This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract.

, 2013, 14, 297--311 297

Cite this: Chem. Educ. Res. Pract., 2013,
14, 297
Using computer simulations in chemistry
problem solving
Spyridon Avramiotis
ab
and Georgios Tsaparlis*
c
This study is concerned with the eects of computer simulations of two novel chemistry problems on
the problem solving ability of students. A controlexperimental group, equalized by pair groups (n
Exp
=
n
Ctrl
= 78), research design was used. The students had no previous experience of chemical practical
work. Student progress was checked twice, once 15 minutes after they had started looking for a
solution, before the experimental group was exposed to the simulation, and again after completion of
the test. The 15 minutes check confirmed the equivalence of the two groups. The findings both verified
the diculty of the problems, and indicated improved mean achievement of the experimental group
(students who were shown the problem simulations), in comparison to the control group (students
who solved the problem in the traditional way). Most students assumed that the major benefit of the
simulations was to help them with the proper application of the equations. The eects of scientific
reasoning/developmental level and of disembedding ability were also examined. The performance level
for formal reasoners was found to be higher than that for transitional reasoners and that for
transitional reasoners higher than for concrete ones. Field independent students were found to
outperform field intermediate ones, and field intermediate students were found to outperform field
dependent ones. Finally, in most cases the experimental group outperformed the control group at all
levels of the above two cognitive factors.
Introduction
Problem solving
Problem solving is a higher-order cognitive activity, hence a
composite one, that involves various cognitive functions, not
least of which is the retrieving from long-term memory of a
certain amount of information and the holding and working
with it in working memory; simultaneously, information
derived from the problem statement must also be held and
processed in working memory. According to psychology and
cognitive science, the human information processing capacity
is restricted (Miller, 1956; Simon, 1974) due to the limitations
of working memory (Baddeley, 1992). We can only pay attention
to a certain limited amount of information and perform a
restricted amount of work on a task at any one time. Many
researchers have dealt with the role of working memory
capacity in problem solving in science (e.g. Johnstone and
El-Banna, 1986; Roth, 1988; Johnstone et al., 1993; Tsaparlis,
1998; Tsaparlis and Angelopoulos, 2000; Stamovlasis and Tsaparlis,
2001, 2003, 2012; St Clair-Thompson et al., 2012). In addition, other
psychometric variables, such as scientific reasoning (previously
referred to as developmental level in the Piagetian sense) and
disembedding ability (degree of field dependence/independence)
play an essential role in science problem solving (Niaz, 1991; Niaz
et al., 2000; Tsaparlis, 2005; Overton and Potter, 2011).
Central and crucial for problem solving are also the number
and quality of available relative operative schemata in long-term
memory. Piaget considered a schema as an internal structure or
representation, and operations as the ways in which we manipulate
schemata. The operative schemata entering a problem constitute
the logical structure of the problem. According to Niaz and
Robinson (1992) (see also Tsaparlis et al., 1998), the logical
structure of a problem represents the degree to which it requires
formal operational reasoning.
At this point, it is necessary to distinguish between what
have been termed problems and exercises. Exercises can be
carried out relatively easily by many students, as they require
only the application of well-known and practiced procedures
(algorithms) for their solution. The skills that are necessary for
a
Program of Graduate Studies Chemistry Education and New Educational
Technologies (DiCheNET), Department of Chemistry, University of Athens,
GR-157 71, Athens, Greece
b
Model Experimental Lyceum, Ionidios School of Piraeus, GR-185 35, Piraeus,
Greece. E-mail: spavramiotis@yahoo.com
c
Department of Chemistry, University of Ioannina, GR-451 10, Ioannina, Greece.
E-mail: gtseper@cc.uoi.gr
Received 11th December 2012,
Accepted 1st April 2013
DOI: 10.1039/c3rp20167h
www.rsc.org/cerp
Chemistry Education
Research and Practice
PAPER
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
View Journal | View Issue
298 Chem. Educ. Res. Pract., 2013, 14, 297--311 This journal is c The Royal Society of Chemistry 2013
this are as a rule lower-order cognitive skills (LOCS). On the other
hand, a real/novel problem requires that the solver must be
able to use higher-order cognitive skills (HOCS) (Zoller, 1993;
Zoller and Tsaparlis, 1997). Note that the degree to which a
problem is a real problem or merely an exercise will depend on
both the students background and the teaching (Niaz, 1995).
Thus, a problem that requires HOCS for some students may
require LOCS for others in a dierent context.
A thorough classification of problem types has been made
by Johnstone (1993, 2001). In this work, we are interested
in real/novel problems. Such problems, requiring both the
development of appropriate strategies and HOCS, prove very
dicult for inexperienced students. A number of researchers
(Simon and Simon, 1978; Larkin and Reif, 1979; Reif, 1981,
1983) have studied the dierences between expert and novice
problemsolvers. The basic dierences were: (a) the comprehensive
and complete schemata of the experts, in contrast to the sketchy
ones of the novices; and (b) the extra step of the qualitative analysis
undertaken by the experts, before they move into detailed and
quantitative means of solution. According to Reif (1983), a basic
description of a problem is the essential first stage in problem
solving:
The manner in which a problem is initially described is
crucially important since it can determine whether the subsequent
solution of the problem is easy or dicult or even impossible. The
crucial role of the initial description of a problem is, however,
easily overlooked because it is a preliminary step which experts
usually do rapidly and automatically without much conscious
awareness. A model of eective problem solving must thus, in
particular, specify explicitly procedures for generating a useful
initial description of any problem . . . The basic description
summarizes the information specified and to be found, introduces
useful symbols, and expresses available information in various
symbolic forms (e.g. in verbal statements as well as in diagrams)
(pp. 949950).
Experience, or a lack of experience, on the part of secondary
students with realistic chemical and physicochemical systems,
such as those involved in chemical problems is also relevant to
this work.
Problem solving and practical work
In science in general, and chemistry in particular, problems are
related to phenomena and experiments. To solve a problem,
students must have the previous knowledge and actual experience
to reconstruct the phenomenon or experiment represented verbally
in the problem, to be able to generate an initial basic description
of the problem. This is not an easy task, especially for school pupils
in many countries (including Greece) who, very often, lack
involvement with experiments, and experience chemistry only
through talk, chalk, and books. Even when experiments are
performed by the students, they are usually of the recipe type
and contribute little to problem solving and enquiry learning in
the laboratory (Domin, 1999a; Johnstone and Al-Shuaili, 2001;
Tsaparlis, 2009). A content analysis of 11 general chemistry
laboratory manuals reported that the majority required the
learners to operate predominantly at the three lower cognitive
levels of Blooms taxonomy (knowledge, comprehension, and
application), and ignored the three higher levels of analysis,
synthesis, and evaluation (Domin, 1999b).
Problem solving ability can be enhanced by associated
laboratory activities. Interestingly, Kerr (1963, cited in Johnstone
and Al-Shuaili, 2001) listed among the aims for practical work that
it should provide training in problem solving and Roth (1994)
found that physics problem solving at the upper secondary level
was improved by means of practical work. Bowen and Phelps
(1997) reported that demonstrations tend to improve the problem-
solving capabilities of the students because they help them switch
between various forms of representing problems dealing with
chemical phenomena (for instance, symbolic and macroscopic).
Deese et al. (2000) found that demonstration assessments
promote critical thinking and a deeper conceptual understanding
of important chemical principles.
In previously published work (Kampourakis and Tsaparlis,
2003), a laboratory/practical activity, involving the well-known
ammonia-fountain experiment, was used in order to find out if
it could contribute to the solution of a demanding chemistry
problem on the gas laws (this is the same as problem 1 in the
present study). Furthermore, the extent to which the practical
activity, together with a follow-up discussion/interpretation in
the classroom, could contribute to the improvement of the
problem-solving ability of the students was assessed. The
subjects were from tenth and eleventh grade (1617 year olds).
It was found that students from the experimental groups
achieved higher scores than those obtained by students from
the control groups. The dierences, although not large, were in
many cases, statistically significant. Mean achievement was,
however, low, and only a small proportion of the students
considered that the practical activity had been relevant/useful
to the solution of the problem. It was concluded that many
students lacked a good understanding of the concepts that
relate to the ideal-gas equation. In addition, as is the case with
most Greek students, the students had no previous experience
in working with chemicals and carrying out, or even watching,
experiments. It seems possible that observing the chemical
experiment so dominated their attention that little opportunity
was left for any mental processing of what was going on and
why. In addition, the experiment chosen involved many details
(including the production of ammonia gas from the reaction of
ammonium chloride with sodium hydroxide), and involved
many physics and chemistry concepts. It therefore seems likely
that an overload of students working memory took place,
preventing significant improvement in their ability to solve
the problem.
The present study
In this paper, we report a study that employed computer
simulations to demonstrate two experiments that were relevant
to the solution of two chemistry problems. The computer
program guided students in using the simulation, in performing
the experiment, and in dealing with the problem. One of the
experiments was basically the same as the ammonia fountain
experiment, used in the earlier Kampourakis and Tsaparlis study
Paper Chemistry Education Research and Practice
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract., 2013, 14, 297--311 299
(with some modifications/refinements). The second experiment
concerned the ignition of a sulfur and oxygen mixture. The
problem associated with the first experiment was exactly the
same as that described in the previous study, while a new
problem was specifically designed for the second experiment.
Both problems were based on the use of the ideal-gas state
equation for the calculation of gas volume or gas pressure
respectively, and concentrations of aqueous solutions (molarities)
were also involved in the problems and the calculations.
The central research question being investigated (Research
question 1) is as follows: Does watching a simulation on a
computer screen while attempting to solve a problem have an eect
on students ability to solve the problem? In addition, the study
oers the opportunity to investigate further two important
factors that are known to have an important eect on students
problem solving performance:
(2) Is the ability of students to solve problems related to their
scientific reasoning/developmental level?
(3) Is disembedding ability (degree of field dependence/independence)
connected to students ability in problem solving?
Rationale
Computer simulations in science education
As stated above, one can pay attention to only a limited amount
of information and can perform only a restricted amount of
work on a task at any one time. If the amount of information
processing exceeds ones capacity, it will result in cognitive
overload (or working memory overload) (Johnstone and
El-Banna, 1986; Sweller, 1988; Sweller and Chandler, 1991). In
the case of laboratory experiments to be performed by the
students, instructions can be designed so as to reduce the
cognitive load and hence the possibility of overload. Cognitive
Load Theory, developed by Sweller (Sweller, 1988; Bannert,
2002), which employs information processing theory, paying
attention to the inherent limitations of working memory, is of
relevance here.
Richard E. Mayers cognitive theory of multimedia learning,
which is based on the fact that paying attention to several tasks
simultaneously will result in a portion of the working memory
not being available for learning (Mayer, 1997, 2001; Mayer and
Moreno, 1998), is of relevance to this study. Mayers theory
discusses a number of design principles for ecient multi-
media instruction. The modality principle states that materials
which present both verbal and graphical information should
present the verbal information in an auditory format, and not
as written text (Moreno and Mayer, 1999). On the other hand,
according to the split attention eect, students learn better from
animation and narration rather than from animation, narration,
and on-screen text (Mayer, 2001). Other principles include the
spatial contiguity principle Students learn better when corres-
ponding words and pictures are presented near rather than far
from each other on the page or screen; the temporal contiguity
principle Students learn better when corresponding words and
pictures are presented simultaneously rather than successively;
the coherence principle Students learn better when extraneous
material is excluded rather than included; and the individual
dierences principle Design eects are stronger for low-
knowledge learners than for high knowledge learners, and for
high-spatial learners rather than for low-spatial learners.
Virtual interactive experiments performed on the computer
appear to be a powerful tool (Lajoie, 1993; Josephsen and
Kristensen, 2006). Research shows that a computer-based learning
environment can reduce the time required for performing a task,
and at the same time reduce the cognitive load. Careful design and
instructions can then contribute to enhanced student learning
(Lajoie, 1993; Lajoi et al., 1998, 2001; van Bruggen et al., 2002;
Josephsen and Kristensen, 2006).
According to Oakes and Rengarajan [2002, cited in Akaygun
and Jones (2013)], an animation is a multimedia presentation
that is rich in graphics and sound, but not in interactivity, while
a simulation is defined as an interactive and explorative repre-
sentation. The authors maintain that it is not always possible to
re-create in a simulation an accurate real-world environment;
further, the more sophisticated a simulation, the more accu-
rately it represents and describes the target phenomenon.
Computer simulations have been used in a variety of teaching
situations especially as a substitute for, or complement to, the
chemistry laboratory (Butler and Grin, 1979; Akaygun and Jones,
2013). Early laboratory applications involved simulations of macro-
scopic laboratory procedures. Later simulations extended the
emphasis to representing phenomena at the atomic and molecular
level, as well as to simulating large and expensive laboratory
equipment. Although animations and simulations are dierent in
terms of their level of interactivity, both have been used as eective
tools for chemistry instruction. For instance, a number of studies
have found that students who received instruction that included
computer animations of chemical processes at the molecular level
were better able to comprehend chemistry concepts involving the
particulate level of matter than those who did not (Williamson and
Abraham, 1995; Sanger and Greenbowe, 1997a, 1997b; Burke et al.,
1998; Sanger et al., 2000; Ardac and Akaygun, 2004, 2005).
A main feature of simulations is their dynamic information and
character, which increases the information-processing demand,
and thus may not contribute to improved learning in comparison
with static pictures (Rieber, 1990; Lewalter, 2003; Lowe, 2003).
Lewalter maintains that dynamic visuals may reduce the load of
cognitive processing by supporting the construction of a mental
model, but they may cause higher cognitive load because of their
transitory nature. Focusing on one type of presentation component
may result in missing information from a dierent presentation
component, because of the split attention eect. This eect occurs
not only when one has to attend to multiple presentation stimuli
(such as combinations of picture and text), but also when attending
to a single presentation that includes temporal changes (as is the
case in an animation or a simulation). Control of the variables in a
dynamic visual can reduce the split attention eect.
The eect of scientific reasoning and disembedding ability on
problem solving
Two cognitive variables, scientific reasoning and disembedding
ability, were also considered in this study. These were considered
Chemistry Education Research and Practice Paper
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
300 Chem. Educ. Res. Pract., 2013, 14, 297--311 This journal is c The Royal Society of Chemistry 2013
pertinent for the following reasons: research has shown that
solving chemistry problems involves many of the attributes of
formal reasoning in the Piagetian sense, hence the ability to
reason scientifically might be expected to be reflected in successful
problem solving. For instance, Niaz and Robinson (1992) reported
that the developmental level of students is the most consistent
predictor of success when dealing with the logical complexity of
chemistry problems, so that the number of operative schemata
entering a problemmay be the main contributor to the diculty of
the problem, overidding its mental (or cognitive) demand (the
latter is connected with working memory capacity). Tsaparlis et al.
(1998) confirmed the findings of Niaz and Robinson in the case of
chemical equilibrium problems, by manipulating the logical
structure of these problems. On the other hand, disembedding
ability separates signal from noise and has proved to be an
important factor for performance in real/novel problems, in which
the field eect plays a central role: field-dependent subjects find it
dicult/are unable to separate an item from its context, whereas
field-independent subjects are able to make this distinction
and can therefore focus better on the relevant field or context.
A continuum exists between these two categories, with an
intermediate category of field-intermediate subjects.
In a study on non-algorithmic quantitative physical chemistry
problems that have many of the features of novel/realistic problems
(Tsaparlis, 2005), it was reported that functional mental capacity
and disembedding ability played important roles and were
definitely more significant than either scientific reasoning or
working memory capacity. Note that the psychometric test for
measuring mental capacity (the figural intersection test) involves
disembedding ability in addition to information processing.
The eects of scientific reasoning and disembedding ability
have also been examined in the area of acidbase equilibria,
and were found to be important for student performance, with
the latter ability clearly having the larger eect (Demerouti
et al., 2004). Developmental level was connected with concep-
tual understanding and application, but less so where complex
conceptual situations and/or chemical calculations were
involved. Disembedding ability was important in situations
requiring demanding conceptual understanding, and where
this is combined with chemical calculations. Overton and
Potter (2011) investigated students success in solving, and
their attitudes towards, context-rich open-ended problems in
chemistry, and compared these to algorithmic problems. They
found a positive correlation between algorithmic problem solving
scores and mental capacity (as measured with the figural
intersection test), while scores in open-ended problem solving
correlated with both mental capacity and disembedding ability.
St Clair-Thompson et al. (2012) compared further algorithmic
and open-ended problems with respect to mental capacity and
working memory capacity; for the algorithmic problems working
memory was reported to be the best predictor, while for open-
ended problems both working memory capacity and mental
capacity were important. On the other hand, BouJaoude et al.
(2004) found that among a number of cognitive variables
(learning orientation, developmental level, mental capacity,
but not including disembedding ability), developmental level
had the highest power to predict student performance in
conceptual problems.
Method
The study was carried out over two consecutive school years,
20012002 and 20022003. In the first year, 101 tenth-year
students, who were attending an upper secondary school
(lykeion/lyceum) in an urban district of Piraeus, Greece (school
A) participated. The students were divided into four classes,
and 82 students who had been matched by pairs (see below)
were selected. These 82 students formed the two study groups
I
A
and II
A
, each with n = 41. The intervention was carried out
during two teaching sessions. The research activity was carried
out over two weeks, with one study session per week and one
standard teaching period separating them. Group I
A
acted as
the experimental group (EG) for problem 1 and control group
(CG) for problem 2 and vice versa for group II
A
.
In the second year, 120 tenth-year students, coming from a
dierent urban upper secondary school also in the Piraeus
district (school B) participated. These students were also
divided into four classes, and 74 students who had been
equalized by pairs were selected. These 74 students formed
the two study groups, I
B
and II
B
, each with n = 37. One group of
school B acted as EG and the other as CG. Because of lack of
instructional time, only problem 2 was used in the second year
of the study.
The first author was a teacher in both schools. A chemistry
teacher acted as the second teacher in school A, while a
physicist took on this role in school B. All three were experi-
enced teachers, with the first two holding postgraduate degrees
in chemistry education. In both schools and for both problems,
each teacher taught one EG and one CG of students.
All students in both schools had the same experience in
working with chemicals in the lab, and all had attended the
same computer simulations. Matching by pairs of the two
groups for each year of the study was carried out on the basis
of the following parameters: (i) mean achievement in three tests
plus a final in-term exam in the chemistry course; (ii) scientific
reasoning (developmental level) of the students (see below). In
addition each pair of students had similar mean achievement
in two final in-term exams in the physics course; Table 1
contains details about this matching by pairs. Statistical
comparisons using a Student t-test for independent samples
and Pearson correlations demonstrate the equivalence of the
groups.
The testing procedure
At the outset, the students were informed about the research
character of the teaching and testing methodology. The students
were asked to solve a given problem each time. Testing took
35 minutes. For the EGs it took place inside the computer school
labs, which were each equipped with 12 and 10 PCs for schools A
and B respectively. The students wrote on special double copy
paper. For both the EG and the CG, 15 minutes after the start of
the test, the teacher asked the students to hand in one copy of
Paper Chemistry Education Research and Practice
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract., 2013, 14, 297--311 301
their paper, with what they had written up to that time. This
allowed for an evaluation of the students achievement up to that
time. Following that, the students of the CG were allowed to
continue their eort for 20 more minutes, after which the
teacher collected their final papers. In the case of the EG, after
the students handed in the copy with their work up to the 15th
minute, they were instructed to use their PC to watch the
problem simulation for 57 minutes. During this time, they were
free to run and watch the simulation as many times as they
wanted. Then, they were allowed to continue their eort in
solving the problem for 15 more minutes, after which time their
final papers were also collected.
The problems
Two problems were used. Problem 1 was taken from an ocial
book for tenth-grade students, supplied by the Greek Ministry
of Education, and was identical to the problem used in the
Kampourakis and Tsaparlis (2003) study. Problem 2 was
devised and constructed by the second author.
Problem 1: A vessel contained gaseous ammonia (NH
3
) at a
pressure p
1
= 2 atm, and a temperature of 27 1C. Part of the
ammonia gas was transferred to a container containing water,
where it dissolved completely to produce 2 L of a 0.1 M aqueous
ammonia solution. If the pressure in the vessel is reduced to
1.18 atm, find the volume of the vessel.
The logical structure of problem 1 involves two main
schemata: the ideal gas equation and concentration of aqueous
solutions (molarity). The problem caused various diculties to
the students. In particular, they failed to connect the fall in the
gas pressure with the ammonia solution that was formed. To
make the problem as close as possible to a real problem (and
not a traditional exercise), no further comments about the
problem were provided, and the value of the ideal gas constant
was not given, so the students needed to know or be able to
calculate the value of the ideal gas constant in the proper units.
Most successful solvers applied the ideal gas equation twice,
for both the initial and the final state of the ammonia gas in the
flask: p
1
V = n
1
RT (1) and p
2
V = n
2
RT (2). From these two
equations, they arrived at the relationship n
1
/p
1
= n
2
/p
2
(3). If
n
s
is the number of moles of ammonia dissolved in water, then
n
2
= n
1
n
s
. Substituting into eqn (3) provides solution of the
resulting equation for n
1
. Finally, substitution of the value for
n
1
into eqn (1) leads to the calculation of V. A much smaller
proportion of students did not use the ideal gas equation, but
instead started with the equation p
1
/p
2
= n
1
/n
2
(at constant V
and T), and went on as mentioned above. A number of students
(11 out of 41, 26.8%) made (eventually successful or unsuccessful)
use of the dierence between the two given pressures in their
solution, which led to the relationship p
1
p
2
= n
s
RT/V. This
alternative method of solution is easier and faster, so the students
who employed it demonstrated a good conceptual understanding
of the problem. Note that the existence of a pressure gauge in the
experimental setting is likely to have contributed substantially to
the idea to subtract the two pressures, a fact that was admitted by
the students who employed this method.
The marking scheme for problem 1 is given in Appendix 1
(Table 6). Partial marks were allocated to the various steps in
the solution procedure as follows: 10.0 + 2.5 = 12.5 marks for
calculation of moles of ammonia gas dissolved in water; 11.25
marks each for eqn (1) and (2); 2.5 marks each for conversion of
degrees Celsius to degrees Kelvin and for knowing or estimat-
ing the value of R; 20.0 marks for the relationship n
2
= n
1
n
s
;
22.5 marks for the algebraic manipulations that lead to a final
expression for V; 10.0 marks for intermediate numerical calculations;
finally, an additional 7.5 marks for the correct numerical result with
proper units. Four experienced chemistry teachers independently
marked fifteen randomly selected papers, according to the agreed
marking scheme. The Pearson correlation coecients between the
four markers varied between 0.94 and 0.99.
Problem 2: A closed vessel with a volume of 4 L, at a
temperature of 207 1C, contains a mixture of sulfur (S) and
precisely the quantity of oxygen gas (O
2
) required for complete
combustion. The vessel is supplied with a piston and an
exhaust gas valve. The mixture is ignited so that all the sulfur
is burnt and all of the oxygen is consumed [S(s) + O
2
(g) -
SO
2
(g)]. By pressing the piston, all the combustion gas produced
is transferred into a beaker containing 0.5 L cold water, where
part of the gas is dissolved to give a solution with a concentration
of 0.96 mol L
1
. The remaining gas is collected in an inverted
test-tube, and was found to occupy a volume of 0.448 L at STP.
Calculate the pressure in the closed vessel before igniting the
mixture and compressing the piston.
Here again the value of the ideal gas constant was not
supplied for the same reason as for problem 1. On the other
hand, so as not to further increase the complexity of the
problem, we included in it the relevant chemical equation. To
solve this problem one has to consider the moles n
1
of SO
2
that
were dissolved in water and the moles n
2
that were collected in
the test-tube. The total amount of SO
2
is then n
1
+ n
2
. Next, the
moles of O
2
which were present in the vessel before combustion
have to be calculated from the reaction stoichiometry. Finally,
Table 1 Description of the matched by pairs experimental and control groups
of students for the two schools used in the study. Mean percentage achievement
(with standard deviations in parentheses) in chemistry, physics, and in the Lawson
test, plus statistical tests (t test and correlation coecients)
Chemistry Physics Lawson test
School A
Group I
A
54.5 (21.5) 61.5 (17.0) 53.6 (20.0)
(n = 41)
Group II
A
54.5 (22.0) 62.5 (18.5) 54.0 (18.9)
(n = 41)
Tests for equivalence
t-Test (p-value) 0.16 (0.87) 0.20 (0.84) 0.38 (0.70)
Pearson r 0.97 0.76 0.86
School B
Group I
B
(experimental) 73.8 (16.8) 69.9 (22.0) 53.7 (20.8)
(n = 37)
Group II
B
(control) 75.0 (17.4) 69.9 (20.4) 51.6 (18.9)
(n = 37)
Tests for equivalence
t-Test 0.97 (0.34) 0.02 (0.98) 1.03 (0.31)
Pearson r 0.97 0.79 0.81
Chemistry Education Research and Practice Paper
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
302 Chem. Educ. Res. Pract., 2013, 14, 297--311 This journal is c The Royal Society of Chemistry 2013
the ideal-gas equation has to be applied to calculate the initial
pressure in the vessel.
Problem 2 is more demanding than problem 1, having a
more complex logical structure. In addition to the schemata of
the solvation and the ideal-gas equation, it involves the extra
schema of the stoichiometry of the chemical reaction (combustion
of sulfur). A further complication arises from the fact that part of
the gas produced is dissolved in water and the remainder is
collected separately.
The marking scheme for problem 2 is given in Appendix 1
(Table 7). All student papers were marked by the first author,
who was a chemistry teacher in school A and one of the two
teachers in school B. Validity and reliability of the marking
schemes was judged by having 40 papers (20 for each group, EG
and CG) marked by another teacher chemist. The correlation
between the two markings was high (r = 0.96).
The simulations
The simulations for the two problems are described in Appendix 2.
The problems themselves provided the scenarios for design and
construction of the simulations. For technical reasons, audio
material was not included, so some of Mayers design principles
for eective multimedia instruction were not employed (see
Rationale). However, eorts were made to ensure that the
simulations were clear and easy for the students to use, suitable
for the students age and level of knowledge, and simple,
without extra added information (noise), which would have
been irrelevant to the solution of the problems (Mayers coherence
principle). Also, little time was required for the handling and
watching of each simulation, and care was taken to ensure that
the simulation did not reveal the solution. The duration of each
simulation did not exceed 1.5 minutes, so that students could
watch it several times within the 57 minutes available.
The same procedure and construction perspective was
adopted for the simulations of both problems. Three screens
were created; the one that appears first is the simulation of the
experimental set-up, while the other two screens play supporting
roles: one is descriptive/explanatory about the equipment and its
connection with the problem; and the other provides functional
help. When execution of the problem is initiated, the main page,
which shows the experimental set up, appears. Navigation is
horizontal, that is, there is ability to switch between the three
screens: Description of the experiment; Restart the experiment;
Help, by using three buttons, which are always present and
active. Further details about the simulations are provided in
Appendix 2.
Psychometric measures
In this study, scientific reasoning was assessed by means of the
Lawson paper-pencil test, which included 15-open-ended-items,
and tested the following reasoning modes: conservation of weight,
displacement of volume, control of variables, proportional
reasoning, combinational reasoning, and probabilistic reasoning
(Lawson, 1978). The two schools involved in our study were
matched by pairs in terms of scientific reasoning as follows: In
the two groups, I
A
(n = 41) and II
A
(n = 41), of school A there was
precisely the same number of students in each of the three
developmental levels: 6 concrete, 23 transitional, and 12 formal
reasoners. Similarly for the two groups, I
B
(n = 37) and II
B
(n = 37), of school B, there were 6 concrete, 21 transitional and
10 formal reasoners in each.
Disembedding ability was assessed by means of the Hidden
Figures Test, which is a 20 minute test that was devised and
calibrated by El-Banna (1987), and is similar to the Group
Embedded Figures Test, GEFT, devised by Witkin (Witkin
et al., 1971; Witkin, 1978). The two groups from school A were
NOT matched in terms of this ability: in group I
A
there were
21 field dependent, 14 field intermediate, and 6 field independent
students while in group II
A
there were 15 field dependent, 14 field
intermediate, and 12 field independent students. On the other
hand, the two groups from school B were matched, with each
group having 6 field dependent, 20 field intermediate, and 11 field
independent students.
Results
Table 2 gives data and statistical comparisons for student
achievement during the first 15 minutes. For testing significance
of dierences of mean achievements between the two matched
samples, we used the parametric paired Students t test (t-test for
matched pairs) and the non-parametric Wilcoxon signed-rank test
(W) (Cohen and Holliday, 1982). Since we were interested in
testing whether one process was better or worse than the other,
we employed two-tailed criteria. It is evident that during the first
15 minutes the two groups EG and CG did not dier in their
treatment, as the use of the simulation was only introduced after
their papers had been collected; therefore, the student achieve-
ment during the first 15 minutes provides a further check on the
equivalence of the two groups. With the exception of school A for
problem 2, the EG and the CG were equivalent statistically. The
considerable improvement demonstrated by the CG students
from school A, in the case of problem 2, might be attributed
(a speculation) to the fact that the CG students for problem 2
Table 2 Total percentage achievement (mean values with standard deviations
in parentheses) at time 15 minutes after the start of the test (paired t-test and
Wilcoxon signed-rank test)
School
Experimental
group
Control
group t Value ( p) W Value ( p)
Problem 1
A 16.3 (23.3) 13.4 (16.4) 0.91 (0.370) 0.51 (0.609)
(n = 41) (n = 41)
Problem 2
A 11.4 (12.7) 29.4 (27.5) 4.93 (o0.001) 4.23 (o0.001)
(n = 41) (n = 41)
B 8.65 (8.18) 10.5 (15.2) 0.79 (0.44) 0.71 (0.479)
(n = 37) (n = 37)
Problems 1 and 2
a
A + B
a
12.7 (18.0) 12.1 (15.8) 0.314 (0.755) 0.036 (0.971)
(n = 78) (n = 78)
a
School A enters here through problem 1 and school B through
problem 2.
Paper Chemistry Education Research and Practice
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract., 2013, 14, 297--311 303
had previously acted as the EG for problem 1, so they had
attended the simulation program for problem 1. It was evident
to these students (now forming the CG) that they were not
going to use the computer this time, so they worked diligently
on the problem, while the students from the EG were waiting to
see the simulation program, and were less eager to begin work
on the solution before that.
Achievement in solving the problems
Table 3 shows the achievement of the EGs and CGs for schools
A and B, respectively, as well as for the sum of schools A and B,
but with school A entering in the latter sum only through
problem 1 (the reason will become clear below). Recall that
school A did both problems, while school B tackled only
problem 2. The table also contains data for the statistical
comparisons (t and W values). As a matter of fact, students
from the EG might have been handicapped by (i) the interruption
of the solution process for the EG while they viewed the simula-
tion and/or (ii) any misunderstanding that might have been
introduced because of the simulation.
The following conclusions can be drawn from the data: in all
cases, performance was lowand below a 50% value. Achievement
by the EG was higher for both problems. There were statistically
significant dierences in favor of the EG for the following cases:
school A for problem1 and sumof schools A for problem1 and B
for problem 2.
Achievement of school A was much higher for problem 2
(especially for the CG), and this can be attributed to the
experience these students gained, both from solving problem 1
and from their practice with the simulation program. The
considerable improvement displayed by the CG from school A
on problem 2 (showing only a very small and statistically
insignificant inferior achievement to that for the corresponding
EG from school A) might be attributed to the fact that the CG
students for problem 2 acted as the EG for problem 1, so they
had previously experienced the simulation program for problem
1. While for problem 1 there is a statistically significant
dierence, in the case of problem 2 only for school B was the
dierence substantial and near statistical significance.
An important finding relates to the number of successful
solvers, both in each problem, and with regard to the eect of
the simulations. We identified students who achieved a mark
over 80% in each problem as successful solvers. From a total of
41 students, there were 6 successful solvers of problems 1
(14.6%), of which 5 belonged to the EG and 1 to the CG. Also,
from a total of 78 students, there were 23 successful solvers of
problem 2 (29.3%), 11 belonged to the EG and 12 to the CG.
Recall the eect of practice on the achievement of the CG from
school A on problem 2.
It is of particular interest to examine the improvement of
mean achievement from 15 minutes until the end of testing
(see Tables 2 and 3). The highest positive eect of the simula-
tion was noted in the case of school B for problem 2, in which
case the EG improved its mean achievement from 15 minutes
until the end of testing by 4.2 times (8.65 -36.4), while the CG
improved by only 2.5 times (10.5 -26.4). On the other hand,
the group from school A that acted as EG for problem 1
improved its mean achievement by about a factor of 2
(16.3 - 32.1) for problem 1, but much less (only 1.3 times)
for problem 2 (29.4 - 38.8) when it acted as CG. This can be
attributed to two factors: this group knew that as CG they were
not going to watch a simulation, hence they worked steadily
during the first 15 minutes, thus advancing their solution
further (29.4), and, as a result, their further improvement by
the end of time (38.8) was not particularly impressive. However,
the fact that their overall achievement (38.8) is comparable to
that of the corresponding EG (40.6) is impressive; their experi-
ence of working with the simulation for problem 1 could well
have played a role. Finally, the group from school A that acted
as CG for problem 1 improved its mean achievement by about
a factor of 2 (13.4 - 23.1) for problem 1, but much more
(3.6 times) for problem 2 (11.4 -40.6) when it acted as EG. It is
apparent that all groups got better with more time, but using
the simulation was far more eective in most cases (except for
the case of school A for problem 2). Note that all the above
changes of score from time 15 minutes until the end of testing
are statistically significant ( p o0.001), as judged by using the t
test for dependent samples.
Turning to some qualitative aspects of the use of the
simulations, discussions with the students after the interven-
tion showed that most students initially assumed that the
simulations did not help them in the solution of the problems
but were useful in helping with the proper application of the
equations. Further discussion revealed some interesting
aspects of the students actions and attitudes, with several of
them admitting that through the simulations they cleared out
something (in their minds). We quote below some of the most
useful comments that were collected:
Using the computer did not disturb them, but on the contrary
they found it interesting.
The instructions were clear.
They had no problem using the quick instructions of the
software, and they did not need practice.
Table 3 Total percentage achievement (mean values with standard deviations
in parentheses) in solving the two problems for the experimental and control
groups (plus statistical comparisons)
School
Experimental
group
Control
group t Value ( p) W Value ( p)
Problem 1
A 32.1 (26.6) 23.1 (20.5) 2.12
(0.030)
2.32 (0.010)
(n = 41) (n = 41)
Problem 2
A 40.6 (26.5) 38.8 (32.7) 0.43
(0.700)
0.52 (0.600)
(n = 41) (n = 41)
B 36.4 (30.5) 26.4 (26.6) 1.81
(0.080)
1.73 (0.080)
(n = 37) (n = 37)
Problems 1 and 2
a
A + B
a
34.1 (28.4) 24.6 (23.5) 2.81
(0.006)
2.96 (0.003)
(n = 78) (n = 78)
a
School A enters here through problem 1 and school B through
problem 2.
Chemistry Education Research and Practice Paper
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
304 Chem. Educ. Res. Pract., 2013, 14, 297--311 This journal is c The Royal Society of Chemistry 2013
They did not assume that they were helped with the final
solution of the problems.
In the simulation of problem 1, the pressure gauge helped
them realise that pressure changed.
They received an overall help from the simulation of problem
2 because it was long and it was dicult for them to keep in mind
the procedure.
Some students protested that they had never before solved a
similar problem before taking the test with problem 1.
Eect of scientific reasoning on the eectiveness of the
simulation
Table 4 shows comparisons for students according to their
levels of scientific reasoning (developmental level). For statistical
comparisons, values of the paired t statistic as well as of the
Wilcoxon signed-rank test are given, because the groups are
matched by pairs. Recall that here all samples were matched
by pairs in terms of levels of scientific reasoning, so we have
equal numbers for each level for the EG and the CG. Formal
students performed better than transitional, and transitional
better than concrete students (except for the EG from school A
for problem 2). In all cases (except for school A for problem 2),
the dierence is higher in the case of the formal students. Also,
in most cases the EG students performed better than the CG
students at all levels of logical thinking. However, due to the small
samples, the dierences are not statistically significant, except in
the case of the formal students for the sum A (problem 1) + B
(problem 2) and nearly so for the formal students of school A for
problem 1.
Eect of disembedding ability on the eectiveness of the
simulation
Table 5 shows a comparison of students according to their level
of disembedding ability. For statistical comparisons, values of
the parametric t statistic for independent samples as well as of
the MannWhitney test (U or z for one or both samples 42025)
(U is a non-parametric counterpart for independent observa-
tions.) (Recall that in this case, the EGs and CGs were not
matched by pairs in terms of levels of disembedding ability,
except for school B (for problem 2) but even for school B the
equalization was in numbers of students per level, and not by
pairs.) Field independent students outperformed field inter-
mediate students (except for the EG from school A for problem
2), and field intermediate students outperformed field depen-
dent ones [except for the CG from school A for problem 1, the EG
from school B for problem 2, and for the sum (school A for
problem 1 + school B for problem 2) for the EG]. Also, with one
exception, the EG students outperformed the CG students at all
levels of disembedding ability, but, due to the small samples, the
dierences are not statistically significant. In the case of school A
for problem 2, the field independent students from the CG
(n = 6) performed much higher than the corresponding students
of the EG (n = 12), but their sample size was very small.
Conclusion and answers to the research
questions
The two problems proved very dicult for the students in our
sample. Student performance was disappointing (under the
50% mark) for both problems and both the experimental group
(EG) and the control group (CG). This can be attributed to the
involvement of various complex concepts, such as the ideal gas
law equation, volumetric analysis, and, in the case of the
problem 2, reaction stoichiometry. As a matter of fact, the
concept of the ideal gas and the ideal gas law are even dicult
for first and second-year undergraduate students (Kautz et al.,
1999). In addition, our students had no previous practice with
similar problems, had no experience in carrying out experi-
ments, and were not even familiar with laboratory equipment.
To ensure that the problems were not straightforward algorith-
mic exercises but had features of real/novel problems, we did
not provide the value of the ideal gas constant for the students.
Regarding our three research questions, our conclusions are as
follows:
Research question 1. Does watching a simulation on a computer
screen while attempting to solve a problem have an eect on
students ability to solve the problem?
Table 4 Total percentage achievement (mean values with standard deviations
in parentheses) for the three levels of scientific reasoning, and statistical
comparisons
Levels of
scientific
reasoning
Experimental
group
Control
group t Value (p) W Value (p)
School A (problem 1)
Concrete 15.0 (8.94) 6.7 (0.30) 1.494 (0.195) 1.355 (0.176)
(n = 6) (n = 6)
Transitional 27.6 (20.1) 23.8 (19.5) 0.840 (0.410) 1.326 (0.185)
(n = 23) (n = 23)
Formal 49.2 (34.9) 29.8 (22.9) 2.143 (0.055) 2.161 (0.031)
(n = 12) (n = 12)
School A (problem 2)
Concrete 34.2 (36.9) 13.3 (8.12) 1.176 (0.293) 0.841 (0.400)
(n = 6) (n = 6)
Transitional 31.9 (19.2) 29.4
(23.95)
0.422 (0.677) 0.770 (0.441)
(n = 23) (n = 23)
Formal 60.4 (24.4) 70.0 (27.3) 0.846
(0.415)
0.831
(0.406) (n = 12) (n = 12)
School B (problem 2)
Concrete 25.3 (30.1) 20.3
(13.98)
0.430 (0.680) 0.339 (0.735)
(n = 8) (n = 8)
Transitional 33.8 (29.4) 26.0 (27.5) 1.069 (0.298) 1.083 (0.279)
(n = 20) (n = 20)
Formal 51.9 (30.6) 32.5 (33.9) 1.491 (0.174) 1.718 (0.086)
(n = 9) (n = 9)
School A (problem 1) and B (problem 2)
Concrete 20.9 (23.4) 14.5 (12.4) 0.937 (0.366) 0.550 (0.582)
(n = 14) (n = 14)
Transitional 30.5 (24.7) 24.8 (23.3) 1.373 (0.177) 1.662 (0.096)
(n = 43) (n = 43)
Formal 50.4 (32.3) 30.9 (27.4) 2.620 (0.016) 2.748 (0.006)
(n = 21) (n = 21)
Paper Chemistry Education Research and Practice
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract., 2013, 14, 297--311 305
Our results showed improved mean achievement for the EG,
that is for the students who used/viewed the problem simulations,
in comparison to the CG, who solved the problem in the traditional
way by thinking and writing on paper. This was more evident when
the results for the two schools were combined (school A for problem
1 and school B for problem 2) to give a larger sample (sum A + B).
Achievement levels at school A were much higher for
problem 2 (especially for the CG), and this can be attributed
both to the experience these students gained from solving
problem 1 and from their practice with the simulation pro-
gram. The considerable improvement in problem 2 for the CG
at school A might be attributed to the fact that the CG for
problem 2 had previously acted as the EG for problem 1, so it
had experienced the benefits of the simulation program for
problem 1. An additional fact that supports the above specula-
tion is that in the interlude between the two teaching periods
when the two problems were administered to the students of
school A, the solution to problem 1 was presented and relevant
discussions were carried out by both the EG and the CG. We
assume that the model solution and the related discussion
must have contributed greatly to the improved results of the
students from school A in problem 2.
We repeat that the equivalence of the two groups of students
(EG and CG) was established by a proper composition that was
based on their achievement in chemistry and physics in-term
exams, as well as in the Lawson test of scientific reasoning.
In addition, the equivalence was checked by comparing
achievement, during the first 15 minutes of the test for each
of the two problems of this study, where the performance of the
two groups was found to be similar.
A deeper analysis of the students solution procedures
showed that marks were low irrespective of the use or non-
use of the simulations. This suggests that there is a mental
jump from the solution of the component subproblems to the
synthesis and solution of the whole problem. It appears that the
eect of the simulations for the relevant experimental set-ups
did not really lead to the final solution of the problems. The
simulations rather appeared to help with the solution of the
subproblems or component steps in a problem. Therefore,
the overall performance on the problem improved, because
after viewing the simulations more students were successful
in using the equations, relationships, and data appropriately.
Examples of students attempting to solve exercises or problems
by using equations, relationships, and data unsuccessfully
or combining them at random (or by using incorrect equations
and relationships) are well-known both to experienced
teachers and in the problem solving literature. According to
our marking schemes (see Appendix 1), a total mark of 3040%
could be achieved by a student who just knew the proper
equations and relationships, and applied them correctly.
In particular, the handling of volumes (volume of the
vessel, volume of water, volume of the gas) proved dicult. It
Table 5 Total percentage achievement (mean values with standard deviations in parentheses) and statistical tests (t-test for independent samples and Mann
Whitney U test) for the three levels of disembedding ability
Levels of disembedding ability Experimental group Control group t Value (p) U Value (p)
School A (problem 1)
Field dependent 27.6 (25.6) 22.3 (24.4) 0.628 (0.535) 0.988 (0.323)
(n = 21) (n = 15)
Field intermediate 32.5 (21.3) 18.6 (9.99) 2.217 (0.036) 2.700 (0.007)
(n = 14) (n = 14)
Field independent 46.7 (39.2) 29.2 (24.1) 1.18 (0.255) 1.42 (0.154)
(n = 6) (n = 12)
School A (problem 2)
Field dependent 31.0 (24.7) 21.4 (16.8) 1.387 (0.0174) 1.241(0.215)
(n = 15) (n = 21)
Field intermediate 48.6 (25.7) 48.2 (31.5) 0.033 (0.974) 0.500 (0.617)
(n = 14) (n = 14)
Field independent 43.3 (28.2) 78.3 (25.6) 2.556 (0.021) 2.204 (0.028)
(n = 12) (n = 6)
School B (problem 2)
Field dependent 39.2 (35.5) 18.3 (15.9) 1.312 (0.219) 1.203 (0.229)
(n = 6) (n = 6)
Field intermediate 28.1 (27.0) 25.9 (26.8) 0.260 (0.797) 0.461 (0.648)
(n = 20) (n = 20)
Field independent 49.8 (31.5) 31.6 (31.5) 1.353 (0.191) 1.776 (0.076)
(n = 11) (n = 11)
School A (problem 1) and B (problem 2)
Field dependent 30.2 (27.8) 21.2 (22.0) 1.216 (0.230) 1.41 (0.158)
(n = 27) (n = 21)
Field intermediate 29.9 (24.6) 22.9 (21.6) 1.254 (0.214) 1.636 (0.102)
(n = 34) (n = 34)
Field independent 48.7 (32.3) 30.3 (27.3) 1.918 (0.063) 2.183 (0.030)
(n = 17) (n = 23)
Chemistry Education Research and Practice Paper
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
306 Chem. Educ. Res. Pract., 2013, 14, 297--311 This journal is c The Royal Society of Chemistry 2013
is at this point that the simulations appeared particularly
helpful.
Research question 2. Is the ability of students to solve problems
related to their scientific reasoning/developmental level?
As mentioned earlier, research has shown that the developmental
level of students is the most consistent predictor of success when
dealing with significant changes in the logical structure of
chemistry problems (Niaz and Robinson, 1992; Tsaparlis
et al., 1997). Our results indicate that the two problems of this
study (especially problem 2) had a rich logical structure (see
comments after the two problems). With one exception, formal
students performed better than transitional students, and
transitional students better than concrete ones, with the
dierences between EG and CG being highest in the case of
the formal students. In most cases, the EG students performed
better than the CG students at all levels of logical thinking, but,
due to the small sample sizes, the dierences are not statisti-
cally significant except in the case of formal students in the
sum A (problem 1) + B (problem 2).
Research question 3. Is disembedding ability (degree of field
dependence/independence) connected to students ability in pro-
blem solving?
It is known from the literature that in the solving of realistic/
novice problems, disembedding ability plays an important and
dominant role (Demerouti et al., 2004; Tsaparlis, 2005; Overton
and Potter, 2011). Our results indicate that both problems used
in this study have features of novelty/were real problems for
our students. With some exceptions, field independent
students outperformed field intermediate students, and field
intermediate students outperformed field dependent ones.
Also, with one exception, the EG students performed better
than the CG students at all levels of disembedding ability.
However, due to small sample sizes, the dierences are not
statistically significant.
Final comments
The data undoubtedly demonstrate that the use of computer
simulations can be helpful in improving problem solving
scores. We recognize that other types of intervention, for
example, just talking through the problem with the students,
demonstrating the experiment, or by having the students
perform the experiment themselves, might have been equally
eective, but the issue here was to check whether a particular
approach (the computer simulation) would be eective.
Our previous study examined the eect of a laboratory/
practical activity involving the ammonia-fountain experiment
on the solution of problem 1 (Kampourakis and Tsaparlis,
2003). While both studies were carried out with tenth-grade
general education Greek students (around 16 years old), the
previous study also involved some eleventh-grade students
(around 17 years old), who were following a stream of studies
that included advanced chemistry among its main subjects.
The tenth-grade students of the experimental group of this
previous study had a mean achievement of 18.6% in problem 1,
while the corresponding eleventh-grade students achieved a
mean score of 37.0%. The second score is about the same as the
marks achieved by the EG students in the present study. The
dierence for tenth grade students between the present and the
previous study could be attributed, in part, to the fact that
chemistry was taught as a one-period per week course in the
previous study, whereas the teaching time had been doubled in
the present study. Finally, the students involved in the present
study came from an urban region of Piraeus, while those from
the previous study came from a semi-urban region in north-
western Greece.
It follows from a comparison of the two studies, that we
cannot with any certainty assert that the simulations are more
eective than, and hence preferable to, real practical activities.
However, we would point out that actual experiments often
involve extra, background information that is not relevant to a
particular problem. This may prevent students from paying
attention to key stimuli relevant to the problem, by causing an
overload of the working memory [Kempa and Ward, 1988;
Johnstone and Letton, 1990 see the relevant discussion in
Kampourakis and Tsaparlis (2003)]. It is also important to
appreciate that simulations are, usually, safer, faster, more
economical and easier to perform and repeat than real experi-
ments. A good understanding of the relevant theory is of course
very important for problem solving. Students who lack the
requisite theoretical framework will not know where to look, or
how to look, in order to make observations appropriate to the
task in hand, or how to interpret what they see. Consequently,
much of the activity will be unproductive (Johnstone and Al-
Shuaili, 2001). Knowing what to observe, knowing how to
observe it, observing it and describing the observations are all
theory-dependent and therefore fallible and biased (Hodson,
1986). Last but not least, let us not forget that an important first
step in problem solving in science (after reading the problem)
is to make a drawing of the problem situation (Mettes et al.,
1980; Reif, 1981, 1983; Genya, 1983). Simulations are capable of
providing a better picture of a problem than is possible with a
simple drawing.
Appendix 1. Marking schemes for the two
problems
Tables 6 and 7
Table 6 Marking scheme for problem 1
Solution step
Solution
procedure
Numerical
computation
Moles n
s
of NH
3
dissolved in water 10.0 2.5
p
1
V
1
= n
1
RT 11.25
p
2
V
2
= n
2
RT 11.25
Conversion of 1C into K. 2.5
Knowledge or estimation of the value of R 2.5
Final moles of NH
3
in the flask (n
2
= n
1
n
s
) 20.0
Algebraic calculations 22.5
Numerical computation 10.0
Correct result (value and units) 7.5
Total marks 75.0 25.0
Paper Chemistry Education Research and Practice
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract., 2013, 14, 297--311 307
Appendix 2. Further details about the
simulations
Simulation of problem 1
Three screens were constructed for this visualization. The main
screen provides a visualization of the experimental set up, while
the other two play an auxiliary role. On starting the program,
picture 1 appears on the main page showing the experimental
set up. Initially the student is led to the screen which describes
the experiment; here he/she can read basic instructions and a
brief description of the software (Fig. 1). One can change
screens by using three buttons that are always shown and active
Table 7 Marking scheme for problem 2
Solution step Solution procedure Numerical computation
Moles n
1
of SO
2
dissolved in water 10.0 2.5
Knowledge of STP 2.5
Moles n
2
of SO
2
that were collected in a reversed tube 10.0 2.5
Conversion of 1C into K. 2.5
Knowledge or calculation of R 2.5
Moles n of SO
2
produced (n = n
1
+ n
2
) 20.0
Stoichiometric calculation of moles of O
2
(from chemical equation) 20.0
Calculation of pressure of O
2
(from the ideal-gas equation) 12.5
Algebraic calculations 10.0
Correct result (value and units) 5.0
Total marks 75.0 25.0
Fig. 1 Two screens of the simulation for problem 1, showing the Description of the system (a), and the instructions provided on opening the Help link (b).
Chemistry Education Research and Practice Paper
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
308 Chem. Educ. Res. Pract., 2013, 14, 297--311 This journal is c The Royal Society of Chemistry 2013
for a horizontal navigation within the software. Next, the
student can return to the start of the experiment and com-
mence and interact with the simulation.
As can be seen from Fig. 2, the experimental set up consists
of a vessel that contains gaseous ammonia. The vessel is
supplied with a pressure gauge that initially shows a reading
of 2 atm. The vessel is also supplied with a thermometer that
shows a reading of 27 1C. From the vessel, a tube, supplied with
a stopcock, joins the vessel to a beaker filled with water. Over
the beaker there is a dropper that contains phenolphthalein
indicator. This allows the student to check for the presence of
base (ammonia) in the beaker by introducing a few drops of the
indicator. The students can turn on the stopcock by moving the
cursor over it and by left clicking. When the stopcock is turned
on, the pressure gauge shows a rapid fall in pressure. The
student can turn o the stopcock and stop the flow of ammonia
gas. Eventually the pressure falls to 1 atm. Some bubbles
Fig. 2 The three screens of the visualization for the ammonia problem (problem 1).
Fig. 3 Three screens of the visualization for the sulfur combustion problem
(problem 2). The piston is gradually pushed.
Paper Chemistry Education Research and Practice
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract., 2013, 14, 297--311 309
appear at the end of the tube inside the water making the flow
of gas perceptible. Ammonia molecules in constant motion are
shown within the tube (when it is turned on) as well as in the
solution. The student is free to choose his/her actions; for
instance, if he/she chooses to drop indicator before passing
ammonia gas, there will be no color change in the solution. In
this case, however, if ammonia is passed after that, the solution
will immediately turn red.
Simulation of problem 2
The same design was used for visualizing problem 2. As can be
seen from Fig. 3, the experimental set up consists of a vessel
that contains, at the bottom, an amount of sulfur, as well as
oxygen gas. Oxygen molecules are shown as small blue balls,
while sulfur atoms are represented in yellow. The vessel is
closed at the bottom with a piston, which can be pushed by
the student. There is no pressure gauge or thermometer in
this case.
A tube which starts with a stopcock and ends in a beaker
filled with water emerges from the vessel. In the beaker there is
also an inverted graduated tube filled with water. This tube is
able to collect the sulfur dioxide gas that is not dissolved in the
water. The vessel carries a switch that can ignite the mixture.
As in the case of the simulation for problem 1, the Descrip-
tion of the experiment button provides a user guide, directing
the simulation in an ordered manner: first they ignite the
mixture and see the video, then they turn on the stopcock,
and finally they push the piston. The Help feature is also
present.
The student can start and watch a video, showing the
combustion of sulfur. Note that the students had previously
observed a demonstration of the combustion of sulfur, the
production of sulfur dioxide and its dissolution in water. After
combustion is complete, the student can transfer all the
produced gas into the beaker containing water by pushing
the piston. In the beaker, part of the gas is dissolved in the
water, and part is collected in the inverted tube. While this
occurs, gas bubbles appear within the inverted tube replacing
water. Sulfur dioxide molecules appear in constant motion in
the solution.
Fig. 3 shows a series of three shots, in which the piston is
gradually pushed, so that the produced gas is transferred into
the beaker containing water and the inversed test tube.
Acknowledgements
The authors are grateful to Dr Bill Byers for his significant
contribution to improving the readability of the paper and to
make clear some methodological and presentational issues.
References
Akaygun S. and Jones L. L., (2013), Dynamic visualizations:
tools for understanding particulate nature of matter, in
Tsaparlis G. and Sevian H. (ed.), Concepts of matter in science
education, vol. 19, Innovations in Science Education and
Technology, Springer, ISBN 978-94-007-5913-8.
Ardac D. and Akaygun S., (2004), Eectiveness of multimedia-
based instruction that emphasizes molecular representa-
tions on students understanding of chemical change, J.
Res. Sci. Teach., 40, 317337.
Ardac D. and Akaygun S., (2005), Using static and dynamic
visuals to represent chemical change at molecular level, Int.
J. Sci. Educ., 27, 12691298.
Baddeley A. D., (1992), Working memory, Science, 255, 556559.
Bannert M., (2002), Managing cognitive load recent trends in
cognitive load theory, Learn. Instruct., 12, 139146.
BouJaoude S., Salloum S. and Abd-El-Khalick F., (2004), Relation-
ships between selective cognitive variables and students ability
to solve chemistry problems, Int. J. Sci. Educ., 26, 6384.
Bowen C. W. and Phelps A. J., (1997), Demonstration-based
cooperative testing in general chemistry: a broader assess-
ment-of-learning technique, J. Chem. Educ., 74, 715719.
Burke K., Greenbowe T. and Windschitl M., (1998), Developing
and using conceptual computer animations for chemistry
instruction, J. Chem. Educ., 75, 16581661.
Butler W. M. and Grin H. C., (1979), Simulations in the
general chemistry laboratory with microcomputers, J. Chem.
Educ., 56, 543.
Cohen L. and Holliday M., (1982), Statistics for social scientists,
London: Harper & Row.
Deese W., Ramsey L. L., Walczyk J. and Eddy D., (2000), Using
demonstration assessments to improve learning, J. Chem.
Educ., 77, 15111516.
Demerouti M., Kousathana M. and Tsaparlis G., (2004), Acid
base equilibria, part II, eect of developmental level and
disembedding ability on students conceptual understand-
ing and problem-solving ability, Chem. Educator, 9, 132137.
Domin D. S., (1999a), Review of laboratory instructional styles,
J. Chem. Educ., 76, 543547.
Domin D. S., (1999b), A content analysis of general chemistry
laboratory manuals for evidence of higher-order cognitive
tasks, J. Chem. Educ., 76, 109112.
El-Banna H. A. A. M., (1987), The development of a predictive
theory of science education based upon information processing
hypothesis, PhD thesis, University of Glasgow, UK.
Genya J., (1983), Improving students problem-solving skills a
methodological approach for a preparatory chemistry
course, J. Chem. Educ., 60, 478482.
Hodson D., (1986), The nature of scientific observation, Sch. Sci.
Rev., 68 (242), 1729.
Johnstone A. H., (1993), Introduction, in Wood C. and Sleet R.
(ed.), Creative problem solving in chemistry, London: The
Royal Society of Chemistry, pp. ivvi.
Johnstone A. H., (2001). Can problem solving be taught? Univ.
Chem. Educ., 5, 6973.
Johnstone A. H. and Al-Shuaili A., (2001), Learning in laboratory:
some thoughts from the literature, Univ. Chem. Educ., 5, 4251.
Johnstone A. H. and El-Banna H., (1986), Capacities, demands
and processes a predictive model for science education, J.
Educ. Chem., 23, 8084.
Chemistry Education Research and Practice Paper
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
310 Chem. Educ. Res. Pract., 2013, 14, 297--311 This journal is c The Royal Society of Chemistry 2013
Johnstone A. H., Hogg W. and Ziane M., (1993), A working
memory model applied to physics problem solving, Int. J.
Sci. Educ., 15, 663672.
Johnstone A. H. and Letton K. M., (1990), Investigating under-
graduate laboratory work, Educ. Chem., 27(1), 911.
Josephsen J. and Kristensen A. K., (2006), Simulation of laboratory
assignments to support students learning of introductory
inorganic chemistry, Chem. Educ. Res. Pract., 7, 266279.
Kampourakis C. and Tsaparlis G., (2003), A study of the eect of
a practical activity on problem solving in chemistry, Chem.
Educ. Res. Pract., 4, 319333.
Kautz C. H., Lovrude M. E., Herron P. R. L. and McDermott L.
C., (1999), Research on student understanding of the ideal
gas law, Proceedings, 2nd International Conference of the
European Science Education Research Association (ESERA)
vol. 1, Germany: Kiel, pp. 8385.
Kempa R. F. and Ward J. E., (1988), Observational thresholds in
school chemistry, Int. J. Sci. Educ., 10, 275284.
Kerr J. F., (1963), Practical work in school science: an account of an
inquiry into the nature and purpose of practical work in school
science teaching in England and Wales, Leicester: Leicester
University Press.
Lajoie S. P., (1993), Computer environments as cognitive tools
for enhancing learning, in Lajoie S. P. and Derry S. J. (ed.),
Computers as cognitive tools, New Jersey: Lawrence Erlbaum
Hillsdale, pp. 261288.
Lajoie S. P., Azevedo R. and Fleiszer D. M., (1998), Cognitive
tools for assessment and learning in a high information flow
environment, J. Educ. Comp. Res., 18, 205235.
Lajoie S. P., Lavigne M. C., Guerrera C. and Munsie S. D., (2001),
Constructing knowledge in the context of BioWorld, Instr.
Sci., 29, 155186.
Larkin J. H. and Reif F., (1979), Understanding and teaching
problem solving in physics, Eur. J. Sci. Educ., 1, 191203.
Lawson A. E., (1978), Development and validation of the class-
room test of formal reasoning, J. Res. Sci. Teach., 15, 1124.
Lewalter D., (2003). Cognitive strategies for learning from static
and dynamic visuals. Learn. Instruct., 13, 177189.
Lowe R. K., (2003). Animation and learning: selective processing of
information in dynamic graphics, Learn. Instruct., 13, 157176.
Mayer R. E., (1997). Multimedia learning: are we asking the
right questions?, Educ. Psychol., 32, 119.
Mayer R. E., (2001). Multimedia learning, New York: Cambridge
University Press.
Mayer R. E. and Moreno R., (1998), A split-attention eect in
multimedia learning: evidence for dual information processing
systems in working memory. J. Educ. Psychol., 90, 312320.
Mettes C. T. C. W., Pilot A., Roossing H. J. and Kramers-Pal H.,
(1980), Teaching and learning problem solving in science,
Part I, General strategy, J. Chem. Educ., 57, 882885.
Miller G. A., (1956), The magical number seven plus or minus
two: some limits on our capacity for processing information,
Psych. Rev., 63, 8197.
Moreno R. and Mayer R., (1999), Cognitive principles of multi-
media learning: the role of modality and contiguity. J. Educ.
Psychol., 91, 358368.
Niaz M., (1991), Correlates of formal operational reasoning: a
neo-Piagetian analysis, J. Res. Sci. Teach., 28, 1940.
Niaz M., (1995), Relationship between student performance on
conceptual and computational problems of chemical equili-
brium, Int. J. Sci. Educ., 17, 343355.
Niaz M., de Nunez S. G. and de Pineda R. I., (2000), Academic
performance of high school students as a function of mental
capacity, cognitive style, mobility-fixity dimension, and crea-
tivity, J. Creat. Behav., 34, 1829.
Niaz M. and Robinson W. R., (1992), Manipulation of logical
structure of chemistry problems and its eect on student
performance, J. Res. Sci. Teach., 29, 211226.
Oakes K. and Rengarajan R., (2002), Practice makes perfect
E-learning simulation in training, http://www.questia.com/
library/1G1-94174474/practice-makes-perfect-e-learning, accessed
17 March 2013.
Overton T. L. and Potter N. M., (2011), Investigating students
success in solving and attitudes towards context-rich open-
ended problems in chemistry, Chem. Educ. Res. Pract., 12,
294302.
Reif F., (1981), Teaching problem solving a scientific
approach, Phys. Teach., 19, 329363.
Reif F., (1983), How can chemists teach problem solving?
Suggestions derived from studies of cognitive processes, J.
Chem. Educ., 60, 948953.
Rieber L. P., (1990). Using computer animated graphics in
science instruction with children, J. Educ. Psych., 82, 135140.
Roth W.-M., (1988), Short-term memory and problem solving in
physics education, Ontario Canada: Dept. of Science, Appleby
College Oakville.
Roth W.-M., (1994), Experimenting in a constructivist high
school physics laboratory, J. Res. Sci. Teach., 31,197223.
Sanger M. and Greenbowe T., (1997a), Common student
misconceptions in electrochemistry: galvanic, electrolytic,
and concentration cells, J. Res. Sci. Teach., 34, 377398.
Sanger M. and Greenbowe T., (1997b), Students misconcep-
tions in electrochemistry: current flow in electrolyte solu-
tions and the salt bridge, J. Chem. Educ., 74, 819823.
Sanger M., Phelps A. and Fienhold J., (2000), Using a computer
animation to improve students conceptual understanding of
a can-crushing demonstration, J. Chem. Educ., 77, 15171520.
Simon H. A., (1974), How big is a chunk? Science, 183, 482488.
Simon D. P. and Simon H. A., (1978), Individual dierences in
solving physics problems, in Siegler R. S. (ed.), Childrens
thinking: what develops? Hillstate NJ: Lawrence Erlbaum
Associates, pp. 325348.
Stamovlasis D. and Tsaparlis G., (2001), Application of complex-
ity theory to an information processing model in science
education, Nonlinear Dynamics Psychol. Life Sci., 5, 267286.
Stamovlasis D. and Tsaparlis G., (2003), A complexity theory
model in science education problem solving: random walks
for working memory and mental capacity, Nonlinear
Dynamics Psychol. Life Sci., 7, 221244.
Stamovlasis D. and Tsaparlis G., (2012), Applying catastrophe
theory to an information-processing model of problem
solving in science education, Sci. Educ., 96, 392410.
Paper Chemistry Education Research and Practice
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online
This journal is c The Royal Society of Chemistry 2013 Chem. Educ. Res. Pract., 2013, 14, 297--311 311
St Clair-Thompson H., Overton T. and Bugler M., (2012), Mental
capacity and working memory in chemistry: algorithmic versus
open-ended problemsolving, Chem. Educ. Res. Pract., 13, 484489.
Sweller J., (1988), Cognitive load during problem solving: eects
on learning, Cogn. Sci., 12, 257285.
Sweller J. and Chandler P., (1991), Cognitive load theory and the
format of instruction, Cogn. Instr., 8, 292332.
Tsaparlis G., (1998), Dimensional analysis and predictive
models in problem solving, Int. J. Sci. Educ., 20, 335350.
Tsaparlis G., (2005), Non-algorithmic quantitative problem solving
in university physical chemistry: a correlation study of the role of
selective cognitive variables, Res. Sci. Tech. Educ., 23, 125148.
Tsaparlis G., (2009), Learning at the macro level: the role of
practical work, in Gilbert J. K. and Treagust D. F. (ed.),
Multiple representations in chemical education, Milton Keynes:
Springer, pp. 109136.
Tsaparlis G. and Angelopoulos V., (2000), A model of problem-
solving: its operation, validity, and usefulness in the case of
organic-synthesis problems, Sci. Educ., 84, 151153.
Tsaparlis G., Kousathana M. and Niaz M., (1998), Molecular-
equilibrium problems: manipulation of logical structure
and of M-demand, and their eect on student performance,
Sci. Educ., 82, 437454.
van Bruggen J. M., Kirschner P. A. A. and Jochems W., (2002),
External representation of argumentation in CSCL and the
management of cognitive load, Learn. Instruct., 12, 121138.
Williamson V. M. and Abraham M. R., (1995), The eects of
computer animation on the particulate mental models of
college chemistry students, J. Res. Sci. Teach., 32, 521534.
Witkin H. A., (1978), Cognitive styles in personal and cultural
adaptation, Worcester, MA: Clark University Press.
Witkin H. A., Oltman P. K., Raskin E. and Karp S. A., (1971),
A manual for the embedded figures test, Palo Alto: Consulting
Psychologists Press.
Zoller U., (1993), Are lecture and learning compatible? Maybe
for LOCS: unlikely for HOCS, J. Chem. Educ., 70, 195197.
Zoller U. and Tsaparlis G., (1997), HOCS and LOCS students:
the case of chemistry, Res. Sci. Educ., 27, 117130.
Chemistry Education Research and Practice Paper
P
u
b
l
i
s
h
e
d

o
n

0
7

M
a
y

2
0
1
3
.

D
o
w
n
l
o
a
d
e
d

o
n

0
6
/
0
2
/
2
0
1
4

2
3
:
5
7
:
4
2
.

View Article Online