International Journal of Science Education

ISSN: 0950-0693 (Print) 1464-5289 (Online) Journal homepage: http://www.tandfonline.com/loi/tsed20

Making inquiry-based science learning visible: the

influence of CVS and cognitive skills on content
knowledge learning in guided inquiry

Anita Stender, Martin Schwichow, Corinne Zimmerman & Hendrik Härtig

To cite this article: Anita Stender, Martin Schwichow, Corinne Zimmerman & Hendrik Härtig
(2018): Making inquiry-based science learning visible: the influence of CVS and cognitive skills
on content knowledge learning in guided inquiry, International Journal of Science Education, DOI:
10.1080/09500693.2018.1504346

To link to this article: https://doi.org/10.1080/09500693.2018.1504346

Published online: 13 Aug 2018.

Submit your article to this journal

Article views: 8

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=tsed20
INTERNATIONAL JOURNAL OF SCIENCE EDUCATION
https://doi.org/10.1080/09500693.2018.1504346

Making inquiry-based science learning visible: the influence of

CVS and cognitive skills on content knowledge learning in
guided inquiry
a b
Anita Stender , Martin Schwichow , Corinne Zimmermanc and Hendrik Härtig a

a
Department of Physics Education, University of Duisburg-Essen, Duisburg, Germany; bDepartment of Physics
Education, University of Education Freiburg, Freiburg im Breisgau, Germany; cDepartment of Psychology,
Illinois State University, Normal, IL, USA

ABSTRACT ARTICLE HISTORY

Many science curricula and standards emphasise that students Received 26 March 2018
should learn both scientific knowledge and the skills associated Accepted 22 July 2018
with the construction of this knowledge. One way to achieve this
KEYWORDS
goal is to use inquiry-learning activities that embed the use of Inquiry learning; analogical
science process skills. We investigated the influence of scientific reasoning; reading ability;
reasoning skills (i.e. conceptual and procedural knowledge of the control-of-variables strategy
control-of-variables strategy) on students’ conceptual learning
gains in physics during an inquiry-learning activity. Eighth graders
(n = 189) answered research questions about variables that
influence the force of electromagnets and the brightness of light
bulbs by designing, running, and interpreting experiments. We
measured knowledge of electricity and electromagnets, scientific
reasoning skills, and cognitive skills (analogical reasoning and
reading ability). Using structural equation modelling we found no
direct effects of cognitive skills on students’ content knowledge
learning gains; however, there were direct effects of scientific
reasoning skills on content knowledge learning gains. Our results
show that cognitive skills are not sufficient; students require
specific scientific reasoning skills to learn science content from
inquiry activities. Furthermore, our findings illustrate that what
students learn during guided inquiry activities becomes visible
when we examine both the skills used during inquiry learning and
the process of knowledge construction. The implications of these
findings for science teaching and research are discussed.

Science is not just a body of knowledge that reflects current understanding of the world; it
is also a set of practices used to establish, extend, and refine that knowledge (National
Research Council, 2012). This twofold meaning of science is reflected in science curricula
of various countries (e.g. USA: NGSS Lead States, 2013; Germany: KMK, 2005). As wide-
spread access to scientific knowledge is available to anyone with Internet access, the ability
to find, interpret and evaluate the quality of the resources used for generating knowledge is
becoming increasingly important. Many reforms and innovations in science education
focus on scientific reasoning and argumentation skills, as they are crucial for participating

CONTACT Anita Stender anita.stender@uni-due.de

© 2018 Informa UK Limited, trading as Taylor & Francis Group
2 A. STENDER ET AL.

in modern knowledge-based societies (Chiappetta, 1997; NRC, 2010). Understanding the

process of knowledge generation in science is thus one of the so-called twenty-first century
skills (Fischer et al., 2014; NRC, 2010; Osborne, 2013).
Many science educators and curriculum developers argue that students should apply
scientific methods and answer research questions by generating and analysing data in
order to acquire both science knowledge and scientific reasoning skills (Abd-El-Khalick
et al., 2004). Such active learning processes, which are consistent with constructivist the-
ories of learning (Solomon, 1994) are typically referred to as inquiry learning (Bell,
Smetana, & Binns, 2005). However, there is little empirical evidence for the positive
effects of unassisted inquiry learning on science achievement (for a review see Kirschner,
Sweller, & Clark, 2006). Rather, studies that show achievement gains utilise scaffolding
techniques to guide students through the inquiry process (Alfieri, Brooks, Aldrich, &
Tenenbaum, 2011).

Inquiry learning
For at least three decades, the term inquiry learning has been used in science education as a
synonym for ‘good and meaningful’ science teaching (Abd-El-Khalick et al., 2004; Ander-
son, 2002). The assertion that students should learn science by mimicking the process of
knowledge construction in science is not new (Dewey, 1910) and has been a leading idea in
science education reforms since the 1950s (Chiappetta, 1997). In science education,
inquiry learning describes an approach in which students learn by actively using scientific
methods to answer research questions (Anderson, 2002; Bell et al., 2005; Chiappetta,
1997). This definition implies two minimum requirements for characterising an activity
as inquiry learning: (a) students answer research questions by (b) applying scientific
methods. Accordingly, activities such as building models of atoms or assembling wildfl-
ower collections are not inquiry-learning activities because students are not answering
research questions. Collecting information in a library or on the Internet to find out
how liquid oxygen is produced is not an inquiry-learning activity because even though stu-
dents are answering a question, they are not applying scientific methods. This analysis of
what counts as ‘inquiry learning’ does not imply that these activities are not important, it
just means that they are not sufficient to meet a strict definition of inquiry learning (Bell
et al., 2005).
However, the whole inquiry learning cycle of asking research questions, collecting data,
and interpreting the results is not always used in science classes. In praxis, activities vary.
In some cases, the teacher provides the research questions, methods of data collection, and
interpretation; in other cases, the teacher leaves the responsibility for some or all of these
components of inquiry learning to students. Bell et al. (2005) define four levels of inquiry

Table 1. Levels of Inquiry Outlined by Bell et al. (2005).

Source of the Research Source of Data Collection Source of Interpretation of
Levels Questions Methods Results
Level 0: Given by teacher Given by teacher Given by teacher
Verification
Level 1: Structured Given by teacher Given by teacher Open to student
Level 2: Guided Given by teacher Open to student Open to student
Level 3: Open Open to student Open to student Open to student
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 3

learning based on the amount of guidance from the teacher (Table 1). In verification (level
0) the teacher provides the research questions, methods of data collection, and guidance
for how to interpret the data to answer the research questions. Level 0 activities are tra-
ditional ‘cookbook-like’ activities that provide a recipe for answering research questions.
In structured inquiry learning (level 1) students are provided with the research questions
and the data collection methods but are not given guidance on how to interpret the data.
In guided inquiry learning (level 2) students are given the research questions but no further
guidance. In open inquiry learning (level 3) students choose a data collection method and
interpret data to answer their own research questions without any guidance. This rubric
captures the varying levels of complexity of inquiry learning in classroom settings and
the amount of guidance provided. Hence, younger and less experienced students should
work on lower-level activities, whereas students with more experience should work on
more complex inquiry-learning activities to practice and improve their scientific process
skills. Researchers can also use this rubric to interpret evidence about the effect of
inquiry learning, as it provides criteria to systematically compare different inquiry-learn-
ing activities used in classrooms or labs.

Guidance and inquiry learning

Recent reviews and meta-analyses of intervention studies contrasting inquiry learning and
more explicit or didactic teaching methods found that inquiry learning has no general
positive advantage for student achievement. However, the degree of guidance moderates
student learning (Alfieri et al., 2011; Minner, Levy, & Century, 2010). For example,
Alfieri et al. (2011) found that unguided inquiry learning settings (level 3 of Bell’s
rubric) had an average effect size that was negative, indicating that explicit teaching was
more beneficial than unguided inquiry for several different outcome measures, including
posttest scores (e.g. science content knowledge), process scores (e.g. problem-solving
skills), and self-ratings. One proposed explanation for this finding is that level 3 inquiry
activities require a greater number of mental operations, and therefore a larger working
memory capacity as well as better executive control of attention, in comparison to the
other levels. Therefore, unguided inquiry learning settings are unlikely to be effective
because they ignore the limitations of working memory and executive control (Kirschner
et al., 2006). Similarly, a meta-analysis conducted by Carolan, Hutchins, Wickens, and
Cumming (2014) revealed that students who receive adequate guidance during the
inquiry learning process (i.e. level 1 or 2) learn more (g = 0.15) than students who are
taught the same content with less guidance (i.e. level 3). According to Kirschner et al.
(2006) guidance can lower the high demands on working memory and executive
control that are required by certain levels of inquiry learning and thus enable students
to encode and store novel information in long-term memory.
The influence of guided inquiry on student achievement is analysed in more detail in a
meta-analysis by Lazonder and Harmsen (2016). They compared effect sizes of studies con-
trasting guided inquiry-learning activities against a reference group that did not receive gui-
dance. In their analysis, they outlined six types of guidance (i.e. process constraints, status
overviews, prompts, heuristics, scaffolds, and explanation) and reported findings with
respect to typical learning outcomes assessed after an inquiry activity as well as performance
measures and the quality of the products created during the activity. They found that
4 A. STENDER ET AL.

learning outcomes as well as the process quality measures did not differ between different
types of guidance, whereas the quality of the products created during inquiry was greater
when students received a specific explanation of how to perform an action (Lazonder &
Harmsen, 2016). Providing students with explanations for science processes during struc-
tured or guided inquiry (level 1 and 2) might, therefore, be an important step for students to
generate an adequate understanding of the conceptual content under investigation (Lazon-
der & Harmsen, 2016). To further examine this assumption, we describe empirical findings
regarding the impact of students’ understanding and application of science process skills
during guided inquiry activities on their learning of science content.

Scientific reasoning skills

Scientific reasoning is the intentional construction of knowledge via generating, testing, and
revising hypotheses (Klahr, 2000; Zimmerman, 2007). Accordingly, scientific reasoning
skills are crucial for generating new knowledge via inquiry-based learning activities. Many rel-
evant scientific reasoning skills are brought together under the term control-of-variables strat-
egy (CVS). CVS enables students to design an experiment in a way that alternative causal effects
or interactions can be excluded. Therefore, all variables except the one being investigated
should ideally be held constant (or ‘controlled’) across experimental conditions (Dewey,
2002; Popper, 1966). Chen and Klahr (1999) outline the key components of CVS. Procedurally,
CVS includes the ability to recognise confounded and unconfounded experiments and the
ability to design experiments in which conditions differ with respect to a single contrasting
variable. It is also important to understand CVS conceptually, which involves the ability to
make appropriate inferences from the results of unconfounded experiments (e.g. only infer-
ences about the variable being tested are warranted), as well as an awareness of ‘the inherent
indeterminacy of confounded experiments’ (Chen & Klahr, 1999, p. 1098).
Even though CVS is crucial for inquiry learning, evidence from multiple studies shows that
most students and even adults do not have a generalised understanding of CVS because their
ability to identify, select, or design controlled experiments depends on the task context or
situational factors (Croker & Buchanan, 2011; Koslowski, 1996; Linn, Clement, & Pulos,
1983; Zimmerman & Croker, 2013). For example, the results of two meta-analyses on CVS
intervention studies show significantly larger effect sizes for studies utilising hands-on assess-
ment instruments compared to studies utilising multiple-choice or open-ended assessments
(Ross, 1988; Schwichow, Croker, Zimmerman, Höffler, & Härtig, 2016). From research on
science performance assessments, it is well known that students’ scores on paper-and-
pencil tasks are higher than their scores on performance tasks that require interacting with
a physical apparatus (Baxter & Shavelson, 1994; Ruiz-Primo & Shavelson, 1996). The discre-
pancy between paper-and-pencil and performance measures may reflect a production
deficiency: students who have a conceptual understanding of a strategy are not necessarily
automatically able to apply this strategy in different situations (Marschner, Thillmann,
Wirth, & Leutner, 2012). In order to promote a generalised understanding of CVS, it may
be necessary to train students on a variety of different tasks.
Evidence for the positive impact of CVS on learning content knowledge comes from
intervention studies that focus on CVS but that additionally measure students’ content
knowledge. Students who learned and applied CVS obtain correct knowledge about the
causal relations being investigated when compared to students who have not mastered
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 5

the conceptual and procedural demands of CVS (Chen & Klahr, 1999; Klahr, Chen, &
Toth, 2001; Künsting, Thillmann, Wirth, Fischer, & Leutner, 2008; McElhaney, 2011).
However, these studies only coded whether students showed a change in beliefs regarding
the causal status of the variables under investigation and not if students could transfer or
generalise this newly acquired knowledge to related scientific concepts. For example, when
learning how to control variables to investigate the effect of mass on the period of a pen-
dulum, the speed of a sinking object, or the distance an object travels down a ramp, does
this belief change about mass translate into more scientifically accurate conceptions about
the physics of mass? It is an open question whether students can generalise the belief
changes that occur about the causal status of the variables under investigation into a
more robust understanding of the science content.

Cognitive skills
Scientific reasoning skills are crucial for inquiry learning, but factors that underlie individ-
ual differences are largely unknown (Chen & Klahr, 1999; Klahr et al., 2001; Künsting
et al., 2008; McElhaney, 2011). General information-processing skills such as analogical
reasoning, encoding, reading skills, and strategy development seem to be necessary to
coordinate scientific reasoning processes (Morris, Croker, Masnick, & Zimmerman,
2012; van der Graaf, Segers, & Verhoeven, 2016). These skills help students to acquire,
coordinate and store new information in memory.
There are two lines of research having investigated the relationship between scientific
reasoning and cognitive skills. One line of research has focused on whether scientific reason-
ing is a unique skill that is independent of other cognitive skills (Koerber, Mayer, Osterhaus,
Schwippert, & Sodian, 2015; Mayer, Sodian, Koerber, & Schwippert, 2014; Nehring, Nowak,
Belzen, & Tiemann, 2015). For example, Koerber et al. (2015) developed a 66-item scale that
assessed five different aspects of scientific thinking to investigate whether scientific reasoning
develops as a unitary skill or as multiple non-related sub-skills. In these studies, cognitive skills
(i.e. analogical reasoning and text comprehension) were measured to demonstrate the discri-
minant validity of measures of cognitive skills from measures of scientific reasoning. Text
comprehension was of interest because many scientific reasoning tests and inquiry-learning
materials require reading and writing. Reported correlations between analogical reasoning
and scientific reasoning ranged from r = .15 (Koerber et al., 2015) to r = .39 (Nehring et al.,
2015) and between reading skills and scientific reasoning from r = .17 (Nehring et al.,
2015) to r = .44 (Mayer et al., 2014).
A second line of research focuses on explaining individual differences in scientific
reasoning as a function of differences in cognitive skills (Osterhaus, Koerber, & Sodian,
2017; van der Graaf et al., 2016). These researchers propose that verbal skills are crucial
for the development of scientific reasoning skills because of the need to encode and
store information in memory via the verbal route (Baddeley, 2000). In contrast to the
research described above, cognitive skills do not serve as control variables but as variables
to test theories about the development of scientific reasoning. For example, van der Graaf
et al. (2016) report correlations between students’ performance on CVS tasks and their
scores on grammar and vocabulary tests. As these studies investigate the impact of mul-
tiple additional variables (e.g. understanding the nature of science, advanced theory of
mind) on individual differences in scientific reasoning, they typically use structural
6 A. STENDER ET AL.

equation modelling to consider covariance between all assessed variables. They report zero
and even negative factor loadings for the direct path between cognitive skills and scientific
reasoning skills. This pattern shows that even though cognitive abilities have medium-size
correlations with scientific reasoning skills, their effects are mediated rather than direct.
This finding is supported by results of the Munich longitudinal study, which shows posi-
tive relationships between cognitive skills and multiple scientific reasoning measures, but
also that early scientific reasoning skills are a better predictor of later scientific reasoning
than cognitive skills (Bullock, Sodian, & Koerber, 2009).

Research questions
As in many other contexts, ‘knowing that’ (i.e. declarative knowledge) is not the same as
‘knowing how’ (i.e. procedural knowledge). It is important, therefore, to differentiate
between having a conceptual understanding of the control-of-variables strategy (CVS
understanding) and having the ability to apply the control-of-variables strategy in a
hands-on task (CVS hands-on skills) (Marschner et al., 2012). Additionally, a meaningful
relationship between cognitive skills and scientific reasoning skills has been shown
(Koerber et al., 2015; Mayer et al., 2014; Nehring et al., 2015). As a consequence, cognitive
skills can also have a meaningful impact on the construction of knowledge during inquiry.
However, the common impact of cognitive skills, conceptual understanding of CVS, and
CVS hands-on skills on science content knowledge during guided inquiry learning has not
been empirically demonstrated. In the current study, we report a re-analysis of the data
from a study that investigated the differential effects of hands-on versus paper-and-
pencil training on students’ learning of CVS skills (Schwichow, Croker, Zimmerman, &
Härtig, 2016). This current analysis differs from the previous study in that we address
the following research questions:

(1) To what extent does the learning of content knowledge during an inquiry-learning
activity depend on scientific reasoning skills (i.e. CVS understanding, CVS hands-
on skills) and cognitive skills (i.e. analogical reasoning, reading ability)?
(2) Is a conceptual CVS understanding sufficient for successful learning of content knowl-
edge in a guided-inquiry activity, or are CVS hands-on skills necessary as well?

As scientific reasoning skills have been shown to support learning of content knowledge
in guided inquiry-learning activities (Chen & Klahr, 1999; Künsting et al., 2008; McElha-
ney, 2011), we assume for the first research question, that CVS skills should have a higher
impact on content learning than cognitive skills. For the second research question, we pre-
dicted that CVS hands-on skills would be a more important contributor to content knowl-
edge learning than CVS understanding.

Method
Participants
This study is based on a reanalysis of data from an intervention study investigating the differ-
ential effects of hands-on and paper-and-pencil training (Schwichow, Zimmerman, Croker,
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 7

& Härtig, 2016). The original study assessed control-of-variables strategy skills (CVS under-
standing and CVS hands-on skills) at the group level (hands-on vs. paper-and-pencil
training); this reanalysis investigates the impact of cognitive skills and CVS skills on
physics content knowledge gains at the individual level. A total of 189 students (aged 12
to 15 years, 54% female) from eight classes of two comprehensive schools in a suburban
area in northern Germany participated in the original study. As data collection occurred
over two days we only have complete datasets for 161 students. Both schools were
academically diverse, including students with special educational needs as well as students
approaching a university entrance exam. Both schools enrol equivalent numbers of low-,
medium-, and high-achieving students according to their elementary school reports.
Specific demographic information regarding student ethnicity, socioeconomic status, and
school achievement was not collected due to a lack of permission to collect this information.

Design and procedure

The original study took place during the last two weeks of the school year over the course
of two days. One author and an assistant provided the instruction in all classes, and all
activities were carried out identically in all science classes as illustrated in Figure 1.
During the pretest we measured students’ physics content knowledge, their conceptual
understanding of CVS, analogical reasoning, and reading abilities. The inquiry-based
learning activities consisted of three phases: (a) CVS introduction, (b) CVS training,
and (c) individual experimentation. In the CVS introduction phase, students were
taught about the importance of controlling variables via a short lesson. The instructor
introduced a situation in which the goal was to determine the factors that influenced
the height of bouncing balls (iron vs. ping-pong). After students predicted that the
ping-pong ball would bounce higher, the instructor’s demonstration induced cognitive
conflict by confounding the type of ball, the height at which the ball was dropped, the
height of the surface the ball was dropped on, and the material of that surface (Styrofoam
vs. steel). Thus, in contrast to the students’ prediction, the iron ball bounced higher. After a
group discussion in which students noted that the test was ‘unfair’, the teacher read a

Figure 1. Study design and sequence of events.

Note: All inquiry-based learning activities are shaded grey and all assessment phases in white. CVS means control-of-vari-
ables strategy. Students in both CVS training conditions tested hypotheses about variables influencing the force of elec-
tromagnets by analysing data. CVS hands-on measures were taken during individual experimentation.
8 A. STENDER ET AL.

standardised explanation and performed a controlled experiment suggested by students.

This demonstration/lesson ensured that all students were introduced to the logical and
conceptual aspects of CVS via an established instructional technique (Lawson &
Wollman, 1976).
In the CVS training phase, students were assigned to one of two training conditions
(hands-on or paper-and-pencil) based on their CVS pretest scores. One student from
each pair of students with equal pretest scores was randomly assigned to one of the two
training conditions; the other student from the pair was then place in the other training
condition. For 30 minutes, the two groups worked in separate rooms on two CVS training
tasks. Students in both training conditions worked in pairs or triads (never individually)
on an electromagnetism task. In the first training task, students were asked to plan and
interpret experiments to test whether the core material of an electromagnet (iron vs. alu-
minium) impacts its force. In the second training task, they investigated whether the
battery type (1.5 V vs. 9 V) impacts the force of an electromagnet. In both tasks, and
for both conditions, students were asked to draw a sketch of the experimental set-up
they had planned. No feedback was provided. Students in the hands-on group then con-
ducted the experiment they planned using physical equipment. Students in the paper-and-
pencil group were given a photograph of a controlled experiment, including the resulting
electromagnetic force. Students interpreted their own results (hands-on) or the results pre-
sented in the photograph (paper-and-pencil) and decided whether the hypothesis under
consideration was supported by the results or not by selecting the corresponding mul-
tiple-choice option (see https://goo.gl/whzRR8 for instructional materials). In summary,
the only intended difference between the two training conditions was that the hands-on
group had to plan, run, and interpret experiments using a physical apparatus (i.e. coils,
wires, and batteries) whereas the paper-and-pencil group planned experiments without
the physical apparatus and interpreted experiments presented to them in a photograph.
Type of training condition had no impact on students’ CVS understanding after this
phase in the original study (Schwichow et al. 2016) Hence, both training conditions pro-
vided the same learning opportunities for developing a conceptual understanding of CVS
and of the content.
In the individual experimentation phase students individually planned and conducted
experiments and then interpreted the outcomes with respect to the hypotheses under con-
sideration. One task was identical to the training task on electromagnetism (testing the
impact of core material and battery type on the force of an electromagnet). The other
task was a transfer task in which students had to investigate whether the material of
wires (constantan vs. copper) used to connect a light bulb with a battery and the
battery type (1.5 V vs. 9 V) have an impact on the brightness of light bulbs. Students con-
trasted wires made of copper and constantan (a copper–nickel alloy also known as Eureka)
because constantan has a ten times higher specific resistance than copper, which cause a
visible difference in the brightness of light bulbs. Both the electromagnetism and wire tasks
had two independent variables with two values each and one dependent variable. The four
hypotheses (two in each task) were introduced sequentially in a booklet with cover stories
about fictitious students who wanted to test causal relationships (see https://goo.gl/
whzRR8). Students had to choose appropriate materials to design controlled experiments
in which they contrasted two experimental conditions. After planning the experiments,
but before conducting the experiments, a research assistant took a photograph of the
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 9

students’ experimental designs. After conducting each experiment, students had to choose,
based on their observation of the experimental contrast, whether the hypothesis was sup-
ported or not.
It is important to note that the CVS training phase and the individual experimentation
phase have the characteristics of guided inquiry learning. As students answered provided
research question by drawing inferences from experiments they created and without step-
by-step instructions, these tasks can be characterised as level 2 guided inquiry (Bell et al.,
2005). Students received no further guidance, feedback, or information about the expected
outcome of the experiments.

Measures
CVS understanding
A multiple-choice pretest was administered before the learning activities to test students’
conceptual understanding of the control-of-variables strategy (CVS). All 23 items involved
CVS problems in the context of heat/temperature or electricity/electromagnets, which are
two content domains that are part of the state science curriculum for middle schools in
northern Germany. The items were designed based on the theoretical conceptualisation
of CVS by Chen and Klahr (1999) and cover the three CVS sub-skills: identifying con-
trolled experiments, interpreting the outcome of controlled experiments, and interpreting
the outcome of confounded experiments (i.e. indeterminacy). For items about identifying
controlled experiments, students had to identify the one controlled experiment from a set
of four experiments (including three confounded experiments). For items about interpret-
ation, students were presented with a drawing of either a controlled or a confounded
experiment. For controlled experiments, students had to draw the correct conclusion
based on the presented outcome. For confounded experiments, they had to choose that
no valid conclusion could be drawn from this experiment (see https://goo.gl/whzRR8
for items and Schwichow, Christoph, Boone, and Härtig (2016) for results of a validation
study). We constructed three different test booklets to prevent students from copying
answers from other students. Each booklet consisted of 12 out of 23 possible items: six
heat/temperature items and six electricity/electromagnetism items. Six of each of the
two problem types were included. There was an overlap between the test booklets of at
least six items. All items presented experiments with three independent variables (two
levels each). Students received one point for every correct answer.

Figural analogical reasoning

Students’ analogical reasoning skills were measured by the non-verbal figural analogies
scale of the German-wide established test for middle-school children (Kognitiver Fähig-
keitstest [KFT]; Heller & Perleth, 2000). Each of the items starts with a pair of figures
that are related according to a specific rule. In the item, a third figure is given, which is
the first figure of a second pair. Students have to choose one figure from a set of five
choices to be paired with the third figure according to the same rule that links the first
pair of figures. For example, if the left side shows a black and a white triangle and the
right side shows a black square, students have to choose the white square from the
answer options because the rule is ‘same shape but different colour’. Students had eight
minutes to answer 25 items.
10 A. STENDER ET AL.

Test of reading abilities

Reading ability was measured using an established instrument for middle- and high-
school students (Lesegeschwindigkeits-und-verständnistest [LGVT]; Schneider, Schlag-
müller, & Ennemoser, 2007). The instrument consists of a text of 1700 words that students
had to read within four minutes. The text contains gaps with missing words that students
were asked to fill in by choosing the correct term from a set of three words. They were
instructed to mark the last word they read within four minutes. Students’ answers were
coded by (a) counting the correct filled gap words (reading comprehension measure)
and (b) by counting the number of words students read within four minutes (reading
speed measure).

Physics content knowledge test

An identical physics content knowledge test was administered before and after the
inquiry-based learning activities. The 14 multiple-choice and 4 open-response items
covered concepts of electric circuits, resistance, and electromagnets. None of the
items required mathematical skills, as the focus was on knowledge (e.g. What are
the components of an electromagnet?) and conceptual understanding (e.g. Why is
the wire of an electromagnet coiled?) of physics concepts. Students got one point for
every correct answer. One rater coded all open-ended responses, and a second rater
coded 20% of the responses. The inter-rater agreement was high (between κ = .84
and κ = 1).

CVS hands-on measure

To evaluate the quality of students’ individual experiments we analysed the photographs of
experiments and responses relating to the hypotheses under consideration. One point was
assigned to an experiment when (a) the corresponding photograph showed a controlled
experiment, and (b) the student correctly evaluated whether the hypothesis was supported
or unsupported (see https://goo.gl/whzRR8 for sample worksheets). Students could receive
a maximum of four points (two for electromagnets and two for light bulbs). All photo-
graphs were coded by the second author and a research assistant. Inter-rater agreement on
a double-coded sample of 25% of the experiment photographs was high (between κ = .83
and κ = .93).

Data analysis
The aim of the reanalysis was to identify the common effect of cognitive skills, CVS under-
standing, and CVS hands-on skills on physics content knowledge gains. Therefore, we
simultaneously analysed the effect of cognitive skills (reading ability and analogical
reasoning) and inquiry skills (CVS understanding and CVS hands-on skills) on student
gains in content knowledge by analysing the data using structural equation modelling
(SEM). We utilised full information maximum likelihood (FIML) to deal with missing
values and robust maximum likelihood estimators to calculate parameters and standard
errors (Kaplan, 2005). The SEM models were calculated with the lavaan-package in R
(Rosseel, 2012).
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 11

Scaling
To create scales for reading comprehension, reading speed, and figural analogies we trans-
formed raw values into t-values. Based on the sum of correct responses we assigned to
every student a corresponding t-value according to the grade-specific transformation
tables in the test manuals. However, as we do not have equivalent information about
the population distribution of students’ abilities on other measures (CVS, content knowl-
edge) we used a one-dimensional Rasch model to create scales for these variables (see
Table 2). The Rasch model estimates student abilities and item difficulties based on
student answers by calculating the probability of solving an item for every item and
every student. To create unrestricted scales not limited to probabilities of 0 to 1 probability
measures are transformed to a logit scale extending from negative infinity to positive
infinity. Lower logit values represent easier items or less able students and higher values
correspond to more challenging items or more able students (Boone, Staver, & Yale,
2014). We calculated CVS understanding and content knowledge person measures as
weighted likelihood estimations (Warm, 1989) in the TAM-package in R (Kiefer,
Robitzsch, & Wu, 2017). We did not utilise Rasch-scaling for the CVS-hands-on items,
as WLE estimates based on four items are highly unreliable. Instead, we calculated a
latent CVS hands-on factor in the SEM model.

Computing change scores

The aim of this study is to explain students’ gains in physics content knowledge after par-
ticipating in guided inquiry. Thus, scores of change in knowledge for every student serve as
the dependent variable in the SEM models. To calculate change scores based on the Rasch-
model we transferred the three-dimensional data structure (items X persons X time) into a
two-dimensional structure (items X persons) by stacking students’ answers to pre- and
post-tests. The resulting data matrix contains twice as many rows as there are students.
After calculating person measures for every entry of the data matrix we separated students’
measures again to receive a pre- and posttest measure for every student (Embretson,
1991). Based on these measures we computed gain scores by subtracting pre- from posttest
measures.

Comparison of SEM Models

We analysed the common impact of analogical reasoning and reading skills, CVS under-
standing, and CVS hands-on skills on gains in physics content knowledge during inquiry
learning using SEM. To decide whether CVS skills or general cognitive skills (analogical

Table 2. Characteristics of weighted likelihood estimates (WLE) obtained by Rasch-analyses.

MNSQ Infit
Instrument Number of items min. – max. WLE-Reliability
Physics knowledge pretest 18 0.86–1.18 .63
Physics knowledge posttest 18 0.88–1.25 .71
CVS understanding 23 0.85–1.15 .73
Note: WLE = Weighted Likelihood Estimates (Person estimates generated form a Rasch-Analyses); MNSQ – Infit = Mean-
square inlier-sensitive fit (fit of items to total scale). Items fit to the scale when MNSQ-Infit is between 0.8 and 1.2;
WLE-Reliability = Reliability of the total scale generated by the Rasch-analysis (interpretation similar to Chronabach’s
Alpha; see Lincare, 2002). We conducted no Rasch analyses for the other constructs because (a) reading speed,
reading comprehension, and analogical reasoning were scaled with t-values, and (b) CVS Hands-On Skills were estimated
as latent factors with two manifest indicators.
12 A. STENDER ET AL.

Figure 2. Proposed SEM models for comparing the influence of CVS skills, analogical reasoning, and
reading abilities on content knowledge gains.
Note: CVS means control-of-variables strategy. The arrows represent directed effects from one variable to another; oval
shapes represent latent variables, whereas rectangular shapes represent observed variables. Measurement error terms
and model specification error terms are omitted for simplicity.

reasoning and reading skills) are necessary or sufficient for learning content knowledge in
inquiry learning settings we compared three proposed SEM Models (Figure 2). We assume
in the CVS model, that only CVS understanding and CVS hands-on skills influence gains
in physics content knowledge. In the Analogical reasoning and reading model, students’
gains in content knowledge are predicted only by reading and analogical reasoning. In
the Analogical reasoning, reading, and CVS model, we assume that analogical reasoning,
reading skills, CVS understanding, and CVS hands-on skills predict content knowledge
gains. The empirical decision as to which model best fit the data structure was made on
the basis of Root Mean Square Error of Approximation (sufficient fit RMSEA < .06) as
the overall fit index and the Comparative Fit Index (sufficient fit CFI ≥ .95) and the
Tucker-Lewis Index (sufficient fit TLI ≥ .95) as incremental fit indices. Models which
sufficient fit indices and / or higher values on fit indices are more sufficient for explaining
the data structure than models with insufficient or lower values on fit indices (Schreiber,
Nora, Stage, Barlow, & King, 2006).

Results
The aim of this study was to estimate the effect of cognitive skills (reading ability and ana-
logical reasoning) and scientific reasoning (understanding and hands-on skills of the
control-of-variables strategy (CVS)) on gains in physics content knowledge. To get an
overview of the data structure we first analysed pairwise correlations between all assessed
variables (Table 3). Concerning our first research question the following correlation
pattern is interesting: CVS understanding correlates with CVS hands-on (r = .35, p
< .001) and gain scores in physics knowledge only correlate with measurements of CVS
(CVS understanding: r = .18, p =.02; CVS hands-on: r = .17, p =.03) but not with
reading or analogical reasoning. The pattern of pairwise correlations provides evidence
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 13

Table 3. Pairwise correlations between scales.

2 3 4 5 Knowledge gain
1 Reading speed .55*** .17* .26* .15* .11
2 Reading comprehension .17* .41*** .27* .14
3 Analogical reasoning .42*** .30** .10
4 CVS understanding .35*** .18*
5 CVS hands-on .17*
Note: * p < .05.
**p < .01.
***p < .001.

that students’ learning of content knowledge in our guided inquiry tasks is linked to scien-
tific reasoning skills and not to cognitive skills. However, pairwise correlations do not con-
sider the effects of multiple covariates on the dependent variable and thus further analysis
is necessary. To this end, we compared three structural equation models (Figure 2).
An examination of the fit indices (Table 4) yields an insufficient fit of the Analogical
reasoning and reading model to the data. The CVS model has a good fit to the CFI, but
an insufficient fit with respect to the TLI and the RMSEA. In contrast, the Analogical
reasoning, reading, and CVS model show a good fit to the data according to all indices.
Figure 3 shows the path diagram for the Analogical reasoning, reading, and CVS
model. We represent only the significant paths in Figure 3, although all possible direct
and indirect effects (see Figure 2) of reading abilities, analogical reasoning, CVS under-
standing, and CVS hands-on skills on content knowledge gain were analysed in the model.
Our first research question concerned whether physics content knowledge gains result
from general cognitive skills or scientific reasoning skills. Two results indicate that analogical
reasoning and reading skills are not sufficient for successfully learning content in inquiry-
based learning environments. First, the Analogical reasoning and reading model did not
adequately fit the data structure (Table 4). Second, the results of the analogical reasoning,
reading, and CVS model (Figure 3) show that analogical reasoning and reading ability
have no direct effect on content knowledge gains (indicated by no arrows from analogical
reasoning and reading ability to content knowledge gains in Figure 3). We only found a
medium-sized direct effect of these variables on CVS understanding (Analogical reasoning:
g = .35, p , .001; Reading ability g = . 36, p , .001). Furthermore, analogical
reasoning had a weak direct effect on CVS hands-on skills (g = .19, p = .04).

Table 4. Global fit indices of SEM models.

Value
Analogical reasoning Analogical reasoning,
Indices Criterion CVS Model & reading model reading, & CVS model
χ 2 (df, p) p >.05 10.45 (6, .10) 6.54 (2, .04) 26.53 (19, .12)
CFI CFI ≥ .95 .97 .94 .97
TLI TLI ≥ .95 .93 .80 .95
RMSEA RMSEA < .06 .08 .11 .05
Note: CFI = Comparative Fit Index; TLI = Tucker–Lewis Index; RMSEA = Root Mean Square Error of Approximation. The CFI
and TLI are measures of goodness of fit, with higher values indicating greater congruence between the model and the
data (with values above .95 typically accepted as indicators of excellent fit, see Schreiber et al., 2006). The χ 2 and RMSEA,
in contrast, are measures of badness of fit, with higher values indicating worse fit between the model and the observed
data. The p-values associated with χ 2 statistics require an unduly restrictive assumption of multivariate normality and are
not robust when this assumption is violated. Further, these p-values are only asymptotically correct (e.g. are better
approximations of the true value as sample sizes increase). Schreiber et al. (2006) again provide criteria for evaluating
the RMSEA, suggesting values below .06 represent good fit.
14 A. STENDER ET AL.

Figure 3. Analogical reasoning, reading ability, and CVS model. The model is specified according to
model 3 in Figure 2 with all direct and indirect paths on content knowledge gains. Manifested variables
are in rectangles, and latent factors are in ellipses. Arrows that are missing compared to proposed model
in Figure 2 are non-significant paths (p > 0.05). The residual variance components (measurement error
variance is indicated by small arrows on latent variables) indicate the amount of unexplained variance.
Thus, for every observed Variable, R² = 1-error variance. The path coefficients associated with bold
arrows represent standardised estimates, whereas the coefficients listed next to light-face arrows rep-
resent standardised factor loadings. Levels of confidence are *p < .05; **p < .01; ***p < .001.

Our second research question addressed whether CVS hands-on skills are needed in
addition to CVS conceptual understanding for knowledge gains. CVS hands-on skill
had a direct effect on content knowledge gain (g = .22, p = .04). We found no direct
effect of conceptual understanding of CVS on content knowledge gain. However, we
did identify a mediation effect of CVS understanding via CVS hands-on skills on
content knowledge gain (CVS understanding –> CVS hands-on; g = . 26, p = .008
and CVS hands-on –> content knowledge gain; g = .22, p = .04).
To further illustrate the importance of CVS hands-on skills for learning content knowl-
edge we compared students who mastered CVS skills (conducting three or four out of four
controlled experiments, n = 121) to students who ran two or fewer controlled experiments
(n = 69). We rescaled logit values for the Rasch-model to the PISA-metric of a mean of 500
and SD of 100. The difference between CVS mastery students (M = 510; SD = 101) and
CVS non-mastery students (M = 477; SD = 95) was significant, t(189) = 1.96, p = .05.
Thus students whose facility with CVS enabled them to conduct controlled experiments
showed greater gains in content knowledge relative to students with less well-developed
hands-on CVS skills.

Discussion
In this study, we investigated which skills students need to achieve content knowl-
edge gains in an inquiry-based learning environment. We utilised structural
equation modelling in order to consider the covariance structure between the
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 15

assessed variables. We first discuss the effect of CVS skills versus cognitive skills
(reading skills and analogical reasoning) on students’ content knowledge learning.
Second, we discuss our results regarding the effect of CVS understanding (i.e. con-
ceptual) versus CVS hands-on skills (i.e. procedural) on content knowledge learn-
ing. Third, we address the importance of inquiry activities that offer students
guidance and support. Lastly, we consider the implications of our results for
science education.

Cognitive versus scientific reasoning skills

One aim of this study was to investigate to what extent student learning of content knowl-
edge in an inquiry-based learning environment depends on reading ability and analogical
reasoning skills or on CVS skills. The fit indices of the SEM indicate that the combined
Analogical reasoning, reading, and CVS model describes the relationships between stu-
dents’ skills and their content knowledge achievement better than the other assumed
models. This model shows that students’ gain in content knowledge depended solely on
their ability to design and interpret controlled experiments, as only the path from CVS
hands-on skills to content knowledge gain was significant. Analogical reasoning and
reading ability had only mediated effects via students’ conceptual understanding of CVS
and CVS hands-on skill on their gains in content knowledge. This result is in line with
previous research findings that show that cognitive skills such as reasoning or reading
skills have only mediation effects on inquiry skills (Osterhaus et al., 2017; van der Graaf
et al., 2016). Students with strong skills in reading and analogical reasoning are not
immediately able to gain content knowledge through inquiry; rather, they need a concep-
tual and procedural understanding of CVS to learn from inquiry activities. That is, cogni-
tive skills help students to develop CVS skills, which in turn support knowledge gains. This
finding provides evidence that students can indeed learn new content knowledge by using
inquiry skills to answer research questions.
A limitation of the current study was our focus on inquiry skills, reading skills, and
analogical reasoning. Previous studies have shown that variables such as students’
understanding of the nature of science (NoS), general reasoning skills, or advanced
theory of mind skills have additional effects on scientific reasoning skills (e.g. Oster-
haus et al., 2017). It is, therefore, an open question of how these variables influence
student learning of content knowledge from inquiry activities. In particular, NoS
understanding could have a direct effect on knowledge gain as it embraces students’
conceptions about scientific knowledge and how that knowledge is generated. These
conceptions might help students to realise that the function of inquiry is to find some-
thing out and to store this new knowledge in memory. An additional limitation of the
study is that due to time constraints and the age group selected, students engaged in a
simplified version of scientific inquiry. In an extended investigation, students may have
been able to, for example, make multiple measurements or further explore the nature
of the relationships among variables. Thus, in having students only make one measure-
ment across two conditions and draw inferences for that situation, we used a stripped
down version of what a teacher would be able to implement in a longer sequence of
classroom sessions.
16 A. STENDER ET AL.

Conceptual understanding of CVS versus CVS hands-on skills

Our second aim was to investigate whether a conceptual understanding of the control-of-
variables strategy (CVS) is sufficient for successful learning of content knowledge during a
guided inquiry or if students also need CVS hands-on skills. To answer this question, we
constructed a model with direct paths for both CVS understanding and CVS hands-on
skills on students’ content knowledge gain. The only significant direct path on students’
knowledge gain in the final SEM model was from CVS hands-on skills. CVS understand-
ing had only indirect effects on content knowledge gains via CVS hands-on skills. This
finding illustrates a potential production deficit. That is, students who have a conceptual
understanding of CVS are not automatically able to apply this knowledge when they
plan, run, and interpret the outcomes of experiments (e.g. Marschner et al., 2012). A con-
ceptual understanding of CVS is therefore necessary but not sufficient for gaining content
knowledge during guided inquiry. Instead, students need to use and practice their pro-
cedural CVS hands-on skills to generalise and transfer their conceptual understanding
of CVS and thus to achieve gains in content knowledge.

The role of guided inquiry

Our results show that students who understand CVS strategies and thus performed more
controlled experiments during the individual experimentation phase learned more physics
content knowledge. This finding is in line with the results of the meta-analysis by Lazonder
and Harmsen (2016) who found that providing students with heuristics or explanations
regarding scientific reasoning skills through structured or guided inquiry (i.e. levels 1
and 2 of Bell et al.’s levels of inquiry) can be a scaffold for students during inquiry. There-
fore, our results provide evidence that scaffolding students’ conceptual understanding of
CVS (as with our cognitive conflict procedure) and providing them with opportunities
to practice CVS hands-on skills can serve for students as a necessary guidance they
need for successful inquiry learning. In summary, we were able to show that learning of
science content through inquiry learning is indeed possible, as students not only con-
structed new knowledge through engagement with inquiry tasks but they did so by
using scientific reasoning skills. This is not surprising as learning by performing scientific
process skills forms the theoretical basis of inquiry learning, but it is not self-evident as
many studies on inquiry learning fail to show learning from those activities (Alfieri
et al., 2011; Minner et al., 2010). Like Lazonder and Harmsen (2016), we found that
inquiry-based learning becomes visible when quality measures of the learning process
of learning by inquiry (e.g. by utilising CVS hands-on skills) are investigated.

Implications for science teaching

Our findings have straightforward implications for science teaching. First, students can
acquire new content knowledge by utilising scientific reasoning skills during inquiry learn-
ing. Accordingly, inquiry learning can and should be used in science classes due to its
alignment with current aims of science education. The importance of inquiry learning
will increase further as science education becomes crucial for participation in modern
knowledge-based societies and economic systems (NRC, 2010).
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 17

Second, introducing students to both the conceptual and procedural aspects of the
control-of-variables strategy and other scientific process skills is a requirement for the
effective learning of content knowledge from guided inquiry. To master such skills stu-
dents need both an introduction to the concepts (e.g. through the cognitive conflict tech-
nique we employed) and training. The results of previous studies show that students can
be trained using paper-and-pencil, computer, or hands-on tasks as long as the training
addresses all relevant sub-skills (Klahr, Triona, & Williams, 2007; Schwichow et.al., 2016).
Finally, to construct effective inquiry-learning activities teachers have to align students’
current scientific reasoning skills and the demands of a particular learning activity. For
example, students could be trained on the required skills, and given the opportunity to
practices those skills. Additionally, teachers can adapt the demands of the inquiry activity
to students’ current skill level. The levels of inquiry outlined by Bell et al. (2005) provide
ways to alter teacher guidance and support during inquiry activities, such as providing
research questions or data. Unguided inquiry (level 3), in which students must perform
and coordinate all components of inquiry without scaffolding, requires practice and exper-
tise. Such activities should be the end and not the starting point of learning trajectories for
inquiry learning.

Disclosure statement
No potential conflict of interest was reported by the authors.

ORCID
Anita Stender http://orcid.org/0000-0001-6478-4762
Martin Schwichow http://orcid.org/0000-0001-9694-7183
Hendrik Härtig http://orcid.org/0000-0002-6171-9284

References
Abd-El-Khalick, F., BouJaoude, S., Duschl, R., Lederman, N. G., Mamlok-Naaman, R., Hofstein, A.,
… Tuan, H.-l. (2004). Inquiry in science education: International perspectives. Science
Education, 88(3), 397–419. doi:10.1002/sce.10118
Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011). Does discovery-based instruc-
tion enhance learning? Journal of Educational Psychology, 103(1), 1–18. https://doi.org/10.1037/
a0021017
Anderson, R. D. (2002). Reforming science teaching: What research says about inquiry. Journal of
Science Teacher Education, 13(1), 1–12. doi:10.1023/A:1015171124982
Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in
Cognitive Sciences, 4(11), 417–423.
Baxter, G. P., & Shavelson, R. J. (1994). Science performance assessments: Benchmarks and surro-
gates. International Journal of Educational Research, 21(3), 279–298. doi:10.1016/S0883-0355
(06)80020-0
Bell, R. L., Smetana, L. K., & Binns, I. C. (2005). Simplifying inquiry instruction. The Science
Teacher, 72(7), 30–33.
Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Dordrecht:
Springer.
Bullock, M., Sodian, B., & Koerber, S. (2009). Doing experiments and understanding science:
Development of scientific reasoning from childhood to adulthood. In W. Schneider & M.
18 A. STENDER ET AL.

Bullock (Eds.), Human development from early childhood to early adulthood. Findings from
the Munich longitudinal study (pp. 173–197). Mahwah, NJ: Lawrence Erlbaum.
Carolan, T. F., Hutchins, S. D., Wickens, C. D., & Cumming, J. M. (2014). Costs and benefits of
more learner freedom: Meta-analyses of exploratory and learner control training methods.
Human Factors, 56(5), 999–1014. doi:10.1177/0018720813517710
Chen, Z., & Klahr, D. (1999). All other things being equal: Acquisition and transfer of the control of
variables strategy. Child Development, 70(5), 1098–1120. doi:10.1111/1467-8624.00081
Chiappetta, E. L. (1997). Inquiry-based science: Strategies and techniques for encouraging inquiry
in the classroom. Science Teacher, 64(7), 22–26.
Croker, S., & Buchanan, H. (2011). Scientific reasoning in a real-world context: The effect of prior
belief and outcome on childrens hypothesis-testing strategies. British Journal of Developmental
Psychology, 29, 409–424.
Dewey, J. (1910). Science as subject-matter and as method. Science, 31(787), 121–127. doi:10.1126/
science.31.787.121
Dewey, J. (2002). Logik : Die theorie der forschung [logic: The theory of inquiry] (1st ed.). Suhrkamp:
Frankfurt am Main.
Embretson, S. E. (1991). A multidimensional latent trait model for measuring learning and change.
Psychometrika, 56(3), 495–515. doi:10.1007/BF02294487
Fischer, F., Kollar, I., Ufer, S., Sodian, B., Hussmann, H., Pekrun, R., … Eberle, J. (2014). Scientific
reasoning and argumentation: Advancing an interdisciplinary research agenda in education.
Frontline Learning Research, 28–45. doi:10.14786/flr.v2i3.96
Heller, K. A., & Perleth, C. (2000). Kognitiver fähigkeitstest für 4. Bis 12. Klassen [cognitive abilities
test of students from 4th to 12th grade]. Göttingen: Hogrefe.
Kaplan, D. (2005). Structural equation modeling: Foundations and extensions (Nachdr). Advanced
quantitative techniques in the social sciences: Vol. 10. Thousand Oaks, CA: Sage Publ.
Kiefer, T., Robitzsch, A., & Wu, M. (2017). TAM: Test Analysis Modules: R package version
1.99999-31. Retrieved from https://CRAN.R-project.org/package = TAM
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction
does not work: An analysis of the failure of constructivist, discovery, problem-based, experi-
ential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. doi:10.1207/
s15326985ep4102_1
Klahr, D. (2000). Exploring science: The cognition and development of discovery processes.
Cambridge, MA: MIT Press.
Klahr, D., Chen, Z., & Toth, E. (2001). From cognition to instruction to cognition: A case study in
elementary school science instruction. In K. D. Crowley, C. D. Schunn, & T. Okada (Eds.),
Designing for science. Implications from everyday, classroom, and professional settings
(pp. 209–250). Mahwah, NJ: Lawrence Erlbaum Associates.
Klahr, D., Triona, L. M., & Williams, C. (2007). Hands on what? The relative effectiveness of phys-
ical versus virtual materials in an engineering design project by middle school children. Journal
of Research in Science Teaching, 44(1), 183–203. doi:10.1002/tea.20152
KMK. (2005). Bildungsstandards im fach physik für den mittleren schulabschluss: Beschluss vom
16.12. 2004. München: Wolters Kluwer Deutschland GmbH.
Koerber, S., Mayer, D., Osterhaus, C., Schwippert, K., & Sodian, B. (2015). The development of
scientific thinking in elementary school: A comprehensive inventory. Child Development, 86
(1), 327–336. doi:10.1111/cdev.12298
Koslowski, B. (1996). Theory and evidence: The development of scientific reasoning (1st ed.). learn-
ing, development, and conceptual change. Cambridge, MA: MIT Pr.
Künsting, J., Thillmann, H., Wirth, J., Fischer, H. E., & Leutner, D. (2008). Strategisches experimen-
tieren im naturwissenschaftlichen unterricht. Psychologie in Erziehung und Unterricht, 55(1), 1–15.
Retrieved from http://www.ruhr-uni-bochum.de/lehrlernforschung/website_eng/pdf/kuensting_
et_al_2008.pdf
Lawson, A. E., & Wollman, W. T. (1976). Encouraging the transition from concrete to formal cog-
nitive functioning-an experiment. Journal of Research in Science Teaching, 13(5), 413–430.
doi:10.1002/tea.3660130505
 INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 19

Lazonder, A. W., & Harmsen, R. (2016). Meta-Analysis of inquiry-based learning: Effects of gui-
dance. Review of Educational Research, 86(3), 681–718. doi:10.3102/0034654315627366
Lincare, M. J. (2002). What do infit and outfit, mean-square and standardized mean? Rasch
Measurement Transactions, 16(2), 878.
Linn, M. C., Clement, C., & Pulos, S. (1983). Is it formal if it’s not physics? The influence of content
on formal reasoning). Journal of Research in Science Teaching, 20(8), 755–770. doi:10.1002/tea.
3660200806
Marschner, J., Thillmann, H., Wirth, J., & Leutner, D. (2012). Wie lässt sich die experimentierstra-
tegie-nutzung fördern? [How can the use of strategies for experimentation be fostered? – A com-
parison of differently designed prompts]. Zeitschrift für Erziehungswissenschaft, 15(1), 77–93.
doi:10.1007/s11618-012-0260-5
Mayer, D., Sodian, B., Koerber, S., & Schwippert, K. (2014). Scientific reasoning in elementary
school children: Assessment and relations with cognitive abilities. Learning and Instruction,
29, 43–55. doi:10.1016/j.learninstruc.2013.07.005
McElhaney, K. W. (2011). Making controlled experimentation more informative in inquiry inves-
tigations (Dissertation). University of California, Berkeley, CA.
Minner, D. D., Levy, A. J., & Century, J. (2010). Inquiry-based science instruction-what is it and
does it matter? Results from a research synthesis years 1984 to 2002. Journal of Research in
Science Teaching, 47(4), 474–496. doi.org/10.1002/tea.20347
Morris, B. J., Croker, S., Masnick, A., & Zimmerman, C. (2012). The emergence of scientific reason-
ing. In H. Kloos (Ed.), Current topics in children’s learning and cognition (pp. 61–82). InTech.
doi.org/10.5772/53885
National Research Council. (2010). Exploring the intersection of science education and 21st century
skills: A workshop summary. Washington, DC: National Academies Press.
National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting
concepts, and core ideas. Washington, DC: The National Academies.
Nehring, A., Nowak, K. H., Belzen, A. U. z., & Tiemann, R. (2015). Predicting students’ skills in the
context of scientific inquiry with cognitive, motivational, and sociodemographic variables.
International Journal of Science Education, 37(9), 1343–1363. doi:10.1080/09500693.2015.
1035358
NGSS Lead States. (2013). Next generation science standards: For states, by states. Washington, DC:
The National Academies Press.
Osborne, J. (2013). The 21st century challenge for science education: Assessing scientific reasoning.
Thinking Skills and Creativity, 10, 265–279. doi:10.1016/j.tsc.2013.07.006
Osterhaus, C., Koerber, S., & Sodian, B. (2017). Scientific thinking in elementary school: Children’s
social cognition and their epistemological understanding promote experimentation skills.
Developmental Psychology, 53(3), 450–462. doi:10.1037/dev0000260
Popper, K. R. (1966). Logik der forschung [The logic of scientific discovery]. Tübingen: J.C.B. Mohr.
Ross, J. A. (1988). Controlling variables: A meta-analysis of training studies. Review of Educational
Research, 58(4), 405–437. doi:10.3102/00346543058004405
Rosseel, Y. (2012). Lavaan: An R Package for Structural Equation Modeling. Journal of Statistical
Software, 48(2), 1–36.
Ruiz-Primo, M. A., & Shavelson, R. J. (1996). Rhetoric and reality in science performance assess-
ments: An update. Journal of Research in Science Teaching, 33(10), 1045–1063.
Schneider, W., Schlagmüller, M., & Ennemoser, M. (2007). LGVT 6-12 lesegeschwindigkeits- und
-verständnistest für die klassen 6-12 [reading speed and reading comprehension test for grade 6
to 12]. Göttingen: Hogrefe.
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation
modeling and confirmatory factor analysis results: A review. The Journal of Educational
Research, 99(6), 323–338. doi:10.3200/JOER.99.6.323-338
Schwichow, M., Croker, S., Zimmerman, C., Höffler, T., & Härtig, H. (2016). Teaching the control-
of-variables strategy: A meta-analysis. Developmental Review, 39(1), S. 37–S. 63. doi:10.1016/j.dr.
2015.12.001
20 A. STENDER ET AL.

Schwichow, M., Zimmerman, C., Croker, S., & Härtig, H. (2016). What students learn from hands-
on activities. Journal of Research in Science Teaching (JRST), 53(7), S. 980–S.1002. doi:10.1002/
tea.21320
Solomon, J. (1994). The rise and fall of constructivism. Studies in Science Education, 23(1), 1–19.
doi:10.1080/03057269408560027
van der Graaf, J., Segers, E., & Verhoeven, L. (2016). Scientific reasoning in kindergarten: Cognitive
factors in experimentation and evidence evaluation. Learning and Individual Differences, 49,
190–200. doi:10.1016/j.lindif.2016.06.006
Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory.
Psychometrika, 54(3), 427–450. doi:10.1007/BF02294627
Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle
school. Developmental Review, 27(2), 172–223. Retrieved from http://www.cogsci.ucsd.edu/~
deak/classes/EDS115/ZimmermanSciThinkDR07.pdf
Zimmerman, C., & Croker, S. (2013). Learning science through inquiry. In G. J. Feist, & M. E.
Gorman (Eds.), Handbook of the psychology of science (pp. 49–70). New York: Springer
Publishing Company.