Computers & Education 149 (2020) 103830

Contents lists available at ScienceDirect

Computers & Education

journal homepage: http://www.elsevier.com/locate/compedu

Students’ guided inquiry with simulation and its relation to school

science achievement and scientific literacy
Cai-Ting Wen a, Chen-Chung Liu b, Hsin-Yi Chang c, *, Chia-Jung Chang d,
Ming-Hua Chang b, Shih-Hsun Fan Chiang b, Chih-Wei Yang e, Fu-Kwun Hwang f
a
Graduate Institute of Network Learning Technology, National Central University, Taiwan
b
Department of Computer Science and Information Engineering, National Central University, Taiwan
c
Program of Learning Sciences, School of Learning Informatics, National Taiwan Normal University, Taiwan
d
Department of Information Communication, Yuan Ze University, Taiwan
e
Graduate Institute of Educational Information and Measurement, National Taichung University of Education, Taiwan
f
Department of Physics, National Taiwan Normal University, Taiwan

A R T I C L E I N F O A B S T R A C T

Keywords: The study investigates the effect of an interactive simulation with embedded inquiry support,
Applications in subject areas which was seamlessly embedded within the simulation, on students’ scientific literacy and school
Simulations science achievement, and explores the relationships among prior school science achievement,
Pedagogical issues
inquiry processes and scientific literacy. A total of 49 eighth-grade students at a public junior high
Improving classroom teaching
Secondary education
school in northern Taiwan participated. Data collected include the students’ pre- and post-scores
Inquiry for school science achievement, logging data that indicate their inquiry processes, and pretest,
Scientific literacy posttest, and delayed-test data that measure their scientific literacy. The results provide evidence
that the designed simulation and inquiry support had a long-term effect on the students’ scientific
literacy. Replacing conventional teaching with inquiry activities did not harm the students’
school science achievement performances. Moreover, compared to school science achievement,
students’ scientific literacy seems a better predictor of their inquiry behavior, especially in the
aspect of making conclusions. Analyses of the students’ inquiry processes indicate that the so-
called low science achieving students conducted more data analyses than the other students,
and demonstrated adequate inquiry engagement. The students with middle level school science
achievement demonstrated the most active engagement in inquiry and showed good gains of
scientific literacy after the learning. These results indicate that a guided inquiry learning envi­
ronment can support students with different levels of school science achievement to highly
engage in science inquiry. Implications and future studies are discussed.

1. Introduction

Science education standards worldwide stress the importance of engaging students in learning science through inquiry (e.g., NGSS
Lead States, 2013; Taiwan Ministry of Education, 2014). This learning approach is called inquiry learning, which situates learning in
investigations of complex problems or phenomena with an emphasis on learning activities that involve, but are not limited to, the
process of posing questions and investigating them with empirical data (Bell, Urhahne, Schanze, & Ploetzner, 2010; Hmelo-Silver,

* Corresponding author. Program of Learning Sciences, National Taiwan Normal University, #162, Sec.1, Heping E. Rd., Taipei City, 106, Taiwan.
E-mail address: hychang@ntnu.edu.tw (H.-Y. Chang).

https://doi.org/10.1016/j.compedu.2020.103830
Received 5 September 2019; Received in revised form 7 January 2020; Accepted 2 February 2020
Available online 4 February 2020
0360-1315/© 2020 Elsevier Ltd. All rights reserved.
C.-T. Wen et al. Computers & Education 149 (2020) 103830

Duncan, & Chinn, 2007).

Specifically, computer simulations can be incorporated into inquiry learning environments to enable students’ investigations.
Research has found positive impacts of student inquiry with simulations on students’ understanding of science concepts (Chiu,
DeJaegher, & Chao, 2015), development of inquiry skills (Efstathiou et al., 2018) or science epistemic beliefs (Huang, Ge, & Eseryel,
2017). However, few studies have investigated whether and how the use of simulations in science inquiry learning environments may
facilitate students’ scientific literacy, given that developing students’ scientific literacy has been recently emphasized in science ed­
ucation standards globally and locally (NGSS Lead States, 2013; OECD, 2016; Taiwan Ministry of Education, 2014).
Moreover, a practical concern involves a common reluctance of science teachers to engage students in inquiry learning because
many teachers may concern that implementing inquiry learning activities may result in a decrease in students’ school science
achievement scores. However, we found little research that was able to address this practical concern since studies often developed
their own instruments to measure students’ conceptual understanding of science (e.g., Chang, 2017; Chiu et al., 2015; Thacker &
Sinatra, 2019) rather than using school science achievement scores. To address the issues that have not been investigated much in past
studies, in this study, we investigated the effect of the guided inquiry with simulations environment on learners’ scientific literacy and
school science achievements.
Another central issue remains regarding how to design and guide student inquiry with simulation to best benefit learning (Don­
nelly, Linn, & Ludvigsen, 2014; Scalise et al., 2011; Rutten, van Joolingen, & van der Veen, 2012). For example, students may have
difficulties conducting mindful and purposeful inquiry with simulations, given the openness of an interactive simulation environment
(McElhaney & Linn, 2011). In this study, a new interactive simulation focusing on the concept of buoyancy has been developed via the
CoSci platform (http://cosci.tw/) (Wen et al., 2018) and was used in this study. Learners’ inquiry with the simulation is guided
through students’ self-generated inquiry maps. This guidance employs an innovative approach to support students’ inquiry by
providing elements of inquiry and allowing students to create their own inquiry map (detailed in the methods section). We examine the
effectiveness of the design in this study.
Despite the existence of the studies documenting successful interventions of science inquiry with simulations and their effects on
learning outcomes, there is also still a need for studies investigating students’ inquiry processes and linking these processes to their pre-
and post-performances. For example, Gal, Uzan, Belford, Karabinos, and Yaron (2015) analyzed log files collected from an open-ended
virtual laboratory environment to make sense of students’ actions during inquiry. Their purpose was to develop an automatic approach
to analyzing students’ problem-solving processes that can be used in the future to develop technology for adaptive feedback. With this
focus the study only investigated the learning process. Issues still remain such as what kinds of inquiry processes are more or less
productive in terms of leading to a more or less successful inquiry learning outcome. As initial steps to address such issues, in this study
we linked the students’ inquiry learning processes to their inquiry learning outcomes and characterized the students based on their
learning processes and performances. Such work can provide insight into the development of science inquiry learning environments
that address the needs of different students.
The research questions addressed include: (1) What is the quality of the students’ inquiry with the simulation? (2) What is the effect
of the guided inquiry with the interactive simulation on students’ scientific literacy and school science achievement? (3) What are the
relationships, if any, among the students’ prior school science achievement, their inquiry process with the interactive simulation, and
their developed scientific literacy? (4) How can the students’ performance be characterized according to their school science
achievement, quality of inquiry process and scientific literacy? The present study aimed to provide insight into the question of how to
augment effects of inquiry with simulation learning environments, such as effective designs of inquiry with interactive simulations
adapted for students with different science achievement levels to promote scientific literacy. It is hoped that the results of the study
contribute to theory and practice in that they address the theoretical perspectives and practical concerns regarding whether and how
inquiry learning with simulations has impact on students’ scientific literacy and school science achievement.

1.1. Background

1.1.1. Guided science inquiry with interactive simulation

Simulations refer to computer programs that simulate scientific phenomena through systems of representations or models.
Interactive features are usually programmed so that students can interact with the simulation to conduct virtual experiments. Virtual
experiments involve the use of technologies to engage students in “What-If” explorations where the outcomes of the experiments can be
immediately accessed through the use of a simulation (Hennessy et al., 2007). Virtual experiments are particularly beneficial for
supporting students’ explorations on phenomena that cannot be easily observed or investigated in real-life situations.
Reviews on the use of computer simulations to support learning have indicated overall positive effects of computer simulations on
learning outcomes (e.g., Donnelly et al., 2014; McElhaney, Chang, Chiu, & Linn, 2015; Rutten et al., 2012; Scalise et al., 2011). For
example, Scalise et al. (2011) reviewed 79 articles reporting on the learning effects of using computer simulations to support science
learning, and found that 53.2% of the studies reported gains and 17.7% reported gains under the right conditions, such as effective
interfaces that consider students’ cognitive load, or inquiry supports that guide students’ learning with the simulation, whereas 25.3%
had mixed results, and only 3.8% showed no gains. It can be evidenced that computer simulations often support science learning, but a
central issue remains, that is, how to design and guide student inquiry with simulation to best benefit learning.
Blanchard et al. (2010) categorized inquiry instruction into four levels: Level 0 refers to verification laboratory instruction in which
the teacher provides students with the question and methods of investigation, and guides them toward an expected conclusion; Level 1
refers to structured inquiry in which students are provided with a question and a method but are responsible for interpreting the result;
Level 2 refers to guided inquiry in which students design their own method of investigation and generate interpretation of the results;

2
C.-T. Wen et al. Computers & Education 149 (2020) 103830

Level 3 refers to open inquiry in which students generate a question and take responsibility for all major aspects of the investigation.
Studies have shown that the verification or cookbook-style laboratory instruction has an unfavorable influence and the least effect on
student learning (Blanchard et al., 2010; Scalise et al., 2011). In contrast, the results from the study by Adams, Paulson, and Wieman
(2009) suggest that students are more engaged and learn better with minimal but nonzero guidance as they conduct inquiry with
simulations. Similarly, one study found that the participants in the guided inquiry with simulation condition conducted more sys­
tematic and comprehensive investigations and reported a lower level of cognitive load than the participants in the unguided inquiry
with simulation condition (Moon & Brockway, 2019). In light of the relative effect of structured and guided inquiry with simulation,
Chang (2017) found that structured inquiry supported students in conducting more virtual experiments with simulations, but such
student behavior did not lead to better conceptual understanding of the science concepts targeted in the simulation, given that both the
students in the guided and structured inquiry conditions developed equally good conceptual understanding. As a result, the guided
inquiry approach led to better learning efficiency than the structured inquiry approach did, because the guided inquiry group con­
ducted fewer experiments but gained comparable conceptual understanding of the science concepts. Donnelly et al. (2014) reviewed
30 effective science inquiry learning environments with visualization or simulation, and found that the majority of the environments
employed guided inquiry. Overall, consistent evidence indicates that guided inquiry with simulations can benefit student learning.
However, there are variations of guided inquiry and their effects need further differentiation and investigation.
The guided inquiry approach is also employed in the current study in which elements of inquiry are provided for students to create
their inquiry map which guides students’ inquiry process (detailed in the methods section). The design of such guidance is consistent
with design principles suggested for inquiry learning environments (Donnelly et al., 2014; Quintana et al., 2004; Scalise et al., 2011).
Salient components of inquiry are revealed via the inquiry map, and hints or prompts are provided in each component to support
students’ thinking and inquiry process. Multiple paths are allowed so that students can set and test their own hypotheses with virtual
experiments. We examine the effectiveness of such a design in the current study.

1.1.2. Learning outcomes of inquiry with simulation

Well-designed science inquiry with simulation learning environments can be effective in promoting at least three types of learning
outcomes, as research has provided evidence. First, guided interaction with simulations can promote students’ conceptual under­
standing of the content targeted in the simulation (e.g., Chang, 2017; Chiu et al., 2015; Thacker & Sinatra, 2019). For example, Chiu
et al. (2015) engaged 45 eighth-grade students in guided science inquiry with a molecular simulation. They found that the students
made significant gains in terms of their understanding of gas properties from the pretests to the posttests.
Second, guided inquiry with simulations can enhance students’ general inquiry skills (Efstathiou et al., 2018). In the study by
Efstathiou et al. (2018), the fifth-grade students used the Go-Lab simulation and computer-supported inquiry learning environment in
which they were guided to conduct virtual experiments to learn science concepts related to sinking, floating and relative density. The
Experiment Design Tool of the Go-Lab environment provides the experimental group students with feedback regarding their design of
virtual experiments. In contrast, the control group only used paper-based worksheets that supported the students’ design of virtual
experiments without feedback. The experimental group outperformed the control group on assessment measuring inquiry skills
including four aspects of designing an experiment: identifying variables, stating hypotheses, operationally defining, and designing
investigations. Since the items of the assessment involve the four aspects of inquiry skills across various contexts and events, the
assessed inquiry skills indicate more domain-general than domain-specific inquiry skills. The results of the study provide evidence for
the use of guided inquiry with simulation environments to facilitate students’ general inquiry skills.
Third, one study found that engaging students in guided inquiry with simulations also enhanced the students’ science epistemic
beliefs (Huang et al., 2017). The students significantly improved in their beliefs about the source or certainty of science knowledge
after the intervention lasting 10 days with one 45-min session each day in which the students were guided to conduct an inquiry with
two PhET simulations about motion and force. The guidance was prompting questions to engage students in inquiry phases of
predict-observe-explain (White & Gunstone, 1989). The study provided evidence that students’ fundamental beliefs about science can
also be changed and developed during guided inquiry with computer simulations.
In the current study we focus on scientific literacy, which has received relatively little attention as a learning outcome in research
on guided inquiry with simulations. Developing future citizens’ scientific literacy involves the goal of educating young people to
become critical users of scientific knowledge, including developing their ability to explain phenomena scientifically, evaluate and
design scientific inquiry, and interpret data and evidence scientifically (OECD, 2016). This goal has been emphasized in science ed­
ucation standards in Taiwan (Taiwan Ministry of Education, 2014) and globally (e.g., NGSS Lead States, 2013). Compared to general
inquiry skills often measured in studies, assessment of scientific literacy requires students to apply science knowledge or concepts and
may need to take advantage of multiple formats of assessment such as multiple-choice and constructed-response items. Therefore, in
this study we developed assessment of scientific literacy in the case of science concepts relating to buoyancy. By examining the guided
inquiry activities the simulation environment provides, we hypothesize that the intervention can enhance the students’ scientific
literacy in the case of buoyancy in five aspects: (1) identifying the question explored in a given scientific study, (2) offering explanatory
hypotheses, (3) interpreting data and drawing appropriate conclusions, (4) evaluating ways of exploring a given question scientifically,
and (5) proposing a way of exploring a given question scientifically (OECD, 2016). Moreover, we further examined the students’
inquiry processes and related them to their scientific literacy performance. Such links between inquiry processes and outcomes reveal
patterns of students’ inquiry that lead to productive or less productive learning outcomes, and provide insight into designing adaptive
learning activities for students with different characteristics.
However, researchers have noticed that it is challenging to use specific innovations such as inquiry-based instruction to raise scores
on external achievement tests such as some high-stakes achievement tests or school achievement science tests, probably because of the

3
C.-T. Wen et al. Computers & Education 149 (2020) 103830

way the achievement tests are constructed (Hickey & Zuiker, 2012). Many of the science achievement tests at schools in Taiwan consist
of only multiple-choice items, cover a wide range of science concepts, and do not focus on the critical thinking or reasoning that
scientific literacy emphasizes. Therefore, researchers suggest using such achievement tests with caution since they may be faulty
indicators of how much individuals have learned from an intervention (Hickey & Zuiker, 2012). In this regard, we did not expect that
the inquiry activity in the treatment of this study would better increase students’ school science achievement scores after the inter­
vention than did the traditional instruction. In fact, many science teachers in Taiwan think that implementing inquiry learning ac­
tivities may result in a decrease in students’ achievement scores because traditional science instruction is usually specifically tailored
for helping students perform well on achievement tests whereas inquiry instruction is not. However, we think that this perspective may
be a misconception because research has shown that inquiry can foster students’ conceptual understanding of science concepts (e.g.,
Chang, 2017; Chiu et al., 2015; Thacker & Sinatra, 2019). Therefore, in this study we compared the impact of the treatment which used
an inquiry approach versus the control which used the traditional approach on students’ school science achievement scores after the
unit. Our hypothesis was that the inquiry treatment would not harm the students’ performance on the achievement tests. We also
collected the students’ school science achievement scores before the intervention to help characterize the students’ school science
achievements and to relate them to their inquiry processes. Doing so would provide insight into understanding how students with
different prior school science achievements may perform and learn during guided inquiry with simulation to spur discussion and
reflection on designing adaptive activities for different students.

2. Methods

We employed a quasi-experimental research design to investigate the impact of the guided inquiry with simulation on students’
scientific literacy and school science achievement. We also employed content analysis of students’ written responses and logging data
to examine the students’ inquiry processes. Both the impact of inquiry with simulation on scientific literacy and school science
achievement and the learning processes of inquiry with simulation in relation to the learning outcomes have been little addressed in
previous studies.

2.1. Participants

The participants in this study were 49 eighth-grade students (26 female, 23 male) in two classes taught by the same science teacher
at a public junior high school in northern Taiwan. One of the classes was randomly assigned as the treatment group (N ¼ 24) who used
the guided inquiry environment to interact with a simulation relating to the phenomenon of sinking and floating to learn science
concepts relating to buoyancy. The treatment group conducted inquiry with the simulation in a computer laboratory, and each of the

Fig. 1. The buoyancy simulation.

4
C.-T. Wen et al. Computers & Education 149 (2020) 103830

students worked on one desktop computer while they were encouraged to discuss with their peers and the teacher.
In contrast, the other class was assigned as the control group (N ¼ 25) which received conventional instruction on buoyancy in a
regular classroom. The teacher followed the textbook and delivered lectures on concepts relating to buoyancy, and had the students
practice test problems. Both the treatment and control groups spent three class periods (45 min per period) learning the concepts
relating to buoyancy. The design of the research comparison was driven by a practical issue about the impact of inquiry versus
traditional instruction. The inclusion of the control group is only to provide the baseline information regarding how the learning
outcomes of the treatment group compared to the learning outcomes of students receiving conventional instruction. No students in the
treatment or control group had learned scientific concepts relating to buoyancy before this study. None of them had prior experience
with the guided inquiry learning environment used in this study. Moreover, their school science achievements as measured by the
school science mid-term examination prior to this study indicate no significant difference between the two groups in terms of their
school science achievements prior to this study (t ¼ 0.007, p ¼ .995). Neither did the two groups differ in terms of their prior scientific
literacy as measured by the pretests (t ¼ 1.44, p ¼ .156).
The science teacher has 7 years’ science teaching experience. She received her bachelor’s degree in science and her master’s degree
in educational technology. She has rich experience in using technology to support her teaching, and has experience in guiding stu­
dents’ learning with web-based inquiry science environments. She is also very confident and comfortable employing conventional
instructional approaches such as textbook-based teaching since such approaches are still dominant in science classrooms and are
generally requested by parents in Taiwan. We designed the treatment group inquiry activities as follows.

2.2. Treatment using the guided inquiry with simulation learning environment

In the first class period, the science teacher explained to the treatment group that the inquiry task was to investigate why objects
sink or float on a liquid by interacting with and running the simulation (Fig. 1). Students could change values of three variables: the
density of the liquid, the object’s mass and the object’s volume, and could run the simulation to observe the status of the object in terms
of sinking into or floating on a liquid. The teacher also demonstrated how to use the functions provided in the learning environment
step by step.
In the next two class periods the students in the treatment group conducted their inquiry with the simulation by following the
guidance provided in the environment. An interface has been developed on the CoSci platform (http://cosci.tw/) for students to create
their own inquiry map that supports learners’ inquiry process (Wen et al., 2018, Fig. 2). The interface provides six inquiry phases for
students to create their own inquiry map that guides the inquiry process, including understanding the task, generating hypotheses,
designing experiments, collecting data, analyzing data and making conclusions. The details of each phase are described as follows.

Understand the task: The learning task was described during this phase. The students clicked the icon to enter the phase which
provided them with an explanation of the context of the simulation, and the goal of the learning task.
Generate hypotheses: The students clicked the “Generate hypotheses” icon to enter this phase in which they were asked to generate
hypotheses by selecting one of the provided hypotheses (such as: the heavier the object, the greater its buoyancy), or generating
their own hypotheses.
Design experiments: The students were prompted to design experiments by setting the values of the three variables in this phase
which would be used for conducting the experiment in the next phase. They were allowed to set the mass of the object, the volume
of the object and the density of the liquid to explore the relationship between those variables and the magnitude of the buoyancy
force, and if those relate to whether the object sinks or floats. They were also asked to explain their design ideas.
Collect data: The interactive simulation about buoyancy was provided in this phase. The students could conduct the experiment
based on the previous phase. The data were automatically recorded by the system and were shown in a dynamic data table. The
students could select the data that they needed from the data table in this phase.

Fig. 2. A screenshot of an inquiry map consisting of nodes created by a student.

5
 C.-T. Wen et al.
Table 1
The scoring rubric for the quality of the students’ inquiry process.
Inquiry Phase Criteria Scoring Rubric

Score 0 Score 1 Score 2

Design Design experiments A -Student designed Irrelevant description, blank, or not Description of the designs aligning with the inquiry
experiments experiments aligned with the selected inquiry aligned with the inquiry question question
question.
Design experiments B -Student designed Designs not based on the experiment Designs based on the experiment principle
experiments based on the experiment principle principle, namely, varying one variable at
6

one time
Collect and Trials conducting - Student conducted trials of Irrelevant description, blank, or trials not Correct description of what data were selected from
analyze data experiments aligned with the selected inquiry aligned with the inquiry question the trials that were used to address the inquiry
question. question
Data analysis - Student identified variables Irrelevant description or blank Identification of only some of the variables related Identification of all variables related to the
needed to be analyzed in order to address the to the inquiry question inquiry question
inquiry question.
Make Conclusions - Student made proper conclusions Irrelevant description or blank. Conclusion made consistent with the scientific Conclusion made consistent with the scientific
conclusions based on the evidence. principle of buoyancy principle of buoyancy and data used as
evidence

Computers & Education 149 (2020) 103830

C.-T. Wen et al. Computers & Education 149 (2020) 103830

Analyze data: In this phase, graphing and other tools were provided for the students to organize and analyze the data collected by
running the interactive simulation. For example, the data browsing tool allows students to see all the details of the data collected
from previous phases. The graphing tool supports them to generate the diagrams to represent relationships between variables. The
table generating tool enables them to select values of variables from the experiments to form a table.
Make conclusions: The students were asked to make conclusions based on the results of the data analysis phase to verify if the
hypothesis is correct or needs revision. The students were prompted to use the data as scientific evidence to support their
conclusions.

To facilitate the students’ awareness of the inquiry process, they were asked to go through each phase strictly. In other words, only
when one phase was complete were they allowed to create and access the next phase. The students could proceed by creating a node
from the previous phase. They could also generate multiple nodes to form more than one inquiry cycle. For example, they could reflect
on the result and create a new hypothesis phase node to form another inquiry cycle based on it. All the behavior during the inquiry
process was logged for us to investigate the quality of the students’ inquiry with the simulations.

2.3. Data collection and analysis

2.3.1. Pre-, post-, and delayed-tests to assess scientific literacy in the context of sinking and floating
All of the participants in the two groups took the scientific literacy test (45 min) before (the pretests) and after (the posttests) the
learning activity using the inquiry map. Five months later the two groups took the scientific literacy delayed test. The science teacher
confirmed that during the five months no other inquiry activities or review sessions on the concepts of buoyancy were implemented.
The pretests consisted of eight constructed-response items that asked students to make a claim and provide a reason for the claim.
The eight items were developed to measure scientific literacy in the context of applying scientific concepts relating to buoyancy, in the
four aspects that the inquiry learning environment particularly aims to facilitate, including (1) identifying the question explored in a
given scientific study, (2) offering explanatory hypotheses, (3) interpreting data and drawing appropriate conclusions, and (4)
evaluating ways of exploring a given question scientifically. The posttests consisted of three additional constructed-response items (a
total of 11 items, among which eight were identical to the pretests) that measured another important aspect of scientific literacy, which
is (5) proposing a way of exploring a given question scientifically. This aspect was only tested in the posttests and delayed posttests
since it involves an advanced level of scientific literacy that should be developed after students have experiences of conducting sci­
entific inquiry. The delayed posttests were very similar to the posttests. The only difference was that in the delayed posttests the three
additional items were revised from a pure constructed-response format to a two-tier multiple-choice format. In the two-tier multiple-
choice format, four choices of a claim were provided for students to select from, followed by four choices of the reason for the claim
also provided for students’ selection. We generated the choices by examining students’ responses in the posttests.
The tests went through several rounds of revision by two science educators and one assessment expert to establish the construct and
content validity. Detailed scoring rubrics were developed to code the students’ responses to the pre-, post- and delayed-tests. In
general, for the claim part, one point was given for an appropriate claim and zero for an inappropriate one. For the reason part, two
points were given for a completely adequate response, one point for a partially adequate response, and zero for an incomplete or
irrelevant one. Two independent raters coded 10 tests and the inter-rater reliability (Lombard, Snyder-Duch, & Bracken, 2002) reached
95%. Cohen’s Kappa is 0.91. Inconsistent codes were discussed and resolved.
Tests of normality indicated that the pre-, post-, and delayed-tests were all normally distributed (pretests: W ¼ 0.98, p ¼ .66;
posttests: W ¼ 0.98, p ¼ .69; delayed tests: W ¼ 0.96, p ¼ .07). The paired sample t-test was employed for the identical eight items of
both the pre- and post-tests to indicate whether there were significant gains in the scientific literacy from the pretests to the posttests.
Independent sample t tests were employed for the comparison between the experimental and control groups.

2.3.2. Students’ inquiry process logged via the learning environment

The students’ behavior to create inquiry maps and responses to the embedded prompting questions in the inquiry phases were
logged by the system and collected and analyzed in this study to indicate students’ inquiry processes. Such data were only collected for
the experimental group since the control group did not conduct inquiry. Detailed scoring rubrics were developed to code the students’
responses to the inquiry prompting questions embedded during the students’ inquiry with the simulation. Table 1 presents the overall
scoring rubric for coding the quality of the students’ inquiry process. The phase “Understand the task” was not coded since it only
provided contextual information for students and did not require them to write down their responses. The reliability of coding the
students’ quality of inquiry process is Cohen’s Kappa 0.88. In addition, the numbers of nodes generated by students in each phase were
summed and calculated to indicate their inquiry behavior.

2.3.3. Students’ pre- and post-scores for school science achievement

The students’ school science achievement scores as measured by the school science mid-term examinations prior to and after this
study were collected for both the experimental and control groups. The school science mid-term examinations consisted of only
multiple-choice items that mainly measured factual knowledge covering a wide range of science topics based on the science textbooks.
The school science mid-term examination after the study consisted of items measuring factual knowledge relating to concepts of
buoyancy.
To investigate the relationships among students’ prior school science achievement, their inquiry process during the learning ac­
tivity, and their scientific literacy developed after the learning, only the data from the experimental group were analyzed since the

7
C.-T. Wen et al. Computers & Education 149 (2020) 103830

control group did not conduct inquiry. Pearson correlation was employed. In addition, the k-means cluster analysis was performed for
examining how different types of students (in terms of their prior school science achievement and performance of inquiry process)
would gain different degrees of scientific literacy after their learning with the simulation.

3. Results

3.1. Students’ inquiry process with the simulation

To examine how the students engaged in the inquiry with the support of the environment, the numbers of nodes generated by
students in each phase were calculated. The results are listed in Table 2, which shows that most of them completed more than one
inquiry cycle. In other words, they were able to experience more than one inquiry by going through every phase in the learning
environment.
In order to investigate the students’ inquiry process in each node, we further analyzed their behavior and content logged inside the
nodes. The results are summarized in Tables 3 and 4. As shown in Table 3, on average each student generated more than one hypothesis
(M ¼ 1.71, SD ¼ 0.7), which suggests that the students tried to explore the simulation from multiple viewpoints. Note that the average
number of generated hypotheses is less than the average number of the hypothesis nodes they created. This is because some of the
students abandoned some nodes by keeping them blank, which caused the numbers of hypotheses generated to be less than the number
of hypothesis nodes generated. In the “Design Experiment” node, students can plan multiple experiments. On average, each student
planned eight experiments (M ¼ 8.00, SD ¼ 2.72) to test their hypotheses. In the “Collect Data” node, the students can run the
simulation as trials of their experiments. On average, each student conducted 19 trials (M ¼ 19.08, SD ¼ 9.44). These results indicate
the students’ behavior during the inquiry process, showing that overall the students were highly engaged behaviorally in the learning
environment. This also provides evidence for the effectiveness of the inquiry map approach supporting students in highly engaged
inquiry.
The quality of the students’ inquiry process is indicated by our analysis of the quality of the content in the nodes using the scoring
rubric in Table 1. Note that in Table 1, for example, the maximum score for the first aspect, experiment design quality A, is one. Since a
student may create multiple nodes, the actual maximum score each student obtained can be more than one. Table 4 lists the actual
maximum scores of the students’ data. It shows that, for example, the maximum score for the first aspect, experiment design quality A,
is two, which indicates that the best performance of the students was a student creating two Design Experiment nodes, and the content
of the nodes was rated 1 for each, and summed as 2 as a result. The score of each student in the five aspects was summed respectively
for each aspect, and the average of the summed scores gained by the students of the experiment group was calculated. The results are
summarized in Table 4. Most of the students designed experiments based on their selected inquiry question (M ¼ 1.00, SD ¼ 0.72),
were able to identify variables that were needed for analysis (M ¼ 1.46, SD ¼ 1.18) and made proper conclusions using the evidence
(M ¼ 1.00, SD ¼ 1.18). However, the results show that the students encountered difficulty in conducting consistent trials and selecting
data from the trials to address the inquiry question (M ¼ 0.08, SD ¼ 0.28). The results may suggest that the complexity of the real-time
data generated from the execution of the trials might hinder the students conducting inquiry.

3.2. Effects of the guided inquiry with the simulation learning environment on scientific literacy and school science achievement

The results of the paired sample t tests indicate that both the experimental and control groups made significant gains from the
pretests (eight items) to the posttests (considering only the eight items identical to the pretests) [Experimental group: t ¼ 5.64, p <
.001; Control group: t ¼ 4.01, p ¼ .001]. The results indicate that both the conventional textbook-based instruction and the guided
inquiry with simulation approach can facilitate students’ scientific literacy in the case related to the phenomenon of sinking and
floating.
The results of independent sample t tests further indicate that there is no significant difference between the treatment group and the
control group in the posttest scores of the scientific literacy assessment (t ¼ 0.689, p ¼ .494). However, the treatment group out­
performed the control group on the delayed-posttests (t ¼ 2.522, p ¼ .015) (Table 5). This suggests that the guided inquiry with
simulation learning environment is helpful for students to develop more permanent scientific literacy.
In addition to scientific literacy, we compared the two groups’ performance on their school science achievement scores after the
study. The result of the independent sample t-test indicates that there is no significant difference between the treatment group and the
control group in their school science achievements after the unit (t ¼ 0.177, p ¼ .860).

Table 2
The average numbers of nodes generated by the students in each phase.
M S.D. Max Min

Generate Hypotheses Node 1.83 0.70 3 1

Design Experiment Node 2.33 1.09 6 1
Collect Data Node 2.04 0.75 4 1
Analyze Data Node 2.25 1.03 5 1
Make Conclusions Node 1.46 0.66 3 0
Total 9.92 2.54 3 1

8
C.-T. Wen et al. Computers & Education 149 (2020) 103830

Table 3
Indicators of the students’ inquiry process inside the nodes.
M S.D. Max Min

Number of hypotheses generated 1.71 0.69 3 1

Number of experiments planned 8.00 2.72 14 3
Number of trials conducted 19.08 9.44 44 5

Table 4
The quality of the students’ inquiry process.
Aspects of the inquiry process Average S.D. Max Min

Experiment design quality A 1.00 0.72 2 0

Experiment design quality B 0.96 0.75 2 0
Trials conducting quality 0.08 0.28 1 0
Data analysis quality 1.46 1.18 4 0
Conclusions quality 1.00 1.18 4 0
Total 4.50 3.20 12 0

Table 5
Means and standard deviations of the posttest, delayed-test, and post achievement scores.
Scientific Literacy Posttest Scientific Literacy Delayed-test Post School Science Achievement Score

M S.D. M S.D. M S.D.

The treatment group 14.92 6.23 19.08 6.53 64.20 15.01

The control group 13.73 5.94 14.44 6.58 63.44 15.75
Independent sample t tests t ¼ 0.689, p ¼ .494 t ¼ 2.522, p ¼ .015 t ¼ 0.177, p ¼ .860

3.3. Relationships among prior school science achievement, process of inquiry, and scientific literacy

Table 6 shows the relationships among the students’ prior school science achievement, process of inquiry, and scientific literacy.
Overall, judged by the total scores of the inquiry quality, the students’ inquiry process was significantly related to their (both pre- and
post-instructional) scientific literacy and prior school science achievement. We further differentiated the results by examining in detail
the students’ performance on the phases or aspects of their inquiry process. The results show positive relations between the number of
nodes generated for making conclusions and both the pre- and the post-instructional scientific literacy. This suggests that students who
have higher prior scientific literacy would try to make more different conclusions, and would have better scientific literacy perfor­
mance after the activity. However, students’ prior school science achievement was not related to any of their node-generation process
or inquiry behavioral engagement. Compared to school science achievement, students’ scientific literacy seems a better predictor of
their inquiry behavior, especially in the aspect of making conclusions.
On the other hand, both prior school science achievement and scientific literacy were positively related to the quality of the
students’ inquiry process. Specifically, how well the students designed experiments based on the inquiry question and experiment

Table 6
Pearson bivariate correlation coefficients *<0.05; **<0.01.
Inquiry Process Phases or Aspects Pretest Scientific Posttest Scientific Prior school science
literacy literacy achievement

Number of nodes made Generate Hypotheses Node .023 .044 .098

Design Experiment Node -.139 -.065 .150
Collect Data Node -.175 -.052 .179
Analyze Data Node -.113 -.124 -.090
Make Conclusions Node .405* .435* .389

Behavioral engagement of Number of hypotheses .092 .102 .108

inquiry generated
Number of experiments planned .036 .235 .382
Number of trials conducted .151 .259 .378

Quality of inquiry Experiment design quality A .585** .563** .661**

Experiment design quality B .536** .491* .543**
Trials conducting quality .029 .031 -.207
Data analysis quality .364 .342 .321
Conclusions quality .762** .731** .624**
Total .675** .640** .606**

9
C.-T. Wen et al. Computers & Education 149 (2020) 103830

principle, and made adequate conclusions based on the data collected as evidence, were highly or moderately correlated with their
scientific literacy and school science achievement. In comparison, how well the students were able to conduct aligned trials and to
identify variables for analysis were not related to their scientific literacy or their school science achievement.

3.4. Characterizing students according to prior school science achievement, inquiry process and scientific literacy

We conducted cluster analysis to characterize the students. Since the number of participants is small, we used the cluster analysis
results for only exploratory purposes. We considered the results with descriptive statistics and their limitations. The results are pre­
sented in Table 7 and Figs. 3–5. Cluster 3 (M) consisted of five students at the middle level of prior school science achievement. They
highly engaged in the inquiry, especially for the aspects of planning experiments (10.99) and conducting trials (27.96 times) to collect
data. Their high engagement seemed to reflect on their improvement (the posttest scientific literacy scores minus the pretest scientific
literacy scores) of scientific literacy (a mean of 3.4 points), which is the greatest gain among the 4 clusters.
Compared to Cluster 3, Cluster 4 (H) consisted of eight students with higher level of prior school science achievement. These
students made more nodes, especially nodes of making conclusions (2.12). Their other behavioral engagement of inquiry was also
higher than the average, such as the number of hypotheses they generated (2.12) and the trials they conducted (21.16 times).
Meanwhile, they maintained a high quality inquiry process, especially for the aspects of experiment design (1.75 points) and data
analysis (2.75 points). Their scientific literacy also improved (2.5 points), and was only inferior to Cluster 3. There might be a ceiling
effect since the Cluster 4 students had already scored higher than the other students in the pretests.
On the other hand, Clusters 1 (L1) and 2 (L2) consisted of students with low level prior school science achievement but demon­
strated different types of engagement in the inquiry process. More specifically, the Cluster 1 students generated more nodes and
conducted more trials than the Cluster 2 students who on average focused on only one hypothesis. However, both groups of students
needed to improve the quality of their inquiry performance. This indicates that the Cluster 1 students might have struggled during the
inquiry process and used the strategy of trial and error repeatedly as they generated a high number of hypothesis nodes (2.20) but
demonstrated a low quality inquiry process. Also, neither cluster of students performed well on the scientific literacy assessment. It is
suggested that more adaptive guidance should be provided to these students who engaged differently in the guided inquiry
environment.
Fig. 3 shows the patterns of the four clusters in their node-generation process. It is encouraging to observe that the guided inquiry
with simulation learning environment enabled not only the high but also the low school science achievement students to engage in
inquiry by making the nodes. Cluster 2 (L2) could be characterized as motivated data analyzers since they created more data analysis
nodes than the others. We also observed that Cluster 1 (L1) demonstrated similar patterns to Cluster 4 (H) except that the high
achieving (H) students created more “Collect Data,” “Analyze Data” and “Make Conclusions” nodes.
Moving beyond the preliminary engagement in inquiry by making the nodes, the patterns of the four clusters in terms of their main
behavioral engagement of inquiry are presented in Fig. 4. It is apparent that Cluster 3 (M) students were active inquiry engagers since
they planned more experiments and conducted more trials. This result is also encouraging as it shows that the inquiry learning
environment can successfully engage students who might not score the highest in school science tests. We believe that such
engagement is important since it provides different opportunities for these students to get to know what science is about, which may in
turn help foster their motivation to learn and achieve in science. The Cluster 3 students gained the most in the scientific literacy
assessments in this study. In comparison, Cluster 4 (H) can be characterized as moderate inquiry engagers, and Cluster 1 (L1) as

Table 7
The cluster analysis results.
Characteristic Variable Mean Cluster 1 (L1) (n Cluster 2 (L2) (n Cluster 3 (M) (n Cluster 4 (H) (n ¼
¼ 5) ¼ 6) ¼ 5) 8)

Prior school science Prior mid-term examination 60.04 44.66 45.96 67.62 75.42
achievement score

Number of nodes made Generate Hypotheses Node 1.83 2.20 1.00 1.80 2.25
Design Experiment Node 2.33 2.59 1.50 2.20 2.88
Collect Data Node 2.04 1.80 1.50 2.80 2.12
Analyze Data Node 2.25 1.60 3.00 2.00 2.25
Make Conclusions Node 1.46 1.20 1.00 1.20 2.12

Behavioral engagement of Number of hypotheses 1.71 1.80 1.00 1.80 2.12

inquiry generated
Number of experiments 8.00 6.99 5.50 10.99 8.63
planned
Number of trials conducted 19.08 17.01 10.68 27.96 21.16

Quality of inquiry process Experiment design quality A 1.00 0.60 0.67 0.60 1.75
Experiment design quality B 0.96 0.60 0.67 0.40 1.75
Trials conducting quality 0.08 0.40 0.00 0.00 0.00
Data analysis quality 1.46 0.80 1.16 0.40 2.75
Conclusions quality 1.00 0.80 0.50 0.20 2.00

Scientific literacy Total Gain 1.92 0.60 1.00 3.40 2.50

10
C.-T. Wen et al. Computers & Education 149 (2020) 103830

Fig. 3. The four clusters in relation to the numbers of nodes made.

Fig. 4. The four clusters in relation to their behavioral engagement of inquiry.

Fig. 5. The four clusters in relation to their inquiry process quality.

average inquiry engagers.

The patterns of the students’ quality of inquiry process are presented in Fig. 5. Overall Cluster 4 (H) students maintained high
quality of their inquiry process. In comparison, the motivated data analyzers (L2), the average inquiry engagers (L1), and the active
inquiry engagers (M) need support to improve their quality of the four aspects of inquiry, namely, designing experiments aligned with
the selected inquiry question, designing experiments based on the experiment principle, identifying variables needed to be analyzed in

11
C.-T. Wen et al. Computers & Education 149 (2020) 103830

order to address the inquiry question, and making proper conclusions based on the evidence. The average inquiry engagers (L1)
demonstrated better quality of conducting trials of experiments aligned with the selected inquiry question, but judged by the scores
they received, all students need further support in this aspect.

4. Discussion and conclusions

The study involved a relatively small number of participants engaging in guided inquiry with an interactive simulation. The
generalizability of the results may be limited due to this small number and the context of the study. Nevertheless, we were able to
collect and analyze the data from the students’ inquiry process as well, aiming to provide a full picture of students’ inquiry. We discuss
the lessons learned, implications and future studies as follows.

4.1. Effects of the guided inquiry with simulation learning environment on scientific literacy and school science achievement

Both the experimental and control groups made significant gains from the scientific literacy pretests to the posttests. This is
reasonable given that both instructional approaches helped the students learn science concepts relating to sinking and floating, and
that proficiency in scientific literacy requires students to make critical use of science knowledge (OECD, 2016), and in many science
inquiry performances science knowledge plays a significant and central role (Chang, 2018).
In terms of the between group comparison, the two groups did not differ in the scientific literacy posttests. However, the exper­
imental group outperformed the control group in the scientific literacy delayed posttests. A learning curve predicted by Horton (2001)
(Fig. 6) indicates that students’ knowledge may decrease but proficiency of performance may increase some time after an intervention
takes place. Our findings further suggest that different learning curves exist for different instructional approaches. The guided inquiry
with simulation learning environment did a better job than the conventional science teaching at retaining the students’ scientific
literacy, which requires both application of science knowledge and performance of science inquiry.
Research has indicated that students’ inquiry learning with computer simulations can facilitate students’ understanding of science
concepts (Chiu et al., 2015), development of inquiry skills (Efstathiou et al., 2018) or science epistemic beliefs (Huang et al., 2017).
This current study is among the very few studies that have started to examine the effect of inquiry learning with computer simulations
on students’ development of scientific literacy, an aspect that has been emphasized in recent science education reforms (NGSS Lead
States, 2013; OECD, 2016; Taiwan Ministry of Education, 2014). The results of the study indicate that the guided inquiry with a
simulation learning environment has an effect on the students’ scientific literacy. Moreover, this effect has a long-term influence on
students’ scientific literacy.
In addition to scientific literacy, we compared the experimental and control groups’ performance on their school science
achievement scores after the study. This comparison addresses a practical concern that many science teachers may fear that imple­
menting inquiry learning activities may result in a decrease in students’ achievement scores. However, we think that this concern may
stem from a misconception, because research has shown that inquiry can foster students’ conceptual understanding of science concepts
(e.g., Chang, 2017; Chiu et al., 2015; Thacker & Sinatra, 2019). Nevertheless we found little research that was able to address this
practical concern since these studies did not use school science achievement scores but rather developed their own instruments to
measure students’ conceptual understanding of science. In the current study, we found that replacing lectures on concepts relating to
buoyancy with the buoyancy simulation did not harm the students’ school science achievement performances. Despite the fact that
high-stakes achievement tests or school achievement science tests may not be sensitive enough to indicate the benefits of some
innovative learning approaches such as the inquiry-based learning approach (Hickey & Zuiker, 2012), in the current study we used and
compared the different groups’ school science achievement scores to provide evidence that the inquiry-based learning approach has a
similar effect on aspects of science learning outcomes that are valued in school science achievement tests.

4.2. Productive inquiry process connecting to scientific literacy

Little research has linked students’ inquiry processes to their performances after inquiry. Only a few qualitative studies have re­
ported on cases of students’ learning trajectories during inquiry with simulations (e.g., Thacker & Sinatra, 2019). In our study, we
found that how well the students purposefully designed their experiments to address their inquiry question and adequately made
conclusions based on the evidence obtained from the inquiry were significantly correlated to their performance on the scientific lit­
eracy assessments. This result suggests that more attention or scaffolding is needed to support students’ mindful engagement in these
aspects of science inquiry, in order to develop scientific literacy that is important for the future citizens of the 21st century (NGSS Lead
States, 2013; Taiwan Ministry of Education, 2014). A recent study has focused on scaffolding students’ learning using an experiment
design tool to apply the ‘‘vary one thing at a time’’ [VOTAT] heuristic when designing their experiments (Efstathiou et al., 2018). The
experiment design tool has found it to be effective for fostering students’ domain-general inquiry skills. Based on our results, we further
suggest other areas of scaffolding, including students’ reflection on how well their designs address the inquiry question, and how well
their conclusions are supported by the evidence from the inquiry. One study found that their designed meta-conceptual scaffolding (e.
g., prompting students to reflect on and elaborate their thinking when they predicted and explained a simulation) enhanced the
students’ conceptual understanding of the concepts represented in the simulation (Huang et al., 2017). For future studies, it seems
equally important to design and incorporate meta-inquiry scaffolding and to investigate its impact on students’ inquiry processes and
outcomes.
In comparison, the students’ behavioral engagement of inquiry, namely, number of nodes created, hypotheses tested, experiments

12
C.-T. Wen et al. Computers & Education 149 (2020) 103830

Fig. 6. Predicted learning curve by Horton (2001).

planned, or trials conducted, was not significantly related to their scientific literacy or school science achievement performances.
"Chang(2017)" found that structured inquiry may increase students’ behavioral engagement of inquiry, but this behavioral engage­
ment may not necessarily lead to better learning outcomes. In our guided inquiry environment, the students’ variations of behavioral
engagement were allowed. In general, the students were highly behaviorally engaged, indicating that the learning environment was
able to engage students in conducting inquiry. Students’ behavioral engagement in inquiry can be a basic indicator of the success­
fulness of an inquiry environment. The students in this study, regardless of their prior school science achievement levels, were highly
engaged in the inquiry activities. This provides evidence for the effectiveness of the inquiry map support. Future studies are needed to
investigate how to take advantage of students’ behavioral engagement to further scaffold them to conduct quality inquiry with
interactive simulation.

4.3. Designing guided inquiry with simulation learning environments for high and low prior school science achievement students

Conventional, high-stakes achievement tests continue to dominate educational systems, while researchers have started to call for
alternative assessments that are sensitive to innovative science curricula (Hickey, Taasoobshirazi, & Cross, 2012; Hickey & Zuiker,
2012). In this study we demonstrated how using logging data of students’ inquiry combined with their prior and post performance in
science can provide evidence of the effectiveness of innovative science learning activities. Specifically in Taiwan, many school science
achievement tests such as mid-term examinations follow the standardized tests, consisting of only multiple-choice items covering a
wide range of science facts and drill-and-practice problems. Students’ motivation to learn science and their ideas about science may
suffer when too much emphasis is put on the achievement tests and too little attention is paid to innovative and authentic ways of doing
and learning science and to formatively assessing and revealing their impacts on student learning of science.
In this study, we explored the impact of the guided inquiry with simulation learning environment by characterizing the students
considering their prior school science achievements, inquiry processes, and gained scientific literacy. The results suggest that the so-
called low science achieving students can be motivated data analyzers or average inquiry engagers in an inquiry learning environment.
Moreover, the students with middle level school science achievement demonstrated the most active engagement in inquiry and showed
good gains in scientific literacy after the learning. These results provide evidence of how a guided inquiry with a simulation learning
environment can promote students with different school science achievement levels to learn science with different trajectories.
Specifically, high-achieving students were more able to engage in scientific inquiry focusing on designing experiments based on the
inquiry question and experiment principle and making conclusions based on the collected data as evidence. In comparison, low-
achieving students tended to practice their inquiry through generating more hypothesis nodes, conducting more trials, and creating
more data analysis nodes. These are good inquiry practices to start with, and future designs of interventions need to address how to
scaffold students’ inquiry learning from personal inquiry (e.g., trial-and-error or free exploration) to scientific inquiry (e.g., conducting
valid experiments to make adequate conclusions to address the inquiry question). Future studies are needed to address the common
difficulties in inquiry with simulations, including designing experiments and conducting trials aligned with the selected inquiry
question, designing experiments based on the experiment principle, identifying variables which need to be analyzed, making judg­
ments about what data are needed in order to address the inquiry question, and making proper conclusions based on the evidence.

Author contribution section

Cai-Ting Wen: Conceptualization, Methodology, Software, Formal analysis, Writing - original draft, Writing - review & editing,
Chen-Chung Liu: Conceptualization, Methodology, Formal analysis, Writing - original draft, Writing - review & editing, Hsin-Yi Chang:
Conceptualization, Methodology, Formal analysis, Investigation, Writing - original draft, Writing - review & editing, Chia-Jung Chang:
Methodology, Software, Validation, Ming-Hua Chang: Software, Validation. Shih-Hsun Fan Chiang: Software, Validation. Chih-Wei

13
C.-T. Wen et al. Computers & Education 149 (2020) 103830

Yang: Validation, Supervision. Fu-Kwun Hwang: Validation, Supervision.

Source of funding

This work was supported by Ministry of Science and Technology, Taiwan, under Grants MOST106-2511-S-008-012-MY3,
MOST107-2511-H-008-003-MY3, MOST107-2811-H-008-006, and MOST108-2628-H-003-001-MY3. It was also financially supported
by the Institute for Research Excellence in Learning Sciences of National Taiwan Normal University (NTNU) from the Featured Areas
Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan.

Declaration of competing interest

All authors of this manuscript declare that we have no conflict of interest.

References

Adams, W. K., Paulson, A., & Wieman, C. E. (2009). What levels of guidance promote engaged exploration with interactive simulations? In H. Charles, S. Mel, &
H. Leon (Eds.), Proceedings of AIP conference: Vol. 1064. 2008 physics education research conference (pp. 59–62). Edmonton, Alberta, Canada: American Institute of
Physics. https://doi.org/10.1063/1.3021273.
Bell, T., Urhahne, D., Schanze, S., & Ploetzner, R. (2010). Collaborative inquiry learning: Models, tools, and challenges. International Journal of Science Education, 32,
349–377. https://doi.org/10.1080/09500690802582241.
Blanchard, M. R., Southerland, S. A., Osborne, J. W., Sampson, V. D., Annetta, L. A., & Granger, E. M. (2010). Is inquiry possible in light of accountability? A
quantitative comparison of the relative effectiveness of guided inquiry and verification laboratory instruction. Science Education, 94, 577–616. https://doi.org/
10.1002/sce.20390.
Chang, H.-Y. (2017). How to augment the learning impact of computer simulations? The designs and effects of interactivity and scaffolding. Interactive Learning
Environments, 25, 1083–1097. https://doi.org/10.1080/10494820.2016.1250222.
Chang, H.-Y. (2018). Students’ representational competence with drawing technology across two domains of science. Science Education, 102, 1129–1149. https://doi.
org/10.1002/sce.21457.
Chiu, J. L., DeJaegher, C. J., & Chao, J. (2015). The effects of augmented virtual science laboratories on middle school students’ understanding of gas properties.
Computers & Education, 85, 59–73. https://doi.org/10.1016/j.compedu.2015.02.007.
Donnelly, D. F., Linn, M. C., & Ludvigsen, S. (2014). Impacts and characteristics of computer-based science inquiry learning environments for precollege students.
Review of Educational Research, 84, 572–608. https://doi.org/10.3102/0034654314546954.
Efstathiou, C., Hovardas, T., Xenofontos, N. A., Zacharia, Z. C., deJong, T., Anjewierden, A., et al. (2018). Providing guidance in virtual lab experimentation: The case
of an experiment design tool. Educational Technology Research & Development, 66, 767–791. https://doi.org/10.1007/s11423-018-9576-z.
Gal, Y.a., Uzan, O., Belford, R., Karabinos, M., & Yaron, D. (2015). Making sense of students’ actions in an open-ended virtual laboratory environment. Journal of
Chemical Education, 92, 610–616. https://doi.org/10.1021/ed500531a.
Hennessy, S., Wishart, J., Whitelock, D., Deaney, R., La Velle, L., McFarlane, A., … Winterbottom, M. (2007). Pedagogical approaches for technology-integrated
science teaching. Computers & Education, 48, 137–152. https://doi.org/10.1016/j.compedu.2006.02.004.
Hickey, D. T., Taasoobshirazi, G., & Cross, D. (2012). Assessment as learning: Enhancing discourse, understanding, and achievement in innovative science curricula.
Journal of Research in Science Teaching, 49, 1240–1270. https://doi.org/10.1002/tea.21056.
Hickey, D. T., & Zuiker, S. J. (2012). Multilevel assessment for discourse, understanding, and achievement. The Journal of the Learning Sciences, 21, 522–582. https://
doi.org/10.1080/10508406.2011.652320.
Hmelo-Silver, C., Duncan, R., & Chinn, C. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark
(2006). Educational Psychologist, 42, 99–107. https://doi.org/10.1080/00461520701263368.
Horton, W. (2001). Evaluating e-learning. Alexandria, VA: American Society for Training & Development.
Huang, K., Ge, X., & Eseryel, D. (2017). Metaconceptually-enhanced simulation-based inquiry: Effects on eighth grade students’ conceptual change and science
epistemic beliefs. Educational Technology Research & Development, 65, 75–100. https://doi.org/10.1007/s11423-016-9462-5.
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human
Communication Research, 28, 587–604. https://doi.org/10.1111/j.1468-2958.2002.tb00826.x.
McElhaney, K. W., Chang, H.-Y., Chiu, J. L., & Linn, M. C. (2015). Evidence for effective uses of dynamic visualizations in science curriculum materials. Studies in
Science Education, 51, 49–85. https://doi.org/10.1080/03057267.2014.984506.
McElhaney, K. W., & Linn, M. C. (2011). Investigations of a complex, realistic task: Intentional, unsystematic, and exhaustive experimenters. Journal of Research in
Science Teaching, 48(7), 745–770.
Ministry of Education (MOE). (2014). Curriculum outlines of the 12-year basic education curriculum-the master outline. Taipei: National Research Council Press.
Moon, J. A., & Brockway, D. (2019). Facilitating learning in an interactive science simulation: The effects of task segmentation guidance on adults’ inquiry-based
learning and cognitive load. Journal of Research on Technology in Education, 51, 77–100. https://doi.org/10.1080/15391523.2019.1566038.
NGSS Lead States. (2013). Next generation science standards: For states, by States. Washington, DC: The National Academies Press.
OECD. (2016). PISA 2015 Assessment and analytical framework: Science, reading, mathematic and financial literacy. Paris: OECD Publishing.
Quintana, C., Reiser, B. J., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., … Soloway, E. (2004). A scaffolding design framework for software to support science
inquiry. The Journal of the Learning Sciences, 13, 337–386. https://doi.org/10.1207/s15327809jls1303_4.
Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2012). The learning effects of computer simulations in science education. Computers & Education, 58, 136–153.
https://doi.org/10.1016/j.compedu.2011.07.017.
Scalise, K., Timms, M., Moorjani, A., Clark, L., Holtermann, K., & Irvin, P. S. (2011). Student learning in science simulations: Design features that promote learning
gains. Journal of Research in Science Teaching, 48, 1050–1078. https://doi.org/10.1002/tea.20437.
Thacker, I., & Sinatra, G. M. (2019). Visualizing the greenhouse effect: Restructuring mental models of climate change through a guidedonline simulation. Education
Sciences, 9, 14. https://doi.org/10.3390/educsci9010014.
Wen, C. T., Chang, C. J., Chang, M. H., Chiang, S. H. F., Liu, C. C., Hwang, F. K., & Tsai, C. C. (2018). The learning analytics of model-based learning facilitated by a
problem-solving simulation game. Instructional science, 46(6), 847–867.
White, R. J., & Gunstone, R. F. (1989). Metalearning and conceptual change. International Journal of Science Education, 11, 577–586. https://doi.org/10.1080/
0950069890110509.

14