Journal of Science Education and Technology (2018) 27:322–333

https://doi.org/10.1007/s10956-017-9726-9

Integrating Computational Science Tools into a Thermodynamics Course

Camilo Vieira 1 & Alejandra J. Magana 1 & R. Edwin García 2 & Aniruddha Jana 2 & Matthew Krafcik 2

Published online: 24 January 2018

# Springer Science+Business Media, LLC, part of Springer Nature 2018

Abstract
Computational tools and methods have permeated multiple science and engineering disciplines, because they enable scientists
and engineers to process large amounts of data, represent abstract phenomena, and to model and simulate complex concepts. In
order to prepare future engineers with the ability to use computational tools in the context of their disciplines, some universities
have started to integrate these tools within core courses. This paper evaluates the effect of introducing three computational
modules within a thermodynamics course on student disciplinary learning and self-beliefs about computation. The results suggest
that using worked examples paired to computer simulations to implement these modules have a positive effect on (1) student
disciplinary learning, (2) student perceived ability to do scientific computing, and (3) student perceived ability to do computer
programming. These effects were identified regardless of the students’ prior experiences with computer programming.

Keywords Computational science . Simulations . Thermodynamics . Self-beliefs . Conceptual change . Computational thinking

Introduction complex problems of a global society (ACM 2013; National

Computational science and engineering (CSE) has been
denominated the third pillar of science along with theoretical
and experimental approaches (PITAC 2005; Wofford 2009).
CSE integrates multiple disciplines with mathematical and
computational tools to enable scientists and engineers to solve
complex problems (Shiflet and Shiflet 2006; Weintrop et al.
2016). The body of knowledge of the ACM Computer
Science (CS) curricula in 2013 defined CSE as Bthe applica-
tion of computer science to solve problems across a range of
disciplines^ (The Joint Task Force on Computing Curricula—
ACM/IEEE-Computer Society 2013, p. 68). CSE involves
several techniques such as modeling, simulation, visualiza-
tion, data processing, and numerical analysis.
Several reports and agencies highlight that engineering cur-
ricula need to keep the pace with advances in computational
science (Wofford 2009). Engineering graduates need to be
prepared with the CSE knowledge and skills to address the
* Camilo Vieira
cvieira@purdue.edu
1
Department of Computer and Information Technology, Purdue
University, 401 N. Grant Street, West Lafayette, IN 47907, USA
2
School of Materials Engineering, Purdue University, Neil Armstrong
Hall of Engineering, 701 West Stadium Avenue, West
Lafayette, IN 47907-2045, USA
Science Foundation 2011; PITAC 2005; The Minerals,
Metals, and Materials Society 2013). While graduate students
in engineering value the application of computational tools
and methods in their disciplines, they usually feel underpre-
pared to use/develop those (Magana and Mathur 2012).
To address this issue and at the same time integrate CSE into
undergraduate programs, several alternatives have been sug-
gested (Turner et al. 2002): creating concentrations in CSE,
creating multidisciplinary project-based courses, enhancing
existing courses with the use of CSE tools and methods, and
using computer visualizations as a vehicle for teaching and
learning disciplinary knowledge. Hence, some universities
and colleges have started to integrate CSE tools and methods
into their disciplinary engineering curricula (Magana et al.
2013; Alabi et al. 2015), science courses (Wofford 2009), and
liberal arts environments (Turbak and Berg 2002). For instance,
the Department of Materials Science and Engineering at a mid-
Atlantic coast university created a curricular innovation that
involved one computation and programming course for mate-
rials scientists and twelve computational modules integrated
into six core courses (Magana et al. 2013; Magana et al.
2016). Similarly, the materials engineering department at
University of Illinois at Urbana-Champaign has integrated
computational modules throughout their undergraduate curric-
ulum (Mansbach et al. 2016).
This paper describes the implementation of three computa-
tional modules as part of a Thermodynamics of Materials
J Sci Educ Technol (2018) 27:322–333
course at a large Midwestern university. Each module
consisted of a 50-min lab session in which students used the
Virtual Kinetics of Materials Laboratory (VKML), hosted in
nanoHUB.org (https://nanohub.org/tools/vkmllive). VKML is
based on Gibbs (Cool et al. 2010), a set of Python-based
libraries that enables students to represent complex thermody-
namics phenomena using modeling techniques and simple
python programming. In order to assess this micro curricular
innovation, the research team explored the following research
questions (RQ):
RQ1: What is the effect of introducing computational
modules as part of a thermodynamics course on student
disciplinary learning?
RQ2: What is the effect of introducing computational
modules as part of a thermodynamics course on student
self-beliefs about computation?
RQ3: What are the factors that affect changes on student
understanding of disciplinary phenomena after being ex-
posed to computational modules as part of a thermody-
namics course?
RQ4: What are the factors that affect changes on student
self-beliefs about computation after being exposed to
computational modules as part of a thermodynamics
course?
Thermodynamics Teaching and Learning
Thermodynamics is an important course for several disci-
plines in engineering and the sciences (Probst and Zhang
2013), including mechanical, chemical, and materials engi-
neering, as well as physics and chemistry. Yet, thermodynam-
ics is a complex area that students struggle to understand and
visualize (Mulop et al. 2012), and at the same time, instructors
usually struggle to find effective ways to teach it (Cobourn
and Lindauer 1994; Probst and Zhang 2013). Some of the
common misunderstandings and difficult concepts in thermo-
dynamics include (Mulop et al. 2012; Olds et al. 2004) work,
heat, enthalpy, entropy, and equilibrium, as well as applying
these concepts for problem-solving and visualizing these ab-
stract phenomena.
Researchers advocate for constructivist learning environ-
ments to meaningfully engage students in the learning process
of thermodynamics (Mulop et al. 2012). Along these lines,
computer simulations have been shown to be an effective
approach to support this complex teaching and learning pro-
cess (Baher 1999; Cobourn and Lindauer 1994; Yang et al.
2012). Some of the affordances of these simulations in ther-
modynamics are (Baher 1999; Mulop et al. 2012; Khan 2011)
(1) visualizing abstract concepts, (2) identifying relationships
among variables, (3) supporting monitoring learning
323
activities, (4) scaffolding detailed mathematical activities,
and (5) applying the course concepts into real life situations.
However, when these simulations involve computer program-
ming, students face challenges in mapping the conceptual
model, the mathematical representation, and the algorithmic
representation (Vieira et al. 2017), and therefore, they may
benefit from additional support.
This study explores the introduction of Gibbs, a Python-
based computer simulation software, into a course of thermo-
dynamics of materials. During the implementation of three
computational modules, students had access to the Python
code that created the visualizations and simulations. In order
to reduce the cognitive load from these activities, students
were provided with a worked example that was introduced
by the instructor. The meaningful exploration of these exam-
ples was encouraged through self-explanation activities within
the homework assignments.
Constructivist Approaches to Conceptual
Change
Conceptual change is the theoretical framework that guided
our investigation. Conceptual change has been broadly re-
ferred to as the process by which the individuals’ conceptual
frameworks are replaced by another set of beliefs which are
incompatible with the first (Posner et al. 1982). Researchers
have explained conceptual change as a the process of acquir-
ing new knowledge based on models of philosophy of science
(Posner et al. 1982), as ontological shifts (Chi et al. 2012), or
as processes for restructuring the learners’ preinstructional
conceptual structures in order to allow understanding of sci-
ence concepts (Duit and Treagust 2003). Although there is still
not a common agreement of the cognitive processes inherent
to conceptual change, there seems to be an agreement that
constructivist approaches to science learning may result in
conceptual change (Duit 1999).
Conceptual change-based strategies focus on externalizing
an existing the learner’s knowledge structure and then modi-
fying it via instruction. Instructional elements for conceptual
change include helping students to identify or externalize their
own misconceptions, fragmented or incomplete conceptions;
then provide instruction where students have an opportunity to
apply an accurate scientific conception; and finally reinforce
the accurate conceptions via repetition (Anderson and Smith
1987).
Computer simulations have been used as a constructivist
approach for conceptual understanding in science (Khan
2011; Windschitl 1995; Wofford 2009). The use of computer
simulations with proper instruction can facilitate the four con-
ditions to promote conceptual change: (1) dissatisfaction with
existing concepts, (2) the new concept should be explainable,
(3) the new concept should be intelligible, and (4) the new
324
concept should be able to be extended (Posner et al. 1982). We
argue that in the same way computer simulations can promote
conceptual change in science, they can also promote concep-
tual change in engineering education.
Methods
Participants
Students enrolled in a sophomore thermodynamics of mate-
rials course (herein called THERMO) were the participants for
this study. During the spring semester 2015, 62 students par-
ticipated in this study, while in the spring 2016 semester, 46
students participated too. At the beginning of the semester, the
participants were asked about the number of programming
courses they had taken before (either high school or college
level). Twenty-two students had not taken any programming
courses before, 37 students had taken one course, 33 students
had taken two courses, and 16 students had taken three or
more courses. Eighty-eight students mentioned to be familiar
with at least one programming language, and 30 of these stu-
dents reported to be familiar with Python. This study was
approved by the institutional review board (IRB) at the partic-
ipating institutions.
Procedures and Data Collection
Figure 1 summarizes the research procedures along with the
data collection instruments for this study. The research process
started with a baseline survey that prompted students to report
their self-beliefs regarding their ability to write computer pro-
grams. The survey on self-beliefs was adapted from Magana
et al. (2016) as described in Table 1. The original survey only
included the following constructs: (1) control appraisals, de-
fined as the students’ Blevel of control of the outcome^ (p.
430); and (2) value appraisals, defined as the Bperceived im-
portance of achieving that outcome^ (p. 430). The survey for
Fig. 1 Intervention in the
thermodynamics course. 1
Baseline survey on self-beliefs, 2
three computational modules
including pretest/posttest and
student explanations of a worked
example, 3 project interview, 4
end-of-the-semester interview
1. BASELINE
Start
J Sci Educ Technol (2018) 27:322–333
this study was adapted to include two different areas in which
the students could develop their ability to control the outcome:
designing and writing an algorithm or computer program and
using computation for scientific purposes (e.g., visualizing a
phase diagram or solving a set of linear equations).
Throughout the entire semester, students attended regular
bi-weekly lectures and one laboratory session were they
worked on the computational modules. Each computational
module started with a 5-min pretest regarding THERMO con-
cepts that were explained during the lectures. Appendix 1
includes the pre/posttest instruments. After the pretest, stu-
dents worked with the Gibbs software (Cool et al. 2010), as
implemented in the VKML (Bartol et al. 2014).
Three computational modules were introduced as part of
the THERMO course (see Table 2). All the modules materials
and the worked examples were accessible to students through
the VKML hosted on nanoHUB.org. The activities within
each module consisted of using VKML to represent the
thermodynamics phenomena. After getting familiar with the
worked example, students were asked to complete a
homework assignment to write in-code comments explaining
what the code was doing. In 2015, students were allowed to
complete the assignment individually or in groups, and this
assignment was extra-credit for the students. Since most of the
students chose not to complete these extra-credit activities, the
instructor and the research team decided to make them indi-
vidually graded as part of the homework assignment in 2016.
One week after the computational module was delivered, a
posttest was administered, often together with the correspond-
ing pretest for the following module.
The first module was implemented as part of the topic of
state variables and state functions. A state function is Ba prop-
erty of a material system that mathematically describes the
equilibrium state of the system independently on what process
the system followed to arrive at that condition^ (DeHoff
2006). These concepts had been introduced to the students
during that week lecture sessions. The pretest and posttest
consisted of a single yes-or-no question that required students
SAMPLE MODULE
SELF-EXPLAINING
ONE EXAMPLE
PRETEST Students write POSTTEST
Instructor walks
Thermo
students through
comments within Thermo
Concepts the code to self- Concepts
the example
explain it
3. PROJECT
SURVEY
3. EXIT
MODULE 1 MODULE 2 MODULE 3 End
2. COMPUTATIONAL MODULES
SPRING 2015, 2016
J Sci Educ Technol (2018) 27:322–333 325

Table 1 Self-belief survey items as adapted from Magana et al. (2016)

Original Adapted

Construct Question Construct Question

Control I have the ability to design an algorithm.
I have the ability to write a computer program.
I have the ability to visualize data using a
computer.
I have the ability to implement a graphical
user interface.
N/A
N/A
Value I feel computation (data visualization, modeling
and simulation algorithm design) will be
useful in my studies.
I feel computation (data visualization, modeling
and simulation algorithm design) will be
useful in my career.
I intend to purposefully seek courses that will
allow me to increase my knowledge about
computation.
I intend to use computation (data visualization,
modeling, and simulation algorithm design)
in my future career.
Programming abilities
Scientific computing abilities
Perceived value of computation
Q1. I have the ability to design an algorithm.
Q2. I have the ability to write a computer
program.
Q3. I have the ability to visualize data using
a computer.
Q4. I have the ability to build phase diagrams
using computation.
Q5. I have the ability to use a computer to solve
a set of linear equations.
Q6. I have the ability to implement a numerical
model based on a simple partial differential
equation
Q7. I feel computation (data visualization,
modeling, and simulation algorithm design)
will be useful in my studies.
Q8. I feel computation (data visualization,
modeling, and simulation algorithm
design) will be useful in my career.
Q9. I intend to purposefully seek courses that
will allow me to increase my knowledge
about computation.
Q10. I intend to use computation (data
visualization, modeling, and simulation
algorithm design) in my future career.

to show their procedures (see Appendix 1). Students were
given five extra-credit points for completing each of these
instruments correctly, and the teaching assistant (TA) was in
charge of the scoring process.
The session continued with the course instructor describing
simple actions in VKML such as create a new file, declare
variables, write a Bhello world^ program, write a for-loop,
execute a program, import Python packages, and insert com-
ments within the code. The instructor also highlighted the
importance of commenting the code to be able to collaborate
and better understand a program. The next part of the module
was intended to write a program in VKML that would identify
whether a given equation was a state function or not. First, the
group discussed the activity conceptually and mathematically.
Then, the instructor guided the implementation along with a
worked example of this program. The student assignments for
the following week included commenting the code to describe
each step and modify the program so that it evaluated a dif-
ferent function. The worked example for this module is pre-
sented in Fig. 2. The code first declared the thermodynamic
variables to create the equation A on section 3. On section 5,
the program computed the partial derivatives of the equation,
which were differentiated again on section 7. On section 9, the
worked example verified whether both derivatives were the
same or not to identify equation A as a state function.
The other two computational modules followed the same
procedures as in the first one, but with subsequent topics from
the course. Module II focused on stability of binary solutions

Table 2 Learning outcomes of

the three computational modules Module Learning outcome

Module [Disciplinary] Learners will identify the conditions that define thermodynamic stability,
1 metastability, and instability[Computational] Learners create, execute, and interpret python
programs using VKML
Module [Disciplinary] Learners will identify the free energy curve for eutectic, peritectic, and lenticular
2 (isomorphous) phase diagrams[Computational] Learners will create binary phase diagrams using
VKML
Module [Disciplinary] Learners will identify the phases of the peritectic phase diagram and the solubility
3 limit of a given component[Computational] Learners will create phase diagrams using VKML
326
Fig. 2 Worked example for module no. 1: identifying whether a given
function is a state function or not
with phase diagrams while module III introduced phase dia-
gram analysis of composition range(s). These two modules
included a graphical user interface where students could
change parameter values to identify their effect on the phase
diagrams. Table 2 summarizes the learning outcomes of the
modules. All the python codes are accessible through VKML
(Bartol et al. 2014).
Validity and Reliability
In order to validate the survey items and constructs, we
computed the Pearson correlation coefficient among the
ten questions using students to the survey instruments.
The number of complete observations that were used for
this analysis was 198. Table 3 shows the Pearson correla-
tion values among all the survey questions. All the corre-
lation values were statistically significant (i.e., p value <
0.05). The item Q4 (I have the ability to build phase
diagrams using computation) showed a moderate correla-
tion to several items within the perceived value constructs
(e.g., Q7, Q8, and Q10). Since students were part of a
Materials Science and Engineering program, this relation-
ship makes sense: students who know how to visualize
phase diagrams using computation, also, see the value of
using computation in their career.
We then performed a factor analysis using the maxi-
mum likelihood factoring method to validate the three
constructs (see Table 4). Since the factors were correlated
to each other, we used oblimin as the rotation method.
J Sci Educ Technol (2018) 27:322–333
Table 3 Pearson correlation among survey questions
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
Q1 1
Q2 0.53 1
Q3 0.34 0.49 1
Q4 0.38 0.44 0.61 1
Q5 0.29 0.24 0.41 0.47 1
Q6 0.46 0.34 0.44 0.46 0.44 1
Q7 0.36 0.27 0.39 0.43 0.21 0.29 1
Q8 0.27 0.34 0.4 0.42 0.18 0.33 0.67 1
Q9 0.22 0.31 0.25 0.33 0.19 0.17 0.59 0.56 1
Q10 0.27 0.4 0.36 0.51 0.21 0.28 0.58 0.6 0.6 1
All the correlation values were statistically significant: p value < 0.05.
All italicized entries have a p value < 0.001
Our factor analysis showed similar results compared to
Magana et al. (2016 regarding the perceived value. The
construct scientific computing abilities also shared the
item Q2 with the construct on programming abilities.
Table 4 also shows the internal consistency measure for
all the constructs. Cronbach’s alpha (Cronbach 1951) was
computed for the constructs scientific computing abilities
and perceived value, but not for programming abilities.
Cronbach’s alpha is not recommended for constructs with
only two items (Eisinga et al. 2013), and therefore, we
report the Spearman-Brown coefficient as the reliability
measure for this construct. An alpha of 0.7 or higher has
been suggested to be acceptable in the social sciences
(Kline 1999).
The content validity of the pretest/posttest instruments
is given by experts iteratively designing and reviewing
these instruments. First, the course instructor and the
teaching assistant met to discuss what type of questions
would be relevant for a given module. Then, the teaching
assistant created sample pretest and posttest questions,
which the course instructor made comments on. The teach-
ing assistant refined the instruments until agreement was
reached. The scoring process was also discussed between
the course instructor and the teaching assistant until agree-
ment was reached.
Data Analysis
The pretest/posttest instruments, assessing conceptual
knowledge, were scored by the TA considering not only
the students’ correct response but also their procedure or
explanation. Each of these pretest-posttest pairs from the
J Sci Educ Technol (2018) 27:322–333 327

Table 4 Factor analysis and

Cronbach’s alpha for the three Construct Internal consistency Item Factor 1 Factor 2 Factor 3
constructs
Programming abilities 0.69* Q1 0.99
Q2 0.31 0.35
Scientific computing abilities 0.77** Q3 0.74
Q4 0.75
Q5 0.62
Q6 0.52
Perceived value 0.86** Q7 0.77
Q8 0.77
Q9 0.8
Q10 0.7
Eigenvalue 2.44 2.07 1.29
% of variance 24% 21% 13%

***Spearman-Brown formula value; Cronbach’s alpha

four computational modules were also compared using a
paired t test, aimed at identifying statistically significant
differences in the students’ disciplinary understanding af-
ter being exposed to the computational modules (RQ1).
The assumptions of normality and homogeneity of vari-
ance were satisfied. In order to identify the factors that
influence these changes (RQ3), we used a one-way
ANOVA with the gain from pretest to posttest within each
module as a dependent variable. The independent vari-
ables that we evaluated are (1) the course offering, 2015
or 2016; (2) the number of programming courses the stu-
dents were previously exposed to; and (3) whether the
student had prior experience with Python or not.
We computed the three constructs from the survey (i.e.,
programming abilities, scientific computing abilities, and
perceived value of computation) as a linear combination
of the items that resulted from the factor analysis, using
the corresponding factor loadings. We then identified sig-
nificant changes between pre and posttest for each of
these constructs using a paired t test (RQ2). The assump-
tions for the parametric test were satisfied for the three
constructs. Finally, we compared the gain from pretest to
posttest within each construct using one-way ANOVA
with the same independent variables as before (RQ4).
The purpose of this analysis was to identify whether any
of these variables had an effect on the changes of the
students’ self-beliefs.
Results
This study explores the changes on student learning and self-
beliefs after being exposed to three computational science
modules within a thermodynamics course, a materials engi-
neering core course. We first present the descriptive statistics
for each of the measures and basic inferential statistics to
identify potential changes pretest to posttest. The next step is
then to identify whether the students’ prior experiences with
programming or the graded self-explanation activities had an
effect on these changes.
Learning Gains
Table 5 lists the average score for the pretest and posttest
instruments in the three computational modules. These
instruments were scored in a scale from zero to five: a
low score is considered to be below two, a moderate score
is between two and 3.5, and a high score is considered to
be higher than 3.5. Overall student performance on the
pretest can be considered from low to moderate, except
for module two where on average the student’s performed
over 50%. Student performance on the posttest is higher
for all the modules compared to the pretest. Modules one
and two moved towards a high score, while module three
resulted now as moderate.
The scores for pretest and posttest within each compu-
tational module were compared using a paired t test. We
used Bonferroni correction to protect from type I error.
Our initial alpha level of 0.05 was divided by the three
tests that we conducted for the three modules. Thus, in
order to be a significant result, the p value must be below
0.0167. The three modules showed positive significant
differences from pretest to posttest, suggesting that stu-
dent exposure to these modules actually increased their
conceptual understanding. The effect sizes for these dif-
ferences were computed using Cohen’s d (Cohen1988).
328 J Sci Educ Technol (2018) 27:322–333

Table 5 Descriptive and

inferential statistics of pretest and Module no. Pretest Posttest t test
posttest instruments for the three
computational modules Mean SD N Mean SD N t DF p value d

1 2.7 2.13 107 3.84 1.92 105 − 5.37 103 < 0.01* 0.56
2 3.22 2.01 105 4.01 1.47 102 − 3.52 98 < 0.01* 0.45
3 2.18 1.85 107 3.43 1.96 106 − 5.96 104 < 0.01* 0.66

*p value < 0.0167, significant difference with Bonferroni correction

These values can be interpreted as follows (Rubin 2012):
weak—|d| < 0.2; weak to moderate—0.2 < |d| < 0.4; mod-
erate—0.4 < |d| < 0.65; moderate to strong—0.65 < |d| <
0.8; and strong—0.8 < |d|. Modules one and two showed
a moderate effect size, while module three showed a mod-
erate to strong effect size.
show a significant effect from pretest to posttest. The effect
size was weak to moderate for the programming abilities and
strong for the scientific computing abilities.
Controlling Factors

Self-Beliefs In order to identify factors that could affect student conceptual

The descriptive statistics for the three self-belief con-
structs assessed in the beginning and the end of the
semester survey are presented in Table 6. Since these
scores were computed using the factor loadings derived
from the factor analysis process, we normalized their
resulting value so that they are comparable to each oth-
er. Values in the range 0–30% were considered negative,
the range 31–70% was considered neutral, and anything
above 70% was considered positive self-beliefs. The re-
sults suggest that the two constructs related to ability
increased, although the programming abilities remained
neutral. The scientific computing abilities moved from
neutral to positive, and the perceived value of computa-
tion remained positive in spite of a small negative
change from pretest to posttest.
These three constructs were then compared from pretest to
posttest using paired t tests. The two constructs related to
student ability to control certain outcome showed significant
positive differences after students had been exposed to the
computational modules. Only the perceived value did not
change or changes in student self-beliefs, the research team
carried out a one-way ANOVA using the following indepen-
dent variables: (1) course offering: 2015 or 2016, (2) number
of previous programming courses, and (3) prior Python
experience.
The students’ performance between pretest and posttest
did not show any significant differences for module one
regarding the course offering (F(95 ,1) = 2.98, p value =
0.09), previous programming courses (F(95, 6) = 1, p val-
ue = 0.09), or prior Python experiences (F(95, 1) = 2, p
value = 0.16), neither did they show for module two:
course offering (F(95, 1) = 1.07, p value = 0.31), previous
programming courses (F(95, 6) = 1.21, p value = 0.31), or
prior Python experiences (F(95, 1) = 1.07, p value = 0.31);
and module three: course offering (F(95, 1) = 3.9, p val-
ue = 0.05), previous programming courses (F(95, 6) =
0.73, p value = 0.63), or prior Python experiences (F(95,
1) = 0.05, p value = 0.8). These results suggest that the
students’ previous experiences with programming did
not affect their learning gains regarding the disciplinary
knowledge in any of the modules.

Table 6 Descriptive and inferential statistics of pretest and posttest instruments for the three computational modules

Construct Pretest Posttest t test

Mean (%) SD (%) N Mean (%) SD (%) N t DF p value d

Programming abilities 56.53 22.95 108 64.37 22.95 91 − 2.66 90 < 0.01* 0.34
Scientific computing abilities 61.73 16.88 108 75.25 16.17 92 − 6.49 91 < 0.01* 0.82
Perceived value 74.59 18.34 107 71.96 18.59 92 0.87 90 0.39 0.14

*p value < 0.0167, significant difference with Bonferroni correction

J Sci Educ Technol (2018) 27:322–333
The changes from the pretest survey to the posttest that
were identified in the programming ability construct did
not depend on the course offering (F(82, 1) = 0.16, p val-
ue = 0.69), the previous programming courses (F(82, 6) =
0.92, p value = 0.49), or the students’ prior Python expe-
riences (F(82, 1) = 3.85, p value = 0.05). Although the
overall perceived value of computation did not change
significantly from pretest to posttest, we still conducted
this analysis to identify whether certain groups of students
(e.g., students with certain previous experiences) had had
specific changes. However, the results were not signifi-
cant for any of the three independent variables as related
to the students’ perceived value of computation. Only the
changes in scientific computing ability showed a signifi-
cant difference between students who had prior experi-
ence with Python and those who did not have them
(F(83, 1) = 6.157, p value = 0.02). Students who had prior
experience with Python showed a smaller increase
(mean = 5.12%, SD = 17.88%) as compared to those stu-
dents who did not have this previous experiences (mean =
16.58%, SD = 20.10%).
Discussion
This study explored the effect of introducing computa-
tional modules into a thermodynamics course on student
learning and student self-beliefs. Three computational
modules employed the VKML computer simulations to
represent thermodynamics abstract phenomena and to ex-
pose students to computational tools that can be useful in
their careers. The next subsections explore:(1) changes on
student learning, (2) changes on student self-beliefs, (3)
factors affecting student learning, and (4) factors affecting
student self-beliefs.
RQ1: What is the effect of introducing computational
modules as part of a thermodynamics course on student
disciplinary learning?
Students attended the computational modules after being ex-
posed to the classroom lectures. The modules started with a
pretest assessing student understanding of the concepts that
were going to be represented using VKML computer simula-
tion. Students then interacted with the computer simulation
using a worked example that helped them represent such ab-
stract concepts. One week later, when the next computational
module was starting, students completed the corresponding
posttest.
The results from the pretest/posttest analysis suggest
that students benefited from the computational modules
to increase their understanding about thermodynamics
concepts taught during the lecture time. Student scores
329
for the three modules showed a positive significant differ-
ence between pretest and posttest measures. The results
suggest that the constructivist approach (Mulop et al.
2012) using the VKML computer simulation supported
the students’ conceptual change (Duit and Treagust
2003; Pyatt and Sims 2012) in the context of thermody-
namics. Having students reflect on how the examples
worked, and use the interactive visualizations, may have
helped them to better understand abstract concepts taught
during the lecture sessions (Magana et al. 2016; Anderson
and Smith 1987; Weintrop et al. 2016). In a way, the pretest
instruments might have also helped students to identify their
fragmented or incomplete conceptions, while the computer
simulations helped students to repair these misconceptions
through the visualization of abstract phenomena (Magana
et al. 2016; Anderson and Smith 1987).
RQ2: What is the effect of introducing computational
modules as part of a thermodynamics course on student
self-beliefs about computation?
Student self-beliefs about computation also changed as a
result of the computational modules. The construct that
had the largest positive effect was scientific computing
abilities, which was aimed to assess the students’ ability
to use computation to represent disciplinary phenomena.
The programming ability showed a weak to moderate pos-
itive change, suggesting that even though the modules did
not require students to create programs, these still helped
students to increase their perceived ability to do computer
programming. Finally, the construct assessing the per-
ceived value of computation did not show a significant
change. However, this construct had shown a positive
perceived value from the pretest, which suggests that stu-
dents already found computation to be an important ele-
ment in their careers. These findings are aligned to other
studies showing that supplementing disciplinary courses
with computational modules have a positive effect on
the students’ perceived abilities with computation, but
not on the already positive perceived value of computa-
tion (Magana et al. 2016; Mansbach et al. 2016).
RQ3: What are the factors that affect changes on student
understanding of disciplinary phenomena after being
exposed to computational modules as part of a thermo-
dynamics course?
Prior research has shown that computational modules involv-
ing computer programming may require to have students be
exposed to prior programming experiences in order to have a
positive effect on disciplinary learning (Magana et al. 2016).
In order to identify whether these prior experiences have an
effect on student learning, the research team conducted one-
330
way ANOVA controlling for the following variables: (1)
course offering: 2015 or 2016; (2) number of previous pro-
gramming courses; and (3) prior Python experience. The rel-
evance of including the course offering lies on the fact that the
students’ self-explanation activities were graded for the sec-
ond offering (2016) of the course, but not for the first one
(2015).
The results from this study suggest that the students’
conceptual change after being exposed to the computa-
tional modules was not affected by their prior experiences
nor by the course offering. Although the computational
modules allowed students to access the Python code, this
was not a requirement for students to take advantage of
the computational visualizations. Also, providing students
with the worked examples may have been enough scaf-
folding for students to be able to focus on the interpreta-
tion of the output instead of actually building the tool.
Even in 2016, when students were asked to write com-
ments within the code as a self-explanation strategy, the
learning gains for disciplinary knowledge were not signif-
icantly different when compared to 2015.
RQ4: What are the factors that affect changes on student
self-beliefs about computation after being exposed to
computational modules as part of a thermodynamics
course?
A similar analysis was conducted to identify whether
these factors had an effect on the changes of the students’
self-beliefs after participating in the computational mod-
ules. In this case, the results suggested significant differ-
ences for changes in scientific computing ability when
students had prior experience with Python. Students with-
out this experience showed a larger positive effect from
the computational modules in their ability to use it for
scientific computing when compared to students with this
experience. This result suggests that the approach evalu-
ated here (i.e., VKML simulation tool + worked examples
in thermodynamics) helped students to increase their self-
perceived ability, even if they had no experience with the
programming language.
Conclusion, Limitations, and Next Steps
This study explored the effect of introducing three computa-
tional modules as part of a thermodynamics course on student
disciplinary learning and student self-beliefs. The computa-
tional modules were taught after students attended the class-
room lectures, and these were designed to support the stu-
dents’ understanding of abstract thermodynamics phenomena.
The modules used the Virtual Kinetics of Materials
Laboratory (VKML), hosted in nanoHUB.org (https://
J Sci Educ Technol (2018) 27:322–333
nanohub.org/tools/vkmllive). VKML is based on Gibbs
(Cool et al. 2010), a set of Python-based libraries. Students
were provided with a worked example code that created a
representation of a given phenomenon, and they could either
interact with the code directly, or just with the user interface to
modify parameters.
The results from this study suggest that the construc-
tivist approach using computer simulations can help stu-
dents to better understand abstract disciplinary concepts.
At the same time, student self-beliefs about computation
changed towards positive ones, especially as related to
ability to do scientific computing. Moreover, the fact that
some students did not have prior experiences with com-
puting did not prevent them from learning and having this
positive effect in their self-beliefs. In fact, students with
no experiences in Python actually showed a larger posi-
tive effect in their perceived ability to do scientific
computing.
While this study evaluated the effect of the modules on
student disciplinary learning and self-beliefs of computa-
tion, we still do not know how much students actually
learned about computation, and how is that related to dis-
ciplinary learning or student self-beliefs. Some students
used the VKML to complete their course projects, but this
was not a requirement for the project itself. Consequently,
the next steps in this research will look into changes on
student learning of computation and how these can trans-
fer to other disciplinary contexts.
An additional limitation of this study is the lack of a
control group to compare this constructivist approach to
more traditional approaches (e.g., classroom lectures).
Since students participating in this study spent additional
time working on these concepts after the lectures, it is
possible that such time on task had an effect on the out-
comes of this study as related to student learning. A future
study may look at comparing two groups that spend the
same time working on these concepts to identify the
affordances of using computational models and simula-
tions for thermodynamics concepts.
Acknowledgements The research reported in this paper was supported in
part by the US National Science Foundation under the awards no.
EEC1449238 and no. EEC1329262. The content is solely the responsi-
bility of the authors and does not necessarily represent the official views
of the National Science Foundation.
Compliance with Ethical Standards All procedures in this study that in-
volved human participants were in accordance with the ethical standards
of the institution and were approved by the Institutional Review Board
(IRB) before the implementation of the research procedures. All proce-
dures performed in studies involving human participants were in accor-
dance with the ethical standards of the institutional research committee
and with the 1964 Helsinki Declaration and its later amendments or com-
parable ethical standards. For this type of study, formal consent is not
required.
J Sci Educ Technol (2018) 27:322–333 331

Appendix 1

Pretest Module I

Q: Is the following f = ( )

(a) Yes (b) No

[Encircle the correct answer. Show all your working steps below. Use the back side of this page if necessary.]

Module I

Q: Is the following f = /

(a) Yes (b) No

[Encircle the correct answer. Show all your working steps below. Use the back side of this page if necessary.]

Pretest Module II
332 J Sci Educ Technol (2018) 27:322–333

Pretest Module III

Posest Module III

J Sci Educ Technol (2018) 27:322–333
References
Alabi, O., Magana, A.J. and Garcia, R.E. (2015). Gibbs, computational
simulation as a teaching tool for students’ understanding of thermo-
dynamics of materials concepts. Journal of Materials Education,
37(5-6), 239-260.
Anderson, C., & Smith, E. (1987). Teaching science. In V. Richardson-
Koehler (Ed.), The educator’s handbook: a research perspective (pp.
84–111). New York: Longman.
Baher, J. (1999). Articulate virtual labs in thermodynamics education: a
multiple case study. J Eng Educ, 88(4), 429–434. https://doi.org/10.
1002/j.2168-9830.1999.tb00470.x.
Bartol, A., Cool, T., & García, R. E. (2014). The virtual kinetics of materials
laboratory. Retrieved from https://www.nanohub.org/tools/vkmllive/.
Chi, M. T. H., Roscoe, R. D., Slotta, J. D., Roy, M., & Chase, C. C. (2012).
Misconceived causal explanations for emergent processes. Cogn Sci,
36(1), 1–61. https://doi.org/10.1111/j.1551-6709.2011.01207.x.
Cobourn, W. G., & Lindauer, G. C. (1994). A flexible multimedia instruc-
tional module for introductory thermodynamics. J Eng Educ, 83(3),
271–277. https://doi.org/10.1002/j.2168-9830.1994.tb01115.x.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences
(pp. 20–26). Hillsdale: Lawrence Earlbaum Associates.
Cool, T., Bartol, A., Kasenga, M., Modi, K., & García, R. E. (2010).
CALPHAD: computer coupling of phase diagrams and thermo-
chemistry Gibbs: phase equilibria and symbolic computation of ther-
modynamic properties. CALPHAD: Computer Coupling of Phase
Diagrams and Thermochemistry, 34(4), 393–404. https://doi.org/10.
1016/j.calphad.2010.07.005.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of
tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/
BF02310555.
DeHoff, R. (2006). Thermodynamics in materials science (2nd ed.). Boca
Raton; Taylor & Francis.
Duit, R. (1999). Conceptual change approaches in science education. In
W. Schnotz, S. Vosniadou & M. Carretero (Eds.), New perspectives
on conceptual change (pp. 263–282). Oxford: Elsevier Science
Duit, R., & Treagust, D. F. (2003). Conceptual change: a powerful frame-
work for improving science teaching and learning. Int J Sci Educ,
25(6), 671–688. https://doi.org/10.1080/09500690305016.
Eisinga, R., Grotenhuis, M.. & Pelzer, B. (2013) Int J Public
Health 58(4), 637–642. https://doi.org/10.1007/s00038-012-0416-3
Khan, S. (2011). New pedagogies on teaching science with computer
simulations. J Sci Educ Technol, 20(3), 215–232. https://doi.org/
10.1007/s10956-010-9247-2.
Kline, P. (1999). The handbook of psychological testing (2nd ed.).
London: Routledge.
Magana, A. J., & Mathur, J. (2012). Motivation, Awareness and
Perceptions of Computational Science. Computing in Science and
Engineering (CiSE). IEEE Computer Society, 14(1), 74–79.
Magana, A. J., Falk, L. M., & Reese, J. M. (2013). Introducing Discipline-
Based Computing in Undergraduate Engineering Education. ACM
Transactions on Computing Education, 13(4), 1–22.
Magana, A. J., Falk, M. L., Vieira, C., & Reese, M. J. (2016). A case
study of undergraduate engineering students' computational literacy
and self-beliefs about computing in the context of authentic prac-
tices. Computers in Human Behavior 61, 427–442. https://doi.org/
10.1016/j.chb.2016.03.025.
Mansbach, R., Ferguson, A., Kilian, K., Krogstad, J., Leal, C., Schleife,
A., et al. (2016). Reforming an undergraduate materials science
curriculum with computational modules. J Mater Educ, 38(3–4),
161–174.
Mulop, N., Yusof, K. M., & Tasir, Z. (2012). A review on enhancing the
teaching and learning of thermodynamics. Procedia-Social and
Behavioral Sciences, 56, 703–712. https://doi.org/10.1016/j.
sbspro.2012.09.706.
333
National Science Foundation. (2011). Empowering the nation through
discovery and innovation - NSF strategic plan for fiscal years (FY)
2011–2016. Washington, DC: Author. Retrieved from http://www.
nsf.gov/news/strategicplan/nsfstrategicplan_2011_2016.pdf.
Olds, B. M., Streveler, R. A., Miller, R. L., & Nelson, M. A. (2004).
Preliminary results from the development of a concept inventory in
thermal and transport scienceage, 9, 1.
PITAC. (2005). Computational science: ensuring America’s
competitiveness. Retrieved from http://www.nitrd.gov/pitac/
reports/20050609_computational/computational.pdf
Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982).
Accommodation of a scientific conception. Toward a theory of con-
ceptual change. Sci Educ, 66(2), 211–227. https://doi.org/10.1002/
sce.3730660207.
Probst, D. K., &Zhang, Y. (2013) A gentle bridge between dynamics and
thermodynamics. In proceedings of the 2013 ASEE Annual
Conference & Exposition, Atlanta, Georgia. https://peer.asee.org/
19062
Pyatt, K., & Sims, R. (2012). Virtual and physical experimentation in
inquiry-based science labs: attitudes, performance and access. J
Sci Educ Technol, 21(1), 133–147. https://doi.org/10.1007/s10956-
011-9291-6.
Rubin, A. (2012). Statistics for evidence-based practice and evaluation
(research, statistics and program evaluation) (3rd ed.). Nashville:
Cengage Learning.
Shiflet, A. B., & Shiflet, G. W. (2006). Introduction to computational
science: Modeling and simulation for the sciences. Princeton:
Princeton University Press.
The Joint Task Force on Computing Curricula—ACM/IEEE-Computer
Society. (2013). Computer Science Curricula 2013: Curriculum
Guidelines for Undergraduate Degree Programs in Computer
Science. Practice.https://doi.org/10.1145/2534860.
The Minerals, Metals & Materials Society. (2013). Integrated
Computational Materials Engineering (ICME): implementing
ICME in the aerospace, automotive, and maritime industries.
Warrendale. Retrieved from www.tms.org
Turbak, F., & Berg, R. (2002). Robotic design studio: Exploring the big
ideas of engineering in a liberal arts environment. J Sci Educ
Te c h n o l , 11 ( 3 ) , 2 3 7 – 2 5 3 . h tt p s : / / d o i . o rg / 1 0 . 1 0 2 3 / A :
1016376818781.
Turner, P. R., Cunningham, S., Phillips, A. T., Claire, E., Shiflet, A. B.,
Stewart, K., …Shiflet, A. (2002). Undergraduate Computational
Science and Engineering Programs and Courses. In SIGCSE (pp.
96–97). Covington, Kentucky. https://doi.org/10.1145/563340.
563374
Vieira, C., Magana, A. J., Falk, M. L., & Garcia, R. E. (2017). Writing in-
code comments to self-explain in computational science and engi-
neering education. ACM Transactions on Computing Education,
17(4), 1–21. https://doi.org/10.1145/3058751.
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., &
Wilensky, U. (2016). Defining computational thinking for mathe-
matics and science classrooms. J Sci Educ Technol, 25(1), 127–147.
https://doi.org/10.1007/s10956-015-9581-5.
Windschitl, M. A. (1995). Using computer simulations to enhance con-
ceptual change: the roles of constructivist instruction and student
epistemological beliefs. (Doctor of Philosophy), Iowa State
University Ames, Iowa.
Wofford, J. (2009). K-16 computationally rich science education: A ten-
year review of the journal of science education and technology
(1998–2008). J Sci Educ Technol, 18(1), 29–36. https://doi.org/10.
1007/s10956-008-9127-1.
Yang, D., Streveler, R. A., Miller, R. L., Slotta, J. D., Matusovich, H. M.,
& Magana, A. J. (2012). Using computer-based online learning
modules to promote conceptual change: helping students understand
difficult concepts in thermal and transport science. Int J Eng Educ,
28(3), 686.