Accepted Manuscript

Customizing scaffolds for game-based learning in physics: Impacts on knowledge

acquisition and game design creativity

Gloria Yi-Ming Kao, Chieh-Han Chiang, Chuen-Tsai Sun

PII: S0360-1315(17)30134-3
DOI: 10.1016/j.compedu.2017.05.022
Reference: CAE 3191

To appear in: Computers & Education

Received Date: 14 October 2016

Revised Date: 30 May 2017
Accepted Date: 31 May 2017

Please cite this article as: Kao G.Y.-M., Chiang C.-H. & Sun C.-T., Customizing scaffolds for game-
based learning in physics: Impacts on knowledge acquisition and game design creativity, Computers &
Education (2017), doi: 10.1016/j.compedu.2017.05.022.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to
our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and all
legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT
Title page

Title: Customizing Scaffolds for Game-Based Learning in Physics: Impacts on

Knowledge Acquisition and Game Design Creativity

PT
Authors: Gloria Yi-Ming Kao a,*, Chieh-Han Chiang b, Chuen-Tsai Sun c

RI
Graduate Institute of Digital Learning and Education, National Taiwan
University of Science and Technology, Taipei 106, Taiwan, ROC
b
Degree Program of E-Learning, College of Science, National Chiao Tung

SC
University, Hsinchu 300, Taiwan, ROC
c
Department of Computer Science, National Chiao Tung University,
Hsinchu 300, Taiwan, ROC

U
AN
M

Corresponding author:
Dr. Gloria Yi-Ming Kao
D

Graduate Institute of Digital Learning and Education

National Taiwan University of Science and Technology
TE

43 Keelung Rd., Sec. 4

Taipei 106, Taiwan
EP

Tel: +886-2-2733-3141 Ext. 7467

Fax: +886-2-2737-6433
Email: gloriakao@gmail.com
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 1
ACCEPTED MANUSCRIPT

Customizing Scaffolds for Game-Based Learning in Physics:

Impacts on Knowledge Acquisition and Game Design Creativity

Abstract

PT
Scaffolds in games have the potential to facilitate learning effects in addition
to assisting the gaming process. However, studies on game-based science learning

RI
usually only adopt questionnaires to evaluate scientific concepts, and use interviews
or observations to assess problem solving ability, neglecting the importance of

SC
investigating game-making outcomes. We customized a digital game, “Crayon
Physics Deluxe,” with varied scaffolding designs to evaluate their effects on science
learning. A total of 126 participants were divided into four groups: demonstration

U
scaffolding, non-scaffolding, marking critical features scaffolding, and the no-game
group. Learning outcomes were examined in terms of physics knowledge acquisition
AN
(in the form of concept maps) and design creativity (in the form of game episode
designs). Students were asked to transform their roles from problem solvers to
M

problem designers, which might not only demonstrate their ability to solve scientific
problems, but also develop their creativity potential in designing scientific puzzles.
The results indicated that the marking critical features scaffolding group performed
D

significantly better than the demonstration scaffolding group in both conceptual

TE

knowledge acquisition and the sensitivity dimension of design creativity, while the
group with demonstration scaffolding scored higher in the flexibility dimension of
design creativity. These findings suggest that proper scaffolds could be designed to
EP

function as learning scaffolds rather than just as gaming scaffolds, and different
learning purposes require various scaffolding designs. The content of the scaffolds, as
well as the timing of their provision should be carefully designed according to the
C

game features to achieve specific instructional purposes.

AC

Keywords: applications in subject areas; improving classroom teaching; media in

education; pedagogical issues; teaching/learning strategies

1
 2
ACCEPTED MANUSCRIPT

1. Introduction
Digital games have been used as effective tools for facilitating learning in
various domains such as language, software engineering, creativity, and science
education (Barzilai & Blau, 2014; Connolly, Stansfield, & Hainey, 2007; Hao
et al., 2010; Liu & Chu, 2010; Ott & Pozzi, 2012). Raybourn and Bos (2005)
pointed out that a game-based learning environment could provide learners

PT
with opportunities to experience abstract concepts and enhance their learning.
For example, Rosas et al. (2003) introduced digital games into the classroom as
instructional tools, and found that they could exert positive effects on learning

RI
outcomes, learning motivation and classroom dynamics; in particular, their
results showed significant improvements in reading comprehension, spelling,

SC
and mathematical skills for the gaming group. Ott and Pozzi (2012) also
indicated an increasing trend in students’ creative skills and attitudes related to
figuring out and enacting original solution strategies in their three-year

U
game-based learning project in which they adopted indicators from cognitive,
AN
meta-cognitive and affective aspects to evaluate creativity.

Game characteristics such as goals, rules, interactivity, feedback, and

M

challenges are essential in creating engaging gaming experience (Prensky,

2001). Among these, appropriate challenges that correspond to the skills
acquired to get through the game allow players to experience flow
D

(Csikszentmihalyi, 1997). In the flow state, players are neither bored nor
TE

anxious. Suits (1978) described game playing as the voluntary attempt to

overcome unnecessary obstacles, which underscores the importance of
autonomy. A successful game design allows players to explore the game world
EP

at their own pace and satisfy their needs as explorers (Bartle, 1996; Rapp,
2015). When confronted with obstacles, players are willing to face the
challenges and continue playing since they are highly motivated and want to
C

satisfy their needs by attaining a long-term goal, instead of choosing an easier

AC

way (using scaffolds or tips) to attain a short-term goal. This concept is similar
to the concept of desirable difficulties (Bjork & Bjork, 2011), which suggests
that introducing certain difficulties into the learning process can greatly
improve long-term retention of the learned material. To sustain “appropriate”
challenges, games usually use scaffolds in their game design and allow players
to decide whether or when to use them, to maintain a balance between
frustration and “desired difficulty.”

2
 3
ACCEPTED MANUSCRIPT
One of the main challenges of adopting games in learning scenarios is to
maintain a sense of enjoyment in the game while still achieving the desired
learning effects (Charsky & Ressler, 2011). Meanwhile, games should be well
designed to provide opportunities for learners to actively experience, practice,
and reflect on their concepts in authentic problem-based situations in addition
to retaining a fun element (Oblinger, 2004; Squire, 2008). To facilitate positive
learning outcomes and provide engaging learning experience, designers or

PT
teachers should consider the learners’ perspective, and incorporate suitable
instructions in the process of the game design (Becker, 2007).

RI
However, it takes teachers a great deal of effort to design a completely new
game; a possible alternative is therefore to choose an appropriate game and

SC
embed scaffolds into it for personal instructional purposes. In this way, the
burden of developing a digital game from the very beginning could be largely
relieved for teachers. Researchers have suggested supplementing games with

U
external scaffolds or instructional activities that support the connections
AN
between what they have achieved in the game and the content knowledge
learned at school (Barzilai & Blau, 2014; Charsky & Mims, 2008; Quintana et
al., 2004). For example, Barzilai and Blau (2014) added an external conceptual
M

scaffold as formal knowledge representations to a business simulation game to

assist learners in developing financial-mathematical problem-solving abilities.
D
TE

1.1 Game based learning in science education

In particular, digital games have been advocated as an effective approach
EP

to increasing students’ interest and to facilitating learning outcomes for science

education (Cheng, She, & Annetta, 2015; Mayo, 2007; Quintana et al., 2004).
For example, Scaffolding Understanding by Redesigning Games for Education
C

(SURGE: D‘Angelo, 2010) helped players make explicit connections from the
AC

game to specific science content, thus bridging the gap between student
learning in non-formal game environments and the formalized knowledge
structures at school. Cheng et al. (2015) indicated that players did in fact learn
from playing a serious educational game, and the game immersion experience
did impact on their science learning and led to higher gaming performance.
Chen, Wang, and Lin (2015) found that students who played in a solitary or
collaborative mode of game-based learning demonstrated improvement in their
science learning outcomes.
However, Wu, Chiou, Kao, Hu, and Huang (2012) reported that most

3
 4
ACCEPTED MANUSCRIPT
game designers have mainly specifically described the game design processes
and the technological applications themselves, while rarely focusing on the
learning theory foundations of their educational games. Without emphasizing
the learning theory foundations, the effects of applying games to science
learning would be much more limited. On the other hand, Li and Tsai (2013)
reviewed empirical research articles regarding game-based science learning
published from 2000 to 2011 and found that most of the digital games were

PT
utilized to promote scientific knowledge/concept learning, while less than
one-third were implemented to facilitate the students’ problem-solving skills,

RI
and only a few studies have explored game-based science learning from the
aspects of scientific processes, affect, engagement, and socio-contextual
learning.

SC
Scientific knowledge/concept learning refers to acquiring or improving the
concepts of a specific science domain (e.g., physics, chemistry, biology). For

U
example, Hsu and Tsai (2013) incorporated self-explanation into a computer
game to facilitate primary school students’ acquisition of scientific concepts
AN
regarding light and shadow. Problem solving emphasizes learning to solve
problems from a cognitive perspective (e.g., understanding, characterizing,
representing, solving, reflecting, communicating, and reasoning). For example,
M

Hwang, Hung, and Chen (2014) used a peer assessment-based game to improve
students' learning achievements, motivations, and problem-solving skills in
D

a science course at an elementary school. Scientific processes refer to

performing scientific methods that include observing, explaining, predicting,
TE

investigating, interpreting, and concluding in an inquiry process. For example,

Spires, Rowe, Mott, and Lester (2011) asked middle grade students to solve a
EP

science mystery based on microbiology content, and the results indicated that
effective exploration of the hypothesis strategies within a science task could
predict science content learning and in-game performance.
C

To enhance the effects of game-based learning in science education, we

AC

used established learning theories to customize a digital game “Crayon Physics

Deluxe” (Kloonigames, 2014) with scaffold designs. This game requires
participants to solve physics puzzles with personal 2D drawings. Hmelo-Silver,
Duncan, and Chinn (2007) proposed that tasks such as quests or puzzles can be
used as a form of problem-based learning in gaming environments, particularly
when math and science tasks are involved. Therefore, Crayon Physics Deluxe
is by nature a problem-based learning environment in which students can
engage in lifelike situations and try to solve problems by using their experience
and newly learned concepts from previous episodes. Based on this game, we

4
 5
ACCEPTED MANUSCRIPT
attempted to extend the ways in which learning takes place through adding
external scaffolds that support students’ individual learning progress. During
game play, students have to actively construct possible approaches to solve the
puzzles by externalizing their physics concepts with drawings. Constructivism
advocates individuals being active constructors of knowledge and actively
creating their own representations of objective reality rather than passively
adopting it from external sources (Bodner, 1986). Our design for game-based

PT
learning fits the “problem-based learning” principle of the “constructivism”
learning theory (Li & Tsai, 2013).

RI
SC
1.2 Gaming scaffolds vs. learning scaffolds
Scaffolding is defined as the process by which a more knowledgeable peer
or teacher offers assistance that enables learners to solve tasks that would

U
otherwise be hard for them to complete independently (Wood, Bruner, & Ross,
1976). The concept of scaffolding comes from Vygotsky’s (1978) zone of
AN
proximal development (ZPD). Vygotsky believed that proper and temporary
guidance helps learners advance their learning within the ZPD. Palinscar and
M

Brown (1984) pointed out that scaffolding is an essential instructional tool or

strategy that can guide learners to develop their learning abilities. When the
scaffolding provided is systematic and matches the learners’ mental models,
D

new concepts can be more easily internalized and integrated into existing
TE

knowledge structures (Bruner, 1985). As learning occurs not only in traditional

classrooms but also in a digital space with software (computer-assisted learning)
or games (game-based learning), the concept of scaffolding has been extended
EP

to digital tools that support learners with the needed structure to confront
complex tasks (Quintana et al., 2004). Therefore, designing scaffolds to fit the
task properties as well as the tools and interfaces learners use is an important
C

issue (Soloway, Guzdial, & Hay, 1994).

AC

Scaffolding in games is used to bridge the gap between the player’s

current skills and those needed to be successful in the next game level. In
commercial games, various types of scaffolding are used to explain game rules,
explore game strategies, and familiarize players with the necessary skills to
move through the levels without too much frustration. When players get stuck,
appropriate scaffolding can make them feel more successful and motivate them
to continue playing (Chen & Law, 2016). Proper scaffolding provides a
satisfying game experience for players. Likewise, gameplay should be

5
 6
ACCEPTED MANUSCRIPT
supported with appropriate feedback and scaffolding to meet students’ learning
requirements so that they can complete the tasks and solve the problems
(Jabbar & Felicia, 2015). However, researchers (Barzilai & Blau, 2014; Sun,
Wang, & Chan, 2011) have proposed that scaffolding a game may involve
trade-offs between learning gains and students’ perceptions regarding the fun
of playing the game. Dependence on the scaffolding could lead them to use it
just to get enough information to move through the game without actually

PT
learning the material in such a way that it can be retained. It could also reduce
students’ ability to apply the knowledge and skills in a creative way. Fisch

RI
(2005) indicated that well-designed scaffolds should not only help players to
refine their gaming strategies, but should also increase the learning effects.
Therefore, when scaffolds are integrated into games, designers or instructors

SC
should consider whether they are to be used as tools to assist advancement
through the game levels, and thus function as “gaming scaffolds,” or even

U
better, to facilitate specific learning effects, thus functioning as “learning
scaffolds.”
AN
In addition, what kinds of scaffold types do learners need? And how much
assistance is appropriate? Various forms of scaffolding design for software or
M

gameplay might support learning in different ways (Jabbar & Felicia, 2015;
Quintana et al., 2004). According to Wood et al. (1976), proper scaffolding
D

supports problem-based learning in three ways. First, it maintains motivation

by reducing and controlling the difficulties and frustrations that learners might
TE

face. Second, it provides appropriate assistance to reduce uncertainty, and

guides the play toward a goal with greater clarity. Finally, it highlights critical
information or divides a complex problem so that learners can identify the
EP

required concepts they need to solve the tasks. In science, there are different
types of knowledge or skills to be learned which might require different
C

learning strategies or scaffolds. To assist students in the problem-based

learning process, providing a full example or just giving a few hints may have
AC

two diverse effects on their learning outcomes.

The debates on the effects of different instructional guidance remain

controversial. While some researchers argue that people learn best with
unguided or minimally guided instruction from the perspective of constructive
learning (e.g., Adams, Paulson, & Wieman, 2009; Chang, 2016; Steffe & Gale,
1995), other researchers suggest that students should be assisted with direct
instructional guidance on the concepts and procedures to reduce possible
cognitive load (e.g., Kirschner, Sweller, & Clark, 2006; Moreno, 2004). For

6
 7
ACCEPTED MANUSCRIPT
example, Adams et al. (2009) investigated what levels of guidance promote
engaged exploration with interactive simulations, and found that minimal but
nonzero guidance promotes optimum engaged exploration and learning in
physics simulations. Chang (2016) also reported that scaffolding using minimal
but critical guidance such as the driving question approach may benefit
students’ learning in terms of their cognitive engagement. On the other hand,
Moreno (2004) found that novice students’ cognitive load could be reduced and

PT
they could learn better from strongly guided feedback than from discovery.
Kirschner et al. (2006) also pointed out that minimal guidance during

RI
instruction is less effective and less efficient than guidance specifically
designed to support the cognitive processing necessary for learning. Therefore,
for the purpose of increasing novice students’ overall conceptual understanding

SC
on a particular subject matter, we assume that they should learn better with
direct guidance. Furthermore, most research on applying scaffolds in science

U
education has usually looked at conceptual understanding or learning efficiency
(e.g., Adams et al., 2009; Chang, 2016). Little research has been done to report
AN
the effects of various scaffolds on design creativity in science education. In the
domain of creativity research, direct guidance is usually regarded as being
harmful to creative thinking. For example, Belski and Belski (2015) pointed
M

out that the extensive professional experience and the possession of large
amounts of domain knowledge could have a detrimental effect on creativity
D

and impede problem solvers’ ability to generate creative ideas. Therefore, for
the purpose of increasing students’ potential creativity in designing products,
TE

we assume that they should perform better with minimal guidance.

Among the scaffold types proposed by Wood et al. (1976), we chose to

EP

focus on “Demonstration” and “Marking critical features,” which correspond to

direct guidance and minimal guidance, respectively. “Demonstration” offers
C

examples for learners to model or imitate the process, and thus helps them to
develop appropriate abilities. Students could learn the way experts undertake
AC

the task and try to understand the reasons behind those moves leading to the
solution. “Marking critical features” helps learners recognize the critical
features of a task so that they can obtain the essential information needed to
tackle it. Through this process, students learn to identify relationships between
the tasks and questions confronted. They also learn to better understand the
distance between their current abilities and the desired outcomes. For instance,
Sun et al. (2011) discussed how different scaffold types (frustration control and
demonstration) directed problem-solving behaviors in a digital Sudoku game in

7
 8
ACCEPTED MANUSCRIPT
different ways, and found that demonstration scaffolds mostly encouraged the
development of solving strategies.

1.3 Assessment of science learning outcomes

In Li and Tsai’s (2013) review of relevant research for game-based
learning in science education, they found a gap between what students may

PT
have learned from the games and their learning outcomes which were evaluated.
For example, game environments could allow students to experience a variety

RI
of learning (e.g., enhancing their scientific literacy and problem solving ability),
but most of the assessments of learning outcomes were limited to investigating

SC
scientific concepts. In addition, this evaluation was mostly undertaken by using
standard or self-developed questionnaires, while the evaluation of
problem-solving ability was assessed by open-ended questions, field

U
observations, or interview findings.
AN
Firstly, an alternative to measuring scientific knowledge/concepts is to use
a concept map. Concept maps, developed by Novak and Gowin (1984), are
diagrams in which nodes represent concepts, and links between nodes represent
M

cognitive relationships. Concept maps externalize the conceptual structure of a

content area and are commonly used to visually show learners’ understanding
D

and organization of a learning topic. First applied in science education, concept

maps can facilitate logical thinking and global comprehension of content. A
TE

concept map can also be used to help identify learners’ misconceptions (Akbaş
& Gençtürk, 2011; Palmer, 2003; Reinfried, 2006). In addition, concept maps
EP

can be used as an evaluation tool to compare student concept maps with expert
concept maps to determine the students’ levels of conceptual understanding
(Kao, Lin, & Sun, 2008).
C

Secondly, one potential approach to measuring problem-solving ability is

AC

through game design. Engaging in designing activities is conducive to

self-regulated learning, and can increase motivation (De Jong & Pieters, 2006;
Kafai & Resnick, 1996). In particular, game design is regarded as an ideal
context to increase children’s conceptual understanding and motivation as well
as to develop their problem-solving skills (Akcaoglu, 2014; Baytak & Land,
2010; Prensky, 2008). Game design activities could be used in the process to
benefit the learning effects, or could be applied at the end of experiencing the
game to evaluate the students’ learning outcomes. Baytak and Land (2010)
investigated how children created computer games to represent their

8
 9
ACCEPTED MANUSCRIPT
understanding of nutrition, while Akcaoglu (2014) found that students'
problem-solving skills were significantly improved after attending a
game-design summer program. Evaluations of those game designs could
provide clues as to the effectiveness of including learners in the design of their
own learning environment.

However, Li and Tsai (2013) reported that the majority of the reviewed

PT
studies in science education adopted the game-playing feature, while only two
required students to design their own digital games to facilitate the learning
effects. For example, Li (2010) analyzed students’ experiences and responses

RI
as they created computer games to teach the concept of Isaac Newton’s Three
Laws of Motion. They found that student engagement in the game-building

SC
process could enhance not only their understanding of the subject matter, but
also their general problem-solving abilities. Meanwhile, they identified that
creativity, engagement and new identity were three salient traits demonstrated

U
by the students when learning with digital game-building. Therefore,
AN
investigating game-making outcomes for game-based science learning is an
important issue that deserves extensive attention. A good quality game-making
outcome for a specific science topic represents a comprehensive understanding
M

of the problem being studied, and demonstrates the potential ability for
problem-solving, and even creativity in designing new game scenarios in that
D

subject matter.
TE

1.4 Creativity in science education

EP

Creativity is the ability to create works that have both originality and
effectiveness (Sternberg & Lubart, 1996, 1999). Originality is often labeled as
C

unusual, novel, unique, or unexpected, while effectiveness takes the form of

adaptivity, usefulness, or appropriateness. In an attempt to provide a unitary
AC

definition for creativity, Rhode (1961) proposed a scheme of creative

dimensions known as the 4 P’s: person, process, product, and press (i.e.,
environmental influences). In the past decades, researchers have chosen to
explore different creative dimensions. For example, researchers who focus on
“process” are inclined to view creativity as a process of creative problem
solving. During the process of problem solving, individuals have the possibility
to demonstrate creative solutions to problems by understanding, representing,
solving, reflecting, and reasoning in a unique way. Parnes (1967) proposed the
concept of Creative Problem Solving (CPS), which is a high-order cognitive

9
 10
ACCEPTED MANUSCRIPT
ability including the thinking mechanisms of cognition, memory, divergent
thinking, convergent thinking, and evaluation (Parnes, Noller, & Biondi, 1977).
On the other hand, many researchers conceptualize creativity as a creative
“product” by looking at the outcomes of creative thoughts and evaluating
whether the features of the product could be subjectively judged by experts as
being creative (Amabile, 1983; Amabile & Tighe, 1993; Shieh & Chang, 2014;
Siew, Chong, & Lee, 2015). For example, Shieh and Chang (2014) asked

PT
junior-high school students to create functional boats from scratch to observe
how they resolved the problems encountered during the process, and found that

RI
the experience of designing scientific artefacts does in fact enhance students’
creative skills and problem-solving abilities. Guilford (1977) proposed
sensitivity, fluency, flexibility, originality, and elaboration as the criteria of

SC
creativity assessment, which are commonly adopted by most researchers in
evaluating product creativity. In Siew, Chong, and Lee’s (2015) study, they

U
attempted to improve fifth graders’ scientific creativity through problem-based
learning, and found that students’ performance in each product dimension of
AN
scientific creativity improved in terms of fluency, flexibility, and originality.

Creativity and imagination are the main sources of scientific knowledge

M

(Aydeniz & Bilican, 2014; Urhahne, Kremer & Mayer, 2010). As a critical
domain of the nature of science, creativity is essential to science education
D

(Quigley, Pongsanon, & Akerson, 2010; Yager, Dogan, Hacieminoglu, &

Yager, 2012). In science education, creativity involves not only generating
TE

something new, but is an essential element in improving students’ thinking and

problem-solving abilities (Cremin, Glauert, Craft, Compton, & Styliandou,
2015; Plucker, Beghetto, & Dow, 2004), or assisting them to confront problems
EP

yet to occur (Eckhoff & Urbach, 2008). Besides, learners’ attitudes toward
science could be improved with creative activities, thus promoting their
C

endeavors to learn science (Hendrix, Eick, & Shannon, 2012; Yager et al.,
2012). However, Al-Abdali and Al-Balushi’s (2016) study indicated that most
AC

science teachers follow cookbook-style activities, and believe that they are not
prepared or do not have enough time to teach creativity in their classes.
Therefore, many more efforts should be put into encouraging students’
creativity in science education.

1.5 Research questions

Based on the above literature survey and discussion, we propose the

10
 11
ACCEPTED MANUSCRIPT
following research questions:
1. What are the effects of digital games on learning, particularly in the
domain of physics?
2. How does scaffolding increase the effectiveness of the digital game for
student conceptual learning outcomes?
3. How does scaffolding increase the potential of the digital game for
student creativity in designing new game episodes?

PT
4. How does adding specific types of scaffolding fulfill specific
instructional purposes? Which types of scaffolding are more effective?

RI
2. This Study

SC
2.1 Study purpose

U
This study intentionally used a digital educational game, Crayon Physics
Deluxe (CPD) (Kloonigames, 2014), to explore the effects of injecting
AN
self-design scaffolds into students’ learning outcomes. The learning outcomes
which this study sought to address are: knowledge acquisition and design
creativity. The game was tailored to determine if learners could acquire better
M

physics knowledge (in the form of concept maps) and show good design
creativity (in the form of game episode designs) based on the type of
D

scaffolding provided in the game. It also aimed to determine whether the use of
scaffolding can extend learners' thinking beyond the original concept
TE

boundaries, or encourage flexible thinking.

EP

2.2 Conceptual framework

C

During the process of playing Crayon Physics Deluxe (CPD), the

AC

participants were required to actively construct possible approaches to solve

puzzles by externalizing their physics concepts with drawings. This game thus
provides a potential environment for problem-based learning in physics
education. To facilitate the learning effects of this digital game, we
incorporated different scaffold designs to assist the students. As shown in
Figure 1, this study chose to compare the effects of “Demonstration” and
“Marking critical features” scaffolding on students’ overall understanding of
basic machinery in physics, and their potential creativity in designing new
physics puzzles for Crayon Physics Deluxe, because different levels of

11
 12
ACCEPTED MANUSCRIPT
guidance or scaffolding might result in different learning outcomes (Please
refer to Section 1.2; Adams et al., 2009; Chang, 2016; Kirschner et al., 2006;
Moreno, 2004; Steffe & Gale, 1995). Demonstration, in which procedures are
modeled so that learners can imitate them, and marking critical features, where
learners are provided with the critical information they need to complete the
assigned task, provide us with the opportunity to observe the effects of two
diverse forms of instructional guidance (direct vs. minimal) on learning

PT
outcomes. Little research has compared the effectiveness of the demonstration
versus marking critical features scaffolding in the domain of game-based

RI
science learning. In particular, previous research on applying scaffolds in
science education mostly looked at conceptual understanding and learning
efficiency (e.g., Adams et al., 2009; Chang, 2016). To the best of our

SC
knowledge, there has been no empirical study reporting the effects of various
scaffolds on design creativity in science education. Neither theoretical

U
perspectives nor empirical results provide sufficient information to show which
kinds of scaffolds are effective in different aspects of learning outcomes for
AN
game-based science learning.
M

Insert Figure 1 about here

D

In this study, the students had two different opportunities to demonstrate

TE

their creativity, one while they were playing the Crayon Physics Deluxe, and
the other while they were designing their own game episodes. Our study design
EP

fits the “problem-based learning” principle of the “constructivism” learning

theory. Students could come up with creative approaches to solve the puzzles
in the game playing session while also having the chance to demonstrate
C

creativity in the game episode designing session. We purposely chose to focus

AC

on the latter one, design creativity, since this aspect requires a greater capacity.
To design a game episode for Crayon Physics Deluxe, the students had to first
comprehend the core of the physics principle in basic machinery, and then try
to transform their roles from problem solvers to problem designers. Through
the process of designing products (game episodes), they could not only
demonstrate their problem-solving abilities, but also develop more advanced
problem-designing abilities and even creativity potential in designing scientific
puzzles. This attempt to explore design creativity in game-based learning for
science education is critical, but has rarely been investigated. To assess game

12
 13
ACCEPTED MANUSCRIPT
design creativity, we adopted sensitivity, fluency, flexibility, originality and
elaboration (Baer, 1993; Guilford, 1968, 1977) as the five assessment criteria,
which encompass the evaluation for problem solving ability and creativity in
designing puzzles (please refer to Table 4).

2.3 Research hypotheses

PT
Based on the literature survey in Section 1, we propose the following
research hypotheses:

RI
The first evaluation required learners to observe the tasks and environment
of the game, and then to generate a concept map of their physics knowledge to

SC
measure the bottom-up processing of knowledge construction. When novice
learners watch the demonstration, they might gain a full understanding of how

U
experts solve problems and advance to another level of the game (e.g.,
Kirschner et al., 2006). On the other hand, novice learners who are simply
AN
presented with critical features of the game may only acquire some fragmental
concepts. We therefore hypothesized that learners would achieve better physics
knowledge acquisition in terms of concept map construction in the
M

demonstration condition than in the marking critical features condition.

D

The second evaluation requested participants to design new game episodes

as creative products after they had navigated the game in the different
TE

scaffolding conditions. From the game playing process, the participants

familiarized themselves with the game content and acquired a certain degree of
knowledge of physics machinery. The learners then created game puzzles that
EP

could be solved by applying appropriate physics concepts. When learners

watch a demonstration, a lengthy video might provide too much information,
C

resulting in poor concentration or limited room for imagination (e.g., Belski &
Belski, 2015). On the other hand, learners who are only given information on
AC

the critical features of the game might use that partial information to flexibly
create game episodes. We therefore hypothesized that learners in the marking
critical features condition would design episodes that were more creative than
those in the demonstration condition.

The hypotheses were:

(1) Learners in the three groups using the game (demonstration,

13
 14
ACCEPTED MANUSCRIPT
non-scaffolding, and marking critical features groups) would perform
significantly better in the post-test construction of concept maps than the
non-game group learners.

(2) Learners in the two scaffolding groups (demonstration and marking critical
features groups) would perform significantly better in concept map
construction than those in the non-scaffolding group.

PT
(3) Learners in the demonstration group would demonstrate more physics
knowledge acquisition when constructing concept maps than the marking

RI
critical features group.

(4) For the design creativity (sensitivity, fluency, flexibility, originality and

SC
elaboration) of the game episodes, the scaffolding groups (demonstration
and marking critical features group) would perform significantly better than

U
the non-scaffolding group. AN
(5) Learners in the marking critical features group would produce more creative
designs in terms of sensitivity, fluency, flexibility, originality and
elaboration than those in the demonstration group.
M
D

3. Method
Since it is easier to implant a quasi-experiment design in a real classroom
TE

setting than a randomized experiment, and as this design could still be used to
compare the pre- and post-performances of either multiple groups or multiple
EP

waves of measurement, a quasi-experiment was designed to examine the effect

of different types of scaffolding on two different aspects of learning outcomes:
knowledge acquisition (in the form of concept maps) and design creativity (in
C

the form of game episode designs). Four classes of participants were divided
AC

into four groups, three experimental groups and one control group. The three
experimental groups played the Crayon Physics Deluxe game; one group
received demonstration scaffolding; one received the game with no scaffolding;
and the third group received marking critical features scaffolding. The fourth
group, the control group, did not play the game.

3.1 Participants

14
 15
ACCEPTED MANUSCRIPT
From a junior high school in northern Taiwan, 126 participants in four
classes were selected to participate as a class in one of the four group
conditions: demonstration scaffolding (N=31), non-scaffolding (N=30),
marking critical features scaffolding (N=33), and the non-game (N=32) groups.
Students in this school are normally distributed into classes according to their
achievements, meaning that the academic performance for these groups was
similar. Students were not aware of the real research purpose of this experiment

PT
and did not know that there were other groups (classes) participating in this
activity. During the activity, students were required not to talk or discuss the

RI
activity with others. Excluding missing values due to absence or incomplete
data, the valid number of participants in some of the statistical analyses varies
slightly. Students were not informed of the study purpose or other group

SC
conditions in order to prevent interference.

3.2 Procedure
U
AN
The entire task was divided into sessions that spanned approximately two
months. The procedure along with the time allotted for each session is detailed
M

in Figure 2. One science teacher was in charge of teaching the basic concepts
required for the experiment to all four groups. In stage 1, all participants were
D

first introduced to the concept mapping technique and then completed a pre-test
by drawing a concept map to show their original understanding of “basic
TE

machinery” in Physics. In stage 2, the three experimental groups were

introduced to the Crayon Physics Deluxe game and instructed on how to play it
EP

within the assigned condition (demonstration, non-scaffolding, and marking

critical features scaffolding). The control group did not play the game; instead,
they received traditional teaching instruction on the same topics. To monitor
C

whether the learners overused the scaffolds, each participant was given a paper
AC

card to record the scaffolds used. In addition, a screen recorder was used to
track the process and the time spent on gameplay. In stage 3, a post-test
involving concept map constructions was administered to measure all
participants’ comprehension regarding basic machinery. These concept maps
allowed the researchers to evaluate how well the learners combined their
existing physics knowledge with the newly learned knowledge.

In stage 4, the three experimental groups were asked to create their own
personal game episodes to demonstrate their levels of design creativity. During
the process, they were not allowed to discuss the process or their ideas with

15
 16
ACCEPTED MANUSCRIPT
their peers. The game episode designs were undertaken in two steps. First,
students were asked to use the functions provided in CPD to design episodes
directly in the game, and they could design several in-game episodes if they
wanted. Second, students were asked to choose the best design to draw on the
game episode design (GED) sheets (Table 3). On the GED sheets, the students
had to specify the scenarios and their problem-solving tactics in detail. Students
have to firstly fully understand the core of the problem, and then try to

PT
transform their roles from problem solvers (playing game episodes) to problem
designers (designing game episodes). Through the process of designing game

RI
episodes, the students could not only demonstrate their problem-solving
abilities, but could also develop more advanced problem-designing abilities and
even creativity in designing new game episodes. Finally, both the pre- and

SC
post-test concept maps and the game episode designs were scored by the
experts. Separate rubrics (measurement guidelines) were provided for the

U
concept maps and the game episode designs, as shown in Sections 3.5 and 3.6.
AN
Insert Figure 2 about here
M
D

3.3 The educational game “Crayon Physics Deluxe”

TE

Crayon Physics Deluxe (Kloonigames, 2014) is a stand-alone commercial

digital game in which participants solve physics puzzles with creative 2D
drawings. This game not only allows players to solve the game puzzles, but
EP

also to apply their physics concepts to create their own game designs.
Therefore, this game allows students to learn physics concepts from the
perspective of “problem-based learning” as well as to demonstrate possible
C

creativity in game design. Since the physics concepts concealed in the game
AC

content might not be explicit enough for students to connect with the curricular
knowledge, and students might get stuck during game playing and thus lose the
opportunity to benefit from the game-based learning, we embedded two chosen
scaffolds into this game. Other alternative games include “Algodoo” and
“Magic pen,” which also allow players to play around in a cartoon-style 2D
physics sandbox.

Three junior high school science teachers with at least 5 years of

experience teaching “nature and living technology” selected which game

16
 17
ACCEPTED MANUSCRIPT
episodes from CPD the students would play to cover the targeted physical
concepts. Out of 81 CPD difficulty levels, 19 were selected, covering basic
physics/machinery principles, such as rotation, levers, gears, fulcrums, inclined
planes, and momentum.

To pass each game level, the player moves a red ball across the
valley/canyon to the star by drawing objects and using tactics involving

PT
physical concepts (Figure 3). The drawings were automatically transformed
into real physical objects in an attempt to solve the puzzle. During this process,
the students’ flexibility in applying physics concepts to solve the puzzle can be

RI
fully demonstrated. A participant can try any possibility to pass a level,
connecting old experiences with new ones. There may be times when a player

SC
tries several ideas and spends a considerable amount of time learning the
concepts needed before realizing the solution to a problem. One player may use
a routine and ineffective way to solve a puzzle, while another may compose a

U
clever device involving a chain reaction as their solution. As a student moves
AN
from one level to another, experience is accumulated and could be used in
subsequent episodes.
M

In addition, numerous functions are provided for designing game episodes

(Table 1).
D
TE

Insert Figure 3 about here

Insert Table 1 about here

C EP

3.4 Embedded scaffolding

AC

The same three teachers who selected the game episodes also assisted in
the process of embedding scaffolds into the game. CPD is a problem-based
learning environment that contains puzzles of different levels of difficulty.
Each puzzle presents a meaningful and lifelike situation rather than a
cookbook-style problem, and requires the players to recognize critical objects
to pass each game level. Students might get stuck due to lack of experience or
insufficient concepts or skills. Therefore, we added external scaffolds to
motivate them to continue playing, and also in an attempt to facilitate specific
learning effects. We hoped that these scaffolds could function as “gaming

17
 18
ACCEPTED MANUSCRIPT
scaffolds” to help the students pass through the game levels without too much
frustration, and also to act as “learning scaffolds” that would naturally guide
them to construct their own understanding or develop their own creative ideas
in the game design. The focus of this study was therefore on comparing how
the “demonstration” and “marking critical features” scaffolding proposed by
Wood et al. (1976) would impact the students’ physics knowledge acquisition
and game episode design creativity, as it has been argued that different levels

PT
of guidance or scaffolding might have diverse effects on learning outcomes
(please refer to Section 1.2; Adams et al., 2009; Chang, 2016; Kirschner et al.,

RI
2006; Moreno, 2004; Steffe & Gale, 1995).

For demonstration scaffolding (Figure 4), participants could view videos

SC
with instructions and details for solving the puzzles whenever they had
difficulty progressing to the next level. In the videos, the three experts gave
clear instructions and detailed the procedures that the students would need to

U
advance to the next level. They could watch as many or as few of the videos as
AN
they needed. Although they were likely to use the demonstrated procedure to
solve the puzzles, some students could push themselves to come up with their
own ideas.
M

For the marking critical features scaffolding (Figure 5), participants were
given clues that showed them key concepts to help them advance to the next
D

level. While CPD already had some embedded clues, additional clues were
TE

generated by the three experts and included in the game. If the students got
stuck on a certain level, they could use the screenshots that had been added to
the game. For example, the clues in these screenshots included illustrations of
EP

physics concepts such as levers, pivot points, and pulleys, as well as

step-by-step instructions on how to use certain objects to advance to the next
level. The participants could use the clues provided in the marking critical
C

features scaffolding to assist them in understanding the key points they needed
AC

to solve the game puzzles.

Insert Figure 4 about here

Insert Figure 5 about here

3.5 Concept map assessment

18
 19
ACCEPTED MANUSCRIPT
Concept maps, developed by Novak and Gowin (1984), were used for the
measurement of conceptual understanding. The maps were scored according to
six criteria: propositions, hierarchies, cross-links and examples (Novak &
Gowin, 1984) and branching and extra propositions (Markham, Mintzes &
Jones, 1994; Moreira, 1979). A student’s map on basic machinery in Physics is
shown in Figure 6 (the concept map content was translated into English). Table
2 illustrates how to score the map based on these six criteria.

PT
RI
Insert Figure 6 about here

Insert Table 2 about here

U SC
One expert in science education constructed his concept map on the same
topic—machinery—to be used as a standard. By using the concept map
AN
measurement, the score of the expert map was derived. Likewise, two rounds
of each participant’s concept maps were scored using the same criteria. To
M

obtain an indicator showing a student’s mastery of the learning content, we

used the formula:
D

（score of participant’s concept map）

TE

x 100 ＝ indicator of the level of knowledge

（score of the standard concept map）
EP

For example, if participant A scored 70 on his concept map after playing

the game and the score of the expert’s concept map was 76, a value of 92.1 was
C

calculated to represent the participant’s knowledge acquisition. Note that it was

possible for a participant’s concept map to obtain a higher score than the
AC

expert’s map, in which case, the calculated value would be higher than 100.

3.6 Game design creativity assessment

The Consensual Assessment Technique (CAT) (Amabile, 1983) was used

to evaluate participant creativity in designing game episodes. The CAT is often
used to assess creativity products such as artwork, poems, and compositions.
This product-based assessment requires a strict scoring procedure and experts’

19
 20
ACCEPTED MANUSCRIPT
experience in assessments. Experts are given clear rules they should follow
during the evaluation. For example, they should hold the same fair attitude to
assess each product during the evaluation, and their ratings should distinguish
products to different quality levels. Experts are not allowed to interact during
the process, and products are presented in a different order to each expert to
prevent fatigue or presentation order issues. Ratings have to reach stable
consensus so as to produce a satisfactory inter-rater reliability that shows

PT
consistency.

Three experts with 4-8 years of experience teaching “drawing and art

RI
design” were trained on how to operate the Consensual Assessment Technique
(CAT) and use it to assess the designs on the GED sheets (see Table 3).

SC
Designs were scored based on the five dimensions of creativity assessment:
sensitivity, fluency, flexibility, originality and elaboration (Baer, 1993; Guilford,
1968, 1977). One science education teacher was responsible for providing the

U
scoring criteria, training the three expert raters and making sure they had the
AN
same background knowledge required to perform the assessment. Table 4
provides the definitions and application of these dimensions.
M

Insert Table 3 about here

D

Insert Table 4 about here

TE
EP

4. Results
Descriptive data and statistical analyses are provided in this section. We
used statistical techniques to control extraneous variables by running Analysis
C

of Covariance (ANCOVA) to remove covariates from the list of possible

AC

explanations of variance in dependent variables. In this study, we used the

pre-test score of the concept map as the covariate in the corresponding tests. In
addition, since there are five correlated dependent variables (sensitivity, fluency,
flexibility, originality and elaboration) for game episode design creativity, we
applied a multivariate analysis of variance (MANOVA) to yield an overall
statistical result on this set of variables instead of performing multiple
individual tests.

20
 21
ACCEPTED MANUSCRIPT
4.1 Frequency of scaffolds used and time spent to advance through the game

In this study, the participants were required to solve 19 game levels in total.
Each group used the scaffolding approximately three times (Demonstration: M
= 3.10; Marking critical features: M = 3.77). It is possible that the participants
were able to apply earlier clues and strategies to solve puzzles in later levels.
The average time (in seconds) spent playing the game was: demonstration

PT
scaffolding group (M = 830.14), non-scaffolding (M = 1,212.53), and marking
critical feature groups (M = 1,085.33).

RI
4.2 Effect of playing Crayon Physics Deluxe on physics knowledge acquisition

SC
The scores of the participants’ concept maps ranged from 0 to 100. In
order to examine the effect of the educational games on physics learning, an

U
ANCOVA was performed. The post-test scores of the concept maps were
AN
used as the dependent variable, and the pre-test scores of the concept maps
were used as the covariate. Whether or not the student played the game
(yes/no) was used as an independent variable in the ANCOVA model. As seen
M

in Table 5, there was no interaction between the game playing (yes/no)

condition (F = .89, p = .35), indicating that the assumption of homogeneity of
D

the regression slopes was met. Levene's test of variance homogeneity was also
met (p > .05). The effect sizes for eta-squared were classified into small (>.01),
TE

medium (>.06) and large (>.14) (Cohen, 1988). The ANCOVA results show
that there was a significant difference with a large effect size (F(1, 100) =
22.00, p < .001, ηp2 = .18) on knowledge acquisition. Participants who
EP

experienced CPD (M = 52.55) scored significantly higher on their concept

maps than those who did not play the game (M = 49.59).
C
AC

Insert Table 5 about here

4.3 Effect of scaffolds on physics knowledge acquisition

Another ANCOVA was performed to determine whether there was a

difference in the concept map scores among the three game-playing groups: the
demonstration scaffolding, non-scaffolding, and marking critical features

21
 22
ACCEPTED MANUSCRIPT
scaffolding groups. While the pre-test scores of the concept maps were
controlled for, the scaffolding groups were used as an independent variable,
and the post-test scores of the concept maps were used as the dependent
variable. No interaction among the three groups was found (F = .10, p = 0.91),
indicating that the assumption of homogeneity of the regression slopes was met.
Levene's test of variance homogeneity was also met (p > .05). The ANCOVA
results showed that there was a significant difference with a medium effect size

PT
among the three scaffolding groups (F(2, 73) = 4.07, p < .05, ηp2 = .1), as
shown in Table 6. Post-hoc comparisons show that the marking critical features

RI
scaffolding group (M = 52.64) scored significantly higher than both the
demonstration (M = 50.53, p＜.05) and the non-scaffolding groups (M = 50.90,
p ＜ .05). There was no significant difference between the demonstration

SC
scaffolding and the non-scaffolding groups (p > .05).

U
Insert Table 6 about here
AN
M

4.4 Effect of scaffolds on game episode design

Before evaluating the game episode designs on the five dimensions of

D

creativity, the ratings of the three experts were examined for inter-rater
TE

reliability using Kendall’s coefficient of concordance. The inter-rater reliability

refers to the extent to which all the categorizations coincide. If the inter-rater
reliability is high, then the categorization would not be affected by a significant
EP

rater factor. The results were .65 for sensitivity, .73 for fluency, .75 for
flexibility, .54 for originality and .51 for elaboration (p < .001), indicating
moderate to substantial inter-judge reliability (Kendall & Gibbons, 1990).
C
AC

To determine the effect of the three types of scaffolding (the

demonstration scaffolding, non-scaffolding, and marking critical features
scaffolding) on the five dimensions of creativity for the game episode designs,
a MANOVA was performed. Levene's test of variance homogeneity was met (p
> .05). As seen in Table 7, there is a significant difference with a medium
effect size among the three scaffolding conditions on the overall creativity
performance (Wilk’s Lambda = .78, F(10, 168) = 2.20, p < .05, ηp2 = .12).
Follow-up ANOVAs on each dimension were employed to identify variables
that maximized differences among the scaffolding groups. Sensitivity (F = 3.69,

22
 23
ACCEPTED MANUSCRIPT
p < .05, ηp2=.08) and flexibility (F = 3.26, p < .05, ηp2=.07) contributed to the
significant multivariate effects; however, fluency (F = .87, p > .05, ηp2=.02),
originality (F = 1.38, p > .05, ηp2=.03), and elaboration (F = .99, p > .05,
ηp2=.02) did not.

Post-hoc comparisons showed that the marking critical features

scaffolding group (M = 2.71) had significantly higher levels of sensitivity than

PT
the non-scaffolding group (M = 2.18), while the demonstration scaffolding
condition (M = 3.65) reported higher levels of flexibility than the
non-scaffolding condition (M = 3.03).

RI
SC
Insert Table 7 about here

5. Discussion
U
AN
5.1 Effect of playing Crayon Physics Deluxe on physics knowledge acquisition
M

We hypothesized that learners in the three groups using the game

(demonstration, non-scaffolding, and marking critical features groups) would
D

perform significantly better in the post-test construction of concept maps than

those in the non-game group.
TE

Participants in the three groups who played CPD (M = 52.55) scored

significantly higher on constructing the concept map than those who did not
EP

play the game (M = 49.59). These results suggest that this well-designed
educational game might be more beneficial than the no-game condition in
terms of promoting learning physics concepts. The learning benefits might be
C

attributable to the context of CPD in which game episodes encourage

AC

connections between existing and new experience/knowledge, making

learning more meaningful and concrete. This implies that educational games
on a related knowledge domain have the potential to enhance learning
outcomes by helping learners associate the strategies required for passing
through the game with the physics concepts they have learned.

5.2 Effect of scaffolds on physics knowledge acquisition

We further hypothesized that learners in the two scaffolding groups (the

demonstration and marking critical features groups) would perform

23
 24
ACCEPTED MANUSCRIPT
significantly better on concept map construction than the non-scaffolding group.
In particular, learners in the demonstration group would demonstrate more
physics knowledge acquisition when constructing concept maps than the
marking critical features group.

Post-hoc comparisons showed that the marking critical features

scaffolding group (M = 52.64) scored significantly higher than both the

PT
demonstration (M = 50.53, p＜.05) and the non-scaffolding groups (M = 50.90,
p ＜ .05). There was no significant difference between the demonstration
scaffolding and the non-scaffolding groups (p > .05).

RI
Contrary to our expectations, the concept map scores were higher for

SC
students in the marking critical features group than in the demonstration group.
One possible explanation could be that the participants in the marking critical
features group were only provided with certain critical clues, so they had to

U
figure out the ways to use those clues, discover the relationships among the
concepts, and try to connect resources together to solve the puzzle. The process
AN
of solving the puzzle largely relied on figuring out the relationship between
those clues, available objects, and physical principles on their own, which is in
M

accordance with the process of concept mapping (Novak & Cañas, 2007) and
the theory of constructivism. Hence, learners in the marking critical features
condition could effectively perform better in knowledge acquisition in terms of
D

concept map construction. This finding is in accordance with Adams et al.’s

TE

(2009) observation that students could better explore the physics simulations
under minimal guidance where they gained physical insights into the
phenomena via their own questioning and constructing of scientific concepts.
EP

However, the learners in the demonstration scaffolding group saw a full

demonstration of how to pass through the game, and may not have been
prompted to think about the concepts or physics principles underlying the
C

solution. Therefore, they might have tended to directly use the solution
AC

provided without actively thinking about or constructing the relationships

between concepts.

5.3 Effect of scaffolds on game episode design

In terms of creativity in the design of game episodes, we hypothesized that

the scaffolding groups (the demonstration and marking critical features group)
would perform significantly better than the non-scaffolding group. In particular,
we expected that learners in the marking critical features group would produce
more creative designs than those in the demonstration group. In this study, a

24
 25
ACCEPTED MANUSCRIPT
good quality game episode design represents a comprehensive understanding of
the problem being studied, and demonstrates the potential ability for
problem-solving and even creativity in designing new game scenarios for the
Crayon Physics Deluxe game.
Post-hoc comparisons show that the marking critical features scaffolding
group (M = 2.71) had significantly higher levels of sensitivity than the
non-scaffolding group (M = 2.18). This finding suggests that the marking

PT
critical features scaffolding effectively helped the participants to better
understand how to apply key objects to solve the puzzles and navigate through

RI
the game. Since the participants were only given critical clues, they still had to
think about how to apply them, which may have trained them to think more
deliberately about the game playing tactics. Therefore, their game designs had

SC
better feasibility and were more likely to be successfully executed.

Post-hoc comparisons also indicated that the demonstration scaffolding

U
condition (M = 3.65) reported higher levels of flexibility than the
AN
non-scaffolding condition (M = 3.03). The clues provided in the demonstration
condition effectively used a wide range of knowledge and tactics to complete
the game episodes, potentially boosting the participants’ adaptability to change
M

and their ability to think flexibly. Hence, learners in the demonstration

condition could perform significantly better in terms of flexibility in their
D

creative game episode designs than the non-scaffolding group. Meanwhile, it

should be noted that flexibility in the demonstration condition (M = 3.65) was
TE

also higher than that in the marking critical features condition (M = 3.18),
although not at a significant level. This finding is a little contrary to our
expectations; we assumed that learners would exhibit better overall design
EP

creativity with minimal guidance (the marking critical features scaffolding)

rather than with direct guidance (the demonstration scaffolding). This
C

phenomenon suggests that different kinds of scaffolds might be required to

improve different dimensions of design creativity; a single kind of scaffolding
AC

was insufficient for bringing out all of their creative potential; this preliminary
result is worth exploring in future research.

There was no significant difference in fluency among the three scaffolding

conditions. One possible explanation might be that the amount of time required
to try out ideas for a new game episode design and work on revisions was
limited, which might have led to fewer game episode designs being produced.
There was also no significant difference in originality among the three
scaffolding conditions. This finding suggests that originality cannot be

25
 26
ACCEPTED MANUSCRIPT
developed in a short period of time and that it may take longer for the effects of
scaffolding to come into play. This finding is in line with Yin (2008), who
described originality as an extraordinary way of thinking, which might be
cultivated rather than learned. Finally, there was no significant difference in
elaboration among the three scaffolding groups. One explanation is that the
participants tended to focus on coming up with a workable design, but did not
spend much time elaborating on their designs.

PT
5.4 Design knowledge for game-based scaffolds

RI
Successful commercial digital games embed proper scaffolding
functionality to keep the players engaged. However, commercial digital games

SC
or even educational games cannot meet all specific teaching purposes. In
practice, it is also not cost-effective to design a new game from the very
beginning to include sufficient scaffolding to meet all possible educational

U
objectives. As with the findings of Barzilai and Blau (2014), external
scaffolding is effective for maximizing learning effects after enabling students
AN
to experience the game. Individual instructors could include additional
scaffolding in their game-based learning scenarios or activities in an attempt to
M

achieve their personal teaching objectives. Such a feature could be significant

in supporting learning in formal educational settings.
D

This study demonstrates that teachers themselves could tailor a digital

game, such as Crayon Physics Deluxe, to fulfill personal instructional goals by
TE

transforming certain default game assistant features into learning scaffolds, or

injecting self-designed scaffolds into the game (Figure 7). Therefore, teachers
EP

could focus their attention on designing proper learning scaffolds to maximize

the learning effects of the current available digital games without developing an
educational game from the beginning. Teachers should first search for digital
C

games which have the potential to introduce relevant curricular concepts

AC

related to their personal instructional purposes. Then, they should embed

proper scaffold designs according to the game features and the computer tools
they are familiar with to create learning scaffolds.

Insert Figure 7 about here

Furthermore, our findings provide empirical results indicating that

26
 27
ACCEPTED MANUSCRIPT
knowledge acquisition and different perspectives of creativity might require the
assistance of various types of scaffolds. Marking critical features scaffolding
was effective in helping the participants connect the new concepts with their
existing experience to construct their own understanding of basic machinery in
Physics, and in cultivating the sensitivity dimension of design creativity, while
demonstration scaffolding was effective in facilitating the flexibility dimension
of design creativity. When science teachers or future researchers attempt to

PT
design their own game-based scaffolds to facilitate learning effects, they should
pay special attention to which kinds of knowledge or ability they would like

RI
their students to cultivate.

SC
6. Conclusion

U
Most educational games only provide “gaming scaffolds” for students to
pass through the game levels without too much difficulty and frustration, with
AN
little emphasis on providing “learning scaffolds” that naturally guide students
through various levels of learning complexity. In addition, studies on
game-based science learning have usually only adopted questionnaires to
M

evaluate scientific concepts, and have used interviews or observations to assess

problem-solving ability, neglecting the importance of investigating
D

game-making outcomes (Li & Tsai, 2013). This study attempted to embed two
types of scaffolds (demonstration and marking critical features) to see if they
TE

could work as learning scaffolds to facilitate physics learning, or at least work

as gaming scaffolds to move the player through the game. Most importantly,
EP

the outcomes of science learning in this study were measured with concept
maps to determine the mastery level of the scientific concepts in the physics
machinery as well as measuring the game episode designs to determine the
C

students’ problem-solving ability and even creativity in designing new game

AC

scenarios. The students were assisted with our scaffolding design to solve the
game puzzles during the game playing session in which they were naturally
engaged in a problem-solving learning environment. They were then asked to
transform their roles from problem solvers to problem designers during the
game episode design session.

Our results showed that, in general, those students who experienced CPD
(with or without scaffolding) performed better in terms of knowledge
acquisition as measured by concept mapping. In particular, we found that
marking critical feature prompts and clues were effective in helping the

27
 28
ACCEPTED MANUSCRIPT
participants connect the new concepts with their existing experience. They
needed to figure out ways to use those clues, determine the relationships
between the concepts, and try to connect resources together to solve the puzzles.
By constructing their understanding of basic machinery in Physics through the
use of these clues to pass the game levels, they gained a better understanding of
the physics concepts, and performed better on the concept mapping task. In
addition, they acquired an increased capability to perform detailed or thorough

PT
thinking, which can result in more feasible game episode designs and lead to
better performance in the sensitivity dimension of creativity.

RI
On the other hand, the demonstration scaffolding group saw a full
demonstration of how to progress through the game, which may have resulted

SC
in the participants simply using the same methods to solve the puzzles or even
to create their own game episode designs. The participants in this group may
have lacked the practice to actively construct the relationships between tools or

U
concepts, and were not prompted to think about the concepts or physics
AN
principles underlying the solution, resulting in a poorer performance in concept
mapping. Meanwhile, this group could grasp the ideas of the diversity of game
passing objects or tactics from watching the demonstration, resulting in better
M

adaptability to change in a familiar context, and better performance in the

flexibility dimension of creativity.
D
TE

6.1 Limitations and suggestions for future research

In this study, concept maps were used to visually show students’

EP

understanding and organization of the learning topic “basic machinery.”

Although concept maps are capable of representing different types of
knowledge (factual, conceptual, and procedural knowledge) (Anderson,
C

Krathwohl, & Bloom, 2002), it was not our main purpose to differentiate the
AC

overall conceptual understanding into these three categories in this study.

Therefore, while we administered a pre-test and a post-test involving concept
map constructions, we simply asked students to construct concept maps to
show their comprehension of “basic machinery,” but did not ask them to draw
procedural knowledge on basic machinery in particular. Future researchers
should consider this issue in experimental design if they are interested in
knowing the impacts of different scaffold types on these different knowledge
types. In addition, all of the concept maps were scored based on the six criteria
shown in Table 2. Since there is a standard for evaluating concept maps

28
 29
ACCEPTED MANUSCRIPT
(Markham, Mintzes & Jones, 1994; Moreira, 1979; Novak & Gowin, 1984), we
only asked one junior high school science teacher to do this assessment. If
more than one expert is involved in the assessment, inter-rater reliability should
be reported.
The Consensual Assessment Technique (CAT) (Amabile, 1983) was used
to assess the students’ creativity in game episode design. We asked three
experts to perform individual evaluations based on the assessment criteria

PT
shown in Table 4. Future researchers might consider adopting a qualitative
approach to analyzing the game design content and report the coding process

RI
and detailed qualitative results. In this study, three raters who teach “drawing
and art design” were invited as experts to perform this assessment. Although
these three raters might not be experts in science education, they all have 4-8

SC
years of teaching experience in their domain. In addition, they received training
from one science education teacher to make sure they understood the CAT

U
procedure and had the same background knowledge required to perform this
assessment. Therefore, the three raters could not be considered as true
AN
non-experts, but were regarded as suitable experts for this evaluation.
Sometimes, it is difficult or expensive to assemble teams of experts required by
the CAT. A few researchers have tried to compare the assessments made by
M

expert and non-expert raters in the hope of proving that suitable panels of
non-expert raters could replace more costly panels of experts (Kaufman, Baer,
D

Cole, & Sexton, 2008; Runco, McCarthy, & Svenson, 1994). Kaufman et al.
(2008) noted that it is still not exactly clear whether it is essential to use expert
TE

raters when using the CAT. It is also unclear what kind of expertise is required to
be considered an expert rater for a certain research purpose in a particular
EP

domain. Future researchers should pay special attention to the composition of

raters’ expertise in order to choose the most appropriate judges for evaluating
specific creative products. While it is common to invite experts to evaluate
C

creative products, Runco et al. (1994) proposed other alternatives including

AC

self-ratings and peer ratings, and suggested that professional artists might not be
the best raters of nonprofessional artwork since expert assessments might be
overly critical compared with peer or self-assessments. Due to peer assessment
having been verified as a valid evaluation approach to assess students’ products
(Kao, 2013; Tinapple, Olson, & Sadauskas, 2013; Topping, 1998), future
research might consider peer assessment as an alternative approach to
analyzing the students’ game episode designs. The process of peer assessment
could also have benefits in terms of helping students to think from an
assessor’s perspective, and thus facilitate the students’ understanding or

29
 30
ACCEPTED MANUSCRIPT
enhance their ability in that subject matter.

For the demonstration scaffolding, experts’ gaming behaviors were

pre-recorded and shown to the participants on demand during the game play.
For the marking critical features scaffolding, snapshots of critical steps were
provided as clues to facilitate the learning process. Although the instructional
scaffolds in this study focused on screen recording and snapshot tools, teachers

PT
could experiment with other available computer tools, such as drawing tools to
generate specific instructional scaffolds. Also, Hill and Hannafin (2001)
suggested four other types of scaffolding: (1) conceptual, (2) metacognitive, (3)

RI
procedural, and (4) strategic, to accommodate the needs of different learners in
a variety of contexts. Instructors could consider which scaffold type to use

SC
according to the game properties and personal instructional purpose.

Scaffolding can help build students’ confidence and provide essential

U
support for learning. A proper scaffolding design not only increases learning
motivation, but also facilitates different aspects of learning (i.e., the cognitive
AN
aspects of knowledge acquisition or creative product design) in a
problem-based learning environment. More studies are needed in order to
M

identify which learning strategy is effective in terms of transforming gaming

scaffolding into learning scaffolding. To more effectively assist learning, the
number and content of the scaffolds, as well as the timing of providing
D

scaffolds, should be carefully designed according to the game features to

TE

achieve specific instructional purposes. To continue using the game for learning,
students need to find a balance between the desired difficulty (Bjork & Bjork,
2011) and the frustration confronted through determining whether to use
EP

scaffolds and what kinds of scaffolds to use. Future studies may provide
multiple scaffold types for students to choose, and determine possible variation
in scaffold usage to exploit how students choose different scaffold types
C

depending on their personal needs during the different stages of a learning

AC

process. These issues in educational games should be further explored in future

research.

30
 31
ACCEPTED MANUSCRIPT
References

Adams, W. K., Paulson, A., & Wieman, C. E. (2009). What levels of guidance
promote engaged exploration with interactive simulations? In H. Charles, S.
Mel & H. Leon (Eds.), 2008 Physics Education Research Conference. AIP
Conference Proceedings (Vol. 1064, pp. 59-62). Edmonton, Alberta: AIP Press.

Akbaş, Y., & Gençtürk, E. (2011). The effect of conceptual change approach to

PT
eliminate 9th grade high school students’ misconceptions about air pressure.
Educational Sciences: Theory & Practice, 11(4), 2217-2222.

RI
Akcaoglu, M. (2014). Learning problem-solving through making games at the game
design and learning summer program. Educational Technology Research and

SC
Development, 62(5), 583-600.
Al-Abdali, N. S., & Al-Balushi, S. M. (2016). Teaching for creativity by science
teachers in grades 5–10. International Journal of Science and Mathematics

U
Education, 14(2), 251-268.
AN
Amabile, T. M. (1983). The social psychology of creativity: A componential
conceptualization. Journal of Personality and Social Psychology, 45(2),
357-376.
M

Amabile, T. M., & Tighe, E. (1993). Questions of creativity. In J. Brockman, (Ed.),

Creativity (pp. 7-27). New York: Simon & Schuster.
D

Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning,
TE

teaching, and assessing: A revision of Bloom's taxonomy of educational objectives.

Allyn & Bacon.
Aydeniz, M., & Bilican, K. (2014). What do scientists know about the nature of
EP

science? A case study of novice scientists’ views of NOS. International Journal

of Science and Mathematics Education, 12(5), 1083-1115.
C

Baer, J. (1993). Creativity and divergent thinking: A task-specific approach. Hillsdale,

New Jersey: Lawrence Erlbaum Associates.
AC

Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal
of MUD research, 1(1), 19.
Barzilai, S., & Blau, I. (2014). Scaffolding game-based learning: Impact on learning
achievements, perceived learning, and game experiences. Computers &
Education, 70, 65-79. doi: 10.1016/j.compedu.2013.08.003
Baytak, A., & Land, S. M. (2010). A case study of educational game design by kids
and for kids. Procedia-Social and Behavioral Sciences, 2(2), 5242-5246.
Becker, K. (2007). Pedagogy in commercial video games. Games and simulations in
online learning: Research and development frameworks, Hershey, PA:

31
 32
ACCEPTED MANUSCRIPT
Information Science Publishing.
Belski, I., & Belski, I. (2015). Application of TRIZ in improving the creativity of
engineering experts. Procedia Engineering, 131, 792-797.
Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good
way: Creating desirable difficulties to enhance learning. In M. A. Gernsbacher, R.
W. Pew, L. M. Hough, & J. R. Pomerantz (Eds.), Psychology and the real world:
Essays illustrating fundamental contributions to society (pp. 56-64). New York:

PT
Worth Publishers.
Bodner, G. M. (1986). Constructivism: A theory of knowledge. Journal of Chemical

RI
Education, 63(10), 873-878.
Bruner, J. (1985). Vygotsky: A historical and conceptual perspective. In I.V. Wertsch
(Ed.), Culture, communication and cognition: Vygotskian perspectives (pp.

SC
21-34). Cambridge University Press, England.
Chang, H. Y. (2016). How to augment the learning impact of computer simulations?

U
The designs and effects of interactivity and scaffolding. Interactive Learning
Environments, 1-15.
AN
Charsky, D., & Mims, C. (2008). Integrating commercial off-the-shelf video games
into school curriculums. TechTrends, 52(5), 38-44.
Charsky, D., & Ressler, W. (2011). "Games are made for fun": Lessons on the effects
M

of concept maps in the classroom use of computer games. Computers &

Education, 56(3), 604-615. doi: 10.1016/j.compedu.2010.10.001
D

Chen, C. H., & Law, V. (2016). Scaffolding individual and collaborative game-based
learning in learning performance and intrinsic motivation. Computers in Human
TE

Behavior, 55, 1201-1212.

Chen, C. H., Wang, K. C., & Lin, Y. H. (2015). The comparison of solitary and
EP

collaborative modes of game-based learning on students' science learning and

motivation. Educational Technology & Society, 18(2), 237-248.
Cheng, M. T., She, H. C., & Annetta, L. A. (2015). Game immersion experience: its
C

hierarchical structure and impact on game based science learning. Journal of

AC

Computer Assisted Learning, 31(3), 232-253.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
Hillsdale, NJ: Erlbaum.
Connolly, T. M., Stansfield, M., & Hainey, T. (2007). An application of games-based
learning within software engineering. British Journal of Educational Technology,
38(3), 416-428.
Cremin, T., Glauert, E., Craft, A., Compton, A., & Styliandou, F. (2015). Creative
Little Scientists: exploring pedagogical synergies between inquiry-based and
creative approaches in Early Years science. Education 3-13, 43(4), 404-419.

32
 33
ACCEPTED MANUSCRIPT
Csikszentmihalyi, M. (1997). Finding flow: The psychology of engagement with
everyday life: Basic Books.
D‘Angelo, C. (2010). Scaffolding vector representations for student learning inside a
physics game. Unpublished doctoral dissertation, Arizona State University.
De Jong, T., & Pieters, J. (2006). The design of powerful learning environments. In P.
A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd
ed., pp. 479-510). NJ: Lawrence Erlbaum Associates.

PT
Eckhoff, A., & Urbach, J. (2008). Understanding imaginative thinking during
childhood: Sociocultural conceptions of creativity and imaginative thought.

RI
Early Childhood Education Journal, 36(2), 179-185.
Guilford, J. P. (1968). Intelligence, Creativity and Their Educational Implications (1st
ed.). CA., Robert R. Knapp.

SC
Guilford, J. P. (1977). Way beyond the IQ. New York: Creative Education
Foundation.

U
Hao, Y., Hong, J. C., Jong, J. T., Hwang, M. Y., Su, C. Y., & Yang, J. S. (2010).
Non-native Chinese language learners’ attitudes towards online vision-based
AN
motion games. British Journal of Educational Technology, 41(6), 1043-1053.

Hendrix, R., Eick, C., & Shannon, D. (2012). The integration of creative drama in an
inquiry-based elementary program: The effect on student attitude and conceptual
M

learning. Journal of Science Teacher Education, 23, 823-846.

Hill, J. R., & Hannafin, M. J. (2001). Teaching and learning in digital environments:
D

The resurgence of resource-based learning. Educational Technology Research

and Development, 49(3), 37-52.
TE

Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and

achievement in problem-based and inquiry learning: A response to Kirschner,
EP

Sweller, and Clark (2006). Educational Psychologist, 42(2), 99-107.

Hsu, C. Y., & Tsai, C. C. (2013). Examining the effects of combining self-explanation
principles with an educational game on learning science concepts. Interactive
C

Learning Environments, 21(2), 104-115.

AC

Hwang, G. J., Hung, C. M., & Chen, N. S. (2014). Improving learning achievements,
motivations and problem-solving skills through a peer assessment-based game
development approach. Etr&D-Educational Technology Research and
Development, 62(2), 129-145. doi: 10.1007/s11423-013-9320-7
Jabbar, A. I. A., & Felicia, P. (2015). Gameplay engagement and learning in
game-based learning: A systematic review. Review of Educational Research,
85(4), 740-779. doi: 10.3102/0034654315577210
Kafai, Y. B., & Resnick, M. (Eds.). (1996). Constructionism in practice: Designing,
thinking, and learning in a digital world. NJ: Lawrence Erlbaum Associates.

33
 34
ACCEPTED MANUSCRIPT
Kao, G. Y. M. (2013). Enhancing the quality of peer review by reducing student “free
riding”: Peer assessment with positive interdependence. British Journal of
Educational Technology, 44(1), 112-124.
Kao, G. Y. M., Lin, S. S. J., & Sun, C. T. (2008). Breaking concept boundaries to
enhance creative potential: Using integrated concept maps for conceptual
self-awareness. Computers & Education, 51(4), 1718-1728. doi:
10.1016/j.compedu.2008.05.003

PT
Kaufman, J. C., Baer, J., Cole, J. C., & Sexton, J. D. (2008). A comparison of expert
and nonexpert raters using the consensual assessment technique. Creativity

RI
Research Journal, 20(2), 171-178.
Kendall, M., & Gibbons, J. D. (1990). Rank correlation methods. Oxford: Oxford
University Press.

SC
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during
instruction does not work: An analysis of the failure of constructivist, discovery,

U
problem-based, experiential, and inquiry-based teaching. Educational
psychologist, 41(2), 75-86.
AN
Kloonigames (2014). Crayon physics deluxe. Retrieved February 6, 2014, from
http://www.crayonphysics.com/
Li, M. C., & Tsai, C. C. (2013). Game-based learning in science education: A review
M

of relevant research. Journal of Science Education and Technology, 22(6),

877-898.
D

Liu, T. Y., & Chu, Y. L. (2010). Using ubiquitous games in an English listening and
speaking course: impact on learning outcomes and motivation. Computers &
TE

Education, 55(2), 630-643.

Markham, K. M., Mintzes, J. J. & Jones, M. G. (1994). The concept as a research and
EP

evaluation tool: Further evidence of validity. Journal of College Science Teaching,

31, 91-101.
C

Mayo, M. J. (2007). Games for science and engineering education. Communications

AC

of the ACM, 50(7), 30-35.

Moreira, M. (1979). Concept maps as tools for teaching. Journal of College Science
Teaching, 8, 283-286.

Novak, J. D., & Gowin, D. B. (1984). Learning How to Learn. New York: Cambridge
University Press.

Novak, J. D., & Cañas, A. J. (2007). Theoretical origins of concept maps, how to
construct them, and uses in education. Reflecting Education, 3(1), 29-42.

34
 35
ACCEPTED MANUSCRIPT
Oblinger, D. G. (2004). The next generation of educational engagement. Journal of
Interactive Media in Education, 2004(1), p.Art. 10.

Ott, M., & Pozzi, F. (2012). Digital games as creativity enablers for children.
Behaviour & Information Technology, 31(10), 1011-1019.

Palmer, D. H. (2003). Investigating the relationship between reputational text and

conceptual change. Science Education, 87, 663-684.

PT
Palinscar, A., & Brown, A. (1984). Reciprocal teaching of comprehension-fostering

RI
and comprehension-monitoring activities. Cognition and Instruction, 1, 117-175.
Parnes, S. J. (1967). Creative behavior guidebook. Scribner.

SC
Parnes, S. J., Noller, R. B., & Biondi, A. M. (1977). Guide to creative actions: revised
edition of the creative behavior guidebook. New York: Charles Scribner's Sons.

U
Plucker, J. A., Beghetto, R. A., & Dow, G. T. (2004). Why isn’t creativity more
important to educational psychologists? Potentials, pitfalls, and future directions
AN
in creativity research. Educational Psychologist, 39(2), 83-96.

Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.

M

Prensky, M. (2008). Students as designers and creators of educational computer

D

games: Who else? . British Journal of Educational Technology, 39(6),

1004-1019.
TE

Quigley, C., Pongsanon, K., & Akerson, V. L. (2010). If we teach them, they can learn:
Young students’ views of nature of science aspects to early elementary students
during an informal science education program. Journal of Science Teacher
EP

Education, 21(7), 887-907.

Quintana, C., Reiser, B. J., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., . . .
C

Soloway, E. (2004). A scaffolding design framework for software to support

science inquiry. Journal of the Learning Sciences, 13(3), 337-386.
AC

Rapp, A. (2015). Designing interactive systems through a game lens: An ethnographic

approach. Computers in Human Behavior.
Raybourn, E. M., & Bos, N. (2005). Design and evaluation challenges of serious
games. Paper presented at the CHI'05 extended abstracts on human factors in
computing systems.
Reinfried, S. (2006). Conceptual change in physical geography and environmental
sciences through mental model building: The example of groundwater.
International Research in Geographical and Environmental Education, 15(1),
41-61.

35
 36
ACCEPTED MANUSCRIPT
Rhodes, M. (1961). An analysis of creativity. The Phi Delta Kappan, 42(7), 305-310.
Rosas, R., Nussbaum, M., Cumsille, P., Marianov, V., Correa, M., Flores, P., et al.
(2003). Beyond Nintendo: design and assessment of educational video games for
first and second grade students. Computers & Education, 40(1), 71-94.
Runco, M. A., McCarthy, K. A., & Svenson, E. (1994). Judgments of the creativity of
artwork from students and professional artists. Journal of Psychology, 128,
23-31.

PT
Shieh, R. S., & Chang, W. (2014). Fostering student’s creative and problem-solving
skills through a hands-on activity. Journal of Baltic Science Education, 13(5),

RI
650-661.
Steffe, L. P., & Gale, J. E. (Eds.). (1995). Constructivism in education. Hillsdale, NJ:
Lawrence Erlbaum.

SC
Siew, N, M., Chong, C. L., & Lee, B. N. (2015). Fostering fifth graders’ scientific
creativity through problem-based learning. Journal of Baltic Science Education,

U
14(5), 655-669.
Soloway, E., Guzdial, M., & Hay, K. E. (1994). Learner-centered design: The
AN
challenge for HCI in the 21st century. interactions, 1(2), 36-48.
Spires, H. A., Rowe, J. P., Mott, B. W., & Lester, J. C. (2011). Problem solving and
game-based learning: effects of middle grade students’ hypothesis testing
M

strategies on learning outcomes. Journal of Educational Computing Research,

44(4), 453-472. doi: 10.2190/EC.44.4.e
D

Squire, K. D. (2008). Video games and education: Designing learning systems for an
interactive age. Educational Technology, 48(2), 17-26.
TE

Sternberg, R. J., & Lubart, T. I. (1996). Investing in creativity. American Psychologist,

51, 677-688.
EP

Sternberg, R. J., & Lubart, T. I. (1999). The concept of creativity: Prospects and
paradigms. In R. J. Sternberg (Ed.), Handbook of creativity (pp. 3-15).
C

Cambridge, UK: Cambridge University Press.

AC

Sun, C. T., Wang, D. Y., & Chan, H. L. (2011). How digital scaffolds in games direct
problem-solving behaviors. Computers & Education, 57(3), 2118-2125.
Suits, B. (1978). The Grasshopper: Games, Life and Utopia. Toronto: University of
Toronto Press.
Tinapple, D., Olson, L., & Sadauskas, J. (2013). CritViz: Web-based software
supporting peer critique in large creative classrooms. Bulletin of the IEEE
Technical Committee on Learning Technology, 15(1), 29.
Topping, K. J. (1998). Peer assessment between students in colleges and universities.
Review of Educational Research, 68, 3, 249-276.

36
 37
ACCEPTED MANUSCRIPT
Urhahne, D., Kremer, K., & Mayer, J. (2010). Conceptions of the nature of science—
Are they general or context specific? International Journal of Science and
Mathematics Education, 9(3), 707-730.

Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press.

Wood, D. J., Bruner, J. S., ＆ Ross, G. (1976). The role of tutoring in problem
solving. Journal of Child Psychology and Psychiatry, 17, 89-100.

PT
Wu, W. H., Chiou, W. B., Kao, H. Y., Hu, C. H. A., & Huang, S. H. (2012).
Re-exploring game-assisted learning research: The perspective of learning

RI
theoretical bases. Computers & Education, 59(4), 1153-1161. doi:
10.1016/j.compedu.2012.05.003

SC
Yager, S., Dogan, O., Hacieminoglu, E., & Yager, R. (2012). The role of student and
teacher creativity in aiding current reform efforts in science and technology
education. National Forum of Applied Educational Research Journal, 25(3),

U
1-24.
Yin, J. (2008). A discussion on the cultivation of artistic originality. Asian Social
AN
Science, 4(9), 73-75.
M
D
TE
C EP
AC

37
 38
ACCEPTED MANUSCRIPT

Figure Captions

Figure 1. Conceptual framework

Figure 2. Study procedure

PT
Figure 3. The Crayon Physics Deluxe gaming environment

RI
Figure 4. Customizing a demonstration scaffold

Figure 5. Customizing a marking critical features scaffold

SC
Figure 6. A student’s concept map on basic machinery in Physics

U
Figure 7. An effective educational game with learning scaffolds
AN
M
D
TE
C EP
AC

38
 39
ACCEPTED MANUSCRIPT

Table Captions

Table 1. “Crayon Physics Deluxe” Episode Design Object Functions

PT
Table 2. Concept map assessment criteria

Table 3. Game episode design sheet

RI
Table 4. Five evaluation dimensions to assess creativity of game episode design

SC
Table 5. ANCOVA results for concept map scores by game-playing condition

U
AN
Table 6. ANCOVA results for concept map scores by scaffolding condition

Table 7. Descriptive statistics and MANOVA results for the five dimensions of
M

creative game episode design

D
TE
C EP
AC

39
 ACCEPTED MANUSCRIPT
Table 1. “Crayon Physics Deluxe” Episode Design Object Functions
Object Icon Function

Draw Draw shapes and lines.

Move Move items to interest areas.

Rope Draw rope function lines.

PT
Pin one item on or to another; one pin allows item rotation, two
Pins
pins indicate secure placement.

RI
Ball Add red balls to interest areas.

Star Add yellow stars to interest areas.

SC
Play Preview game episode design.

U
Stop Stop game episode design preview.
AN
Game Pins Serve same function as pins. Game episode designers could use
this game pins as hints for solving puzzles.
A source of power. Rockets are launched when touched by other
Rocket
M

objects.
Force A source of power. Force is required to work with “Pins.”
D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

Table 2. Concept map assessment criteria

Criterion Score description
Proposition Is the relationship between two concepts indicated by a connecting
line and linking word(s)? For each meaningful, valid proposition
shown, score 1 point.

PT
Hierarchy Does the map show hierarchy? Is each subordinate concept more
specific and less general than the concept drawn above it? Score 5
points for each valid level of the hierarchy.

RI
Cross-link Does the map show meaningful connections between one segment of
the concept hierarchy and another segment? Score 10 points for each
cross link that is both valid and significant and 5 points for each cross

SC
link that is valid.

Example Specific events or objects that are valid instances of those designated

U
by the concept label can be scored 1 point each.
AN
Branch For each meaningful and valid branch of a concept, score 1 point.

Extra proposition Proposition that is not included in the learning material but is shown
on the map. For each meaningful, valid extra proposition shown, score
M

2 points.
D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

Table 3. Game episode design sheet

Group: (please leave it blank) Name of the game episode: Fly fly Your name:

Number of objects used: 24 Difficulty: high medium low Number of functions

used: 9

PT
Please draw your game episode
design

Description of the game scenario
(Please describe the features.)
RI
SC
In general, it is a cute game episode with elements such as a hot
air balloon and rocket. This game has an appropriate level of
difficulty; everyone can finish it in a reasonable amount of
time.

U
Description of the problem-solving 1. Connect the hot air balloon and windmill by drawing two
tactics (Please describe the steps.) pins. 2. Draw an object for the ball to roll down to get the first
AN
star. 3. Draw an object in the upper right corner of the windmill
and remove the object that connects the two items to get the
second star. 4. Draw some objects on the rocket to let the rocket
M

bump into the ball to get the stars.

D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

Table 4. Five evaluation dimensions to assess creativity of game episode design

Dimension Description Scoring
Sensitivity The ability to see key points in a Based on the feasibility to execute game
question or context; awareness of a episode design. The game design reflects
need for change. an awareness of applying key objects to
solve a physics puzzle. Points: 1 (Low) to
5 (High).

PT
Fluency The ability to rapidly give multiple Based on the number of game episode
answers in response to the same designs created. A valid and significant
information. design earns 2 points; similar design earns

RI
1 point.

Flexibility Ability to make changes in a familiar
context, adaptability to change
instructions, with evidence of active
thinking.
SC
Based on the flexibility to organize objects
in the game design. Every compound
object earns 2 points; every simple object
earns 1 point.

Originality
U
The ability to generate unconventional Based on the uniqueness of game episode
AN
associations or to respond to tasks in design. Points: 1 (Low) to 10 (High).
novel ways.

Elaboration Realization or transformation of an Based on the richness of game episode

M

idea; the ability to add enriching design. Points: 1 (Low) to 5 (High).

details, fill in gaps, or add finishing
touches.
D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

Table 5. ANCOVA results for concept map scores by game-playing condition

Group Number of Estimated SD F-value Pairwise
students marginal mean (p-value) comparison
(1) Game 77 52.55 .55 22.00***(.00) (1) > (2)
playing

PT
(2) Non-game 26 49.59 .32
playing

RI
SC
*** p < .001

U
AN
M
D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

Table 6. ANCOVA results for concept map scores by scaffolding condition

Number of Estimated F-value Pairwise
Group SD
students marginal mean (p-value) comparison
(1) Demonstration 26 50.53 .57 4.07*(.00) (3) > (1)
(2) Non-scaffolding 23 50.90 .61 (3) > (2)

PT
(3) Marking critical features 28 52.64 .55
* p < .05

RI
U SC
AN
M
D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

Table 7. Descriptive statistics and MANOVA results for five dimensions of creative game episode
design
Dimension Group N Mean SD Value F df Error df Sig PES Post hoc
(LSD)
Sensitivity 1 30 2.38 .76
2 30 2.18 .80 3.69* 2 .02 .08 3>2
3 31 2.71 .73

PT
Fluency 1 30 3.52 1.10
2 30 3.15 1.15 .87 2 .21 .02
3 31 3.32 .97
Flexibility 1 30 3.65 1.15

RI
2 30 3.03 .96 3.26* 2 .02 .07 1>2
3 31 3.18 .79
Originality 1 30 2.60 1.49

SC
2 30 2.32 1.03 1.38 2 .13 .03
3 31 2.84 1.11
Elaboration 1 30 1.92 .94
2 30 1.88 .74 .99 2 .19 .02

U
3 31 2.15 .69
Wilk’s lambda .78 2.20 10 168 .01 .116
AN
Group categories: 1 - demonstration; 2 - non-scaffolding; 3 - marking critical features
* p < .05.
M
D
TE
C EP
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

PT
RI
U SC
AN
M
D
TE
EP
C
AC
 ACCEPTED MANUSCRIPT

Customizing Scaffolds for Game-Based Learning in Physics:

Impacts on Knowledge Acquisition and Game Design Creativity

PT
Highlights
• Customized digital games have the potential to meet specific teaching goals.

RI
• Appropriate scaffolding could enhance knowledge acquisition & design
creativity.
• Concept maps and game design outcomes were used to assess learning effects.

SC
• Scaffolding types and timing should be considered when customizing digital
games.

U
AN
M
D
TE
C EP
AC