Luo 2022

Education and Information Technologies
https://doi.org/10.1007/s10639-021-10642-9
Gamification for educational purposes: What are the factors

contributing to varied effectiveness?
Zhanni Luo1
Received: 25 March 2021 / Accepted: 16 June 2021

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature
2021
Abstract
Gamification refers to the use of game-design elements or mechanisms in non-
game contexts to promote the expected behaviours. Though theoretically promising,
empirical studies reported mixed results as to the effectiveness of gamification in
educational practices. To understand this issue better, this author selected 44 arti-
cles on the topic of educational gamification with the use of a bibliometric analy-
sis software HistCite, followed by a descriptive content analysis on these articles.
This author examined three issues, including how effective the educational gamifica-
tion implementations were in previous empirical studies, how the effectiveness has
been measured, and what factors contribute to varied effectiveness results. One sig-
nificant output of the current study is the redefinition of two terms: game elements
refer to “the obvious game-like elements that are frequently used in digital games or
gamification, which are concrete nouns”, and gamification mechanisms refer to “the
underlying guidelines that make gamification activities engaging, which are abstract
nouns that relate to humans’ innate psychological needs”. What’s more, the content
analysis helped summarise a framework describing what makes gamification engag-
ing, which contains goal, visualisation, immediate feedback, adaptation, challenge,
competition, reward, and fun failure. Limitations and suggestions for future studies
have been discussed.
Keywords Gamification · Gamified learning · Effectiveness assessment · Learning

engagement · Descriptive content analysis
* Zhanni Luo
zhanni.luo@pg.canterbury.ac.nz
1
School of Educational Studies and Leadership, University of Canterbury, Room 415, Rehua
Building, Christchurch 8041, New Zealand
13
Vol.:(0123456789)
1 Introduction
Gamification, the use of game design elements into non-game contexts (Deterding
et al., 2011), has gained popularity in education in the last two decades due to its
potential in enhancing students’ engagement and academic performances (Chen
et al., 2015; Ibanez et al., 2014; Martí-Parreño et al., 2016). Educational gamifi-
cation, also named gamified learning, gamification in education, or gamification
for educational purposes, refers to the use of gamification techniques to facilitate
teaching or learning.
A large body of empirical studies attempted to confirm whether gamification
is significantly effective in engaging learners in pedagogical activities. It was
assumed that gamification positively impacts learning; however, with the enrich-
ment of empirical studies, researchers began to realise that educational gamifica-
tion might bring no difference or even negative impacts (Adukaite et al., 2017;
Chen et al., 2015; De-Marcos et al., 2014; DomíNguez et al., 2013; Hanus & Fox,
2015; Ibanez et al., 2014). Therefore, Buckley and Doyle (2017) proposed to treat
gamification with caution in the field of education.
The confounding effectiveness results reported in previous studies raised the
following questions: is educational gamification effective in pedagogical prac-
tices? What measures have been employed in assessing the effectiveness of edu-
cational gamification? What factors contribute to varied effectiveness results? If
there is a gap between theories and practice, what brings the differences and how
to minimise the gap? To this author’s knowledge, these issues were understudied.
The current study aims to answer three research questions (RQs):
RQ 1: How effective the gamification implementations were in previous empir-

ical studies in the field of education?
RQ 2: How previous studies measure the effectiveness of educational gamifica-
tion?
RQ 3: In the selected articles, what are the factors contributing to the varied
effectiveness of educational gamification?
The first round of data analysis revealed that the involvement of different game
elements or gamification mechanism is a significant factor affecting effective-
ness results. Accordingly, this author added a fourth research question, which
is:
RQ 4: In theory, what game elements or gamification mechanisms make edu-
cational gamification effective? In practice, what game elements or gamifica-
tion mechanisms were used in empirical studies? If there are differences, what
causes the differences?
“Gamification mechanism” in RQ 4 is a new term proposed by this author

based on the content analysis results. Since the definitions of game elements and
gamification mechanisms overlapped in previous studies, this author also rede-
fined the term “game elements”. The redefined definitions are as below (which
will be detailed in the section “Findings of the descriptive content analysis”):
13
• Game elements: the obvious game-like elements that are frequently used in digi-
tal games or gamification, which are concrete nouns.
• Gamification mechanisms: the underlying guidelines that make gamification
activities engaging, which are abstract nouns that relate to humans’ innate psy-
chological needs.
2 Methodology
2.1 Research design
The current study is a descriptive content analysis based on citation analysis results
generated by the bibliometric analysis software HistCite.
Bibliometric analysis, first proposed by Pritchard (1969) as “the application
of mathematics and statistical methods to books and other media of communica-
tion” (p. 348), is “a research technique using quantitative and statistical analyses to
describe distribution patterns of research articles with a given topic” (Martí-Parreño
et al., 2016, p. 666). In most cases, bibliometric analysis involves works in a large
time span; for example, Song et al. (2019) investigated research on classroom dia-
logues based on 3914 papers published from 1999 to 2018 from the Web of Science
(WoS) database; Zawacki-Richter et al. (2010) explored the growing impact of 12
open access distance educational journals based on 1123 articles published between
2003 to 2008; Huang et al. (2020) proposed to explore the hot topics in educational
research, followed by a bibliometric analysis based on 19,084 papers published from
2000 to 2017. Due to the large-time-span nature, bibliometric analysis was regarded
as a helpful approach that provides facilitative research retrospectives, identifies
research hotspots, and reveals development trends in the targeted discipline (Song
et al., 2019).
Citation linkage analysis is a common technique in bibliometric analysis, which
is “not a new practice” in examining published academic works (Budd, 1988, p.
180). With the use of citation linkage analysis, bibliometric analysis software (e.g.
HistCite) helps identify the key literature in a research field by pinpointing impor-
tant papers, identifying the most prolific scholars and enlisting the highly cited jour-
nals. It also visualises the history of a research field by creating historiographies that
show the citation relationship among important papers (Garfield et al., 2006; Pan
et al., 2018).
This study used two outputs generated by the bibliometric analysis software
HistCite, including the articles with high citation scores and the articles in histori-
ography (also named citation graph, a visual presentation of the linkages among the
frequently-cited works among the selected articles). The original data is available
online (see Supplementary data at the end of this study).
The results generated by bibliometric analysis (including citation analysis) lack
in-depth analysis; therefore, to release the full potential of bibliometric analysis,
researchers attempted to combine it with qualitative interpretation approaches, such
as descriptive content analysis and systematic literature review (Arici et al., 2019;
Bartolini et al., 2019; Goksu et al., 2020). However, to this author’s knowledge, the
13
extant related publications did not yet cover the field of gamification or educational
gamification.
2.2 Data collection and data presentation
On the dataset Web of Science, this author retrieved 175 published works after
performing the following search strategy and keywords in the field topic: (“gami-
fication” OR “gamify” OR “gamified” OR “gamifying”) AND (“education”) AND
(“effectiveness”). Next, this author exported the data by following the path of
“Export”- “Other file format”- “Record content (Full record and cited references)”-
“File format (Plain text)”. After imported the data into the software HistCite, this
author obtained two documents: the list of articles with high citation score from
Analyses- Cited paper and the historiography from Graph Maker.
The top 30 articles from each document were selected. After deleting 16 articles,
including 13 not found, two not highly related, and one appears in both the histo-
riography and the list of articles with high citation score, this author obtained 44
articles, in which 23 were empirical studies that involved participation recruitment,
while the other 21 were literature reviews, perspectives or statement papers.
Due to limited space, the coding process was not attached in the manuscript (see
Supplementary data at the end of this study for detailed information). Findings of the
current study were concluded based on the content analysis results of the selected
data, except those begin with “this author believes” and “this author noticed”.
2.3 Data analysis
Due to the exploratory nature of the current study, this author did not conduct a sys-
tematic literature review based on the generated data; instead, this author selected
the descriptive content analysis approach.
The 23 empirical studies were the main sources in presenting big-picture infor-
mation such as whether gamification is effective in education (RQ 1), how to meas-
ure the effectiveness of gamification in education (RQ 2), as well as which game
element or gamification mechanism has been used in practice (RQ 4). The 21
descriptive papers were supportive in providing direct answers to the question of
“what are the factors contributing to the varied effectiveness of educational gami-
fication” (RQ 3), as well as presenting insightful explanations to other research
questions.
The descriptive content analysis followed the thematic analysis instructions pro-
posed by Braun and Clarke (2006).
2.4 Inclusion and exclusion criteria
Though not necessarily connected with digital devices, most of the extant tools for
gamified learning are digital-based in the form of software, computer systems or mobile
phone applications; together with the fact that gamified learning studies are at the
crossroads of psychology and pedagogy (All et al., 2016), gamification for educational
13
purposes is currently an interdisciplinary area that involves researchers of varied back-

ground, such as engineers with information technology (IT) background, programmers
with game-design background, psychologists with behaviour science background, edu-
cationists with teaching backgrounds, etc. Accordingly, the publications in the target
field are not only for classroom learning. To be specific, the publications selected in
the current manuscript were about four main issues: 1) the development of a specific
gamification learning tool (which could be mechanical), 2) the exploration of gamifica-
tion design principles for enhanced learning output (which could be psychological), 3)
the use of existing gamification tools for pedagogical purposes (which are educational),
and 4) the effectiveness assessment of a specific gamification tool in classroom settings
(which are educational).
It worth noticing that the keywords game elements, non-game contexts, digital
games and game-based learning must be excluded. This author conducted four rounds
of trial studies with these keywords, found that the data involved a large number of
documents in other fields such as mathematics and psychology.
3 Findings of the systematic literature review
3.1 The effectiveness of educational gamification
Silpasuwanchai et al. (2016) claimed that the effectiveness of gamification for learning
remains controversial: though gamification has been gaining increasingly more atten-
tion, “there is little empirical evidence to support their effectiveness” on engaging stu-
dents or improving academic performances (Denny, 2013, p. 763). This viewpoint was
supported by other researchers such as Hakulinen et al. (2013). Table 1 presents the
effectiveness results of educational gamification reported in the selected articles. Gen-
erally speaking, the selected empirical studies reported significantly different results
on the effectiveness of educational gamification: among the 23 empirical studies, 11
reported positive results, two negative, three mixed, two partial, and two no differences;
the effectiveness result report was missing in three empirical studies.
Gamified learning with positive effects A large number of empirical studies con-
cluded that gamified learning is promising as to its effectiveness in education
(n = 11). Thom et al. (2012) proposed to assess the effectiveness of gamification in
a social networking service (SNS) by removing the points system including points
and badges. The results suggested that the removal of the points system has a “sig-
nificant negative impact” on user activity (Thom et al., 2012, p. 1069). Namely, the
points system helps improve user activity. Similarly, Hamari (2017) investigated the
Table 1 The reported effectiveness of educational gamification in empirical studies

Positive Negative Mixed Partial No effects n/a
Number (out of 23) 11 2 3 2 2 3
13
effectiveness of an individual mechanic “badges” on a trading website Sharetribe.

The results showed that users in the gamification group were “significantly” more
likely to take part in online activities (p. 469). The research results of other schol-
ars also support the positive effect of gamified learning (Fernandes et al., 2012;
González et al., 2016; Hamari, 2013; Ibanez et al., 2014; Osipov et al., 2016; Papastergiou,
2009; Rai & Beck, 2017; Riva et al., 2014; Stanculescu et al., 2016; Su & Cheng,
2013).
Kim and Lee (2015) proposed a dynamic model in regard to the effectiveness
of educational gamification. According to Kim and Lee (2015), gamified learning
or game-based learning starts with lower educational effectiveness than the tradi-
tional way due to hindrances (e.g., adaptation time), and then “increases rapidly and
exceeds the educational effectiveness of traditional learning” (p. 8489). Eventually,
gamified learning helps achieve better learning outcomes than the traditional way.
Empirical studies that confirmed the positive effect of gamified learning also
implied that educational gamification should be implemented with more consid-
erations, such as whether it works on all the students and whether it works to the
same extent. For example, Denny (2013) surveyed to explore whether a gamification
element “badge” engages students in online learning settings. The overall data sug-
gests a positive result, however, Denny (2013) stated that “not all students are moti-
vated… to the same extent” (p. 770). In other words, a certain group of students are
not engaged in gamified learning though it is not revealed in the data.
Gamified learning with negative effects Hanus and Fox (2015) believed that one
central goal of education is to get students intrinsically motivated, which occurs
when the desire to learn comes from within the students. However, educational
gamification tends to provide extrinsic incentives or rewards, such as badges,
points and competition. According to Hanus and Fox (2015), providing “tangible”
and “expected” rewards to learners who are already interested in the current task
“may cause them to shift motivations from intrinsic (e.g., because they wanted to) to
extrinsic (e.g., because they want to earn a reward)”. This opinion was supported by
various scholars such as Deci and Ryan (2000), Lepper et al. (1973) and Tang and
Hall (1995).
Hanus and Fox (2015) tested their hypotheses on 71 students across two courses
during a 16-week semester in educational settings. The results showed that the stu-
dents in the gamified group were less motivated, satisfied or empowered than stu-
dents in the controlled group. At the same time, students in the gamified course
gained lower scores in the final examination.
The study of Chen et al. (2015) revealed similar results, as users felt disen-
gaged and confused about the customised gamification system. Chen et al. (2015)
explained that the failure might be caused by technical and logistical implementation
issues, such as poor level of the perceived ease of use to the gamification system.
Gamified learning with mixed or partial effects Consistent with the conclusion of
Adukaite et al. (2017), there was an emerging body of research suggesting that gam-
ified learning is not always beneficial or effective. The study of Ibanez et al. (2014)
13
reported mixed results on the effectiveness of gamified learning activities targeted

at the learning of the C-programming language. Besides analysing students’ login
information from log files, the authors requested the 22 participants to complete five
questionnaires, which contained four open-ended questions focusing on students’
reasons to get involves and students’ main focus during the learning activity. The
login information and questionnaire results suggested general “positive” effects on
student engagement toward gamified learning activities and a “moderate” improve-
ment in learning outcomes (p. 291); though, the open-ended question responses pre-
sent a portion of negative answers. Participants leave comments such as “There were
no new things to explore”, “I did not feel engaged” and “I feel bored” (Ibanez et al.,
2014, pp. 297, 299).
Similarly, the qualitative comments in the study of DomíNguez et al. (2013)
indicated a negative impact of gamification techniques on students’ engagement.
According to DomíNguez et al. (2013), good results do not apply to every individ-
ual. The system was “not motivating enough”, not fun, and even “discouraging” for
certain types of users (p. 391). DomíNguez et al. (2013) designed a gamification
plugin containing PBL (points, badges, and leaderboards) for a well-known e-learn-
ing platform and tested students’ achievement and attitude. They summarised a
specific conclusion that gamification partially impacts students as students in gami-
fied group “got better scores in practical assignments and in overall score”, but they
“performed poorly on written assignments and participated less on class activities,
although their initial motivation was higher” (DomíNguez et al., 2013, p. 380).
The same with DomíNguez et al. (2013), other empirical studies reported that
gamified learning does not impact every aspect of learning: gamified and traditional
learning have different functions and impacts. De-Marcos et al. (2014) found that
students in the gamified learning group performed better on practical assignments
but worse in knowledge assessments; at the same time, even though students in the
gamified group are positive in attitude, they gain comparatively low scores and rates.
With similar results, Hakulinen et al. (2013) attempted to differentiate behaviour
engagement from general engagement, and the results of their study indicated that
badges affect the “behaviour of students” but not the academic “grading” (p. 47).
Gamified learning with slight or no effects Surprisingly, there were also stud-

ies reporting that gamification brings slight or even no effect on education. Both
DomíNguez et al. (2013) and De-Marcos et al. (2014) found that in a final examina-
tion for the assessment of conceptual knowledge, students in the gamified gained
similar academic scores with students in the controlled group. DomíNguez et al.
(2013) concluded that “cognitive impact of gamification over students is not very
significant” (p. 391).
3.2 The measures of the effectiveness of educational gamification
Though numerous benefits of educational gamification have been nominated, the

most-recognised one is the potential in improving students’ learning engagement
(Landers & Armstrong, 2017). According to Ibanez et al. (2014), among the various
13
conceptualisations of engagement categories, a frequently-used theory is the three-

component engagement: behavioural engagement, affective engagement, and cogni-
tive engagement.
Behaviour engagement is the task engagement people can observe based on users’
behaviours. Indicators of behavioural engagement could be time on task, homework
completion, attendance, and classroom activity participation (Appleton et al., 2006,
2008; ; Ge & Ifenthaler, 2018; Ibanez et al., 2014; Silpasuwanchai et al., 2016).
In the selected studies, researchers measured behavioural engagement with indi-
cators such as time of use (Osipov et al., 2016; Silpasuwanchai et al., 2016; Stanculescu
et al., 2016), the number of answers or connections (Denny, 2013; Hamari, 2017;
Osipov et al., 2016; Stanculescu et al., 2016), earned points (Osipov et al., 2016), or
simply the participation (DomíNguez et al., 2013; Hamari, 2013; Thom et al., 2012).
A large number of researchers collect behavioural engagement data via observation
or from the log files in the gamified learning system.
Affective engagement, also named emotional engagement, is the task engagement
linked to emotional reactions. According to Boykin and Noguera (2011), affective
engagement reflects on various aspects, such as students’ “interest level, positive
affect, positive attitude, positive value held, curiosity, and task absorption (and the
less the anxiety, sadness, stress, and boredom” (p. 43).
Researchers measure affective engagement with factors such as achievement
(Fleming, 2001; Su & Cheng, 2013), attitude (De-Marcos et al., 2014; DomíNguez
et al., 2013; González et al., 2016; Rai & Beck, 2017), efficacy (Buckley & Doyle,
2017), empowerment (Hanus & Fox, 2015; Riva et al., 2014), enjoyment (Silpasuwanchai
et al., 2016), motivation (Fernandes et al., 2012; Hanus & Fox, 2015), satisfaction
(Fernandes et al., 2012; Hanus & Fox, 2015) and the sense of encouragement (Chen
et al., 2015). There are also other indicators such as the easiness to understand, easi-
ness to play, perceived usefulness, domination, immersion, self-determination and
amusement (Fernandes et al., 2012; Fleming, 2001; Koivisto & Hamari, 2014; Riva
et al., 2014). The main measures of these indicators are self-report questionnaires or
interviews (Hew et al., 2016).
Cognitive engagement is defined as a psychological state in which students are
willing to put time and efforts to “truly understand a topic” and in which students
“persist studying over a long period of time” (Rotgans & Schmidt, 2011, p. 465).
The indicators of cognitive engagement are comparatively complex, as Appleton
et al. (2006) proposed that cognitive engagement has been traditionally measured
by homework completion, class attendance and interactions with the teachers, which
are also “indicators of behavioral engagement” (p. 429). This author agrees with the
further discussion of Appleton et al. (2006), which explains that cognitive engage-
ment includes “less observable” and “more internal” indicators, such as self-regula-
tion, the value of learning, personal goals and learning autonomy (p. 429). Charac-
teristics of cognitive engagement were later simplified as “learners’ psychological
investment in learning” (Silpasuwanchai et al., 2016, p. 462).
It worths noticing that some researchers regarded academic performance as
the indicator of cognitive engagement; therefore, they assessed academic scores
in exams, number of correct responses in quizzes, or number of recalled items in
post-tests (De-Marcos et al., 2014; DomíNguez et al., 2013; Hanus & Fox, 2015;
13
Silpasuwanchai et al., 2016). Academic assessments indeed involve cognitive activi-

ties; however, this author believes that cognitive involvement is not equivalent to
cognitive engagement. If exclude studies that assess academic performances, it can
be seen that cognitive engagement remains under-investigated. As stated by Silpasuwanchai
et al. (2016), whether gamification can help students “think deeply” and “reflect”
“remains inconclusive” (p. 462).
To conclude, previous studies mainly assessed the effectiveness of educational
gamification by measuring students’ behavioural and affective engagement, with
the involvement of cognitive engagement remained controversial. It is still in debate
whether academic performance improvement can be regarded as enhanced cognitive
engagement: if the answer is positive, cognitive engagement was frequently assessed
by previous empirical studies, which was strongly related with academic perfor-
mances tested via quizzes or examines; if the answer is negative, cognitive engage-
ment was rarely assessed in previous educational gamification studies, which should
be addressed in future studies.
3.3 Factors contributing to varied effectiveness results of educational

gamification
3.3.1 Factors affecting the effectiveness of gamification in education
In regard to factors affecting the effectiveness of educational gamification, five

main factors emerged from data analysis, including gender, game experience, indi-
vidual differences (learning styles, personality traits, etc.), and resistance to new
technologies.
Age, gender, and game experience were frequently mentioned as the demographic
variables contributing to the varied effectiveness results of educational gamifica-
tion (Yee, 2006). According to the selected articles in the current study, age does
not bring significant differences in the effectiveness of educational gamification
(Koivisto & Hamari, 2014), while gender and game experience do result in varied
impacts (Koivisto & Hamari, 2014; Silpasuwanchai et al., 2016). More specifically,
researchers reported that students of different gender perceive the benefits of gami-
fied learning differently, and students with more game experience are more possible
to benefits more from the gamification design (Koivisto & Hamari, 2014; Landers &
Armstrong, 2017).
Individual differences was an important research branch in studying the factors
influencing the effectiveness of educational gamification, as gamified learning does
not impact the same on every student. Researchers studied the impact of individ-
ual differences from various perspectives including learning style and personality
traits (Buckley & Doyle, 2017), player types (Barata et al., 2017; DomíNguez et al.,
2013), personal preferences (Denny, 2013), player motivation (Yee, 2006), etc.
If the empirical studies involved specific gamified learning tools such as websites
and mobile apps (rather than non-physical gamification approaches), resistance to
new technologies was a frequent-mentioned factor affecting the result. The gamified
instruction may lead to poorer outcomes if students were not comfortable with the
13
Table 2 Factors and barriers influencing the effectiveness of gamified learning

Factors influencing the result Barriers of gamification studies
1) Gender 1) Inappropriate or superficial use of gamification

2) Usage experience techniques
3) Individual difference (e.g., personality traits, 2) Ambiguous or broad definition of the term
learning styles) 3) Poor integration of game features and peda-
4) Resistance to new technologies gogical content
4) Difficulties of designing adaptive gamification
systems
5) Heavy reliance on computer science skills
6) Methodological limitations
new technology (Landers & Armstrong, 2017) or students perceived the tool as a
difficult-to-use one (DomíNguez et al., 2013).
Due to the elementary nature of gamification studies, there was a limited number
of empirical studies specifically focusing on the contributing factors of effectiveness
assessment. This author followed an in-depth content analysis by viewing the full
texts of the selected articles, especially in the section of discussions and research
limitations. The content analysis helped reveal several barriers that impede reach-
ing consistent effectiveness results in educational gamification studies, as shown in
Table 2 and detailed in the next section.
3.3.2 Barriers to reaching consistent effectiveness results in educational

gamification studies
Inappropriate or superficial use of gamification techniques was an obvious theme

that emerged from the content analysis. Gamification has been defined as “using
gamification mechanisms or game elements to non-game contexts”, which indicates
that gamification is the “use of design (rather than game-based technology or other
game-related practices) elements (rather than fully developed games)” (Deterding
et al., 2011, p. 5). However, for a long time in practice, the tools for gamification
studies were customised video games, which brought concerns such as poor inte-
gration with educational content, irrelevant tasks, homogenous content, waste of
classroom time, and possibilities of getting students addicted (Kim & Lee, 2015; Kirriemuir &
McFarlane, 2004; Morris et al., 2013; NFER, 2009; Van Rosmalen et al., 2015).
As a result, the video games and formal learning in schools “each make(s) the
other worse” (Lee & Hammer, 2011, p. 4). Afterwards, researchers began to focus
on educational websites or mobile applications with the use of game elements,
which aligned with the gamification definition; however, as will be detailed later in
Table 3, though dozens of game elements are available, the most explored ones were
limited to points, badges and leaderboard only. In this way, gamification is more
likely to be “pointsfication” only (the use of point-related elements only), which is
far from reaching the core of gamification design. In other words, empirical studies
may claim that they were assessing gamification, while they were actually assessing
something else, which brings biased or even misleading results to the research field.
13
Table 3 Redefinition of game elements and gamification mechanisms

Feature Term Definition
Game-like Explicit gamification The strategies that utilise obviously game-like appli-
cations. (Chou, 2015)
Game elements The obvious game-like elements that are frequently
used in digital games or gamification, which are
concrete nouns.
Human-desire based Implicit gamification The human-focused design that utilises game ele-
ments in non-game contexts (Chou, 2015)
Gamification mechanisms The underlying guidelines that make gamification
activities engaging, which are abstract nouns that
relate to humans’ innate psychological needs.
The ambiguous or broad definition of related terms also impedes consistency in

concluding the effectiveness of educational gamification (Seaborn & Fels, 2015).
The ambiguous definition makes researchers design something they assumed to
be meaningful but not perceived as real gamification by the users. For example,
in the study of Chen et al. (2015), authors believed that the customised learning
system was with meaningful gamification design, in which students can “create a
successful business and accumulate individual monetary value” (p. 41); however,
participants reported that they were confused. One participant described the sys-
tem as the “elephant in the room” in which everyone “acknowledge(s) it, but no
one consistently explained or understood it” (p. 42). Broad definition results in
similar phenomenon; for example, Lee and Hammer (2011) regarded the gami-
fication concept from a broader perspective by forcing the existing components
in conventional learning contexts into gamification frames: they translated the
grades students receive for their assignments as “points” or “badges”, the upgrade
to next level as “level up”, and the rewards or punishments for the desired/unde-
sired behaviours as “common currency” or “a reward system”. Consequently,
the correspondent studies were still working on the traditional teaching, though
coated with gamification concepts, which results in mixed or confusing effective-
ness results.
Thirdly, previous studies revealed that there was a poor integration of game fea-
tures and pedagogical content. According to Lee and Hammer (2011), one key fea-
ture of game experience is freedom: the freedom “to experiment, to fail, to explore
multiple identities, (and) to control one’s own investment and experience” (p. 4).
However, teaching is kind of making play mandatory by setting rules and goals. By
putting these two things together, the expected “chocolate and peanut butter” turn
out to be “chocolate-covered broccoli” (Lee & Hammer, 2011, p. 4). Simply dress-
ing up an activity as a gamified one is far from being intrinsically engaging (Lieberoth,
2015). This statement explains why some gamified attempts ended up with negative
outcomes: they were just not the true gamification that is able to provide a meaning-
ful gamification experience.
The lack of adaptation was another criticism of gamification in education (Buckley
& Doyle, 2017; Landers & Armstrong, 2017). Most current gamified learning tools
13
serve for average students, which are hardly engaging for high-performing or left-
behind students. At the same time, a limited number of gamified learning tools can
provide learners with tasks “just barely out of reach” to trigger flow status (Cain &
Piascik, 2015, p. 3).
Fifthly, there were high entry barriers to design adequate tools to support educa-
tional gamification (Cain & Piascik, 2015; Chen et al., 2015). Educators with little
or no training in computer science are very possible to create “bad games” “with lit-
tle or no effect on learning” (Cain & Piascik, 2015, p. 4). Again, the involved gami-
fied learning tools might be not mature enough to support true gamification that is
able to provide meaningful gamification experiences.
A considerable number of researchers criticised that there were methodological
limitations in most existing studies of gamification in education, including short
experiment timeframe, small sample size, absence of control groups, ignorance
of novelty impact, the involvement of one single gamification mechanism, and the
lack of the combination of qualitative and quantitative analysis (Chen et al., 2015;
De-Marcos et al., 2014; Seaborn & Fels, 2015; Stanculescu et al., 2016).
4 Findings of the descriptive content analysis
As specified in the section Introduction, the first round of data analysis revealed that
the involvement of different game elements is a significant factor contributing to the
varied effectiveness results. Therefore, another round of content analysis was con-
ducted focusing on inspecting the game elements involved in theory and practice.
Findings indicated that there is a need to distinguish a new term “gamification
mechanisms” from the general concept “game elements”. In considering readability,
this author introduced the findings (redefined terms) before presenting the process,
as detailed in the following sub-sections.
4.1 Game elements and gamification mechanisms: a redefinition
Though Deterding et al. (2011) specified gamification is the use of game elements
in non-game contexts, “there is no universal list of game elements”, which brings
problematic “inherent uncertainty” (Werbach, 2014, p. 267). Because the gamifica-
tion concept originated from full-fledged video games, its elements were originally
borrowed from the game elements in video game design theories. Therefore, previ-
ous gamification studies involved video-game-related elements, such as avatar, boss
fights, combat, and virtual goods (Buckley & Doyle, 2017).
Notably, on the other hand, other researchers proposed to deprive the gamifica-
tion concept from video games, by dividing game elements into two groups: the ele-
ments which are obviously game-like, and the elements which are not. For example,
Denden et al. (2017) believed that game elements can be divided into “game mechan-
ics” and “game dynamics”, with game mechanics are close to the components of tra-
ditional video games (levels, points, badges, virtual goods, leaderboards, and feed-
back) and game dynamics are those close to the needs of people (progression, status,
13
achievement, altruism, competition, and self-expression). Blohm and Leimeister

(2013) supported the idea that “game mechanics” and “game dynamics” consist of
the general idea of “game elements”, but its corresponding components are differ-
ent. According to Blohm and Leimeister (2013), the game mechanics contain seven
elements: documentation of behaviour, scoring systems/badges/trophies, rankings,
levels or reputation points, group tasks, time pressure, and avatars/virtual worlds/
virtual trade. The game dynamics also contain seven elements, namely explora-
tion, collection, competition, acquisition of status, collaboration, challenge, and
development/organization.
Holding the same viewpoint, Chou (2015) proposed the terms “explicit gamifi-
cation” and “implicit gamification”. Explicit gamification involves strategies uti-
lising applications that are obviously game-like, while implicit gamification refers
to human-focused design that utilises game elements in non-game contexts. Chou
(2015) highlighted the importance of implicit gamification, saying that implicit
gamification focuses on individuals’ natural desires, provides motivational drives,
and eventually results in human behaviours. Chou (2015) made a statement that eve-
rything people do is based on one or more of the implicit gamification components,
and if there is a lack of one or more implicit gamification components, there will be
no motivation, nor the expected behaviours.
However, though Chou (2015) clearly concluded that explicit gamification
involves “obviously game-like strategies” while implicit gamification involves
“human-focused design that involves game elements”, the specific indication of
“strategies” or “game elements” remain obscure.
This author proposes to use the term “game elements” to refer to the obviously
game-like strategies that construct explicit gamification, and the term “gamification
mechanisms” to refer to the human-focused designs that help realise implicit gamifi-
cation. To be more specific, this author prefers to redefine the two terms, with game
elements referring to the obvious game-like elements that are frequently used in typ-
ical digital or board games, while gamification mechanisms indicating the underly-
ing guidelines that make an activity engaging, which originate from human desires.
The term gamification mechanisms appeared in previous studies; however, the
explanations were highly inconsistent. For example, Schacht and Schacht (2012)
described gamification mechanisms as the summarised name of a cluster including
“achievements, bonuses, levelling up, progression, leaderboards, cascading informa-
tion and virality” and “additional” ones when needed (p. 195). From this perspec-
tive, gamification mechanisms are similar to game elements. Arakawa and Matsuda
(2016) described gamification mechanism as the providence of tasks, rewards and
communication which triggers value, ideology and fun. According to Arakawa and
Matsuda (2016), gamification mechanism is a process rather than a concretised item.
What’s worse, to this author’s knowledge, no researchers secured the definition of
gamification mechanism yet.
To minimise confusion, this author redefined the term gamification mechanisms
after analysing the selected articles (see Table 3). Since the definitions of game ele-
ments and gamification mechanisms overlap in previous studies, this author also
redefined the term game elements. The detailed process will be presented in the fol-
lowing three sub-sections.
13
4.2 Gamification mechanisms proposed in theory: a framework construction
This section aims to explore “what makes gamification engaging for learners” in the
literatures of the selected articles. Based on the content analysis results, this author
summarised a framework with eight components, including goal, visualisation, imme-
diate feedback, adaptation, challenge, competition, reward, and fun failure (see Fig. 1).
Several researchers discussed the importance of goals, feedback, and flow in edu-
cational gamification. According to McGonigal (2011), the key features of gamifi-
cation are goals, rules, a feedback system and voluntary participation. Since McGonigal
(2011) was a digital game designer, though s/he emphasises the use of digital games
in non-game contexts, the four features still tend to serve for fully-developed dig-
ital games. Cain and Piascik (2015) discussed the five essential elements of seri-
ous games for pharmacy education, which contains the four features proposed by
McGonigal (2011) and one more named flow.
Flow is a concept frequently appeared in gamification studies. First dis-
cussed by Csikzentimihalyi (1975), flow refers to a state in which the users are so
involved in something that they could even feel a reduced awareness of self or time
(Csikzentimihalyi, 1975; Hamari & Koivisto, 2014; Kirriemuir & McFarlane, 2004).
Flow states often occur when the tasks are not too easy or too difficult, which enable
the users to involve in the tasks “on the height of his/her skills” (Hamari & Koivisto,
2014, p. 134). In digital games contexts, flow occurs when players are given goals
and objectives that are not too challenging (which provokes anxiety) nor too simple
(which triggers boredom) for their current skills (Ibanez et al., 2014).
This author did not consider flow as a game mechanic but as a pursuit: designers
or instructors are not able to provide flow; instead, they provide game elements or
gamification mechanisms to induce gamification experience (Werbach, 2014), which
is possible to trigger the flow status. Nevertheless, this author found that several
Fig. 1 The gamification mecha-

nisms framework: what makes
gamification engaging?
13
principles contribute to flow status, including immediate feedback, adaptation, and

challenge. Morris et al. (2013) emphasised the importance of providing adaptive
learning activities or materials based on learners’ level of mastery and Cain and
Piascik (2015) proposed that students should receive “dynamic and immediate feed-
back” regarding their skills, knowledge, strategies, etc. (p. 3). Namely, adaptation is
a comparatively novel concept indicating the providence of customised and dynamic
task difficulties in the gamified learning context. Adaptation addresses the criticism
of gamification’s poor applicability in educational practices. As criticised by Lister
(2015) and Hakulinen and Auvinen (2014), most of the educational tools serve aver-
age students only, leaving high-performance students enduring slow-paced learn-
ing progress while low-performance students suffering from the lack of scaffolding
guidance. Adaptation enables each learner to focus on tasks that are not too easy or
too difficult based on the diagnosis of learners’ competency. From this perspective,
immediate feedback forms the basis of adaptation, adaptation provides challenges,
and adequate challenges trigger flow state, therefore the three principles established
an engagement loop for the possible establishment of flow status.
Cain and Piascik (2015) nominated an uncommon but important feature of success-
ful gamification: fun failure. Players in digital games never worry about failure since
the playing process is fun and repeatable: if they fail, they could start over. This kind
of anxiety-free environment also triggers learner motivation in educational contexts.
Researchers also emphasised the importance of visualised progression (Hamari
et al., 2014; Morris et al., 2013), competition (Hamari et al., 2014; Hanus & Fox,
2015) and rewards (Hamari et al., 2014; Ibanez et al., 2014). These three character-
istics were frequently be achieved with the use of the three popular game elements:
points, badges, and leaderboards (PBL).
This author excluded several other principles or theories due to varied research
scopes. For example, McGonigal (2011) and Kim and Lee (2015) highlighted the
importance of rules, voluntary participation, graphics and sound effects, which are
not necessarily the key components of gamification in educational contexts; they
also supported that “curiosity” and “fantasy” are essential characteristics for com-
puter games, which are more related to user experience rather than gamification
mechanisms; Koivisto and Hamari (2014) believed successful gamification need to
include three components: the affordances implemented to a system or service lead
to psychological outcomes, the gameful experience lead to behaviour outcomes, and
the activities and behaviours that the gamification aims to support and motivate. To
take a closer look at these three components, this author noticed that they are gen-
eral concepts rather than game elements or gamification mechanisms.
Based on the aforementioned analysis, this author simplified the mechanisms that
make gamification engaging in a framework, as illustrated in Fig. 1.
4.3 Game elements and gamification mechanisms involved in empirical studies
Table 4 presents the game elements or gamification mechanisms applied to the

selected 23 empirical studies. The most frequently used game element was badges
13
Table 4 The game elements or gamification mechanisms applied in the selected empirical studies
Game element/Gamification mechanism Frequency Proportion
(All = 23)
Badges (trophies) 13 57%

Leaderboard (ranking) 9 39%
Points (XP) 6 26%
Interaction, level, progress bar 2 9%
Achievement, adaptation, challenge, competition, content 1 4%
unlocking, fantasy, feedback, goal, student control, uncertainty,
virtual currency
(also named trophies), which has been used in 13 articles among 23. The next two
popular game elements were leaderboard (also named ranking) and points (also
named experience, abbreviated as XP), with nine and six articles testing their
effectiveness. Consistent with previous findings, points, badges and leaderboards
have been identified as the most popular game elements (de los Arcos et al., 2017;
Dicheva et al., 2015).
Other game elements or gamification mechanism were not as popular as PBL
(points, badges and leaderboard), as only three game elements or gamification
mechanisms appeared twice in the 23 studies (interaction, level and progress bar),
and all others only appeared once (achievement, adaptation, challenge, competition,
content unlocking, fantasy, feedback, goal, student control, uncertainty and virtual
currency).
4.4 Game elements and gamification mechanisms: definition, feature

and examples
It is obvious that the proposed effective game elements or gamification mechanisms

in theory (see Fig. 1) are not those frequently used in practice (see Table 4). Several
game elements or gamification mechanisms do not even appear in the 23 selected
empirical studies, such as goal, visualisation, competition and reward and fun fail-
ure. However, taking a closer look at the results, this author noticed that the com-
ponents in Fig. 1 and Table 4 explain each other: it seems that components in Fig. 1
reflect the eventual expectations, while components in Table 4 tend to be specific
tools to help achieve the expectations. For example, points and badges present goals
and serve as rewards; leaderboard illustrates competition; at the same time, all the
three elements of PBL provide visualised feedback.
As discussed in previous sections, this author prefers to redefine the two terms,
with game elements referring to the obvious game-like elements that are frequently
used in typical digital or board games, while gamification mechanisms indicating
the underlying guidelines that make an activity engaging, which originate from
human desires.
What’s more, the game elements and gamification mechanisms are different in
types of nouns, either concrete or abstract. Concrete and abstract nouns are nouns
13
Table 5 Game elements and gamification mechanisms: definition, feature, and examples

Term Definition Feature Examples
Game elements The obvious game-like ele- Concrete nouns Points, badges, leaderboard,
ments that are frequently that are obvi- trophies, levels, tasks,
used in digital games or ously game- scoring systems, avatar,
gamification. like boss fight, virtual goods,
virtual trade, alternative
activities, group
Gamification mechanisms The underlying guidelines Abstract nouns Feedback, achievement
that make gamification that relate to (accomplishment),
activities engaging, human needs meaning, self-expression,
which originate from or desires competition, curiosity,
human needs and desires. exploration, collabora-
tion, fantasy, challenge,
self-determination,
social influence (social
recognition), unpredict-
ability (uncertainty), fun
(fun failure), avoidance,
scarcity, ownership,
engagement, enjoyment,
interaction, user control
that people can or cannot see, hear, smell, touch, and taste. There are eight abstract
nouns in Table 4, including interaction, achievement, competition, challenge, fan-
tasy, feedback, student control and uncertainty. When reference to the original notes
for literature, this author noticed that almost all the abstract nouns were the sum-
maries initialled by authors rather than the name of specific elements presented in
the gamified learning tools. For example, Papastergiou (2009) used a system named
LearnMem1 to teach computer memory concepts, which involved an abstract noun
“fantasy”. To realise fantasy, learners in the system LearnMem1 are initially placed
in one room “in the form of maze” and they are expected to assume “the role of a
hero” by answering questions (p. 4). Instead of the abstract noun fantasy, the actual
elements presented in the system LearnMem1 are concrete maze and avatar. There-
fore, this author summarised that the game elements are concrete nouns while gami-
fication mechanisms are abstract nouns. The definition, feature and examples of the
two terms are shown in Table 5.
5 Conclusion
The current study is a descriptive content analysis based on 44 articles selected from
the outputs of a bibliometric analysis software HistCite. The results showed that
most previous researchers considered engagement as the key indicator in measur-
ing the effectiveness of educational gamification, and they mainly focused on the
assessment of students’ behavioural engagement, affective engagement, and aca-
demic performance. Academic performance was widely-treated equivalent with
cognitive engagement; however, if treat these two concepts as separate ones, cogni-
tive engagement was under-studied in educational gamification research. Previous
13
studies reported confounding results in regard to the effectiveness of educational

gamification, as some concluded that the results are positive, some were negative,
some brought partial or mixed results, and some others demonstrated no differences.
Several factors were considered contributing to varied effectiveness results, such as
gender, game experience, individual differences, and resistance to new technologies.
Besides, there are also barriers that hindering educational gamification to release its
full potential, including inappropriate or superficial use of gamification techniques,
ambiguous or broad definition of the term, poor integration of game features and
pedagogical content, difficulties in designing adaptive gamification tools for edu-
cation, heavy reliance on computer science skills, and methodological limitations
(absence of control groups, small sample size, etc.).
Content analysis of the selected articles revealed several research directions for
future studies, as shown below:
Test the effectiveness of individual game element or gamification mechanism in

learning Since gamification can take many forms and can combine game design
elements in many different ways, it is inappropriate to study the motivational effects
of gamification as a generic construct (Sailer et al., 2017; Seaborn & Fels, 2015).
Therefore, researchers could experimentally investigate the effectiveness of individ-
ual game element on learning motivation and performance (Deterding et al., 2011;
Linehan et al., 2011; Mekler et al., 2017). This goal can be achieved by removing
certain game elements from a gamified learning tool and compare the results of con-
trolled and experimental groups (Seaborn & Fels, 2015).
This author noticed that previous studies investigated the impact of individual game
elements, including points (Attali & Arieli-Attali, 2015), badges (Anderson et al.,
2013; Antin & Churchill, 2011; Botra et al., 2014; Bustillo et al., 2017; Hakulinen
et al., 2013; Hakulinen et al., 2015; Hamari, 2017; Kyewski & Krämer, 2018; Sailer
et al., 2017), leaderboard (Landers et al., 2017; Pedersen et al., 2017; Sailer et al.,
2017), performance graphs (Sailer et al., 2017), and the combination of points,
badges and leaderboard (PBL) (Mekler et al., 2017). A very limited number of stud-
ies investigated the impact of individual gamification mechanisms, such as achieve-
ment (Groening & Binnewies, 2019) and competition (Sepehr & Head, 2013).
Future studies could lay more emphasis on investigating the impact of an individ-
ual game element or gamification mechanisms, especially those besides PBL. Even
more attention can be address to understanding how gamification mechanisms work
in learning practices and impact the final results.
Investigate how gamification impacts different types of students Though some

researchers held a positive attitude to the effectiveness of educational gamifica-
tion, they still call for cautions in implementing this approach as not everyone ben-
efits from it (Buckley & Doyle, 2017; Hwang et al., 2012). There is a need to study
learner differences and how they influence the effectiveness of educational gamifi-
cation. Notably, it is common to build an involuntary connection between learner
differences and player types, especially the gamification user type measured by the
24-item Gamification User Types Hexad Scale (Tondello et al., 2016); however, Luo
13
et al. (2020) criticised the rationality of categorising students into a certain number
of groups, since there is a lack of theoretical coherence or reliability in making the
categorisation, while learners are possible to be negatively influenced by the accom-
panying self-suggestion (which make the learners to behave in the way they thought
they are expected to, namely unconsciously put themselves into pre-set stereotypes).
Luo et al. (2020) suggested to fix the gap by focusing on “learner difference”, which
avoids categorising students into groups; instead, it investigates learners’ prefer-
ences and the providence of customised learning materials. The learner difference
study also applies to the refinement of educational gamification.
Investigate the methods assessing and promoting cognitive engagement in edu-

cational gamification The results showed that previous studies have laid too much
attention on measuring students’ behavioural and affective engagement with cog-
nitive engagement understudied (e.g., focused attention, self-reflection, analysis,
evaluation, etc.), if exclude academic performance from the indicators of cognitive
engagement (Hew et al., 2016). This author proposes future studies to explore three
issues, including “is there any method to assess the cognitive engagement in educa-
tional gamification”, “whether the involvement of behavioural and affective engage-
ment connected with positive learning outcomes or enhanced learning experiences”,
and “how reliable and valid are the assessments of cognitive engagement in educa-
tional gamification”.
Explore the techniques of gamifying class without digital games (Sánchez Mena &
Martí Parreño, 2017) As highlighted by Hanus and Fox (2015), one important con-
ceptual implication of gamification is that “game” is not necessary for gamified
learning. Rather than using digital games or other forms of games, teachers could
gamify the learning process and therefore “make the class itself a game” (Hanus &
Fox, 2015, p. 152). In this way, gamification could break the constraints of game
features such as high cost, heavy reliance on technology, time-wasting, and poor
integration with pedagogical curriculum (Cain & Piascik, 2015; Fernandes et al.,
2012; Lee & Hammer, 2011; Morris et al., 2013).
What’s more, future studies could focus on the investigation of factors contrib-
uting to teachers’ acceptance intention to gamification in the classroom (Adukaite
et al., 2017; Bourgonjon et al., 2013; Vandercruysse et al., 2013).
Besides, this author proposes to differentiate gamification mechanisms from the
general game elements, as highlighted in the section Introduction and Findings.
There is a gap in the communication between educationalists and game designers:
educationalists firstly consider the expected results (abstract gamification mecha-
nisms) in designing a gamified tool, while game designers intuitively make connec-
tions with specific game elements. When the two terms indicating different things,
the communication cost between educationalist and game designers is huge. Educa-
tionalists could potentially be unhappy with products with pointsfication only (the
use of points only in gamification), while game designers complain about “unreal-
istic requests” such as providing fantasy or adding fun failure effects. Accordingly,
future studies that hope to improve the effectiveness of gamified learning could
13
work on issues such as “to achieve one certain gamification mechanism, the use of
which game elements could be useful” or “to use one certain game element, which
gamification mechanisms will be triggered”. Just match the two concepts together,
the effectiveness of gamified learning tools will be much more guaranteed, as well
as the communication between educationalists and game designers.
Though rarely proposed in previous studies, this author believes that there is a
lack of standardised measures or easy-to-use techniques to assess the effective-
ness of educational gamification, such as a validated scale to measure the perceived
usefulness or the perceived limitations. All et al. (2016) recommended developing
a common methodology to assess and compare the effectiveness of digital game-
based learning, which is also understudied in the field of educational gamifica-
tion. Future studies could also focus on the investigation of factor contributing to
users’ acceptance willingness, because the effectiveness of educational gamification
will not be optimised if the users are reluctant to accept it (Adukaite et al., 2017;
Bourgonjon et al., 2013; Vandercruysse et al., 2013).
One arguable weakness of the current review is the involvement of publications
from the one dataset only. At the same time, though successfully removed inappro-
priate keywords in pilot studies, this author did not include “learning” or “teaching”
in the current research. Systematic literature reviews could be conducted based on
literature collected from more databases with the use of a more accurate Boolean
searching strategy.
Notwithstanding these limitations, this study is significant in the following
aspects: it redefined the term game elements and gamification mechanisms, synthe-
sised the effectiveness of gamified learning in the selected empirical studies, dis-
cussed the methods to assess the effectiveness of educational gamification, enlisted
the factors and barriers leading poor effectiveness results, and provided potential
research areas for future studies. Results of this study help enhance the under-
standing of the current situation of educational gamification, as well as facilitating
researchers to quickly grasp the hotspots in the field.
Abbreviations PBL: Points, badges, and leaderboard
Authors’ contributions Zhanni Luo as the sole author is responsible for the completion of the current
manuscript including research design, data collection, data analysis, manuscript draft, revision, and
proofreading.
Data availability The datasets generated and analysed during the current study are available from the cor-
responding author on reasonable request.
Declarations
All procedures performed in studies involving human participants were in accordance with ethical stand-
ards. Informed consent was obtained from all individual participants included in the study.
Competing interests This author declares that there are no competing interests.
13
References
Adukaite, A., van Zyl, I., Er, Ş., & Cantoni, L. (2017). Teacher perceptions on the use of digital gamified
learning in tourism education: The case of South African secondary schools. Computers & Educa-
tion, 111, 172–190. https://doi.org/10.1016/j.compedu.2017.04.008.
All, A., Castellar, E. P. N., & Van Looy, J. (2016). Assessing the effectiveness of digital game-based
learning: Best practices. Computers & Education, 92, 90–103. https://doi.org/10.1016/j.compedu.
2015.10.007.
Anderson, A., Huttenlocher, D., Kleinberg, J., & Leskovec, J. (2013). Steering user behavior with badges.
Paper presented at the Proceedings of the 22nd international conference on World Wide Web.
https://doi.org/10.1145/2488388.2488398.
Antin, J., & Churchill, E. F. (2011). Badges in social media: A social psychological perspective. Paper
presented at the CHI 2011 Gamification Workshop Proceedings.
Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psycho-
logical engagement: Validation of the Student Engagement Instrument. Journal of School Psychol-
ogy, 44(5), 427–445. https://doi.org/10.1016/j.jsp.2006.04.002.
Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical
conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369–386.
https://doi.org/10.1002/pits.20303.
Arakawa, Y., & Matsuda, Y. (2016). Gamification mechanism for enhancing a participatory urban sens-
ing: Survey and practical results. Journal of Information Processing, 24(1), 31–38. https://doi.org/
10.2197/ipsjjip.24.31.
Arici, F., Yildirim, P., Caliklar, Ş., & Yilmaz, R. M. (2019). Research trends in the use of augmented
reality in science education: Content and bibliometric mapping analysis. Computers & Education,
142, 103647. https://doi.org/10.1016/j.compedu.2019.103647.
Attali, Y., & Arieli-Attali, M. (2015). Gamification in assessment: Do points affect test performance?
Computers & Education, 83, 57–63. https://doi.org/10.1016/j.compedu.2014.12.012.
Barata, G., Gama, S., Jorge, J., & Gonçalves, D. (2017). Studying student differentiation in gamified edu-
cation: A long-term study. Computers in Human Behavior, 71, 550–585. https://doi.org/10.1016/j.
chb.2016.08.049.
Bartolini, M., Bottani, E., & Grosse, E. H. (2019). Green warehousing: Systematic literature review and
bibliometric analysis. Journal of Cleaner Production, 226, 242–258. https://doi.org/10.1016/j.jclep
ro.2019.04.055.
Blohm, I., & Leimeister, J. (2013). Gamification: Design of IT-based enhancing services for motivational
support and behavioral change. Business and Information Systems Engineering, 5(4), 275–278.
https://doi.org/10.1007/s12599-013-0273-5.
Botra, A., Rerselman, M., & Ford, M. (2014). Gamification beyond badges. Paper presented at the 2014
IST-Africa Conference Proceedings. https://doi.org/10.1109/ISTAFRICA.2014.6880651.
Bourgonjon, J., De Grove, F., De Smet, C., Van Looy, J., Soetaert, R., & Valcke, M. (2013). Acceptance
of game-based learning by secondary school teachers. Computers & Education, 67, 21–35. https://
doi.org/10.1016/j.compedu.2013.02.010.
Boykin, A. W., & Noguera, P. (2011). Creating the opportunity to learn: Moving from research to prac-
tice to close the achievement gap. Ascd.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychol-
ogy, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa.
Buckley, P., & Doyle, E. (2017). Individualising gamification: An investigation of the impact of learning
styles and personality traits on the efficacy of gamification using a prediction market. Computers &
Education, 106, 43–55. https://doi.org/10.1016/j.compedu.2016.11.009.
Budd, J. M. (1988). A bibliometric analysis of higher education literature. Research in Higher Education,
28(2), 180–190. https://doi.org/10.1007/BF00992890.
Bustillo, J., Rivera, C., Guzmán, J. G., & Acosta, L. R. (2017). Benefits of using a mobile application
in learning a foreign language. Sistemas & Telemática, 15(40), 55–68. https://doi.org/10.18046/syt.
v15i40.2391.
Cain, J., & Piascik, P. (2015). Are serious games a good strategy for pharmacy education? American
Journal of Pharmaceutical Education, 79(4), 47. https://doi.org/10.5688/ajpe79447.
13
Chen, Y., Burton, T., Mihaela, V., & Whittinghill, D. (2015). Cogent: a case study of meaningful gamifi-
cation in education with virtual currency. iJET–International Journal of Emerging Technologies in
Learning, 10, 133–147. https://doi.org/10.3991/ijet.v10i1.4247.
Chou, Y.-K. (2015). Actionable gamification: Beyond points, badges, and leaderboards. Octalysis Group.
Csikzentimihalyi, M. (1975). Beyond boredom and anxiety: Experiencing flow in work and play.
Jossey-Bass.
de los Arcos, B., Faems, B., Comas-Quinn, A., & Pulker, H. (2017). Teachers’ use and acceptance of
gamification and social networking features of an open repository. European Journal of Open, Dis-
tance and E-learning, 20(1), 127–138. https://doi.org/10.1515/eurodl-2017-0008.
Deci, E. L., & Ryan, R. M. (2000). The" what" and" why" of goal pursuits: Human needs and the self-
determination of behavior. Psychological Inquiry, 11(4), 227–268. https://doi.org/10.1207/S1532
7965PLI1104_01.
De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study com-
paring gamification and social networking on e-learning. Computers & Education, 75, 82–91.
https://doi.org/10.1016/j.compedu.2014.01.012.
Denden, M., Tlili, A., Essalmi, F., & Jemni, M. (2017). An investigation of the factors affecting the
perception of gamification and game elements. Paper presented at the 2017 6th International
Conference on Information and Communication Technology and Accessibility (ICTA). https://
doi.org/10.1109/ICTA.2017.8336019.
Denny, P. (2013). The effect of virtual achievements on student engagement. Paper presented at the
Proceedings of the SIGCHI conference on human factors in computing systems. https://doi.org/
10.1145/2470654.2470763.
Deterding, S., Sicart, M., Nacke, L., O’Hara, K., & Dixon, D. (2011). Gamification. using game-
design elements in non-gaming contexts. Paper presented at the CHI’11 extended abstracts on
human factors in computing systems. https://doi.org/10.1145/1979742.1979575.
Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Gamification in education: A systematic
mapping study. Educational Technology & Society, 18(3), 75–88.
DomíNguez, A., Saenz-De-Navarrete, J., De-Marcos, L., FernáNdez-Sanz, L., PagéS, C., & Mar-
tíNez-HerráIz, J.-J. (2013). Gamifying learning experiences: Practical implications and out-
comes. Computers & Education, 63, 380–392. https://doi.org/10.1016/j.compedu.2012.12.020.
Fernandes, J., Duarte, D., Ribeiro, C., Farinha, C., Pereira, J. M., & da Silva, M. M. (2012). iThink: A
game-based approach towards improving collaboration and participation in requirement elicita-
tion. Procedia Computer Science, 15, 66–77. https://doi.org/10.1016/j.procs.2012.10.059.
Fleming, N. D. (2001). Teaching and learning styles: VARK strategies. IGI global.
Garfield, E., Paris, S., & Stock, W. G. (2006). HistCiteTM: A software tool for informetric analysis of
citation linkage. Information Wissenschaft und Praxis, 57(8), 391–400.
Ge, X., & Ifenthaler, D. (2018). Designing engaging educational games and assessing engagement in
game-based learning. In Gamification in Education: Breakthroughs in Research and Practice
(pp. 1–19): IGI Global. https://doi.org/10.4018/978-1-5225-5198-0.ch001.
Goksu, I., Ozkaya, E., & Gunduz, A. (2020). The content analysis and bibliometric mapping of CALL
journal. Computer Assisted Language Learning, 1–31. https://doi.org/10.1080/09588221.2020.
1857409.
González, C. S., Gómez, N., Navarro, V., Cairós, M., Quirce, C., Toledo, P., & Marrero-Gordillo, N.
(2016). Learning healthy lifestyles through active videogames, motor games and the gamification
of educational activities. Computers in Human Behavior, 55, 529–551. https://doi.org/10.1016/j.
chb.2015.08.052.
Groening, C., & Binnewies, C. (2019). “Achievement unlocked!”-The impact of digital achievements
as a gamification element on motivation and performance. Computers in Human Behavior, 97,
151–166. https://doi.org/10.1016/j.chb.2019.02.026.
Hakulinen, L., & Auvinen, T. (2014). The effect of gamification on students with different achievement
goal orientations. Paper presented at the 2014 international conference on teaching and learning
in computing and engineering. https://doi.org/10.1109/LaTiCE.2014.10.
Hakulinen, L., Auvinen, T., & Korhonen, A. (2013). Empirical study on the effect of achievement
badges in TRAKLA2 online learning environment. Paper presented at the 2013 Learning and
teaching in computing and engineering. https://doi.org/10.1109/LaTiCE.2013.34.
Hakulinen, L., Auvinen, T., & Korhonen, A. (2015). The effect of achievement badges on students’
behavior: An empirical study in a university-level computer science course. International
13
Journal of Emerging Technologies in Learning (iJET), 10(1), 18–29. https://doi.org/10.3991/ijet.

v10i1.4221.
Hamari, J. (2013). Transforming homo economicus into homo ludens: A field experiment on gamifi-
cation in a utilitarian peer-to-peer trading service. Electronic Commerce Research and Applica-
tions, 12(4), 236–245. https://doi.org/10.1016/j.elerap.2013.01.004.
Hamari, J. (2017). Do badges increase user activity? A field experiment on the effects of gamification.
Computers in Human Behavior, 71, 469–478. https://doi.org/10.1016/j.chb.2015.03.036.
Hamari, J., & Koivisto, J. (2014). Measuring flow in gamification: Dispositional flow scale-2. Com-
puters in Human Behavior, 40, 133–143. https://doi.org/10.1016/j.chb.2014.07.048.
Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does Gamification Work?-A Literature Review of Empir-
ical Studies on Gamification. In 2014 47th Hawaii international conference on system sciences
(pp. 3025–3034). IEEE. https://doi.org/10.1109/HICSS.2014.377.
Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal
study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance.
Computers & Education, 80, 152–161. https://doi.org/10.1016/j.compedu.2014.08.019.
Hew, K. F., Huang, B., Chu, K. W. S., & Chiu, D. K. (2016). Engaging Asian students through game
mechanics: Findings from two experiment studies. Computers & Education, 92, 221–236. https://
doi.org/10.1016/j.compedu.2015.10.010.
Huang, C., Yang, C., Wang, S., Wu, W., Su, J., & Liang, C. (2020). Evolution of topics in education
research: a systematic review using bibliometric analysis. Educational Review, 72(3), 281–297.
https://doi.org/10.1080/00131911.2019.1566212.
Hwang, G.-J., Sung, H.-Y., Hung, C.-M., Huang, I., & Tsai, C.-C. (2012). Development of a personalized
educational computer game based on students’ learning styles. Educational Technology Research
and Development, 60(4), 623–638. https://doi.org/10.1007/s11423-012-9241-x.
Ibanez, M.-B., Di-Serio, A., & Delgado-Kloos, C. (2014). Gamification for engaging computer science
students in learning activities: A case study. IEEE Transactions on Learning Technologies, 7(3),
291–301. https://doi.org/10.1109/TLT.2014.2329293.
Kim, J. T., & Lee, W.-H. (2015). Dynamical model for gamification of learning (DMGL). Multimedia
Tools and Applications, 74(19), 8483–8493. https://doi.org/10.1007/s11042-013-1612-8.
Kirriemuir, J., & McFarlane, A. (2004). Literature review in games and learning. Futurelab Series, 1–35.
Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from gamification.
Kyewski, E., & Krämer, N. C. (2018). To gamify or not to gamify? An experimental field study of the
influence of badges on motivation, activity, and performance in an online learning course. Comput-
ers & Education, 118, 25–37. https://doi.org/10.1016/j.compedu.2017.11.006.
Landers, R. N., & Armstrong, M. B. (2017). Enhancing instructional outcomes with gamification: An
empirical test of the Technology-Enhanced Training Effectiveness Model. Computers in Human
Behavior, 71, 499–507. https://doi.org/10.1016/j.chb.2015.07.031.
Landers, R. N., Bauer, K. N., & Callan, R. C. (2017). Gamification of task performance with leader-
boards: A goal setting experiment. Computers in Human Behavior, 71, 508–515. https://doi.org/10.
1016/j.chb.2015.08.008.
Lee, J., & Hammer, J. (2011). Gamification in education: What, how, why bother? Academic Exchange
Quarterly, 15(2), 1–5.
Lepper, M. R., Greene, D., & Nisbett, R. E. (1973). Undermining children’s intrinsic interest with extrin-
sic reward: A test of the" overjustification" hypothesis. Journal of Personality and Social Psychol-
ogy, 28(1), 129–137. https://doi.org/10.1037/h0035519.
Lieberoth, A. (2015). Shallow gamification: Testing psychological effects of framing an activity as a
game. Games and Culture, 10(3), 229–248. https://doi.org/10.1177/1555412014559978.
Linehan, C., Kirman, B., Lawson, S., & Chan, G. (2011). Practical, appropriate, empirically-validated
guidelines for designing educational games. Paper presented at the Proceedings of the SIGCHI con-
ference on human factors in computing systems. https://doi.org/10.1145/1978942.1979229.
Lister, M. (2015). Gamification: The effect on student motivation and performance at the post-secondary
level. Issues and Trends in Educational Technology, 3(2), 1–22. https://doi.org/10.2458/azu_itet_
v3i2_lister.
Luo, Z., O’Steen, B., & Brown, C. (2020). The use of eye-tracking technology to identify visualisers and
verbalisers: Accuracy and contributing factors. Interactive Technology and Smart Education, 17(2),
229–247. https://doi.org/10.1108/ITSE-12-2019-0087.
13
Martí-Parreño, J., Méndez-Ibáñez, E., & Alonso-Arroyo, A. (2016). The use of gamification in education:
a bibliometric and text mining analysis. Journal of Computer Assisted Learning, 32(6), 663–676.
https://doi.org/10.1111/jcal.12161.
McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world.
Penguin.
Mekler, E. D., Brühlmann, F., Tuch, A. N., & Opwis, K. (2017). Towards understanding the effects of
individual gamification elements on intrinsic motivation and performance. Computers in Human
Behavior, 71, 525–534. https://doi.org/10.1016/j.chb.2015.08.048.
Morris, B., Croker, S., Zimmerman, C., Gill, D., & Romig, C. (2013). Gaming science: the “Gamifi-
cation” of scientific thinking. Frontiers in Psychology, 4, 607. https://doi.org/10.3389/fpsyg.2013.
00607.
NFER. (2009). NFER Teacher Voice Omnibus February 2009 Survey- Using computer games in the
classroom. Retrieved from https://www.nfer.ac.uk/media/2752/futl27teachervoicereport.pdf.
Osipov, I. V., Volinsky, A. A., Nikulchev, E., & Prasikova, A. Y. (2016). Communication and gami-
fication in the web-based foreign language educational system: Web-based foreign language
educational system. International Journal of Web-Based Learning and Teaching Technologies
(IJWLTT), 11(4), 22–34. https://doi.org/10.4018/IJWLTT.2016100102.
Pan, X., Yan, E., Cui, M., & Hua, W. (2018). Examining the usage, citation, and diffusion patterns
of bibliometric mapping software: A comparative study of three tools. Journal of Informetrics,
12(2), 481–493. https://doi.org/10.1016/j.joi.2018.03.005.
Papastergiou, M. (2009). Digital game-based learning in high school computer science education:
Impact on educational effectiveness and student motivation. Computers & Education, 52(1),
1–12. https://doi.org/10.1016/j.compedu.2008.06.004.
Pedersen, M. K., Rasmussen, N. R., Sherson, J. F., & Basaiawmoit, R. V. (2017). Leaderboard effects
on player performance in a citizen science game. arXiv preprint arXiv:1707.03704.
Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25(4), 348–
349. https://doi.org/10.1108/eb026482.
Rai, V., & Beck, A. L. (2017). Play and learn: serious games in breaking informational barriers in
residential solar energy adoption in the United States. Energy Research & Social Science, 27,
70–77. https://doi.org/10.1016/j.erss.2017.03.001.
Riva, S., Camerini, A.-L., Allam, A., & Schulz, P. J. (2014). Interactive sections of an Internet-based
intervention increase empowerment of chronic back pain patients: randomized controlled trial.
Journal of Medical Internet Research, 16(8), e180. https://doi.org/10.2196/jmir.3474.
Rotgans, J. I., & Schmidt, H. G. (2011). Cognitive engagement in the problem-based learning
classroom. Advances in Health Sciences Education, 16(4), 465–479. https://doi.org/10.1007/
s10459-011-9272-9.
Sailer, M., Hense, J. U., Mayr, S. K., & Mandl, H. (2017). How gamification motivates: An experi-
mental study of the effects of specific game design elements on psychological need satisfaction.
Sánchez Mena, A. A., & Martí Parreño, J. (2017). Drivers and barriers to adopting gamification:
teachers’ perspectives. Electronic Journal of e-Learning, 15(5), 434–443.
Schacht, M., & Schacht, S. (2012). Start the game: Increasing user experience of enterprise systems
following a gamification mechanism. In Software for People (pp. 181–199): Springer. https://doi.
org/10.1007/978-3-642-31371-4_11.
Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal
of Human-Computer Studies, 74, 14–31. https://doi.org/10.1016/j.ijhcs.2014.09.006.
Sepehr, S., & Head, M. (2013). Competition as an element of gamification for learning: an explora-
tory longitudinal investigation. Paper presented at the Proceedings of the First International Con-
ference on Gameful Design, Research, and Applications. https://doi.org/10.1145/2583008.25830
09.
Silpasuwanchai, C., Ma, X., Shigemasu, H., & Ren, X. (2016). Developing a comprehensive engage-
ment framework of gamification for reflective learning. Paper presented at the Proceedings of
the 2016 ACM Conference on Designing Interactive Systems. https://doi.org/10.1145/2901790.
2901836.
Song, Y., Chen, X., Hao, T., Liu, Z., & Lan, Z. (2019). Exploring two decades of research on classroom
dialogue by using bibliometric analysis. Computers & Education, 137, 12–31. https://doi.org/10.
1016/j.compedu.2019.04.002.
13
Stanculescu, L. C., Bozzon, A., Sips, R.-J., & Houben, G.-J. (2016). Work and play: An experiment in
enterprise gamification. Paper presented at the Proceedings of the 19th ACM Conference on Com-
puter-Supported Cooperative Work & Social Computing. https://doi.org/10.1145/2818048.2820061.
Su, C.-H., & Cheng, C.-H. (2013). A mobile game-based insect learning system for improving the learn-
ing achievements. Procedia-Social and Behavioral Sciences, 103, 42–50. https://doi.org/10.1016/j.
sbspro.2013.10.305.
Tang, S. H., & Hall, V. C. (1995). The overjustification effect: A meta-analysis. Applied Cognitive Psy-
chology, 9(5), 365–404. https://doi.org/10.1002/acp.2350090502.
Thom, J., Millen, D., & DiMicco, J. (2012). Removing gamification from an enterprise SNS. Paper pre-
sented at the Proceedings of the acm 2012 conference on computer supported cooperative work.
https://doi.org/10.1145/2145204.2145362.
Tondello, G. F., Wehbe, R. R., Diamond, L., Busch, M., Marczewski, A., & Nacke, L. E. (2016). The
gamification user types hexad scale. Paper presented at the Proceedings of the 2016 annual sympo-
sium on computer-human interaction in play. https://doi.org/10.1145/2967934.2968082.
Van Rosmalen, P., Wilson, A., & Hummel, H. G. (2015). Games for and by teachers and learners. In
Gamification: Concepts, Methodologies, Tools, and Applications (pp. 1706-1733): IGI Global.
https://doi.org/10.4018/978-1-4666-8200-9.ch086.
Vandercruysse, S., Vandewaetere, M., Cornillie, F., & Clarebout, G. (2013). Competition and students’
perceptions in a game-based language learning environment. Educational Technology Research and
Development, 61(6), 927–950. https://doi.org/10.1007/s11423-013-9314-5.
Werbach, K. (2014). (Re) defining gamification: A process approach. Paper presented at the International
conference on persuasive technology. https://doi.org/10.1007/978-3-319-07127-5_23.
Yee, N. (2006). Motivations for play in online games. Cyberpsychology & Behavior, 9(6), 772–775.
https://doi.org/10.1089/cpb.2006.9.772.
Zawacki-Richter, O., Anderson, T., & Tuncay, N. (2010). The growing impact of open access distance
education journals: A bibliometric analysis. Journal of Distance Education, 24(3), 54–73.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published
maps and institutional affiliations.
13

Luo 2022

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Luo 2022

Uploaded by

Copyright:

Available Formats

Education and Information Technologies

Gamification for educational purposes: What are the factors

Received: 25 March 2021 / Accepted: 16 June 2021

Keywords Gamification · Gamified learning · Effectiveness assessment · Learning

RQ 1: How effective the gamification implementations were in previous empir-

“Gamification mechanism” in RQ 4 is a new term proposed by this author

2.2 Data collection and data presentation

2.4 Inclusion and exclusion criteria

purposes is currently an interdisciplinary area that involves researchers of varied back-

3 Findings of the systematic literature review

3.1 The effectiveness of educational gamification

Table 1 The reported effectiveness of educational gamification in empirical studies

Number (out of 23) 11 2 3 2 2 3

effectiveness of an individual mechanic “badges” on a trading website Sharetribe.

reported mixed results on the effectiveness of gamified learning activities targeted

Gamified learning with slight or no effects Surprisingly, there were also stud-

3.2 The measures of the effectiveness of educational gamification

Though numerous benefits of educational gamification have been nominated, the

conceptualisations of engagement categories, a frequently-used theory is the three-

Silpasuwanchai et al., 2016). Academic assessments indeed involve cognitive activi-

3.3 Factors contributing to varied effectiveness results of educational

3.3.1 Factors affecting the effectiveness of gamification in education

In regard to factors affecting the effectiveness of educational gamification, five

Table 2 Factors and barriers influencing the effectiveness of gamified learning

1) Gender 1) Inappropriate or superficial use of gamification

3.3.2 Barriers to reaching consistent effectiveness results in educational

Inappropriate or superficial use of gamification techniques was an obvious theme

Table 3 Redefinition of game elements and gamification mechanisms

The ambiguous or broad definition of related terms also impedes consistency in

4 Findings of the descriptive content analysis

4.1 Game elements and gamification mechanisms: a redefinition

achievement, altruism, competition, and self-expression). Blohm and Leimeister

4.2 Gamification mechanisms proposed in theory: a framework construction

Fig. 1 The gamification mecha-

principles contribute to flow status, including immediate feedback, adaptation, and

4.3 Game elements and gamification mechanisms involved in empirical studies

Table 4 presents the game elements or gamification mechanisms applied to the

Badges (trophies) 13 57%

4.4 Game elements and gamification mechanisms: definition, feature

It is obvious that the proposed effective game elements or gamification mechanisms

Table 5 Game elements and gamification mechanisms: definition, feature, and examples

studies reported confounding results in regard to the effectiveness of educational

Test the effectiveness of individual game element or gamification mechanism in

Investigate how gamification impacts different types of students Though some

Investigate the methods assessing and promoting cognitive engagement in edu-

Abbreviations PBL: Points, badges, and leaderboard

Journal of Emerging Technologies in Learning (iJET), 10(1), 18–29. https://​doi.​org/​10.​3991/​ijet.​

You might also like

2.2 Data collection and data presentation

2.4 Inclusion and exclusion criteria

3 Findings of the systematic literature review

3.1 The effectiveness of educational gamification

Table 1 The reported effectiveness of educational gamification in empirical studies

3.2 The measures of the effectiveness of educational gamification

3.3 Factors contributing to varied effectiveness results of educational

3.3.1 Factors affecting the effectiveness of gamification in education

Table 2 Factors and barriers influencing the effectiveness of gamified learning

3.3.2 Barriers to reaching consistent effectiveness results in educational

Table 3 Redefinition of game elements and gamification mechanisms

4 Findings of the descriptive content analysis

4.1 Game elements and gamification mechanisms: a redefinition

4.2 Gamification mechanisms proposed in theory: a framework construction

Fig. 1 The gamification mecha-

4.3 Game elements and gamification mechanisms involved in empirical studies

4.4 Game elements and gamification mechanisms: definition, feature

Table 5 Game elements and gamification mechanisms: definition, feature, and examples

Journal of Emerging Technologies in Learning (iJET), 10(1), 18–29. https://doi.org/10.3991/ijet.