IMPACT OF LEARNING BADGES IN A SERIOUS GAME

A. Teles Vieira
CITI - Universidade Nova de Lisboa (PORTUGAL)

Abstract
There are several experiences in the framework of serious games. The use of learning badges has
been a recurring in this area. When you are playing there are some elements insusceptible to create
motivation. This can be the case of learning medals (learning badges). These are considered a
premium and are used by students to show that they meet their objectives. For some authors, badges
can provide focus on goals, more challenging tasks, better educational performance, power of choice
and authenticity.
This paper focuses on the controlled experiment in a school with adolescent students. The serious
game "Alpha Patrol" has been tested and analyzed in various aspects. In this case, it is important to
report the analysis done and the results obtained regarding the learning badges.
This paper intends to reflect how the implementation of the learning badges in the serious game may
not be as promising or advantageous as expected.
In this way, the article pretends to show that the effective utilization of the learning badges, as an
object of motivation, may perhaps not be as an agent of motivation as we may think.
Keywords: Serious games, learning badges, experience, motivation.

1 INTRODUCTION
When playing there are some elements precluded to build motivation. This is the case of the learning
badges. These are considered a prize and are used by students to show that they have fulfilled their
objectives. These kind of medals started with the Open Source Community Mozilla Firefox in 2011, the
year in which was launched the first set of badges.
In recent years, many were the badges implemented either by the Foundation Mozilla Firefox, or by
other entities that followed the trend: “the Open Badges framework is designed to allow any learner to
collect badges from multiple sites, tied to a single identity, and then share them out across various
sites, including personal blogs to social networking channels. It is critical for this infrastructure to be
open to give learners control over their own learning and credentials, allow anyone to issue badges,
and for each learner to carry their badges with them across the Web and other contexts.”[1]
The main objective is for each student to use their badges to demonstrate the different goals [1]. The
player collects the badges and puts them in his/her virtual backpack and is then able to show them in
any digital platform as a personal website, social network or on a blog, “a collection of badges can
function as a distributed portfolio that may eventually be accessible from a variety of social media
sites, such as LinkedIn, Facebook, and Google Plus. When badges serve as part of a résumé or
portfolio, they tell prospective employers a more detailed story about the projects and activities that
define a student’s learning, including both the hard and soft skills that were acquired.” [2]
With this "digital backpack" and the demonstration of badges, one seeks to recognize the capabilities
that are not easily assessed on a scale of traditional teaching. These acquired skills are recognized
through the badges and are achieved, most of the times, outside the classroom context.
EDUCAUSE website refers that these medals of learning “offer talking points for a job interview,
opening a friendly dialogue between a manager and an interviewee and allowing the latter to speak
about accomplishments and interests that might not otherwise arise in conversation”. [3]

2 LEARNING BAGDES AND MOTIVATION

There are several institutions that use badges, for example of the Khan Academy, which uses badges
to motivate their students: “offers a lengthy web page of them, such as an “Atomic Clockwork” badge,
which requires a student to watch videos or hone a skill for each of 30 consecutive days”. [3]
Also the MITx uses badges in their study groups. In OpenStudy whenever a student gives a proper
and timely reply, there is a reward in form of a badge.
Various projects were recognized by the use of digital badges in education in the annual projects
competition on Digital Badges, John D. and Catherine T. MacArthur Foundation, Gates Foundation
and Mozilla. In 2012, in the badges competition for Lifelong Learning, there were over 2 million dollars
in research scholarships for the development of innovative systems using badges.
With the construction of new usage models for badges it is expected that the idea to deliver a symbolic
'prize' by the goals achieved can positively influence the involvement and learning. For example, the
badges can provide focus on objectives, more challenging tasks, better educational performances,
power of choice and authenticity [4].
Caryn Swark refers that “badges are just another form of grading, or - worse - extrinsic rewards (…)
Badges can be extrinsic rewards, but they can also provide intrinsic motivation, especially for students
who like clearly spelled out expectations. Provided they're optional, they shouldn't negatively impact
any student's learning.”[5].
However, Resnick denotes that, “the problem, for me, lies in the role of badges as motivators. In many
cases, educators are proposing badge systems in order to motivate students. It’s easy to understand
why educators are doing this: most students get excited and engaged by badges. But towards what
end? And for how long? I worry that students will focus on accumulating badges rather than making
connections with the ideas and material associated with the badges (...).” [6]
For its part, Henry Jenkins refers that “there is a value in helping these youths find ways to value what
they are doing as intellectual pursuits, and there is a value in seeking to validate these experiences
and help them learn how to mobilize that knowledge as they learn to work through the formal
structures that exert power over their lives. But making badges too central to the process may alienate
them before they have a chance to exert ownership over the knowledge they are acquiring." [7]
It is important to mention that the empirical studies reflect on the effects of the badges in education,
Abramovich et al (2013) analyze the relationship of motivation with badges “the motivation relationship
between badges and motivation for low performing students was limited to participatory badges. Skill
badges earned by the low-performing students did not correlate with the change in performance
avoidance goals. Not only do we conclude that different types of badges will have different effects on
student motivation to learn but we also conclude that different types of badges will also affect learning
performance”. [8].
On the other hand Epper, Derryberry e Jackson (2012) identify five trends that can boost the adoption
of games in learning:
● The students' expectations. Digital natives grew up with computer games. The time
spent during the game can be considered an opportunity to learn, and the different learning
opportunities may be present in games with distinctive characters. The simulation can be
considered as the heart of the game “Digital simulations provide a learning environment in
which the student can practice difficult, exacting, life - threatening, or mission - critical skills.
(...) we are reminded that it is often the quality of the insight rather than the fidelity of the
representation that most often determines the appeal of a game. This explains the continued
popularity of games such as Go, Chess, Bridge, Monopoly, and other board games. High -
fidelity simulations paired with gameplay dynamics and competitive game elements can
increase student engagement and enhance learning.“ [9]
● Data analysis: games are a constant interaction and can help in obtaining significant
data about the student’s - what is his/her main difficulties, which is his/her skills, how he/she
works individually and as a group. These data in conjunction with other may be the key to
building a pedagogical approach and disciplinary action to transform the curriculum to be more
appealing to students.
● Badges (medals) for learning: the digital learning badges can operate as a data
collector of each student “While badges for learning are not, in and of themselves, games, the
activities (usually called challenges, missions, or quests) that support the acquisition of
badges can have a game - like quality. Further, the skill - building structure of games — which
is both recursive and accretive — pairs naturally with badges for learning.” [9]
● Mobile devices: the access to mobile devices has been increasing among students.
The use of the QR code, geotagging, online check-in and accelerometers, are characteristics
 that encourage the use of games in the educational context. The use of these tools can
improve the use of games and make the experience more enjoyable.
● Increasing use of social media: two to three thirds of Internet users use social media:
make purchases, share photos and videos and use online games applications. “As social
media tools become more thoroughly integrated throughout postsecondary education, their
usage within game - based learning is inevitable.” [9]

3 METHODOLOGY
As stated the use of badges can boost the adoption of games in learning. In this paper we present the
serious game “Alpha Patrol”. So, it is important to report the analysis done and the results obtained
regarding the learning badges.
A controlled experiment with the use of the serious game “Alpha Patrol” took place in School EB 2, 3
Filipa de Lencastre in Lisbon. The class, chosen randomly, was the 7ºA with 30 students: 18 students
were female and 12 male students. However, and as the experience took place in the School Library,
4 other students wanted to join the experience: 4 male students from class 7ºC.
First, we asked the students to fill, individually, the original inquire on expectations, game habits, prior
knowledge on the subject and the use of games in the classroom context. It was a short survey with
only 8 mandatory answers required.
After the game – it was used an offline version of the game with the levels unlocked so they could
have a preview at the entire game – the students responded to the final survey. This survey, filled out
individually, aimed to analyze what they had learned, their general thoughts on the game and the
relationship they established with the professors Isabel Alçada and Ana Maria Magalhães’ book. The
final survey was a little more extensive: 30 questions - 29 compulsory and 1 free response question
(and not compulsory) for comments and suggestions.
An essential aspect of this game is the quizzes: small multiple-choice questionnaires that allow the
player to move forward in the game and get the badges related to the question/answer system.
In the construction of a game are several characteristics that may be created and added.
The badges work by learning objectives, i.e., in the badges menu can we identify the badges available
and what the player should do to get them. Therefore, the medals are of different categories, which
recognize different skills of various: exploration, security, speed, combat and knowledge acquired.
This is once again essential for this game to be integrated in the serious game group.
There were created 49 learning badges for this game. For the player to achieve the badges he/she
should follow the indications. The following table explains what the player should do to achieve each
medal:

Learning Medal What to Do

The restless Give 5 hops

The Patient Spend 2 minutes at any level of the game without

doing anything.

The Very Patient Spend 10 minutes at any level of the game

without doing anything.

The incredibly Patient Spend 30 minutes at any level of the game

without doing anything.

The Without life Spend 2 hours at any level of the game without
doing anything.

The does not stop quiet Give 50 hops

And who do not jump is not of the team! Give 200 jumps

Premium Nelson Évora Give 500 jumps

Where there is no smoke fire Achieve the armature of fire.

They have sown the wind shall reap the Achieve the armature of wind.
whirlwind

Puts the wax, takes away the wax Attack 20 times without weapon.

Young grasshopper Attack 80 times without weapon.

Karate Kid Tackle 250 times without weapon.

Kung Fu Panda Tackle 400 times without weapon.

Soldier Shoot 12 times.

Sargeant Shoot 30 times.

Lieutenant Shoot 60 times.

Lieutenant General Shoot 150 times

The defender of the earth Destroy 1 enemy

Steven Seagel 5 Destroy enemies

Chuck Norris Destroy 20 enemies

Terminator Destroy 50 enemies

Treasure Hunter Find 2 treasures

Explorer of maps Find 5 treasures

Pedro Alvares Cabral Find 9 treasures

Vasco da Gama Find 18 treasures

John Indian Find 27 treasures

Even now started Conclusion of the first part of level 1.

London is already Completion of level 1.

Smells well, smell of Lisbon Completion of level 2.

Domo arigato Mr Roboto Completion of level 3.

Tekken the Take more than 4 minutes to finish a level

Zzzzzzzzzzzzzzzzzzz Take more than 10 minutes to finish a level

Obikwelo Take less than 4 minutes to finish a level

 Usain bolt Take less than 2 minutes to finish a level

The incredible flash Take less than 1 minute to end a level

The Pacifist Ending a level without destroying more than 2

enemies

The Hippie Ending a level without using the weapon

Poor in gold but rich in dreams Ending a level without catching treasures

The Cult Set in 1 quiz

The ensured Set in 3 quizzes

The Responsible Set in 4 quizzes

The Intelligent Set in 6 quizzes

Note Maximum Set in 15 quizzes

Irresponsible Err 2 quizzes

I will draw your Guardian Err 15 quizzes

The Salvador Redeem 2 people

Freedom Redeem 5 people

Mother Teresa of Calcutta Redeem 15 people

Figure 1 – Learning badges

These badges have as objective to recognize competences, “digital badges are a portable way to
recognize achievement; any organization, application, or platform can easily issue and display them.
And organizations that issue digital badges increase their potential impact by reaching new audiences
and providing learning opportunities that can be recognized.” [10]
The use of these custom badges may help to improve the skills of each player. However, it is noted
that badges may recognize the good skills and also recognize the difficulties of the player regarding
the type of game played. For example, if the player takes a long time to a win a level or miss the
questions proposed. In this sense, the learning badges can be used for to study if the player can or
should improve his/her performance in the scope of the game and the set of knowledge acquired in
the game course.

4 RESULTS
After the application of the pilot session and the surveys, the last two questions on the user experience
subject, we asked each respondent to indicate the three strong points of the game and the three
weaknesses of the game within the categories listed: characters, story, game levels, learning badges,
quizzes, animations, struggles, 3D graphics, armours, challenges and robots.
The three strengths identified were: 41.2% (N=14) the armours; 28.2% (N=13) the challenges and as a
third strong point ex exequo, with 35.3% (N=12), were the categories of history, game levels and
quizzes.

Figure 2 Alpha Patrol badges

 Figure 2 - Three strengths

The three weaknesses identified were: 52.9% (N=18) to the robots; 47.1% (N=16) to the characters
and, with 38.2% (N=13), the quizzes.

Figure 3 - Three weaknesses

After the results one aspect that deserves consideration is the integration of the learning badges. In
fact, in the conceptualization of the project, when integrating the learning badges as motivating agents,
the research team thought they would be more appreciated. However, it was found that students
considered it a minor aspect in the learning acquisition: despite the initial interest, the players did not
consider the badges interesting enough for collecting. They were more interested in getting ahead in
the narrative of the game.
This aspect forces us to reflect on the reasons that may cause badges not to be considered as
elements of motivation of a serious game. In the specific case of the controlled experiment, both the
lack of visibility of the learning badges in the serious game, or indifference by the possible advantages
of badges may have contributed to the low compliance. The age range can be one of the reasons for
the lack of interest in the acquisition and demonstration of badges. Davis & Klein (2015), about a study
with secondary school pupils, reported that “the youth participants in the current investigation were not
interested in using badges to connect their science center experiences with their school lives, and they
did not want to share these experiences with their friends or family members. They did value the ability
to use their badges in the college admissions process, but they raised concerns that such external
audiences may find it difficult to assess the worth of a particular badge without some level of
standardization in the badge awarding process.” [11]
However, we cannot fail to mention that there is still a way to go when it comes to analysis of the
learning badges as an agent of motivation. Davis & Klein (2015) mention “considerable interpretative
work is required to make sense of badge collections, raising challenges related to validating the
credibility of badges earned in diverse contexts.” [11]
5 CONCLUSIONS
It is hoped that for future investigations, the use of learning badges is deepened in order to identify
elements of motivation in the use of serious games. The use of learning badges depending on the
target audience is to be emphasized. It became clear, however, that the type of usage that may be
developed, is directly related to the audience. As regards, Easley and Ghosh (2016) state that “It is
important to note here that different users may value these badges differently; in fact, it is conceivable
that there are subpopulations of users who value absolute accomplishment more highly than relative
accomplishment, and vice versa. If this is indeed the case, the nature of the user population attracted
to the site may vary endogenously depending on the nature of the reward offered by the site.” [12]
Thus, understanding the relationship between the users and the badges is critical in future studies
“developing such a nuanced understanding of how users value the social-psychological rewards
created by badges is essential to designing effective reward mechanisms for incentivizing user
contribution and participation on the Web.” [12]

REFERENCES
[1] The Mozilla Foundation and peer 2 peer University (2012) Open badges for Lifelong Learning.
Retrieved from https://wiki.mozilla.org/images/b/b1/openbadges- working-paper_092011.pdf.
A.A. Author, "Journal/Conference Article Title," Periodical Title, vol. Volume, no. Issue, pp.-pp.,
Publication Year.
[2] JACKSON, N., (2013) The Developmental Challenge: An Ecological Perspective. Retrieved from
http://www.academia.edu/7794659/The_Developmental_Challenge_An_Ecological_Perspective
[3] EDUCAUSE (2012) 7 things you should know about badges, Retrieved from
http://net.educause.edu/ir/library/pdf/eli7085.pdf.
[4] DICKEY, M. (2005). Engaging by design: how engagement strategies in popular computer and
video games can inform instructional design. Educational Technology Research and
Development
[5] SWARK, C. (2012) P2pu,mozilla, open badges... Oh my!, Retrieved from
http://gamifymyclass.blogspot.ca/
[6] RESNICK, M. (2012) Still a badge skeptic. HASTAC, Retrieved from
http://hastac.org/blogs/mres/2012/02/27/still-badge- skeptic.
[7] ASH, K. (2012). Digital badges would represent students’ skill acquisition. Education week.
Retrieved from: http://www.edweek.org/dd/articles/2012/06/13/03badges.h05.html?tkn=orsf1
[8] ABRAMOVICH, S., SCHUNN, C., HIGASHI R., (2013) Are badges useful in education?: it
depends upon the type of badge and expertise of learner. Retrieved from
http://www.lrdc.pitt.edu/schunn/research/papers/Abramovich-Schunn-Higashi.pdf
[9] EPPER, R., DERRYBERRY, A., JACKSON, S., (2012) Game-based Learning. Retrieved from
https://net.educause.edu/ir/library/pdf/ERB1208.pdf
[10] FINKELSTEIN, J., KNIGHT E., MANNING S., (2013) «The Potential and Value of Using Digital
Badges for Adult Learners». American Institutes for Research. Retrievede from
https://lincs.ed.gov/publications/pdf/AIR_Digital_Badge_Report_508.pdf
[11] DAVIS, K., KLEIN, E., (2015) Investigating High School Students’ Perceptions of Digital Badges in
Afterschool Learning. Retrieved from
http://katiedavisresearch.com/wp-content/uploads/2014/01/2015_CHI_davis.klein_.badges.pdf
[12] EASLEY, D., GHOSH, A., (2016) Incentives, gamification, and game theory: An economic
approach to badge design. ACM Trans. Econ. Comput.