Computers in Human Behavior 45 (2015) 213221

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier.com/locate/comphumbeh

The effect of a persuasive social impact game on affective learning

and attitude
Dana Ruggiero ,1
Purdue University, Curriculum and Instruction, 100 University Street, West Lafayette, IN 47905, United States

a r t i c l e i n f o a b s t r a c t

Article history: To investigate whether a persuasive social impact game may serve as a way to increase affective learning
Available online 31 December 2014 and attitude towards the homeless, this study examined the effects of persuasive mechanics in a video
game designed to put the player in the shoes of an almost-homeless person. Data were collected from
Keywords: 5139 students in 200 middle/high school classes across four states. Classes were assigned to treatment
Affective learning groups based on matching. Two treatment conditions and a control group were employed in the study.
Attitude All three groups affective learning and attitude scores decreased from the immediate posttest but the
Hierarchical liner modeling
game group was signicantly different from the control group in a positive direction. Students who
Persuasive games
played the persuasive social impact game sustained a signicantly higher score on the Affective Learning
Scale (ALS) and the Attitude Towards Homelessness Inventory (ATHI) after three weeks. Overall, ndings
suggest that when students play a video game that is designed using persuasive mechanics an affective
and attitude change can be measured empirically.
2014 Elsevier Ltd. All rights reserved.

1. Introduction gap in empirical knowledge about how persuasive games actively

In the games research community the power of games to
change an emotive process such as attitude and affect is subject
to continuous debate. Researchers invested in games as a signi-
cant medium of learning are often faced with the difculty of argu-
ing both sides of an argument. One on side, games are being
attacked as the cause of youth violence (Engelhardt, Bartholow,
Kerr, & Bushman, 2011) leading games scholars to be on the defen-
sive. On the other side, games are held up as creative solutions for
learning from early education to corporate training (Gee, 2007;
Ruggiero, 2014). While researchers advocating for either of these
positions have to be prepared to accept the ethical implications
of the other, a horse of a new color has appeared on the games
horizon-persuasive games.
Video games are a vibrant and integral part of society that is
growing, and it is important to study their possible effects on
affective behavior. Using theoretical models and case studies, aca-
demics have shown that different elements of persuasion can be
used to dene elements in video games that make them persuasive
(Bogost, 2007; Evans, 2011; Ruggiero, 2013). However there is a
Tel.: +1 765 414 9475.
E-mail addresses: druggier@purdue.edu, d.ruggiero@bathspa.ac.uk
1
Permanent Address: Bath Spa University, School of Education, Newton St. Loe,
Bath BA29BN, UK. Tel.: +44 07447 790738.
affect the affective learning process and attitude.
A relatively new eld in games research, persuasive games, or
games meant to change the attitude or behavior of the player
through game play (Bogost, 2007; Ruggiero, 2012) have the
potential to act as powerful vehicles for learning through
persuasive mechanics. To explore the idea of a persuasive game
as a game changer for attitude and affective learning, the current
research investigated the effect of one game, Spent developed by
the Durham Homeless Coalition, across 5139 students in 200
classrooms spanning four states.
Before discussing the methods and results of this study it is nec-
essary to understand the background of persuasive games and
those specically designed as vehicles of social impact. Persuasive
games have been studied in various contexts with differing
denitions over the past forty years. Predating the invention of
the computer, humans have used play and games for teaching nec-
essary skills and socialization for millennia (Abt, 1970; Huizinga,
1955). Games explicitly created to change attitudes and behavior
date back to 1790, when British publishers of the New Game of
Human Life advised parents to play the board game with their chil-
dren and request their attention to a few moral and judicious
observations explanatory of each character as they proceed &
contrast the happiness of a virtuous and well spent life with the
fatal consequences arriving from vicious & immoral pursuits
(Lepore, 2007 para. 3).

http://dx.doi.org/10.1016/j.chb.2014.11.062
0747-5632/ 2014 Elsevier Ltd. All rights reserved.
214 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221
In 1843 a board game released in the US called Mansion of Hap-
piness gave instructions that instructed players to make good and
moral decisions to gain the seat of happiness. The 1960s and
1970s witnessed a surge of multiplayer simulations. Given
credibility by the Rand Foundation, which developed a number of
persuasive games for use in the Cold and Vietnam wars, most of
these were intended for education, training, and exploring
alternative courses of action (Abt, 1970) with some persuasive
purposes. Persuasive games today are a growing part of video game
landscapes. They have attracted the attention of the media
(Ochalla, 2007), academics (Bogost, 2003, 2007, 2008; Frasca,
2001; Gee, 2003, 2007), and funding agencies such as the MacAr-
thur Foundation (MacArthur Foundation, 2012).
1.1. How persuasive games persuade
Attempts at analyzing persuasion date back to ancient Greece;
according to Aristotle persuasion is achieved through rhetoric,
and three parts that include ethos, pathos, and logos (Cooper,
1932). Ethos uses claims about the persuaders moral character
and his or her trustworthiness, an important aspect of the persua-
sion process if it is to be effective (Evans, 2011). Pathos is an emo-
tional appeal to secure the goodwill of the listener while logos is
the reasoned argument that appeals to the listeners rational mind.
Aristotles categorization has been elaborated over time but is still
useful for analysis of persuasion (Bogost, 2007). Factors such as the
interest of receivers of the message, their level of education, their
knowledge of the issue, their cultural background, their feelings
about the originator of the message, the medium used for the per-
suasive message, and competing factors all inuence the success or
failure of an attempt at persuasion.
Persuasive games use several mechanisms of persuasion stem-
ming from advertising and marketing that have been posited by
game researchers, including: immersion, ow, engagement, per-
suasive rhetoric, and persuasive ethos.
1.1.1. Immersion
Immersion is the experience of being transported to an elabo-
rately simulated place that takes over all attention and becomes
enveloping (Murray, 1997). Technology has increased the power
of immersion through video games, it seems that games as per-
suasive technology hold much promise for changing peoples atti-
tudes: games are by nature interactive, and people tend to retain
more impressions (Khaled, 2007, p. 17).
1.1.2. Flow
Flow has been a theory posited by some game theorists (Amory,
2006) that games are compelling because players are in a highly
energized state of concentration and focus (Csikszentmihalyi,
1990). Flow is achieved when the level of the challenge and the
level of the players skill in dynamic tension, creating a highly
focused state of mind. Amory (2006) posits that the player can
assimilate tacit knowledge through the process that is then assim-
ilated and constructed after emerging from the state of ow.
1.1.3. Engagement
Engagement is closely related to ow where the player nds the
game so engrossing that they assimilate facts and values without
realizing they are doing so (Quinn, 2005). Research (Padgett,
Strickland, & Coles, 2006; Thomas & Cahill, 1997; Tuzun, 2007)
supports the claims that games increase engagement through ow,
immersion, and agency. Accordingly when players are more
engaged with the game, they are more likely to see the situation
from the perspective presented in the game.
1.1.4. Procedural rhetoric and ethos
Both procedural rhetoric and ethos are Aristotelian theories of
persuasion updated by Bogost (2007) and Evans (2011). While
Bogost denes procedural rhetoric as the practice of using pro-
cesses persuasively (2007, p. 2), Evans argues that rhetoric is
not enough and includes ethos or persuasion by empathy, fact,
and integrity (2011, p. 71). By demonstrating that every action
in a game has consequences, which are built into the game struc-
ture by the designers, the rhetoric and ethos of these procedures
not only allow the player to learn through game play but also are
a more effective and longer-lasting way of assimilating informa-
tion (Bogost, 2007). Evans (2011) argues that rhetoric alone is
not enough and examines not only the message being sent but also
the messenger and the nal outcome for integrity, empathy, and
fact. Procedural rhetoric and ethos both play a part in the overall
impact of persuasive games.
1.2. Measuring impact of persuasive games
Measuring the impact of a persuasive social impact game
remains a gap in the current research. How many students who
play Food Force, a game that puts them in the shoes of a World
Health Organization food worker and that has been downloaded
4 million times, retain an interest in the politics of food distribu-
tion after game play concludes? Americas Army, a game developed
by the U.S. Army, has had players dedicate more than 160 million
hours of game time (Clarren, 2006) but how many of those players
actually enlist in the Army? Games that are intended to lead to
actions are easier to evaluate because you can measure the effect.
For example, the effectiveness of a game that is aimed at persuad-
ing people to visit a website can be calculated by the number of
players who clicked through from the game to the website.
When measuring a game without such concrete goals, such as
the intention to inuence the players change in affective learning
and attitude towards the homeless by experiencing what its like to
be almost homeless, this inuence is more difcult to measure. In
order to measure a change in affective learning and attitude
towards the homeless by playing a persuasive social impact game
the rst step is to establish whether or not there is a signicant
change when playing a game designed to be extremely persuasive
through the use of both mechanics and topic. Thus, the following
research questions were proposed:
R1: To what extent does affective learning differ immediately and
three weeks after playing the persuasive social impact game Spent
for the game treatment group, controlling for gender and hours
playing video games, as measured by the Affective Learning Scale
(ALS) compared to the control and comparison groups?
R2: To what extent does attitude towards the homeless differ
immediately and three weeks after playing the persuasive social
impact game Spent for the game treatment group, controlling for
gender and hours playing video games, as measured by the Attitude
Towards the Homeless Inventory (ATHI) compared to the control
and comparison groups?
2. Methodology
2.1. Participants
This study used a quasi-experimental design with control and
two treatment groups to compare the effects of playing a persua-
sive social impact game on affective learning and attitude towards
homelessness. The population of this study consisted of
adolescents in grades 7 through 12 (ages 1218) in formal
education settings in the Midwestern U.S. The sample consisted
 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221
of public, private, and religious schools in four states of the Mid-
western U.S. Using a purposive sampling (Johnson & Christensen,
2012), an email was sent to all principals of schools that matched
the sampling criteria in Indiana, Michigan, Illinois, and Ohio asking
them to forward a letter of introduction to their teachers.
An a priori power analysis was conducted using G-Power
(Faul, Erdfelder, Lang, & Buchner, 2007) and it was found that a
total of 341 classrooms would be required for a .95 power. Given
the nature of the study it was nearly impossible to get a probabi-
listic representational sample and a total of 346 classrooms
agreed to participate. Ninety-four classrooms agreed to partici-
pate but did not start the project or completed only the demo-
graphic information; thirty-one classrooms did not complete the
three-week posttest; twenty classrooms had fty percent or more
unlabelled student data; and one classroom had only three stu-
dents. In total, 200 classrooms (n = 5139 students) completed
the study. A post hoc power analysis demonstrated that a total
of 200 classrooms resulted in a .81 power. Twenty-four class-
rooms were from religious schools, eleven were from private
schools, and 165 were from public schools. As this study did
not introduce novel teaching methods it was approved through
the Institutional Review Board (IRB) as exempt research and
every teacher who agreed to participate had their entire class
enrolled in the study.
2.2. Design
Pretestposttest control group design (Campbell, Stanley, &
Gage, 1963) occurred in two steps. In the rst step matching was
employed on individual classrooms to ensure that there was equal
representation in the pretest and no pretest groups. Second, ran-
domization occurred for the experimental condition. For example,
using a Solomon design, classes were assigned to game, reading, or
control with a pretest and two posttests: one, immediately after
treatment and a second at three weeks after the treatment (see
Table 1). The pretest was used to ensure that the control and treat-
ment groups were equivalent. Additionally classrooms were
matched using the following variables: state, setting, and grade.
This was done to minimize variance in the scores due to contextual
effects and to create equal numbers for each of the treatment and
control groups. To ensure that across states, setting, and grade
there were an equal number of classrooms participating in each
of the groups the total number of classes participating in each
grade were split by state and the type of school setting. Finally
classes were randomly assigned to one of the three groups (control,
reading, or game) (see Table 2).
2.3. Materials
2.3.1. Affective Learning Scale (ALS)
The dependent variable was dened as the average score on the
Affective Learning Scale (ALS) (Scott & Wheeles, 1975). ALS has
participants indicate their level of agreement with statements
pertaining to the experience of that particular day in class. Scott
Table 1
Solomon design with six groups.
Group Pretest Treatment Post-immediate Post-3 weeks
T1 (Game) O1 X O2 O3
T1 (Game) X O4 O5
Control O6 O7 O8
Control O9 O10
T2 (Reading) O11 X O12 O13
T2 (Reading) X O14 O15
Key: T = Treatment, O = Observation, C = Control.
215
Table 2
Breakdown for control, game, and reading groups.
Group No Pretest Pretest Total
Control 14 10 24
Game 46 70 116
Reading 31 29 60
Total 91 109 200
and Wheeless (1975) reported a .96 Cronbachs alpha reliability
for ALS, demonstrating a very high internal consistency that is
explained by the semantic structure of the survey questions. Preli-
minary analysis of the ALS scores for this study found a similar
high Cronbachs alpha reliability (.944).
2.3.2. Attitude Towards Homeless Inventory (ATHI)
The dependent variable was dened as the total score obtained
by computing responses on the 11-item 6-point Likert scale on the
ATHI (Kingree & Daves, 1997). Initially the ATHI was chosen for its
ability to measure four relevant dimensions of homelessness; (1)
personal causation, (2) societal causation, (3) afliation, and (4)
solutions to homelessness as one score. Four studies were collec-
tively implemented to provide the internal validation for the 11-
item inventory with Cronbachs alpha of .72, indicating relative
consistency of scores across items (Kingree & Daves, 1997). To
examine reliability for the current sample, the measure was
checked after pretest and the reliability coefcient, Cronbachs
alpha, with all 11 items was .633. A correlation analysis later
revealed that one of the items (I feel comfortable sharing lunch
with a homeless person) was not internally consistent with the rest
of the measure for this population.
2.3.3. Independent variables
The independent variables were group membership (i.e., game,
reading, and control group assignments) controlling for gender,
hours playing video games at home, and pretest scores on the
ALS and ATHI. Students were asked to provide demographic infor-
mation such as gender, age, number of hours playing video games,
and volunteer experience with the homeless.
2.3.4. Control, game, and reading group assignment
Using matching protocols (Campbell et al., 1963) classes were
matched rst by variables (e.g., grade, state, setting) and then ran-
domly assigned to either the game, reading, or control group to
ensure equivalency across the participant. This was done to ensure
that an equal number of students were assigned to each of the con-
trol, reading, and game groups from each of the grades, states, and
school settings.
2.3.4.1. The game. Spent is a text based game developed by the
McKinney advertising studio in cooperation with the Durham
Homeless Coalition. The player starts off with a scenario that they
are a single parent, have no job, and just lost their home. The game
play starts when the player is challenged to see if they could sur-
vive on $1000 for the next month. After securing a job the player
is faced with a series of choices including nding housing, paying
for car insurance, joining the union, and buying food. Every deci-
sion has an effect on the players money barometer and the chal-
lenge is to make the choices you would normally make in your
own life but on the limited nancial means that the player cur-
rently has available.
2.3.4.2. The reading. Written in the rst person this account is of
the rst night of homelessness and the experiences that lead to
and follow from living on the edge of nancial ruin. Published by
216 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221

the Hufngton Post the author leads the reader through carrying
all of their belongings, nding a safe place to sleep, getting a job
interview, buying food, and applying for benets. The reading level
of this article is grade 5.

2.4. Procedure

In the fall of 2011 9816 emails were sent to every school in Indi-
ana, Ohio, Illinois, and Michigan asking principals to forward an
email invitation to participate in the study. A total of 200 class-
rooms (n = 5139 students) completed the study by June 2012.
The researcher contacted each teacher that expressed interest in
participating in the study via email in March 2012. Upon receiving
the approval of the Institutional Review Board (IRB) at Purdue the
researcher began recruiting individual classrooms for the experi-
Fig. 1. Example of level 1 model.
ment to be conducted in the spring 2012.
A week prior to the research date, teacher participants received
a reminder email with a basic survey that asked them to provide
classroom data such as volunteer activities related to homeless-
ness, how often they used computer games in their classroom,
and subject area specialization, including whether or not they have
talked about homelessness and any community service projects
that the school is involved in relating to homelessness. This was
used in the data conditioning process to determine if any outliers
were the result of these individual classroom variables. The survey
was in Qualtrics with multiple-choice and open-ended ques-
tions. This information was used to gain a clearer understanding
of how the variables related to the school and classroom settings.
For game and reading groups, on the assigned research day the
teacher escorted their class to the computer lab and started the
study. Teachers read a brief introduction telling the students how
to log on to Qualtrics and take the pretest, the length of time
to play the game or read the story and how to access the posttest.
Each teacher received a short script to be followed for the treat-
Fig. 2. Example of level 2 model.
ment section and the follow up. Students logged onto individual
computers and accessed the Internet. Each student received a slip
of paper that had the URL listed for the game (www.playspent.org) took the ALS and ATHI without playing the game. Each group took
or reading and the tinyURL (www.tinyurl.com) for the Qualtrics the ALS and ATHI three weeks later to test for long-term effect of
survey. First students took the pretest on Qualtrics. They then change in affective learning and attitude towards homelessness.
played Spent or read a rst-person narrative about being homeless
for approximately 30 min. Then they accessed the Qualtrics sur-
2.5. Data analysis
vey site again to complete the posttest measure. The pretest and
posttest measure asked for the students name, gender, school,
Hierarchical Linear Modeling (HLM) was chosen for the data
state, and grade to ensure proper identication of the student on
analysis because the data were reported by students nested in
all three tests. To ensure condentiality surveys were matched
classrooms, with the classroom receiving the assignment of con-
and de-identied by a third party not associated with the research
trol, game, or reading group. HLM is a multi-level multiple regres-
study prior to any data analysis. This occurred for all research
sion technique that is useful in analyzing nested data (Bryk &
groups.
Raudenbush, 1992). In order to proceed with HLM analysis, the
For control groups, on the assigned research day the teacher
number of levels in the data needed to be specied and models
escorted their class to the computer lab and started the pretest.
needed to be constructed. The current study data were best
Students logged onto individual computers and accessed the Inter-
described in two levels: student level (level 1), and classroom
net. Each student received a slip of paper that had the tinyURL for
and teacher level (level 2). Level 1 (see Fig. 1) was represented
the Qualtrics survey. Students accessed the Qualtrics survey
by student background such as; hours playing games at
site to complete the pretest measure. Thirty minutes later they
home and at school, pretest score, and gender which are unique
completed the post test measure after a structured period of free
across students. Teacher and classroom background variables such
time dependent on each teacher. Some examples of free time activ-
as treatment grouping and grade level represented level 2
ities provided by the teachers are silent reading, working on other
(see Fig. 2).
classwork, and playing a game such as Hangman. Three weeks later
the class returned to the computer lab on the assigned research
day. Students logged onto individual computers and accessed the 3. Results
Internet. Each student received a slip of paper that had the tinyURL
for the Qualtrics survey. Students accessed the Qualtrics sur- Playing the persuasive social impact game Spent increased the
vey site to complete the measure. affective learning scores of students both immediately and three
Participants in the game, reading and control groups took the weeks later over the control group and reading group. There was
ALS and ATHI. The treatment groups took the ALS and ATHI directly no immediate change for attitude towards the homeless for the
before and after playing the game or reading and the control group game group, however three weeks later they were signicantly
 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221 217

more positive towards the homeless than the reading or control gender, is not a signicant predictor of the immediate posttest
group. Overall, students who played the game Spent retained a score on ALS, but hours playing games at home and ALS pretest
higher score on the ALS and ATHI than the students in the reading are signicant predictors of ALS immediate posttest score (see
or control groups. Table 3). This may imply that the more hours students play games
at home, the more engaging the game Spent.
3.1. Affective learning Overall, the reading and game groups showed a signicant dif-
ference from the control group score (see Table 3). One thing that is
Students in both the reading and game group increased their not explained by the data is the lower pretest score for the control
affective learning score immediately after treatment and both fac- group. One possible explanation is the lower number of control
tors were statistically signicant in difference from the control group classrooms (n = 24). An additional possible explanation is
group. While students in the reading group scored .667 that teachers informed their students of the control group assign-
(p < .0001) points higher than the control group, the game group ment and this affected the control group score.
had a 1.57 (p < .0001, r = .882) increase over the control group Three weeks after the treatment (see Table 4) the game group
(see Table 3). showed a signicant difference, .736 points (p < .001, r = .507) from
The change from ALS pretest to ALS immediate posttest for the the control group while the reading group did not, .102 points
control group can be explained by carry over effect as these tests (p = .553). One possible explanation for this nding is that the
were taken back to back. The game group gained an approximate game Spent had a sleeper effect on affective learning. All three
2.4 points, which is a signicant change. Additionally, the reading groups decreased in their affective learning scores but only the
group gained an approximate 1.5 points immediately after treat- game group showed a signicant change from the posttest to the
ment. The changes in scores may be explained by the fact that stu- three week posttest as measured by paired sample t-tests. This
dents had to travel to a computer lab and take part in a special could be interpreted to mean that playing the game increases
assignment rather than the normal day-to-day classwork. affective learning temporarily but the effect is not sustained over
In summary the model as a whole is signicant in predicting the the long term.
immediate posttest score on ALS. Individual level 1 variable In summary none of the groups sustain their ALS score from
immediate posttest to three week posttest (see Fig. 3). However,
Table 3 the game group does score signicantly higher on the three week
Output for immediate posttest (ALS).
posttest than the control group. Both the reading group and the
Source DV: immediate Coefcient Standard Approx. P-value control group decrease to pretest level after three weeks and there
posttest ALS error d.f. is no signicant difference between these groups.
Control 3.833 .059 197 p < 0.01
Game 1.578 .170 197 p < 0.01
Reading .667 .180 197 p < 0.01 3.2. Attitude towards the homeless
Gender .057 .048 2330 .244
Games home .011 .005 2330 .042
During the initial analysis of the data the scores from the ATHI
ALS pretest .545 .019 2330 p < 0.01
proved too homogenous to indicate an HLM model, therefore an
ANCOVA was used for analysis. A signicant difference between
groups for immediate posttest score of attitude towards the home-
Table 4 less, and the ANCOVA model as a whole is signicant (p < 0.01). The
Output for three week posttest (ALS). signicance of the model as a whole demonstrates that it is possi-
Source DV: three week Coefcient Standard Approx. P-value ble to predict ATHI immediate posttest score with the included
posttest ALS error d.f. variables. Table 4 shows that TTC group membership (control,
Control 3.026 .046 197 p < 0.01 game, and reading) is a signicant predictor of ATHI immediate
Game .736 .164 197 p < 0.01 posttest (p < 0.01). Hours playing video games at home and at
Reading .102 .173 197 .553 school and gender do not signicantly predict ATHI immediate
Gender .001 .051 3451 .987
posttest score.
Games HOME .003 .003 3451 .241
Games SCH .006 .006 3451 .299 Students in the game group increased their score (see Table 5)
ALS immediate posttest .010 .018 3451 .584 and it was not signicantly different from the control group by
.990 points (p = .115, r = .147), but most interesting is the nding

Fig. 3. ALS scores by test and group.

218 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221

Table 5 Table 7
Immediate posttest ATHI model results. Three week posttest ATHI model results.

Source DV: immediate df Mean F Sig. Source DV: three week df Mean F Sig.
posttest ATHI square posttest ATHI square
Corrected model 6 4823.024 102.441 p < 0.01 Corrected model 6 433.354 6.474 p < 0.01
Intercept 1 25518.167 542.004 p < 0.01 Intercept 1 164777.263 2461.706 p < 0.01
gender 1 33.500 .712 .399 Gender 1 2.296 .034 .853
Games home 1 159.990 3.398 .065 Games home 1 23.516 .351 .553
Games SCH 1 2.886 .061 .804 Games SCH 1 65.425 .977 .323
Pretest 1 26526.386 563.418 p < 0.01 Immediate posttest 1 110.916 1.657 .198
TTC 2 1279.620 27.179 p < 0.01 TTC 2 1147.367 17.141 p < 0.01
Error 2318 47.081 Error 3451 66.936
Total 2325 Total 3458
Corrected total 2324 Corrected total 3457

a. R Squared = .210 (Adjusted R squared = .208). a. R squared = .011 (Adjusted R Squared = .009).

that students in the reading group decreased their score although Table 8
it was not signicantly different from the control group by Three week posttest ATHI results.
1.264 points (p = .057). On average students in the game group
Source DV: three week Coefcient Standard Approx. P-value
scored approximately one point higher than students in the control posttest ATHI error d.f.
group, with ATHI pretest score serving as the largest predictor,
Control 36.397378 .207350 197 p < 0.01
after grouping, of the ATHI immediate posttest score. Game 1.949401 .730179 197 p < 0.01
There are 11 items on the ATHI, with a total possible score of 66. Reading .518693 .768835 197 .500
The control group average score of 37.739 represents a slightly Gender .335573 .305337 2318 .272
higher than median score of the belief that homelessness has soci- Games home .031543 .019336 2318 .103
Games Sch .003325 .037998 2318 .931
etal causes, a willingness to afliate with homeless people, the ATHI immediate posttest .499488 .021568 2318 p < 0.01
belief that homelessness is caused by personal characteristics,
and the belief that homelessness is a solvable problem. Controlling
for all other variables, the game group scored 38.730 and the read-
ing group 36.475 immediately after treatment (see Table 5). The group shows a signicant difference from the control group. The
only signicant difference from the control group was the reading reading group students showed a signicant difference from the
group and that group had a negative change from the pretest and a control group from pretest to immediate posttest where the score
negative difference from the control group. This was not expected decreases. From immediate posttest to three week posttest the
but could imply that reading about being homeless decreased stu- scores increase slightly but are not signicantly different from
dents attitude towards the homeless. the control group.
Three weeks after treatment, grouping (TTC) remains the only In summary the ANCOVA model for the ATHI three week post-
factor signicantly able to predict the individual score on the three test is a signicant predictor of three week posttest score. Addi-
week posttest (see Table 6). However, the overall model (p < .001), tionally, the game group differs signicantly from the control
is able to predict the ATHI three week posttest score. Students in group after three weeks (p = .009, r = .286), but the reading group
the game group had a base score of 1.94 points (p = .009) higher does not (p = .500). This suggests that it was not the topic, home-
than control group students and 1.472 points higher than the read- lessness that acted as the catalyst for a sustained higher score on
ing group (Table 7). The reading group had .518 points higher than the ATHI three week posttest score, but the persuasive social
the control group but this was not found to be signicant (p = .5). impact game mechanics of Spent.
These ndings could be interpreted to mean that students in the
game group scored higher on the ATHI and maintain a higher score 4. Discussion
on the ATHI after three weeks. The change from immediate post-
test to three week posttest are statistically signicant and the First and foremost this study is signicant because it is cur-
scores are signicantly different from the control group (see rently one of very few studies that empirically measured the per-
Table 8). suasive effectiveness of a game aimed at a social impact and
Fig. 4 illustrates how the scores from each group changed from found statistically signicant results. There are multiple anecdotal
test to test. Visual examination demonstrates that both the game reports and case studies available on social impact games, but this
and control groups had an increase in scores after the treatment is one of the rst that took a large-scale quantitative approach to
on the immediate posttest. The increase of the game group was sig- measuring affective learning and attitude towards the homeless
nicant. From immediate posttest to three week posttest the game using a publicly available social impact game. Additionally, the
study used an empirical methodology that is veriable with mea-
Table 6
sures routed in valid and reliable statistical approaches. Simply
Immediate posttest ATHI results. stated, this study found statistically signicant results within the
protocols of the study, using parametric tests and reached conclu-
Source DV: immediate Coefcient Standard Approx. P-value
posttest ATHI error d.f.
sions that are supported by current literature.

Control 37.739 .221 197 p < 0.01

Game .990 .626 197 .115 4.1. Affective learning
Reading 1.264 .660 197 .057
Gender .335 .305 2318 .272 Playing Spent or completing the reading signicantly increased
Games home .031 .019 2318 .103 the affective learning score immediately after treatment with the
Games SCH .003 .037 2318 .931
ATHI pretest .499 .021 2318 p < 0.01
game group scoring 1.57 points higher and the reading group
scoring .66 points higher out of a score of 6. This nding showed
 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221
Fig. 4. ATHI scores by test and group.
that there is a signicant impact on affective learning whether the
students read or play the game. The game group scored both signif-
icantly higher than the control group and almost a full point higher
than the reading group in the immediate posttest. The data sup-
ports prior research (Lavender, 2008; Lieberman, 2001; Ruggiero,
2014) that playing the game would lead to a higher level of affec-
tive learning. Although subjects who were a part of the game group
scored signicantly higher than the reading group, the game group
was not signicantly different than the reading group. Both activ-
ities positively affected the affective learning score signicantly
over the control group. However, it is interesting to note that stu-
dents in the game group increased two times over the reading
group. One possible explanation for this nding is that the game
elements and interactivity increased affective learning. Prior
research supports this claim (Alkan & Cagiltay, 2007; Barab et al.,
2009; Cameron & Dwyer, 2005; Dormann, Whitson, & Neuvians,
2013), and the students in the game group were experiencing the
same content as the students reading in addition to the persuasive
social impact game.
Another potential explanation for the signicant statistical and
practical increase in scores on the ALS for the game group is that
the pretest score was based off of subjects not having any prior
experience with the game. Therefore the pretest scores were hom-
ogenously low across all groups and the game group was signi-
cantly different than the control group at the immediate posttest
due to the game elements. The combined explanation of the prior
research supporting an increase in affective learning and no prior
activity resulting in low pretest scores indicates the differential
impact of game play on affective learning over reading or control
group scores.
This nding is both interesting and expected. It was expected
that the students who played the game to be more engaged and
affected than those who did nothing. What was unexpected was
that students who read would not be statistically different than
the students who play the game. However, it has been seen before
in games research (Van Eck, 2006) where there is no statistical dif-
ference between groups when measuring abstract concepts like
affective learning.
All three groups affective learning decreased from the immedi-
ate posttest, but the game group was signicantly different from
the control group in a positive direction. This means that after
three weeks the students that played the game were still more
engaged and motivated in the topic of homelessness than the stu-
dents who had not played the game. While affective learning
peaked at the immediate posttest and then went down during
the three-week interval, the game group was still more positive
than the other groups displaying a sleeper effect. However, while
the reading group and control group decreased to pretest level
the game group was signicantly higher statistically than the
219
control group after three weeks. One explanation for this nding
may be that the mechanics of the game that initially increased
the ALS scores were sustainable after three weeks leading to a
higher, albeit less peaked, affective learning scores than the control
or reading groups.
One possible interpretation of this nding is that the game
Spent, with its persuasive mechanics, has a higher residual effect
on affective learning when students think about homelessness.
The more positive affective learning could possibly link to a more
positive attitude towards the homeless. Prior research (Barker,
Brinkman, & Deardorff, 1995; Bartholomew et al., 2000; Cahill,
1994; Goldsworthy, Barab, & Goldsworthy, 2000; Tobias, Fletcher,
& Wind, 2014; Yee & Bailenson, 2006) supports this nding. Prac-
tical signicance of this nding links to the statistical signicance.
Students that are more engaged and motivated by the game imme-
diately after playing are able to sustain that engagement and moti-
vation in the topic of homelessness over the course of three weeks.
Games can manufacture psychological changes as shown here
and supported by the work of the above studies as well as Foster
and Mishra (2009), Mishra and Foster (2007), and Park, Plass, and
Brnken (2014).
4.2. Attitude towards the homeless
Playing the game did not immediately change students attitude
towards the homeless score signicantly as compared to the con-
trol group. Additionally, reading about homelessness negatively
affected students attitude towards the homeless score as com-
pared to the control group.
It is the latter nding that is most interesting. Why does reading
about the homeless signicantly affect the score on the ATHI while
playing the game does not? Plus, why is that change negative? One
potential explanation for the negative change from pretest to
immediate posttest for the reading group is that the students did
not link the two activities as being related. Another explanation
is the reading level was either too high or too low for the students.
While the reading level was tested at fth grade, the range of stu-
dents who participated was not limited to only those reading at
grade level. The reading group was signicantly different nega-
tively than the control group for the immediate posttest, indicating
that reading about being homeless was a negative experience for
the subjects. One additional potential explanation for nding a
strong difference between reading about homelessness and the
control group was the context of the reading material. The subjects
read a rst-person account of being homeless for one night and the
realities that are encountered by this person. It is clear that the
subjects as a group had a statistically signicant negative reaction
to the reading.
220 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221
One potential explanation for weak differences between playing
the game and the control group was the measure of assessment.
Spent was not intended to teach about homelessness but to support
conceptual learning, best measured by attitude items. However,
initial analysis of the attitude items, a part of the immediate post-
test, produced low Cronbachs alpha coefcient for the total 11
items, limiting inferences from the attitude towards the homeless
measure (ATHI). An additional troubling problem is that the corre-
lation between the pretest and posttest were high and that the
actual score values before and after playing the game were similar.
Although recent literature (Shute, Ventura, & Ke, 2015) shows that
pretest performance can predict posttest scores, it is clear that the
subjects as a group were relatively homogenous in their attitude
towards the homeless (less than 10% ICC on the HLM analysis).
While the difference between the game group and control group
was not statistically signicant, it was practically signicant. The
students in the game group did gain more than 2 points from the
pretest showing a positive change in attitude towards the home-
less immediately after playing the game. To put this in perspective,
we can look to recent research in the eld of games and learning.
As discussed in the introduction there are studies that demonstrate
a short-term change in affective and cognitive processes. The nd-
ings from this study are not unlike the other studies with students
in this age group (Van Eck, 2006) where a large heterogeneous
group does not show a statistically signicant change without
aggregating to grade level or breaking down the results by addi-
tional variables.
While the ndings for the short-term research questions were
interesting, the three week scores are where the study demon-
strates both practical and statistical signicance across all areas.
After three weeks the game group scores signicantly higher on
the ATHI than the control group but not signicantly different from
the immediate posttest. This nding implies that students who
were in the game group were still signicantly higher in terms of
attitude towards the homeless than the control group, implying a
sleeper effect. This nding is one of the most notable of the whole
study-there is both statistical and practical signicance within this
one nding. Students in the game group, who got the same content
as the students in the reading group, maintained a more positive
attitude towards the homeless than either the reading or the con-
trol groups. This can be interpreted to mean that something about
the game helped sustain a more positive attitude towards the
homeless after a period of three weeks.
One potential explanation for nding strong differences
between playing the game and the control group was that the
increased positive attitude students had displayed on the immedi-
ate posttest was sustainable over three weeks while the control
group returned to stasis. Taken together with the immediate post-
test score, the relationship between being in the game group and
attitude towards the homeless was signicantly more positive sta-
tistically than the control group.
Other studies that have examined long term sustainability of
attitude have shown that when there is a personal connection
between what is being presented, and the level of cognitive impor-
tance associated with the topic sustained learning takes place
(Lieberman, 2001; Padgett et al., 2006; Shute et al., 2015; Tuzun,
2007). One way to interpret this nding is that the game mechan-
ics, in particular the persuasive mechanics, are responsible for the
sustained change over time. This interpretation is not supported by
the prior research in persuasive games simply because there is
none. One statement that appears over and over again in reports
by Future Labs on serious games (Ulicsak & Wright, 2010) is one
element that has not been investigated by (insert game here) is
the impact on the trials on knowledge retention and behavior
change. Currently there are frameworks on persuasive technology
(Fogg, 2003; Karppinen & Oinas-Kukkonen, 2013) that support this
nding because it falls within the motivation and ability levels of
the students while using persuasive triggers that are meant to
affect attitude change. However, to reiterate the signicance of this
study, and these ndings in particular, is that they are a rst step in
the otherwise unconquered eld of measuring persuasive game
effectiveness.
5. Conclusion
In the big picture this study was both statistically and practi-
cally signicant in the eld of measuring persuasive games. This
is a rst step in a barren area of research and for that reason alone
this study will open up new paths to future research. All of the
major ndings were statistically signicant with a solid methodol-
ogy and research framework that created a replicable study that
can be used with any persuasive game that ts the criteria dis-
cussed in the introduction. Moreover, this study is the rst step
on lining up more research on emotive processes such as affective
learning and attitude. Moving forward is the key to the utility of
this study and lessons learned during the process will help further
this eld of research.
This study examined the effects of playing a persuasive social
impact game on affective learning. Data were collected from
5139 middle and high school students across 200 schools in four
states. The treatment conditions were randomly assigned to each
classroom after matching and included a control group (only took
the tests), a game group, and a reading group each within a Solo-
mon design (no pretest for half the participants). There were statis-
tically signicant positive changes to the game group after three
weeks, a rst step in closing the gap that exists in persuasive game
research.
References
Abt, C. (1970). Serious games. New York: The Viking Press.
Alkan, S., & Cagiltay, K. (2007). Studying computer game learning experience
through eye tracking. British Journal of Educational Technology, 38(3), 538542.
Amory, A. (2006). Game object model version II: A theoretical framework for
educational game development. Educational Technology Research and
Development, 54, 127.
Barab, S. A., Scott, B., Siyahhan, S., Goldstone, R., Ingram-Goble, A., Zuiker, S., et al.
(2009). Conceptual play as a curricular scaffold: Using videogames to support
science education. Journal of Science Education and Technology, 18(1), 305320.
Barker, J., Brinkman, L., & Deardorff, M. (1995). Computer intervention for
adolescent children of divorce. Journal of Divorce and Remarriage, 23(3),
197213.
Bartholomew, L. K., Gold, R. S., Parcel, G. S., Czyzewski, D. I., Sockrider, M. M.,
Fernandez, M., et al. (2000). Watch, discover, think, and act: Evaluation of
computer-assisted instruction to improve asthma self-management in inner-
city children. Patient Education and Counseling, 39(23), 269280.
Bogost, I. (2003). Persuasive games. <www.persuasivegames.com> [Retrieved
11.09.11].
Bogost, I. (2007). Persuasive games: The expressive power of video games. Cambridge,
MA: The MIT Press.
Bogost, I. (2008). How wrong i was about political games in 2008.
<www.watercoolergames.org/archives/000888.shtml> [Retrieved 11.09.11].
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models in social and
behavioral research: Applications and data analysis methods (1st ed.). Newbury
Park, CA: Sage Publications.
Cahill, J. M. (1994). Health works: Interactive AIDS education videogames.
Computers in Human Services, 11(1), 159176.
Cameron, B., & Dwyer, F. (2005). The effect of online gaming, cognition and feedback
type in facilitating delayed achievement of different learning objectives. Journal
of Interactive Learning Research, 16(3), 243258.
Campbell, D. T., Stanley, J. C., & Gage, N. L. (1963). Experimental and quasi-
experimental designs for research. Boston: Houghton Mifin.
Clarren, R. (2006). Virtually dead in Iraq, Salon, September 16, 2006.
www.salon.com/ent/feature/2006/09/16/americasarmy/ [Retrieved 12.010.11].
Cooper, L. (1932). The rhetoric of Aristotle. Englewood Cliffs, NJ: Prentice-Hall.
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York:
Harper & Row.
Dormann, C., Whitson, J. R., & Neuvians, M. (2013). Once more with feeling game
design patterns for learning in the affective domain. Games and Culture.
1555412013496892.
 D. Ruggiero / Computers in Human Behavior 45 (2015) 213221
Engelhardt, C. R., Bartholow, B. D., Kerr, G. T., & Bushman, B. J. (2011). This is your
brain on violent video games: Neural desensitization to violence predicts
increased aggression following violent video game exposure. Journal of
Experimental Social Psychology, 47(5), 10331036.
Evans, M. A. (2011). Procedural ethos: Conrming the persuasive in serious games.
International Journal of Gaming and Computer-Mediated Simulations (IJGCMS),
3(4), 7080. http://dx.doi.org/10.4018/jgcms.2011100105.
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). GPower 3: A exible
statistical power analysis program for the social, behavioral, and biomedical
sciences. Behavior Research Methods, 39, 175191.
Fogg, B. J. (2003). Persuasive technology: Using computers to change what we think and
do. Amsterdam: Morgan Kaufmann Publishers.
Foster, A. N., & Mishra, P. (2009). Games, claims, genres and learning. In R. E. Ferdig
(Ed.), Handbook of research on effective electronic gaming in education:
Information science reference.
Frasca, G. (2001). Videogames of the oppressed: Videogames as a means for critical
thinking and debate. Atlanta, GA: Georgia Institute of Technology.
Gee, J. P. (2003). What video games have to teach us about learning and Literacy. New
York, NY: Palgrave Macmillan.
Gee, J. P. (2007). Good video games + good learning: Collected essays on video games,
learning and literacy. New York: Peter Lang.
Goldsworthy, R. C., Barab, S. A., & Goldsworthy, E. L. (2000). The STAR project:
Enhancing adolescents social understanding through video-based, multimedia
scenarios. Journal of Special Education Technology, 15(2), 1326.
Huizinga, J. (1955). Nature and signicance of paly as a cultural phenomenon. In K.
Salen & E. Zimmerman (Eds.), The game design reader: A rules of play anthology
(pp. 96120). Cambridge: MIT Press.
Johnson, B., & Christensen, L. B. (2012). Educational research: Quantitative,
qualitative, and mixed approaches. Thousand Oaks, Calif: SAGE Publications.
Karppinen, P., & Oinas-Kukkonen, H. (2013). Three approaches to ethical
considerations in the design of behavior change support systems. In
Persuasive technology (pp. 8798). Berlin, Heidelberg: Springer.
Khaled, R. (2007). Culturally-relevant persuasive technology. Wellington, NZ: Victoria
University of Wellington.
Kingree, J. B., & Daves, W. F. (1997). Preliminary validation of the Attitudes Toward
Homelessness Inventory. Journal of Community Psychology, 25, 265288. http://
dx.doi.org/10.1002/(SICI)1520-6629(199705)25:3<265::AID-JCOP4>3.0.CO;2-V.
Lavender, T. (2008). Games Just Wanna Have Fun . . . Or Do They? Measuring the
effectiveness of persuasive games. Loading, 1(1).
Lepore, J. (2007). The meaning of life. New Yorker, 83(13), 3843.
Lieberman, D. A. (2001). Management of chronic pediatric diseases with interactive
health games: Theory and research ndings. The Journal of Ambulatory Care
Management, 24(1), 2638.
MacArthur Foundation (2012). Digital learning competition, September 15, 2011.
<http://www.dmlcompetition.net/> [Retrieved on 01.02.12].
Mishra, P., & Foster, A. (2007). The claims of games: A comprehensive review and
directions for future research. In C. Crawford, D. A. Willis, R. Carlsen, I. Gibson, K.
McFerrin, & J. Price, et al. (Eds.). Society for information technology & teacher
221
education: 2007 18th international conference (Vol. 2007, pp. 22272232). San
Antonio, Texas: Association for the Advancement of Computing in Education
(AACE).
Murray, J. (1997). Hamlet on the Holodeck: The future of narrative in cyberspace.
Cambridge, MA: MIT Press.
Ochalla, B. (2007). Who says video games have to be fun? The rise of serious games.
http://www.gamasutra.com/view/feature/1465/who_says_video_games_have_to_
be_php [Retrieved 03.11.11].
Padgett, L. S., Strickland, D., & Coles, C. D. (2006). Case study: Using a virtual reality
computer game to teach re safety skills to children diagnosed with fetal
alcohol syndrome. Journal of Pediatric Psychology, 31(1), 6570.
Park, B., Plass, J. L., & Brnken, R. (2014). Cognitive and affective processes in
multimedia learning. Learning and Instruction, 29, 125127.
Quinn, C. (2005). Engaging learning: Designing e-learning simulation games. Hoboken,
NJ: Pfeiffer Essential Resources.
Ruggiero, D. (2012, June). Conceptualizing a persuasive game framework. In World
conference on educational multimedia, hypermedia and telecommunications (Vol.
2012, No. 1, pp. 11811185).
Ruggiero, D. (2013). Persuasive games as social action agents: Challenges and
implications in learning and society. International Journal of Gaming and
Computer-Mediated Simulations (IJGCMS), 5(4).
Ruggiero, D. (2014). Problem solving through video game creation. In Curriculum
models for the 21st century (pp. 295311). New York: Springer.
Scott, M. D., & Wheeless, L. R. (1975). Communication apprehension, student
attitudes, and levels of satisfaction. Western Journal of Speech Communication,
41, 188198.
Shute, V. J., Ventura, M., & Ke, F. (2015). The power of play: The effects of Portal 2
and Lumosity on cognitive and noncognitive skills. Computers and Education, 80,
5867.
Thomas, R., & Cahill, J. (1997). Using an interactive computer game to increase skill
and self-efcacy regarding safer gender. Health Education and Behavior, 24(1),
7186.
Tobias, S., Fletcher, J. D., & Wind, A. P. (2014). Game-based learning. In Handbook of
research on educational communications and technology (pp. 485503). New
York: Springer.
Tuzun, H. (2007). Blending video games with learning: Issues and challenges with
classroom implementations in the Turkish context. British Journal of Educational
Technology, 38(3), 465477.
Ulicsak, M., & Wright, M. (2010). Games in Education: Serious games. Bristol:
Futurelab.
Van Eck, R. (2006). The effect of contextual pedagogical advisement and
competition on middle-school students attitude toward mathematics and
mathematics instruction using a computer-based simulation game. Journal of
Computers in Mathematics and Science Teaching, 25(2), 165195.
Yee, N. & Bailenson, J. N. (2006). Walk a mile in digital shoes: The impact of
embodied perspective-taking on the reduction of negative stereotyping in
immersive virtual environments. In Paper presented at PRESENCE 2006, Palo Alto,
CA.