1

Running head: EVALUATION OF DIGITAL GAME-BASED LEARNING

Evaluation of Digital Game-Based Learning:

A Research Proposal into the Impact of Fallout 4 in an English Language Arts Classroom

Thomas K. Wheeler

Towson University
 2
EVALUATION OF DIGITAL GAME-BASED LEARNING
Abstract

Reading comprehension is the ability for students to accurately understand written material.

Students with high levels of reading comprehension ability are able to decode what they read in

order to make connections between what they read and what they already know. Reading

comprehension is a significant indicator of progress in any ELA curriculum and grants students

the opportunity to digest high level anchor texts. In the Howard County school system, reading

comprehension is part of the Strategic Plan to empower students to be college and career ready.

The research conducted for this paper examined the different implementation strategies for

digital game-based learning as well as its impact on student achievement.

Keywords: digital game-based learning, technology, engagement

 3
EVALUATION OF DIGITAL GAME-BASED LEARNING
Evaluation of Digital Game-Based Learning:

A Research Proposal into the Impact of Fallout 4 in an English Language Arts Classroom

Research Proposal Background

In the Strategic Plan for the Howard County Public Schools System, Superintendent

Michael Martirano identified what he referred to as, “the fierce urgency of now.” This call to

action identifies its mission to ensure success. This success is both academic and social-

emotional in nature. Overall the well-being of our students is the focal calling of our school

system. One of the four pillars of this call to action is to empower its student body to acquire the

attributes and skills necessary to become global citizens and to obtain meaningful skills through

their education (Howard County Public Schools System, 2016).

The English Language Arts (ELA) curriculum is deeply invested in Superintendent

Martirano’s call to action and provides opportunity to empower their students in every unit. The

county provided curriculum for ELA is extensive in its study of many of the elements of the

English language. Generally speaking, anchor texts are used to teach these concepts including

grammar, vocabulary, and literary criticism. A major element of high school English is using

text-based evidence to characterize figures from popular texts. While teaching both direct and

indirect characterization through texts is accessible and effective, giving students the opportunity

to explore characterization in gamified situations could be more impactful to their understanding.

Digital game-based learning (DGBL) is an instructional method that incorporates

educational content or learning principles into video games with the goal of engaging learners.

While “engagement” in the realm of education is a subjective field, video games might be

particularly engaging to the current and future student populations. If that engagement that

students exhibit lead to an increase in reading comprehension skills, then it is an avenue worth
 4
EVALUATION OF DIGITAL GAME-BASED LEARNING
exploring and worthy of further research. Digital game-based learning in an English classroom

aligns with the strategic plan for Howard County in both theory and practice. In an ever-

developing curriculum, there are many unknowns about what the future will hold for ELA

classrooms. My proposal is to research how students react to digital game-based learning in an

English classroom.

Fallout 4 is an open world role playing game and thus is meant for the player to have free

reign over his or her actions. Students will create their own virtual identity including appearance,

personality, and attributes. This projected identity will serve as their liaison into the virtual post-

apocalyptic United States of America. The students will be playing the game as they desire but

they will be analyzing how the social climate of the virtual universe shapes the characterization

of the NPCs as well as their own virtual identity.

Statement of the Problem

Despite the Strategic Plan to empower students as well the common core state standards

to increase reading comprehension, students in Reservoir High School English classes are testing

lower than in the past. The Howard County English Office has made considerable investments in

different tools for English during the Covid-19 pandemic including the award-winning

curriculum platform, Actively Learn. Despite the funding allocations, pre-post measures have

shown that in English classes, reading comprehension has not only decreased at every grade

level, but students are struggling with capturing basic comprehension in source material. With

the Howard County goal of all students becoming college and career ready in mind, teachers

must be more determined than ever to instill a strong sense of literacy in each of their students.

At the risk of falling behind their peers and the future work force, students need to increase their

reading comprehension skills because their futures are at stake.

 5
EVALUATION OF DIGITAL GAME-BASED LEARNING

Statement of the Hypothesis

Reading comprehension growth can be increased through the use of Fallout 4 (Bethesda,

2015) in an ELA classroom.

Review of Literature

Students demonstrate improved reading comprehension when engaged in game-based

learning. Rusman, Ternier, and Specht (2018) conducted a theory-informed study into the

effectiveness of a mobile game in language comprehension using a design research approach.

The game aimed to familiarize young children (aged 4-8) with vocabulary, sounds, and

pronunciations. The qualitative research conducted found the use of the game to show a positive

impact on student familiarization of vocabulary. This qualitative research was focused around

series of questionnaires provided to both children who participated and parents who observed.

The quantitative research conducted was based on a post-test provided and it showed positive

impact on student memory in a 25-minute window after the game was played. No research was

provided on long term sustainability.

At the high school level, students are expected to meet standards of reading

comprehension that include the ability to analyze a complex set of ideas or sequence of events

and explain how specific individuals, ideas, or events interact and develop over the course of the

text. That ability to understand and analyze can sometimes be engaged through stimulating

environments with light and sound. Zi-Yu, Shaikh, Gazizova (2019) conducted a study that

demonstrated that, with game-based learning, students showed an increase in motivation and

attraction to learning. Furthermore, those same students that were invested in game-based

learning showed higher learning outcomes compared to the control group. This difference in
 6
EVALUATION OF DIGITAL GAME-BASED LEARNING
objective data was significant showing that the number of students that gave correct answers

over 50% of the time for game-based learners was 1.72 times higher than the non-game-based

learners.

In the Marino and Hayes (2012) quantitative study, the online digital game WhyPox was

used to investigate the impact of teacher-led digital game-based learning on the comprehension

of biology material. Marino and Hayes (2012) sampled online users of various ages from 10-18

with a median age of 12.4 years old. 285 online users who consented to participate in the

research study were recruited via postings and online town hall meetings on the site. In order to

participate, users had to print out consent and assent forms and mail or fax them to the

university. The study was designed to examine the digital-games impact on biology

comprehension when playing for leisure at home as opposed to playing in a classroom under the

guidance of a teacher. Users showed an increase in comprehension of the understanding of a

computer virus as well as the accuracy in depicting what it would look like in both groups.

Though there were generally no differences between pre / post survey response for both

classroom and online participants, results showed that classroom students provided significantly

more computational responses and significantly fewer ‘don’t know’ response across the board.

These results can be interpreted in a number of ways. All users of the digital game demonstrated

growth in biology comprehension which can be indictive of engagement or educational value.

Furthermore, while the users that were in the classroom being taught the game did show

significantly higher growth patterns, it should be noted that the degree of bias was not addressed

in the study. This raises the question of the validity of the study entirely. There is a possibility

that the instructor knowingly or unknowingly led students in specific instruction to “teach to the

test” so to speak.
 7
EVALUATION OF DIGITAL GAME-BASED LEARNING
The Lawrence & Sherry (2021) experimental study of five seventh grade English

Language Arts classrooms most closely resembles the study that I will be proposing. The

comparative based experiment measured students argumentative writing responses on the basis

of quality as determined through a rubric of their creation. The control group of this experiment

did not play the digital game, Quandary, while the experimental group did. The findings of this

experiment show that students that engaged in the game show increased argumentative literacy.

Studies involving students and digital game-based learning are important, but so are the

revelations that come from studying teachers. Hayak & Avidov-Ungar (2020) performed a

qualitative study on 28 elementary school teachers that were actively integrating digital game-

based learning into their curriculum. The comparative study was researching the willingness and

comfort to integrate digital game-based learning based on the length of time that a teacher was

employed. Hayak & Avidov-Ungar (2020) made their determination by codifying responses

given through recorded interviews and found that the teachers’ career stage revealed

characteristic differences. Teachers that were teaching for under seven years were more

amenable to incorporating digital games into their unit plans. The findings of the study have

practical implications for teachers’ professional development at the various career stages.

Method

Participants

The numbers of students enrolled in my Reservoir High School 11th grade English honors

classes total 76. Those 76 students are split into three courses with totals of 25, 23, and 28

respectively. In the entire sample, 51% of students are Male, 49% are Female. 41% percent of

students are White/Caucasian, 39% are Black/African American, 11% are

Hispanic/Latino/Latina, 6% are Asian/Pacific Islander, and 3% are other ethnicities.

 8
EVALUATION OF DIGITAL GAME-BASED LEARNING
Measures

The Maryland Comprehensive Assessment Program (MCAP) provides information to

educators, students, parents, and stakeholders on student progress towards proficiency on the

Maryland College and Career Ready Standards. It is administered in the spring of the school year

and is required of all high school English students. Due to Covid-19 the passing of the MCAP

requirement has been lifted for the time being, but historically the student assessment must meet

a passing score. The ELA portion of the test is broken into four, 90-minute sessions. These

sessions take place over the course of several days. The test contains both multiple choice

questions as well as a writing section. One of the primary testing targets of the MCAP is reading

comprehension and that direct focus is embedded into the Howard County Public School

System’s English Language Arts curriculum.

Design

Given that I am seeking to ascertain whether or not that reading comprehension can be

improved through the use of the game Fallout 4, a mixed methods approach would be the best fit

for my purposes. This study will take place during our American literature unit focused on the

dystopian literature of the 1950s. After WWII and the world saw the threat of atomic warfare,

there was an increase in speculative fiction and dystopian literature. My control group will be

learning about the unit using traditional methods of reading a series of short stories. My

experimental group will be playing the game Fallout 4 in class to explore a nuclear fallout’s

impact on society in America. 

The reason I settled on this approach is because I wanted to incorporate an open-ended

questionnaire to garner student feedback regarding the use of a video game in English class to
meet objectives. Furthermore, I want to explore the connection between student feedback and
performance on a unit pre/post test to determine if there is a correlation between reading
comprehension and engagement. This study design will be using a simple random sampling to
ensure that every student has an equal opportunity to be selected. The breakdown of my classes
 9
EVALUATION OF DIGITAL GAME-BASED LEARNING
is split almost evenly between men and women and while the composition of my classes leans
towards a Caucasian and black majority (80%), I have no concerns of under or over
representation. I will be assigning each student a numerical value and then I will use a random
number generator for both my control class and my experimental class.
Validity
The quantitative study instrument, the pre and post-test, contains questions that will align
with the common core state standards of reading comprehension. In many cases, the questions
themselves will be modeled after the stylistic approach that is seen in the Maryland
Comprehensive Assessment Program (MCAP). The questions, much like the state assessment,
will be varied to fairly capture the entire domain that the test seeks to measure. This includes plot
details as well as characterization elements. While there are students that struggle in testing
formats, precautions will be taken to alleviate that stress including extended time and frequent
breaks as requested.
The qualitative study instrument, the questionnaire, will contain questions that are written
in such a way that reflect no bias. The neutral questions will not contain charged language nor
will it persuade students in one direction or another.
In order to ensure the validity of the study, the instructor must also be reliable in their
teaching methods. The teacher must display identical patterns in teaching for both the control
group and the experimental group in areas that can be maintain. These fields include enthusiasm,
expectations, and content knowledge.
Procedures and Timeline

The procedures for the data collection will occur at different intervals. Both the control

group and the experimental group will begin the abbreviated unit with identical pretests. From

there, using an assortment of lesson plans, teachers will conduct five fifty-minute lesson plans.

These lesson plans are teacher developed but the control group will be focused on short stories

while the experimental group will be using the digital game Fallout 4. At the end of the unit,

students will take identical post-tests to measure their growth. At the conclusion of the unit, all

students will be asked to complete a questionnaire containing open-ended survey questions. The

nature of the open-ended responses means that they are not limited to a set of predetermined

options. Once the data has been collected, it will then be analyzed by the teacher of record.

Data Analysis

My study seeks to answer the question whether or not reading comprehension growth can

be increased through the use of Fallout 4 in an ELA classroom. My mixed methods approach
 10
EVALUATION OF DIGITAL GAME-BASED LEARNING
will require two sets of data collection that will include both a quantitative measure of growth in

reading comprehension and a qualitative measure of engagement.

In my quantitative study, student scores will be separated into two data sets for both my

control group and my experimental group using a group comparative design. The students in

both groups will be taking the same pre-test and the same post-test in order to ensure an accurate

demonstration of growth. In this dataset, student scores of the post-test will be compared to their

original scores on the pre-test in order to determine growth. The validity of test has already been

established earlier in the proposal. The difference in test scores will be demonstrated by a t-test.

T-tests are used to compare average outcomes when there are two groups. In my study, the two

groups will be the control group and the experimental group and the difference in reading

comprehension growth will be tested for statistical significance using a t-test.

In terms of my qualitative analysis, students will be assessed on their responses to an

open-ended questionnaire. I will take their responses and code them before grouping the

responses based on the language that they use in regards to how they interpreted their own

engagement levels as well as the relationship to reading comprehension. The validity of test has

already been established earlier in the proposal.

Anticipated Outcomes and Implications

I anticipate that the quantitative data collected will demonstrate stronger reading

comprehension growth with the experimental group, those that played the video game, than the

control group, those that learned the unit using traditional short stories. The extent of that

growth, however, will be moderate. Furthermore, the students that played the video game will

express stronger levels of engagement through their responses to the open-ended questionnaire.
 11
EVALUATION OF DIGITAL GAME-BASED LEARNING
The anticipated implications are that digital game-based learning would become a more common

method of teaching in ELA classrooms.

 12
EVALUATION OF DIGITAL GAME-BASED LEARNING
References

Rusman, E., Ternier, S., & Specht, M. (2018). Early Second Language Learning and Adult

Involvement in a Real-World Context: Design and Evaluation of the “ELENA Goes

Shopping” Mobile Game. Journal of Educational Technology & Society, 21(3), 90–103.

Zi-Yu Liu, Shaikh, Z. A., & Gazizova, F. (2020). Using the Concept of Game-Based Learning in

Education. International Journal of Emerging Technologies in Learning, 53–64.

https://doi-org.proxy-tu.researchport.umd.edu/10.3991/ijet.v15i14.14675

Marino, M., & Hayes, M. (2012). Promoting inclusive education, civic scientific literacy, and

global citizenship with videogames. Cultural Studies of Science Education, 7(4), 945–

954. https://doi-org.proxy-tu.researchport.umd.edu/10.1007/s11422-012-9429-8

Lawrence, A. M., & Sherry, M. B. (2021). How Feedback From an Online Video Game Teaches

Argument Writing for Environmental Action. Journal of Literacy Research, 53(1), 29–

52. https://doi-org.proxy-tu.researchport.umd.edu/10.1177/1086296X20986598

Hayak, M., & Avidov-Ungar, O. (2020). The Integration of Digital Game-Based Learning into

the Instruction: Teachers’ Perceptions at Different Career Stages. TechTrends: Linking

Research & Practice to Improve Learning, 64(6), 887–898. https://doi-org.proxy-

tu.researchport.umd.edu/10.1007/s11528-020-00503-6