Running head: DATA ANALYSIS 1

Data Analysis

Kyle Johnson

EDD8422

Transformational Thinking to Meet 21st-Century Skills

Telephone: 678-477-2349

Email: kyle.johnson112991@gmail.com

Instructor: Dr. Brie Nelson

ACTION RESEARCH: DATA ANALYSIS 2

Data Analysis

To effectively assess the success of a program that is implemented within a school, data

collection must be included in the study to support the positive or negative growth that is being

achieved with the program. Once data is collected, the researcher must analyze the data to reveal

the success or failure of the program being evaluated. Through this course project, the researcher

aimed to evaluate the success of Kahoot on student learning. In the process, teacher perceptions

of the success and benefits of Kahoot were also analyzed. By using a mixed methods analysis of

the success of using digital based games such as Kahoot, “…data can be analyzed and integrated

during the process of research so that new, synergistic understandings can emerge that extend the

findings and interpretations that can be achieved from reliance on only one methodological

perspective” (Sammons, 2010). Given both the qualitative and quantitative data, a

comprehensive analysis was conducted. This paper reflects on those findings.

The Learning Environment

Stakeholder Receptivity and Perceptions

At Cedar Grove High School, technology integration plays a crucial role in assisting

teachers in developing engaging lessons that foster student collaboration and challenge students’

twenty-first century skills. For digital-based games such as Kahoot, teachers have adopted this

program as a means to remediate student learning for upcoming assessments. However, the

degree in which teachers interact with the game as students play Kahoot, their perception of the

game’s ability to aide in student mastery of standards, and the game’s impact on remediating

content effectively to increase student assessment scores is the focus of this study.

Twelve teacher participants were given a survey with five Likert-scale questions and

three short answer questions. According to Figure 1: Teacher Perceptions of Kahoot, of the
ACTION RESEARCH: DATA ANALYSIS 3

twelve participants surveyed, 33% of participants indicated they identified themselves as experts

and use the reports to remediate missed content even further. 50% of teachers indicated they use

Kahoot at least five times a month to remediate content covered within a class for remediation

purposes. However, one survey question asked teachers if they thought student assessment scores

were directly impacted by student involvement in playing Kahoot. One teacher indicated that

Kahoot engages students in a way that challenges them, yet helps them learn. Based on this

information, 92% of teachers survey feel that Kahoot is beneficial for student learning, but the

effect it has on student assessment outcomes is minimal.

Figure 1: Teacher Perceptions of Kahoot

Teacher Perceptions of Kahoot

6

0
John Sandy David Glenda Carla Mike Ruth Wendy Carrie Jacob Willliam Hannah

How comfortable are you with using Kahoot?

How often do you use games such as Kahoot with one of your class periods?
Do you feel Kahoot is beneficial for student learning?
Do you think student assessment scores are directly impacted by student involvement in playing Kahoot?
Based on your last assessment, were scores impacted by those students who played Kahoot seriously?
ACTION RESEARCH: DATA ANALYSIS 4

Kahoot Averages Compared to Interim Averages

To gather Kahoot averages, participants were asked to provide those averages for each

class period. This data was consolidated and averaged together for each teacher participant. The

researcher then logged into the assessment portal provided by the district and gather each

teacher’s Interim averages for their respective content areas. Based on these findings and

comparisons, as shown in Figure 2: Kahoot vs. Interim Average Scores, three teachers had a

positive growth differential in their Kahoot average compared to their Interim average. Four

teachers had a negative growth differential, and five teachers did not engage their students in the

Kahoot review prior to giving the Interim assessment. Based on these findings, there is no

significant relationship to those teachers who engage in Kahoot and the impact Kahoot has on

increasing student assessment averages. However, based on teacher perception of Kahoot, and

those teachers who opted out of playing the Kahoot review prior to giving their assessment, a

significant correlation can be drawn from the correlated data.

ACTION RESEARCH: DATA ANALYSIS 5

Figure 2: Kahoot vs. Interim Average Scores

Kahoot vs. Interim Average Scores

90
81 80
80

70
61 62
60 58 58
53 54 55
52 52 52 51 52
49 47 48
50

40 37
32
30

20

10
0 0 0 0 0
0
John Sandy David Glenda Carla Mike Ruth Wendy Carrie Jacob Willliam Hannah

Kahoot Interim Average County Interim Average

Analysis of the Correlated Data

Ruth is a College Prep World History teacher who has been teaching for seven years.

Based on her perception survey, she has a less than average opinion on Kahoot. Based on her

response to feeling as though Kahoot is beneficial for student learning, she stated, “I think it
ACTION RESEARCH: DATA ANALYSIS 6

depends on the student, but I do believe for some it causes them to engage with the material. It is

familiar (game platform) and competitive.” Based on her Interim average, 52, it was comparable

to other participant scores that did engage in the Kahoot review prior to the Interim assessment.

Carla is an Honors World History teacher who has been teaching for 13 years. She had a

very high Kahoot average of 80, yet her Interim average score 58. This reflects her perceptions

on Kahoot where she indicated an average belief that Kahoot is beneficial for student learning.

She indicated, “Those who actually pay attention and are actively engaged benefit.” Although

Carla indicated that she believed those who engaged in the Kahoot activity were rewarded with

higher Interim scores, the data does not reflect this assumption.

Sandy is an Honors US History teacher who has been teaching for 14 years. Her Kahoot

average was 61, and her Interim average was 81. She indicated that she felt that Kahoot was not

at all beneficial for student learning. She stated, “I see its benefits, but I don't think it is up to the

level of rigor students need for the test.” Although she had this belief, she still engaged students

in the Kahoot and her averages were in direct contrast to Carla’s, who did not indicate any issue

of rigor with the Kahoot questions. Based on these three instances of data correlated and

analyzed, no conclusive correlation can be drawn between teacher perception of Kahoot and the

impact Kahoot games have on overall averages of Interim assessment data. Although teacher

perceptions of using Kahoot to remediate content were average, there is no strong correlation

between the frequency of students playing Kahoot and the positive outcomes students have on

their Interim assessment scores.

Equity, Diversity, and Ethical Considerations

To ensure ethics and bias were protected, participants included in this study came from

varying experience levels and teach Honors, College Prep, and Resource classes. 42% of the
ACTION RESEARCH: DATA ANALYSIS 7

participants were male and 58% were female. Although names were used in the perception

survey, pseudonyms were used in the data collection and chart that consolidated data. When

collecting Kahoot and county Interim averages, individual student data was not shared with the

researcher and was not analyzed. Only the county Interim averages for each teachers’ class was

consolidated into one localized average. This ensured that no bias was given with a group of

students with lower cognitive ability versus higher cognitive ability. The same method was used

for the multiple Kahoot games that teachers used to review for each class period.

Areas of Implementation

Skills Needed for 21st Century Learners

In order for digital-based games to work effectively, and in this case, reap the benefits of

ensuring content covered within the game yields lasting results as students take assessments,

teachers need to provide remediation activities that help develop students’ 21st century skills..

For example, when creating Kahoot games, ensuring the questions asked are aligned to the rigor

and content depth of knowledge is essential to challenge student cognitive abilities. As Hamari,

Shernoff, Rowe, Coller, Ashell-Clarke, and Edwards (2016) state, “…the challenge of the game

should be able to keep up with the learners growing abilities and learning in order to endorse

continued learning in game-based learning environments” (p. 170). Without the rigor, the digital-

based game will not provide students with the level of content development needed to be

successful on the assessment. As one teacher noted, they are concerned that Kahoot is just a ‘fun’

way for students to learn and does not quite challenge student thinking. This, however, can be

controlled by teacher creation of games, which is a strength of Kahoot.

Strengths and Challenges

ACTION RESEARCH: DATA ANALYSIS 8

With the ability to create Kahoots, teachers can create questions that are as rigorous as

they allow. Different game designs also give teachers the flexibility to create different question

options, including several answer choices, true or false, puzzle, poll, word cloud, and slide for

more context-rich questioning. According to Qian and Clark (2016), digital-based games that

promote 21st century skills, “[employ] multiple game-design elements, with collaboration, role

playing, narrative, exploration, and complexity being the most popular ones” (p. 56). In other

words, complex game designs are much more likely to illicit 21st century skills than traditional

drill and practice games. Although Kahoot offers these advanced game options, there are still

challenges that keep Kahoot from being fully engaging for students.

Due the mobile ability of Kahoot, students use their phones as answer documents as they

engage in the game. All participants indicated that this way of engaging students in a fun

learning environment was the main reason they choose to play this game with students. By using

devices students use on a daily basis, students are able to have fun learning. However,

participants expressed the need for more controlling factors in the games ability to deliver

feedback and keep students focused on the task. Carla noted, “Once they log in, make it so they

can’t switch screens (to check texts, social media, etc.).” Due to the use of personal phones as

answer devices, students can often get distracted with other phone notifications, which hinder

them from staying focused on the remediation activities. Additionally, William indicated the

need for an elaboration screen after each question that details more information regarding the

correct answer. With this additional feature added to student phones, it could cause them to

become more engaged instead of simply using the phone to pick an answer choice. With the

strengths and weaknesses noted, further measures can be in place to give teachers using a Kahoot
ACTION RESEARCH: DATA ANALYSIS 9

for remediation activities a solid foundation for structured review sessions that will assist them in

mastering the content covered on local, district, and state assessments.

Strategic Goals and Further Development

Based on the data analysis, there was no correlation found between teacher perception,

student engagement in Kahoot, and increased assessment averages. Moving forward, careful

consideration should be taken in ensuring questions covered within the game are aligned with the

rigor and depth of knowledge of the potential assessment. Additionally, student participation

needs to be promoted to ensure students are engaged and receiving the information needed to be

successful on the assessments.

Conclusion

To evaluate a program within a school, data needs to be collected and analyzed. With this

data, the effectiveness of the program can be assessed and measures can be put in place to ensure

strengths of the program are maintained and weaknesses can be built upon. Based on twelve

teachers’ perceptions and student assessment averages, the use of Kahoot to remediate content

could have the potential to create unique and lasting learning opportunities for students.

However, further evaluations must be done to truly get to the root of how digital-based games

such as Kahoot can be used to illicit critical-thinking skills and promote higher depth of

knowledge.

References

Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016).
Challenging games help students learn: An empirical study on engagement, flow and
immersion in game-based learning. Computers in Human Behavior, 54, 170-179.
doi:10.1016/j.chb.2015.07.045
ACTION RESEARCH: DATA ANALYSIS 10

Qian, M., & Clark, K. R. (2016). Game-based learning and 21st century skills: A review of
recent research. Computers in Human Behavior, 63, 50-58.
doi:10.1016/j.chb.2016.05.023

Sammons, P. (2010). The contribution of mixed methods to recent research on educational

effectiveness. Sage handbook of mixed methods in social and behavioral research, 697-
723.