Journal of Creative Communications

This manuscript has been submitted to Journal of Creative Communications

Journal Name: Journal of Creative Manuscript ID: CRC-2020-0227.RV2

Communications
Manuscript Type: Original article
Manuscript Title: The role of creative
communications and gamification in
customer engagement in higher
education: a sentiment analysis
approach

Keywords: Engagement behaviors, Gamification, Gamification-based tools,

Kahoots!, student engagement

MeSH terms:

Abstract: This paper aims to explore gamification tools in services and higher
education environments and their role in creating student engagement. The
research adopts a qualitative method based on quasi-experimental design.
Participants from a higher education institution are exposed to gamification
activities during a full semester. Researchers use sentiment analysis
technique based on a text-mining approach to analyse the data. Findings
reveal that participants perceive gamification in service settings as a useful
tool. The global sentiment analysis reveals a positive sentiment about the
gamification approach that contributes to increasing participants’
engagement. This study’s novelty arises from quasi-experimental research to
measure gamification activities’ impact on students’ engagement, measured
through sentiment analysis of their opinions.
v iew
Re
er
Pe
 JOURNAL OF
CREATIVE COMMUNICATIONS
Author Response Form
1
Manuscript
2 ID: CRC-2020-0227
Title:3“The role of creative communications and gamification in customer engagement in higher education: a
sentiment
4 analysis approach”.
Journal:
5 Journal of Creative Communications
6
We thank
7 the Reviewers and Editor for their thorough review and highly appreciate their comments and
suggestions,
8 which have significantly improved our paper’s quality. Please find below a detailed response to
each comment
9 in green. Revisions are shown in the manuscript, also in green colour.
10

11 AUTHORS’ RESPONSES TO REVIEWERS’ COMMENTS

12
13 REVIEWER 1
Suggestions/comments from the reviewer Response from the Author(s)
This is much improved from the first version I Thank you very much for your comments and
looked at. I appreciate the inclusion of gamification encouragement. Both reviewers’ comments are
critiques – although these remain quite marginal in truly helping us to improve our work.
the paper – and there is generally a lot more detail Concerning the suggestion to consider rephrasing
and analysis here. I appreciate the authors engaging the word customer, we have acted accordingly and
so strongly with both my and the other reviewer’s changed to the word ‘customer’ to ‘student’.
comments. Similarly, extra information about text Thanks for pointing this out.
mining is strongly beneficial. I personally remain Thank you once again for your time to read,
unconvinced by this method compared to human comment, and helping us improve our research.
analysis, BUT that’s just a personal methodological
thing, as the authors now describe and defend the
choice quite well, and I’m happy to say the paper
iew

should move forward. The new discussion is also

an excellent addition. One thought: maybe remove
the word “customers” for describing “students”?
v

Given all the negative discussion around the

customer-model of higher education learning, I
Re

think this is an unfortunate bit of phrasing.

Otherwise, I think this paper is much improved and
offers an interesting addition to the literature in this
area.
er

14
15
Pe

16
17
18
 JOURNAL OF
CREATIVE COMMUNICATIONS
Author Response Form
19
20 REVIEWER 2
Suggestions/comments from the reviewer Response from the Author(s)
Congratulations on the completion of this revision. Thank you very much for your kind words and
It is suitable for the journal in my opinion. encouragement.
However, some minor modifications will augment
the publication further.

1) There are minor language errors across the Thank you for your suggestion. We have revised
paper. Running the paper through Grammarly like the entire manuscript in terms of language
software will help improvise the language further. improvement. Changes are highlighted in green
colour.

2) Just before the variables are discussed in the
Theoretical development section, the author can
choose to mention about the S-0-R model, which
suits this paper as an overarching theory.
Thank you very much for your suggestion, which
we believe helps improve our manuscript’s quality.
We are now proposing S-O-R framework as the
overarching theory for this study. That can be seen
in both the introduction section and the theoretical
background section.

3) Please create contributions to theory and Thank you for your suggestion. We have acted
managerial implications as separate sections for accordingly and separated these two sub-sections.
better readership. Please see page 12, 13 and 14 for these changes.

4) Again in the contribution to theory connect the
overarching S-0-R theory with the results obtained
in this study. Such practice will augment the
contribution quality of this otherwise nice study.
iew
Thank you for your suggestion. As mentioned
above, we are now proposing S-O-R framework as
the overarching theory for this study and its
connection with our results. Theoretical
contributions are added based on this adoption.
Please see page 13 for that change. Additionally,

suggestions for future research are also offered

based on the use of S-O-R framework.
v

5) Please cite a few papers from JOCC to make this We have acted accordingly and added relevant
paper relevant to the Journal. literature made available by JOCC about this topic.
Re

Please see the reference list for these changes,

highlighted in green colour.

All the best for your publication. Thank you.

er

21
Pe

22
23
 1 The role of creative communications and gamification in student
2 engagement in higher education: a sentiment analysis approach

4 Abstract

5 This paper aims to explore gamification tools in services and higher education
6 environments and their role in creating student engagement. The research adopts a
7 qualitative method based on quasi-experimental design. Participants from a higher
8 education institution are exposed to gamification activities during a full semester.
9 Researchers use sentiment analysis technique based on a text-mining approach to analyse
10 the data. Findings reveal that participants perceive gamification in service settings as a
11 useful tool. The global sentiment analysis reveals a positive sentiment about the
12 gamification approach that contributes to increasing participants’ engagement. This
13 study’s novelty arises from quasi-experimental research to measure gamification
14 activities’ impact on students’ engagement, measured through sentiment analysis of their
15 opinions.

16

17 Keywords: Gamification; Gamification-based tools; Engagement behaviours; Student

18 engagement; Kahoot’s!;

19

20 Type of Paper: Research Paper

iew

21

22
v
Re
er
Pe
Introduction

This paper aims to understand the role of gamification in student engagement in a higher

education setting, following recent calls for further research about the role of technology

development on stakeholders engagement (Leonidou, Christofi, Vrontis, & Thrassou, 2018;

Loureiro, Romero, & Bilro, 2020; Watson, Wilson, Smart, & Macdonald, 2018) and customer

engagement (Bilro & Loureiro, 2020; Hollebeek, Clark, Andreassen, Sigurdsson, & Smith,

2020). Student engagement may be defined as behavioural and psychological involvement in

learning activities (Appleton, Christenson, & Furlong, 2008). The literature already addresses

engaged students as an active part in the process of searching, sharing, and learning new skills

and theories about a topic (e.g., Damnjanovic, Proud, & Milosavljevic, 2020; Iosup & Epema,

2014), especially when using gamification tools in educational environments (e.g., Faiella &

Ricciardi, 2015; Galbis-Córdova, Martí-Parreño, & Currás-Pérez, 2017; Looyestyn et al.,

2017). However, pieces of evidence about how the use of gamification can successfully create

engagement in participants of a service environment, such as higher education service, are still

scarce (Kayimbaşioǧlu, Oktekin, & Haci, 2016; Martín-Gutiérrez, Mora, Añorbe-Díaz, &

González-Marrero, 2017).

The academic discussion on the topic has focused the attention on gamification as a way

to increase attachment (Oleksy & Wnuk, 2017), to build co-creation experiences (Nobre &
iew

Ferreira, 2017), to foster the intention of engagement and brand attitude (Lim & Puspita, 2020;
v

Yang, Asaad, & Dwivedi, 2017), or to develop interaction and participation at higher education
Re

environments (Piñeiro-Otero & Costa-Sánchez, 2015). Contrariwise, several authors have

discussed the subversive ludification of society that gamification may represent (e.g., Fuchs,
er

2018; Woodcock & Johnson, 2017). The adoption and practice of game-based techniques or
Pe

gamification in higher educational environments may promote positive behaviour changes,

leading to increased motivation, relevance, or immersion (Kapp, 2012; Kapp & Coné, 2012).

1
However, gamification’s use as a tool to leverage student engagement in services is still an

under-explored topic (e.g., Leclercq, Hammedi, & Poncin, 2018; Rosado-Pinto & Loureiro,

2020).

This paper adopts a quasi-experimental approach to test the causal effects between an

intervention and its outcome (Harris et al., 2006), aiming to answer the following research

question: does gamification tools influence students’ engagement in services, such as higher

education settings? Specifically, this research aims to shed light on gamification’s implications

in students’ engagement in higher education environments, analysing the data through a

sentiment analysis technique based on a text-mining approach. For that, authors lay hands to

stimulus-organism-response (SOR) theoretical framework (Eroglu, Machleit, & Davis, 2003;

Mehrabian & Russell, 1974; Roschk, Loureiro, & Breitsohl, 2017). This research shows

gamification’s potential outside online environments and in distinct practical application areas,

collecting evidence of its positive impact in service settings. The findings unveil that students

understood the experience as an educational element, and only after as entertainment. The

findings also show that our participants’ overall experience was positive or very positive and

that participants feel engaged by using gamification tools in service settings such as educational

environments.

The paper’s contribution is two-folded. First, the findings add to the existing stakeholder
iew

and engagement literature, namely on services and technology transformation topics. Second,
v

the authors claim that services, specifically higher education institutions, can adopt new
Re

technological approaches such as gamification to improve students’ service (i.e., education).

As far as authors are aware, this paper is one of the pioneers to further understand students’
er

engagement process through technological tools such as gamification in an extended period

Pe

(such as a semester) in an actual/physical environment.

2
 The remainder of the paper is structured as follows. The paper reviews the relevant

literature on gamification in educational settings and student engagement in the next section.

Next, the authors present the research methodology and the results of our study. Finally, we

discuss the findings and present paper’s theoretical and managerial implications, along with

research limitations and opportunities for further research.

Theoretical background

Gamification and its use in educational settings

The gamification concept is still in its early days and open to a more established definition

(Hamari, Koivisto, & Sarsa, 2014). The Oxford dictionary describes gamification as applying

typical game playing elements (such as competition with others, point scoring, among others)

to distinct fields (Oxford, 2019). Literature defines gamification as the use of game design

elements in non-game contexts (Deterding, Dixon, Khaled, & Nacke, 2011; Priya & Kalpana,

2014), by incorporating game elements into non-game settings, or by using game mechanics,

dynamics, and frameworks to encourage specific behaviours (Ip, Law, & Lee, 2011; Pagowsky,

2012; Sheldon, 2012; Stott & Neustaedter, 2013). Additionally, other authors consider

gamification more broadly and describe it as the concept of using game-based processes and

game thinking to involve people, to motivate actions, to promote learning processes, or to solve
iew

problems (Chaudhary, 2010; Kapp, 2012).

v

In this paper, the authors adopt the definition of Deterding et al. (2011, p. 11), who claim
Re

that gamification is “the use of game design elements in non-game contexts”. One of those non-

game contexts where gamification techniques are evolving is the educational setting,
er

specifically the Higher Education (e.g., Faghihi et al., 2014; Galbis-Córdova et al., 2017). In
Pe

this paper, we also accept Robson and colleagues definition of gamification: “The application

of lessons from the gaming domain to change behaviours in non-game situations” (Robson,

3
Plangger, Kietzmann, McCarthy, & Pitt, 2015, p. 2). Both definitions point out how to use

gamification in different surroundings, such as Higher Education Institutions (HEI).

Various studies have observed participants’ behavioural changes in gamified environments

(Koivisto & Hamari, 2014; Turan, Avinc, Kara, & Goktas, 2016; Barata, Gama, Jorge, &

Gonçalves, 2017), with several others reinforcing the positive aspects of gamification’s

application at the HEI level (Yildirim, 2017; Subhash & Cudney, 2018). Still, since its

beginning, gamification has sparked controversy between academics and practitioners devoted

to human-computer interaction (Mahnič, 2014; Woodcock & Johnson, 2017). This controversy

is reflected in some literature addressing the topic, putting in evidence that gamification’s effect

on motivation or participation is lower than the expectations created by the hype (Broer, 2014;

Fuchs, 2018), or not showing a positive relationship between gamification tools usage and

improved success (Frost, Matta, & MacIvor, 2015). Although gamification in educational

settings is a growing phenomenon, the literature reveals that not sufficient evidence exists to

support the long-term benefits of gamification in educational contexts (Dichev & Dicheva,

2017) and that the knowledge about the gamification outcome in an educational context is still

scarce (Dichev & Dicheva, 2017; Dicheva, Dichev, Agre, & Angelova, 2015). However, the

levels of motivation and engagement among individuals joining gamified scenarios seem to

increase, leading to improved performance and positive results (Looyestyn et al., 2017).
iew

This study intends to contribute to the ongoing debate shedding light on gamification’s role
v

in students' engagement. For that, authors lay hands to stimulus-organism-response (SOR)

Re

theoretical framework (Eroglu et al., 2003; Roschk et al., 2017) initially proposed by

(Mehrabian & Russell, 1974). The framework in use represents the role of stimuli (i.e., in-class
er

gamification activities) in influencing consumers’ emotional and cognitive states (i.e. the
Pe

organism), which, in turn, result in approach or avoidance behaviour (i.e. student engagement).

4
Students’ engagement in gamified environments

The literature already shows the relevance of distinct stakeholders’ engagement, such as

consumers, shareholders, providers, or students, from various viewpoints (Loureiro, Romero,

et al., 2020), and engagement between companies and consumers (e.g., Brodie, Hollebeek,

Jurić, & Ilić, 2011; Hollebeek, Glynn, & Brodie, 2014; Loureiro, Bilro, & Japutra, 2020; Sprott,

Czellar, & Spangenberg, 2009). The topic of engagement is also discussed and studied in other

areas such as psychology, sociology, or education (Garczynski, Waldrop, Rupprecht, &

Grawitch, 2013; Loureiro, Bilro, & Angelino, 2020; Morimoto & Friedland, 2013). Literature

defines student engagement as a behavioural and psychological involvement in learning

activities (Appleton et al., 2008). Student engagement is also seen as a multidimensional

construct, comprising three dimensions: cognitive engagement, behavioural engagement, and

emotional/affective engagement (Fredricks, Blumenfeld, & Paris, 2004; Jimerson, Campos, &

Greif, 2003).

First, cognitive engagement can be recognised as expending additional effort to understand

multifaceted concepts and/or overcome upscaled expertise (Finn & Zimmer, 2012; Fredricks et

al., 2004). Diverse authors evidence the relevance of the cognitive engagement dimension in

many online contexts (e.g., Burkey, 2019; Ghosh, 2019; Putman, Ford, & Tancock, 2012; Zhu,

2006). Cognitive engagement in an online context can be perceived as the attention and effort
iew

students apply to interact with peers and tutors in discussions, comments, or posts (Bilro,
v

Loureiro, & Ali, 2018; Putman et al., 2012). Moreover, it includes students’ upscale expertise,
Re

such as analysing, reviewing, or reasoning (Putman et al., 2012; Shukla & Sharma, 2018; Zhu,

2006). Second, behavioural engagement can usually be perceived in technological contexts

er

based on discussions or replies (Amegbe, Owino, & Kerubo, 2017; Bilro & Loureiro, 2020;
Pe

Leclercq et al., 2018). The literature debates if the number of interactions among peers is a

measure of behaviour engagement (Goggins & Xing, 2016) and the linkage between

5
discussions among students and achievements (Ramos & Yudko, 2008). Lastly, the literature

defines emotional engagement as the students’ psychological response to academic settings,

such as boredom or enjoyment from the learning activities (Finn & Zimmer, 2012). The

relationship between tutors and students, the connection between students and their peers, and

students’ interests or enjoyment in making part of the online discussions also influences this

dimension (Fredricks et al., 2004).

Methods

Qualitative research based on a quasi-experimental design is adopted in this study, as we aim

to demonstrate causality between an intervention and its outcome (Harris et al., 2006). To

operationalise the study, we conduct a quasi-experimental design without a control group

(Shadish, Cook, & Campbell, 2002) and expose 91 undergraduate students to various

gamification moments during a full Semester in a specific curricular unit. Participants were all

management degree students, ranging from 19 to 25 years old, and gender-balanced was

achieved (57,14% female). We have built and assess a services scenario using a gamification

tool, Kahoot!, “a game-based learning platform that makes it easy to create, share and play

fun learning games or trivia quizzes in minutes. Users can play Kahoot! on any mobile device

or computer with an internet connection.” (Kahoot!, 2019). During the semester, researchers
iew

ask participants to use Kahoot! in class, as final modules quizzes, as an assessment tool for
v

other colleagues’ group presentations, or others. At no moment during the semester was
Re

explained to students why they were using Kahoot!. The goal was explained only at the end of

the semester, and students opinions were collected. Researchers collected the data through one
er

single open-ended question made available on Qualtrics, assessing their opinion about the
Pe

advantages and disadvantages of this type of gamification-based tool: “What is your opinion

about the use of Kahoot! inside the classroom? Which are the main advantages and

6
disadvantages that you see in using this type of Gamification tools?”. Participants were first

asked to provide informed consent, fill out a pre-experiment questionnaire, and anonymity was

guaranteed. Students access the form during a class through their own devices, answering

individually. Researchers also conducted a post-experimental debrief to participants, as we are

dealing with experience-based learning activities (Stewart, 1992). From a total of 91

participants, it was possible to collect 73 valid answers after blank responses or other non-

suitable answers were eliminated (80,2% success rate). The sample size is considered adequate

(Simmons, Nelson, & Simonsohn, 2013) and in line with the current practise (e.g., Javornik,

2016; Zhao & Patrick Rau, 2020).

To extract, examine, and transpose the vast amount of information into valuable

knowledge, it became necessary to proceed with a text mining technique (Fan, Wallace, Rich,

& Zhang, 2006; Költringer & Dickinger, 2015; Srivastava & Sahami, 2009). Researchers

performed a sentiment analysis (SA) procedure to understand the participants’ sentiments and

attitudes toward the gamification moments. Data and text mining techniques allow researchers

to analyse information and process non-structured text to find relevant knowledge decoded into

actionable information (Fan et al., 2006; Zhang, Zeng, Li, Wang, & Zuo, 2009). The text mining

techniques usually include distinct actions such as text clustering, topic extraction, text

categorisation, among others (Li & Wu, 2010). For this study, we use the MeaningCloud
iew

software, a powerful tool to extract meaningful knowledge from all types of unstructured
v

content, allowing researchers to perform text analysis, text classification, or sentiment analysis
Re

(MeaningCloud, 2019b). MeaningCloud Sentiment Analysis API uses semantic approaches

based on advanced natural language process (NLP) in all aspects of morphology, syntax,
er

semantics and pragmatics (MeaningCloud, 2019b). Distinct research has opted for this software
Pe

in different scientific domains (e.g., Bilro, Loureiro, & Guerreiro, 2019; Kaur & Chopra, 2016;

Martínez et al., 2016; Segura-Bedmar, Martínez, Revert, & Moreno-Schneider, 2015). This tool

7
can analyse a vast amount of data through NLP. NLP aid computers to comprehend the human’s

natural language, allowing machines to interpret the relevant elements of the human language

sentence and produces an interpretation of the text so it can be analysed (Godbole,

Bhattacharya, Gupta, & Verma, 2010; Loureiro, Bilro, & Japutra, 2020; Mostafa, 2013). To

assess the validity and reliability of the text mining outcomes, the authors lay the hands into

existing literature measuring and comparing MeaningCloud tool results. Previous studies

highlight that MeaningCloud presents high validity and reliability when compared with several

other sentiment analysis tools (van Aggelen, 2015). van Aggelen (2015) argues that the

operation made by MeaningCloud of decomposing the text provides concurrently valid and

consistent sentiment analysis, as the outcomes agree with other concurring tools, with only

minor deviations. Moreover, literature also put in evidence that when compared with other

tools, the results obtained by MeaningCloud are very good, presenting a high percentage of

correctly classified text, with minor detected errors (Gonzalez-Marron, Mejia-Guzman, &

Enciso-Gonzalez, 2017).

Results

The data analysis was made using distinct text-mining procedures. The first procedure is topic

analysis, and topic extraction was made from the data on MeaningCloud. Table 1 shows the ten
iew

most mentioned topics discussed by the students in their responses.

v
Re

[INSERT TABLE 1 ABOUT HERE]

er

The main topics in students’ answers are connected to the classes themselves, such as class,
Pe

student, classroom, or subject. These are followed by topics connected to match, competition,

or interactive. These results are in line with the expected outcome. Therefore, students perceive

8
gamification firstly as an educational tool, and only after that, they mention the facets more

connected to the game or the competition itself. Based on these results, it is possible to argue

that students understood the experience they have had as an element of education, and only

after as an element of entertainment.

The second procedure is to analyse the most mentioned words. To analyse and group the

set of words most used, we resort to a WordCloud, a visual illustration of text data, used to

portray keyword metadata or visualise the free form of text. For this purpose, HTML5

WordCloud was used. The respondents’ most mentioned word is ‘class’, followed by ‘student’

(see Figure 1).

[INSERT FIGURE 1 ABOUT HERE]

Other words, such as game, competition, or winner, are also highly ranked. However, as

seen in Figure 1, respondents focus their opinion on words connected to education, such as

class, students, or subjects. Again, respondents seem to understand that the purpose of this type

of tool is related to education rather than entertainment.

Moving on in our text-mining analysis, we perform the third procedure through deep

categorisation of the available text. Deep categorisation “assigns one or more categories of a
iew

predefined taxonomy to different snippets of a text. By applying a powerful semantic rule

v

technology, it provides maximum accuracy in the classification while allowing the fastest and
Re

most efficient definition of models” (MeaningCloud, 2019a). The deep categorisation reveals

that the respondents’ main categories are education, educational assessment, technology and
er

computing, video gaming, and mobile games. Again, the text categories are in line with the
Pe

previous analysis, with a clear focus on educational environments (Table 2).

9
 [INSERT TABLE 2 ABOUT HERE]

To complement the analysis, researchers use a fourth text-mining procedure as it became

relevant to carry out a cluster analysis, to classify our object (in this case, the text corpus) into

related groups that are similar to each other (i.e., clusters) (Li & Wu, 2010; Punj & Stewart,

1983). The software creates clusters and attributes a score to each cluster to identify the clusters

with higher scores. Due to the large group of clusters that emerged, researchers decide to use

only the clusters with higher scores, establishing a cut-off for clusters scoring 150 or higher

(see Table 3).

[INSERT TABLE 3 ABOUT HERE]

The clusters with the highest scores are ‘Good tool’ (602.60), ‘Good way’ (602.55) and

‘Paying attention’ (531.20). We can argue that students understand this type of gamification

tools as a good approach to adopt in educational environments, namely inside classrooms.

Students also highlight that it is a fun way to learn (255.64) and outweighs the disadvantages

(245.31). Some students also refer to a more practical aspect of using this type of technology

in class: internet Wi-Fi should always be available to not detract from the overall experience
iew

(244.80).
v

Finally, to understand the sentiment level of each participant in the study it is performed a
Re

global sentiment analysis, which intends to map the overall sentiments expressed in the text

(i.e., participants’ answers) (Cambria, Schuller, Xia, & Havasi, 2013; Liu, 2015). Through the
er

MeaningCloud software, it is possible to analyse the sentiments expressed in each comment or

Pe

answer and attribute a polarity scale for each one, from Positive + (P+) to Negative + (N+). As

10
seen in Table 4, most answers express positive sentiments (72.60%), and only 8.22 % of the

answers reveal negative sentiments.

[INSERT TABLE 4 ABOUT HERE]

Based on these results, most participants have a positive sentiment about this practice in

higher educational settings. Moreover, less than 10% of respondents revealed negative

sentiments toward the experience, and around 20% reveal neutral sentiments. Based on these

results, we argue that this group of participants’ overall experience is positive or very positive

and that participants feel engaged by using gamification tools in service settings such as

educational environments.

Discussion

This study explores if gamification tools influence students’ engagement in services,

specifically in Higher Education Settings. The literature already points out some behavioural

changes observed in participants of gamified environments, namely online (e.g., Chaudhary,

2010; Koivisto & Hamari, 2014; Turan et al., 2016). With this research, this paper contributes

to the ongoing discussion by putting in evidence the potential of gamification outside online
iew

environments and in distinct areas of practical application, collecting evidence of its possible
v

positive impact in service settings. Moreover, it is possible to realise that students perceive
Re

gamification as an educational tool, and only after that as a game or a competition itself. The

findings show that students understand the experience as an educational element, and only after
er

as entertainment. Based on the results, it is also possible to argue that students understand this
Pe

type of gamification tools as a good approach to implement in educational environments,

namely inside classrooms.

11
 This study also shed light on the impact of gamification activities on student engagement,

which can be seen as a behavioural and psychological involvement in learning activities

(Appleton et al., 2008). Our findings show that participants’ overall experience is positive or

very positive and feel engaged using gamification tools. Additionally, by resorting to sentiment

analysis, it is possible to find and measure the participants’ general sentiment about the

gamification approach to support the learning process. Sentiment analysis is an exciting way to

facilitate understanding of how participants feel about something, about their emotional states

and their opinions, answers or comments expressed freely, without limitations or restrictions

(Altrabsheh, Cocea, & Fallahkhair, 2014; Liu, 2015). Most of our participants’ sentiments

expressed (72.60%) were positive, revealing that this type of gamification tool and its direct

application can contribute to greater interest and motivation.

Conclusions and implications

Theoretical contributions

This paper proposes to study how gamification’s technology-based approach could influence

students’ engagement with the contents made available in services. Using a gamification tool,

we have built and assess a services scenario to collect students’ motivations, interests, and
iew

engagement in higher education settings. The findings may contribute to theory development

and improvement of services and higher education institutions (HEI).

v

First, this paper underlines that gamification influences students’ engagement in services,
Re

such as higher education settings. When asked, students have reported high levels of

engagement behaviours when taking part in the Kahoot! activities. However, the
er

implementation of gamification-based activities within an educational environment requires

Pe

special attention to ensure the best possible outcome from the whole experience for participants

and providers. Following Huang & Soman (2013), caution should be made about the

12
preparatory actions to be taken, such as the definition of objectives, the understanding of the

target audience, or identifying the available resources that could support the application of

gamification elements.

Second, this study uses a novel approach by applying the S-O-R framework to explain the

role of gamification in student engagement in higher education settings, particularly the use of

Kahoot! as a gamification tool. The student engagement, grounded in the S-O-R environmental

psychology framework, posits how environmental stimuli are perceived and processed,

ultimately impacting attitudinal and behavioural responses (Donovan & Rossiter, 1982;

Mehrabian & Russell, 1974). Our study empirically extends the SOR framework to the study

of students’ engagement. Research employing this framework suggests that technological

experiences resulting from controlled stimuli may be critical to engagement (Mosteller &

Mathwick, 2014). Psychological engagement, defined as a cognitive and affective commitment

to a functional relationship (Mollen & Wilson, 2010), may manifest from the gamification

process experienced.

Managerial implications

From a managerial perspective, students’ engagement in gamification activities shows relevant

managerial implications due to the interest and behavioural engagement students reveal in these
iew

activities. First, this study aims not to analyse the gamification tool but highlight the importance
v

of this type of gamification-based tool to stimulate participants and increase their interest in the
Re

contents made available through an active and engaging offer. Kahoot! is a straightforward and

interactive online tool, which can be considered an entertainment tool. Managers should bear
er

in mind that it can be a powerful tool to stimulate participation and interest when used in a
Pe

service setting. In our study, participants consider it a ‘good tool’ and a ‘good way to pay

attention’ and, most of all, a ‘fun way to learn” which is also an exciting condition to promote

13
participant satisfaction and engagement (Borrás-Gené, Martínez-Núñez, & Martín-Fernández,

2019). Second, practitioners (either college administrations, lecturers or others) may understand

and promote this type of tools as a leverage to create participation and engagement in their

audiences. Gamification providers, such as firms, brands or HEI, need to master the best way

to apply gamification elements in a service setting properly and, above all, to be aware of this

technology-based approach advantages or disadvantages.

Limitations and future research

This paper focuses on gamification tools’ effect on students’ engagement in a service setting

such as HEI. Although this research is based on a rigorous method and data analysis, unveiling

relevant findings that broaden literature knowledge, is not without its limitations. First,

limitations arise from using a quasi-experimental design, which has recognised boundaries

(Harris et al., 2006), such as the lack of randomisation or the presence of temporal confounders.

Second, while data mining and text mining have many potential benefits and values, there are

still some technical limitations to their capabilities. Data and text mining can analyse and assign

a comment to one group of ideas (e.g., topic analysis, clusters) based on its content

characteristics (word similarities defined in the NLP). However, that information can contain

multiple cognitive aspects or contradictory categorisation cues that may result in missing data
iew

assigned to specific groups of ideas (Seifert, 2004). Third, using a single gamification-based
v

tool (Kahoot!) can also be perceived as a limitation. Other gamification tools could be used,
Re

and/or a mix of tools and methods could contribute in an aggregated way to influence the

participant’s engagement process. Since applying a single gamification-based tool has shown
er

positive results in participants’ engagement, it can be relevant to test different approaches and
Pe

scenarios, seeking to identify additional effective methods that may contribute to participants’

interest and engagement.

14
 Consistent with our research, future researchers may consider building on the S-O-R

framework applied to student engagement to investigate the structural paths linking

environmental stimuli, psychological processes and engagement-related responses. Moreover,

further research can also be performed by comparing distinct types of gamification tools with

the same sample to understand who ensures the best learning outcomes. Finally, future research

may also use different university courses in distinct cultural contexts to strengthen the findings

and consider students’ emotional, cognitive, and social dimensions and their role in these

gamification educational approaches.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest concerning the research, authorship

and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this

article.
iew

References
Altrabsheh, N., Cocea, M., & Fallahkhair, S. (2014). Sentiment Analysis: Towards a Tool for
Analysing Real-Time Students Feedback. Proceedings - 26th International Conference
v

on Tools with Artificial Intelligence, ICTAI, 2014, 419–423.

Re

https://doi.org/10.1109/ICTAI.2014.70

Amegbe, H., Owino, J. O., & Kerubo, O. L. (2017). Behavioural Responses to Corporate
er

Image Building Through Social Media Advertising: A Study Among Nairobi Students.
Pe

Journal of Creative Communications, 12(3), 223–238.

https://doi.org/10.1177/0973258617725607

Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school:

15
 Critical conceptual and methodological issues of the construct. Psychology in the
Schools, 45(5), 369–386. https://doi.org/10.1002/pits.20303

Barata, G., Gama, S., Jorge, J., & Gonçalves, D. (2017). Studying student differentiation in
gamified education: A long-term study. Computers in Human Behavior, 71, 550–585.
https://doi.org/10.1016/j.chb.2016.08.049

Bilro, R. G., & Loureiro, S. M. C. (2020). A consumer engagement systematic review:

synthesis and research agenda. Spanish Journal of Marketing-ESIC, 24(3), 283–307.
https://doi.org/10.1108/SJME-01-2020-0021

Bilro, R. G., Loureiro, S. M. C., & Ali, F. (2018). The role of website stimuli of experience on
engagement and brand advocacy. Journal of Hospitality and Tourism Technology, 9(2),
204–222. https://doi.org/10.1108/JHTT-12-2017-0136

Bilro, R. G., Loureiro, S. M. C., & Guerreiro, J. (2019). Exploring online customer
engagement with hospitality products and its relationship with involvement, emotional
states, experience and brand advocacy. Journal of Hospitality Marketing and
Management, 28(2), 147–171. https://doi.org/10.1080/19368623.2018.1506375

Borrás-Gené, O., Martínez-Núñez, M., & Martín-Fernández, L. (2019). Enhancing fun

through gamification to improve engagement in MOOC. Informatics, 6(28), 2–20.

Brodie, R. J., Hollebeek, L. D., Jurić, B., & Ilić, A. (2011). Customer engagement:
Conceptual domain, fundamental propositions, and implications for research. Journal of
Service Research, 14(3), 252–271. https://doi.org/10.1177/1094670511411703

Broer, J. (2014). Gamification and the trough of disillusionment. In A. Butz, M. Koch, & J.
iew

Schlichter (Eds.), Mensch & Computer 2014 - Workshopband (pp. 389–395). Berlin: De
Gruyter Oldenbourg.
v

Burkey, B. (2019). Total Recall: How Cultural Heritage Communities Use Digital Initiatives
Re

and Platforms for Collective Remembering. Journal of Creative Communications, 14(3),

235–253. https://doi.org/10.1177/0973258619868045

Cambria, E., Schuller, B., Xia, Y., & Havasi, C. (2013). New avenues in opinion mining and
er

sentiment analysis. IEEE Intelligent Systems, 28(2), 15–21.

Pe

https://doi.org/10.1109/MIS.2013.30

Chaudhary, A. G. (2010). Educational Gaming: An Effective Tool for Learning and Social
Change in India. Journal of Creative Communications, 5(3), 135–152.

16
 https://doi.org/10.1177/0973258612471244

Damnjanovic, V., Proud, W., & Milosavljevic, M. (2020). Mentoring development at student
international business case competitions. EuroMed Journal of Business, Ahead-of-p.
https://doi.org/10.1108/EMJB-12-2018-0092

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to
gamefulness: Defining gamification. Proceedings of the 15th International Academic
MindTrek Conference on Envisioning Future Media Environments - MindTrek ’11, 9–11.
https://doi.org/10.1145/2181037.2181040

Dichev, C., & Dicheva, D. (2017). Gamifying education: what is known, what is believed and
what remains uncertain: a critical review. International Journal of Educational
Technology in Higher Education, 14(9), 1–36. https://doi.org/10.1186/s41239-017-0042-
5

Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Gamification in education: a
systematic mapping study. Educational Technology & Society, 18(3), 75–88.
https://doi.org/10.1109/EDUCON.2014.6826129.

Donovan, R. J., & Rossiter, J. R. (1982). Store Atmosphere: An Environmental Psychology

Approach. Journal of Retailing, 58(1), 34–57. https://doi.org/Article

Eroglu, S. A., Machleit, K. A., & Davis, L. M. (2003). Empirical Testing of a Model of
Online Store Atmospherics and Shopper Responses. Psychology and Marketing, 20(2),
139–150. https://doi.org/10.1002/mar.10064

Faghihi, U., Brautigam, A., Jorgenson, K., Martin, D., Brown, A., Measures, E., &
iew

Maldonado-Bouchard, S. (2014). How gamification applies for educational purpose

specially with college algebra. Procedia Computer Science, 41, 182–187.
https://doi.org/10.1016/j.procs.2014.11.102
v
Re

Faiella, F., & Ricciardi, M. (2015). Gamification and Learning: a Review of Issues and
Research. Journal of E-Learning and Knowledge Society, 11(3), 13–21.
https://doi.org/10.20368/JE-LKS.V11I3.1072
er

Fan, W., Wallace, L., Rich, S., & Zhang, Z. (2006). Tapping the power of text mining.
Pe

Communications of the ACM, 49(9), 76–82. https://doi.org/10.1145/1151030.1151032

Finn, J. D., & Zimmer, K. S. (2012). Student engagement: What is it? Why does it matter? In
S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student

17
 engagement (pp. 97–131). New York: Springer.

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of
the concept, state of the evidence. Review of Educational Research, 74(1), 59–109.
https://doi.org/10.3102/00346543074001059

Frost, R. D., Matta, V., & MacIvor, E. (2015). Assessing the efficacy of incorporating game
dynamics in a learning management system. Journal of Information Systems Education,
26(1), 59–70.

Fuchs, M. (2018). Subversive Gamification BT - Playful Disruption of Digital Media. In D.

Cermak-Sassenrath (Ed.), Playful Disruption of Digital Media (pp. 181–191). Singapore:
Springer Singapore. https://doi.org/10.1007/978-981-10-1891-6_12

Galbis-Córdova, A., Martí-Parreño, J., & Currás-Pérez, R. (2017). Higher education students’
attitude towards the use of gamification for competencies development. Journal of E-
Learning and Knowledge Society, 13(1), 129–146.

Garczynski, A. M., Waldrop, J. S., Rupprecht, E. A., & Grawitch, M. J. (2013).

Differentiation between work and nonwork self-aspects as a predictor of presenteeism
and engagement: Cross-cultural differences. Journal of Occupational Health
Psychology, 18(4), 417–429. https://doi.org/10.1037/a0033988

Ghosh, M. (2019). Analysing the Engagement and Attitude of Elderly Towards Digital
Platforms in India. Journal of Creative Communications, 14(3), 214–234.
https://doi.org/10.1177/0973258619872085

Godbole, S., Bhattacharya, I., Gupta, A., & Verma, A. (2010). Building re-usable dictionary
iew

repositories for real-world text mining. Proceedings of the 19th ACM International
Conference on Information and Knowledge Management - CIKM ’10, 1189.
https://doi.org/10.1145/1871437.1871588
v
Re

Goggins, S., & Xing, W. (2016). Building models explaining student participation behavior in
asynchronous online discussion. Computers and Education, 94, 241–251.
https://doi.org/10.1016/j.compedu.2015.11.002
er

Gonzalez-Marron, D., Mejia-Guzman, D., & Enciso-Gonzalez, A. (2017). Exploiting Data of

Pe

the Twitter Social Network Using Sentiment Analysis. In Applications for future internet
(pp. 35–38). Cham: Springer. https://doi.org/10.1007/978-3-319-49622-1

Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does gamification work? - A literature review of

18
 empirical studies on gamification. Proceedings of the Annual Hawaii International
Conference on System Sciences, 3025–3034. https://doi.org/10.1109/HICSS.2014.377

Harris, A. D., Mcgregor, J., Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D. E., &
Finkelstein, J. (2006). The Use and Interpretation of Quasi-Experimental Studies in
Medical Informatics. J Am Med Inform Assoc., 13, 16–23.
https://doi.org/10.1197/jamia.M1749

Hollebeek, L. D., Clark, M. K., Andreassen, T. W., Sigurdsson, V., & Smith, D. (2020).
Virtual reality through the customer journey : Framework and propositions. Journal of
Retailing and Consumer Services, 55. https://doi.org/10.1016/j.jretconser.2020.102056

Hollebeek, L. D., Glynn, M. S., & Brodie, R. J. (2014). Consumer brand engagement in social
media: Conceptualization, scale development and validation. Journal of Interactive
Marketing, 28(2), 149–165. https://doi.org/10.1016/j.intmar.2013.12.002

Huang, W. H.-Y., & Soman, D. (2013). A Practitioner’s Guide To Gamification Of

Education. In Rotman School of Management. Toronto.

Iosup, A., & Epema, D. (2014). An experience report on using gamification in technical
higher education. Proceedings of the 45th ACM Technical Symposium on Computer
Science Education - SIGCSE ’14, (2008), 27–32.
https://doi.org/10.1145/2538862.2538899

Ip, C., Law, R., & Lee, H. A. (2011). A review of website evaluation studies in the tourism
and hospitality fields from 1996 to 2009. International Journal of Tourism Research,
13(3), 234–265. https://doi.org/10.1002/jtr.815
iew

Javornik, A. (2016). ‘It’s an illusion, but it looks real!’ Consumer affective, cognitive and
behavioural responses to augmented reality applications. Journal of Marketing
Management, 32(9–10), 987–1011. https://doi.org/10.1080/0267257X.2016.1174726
v
Re

Jimerson, S. R., Campos, E., & Greif, J. L. (2003). Toward an Understanding of Definitions
and Measures of School Engagement and Related Terms. The California School
Psychologist, 8(1), 7–27. https://doi.org/10.1007/bf03340893
er

Kahoot! (2019). Kahoot! | Learning Games | Make Learning Awesome! Retrieved February 4,
Pe

2019, from https://kahoot.com/

Kapp, K. M. (2012). The Gamification of Learning and Instruction (John Wiley & Sons Inc.,
Ed.). San Francisco.

19
Kapp, K. M., & Coné, J. (2012). What Every Chief Learning Officer Needs to Know about
Games and Gamification for Learning. White Paper, (Institute for Interactive
Technologies).

Kaur, A., & Chopra, D. (2016). Comparison of text mining tools. 2016 5th International
Conference on Reliability, Infocom Technologies and Optimization, ICRITO 2016:
Trends and Future Directions, 186–192. https://doi.org/10.1109/ICRITO.2016.7784950

Kayimbaşioǧlu, D., Oktekin, B., & Haci, H. (2016). Integration of Gamification Technology
in Education. Procedia Computer Science, 102.
https://doi.org/10.1016/j.procs.2016.09.460

Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from
gamification. Computers in Human Behavior, 35, 179–188.
https://doi.org/10.1016/j.chb.2014.03.007

Költringer, C., & Dickinger, A. (2015). Analysing destination branding and image from
online sources: A web content mining approach. Journal of Business Research, 68(9),
1836–1843. https://doi.org/10.1016/j.jbusres.2015.01.011

Leclercq, T., Hammedi, W., & Poncin, I. (2018). The Boundaries of Gamification for
Engaging Customers: Effects of Losing a Contest in Online Co-creation Communities.
Journal of Interactive Marketing, 44, 82–101.
https://doi.org/10.1016/j.intmar.2018.04.004

Leonidou, E., Christofi, M., Vrontis, D., & Thrassou, A. (2018). An integrative framework of
stakeholder engagement for innovation management and entrepreneurship development.
iew

Journal of Business Research, In press, 1–14.

https://doi.org/10.1016/j.jbusres.2018.11.054

Li, N., & Wu, D. D. (2010). Using text mining and sentiment analysis for online forums
v

hotspot detection and forecast. Decision Support Systems, 48(2), 354–368.

Re

https://doi.org/10.1016/j.dss.2009.09.003

Lim, F., & Puspita, V. (2020). Apple’s Tale: The Engagement and Sense-Making of Apple’s
er

Story-Based Advertisements. Journal of Creative Communications, 15(3), 255–268.

Pe

https://doi.org/10.1177/0973258620924948

Liu, B. (2015). Sentiment analysis: Mining opinions, sentiments, and emotions. New York,
USA: Cambridge University Press.

20
Looyestyn, J., Kernot, J., Boshoff, K., Ryan, J., Edney, S., & Maher, C. (2017). Does
gamification increase engagement with online programs? A systematic review. PloS
One, 12(3), e0173403. https://doi.org/10.1371/journal.pone.0173403

Loureiro, S. M. C., Bilro, R. G., & Angelino, F. J. A. de. (2020). Virtual reality and
gamification in marketing higher education: a review and research agenda. Spanish
Journal of Marketing - ESIC, In press. https://doi.org/10.1108/SJME-01-2020-0013

Loureiro, S. M. C., Bilro, R. G., & Japutra, A. (2020). The effect of consumer-generated
media stimuli on emotions and consumer brand engagement. Journal of Product &
Brand Management, 29(3), 387–408. https://doi.org/10.1108/JPBM-11-2018-2120

Loureiro, S. M. C., Romero, J., & Bilro, R. G. (2020). Stakeholder engagement in co-creation
processes for innovation: A systematic literature review and case study. Journal of
Business Research, 119, 388–409. https://doi.org/10.1016/j.jbusres.2019.09.038

Mahnič, N. (2014). Gamification of politics: start a new game. Teorija in Praksa, 51(1), 143–
161.

Martín-Gutiérrez, J., Mora, C. E., Añorbe-Díaz, B., & González-Marrero, A. (2017). Virtual
technologies trends in education. Eurasia Journal of Mathematics, Science and
Technology Education, 13(2), 469–486. https://doi.org/10.12973/eurasia.2017.00626a

Martínez, P., Martínez, J. L., Segura-Bedmar, I., Moreno-Schneider, J., Luna, A., & Revert,
R. (2016). Turning user generated health-related content into actionable knowledge
through text analytics services. Computers in Industry, 78, 43–56.
https://doi.org/10.1016/j.compind.2015.10.006
iew

MeaningCloud. (2019a). Deep Categorization | MeaningCloud. Retrieved August 6, 2019,

from Deep Categorization website: https://www.meaningcloud.com/products/deep-
categorization
v
Re

MeaningCloud. (2019b). Our Company | MeaningCloud. Retrieved July 10, 2019, from Our
company website: https://www.meaningcloud.com/our-company

Mehrabian, A., & Russell, J. A. (1974). An Approach to Environmental Psychology. In An

er

approach to environmental psychology. Cambrige, MA: MIT Press.

Pe

Mollen, A., & Wilson, H. (2010). Engagement, telepresence and interactivity in online
consumer experience: Reconciling scholastic and managerial perspectives. Journal of
Business Research, 63(9–10), 919–925. https://doi.org/10.1016/j.jbusres.2009.05.014

21
Morimoto, S. A., & Friedland, L. A. (2013). Cultivating Success: Youth Achievement,
Capital and Civic Engagement in the Contemporary United States. Sociological
Perspectives, 56(4), 523–546. https://doi.org/10.1525/sop.2013.56.4.523

Mostafa, M. M. (2013). More than words: Social networks’ text mining for consumer brand
sentiments. Expert Systems with Applications, 40(10), 4241–4251.
https://doi.org/10.1016/j.eswa.2013.01.019

Mosteller, J., & Mathwick, C. (2014). Reviewer online engagement: the role of rank, well-
being, and market helping behavior. Journal of Consumer Marketing, 31(6/7), 464–474.
https://doi.org/10.1108/JCM-05-2014-0974

Nobre, H., & Ferreira, A. (2017). Gamification as a platform for brand co-creation
experiences. Journal of Brand Management, 24(4), 349–361.
https://doi.org/10.1057/s41262-017-0055-3

Oleksy, T., & Wnuk, A. (2017). Catch them all and increase your place attachment! The role
of location-based augmented reality games in changing people - place relations.
Computers in Human Behavior, 76, 3–8. https://doi.org/10.1016/j.chb.2017.06.008

Oxford. (2019). Gamification At Oxford Dictionary.

Pagowsky, N. (2012). Etcv 524 Literature review: Motivation in Gamified Learning

Scenarios. Arizona: University of Arizona.

Piñeiro-Otero, T., & Costa-Sánchez, C. (2015). ARG (Alternate Reality Games).

Contributions, Limitations, and Potentialities to the Service of the Teaching at the
University Level. Comunicar, 22(44), 141–148. https://doi.org/10.3916/C44-2015-15
iew

Priya, B., & Kalpana. (2014). Gamification: the Best Approach for Student Engagement.
International Journal of Management, 5(2), 976–6502.
v

Punj, G., & Stewart, D. W. (1983). Cluster Analysis in Marketing Research: Review and
Re

Suggestions for Application. Journal of Marketing Research, 20(2), 134–148.

https://doi.org/10.2307/3151680
er

Putman, S. M., Ford, K., & Tancock, S. (2012). Redefining online discussions: Using
participant stances to promote collaboration and cognitive engagement. International
Pe

Journal of Teaching and Learning in Higher Education, 24(2), 151–167.

Ramos, C., & Yudko, E. (2008). “Hits” (not “Discussion Posts”) predict student success in

22
 online courses: A double cross-validation study. Computers and Education, 50(4), 1174–
1182. https://doi.org/10.1016/j.compedu.2006.11.003

Robson, K., Plangger, K., Kietzmann, J. H. J., McCarthy, I., & Pitt, L. (2015). Is it all a
game? Understanding the principles of gamification. Business Horizons, 58(4), 411–420.
https://doi.org/10.1016/j.bushor.2015.03.006

Rosado-Pinto, F., & Loureiro, S. M. C. (2020). The growing complexity of customer

engagement: a systematic review. EuroMed Journal of Business, 15(2), 167–203.
https://doi.org/10.1108/EMJB-10-2019-0126

Roschk, H., Loureiro, S. M. C., & Breitsohl, J. (2017). Calibrating 30 Years of Experimental
Research : A Meta-Analysis of the Atmospheric Effects of Music , Scent , and Color.
Journal of Retailing, 93(2), 228–240. https://doi.org/10.1016/j.jretai.2016.10.001

Segura-Bedmar, I., Martínez, P., Revert, R., & Moreno-Schneider, J. (2015). Exploring
Spanish health social media for detecting drug effects. BMC Medical Informatics and
Decision Making, 15(2), S6. https://doi.org/10.1186/1472-6947-15-S2-S6

Seifert, J. W. (2004). Data mining an overview. In analyst in information science and

technology policy resources, science, and industry division. The Library of Congress.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-
experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.

Sheldon, L. (2012). The Multiplayer Classroom: Designing Coursework as a Game. In Course

Technology. Boston, MA: Cengage Learning, Inc. https://doi.org/10.1111/teth.12095

Shukla, S., & Sharma, P. (2018). Emotions and Media Multitasking Behaviour among Indian
iew

College Students. Journal of Creative Communications, 13(3), 197–211.

https://doi.org/10.1177/0973258618790794
v

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2013). Life after p-hacking. Meeting of the
Re

Society for Personality and Social Psychology, 17–19. New Orleans, LA.
https://doi.org/10.2139/ssrn.2205186
er

Sprott, D., Czellar, S., & Spangenberg, E. (2009). The Importance of a General Measure of
Brand Engagement on Market Behavior: Development and Validation of a Scale.
Pe

Journal of Marketing Research, 46(1), 92–104. https://doi.org/10.1509/jmkr.46.1.92

Srivastava, A. N., & Sahami, M. (2009). Text mining: Classification, clustering, and

23
 applications. New York, USA: Chapman & Hall/CRC.

Stewart, L. P. (1992). Ethical Issues in Postexperimental and Postexperiential Debriefing.

Simulation & Gaming, 23(2), 196–211. https://doi.org/10.1177/1046878192232007

Stott, A., & Neustaedter, C. (2013). Analysis of Gamification in Education. Carmster.Com,

1–8.

Subhash, S., & Cudney, E. A. (2018). Gamified learning in higher education: A systematic
review of the literature. Computers in Human Behavior, 87(May), 192–206.
https://doi.org/10.1016/j.chb.2018.05.028

Turan, Z., Avinc, Z., Kara, K., & Goktas, Y. (2016). Gamification and education:
Achievements, cognitive loads, and views of students. International Journal of
Emerging Technologies in Learning, 11(7), 64–69.
https://doi.org/10.3991/ijet.v11i07.5455

van Aggelen, J. M. (2015). Concurrent Validity and Consistency of Social Media Sentiment
Analysis Tools.

Watson, R., Wilson, H. N., Smart, P., & Macdonald, E. K. (2018). Harnessing Difference: A
Capability-Based Framework for Stakeholder Engagement in Environmental Innovation.
Journal of Product Innovation Management, 35(2), 254–279.
https://doi.org/10.1111/jpim.12394

Woodcock, J., & Johnson, M. R. (2017). Gamification: What it is, and how to fight it. The
Sociological Review, 66(3), 542–558. https://doi.org/10.1177/0038026117728620

Yang, Y., Asaad, Y., & Dwivedi, Y. (2017). Examining the impact of gamification on
iew

intention of engagement and brand attitude in the marketing context. Computers in

Human Behavior, 73, 459–469. https://doi.org/10.1016/j.chb.2017.03.066
v

Yildirim, I. (2017). The effects of gamification-based teaching practices on student

Re

achievement and students’ attitudes toward lessons. Internet and Higher Education, 33,
86–92. https://doi.org/10.1016/j.iheduc.2017.02.002
er

Zhang, C., Zeng, D., Li, J., Wang, F. Y., & Zuo, W. (2009). Sentiment analysis of chinese
documents: From sentence to document level. Journal of the American Society for
Pe

Information Science and Technology, 60(12), 2474–2487.

https://doi.org/10.1002/asi.21206

24
Zhao, J., & Patrick Rau, P. L. (2020). Merging and synchronizing corporate and personal
voice agents: Comparison of voice agents acting as a secretary and a housekeeper.
Computers in Human Behavior, 108(March), 106334.
https://doi.org/10.1016/j.chb.2020.106334

Zhu, E. (2006). Interaction and cognitive engagement: An analysis of four asynchronous

online discussions. Instructional Science, 34(6), 451–480.
https://doi.org/10.1007/s11251-006-0004-0

v iew
Re
er
Pe

25
1 Fig. 1.
2 Wordcloud of the most mentioned words
3

4
5

6
v iew
Re
er
Pe
1 Table 1. 10 most mentioned topics.
2
Topic Frequency
class 58
student 35
classroom 29
match 21
subject 10
competition 10
Kahoot 10
people 10
question 9
interactive 9
3
4 Table 2. Text categories.
5
Categories
Education
Business and Finance>Industries
Technology and Computing>Consumer
Electronics
Science>Environment
Education>Educational Assessment
Business and Finance>Business
Video Gaming
Video Gaming>Mobile Games
Technology and Computing>Computing
6

7 Table 3. Word clusters.

8
Cluster Score
Good tool 602.60
iew

Good way 602.55

Paying attention 531.20
Knowledge 370.08
Game 284.24
v

Fun way to learn 255.64

Re

Stop paying attention 250.24

Outweigh the disadvantages 245.31
Wifi 244.80
Feel 243.10
er

Motivation 231.84
Good Idea 231.03
Competitive environment 198.76
Pe

Way of learning 188.90

Serious 174.45
Interactive way 168.70
Limited time 154.72
Way to teach 154.28
 9 Table 4. Sentiment analysis results.
Sum % P-N %
P+ 8 10,96%
72.60%
P 45 61,64%
NEU\NONE 14 19,18% 19.18%
N 6 8,22%
8.22%
N+ 0 0,00%
Total 73 100,00% 100,00%
10

11

12

v iew
Re
er
Pe