Review of Education

Review of Education
Vol. 9, No. 1, February 2021, pp. 27–52

DOI: 10.1002/rev3.3231
Does the ARCS motivational model affect

students’ achievement and motivation?
A meta-analysis
Idris Goksu1* and Yusuf Islam Bolat2
1
Department of Educational Sciences, Mardin Artuklu University, Mardin, Turkey,
2
Department of Computer Education and Instructional Technology, Dicle University,
Diyarbakir, Turkey
In this meta-analysis, the aim is to determine the overall effect of the ARCS (attention, relevance,
confidence, satisfaction) model of motivation on students‘ academic achievement, motivation,
attention, relevance, confidence and satisfaction. Additionally, the effect of the model is analysed
according to the learning environment in which the model is integrated, discipline area, education
level and sample size. The primary studies included in the study were obtained from Web of
Science, Scopus, Science Direct, ERIC and PsycARTICLE databases. A total of 38 controlled
experimental studies in the form of peer-reviewed articles were coded, resulting in 110 extracted
effect size (ES). The sample size of the primary studies involves 8690 students from K-12 (kinder-
garten to 12th grade) and higher education. Random-effects model was used for overall ES, mixed-
effects model for categorical moderators and meta-regression analysis for integer moderators. As a
result of the study, it was determined that the overall effect of ARCS on achievement was at medium
level (ES = 0.74) and the overall effect on motivation was at small level (ES = 0.43). ESs of
achievement differed by the moderators of discipline and the ESs of motivation differed by the mod-
erator of education level. There was not a significant relationship between the effect of ARCS on
achievement and motivation, and moderator of sample size. Remarkable ESs have been obtained in
the learning environment moderator related to the dependent variables of blended learning, robots,
augmented reality; in the discipline moderator related to architecture, computer technologies,
sciences and maths; in the level of education moderator related to undergraduate variables. In addi-
tion, very large ES related to the attention component of ARCS was obtained. The results obtained
in this study do not represent the strength of the evidence, as it is based on the validity and reliability
of primary studies.
Keywords academic achievement, ARCS, meta-analysis, motivation.
Introduction
Technological developments play an important role in education as well as in many
other areas. The development of educational technologies also changes students’
expectations from their learning environments, needs, and individual characteristics.
This situation necessitates the change of learning approaches in the teaching process.
Instructional design has an important role in the process of determining the appropri-
ate learning approaches, identifying the needs and developing the appropriate
This study was presented as an oral presentation at the 13th International Computer and Instruc-
tional Technologies Symposium (ICITS-2019), Kırsßehir, Turkey, 2–4 May 2019.
*Corresponding author: Department of Educational Sciences, Mardin Artuklu University, 47100,
Mardin, Turkey. Email: idrisgoksu@artuklu.edu.tr
© 2020 British Educational Research Association

28 I. Goksu, Y. Islam Bolat
instructional materials and gaining new knowledge and skills that the students need.
Today, when the constructivist learning approach is adopted, it is important for the
student to be active in the learning process and to participate effectively. The motiva-
tion factor which is thought to have a strong influence on learning while designing the
teaching (Maehr & Archer, 1985; Huett, 2006), keeps the student active in the pro-
cess and facilitates active participation in the course.
Motivation is a decisive factor in the process of the student’s effort in the related
activity, the continuation of the related activity, willingness and learning by feeling
connected to the related activity (Di Serio et al., 2013). Keller defines motivation as
‘what people desire, what they choose to do, and what they commit to do’ and states
that investigation of motivation answers the question of ‘why we do the things we do’
(Keller, 2010, p. 3). Motivation helps teachers learn how students can learn better
and also plays a key role in students’ learning (Pintrich & De Groot, 2003). Students
with high motivation can become better interrogators (Martens et al., 2004) and this
enables students to develop. Therefore, the change in the level of motivation is
reflected in the behaviour of students. However, it is also claimed that students with
higher motivation may not have better learning outcomes (Martens et al., 2004). On
the other hand, there are also studies emphasising that motivation has a significant
effect on academic achievement (Sarier, 2016; Fong et al., 2017). Drawing attention to
the importance of motivation in the instructional design process, Keller (1987) devel-
oped the ARCS (attention, relevance, confidence, satisfaction) model of motivation.
ARCS model of motivation

The ARCS model emerged from the need to find more effective and systematic ways
to understand the main effects on learning motivation, and studies on the develop-
ment of the model first began in 1979 (Keller, 1987a). This model, which aims to
improve the motivational feature of teaching materials, has four components (atten-
tion, relevance, confidence, satisfaction) derived from a synthesis of research on moti-
vation (Abdessettar et al., 2016) and defines the motivation. At the same time, the
ARCS model includes strategies that improve the motivational appeal of teaching
and offers a systematic design process (Keller, 1987a). The ARCS model, which is
also known for its motivating feature in technology-based learning environments,
adopts the design and problem-solving approach, revealing its distinctive feature.
The model follows a linear learning process in which the attention component of this
model stimulates students’ curiosity; the relevance component combines students’
experiences, goals and perceived importance of the learning process with instructional
content; the confidence component improves students’ beliefs that they will excel in
learning processes; the satisfaction component enables students to evaluate feedback
as a result of the learning process (Keller, 1987a, 1987b, 1987c). The components of
the ARCS model and the strategies it contains are summarised in Figure 1.
According to Keller (2010), first, a course should attract and arouse the interest of
students. At the same time, the continuity of this attention and interest should be
ensured. Attracting attention only is not enough to be motivated. Students should
believe that the learning content is related to their personal goals or motives and they
should feel connected to the learning environment. In addition, motivation may not

The effect of ARCS Model 29
Figure 1. Components and strategies of the ARCS model (Keller, 1987a)
be at the proper level if the expectation of confidence and success is too low or too
high. Students should get rid of well-established fears that hinder their learning about
lessons or attainments and should not miss important details in their learning activi-
ties. Finally, students should be satisfied with both the learning process and the
results for the continuity of the desire to learn. Considering these aspects, which are
based on the components of the ARCS model, will help make instruction stimulating
and help students feel responsible for their achievement (Keller, 2010).
According to the ARCS model—which is the most common motivation model and
has a history of about 40 years—to motivate the student, the teacher or instructional
material should attract the attention of students, ensure the continuity of this atten-
tion, indicate why they should learn, convince them that they will succeed if they
make an effort and help them to feel a sense of reward and pride (Keller, 1987a,
2010). This model can be used in many fields such as lesson design, teacher training,
multimedia design, science education and instructional message design (Suzuki et al.,
2004). However, it is mostly preferred in computer-based applications (Li & Keller,
2018). Instead of using an isolated strategy in the instructional design process, it is
recommended that the ARCS model be integrated into the process (Keller, 2010). If
the instructional design is based on the ARCS model, difficulties in managing learn-
ing and motivation components in learning environments and especially in the inte-
gration of technology and innovative systems may be reduced (Keller, 2016). It is
suggested that the ARCS model works well with instructional design principles and is
useful for improving student engagement (Reynolds et al., 2017). The ARCS model
is encouraging and motivating for students to continue learning (Huett, 2006). When
the review studies are examined, it is seen that the ARCS model has widespread use
and is based on many scientific research studies. Li & Keller (2018) examined the
experimental studies based on the ARCS model and found that the model was

applied to educational environments in many countries. Goksu et al. (2014, 2017)

examined a number of instructional design studies in their content analysis research
and found that the ARCS model is the most preferred instructional design model
after the ADDIE (analysis, design, development, implementation, evaluation) model.
All the components of the ARCS model are generally used in research based on
ARCS (Li & Keller, 2018).
It is noteworthy that experimental studies on the ARCS model are more common in
some subject areas (e.g. STEM [science, technology, engineering, mathematics], tech-
nical, professional and vocational, English as a second language—Li & Keller, 2018).
The ARCS model also has an effective design tool for the motivational development of
distance learning environments (Huett, 2006). In addition, the ARCS model is an
effective model in technology-supported foreign language education as it provides
active learning and the model significantly increases academic achievement in commu-
nication skills (Piriyasurawong, 2019). It is suggested that if the ARCS model is applied
in language learning, students’ motivation and academic performance will increase and
they will participate more in cooperative learning activities (Refat et al., 2019). The
ARCS-based game-based learning (GBL) can be a unique and effective approach for
high-quality education, such as academic research skills (Abbott, 2019). Furthermore,
augmented reality (AR) applications based on the ARCS model contribute significantly
to motivation in vocational education (Bacca et al., 2004).
The ARCS model can be applied at all educational levels (e.g., K-12 [kindergarten
to 12th grade], higher education etc.—Li & Keller, 2018). Computer-based instruc-
tion designed in line with the ARCS model in the 4th-grade level mathematics course
positively affected both academic achievement and attitude towards the course
(Karakisß et al., 2016). It is seen that AR applications developed according to the
ARCS model are more effective in terms of motivation than traditional methods in a
5th-grade English course, but do not make a significant difference in terms of learning
effectiveness (Hao & Lee, 2019). In a study of midwifery education in higher educa-
tion, the ARCS model was found useful in coping with motivational difficulties asso-
ciated with the technology-based learning experiences of educators (Stockdale et al.,
2019). In addition to academic achievement and motivation, students generally
showed positive attitudes towards the environments where the ARCS model was inte-
grated (Li & Keller, 2018).
The fact that the ARCS model was developed specifically to increase students’
motivation is seen as an important reason for researchers to conduct experimental
research on this model (Li & Keller, 2018). It is seen that quantitative methods are
preferred in the empirical studies about the ARCS model (Li & Keller, 2018) and in
these studies, the variables of academic achievement and motivation are examined. It
has been found that the effect of technology on students’ motivation has been mea-
sured with a questionnaire containing ARCS model components in many studies
(Bacca et al., 2004; Ho et al., 2019; Julia & Antolı, 2019; Khan et al., 2019). Li & Kel-
ler (2018) emphasise this situation in their review study. The fact that the results
obtained from the studies on the ARCS model are inconsistent (Li & Keller, 2018),
and that the model remains weak in explaining the integration of information-pro-
cessing elements into the learning process and explaining the interaction with motiva-
tion (Woo, 2014) is seen as an issue that needs to be emphasised. When the literature

was examined, no studies were found that compiled and deduced research related to
the ARCS model other than the meta-review study of Li and Keller (2018). The fact
that the mentioned study is focused only on qualitative results indicates that there is a
need for a quantitative study. In addition, the absence of any studies inferring the
quantitative results of experimental research on the ARCS model reveals the impor-
tance of our study. In this context, the meta-analysis method is followed in this study
and it is aimed to answer the following research questions (RQ):
RQ 1. What is the overall effect of the ARCS model on students’ academic achievement?
RQ 1.1. How does the effect on academic achievement differ according to the learning
environment and the field of discipline?
RQ 1.2. How does the effect on academic achievement differ according to the educational
level and sample size?
RQ 2. What is the overall effect of the ARCS model on students’ motivation?
RQ 2.1. How does the effect on motivation differ according to the learning environment
and the field of discipline?
RQ 2.2. How does the effect on motivation differ according to the educational level and
sample size?
RQ 2.3. What is the overall effect of the ARCS model on attention, relevance, confidence,
and satisfaction?
Method
In this study, the meta-analysis design was used to determine the overall effect of the
ARCS model on students’ academic achievement and motivation. The aim is to make a
general inference by combining the results of the experimental studies examining the
effect of this model. The meta-analysis method used for this purpose allows us to com-
bine the quantitative data of the research studies focusing on a specific subject and to
make a general judgment (C ß ogaltay & Karadag, 2015; Tatsioni & Ioannidis, 2017). It is
suggested that the sample is enlarged by bringing together the results of experimental
research and thus the validity of the research is increased (Ellis, 2012). Card (2012) sug-
gests that the method in meta-analysis studies should consist of literature search proce-
dures, study inclusion and exclusion criteria, coding of study characteristics and data
analytic strategy. Lam and Kennedy (2005) point out that the importance of meta-anal-
ysis results depends on the quality of the studies included in the meta-analysis.
Literature search procedures

The research studies included in this study were obtained by searching Web of
Science, Scopus, ERIC, PsycARTICLE and Science Direct databases. On 16

January 2020, the results of [‘ARCS’ AND (‘instructional’ OR ‘learning’)] query and
peer-reviewed articles were refined and 1495 results were obtained. There is no year
or language limitation when searching.
Inclusion and exclusion criteria

Abstracts of the studies obtained from the search were examined by the researchers; a
total of 1302 studies with multiple copies, not related to the ARCS model or non-arti-
cle type were excluded from the study. Thus, a total of 193 articles were obtained.
The full texts of the related articles were examined by the authors, and articles meet-
ing the following criteria were included in the scope of the study:
a)Conducted with a controlled experiment group.
b)The experimental learning environment designed in accordance with the ARCS
model.
c)At least one of the data sets are ‘mean, standard deviation and sample size’, ‘sample
size and p-value’, ‘Cohen’s d and sample size’ or ‘mean-difference, t value and sam-
ple size’.
d)Contain parametric analyses.
e)The equivalence of the groups is statistically provided or randomly assigned.
f)Participants should be in grades K-12 or higher education.
g)Includes investigation of academic achievement and/or motivation variables.
Articles that did not meet the above criteria were excluded. In some studies where
motivation was examined, the motivation results of the related studies were not
included in the study, since the item-based analysis was performed instead of examin-
ing the total effect of the ARCS model or its effect on the basis of dimensions. As a
result, 155 articles have been eliminated and 38 articles constituted the scope of this
study (see Appendix D). The sample of these studies included 8690 (Experi-
ment = 4344, Control = 4346) students.
Coding of study characteristics

The full texts of the 38 articles included in the study were examined (by conducting
simultaneous meetings with TeamViewer software) by researchers from different uni-
versities (the first researcher is a CEIT doctoral graduate and still working as an assis-
tant professor, and the second researcher has passed the CEIT doctoral qualification
and is also a research assistant in the CEIT department) and the data was processed
into a form developed with Google Sheets. The articles were read together by two
researchers and the data were processed into the related form by a joint decision. The
related form consists of these characteristics: study id (A1, A2, . . .), dependent vari-
able (academic achievement as AA, motivation as M and/or sub-dimensions of moti-
vation as M-A, M-R, M-C or M-S), level of education, discipline, learning
environment, sample size, experimental group (Mean, SD, participant), control
group (Mean, SD, participant), p-value, Cohen’s d, mean difference and t-value.
Level of education was coded as: elementary school (Grade 1 to 4, 6–10 age),

middle school (Grade 5–8, 10–14 age), high school (Grade 9 to 12, 14–18 age),
college (pre-undergraduate stage, usually involving vocational training) and under-
graduate. The courses in which primary studies are applied were coded as the cate-
gories of architecture, computer technologies, English language, health sciences,
maths, science and social sciences. Learning environments in which the ARCS model
is applied in primary studies were coded as augmented reality, blended learning, class-
based activities, computer-assisted learning, e-mail messages, game-based learning,
mobile learning, printed material and robots categories. While characteristics of pri-
mary studies are coded, research patterns, duration of interventions and the scale types
are also determined. These characteristics are presented in Table 1. In order to make
Table 1 shorter and more understandable, instead of giving all data for each effect size
(ES) (a total of 110 ES was calculated in our study), some data were presented inter-
mittently (min-max) or shortened. Some evaluations made by the researchers about
some situations detected during the coding process are presented below:
UThe sample group in A1 and A16 was evaluated as undergraduate.
UThere are three different groups in A3. The most suitable ones were evaluated as
experimental and control groups.
UStudents in A4 were separated into two groups (higher level of intrinsic motivation
group and lower level of intrinsic motivation group). Consequently, two separate
ESs regarding both academic achievement and motivation were calculated.
UIn A6, motivational outcomes were not taken into account because the analyses
cover only one dimension of the motivation scale.
UIn A7, motivational outcomes were not taken into account because the analyses
cover only one dimension of the ‘Students’ Motivation towards Science Learning
Scale’.
UIn A19, the writing-based group, which is the most far from the ARCS features,
was evaluated as the control group.
UIn A20, since the p-value was given as < 0.0001, the p-value was evaluated as
0.00009.
UIn A28, two different ESs were calculated for academic achievement as Practice
and Theory.
UIn A30, the permanence test was taken into account.
UIn A32, the values that could not be reached were obtained from the master thesis
from which it was derived.
UIn A35, four different ESs were calculated; two ESs (high, low anxiety) for geome-
try and two ESs (high, low anxiety) for algebra.
UIn A37, since the motivation scale was not valid and reliable, motivational results
were not included in the study.
UIn A38, five different ESs were calculated regarding academic achievements such
as structure, dimension, flow, layout and symbol.
Data analysis
When the data of 38 studies were examined, it was seen that in four different combi-
nations—(1) mean, standard deviation, sample size, (2) sample size, p-value, (3)

Table 1. Coded studies
34
Exp. Mean Exp. SD Exp. Cnt.Mean Cnt. SD Control

Id DV(Scale) RD Duration Grade Discipline LE (Min-Max) (Min-Max) Sample (Min-Max) (Min-Max) Sample p/t/d/ Mean diff
A1 AA (?), M ! ? G5 Social sciences C7 Printed material 33 33 0.716, 0.06235

(IMMS) LE8
A2 AA (1), M RCT ? G5 Physiology C4, Printed material 12.80–43.6 3.40–16.1 25 7.9–48.00 2.5–13.9 25
(IMMS) Statistics C5 LE8
A3 AA (1), M RCT 1d G3 (10th) Biology C6 CAL LE4 7.94–52.73 2.5–12.09 22 6.25–42.84 3.35–13.75 19
(IMMS)
A4 AA (2), M RCT 2 wk G5 EFL C3 Interactive 11.99–130.62 4.49–16.45 79–80 8.92–120.43 3.09–15.26 76–78
(IMMS) multimedia LE4
A5 AA (1,3), M (8) RCT ? G5 Multimedia C7 Web-Lecture LE4 22 22 0.009, 0.203
A6 AA (?) RCT 3 wk G5 Research design C7 Instructional text 25.69 4.90 26 25.06 6.33 31
I. Goksu, Y. Islam Bolat
LE8
A7 AA (1) Q-ex 10 hr G3 (11th) Chemistry C6 Learning activities 68.49 10.69 52 59.90 10.99 51
LE3
A8 AA (?), M (CIS) RCT one wk G5 Archaeological C7 E-mail messages 2.56–7.89 0.71–2.65 30 2.03–8.96 0.88–2.48 71
LE5
A9 M (SQ) Q-ex 12 wk G3 Tech and engineering Learning activities 138.82 14.53 45 122.04 19.80 48
C2 LE3
A10 M (9) RCT 2 wk G5 Computer apps C2 E-mail messages 54 37 0.85, 0.94, 0.122,

LE5 0.71, 0.94
A11 AA(IPT), M RCT 5(1/2) wk G5 Computer Course C2 E-mail messages 31.77–93.4 5.43–7.28 30–35 28.70–86.1 7.35–7.39 26–37
(IMMS) LE5
A12 M (8) RCT 6 wk G5 Instructional Tech C7 Blended learning 25 25 15.169, 2.38667,
LE2 7.597, 1.68,
4.585, 0.9,
2.963, 0.56
A13 AA (1,LQ) Q-ex 6 wk G2 (7th) English GBL LE6 89.44 7.45 32 81.25 9.59 32
(Listen&Speak) C3
A14 AA (1), M (8) Q-ex 16 wk G4 Information Tech C2 Multimedia LE4 63.7–81.76 8.46–10.25 110 65.3–80.78 7.41–12.51 110
A15 M (9), AA (1) RCT 3d G4 Maths C5 Video LE4 AA(15.59) AA(0.397) 22 AA(15.26) AA(0.43) 21 0.346
A16 M(IMMS), AA Q-ex 1 wk G5 English reading C3 RTA LE9 3.68 0.37 32 3.38 0.51 30 0.03
(ERQ)
A17 AA (?), M RCT 3 wk G5 Access database C2 Blended learning 22.62–128.08 3.9–16.83 45 19.48–117.48 4.26–16.7 45
(IMMS) LE2
A18 AA (?) Q-ex 8 wk G5 Software engineering GBL LE6 80.24 9.33 33 72.14 12.01 30
C2
A19 AA (1), M RCT 90 min G5 English C3 DGBL LE6 3.00–50.8 0.6–14.0 41 2.7–49.0 0.6–12.6 41
(IMMS)
A20 AA (1,0) Q-ex 1 wk G4 Architecture C1 GBL LE6 77.61 36 55.08 36 0.0001
A21 AA (?) RCT 3 wk G1 (4th) Science C6 GBL LE6 82.94 10.01 34 74.71 8.96 34
A22 AA (1) ! 8 wk G5 Physics C6 Flipped Learning 74.63 11.69 36 62.12 15.92 30
LE2
Table 1. (Continued)
Exp. Mean Exp. SD Exp. Cnt.Mean Cnt. SD Control
Id DV(Scale) RD Duration Grade Discipline LE (Min-Max) (Min-Max) Sample (Min-Max) (Min-Max) Sample p/t/d/ Mean diff
A23 AA (1,4), M Q-ex 6 wk G5 EFL/C3 Mobile inquiry 24.2–56.0 3.24–18.02 35 22.28–51.88 2.12–17.3 32
(IMMS) learning LE7
A24 AA (1), M RCT 30 min G5 Anathomy C4 Computer 3d 2.7–14.2 0.7–2.8 49 2.5–13.9 0.8–3.7 51
(IMMS) material LE4
A25 AA (1) Q-ex 4 wk G2 (5th) Maths C5 CAL LE4 11.33 4.20 24 8.72 3.52 22
A26 AA (Rubric) RCT 16 wk G5 Computer prog C2 Robots LE9 86.15 8.67 33 70.41 10.53 27
A27 M (8) Q-ex 1 sem G3 (11th) Physics C6 Blended learning 20.06–31.62 1.25–1.97 34 18.67–30.07 1.32–2.25 27
LE2
A28 AA-p,t (?) Q-ex 56 hr G5 Civil engineering C1 Mobile Learning 90.21, 97.34 5.32, 6.54 50 60.51, 67.72 3.22, 4.12 50
LE7
A29 AA (EP) RCT 2 hr G2 (6th) Science C6 Mobile learning 93.49 4.19 43 87.72 4.96 43
LE7
A30 AA (DKT), M RCT 3 hr G5 Library instruction C7 GBML LE6 22.8–50.73 2.47–6.8 18 15.5–44.18 2.18–5.28 18
(IMMS)
A31 AA (?) ! 12 mo G5 Biology C6 Class-based LE3 70 69 0.002
A32 AA (1), M ! 4 wk G5 Maths C5 Interactive 23.13–97.70 5.05–15.51 48 20.35–88.13 5.32–23.1 46
(IMMS) multimedia LE4
A33 AA (1,4,5) Q-ex 7 wk G5 English (vocabulary) GBL LE6 72.22 4.45 32 69.17 3.17 30

C3
A34 AA (1) ! 1 wk G5 Biology C6 E-book LE4 65.71 21.73 35 47.43 17.55 35
A35 AA (1,5,6) ! 3 wk G2 (6th) Geometry C5, Algebra Mobile AR LE1 79.65–89.64 4.68–6.28 18–22 58.57–76.13 5.55–6.34 19–23
C5
A36 AA(1,3,4), M(8) ! 4 wk G2 (5th) English C3 AR learning 4.51–84.78 0.63–21.28 71 3.07–81.40 0.9–21.50 76
(mobile) LE1
A37 AA (KT) RCT 4 wk G4 Computer prog C2 GBL LE6 15.32 1.86 25 12.88 2.44 25
A38 AA (7) ! 15 wk G5 Interior design C1 Mobile AR LE1 16.04–16.94 1.69–2.32 52 2.13–3.17 13.35–15.52 48
AA: Academic achievement, M: Motivation, M-A: Motivation-Attention, M-R: Motivation-Relevance, M-C: Motivation-Confidence, M-S: Motivation-Satisfaction, DV:
Dependent variable, LE: Learning Environment, Q-ex: Quasi-experimental, GBL: Game-based Learning, !: RCT or Quasi-experiment, LQ: Listening Questions, RD:
Research Design, AR: Augmented Reality, G1: Elementary, G2: Middle School, G3: High School, G4: College, G5: Undergraduate, 0: Open-ended, 1: Multiple choice
questions, 2: Problem-solving question, 3: Y/N or T/F question, 4: Fill-in-the-blanks, 5: Cloze test, 6: Calculation questions, 7: Google forms quest, 8: ARCS based scale, 9:
ARCS based CIS, C1: Architecture, C2: Computer Technologies, C3: English Language, C4: Health Sciences, C5: Maths, C6: Science, C7: Social Sciences moderator,
LE1: Augmented Reality, LE2: Blended Learning, LE3: Class-based activities, LE4: Computer-assisted learning, LE5: e-mail messages, LE6: Game-based Learning, LE7:
Mobile Learning, LE8: Printed Material, LE9: Robots moderator, RCT: Randomized Controlled Trial, DKT: Declarative Knowledge Test, IPT: Interactive Performance
test, KT: Knowledge Test, ERQ: English Reading Quizzes, A1-A38: Refers to the Id of the article (References in Appendix D), ?: No information was reported in the study
or not clear, IMMS: Instructional Material Motivation Scale, CIS: Course Interest Survey, EFL: English as a foreign language, CAL: Computer-assisted learning, RTA:
Robot Teaching Assistant, DGBL: Digital Game-based Learning, EP: Exam Papers.[Correction added on 02 October 2020 after first online publication: The Table footnote
The effect of ARCS Model
has been included in this version].

35
Cohen’s d, sample size and (4) sample size, mean difference, t-value—totally 110
independent results were reached. The obtained data were transferred to Compre-
hensive Meta-Analysis (CMA) V.3 Software which calculates the Hedges’ g effect
size. In the first combination, 93 results were evaluated, 9 results in the second com-
bination, 4 results in the third combination and 4 results in the fourth combination.
Of these results, 44 are related to academic achievement (see Appendix A), 12 to
overall motivation, and 54 to the sub-dimensions of the ARCS model (Attention: 13,
Relevance: 13, Confidence: 15, Satisfaction: 13—see Appendix B). In this study, the
random-effects model was used to calculate the effect size because the primary studies
were gathered from the published literature (Borenstein et al., 2009). For the modera-
tors (educational level, discipline, learning environment) coded according to the
research questions, the mixed-effects analysis was preferred. Meta-regression was
performed for the sample size moderator. Also, the heterogeneity test (Q statistic)
was used to determine the existence and significance of variance between moderators
(Borenstein et al., 2009). Funnel Plot, Rossenthall Fail Safe N and Duval and Twee-
die’s Trim and Fill tests were used to identify potential publication bias. In Funnel-
Plot, the horizontal axis represents ES, and the vertical axis represents the standard
error. While interpreting the graph, in order to reach the conclusion that there is no
publication bias, it is expected that ESs will show symmetric distribution according to
the middle vertical line and standard error values will concentrate on the graph close
to zero (Borenstein et al., 2009). The Rossenthall Fail-Safe N test calculates the num-
ber of studies that do not have a meaningful value for the overall ES to decrease to an
insignificant level (Rosenthal, 1979). Duval and Tweedie’s Trim and Fill are meth-
ods of estimating the number of missing studies (Duval & Tweedie, 2000). Black cir-
cles indicate missing studies in Funnel Plot. The effect sizes obtained according to
the rules in the literature (very small: ES ≥ 0.01 and < 0.2, small: ES ≥ 0.2
and < 0.5, medium: ES ≥ 0.5 and < 0.8, large: ES ≥ 0.8 and < 1.2, very large:
ES ≥ 1.2 and < 2.0, Huge: ES ≥ 2.0) (Cohen, 1988; Sawilowsky, 2009).
Publication bias
Academic achievement: Testing publication bias in meta-analysis studies is considered
important for the correct interpretation of meta-analysis. First, Forest Plot graph of
44 ES related to academic achievement was created (see Appendix A) and four out-
liers (6.194, 5.702, 5.099, and 4.364), which may affect publication bias, were
deleted. Recalculation was made with the remaining 40 ES. Funnel Plot, Rossenthall
Fail-Safe N and Duval and Tweedie’s Trim and Fill test results were evaluated to test
potential publication bias. Funnel Plot, which shows the distribution of standard
errors and ES, is presented in Figure 2.
When Funnel Plot is examined, it can be seen that ESs are distributed symmetri-
cally. This situation reveals the conclusion that potential bias is low. Indeed, as a
result of the Duval and Tweedie’s Trim and Fill test, there were no missing studies
(studies trimmed = 0, observed and adjusted value are equal: 0.7415). According to
the Fail-Safe N test results, 3751 new studies should be included in order to decrease
the overall effect to a non-significant level.

Figure 2. Funnel plot of overall ES for achievement
Motivation: A total of 12 ES related to motivation and 54 ES (A: 13, R: 13, C: 15,

S: 13) related to the sub-dimensions of the ARCS model were calculated. Forest Plot
showing both the motivation and the distribution of the ESs belonging to A-R-C-S
dimensions is given in Appendix B. Forest Plot was examined and one ES from the
attention and one ES from relevance dimensions was evaluated as outlier
(ESA = 4.223, ESR = 3.524). These values are not included in the calculation
because they are thought to increase potential publication bias. According to the
results of the Duval and Tweedie’s Trim and Fill test on overall motivation, it turned
out that there were only two missing studies. As a result of Rosenthall Fail Safe N test,
it was found as N = 143. These tests, which were performed to evaluate potential
publication bias, show that the bias of the calculated ESs related to overall motivation
is low. Funnel Plot of general motivation is presented in Figure 3.
Funnel Plots that are created separately about the sub-dimensions of motivation
also show that publication bias is low (see Appendix C). In addition, according to the
results of Duval and Tweedie’s Trim and Fill test, it was determined that the missing
studies were two for attention, three for relevance, two for confidence and zero for
satisfaction. The results of the Rossenthall Fail-Safe N test revealed that many more
studies (NA = 469, NR = 217, NC = 467, NS = 469) should be included in order for
the overall effect to decrease to an insignificant level. All these results show that
potential publication bias is low for the sub-dimensions of motivation.
Results
Using the 110 different ESs obtained from 38 primary studies, the overall effect of
the ARCS model on academic achievement, motivation and A-R-C-S was examined.
In addition, the change of ES according to moderators of learning environment,

Figure 3. Funnel plot of overall ES for motivation
discipline, level of education and sample size was analysed in terms of motivation and
academic achievement. The random effects model for overall ES, the mixed-effects
model for categorical moderators and meta-regression analysis for integer moderators
were used. According to the results, it was revealed that the overall ES of the ARCS
model on the academic achievement was medium and its overall ES on the motiva-
tion was small. It was observed that the effect on academic achievement differed sig-
nificantly according to discipline, while it did not differ according to learning
environments and level of education. Also, the effect on motivation differed signifi-
cantly according to the level of education, but it did not differ according to discipline.
Meta-regression results related to the sample size moderator showed that as the num-
ber of samples increased, the effect of the ARCS model on academic achievement
and motivation decreased but this decrease was not significant. Finally, the ES
obtained for attention components of the ARCS model was found to be very large;
for relevance, confidence and satisfaction components it was medium level. The find-
ings are presented under each research question.
Overall effect of ARCS model on achievement

The overall effect of the ARCS model on academic achievement was calculated as
overall ES by combining 40 independent results from 34 primary studies. According
to the random effects model, the overall effect of the ARCS model on academic
achievement was calculated as 0.74 (CI: 0.56–0.92) with a standard error of 0.09.
This finding showed that the ARCS model has a medium effect on academic achieve-
ment. As a result of the analyses performed according to the random effects model,
each independent Hedges’ g value and some statistical data related to this value are
presented in Appendix A. Twelve of the calculated ESs showed no significant

difference between experiment group (ARCS based) and control group (non-ARCS
based), while 28 showed a significant difference in favour of experiment group. Three
of the calculated ESs are negative and 37 of them are positive. In addition, the highest
ES was calculated as 2.24 and the lowest as ES 0.07. In addition, six of the indepen-
dent ESs showing the effect of ARCS model on academic achievement were very
small, four were small, twelve were medium, nine were large, eight were very large
and one was huge.
Effect sizes of ARCS model on achievement by the moderators of learning environment and
discipline
The ES values calculated according to academic achievement and the results of the
mixed-effects analysis (variables with more than one ES) and heterogeneity test are
presented in Table 2 to determine the level of ARCS model according to the learning
environments in which it is integrated and whether it differs significantly from each
other.
As shown in Table 2, the effect of the ARCS model on academic achievement was
recalculated by grouping according to the learning environments in which it was inte-
grated. It was calculated as very large for blended learning (ES = 1.57, CI: 0.25–
2.89, p < 0.05), large for robots (ES = 1.08, CI: 0.03–2.13, p < 0.05) and aug-
mented reality (ES = 1.06, CI: 0.66–1.45, p < 0.05), medium for mobile learning
(ES = 0.74, CI: 0.25 1.73, p > 0.05), GBL (ES = 0.70, CI: 0.44 0.96, p < 0.05),
class-based activities (ES = 0.64, CI: 0.38–0.89, p < 0.05), and computer-assisted
learning (ES = 0.54, CI: 0.26–0.83, p < 0.05), small for printed material (ES = 0.35,
CI: 0.36–1.05, p > 0.05) and e-mail messages (ES = 0.34, CI: 1.17–1.85,
p > 0.05). As a result of the heterogeneity test conducted to determine whether there
was a significant difference according to the learning environment moderator, homo-
geneity value between groups was calculated as (QB) = 7.93 and p = 0.44. It was con-
cluded that the variance between the independent ESs calculated for the learning
environment moderator was not significant (QB(8) < 15.507, p > 0.05). These find-
ings show that the effect of the ARCS model on academic achievement does not make
Table 2. The effect sizes of ARCS model on achievement by the moderator of learning
environment
Learning environment n ES+ ES level SE p Lower CI Upper CI
Blended learning 2 1.57 Very large 0.67 0.02 0.25 2.89

Robots 2 1.08 Large 0.53 0.04 0.03 2.13
Augmented reality 8 1.06 Large 0.20 0.00 0.66 1.45
Mobile learning 2 0.74 Medium 0.51 0.14 0.25 1.73
Game-based learning (GBL) 8 0.70 Medium 0.13 0.00 0.44 0.96
Class-based activities 2 0.64 Medium 0.13 0.00 0.38 0.89
Computer assisted learning 10 0.54 Medium 0.14 0.00 0.26 0.83
Printed material 4 0.35 Small 0.36 0.34 0.36 1.05
E-mail messages 2 0.34 Small 0.77 0.66 1.17 1.85
QB = 7.93, p = 0.44, df(Q) = 8

a significant difference according to the integrated learning environment. The find-

ings showing the effect of the ARCS model on academic achievement according to
discipline are presented in Table 3.
The effect of the ARCS model on academic achievement was calculated according
to the discipline. It was very large for architecture (ES = 1.31, CI: 1.13–1.49,
p < 0.05), large for computer technologies (ES = 1.09, CI: 0.27–1.92, p < 0.05) and
science (ES = 0.81, CI: 0.63–1.00, p < 0.05), medium for maths (ES = 0.79, CI:
0.50–1.08, p < 0.05) and English language (ES = 0.53, CI: 0.30–0.76, p < 0.05),
very small for social sciences (ES = 0.10, CI: 0.28–0.49, p > 0.05) and health
sciences (ES = 0.05, CI: 0.41 0.32, p > 0.05). As a result of the heterogeneity
test conducted to determine whether there was a significant difference in terms of the
effect on academic achievement between the groups formed according to the disci-
pline moderator where the ARCS model is applied, it was calculated as (QB) = 70.53
and p = 0.00. It was concluded that the variance between independent ESs calculated
for the discipline moderator was significant (QB(6) > 12.592, p < 0.05). This finding
reveals that the effect of the ARCS model on academic achievement differs according
to the discipline field/course.
Effect sizes of ARCS model on achievement by the moderators of education level and sample
size
Mixed-effects analysis was carried out to determine the level of effect of the ARCS
model on academic achievement according to the education level of students. Also, a
heterogeneity test was performed to reveal whether ESs differ according to the level
of education. The findings are presented in Table 4.
The effect of the ARCS model on academic achievement is calculated as medium
for all levels of education. As a result of the heterogeneity test conducted to determine
whether there was a significant difference in terms of the effect on academic achieve-
ment between the groups formed according to the moderator of educational level, it
was calculated as (QB) = 0.13 and p = 0.99. It was concluded that the variance
between independent ESs calculated for the moderator of education level was not sig-
nificant (QB(3) < 7.816, p > 0.05). This finding shows that the effect of the ARCS
model on academic achievement does not differ by level of education.
Table 3. The effect sizes of ARCS model on achievement by the moderator of discipline
Field/Course n ES+ ES level SE p Lower CI Upper CI
Architecture 6 1.31 Very large 0.09 0.00 1.13 1.49

Computer technologies 6 1.09 Large 0.42 0.01 0.27 1.92
Science 7 0.81 Large 0.09 0.00 0.63 1.00
Maths 6 0.79 Medium 0.15 0.00 0.50 1.08
English Language 8 0.53 Medium 0.12 0.00 0.30 0.76
Social sciences 5 0.10 Very small 0.20 0.60 0.28 0.49
Health sciences 2 0.05 Very small 0.18 0.81 0.41 0.32
QB = 70.53, p = 0.00, df(Q) = 6

Table 4. The effect sizes of ARCS model on achievement by the moderator of education level
Level of education* n ES+ ES level SE p Lower CI Upper CI
Undergraduate 27 0.76 Medium 0.12 0.00 0.53 0.99

Middle school 6 0.72 Medium 0.19 0.00 0.34 1.10
High school 2 0.72 Medium 0.17 0.00 0.39 1.06
College 4 0.64 Medium 0.38 0.09 -0.10 1.37
QB = 0.13, p = 0.99, df(Q) = 3

*Elementary school with fewer than two valid studies was eliminated from the analysis.
Meta-regression analysis was performed to reveal the relationship between sample

size and ES. The results obtained are presented in Figure 4. Figure 4 shows that the
sample size ranges from 36 to 220 and calculated ESs range from 0.420 to 2.243.
The regression line shows that ES decreases as the sample increases. Regression coef-
ficient was calculated as 0.0022 (z = 1, p = 0.316, CI: 0.0065–0.0021).
According to these findings, it can be argued that as the sample size increases in
experimental studies based on the ARCS model, the resulting ES decreases. How-
ever, this decline was not significant.
Figure 4. Meta-regression analysis of sample size and ESs by random effect model (achievement)

Overall effect of ARCS model on motivation

The overall effect of the ARCS model on motivation was calculated by combining 12
independent results from 11 primary studies. According to the random effects model,
the overall effect of the ARCS model on motivation was 0.43 (CI: 0.27–0.58) with a
standard error of 0.08. This finding shows that the ARCS model has a small overall
effect on motivation. As a result of the analyses performed according to the random-
effects model, each independent Hedges’ g value and some statistical data related to
this value are presented in Appendix B. Six of the calculated ESs showed that there
was not a significant difference between the experiment group (ARCS based) and
control group (non-ARCS based), while six showed that there was a significant differ-
ence in favour of the experiment group. All of the ESs were positive. The highest ES
value was 0.84 and the lowest ES value was 0.08. In addition, one of the independent
ESs showing the effect of the ARCS model on motivation was large, four of them
were medium, six of them were small and one of them was very small.
Effect sizes of ARCS model on motivation by the moderators of learning environment and
discipline
Mixed-effects analysis was carried out to determine the level of effect of the ARCS
model on motivation according to the learning environment. As a result of the analy-
sis, more than one ES (n = 8) related to computer-assisted learning was calculated
and this ES was small (ES = 0.35, CI: 0.17–0.52, p < 0.05). Blended learning, e-mail
messages, mobile learning and robots with fewer than two valid studies were elimi-
nated from the analysis. Since there was more than one ES related to ‘Computer-as-
sisted learning’ only for the learning environment moderator, a heterogeneity test was
not conducted between the groups. The results of the mixed-effects analysis and
heterogeneity test, which reveal the effect of the ARCS model on the motivation
according to the discipline, are presented in Table 5.
The results of the mixed-effects analysis show that the ARCS model has a positive
and small effect on motivation in computer technologies, English language and maths
courses. Also, the results of the heterogeneity test which was conducted to determine
whether there was a significant difference between the groups formed according to
the discipline moderator where the ARCS model was applied, it was calculated as
(QB) = 0.06 and p = 0.97. It was concluded that the variance between independent
ESs calculated for the discipline moderator was not significant (QB(2) < 5.991,
Table 5. The effect sizes of ARCS model on motivation by the moderator of discipline
Field/Course* n ES+ ES level SE p Lower CI Upper CI
Computer technologies 3 0.49 Small 0.25 0.05 0.01 0.98

English Language 4 0.44 Small 0.13 0.00 0.18 0.70
Maths 2 0.42 Small 0.17 0.01 0.09 0.76
QB = 0.06, p = 0.97, df(Q) = 2

*Health science, social sciences and science with fewer than two valid studies were eliminated from the analysis.

p > 0.05). This finding reveals that the effect of the ARCS model on motivation does
not differ by the moderator of discipline.
Effect sizes of ARCS model on motivation by the moderators of education level and sample
size
The findings related to the ESs obtained regarding the undergraduate and college
level as a result of the mixed-effects analysis made according to the level of education
moderator are presented in Table 6. Also, the findings obtained as a result of the
heterogeneity test performed to determine whether there is a significant difference
according to the level of education are presented in Table 6.
When the effect of ARCS model on motivation according to educational level is
examined, the effect size was small for undergraduate (ES = 0.48, CI: 0.32–0.63,
p < 0.05) and very small for college (ES = 0.12, CI: 0.12 to 0.36, p > 0.05). This
finding shows that the ARCS model has a positive effect on motivation at both levels,
but this effect is greater at the undergraduate level. As a result of the heterogeneity
test conducted to determine whether there is a significant difference between the
groups formed by the moderator of the level of education, it was found as
(QB) = 5.90 and p = 0.02). It was concluded that the variance between independent
ESs calculated for the moderator of the level of education was significant
(QB(1) > 3.841, p < 0.05). This finding shows that the effect of the ARCS model on
motivation differs by the level of education.
Meta-regression analysis was performed to reveal the relationship between the
motivational ESs of the ARCS model and the sample size. The results obtained are
presented in Figure 5. Figure 5 shows that the sample size ranges from 41 to 220 and
the calculated ESs range from 0.085 to 0.843. The regression line shows that ES
decreases as the sample increases. Regression coefficient was calculated as 0.0022
(z = 1.91, p = 0.057, CI: 0.0044 to 0.0001). These findings show that as the sam-
ple size increases in experimental studies on which the ARCS model is based, the
motivation-related ES decreases. However, this decrease was not significant at the
0.05 level.
The effect sizes of ARCS model on attention, relevance, confidence and satisfaction
The results of the mixed-effects analysis and heterogeneity test, which were con-
ducted to determine the effect level of attention, relevance, confidence and
Table 6. The effect sizes of ARCS model on motivation by the moderator of education level
Level of education* n ES+ ES level SE p Lower CI Upper CI
Undergraduate 9 0.48 Small 0.08 0.00 0.32 0.63

College 2 0.12 Very small 0.12 0.33 0.12 0.36
QB = 5.90, p = 0.02, df(Q) = 1

*High school with fewer than two valid studies was eliminated from the analysis.

Figure 5. Meta-regression analysis of sample size and ESs by random effect model (motivation)
satisfaction according to the components of the ARCS model and whether the effect
differed according to the components, are presented in Table 7.
When Table 7 is examined, it is seen that the largest effect size was calculated for
attention (ES = 1.20, CI: 0.70–1.70, p < 0.05) and the smallest effect size was calcu-
lated for satisfaction (ES = 0.70, CI: 0.45–0.96, p < 0.05. According to the results of
the heterogeneity test, there was not a significant difference between the components
(QB(3) < 7.815, p > 0.05). The distribution of all ESs calculated by the moderators
in this study is summarized in Table 8.
Table 7. The effect sizes of ARCS model on attention, relevance, confidence, satisfaction
Components n ES+ ES level SE p Lower CI Upper CI
Attention 13 1.20 Very large 0.25 0.00 0.70 1.70

Relevance 15 0.72 Medium 0.14 0.00 0.44 1.00
Confidence 13 0.78 Medium 0.19 0.00 0.40 1.15
Satisfaction 13 0.70 Medium 0.13 0.00 0.45 0.96
QB = 3.21, p = 0.36, df(Q) = 3

Table 8. Calculated effect sizes and p values
Achievement Motivation A R C S
ARCS Overall ES 0.74 0.43 1.20 0.72 0.78 0.70

Learning Blended learning 1.57 * According to the
Environment Robots 1.08 * learning
Augmented reality 1.06 * environment, ES of
Mobile learning 0.74 * academic
Game-based learning (GBL) 0.70 * achievement does not
Class-based activities 0.64 * differ significantly
Computer assisted learning 0.54 0.35 (pa = 0.44).
Printed material 0.35 *
E-mail messages 0.34 *
Discipline Architecture 1.31 * According to the
Computer technologies 1.09 0.49 discipline, while ES of
Science 0.81 * academic
Maths 0.79 0.42 achievement differs
English Language 0.53 0.44 significantly
Social sciences 0.10 * (pa = 0.00), ES of
Health sciences 0.05 * motivation does not
differ (pm = 0.97).
Level of Undergraduate 0.76 0.48 According to education
education Middle school 0.72 * level, while ES of
High school 0.72 * motivation differs
College 0.64 0.12 significantly
(pm = 0.02),
academic
achievement does not
differ (pa = 0.99).
Sample Size There is no significant
relationship between
the effect of the ARCS
model on academic
achievement and
motivation and sample size
(pa = 0.316, pm = 0.057).
Note: *Moderators with fewer than two valid studies were eliminated from the analysis. pa: p-value of academic
achievement, pm: p-value of motivation.
Discussion and conclusion

In this study, the level of the overall effect of the ARCS model of motivation on stu-
dents’ academic achievement and motivation was determined via meta-analysis. We
also aimed to investigate whether this effect differed according to the learning envi-
ronment in which the ARCS model was integrated, the discipline, the level of educa-
tion, and the sample size. In addition, the effect of the ARCS model’s components
(attention, relevance, confidence and satisfaction) according to its size and whether it
differed or not was determined. A total of 110 independent ESs were calculated using
data obtained from 38 controlled experiment primary studies which were comparing
the ARCS model-based and non-ARCS model-based teaching methods. It is also

recommended that readers take into account that the findings of calculated effect
sizes do not represent the strength of the evidence. The findings obtained as a result
of random effects, mixed-effects and meta-regression analysis were discussed in the
literature and recommendations were made to researchers and practitioners.
As a result of this study, the overall effect of the ARCS model on academic achieve-
ment was medium (ES = 0.74) and the overall effect on motivation was small
(ES = 0.43). This result showed that the ARCS model contributed positively to both
academic achievement and motivation. However, it was shown that it contributes
more on academic achievement. The fact that instructional design enables students
to concentrate on learning (Darabi et al., 2018) explains this result. Huett (2006) also
stated that the motivation factor has a strong effect on learning. Similarly, it is argued
that motivation is a determining factor in the learning process (Di Serio et al., 2013;
Sarier, 2016; Fong et al., 2017). However, high motivation does not mean that aca-
demic achievement will be high (Martens et al., 2004). Although the ARCS model is
a motivation model, it is noteworthy that it has an effect on academic achievement.
When the results obtained in our study were evaluated, it was found that the ARCS
model of motivation should be preferred by teachers when designing teaching
because motivation facilitates the teaching process and makes an important contribu-
tion to the learning of students (Pintrich & De Groot, 2003; Reynolds et al., 2017).
The fact that the ARCS model has been applied to educational environments in many
countries (Li & Keller, 2018) reveals the importance of the model. Although the over-
all effect of the ARCS model on motivation is small, it is important that all compo-
nents are calculated at very large (ESA = 1.20) or medium-level (ESR = 0.72,
ESC = 0.78, ESS = 0.70). This result may be due to the fact that the ARCS model
mostly focuses on the effect of the sub-dimensions rather than the overall effect on
motivation in the primary studies. The fact that all the components of the ARCS
model are found to be used in researches based on the ARCS model (Li & Keller,
2018) supports the results of our study. In our study, it was noteworthy that ES was
detected at a very large level in the attention component and at medium level in other
components. Based on these results, it is seen that the attention dimension is applied
more effectively in learning environments where the ARCS model is based. However,
it should not be forgotten that attention is not sufficient for motivation and other
components should be at an appropriate level (Keller, 2010). In our study, the overall
effect of motivation was found lower than the A-R-C-S components, which may
result from the fact that all components could not be applied at the expected level.
When the effect of the ARCS model on academic achievement according to the
learning environments is considered, the ESs of blended learning (ES = 1.57), robots
(ES = 1.08) and AR (ES = 1.06) are remarkable. In addition, the ARCS model has a
medium effect on academic achievement in GBL, classroom-based, mobile-based
and computer-based learning environments. Abbott (2019) argues that the ARCS
model can be an effective approach if integrated into GBL. The suggestion that gami-
fied and interactive learning strategies are the most effective strategies (Davis et al.,
2018) and that ARCS-based AR applications increase motivation (Bacca et al.,
2004), support the findings of our study. It had a small effect when integrated into
print-material and e-mail messages. There is emphasis that the ARCS model can be
used in course design, multimedia design and instructional message design (Suzuki

et al., 2004). As a matter of fact, our study revealed that the ARCS model has a signif-
icant effect on many learning environments. These results show that the ARCS model
increases academic achievement mostly in technology-based learning environments.
In this study, the computer-based learning environment was emphasised regarding
the effect of the ARCS model on motivation. Li and Keller (2018) showed that the
ARCS model is preferred in computer-based applications. In studies where the effect
of the ARCS model on motivation is examined, the model is mostly integrated into
the computer-based learning environments. It is noteworthy that the number of pri-
mary studies examining the effect of the ARCS model in other learning environments
is very low. Therefore, we argue that more research is needed on this subject. In sum-
mary, the ARCS model allows integration in many different learning environments
within the context of academic achievement, but the environments in which it is inte-
grated with a motivational context are limited. Keller (2010) points out the impor-
tance of integrating the ARCS model into the process rather than using a single
strategy in the instructional design process. Thus, teachers can manage learning envi-
ronments to stimulate and sustain motivation (Keller, 1999). Moreover, Keller
(2016) argues that if the design of instruction is based on the ARCS model, difficul-
ties in managing learning and motivation components and technology integration
may be reduced.
When the effect of the ARCS model on academic achievement and motivation is
examined according to the course it is applied to, architecture (ES = 1.31), computer
technologies (ES = 1.09), science (ES = 0.81) and maths (ES = 0.79) stand out in
terms of the effect on academic achievement. Suzuki et al. (2004) state that one of the
areas where the ARCS model can be used is science education. Karakisß et al. (2016)
found that the ARCS model positively reflected academic achievement and attitude
in maths courses. It is maintained that the ARCS model may affect academic achieve-
ment in the English language at a medium level. In our study, the ARCS model had a
negative but very small effect in the field of health. On the other hand, Stockdale et al.
(2019) argued that the ARCS model has a significant contribution to motivation in
midwifery education, supporting the use of the ARCS model in the field of health.
Primary studies did not contain any motivational ES results, so no result could be
obtained related to it. So we can say that there is a need for experimental studies on
the effect of the ARCS model on motivation in the field of health. Furthermore,
according to the results of our study, the fact that the ARCS model has a significant
effect in the fields of science, computer technologies, architecture, and maths indi-
cates that it can be used in STEM education. This is supported by the fact that Li
and Keller (2018) found that an important part of the experimental studies on the
ARCS model focused on STEM. In this study, the ARCS model was found to be
effective in language education. Some studies in the literature also mention the
importance of the ARCS model in language education (Li & Keller, 2018; Piriyasura-
wong, 2019; Refat et al., 2019). Hao and Lee (2019) revealed that the ARCS model
had an effect on motivation in English, but did not significantly affect academic
achievement. Although the effect of the ARCS model on both academic achievement
and motivation is similar in many courses, the effect on academic achievement in
computer technologies and maths courses is higher than motivation. While the effect

of the ARCS model on academic achievement differs significantly according to disci-

pline (p = 0.00), its effect on motivation does not differ significantly (p = 0.97).
When the effect of ARCS model on academic achievement according to level of
education is examined, it has a medium effect at all levels (ESundergraduate = 0.76;
ESmiddle school = 0.72; EShigh school = 0.72; EScollege = 0.64). This result shows that
the model has a greater effect on academic achievement at undergraduate level
(n = 27). Glover (2014) emphasises that instructional design has become an impor-
tant approach in higher education. When the effect of the ARCS model on motivation
is examined, it has a medium effect at undergraduate (ES = 0.48) and a very small
effect at college level (ES = 0.12). Although the effect of the ARCS model varies
according to the level of education, it has a positive effect at all levels. Li and Keller
(2018) also suggested that the ARCS model can be applied at all levels of education.
While the effect of the ARCS model on academic achievement at the college level is
medium, it has a very small effect on motivation. This result may be due to the
methodological differences of the experimental studies that constitute the scope of
this study. In this study, while ES of motivation differs significantly according to the
level of education (p = 0.02), academic achievement does not differ (p = 0.99).
Another result obtained regarding the sample size of primary studies showed that as
the size of samples increases, the effect of the ARCS model on academic achievement
and motivation decrease but this decrease is not significant. In addition, it should not
be ignored that the effect size is generally larger in small sample studies than large
sample studies (Slavin & Smith, 2009).
In our study, the effects of the ARCS model on both academic achievement and
motivation has a wide potential for use in the context of the level of education, disci-
pline and learning environments. Li and Keller (2018) stated that this model not only
helps academic achievement and motivation but also enables students to show posi-
tive attitudes in the learning process. However, this study draws conclusions based on
the results obtained from previous experimental studies. Therefore, when interpret-
ing the results obtained from such review studies, it should not be ignored that pri-
mary studies may have some limitations. In addition, considering that the results of
experimental studies based on the ARCS model may show inconsistency (Li & Keller,
2018), it is recommended that researchers and practitioners interpret the results
obtained in our study with precision. Based on the results obtained in this study, the
following recommendations can be made to researchers and practitioners:
UThe ARCS model has been integrated into many learning environments and its
effect on academic achievement has been examined, but a limited number of learn-
ing environments are examined in terms of motivation. Therefore, researchers
investigating the effect of the ARCS model on motivation in various learning envi-
ronments can contribute to the field (Li & Keller, 2018).
UIt can be argued that teachers integrating the ARCS model into learning environ-
ments may contribute positively to both the academic achievement and motivation
of students.
UIt can be claimed that blended learning can be a suitable learning environment for
the ARCS model. However, considering that only the results of two studies based

on blended learning are combined to obtain the effect size in this study, it is consid-
ered important to carry out new research on this subject.
USince the 21st-century skills are of great importance nowadays, the ARCS model
has the potential to contribute to the acquisition of both computer technologies
and mathematical skills, so teachers are encouraged to make more use of the
model.
USince the ARCS model positively contributes to both academic achievement and
motivation in the fields of STEM (computer technologies, maths, etc.), it is consid-
ered important to make use of this model in the design of STEM education. In this
context, conducting researches that combine STEM and ARCS model may con-
tribute to the literature.
Limitations
This study includes only WoS, Science Direct, Scopus, ERIC and PsycARTICLE
databases and peer-reviewed article type publications. Future research may include
PubMed, Pro-Quest, JSTOR, EBSCO databases as well as publications other than
peer-reviewed articles (theses, conference papers, reviews, unpublished studies, etc.).
In addition, it is a limitation that the primary studies of this research have been
obtained with the keywords [‘ARCS’ AND (‘instructional’ OR ‘learning’)]. Although
we have determined inclusion and exclusion criteria to improve the quality of the
results (e.g. controlled experimental, equalised pre-tests), it can be considered as a
limitation that the included primary studies have various differences in terms of
methodological or statistical analysis. It is important to note that in future studies, pri-
mary studies should have more similar features. For example, new meta-analyses can
be limited to primary studies where features such as level of education, sample size,
learning environment are similar. Also, the academic achievement and motivation
effect sizes of the ARCS model in our study are affected by the validity and reliability
measures taken in primary studies. So the calculated ESs do not represent the
strength of the evidence.
Declaration of interest statement

The authors certify that they have no affiliations with or involvement in any organisa-
tion or entity with any financial interest or non-financial interest in the subject matter
or materials discussed in this manuscript.
Funding
This research was supported by Mardin Artuklu University Scientific Research Pro-
€
jects Coordination Unit. Project number: MAU.BAP.18.EF.014.
Conflict of interest
The authors declare that they have no conflict of interest.

References
Abbott, D. (2019) Game-based learning for postgraduates: an empirical study of an educational
game to teach research skills, Higher Education Pedagogies, 4(1), 80–104. https://doi.org/10.
1080/23752696.2019.1629825.
Abdessettar, S., Gardoni, M., Hotte, R. & Abdulrazak, B. (2016) Persuasive technologies for effi-
cient adaptable self-education, The Eighth International Conference on Mobile, Hybrid, and On-
Line Learning, 70–75.
Bacca Acosta, J. L., Baldiris Navarro, S. M., Fabregat Gesa, R. & Kinshuk, K. (2004) Framework
for designing motivational augmented reality applications in vocational education and training,
Australasian Journal of Educational Technology, 35(3), 102–117.
Borenstein, M., Hedges, L. V., Higgins, J. P. T. & Rothstein, H. R. (2009) Introduction to meta-
analysis. https://doi.org/10.1002/9780470743386.
Card, N. A. (2012) Applied meta-analysis for social science research (New York, Guilford Press).
ß o
C galtay, N. & Karadag, E. (2015) Introduction to meta-analysis. In E. Karada g (Ed.), Leadership
and Organizational Outcomes 19–28.
Cohen, J. (1988) Statistical power analysis for the behavioral sciences, 2nd ed. (USA, Lawrence Erl-
baum Associates).
Darabi, A., Arrington, T. L. & Sayilir, E. (2018) Learning from failure: A meta-analysis of the
empirical studies, Educational Technology Research and Development, 66(5), 1101–1118.
https://doi.org/10.1007/s11423-018-9579-9.
Davis, D., Chen, G., Hauff, C. & Houben, G.-J. (2018) Activating learning at scale: A review of
innovations in online learning strategies, Computers & Education, 125, 327–344. https://doi.
org/10.1016/J.COMPEDU.2018.05.019.
Iban
Di Serio, A., ~ ez, M. B. & Kloos, C. D. (2013) Impact of an augmented reality system on stu-
dents’ motivation for a visual art course, Computers & Education, 68, 586–596. https://doi.org/
10.1016/J.COMPEDU.2012.03.002.
Duval, S. & Tweedie, R. (2000) A nonparametric “Trim and Fill” method of accounting for publi-
cation bias in meta-analysis, Journal of the American Statistical Association, 95(449), 89–98.
https://doi.org/10.1080/01621459.2000.10473905.
Ellis, P. D. (2012) The essential guide to effect sizes. https://doi.org/10.1017/cbo9780511761676.
Fong, C. J., Davis, C. W., Kim, Y., Kim, Y. W., Marriott, L. & Kim, S. (2017) Psychosocial factors
and community college student success, Review of Educational Research, 87(2), 388–424.
https://doi.org/10.3102/0034654316653479.
Glover, K. (2014). Certifiable: Going rogue with non-library certifications. In M. K. Aho & E. Ben-
nett (Eds.), The machiavellian librarian: Winning allies, combating budget cuts, and influencing
stakeholders. https://doi.org/10.1533/9781780634364.2.133
G€oksu, I., €
_ Ozcan, oktasß, Y. (2014) Studies related to instructional design mod-
ß akır, R. & G€
K. V., C
els in Turkey, Elementary Education Online, 13(2), 694–709.
G€oksu, I., €
_ Ozcan, K. V., Cß akır, R. & G€oktasß, Y. (2017) Content analysis of research trends in
instructional design models: 1999–2014, Journal of Learning Design, 10(2), 85–109. https://doi.
org/10.5204/jld.v10i2.288.
Hao, K.-C. & Lee, L.-C. (2019) The development and evaluation of an educational game integrat-
ing augmented reality, ARCS model, and types of games for English experiment learning: an
analysis of learning, Interactive Learning Environments, 1–14, https://doi.org/10.1080/
10494820.2019.1619590.
Ho, L.-H., Sun, H., Tsai, T.-H., Ho, L.-H., Sun, H. & Tsai, T.-H. (2019) Research on 3D paint-
ing in virtual reality to improve students’ motivation of 3D animation learning, Sustainability,
11(6), 1605. https://doi.org/10.3390/su11061605.
Huett, J. B. (2006) The effects of ARCS-based confidence strategies on learner confidence and performance
in distance education (University of North Texas), Available online at: https://digital.library.
unt.edu/ark:/67531/metadc5268/m2/1/high_res_d/dissertation.pdf (accessed 15 August 2019).

(2019) Impact of implementing a long-term STEM-based active learning
Julia, C. & Antolı, J. O.
course on students’ motivation, International Journal of Technology and Design Education, 29(2),
303–327. https://doi.org/10.1007/s10798-018-9441-8.
Karakisß, H., Karamete, A. & Okcßu, A. (2016) The effects of a computer-assisted teaching material,
designed according to the ASSURE instructional design and the ARCS Model of motivation,
on students’ achievement levels in a mathematics lesson and their resulting attitudes, European
Journal of Contemporary Education, 15(1), 105–113. https://doi.org/10.13187/ejced.2016.15.
105.
Keller, J. M. (1987a) Development and use of the ARCS model of instructional design, Journal of
Instructional Development, 10(3), 2–10. https://doi.org/10.1007/BF02905780.
Keller, J. M. (1987b) Strategies for stimulating the motivation to learn, Performance and Instruction,
26(8), 1–7. https://doi.org/10.1002/pfi.4160260802.
Keller, J. M. (1987c) The systematic process of motivational design, Performance and Instruction, 26
(9–10), 1–8. https://doi.org/10.1002/pfi.4160260902.
Keller, J. M. (1999) Motivation and methods, New Directions for Teaching and Learning, 78, 37–47.
https://doi.org/10.1002/tl.7804.
Keller, J. M. (2010) Motivational design for learning and performance: The ARCS model approach
(New York, Springer).
Keller, J. M. (2016) Motivation, learning, and technology: Applying the ARCS-V motivation
model, Participatory Educational Research, 3(2), 1–15. http://dx.doi.org/10.17275/per.16.06.3.
2.
Khan, T., Johnston, K. & Ophoff, J. (2019) The impact of an augmented reality application on
learning motivation of students, Advances in Human-Computer Interaction, 2019, 1–14. https://d
oi.org/10.1155/2019/7208494.
Lam, R. W. & Kennedy, S. H. (2005) Using metaanalysis to evaluate evidence: Practical tips and
traps, Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie, 50(3), 167–174.
https://doi.org/10.1177/070674370505000306.
Li, K. & Keller, J. M. (2018) Use of the ARCS model in education: A literature review, Computers
& Education, 122, 54–62. https://doi.org/10.1016/J.COMPEDU.2018.03.019.
Maehr, M. L. & Archer, J. (1985) Motivation and school achievement (ERIC Publications) Available
online at: https://files.eric.ed.gov/fulltext/ED265938.pdf (accessed 15 August 2019).
Martens, R., Gulikers, J. & Bastiaens, T. (2004) The impact of intrinsic motivation on e-learning in
authentic computer tasks, Journal of Computer Assisted Learning, 20(5), 368–376. https://doi.
org/10.1111/j.1365-2729.2004.00096.x.
Pintrich, P. R. & De Groot, E. V. (2003) A motivational science perspective on the role of student
motivation in learning and teaching contexts, Journal of Educational Psychology, 95(4), 667–
686. https://doi.org/10.1037/0022-0663.95.4.667.
Piriyasurawong, P. (2019) Active learning using ARCS motivation on Social Cloud Model to
enhance communication skills in foreign language, TEM JOURNAL - Technology, Education,
Management, Informatics, 8(1), 290–297. https://doi.org/10.18421/TEM81-40.
Refat, N., Rahman, M. A., Asyhari, A. T., Kurniawan, I. F., Bhuiyan, M. Z. A. & Kassim, H.
(2019) Interactive learning experience-driven smart communications networks for cognitive
load management in grammar learning context, IEEE Access, 7, 64545–64557. https://doi.org/
10.1109/ACCESS.2019.2915174.
Reynolds, K. M., Roberts, L. M. & Hauck, J. (2017) Exploring motivation: integrating the ARCS
model with instruction, Reference Services Review, 45(2), 149–165. https://doi.org/10.1108/
RSR-10-2016-0057.
Rosenthal, R. (1979) The file drawer problem and tolerance for null results, Psychological Bulletin,
86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638.
Sarier, Y. (2016) The factors that affects students’ academic achievement in Turkey: A meta-analy-
sis study, H. U. Journal of Education, 31(3), 609–627. https://doi.org/10.16986/HUJE.
2016015868.
Sawilowsky, S. S. (2009) New effect size rules of thumb, Journal of Modern Applied Statistical Meth-
ods, 8(2), 597–599. https://doi.org/10.22237/jmasm/1257035100.
Slavin, R. & Smith, D. (2009) The relationship between sample sizes and effect sizes in systematic
reviews in education, Educational Evaluation and Policy Analysis, 31(4), 500–506. https://doi.
org/10.3102/0162373709352369.
Stockdale, J., Hughes, C., Stronge, S. & Birch, M. (2019) Motivating midwifery students to digi-
talise their enquiry-based learning experiences: An evaluative case study, Studies in Educational
Evaluation, 60, 59–65. https://doi.org/10.1016/j.stueduc.2018.11.006.
Suzuki, K., Nishibuchi, A., Yamamoto, M. & Keller, J. M. (2004) Development and evaluation of
website to check instructional design based on the ARCS Motivation Model, Information and
Systems in Education, 2(1), 63–69.
Tatsioni, A. & Ioannidis, J. P. A. (2017) Meta-analysis, International Encyclopedia of Public Health,
117–124, https://doi.org/10.1016/B978-0-12-803678-5.00291-5.
Woo, J.-C. (2014) Digital game-based learning supports student motivation, cognitive success, and
performance outcomes, Educational Technology & Society, 17(3), 291–307.
SUPPORTING INFORMATION
Additional Supporting Information may be found online in the supporting informa-

tion tab for this article:
Appendix A. Forest plot of effect sizes of academic achievement.

Appendix B. Forest plot of effect sizes of motivation and A-R-C-S dimensions.
Appendix C. Funnel plots of effect sizes of A-R-C-S dimensions.
Appendix D. References of primary studies included in this study.

Review of Education - 2020 - Goksu - Does The ARCS Motivational Model Affect Students Achievement and Motivation A

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Review of Education - 2020 - Goksu - Does The ARCS Motivational Model Affect Students Achievement and Motivation A

Uploaded by

Copyright:

Available Formats

Vol. 9, No. 1, February 2021, pp. 27–52

Does the ARCS motivational model affect

© 2020 British Educational Research Association

ARCS model of motivation

© 2020 British Educational Research Association

Figure 1. Components and strategies of the ARCS model (Keller, 1987a)

© 2020 British Educational Research Association

applied to educational environments in many countries. Goksu et al. (2014, 2017)

© 2020 British Educational Research Association

RQ 2. What is the overall effect of the ARCS model on students’ motivation?

Literature search procedures

© 2020 British Educational Research Association

Inclusion and exclusion criteria

Coding of study characteristics

© 2020 British Educational Research Association

© 2020 British Educational Research Association

Exp. Mean Exp. SD Exp. Cnt.Mean Cnt. SD Control

A1 AA (?), M ! ? G5 Social sciences C7 Printed material 33 33 0.716, 0.06235

© 2020 British Educational Research Association

© 2020 British Educational Research Association

has been included in this version].

© 2020 British Educational Research Association

Figure 2. Funnel plot of overall ES for achievement

Motivation: A total of 12 ES related to motivation and 54 ES (A: 13, R: 13, C: 15,

© 2020 British Educational Research Association

Figure 3. Funnel plot of overall ES for motivation

Overall effect of ARCS model on achievement

© 2020 British Educational Research Association

Learning environment n ES+ ES level SE p Lower CI Upper CI

Blended learning 2 1.57 Very large 0.67 0.02 0.25 2.89

QB = 7.93, p = 0.44, df(Q) = 8

© 2020 British Educational Research Association

a significant difference according to the integrated learning environment. The find-

Field/Course n ES+ ES level SE p Lower CI Upper CI

Architecture 6 1.31 Very large 0.09 0.00 1.13 1.49

QB = 70.53, p = 0.00, df(Q) = 6

© 2020 British Educational Research Association

Level of education* n ES+ ES level SE p Lower CI Upper CI

Undergraduate 27 0.76 Medium 0.12 0.00 0.53 0.99

QB = 0.13, p = 0.99, df(Q) = 3

Meta-regression analysis was performed to reveal the relationship between sample

© 2020 British Educational Research Association

Overall effect of ARCS model on motivation

Field/Course* n ES+ ES level SE p Lower CI Upper CI

Computer technologies 3 0.49 Small 0.25 0.05 0.01 0.98

QB = 0.06, p = 0.97, df(Q) = 2

© 2020 British Educational Research Association

Level of education* n ES+ ES level SE p Lower CI Upper CI

Undergraduate 9 0.48 Small 0.08 0.00 0.32 0.63

QB = 5.90, p = 0.02, df(Q) = 1

© 2020 British Educational Research Association

Components n ES+ ES level SE p Lower CI Upper CI

Attention 13 1.20 Very large 0.25 0.00 0.70 1.70

QB = 3.21, p = 0.36, df(Q) = 3

© 2020 British Educational Research Association

Table 8. Calculated effect sizes and p values

ARCS Overall ES 0.74 0.43 1.20 0.72 0.78 0.70

Discussion and conclusion

© 2020 British Educational Research Association

© 2020 British Educational Research Association

© 2020 British Educational Research Association

of the ARCS model on academic achievement differs significantly according to disci-