International Journal of Innovation Studies 4 (2020) 134e147

Contents lists available at ScienceDirect

International Journal of Innovation Studies

journal homepage: http://www.keaipublishing.com/en/journals/in-
ternational-journal-of-innovation-studies

Artificial intelligence innovation in education: A twenty-year

data-driven historical analysis
Chong Guan a, Jian Mou b, *, Zhiying Jiang a
a
School of Business, Singapore University of Social Sciences (SUSS), Singapore
b
School of Business, Pusan National University, Busan, 46241, South Korea

a r t i c l e i n f o a b s t r a c t

Article history: Reflecting on twenty years of educational research, we retrieved over 400 research article
Available online 29 September 2020 on the application of artificial intelligence (AI) and deep learning (DL) techniques in
teaching and learning. A computerised content analysis was conducted to examine how AI
Keywords: and DL research themes have evolved in major educational journals. By doing so, we seek
Artificial intelligence to uncover the prominent keywords associated with AI-enabled pedagogical adaptation
Systematic review
research in each decade, due to the discipline’s dynamism. By examining the major
Intelligent tutoring systems
research themes and historical trends from 2000 to 2019, we demonstrate that, as
Virtual reality
Educational data mining
advanced technologies in education evolve over time, some areas of research topics seem
have stood the test of time, while some others have experienced peaks and valleys. More
importantly, our analysis highlights the paradigm shifts and emergent trends that are
gaining prominence in the field of educational research. For instance, the results suggest
the decline in conventional tech-enabled instructional design research and the flourishing
of student profiling models and learning analytics. Furthermore, this paper serves to raise
awareness on the opportunities and challenges behind AI and DL for pedagogical adap-
tation and initiate a dialogue.
© 2020 China Science Publishing & Media Ltd. Publishing Services by Elsevier B.V. on
behalf of KeAi. This is an open access article under the CC BY-NC-ND license (http://
creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Artificial Intelligence (AI), a jargon often found in science fiction, has been widely accepted as it increasingly becomes part
of our daily lives. It rapidly transforms industries such as healthcare, transportation, retail and finance (Russell and Norvig,
2018). Education is another area that offers incredible potential for the application of AI technology. In fact, AI innovation
in education has evolved from idealised laboratory scenarios to real-life learning contexts with more complexity. Companies
in the educational technology (EdTech) industry have developed Individual Adaptive Learning System that allows personalized
learning, Aided Teaching System that assists classroom environment management, grading, evaluation and second-language
problem; and Institute Administration System that helps with student enrolment and enquiries, etc. From 2008 to 2017, the
global venture capital investment that flowed into AI-based education reached $1047 billion (Mou, 2019). With companies
proceeding apace in AI-based innovation, issues arise such as how to integrate AI-based education systems with macro
education institutions and micro education contexts, roles played by the stakeholders in the AI-based education ecosystem,

* Corresponding author. School of Business, Pusan National University, 2, Busandaehak-ro 63beon-gil, Geumjeong-gu, Busan, 46241, South Korea.
E-mail address: jian.mou@outlook.com (J. Mou).

https://doi.org/10.1016/j.ijis.2020.09.001
2096-2487/© 2020 China Science Publishing & Media Ltd. Publishing Services by Elsevier B.V. on behalf of KeAi. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147 135

and development of business versus consumer facing AI-based technology platforms. Have these issues been addressed in AI
in education (AIEd) literature? With the changing role of AI technologies, how will the educational paradigms evolve? Where
is AIEd research heading?
Though AI-based education has burgeoned rapidly in the last 10 years, research on AIEd emerged as early as in the se-
venties. From 1964 to 1966, Eliza, an early natural language processing (NLP) programme was built by Joseph Weizenbaum
(1966) in MIT. Eliza was then incorporated by various programmes as the machine and human interface. Then Jaime Carbonell
coded a student facing instructional programme called SCHOLAR, which posed or answered questions regarding South
American geography and gave instant feedback about the quality of a learner’s responses in natural language (Carbonell,
1970). This system was later referred to as an Intelligent Tutoring System (ITS) (Sleeman and Brown, 1979). Another
important prototype that sets the foundation for ITS development is MYCIN (Shortliffe et al., 1975), an expertise system that
helps physicians with diagnosis and therapies of bacteria infected patients. This seminal work has embedded tutoring ap-
proaches with domain expertise so that knowledge becomes accessible to learners. The AIEd literature then rolled out from
the ITS strand and evolved into a broad field with different themes and paradigms. The scale and scope of AIEd research has
escalated quickly with the advancement of AI technology in education.
Thematic reviews (e.g. Roll and Wylie, 2016; Hinojo-Lucena et al., 2019; Chan and Zary, 2019) on AIEd have been published
with different goals and focuses. These reviews help us identify and understand the sources, foci and context of the vast
literature. As comprehensive as these reviews attempt to be, they are either single-journal bibliometric analysis or cross-
journal analysis with a shorter period. The limitation with single-journal bibliometric analysis is its representativeness of
AIEd as a research field. The issue with a review based on a shorter period is that it only reflects the latest trends in AIEd. We
therefore believe a temporal multiple-journal bibliometric analysis is needed to piece together the evolution of AIEd research
in the past two decades. By doing so, we are able to address the following research questions:

What have been the major paradigms in the history of AIEd literature?

What are the lifecycles of different research paradigms? How has AIEd research shifted from one to another?

What are the emerging themes that have started to gain prominence in the AIEd research community?

By depicting the evolution of AIEd research as a field, it helps us understand the nature and position of this on-going
research stream. It also serves as a spur to initiate dialogue regarding the up-and-coming AIEd research themes in the
next decade.
To achieve our research goal, an expansive computer-assisted content analysis was implemented on the studies published
in reputable journals. The titles and abstracts of all selected AIEd journal articles (n ¼ 425) published between 2000 and 2019
were collected from the publishers’ website and Google Scholar. This text data was then fed on the Leximancer for in-depth
text analysis. In addition to automatic computer-assisted content analysis, we have supplemented the study by undertaking
manual analysis on the representativeness of topics illustrated in each concept map.
With the exploration and analysis based on such a large pool, our research uncovered the major research themes and the
paradigm shifts. With the prevalence of distant education, 2000e2009 saw an increase of research outputs in implementation
and design of online education. Virtual Reality (VR) emerged as a sub-theme under this strand of research to bring vivid
experiences to the classrooms. Then when big data came on the scene, student profiling models and learning analytics
became a dominant focus in 2010e2019. With publication data from multiple journals in the past twenty years, our research
engineered a temporal map of paradigm shifts in AIEd research. The underpinnings of these shifts were encouraged by the
advancement of AI technology.

2. Literature review

2.1. Conceptualization of artificial intelligence in education (AIEd)

The term AI, coined by John McCarthy in 1955, is defined as a computer with the capability to perform a variety of human
cognitive tasks, such as communicating, reasoning, learning, and/or problem-solving (Nilsson, 1998). Baker and Smith (2019)
further explain that AI represents a generic term to describe a wide collection of different technologies and algorithms (e.g.,
machine learning, NLP, data mining, and neural networks). In the earlier stage, AI in education area normally refers to
intelligent tutoring systems, which aim to solve problems such as enhancing operator performance automatically (e.g., Ross,
1987; Hwang, 2003). Currently, AI refers to using big data to perform complex tasks. The definitions of AI in education are
reported in Table 1.
AI applications have been used to advance numerous industries in today’s society (Russell and Norvig, 2018). In this
research, we confine the concept of Artificial Intelligence to the educational context. We use Baker and Smith (2019)’s
categorization of AIEd applications that are available today: a) learner-oriented AIEd; b) instructor-oriented AIEd; and c)
institutional system-oriented AIEd. A learner-oriented AI enables students to study a subject domain, i.e. an adaptive or
personalized learning management system. An instructor-oriented tool can automate tasks such as administrative proced-
ures, assessments, plagiarism detection and provision of feedback. These systems can also help teachers monitor students’
learning progress so that intervention could be provided proactively. With advances such as individualized ITS (Steenbergen-
136 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147

Table 1
The definition of AI in education.

No. Authors Definition

1 Ross (1987)
2 Hwang (2003)
3 Johnson et al. (2009)
4 Popenici and Kerr (2017)
5 Chatterjee and
Bhattacharjee (2020)
AI techniques can permit the intelligent tutoring systems itself to solve the problems which it sets for the user, in a
human-like and appropriate way, and then reason about the solution process and make comments on it.
Summarized AI in education context as intelligent tutoring system that helps to organize system knowledge and
operational information to enhance operator performance and automatically determining exercise progression and
remediation during a training session according to past student performance.
The authors summarized AI as artificially intelligent tutors that construct responses in real-time using its own ability to
understand the problem and assess student analyses.
AI is defined as computing systems that are able to engage in human-like processes such as learning, adapting,
synthesizing, self-correction and use of data for complex processing tasks.
AI is defined as computing systems capable of engaging in human-like processes such as adapting, learning,
synthesizing, correcting and using of various data required for processing complex tasks.

Hu and Cooper, 2014), support platforms (Heffernan and Heffernan, 2014), and “teacher bots” (Bayne, 2015), AIEd has
demonstrated its potential to help learners identify knowledge gaps and receive specialized support, thereby freeing in-
structors from daily menial tasks and allowing them to respond to students more effectively. Lastly, at an institutional level,
AIEd tools can provide useful insights to administrators and decision makers, like enrolment and attrition patterns across
disciplines or colleges. The application of AIEd remains a topic of keen interest among researchers.

2.2. Conceptualization of deep learning

Deep learning (DL) is a subset of AI but the core of AI. DL allows computers to improve using experience and data
(Goodfellow et al., 2016). It achieves great power and flexibility by representing the world as a nested architecture of concepts
that differ in abstract level. Some common examples of deep learning include NLP-based Google translate, instantaneous and
personalized recommendation systems on Amazon, and algorithmic trading in financial markets, etc.
In this research, again we focus on DL applications in the context of education. As a subset of AI, Baker and Smith (2019)’s
categorization of AIEd applications still apply. Specifically, the following areas see most applications of DL technologies in
education: 1) Adaptive assessment and grading, 2) performance predication, and 3) student retention. Pattern recognition
allows DL application to grade text assignments and spot students at risk of course failure (Abbott, 2006; Wang et al., 2008;
Wu et al., 2018). DL-based assessments allow constant feedback to both students and teachers and suggest adaptive paths to
achieve the learning goals (Vaculík et al., 2013). DL is also able to predict student performance and identify their strengths and
weaknesses and suggests ways to improve with tests or practices (Anand et al., 2018; Alam et al., 2018). Last but not least,
student retention is essential for school enrolment management and it affects key metrics such as reputation, ranking and
financial performance. Studies that use DL to detect students at risk for retention management include Delen (2010) and
Lykourentzou (2009).

2.3. Single-journal bibliometric analysis and analyses across journals

We first review previous work that coalesces past AIEd literature and synthesizes significant research streams. Roll and
Wylie (2016) investigated 47 articles published in the year 1994, 2004, and 2014 in International Journal of Artificial Intel-
ligence in Education (IJAIED). Their analysis identified the foci and common scenarios in AIEd research and called for two
parallel strands of AIEd research in the coming decades: a) an evolutionary process, revolving around collaborating with
teachers, advancing current instructional design and classroom practices, and diversifying application technologies and
disciplines; and b) a revolutionary process, focusing on embedding technologies within each learner’ daily life, supporting
his/her personal goals, culture development and community empowerment. Three papers in the past 5 years have studied the
scientific production on AIEd across multiple journals. Hinojo-Lucena et al. (2019) examined the published work (132 papers)
on AI in higher education indexed by Web of Science (WoS) and Scopus databases between 2007 and 2017. From the results
obtained, they concluded that, although AI has been receiving worldwide interest, research on its application in higher
education is at an incipient stage. In the same year, Bond et al. (2019) explored 146 articles in EdTech journals published
between 2007 and 2018. They identified four pillars of AIEd applications in academic, administrative and institutional support
services: profiling and forecasting, assessment and evaluation, adaptive systems and customization, and ITSs. Their synthesis
of results highlighted that there had been insufficient critical consideration of risks and challenges in AIEd research. In
addition, they emphasized the gap for future research on ethical educational approaches and connection to theoretical
perspectives on AI in higher education. Chan and Zary (2019) also explored the research trends in AIEd, with a special focus on
medical education. A total of 37 articles published between 1980s till date were examined. They reported three fundamental
applications: a) learning support, b) assessment of learning progress, and c) curriculum review. The results revealed the
importance of addressing the technical difficulties and assessing the effectiveness of AI, in order to accelerate adoption. In
sum, the main advantage of AIEd application is the capability of providing an adaptively guided learning pathway and
individualized feedback at a reduced cost.
 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147 137

Understanding the historical and emerging trends of AIEd research has been challenging due to its “meta” nature and the
large number of potential publication outlets. We therefore propose a multiple-journal analysis covering the entire history of
AIEd research over the past twenty yearsea much longer period than previous cross-journal bibliometric studies.

3. Method

In the current study, we sought to examine the past research themes and trends in the field of AI and DL applications in
education via a comprehensive analysis of the literature published in reputable journals over the last two decades.
We utilize an automated method (web-scrapping) to retrieve and compile journal articles on major publishers’ databases,
including Association for Computing Machinery (ACM), EBSCO, Emerald, IEEE, JSTOR, ScienceDirect, Taylor & Francis, and
Wiley.
The following criteria were used to define the studies to be included in the review:

1. A focus on AI and/or DL application.

2. Deployment of AI/DL in education context.
3. Being published in an influential academic journal.

We operationalised AIEd application as an umbrella concept to describe systems/machines that mimic human responses
to stimulation. In the selection of the journal articles to be included in the review, we included of broad variety of intelligent
learning solutions and the implementation predictive analytics of student performance that liberates teachers, learners and
education administrators from spatial or temporal constraints (either partially or completely) (e.g. ITS, intelligent computer-
aided instruction, artificial neural network-enabled assessments, machine learning techniques for student performance
prediction etc.). Based on the above criteria, we searched in the above databases for all journal articles with titles or abstracts
that included the following keywords: AI AND (learning OR education OR school OR instruction), DL AND (learning OR ed-
ucation OR school OR instruction), Neural Network AND (learning OR education OR school OR instruction), VR AND (learning
OR education OR school OR instruction), AR AND (learning OR education OR school OR instruction). These keywords were
carefully chosen to represent the wide range of implementation of technology-rich educational innovations.
Further reading by an expert panel was conducted to confirm the criteria 1e3 for all articles correspond to criteria 1e2 via
keyword search.
The journals were chosen to reflect the different areas that we considered to be potential outlets for research on the
implementation of AI and DL on learning in the academia. British Journal of Educational Technology (BJET), Education and
Information Technologies, Journal of Educational Technology & Society (JETS), Educational Technology Research and
Development and Programmed Learning & Educational Technology (PLET) were selected to represent research on AIEd.
Computers & Education (CAE), IEEE Transactions on Education and Interactive Learning Environments were chosen as these
periodicals focus on ITS, augmented reality (AR) and virtual reality (VR) applications in learning. As there are many more
journals that publish research on discipline specific AI-enabled learning, we included other possible outlets such as Computer

Table 2
Number of included articles (n ¼ 425) by journal.

Journal Number of included articles

Computers & Education 232
IEEE Transactions on Education 19
British Journal of Educational Technology 19
Computer Applications in Engineering Education 18
Interactive Learning Environments 15
Journal of Computer Assisted Learning 15
International Journal of Artificial Intelligence in Education 14
Journal of Educational Technology & Society 9
Education and Information Technologies 7
eLearning & Software for Education 7
ACM Transactions on Computing Education 3
Behaviour & Information Technology 3
Decision Support Systems 3
Educational Technology & Society 3
International Journal of Emerging Technologies in Learning 3
Education Sciences 2
Educational Media International 2
Educational Technology Research and Development 2
IEEE Transactions on Learning Technologies 2
Innovation in Teaching and Learning in Information and Computer Sciences 2
Journal of Engineering Education 2
others with one article 43
138 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147

Applications in Engineering Education and Computer Science Education. All selected periodicals are considered highly
reputable within their respective research communities. Notwithstanding, they each have a distinct target audience profile.
Table 2 presents the number of selected articles from each of these journals. It was consistent with our expectation that the
selected scope would be more extensively covered in the more recent publications due to the exponential growth of edu-
cation technologies. As a result, our research focus on the AIEd research from 2000 to 2019. The field of research on AI has
blossomed in this time (Wang and Siau, 2019).

3.1. Computer-assisted content analysis

To answer the research question (How have the publications on AIEd evolved over time?), the titles and abstracts of
selected articles were qualitatively analysed by Leximancer. Unlike other computerised content analysis software (e.g., LIWC,
NVivo, and ATLAS.ti), Leximancer does not administer word frequency or encode vocabularies and phrases. Its algorithms
analyse the meanings within paragraphs of texts by identifying and extracting their key concepts and schemes. By tokeni-
sation of all nouns, verbs, adjectives and adverbs, and removing pronouns (e.g. I, you, she), determiners (e.g. the, a) and
prepositions (e.g., in, on), it identifies words with similar semantics together and visually summarises and displays the
extracted information and relationships. By applying a quantitative approach to qualitative analysis (Indulska et al., 2011),
Leximancer has been deployed to examine human language and psychology (Smith and Humphreys, 2006), online com-
munications in tourism (Tseng et al., 2015), and conceptual analysis in literature review studies (Cretchley et al., 2010; Dann,
2010; Indulska et al., 2011).
In this study, we used Leximancer to analyse twenty-years of unstructured textual data collected from published articles,
to reveal core research topic steams across academic publications. Computer-aided content analysis has been found to be a
suitable approach for mapping a research area, whilst cutting down time and cost, and limiting bias from human interpre-
tation (Bond et al., 2019; Indulska et al., 2011).
Titles and abstracts are “lexically dense” and typically focus on the core issues raised in articles (Cretchley et al., 2010). Past
content analyses have also employed titles and abstracts to examine topics and research trends over time. These studies
include the analysis of Distance Education from 1980 to 2014 (Zawacki-Richter and Naidu, 2016), the analysis of the
instructional design and technology (IDT) scholarship between 2007 and 2017 (Bodily et al., 2019), and the analysis of five
decades of Edtech research in BJET (Bond et al., 2019). Thus, such data represent the most appropriate resources for devel-
oping an understanding of a particular research field (see Fig. 1).
To analyse how the key research topics have evolved, converged or diverged over time, the titles and abstracts of all
selected AIEd journal articles published between 2000 and 2019 were collected from publishers’ websites and Google Scholar.
The text data were then collated in a.csv file and imported into Leximancer for analysis. The results of the qualitative content
analysis of the maps were then compared to the most cited articles published during each decade (see Appendix A).
In addition to computer-assisted text analysis, we also conducted manual content analysis (Weismayer and Pezenka, 2017)
to supplement the results with qualitative illustrations of emerging topics and “concept paths” within the concept maps (see
Figs. 2 and 3). Coding categories were developed based on the prior literature and the data pattern. First, some initial coding
categories were generated from the previous literature. These categories include various applications of AIEd, such as AI
computer-assisted instruction (AI CAI) system (Kearsley, 1978; Gallagher, 1981), assessment design (Bejar, 1984), and ITS
(Sleeman and Brown, 1979). Secondly, additional categories were generated based on the data patterns. These categories were
generated following the procedures suggested by Bogdan and Biklin (1998). Initial coding was conducted by generating
numerous category codes as reading the responses, labeling data that was related without worrying about the variety of
categories. Then, focused coding was used to eliminate and combine repeating ideas. These additional categories derived
from the data include adaptive learning, VR, AR, learning analytics and so on. An expert coder linked all specific paper to
identified categories.

Fig. 1. Human-Technology-Outcome framework.

 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147 139

Fig. 2. Concept map for the time period between 2000 e 2009 (n ¼ 114 articles).

4. Results and discussion

4.1. Overall scope of AI and DL in educational research and practice

Based on manual coding, eleven research themes have emerged in the field of AIEd research across the past twenty-years
(2000e2019). These themes reflect the diverse applications of AIEd, including AI computer-assisted instruction (AI CAI)
system, VR, ITS, AR, educational games, predictive modelling, adaptive learning, assessment design, educational agents and
teaching elevation. Table 3 illustrates the major research streams covered in the published articles (n ¼ 425).
The manual coding results are then compared with top concepts generated via automatic content analysis, where the
emergence of themes in conceptual analysis is automatic, depending upon presence and frequency of concepts. Table 4
presents a list of representative text excerpts for each concept, which illustrates how the concepts are related to AIEd
140 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147

Table 3
Qualitative results on research themes based on manual coding.

Research themes and description Frequency Percentage

AI computer-assisted instruction (AI CAI): an interactive instructional technique, offline/online, involving a variety of 97 22.8%
programmed instructional materials, such as drill-and-practice, tutorial, or simulation activities (Tan et al., 2013)
Virtual reality (VR) in education: a simulated experience that enhances learning and engagement by allowing user to view and 91 21.4%
interact with virtual features or items (Monahan et al., 2008)
Intelligent tutoring system (ITS): a computer system that aims to provide immediate and customized instruction or feedback to 77 18.1%
learners, usually without requiring intervention from a human teacher (Chi and VanLehn, 2010; Steenbergen-Hu and Cooper,
2014)
Augmented reality (AR) in education: bringing an interactive experience of a real-world environment into the classroom, 48 11.3%
where the objects that reside in the real world are enhanced by computer-generated perceptual and sensory information (Lai
et al., 2019)
Educational games: a game designed for a primary purpose other than pure entertainment, rather learning or practicing a skill 30 7.1%
(Annetta et al., 2009)
Predictive modelling in education: implementing predictive analytics of student performance (Lykourentzou et al., 2009), 22 5.2%
satisfaction (Hew et al., 2020), mood (Moridis and Economides, 2009) or course selection (Kardan et al., 2013)
Adaptive learning/adaptive teaching: an educational method which uses computer algorithms to orchestrate the interaction 16 3.8%
with the learner and deliver customized resources and learning activities to address the unique needs of each learner
(Normadhi et al., 2019)
Assessment design: the formulation of assessment instruments using machine learning, neural network, automatic scoring or 16 3.8%
other AI techniques that could provide more conducive and diagnostic outcomes than what conventional tests were capable of
offering (El-Alfy and Abdel-Aal, 2008; Sung et al., 2016)
Learning analytics: Using sophisticated machine learning (ML) algorithms and rich data about learners and their contexts, 14 3.3%
facilitates inference-making about several behavioral aspects (including effortful behavior) for purposes of understanding and
optimising learning and the environments in which it occurs (Sharma and Papamitsiou, 2019)
Educational agents: a learning companion system that assumes two roles, one as an intelligent tutor and another as a learning 12 2.8%
companion (Chou at al. 2003)
Teaching evaluation: a teaching quality evaluation model build based on advanced techniques (Wang et al., 2017) 2 0.5%

research in the article. The overall results indicate that, research efforts in this area have predominantly focused on the
learning practices, instead of concentrating specifically on the technology.
As shown in Table 4, the top concepts are related to AIEd in four ways: a) application context, such as pedagogical
deployment of AIEd (e.g. intelligent tutoring systems, expert systems); b) targeted outcomes (e.g. predicting student per-
formance; identification of learning styles); c) technologies being deployed (e.g. VR, mobile educational features); and d)
learning environment (project-based learning environment).
The overall review shows that the published research focuses on how technologies provide effective teaching and learning
environments. In the early 2000s, AI systems began to be widely implemented for ITS (Steenbergen-Hu and Cooper, 2014).
The analysis reveals an emphasis on the importance of such systems in these papers, which appear as top concepts in both
decades. One of the most challenging issues with regards to education is that each student may have a different learning style
and pace. Research on adaptive learning and assessments seems to focus on addressing this issue by developing customised
profiles and training materials for each learner based on his/her aptitude, ability, experience and preferred mode of learning.
During the past two decades, there has been clear focus of research topics on technologies for learning analytics and pre-
dictive modelling; with greater emphasis on detection of specific learner problems with early intervention and personalized
instruction.
As will be presented in the next section, prominent research thrusts that have emerged throughout AIEd research history
comprise of the evolution of technology for instructional design, the integration of new technologies under a variety of
teaching and learning contexts; and issues with implementing new systems and platforms, including appropriate techno-
logical and pedagogical adjustments.
Furthermore, to build a framework in the education area, we considered the technology-organization-environment (TOE)
framework. It is originally developed by Tornatzky and Fleischer (1990) to help understand innovation adoptions at the firm
level. This framework addresses three elements, namely the technological context, the organizational context and the
environmental context. These elements interact with each other and finally lead to technological innovation (Baker, 2011).
While, in our study, we argue that the interactions among the elements would lead to positive outcomes such as enhanced
teaching, tutoring, learning and education. In the TOE framework, the technology refers to IT infrastructure such as AI ed-
ucation systems (Pan and Jang, 2008). The environmental context includes policies and government regulations (Baker, 2011).
Lastly, the organizational context includes human resources (Yeh et al., 2015). Therefore, according to TOE framework along
with the top ranked concepts, we built a Human-Technology-Outcome framework.
In the framework, human refers to the participants in the education environment such as students. Technology refers to
the technological innovations to help perform the AI based education such as virtual reality and other systems. Outcome
refers to what we can do and what kind of performance we can gain through AI based education systems. The whole
framework is interactive in the education environment.
 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147 141

Table 4
Top ten ranked concepts and the exemplary text excerpts of each concept by decade.

2000e2009 2010e2019

Top Concepts Exemplary text excerpts Top Concepts Exemplary text excerpts
Student/students Predicting student performance (Van Heerden Learning Measuring student learning outcomes (Galbraith et al.,
et al., 2008) 2012)
Learning Identification of learning styles online (Lo and Student/students Student experience and educational effectiveness (Ib
~ ez
an
Shu, 2005) et al., 2014)
System/systems Intelligent tutoring system (Ford, 2008) System/systems Intelligent tutoring systems (Chi and VanLehn, 2010)
Environment/ Interactive learning environment (Yang et al., Reality An augmented reality-based learning approach (Lai et al.,
environments 2010) 2019)
Education/ Mobile educational features (Virvou and Alepis, Virtual Using virtual reality in the classroom (Cooper et al., 2019)
educational 2005)
Virtual Virtual reality technology (Yang et al., 2010) Education New technology trends in education (Martin et al., 2011)
Model Learner-initiating instruction model (Wong Environment/ Project-based learning environment (Larraza-Mendiluze
et al.,2007) environments et al., 2016)
Tutoring Personalized tutoring (Virvou and Alepis, 2005) Teaching Teaching practices and student learning (Viegas et al.,
2018)
Design Interface design using neural networks (Curilem Knowledge Learners’ collaborative knowledge construction
et al., 2007) performances (Lin et al., 2013)
Reality Virtual reality for collaborative e-learning Design The design of cognitive tasks (Nadolny, 2017)
(Monahan et al., 2008)

We found that the learning outcomes are important in studying AI-based education during last few decades. While, the
research published in 2000e2009 focuses more on developing the whole AI education environment. In contrast, the year
between 2010 and 2019 are focusing more on the learning outcomes.

4.2. The instructional design and implementation of online learning (2000e2009)

Notably, 2000 to 2009 saw an increased interest in learner-oriented approaches, compared to previous decades. The
thematic summary shows that student/students has become the focus, with the most direct mentions at 187 hits (100% relative
count), followed closely by learning (97%), system/systems (80%), environment/environments (48%), education/educational
(44%), virtual (42%), model (28%) and tutoring (27%). The top cited publications in this decade also reflect the emerging trends
that surface from the concept map (see Fig. 2), with strong emphasis on VR and a more proactive student-centered approach.

4.2.1. Intelligent tutoring systems (ITS)

ITSs have come into focus since the early 2000s, as reflected in some of the highest cited sources in this decade (see
Appendix A). An ITS refers to a computer program that provides immediate and tailored instruction or feedback to students
(Steenbergen-Hu and Cooper, 2014), generally without the direct intervention from a human instructor. Research on ITS
focused on two fundamental related pedagogical issues: a) to provide sophisticated instructional guidance on a one-on-one
basis that was superior to what had been achieved with a conventional computer-aided instructional system; and b) to
advance existing models on the intellective processes associated with teaching and learning. The cognitive capability of an ITS
was derived from its implementation of AI techniques. These techniques were deployed under four interacting components:
a) The knowledge base, representing the central part in an instructional process; b) the student model, referring to a learner’s
existing knowledge state; c) the pedagogical module, representing the most appropriate instructional approaches which were
contingent on the assessment of a student model; and d) the user interface, facilitating an effective conversation between an
ITS and a student (Sedlmeier, 2001). Special attention was given to the functionality assessment of these components, such as
their granularity, generality, interoperability, and reliability. Based on their functionalities, ITSs could be classified as: course
content curation and delivery ITSs (e.g., Curilem et al., 2007); and ITSs that monitor a student’s performance and offer
automatic diagnostic feedback (e.g., Hwang, 2003). Whilst there was interest in this domain, many issues remained unad-
dressed, such as the role of theoretical frameworks in the design, implementation and validation of an ITS.

4.2.2. Virtual reality (VR)

VR is a computer application which provides users with immersive experience via three-dimensional (3-D) visual and
auditory simulations (Antonietti and Cantoia, 2000). VR is typically characterized for its involvement in exploring the sur-
roundings and interaction with virtual objects. Over the last 20 years, EdTech researchers and practitioners have been
building VR applications to bring vivid experiences into the classroom, for the purpose of increasing student engagement and
enhancing learning (see virtual e technology e environment). The implementation of learning activities enabled by virtual
environments like Second Life (Bell, 2009; Edirisingha et al., 2009; Salmon, 2009), Active Worlds (Dickey, 2005) and
massively multiplayer online role-playing games (MMORPGs) such as World of Warcraft (Anderson, 2019) have been eval-
uated by researchers in the field. Notwithstanding, Warburton (2009) highlighted that an in-depth understanding of digital
literacies and virtual identities should be required, before 3-D virtual representations could be best deployed to bridge the
142 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147

gaps between educators and learners. Though many researchers reported a direct positive correlation between VR adoption
and improved user engagement, a literature review by Barbou and Reeves (2009) argued that only the intrinsically motivated,
self-directed, and digitally proficient learners could benefit the most from VR.

4.2.3. A paradigm shift from teacher-centered to learner-centered approach

Instructional design of online environments has been incorporating more student-led and competency-based pedagogical
activities (Carr-Chellman and Duchastel, 2000). Learner-oriented instructional models have been established (e.g., Alonso
et al., 2005) to provide further support to educators and system architects in constructing a more flexible and self-
directed learning space (see students e support e system e tools), informed by and applying theoretical frameworks (e.g.,
Cheng and Yeh, 2009).

4.3. Emergence of student profiling models and learning analytics (2010e2018)

In Fig. 3, the concept map describes the primary research thrusts and concepts of articles published between 2010 and
2019 (n ¼ 311). It indicates an increased emphasis on the learning processes of students, with the help of AI applications. Both
learning (100% relative count) and student/students (77%) surface considerably more during this decade compared to the
preceding decades, with 752 and 578 hits, respectively. System/systems is the next strongest research stream with 38% relative
count, followed by reality (27%), virtual (24%) and education (18%). Overall, this concept map indicates a continuous focus on
VR research, including how to improve student experience by harnessing VR in education contexts. Other studies covered how
various AI tools could increase students’ engagement, as identified in some of the highest cited articles (see Appendix A). The
emergence of a new theme data indicates a more evidence-based approach to AIEd practices. The concept computer is again
missing from the concept map, leaving concepts tutoring, intelligence, and skills.

Fig. 3. Concept map for the time period between 2010 e 2019 (n ¼ 311 articles).
 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147 143

4.3.1. Student profiling and academic achievement prediction

The basis for a number of AIEd predictive applications are learner models that enable active tracking of student knowledge,
engagement, academic performance, and the likelihood of someone dropping out of a course. The aim is to detect students at
risk and to provide timely administrative support and/or subject matter guidance throughout their academic progression. In
this respect, classification algorithms under predictive modelling have been an essential element of educational data mining
(Krishna et al., 2018). Many studies concerned with profiling addressed issues related to students’ academic achievements at
the course level (e.g., Dahman and Dag . 2019). Other drop-out and retention studies sought to develop early alert systems to
identify at-risk students during their first academic year (e.g., Hoffait and Schyns, 2017), with an intention to provide
appropriate remediation and reorientation. Most studies examined the models’ overall accuracy, sensitivity and precision
(e.g. Lykourentzou et al., 2009). For instance, Delen (2010) used institutional data e students’ demographic variables, aca-
demic performance and financial characteristics and three classification techniques e artificial neural networks, decision
trees and logistic regression to predict drop-out rates. His study concluded that the most critical predictors with regards to
student drop-out rates were their past and present academic achievements, and whether they receive financial aid on tuition
and other school expenses. Overall, these studies showed that AI-enabled student profiling and modelling solutions could
serve to relieve academic and administrative staff from manual work, allowing them to focus on the more challenging cases.

4.3.2. Evolution of learning analytics

Real-time data analytics has emerged as one of the most promising AIEd research trends in this decade (see lear-
ningeusedebasedeanalysis). Learning analytics could expand the understanding of the teaching-learning processes (e.g.,
Rodríguez-Triana et al., 2015) and help optimize learning practices. At a macro level, studies have examined how the insights
derived from big data analytics on pedagogical patterns could help inform instructional design (e.g., Rienties and Toetenel,
2016) (see tutoringesystemestudentedata). Studies on the influence of AIEd on pedagogical contexts and assessment
regime (Bennett et al., 2017) (see analysiseinformationeevaluation) have also emerged in the past decade (Ellis, 2013). For
instance, researchers found that learner-centered assessments could encourage deeper student reflections (e.g., Nix and
Wyllie, 2011; Whitworth and Wright, 2015). The breadth of these articles also suggests an area of increasing interest and a
paradigm shift to evidence-based pedagogical approaches (e.g., McKenney and Mor, 2015).
At a micro level, learning analytics allow students to assess their own individual competences and thus receive support
directly (e.g., Daley et al., 2016; Gonza
lez-Marcos et al., 2016). Later studies have looked into other innovative ways of
deploying learning analytics. The study by Spikol et al. (2018) adopted an automatic face and hand gesture tracking system in
workshops to predict students’ results in project-based learning. Lastly, Pardo and Siemens (2014) highlighted the ongoing
debate surrounding ethical considerations and personal information privacy in the use of student data for institutional
analytics.
The top cited papers across in each decade listed in Appendix A confirm our findings on the paradigm shifts presented in
the conceptual maps. The papers with the most citations typically were published about an up-and-coming technology early
in its adoption, which led to a large number of citations as the technology gained greater popularity. During the past two
decades, VR was the most frequently cited technology (e.g. Merchant et al., 2014). Most top cited articles towards the last two
decades took on a “technology centric” approach, with greater emphasis on using technologies for solving specific peda-
gogical problems such as educational agents (Chou et al., 2003) and personalized learning (Chen, 2008).

5. Conclusion and research limitations

This research surveyed over 400 journal articles in the domain of AI and DL in education published in reputable journals
from 2000 to 2019. With the aim to uncover the history of research evolution in the field of AIEd, four main research paradigm
shifts have been identified in accordance with the changing role of AI technology in education. With the prevalence of distant
education, 2000e2009 saw an increase of research outputs in implementation and design of online education in schools. VR
emerged as a sub theme under this strand of research as a technology to bring vivid experiences to classrooms. Then when big
data came onto the scene, student profiling models and learning analytics that supported personalized learning design
became a dominant focus in 2010e2019. With such temporal bibliometric analysis based on multiple-journals of AIEd
research in the past two decades, our research contributes to the AIEd literature by exploring major research paradigms in the
history, and the lifecycles of different research paradigm shifts. It helps to position on-going AIEd research in the literature.
The study also serves as a spur to initiate dialogue regarding the up-and-coming AIEd research themes in the next decade.
Despite our effort to make the coverage of this research as comprehensive as possible, our research is limited in the
following aspects. We use the density of publications to construct the concepts map and major research paradigms. In this
sense, our method is constrained to what have merged or happened in the field. This can be an issue especially in predicting
future research trends. For themes that have just emerged and have yet diffused in the field, considering the lag of 1e2 year
from research idea generation to publication, they might not be spotted as a possible emerging area. Secondly, we have
focused on AIEd literature to study the AI technology in education. However, it is important to realize that EdTech is not only
about technology, but also about the pedagogical, cultural, social, economic, ethical and psychological dimensions of edu-
cation. In this sense, AIEd research is cross-disciplinary in nature and any major development in the above disciplines may
induce a paradigm shift in AIEd research. Hence, for future studies, reviews on AIEd research should include publications in
these disciplines as well.
144 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147

Statements on open data, ethics and conflict of interest

All of the data were retrieved from publishers’ databases and Google Scholar. It does not pose any risks to individuals or
institution. No potential conflict of interest was reported by the authors.

Acknowledgments

This work was supported by Pusan National University Research Grant (202004140001).

Appendix A. The top cited papers in each decade across all journals in our analysis

Paper Title Journal Author(s) Year Cites

Redefining the learning companion: the past, present, and future of Computers & C. Chou, T. Chan & C. Lin 2003 332
educational agents Education
Web3D technologies in learning, education and training: Motivations, issues, Computers & L. Chittaro & R. Ranon 2007 436
opportunities Education
Virtual reality for collaborative e-learning Computers & T. Monahan, G. McArdle & M. Bertolotto 2008 380
Education
Intelligent web-based learning system with personalized learning path Computers & C. Chen 2008 332
guidance Education
Virtual world teaching, experiential learning, and assessment: An Computers & L. Jarmon, T. Traphagan, M. Mayrath & A. 2009 604
interdisciplinary communication course in Second Life Education Trivedi
A comparative analysis of machine learning techniques for student retention Decision D. Delen 2010 179
management. Support
Systems
Effectiveness of virtual reality-based instruction on students’ learning Computers & Z. Merchant, E. T. Goetz, L. Cifuentes, W. 2014 656
outcomes in K-12 and higher education: A meta-analysis Education Keeney-Kennicutt & T. J. Davis
Personalized e-learning system using Item Response Theory Computers & C. Chen, H. Lee, & Y. Chen 2005 541
Education
Investigating learners’ attitudes toward virtual reality learning environments: Computers & H. Huang, U. Rauch & S. Liaw 2010 447
Based on a constructivist approach Education
New technology trends in education: Seven years of forecasts and Computers & S. Martin, G. Diaz, E. Sancristobal, R. Gil, M. 2011 496
convergence Education Castro & J. Peire

References

Abbott, R. G. (2006). Automated expert modeling for automated student evaluation. In International Conference on Intelligent Tutoring Systems (pp. 1e10).
Berlin, Heidelberg: Springer.
Alam, M. M., Mohiuddin, K., Das, A. K., Islam, M. K., Kaonain, M. S., & Ali, M. H. (2018). March). A Reduced feature based neural network approach to classify
the category of students. In Proceedings of the 2nd International Conference on Innovation in Artificial Intelligence (pp. 28e32). ACM.
Alonso, F., Lopez, G., Manrique, D., & Vines, J. M. (2005). An instructional model for web-based E-learning education with a blended learning process
approach. Br. J. Educ. Technol., 36, 217e235. https://doi.org/10.1111/j.1467-8535.2005.00454.x
Anand, V. K., Rahiman, S. A., George, E. B., & Huda, A. S. (2018, March). Recursive clustering technique for students’ performance evaluation in programming
courses. In Majan International Conference (MIC) (pp. 1e5). IEEE, 2018.
Anderson, B. (2019). Revisiting MMORPGs in support of learning: changes in the last decade. In K. Becnel (Ed.), Emerging Technologies in Virtual Learning
Environments (pp. 187e214). Hershey, PA: IGI Global. https://doi:10.4018/978-1-5225-7987-8.ch010.
Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M.-T. (2009). Investigating the impact of video games on high school students’ engagement and learning
about genetics. Comput. Educ., 53(1), 74e85.
Antonietti, A., & Cantoia, M. (2000). To see a painting versus to walk in a painting: an experiment on sense-making through virtual reality. Comput. Educ.,
34(3e4), 213e223. http://doi:10.1016/s0360-1315(99)00046-9.
Barbou, M. K., & Reeves, T. C. (2009). The reality of virtual schools: a review of the literature. Comput. Educ., 52(2), 402e416. https://doi.org/10.1016/j.
compedu.2008.09.009
Baker, J. (2011). The technologyeorganizationeenvironment framework. Integrated Series in Information Systems 28. In Y. K. Dwivedi, et al. (Eds.). In-
formation Systems Theory: Explaining and Predicting Our Digital Society (vol. 1). Springer ScienceþBusiness Media, LLC, 2011.
Baker, T., & Smith, L. (2019). Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges. Retrieved from Nesta Foundation
website https://media.nesta.org.uk/documents/Future_of_AI_and_education_v5_WEB.pdf.
Bayne, S. (2015). Teacherbot: interventions in automated teaching. Teach. High. Educ., 20(4), 455e467. https://doi.org/10.1080/13562517.2015.1020783
Bejar, I. (1984). Educational diagnostic assessment. J. Educ. Meas., 21(2), 175e189. http://www.jstor.org/stable/1434541.
Bell, D. (2009). Learning from second life. Br. J. Educ. Technol., 40(3), 515e525. https://doi.org/10.1111/j.1467-8535.2009.00943.x
Bennett, S., Dawson, P., Bearman, M., Molloy, E., & Boud, D. (2017). How technology shapes assessment design: findings from a study of university teachers.
Br. J. Educ. Technol., 48(2), 672e682. https://doi.org/10.1111/bjet.12439
Bodily, R., Leary, H., & West, R. (2019). Research trends in instructional design and technology journals. Br. J. Educ. Technol., 50(1), 64e79. https://doi.org/10.
1111/bjet.12712
Bogdan, R. B., & Biklin, S. K. (1998). Qualitative Research for Education: an Introduction to Theory and Methods (third ed.). Needham Heights, MA: Allyn and
Bacon.
Bond, M., Zawacki-Richter, O., & Nichols, M. (2019). Revisiting five decades of educational technology research: a content and authorship analysis of the
British Journal of Educational Technology. Br. J. Educ. Technol., 50, 12e63. http://doi:10.1111/bjet.12730.
Carbonell, J. (1970). AI in CAI: an artificial intelligence approach to computer aided instruction. Science, 167, 190e202.
 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147 145

Carr-Chellman, A., & Duchastel, P. (2000). The ideal online course. Br. J. Educ. Technol., 31, 229e241. http://doi:10.1111/1467-8535.00154.
Chan, K. S., & Zary, N. (2019). Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med. Educ.,
5(1), 1e39. https://doi: 10.2196/13930.
Chatterjee, S., & Bhattacharjee, K. K. (2020). Adoption of Artificial Intelligence in Higher Education: a Quantitative Analysis Using Structural Equation Modelling.
Forthcoming in Education and Information Technologies.
Chen, C. M. (2008). Intelligent web-based learning system with personalized learning path guidance. Comput. Educ., 51(2), 787e814.
Chen, C. M., Lee, H. M., & Chen, Y. H. (2005). Personalized e-learning system using item response theory. Comput. Educ., 237e255.
Cheng, Y.-C., & Yeh, H.-T. (2009). From concepts of motivation to its application in instructional design: reconsidering motivation from an instructional
design perspective. Br. J. Educ. Technol., 40(4), 597e605. https://doi.org/10.1111/j.1467-8535.2008.00857.x
Chi, M., & VanLehn, K. (2010). Meta-cognitive strategy instruction in intelligent tutoring systems: how, when, and why. Educ. Technol. Soc., 13(1), 25e39.
https://www.jstor.org/stable/10.2307/jeductechsoci.13.1.25.
Chittaro, L., & Ranon, R. (2007). Web3D technologies in learning, education and training: motivations, issues, opportunities. Comput. Educ., 49(1), 3e18.
Cooper, G., Park, H., Nasr, Z., Thong, L. P., & Johnson, R. (2019). Using virtual reality in the classroom: preservice teachers’ perceptions of its use as a teaching
and learning tool. Educ. Media Int., 56(1), 1e13. https://doi.org/10.1080/09523987.2019.1583461
Chou, C. Y., Chan, T. W., & Lin, C. J. (2003). Redefining the learning companion: the past, present, and future of educational agents. Comput. Educ., 40(3),
255e269.
Cretchley, J., Rooney, D., & Gallois, C. (2010). Mapping a 40-year history with Leximancer: themes and concepts in the journal of cross-cultural psychology. J.
Cross Cult. Psychol., 41(3), 318e328. https://doi.org/10.1177/0022022110366105
Curilem, S. G., Barbosa, A. R., & de Azevedo, F. M. (2007). Intelligent tutoring systems: formalization as automata and interface design using neural networks.
Comput. Educ., 49(3), 545e561. https://doi.org/10.1016/j.compedu.2005.10.005
Daley, S. G., Hillaire, G., & Sutherland, L. M. (2016). Beyond performance data: improving student help seeking by collecting and displaying influential data
in an online middle-school science curriculum. Br. J. Educ. Technol., 47(1), 121e134. https://doi.org/10.1111/bjet.12221
Dahman, M. R., & Dag , H. (2019). Machine learning model to predict an adult learner’s decision to continue ESOL course. Educ. Inf. Technol., 24(4),
2429e2452. https://doi.org/10.1007/s10639-019-09884-5
Dann, S. (2010). Redefining social marketing with contemporary commercial marketing definitions. J. Bus. Res., 63(2), 147e153. https://doi.org/10.1016/j.
jbusres.2009.02.013
Delen, D. (2010). A comparative analysis of machine learning techniques for student retention management. Decis. Support Syst., 49(4), 498e506.
Dickey, M. D. (2005). Three-dimensional virtual worlds and distance learning: two case studies of active worlds as a medium for distance education. Br. J.
Educ. Technol., 36(3), 439e451. https://doi.org/10.1111/j.1467-8535.2005.00477.x
Edirisingha, P., Nie, M., Pluciennik, M., & Young, R. (2009). Socialisation at a distance in a 3-D multi-user virtual environment. Br. J. Educ. Technol., 40,
458e479. http://doi:10.1111/j.1467-8535.2009.00962.x.
El-Alfy, E. M., & Abdel-Aal, R. E. (2008). Construction and analysis of educational tests using abductive machine learning. Comput. Educ., 51(1), 1e16. https://
doi.org/10.1016/j.compedu.2007.03.003
Ellis, C. (2013). Broadening the scope and increasing the usefulness of learning analytics: the case for assessment analytics. Br. J. Educ. Technol., 44(4, SI),
662e664. https://doi.org/10.1111/bjet.12028
Ford, L. (2008). A new intelligent tutoring system. Br. J. Educ. Technol., 39, 311e318. https://doi.org/10.1111/j.1467-8535.2008.00819.x
Galbraith, C. S., Merrill, G. B., & Kline, D. M. (2012). Are student evaluations of teaching effectiveness valid for measuring student learning outcomes in
business related classes? A neural network and Bayesian analyses. Res. High. Educ., 53, 353e374. https://doi.org/10.1007/s11162-011-9229-0
Gallagher, J. P. (1981). The effectiveness of man-machine dialogues for teaching attribute blocks problem-solving skills with an artificial intelligence CAI
system. Instr. Sci., 10, 297e332. https://doi.org/10.1007/BF00162731
Gonz
alez-Marcos, A., Alba-Elías, F., & Ordieres-Mere
, J. (2016). An analytical method for measuring competence in project management. Br. J. Educ. Technol.,
47(6), 1324e1339. https://doi.org/10.1111/bjet.12364
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. Cambridge: The MIT Press.
Heffernan, N. T., & Heffernan, C. L. (2014). The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally
invasive research on human learning and teaching. Int. J. Artif. Intell. Educ., 24(4), 470e497. https://doi.org/10.1007/s40593-014-0024-x
Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: a gradient boosting trees supervised machine learning and
sentiment analysis approach. Comput. Educ., 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724
Hinojo-Lucena, F., Aznar-Díaz, I., Ca
ceres-Reche, M. P., & Romero-Rodríguez, J. (2019). Artificial intelligence in higher education: a bibliometric study on its
impact in the scientific literature. Educ. Sci., 9(1), 51. https://doi.org/10.3390/educsci9010051
Hoffait, A.-S., & Schyns, M. (2017). Early detection of university students with potential difficulties. Decis. Support Syst., 101, 1e11. https://doi.org/10.1016/j.
dss.2017.05.003
Huang, H. M., Rauch, U., & Liau, S. S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: based on a constructivist
approach. Comput. Educ., 1171e1182.
Hwang, G. (2003). A conceptual map model for developing Intelligent Tutoring Systems. Comput. Educ., 40(3), 217e235. https://doi: 10.1016/S0360-1315(02)
00121-5.
Ib

an~ ez, M. B., Di Serio, A.,
Villara

n, D., & Kloos, C. D. (2014). Experimenting with electromagnetism using augmented reality: impact on flow student
experience and educational effectiveness. Comput. Educ., 71, 1e13. https://doi.org/10.1016/j.compedu.2013.09.004
Indulska, M., Hovorka, D., & Recker, J. (2011). Quantitative approached to content analysis: identifying conceptual drift across publication outlets. Eur. J. Inf.
Syst., 21(2), 49e69. https://doi.org/10.1057/ejis.2011.37
Jarmon, L., Traphagan, T., Mayrath, M., & Trivedi, A. (2009). Virtual world teaching, experiential learning, and assessment: an interdisciplinary commu-
nication course in Second Life. Comput. Educ., 53(1), 169e182.
Johnson, B. G., Phillips, F., & Chase, L. G. (2009). An intelligent tutoring system for the accounting cycle: enhancing textbook homework with artificial
intelligence. J. Account. Educ., 27, 30e39.
Kardan, A. A., Sadeghi, H., Ghidary, S. S., & Sani, M. R. F. (2013). Prediction of student course selection in online higher education institutes using neural
network. Comput. Educ., 65, 1e11. https://doi.org/10.1016/j.compedu.2013.01.015
Kearsley, G. (1978). The relevance of AI research to CAI. J. Educ. Technol. Syst., 6(1), 229e250. https://doi.org/10.2190/F9WPY-EQ0J-8MT0-RGWR
Krishna, P. K. V., Kumar, M., & Aruna Sri, P. S. G. (2018). Student information system and performance retrieval through dashboard. Int. J. Eng. Technol., 7,
682e685. https://doi.org/10.14419/ijet.v7i2.7.10922
Lai, A.-F., Chen, C.-H., & Lee, G.-Y. (2019). An augmented reality-based learning approach to enhancing students’ science reading performances from the
perspective of the cognitive load theory. Br. J. Educ. Technol., 50, 232e247. https://doi.org/10.1111/bjet.12716
Larraza-Mendiluze, E., Garay-Vitoria, N., Soraluze, I., Martín, J., Muguerza, J., & Ruiz-Va
zquez, T. (2016). Using a real bare machine in a project-based learning
environment for teaching computer structure: an analysis of the implementation following the action research model. J. Educ. Resour. Comput., 16(3),
1e17. https://doi.org/10.1145/2891415
Lin, T.-J., Duh, H. B.-L., Li, N., Wang, H.-Y., & Tsai, C.-C. (2013). An investigation of learners’ collaborative knowledge construction performances and behavior
patterns in an augmented reality simulation system. Comput. Educ., 68, 314e321. https://doi.org/10.1016/j.compedu.2013.05.011
Lo, J.-J., & Shu, P.-C. (2005). Identification of learning styles online by observing learners’ browsing behaviour through a neural network. Br. J. Educ. Technol.,
36, 43e55. https://doi.org/10.1111/j.1467-8535.2005.00437.x
Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of
machine learning techniques. Comput. Educ., 53(3), 950e965. https://doi.org/10.1016/j.compedu.2009.05.010
146 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147

Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: seven years of forecasts and convergence.
Comput. Educ., 57(3), 1893e1906. https://doi.org/10.1016/j.compedu.2011.04.003
Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning
outcomes in K-12 and higher education: a meta-analysis. Comput. Educ., 70, 29e40.
McKenney, S., & Mor, Y. (2015). Supporting teachers in data-informed educational design. Br. J. Educ. Technol., 46(2 SI), 265e279. https://doi.org/10.1111/bjet.
12262
Monahan, T., McArdle, G., & Bertolotto, M. (2008). Virtual reality for collaborative e-learning. Comput. Educ., 50(4), 1339e1353.
Moridis, C. N., & Economides, A. A. (2009). Prediction of student’s mood during an online test using formula-based and neural network-based method.
Comput. Educ., 53(3), 644e652. https://doi.org/10.1016/j.compedu.2009.04.002
Mou, X. (2019, September). Artificial intelligence: investment trends and selected industry uses. https://www.ifc.org/wps/wcm/connect/7898d957-69b5-
4727-9226-277e8ae28711/EMCompass-Note-71-AI-Investment-Trends.pdf?MOD¼AJPERES&CVID¼mR5Jvd6.
Nadolny, L. (2017). Interactive print: the design of cognitive tasks in blended augmented reality and print documents. Br. J. Educ. Technol., 48, 814e823.
https://doi.org/10.1111/bjet.12462
Nilsson, N. J. (1998). Artificial Intelligence: A New Synthesis. San Francisco: Morgan Kaufmann Publishers, Inc.
Nix, I., & Wyllie, A. (2011). Exploring design features to enhance computer-based assessment: learners’views on using a confidence-indicator tool and
computer-based feedback. Br. J. Educ. Technol., 42(1), 101e112. https://doi.org/10.1111/j.1467-8535.2009.00992.x
Normadhi, N. B. A., Shuib, L., Nasir, H. N. M., Bimba, A., Idris, N., & Balakrishnan, V. (2019). Identification of personal traits in adaptive learning environment:
systematic literature review. Comput. Educ., 130, 168e190. https://doi.org/10.1016/j.compedu.2018.11.005
Pan, M. J., & Jang, W. Y. (2008). Determinants of the adoption of enterprise resource planning within the technology-organization-environment framework:
Taiwan’s communications industry. J. Comput. Inf. Syst., 48(3), 94e102.
Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. Br. J. Educ. Technol., 45(3), 438e450. https://doi.org/10.1111/bjet.12152
Popenici, S. A. D., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Res. Pract. Technol. Enhanc.
Learn. (RPTEL), 12(22), 1e13.
Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: a cross-institutional comparison across
151 modules. Comput. Hum. Behav., 60, 333e341. https://doi.org/10.1016/j.chb.2016.02.074
Rodríguez-Triana, M. J., Martínez-Mone
s, A., Asensio-Pe
rez, J. I., & Dimitriadis, Y. (2015). Scripting and monitoring meet each other. Br. J. Educ. Technol., 46(2),
330e343. https://doi:10.1111/bjet.12198.
Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. Int. J. Artif. Intell. Educ., 26(2), 582e599.
Ross, P. (1987). Intelligent tutoring systems. J. Comput. Assist. Learn., 3, 194e203.
Russell, S., & Norvig, P. (2018). Artificial Intelligence: a Modern Approach (global edition). Harlow, United Kingdom: Pearson Education Limited.
Salmon, G. (2009). The future for (second) life and learning. Br. J. Educ. Technol., 40, 526e538. https://doi.org/10.1111/j.1467-8535.2009.00967.x
Sedlmeier, P. (2001). Intelligent tutoring systems. In Neil. J. Smelser, & Paul. B. Baltes (Eds.), International Encyclopedia of the Social & Behavioral Sciences (pp.
7674e7678). Pergamon, ISBN 9780080430768. https://doi.org/10.1016/B0-08-043076-7/01618-1.
Sharma, K., Papamitsiou, Z., & Giannakos, M. (2019). Building pipelines for educational data using AI and multimodal analytics: a “grey-box” approach. Br. J.
Educ. Technol., 50, 3004e3031. https://doi.org/10.1111/bjet.12854
Shortliffe, E. H., Davis, R., Axline, S. G., Buchanan, B. G., Green, C. C., & Cohen, S. N. (1975). Computer-based consultations in clinical therapeutics: explanation
and rule acquisition capabilities of the MYCIN system. Comput. Biomed. Res., 8(4), 303e320. https://doi.org/10.1016/0010-4809(75)90009-9
Intelligent tutoring systems [Special issue]Sleeman, D. H., & Brown, J. S. (Eds.). Int. J. Man Mach. Stud., 11, (1979), 1e3. https://doi:10.1016/S0020-7373(79)
80002-4.
Smith, A. E., & Humphreys, M. S. (2006). Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping. Behav. Res.
Methods, 38(2), 262e279. https://doi.org/10.3758/BF03192778
Spikol, D., Ruffaldi, E., Dabisias, G., & Cukurova, M. (2018). Supervised machine learning in multimodal learning analytics for estimating success in project-
based learning. J. Comput. Assist. Learn., 34(4), 366e377. https://10.1111/jcal.12263.
Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. J. Educ.
Psychol., 106(2), 331e347. https://psycnet.apa.org/doi/10.1037/a0034752.
Sung, Y.-T., Liao, C.-N., Chang, T.-H., Chen, C.-L., & Chang, K.-E. (2016). The effect of online summary assessment and feedback system on the summary
writing on 6th graders: the LSA-based technique. Comput. Educ., 95, 1e18. https://doi.org/10.1016/j.compedu.2015.12.003
Tan, D., Ji, S., & Jin, M. (2013). Intelligent computer-aided instruction modeling and a method to optimize study strategies for parallel robot instruction. IEEE
Trans. Educ., 56(3), 268e273.
Tseng, C., Wu, B., Morrison, A. M., Zhang, J., & Chen, Y. (2015). Travel blogs on China as a destination image formation agent: a qualitative analysis using
Leximancer. Tourism Manag., 46, 347e358. https://doi.org/10.1016/j.tourman.2014.07.012
Tornatzky, L. G., & Fleischer, M. (1990). The Processes of Technological Innovation. Lexington, MA: Lexington books.
Vaculík, K., Popelínský, L., Mra
kova
, E., & Jurco, J. (2013, October). Tutoring and automatic evaluation of logic proofs. In European Conference on e-Learning (p.
495). Academic Conferences International Limited.
Van Heerden, B., Aldrich, C., & Du Plessis, A. (2008). Predicting student performance using artificial neural network analysis. Med. Educ., 42, 516e517. https://
doi:10.1111/j.1365-2923.2008.03052.x.
Viegas, C., Pavani, A., Lima, N., Marques, A., Pozzo, I., Dobboletta, E., Atencia, V., Barreto, D., Calliari, F., Fidalgo, A., Lima, D., Tempor~ ao, G., & Alves, G. (2018).
Impact of a remote lab on teaching practices and student learning. Comput. Educ., 126(July), 201e216.
Virvou, M., & Alepis, E. (2005). Mobile educational features in authoring tools for personalised tutoring. Comput. Educ., 44(1), 53e68. https://doi.org/10.
1016/j.compedu.2003.12.020
Wang, H. C., Chang, C. Y., & Li, T. Y. (2008). Assessing creative problem-solving with automated text grading. Comput. Educ., 51(4), 1450e1466.
Wang, W., & Siau, K. (2019). Artificial intelligence, machine learning, automation, robotics, future of work and future of humanity: a review and research
agenda. J. Database Manag., 30(1), 61e79. https://doi:10.4018/JDM.2019010104.
Wang, B., Wang, J., & Hu, G. (2017). College English classroom teaching evaluation based on particle swarm optimization e extreme learning machine
model. Int. J. Emerg. Technol. Learning, 12(5), 82e97. https://doi.org/10.3991/ijet.v12i05.6782
Warburton, S. (2009). Second Life in higher education: assessing the potential for and the barriers to deploying virtual worlds in learning and teaching. Br. J.
Educ. Technol., 40(3), 414e426.
Weismayer, C., & Pezenka, I. (2017). Identifying emerging research fields: a longitudinal latent semantic keyword analysis. Scientometrics, 113(3), 1757e1785.
https://doi.org/10.1007/s11192-017-2555-z
Weizenbaum, J. (1966). ELIZA d a computer program for the study of natural language communication between man and machine. Commun. ACM, 26(1),
23e28. https://doi.org/10.1145/357980.357991. January 1983.
Whitworth, D. E., & Wright, K. (2015). Online assessment of learning and engagement in university laboratory practicals. Br. J. Educ. Technol., 46(6),
1201e1213. https://doi.org/10.1111/bjet.12193
Wong, W.-K., Hsu, S.-C., Wu, S.-H., Lee, C.-W., & Hsu, W.-L. (2007). LIM-G: learner-initiating instruction model based on cognitive knowledge for geometry
word problem comprehension. Comput. Educ., 48(4), 582e601. https://doi.org/10.1016/j.compedu.2005.03.009
Wu, J. Y., Hsiao, Y. C., & Nian, M. W. (2018). Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in
predicting learning achievement within the personal learning environment. Interact. Learn. Environ., 1e16.
Yang, J. C., Chen, C. H., & Jeng, M. C. (2010). Integrating video-capture virtual reality technology into a physically interactive learning environment for English
learning. Comput. Educ., 55(3), 1346e1356. https://doi.org/10.1016/j.compedu.2010.06.005
 C. Guan, J. Mou and Z. Jiang / International Journal of Innovation Studies 4 (2020) 134e147 147

Yeh, C. H., Lee, G. G., & Pai, J. C. (2015). Using a technology-organization-environment framework to investigate the factors influencing e-business infor-
mation technology capabilities. Inf. Dev., 31(5), 435e450.
Zawacki-Richter, O., & Naidu, S. (2016). Mapping research trends from 35 years of publications in distance education. Dist. Educ., 37(3), 245e269. https://doi.
org/10.1080/01587919.2016.1185079

Chong Guan is an Associate Professor in the School of Business, Singapore University of Social Sciences (SUSS), Singapore. Her research interest lies in the
area of user decision making on digital platforms. Email: guanchong@suss.edu.sg.

Jian Mou is an Assistant Professor of Management Information Systems in the School of Business, Pusan National University. His research interests include e-
commerce, social media, human computer interaction, trust and risk issues in electronic services, and consumer health informatics. Email: jian.mou@
outlook.com

Zhiying Jiang is the Head of Graduate Programme in Digital Marketing in the School of Business, Singapore University of Social Sciences (SUSS), Singapore.
Her research interests include digital user analytics and mobile economics. Email: jiangzhiying@suss.edu.sg.