Journal of Research on Technology in Education

ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/ujrt20

A systematic review of conversational AI in

language education: focusing on the collaboration
with human teachers

Hyangeun Ji, Insook Han & Yujung Ko

To cite this article: Hyangeun Ji, Insook Han & Yujung Ko (2023) A systematic review
of conversational AI in language education: focusing on the collaboration with
human teachers, Journal of Research on Technology in Education, 55:1, 48-63, DOI:
10.1080/15391523.2022.2142873

To link to this article: https://doi.org/10.1080/15391523.2022.2142873

Published online: 30 Nov 2022.

Submit your article to this journal

Article views: 4197

View related articles

View Crossmark data

Citing articles: 4 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ujrt20
Journal of Research on Technology in Education
2023, VOL. 55, NO. 1, 48–63
https://doi.org/10.1080/15391523.2022.2142873

Review

A systematic review of conversational AI in language

education: focusing on the collaboration with human
teachers
Hyangeun Jia , Insook Hana,b and Yujung Koc
a
Department of Teaching and Learning, College of Education and Human Development, Temple University,
Philadelphia, PA, USA; bDepartment of Education, Korea University, Seongbuk-gu, South Korea; cDepartment of
Platform EdTech, Hanyang Cyber University, Seongdong-gu, South Korea

ABSTRACT ARTICLE HISTORY

Despite the increasing use of conversational artificial intelligence (AI) in Received 10 April 2022
language learning, few studies explored how to develop collaborative part- Revised 15 October 2022
nership between AIs and humans. This systematic review examines empirical Accepted 18 October
evidence of human-computer collaboration from 24 studies conducted in 2022
an AI-integrated language learning environment and published between KEYWORDS
2015 and 2021. The roles of conversational AIs and teachers in each lan- Conversational AI;
guage learning phase with challenges of and suggestions for conversational language learning;
AI-integrated language learning were identified. Although limited evidence intelligence amplification;
for collaboration between conversational AIs and human teachers was systematic review
found, future language education should integrate conversational AIs to
promote intelligence amplification and decrease human teachers’ workload
through classroom orchestration. The study concludes with guidelines and
recommendations for teachers and AI researchers.

Introduction
Artificial intelligence (AI) has gained tremendous attention recently as a potential tool to inno-
vate teaching and learning in educational settings. However, the idea of applying AI in education
is not entirely new. Since Turing (1950) explored the mathematical possibility of AI by suggesting
the Turing Machines, the development of AI and research on learning and educational applica-
tions have experienced rises and falls. In particular, Minsky and Papert (1968) pioneered AI
research and applied computational ideas to emulate human psychological processes in machines
so that a computer could solve problems and make decisions based on available information as
humans do. Since then, early research on AI and education focused on developing and testing
intelligent tutoring systems (ITS). Coupled with the pedagogical approach of learning by teaching,
some ITSs have evolved into teachable agents that help students learn while teaching computer
agents (Blair et al., 2007; Leelawong & Biswas, 2008; Silvervarg et al., 2021).
With a recent dramatic increase in the investment in AI industries (Pan, 2016), AI has gained
new momentum. It has driven the advancement of AI technology, such as machine learning,
neural network, natural language processing (NLP) with automatic speech recognition (ASR),
and advanced image processing (Zawacki-Richter et al., 2019) and increased access to commer-
cially available AI devices and mobile applications. Wider and easier access to AI technologies
allowed a potential increase in its application in educational settings. This was supported by a

CONTACT Insook Han hanis79@gmail.com  Department of Teaching and Learning, College of Education and Human
Development, Temple University, 439 Ritter Hall, 1301 Cecil B. Moore Ave, Philadelphia, PA19122-6091, USA
© 2022 ISTE
 Journal of Research on Technology in Education 49

significant increase in the number of papers on this topic published since 2015 (Chen et al.,
2020). Indeed, a review by Chen et al. (2020) provided a wide array of educational scenarios,
where recent AI technologies have been applied in administration, instruction, and learning.
The scenarios included personalized intelligent teaching, assessment and evaluation, smart school,
and online and mobile remote education.
One notable example is the rapid growth in the development of chatbots (Adamopoulou &
Moussiades, 2020), a type of conversational AI that allows natural language conversations between
humans and machines by texts or voice. While ITS is often used as an umbrella term for
knowledge-based adaptive tutoring systems that often involve both pedagogical and conversational
abilities, conversational AI refers to a system that emphasizes the conversational abilities of an
ITS whose conversation usually takes place using NLP technology (Fadel et al., 2019). Conversational
AI has been increasingly adopted by educational researchers, particularly in language education,
with its potential to provide learners with chances to interact and support communicative lan-
guage teaching (Fryer et al., 2019; Hapsari & Wu, 2022). With the increasing interest, many
previous reviews that included conversational AI in language learning looked at its broad appli-
cation in education (Pérez et al., 2020). A few recent reviews solely focused on the use of
conversational AI in language learning (Chen et al., 2021; Huang et al., 2021; Liang et al., 2021).
However, these reviews concentrated on the effectiveness of conversational AI in language edu-
cation or viewed conversational AI either as a medium or a replacement of a human teacher.
Acknowledging the limitations of automated interventions with ITSs (Baker, 2016; Roschelle
et al., 2020), this special issue calls for rethinking the ways of using AI for amplifying human
intelligence and developing collaborative partnerships between humans and machines. Compared
to many promises that AI-enabled technologies could bring to language learning settings, few
guidelines are available for teachers on how they utilize AI technologies for enhancing student
learning (Kim et al., 2022). Previous language learning literature defined various teachers’ roles
(Alrayah, 2018; Brandl, 2007), some of which could be supported, enhanced or modified by
using AI technologies. Since the use of AI in classrooms is a rapidly emerging area, we need
to understand how teachers can collaborate with AI technologies while engaging in enacting
instructional roles to support student learning. Therefore, this study aims to explore how a
teacher uses a conversational AI, what instructional roles conversational AI play, and how it
supports or modifies traditional language teachers’ roles.

Previous studies
Potential of conversational AI as a language learning tool
The affordances of conversational AIs can be best supported by the interaction theory, which
emphasizes the language learners’ collaborative efforts to interact and communicate with another
speaker (Chapelle, 2005; Li, 2018; Long, 1996). However, language learners often have limited
chances to practice speaking and receive feedback (Terhune, 2016). Furthermore, chances for
language learners to engage in authentic communication are insufficient (Alharbi, 2015; Jo, 2008).
Recent advances in machine learning, ASR, and NLP technologies have made conversational
AI-integrated language learning a more suitable and economical approach. They provide language
learners access to language learning resources and an authentic environment for communication
in a target language.
Furthermore, conversational AI can reduce foreign language anxiety, a constant concern that
hinders language learning performance and achievement (Horwitz, 2001; Shao et al., 2019;
Teimouri et al., 2019). Shao et al. (2019) emphasized the role of students’ positive emotions such
as hope, pride, and contentment in language learners’ motivation and performance. It was also
argued that teachers should strive to reduce students’ foreign language anxiety in and outside
the language classroom. Among several techniques and interventions to reduce or overcome
language anxiety, drill and repetition (Chen & Chang, 2004; Gregersen et al., 2014), teachers’
positive feedback (Alnuzaili & Uddin, 2020) have been discussed as viable solutions by many
50 H. JI ET AL.

language researchers. Furthermore, language learners can interact with conversational AI in a

more flexible manner (Zhang & Zou, 2020). They can also receive scaffolding and feedback in
a less-threatening environment (Bibauw et al., 2019; Istrate, 2018) than in traditional classrooms.

Role of teachers in conversational AI-integrated language learning

Interaction theory also emphasizes language teachers’ role while interacting with learners and
giving feedback (Kinginger, 2001). Alrayah (2018) described the language teacher’s role as “be[ing]
a facilitator of learning, monitoring and collecting data on learner performance, as well as
intervening when the group needs assistance in doing the task.” (p. 29) Additionally, teachers
should also be able to encourage collaborative learning processes while providing constructive
feedback and promoting a safe feeling to learn. Over the few decades, many language researchers
suggested the need to reconceptualize the evolving teacher roles, one of which involves the
consideration of technology incorporation. Hedayati et al. (2017) explained how language learning
is becoming more resourceful, and learning contexts are evolving owing to technological devel-
opment. Furthermore, the teachers’ roles should also be reconceptualized to capture this
dynamicity.
AI is expected to transform teachers’ roles in classrooms (Baker, 2016; Chaudhry & Kazim,
2022; Holstein et al., 2020; Kim et al., 2022). Similar to many predictions that automated AI will
replace future jobs in the next 15 years (Zeira, 1998), AI is also expected to automate teachers’
repetitive and time-consuming tasks. This includes tasks such as tracking student progress,
grading, or taking attendance (Bushweller, 2020). The automation will reduce teachers’ daily
workload (Bryant et al., 2020). Consequently, teachers could focus more on making data-driven
instructional decisions based on AI-assisted student data to make instruction more personalized
and adaptive to individual students (Wang, 2021). Notably, Baker (2016) claimed that intelligence
should be placed in teachers rather than in the tools they use, and the ultimate goal of AI in
education (AIEd) should be to promote education, not artificial intelligence.

Previous reviews on AIEd in language learning

With the increasing interest in AI in language learning, several review papers in AIEd have
discussed different aspects of using AIs for educational purposes. For example, the reviews
focused on the range of chatbot types and their technology (Pérez et al., 2020) and ethical
implications and risks (Zawacki-Richter et al., 2019). Pérez et al. (2020) and Smutny and
Schreiberova (2020) commonly reported that conversational AIs had been substantially adopted
in the language learning domain due to their advantages, such as motivating language learners.
Other reviews also provided comprehensive summaries in highlighting various issues in language
learning. They included technological, pedagogical, and social affordances (Huang et al., 2021),
article trends, top journals, countries/regions and institutions, AI technology types, learning
outcomes, support, participants, scientific collaborations, and co-citation relations (Chen
et al., 2021).
While the aforementioned studies reviewed AI in language learning from technological and
learning perspectives, a few addressed different roles AI and language teachers would play to
enhance teaching practices. For instance, Liang et al. (2021) broadly reviewed AIEd research in
language learning from 1990 to 2020. It was established that AI took four different roles of
profiling and prediction, assessment and evaluation, adaptive systems and personalization, and
intelligent tutoring systems. However, it did not specifically focus on the new advancement of
AI technologies that might afford different dynamics in AI roles. Further, to the best of our
knowledge, no previous reviews have investigated the evolving role of teachers in AI-integrated
language teaching. Therefore, to maximize the potential of AI in language classrooms, we need
to better understand how humans and AI can collaborate in transforming language classrooms
to become more individualized and adaptive.
 Journal of Research on Technology in Education 51

Purpose of the study

This study aims to close the research gap by suggesting possible collaborative opportunities
between humans and machines to ensure that AI can extend human intelligence (Baker, 2016).
To this end, we conducted a systematic review analysis to establish the current status of using
conversational AI in language learning, the challenges in using AI, and the potential for intel-
ligence amplification by asking the following research questions:

1. What are the instructional roles of conversational AI and teachers in conversational-AI-in-

tegrated language learning?
2. What challenges emerged when integrating conversational AI into language learning?
3. How can human-computer collaboration deal with the challenges in conversational-AI-in-
tegrated language learning?

Methods
For the systematic review, we followed the four-phase process described in the Preferred Reporting
Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline (Liberati et al., 2009). The
process involved (a) identification of the possible studies, (b) abstract screening, (c) full-text
assessment for eligibility, and (d) qualitative synthesis of the included studies. Figure 1 provides
an overview of the process and the number of papers included and excluded at each phase.

Figure 1. PRISMA flowchart (Liberati et al., 2009).

52 H. JI ET AL.

Search procedures
To identify possible studies, six databases in the educational field (EBSCO, Wiley Online Library,
Science Direct, Web of Science, Taylor and Francis, and ProQuest) were searched. To reduce
publication bias, ProQuest was added to include conference proceedings and dissertations. We
limited the systematic literature search to articles published between January 2015 and January
2022, which is when the AI industry drove a dramatic technological advancement and related
papers increased in number. We conducted full-text searches using comprehensive keywords on
conversational AI (“conversational agents” OR “chatbot” OR “chat bot” OR “intelligent personal”
OR “intelligent agents”) and language learning (“language learning” OR “language teaching” OR
“language classroom” OR “language education”). Following the database search screening, a hand
search on the titles of the reference list from the screened articles was conducted.

Inclusion and exclusion criteria

To select the articles to be included in the analyses, we used the following criteria: (1) the paper
must be written in English, (2) the study must be related to second language learning, (3) the
study must be conducted with conversational AI, (4) the study must include empirical data
(qualitative, quantitative or both), (5) the study must be conducted for student learning. Based
on these criteria, the studies exploring second language learning-related outcomes (e.g., reading,
listening, speaking, writing, or affective outcomes) were included. However, those focused on
first language, sign language, or computer language learning were excluded. Conversational AI
includes chatbots, intelligent personal assistants (IPAs), robots, and virtual or other multimedia
platforms that incorporate AI technology. Chatbots were either research-built or ready-made
applications such as Cleverbot and Replika, and IPAs included Amazon’s Alexa or Google
Assistant. Other multimedia platforms, such as virtual reality, extended reality, or embodied
conversational AI, were included only when equipped with AI technologies. Thus, online trans-
lators or pre-programmed pedagogical, animated, embodied agents were excluded. Lastly, to
include the studies that focused on student learning, we excluded studies involving only teachers
and teacher education, the design, development, evaluation of systems, or usability surveys.
The initial database search yielded 673 articles, including peer-reviewed journal articles, book
chapters, conference proceedings, and dissertations. After removing duplicates, 636 of them were
included in the abstract screening. The three researchers independently reviewed and coded the
articles. Two researchers screened each article and selected 181 articles for full-text screening.
Inter-reliability was calculated by dividing the number of agreements made on each article by
the total number of ratings and multiplying 100 (Belur et al., 2021). The overall inter-rater
reliability for abstract screening was 95.75, which showed a good agreement. 100% agreement
was achieved after the discussion. Full-text screening was then performed, and the three research-
ers came to a good agreement with an inter-rater reliability of 83.9. 100% agreement was achieved
after three discussion sessions with thorough reading of the disagreed articles, which resulted
in 19 articles for inclusion.
In addition to the electronic search, a manual search of the reference list for the 19 articles
was performed by two researchers. After removing duplicates, 14 articles were screened using the
same inclusion and exclusion criteria as the abstract and full-text screening. Five studies met the
criteria and were added to the final list, making a total of 24 articles included in the study.

Data extraction and analysis

We conducted a content analysis to answer our research questions by applying inductive and
deductive approaches (Mayring, 2000). First, to understand the overall characteristics of the 24
articles, we summarized them based on study type, study design, study setting, learner type,
target language, and language learning outcome. Then, to answer the first research question, we
 Journal of Research on Technology in Education 53

conducted an inductive category development by formulating a criterion of extracting themes

based on the research question, as suggested by Mayring. Specifically, we inductively coded the
studies by closely reading the methods, discussion, and conclusion sections and found emerging
themes regarding the roles of conversational AI and teachers. Subsequently, we took a deductive
approach to map the themes onto the existing framework that classified language teacher roles
to identify what traditional teacher roles were replaced by conversational AI, what roles were
still performed by teachers, and what additional roles emerged. For this, we based the review
on Brandl (2007)’s classification of language teacher roles. The framework is one of the very
few that solely focuses on language teachers’ instructional practices and captures the compre-
hensive teacher roles at each phase of traditional language instruction. Furthermore, the frame-
work is tailored for communicative language teaching, the approach to language learning that
emphasizes interactive conversations and has been widely used in the language classrooms
(Kukulska-Hulme & Lee, 2020). Considering that conversational AIs could effectively support
communicative practices and tasks in language learning (Fryer et al., 2019; Hapsari & Wu, 2022),
Brandl’s categorization could be also applied to the identification of language teachers’ and AIs’
new roles in language classrooms.
Specifically, Brandl defines eleven language teachers’ roles in the four different language
learning phases. In the input phase, a teacher introduces learning content as a presenter. In the
assimilation phase, a teacher leads the students along the sequence of learning tasks while serving
as a designer, organizer, or guide. In the application and extension phase, a teacher sets up, and
models student-centered activity as a facilitator, walks through the classroom, monitors student
work, or answers questions as a resource provider, group process manager, or needs analyst. Lastly,
in the follow-up or synthesis phase, a teacher serves as a discussion leader, classroom manager/
facilitator, or evaluator who draws conclusions about the students’ performance. Furthermore,
to address the second and third research questions, we used an inductive approach to categorize
the challenges of conversational AI-integrated language learning and suggestions for collaboration
between conversational AI and teachers.

Results
Study characteristics
Table 1 describes the overall characteristics of 24 articles included in this review. The number
of articles published each year seemed to be increasing between 2015 and 2021, with only a
slight drop in 2017 and 2018 (see Figure 2). Among the 24 studies, the majority of them were
journal articles (n = 17), followed by conference papers (n = 4), dissertations (n = 2), and a book
chapter (n = 1). Half of the reviewed articles employed a quantitative research design approach
(n = 12), immediately followed by a mixed-method approach (n = 11), and only one study con-
ducted qualitative research (n = 1).
More than half of the studies (n = 13) were performed in formal learning settings during
regular class time. However, five studies were set in informal learning environments, and par-
ticipants used AI technology more freely. Also, another five studies were set in lab environments.
Figure 3 reveals that the reviewed studies were dominantly conducted with learners pursuing
higher education (n = 20). The rest of the studies had participants from elementary education,
secondary education, and adult learning, each with one study that did not specify the education
level of the learners.
In most of the reviewed articles, the target language for learning was English (n = 20), with
learners studying English as a second or foreign language. The participants in the remaining
four articles intended to learn Chinese, French, Japanese, and Russian. In measuring the impact
of language learning with conversational AI, some studies focused solely on one outcome while
others employed multiple outcome measurements, which made duplicate counting for outcome
measurement for each study. In most studies (n = 17), speaking was the main focus, and
 54

Table 1. Study characteristics and roles of conversational AI.

Language learning
Study Study type Study design Study setting Learner type Target language outcome Roles of conversational AI
Al-Kaisi et al. (2019) Conference Paper Quantitative Formal learning Higher Education Russian Speaking, Vocabulary, Conversation partner (oral)
Grammar
Ayedoun et al. Journal Article Quantitative Lab Higher Education English Speaking Conversation partner (oral)
H. JI ET AL.

(2015)
Bao (2019) Journal Article Quantitative Informal learning Adult Learning English Speaking, Perception Conversation partner (oral)
Cai et al. (2020) Conference Paper Quantitative Formal learning Higher Education Japanese Speaking Feedback provider
Divekar (2020) Dissertation Mixed-methods Informal learning Higher Education Chinese Speaking, Listening, Conversation partner (oral)
Vocabulary,
Perception
Dizon (2020) Journal Article Quantitative Formal learning Higher Education English Speaking, Listening, Conversation partner (oral)
Perception
El Shazly (2021) Journal Article Mixed-methods Formal learning Higher Education English Speaking Conversation partner (written/
oral), Resource provider
Fryer et al. (2017) Journal Article Quantitative Formal learning Higher Education English Interest Conversation partner (oral)
Fryer et al. (2019) Journal Article Mixed-methods Lab Higher Education English Listening, Reading, Conversation partner (oral)
Interest, Perception
Fryer et al. (2020) Journal Article Quantitative Formal learning Higher Education English Interest, self-efficacy, Conversation partner (oral)
self-concept
Hassani et al. (2016) Journal Article Quantitative Lab Higher Education English Speaking, Listening Conversation partner (oral),
Evaluator
Hsu et al. (2021a) Journal Article Mixed-methods Formal learning Higher Education English Speaking, Listening, Conversation partner (oral),
Perception Resource provider
Hsu et al. (2021b) Journal Article Quantitative Informal learning Higher Education English Speaking Conversation partner (oral),
Feedback provider
Jeon (2021) Journal Article Mixed-methods Formal learning Elementary English Vocabulary Feedback provider, Needs
Education analyst, Evaluator
Kılıçkaya (2020) Book Chapter Qualitative Formal learning Higher Education English Perception Conversation partner (written)
Kim (2016) Journal Article Quantitative Not specified Higher Education English Speaking, Perception Conversation partner (oral)
Liakin et al. (2014) Journal Article Quantitative Informal learning Higher Education French Speaking Conversation partner (oral),
Feedback provider
Lin and Mubarok Journal Article Mixed-methods Formal learning Higher Education English Speaking, Behaviors Conversation partner (oral),
(2021) Resource provider
McCrocklin (2016) Journal Article Mixed-methods Formal learning Higher Education English Beliefs, Behaviors Conversation partner (oral)
Moussalli and Conference Paper Mixed-methods Lab Not specified English Perception Conversation partner (oral)
Cardoso (2016)
Ruan (2021) Dissertation Quantitative Lab Higher Education English Speaking, Vocabulary Conversation partner (oral),
Feedback provider
Tai and Chen (2020) Journal Article Mixed-methods Formal learning Secondary Education English Speaking, Perception Conversation partner (oral)
Thompson et al. Conference Paper Mixed-methods Formal learning Higher Education English Interest, Perception Conversation partner (oral)
(2018)
Yin and Satar (2020) Journal Article Mixed-methods Informal learning Higher Education English Speaking, Perception Conversation partner (written/
oral)
 Journal of Research on Technology in Education 55

Figure 2. Number of articles published by year.

Figure 3. Learner types.

perception of language learning with conversational AI was also of interest (n = 11). Other out-
come measurements included listening (n = 5), vocabulary (n = 4), interest (n = 4), behavior (n = 2),
reading (n = 1), grammar (n = 1), self-efficacy (n = 1), self-concept (n = 1), and belief (n = 1).

Role of conversational AI in language learning

From the content analysis based on Brandl’s classification (2007), we found that conversational
AI served as an evaluator, resource evaluator, and needs analyst out of eleven roles, in selected
studies. In some studies, conversational AIs took multiple roles (Table 1).
Specifically, as a resource provider, conversational AI presented learners with authentic videos
that helped learners expand their lexical and structural competency. Furthermore, it improved
phonological awareness during oral and writing practices (El Shazly, 2021), provided information
about unknown words using online searches (Hsu et al., 2021a), or sent videos, pictures, memes,
56 H. JI ET AL.

and songs to learners to aid speaking activities (Lin & Mubarok, 2021). Subsequently, as an
evaluator, conversational AI provided diagnostic information about learners’ vocabulary ability
(Jeon, 2021) or estimated learners’ conversational proficiency level (Hassani et al., 2016).
Furthermore, Hassani et al. (2016) established that conversational AI adjusted its proficiency
level and offered adaptive feedback based on the evaluations it made. Lastly, as a needs analyst,
conversational AI tailored instructions based on the learner’s specific needs (Jeon, 2021). For
example, if a learner needed an implicit prompt, the conversational AI asked the learner to read
the sentence again and guess the meaning. If a learner needed a more explicit prompt, it pro-
vided another sentence that includes the word in a new, more specific context.
Beyond the roles defined by Brandl (2007), two additional roles emerged from the data
analysis. They included the conversation partner role in a written or oral form and the feedback
provider role which provided learners with individualized feedback. The most prominent role
of conversational AI was a conversational partner. Most studies (n = 22, 91.7%) used conversa-
tional AI as a conversational partner for speaking (n = 19) using the learners’ voice input, for
writing tasks (n = 1) using the learners’ text input, or both (n = 2, El Shazly, 2021; Yin & Satar,
2020). In these studies, conversational AIs produced audio and textual outputs with facial
expressions. For example, in El Shazly (2021), language learners in higher education practiced
their pronunciation using a task-oriented dialogue chatbot and extensively interacted with web
chatbots for written conversations. Both chatbots employed NLP and human-like intelligence,
allowing participants to receive acceptable comprehensible input and produce multiple foreign
language outputs. In addition to providing language practice opportunities, conversational AI
was designed as a conversational partner to help language learners with foreign language learning
anxiety (Bao, 2019), willingness to communicate (Ayedoun et al., 2015; Tai & Chen, 2020),
language course/task interest (Fryer et al., 2017; Thompson et al., 2018), language course
self-efficacy (Fryer et al., 2020), language learning interest (Fryer et al., 2019), autonomous
learning behavior (McCrocklin, 2016), and negotiation for meaning (Yin & Satar, 2020).
Another remarkable role identified was feedback provider. In five studies (Cai et al., 2020; Hsu
et al., 2021b; Jeon, 2021; Liakin et al., 2014; Ruan, 2021), conversational AI provided individual
feedback to each learner through implicit/explicit prompts, voice messages, feedback sentences, or
written visuals. Notably, conversational AI in the two studies (Jeon, 2021; Ruan, 2021) modified
its feedback based on the learners’ input. For example, in Ruan (2021), conversational AI provided
adaptive feedback by comparing learners’ responses with the correct response using semantic
similarity, length of the response, and classifying the responses into three performance levels, which
was effective in engaging, motivating learners, and improving language learning outcomes.

Challenges of conversational AI-integrated language learning

In contrast to the explicit role of conversational AI, most studies (n = 18, 75%) did not clearly
define the role of language teachers, or teachers were minimally involved or completely absent
during conversational AI-integrated language instruction. It caused various challenges in student
learning with conversational AI, as reported in 12 studies with empirical evidence.
The most frequent challenge was interaction challenge, and the sources of this challenge were
multifold. First, many studies reported that misinterpretation caused by ASR systems and NLP
technologies in conversational AI led to communication breakdown (El Shazly, 2021; Kim, 2016;
Dizon, 2020; Divekar, 2020) or off-topic responses (Kılıçkaya, 2020). Kim (2016) found that
beginning-level learners experienced more communication breakdowns and were more confident
when conversing with human peers. Additionally, four studies addressed the limitations found
in the interaction between learners and conversational AI even when communication breakout
did not occur. Namely, the conversation practices were often not goal-oriented (El Shazly, 2021)
or overlooked learners’ grammatical mistakes (Al-Kaisi et al., 2019). Two other studies argued
that conversational AI should create more immersive communication, reproduce social conver-
sation (Ayedoun et al., 2015), and provide in-depth and involved conversation (Bao, 2019).
 Journal of Research on Technology in Education 57

Additionally, four studies addressed psychological challenges. El Shazly (2021) reported that
conversational AI in the study may have failed to match the learners’ topical interests and meet
their emotional needs. Similarly, learners in Kılıçkaya (2020) pointed out the lack of emotional
expressions in conversational AI. Furthermore, Fryer et al. (2017) and Fryer et al. (2019) found
that students lost interest in conversing with the chatbot after the initial practice, suggesting
the possibility of a novelty effect. To tackle these challenges, El Shazly (2021) and Fryer et al.
(2019) commonly emphasized the role of teachers to maintain learner interest and provide
support and empathy in social interactions. Finally, technical difficulties were found in Kılıçkaya
(2020), where learners had connection problems and experienced learning application crashes.

Collaboration between conversational AI and language teachers

Although many studies used conversational AI without considering the role of teachers, two of
these studies (Divekar, 2020; Hsu et al., 2021a) described how conversational AI and teachers
collaborated in the application and extension learning phase. In Divekar (2020), learners practiced
speaking with conversational AI while teachers supported the learning process as feedback pro-
viders and group process managers. Similarly, in Hsu et al. (2021a), conversational AI played
the role of a conversational partner, while teachers played the role of a needs analyst and group
process manager. Specifically, the researcher and teacher walked around the class during speaking
practices with conversational AI to check students’ participation and provide help when necessary.
Several studies also proposed ways how conversational AI and language teachers could col-
laborate in and outside classrooms to address some of the aforementioned challenges. To address
interaction challenges, Jeon (2021) stated that teachers should more actively carry out a mediator
role during conversational practices by correcting students’ requests and explaining difficult
answers given by conversational AI. Similarly, to tackle the psychological needs of learners, El
Shazly (2021) emphasized language teachers’ facilitative role in language learning anxiety that
constructively steers learners’ affective condition. Furthermore, Divekar (2020) found that having
multiple conversational AIs and/or teachers was beneficial as they make class conversations more
dynamic and increase student engagement.
Besides the collaboration during classroom time, three studies suggested how conversational
AI and language teachers could collaborate outside the classroom to maximize learning and
teaching efficiency. Liakin et al. (2014) and McCrocklin (2016) commonly argued that teachers
could use more classroom time for meaningful communicative activities with human peers and
teachers if they assign repetitive pronunciation practices to conversational AI outside the class-
room. Additionally, Jeon (2021) discussed that language teachers could utilize diagnostic infor-
mation provided by conversational AI to make pedagogical decisions for subsequent learning.

Discussion
This systematic review aimed to examine (1) the roles of conversational AI and teachers iden-
tified in the current language learning literature and (2) how collaboration between conversational
AI and language teachers could be achieved to tackle the challenges identified in the literature
and help progress toward intelligence amplification.

Roles of conversational AI and teachers

From the analysis of 24 articles based on Brandl’s classification (2007), we found that conver-
sational AIs were mostly used in the application and extension phase, playing roles as a con-
versational partner, feedback provider, resource provider, and needs analyst. In two studies,
conversational AI was also used in the follow-up or synthesis phase as an evaluator. Many of
the reviewed studies confirmed the potential of conversational AIs in addressing the limitation
58 H. JI ET AL.

of current language instruction identified in previous studies, such as foreign language learning
anxiety (Hsu et al., 2021b), lack of individualized feedback (Luan & Tsai, 2021), or lack of
opportunity for practicing speaking (Muhammad et al., 2020). By providing emotionally safe
learning environments where language learners can practice conversation with individualized
feedback, conversational AI can support communicative practices.
Unlike the prominent roles of conversational AIs, the roles of teachers were not clearly defined
in the majority of studies. This finding is a stark contrast to many arguments that emphasize the
role of teachers in AIEd (Baker, 2016; Chaudhry & Kazim, 2022; Holstein et al., 2020; Kim et al.,
2022; Zheng, 2020). However, our study provides insights into human teachers’ roles in language
classrooms. According to the results of our study, conversational AIs are rarely involved in the
designing, evaluating, and decision-making processes. In these instructional processes, human
intelligence is required for interpreting student data, designing learning activities, or making deci-
sions to adapt to instructional situations emerged during teaching (Cukurova et al., 2019). Therefore,
we expect that teachers continue performing traditional language teacher’s roles, such as presenting
new content, providing learning tasks, facilitating activities, and leading discussions.
Moreover, teachers can play a significant role in addressing the challenges of conversational
AI-integrated language learning identified in the current study. Our findings revealed that the
use of conversational AI without human inputs could render several challenges: interaction
challenges that involve misinterpretation of learner inputs and conversations that are off-topic,
not goal-oriented, or less immersive and social; psychological challenges that fail to meet learners’
interest and emotional needs; and technical difficulties, such as connection problems or appli-
cation crashes. In dealing with these challenges, teachers can play a unique role as a facilitator
and feedback provider by offering meaningful language practices, feedback, and emotional support
as suggested in some of the reviewed studies (Divekar, 2020; El Shazly, 2021; Jeon, 2021).

Collaboration between humans and machines

We also tried to understand how conversational AI and human teachers can collaborate in
language classrooms. Although many researchers suggested ways for intelligence amplification
to be achieved in conversational AI-integrated classrooms (Baker, 2016; Chaudhry & Kazim,
2022; Holstein et al., 2020; Kim et al., 2022), given that only two reviewed studies provided
evidence of human-computer collaboration, we argue that evidence from the empirical studies
in the language learning domain is still surprisingly weak. Future innovations in AI are expected
to overcome technical difficulties and interaction challenges (Divekar et al., 2021). However, to
transform conversational AI-integrated language learning to be truly meaningful to individual
students, language teachers should understand how to collaborate with conversational AIs in
different learning/teaching phases. In the technology-enhanced learning community, researchers
claim that this collaboration between humans and machines can be achieved through classroom
orchestration, where human teachers are at the core of human-computer collaboration and drive
the whole learning activity (Dillenbourg & Jermann, 2010; Roschelle et al., 2013). The result of
this review study and other studies on AI and learning suggest implications on how classroom
orchestration could be achieved to increase teaching efficiency and reduce the workload of
teachers by supporting human teachers in each language learning phase.
First, it was suggested that conversational AI could support teachers’ role as decision-makers
in the pre-lesson and planning process by providing diagnostic information collected in the
previous lessons (Jeon, 2021). Also, conversational AIs could assist teachers with providing guided
learning tasks (e.g., pronunciation drills) during the assimilation phase so that teachers could
spend more class time on communicative tasks during the application and extension phase
(Liakin et al., 2014; McCrocklin, 2016). In a similar vein, Holstein et al. (2019) developed an
AI-based classroom orchestration tool that supports K-12 teachers. They claimed that the tool
could help teachers to better control their classrooms and support decision-making processes
by providing real-time learning analytics data.
 Journal of Research on Technology in Education 59

Similarly, in the report based on the panel discussion on AI and learning, Roschelle et al.
(2020) suggested practical guidelines for classroom orchestration that can be used in the appli-
cation, extension, and follow-up or synthesis phases. Although the report was not tailored for
language learning, the researchers argued that future conversational AI-integrated learning envi-
ronments should focus more on social learning rather than highly individualistic learning. For
example, the report illustrated a middle school classroom lesson designed to explore the 16th
century Venice in a virtual classroom with an AI tour guide and how this guide could assist
human teachers’ group process manager, needs analyst, evaluator, and facilitator roles. It was
suggested that teachers and AI collaborate in forming groups, nurturing better conversations,
grading essays, responding to students’ emotions, and lesson planning. During this collaboration,
human teachers, who have better knowledge about students’ backgrounds and administrative
aspects, could form initial groups, while AIs help teachers by providing anticipated issues and
suggestions. Furthermore, teachers could utilize the AI’s learning analytics data about students’
sequence of behavior and emotions collected in the virtual reality simulation, which otherwise
would have been difficult to pick up.

Limitations and suggestions for future research

There are several notable limitations in the current study. First, only six search databases were
included, which left out the grey literature in the field. In addition, it only included papers
written in English. This may have limited the target language of the selected studies to be mostly
English as a second or foreign language. Given that the field is still emerging and the number
of articles has sharply increased in the last couple of years, future review studies should consider
expanding the scope of search and publication venues. Finally, the current study jointly reported
the roles of conversational AIs and teachers in different learning environments (e.g., extended
reality, virtual reality). Future research might also consider looking at how the affordances and
challenges of conversational AI-integrated learning differ in these learning environments.
The use of conversational AIs in language learning is still a nascent field of inquiry, leaving
many possible directions for future conversational AI research. First, given that five studies were
conducted in the lab environments, future research should design a study that closely resembles
the actual language classroom. Furthermore, as suggested in other human-computer interaction
studies (Chatti et al., 2020), there is a huge gap between how a study is designed by researchers
and how AI technology is used by teachers in the actual language classroom. Second, no studies
discussed how conversational AI-integrated language learning could support students’ cultural
awareness during language learning, which has been increasingly emphasized in recent decades.
In addition, only limited studies discussed how conversational AIs could be used to minimize
the labor intensity of teachers. Subsequently, as suggested by Divekar (2020), multiple teachers
and conversational AIs could produce more useful oral practices than in dyadic settings. Thus,
it would be meaningful for future studies to investigate the optimal amount of conversational
AIs and/or teachers in maximizing learning efficiency. Lastly, given that human-computer inter-
action is becoming diversified, a better theoretical framework for language teachers’ roles that
captures this dynamicity of the learning environment is needed.

Conclusion
In contrast to many emphases made by AIEd researchers on intelligence amplification and
human-computer collaboration, our review revealed that the collaboration between conversational
AI and teachers is not strongly supported by the current empirical studies. Nonetheless, researchers
in many reviewed studies were conscious of the challenges of conversational AI-integrated lan-
guage classrooms and provided insights regarding ways wherein teachers and conversational AIs
could collaborate in future studies. Roschelle et al. (2020) expected that stronger orchestration
would be required in future classrooms. Accordingly, it is imperative for language teachers to be
60 H. JI ET AL.

equipped with instructional strategies that maximize human-computer collaboration from design
and implementation to evaluation of language lessons. To support teachers, AIEd researchers
should be able to provide practical guidelines through conducting evidence-based research.

Reciprocal perspectives on AI and human intelligence in education

The focus of AI in language learning is shifting from an intelligent tutoring system to a more
adaptive and personalized system, which enables the collaboration between human teachers and
AI. However, conversational AI is still in its developing stage, and no previous review has
investigated the evolving role of human teachers in AI-enhanced teaching. Therefore, the current
review paper aimed at examining the roles of language teachers and the remaining challenges
manifested in the current AI research and suggesting critical implications for how intelligent
amplification, as discussed in Baker (2016), could be achieved in the conversational AI-incorporated
language learning. Based on our study findings, we suggest that future research examine the
use of conversational AIs that allow stronger partnership with human language teachers, reduce
human teachers’ workload, and enhance language learners’ cultural awareness and evaluate
human-computer collaboration in a research context that better aligns with the actual language
classroom. We also call for a more holistic framework that reflects the evolving role of language
teachers in AI-integrated learning environments.

Data sharing statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Disclosure statement
The author declares that this study does not present any conflict of interest.

Notes on contributors
Hyangeun Ji is Ph.D. student and research assistant in Science, Math, and Educational Technology at Temple
University. Before coming to Temple, she worked as a computer programmer and EFL teacher in South Korea.
Her research interest includes but is not limited to instructional technology, computer-assisted language learning,
online and virtual learning, and artificial intelligence in education.
Insook Han was Associate Professor in the Department of Teaching and Learning at Temple University and is
currently at Korea University. Her major research interests involve the design of learning environments, embodied
cognition, emerging technologies in classrooms, and pre-service education for technology integration.
Yujung Ko, is Assistant Professor in the Department of Platform EdTech, Division of Social Sciences at Hanyang
Cyber University. Her research interests focus on the educational use of emerging technologies, the design of
learning with technology, and technology integration of pre- and in-service teachers.

ORCID
Hyangeun Ji http://orcid.org/0000-0002-0257-074X
Insook Han http://orcid.org/0000-0002-3344-3356
Yujung Ko http://orcid.org/0000-0002-7535-5549

References
* indicates the selected studies for review
Adamopoulou, E., & Moussiades, L. (2020). Chatbots: History, technology, and applications. Machine Learning
with Applications, 2, 100006. https://doi.org/10.1016/j.mlwa.2020.100006
 Journal of Research on Technology in Education 61

* Al-Kaisi, A. N., Arkhangelskaya, A. L., & Rudenko-Morgun, O. I. (2019). Voice assistants as a training tool in
a foreign language class [Paper presentation]. INTED2019–13th International Technology, Education and
Development Conference Proceedings, Valencia. 1236–1246. https://doi.org/10.21125/inted.2019.0399
Alharbi, H. A. (2015). Improving students’ English speaking proficiency in Saudi public schools. International
Journal of Instruction, 8(1), 105–116. https://doi.org/10.12973/iji.2015.818a
Alnuzaili, E. S., & Uddin, N. (2020). Dealing with anxiety in foreign language learning classroom. Journal of
Language Teaching and Research, 11(2), 269–273. https://doi.org/10.17507/jltr.1102.15
Alrayah, H. (2018). The effectiveness of cooperative learning activities in enhancing EFL learners’ fluency. English
Language Teaching, 11(4), 21–31. https://doi.org/10.5539/elt.v11n4p21
* Ayedoun, E., Hayashi, Y., & Seta, K. (2015). A conversational agent to encourage willingness to communicate
in the context of English as a foreign language. Procedia Computer Science, 60, 1433–1442. https://doi.
org/10.1016/j.procs.2015.08.219
Baker, R. S. (2016). Stupid tutoring systems, intelligent humans. International Journal of Artificial Intelligence in
Education, 26(2), 600–614. https://doi.org/10.1007/s40593-016-0105-0
* Bao, M. (2019). Can home use of speech-enabled artificial intelligence mitigate foreign language anxiety –
Investigation of a concept. Arab World English Journal, 5, 28–40. https://doi.org/10.24093/awej/call5.3
Belur, J., Tompson, L., Thornton, A., & Simon, M. (2021). Interrater reliability in systematic review methodolo-
gy: Exploring variation in coder decision-making. Sociological Methods & Research, 50(2), 837–865. https://
doi.org/10.1177/0049124118799372
Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research
synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32(8),
827–877. https://doi.org/10.1080/09588221.2018.1535508
Blair, K., Schwartz, D. L., Biswas, G., & Leelawong, K. (2007). Pedagogical agents for learning by teaching:
Teachable agents. Educational Technology & Society, 47, 56–61.
Bushweller, K. (2020, January 7). Teachers, the robots are coming. But that’s not a bad thing. Education Week.
https://www.edweek.org/technology/teachers-the-robots-are-coming-but-thats-not-a-bad-thing/2020/01
Bryant, J., Heitz, C., Sanghvi, S., & Wagle, D. (2020). How artificial intelligence will impact K-12 teachers. McKinsey
& Company.
Brandl, K. (2007). Communicative language teaching in action: Putting principles to work. Pearson.
* Cai, M. Y., Wang, J. Y., Chen, G. D., Wang, J. H., & Yang, S. H. (2020). A digital reality theater with the mech-
anisms of real-time spoken language evaluation and interactive switching of scenario & virtual costumes: Effects on
motivation and learning performance [Paper presentation]. 2020 IEEE 20th International Conference on Advanced
Learning Technologies (ICALT), July (pp. 295–299). IEEE. https://doi.org/10.1109/ICALT49669.2020.00096
Chapelle, C. (2005). Interactionist SLA theory in CALL research. In J.L. Egbert & G.M. Petrie (Eds.), CALL
research perspectives (pp. 53–64). Lawrence Erlbaum Associates.
Chatti, M. A., Muslim, A., Guesmi, M., Richtscheid, F., Nasimi, D., Shahin, A., & Damera, R. (2020). How to
design effective learning analytics indicators? A human-centered design approach [Paper presentation]. In European
Conference on Technology Enhanced Learning, September (pp. 303–317). Springer.
Chaudhry, M. A., & Kazim, E. (2022). Artificial intelligence in education (AIEd): A high-level academic and
industry note 2021. AI and Ethics, 2(1), 157–165. https://doi.org/10.1007/s43681-021-00074-z
Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access. 8, 75264–75278.
https://doi.org/10.1109/ACCESS.2020.2988510
Chen, T. Y., & Chang, G. B. (2004). The relationship between foreign language anxiety and learning difficulties.
Foreign Language Annals, 37(2), 279–289. https://doi.org/10.1111/j.1944-9720.2004.tb02200.x
Chen, X., Zou, D., Cheng, G., & Xie, H. (2021). Artificial intelligence-assisted personalized language learning:
Systematic review and co-citation analysis [Paper presentation]. 2021 International Conference on Advanced
Learning Technologies (ICALT), July (pp. 241–245). IEEE. https://doi.org/10.1109/ICALT52272.2021.00079
Cukurova, M., Kent, C., & Luckin, R. (2019). Artificial intelligence and multimodal data in the service of human
decision‐making: A case study in debate tutoring. British Journal of Educational Technology, 50(6), 3032–3046.
https://doi.org/10.1111/bjet.12829
Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. In M. S. Khine, & I. M. Saleh
(Eds.), New science of learning (pp. 525–552). Springer Science + Business Media.
* Divekar, R. R. (2020). AI enabled foreign language immersion: Technology and method to acquire foreign lan-
guages with AI in immersive virtual worlds [Doctoral dissertation]. Rensselaer Polytechnic Institute.
Divekar, R. R., Lepp, H., Chopade, P., Albin, A., Brenner, D., Ramanarayanan, V. (2021). Conversational agents
in language education: Where they fit and their research challenges. International Conference on Human-Computer
Interaction (pp. 272–279). Springer.
* Dizon, G. (2020). Evaluating intelligent personal assistants for L2 listening and speaking development. Language
Learning & Technology, 24(1), 16–26.
* El Shazly, R. (2021). Effects of artificial intelligence on English speaking anxiety and speaking performance: A
case study. Expert Systems, 38(3), e12667. https://doi.org/10.1111/exsy.12667
Fadel, C., Holmes, W., & Bialik, M. (2019). Artificial intelligence in education: Promises and implications for
teaching and learning. The Center for Curriculum Redesign.
62 H. JI ET AL.

* Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest
in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human
Behavior, 75, 461–468. https://doi.org/10.1016/j.chb.2017.05.045
* Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences,
interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023
* Fryer, L. K., Thompson, A., Nakao, K., Howarth, M., & Gallacher, A. (2020). Supporting self-efficacy beliefs
and interest as educational inputs and outcomes: Framing AI and human partnered task experiences. Learning
and Individual Differences, 80, 101850. https://doi.org/10.1016/j.lindif.2020.101850
Gregersen, T., Macintyre, P. D., & Meza, M. D. (2014). The motion of emotion: Idiodynamic case studies of learn-
ers’ foreign language anxiety. The Modern Language Journal, 98(2), 574–588. https://doi.org/10.1111/modl.12084
Hapsari, I. P., Wu, T. T. (2022). AI chatbots learning model in English speaking skill: Alleviating speaking anx-
iety, boosting enjoyment, and fostering critical rhinking. International Conference on Innovative Technologies
and Learning (pp. 444–453). Springer.
* Hassani, K., Nahvi, A., & Ahmadi, A. (2016). Design and implementation of an intelligent virtual environment
for improving speaking and listening skills. Interactive Learning Environments, 24(1), 252–271. https://doi.org
/10.1080/10494820.2013.846265
Hedayati, M., Reynolds, B., & Bown, A. (2017). Investigating CALL teachers’ contextualised views of their evolving
roles [Paper presentation]. In Proceedings of the 10th International Conference ICT for Language Learning,
October) (pp. 184–188). Libreria Universitaria.
Holstein, K., Aleven, V., Rummel, N. (2020). A conceptual framework for human–AI hybrid adaptivity in edu-
cation. In International Conference on Artificial Intelligence in Education (pp. 240–254). Springer.
Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support
teacher–AI complementarity. Journal of Learning Analytics, 6(2), 27–52. https://doi.org/10.18608/jla.2019.62.3
Horwitz, E. (2001). Language anxiety and achievement. Annual Review of Applied Linguistics, 21, 112–126. https://
doi.org/10.1017/S0267190501000071
* Hsu, H. L., Chen, H. H. J., & Todd, A. G. (2021a). Investigating the impact of the Amazon Alexa on the
development of L2 listening and speaking skills. Interactive Learning Environments, 1–14. https://doi.org/10.1
080/10494820.2021.2016864
* Hsu, M. H., Chen, P. S., & Yu, C. S. (2021b). Proposing a task-oriented chatbot system for EFL learners
speaking practice. Interactive Learning Environments, 1–12. https://doi.org/10.1080/10494820.2021.1960864
Huang, W., Hew, K. F., & Fryer, L. K. (2021). Chatbots for language learning—Are they really useful? A system-
atic review of chatbot‐supported language learning. Journal of Computer Assisted Learning, 38(1), 1–21.
Istrate, A. M. (2018). Artificial intelligence and machine learning—Future trends in teaching ESL and ESP.
eLearning & Software for Education, 2, 471–476.
* Jeon, J. (2021). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis.
Computer Assisted Language Learning, 1–27. https://doi.org/10.1080/09588221.2021.1987272
Jo, S. (2008). English education and teacher education in South Korea. Journal of Education for Teaching, 34(4),
371–381. https://doi.org/10.1080/02607470802401594
* Kılıçkaya, F. (2020). Using a chatbot, Replika, to practice writing through conversations in L2 English: A Case
study. In New Technological Applications for Foreign and Second Language Learning and Teaching (pp. 221–238).
IGI Global.
Kim, J., Lee, H., & Cho, Y. H. (2022). Learning design to support student-AI collaboration: Perspectives of
leading teachers for AI in education. Education and Information Technologies, 27, 6069–6104. https://doi.
org/10.1007/s10639-021-10831-6
* Kim, N. Y. (2016). Effects of voice chat on EFL learners’ speaking ability according to proficiency levels.
Multimedia-Assisted Language Learning, 19(4), 63–88.
Kinginger, C. (2001). Sociocultural approaches to teaching and teachers’ research. In R. Z. Lavine (Ed.), Beyond
the boundaries: Changing contexts in language learning (pp. 201–226). McGraw-Hill and Northeast Conference
Reports.
Kukulska-Hulme, A., & Lee, H. (2020). Intelligent assistants in language learning: An analysis of features and
limitations. In K.-M. Frederiksen, S. Larsen, L. Bradley & S. Thouësny (Eds), CALL for widening participation:
Short papers from EUROCALL 2020 (pp. 172–176). Research-publishing.net.
Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: The Betty’s Brain system. International
Journal of Artificial Intelligence in Education, 18(3), 181–208.
Li, S. (2018). Corrective feedback in L2 speech production. The TESOL Encyclopedia of English Language Teaching,
1–9.
* Liakin, D., Cardoso, W., & Liakina, N. (2014). Learning L2 pronunciation with a mobile speech recognizer:
French/y/. CALICO Journal, 32(1), 1–25. https://doi.org/10.1558/cj.v32i1.25962
Liang, J. C., Hwang, G. J., Chen, M. R. A., & Darmawansah, D. (2021). Roles and research foci of artificial
intelligence in language education: An integrated bibliographic analysis and systematic review approach.
Interactive Learning Environments, 1–27.
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux,
P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses
 Journal of Research on Technology in Education 63

of studies that evaluate health care interventions: Explanation and elaboration. Journal of Clinical Epidemiology,
62(10), e1–e34.
* Lin, C. J., & Mubarok, H. (2021). Learning analytics for investigating the mind map-guided AI chatbot approach
in an EFL flipped speaking classroom. Educational Technology & Society, 24(4), 16–35.
Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. C. Ritchie &
T. K. Bhatia (eds.), Handbook of research on language acquisition (pp. 413–468). Academic Press.
Luan, H., & Tsai, C. C. (2021). A review of using machine learning approaches for precision education. Educational
Technology & Society, 24(1), 250–266.
Mayring, P. (2000). Qualitative content analysis. In U. Flick, E. Kardoff, & I. Steinke (Eds.), A companion to
qualitative research (pp. 266–269). Sage.
* McCrocklin, S. M. (2016). Pronunciation learner autonomy: The potential of automatic speech recognition.
System, 57, 25–42. https://doi.org/10.1016/j.system.2015.12.013
Minsky, M. L., & Papert, S. A. (1968). Perceptrons: An introduction to computational geometry. MIT Press.
* Moussalli, S., & Cardoso, W. (2016). Are commercial ‘personal robots’ ready for language learning? Focus on
second language speech. In S. Papadima-Sophocleous, L. Bradley, & S. Thouesny (Eds.), CALL communities
and culture – short papers from EUROCALL 2016 (pp. 325–329). Research-publishing.net.
Muhammad, A. F., Susanto, D., Alimudin, A., Adila, F., Assidiqi, M. H., & Nabhan, S. (2020). Developing English
conversation chatbot using Dialogflow [Paper presentation]. 2020 International Electronics Symposium (IES),
September (pp. 468–475). IEEE. https://doi.org/10.1109/IES50839.2020.9231659
Pan, Y. (2016). Heading toward artificial intelligence 2.0. Engineering, 2(4), 409–413. https://doi.org/10.1016/J.
ENG.2016.04.018
Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A sys-
tematic literature review. Computer Applications in Engineering Education, 28(6), 1549–1565. https://doi.
org/10.1002/cae.22326
Roschelle, J., Dimitriadis, Y., & Hoppe, U. (2013). Classroom Orchestration: Synthesis. Computers & Education,
69, 523–526.
Roschelle, J., Lester, J., & Fusco, J. (2020). AI and the future of learning: Expert panel report. Digital Promise.
* Ruan, S. S. (2021). Smart tutoring through conversational interfaces [Doctoral dissertation]. Stanford University.
Shao, K., Pekrun, R., & Nicholson, L. J. (2019). Emotions in classroom language learning: What can we learn
from achievement emotion research? System, 86, 102121. https://doi.org/10.1016/j.system.2019.102121
Silvervarg, A., Wolf, R., Blair, K. P., Haake, M., & Gulz, A. (2021). How teachable agents influence students’
responses to critical constructive feedback. Journal of Research on Technology in Education, 53(1), 67–88. https://
doi.org/10.1080/15391523.2020.1784812
Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook
Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
* Tai, T. Y., & Chen, H. H. J. (2020). The impact of Google Assistant on adolescent EFL learners’ willingness to
communicate. Interactive Learning Environments, 1–18. https://doi.org/10.1080/10494820.2020.1841801
Teimouri, Y., Goetze, J., & Plonsky, L. (2019). Second language anxiety and achievement: A meta-analysis. Studies
in Second Language Acquisition, 41(2), 363–387. https://doi.org/10.1017/S0272263118000311
Terhune, N. M. (2016). Language learning going global: Linking teachers and learners via commercial Skype-based
CMC. Computer Assisted Language Learning, 29(6), 1071–1089. https://doi.org/10.1080/09588221.2015.1061020
* Thompson, A., Gallacher, A., Howarth, M. (2018). Stimulating task interest: Human partners or chatbots?.
Proceedings of the Future-Proof CALL: Language Learning as Exploration and Encounters, 302–306.
Turing, A. M. (1950). Computing machinery and intelligence. Mind, LIX(236), 433–460. https://doi.org/10.1093/
mind/LIX.236.433
Wang, Y. (2021). When artificial intelligence meets educational leaders’ data-informed decision-making: A cau-
tionary tale. Studies in Educational Evaluation, 69, 100872. https://doi.org/10.1016/j.stueduc.2020.100872
* Yin, Q., & Satar, M. (2020). English as a foreign language learner interactions with chatbots: Negotiation for
meaning. International Online Journal of Education and Teaching, 7(2), 390–410.
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial
intelligence applications in higher education–Where are the educators? International Journal of Educational
Technology in Higher Education, 16(1), 1–27. https://doi.org/10.1186/s41239-019-0171-0
Zeira, J. (1998). Workers, machines, and economic growth. The Quarterly Journal of Economics, 113(4), 1091–1117.
https://doi.org/10.1162/003355398555847
Zhang, R., & Zou, D. (2020). Types, purposes, and effectiveness of state-of-the-art technologies for second and
foreign language learning. Computer Assisted Language Learning, 35(4), 696–742.
Zheng, G. (2020). The optimization and application of blended teaching based on artificial intelligence [Paper
presentation]. 2020 3rd International Conference on Advanced Electronic Materials, Computers and Software
Engineering (AEMCSE), April (pp. 71–74). IEEE. https://doi.org/10.1109/AEMCSE50948.2020.00023