The battle of language learning apps:

a cross-platform overview
Athanasios Karasimos
akarasimos@enl.auth.gr| akarasimos@gmail.com
Centre for Greek Language | Aristotle University of Thessaloniki

Abstract
Learning foreign languages has become a very appealing and important ability in the
contemporary world. Technology, nowadays, permeates all aspects of people’s lives, and it
has consequently also reached the domain of education. Oftentimes, learning a foreign
language involves using language learning platforms (LLPs). There is an abundance of
language learning apps available, but there are very few studies cross-investigating their
effectiveness in language learning. This paper presents a comparative research of the five
top-rated learning apps, such as Duolingo, Memrise, LingQ, Busuu and Rosetta stone. In this
comprehensive cross-evaluation, several elements and features of these apps are examined
and their strengths and advantages are underlines, while addressing certain limitations and
comments about their overall performance.

Keywords: language learning platforms, asynchronous education, distance learning,

Duolingo, Memrise, Rosetta stone, LingQ, Busuu

1. Introduction
Mobile-assisted language learning (MALL) is one of the most important recent developments
in the dominant field of educational technology and computer-assisted language learning.
Building on smartphones and tablets, a plethora of learning apps have been developed,
designed and are widely available in almost all operating systems and mobile platforms. The
rapidly growth of these apps and the increasing accessibility of the apps is underlying the
necessity to build a sufficient set with selection criteria. In the past decade, the smartphones
have become dominant and ubiquitous; mobile applications with huge variety of topics and
categories have an impact factor on the daily life for entertainment, business and
educational purposes, since they offer an enormous pool of resources. An increasing amount
of study emanate the potential of language learning apps to learn, practise, and enhance all
four language skills, grammar, vocabulary and overall knowledge of the regions where target
languages are spoken (Rosell-Aguilar, 2009; 2018).

This paper explores five language learning apps (Duolingo, LingQ, Busuu, Memrise, Rosetta
Stone) that have the highest review score in Google Play/ Apple store and provide a cross-
evaluation of them. It introduces a framework of selection criteria for language learning
apps, which guides language learners to critically evaluate linguistic and technical aspects of
language learning apps and choose more easily. The chapter also presents a short literature
review of theory behind autonomous and distance learning with a short recap of studies
analysing each of the aforementioned apps. Finally, the cross-evaluation data from a
questionnaire and short interviews are presented and discussed.

2. The theoretical background: re-defining autonomous learning

2.1. Towards a theoretical foundation

Learning in the 21st century has developed beyond physical and traditional borders. Learners
have access to knowledge since technology and web 3.0 (nowadays we are in dawn of web
4.0) is present and dominant in the learning procedure. The Zgen (and not only) learners
urged by the pandemic impact and some additional reasons choose distance, autonomous
and asynchronous learning rather than the face-to-face teaching situations. King et al. (2001)
provided separate definitions for the terms distance learning and education since they
defined “distance learning” as the activity of learning at a distance and not be an alternative
version of “education”. Additionally, place, medium and time are the major limitations which
are associated with the term distance (Guilar & Loring, 2008).

Asynchronous education and close-end apps and software are achieved regardless the place
and time of attendance. The learners may attend courses, participate, and finish their tasks
and activities according to their own tempo and schedule. On the contrary, synchronous
education is immediate, unpersonalized, real-time and easier for the learners to interact
with their instructors or co-learners. Asynchronous education is based on three significant
characteristics, such as flexibility, availability and autonomy. It is a balanced combination of
three dominant and well-known theories: (a.) Theory of independence and autonomy
(Wedemayer, 1981), (b.) Theory of industrialization (Peters, 1988) and (c.) Theory of
interaction and communication (Holmberg, 1986).

Despite the enormous tech evolution of distance education platforms, apps and services, the
theory behind-the-scenes of distance learning remains chaotic, uncharted and confused. As
Moore (1997) opposes, there is no national policy, nor anything approaching a consensus
among educators of the value, a commonly accepted definition, a concrete or broad
methodology or even the concept of distance education. Since Shale’s (1990) last call for
theoreticians and practitioners to stop emphasizing points of difference between distance
and traditional education, there was not much effort to identify common educational
problems and to build a theoretical framework to support all these apps, tools, services and
features.

2.2. Theoretical constructs of distance and autonomous learning

Mapping the theoretical constructs of distance learning into a language learning platform
framework, it fulfills the potential gaps of its theory. The four pillars of distance learning
ensure a well-known and properly documented foreground to build a theory. Transactional
distance, interaction, control and social context construct a non-chaotic and clear
perspective for learning apps. More specific, transactional distance encompasses the
existential distance between all educational relationships. Such a distance is determined by
the amount of dialogue that occurs between the learner and the educator, and the amount
of structure that exists in the design of the course. Moreover, the interaction between
educator and learner is that construct component that provides feedback, motivation, and
dialogue between them. Additionally, learner-content interaction is the method by which
students obtain intellectual information from the material, and finally, interaction between
learners is the exchange of information, ideas, and discussion between learners about the
course, whether this is structured or non-structured. With control, students relate that their
academic success is a result of their own personal accomplishments; therefore, they are
more likely to persist in their education. Students with an external locus of control feel that
their success, or lack of it, is due largely to events such as luck or fate outside their
control. Finally, a crucial question is how the social environment affects motivation,
attitudes, teaching, and learning. The guidelines always demand technology to be culturally
and religiously neutral, to be easily used in a variety of settings.

2.3. Identified essential elements of autonomous learning and language

learning apps
The core notion behind autonomous learning is structured with all the necessary and
essential elements to create and support the independent learners. Furthermore, distance
learning promotes a greater student responsibility and a widely available instruction to cover
each user’s needs, preferences and different learning pattern and behavior. For instance, the
framework of RASE (Resources, Activity, Support, and Evaluation) learning design (Churchill,
King, & Fox, 2013) is a blended learning theory with the aforementioned significant elements
of autonomous learning focused on writing skill. Students can access the resources on the
content management systems, such Edmodo, Canvas, Schoology, Moodle or Open class
(Rakhmawati, 2020). Moreover, effective mix of media and methods under the umbrella of
blended learning offers adaptation to individual differences and a wide variety of start, stop,
and learning times.

MALL has been promoted recently to cut down the barriers on language teaching and
learning. MALL can also enable students become more active, autonomous and collaborative
because they contribute to their own learning path. Rewinding to Wedemeyer’s (1981)
theory, the designers of language learning apps should reconsider the theory behind-the-
scenes of their code. Meaningful learning and new learning anchored in the cognitive
structure (not rote learning) are key elements to boost the effectiveness of these apps.
Several research (Citrayasa, 2019; Malerba, 2015; Munday, 2017; Psychogyiou & Karasimos,
2019; Stockwell & Hubbard, 2013) present data from users who request apps centered on
interest, with individualization of teaching and learning and to support encouragement of
critical thinking. Unfortunately, the astounding growth of new language learning apps (more
than 70) glamorized by technology amplifies the poor conceptual framework and the usual
absence of identified essential elements and the 4-Square Map of distance education
technology options (Stock McIsaac & Gunawardena, 2004)

3. Language learning platforms: Learning a language from distance

3.1. Previous research on the use of apps for language learning

Several research studies into the use of language learning platforms and apps have
presented positive results on language skill improvement, learner motivation and
engagement; nevertheless, as Rosell-Aguilar (2018) points out, the researchers have mostly
focused on rather small subject groups using pre-selected apps or platform rather than on
the choices made by the language users.

The major impact factor of using language learning apps of many studies (Castaneda & Cho,
2016; Kim, 2013; Morgana, 2015; Steel, 2012) is the significant and prominent
improvements in all language skills, grammar and vocabulary, and even in more advanced
linguistic abilities. The effectiveness of LLPs has also been found comparable to face-to-face
teaching at several CEFR level students. Rachels and Rockinson-Szapkiw (2017) found that
there were no significant differences between the Spanish subjects that learned a language
via Duolingo and the other group that participated with a traditional teaching approach.
Similarly, Psychogyiou and Karasimos (2019) support that the vast majority of the
participants was confident about the result, although there were some contrasting views
which cause a debate about the effectiveness of the Duolingo app. However, the users were
pleased with learning new vocabulary and pronunciation compared to the typical
coursebook approach.

The language learning apps not only promote autonomous language learning, but
additionally they offer an alternative reality to the educators and students that want to use
these apps within a classroom context. Chen and Kessler (2013) maintain that smartphones
and tablets may enhance learner autonomy and ubiquitous learning in informal settings.
Unfortunately, only a few studies (Steel, 2012; Stockwell & Hubbard, 2013) present results
from LLPs and MALL activities in a face-to-face classroom compared to how the users engage
in mobile-assisted language learning outside-of-the-box (a traditional or even a techy
classroom). Moreover, Stockwell and Hubbard (2013) study the learner’s use of LLPS outside
the default and designer’s guide and suggestions, since regularly the learners’ personalised
use of apps may be deviated from what the best practices or ideal usage of each application
is considered. In Mason and Zhang’s (2017) study looking at mobile app use with 140
learners of Chinese, found that 94% of learners utilised apps independently and the majority
of them recognised the value of apps to support their language learning, but learners only
used a proportion of the functionalities available (Rosell-Aguilar, 2018, p. 6).

For an extensive presentation of previous research on the use of apps for language learning,
check Rosell-Aguilar (2018) and Goodwin-Jones (2011) for early language mobile apps.

3.2. Selection criteria for language learning platforms

The on-going and impressive battle between the learning language apps has raised the need
to provide and categorize some concrete and straightforward selection criteria for LLPs. On
the other hands, the users reviews and the learners’ community feedback generated enough
arguments for the researchers to modify and shape a best practices guide for the LLPs
designers and developers.

Whilst it is arguable whether apps can at this point be considered as a single solution to
language learning, they can effectively support learner autonomy and interest in learning a
language (Goodwin-Jones, 2011). Apps can provide a good supplement for language learners
who are enrolled in formal instruction as well as a good starting point for beginner
independent learners. They can also provide regular practice for language learners who are
no longer formally studying a language but wish to keep practising it (Rosell-Aguilar, 2017;
2018). The number of language learning apps grows rapidly the last decade and the
increasing accessibility of the apps generates the necessity for the appropriate selection of
the apps. This session provides several evaluation systems of language learning apps.

Heil, Wu and Lee (2016) using the app analytics engine App Annie identify and compile a list
of the top-50 apps in both Google and Apple stores (first trimester 2015). App Annie collects
information from users and uses it to estimate rankings of apps. They analyse findings which
provide a concrete picture of available language-learning apps (the last 5-years). Their
criteria are languages and platforms supported, monetization, user input, assessment and
instructional focus, implicit and explicit grammar instruction, corrective feedback, and user
interaction - listening, reading and writing. Their study points out that is a common pattern
on teaching language as isolated basic and advanced vocabulary words via LLPs rather than
contextualized usage. Heil, Wu and Lee (2016) observe that some drill-type techniques offer
very little explanatory corrective feedback, and there is little adaptation to the needs of
individual learners. Despite advances in language teaching that have stressed the
importance of communicative competence in language learning, MALL technology is still
primarily utilized for vocabulary instruction rather than fluency-building.

Additionally, Son (2016) introduces a language learning app review form, which guides
language teachers to critically evaluate pedagogical and technical aspects of language
learning apps. His evaluation form contains 15 selection and reviewing criteria: purpose,
accuracy, usefulness, flexibility, authenticity, engagement, feedback, integration, support,
price, reliability, presentation, organization, navigation and multimedia.

As Smith and Ragan (2004) pointed out examining the content, task, and context of the
instructional materials or programs was important. Thus, evaluating the quality of the
content provided by apps is important. Given that the apps are for used for learning, it is
vital to assess the pedagogy coherence of language skills within the learning activities. Apps
are software installed on mobile devices, thus evaluating the usability, customization, and
sharing options provided by such app is valuable. In summary, seven elements are identified
to evaluate language-learning mobile apps: feedback and self-correction, customization,
pedagogical coherence on language skills, content quality, usability, motivation, and sharing
(Chen, 2016).

Despite the perplexity or simplicity of a straightforward, well-organized and clear selection

grid, at the end of the day choosing the right language-learning software is a highly personal
decision and strongly related to each user’s needs and requirements. Bulatovych (2017)
suggests creating an app for effective language learning by making learning easy and
engaging for beginners and challenging advanced learners. She comments that several LLPs
take a more serious approach to vocabulary teaching, giving workloads that only advanced
or intermediate learners can cope with; therefore, these apps usually offer complex and
challenging quizzes with an emphasis on reading and memorizing words and phrases. Finally,
she filtered plenty app reviews to propose specific categories/ criteria, such as personalized
approaches to learning vocabulary and grammar, listening comprehension, pronunciation
checks, language practice, training and checking progress, motivation and support of
learners.

The user wants a program that's right for his/her language education level, whether he/she
is a total beginner or an advanced learner in need of some brushing up or vocabulary
refreshing. The user also has to make sure that the language he/she wants to study is
available, well-supported and not in early-access version with issues and bugs. What about
allowing you to study while driving? Almost every user’s life is becoming ever faster and this
situation enables users to take advantage of an app on the fly. Therefore, the option to swift
from active to passive learning procedure due to some situations (driving, travelling,
working, writing, etc.) is crucial.

To help a learner choose the proper language learning app, it is suggested a list of self-
questions on how to use an engaging language learning app. Do you need to acquire the
basic grammar or just be able to speak some specific phrases for entertainment or travel? Is
the program you want in your price range? Does the learner need virtual tutoring? Does the
user need to master core grammar or just be able to speak some essential phrases for
travel? What about making sure he/she can study without a teacher? Flipped classrooms,
problem-based learning, gamified activities and tasks and other technology supported
learning approaches are on the top of the users requirement list when they want to choose
carefully an elegant and efficient app. Towards these goals and requests, researchers and
designers try to combine successfully different theoretical frameworks to enhance the
learning outcomes. Assuredly additional principles should be identified and added to the
Basic Principles List.

The different perspectives of learners and designers are mainly focused on intentions and
experiences (there is a significant gap between them based Stracke, et al., (2018)’s and Hei,
Wu and Lee (2016)’s research). For this reason, we try to build a concrete and compact
proposal to overcome this gap and fulfill their needs. Our selection and evaluation model,
Framework of Selection and Evaluation Criteria for language learning apps, F-SEC) is mostly
based on Heil, Wu and Lee (2016), but expanded and modified to cover a broader area of
language learning apps and their features, both from selection point of view and from
evaluation perspective.

Framework of Selection and Evaluation Criteria for language learning apps (F-SEC)
Selection and evaluation criteria Values and guidelines
Languages Number and variety of language courses (including
beta courses and constructed language courses)
Platforms Operating systems: Android, iOS, Windows, Linux
Registration and subscription Free registration, trial version or premium account
(monthly cost or lifelong cost)
User interaction Input options, such writing on keyboard/ touchboard,
speaking into mic, typical or advanced use of touch
screen, using of mouse
Support and updates Frequency of updates, technical support, adding new
features, bug and fixes
Vocabulary teaching and assessment vocabulary in isolation (with flashcards, video
selection, picture pairing/ matching), vocabulary in
context (inside monologue, dialogue or a story),
supporting dictionary/lexicon
 Grammar teaching and assessment Autonomous grammar section, grammar in isolation
(examples, tables), grammar in context, absence of
grammar instruction
Reading, Writing, Listening and Speaking (+ Reading texts and comprehension tasks, word
Pronunciation) formation processes, sentence structure building,
listening phrases or passages and comprehension or
transcription tasks, dialogues or speaking interaction
Corrective Feedback Sound effects, visual feedback, textual corrections,
textual explanations, no feedback
Gamification Gamification features (experience points, levels,
leaderboards, badges, challenges, etc.), motivation
messages (emails, messages, pop-up notifications),
gamification absence
Placement test Placement test for language levelling, progress tests,
no tests
Community engagement Connect with social media friends, connect with other
app users, forum, bulletin lists

4. Choosing the proper language learning app

4.1. The Duolingo application

Duolingo exists since 2009 and was nominated as iPhone App of the Year 2013 and Google’s
Best of the Best app 2014; it offers 68 different language courses across 23 languages with
more than 300 million registered active users. Duolingo provides written lessons, dictation,
an interactive dictionary (plus Stories and Events for several languages). It has a gamified
skill tree that users can progress through and a vocabulary section where learned words can
be practiced (Huynh & Iida, 2017; Psychogyiou & Karasimos, 2019; Rachels & Rockinson-
Szapkiw, 2018).

Duolingo enhances the learning experience by increasing access to the target language and
allowing learners to work at their own time, level and pace, thus making the task of learning
more exciting and personalized (Nushi & Eqbali, 2017). Moreover, Munday (2017) studies
how the translation-based platform offers different possibilities for Spanish learners, while
its gamified features make it fun for students. Rachels and Rockinson-Szapkiw (2018)
provide evidence that students learning a language with a free app like Duolingo, which
applies gamification principles, exhibit equivalent learning outcomes with students being
taught in a traditional face-to-face setting; their research indicates that Duolingo can have a
substantial impact on foreign language learning. Finardi, Leão and Amorim’s (2016) study
present that Duolingo may engage and aid users in foreign language learning, but up to a
certain extent. That is, the participants perceive that the presence of a teacher is still
necessary to guide them in their learning process, and to foster interaction and production
skills. When it comes to teaching and learning with technology, the need for teacher
presence also depends on the age of the learner(s) and personal needs.
Picture 1: Samples from Duolingo showing a Stories scenario (listening mode) and the hierarchical skill tree of a
language course (German)

4.2. Busuu
Busuu is language learning app with an exponential increasement of its popularity with over
100 million people choose it to learn Spanish, pick up a few French phrases, try Japanese, or
one of the other 12 different supported languages. Several studies (Nushi & Jenabzadeh,
2016; Shibata, 2020; Winans, 2019; Citrayasa, 2019; Malerba, 2015; Saona-Vallejos, 2018;
Rosell-Aguilar, Autonomous language learning through a mobile application: a user
evaluation of the busuu app, 2018) have been conducted about its efficiency, usefulness and
impact for almost all language skills. Busuu provides well-crafted learning tools such as flash-
cards, conversation models, correction by native speakers, and grammar extracts for its
users plus the community help center. Busuu offers exercises for the four major language
skills through its concentrated and well-formed exercises. However, its huge downside of
the free version is the small variety of exercises that it offers, which gets boring after a while
(Nushi & Jenabzadeh, 2016).

Shibata’s (2020, p. 202) review underlines that “most of the activities in Busuu, however, still
seem to be based on the theoretical perspectives of behaviourism due to the noticeable
number of audiolingual activities and repetitive mechanical drills” and suggests the need to
add some opportunities to collaborate and discuss senses with other users to boost the its
effectiveness. On the other hand, Winans (2019) supports that interactivity and social
connection with native speakers of the target language can be motivating, intriguing and
challenging for users since they provide real feedback about daily and realistic use of each
language. Results of Malerba’s (2015) study revealed that learners with an autonomous
attitude tended to have a more effective learning experience, to find interesting topics and
to be able to combine social and pedagogical paths even if it is not properly supported.
Additionally, most of the users have positive feedback on their counter to a community of
practice where learners can share goals and be engaged in continuous collaborative and
meaningful activity (Saona-Vallejos, 2018). Finally, it is impressive the fact that the longer
learners have been using the Busuu, the more likely they are to pay for premium content
(Rosell-Aguilar, 2018).
Picture 2: Samples from Busuu showing a vocab revision and a typical tip during a new lesson session.

4.3. Memrise
Memrise is a language learning app founded in 2010; it has quickly grown and now more
than 50 million people and contains Rich, real-life language content. Memrise’s courses have
real-life language; to add to the richness, they are packed with thousands of video clips of
native speakers speaking in their native language. Several studies ( (Widyaningrum & Putro,
2019; Zhang, 2019; Nuralisah & Kareviati, 2020; Cedercreutz & Ropicki, 2018; Reknasari &
Putro, 2019; Karjo & Andreani, 2018)

Nuralisah and Kareviati (2020) maintain that vocabulary learning using Memrise is effective
for improving students' language skills. The advantage of using this application is to make
students motivated and happier in learning English and make it easier for students to
memorize words. This can also increase the sense of cooperation with each other.
Additionally, Zhang (2019, p. 161) concludes that “although adopting the audiolingual
method in nature, Memrise has the strength of promoting learners’ memorization of words
and phrases. However, the lack of contextual elements, of oral production, and of the
capability to help students carry out authentic interactive conversation in real-life scenarios
limits Memrise’s ability to function as the sole source for second language learning”.
According to Reknasari and Putro (2019), Memrise offers comprehensive vocabulary learning
through reviewing in order to reinforce the students’ word retention. Finally, Karjo and
Andreani (2018) believe that the reviewing of the previously learned words before learning
new words, it becomes annoying and frustrating very soon.
Picture 3: Samples from Memrise showing the overall overview of language progress and a vocabulary task.

4.4. LingQ
LingQ was launched in late 2007 by Steve and Mark to develop better tools for language
learners. The basis of the LingQ system enables thousands of learners from all over the
world to enjoy language learning and successfully reach their goals. LingQ is a fun, simple
way to learn languages from authentic, interesting content; learning the language from the
language itself, just as most children learn their own language. Along with over 1,000,000
LingQ member, LingQ offers 42 language courses. Winda (2019) notes that LingQ provides
some listening activities to enhance more vocabulary remembering instead of improving the
skill itself. Despite the active community and the 14-years of existence, there is a significant
lack of publications and reviews about LingQ.

Picture 4: Samples from LingQ showing a flashcard exercise and the overview of the lessons.

4.5. Rosetta stone

Rosetta Stone’s Dynamic Immersion method uses interactive and contextual language
lessons blended with Extended Learning features. With a variety of speaking-focused lessons
and features, instantaneous pronunciation feedback with TruAccent, a track record of
getting people speaking any language confidently and a near-five star rating in the app store,
the award-winning Rosetta Stone mobile app (2019 PCMag Editors’ Choice, 2019 Tabby
Awards Winner, 2019 Best Mobile App Awards: Best Designed App and Best Overall App) is
an excellent way to learn and speak new languages with subscription.

Lafford, Lafford and Sykes (2007) evaluates Rosetta Stone among other LLPs, based on a
features group of previous SLA research that has been proved quite significant and relevant
in the acquisition process, such as the relevant contextualization of language, opportunities
for interaction, etc. Santos’ (2011) review assesses the Rosetta Stone app by noting its fairly
significant lack of context in the materials and an inability to respond to spontaneous
student speech. Santos sums up that Rosetta Stone’s interaction is a quite poor and
downgrade version of real-life conversations and speech encounters. Moreover, DeWaard
(2013) explores the use of Rosetta Stone as a replacement of classroom instruction and
points out the interface of this app is appealing but lacking in several areas. Specifically,
DeWaard (2013) stresses out the failure of cultural authenticity and its major limitations of a
flawed machine learning system. DeWaard (2013, p. 61) concludes that Rosetta Stone is “not
a viable replacement of current instruction at the postsecondary level”. Finally, the major
study by Lord (2015; 2016) presents data that confirm Rosetta Stone’s marketing claims of
being superior to in-class learning, in spite of learners’ appreciation for the flexibility and
usability of the program. Actually, the learner outcomes indicate that this app does not seem
as adept at helping learners develop crucial communicative strategies in the foreign
language, it is capable of presenting isolated, decontextualized language elements.

Picture 5: Samples from Rosetta Stone showing a common listening/vocabulary task and expore-all-content
overview of a course

5. The effective of learning a language: the comparison mode

5.1. Participants and research tools

To cross evaluate these five apps with the highest user’s score in Google Play and Apple
store, there are ten (10) participants agreed to test each apps for 4 to 6 hours and then give
their opinion about their experience with these applications. They are native Greek
speakers, and they belong to the C1-C2 level of English language proficiency. The
participants are 32-44 years old, 6 men and 4 women, and they come from a variety of
professional backgrounds and they have at least a BA diploma. Some participants have
already used a LLP before and they all tested the same language in all the apps, which was
German or Italian. The participants are asked to complete a post-questionnaire to evaluate
each app and all together based on the aforementioned criteria (see 3.2).

An online survey via Google Forms was developed based on other studies into use of apps
for language learning (Karjo & Andreani, 2018; Psychogyiou & Karasimos, 2019; Rosell-
Aguilar, 2017; 2018; Saona-Vallejos, 2018) but tailored to the specific features of these five
LLPs. The survey consists of 15 items, most of which were multiple choice (more than one-
option available) with two open follow-up questions asking participants to write some
advantages or disadvantages of these apps. Moreover, there was some short open-ended
interviews (via Google Meet) with each participant to provide further details depending on
their replies and to justify their final winner language learning app. The responses to open
questions were categorised following the thematic analysis process suggested by Braun and
Clarke (Brown & Clarke, 2006).

5.2. Compared data analysis and discussion

Most of the participants (70%) have already used a language learning app to get familiar
with at least one language; a further 20% were using an app to learn or brush up three
languages usually during a travel preparation (60%) or for business purposes (50%). Only a
minority of the participants (20%) used a LLP that wasn’t in the top-10 of user’s choices;
however, they will reconsider their options based on their testing period of the five selected
learning apps. All of them considered that a daily use for a week is a sufficient time to review
an app and decide to keep up with it or drop it.

Based on their evaluation choices for the languages skills and linguistics items, Rosetta stone
received the highest (mean) score of these apps (69%) with a very short distance (65%) for
Duolingo. For vocabulary acquisition, Duolingo and Busuu provide a greater variety of new
words (80%) by building a more wide-category group of words comparing to other apps.
Memrise was described as the most sufficient (80%) to cover grammatical phenomena, to
teach the rules and to provide some straightforward guidelines of morphosyntactic features.
Listening activities are not the main focus for most apps; nevertheless, Rosetta stone (100%)
and Memrise (90%) have embedded interesting and challenging listening tasks 1 that satisfy
the users for the listening training.

Two weak points of the language learning apps are the speaking and writing activities.
Nevertheless, the live session booking of Rosetta stone with native speakers offers to the
respondents (80%) a good practice and a sufficient e-nvironment for simulating real-life
speaking conditions; moreover, it seems that Busuu and Rosetta stone can handle at least
mediocre the needs of participant to improve their writing skills. Very few apps offer the
option for extensive writing, such as Schoology, iWrite Words and iTrace, since the machine
language learning algorithms cannot provide yet proper feedback and some basic evaluation
comments. Unfortunately, a decent quality of automated feedback for writing activities is
limited to correctly translate or transcribe a sentence.

1
The transformation of Duolingo Stories (as a new feature) from a reading activity into also an
extensive listening activity will change its percentage.
Two linguistic properties that are quite new in the learning apps are pronunciation and
dictionary. For pronunciation, Duolingo supports a built-in speech recognition agent that
fulfills the participants’ (80%) demands, although Rosetta stone provides a different level of
acceptance for each pronunciation (green, orange and red indicator) and technically, it is a
more advance recognition system compared to the rest. Finally, the accompanied
dictionaries of Rosetta stone (90%), LingQ (80%) and Memrise (80%) includes multi-level
hierarchical information for the lexemes providing definitions, pronunciation, examples and
usually a scale of word-meaning acquisition. Specially, Rosetta stone creates subcategories
of the lexemes not only related to specific lesson and thematic sessions, but also to how
exposed is the learner to these words and how recently and often are repeated.

Language learning apps and language skills

Pronuncation

Dictionary

Writing

Reading

Speaking

Listening

Grammar

Vocabulary

0 10 20 30 40 50 60 70 80 90 100

Rosetta Stone Memrise LingQ Busuu Duolingo

Respondents were provided with a number of questions about the design choices, the
interface and what else they would like the app to provide. They were allowed to select all
the choices they considered relevant and point out their favorite likes and dislikes for
specific apps. Despite the massively growth of registered users, most LLPs do not have a
functional interface that satisfies and intrigues the users, since it is the first impression of an
app when it is opened by them. Duolingo verifies its rumors that its interface is functional,
light, user-friendly and fast. Almost all participants (90%) believe that this is the best way to
build and design a learning app. Moreover, gamification is a dominant essential element of
Duolingo. Its strategic attempt to enhance language learning and activities in order to create
similar experiences to those experienced when playing games, since it is crucial to motivate
and engage users. This is generally accomplished only in Duolingo through the application of
game-design elements (progress bar, leaderboard, badges, achievements, etc.) and game
principles (dynamics and mechanics) in language learning contexts. The rest apps failed to
provide at least a few gamification features (max 20% of participants).

To produce an overall clear picture of how useful participants found the app, they were
asked to what extent they are satisfied with community engagement, thematic sessions and
reviewing/feedback to help them improve their knowledge. Most respondents (70%-90%)
agree that all these apps implement proper the community engagement by creating close or
open groups for discussion and interaction, or arranging face-to-face meetings for language
practice or posting topics in forums for debate. Furthermore, the language learning apps do
not go off course from traditional course books and textbooks; their thematic areas are
similar and contain almost identical phrases and lexemes learning or grammar instructions.
However, Busuu, LingQ and Rosetta stone pre-shape their language courses according the
actual needs of each user (family and friends, business, entertainment, advance level, etc.)
and it was expected to represent some major differences (only 10%) compared to the other
two apps. To conclude, another significant essential element is reviewing and feedback.
Once again, Duolingo was considered (90% of participants) as an ideal app to receive a
proper reviewing/feedback of their effort; actually, Duolingo has an embedded routine to
include some repeated errors and mistakes in the new thematic sessions to revise them.
Rosetta stone offer a specific tab to review the previous knowledge.

Language learning apps and design evaluatoion

100
90
80
70
60
50
40
30
20
10
0
Interface Gamification Community Thematic sessions Reviewing

Duolingo Busuu LingQ Memrise Rosetta Stone

Overall, the participants describe that the five apps justify their high score reviewing with
minor differences between them. Although Rosetta stone received the most responses,
nevertheless they will not consider to pay any subscription to access Rosetta stone basic and
advance package2. The final winner is Duolingo, since it has the best implement mobile app,
a catchy interface, well-embedded gamification features and sufficient language skill
activities. We have to consider that the massive updates of the apps in the future may tip
the scale of these performance, since the designers always try to offer new features and
elements that improve the overall performance and effectiveness.

5.3. Limitations and further research

The data collected for this study is self-reported and therefore subject to the limitations that
such research methods have (Paulhus & Vazire, 2007). Nonetheless, these results must be
interpreted with caution and a number of limitations should be borne in mind. Although no
incentive was offered for participating in the research and it was tried to minimize a possible
2
Rosetta stone offers an unlimited access to their courses for three days.
bias effect, it is a possibility that some respondents may have been tired by using so many
different apps in a short period and confused of which features is supported by which app.
Moreover, it seems that the latest used app (Rosetta stone in our case) was boosted in their
ratings since it was the most recent and it was easier to recall its features. When conducting
a comparative study, it is important to have a sufficient sample size in order to conclude a
valid research result. Therefore we need a larger sample, to have more precise results
although it is quite challenging to collect a large group of participants that will test different
apps. Moreover, our limited qualitative data from the short interviews do not allow us to
generalize the research findings. Based on these data, it is necessary to expand the
participant group, randomize the order of app testing and allow the participants to choose
any language for each app.

6. Concluding remarks
The purpose of this paper was to explore and compare the overall experience of five top-
rated language learning apps and their effectiveness in learning a language and developing
language skills. The majority of the participants enjoy the use of these apps and they use
them for entertainment/travel and educational purposes. They stated some of the
advantages and disadvantages of studying autonomously based on their personal
experience. It is important that the learners have the opportunity to personalize their course
plan and adjust the learning procedure based on their time schedule and preferences.

The vast majority of the participants were satisfied by Rosetta stone and Duolingo. However,
when it comes to the development of specific language skills, such as speaking and writing,
there are some contrasting views of problematic implementation and dissatisfaction of their
effort. Usually, the participants were pleased with learning new vocabulary, grammar and
pronunciation. Additionally, gamification and reviewing/feedback are some essential
elements that are correlated with the app score and its effectiveness. Cross-evaluation with
the top-10 rated LLPs is needed with sufficient qualitative and quantitative data. Evaluating
and constantly developing new learning features and properties will only have positive
results having in mind the learners preferences and suggestions.

References
Brown, V., & Clarke, M. (2006). Using thematic analysis in psychology. Qualitative Research,
3(2), 77-101.

Bulatovych, D. (2017, 05 17). Yalantis. Ανάκτηση 06 22, 2021, από How to Build a Prosperous
Language Learning App: https://yalantis.com/blog/how-to-build-a-engaging-
language-learning-app/

Castaneda, D., & Cho, M. (2016). Use of a game-like application on a mobile device to.
Computer Assisted Language Learning, 27(9), 1195–1204.

Cedercreutz, E., & Ropicki, M. (2018). Efficient, Effective Vocabulary Learning with Memrise.
53rd ASOCOPI Annual Congress. Bolivar, Colombia: Universidad Tecnologica de
Bolivar.
Chen, X. (2016). Evaluating Language-learning Mobile Apps for Second-language Learners.
Journal of Educational Technology, 9(2), 39-51. doi:10.18785/jetde.0902.03

Chen, X., & Kessler, G. (2013). Action research tablets for informal language learning:
Student usage and attitudes. Language Learning & Technology, 17(1), 20-36.

Churchill, D., King, M., & Fox, B. (2013). Learning Design for Science Education in the 21st
Century. Journal of the Institute for Educational Research, 45, 404-421.

Citrayasa, V. (2019). Junior high school students’ lived experiences of learning English using
Busuu. Indonesian EFL Journal, 5(2), 85-92. doi:0.25134/ieflj.v5i2.1900

DeWaard, L. (2013). Is Rosetta Stone a viable option for L2 learning? ADFL Bulletin, 42(2), 61-
72.

Finardi, K., Gomes Leão, R., & Amorim, G. (2016). Mobile Assisted Language Learning:
Affordances and Limitations of Duolingo. Education and Linguistics Research, 2(2),
48-65. doi:10.5296/elr.v2i2.9842

Goodwin-Jones, R. (2011). Emerging technologies: Mobile apps for language learning.

Language Learning & Technology, 15(2), 2-11.

Guilar, J., & Loring, A. (2008). Dialogue and Community in Online Learning: Lessons from
Royal Roads University. Journal of Distance Education, 22(3), 19-40.

Heil, C., Wu, J., & Lee, J. (2016). A review of mobile language learning applications: trends,
challenges and opportunities. The EuroCALL Review, 24(2).

Holmberg, B. (1986). Growth and Structure of Distance Education. London: Croom Helm.

Huynh, D., & Iida, H. (2017). An Analysis of Winning Streak's Effects in Language Course of
"Duolingo. Asia-Pacific Journal of Information Technology and Multimedia Jurnal
Teknologi Maklumat dan Multimedia Asia-Pasifi, 6(2), 23-29. doi:10.17576/apjitm-
2017-0602-03

Karjo, C. H., & Andreani, W. (2018). Learning Foreign Languages With Duolingo and Memrise.
ICDEL 18 Proceedings of the 2018 International Conference on Distance Education
and Learning (σσ. 109-112). New York, NY: Association for Computing Machinery.
doi:10.1145/3231848.3231871

Kim, H. (2013). Emerging mobile apps to improve English listening skills. MultimediaAssisted
Language Learning, 16(2), 11-30.

King, F., Young, M., Drivere-Richmond, K., & Schrader, P. (2001). Defining Distance Learning
and Distance Education. AACE Journal, 9(1), 1-14.

Lafford, B., Lafford , P., & Sykes, J. (2007). Entre dicho y hecho …: An assessment of the
application of research from second languageacquisition and related fields to the
creation of Spanish CALL materials for lexical acquisition. CALICO Journal, 24(3), 427-
529.
Lord, G. (2015). An Exploratory Study into the Effectiveness of Rosetta Stone for Language
Learning. Florida: University of Florida.

Lord, G. (2016). Rosetta Stone for Language Learning. IALLT Journal of Language Learning
Technologie, 45(1), 1-35. doi:10.17161/iallt.v46i1.8552

Malerba, M. (2015). LEARNERS’ BEHAVIOURS AND AUTONOMY IN LIVEMOCHA AND BUSUU

ONLINE COMMUNITIES. Conference: EDEN 2015 Annual Conference, (pp. 1-6).
Barcelona.

Mason, A., & Zhang, W. (2017). An exploration of the use of mobile applications to support
the learning of Chinese characters employed by students of Chinese as a foreign
language. Στο Q. Kan, & S. Bax (Eds.), Beyond the language classroom: Researching
MOOCs and other innovation (pp. 99-112). Dublin, Ireland: Research-publishing.net.

Moore, M. (1997). Theory of transactional distance. Στο D. Keegan (Επιμ.), Theoretical

Principles of Distance Education (σσ. 22-38). London: Routledge.

Morgana, V. (2015). Investigating students’ and teachers’ perceptions of using the iPad in
the EFL classroom. EuroCALL 2015 Conference. Padova, Italy.
doi:10.13140/RG.2.2.12732.64641

Munday, P. (2017). Duolingo. Gamified learning through translation. Journal of Spanish

Language Teaching, 4(2), 94-98. doi:10.1080/23247797.2017.1396071

Nuralisah, A., & Kareviati, E. (2020). THE EFFECTIVENESS OF USING MEMRISE APPLICATION
IN TEACHING VOCABULARY. Professional Journal of English Education, 3(4), 494-500.

Nushi, M., & Eqbali, M. (2017). Duolingo: A mobile application to assist second language
learning. Technology and Language Learning/Teaching, 17(1), 89-98.

Nushi, M., & Jenabzadeh, H. (2016). Busuu: A Mobile App. TESL Reporter, 49(2), 30-38.

Peters, O. (1988). Distance Teaching and Industrial Production: A Comparative Interpretation

in Outline. Στο D. Sewart, D. Keegan, & B. Holmberg (Επιμ.), Distance Education:
International Perspectives (σσ. 95-113). New York: Routledge.

Psychogyiou, A., & Karasimos, A. (2019). The effectiveness of learning a foreign language via
a distance learning tool: testing the Duolingo application. Στο N. Topintzi, N. Lavidas,
& M. Moumtzi (Επιμ.), Selected papers of International Symposium of Theoretical
and Applied Linguistics 23 (ISTAL23) (σσ. 364-380). Thessaloniki: Aristotle University
of Thessaloniki.

Rachels, J., & Rockinson-Szapkiw, A. (2017). The effects of a mobile gamification app on
elementary students’ Spanish achievement and self-efficacy. Computer Assisted
Language Learning, 31(1-2), 72-89.

Rachels, J., & Rockinson-Szapkiw, A. (2018). The effects of a mobile gamification app on
elementary students’ Spanish achievement and self-efficacy. Journal Computer
 Assisted Language Learning, 31(1-2), 72-89. doi:10.1080/09588221.2017.1382536

Rakhmawati, I. (2020). Promoting Autonomous Learning Through Mobile Applications in a

Blended English Language Classroom. Advances in Social Science, Education and
Humanities Research, 34, 97-101.

Reknasari, A., & Putro, N. (2019). Using the Memrise mobile application to assist foreign
language learning. Στο J. Ashadi, A. Basikin, & N. Putro (Eds.), Teacher Education and
Professional Development in Industry 4.0 (pp. 238-243). London: CRC Press.

Rosell-Aguilar, F. (2009). In the decade or so since smartphones were introduced, these and
subsequent. Στο L. Lomicka, & G. Lord (Επιμ.), The next generation: Social
networking and online collaboration in foreign language learning (σσ. 13-34). San
Marcos, TX: Calico.

Rosell-Aguilar, F. (2017). State of the app: A taxonomy and framework for evaluating
language learning mobile application. The CALICO Journal, 34(2), 243-258.

Rosell-Aguilar, F. (2018). Autonomous language learning through a mobile application: a user

evaluation of the busuu app. Computer Assisted Language Learning,, 1-28.
doi:10.1080/09588221.2018.1456465

Rosell-Aguilar, F. (2018). Autonomous language learning through a mobile application: a user

evaluation of the busuu app. Computer Assisted Language Learning, 37(8), 1-28.
doi:10.1080/09588221.2018.1456465

Santos, V. (2011). Review of Rosetta Stone Portuguese (Brazil), Levels 1, 2 &3. CALICO
Journal, 29(1), 177-194.

Saona-Vallejos, M. (2018). Busuu: how do users rate this app for language learning? Στο F.
Rosell-Aguila, T. Beaven, & M. Gutiérrez (Επιμ.), Innovative language teaching and
learning at university: integrating informal learning into formal language education
(σσ. 27-36). doi:10.14705/rpnet.2018.22.773

Shibata, N. (2020). The Usefulness of Busuu Online Courses for Foreign Language Learning.
Computer-Assisted Language Learning Electronic Journal,, 21(2), 197-203.

Smith, P., & Ragan, T. (2004). Instructional Design (3rd εκδ.). Merill, MY: Wiley Jossey-Bass.

Son, J.-B. (2016). Selecting and evaluating mobile apps for language learning. Στο The
international handbook of mobile-assisted language learning (σσ. 161-179.). Beijing:
China Central Radio & TV University Press.

Steel, C. (2012). Fitting learning into life: Language students’ perspectives on benefits of
using mobile apps. Στο M. Brown , M. Hartnett, & T. Stewart (Επιμ.), ASCILITE 2012 -
Annual conference of the Australian Society for Computers in Tertiary Education, (σσ.
875–880). Wellington, New Zealand.
Stock McIsaac, M., & Gunawardena, C. (2004). Distance learning. Στο D. Jonassen, & M.
Driscoll (Επιμ.), Handbook of Research on Educational Communications and
Technology (2nd εκδ., σσ. 347-388). New York: Routledge.

Stockwell, G., & Hubbard, P. (2013). Some emerging principles for mobile-assisted language
learning. Monterey, CA: The International Research Foundation for English Language
Education.

Stracke, C., Tan, E., Moreira Texeira, A., do Carmo Texeira Pinto, M., Vassiliadis, B., Kameas,
A., & Sgouropoulou, C. (2018). Gap between MOOC designers' and MOOC learners'
perspectives on interaction and experiences in MOOCs: Findings from the Global
MOOC Quality Survey. 2018 IEEE 18th International Conference on Advanced
Learning Technologies (ICALT) (σσ. 1-5). Mumbai, India: IEEE.

Wedemayer, C. (1981). Learning at the Back Door: Reflections on Non-Traditional Learning in

the Lifespan. Madison, WI: University of Wisconsin Press.

Widyaningrum, N., & Putro, N. (2019). Using Memrise to promote students’ listening and
speaking abilities. Στο J. Ashadi, A. Basikin, & N. Putro (Επιμ.), Teacher Education and
Professional Development in Industry 4.0 (σσ. 1-5). London: CRC Press.

Winans, M. (2019). Busuu: A Social Network Application to Learn Languages. CALICO Journal,
37(1), 117-126. doi:10.1558/cj.37781

Winda, A. (2019). Teaching Listening Practice by Using Aplication on LingQ. INA-Rxiv Papers,
1-8. doi:10.31227/osf.io/7q8fv

Zhang, X. (2019). Memrise. CALICO Journal, 36(2), 152-161. doi:10.1558/cj.37857