Project 4 White Paper With Results

ABSTRACT
The world is becoming smaller and smaller

with technology and you can now meet
individuals from all over the world, with
different backgrounds and different
cultures. Many people turn to language
learning applications to communicate
better when they are traveling or mainly
just because they are curious. There are
many possible mobile apps out there but
LEARNING A many of them have different interfaces and
benefits and a usability test will be done on
two of the most popular language courses,
Duolingo and Busuu.
LANGUAGE WITH Nicole Yu
ENG 317 Technical Writing
MOBILE APPS
3/24/2019
Contents
Introduction ................................................................................................................................................... 2
Methodology ................................................................................................................................................. 2
Equipment and Test Environment............................................................................................................. 2
Task List ................................................................................................................................................... 2
Test Metrics .............................................................................................................................................. 2
Likert Scale ............................................................................................................................................... 3
Results ........................................................................................................................................................... 3
Task 1 Installation of the Application ....................................................................................................... 4
Task 2 Creating an Account ...................................................................................................................... 5
Task 3 Choosing a language (French)....................................................................................................... 7
Task 4 Complete a Tutorial or Onboarding experience ............................................................................ 9
Task 5 Switching a language (French  Japanese)................................................................................ 11
Task 6 Completing the First Lesson ....................................................................................................... 13
Task 7 Completing a Placement Test (German) ..................................................................................... 15
Task 8 Conversation where you must speak into phone ......................................................................... 17
Task 9 Completing a Review Section ..................................................................................................... 18
Task 10 Finding FAQ section ................................................................................................................. 19
Conclusions and Recommendations ........................................................................................................... 21
Appendix ..................................................................................................................................................... 23
Likert Scale Task 1 ................................................................................................................................. 23
Likert Scale Task 10 ............................................................................................................................... 27
Introduction
Many people now are exposed to individuals that speak many different languages and now there
are a growing amount of people that want to learn another language. Due to ease of access with mobile
phones the popularity of mobile language learning applications has grown more and more popular.
Duolingo and Busuu are two of the most popular mobile language learning applications for android and
iPhone. Typically learning a new language takes a long amount of time and dedication on the user’s parts,
with focuses on vocabulary, grammar, and speaking skills.
This white paper will evaluate Duolingo’s mobile application for ease of use, the successful use
of the application and user’s experience with the application. The user will be a general individual who
wants to learn and a new language and has no or minimal prior experience with the language of choice.
The usability test is based on ten common tasks that the user will encounter. By using the same metrics
for both apps, a direct comparison can be drawn between the two applications. The typical user is just
assumed to be an individual with no prior knowledge of the language they are learning.
Methodology
Equipment and Test Environment
This usability test for both Duolingo (Version 6.10.0) and Busuu (Version 19.2.0) were
conducted on a weekday evening using an Apple iPhone 8. This test was done in a quiet and private
environment with minimal interference or distractions in the surroundings. This was done because
learning a language is often one that requires a lot of focus from the user with minimal distractions. The
user was not able to use any outside resources to answer any language questions they may have had.
Task List
These tasks below were chosen to test the usability of the two applications. Each task are general tasks a
user may do within the first few uses of the apps. The designated user will be an individual whose current
language is English and wants to learn a new language and has no or minimal prior experience with the
language of choice. Also the user will be using the applications without paying for any upgraded plans
since both plans have an updated paid ad-free version.
1. Installation of the Application
2. Creating an Account
3. Choosing a language (French) or Onboarding experience
4. Complete a Tutorial
5. Switching a language (French  Japanese)
6. Completing the First Lesson
7. Completing a Placement Test (German)
8. Conversation where you must speak into phone
9. Completing a Review Section
10. Finding FAQ section
Test Metrics
The user will complete all the tasks listed in the section above to the best of their ability. Once a task has
been completed they will rate their experience with each metric below using the Likert Scale.
Effective: The task can be completed and teach some aspect of a new language.
Efficient: The task can be completed in a practical amount of time with a reasonable amount of effort.
Engaging: The application’s style and appearance encouraged the user to use it.
Error Tolerant: The application allowed the user to recognize and remedy errors.
Easy to Learn: The task could be completed easily and relatively straightforward.
Likert Scale
Results
To determine which application had a better user interface, these metrics were used and answered
on a 5-point Likert scale. Each application was judged on the same scales and task. Each section will just
list the overall score out of 25, but the usability surveys will be included in the Appendix section of this
white paper. A quick overview is provided in the beginning of each task section and then screenshots
from each application follow.
Task 1 Installation of the Application
initially the user must first install the applications on their phone or laptop. I searched both “language”
and “language learning” in the app store. Duolingo is the first result for both searches while Busuu shows
up as the 6th or 7th option. This means that Duolingo is more effective and has a higher ease of access
since it is the first shown result. In terms of choice of wording and design on their app store page, it seems
that Busuu markets itself more as a language learning application than Duolingo does.
Task Score
Busuu: 23
Duolingo: 25
Fig. 1. The application description screen for both applications in the Apple App Store.
Task 2 Creating an Account
Creating an account would be useful for the user since it allows them to access their information
immediately and across multiple platforms. Busuu immediately has the user enter in their information
before you can even use the app. Duolingo allows the user to complete a full lesson before they are
asked to create an account, but it is not even necessary. In the section to create the account there is an
option to click later, which due to the contrasting background option, seems to be a more visually
appealing choice. It seems that Duolingo does not put as much of an emphasis on creating an account as
Busuu which led to its lower task score. Both applications require a similar amount of information.
Task Score
Busuu: 22
Duolingo: 13
Busuu
Fig. 2. The Busuu screens for creating a profile.

Duolingo
Fig. 3. Duolingo prompt that shows up after completing a lesson.

Task 3 Choosing a language (French)
The user often has many different options for languages that they are able to choose, this choice often
comes immediately after the home screen. Languages are organized by the popularity of the language
and both applications use flags to signify the languages as opposed to alphabetically. This might be
difficult for users that are trying to find more ‘obscure’ or less popular languages since both of them sort
by popularity, not alphabetically. The design is very simple and straight to the point.
Task Score
Busuu: 25
Duolingo: 25
Busuu
Fig. 4. The language choosing screen for Busuu.

Duolingo
Fig. 5. Choosing a Course for Duolingo.

Task 4 Complete an Onboarding experience
Applications often run differently from each other. An assessment was made about the onboarding or
tutorial experience. Busuu provides a quick introduction similar to the screencaps where they tell you
what you are going to accomplish with a quick sentence and casual images of people learning in
different settings. You must press continue below the sentence to move on. For Duolingo it immediately
jumps into the lesson with no explanation but throughout the lesson there are short instructions to
complete the tasks. Busuu uses emphasizes more of the travel benefits of learning a language while
Duolingo is more focused on the user.
Task Score
Busuu: 21
Duolingo: 15
Busuu
Fig. 6. The travel imagery Busuu uses when creating the application..
Duolingo
Fig. 7. When entering Duolingo it goes straight into the lesson or logging into your account. It is much simpler, emphasizing just
learning a language forever.
Task 5 Switching a language (French  Japanese)
Many times, user would like to switch around and change their language of choice. Both applications
have a small circular icon with the flag of your current language in the upper left hand corner. To change
the language, you must click on this button. Since both applications have a similar design, they both
received the same task score, but since the icon does not explicitly state that is the location to change
the language I docked points on ease of access.
Task Score
Busuu: 23
Duolingo: 23
Busuu
Fig. 8. The Busuu screen after the flag icon is clicked. Provides short descriptions for each language.
Duolingo
Fig. 8. The Duolingo screen after clicking the flag icon in the corner. Duolingo allows you to see all your courses in this small tab
after a first lesson has been created..
Task 6 Completing the First Lesson
The first lesson of each application is the best introduction to the layout and design of the app. Both
applications have many different tasks that must be accomplished showing a well-rounded approach to
learning. Both apps have a completion tab at the very top which shows the users progress into the
lesson. Both apps have your bar go down with a mistake but Duolingo implements a health system and
too many mistakes will stop the user from continuing the lesson since the health bar decreases. I feel
like this provides extra incentive for the user to perform well. Also Busuu uses a generally blue and
white theme that is very monotonous. Duolingo uses a lot of color and imagery that really provides
incentive for the user, and even provides motivating messages from the Duolingo owl mascot
throughout the lesson. Duolingo also has the continue icon at the bottom throughout the lesson, this
remains grey when the task has not been attempted and then turns green when completed. Busuu on
the other hand does not even show the completed icon until an attempt has been made. Both
applications have the same exit ‘X’ symbol in top lefthand corner if you wish to exit application. Busuu
keeps it in blue while Duolingo has it in greyscale which might misled the user into thinking they cannot
click it.
Task Score
Busuu: 19
Duolingo: 23
Busuu
Fig. 9. Sample screens of Busuu lesson screens, the top shows the health bad and the exit logo. The lesson occupies most of the
screen and the continue pops up at the bottom of the screen.
Duolingo
Fig. 10. Sample screens of Duolingo lesson screens, the top shows the health bad, the exit logo and the amount of health the
user has left. The lesson occupies most of the screen and the continue button is grey if not attempted and turns into a green
‘check’ when done.
Task 7 Completing a Placement Test (German)
The user may desire to partake in a placement test. The placement test will allow them to skip certain
lessons and move forward faster in their learning. Duolingo and Busuu both emphasize more grammar
options and use a similar system to their lessons just with a larger variety of vocabulary. Duolingo
though at completion shows what lessons or vocabulary words you have mastered and can move on
with, which Busuu does not. I feel like that is more effective in conveying to the user what sections they
are successful in.
Task Score
Busuu: 17
Duolingo: 21
Busuu
Fig. 11. Sample screens of Placement tests which can be accessed in the same way switching a language can be done.
Fig. 12. Sample screens of Duolingo completion screens which can be accessed in after the placement test and tell the user which
lessons they have passed.
Task 8 Conversation where you must speak into phone
The user also has the option to participate in speaking and listening lessons. This is a subset within the
lessons but also shows more enhanced capabilities of the lesson. Both applications use a large
microphone shape and a prompt to speak into the phone. At completion though Busuu just tells you if it
was correct or not, Duolingo on the other hand shows word by word which you were able to pronounce
correctly.
Task Score
Busuu: 20
Duolingo: 24
Duolingo
Fig. 13. Duolingos speaking lesson which at completion highlights which section of the sentence you have accurately said.
Task 9 Completing a Review Section
Learning a language requires constant repetition of the vocabulary and grammar. Both applications
utilize a review system but Duolingo provides more incentives for completion by having a rewards
system. This is also when I realized the difficulties or challenges with the life system, Duolingo requires
the user to only continue lessons if they have ‘health’ left. This screen will show up in the middle of
review or lesson sections.
Task Score
Busuu: 19
Duolingo: 23
Duolingo
Fig. 14. Duolingo’s health boosting screen which will show when the user has run out of current lives.
Task 10 Finding FAQ section
Users often have questions that must be answered. This task accesses the accessibility of finding the
FAQ. You must go to your profile page to find the support section. Busuu just uses a small gear in the
upper right hand corner while Duolingo has a tab that says “Help Center”. Busuu leads you to a contact
us form while Duolingo leads you first to a FAQ and then to a contact us form. It seems that Duolingo is
more proactive in answering questions by having commonly asked questions in the FAQ section.
Task Score
Busuu: 14
Duolingo: 19
Busuu
Fig. 15. The screenshots to complete a Contact Us Form, very generic form and there is only one place for all types of questions.
Duolingo
Fig. 16. The Duolingo FAQ section which is helpful since many users have common questions that can be quickly answered.
Conclusions and Recommendations
Overall after completing the usability tests for both applications, Duolingo was deemed as more usable.
Duolingo received a scored of 211/250 while Busuu scored 203/250.
Individual Task Scores

25
20
15
Score
Duolingo
10
Busuu
5
0
Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 Task 8 Task 9 Task 10
Task
Fig. 17. Histogram of Individualized Task Scores for Duolingo vs. Busuu.
Overall Task Scores

250
200
150
Score
Duolingo
100
Busuu
50
0
Task 1
Task
Fig. 18. Histogram of Overall Task Scores for Duolingo vs. Busuu.
Duolingo and Busuu are both very useful language learning applications but I feel that personally for me
Duolingo provides a more colorful, cheerful ambiance and provides more incentives to keep the user
using the application. The language lessons are simpler, and the application design is easier to navigate in
terms of design and answering questions the user may have. Busuu might be more useful for someone
that desires a more upfront learning experience without the extra stress of maintaining lives or a health
system or having icons throughout. Though I feel that the imagery that Duolingo uses is very specific to
the vocabulary being used while Busuu uses more of complete imagery, so if you are learning about
animals, Busuu would show farm while Duolingo shows cartoon icons of individual animals. Also I feel
that in the lesson screen Duolingo is more straightforward in the users ability to exit the screen and what
needs to be done to complete the lesson, since the layout does not vary drastically from lesson to lesson.
Improvements that I would recommend to Busuu are:
1. During speech recognition user must repeat phrases multiple times usually and there is often a lag
or delay in the phone being able to recognize it.
2. Allow user to have more lives, or choose the difficulty setting they are at. The 5-life limit may be
too small for some users and cause them to abandon the application.
3. Make users create accounts initially so that their progress may be saved.
4. Include an option for visually challenged users to change the font size initially.
Improvements that I would recommend to Duolingo are:
1. During speech recognition address which sections the user is failing to pronounce correctly.
2. Allow users to switch between languages if they have not gone too far into the lesson, currently
the first option you choose is your permanent lesson unless you upgrade.
3. Include an option for visually challenged users to change the font size initially.
4. Make use of simple imagery as opposed to realistic photos in lessons so that the relation between
new vocabulary and image is easier.
Bibliography
Busuu. Version 19.3.0. Busuu Limited, 2008. Apple App Store, https://itunes.apple.com/us/app/busuu-
language-learning/id379968583?mt=8
Duolingo. Version 6.13.0. Duolingo, 2011. Apple App Store,

https://itunes.apple.com/us/app/duolingo/id570060128?mt=8.
Appendix
Likert Scale Task 1
Mobile Application
Duolingo Busuu
Task Installation of the Application
Effective The task can be completed and teach some aspect of a new language. The task can be completed and teach some aspect of a new language.
Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

1 2 3 4 5 1 2 3 4 5
The task can be completed in a practical amount of time with a reasonable amount The task can be completed in a practical amount of time with a reasonable amount
Efficient of effort. of effort.
1 2 3 4 5 1 2 3 4 5
Engaging The application’s style and appearance encouraged the user to use it. The application’s style and appearance encouraged the user to use it.
1 2 3 4 5 1 2 3 4 5
Error
The application allowed the user to recognize and remedy errors. The application allowed the user to recognize and remedy errors.
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
The task could be completed easily and relatively straightforward. The task could be completed easily and relatively straightforward.
Learn

1 2 3 4 5 1 2 3 4 5
Additional Around 5th or 6th in the search. Seems to have a significant amount of downloads
First in Apple Search as long as you are not including Ads.
Notes but does not load first.
Likert Scale Task 2

Mobile Application
Duolingo Busuu
Task Creating an Account

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional Requires: Age, Name, Email, Password. Done after the first lesson, prompt screen
Name, Email, Password. Done immediately in the beginning.
Notes shows up.
Likert Scale Task 3
Mobile Application
Duolingo Busuu
Task Choosing a Language

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional Done immediately after home screen, countries are shown in linear list with a small Done immediately after home screen, countries are shown in linear list with a small
Notes circualr icon with the flag next to it. Goes straight to lesson. circualr icon with the flag next to it. After clicking small description screen shows.
Likert Scale Task 4

Mobile Application
Duolingo Busuu
Task Completing a Tutorial (Or any Onboarding Experience)

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional Small onboarding experience in the beginning where they go through each type of
No onboarding experience, goes straight into the tutorial.
Notes lesson and describe what needs to be done.
Likert Scale Task 5
Mobile Application
Duolingo Busuu
Task Switching to a Different Language

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional Switching is done by clicking a small country symbol at the top lefthand corner, Switching is done by clicking a small country symbol at the top lefthand corner,
Notes only available in home screen. exact same design as Duolingo.
Likert Scale Task 6

Mobile Application
Duolingo Busuu
Task Completing a First Lesson

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Lesson is composed of many small tasks, small description at the top. There is a Lesson is composed of many small tasks, small description at the top. There is a
Additional complete button that must be clicked at the bottom to continue, is grey when task complete button that must be clicked at the bottom to continue, does not appear til
Notes not attempted. task has been completed.
Likert Scale Task 7
Mobile Application
Duolingo Busuu
Task Completing a Placement Test

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional Placement test is done in same style as lesson, more heavy in speaking and Placement test is done in same style as lesson, more heavy in speaking and
Notes grammer than vocabulary. grammer than vocabulary, much more conversation heavy than Duolingo.
Likert Scale Task 8

Mobile Application
Duolingo Busuu
Task Conversation where you speak into Phone

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional Once you speak into the phone, app will highlight green which specific words were Once you speak into phone and complete task, only tells you if successul or not,
Notes said correctly. Very targeted learning. not as helpful as Duolingo in recognizing which words were said incorrectly.
Likert Scale Task 9
Mobile Application
Duolingo Busuu
Task Completing a Review Section

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional Review section is prompted by clickin on lesson icon in home screen that you Review section is prompted by clickin on lesson icon in home screen that you
Notes would like to review. would like to review but then must click additional quiz section.
Likert Scale Task 10

Mobile Application
Duolingo Busuu
Task Contacting Support

1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
1 2 3 4 5 1 2 3 4 5
Error
Tolerant

1 2 3 4 5 1 2 3 4 5
Easy to
Learn

1 2 3 4 5 1 2 3 4 5
Additional From home screen must click around 3 times to get into section of FAQ with No FAQ section, from home screen around 2 clicks to get to contact us section.
Notes many pre prepared questions. Not as prompt with help as Duolingo.

Project 4 White Paper With Results

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Project 4 White Paper With Results

Uploaded by

Copyright:

Available Formats

ABSTRACT

The world is becoming smaller and smaller

Fig. 2. The Busuu screens for creating a profile.

Fig. 3. Duolingo prompt that shows up after completing a lesson.

Fig. 4. The language choosing screen for Busuu.

Fig. 5. Choosing a Course for Duolingo.

Individual Task Scores

Overall Task Scores

Duolingo. Version 6.13.0. Duolingo, 2011. Apple App Store,

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Likert Scale Task 2

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Likert Scale Task 4

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Likert Scale Task 6

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Likert Scale Task 8

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Likert Scale Task 10

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

Strongly Disagree Strongly Agree Strongly Disagree Strongly Agree

You might also like