Tech Know Learn

DOI 10.1007/s10758-017-9339-7

ORIGINAL RESEARCH

Evaluation of an Extendable Context-Aware ‘‘Learning

Java’’ App with Personalized User Profiling

Jane Yin-Kim Yau1 • Zornitsa Hristova2

 Springer Science+Business Media B.V. 2017

Abstract In recent times, there has been an uptake of mobile learning (hereafter, abbre-
viated as m-learning), i.e. learning with mobile technologies, especially in the form of
mobile apps. Apps are particularly useful as they are small applications usually with a
single-purpose and allow the users to view learning content offline from their mobile
devices. They can be a good supplement to formal learning/training and can be very
efficient informal learning tools, allowing learners to learn anytime and anywhere. In this
paper, we present our ‘‘Learning Java’’ app, which was designed based on the theoretical
framework ‘‘context-aware personalized m-learning application with m-learning prefer-
ences’’ (Yau and Joy 2011). The app utilizes a personalized user profile consisting of
location, noise and time of day, as well as the learner’s knowledge level. Additionally, an
understanding of different m-learning preferences by learners is represented in our app as
their individual user profile, for example, a learner may concentrate the best in a quiet
library and the app will select appropriate (more difficult and longer) material based on this
information, as opposed to shorter and easier materials. Video materials are also used by
learners. This app was tested by 40 volunteers; 10 of which completed a long questionnaire
regarding the usage of the app in terms of personalized user profile, context-awareness
factors and whether the app helped increase their motivation for learning and their learning
effectiveness for the subject. The results highlighted that participants could optimize their
spare times for most effective learning (e.g. video-watching with headphones) in busy and/
or noisy environments. Findings also showed other chosen learning strategies by learners
to make their learning more effective. Future work includes (1) extending the app for other
subjects and disseminating it for use by remote learners, for example, those who are

& Jane Yin-Kim Yau

jyau@mail.uni-mannheim.de
Zornitsa Hristova
zphristova@gmail.com
1
Chair of Learning, Design and Technology, School of Business, University of Mannheim,
Mannheim, Germany
2
Department of Computer Science, Malmö University, Malmö, Sweden

123
 J. Y.-K. Yau, Z. Hristova

situated in developing countries without frequent access to wireless internet and/or edu-
cational materials, and (2) including learning analytical support to students to enhance their
study success.

Keywords Mobile learning
Context-awareness
Personalization
User

profile
Learning analytics

1 Introduction

The m-learning field has evolved significantly in the last 15 years via the advancement of
technologies, the widespread ownership and utilization of mobile devices in both devel-
oped and developing countries. In these recent times, learning with mobile technologies
has become an everyday activity and these technologies are often a powerful tool for
bringing rich educational materials as well as building new communities of learners, where
they previously did not exist before such as in Cambodia, Sierra Leone and remote areas of
Pakistan (Moore and Hu 2007; West and Vosloo 2013). The unique opportunities facili-
tated by mobile technologies include personalization of learning, providing immediate
feedback and assessment, enabling anytime and anywhere learning, supporting situated
learning, enhancing seamless learning, and bridging formal and informal learning. Mobile
technologies can also be used as a tool to minimize educational disruption in conflict and
disaster areas, assist learners with disabilities, as well as providing more educational
opportunities to remote learners such as in Africa (Ibid).
M-learning applications/apps have also become abundant. Applications have been
defined as a system, which has multiple functions, whereas an app usually has just one
function and uses content that may be viewed offline (Godwin-Jones 2011). Applications
have been implemented in a wide variety of subjects in order to enhance students’ learning
experiences and effectiveness including the learning of geometry mathematics (Yau et al.
2011), learning of history via museum visits (Nordmark and Milrad 2015), and learning of
science by measuring soil and water quality (Yau and Salavati 2011; Vogel et al. 2010).
Godwin-Jones (2011) described a number of mobile apps as new emerging technologies
for language learning in different languages such as Japanese, Chinese, English, French
and ESL. The sophistication of today’s apps requiring minimal user knowledge on the use
of technology can be illustrated by the Google Translate’s live camera translation feature
which requires the user to only hold their device camera to point to Japanese characters to
obtain an instant translation of it into English (Davenport 2017).
The learning analytics field is an emergent field since around 2010 and has developed at
a fast pace. Learning analytics is defined as ‘‘the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of understanding and
optimizing learning and the environments in which it occurs’’ (LAK 2011). Specifically, by
providing learning analytical support to learners whilst they are utilizing m-learning
applications, data about learner’s m-learning activities can be obtained and used to enhance
their learning experiences and effectiveness. In the review conducted by Ferguson et al.
(2016), a number of learning analytical tools have been presented including ASSISTments,
Bettermarks, Bingel, Cito LUVS, Cognitive tutor software. This report provides a state-of-
the-art on current research in this field in Europe.
Learning contexts are often utilized in m-learning applications/apps. These are defined
as ‘‘any information that can be used to characterize the situation of learning entities that
are considered relevant to the interactions between a learner and an application’’ (Wang

123
Evaluation of an Extendable Context-Aware ‘‘Learning Java’’…

2004). Context information (such as location, noise) is made known to applications via the
Global Positioning System (GPS), Radio-Frequency Identification (RFID) or microphone
respectively and is reasonably fast and accurate. Other contexts such as detecting a lear-
ner’s motivation level are currently still very challenging, if not impossible. Context-
awareness refers to ‘‘the use of context detected by the device to provide task-relevant
information and/or services to a user’’ (Abowd et al. 1999).
Empirical evidence suggests that learners have different learning styles when learning in
traditional classroom contexts and they have different learning preferences in the educa-
tional process since learners differ in various aspects such as background, interests and
talents (Keefe 2007). In m-learning settings, individual learning preferences also apply. In
our previous work, we designed, developed and evaluated a context-aware m-learning
schedule framework (Yau 2011). The results of an interview study that constituted part of
this research demonstrated that learners have different learning preferences in m-learning
settings such as one learner may strongly prefer to study in a quiet library, whereas another
may strongly prefer to study in a busy café. M-learning allow learners to choose flexibly an
optimum learning environment/location that suits their needs. Some learners may also be
aware of the types of locations they may best and least concentrate in. In this paper, these
individual learning preferences form the foundation of our context-aware ‘‘Learning Java’’
app, which consists of a personalized user profile for each learner and provides interactive
feedback to students. The app detects three context factors: location, time of day, and noise
level. A user profile is formed for each user storing their preferred situation for study and
their preferred study materials for different locations such as reading longer texts in a quiet
environment, and watching video lectures in a noisier environment. The aim of this per-
sonalization is to increase students’ motivation in different learning settings to maximize
their learning effectiveness as well as boost their learning experiences. Motivation is often
argued to be the number one determinant factor in the success of any learning situation,
which substantially influences the learner’s process and outcomes (Williams and Williams
2011). The current version of our app is intended for Java programming language learners
utilizing learning objects as the source of learning materials; however, the materials can
easily be changed to extend to any subject and difficulty level.
In Sect. 2 of this paper, related literature to our work is described, divided into three
sub-sections which are substantial for increasing learners’ motivation to participate in
m-learning: personalization and user-profiling, enabling anytime anywhere learning via
the use of (offline) learning objects repositories, and learning analytical support. In
Sect. 3, the design, implementation of our app is illustrated. In Sect. 4, the results from the
evaluation of the app are presented. Finally in Sect. 5, we discuss the conclusions and
future work.

2 Literature Review

2.1 Personalization and User Profiling

Nowadays, mobile devices are usually owned by their users. Many of today’s functions on
these devices are easily customizable such as the amount of light on the screen, and the
volume of any media used. Information can also be used to inform different device
applications to change the font-color, text background, and icons. More sophisticated
features can be customized such as the user’s background knowledge and skills on various

123
 J. Y.-K. Yau, Z. Hristova

topics. A profile would then be useful to capture details about the learner such as personal
details, which may describe their spoken languages (to influence choice of language on the
device), and subject knowledge areas (specific details about their proficiency levels in
different subjects). Having the user-profiling information allows algorithms to be used to
link appropriate content to profile. Without a user profile, personalization is more difficult
to take place (or on-the-fly) but is still possible using other means such as via the tracking
of browser or interaction history [such as tracking of keyword search in Amazon (Pazzani
and Billsus 2007)]. Personalized learning has more advantages than non-personalized
learning i.e. one content-fits-all-approach. Utilizing a user-profile can inform appropriate
materials the learner would be interested in order to encourage them to participate actively.
The learner’s interaction history is built when there are successful matches of content to
context. Time spent on repetitive tasks can also be decreased as the application is aware of
learning materials that have already been utilized via the history. Zhou and Rechert (2008)
divided personalization aspects into eight categories for recommendation or learning
applications to utilize; these are: personal profile, goals, knowledge, interests, preferences,
interaction and presentation history, devices, and other contexts factors. Learners’
knowledge levels are different and their capabilities to acquire new knowledge vary
(Baniardalan and Mahdavi 2012). These differences need to be taken into account by the
m-learning application (Zhou and Rechert 2008). The purpose of using learners’ knowl-
edge levels when suggesting learning materials is to present the learners with materials that
satisfy their goals, current knowledge levels, curiosity, and materials that will not be too
difficult.

2.2 Enable Anytime and Anywhere Learning Via Offline Learning Objects
Repository

The possibility for the learners to access learning objects (hereafter, abbreviated as LOs)
even when their mobile devices are offline can be very beneficial. In particular, this can
provide seamless learning in all learning situations with mobile devices regardless of space
and time (Trifonova and Ronchetti 2005). Thus, offline learning is a major part of every
m-learning application to obtain ‘true mobility’. Offline learning is especially important in
developing countries where online and wireless access to the internet are not (always)
possible. Similarly, in refugee and emergency situations, the ability to access offline
materials (particularly, in language learning) has demonstrated immense benefits to the
integration of refugees’ and immigrants’ lives in another country (West and Vosloo 2013).
Our previous works in LOs have shown a good number of high-quality LOs situated in LOs
repositories around the world in the subject of computing and these repositories can easily
be incorporated and utilized in various m-learning applications (Sinclair et al. 2013).
Online LO repositories are usually situated on servers of academic institutions such as
universities, schools and academies as well as other websites such as Youtube. For
example, Connexions LO repository is stored on a website of Rice University, USA.
YouTube allows teachers and students to share and reuse LOs (Chenail 2011). Video-based
LOs have also become increasingly common and their interactivity provides the ability of
stopping, rewinding, and replaying of their content. Finally, video content has advantages
over images and graphics when it comes to explaining topics that require motions such as
the concept of a while-loop in programming languages. Other websites such as Khan
Academy LOs (www.khanacademy.org) store an extensive amount of learning materials in
a variety of subjects.

123
Evaluation of an Extendable Context-Aware ‘‘Learning Java’’…

2.3 Learning Analytical Support

Systems with learning analytic support allow users to have real-time and interactive
feedback. Ferguson et al. (2016) provided an extensive list of current learning analytic
applications as well as policy initiatives by different national governments in Europe such
as in Denmark, Norway and Netherlands, also in Australia and USA. Shum (2012)
described ‘‘analytics’’ as a term used in ‘‘business and science to refer to computational
support for capturing digital data to help inform decision-making’’. Following from this
definition, the older works of intelligent tutoring and adaptive learning systems which
utilize students’ correct/incorrect answers and present to users easier/more difficult
materials (Nwana 1990) can also be considered as ‘learning analytics’, prior to the launch
of this field at the first Learning Analytics and Knowledge conference in 2011 (LAK 2011).
Put broadly, learning analytics uses collective intelligence to interpret learner behavior and
signals (in the form of learner data and interactions with the system) and adapt the system’s
behavior accordingly in order to produce better learning outcomes (Shum 2012).
One way of motivating students to enhance their current, ongoing and future learning
progress is to show their achievements, strengths and weaknesses in a user-friendly
dashboard. Some systems may provide their learners with results from peers to motivate
them to achieve the same level or better results as their peers. Predictive analytics is used
to make interventions and to help reduce dropout rates of students; these analytics utilize
static (achievement targets) and dynamic (quantity and quality of forum posts) learner data.
Other areas of learning analytics include adaptive learning analytics, social network
analytics, and discourse analytics (Shum 2012). There are important ongoing debates
regarding the ethical use of students’ data to inform the system about their next steps, and
that the retrieved data may not accurately reflect students’ motivation or lack of motivation
in learning. Nevertheless, small scales of learning analytic evaluations have shown and
documented empirical evidence that these to an extent can reach and intervene students
who may have dropped out of their course otherwise.
Suchithra et al. (2015) defined ten categories of learning analytical support, as follows
(our work presented in this paper can be categorized under the personalized suggestions
category):
1. About the performance of learners through assessment
2. About the participation of learners in traditional class rooms
3. About the participation of learners in online learning rooms
4. Participation in discussions
5. About the learners skill which helps the teacher
6. Personalized suggestions
7. Decision-making (Enrollments for universities)
8. Success factors of students
9. Feedback system
10. Pedagogy
Tabuenca et al. (2015) developed a mobile tool, which utilizes learning analytics for
self-regulated learning. Their results revealed positive learning effects for students having
their time tracked and their time management skills improved as a result. Their contri-
butions include suggestions on how mobile notifications should be designed to enhance
students’ self-regulated learning in online courses.

123
 J. Y.-K. Yau, Z. Hristova

3 An Extendable Context-Aware ‘‘Learning Java’’ App with Personalized

User Profiling and Potential Utilization of Learning Analytics

In this section, we present our research aims and questions in Sect. 3.1, the theoretical
framework of our ‘‘Learning Java’’ app in Sect. 3.2 followed by the implementation details
with graphical user interface and scenario in Sect. 3.3. In Sect. 3.4, we discuss the tech-
nical constraints we encountered and resolved in the system implementation as well as our
research limitations.

3.1 Research Aims and Questions

The aim of this research was to improve an existing theoretical framework of a person-
alized context-aware m-learning system (Yau and Joy 2011). We aimed to find
improvements via our literature review. We also aimed to evaluate if the developed system
based on this framework would be able to motivate users to study independently by
providing them with personalized suggestions for LOs based on their current contexts and
profile. Additionally, we aimed to explore the development feasibility of this framework
and evaluate its acceptance amongst users. Our research questions are as follows:
1. How can a prototype of a personalized context-aware m-learning system be
implemented?
2. How can a personalized context-aware m-learning system increase learners’ motiva-
tion for studying?
3. How can suggestion rules for the system be developed?

3.2 Theoretical Framework

Our ‘‘Learning Java’’ app is designed based on the proposed framework by Yau and Joy
(2011), as shown in Fig. 1. It consists of three main blocks: the user profile, the person-
alization mechanism, and the learning object repository. The innovative components of
this framework comprise the inclusion of a personalized user profile enabled by the two
main blocks—user profile and personalization mechanism. The user profile consists of
three main elements—location, noise and time of day. In our previous work, result findings
from a detailed interview consisting of 37 participants informed that some students had
very strong preferences to study in particular circumstances such as a quiet library, and
others may like to study in a noisy café. We found that these preferences were subjective
and could differ from learner to learner (Yau 2011). Therefore, we created the personalized
user profile and learners would be selected appropriate learning content to them based on
their current contexts with consideration to how they like to study under these circum-
stances (Fig. 2).
This proposed framework was developed into the ‘‘Learning Java’’ app and subsequent
work was completed succeeding Yau and Joy (2011) including a further literature review,
which informed us the following two additions to the app: (1) We enriched the user profile
by including the addition of information about the learner’s interaction history and subject
knowledge level (in this case: Java). (2) A further component was added which was a local
LO repository to support offline learning as well as video LOs, as opposed to only textual
ones. Videos were utilized via the Youtube website as a remote LO repository and can
provide students, teachers and practitioners with the opportunity to share reusable LOs

123
Evaluation of an Extendable Context-Aware ‘‘Learning Java’’…

Fig. 1 ‘‘Learning Java’’ app based on (Yau and Joy 2011) with user profile (based on contexts),
personalization mechanism and LO repository

(Chenail 2011). We selected a list of LOs that are suitable for watching both on devices
with small and large screens and these videos do not contain text with small font that
cannot be read easily on a small screen. Our previous work has demonstrated that this
framework can be used successfully as a motivating learning approach for supporting self-
regulated learners (Yau and Joy 2008, 2009).
The proposed advantages of this app include (1) utilizing free unexpected time such as
waiting times (Evans 2008), (2) reducing learner frustration and increasing learner moti-
vation by selecting appropriate content based on their contexts and personalized profile
such as recommending users to more difficult study materials which require high levels of
concentration in their preferred place of study, as opposed to easier study materials in a
situation where they can concentrate less, (3) maintaining the engagement of learners in
their learning activities as well as enhance their learning effectiveness and enjoyability in
their learning process via the use of the personalization mechanism, and (4) improving
learning efficiency by minimizing time spent on repetitive tasks. The further literature
review that we conducted also informed us three criteria for our app, which we took into
consideration for the implementation, are as follows:
1. Changing Values of Contexts A learner’s location may change during a learning session
due to the mobility and flexibility of the nature of m-learning. However, the changes
during a learning session will not be taken into account and a learner’s learning materials
will not be altered during a learning session. The values of learning contexts are
considered from the beginning of the learning session only and learning materials for a
learning session are selected based on these values. In particular, a change in the location
may lead to a drastic change in other contexts such as quality and amount of light e.g.
sunlight. However, as mentioned, due to the usually short duration of study sessions, we
assume that the increase/decrease of sunlight will not affect the learning process.
2. Relationship Between the Time of Day Context and Learner’s Concentration Level A
study was undertaken by Rodrigues et al. (2002) who examined students’ grades in
relation with how sleepy they were whilst undertaking exams. They concluded that
sleepier students achieved lower levels of results in comparison with their classmates
who were more awake. In our app, we utilize the knowledge that learners typically

123
 J. Y.-K. Yau, Z. Hristova

Fig. 2 Screenshots of the ‘‘Learning Java’’ app

know what time of day their sleepiness is low and respectively their concentration
level is high (and will select the appropriate concentration level accordingly).
3. Relationship Between Chronic Noise Exposure and Learner’s Reading Abilities There
were many studies that demonstrated the decrease of learner’s reading abilities due to noise
(Evans and Maxwell 1997). Therefore, in the user-profile of our app, we enable learners to
choose their preferred study action when the noise level is high, medium or low.

3.2.1 User Profile Initialization (Based on the Criteria) Via User Input of Context
Values

At the beginning of each study session, the learner is requested to input the values of four
contexts which correspond to their current learning situation: location, noise, time of day

123
Evaluation of an Extendable Context-Aware ‘‘Learning Java’’…

and knowledge level. In the location context option, users are requested to name their
current location and select from a predefined list, how good the lighting is in this location.
The app will change the font size of text when there is bad lighting, as detected by the
camera. In the noise context option, users are requested to input the study actions they
prefer to perform when the noise level is high, medium or low. Users can choose between
four options: none, reading, watching video, or both. Users define their own boundaries for
low, medium and high noise levels; these levels are then recorded to their user profile. In
the time of day context option, users are requested to input the time intervals of the day and
related to them their level of concentration. Users specify at least one of the three possible
time intervals when their level of concentration is respectively high, medium, and low. Due
to time constraints, working days and weekends differences were not taken into account,
which can form the future work relating to our study. In the knowledge level option, 14
screens are used with each screen showing a question and four possible multiple-choice
answers. These questions were based on the Java knowledge tests published in india-
bix.com (IndiaBIX 2013). The goal of the knowledge test was to capture the users’ initial
knowledge level in the Java programming language. According to the number of correctly
answered questions, the users will be classified as beginners or advanced learners. The app
keeps track of all learning objects that have been opened by the learner in the Interaction
History, and distinguishes between the ones that have been completed and those, which
have not. Note that: the app will use default values for any missing/incompleted answers to
the context values. If there is insufficient context information, suggested LOs to users will
not be personalized.

3.3 Graphical User Interface, Implementation and Testing

Our ‘‘Learning Java’’ app was implemented on Eclipse using Java in the Android devel-
opment tools plug-in. An external library called Gson was used to convert the responses
from the server from JavaScript Object Notation (JSON) to Java objects. The reasons for
choosing Android as the development platform are to (a) reduce the effects of the types of
user’s devices on their user experiences as these can be impacted by the different operating
systems, screen sizes, software, and capabilities; and (b) automatically adapt the repre-
sentation of learning materials to different devices; and (c) eliminate the need for addi-
tional software to be installed (Zhou and Rechert 2008). The app has three main functional
components: (1) learner profile—stores the learner’s context factors preference, learner’s
knowledge level and interaction history; (2) personalization mechanism—detects the
current values of the context factors via embedded GPS, microphone, and internal clock,
suggests LOs to users and handles the interaction between the LO repository and users; and
(3) learning object repository—stores LOs and maps them with preference criteria. The
local LO repository allows the learners to download content for use later. The video
website allows the reuse and sharing of video materials, such as lectures, short introduction
videos, and documentaries and have the advantages of adding motions and step-by-step
instructions in relation to conventional images.
The learner profile is stored in two XML files—(1) stores learner’s context factors
preferences and learner’s knowledge level; (2) stores interaction history. In order to read
and write data in both XML files, XML parsers were developed. It has three system
requirements: (1) offline usage, (2) personalized suggestion of LOs, and (3) users should be
able to review and continue from previously studied materials. The application accesses
the user’s profile information from the device storage, which contains the values of context
information retrieved from the embedded GPS, microphone and internal clock, then sends

123
 J. Y.-K. Yau, Z. Hristova

a query for LOs to the web server. This query contains the knowledge level, concentration
level and LO format. If one of these parameters is missing, the application uses default
values and selects LOs, which are not personalized to the learner. All LOs in the database
are tagged to different knowledge levels.
The app was designed with resizable components to ensure similarity of the user
interface design between different device types. Screen (a) shows the start screen, which
has four buttons to access four different functions and their respective screens—File
History (interaction history), Downloads (previously downloaded LOs), User Profile and
About. Screen (b) shows the User Profile, the user is requested to enter information
regarding Location (c), Noise (d), Time contexts (e) and Knowledge Test (f) screens. In
Location (c), users enter the name of their current location and the usual quality of light in
this location by selecting from a drop-down list with predefined values; which is then
automatically be saved and the user is not requested the same information again. In Noise
(d), the user defines his/her personal preferences by setting (1) the boundaries for the low/
medium/high intervals for the noise level that he/she is comfortable with studying in; and
(2) the type of action that he/she would like to perform when the noise level lies within
these intervals. In Time (e), the user defines the current time of day and select from a drop-
down list with predefined values of concentration: low, medium or high. In Knowledge Test
(f), users answer all questions by selecting a correct multiple-choice answer and their
knowledge level in Java is then updated accordingly. All of this information relating to the
user is stored in the respective User Profile. (g) shows a textual LO, whereas (h) shows a
video LO. In History Activity (i) and Downloads Activity (j), a list of LOs that have been
studied during the application’s last use is displayed to enable continued usage. In (i),
internet connection is required whereas in (j) it is not as the LOs have been stored in the
device’s memory; this only applies to textual LOs. Note that video LOs can only be
streamed but not be downloaded.
The critical aspects of the app were tested prior to the evaluation for any errors or
problems. The tests included ensuring that the information stored on the three XML files
which contained the learner profile, interaction history, and downloaded LOs respectively
were stored accurately; queries sent to the server functioned appropriately; external links to
Youtube videos were checked to ensure their validity; and that the hardware—GPS,
microphone and internal clock functioned accurately. A findings and solution from the
testing include: false readings from the microphone produced due to background noise
(people talking or laughing or if the microphone was close to their mouths) were rectified
by manually/artificially lowering the noise level in the calculations that were made or when
noise values were displayed and entered on the screen.

3.4 Technical Constraints and Research Limitations

• Due to the small number of test mobile devices that we used during the implementation
and the evaluation stages, it was not possible to assure the proper functioning of the app
on all Android-based mobile devices. Two mobile devices to test the prototype were
used—HTC One and Samsung Galaxy Nexus. A few volunteers reported that they were
not able to install the application on other Android-based mobile devices.
• Due to time limitation and otherwise, (a) audio Java LOs were not located; (b) learning
styles for traditional learning were not considered as we focused on m-learning only;
(c) having only one topic (Java) would reduce the variety of users utilizing the app;
(d) small sample size of volunteers for the evaluation of the app which can make it
difficult to draw general conclusions.

123
Evaluation of an Extendable Context-Aware ‘‘Learning Java’’…

4 Evaluation of the ‘‘Learning Java’’ App

In this section, we present the research methodology for evaluating the app in Sect. 4.1. In
the subsequent sections, we present the research results—how effective were the person-
alized user-profiling and context factors relationships (Sect. 4.2), strategies to increase
students’ motivation learning (Sect. 4.3) and the personalization mechanism for suggesting
of LOs based on m-learning preferences (Sect. 4.4) respectively. In Sect. 4.5, we present
answers to our research questions presented in Sect. 3.1 as well as our contributions to the
field of context-aware m-learning.

4.1 Research Methodology

40 volunteers spent time for a week to learn Java and evaluated our app by filling out a
survey containing feedback questions. Ten out of 40 returned their questionnaires i.e. 25%
response rate. Three were undergraduate and seven were postgraduates. Three studied at
Malmo University, Sweden, six studied at Technical University of Sofia, and one studied at
Sofia University ‘‘St. Kliment Ohridski’’, Bulgaria. Eight were male and two were female.
Nine were aged from 22 to 27 years, one was 35 years. The volunteers had different levels
of Java programming experience—ranging from none to professional and some worked
three or more years as software developers. Volunteers were made aware that they may
withdraw from the study at any time.

4.2 Evaluation of the Personalized User Profile and Contexts

In this section, we describe and display the statistics (in Fig. 3) obtained for the volunteers
who were asked three questions about the deployed contexts in the app. Relating to
location, seven out of 10 stated that their study location affected their study processes, and
changes to other locations lead to changes in other contexts values such as noise. Three
participants stated that the change of their location did not affect their study process and
therefore would not require any change of LOs (in terms of easier vs. more difficult, shorter
vs. longer) for learning in that situation (#1). Relating to noise, Nine out of ten participants
confirmed that noise does strongly affect their concentration. One participant strongly
maintained that he ‘‘cannot memorize a single word if the place is not quiet enough’’ (#2).
Relating to time of day, six out of ten respondents agreed that this does affect their learning
process and preferred to study in the mornings when ‘‘their brains and eyes were fresh’’.
Another participant strategized their study sessions for more difficult topics in the morn-
ings (higher concentration) and easier ones in the afternoons and evenings (lower con-
centration) (#3).
Reasons for the different times of day, which affect the learning process, include habits,
sleepiness and lighting. Participants were asked to rank the contexts, which affected most
and least their concentration and it was found that noise affected most their concentration,
followed by the location, then the time of the day. Concerning the lighting levels for
different times of the day, (some) mobile devices automatically react to light changes by
changing the screen light intensity. Future work includes taking the geographical latitude
into account, which affects the sunlight levels in different parts of the world. We asked
volunteers to inform us of any other factors that may affect their concentration—half of the
respondents did not find additional factors whereas the other half named four factors: (1)
receiving notifications/distractions/disruptions (such as email, phone call, text message),

123
 J. Y.-K. Yau, Z. Hristova

Fig. 3 Feedback from participants on the contexts and the app

(2) other people surrounding the learner, (3) whether they were hungry/thirsty, and (4)
sleepiness. Participants suggested future improvements of the app including being able to
edit the user profile, incorporating a calendar into the app, and a way to disable incoming
calls, messages, and/or emails.

4.3 Evaluation of Strategies to Increase Students’ Motivation

In this section, we describe and display the statistics obtained for the volunteers who
studied LOs that were suggested to them based on their available time, motivation and Java
knowledge level. Two questions were asked relating to the effect of the app in terms of
increasing their motivation. Half confirmed that the prototype motivated them to study
whereas the other half stated that it did not affect their motivation level (Fig. 3 above, #4).
Additional strategies proposed by volunteers for increasing students’ motivation to learn/
use the app include adding new LOs (especially interactive ones) to the repository to
increase the variety of the materials. The interactive feedback as suggested by participants
as beneficial for increasing their learning effectiveness and motivation. These include
adding (1) instant real-time feedback given to learners based on their answers as a form of
assessment of their topic knowledge, (2) social connections to peers for discussions, and
(3) visualization of the learning progress of the completed LOs and their difficulty. Vol-
unteers considered that the content of the LO was more important than the current learning
contexts. They stated that when they were interested in a particular topic, they were more
motivated to learn which affected positively their learning process. Some of these strate-
gies were also proposed by Suchithra et al.’s (2015) categories of learning analytical
support (mentioned in Sect. 2.2), as follows:
• About the performance of learners through assessment
• Participation in discussions
• Feedback system

123
Evaluation of an Extendable Context-Aware ‘‘Learning Java’’…

4.4 Evaluation of the Personalized Mechanism and Suggestion Rules

Volunteers revealed their different study habits and preferences towards their study
environment, and the context-awareness of the app was received well and was considered
user-friendly as well as interesting for novel users. As shown in Fig. 4, all participants
expressed positive feedback of the contexts utilized in the app and the suggestions of LOs
based on their m-learning preferences stored in the user profile (#1). They stated that time
of day, lighting level, and noise affect their concentration and learning process. Half of the
volunteers stated that the app was able to motivate them to study (#2). Future work
includes displaying and giving learners the choice of selecting both LOs that are appro-
priate and inappropriate for the current context. Five volunteers were satisfied with the
suggested LOs and stated that they were appropriate for the learning contexts (#3). Indeed,
they stated that these were more enjoyable and useful than using LOs that were not
appropriate for the current learning contexts. The suggestion rules used the preferences
towards the three context factors stored in the learner’s profile and the learner’s knowledge
level to create a combination that is matched against LOs metadata. The metadata of the
LOs included: their type (text or video), the intended knowledge level, and the concen-
tration level that is needed to study the LO. Volunteers expressed positive remarks
regarding the personalization mechanism.

4.5 Answers to Research Questions and Contributions

In order to answer our first research question ‘‘How can a prototype of a personalized
context-aware m-learning system be developed?’’, we used the findings from our literature
review, implemented the prototype and extended the framework developed by Yau and Joy
(2011). Prior to the implementation, we improved the learner profile by adding the in-
teraction history component (Contribution 1). This allows the app to keep track of already
opened LOs and to allow the learners to review their history of learning materials. We
developed a local LOR, which allows learners to download content that is interesting to

Fig. 4 Feedback from the participants on the suggestion mechanism

123
 J. Y.-K. Yau, Z. Hristova

them for later use, as a complement to our remote LOR. This option increases the mobility
of the learning process as downloaded material can be studied without an Internet con-
nection (Contribution 2). Our remote LOR is able to access Youtube and allows the reuse
and share of video materials such as lectures, short informative videos and documentaries
(Contribution 3).
In order to answer our research question ‘‘How can a personalized context-aware
m-learning system increase learner’s motivation?’’, we built our app based on a theoretical
framework that has the potential to motivate learners. 40 participants used this app for a
week and filled in a questionnaire concerning different aspects (including motivation). Five
out of ten respondents indicated that the app did motivate them to study Java while the
other five stated that the app had no effect on their motivation. We can conclude that
although the app does not motivate all learners, it does not demotivate them. The most
motivating components of the app are the personalization mechanism and its context-
awareness, according to the participants—positive feedback of this was received from all
participants (Contribution 4). Most of them are unfamiliar with context-awareness. A
suggestions was made by a participant which was to add interactive LOs that can give
instant feedback. Another participant expressed that a system that would constantly add
new learning materials will motivate him/her. Increased motivation ensures learning
experiences and effectiveness are improved leading to enhanced study success.
In order to answer our research question ‘‘How can suggestion rules for the prototype
system be developed?’’, we designed our app to allow the users to create their own
suggestion rules by filling in their learner profiles (Contribution 5). The learners were able
to associate the quality of light, type of LOs and their level of concentration respectively
with different values of the three context factors: location, noise level and time of day.
First, learners were able to add the locations in which they tend to study and the related
lighting quality levels while they were in them. The users defined three noise intervals
perceived as quiet, medium and noisy and chose learning activities that they would per-
form in each of these scenarios. Third, learners defined the time interval in the 24 h of the
day when they study and defined their level of concentration during these time slots. The
suggestion mechanism implemented in the apposed the preferences towards the three
context factors stored in the learner’s profile and the knowledge level to create a combi-
nation that is matched against the LOs’ metadata (type—text/video, intended knowledge
level, and required concentration level) (Contribution 6).

5 Conclusions and Future Work

This paper has discussed and demonstrated how a context-aware ‘‘Learning Java’’ app with
personalized user profile was designed and implemented. The app was evaluated for a
week with volunteers of different ages, nationalities, and knowledge of Java. Positive
feedback was received from participants concerning particularly the personalization aspect
of the system as it suggests learning materials to them according to their preferences and
knowledge level. The context-awareness of the app was perceived as an innovative and
interesting feature. Some participants stated that they liked that the app suggested LOs to
them according to their current contexts and profile. They stated that when they were
interested in the topics, their motivation was higher than usual which positively affected
their learning process and experience.

123
Evaluation of an Extendable Context-Aware ‘‘Learning Java’’…

5.1 Future Works

1. The app can be upgraded to use LOs on a single other topic such as law; or a range of
topics such as mathematics, science and history. For example, the Connexions (2013)
repository provides LOs on different topics, such as Arts, Business, Humanities,
Mathematics and Statistics. We currently utilize LOs from this repository and as all
LOs here are stored in the same format, the app can easily be upgraded to provide
learners with learning materials on topics that are not related to Java programming.
2. The user profile can include information the type of interests that users have in certain
scientific topics as suggested by Zhou and Rechert (2008). For example, users of the
system can be interested in history because it is a part of their curriculum at university
or because they want to learn new interesting facts about their country. These types of
interest can be classified as professional or hobby. Respectively, the LOs in the
database should be tagged in order to make good suggestions to learners with LOs that
are of interest to them.
3. The user profile can be added with additional information about the history of their
completed LOs. To support this presentation history, the app keeps track of the status
of every LO. The LOs are marked either as new, completed, pending, or disliked. The
app prompts the users to change the status of the studied LO upon closing, if at least
95% of this LO was not completed. The status of LOs affect the results of searching
for new LOs. LOs that are completed or disliked are shown at the end of the list.
Different colors in the search results are used for different status of LOs.
4. The system can be upgraded to recognize the change and update of users’ knowledge
level as suggested by Zhou and Rechert (2008). The knowledge level represents
learners’ current knowledge on Java. The system edits users’ knowledge level after
they have finished the knowledge level part of the user profile input. In the future, the
value of their knowledge level can increase after users finish a self-test or read
sufficient number of LOs, which advance their knowledge level. Moreover, the system
can be upgraded to request users to review LOs that they have completed after some
period of time to provide and share feedback with others the qualities of the LOs.
5. The system can be incorporated with mobile learning analytics, and a dashboard for
learners to visualize their learning progress in different mobile and contextual
situations, similar to the work of Tabuenca et al. (2015). We intend to incorporate
Suchitra’s (2015) categories of learning analytics into the design and evaluation of our
future studies.
6. We will evaluate the system with a larger number of participants and also involving
other topics and difficulty levels. Based on the feedback, we will improve the app to
increase students’ learning and their study success.

References
Abowd, G. D., Dey, A. K., Brown, P. J., Davies, N., Smith, M., & Steggles, P. (1999). Towards a better
understanding of context and context-awareness. In H. W. Gellersen (Ed.), Handheld and ubiquitous
computing (pp. 304–307). Berlin: Springer.
Baniardalan, M., & Mahdavi, M. (2012). Personalizing m-learning systems using recommender systems.
International Journal of Scientific & Engineering Research, 3(4), 1–5.
Chenail, R. J. (2011). YouTube as a qualitative research asset: Reviewing user generated videos as learning
resources. The Qualitative Report, 16(1), 229–235.

123
 J. Y.-K. Yau, Z. Hristova

Connexions. (2013). Retrieved from http://cnx.org/.

Davenport, C. (2017). Google Translate adds instant camera English/Japanese translation. http://www.
androidpolice.com/2017/01/24/google-translate-adds-instant-camera-englishjapanese-translation-apk-
download/.
Evans, C. (2008). The effectiveness of m-learning in the form of podcast revision lectures in higher
education. Computers & Education, 50(2), 491–498.
Evans, G. W., & Maxwell, L. (1997). Chronic noise exposure and reading deficits the mediating effects of
language acquisition. Environment and Behavior, 29(5), 638–656.
Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., Ullmann, T.,
Vuorikari, R. (2016). Research evidence on the use of learning analytics—implications for education
policy. In R. Vuorikari & J. Castaño Muñoz (Eds.). Joint Research Centre Science for Policy Report.
Godwin-Jones, R. (2011). Emerging technologies—mobile apps for language learning. Journal of Language
Learning & Technology, 15(2), 2–11.
IndiaBIX. (2013). Retrieved from http://www.indiabix.com/online-test/java-programming-test/.
Keefe, J. W. (2007). What is personalization? Phi Delta Kappan, 89(3), 217.
LAK. (2011). In International conference on learning analytics and knowledge. https://tekri.athabascau.ca/
analytics/about.
Moore, P., & Hu, B. (2007). A context framework with ontology for personalised and cooperative mobile
learning. Computer Supported Cooperative Work in Design, III, 727–738.
Nordmark, S. & Milrad, M. (2015). Influencing everyday teacher practices by applying mobile digital storytelling
as a seamless learning approach. In Proceedings of world conference on mobile learning (pp. 1–17).
Nwana, H. (1990). Intelligent tutoring systems: An overview. Artificial Intelligence Review, 4, 251–277.
Pazzani, M. J., & Billsus, D. (2007). Content-based recommendation systems. In P. Brusilovsky (Ed.), The
adaptive web (pp. 325–341). Berlin: Springer.
Rodrigues, R., Viegas, C. A., Abreu e Silva, A. A., & Tavares, P. (2002). Daytime sleepiness and academic
performance in medical students. Arquivos de Neuro-Psiquiatria, 60(1), 6–11.
Shum, S. (2012). Learning analytics. Washington, D.C.: UNESCO Policy Brief.
Sinclair, J., Joy, M., Yau, J., & Hagan, S. (2013). A practice-oriented review of learning objects. IEEE
Transactions on Learning Technologies, 6(2), 177–192.
Suchithra, R., Vaidhehi, V., & Iyer, N. E. (2015). Survey of learning analytics based on purpose and
techniques for improving student performance. International Journal of Computer Applications,
111(1), 22–26.
Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015). Time will tell: The role of mobile learning
analytics in self-regulated learning. Computers & Education, 89, 53–74.
Trifonova, A., & Ronchetti, M. (2005). Prepare for a bilingualism exam with a PDA in your hands. Methods
and Technologies for Learning, 34, 343.
Vogel, B., Kurti, A., Spikol, D., & Milrad, M. (2010). Exploring the benefits of open standard initiatives for
supporting inquiry-based science learning. In Proceedings of European conference on technology
enhanced learning, lecture notes in computer science (pp. 596–601).
Wang, Y. K. (2004). Context awareness and adaptation in mobile learning. In Proceedings of wireless and
mobile technologies in education (pp. 154–158).
West, M. & Vosloo, S. (2013). UNESCO policy guidelines for mobile learning. UNESCO report.
Williams, K. C. & Williams, C. C. (2011). Five key ingredients for improving student motivation. Research
in Higher Education Journal, 121–123.
Yau, J. (2011). A mobile context-aware learning schedule framework with Java learning objects. Ph.D.
dissertation, University of Warwick, UK.
Yau, J. Y.-K., & Joy, M. (2011). A context-aware personalised m-learning application based on m-learning
preferences. International Journal of Mobile Learning and Organisation, 5(1), 1–14.
Yau, J., Gil, D., Milrad, M., Petersson, O., Salavati, S., & Sollervall, H. (2011). Identifying the potential
needs to provide mobile context-aware hints to support students’ learning. In International conference
on computers in education (pp. 365–372).
Yau, J., & Joy, M. (2008). A self-regulated learning approach: mobile context-aware learning schedule.
International Journal of Interactive Mobile Technologies, 2(3), 52–57.
Yau, J. & Joy, M. (2009). A mobile context-aware framework for supporting self-regulated learners. In
Kinshuk, D. G. Sampson, J. M. Spector, P. Isaı́as & D. Ifenthaler (Eds.), IADIS cognition and
exploratory learning in digital age (pp. 415–418).
Yau, J. & Salavati, S. (2011). Researchers’ and teachers’ perspectives on the use of mobile technologies for
inquiry-based learning. In I. Sanchez et al. (Eds.), IADIS mobile learning (pp. 136–143).
Zhou, R. & Rechert, K. (2008). Personalization for location-based e- learning. In Next generation mobile
applications, services and technologies, 2008. NGMAST’08 (pp. 247–253).

123