Computers & Education 102 (2016) 79e89

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

The use of a mobile learning management system and

academic achievement of online students
Insook Han a, Won Sug Shin b, *
a
Department of Teaching & Learning, Temple University, Philadelphia, PA 19122, USA
b
Department of Education, Korea University, 1, 5-Ka, Anam-dong, Seongbuk-Gu, Seoul 136-075, South Korea

a r t i c l e i n f o a b s t r a c t

Article history: Mobile learning has become widespread, and higher education institutions have started
Received 6 August 2015 adopting mobile technology to cope with the needs of students. Despite its adoption in
Received in revised form 14 July 2016 higher education settings, little research has been done to examine factors influencing the
Accepted 16 July 2016
adoption of mobile learning management systems (LMSs) and the learning effects on
Available online 18 July 2016
students' academic achievement. To explore the relationships among factors and the
educational effectiveness of mobile LMSs, students' demographic backgrounds (age and
Keywords:
employment status), self-reported psychological data (self-efficacy, innovativeness,
Mobile learning
Mobile LMS
perceived ease of use, and perceived usefulness of mobile LMSs), and external factors
Online students' academic achievement (subjective norms) were collected from 1604 students from 10 repetitions of the same
Post-secondary education course in an online university in Korea, in addition to their test scores. The logistic
regression results showed that age and employment status were significant factors in
predicting students' adoption of mobile LMSs and that there were potential connections
between mobile LMS use and students' gender, age, and psychological characteristics. In
addition, the study demonstrated that the use of a mobile LMS positively influenced online
students' academic achievement. The findings from this empirical study present a better
understanding of students' usage of mobile devices in higher education.
© 2016 Elsevier Ltd. All rights reserved.

1. Introduction

Mobile learning has been defined as learning facilitated by mobile devices such as mobile phones, tablet PCs, and personal
media players (Herrington & Herrington, 2007; Valk, Rashid, & Elder, 2010) in both formal and informal educational settings
(Quinn, 2011; Traxler, 2010). Mobile learning has become widespread as the development of mobile devices with advanced
wireless communication technology has encouraged learning “on the move,” using mobile devices in educational settings. It
allows students to access learning content from various locations and times (Jones, Scanlon, & Clough, 2013; Hyman, Moser, &
Segala, 2014; Garcia-Cabot, de-Marcos, & Garcia-Lopez, 2015), and share learning contents with others (Woodill, 2011).
This technological innovation has encouraged higher education institutions to increase the use of mobile technology to
fulfill their students' expectations and needs. At present, many undergraduate students bring their own digital devices to
university, especially small, portable ones such as smartphones and tablets (Dahlstrom, Walker, & Dziuban, 2012; O'Bannon &
Thomas, 2015), and they expect to access academic resources using their mobile devices. Additionally, according to the

* Corresponding author.
E-mail addresses: hanis79@gmail.com (I. Han), wss2105@gmail.com (W.S. Shin).

http://dx.doi.org/10.1016/j.compedu.2016.07.003
0360-1315/© 2016 Elsevier Ltd. All rights reserved.
80 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89

statistics from the US Department of Education, the population of online students has dramatically increased. In 2012, about
5.5 million students were enrolled in online courses in the US, which was roughly one quarter of the total enrollment in
higher education institutions (Straumsheim, 2014). Among those 5.5 million students, about 2.6 million were enrolled in fully
online programs (i.e., they did not physically attend classes at all). This phenomenon of an increasing number of students
taking online courses can also be found internationally (Dahlstrom et al., 2012). To serve this growing population of on-
campus mobile users and increase their access to learning contents and activities, many higher educational institutions
have endeavored to develop mobile learning management systems (LMSs) that provide similar functions to a traditional, PC-
based LMS but via mobile devices. Their functions include accessing course materials and grades, sharing resources with
other students or instructors, uploading assignments, and collaborating with classmates. Moreover, a mobile LMS has the
advantage over a traditional LMS in that students are able to access their courses anytime and anywhere by taking advantage
of the unique features of mobile devices (Lowenthal, 2010).
Regardless of the recent deployment of formal learning management systems in a mobile context, however, research on
mobile learning still remains focused on exploring the potential of informal mobile learning (Chen & Denoyelles, 2013;
Hwang & Chang, 2011; Jones et al., 2013; Martin & Ertzberger, 2013) and on its use as a support tool in formal learning
delivered at least partially by other means (e.g., as traditional online or face-to-face learning) (Gikas & Grant, 2013). Among
several previous studies that have addressed the provision of mobile learning as a tool to provide full access to academic,
social, and administrative materials in the same way as a traditional LMS, only a few studies have examined students' per-
ceptions and behaviors when using this new technology (Cavus, 2011; Han & Han, 2014; Mo €dritscher, Neumann, & Brauer,
2012). For example, Cavus (2011) examined students' perspectives on the use of mobile LMSs in higher education settings,
and found that most of the students had favorable perceptions toward the use of mobile devices in accessing an LMS. In a
further examination of the differences in perceptions between users and non-users of mobile LMSs, one study revealed that
even while acknowledging the advantages of mobile LMSs, non-users were reluctant to adopt them due to their perception
that they were overly complex, and thus were psychologically resistant to using them (Han & Han, 2014). Meanwhile, users
adopt the use of a mobile LMS because they perceive the new system to be less challenging and complex. The reasons users
and non-users differ in terms of their perception of the challenges of mobile LMSs can depend on individual backgrounds and
psychological characteristics as well as external factors; however, this has yet to be explored in detail. Since the mere ex-
istence of a mobile LMS does not guarantee its use by students, further investigation of who chooses to use a mobile LMS and
what characteristics they have is necessary. This study thus aims to examine (1) which factors affect the use of mobile LMSs
and (2) the relationships that exist among these factors.
Additionally, since mobile learning in higher education is still in its early stages (Cheon, Lee, Crooks, & Song, 2012; Hwang
& Wu, 2014; Park, 2011), few studies have reported how mobile LMS use affects students' academic achievement in formal
educational settings. Hence, more research needs to be conducted to explain why students use mobile LMSs in formal ed-
ucation and how mobile LMSs influence student academic achievement.
Considering the above-mentioned research needs, the purpose of this study is to examine factors influencing the use of
mobile LMSs by online students. More specifically, this study investigates influential variables and the relationships among
them by comparing regression models including different sets of variables, such as students' individual backgrounds and
psychological characteristics, as well as external factors. In addition, this study also examines the effect of using mobile LMSs
on online students' academic achievement.

2. Background

2.1. Factors affecting the use of mobile LMSs

Since the use of mobile LMSs in formal learning settings is a relatively new technological advancement, until now, few
studies have explored the factors that can affect their use. However, some studies have examined factors influencing the
acceptance of various new technologies such as studies that applied the Technology Acceptance Model (TAM) framework
originally proposed by Davis in 1989. The TAM emphasizes the perceived usefulness and perceived ease of use as determinant
factors that influence the behavioral intention of using new technologies (Davis, 1989). This model has been extended by
various other researchers, who studied the acceptance and usage of various forms of technologies. For example, technological
complexity, computer self-efficacy, and organizational support were claimed to be included in the model (Teo, 2010). Sub-
jective norms and system accessibility (Park, 2009), as well as personal innovativeness (Liu, Li, & Carlsson, 2010) were also
explored as factors to be considered in understanding technology adoption.
Among the various previous studies that applied the TAM or the extended TAM in technology adoption, studies focusing
on mobile learning in general may provide us insight into the factors that might affect the use of mobile LMSs. The most
examined factors in this regard have been users' psychological characteristics and external factors. Previous studies have
found that the users' psychological characteristics, including self-efficacy, innovativeness, perceived usefulness, perceived
ease of use, and attitude toward technology are significant predictors of the use of mobile devices in learning. For example,
Park, Nam, and Cha (2012) investigated the intention to engage in mobile learning among Korean college students and
determined that attitude was the most important construct in explaining the acceptance of mobile learning. Meanwhile, Liu
et al. (2010) found that perceived usefulness and innovativeness were the two most influential factors in adopting mobile
learning among Chinese college students. These factors have also been recognized as influential in other studies (e.g., Cheon
 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89 81

et al., 2012; Huang, Hsiao, Tang, & Lien, 2014). Furthermore, the Shin and Kang (2015) found that learners' self-efficacy toward
using mobile devices increased the use of mobile devices in their learning. According to these previous studies, the more
positive the attitude toward technology, the more self-efficacy and innovativeness learners have; moreover, the greater the
perceived usefulness and ease of use of mobile devices, the more likely students are to adopt mobile devices (including
mobile LMSs) in their learning.
In addition to psychological characteristics, the social and environmental factors around users can also influence their
decision on the adoption of mobile learning. Subjective norms and perceived systems accessibility are external factors that
have been found to be influential in previous mobile learning adoption research. “Subjective norms” refers to the perceived
social pressure to adopt mobile learning (Huang et al., 2014; Shin & Kang, 2015), whereas perceived systems accessibility, for
instance perceived institutional financial support for mobile learning initiatives (McGill, Klobas, & Renzi, 2014) and systems
accessibility (Shin & Kang, 2015) also affect the adoption of mobile learning. Since mobile LMSs are provided by institutions
free of charge, we will only consider systems accessibility as a potential factor. Hence, it can be hypothesized that the greater
the perceived social pressure and accessibility of mobile LMSs, the more likely learners are to use mobile LMSs in higher
educational settings.
Finally, online students' individual backgrounds are another factor that might affect their use of mobile LMSs. According to
previous literature, US graduate students were enrolled in fully online programs at double the rate of undergraduate students
in 2012 (Straumsheim, 2014). In a fairly similar finding, students in South Korean online universities are mostly adult learners
with full-time jobs and a wider age range than university students in general (Han & Han, 2014). Thus, age and employment
status may affect the use of mobile LMSs as well.
As discussed, some studies have examined the factors influencing the use of mobile devices in students' learning in general
but not many studies have specifically focused on the use of mobile LMSs. In addition, those studies that investigated various
factors in mobile learning have not explored the relationship among factors and the possible interplays that the factors have
in predicting the use of mobile LMSs. Thus, this study not only investigated the influential variables in mobile LMS use but also
the relationships among them by comparing regression models including different sets of variables, such as students' indi-
vidual backgrounds and psychological characteristics, as well as external factors. Even though there have been many studies
examining factors influencing the use of new technologies in education, very few studies have attempted to differentiate the
effects of specific factors by controlling others or to examine the relationships among variables.

2.2. Mobile LMSs and academic achievement

Mobile devices are considered promising learning tools that can support learners' needs and participation in unique ways
(Hwang & Wu, 2014) and engage them in educational activities (Wu et al., 2012). However, to the best of our knowledge, there
is still no empirical research investigating the relationship between college students' acceptance of mobile learning and their
academic achievements: the majority of studies examining mobile learning have instead focused on factors affecting users'
intentions to adopt mobile learning (Cheon et al., 2012). Indeed, one review of published articles on mobile learning and
learning achievement showed no empirical data supporting the assertion of whether mobile learning is related positively or
negatively to students' learning achievement (Hwang & Wu, 2014). In this absence of hard evidence, there is a concern that
mobile devices might distract students' attention from learning content (Gehlen-Baum & Weinberger, 2014).
Despite the lack of empirical research and doubts concerning its effectiveness, mobile learning is still considered to have
potential due to its accessibility, flexibility, ease of assessment and feedback, and access to online repositories and com-
munities of practice (Jacob & Issac, 2008a; O'Bannon & Thomas, 2015). Thus, empirical studies are needed to further confirm
the effects of mobile device use on learners' academic achievement (Zydney & Warner, 2016). Mobile LMSs specifically
provide students with unique opportunities to view lectures, participate in discussion, interact, and share ideas with others
anywhere and anytime. Considering that the majority of students are cell phone users, as mentioned above (Smith, Rainie, &
Zickuhr, 2011), higher education institutions are likely to remain interested in providing a mobile LMS and determining the
impact that the use of this system will have on their students' academic achievement. There is at least one extant study that
has examined the structural relationships among individual, social, and systemic factors affecting students' behavioral in-
tentions to use mobile LMSs and their influence on students' satisfaction with and academic achievement in online learning
(Shin & Kang, 2015). The authors of that study found that the intention to use mobile LMSs is related to students' learning
satisfaction, which in turn predicts academic achievement. However, this study did not investigate whether mobile LMS use
has a direct relationship with student academic achievement, that is, when controlling other potentially influential variables.

2.3. Research questions

Based on the literature review focusing on potential variables that might have an influence on mobile learning and the use
of mobile LMSs, we selected variables that have been examined and proved to be consistently influential in the adoption of
mobile learning in previous studies. In order to examine and confirm whether those variables still have an influence on
mobile LMS use in our own research context, the current study investigated: (1) the factors that influence the adoption of
mobile LMSs, and (2) the impact that the use of mobile LMSs has on students' academic achievement when controlling other
influential factors. Therefore, the following research questions, which are twofold, need to be asked in this study:
82 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89

1) What are the relationships among factors that influence the adoption of mobile LMSs? Here, we hypothesize that there
would be possible interplays among individual background, psychological, and external factors in influencing the adoption
of mobile LMSs.
2) Will the use of mobile LMSs have a positive influence on students' academic achievement? Due to accessibility and
flexibility of time and space for learning, we hypothesize that the use of a mobile LMS will be positively related to students'
academic achievement.

3. Methods

3.1. Data

Data were generated from a population of 2000 online undergraduate students enrolled in 10 different sessions of the
same undergraduate course in an online university in South Korea. The course was Introduction to Cyber Learning, a required
course for all students, to be completed before graduation. Due to the limited capacity of 200 students per online course, the
university needs to open multiple sections of this course to accommodate student demand. For quality control purposes, all 10
repetitions of the same course were designed and taught by the same instructor and had the same learning materials, ac-
tivities, and assessment methods.
A survey instrument was distributed using the university's LMS at the end of the spring semester in 2014. Students were
asked to participate voluntarily in this survey, and 1608 students responded. Four invalid responses (due to many missing
answers) were excluded from the final analyses. Among the 1604 respondents, 1115 were mobile LMS users and 489 were not.
There were 697 male students, 1116 students employed full-time, 133 employed part-time, and 259 unemployed. Since it was
a compulsory course, the survey participants were students from six different departments with 16 different majors (see
Table 1).
Along with the survey, their mid-term and final exam scores were collected from the LMS but any identifying information
was removed. The data collection process proceeded with permission from survey participants, lecturers, and the educational
institution.

3.2. Mobile LMSs in online universities

The university where this study was conducted is one of 13 online universities accredited by the Korean Ministry of
Education as legitimate higher educational institutions, and it offers degrees completely online. This particular university
offers bachelor degrees in 16 different majors in six different departments.
Recently, online universities introduced mobile LMSs, although not all functions in the traditional, PC-based learning
management system (LMS) were implemented in the early stages. Mobile LMSs are applications that can be downloaded onto
individual students' mobile devices. They allow students to watch video lectures and participate in class activities with
increased accessibility to learning resources via mobile devices, overcoming previous limitations caused by mobile and web
version incompatibilities. Moreover, mobile LMSs enable enhanced resolution and simplified screen navigation that optimize
students' experiences of using LMSs on mobile devices. Mobile LMSs have evolved according to what was learned from trial
applications. The features of current mobile LMSs include attending online courses, posting questions, checking messages,
and monitoring academic calendars. Moreover, the mobile application is connected with a traditional LMS; that is, any

Table 1
Survey participants' majors.

Schools/Majors Frequency Percent

School of Social Welfare Welfare Facility Management 49 3.1
Social Welfare 320 20.0
Elderly Welfare 35 2.2
School of Counseling And Psychology Family Counseling 68 4.2
Military & Police Counseling 122 7.6
Counseling Psychology 421 26.2
School of Social Science Healthcare Administration 89 5.5
Real Estate 96 6.0
Law & Public Administration 59 3.7
School of Business Business Administration 114 7.1
Finance Insurance 51 3.2
International Trade and Logistics 39 2.4
School of IT & Design Computer & Information Communication 67 4.2
New Media Contents Technology 11 0.7
Multimedia Design 23 1.4
School of Culture and Arts Culture & Art Management 28 1.7
No response 12 0.7
Total 1604 100.0
 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89 83

learning activity that occurred in the mobile LMS is recorded as regular attendance and participation in the traditional one.
Therefore, learners can be marked as attending online courses and posting messages on bulletin boards while commuting or
at any other time when they are available.
Since the entire teaching and learning experience is only delivered online, the use of a traditional LMS is required for
students. All students in online universities are required to use the PC-based LMS, and after being introduced to the mobile
LMS, students can choose whether they wish to adopt its use for their own learning.

3.3. Survey instrument

In order to answer the research questions, this study partially developed survey items as well as some previously
developed items. The first section of the survey asked questions concerning the students' individual backgrounds such as age,
gender, year of college, employment status, and use of mobile LMSs. The question concerning employment status had three
categories: full-time employed, part-time employed, or unemployed. The question concerning the use of mobile LMSs
consisted of a yes/no question that asked whether the student had been using a mobile LMS for the course in which the survey
was conducted.
The second section of the survey included items that were adopted from previous studies and these items measured the
psychological and external factors that predict people's acceptance and use of new technology. The items were reworked to
focus on mobile LMSs specifically. All survey items were answered using a five-point Likert scale. The psychological factors
included two questions on self-efficacy and three questions on students' own innovativeness, attitude toward (mobile LMS)
technology, and perceived usefulness and ease of use of mobile LMSs. The external factors comprised three questions each for
subjective norms and system accessibility. In order to conduct the data collection among the Korean student population, the
English survey items were translated into Korean. Three experts in related fields reviewed each question to verify the content
validity for the translated version. In addition, a reliability test for the survey items was conducted. The definitions, sources,
and reliability scores for each construct are provided in Table 2.

3.4. Analysis

Research question 1: What are the relationships among factors that influence the adoption of mobile LMSs?
To answer research question 1, we analyzed the likelihood of mobile LMS use using logistic regression with three esti-
mated models. The first model, which only included students' individual background variables, examined individual dif-
ferences in the likelihood of using a mobile LMS. The second model added psychological characteristics for adopting new
technologies to examine how these characteristics were related to the likelihood of using a mobile LMS. The last model added
external factors, such as the influence of subjective norms and perceived systems accessibility on the adoption of new
technologies. This phased, modeling process allowed us to more systemically identify the factors influencing the use of a
mobile LMS by examining the relationships among the variables.
Research question 2: Will the use of mobile LMSs have a positive influence on students' academic achievement?
To test research question 2, we conducted an ordinary least squares (OLS) regression analysis to examine how the use of a
mobile LMS is related to students' final exam scores. We controlled the students' individual background, psychological
characteristics, and external factors to investigate the net effect of mobile LMS use; we also controlled the students' midterm
scores to estimate the “value added” effect of the use of a mobile LMS.

Table 2
Survey instrument.

Construct Operational definition Items Source Alpha

Self-efficacy Students' ability to use a mobile LMS to accomplish a learning job or task 2 Cheon et al. 0.862
e.g., I am confident about using a mobile device for my courses. (2012)
Innovativeness Willingness to adopt a mobile LMS before others 3 Van Braak (2001) 0.856
e.g., I like to experiment with new information technologies.
Attitude toward technology Individual's positive or negative feelings concerning the use of a mobile LMS 3 Cheon et al. 0.878
e.g., Using m-learning in my coursework would be a pleasant experience. (2012)
Perceived usefulness Degree to which a student believes using a mobile LMS will enhance his or her learning 3 Liu et al. (2010) 0.850
e.g., I think using a mobile LMS can increase the efficiency of my studies and work.
Perceived ease of use Degree to which a student believes using a mobile LMS would be simple and 3 Liu et al. (2010) 0.841
straightforward
e.g., I think using a mobile LMS is easy.
Subjective norm Perceived social pressure to engage or not engage in using a mobile LMS 3 Park et al. (2012) 0.853
e.g., I need to experience mobile learning for my future job.
Perceived systems Extent to which students are granted stable access to a mobile LMS 3 Park et al. (2012) 0.795
accessibility e.g., Mobile devices have good compatibility with other computer devices.
84 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89

4. Results

4.1. Descriptive findings

Table 3 presents descriptive statistics for the variables measured. Of the 1115 students who reported that they had used a
mobile LMS, 54% were female. In contrast, 63% of the students who reported that they had not used a mobile LMS were
female. With respect to age, the results showed an age difference favoring younger studentsdfor example, more students
under 50 reported that they had used a mobile LMS than students over 50, and 40% of those who reported that they had not
used a mobile LMS were in their 40s. In addition, more students who had a full-time job reported that they had used a mobile
LMS than those who had a part-time job or were unemployed. Finally, students who reported to have used a mobile LMS
showed higher scores on all psychological characteristics and external factors. In sum, descriptive statistics showed a greater
tendency of self-reporting on mobile LMS use among younger, male, and full-time employed students, and those who showed
higher scores on psychological and external factors (than among older, unemployed female students with lower scores on
these factors). As for academic achievement, those who reported to have used a mobile LMS achieved similar, or somewhat
higher, scores on midterm and final exams than those who did not. In order to distinguish whether these differences were
significant, we conducted a statistical analysis by using logistic regression and an OLS regression.
Research question 1: What are the relationships among factors that influence the adoption of mobile LMSs?
The first analysis examined factors influencing the likelihood students in an online university would adopt a mobile LMS.
The first model consisted of a set of individual background variables, among which only gender, age, and employment status
were significant predictors of the students' likelihood of adopting a mobile LMS (see Table 4). Regarding gender, male stu-
dents were 24% more likely to use a mobile LMS than female students. Moreover, the odds of students in their 20s using a
mobile LMS were 29% and 56% higher than students in their 40s and in their 50s or above, respectively. In terms of

Table 3
Descriptive statistics.

Mobile learning Total

Yes No

Mean or proportion SE Mean or proportion SE Mean or proportion SE

Individual background
Female** 0.54 0.63 0.57
Age***
29 or below 0.25 0.20 0.24
30e39 0.30 0.21 0.27
40e49 0.35 0.40 0.36
50 or above 0.10 0.19 0.13
Grade level
Freshmen 0.35 0.35 0.35
Sophomores 0.06 0.06 0.06
Juniors 0.43 0.43 0.43
Seniors 0.17 0.16 0.17
Employment status***
Full-time 0.76 0.66 0.73
Part-time 0.11 0.11 0.11
Unemployed 0.13 0.23 0.16

Psychological characteristics
Self-efficacy*** 11.07 2.63 9.09 2.73 10.46 2.81
Innovativeness*** 10.69 2.58 8.83 2.46 10.13 2.68
Ease of use*** 10.79 2.40 8.97 2.44 10.23 2.55
Usefulness*** 11.46 2.25 10.38 2.30 11.13 2.32
Attitude*** 10.71 2.41 9.69 2.33 10.40 2.43

External factors
Subjective norm*** 11.47 2.51 10.56 2.48 11.19 2.53
Systems accessibility*** 17.64 4.09 16.46 3.66 17.28 4.00

Achievement
Mid-term 15.46 4.68 15.25 4.60 15.39 4.66
Final 23.46 8.89 22.73 9.45 23.24 9.07
N 1115 489 1604

**, *** denote significant differences at p < 0.01 and p < 0.001 (two-tailed tests), respectively.
 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89 85

Table 4
Factors influencing the likelihood of using a mobile LMS: Logistic regression results.

Model 1 Model 2 Model 3

B SE OR B SE OR B SE OR
Individual background
Female
0.27* 0.12 0.76
0.01 0.13 0.99
0.01 0.13 0.99
Age
29 or below (reference group)
30e39 0.14 0.17 1.15 0.18 0.18 1.19 0.15 0.18 1.16
40e49
0.35* 0.15 0.71
0.26 0.16 0.77
0.29 0.16 0.75
50 or above
0.83*** 0.19 0.44
0.60** 0.21 0.55
0.64*** 0.21 0.53
Grade level
Freshmen (reference group)
Sophomores
0.16 0.25 0.85
0.19 0.27 0.82
0.17 0.27 0.84
Juniors
0.02 0.13 0.98 0.06 0.14 1.06 0.07 0.14 1.07
Seniors
0.04 0.17 0.96 0.16 0.18 1.18 0.20 0.18 1.22
Employment status
Full-time (reference group)
Part-time 0.00 0.18 1.00 0.02 0.19 1.02 0.02 0.19 1.02
Unemployed
0.60*** 0.15 0.55
0.64*** 0.16 0.53
0.65*** 0.16 0.52

Psychological characteristics
Self-efficacy 0.11** 0.03 1.12 0.11*** 0.03 1.12
Innovativeness 0.14*** 0.03 1.15 0.13*** 0.03 1.14
Ease of use 0.12** 0.04 1.13 0.12*** 0.04 1.13
Usefulness 0.07 0.04 1.08 0.08** 0.05 1.08
Attitude
0.01 0.04 0.99 0.00 0.04 1.00

External factors
Subjective norm 0.06* 0.03 1.06
Systems accessibility
0.04 0.02 0.96

Constant 1.32 0.15 e
3.29*** 0.39 e
3.12*** 0.41 e

Log likelihood
953.772
845.211
843.105
Pseudo R2 0.033 0.143 0.145

***p < 0.001, **p < 0.01, *p < 0.05 (two-tailed tests).

employment status, students with full-time jobs were 45% more likely to use a mobile LMS than those unemployed. The
pseudo-R2 of the first model was 0.033.
In Model 2, in which the psychological characteristics of students were taken into account, significant differences in the
likelihood of using a mobile LMS were observed between students in their 20s and students in their 50s or older, as well as
between full-time employed and unemployed students. However, when comparing Model 2 with Model 1, the differences
between male and female gender, and between the 20s and 40s age range became non-statistically significant after con-
trolling these psychological characteristics. Among the psychological characteristics, self-efficacy, innovation, and ease of use
were significant predictors of the likelihood of use of a mobile LMS by positively influencing the usage decision. Model 2
increased the pseudo-R2 from 0.033 to 0.143.
In Model 3, the external factors were also included. Among them, the subjective norm was found to be a significant
predictor of mobile LMS use among students. Even after controlling the subjective norm variable and when comparing Model
2 with Model 3, the observed differences between students in their 20s and in their 50s or above, and between full-time
employed and unemployed students, remained significant in Model 3. So were the effects of self-efficacy, innovation, and
ease of use. More specifically, students in their 20s were 47% more likely to use a mobile LMS than students in their 50s or
above, while full-time employed students were 48% more likely to use a mobile LMS than unemployed students. Furthermore,
one-unit changes in the psychological independent variables of self-efficacy, innovativeness, and perceived ease of use
increased the odds of using a mobile LMS by 12%, 14%, and 13%, respectively. In addition, the perception of usefulness was
found to be a significant predictor that increased the odds of mobile LMS use by 8% with a one-unit change in usefulness.
Finally, a one-unit change in the subjective norm, an external factor, increased the odds of mobile LMS use by 6%. Overall,
Model 3 increased the pseudo-R2 from 0.143 to 0.145. Table 4 displays the results of the logistic regression analyses:
Research question 2: Will the use of mobile LMSs have a positive influence on students' academic achievement?
The second research question examined the extent to which the academic achievement of online university students
benefited from the use of mobile LMSs. Results of an OLS regression predicting students' final scores, displayed in Table 5,
showed that the use of a mobile LMS was marginally associated with higher final exam scores after controlling students'
individual background, psychological characteristics, external factors, and midterm scores at the 0.10 significance level, with a
86 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89

Table 5
The use of a mobile LMS and Academic achievement: Ordinary least squares results.

Model 1 Model 2 Model 3 Model 4

Coef. SE Coef. SE Coef. SE Coef. SE

Mobile 0.49 0.41 0.68y 0.41 0.71 0.44 0.44
Midterm 1.03*** 0.05 1.02*** 0.05 1.02*** 0.05 0.05
Student characteristics
Female 1.04** 0.39 0.89** 0.40 0.88* 0.41
Age
29 or below (reference group)
30e39 1.30* 0.53 1.26* 0.53 1.28* 0.53
40e49 1.55** 0.49 1.44** 0.50
1.46** 0.51
50 or above 2.15** 0.64 1.96** 0.67 1.97** 0.68
Grade level
Freshmen (reference group)
Sophomores 0.96 0.84 0.93 0.84 0.90 0.84
Juniors
0.45 0.43
0.47 0.43
0.48 0.43
Seniors
1.25* 0.56
1.27* 0.56
1.30* 0.56
Employment status
Full-time (reference group)
Part-time
0.24 0.61
0.31 0.61
0.30 0.61
Unemployed
0.72 0.53
0.66 0.53
0.66 0.53

Psychological characteristics
Self-efficacy 0.11 0.10 0.11 0.10
Innovativeness
0.03 0.10
0.03 0.10
Ease of use
0.25* 0.12
0.25* 0.12
Usefulness 0.28* 0.13 0.27 0.14
Attitude
0.08 0.12
0.10 0.11

External factors
Subjective norm
0.03 0.11
Systems accessibility 0.04 0.74

Constant 7.19*** 0.86 5.94*** 0.98 5.47*** 1.41 5.45*** 1.43

R2 0.2140 0.2286 0.2326 0.2327

***p < 0.001, **p < 0.01, *p < 0.05, yp < 0.10 (two-tailed tests).

coefficient value of 0.73 (Model 4). In order to examine whether different groups of variables influence the effect of using a
mobile LMS on students' academic achievement, we sequentially added individual background, psychological characteristics,
and external factors into the models. Model 1, which included the use of a mobile LMS and midterm scores, showed that
midterm scores were positively associated with students' academic achievement. This model explained about 21% of the total
variance in students' academic achievement. In contrast to Model 1, Model 2 included individual background factors and
revealed that mobile LMS use was marginally associated with higher achievement. However, adding individual background
factors to Model 1 increased the explained variance of students' academic achievement by only 1%. Likewise, in Models 3 and
4, when psychological and external factors were added to the model, each only increased the explained variance of students'
academic achievement by 1% or less. This means that individual background, psychological characteristics, and external
factors did not have any influence on the effect of using a mobile LMS on academic achievement.

5. Discussion

The purpose of this study was to examine factors influencing mobile LMS use among students at an online university in
South Korea. In addition, this study further investigated the benefits of using mobile LMSs on the academic achievement of
online students.
The first hypothesis was that individual background, psychological characteristics, and external factors would all affect the
use of a mobile LMS. From the last regression model, Model 3, which was best able to predict the use of a mobile LMS, this
study found that self-efficacy, innovativeness, perceived ease of use, and perceived usefulness among the psychological
factors and the subjective norm among the external factors were significant predictors of mobile LMS use among the students
surveyed. As presented in the Results section, one-unit changes in self-efficacy, innovativeness, perceived ease of use, and
perceived usefulness increased the odds of mobile LMS use by 12%, 14%, 13%, and 8% respectively. Furthermore, a one-unit
change in the subjective norm increased the odds of mobile LMS use by 6%. This result implies that self-efficacy, innova-
tiveness, and perceived ease of use are, in general, stronger predictors than psychological characteristics and external factors.
It is also consistent with previous studies highlighting the importance of psychological factors on the adoption of new
technology (e.g., Teo, 2010). For example, this study reconfirmed the importance of self-efficacy (Shin & Kang, 2015),
 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89 87

innovativeness (Liu et al., 2010), and perceived ease of use (Cheon et al., 2012). Liu et al. (2010) stressed that personal
innovation influenced the adoption of m-learning when they investigated the factors of m-learning adoption. In addition, it
implies that a new mobile learning system should be easy to use, considering the technical limitations of devices, and higher
levels of self-efficacy lead to higher levels of behavioral intention, as Cheon et al. (2012) mentioned.
Among the individual background factors, age and employment status were significant factors predicting adoption of
mobile LMSs. Considering that students in online universities are mostly adult learners from a wider age range and tend to be
employed full-time (Han & Han, 2014), it is expected that mobile usage patterns will vary among them. It was found in this
study that students in their 50s or above used mobile LMSs significantly less compared to students in their 20s, a difference
remaining significant after controlling other factors and therefore solely accounted for by age. Previous studies have
mentioned small screen size as a potential disadvantage of mobile device use (Jacob & Issac, 2008b). One possible reason for
this difference might be worsening eyesight with age. In addition, students with a full-time job used mobile LMSs significantly
more than those without a job. Physical mobility is one of the biggest advantages of using a mobile LMS, which enables full-
time workers to access lectures and learning activities anytime and anywhere (e.g., while commuting or on a work break)
(Han & Han, 2014). Considering that students in their 20s and those who were full-time employed had 47% and 48% greater
odds of using a mobile LMS, respectively, than their counterparts, student age and employment status were shown to be the
strongest predictors of mobile LMS use among the individual, psychological, and external factors examined by this study.
From the results of the first analysis, we further investigated the relationships among variables by comparing three logistic
regression models adding sets of variables. By comparing models, we could identify that some significant predictors in the
previous model lost statistical significance after controlling other variables that were newly added into the following models.
This means that there is a possible interplay between factors having changed in their significance level and the factors
included in the new model. More specifically, we identified the significance of gender and age differences between students
in their 20s and 40s in our first logistic regression model, however, both were no longer found to be statistically significant
after controlling the psychological factors in the second model. Female students seemed to use mobile LMSs less than male
students in the first model, but after three significant psychological factors (i.e., self-efficacy, innovativeness, and perceived
ease of use) were controlled, it turned out that both genders used mobile LMSs at similar levels. This result seems to
demonstrate relationships between these psychological factors and gender. Self-efficacy, innovativeness, and perceived ease
of use had a positive relationship with mobile LMS use as observed in Model 2, while being a female had a negative rela-
tionship with mobile LMS use as observed in Model 1. Considering that the level of female students' mobile LMS use increased
from Model 1 to Model 2 after controlling the abovementioned psychological variables that had a positive relationship with
mobile LMS use, it implies that female students tend to have low levels of self-efficacy, innovativeness, and perceived ease of
use, and these factors might discourage them from using a mobile LMS. A similar explanation can account for the difference
between students in their 20s and in their 40s: students in their 40s had lower self-efficacy, innovativeness, and perceived
ease of use than those in their 20s. However, as discussed previously, the difference between students in their 20s and those
in their 50s or above remained significant after controlling psychological and external factors (although the coefficient value
decreased), which means that there are other reasons that account for older students' reluctance to use mobile LMSs, which
may potentially include physical limitations, as suggested above.
Second, we hypothesized that the use of mobile LMSs would have a positive relationship with students' academic
achievements due to the unique features of mobile devices. Even though it was not statistically significant, with a significance
level of 0.05, the use of mobile LMSs was marginally associated with online students' academic achievement after controlling
the variables related to individual background, psychological characteristics, external factors, and midterm scores (as a
baseline measure of academic achievement). However, we did not find any interactional relationship between factors and the
use of mobile LMSs in influencing students' academic achievement. As expected, based on previous studies (Jacob & Issac,
2008a), the use of mobile LMSs may have increased the accessibility of online course materials and activities, and the flex-
ibility of being able to use an LMS while on the move may have also been beneficial (O'Bannon & Thomas, 2015). This unique
learning experience could allow students to plan and execute a study routine even when they are not in front of a computer,
which could ultimately affect their academic achievement. However, since the results of this study only showed a weak
association between mobile LMS use and students' academic achievement, a cautious interpretation is needed. As discussed
in a previous study (Han & Han, 2014), in addition to the flexibility of being able to use an LMS anytime and anywhere, mobile
LMSs could be designed to have more advantages drawn from the uniqueness of mobile devices, which may improve stu-
dents' learning. For example, by possessing particular functions such as planning, monitoring, self-assessment, interaction/
feedback, or time management, using a mobile LMS could support students' self-directed learning with increased meta-
cognitive activities (Chung, 2009), which may ultimately enhance their learning.

6. Conclusion

The current study provides empirical evidence that enables a better understanding of online students' usage of mobile
devices. The study not only examined factors influencing the use of mobile LMSs, but also attempted to investigate the re-
lationships among variables pertaining to individual background, psychological characteristics, and external factors. By
analyzing the relationships among variables more closely, we can gain insights into how to best interpret individual students'
decisions to use mobile LMSs and determine how institutions can help. For example, the reasons why there is a dispropor-
tionate lack of mobile LMS use among female and middle-aged students is probably due to their lack of innovativeness and
88 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89

self-efficacy in using new technology. Therefore, more institutional support to provide them with appropriate training and
opportunities to discover the advantages of using a new system might be necessary (Kang & Shin, 2015). Second, as discussed
above, whether the use of mobile LMSs has a positive relationship with online students' academic achievement is an
important question, since mobile technology has been widely adopted in education with the expectation that it can bring
such an advantage. However, numerous variables can influence academic achievement, and thus it may be difficult to confirm
the effects of mobile technology on learning as a result. Therefore, this study attempted to control other variables (i.e., in-
dividual, psychological, and external factors) and clarify the relationship between the use of a mobile LMS itself and academic
achievement. With reference to these empirical results, this study contributes to a deepening of our understanding of how to
integrate mobile LMSs into higher education and the effects of LMS use on online learning.
Although this study yields significant implications for the development of guidelines to support mobile learning in higher
education, it also has limitations in terms of the generalizability of the results. First, the study was conducted in a South
Korean online university where students complete their coursework exclusively online. Since LMSs are more critical in online
universities than in regular higher education institutions, the results may not be transferrable to other educational systems.
Therefore, further studies need to be conducted in different educational contexts (and include other variables that are
influential in those specific contexts) in order to examine different usage patterns among students. Second, this study only
examined the effect of using mobile LMSs on academic achievement in a general sense, and did not specifically investigate
how much time students spent on their studies and which tools and functions of the mobile LMS the students used and did
not use, all of which might have also affected their achievement. Future studies can answer this question by exploring stu-
dents' behaviors within mobile LMSs using log data. Finally, our analysis was based on self-reported information, which can
be subject to response bias. From a methodological perspective, in-depth interviews with students would be useful to
reinforce the results of this study. In addition, this study was a quasi-experimental study; therefore, we were cautious about
interpreting the results causally, instead focusing on correlation. Since the causality between the use of mobile LMSs and
academic achievement is unclear in this study, in order to examine whether the use of mobile LMSs is indeed a factor that
increases academic achievement, a controlled experimental study would need to be conducted.
Continuing advances in wireless technology enable learning on the move. Educational stakeholders such as school ad-
ministrators, teachers, and learning tool developers have tried to increase the use of mobile learning management systems to
correspond to students' needs, despite the lack of robust theoretical support and empirical data. This requires more research
on mobile learning from the perspectives of the education providers (i.e., teachers and administrators) and the consumers
(i.e., students), an end to which this study contributes.

References

Cavus, N. (2011). Investigating mobile devices and LMS integration in higher education: Student perspectives. Procedia Computer Science, 3, 1469e1474.
Chen, B., & Denoyelles, A. (2013). Exploring students' mobile learning practices in higher education. EDUCAUSE Review Online. Retrieved from http://www.
educause.edu/ero/article/exploring-studentsmobile-learning-practices-higher-education.
Cheon, J., Lee, S., Crooks, S. M., & Song, J. (2012). An investigation of mobile learning readiness in higher education based on the theory of planned behavior.
Computers & Education, 59(3), 1054e1064.
Chung, A. K. (2009). Development of a mobile platform (mobile embedded in LMS) to support self-regulated learning. Unpublished doctoral dissertation. Ewha
Womans University.
Dahlstrom, E., Walker, J. D., & Dziuban, C. (2012). ECAR study of undergraduate students and information technology, 2012. Research Report. Louisville, CO:
EDUCAUSE Center for Applied Research. Retrieved from http://www.educause.edu/eca.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 319e340.
Garcia-Cabot, A., de-Marcos, L., & Garcia-Lopez, E. (2015). An empirical study on m-learning adaptation: Learning performance and learning contexts.
Computers & Education, 82, 450e459.
Gehlen-Baum, V., & Weinberger, A. (2014). Teaching, learning and media use in today's lectures. Computers in Human Behavior, 37, 171e182.
Gikas, J., & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social
media. The Internet and Higher Education, 19, 18e26.
Han, I., & Han, S. (2014). Adoption of the mobile campus in a cyber university. The International Review of Research in Open and Distributed Learning, 15(6),
237e256, 140e157.
Herrington, A., & Herrington, J. (2007). Authentic mobile learning in higher education. In AARE 2007 international educational research conference, 28
november 2007. Fremantle: Western Australia.
Huang, R. T., Hsiao, C. H., Tang, T. W., & Lien, T. C. (2014). Exploring the moderating role of perceived flexibility advantages in mobile learning continuance
intention (MLCI). The International Review of Research in Open and Distributed Learning, 15(3).
Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of
students. Computers & Education, 56(4), 1023e1031.
Hwang, G. J., & Wu, P. H. (2014). Applications, impacts and trends of mobile technology-enhanced learning: A review of 2008e2012 publications in selected
SSCI journals. International Journal of Mobile Learning and Organisation, 8(2), 83e95.
Hyman, J. A., Moser, M. T., & Segala, L. N. (2014). Electronic reading and digital library technologies: Understanding learner expectation and usage intent for
mobile learning. Educational Technology Research and Development, 62(1), 35e52.
Jacob, S. M., & Issac, B. (2008a). The mobile devices and its mobile learning usage analysis. In , vol. 1. Proceedings of the international MultiConference of
engineers and computer scientists.
Jacob, S. M., & Issac, B. (2008b). The mobile devices and its mobile learning usage analysis. IEEE Education Society, 2(2), 19e21.
Jones, A. C., Scanlon, E., & Clough, G. (2013). Mobile learning: Two case studies of supporting inquiry learning in informal and semiformal settings.
Computers & Education, 61, 21e32.
Kang, M., & Shin, W. S. (2015). An empirical investigation of student acceptance of synchronous e-learning in an Online University. Journal of Educational
Computing Research, 52(4), 475e495.
Liu, Y., Li, H., & Carlsson, C. (2010). Factors driving the adoption of m-learning: An empirical study. Computers & Education, 55(3), 1211e1219.
Lowenthal, J. N. (2010). Using mobile learning: Determinates impacting behavioral intention. The American Journal of Distance Education, 24(4), 195e206.
Martin, F., & Ertzberger, J. (2013). Here and now mobile learning: An experimental study on the use of mobile technology. Computers & Education, 68, 76e85.
McGill, T. J., Klobas, J. E., & Renzi, S. (2014). Critical success factors for the continuation of e-learning initiatives. The Internet and Higher Education, 22, 24e36.
 I. Han, W.S. Shin / Computers & Education 102 (2016) 79e89 89

Mo€dritscher, F., Neumann, G., & Brauer, C. (2012, July). Comparing LMS usage behavior of mobile and web users. In Advanced learning technologies (ICALT),
2012 IEEE 12th international conference on (pp. 650e651). IEEE.
O'Bannon, B. W., & Thomas, K. M. (2015). Mobile phones in the classroom: Preservice teachers answer the call. Computers & Education, 85, 110e122.
Park, S. Y. (2009). An analysis of the technology acceptance model in understanding university students' behavioral intention to use e-learning. Educational
Technology & Society, 12(3), 150e162.
Park, Y. (2011). A pedagogical framework for mobile learning: Categorizing educational applications of mobile technologies into four types. International
Review of Research in Open and Distance Learning, 12(2), 78e102.
Park, S. Y., Nam, M. W., & Cha, S. B. (2012). University students' behavioral intention to use mobile learning: Evaluating the technology acceptance model.
British Journal of Educational Technology, 43(4), 592e605.
Quinn, C. N. (2011). Designing mLearning: Tapping into the mobile revolution for organizational achievement. San Francisco: Pfeiffer.
Shin, W. S., & Kang, M. (2015). The use of a mobile learning management system at an online university and its effect on learning satisfaction and
achievement. The International Review of Research in Open and Distributed Learning, 16(3).
Smith, A., Rainie, L., & Zickuhr, K. (2011). College students and technology. Washington, DC: Pew Internet and American Life Project. Retrieved from http://
www.pewinternet.org.
Straumsheim, C. (2014). Identifying the online student. Inside higher ed.. Retrieved from https://www.insidehighered.com.
Teo, T. (2010). A path analysis of pre-service teachers' attitudes toward computer use: Applying and extending the technology acceptance model in an
educational context. Interactive Learning Environments, 18(1), 65e79.
Traxler, J. (2010). Will student devices deliver innovation, inclusion, and transformation? Journal of the Research Center for Educational Technology, 6(1),
3e15.
Valk, J. H., Rashid, A. T., & Elder, L. (2010). Using mobile phones to improve educational outcomes: An analysis of evidence from Asia. International Review of
Research in Open & Distance Learning, 11(1), 117e140.
Van Braak, J. (2001). Individual characteristics influencing teachers' class use of computers. Journal of Educational Computing Research, 25(2), 141e157.
Woodill, G. (2011). The mobile learning edge: Tools and technologies for developing your teams. New York: McGraw-Hill Professional.
Wu, W. H., Wu, Y. C. J., Chen, C. Y., Kao, H. Y., Lin, C. H., & Huang, S. H. (2012). Review of trends from mobile learning studies: A meta-analysis. Computers &
Education, 59(2), 817e827.
Zydney, J. M., & Warner, Z. (2016). Mobile apps for science learning: Review of research. Computers & Education, 94, 1e17.