Interactive Learning Environments

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/nile20

Predicting the actual use of m-learning systems:

a comparative approach using PLS-SEM and
machine learning algorithms

Muhammad Alshurideh , Barween Al Kurdi , Said A. Salloum , Ibrahim Arpaci

& Mostafa Al-Emran

To cite this article: Muhammad Alshurideh , Barween Al Kurdi , Said A. Salloum , Ibrahim Arpaci
& Mostafa Al-Emran (2020): Predicting the actual use of m-learning systems: a comparative
approach using PLS-SEM and machine learning algorithms, Interactive Learning Environments,
DOI: 10.1080/10494820.2020.1826982

To link to this article: https://doi.org/10.1080/10494820.2020.1826982

Published online: 01 Oct 2020.

Submit your article to this journal

Article views: 59

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=nile20
INTERACTIVE LEARNING ENVIRONMENTS
https://doi.org/10.1080/10494820.2020.1826982

Predicting the actual use of m-learning systems: a comparative

approach using PLS-SEM and machine learning algorithms
a,b
Muhammad Alshurideh , Barween Al Kurdic, Said A. Salloumd, Ibrahim Arpaci e
and
Mostafa Al-Emran f
a
Department of Management, College of Business Administration, University of Sharjah, Sharjah, UAE; bMarketing
Department, School of Business, The University of Jordan, Amman, Jordan; cMarketing Department, Amman Arab
University, Amman, Jordan; dResearch Institute of Sciences & Engineering, University of Sharjah, Sharjah, UAE;
e
Department of Computer Education and Instructional Technology, Tokat Gaziosmanpasa University, Tokat, Turkey;
f
Applied Computational Civil and Structural Engineering Research Group, Faculty of Civil Engineering, Ton Duc
Thang University, Ho Chi Minh City, Vietnam

ABSTRACT ARTICLE HISTORY

Despite the plethora of m-learning acceptance studies, few have tackled Received 20 December 2019
the importance of examining the actual use of m-learning systems from Accepted 18 September
the lenses of social influence, expectation-confirmation, and satisfaction. 2020
Additionally, most of the prior technology adoption literature tends to
KEYWORDS
use the structural equation modeling (SEM) technique in analyzing the Mobile learning; higher
structural models. To address these limitations, this study extends the education; TAM; ECM; PLS-
technology acceptance model (TAM) with the expectation-confirmation SEM; machine learning
model (ECM) and social influence to predict the actual use of m- algorithms
learning systems. A comparative approach using the partial least
squares-structural equation modeling (PLS-SEM) and machine learning
algorithms was employed to test the proposed model with data
collected from 448 students. The results revealed that both techniques
have successfully provided support to all the hypothesized relationships
of the research model. More interesting, the J48 classifier has performed
better than the other classifiers in predicting the dependent variable in
most cases. The employment of a comparative analytical approach is
believed to add a significant contribution to the information systems
(IS) literature in general, and the m-learning domain in specific.

1. Introduction
Mobile learning (m-learning) is defined as “learning that occurs when learners have access to infor-
mation anytime and anywhere via mobile technologies to perform authentic activities in the context
of their learning” (Martin & Ertzberger, 2013). M-learning offers an exclusive opportunity to leverage
learners’ experiences in formal and informal learning (Joo et al., 2016). The portability and flexibility
of mobile computing devices allow learners to situate their learning in an informative way, experi-
ence learning with real-world problems, and personalize their learning (Traxler, 2009).
Since the appearance of the m-learning concept until this moment, information system (IS) and
educational scholars have examined how to incorporate m-learning in instructional practices. The
continuous insistence of those scholars stems from the fact that m-learning systems enable students
to access their learning materials at “anytime anywhere” settings using wireless networks (Al-Emran
et al., 2019; Sarrab et al., 2018). Despite this enthusiasm, the investment in m-learning systems
requires the understanding of students’ lack of motivation to use these systems for instructional

CONTACT Mostafa Al-Emran al.emran@tdtu.edu.vn

© 2020 Informa UK Limited, trading as Taylor & Francis Group
2 M. ALSHURIDEH ET AL.

activities (Tan et al., 2014). The existence of m-learning platform does not guarantee that learners
would use it for educational practices; students need to be aware of its advantages and adopt it
in their academic lifestyle (Nguyen et al., 2015).
While several studies were conducted in the past to understand the determinants influencing the
adoption of m-learning (Al-Emran et al., 2018a), various issues still need to be discussed. First, the
employment of m-learning in the higher education context is still in its early stages, and its theor-
etical basis is not yet matured (Kumar & Chand, 2019). Second, the students’ use of m-learning
does not always meet the expectations of several educational environments (Aburub & Alnawas,
2019). This stems from the influential factors which mainly rely on the variation in the context, infra-
structure, and students’ readiness. Besides, there are several technical and non-technical issues
regarding the use of mobile computing devices in classrooms (Adegbija & Bola, 2015). Third,
there is a little debate concerning the factors influencing the continued use of m-learning
systems (Al-Emran et al., 2020). The identification of these factors would facilitate the understanding
of students’ actual use of m-learning. Fourth, by analyzing the studies published on m-learning
systems in a recent systematic review (Al-Emran et al., 2018a), it has been noticed that there is a
limited number of studies concerning the continued use of these systems. It has been also observed
that there is a knowledge gap in explaining the impact of social influence on perceived ease of use
and perceived usefulness and how this, in turn, would affect the continued use of m-learning
systems. Fifth, despite the fact that most of the relevant m-learning studies have employed structural
equation modeling (SEM) techniques in explaining the causal-effect among the theoretical model
constructs, there is a scarce of knowledge regarding the use of other analytical techniques, such
as machine learning algorithms (Arpaci, 2019) and neural networks (Al-Shihi et al., 2018).
To handle the aforementioned issues, this research aims to investigate the determinants influen-
cing the students’ continued use of m-learning systems. More specifically, the role of social influence
would be examined in relation to the perceived usefulness and perceived ease of use of m-learning.
In doing so, this study develops a theoretical model based on the integration of the Technology
Acceptance Model (TAM) (Davis, 1989) and Expectation-Confirmation Model (ECM) (Bhattacherjee,
2001). To validate the developed model, this study employs a comparative analytical approach
using the partial least squares-structural equation modeling (PLS-SEM) alongside machine learning
algorithms to develop a high-performance predictive model. PLS-SEM is a widespread multivariate
analysis technique that is used to empirically test theoretical models. On the other hand, the use of
machine learning algorithms and data mining has become an innovative technique for developing
predictive models that explain, predict, and describe the social/human behavior (Osoba & Davis,
2019). This study used the PLS-SEM approach to identify the causal relationships between the
endogenous (independent) and exogenous (dependent) variables. Further, the study employed
the classifier model to predict dependent variables based on the independent variables. Although
the presentation of the two analyses (PLS-SEM vs. machine learning algorithms) seems to produce
two separate findings regarding the operationalization of the same variables in both techniques,
it is in fact, considered a hybrid modeling approach.

2. Research model and hypotheses development

In line with the analysis of m-learning studies provided in a recent systematic review (Al-Emran et al.,
2018a), it has been observed that there is little empirical research on students’ continuous intention
towards m-learning systems with a little explanation of how social influence would affect the per-
ceived usefulness (PU) and perceived ease of use (PEOU) of these systems. Therefore, this study
develops a theoretical model to examine the role of social influence grounded on the integration
of TAM and ECM. Those two theories are among the most cited theories in the contexts of technol-
ogy acceptance and post-adoption behavior. The rationale behind the integration of those two the-
ories stems from the reason that TAM involves the pre-adoption expectation, while ECM accounts for
the post-adoption expectation (Bhattacherjee, 2001; Davis, 1989). As such, the incorporation of both
 INTERACTIVE LEARNING ENVIRONMENTS 3

theories would better explain the students’ continued use of m-learning and how that would
influence their actual use. While the integration of those two theories has been well-supported in
the previous literature (Al-Emran et al., 2020; Joo et al., 2016; Luo et al., 2017), it is believed that
adding the social influence factor would have a significant value in explaining the interrelationships
among the constructs in those theories.
It is suggested that social influence would have significant effects on both PEOU and PU of m-learn-
ing systems. Besides, it is anticipated that the expectation-confirmation would have significant effects
on PU and satisfaction. Further, it is proposed that the continuous intention to use m-learning systems
would be affected by PEOU, PU, and satisfaction. It is also assumed that continuous intention would
affect the actual use of m-learning systems. Figure 1 depicts the proposed theoretical model.

2.1. Social influence

Social influence (SI) refers to “the degree to which an individual perceives that important others
believe he or she should use the new system” (Venkatesh et al., 2003). Through the use of several
technologies, prior research showed that social influence has a significant effect on the PEOU
(Wamba & Queiroz, 2019; Zhang et al., 2020) and PU (Hassan et al., 2020; Nikou & Economides,
2017; Vanduhe et al., 2020; Wamba & Queiroz, 2019; Zhang et al., 2020). By extending this relation-
ship to the context of m-learning, we hypothesize the following:
H1: Social influence would predict the PEOU of m-learning systems.

H2: Social influence would predict the PU of m-learning systems.

2.2. Expectation-confirmation
Expectation-confirmation refers to “users’ perceptions of the congruence between the expectation
of information system usage and its actual performance” (Bhattacherjee, 2001). Previous research
suggested that expectation-confirmation has a significant impact on satisfaction and PU of
several mobile technologies (Al-Emran et al., 2020; Le et al., 2020; Nascimento et al., 2018). Hence,
we suggest the following:
H3: Expectation-confirmation would predict the PU of m-learning systems.

Figure 1. Research model.

4 M. ALSHURIDEH ET AL.

H4: Expectation-confirmation would predict the satisfaction of m-learning systems.

2.3. Perceived ease of use

Perceived ease of use (PEOU) is defined as “the degree to which a person believes that using a par-
ticular system would be free of effort” (Davis, 1989). Previous studies revealed that PEOU has a sig-
nificant impact on the continuous intention to use m-learning systems (Al-Emran et al., 2020; Joo
et al., 2016). Thus, we propose the following:
H5: PEOU would predict the continuous intention of using m-learning systems.

2.4. Perceived usefulness

Perceived usefulness (PU) is defined as “the degree to which a person believes that using a particular
system would enhance his or her job performance” (Davis, 1989). Earlier studies pointed out that PU
has a significant effect on the continuous intention to use various mobile technologies (Joo et al.,
2016; Le et al., 2020; Nascimento et al., 2018). Consequently, we suggest the following:
H6: PU would predict the continuous intention of using m-learning systems.

2.5. Satisfaction
Satisfaction is defined as “the affective attitude towards a particular computer application by an end
user who interacts with the application directly” (Doll et al., 1998). Prior research found that satisfac-
tion has a significant effect on the continuous intention to use different mobile technologies (Le
et al., 2020; Nascimento et al., 2018; Tam et al., 2020). Accordingly, we hypothesize the following:
H7: Satisfaction would predict the continuous intention of using m-learning systems.

2.6. Continuous intention

Continuous intention (CI) is defined as “users’ intention to continue using the information system”
(Bhattacherjee, 2001). Earlier studies showed that CI has a significant influence on the actual use of
m-learning (Al-Emran et al., 2020; Joo et al., 2016). Thus, this leads to the following:
H8: Continuous intention would predict the actual use of m-learning systems.

3. Research methodology
3.1. Context and subjects
The participants of this study are the students enrolled at the British University in Dubai and the Uni-
versity of Fujairah, in which both located in the United Arab Emirates (UAE). In both universities, the
students used the m-learning system to register their courses, receive notifications, and communi-
cate with their colleagues and instructors through the learning management system (LMS). The stu-
dents are also able to access their lectures, download and upload their assignments, and check the
similarity score of their work using the similarity checking system in each university. The data were
collected between October and November 2018 through the use of self-administrated surveys. The
participants volunteered to fill out the surveys without getting any compensation for their involve-
ment. This research uses the convenience sampling approach in gathering the data. While this
approach is commonly used in quantitative studies (Etikan et al., 2016), it also raises some issues con-
cerning the subjective nature in selecting the respondents and data bias. To handle these issues, we
have employed the common method bias (CMB) test (Podsakoff et al., 2003). Out of 500 surveys
 INTERACTIVE LEARNING ENVIRONMENTS 5

distributed, a total of 448 students have successfully filled out the entire survey with a response rate
of 89.6%. Of those, there were 230 females and 218 males. 56% of the participants were aged
between 18 and 29 years old. Further, 56% of the participants were bachelor’s degree students, fol-
lowed by master’s degree students (20%), PhD students (19%), and diploma students (5%).

3.2. Research instrument

The research instrument of this study consists of two parts. The first part is devoted to collect the
demographic data of the participants, whereas the second one is designed to collect responses
regarding the factors in the conceptual model. The indicators in the second part were measured
using a “5-point Likert scale”. The indicators used to evaluate the PEOU and PU were adopted
from Davis (1989). The items used to evaluate expectation confirmation, satisfaction, and continuous
intention were adopted from Bhattacherjee (2001). The social influence items were adopted from
Venkatesh et al. (2003), while the indicators of actual use were adopted from Mohammadi (2015).
The constructs and their underlying items are listed in the Appendix.

3.3. Data analysis

This study employs two different techniques for evaluating the developed research model. Concern-
ing the first approach, this research uses the partial least squares-structural equation modeling (PLS-
SEM) via the SmartPLS software (Ringle et al., 2015). The main reason behind the employment of PLS-
SEM in this research stems from the fact that PLS-SEM provides concurrent analysis for both
measurement and structural model, which in turn, leads to more precise results (Barclay et al.,
1995). In terms of the second technique, this study uses the machine learning algorithms through
Weka to predict the dependent constructs in the theoretical model (Arpaci, 2019).

3.4. Common method bias

To ensure that the collected data do not have CMB, the Harman’s single-factor has been carried out
with seven factors (social influence, expectation-confirmation, PEOU, PU, satisfaction, continuous
intention, and actual use) (Podsakoff et al., 2003). The seven factors were then loaded into a
single factor. The analysis shows that the largest variance explained by the newly created factor is
23.21%, which is below the threshold value of 50% (Podsakoff et al., 2003). Hence, there were no
concerns regarding the CMB in the collected data.

4. Results
4.1. Measurement model assessment
The measurement model is assessed through testing the reliability and validity (Hair et al., 2016). For
reliability testing, the Cronbach’s alpha and composite reliability (CR) measures were used. The rec-
ommended values for each of these measures should be ≥ 0.70 (Hair et al., 2016). According to the
results in Table 1, the values of both measures are considered satisfactory, and thus, the reliability is
confirmed.
Concerning validity testing, Hair et al. (2016) suggested evaluating the convergent and discrimi-
nant validities. For convergent validity, the average variance extracted (AVE) and factor loadings
were tested. The values of AVE should be ≥ 0.50 (Fornell & Larcker, 1981), whereas the values of
factor loadings should be ≥ 0.70 (Hair et al., 2010). As per the results in Table 1, the values of
both measures are accepted, and therefore, the convergent validity is ascertained. For discriminant
validity, Henseler et al. (2015) suggested testing the “Heterotrait-Monotrait ratio (HTMT)” of
6 M. ALSHURIDEH ET AL.

Table 1. Reliability and convergent validity results.

Constructs Items Factor Loadings Cronbach’s Alpha CR AVE
Actual Use AU1 0.928 0.852 0.856 0.931
AU2 0.938
Continuous Intention CI1 0.889 0.849 0.877 0.906
CI2 0.871
CI3 0.858
Expectation-Confirmation EXP1 0.890 0.891 0.971 0.922
EXP2 0.933
EXP3 0.952
Perceived Ease of Use PEOU1 0.909 0.831 0.848 0.899
PEOU2 0.799
PEOU3 0.883
Perceived Usefulness PU1 0.796 0.731 0.758 0.850
PU2 0.906
PU3 0.814
Satisfaction SAT1 0.857 0.787 0.789 0.877
SAT2 0.905
SAT3 0.750
Social Influence SI1 0.904 0.837 0.849 0.890
SI2 0.826
SI3 0.798
SI4 0.742

correlations. The values of HTMT should be < 0.85. As per the readings in Table 2, all the values are
accepted, and hence, the discriminant validity is established.

4.2. Hypotheses testing using PLS-SEM

Table 3 describes the beta (β) values, t-values, and p-values for each of the developed hypotheses
based on the generated results through the PLS-SEM technique. The first hypothesis describes
the relationship between social influence and PEOU (β = 0.460, t = 11.079). The result of this
hypothesis indicates that social influence has a significant positive impact on the PEOU of m-
learning systems. Thus, H1 is supported. The second hypothesis shows the correlation between
social influence and PU (β = 0.175, t = 2.274). The result of this hypothesis reveals that social
influence has a significant positive effect on the PU of m-learning systems. Therefore, H2 is
supported.
The third hypothesis demonstrates the relationship between expectation-confirmation and PU (β
= 0.578, t = 14.434). The result of this hypothesis suggests that expectation-confirmation has a sig-
nificant positive influence on the PU of m-learning systems. Hence, H3 is supported. The fourth
hypothesis characterizes the correlation between expectation-confirmation and satisfaction (β =
0.458, t = 9.952). The result of this hypothesis indicates that expectation-confirmation has a signifi-
cant positive influence on satisfaction. Thus, H4 is supported.
The fifth hypothesis exhibits the relationship between PEOU and continuous intention (β = 0.363,
t = 4.552). The result of this hypothesis suggests that PEOU has a significant positive influence on the

Table 2. HTMT results.

AU CI EXP PEOU PU SAT SI
AU
CI 0.476
EXP 0.576 0.566
PEOU 0.459 0.426 0.795
PU 0.646 0.751 0.768 0.662
SAT 0.750 0.514 0.561 0.551 0.454
SI 0.410 0.481 0.589 0.537 0.440 0.394
 INTERACTIVE LEARNING ENVIRONMENTS 7

Table 3. Hypotheses testing results through PLS-SEM.

H Relationship Beta t-value p-value Remarks
H1 Social Influence → Perceived Ease of Use 0.460 11.079 0.000 Supported
H2 Social Influence → Perceived Usefulness 0.175 2.274 0.013 Supported
H3 Expectation-Confirmation → Perceived Usefulness 0.578 14.434 0.000 Supported
H4 Expectation-Confirmation → Satisfaction 0.458 9.952 0.000 Supported
H5 Perceived Ease of Use → Continuous Intention 0.363 4.552 0.001 Supported
H6 Perceived Usefulness → Continuous Intention 0.477 8.598 0.000 Supported
H7 Satisfaction → Continuous Intention 0.396 6.663 0.000 Supported
H8 Continuous Intention → Actual Use 0.852 58.592 0.000 Supported

continuous intention to use m-learning systems. Therefore, H5 is supported. The sixth hypothesis
describes the correlation between PU and continuous intention (β = 0.477, t = 8.598). The result of
this hypothesis reveals that PU has a significant positive effect on the continuous intention to use
m-learning systems. Hence, H6 is supported.
The seventh hypothesis describes the relationship between satisfaction and continuous intention
(β = 0.396, t = 6.663). The result of this hypothesis suggests that satisfaction has a significant positive
influence on the continuous intention to use m-learning systems. Thus, H7 is supported. The eighth
hypothesis shows the correlation between continuous intention and actual use (β = 0.852, t =
58.592). The result of this hypothesis indicates that the continuous intention to use m-learning
systems has a significant positive impact on its actual usage. Therefore, H8 is supported.
Concerning the coefficient of determination (R2) results in Figure 2, it can be observed that social
influence explains 43.7% of the variance in PEOU. Moreover, social influence and expectation-confir-
mation together explain 49% of the variance in PU. Besides, expectation-confirmation explains 44.6%
of the variance in satisfaction. Further, PEOU, PU, and satisfaction together explain 39.4% of the var-
iance in continuous intention. Additionally, the continuous intention explains 32.7% of the variance
in actual use. In comparison with the proposed R2 values (Chin, 1998), the values observed in this
study are regarded to be acceptable.

4.3. Hypotheses testing using machine learning algorithms

This research employs machine learning classification algorithms by applying a wide range of meth-
odologies, including Bayesian networks, decision trees, if-then-else rules, and neural networks, to

Figure 2. PLS algorithm results.

8 M. ALSHURIDEH ET AL.

Table 4. Predicting the PEOU by social influence.

Classifier CCI1 (%) TP2 Rate FP3 Rate Precision Recall F-Measure
BayesNet 72.79 .728 .523 .683 .728 .674
Logistic 71.12 .711 .610 .690 .711 .585
LWL 72.79 .728 .523 .683 .728 .674
AdaBoostM1 72.79 .728 .523 .683 .728 .674
OneR 71.84 .718 .540 .606 .718 .650
J48 73.03 .730 .538 .698 .730 .665
1
CCI: Correctly Classified Instances. 2TP: True Positive, 3FP: False Positive.

predict the relationships in the proposed theoretical model (Arpaci, 2019, 2020). Weka (ver. 3.8.3)
was used to test the predictive model based on several classifiers, including Bayesian classifier
(BayesNet), meta classifier (AdaBoostM1), lazy classifier (LWL), logistic regression classifier (Logistic),
decision tree (J48), and rule learner (OneR) (Frank et al., 2009). Based on the results in Table 4, it can
be observed that J48 performs better than the other classifiers in predicting the PEOU of m-learning
systems. The J48 predicted the PEOU with an accuracy of 73.03% for the 10-fold cross-validation.
Hence, H1 is supported. This classifier had a better performance in terms of the TP rate (.730), pre-
cision (.698), and recall (.730) as compared to the other classifiers.
The results also suggested that the J48 had a better classification performance than the other
classifiers in predicting the PU, as shown in Table 5. The J48 predicted the PU by the attributes of
social influence and expectation-confirmation with an accuracy of 78.29%, and thereby, H2 and
H3 were both supported.
As demonstrated in Table 6, the results showed that both OneR and J48 classifiers performed
better than the other classifiers in predicting the satisfaction by expectation-confirmation. The
OneR and J48 classifiers predicted the satisfaction with an accuracy of 63.72%. Accordingly, H4 is
supported.
The results presented in Table 7 indicated that the J48 performed better than the other classifiers
in predicting the continuous intention by the attributes of PEOU, PU, and satisfaction. The J48 pre-
dicted the continuous intention with an accuracy of 88.31%. Therefore, H5, H6, and H7 were
supported.
The results also indicated that Logistic performed better than the other classifiers in predicting
the actual use by the continuous intention, as shown in Table 8. The Logistic classifier predicted
the actual use with an accuracy of 78.99%. Accordingly, H8 is supported.

5. Discussion
5.1. Research hypotheses findings
The present study proposed a conceptual model to examine the students’ actual use of m-learning.
To validate the proposed model, this research employed a comparative analytical approach, includ-
ing PLS-SEM and machine learning algorithms.
In terms of predicting the PEOU, the PLS-SEM results suggested that social influence has a signifi-
cant impact on the PEOU of m-learning systems. Further, the results of machine learning indicated
that the J48 classifier showed better performance than the other classifiers in predicting the PEOU of
m-learning systems with an accuracy of 73.03%. These findings seem to be consistent with those
found in the previous literature (Wamba & Queiroz, 2019; Zhang et al., 2020). This indicates that
the positive opinions and reports of other students concerning the use of m-learning systems
would significantly affect the PEOU of these systems. With respect to predicting the PU, the PLS-
SEM results showed that the PU of m-learning systems was significantly and positively affected by
social influence and expectation-confirmation together. Besides, the results of machine learning indi-
cated that the J48 showed better performance by predicting the PU with an accuracy of 78.29%.
These results also accord with the observations reported in previous literature, which suggested
 INTERACTIVE LEARNING ENVIRONMENTS 9

Table 5. Predicting the PU by social influence and expectation-confirmation.

Classifier CCI (%) TP Rate FP Rate Precision Recall F-Measure
BayesNet 69.45 .695 .247 .715 .695 .683
Logistic 68.74 .687 .223 .713 .687 .680
LWL 68.26 .683 .265 .737 .683 .667
AdaBoostM1 59.19 .592 .321 .599 .592 .665
OneR 67.78 .678 .263 .706 .678 .663
J48 78.29 .783 .160 .791 .783 .780

that the PU is positively affected by social influence (Hassan et al., 2020; Nikou & Economides, 2017;
Vanduhe et al., 2020; Wamba & Queiroz, 2019; Zhang et al., 2020) and expectation-confirmation (Al-
Emran et al., 2020; Le et al., 2020; Nascimento et al., 2018). This means that the usefulness of using m-
learning systems is increased when the students perceive that these systems are socially accepted by
their colleagues. The usefulness of m-learning systems can also be increased if the students have
positive expectations that these systems would enhance their performance.
Overall, the reason why social influence is being such a key driver in affecting the PU and PEOU of
m-learning systems is that the study was conducted in the UAE, which characterized by collectivistic
culture. In collectivistic cultures, the perceptions of individuals are more likely to be affected by
others’ decisions and opinions (Zhou & Li, 2014). Hence, social influence is expected to have a
more substantial impact on m-learning acceptance in the UAE culture compared with other cultures.
Concerning the prediction of satisfaction, the PLS-SEM findings pointed out that expectation-
confirmation has a positive impact on satisfaction. The results of machine learning also indicated
that both OneR and J48 classifiers performed better than the other classifiers in predicting the sat-
isfaction by expectation-confirmation. These findings also correspond to those observed in prior
research (Al-Emran et al., 2020; Le et al., 2020; Nascimento et al., 2018). It can, therefore, be suggested
that the students would be satisfied with using the m-learning systems if they impose positive
expectations that these systems would enhance their learning performance.
With regard to predicting the continuous intention, the PLS-SEM results revealed that the con-
tinuous intention to use m-learning systems was significantly affected by PEOU, PU, and satisfaction.
Moreover, the results of machine learning revealed that the J48 showed better performance by pre-
dicting the continuous intention with an accuracy of 88.31%. Similarly, these findings match those
noticed in earlier investigations, which reported that continuous intention is positively affected by
PEOU (Al-Emran et al., 2020; Joo et al., 2016), PU (Joo et al., 2016; Le et al., 2020; Nascimento
et al., 2018), and satisfaction (Le et al., 2020; Nascimento et al., 2018; Tam et al., 2020). These
results suggest that students’ intentions will continue to use m-learning systems if these systems
keep satisfying them, specifically when the systems impose free efforts and enhance their learning
performance.
With respect to predicting the actual use, the PLS-SEM results pointed out that the actual use of
m-learning systems was significantly and positively influenced by its continued use. Further, the
results of machine learning indicated that the Logistic classifier showed better performance by pre-
dicting the actual use with an accuracy of 78.99%. These results were also in agreement with those
reported in the previous m-learning literature (Al-Emran et al., 2020; Joo et al., 2016). This indicates
that the continued use of m-learning was directly and positively associated with the students’ actual

Table 6. Predicting satisfaction by expectation-confirmation.

Classifier CCI (%) TP Rate FP Rate Precision Recall F-Measure
BayesNet 62.77 .628 .341 .654 .628 .609
Logistic 59.19 .592 .381 .605 .592 .566
LWL 61.34 .613 .344 .624 .613 .597
AdaBoostM1 55.85 .558 .422 .510 .558 .496
OneR 63.72 .637 .341 .687 .637 .618
J48 63.72 .637 .341 .687 .637 .618
10 M. ALSHURIDEH ET AL.

Table 7. Predicting the continuous intention by PEOU, PU, and satisfaction.

Classifier CCI (%) TP Rate FP Rate Precision Recall F-Measure
BayesNet 69.69 .697 .241 .703 .697 .697
Logistic 69.69 .697 .235 .705 .697 .698
LWL 73.03 .730 .218 .737 .730 .730
AdaBoostM1 65.63 .656 .368 .759 .656 .778
OneR 71.84 .718 .204 .731 .718 .722
J48 88.31 .883 .071 .891 .883 .884

Table 8. Predicting the actual use by continuous intention.

Classifier CCI (%) TP Rate FP Rate Precision Recall F-Measure
BayesNet 76.85 .768 .246 .776 .768 .750
Logistic 78.99 .790 .253 .787 .790 .778
LWL 72.55 .726 .336 .716 .726 .797
AdaBoostM1 72.55 .726 .336 .716 .726 .797
OneR 77.80 .778 .276 .774 .778 .760
J48 77.80 .778 .280 .775 .778 .758

use. This means that students who used the m-learning system with a positive or negative experi-
ence did not change their usage intentions after their initial trial. In this sense, the students either
continued to use the m-learning system in their instructional activities, or they did not use it
consistently.

5.2. Research implications

We have validated the proposed model through a comparative method using machine learning
algorithms and PLS-SEM. While the use of data mining techniques in analyzing theoretical models
has been well-perceived in the previous literature (Berrado et al., 2013; Verbraken et al., 2014; Wu,
2011), the employment of a comparative analytical approach (i.e. machine learning and PLS-SEM)
is believed to add a significant contribution to the IS literature in general, and the m-learning
domain in specific. It is important to report that PLS-SEM can be used for examining the dependent
construct and validating the conceptual model based on the extension of an existing theory (Al-
Emran et al., 2018b). Likewise, supervised machine learning algorithms (i.e. having a pre-defined
dependent construct) can be used for predicting the dependent variable based on independent vari-
ables (Arpaci, 2019). It is also essential to indicate that this study has employed several classification
algorithms with different methodologies, including association rules, decision trees, if-then-else
rules, Bayesian networks, and neural networks. More specifically, the findings suggested that J48
(a decision tree) performed better than other classifiers in most cases. It is worth mentioning that
the decision tree (nonparametric) was used for classifying both continuous (numerical) and categori-
cal variables by splitting the sample into homogeneous sub-samples based on the most significant
independent variable (Arpaci, 2020). On the other hand, PLS-SEM (a nonparametric procedure) was
used to test the significant coefficients with replacements from the sample to draw a large number
of sub-samples randomly. Although the presentation of the two analyses (PLS-SEM vs. machine
learning algorithms) seems to produce two separate findings regarding the operationalization of
the same variables in both techniques, it is in fact, considered a hybrid modeling approach.

5.3. Theoretical contributions

The theoretical contributions of this research can be summarized in three points. First, this study
explored the role of social influence grounded on the combination of TAM and ECM to examine
the actual use of m-learning. It is believed that the proposed model would add value to the ECM
 INTERACTIVE LEARNING ENVIRONMENTS 11

and TAM at one hand, and the m-learning domain on the other hand. Second, most of the earlier
studies in m-learning have investigated the determinants affecting either the continuous intention
or actual use. To the best of the authors’ knowledge, this is one of the few attempts that examined
the factors affecting the continuous intention and how that would influence the students’ actual use
of m-learning. Third, PU and PEOU have been well-perceived in the previous m-learning literature to
be the key drivers of behavioral intention. With the introduction of the proposed model, this
research shows that other essential factors need to be considered, whether directly or indirectly,
when approaching the continued use of m-learning, such as social influence, expectation-confir-
mation, and satisfaction.

5.4. Practical implications

The evidence from this research imposes various practical implications. First, the findings indicated
that PU (β = 0.477) is the most crucial factor which affects the continuous intention to use m-learn-
ing. Hence, educators and practitioners are required to ensure that the incorporated content
materials in m-learning systems are useful and would enhance the learning performance of students
in order to sustain their continued use. While taking other factors into consideration, the decision-
makers should focus more on the usefulness when purchasing new m-learning systems or upgrading
the existing ones. Second, the findings revealed that social influence has a significant impact on
increasing the ease of use and usefulness of m-learning systems. While this might be due to the
characteristics of the studied sample, researchers might explore the impact of social influence in
other non-collectivistic cultural environments. Third, since all the hypotheses were supported, m-
learning developers should consider the importance of the studied factors in designing more colla-
borative m-learning systems in order to sustain the continued use of these systems. M-learning
systems should not require much time and effort from their students to get through the learning
material (Amoroso & Lim, 2017). Therefore, m-learning developers should emphasize on designing
user-friendly systems with a better reading layout and richer presentation.

6. Conclusion
While several studies were conducted in the past to examine the factors affecting the m-learning
acceptance, few have studied what impacts the actual use of m-learning from the perspective of
social influence, expectation-confirmation, and satisfaction. At the same time, most of the prior
studies have mainly relied on the SEM approach concerning the analysis of structural models. There-
fore, this study aimed to address these limitations by extending the TAM with ECM and social
influence to predict the actual use of m-learning systems. To validate the proposed model, this
research employed a comparative approach using PLS-SEM and machine learning algorithms. The
findings indicated that both techniques provided support to all the suggested hypotheses. More
interesting, the J48 classifier has performed better than the other classifiers in predicting the depen-
dent variable in most cases. It is, therefore, believed that the use of a comparative analytical
approach would add a valuable contribution to the IS literature in general, and the m-learning
domain in particular.
A number of important limitations need to be reported. First, the generalization of the results to
the other higher educational institutions in the UAE or other countries should be treated with
caution. This stems from two reasons: (a) the concentration on only two institutions for picking
up the samples, and (b) the employment of a convenience sampling technique for selecting the par-
ticipants. Further research needs to consider these issues to enhance the opportunity of results gen-
eralization. Second, the study focused on examining the actual use of m-learning systems by
students only. Future attempts are highly encouraged to measure the instructors’ actual use of
m-learning systems in order to get further insights into the affecting determinants and draw a com-
prehensive picture of the implementation of these systems.
12 M. ALSHURIDEH ET AL.

Acknowledgment
This is an extended version of a conference paper published by the International Conference on Advanced Intelligent
Systems and Informatics 2019.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Notes on contributors
Muhammad Alshurideh is working for the College of Business Administration – University of Sharjah in UAE as a full-
time faculty member and for the School of Business – University of Jordan in Jordan. Regarding the teaching, he has the
responsibilities to teach a wide range of marketing and business topics for both undergraduate and postgraduate stu-
dents. He has more than 40 published papers in different marketing and business topics mainly CRM and Customer
Retention. He has published in good ranked journals such as Journal of Marketing Communications and International
Journal of Electronic Customer Relationship Management.
Barween Al Kurdi is an Assistant Professor in Marketing and she is working for Amman Arab University – Faculty of
Business – Marketing Department. She is a member of large number of committees and mainly the social committee.
She used to publish in good ranked journals such as Journal of Marketing Communications and International Journal of
Marketing Studies.
Said A. Salloum had graduated from The British University in Dubai with a distinction with MSc in Informatics (Knowl-
edge and Data Management). He got his bachelor’s degree in Computer Science from Yarmouk University. Currently, he
is working at the University of Sharjah “Research Institute of Sciences and Engineering (RISE)” as a researcher on
different research areas in Computer Science such as data analysis, machine learning, knowledge management, and
Arabic Language Processing. Salloum is an Oracle expert since 2013 along with various recognized international certifi-
cates that are issued by Oracle.
Ibrahim Arpaci is an Associate Professor and Chair of the Department of Computer Education and Instructional Tech-
nology at Gaziosmanpasa University, where he is also Director of the Distance Education Application and Research
Center. He was a visiting scholar at Ryerson University, Ted Rogers School of Information Technology Management,
Toronto, ON, Canada (2012-2013). He holds a BSc in Computer Education and Instructional Technology (2005) from
Anadolu University, an MSc in Information Systems (2009) and a PhD in Information Systems (2013) both from
Middle East Technical University. His research interests are in instructional systems design and technology, cyberpsy-
chology and behavior, culture, learning and technology.
Mostafa Al-Emran obtained his Ph.D. degree in Computer Science from Universiti Malaysia Pahang. He received the MSc
degree in Informatics from The British University in Dubai with a distinction level along with the top Academic Excel-
lence Award, and the BSc degree with honors in Computer Science from Al Buraimi University College. He has published
over 70 research articles in highly reputed journals such as Computers & Education, Computers in Human Behavior, Inter-
national Journal of Information Management, Telematics and Informatics, IEEE Access, Technology in Society, Interactive
Learning Environments, Journal of Educational Computing Research, and International Journal of Engineering Education,
among many others. Most of his publications were indexed under the ISI Web of Science and Scopus. He has also
edited a number of books published by Springer. He is a certified recognized reviewer by several leading journals in
Elsevier. His current research interests include mobile learning, knowledge management, technology acceptance,
and wearable technology.

ORCID
Muhammad Alshurideh http://orcid.org/0000-0002-7336-381X
Ibrahim Arpaci http://orcid.org/0000-0001-6513-4569
Mostafa Al-Emran http://orcid.org/0000-0002-5269-5380

References
Aburub, F., & Alnawas, I. (2019). A new integrated model to explore factors that influence adoption of mobile learning in
higher education: An empirical investigation. Education and Information Technologies, 24(3), 2145–2158. https://doi.
org/10.1007/s10639-019-09862-x
 INTERACTIVE LEARNING ENVIRONMENTS 13

Adegbija, M. V., & Bola, O. O. (2015). Perception of undergraduates on the adoption of mobile technologies for learning
in selected universities in Kwara state, Nigeria. Procedia - Social and Behavioral Sciences, 176, 352–356. https://doi.org/
10.1016/j.sbspro.2015.01.482
Al-Emran, M., Arpaci, I., & Salloum, S. A. (2020). An empirical examination of continuous intention to use m-learning: An
integrated model. Education and Information Technologies, 25, 2899–2918. https://doi.org/10.1007/s10639-019-
10094-2
Al-Emran, M., Mezhuyev, V., & Kamaludin, A. (2018a). Technology acceptance model in M-learning context: A systematic
review. Computers & Education, 125, 389–412. https://doi.org/10.1016/j.compedu.2018.06.008
Al-Emran, M., Mezhuyev, V., & Kamaludin, A. (2018b, September 3–5). PLS-SEM in information systems research: A com-
prehensive methodological reference. 4th International Conference on Advanced Intelligent Systems and Informatics
(AISI 2018) (pp. 644–653).
Al-Emran, M., Mezhuyev, V., & Kamaludin, A. (2019). An innovative approach of applying knowledge management in M-
learning application development: A pilot study. International Journal of Information and Communication Technology
Education (IJICTE), 15(4), 94–112. https://doi.org/10.4018/IJICTE.2019100107
Al-Shihi, H., Sharma, S. K., & Sarrab, M. (2018). Neural network approach to predict mobile learning acceptance.
Education and Information Technologies, 23(5), 1805–1824. https://doi.org/10.1007/s10639-018-9691-9
Amoroso, D., & Lim, R. (2017). The mediating effects of habit on continuance intention. International Journal of
Information Management, 37(6), 693–702. https://doi.org/10.1016/j.ijinfomgt.2017.05.003
Arpaci, I. (2019). A hybrid modeling approach for predicting the educational use of mobile cloud computing services in
higher education. Computers in Human Behavior, 90, 181–187. https://doi.org/10.1016/j.chb.2018.09.005
Arpaci, I. (2020). What drives students’ online self-disclosure behavior on social media? A hybrid SEM and artificial intel-
ligence approach. International Journal of Mobile Communications, 18(2), 229–241. https://doi.org/10.1504/IJMC.
2020.10017999
Barclay, D., Higgins, C., & Thompson, R. (1995). The partial least squares (pls) approach to casual modeling: Personal com-
puter adoption ans use as an illustration.
Berrado, A., Elfahli, S., & El Garah, W. (2013). Using data mining techniques to investigate the factors influencing mobile
payment adoption in Morocco. 2013 8th International Conference on Intelligent Systems: Theories and Applications
(SITA) (pp. 1–5). https://doi.org/10.1109/SITA.2013.6560791.
Bhattacherjee, A. (2001). Understanding information systems continuance: An expectation-confirmation model. MIS
Quarterly, 25, 351–370. https://doi.org/10.2307/3250921
Chin, W. W. (1998). The partial least squares approach to structural equation modeling. Modern Methods for Business
Research, 295(2), 295–336.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS
Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
Doll, W. J., Hendrickson, A., & Deng, X. (1998). Using Davis’s perceived usefulness and ease-of-use instruments for
decision making: A confirmatory and multigroup invariance analysis. Decision Sciences, 29(4), 839–869. https://doi.
org/10.1111/j.1540-5915.1998.tb00879.x
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American
Journal of Theoretical and Applied Statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement
error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312
Frank, E., Hall, M., Holmes, G., Kirkby, R., Pfahringer, B., Witten, I. H., & Trigg, L. (2009). Weka-A machine learning work-
bench for data mining. In Data mining and knowledge discovery handbook. https://doi.org/10.1007/978-0-387-09823-
4_66
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis. Prentice Hall.
Hair Jr., J. F., Hult, G. T. M., Ringle, C., & Sarstedt, M. (2016). A primer on partial least squares structural equation modeling
(PLS-SEM). Sage Publications.
Hassan, F., Amin, M. K., Khan, T., Emon, M. M. H., & Amin, A. (2020). Roles of social influence in expediting online learning
acceptance: A preliminary study on Bangladeshi learners. Proceedings of the International Conference on Computing
Advancements (pp. 1–6). https://doi.org/10.1145/3377049.3377087
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based
structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.
1007/s11747-014-0403-8
Joo, Y. J., Kim, N., & Kim, N. H. (2016). Factors predicting online university students’ use of a mobile learning manage-
ment system (m-LMS). Educational Technology Research and Development, 64(4), 611–630. https://doi.org/10.1007/
s11423-016-9436-7
Kumar, B. A., & Chand, S. S. (2019). Mobile learning adoption: A systematic review. Education and Information
Technologies, 24(1), 471–487. https://doi.org/10.1007/s10639-018-9783-6
Le, T. T., Pham, H. M., Chu, N. H., Nguyen, D. K., & Ngo, H. M. (2020). Factors affecting users’ continuance intention
towards mobile banking In Vietnam. American Journal of Multidisciplinary Research & Development (AJMRD), 2(4),
42–51.
14 M. ALSHURIDEH ET AL.

Luo, Y., Ye, Q., & Meng, L. (2017). An empirical research on influence factor of college students’ continued intentions of
online self-regulated learning based on the model of ECM and TAM. 2017 3rd International Conference on Social
Science and Higher Education (pp. 521–528). Atlantis Press. https://doi.org/10.2991/icsshe-17.2017.132
Martin, F., & Ertzberger, J. (2013). Here and now mobile learning: An experimental study on the use of mobile technol-
ogy. Computers & Education, 68, 76–85. https://doi.org/10.1016/j.compedu.2013.04.021
Mohammadi, H. (2015). Social and individual antecedents of m-learning adoption in Iran. Computers in Human Behavior,
49, 191–207. https://doi.org/10.1016/j.chb.2015.03.006
Nascimento, B., Oliveira, T., & Tam, C. (2018). Wearable technology: What explains continuance intention in smart-
watches? Journal of Retailing and Consumer Services, 43, 157–169. https://doi.org/10.1016/j.jretconser.2018.03.017
Nguyen, L., Barton, S. M., & Nguyen, L. T. (2015). IPads in higher education - hype and hope. British Journal of Educational
Technology, 46(1), 190–203. https://doi.org/10.1111/bjet.12137
Nikou, S. A., & Economides, A. A. (2017). Mobile-based assessment: Investigating the factors that influence behavioral
intention to use. Computers & Education, 109, 56–73. https://doi.org/10.1016/j.compedu.2017.02.005
Osoba, O., & Davis, P. K. (2019). An artificial intelligence/machine learning perspective on social simulation: New data
and new challenges. In P. K. Davis, A. O’Mahony, & J. Pfautz (Eds.), Social-behavioral modeling for complex systems
(pp. 443–476). John Wiley & Sons, Inc.
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A
critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://
doi.org/10.1037/0021-9010.88.5.879
Ringle, C. M., Wende, S., & Becker, J. (2015). SmartPLS 3. Bönningstedt: SmartPLS. http://www.smartpls.com
Sarrab, M., Al-Shihi, H., Al-Manthari, B., & Bourdoucen, H. (2018). Toward educational requirements model for mobile
learning development and adoption in higher education. TechTrends, 62(6), 635–646. https://doi.org/10.1007/
s11528-018-0331-4
Tam, C., Santos, D., & Oliveira, T. (2020). Exploring the influential factors of continuance intention to use mobile Apps:
Extending the expectation confirmation model. Information Systems Frontiers, 22(1), 243–257. https://doi.org/10.
1007/s10796-018-9864-5
Tan, G. W. H., Ooi, K. B., Leong, L. Y., & Lin, B. (2014). Predicting the drivers of behavioral intention to use mobile learning:
A hybrid SEM-neural networks approach. Computers in Human Behavior, 36, 198–213. https://doi.org/10.1016/j.chb.
2014.03.052
Traxler. (2009). Current state of mobile learning. In M. Ally (Ed.). Mobile learning transforming the delivery of education
and training (pp. 9–24). AU Press.
Vanduhe, V. Z., Nat, M., & Hasan, H. F. (2020). Continuance intentions to use gamification for training in higher edu-
cation: Integrating the technology acceptance model (TAM), social motivation, and task technology Fit (TTF). IEEE
Access, 8, 21473–21484. https://doi.org/10.1109/ACCESS.2020.2966179
Venkatesh, V., Morris, M., Davis, G., & Davis, F. (2003). User acceptance of information technology: Toward a unified view.
MIS Quarterly, 27(3), 425–478. https://doi.org/10.2307/30036540
Verbraken, T., Goethals, F., Verbeke, W., & Baesens, B. (2014). Predicting online channel acceptance with social network
data. Decision Support Systems, 63, 104–114. https://doi.org/10.1016/j.dss.2013.08.011
Wamba, S. F., & Queiroz, M. M. (2019). The role of social influence in blockchain adoption: The Brazilian supply chain
case. IFAC-PapersOnLine, 52(13), 1715–1720. https://doi.org/10.1016/j.ifacol.2019.11.448
Wu, W. W. (2011). Mining significant factors affecting the adoption of SaaS using the rough set approach. Journal of
Systems and Software, 84(3), 435–441. https://doi.org/10.1016/j.jss.2010.11.890
Zhang, T., Tao, D., Qu, X., Zhang, X., Zeng, J., Zhu, H., & Zhu, H. (2020). Automated vehicle acceptance in China: Social
influence and initial trust are key determinants. Transportation Research Part C: Emerging Technologies, 112, 220–233.
https://doi.org/10.1016/j.trc.2020.01.027
Zhou, T., & Li, H. (2014). Understanding mobile SNS continuance usage in China from the perspectives of social influence
and privacy concern. Computers in Human Behavior, 37, 283–289. https://doi.org/10.1016/j.chb.2014.05.008
 INTERACTIVE LEARNING ENVIRONMENTS 15

Appendix: Constructs and items

Social influence

SI1: People who influence my behavior think that I should use the m-learning system.
SI2: People who are important to me think that I should use the m-learning system.
SI3: The institution management has been helpful in the use of the m-learning system.
SI4: In general, the institution has supported the use of the m-learning system.

Expectation confirmation

EXP1: My experience with using m-learning was better than what I expected.
EXP2: The service level provided by m-learning was better than what I expected.
EXP3: Overall, most of my expectations from using m-learning were confirmed.

Perceived ease of use

PEOU1: M-learning is easy to use.

PEOU2: Interaction with m-learning is clear and understandable.
PEOU3: M-learning is convenient and user-friendly.

Perceived usefulness

PU1: M-learning enhances my efficiency.

PU2: M-learning enables me to accomplish tasks more quickly.
PU3: M-learning improves my performance.

Satisfaction

SAT1: I am satisfied with using m-learning as a learning assisted tool.

SAT2: I am satisfied with using m-learning functions.
SAT3: I am satisfied with multimedia instructions.

Continuous intention

CI1: I intend to continue using m-learning rather than discontinue its use.
CI2: I intend to continue using m-learning than other alternative means.
CI3: If I could, I would like to continue my use of m-learning.

Actual use

AU1: I use the m-learning system on daily basis.

AU2: I use the m-learning system frequently.