This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES 1

Design and Evaluate Immersive Learning

Experience for Massive Open Online Courses
(MOOCs)
Horace Ho Shing Ip, Chen Li, Selena Leoni, Yangbin Chen, Ka-Fai Ma, Calvin Hoi-to Wong, and Qing Li

Abstract—Massive Open Online Courses (MOOCs), a unique form of online education enabled by web-based learning technologies,
allow learners from anywhere in the world with any level of educational background to enjoy online education experience provided by
many top universities all around the world. Traditionally, MOOC learning contents are always delivered as text-based or video-based
materials. Although introducing immersive learning experience for MOOCs may sound exciting and potentially significative, there are a
number of challenges given this unique setting. In this paper, we present the design and evaluation methodologies for delivering
immersive learning experience to MOOC learners via multiple media. Specifically, we have applied the techniques in the production of
a MOOC entitled Virtual Hong Kong: New World, Old Traditions, led by AIMtech Centre, City University of Hong Kong, which is the first
MOOC (as our knowledge) that delivers immersive learning content for distant learners to appreciate and experience how the
traditional culture and folklore of Hong Kong impact upon the lives of its inhabitants in the 21st Century. The methodologies applied
here can be further generalised as the fundamental framework of delivering immersive learning for future MOOCs.

Index Terms—Massive Open Online Course, Virtual Reality, Immersive Learning, e-Learning.

1 I NTRODUCTION

T HE concept of Massive Open Online Course (MOOC)

materialised in 2008, when the course Connectivism and
Connective Knowledge (CCK08) became online as an experi-
ment of exploring a new pedagogy in the online environ-
ment [13]. In 2011, the concept of MOOC became well-
known to the general public, when the course Artificial
Intelligence of Stanford University attracted over 160,000
learners across nearly 200 countries in the world. Unlike
CCK08 and several other experimental MOOCs adopting
the connectivism approach [48], the Stanford AI course
follows the traditional behaviourist approach [49]. Stephen
Downes introduced the terminology c-MOOC and x-MOOC
to distinguish MOOCs applying two different pedagogical
approaches. c-MOOCs emphasise the openness and net-
working among learners and facilitators; anyone can con-
tribute to the contents. On the contrary, x-MOOCs are more
facilitator-centric; the contents are prepared by the facilita-
tors. Traditional behaviourist approach treats learning as the
process of conditioning to respond to the environment [52]
[21]. Recent research studies show that besides behavioural
responses to the environment, affects can also significantly
contribute to learning, yielding new theories such as the
cognitive-affective theory of learning [38]. Hence, facilitators
of modern MOOCs are expected not only to create appro-
priate learning environments for conditioning behavioural
responses, but also to make the environments affectively
attractive to learners. In practice, this tends be challenging.
• Horace Ip and Chen Li are with AIMtech Centre and the Depart-
ment of Computer Science, Selena Leoni is with AIMtech Centre,
Yangbin Chen and Qing Li are with the Department of Computer
Science, and Ka-Fai Ma and Calvin Wong are with the Department
of Chinese and History, City University of Hong Kong, Hong Kong
SAR. Contacts: see http://aimtech.cityu.edu.hk/, http://cs.cityu.edu.hk/,
and http://cah.cityu.edu.hk/
Due to the limitation of web technologies, the learning ac-
tivities of MOOCs are usually limited to watching video lec-
tures and/or tutorials, taking quizzes and/or examinations,
joining discussions, etc. There is not enough facilitation
for MOOC educators to practice various instructional plan
and/or pedagogical designs given the setting. Actually, a
recent survey raises concerns about the instructional quality
of MOOCs [34].
With the emergence of immersive learning approaches
enabled by Virtual Reality (VR) technologies, educators
would have greater freedom in MOOC designing, given the
distant learning environments. During immersive learning,
learners are expected to feel being telepresent in virtual sce-
narios [51]. The three-dimensional virtual scenarios could
be recreations of real life scenes, or be purely imagined to
facilitate the practice of particular learning theories, such
as the behaviourist approach and the cognitive-affective
approach [20] [23]. Although it sounds promising, there are
a number of challenges to deliver immersive learning in
MOOC settings; and the approach is yet to be empirically
studied. These challenges and questions may include (1)
given the various accessibility to the enabling technologies
and the highly diverse levels of Information Communica-
tion Technology (ICT) skills of the MOOC learners, how
to effectively support and facilitate immersive learning in
the setting; (2) how to design and deliver the immersive
learning contents to achieve satisfied immersive learning
experience while minimising the technical requirements;
and (3) how introducing immersive learning experience for
MOOCs could impact the overall learning experience after
all.
In this paper, we present a pioneering x-MOOC entitled
Virtual Hong Kong: New World, Old Traditions (VirtualHK),
which is developed by an interdisciplinary team at City

1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES
University of Hong Kong aiming to take on the challenges
and to investigate the questions regarding the delivery of
immersive learning in MOOC settings. The development
of the VirtualHK MOOC follows an iterative approach
based on evaluations during three stages of the MOOC
development, namely the course planning and design stage,
prototyping stage, and post-launch stage. The whole devel-
opment process can be considered as a guiding framework
for future works. We organise this paper as follows. In the
next section, we review the articles on immersive learning,
evaluation methods on immersive learning experience, and
MOOC. Section 4 describes the design and production of
the VirtualHK MOOC, with the evaluation methods we
employed in this study. Section 5 shows the results of
three levels of evaluations. In section 6, we discussed the
results and how the results could help answer the research
questions we proposed. Finally, we conclude the study in
the last section.
2 R ELATED W ORK
2.1 Immersive Learning and Virtual Reality
The concept of immersive learning is to use technologies,
especially computer graphics and human-computer inter-
action technologies, to create simulated virtual worlds, in
which learning can take place by employing appropriate
instructional and pedagogical approaches [20] [8] [42]. The
simulations can be designed for creating a generic virtual
world where learners could learn collaboratively and syn-
chronously. For example, Second Life1 utilises the web-
based graphics rendering technologies to create a virtual
world where people can gather. The Second Life virtual
world is widely used for educational purposes, because it
allows learners to create avatars representing themselves
in the virtual world, interact with each other, the virtual
environments and the virtual objects, and enables collabo-
rative learning in the virtual world [2]. In order to provoke
learners’ motivation and engagement with learning in vir-
tual worlds, a reward system with clearly defined rules can
also be incorporated and the completions among learners
are somehow encouraged. This technique is usually referred
as gamification [17] [18]. Another common design of the
simulations is to depict very specific real life scenarios. In
this type of simulations, learners are allowed and encour-
aged to test their hypotheses of the effects of inputs on the
intended outcomes. For example, VfrogTM is a very specific
simulation that allows learners to conduct frog dissection
using virtual apparatus [29], while Mr. VetroTM simulates
real life medical scenarios that provides medical students
the opportunities to practice their newly gained skills in
the simulated virtual world before practicing the skills on
real patients [22]. This type of simulations do not require
learning happen in a synchronous manner, which might be
easier to fit into the MOOC learning setting.
Speaking of the design of immersive learning contents
and experience, pedagogy is a very important factor to
the success, hence a few pedagogical models have been
proposed. Dalgarno and Lee’s model of learning in three-
dimensional virtual worlds is one of the well-known model
1. http://www.secondlife.com/
1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
2
that describes immersive learning experience and pedagogy
[6]. Dalgarno and Lee defined three-dimensional virtual
worlds for education purposes by three characteristics, ie.
the illusion of three dimensions, smooth temporal and phys-
ical changes, and a high level of interactivity. This can be
considered as an extension to Steuer’s definition on virtual
reality [51]. Dalgarno and Lee believed that with these
characteristics fulfilled, many pedagogical approaches, such
as experiential learning and contextual learning, could be
employed in the virtual worlds. Fowler extended the peda-
gogy of immersive learning based on Dalgarno and Lee’s
model [16]. Instead of emphasising on the technological
affordances, Fowler focused more on the pedagogical re-
quirements. The proposed model aligned learning stage,
learning objectives, and learning activities, resulting a more
complete model that integrates learning and the technology.
In most cases, to achieve certain degree of immersion,
special VR environments and/or devices with the ability of
enabling stereoscopic viewing are employed for the delivery
of immersive learning experience. For example, Ip et al.
used a 4-side Cave Automatic Virtual Environment (CAVE)
[4] [5] to deliver immersive learning experience to school-
aged children with Autism Spectrum Disorders (ASD) [24].
The CAVETM consists of three back projection screens and
one front projection screen. All four screens and correspond-
ing projectors are mounted on a customised supporting
structure. A similar environment was also used for chem-
istry education [31]. These project-based VR environments
typically require special venues and are very challenging
to be employed in large quantities, limiting the number of
potential beneficiaries. On the other hand, the new gener-
ation of Head-mounted Displays (HMDs), such as Oculus
Rift and HTC VIVE, do not require special venues and are
highly portable. Hence, delivering immersive learning via
HMDs is practical in some settings. For example, Lartigue
et al. employed Oculus Rift HMDs in elementary schools
for biology education [28]. Pulijala, Ma and Ayoub used the
same device with Leap Motion for training oral and max-
illofacial surgeons [45]. As the new generation of HMDs are
getting more and more popular and accessible, we expect to
see more research studies that employ HMDs for education
purposes. However, at the current stage, HMDs are not
widely used for delivering immersive learning experience
to mass audiences.
In 2014, Google introduced a low-cost personal virtual
reality solution called Google Cardboard2 (see Fig. 1) to
deliver immersive contents to smartphone users [50]. Unlike
high performance HMDs such as Oculus Rift and HTC VIVE
which are driven by powerful rendering computers and are
most likely aiming at gaming and entertainment, this low-
cost cardboard made device works with smartphones and
does not require other special hardware configurations. The
popularity of smartphones all around the world and the
great smartphone compatibility of the device make virtual
reality accessible to the general public for the first time.
Papachristos et al. studied whether the low-cost personal
virtual reality device could actually affect learning [43].
Results show that there are no significant differences in
the variables studied, including spatial presence, usabil-
2. http://vr.google.com/cardboard
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES
ity, workload, simulator sickness, satisfaction, and learning
gain; Cardboard-based VR systems could provide accept-
able levels of immersive experience to users and contribute
to the pedagogical use of VR. However, it has not been used
to deliver immersive learning experience in MOOC settings.
Fig. 1. The Google Cardboard headset we used for this study.
2.2 Design and Evaluation of Immersive Learning
Since the early days of immersive learning, there is an
emphasis on the evaluation of learning effectiveness [3].
Immersive learning allows learners to freely explore, ex-
perience, interact with objects and characters, and try out
new ideas and solutions in the virtual environment through
experiential learning approach. The uniqueness of immer-
sive learning brings both opportunities and challenges to
the design of the virtual learning environment. Most of
the recent research studies utilise pre-post tests and/or
randomised controlled trials to evaluate the design of im-
mersive learning experience [23] [37]. However, the tests
and trials can only be carried out when the entire immersive
learning contents or at least a prototype is ready. Design
principles and guidelines for designing immersive learning
experience and its associated contents are needed.
The four-dimensional framework for designing and
evaluating immersive learning experience in a virtual world
proposed by De Freitas et al. is one of these guidelines
[10]. The four dimensions need to be considered when
designing immersive learning experience are (a) learner
specifics, (b) pedagogy, (c) representation, and (d) context.
Considering the learner specifics help designers better un-
derstand their learning needs and their environment. Also,
learners’ information and communication technology (ICT)
competencies also need to be considered since immersive
learning usually rely on advanced technologies, particu-
larly virtual reality and natural interface technologies. The
pedagogy dimension analyses which teaching and learning
approaches should be applied and which kind of teaching
and learning activities should be carried out in the virtual
environment as well as outside the virtual environment
as a supplementary to help with knowledge transfer from
the virtual environment to real world scenarios. The third
dimension, representation, analyses the contents. Similar to
Steuer’ two definitive dimensions of virtual reality namely
vividness and interactivity [51], the representation dimen-
sion considers the fidelity of the contents as well as how
the environment and objects inside the environment should
respond to learners’ actions. The last dimension takes into
account the setting where immersive learning takes place
in the overall design. This frequently ignored dimension is
1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
3
very important because the availability of the technology
in a given setting and the level of support provided by
the educators had been proven to be critical to the final
outcomes of immersive learning [23] [37].
2.3 Massive Open Online Courses (MOOCs)
Massive Open Online Courses (MOOCs) are believed to
be a way of changing education by delivering high qual-
ity educational contents via the Internet thus overcomes
geographic and economic barriers. The concept of MOOC
origins from the online course Connectivism and Connective
Knowledge (CCK08) developed by the University of Man-
itoba. CCK08 employed the connectivist approaches. The
course follows a loose and decentralised structure, empha-
sising the learner-learner and learner-knowledge connec-
tion while weakening the connection between learners and
educators Similar MOOCs that employ the connnectivist
approaches are categorised as c-MOOCs, in order to be dis-
tinguished from x-MOOCs, which follows a more traditional
linear plan of instruction [13]. From the educators’ perspec-
tive, content preparation of both types of MOOCs tends to
be challenging due to the nature of MOOCs, including the
complicated and unforeseeable demographics of learners
[26].
There are more recent studies focusing on the quality of
MOOCs. Margaryan et al. pointed out that although most
of the 76 MOOCs randomly sampled by their team are well
organised and have well-presented course materials, the
instructional design quality of these MOOCs are relatively
low [34]. As retention rate is a very important metric to mea-
sure MOOC, Adamopoulos employs the Grounded Theory
Method (GTM) to quantitatively analyse 133 MOOCs and
find that professors (educators), assignments and course
materials are the key factors which positively influences the
completion rate of MOOCs [1]. Moreover, the varieties of
learning activities in MOOC settings are relatively limited.
The question, whether these browser-based video-centric
learning activities could negatively affect learning experi-
ence of learners particularly in appreciating and acquiring
certain tacit knowledge that the course content aim to de-
liver, still remains.
3 I MMERSIVE L EARNING C ONTENTS
VirtualHK MOOC combines contents delivered in the tra-
ditional forms together with immersive learning contents.
The combination of both forms of learning contents could
enhance the learning experience and enable learners to
better acquire certain tactic knowledge and to appreciate the
spiritual aspect of Hong Kong cultures by having the experi-
ence of being presence in the actual events or environments.
In Section 3.1 and 3.2, we only cover the production and
delivery of the immersive learning contents.
3.1 Content Design and Production
The three-week contents of VirtualHK MOOC are structured
by following the three important elements in the eastern
culture, saying Heaven, Earth and Mankind. There are a
total number of three types of contents need to be produced
for immersive learning. The first type, Virtual Guided Tour,
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES 4

tours learners around various sites. Learners can extract

information by interacting with the displays and objects
in the virtual scenarios, which are the recreations of the
sites in real life. The information may be presented in the
form of audio narratives, video clips, photo slides, three-
dimensional models and animations, etc. The second type,
Spherical Panoramic Documentary, documents events, such (a) Raw images captured by the digital camera
as ritual practices, carnivals, parades, etc., from the first
person perspective3 . The learners are allowed to teleport
from a few locations where the spherical panoramic cameras
were placed to record the events. Both the virtual guided
tour and the spherical panoramic documentary heavily
rely on image and video processing techniques. The third
type, Animated Interactive Story, uses computer animation
techniques to recreate folklore and historical events with
narrations. By introducing these three types of contents, we
hope that learners could better understand the traditions (b) Stitched and post-processed spherical
and the current social statues in Hong Kong; and such panoramic image
understandings might arouse certain affects of the learners,
such as being motivated and feeling of enjoyment. The
learners are expected not only to gain some personal feeling
about Hong Kong, its people and its cultures, but also to
value and organise based on their understandings towards
the communal environments and spirits of this dynamic
society [27].
The production of the virtual guided tours begins by
capturing high resolution digital pictures on site. Specifi-
cally, for each of the scenes we have developed, several
(c) Stitched spherical panoramic image viewed in
locations are chosen based on how the learning contents Cardboard VR device
are distributed in the physical space. Locations belonging to
the same scene will be interconnected in the later stages Fig. 2. Captured images and the stitched spherical panoramic image.
when we develop the VR software in Unity3D, so that
learners can ”teleport” to any location freely. For each of
neighbouring pictures and transforming the pictures based
the locations, 38 pictures are taken by the digital camera
on the starting perspective using the homography matrix
with the 16mm wide-angle lens mounted on a panoramic
H . The final spherical panorama is then further edited in
tripod head (see Fig. 2). The panoramic tripod head guaran-
Adobe Photoshop to remove any artefacts. The panoramas
tees that the camera is always rotated around the entrance
are imported into Unity3D with other materials, such as
pupil of the lens. In many cases, the pictures need to be
audio recording, to enable interactions. The production of
further processed before being stitched together to form the
the spherical panoramic documentaries is similar to that
spherical panorama. The stitching process consists of four
of the virtual guided tours. The only difference is that the
steps, namely feature extraction, registration, homography
videos are captured by 12 GoPro cameras mounted on a
calculation, and warping and stitching. First, Scale Invari-
3D-printed rigid. Although 6 GoPro cameras are enough
ant Feature Transform (SIFT) features [33] are extracted
to capture the spherical panoramic video, true stereoscopy
from each of the 38 pictures. These features describe the
is enabled by employing 12 cameras, of which 6 cameras
local features in the pictures. The scale invariance nature
are used to capture the video for the left-eye view while
of SIFT algorithm can help with the registration process,
the other 6 are used to produce for the right-eye view. The
during which two points in two overlapped pictures (a.k.a.,
underlying algorithm for stitching is the same as we used to
neighbouring pictures) can be paired. Specifically, we em-
stitch still images.
ployed the Fast Library for Approximate Nearest Neighbors
(FLANN) [39] to achieve the registration. For each two
neighbouring pictures, namely picture i and picture j , the 3.2 Content Delivery
registration process outputs the paired points as a list of The VirtualHK MOOC consists of both the immersive learn-
tuples (Ii , Ij ), where Ii and Ij are the coordinates of the ing contents and the traditional (non-immersive) learning
matching points in the two overlapped pictures respectively. contents. The traditional learning contents are delivered
RANSAC is the chosen algorithm to calculate the homogra- through FutureLearn.com4 . The challenges of delivering the
phy H , a 3-by-3 matrix, by solving the equation Ii = H × Ij immersive learning contents include (1) how to ensure the
[15]. With the homography matrix H calculated for each technology barriers do not hinder learners from experienc-
picture pair, the stitching is achieved by matching two ing the immersive learning contents; and (2) how to connect
the immersive learning contents with the traditional.
3. One of the spherical panoramic documentaries can be viewed by
visiting https://youtu.be/dhHNIypYsmo. 4. https://www.futurelearn.com/courses/virtual-hong-kong

1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES
To cater learners who may have different level of acces-
sibility to the necessary technologies, we provide two ways
of experiencing the immersive learning contents. The first
way is to view the contents non-interactively via YouTube.
YouTube supports spherical panoramic video playback and
allows learners to change the viewing perspective by hold-
ing the left mouse button down and drag. Most of the mod-
ern internet browsers on PCs, such as Chrome, Opera, Fire-
fox, Edge, etc., natively support these features. However, the
videos are pre-rendered and are not stereoscopic, giving a
less interactive and less immersive experience. The second
way is to experience the immersive learning contents via
the VirtualHK smart phone App5 in the Google Cardboard
headset (see Fig. 1). The App is developed for both the
iPhones and the Android phones. Learners can download
and install the App to their compatible smart phones via
Apple App Store or Google Play Store respectively. Learners
are free to choose the way of accessing the relevant contents.
To facilitate a smooth transition from the traditional
learning contents, which come in the form of textual articles
and videos, to the immersive learning (3D) contents, we
have embedded a QR code scanner into the VirtualHK smart
phone App. The QR code scanner can be used to scan the
QR codes, which are embedded as context-aware tags into
the traditional learning contents and are designed to work
as access points to the immersive learning contents. For
example, a traditional non-immersive lecture video about
the Man Mo Temple may have a QR code shown up during
video playback and by scanning the QR code using the
App’s built-in QR code scanner, the immersive learning
contents about the Man Mo Temple will be loaded auto-
matically, seamlessly providing a context-aware immersive
learning experience. These QR codes are embedded into
the content pages of articles, discussion topics, and even
assessments as a way of directly accessing the immersive
learning contents in the corresponding learning contexts. To
fully facilitate learners with different levels of information
technology literacy, a tutorial video on the VirtualHK smart
phone App is provided to learners at the beginning of the
course. For learners who can only access the immersive
learning contents on YouTube, we also provide direct links
with time codes.
4 M ETHOD
During each stage of the development of the MOOC, var-
ious evaluations were carried out to ensure the learning
experience. There are mainly three stages of the develop-
ment process, namely course planning and design stage,
prototyping stage, and post-launch stage, during which
evaluations were carried out. During the course planning
and design stage, we follow the four-dimensional framework
for designing and evaluating immersive learning experience in a
virtual world proposed by De Freitas et al. [10], which covers
learner specifics, pedagogy, representation, and context of
the given setting.
When the prototype of the smart phone App and the
relevant traditional contents are ready, we conducted a ran-
5. The VirtualHK smart phone App can be downloaded from App
Store or Google Play Store by search ”VirtualHK” or ”Virtual Hong
Kong”.
1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
5
domised controlled trial on two groups (experiment group
and control group) from 60 (29 males and 31 females) non-
local undergraduate students who have spent less than 24
consecutive months in Hong Kong. The participants of the
experiment group were asked to experience a designated
part of the MOOC with immersive learning contents. We
converted the immersive learning contents into an article
by capturing normal pictures of the scenes and describing
relevant information as text, in order to make a dummy for
the control group, so that participants of the control group
can access the similar level of detailed information in a
traditional (non-immersive) way. Because previous research
studies suggest that introducing immersive learning can
make the learning process more interesting and enjoyable
[7] [9] [47], and can make learners feel more motivated
[12], leading to better performance, the evaluation exper-
iment aims to answer three research questions regarding
the immersive learning experience based on learners with
similar pre-knowledge on Hong Kong culture and society.
The research questions are
1) Is the knowledge gain of the participants from the
experiment group better than that of the partici-
pants from the control group?
2) Compared to the participants of the control group,
do the participants from the experiment group feel
more motivated when experiencing the MOOC?
3) Compared to the participants of the control group,
do the participants from the experiment group feel
more enjoyable when experiencing the MOOC?
The instruments we used here are adapted from [25] for
measuring the motivation, and from [40] and [14] for mea-
suring enjoyment, consisting a total number of 48 5-point
Likert scale questions. We also developed 9 multiple choice
questions for knowledge test. Note that only the knowledge
tests were administrated before and after learners experienc-
ing the designated part of the VirtualHK MOOC; the mo-
tivation and enjoyment questionnaires were administrated
only after learner experiencing the designated part of the
MOOC.
The first run of the VirtualHK MOOC was launched on
April 26, 2017. A total number of 2,747 learners joined the
MOOC, of which 350 are social learners who have posted as
least one comment. In the post-launch stage of evaluation,
the collected comments were constantly analysed using
the classic lexicon-based sentiment analysis method [32],
in order to help educators evaluate the learning experi-
ence and monitor learner feedbacks more effectively in the
asynchronous learning environment. Compared to machine
learning based approaches, lexicon-based approach does not
require a huge amount of data to train the model. However,
choosing the right dictionary could impact the performance
of the lexicon-based approach significantly [46]. The dictio-
nary we chose is a classic dictionary [32], which has been
applied for opinion mining and sentiment analysis in online
learning settings before (e.g., [30] [41] and [19]). By the end
of the first run of the VirtualHK MOOC, a total number of
2,298 comments and feedbacks were posted by the learners.
The rest of the paper will only cover the analysis of the
posts under those 6 steps, saying step 1.3 What do you know
about Immersive Learning?, step 1.4 Practical guide to set up the
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES
VR equipment, step 1.7 Man Mo Temple and Hong Kong, step
2.2 Cheung Chau Tai Ping Ching Chiu: the origins, step 2.9 A
bun and a village and 3.7 Isolation and the quest for community
in a cage home, where learners were exposed to immersive
learning experience due to the scope of this paper.
5 R ESULT
5.1 Course Planning and Design Evaluation
Following the four-dimensional framework for designing and
evaluating immersive learning experience in a virtual world
proposed by De Freitas et al. [10], the course planning
and design evaluation aims to systematically review every
aspects of the VirtualHK MOOC involving immersive learn-
ing, in order to better align the to-be-delivered experience
with the overall objective of the MOOC, which is to help
the general public, both local and non-local, to understand
thus appreciate the uniqueness of culture and the society in
Hong Kong.
For the first domain, learner specifics, our target learners
are MOOC learners. Due to the nature of the MOOC setting,
the learners’ demographics tend to be highly diverse. As a
result, the information technology literacy of learners is hard
to predict. Hence, the design of the smart phone App, espe-
cially the user experience design, needs to be as intuitive as
possible, so that learners with various levels of information
technology literacy can use the App easily. In the App,
we visualise a virtual pointer at the centre of the viewing
space. The pointer can be pointed to navigation markers and
hotspots in the virtual scenes, enabling learners to teleport
from one location to another and extract learning contents
in multimedia forms. To further ease the process of content
delivery, we decided to use the QR codes as contextual tags
and build a QR code scanner inside the App. The relevant
immersive learning contents can be automatically loaded
by scanning the QR codes embedded in the traditional
contents.
The second domain is pedagogy. The pedagogical ap-
proach we apply for this project, is experiential learning
and affective learning. Virtual reality environment is ideal
for applying experiential learning. The environment is safe
and controllable, so learners can experience, experiment and
reflect in the virtual scenes freely. Virtual reality environ-
ment is also ideal for affective learning. Here, we hope
the virtual reality environment can provide learners the
feel of presence, so that they can appreciate Hong Kong
culture in the scene ”Man Mo Temple and Hong Kong”,
and can reflect and empathise towards social welfare and
community solidarity in the scene ”Isolation and the Quest
for Community in a Cage Home”. These objectives are based
upon receiving the information and knowledge embedded
in the virtual scenes, waiting for the learners’ discovery and
responding.
The third domain, representation, takes the design of
the immersive learning contents into consideration. To bet-
ter invoke appreciation and empathy of the learners, the
high resolution spherical panoramic images are captured
and produced in the virtual reality environment, which
maximise the feel of presence in the scene. As reported by
learners, the overall experience of immersion is very high.
Several local learners even reported that they notice new
1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
6
TABLE 1
The subjects’ durations of residence in Hong Kong.
Experiment Group Control Group
(n=30) (n=30)
0-3 months 3 (10.00%) 3 (10.00%)
4-6 months 5 (16.67%) 8 (26.67%)
7-9 months 11 (36.67%) 8 (26.67%)
10-12 months 4 (13.33%) 0 (0.00%)
13-24 months 7 (23.33%) 11 (36.67%)
things in the virtual scenes even they go to the Man Mo
Temple to pray regularly.
Lastly, the context domain, which considers the learning
settings. For the given setting, the learning happens without
direct facilitation from the educators or mentors. Hence,
it is quite important to provide enough resources for the
learners to get the necessary equipment and to provide
alternative ways of content delivery when the necessary
equipment is not accessible. As a result, we not only drafted
documentations and tutorial videos to help the learners, but
also provided alternative ways of delivering the contents.
5.2 Prototype Evaluation
A total number of 60 subjects participated in the prototype
evaluation study. 29 of them are males, 31 are females. The
subjects are all the University’s non-local students. 22 of the
60 subjects speak English as first language. Because those
subjects who speak other languages as their first languages
need to meet the University’s requirements on English profi-
ciency, we assume that the medium of teaching, ie. English,
would not affect the subjects’ comprehension of the MOOC
contents. Of all the subjects, no one has reported that they
have lived in Hong Kong for more than 24 consecutive
months. The durations of the subjects’ residence in Hong
Kong are described in Table 1. The subjects were randomly
assigned to either the experiment group, of which the sub-
jects were exposed to the immersive learning contents, or
the control group, of which the subjects were exposed to the
same contents in the traditional ways.
To investigate whether immersive learning results better
knowledge gain, we used t-tests on the subjects’ perfor-
mances in the knowledge test. The mean score of the exper-
iment group (Mean=3.07, SD=1.66) is only slightly higher
than that of the control group (Mean=3.03, SD=1.40) in the
pre-tests. T-test shows that the difference is insignificant
(t=0.084, p=0.933). Hence, the pre-knowledge levels of the
subjects from two groups are very similar. The post-test
results tell a similar story. The mean score of the experiment
group (Mean=8.30, SD=0.79) is again higher than that of
the control group (Mean=8.07, SD=0.94). However, t-test
shows that the difference is insignificant (t=1.036, p=0.305).
Hence, based on the results, although the knowledge gain
of the subjects from both groups is observed, there is no
significant difference in the knowledge gain between the
two groups. By adjusting for the pre-test results, analysis of
covariance (ANCOVA) further confirms the results (Mean
Square=0.764, p=0.282). The details are presented in Table 2.
To measure and compare the subjects’ motivation during
learning, the adapted version of the Instructional Materials
Motivation Survey (IMMS) [25] was administrated. The
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES 7

TABLE 2
Comparison of performances in the knowledge test.

Experiment Group Control Group

(n=30) (n=30)
Mean SD Mean SD t df p Mean Diff.
Pre test 3.07 1.66 3.03 1.40 0.084 58 0.933 0.033
Post test 8.30 0.79 8.07 0.94 1.036 58 0.305 0.233
*p < .05, **p < .01, ***p < .001

TABLE 3
Comparison of the four sub-scales of IMMS.

Experiment Group Control Group

(n=30) (n=30)
Mean SD Mean SD t df p Mean Diff.
Attention 52.00 2.729 49.43 4.987 2.473 58 0.016* 2.567
Relevance 35.10 2.510 36.00 3.301 -1.189 58 0.239 -0.900
Confidence 37.10 2.412 36.27 3.750 1.024 58 0.310 0.833
Satisfaction 26.20 2.024 24.77 2.873 2.234 58 0.029* 1.433
*p < .05, **p < .01, ***p < .001

adapted IMMS consists of 36 5-point Likert scale questions
(see Appendix A), covering four sub-scales, namely atten-
tion, relevance, confidence, and satisfaction. The subjects
from the experiment group reported better motivation in
the attention, confidence, and satisfaction sub-scale, but not
in the relevance sub-scale, when being compared to the
subjects from the control group. Specifically, for the attention
sub-scale, the accumulated mean score of the experiment
group (Mean=52.00, SD=2.729) is higher than that of the
control group (Mean=49.43, SD=4.987). T-test shows that the
difference in accumulated mean score is significant (t=2.473,
p=0.016). For the relevance sub-scale, although the accu-
mulated mean score of the experiment group (Mean=35.10,
SD=2.510) is slightly lower than that of the control group
(Mean=36.00, SD=3.301), t-test shows that the difference
is insignificant (t=-1.189, p=0.239). For the confidence sub-
scale, the accumulated mean score reported by the subjects
of the experiment group (Mean=37.10, SD=2.412) is higher
than that reported by the subjects of the control group
(Mean=36.27, SD=3.750), but t-test shows that the difference
is insignificant (t=1.024, p=0.310). Lastly, for the satisfaction
sub-scale, the accumulated mean score of the experiment
group (Mean=26.20, SD=2.024) is higher than that of the con-
trol group (Mean=24.77, SD=2.873). And t-test confirms the
difference is significant (t=2.234, p=0.029). Detailed results
are reported in Table 3.
To measure learners’ enjoyment, a total number of 12 5-
point Likert scale questions adapted from [40] and [14] (see
Appendix B) were administrated. The instrument covers
three dimensions of enjoyment, namely affect, behaviour,
and cognition. For all three dimensions, subjects of the ex-
periment group reported higher level of enjoyment during
learning in terms of accumulated mean ratings, when being
compared with subjects of the control group did. Specifi-
cally, for the affect dimension, the accumulated mean score
of the experiment group (Mean=20.97, SD=1.691) is higher
than that of the control group (Mean=19.87, SD=2.300). For
the behaviour dimension, the accumulated mean score of
the experiment group (Mean=15.30, SD=1.579) is higher
than that of the control group (Mean=13.97, SD=2.671).
Similar difference can be seen for the cognition dimen-
sion; the accumulated mean score of the experiment group
(Mean=12.17, SD=1.464) is higher than that of the control
group (Mean=10.87, SD=1.852). T-tests show that the dif-
ferences in reported level of enjoyment are all statistically
significant. Detailed results are reported in Table 4.
5.3 Post-launch Evaluation
The first course run of the VirtualHK MOOC was launched
on April 26, 2017. There are a total number of 2,747 learners
joined the first course run, among which 1,813 are learners
who have visited the course page, 350 are social learners
who have posted as least one comment. 20.3% of the learners
finished more than 90% of the course contents. The detailed
course measures in terms of the numbers of learners can be
found in Table 5.
After the first course run of the three-week course, a total
number of 2,298 comments and feedbacks were posted by
the learners. Lexicon-based approach is chosen to analyse
these posts. This approach classifies each of these posts
as ”positive”, ”negative”, or ”neutral”. Only the analysis
results for those 6 steps, involving a total number of 296
posts, where learners were heavily exposed to the immer-
sive learning experience are reported in Table 6 due to the
scope of this paper. Specifically, step 1.3 What do you know
about Immersive Learning? is an introductory video lecture
that gives an overview of immersive learning. There were
a total number of 55 comments under this step. Sentiment
analysis shows that 29 (52.73%) of them are ”positive” and
26 (47.27%) of them are ”neutral”. Most of the ”positive”
comments are from learners who feel interested and excited
about immersive learning. There were learners who also
raised questions regarding the availability and accessibility
to the necessary equipment. These questions were mostly
categorised as ”neutral” by the sentiment analysis algo-
rithm. Step 1.4 Practical guide to set up the VR equipment
is a video lecture with textural instructions on the VR
equipment as well the software. Under this step, there were
a total number of 72 comments, of which 40 (55.56%) were
categorised as ”positive”, 2 (2.78%) as ”negative”, and 30

1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES 8

TABLE 4
Comparison of the enjoyment measures.

Experiment Group Control Group

(n=30) (n=30)
Mean SD Mean SD t df p Mean Diff.
Affect 20.97 1.691 19.87 2.300 2.110 58 0.039* 1.100
Behaviour 15.30 1.579 13.97 2.671 2.353 58 0.022* 1.333
Cognition 12.17 1.464 10.87 1.852 3.016 58 0.004** 1.300
Combined 48.43 2.921 44.70 5.700 3.193 58 0.002** 3.733
*p < .05, **p < .01, ***p < .001

TABLE 5
Detailed course measures in terms of the numbers of learners.

Total Week 1 Week 2 Week 3

Joiners 2,747 N/A
Learners 1,813 (66.0%) 1,806 697 618
Active Learnersa 1,294 (71.4%) 1,283 600 534
Social Learnersb 350 (19.3%) 306 148 149
Learners with ≥ 50% step completion 530 (29.2%) N/A
Learners with ≥ 90% step completion 368 (20.3%) N/A
a
Learners who have finished at least one step of the course.
b
Learners who have posted at least one comment or feedback.

TABLE 6
Sentiment analysis results of comments under course steps related to immersive learning.

Step Total Positive (%) Negative (%) Neutral (%)

1.3 55 29 (52.73) 0 (0.00) 26 (47.27)
1.4 72 40 (55.56) 2 (2.78) 30 (41.67)
1.7 39 25 (64.10) 2 (5.13) 12 (30.77)
2.2 34 16 (47.06) 4 (11.76) 14 (41.18)
2.9 27 22 (81.48) 3 (11.11) 4 (14.81)
3.7 69 25 (36.23) 22 (31.88) 22 (31.88)

(41.67%) as ”neutral”. Like step 1.3, most of the learner 6 D ISCUSSION

asked questions regarding the VR equipment and other
technology-related topics were categorised as ”neutral”. We
looked at the two negative comments and found that both
comments were complaining about the availability of the
equipment in their countries/regions. The mentors of the
MOOC quickly provided ways of purchasing the equipment
to the learners and informed them about the alternative
ways of enjoying the immersive learning contents without
the VR equipment. The learners were satisfied and further
posted that they would like to continue learning without the
VR equipment. The distributions of sentiment of comments
under step 1.7 Man Mo Temple and Hong Kong, step 2.2
Cheung Chau Tai Ping Ching Chiu: the origins, and step 2.9
A bun and a village are very similar. There were a total
number of 39 comments posted under step 1.7, of which
25 (64.10%) were ”positive”, 2 (5.13%) were ”negative”, and
12 (30.77%) were ”neutral”. There were a total number of 34
comments posted under step 2.2, of which 16 (47.06%) were
”positive”, 4 (11.76%) were ”negative”, and 14 (41.18%) were
”neutral”. And there were a total number of 27 comments
posted under step 2.9, of which 22 (81.48%) were ”positive”,
3 (11.11%) were ”negative”, and 4 (14.81%) were ”neutral”.
Unlike these three steps, step 3.7 Isolation and the quest for
community in a cage home received much higher number of
comments (69 posts) and the percentage of ”negative” com-
ments was also the highest (31.88%) among all immersive
learning related steps.
The course planning and design evaluation allows us to
systematically review the immersive learning experience we
were designing for the VirtualHK MOOC. By following the
four-dimensional framework, the team realised that the biggest
challenges of delivery immersive learning experience in
MOOC settings are availability and accessibility of the en-
abling technologies, and the supportive resources. To deal
with the potential problem related to the limited accessibil-
ity of the enabling technologies, including the availability of
compatible smart phones and Google Cardboard goggles,
the team decided to deliver the immersive learning contents
through various mediums. The first medium we used is the
smart phone App developed by us. In this case, learners
need to have both compatible smart phones and Google
Cardboard goggles in order to use the App; this solution can
provide the best immersion and highest level of interactions
to the learners at low-cost. Besides, the team decided to have
a VR off mode built in the App (see Fig. 3). By enabling this
mode, the App can work without a Google Cardboard; the
immersion will be sacrificed because the rendered images
cannot be viewed stereoscopically, but interactions are kept
the same as they were in the VR on mode. The second
medium we used is YouTube. YouTube supports spherical
panoramic video playback and allows learners to change the
viewing perspective by holding the left mouse button down
and drag. In this case, as long as the learner uses a modern
web browser to access the FutureLearn platform on which

1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES
(a) VR mode is on (b) VR mode is off
(c) The cage home scene viewed in VR on mode
(d) The cage home scene viewed in VR off mode
Fig. 3. Learners can toggle the VR on/off mode by tapping the VR mode
icon on the bottom left of the App’s main menu.
the MOOC is presented, he/she can access the immersive
learning contents in a less immersive and less interactive
way. To tackle the second challenge, the team prepared
detailed instructions and a tutorial video on how to get
up the VR equipment. Moreover, the course mentors were
trained so that they could provide enough online support to
learners in case they encounter difficulties in accessing the
immersive learning contents.
The prototype evaluation provides us some interesting
insights regarding the immersive learning experience de-
sign in a controlled setting on the population with similar
education backgrounds. Both the experiment group and the
control group show significant knowledge gain after experi-
encing the MOOC. However, there is no statistical difference
between the two groups in the post knowledge tests. One
possible explanation to this phenomenon is that subjects
from both groups performed so well in the post knowledge
tests. So even the mean scores of the experiment group,
which is 8.30 out of 9, is slightly higher than that of the con-
1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
9
trol group, there is simply not enough margin left, resulting
a statistical insignificance. Another possible explanation is
based on the cognitive load theory [44]. Previous research
studies suggest that introducing multimedia contents to
learning through multiple sensory channels could increase
demands of learners’ cognitive process [35] [36]. The smart-
phone App is designed to present the contents in the form
of pictures and words. The words are presented not as texts
displayed on the screen but as audio narrations. Hence,
when learners are using the App, both sensory channels will
be absorbing information at the same time. This might be
challenging for some learners since both channels might be
overloaded with essential processing demands in working
memory. To further investigate this possible explanation, we
interviewed five subjects from the experiment group who
were chosen randomly within a week after they experienced
the prototype. Four of them reported that when using the
App, as they are so attracted to the photo realistic virtual
scenes, they sometimes did not realise the narrations have
already been playing or simply missed the information
partially. Refer to [36], this might be the case of type 2
overload. To avoid the potential overload, the team decided
to divide the continuous narrations into learner-controlled
segments. Specifically, the relevant narration will be played
only if the learner tapped on the corresponding displays or
objects in the virtual scenes. By implementing this mecha-
nism, the learner can control his/her own learning pace and
the information can be present via two sensory channels
sequentially instead of in parallel.
The adapted IMMS instrument also reveals insights re-
garding the effects on learners’ motivation by introducing
immersive learning. Results show that on two sub-scales,
namely attention and satisfaction, the experiment group
reported statistically higher level of motivation than control
group. Although the level of motivation reported by the
experiment group on the confidence sub-scale is higher
than that of the control group, the difference is statisti-
cally insignificant. This matches with our observations and
previous studies using the IMMS as the instrument (e.g.,
[11], which deployed Augmented Reality (AR) technologies
for art education). For the relevance sub-scale, the result is
interesting. According to [25], this sub-scale refers to learn-
ers’ feelings or perception of attraction toward desired outcomes.
Results show that subjects of the experiment group reported
insignificant lower level of motivation in this sub-scale,
when be compared with the control group. To investigate
this may require further studies. A possible way of conduct-
ing such studies would be by introducing instruments for
measuring similar domains of motivation. The enjoyment
measures show that subjects from the experiment group
reported significantly higher level of enjoyment those from
the control group did. Generally speaking, introducing the
immersive learning experience positively contributed to
learners’ motivation and enjoyment in the controlled setting.
By utilising the Lexicon-based sentiment analysis ap-
proach, the course facilitators and educators can monitor
learners’ feedbacks more effectively. There are 6 steps of
the VirtualHK MOOC that are closely related to the im-
mersive learning approach. It is worth mentioning that
the comments under step 3.7, especially those which were
categorised as ”negative”, clearly showed the learners’ con-
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES 10

cerns regarding the living condition of the low-income 9) There were examples that showed me how the
class in Hong Kong. The Lexicon-based sentiment analysis MOOC could be important to some people in the
algorithm shows that the top five most frequent words learning setting.
which implies negative sentiment are ”poor”, ”problem”, 10) Completing the MOOC was important to me.
”shocked”, ”hard”, and ”awful”. The results clearly demon- 11) The quality of the writing in the MOOC helped to
strate that introducing immersive learning experience can hold my attention.
arouse certain affects of the learners, which is one of the core 12) The content of the MOOC is so abstract that it was
objectives of this humanity and cultural heritage MOOC. hard to keep my attention on it. (Reversed)
13) As I watched the MOOC, I was confident that I
could learn the content.
7 C ONCLUSION AND F UTURE W ORK 14) I enjoyed the MOOC so much that I would like to
In this paper, we present the empirical study of designing know more about this topic.
and evaluating immersive learning experience for Massive 15) The design of the MOOC looks dry and unappeal-
Open Online Courses (MOOC). The VirtualHK MOOC, as ing. (Reversed)
our knowledge, is the first MOOC that brings immersive 16) The content of the MOOC is relevant to my interests.
learning experience to the MOOC setting. Evaluation results 17) The way the information is arranged in the MOOC
during the prototyping stage have shown that immersive helped keep my attention.
learning experience may not directly impact the knowledge 18) There are explanations or examples of how people
gain of learning, but can improve the overall learning expe- use the knowledge in the MOOC.
rience, especially in better motivating learners and making 19) The activities in the MOOC were too difficult. (Re-
the learning more enjoyable, if it is carefully designed and versed)
implemented. There are also evidences from the Lexicon- 20) The MOOC has things that stimulated my curiosity.
based sentiment analysis on the learners’ comments show- 21) I really enjoyed learning with the MOOC.
ing that immersive learning experience contributes to the 22) The amount of repetition in the MOOC caused me
affective domain of learning. The Lexicon-based sentiment to get bored sometimes. (Reversed)
analysis approach we applied for this study may be lim- 23) The content and style of writing in the MOOC
ited in terms of distinguishing the topic(s) each comment convey the impression that its content is worth
corresponds to. Although it helps the course facilitators to knowing.
better monitor the learners’ feedbacks, it is worth to further 24) I learned some things that were surprising or unex-
investigate on using more advanced Natural Language Pro- pected with the MOOC.
cessing (NLP) approaches to overcome the mentioned short- 25) After working on the MOOC for a while, I was
coming of the Lexicon-based approach. The methodologies confident that I would be able to pass a test on the
and procedures we applied for the VirtualHK MOOC can content.
be considered as a fundamental framework for delivering 26) The MOOC was not relevant to my needs because I
immersive learning experience for future MOOCs. This already knew most of it. (Reversed)
framework emphases the iterative approach of constant 27) The wording of feedback, or of other comments in
monitoring and evaluation throughout different stages of the MOOC, helped me feel rewarded for my effort.
the MOOC design and production. 28) The variety of reading passages, activities, illustra-
tions, etc., helped keep my attention on the MOOC.
A PPENDIX A 29) The style of writing in the MOOC is boring. (Re-
versed)
A DAPTED I NSTRUCTIONAL M ATERIALS M OTIVATION 30) I could relate the content of the MOOC to things I
S URVEY (IMMS) Q UESTIONS have seen, done or thought about in my own life.
1) When I first looked at the Massive Open Online 31) There are so many words on each MOOC
Course (MOOC), I had the impression that it would screen/page that it is irritating. (Reversed)
be easy for me. 32) If felt good to successfully complete the MOOC.
2) There was something interesting at the beginning of 33) The content in the MOOC will be useful to me.
the MOOC that attracted my attention. 34) I could not really understand quite a bit of the
3) The MOOC was more difficult to understand than I material in the MOOC. (Reversed)
would like it to be. (Reversed) 35) The good organization of the content in the MOOC
4) After watching the introductory video, I felt confi- helped me be confident that I would learn this
dent that I knew what I was supposed to learn from material.
the MOOC. 36) It was a pleasure to work on such a well-designed
5) Completing the MOOC gave me a satisfying feeling MOOC.
of accomplishment.
6) It is clear to me how the content of the MOOC is A PPENDIX B
related to things I already know.
7) The MOOC had so much information that it was
A DAPTED I NSTRUMENT OF M EASURING E NJOY -
hard to pick out and remember the important MENT DURING I MMERSIVE L EARNING
points. (Reversed) 1) I felt attracted to this Massive Open Online Course
8) The interface design of the MOOC is eye-catching. (MOOC).

1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES
2) I became less aware of my surroundings because I
was engrossed in studying the MOOC.
3) I think this MOOC is a good way of learning Hong
Kong culture and traditions.
4) I felt emotionally attached to the MOOC.
5) I think it is worthwhile to study the MOOC.
6) I felt emotionally involved in the MOOC.
7) I think this instructional program is an interesting
way of learning Hong Kong culture and traditions.
8) I lost track of time while studying the MOOC.
9) I felt emotionally affected while studying the
MOOC.
10) I was less self-aware while studying the MOOC.
11) I think this MOOC is an effective way of learning
Hong Kong culture and traditions.
12) I was less worried about my life while studying the
MOOC.
ACKNOWLEDGMENTS
This project is funded by City University of Hong Kong.
R EFERENCES
[1] Panagiotis Adamopoulos. What makes a great mooc? an interdisci-
plinary analysis of student retention in online courses. 2013.
[2] Suzanne C Baker, Ryan K Wentz, and Madison M Woods. Using
virtual worlds in education: Second lifeTM as an educational tool.
Teaching of Psychology, 36(1):59–64, 2009.
[3] Meredith Bricken. Virtual reality learning environments: potentials
and challenges. ACM SIGGRAPH Computer Graphics, 25(3):178–184,
1991.
[4] Carolina Cruz-Neira, Daniel J Sandin, and Thomas A DeFanti.
Surround-screen projection-based virtual reality: the design and
implementation of the cave. In Proceedings of the 20th annual
conference on Computer graphics and interactive techniques, pages 135–
142. ACM, 1993.
[5] Carolina Cruz-Neira, Daniel J Sandin, Thomas A DeFanti, Robert V
Kenyon, and John C Hart. The cave: audio visual experience auto-
matic virtual environment. Communications of the ACM, 35(6):64–73,
1992.
[6] Barney Dalgarno and Mark JW Lee. What are the learning affor-
dances of 3-d virtual environments? British Journal of Educational
Technology, 41(1):10–32, 2010.
[7] Sara De Freitas. Learning in immersive worlds: A review of game-
based learning. 2006.
[8] Sara De Freitas and Tim Neumann. The use of ’exploratory learn-
ing’ for supporting immersive learning in virtual environments.
Computers & Education, 52(2):343–352, 2009.
[9] Sara de Freitas, Genaro Rebolledo-Mendez, Fotis Liarokapis,
G Magoulas, and Alexandra Poulovassilis. Developing an evalu-
ation methodology for immersive learning experiences in a virtual
world. In Games and Virtual Worlds for Serious Applications, 2009.
VS-GAMES’09. Conference in, pages 43–50. IEEE, 2009.
[10] Sara De Freitas, Genaro Rebolledo-Mendez, Fotis Liarokapis,
George Magoulas, and Alexandra Poulovassilis. Learning as im-
mersive experiences: Using the four-dimensional framework for
designing and evaluating immersive learning experiences in a
virtual world. British Journal of Educational Technology, 41(1):69–85,
2010.
[11] Ángela Di Serio, Marı́a Blanca Ibáñez, and Carlos Delgado Kloos.
Impact of an augmented reality system on students’ motivation for
a visual art course. Computers & Education, 68:586–596, 2013.
[12] Michele D Dickey. Engaging by design: How engagement strate-
gies in popular computer and video games can inform instructional
design. Educational Technology Research and Development, 53(2):67–83,
2005.
[13] Stephen Downes. Places to go: Connectivism & connective knowl-
edge. Innovate: Journal of Online Education, 5(1):6, 2008.
1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
11
[14] Xiaowen Fang, Susy Chan, Jacek Brzezinski, and Chitra Nair.
Development of an instrument to measure enjoyment of computer
game play. INTL. Journal of human–computer interaction, 26(9):868–
886, 2010.
[15] Martin A Fischler and Robert C Bolles. Random sample consensus:
a paradigm for model fitting with applications to image analysis
and automated cartography. Communications of the ACM, 24(6):381–
395, 1981.
[16] Chris Fowler. Virtual reality and learning: Where is the pedagogy?
British journal of educational technology, 46(2):412–422, 2015.
[17] James Paul Gee. What video games have to teach us about learning
and literacy. Computers in Entertainment (CIE), 1(1):20–20, 2003.
[18] James Paul Gee. Good video games+ good learning. New York: Peter
Lang, 2007.
[19] Steven C Harris, Lanqin Zheng, Vive Kumar, et al. Multi-
dimensional sentiment classification in online learning environ-
ment. In Technology for Education (T4E), 2014 IEEE Sixth International
Conference on, pages 172–175. IEEE, 2014.
[20] Jan Herrington, Thomas C Reeves, and Ron Oliver. Immersive
learning technologies: Realism and online authentic learning. Jour-
nal of Computing in Higher Education, 19(1):80–99, 2007.
[21] Ernest Ropiequet Hilgard and Donald George Marquis. Condition-
ing and learning. Appleton-Century, 1940.
[22] Andri Ioannidou, Alexander Repenning, David Webb, Diane
Keyser, Lisa Luhn, and Christof Daetwyler. Mr. vetro: A collective
simulation for teaching health science
[23] Horace HS Ip and Chen Li. Virtual reality-based learning envi-
ronments: recent developments and ongoing challenges. In Interna-
tional Conference on Hybrid Learning and Continuing Education, pages
3–14. Springer, 2015.
[24] Horace H.S. Ip, Simpson W.L. Wong, Dorothy F.Y. Chan, Julia
Byrne, Chen Li, Vanessa S.N. Yuan, Kate S.Y. Lau, and Joe Y.W.
Wong. Enhance emotional and social adaptation skills for children
with autism spectrum disorder: A virtual reality enabled approach.
Computers & Education, 117:1 – 15, 2018.
[25] John M Keller. Motivational design for learning and performance: The
ARCS model approach. Springer Science & Business Media, 2009.
[26] René F Kizilcec, Chris Piech, and Emily Schneider. Deconstructing
disengagement: analyzing learner subpopulations in massive open
online courses. In Proceedings of the third international conference on
learning analytics and knowledge, pages 170–179. ACM, 2013.
[27] David R Krathwohl, Benjamin S Bloom, and Bertram B Masia.
Taxonomy of educational objectives, handbook ii: affective domain.
new york: David mckay company. Inc. ISBN 0-679-30210-7, 0-582-
32385, 1, 1964.
[28] Jonathan Lartigue, Tyler Scoville, and Minh Pham. Promoting k-
8 learning using oculus rift: Employing virtual reality to increase
learning outcomes in elementary biology. In E-Learn: World Confer-
ence on E-Learning in Corporate, Government, Healthcare, and Higher
Education, pages 1100–1105. Association for the Advancement of
Computing in Education (AACE), 2014.
[29] EA-L Lee, Kok Wai Wong, and Chun Che Fung. Learning effec-
tiveness in a desktop virtual reality-based learning environment.
2009.
[30] Chee Kian Leong, Yew Haur Lee, and Wai Keong Mak. Mining
sentiments in sms texts for teaching evaluation. Expert Systems with
Applications, 39(3):2584–2589, 2012.
[31] Maria Limniou, David Roberts, and Nikos Papadopoulos. Full
immersive virtual environment cave tm in chemistry education.
Computers & Education, 51(2):584–593, 2008.
[32] Bing Liu. Sentiment analysis and subjectivity. Handbook of natural
language processing, 2:627–666, 2010.
[33] David G Lowe. Distinctive image features from scale-invariant
keypoints. International journal of computer vision, 60(2):91–110, 2004.
[34] Anoush Margaryan, Manuela Bianco, and Allison Littlejohn. In-
structional quality of massive open online courses (moocs). Com-
puters & Education, 80:77–83, 2015.
[35] Richard E Mayer. Multimedia learning. Psychology of learning and
motivation, 41:85–139, 2002.
[36] Richard E Mayer and Roxana Moreno. Nine ways to reduce
cognitive load in multimedia learning. Educational psychologist,
38(1):43–52, 2003.
[37] Zahira Merchant, Ernest T Goetz, Lauren Cifuentes, Wendy
Keeney-Kennicutt, and Trina J Davis. Effectiveness of virtual
reality-based instruction on students’ learning outcomes in k-12
and higher education: A meta-analysis. Computers & Education,
70:29–40, 2014.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TLT.2018.2878700, IEEE
Transactions on Learning Technologies
IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES 12

[38] Roxana Moreno. Instructional technology: Promise and pitfalls. Chen Li received his BSc in computer science and technology from
Technology-based education: Bringing researchers and practitioners to- Nanjing University in 2008, MSc and PhD in computer science from City
gether, pages 1–19, 2005. University of Hong Kong in 2011 and 2018 respectively. He is currently
[39] Marius Muja and David G Lowe. Fast approximate nearest working as senior research associate in AIMtech Centre (Centre for
neighbors with automatic algorithm configuration. VISAPP (1), Innovative Applications of Internet and Multimedia Technologies) at
2(331-340):2, 2009. City University of Hong Kong. His research interests include computer
[40] Robin L Nabi and Marina Krcmar. Conceptualizing media en- graphics, virtual reality, interactive media, human-computer interface,
joyment as attitude: Implications for mass media effects research. and innovative technologies for education.
Communication theory, 14(4):288–310, 2004.
[41] Alvaro Ortigosa, José M Martı́n, and Rosa M Carro. Sentiment
analysis in facebook and its application to e-learning. Computers in
Human Behavior, 31:527–541, 2014.
[42] Zhigeng Pan, Adrian David Cheok, Hongwei Yang, Jiejie Zhu, and
Jiaoying Shi. Virtual reality and mixed reality for virtual learning Selena Leoni received her bachelor’s degree of arts and science
environments. Computers & Graphics, 30(1):20–28, 2006. in new media from the School of Creative Media, City University of
[43] Nikiforos M Papachristos, Ioannis Vrellis, and Tassos A Mikropou- Hong Kong in 2015. She is currently working as a research assistant
los. A comparison between oculus rift and a low-cost smartphone in AIMtech Centre (Centre for Innovative Applications of Internet and
vr headset: Immersive user experience and learning. In Advanced Multimedia Technologies), City University of Hong Kong on multimedia
Learning Technologies (ICALT), 2017 IEEE 17th International Conference content production. Her specialties include documentary filming, spher-
on, pages 477–481. IEEE, 2017. ical panoramic photography and videography, and production of other
[44] Jan L Plass, Roxana Moreno, and Roland Brünken. Cognitive load visual media contents.
theory. Cambridge University Press, 2010.
[45] Yeshwanth Pulijala, Minhua Ma, and Ashraf Ayoub. Vr surgery:
Interactive virtual reality application for training oral and maxillo-
facial surgeons using oculus rift and leap motion. In Serious Games
and Edutainment Applications, pages 187–202. Springer, 2017.
[46] Kumar Ravi and Vadlamani Ravi. A survey on opinion min- Yangbin Chen is currently a PhD candidate at City University of Hong
ing and sentiment analysis: tasks, approaches and applications. Kong. He received the B.Eng. degree from University of Science and
Knowledge-Based Systems, 89:14–46, 2015. Technology of China, and the MPhil degree from City University of Hong
[47] Jon Scoresby and Brett E Shelton. Visual perspectives within Kong. His main research interests include natural language processing
educational computer games: effects on presence and flow within and deep learning.
virtual immersive learning environments. Instructional Science,
39(3):227–254, 2011.
[48] George Siemens. Connectivism: Learning as network-creation.
ASTD Learning News, 10(1):1–28, 2005.
[49] Burrhus Frederic Skinner. About behaviorism. Vintage, 2011.
[50] B Smus, C Plagemann, and D Coz. Cardboard: Vr for android. In
Ka-Fai Ma received the PhD degree in Sociology from the University
Google I/O Conference, 2014.
of Wisconsin-Madison. He is currently an assistant professor of the
[51] Jonathan Steuer. Defining virtual reality: Dimensions determining
department of Chinese and History in The City University of Hong Kong.
telepresence. Journal of communication, 42(4):73–93, 1992.
His major research areas are in Sociology of Culture, Class Analysis,
[52] John B Watson. Psychology as the behaviorist views it. Psycholog-
and Political Sociology. He is devoted in teaching subjects of Creative
ical review, 20(2):158, 1913.
Writing, Cultural Industry, and Media Analysis. He has published over
20 books of essays, many of them bestsellers. Once Upon A Time is
his debut novel which will be published in various languages including
French, Korean and English. It will also be turned into a film project by
the international well-know director Johnny To.

Calvin Hoi-to Wong is a doctoral candidate in Chinese Studies at the

University of Edinburgh, United Kingdom and currently an instructor in
the Department of Chinese and History at the City University of Hong
Kong teaching ancient Chinese history, urban and women history of
Horace Ho Shing Ip is Chair Professor of Com- China, and modern world history. His current research interests include
puter Science and the founding director of the social and cultural history, publishing history and print culture, history of
AIMtech Centre (Centre for Innovative Applica- China’s foreign relations, early medieval China, and modern China.
tions of Internet and Multimedia Technologies)
PLACE at City University of Hong Kong. He graduated
PHOTO from University College London (UCL), UK, with
HERE a BSc(Hons) and PhD degrees in 1980 and 1983
respectively. His research interests include in-
teractive multimedia, pattern recognition, virtual Qing Li is a Professor at City University of Hong Kong. He received the
environment and e-Learning technologies. He B.Eng. degree from Hunan University, China, and the M.Sc. and Ph.D.
has received regional and international awards degrees from University of Southern California, Los Angeles, USA, all in
for his work including a Gold Medal from the Salon International Des Computer Science. His research interests include database modelling,
Inventions, Geneva; Prix Ars Electronica, Austria; and the Asia-Pacific Web services, multimedia retrieval and management, and e-learning
Information and Communications Technology Award. In 2007, Prof. Ip systems. He has authored over 300 papers in technical journals and in-
was recognized as one of world’s distinguished innovators by the TIME ternational conferences. He is actively involved in the research commu-
magazine. He is a Fellow of the Hong Kong Institution of Engineers nity by serving as a journal reviewer, program committee chair/co-chair,
(HKIE), a Fellow of the Institution of Engineering and Technology (IET) and organizer/co-organizer of several international conferences. Prof. Li
in the United Kingdom, a Fellow of the British Computer Society. Prof. Ip serves as the Chairman of the Hong Kong Web Society, a Councilor of
was awarded in 2004 the prestigious Fellow of International Association the Database Society of China Computer Federation, and a Steering
for Pattern Recognition (IAPR) for contributions to the field of Pattern Committee Member of DASFAA, ICWL, WAIM, and International WISE
Recognition. He has published over 300 papers in international journals Society.
and conference proceedings.

1939-1382 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.