The Impact of Immersive Virtual Reality Environments on

Learning Outcomes and Engagement in Online Higher Education:

A Systematic review and Meta Analysis

ABSTRACT
INTRODUCTION

In a constantly changing information society where knowledge organizations and

educational institutions support a knowledge-based economy and the citizens’ well-being,
there is a social need to continuously improve the quality of all modes of learning. The
quality improvements should aim to enhance or at least bring about meaningful reforms to
all levels of formal education, supporting the learners’ styles of learning and their
personalized learning needs. Higher education (HE) is often the most turbulent domain of
externally or/and internally or time-driven changes. Such rapid or slow changes are, for
instance, the current coronavirus pandemic or various socio-economical changes from
financial crises, which most often define the needs for a specific type and mode of learning,
re-shaping pedagogic purposes of state-based or other formal education. For learning
processes to be both effective with desirable results (learning outcomes) and also efficient
regarding the performance of instructors and learners, long- and short-term educational
objectives must be clearly outlined and smoothly streamlined. In return, the learning
procedures must be thoroughly (re-)considered and often re-designed or adapted to
address the newly risen pedagogical, socio-cultural, or socio-economical needs.

Dalgarno, B., & Lee, M. J. (2010). What are the learning affordances of 3‐D virtual
environments?. British journal of educational technology, 41(1), 10-32.

Immersive Virtual Reality (IVR) systems offer several learning affordances, that is,
characteristics that can elicit learning behaviors promoting learning outcomes in educational
contexts. In particular, the immersive and interactive properties of these systems represent a
strong opportunity to support the effective learning experiences.

Di Natale, A. F., Repetto, C., Riva, G., & Villani, D. (2020). Immersive virtual reality in K ‐12
and higher education: A 10‐year systematic review of empirical research. British Journal of
Educational Technology, 51(6), 2006-2033.

VR has the potential to act as an active learning tool and enhance the educational experience in
bioscience, as it encourages active participation and self-directed learning of a student through
high-levels of interactivity with the software. This is important as medical and bioscience
education often requires students to be able to physically interact with objects, such as organs, to
gain a better understanding of their form through self-directed inquiry and exploration (Maresky
et al., 2019). Many programmes designed for HMD VR allow students to interact with the virtual
environment in a variety of ways, one being through the use of handheld controllers which
deliver haptic feedback or technology such as ‘Leap Motion’ which tracks hand and digit
movements. A number of papers have studied interactive VR programmes as educational tools,
exhibiting a myriad of associated benefits, such as improvements in visual understanding, long
term retention and revision of concepts, as well as engagement with and enjoyment of the
learning activity (Maresky et al., 2019; Marsh, Giffin, & Lowrie, 2008; Stepan et al., 2017;
Teranishi & Yamagishi, 2018), as described further in Section 4. However, when students are
constrained by a lack of appropriate, intuitive hand controls, improvements in learning could be
limited (Makransky, Terkildsen, & Mayer, 2019). Moreover, simply having a VR programme in
a course does not generate active learning as the tool can be used in a very passive manner as
well. For example, Stepan and colleagues used a “flyover” video in the first half of their VR
intervention phase which subjects watched through HMD VR (Stepan et al., 2017). Additionally,
it should be noted that VR technology, though no longer nascent, is still young and therefore is
not perfect. Some limitations of varying severity range from motion sickness to an inability for
HMDs to comfortable fit around large religious headwear. Though VR cannot, as yet, fully
replicate the tactile sensation experienced from physically interacting with an object (thus
limiting the sense of presence conveyed), it does allow for the digital simulation to be
manipulated and explored in a similar manner to the real thing, highlighting the interactive
capability of the technology and its potential use as an active learning tool.

Maresky, H. S., Oikonomou, A., Ali, I., Ditkofsky, N., Pakkal, M., & Ballyk, B. (2019). Virtual reality and cardiac
anatomy: Exploring immersive three-dimensional cardiac imaging, a pilot study in undergraduate medical anatomy
education. Clinical Anatomy, 32(2), 238-243.

Stepan, K., Zeiger, J., Hanchuk, S., Del Signore, A., Shrivastava, R., Govindaraj, S., & Iloreta, A. (2017).
Immersive virtual reality as a teaching tool for neuroanatomy. International Forum Of Allergy & Rhinology, 7(10),
1006-1013. doi:10.1002/alr.21986

Marsh, K. R., Giffin, B. F., & Lowrie, D. J., Jr. (2008). Medical Student Retention of Embryonic Development:
Impact of the Dimensions Added by Multimedia Tutorials. Anatomical Sciences Education, 1(6), 252-257
Teranishi, S., & Yamagishi, Y. (2018). Educational Effects of a Virtual Reality Simulation System for Constructing
Self-Built PCs. Journal of Educational Multimedia and Hypermedia, 27(3), 411-423.

Despite the problems of early virtual reality technologies, the rapid increase
in the processing power of the computer led to the deployment of desktop-
based virtual reality technology in K-12 and higher education. The drastic
reduction in the cost of technology and availability of high-speed Internet
connection further increased the use of this less immersive form of virtual
reality technology (Dickey, 2005, McLellan, 2004). Although desktop-
based 3-D virtual environments cannot provide fully immersive experience,
their photo realistic computer graphics been shown to enhance learners'
engagement (Dickey, 2003). Advances in the technology have made it
possible to use low cost peripheral devices such as headphones, shutter
glasses, and data gloves. In addition, with the further advancement of Web
technologies, new possibilities of simultaneously allowing more than one
user in a virtual environment to work collaboratively have also emerged
(Chen and Teh, 2000, Kamel Boulos and Wheeler, 2007).

The assumption underlying the rapid rise in the use of desktop-based

virtual reality technology in instruction is the unique affordances that it
offers in enhancing learners' cognitive skills. Many educators have
integrated a variety of desktop-based virtual reality technologies into their
instruction.

A search of the literature revealed three meta-analyses (Lee, 1999,

Sitzmann, 2011, Vogel et al., 2006) and a systematic review summarizing
qualitative research on 3D virtual worlds (Hew & Cheung, 2010). Lee
conducted a meta-analysis of 19 studies and found a positive impact of
using simulation on learning outcomes but a negative impact on students'
attitudes towards using this form of technology for learning. The major
focus of Lee's meta-analysis was on moderator variables such as mode of
simulation (presentation or practice), presence of expository instructional
features, and guided versus unguided simulations. According to the results
of this meta-analysis, simulations are effective for both presentation and
practice if used in conjunction with other methods of instruction. Lee also
found that students' performed better when some form of guidance was
provided even in practice mode compared to those where there was no
guidance provided.
More recently, Sitzmann (2011) and Vogel et al. (2006) conducted meta-
analyses in which they analyzed the effects of interactive computer-based
games and simulations and found statistically significant positive impacts
on learning outcomes. Vogel et al. studied the moderation effects of gender,
learner control, age, realism, and learner collaboration on learning
outcomes. According to their report, students performed better when they
were in control of their navigation through the virtual learning
environment compared to when the teacher controlled the learning
environment. In addition, students in the traditional group outperformed
the students in the virtual learning environment when, the sequence of
learning activities was controlled by the computer programs compared to
when students could select the sequence.

Researchers have explored the benefits and applications of virtual reality

(VR) in different scenarios. VR possesses much potential and its application
in education has seen much research interest lately. However, little
systematic work currently exists on how researchers have applied
immersive VR for higher education purposes that considers the usage of
both high-end and budget head-mounted displays (HMDs). Hence, we
propose using systematic mapping to identify design elements of existing
research dedicated to the application of VR in higher education.
Materials & Methods
The recent meta analysis have been conducted in accordance to guidelines of preffered reporting
items for systematic review and meta analyses (PRISMA) statement [39]. The PRISMA
guidelines decided the inclusion criteria for selection of research articles and search strategy for
recent meta analysis.

2.1 Search strategy & Study Selection

The research papers related to “association among use of mouthwash and risk of hypertension
incidence” have been searched through databases such as PubMed, EMBASE, MEDLINE, and
Cochrane library. The data was searched by using MesH keywords such as “hypertension”,
“mouthwash use”, “risk of HTN”, “HTN incidence” “nitric oxide bioavailibilty” and “cohort
studies”. Only those research articles were extracted which were based on cohort and pilot
studies of HTN population through above mentioned electronic databases. We extracted the
studies published upto August 2023 for authentic data selection.

2.2 Data Extraction

The information collected from research articles include authors, study sample, year of
publication, study aims, type of methodology, primary outcomes, and findings. We designed the
PICO model for above mentioned research aims of recent study to search and select the articles.
The PICO question comprised four parts including population (P), intervention (I), comparison
(C) among experimental and control groups and outcome (O) [40].
The PICO question of recent systematic review and meta analysis have been designed as follows:
P—adult population (40 to 60 years)
I—Use of mouthwash (risk factor)
C—mouthwash users versus non mouthwash users
O—Outcomes (hypertension incidence)

2.3 Inclusion Criteria

The predefined inclusion criteria helped in screening of research articles after search and
extraction [41]. We included only those studies in recent meta analysis which met following
criteria:
 1. The studies involving adult population with mild hypertension and used medication for
mnanagement once in their lifetime.
2. The cohort, epidemiological, pilot and short clinical trail based studies.
3. Only those studies included in which only use of mouthwash was discussed as risk of
hypertension rather than toothpaste.
4. The outcomes of included studies would be hypertension and linked diseases such as
cardiovascular issues as well as stroke.
5. All those trials were published in English from 2010 till 2023.

2.4 Outcomes assessed

Our primary outcome was the short term risk of hypertension in result of mouth rinsing with
flouride containing mouthwash among adults for once or twice daily. Among included cohort
studies, most of adult population have already hypertension symptoms. In this meta analysis, we
found the short term effect of mouthwash use on risk of hypertension incidence and management
by medication among adult population.

2.4.1 Defination of Hypertension

Hypertension is defined as condition in which systolic blood pressure prevails ≥140 mmHg and
diastolic blood pressure ≥90 mmHg and patients have been using medication to control or
manage this issue [42]. But reports of blood pressure readings and other case descriptions, such
as self-reported high blood pressure and other criteria (high normal/pre-hypertension), were also
recorded.

2.5 Quality Assessment

Two reveiwers evaluated the quality of included studies independently by using the risk bias tool
of Cochrane Collaboration of RevMan software version 5.3.0. In case of disagreement, the third
reviewer would be involved for discussion and resolution.
2.6 Statistical Analysis
The RevMan 5.3 software was used for statistical analysis of data from included studies in recent
meta analysis. [43] The p-value < 0.05 was considered significant statistically while presenting
results as odds ratio (ORs) with 95% confidence interval (CI). Moreover, the heterogeneity was
measured by using the the Q test and I 2 statistics. A fixed-effects model and a random-effects
model were applied in case of no significant difference detected from heterogenity test.

Materials & Methods

2.1 Search strategy & Study Selection
The recent systematic review and meta analysis have been conducted in accordance with
PRISMA guidelines that decided the search strategy and inclusion criteria for selection of
research articles. The databases such as JSTOR, ProQuest, ERIC and Researchgate for data
research and extraction up to October 2023. For data extraction, we used MesH keywords
including “Community of inquiry OR CoI”, “Blended course AND Blended learning”, “English
as foriegn language OR EFL”, “Integration of CoI with blended course”, “English speaking
skills”, “online classes AND offline classes” and “outcome of CoI in blended course”. Only
those research articles extracted which were based on studies conducted among English course
students and different degree programs in Universities and data research were conducted by two
authors independently through electronic databases. For authentic data selection, we extracted
the studies published from 2013 to october 2023.
The extracted data from papers include the authors, year of publication, country or area of study,
study sample; number of individuals in experimental as well as control group, type of model,
study aims, course duration and findings.
2.2 Inclusion Criteria
The objectives of systematic review kept in accordance while defining eligibility criteria. After
data search and extraction, we screened the data according to predefined inclusion criteria or
eligibility criteria. In recent systematic review, we included only those studies that met the
following criteria:

1. The studies involve students who attending the EFL course or university under graduate
degree in which they were opting english skills course.
 2. The mixed method study, cross sectional studies and narrative studies were extracted
rather than already done reviews or meta analysis
3. Only those studies included which have experimental group of students learning english
speaking or EFL course integrated with CoI versus control group with traditional EFL.
4. The age of study participants ranged from 16 to 30 years old whose primary language
was not english.
5. The outcomes of included studies would include improvements in speaking english,
effective cognitive performances and enhanced test scores.
6. All those studies were published in English from 2013 till 2023.

2.3 Definition of terminologies

2.3.3 EFL Blended course

EFL or English as foreign language is defined as using and learning english as an additional
language while primary language is non english. In other words, ESL refers to learning of
english language as second language for communication purpose in non-native country. Instead
than considering the online element as an addition to or duplication of what is being covered in
the learning environment, a blended course integrates both offline and online tasks such that they
increase, develop on, and support one another. EFL blended course refers to integration of online
techniques to learn english speaking skills and it may include video audio calls, conferences and
meetup through online platforms (Rahim, 2019).

2.3.4 Community of inquiry (CoI)

The Community of Inquiry (CoI) framework acknowledges the significance of relationships

among the teaching, social, and cognitive the existence in generating a supportive and
cooperative learning environment. The Community of Inquiry theoretical framework describes
an approach for developing three interconnected elements; social, cognitive, and teaching
presence in order to produce a profoundly important (collaborative-constructivist) educational
experience (Shea et al., 2012).
Fig no. 1: Elements of Community of Inquiry (CoI) framework

2.4 Outcomes of study

Primary outcomes of study were the grammatical knowledge, english speaking, cognitive
performances, enhanced accountability, improved adaptability, increased engagement in class
and better english speaking scores.
2.5 Quality Assessment

All included articles were

independently assessed for
methodological quality by the two
reviewers. We used
adapted Mixed Methods Appraisal
Tool (MMAT) for
quality assessment [15]. This tool was
judged suitable to
be used in mixed method reviews,
being with substantial
validity and reliability. The MMAT
used in our study con-
tained three sets of criteria: 1) a
“qualitative” set for quali-
tative studies, and qualitative
components of mixed
methods research; 2) an “observational
descriptive” set for
observational descriptive quantitative
studies, and obser-
vational descriptive components of
mixed methods re-
search; and 3) a “mixed methods” set
for mixed methods
research studies. Each study type was
judged within its
methodological domain. For each
criterion, the presence/
absence was scored 1 and 0,
respectively [16]. Discrepan-
cies in quality assessment between
reviewers related pri-
marily to how findings related to
researchers’ influence
and sampling strategy of quantitative
data, and were set-
tled through consensus. A quality score
for each article
was then calculated through dividing
the total points
scored by the total points possible.
Each article was classi-
fied as weak (≤0.50), moderate-weak
(0.51 to 0.65),
moderate-strong (0.66 to 0.79), or
strong (≥0.80) in terms
of study quality [17]
All included articles were
independently assessed for
methodological quality by the two
reviewers. We used
adapted Mixed Methods Appraisal
Tool (MMAT) for
quality assessment [15]. This tool was
judged suitable to
be used in mixed method reviews,
being with substantial
validity and reliability. The MMAT
used in our study con-
tained three sets of criteria: 1) a
“qualitative” set for quali-
tative studies, and qualitative
components of mixed
methods research; 2) an “observational
descriptive” set for
observational descriptive quantitative
studies, and obser-
vational descriptive components of
mixed methods re-
search; and 3) a “mixed methods” set
for mixed methods
research studies. Each study type was
judged within its
methodological domain. For each
criterion, the presence/
absence was scored 1 and 0,
respectively [16]. Discrepan-
cies in quality assessment between
reviewers related pri-
marily to how findings related to
researchers’ influence
and sampling strategy of quantitative
data, and were set-
tled through consensus. A quality score
for each article
was then calculated through dividing
the total points
scored by the total points possible.
Each article was classi-
fied as weak (≤0.50), moderate-weak
(0.51 to 0.65),
moderate-strong (0.66 to 0.79), or
strong (≥0.80) in terms
of study quality [17]
Two reviewers assessed independently the methodological quality of included studies by using
Mixed Method Appraisal tool (MMAT). This tool is used for quality assessment of studies
discussing qualitative, descriptive and mixed methods for data analysis. It assess the sustainable
validity and reliability of included studies, used for mixed method reviews quality assessment
(Pace, 2012). The recent tool used in our study was a 11 point checklist with three sets of criteria
a). qualitative component of qualitative studies and qualitative components related standards
b). Qualitative description related criteria of qualitative descriptive studies and
c). mixed methods criteria set for studies with mixed method analysis.
A quality score of each study was calculated which recognized the quality of that specific paper
for instance: total score categorizes the studies into three categories as weak with less than 5
scores, moderate with score 5 and strong with score more than 5.

All included articles were

independently assessed for
methodological quality by the two
reviewers. We used
adapted Mixed Methods Appraisal
Tool (MMAT) for
quality assessment [15]. This tool was
judged suitable to
be used in mixed method reviews,
being with substantial
validity and reliability. The MMAT
used in our study con-
tained three sets of criteria: 1) a
“qualitative” set for quali-
tative studies, and qualitative
components of mixed
methods research; 2) an “observational
descriptive” set for
observational descriptive quantitative
studies, and obser-
vational descriptive components of
mixed methods re-
search; and 3) a “mixed methods” set
for mixed methods
research studies. Each study type was
judged within its
methodological domain. For each
criterion, the presence/
absence was scored 1 and 0,
respectively [16]. Discrepan-
cies in quality assessment between
reviewers related pri-
marily to how findings related to
researchers’ influence
and sampling strategy of quantitative
data, and were set-
tled through consensus. A quality score
for each article
was then calculated through dividing
the total points
scored by the total points possible.
Each article was classi-
fied as weak (≤0.50), moderate-weak
(0.51 to 0.65),
moderate-strong (0.66 to 0.79), or
strong (≥0.80) in terms
of study quality [17]
Results

3.1 Included Studies

In recent systematic review, the selection and screening of research articles related to study aims
“integration of community of inquiry (CoI) with blended learning to enhance EFL speaking
skills” was conducted according to PRISMA guidelines. Approximately, 345 research articles
found from above mentioned electronic databases. Only 100 research articles found eligible for
screening by inclusion criteria. Out of those, 8 studies met the inclusion criteria and analyzed for
final results, as mentioned in Fig no.2.
 3.2 Risk Assessment
Author, Year Qualitative Qualitative descriptive Mixed Methods

Sources Analysis Consideratio Considerations Sampling Sample Measureme Accept Mixed Integration of Consideration
of process ns of findings of findings strategy representativ nt able method results relevant of limitations
Total scores
relevant relevant relevant to relate to relevant to eness appropriate response research of objective with
to to context researchers objective rate design integration
objectives objective influence relevant to
s objective
Wang (2021) 1 1 1 0 1 1 1 1 1 0 0 8/11

Duha et al., (2022) N/A N/A N/A N/A 1 1 0 1 1 0 0 4/11

Xiaoxing & Deris, 1 1 1 0 1 1 1 1 1 0 0 8/11

(2022)

Mo & Lee (2017) 1 1 1 0 1 1 1 1 1 0 0 8/11

Cheng (2022) N/A N/A N/A N/A 1 0 1 1 1 0 0 4/11

Chen (2022) 1 1 1 0 1 1 1 1 1 0 0 8/11

Purwandari et al., 1 1 1 0 1 0 1 1 N/A N/A N/A 6/11

(2022)

Wahyuningsih, & 1 1 1 0 1 1 1 1 N/A N/A N/A 7/11

Afandi, (2022)

N/A means

The MMAT tool recognized the three main categories of quality assessment of 8 included studies. Among those, 6 studies (Wang,
2021; Xiaoxing & Deris, 2022; Mo & Lee 2017; Chen, 2022 Purwandari et al., 2022 & Wahyuningsih, & Afandi, 2022) were strong,
2 studies (Cheng, 2022 & Duha 2022) were weak. No study found to be moderate. The overall scores of included studies reflect that
we have selected stronger studies for recent systematic review, as shown in Table no. 1.

Table no.1: Quality assessment of the included studies using Mixed Methods Appraisal Tool (MMAT)

3.2 Study characteristics

About 806 EFL students from different universities and institutions were analyzed to evaluate efficacy of CoI integrated blended
learning to enhance EFL speaking skills from different countries. These non native english countries include Taiwan, China,
Indonesia, Honk Kong, Bangladesh and Korea. The mixed method studies, descriptive analysis, and qualitative studies were included
in recent systematic review. The age of all participants ranges from 16-30 years whose primary language was non english.
 Table no. 2: Characteristics of included studies

Author name Area Type of Sample size Type of sampling Type of Study Aim Findings
publication famework

Wang (2021) Taiwan Jounal Article 136 participants Pre-experimental ICT & Blended To investigate the ways of There is positive effect of
method leaning improving communicative blended english speaking
(Hitutor) peformance and feedback during course among non english
online and offline english native speakers and enhance
teaching among instructors and the EFL outcomes through
learners. lectures and ICT.

Duha et al., (2022) Bangladesh Journal article 50 participants Convergent parallel Community of To assess the face book use as an The use of face book
(control =28, Mixed method inquiry way of improving English improved English speaking
experimental group = sampling informed language teaching among scores among experimental
22) facebook undergraduate students group’s participants.
discussions

Xiaoxing & Deris, (2022) China Journal Article 97 under-graduate Mixed method study Community of To examine the ways of Student engagement in
students inquiry model improving learning outcomes in course improved by CoI
(CoI) form of active participations and model; as grading, teaching
effective teaching practices in practices and feedback
online class of blended EFL positively enhanced in EFL
course. blended course
Mo & Lee (2017) Korea Journal article 124 students Co relational Analysis Community of To figure out how students The L2 online learning
Inquiry (CoI) evaluate an L2 online community enhances the
framework educational community for learning of languages and
English speaking through self-confidence in L2 use
community of inquiry CoI model according to the student's
impressions of it.
Cheng (2022) Hong Kong Journal article 143 students Mixed method study Community of To investigate the efficacy of The BOYD integrated CoI
Inquiry (CoI) Community of Inquiry (CoI) framework is promoted the
framework framework for implementation EFL learning in blended
of EFL blended course. english course.

Chen (2022) Taiwan Journal Article 67 under-graduate Qualitative Study Community of to investigate the integration of The three variables of
freshman English Inquiry (CoI) online learning groups and community of inquiry,
framework intentional activity in a blended reported knowledge, and
learning context, to enhance the educational engagement are
cognitive effectiveness and statistically significantly
participation of English as a correlated.
foreign language (EFL) students.
Purwandari et al., (2022) Indonesia Journal article 187 students Qualitative study E learning To evaluate the learning Students showed positive
Community of experiences of engineering feedback towards e learning
Inquiry (CoI) students by using CoI framework CoI model for language
framework for langauge learning. learning

Wahyuningsih, & Indonesia Journal article 2 lecterures Narrative study E learning to investigate how the two English professors must
Afandi, (2022) Community of English language lecturers used become more professional in
Inquiry (CoI) blended learning to incorporate their use of blended learning,
framework English skills throughout the which can be accomplished
COVID-19 epidemic. by participating in training
on the topic and offering
pertinent English speaking
skills.