Journal of Research on Technology in Education

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/ujrt20

Comparing immersive VR and non-immersive VR

on social skill acquisition for students in middle
school with ASD

Adam Carreon, Sean J. Smith, Bruce Frey, Amber Rowland & Maggie Mosher

To cite this article: Adam Carreon, Sean J. Smith, Bruce Frey, Amber Rowland & Maggie Mosher
(02 Mar 2023): Comparing immersive VR and non-immersive VR on social skill acquisition for
students in middle school with ASD, Journal of Research on Technology in Education, DOI:
10.1080/15391523.2023.2182851

To link to this article: https://doi.org/10.1080/15391523.2023.2182851

Published online: 02 Mar 2023.

Submit your article to this journal

Article views: 386

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ujrt20
Journal of Research on Technology in Education
https://doi.org/10.1080/15391523.2023.2182851

Comparing immersive VR and non-immersive VR on social

skill acquisition for students in middle school with ASD
Adam Carreona , Sean J. Smithb, Bruce Freyb, Amber Rowlandb and
Maggie Mosherb
a
Georgia Southern University, Statesboro, GA; bUniversity of Kansas, Lawrence, KS

ABSTRACT ARTICLE HISTORY

Virtual reality (VR) has evolved to include non-immersive to fully-immersive Received 20 October
experiences for the classroom. This study seeks to understand the potential 2022
effects VR may offer, specifically characteristics associated with how a stu- Revised 8 February 2023
Accepted 12 February
dent experiences a VR intervention and the effects of using a VR device
2023
for learning. A group experimental design was used to compare a
fully-immersive VR intervention and an identical non-immersive VR inter- KEYWORDS
vention for a group of middle school participants with ASD (N = 22). Virtual reality; social skills;
Participants were randomly assigned a screen-based VR experience or a autism; instructional
head-mounted display VR experience through rolling randomization. Results technology; assistive
indicate that while the more immersive condition did not produce higher technology; mixed reality;
acquisition than the non-immersive condition, both device conditions did special education
produce significant increases in learning.

Introduction
Social emotional learning (SEL) plays a vital role in the education of all learners, particularly
those students with identified disabilities (Schonert-Reichl, 2017). Thus, supporting the development
of social emotional skills is crucial to facilitating empowered, independent learners. SEL often is
referred to as the process through which individuals learn and apply social, emotional, and related
social skills, attitudes, behaviors, and values that direct their thoughts, actions, and feelings to
succeed within the demands of school and greater society (Kendziora & Yoder, 2016). For students
with disabilities (SWDs) (e.g., learning disabilities, autism spectrum disorder), challenges in SEL
are often exhibited through a student’s inability to interpret one’s social environment and the
various social cues, having poor impulse control, misinterpreting appropriate social behavior, and
as a result, the inability to develop meaningful relationships with peers (Kendziora & Yoder, 2016).
While many SWDs exhibited social skill deficits before the pandemic, remote learning and
the constant disruptions to K-12 instruction have exasperated social skill deficits (Yoder et al.,
2020). The pandemic and the need to isolate challenged social interaction thus, further impeded
opportunities to learn, practice, and apply social skills to the educational and community envi-
ronments for all learners (Yoder et al., 2020). As a result, educators realized that countless
students are ill-equipped to exhibit appropriate social skill competencies expected to facilitate
classroom instruction and subsequent student learning. Instead, schools reported challenges in
addressing academic regression while balancing the need to employ specific SEL instruction to
address growing social emotional deficits.
The need for remote learning has underlined the importance of effective digital solutions to
promote student learning, including SEL. Thus, while learning at a distance through various
technology tools may have enhanced social skill deficits, these same tools are increasingly being

CONTACT Adam Carreon acarreon@georgiasouthern.edu

© 2023 ISTE
2 A. CARREON ET AL.

used to personalize SEL (Howard & Gutworth, 2020). Be it game-based simulation, video mod-
eling, robots, or apps and online resources (e.g., Social Express, Floreo, etc.); educators are
seeking alternatives to face-to-face SEL approaches. These sought after approaches need to offer
a safe learning environment, opportunities for explicit instruction, provide ample student practice,
and can scaffold the learning experience allowing students to build upon social skill understand-
ing and subsequent development.

Virtual reality for SEL

Essential to the various digital tools being considered for SEL, virtual reality (VR) is rapidly becom-
ing a popular tool for use in the classroom (Elmqaddem, 2019). VR is defined by Merriam-Webster
(2020) as an artificial environment that can be experienced and interacted with through sensory
stimuli provided by computers and other mobile devices. VR uses a digital simulation, often
three-dimensional (3D) environments, through which a student can physically interact (Oigara,
2018). VR technology has been available for decades but is limited for classroom use due to the
high costs of acquiring hardware and developing VR applications (Salem et al., 2012). However,
the emerging development of hardware and software now allows for cost-efficient and attainable
solutions that can be accessed to support academic, behavioral, and social-emotional development.
For SWDs who can benefit from social skill support, VR applications offer a safe and authentic
digital environment to experience, practice, learn, enhance and replicate skills (Bellani et al., 2011).
As VR has evolved, three primary applications of VR have emerged: (a) non-immersive, (b)
semi-immersive, and (c) fully-immersive (Carreon et al., 2022), each providing different levels
of immersion during use. For example, non-immersive VR (often referred to as desktop/laptop
virtual environments) allows users to view the VR environment on the screen of a digital device
interacting via keyboard, mouse, touchpad, or joystick. In semi-immersive VR, users view the
environment on a digital device or through a simulator (e.g., Flight Simulator). Users often may
be immersed visually and auditorily but are still aware of and/or can see their surrounding
physical environment. Finally, fully-immersive VR utilizes a head mounted display (HMD) or
similar headset and can use a tracking device (e.g., LiDar scanning) to navigate within the
environment. The user interacts with digital three-dimensional objects and has the sensation of
being a full participant in the virtual environment. Users in full immersion are unaware of their
surrounding environment because they cannot see or hear their natural surroundings due to
the HMD and similar fully-immersive equipment.

Immersion and VR interventions

Overall, the use of VR for students with ASD is promising. However, recent reviews recognize
that research has not kept up with the growth of immersion available in VR technology (Carreon
et al., 2022; Mosher et al., 2022; Vasquez et al., 2015). This includes describing the immersion
and understanding its impact on providing successful interventions for students with ASD.
Current research suggests a correlation between higher levels of immersion and the more
significant potential to enhance the learning experience (Radianti et al., 2020). As previously
noted, the level of immersion is determined by removing real-world stimuli and substituting the
digital environment stimuli (Mestre, 2005; Witmer & Singer, 1998). Research has often measured
the effectiveness of the degree of immersion through presence which requires the participant to
imagine the environment as their present location to navigate the virtual world (Herrera et al.,
2008). The degree VR induces presence is based upon both realism of the graphics and immer-
sive technology and how realistic the interactions are within the environment (Slater, 2018).
Miller and Bugnariu (2016) suggest, in their review of VR immersion for students with ASD,
that VR that provides a higher level of presence for participants may be more successful in
delivering social skill interventions. This means that if a VR intervention can be delivered in a
 Journal of Research on Technology in Education 3

more immersive or realistic manner, it may provide further opportunities for learning social
skills that can generalize outside of the VR environment.

Virtual reality social skill interventions for students with ASD

With the advancements in VR, including lower costs and increased classroom availability, educators
are beginning to realize the potential effects VR can have on students with and without disabilities’
instructional, social-emotional, and behavioral outcomes (Oigara, 2018). Current research finds
several studies focusing on VR’s use and impact in the K-12 classroom (Freina & Ott, 2015). For
example, a recent study suggests that students may retain information and better apply what they
learn following a VR intervention (Krokos et al., 2019). Mikropoulos and Natsis (2011) reviewed
53 articles on VR and general education and found that VR has the potential to aid students in
understanding and learning. Additionally, in a general education context, VR interventions have
been implemented to support higher-order thinking skills associated with all academic content
areas (i.e., mathematics, English) (Mikropoulos and Natsis (2011). These interventions are often
used as a curriculum enhancement for students to practice academic skills.
Further examination of the VR literature specific to SWDs finds an overwhelming focus on
social skills (McCoy et al., 2016). While a smaller sample of studies is available than in general
education VR research, a majority focus on students with autism spectrum disorder (ASD). This
is not particularly surprising because the predominant number of VR studies feature students with
ASD, a disability where social skill deficits are a primary attribute (Simpson et al., 2012). Researchers
have used VR for the instruction of students with ASD to successfully increase communication
(Adjorlu & Serafin, 2018; Taryadi & Kurniawan, 2018), improve emotional recognition (Chen et al.,
2015; Ip et al., 2018), and increase social relationships (Cheng et al., 2015; Lee et al., 2018).
While literature reviews have been conducted on the effectiveness of VR in education, few lit-
erature reviews have been conducted on VR and students with ASD, and even fewer literature
reviews have been conducted comparing levels of immersion in the technology. Mosher et al. (2022)
conducted a review of VR technology to present social skills to school-age students with autism.
Their review found 41 studies using VR for students with ASD, and of the 41 studies, 32 studies
reported significant positive social skills outcomes. However, they noted that most studies utilized
a non-immersive screen-based technology. Similarly, Vasquez et al. (2015) completed a literature
review of VR and students with ASD. All 19 articles reviewed targeted social skills and found
positive results for students with ASD using VR to address social skills. Of the studies reviewed,
only five were identified with immersive qualities established by Carreon et al. (2022). After further
review of those five articles, all would be excluded from the definition of a fully-immersive inter-
vention. Only one review found by the researchers compared the immersion by technology and
the current VR literature for SWDs. In this review, they found that 18 of 25 articles targeted social
skills for K-12 students with ASD. Of the 18 studies, 16 reported positive results in gaining social
skills from the VR intervention. Of the 18 identified articles on social skills, fifteen were deemed
non-immersive, according to the criteria set by the researchers (Carreon et al., 2022).

The potential of current VR

VR interventions can enhance traditional methods of social skill instruction in the classroom
for students with ASD. The past few decades have shown promising results of VR in disability
research. Yet, research has not embraced the technological advancements associated with immer-
sive VR. A majority of research still encompasses non-immersive VR that does not fully capitalize
on the capabilities of VR. While the concept of VR having a simulation or digital depiction of
reality has not changed, the ability to immerse a student into the simulation or digital depiction
continues to improve. Therefore, the levels of immersion a user experiences will continue to
improve, advance, and become available to special educators. However, current research is limited
4 A. CARREON ET AL.

in understanding immersion, specifically measuring presence, and how it might enhance how
students with ASD acquire social skills. In their review of VR for students with ASD, Vasquez
et al. (2015) reinforce that VR hardware and software will change, altering the potential to
impact student outcomes. Therefore, this study seeks to understand whether a fully-immersive
VR intervention will further improve student acquisition of skills compared to a non-immersive
VR intervention. The following questions will guide this study:

1. What are the effects of fully-immersive VR and non-immersive VR on the knowledge

acquisition of expressive communication skills?
2. Is there a significant difference in the impact of a Social Skill intervention delivered via
non-immersive or immersive VR devices?
3. Does a student’s perceived level of immersion measured through a presence scale improve
knowledge acquisition of expressive communication skills?

Method
This study was designed to compare the effectiveness of a fully-immersive expressive communi-
cation VR intervention and an identical non-immersive VR expressive communication intervention
for a group of middle school participants with ASD (N = 22). Participants experienced VOISS, an
application developed with 25 expertly written social skills scenarios presented on either a
screen-based application or an Oculus Quest 2 stand-alone VR HMD (Now Meta Quest 2). A
group experimental design consisting of training, pretest, and the intervention phase was conducted.
Further descriptions will be expanded upon in the sections below.

Participants
The participants selected for this study were students with ASD attending middle school in
public or private schools in the Midwestern United States. Participants in the study ranged in
age from 11 to 15 years old, attending 5th through 8th grade across eight different schools. All
participants had to meet the following criteria to participate: (a) have the fine motor skills to
use a touch screen, mouse, trackpad, or haptic controller, (b) be able to sit in a chair or stand
in one place for at least 20 minutes, and (c) be able to identify answers to on-screen questions,
all via teacher recommendation and knowledge. Additionally, all participants had to: (a) qualify
for special education eligibility of autism or have a medical diagnosis of ASD as identified by
an individualized education program (IEP) or individualized service plan (ISP), (b) be aged 11
to 15 years old, and (c) have a demonstrated deficit in expressive communication social skills.
Informed consent was attained from parents prior to intervention. Additionally, verbal assent
was also attained from participants prior to intervention.
A total of 29 students were recruited as participants for this study. Parents of seven students
declined the opportunity for their child to participate before intervention, resulting in 22 total
students for intervention. Rolling recruitment and randomization were used to place participants
into condition groups. Upon receiving parental consent, students were entered into a random
number generator that consisted of one or two. If the generator depicted a one, the participant
was assigned to the non-immersive condition. If the number generator displayed a two, the
participant was assigned to the fully-immersive condition. Table 1 provides details on all included
participants.

Setting
The study was conducted in two settings due to the restrictions of the COVID-19 health pro-
tocols. Twenty-one of the twenty-two students completed the VOISS application in their middle
 Journal of Research on Technology in Education 5

Table 1. Participant characteristics.

Participant Group Age Grade Reported Eligibility
1 Non-immersive 13 7th ASD
2 Non-immersive 11 6th ASD, ADHD
3 Non-immersive 11 5th ASD, ADHD
4 Non-immersive 11 5 th
ASD, ADHD
5 Non-immersive 11 5th ASD
6 Non-immersive 12 6th ASD
7 Non-immersive 12 6th ASD, ADHD
8 Non-immersive 13 7 th
ASD
9 Non-immersive 11 5th ASD, DMDD
10 Non-immersive 14 8th ASD, Anxiety
11 Non-immersive 14 8th ASD
12 Immersive 11 6 th
ASD, ADHD
13 Immersive 12 6th ASD, ADHD
14 Immersive 12 6th ASD
15 Immersive 14 8th ASD
16 Immersive 13 7 th
ASD
17 Immersive 12 6th ASD, CP, ADHD
18 Immersive 11 5th ASD, OCD, ODD, TD, SLD
19 Immersive 13 7th ASD, ADHD
20 Immersive 11 6 th
ASD, ADHD
21 Immersive 11 5th ASD, ADHD
22 Immersive 12 7th ASD, ADHD, Anxiety
Notes: ASD = autism spectrum disorder; ADHD = attention deficit hyperactivity disorder; DMDD = disruptive mood dysregulation
disorder; CP = cerebral palsy; OCD = obsessive compulsive disorder; ODD = oppositional defiance disorder; TD = tic disorder;
SLD = specific learning disorder.

school classroom. One participant remained home due to COVID safety-related considerations
and remotely completed VOISS during the academic day. Due to COVID-19 school policy
restrictions, the first author conducted initial student training, all data collection, and pre-and
post-test data collection via virtual video conference (i.e., Zoom meetings). Student completion
of the VOISS app was completed independently by the student. While classroom teachers ensured
five VOISS scenarios were completed per session, no additional instruction or support was
provided on the part of the classroom teacher.

VOISS intervention
VOISS is a tool designed as a stand-alone interactive computer or HMD based VR simulation
experience designed to enhance the social skills of participants with ASD. VOISS is available
via computer, Chromebook, tablet (e.g., iPad), or HMD (i.e., Oculus/Meta Quest 2). It is import-
ant to note that while VOISS was access by specific devices in this manuscript, VOISS is available
on any device currently that runs Android, Apple iOS, and Oculus/Meta (a version of Android)
operating systems. Users can search VOISS on any application store from the previous list of
operating systems.
In VOISS, participants are presented with social skill scenarios they must interact with,
problem-solve through, and, overall, experience authentic social communication situations. VOISS
features 140 scenarios across ten social skill domains. Researchers of this study assisted in the
design and development of the VOISS application. All scenarios were developed by experts in
the special education field and checked by experts for fidelity and reliability. Further, the sce-
narios were written by teachers in the field of special education across the United States. The
scenarios were then checked for reliability and fidelity by professors from various institutions
of special education across the country in the field of special education and ASD. Updates were
made if needed and finished scenarios were programmed in the technology by an outside firm.
For this investigation, one social skill domain, expressive communication, was utilized.
Expressive communication represents a critical social skill for participants to acquire to be
successful inside and outside school. Overall, the expressive communication domain contains 25
6 A. CARREON ET AL.

scenarios each participant experienced. The 25 scenarios encompass seven skills identified by
the above-stated experts as critical characteristics of appropriate expressive communication
social skills.
Participants navigated the scenarios and locations using a pointing device on a screen, touch-
screen, or a haptic controller via VR HMD. Inside scenarios, participants were presented with
social situations where they had to select a correct multiple-choice response or orally respond to
a request on the screen. Each scenario varied in length and contained various multiple-choice or
oral participation questions. In the event of an incorrect response, a narration explained why the
answer was not socially appropriate and provided direct instruction for the appropriate response.
The participant had the ability to select the correct response following the incorrect response.

Research design
The study design was a group experimental design measured with a repeated-measures analysis
of variance (ANOVA; von Ende, 2001). The study compared the effect of non-immersive VR
(iPad/Chromebook) and fully-immersive VR (Oculus/Meta Quest 2) on the knowledge acquisition
of expressive communication social skills. Prior to intervention, all participants (n = 22) estab-
lished a baseline utilizing the Clinical Assessment of Pragmatics (CAPs; https://www.wpspublish.
com/caps-clinical-assessment-of-pragmatics) assessment as a pretest. Following the baseline, all
participants were trained by navigating a scenario in a different domain (Self-regulation).
Participants were shown what each button does, how to navigate, and how to respond to
in-scenario prompts. Participants were then expected to independently navigate the scenario.
Participants could not enter intervention until independently completing the training scenario.
Following training, participants randomized to treatment group one received the iPad VOISS
intervention, and participants randomized to treatment group two received the Oculus/Meta
VOISS intervention. Following the completion of the intervention for both groups, the CAPS
was also used as the posttest. The CAPs was replicated for the post-test.
Additional data was collected via a presence scale. Participants were given the presence scale
upon completing the randomly assigned intervention and posttest. This presence scale was utilized
to detect the participants feeling of being in the virtual environment. Data from the presence scale
allowed for a greater understanding of how levels of immersion impact the participants.

Measures
The Clinical Assessment of Pragmatics (CAPs; Lavi et al., 2016) was used to measure student
knowledge of expressive communication and social skills. The CAPs is a norm-referenced
video-based assessment designed for use with students aged seven to eighteen. For this study,
the three subtests associated with expressive communication (instrumental performance, affective
expression, and paralinguistic signals) were utilized due to the VOISS intervention providing
participants with instruction for expressive communication (Lavi et al., 2016). The three subtests
used of CAPs provided a series of eight video scenarios each, for a total of 24 scenarios. After
completing all three subtests, a total combined score was utilized to represent knowledge of EC
scores. Responses to the questions were recorded on a score sheet and scored between zero,
one, or two. A zero was scored as not being a correct social response to the situation. A one
was scored if a participant responded adequately but did not provide supporting social interac-
tion. Finally, a two was recorded if the participant responded adequately and supported their
response sufficiently.
A presence scale developed and validated by Witmer and Singer (1998) was used to under-
stand the participant’s sense of presence. The scale uses a seven-point Likert scale for responses.
Witmer and Singer found the PQ with 19 items had high reliability with a Cronbach’s alpha of
.81. Silva et al. (2016) validated the instrument and found an alpha value of 0.89. The presence
 Journal of Research on Technology in Education 7

scale was administered to participants following intervention. Participants answered all 19 ques-
tions utilizing the seven-point Likert scale through a survey online. Data were analyzed using
a total score of all 19 questions.

Reliability and fidelity measurement

An independent graduate student performed interrater reliability checks on the Clinical Assessment
of Pragmatics (CAPs). Sample tests were scored for training and compared until 100% agreement
occurred. The assessment is scored on a zero to two scale. All assessments were conducted via vid-
eoconferencing software and recorded to be scored at a later date. To verify the accuracy of the CAPs
assessment for the participants, the scores from 25% of sessions scored by the researcher were com-
pared to scores obtained by the trained independent graduate student. The sessions were recorded,
and the 24-question assessment was scored. Each session observed was approximately 20 minutes in
length. The obtained agreements and disagreements were calculated using [agreements/(agreements + dis-
agreements) x100 = percent of agreement]. Interrater reliability for CAPs measurement overall was 92%.
This finding indicates an acceptable percent of agreement for CAPs data was obtained.
A fidelity checklist was developed to measure the researcher’s adherence to the training of both
VR conditions. The sessions were recorded, and the checklist was utilized later. The researcher
and independent graduate student viewed the recorded sessions, completed the checklist, and
compared the checklist for discrepancies and agreements. Fidelity to the training in both conditions
was calculated by [(Number of sessions with 100% fidelity)/(total number of sessions) X 100 = per-
cent of fidelity. A total of 6 sessions or 27% of intervention was observed (3 in each condition).
There were five sessions with 100% fidelity and one session where a step was missed, resulting
in 83.3% fidelity. The step did not result in an issue of the student completing training successfully.
The data indicates the treatment was delivered with a high degree of fidelity.
Finally, a fidelity checklist was developed to measure teachers’ adherence to the treatment in
both immersion conditions. The researcher and independent graduate student viewed either live
or recorded videoconference meetings and completed the checklist to measure adherence to the
treatment assigned. Fidelity to the intervention in both conditions was calculated by [(Number
of sessions with 100% fidelity)/(total number of sessions) X 100 = percent of fidelity. A total of
44 sessions, or 20% of intervention, was observed. All 44 sessions were recorded at 100%. The
data indicates the treatment was delivered to a very high degree of fidelity.

Results
Analysis of pretest posttest CAPs change
The participants were administered three subtests of the CAPs (Lavi et al., 2016) to assess
expressive communication social skills. The scores obtained from these assessments were analyzed
to compare the effectiveness of the two interventions on the acquisition of social skills for
middle school students with ASD. Descriptive and inferential statistics were used to compare
the scores on the CAPs. Further, a repeated measures analysis of variance (ANOVA) was con-
ducted to test for significant change across time. Four repeated ANOVAs were conducted to
test for change across time. A multivariate approach was used to test the repeated measures
main effects. The independent variable was which condition the participants were assigned. The
dependent variables were each of the subscale scores and the total score. See Table 2 for means
and standard deviation scores across pre- and posttest data used for analysis.

Instrumental performance subtest

Instrumental performance is a subtest of the CAPs that assesses a participant’s ability to use
appropriate social routine language (Lavi et al., 2016). The repeated measures analysis of variance
8 A. CARREON ET AL.

with instrumental performance as the dependent variable did not find a significant interaction,
Wilks’ Lambda F(1, 20) = .11, p = .75. The main effect was significant [F(1, 20) = 32.152, p <
.001] with a very large effect size, partial eta squared of 0.62. This indicates that the participants
in both groups increased their instrumental performance social skills after receiving the inter-
vention, regardless of the format.

Affective expression performance subtest

Affective expression is a subtest of the CAPs that assesses the participant’s ability to recognize
and express emotions appropriately (Lavi et al., 2016). The repeated measures analysis of variance
with affective expression as the dependent variable found a significant interaction, Wilks’ Lambda
F(1,20) = 8.37, p = .01, though the non-immersive group actually increased more than the
immersive group. The main effect was significant [F(1, 20) = 30.63, p < .001] with a very large
effect size, partial eta squared of 0.61. Participants in both groups increased their affective
expression social skills after receiving the intervention, regardless of the format.

Paralinguistic signals performance subtest

Paralinguistic signals is a subtest of the CAPs that assesses the participant’s ability to recognize
and use nonverbal skills, including facial expression, tone of voice, gestures, and overall body
language (Lavi et al., 2016). The repeated measures analysis of variance with paralinguistic signals
as the dependent variable found an interaction that was nearly significant, Wilks’ Lambda F(1,
20) = 4.149, p = .06, though the non-immersive group actually increased more than the immersive
group. The main effect was significant [F(1, 20) = 68.15, p < .001] with a very large effect size,
partial eta squared of 0.77. As with the previous two analyses, participants in both groups increased
their paralinguistic signals social skills after receiving the intervention, regardless of group.

Total performance
Total performance on the scale is referred to as pragmatic performance (Lavi et al., 2016). The
repeated measures analysis of variance with the total pragmatic performance as the dependent
variable found a significant interaction, Wilks’ Lambda F(1,20) = 10.395, p = .01, though, once
again, the non-immersive group increased more than the immersive group. The main effect was
significant [F(1, 20) = 120.58, p < .001] with a very large effect size, partial eta squared of 0.86.
Participants in both groups increased their total pragmatic performance social skills score after
receiving either intervention format.

Analysis level of presence

Researchers compared the effect of presence in device conditions on social skill acquisition by
running a correlation. There were no significant relationships between the change scores (social
skill learning) and the levels of presence reported. Correlations are shown in Table 3.

Table 2. Summary of means and standard deviations for CAPs data across the two interventions.
NI standard Immersive Immersive standard
Assessment Subtest NI mean deviation NI N mean deviation Immersive N
Pretest Instrumental performance 6.27 3.32 11 8.64 3.72 11
Affective expression 2.09 1.45 11 7.55 5.82 11
Paralinguistic signals 2.55 2.42 11 5.18 3.55 11
Total score 10.91 2.39 11 24.73 4.36 11
Posttest Instrumental performance 9.64 2.91 11 11.64 4.39 11
Affective expression 8.18 2.71 11 9.45 4.76 11
Paralinguistic signals 6.91 3.15 11 7.82 4.31 11
Total Score 21.36 2.92 11 28.91 4.48 11
 Journal of Research on Technology in Education 9

Table 3. Level of presence correlations.

Change score values N Pearson correlation Significance
Instrumental performance 22 .05 0.82
Affective expression 22 −0.40 0.07
Paralinguistic signals 22 −0.13 0.58
Total for pragmatic performance 22 −0.28 0.21
No significant correlations were found between the level of presence and dependent variables.

Discussion
There are three primary findings with this study: (a) VR has a positive impact on social skill
development for SWDs, (b) VR has the ability to facilitate student engagement and knowledge
acquisition across multiple devices, and (c) levels of presence seem to not influence the acqui-
sition of social skills.
First, this study finding on social skill growth and development are consistent with current
VR research for SWDs learning social skills (Yuan & Ip, 2018). Our efforts reinforce that
ever-changing VR environments continues to provide a platform for delivering social skill
instruction. In this study, VOISS provided a platform that individualizes the learning experience
specific to the needs of the student and thus, further personalizes social skills for the teacher
and, of course, the end user, the student. This study adds to the research base that VR is a
viable outlet for students with ASD to safely replicate authentic environments often necessary
for social skill learning (Lorenzo et al., 2016). While our study supports the use of VR to teach
social skills, further research is needed to examine the role of VR in not only increasing knowl-
edge and understanding but also subsequent application of these learned skills in the real-world
environment (e.g., the classroom, school hallways).
Second, this study also documented important differences in the impact levels of immersion
have on student social skill knowledge and understanding. Our study determined that differences
in VR platforms did not lead to significant learning outcomes for students. In contrast, Halabi
et al. (2017) found a greater degree of immersion technology significantly improved performance
for students with ASD. Similarly, findings indicate the student preferred higher-level VR tech-
nology to older, less immersive VR technology.
Our findings contradict previous assumptions that student learning may increase at higher
levels through more immersive VR experiences (Lorenzo et al., 2016). Students using the
iPad or Chromebook, less immersive VR platforms increased their knowledge and under-
standing similar to that of participants instructed through the fully immersive Oculus VR
platform. To control for the potential impact of content on students learning, this study
utilized identical content across the various levels of immersion for the VR intervention.
We found a significant improvement across both device conditions across all three subtests
(i.e., instrumental performance, affective expression, paralinguistic signals) and the total
pragmatic performance score. Furthermore, in comparing the two conditions, the
non-immersive group increased significantly versus the immersive condition over the first
two subtests (i.e., instrumental performance, affective expression) and performed slightly
better on the third subtest (i.e., paralinguistic signals). Results of no significant differences
between the iPad/Chromebook platforms (less immersive) and the Oculus/Meta platform
(fully immersive) indicate that level of immersion may not impact student learning in the
application of VR. In short, the non-immersive condition with VOISS seems to consistently
outperform the immersive condition with VOISS. While the assumption is that the more
immersed with a headset you are, the more you may attain the intended material, this study
indicates that might not be the case. Findings suggest that the interactive features typically
associated with full-immersion (i.e., graphics, sound, engagement) are now being found on
non-immersive devices (i.e., iPads and Chromebooks). This integration could be negating
the impact of presence in the fully immersive experience.
10 A. CARREON ET AL.

In addition, this study offers preliminary evidence suggesting that the type of VR platform,
which facilitates the level of user immersion, may not be a significant factor when the VR instruc-
tional content is consistent across the various immersive learning experiences. Both device conditions
created a similar learning environment resulted in significantly improved knowledge of expressive
communication social skills. With that said, current research (Mosher et al., 2022) does not appear
to recognize the need for consistent instructional content across levels of VR immersion, and/or
the field has yet to factor in consistent content in examining the impact of student immersion
when determining VR effectiveness. Critical next steps for this line of research are to explore further
the extent to which levels of immersion impact student learning and associated outcomes when
consistent instructional experiences are present across all levels of immersion. This would be par-
ticularly relevant in the needed scholarship that extends beyond knowledge and, instead, considers
VR and its impact on application or generalization to the real-world learning environment.
Third, we documented substantial similarities across the two device conditions in participants’
level of presence. The level of presence was an added measure to determine students’ perspective
on their level of immersion when interacting with the VR-based tool. Again, the higher the score,
the higher level of perceived immersion and, thus, the assumption the students would learn through
the engaging experience. The similarities in feelings of presence regardless of the device or level
of immersion (less versus full) may be the primary factor contributing to nearly identical growth
in social skill knowledge and understanding, irrespective of the level of immersion or device used.
Students with ASD indicated that both devices engaged them in the virtual environment and
provided a sense of realism from which to interact and, in return, learn. For students with ASD,
the more immersive technology features (e.g., headsets, graphics) have often impacted and influ-
enced learning differences between the less and more immersive VR experiences (Miller & Bugnariu,
2016). However, the results of this study suggests the non-immersive condition not only compares
to the expensive immersive VR condition but can also outperform the immersive VR condition.
This may indicate ever-changing/improving technology features in non-immersive devices (e.g.,
iPad, Chromebook) may provide the necessary features needed to induce a similar level of pres-
ence. This includes but is not limited to audio immersion, realistic graphics, and interactivity with
the virtual environment. As device features evolve and improve, it is important that additional
research continue to try to understand the impact features, technology elements, and similar
interactive attributes of VR have on the level of immersion and thus, further determine the foun-
dational elements necessary for learning in a virtual environment. We should note, this might
necessitate further development of presence measures designed for students that include items that
measure student attributes relevant to their level of understanding, attributes that enhance engage-
ment, and in turn, further allow researchers to determine a true feeling of immersion.

Implications for educators

The results of this study show that both device conditions significantly increase social skill
development. It appears that the level of immersion does not matter as much as previously
suggested (Halabi et al., 2017). Districts are investing in technology at an exponential rate,
putting iPads and Chromebooks in the classroom. According to Edweek, 90 percent of students
in the United States have access to a Tablet or Chromebook for school (Klein, 2021). With this
increase in access to technology, students and teachers may already have access to technology
that can provide an immersive experience capable of quality social skill instruction. Tablets and
Chromebooks today, by virtue of updated hardware and software, may contain the necessary
features to provide a level of presence comparable with an immersive VR experience.
Overall, social skills are a primary challenge that may be addressed by using VR for students
with ASD (Didehbani et al., 2016). Past research supports using VR to address social skills, yet
research seems to rely on products that are not readily available to classroom teachers. Classroom
teachers need to understand what tools are available in their classroom and how they were
 Journal of Research on Technology in Education 11

created. This study indicates that the type of device may not influence intervention results, but
the features of VR may make the difference. This study utilized a social skill intervention that
was expert-created and expert-reviewed and delivered via two types of virtual environments.
Additionally, it allowed this study to intervene with the same intervention across devices, allowing
the comparison of the device features and not the intervention. The clarity and availability of
the intervention need to continue as technologies become available. Many current devices can
provide similar features necessary for successful social skill intervention for students with ASD.

Limitations
No research study exists without limitations. First, due to school closures and other COVID-19
pandemic barriers, this study had limited participants, making it harder to generalize to a larger
population. Second, the randomization of participants into different conditions was intended to
strengthen the results. Unfortunately, in this study, participants were randomly placed into
immersive and non-immersive groups. The participants in the non-immersive group started at
a significantly lower pretest score, 10.91, versus 24.73. The design of the CAPs provides a limited
number of points possible, 16 on each subtest for a total of 48 possible points, which could lead
to the non-immersive group having more opportunities to grow from intervention. A screener
such as the Social Skills Improvement System Rating Scales (SSIS-RS) delivered before random-
ization would allow the researcher to equate groups. Equating groups would allow for more even
baselines across groups. A final limitation to this study was the overall length of the intervention.
This study was conducted over two weeks and contained five sessions of approximately 20-minute
increments which does not allow for long-term retention and generalization.

Recommendations for future research

Implementing VR in middle school settings may offer a practical, feasible, and authentic way
to reduce barriers to implementing social skill interventions. The implementation of VR can
reduce the time constraints of traditional social skill instruction for teachers. This study rep-
resents an introductory study into how similar interventions are delivered on a wider variety of
readily available devices and the impact on learning for students with ASD.
Immersive technology is an inevitable reality for classrooms, and researchers and teachers
must understand the effect that the content and realism have on various students. Today’s class-
rooms contain a variety of learners with diverse backgrounds. Additional research is needed to
understand how the immersion type and level of presence in a VR intervention affect student
learning. The data of this study suggest that the components of the VR intervention, specifically
the level of immersion and level of presence, are not as important as the specific intervention.
With the explosion of VR in current society and its adoption in the classroom, further research
is needed to understand the effects of improving the immersive values of intervention.
Understanding immersion and presence will allow content creators to understand the critical
components for developing effective software for teachers.
The use of current technology often coincides with the term novelty effect. The novelty effect
refers to a phenomenon where a person has a higher level of motivation when using new tech-
nology, and motivation fades over time (Koch et al., 2018). This study means that students may
have participated and learned over a short period due to the device and the engaging content.
The novelty effect would suggest that, over time, the attention and learning by students in VR
would wane. Similarly, this study reported primarily positive results from the 22 participants over
two weeks. The data of this study aligns with the current research by also showing significant
results over a short amount of time. Future research needs to be conducted over more extended
periods of time, which could include longitudinal studies. Researchers will understand the long-term
effects and understand if there is an initial or novel effect by conducting longer-term studies.
12 A. CARREON ET AL.

Summary
This study sought to identify the effectiveness VR has on social skills acquisition. Overall, VR
seems to present a promising option for SWDs to acquire social skills. Additionally, this study
sought to understand whether levels of immersion measured through presence, made a difference
in levels of acquisition. This study shows that a social skills intervention delivered via varied
levels of immersive VR may provide an intervention that impacts teacher time minimally with
the enormous potential to achieve significant social skill knowledge. Furthermore, this indicates
that teachers and students may already have access to the VR technology as students improved
their knowledge regardless of device utilized.

Notes on contributors
Adam Carreon is an assistant professor of special education in the Department of Elementary and Special
Education at Georgia Southern University. Dr. Carreon’s research centers around innovative technology to support
students with disabilities and special education teachers.
Sean J. Smith is a professor of special education in the Department of Special Education at the University of
Kansas. Dr. Smith’s research centers on innovations and technology solutions to support struggling learners.
Bruce Frey is a professor of special education in the Department of Educational Psychology at the University
of Kansas. Dr. Frey’s research centers on classroom assessment, instrument development, and program
evaluation.
Amber Rowland is an associate research professor with the Center for Research on Learning and the co-director
of the Advanced Learning Technologies Division at the University of Kansas. Dr. Rowland’s research centers on
supporting professional learning at a distance and pursuing methods for tapping into the collective capacity of
educators and students.
Maggie Mosher, Ph.D. ABD, is the Social Skill Content Strategist for Virtual-Reality Opportunities to Implement
Scaffolded Skills (VOISS). She specializes in social-emotional development, innovative technology, multisensory
instructional methods, tiered systems of support, and individualized instruction.

ORCID
Adam Carreon http://orcid.org/0000-0001-9677-8025
Maggie Mosher http://orcid.org/0000-0001-9079-231X

References
Adjorlu, A., & Serafin, S. (2018). Head-mounted display-based virtual reality as a tool to teach money skills to
adolescents diagnosed with autism spectrum disorder. In Interactivity, game creation, design, learning, and
innovation (pp. 450–461). Springer. https://doi.org/10.1007/978-3-030-06134-0_48
Bellani, M., Fornasari, L., Chittaro, L., & Brambilla, P. (2011). Virtual reality in autism: State of the art. Epidemiology
and Psychiatric Sciences, 20(3), 235–238. https://doi.org/10.1017/S2045796011000448
Carreon, A., Smith, S. J., Mosher, M., Rao, K., & Rowland, A. (2022). A review of virtual reality intervention
research for students with disabilities in K-12 settings. Journal of Special Education Technology, 37, 82–99.
https://doi.org/10.1177/0162643420962011
Chen, C. H., Lee, I. J., & Lin, L. Y. (2015). Augmented reality-based self-facial modeling to promote the emo-
tional expression and social skills of adolescents with autism spectrum disorders. Research in Developmental
Disabilities, 36, 396–403. https://doi.org/10.1016/j.ridd.2014.10.015
Cheng, Y., Huang, C. L., & Yang, C. S. (2015). Using a 3D immersive virtual environment system to enhance
social understanding and social skills for children with autism spectrum disorders. Focus on Autism and Other
Developmental Disabilities, 30(4), 222–236. https://doi.org/10.1177/1088357615583473
Didehbani, N., Allen, T., Kandalaft, M., Krawczyk, D., & Chapman, S. (2016). Virtual reality social cognition
training for children with high functioning autism. Computers in Human Behavior, 62, 703–711. https://doi.
org/10.1016/j.chb.2016.04.033
Elmqaddem, N. (2019). Augmented reality and virtual reality in education. Myth or reality? International Journal
of Emerging Technologies in Learning, 14(3), 234–242. https://doi.org/10.3991/ijet.v14i03.9289
 Journal of Research on Technology in Education 13

Freina, L., & Ott, M. (2015). A literature review on immersive virtual reality in education: state of the art and
perspectives. In the International Scientific Conference Elearning and Software for Education, 133, 1–8.
Halabi, O., El-Seoud, S. A., Alja’am, J. M., Alpona, H., Al-Hemadi, M., & Al-Hassan, D. (2017). Design of im-
mersive virtual reality system to improve communication skills in individuals with autism. International Journal
of Emerging Technologies in Learning, 12(5), 50–64. https://doi.org/10.3991/ijet.v12i05.6766
Herrera, G., Alcantud, F., Jordan, R., Blanquer, A., Labajo, G., & De Pablo, C. (2008). Development of symbolic
play through the use of virtual reality tools in children with autistic spectrum disorders: Two case studies.
Autism, 12(2), 143–157. https://doi.org/10.1177/1362361307086657
Howard, M. C., & Gutworth, M. B. (2020). A meta-analysis of virtual reality training programs for social skill
development. Computers & Education, 144, 1–19. https://doi.org/10.1016/j.compedu.2019.103707
Ip, H. H., Wong, S. W., Chan, D. F., Byrne, J., Li, C., Yuan, V. S., Lau, K. S. Y., & Wong, J. Y. (2018). Enhance
emotional and social adaptation skills for children with autism spectrum disorder: A virtual reality enabled
approach. Computers & Education, 117, 1–15. https://doi.org/10.1016/j.compedu.2017.09.010
Kendziora, K., & Yoder, N. (2016). When districts support and integrate social and emotional learning (SEL):
Findings from an ongoing evaluation of districtwide implementation of SEL. Education Policy Center at American
Institutes for Research.
Klein, A. (2021, April). During COVID-19, schools have made a mad dash to 1-to-1 computing. What happens
next? Education Week. https://www.edweek.org/technology/during-covid-19-schools-have-made-a-mad-dash
-to-1-to-1-computing-what-happens-next/2021/04
Krokos, E., Plaisant, C., & Varshney, A. (2019). Virtual memory palaces: Immersion aids recall. Virtual Reality,
23, 1–15. https://doi.org/10.1007/s10055-018-0346-3
Koch, M., von Luck, K., Schwarzer, J., & Draheim, S. (2018). The novelty effect in large display
deployments-Experiences and lessons-learned for evaluating prototypes. In Proceedings of 16th European con-
ference on computer-supported cooperative work-exploratory papers. European Society for Socially Embedded
Technologies (EUSSET).
Lavi, A., Mainess, K. J., & Daher, N. (2016). Clinical Assessment of Pragmatics (CAPs): A validation study of a
video-based test of pragmatic language in adolescent students. Autism Open Access, 6(2), 1–6. https://doi.
org/10.4172/2165-7890.1000172
Lee, I.-J., Chen, C. H., Wang, C. P., & Chung, C. H. (2018). Augmented reality plus concept map technique to
teach children with ASD to use social cues when meeting and greeting. The Asia-Pacific Education Researcher,
27(3), 227–243. https://doi.org/10.1007/s40299-018-0382-5
Lorenzo, G., Lledó, A., Pomares, J., & Roig, R. (2016). Design and application of an immersive virtual reality
system to enhance emotional skills for children with autism spectrum disorders. Computers & Education, 98,
192–205. https://doi.org/10.1016/j.compedu.2016.03.018
McCoy, L., Lewis, J. H., & Dalton, D. (2016). Gamification and multimedia for medical education: A landscape
review. The Journal of the American Osteopathic Association, 116(1), 22–34. https://doi.org/10.7556/jaoa.2016.003
Mestre, D. (2005). Immersion and presence, movement & perception. CNRS & University of the Mediterranean, 2, 1–10.
Mikropoulos, T. A., & Natsis, A. (2011). Educational virtual environments: A ten-year review of empirical research
(1999-2009). Computers & Education, 56(3), 769–780. https://doi.org/10.1016/j.compedu.2010.10.020
Miller, H. L., & Bugnariu, N. L. (2016). Level of immersion in virtual environments impacts the ability to assess
and teach social skills in autism spectrum disorder. Cyberpsychology, Behavior and Social Networking, 19(4),
246–256. https://doi.org/10.1089/cyber.2014.0682
Mosher, M., Carreon, A., Craig, S., & Ruhter, L. (2022). Immersive technology to teach social skills to students
with autism spectrum disorder: A literature review. Review Journal of Autism and Developmental Disorders,
9(3), 334–350. https://doi.org/10.1007/s40489-021-00259-6
Oigara, J. N. (2018). Integrating virtual reality tools into classroom instruction. In Handbook of research on
mobile technology, constructivism, and meaning ful learning (pp. 147–159). IGI Global. https://doi.
org/10.4018/978-1-5225-3949-0.ch008
Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual
reality applications for higher education: Design elements, lessons learned, and research agenda. Computers &
Education, 147, 1–29. https://doi.org/10.1016/j.compedu.2019.103778
Salem, Y., Gropack, S. J., Coffin, D., & Godwin, E. M. (2012). Effectiveness of a low-cost virtual reality system
for children with developmental delay: a preliminary randomized single-blind controlled trial. Physiotherapy,
98(3), 189–195. https://doi.org/10.1016/j.physio.2012.06.003
Schonert-Reichl, K. A. (2017). Social and emotional learning and teachers. The Future of Children, 27, 137–155.
https://doi.org/10.1353/foc.2017.0007
Silva, G. R., Donat, J. C., Rigoli, M. M., de Oliveira, F. R., & Kristensen, C. H. (2016). A questionnaire for
measuring presence in virtual environments: factor analysis of the presence questionnaire and adaptation into
Brazilian Portuguese. Virtual Reality, 20(4), 237–242. https://doi.org/10.1007/s10055-016-0295-7
Simpson, R. L., Ganz, J. B., & Mason, R. (2012). Social skills interventions and programming for learners with
autism spectrum disorders. Educating students with autism spectrum disorders: Research-Based Principles and
Practices. Routledge.
14 A. CARREON ET AL.

Slater, M. (2018). Immersion and the illusion of presence in virtual reality. British Journal of Psychology (London,
England: 1953), 109(3), 431–433. https://doi.org/10.1111/bjop.12305
Taryadi, B. R., & Kurniawan, I. (2018). The improvement of autism spectrum disorders on children communi-
cation ability with pecs method multimedia augmented reality-based. Journal of Physics: Conference Series, 947,
1–7. https://doi.org/10.1088/1742-6596/947/1/012009
Vasquez, E., Nagendran, A., Welch, G. F., Marino, M. T., Hughes, D. E., Koch, A., & Delisio, L. (2015). Virtual
learning environments for students with disabilities: A review and analysis of the empirical literature and two
case studies. Rural Special Education Quarterly, 34(3), 26–32. https://doi.org/10.1177/875687051503400306
Virtual Reality. (2020). Merriam-Webster.com. Retrieved August 23, 2020, from https://www.merriam-webster.
com/dictionary/virtual_reality
von Ende, C. N. (2001). Repeated-measures analysis. Design and Analysis of Ecological Experiments, 8, 134–157.
Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire.
Presence, 7(3), 225–240. https://doi.org/10.1162/105474698565686
Yoder, N., Atwell, M. N., Godek, D., Dusenbury, L., Bridgeland, J. M., & Weissberg, R. (2020). Preparing youth for
the workforce of tomorrow: Cultivating the social and emotional skills employers demand. In SEL for Workforce
Development. Collaborating States Initiative. Collaborative for Academic, Social, and Emotional Learning.
Yuan, S. N. V., & Ip, H. H. S. (2018). Using virtual reality to train emotional and social skills in children with
autism spectrum disorder. London Journal of Primary Care, 10(4), 110–112. https://doi.org/10.1080/17571472.
2018.1483000