Cognitive Overload in Mixed-Reality

Interactions: A Qualitative Analysis

Sachin Palihawadana

Master’s Thesis Project

Master’s Programme in Cognitive Science
Spring 2023
Supervisors: Juan Carlos Nieves, Andreas Brännström
 2

Abstract

Mixed reality (MR) technology is trending in various sectors at the present time. In MR
applications, due to the interactions between two realities, the potential of experiencing
Cognitive Overload is high. Hence, the present qualitative study has been conducted to broaden
the understanding of Cognitive Load in MR interventions. Six participants were recruited for
the study who were University students. The study was conducted with an activity, based on a
car chasing scenario in police interventions. The participants were instructed to complete a
questionnaire after the activity, which was designed based on Cognitive Load Theory (CLT),
and thereafter semi-structured interviews were conducted. The data were analysed through the
Thematic Analysis method (TA), which resulted in generating three themes; Prevention of
Intrinsic Load, Prevention of Extraneous Load, and Increase of Germane Load. Each theme
was divided into categories, and the codes were generated and analysed under the categories.
Our findings elaborate on several elements that influence cognitive load in MR interactions.
The major elements of our findings include; Expertise in operating HoloLens and MR,
Prevention of Split-attention, Prioritizing and Grouping of information, Vigilance, Design of
HoloLens and MR, and Comprehensibility. The study findings can be used in designing MR
applications, especially for the use case of police interventions. Yet, we believe that further
studies are needed to broaden the understanding of causes and prevention methods of Cognitive
overload in MR interactions.

Keywords: Mixed Reality Interaction, Cognitive Load Theory, Context

Understanding
 3

I. Background
The past few years have been a transformative time period of technology which occurred with
the emergence of technological advancements such as Virtual Reality (VR), Augmented
Reality (AR), and Mixed Reality (MR). Mixed Reality is generated with both actual reality and
digital/virtual elements. Unlike AR and VR, MR creates a realistic context where the user is
able to interact simultaneously with both the real-world elements in their physical surrounding
and the digital elements in the virtual context. Therefore, MR has been recognized and utilized
highly as a novel and effective trend for major industry sectors such as healthcare (Gregory et
al., 2018), engineering (Mesaros et al., 2016; Zhaotong, et al., 2017), military defense (Guzman
et al., 2022), education (Smith et al., 2021; Touel et al., 2020), architecture (Schnabel & Wang,
2009), and entertainment (Gochfeld et al., 2018; Prattico & Lamberti, 2021), which has many
useful applications in supporting to improve the efficiency and creativity of the performance.
For instance, Mixed reality has been even used for surgical visualization as well, in order to
assist with plastic and reconstructive surgery (Gregory et al., 2018; Tepper et al., 2017). Hence,
MR could also be a useful tool to the first safety providers such as police, firefighter, and
emergency medical staff, to enhance their efficiency in performing the tasks. Consequently,
MR-based applications are rapidly growing all over the different sectors, especially with the
use of head-mounted displays (HMD) (Park et al., 2021; Rokhsaritalemi et al., 2020).
A head-mounted display (HMD) is a type of device that is worn on the head and it typically
consists of a small screen or screens placed in front of the eyes, along with other components
such as speakers and sensors. HMDs are built in various forms such as helmets, goggles, and
glasses (Rokhsaritalemi et al., 2020). Microsoft HoloLens is a type of HMD that allows users
to see and interact with digital content in a MR environment. But unlike most of other HMDs,
it does not require to be connected to a computer to function, rather all the systems are in-built
(Park et al., 2021). Hence, Microsoft HoloLens is often used to facilitate MR functions.

Mixed reality holds the potential to enhance users’ knowledge and awareness in a situation by
supporting users to access digital information. But some information might be irrelevant or
more complex to process while performing the task. Also, receiving information from both
physical and digital contexts would make the user overwhelmed with a great level of
information (Kockord & Bodensiek, 2021). This occurs due to the fact that our working
memory has a limited capacity to capture and retain information in a brief period of time
(Constantinidis & Klingberg, 2016). The working memory mainly works as an information
manipulation system that filters the information received by the sensory inputs of the individual
through attentional focus (Oberauer & Hein, 2012). Because of its limited capacity to hold the
information, this mental workspace tends to experience an overload when the MR user receives
a larger amount of information through the digital reality while they also receive sensory
information and perform in the physical world simultaneously. This results in MR users
experiencing mental exhaustion, which is called, “Cognitive Overload”, where the working
memory exceeds its capacity of retaining information. This inhibits user’s potential of
performing their best at a particular task (Kirsh, 2000). In order for users to perform well in a
task with the support of MR, the causes of cognitive overload need to be managed and the
technological changes need to be facilitated to emit or decrease the cognitive load.

Therefore, the present study was conducted to understand the elements that cause a high
cognitive load in the users when using MR to complete a task. The elements the study
discovered have general applicability to any setup of MR application that would help to
understand what factors could cause the cognitive overload and how those causes could be
 4

managed or prevented to reduce the cognitive load of users when interacting with MR in a
particular task. However, the study has been more focused on integrating the factors related to
the usefulness of MR interactions in police interventions. Police officers are considered as one
of the most important first safety providers who need the support of additional information in
their day-to-day tasks in order to ensure public safety (Verhage, 2018). Currently, police
officers use Walkie Talkies, bodycam, and smartphones to receive information during
interventions (Ilori et al., 2016). Though these tools are efficient, the scope of the information
they could receive is limited (Ilori et al., 2016). Hence, MR would be an efficient and
convenient tool the officers could utilize when acquiring the most prominent and relevant
information they would require in the intervention context, especially when the decision-
making becomes more challenging and stressful (Verhage, 2018). Since the support and
information required vary from one intervention to another, this sort of information could not
be predetermined and kept in a display. Therefore, it would require an interactive information-
acquiring process that would function efficiently to enhance their performance during the
intervention. Hence, our study could be helpful when MR technology would be used in police
interventions. However, as more learning would need to make a use case of MR for police
officers, this study only attempts to support the knowledge needed to avoid the cognitive
overload of the officers when interacting with MR applications during interventions.

Problem Statement
Even though there are a lot of studies have been conducted to explore the applications of MR
for various tasks, less attention has been given to performance factors at a human cognitive
level. To be efficient in using these technologies, a certain level of cognitive effort is needed
(Thees et al., 2020). In order to gain the maximum potential of using MR in completing a task,
the cognitive process of the individual should function efficiently. Therefore, studies are
required to explore the causes and the prevention methods of generating cognitive overload
which hinders the performance of the user.
In this study we aim to learn what elements would cause cognitive overload in mixed-reality
interactions, and how they can be prevented, particularly focusing on a car-chasing scenario
inspired by police interventions. The study is conducted based on Cognitive Load Theory
(CLT), which contributes a foundation to generate our results expressively in relation to the
current literature. We use CLT in our study because it helps us to understand how the learning
process generally take place, and the three components of CLT help to comprehend cognitive
load in individuals and how it influences their performance with a clear logical process
(Merrienboer & Sweller, 2005; Paas et al., 2004).

Cognitive Load Theory (CLT)

Cognitive Load Theory (CLT) was introduced by Sweller (1988) based on the Hitch and
Baddeley's model of working memory (1976). Initially, it was developed as an instructional
design tool in the field of Education. Later it was adapted for several other fields. CLT has been
developed mainly based on the integration between working memory and long-term memory.
The information is stored in the long-term memory in a specific form called, “Schema” (Gobet
et al., 2015). CLT states that the capacity of working memory is very limited, thus, the
information acquired is proceed to LTM and is stored as a single schema while connecting to
the previously learned schemas. When the working memory is overloaded, the learning does
not process effectively. Hence, Cognitive Overload decreases the performance of the individual
(Merrienboer & Sweller, 2005). Therefore, in order to increase performance, the cognitive load
should be prevented. Thus, CLT explains the cognitive processes behind the occurrence of
 5

cognitive load, in order to explore the prevention methods. CLT theory consists of three factors
of Cognitive Load: Intrinsic cognitive load, Extraneous cognitive load, and Germane cognitive
load.

The Intrinsic cognitive load (ICL) is developed by the complexity or the difficulty of the task
and the efficiency of the working memory to be focused and acquire the information related to
the task (Clark et al., 2006; Sweller et al., 1998). It is mainly determined by the expertise of
the learner, the complexity of the task, and the instructional methods in the learning
environment (Clark et al., 2006).
The extraneous cognitive load (ECL) refers to the irrelevant or unnecessary elements;
information or processes, that are not related to the task goal, which would interfere with
learning. ECL does not support schema acquisition and automation in long-term memory.
Hence, it does not enhance the learning related to the task, thus, it needs to be maintained at a
low level possible to reduce the Cognitive Load. For instance, when using MR to complete a
task, the visual information appears sometimes can be interesting, but it might be irrelevant to
the task (Frederiksen et al., 2020; Parong & Mayer, 2018). Therefore, it might cause a
distraction and will split the learner’s attention on unnecessary information. The main purpose
of reducing ECL is to provide more memory space in the limited capacity of working memory
to gain information only related to the task, and develop a better integration between working
memory and long-term memory, generating a maximum potential to have a smooth process for
germane cognitive function (Chen, Grierson, & Norman, 2015; Sweller et al., 1998). Therefore,
all the irrelevant cognitive processes which interfere with learning; schema acquisition, and
automation in long-term memory, need to be eliminated (Sweller, et al., 1998).
The Germane cognitive load (GCL) refers to the cognitive effort, which is required for learning
by schema construction; linking the newly received information with the existing information
stored in long-term memory (Sweller et al., 1998). Unlike the intrinsic and extraneous
cognitive loads, germane load positively affects the schema construction, which means the
more germane load is generated, the more cognitive effort would be placed on learning, thus,
it enhances efficiency and performance (Chandler & Sweller, 1991; Seufert, 2018).

In order to optimize the process of germane cognitive functioning, ECL has to be reduced and
provide the maximum potential for working memory to function without exceeding its
resources, while the ICL needs to be reduced by the smooth process of integration of the
information in the working memory (Ayres, 2006). With the smooth function of the germane
cognitive process, more cognitive effort and well-integrated information would support for
individual’s problem-solving ability, thus, it would increase performance (Paas et al., 2004).

Application for Police Interventions

VR applications have been already used for police training which could provide a realistic and
safe training experience for the officers in several scenarios (Zechner et al., 2023). The AR
technology has been recommended to use in police investigations which could support officers
to have on-site access to relevant data and information during crime scene investigations (Haque
& Saleem, 2020). Studies also express the development of an "Augmented reality based
criminal investigation system (ARCRIME)" which would create a 3D model of the crime scene,
helping the investigating officers to perceive the situation from different angles or perspectives,
while they could get access to the data and information related to the crime scene such as
fingerprints, evidence, and suspect profiles (Haque & Saleem, 2020). But when it comes to real-
time police interventions, the technological applications need to function on-site while officers
keep a high level of attention on the situation. In these interventions, the applications are needed
 6

to enhance situational awareness, which would help the officers to be more aware of the data
and information about the specific situation. Thus, MR-based applications are used to support
police officers in order to manage and control the situation in a better and safer manner resulting
in lesser conflicts and damage.

Moreover, according to Criminal Psychology, behavioral cues; which are commonly known as
non-verbal body language, are used to perceive certain information about the individual or
suspect, which broaden the awareness of the first responders. Studies show that the recognition
of behavioral cues of criminals or individuals who have intentions to do crimes could support
police officers to early detect criminal intentions and have a proactive approach to prevent the
crimes (Koller et al., 2015). But on the negative side, the officers could be biased to their
stereotypes depending on their personal beliefs, misinformation, and their prior experiences in
work (Bogaard et al., 2016). When using MR-based AI, the officers could minimize their biases
and false alarms in recognizing the cues of the suspects since the AI would provide only relevant
and accurate data. The lie detection systems make evidence for the capability of machines or
AI systems to accurately detect the behavioral cues of humans through behavioral variables
such as signs of facial expressions, eye movements, measured time to respond, and even
detection of personality traits (Gonzalez-Billandon et al., 2019). Thus, the support of AI in
detecting behavioral cues could help police officers to act efficiently during the interventions.

Often, the officers need to make quick decisions and act hastily during police interventions
(Verhage, 2018). In these moments, receiving more information and support from AI is highly
important to well recognize the surrounding and to get more details about the suspect, so that
the officers could make more efficient and accurate decisions. However, at the same time, the
officers need to have undivided attention and be on alert about their surrounding when they are
in the middle of an intervention. Therefore, using HoloLens could also become a distractor for
police officers (Borglund, & Hansson, 2022). The MR-based AI system attached to the
HoloLens is helpful for providing an immediate unbiased perspective during the intervention
period where the officers need to make decisions within a very short-time period. Thus,
biological sensory information and the information gained through the AI are equally important
to create a better situational awareness which helps them make quick decisions and act
immediately. Hence, the information gained through AI should not distract or overwhelm the
police officer and it should ensure that the police officer would not lose being on alert in the
physical surrounding. However, only a very few research studies have been conducted on the
applications of MR for first responders (Borglund, & Hansson, 2022). Hence, the findings of
this study could be used for the MR applications in police interventions, in order to explore the
elements that reduce the cognitive load of the officers and enhance their performance.
 7

II. Method
A qualitative study is conducted in order to understand the cognitive overload of mixed-reality
interactions. Six students from Umeå University have been taken as the sample of the study
(See Table 01). Serious games approach is used in the study in order for participants to have
an interactive experience of using mixed-reality (Condino et al., 2022).

Participant Designation Faculty Gender Prior experience in

using MR
001 Student MIT M No
002 Student Management M No
003 Student MIT M No
004 Student Management F No
005 Student Management F No
006 Student MIT M Yes

Table 01: Participant Details

The study used a scenario of following a car and checking the ID of the person. The activity
consisted of two phases (See Figure 01). The first phase was playing the activity without the
HoloLens, and the second phase was replaying the same scenario while experiencing the MR,
by playing a virtual game where they had to kill robot enemies. The activity was held in an
office room where there were several obstacles such as chairs and tables participants had to
confront while doing the activity. After each phase of the activity, the participants were
instructed to complete a questionnaire, which was designed based on the CLT’s three factors.
The questionnaire was designed based on “The NASA Task Load Index” (NASA-TLX) which is
a multidimensional scale for measuring cognitive overload (Hart, 2006), and some useful
statements were adapted from other user experience questionnaires (Brooke, 1996; Krieglstein,
2023). The questionnaire had five-point responses, from “strongly disagree” (1) to “strongly
agree” (5). The main purpose of the questionnaire was to gather the support evidence for the
analysis of the interview data. We considered six measurement factors as stated in NASA-TLX
when developing the questionnaire (Hart, 2006).

1. Mental demand: How much mental effort took the participants to perform the activity/task
2. Physical demand: The level of physical effort needed to perform the task.
3. Temporal demand: The amount of time consumed to complete the task.
4. Effort: Overall considering both mental and physical effort, how difficult was the
participants to achieve their performance level
5. Performance: How satisfied were the participants with their performance
6. Frustration level: How insecure, irritated, overwhelmed, stressed, or discouraged, the
participants experienced during the completion of the task.

The questionnaire consisted of 19 questions under the categories of General awareness;

included 03 questions to understand the participants’ awareness and prior experience in using
mixed-reality, Intrinsic Load; contained 10 questions related to the understanding of intrinsic
load, Extraneous Load; included 10 questions to understand the level of extraneous load
experienced by the participants, and Germane Load; consisted 06 questions related the learning
 8

experience of the participants. All the participants completed the questionnaire soon after they
performed the activity and they were requested to get the statements clarified if they do not
comprehend well.
After the completion of the questionnaire, each participant was individually interviewed with
semi-structured questions in order to gather qualitative data on their experience in having
mixed-reality interactions. The interviews were voice recorded and all the recordings were
destroyed after generating the transcripts. All the gathered information including the interview
recordings and the transcriptions are kept highly confidential and will not be shared with any
external parties. The collected data were analysed through thematic analysis (TA), which helps
to analyse the research data closely by clustering the data into themes (Braun & Clarke, 2006).

Phase
01
Briefing of the
activity scenario

Follow the person with Call and ask for

Verify the ID
obstacles in the context the ID check

Playing the virtual game and Wearing the HoloLens

experiencing Mixed-reality and learning to operate it

Follow the person (with the Call and ask for Verifing the ID while
obstacles in the context) while the ID check on mixed-reality
playing the game in mixed-reality

Phase End of the

02 activity

Figure 01: Steps of the study activity

 9

III. Results
The collected data is analysed by thematic analysis. The deductive analysis method was used to
generate themes for the study. Three main themes were generated based on the Cognitive Load
Theory, as Prevention of Intrinsic Load, Prevention of Extraneous Load, and Increase of
Germane Load. Each of these themes has been divided into two categories as Interactions and
Information. The interaction category refers to the participants’ experience in interacting with
mixed reality, HoloLens, and the physical surrounding. Based on the activities, the interactions
were subdivided into the subjective experience participants discussed during the interviews and
the behavioural cues related to cognitive overload, overserved by the researcher during the
activity. The other category is Information which includes the participants’ experience of the
information received from mixed reality, and their suggestions/ insights on improving the
effectiveness and alternative methods. It represents how and what information should be
presented through mixed reality. Based on the data several codes were identified for each of
the categories (See Table 02).

Theme Category Subcategory Codes

Expertise in operating
Subjective experience HoloLens and MR
Interactions Avoid Split-attention
Prevention of Behavioural cues Reaction difficulty
Intrinsic Load (From observations) Tense Behaviour
Ability to grasp quickly
Information - Prioritizing and
Grouping

Subjective experience Design of the HoloLens

Prevention of Vigilance
Interactions Behavioural cues
Extraneous Load Task-oriented behaviour
(From observations)
Relevant details

Ease of Learning
Subjective experience
Product Improvements
Interactions
Behavioural cues
Efficiency in movements
Increase of (From observations)
Germane Load Visualize the next steps
Processing of Long-term
Information -
Memory
Comprehensibility

Table 02: Themes, Categories, and Codes of the analysis

Prevention of Intrinsic Load

Under the theme of prevention of intrinsic load, we illustrate the research data in relation to the
complexity of the task and the attention the participants could convey during the activity.
 10

Subjective experience in interactions (ICL)

Expertise in operating HoloLens and MR: Having the knowledge and familiarity in using the
HoloLens and the relevant MR application causes the user to operate it effortlessly, which
would require less mental effort. This provides more capacity for the working memory to
function on the task and for the user to keep his focus on the task problem. The participants
stated that if they get used to using the HoloLens and receive more time to practice the
instructions of MR, they might experience less difficulty in the activity.

”It takes time to operate. One time is not enough.” (001)

”Initially it will not be so comfortable, but when you get used to that, it will be very useful. I
felt that during the activity.” (006)

It was a bit uncomfortable when adjusting it to my head. I was afraid if it would fall down.
(002)

”When I was putting on the HoloLens, I felt a bit overwhelmed. But after some time, I got
used to it.” (002)

”when I get used to it, it would be easy.” (003)

Avoid Split-attention: When the attention is divided, the limited capacity of the working
memory is shared between the two elements. Therefore, the user is unable to pay complete
attention to the task. The study data shows that the task gets complex due to the intrinsic
cognitive overload, when the users’ experience split-attention during the activity. Since MR
allows to experience two realities at the same time, it could increase the intrinsic load of their
cognitive process.

”I should say that the activity made me a little bit irritated.” (004)

”...because I had a difficulty on concentrating. I was a bit concerned about how to

concentrate with two realities.” (004)

Behavioural cues observed in interactions (ICL)

Reaction difficulty: It was able to observe that some participants’ voice was shuttered for a
moment when they started to speak during the second phase of the activity. Furthermore, in
general, the speech responses were a bit delayed in the second phase of the activity. When
compared to the first phase it was quite noticeable.

Tense Behaviour: Most of the participants showed a bit of tense behaviour through their body
movements in the second phase of the activity. When comparing the two phases, they were
able to do the task in a much more relaxed manner during the first phase.

Presentation of the Information (ICL)

Ability to grasp quickly: The capability of grasping the appeared information quickly provides
evidence for having less intrinsic load on the working memory. The participants expressed their
preference for the method of receiving the information, between visual and auditory. There are
 11

some discrepancies in these user preferences, in which we cannot make conclusions about what
would be more effective to decrease the cognitive load.

”visual information is quicker to grasp” (003)

”I prefer the combination of both visual and auditory information, because it gives me much
information to have a clear picture of the situation/information.” (001, 006)

Prioritizing and Grouping: In order to have a low level of intrinsic load, the relevant
information appearing on the HoloLens should be prioritized. Also, the Grouping of the
information supports to grasp the information quickly when appeared. Furthermore, it enhances
the ability to efficiently link the information to the existing schemas in the long-term memory
as it receives as ”chunks”, which optimizes the germane process. At the end of the interview,
each participant was requested to imagine being in an actual situation of following a car, and
asked them to draw the appearance of the information on the whiteboard which they would
require during a real time situation of chasing a car (See Figure 02).

Figure 02: Illustration used for participants to imagine the real scenario

Participants marked the important information they would require on the illustration. Also, they
were asked to decide what information should be on their left and right views, as a subjective
grouping of the information (See Figure 03).
(Other illustrations are presented in the appendix)

Figure 03: Illustration with the marked information by a participant: 003

 12

In general, the prioritized information consisted of, (1) Speed of the following vehicle, (2)
Vehicle plate number, (3) Vehicle owner profile details; including name, age, and address
(through scanning of vehicle plate number), (4) Picture of the vehicle owner/driver, (5) Road
map, (6) Vehicle type, (7) Criminal records of the person, (8) Number of passengers in the car,
(9) Car registration details; to check whether the car is stolen, (10) Distance between the
participant’s car and the following car.

”Left side – Road map

Right Side – Vehicle plate number, Speed of the following vehicle, Owner profile details: Pic
& name” (003)

”Left Side: Picture of the owner

Right Side: Name, Age, Address, Criminal records of the person, and speed of the following
car” (002)

”Left side: Driver’s name, Car registration details

Right side: Speed of the following vehicle” (005)

In visualizing the appearance of information of MR, participants specifically mentioned what

information they would prefer to have on the left and the right side of their MR view. The
grouping of the information required by the participants is summarized below with the data
acquired from the illustrations and the interviews (See Table 03).

Vehicle
Picture
Speed of owner Criminal Number Distance
Vehicle of the Car
Grouping the profile Road Vehicle records of between
Participant plate vehicle registration
type following details: map type of the passengers two
number owner/ details
vehicle name, age, person in the car vehicles
driver
address
001 ✓ - - - - - - - - ✓
002 - - - ✓ - - - - - -

Left side of 003 - - - - ✓ - - - - -

the view 004 - ✓ - - - - - - - -
005 - - ✓ - - - - - ✓ -
006 ✓ ✓ - - - - - - - -
001 - - - - - ✓ - ✓ - -
002 ✓ - ✓ - - - ✓ - - -

Right side of 003 ✓ - ✓ ✓ - ✓ - - - -

the view 004 - - - - - - - ✓ - -
005 ✓ - - - - - ✓ - - -
006 - - ✓ - - - - - - -

Table 03: Grouping in appearance of the information by the participants

 13

Prevention of Extraneous Load

As Extraneous load refers to every element or process that is not unnecessary to complete the
task, under this theme the distractions and interruptions that influence the working memory are
presented. The extraneous load needs to be eliminated completely or kept at the lowest level,
in order to let the full capacity of the working memory be efficiently used for the task.

Subjective experience in interactions (ECL)

Design of the HoloLens: The design of the HoloLens (also the MR application; but it varies
depending on the task and industry) needs to be developed in favour of reducing the cognitive
load. If it’s not users would experience an additional cognitive load when attempting to interact
with the HoloLens. This increases the Extraneous Load since it is an unnecessary task or a
problem to solve by the user while working on the real task. Therefore, the design of the
HoloLens should make the users operate the HoloLens easily, requiring minimum cognitive
effort. Our research data presents two types of design-based cognitive load.

• Commanding: Most of the participants emphasized that the commanding part of the
HoloLens was distracting for them, especially if they use it while driving a vehicle.

Commanding is very sensitive for the moments (003)

The commanding part was quite a bit difficult for me. Even though I attacked the
enemy properly, it didn’t work well sometimes. (004)

Data explains that the commanding difficulties made them to experience Cognitive
Overload, by being confused, which would demand the cognitive processes to prioritise
solving the commanding issue over the actual task.

It was a little bit difficult to command. I was a bit confused. (001)

Participants made several suggestions to avoid cognitive load which caused by the issue
of commanding. Some of the participants preferred to have voice commands over
gestures. Some wanted to improve the gesture commands such as to use of fingertips
(only one finger) to command rather than using two fingers, and another participant
preferred to use the head gestures to command than using the fingers. Furthermore, as
suggested, there need to be considerable intervals between clicks.

”should be voice activated. Not gestures.” (001; 003)

”I prefer to command with just one finger like pressing the buttons. Just to give a
simple touch with the fingertip and move forward.” (004)

”I would prefer to command by waving my head while driving. Because using one
hand to command it while driving is quite difficult: Gestures by head, not hand”
(006)

”I prefer to have gaps in between the options when commanding: intervals

between clicks” (003)
 14

• User interference: Participants stated that the HoloLens has a very narrow view, thus,
they require to have a very stable HoloLens view that would support them to fully focus
on the task.

I had to always move my head to get the perfect view. That was difficult. (003)

we have to focus on our vehicle and the side mirrors, and everything surrounding
us. So, I prefer to have a very stable HoloLens view. (003)

The visuals should be more straight-focused since we’re driving the car. (006)

The lens has a very narrow strip, so it’s hard to have clear view of the Mixed-
reality information. (006)

I couldn't see the screen properly in the beginning, so I was a bit upset. (004)

The lens has a very narrow strip, so it’s hard to have clear view of the Mixed-
reality information. (006)

Behavioural cues observed in interactions (ECL)

Vigilance: It was observed that while the participants were chasing during the second phase,
some of them hit their legs on the stuff around, yet it did not seem that they even noticed it.
Furthermore, after they completed the task (checking the ID), they again went back to virtual
reality for a moment, losing their awareness of the physical world. Thus, a loss of vigilance
was noticed during the activity.

Task-oriented behaviour: During the second phase, most of the participants did not call the
person being followed (researcher) to stop before checking the ID, until the researcher stopped
and looked back at them. The sense of their role leading the activity had been missed. Hence,
the participant’s ability to maintain task-oriented behaviour, which is a critical element for
performance, was interrupted during the second phase of the activity due to the interactions in
the virtual reality.

Presentation of the Information (ECL)

Relevant details: Participants stated that they want to stick only with the relevant details to
their task. For instance, it was mentioned that they do not want to clutter their MR view with
too many details. They emphasized that only relevant information would be enough to appear
through the lens. This explains that they prevented having unnecessary information which
generates extraneous load.

”I do not want to clutter my view, so just a few important details would be enough.” (001;
003)

”I don't need any visual experience to follow the vehicle, so I prefer to have Auditory
information.” (005)

”Number plate we can see directly. So, we don’t need it.” (001)
 15

Increase of Germane Load

Germane load is the cognitive effort required for learning. The increase of the germane load
enhances the learning outcomes and the efficiency. Therefore, in this theme, we present the
data that supports optimizing the germane cognitive functioning.

Subjective experience in interactions (GCL)

Ease of Learning: The ease of learning determines the user’s potential for the highest
performance on completing the task. Enjoying the activity could be comprehended as
participants’ likelihood of learning and having a smooth functioning of the germane process.
Research data shows that most of the participants really enjoyed the learning experience
through the activity. The questionnaire data supports this as it has received four out of five
(4/5) responses on the questions favourable for an enjoyable learning experience.

” It was not really difficult for me to understand what's in the surrounding. The activity was
quite interesting for me.” (002)

” Overall activity was comfortable.” (001)

I enjoyed the learning experience. (Q) – 4/5

I felt like a burnout at the end of the activity. (Q) – 1/5

Mixed-reality experience would help me to make better decisions. (Q) – 4/5

Product Improvements: The participants’ suggestions for improvements for the HoloLens and
the MR application define how they want to enhance their learning process while minimizing
the cognitive load. For instance, having the “zoom in” facility in the HoloLens has been
suggested by one of the participants. He explains that he could observe the vehicle easily while
driving in a distance, and receive information such as the vehicle plate number, and the number
of passengers, by direct observations. Moreover, it was suggested to have a GPS tracking
system in the MR application, so that the user could keep track of the following vehicle with
just a simple command.

“Better if we have a zoom in facility. So that we can zoom in while driving and observe the
vehicle details.” (001)

“When you chase a vehicle (suspect) and you lose it after a while, then you can trace it
through the number plate details and get the direction of it via satellite/GPS technology. All
at the same time you drive the car.” (006)

Some suggested having alternative products to receive the information instead of the HoloLens,
such as having a microphone, in which the users would receive the auditory information, or
having a screen attached to the vehicle; similar to using Google Maps through the smartphone
while driving.

“If can use a microphone instead of HoloLens I can easily follow the vehicle.” (005)
 16

“I prefer if I can use a screen on the vehicle instead of the HoloLens. Like using the Google
map, it will be easy and helpful if we could get additional info. into that screen. Less
distractive.” (005)

Furthermore, a participant recommended changing the lens into a sunglass-type product. She
argued that the HoloLens is too huge to wear when driving a vehicle, and it would interrupt her
vision and disrupt her mental process, which would increase her cognitive load.

“I prefer to wear the HoloLens if it can be made as a Sunglass or a tiny tool. It would be
more comfortable then. But this equipment is so huge. It will disrupt me to follow the
vehicle.” (005)

Behavioural cues observed in interactions (GCL)

Efficiency in movements: Compared to the first phase, it was observed that participants had
slow movements (walking and interacting) when following, taking more time to complete the
second phase of the activity. This observation was also supported by one of the participants, as
she had experienced it during the activity.

“When we compare to the first activity, the second activity is more time consuming.” (004)

Presentation of the Information (GCL)

Visualize the next steps: The ability to visualize the steps that need to be taken ahead, is an
indicator of well-comprehending the information and having an effective learning process.
Hence, the following data represent the participants’ perspective on visualizing the steps they
need to take after receiving the information through the MR application. They argue that having
the information prior would ease the cognitive load of the next steps of the task, requiring less
cognitive effort and time to initiate the task. This provides a better scope for situational
awareness, where the participants could observe and learn more information when they reach
to the next steps since they have more capacity in the working memory due to the prior learning.

”I would have this information before. So, once I reach to the passenger, since I already knew
some information, it would not take much mental effort and time to verify the identification.”
(006)

” When I check his ID, I can make sure whether that is the exact person who is in the
profile.” (003)

Processing of Long-term Memory: Ability to remember and understand the information

indicates a better functioning of the germane process, as the learning has occurred effectively.
Some participants preferred to use visual information, mentioning that it would be easy for
them to remember the information. But there were some discrepancies as well since some
participants dislike using visual information, rather preferring to use auditory information to
remember well.

” Visual information is very easy to understand and remember” (003)

” I usually prefer visual information. Because it's easy to remember and it does not distract my
mind.” (004)
 17

” I prefer auditory information, because we need to remember the visual info. when it
disappears.” (002)

Comprehensibility: Easy to comprehend the information is one of the factors that would
enhance the germane load. During the study activity participants showed that they were able to
comprehend the information, which indicates a smooth process of linking new information to
the existing schema in the long-term memory.

I felt confused and panic not knowing what to do with the information appeared. (Q) – 1/5

Visualization of Questionnaire data

As stated in the methodology, the questionnaire was designed with a five-scale response
structure, in which the participants were allowed to choose between one to five depending on
whether they strongly disagree (1) or strongly disagree (5) with a statement based on their
experience of the activity. Through the questionnaire data, we developed a comparison of the
cognitive load experienced by the participants between the first phase and the second phase of
the activity. A Radar chart was generated to forecast the comparison (See Figure 04). Eleven
questions were taken into the comparison which were related to both the phase one and two
activities. For each question, the maximum value or the median value of the answers was taken
to indicate the cognitive load. In the chart, the highest number (5) is demonstrated as the
maximum cognitive load, and the lowest number (1) is demonstrated as the minimum cognitive
load. Hence, the area of each polygon defines the level of cognitive overload.

Contrast of CL in Two phases of the activity

1: IN
11: GE 2: IN

10: GE 3: IN

9: GE 4: EX

8: GE 5: EX

7: EX 6: EX

Without MR (First Phase) With MR (Second Phase)

IN: Intrinsic Load | EX: Extraneous Load | GE: Germane Load

Figure 04: Contrast of Cognitive Overload with and without MR
- The figure illustrates a comparison of 11 questions in the questionnaire -

The chart demonstrates that the participants experienced a higher level of cognitive load when
they were using mixed reality during the activity, when compared to the first phase of the
activity where they only experienced the physical reality.
 18

In order to visualize the variance of the three cognitive loads of the activity when using MR,
another radar chart was generated (See Figure 05). In the chart, it can be observed that the areas
of all three polygons do not have a considerable level of variance.

Comparison of Cognitive Load Factors - with MR

1
5

8 4 2
3
2
1
7 0 3

6 4

Intrinsic Load Extraneous Load Germane Load

Figure 05: Comparison of Cognitive Load factors with MR (Second Phase)

- The figure illustrates a comparison of 08 questions in the questionnaire -

When making a comparison between the Cognitive Load factors between the first phase and
the second phase of the activity, it could be observed that when using MR during the activity a
greater level of cognitive load has been increased. (See figures 05 & 06)

Comparison of Cognitive Load Factors - without MR

1
5

8 4 2
3
2
1
7 0 3

6 4

5
Intrinsic Load Extraneous Load Germane Load

Figure 06: Comparison of Cognitive Load factors without MR (First Phase)

- The figure illustrates a comparison of 08 questions in the questionnaire -
 19

Mixed Reality Cognitive Load Model

In this following diagram (See Figure 07), we have summarized the process of eliminating
cognitive overload, and optimizing performance. Here we present how our data codes support
in reducing cognitive overload and generate the maximum potential for learning.

Increased Performance

Efficiency in Problem solving

and Decision making

Increased Situation Rich relevant Competence Task Expertise: Skills &

Awareness information Experience

Efficiency
in learning
Ease of Learning

Optimization of Direct Product Improvements

causes
Increase of
Germane Germane Load
Functioning Efficiency in movements

Visualize the next steps

Indirect
causes
Processing of
Long-term Memory

Prevention of Prevention of
Intrinsic Load Extraneous Comprehensibility
Load
Unleash full capacity
of working memory

Expertise in operating Design of the HoloLens

HoloLens and MR
Vigilance

Avoid Split-attention
Task-oriented behaviour

Reaction difficulty
Irrelevant details

Tense Behaviour

Ability to grasp quickly

Prioritizing and Grouping

 20

Figure 07: Study diagram for optimizing the performance

IV. Discussion
MR is designed to operate in hybrid reality which merges the real world and the virtual world.
Therefore, users often could experience difficulty in interacting with two realities and receiving
information from both the physical and virtual worlds at the moment. This difficulty occurs
due to the cognitive load generated in users that interfere with their process of learning from
both realities. The purpose of technological advancements is to ease the workload of
individuals and increase the efficiency of completing a specific task (Flavián et al., 2019). This
supports individuals to enhance their performance. MR is considered one of the most beneficial
technological tools that could enhance the potential of human performance in a specific task.
Therefore, if certain elements interfere with gaining maximum useability of MR by generating
a level of cognitive overload, those elements need to be eliminated or kept at a low level. Hence
through this qualitative study, we attempted to find such elements and explored them to
broaden our understanding of preventing the cognitive overload in the use of MR applications.
One of the elements we found is the user’s knowledge and skills in operating the MR
application and the HoloLens. The more the users get familiar with and develop their expertise
in operating the MR, the lesser cognitive effort would take for them to get the maximum
support of it to complete the task. In other words, they would experience lesser task difficulty
with the maximum support of MR, which refers to a decrease in intrinsic cognitive load. Studies
express that the more expertise we build in a task, the lesser capacity of working memory would
need to complete the task (Ericsson, 2018). Therefore, having good practice and knowledge in
operating the MR and HoloLens would decrease the potential of generating cognitive overload,
since more capacity of the working memory would be available for the function of the main
task.
The split-attention emerged as the most critical factor in using MR. Keeping the focus between
two realities is a bit difficult. Some of the participants stated that they experienced split-
attention since they had to act in two realities simultaneously. Split-attention occurs when two
or more mutually dependant, yet spatially separated sources of visual information process
together in order to generate a proper comprehension of the situation (Schnotz & Kürschner,
2007). Split-attention disturb working memory to function properly, thus, the information
cannot enter to long-term memory for schema construction. Hence, occurrence of split-
attention needs to be at the lowest level to reduce the cognitive overload.
When using MR applications to receive information, for instance, in a scenario of chasing a
vehicle, the information that appears in the mixed reality display should be very relevant and
supportive to accomplish the task in the real world. Therefore, the information should be highly
comprehensible and relatable to the situation. Thus, appearing information must be prioritized
and grouped in a certain way that the user could comprehend it at a glance. Grouping the
information in MR support working memory to develop chunks in the mental state (Xu, 2016).
Chunking makes it easy to rehearse the information easily, thus, the information could get into
long-term memory where learning takes place (Ricker et al., 2010). Hence, effectively
grouping the information appearing through MR helps the user to comprehend the information
at ease which results in decreasing cognitive overload.
The design of HoloLens needs to be improved with the user experience testing according to
the field and task it is being used in. Also, the MR application has to be updated with relevant
features depending on the task and user’s preferences (Hughes et al., 2005). But when
considering user preferences, there might occur difference discrepancies. In the present study,
 21

some participants preferred to have only visual information when following a vehicle, while
some preferred to have auditory information. The preference was reasoned based on whether
the information-receiving mode makes any distractions for them or whether it helps them to
remember the information with ease. According to user experience studies, users tend to have
different preferences for interactions, and also, they might change their preferences due to their
mood or the time period they use the tool (Goguey et al., 2021). For instance, when chasing
the car, a person might prefer to use visual information in the summer, and he might change
his preference to auditory information during the winter, since his view is more cluttered with
snow and road safety signals, thus, he might prefer to avoid getting his vision distracted by the
virtual information. Therefore, these discrepancies need to be tackled by gaining more
knowledge through user experience studies in MR applications (Goguey et al., 2021).
However, the users need to be able to have an efficient mode of receiving information that
makes them better remember the information, which facilitates a smooth function of the
germane process. So that they could learn the information quickly and make use of the
information in solving their task problems.
Moreover, in the present study, participants made a few suggestions for improving the
HoloLens and MR application in the use of receiving information when following a vehicle.
One of the suggestions was to develop a tiny lens for the HoloLens; a design like a Sunglass,
so that it would not be disturbing their visual experience and would be more comfortable to
wear while driving. But on the other hand, even though the lens gets a bit smaller and easier to
wear, participants stated that the MR user interface; the parameter of the virtual display, needs
to be broader, so that they could grasp the information clearly and efficiently. This model has
been tested by Google, and they have already introduced a Sunglass-like lens called, “Google
Glass” (He et al., 2018). The Google Glass tool has been tested for driving performance as
well. The studies state that it has been very user-friendly even when driving (He et al., 2018).
Therefore, this tool also could be considered as most effective to use for the situation of chasing
a car, thus it would be better use for the MR application in police interventions. Another feature
improvement stated by the participants is to develop a “zoom in” ability to observe the
following vehicle closely and to have a GPS tracker facility to keep track of the following
vehicle. The developments of these features would help the users to gain more relevant
information in performing the task at ease, thus it would reduce the cognitive load.
One other element we noticed during the activity was participants losing the “Vigilance” in
the situation. During the activity, it was observed that participants were hitting their legs on the
obstacles around while they were playing in the mixed reality. But they didn’t seem to pay a
considerable amount of attention to the physical world even when they got hit. Furthermore, a
delay in response and a loss of sense of their role in the task was noticed. This means there was
a decrease in the situational awareness of the participants. Previous studies support these
observations stating that the mixed reality generates a delay in response time, while making a
drop in Vigilance, due to dual-task performance in two realities (Arif et al., 2022). This occurs
when the working memory is fully occupied which means the cognitive load is higher at this
moment (Helton & Russell, 2012). Police interventions often consists of vigilance tasks, thus
being on alert is highly important since the officers need to take quick decisions and make
immediate reactions (Verhage, 2018). In case, if they lose their focus during an intervention,
support should be provided through the mixed reality application, ensuring that no harm would
occur to the public or the officer. An in-built Obstacle Detection system could be developed in
MR applications for the safety purpose, as it has been already developed for AR users (Połap
et al., 2017).
These are the learning from our study that answers our research question of what causes
cognitive overload in mixed reality interactions. The above-mentioned elements are the main
findings of our study, especially in relation to the use case of police interventions. We believe
 22

that successful management of these causes of cognitive load would help to enhance the
performance of the users of Mixed reality. The present study findings could be mainly used to
develop MR applications for police interventions. But it would not be necessarily limited to
the MR application for one sector. As final remarks, we believe that our study broadened the
scope of understanding of Cognitive Overload in Mixed-Reality Interventions.

V. Limitations & Future directions

Our work had some limitations. (1) One of the limitations recognized is that there was no
control group to compare the performance and the cognitive load generated during the activity
conducted. We did compare the cognitive load of the same participants with having two phases
in the activity. However, we believe it would provide more rich data if a control group could
be used in future studies. Also, it would be great to compare the application of another
information-receiving tool e.g., mobile phone with the MR-Hololens by having a third group
in the study. (2) Another limitation that can be considered is that our study was based on only
the Cognitive Load Theory, therefore, for future studies we suggest considering other
Cognitive Load related theories as well when designing the studies (Sternberg et al., 2001;
Tennyson & Rasch, 1988). (3) The findings of the study are based on participants’ self-
assessments which limit the results to their subjective experiences. Therefore, it is
recommended to also include quantitative study methods in future studies in order to
objectively examine and weigh the level of relationships between these causing elements and
the users’ cognitive load. (4) Also, the present study was carried out with convenience-based
participant selection which has made limitations in the study in terms of real-time applications
in police interventions. For instance, the protocols police officers use during an intervention
would have a greater impact on determining what elements would cause the most in generating
cognitive load when using MR. Yet, since our study findings are not limited to the police
intervention applications, the usefulness is valid to any context in the interactions of MR.
However, we recommend conducting a study with the actual police officers, since that would
provide rich data through their work experience and the perspectives of being in real-time
interventions. (5) Moreover, we conducted the present study based on serious games, but it is
recommended to conduct future studies with a developed MR application that would be task-
specific and appear only relevant information to the task. We trust that more studies need to be
conducted in this area of managing cognitive load in MR applications. Furthermore, we
recommend to further develop our MR Cognitive Load Model in future studies with the new
findings of causing elements and preventing strategies of cognitive load.
 23

VI. References
Arif, S. M., Brizzi, M., Carli, M., & Battisti, F. (2022). Human reaction time in a mixed
reality environment. Frontiers in Neuroscience, 16.
https://doi.org/10.3389/fnins.2022.897240

Ayres, P. (2006). Using subjective measures to detect variations of intrinsic load within
problems. Learning and Instruction, 16, 389–400

Bogaard, G., Meijer, E. H., Vrij, A., & Merckelbach, H. (2016). Strong, but wrong: Lay
people’s and police officers’ beliefs about verbal and nonverbal cues to deception.
PLOS ONE, 11(6). https://doi.org/10.1371/journal.pone.0156615

Borglund, E., & Hansson, J. (2022). Tactical Police Interventions: Design Challenges for
Situational Awareness. In Rob Grace, & Hossein Baharmand (Eds.), ISCRAM 2022
Conference Proceedings – 19th International Conference on Information Systems for
Crisis Response and Management (pp. 1037–1047). Tarbes, France.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research
In Psychology, 3(2), 77-101. doi: 10.1191/1478088706qp063oa

Brooke, J. (1996). Sus: A “quick and dirty” usability scale. Usability Evaluation in Industry,
189, 4–7. https://doi.org/10.1201/9781498710411-35

Chandler, P. & Sweller, J. (1991). Cognitive load theory and the format of instruction.
Cognition and Instruction, 8(4), 293-332.

Chen, R., Grierson, L., & Norman, G. (2015). Manipulation of cognitive load variables and
impact on auscultation test performance. Advances in Health Sciences Education,
20(4), 935–952. https://doi.org/10.1007/s10459-014-9573-x

Clark, R. C., Nguyen, F., & Sweller, J. (2006). Efficiency in learning: Evidence-based
guidelines to manage cognitive load. San Francisco: Pfeiffer.

Condino, S., Gesi, M., Viglialoro, R. M., Carbone, M., & Turini, G. (2022). Serious games
and mixed reality applications for Healthcare. Applied Sciences, 12(7), 3644.
https://doi.org/10.3390/app12073644

Constantinidis, C., & Klingberg, T. (2016). The neuroscience of working memory capacity
and training. Nature Reviews Neuroscience, 17(7), 438–449.
https://doi.org/10.1038/nrn.2016.43

Ericsson, K. A. (2018). Superior working memory in experts. In K. A. Ericsson, R. R.

Hoffman, A. Kozbelt, & A. M. Williams (Eds.), The Cambridge handbook of expertise
and expert performance (pp. 696–713). Cambridge University
Press. https://doi.org/10.1017/9781316480748.036
 24

Flavián, C., Ibáñez-Sánchez, S., & Orús, C. (2019). The impact of virtual, augmented and
mixed reality technologies on the customer experience. Journal of Business Research,
100, 547–560. https://doi.org/10.1016/j.jbusres.2018.10.050

Frederiksen, J. G., Sørensen, S. M., Konge, L., Svendsen, M. B., Nobel-Jørgensen, M.,
Bjerrum, F., et al. (2020). Cognitive load and performance in immersive virtual reality
versus conventional virtual reality simulation training of laparoscopic surgery: A
randomized trial. Surgical Endoscopy, 34(3), 1244–1252. https://doi.org/
10.1007/s00464-019-06887-8

Gobet, F., Lane, P.C. and Lloyd-Kelly, M. (2015) ‘Chunks, schemata, and retrieval
structures: Past and current Computational Models’, Frontiers in Psychology, 6.
doi:10.3389/fpsyg.2015.01785.

Gochfeld, D., Brenner, C., Layng, K., Herscher, S., DeFanti, C., Olko, M., Shinn, D., Riggs,
S., Fernández-Vara, C., & Perlin, K. (2018). Holojam in Wonderland: Immersive
Mixed Reality Theater. SIGGRAPH ’18: ACM SIGGRAPH 2018 Art Gallery.
https://doi.org/10.1145/3202918.3203091

Goguey, A., Gutwin, C., Chen, Z., Suwanaposee, P., & Cockburn, A. (2021). Interaction pace
and user preferences. Proceedings of the 2021 CHI Conference on Human Factors in
Computing Systems. https://doi.org/10.1145/3411764.3445772

Gonzalez-Billandon, J., Aroyo, A. M., Tonelli, A., Pasquali, D., Sciutti, A., Gori, M.,
Sandini, G., & Rea, F. (2019). Can a robot catch you lying? A machine learning system
to detect lies during interactions. Frontiers in Robotics and AI, 6.
https://doi.org/10.3389/frobt.2019.00064

Gregory, T. M., Gregory, J., Sledge, J., Allard, R., & Mir, O. (2018). Surgery guided by
mixed reality: Presentation of a proof of concept. Acta Orthopaedica, 89(5), 480–483.
https://doi.org/10.1080/17453674.2018.1506974

Guzman, B., Deresky, S., Taylor, S., Wimmer, H., Momen, A., Tossell, C., Boyce, M.,
Cartwright, J., Amburn, C., & Sawyer, B. (2022). Evaluating mixed reality and tablet
technologies in military planning. 2022 Systems and Information Engineering Design
Symposium (SIEDS). https://doi.org/10.1109/sieds55548.2022.9799294

Haque, S., & Saleem, S. (2020) "Augmented reality based criminal investigation system
(ARCRIME)," 2020 8th International Symposium on Digital Forensics and Security
(ISDFS), Beirut, Lebanon, 2020, pp. 1-6, doi: 10.1109/ISDFS49300.2020.9116204.

Hart, S. G. (2006). NASA-Task Load Index (NASA-TLX); 20 years later. Proceedings of the
Human Factors and Ergonomics Society Annual Meeting, 50(9), 904–908.
https://doi.org/10.1177/154193120605000909

He, J., McCarley, J. S., Crager, K., Jadliwala, M., Hua, L., & Huang, S. (2018). Does
wearable device bring distraction closer to drivers? comparing smartphones and Google
Glass. Applied Ergonomics, 70, 156–166. https://doi.org/10.1016/j.apergo.2018.02.022
 25

Helton, W. S., & Russell, P. N. (2012). Visuospatial and verbal working memory load:
Effects on visuospatial vigilance. Experimental Brain Research, 224(3), 429–436.
https://doi.org/10.1007/s00221-012-3322-2

Hughes, C. E., Stapleton, C. B., Hughes, D. E., & Smith, E. M. (2005). Mixed reality in
education, entertainment, and training. IEEE Computer Graphics and Applications,
25(6), 24–30. https://doi.org/10.1109/mcg.2005.139

Ilori, A., Li, Y., Mahesh, V., & Craig, B. (2016). Effect of position: An ergonomics
evaluation of police’s wearable equipment. Advances in Intelligent Systems and
Computing, 199–207. https://doi.org/10.1007/978-3-319-41685-4_18

Kirsh, D. (2000). A few thoughts on cognitive overload. Intellectica. Revue de l’Association

Pour La Recherche Cognitive, 30(1), 19–51. https://doi.org/10.3406/intel.2000.1592

Kockord, R., & Bodensiek, O. (2021). Cognitive load during first contact with mixed reality
learning environments. Mensch Und Computer 2021.
https://doi.org/10.1145/3473856.3474003

Koller, C. I., Wetter, O. E., & Hofer, F. (2015). ‘who's the thief?’ the influence of knowledge
and experience on early detection of criminal intentions. Applied Cognitive Psychology,
30(2), 178–187. https://doi.org/10.1002/acp.3175

Krieglstein, F., Beege, M., Rey, G. D., Sanchez-Stockhammer, C., & Schneider, S. (2023).
Development and validation of a theory-based questionnaire to measure different types
of cognitive load. Educational Psychology Review, 35(1).
https://doi.org/10.1007/s10648-023-09738-0

Merrienboer, J., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent
developments and future directions. Educational Psychology Review, 17, 147–177

Mesaros, P., Mackova, D., Spisakova, M., Mandicak, T., & Behunova, A. (2016). M-learning
tool for modeling the building site parameters in mixed reality environment. 2016
International Conference on Emerging eLearning Technologies and Applications
(ICETA). https://doi.org/10.1109/iceta.2016.7802094

Oberauer, K., & Hein, L. (2012). Attention to information in working memory. Current
Directions in Psychological Science, 21(3), 164–169.
https://doi.org/10.1177/0963721412444727

Paas, F., Renkl, A., & Sweller, J. (2004). Cognitive load theory: Instructional implications of
the interaction between information structures and cognitive architecture. Instructional
Science, 32, 1–8.

Park, S., Bokijonov, S., & Choi, Y. (2021). Review of Microsoft hololens applications over
the past five years. Applied Sciences, 11(16), 7259.
https://doi.org/10.3390/app11167259

Parong, J., & Mayer, R. E. (2018). Learning science in immersive virtual reality. Journal of
Educational Psychology, 110(6), 785–797. https://doi.org/10.1037/ edu0000241
 26

Prattico, F. G., & Lamberti, F. (2021). Mixed-reality robotic games: Design Guidelines for
Effective Entertainment with consumer robots. IEEE Consumer Electronics Magazine,
10(1), 6–16. https://doi.org/10.1109/mce.2020.2988578

Ricker, T. J., AuBuchon, A. M., & Cowan, N. (2010). Working memory. WIREs Cognitive
Science, 1(4), 573–585. https://doi.org/10.1002/wcs.50

Rokhsaritalemi, S., Sadeghi-Niaraki, A., & Choi, S. (2020). A review on Mixed reality:
Current trends, challenges and prospects. Applied Sciences, 10(2), 636.
https://doi.org/10.3390/app10020636

Schnabel, M. A., & Wang, X. (2009). Mixed Reality in Architecture, Design and
Construction. https://doi.org/10.1007/978-1-4020-9088-2

Schnotz, W., & Kürschner, C. (2007). A reconsideration of cognitive load theory.

Educational Psychology Review, 19, 469-508.

Seufert, T. (2018) ‘The interplay between self-regulation in learning and Cognitive Load’,
Educational Research Review, 24, pp. 116–129. doi:10.1016/j.edurev.2018.03.004.

Smith, E., McRae, K., Semple, G., Welsh, H., Evans, D., & Blackwell, P. (2021). Enhancing
vocational training in the post-COVID ERA through mobile mixed reality.
Sustainability, 13(11), 6144. https://doi.org/10.3390/su13116144

Sternberg, R. J., & Zhang, L. (2001). Experiential Learning Theory: Previous Research
and New Directions. In Perspectives on thinking, learning, and cognitive styles. L.
Erlbaum Associates.

Sweller, J., van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and
instructional design. Educational Psychology Review, 10(3), 251–296. https://doi.
org/10.1023/A:1022193728205

Tennyson, R. D., & Rasch, M. (1988). Linking cognitive learning theory to instructional
prescriptions. Instructional Science, 17(4), 369–385.
https://doi.org/10.1007/bf00056222

Tepper, O. M., Rudy, H. L., Lefkowitz, A., Weimer, K. A., Marks, S. M., Stern, C. S., &
Garfein, E. S. (2017). Mixed reality with hololens. Plastic and Reconstructive Surgery,
140(5), 1066–1070. https://doi.org/10.1097/prs.0000000000003802

Touel, S., Marfisi-Schottman, I., & George, S. (2020). Analysis of mixed reality tools for
learning math in primary and Secondary School. Lecture Notes in Computer Science,
112–121. https://doi.org/10.1007/978-3-030-63464-3_11

Verhage, A., Noppe, J., Feys, Y., & Ledegen, E. (2018). Force, stress, and decision-making
within the Belgian police : the impact of stressful situations on police decision-making.
JOURNAL OF POLICE AND CRIMINAL PSYCHOLOGY, 33(4), 345–357.
https://doi.org/10.1007/s11896-018-9262-4
 27

Xu, F. (2016). Short-term working memory and chunking in SLA. Theory and Practice in
Language Studies, 6(1), 119. https://doi.org/10.17507/tpls.0601.16

Zechner, O., Kleygrewe, L., Jaspaert, E., Schrom-Feiertag, H., Hutter, R. I., & Tscheligi, M.
(2023). Enhancing operational police training in high stress situations with virtual
reality: Experiences, tools and Guidelines. Multimodal Technologies and Interaction,
7(2), 14. https://doi.org/10.3390/mti7020014

Zhaotong, S., Bing, H., & Yi C. (2017). Study on Mixed Reality Application in Construction
Engineering. Journal of Information Technology in Civil Engineering and Architecture,
9(3): 43-46. doi: 10.16670/j.cnki.cn11- 5823/tu.2017.03.07
 28

VII. Appendix

01. Illustrations of Prioritizing and grouping information

Figure 08: Illustration of the participant: 002

Figure 09: Illustration of the participant: 003

 29

Figure 10: Illustration of participant:004

Figure 11: Illustration of participant:005

 30

Figure 12: Illustration of participant:006

 31

02. Interview Guide

Step 01: Brief introduction about the study
We conduct this study to understand how to reduce the Cognitive load (having too much
information and distractions) when using Mixed Reality (MR). We aim to use the findings of
this study to make applications of Mixed reality in police cases.
Now we conduct a simple activity with you and you may fill a questionnaire after the activity.
Also, there will be a short interview with you to understand your experience during the activity.

Data Privacy policy explained: The interview will be voice recorded in order for us to generate
the transcriptions of it. After transcribing the voice record, it will be permanently deleted. All
the data we gather will be confidential and will not be shared with a third party. Also, at any
given moment, during the interviews or till the study completes, you have the right to withdraw
if you wish to. At that point, the data collected from you will be removed. It can be done by
sending an email: sachpalihawadana@gmail.com

Step 02: Taking the informed consent and personal data protection form

Step 03: Filling the Demographic questionnaire

Step 04: Activity – 10 min

Step 05: Filling the scale questionnaire – 10 min

Step 04: The interview (Duration: 15 - 20 mins)

Interview Questions

1. How did you experience the activity?

- Did you feel irritated or overwhelmed? If so, why?
2. How did you feel wearing the HoloLens?
3. How difficult were you to keep your focus on the activity?
4. Do you prefer auditory or visual information? Why?
5. How do you feel using it during an actual situation?
- Would it be effective to use? Why?
6. How easy were you to operate the HoloLens?
- Any suggestions to improve the commands or appearance?
 32

03. Questionnaire

Reference number:

Title of project: A Qualitative Analysis of Understanding the Cognitive Overload in Mixed-

Reality Interactions

General Awareness

Question Strongly Strongly

disagree agree
1 I had experience in using mixed
reality before this activity.
1 2 3 4 5
2 I had experience in wearing
HoloLens.
1 2 3 4 5
3 I have never heard about using
HoloLens or/and mixed reality until
now.
1 2 3 4 5
 33

Intrinsic Load

Question Strongly Strongly

disagree agree
1 I think that I would like to use
mixed reality frequently in my daily
tasks.
1 2 3 4 5
2 The activity was unnecessarily
complex.
1 2 3 4 5
3 I followed the steps of the activity
very easily with the information
given through the HoloLens.
1 2 3 4 5
4 I think I would need technical
support from someone when I use
the HoloLens next time.
1 2 3 4 5
5 The activity made me feel
overwhelmed.

1 2 3 4 5
6 I couldn’t grasp all the information
provided through the HoloLens.
1 2 3 4 5
7 I feel that most of my peers/
colleagues would learn to use mixed
reality very quickly.
1 2 3 4 5
8 I felt very comfortable wearing the
HoloLens during the activity.
1 2 3 4 5
9 I felt very confident using the
mixed-reality to receive information.

1 2 3 4 5
10 I did the activity with a
minimum mental effort.
1 2 3 4 5
 34

Extrinsic Load
Question Strongly Strongly
disagree agree
1 The order of the information
delivered through the HoloLens was
easy to follow.
1 2 3 4 5
2 I felt stressed/irritated with the
HoloLens sometimes during the
activity.
1 2 3 4 5
3 I noticed that some of the
information received through
HoloLens are unnecessary.
1 2 3 4 5
4 I experienced this very challenging
and tense with all the information
delivered quickly.
1 2 3 4 5
5 I found the various functions in the
HoloLens app were well integrated
with the surrounding environment.
1 2 3 4 5
6 I felt connected with others and
found easy to communicate during
the activity.
1 2 3 4 5
7 I found the visual information easier
to follow than the auditory
information.
1 2 3 4 5
8 I prefer auditory information than
the visual information.
1 2 3 4 5
9 It took me more time than usual to
react and make my movements.

1 2 3 4 5
10 I was worried about what others
would feel seeing me wearing the
HoloLens.
1 2 3 4 5
 35

Germane Load

Questions Strongly Strongly

disagree agree
1 I understood the appeared
information very clearly.
1 2 3 4 5
2 I felt like a burnout at the end of the
activity.
1 2 3 4 5
3 Mixed-reality experience would help
me to make better decisions.
1 2 3 4 5
4 The information I received made me
easy to recall the things I learnt
before.
1 2 3 4 5
5 I felt confused and panic not
knowing what to do with the
information appeared.
1 2 3 4 5
6 It was really an enjoyable learning
experience.
1 2 3 4 5