Professional Documents
Culture Documents
Virtual Reality Support of Cognitive Processes in Firefighter Skills Training
Virtual Reality Support of Cognitive Processes in Firefighter Skills Training
Department of Safety, Chemistry and Biomedical Department of Computer Science, Electrical Engineering
Laboratory Sciences and Mathematical Sciences
Western Norway University of Applied Sciences Western Norway University of Applied Sciences
Haugesund, Norway Bergen, Norway
cecilia.hammar.wijkmark@hvl.no ilona.heldal@hvl.no
Abstract—Practice-based training in realistic settings is steel-and-concrete buildings, which will have already sustained
necessary to acquire and retain competence as a firefighter. Hot- numerous fires and have limited similarity to buildings within
Fire Live Simulation (HF-LS) training in real fire and smoke, real society) provide limited variation in terms of buildings,
using training buildings, vehicles, and equipment, has been the objects, and interaction possibilities.
only available practice-based training format available, and
therefore accepted despite its limitations. Virtual Reality (VR) During the last decade, the interest in Virtual Reality (VR)
development has provided possibilities for supplementing HF-LS training has increased for training applications in different
training. However, hesitation exists, often because of doubt domains, such as medicine, industry, and the military, where
regarding the represented realism of virtual fire situations. This practice-based skills training is associated with high risk or high
paper investigates how immersive VR supports cognitive processes cost [1-3]. Simulation-based training (HF-LS or VR) aims to
and affects fidelity in firefighter training, compared with HF-LS. reproduce realistic situations with adequate perceptual
Data were collected from firefighter recruits, experienced information and thus trigger behavior similar to a real situation.
firefighters, and instructors at a Fire and Rescue Service in In VR applications, the user can interact with the computer-
Sweden, 33 persons in total, during VR and HF-LS training. VR generated graphical environment and representations of objects,
experiences were compared with previous experiences of HF-LS. based on rules, feedback, and experiences influenced by the
The results emphasize the current benefits and limitations of VR allowed interaction possibilities and technology used.
versus HF-LS training in relation to the support of cognitive
Consequently, it is crucial that users understand the concordance
processes and the representations’ fidelity.
between the simulated environment and its objects, and the
Keywords— Virtual Reality; immersive; firefighter; training; corresponding real-world objects as well as the task [3]. This
Live Simulation; cognitive processes; fidelity understanding relies on the perceived realism of the VR
representations; in other words, it depends on the ability of VR
I. INTRODUCTION to support the CPs [4]. This paper uses the term fidelity for the
degree of exactness with which virtual representation copy or
Firefighters are trained to handle a wide spectrum of
reproduce objects, people, phenomena, interactions or real-life
emergency situations. All fires are different and the situation can
environments. We also use the broader term realism which also
change according to fuel, location, and other circumstances.
refers to how believable abstractions are represented in VR.
Firefighters are expected to be able to handle each situation,
which requires awareness and the ability to recognize risks and Notwithstanding current interest and availability, there is
make quick and correct decisions in critical circumstances where hesitation toward VR training in Fire and Rescue Service (FRS)
lives may be at risk. organizations, which can be explained by mistrust in the
conveyed realism of representations in VR training and fear that
Practice-based Hot-Fire Live Simulation (HF-LS) in the
VR training will replace HF-LS [5]. However, the critical voices
controlled environment on training grounds, using real fire
also recognize the possible complementary value of VR [6].
(burning wood or gas), smoke, and vehicles, is currently the gold
standard for drill training. It allows firefighters to familiarize Based on a field study, this paper discusses how immersive
themselves with available equipment and work in high VR, in relation to HF-LS, can support a number of higher CPs
temperatures. The physical objects, buildings, and environment identified as crucial in firefighter skills training as well as the
in HF-LS are considered to enable sufficiently naturalistic relation to fidelity. The study involved 33 persons; firefighter
experiences, and thereby trigger Cognitive Processes (CP) recruits, experienced firefighters, and instructors from the FRS
similar to real incidents. However, HF-LS has certain Östra Skaraborg, Sweden. None of the participants had any
limitations. Training facilities (involving a limited number of previous experience of VR-based training. The study was
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.
C. Hammar Wijkmark et al. • Virtual Reality Support of Cognitive Processes in Firefighter Skills Training
conducted using a previously published method [7] based on higher CPs recognition, comprehension, learning and decision
data collected through questionnaires and observations of VR making are identified as necessary for training and measurable
and HF-LS training and problem-solving, [7]. This research using self-reports and observations, in non-immersive VR
investigates how immersive VR supports the following CPs training for Incident Commanders [7]. In the current study, we
recognition, comprehension, learning and decision making, in examine the same CPs for immersive VR training for
relation to HF-LS training. firefighters` skill training . The following higher CPs (Layer 6
of the LRMB) are discussed in this report:
II. BACKGROUND
• Recognition: the ability to identify an object, which
A. Learning approach and limitations relates to the realism of buildings, vehicles, avatars,
Firefighter training often incorporates two different HF-LS flames, and smoke. Recognition of subtle risks and
training formats involving real fire and smoke: 1) container warning signals, (e.g., changing smoke behavior is
training, used for basic training in understanding the fire essential for firefighter preparation).
behavior and use of extinguishing methods, and equipment (see
Fig.1), and scenario training in concrete buildings or steel ship • Comprehension: the capability to understand the
container constructions used for full incident scenario training relationship between the objects and attributes of the
(HF-LS Scenario training). Both formats require the firefighters incident scene (e.g., how the fire is spreading). This
to use Breathing Apparatus (BA) and to perform smoke diving. aspect can be related to situational awareness (see
HF-LS scenario training involves a Breathing Apparatus Entry Endsley [11, 12]).
(BAE) team, whereby two firefighters enter the building (smoke • Learning: the ability to perform in steps and practical
diving) while a BAE leader keeps track of time and available air procedures, and thereby update the cognitive models of
and should be prepared to enter the building if necessary. the brain in the long-term memory. This draws parallels
In HF-LS, the available training facilities (buildings, fire to applying knowledge [13] and gaining a new concrete
trucks, and equipment) can be used to simulate several scenarios experience [14, 15].
to train technical skills in a controlled environment, as close to • Decision-making: the process of choosing a course of
the realism of actual incidents as possible. action from a set of alternatives. In fire situations, the
These steel and concrete buildings at FRS training grounds time constraints and the levels of risk significantly affect
around the world have been built to withstand fire and water this process, and it is often highly critical to act and
numerous times per week for many years. They must represent evaluate the courses of action quickly. The importance of
different types of buildings from the society, e.g., sometimes a rapid decision-making in these situations has been
supermarket, other times an office building. Thus, they cannot described in the Recognition-Primed Decision (RPD)
resemble many real-life buildings that the firefighters will model [16].
encounter in real fire situations. Additionally, safety precautions
III. METHODOLOGY
and environmental regulations limit the quantities and types of
fuel that can be used, resulting in limited fire development. This The field study was conducted at the FRS Östra Skaraborg
limits the possibility for the simulation of cues, events, and training facilities, where 33 employees participated over a period
consequences of incorrect or no actions. The smoke in HF-LS of six days in October 2020. The participants were divided into
also contains carcinogenic substances, harmful if inhaled and/or three groups depending on previous firefighting experience. The
through skin exposure, since they penetrate fire protective groups were:
clothing [8]. Stud (students): Firefighter recruits (15 participants)
participating in the introductory program. At the time of the
study, they were in their second week of the three-week program
designed to prepare them to work as part-time firefighters. Prior
to the study, they had performed three or four BA-entries in
container training. The decision to schedule VR training at this
point in the schedule was motivated by the expected value to
prepare for the HF-LS scenario training the following week.
Instr (instructors): The instructors (5 participants) were
experienced firefighters who had been assigned as instructors.
Exp (experienced firefighters): 13 professional firefighters
Fig. 1. HF-LS container, exterior (left) and interior (right). with more than 3 years of practice.
A. Study Design
B. Higher Cognitive Processes
The study was conducted in two steps: A) the VR training
Education research often considers findings from Cognitive
(Stud, Instr and Exp), and B) the HF-LS scenario-based training
Science, particularly for defining aspects that improve learning
(Stud).
[9], as well as how the human brain functions. In the Layered
Reference Model of the Brain, LRMB [10], the main CPs are A. The field study of the VR training was performed using
decomposed into six layers and a total of 37 basic CPs. The the study design previously published [17]. The participants
000028
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.
IEEE 1st International Conference on Cognitive Aspects of Virtual Reality • CVR 2022 • 11-12 May 2022, Online
000029
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.
C. Hammar Wijkmark et al. • Virtual Reality Support of Cognitive Processes in Firefighter Skills Training
in a minimum of two HF-LS sessions annually. However, longer and 73% of the Stud scored this as Likert 3 or higher. When
intervals may occur, as was the case for five of the experienced asked, “To what extent did you experience a feeling of this
participants in the study. All Stud participated in their first HF- incident happening for real?” 100% of Exp, 80% of Instr, and
LS container training the week prior to the study. When the 73% of Stud scored this as Likert 3 or higher. When asked
experienced firefighters and instructors were asked to estimate further about the more physical feeling of extinguishing in VR,
the length of time they had been actively training, in the last HF- and compared with the feeling of extinguishing a fire in HF-LS,
LS, the Instr stated an average of 20 minutes and the Exp 14 85% of the Exp and 40% of both Inst and Stud scored the
minutes. The total time spent in the training session, including similarity between VR and HF-LS as acceptable (3 on the Likert
all preparations and restoring equipment, was on average two scale or higher). Detailed responses included: “I think it was a
hours for the Instr and 1.5 hours for Exp. Therefore, HF-LS bit harder to extinguish in VR compared with the container
provides a time efficiency of 16%, calculated by dividing the (wood and LPG) fires.” (Stud 10) and “The few times I have
active training time by the total time required. By comparison, extinguished for real [in HF-LS container], I have had more
three VR scenarios were conducted in 20 minutes without effect and extinguished faster” (Stud 3). Further research is
preparation and with only minutes of restart before the next needed to explore whether extinguishment in HF-LS or by using
participant. In 120 minutes, the firefighter could perform in 17 VR tools provides the closest fidelity to real incident fire
VR scenarios at a pace corresponding to this study (an average behavior and extinguishment.
of 6.6 minutes per scenario).
When starting the scenarios, the participants were
TABLE 2. Results from Questionnaire 2, after immersive VR training, for the three groups of firefighter recruits (Stud), instructors (Inst) and
Experienced firefighters (Exp). The first column in each group presents the percentage that scores acceptable or higher (Likert >=3). The
second column presents the average (Avg.) and the third column presents the standard deviation (SD.)
Think of the experience. To what extent did you experience a 73% 3,07 1,00 80% 3,20 1,02 100% 3,92 0,73
feeling of this incident happening for real?
Compared to the feeling of extinguishing a fire in hot fire 40% 2,33 1,07 40% 2,60 0,80 85% 3,23 0,80
training, how similar would you say the feeling of
extinguishing in VR was?
Visual realism
To what extent did you find the virtual representation of the 87% 3,20 1,11 100% 3,80 0,40 92% 3,62 0,84
FIRE realistic?
To what extent did you find the virtual representation of the 87% 3,13 0,96 80% 3,40 0,80 92% 3,54 0,75
SMOKE realistic?
To what extent did you find the virtual representation of the 73% 3,20 1,17 100% 3,80 0,40 100% 3,92 0,47
WATER realistic?
To what extent did you find the task realistic? 93% 3,87 0,96 100% 4,40 0,49 92% 3,85 0,66
Overall
In general, how positive are you about VR training? 93% 4,33 0,87 100% 4,40 0,80 100% 4,77 0,58
B. Experiences of Immersive VR training realistically informed about the situation and safety measures,
A summary of the answers from Q2 is presented in Table 2. by the instructor playing the role of the BAE-leader. This
In response to the question, “Think of some previous HF-LS information only contained the information available when
training session when you experienced a high presence. arriving at a scene, not more, such as “ the fire is on the second
Compared with that, to what extent did you experience presence floor of a family house”. As in real-life incidents, after arriving,
in the VR simulation today?”, 92% of the Exp, 80% of the Inst, the firefighter can, and is required to, recognize the objects
involved, comprehend what is happening, and identify risks
000030
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.
IEEE 1st International Conference on Cognitive Aspects of Virtual Reality • CVR 2022 • 11-12 May 2022, Online
directly by looking at the high fidelity virtual 3D scene. No risks. The fire cannot spread and therefore dynamic effects of
additional verbal information is necessary to recognize an wrong actions cannot be demonstrated, HF-LS does not support
object. When entering the building, the firefighter can identify updating the cognitive models of the brain in the long-term
different rooms, such as a kitchen or a living room, by observing memory regarding wrong actions, thus limiting learning
the virtual objects. opportunities.
The results (Table 2) also illustrate the recognition of Seven out of the total of 15 Stud performed in the BAE team
objects, the perceived realism of fire, smoke, water, and the task in the observed HF-LS training. A total of 86% of them scored
to be performed in the scenario, such as extinguishing the fire. acceptable or higher (Likert >=3) for the question “Think of the
The average perceived realism was 3, with the highest value for
water (average 3.92 from Inst) and the lowest for smoke
(average 3.13 from Stud). Explanations related to lower
experienced realism of smoke in the Stud group included: “I did
not experience the smoke in the same way as in real training
[HF-LS]” (Stud 4), and “it was unrealistic that the smoke did not
get lower” (Stud 10). One Exp and one Inst stated that more
smoke would increase the realism, while others made contrary
statements such as “the flames and their color were realistic”
(Exp 6), “the intensity of the flames” (Exp 5), “the clear smoke
layer” (Exp 4), and “the smoke was thick and rolling” (Exp 5).
Regarding the support for learning, and the possibility of
updating the cognitive models in the brain in the long-term
memory, the immersive VR training supports active
experimentation, through repetitive “runs”, e.g., also trying to
extinguish the fire using a “wrong” method. Correspondingly,
decision-making is supported by the dynamic scenario; for Fig. 3. HF-LS scenario training building
example, the fire will spread if the correct decisions on actions
are not made. previous HF-LS container training. Compared with that, to what
extent did you experience presence in the VR simulation today?”
In response to the question “In general, how positive are you The result was the same (86%, Likert >=3) for the question “To
about VR training?”, 100% of the Exp and Inst scored Likert 3 what extent did you experience a feeling of this incident
or above, and 93% of the Stud, namely one participant (Stud 5) happening for real?” When asked to compare the feeling of
scored Likert 2 for this question, giving no explanation. extinguishing in HF-LS with the feeling of extinguishing in VR,
C. Hot-Fire Live-Simulation 86% scored Likert 3 or higher. Comments related to the
similarity between VR and HF-LS realism of fire, smoke, and
The HF-LS scenario-based training observed in this study
water included: “the handling of the nozzle, the amount of water
was performed in the building shown in Fig. 3, which is a
and the recoil was extremely similar in VR, compared with the
standard building found at numerous Swedish FRS training
reality” (Stud 10), “the extinguishing feels very similar and the
facilities that represents an apartment building. Although the
functions of the nozzle were very similar” (Stud 10). One Stud
fire, flames, smoke, and heat are real, they are limited because
(Stud 8) underlined in the comments that “Reality is reality, and
of safety and the amount of fuel allowed. The interior has a
VR is a simulation and you will never get away from the fact
staircase and rooms, but it is difficult to recognize different
that it is not for real”.
rooms owing to the lack of furniture and the black soot covering
all surfaces. The instructor plays the role of the dispatcher, the D. The support for Cognitive Processes
commander, and the house owner. The Stud firefighter team The compared training formats, HF-LS and immersive VR,
receives the call in their firetruck and drives to the scene. Until support the higher CPs to a different extent. This is summarized
this point, the communication is verbal, similar to the HF-LS, in Table 3. The differences highlight the complementarity of the
VR, and real life. The difference between HF-LS and VR methods and may enhance the knowledge about benefits and
becomes more apparent when firefighters arrive at the scene. limitations of each.
Through role-play and verbal information, the instructor
describes the objects and the situation, with instructions such as V. CONCLUSIONS
“the fire is on the second floor, to the left”, “there is an attic”, or
If accurate simulation of real-life incidents is necessary for
“the building on the left is a garage”. The firefighter cannot
good training, neither VR nor HF-LS are sufficiently realistic
recognize what type of (real-world) building this is representing
today. The challenge is to identify their specific benefits and
and cannot comprehend what is going on, namely the situation,
limitations and utilize a combination that would provide the best
cues, and risks. With no additional information, the situation can
training value for handlinig future incident situations. Enhanced
be interpreted as: “there is smoke coming out from a concrete
training values lie in knowing how to supplement training.
construction with several openings and doors. There are no open
flames visible. Next to this construction is a steel container”. It The results highlight the possibility of using immersive VR
is not possible to simulate different fire or smoke behaviors to support the CPs for recognition (e.g., perceive real-like
allowing the recognition and comprehension of cues or specific objects, comprehension (e.g., perceive realistic dynamic
000031
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.
C. Hammar Wijkmark et al. • Virtual Reality Support of Cognitive Processes in Firefighter Skills Training
situations), and learning (e.g., explore the possibility of trying [12] M. Endsley, “Endsley, M.R.: Toward a Theory of Situation Awareness in
different – including wrong - actions), and decision making, Dynamic Systems. Human Factors Journal 37(1), 32-64,” Human Factors:
The Journal of the Human Factors and Ergonomics Society, vol. 37, pp.
(through evaluating the effects of actions or non-actions). 32-64, 03/01, 1995.
The presence experience was reported to be higher in the Exp [13] W. A. L. Blyth, B. S. Bloom, and D. R. Krathwohl, "Taxonomy of
and Instr groups than in the Stud group, as was the perceived Educational Objectives. Handbook I: Cognitive Domain," Faber & Faber
Limited, 1966, pp. 119-119.
realism of flames, smoke, and water representation.
[14] D. Kolb, Experiential Learning: Experience As The Source Of Learning
Additional values of using VR illustrated by this study And Development, 1984.
included the time efficiency and no particle of heat exposure, [15] D. A. Kolb, R. E. Boyatzis, and C. Mainemelis, "Experiential Learning
which firefighters encounter in HF-LS [8]. The mandatory two Theory: Previous Research and New Directions," pp. 227-248: Routledge,
2001.
HF-LS trainings per year relate to heat exposure, which is an
[16] R. Eisenberger, “Decision making in action: Models and methods, Klein,
important aspect of BAE. Scenario-based HF-LS is seldom G.A., Orasanu, J., Calderwood, R. and Zsambok, C.E. (eds). Norwood,
performed, presumably because of the associated limits in NJ: Ablex, 1993, 480 pp. ISBN 0–89391–794–X (pb),” Journal of
supporting CPs. This study demonstrates the acceptance of Behavioral Decision Making, vol. 8, no. 3, pp. 218-219, 1995/09/01,
scenario based- training employing immersive VR, as a 1995.
supplement to the other training methods, in a time efficient and [17] C. H. Wijkmark, M. M. Metallinou, and I. Heldal, “Remote Virtual
safe format. Simulation for Incident Commanders—Cognitive Aspects,” Applied
Sciences, vol. 11, no. 14, pp. 6434, 2021.
Future research is needed to investigate the support for other [18] "FLAIMTrainer," https://www.flaimsystems.com/flaim-trainer/.
basic CPs, identify what aspects of realism contributing to the [19] R. Schroeder, A. Steed, A.-S. Axelsson, I. Heldal, Å. Abelin, J.
support for the CPs, the experienced fidelity and presence and to Wideström, A. Nilsson, and M. Slater, “Collaborating in networked
explore how this impacts training value. immersive spaces: as good as being there together?,” Computers &
Graphics, vol. 25, no. 5, pp. 781-788, 2001.
ACKNOWLEDGMENT [20] M. Slater, A. Sadagic, M. Usoh, and R. Schroeder, “Small-group behavior
in a virtual and real environment: A comparative study,” Presence:
We would like to thank the Fire and Rescue Service Östra Teleoperators & Virtual Environments, vol. 9, no. 1, pp. 37-51, 2000.
Skaraborg, the participants and the coordinator, Mr. Pär [21] M. Sommer, G. S. Braut, and O. Njå, “A model for learning in emergency
Hagbohm, and Mr. Sune Fankvist, for assistance in the study. response work,” International Journal of Emergency Management, vol. 9,
pp. 151, 2013.
REFERENCES
[1] D. Checa, and A. Bustillo, “A review of immersive virtual reality serious
games to enhance learning and training,” Multimedia Tools and
Applications, vol. 79, no. 9, pp. 5501-5527, 2020.
[2] I. Heldal, “The usability of collaborative virtual environments: Towards
an evaluation framework,” Ph.D thesis vid Chalmers Technical
University, 2004.
[3] D. Harris, M. Wilson, and S. Vine, “Development and validation of a
simulation workload measure: the simulation task load index (SIM-
TLX),” Virtual Reality, vol. 24, no. 4, pp. 557-566, 2020.
[4] I Horváth, Borbála Berki, Anna Sudár, Péter Baranyi, “Definition,
Background and Research Perspectives Behind ‘Cognitive Aspects of
Virtual Reality’ (cVR),” Journal of Cognitive Infocommunications, vol.
1, no. 1, pp. 1-6, 2022.
[5] H. Wijkmark, Cecilia, I. Heldal, and M. M. Metallinou, “Can Remote
Virtual Simulation Improve Practice-Based Training?–Presence and
performance in incident commander education,” PRESENCE: Virtual and
Augmented Reality, pp. 1-57, 2021.
[6] F. M. Williams-Bell, B. M. Murphy, B. Kapralos, A. Hogue, and E. J.
Weckman, “Using Serious Games and Virtual Simulation for Training in
the Fire Service: A Review,” Fire Technology, vol. 51, no. 3, pp. 553-584,
May, 2015.
[7] C. H. Wijkmark, I. Heldal, and M.-M. Metallinou, "Experiencing
Immersive VR Simulation for Firefighter Skills Training."
[8] H. Wingfors, J. R. Nyholm, R. Magnusson, and C. H. Wijkmark, “Impact
of fire suit ensembles on firefighter PAH exposures as assessed by skin
deposition and urinary biomarkers,” Annals of work exposures and health,
vol. 62, no. 2, pp. 221-231, 2018.
[9] J. D. Bransford, A. L. Brown, and R. R. Cocking, "How people learn:
Brain, mind, experience, and school."
[10] Y. Wang, Y. Wang, S. Patel, and D. Patel, “A layered reference model of
the brain (LRMB),” Systems, Man, and Cybernetics, Part C: Applications
and Reviews, IEEE Transactions on, pp. 124-133, 04/01, 2006.
[11] M. R. Endsley, “Design and Evaluation for Situation Awareness
Enhancement,” Proceedings of the Human Factors Society Annual
Meeting, vol. 32, no. 2, pp. 97-101, 1988/10/01, 1988.
000032
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.
IEEE 1st International Conference on Cognitive Aspects of Virtual Reality • CVR 2022 • 11-12 May 2022, Online
000033
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.
C. Hammar Wijkmark et al. • Virtual Reality Support of Cognitive Processes in Firefighter Skills Training
000034
Authorized licensed use limited to: Viettel Group. Downloaded on June 05,2023 at 06:10:17 UTC from IEEE Xplore. Restrictions apply.