Professional Documents
Culture Documents
net/publication/359459606
Driving Simulator for Drivers Education with Artificial Intelligence Traffic and
Virtual Reality: a Review
CITATIONS READS
2 952
4 authors, including:
SEE PROFILE
All content following this page was uploaded by Ji-Jian Chin on 23 June 2022.
Abstract Simulators are games that give a sense of realism and simulate of the real
world in a virtual world, such as racing games or serious medical training games.
These have been widely documented to have positive effects on the specific techni-
cal skills training on their respective field of users. However, in the area of simula-
tors for driving education, there has been a lack of a comprehensive survey which
encompasses the various contributions in this area. In this paper, we fill that gap by
providing a review of previous work in literature that focuses on driver’s education
using simulators. In addition, we include also works on Artifical Intelligence (AI)
and Computational Intelligence (CI) in games that could be used to enhance the
project and the effects of implementation of virtual reality in such studies.
1 Introduction
Simulators have been in existence for quite awhile. These computer programs pro-
vide a platform for simulating the real world in a virtual one where mistakes can be
forgiven and learning can be encouraged. These platform have been used in appli-
cations such as racing simulators and medical training simulators to teach and train
user skills that are specific to the field. With the technology of virtual reality, these
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 483
R. Alfred and Y. Lim (eds.), Proceedings of the 8th International Conference
on Computational Science and Technology, Lecture Notes in Electrical
Engineering 835, https://doi.org/10.1007/978-981-16-8515-6_38
484 Y.-Q. Cheng et al.
platforms now can be further enhanced to provide users with immersive learning
where the user is immersed in the simulated environment to enhance the learning
effectiveness.
Simulators such as racing sims and medical training serious games have been
studied and widely documented in [4]. For example, Kandhai et al.’s paper [13]
presented an analysis on Immersive Driving Simulation for Driving Education. The
authors reported positive results on favourable learning experiences by the users in
their early testing. Chan et al. [5] also argued that driving simulators can be effective
tools for evaluating novice drivers in hazard anticipation, speed management and
attention maintenance. Finally, Burkhardt et al. mentioned in his paper [4], to a lesser
extend of the simulators and itself, VR have been documented copiously for research
purpose in road safety and (to a less extend) driving education. These technologies
however have not received as much attention in the training and learning purposes
of the simulators in driving schools [4]. Also, as the publication years suggest, these
works have not received recent attention.
For these driving simulators, they will need to be simulated traffic to simulate a
realistic virtual traffic environment. Artificial Intelligence (AI) and Computational
Intelligence (CI) have been linked to computer games since the first program where it
was designed to play chess [24, 29]. In modern computer games, AI and CI have been
uniquely designed for their specific purposes in their games. In this review paper, we
mainly focus on racing games, simulators and open world games that have simulated
traffic. AI and CI in racing are not just for running laps in the track mindlessly. It is
there to give challenge and immersion to the user. In some cases with modern open
world racing games and games, it could be an integral part of making the virtual
world come alive. Traffic in games like the Forza Horizon 4, Cyberpunk 2077 and
Grand Theft Auto V give the world a sense of liveliness and realism.
In the context of driving simulator, an adaptive AI traffic could provide the user
with realism and challenges as the AI technique might act randomly to interfere with
the user when he is driving. As mentioned by Aiolli et al. [2], an AI that can adapt to
the user’s behaviour and can react with more unpredictable behaviours. MacKinnon
et al. [15] also found that adaptive AI gave users a more challenging and enjoyable
experience. These factors suggest that AI in a driving simulator can be adaptive to
provide certain challenges to the user.
Virtual Reality (VR) has been in trend recently with simulators such as Microsoft’s
Flight Simulator. But according to Cipresso et al. [7], the technology has been in
existence for quite some time, formulated during the 1960s with the first commercial
VR tools appearing in the late 1980s. VR has attracted the interest of investors and the
general public after Mark Zuckerberg purchased Oculus. Now big corporations such
as Samsung, Sony, HTC, and Google are investing heavily into VR/AR technologies.
With the recent emergence of consumer focused VR technologies like HTC VIVE,
Sony PlayStation VR and the Oculus Rift, VR is attracting attention of users and
researchers alike, suggesting it may be the next stepping stone in technological
innovation [7]. Gutierrez et al. [18] mentioned in his paper that it is sound to take
advantage of these technologies to facilitate learning with the increased affordability
Driving Simulator for Drivers Education with Artificial … 485
and accessibility of VR/AR technologies. This suggests that more fields of education
could be experimented with VR/AR.
In this article, we set out to achieve two objectives: first we review existing work in
literature that focuses on driver’s education using simulators, AI and CI in games that
could be used in the projects. Secondly we discuss the effects of VR towards filling
the gap that we found to be available for further studies. It also serve as a call for
using available technology to improve user learning engagement and effectiveness,
while improving the safety, controllability and repeatability of driver education. We
note that prior to this work, to the best of our knowledge no such work has been done
to consolidate existing studies done in these three main areas of driving simulation,
AI and CI as well as VR. This article is also intended to provide a rough guideline
on designing the requirements of a driving simulator for driving education studies.
The major contributions are summarised as below.
1. We reviewed papers that analysed the effects of simulators for training and edu-
cation and found a gap in their use in driving education studies.
2. We surveyed AI and CI techniques used in racing games that can help create an
environment with simulated traffic that can mimic real world randomness and
provide the user with an immersive experience.
3. We also examined papers that study VR and their effects on improving skill
training, hazard perception and learning effectiveness. In these VR-related papers,
we found that there is also a gap in driving education studies, while exploring the
possible use to improve learning effectiveness in driving education.
This article explored ten papers related to driving simulators used for driving edu-
cation: five papers regarding AI and CI techniques in games to enhance experiences
and five papers regarding VR and its effect in educational and training uses. All of
the papers are published between 2005 and 2021.
The organisation of this article is as follows. Section 2 presents the literature
review of driving simulators use for driving education and serious games. Section 3
discusses about the AI and CI techniques used in games, including its methods used
and how would it help with our upcoming project. Effects of virtual reality used in
education will be subsequently discussed in Sect. 4. Section 5 will be the conclusion
of the article.
2 Driving Simulators
We arrange our review in accordance to chronology, with one exception of the final
article which focuses on drivers’ education for the Malaysian road system. In this
section, we reviewed papers regarding driving simulators and serious games on their
effects on education. Exploring the papers to see if there is really a gap in drivers
education in relation to driving simulators.
A study did by Wade et al. [19] in 2005 highlighting the effects of Driving Simula-
tor Fidelity on over 500 novice drivers accompanied by driver education instructors
486 Y.-Q. Cheng et al.
who have no prior experience in running human subject experiments. Three types of
setups with different fidelity, from least to highest, was used to run against training
groups that did Literature Training and among themselves [19]. In their results, we
focused solely on the results of different fidelity of the simulators used and discover
that greater fidelity produce better results in driving training.
The issue here is that this paper was published in 2005. Computer graphics fidelity
and the commercialisation of VR technology is getting better and more common in
2021 compared to 2005. Hence, with the technology now, we can improve further
on the fidelity of the driving simulator and in turn, improve the results of our project.
Fisher et al. designed a program that is pc-based to train novice drivers’ hazard
detection. The authors’ method was a pc training session where it has 3 sections, a
pre-test, training and post-test [10]. The program will ask their users to manually
click on and identify the hazard or where to be aware in the situation. The users are
supposed to look at the scenario photo and drag circles indicating where they should
monitor [10] and where potential hazards can be [10]. After the pre-test the users
were required to go to a simulator to be trained for risk awareness. Then they will take
another post-test to evaluate their learnings [10]. We have decided to take inspiration
from their methods of designing pre-programmed scenarios to train our user’s risk
awareness but will incorporate the scenarios in the simulator itself including the
evaluation. Their result shows reduced likelihood of crashes proving novice drivers
can be trained to gain information and reducing their likelihood to crash. It also
indicates possible improvement if driving simulators were used for training.
Wang et al. [28] paper concludes that driving simulator is slowly becoming an
important approach for driving safety studies due to its repeatability and controlla-
bility of experiments. It is also safer and have lower cost [28]. The authors signify
that the masses can benefit from the widespread consumer use of mid-level and
entry-level sims as computer technology progresses [28]. In modern times (2021),
the Covid-19 Pandemic had pushed entry-level sims in trend as cost to own is low.
Signifying that the use of low-cost simulators for driving education is viable.
Backlund et al.’s paper [3] concluded that there is a positive link between specific
skill-oriented aspects of driving and gaming. The authors did this by constructing a
simulator with off-the-shelf hardware running on an open-source software [3]. Data
was collected from 70 participants and analysed to study the possible individual
learning effects and it came out positive [3]. They showed a simulation based on
game is viable to be employed as a catalyst to enhance various aspects of learning in
traffic safety context and driving education [3]. Kandhai et al. [13] did their analysis
on immersive driving simulation for driver’s education by putting the users into a
predefined starting point in the simulator they built and asking the users to follow
the voice instructions en route. The faults were recorded and compared with users’
simulation experience report. The results showed that users commit more faults
than they perceive, suggesting users can improve using the simulator [13] while
reinforcing Backlund et al.’s statement [3].
Underwood et al. [27] assessed the comparability between driving on the road and
in a simulator and chose to evaluate [13] simulator. The authors compared the sim-
ulators hazard detection while watching recorded short clips from a vehicle moving
Driving Simulator for Drivers Education with Artificial … 487
for their drivers’ education. Another factor in exploiting the potential of driving sim-
ulators for drivers’ education, the driving simulator need to have a balanced mix of
entertainment and specific educational content to get users actively engaged in the
simulator.
player, instead it was to give the player an enjoyable and challenging experience.
The AI is incorporated with AI techniques that enables the cars to be controlled in
a manner that mimics natural driving [6]. The AI is designed with the combination
of Waypoint systems with Vector Calculation, Conditional Monitoring System and
Artificial Environment Perception [6]. These methods enable the AI to find where
they should go, traverse the track with natural steering and throttle/brake inputs and
be able to determine their surroundings and avoid obstacles such as walls. With the
AI techniques, we believe that it will provide the player with a sense of realism
in the driving simulator. The authors mentioned Unity has native support for the
development effort and reduces the implementation effort of the simulator game. It
is good news, as Unity is our platform of choice for our project.
Tomlinson et al. discussed about the AI architecture for racing games, touching on
racing behaviours. In this section, the authors mentioned that most realistic games,
AI can make occasional mistakes and recover from their mistakes [25]. As driving
simulators are not focused on racing, but the mistake and recovery design can help
make the experience of driving simulator more engaging and realistic in regards to
natural driving AI.
In our project, we wanted the AIs to know how to maneuver out of obstacles
and to prevent them crashing into obstacles all the time. We can refer Muhammad
Salman paper’s [23] look ahead, and overtaking methods. In his paper, his look
ahead method was used for path finding and obstacle avoidance. In his use case, the
lookahead points position on the track were determined by a general formula. The
AI will use the lookahead points to “lookahead” for obstacles and track scanning to
manoeuvre itself. As for his overtaking method, he uses a few inputs of information
to determine the overtake. The AI would first determine which opponent to overtake,
when the AI got close, a negative potential field indicating a crash was given. The
AI then made adjustments to the steer value and speed while adhering to the track
and cornering limits to make an overtake move. His paper concluded by stating that
his implementation has achieved the task of collision detection and overtaking while
driving in a car racing game.
In summary, driving simulators should be engaging and entertaining to get users
motivated and actively engaged in learning. AI and CI techniques discussed here can
provide the users with a simulated environment that is entertaining, challenging and
lively. The difficulty here is the implementations as all papers here have different
systems with different implementations, developing from scratch would require a
significant amount of time. We also believe that we may found a gap in using adaptive
AI techniques in driving simulators for driving education that could be filled with
more experimentation in terms of giving a real world randomness in driving behaviour
of the simulated traffic.
490 Y.-Q. Cheng et al.
4 Virtual Reality
The findings of the authors proves that VR can be an efficient way in improving
the learners’ level of anatomy knowledge [30]. In accordance with [19] the fidelity
of a driving simulator will produce a better result, VR can provide the users with
better fidelity than screens. VR has been confirmed to have an impact and would be
beneficial to include it in our project.
Masoumzadeh et al. [16] did a pilot study on improving spatial cognition in older
adults. The pilot study was done on older patients with varying degrees of dementia.
The experiment uses an in house designed virtual reality driving simulator and a
sample size of 10 participants. The author’s result shows that with repeated practice
with their designed virtual reality driving simulator, it has positive effects on older
adults’ spatial cognition and giving the patients an uplifting experience even on those
with different degrees of dementia. This gave the project a motivation to incorporate
VR into the project as it has been shown to help with older adults, it could benefit
the younger adults as well.
Dobrowolski et al. [8] done an experiment on learning transfer effects of complex
skill after training in younger and older adults with Immersive VR. The methods were
splitting 118 participants into 2 training groups: the non-interactive training group,
which uses a manual and training videos and the IVR training group which uses a
virtual simulation of the real-world task. The authors concluded that IVR training
of a complex skill can be transferred into a real-world analog, and is vastly more
effective than text-based and video methods. It is also equally beneficial to both age
ranges of adults [8].
A systemic review done by Renganayagalu et al. [22] recently concluded that VR
is considered as a technology with the potential to enhance learning processes with
trainees due to the ability of being able to provide experiential learning with elevated
levels of interaction and encourage active learner engagement. The review identified
training in these fields with VR is highly valuable to are such as aerospace, defence,
industrial, health care, safety and emergency preparedness training, indicating there
is a gap in drivers education. The authors mentioned that more rigorous research
are needed to examine the most promising uses of VR-Head Mounted Displays (VR
HMDs) in authentic training context [22] which gave motivation to our project.
Summarising this section, the VR papers we reviewed here are all quite recent,
indicating a boom of studies in recent years. There is still a gap in studies related
to drivers’ education despite the boom. The difficulty that we can possibly see is
motion sickness in using the VR HMDs. In general, we found a positive outlook on
the effects of using VR to boost learning effectiveness regardless of age as in [16]
paper was done with older adults and in [1] is done with younger adults. Hence we
came with the conclusion that including an immersive experience with VR or VR
headsets can positively affect the learning effectiveness of the driving simulator.
492 Y.-Q. Cheng et al.
5 Conclusion
In conclusion, we reviewed various papers and affirm that there is an area for appli-
cation in drivers’ education with driving simulators, and the use of Virtual Reality in
drivers’ education. To design a driving simulator for driving education, we found that
the entertainment factor is as important as the educational, and by using AI and CI
techniques used in racing games, a challenging and unpredictable simulated traffic
can be designed. In relation to all the sections, we conclude that driving simulators
that uses AI simulated traffic with VR fidelity for driving education can and should
further developed as the potential of using these technologies to improve learning
effectiveness seems promising. Future works’ direction should include attempts in
developing a driving simulator that has the mentioned qualities to be tested on human
subjects.
Acknowledgements The authors would like to thank the Ministry of Education of Malaysia
in providing financial support for this work through the Fundamental Research Grant Scheme
(FRGS/1/2020/SS0/MMU/02/1)
References
1. Agrawal, R., Knodler, M., Fisher, D.L., Samuel, S.: Virtual reality headset training: Can it be
used to improve young drivers’ latent hazard anticipation and mitigation skills. Transportation
Research Record 2672(33), 20–30 (2018). DOI 10.1177/0361198118758311
2. Aiolli F, Palazzi CE (2008) Enhancing artificial intelligence in games by learning the opponent’s
playing style. In: Ciancarini P, Nakatsu R, Rauterberg M, Roccetti M (eds) New frontiers for
entertainment computing. Springer, US, Boston, MA, pp 1–10
3. Backlund P, Engström H, Johannesson M, Lebram M (2010) Games for traffic education:
an experimental study of a game-based driving simulator. Simul Gaming 41:145–169. DOI:
10.1177/1046878107311455.
4. Burkhardt JM, Corneloup V, Garbay C, Bourrier Y, Jambon F, Luengo V, Job A, Cabon P, Benab-
bou A, Lourdeaux D (2016) Simulation and virtual reality-based learning of non-technical skills
in driving: critical situations, diagnostic and adaptation. IFAC-PapersOnLine 49(32):66–71.
https://doi.org/10.1016/j.ifacol.2016.12.191, cyber-Physical & Human-Systems CPHS 2016
5. Chan E, Pradhan A, Pollatsek A, Knodler M, Fisher D (2010) Are driving simulators effec-
tive tools for evaluating novice drivers’ hazard anticipation, speed management, and attention
maintenance skills. Transp Res Part F Traffic Psychol Behav 13:343–353. https://doi.org/10.
1016/j.trf.2010.04.001
6. Chan MT, Chan CW, Gelowitz C (2015) Development of a car racing simulator game using
artificial intelligence techniques. Int J Comput Games Technol 2015:839721. https://doi.org/
10.1155/2015/839721
7. Cipresso P, Giglioli IAC, Raya MA, Riva G (2018) The past, present, and future of virtual
and augmented reality research: A network and cluster analysis of the literature. Front Psychol
9:2086–2086. https://doi.org/10.3389/fpsyg.2018.02086, 30459681[pmid]
8. Dobrowolski P, Skorko M, Pochwatko G, Myśliwiec M, Grabowski A (2021) Immersive virtual
reality and complex skill learning: Transfer effects after training in younger and older adults.
Front Virtual Reality 1:40. https://doi.org/10.3389/frvir.2020.604008
Driving Simulator for Drivers Education with Artificial … 493
9. Erhel S, Jamet E (2013) Digital game-based learning: Impact of instructions and feedback
on motivation and learning effectiveness. Comput Educ 67:156–167. https://doi.org/10.1016/
j.compedu.2013.02.019
10. Fisher D, Pollatsek A, Pradhan A (2006) Can novice drivers be trained to scan for information
that will reduce their likelihood of a crash? Injury Prev J Int Soc Child Adolesc Injury Prev
12(Suppl 1):i25–9. https://doi.org/10.1136/ip.2006.012021
11. Guillén Nieto V, Alesón-Carbonell M (2012) Serious games and learning effectiveness: the
case of it’s a deal! Comput Educ 58:435–448. https://doi.org/10.1016/j.compedu.2011.07.015
12. Ismail R, Md Saad MH, Mohd Nor J, Rosli R (2016) Driver’s readiness to use car driving
simulator as a tool to improve driving skill: a preliminary study. Jurnal Teknologi 78, 137–142.
https://doi.org/10.11113/jt.v78.9161
13. Kandhai K, Smith M, Kanneh A (2011) Immersive driving simulation for driver education and
analysis. In: 2011 16th International conference on computer games (CGAMES), pp 288–292
14. Libâneo JC, Alves N (2017) Temas de pedagogia: Diálogios entre didática e currículo. Cortez
Editora
15. MacKinnon B (2012) Adaptive ai in a racing simulator: a challenging and fun environment for
any user. Comput Games J 1(1):71–89. https://doi.org/10.1007/BF03392329
16. Masoumzadeh, S., Moussavi, Z.: Does practicing with a virtual reality driving simula-
tor improve spatial cognition in older adults? a pilot study. Neuroscience Insights 15,
2633105520967930 (2020). DOI 10.1177/2633105520967930
17. Mazer B, Gélinas I, Duquette J, Vanier M, Rainville C, Chilingaryan G (2015) A randomized
clinical trial to determine effectiveness of driving simulator retraining on the driving perfor-
mance of clients with neurological impairment. Br J Occup Ther 78. https://doi.org/10.1177/
0308022614562401
18. Mora, C.E., Martín-Gutiérrez, J., Añorbe-Díaz, B., González-Marrero, A.: Virtual technologies
trends in education. EURASIA Journal of Mathematics, Science and Technology Education
13(2), 469–486 (2017). DOI 10.12973/eurasia.2017.00626a
19. Park G, Allen R, Rosenthal T, Fiorentino D (2005) Training effectiveness: how does driv-
ing simulator fidelity influence driver performance? In: Proceedings of the human factors
and ergonomics society annual meeting, vol 49, pp 2201–2205. https://doi.org/10.1177/
154193120504902518
20. Queiroz ACM, Nascimento AM, Tori R, da Silva Leme MI (2019) Immersive virtual environ-
ments and learning assessments. In: Beck D, Peña-Rios A, Ogle T, Economou D, Mentzelopou-
los M, Morgado L, Eckhardt C, Pirker J, Koitz-Hristov R, Richter J, Gütl C, Gardner M (eds)
Immersive learning research network. Springer International Publishing, Cham, pp 172–181
21. Rabin SE (2014) A rubber-banding system for gameplay and race management, pp 504–512.
https://doi.org/10.1201/b16725
22. Renganayagalu SK, Mallam SC, Nazir S (2021) Effectiveness of VR head mounted displays in
professional training: a systematic review. In: Technology, Knowledge and Learning. https://
doi.org/10.1007/s10758-020-09489-9
23. Salman M (2013) Collision detection and overtaking using artificial potential fields in car racing
game TORCS using multi-agent based architecture. Master’s thesis, , Department of Computer
Science and Engineering
24. Shannon CE (1950) Xxii. programming a computer for playing chess. London Edinburgh
Dublin Philos Mag J Sci 41(314):256–275. https://doi.org/10.1080/14786445008521796
25. Tomlinson S, Melder N (2015) An architecture overview for AI n racing games
26. Uhr M, Felix D, Williams B, Krueger H (2020) Transfer of training in an advanced driving
simulator - comparison between real world environment and simulation in a manoeuvring
driving task
27. Underwood G, Crundall D, Chapman P (2011) Driving simulator validation with hazard per-
ception. Transp Res Part F Traffic Psychol Behav 14:435–446. https://doi.org/10.1016/j.trf.
2011.04.008
28. Wang Y, Zhang W, Wu S, Guo Y (2007) Simulators for driving safety study—a literature
review. In: Shumaker R (ed) Virtual reality. Springer, Berlin Heidelberg, Berlin, Heidelberg,
pp 584–593
494 Y.-Q. Cheng et al.
29. Westera W, Prada R, Mascarenhas S, Santos PA, Dias J, Guimarães M, Georgiadis K, Nyam-
suren E, Bahreini K, Yumak Z, Christyowidiasmoro C, Dascalu M, Gutu-Robu G, Ruseti S
(2020) Artificial intelligence moving serious gaming: presenting reusable game AI compo-
nents. Educ Inf Technol 25(1):351–380. https://doi.org/10.1007/s10639-019-09968-2
30. Zhao J, Xu X, Jiang H, Ding Y (2020) The effectiveness of virtual reality-based technol-
ogy on anatomy teaching: a meta-analysis of randomized controlled studies. BMC Med Educ
20(1):127. https://doi.org/10.1186/s12909-020-1994-z