1
A Survey of Simulators for Autonomous Driving:
Taxonomy, Challenges, and Evaluation Metrics
Yueyuan Li, Wei Yuan, Weihao Yan, Qiyuan Shen, Chunxiang Wang, and Ming Yang
Abstract—Simulators have irreplaceable importance for the
research and development of autonomous driving. Besides saving
resources, labor, and time, simulation is the only feasible way
arXiv:2311.11056v1 [cs.RO] 18 Nov 2023
to reproduce many severe accident scenarios. Despite their
widespread adoption across academia and industry, there is an
absence in the evolutionary trajectory of simulators and critical
discourse on their limitations.
To bridge the gap in research, this paper conducts an in-
depth review of simulators for autonomous driving. It delineates
the three-decade development into three stages: specialized de-
velopment period, gap period, and comprehensive development,
from which it detects a trend of implementing comprehensive
functionalities and open-source accessibility. Then it classifies the
simulators by functions, identifying five categories: traffic flow
simulator, vehicle dynamics simulator, scenario editor, sensory
data generator, and driving strategy validator. Simulators that
amalgamate diverse features are defined as comprehensive simu-
lators. By investigating commercial and open-source simulators,
this paper reveals that the critical issues faced by simulators
primarily revolve around fidelity and efficiency concerns. This
paper justifies that enhancing the realism of adverse weather
simulation, automated map reconstruction, and interactive traffic
participants will bolster credibility. Concurrently, headless sim-
ulation and multiple-speed simulation techniques will exploit the
theoretic advantages. Moreover, this paper delves into potential
solutions for the identified issues. It explores qualitative and
quantitative evaluation metrics to assess the simulator’s per-
formance. This paper guides users to find suitable simulators
efficiently and provides instructive suggestions for developers to
improve simulator efficacy purposefully.
Index Terms—Autonomous Driving, Simulator, Testing.
I. I NTRODUCTION
Simulators play a crucial role in the field of autonomous
driving. According to Kalra et al., validating the performance
of an autonomous vehicle with statistical significance would
require an impractical 215 billion miles of real-world testing
[1]. With simulators, this process can speed up with lower
resource costs. Simulators offer the ability to generate raw
sensor data and ground truth under adverse weather condi-
tions, which helps to enhance the robustness of perception
algorithms [2]. They also enable the simulation of complex
traffic scenarios, including severe accidents, which aids in
This work is supported by the National Natural Science Foundation of
China (62173228). Ming Yang is the corresponding author.
Yueyuan Li, Weihao Yan, Qiyuan Shen, Chunxiang Wang, and Ming Yang
are with the Department of Automation, Shanghai Jiao Tong University,
Key Laboratory of System Control and Information Processing, Ministry of
Education of China, Shanghai, 200240, CN (email: MingYANG@sjtu.edu.cn).
Wei Yuan is with the Innovation Center of Intelligent Connected Vehicles,
Global Institute of Future Technology, Shanghai Jiao Tong University, Shang-
hai, 200240, CN
the improvement of driving policy and route planning models
[3]. Moreover, simulators facilitate testing the accuracy and
stability of control methods by simulating the dynamics behav-
ior of diverse vehicle types under various physics conditions
[4]. Additionally, simulators find applications in tasks such
as vehicle cooperation and trajectory prediction related to
autonomous driving [5].
The simulators are widely adopted in academia and indus-
trial communities. In 2022, nearly half of the published meth-
ods related to autonomous driving in IEEE Transactions on
Intelligent Transportation Systems were verified by simulators
[6]. Leading technology companies in the autonomous driving
industry, such as Waymo, Pony.ai, and Didi, emphasize the
development and application of simulators [7, 8, 9]. Waymo’s
impressive performance, as reflected in the Disengagement
Report from California between 2020 and 2023 [10], serves as
compelling evidence of the critical role that realistic simulators
play in achieving outstanding results [7].
A high-quality open-source simulator, which grants users
access to the executable program and source code, holds
significant advantages for both academic and industrial com-
munities. Firstly, it lowers barriers for individual researchers
and small institutions, who often find real-world testing unaf-
fordable and creating a simulation platform time-consuming.
With a free and open-source simulator, more researchers are
encouraged to engage in the development of autonomous
driving technologies, fostering innovation and collaboration
within the field. Secondly, an ideal open-source simulator
serves as a valuable platform for data sharing. This enables
researchers to exchange data related to vehicles, sensors,
and traffic scenarios, leading to substantial time and cost
savings in data collection efforts. Additionally, the diversity
of shared data contributes to comprehensive improvements in
autonomous driving systems. Furthermore, data sharing within
the simulator ecosystem can promote the establishment of
standards in data format and communication protocols. Rec-
ognizing the importance of public simulators, influential insti-
tutions like Computer Vision Center (CVC) and Baidu have
developed simulators such as CARLA and Apollo [11, 12].
These initiatives reflect the growing acknowledgment of the
value of open-source simulators for autonomous driving.
The history of traffic flow simulators serves as a valuable
reference, highlighting the evident shift towards open-source
simulators as the future trend. Initially, most traffic flow sim-
ulators, such as Paramics and Aimsun, were commercial soft-
ware [13, 14]. However, with the decreasing cost of software
development, the open-source simulator SUMO emerged and
steadily gained popularity, ultimately becoming the preferred

choice for traffic flow simulations [15], while Paramics and
Aimsun faced the challenge of acquisition. Advancements in
key techniques required for simulator development, such as
3D modeling, graphical rendering, and parallel computing,
have become more accessible and affordable. The cost of
developing simulators is expected to decrease, leading to a
decline in simulator license prices. As a result, users will
naturally gravitate towards more economical choices, namely,
open-source simulators.
This paper serves as a comprehensive guide for researchers
in the field of autonomous driving, assisting them in selecting
suitable simulators for their specific tasks. It emphasizes
the current limitations of simulators, alerting researchers and
developers of the potential factors that may defect the validity
of simulation-based experiments. This paper further offers
insightful suggestions to the developers of open-source sim-
ulators, directing their attention toward critical issues worthy
of addressing. The contribution of this paper is fourfold:
1) It presents a comprehensive overview of the evolution
of simulators over the past thirty years and predicts the
likely development tendencies in the future.
2) It categorizes the simulators based on their utility. Spe-
cial attention is given to identifying free and open-source
simulators for potential users.
3) It identifies and defines the critical issues in current
simulators from a developer’s perspective. It further in-
vestigates the potential solutions to address these issues.
4) It reviews the evaluation metrics to measure the effi-
ciency and fidelity of a simulator, which is organized
by the mechanism of algorithms.
II. R ELATED W ORKS
The limited number of reviews on simulators for au-
tonomous driving is primarily due to the lack of simulators.
Most surveys and reviews in the field of autonomous driving
have focused on other aspects, such as tasks, algorithms,
and sensors. Though these reviews briefly mention the use
of simulators in various scenarios, they often do not exten-
sively cover the topic [16, 17, 18]. For example, Yurtsever
et al. demonstrated the capability of simulators to generate
diverse data that captures different weather conditions [19].
This data augmentation method enhances the robustness of
perception models, enabling them to perform effectively in
a wide range of scenarios [20]. Similarly, Grigorescu et al.
emphasized the importance of simulation for deep learning-
based driving decision-making models, as these models require
diverse interaction behavior to improve their decision-making
capabilities [21]. This viewpoint is echoed in other reviews that
focus on decision-making algorithms [22, 23, 24, 25]. What’s
more, Tampuu et al. expressed concerns about the fidelity of
simulators, highlighting the challenges of transferring decision
models trained in virtual environments to the real world
[26]. While these reviews provide strong evidence of the
widespread application of simulators across various tasks, it is
worth noting that most of these reviews merely list alternative
simulators without delving into their quality [27].
It was not until 2020, with the increasing number of public
simulators, that reviews and discussions focusing on simulators
2
began to emerge. Zhou et al. conducted a detailed survey
of seven popular simulators, including CARLA, LGSVL,
CarSim, AirSim, Prescan, Matlab/Simulink, and CarMaker.
Their work explored the functionalities of these simulators and
summarized their respective application scopes [28]. Holen et
al. conducted a similar study but expanded the range of simu-
lators, offering a broader perspective on the available options
[29]. The simulators mentioned in Kaur et al.’s research were
mostly consistent with Zhou’s work, with the main difference
being the replacement of AirSim with Gazebo in their survey.
This survey qualitatively describes the performance of the
simulators, along with a summary of the desired functionalities
and characteristics of simulators [30]. The existing reviews
tended to assist researchers to choose simulators, so their
main content is to list the available alternatives and check the
detailed functions. There remains a need for more comprehen-
sive and in-depth reviews to discuss simulators’ performance,
limitations, and potential directions for improvement.
Over the past decade, discussions regarding critical issues
in simulators have been scattered, with various researchers
highlighting specific challenges. Hu et al. stressed the need
to narrow the gap between simulators and the real world
by automating the reconstruction of simulation scenarios and
improving the sim2real model transfer efficiency [31]. Mean-
while, Zhang et al. identified that enhancing the intelligence of
traffic participants’ behaviors is a promising avenue to boost
simulator performance [3]. Similarly, Siebke et al. emphasized
the importance of including human errors in the traffic sim-
ulation process to enhance realism [32]. While these papers
succeeded in justifying the importance and urgency of these
issues, they often fell short in relating the existing technology
to potential solutions and providing compelling evaluation
metrics to measure the effectiveness of proposed resolutions.
Consequently, the development community has not agreed
on the simulators’ insufficiency, leading to unorganized code
development and aimless release of new simulators.
This paper aims to address this gap in the review of
simulators. In addition to investigating simulators’ history,
category, and available items, this paper will analyze and
connect existing technologies to viable resolutions, providing
a clear roadmap for developers to focus their efforts and
create more effective and efficient simulators. By proposing
well-defined evaluation metrics, this paper seeks to establish
a standardized way to measure the performance and quality
of simulators. By filling this void in the literature, this paper
benefits researchers, developers, and the autonomous driving
community as a whole.
III. P ROBLEM R EFINEMENT
Defining the scope of discussion is crucial to ensuring a
clear and focused review. In the context of this paper, the
term simulator refers to software tools that replicate the real
world with sufficient detail for developing functions within
autonomous driving systems. The scope excludes simulators
integrated with hardware that aim to mimic a realistic driving
experience by technology like digital twins. Considering the
substantial engineering effort required to develop usable sim-
ulators, the number of mature and readily available options is

limited. Hence, this review will not be restricted to comprehen-
sive simulators capable of supporting multiple research topics
across various areas such as perception, planning, and control.
Instead, it will encompass various specific-purpose simulators.
Since commercial simulators constitute a significant portion
of available options and hold a dominant position in some
categories, they will be included in the review of history and
taxonomy. However, due to commercial copyright restrictions,
performance evaluations of these commercial simulators will
not be provided in this paper. Instead, the focus will be on
free and open-source simulators when discussing evaluation
metrics, performance, and existing issues.
IV. H ISTORY OF S IMULATORS
The history of simulators for autonomous driving provides
valuable insights into the technological evolution within this
domain. Over the past three decades, the development of sim-
ulators has progressed through distinct stages: the specialized
development period, the gap period, and the comprehensive
development period. Two noteworthy trends have emerged
in the development of simulators. Firstly, there is a notable
expansion of simulator capabilities, enabling the integration of
multiple functions on a single platform. This versatility allows
for the efficient development of diverse tasks within the same
simulator environment. Secondly, the idea of free and open-
source simulators has grown popular. An increasing number
of companies and organizations are contributing open-source
software to establish their influence on evaluation benchmarks.
The period before 2002 is described as the specialized
development period due to the focus of simulators on singular
objectives. Most of the simulators worked on two topics, traffic
flow, and vehicle dynamics, respectively. Traffic flow simu-
lation proved instrumental in road planning and optimizing
traffic efficiency, while vehicle dynamics simulation facilitated
the assessment of component adjustments on vehicle physics
behavior. The maturity of relevant theories and technologies
during this era significantly contributed to the emergence of
traffic flow and vehicle dynamics simulators. These simulators
were built upon explicit rules and physics models, requiring
relatively modest computational resources. In 1992, PTV
Group introduced PTV Vissim for traffic flow and logistics
system simulations [33]. Later, in 1994, Paramics appeared to
offer similar functionalities [13]. Alongside these commercial
options, the open-source simulator SUMO played a critical
role in traffic flow simulation [34]. Concurrently, in the realm
of vehicle dynamics, CarSim and IPG CarMaker, established
during the 1990s, fostered enduring partnerships with nu-
merous vehicle manufacturers [35, 36]. Their prominence in
this domain was reinforced by the extensive real-world data
they accumulated. RFpro, which appeared a decade later in
2007, was one the few simulators boasting a vehicle dynamics
fidelity comparable to CarSim and IPG CarMaker, owing to
the real-world data collection from racing cars [37].
From 2003 to 2013, the development of simulators faced
significant challenges as related technologies encountered bot-
tlenecks. Primarily, computer hardware, especially graphics
cards, lacked the capability to handle large-scale rendering.
3
Additionally, software techniques such as rendering, object
modeling, and physics engine were under development. Con-
sequently, constructing a high-fidelity simulator was daunting
and costly. During that period, the academic community had
not recognized the potential of learning-based methods, so
there was no urgent demand for generating a large volume
of high-fidelity data. The combination of technological limi-
tations and absent motivations led to a ten-year gap. Only a
few simulators managed to survive and make a lasting impact
during the harsh period. Notable among them were PreScan,
supported by Siemens in terms of funding and hardware [38];
rFpro, which benefited from the financial backing and data
support of Formula 1 (F1) racing groups [37]; and VI-grade,
a comprehensive investment by HBK Investment [39]. These
simulators’ survival relied heavily on substantial investments
from world-leading profit organizations.
The resurgence of simulators began in 2013, and the focus
evolved from singular functionalities to the diverse needs of
multiple tasks. One influential factor was the release of the
video game GTA V in 2013, which served as an effective
tool for data generation owing to its advanced rendering
technology. Researchers leveraged this platform to collect
images under various weather conditions at a low cost [40].
The expansive city map of GTA V also attracted researchers
to employ it in driving decision-making models [41]. Despite
Take-Two Interactive, the developers of GTA V, rejecting
the autonomous driving interface in 2016, the application
demonstrated the immense potential of simulators. It showed
the advantages of simulators to replicate diverse weather
conditions, offering researchers the flexibility to study various
scenarios. Some subsequent 3D simulators emerged, aiming to
approximate the real world and provide diverse synthetic sen-
sor data, actively offering interfaces to access proxy vehicles
for validating driving decision algorithms [11, 42, 43]. In 2018,
Waymo utilized bird-eye view (BEV) semantic images to train
driving decision models, which demonstrated an impressive
performance in trajectory prediction with an easily accessible
input [44]. This approach encouraged the development of
2D BEV simulators such as Highway-env and CommonRoad,
tailored to specific simulation requirements and research ob-
jectives [45, 46]. These developments marked a significant
shift in the applications of simulators for autonomous driving.
Besides the shift in functionality, the history of simulators
also reflects a trend toward open-source. Fig. 1 illustrates
the changes in the number of open-source and closed-source
simulators over the past three decades. The data clearly
reveals a consistent growth of free and open-source simulators.
This trend can be attributed to two main factors. Firstly, the
technologies required for developing simulators have become
more accessible over time. The decreasing costs and increasing
availability of computing resources and software tools have
made it easier for developers to create and distribute open-
source simulators. Therefore, more researchers and devel-
opers are contributing to open-source simulators. Secondly,
the emergence of open-source simulators is beneficial for
the entire autonomous driving industry. These platforms can
promote the establishment of standardized data formats and
evaluation criteria, and further foster a more cohesive and

Fig. 1. The number of simulators growing by year. This statistical result includes the popular commercial simulators used by famous vehicle manufacturers
or technology companies and open-source simulators with over 100 stars on GitHub or more than 100 citations on paper.
efficient research environment, where findings can be better
shared and validated. These two advantages together propel
the growth of open-source simulators.
V. C ATEGORIZATION OF S IMULATORS
The development history of simulators reveals their origin
from integrated programs designed to automate the training
and testing processes for various functions within autonomous
driving systems. Therefore, it is natural to categorize the sim-
ulators based on their functionalities. Employing this frame-
work, simulators are classified into five specialized classes:
traffic flow simulator, vehicle dynamics simulator, scene editor,
sensory data generator, and driving strategy validator. When
a simulator can handle multiple tasks, it is positioned within
the sixth category, a comprehensive simulator.
TABLE I demonstrates statistics on the influential sim-
ulators. The table includes essential information about the
simulator’s name, type, first release date, last maintenance
time, developer, accessibility, and supported systems. The rows
are sorted according to the simulators’ categories, focusing on
their most representative features, even as they may expand to
include new functionalities, which ensures clarity and avoids
ambiguity in the classification process. The statistical results
consider both commercial and open-source simulators, filtered
by their performance and influence. This table includes com-
mercial simulators used by renowned vehicle manufacturers
or technology companies, as well as free and open-source
simulators that have gained substantial attention, with over
100 stars on GitHub or more than 100 citations on Google
Scholar. Especially, some simulators had a significant impact
but are no longer actively maintained. To acknowledge their
4
influence while indicating their current status, such simulators
are denoted with gray text.
A. Traffic Flow Simulator
Traffic flow simulators play a crucial role in simulating
and analyzing the behavior of vehicles and pedestrians in
a traffic system. Their primary objective is to assess and
optimize traffic systems’ performance, safety, and efficiency,
providing valuable insights for engineering tasks such as urban
planning and road arrangement. These simulators commonly
offer functionalities to edit the road network, traffic signs, and
signal timing and adjust the behavior patterns of vehicles and
pedestrians, including factors like average vehicle speed, den-
sity, and lane-changing frequency. Simulation outputs typically
encompass metrics such as congestion, emissions, and traffic
capacity. The focus of these simulators is on macroscopic
aspects of traffic conditions rather than individual traffic
participant movement trajectories, making high accuracy in
vehicle dynamics and map details unnecessary.
Since the 1990s, various traffic flow simulators like PTV
Vissim, Aimsun, Paramics, and SUMO have been extensively
used [13, 14, 15, 33]. In recent years, there is no new simu-
lators for traffic flow simulation because the functionalities of
existing ones have been stabilized. Notably, the open-source
simulator SUMO continues to update its microscopic vehicle
models. Its open-source nature has garnered support from the
community, with many developers connecting their driving
strategy validators to SUMO to benefit from its ability to
control the background vehicles in the virtual environment
[61].
 5

TABLE I
BASIC INFORMATION OF CATEGORIZED SIMULATORS .
I F THE SIMULATOR IS NO LONGER UNDER MAINTENANCE , THE TEXT WILL BE COLORED GRAY.

Open-source Supported systems

Simulator Category Maintenance Provider
and free Windows Linux
√
PTV Vissim [33] Traffic flow 1992-now PTV Group √ √
Paramics [13] Traffic flow 1994-now SYSTRA √ √
Aimsun [14] Traffic flow 1999-now Siemens √ √ √
SUMO [15] Traffic flow 2001-now Eclipse Foundation
√ √
CarSim [35] Vehicle dynamics 1996-now Mechanical Simulation √ √ √
Webots [47] Vehicle dynamics 1998-now Cyberbotics √ √
CarMaker [36] Vehicle dynamics 1999-now IPG Automotive √ √ √
Gazebo [48] Vehicle dynamics 2002-now Open Robotics √ √
rFpro [37] Vehicle dynamics 2007-now rFpro
√ √
VI-Road [39] Scenario editor 2007-now VI-grade √ √
RoadRunner [49] Scenario editor 2020-now MathWorks
√ √
Parallel Domain [50] Sensory data 2017-now Parallel Domain √ √ √
AirSim [42] Sensory data 2017-2022 Microsoft √ √ √
SVL [43] Sensory data 2018-2021 LG Electronics √ √
VI-WorldSim [39] Sensory data 2020-now VI-grade √ √
Nvidia Drive Sim [51] Sensory data 2020-now Nvidia √ √ √
Vista [52] Sensory data 2020-2022 Vista Group
√ √ √
Torcs [53] Driving strategy 1997-2020 Torcs Group √ √ √
CarRacing [54] Driving strategy 2014-now Farama Foundation √ √ √
VDrift [55] Driving strategy 2014-now VDrift Group √ √ √
Udacity Self-driving [56] Driving strategy 2016-2022 Udacity √ √
Simulink Driving strategy 2017-now Mathworks √ √
Apollo [12] Driving strategy 2017-now Baidu √ √ √
CommonRoad [46] Driving strategy (2D) 2017-now TUM √ √ √
highway-env [45] Driving strategy (2D) 2018-now Eduoard Leurent √ √
SMARTS [57] Driving strategy (2D) 2020-now Huawei Noah √ √ √
SUMMIT [58] Driving strategy 2020-2022 AdaCompNUS
√ √
SCANeR Studio [59] Comprehensive 1990-now AVSimulation √ √
Virtual Test Drive [60] Comprehensive 1998-now Hexagon √
Prescan [38] Comprehensive 2006-now Siemens √ √ √
CARLA [11] Comprehensive 2016-now Computer Vision Center

B. Vehicle Dynamics Simulator
Vehicle dynamics simulators are specifically designed to
simulate the dynamic behavior of vehicles using principles
of physics. These simulators take into account both internal
factors, such as the vehicle’s powertrain, suspension, and
aerodynamics, and external conditions, such as road friction,
wind resistance, and slope. Considering these variables, the
simulator can accurately estimate the dynamics characteristics
of vehicles, including acceleration, braking, and steering re-
sponse to different control commands. Their widespread use
includes evaluating vehicle performance, optimizing vehicle
design, and enhancing safety performance.
Due to the well-established theories in vehicle dynamics,
the corresponding simulators have been available for a con-
siderable period. However, because of the strong reliance
on real-world experimental data, the most reliable vehicle
dynamics simulators are typically developed for commercial
use in close collaboration with car manufacturers. Leading
examples of state-of-the-art (SOTA) simulators in vehicle
dynamics are CarSim and IPG CarMaker, famous for their
outstanding performance [35, 36]. For vehicle manufacturers
seeking highly accurate estimation of vehicles’ physical behav-
ior, collaborating with these simulators is often the preferable
choice. In the domain of F1 racing, the rFpro simulator has
been developed to provide a realistic dynamic system for
racing cars and is widely utilized by F1 teams [37]. Among
open-source options, Gazebo and Webots are reliable choices
due to their rich experience in physics simulation in robotics
[47, 48]. These two simulators keep improving their vehicle
dynamics simulations to meet the growing demands of users.
C. Scenario Editor
A scene editor provides a visual interface for editing maps
and trajectories of traffic participants. The editing of maps
involves modifying the geometric properties and topological
relationships of roads, as well as adding points of interest
such as traffic lights and road signs. On the other hand, editing
traffic participants entails adjusting their appearance, type, and
trajectories within the scene. As is demonstrated in Fig. 2, the
purpose of scene editors is to provide data for downstream
tasks like traffic flow simulation, sensor data generation, and
the validation of driving strategies, where the edited data is
further processed and utilized. Therefore, scene editors are
typically integrated as submodules within other simulators
rather than existing as standalone tools.
Among the popular scene editors, the 2D scene editor
attached to SUMO stands out as an effective and widely used
tool [15]. For 3D scene editing, MathWorks’ RoadRunner

Fig. 2. The common tasks in developing an autonomous driving system and their corresponding simulator types [11, 15, 34, 35, 42, 46, 49]. If a simulator
can cover all the tasks within the system, it can be regarded as a comprehensive simulator.
is a notable choice and distinguishes itself as a standalone
simulator [49]. It is essential to note the significant difference
in how maps and vehicle trajectories are represented between
2D and 3D scene editors. Consequently, manual adjustments
may be required when converting data formats [34, 62]. The
issue of the map’s data format will be further discussed in
detail in Section VI-B.
D. Sensory Data Generator
Sensory data generators are tools that can synthesize raw
sensor data and ground truth annotations using rendering and
numerical calculation techniques. The sensor data typically
includes RGB images, lidar points, and radar points, while
ground truth annotations contain objects’ bounding boxes,
semantic labels, and depth values. These generators play a
crucial role in improving the robustness of perception al-
gorithms, as most SOTA algorithms are data-driven. Since
collecting real-world data under diverse conditions is costly
and time-consuming, researchers have to depend on sensory
data generators to enhance the diversity and complexity of
their datasets [1]. Despite the significance of these generators,
their numbers remain limited due to the ongoing development
of synthesizing algorithms [63, 64, 65]. Parallel Domain is one
of the few companies that have successfully commercialized
this data synthesis service [50].
Perception data synthesis is often conducted through the
rendering functionality within simulators. Since realistic scene
6
rendering typically requires game engines, many perception
data generators offer interfaces for controlling agent vehi-
cles. The distinction between a perception data generator
and a comprehensive simulator lies in vehicles’ physical
performance and traffic participants’ behavior patterns. These
features determine whether a simulator can serve functions
beyond data synthesis. Accordingly, simulators like LGSVL
and AirSim are classified as perception data generators, given
their primary focus on generating sensor data rather than
providing a comprehensive simulation environment [42, 43].
E. Driving Strategy Validator
Driving strategy validators examine the performance of
algorithms in tasks such as path planning, trajectory prediction,
and driving policy generation by providing executable traffic
scenarios. A branch of driving strategy validators emphasizes
decision-making as a submodule of an autonomous driving
system, thus pursuing a high fidelity in sensor data and
vehicle dynamics. The main difference between this branch
of driving strategy validators and comprehensive simulators
lies in scene editing. Due to the high cost and complexity
of obtaining map and object trajectory data in a scene, most
driving strategy validators prefer to provide these data as part
of their paid services without giving users sufficient freedom
in editing. Examples of simulators in this category include
Apollo, Udacity self-driving platform, and Vista [12, 52].

Fig. 3. The critical issues in simulators [63, 66, 67]. The fidelity problems affect the reliability of the simulation. The efficiency problems affect the time
and computation resource cost of the simulation.
Since high-quality validators are usually commercial soft-
ware or paid services with high hardware requirements, they
are unaffordable for many researchers. This situation motivates
the development of small and independent driving strategy
validators, which use simplified vehicle dynamics models and
idealized perception data. These validators focus on verifying
the effectiveness of algorithms while underlooking the errors
brought by sensors and vehicle control. In the open-source
community, this branch of simulators has gained widespread
attention. Two good examples include TorcsRL, which focuses
on validating reinforcement learning-based control models
in racing scenarios [53], and Highway-env, which provides
four environments (highway, intersection, roundabout, and
parking lot) for evaluating the performance of driving policy
models [45]. These open-source simulators provide convenient
and flexible tools that allow researchers to quickly validate
their theories and algorithms without worrying about complex
dynamics models and expensive perception data collection.
F. Comprehensive Simulator
Simulators mentioned above are tailored for specific tasks.
However, variations in data format between the simulators
from different developers can lead to errors and even sys-
tem crashes during testing. Comprehensive simulators offer a
unified environment that promotes seamless interaction among
components, resolving format inconsistencies and ensuring a
smoother testing process for the entire system. Fig. 2 shows
the tasks in autonomous driving systems that can be fully
developed within a virtual environment and their relation with
the task-specific simulators. Comprehensive simulators stand
out as entities that integrate a majority of functionalities, which
endows them with the capacity to handle relatively complete
autonomous driving systems. These simulators encompass a
traffic scenario editor, generate highly realistic perception data,
and provide interfaces for program-controlled vehicles. In the
simulation process, comprehensive simulators consider the
vehicles’ physics behavior and interactions with surrounding
7
traffic participants. A comprehensive simulator of high quality
can notably accelerate the research of autonomous driving.
Nevertheless, the development process requires multidisci-
plinary technologies and substantial financial support.
The Virtual Test Drive (VTD) is a widely acclaimed com-
prehensive simulator, offering highly integrated functionalities
that satisfy diverse needs in developing an autonomous driving
system [60]. A similar product, SCAnER, presents comparable
attributes, yet its popularity remains relatively limited due
to a lack of promotion [59]. Waymo is reputed to possess
an advanced comprehensive simulator, though the technical
intricacies have not been disclosed publicly [7]. Within the
realm of open-source simulators, CARLA stands out as the
sole platform demonstrating the inclination and capability to
evolve into a comprehensive simulator. Still, there is much
space for improving its performance [11].
Despite the challenges in developing a comprehensive sim-
ulator, it is meaningful for autonomous driving. Such a simu-
lator can present realistic scenarios and situations, allowing
researchers and developers to assess the performance and
safety of autonomous driving systems thoroughly. Therefore,
continued investment and support for developing simulators
are crucial for advancing autonomous driving technology.
VI. C RITICAL I SSUES
Simulators for autonomous driving face challenges in fi-
delity and efficiency (see Fig. 3). Fidelity in simulation per-
tains to the discernible variance between synthesized data and
real-world data. In tasks including adverse weather conditions,
automated map reconstruction, and interactive traffic partici-
pants, the output produced by simulators exhibits significant
disparities from the real-world data [31, 32, 68]. Efficiency
in simulation centers around code optimization, thereby ex-
pediting the training and testing phases. Headless simulation
enables the deployment of simulators across large-scale, re-
mote servers without a physical display, which makes better
use of computation resources [69]. High frame-rate simulation
 8

underscores the refinement of simulation algorithms, culminat-

ing in accelerated individual simulation processes [70, 71, 72].
Addressing these challenges is crucial as current simulators
often suffer from distortion, undermining the credibility of
simulating experiments; the high hardware requirements and
limited simulation speed indicate that simulators have not fully
realized their theoretical advantages.
By observing the free and open-source simulators, this
section delves into the critical issues in simulators, providing
definitions and emphasizing their importance. It will also
review existing methods to tackle these problems, offering
potential solutions for enhancing simulators’ performance.

A. Adverse Weather Simulation

The common sensors in autonomous driving systems in-
clude the camera, lidar, radar, IMU, speedometer, and GNSS.
Among these, the camera, lidar, and radar take center stage
in local perception tasks, serving to dynamically comprehend
the traffic environment encompassing an autonomous vehicle.
Regrettably, these cameras dedicated to real-time local percep-
tion are notably vulnerable to adverse weather conditions [73].
Factors like illumination, visibility, and humidity fluctuate in Fig. 4. Particle-based snowfall simulation for lidar points [65]. From the
different weather, introducing noise into sensor data, so the comparison, the augmented data improve the accuracy of object detection,
precision and range of perception outcomes are compromised
[74]. Adverse weather conditions also alter road reflectance,
further perturbing the efficacy of perception algorithms [75].
Hence, improving perception algorithms’ robustness in diverse
weather has been a paramount objective for a long time.
Collecting authentic sensor data from real-world scenar-
ios is costly and time-demanding. Given the erratic nature
of weather, collected data might deviate unpredictably from
anticipated norms. In contrast, simulators offer precise control
over weather conditions through adjustable parameters. They
circumvent the need for prolonged waits to encounter specific
weather scenarios [1]. Moreover, the labor-intensive task of
manually annotating ground truth data is obviated. These
inherent advantages suggest that if simulators can proficiently
generate high-fidelity data in adverse weather conditions, they
possess the potential to supplant real-world data and promote
relevant research [74, 76].
Most existing simulators focus on the presence of ad-
verse weather simulation functionality without considering the
actual quality of the implementation. Since the simulators
usually rely on game engines for development, the common Fig. 5. GAN-based rain simulation for RGB images[64]. Tremblay et al. apply
choice is to use the engines’ default particle system to generate a GAN-based method to simulate a rainy scenario with 50 mm/hr precipitation.
adverse weather [11, 42, 43]. In a particle system, rain and
snow are treated as particles with physics properties. During
the simulation process, the emphasis is on the movement tra- the distribution, and the original data merge with the mask
jectory of particles due to wind speed and gravity, but less at- through post-processing. Modeling the noise distribution pat-
tention is paid to the texture templates of particles [63, 64, 77]. tern with particle systems and physical properties was one
Thus the image data generated based on particle systems lacks of the earliest practices [63, 78, 79], which was initially
realism. Furthermore, existing weather simulations often only applied to image generation and extended to enhance lidar
affect rendering, meaning they can only generate image data point data (see Fig. 4) [65]. In recent years, learning-based
for adverse weather conditions without adding corresponding data synthesis methods have been increasing [80, 81]. Among
noise interference to lidar and millimeter-wave radar. them, approaches based on Generative Adversarial Networks
In most synthesis processes, the distribution pattern of (GANs) and Neural Radiance Fields (NeRF) have achieved
noise is first extracted, then a mask is generated based on impressive results [82, 83, 84, 85]. Fig. 5 illustrates rain

data synthesized by Tremblay et al. using a combination
of CycleGAN and physics-based rendering. This synthesis
maintains visual realism while adhering well to physical laws.
In order to enhance simulators’ performance in simulat-
ing adverse weather conditions, the simulator should pro-
vide a flexible and user-friendly interface that allows users
to integrate customized data synthesis models. Advanced
weather simulation algorithms are often available as open-
source projects online [76, 81, 85]. These algorithms vary in
environment configuration and hardware requirements, posing
challenges for users looking to adopt them quickly. A solution
could be establishing a standardized parameter interface within
the simulator, allowing users to effectively adjust various
weather parameters to simulate different weather conditions.
B. Automated Map Reconstruction
High-definition maps are pivotal in crafting the testing
environments for autonomous driving systems. A meticulously
annotated map is the foundation for generating sensory data
and formulating driving strategies. Given that the intricate
road layouts and exceptional vehicle behaviors predominantly
originate from real-world scenarios, generating traffic scenar-
ios, especially the maps, hinges heavily on authentic data.
The conventional map reconstruction process manually ex-
tracts road structures, labels road network relationships, and
annotates traffic signals from lidar points and images, which is
labor-consuming. By comparison, automated map reconstruc-
tion utilizes algorithms to process 2D pixels and 3D points
and generate structured maps. Achieving partial or complete
automated map construction can significantly reduce manual
effort and improve the efficiency of building scenarios.
Automated map reconstruction is advancing along two
distinct trajectories. One avenue is oriented towards 3D re-
construction, characterized by an emphasis on achieving high-
level realism and details. The general idea is to extract features
from 2D images and match them with 3D points and pose
information. By establishing correlations between object mod-
els and textures, this method enables effective automatic map
reconstruction [86, 87, 88]. Traditional 3D map reconstruction
methods use explicit priors to process large-scale raw data
rapidly [88, 89]. The new rising learning-based methods ex-
tract data-driven priors by neural networks to handle noise and
data sparsity [90, 91]. Fig. 6 depicts the reconstruction result
of an advanced 3D map reconstruction method, Neural Kernel
Surface Reconstruction [92]. A recent breakthrough in this
area is the NeRF algorithm. It learns the implicit features of
space from images captured from different angles, representing
3D space as a continuous radiance field [93]. NVIDIA has
implemented 3D surface reconstruction and rendering based
on the NeRF algorithm, demonstrating its great potential
for automated 3D map reconstruction [67]. Another avenue
focuses on reconstructing the road structure and relationships,
exhibiting a greater tolerance for deviation in map details
[94]. These techniques prioritize comprehensively depicting
the traffic environment, encompassing elements such as the
road network, traffic signals, and dynamic objects [66, 95].
With the overarching structure and interplay dynamics of the
9
Fig. 6. Surface reconstruction based on lidar points [92]. This scene is
reconstructed based on data from Waymo’s dataset, which consists of 10M
points, taking only 20s.
traffic scenario, the 2D maps generated by these approaches
become preferable to the development of driving decision-
making algorithms [68, 96].
These two categories of automated map reconstruction
methods can reduce the need for manual intervention in
constructing simulation scenarios for autonomous driving [97].
However, they are unsuitable for practical implementation at
current. The 3D map construction techniques exhibit sensitiv-
ity to data distribution and susceptibility to noise interference
[66]. The precision and speed of these methods fall short
of the standard for industrial application [93]. It is worth
noting that the output of 3D map reconstruction is the 3D
mesh model, which lacks structural information on roads and
buildings. Also, semantic annotations about signs and signals
remain absent within the 3D mesh model. Compared to the
widely adopted 3D map data format OpenDRIVE, 3D mesh
models exhibit significant data gaps [62]. Although 2D map
reconstruction approaches furnish data on road structure, the
seamless integration of their outputs with 3D map reconstruc-
tion poses challenges. Typically, the outcomes of 2D map
reconstruction are expressed in XML-based formats like Open-
StreetMap (OSM) and Lanelet [98, 99]. Due to the different
mechanisms by which OSM and Lanelet store road geometry
compared to the OpenDRIVE, data conversion often raises
errors necessitating manual correction. Therefore, matching
the road structure with the 3D mesh model is a task that
remains accomplished. Addressing these challenges is critical
for advancing the state of automated map reconstruction and
realizing its potential in autonomous driving simulation.
C. Interactive Traffic Participants
A prominent limitation of current simulators is the absence
of intelligence in traffic participants. Traffic participants en-
compass vehicles, cyclists, pedestrians, and dynamic objects
within a traffic scenario surrounding the self-driving vehicle.
Ideally, these participants should be able to perceive their
surroundings, comprehend the actions of other participants,
and navigate within the scenario efficiently [95].
Presently, the traffic participants featured in simulators rely
on rule-based behavioral models, which exhibit simplistic

Fig. 7. A series of generated unsafe scenarios [95]. These attackers can help
test the ability of the autonomous vehicle to avoid accidents.
behavior patterns [11]. These models are constrained to actions
like maintaining a specific distance and stopping, lacking the
flexibility to circumvent obstacles or accelerate through scenes
[100, 101, 102]. The notable disparity between the behavioral
models of simulated traffic participants and their real-world
counterparts undermines the credibility of simulation experi-
ments in driving decision-making task [32].
Indeed, several methods have been devised to enhance the
interaction capabilities of traffic participants [23, 27]. A preva-
lent approach involves adding an element of randomness to the
behavior of these participants. The typical procedure entails
the behavior model initially outlining potential choices to
navigate the traffic participant to pass the scenario successfully.
Then a trajectory is randomly chosen based on historical data
distributions accumulated from analogous situations. These
data distributions can be represented through explicit formulas
or harnessed through neural networks [3, 103, 104]. This tech-
nique aids in imbuing traffic participants with unpredictability,
contributing to a more authentic and varied simulation environ-
ment. The flexibility and complexity of generating interactive
traffic participants still have much room for improvement.
Besides improving traffic participant controlling algorithms,
integrating algorithms with the simulators is another pivotal
challenge. Interactive driving strategies tend to exhibit more
aggressiveness, calling for an efficient way to detect traffic rule
violations and collisions. One possible solution is to adopt a
multi-agent driving strategy model to substitute the traditional
traffic participants. In this way, all vehicles within the traffic
scenario assume the role of controllable entities governed
by the driving strategy model. This approach enhances the
intelligence of traffic participants while concurrently ensuring
that the driving strategy models are meticulously developed
and rigorously tested, so it has gained support from some
of the simulators and algorithms [11, 46, 57, 105]. The
other solution to accelerate simulation and save computation
resources will be discussed in Section VI-E.
Moreover, open-source simulators should offer an interface
for users to customize traffic participants. By equipping simu-
lators with a dependable traffic event detector and a framework
to tailor behavior models, the creativity of the open-source
community can facilitate realizing more interactive traffic
participant controllers through collaborative contributions.
10
D. Headless Simulation
Headless simulation empowers a simulator to operate on
remote servers without a physical display. By eliminating the
necessity for a Graphical User Interface (GUI), the simula-
tor can allocate computational resources more streamlined,
leading to optimal resource utilization. This approach proves
particularly beneficial for the seamless deployment of simula-
tors within extensive computational infrastructures, including
cloud platforms and computer clusters. The versatility and
scalability offered by headless simulation enable the effective
management of large-scale simulation tasks [69].
Most existing simulators are built upon game engines or
graphical frameworks [11, 45, 46]. The presence of headless
simulation in open-source simulators is examined in TABLE
II. Achieving headless simulation necessitates comprehensive
customizations spanning diverse domains: fine-tuning ren-
dering protocols, optimizing data transmission and storage
mechanisms, and implementing performance enhancements
meticulously tailored to the headless simulation paradigm.
This intricate endeavor is pivotal for effectively dissociating
the simulator from graphical interfaces, thus enabling seamless
operation within remote server environments.
E. Multiple-speed Simulation
Multiple-speed simulation refers to accelerating the simu-
lation speed without compromising the accuracy of rendering
and physics updates. This feature significantly improves the
efficiency of simulation testing, enabling engineering tests that
typically take several months in the real world to be completed
in half or even less time. The implementation of multiple-
speed simulation follows a similar framework to high frame-
rate simulation. However, based on investigations of open-
source and actively maintained simulators, it is apparent that
the frame rate rarely exceeds 60 Hz (as shown in TABLE II),
and most simulators do not support multiple-speed simulation.
The frame rate further drops when the simulation grows
complex, such as high traffic density and adverse weather,
which indicates a pressing need for performance optimization.
Here are some feasible directions for improvement:
1) Algorithm optimization: Improve the data structures and
algorithms for collision detection, physics update, rendering,
and other computational tasks to increase efficiency.
2) Code optimization: Optimize the code of the simulator
to eliminate unnecessary computations, memory usage, and
delays, thereby improving overall performance.
3) Data compression: Use efficient data compression al-
gorithms to reduce storage and transmission overhead of
simulation data, improving simulation efficiency.
4) Level of details: For certain scenarios or testing ob-
jectives, the level of detail can be selectively expressed to
accelerate the simulation speed. This can be achieved through
simplifying models, lowering texture resolution, or reducing
the precision of physics calculations, etc [71].
5) Parallel and distributed computation: To accelerate sim-
ulation speed, a viable solution is to leverage the power of
multi-core processors and distributed computation by distribut-
ing tasks across multiple processing units [72].
 11

TABLE II
T HE HARDWARE REQUIREMENTS AND PERFORMANCE OF THE FREE AND OPEN - SOURCE SIMULATORS THAT ARE ACTIVELY MAINTAINED

Minimal hardware requirements Headless Parallel Distributed Maximal

Simulator
Disk space Memory GPU simulation computation computation FPS
SUMO 1 GB 512 MB None 120 Hz
√ √
Webots 1 GB 2 GB 512 MB √ √ √ 60 Hz
Gazebo 500 MB 4 GB 1 GB 500 Hz
√
CarRacing 1 MB 128 MB None 60 Hz
VDrift 512 MB 1GB None √ √ 60 Hz
Apollo 2 GB 16 GB None 60 Hz
CommonRoad 32 MB 2 GB None 60 Hz
highway-env 1 MB 128 MB None 60 Hz
SMARTS 128 MB 2 GB None 60 Hz
√ √ √
CARLA 20 GB 8 GB 8GB 60 Hz

VII. E VALUATION M ETRICS
After comprehending the critical issues of simulators and
their potential remedies, it is essential to employ qualitative
and quantitative metrics for evaluating simulator performance
in specific aspects. This section reviews and explains the
evaluation metrics for assessing simulation fidelity, organized
by the metrics’ mechanisms. Given the limited number of
research addressing critical issues and the divergence in the
metric selection, this section will only explain the indicators
employed across multiple research papers in detail. The evalu-
ation of simulation efficiency, being more straightforward and
unified, will be condensed and presented in the last subsection.
A. Visualization
Visualization is a straightforward and intuitive approach
to assess the realism of image synthesis. Early research on
adverse weather simulation commonly involved comparing
synthesized images with real-world images and outputs of
other algorithms (as shown in Fig. 5) [64, 79]. Visualization
is equally applicable to synthesized lidar points and radar
data (as shown in Fig. 4) [65, 81]. While visualization alone
may not be sufficient to fully validate the significance of data
synthesis algorithms, it remains a crucial aspect of evaluation.
Quantitative metrics can test the performance of data synthesis
algorithms in specific aspects, but they may not capture the
finer details and realism of image synthesis. By visually
presenting the differences between synthesized images and
real-world images, visualization aids researchers in better
understanding the output and identifying potential issues for
further improvements.
Despite its advantages, visualization also has limitations.
First, it is a subjective method, so every individual may have
a unique perception [85]. Second, visual results lack specific
quantitative metrics to directly measure the difference between
synthesized and real-world images. Therefore, quantitative
metrics and visual results are both necessary for a compre-
hensive assessment of data synthesis algorithms.
B. Distribution-based Metric
Assessing synthesizing algorithms through distribution-
based metrics is a practical and widely applicable approach.
This method does not directly evaluate data generation perfor-
mance but involves using intermediate algorithms to compare
the synthesized data distributions. The underlying theory is
as follows: assume that real-world data follows a distribution
pattern D, data generated by method I follows distribution D′ ,
and data generated by method II follows distribution D′′ . By
training intermediate algorithms α and β on datasets produced
by method I and method II, respectively, the prediction
accuracy can be assessed. Upon transferring the models to real-
world datasets, if the accuracy of method I surpasses that of
method II, it can be inferred that the synthesized distribution
D′ is closer to the real-world data distribution D, indicating
the superiority of method I over method II.
Numerous algorithms for synthesizing images [64, 84], lidar
points [65], radar [81] and other sensor data have utilized
SOTA perception algorithms as intermediaries to demonstrate
their effectiveness. The widely used accuracy metrics for
intermediate perception algorithms include Average Precision
[63, 83], mean Average Precision [64], Intersection of Union
(IoU) [80, 97], and F1 score. These metrics connect synthe-
sizing methods’ performance and practical tasks, making them
valuable for real-world applications. However, this evaluation
method also has certain drawbacks. The main limitation is
the time cost, as the evaluation process requires significant
data processing and model training [76]. Another concern is
the lack of theoretical research to justify the statistical signif-
icance. It remains unclear how much percentile of accuracy
the media algorithm should reach to prove one synthesizing
method is an advantage over others convincingly.
C. Accuracy-based Metric
Using accuracy metrics for map reconstruction when ground
truth is available is a viable approach. When the reconstructed
map contains models or structured organization, metrics like
IoU or F1 scores can be used to compare the accuracy of
the reconstruction with real-world data [68, 106]. For map
reconstruction models based on NeRF and other neural kernel-
based methods, image quality assessment metrics such as Peak
Signal-to-Noise Ratio (PSNR) and Structural Similarity Index
(SSIM) can also be introduced [67, 93].

PSNR is calculated using the Mean Squared Error (MSE)
formula and is generally expressed as
P SN R = 20 · log10 (M AXI ) − 10 · log10 (M SE) (1)
where M AXI represents the maximum possible pixel value in
the image, and M SE is the difference between the generated
image and the ground truth. A higher PSNR value indicates
smaller noise and better reconstruction results.
SSIM, on the other hand, assesses the algorithm’s quality
by luminance, contrast, and structure and is represented as
SSIM (x, y) = [l(x, y)α · c(x, y)β · s(x, y)γ ] (2)
where x and y are the generated image and the real-world
image, and l(x, y), c(x, y), and s(x, y) are the evaluation
functions for brightness, contrast, and structure, respectively.
α, β, γ are adjustable parameters [107].
Despite their advantages, accuracy-based evaluation meth-
ods have some drawbacks. They often require manual an-
notations on real-world data, which can be time-consuming
and labor-intensive [106]. Additionally, accurately matching
corresponding points between synthetic and real-world data
is crucial to avoid inaccuracies in the evaluation results due
to matching errors [88]. Furthermore, accuracy-based metrics
only provide specific insights into data quality and may not
comprehensively evaluate algorithm performance. A combina-
tion of different evaluation methods is often necessary for a
comprehensive assessment.
D. Metrics for Simulation Efficiency
Validating headless simulation is a pure engineering task.
It involves running the simulator across various operating
systems and environmental configurations to confirm its in-
dependence from the GUI. The assessment process entails
numerous iterations of trial and error. The outcome of this
testing is a straightforward true-false answer about whether
headless simulation is successfully implemented.
The assessment of multiple-speed simulation involves mon-
itoring the simulator’s execution speed and allocated com-
putational resources. Tracking execution speed is typically
achieved through plugins and programs, with specifics deter-
mined by the simulator’s programming language and frame-
work architecture. In terms of computational resource usage,
critical metrics to monitor are CPU utilization, memory al-
location, GPU utilization, and GPU memory allocation. For
high-performance simulators supporting parallel processing,
performance evaluation often employs metrics like the speedup
ratio S and speedup efficiency E, defined as follows:
serial execution time
S= (3)
parallel execution time
S pp. 24 474–24 487, 2022.
E= (4)
P [4] S. Wang, R. Stern, and M. W. Levin, “Optimal con-
where P denotes the number of processors. The speedup
efficiency E is expected to be greater or equal to 1. If E ≪
1, the optimization is invalid [108].
12
VIII. C ONCLUSIONS
This paper presents a comprehensive overview of the history
of simulators for autonomous driving over the past three
decades. The timeline is divided into three distinct phases:
the specialized development period (1990s-2002), the gap pe-
riod (2002-2013), and the comprehensive development period
(2013-present). A tendency toward comprehensive functional-
ity and open-source accessibility is detected in the evolution
process. After investigating influential commercial and open-
source simulators, they are classified into five specialized
classes: traffic flow simulators, vehicle dynamics simulators,
scene editors, sensor data synthesizers, and driving strategy
validators. Additionally, simulators that enclose multiple func-
tionalities are grouped as comprehensive simulators.
After obtaining a systematic understanding of the simula-
tors, this paper identifies the critical issues that defect the
usage of simulators by observing the open-source simula-
tors. The primary concerns are two aspects, data fidelity,
and simulation efficiency. This paper argues that addressing
challenges related to adverse weather simulation, automatic
map reconstruction, and interactive traffic participants will
greatly enhance the realism of simulators. Additionally, devel-
oping a headless simulation pipeline and achieving high frame
rates can significantly improve the efficiency of simulators.
This paper provides a clear definition of the critical issues,
emphasizes their importance, and outlines feasible solutions as
valuable guidelines for researchers who tend to use or improve
simulators. Moreover, the paper explores both qualitative and
quantitative metrics to evaluate the effectiveness of resolving
these challenges within simulators, elucidating the principles
and scope of applicability of these metrics.
This paper is written as a call for more attention to the
simulators for autonomous driving. Continuous and systematic
improvements in simulators are essential to the success of
autonomous driving systems. As the field progresses, further
advancements in simulators will fasten the realization and
application of autonomous driving technology.
R EFERENCES
[1] N. Kalra and S. M. Paddock, “Driving to safety: How
many miles of driving would it take to demonstrate au-
tonomous vehicle reliability?” Transportation Research
Part A: Policy and Practice, vol. 94, pp. 182–193, 2016.
[2] A. Shafaei, J. J. Little, and M. Schmidt, “Play and learn:
Using video games to train computer vision models,”
arXiv preprint arXiv:1608.01745, 2016.
[3] Q. Zhang, Y. Gao, Y. Zhang, Y. Guo, D. Ding, Y. Wang,
P. Sun, and D. Zhao, “Trajgen: Generating realistic
and diverse trajectories with reactive and feasible agent
behaviors for autonomous driving,” IEEE Transactions
on Intelligent Transportation Systems, vol. 23, no. 12,
trol of autonomous vehicles for traffic smoothing,”
IEEE Transactions on Intelligent Transportation Sys-
tems, vol. 23, no. 4, pp. 3842–3852, 2021.

[5] Y. Wang, S. Zhao, R. Zhang, X. Cheng, and L. Yang,
“Multi-vehicle collaborative learning for trajectory pre-
diction with spatio-temporal tensor fusion,” IEEE
Transactions on Intelligent Transportation Systems,
vol. 23, no. 1, pp. 236–248, 2020.
[6] IEEE Transactions on Intelligent Transportation Sys-
tems, “Ieee transactions on intelligent transportation
systems all issues 2022,” https://ieeexplore.ieee.org/xpl/
issues?punumber=6979&isnumber=10175858, accessed
Aug. 14, 2023.
[7] The Waymo Team, “Simulation city: Introducing
waymo’s most advanced simulation system yet for
autonomous driving,” https://waymo.com/blog/2021/06/
SimulationCity.html, accessed Aug. 14, 2023.
[8] Pony.ai, “Technology,” https://pony.ai/tech?lang=en, ac-
cessed Aug. 14, 2023.
[9] DiDi, “Didi’s autonomous driving,” https:
//www.didiglobal.com/science/intelligent-driving,
accessed Aug. 14, 2023.
[10] A. Sinha, S. Chand, V. Vu, H. Chen, and V. Dixit,
“Crash and disengagement data of autonomous vehicles
on public roads in california,” Scientific data, vol. 8,
no. 1, p. 298, 2021.
[11] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and
V. Koltun, “Carla: An open urban driving simulator,” in
Conference on robot learning. PMLR, 2017, pp. 1–16.
[12] Baidu, “Apollo,” https://github.com/ApolloAuto/apollo,
accessed Aug. 14, 2023.
[13] M. Smith, G. Duncan, and S. Druitt, “Paramics: micro-
scopic traffic simulation for congestion management,”
1995.
[14] J. Barceló and J. Casas, “Dynamic network simula-
tion with aimsun,” in Simulation approaches in trans-
portation analysis: Recent advances and challenges.
Springer, 2005, pp. 57–98.
[15] M. Behrisch, L. Bieker, J. Erdmann, and D. Krajzewicz,
“Sumo–simulation of urban mobility: an overview,” in
Proceedings of SIMUL 2011, The Third International
Conference on Advances in System Simulation. Think-
Mind, 2011.
[16] D. Feng, A. Harakeh, S. L. Waslander, and K. Di-
etmayer, “A review and comparative study on proba-
bilistic object detection in autonomous driving,” IEEE
Transactions on Intelligent Transportation Systems,
vol. 23, no. 8, pp. 9961–9980, 2021.
[17] A. Haydari and Y. Yılmaz, “Deep reinforcement learn-
ing for intelligent transportation systems: A survey,”
IEEE Transactions on Intelligent Transportation Sys-
tems, vol. 23, no. 1, pp. 11–32, 2020.
[18] Y. Li, R. Chen, X. Niu, Y. Zhuang, Z. Gao, X. Hu,
and N. El-Sheimy, “Inertial sensing meets machine
learning: Opportunity or challenge?” IEEE Transactions
on Intelligent Transportation Systems, 2021.
[19] E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda,
“A survey of autonomous driving: Common practices
and emerging technologies,” IEEE Access, vol. 8, pp.
58 443–58 469, 2020.
[20] M. Johnson-Roberson, C. Barto, R. Mehta, S. N. Srid-
13
har, K. Rosaen, and R. Vasudevan, “Driving in the
matrix: Can virtual worlds replace human-generated
annotations for real world tasks?” arXiv preprint
arXiv:1610.01983, 2016.
[21] S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu,
“A survey of deep learning techniques for autonomous
driving,” Journal of Field Robotics, vol. 37, no. 3, pp.
362–386, 2020.
[22] C. Katrakazas, M. Quddus, W.-H. Chen, and L. Deka,
“Real-time motion planning methods for autonomous
on-road driving: State-of-the-art and future research
directions,” Transportation Research Part C: Emerging
Technologies, vol. 60, pp. 416–442, 2015.
[23] W. Schwarting, J. Alonso-Mora, and D. Rus, “Planning
and decision-making for autonomous vehicles,” Annual
Review of Control, Robotics, and Autonomous Systems,
vol. 1, pp. 187–210, 2018.
[24] L. Claussmann, M. Revilloud, D. Gruyer, and S. Glaser,
“A review of motion planning for highway autonomous
driving,” IEEE Transactions on Intelligent Transporta-
tion Systems, vol. 21, no. 5, pp. 1826–1848, 2019.
[25] U. M. Gidado, H. Chiroma, N. Aljojo, S. Abubakar,
S. I. Popoola, and M. A. Al-Garadi, “A survey on deep
learning for steering angle prediction in autonomous
vehicles,” IEEE Access, vol. 8, pp. 163 797–163 817,
2020.
[26] A. Tampuu, T. Matiisen, M. Semikin, D. Fishman,
and N. Muhammad, “A survey of end-to-end driving:
Architectures and training methods,” IEEE Transactions
on Neural Networks and Learning Systems, vol. 33,
no. 4, pp. 1364–1384, 2020.
[27] L. Wei, Z. Li, J. Gong, C. Gong, and J. Li, “Autonomous
driving strategies at intersections: Scenarios, state-of-
the-art, and future outlooks,” in 2021 IEEE Interna-
tional Intelligent Transportation Systems Conference
(ITSC). IEEE, 2021, pp. 44–51.
[28] J. Zhou, Y. Zhang, S. Guo, and Y. Guo, “A survey
on autonomous driving system simulators,” in 2022
IEEE International Symposium on Software Reliability
Engineering Workshops (ISSREW). IEEE, 2022, pp.
301–306.
[29] M. Holen, K. Knausgård, and M. Goodwin, “An eval-
uation of autonomous car simulators and their appli-
cability for supervised and reinforcement learning,” in
International Conference on Intelligent Technologies
and Applications. Springer, 2021, pp. 367–379.
[30] P. Kaur, S. Taghavi, Z. Tian, and W. Shi, “A survey
on simulators for testing self-driving cars,” in 2021
Fourth International Conference on Connected and Au-
tonomous Driving (MetroCAD). IEEE, 2021, pp. 62–
70.
[31] X. Hu, S. Li, T. Huang, B. Tang, and L. Chen, “Sim2real
and digital twins in autonomous driving: A survey,”
arXiv preprint arXiv:2305.01263, 2023.
[32] C. Siebke, M. Mai, and G. Prokop, “What do traffic
simulations have to provide for virtual road safety
assessment? human error modeling in traffic simula-
tions,” IEEE Transactions on Intelligent Transportation

Systems, vol. 24, no. 2, pp. 1419–1436, 2022.
[33] PTV Group, “Ptv vissim: Multimodel traffic
simulation software,” https://www.myptv.com/en/
mobility-software/ptv-vissim, accessed Aug. 14, 2023.
[34] K. G. Lim, C. H. Lee, R. K. Y. Chin, K. B. Yeo,
and K. T. K. Teo, “Sumo enhancement for vehicular
ad hoc network (vanet) simulation,” in 2017 IEEE
2nd international conference on automatic control and
intelligent systems (I2CACIS). IEEE, 2017, pp. 86–91.
[35] Mechanical Simulation, “Carsim,” https://www.carsim.
com, accessed Aug. 14, 2023.
[36] IPG Automotive, “Solutions for virtual test driving,”
https://ipg-automotive.com/en, accessed Aug. 14, 2023.
[37] rFpro, “The world’s most accurate simulation environ-
ment,” https://rfpro.com, accessed Aug. 14, 2023.
[38] Siemens, “Simcenter prescan software,”
https://plm.sw.siemens.com/en-US/simcenter/
autonomous-vehicle-solutions/prescan, accessed
Aug. 14, 2023.
[39] VI-grade, “Driving simulators: An invaluable set
of tools to successfully accelerate your develop-
ment process,” https://www.vi-grade.com/en/solutions/
driving simulators, accessed Aug. 14, 2023.
[40] M. Martinez, C. Sitawarin, K. Finch, L. Meincke,
A. Yablonski, and A. Kornhauser, “Beyond grand theft
auto v for training, testing and enhancing deep learning
in self driving cars,” arXiv preprint arXiv:1712.01397,
2017.
[41] D. Wang, J. Wen, Y. Wang, X. Huang, and F. Pei, “End-
to-end self-driving using deep neural networks with
multi-auxiliary tasks,” Automotive Innovation, vol. 2,
pp. 127–136, 2019.
[42] S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Air-
sim: High-fidelity visual and physical simulation for
autonomous vehicles,” in Field and Service Robotics:
Results of the 11th International Conference. Springer,
2018, pp. 621–635.
[43] G. Rong, B. H. Shin, H. Tabatabaee, Q. Lu, S. Lemke,
M. Možeiko, E. Boise, G. Uhm, M. Gerow, S. Mehta
et al., “Lgsvl simulator: A high fidelity simulator
for autonomous driving,” in 2020 IEEE 23rd Interna-
tional Conference on Intelligent Transportation Systems
(ITSC). IEEE, 2020, pp. 1–6.
[44] M. Bansal, A. Krizhevsky, and A. Ogale, “Chauffeurnet:
Learning to drive by imitating the best and synthesizing
the worst,” arXiv preprint arXiv:1812.03079, 2018.
[45] E. Leurent, “An environment for autonomous
driving decision-making,” https://github.com/eleurent/
highway-env, accessed Aug. 14, 2023.
[46] M. Althoff, M. Koschi, and S. Manzinger, “Common-
road: Composable benchmarks for motion planning on
roads,” in 2017 IEEE Intelligent Vehicles Symposium
(IV). IEEE, 2017, pp. 719–726.
[47] O. Michel, “Cyberbotics ltd. webots™: professional
mobile robot simulation,” International Journal of Ad-
vanced Robotic Systems, vol. 1, no. 1, p. 5, 2004.
[48] N. Koenig and A. Howard, “Design and use paradigms
for gazebo, an open-source multi-robot simulator,” in
14
2004 IEEE/RSJ International Conference on Intelli-
gent Robots and Systems (IROS) (IEEE Cat. No.
04CH37566), vol. 3. IEEE, 2004, pp. 2149–2154.
[49] MathWorks, “Get started with roadrunner,”
https://www.mathworks.com/help/roadrunner/get-
started-with-roadrunner.html, accessed Aug. 14, 2023.
[50] Parallel Domain, “Parallel domain,” https:
//paralleldomain.com/, accessed Aug. 14, 2023.
[51] Nvidia, “Nvidia drive end-to-end platform for
software-defined vehicles,” https://www.nvidia.com/
en-us/self-driving-cars/, accessed Aug. 14, 2023.
[52] A. Amini, T.-H. Wang, I. Gilitschenski, W. Schwarting,
Z. Liu, S. Han, S. Karaman, and D. Rus, “Vista 2.0:
An open, data-driven simulator for multimodal sensing
and policy learning for autonomous vehicles,” in 2022
International Conference on Robotics and Automation
(ICRA). IEEE, 2022, pp. 2419–2426.
[53] B. Wymann, E. Espié, C. Guionneau, C. Dimitrakakis,
R. Coulom, and A. Sumner, “Torcs, the open racing car
simulator,” Software available at http://torcs. source-
forge. net, vol. 4, no. 6, p. 2, 2000.
[54] C. Campbell, “Box2d c++ tutorials - top-down car
physics,” http://www.iforce2d.net/b2dtut/top-down-car,
accessed Aug. 14, 2023.
[55] F. Kehrle, J. V. Frasch, C. Kirches, and S. Sager,
“Optimal control of formula 1 race cars in a vdrift
based virtual environment,” IFAC Proceedings Volumes,
vol. 44, no. 1, pp. 11 907–11 912, 2011.
[56] Udacity, “Udacity self-driving platform,” https://github.
com/udacity/self-driving-car-sim, accessed: 2023-07-
15.
[57] M. Zhou, J. Luo, J. Villella, Y. Yang, D. Rusu, J. Miao,
W. Zhang, M. Alban, I. Fadakar, Z. Chen et al.,
“Smarts: Scalable multi-agent reinforcement learning
training school for autonomous driving,” arXiv preprint
arXiv:2010.09776, 2020.
[58] P. Cai, Y. Lee, Y. Luo, and D. Hsu, “Summit: A
simulator for urban driving in massive mixed traffic,” in
2020 IEEE International Conference on Robotics and
Automation (ICRA). IEEE, 2020, pp. 4023–4029.
[59] AVSimulation, “Scaner 2023.2 release note,” https://
www.avsimulation.com/scaner-2023-2-release-note, ac-
cessed Aug. 14, 2023.
[60] Hexagon, “Virtual test drive: Complete tool-chain for
driving simulation applications,” https://hexagon.com/
products/virtual-test-drive, accessed Aug. 14, 2023.
[61] P. Li, A. Kusari, and D. J. LeBlanc, “A novel traffic sim-
ulation framework for testing autonomous vehicles us-
ing sumo and carla,” arXiv preprint arXiv:2110.07111,
2021.
[62] M. Dupuis, M. Strobl, and H. Grezlikowski, “Opendrive
2010 and beyond–status and future of the de facto
standard for the description of road networks,” in Proc.
of the Driving Simulation Conference Europe, 2010, pp.
231–242.
[63] A. Von Bernuth, G. Volk, and O. Bringmann, “Simulat-
ing photo-realistic snow and fog on existing images for
enhanced cnn training and evaluation,” in 2019 IEEE

Intelligent Transportation Systems Conference (ITSC).
IEEE, 2019, pp. 41–46.
[64] M. Tremblay, S. S. Halder, R. De Charette, and J.-F.
Lalonde, “Rain rendering for evaluating and improving
robustness to bad weather,” International Journal of
Computer Vision, vol. 129, pp. 341–360, 2021.
[65] M. Hahner, C. Sakaridis, M. Bijelic, F. Heide, F. Yu,
D. Dai, and L. Van Gool, “Lidar snowfall simulation
for robust 3d object detection,” in Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern
Recognition, 2022, pp. 16 364–16 374.
[66] S. Tan, K. Wong, S. Wang, S. Manivasagam, M. Ren,
and R. Urtasun, “Scenegen: Learning to generate real-
istic traffic scenes,” in Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recogni-
tion, 2021, pp. 892–901.
[67] Z. Li, T. Müller, A. Evans, R. H. Taylor, M. Unberath,
M.-Y. Liu, and C.-H. Lin, “Neuralangelo: High-fidelity
neural surface reconstruction,” in Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern
Recognition, 2023, pp. 8456–8465.
[68] Y. Tian, A. Carballo, R. Li, and K. Takeda, “Real-to-
synthetic: Generating simulator friendly traffic scenes
from graph representation,” in 2022 IEEE Intelligent
Vehicles Symposium (IV). IEEE, 2022, pp. 1615–1622.
[69] A. Afzal, D. S. Katz, C. L. Goues, and C. S. Timperley,
“A study on the challenges of using robotics simulators
for testing,” arXiv preprint arXiv:2004.07368, 2020.
[70] S. J. Seyed Aboutorabi and M. H. Rezvani, “An opti-
mized meta-heuristic bees algorithm for players’ frame
rate allocation problem in cloud gaming environments,”
The Computer Games Journal, vol. 9, pp. 281–304,
2020.
[71] B. Varga, D. Doba, and T. Tettamanti, “Optimizing vehi-
cle dynamics co-simulation performance by introducing
mesoscopic traffic simulation,” Simulation Modelling
Practice and Theory, vol. 125, p. 102739, 2023.
[72] A. Rousset, B. Herrmann, C. Lang, and L. Philippe, “A
survey on parallel and distributed multi-agent systems
for high performance computing simulations,” Com-
puter Science Review, vol. 22, pp. 27–46, 2016.
[73] J. Vargas, S. Alsweiss, O. Toker, R. Razdan, and
J. Santos, “An overview of autonomous vehicles sensors
and their vulnerability to weather conditions,” Sensors,
vol. 21, no. 16, p. 5397, 2021.
[74] C. Goodin, D. Carruth, M. Doude, and C. Hudson,
“Predicting the influence of rain on lidar in adas,”
Electronics, vol. 8, no. 1, p. 89, 2019.
[75] S. Zang, M. Ding, D. Smith, P. Tyler, T. Rakotoarivelo,
and M. A. Kaafar, “The impact of adverse weather
conditions on autonomous vehicles: How rain, snow,
fog, and hail affect the performance of a self-driving
car,” IEEE Vehicular Technology Magazine, vol. 14,
no. 2, pp. 103–111, 2019.
[76] W. Li, Q. Zhang, J. Zhang, Z. Huang, X. Tian,
and D. Tao, “Toward real-world single image derain-
ing: A new benchmark and beyond,” arXiv preprint
arXiv:2206.05514, 2022.
15
[77] D. Dunkerley, “Rain event properties in nature and in
rainfall simulation experiments: a comparative review
with recommendations for increasingly systematic study
and reporting,” Hydrological processes: An interna-
tional journal, vol. 22, no. 22, pp. 4415–4435, 2008.
[78] S. Starik and M. Werman, “Simulation of rain in
videos,” in Texture Workshop, ICCV, vol. 2, 2003, pp.
406–409.
[79] C. Wang, Z. Wang, T. Xia, and Q. Peng, “Real-time
snowing simulation,” The Visual Computer, vol. 22, pp.
315–323, 2006.
[80] C. Sakaridis, D. Dai, and L. Van Gool, “Semantic foggy
scene understanding with synthetic data,” International
Journal of Computer Vision, vol. 126, pp. 973–992,
2018.
[81] D.-H. Paek, S.-H. Kong, and K. T. Wijaya, “K-radar: 4d
radar object detection for autonomous driving in various
weather conditions,” Advances in Neural Information
Processing Systems, vol. 35, pp. 3819–3829, 2022.
[82] M. Zhang, Y. Zhang, L. Zhang, C. Liu, and S. Khurshid,
“Deeproad: Gan-based metamorphic testing and input
validation framework for autonomous driving systems,”
in Proceedings of the 33rd ACM/IEEE International
Conference on Automated Software Engineering, 2018,
pp. 132–142.
[83] Z. Yang, Y. Chai, D. Anguelov, Y. Zhou, P. Sun,
D. Erhan, S. Rafferty, and H. Kretzschmar, “Surfelgan:
Synthesizing realistic sensor data for autonomous driv-
ing,” in Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, 2020, pp.
11 118–11 127.
[84] V. Mus, at, I. Fursa, P. Newman, F. Cuzzolin, and
A. Bradley, “Multi-weather city: Adverse weather stack-
ing for autonomous driving,” in Proceedings of the
IEEE/CVF International Conference on Computer Vi-
sion, 2021, pp. 2906–2915.
[85] Y. Li, Z.-H. Lin, D. Forsyth, J.-B. Huang, and S. Wang,
“Climatenerf: Physically-based neural rendering for ex-
treme climate synthesis,” arXiv e-prints, pp. arXiv–
2211, 2022.
[86] C. Früh and A. Zakhor, “An automated method for
large-scale, ground-based city model acquisition,” Inter-
national Journal of Computer Vision, vol. 60, pp. 5–24,
2004.
[87] N. Snavely, S. M. Seitz, and R. Szeliski, “Photo tourism:
exploring photo collections in 3d,” in ACM siggraph
2006 papers, 2006, pp. 835–846.
[88] S. Agarwal, Y. Furukawa, N. Snavely, I. Simon, B. Cur-
less, S. M. Seitz, and R. Szeliski, “Building rome in a
day,” Communications of the ACM, vol. 54, no. 10, pp.
105–112, 2011.
[89] F. Williams, M. Trager, J. Bruna, and D. Zorin, “Neural
splines: Fitting 3d surfaces with infinitely-wide neural
networks,” in Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition, 2021, pp.
9949–9958.
[90] J. Pan, X. Han, W. Chen, J. Tang, and K. Jia, “Deep
mesh reconstruction from single rgb images via topol-

ogy modification networks,” in Proceedings of the
IEEE/CVF International Conference on Computer Vi-
sion, 2019, pp. 9964–9973.
[91] A. Boulch and R. Marlet, “Poco: Point convolution
for surface reconstruction,” in Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern
Recognition, 2022, pp. 6302–6314.
[92] J. Huang, Z. Gojcic, M. Atzmon, O. Litany, S. Fidler,
and F. Williams, “Neural kernel surface reconstruction,”
in Proceedings of the IEEE/CVF Conference on Com-
puter Vision and Pattern Recognition, 2023, pp. 4369–
4379.
[93] M. Tancik, V. Casser, X. Yan, S. Pradhan, B. Milden-
hall, P. P. Srinivasan, J. T. Barron, and H. Kretzschmar,
“Block-nerf: Scalable large scene neural view synthe-
sis,” in Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, 2022, pp.
8248–8258.
[94] A. Mondal, P. Tigas, and Y. Gal, “Real2sim: Automatic
generation of open street map towns for autonomous
driving benchmarks,” in Machine Learning for Au-
tonomous Driving Workshop at the 34th Conference
on Neural Information Processing Systems (NeurIPS),
2020.
[95] Z. Ghodsi, S. K. S. Hari, I. Frosio, T. Tsai, A. Troccoli,
S. W. Keckler, S. Garg, and A. Anandkumar, “Gener-
ating and characterizing scenarios for safety testing of
autonomous vehicles,” in 2021 IEEE Intelligent Vehicles
Symposium (IV). IEEE, 2021, pp. 157–164.
[96] J. Gao, C. Sun, H. Zhao, Y. Shen, D. Anguelov, C. Li,
and C. Schmid, “Vectornet: Encoding hd maps and
agent dynamics from vectorized representation,” in Pro-
ceedings of the IEEE/CVF Conference on Computer Vi-
sion and Pattern Recognition, 2020, pp. 11 525–11 533.
[97] A. Kar, A. Prakash, M.-Y. Liu, E. Cameracci, J. Yuan,
M. Rusiniak, D. Acuna, A. Torralba, and S. Fidler,
“Meta-sim: Learning to generate synthetic datasets,” in
Proceedings of the IEEE/CVF International Conference
on Computer Vision, 2019, pp. 4551–4560.
[98] M. Haklay and P. Weber, “Openstreetmap: User-
generated street maps,” IEEE Pervasive computing,
vol. 7, no. 4, pp. 12–18, 2008.
[99] F. Poggenhans, J.-H. Pauls, J. Janosovits, S. Orf,
M. Naumann, F. Kuhnt, and M. Mayr, “Lanelet2: A
high-definition map framework for the future of auto-
mated driving,” in 2018 21st International Conference
on Intelligent Transportation Systems (ITSC). IEEE,
2018, pp. 1672–1679.
[100] S. Krauß, “Microscopic modeling of traffic flow: Inves-
tigation of collision free vehicle dynamics,” 1998.
[101] J. Erdmann, “Sumo’s lane-changing model,” in Model-
ing Mobility with Open Data: 2nd SUMO Conference
2014 Berlin, Germany, May 15-16, 2014. Springer,
2015, pp. 105–123.
[102] S. Panwai and H. Dia, “Comparative evaluation of mi-
croscopic car-following behavior,” IEEE Transactions
on Intelligent Transportation Systems, vol. 6, no. 3, pp.
314–325, 2005.
16
[103] A. Kesting, M. Treiber, and D. Helbing, “General
lane-changing model mobil for car-following models,”
Transportation Research Record, vol. 1999, no. 1, pp.
86–94, 2007.
[104] F.-Y. Wang, “Parallel control and management for intel-
ligent transportation systems: Concepts, architectures,
and applications,” IEEE Transactions on Intelligent
Transportation Systems, vol. 11, no. 3, pp. 630–638,
2010.
[105] S. Shalev-Shwartz, S. Shammah, and A. Shashua, “Safe,
multi-agent, reinforcement learning for autonomous
driving,” arXiv preprint arXiv:1610.03295, 2016.
[106] A. Knapitsch, J. Park, Q.-Y. Zhou, and V. Koltun,
“Tanks and temples: Benchmarking large-scale scene
reconstruction,” ACM Transactions on Graphics (ToG),
vol. 36, no. 4, pp. 1–13, 2017.
[107] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simon-
celli, “Image quality assessment: from error visibility
to structural similarity,” IEEE Transactions on Image
Processing, vol. 13, no. 4, pp. 600–612, 2004.
[108] S. Sahni and V. Thanvantri, “Parallel computing: Per-
formance metrics and models,” IEEE Parallel and Dis-
tributed Technology, vol. 4, no. 1, pp. 43–56, 1996.
Yueyuan LI received a Bachelor’s degree in Electri-
cal and Computer Engineering from the University
of Michigan-Shanghai Jiao Tong University Joint
Insitute, Shanghai, China, in 2020. She is working
towards a Ph.D. degree in Control Science and
Engineering from Shanghai Jiao Tong University.
Her main fields of interest are the security of the
autonomous driving system and driving decision-
making. Her current research activities include driv-
ing decision-making models, driving simulation, and
virtual-to-real model transferring.
Wei YUAN received his Master’s and Ph.D. degrees
in Automation from Shanghai Jiao Tong University,
Shanghai, China, in 2017 and 2021, respectively.
Presently, he is a postdoctoral researcher at Shanghai
Jiao Tong University.
His main fields of interest are autonomous driving
system, computer vision, deep learning, and vehicle
control. His current research activities include end-
to-end learning-based vehicle control and decision-
making.
Weihao YAN received a Bachelor’s Adegree in
Automation from Shanghai Jiao Tong University,
Shanghai, China, in 2020. He is working towards
a Ph.D. degree in Control Science and Engineering
from Shanghai Jiao Tong University.
His main fields of interest are autonomous driving
system, computer vision, and domain adaptation.
His current research activities include virtual-to-real
transfer learning, scene segmentation, and founda-
tion models.
 17

Qiyuan SHEN received a Bachelor’s degree in Au-

tomation from Northeastern University, Shenyang,
China, in 2022. He is currently working towards a
Ph.D. degree in Control Science and Engineering at
Shanghai Jiao Tong University.
His main fields of interest are the localization of
autonomous driving systems and SLAM. His current
research activities include multi-modal mapping,
cross-modal localization, and geometrical calibration
for vision and LiDAR.

Chunxiang WANG received a Ph.D. degree in

Mechanical Engineering from Harbin Institute of
Technology, China, in 1999. She is currently an
associate professor in the Department of Automation
at Shanghai Jiao Tong University, Shanghai, China.
She has been working in the field of intelligent
vehicles for more than ten years and has participated
in several related research projects, such as European
CyberC3 project, ITER transfer cask project, etc.
Her research interests include autonomous driving,
assistant driving, and mobile robots.

Ming YANG received his Master’s and Ph.D. de-

grees from Tsinghua University, Beijing, China, in
1999 and 2003, respectively. Presently, he holds
the position of Distinguished Professor at Shanghai
Jiao Tong University, also serving as the Director
of the Innovation Center of Intelligent Connected
Vehicles. Dr. Yang has been engaged in the research
of intelligent vehicles for more than 25 years.