Sasimitha R

20TUEE121

Advanced Charging System for Plug-in Hybrid Electric

Vehicles and Battery Electric Vehicles

Abstract
The increase of plug-in hybrid electric vehicles (PHEVs) and battery electric vehicles (BEVs) results in higher
electricity demand for their charging. In addition, the uncontrolled and timely concentrated charging is potential
to decrease the quality of electricity. This condition has encouraged the development of advanced charging
system, which is able to facilitate quick charging with minimum impacts on the electrical grid. This chapter
explains some issues related to charging of PHEVs and BEVs including some available charging systems,
charging behaviour and developed charging system employing battery for assistance during charging. In
analysis of charging behaviour, the effect of ambient temperature to charging rate is clarified. Higher ambient
temperature, such as during summer, leads to higher charging rate compared to one during winter. As advanced
charging system, a battery-assisted charging system for PHEVs and BEVs is also described. The evaluation results
in terms of their performance to facilitate a quick, simultaneous charging as well as reduce the stress of electrical
grid due to massively uncontrolled charging are also provided. This system is considered as one of the
appropriate solutions that can be adopted in the near future to avoid problems on electrical grid due to massive
charging of PHEVs and BEVs.

Keywords: simultaneous charging, battery assistance, charging behaviour, charging rate

1. Introduction

Electric vehicles (EVs) have received an intensive attention during the last decade due to their characteristics as
vehicles as well as other additional benefits that cannot be offered by conventional vehicles. A massive deployment
of electric vehicles can reduce the total consumption of fossil fuel, therefore, cuts down the greenhouse gas
emission [1]. In addition, as they have higher energy efficiency, lower running cost can be achieved than
conventional internal combustion-engine vehicles. Recently, value-added utilization of electric vehicles also has
been proposed and developed including the ancillary services for the electrical grid and electricity support to
certain energy management system [2–5]. Therefore, the economic performance of the electric vehicles can be
significantly improved.
Some literatures have proposed and described well the grid integration, especially the introduction of renewable
energy, and electric vehicles [6]. The fluctuating renewable energy sources, such as wind and solar, require a
fastresponse energy buffer to cover their intermittency as well as and to store the surplus electricity due to higher
supply side than demand side. Electric vehicles are considered as the appropriate resource to balance and store
these kinds of renewable energy sources [7]. The battery owned by the electric vehicles can absorb and release the
electricity from and to the electrical grid, respectively, to balance the electrical grid promptly.

In general, there are four types of electric vehicles currently running and developed: (i) conventional hybrid electric
vehicle (HEV), (ii) plug-in hybrid electric vehicle (PHEV), (iii) battery electric vehicle (BEV) and (iv) fuel-cell electric
vehicle (FCEV). HEV combines electric motor and internal combustion engine; hence, it is also fitted with a battery
to power the motor as well as store the electricity. The energy to power the motor comes from the engine and
regenerative breaking. However, recently, many HEVs have been redeveloped and shifted to PHEV due to the
excellent characteristics and higher flexibility of PHEV than HEV. Like HEV, PHEV also owns electric motor and
internal combustion engineOn the other hand, FCEV uses only electric motor like BEV. However, it utilizes
hydrogen as the main fuel that is stored in the tank. The oxidation of hydrogen produces electricity to power the
electric motor and if there is any surplus it is stored in the battery. In practice, as the hydrogen refuelling can be
performed in a very short time, almost similar to one of the gasoline refuelling, FCEV basically facilitates no
charging from the external charger.
2 Charging system for PHEV and BEV

Charging of PHEVs and BEVs correlates strongly with some parameters including charging devices, cost, charging
rate, location, time and grid condition. Therefore, relevant selection and distribution of chargers are very crucial to
be able to accommodate those parameters appropriately. PHEV and BEV basically share the same charging
standards; therefore, there is no peculiar charger features or requirements for each vehicle. Charger is designed to
be able to communicate with the vehicle to ensure the safety and appropriate electricity flow. In addition, charger
also monitors the earth leakage at the surrounding ground.

a. Level-1 charging
Table 1. Specification of charging standards for DC fast charging.
On the other hand, CCS standards, including Combo 1 and 2, are capable to facilitate both AC charging, including
level-1 and level-2 charging, and DC charging. It was developed by several European and US car manufactures in
around 2012. Society of Automotive Engineers (SAE) and European Automobile Manufacturer’s Association
(ACEA) strongly supported this initiative with the main purpose of facilitating both AC and DC charging with
only single charging inlet in the vehicle. CCS is able to facilitate AC charging at maximum charging rate of 43 kW
and DC charging at maximum charging rate of 200 kW with the future perspective of up to 350 kW [22]. CCS
chargers are currently installed mainly in Europe and the USA with approximate numbers of 2500 and 1000,
respectively.
.

achieved. To charge to battery SOC of 80% from about SOC of 30%, the required charging durations in both winter
and summer are 35 and 20 min, respectively. During summer, a relatively high charging rate (about 40 kW) can be
achieved up to an SOC of about 50%. However, the charging rate decreases moderately in accordance with the
increase of battery SOC. The charging rate at battery SOC of 80% is about 16 kW. On the other hand, during winter,
the charging rate reaches about 35 kW instantaneously in relatively short duration and Ba

Stationary battery releases its electricity to assist the charging. Therefore, vehicle charging is conducted using
electricity received from the electrical grid and discharged from the stationary battery. This mode is introduced
when a simultaneous quick charging of multiple vehicles occurs, especially in case of high electricity price.
Electricity in the battery discharging mode can be shown as follows:

Pgrid + Pbat = PQC1 + PQC2 + Ploss (1)

where Pgrid, Pbatt, Pqc and Ploss are electricity received from electrical grid, charged (negative value) or discharged
(positive value) electricity from stationary battery, discharged electricity for quick charging of vehicles and
electricity loss, respectively.

a. Battery charging mode

When there is remaining electricity (margin between the contracted power capacity and the used electricity) or
the electricity prices is getting down (because of surplus electricity in the grid, night time, etc.), the stationary
battery is charged to store electricity. The flow in this mode is expressed as Eq. (2).

Pgrid − Pbat = PQC1 + PQC2 + Ploss . (2) c. Battery idling mode

Stationary battery might be in the idling (stand-by) mode in case of several conditions: (a) contracted power
capacity can sufficiently cover the electricity demand for simultaneous charging of vehicles (low charging
demand), (b) stationary battery is empty or under
In contrast, in case of charging using the BAC, the first and second vehicles can enjoy almost the same charging
rate, and both vehicles reach battery SOC of 80% in almost the same time (about 35 min). Furthermore, the
electricity from electrical grid can be kept below the contracted power capacity, although the total charging rate
for both vehicles is larger than the contracted power capacity. This is because the battery assisting the system was
discharged to supply electricity. Hence, compared to a conventional charging system, BAC is able to achieve
highquality charging with higher charging rate during simultaneous charging.

Figure 5. Charging of two BEVs during summer with different charger: (a) conventional charger, (b) developed BAC.

6. Conclusion

As the number of PHEVs and BEVs is massively increasing, their charging becomes a very important issue due to
fluctuating and high demand of electricity. Therefore, it is very important to manage their charging through
coordinated charging, battery-assisted charging and demand respond. Among these three methods, coordinated
charging and demand respond require advanced theoretical development, massive demonstration and
coordination in the electrical grid, therefore, they need couple of years in the future for realization. On the other
hand, battery-assisted charging is considered very applicable in terms of economy and technology.

Charging behaviour of PHEV and BEV is strongly influenced by ambient temperature. Charging under relatively
high ambient temperature (summer) leads to higher charging rate; therefore, shorter charging time can be realized.
In addition, battery-assisted charger (BAC) has been developed especially to facilitate simultaneous charging of
multiple vehicles under certain limited contracted power capacity. The demonstration test of BAC proves that it
can facilitate high quality of charging, while minimizing the electrical grid stress due to massive and concentrated
charging of PHEVs and BEVs.

References

[1] Aziz M, Oda T, Kashiwagi T. Extended utilization of electric vehicles and their reused batteries to support
the building energy management system. Energy Procedia. 2015;75:1938-1943. DOI:
10.1016/j.egypro.2015.07.226

[2] White CD, Max Zhang K. Using vehicle-to-grid technology for frequency regulation and peak-load reduction.
Journal of Power Sources. 2011;196:3972-3980. DOI: 10.1016/j. jpowsour.2010.11.010

[3] Aziz M, Oda T, Mitani T, Watanabe Y, Kashiwagi T. Utilization of electric vehicles and their used batteries
for peak-load shifting. Energies. 2015;8(5):3720-3738. DOI: 10.3390/ en8053720
 Various sensors used in vehicles

Abstract:
At present, the modern automobile designing can be done using different types of sensors. These
are arranged into the car engine to recognize & solve possible problems like repairs, servicing, etc. The
sensors used in automobiles will check the function of the vehicle. An owner of a vehicle doesn’t know
the status of how many sensors are used in their vehicles. There are several largest sensor organizations
available worldwide, which can offer an innovative solution to the customers. In recent automobiles,
sensors are used for detecting as well as responding to change the conditions inside & outside of the
car. So that travelers in the vehicle can move efficiently and safely. By using these sensors data we can
increase comfort, efficiency, and safety

Introduction:
Automobile sensors are intelligent sensors which can be used to control and process the
pressure of oil, temperature, level of emission, coolant levels, etc. There are different types of
sensors used in automobiles, but knowing the working of these sensors is essential. In order to the
function of these sensors, here we have listed some popular sensors used in automobiles which include
the following.
• Mass airflow sensor
• Engine Speed Sensor
• Oxygen Sensor
• Spark Knock Sensor
• Coolant Sensor
• Manifold Absolute Pressure (MAF) Sensor
• Fuel Temperature Sensor
• Voltage sensor
• Camshaft Position Sensor
• Throttle Position Sensor
• Vehicle Speed Sensor

Mass Air Flow Sensor:

The MAF or Mass airflow sensor is one of the essential sensor used in automobiles. This sensor is
used in an engine of the car. This sensor can be controlled by a computer and can calculate the air density in the
engine. If the working of this sensor halts, then the running of the vehicle will be stopped. In addition, the usage
of petroleum will be high. These sensors are classified into two types namely vane meter & hot wire
Engine Speed Sensor:

The engine speed sensor in the automobile can be connected to the crankshaft. The main purpose
of this sensor is to monitor the crankshaft’s rotating speed. So that fuel injection & the engine timing can be
controlled. There are different ways for the vehicle engine to stop unexpectedly. So this sensor will stop that for
car drivers.

Oxygen Sensor:

Located in the exhaust stream, usually near the exhaust manifold and after the catalytic converter, the
oxygen sensor (or O2 sensor) monitors the content of exhaust gases for the proportion of oxygen. The information
is compared to the oxygen content of ambient air and is used to detect whether the engine is running a rich fuel
ratio or a lean one. The engine computer uses this information to determine fuel metering strategy and emission
controls.

Spark Knock Sensor:

The spark knock sensor is used to ensure whether the fuel is burning smoothly, otherwise, it will cause an
unexpected ignition. This ignition is very dangerous which will cause damage in the engine of the car like damage
of rings, head gasket, and rod bearings. Fitting these parts can be costly. So this sensor is used to save all the
troubles occurred in an engine of the car.

spark-knock-sensor

Coolant Sensor:

The coolant sensor is the most significant sensor used in automobiles. Because the computer depends on the
sensor inputs to control all the functions. For instance, turn ON/OFF the EFE system (Early Fuel Evaporation),
retard, spark advance, the flow of EGR, and canister purge.

coolant-sensor
Generally, this sensor can be connected on the board. If the sensor is failed, then there will be some indications
stalling, like poor fuel mileage, etc. So, the status of the sensor should be checked whether it is defective or not. If
it is damaged, then it will be a problem.

Manifold Absolute Pressure Sensor:

The short term of the manifold absolute pressure is MAP. The main function of this sensor in an automobile is to
monitor the load of an engine. Mostly, it measures the dissimilarity among manifold pressure. This can be received
from the outside pressure by the car to make sure that the car engine is capable to receive petroleum depending
on the changes within the pressure.

manifold-absolute-pressure-sensor

Fuel Temperature Sensor:

The fuel sensor is used to check the temperature of the fuel continually whether the fuel utilization is
optimum or not. If the fuel of the engine is cold, then it will take much time to burn due to its high density.
Similarly, if the fuel is warm then it will take less time to burn. Here, the main problem is the inflow varying levels.
So this can injure other parts of an automobile. This sensor will monitor the petroleum is injected at the right speed
and temperature. So that engine of the automobile works properly.

fuel-temperature-sensor

Voltage Sensor

Voltage Sensor is one type of sensor used in automobiles. The main function of this sensor is to manage
the car speed and to make sure the speed of is increased (or) decreased as required. So it is essential to have in your
car.

voltage-sensor

Throttle Position Sensor:

The throttle position sensor in automobile mainly uses feedback carburetion & electronic fuel injection (EFI). It
informs the computer regarding the throttle opening rate as well as the position of the relative throttle. This sensor
is a variable resistor, which is used to change the resistance as the throttle opens.

throttle-position-sensor
It is not complex to identify the faulty throttle position sensor symptoms. As there is a fall while speeding up, then
you can identify the faulty of the sensor. It is the major sign of a faulty throttle position sensor. Whenever you
change this sensor, you cannot adjust it properly.

Vehicle Speed Sensor:

As the name suggests, this VSS sensor has the capability to verify the speed of the car wheels. It is a type of
tachometer. This sensor is arranged within the anti braking system which is known as ABS. Additionally, the
output of this sensor is also utilized for the odometer to read the speed of the vehicle to control gears depending
on the vehicle speed.

vehicle-speed-sensor

Conclusion:
Thus, this is all about the different types of sensors used in automobiles. These sensors are smart systems which
are used for controlling different parts like coolant levels, temperature, the pressure of oil, levels of emission, etc.
These automobile sensors are complex to allow a variety of values, decide and process the accurate combination.
Here is a question for you, what is a parking sensor?
 Vehicle Intelligence System

Abstract :
One of the found techniques in Artificial Intelligence is intelligent agent technology. The concept of Agents
has become important both in Artificial Intelligence and in mainstream of Computer Science. Artificial
Intelligence is defined as the branch of Computer Science that deals in developing intelligent agents and
consequently, Intelligent Agent is defined as an automated agent that can replicate the functioning of human
beings. It perceives its environment, analyses it, and takes actions that will maximize the probability of its
success Intelligent agents continuously perform three functions: perception of dynamic conditions in the
environment; reasoning to interpret perceptions, solve problems, draw inferences, and determine actions. In this
paper, we begin with the basic terminologies related to intelligent agents, and then proceed towards the various
environments that an agent may have to perceive. We discuss about the Multi Agent Systems then finally site the
most extravagant intelligent agent i.e., Autonomous Cars with Google’s driverless car technology. Keywords—
Agent, intelligent agent, agent environment, multi agent system, autonomous car, driverless car.

Introduction:
The Encarta World English Dictionary says that the word Agent comes from the Latin word Agere, which
word gave also the words act, active, agile, agitate etc. This etymologic approach is interesting to have a first
idea of what it is. In general, we can view an agent to be an entity that performs some actions on behalf of
others on request. In real life, we come across a number of human agents such as Travel agent, Business
agent, Police agent etc. But here our concern is the study and development of computational automated
agents that will replicate the functioning of human agents. More generally, in their book "Artificial
intelligence, a modern approach", S. Russel and P. Norvig highlighted the importance of the environment,
defining an agent as something which percepts through sensors and acts through effectors. A definition
close to present-day reality is that of Ted Selker from the IBM Almaden Research Center: ‗An agent is a
software thing that knows how to do things that you could probably do yourself if you had the time‘. Agents
come in many different flavors.
Depending on their intended use, agents are referred to by an enormous variety of names, e.g.,
knowbot, softbot, taskbot, userbot, robot, personal (digital) assistant, transport agent, mobile agent, cyber
agent, search agent, report agent, presentation agent, navigation agent, role agent, management agent,
search and retrieval agent, domain-specific agent, packaging agent. The word ‗agent‘ is an umbrella term
that covers a wide range of specific agent types. Most popular names used for different agents are highly
non-descriptive. It is therefore preferable to describe and classify agents according to the specific properties
they exhibit. Figure-1 is a pictorial representation of functioning of an agent. 2. Characteristics of agents The
idea of intelligent software agents has captured the popular imagination. Let‘s address the question of what
makes an agent intelligent by explaining the characteristics of intelligent agents. 2.1 Primary characteristics
of agents The most important attributes of an agent are referred to as primary attributes; less important or
secondary attributes, are listed below. The primary attributes include the following— Autonomy: reflects the
ability of agents to operate on their own, without immediate human guidance, although the latter is
sometimes invaluable.
Co-operation: refers to the ability to exchange high-level information with other agents: an attribute which is
inherent in multiple agent systems (MAS). Learning: refers to the ability of agents to increase performance
over time when interacting with the environment in which they are embedded. Mobility: This refers to the
property of the agent of being movable to and from various places.
Secondary characteristics of agents:
Agents can be classified according to a number of other attributes, which could be regarded as
being secondary to the ones described above. Rather than a comprehensive list, some examples of secondary
attributes that agents may exhibit will be given. Agents may be classified, for example, by their pro-active
versatility – the degree to which they pursue a single goal or engage in a variety of tasks. Furthermore, one
might attribute social abilities to agents, such as truthfulness, benevolence and emotions, although the last is
certainly controversial. One may also consider mental attitudes of agents, such as beliefs, desires, and
intentions. By combining the primary and secondary properties and characteristics, hybrid agents and
heterogeneous agents can be constructed. With hybrid agents two or more properties and/or attributes are
combined in the design of a single agent. This results in the combination of the strengths of different agent-
design philosophies in a single agent, while at the same time avoiding their individual weaknesses. It is not
possible to separate such an agent into two other agents.Heterogeneous agents combine two or more
different categories of agents in such way that they interact via a particular communication language.
History of intelligent agents
The notion of intelligent agents has been around for the past 50 years; it was first introduced by McCarthy
(1956, 1958) and later coined by the prominent MIT Lincoln Laboratory computer scientist Oliver Selfridge.
In the 1950s, John McCarthy conceived the Advice Taker (McCarthy 1958), a software robot living and
working in a computer network of information utilities (much like today‘s Internet).When given a task by a
human user, the software robot could take the necessary steps or ask advice from the user when it got stuck.
The futuristic prototypes of intelligent personal agents, such as Apple Computer‘s Phil or Microsoft‘s Bob,
perform complicated tasks for their users following the same functions laid out by McCarthy in his Advice
Taker.
Although modern approaches to software agency can trace their roots to these earlier visions, current
research started in the mid-1980s and has been influenced by work done in a number of fields including
artificial intelligence (e.g., reasoning theory and artificial life), software engineering (e.g., object-oriented
programming and distributed processing), and humancomputer interaction (e.g., user modeling and
cognitive engineering).
Agent environments
The critical decision an agent faces is determining which action to perform to best satisfy its design
objectives. Agent environments are classified based on different properties that can affect the complexity of
the agent‘s decision-making process. They include— Accessible vs. inaccessible An accessible environment is
one in which the agent can obtain complete, timely and accurate information about the state of the
environment. The more accessible an environment, the less complicated it is to build agents to operate
within it. Most moderately complex environments are inaccessible.
Deterministic vs. non-deterministic
Most reasonably, complex systems are nondeterministic – the state that will result from an action is not
guaranteed even when the system is in a similar state before the action is applied. This uncertainty presents
a greater challenge to the agent designer.
Episodic vs. non-episodic
In an episodic environment, the actions of an agent depend on a number of discrete episodes with no link
between the performances of the agent in different scenarios. This environment is simpler to design since
there is no need to reason about interactions between this and future episodes; only the current environment
needs to be considered.
Static vs. dynamic
Static environments remain unchanged except for the results produced by the actions of the agent. A
dynamic environment has other processes operating on it thereby changing the environment outside the
control of the agent. A dynamic environment obviously requires a more complex agent design. Discrete vs.
continuous If there are a fixed and finite number of actions and percepts, then the environment is discrete. A
chess game is a discrete environment while driving a taxi is an example of a continuous one. From the
above, it is clear that the combination of inaccessible, non-deterministic, non-episodic, dynamic and
continuous environments is the toughest to perceive and an agent capable of perceiving this combination of
environments will have the highest level of intelligence.
Types of agents
Based on the way an agent handles a request or takes an action upon perceiving its environment, intelligent
agents can be classified into four categories—
i. Simple reflex agents
ii. Agents keeping track of the World
iii. Goal based agents
iv. Utility based agents We shall discuss each one of them in brief details.
Simple reflex agents
A simple reflex agent is an agent that performs actions based on certain conditions being fulfilled. It
monitors it environment, and performs the same action every time the same condition occurs. A simple
reflex agent can be implemented by the simple conditional clauses such as ‗if‘. For example, for a car, we can
implement the following condition— if car-in-front-is-braking then initiate-braking The above statement
explains the fact that if the car running in front brakes and the brake lights are on, then the driver of the car
behind this should initiate brakes to avoid collision. The functioning of simple reflex agents has been
depicted in figure-2.
Agents keeping track of the World
The simple reflex agent described before will work only if the correct decision can be made on the
basis of the current percept. But it is very often that the external environment changes without notice (in case
of dynamic environments) and then simple reflex agents fail to act rationally. For example, if the brake lights
come on, it does not necessarily mean that the car in front in stopping, it may be on because it wants to
change its directions, which is a very common case in real life. In such cases, the agent may need to maintain
some internal state information in order to distinguish between world states that generate the same
perceptual input but nonetheless are significantly different. Updating this internal state information as time
goes by requires two kinds of knowledge to be encoded in the agent program. First, we need some
information about how the world evolves independently of the agent—for example, that an overtaking car
generally will be closer behind than it was a moment ago. Second, we need some information about how the
agent‘s own actions affect the world—for example, that when the agent changes lanes to the right, there is a
gap (at least temporarily) in the lane it was in before, or that after driving for five minutes northbound on
the freeway one is usually about five miles north of where one was five minutes ago. Figure-3 shows the
functioning of Agents that keep track of the World.
Goal based agents
Knowing about the current state of the environment is not always enough to decide what to do. For
example, at a road junction, the taxi can turn left, right, or go straight on. The right decision depends on
where the taxi is trying to get to. In other words, as well as a current state description, the agent needs some
sort of goal information, so that it can act accordingly to fulfill that goal. Figure-4 represents the functioning
of a Goal based agent.
Utility based agents
Goals alone are not really enough to generate high quality behavior. For example, there are many action
sequences that will get the taxi to its destination, thereby achieving the goal, but some are quicker, safer,
more reliable, or cheaper than others. Goals just provide a crude distinction between ―happy‖ and
―unhappy‖ states, whereas a more general performance measure should allow a comparison of different
world states (or sequences of states) according to exactly how happy they would make the agent if they
could be achieved. Because ―happy‖ does not sound very scientific, the customary terminology is to say
that if one world state is preferred to another, then it has higher utility for the agent. Utility is therefore a
function that maps a state onto a real number, which describes the associated degree of happiness. A
complete specification of the utility function allows rational decisions in two kinds of cases where goals have
trouble. First, when there are conflicting goals, only some of which can be achieved (for example, speed and
safety) the utility function specifies the appropriate trade-off. Second, when there are several goals that the
agent can aim for, none of which can be achieved with certainty, utility provides a way in which the
likelihood of success can be weighed up against the importance of the goals.
 Figure 6: Multi-agent system

Figure 7: Google‘s driverless car technology

Acknowledgement

It our heartiest pleasure to present our sincere thanks and gratitude to Dr. Priyesh Kanungo, Reader and
Senior System Engineer, Devi Ahilya University, Indore, India for his constant support and encouragement
during the course of Artificial Intelligence and Neural Networks. We are also very thankful to Dr. V. B.
Gupta, Head, School of Future Studies and Planning, Devi Ahilya University, Indore, India for his constant
support and help. Finally, we would like to thank our friends and family members for their valuable
presence in our life
 Architecture for vision based autonomous road
vehicles features and applications.

Abstract:

In the past decade, autonomous vehicle systems (AVS) have advanced at an exponential rate, particularly
due to improvements in artificial intelligence, which have had a significant impact on social as well as road safety
and the future of transportation systems. However, the AVS is still far away from mass production because of the
high cost of sensor fusion and a lack of combination of top-tier solutions to tackle uncertainty on roads. To reduce
sensor dependency and to increase manufacturing along with enhancing research, deep learning-based
approaches could be the best alternative for developing practical AVS. With this vision, in this systematic review
paper, we broadly discussed the literature of deep learning for AVS from the past decade for real-life
implementation in core fields. The systematic review on AVS implementing deep learning is categorized into
several modules that cover activities including perception analysis (vehicle detection, traffic signs and light
identification, pedestrian detection, lane and curve detection, road object localization, traffic scene analysis),
decision making, end-to-end controlling and prediction, path and motion planning and augmented reality-based
HUD, analyzing research works from 2011 to 2021 that focus on RGB camera vision. The literature is also analyzed
for final representative outcomes as visualization in augmented reality-based head-up display (AR-HUD) with
categories such as early warning, road markings for improved navigation and enhanced safety with overlapping
on vehicles and pedestrians in extreme visual conditions to reduce collisions. The contribution of the literature
review includes detailed analysis of current state-of-the-art deep learning methods that only rely on RGB camera
vision rather than complex sensor fusion. It is expected to offer a pathway for the rapid development of cost-
efficient and more secure practical autonomous vehicle systems.

Keywords: autonomous controlling; deep learning; decision making; intelligent vehicle; perception; self-driving

Introduction
Recently, the autonomous vehicle system (AVS) has become one of the most trending research domains that
focus on driverless intelligent transport for better safety and reliability on roads [1]. One of the main motives for
enhancing AVS developments is its ability to overcome human driving mistakes, including distraction, discomfort
and lack of experience, that cause nearly 94% of accidents, according to a statistical survey by the National Highway
Traffic Safety Administration (NHTSA) [2]. In addition, almost 50 million people are severely injured by road
collisions, and over 1.25 million people worldwide are killed annually in highway accidents. The possible reasons
for these injuries may derive from less emphasis on educating drivers with behavior guidance and poorly
developed drivers’ training procedures, fatigue while driving, visual complexities, that is, human error, which can
be potentially solved by adopting highly efficient self-driving vehicles [3,4]. The NHTSA and the U.S. Department
of Transportation formed the SAE International levels of driving automation, identifying autonomous vehicles
(AV) from ‘level 00 to the ‘level 50 [5], where levels 3 to 5 are considered to be fully AV. However, as of companies
such as Toyota, BMW and Tesla are already working on launching AVSes within the first half of this decade [12].
Although different sensors such as radar, lidar, geodimetric, computer views, Kinect and GPS are used by
conventional AVS to perceive the environment [13–17], it is indeed expensive to equip vehicles with these sensors
and the high costs of these sensors are often limited to on-road vehicles [18]. Table 1 shows a comparison of three
major vision sensors based on a total of nine factors. While the concept of driverless vehicles has existed for
decades, the exorbitant costs have inhibited development for large-scale deployment [19]. To resolve this issue and
build a system that is cost efficient with high accuracy, deep learning applied vision-based systems are becoming
more popular where RGB vision is used as the only camera sensor. The recent developments in this field of deep
learning have accelerated the potential of profound learning applications for the solution of complex real-world
challenges
 VS VR FoV Cost PT DA AAD FE LLP AWP
Camera High High Low Medium Medium High High Medium Medium
Lidar High Medium High Medium High Medium Medium High Medium
Radar Medium Low Medium High High Low Low High Low

In this systematic review paper, a broad discussion and survey of the implementation of deep learning are applied
to aspects of AVS such as vehicle detection (VD), traffic signs and light identification (TSL), pedestrian detection
(PD), lane detection and tracking (LDT), traffic scene analysis (TSA), decision making (DM), end-to-end controlling
and prediction (E2EP), path and motion planning (PMP) and augmented reality-based HUD (ARH) analyzing
research articles from 2011 to 2021 research articles on deep learning-applied AVS to reduce the dependency on
sensor fusion and the high cost of manufacturing and to enhance the focus on developing a level 5 autonomous
driving vehicle. We represent and thoroughly discuss the best deep learning algorithms for each domain, provide
solutions to their limitations and analyze their performance for increasing practical implementation concepts.
Moreover, this systematic review explored the most complete and predominate domains compared to other
surveys from [21–33] (shown in Table 2) that indicates its impact on AVS implementing deep learning where the
review article covered all aspects of the human–machine interface (HMI). The overall contribution of the research
is set out below:
• Analyzed recent solution of state-of-the-art deep learning algorithms for
cost-efficient AVS using RGB camera.

• Detailed literature review covering major domains and most subcategories

to decrease vision sensor complexities.

• Discussed the key advantages and disadvantages of deep learning methods

applied to AVS.

Survey Coverage
Ref. Year
VD LRCD PD TSL E2EC TSA PMP DM ARH HMI

[21] 2019 4 X 4 X X X 4 4 X 4
[22] 2020 X X X 4 X 4 4 4 X X
[23] 2016 X X X X 4 X 4 X X X
[24] 2020 X X X X 4 X 4 X X 4
[25] 2018 X X X X 4 X 4 X X X
[26] 2018 X X X X 4 X 4 X X X
[27] 2021 X X X 4 4 X 4 4 X 4
[28] 2020 4 4 4 4 4 4 X X X 4
[29] 2018 X X 4 X X X X X X X
[30] 2020 4 4 4 X X X 4 X X X
[31] 2020 X 4 4 4 4 4 4 X X X
[32] 2021 X 4 X X X X 4 4 X X
[33] 2020 4 X X X 4 4 4 X X 4
Ours 2022 4 4 4 4 4 4 4 4 4 X

Methodology
Review Planning
The study is based on a systematic review methodology, an approach for analyzing and
evaluating accessible studies related to a particular issue of current research where the core three phases are
preparing the review, conducting the review, and creating a report that summarizes the review. In this systematic
review paper, the researchers have included 142 papers containing deep learning and belonging to a different
domain of AVS. To finalize the papers, we initially focused on the entire domain of autonomous driving, then we
restricted our search to the usage of deep learning in AVS. Only papers with full text in English from renowned
journals, conferences and book chapters that were published between 2011 and 2021 were selected. Due to an
increase in the scope of advanced autonomous transportation, we finally limited our search to the vision-based
application of deep learning in AVS, and the rest were rejected. We also took the most complete edition to avoid
dealing with duplication. The key plan and protocol of the review includes source of data, searching criteria and
procedure, research questions, data selection and data extraction.

Sources of Data
Research papers were gathered from various famous research databases to incorporate specific field
and research questions. Irrelevant research papers that could not address or endorse our research questions were
dismissed. To achieve a broad coverage for the literature review, we used the following databases as our key
resources: Web of Science, Scopus, IEEE Xplorer, ScienceDirect, MDPI, Springer, Wiley Library and ACM.

Research Questions
Research questions were formed to refine the survey and maintain the aim of the topic. The following
research questions are answered throughout the discussion in the different sections of the paper.

• How does deep learning reduce sensor dependency?

• How are on-road objects detected and localized?

• What decision-making processes are solved for AVS?

• How does deep learning contribute to end-to-end controlling and path

planning?

• How should final outcomes be represented in AR-HUD?

Searching Criteria
To find research papers according to the methodology, a pattern was followed to gather suitable papers
which were mostly necessary for our study. We adopted a Boolean searching method with multiple AND, OR in
the advance search options of each data source. During the search for the relevant papers, we selected
“Autonomous Driving” and “Autonomous Vehicle” or “Intelligent Vehicle” or “Self-Driving” and “Deep
Learning” as the main phrases. For a further refined search, various keywords were included to obtain the desired
research papers according to our aim in this review. The following queries were developed based on Boolean
operations:
• ((Autonomous Driving) OR (Autonomous Vehicle) OR (Intelligent Vehicle) OR (SelfDriving) AND (Deep
Learning) AND (Object) AND ([Vehicle] OR [Pedestrians] [Traffic Sign] AND [Traffic Light]))

• ((Autonomous Driving) OR (Autonomous Vehicle) OR (Intelligent Vehicle) OR (Self-Driving) AND (Deep

Learning) AND ([Traffic Scene] OR [Localization] OR [Segmentation]))

• ((Autonomous Driving) OR (Autonomous Vehicle) OR (Intelligent Vehicle) OR (Self-

Driving) AND (Deep Learning) AND (Lane) AND ([Track] OR [Shift] OR [Segmentation]))
• ((Autonomous Driving) OR (Autonomous Vehicle) OR (Intelligent Vehicle) OR (SelfDriving) AND (Deep
Learning) AND (Control) AND ([Steering] OR [Motion]))

• ((Autonomous Driving) OR (Autonomous Vehicle) OR (Intelligent Vehicle) OR (SelfDriving) AND ([Deep

Learning] OR [Deep Reinforcement Learning]) AND (Decision Making) AND ([Uncertainty] OR [Lane
Keeping] OR [Overtaking] OR [Braking] OR [Acceleration]))

• ((Autonomous Driving) OR (Autonomous Vehicle) OR (Intelligent Vehicle) OR (SelfDriving) AND (Deep

Learning) AND ([Augmented Reality] AND [Head Up Display] OR [HUD]))

2.5. Searching and Extraction Procedure

The selection procedure for choosing papers includes four core iteration filtering processes. As the aim of the study
is to discuss implementation of deep learning and comprehensive literature searches to analyze the frameworks
and system designs, first, a total of 760 papers were selected from eight data sources based on the queries
mentioned in the searching criteria (Section 2.4). Web of Science had the highest 151 and ACM had the lowest 40
papers. Then, the selected papers had to be processed through an eligibility stage where 209 duplicated papers
were eliminated at first.
Furthermore, 121 papers were screened out during abstract scanning and 276 papers were chosen after full text
reading. In the next iteration, studies containing domains of deep learning in relation to AVS were selected where
all the papers were published between 2011 and 2021. The final dataset contains a total of 142 papers that covers
the literature on the implementation of deep learning methods for AVS. The structure of the whole selection
process is presented in Figure 1. Table 3 presents the final calculation for the selection of 142 papers according to
these steps and based on the most relatable topics and in-depth analysis.

Analysis of Publication by Year

Out of 142 final papers for review, the studies published between 2011 and 2021 were selected. The year
2019 had the highest number of selected research papers, with 31, which is 17.9% of the total, and 2011 had the
lowest number of papers (2). The distribution of publication is visualized in Figure
2.7. Analysis of Publication by Country
Among the 142 selected papers for the literature review, 56 countries contributed to autonomous
vehicle system development. Figure 3 shows the top 10 countries and the number of papers they contributed before
the final selection. The graphical representation shows that China made the largest contribution, with 34 papers,
and the USA contributed 21 papers, which was the second largest.
Conclusions
The results of the mixed method studies in the field of implementation and application of deep learning algorithms
for autonomous driving systems help us to achieve a clear understanding of the future of transportation. These
results prove that it has the ability to provide intelligent mobility for our constantly evolving modern world as
deep learning was one of the key components to resolve the limitations and bottlenecking of traditional techniques.
Despite containing a good number of studies on autonomous driving systems, only a few make an impact on
recent developments in the autonomous driving industry. To overcome this challenge and build a safer and more
secure sensor-independent transportation system with the aim of building infrastructure of futuristic smart cities,
in this paper, through a systematic review of the literature, studies of AV were selected that used deep learning
and the field was reviewed in terms of decision making, path planning and navigation, controlling, prediction and
visualizing the outcomes in augmented reality-based head-up displays. We analyzed the existing proposal of deep
learning models in real-world implementation for AVS, described the methodologies, designed the flow of
solutions for the limitations of other methodologies, and compared outcomes and evaluation techniques.
Nevertheless, as the research field of autonomous driving systems is still growing, many of the theoretical
methodologies were not applied practically, but along with the research trend of this expanding field, these are
potentially excellent solutions that require further development. Thus, the large-scale distributions of the paper in
the major areas of autonomous driving systems will be essential for further research and development of the
autonomous vehicle industry into a cost-efficient, secure intelligent transport system.