Artificial intelligence

algorithms and challenges

for autonomous vehicles
 August 3, 2020 Anton Hristozov
Autonomous vehicles (AVs) are starting to become a real possibility in some
parts of industry. Agriculture, transportation and military are some of the
examples. The day when we are going to see autonomous vehicles in everyday 
life for the regular consumer is quickly approaching. Many of the operations that
vehicles have to perform are based on sensor information and some artificial
intelligence (AI) algorithms. Vehicles need to collect data, plan their trajectory
and execute the trajectory. These tasks, especially the last two require non-
traditional programming approaches and rely on machine learning techniques,
which are part of AI. This two-part series illustrates the AI applications that
make autonomous vehicles a reality, presenting their challenges and
accomplishments. Part 1 offered an analysis of AI and its applications in AVs.
This article explores the algorithms and challenges of AI in AVs.

3. AI algorithms used in autonomous vehicles

3.1. Route Planning and Control Algorithms
Traditional algorithms from computer science that are heuristic in nature can be
used for this task. These are algorithms like Bellman-Ford and Dijkstra’s
algorithm (Bugala, 2018). For these algorithms to work we need to have
localization of the vehicle during the whole time. Localization is accomplished
through sensors such as GPS as well as simultaneous localization and mapping
(SLAM) techniques.
SLAM is used when there is no GPS availability such as underground or enclosed
spaces for example. SLAM generates a map of the environment and at the same
time estimates the state of a vehicle (Cadena et al., 2016).  The map is composed
of landmarks or obstacles  in order to represent the environment. SLAM is used
in applications where the map is not available and needs to be created. It uses
sensors and special algorithms that create models of the data in order to
produce the map.
3.2. Object Detection Algorithms
Object detection is one of the most important tasks that AI has to handle in a
moving vehicle. These algorithms are an area of active research and they rely on
different sensors. Object detection can be based on cameras or lidars, radars
and other type of sensors. The algorithms used are normally deep learning
algorithms which use some type of a neural network to do the job (Redmon et
al., 2016).
ADVERTISING

One requirement for this task is that it needs to be fast.  The reason is because
there is a succession of images that need  to be processed as the vehicle moves.
Some of the latest techniques here are based on the use of convolutional neural
networks (CNN). These are R-CNN, Fast R-CNN and You Only Look Once (Yolo)
methods (Redmon et al., 2016). RCNN first finds regions that contain potential
objects in an image and then tries to analyze each region. This makes R-CNN
somewhat slow and that is why there have been fast R-CNN methods developed
and the Yolo method. Yolo works simultaneously to find the regions and to
classify the objects in the regions by using a single convolutional neural
network.This makes Yolo very fast compared to the other methods. In addition
Yolo is able to see the entire image and does not suffer from the issues in R-CNN
like mistaking background images for objects.
3.3. Decision Making Algorithms
Decision making determines the actions of the vehicle based on information
from sensors. A vehicle constantly makes decision, based on its policy and the
environment. The algorithms used for decision making are the following
(Bugala, 2018):

 Decision Trees
 Support Vector Machine (SVM) Regression
 Deep Reinforcement Learning

4. Challenges of AI in Autonomous Vehicles
Some of the challenges of using artificial intelligence algorithms for
autonomous vehicles are the same challenges that are universal for many other
AI applications. The autonomous vehicles domain introduces some additional,
unique challenges. These are concepts like real time, safety and machine ethics.
The following subsections describe these areas in detail.
4.1 Real  time response
Real time systems are specific class of embedded systems which have the
characteristics to produce outputs or reactions within certain time restrictions.
They have to be deterministic and minimalist in their design, so that they can
always meet the expected real time behavior. For this to happen they often use
special real time operating systems (RTOS) or bare metal executives that
interact directly with the hardware, avoiding interpreted languages and
dynamic memory allocation. In some cases real time systems intentionally use a
subset of a programming language in order to guarantee speed and
determinism.
The artificial intelligence solutions are normally on the opposite side of the
spectrum — using higher level programming languages, full operating system
dependencies, techniques like dynamic memory allocation and garbage
collection. This creates a challenge in making real time guarantees with these
systems.
Another aspect of these systems is whether they are centralized or distributed
as described in (Carmody, Thomas, 2019). Centralized systems are easier to
architect, but rely heavily on internal communications and a powerful central
processor. Distributed system use dedicated CPUs that can handle different
subsystems and sensors thus alleviating the need for a complex and powerful
central CPU. Distributed architecture allow for systems with lower power
consumption. They are also more flexible and can be cheaper.
4.2 Computational complexity
Artificial intelligence algorithms, especially deep learning, require special
hardware solutions due to the amount of data and the complexity of
computations. Hardware, like graphic processing units (GPU) or tensor
processing units (TPU) can be highly optimized for fast parallel computation. 
The speed comes at a price of higher energy consumption and higher cost. Even
with using specialized hardware there is still no certainty that a particular
algorithms will be able to reach a solution in real time constraints. Therefore the
selection of algorithms with regards to their complexity and CPU demands is a
serious factor in real time systems. Usage of CPU computation and memory are
the measures that can help determine if an algorithm is suitable for use in a
typical CPU used in modern vehicles.
4.3 Black box behavior
AI algorithms have been criticized for being harder to analyze than standard
computer algorithms. This comes from the fact that AI algorithms have a higher
level of complexity and rely on a lot of data. A complex neural network can
perform several AI tasks without understanding the process it controls.
In fact a deep neural network is approximating a function and this is part of its
nature and explains why it can be so successful. The deep  neural networks can
have tens of thousands of nodes that can be trained to reach certain state. In
addition they can have many hidden layers and many inputs and outputs. All
this leads to problems when a system crashes and there is the need to do
forensics analysis and find out the precise cause for the crash. This is an active
area of research and successes in this area will solve many legal and technical
issues related to using AI.
One idea is to use hybrid solutions which combine AI with traditional control
algorithms (Abduljabbar and Dia, 2019). This issue is particularly important for
accident prediction and handling (Narayanan, 2019). The decision history of
why one decision was  made versus another has to be justified by  the AI
algorithm and this is hard or impossible if the algorithm can be only analyzed
and viewed as a black box.
4.4 Accuracy  and Reliability
The computer vision applications used in autonomous vehicles may not be
ready for prime time. The reason is that they can work in pristine conditions, but
can fail with some even small disturbances at the sensor inputs. The training of
AI algorithms happens slowly with training data that has certain characteristics.
Changing the data can make the classification and prediction algorithms
change their behavior dramatically with catastrophic results. For example a
person who carries a large bag, may not be perceived as a person by the vehicle.
In addition it is hard to predict what can happen with arbitrary data that can
enter the system. This issue has been exploited by attackers who were able to
fool deep learning algorithms by data in images that is not seen by humans, but
makes the neural network stop classifying properly. There are clearly a lot of
improvements needed to improve the reliability and accuracy of the machine
learning algorithms.
4.5 Safety
The complexity of AI can lead to problems with safety. A more complicated
system is one that is more difficult to develop, test and deploy.  This   is only
part of the problem. Another reality is that new safety standards are now
emerging and not yet adopted by the transportation industry (Carmody,
Thomas, 2019).
What makes things even more complicated is that analysis and verification of AI
systems is something that is still in its early stages and may not be tractable,
because of their complexity. In general, formal verification is hard, even for
traditional software systems and it is even harder for AI based software. The
problem with creating robust, verified and validated AI solutions is something
that we have to solve in general, not only for autonomous vehicles, but for all AI
applications.
4.6 Security and AI
AI systems are so central to the autonomous vehicle that their security is
directly related to robustness of the whole system. Machine learning systems
are susceptible to attacks and there have been many examples of such cases
(Newman, 2019).
The biggest threats are when an adversary can manipulate data that comes
from sensors to the vehicle thus making the system make incorrect decisions.
This behavior can be improved by training systems with data that has some
adversarial characteristics. An experiment with a deep learning model for
recognizing road signs showed that ”adding just a few black and white stickers
to a stop sign tricked the algorithm into thinking it was a 45mph speed limit
sign.” (Newman, 2019,p.17).
A big threat for affecting how AI operates in an autonomous vehicle is through
perturbing sensor operation, which leads to changes in the sensor data stream
and thus can completely confuse the AI algorithms. Solutions like hybrid
systems, fuzzy logic and traditional control systems based on models to
complement AI are some ways that researchers are trying to explore. The stakes
are high, since AI can be found in some vehicle subsystems already and will be
in every vehicle in the near future.
4.7 Ethics and AI
At this point, the ethical aspects of AI implementations in autonomous and
semi-autonomous vehicles are not mature and are not significantly developed.
Ethical values are a human quality, which is hard to formalize and implement in
machines. In other words deciding what is right or wrong is kind of nebulous to
a machine (Narayanan, 2019). Full development of machine ethics may be
decades away. This is an area of active research which  is a cross discipline
between human psychology, machine learning and social policy. Possible some
regulations can help in this direction while the technology becomes more
mature.

5. Conclusion
AI solutions are everywhere today. They are part of devices like Alexa and
Google Home, part of grass cutting robots and part of autonomous vehicles we
expect to drive soon. The challenges and achievements of AI are common for
many industries, but in the transportation industry and especially with
autonomous vehicles we have extra challenges related to security and safety.
Part of it is because of the complexity of the task to create a self driving vehicle
and part of it is because of the level of development of the field of AI and its
suitability to solve complex problems like object detection, route planning and
real time decision making. Many of these obstacles will be resolved as we move
forward towards more and more penetration of autonomous solutions in
certain sectors of the economy. Maybe the right question to ask for this industry
is ”Are we going to get there safely?” and not just ”Are we there yet ?”.

References
Abduljabbar, R., & Dia, H. (2019). Applications of Artificial Intelligence in
Transport: An Overview. Sustainability, 1(11), 189.
Beachler, R. (2019). How Man-Made Neural Networks Emulate Biology to
Advance AI and ML. Electronic Design.
Boucherat, X. (2019). Suppliers Forge the Relationship between AI and
Sensors. Automotive World.
Bugala, M. (2018). Algorithms Applied in Autonomous Vehicle
Systems. Szybkobiene Pojazdy Gasienicowe, 50(4), 119–138.
Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., &
Leonard, J. (2016). Past, Present, and Future of Simultaneous Localization
And Mapping: Towards the Robust-Perception Age. IEEE Transactions on
Robotics, 32(6), 309–1332.
Carmody, Thomas. (2019). Whitepaper: AI in the driving Seat: Key Challenges for
the Future of Advanced Driver Assistance. Cambridge Consultants.
Carol, T. (2017). Insurance and the Evolution of Automated Driving
Systems. National Association of Mutual Insurance Companies, 2–11.
Khayyam, H., Javadi, B., Jalili, M., & Jazar, R. (2019). Artificial Intelligence and
Internet of Things for Autonomous Vehicles. Nonlinear Approaches in
Engineering Applications  (Book), 39–68.
Lentin, J. (2017). Ros robotics projects. Packt Publishing.
Narayanan, A. (2019). Ethical Judgement in Intelligent Control Systems for
Autonomous Vehicles. Australian New Zealand Control Conference (ANZCC),
Auckland, New Zealand, 231–236.
Newman, J. (2019). Whitepaper: Toward AI Security – Global Aspirations for a
More Resilient Future. CLTC White Paper Series.
Prytz, R. (2014). Machine learning methods for vehicle predictive maintenance
using off-board and on-board data. Halmstad University Dissertations no. 9,
1–32.
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A.  (2016).  You  Only Look Once:
Unified, Real-Time Object Detection. IEEE Conference on Computer Vision
and Pattern Recognition (CVPR),  Las  Vegas, NV, 779–788.
Sagar, V., & Nanjundeswaraswamy, T. (2019). Artificial Intelligence in
Autonomous Vehicles – A Literature Review. I-manager’s Journal of Future
Engineering & Technology, 14  (3), 56–62