2020 Intermountain Engineering, Technology and Computing (IETC)

Self-Driving Cars: A Platform for Learning and

Research
1st Jacob Newman 2nd Zheng Sun 3rd Dah-Jye Lee
Brigham Young University Brigham Young University Brigham Young University
Dept. of ECEn Dept. of ECEn Dept. of ECEn
Provo, Utah, USA Provo, Utah, USA Provo, Utah, USA
jacobnewman86@gmail.com zhengsun@pm.me djlee@byu.edu

Abstract—We present a self-driving car platform that can be
used by both students and researchers. Students can use the
platform to learn about the technology and challenges that exist
in the self-driving car industry, while researchers can use it to
test and iterate through potential solutions to self-driving car
challenges. The platform is low-cost and small-scale, which makes
it a desirable tool for both research and learning. The main
technologies used on autonomous vehicles, such as computer
vision and object detection, are implemented on this platform. We
begin with a high-level overview of how self-driving cars work.
Then we present the platform we have created, followed by a
description of how it compares with industry level self-driving
cars. Finally, we discuss the future work of this research.
Index Terms—autonomous vehicle, self-driving car, computer
vision, learning platform
an open source software library used for computer vision
applications, is used to assist in lane detection and lane
following. YOLOv3, an object detection system, is used for
object detection and classification, while an Indoor Positioning
System (IPS) provides real-time coordinates to each car over
a wireless network.
There are three main aspects of the self-driving car platform:
The cars, the IPS system, and the track. We created five nearly
identical cars, which proved useful in testing the cars ability
to interact with other cars while driving around the track.

I. I NTRODUCTION
As self-driving car technology becomes available at lower
costs, it has become feasible to simulate self-driving cars on
a smaller, more feasible and affordable scale. This enables a
wider variety of people to work with, understand, and improve
the technologies being used in autonomous vehicles today.
Industry level self-driving cars consist of technological
components that set them apart from human operated cars.
These components must replace the awareness and control that
a human driver would normally give to keeping the car safely
on the road. For example, an autonomous vehicle must make
use of various sensors to model the surrounding environment
in real time. Cameras, ultrasonic sensors, GPS, LiDAR, and
radar are the main sensors used to take in information. The Fig. 1. The Car
combined data from these sensors must be synthesized into
information the car can use to safely navigate the environment. A. The Car
This includes detecting and tracking relevant objects, planning
The car (Fig. 1) consists of a Jetson TX1 module installed
the car’s movements, and maneuvering safely through the
on a Traxxas Slash 4x4 RC car with the Intel RealSense D435i
immediate surroundings. While human drivers have learned
camera mounted on top. The system is powered with a lithium
to do these tasks fairly easily, it is difficult to build a car that
polymer battery. An Arduino Nano is used to send commands
performs these actions with the same amount of precision.
from the Jetson to the car’s ESC (Electronic Speed Control),
There are many challenges that must be addressed in order to
giving the Jetson the ability to control the car’s movement
design and build a car that provides safe transportation
To assist in overcoming these challenges, we have created and speed. An encoder, which is attached to the car near the
a small scale, low-cost self-driving car platform. rotating motor shaft, measures the car’s speed. It is connected
to the Arduino, which provides the Jetson with the car’s speed.
II. S ELF -D RIVING C AR P LATFORM This data is then used in controlling the car’s movement. Each
The self-driving car platform we have created uses an Jetson is capable of connecting to Wi-Fi, allowing it to receive
onboard computer a RGBD camera and an RC car. OpenCV, its location over a local network (See Section II.C) (See Fig.

978-1-7281-4291-3/20/$31.00 ©2020
Authorized licensed use limited to: Robert IEEE
Gordon University. Downloaded on May 28,2021 at 10:29:58 UTC from IEEE Xplore. Restrictions apply.
 2020 Intermountain Engineering, Technology and Computing (IETC)
2 for a diagram of the car system). All of the hardware except
the camera is installed underneath the car cover, giving the car
a relatively normal look. The five different cars are painted five
different colors, which enables the IPS to properly track each
car (See Section II.C).
Fig. 2. Car Diagram
While the cars navigate the track, they do so while inter-
acting with the other cars. Using the Intel RealSense D435i
camera, each car uses the depth data provided to detect when
an object is in front of the car. This enables the car to better
interact with objects on the road. For example, when two cars
drive in a line and the car in the front drives slower, the
car in the back can respond by slowing down or stopping.
Additionally, when objects are found on the road, the car can
stop to avoid a collision.
B. The Track
The track consists of buildings, road lines, street signs, and
a traffic light at the intersection (Fig. 3). The average lane
width on a life-size road is between 10 and 15 feet, and the
average width of a car is about 6 feet. We scaled down the
size of our track to achieve a lane width of 12 inches. The RC
car used for our platform has a width of just over 7 inches.
This provides the small car with a lane size comparable to real
roads.
Fig. 3. Overhead view of track
The traffic light consists of four 3D printed traffic light
frames, each with a red, yellow, and green LED. The lights
are constructed to appear as real lights so it can be recognized
by the YOLOv3 system. An Arduino Nano controls the timing
of the LEDs, imitating the common element of a traffic light
intersection in real driving.
Authorized licensed use limited to: Robert Gordon University. Downloaded on May 28,2021 at 10:29:58 UTC from IEEE Xplore. Restrictions apply.
C. The Indoor Positioning System
Using OpenCV, the Indoor Positioning System (IPS) pro-
vides real-time tracking data for each of the five cars. To
distinguish the cars from one another, each car is painted one
of five distinct colors: red, green, blue, purple, and yellow.
This allows the IPS to track each car by its unique color.
With three web cameras mounted on the ceiling, the three
camera feeds get stitched into one large frame, providing a
full view of the track (Fig. 3). Using a mask, the unneeded
portions of the image are discarded (Fig. 4). The RGB frame
is converted into the HSV color space (Fig. 5), after which
the hue and the value channels are discarded. This leaves only
the saturation channel (Fig. 6), which is then converted into a
binary image (Fig. 7). To remove noise and yellow lines from
this binary image, a morphological technique called opening is
used, where an erosion of the pixels is followed by a dilation.
This binary image is then used as a mask to locate the positions
of each car. The position is determined by the current (x, y)
pixel value of the car. Each car’s position is published to a
local web server, where the cars can access it in real time
over the local network.
To avoid the IPS from detecting yellow road lines as the
yellow car, the mask partially covers the yellow lines. Initially,
the mask completely covered the yellow lines, but doing this
made it difficult for the IPS to detect cars when they drove over
the yellow lines. Partially masking the yellow lines enables
the IPS to detect cars driving over the yellow lines while not
identifying the road lines as the yellow car.
To see a video of the cars driving on the track, visit the
Brigham Young University YouTube channel.
III. C OMPARISON TO I NDUSTRY L EVEL S ELF -D RIVING
C ARS
Though a small-scale self-driving car platform is not able
to recreate all of the challenges an industry level autonomous
vehicle may face, it does provide students and researchers with
a platform capable of recreating a number of the challenges
associated with self-driving cars.
A self-driving car can be divided into four subsystems: Sen-
sors, Perception, Planning, and Control [1]. These subsystems
are shown in Fig. 8. We compare our platform to industry level
Fig. 4. Masked overhead view of track
 2020 Intermountain Engineering, Technology and Computing (IETC)
Fig. 5. Overhead view of track in HSV color space
Fig. 6. Saturation channel of overhead view of track
self-driving cars to determine the advantages and limitations
that exist with a small-scale self-driving car platform, using
the four subsystems as a guide.
A. Sensors
The industry leaders in self-driving vehicles use a variety
of sensors, including cameras, radar, LiDAR, and ultrasonics.
Additionally, GPS is a common tool used in autonomous
driving.
As some of these sensors are currently too expensive for
a small-scale platform, our platform uses low-cost sensors. A
forward-facing camera provides RGB and depth information,
while an IPS, as described in section II.C., provides real-
time positioning of the car. These sensors provide a fair
representation of the real-world environment in which actual
self-driving cars drive.
While industry standard self-driving cars use multiple cam-
eras to gather information from all directions, the camera on
our platform provides only one direction, forward, in which
to gather information. While this does limit the car’s ability to
detect and react to events happening behind and to the car’s
sides, it does allow the car to make driving decisions based
on what is in front of the car. The depth capabilities of the
camera assist in ways a LiDAR system might assist an industry
standard autonomous vehicle. For example, it can detect how
close an object is to the front of the car and react accordingly.
Additionally, the IPS used on the platform provides real-time
Authorized licensed use limited to: Robert Gordon University. Downloaded on May 28,2021 at 10:29:58 UTC from IEEE Xplore. Restrictions apply.
Fig. 7. Binary overhead view of track
Fig. 8. Autonomous vehicle subsystem diagram
location information about the cars, just as GPS does to real
vehicles.
B. Perception
Perception is how the car interprets its surrounding environ-
ment. The perception subsystem can be further divided into
two main areas: detection and localization.
1) Detection: Using data from various sensors, self-driving
cars make sense of the environment immediately surrounding
them. This includes the ability to detect road lanes, to classify
traffic signs and traffic lights, and to track various objects,
such as other vehicles and pedestrians. They must also be able
to detect free space, or space that is clear of obstacles. Our
platform uses OpenCV, YOLOv3, and the depth information
from the camera to assist in these activities. Using OpenCV
and the RealSense camera, the car is able to track the lines on
the road, which assists in the car’s ability to stay in the lanes
when driving. YOLOv3 is used to detect traffic lights, after
which OpenCV is used to determine the current color of the
traffic light. The depth information assists in determining the
distance of objects from the front of the car.
2) Localization: Localization is used to determine the po-
sition of the autonomous car. There are various methods used
to assist in localizing autonomous vehicles, including SLAM
(Simultaneous Localization and Mapping) [2], Kalman filters,
[3] and visual odometry [4]. These methods, combined with
GPS, can be used to create a map of a given area, giving useful
information about specific places.
 2020 Intermountain Engineering, Technology and Computing (IETC)

Portions of this same idea were used in our platform. For  0  

example, the car started processing incoming images through xi xi
the YOLOv3 network only when the IPS system confirmed ti  yi0  = M ·  yi  (1)
it was driving near a traffic light. This saved computational 1 1
resources so the car could better perform other activities while
it was not near the traffic light. Equation (2) shows how to apply M on the whole frame.
The resulting image is shown in Figure 10.
C. Planning
After gathering data from the surrounding environment,  !
1 0 0 v0 0 1 0 v0
  
autonomous vehicles plan a safe route to their destination. Idst (x, y) = Isrc   ,  (2)
This involves setting waypoints at a number of intermediate 0 0 1 v0 0 0 1 v0
locations, with the last waypoint set to the destination. This
is one strategy that was used on our platform. We used the where  
IPS to identify desired waypoints on the track to assist in path x
planning. Once the desired waypoints were set, the car began v0 = M  y 
the task of controlling the car. 1

D. Control Idst is source image array

Using the combined data from the previous three sub- Isrc is destination image array
systems, autonomous vehicles are expected to control the
car in a safe, smooth fashion. The angle of steering that is
needed to reach a desired waypoint must be determined. One
common method of control is Proportional-Integral-Derivative
(PID) control. This is a feedback control loop mechanism that
continuously calculates an error value. This error value is the
difference between a desired setpoint (SP) and a measured
process variable (PV).
We use PID control in conjunction with an image-based
visual servo method on the car platform to enable smooth car
control and to keep the car within the lanes of the track. The
car will not always have clear road lines to drive over however.
When the car drives on a section of the road without lane
lines (for example, the traffic light intersection), it still needs
a sense of where to drive. To keep the car driving correctly
under these conditions, we defined the needed lanes as the
desired SP. If the real-time position of a car is available, the Fig. 9. Car front view
measured PV can be found as well. We use the IPS to provide
this measurement. Notably, the lanes defined in our map are
all arcs. We can use only one Euclidean distance to arc center
and one subtraction to arc radius to compute the error.
As the camera detects the lane markings on the road, the
PV can be calculated. The camera input is processed using
edge detection to find the desired SP. On our track we make
the assumption that all of the roads are flat, which saves
computational resources.
We then compute a perspective transform matrix. As Fig. 9
shows, the incoming frame is the front view (original plane), in
which the lines are curved. Before the Hough Line Transform
is applied, the frame must be converted to a top view (target
plane). We chose to use a 4-Point Perspective Transform to do
this. First, we set up a rectangular pattern on the ground, then
placed the car directly in front of this pattern. The vertices of
the rectangle in the original plane are the source points, and
the homologous vertices in the target plane are the destination
points. Using these points and Equation (1), we can calculate Fig. 10. Car top view
the transformation matrix M.

Authorized licensed use limited to: Robert Gordon University. Downloaded on May 28,2021 at 10:29:58 UTC from IEEE Xplore. Restrictions apply.
 2020 Intermountain Engineering, Technology and Computing (IETC)

IV. F UTURE W ORK [2] G. Bresson, Z. Alsayed, L. Yu, and S. Glaser, “Simultaneous localization
and mapping: A survey of current trends in autonomous driving,” IEEE
While this platform provides a comparable model of how Transactions on Intelligent Vehicles, vol. 2, no. 3, pp. 194–220, Sep. 2017.
industry level autonomous vehicles act on the road, there are [3] V. Subramanian, T. F. Burks, and W. E. Dixon, “Sensor
improvements that can be made to make future platforms more fusion using fuzzy logic enhanced kalman filter for autonomous
vehicle guidance in citrus groves,” Transactions of the
robust and realistic. ASABE, vol. 52, no. 5, pp. 1411–1422, 2009. [Online].
One area of improvement is in visual servo robustness. Available: https://app.dimensions.ai/details/publication/pub.1064896074
Two solutions are presented here that could improve the and http://ncr.mae.ufl.edu/papers/asabe09.pdf
[4] I. Parra, M. A. Sotelo, D. F. Llorca, and M. Ocaña, “Robust visual
performance of the image-based visual servo method. One is to odometry for vehicle localization in urban environments,” Robotica,
use stereo images, while the other is to use deep learning. Both vol. 28, no. 3, p. 441–452, 2010.
of these are computationally intensive and would therefore [5] S. Shalev-Shwartz, S. Shammah, and A. Shashua, “Safe, multi-agent, rein-
forcement learning for autonomous driving,” CoRR, vol. abs/1610.03295,
require more powerful computational resources. 2016. [Online]. Available: http://arxiv.org/abs/1610.03295
Reinforcement learning (RL) is a technique that could [6] G. Máttyus, S. Wang, S. Fidler, and R. Urtasun, “Hd maps: Fine-grained
prove beneficial on industry level self-driving cars [5]. Imple- road segmentation by parsing ground and aerial images,” in 2016 IEEE
Conference on Computer Vision and Pattern Recognition (CVPR). IEEE,
menting this technique on small-scale self-driving car models 2016, pp. 3611–3619.
would therefore be beneficial in more accurately modeling [7] “Intel® realsense™ lidar camera l515.” 2020 (Accessed 26 February
autonomous vehicles. RL involves much trial and error to 2020). [Online]. Available: https://www.intelrealsense.com/lidar-camera-
l515
determine the best course of action for the car. Because of
this, RL is computationally expensive and would also need
powerful resources.
Autonomous vehicles use High Definition (HD) maps [6]
as a reliable way of locating the car and traversing known
areas. This type of map provides high precision information of
the environment surrounding the car. We can use the existing
sensors on our car platform, particularly the IPS and the
camera, to detect various landmarks and build our own HD
map. This stored, update-able map could help with high quality
vehicle localization and route planning.
LiDAR could potentially prove to be a useful tool in a
small-scale self-driving car platform. The technology is getting
cheaper and more accessible [7]. LiDAR could assist in
surveying the environment around the car, providing additional
data the car could use to drive safely.
The next generation of wireless communication (5G) tech-
nology may also prove to be useful in modeling self-driving
cars. 5G has a data rate of up to 1 Gigabit per second,
with a latency as low as 1 millisecond. Using this type of
wireless communication, many more options become available
to a self-driving car platform. The on-board computations can
be migrated to a powerful off-road machine. Also, car-to-car
communication would be faster, enabling more robust and
safer interaction between cars.
V. C ONCLUSION
The platform that has been presented provides a suitable
means of modeling autonomous vehicles. It gives students the
opportunity to learn about the basics of self-driving cars, and
it allows researchers to test and expand on current self-driving
car technologies. When compared to industry level self-driving
cars, this platform provides access to the core technologies
in use on autonomous vehicles. Further improvements can
be made to increase the similarities between this model and
industry level autonomous vehicles.
R EFERENCES
[1] D. Silver, “How the udacity self-driving car works,” 2017 (Accessed 26
February 2020). [Online]. Available: https://medium.com/udacity/how-
the-udacity-self-driving-car-works-575365270a40

Authorized licensed use limited to: Robert Gordon University. Downloaded on May 28,2021 at 10:29:58 UTC from IEEE Xplore. Restrictions apply.