DARPA 2005 Stanley PDF

Stanley: The Robot that Won
the DARPA Grand Challenge

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •
Sebastian Thrun, Mike Montemerlo,

Hendrik Dahlkamp, David Stavens,
Andrei Aron, James Diebel, Philip Fong,
John Gale, Morgan Halpenny,
Gabriel Hoffmann, Kenny Lau, Celia Oakley,
Mark Palatucci, Vaughan Pratt,
and Pascal Stang
Stanford Artificial Intelligence Laboratory
Stanford University
Stanford, California 94305
Sven Strohband, Cedric Dupont,

Lars-Erik Jendrossek, Christian Koelen,
Charles Markey, Carlo Rummel,
Joe van Niekerk, Eric Jensen,
and Philippe Alessandrini
Volkswagen of America, Inc.
Electronics Research Laboratory
4009 Miranda Avenue, Suite 100
Palo Alto, California 94304
Gary Bradski, Bob Davies, Scott Ettinger,

Adrian Kaehler, and Ara Nefian
Intel Research
2200 Mission College Boulevard
Santa Clara, California 95052
Pamela Mahoney
Mohr Davidow Ventures
3000 Sand Hill Road, Bldg. 3, Suite 290
Menlo Park, California 94025
Received 13 April 2006; accepted 27 June 2006
Journal of Field Robotics 23(9), 661–692 (2006) © 2006 Wiley Periodicals, Inc.
Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.1002/rob.20147
662 • Journal of Field Robotics—2006
This article describes the robot Stanley, which won the 2005 DARPA Grand Challenge.
Stanley was developed for high-speed desert driving without manual intervention. The
robot’s software system relied predominately on state-of-the-art artificial intelligence
technologies, such as machine learning and probabilistic reasoning. This paper describes
the major components of this architecture, and discusses the results of the Grand Chal-
lenge race. © 2006 Wiley Periodicals, Inc.
1. INTRODUCTION sult of an intense development effort led by Stanford

University, and involving experts from Volkswagen
The Grand Challenge was launched by the Defense of America, Mohr Davidow Ventures, Intel Research,
Advanced Research Projects Agency 共DARPA兲 in and a number of other entities. Stanley is based on a
2003 to spur innovation in unmanned ground vehicle 2004 Volkswagen Touareg R5 TDI, outfitted with a six
navigation. The goal of the Challenge was to develop
processor computing platform provided by Intel, and
an autonomous robot capable of traversing unre-
a suite of sensors and actuators for autonomous driv-
hearsed off-road terrain. The first competition, which
ing. Figure 2 shows images of Stanley during the race.
carried a prize of $1M, took place on March 13, 2004.
The main technological challenge in the develop-
It required robots to navigate a 142-mile long course
ment of Stanley was to build a highly reliable system,
through the Mojave desert in no more than 10 h. 107
teams registered and 15 raced, yet none of the par- capable of driving at relatively high speeds through
ticipating robots navigated more than 5% of the entire diverse and unstructured off-road environments, and
course. The challenge was repeated on October 8, to do all this with high precision. These requirements
2005, with an increased prize of $2M. This time, 195 led to a number of advances in the field of autono-
teams registered and 23 raced. Of those, five teams mous navigation, as surveyed in this paper. Methods
finished. Stanford’s robot “Stanley” finished the were developed, and existing methods extended, in
course ahead of all other vehicles in 6 h, 53 min, and the areas of long-range terrain perception, real-time
58 s, and was declared the winner of the DARPA collision avoidance, and stable vehicle control on slip-
Grand Challenge; see Figure 1. pery and rugged terrain. Many of these develop-
This paper describes the robot Stanley, and its ments were driven by the speed requirement, which
software system in particular. Stanley was developed rendered many classical techniques in the off-road
by a team of researchers to advance the state-of-the- driving field unsuitable. In pursuing these develop-
art in autonomous driving. Stanley’s success is the re- ments, the research team brought to bear algorithms
Figure 1. 共a兲 At approximately 1:40 pm on Oct 8, 2005, Stanley was the first robot to complete the DARPA Grand
Challenge. 共b兲 The robot is being honored by DARPA Director Dr. Tony Tether.
Journal of Field Robotics DOI 10.1002/rob

Thrun et al.: Stanley: The Robot that Won • 663
Figure 3. A section of the RDDF file from the 2005

DARPA Grand Challenge. The corridor varies in width
and maximum speed. Waypoints are more frequent in
turns.
Figure 2. Images from the race.
CA, approximately 100 miles northeast of Los Ange-

from diverse areas including distributed systems, les, and finished in Primm, NV, approximately
machine learning, and probabilistic robotics. 30 miles southwest of Las Vegas. The 2005 course
both started and finished in Primm, NV.
The specific race course was kept secret from all
1.1. Race Rules teams until 2 h before the race. At this time, each
The rules 共DARPA, 2004兲 of the DARPA Grand Chal- team was given a description of the course on CD-
lenge were simple. Contestants were required to ROM in a DARPA-defined route definition data for-
build autonomous ground vehicles capable of tra- mat 共RDDF兲. The RDDF is a list of longitudes, lati-
versing a desert course up to 175-miles long in less tudes, and corridor widths that define the course
than 10 h. The first robot to complete the course in boundary, and a list of associated speed limits; an
under 10 h would win the challenge and the $2M example segment is shown in Figure 3. Robots that
prize. Absolutely no manual intervention was al- travel substantially beyond the course boundary risk
lowed. The robots were started by DARPA personnel disqualification. In the 2005 race, the RDDF con-
and from that point on had to drive themselves. tained 2,935 waypoints.
Teams only saw their robots at the starting line and, The width of the race corridor generally tracked
with luck, at the finish line. the width of the road, varying between 3 and 30 m
Both the 2004 and 2005 races were held in the in the 2005 race. Speed limits were used to protect
Mojave desert in the southwest United States. The important infrastructure and ecology along the
course terrain varied from high-quality graded dirt course, and to maintain the safety of DARPA chase
roads to winding rocky mountain passes; see Figure drivers who followed behind each robot. The speed
2. A small fraction of each course traveled along limits varied between 5 and 50 mph. The RDDF de-
paved roads. The 2004 course started in Barstow, fined the approximate route that robots would take,

Figure 4. 共a兲 View of the vehicle’s roof rack with sensors. 共b兲 The computing system in the trunk of the vehicle. 共c兲 The
gear shifter, control screen, and manual override buttons.
so no global path planning was required. As a result, 2. VEHICLE

the race was primarily a test of high-speed road
finding, obstacle detection, and avoidance in desert Stanley is based on a diesel-powered Volkswagen
terrain. Touareg R5. The Touareg has four-wheel drive
The robots all competed on the same course; 共4WD兲, variable-height air suspension, and automatic
starting one after another at 5 min intervals. When a electronic locking differentials. To protect the vehicle
faster robot overtook a slower one, the slower robot from environmental impact, Stanley has been outfit-
was paused by DARPA officials, allowing the second ted with skid plates and a reinforced front bumper. A
robot to pass the first as if it were a static obstacle. custom interface enables direct electronic actuation of
This eliminated the need for robots to handle the both the throttle and brakes. A DC motor attached to
case of dynamic passing. the steering column provides electronic steering con-
trol. A linear actuator attached to the gear shifter
shifts the vehicle between drive, reverse, and parking
1.2. Team Composition gears 关Figure 4共c兲兴. Vehicle data, such as individual
The Stanford Racing Team team was organized into wheel speeds and steering angle, are sensed auto-
four major groups. The Vehicle Group oversaw all matically and communicated to the computer system
modifications and component developments related through a CAN bus interface.
to the core vehicle. This included the drive-by-wire The vehicle’s custom-made roof rack is shown in
systems, the sensor and computer mounts, and the Figure 4共a兲. It holds nearly all of Stanley’s sensors.
computer systems. The group was led by researchers The roof provides the highest vantage point of the ve-
from Volkswagen of America’s Electronics Research hicle; from this point, the visibility of the terrain is
Lab. The Software Group developed all software, in- best, and the access to global positioning system
cluding the navigation software and the various 共GPS兲 signals is least obstructed. For environment
health monitor and safety systems. The software perception, the roof rack houses five SICK laser range
group was led by researchers affiliated with Stan- finders. The lasers are pointed forward along the
ford University. The Testing Group was responsible driving direction of the vehicle, but with slightly dif-
for testing all system components and the system as ferent tilt angles. The lasers measure cross sections of
a whole, according to a specified testing schedule. the approaching terrain at different ranges out to
The members of this group were separate from any 25 m in front of the vehicle. The roof rack also holds
of the other groups. The testing group was led by a color camera for long-range road perception, which
researchers affiliated with Stanford University. The is pointed forward and angled slightly downward.
Communications Group managed all media relations For long-range detection of large obstacles, Stanley’s
and fund raising activities of the Stanford Racing roof rack also holds two 24 GHz RADAR sensors,
Team. The communications group was led by em- supplied by Smart Microwave Sensors. Both RADAR
ployees of Mohr Davidow Ventures, with participa- sensors cover the frontal area up to 200 m, with a cov-
tion from all other sponsors. The operations over- erage angle in azimuth of about 20°. Two antennae of
sight was provided by a steering board that included this system are mounted on both sides of the laser
all major supporters. sensor array. The lasers, camera, and radar system

comprise the environment sensor group of the system. ing, whereas the other two executed all other soft-
That is, they inform Stanley of the terrain ahead, so ware. The computers were able to poll the sensors at
that Stanley can decide where to drive, and at what up to 100 Hz, and to control the steering, throttle and
speed. brake at frequencies up to 20 Hz.
Further back, the roof rack holds a number of ad- An important aspect in Stanley’s design was to
ditional antennae: One for Stanley’s GPS positioning retain street legality, so that a human driver could
system and two for the GPS compass. The GPS po- safely operate the robot as a conventional passenger
sitioning unit is a L1/L2/Omnistar HP receiver. To- car. Stanley’s custom user interface enables a driver to
gether with a trunk-mounted inertial measurement engage and disengage the computer system at will,
unit 共IMU兲, the GPS systems are the positioning sensor even while the vehicle is in motion. As a result, the
group, whose primary function is to estimate the lo- driver can disable computer control at any time of the
cation and velocity of the vehicle relative to an exter- development, and regain manual control of the ve-
nal coordinate system. hicle. To this end, Stanley is equipped with several
Finally, a radio antenna and three additional GPS manual override buttons located near the driver seat.
antennae from the DARPA E-Stop system are also lo- Each of these switches controls one of the three major
cated on the roof. The E-Stop system is a wireless link actuators 共brakes, throttle, and steering兲. An addi-
that allows a chase vehicle following Stanley to safely tional central emergency switch disengages all com-
stop the vehicle in case of emergency. The roof rack puter control and transforms the robot into a conven-
also holds a signaling horn, a warning light, and two tional vehicle. While this feature was of no relevance
manual E-stop buttons. to the actual race 共in which no person sat in the car兲,
Stanley’s computing system is located in the ve- it proved greatly beneficial during software develop-
hicle’s trunk, as shown in Figure 4共b兲. Special air ment. The interface made it possible to operate Stan-
ducts direct air flow from the vehicle’s air condition- ley autonomously with people inside, as a dedicated
ing system into the trunk for cooling. The trunk fea- safety driver could always catch computer glitches
tures a shock-mounted rack that carries an array of and assume full manual control at any time.
six Pentium M computers, a Gigabit Ethernet switch, During the actual race, there was of course no
and various devices that interface to the physical sen- driver in the vehicle, and all driving decisions were
sors and the Touareg’s actuators. It also features a made by Stanley’s computers. Stanley possessed an
custom-made power system with backup batteries, operational control interface realized through a
and a switch box that enables Stanley to power-cycle touch-sensitive screen on the driver’s console. This
individual system components through software. interface allowed Government personnel to shut
The DARPA-provided E-Stop is located on this rack down and restart the vehicle, if it became necessary.
on additional shock compensation. The trunk assem-
bly also holds the custom interface to the Volkswagen
Touareg’s actuators: The brake, throttle, gear shifter,
and steering controller. A six degree-of-freedom IMU
3. SOFTWARE ARCHITECTURE
is rigidly attached to the vehicle frame underneath
the computing rack in the trunk.
The total power requirement of the added instru-
3.1. Design Principles
mentation is approximately 500 W, which is pro-
vided through the Touareg’s stock alternator. Stan- Before both the 2004 and 2005 Grand Challenges,
ley’s backup battery system supplies an additional DARPA revealed to the competitors that a stock
buffer to accommodate long idling periods in desert 4WD pickup truck would be physically capable of
heat. traversing the entire course. These announcements
The operating system run on all computers is suggested that the innovations necessary to success-
Linux. Linux was chosen due to its excellent network- fully complete the challenge would be in designing
ing and time sharing capabilities. During the race, intelligent driving software, not in designing exotic
Stanley executed the race software on three of the six vehicles. This announcement and the performance of
computers; a fourth was used to log the race data the top finishers in the 2004 race guided the design
共and two computers were idle兲. One of the three race philosophy of the Stanford Racing Team: Treat au-
computers was entirely dedicated to video process- tonomous navigation as a software problem.

In relation to previous work on robotics architec- ware components, and automatically restart or
tures, Stanley’s software architecture is related to the power-cycle such components when a failure is ob-
well-known three-layer architecture 共Gat, 1998兲, albeit served. In this way, the software is robust to certain
without a long-term symbolic planning method. A occurrences, such as crashing or hanging of a soft-
number of guiding principles proved essential in the ware modules or stalled sensors.
design of the software architecture:
3.1.4. Development Support

3.1.1. Control and Data Pipeline Finally, the software is structured so as to aid devel-
There is no centralized master process in Stanley’s opment and debugging of the system. The developer
software system. All modules are executed at their can easily run just a subsystem of the software, and
own pace, without interprocess synchronization effortlessly migrate modules across different proces-
mechanisms. Instead, all data are globally time sors. To facilitate debugging during the develop-
stamped, and time stamps are used when integrat- ment process, all data are logged. By using a special
ing multiple data sources. The approach reduces the replay module, the software can be run on recorded
risk of deadlocks and undesired processing delays. data. A number of visualization tools were devel-
To maximize the configurability of the system, oped that make it possible to inspect data and inter-
nearly all interprocess communication is imple- nal variables while the vehicle is in motion, or while
mented through publish-subscribe mechanisms. The replaying previously logged data. The development
information from sensors to actuators flows in a process used a version control process with a strict
single direction; no information is received more set of rules for the release of race-quality software.
than once by the same module. At any point in time, Overall, we found that the flexibility of the software
all modules in the pipeline are working simulta- during development was essential in achieving the
neously, thereby maximizing the information high level of reliability necessary for long-term au-
throughput and minimizing the latency of the soft- tonomous operation.
ware system.
3.2. Processing Pipeline

3.1.2. State Management
The race software consisted of approximately 30
Even though the software is distributed, the state of modules executed in parallel 共Figure 5兲. The system
the system is maintained by local authorities. There is broken down into six layers which correspond to
are a number of state variables in the system. The the following functions: Sensor interface, perception,
health state is locally managed in the health monitor; control, vehicle interface, user interface, and global
the parameter state in the parameter server; the glo- services.
bal driving mode is maintained in a finite state au-
tomaton; and the vehicle state is estimated in the 1. The sensor interface layer comprises a num-
state estimator module. The environment state is ber of software modules concerned with re-
broken down into multiple maps 共laser, vision, and ceiving and time stamping all sensor data.
radar兲. Each of these maps are maintained in dedi- The layer receives data from each laser sensor
cated modules. As a result, all other modules will at 75 Hz, from the camera at approximately
receive values that are mutually consistent. The ex- 12 Hz, the GPS and GPS compass at 10 Hz,
act state variables are discussed in later sections of and the IMU and the Touareg CAN bus at
this paper. All state variables are broadcast to rel- 100 Hz. This layer also contains a database
evant modules of the software system through a server with the course coordinates 共RDDF
publish-subscribe mechanism. file兲.
2. The perception layer maps sensor data into
internal models. The primary module in this
3.1.3. Reliability layer is the unscented Kalman filter 共UKF兲
The software places strong emphasis on the overall vehicle state estimator, which determines the
reliability of the robotic system. Special modules vehicle’s coordinates, orientation, and veloci-
monitor the health of individual software and hard- ties. Three different mapping modules build

Figure 5. Flowchart of Stanley software system. The software is roughly divided into six main functional groups: Sensor
interface, perception, control, vehicle interface, and user interface. There are a number of cross-cutting services, such as
the process controller and the logging modules.
two-dimensional 共2D兲 environment maps controllers, one for the steering control and
based on lasers, the camera, and the radar one for brake and throttle control. Both con-
system. A road finding module uses the laser- trollers send low-level commands to the ac-
derived maps to find the boundary of a road, tuators that faithfully execute the trajectory
so that the vehicle can center itself laterally. emitted by the planner. The control layer also
Finally, a surface assessment module extracts features a top level control module, imple-
parameters of the current road for the pur- mented as a simple finite state automaton.
pose of determining safe vehicle speeds. This level determines the general vehicle
3. The control layer is responsible for regulat- mode in response to user commands received
ing the steering, throttle, and brake response through the in-vehicle touch screen or the
of the vehicle. A key module is the path plan- wireless E-stop, and maintains gear state in
ner, which sets the trajectory of the vehicle in case backward motion is required.
steering and velocity space. This trajectory is 4. The vehicle interface layer serves as the in-
passed to two closed-loop trajectory tracking terface to the robot’s drive-by-wire system. It

contains all interfaces to the vehicle’s brakes, Table I. Standard methodology of Stanley’s 15 variables.
throttle, and steering wheel. It also features
the interface to the vehicle’s server, a circuit No. of values State variable
that regulates the physical power to many of 3 Position 共longitude, latitude, and altitude兲
the system components.
3 Velocity
5. The user interface layer comprises the re-
mote E-stop and a touch-screen module for 3 Orientation 共Euler angles: roll, pitch,
and yaw兲
starting up the software.
6. The global services layer provides a number 3 Accelerometer biases
of basic services for all software modules. 3 Gyro biases
Naming and communication services are
provides through Carnegie Mellon Universi-
ty’s 共CMU’s兲 interprocess communication
toolkit 共Simmons & Apfelbaum, 1998兲. A cen- point of view, the sigma point linearization in the
tralized parameter server maintains a data- UKF often yields a lower estimation error than the
base of all vehicle parameters and updates linearization based on Taylor expansion in the ex-
them in a consistent manner. The physical tended Kalman filter 共EKF兲共van der Merwe, 2004兲. To
power of individual system components is many, the UKF is also preferable from an implemen-
regulated by the power server. Another mod- tation standpoint because it does not require the ex-
ule monitors the health of all systems com- plicit calculation of any Jacobians; although those can
ponents and restarts individual system com- be useful for further analysis.
ponents when necessary. Clock synchron- While GPS is available, the UKF uses only a
ization is achieved through a time server. Fi- “weak” model. This model corresponds to a moving
nally, a data logging server dumps sensor, mass that can move in any direction. Hence, in nor-
control, and diagnostic data to disk for replay mal operating mode, the UKF places no constraint on
and analysis. the direction of the velocity vector relative to the ve-
hicle’s orientation. Such a model is clearly inaccurate,
The following sections will describe Stanley’s core but the vehicle-ground interactions in slippery desert
software processes in greater detail. The paper will terrain are generally difficult to model. The moving
then conclude with a description of Stanley’s perfor- mass model allows for any slipping or skidding that
mance in the Grand Challenge. may occur during off-road driving.
However, this model performs poorly during
GPS outages, as the position of the vehicle relies
strongly on the accuracy of the IMU’s accelerometers.
4. VEHICLE STATE ESTIMATION
As a consequence, a more restrictive UKF motion
Estimating vehicle state is a key prerequisite for pre- model is used during GPS outages. This model con-
cision driving. Inaccurate pose estimation can cause strains the vehicle to only move in the direction it is
the vehicle to drive outside the corridor, or build ter- pointed. The integration of the IMU’s gyroscopes for
rain maps that do not reflect the state of the robot’s orientation, coupled with wheel velocities for com-
environment, leading to poor driving decisions. In puting the position, is able to maintain accurate pose
Stanley, the vehicle state comprises a total of 15 vari- of the vehicle during GPS outages of up to 2 min
ables. The design of this parameter space follows long; the accrued error is usually in the order of cen-
standard methodology 共Farrell & Barth, 1999; van der timeters. Stanley’s health monitor will decrease the
Merwe & Wan, 2004兲, as indicated in Table I. maximum vehicle velocity during GPS outages to
An unscented Kalman filter 共UKF兲共Julier & Uhl- 10 mph in order to maximize the accuracy of the re-
mann, 1997兲 estimates these quantities at an update stricted vehicle model. Figure 6共a兲 shows the result of
rate of 100 Hz. The UKF incorporates observations position estimation during a GPS outage with the
from the GPS, the GPS compass, the IMU, and the weak vehicle model; Figure 6共b兲, the result with the
wheel encoders. The GPS system provides both ab- strong vehicle model. This experiment illustrates the
solute position and velocity measurements, which are performance of this filter during a GPS outage.
both incorporated into the UKF. From a mathematical Clearly, accurate vehicle modeling during GPS out-

stacle avoidance. Stanley is equipped with five

single-scan laser range finders mounted on the roof,
tilted downward to scan the road ahead. Figure 7共a兲
illustrates the scanning process. Each laser scan gen-
erates a vector of 181 range measurements spaced
0.5° apart. Projecting these scans into the global co-
ordinate frame according, to the estimated pose of
the vehicle, results in a 3D point cloud for each laser.
Figure 7共b兲 shows an example of the point clouds
Figure 6. UKF state estimation when GPS becomes un-
available. The area covered by the robot is approximately acquired by the different sensors. The coordinates of
100⫻ 100 m. The large ellipses illlustrate the position un- such 3D points are denoted 共Xik Yik Zik兲; where k is the
certainty after losing GPS. 共a兲 Without integrating the time index at which the point was acquired, and i is
wheel motion the result is highly erroneous. 共b兲 The wheel the index of the laser beam.
motion clearly improves the result. Obstacle detection on laser point clouds can be
formulated as a classification problem, assigning to
each 2D location in a surface grid one of three pos-
ages is essential. In an experiment on a paved road, sible values: Occupied, free, and unknown. A loca-
we found that even after 1.3 km of travel without tion is occupied by an obstacle if we can find two
GPS on a cyclic course, the accumulated vehicle error nearby points whose vertical distance 兩Zik − Zjm兩 ex-
was only 1.7 m. ceeds a critical vertical distance ␦. It is considered
drivable 共free of obstacles兲 if no such points can be
found, but at least one of the readings falls into the
5. LASER TERRAIN MAPPING corresponding grid cell. If no reading falls into the
cell, the drivability of this cell is considered un-
known. The search for nearby points is conveniently
5.1. Terrain Labeling organized in a 2D grid, the same grid used as the
To safely avoid obstacles, Stanley must be capable of final drivability map that is provided to the vehicle’s
accurately detecting nondrivable terrain at a suffi- navigation engine. Figure 8 shows the example grid
cient range to stop or take the appropriate evasive map. As indicated in this figure, the map assigns
action. The faster the vehicle is moving, the farther terrain to one of three classes: Drivable, occupied, or
away obstacles must be detected. Lasers are used as unknown.
the basis for Stanley’s short and medium range ob- Unfortunately, applying this classification
Figure 7. 共a兲 Illustration of a laser sensor: The sensor is angled downward to scan the terrain in front of the vehicle as it
moves. Stanley possesses five such sensors, mounted at five different angles. 共b兲 Each laser acquires a three-dimensional
共3D兲 point cloud over time. The point cloud is analyzed for drivable terrain and potential obstacles.

Figure 8. Examples of occupancy maps: 共a兲 An underpass and 共b兲 a road.
scheme directly to the laser data yields results inap- rain, we found that 12.6% of known drivable area is
propriate for reliable robot navigation. Figure 9 classified as obstacle, for a height threshold param-
shows such an instance, in which a small error in the eter ␦ = 15 cm. Such situations occur even for roll/
vehicle’s roll/pitch estimation leads to a massive ter- pitch errors smaller than 0.5°. Pose errors of this
rain classification error, forcing the vehicle off the magnitude can be avoided by pose estimation sys-
road. Small pose errors are magnified into large er- tems that cost hundreds of thousands of dollars, but
rors in the projected positions of laser points because such a choice was too costly for this project.
the lasers are aimed at the road up to 30 m in front The key insight to solving this problem is illus-
of the vehicle. In our reference dataset of labeled ter- trated in Figure 10. This graph plots the perceived
Figure 9. Small errors in pose estimation 共smaller than 0.5°兲 induce massive terrain classification errors, which if ignored
could force the robot off the road. These images show two consecutive snapshots of a map that forces Stanley off the road.
Here, obstacles are plotted in red, free space in white, and unknown territory in gray. The blue lines mark the corridor as
defined by the RDDF.

for a cell which was previously observed. Then, one

or more of three cases will be true:
1. The measurement might be a witness of an

obstacle, according to the probabilistic test. In
this case, Stanley simply marks the cell as an
obstacle and no further testing takes place.
2. The measurement does not trigger as a wit-
ness of an obstacle; but, in future tests, it es-
tablishes a tighter lower bound on the mini-
mum Z value than the previously stored
measurement. In this case, our algorithm
simply replaces the previous measurement
with this one. The rationale behind this is
simple: If the measurement is more restrictive
than the previous one, there will not be a situ-
ation where a test against this point would
Figure 10. Correlation of time and vertical measurement fail, while a test against the older one would
error in the laser data analysis. succeed. Hence, the old point can safely be
discarded.
3. The third case is equivalent to the second, but
with a refinement of the upper value. A mea-
obstacle height 兩Zik − Zjm兩along the vertical axis for a
surement may simultaneously refine the
collection of grid cells taken from flat terrain.
lower and the upper bounds.
Clearly, for some grid cells, the perceived height is
enormous—despite the fact that in reality, the sur- The fact that only two measurements per grid cell
face is flat. However, this function is not random. have to be stored renders this algorithm highly effi-
The horizontal axis depicts the time difference ⌬t兩k cient in space and time.
− m兩 between the acquisition of those scans. Obvi-
ously, the error is strongly correlated with the
elapsed time between the two scans. 5.2. Data-Driven Parameter Tuning
To model this error, Stanley uses a first-order
Markov model, which models the drift of the pose A final step in developing this mapping algorithm
estimation error over time. The test for the presence addresses parameter tuning. Our approach and the
of an obstacle is therefore a probabilistic test. Given underlying probabilistic Markov model possess a
two points 共Xik Yik Zik兲T and 共Xmj
Yjm Zjm兲T, the height number of unknown parameters. These parameters
difference is distributed according to a normal dis- include the height threshold ␦, the statistical accep-
tribution whose variance scales linearly with the tance probability threshold ␣, and various Markov
time difference 兩k − m兩. Thus, Stanley uses a probabi- chain error parameters 共the noise covariances of the
listic test for the presence of an obstacle, of the type process noise and the measurement noise兲.
Stanley uses a discriminative learning algorithm
for locally optimizing these parameters. This algo-
p共兩Zik − Zjm兩 ⬎ ␦兲 ⬎ ␣ , 共1兲 rithm tunes the parameters in a way that maximizes
the discriminative accuracy of the resulting terrain
analysis on labeled training data.
where ␣ is a confidence threshold, e.g., ␣ = 0.05. The data are labeled through human driving,
When applied over a 2D grid, the probabilistic similar in spirit to Pomerleau 共1993兲. Figure 11 illus-
method can be implemented efficiently so that only trates the idea: A human driver is instructed to only
two measurements have to be stored per grid cell. drive over obstacle-free terrain. Grid cells traversed
This is due to the fact that each measurement defines by the vehicle are then labeled as drivable: This area
a bound on future Z values for obstacle detection. corresponds to the blue stripe in Figure 11. A stripe
For example, suppose we observe a measurement to the left and right of this corridor is assumed to be

time, the rate at which the area off the road is labeled
as an obstacle remains approximately constant 共from
22.6% to 22.0%兲. This rate is not 100%, simply be-
cause most of the terrain there is still flat and driv-
able. Our approach for data acquisition mislabels the
flat terrain as nondrivable. Such mislabeling how-
ever, does not interfere with the parameter tuning
algorithm, and hence is preferable to the tedious
process of labeling pixels manually.
Figure 12 shows an example of the mapper in
action. A snapshot of the vehicle from the side illus-
trates that a part of the surface is scanned multiple
times due to a change of pitch. As a result, the non-
Figure 11. Terrain labeling for parameter tuning: The probabilistic method hallucinates a large occupied
area traversed by the vehicle is labeled as “drivable” area in the center of the road, shown in panel 共c兲 of
共blue兲 and two stripes at a fixed distance to the left and the Figure 12. Our probabilistic approach overcomes this
right are labeled as “obstacles” 共red兲. While these labels
error and generates a map that is good enough for
are only approximate, they are extremely easy to obtain
and significantly improve the accuracy of the resulting driving. A second example is shown in Figure 13.
map when used for parameter tuning.
6. COMPUTER VISION TERRAIN ANALYSIS

all obstacles, as indicated by the red stripes in Figure The effective maximum range at which obstacles can
11. The distance between the drivable and obstacle is be detected with the laser mapper is approximately
set by hand, based on the average road width for a 22 m. This range is sufficient for Stanley to reliably
segment of data. Clearly, not all of those cells labeled avoid obstacles at speeds up to 25 mph. Based on the
as obstacles are actually occupied by obstacles; how- 2004 Race Course, the development team estimated
ever, even training against an approximate labeling that Stanley would need to reach speeds of 35 mph in
is enough to improve the overall performance of the order to successfully complete the challenge. To ex-
mapper. tend the sensor range enough to allow safe driving at
The learning algorithm is now implemented 35 mph, Stanley uses a color camera to find drivable
through coordinate ascent. In the outer loop, the al- surfaces at ranges exceeding that of the laser analysis.
gorithm performs coordinate ascent relative to a Figure 14 compares laser and vision mapping side by
data-driven scoring function. Given an initial guess, side. The left diagram shows a laser map acquired
the coordinate ascent algorithm modifies each pa- during the race; here, obstacles are detected at an ap-
rameter one after another by a fixed amount. It then proximate 22 m range. The vision map for the same
determines if the new value constitutes an improve- situation is shown on the right side. This map extends
beyond 70 m 共each yellow circle corresponds to 10 m
ment over the previous value when evaluated over a
range兲.
logged data set, and retains it accordingly. If for a
Our work builds on a long history of research on
given interval size no improvement can be found,
road finding 共Pomerleau, 1991; Crisman & Thorpe,
the search interval is cut in half and the search is
1993兲; see also Dickmanns 共2002兲. To find the road, the
continued, until the search interval becomes smaller vision module classifies images into drivable and
than a preset minimum search interval 共at which nondrivable regions. This classification task is gener-
point the tuning is terminated兲. ally difficult, as the road appearance is affected by a
The probabilistic analysis, paired with the dis- number of factors that are not easily measured and
criminative algorithm for parameter tuning, has a change over time, such as the surface material of the
significant effect on the accuracy of the terrain labels. road, lighting conditions, dust on the lens of the cam-
Using an independent testing data set, we find that era, and so on. This suggests that an adaptive ap-
the false positive rate 共the area labeled as drivable in proach is necessary, in which the image interpretation
Figure 11兲 drops from 12.6% to 0.002%. At the same changes as the vehicle moves and conditions change.

Figure 12. Example of pitching combined with small pose estimation errors: 共a兲 The reading of the center beam of one of
the lasers, integrated over time 共some of the terrain is scanned twice.兲; 共b兲 shows 3D point cloud; 共c兲 resulting map without
probabilistic analysis, and 共d兲 map with probabilistic analysis. The map shown in 共c兲 possesses a phantom obstacle, large
enough to force the vehicle off the road.
The camera images are not the only source of in- quired from the vision routine is to extend the reach
formation about upcoming terrain available to the vi- of the laser analysis. This is different from the
sion mapper. Although we are interested in using vi- general-purpose image interpretation problem, in
sion to classify the drivability of terrain beyond the which no such data would be available.
laser range, we already have such drivability infor- Stanley finds drivable surfaces by projecting
mation from the laser in the near range. All that is re- drivable area from the laser analysis into the camera
Figure 13. A second example of pitching combined with small page estimation errors.

Figure 14. Comparison of the laser-based 共left兲 and the image-based 共right兲 mapper. For scale, circles are spaced around
the vehicle at a 10 m distance. This diagram illustrates that the reach of lasers is approximately 22 m, whereas the vision
module often looks 70 m ahead.
image. More specifically, Stanley extracts a quadrilat- mean RGB color ␮i, a covariance ⌺i, and a count mi of
eral ahead of the robot in the laser map, so that all the total number of image pixels that were used to
grid cells within this quadrilateral are drivable. The train this Gaussian.
range of this quadrilateral is typically between 10 and When a new image is observed, the pixels in the
20 m ahead of the robot. An example of such a quad- drivable quadrilateral are mapped into a smaller
rilateral is shown in Figure 14共a兲. Using straightfor- number of k “local” Gaussians using the EM algo-
ward geometric projection, this quadrilateral is then rithm 共Duda & Hart, 1973兲, with k ⬍ n 共the covariance
mapped into the camera image, as illustrated in Fig-
of these local Gaussians are inflated by a small value
ures 15共a兲 and 15共b兲. An adaptive computer vision al-
so as to avoid overfitting兲. These k local Gaussians are
gorithm then uses the image pixels inside this quad-
rilateral as training examples for the concept of then merged into the memory of the learning algo-
drivable surface. rithm, in a way that allows for slow and fast adap-
The learning algorithm maintains a mixture of tation. The learning adapts to the image in two pos-
Gaussians that model the color of drivable terrain. sible ways; by adjusting the previously found
Each such mixture is a Gaussian defined in the red/ internal Gaussian to the actual image pixels, and by
green/blue 共RGB兲 color space of individual pixels; introducing new Gaussians and discarding older
the total number of Gaussians is denoted as n. For ones. Both adaptation steps are essential. The first en-
each mixture, the learning algorithm maintains a ables Stanley to adapt to slowly changing lighting
Figure 15. This figure illustrates the processing stages of the computer vision system: 共a兲 a raw image; 共b兲 the processed
image with the laser quadrilateral and a pixel classification; 共c兲 the pixel classification before thresholding; and 共d兲 horizon
detection for sky removal.

conditions; the second makes it possible to adapt rap- term makes sure that the Gaussians in memory can be
idly to a new surface color 共e.g., when Stanley moves moved in new directions as the appearance of the
from a paved to an unpaved road兲. drivable surface changes over time.
In detail, to update the memory, consider the jth For finding the drivable surface, the learned
local Gaussian. The learning algorithm determines Gaussians are used to analyze the image. The image
the closest Gaussian in the global memory, where analysis uses an initial sky removal step defined in
closeness is determined through the Mahalanobis Ettinger, Nechyba, Ifju & Waszak 共2003兲. A subse-
distance quent flood-fill step then removes additional sky pix-
els not found by the algorithm in Ettinger et al. 共2003兲.
d共i,j兲 = 共␮i − ␮j兲T共⌺i + ⌺j兲−1共␮i − ␮j兲. 共2兲 The remaining pixels are than classified using the
learned mixture of Gaussian, in the straightforward
Let i be the index of the minimizing Gaussian in the way. Pixels whose RGB value is near one or more of
memory. The learning algorithm then chooses one of the learned Gaussians are classified as drivable; all
two possible outcomes: other pixels are flagged as nondrivable. Finally, only
regions connected to the laser quadrilateral are la-
1. The distance d共i , j兲艋 ␾, where ␾ is an accep- beled as drivable.
tance threshold. The learning algorithm then Figure 15 illustrates the key processing steps.
assumes that the global Gaussian j is repre- Panel 共a兲 in this figure shows a raw camera image,
sentative of the local Gaussian i, and adap- and panel 共b兲 shows the image after processing. Pix-
tation proceeds slowly. The parameters of els classified as drivable are colored red, whereas
this global Gaussian are set to the weighted nondrivable pixels are colored blue. The remaining
mean: two panels in Figure 15 show intermediate process-
ing steps: The classification response before thresh-
m i␮ i m j␮ j olding 关panel 共c兲兴 and the result of the sky finder
␮i ← + , 共3兲关panel 共d兲兴.
mi + mj mi + mj
Due to the ability to create new Gaussians on the
fly, Stanley’s vision routine can adapt to new terrain
within seconds. Figure 16 shows data acquired at the
m i⌺ i m j⌺ j National Qualification Event 共NQE兲 of the DARPA
⌺i ← + , 共4兲
mi + mj mi + mj Grand Challenge. Here the vehicle moves from the
pavement to grass, both of which are drivable. The
sequence in Figure 16 illustrates the adaptation at
mi ← mi + mj , 共5兲 work: The boxed areas toward the bottom of the im-
age are the training region, and the red coloring in the
where mj is the number of pixels in the image image is the result of applying the learned classifier.
that correspond to the jth Gaussian. As is easily seen in Figure 16, the vision module suc-
2. The distance d共i , j兲 ⬎ ␾ for any Gaussian i in cessfully adapts from pavement to grass within less
the memory. This is the case when none of the than 1 s, while still correctly labeling the hay bales
Gaussian in memory are near the local and other obstacles.
Gaussian extracted form the image, where Under slowly changing lighting conditions, the
nearness is measured by the Mahalanobis system adapts more slowly to the road surface, mak-
distance. The algorithm then generates a new ing extensive use of past images in classification. This
Gaussian in the global memory, with param- is illustrated in the bottom row of Figure 17, which
eters ␮j, ⌺j, and mj. If all n slots are already shows results for a sequence of images acquired at the
taken in the memory, the algorithm “forgets” Beer Bottle pass, the most difficult passage in the 2005
the Gaussian with the smallest total pixel Race. Here, most of the terrain has a similar visual ap-
count mi, and replaces it by the new local pearance. The vision module, however, still compe-
Gaussian. tently segments the road. Such a result is only pos-
sible because the system balances the use of past
After this step, each counter mi in the memory is dis- images with its ability to adapt to new camera
counted by a factor of ␥ ⬍ 1. This exponential decay images.

Figure 16. These images illustrate the rapid adaptation of Stanley’s computer vision routines. When the laser predomi-
nately screens the paved surface, the grass is not classified as drivable. As Stanley moves into the grass area, the
classification changes. This sequence of images also illustrates why the vision result should not be used for steering
decisions, in that the grass area is clearly drivable, yet Stanley is unable to detect this from a distance.
Once a camera image has been classified, it is for safe navigation. In other words, the vision analy-
mapped into an overhead map, similar to the 2D map sis serves as an early warning system for obstacles be-
generated by the laser. We already encountered such yond the range of the laser sensors.
a map in Figure 14共b兲, which depicted the map of a In developing the vision routines, the research
straight road. Since certain color changes are natural team investigated a number of different learning al-
even on flat terrain, the vision map is not used for gorithms. One of the primary alternatives to the gen-
steering control. Instead, it is used exclusively for ve- erative mixture of the Gaussian method was a dis-
locity control. When no drivable corridor is detected criminative method, which uses boosting and
within a range of 40 m, the robot simply slows down decision stumps for classification 共Davies & Lienhart,
to 25 mph, at which point the laser range is sufficient 2006兲. This method relies on examples of nondrivable
Figure 17. Processed camera images in flat and mountainous terrain 共Beer Bottle Pass兲.

Table II. Road detection rate for the two primary machine learning methods, broken down into different ranges. The
comparison yields no conclusive winner.
Flat desert roads Mountain roads
Drivable terrain detection rate Discriminative Generative Discriminative Generative

共m兲 training 共%兲 training 共%兲 training 共%兲 training 共%兲
10–20 93.25 90.46 80.43 88.32

20–35 95.90 91.18 76.76 86.65
35–50 94.63 87.97 70.83 80.11
50+ 87.13 69.42 52.68 54.89
False positives, all ranges 3.44 3.70 0.50 2.60
terrain, which were extracted using an algorithm pass filters are implemented as one-dimensional
similar to the one for finding a drivable quadrilateral. Kalman filters 共KFs兲. The state of each filter is the
A performance evaluation, carried out using inde- lateral distance between the road boundary and the
pendent test data gathered on the 2004 Race Course, center of the RDDF. The KFs search for possible ob-
led to inconclusive results. Table II shows the classi- stacles along a discrete search pattern orthogonal to
fication accuracy for both methods; for flat desert the RDDF, as shown in Figure 18共a兲. The largest free
roads and mountain roads. The generative mixture of offset is the “observation” to the KF, in that it estab-
Gaussian methods was finally chosen because it does lishes the local measurement of the road boundary.
not require training examples of nondrivable terrain, So, if multiple parallel roads exist in Stanley’s field
which can be difficult to obtain in flat open lakebeds. of view, separated by a small berm, the filter will
only trace the innermost drivable area.
By virtue of KF integration, the road boundaries
7. ROAD PROPERTY ESTIMATION change slowly. As a result, small obstacles—or mo-
mentary situations without side obstacles—affect the
7.1. Road Boundary road boundary estimation only minimally; however,
persistent obstacles that occur over an extended pe-
One way to avoid obstacles is to detect them and riod of time do have a strong effect.
drive around them. This is the primary function of Based on the output of these filters, Stanley de-
the laser mapper. Another effective method is to fines the road to be the center of the two boundaries.
drive in such a way that minimizes the a priori The road center’s lateral offset is a component in
chances of encountering an obstacle. This is possible
scoring trajectories during path planning, as will be
because obstacles are rarely uniformly distributed in
discussed further below. In the absence of other con-
the world. On desert roads, obstacles–such as rocks,
tingencies, Stanley slowly converges to the esti-
brush, and fence posts–exist most often along the
mated road center. Empirically, we found that this
sides of the road. By simply driving down the
middle of the road, most obstacles on desert roads driving technique stays clear of the vast majority of
can be avoided without ever detecting them! natural obstacles on desert roads. While road center-
One of the most beneficial components of Stan- ing is clearly only a heuristic, we found it to be
ley’s navigation routines, thus, is a method for stay- highly effective in extensive desert tests.
ing near the center of the road. To find the road cen- Figure 18共b兲 shows an example result of the road
ter, Stanley uses probabilistic low-pass filters to estimator. The blue corridor shown there is Stanley’s
determine both road sides based using the laser best estimate of the road. Notice that the corridor is
map. The idea is simple; In expectation, the road confined by two small berms, which are both de-
sides are parallel to the RDDF. However, the exact tected by the laser mapper. This module plays an
lateral offset of the road boundary to the RDDF cen- important role in Stanley’s ability to negotiate desert
ter is unknown and varies over time. Stanley’s low- roads.

Figure 18. 共a兲 Search regions for the road detection module: The occurrence of obstacles is determined along a sequence
of lines parallel to the RDDF; and 共b兲 the result of the road estimator is shown in blue, behind the vehicle. Notice that the
road is bounded by two small berms.
7.2. Terrain Ruggedness shock experienced by the vehicle due to excitation

by the terrain. Empirically, this filtered acceleration
In addition to avoiding obstacles and staying
appears to vary linearly with velocity. 共see Figure
centered along the road, another important compo-
19兲. In other words, doubling the maximum speed of
nent of safe driving is choosing an appropriate ve-
the vehicle over a section of terrain will approxi-
locity 共Iagnemma & Dubowsky, 2004兲. Intuitively
mately double the maximum differential accelera-
speaking, desert terrain varies from flat and smooth tion imparted on the vehicle. In Sec. 9.1, this rela-
to steep and rugged. The type of the terrain plays an tionship will be used to derive a simple rule for
important role in determining the maximum safe ve- setting maximum velocity to approximately bound
locity of the vehicle. On steep terrain, driving too the maximum shock imparted on the vehicle.
fast may lead to fishtailing or sliding. On rugged
terrain, excessive speeds may lead to extreme shocks
that can damage or destroy the robot. Thus, sensing
the terrain type is essential for the safety of the ve- 8. PATH PLANNING
hicle. In order to address these two situations, Stan- As was previously noted, the RDDF file provided by
ley’s velocity controller constantly estimates terrain DARPA largely eliminates the need for any global
slope and ruggedness and uses these values to set path planning. Thus, the role of Stanley’s path plan-
intelligent maximum speeds. ner is primarily local obstacle avoidance. Instead of
The terrain slope is taken directly from the vehi- planning in the global coordinate frame, Stanley’s
cle’s pitch estimate, as computed by the UKF. Bor- path planner was formulated in a unique coordinate
rowing from Brooks & Iagnemma 共2005兲, the terrain system: Perpendicular distance, or “lateral offset” to
ruggedness is measured using the vehicle’s z accel- a fixed base trajectory. Varying the lateral offset
erometer. The vertical acceleration is band-pass fil- moves Stanley left and right with respect to the base
tered to remove the effect of gravity and vehicle vi- trajectory, much like a car changes lanes on a high-
bration, while leaving the oscillations in the range of way. By intelligently changing the lateral offset, Stan-
the vehicle’s resonant frequency. The amplitude of ley can avoid obstacles at high speeds while making
the resulting signal is a measurement of the vertical fast progress along the course.

vehicle safety. Using a base trajectory that is

smoother than the original RDDF will allow
Stanley to travel faster in turns and follow the
intended course with higher accuracy.
• Matched curvature. While the RDDF corri-
dor is parallel to the road in expectation, the
curvature of the road is poorly predicted by
the RDDF file in turns, again due to the finite
number of waypoints. By default, Stanley
will prefer to drive parallel to the base trajec-
tory, so picking a trajectory that exhibits cur-
vature that better matches the curvature of
the underlying desert roads will result in
fewer changes in lateral offset. This will also
result in smoother, faster driving.
Figure 19. The relationship between velocity and im-
parted acceleration from driving over a fixed-sized ob- Stanley’s base trajectory is computed before the
stacle at varying speeds. The plot shows two distinct reac- race in a four-stage procedure.
tions to the obstacle; one up and one down. While this
relation is ultimately nonlinear, it is well modeled by a 1. First, points are added to the RDDF in pro-
linear function within the range relevant for desert portion to the local curvature 关see Figure
driving. 20共a兲兴.
2. The coordinates of all points in the up-
sampled trajectory are then adjusted through
The base trajectory that defines lateral offset is least-squares optimization. Intuitively, this
simply a smoothed version of the skeleton of the optimization adjusts each waypoint, so as to
RDDF corridor. It is important to note that this base minimize the curvature of the path while
staying as close as possible to the waypoints
trajectory is not meant to be an optimal trajectory in
in the original RDDF. The resulting trajectory
any sense; it serves as a baseline coordinate system
is still piece-wise linear, but it is significantly
upon which obstacle avoidance maneuvers are con- smoother than the original RDDF.
tinuously layered. The following two sections will Let x1 , . . . , xN be the waypoints of the base tra-
describe the two parts to Stanley’s path planning soft- jectory to be optimized. For each of these
ware: The path smoother that generates the base tra- points, we are given a corresponding point
jectory before the race, and the online path planner along the original RDDF, which shall be de-
which is constantly adjusting Stanley’s trajectory. noted yi. The points x1 , . . . , xN are obtained by
minimizing the following additive function:
8.1. Path Smoothing argmin 兺兩yi − xi兩2

x1,. . .,xN i
Any path can be used as a base trajectory for plan-
ning in lateral offset space. However, certain quali- 共xn+1 − xn兲 · 共xn − xn−1兲
ties of base trajectories will improve overall perfor- − ␤兺
n 兩xn+1 − xn兩兩xn − xn−1兩
mance.
+ 兺 fRDDF共xn兲, 共6兲
• Smoothness. The RDDF is a coarse descrip- n
tion of the race corridor and contains many
sharp turns. Blindly trying to follow the where 兩yi − xi兩2 is the quadratic distance be-
RDDF waypoints would result in both sig- tween the waypoint xi and the corresponding
nificant overshoot and high lateral accelera- RDDF anchor point yi, and the index variable
tions, both of which could adversely affect i iterates over the set of points xi. Minimizing

Figure 20. Smoothing of the RDDF: 共a兲 Adding additional points; 共b兲 the trajectory after smoothing 共shown in red兲; 共c兲 a
smoothed trajectory with a more aggressive smoothing parameter. The smoothing process takes only 20 seconds for the
entire 2005 course.
this quadratic distance for all points i ensures from a bounded deceleration constraint. The
that the base trajectory stays close to the lateral acceleration constraint forces the ve-
original RDDF. The second expression in Eq. hicle to slow down appropriately in turns.
共6兲 is a curvature term; It minimizes the angle When computing these limits, we bound the
between two consecutive line segments in the lateral acceleration of the vehicle to
base trajectory by minimizing the dot prod- 0.75 m/s2, in order to give the vehicle
uct of the segment vectors. Its function is to enough maneuverability to safely avoid ob-
smooth the trajectory: The smaller the angle, stacles in curved segments of the course. The
the smoother the trajectory. The scalar ␤ bounded deceleration constraint forces the
trades off these two objectives, and is a pa- vehicle to slow down in anticipation of turns
rameter in Stanley’s software. The function and changes in DARPA speed limits.
fRDDF共xn兲 is a differentiable barrier function
that goes to infinity as a point xn ap- Figure 20 illustrates the effect of smoothing on a short
proaches the RDDF boundary, but is near segment of the RDDF. Panel 共a兲 shows the RDDF and
zero inside the corridor away from the the upsampled base trajectory before smoothing.
boundary. As a result, the smoothed trajec- Panels 共b兲 and 共c兲 show the trajectory after smoothing
tory is always inside the valid RDDF cor- 共in red兲, for different values of the parameter ␤. The
ridor. The optimization is performed with a entire data preprocessing step is fully automated, and
fast version of conjugate gradient descent,
requires only approximately 20 s of computation
which moves RDDF points freely in 2D
time on a 1.4 GHz laptop, for the entire 2005 Race
space.
Course. This base trajectory is transferred onto Stan-
3. The next step of the path smoother involves
ley, and the software is ready to go. No further infor-
cubic spline interpolation. The purpose of
mation about the environment or the race is provided
this step is to obtain a path that is differen-
tiable. This path can then be resampled to the robot.
efficiently. It is important to note that Stanley does not
4. The final step of path smoothing pertains to modify the original RDDF file. The base trajectory is
the calculation of the speed limit attached to only used as the coordinate system for obstacle
each waypoint of the smooth trajectory. avoidance. When evaluating whether particular tra-
Speed limits are the minimum of three quan- jectories stay within the designated race course,
tities: 共1兲 The speed limit from corresponding Stanley checks against the original RDDF file. In this
segment of the original RDDF, 共2兲 a speed way, the preprocessing step does not affect the inter-
limit that arises from a bound on lateral ac- pretation of the corridor constraint imposed by the
celeration, and 共3兲 a speed limit that arises rules of the race.

8.2. Online Path Planning edly layering these simple maneuvers on top of the
base trajectory can result in quite sophisticated
Stanley’s online planning and control system is simi-
trajectories.
lar to the one described in Kelly & Stentz 共1998兲. The
The second parameter in the path search allows
online component of the path planner is responsible
the planner to control the urgency of obstacle avoid-
for determining the actual trajectory of the vehicle
ance. Discrete obstacles in the road, such as rocks or
during the race. The goal of the planner is to com-
plete the course as fast as possible, while success- fence posts, often require the fastest possible change
fully avoiding obstacles and staying inside the in lateral offset. Paths that change lateral offset as
RDDF corridor. In the absence of obstacles, the plan- fast as possible without violating the lateral accelera-
ner will maintain a constant lateral offset from the tion constraint are called swerves. Slow changes in
base trajectory. This results in driving a path parallel the positions of road boundaries require slow
to the base trajectory, but possibly shifted left or smooth adjustment to the lateral offset. Trajectories
right. If an obstacle is encountered, Stanley will plan with the slowest possible change in lateral offset for
a smooth change in lateral offset that avoids the ob- a given planning horizon are called nudges. Swerves
stacle and can be safely executed. Planning in lateral and nudges span a spectrum of maneuvers appro-
offset space also has the advantage that it gracefully priate for high-speed obstacle avoidance: Fast
handles GPS error. GPS error may systematically changes for avoiding head on obstacles, and slow
shift Stanley’s position estimate. The path planner changes for smoothly tracking the road center.
will simply adjust the lateral offset of the current Swerves and nudges are illustrated in Figure 21. On
trajectory to recenter the robot in the road. a straight road, the resulting trajectories are similar
The path planner is implemented as a search al- to those of Ko & Simmons’s 共1998兲 lane curvature
gorithm that minimizes a linear combination of con- method.
tinuous cost functions, subject to a fixed vehicle The path planner is executed at 10 Hz. The path
model. The vehicle model includes several kinematic planner is ignorant to actual deviations from the ve-
and dynamic constraints including maximum lateral hicle and the desired path, since those are handled
acceleration 共to prevent fishtailing兲, maximum steer- by the low-level steering controller. The resulting
ing angle 共a joint limit兲, maximum steering rate trajectory is therefore always continuous. Fast
共maximum speed of the steering motor兲, and maxi- changes in lateral offset 共swerves兲 will also include
mum deceleration. The cost functions penalize run- braking in order to increase the amount of steering
ning over obstacles, leaving the RDDF corridor, and the vehicle can do without violating the maximum
the lateral offset from the current trajectory to the lateral acceleration constraint.
sensed center of the road surface. The soft con- Figure 22 shows an example situation for the
straints induce a ranking of admissible trajectories. path planner. Shown here is a situation taken from
Stanley chooses the best such trajectory. In calculat- Beer Bottle Pass, the most difficult passage of the
ing the total path costs, unknown territory is treated 2005 Grand Challenge. This image only illustrates
the same as drivable surface, so that the vehicle does one of the two search parameters: The lateral offset.
not swerve around unmapped spots on the road, or It illustrates the process through which trajectories
specular surfaces, such as puddles. are generated by gradually changing the lateral off-
At every time step, the planner considers trajec- set relative to the base trajectory. By using the base
tories drawn from a 2D space of maneuvers. The trajectory as a reference, path planning can take
first dimension describes the amount of lateral offset place in a low-dimensional space, which we found
to be added to the current trajectory. This parameter to be necessary for real-time performance.
allows Stanley to move left and right, while still
staying essentially parallel to the base trajectory. The
second dimension describes the rate at which Stan-
ley will attempt to change to this lateral offset. The 9. REAL-TIME CONTROL
lookahead distance is speed dependent and ranges
from 15 to 25 m. All candidate paths are run Once the intended path of the vehicle has been de-
through the vehicle model to ensure that obey the termined by the path planner, the appropriate
kinematic and dynamic vehicle constraints. Repeat- throttle, brake, and steering commands necessary to

Figure 21. Path planning in a 2D search space: 共a兲 Paths that change lateral offsets with the minimum possible lateral
acceleration 共for a fixed plan horizon兲; and 共b兲 the same for the maximum lateral acceleration. The former are called
“nudges,” and the latter are called “swerves.”
achieve that path must be computed. This control monitor, the velocity recommender, and the low-
problem will be described in two parts: The velocity level velocity controller. The low-level velocity con-
controller and steering controller. troller translates velocity commands from the first
three modules into actual throttle and brake com-
mands. The implemented velocity is always the
9.1. Velocity Control minimum of the three recommended speeds. The
Multiple software modules have input into Stanley’s path planner will set a vehicle velocity based on the
velocity, most notably the path planner, the health base trajectory speed limits and any braking due to
Figure 22. Snapshots of the path planner as it processes the drivability map. Both snapshots show a map, the vehicle,
and the various nudges considered by the planner. The first snapshot stems from a straight road 共Mile 39.2 of the 2005
Race Course兲. Stanley is traveling 31.4 mph; and hence, can only slowly change lateral offsets due to the lateral accelera-
tion constraint. The second example is taken from the most difficult part of the 2005 DARPA Grand Challenge, a moun-
tainous area called Beer Bottle Pass. Both images show only nudges for clarity.

Figure 23. Velocity profile of a human driver and of Stanley’s velocity controller in rugged terrain. Stanley identifies
controller parameters that match human driving. This plot compares human driving with Stanley’s control output.
swerves. The vehicle health monitor will lower the In bumpy terrain, slowing down also changes the
maximum velocity due to certain preprogrammed frequency at which the bumps pass, reducing the
conditions, such as GPS blackouts or critical system effect of resonance.
failures. The velocity recommender is characterized by
The velocity recommender module sets an ap- two parameters: The maximum allowable shock,
propriate maximum velocity based on estimated ter- and the linear recovery rate. Both are learned from
rain slope and roughness. The terrain slope affects human driving. More specifically, by recording the
the maximum velocity if the pitch of the vehicle ex- velocity profile of a human in rugged terrain, Stan-
ceeds 5°. Beyond 5° of slope, the maximum velocity ley identifies The parameters that most closely
of the vehicle is reduced linearly to values that, in match the human driving profile. Figure 23 shows
the extreme, restrict the vehicle’s velocity to 5 mph. the velocity profile of a human driver in a mountain-
The terrain ruggedness is fed into a controller with ous area of the 2004 Grand Challenge Course 共the
hysteresis that controls the velocity setpoint to ex- “Daggett Ridge”兲. It also shows the profile of Stan-
ploit the linear relationship between filtered vertical ley’s controller for the same data set. Both profiles
acceleration amplitude and velocity; see Sec. 7.2. If tend to slow down in the same areas. Stanley’s pro-
rough terrain causes a vibration that exceeds the file, however, is different in two ways: The robot de-
maximum allowable threshold, the maximum veloc- celerates much faster than a person, and its recovery
ity is reduced linearly such that continuing to en- is linear whereas the person’s recovery is nonlinear.
counter similar terrain would yield vibrations which The fast acceleration is by design, to protect the ve-
exactly meet the shock limit. Barring any further hicle from further impact.
shocks, the velocity limit is slowly increased linearly Once the planner, velocity recommender, and
with distance traveled. health monitor have all submitted velocities, the
This rule may appear odd, but it has great prac- minimum of these speeds is implemented by the ve-
tical importance; it reduces the Stanley’s speed when locity controller. The velocity controller treats the
the vehicle hits a rut. Obviously, the speed reduction brake cylinder pressure and throttle level as two op-
occurs after the rut is hit, not before. By slowly re- posing single-acting actuators that exert a longitudi-
covering speed, Stanley will approach nearby ruts at nal force on the car. This is a very close approxima-
a much lower speed. As a result, Stanley tends to tion for the brake system, and was found to be an
drive slowly in areas with many ruts, and only re- acceptable simplification of the throttle system. The
turns to the base trajectory speed when no ruts have controller computes a single error metric, equal to a
been encountered for a while. While this approach weighted sum of the velocity error and the integral
does not avoid isolated ruts, we found it to be highly of the velocity error. The relative weighting deter-
effective in avoiding many shocks that would other- mines the trade-off between disturbance rejection
wise harm the vehicle. Driving over wavy terrain and overshoot. When the error metric is positive, the
can be just as hard on the vehicle as driving on ruts. brake system commands a brake cylinder pressure

wheels match the global orientation of the trajectory.

This is illustrated in Figure 24. The angle ␺ in this
diagram describes the orientation of the nearest path
segment, measured relative to the vehicle’s own ori-
entation. In the absence of any lateral errors, the con-
trol law points the front wheels parallel to the plan-
ner trajectory.
The basic steering angle control law is given by
kx共t兲
␦共t兲 = ␺共t兲 + arctan , 共7兲
Figure 24. Illustration of the steering controller. With u共t兲
zero cross-track error, the basic implementation of the
steering controller steers the front wheels parallel to the
path. When cross-track error is perturbed from zero, it is
where k is a gain parameter. The second term adjusts
nulled by commanding the steering according to a nonlin- the steering in 共nonlinear兲 proportion to the cross-
ear feedback function. track error x共t兲: The larger this error, the stronger the
steering response toward the trajectory.
Using a linear bicycle model with infinite tire
proportional to the PI error metric; and when it is stiffness and tight steering limitations 共see Gillespie,
negative, the throttle level is set proportional to the 1992兲 results in the following effect of the control
negative of the PI error metric. By using the same PI law:
冉冊
error metric for both actuators, the system is able to
avoid the chatter and dead bands associated with kx共t兲 − kx共t兲
冑冉冊
ẋ共t兲 = − u共t兲sin arctan = ,
opposing single-acting actuators. To realize the com- u共t兲 kx共t兲 2
manded brake pressure, the hysteretic brake actua- 1+
u共t兲
tor is controlled through saturated proportional
feedback on the brake pressure, as measured by the 共8兲
Touareg, and reported through the CAN bus
interface. and hence for small cross track error,
x共t兲 ⬇ x共0兲exp − kt. 共9兲

9.2. Steering Control
The steering controller accepts as input the trajectory Thus, the error converges exponentially to x共t兲 = 0.
generated by the path planner, the UKF pose and The parameter k determines the rate of convergence.
velocity estimate, and the measured steering wheel As cross-track error increases, the effect of the arctan
angle. It outputs steering commands at a rate of function is to turn the front wheels to point straight
20 Hz. The function of this controller is to provide toward the trajectory, yielding convergence limited
closed-loop tracking of the desired vehicle path, as only by the speed of the vehicle. For any value of
determined by the path planner, on quickly varying x共t兲, the differential equation converges monotoni-
potentially rough terrain. cally to zero. Figure 25 shows phase portrait dia-
The key error metric is the cross-track error, x共t兲, grams for Stanley’s final controller in simulation, as
as shown in Figure 24, which measures the lateral a function of the error x共t兲 and the orientation ␺共t兲,
distance of the center of the vehicle’s front wheels including the effect of steering input saturation.
from the nearest point on the trajectory. The idea These diagrams illustrate that the controller con-
now is to command the steering by a control law verges nicely for the full range attitudes and a wide
that yields an x共t兲 that converges to zero. range of cross-track errors, in the example of two
Stanley’s steering controller, at the core, is based different velocities.
on a nonlinear feedback function of the cross-track This basic approach works well for lower
error, for which exponential convergence can be speeds, and a variant of it can even be used for re-
shown. Denote the vehicle speed at time t by u共t兲. In verse driving. However, it neglects several impor-
the error-free case, using this term, Stanley’s front tant effects. There is a discrete variable time delay in

Figure 25. Phase portrait for k = 1 at 10 and 40 m per second, respectively, for the basic controller, including the effect of
steering input saturation.
the control loop, inertia in the steering column, and Sonoran Desert near Phoenix, AZ. In the weeks lead-
more energy to dissipate as speed increases. These ing up to the race, the team permanently moved to
effects are handled by simply damping the differ- Arizona, where it enjoyed the hospitality of Volk-
ence between steering command and the measured swagen of America’s Arizona Proving Grounds. Fig-
steering wheel angle, including a term for yaw ure 26 shows examples of hardware testing in ex-
damping. Finally, to compensate for the slip of the treme offroad terrain; these pictures were taken
actual pneumatic tires, the vehicle is commanded to while the vehicle was operated by a person.
have a steady-state yaw offset that is a nonlinear In developing Stanley, the Stanford Racing Team
function of the path curvature and the vehicle speed, adhered to a tight development and testing sched-
based on a bicycle vehicle model, with slip, that was ule, with clear milestones along the way. Emphasis
calibrated and verified in testing. These terms com- was placed on early integration, so that an end-to-
bine to stabilize the vehicle and drive the cross-track end prototype was available nearly a year before the
error to zero, when run on the physical vehicle. The race. The system was tested periodically in desert
resulting controller has proven stable in testing on environments representative of the team’s expecta-
terrain from pavement to deep off-road mud tion for the Grand Challenge race. In the months
puddles, and on trajectories with tight enough radii leading up to the race, all software and hardware
of curvature to cause substantial slip. It typically modules were debugged and subsequently frozen.
demonstrates tracking error that is on the order of The development of the system terminated well
the estimation error of this system. ahead of the race.
The primary measure of system capability was
“MDBCF”—mean distance between catastrophic
failures. A catastrophic failure was defined as a con-
10. DEVELOPMENT PROCESS AND RACE dition under which a human driver had to inter-
RESULTS vene. Common failures involved software problems,
such as the one shown in Figure 9; occasional fail-
ures were caused by the hardware, e.g., the vehicle
10.1. Race Preparation power system. In December 2004, the MDBCF was
The race preparation took place at three different lo- approximately 1 mile. It increased to 20 miles in July
cations: Stanford University, the 2004 Grand Chal- 2005. The last 418 miles before the National Qualifi-
lenge Course between Barstow and Primm, and the cation Event were free of failures; this included a

Figure 26. Vehicle testing at the Volkswagen Arizona Proving Grounds, manual driving.
single 200-mile run over a cyclic testing course. At them. As a consequence, the team felt that the tech-
that time, the system development was suspended, nical risks associated with the RADAR system out-
Stanley’s lateral navigation accuracy was approxi- weighed its benefits, and made the decision not to
mately 30 cm. The vehicle had logged more than use RADAR in the race.
1,200 autonomous miles.
In preparing for this race, the team also tested
sensors that were not deployed in the final race.
Among them was an industrial strength stereo vi-
sion sensor with a 33 cm baseline. In early experi-
ments, we found that the stereo system provided ex-
cellent results in the short range, but lagged behind
the laser system in accuracy. The decision not to use
stereo was simply based on the observation that it
added little to the laser system. A larger baseline
might have made the stereo more useful at longer
ranges, but was unfortunately not available.
The second sensor that was not used in the race
was the 24 GHz RADAR system. The RADAR uses a
linear frequency shift keying modulated 共LFMSK兲
transmit wave form; it is normally used for adaptive
cruise control. After carefully tuning gains and ac-
ceptance thresholds of the sensor, the RADAR
proved highly effective in detecting large frontal ob-
stacles such as abandoned vehicles in desert terrain.
Similar to the monovision system in Sec. 6, the RA-
DAR was tasked to screen the road at a range be-
yond the laser sensors. If a potential obstacle was
detected, the system limits Stanley’s speed to
25 mph so that the lasers could detect the obstacle in
time for collision avoidance.
While the RADAR system proved highly effec-
tive in testing, two reasons prevented its use in the
race. The first reason was technical: During the
NQE, the USB driver of the receiving computer re-
peatedly caused trouble, sometimes stalling the re- Figure 27. This map shows Stanley’s path. The thickness
ceiving computer. The second reason was pragmati- of the trajectory indicates Stanley’s speed 共thicker means
cal. During the NQE, it became apparent that the faster兲. At the locations marked by the red “x”s, the race
organizers paused Stanley because of the close proximity
probability of encountering large frontal obstacles of CMU’s H1ghlander robot. At Mile 101.5, H1ghlander
was small in high-speed zones; and even if those was paused and Stanley passed. This location is marked
existed, the vision system would very likely detect by a green “x”.

Figure 28. Passing CMU’s H1ghlander robot: The left column shows a sequence of camera images, the center column the
processed images with obstacle information overlayed; and the right column, the 2D map derived from the processed
image. The vision routine detects H1ghlander as an obstacle at a 40 m range, approximately twice the range of the lasers.
10.2. National Qualification Event course in the first run, 13 in the second run, 18 in the
third run, and 21 in the fourth run. Stanley’s times
The NQE took place from September 27 to October 5
on the California Speedway in Fontana, CA. Like were competitive, but not the fastest 共Run 1—10:38;
most competitive robots, Stanley qualified after four Run 2—9:12; Run 3—11:06; and Run 4—11:06兲. How-
test runs. From the 43 semifinalists, 11 completed the ever, Stanley was the only vehicle that cleared all 50

first, the top-seeded H1ghlander robot was the only

robot encountered by Stanley during the race.
As noted in the Introduction of this paper, Stan-
ley finished first, at an unmatched finishing time of
6 h, 53 min, and 58 s. Its overall average velocity
was 19.1 mph. However, Stanley’s velocity varied
wildly during the race. Initially, the terrain was flat
and the speed limits allowed for much higher
speeds. Stanley reached its top speed of 38.0 mph at
Mile 5.43, 13 min and 47 s into the race. Its maxi-
mum average velocity during the race was 24.8 mph,
Figure 29. Laser model of CMU’s H1ghlander robot,
taken at Mile 101.5. which Stanley attained after 16 min and 56 s, at Mile
7.00. Speed limits then forced Stanley to slow down.
Between Mile 84.9 and 88.1, DARPA restricted the
maximum velocity to 10 mph. Shortly thereafter, at
gates in every run, and avoided collisions with all of Mile 90.6 and 4 h, 57 min, and 7 s into the race, Stan-
the obstacles. This flawless performance earned ley attained its minimum average velocity of
Stanley the number two starting position, behind 18.3 mph. The total profile of velocities is shown in
CMU’s H1ghlander robot and ahead of the slightly Figure 30.
faster Sandstorm robot, also by CMU. As explained in this paper, Stanley uses a num-
ber of strategies to determine the actual travel speed.
During 68.2% of the course, Stanley’s velocity was
10.3. The Race limited as precalculated, by following the DARPA
speed limits or the maximum lateral acceleration
At approximately 4:10 am on October 8, 2005, the constraints in turns. For the remaining 31.8%, Stan-
Stanford Racing Team received the race data, which ley chose to slow down dynamically, as the result of
consisted of 2,935 GPS-referenced coordinates, along its sensor measurements. In 18.1%, the slow down
with speed limits of up to 50 mph. Stanley started was the result of rugged or steep terrain. The vision
the race at 6:35 am on October 8, 2005. The robot module caused Stanley to slow down to 25 mph for
immediately picked up speed and drove at or just 13.1% of the total distance; however, without the vi-
below the speed limit. 3 h, 45 min, and 22 s into the sion module Stanley would have been forced to a
race, at Mile 73.5, DARPA paused Stanley for the 25 mph maximum speed, which would have re-
first time, to give more space to CMU’s H1ghlander sulted in a finishing time of approximately 7 h and
robot, which had started five minutes ahead of Stan- 5 min, possibly behind CMU’s Sandstorm robot. Fi-
ley. The first pause lasted 2 min 45 s. Stanley was nally, for 0.6% of the course, Stanley drove slower
paused again only 5 min 40 s later, at Mile 74.9 共3 h, because it was denied GPS readings. Figure 31 illus-
53 min, and 47 s into the race兲. This time the pause trates the effect of terrain ruggedness on the overall
lasted 6 min 35 s, for a total pause time of 9 min velocity. The curve on the top illustrates the magni-
20 s. The locations of the pauses are shown in Figure tude at which Stanley slowed down to accommodate
27. From this point on, Stanley repeatedly ap- rugged terrain; the bottom diagram shows the alti-
proached H1ghlander within a few hundred yards. tude profile, as provided by DARPA. The terrain
Even though Stanley was still behind H1ghlander, it ruggedness triggers mostly in mountainous terrain.
was leading the race. We believe that the ability to adapt the speed to the
5 h, 24 min, 45 s into the race, DARPA finally ruggedness of the terrain was an essential ingredient
paused H1ghlander and allowed Stanley to pass. in Stanley’s success.
The passing happened a Mile 101.5; the location is Stanley also encountered some unexpected diffi-
marked by a green circle in Figure 27. Figure 28 culties along the course. Early on in the race, Stan-
shows processed camera images of the passing pro- ley’s laser data stream repeatedly stalled for dura-
cess acquired by Stanley, and Figure 29 depicts a 3D tions of 300 to 1,100 ms. There were a total of 17
model of H1ghlander as it is being passed. Since incidents, nearly all of which occurred between Mile
Stanley started in second pole position and finished 22 and Mile 35. The resulting inaccurate time stamp-

Figure 30. Stanley’s cumulative velocity.
ing of the laser data led to the insertion of phantom swerved on an open lake bed without any obstacles,
obstacles into the map. In four of those cases, those as shown in Figure 32共b兲. At no point was the ve-
incidents resulted in a significant swerve. The two hicle in jeopardy, as the berm that was traversed was
most significant of these swerves are shown in Fig- drivable. However, as a result of these errors, Stan-
ure 32. Both of those swerves were quite noticeable. ley slowed down a number of times between Miles
In one case, Stanley even drove briefly on the berm 22 and 35. Thus, the main effect of these incidents
as shown in Figure 32共a兲; in the other, Stanley was a loss of time early in the race. The data stream

Figure 31. This diagram shows where the road condi-

tions forced Stanley to slow down along the race course.
Slow down predominately occurred in the mountains.
stalling problem vanished entirely after Mile 37.85. It

only reoccurred once at Mile 120.66, without any vis-
ible change of the driving behavior.
During 4.7% of the Grand Challenge, the GPS
reported 60 cm error or more. Naturally, this num-
ber represents the unit’s own estimate, which may
not necessarily be accurate. However, this raises the
question of how important online mapping and path
Figure 33. Sensor image from the Beer Bottle Pass, the
planning was in this race. most difficult passage of the DARPA Grand Challenge.
Stanley frequently moved away from the center
axis of the RDDF. On average, the lateral offset was
±74 cm. The maximum lateral offset during the race
was 10.7 m, which was the result of the swerve interest is the map in Figure 34共b兲. Here the DARPA-
shown in Figure 32共c兲. However, such incidents were provided corridor is marked by the two solid blue
rare, and in nearly all cases nonzero lateral offsets lines. This image illustrates that the berm on Stan-
were the results of obstacles in the robot’s path. ley’s left reaches well into the corridor. Stanley
An example situation is depicted in Figure 33. drives as far left as the corridor constraint allows.
This figure shows raw laser data from the Beer Figure 35 shows a histogram of lateral offsets for the
Bottle Pass, the most difficult section of the course. Beer Bottle Pass. On average, Stanley drove 66 cm to
An image of this pass is depicted in Figure 34共a兲. Of the right of the center of the RDDF in this part of the
Figure 32. Problems during the race caused by a stalling of the laser data stream. In both cases, Stanley swerved around
phantom obstacles; at Mile 22.37 Stanley drove on the berm. None of these incidents led to a collision or an unsafe driving
situation during the race.

Figure 34. Image of the Beer Bottle pass, and snapshot of the map acquired by the robot. The two blue contours in the
map mark the GPS corridor provided by DARPA, which aligns poorly with the map data. This analysis suggests that a
robot that followed the GPS via points blindly would likely have failed to traverse this narrow mountain pass.
race. We suspect that driving 66 cm further to the precise. We believe that those techniques, along with
left would have been fatal in many places. This the extensive testing that took place, contributed sig-
sheds light on the importance of Stanley’s ability to nificantly to Stanley’s success in this race.
react to the environment in driving. Simply follow- While the DARPA Grand Challenge was a mile-
ing the GPS points would likely have prevented stone in the quest for self-driving cars, it left open a
Stanley from finishing this race. number of important problems. Most important
among those was the fact that the race environment
was static. Stanley is unable to navigate in traffic. For
11. DISCUSSION autonomous cars to succeed, robots, such as Stanley,
must be able to perceive and interact with moving
This paper provides a comprehensive survey of the traffic. While a number of systems have shown im-
winning robot of the DARPA Grand Challenge. Stan- pressive results 共Dickmanns et al., 1994; Hebert,
ley, developed by the Stanford Racing Team in col- Thorpe & Stentz, 1997; Pomerleau & Jochem, 1996兲,
laboration with its primary supporters, relied on a further research is needed to achieve the level of re-
software pipeline for processing sensor data and de- liability necessary for this demanding task. Even
termining suitable steering, throttle, brake, and gear within the domain of driving in static environments,
shifting commands. Stanley’s software can only handle limited types of
From a broad perspective, Stanley’s software mir- obstacles. For example, the present software would
rors common methodology in autonomous vehicle be unable to distinguish tall grass from rocks, a re-
control. However, many of the individual modules search topic that has become highly popular in recent
relied on state-of-the-art artificial intelligence tech- years 共Dima & Hebert, 2005; Happold, Ollis &
niques. The pervasive use of machine learning, both
Johnson, 2006; Wellington, Courville & Stentz, 2005兲.
ahead and during the race, made Stanley robust and
ACKNOWLEDGMENTS
The Stanford Racing Team 共SRT兲 was sponsored by
four Primary Supporters: Volkswagen of America’s
Electronics Research Lab, Mohr Davidow Ventures,
Android, and Red Bull. The Primary Supporters, to-
gether with the Stanford team leaders, formed the
Figure 35. Histogram of lateral offsets on the beer bottle SRT Steering Committee, which oversaw the SRT op-
pass. The horizontal units are in centimeters. erations. The SRT also received support from Intel

Research, Honeywell, Tyzx, Inc., and Coverity, Inc. Happold, M., Ollis, M., & Johnson, N. 共2006兲. Enhancing
Generous financial contributions were made by supervised terrain classification with predictive unsu-
pervised learning. In G. Sukhatme, S. Schaal, W. Bur-
David Cheriton, the Johnson Family, and Vint Cerf.
gard, & D. Fox 共Eds.兲, Proceedings of the Robotics Sci-
A huge number of individuals at Stanford, Volk- ence and Systems Conference, Philadelphia, PA.
swagen, and related facilities helped us in develop- Cambridge, MA: MIT Press.
ing this robot, which is gratefully acknowledged. Hebert, M., Thorpe, C., & Stentz, A. 共1997兲. Intelligent un-
The SRT also thanks DARPA for organizing this manned ground vehicles: Autonomous navigation re-
great race and the many journalists who provided search at Carnegie Mellon University. Dordrecht: Klu-
wer Academic.
press coverage. Our final gratitude goes to the many Iagnemma, K., & Dubowsky, S. 共2004兲. Mobile robots in
friends we made preparing for and during the event, rough terrain: Estimation, motion planning, and con-
and the many people who helped us and other trol with application to planetary rovers. Springer
teams along the way. Tracts in Advanced Robotics 共STAR兲 Series. Berlin:
Springer.
Julier, S., & Uhlmann, J. 共1997兲. A new extension of the
Kalman filter to nonlinear systems. In International
REFERENCES Symposium on Aerospace/Defense Sensing, Simulate
and Controls, Orlando, FL.
Brooks, C., & Iagnemma, K. 共2005兲. Vibration-based terrain Kelly, A., & Stentz, A. 共1998兲. Rough terrain autonomous
classification for planetary exploration rovers. IEEE mobility, part 1: A theoretical analysis of require-
Transactions on Robotics, 21共6兲, 185–1191. ments. Autonomous Robots, 5, 129–161.
Crisman, J., & Thorpe, C. 共1993兲. SCARF: A color vision Ko, N., & Simmons, R. 共1998兲. The lane-curvature method
system that tracks roads and intersections. IEEE for local obstacle avoidance. In Proceedings of the
Transactions on Robotics and Automation, 9共1兲, 49– IEEE/RSJ International Conference on Intelligent Ro-
58. bots and Systems 共IROS兲, Victoria, Canada. New York:
DARPA. 共2004兲. DARPA Grand Challenge rulebook. Re- IEEE Industrial Electronics Society.
trieved from http://www.darpa.mil/ Pomerleau, D.A. 共1991兲. Rapidly adapting neural networks
grandchallenge05/Rules_8oct04.pdf. for autonomous navigation. In R.P. Lippmann, J.E.
Davies, B., & Lienhart, R. 共2006兲. Using CART to segment Moody, & D.S. Touretzky 共Eds.兲, Advances in neural
road images. In Proceedings SPIE Multimedia Con- information processing systems 3 共pp. 429–435兲. San
tent Analysis, Management, and Retrieval, San Jose, Mateo: Morgan Kaufmann.
CA. Pomerleau, D.A. 共1993兲. Knowledge-based training of arti-
Dickmanns, E. 共2002兲. Vision for ground vehicles: History ficial neural networks for autonomous robot driving.
and prospects. International Journal of Vehicle Au- In J.H. Connell & S. Mahadevan 共Eds.兲, Robot learning
tonomous Systems, 1共1兲, 1–44. 共pp. 19–43兲. New York: Kluwer Academic.
Dickmanns, E., Behringer, R., Dickmanns, D., Hildebrandt, Pomerleau, D., & Jochem, T. 共1996兲. Rapidly adapting ma-
T., Maurer, M., Schiehlen, J., & Thomanek, F. 共1994兲. chine vision for automated vehicle steering. IEEE Ex-
The seeing passenger car VaMoRs-P. In Proceedings pert, 11共2兲, 19–27.
of the International Symposium on Intelligent Ve- Simmons, R., & Apfelbaum, D. 共1998兲. A task description
hicles, Paris, France. language for robot control. In Proceedings of the Con-
Dima, C., & Hebert, M. 共2005兲. Active learning for outdoor ference on Intelligent Robots and Systems 共IROS兲, Vic-
obstacle detection. In S. Thrun, G. Sukhatme, S. toria, CA. New York: IEEE Industrial Electronics
Schaal, & O. Brock 共Eds.兲, Proceedings of the Robotics Society.
Science and Systems Conference, Cambridge, MA. van der Merwe, R. 共2004兲. Sigma-point Kalman filters for
Cambridge, MA: MIT Press. probabilistic inference in dynamic state-space models.
Duda, R., & Hart, P. 共1973兲. Pattern classification and scene Unpublished Ph.D. thesis, Oregon Graduate Institute
analysis. New York: Wiley. School of Science and Engineering, Beaverton,
Ettinger, S., Nechyba, M., Ifju, P., & Waszak, M. 共2003兲. Oregon.
Vision-guided flight stability and control for microair van der Merwe, R., & Wan, E. 共2004兲. Sigma-point Kalman
vehicles. Advanced Robotics, 17, 617–640. filters for integrated navigation. In Proceedings of the
Farrell, J., & Barth, M. 共1999兲. The global positioning sys- 60th Annual Meeting of The Institute of Navigation
tem. New York: McGraw-Hill. 共ION兲, Dayton, OH.
Gat, E. 共1998兲. Three-layered architectures. In D. Korten- Wellington, C., Courville, A., & Stentz, A. 共2005兲. Interact-
kamp, R. Bonasso, & R. Murphy 共Eds.兲, AI-based mo- ing Markov random fields for simultaneous terrain
bile robots: Case studies of successful robot systems modeling and obstacle detection. In S. Thrun, G.
共pp. 195–210兲. Cambridge, MA: MIT Press. Sukhatme, S. Schaal & O. Brock 共Eds.兲, Proceedings of
Gillespie, T. 共1992兲. Fundamentals of vehicle dynamics. the Robotics Science and Systems Conference, Cam-
Warrendale, PA: SAE Publications. bridge, MA. Cambridge, MA: MIT Press.

DARPA 2005 Stanley PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DARPA 2005 Stanley PDF

Uploaded by

Copyright:

Available Formats

Stanley: The Robot that Won

the DARPA Grand Challenge

Sebastian Thrun, Mike Montemerlo,

Sven Strohband, Cedric Dupont,

Gary Bradski, Bob Davies, Scott Ettinger,

Received 13 April 2006; accepted 27 June 2006

1. INTRODUCTION sult of an intense development effort led by Stanford

Journal of Field Robotics DOI 10.1002/rob

Figure 3. A section of the RDDF file from the 2005

CA, approximately 100 miles northeast of Los Ange-

Journal of Field Robotics DOI 10.1002/rob

so no global path planning was required. As a result, 2. VEHICLE

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

3.1.4. Development Support

3.2. Processing Pipeline

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

stacle avoidance. Stanley is equipped with five

Journal of Field Robotics DOI 10.1002/rob

Figure 8. Examples of occupancy maps: 共a兲 An underpass and 共b兲 a road.

Journal of Field Robotics DOI 10.1002/rob

for a cell which was previously observed. Then, one

1. The measurement might be a witness of an

Journal of Field Robotics DOI 10.1002/rob

6. COMPUTER VISION TERRAIN ANALYSIS

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Flat desert roads Mountain roads

Drivable terrain detection rate Discriminative Generative Discriminative Generative

10–20 93.25 90.46 80.43 88.32

False positives, all ranges 3.44 3.70 0.50 2.60

Journal of Field Robotics DOI 10.1002/rob

7.2. Terrain Ruggedness shock experienced by the vehicle due to excitation

Journal of Field Robotics DOI 10.1002/rob

vehicle safety. Using a base trajectory that is

8.1. Path Smoothing argmin 兺 兩yi − xi兩2

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

wheels match the global orientation of the trajectory.

x共t兲 ⬇ x共0兲exp − kt. 共9兲

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

first, the top-seeded H1ghlander robot was the only

Journal of Field Robotics DOI 10.1002/rob

Figure 30. Stanley’s cumulative velocity.

Journal of Field Robotics DOI 10.1002/rob

Figure 31. This diagram shows where the road condi-

stalling problem vanished entirely after Mile 37.85. It

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

Journal of Field Robotics DOI 10.1002/rob

You might also like

8.1. Path Smoothing argmin 兺兩yi − xi兩2