Automotive Electronics

Advanced Vehicle Control and Safety
By Prof. Ravi Kumar Puli

1. The Problem
• Vehicles and highways have greatly improved safety: total fatalities are down approximately 30% over the past 35 years • Even with those improvements, there are still approximately 40,000 fatalities / year in India • People haven’t improved: in 90% of all accidents, the driver is a contributing cause

The Solution
• The Intelligent Vehicle Initiative (IVI) is a program being adopted by automotive companies to use advanced electronics to improve vehicles, with the dominant concern being safety. • This presentation is arranged around a series of advanced functions, such as vehicle detection, that contribute to safer and more intelligent vehicles. For each function, the tutorial discusses a set of possible technologies. • The next set of slides show the “user services” for the IVI advanced vehicle control and safety systems. The following charts show which technology functions support each user service. • Note the synergy: each technical function supports many user services.

– navigation and routing – real-time traffic information – driver comfort and convenience features .IVI User Service categories: • Safety: (directly contributing to vehicle safety). – – – – – – – – rear end collision warning roadway departure warning lane change / merge collision warning intersection collision warning railroad crossing collision warning vision enhancement location-specific warnings collision notification • Safety Impacting: (potential to distract or aid the driver).

More Services • Commercial Vehicle Services: – – – – – – – – – – vehicle stability vehicle diagnostics driver condition monitoring cargo identification automated transactions safety recorder obstacle and pedestrian detection precision docking passenger monitoring passenger information • Transit: .

More Services • Specialty Vehicles: – full automation • Supporting Services: – low friction warning – longitudinal control – lateral control .

Technical functions • There is a set of common vehicle functions that underlie those user services: – – – – – – – – – sensing the position of other vehicles sensing obstacles sensing the position of the lane relative to your own vehicle sensing vehicle position and motion estimating braking performance communication reliability miscellaneous functions sensor-friendly vehicles and roadways • The rest of this section shows how each of these functions supports the various user services .

on the sides. • Sensing has to work in all weather. for backup warning and for lane change / merge warning of overtaking vehicles.2 Sensing Other Vehicles • Other vehicles need to be sensed in front for adaptive cruise control and forward collision warning. and behind. for blind spot and lane change / merge warning. and at a variety of ranges .

another lane. on a curving road.1 Basic Geometry Sensing straight ahead is not sufficient. a forwardlooking sensor needs to have a wide field of view. . or on the shoulder. and sensed vehicle position needs to be combined with road geometry to know whether the lead vehicle is in your lane.2.

this is referred to in the radar literature as “clutter”. which are only concerned with moving vehicles. guard rails. parked cars. . • Adaptive Cruise Control (ACC) systems.2. can reject any stopped object as clutter. • Rear-end collision warning systems need to sense stopped vehicles. overpasses. and so need high-acuity sensing of vehicles and lanes in order to separate targets (other vehicles) from clutter.2 Targets and Clutter • Other objects in the field of view can include roadside signs. etc.

so sensing is almost instantaneous. 3 degrees in bearing. Radar works at the speed of light. snow. • Simple radar is a single spot with no information on bearing angle.3 Radar • Radar is an excellent choice for seeing big metal objects through fog. or use two or more beams and various processing schemes to measure bearing and range • Typical resolution (closest objects that can be distinguished) is 1 meter in range. . or light rain • The currently approved frequency is 77 GHz. More sophisticated versions sweep the beam mechanically.2.

Top image is video of the scene. up is left. with corresponding locations marked. bottom is radar data. D is much further and has a bearing between A and B. . Car A is close and he center of the radar return (the video image does not extend as far to the right as the radar). C is further yet and is barely visible above the roof of A. The radar data is range (horizontal) and bearing angle (vertical. down is right). Brightness indicates strength of return. B is further and left.Radar Data Data from a scanning radar.

lidar. it is blocked by the same kinds of effects that impede human vision: fog.2. Instead of sensing a blob with radar. The laser can be scanned over the scene with mirrors to produce a “range image”. and laser rangefinder are all synonyms. . They refer to measuring distance using the travel time of a laser beam. • Lasers can be focused to very small spots (fractions of a degree). a ladar can make many measurements as it scans. snow. and heavy rain will block the signals.4 Ladar • Ladar. so they have much better resolution than radar. and can measure fine details of shape. • Since ladar is near visible light.

Each picture is 480.Ladar Data • The figures on the next page show data from a high-resolution scanning laser rangefinder. • The lower picture shows range data. and which direction they are facing.000 pixels (points). It is possible to build a computer program that can identify which objects are cars. • The top picture shows the reflectance data: this is the amount of laser energy returned from that point in the scene. • Note the fine details of shape and appearance visible in this data. each corresponding to a separate ladar measurement. and is roughly equivalent to a flash photo. . this can give early warning of which vehicles may be on a collision course. Brightness encodes range: points that are further away are displayed more brightly.

Ladar Data .

2. and work in a most weather conditions. • Sonars are commercially available for blind spot sensors and back-up warning sensors.5 Sonar • Sonar works by measuring the time of flight of sound. so sonar is only useful for detecting objects at ranges of a few meters or less. . • Sonars are inexpensive. The initial mass market application was in Polaroid auto-focus cameras. • Sound travels (relatively) slowly though air and is hard to focus.

Side and Rear Sensors Sonars Radar This bus is equipped with rear and side sensors for blind spot coverage .

. or onset of emergency braking. • Vehicles can also broadcast other information. such as speeds. intent to change lanes. developed by the PATH program at UC Berkeley. This is crucial in decreasing inter-vehicle spacing to increase throughput. • Each vehicle can broadcast its current location. while maintaining safety.2. intervehicle communications can be used to determine relative positions. such as automated highways. Platoons rely on communications 20 times a second to keep all vehicles moving smoothly together. • The picture on the next page shows a “platoon” of tightly-spaced automated vehicles.6 Communications • If all vehicles on a roadway are equipped with ITS features. derived from GPS or other positioning systems. • This kind of scheme is most appropriate for high-end IVI systems.

Platoon .

there is a great deal of variability in people’s driving behavior. Unless that vehicle is fully automated. it is necessary to model the behavior of that driver. that driver’s behavior can be estimated.7 Driver models • Sensing the current location of a nearby vehicle is not all: it would be even better to predict future actions of the vehicle. • As shown in the next slide (and as everyone knows from personal experience). • If a particular vehicle can be observed for some time.2. and used to predict future actions. .

and cut the corners by different amounts when the road curves Left curve Straight Right curve . They have different mean lane positions when the road is straight.Driver Differences The five drivers plotted here each have different behaviors for one important component of driving: average lane position.

3 Sensing Obstacles • Obstacle detection is much more difficult than vehicle detection: obstacles can be small. • Obstacle detection is one of the most challenging tasks for an intelligent vehicle . obstacles need to be detected 100 m ahead. the distance is even longer.g. deer running across the road) • For a passenger car at highway speeds. For trucks. and much harder to see • Obstacles can be stationary or moving (e. nonmetallic.

deer. deer. ladders. pallets.  A non-scientific survey of colleagues indicates that people have hit tire carcasses. etc. mufflers.  A survey commissioned by a company that builds litter-retrieval machines reports 185 million pieces of litter / week.1 Obstacles on the Road  State DOTs report cleaning up construction debris. dogs. tire carcasses.  State highway patrols receive reports of washing machines.3.  Rural states report up to 35% of all rural crashes involve animals. fuel spills. car parts. and so forth. . mostly deer but also including moose and elk as well as farm animals. other home appliances. even a toilet.

2 Sensors • Ladar. which probably come from objects sticking up out of the roadway.3. is useful for seeing small objects • A variant is to use the reflectance channel of a ladar. • Sonar has insufficient range • Advanced radar and stereo vision systems may work . and to look for bright returns. in its high-resolution scanning formats.

3. • There is also some evidence that polarimetric radar will give different returns for wet or snowy roads. polarized radar transmitters and receivers can separate the return from different polarization directions. • Polarimetric radars built at U of Michigan are much better than ordinary radar at separating small obstacles from ground clutter. giving some information on road conditions. this provides cues to distinguish horizontal surfaces and from vertical surfaces.3 Polarimetric radar • Radar can be polarized in the same way as light. . • Just as polarized sunglasses help reduce light reflected from shallow angles (glare).

Intersecting the lines of view from the cameras gives the 3D location of the object.4 Stereo vision • Stereo works by finding the same point in two or more cameras.3. .

The overlays on the original image show detected objects. and generate appropriate driver warnings . • Further processing can examine each blob to separate people from fixed obstructions. Right: “blobs” of pixels at the same distance. Center: depth map from stereo. two pedestrians and a car. brighter is close. used for side-looking sensors on a bus.Stereo Guided Segmentation • Low-resolution stereo for detection and recognition of nearby objects. • Left: Original image.

and a distant dip in the road . The objects on the road are 15 cm tall at a range of 100 m from the camera. grass along the road. Besides the obstacles on the road. the sign.Long-Range Stereo Top: One of three images from a stereo set. Bottom: detected objects in black. the system has found the person.

• Requirements are somewhat different for each application. . to know which lane your vehicle is in as well as which lane preceding vehicles are in.4 Sensing Lane Position • Knowing lane position is necessary for automated guidance and for lane departure warning systems. It is also important for rear-end collision warning.

warn and disable if not operating – accuracy: absolute accuracy of better than 30 cm needed.4. extremely high for automated guidance – availability: must be available nearly 100% for automated guidance. no high-frequency jitter allowed for control applications – range: rear-end warning requires knowing lane position of leading vehicle. lower availability acceptable for warning systems provided a warning is given – weather: should operate in most weather.1 Requirements – reliability: high for warning systems. 100m . to approx.

The magnets can be installed north pole up or down. buried in the center of the road. for most applications.4. map location. as shown here. The magnets are sensed by magnetometers underneath the front and rear bumpers of the vehicle to provide lateral position information. providing a simple binary code that can indicate e. The magnets can be inexpensive magnets. for lateral guidance. .2 Magnetics University College of Berkeley has pioneered the use of permanent magnets. or more expensive but much smaller magnets for bridge decks where drilling large holes would damage the structure.g.

since the magnets are not directly underneath the vehicle’s sensors. Here. they are used to mark the edge of the shoulder. to provide a visual indicator to the snow plow operator. there are also efforts to place magnets in lane marking tape. This would be less expensive to install. . but requires more sophisticated sensing. Besides buried magnets.More Magnets An obvious advantage of magnets is that they are not affected by weather.

with pickup coils mounted in triangular frames at both front and back of the truck. and the signal on the cable can be used to send messages (e.4. The automated trucks at the West rack pavement test site use two cables for redundancy.g. .3 Buried cables The oldest way to perform automated guidance. going back to the 1950’s. “speed limit change”). But cable installation and maintenance are difficult. is to follow a buried cable. Buried cables are all-weather.

) punched with specific hole pattern to provide frequency-selective retroreflection .4. • Frequency-dependent tape properties can provide distance and other information higher f lower f (a) R adar High-Frequency Illumination Low-Frequency Illumination R adar-R eflective S tripe (b) • Conventional lane marking tape (3M Corp.4 Radar reflective surfaces • Collision avoidance radar can be used for lateral control with modified lane-marking tape.

. showing the location of radar targets: yellow X for right lane. Experimenter interface shown at bottom.4.5 Vision Typical vision system for lane tracking. Overlayed on the image is data from other sensors. The detected position of the solid line is shown by the blue dots. red X for current lane. the detected dashed line by dark and light blue dots.

can be used in several ways. and position on a map. . depending on the resolution. at a variety of resolutions. For example: • Several different technologies provide ways of measuring absolute position and motion.5 Sensing vehicle position and motion – coarse position (10s of meters) can be used to predict that a corner is coming up – medium position (meters) can be used to warn a driver to slow down. based on the design speed of the upcoming curve – fine positioning (cm) can be used to tell if the driver is drifting out of their lane through the curve • An estimate of vehicle motion.

5. the different in apparent times gives the difference in time-of-flight of the signals from the satellites. It works by broadcasting very accurate time signals from a constellation of orbiting satellites. .2 GPS • The Global Positioning System is a satellite-based navigation system. originally developed by the US military. A ground-based receiver can compare the times from several satellites. Simple geometry gives the location of the groundbased unit and an accurate time. and therefore the difference in distance to each satellite.

in order to reduce the accuracy of the system for potential enemies – Local atmospheric effects refract the signals by varying amounts • The result is that raw GPS has an accuracy of only 10’s of meters .More GPS • This simple picture is distorted by two phenomena – The US government deliberately introduces distortions into the civilian version of the signal.

It continually compares its known position with the GPS reported position. • The accuracy of DGPS is on the order of a few meters. a base station has a GPS receiver at a known location. The base station broadcasts the correction terms to mobile units. The difference is the error caused by selective availability and atmospheric distortion.Differential GPS • In Differential GPS. the mobile units can reduce their errors. By applying the correction. .

can achieve accuracies of 2 cm or better. • A good carrier-phase system. By counting waveforms.Carrier Phase GPS • In carrier phase systems. they can synchronize their positions with each other to a fraction of a wavelength. . with good conditions. the base station and the mobile units watch both the broadcast time code. and the actual waveforms of the carriers.

• In mountainous terrain. and can become confused even going under a large road sign. open terrain. this is not a problem. . or in urban canyons. • Carrier-phase GPS is very sensitive to losing lock on the satellites.GPS Difficulties • GPS requires a clear view of at least 4 satellites. or in flat. GPS signals can be blocked or (worse) can reflect from tall objects and cause mistaken readings. For aircraft applications.

so there is hope for the future.Bottom line on GPS • GPS is very useful for many applications. • Research continues on filling in gaps in GPS coverage. . • It is not yet 100% reliable. so is not ready for control applications. and integrating GPS with other sensors.

This is sufficient for in-vehicle navigation systems. such as design speed of curves. This would aid e. • The first generation of digital maps were produced from paper maps. so the accuracies will improve. and therefore are no more accurate than the paper products. grade of slopes. until more sophisticated uses arise. in warning drivers of excessive speed when entering a curve. . • To be completely useful. etc. there is little market demand for high accuracy. maps should have additional information.g. Typical quoted accuracies are 14 meters. • The next generations of maps will be produced directly from aerial photos and verified by driving selected routes with accurate GPS.Maps • Accurate position is not useful unless combined with accurate maps.

• Inertial measurement is good for sensing braking forces or for comparing wheel speed with ground speed and calculating slip during braking. • Since position is doubly-integrated. • Inertial measurement is useful to fill in short-term gaps in GPS or other measurements. • High-precision inertial navigation is not yet affordable for the automotive market. small errors in acceleration build up rapidly.5. . then integrates acceleration to give velocity and again to give position.3 Inertial • Inertial sensing measures acceleration.

such as bridges or buildings. Road Rally enthusiasts can calibrate their odometry to 0. incorrect estimates of wheel circumference due to changes in tire inflation. etc.1%. . • Standard compasses are affected by nearby metallic objects. this is not practical for most vehicles. • Odometry uses wheel encoders to measure distance traveled.4 Other sensors • “Dead reckoning” uses estimates of distance travelled and direction of travel. It is susceptible to errors due to tire slip.5.

where it is important to measure the speed of farm equipment even with significant tire slip.1% • Doppler radar is used in precision agriculture applications.More Sensors • Image correlators directly measure vehicle motion by watching the ground move by under the vehicle. . These systems are accurate to better than 0.

• To set safe following distance. the braking capability of the lead car. ideally the system should know its own braking capability. and the reaction time of the automated system or of the • Braking performance of vehicles on identical roadways can vary by a factor of 4 .6 Predicting Braking Performance • Braking performance is key to setting many parameters in automated control and in driver warning systems.

typical reaction times range from 100 milliseconds for a fast computer-controlled sensor and brake actuator.1 Basic formulas • The basic formulas for the time and distance required to bring a car to a stop are • Time = reaction time + speed / deceleration • Distance = speed * reaction time + ½ speed2 / deceleration • Typical highway speeds are approximately 30 meters / second. or braking performance. . The dominant unknown factor is deceleration. to up to 2 seconds for a human driver.6.

As the slip increases. the tires begin to slip. This means that gently tapping the brakes is not enough to tell surface type. and therefore it is difficult to predict maximum braking performance without attempting hard braking. but reach different peaks. which results in deceleration force. slip curve. After that point. and the braking force decreases. the wheels begin to lock and skid. Dry surface Force (g) Wet surface Slip (%) .2 Wheel speeds and slip Typical force vs. As the brakes are applied. the force increases to some maximum.6. Note that the curves for wet and dry pavement start off very close to each other.

. water. tuned to detect differences between ice. looking for incipient slip on the driving wheels • None of the methods is completely successful yet.3 Surface condition sensing • Several methods have been attempted to sense current surface conditions: – infrared spectrophotometers.6. and dry pavement – microphones in the wheel wells listening for water splash sounds – roadside mini-weather stations with sensors built into the pavement – careful instrumentation of all wheels of a car.

Several characteristics of automobiles are much different than.g. In the automotive environment. . aircraft:  Cost sensitivity: Usual practices that involve triplex redundancy of critical components may not be affordable in automobiles. preventive maintenance schedules call for replacing electronics before the end of their design life.  Equipment used until end-of-life: In most safetycritical tasks.7 Reliability  Reliability engineering in intelligent vehicles is difficult.. many components are never replaced until they fail. e.

That same probability would cause a failure once every 4. Even though the risk to a passenger might be the same in both cases. one that occurs once in 109 hours. due to the much higher number of vehicles. the public perception of risk could be much higher for cars.More Reliability  Operation in uncontrolled environment: Vehicles operate in harsh environments. .5 days in the US automotive fleet.  Very large scale of deployment: An extremely improbable event. would cause one failure in 73 years in the US commercial air fleet. with relatively unskilled and untrained operators.

the output of all three can be compared. • Triplex redundancy has three copies. For instance. and the majority wins. If a failure is detected in one system. this provides automatic detection and correction of single errors. computer). for computers. some steering authority is available by differentially applying the right or left brakes. if a steering actuator fails on an automated vehicle. . the other can be used.1 Redundancy • Duplex redundancy refers to having two copies of a subsystem (e.g. • Heterogeneous redundancy refers to doing the same function with different means.7.

. the California PATH approach of Platoons is designed to mitigate the effects of an (unlikely) crash by having vehicles so closely spaced that any collision would be at a small relative velocity. In automated highways.7.2 System-level solutions • System level solutions build safety into the system by considering the entire system.

are being developed for other purposes (e. handling). • As drivers become more accustomed to electronics in vehicles. such as electronic controls.g. but will be useful for intelligent vehicles. • Some. prices will fall. consumer acceptance will increase. . and the pace of adoption of new technology could accelerate.8 Emerging technologies • A number of other technologies are being developed that will support intelligent vehicles.

e. i.1 Control • Current IVI applications are focused on driver assistance rather than vehicle control. not automated control. nevertheless. The underlying hardware may need to be modified for optimal automatic control.8. partial and full automation will eventually be important. predictable response for humans. current steering system geometry is designed for “good handling”. . For example. • A wide variety of standard and advanced controls techniques are being applied to road vehicles • Vehicles to date have been designed for human control.

Truck controllers need to be much more adaptable than light vehicle controllers.  Equipment failure: Special controllers need to be designed to cope with tire blowout or loss of power brakes or power steering. and the distribution of the load.Difficulties • Automated control is especially difficult in some situations:  Emergency maneuvers: Control systems optimized for smooth performance at cruise will not work for abrupt maneuvers in emergency situations. vary much more for a heavy truck than for a passenger car. .  Heavy vehicles: The load.

More Difficulties
 Low speeds: Engine and transmission dynamics are hardest to model at slow speeds. Applications such as automated snow plows or semi-automated busses will require careful throttle control design.  Low-friction surfaces: As addressed above, it is difficult to predict the effective coefficient of friction on a particular road surface. This affects not only braking performance but also the design of throttle and steering controllers.

8.2 Actuation
• Full or partial automation will require actuators, i.e. computer-controlled motors that can move the throttle, brake, and steering. • The state of the art is rapidly improving: vehicles are available on the market with electronic fuel injection, electronic power steering, and electronic power brakes, all driven by performance and weight improvements for manually-driven cars. This makes it much easier to add computer control. • Special-purpose actuators will still be needed in some applications, such as quick-response throttles for closelyspaced platoons of cars.

8.3 Driver condition
• It is important to assess driver alertness, both in a drowsy driver warning system, and in an automated system that is preparing to return control to the driver. • Alertness can be sensed indirectly, by watching lane-keeping performance; or directly, by watching for eye blink rate and closure.

note two bright dots for reflections .Perclose Measuring percentage of time eyes are closed. Subtracting the images will create a blank image (if the eyes are closed) or an image with two bright spots (if the eyes are open). This system illuminates the face with two IR wavelengths. one of which reflects from the retina. Top left: image with retinal reflections Top right: no retinal reflections Bottom left: difference image.

g. . 20 times / second. traffic status ahead. • Dedicated Short Range Communication (DSRC) is being developed for warning of local conditions. on managing both inter. sharp curves. or stopped traffic out of sight around a bend. • In the platoon version of full automation.and intra-platoon communication. This creates interesting research questions on creating local nets. and on reliable communications in urban canyons and other difficult environments.8.4 Communications • Infrastructure-based ITS projects are building roadway-to-vehicle communications for traffic and routing information. intent to change lanes. changes in speed limit. vehicles need to communicate with low latency. braking. e. • Vehicle-vehicle communications will be increasingly important for collision warning systems. such as ice. etc. The lead vehicle can communicate speed.

so radar-equipped trailing vehicles can get information as well as range – powering a transponder with radar energy. • Schemes currently under research include: – modulating the radar reflectivity of a surface. again to communicate to a following radar-equipped vehicle – modulating LED brake lights so trailing vehicles equipped with detectors tuned to that particular wavelength can pick up information . using a variety of bands.Communications Technologies • Most communications schemes rely on radio.

9 Sensor-friendly roadways and vehicles • On-board sensing would work better if the environment were designed for sensing. • Current roadways have many objects that cause radar “clutter” (returns from objects that are not of interest). bridge overpasses . • Current roadways have significant variability (Bott’s dots. painted lines. roadside signs. thermoplastic stripes. such as guard rails. etc).

• Sensor friendly systems will improve path prediction by enhancing lane visibility. • They will also improve obstacle detection by reducing clutter from off-road objects and increasing returns from other vehicles. .9.2 Path prediction • “Path Prediction” refers to estimating where the vehicle’s current lane goes. so an obstacle detection system knows where to look for stopped cars and other obstructions.

9. or filters that absorb only a narrow frequency band. but would be marked in the radar signal as known fixed objects  Mapped: The locations and signatures of fixed objects could be stored in a map. . and provided to individual vehicles. They would then still appear in a radar return.1 Dealing with clutter  Clutter can be:  Moved: Sign posts could be placed farther from the travel lanes. could be applied to large objects.  Masked: Radar Absorbing Material (RAM) could be applied to objects such as bridge abutments  Marked: Polarizing reflectors.

Sensor-Friendly Features • Besides clutter suppression. sensor-friendly systems can improve visibility: – Lane markings can be improved with pigments that reflect radar or near-visible wavelengths – Vehicle visibility can be improved with radar reflectors. either fixed or modulated for communications .

[GHz] . radar-based vehicle tracking is difficult.Microstrip patch retroreflector antenna • Without a stable aiming point. Angle • Angle-invariant return provides aim-point stability. vehicle marker. Freq. Compact form factor is easily attached to vehicles. wideband. • Wide bandwidth permits good range resolution. Lead vehicle appears to wander • OSU patch retro-reflector provides a distinctive.

g. lane tracking aids both lane departure and rear-end warnings. the remaining obstacles to deployment are legal and institutional. and can take advantage of better markings.10 Comments and conclusions • Many of these technologies work best in combination: e. • Many of these work best with some infrastructure assistance: e. lane departure systems need at least good road delineation. . the technology is approaching readiness. • In many cases.g.