Vision-Based Lane/Road Detection Methods

Artificial Vision in Road Vehicles (Bertozzi, et al, 2002) http://www.dx.doi.org/10.1109/JPROC.2002.801444 - General Advice 1. Limiting the search for road/lane markers to a specific region of the image speeds things up. This can be done using knowledge of where they were in the last image, or general assumptions of where they usually should be. 2. Assuming continuity and parallelism of lane markers along with smoothly varying lane widths can help speed things up. Once you find a piece of a marker, you can limit the search to nearly-parallel copies of it (extended along it, offset by a lane width + some amount to account for slowly varying widths, or offset by a line spacing width if you are looking for double lines). 3. Reconstructing road geometry can be made much simpler by assumptions on its shape. a. I think that this means assuming it is a flat plane, typically. b. Clothoids have been used. They are a parametric plot which blends a straight segment into a curved segment using integrals of sin(x2) and cos(x2). See http://en.wikipedia.org/wiki/Clothoid for a quick

explanation. The plot looks like: 4. If you have a specific camera calibration and assume a flat road without bumps, then the mapping between image pixels and world coordinates becomes much easier. - Key Difficulties 1. Dealing with shadows. 2. Occlusion of the road and lane markers due to cars and other obstacles. - Example Algorithms 1. ALVINN [9]: Uses neural net training with a large database of training images to automatically follow roads without any explicit feature recognition. 2. SCARF [10] and POSTECH PVR II [11]: Both use color cameras and the assumption that the road will be homogeneously colored to extract the road region from images even on completely unstructured roads.

5. the algorithm can be sped up significantly. It also mentions the Generalized Hough Transform which is supposed to be able to find any shape which can be succinctly parameterized (circles.cs.” Uses parabolas to model the lane markings. MOSFET [13]: color-based technique – “uses a color segmentation algorithm that maximizes the contrast between lane markings and the road. (We could probably find MATLAB code for Hough transforms somewhere with a little looking. 6.org/wiki/Hough_transform gives a good brief introduction. How do you take the perspective out of the image? . It extracts edges and then matches edge points to various patterns of lane markers which are generated to model typical roads. PVR II [11]: analyzes two parallel stripes of the image at a time looking for edges using Gaussian masks and zero crossings. A cool web page with an applet showing how the transform works is http://www. ellipses.tu-bs. This still gives enough time to maneuver on highways and ramps. 4. The result is compared to a typical model and the quality of the fit indicates a confidence as to whether or not it is a lane marking. o Sounds cool. o Hough transform – basically a method where candidate points are all allowed to vote for any line on which they could belong. The system always limits the search to a given distance ahead of the vehicle and removes the perspective effect from that portion of the image so that the lane markers in the image can be compared directly to standard models in which they are parallel (not slanted and/or leaning towards each other). Toyota Central R&D Labs [12]: pattern-based technique which was developed to handle situations where visibility of the lane markers is poor. RALPH [15] uses radar-based obstacle detection to limit the portion of the image to be analyzed for lane markings.). The image portions where the lane markings are obscured can be ignored (for finding lanes) this way. 7.html .de/rob/lehre/bv/HNF. A wikipedia article located here: http://en. The lines with the most number of votes at the end are considered to be the best fits to the points. It explains that the candidate points are usually identified via edge detection.3. Does the fitting with a simplified Hough transform. o This sounds promising – it could still find the road even if lane markers aren’t present in all segments and maybe even in off-road stretches. etc. The pattern which best matches the edge points found in the image is selected as the most likely model of the lane markers.wikipedia. If you use gradient information from this process to limit the directions in which candidate lines can point. They also give a link to the Java source code.) LAKE and SAVE autonomous vehicles [14] only analyze the image portion corresponding to 12 m ahead of the vehicle where it is usually safe to assume that the lane markers will not be obscured by other vehicles.

and computing position and heading within the lane. 13. General Dynamics Robotics Systems and Ohio State University method [16]: uses a gray-level histogram to segment and classify objects as road. 17. Universität der Bundeswehr [20]. The window is selected using statistical methods based on the current state and previous windows. Laboratoire Central des Ponts-et-Chaussees de Strasbourg [17]: Assumes that there should always be a chromatic contrast between road and off-road areas/obstacles in at least one color channel.” o What is chromatic saturation? 10. and Robert Bosch GmbH [22]: Base their road models on clothoids. The lane model is updated continuously and the overall system allows the process to be very robust to noise and poor visibility. planar road assumption to map the image into another domain representing a bird’s eye view of the road (takes out the perspective like the RALPH system). 12. .” 18. Daimler-Benz [21]. Ferraris” [31]: Uses results for lane markings from previous frames to validate the results for the current one. 9. Using a low-order polynomial curve allows dashed lines to be represented. 15. University of Michigan [26]: proposed using concentric circles as a model for lane markings since that is actually how they are laid out and it is a better fit than polynomial approximations. “Uses the concept of chromatic saturation to separate the components. Early version of Ohio State’s algorithm uses polynomials to represent the lane markings [24]. To narrow in on candidate lane markings. ARGO [34]: restricts the search for lane markings to “the nearest neighborhood of the markings previously detected. describing the lane markings. they put the bright regions in a vector list and then use parameters such as convergence at infinity and lane width to decide which ones to keep. lane marking candidates or obstacle candidates. 14. 11. GOLD system [32]: exploits the flat.” Simplifies representation of the road. Transportation College of Jilin University of Technology [29]: uses a linear lane model and represents road markers as sequences of straight lines. Robert Bosch GmbH [19]: uses a model of both the road and the vehicle dynamics to predict where lane marking should be found.8. 16. “In a clothoid the curvature depends linearly on the curvilinear reference. CyCab electric vehicle [28]: “uses an edge linking process based on Contour Chains and Causal Neighborhood Windows” to piece edges together. Then the longest chains with angles close to 45° and 135° are found and picked as the most probably candidates for left and right lanes. LASMEA [18.33]: uses a dynamic window of interest in which lane markings are sought. Istituto Elettrotecnico Nazionale “G.

A method which works well for one set of objectives may not be suited for another. textures. be included in the results. This first processing attempt is then refined using additional information about general lane marking characteristics and/or information from the road model and past frames. bright patches. It also shows that the road surface may not be uniform. like parabolas. Common Simplifying Assumptions: (These make the task much easier. Another problem shown is the drastic change in road appearance that can occur due to shadows from trees and overpasses. Generalized framework for lane detection methods: a.) a. 2. Orientation based enhancement/attenuation [23]. This post processing is where most of the difficulty and opportunity for innovation lies. The road/vehicle model and information from past frames can assist in this step by helping to narrow in on the region of interest for processing rather than just plowing through the whole image. b. This first step won’t single out the lane markers entirely.1109/TITS. obviously. The road/lane width is locally constant.) 4. System. hyperbolas. or even clothoids [16]. Dynamic programming [28]. This step can be made much more robust when combined with the road model and tracking information temporally. or more complex. which was shown to remove outliers more effectively than the Hough transform. It is important to identify the objectives for a lane tracking system and develop relevant performance metrics. and Evaluation (McCall and Trivedi. These may be edges. A road model is the first component (a vehicle model can be included. especially in regions where transitions from asphalt to cement occur.22]. Figure 2(a)-(f) illustrates that lane markers can vary widely in their style and visibility. Methods which have been used include: i.21]. The Hough transform [26.Video-Based Lane Estimation and Tracking for Driver Assistance: Survey. but further processing is needed to isolate them. Cue scheduling [29]. They should.dx. 3. or something else. too.869595 . You try and glean just the lane markers from the initial feature extraction rejecting everything else. but must be kept in mind as potential causes for failure when they are violated. Models used have been as simple as straight line segments.2006. b. 2006) http://www. if data is available and that is of interest). The lane detection system will build around this model. splines [17. There is typically a first shot processing method which extracts some kind of features from the raw images. . The road/lane texture is constant. iv.org/10.doi. (I’m not sure what that is. You need the right performance metrics to tell you whether or not you are getting the job done. ii.General Advice/Observations 1. iii. c. motion vectors. 27]. like in construction zones. piecewise constant curvatures [16.

Springrobot [26]: Uses an adaptive randomized Hough transform for processing detected edges. 5.Example Algorithms 1. Huge comparison table on next page. LOIS [34]: Uses edge magnitude and orientation with a maximum a posterior estimator to find lane position. d. GOLD [20]: Uses an inverse-perspective road image and combines laneposition tracking with obstacle detection. 2. 5. Methods used include: a. Recovering 3D Road Shape [32]: Uses the assumption of constant lane width to estimate changes in elevation of the road. Lane tracking techniques: turns feature extraction and position tracking into a closed-loop feedback system. 16] b.) Got results with a standard deviation of 13 cm. Road markings follow strict rules for appearance or placement. Bertozzi and Broggi [20]: Assumed road markings form parallel lines in an inverse-perspective warped image. – Planar assumption causes errors in estimated road curvature on hills.c. The road is a flat plane or follows a strict model for elevation change. Kalman filtering [12. Particle filtering [29. 4. (Not sure what that means. Nonlinear systems [31] . Obviously. 30] c. 3. . The tracked lane position (combined with information from the vehicle model?) defines an a priori estimate of the location and orientation of the extracted features. 6. this will fail in locations where the lane width isn’t constant.

.

the angles of the maximum and minimum responses can be found by: d. 3. shadows.Their Algorithm – VioLET (VIdeO-based Lane Estimation and Tracking) 1. They wanted accuracy for at least 30-40 m ahead of the vehicle.. a. the second-derivative response at any angle can be calculated via [39]: c. . They were trying to come up with something that would be unprecedented in its robustness to varying road surface. lane marker types. They create a basis set of responses from three separable 2-D Gaussians: b. They used a filter kernel roughly three times the expected width of the lane markers and applied it in the inverse-perspective warped image so that a kernel of constant size could be used everywhere. 2. which they say can be attained with a simple parabolic road model (second order polynomial). Basic feature extraction: Steerable Filters a. Setting the derivative equal to zero. and lighting. Then.

e. their description of their performance metrics is weak. it sounds like their technique is based upon the following: i. very poorly described. then fit the parabola to those points. and the minimum response should be low. There is supposed to be a high variance in the direction of the lane markings and a lower variance in other directions since points from the lane markings should be spread out more in the direction of the road. It looks like they pick a template which excludes the lane markings and depends on their being some recognizable dark patch down the center of the lane or something. So. iii. Basically. They also use a road center template and squared intensity error measure to try and come up with the center of the road. For circular markers. For lane markers. Results: Again. they get a set of points of the estimate of the center of the road at certain distance increments ahead of the vehicle. vehicles changed lanes directly in front of the test bed. The specular highlights and lines of the vehicles caused false positives in the feature extraction. . the maximum response should be in the direction of the lane marker (assumed known from the previous frame?). They throw out points which aren’t near the estimated lane markings according to their road model and prior knowledge. g.) 5. ii. iv. the maximum and minimum response should be roughly equal. but the results look pretty good overall. Quite disappointing. you can pick those out by looking for areas where there is a small difference between the max and min. How they get the points used in this estimation is very. In most of the situations where the tracking was lost. A Hough transform also helps find markings for solid/striped (non-circular reflector) lane markings. f. This lets you attenuate the response for objects/edges which are at the wrong orientation. An important note they make is the following: a. You can also get results which are tuned to a specific angle (like the assumed or last known lane marker angle) by formula (6). 4. (All this is so remarkably unclear in their descriptions. Road curvature estimation. But. a. Somehow they take first and second moments of the candidate points.

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.