This action might not be possible to undo. Are you sure you want to continue?

A

**Stairway Detection Algorithm based on
**

Vision for

Yang Cong

UGV

Stair Climbing

Ji Liu

and Yandong Tang

1

Xiaomao Li

**'Robotic Laboratory, Shenyang Institute ofAutomation, Chinese Academy of Science 110016, China
**

2Graduate School of the Chinese Academy of Science, Beijing 10038, China

images and reconstructing the 3D environment through line matching. In order to make the UGV climb the stairways automatically, two parameters are very important and play key roles: the angle between the vehicle's heading direction and the stair's horizontal direction, and the position of the vehicle on the stairs. The angle is used to guide the vehicle to the right direction, and the position to avoid the vehicle colliding to the walls. For evaluating these two offset parameters accurately a stairway detection algorithm based on vision for UGV stair climbing is presented in this paper. For validating our algorithm, a visual servo system is introduced to control the UGV stair climbing by these two parameters and to guide the UGV to go forward along the lines bisecting the stairs ( to see Fig. 1). A vision algorithm for the visual servo system is proposed to extract a set of long, horizontal lines robustly in any conditions and to estimate the parameters by the set of lines. In practice, the UGV will be confronted with many different conditions, such as the stairs of different styles and materials, different application environments including indoors and outdoors, illumination changes, shadows, etc. In order to deal with these situations

Abstract-In the paper, we present a vision based algorithm used to guide the Unmanned Ground Vehicles (UGV) for autonomous stairways climbing and implement it on UGV successfully. The reliability of guiding UGV to climb stairs requires evaluating two offset parameters: the position of vehicle on stairs and the orientation angle to stairs. The intention of our algorithm is to estimate these two parameters through extracting the stair edges robustly. To achieve this goal, we apply the Gabor filter to eliminate the influence of the illumination and keep edges, and propose a fast method to remove small lines. Finally we link stair edges, and estimate the offset parameters used to steer the vehicle by RANSAC algorithm. Experiments on various stairways including indoor and outdoor are given in various light conditions. The results validate our algorithm. Index Terms- computer vision, UGV, stair climbing, Gabor filter, RANSAC

I. INTRODUCTION

A s the mobile robot technology making remarkable progress, it has become the preferred solution for many tasks. In order to navigate autonomously in unknown environments, mobile robots, such as UGV, must be equipped with the followings: sensors for measuring the terrain, software for interpreting the data, path planning to decide a safe route of action, and control to oversee the plan. The ability to detect and climb the stairways automatically is of a great challenge to UGV. For detecting the stairways based on vision automatically, Stephen Se[1] used monocular to detection curbs and stairways for the partially sighted human; in [2] JS Gutmann introduced stereo vision to guided the human robot to climb up and down the stairways; xiaoyu[3] and R.Turchetto[4] proposed methods to detect and locate curbs and stairways using stereo vision by segmentation

Manuscript received October 1, 2007. This work was supported in part by Shenyang institute of Automation (SIA) of Chinese Academic of Science (CAS). All authors are with Robotics Laboratory, Shenyang Institute of Automation, Chinese Academy of Sciences Shenyang 110016, China (congyang8 1 0gmail.com)

i 1;.i 11in uveraii Ulisign U1 11 n U V, L11nC LWU L1raCK aSIuw Ln1 lUUUL 1 USCU for driving and turning and the other two tracks in front of the robot is used for climbing.

1806

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. Downloaded on March 20,2010 at 06:18:51 EDT from IEEE Xplore. Restrictions apply.

The paper is organized as followed: in section 2. With our proposed fast algorithm the most of the small.2010 at 06:18:51 EDT from IEEE Xplore. In section 4. and under different lighting conditions. different materials. Our algorithm includes the following steps: Firstly. Y) exp[. the two offset parameters are estimated by the RANSAC method. The experimental results validate our algorithm. . and to extract the most similar ones among these lines which should be the stair edges. Finally. a robust edge linking algorithm is used to extract the set of parallel lines. the results of our experiments are shown. horizontal edge (which should be the stair edges) according to some basic constraints. a fast method is proposed to filter out small and vertical lines after the edges are detected. When the vehicle arrives at the linking between two staircases. the generic edge detector is applied to the filtered image. It can improve the efficiency and accuracy of the linking algorithm in the next step.L processing. The appropriate selection of the scale and the frequency of the Gabor filter to maintain the level of the staircase edges will be a precondition for the following .2 The whole flow of our algorithms 1807 Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. the projection of stair edges on the 2D image plane are nearly a group of parallel lines.(. Downloaded on March 20. The Gabor filter is applied to eliminate the influence of variety illumination and shadows effects on an image. Equation 1 shows the Gabor function: 1 I1X2 2 ga . In section 3. And the UGV can be steered by the two parameters to climb the stairs. vertical edges are removed. Furthermore. II. Then. which make it possible to control the UGV to creep the stairs automatically. (x. horizontal lines in an image.2 robustly. The only feature of stairways is that their profile includes a set of parallel lines in 3D space. This step improves the efficiency and the accuracy of the algorithm in the stair edge detecting and linking processing. Finally. The intention of our vision algorithm is to detect long. it will go on climb the stair based on vision. Secondly. If the vehicle detects the stair. y) is the point position in the horizontal and vertical i Fig. A. it restarts the Stair Detection Program to detect the stairs automatically. We end this paper with conclusions and discussions in section 5..9]. Gabor filter is applied to eliminate the influence of the illuminate. Thirdly. Fig. These extracted lines can be used to estimate the current offset angle and the current offset position of the UGV on the stairways. THE WHOLE FLOW OF OUR ALGORITHM Stairways are of different styles. we estimate the two parameters by the RANSAC and describe the strategy of steering the UGV. we estimate the two offset parameters based on the theory of RANSAC algorithm [10][11][12]. The Gabor filter has also been shown to be benefit to the receptive fields of simple cells in the striate cortex and hence their relationship to models for the early vision of manuals [7. The Gaborfilter The Gabor filter can eliminate the influence of the illumination to an image efficiently and reserve the multi-scale and multi-direction information. the Gabor function is used to filter the image in order to eliminate the influence of the illuminate and keep the stair edges. we describe the whole flow of our algorithm. Restrictions apply.8.2 illustrates the whole flow of our algorithm. When the UGV is creeping on the stairways. the remainder adjacent edges are linked into a long.+ y 2 )] exp[jo (x + y)] (1) N y v where (x.

a is the standard deviation of the x direction and y direction respectively. it may be a line preserved. RANSAC or other edge link algorithms directly on the output binary edge images for detecting the stair lines.5). If a line does not get through. and so we elim inate all of the edge pixels in the template.l) If sum > 1. j)xEdge(k. ax. There would be no line. we propose a new fast 'The synthesize image 'The result of the eliminate small lines Fig. and there are still a lot of small lines in the image.6 The result of the eliminate small lines on synthesize image Fig. Fig. we use the multi-channel filtering approach with a set of predetermined filter parameters. and save much time in edge linking. vertical lines which are not wanted.5 shows their results respectively. Fig.3 direction. we assume that all the stair edges are horizontal or nearly horizontal (the slope is less than 0. The method to compute the sum of the neighbor points of the edge points in the template is as follows: sum = edge point adj i_j E adj(i.7 The result of eliminating small lines on real image (a) The edge of no Gabor filtered (b) The edge of Gabor filtered Fig. the width of the template as W=2*r+1 and the image point as img (i. which direction is vertical and the band width equal to 1. The main idea of our template operation is that if a line gets through or partial get through the template boundaries. We define the radius of the template as r. For our current work. j} . The set of neighbor points of the Edge{ i. j). And these parameters are used in all of our experiments. In order to remove most of small. Fig. (y.5 The results of the edge detector 1808 Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. The result of the method on synthesize image is illustrated in Fig. . In our experiments. and Fig. we experimentally apply a series of Gabor filters to stair images in order to choose the filter which has the largest response in vertical direction and the lowest response in horizontal direction as the most suitable one. and o0 is the space frequency of the complex plane-wave. and o to maintain the level of the stair edges. it would be eliminated. it will be conducive to further processing. There would be a line pass through the template. So in order to get the appropriate parameters of Gabor filters. And Fig. 4 The result of Gabor filter B. j} where i = 1 or i = W or j = 1 or j = W . Downloaded on March 20.3 shows the original image. the boundary point of the template as Edge{i. Else if sum < 1.6. j} are denoted as adj{fi. By selecting the appropriate parameters of ax.7 shows the results of this method on the stair binary image.4 shows the result of the image convolved by the Gabor filter. Restrictions apply. Through experiments.2010 at 06:18:51 EDT from IEEE Xplore. This method can remove all of the isolate points and most of the short discontinuous lines. Eliminate small Lines The Gabor filtered image is segmented by the generic edge detector. It is inefficient if we apply Hough transform. the parameters are determined.

. some basic constraints are set to find the stair edges in the image: 1) Stairway edges are always long and straight lines. S'. ESTIMATE THE PARAMETERS AND STEER THE VEHICLE A. 2) Stairways edges are parallel to each other. Let us define some parameters about the ith line segment or region in the binary edge image: ai: the area of the region i. and Syis the product of all its x For i=l: total number of the merged lines If the length of the line i is less than width/4. Fig. and some line segments which belong to the stairways may be only a part of the stair edges. parameters about line ith. Else If x.2010 at 06:18:51 EDT from IEEE Xplore. The offset parameters. Edge Linking After eliminating the small lines. where j-i x.. can Fig. 3) The direction ofthe stairway is nearly horizontal: so we assume the slope the line less than a threshold. is the sum of the square of all its x coordinates. If I k' - yI: the y coordinate of its left endpoint. N=N+ 1. Continue. where N is the total number of the lines. the binary image. x: the x coordinate of its right endpoint.s s S>S1 area y I I (2) (3) Where S' the sum of all its yiscy sum of all its x coordinates. If k1' does not close to zero Reject the line ijth. Delete. The equation of the line segment is y = k'x + b' . The ith region is fitted as a line. A series of rules are designed to link the edges. coordinates and y coordinates. III. > thresholdx Xl IJ. Restrictions apply.N. The remaining linked lines are most likely the staircase edges. Else Insert line ijth into L. threshold] equals to 0. S' is the sum of all its y coordinates. This is caused by the influence of different illumination. and the details of the linking processing are listed below: 1. S'y is the sum of the square of all its y sum of the coordinates.> width/2 or xl <width/2 Delete. Then the parameters k' and b' are computed by equations (2) and (3): Else If or y' -Yr y -yY I Jr Else Continue. And in the experiments. b': the intercept of its fitted line. thresholdx and thresholdy are both 5. Estimate the parameters Not all the detected parallel line segments from the image belong to the stairways.Si Si x SXxx . and denote the set of lines segments as L: L. bWj and the other Merge the line ith and line jth into line ijth. .{1. Continue. We sort the line segments by their area ai in descend order. thresholdy > thresholdy kj.: the x coordinate of its left endpoint. the offset angle AO and offset position Ax. Recompute the kiarea iS' -_'S'ZS k area = =area'S> . > thresholdx -X > k: the slope of its fitted line.Sx'x x I lxI.4 C. Else If xl _x or kj II> thresholdl yx: the y coordinate of its right endpoint.5.8 shows the set of extracted parallel lines from the 2D image.Ill. Continue. 2. The merging process is: Fo-r i=l :X For j=1A4. Downloaded on March 20.2.8 The result of extracting parallel stair edges 1809 Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS.

if the vehicle is not too close to the sides of the stairways and the heading orientation angle of the vehicle is almost along the stairways. In the indoor condition. Fig.pj. 10 shows the results under various conditions.9 shows the process of estimating the offset parameters. In this paper. If both of AO and Ax are less than the preset thresholds. It is robust in the sense of good tolerance to outliers in the experimental data and is capable of interpreting and smoothing data containing a significant percentage of gross errors.N as the total number of the detectedparallel lines. bj} as the line parameters which is fitted by the center points of Pj. registration or detection of geometric primitives[ I1] [ 12]. Through experiments we find out that the offset angle AO and offset position Ax are change slightly in the two continuous frames. M equals to 2. we use RANSAC to estimate the offset parameters. It was first introduced by Fischler and Bolles in 1981 [10].pN} points. The RANSAC algorithm has been applied to a wide range of model parameters estimation problems in computer vision. yj} as the center point of each line wherej=tl . all the stairways are recognized regardless of light conditions.x (4) These two estimated parameters will be used to steer the vehicle. we will change the direction of the vehicle to make it along the right orientation. Downloaded on March 20.. where yj =kj*xj+bj. 1810 Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS.2010 at 06:18:51 EDT from IEEE Xplore. In the outdoor condition. Retain Oj as the best if h(Oj) is the minimum thus far.25zT and 20 pixels. 11 illustrates a steering processing when the UGV is mounted on the side of the stair or inclined on the stair. The white line computed by the RANSAC algorithm shows the estimated center line of the stairways. * h(®j) as the accumulate of the distance of point i from line 64 * S as the largest number of circulation Fig. Gather the inliers to the best estimate. IV. We fit the center points as a line. Fig. The blue line shows the heading direction of the vehicle. When the UGV get to the landing plane.. but if either AO or Ax is great than the threshold..-.S Select a minimal subset {pj. * 64=kj. B. and the white points means the center point of each line. AO =T /2-O Ax= w idth / 2 . and navigated the vehicle to the next stair automatically. So a set of thresholds are preset. And this phonemena is mainly caused by changing of viewpoints and the influence of illumination. Evaluate h(®j) on the N-M points not in the minimal subset. So the UGV can turn the corner and go upstairs or downstairs autonomously.9 Estimate the offset parameter * 1. planed the path. the UGV will climb the stairways safely. such as feature matching. The ideal objection of our algorithm is to make the vehicle stay in the center of the stairways and steer it go ahead along the stairways.l. the orientation of the UGV is in the vertical direction. Steer the vehicle and turn the corner Our algorithm is used to guide the vehicle to climb the stair automatically. Forj Fig. The red lines is the detected parallel lines. the UGV will run along the orientation of the last operation. the stair detection program will be restarted to recognize the stairs. the detected stair edges are labeled by red lines. and the position of the UGV is in width/2 of the image.*. guides the vehicle to the center of the stairways. Oj8. Denote the best as ®j*. and the detail of the RANSAC algorithm is described below: Set of data points: * Pj =xj. . EXPERIMENTS We test our system and algorithm under various lighting conditions on various style and materials of stairs which are indoors and outdoors. Through experiment we preset the thresholds of AO and Ax as 0. The RANSAC algorithm is an algorithm for robust fitting of models.5 be calculated from these parallel lines using RANSAC algorithm. when facing to the sun only a portion of stair edges are not labeled that is because the image is saturated by the sunshine. and steer the vehicle up The direction of the stairway is 0 = arctan(k*) and the average of x coordinates is x xi / N So we can calculate the offset angle AO and offset position Ax as equation (4). In fact. using the equation (2) and (3) in the last page. and refine using (weighted) least-squares. Restrictions apply. .M} from the {pl. Generate an estimate ®j from the minimal subset.N}. our algorithm can still detect the stair edges (red lines).. In practice it is no use to make the UGV always stay in the precision center of the stair.

737-758 (2007) G.A. "Invariance Properties of Gabor Filter-Based Features" Overview and Applications. Downloaded on March 20.Napsac :High noise. 1997 JS Gutmann. J. " Detection and Localization of Curbs and Stairways Using Stereo Vision" Proc.4654 R. "Visual Curb Localization for Autonomous Navigation".Matthies. BMVC97. 24(6):381-395. "Vision-based Detection of Kerbs and Steps". Manduchi. Anastasios I. R. including extracting the edges. Las Vegas. 2002.Myatt. S. No.Proc. pages 458-467. Communication of the ACM. At last. Fig.S. 2000. Matas. Restrictions apply. 15(5):1088 1099 M.d) test. and orientation optimized by two-dimensional visual cortical filters". 11(d) shows the vehicle is adjusted in the center of the stair by our algorithm.Bolls. Vol. Proc. 7. Fig.C. 11 A steering processing when the UGV are mounted on the side of the stair 1811 Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY MADRAS. high dimensional robust estimationit's in the bag.11(c) shows the vehicle inclines on the stairwells and our algorithm can still detect the stair edges and estimate the parameters robustly.Bishop. "Vision-Guided Autonomous Stair Climbing". October2003. On Image Processing. Roumeliotis. spatial-frequency. 2(7):1160-1169. R. Proc.4GHZ) using C language. Mourikis. V.Daugman. "Image Representation Using 2D Gabor Wavelet" IEEE Trans. Fig. The offset parameters marked by the white line are estimated successfully. Brady. UK. Cardiff.Torr. British Machine Vision Conference. IEEE/RSJ IROS'03. Fig. the over time of the systems is less than 60ms.. ICRA. Se and M. Manduchi. pages 448 {457. 2004.ICRA 2005 4648. Kyrki and H.2010 at 06:18:51 EDT from IEEE Xplore. Nikolas Trawny. V. Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography. M Fukuchi. Y. 11(a) shows the vehicle is mounted on the side of the stair. M Fujita "Stair climbing for humanoid robots using stereo vision" Intelligent Robots and Systems. Randomized ransac with t(d. 0.Fischler and R.Craddock. Turchetto and R. Stergios I.1985 Tai Sing Lee. Finally. UK.M.6 to the stairway.R.Xiong and L.2002.Nasuto.J.H. 10 The results under various conditions [12] REFERENCES S.Am. and steering the UGV.Soc. The results proved the validity of our algorithm. 1996. British Machine Visioin Conference. Figi 1(b) shows that our vision system is guiding the vehicle to the center of the stairways. P. estimating the time. [1] [2] [3] The indoor condition fqce to the. .etc "Autonomous Stair Climbing for Tracked Vehicles" The International Journal of Robotics Research. -un [4] [5] [6] [7] [8] [9] [10] [11] The outdoor condition Fig. J. on Pattern Analysis and Machine Intelligence.(IROS 2004) Xiaoye Lu. 26. "Uncertainty relation for resolution in space.OPT. K.1981 and D. The white lines indicate the estimated center of the stairwells. Chum and J. implemented on a laptop computer (Pentuium4 2. Kamarainen. IEEE Trams. 2006. Kalviainen. CONCLUSION We have demonstrated a vision system based on our own algorithm which can guide the UGV to climb the stairs automatically under various conditions. Cardi*). 18(10):959-971 J.

Sign up to vote on this title

UsefulNot useful- SB Release Report
- navPoint-40
- Steep Ramps and Stairs
- Dr Mariah - Seminar 11 Nov 2014
- AB114-1 revit stairs by phill
- Dox
- SSwB
- design-of-staircase-1197052627390217-2
- Building Regulations
- Barrie Scott is Back With Another Home
- Stairs Joinery
- Education of Deacon Rusk
- 931-2-PB
- Codes Homeowners
- Prop Rac
- 07 Stair
- Ejercicios de referencia
- TGD_K
- EN(951)
- Copy of
- OSHA 1910.24 AND 1910.27.pdf
- 37286880-Design-of-Staircase.pdf
- Staircase
- Prosteel ASD
- Technical Specifications Stairs
- Prueba Olimpiada Final 2013 Ingles 4
- ET_327_09-29-15
- Stair
- Stairs
- Presentation 1
- A Stairway Detection Algorithm based on vision