8 BD 7

Corridor Navigation and Obstacle Distance Estimation for Monocular Vision Mobile Robots
Zhengyin Zhou, Tianding Chen, Di Wu, Changhong Yu

International Journal of Digital Content Technology and its Applications. Volume 5, Number 3, March 2011
Corridor Navigation and Obstacle Distance Estimation

for Monocular Vision Mobile Robots
1
Zhengyin Zhou, 2Tianding Chen, 3Di Wu,4Changhong Yu
1,
Zhejiang Gongshang University Hangzhou China, zzysakura@hotmail.com
2,
Zhejiang Gongshang University Hangzhou China, chentianding@163.com
3,
Zhejiang Gongshang University Hangzhou China, zjwudi@tom.com
4,
Zhejiang Gongshang University Hangzhou China, yuch_2007@163.com
doi:10.4156/jdcta.vol5. issue3.19
Abstract
To achieve autonomous indoor mobile robot navigation, the robot needs to identify the direction,
obstacle position and other information. This paper describes a solution for indoor mobile robot
navigation and obstacle distance detection which based on monocular vision. We use Hough transform
to find the straight lines in the corridor environment which were used to find the vanishing point as the
navigation direction for the mobile robot. At the same time, use the priori knowledge of the corridor
environment and the pinhole camera model to calculate the distance of the obstacle. Test results on our
robot system show that the algorithm presented in this paper can detect the direction of the corridor
and estimate the obstacle distance accuracy. This algorithm is relatively simple with high real-time
ability and robustness
Keywords: Monocular Vision, Hough Transform, Vanishing Point, Distance Estimation
1. Introduction
As we all know, the study of indoor navigation robot has been a very long time, the number of the
proposed methods is also considerable large. The major problem to be solved of indoor navigation
robot is the perception and learning of the indoor environment, target location identification and
obstacle avoidance. The actions above are all based on the understanding of the environment received
by the robot. Indoor navigation robot understands the environment mainly depends on various types of
sensors. The most commonly used sensors include infrared, sonar, laser radar and vision sensors, etc. A
relatively simple way to achieve indoor robot navigation is using odometric and sonar sensing [1].
Through these sensors, we can calculate the distance information of various obstacles and reconstruct
the indoor environment structure. However, its disadvantage is also obvious, sonar sensing have poor
performance on detection accuracy, meanwhile, some small obstacle and some sound absorbing
material can not be detected. Another commonly used method is to reconstruction 3D indoor or
outdoor scene based on laser radar [2]. Laser radar can detect the depth information of all objects
within the environment with high accuracy. However there are disadvantages of laser radar too, the
cost of a laser radar is high, the size of the device is large and it do not have good real-time ability for it
requires very large computational. The most widely used method is visual sensor. Because it can
collect abundant data with flexible algorithm. Now the main research of indoor navigation robot by
machine vision is using binocular stereo vision. It can be used to calculate the information of the indoor
environment and the depth of obstacles [3], this method is also widely used in outdoor environments
and have many practical applications [4]. Meanwhile, monocular vision is also wildly used. Here is
another application of robot navigation by contrasting and matching the feature points in environment
with the known ones which were learned and memorized before [5]. Another problem to be solved in
the research of indoor robot is the identification and distance detection of obstacles in the environment.
Antoni and Aude proposed a kind of depth algorithm based on the image structure of the whole image
rather than a specific object, which has some practical value [6]. Besides, the computer science
department of Stanford University proposed a new method use image database. They use 3D laser
range finder to measure the depth information of thousands of pictures and then establish an image
database by these information. Then the depth information of a single monocular image is got by
- 192 -
comparing the image with the pictures in the database [7].what’s more, Dr Bahaa use MVFH
Histogram[8] to improve path planning and obstacle avoidance for a wheeled driven mobile robot. In
short, the major problem to be solved for indoor navigation robot is the identification of the direction
and obstacle avoidance.
2. Identification of Direction
2.1. Use the Vanishing Point as the Forward direction
The most important part of indoor robot navigation is the corridor direction identification. The
algorithm described in this article is based on structural corridor, so the information which should be
processed is much less than that in the complex environment. As a result we can achieve better
real-time performance. In order to get the heading direction, the traditional method used to deal with
the corridor edge line [9]. Actually, it is not always easily to identify the corridor edge line as the
changes of light intensity, obstacles and some other elements.
This paper presents a new method of heading direction identification for indoor robot based on the
structural corridor. We use all kinds of lines in the structural corridor to calculate the vanishing point as
the heading direction of the robot. As we known, there are many lines which can show the heading
direction of the corridor besides the edge line. For example, the intersection of the walls and ceiling.
Meanwhile, many straight lines will be existed with the purposes of decoration. What’s more, two lines
of the door and window’s edges are also parallel to the edge lines of the corridor. All of these lines
mentioned above parallel with each other. The advantage is that this kind of lines almost exists in every
corridor and can be detected by some simple image processing algorithms. Figure.1 shows four
different corridors, we can see that these lines do exist in each corridor and they are easily detected.
These lines constitute the foundation of our algorithm which can tell robot the heading direction.
Figure 1. Four Different Styles of Corridors
Because of the linear perspective, the lines mentioned in structural corridor will be looked like
several lines scattering from one point on the image plane. This point is called vanishing point. As it is
showed in Figure.2 (b), plane AEFB, DHGC, AEHD (AILD), BFGC (BJKC) represent the ceiling,
floor, left wall and right wall of the corridor. Line AE represents intersection of the left wall and
ceiling; line BF represents intersection of the right wall and ceiling. Meanwhile, line DH represents
intersection of the left wall and floor; line CG represents intersection of the right wall and floor. The
- 193 -
Figure.2 (a) shows the 2D image of Figure.2 (b).It is also what we see on image plane. In Figure.2 (a),
we can see that lines AE,BF,DH,CG intersect at point O. Point O is the vanishing point, or the
midpoint of plane ABCD. It is understandable that the vanishing point is the heading direction of the
robot.
Figure 2. The 2D Image and 3D Image of Corridor
2.2. The Detection of Lines
To get the coordinates of vanishing points, we should get the lines which parallel the heading
direction of the corridor first. Here, firstly we transform the color image into grayscale image; then
processes the grayscale image by canny operator for edge detection; at last, detect the lines by liner
Hough transform[10]. What's more, we only need the lines which parallel the heading direction of the
corridor, so we calculate the slope of all lines and pick out the lines whose slope is neither too big nor
too small. The processing pipeline is shown in Figure 3.
Figure 3. The Image Processing Pipeline
Here, we use the open source image processing library OpenCV 1.0 and Visual C++ 6.0 as the
platform for image processing. From Figure 4, we can see that most of the straight lines which parallel
the heading direction of the corridor in Figure 1 are found. What’s more, all these lines are pointing in
the same direction. This direction is the vanishing point. The next assignment is how to find the
vanishing point.
- 194 -
Figure 4. The Result of Line Detection Algorithm
2.3. How to Find the Vanishing Point
We know that all the lines which are detected above intersect at the same point in accordance with
perspective model. But in reality, the intersection points of these lines will be distributed regularly
around the vanishing point because of the noisy and some other reasons. In this paper we use a method
base on three iterative to get the coordinate of the vanishing point. Firstly, we need to calculate all the
coordinates of the intersection points intersected by the lines detected. Here there are two straight lines
A1 x  B1 y  C1  0
(1)
A2 x  B2 y  C 2  0
and point A  x1 , y1  , point B  x2 , y2  on the first line, point C  x3 , y3  , point D  x4 , y4  on the

second line. First, calculate the value of d   y2  y1    x4  x3    y4  y3    x2  x1  . Line AB
parallels line CD if d equals to zero. Else if d do not equals to zero, then line AB intersects line CD at
E  x0 , y0  .So
 x0  x2  x1  x4  x3   y3  y1    y2  y1  x4  x3   x1   y4  y3  x2  x1   x3  / d


y0   y2  y1   y4  y3  x3  x1   x2  x1   y4  y3   y1  x4  x3   y2  y1   y3  / d
(2)
By formula (2) we can get all the intersection points intersected by the lines detected before. We name
these intersections E n (n equals to the number of the intersection points).Their coordinates
are  x1 , y1 , x2 , y2 ....... xn yn  .
Then, we calculate the best coordinate of the vanishing point by three iterative. Suppose En is the
cluster of intersection points. Firstly, we calculate the average value of En in x, y coordinates which
is expressed as E , x, y  
- 195 -
 n n

  xn  y n 
 
x, y   n 1 , n 1
n n


(3)
 
 
 
After get the coordinate of x , y , we calculate the Euclidean distance between x , y and each  
intersection point, and then weed out the intersection points whose Euclidean distance is bigger than
 
' '
the threshold we set. After this process, recalculate the x , y by formula (3). This Iterative will be
done three times, in order to filter out the intersection points which do not meet the rules. Here the
threshold values we select are 0.3d, 0.2d, 0.1d (here d is the length of the image diagonal). After three
iterations those intersection points which are very far from the vanishing point do not participate in the
final calculation, as a result, the coordinate of vanishing point will be relatively accurate. Figure.5 is
the result processed by our iterative algorithm. The red lines are the straight line detected by Hough
algorithm. The black points are the intersection points calculated by formula (2). The red cross, blue
cross and green cross are the first, second and third result of vanishing point calculated by iteration
algorithm. From figure.5, we can see that the black points distribute throughout the whole image
besides the vicinity of the vanishing point. If we do not remove these useless points, the accuracy of the
vanishing point will be affected. From Figure 5 we can see that the more iteration we use, the more
accurate the vanishing point is. Therefore, we can conclude that this vanishing point finding algorithm
based on iteration is correct. See the test and verification in part 4.
Figure 5. The Intersection Cluster and Vanishing Point
3. Obstacle Detection and Distance calculation
3.1. Obstacle Detection
In the design of indoor navigation robot, another problem need to be solved is the obstacle detection
and avoidance. In indoor environment, a simple way used to identify the obstacles is by the color of
obstacles [11] and their movement [12][13]. In this article, we simply use the difference of color
between the obstacle and the floor to identify the obstacle because the complexity of structural corridor
is relatively low. As we have calculated the vanishing point of the corridor, it is easily to distinguish
the area of the floor from the other part of the corridor. What we need to do is judge if there are
obstacles in the area of the floor. It is not necessary to process other area of the image. For the color of
the obstacle is general different from the floor, so we can simply define that the area whose color is
- 196 -
different form the floor is the obstacle. Meantime it is important to remove the area whose proportion is
too small, for this kind of area is usually the result of noisy produced by light and other reasons.
Figure.6 is the image of obstacles detection in corridor. Figure.6 (a) is the original image; figure.6 (b)
is the binary image which is the result of the color detection; figure.6 (c) is the obstacle marked image.
Figure.6 shows that the obstacle detection algorithm proposed in this article is very effective. Its
real-time ability is also very good since its calculation cost is low.
Figure 6. The image of obstacle detection
3.2. The Calculation of Obstacle Distance
The usual method of obstacle distance calculation is by using stereo vision. In order to get the depth
information, the camera should be calibrated firstly, and then the parameters got by camera calibration
will be used in the transformation from the image coordinate system to world coordinate system.
Another usual method is by using laser radar to measure the depth of the obstacle. We have already
discussed the advantages and disadvantages of these methods in introduction. In this article, we
measure the depth of the obstacles by monocular camera with some known parameters.
Generally speaking, if we want to obtain 3-D space position information from 2-D image base on
monocular vision system, some of the constraints (i.e. the height of the camera is constant, camera
parallel to the ground etc) must be taken. However, using stereo vision requires calibration and a lot of
matrix calculation, so it can not be real-time. In this article, we present a new obstacle depth estimation
algorithm base on pinhole model.
In figure.7, plane KLNM is the image plane, plane ABFE is the ground plane of corridor, point A, B,
C, D are four points on the two edge lines, line AB, CD parallel to each other and both of them
perpendicular to the heading direction of the corridor. Points G, H, I, J are four corresponding points of
A, B, D, D on image plane. So we can use this to estimate the distance between corridor and robot.
Form figure.7 we know that Triangle OAB OCD are not strictly similar triangles, but the distance of
point O and plane KLMN is much less than the distance of point O and ling AB, so we can regard
approximately that triangle ABO, CDO are similar triangles. The actual measurement data in part 4
shows that this kind of approximation does not affect the accuracy of estimation.
Figure 7. The Pinhole Model of Camera
- 197 -
Here, we suppose the distance of point O to plane KLMN is f, the width of the corridor (the length
of line AB, CD) is D. The distance of point O to line AB is H1 ; the distance of point O to line CD
is H 2 . The length of line AB on image plane (the length of line GH) is d1 ; the length of line CD on
image plane (the length of line IJ) is d 2 . Then the following relationship is established.
 d1 f
D H
 1
(4)
d2 f
 
 D H2

The value of D in Formula (4) is the width of the corridor which is known, d1 d 2 Is the width of
the corridor on imaging plane which will be measured, H 1 H 2 is the distance between robot and
obstacles which is the aim of this algorithm. Therefore, we can estimate the distance between robot and
corridor if we know the value of f and other parameters. For it is simply similar triangle relationship, so
we can calculate the value of f by multi-pair of d n and H n .
After get the parameters need, we can calculate the distance of the obstacle detect by obstacle
detection algorithm. For we have detect area of the floor and the position of the obstacle on the image,
so it is easily to obtain the plane which the obstacle is on. As what is shown in figure.8, cuboid
ABCD-MNPQ stands for the corridor, the planes of the two obstacles are plane EFGH and plane
EFGH. They are all planes which are perpendicular to the heading direction of the corridor. Line HG is
the intersection line of plane EFGH and plane QDCP. The location of the robot camera is the point O.
The distance from the robots to the first obstacle can be seemed as the distance from point O to line HG.
So we can set up formula (5) by Figure.7 and formula (4):
f
H D (5)
d
The value of f can be calculated by the test images in which the distance of the obstacles is known.
D is the prior knowledge of the width of the corridor. The only needed part is the value of d, in other
words, the width of the corridor where the obstacle is located in the image.
Figure 8. The Estimation of Obstacle Distance
- 198 -
4. Verification of Algorithm and Test Analysis

The algorithm is verified and tested on an autonomous mobile robot (AS-RF) Figure 9.AS-RF is
0.44m tall, weighing about 71kg.The robot subsystems are described in detail below. They are
comprised of:
1) 24V 150W Maxon DC motor and 2000 line encoder;
2) 5 Ultrasonic sensors with the range of 0.41m to 7m;
3) AU-G35-SC18 CCTV camera with Sony Visca PTZ Control Protocol;
4) 24V, 20AH Lithium battery power
The AS-RF can carry 40 kg of equipment at speeds of up to 0.7m/s. In order to get the best behave
of the algorithm, we set the speed of the robot which is about 0.3m/s. Here we use Osprey 100 video
capture card to capture images form the camera. The video streaming we set is 10 frames per second
with the apparent resolution of 640×480. The main computer on the robot is a desktop computer with
CPU of Intel E2160 c 1G DDR2 memory. The programming environment we use is visual C++ 6.0.
Figure 9. The AS-RF Robot
4.1. Verification and Analysis of Vanishing Point Algorithm
We test the vanishing point algorithm by 3 short slips recorded by the CCD camera of robot. These
3 clips were shot in three different corridors with different illumination. We picked 150 frames from
each of the slips, 50 frames from the very starting, 50 frames from the middle of them and another 50
frames form the end of the slips in order to make out experiment result more scientific and more
accurate. Figure 10 is the result of the test. The value of X-axis stands for the number of frames, the
value of Y-axis stands for the Euclidean distance between the estimated vanishing point and the real
vanishing point. The three lines with cross, circle and asterisk stand for the each of the 50 frames at the
very starting of the slips, in middle of them and at the end of the slips. From Figure.10 (a), we can see
that the values of distance between the estimated vanishing point and the real vanishing point in three
clips are distributed in the range 20 pix. It is because this clip is shot in a corridor with appropriate light.
Figure.10 (b) (c) is the result of the clips shot in corridors with bright light and poor light. We can see
that these two results are not satisfactory. The error rate is obviously bigger than clip one. However, if
we control the illumination in a good range, our vanishing point algorithm in this article will have high
accuracy. It can be completely applied to the indoor robot navigation
- 199 -
(a) Appropriate Light
(b) Bright Light (c) Poor Light

Figure 10. The Result of Vanishing Point Detection
4.2. Test and Analysis of Obstacles Distance Estimation
We have stated that parameter d and f must be known before calculate the distance of unknown
obstacles. This means different environments have different values of D and f, and their values must be
calculated beforehand. Our test environment is a corridor with the width of 2.64m and the length of
30m. The height of our camera is 0.41m. The resolution of the camera we use is 640 × 480. Table 1
shows 5 groups of different distance of known obstacles. We can see that the value of D is 2.64m, the
value of f is calculated by different d and H, its average value is 616.8(pix).
Table 1. The Value of f Base on Different Distance of Obstacle

D dn Hn f
2.64m 599pix 2.64m 599
2.64m 369pix 4.4m 615
2.64m 265pix 6.16m 618
2.64m 203pix 7.92m 618
2.64m 173pix 9.68m 634
Now we have got all the priori parameters needed in out algorithm. So it is time to test its
correctness and accuracy. Here we use 11 images to test the algorithm. Figure.11 is 4 marked pictures
picked out form the 11 images. In figure.11 the black point is the vanishing point detected, the red
triangle is the area of the floor, red rectangle is the location of the obstacle. The length of the red line
between two blue points is the value of d which would be estimate by the program.
- 200 -
Figure 11. Four Different Obstacle Distance Images for Obstacle Detection
The distance of the obstacle in figure.11 is (a) 3m (b) 4m (c) 5m (d) 6m.
After calculation, we get the data in Figure 12
Figure 12. The Result of the Obstacle Distance Test
In Figure 12, the X-axis stands for the number of the images, the Y-axis stands for the value of the
obstacle distance. The blue bar stands for the real distance of the obstacles, the red bar on the right
stand for the estimated distance. From Figure 12, we can see that when the distance of the obstacle is
less than 10m, the error can be controlled in 5cm. That is to say, the accuracy of out algorithm meets
the requirements of indoor robot navigation. When the distance is farther than 10m, the error rate
becomes a little bigger, but since out algorithm is used for indoor robot which only needs to detect the
- 201 -
obstacles not very far away. So we can make the conclusion that our algorithm is perfectly acceptable
for indoor robot.
5. Conclusion
In this paper, we propose an indoor robot navigation base on monocular camera. Our main purpose
is to solve two problems: the identification of heading direction in corridor environment; the estimate
of the obstacle distance. In the first part, detect the straight line in the corridor environment which will
intersect at one point base on Principle Perspective. This point is called vanishing point which can be
seem as the heading direction of the robot. This algorithm is relatively simple so it can be used in
real-time system. The test data shows that its error rate is very small. In second part, we focus on the
needs of obstacle detection and distance estimation. Our obstacle detection algorithm is base on the
different color between obstacle and the floor. Then the distance of the obstacle is estimate base on
similar triangles. This method makes it possible to get the depth information of the image by
monocular vision. Meanwhile the test data shows that its accuracy can meet the requirement of indoor
robot. In a word, this article proposes a new method of indoor robot navigation which has some
practical value.
6. References
[1] Ricardo Carelli, Eduardo Oliveira Freire, “Corridor navigation and wall-following stable control
for sonar-based mobile robots”, Robotics and Autonomous Systems 45 pp 235-247,2003
[2] Hartmut Surmann, Andreas Nüchter, “An autonomous mobile robot with a 3D laser range finder
for 3D exploration and digitalization of indoor environments”, Robotics and Autonomous Systems
45 pp 181-198, 2003
[3] Aniket Murarka, Benjamin Kuipers, “A Stereo Vision Based Mapping Algorithm for Detecting
Inclines, Drop-offs, and Obstacles for Safe Local Navigation”，in Proc. of the IEEE/RSJ Conf. on
Intelligent Robots and Systems (IROS), pp 1646-1653, 2009.
[4] Arturo L. Rankin, Andres Huertas, “Stereo vision based terrain mapping for off-road autonomous”,
Proceedings of the SPIE, International Society for Optical Engineering Vol 7332, pp.
733210-733210-17, 2009
[5] Stephen Se, David Lowe, “Vision-based Mobile Robot Localization And Mapping using
Scale-Invariant Features”, In International Conference on Robotics and Automation, Seoul,
Korea,pp. 2051-58, 2001
[6] Antonio torralba,Aude Olive, “Depth Estimation from Image Structure,IEEE Transactions on
Pattern Analysis and Machine Intelligence”,Vol.24,No.9, 2002
[7] Jeff Michels, Ashutosh Saxena, “High Speed Obstacle Avoidance using Monocular Vision and
Reinforcement Learning”, In International Conference on Machine Learning (ICML), pp. 593-600
2005
[8] Dr. Bahaa I.Kazem , Ali H. Hamad , Mustafa M. Mozael, “Modified Vector Field Histogram with a
Neural Network Learning Model for Mobile Robot Path Planning and Obstacle Avoidance”, IJACT,
International Journal of Advancements in Computing Technology, Vol. 2, No. 5, pp. 166-173, 2010
[9] Saitoh,Takeshi; Tada,Naoya, “Indoor Mobile Robot Navigation by Central Following Based on
Monocular Vision”, IEEJ Transactions on Electronics, Information and Systems, Volume 129,
Issue 8, pp. 1576-1584, 2009
[10] Sihai Yang, Duansheng Chen, “Hough Transform Analysis: A global perspective”, Journal of
Computer-Aided Design & Computer Graphics Vol118, No18 Aug1, 2006
[11] Youfu Wu, Jing Wu, “Recognizing Moving Objects Based on Gaussian-Hermite Moments and
ART Neural Networks”, JCIT, Vol. 5, No. 8, pp. 63-70, 2010
[12] Wei Cao, Yixin Yan, Shengming Li, “Unsupervised Color-Texture Image Segmentation Based on
A New Clustering Method”, JNIT, Vol. 1, No. 2, pp 96-102, 2010
[13] Ali H. Hamad, Fatima B. Ibrahim, “Path Planning of Mobile Robot Based on Modification of
Vector Field Histogram using Neuro-Fuzzy Algorithm”, IJACT : International Journal of
Advancements in Computing Technology, Vol. 2, No. 3, pp. 129-138, 2010
- 202 -

8 BD 7

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

8 BD 7

Uploaded by

Copyright:

Available Formats

Corridor Navigation and Obstacle Distance Estimation for Monocular Vision Mobile Robots

Zhengyin Zhou, Tianding Chen, Di Wu, Changhong Yu

Corridor Navigation and Obstacle Distance Estimation

Keywords: Monocular Vision, Hough Transform, Vanishing Point, Distance Estimation

2.1. Use the Vanishing Point as the Forward direction

Figure 1. Four Different Styles of Corridors

Figure 2. The 2D Image and 3D Image of Corridor

2.2. The Detection of Lines

Figure 3. The Image Processing Pipeline

Figure 4. The Result of Line Detection Algorithm

2.3. How to Find the Vanishing Point

and point A  x1 , y1  , point B  x2 , y2  on the first line, point C  x3 , y3  , point D  x4 , y4  on the

 x0  x2  x1  x4  x3   y3  y1    y2  y1  x4  x3   x1   y4  y3  x2  x1   x3  / d

Figure 5. The Intersection Cluster and Vanishing Point

3. Obstacle Detection and Distance calculation

3.1. Obstacle Detection

Figure 6. The image of obstacle detection

3.2. The Calculation of Obstacle Distance

Figure 7. The Pinhole Model of Camera

Figure 8. The Estimation of Obstacle Distance

4. Verification of Algorithm and Test Analysis

Figure 9. The AS-RF Robot

4.1. Verification and Analysis of Vanishing Point Algorithm

(a) Appropriate Light

(b) Bright Light (c) Poor Light

4.2. Test and Analysis of Obstacles Distance Estimation

Table 1. The Value of f Base on Different Distance of Obstacle

Figure 12. The Result of the Obstacle Distance Test

You might also like