2015 8th International Congress on Image and Signal Processing (CISP 2015)

Lane Detection Based on Spiking Neural Network

and Hough Transform
Xue Li, Qingxiang Wu, Yu Kou, Lei Hou, Heng Yang
Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, College of Photonic and
Electronic Engineering,
Fujian Normal University
1026044914@qq.com, qxwu@fjnu.edu.cn

Abstract—In the field of the unmanned automobile and the
automobile auxiliary driving system, the real-time and accurate
detection of the lane is very important. Based on the previous
research on the lane detection, the paper introduces the spiking
neural network with the parallel mechanism to detect the lane.
Firstly, the region of interests (ROI) is set on the origin image
that collected by a vehicle on-board camera. In order to reduce
processing time, areas outside the road are excluded in the ROI.
Then the image preprocessing is applied to the ROI, including
RGB to grayscale, gray stretch and median filtering to eliminate
noise. Edge detection of the lane is the key to determine whether
the Hough transform can detect the lane. In this paper, the
spiking neural network is used to detect the edge of the lane.
Finally, Hough transform is used to detect the lane. Experimental
results show that this method is more accurate and robust than
other methods.
Keywords-lane detection; region of intrests; spiking neural
network; hough transform
I. INTRODUCTION
The detection of road lane line has a key role in the
unmanned automobile and the vehicle assistant driving system.
When the vehicle appears to deviate from the lane or vehicle
safety distance is too small, it can timely alert the driver to
avoid dangerous [1-2]. Because of the low price of machine
vision and the rich information of the image, the machine
vision has been widely used in the lane detection of intelligent
vehicle. Hough transform is a widely used method in the lane
detection. It has very good suppression of noise performance
and is not sensitive to the target which is partially occluded and
covered in the image. But, because of the complexity of Hough
transform, the computation speed is very slow, and the false
detection rate is large. It cannot meet the requirements of
accuracy and real-time requirements. The detection speed is
improved by limiting the polar angle and the polar diameter in
Hough transform space in reference [3]. Reference [4] detected
the lane by finding the best threshold of the image, Random
sample consensus (RANSAC) algorithm is used to detect the
lane in paper [5], but it has high wrong rate. Paper [6] use the
peak point inverse transform to extract line segments in Hough
transforms space. All the methods mentioned above are
modified in the process of Hough transform, do not take into
account the appropriate processing methods in the image
before Hough transform to improve the detection accuracy and
shorten the calculation time.
This article integrates a set of image processing
techniques and spiking neural network to improve accuracy of
the lane detection. Firstly, the ROI is set on the image captured
from the on-board camera. This can greatly reduce the running
time of the algorithm and improving the detection speed;
Secondly, A set of image preprocessing techniques is applied to
the ROI region, such as transform from RGB image to
grayscale, gray stretch and median filtering. Edge detection
based on the spiking neural network with biological
mechanism is proposed to extract efficient edge lines. At last,
the Hough transform is used to detect the lane. In this paper the
new ideas are that the ROI preprocessing is used to speed up
the detection time and spiking neural network is used to obtain
better edge lines to improve the detection accuracy.
II. IMAGE PREPROCESSING
The origin image is captured from the on-board camera.
Because the color image contains a lot of noise and other
interference information, they will affect the final lane
detection. Therefore, the ROI and preprocessing techniques is
used to process the origin image so that the noise and other
interference information can be removed and then, the correct
rate of lane detection can be improved [7].
A. Select ROI in Origin Image
The images collected by the on-board camera are color
images. Each pixel in the image is made up of R, G, and B
three color components. Each color contains 256 levels. So the
change range is about 16000000 (256×256×256) for each pixel
in the image. As seen, the color image contains a lot of
information. Processing these information needs to consume a
lot of time, so that it is difficult to process in real-time because
of time consuming of the color image processing. Through
statistical analysis of the multi frame images acquired by the
on-board camera, the 1/3 area above the image is the sky and
the background area. It is call it invalid area. The remaining 2/3
region is an effective area, which is called the ROI, as shown in
Figure 1. Processing on only the ROI can greatly reduce the
time of algorithm and improve the running speed. The original
image is shown in Figure 2 (a), the ROI region is shown in
Figure 2 (b).

978-1-4673-9098-9/15/$31.00 ©2015 IEEE 626


Figure 1 The interested region of road area
(a) Original image
(b) ROI area
Figure 2 ROI area selection
B. Image Preprocessing
Image preprocessing includes color image to grayscale
image, gray stretch, median filter to eliminate the image noise
and other interference information. In this paper, a weighted
average method is used to convert the color image into
grayscale images. Because gray stretch can increase the
contrast between the lane and the road, this can make the lane
more prominent. Grayscale image is shown in Figure 3a, and
the gray stretch image is shown in Figure 3b.
G ray = 0.02900 × R + 0.58700 × G + 0.11400 × B (1)
The method of image filtering includes the frequency
domain filtering and the spatial domain filtering [8]. Spatial
domain filtering is simpler and faster than the frequency
domain filtering. It is mainly the salt and pepper noise in the
road image, the spatial domain filtering can remove the salt and
pepper noise well and preserve the edge details of the image.
Its main principle is to use the middle value of every pixel in
the neighborhood of one pixel instead of current pixel value.
The image after median filtering is shown in Figure 3c.
f ( x , y ) = m e d ia n{ S f (x,y ) } (2)
Figure 3a Grayscale image
Figure 3b Image after gray stretch
Figure 3c Image after median filter
III. SPIKING NEURAL NETWORK TO EXTRACT THE
EDGE OF THE LANE
The quality of the edge detection of road lane plays an
important role in the lane line detection. Previous researchers'
work is based on the existing edge operator. Such as the
Laplace-Gauss operator (LoG operator), various gradient
operators (such as Roberts, Sobel and Prewitt) and canny edge
detection [9]. LoG operator has some ability to suppress the
noise. But some of the original edge also was smoothed result
in some edges cannot be detected. The gradient operator has
high accuracy of edge detection, but it is easy to lose the edge
information, and it does not have the ability to suppress the
noise. The canny edge detection is considered as a good
compromise between the accuracy and suppressing the noise
interference. In this paper, a novel method of image edge
detection based on spiking neural network is proposed extract
better edge lines. Compared with other methods, the spiking
neural network has more real biological information processing
mechanism.
Human visual system has a high efficient image
processing capability [10]. Neuroscientists have found that
there are various receptive fields from simple cells in the striate
cortex to those of the retina and lateral geniculation nucleus,
and the neuron can be simulated by Hodgkin and Huxley
neuron model [11-13]. Based on these receptive fields and
Hodgkin and Huxley neuron model, a novel method of edge
detection based on spiking neural network is proposed [14].
The edge response of the spiking neuron was shown in Figure
4, and the spiking neural network model was shown in Figure
5.
Figure 4 The edge response of the spiking neuron
In the case of edge excitation, the neuron can produce a
dense pulse output signal, but in the case of flat input, the
neuron can only produce a sparse pulse output signal. So we
can use the density of the pulse output signal generated by the
627
neuron to distinguish whether it is the edge pixel in the input where g ex
x , y (t ) and g x, y (t ) are the conductance for excitatory and
ih
image [15].
inhibitory synapses respectively, τ ex and τ ih are the time
constants for excitatory and inhibitory synapses respectively,
vN 1 (t ) is the membrane potential of neuron N1, Eex and Eih are
the reverse potential for excitatory and inhibitory synapses
respectively, cm represents a capacitance of the membrane,
gl represents the conductance of membrane, ex is short for
excitatory and ih for inhibitory, wup
x, y
_ ex represents the strength

of excitatory synapses, wup x, y

_ ih represents the strength of

inhibitory synapses, Aex and Aih are the membrane surface area
connected to the excitatory synapse and inhibitory synapse
respectively. According to the description of biological
receptive fields, values for wup x, y
_ ex and
wup
x, y
_ ih are expressed as

follows [16].

­0 if (y-yc ) ≤ 0
Figure 5 Spiking neural network model for image edge detection up _ ex ° (7)
w( x, y ) =® −
( x − xc )2 ( y − yc ) 2
−
The first layer was photonic receptors layer, each pixel in °w e δx
2
δ y2
this layer corresponds to a receiver. The second layer was ¯ e max if (y-y c )>0
composed of four different neurons. Each of them has four
different receptive fields respectively. 'X' represent an ­0 if (y-yc )>0
up _ ih ° (8)
excitatory synapses, 'ǻ' represent an inhibitory synapses. The w( x, y ) =® −
2
( x − xc ) ( y − yc )
−
2

third layer was the output layer. The neuron in the output layer °w e δ x
2
δ y2
¯ i max if (y-y c ) ≤ 0
integrated the output from the intermediate layer. In the middle
layer, there are four parallel arrays of neurons, N1, N2, N3 and where ( xc , yc ) is the centre of receptive field RFrcpt .
N4 and only one neuron in each array is shown in Figure 5 for
simplicity. Each of these layers perform the processing for up, ( x, y ) ∈ RFrcpt . δ x and δ y are constants, we max and wi max are
down, left and right edges respectively and connected with the the maximal weights for excitatory synapses and inhibitory
receptive field by different weights. G( x , y ) represent the gray synapses respectively. When the neuron's membrane voltage
reaches a threshold voltage, the neuron generates a pulse
scale. ( x, y ) ∈ RFrcpt . qxex, y and qxih, y were the peak conductance
signal, and then the neuron enters a period of dormancy. When
generated by the excitatory synapses and inhibitory synapse at the membrane voltage reaches the threshold voltage again at
the receiving point (x, y). Each receptor can transform a value the time of dormancy, the neuron generates a pulse signal
of gray scale to peak conductance by the following expressions. again. S N 1 (t ) represent the pulse sequence generated by the
qxex, y = α G( x, y ) qxih, y = β G( x, y ) (3) neurons of N1.
­1 if neuron N1 fires at time t.
α and β are constants. According to the conductance based S N 1 (t ) = ® (9)
integrate-and-fire model, the potential changes of neuron N1 ¯0 if neuron N1 does not fire at time t.
can be expressed in the following expressions. Likewise, Neuron N2, N3, and N4 are governed by a set of
ex equations similar to that for neuron N1. The parameters of the
g ( x, y ) (t ) 1 spiking neural network are set as follows: vth = -60 mv, vreset = -
=− g (exx, y ) (t ) + α G( x , y ) (4)
dt τ ex 70 mv, Eex = 0 mv, Eih = -75 mv , El = -70 mv, gl =1.0
g(ihx , y ) (t ) 1 μs/mm2, cm =10 nF/mm2 ˈ τ ex =4 ms, τ ih =10 ms, τ ref =6
=− g(ihx , y ) (t ) + β G( x , y ) (5)
dt τ ih ms, Aih =0.028953 mm2, Aex =0.014103 mm2 , we max is set to
0.7093 for excitatory synapses, and wi max is set to 0.3455 for
dV N 1 (t ) w(upx,_y )ex g (exx, y ) (t )
cm = gl ( El − VN 1 (t )) + ¦ ( Eex − VN 1 (t )) inhibitory synapses [17]. Figure 6a is the result of edge
dt ( x , y )∈RFrcpt Aex detection of Sobel operator. Figure 6b is the canny edge
detection. Figure 6c is the edge detection based on the spiking
w(upx ,_y ih) g (ihx , y ) (t ) neural network. It is the firing rate map on the output layer,
+ ¦
( x , y )∈RFrcpt Aih
( Eih − VN 1 (t )) (6)
Bright lines show that the corresponding neurons fires with a
high frequency and indicate the edges with high contrast. Dark
lines show that the corresponding neurons fires with a low

628
frequency and indicate the edges with low contrast. Using the y
firing rates, different contrast edges can be separated.

( xi , yi )

ρ
(xj , y j )

Figure 6a Edge detection based on sobel operator

θ
o x
Figure 7 Polar coordinate of straight line

θ ρi = x i cos θ + yi sin θ
Figure 6b Canny edge detection
( ρ0 , θo )

ρ j = x j cos θ + y j sin θ

o ρ
Figure 6c Edge detection based on the spiking neural network
Figure 8 Polar coordinates mapping to the parameter space
IV. LANE DETECTION BASED ON HOUGH TRANSFORM
Parameter space can be considered as discrete in practical
Firstly, we propose three assumptions of road model before application and a two-dimensional accumulator A (λ , θ ) is
Hough transform [18]. established. As a fixed point ș can take all possible values,
(1) The assumption of a linear model. According to the corresponding Ȝ can be calculated according to equation (10).
characteristics of the ROI and the characteristics of the Let A (λ , θ ) add 1 after calculate each (λ , θ ) . That
image captured by the on-board camera, the lane in the is A(λ ,θ ) = A(λ ,θ ) + 1 . Suppose that the size of image
camera field of view is supposed a straight is D × D , as for ȡ and ș, the range of value
line: y = kx + b .
are [ − 2 D / 2, 2 D / 2] and [− π , π ] respectively. Found the
(2) Suppose that the left lane and right lane are parallel. In 2 2
the camera field of view, whether it is a straight line, a maximum value Amax (λi , θi ) in the results of the voting
curve, or an S curve, it is assumed that the two lanes space A(λ , θ ) . λi and θi are the polar coordinate equation with
are parallel to each other, and they intersect in the the largest collinear points of the origin image ˄ collinear
vanishing point.
points is Amax (λi , θi ) ˅. Hough transform as shown in Figure 9.
(3) The direction of the lane is not perpendicular to the Lane detection results are shown in Figure 10.
driving direction.
Hough transform is the most widely used in the lane
detection. The basic idea of Hough transform is to convert the
image coordinate space to the parameter space to realize the
fitting line curve. In addition, it can detect the lane even the
image was partly occluded and damaged. As shown in Figure
7, the line equation can be expressed in polar coordinates.
λ = x cos θ + y sin θ (10)
Ȝ is the distance from the origin point to the straight line, ș is
the angle with the positive X axis, the range of ș is ±90 0 . Two
collinear points ( xi , yi ) and ( x j , y j ) in polar coordinates mapping
Figure 9 Hough transform
to the parameter space are two sine curve, and intersect at one
point [4]. As shown in Figure 8.

629

Figure 10 Lane detection
The experiment detected 6000 frames that collected from
the on-board camera. Two important evaluation criteria are
used as follows. The true positive rate (TPR) and the false
positive rate (FPR) are calculated as TPR= (the number of
detect lanes) / (the number of target lanes), FPR= (the number
of false positives) / (the number of target lanes).The TPR is
95.6%, the FPR is 5.5%. Use the method in reference [4] and
reference [5] to detect the lanes under the same experiment
conditions. The TPR is 92.6% and 90.2%, the FPR is 8.3% and
9.6%. Most of the misdetection was caused by the loss of the
lanes or the fuzzy of the lanes. The mode is simulated by
Matlab program in a PC with CPU 1.2G, it takes about 6
seconds to detect the lane. If the model is implemented in
parallel on hardware, the lane detection can be achieved within
100ms. Therefore, this model can be applied to artificial
intelligent systems. Experimental results show that the novel
algorithm has good accuracy and robustness.
V. CONCLUSION
In this paper, a spiking neural network with the biological
information processing is proposed to detect the lane. Firstly,
select the ROI from the origin image that collected from the
on-board camera. Then, image preprocessing is performed on
ROI. The spiking neural network has used to detect the lane
edge in this paper. Compared with other edge detection
algorithms (Like canny edge detection, Laplace-Gauss operator
and a variety of gradient operators), this edge detection
algorithm based on the spiking neural network can detect the
weak edges of the details, and it contains more real biological
information processing mechanism. At the same time, so it is a
new intelligent edge detection algorithm. Finally, the lane is
detected by Hough transform. Future work is to optimize the
algorithm, so that it can apply to the lane departure warning
system.
ACKNOWLEDGMENT
The authors gratefully acknowledge supports from Fujian
Provincial Key Laboratory for Photonics Technology, the fund
from the Natural Science Foundation of China (Grant
No.61179011) and Science and Technology Major Projects for
Industry-academic Cooperation of Universities in Fujian
Province (Grant No.2013H6008).
REFERENCES
[1] S. C. Yi, Y. C. Chen, C. H. Chang, “A Lane Detection Approach Based
on Intelligent Vision,” Computers and Electrical Engineering,Vol. 42,
pp.23-29,2015
[2] J. Wang, C. J. Lin, S. M. Chen, “Applying Fuzzy Method to Vision-
Based Lane Detection and Departure Warning System,” Expert Systems
with Applications,Vol. 37, pp.113-126,2010
[3] Q. Wang, “An Improved Lane Detection Algorithm Based on Hough
Transform,” Computer and Digital Engineering,Vol. 42, pp.2164-
2167,2014.
[4] Z. L. Jin, G. M. Hu, H. J. Zhang, G. H. Man, “Lane Detection Method
Based on Optimal Threshold and Hough Transform,” Instrumentation
and Measurment,Vol. 28, pp.88-91,2009.
[5] J. Guo, Z. H. Wei, D. Q. Miao, “Lane Detection Method Based on
Improved RANSAC Algorithm,” 2015 IEEE Twelfth International
Symposium on Autonomous Decentralized Systems,pp.285-288,2015
[6] J. R. Cai, J. W. Zhao, “Camera Calibration of Binocular Stereo Vision
System,” Journal of Jiangsu University,Vol. 27, pp.6-9,2006.
[7] X. R. Liu, Z. X. Cai, “Robust Detection and Tracking of Structured
Road Lane,” Photoelectron and Laser,Vol. 18, pp.1834-1838,2012
[8] Y. Yuan, G. L. Zhang, “Improved Median Filtering Method,” Journal of
Chengdu University of Technology,Vol. 31, pp.126-129,2013
[9] Y. Zhao, B. L. Zhou, H. X. Chen, “A New Algorithm for Edge
Detection Based on Canny Algorithm,” Journal of Jilin University,Vol.
50, pp.740-744,2012.
[10] Hosoya. T, Baccus, S. A. Meister, M, “Dynamic predictive coding by
the retina,” Nature,Vol.436, pp.71 -77,2005
[11] Knoblauch. A, Palm. G, “Scene Segmentation by Spike Synchronization
in Reciprocally Ronnected Visual Areas,” Local Effects of Cortical
Feedback,BiolCybern,Vol.87, pp.151-67,2002
[12] Christof J. Schwiening, “A Brief Historical Perspective: Hodgkin and
Huxley,” The Journal of Physiology,Vol. 590, pp.2571-2575,2012
[13] Ant. nio-Carlos Guimar. es de Almeida, Ant. nio Márcio Rodrigues,
Antonio Fernando Catelli Infantosi, “Computational Neuroscience in
Perspective,” Brazilian Journal of Biomedical Engineering,Vol. 30,
pp.205-206,2014
[14] Q. X. Wu, T. M.McGinnity1, Liam Maguire1, Jianyong Cai,
G.D.Valderrama-Gonzalez, “Motion Detection Using Spiking Neural
Network Model,” Advanced Intelligent Computing Theories and
Applications. With Aspects of Artificial Intelligence Lecture Notes in
Computer Science,Vol. 5227, pp.76-83,2008
[15] Dayan. P, Abbott. L, “Theoretical Neuroscience :Computational and
Mathematical Modeling of Neural Systems,” MIT Press , London,
pp.248-254,2001
[16] Kan. E.R, Shwartz. J.H, “Principles of Neural Science,” McGraw-Hill
Press, NewYork, 2000
[17] Hodgkin. A, Huxley. A, “A Quantitative Description of Membrane
Current and It’s Application to Conduction and Excitation in Nerve,”
Journal of Physiology,Vol. 117, pp. 500-544,1952
[18] Y. M. Dai, X. Zhang, Z. F. Wang, “Research on Lane Detection Method
Based on Improved Hough Transform Algorithm,” Journal of Hangzhou
Dianzi University,Vol. 18, pp.91-95,2006.
630