See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/278703606

Single Camera Vehicle Detection Using Edges and Bag-of-Features

Chapter · January 2012

DOI: 10.1007/978-94-007-2792-2_13

CITATIONS READS

2 367

3 authors:

Jahanzeb Pirzada Ehsan Ul Haq

Beihang University (BUAA) International Islamic University, Islamabad
44 PUBLICATIONS   101 CITATIONS    49 PUBLICATIONS   250 CITATIONS   

SEE PROFILE SEE PROFILE

Hyunchul Shin
Chongshin University
39 PUBLICATIONS   194 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

ASSESSMENT OF TRAFFIC NOISE POLLUTION IN ISLAMABAD PAKISTAN. View project

Encryption and Authentication for Satellite Communication View project

All content following this page was uploaded by Jahanzeb Pirzada on 18 December 2018.

The user has requested enhancement of the downloaded file.

 Single Camera Vehicle Detection Using Edges and
Bag-of-Features

Syed Jahanzeb Hussain Pirzada 1, Ehsan Ul Haq 2 and Hyunchul Shin3 ,

1, 2, 3
Department of Electronics and Computer Engineering, Hanyang University,
Ansan, South Korea
{1 jahanzebp, 2ehsan}@digital.hanyang.ac.kr, 3shin@hanyang.ac.kr

Abstract. Vehicle detection is becoming a necessary part of Automatic Cruise

Control (ACC) and Advanced Driver Assistance Systems (ADAS). Our main
focus in this paper is on improving the performance of single camera based
vehicle detection systems. Edges are one of the main characteristics of an
object, which carries most of the information about an object in an image. In
this paper, it was observed that horizontal edges are strong feature for vehicle
detection. Therefore, we generated initial candidate using Horizontal Edge
Filtering (HEF) on canny edge map. These initial candidates are further verified
using Bag-of-Features (BoF) with K nearest neighbor algorithm. A threshold is
used on differences of histograms of training and test images for matching the
vehicles. In this paper, the combination of edges (initial candidate) and bag-of-
features (final verification) has improved detection rate significantly as
compared to other well known methods. Our method has 96% detection rate on
roads inside a city and 98% detection on highways.

Keywords: Vehicle detection system; Horizontal edge filtering; Bag-of-

features; Canny edge detection; K nearest neighbour.

1 Introduction

Nowadays travelling on vehicles has become a necessity instead of luxury. However

at the same time, number of deaths due to the vehicular accidents is also increasing
each year. Worldwide vehicles accidents effects every year between 20 to 50 million
people [1]. Therefore, developing an advanced driver assistance system (ADAS) is
becoming more and more important.
A lot of research is going on developing a vision based vehicle detection system
[2] [3]. The main goal of vision based vehicle detection systems is to accurately and
efficiently identify dangerous conditions [4]. Usually accidents can be avoided by
detecting lanes [5], pedestrian [6] and nearby vehicles [7]. In this paper, we have
presented a new vehicle detection system for detection of nearby vehicles.
Edge is one of the main characteristics of an object, which carries most of the
information of an object in an image. That’s why edge information is very important
in image processing, particularly, in the area of object detection and recognition
algorithms [8]. There are many edge detection methods used for vehicle detection like
Canny [9], Sobel [10] and SUSAN [11] edge detectors. Many papers has also
compared [12] [13] different edge based methods for different applications. In this
paper, based on comparisons [12] [13], we have found canny edge detector as best
technique for finding vehicle edges. It was observed that horizontal edges are strong
feature for vehicle detection [15]. So we have applied Horizontal Edge Filtering
(HEF) to filter the long horizontal connected edges to detect only vehicle from all the
edges in the image.
BoF is becoming as an important object recognition and classification technique.
BoF was derived from ‘bag-of-words’ (BoW) modeling for text retrieval like Google
search. BoW technique is a text analysis algorithm discriminating documents by
occurrence of particular words that each document contains [14]. In this paper, Initial
candidates generated using edges are further verified using Bag-of-Features (BoF)
with K Nearest Neighbour (KNN). BoF is used for verification because this technique
verifies the vehicle and non vehicle candidates effectively.
In past decade, there has been a lot of research on real-time detection of vehicles.
In [15] authors have reviewed most of the techniques. In [15] [16] [17] authors
emphasize that vertical and horizontal edges are very powerful heuristic for initial
candidate detection. Matthews et al [16] has used edge detection to find vertical edges
for detection of left and right position of vehicle. But vertical edges are not very
strong heuristic for on road vehicles as compared to horizontal edges. Moreover,
some obstacle on road may also have same vertical and horizontal edge profile as of a
car therefore we need an additional verification step. Some authors [18] [19] also used
shadow of car and combination of horizontal and vertical edges for vehicle detection.
Handmann et al. [17] proposed a template based on the observation that the
rear/frontal view of a vehicle has a “U” shape (i.e., one horizontal edge, two vertical
edges, and two corners connecting the horizontal and vertical edges). Hence, edges
prove to be effective candidate to be used for initial candidate detection. Therefore we
selected the HEF for initial candidate selection. Although using only edges do not
provide good detection rate so we combine edge based initial candidate detection with
verification using BoF algorithm.
The remaining paper is organized as follow. Section II introduces to Canny Edge
Detection. Section III explains the BoF algorithm. Section IV explains methodology
of proposed technique. Section V provides experimental results. Section VI concludes
this paper.

3 Canny Edge Detection

Canny edge detector performs smoothing with a linear combination of exponential

functions and then detects edges by higher order derivative operators. Canny tried to
optimize a set of criterion to maximize the performance of already available edge
detection methods. First requirement was low error rate, i.e. high response to edges
while minimum response to surfaces or planes. Second criterion was that edges
should be well localized on the boundaries. Third criterion was to have only a single
response to a single edge.
First Canny edge detector smoothes the image to remove noise. Next it calculates
gradient of the image. Then next step is to perform the non maximum suppression. In
this step all those pixels which are not at the maximum are suppressed. The edge map
is further improved by hysteresis, which tracks along the non suppressed pixels.
Canny edge detector next uses two thresholds. If a pixel’s value is above higher
threshold then it is selected as an edge. However, if a pixel’s magnitude is below the
second threshold then it is set to zero. In case if magnitude is between the two
thresholds then it is set to zero unless it is connected to an edge pixel. In this paper,
canny edge detector is used on images with rear view of vehicles as shown in Fig. 1.

Fig. 1. Edge map generated by canny edge detection for rear view of vehicles.

4 Bag-of-Feature Algorithm

BoF modelling is carried out by constructing a vocabulary of visual words from a

set of Scale Invariant Feature Transform (SIFT) descriptors. That is, it should go
through several steps; keypoint detection, extraction of SIFT descriptor, and vector
quantization. Keypoint-based regions are located at salient points such as corners and
blobs. Thus meaningful keypoints are selected. Next step is the extraction of SIFT
descriptor. SIFT descriptors are extracted from a set of detected keypoints. Then
vector quantization is carried out to generate a visual vocabulary using a set of SIFT
descriptors extracted from various images. Clustering algorithms such as k-means and
k-median are used for vector quantization. After k clusters are generated by
clustering, the centre of each cluster becomes a visual word. Therefore, the database
of SIFT descriptors should be sufficiently large and various to generate visual words.
A feature vector of each image is represented by the occurrence of the visual words; it
can take typical histogram of visual words. Histogram expresses the frequency of the
visual word in any given image.

5 Methodology

In this paper, main methodology involves two main steps, initial candidate generation
using edge based method and verification using BoF algorithm as shown in Fig. 2.

5.1 Initial Candidate Generation

Initial candidate generation is very important for vehicle detection as whole

detection accuracy is very much dependent on it. In this paper, for initial candidate
generation, we used HEF of canny edge map.
Fig. 2. Shows the flow chart of main methodology.
For reducing the computation time, we have selected region where there are cars
and roads in the input images. For region of interest (RoI) selection we cropped image
from top and bottom only. Canny edge detector is applied on RoI of input image to
generate edge map as shown in Fig. 3(b). In images with rear view of vehicles it was
noticed that long connected horizontal edges are usually belongs to vehicles.
Depending on this observation HEF is applied on canny edge response for detection
of vehicles. In HEF, each row of whole edge map is searched for at least ten
consecutive 1’s. Whenever a number of consecutive ones are found, they are
preserved as a long connected horizontal edge. Following this criteria whole image is
scanned and only those long connected edges are assigned value 1, while small
unconnected edges is assigned value 0. As a result small unconnected edges are
neglected and only long connected edges are used.
Vehicles rear view generally contains long horizontal edges due to the shape of
bumper and back windscreen and roof of vehicle etc. These long edges are located
close to each other and also vertically aligned. This information can be used to obtain
initial candidate. We have observed that if there are three long edges close to each
other and vertically aligned then it can be considered as an initial vehicle candidate
and labelling rectangle is generated around this region as shown in Fig. 3(c).

5.2 Verification Using Bag-of-Feature

Verification of initial candidate is an important step in our approach, as it reduces

number of false matches and improves accuracy of the detection. In this paper, we
used BoF algorithm for verification. BoF is one of the most used and successful
techniques for object detection.
 (a) (b)

(c)
Fig. 3. (a) Shows region of interest (RoI) generation, (b) shows canny edge map of RoI image,
(c) Shows results of horizontal edge filtering on an image.

In BoF algorithm, there are three main steps; visual vocabulary generation, training
class generation and testing.
In the Visual Vocabulary Generation, keypoints and descriptors are generated by
using Harris corner detector and Scale Invariant Feature Transform (SIFT)
Descriptors [20] respectively. Each descriptor represents a single patch. After patches
are formed from all images k-mean clustering [21] is used to cluster similar patches
together. After k (k= 500) clusters are generated by clustering, the centre of each
cluster becomes a visual word. Visual vocabulary for vehicle and non vehicle class
are generated separately. We use 500 images of vehicles from Caltech database and
500 images from random Google links for non vehicle database for vocabulary
generation.
In the second step training for class generation is performed to generate training
histogram of vehicle class and non vehicle class. Same algorithm is followed in
training images as of vocabulary generation except the last part of vocabulary
generation is replaced by generation of training histograms. In the training cycle 500
images from Caltech vehicle dataset [22] is used for training of vehicle class and
many random images (about 500 images) from different Google links is used for non
vehicle class.
In the third step testing is performed on the labelled images extracted from edge
response during initial candidate detection. For test images, histograms are generated
using same training class generation algorithm. Then K nearest neighbour algorithm
is used for matching the training and test images histograms. If the difference in
neighbour distance between histograms of training and test images is less than a
threshold then it’s acceptable as a vehicle. Otherwise, the image is considered as a
non-vehicle class. A threshold value of 1.50 is used for comparing neighbour
distances of histogram; this value is selected by hit and trial method. Then the non
vehicle class histogram is matched with histogram of test image and if difference is
less than the threshold, then it’s confirmed as a non vehicle. Verification of vehicles
double checks the results from edge response and hence improves detection by
removing false initial detection.
6 Experimental Results

Experiment is performed using MATLAB version R2008b on a PC with Intel

Core2 Duo processor with 2 GB random access memory (RAM). In our experiment
we extracted images from real life video. The video is captured from camera which is
mounted inside a vehicle at a fix place to shoot the road.
Table 1 show the results of vehicle detection by our proposed method. The
experiment is performed on two different cases, roads inside the city and on
highways. Total vehicle candidates represent the amount of total vehicles present in
all the frames of video. Initial Candidates detection represents total candidates
detected. Non-vehicle detection represents the total non vehicle detection, which is
removed by verification process. Verified candidates represent the total vehicles
detected after verification by BoF method. Missed Candidates represents number of
vehicles that are not detected. The missed vehicles are due to overlapping vehicles
and large distance of vehicles from target vehicle. Detection rate represent the rate of
detection of vehicles. Fig. 4(a) shows the detection of vehicles in highways and Fig.
4(b) shows detection of vehicles in city roads. Fig. 4(c) shows non vehicle and wrong
vehicle detection in highways and Fig. 4(d) shows it in city roads after initial
candidate detection. Fig. 4(e) shows missed overlapping vehicles detection. Fig. 4(f)
shows missed vehicle detection which is at large distance from vehicle. There were
many overlapping vehicles on roads inside city as compared with on highways. So the
detection rate is less on roads inside city.

Table 1. Results of vehicle detection

S.No Image Sequence

Inside City Highways

1 Total Vehicle Candidates 240 150
2 Initial Candidates Detection 234 169
3 Non-vehicles detection 3 22
4 Verified Candidates 231 147
5 Missed Vehicles 9 3
6 Detection rate 96 % 98 %

7 Conclusion

In this paper, edge based initial candidate detection is used with bag-of-feature
based verification. The combination of two methods has helped to detect all vehicles
using edge based method and remove false vehicle detection by verification through
BoF algorithm. So, combination of two well known techniques is used for improving
the detection rate of vehicles. Our results show around 98% detection of vehicles on
highways where there is low traffic and 96% detection on roads inside city where
there is high traffic.
 In this work we have emphasized on day time sunny and cloudy weather
conditions. In future, this work will be enhanced for different weather conditions.

(a) (b)

(c) (d)

(e) (f)

Fig. 4. (a) shows detection of vehicles in highway, (b) shows detection of vehicles
inside city roads, (c) shows Wrong detection using inside city roads images, (d)
shows Wrong detection using highway image, (e) shows Undetected vehicle using
inside city roads image. (f) show missed vehicle detection which is at large distance
from vehicle.

Acknowledgment

This work was supported by Ministry of Knowledge Economy (MKE) and IDEC
Platform center (IPC) at Hanyang University. This research work is sponsored by
‘Higher Education Commission (HEC), Govt. Of Pakistan’ under the scholarship
program titled: MS level Training in Korean Universities/Industry.
 References

1. World Health Org., “World Report on Road Traffic Injury Prevention,” [Online]. Available:
http://www.who.int/violence_injury_prevention/publications/road_traffic/world_report/facts
heets/en/index.html
2. Z. Sun, G. Bebis, and R. Miller, “On-Road Vehicle Detection Using Evolutionary Gabor
Filter Optimization,” IEEE Transactions on Intelligent Transportation Systems, vol. 6, pp.
125-137, 2005.
3. Z. Sun, G. Bebis, and R. Miller, "On-Road Vehicle Detection: A Review," IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 28, p. 694-711, 2006.
4. M. M. Trivedi, T. Gandhi, and J. McCall, “Looking-in and looking-out of a vehicle:
Computer-vision-based enhanced vehicle safety,” IEEE Trans. Intell. Transp. Syst., vol. 8,
no. 1, pp. 108–120, Mar. 2007.
5. J. McCall and M. M. Trivedi, “Video-based lane estimation and tracking for driver
assistance: Survey, system, and evaluation,” IEEE Trans. Intell. Transp. Syst., vol. 7, no. 1,
pp. 20–37, Mar. 2006.
6. T. Gandhi and M. M. Trivedi, “Computer vision and machine learning for enhancing
pedestrian safety,” in Computational Intelligence in Automotive Applications. Berlin,
Germany: Springer-Verlag, May 2008, pp. 59–77.
7. T. Gandhi and M. M. Trivedi, “Vehicle surround capture: Survey of techniques and a novel
Omni video based approach for dynamic panoramic surround maps,” IEEE Trans. Intell.
Transp. Syst., vol. 7, no. 3, pp. 293– 308, Sep. 2006.
8. Z. Hongna, W. Qi, “Image measuring technique and its applications,” Journal, Electrical
Measurement & Instrumentation, 2003.
9. J. F. Canny, A computational approach to edge detection. IEEE Transactions. On Pattern
Analysis and MachineIntelligence, 8, pp. 679- 714, 1986.
10. Russ, “The image processing handbook”, CRC Press, 2002.
11. S. M. Smith, and J. M. Brady, SUSAN- A New Approach to Low Level Image Processing.
International Journal of Computer Vision, 23(1), pp. 45-78, 1997.
12. Z. Musoromy, Dr. S. Ramalingam, N. Bekooy, “Edge Detection Comparison for License
Plate Detection”. IEEE Transactions on Control, Automation, Robotics and Vision
(ICARCV), pp. 1133-1138, Dec 2010.
13. C. Fan, Y. Ren, “Study on the Edge Detection Algorithms of Road Image”. Third
International Symposium on information Processing, pp 217-220, 2010.
14. S. Brin and L. Page, "The anatomy of a large-scale hypertextual Web search engine," in
Proc. International WWW Conf. Computer Networks and ISDN Systems, pp. 107-117,
1998.
15. Z. Sun, G. Bebis and R. Miller, “On-road vehicle detection using optical sensors a review,”
IEEE International Conference on Intelligent Transportation Systems, Washington, DC,
October 3-6, 585-590. 2004.
16. N. Matthews, P. An, D. Charnley, and C. Harris, “Vehicle Detection and Recognition in
Greyscale Imagery,” Control Eng. Practice, vol. 4, pp. 473-479, 1996.
17. U. Handmann, T. Kalinke, C. Tzomakas, M. Werner, and W. Seelen, “An Image Processing
System for Driver Assistance,” Image and Vision Computing, vol. 18, no. 5, 2000.
18. C. Tzomakas and W. Seelen, “Vehicle Detection in Traffic Scenes Using Shadows,”
Technical Report 98-06, Institutfur Neuroinfor-matik, Ruht-Universitat, Bochum,
Germany, 1998.
19. N. Srinivasa, “A Vision-Based Vehicle Detection and Tracking Method for Forward
Collision Warning,” Proc. IEEE Intelligent Vehicle Symp., pp. 626-631, 2002.
20. D.G. Lowe, “Distinctive image features from scale invariant keypoints,” International
Journal of Computer Vision, vol. 60, pp. 91- 110, 2004.
21. D. J. C. MacKay. Information Theory, Inference, and Learning Algorithms, 2003
22. Caltech Cars Dataset, http://www.robots.ox.ac.uk/~vgg/data3.html.

View publication stats