IET Image Processing

Review Article

Vehicle detection in intelligent transport ISSN 1751-9659

Received on 21st April 2018
Revised 24th April 2019
system under a hazy environment: a survey Accepted on 19th September 2019
E-First on 28th November 2019
doi: 10.1049/iet-ipr.2018.5351
www.ietdl.org

Agha Asim Husain1 , Tanmoy Maity1, Ravindra Kumar Yadav2

1Department of Mining Machinery Engineering, Indian Institute of Technology (ISM), Dhanbad, India
2Department of Electronics and Communication Engineering, Skyline Institute of Engineering and Technology, Greater Noida, India
E-mail: aghahusain@gmail.com

Abstract: Developing an intelligent transportation system has attracted a lot of attention in the recent past. Moreover, with the
growing number of vehicles on the road most nations are adopting an intelligent transport system (ITS) for handling issues like
traffic flow density, queue length, the average speed of the traffic, and total vehicles passing through a point in a specific time
interval and so on. ITS by capturing traffic images and videos through cameras, helps the traffic control centres in monitoring
and managing the traffic. Efficient and unfailing vehicle detection is a crucial step for the ITS. This study reviews different
techniques and applications used around the world for vehicle detection under various environmental conditions based on video
processing systems. This study also discusses the types of cameras used for vehicle detections, and the classification of
vehicles for traffic monitoring and controlling. This study finally highlights the problems encountered during surveillance under
extreme weather conditions.

1 Introduction
Nowadays, traffic detection is attracting a lot of researchers’
attention in computer vision and intelligent transportation systems
[1, 2]. The rapid upsurge in several vehicles in the last few years
has posed serious problems in the management of the
transportation system vividly. Conditions such as accidents,
blockages on roads, vehicle robberies, and traffic congestion are a
common problem faced in the management of traffic. The
intelligent transport system (ITS) has become a dynamic area of
research since the last decade. It plays an important role in
handling many real-time situations like avoiding traffic congestions
and accidents, in the surveillance of traffic, in the collection of toll
tax, terrorist monitoring, and in traffic flow under hazy
environments [3], and many more. That is why ITS has become a
significant field of study.
Vehicle detection is the key task of the transport monitoring
system, as after the detection of the vehicle, other applications can
be used in a better way. Various technologies [4, 5] are being
undertaken for detecting and categorising vehicles automatically.
Different algorithms (based on shape, size, texture, colour, etc.) are
used to categorise different vehicles from videos, using the image
processing [6], this information is then transferred to the
centralised controlling and monitoring system [7].
Fig. 1 shows the process that is followed in a traffic monitoring
system based on videos, which has a camera mounted on the
roadside, watching over the traffic scene. The camera works as a
sensor device used to capture traffic videos. The captured video
images are converted into a digital form using a processor. The
images in their digital form are processed and examined using
image processing techniques. The extracted information can then
be used by any external user, such as a traffic control centre, for
traffic monitoring and controlling, etc.
Image processing techniques are applied for a video-based
traffic monitoring system in five stages: (i) image acquisition
(CCTV) and digitisation, (ii) vehicle detection through background
subtraction/foreground extraction, (iii) image segmentation, (iv)
vehicle identification under different environmental conditions and
classification, and (v) tracking.
Fig. 2 shows that in the process of vehicle detection, the video
image is processed, and the region where the vehicles are present is
determined. This region of interest can be an alone pixel, a line of
pixels, or a group of pixels [9].
Throughout the traffic parameter extraction, they are obtained
by comparing the vehicle detection status on the region of interest
at the different instant of time frames. In a traffic surveillance
system, background and illumination changes make vehicle
detection a difficult task. The key problems are changing
backgrounds (camera jitter, water rippling, and waving trees, etc.),
illumination changes (gradual or sudden light changes), and
climatic changes [10]. In these issues, the foreground mask by a
Gaussian mixture model (GMM) is used, but there is a big chance
of fault detection. To overcome this problem, various projects are
being taken up to find better vehicle surveillance ways through
which the chances of false detections can be reduced.

Fig. 1  Block diagram of vehicle detection and categorisation

IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10 1

© The Institution of Engineering and Technology 2019

Fig. 2  Filtering acquisition, detection, and identification of vehicles [8]
Fig. 3  Traffic surveillance camera
(a) Gatso, (b) Highway agency CCTV, (c) Specs, (d) Trafficmaster [25]
This paper reviews the vehicle detection process used for
surveillance and includes Section 2, which describes traffic
surveillance cameras, Section 3 explains about moving vehicle
detection techniques and the problems encountered during
surveillance, Section 4 describes the classification of vehicles for
effective traffic monitoring and controlling. Section 5 discusses the
approach used to remove the haziness from image sequences.
2 Traffic surveillance cameras
Traffic cameras are modern equipment that makes functional usage
of video surveillance technology. Their footage can be seen on TV
news during traffic reports. These cameras are implanted at the top
of traffic signals or are mounted along the busy streets, or at the
busy crossings of the highways. The cameras also helped in
recording traffic patterns for future study and helped in monitoring
and controlling the traffic [11]. Therefore, traffic cameras are an
important device of video surveillance [12], which aims to monitor
a specified area for safety and security reasons. Smart traffic
cameras are commonly used in many road transportation systems,
like surveillance, traffic management, driver assistance, and control
access systems, security and law enforcement, etc. [13].
The research-based on vehicle detection methods through
videos taken by camera started in the late 1970s [14]. In 1984, the
University of Minnesota initiated a wide-area multiple video
imaging detection system known as ‘Autoscope’ [15, 16]. It helps
in reducing the installation and maintenance costs and led to the
development of ITS [17, 18]. The earlier main emphasis was given
to the detection of licensed plates only [19], but because of the
rising demand, other ways of vehicle classification like colour,
size, environment, etc. are being taken as a criterion of vehicle
classification [20, 21].
Traffic cameras are generally placed at some common
congestion point to monitor vehicle flow on roads, highways,
major routes. The monitored information then passes to the traffic
control centre for taking necessary action for any mishappening.
Traffic cameras are also important in decisions concerning
surveillance, future road construction, and development, apart from
2 IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10
monitoring the accidents or major closures, jams on roads, etc.
Generally, day-to-day traffic flows do not differ much, but in case
of a road mishap or a closure, a traffic alert can be very helpful for
a time-crunched commuter. Real-time videos from specified
locations are collected through surveillance cameras using Pan Tilt
Zoom, Gatso, Vector camera system, etc. and the images are then
transmitted to the enforcement traffic control centre through the
Internet [22–24]. Figs. 3a–d show some of the good cameras used
in traffic surveillance.
One of the cameras named Gatso is based on a radar signal to
record a vehicle's speed. The camera machine sends a radio signal
to the vehicle, the signal is then returned by the vehicle. The period
taken between the signal sent and received back from the vehicle is
noted. Similarly, another signal is sent and received back, and the
timing is also noted. The difference in the timings enables the
‘Gatso’ camera to know how quickly the vehicle is travelling
between the two points. This camera takes several pictures if the
vehicle crosses the specified speed limit.
There is another camera named ‘Specs’ which can monitor four
lanes simultaneously and is generally mounted on gantries. This
type of camera is enabled with the Automatic Number Plate
recognition system. The camera takes the photograph of each
vehicle that passes it and the vehicle's data is then sent to another
set of cameras, which is mounted at around 200–250 m on the
road. The average time taken by the vehicle in travelling between
these two set points can be calculated and helps in estimating the
speed of the vehicle. These cameras can work night and day, and in
all weather conditions-heat, wind, hazy, ice, and snow, as they are
fitted with infra-red sensors that capture electromagnetic radiations
[26, 27] of objects.
The precautions that are taken while installing these traffic
cameras are
• Cameras are mounted in such a way that the whole intersection
area gets covered.
• The flashlight is installed for recordings at night.
• Cameras are positioned and calibrated to easily record the data
of the license plate of the violating vehicles.
• Local law enforcement agencies are consulted to know the most
bothersome areas/intersections.
• Cameras are placed in environment-controlled housings to
protect from bad weather.
• Cameras are placed such that they have enough visibility and a
decent view of all streets/lanes.
3 Vehicle detection
The concept of vehicle detection is, to select the area of interest as
to where a vehicle exists on the road in real-time. The system has
very high complexity because of real-time acquiring of images, as
the traffic scene's surroundings are continuously changing. The
complexity of vehicle detection also depends upon light change,
weather conditions, shadow, complex background interference,
camera parameters, drift and occlusion of vehicles, etc. So, it is
very difficult to detect vehicles under these changing conditions.
For this, vehicles must be extracted from an image sequence. The
‘moving’ characteristic is used to differentiate a moving vehicle
from a stationary background. The precision or accuracy of vehicle
detection is of great significance for vehicle tracking movement
expression and behaviour understanding and is the basis for
subsequent processing. Generally, the algorithm used for vehicle
detection is divided into four stages: problem domain, image or
video acquisition, frame segmentation, and vehicle recognition [28,
29], as shown in Fig. 4.
Fig. 4 shows the framework of the vehicle detection based on
video processing. By using the problem domain information in the
video, the vehicle is identified from the image sequence. Then,
with the use of embedded technology and smart camera, image
acquisition takes place. Since the vehicle is in motion, acquiring
images can be a bit complex.
Therefore, researchers prepare an algorithm with great
robustness to improve the adaptability and precision of detection,
which helps in recognising the image [30].
© The Institution of Engineering and Technology 2019
Fig. 4  Generalised framework of vehicle detection and recognition
In a broad sense, vehicle detection can be categorised based on
[31]:
(i) prior knowledge,
(ii) motion,
(iii) Wavelets
(iv) machine learning (ML),
(v) deep learning (DL).
3.1 Prior knowledge-based methods
Various approaches to detect a vehicle have been made based on
prior knowledge. They are symmetry, colour, vertical, or horizontal
edge, shadow, wheels, and 3D model, etc. [31, 32].
3.1.1 Symmetry: For both horizontal and vertical directions, there
must be a symmetry while taking the observation from the
stationary camera. The method has not been considered as the
finest method of detection due to noise.
3.1.2 Colour: Colour segmentation, along with motion has been
introduced to detect the moving vehicle. Colour segmentation
generally uses split and merge algorithm, whereas vehicle
segmentation uses frame differentiation.
3.1.3 Vertical and horizontal edges: Conventional gradient-
based and morphological edge detectors are the two kinds of edge
detection methods. The conventional gradient-based uses a Sobel
operator and generalised Hough transform. Morphological edge
detection uses dilation and erosion operation to spot edges, and
they are better than conventional edge detectors based on
computational cost. Apart from the above-mentioned method,
separable morphological edge detector was proposed by Siyal
which depends less on edge directionality and increases the
robustness of vehicle detection [33].
3.1.4 Shadow: As the region under any vehicle is darker as
compared to other regions on the road, this fact becomes the basis
of vehicle detection [34]. A threshold method centred on the
characteristics of the Hue saturation value colour space is
suggested by Cucchiara et al. [35]. Xua et al. [36] suggested
another algorithm which suppresses the moving cast shadow of
vehicle detection.
3.1.5 Wheel: The methods proposed for vehicle detection in the
daytime are not valid for nighttime. This fact was discussed by
Yoneyama et al. [37] stated that methods applied to daytime do not
work properly if it is applied for detection at nighttime. While
considering the problem faced by Yoneyama et al., Iwasaki and
Kurogi [38] defined a new unique method specifically for
nighttime. The method calculates the distance between its wheels
(front and rear), which helps in finding the size of the vehicle.
Also, this method helped in finding the lane where the vehicle
exists. However, for this method, the camera needs to be installed
on the roadside rather than on top of any traffic signal.
3.1.6 3D model: The vehicles’ 3D pose is mapped to a
corresponding model description. Dealing with obstruction is
another benefit of the 3D model [39]. Various approaches of 3D
modelling have been proposed like graph matching [40], viewpoint
consistency check [41], alignment-based 2D method, indexing and
invariants [42], gradient-based method [42, 43], neural network,
self-similarity [44], etc. In all the above-mentioned methods, 2D
image features like line segments, points, and conic sections, are
IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10
© The Institution of Engineering and Technology 2019
matched with corresponding 3D features of the position [39].
Karsten et al. suggested a method that works on the 3D restoration
of a dynamic atmosphere with a fully regulated background for
traffic movements [45]. Lou et al. [39] proposed another method
that disintegrated the two motions, i.e. translation and rotation.
Besides that, Ghosh and Bhanu [46] suggested a 3D model
mapping arrangement where he classified the vehicles into various
types like hatchback, sedan, and wagons.
3.2 Motion-based methods
3.2.1 Optical flow: This method of vehicle detection is based on
optical flow calculations, whereas the above-mentioned methods
have used spatial features. Motion detection is one of the vital
parameters in an ITS. There is a relative motion between the traffic
scene and the sensor because of which the pixels on the image
appear to be moving. This motion because of the vector field is
called optical flow. By using optical flow autonomously any
vehicle can be detected from the camera. However, its computation
is very complex and subtle to noise. Meyer et al. [47] estimated the
concept of a displacement vector field, which initialises a contour-
based tracking process. Real-time analysis of video streams
without any specialised hardware is not that easy. Barron's et al.
[48] have mentioned a detailed discussion on optical flow in his
work.
3.2.2 Background subtraction: Background subtraction, also
named as foreground detection, is one of the vital jobs by which
many other applications like vehicle tracking, recognition, and
irregularity detection can be realised. It is used for motion
segmentation in a static scene [49]. For detection of motion, pixel
by pixel image is subtracted from a background image, which is
considered as a reference image. A certain threshold level is
defined if the difference is above the threshold level, then the
image considered as foreground. The formation of the background
image is recognised as background modelling. Some
morphological post-processing is done after the creation of
foreground pixel maps like erosion, dilation, and closing to reduce
noise and improve the detected region. The reference background
image is updated from time-to-time and the image where the
section of the road is shown with no vehicle serves as [49, 50].
There are various techniques by which background subtraction can
be applied in terms of foreground detection.
Heikkila and Silven [51] gave a very simple concept of
background technique in which a pixel at a location (x, y) in the
present image It is marked as foreground if It(x, y) − Bt(x, y) > Th
is fulfilled; where a pre-defined threshold is denoted as Th. An
impulse response filter updates the background image, as shown in
the equation below:
Bt + 1 = αIt + (1 − α)Bt (1)
where α is a constant that ranges between 0 and 1.
The foreground pixel map construction is followed by
morphological closing and the elimination of small-sized areas [51,
52]. With the help of this easy background model, pixels
corresponding to foreground moving vehicles can be detected by
thresholding any of the distance functions [53]
d0 = It − Bt (2)
d0 = (ItR − BtR)2 + (ItG − BtG)2 + (ItB − BtB)2 (3)
d∞ = max ItR − BtR , ItG − BtG , ItB − BtB (4)
3
where R stands for red, G stands for green, and B stands for blue
channels, and d0 is a measure operating on greyscale images.
Another method is a GMM, which was given by Stauffer and
Grimson [54] for real-time tracking. The algorithm depends upon
the assumption that the background is made of animated textures
like waves or wind-shaken trees. It represents every pixel with a
mixture of K Gaussians. Here, the probability of occurrence of
colour at a given pixel is
K
P(It) = ∑ ωi t N
, μi, t, ∑ (5)
i=1 i, t
Fig. 5  DL is a subset of ML which is again the subset of AI
N(μi, t, ∑ i, t) is the ith Gaussian model, and ωi, t weight [53].
Zivkovic [55] suggested a better GMM using a recursive
computation to continuously update the parameters of the model.
This model is widely adopted by many researchers for the analysis
of traffic [56–58] using a fixed number of Gaussians. Rapid
changes in illumination pose difficulty in applying background
subtraction in a real-world setting. Amali and co-authors [59]
suggested a method using sample-based background subtraction
called rapid background subtraction. The whole algorithm basically
uses three different techniques. In the first technique, the first two
consecutive frames are taken as a background model and this
model can be updated after the threshold period. The next
technique includes the classification of pixels in correspondence to Fig. 6  Graph plotted between accuracy (performance) and size of training
the background pixel model and the shadow detection method. The data
last technique involves updating of the background pixel model by
random pixel locations. By using this technique, accuracy and There are different approaches to solve any problem, and so
efficiency can be greatly increased. there are different ways a machine can learn. Depending on these
For online data, the subspace learning technique has been used approaches, there are three ways a machine can learn [67, 68].
for the background model. Oliver et al. [60] suggested a method The types of ML ways or algorithms are
using the concept of principal component analysis (PCA)
depending upon probability distribution function. In recent times, (a) Supervised learning (task driven-regression/classification).
Bouwmans et al. [61] introduced a detailed review of robust PCA- (b) Unsupervised learning (data driven-clustering/association)
PCP based methods for testing and ranking the existing algorithms
(c) Reinforcement learning (learns to react to an environment).
for foreground detection. The study here is driven by the need for
robust vehicle detection and a classification algorithm that can be
used in a traffic monitoring system to deal with such changes. 3.4.1 Supervised learning: In this algorithm, continuous
Because of critical situations in video surveillance, some monitoring is done to make it understandable and it practices a
researchers introduced the concept of fuzzy logic for background known dataset called the training dataset to make estimates. The
subtraction. training dataset contains input data and response values. As of it,
the supervised learning algorithm seeks out to build a prototype
that can make estimations of the response values for a new dataset.
3.3 Wavelet-based methods
Since the last decade, the wavelet transform has become one of the 3.4.2 Unsupervised learning: In the case of unsupervised
vital tools for signal processing applications. The vehicle motion is learning, the algorithm draws interferences from datasets
taken in 3D spatial–temporal data from the image scene [62–64], consisting of input data without categorised reactions. This is
and this is possible by wavelet transform. The benefit is very low simply a self-learning algorithm where a machine takes its own
computational complexity and an easy application in both spatial decisions.
and temporal directions. Though, these methods depend upon noise
and on variations of timings of movements. Wang and Chen [63] 3.4.3 Reinforcement learning: In this algorithm, the machine can
proposed a new algorithm using the spatial–temporal wavelet take its own decision depending upon its past experiences. It is
transform on videos to obtain motion information of vehicles. A totally relying on behaviourist psychology; whose outcomes are to
fusion technique of wavelet transformation and colour information expand the numerical reward. The learner is not advised which
features for automatic vehicle identification was proposed by Yin move to make; however, rather should find which activity will
et al. [65]. yield the greatest reward.
A major limitation of ML is that it cannot handle multiple
3.4 Machine learning dimensional data. Also, complex AI problems cannot be handled
by ML.
ML [66] is an interesting topic that helps in understanding how a
So, DL concept came into existence, which handled high-
machine updates the code on its own. ML is a subset of artificial
dimensional data and focused on the right features by themselves,
intelligence (AI) as shown in Fig. 5, which provides computers
requiring slight supervision from the system analyst. Fig. 6 shows a
with the ability to learn without being absolutely programmed. The
graph which shows the relation between accuracy and size of
meaning of this is that once the machine is programmed, it will not
training data, and it is found that DL shows better performance
change its own code even when a new problem is encountered. The
with an increase in the size of training data.
objective of ML is to focus on the development of computer
programs that change when exposed to new data. It changes
according to a situation where the machine does not change its 3.5 Deep learning
code. This is a self-learning concept; it learns whatever it has DL [69] is processed through neural networks, which is a multi-
acquired from the past scenario, situations, experiences, etc. So, the layered neural network learning algorithm. Every deep neural
machine comes up with new solutions by using the concept of AI. network (DNN) consists of three layers, the input layer, the hidden
layer, and the output layer. In between the input and output layers,

4 IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10

© The Institution of Engineering and Technology 2019

Fig. 7  Deep network with two hidden layers in between input and output
layers [68]
Fig. 8  Machine neural architecture
(a) Graphical representation of each undirected edge showing dependency for
Boltzmann machine architecture [66], (b) Restricted Boltzmann machine architecture
with stochastic neurons [66]
there can be n-multiple hidden layers, and processing of ‘n’ hidden
layers is possible because of high resources available these days.
Fig. 7 represents a generalised model of DNN where two
hidden layers are taken in between input and output layers.
Since the last decade, DNNs have prompted an arrangement of
hops forward on an assortment of assignments, e.g. computer
vision, machine interpretation, and voice acknowledgment, and so
forth. One of the basic parts achieving these achievements comes
about is convolutional neural networks (CNN) [70]. Krizhevsky et
al. [71] proposed a concept called AlexNet after which, CNNs
have exhibited prevalent execution for picture classification
differentiated and conventional ‘shallow learning’ strategies, and
have similarly been adequately associated for object detection [72],
IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10
© The Institution of Engineering and Technology 2019
video classification [73] and segmentation [24], etc. These
triumphs motivated a new line of research that focused on formed
higher execution CNNs, and the execution of framework models
has been improved by utilising further and more broad structures.
Simonyan and Zisserman [74] proposed VGGNet, empowering the
examination on the use of significant outline in PC vision. Szegedy
et al. [75] showed GoogLeNet which contains Inception modules,
setting the new best in class for the ImageNet Challenge. To deal
with this deprivation issue, He et al. [76] showed a waiting
learning framework named ResNet that can get ready liberally
more significant frameworks than those used in advance.
DL models are quite very interesting and proved to be an
outstanding solution for the detection and classification of vehicles,
so we briefly overview some of the models.
3.5.1 Convolutional neural networks: It is a deep discriminative
network, usually looks like that of the ordinary artificial neural
network, as mentioned in Fig. 7, where transforms have been done
through hidden layers. The deep architecture of CNN is explained
in [66] (refer to Fig. 3c) that discussed the arrangements with a
multi-channelled image as the participation input [77]. To manage
this complicated input, the CNN comprises of a few layers of
convolutions with non-linear actuation capacities to figure the
output. As an outcome, the CNN contains limited associations
whereby every area of the input is associated with a neuron in the
output. Each layer applies a distinctive channel and joins their
outcomes. What is more, the CNN comprises of the pooling layers
for subsampling. During the preparation stage, a CNN
consequently takes in the estimations of its channels considering
the given assignment. In its first layer, CNN may figure out how to
distinguish the edges from the raw pixels. At that point, in the
second layer, the CNN utilises the edges to distinguish
straightforward shapes. In this form, in the consequent higher
layers, by utilising these shapes, CNN might have the capacity to
learn more elevated amount highlights like facial shapes of the
animal present in the image. In the last layer, a classifier is utilised
to exploit these abnormal state highlights. Profound CNN's have
ended up being very fruitful in learning task highlights, which have
given much-enhanced outcomes, especially on computer vision
undertakings, conversely with contemporary ML procedures.
Normally, CNN's are prepared by utilising supervised learning
strategies whereby an expansive number of input–output sets is
important. Notwithstanding, acquiring a considerably huge data set
has been a key test in applying CNN's for settling another
assignment. Likewise, CNN's have been accounted for to be widely
utilised as a part of unsupervised techniques [71, 78, 79].
3.5.2 Deep Boltzmann machine (DBM): To understand the
DBMs, a basic knowledge of basic Boltzmann machines and the
restricted Boltzmann machines [80] is necessary. A network of
binary stochastic units with a defined ‘energy’ for the network is a
Boltzmann machine. A representation of the BM architecture is
shown in Fig. 8a. Although learning is ineffective in a shallow
BM, it can be made relatively efficient through a restricted
Boltzmann machine (RBM), as it does not allow the networks
among units in the same layer, as mentioned in Fig. 8b.
After training RBM, activities of hidden units can be taken as
the data for training an even higher level RBM. This stacking of
RBMs allows training numerous layers of hidden units in an
efficient manner, and this is considered as one of the most common
DL strategies. With each added layer, the overall model of RBM
improves significantly.
3.5.3 Recurrent neural networks (RNNs): RNNs, also known as
long short-term memory networks (LSTMs). This is specifically
useful in the modelling of text and speech-related sequence data.
Fig. 9 shows the deep generative architecture [81] of RNNs, whose
depth may be as much as the length of the input data sequence.
Recently, its use was limited because of the problem of ‘vanishing
gradient’ although it has a high potential strength the recent
literature points towards the new methods of optimising generative
RNNs, that modifies the stochastic gradient reasonably.
5

Fig. 9  Recurrent neural network [66]
Fig. 10  Stacked auto-encoder [66]
Table 1 DL algorithms for vehicles.
Algorithms Method used Purpose
fast region-CNN [88, selective search narrow down of bounding
89] boxes
overfeat [90] sliding window scanning the image at
multiple scales
faster region-CNN [88, region proposal identifying bounding boxes
89] network that must be tested
SSD (single shot bounding box forwards image only once
detector) multibox [88] regression for discretisation
3.5.4 Stacked auto-encoder (SAE): An auto-encoder means an
artificial neural network that helps in active coding [82] by way of
encoding, where a set of data flows shown in Fig. 10. The encoded
data is a compressed form of the entire dataset with reduced
dimensionality. A typical auto-encoder has an input layer with a no
of smaller hidden layers following it, and then an output layer. The
input data set is encoded by the small hidden layers and the output
layer rebuilds the input layer.
In the study, Bourlard and Kamp [83] pointed out that if the
auto-encoder architecture had just a single hidden layer of linear
neurons, the solution obtained from the auto-encoder would be
correlated to PCA. At that point, the component which is found out
by the auto-encoder is utilised to prepare another layer of auto-
encoder, etc. The scholarly weights, by means of this procedure,
are, in the long run used to instate a DNN [84].
• Discussion on the implementation of deep neural algorithms for
object-based instance: One of the DL algorithms for the object
(vehicle, human, animal, etc.) based instance is DNN [85, 86].
In contrast to various video-based vehicle detection [87, 88],
classification and in low illumination algorithms, DNN uses
‘You Look Only Once’ (YOLO) [89] framework method has
6 IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10
shown significantly better results. YOLO is very fast in terms of
identification of vehicles that use around 45 frames per second
compared to state-of-the-art detection systems. It uses a solitary
convolutional network at once predicts multiple bounding boxes
and class probabilities for those boxes. YOLO prepares on full
images and much improves the detection performance. YOLO
lags in terms of accuracy, but the model has several advantages
over previously used methods of vehicle detection. Some of the
practical algorithms, their methods and purpose are mentioned
in Table 1.
• Apart from the above-mentioned algorithms, the large-scale
dataset is of prime importance. There are various datasets
available for the manual and self-driving vehicle, which includes
ImageNet, COCO, KITTI, Cityscapes, CamVid, etc. ImageNet
[90–92] is an image dataset having of 14,197,122 images. This
dataset is mainly used for image classification and detection of
an object based on neural network training. Common Objects in
Context (COCO) [92–94] dataset by Microsoft is a large-scale
dataset feature over 200,000 labelled images with 80 object
categories. This dataset is used for scene analysis in 2D. KITTI
[95, 96] contains a driving 3D dataset of images and contains
ground truth for 400 dynamic scenes. It develops challenges for
stereo vision, optical flow, video odometry. Cityscapes [92, 93]
dataset contain 5000 fully segmented image and 20,000 partial
segmented images that focus on a semantic understanding of
urban street scenes. It works on pixel-level and instance-level
semantic labelling. Cambridge-driving Labelled Video Database
(CamVid) [92, 93, 97] is specified for 700 semantically
segmented images. These datasets are trained through DNN and
then tuned as per application.
• Problems encountered in vehicle detection: Various challenges
which exist in detecting a vehicle are continuous changes in the
background, shadowing, headlight reflection, camouflage,
clutter, video noise, environment, etc. [98]. Proper vehicle
detection is still a challenge in the present scenario. The details
of the various problem encountered are mentioned in the
Appendix (Table 2).
4 Vehicle classification
The classification of vehicles in a surveillance [99] system can be
categorised as two-wheeler, three-wheeler, light motor vehicle,
sports utility vehicle, light commercial and heavy motor vehicles,
as shown in Fig. 11. To differentiate between different types of
vehicles a feature-based optimisation like width, length, height,
etc., is used [100]. These vehicles belong to different classes, but
maybe they have the same colour. Apart from the dimension-based
feature, the texture-based feature is also used to improve the
classification accuracy of vehicles. With the fast progression of the
graphics processing unit, the computation capacity of handling
image has been extraordinarily improved which brought the quick
headway of DL. Looked at with customary feature extraction
calculation, DL has better flexibility and adaptability [100]. Till the
last decade, traditional methods are used in image processing for
classification of the vehicle which is histogram of gradient along
with support vector machine, but that required a lot of calculations.
Recently, more and more researchers are working on DL model for
the vehicle type classification and it becomes very successful in the
identification and recognition of vehicles.
Depending upon the classification of vehicle types, LeCun et al.
[101] gave the concept of the modern structure of the CNN
(LeNet-5) that was used for the classification of handwriting. After
that, many developments had done on CNN [102], Zhang et al.
used the concept of deep CNN [103, 104] which helped to identify
the vehicle of the regular image.
Apart from examining the procedure utilised for classification
of vehicles, feature-based classification (like size or structure) has
been discussed here. Since the last decade, there is a significant
change in the size or structure of two-wheelers. Now the
companies are designing the structure more stylish, robust, and
masculine. Bajaj Pulsar, Suzuki, Ducati, Honda, Harley-Davidson
are few of the companies where the sizes of two-wheelers have
© The Institution of Engineering and Technology 2019
Table 2 Various problems encountered
S. no. Various Brief description
problems
1. continuous change There are some cases when changes in background exist, but due to its relevance, it is considered as a background.
in background These movements can be regular or irregular depending upon its condition like the traffic light, day to night lightning
or vice-versa, waving trees, etc. Managing these continuous changes in the background is a challenging task.
2. shadowing Moving shadow is another problem on a bright sunny day, as one vehicle may cast its shadow on another vehicle.
This shadow may cause false detection of the vehicle size. So, this also becomes a perplexing task.
3. headlight refection Headlight reflection at night is another aspect causing fault detection when using the background differencing
approach for detection of vehicles.
4. camouflage Sometimes a vehicle can poorly differ from their background, making accurate classification difficult. This is most
important in surveillance applications where camouflage is difficult for temporal differencing methods.
5. clutter Task of segmentation is difficult in case of background clutter. Various models have been proposed that continuously
produces the clutter background and split up the moving vehicle in the foreground from the background.
6. motion of camera Unstable video captured when the camera is moving. Slight irregular movement magnitude differs from one video to
the other.
7. environment Moving vehicle detection becomes a very tough job when videos are taken in challenging weather conditions (snow,
storm, fog, cloudy, air turbulence, hazy environment, rainy, etc.).
8. video noise Noise is a completely unwanted signal and with video's, it is commonly superimposed. Background subtraction for
video surveillance must deal with such weak signals affected by various kinds of noise, such as sensor noise or
compression artefacts.

Fig. 11  Moving vehicles classification based on attributes derived from feature space optimisation

Table 3 Classification of vehicles based on their sizes The classification of vehicles based on their sizes includes
S. no. Type of vehicle Width, m Length, m Height, m width, length, and height of the vehicle is mentioned in Table 3.
1. two-wheeler 0.60–0.90 2.10–2.40 1.2–1.5 The approximate dimensions of all types of possible vehicles have
been included in the table. The features of these vehicles have been
2. three-wheeler 1.2–1.4 2.5–2.8 1.6–1.9
recognised and detected using any detection method [105].
3. light motor vehicle 1.3–1.6 3.2–3.8 1.4–1.7 Figs. 12a–f show the classification of vehicles in terms of size or
4. sports utility vehicle 1.6–1.8 3.8–4.1 1.6–1.8 structure.
5. light commercial vehicle 1.5–1.7 4.0–4.2 1.8–2.1 Vehicle type is notable in view of the characterised scope of
6. heavy motor vehicle 2.5–3.0 8.0–12.5 3.6–4.0 vehicle sizes. After the various methodologies applied for
classification of vehicles, the vehicle position in the video frame
sequences does not only depend on the vehicle size; rather it also
been increased significantly. Another segment of the classification depends upon what type of weather environment the vehicle is in.
of vehicle type is three-wheeler. Different types of auto-rickshaws
come under this category, which is used to carry passengers or 5 Hazy environment
goods for shorter distances. ‘Vikram’ from ‘Scooter India’ was a
very famous three-wheeler at one time, but now many companies Detecting a vehicle under challenging weather environment such as
are manufacturing three-wheelers that are small structure-wise and sunny, cloudy, snowy, rainy, hazy, etc., is a challenging job. Many
efficient for work. The light motor vehicle is a small vehicle used researchers have tried to eliminate the effect of bad light from the
to carry passengers that includes a jeep, motor cars, taxi, etc., video, to improve the quality of image sequences. However, these
whereas sports utility vehicle is a rugged vehicle that can carry methods do not detect the velocity and classification of the vehicle.
both passengers and cargo. Some of the sports utility vehicles As hazy weather conditions pose the most common problem in
which use this technique are Hummer, Ford Eco Sport, and Range surveillance systems, thereby reducing the clarity in traffic images
Rover, etc. Another segment is Light Commercial Vehicles, resulting in low accuracy of vehicle recognition and detection.
designed for low goods carriage. Piaggio Ape is one of the good There are various computational methods available for noisy
examples of this classification. Heavy motor vehicle includes images captured in hazy weather conditions and to overcome such
trucks, bus, etc. is bigger in all dimensions. challenges, some of them are discussed as below:

IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10 7

© The Institution of Engineering and Technology 2019
 The soft matting Laplacian matrix M is set by

1
∑ δi, j −
ωm
1 + (Ii − μm)T
m (i, j) ∈ ωm
−1 (9)
∈
+ Σm + U Ii − μm
ωm 3

where Σm is called the covariance matrix of the colours in frame or

window ωm, δi j is known as the Kronecker delta function, µm is the
mean matrix, U3 is known as an identity matrix of size 3 × 3, ɛ is a
normalising parameter, and |ωm| is the overall number of pixels in
frame or window ωm.
The distinguished transmission map t can be achieved by the
following linear system:
~
(M + λU)t = λt (10)

where M is called the soft matting Laplacian matrix, U is

considered as an identity matrix of the same size as M, and λ is
taken as 10−4.

5.4 Recovering the scene radiance

Fig. 12  Vehicle classification in terms of size or structure
At last, a single hazy image can be recovered as scene radiance K
(a) Two-wheeler, (b) Three-wheeler, (c) Light motor vehicle, (d) Sports utility vehicle,
as
(e) Light commercial vehicle, (f) Heavy motor vehicle

I f (m, n) − L f
5.1 Dark channel prior K f (m, n) = + Lf (11)
max (t(m, n), t0)
It is a technique of image restoration that removes hazy from a
single image [106, 107]. The estimation of hazy can be calculated where f∈{r, g, b}, the value of t0 is supposed to be 0.1. In the real
as input image, atmospheric light (L) is the highest intensity pixel as
per its correspondence in the dark channel to the brightest 0.1% of
K dark(m, n) = min min K f i, j (6) pixels.
i, j ∈ Ω m, n f ∈ r, g, b In addition to the above-discussed methods, another traditional
method for detection of moving the vehicle is GMM, but for the
Here ‘f’ belongs to red (r), green (g) and blue (b). K f symbolises a hazy environment [110] due to rain, dust, mist, snow, fog, etc., this
network of colour image K. A local patch denoted by Ω is centred model underperforms. To handle these kinds of challenges DL-
at around (m, n). based framework outperforms over GMM because of lower
The minimum operation min f ε r, g, b K f (i, j) is executed on K f computational cost and accuracy.
and min(i, j) ∈ Ω(m, n) is centred at (m, n), which is executed as a
minimum filter on the local patch. 6 Conclusion
When the outdoor image lacks hazy in that case, the dark Vehicle detection in a complex environment is a key task in an ITS.
channel has low intensity [108]. For a haze-free image, the value of The modern and futuristic practices of vehicle detection that are
the dark channel is approximately zero. For haze region, the dark used across the world have been reviewed in this paper. This paper
channel value is always greater than zero. is categorised into four sections, including the types of traffic
surveillance cameras, vehicle detection techniques, bases of vehicle
5.2 Transmission map estimation classification and detection of a vehicle in the hazy environment.
Among other cameras, Gatso and specs cameras are widely used in
The amount of hazy and transmission map is estimated under traffic surveillance because of their high resolution and accuracy.
normalised atmospheric light L f by dark channel prior as Moreover, the five major techniques used for vehicle detection are
given as (i) knowledge-based, (ii) motion-based, (iii) wavelet-
~ I f (i, j) based, (iv) ML, and (v) DL, which has been discussed briefly in
t (m, n) = 1 − min ( min (7)
(i, j) ∈ Ω(m, n) f ∈ r, g, b Lf this paper. A brief discussion on vehicle detection algorithms
shows the importance of DNN over existing traditional methods.
On the comprehensive removal of hazy, the image becomes Various challenges such as weather condition, varying illumination,
unnatural [108]. Thus, a constant parameter ω ∼0.95 is added to dynamic background scenes, etc. may affect the performance of the
retain a portion of the hazy for distant objects as vehicle detection algorithm. A brief overview is given for vehicle
classification based on a CNN. Finally, the computational methods
I f (i, j) that can be used for rectifying the faulty image detection under a
~
t (m, n) = 1 − ω min ( min (8) hazy environment have been discussed. Although lots of
i, j ε m, n f ∈ r, g, b Lf methodologies-based algorithms are available in the form of the
state-of-the-art for vehicle classification, detection, and tracking,
The dark channel prior to patch size 15 × 15 was proposed by He et still challenges arise due to various issues related to moving
al. [108]. vehicle dataset. Thus, leading the researchers to work on further for
the updating and enhancement of existing state-of-the-art.
5.3 Soft matting
After the estimation of the transmission map, the images recovered
contain certain block effects in a hazy image. Transmission map
has been improved by using soft matting [109], which was used by
Levin et al. to reduce the block effects.
8 IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10
© The Institution of Engineering and Technology 2019
7 References
[1] Buch, N., Velastin, S.A., Orwell, J.: ‘A review of computer vision techniques
for the analysis of urban traffic’, IEEE Trans. Intell. Transp. Syst., 2011, 12,
(3), pp. 920–939
[2] Zhang, J., Wang, F.Y., Wang, K., et al.: ‘Data-driven intelligent transportation
systems: a survey’, IEEE Trans. Intell. Transp. Syst., 2011, 12, (4), pp. 1624–
1639
[3] Sun, Z., Bebis, G., Miller, R.: ‘On-road vehicle detection: a review’, IEEE
Trans Pattern Anal. Mach. Intell., 2006, 28, (5), pp. 694–711
[4] Saran, K.B., Sreelekha, G.: ‘Traffic video surveillance: vehicle detection and
classification’, Int. Conf. on Control Communication and Computing, India,
November 2015, pp. 516–521
[5] Wang, G., Xiao, D., Gu, J.: ‘Review on vehicle detection based on video for
traffic surveillance’, IEEE Int. Conf. Automation and Logistics, Qingdao,
China, September 2008, pp. 2961–2966
[6] Chen, Z., Ellis, T., Velastin, S.A.: ‘Vehicle type categorization: a comparison
of classification schemes’. IEEE Conf. Intelligent Transportation Systems
Proc., ITSC, Washington, DC, USA, 2011, pp. 74–79
[7] Lai, A.S.H., Yung, N.H.C.: ‘Vehicle-type identification through automated
virtual loop assignment and block-based direction-biased motion estimation’,
IEEE Trans. Intell. Transp. Syst., 2000, 1, (2), pp. 86–97
[8] https://i.ytimg.com/vi/xVwsr9p3irA/maxresdefault.jpg
[9] Kim, J.B., Kim, H.J.: ‘Efficient region-based motion segmentation for a video
monitoring system’, Pattern Recognit. Lett., 2003, 24, (1–3), pp. 113–128
[10] Wu, B.F., Juang, J.H.: ‘Adaptive vehicle detector approach for complex
environments’, IEEE Trans. Intell. Transp. Syst., 2012, 13, (2), pp. 817–827
[11] Bishop, R.: ‘Intelligent vehicle applications worldwide’, IEEE Trans. Intell.
Transp. Syst., 2000, 15, (1), pp. 78–81
[12] Rai, M., Yadav, R.K.: ‘A novel method for detection and extraction of human
face for video surveillance applications’, Int. J. Signal Imaging Syst. Eng.,
2016, 9, (3), pp. 165–173
[13] Baran, R., Rusc, T., Fornalski, P.: ‘A smart camera for the surveillance of
vehicles in intelligent transportation systems’, Multimedia Tools Appl., 2016,
75, (17), pp. 10471–10493
[14] Deb, S.K., Nathr, R.K.: ‘Vehicle detection based on video for traffic
surveillance on road’, Int. J. Comput. Sci. Emerg. Technol., 2012, 3, (4), pp.
121–137
[15] Michalopoulos, P.G.: ‘Vehicle detection video through image processing: the
autoscope system’, IEEE Trans. Veh. Technol., 1991, 40, (1), pp. 21–29
[16] Dickmanns, E.D.: ‘The development of machine vision for road vehicles in
the last decade’, IEEE Intell. Veh. Symp. Proc., 2003, 1, pp. 268–281
[17] Sussman, J.M.: ‘Perspectives on intelligent transportation systems’, 2005
[18] Gupte, S., Masoud, O., Martin, R.F.K., et al.: ‘Detection and classification of
vehicles’, IEEE Trans. Intell. Transp. Syst., 2002, 3, (1), pp. 37–47
[19] Janowski, L., Kozłowski, P., Baran, R., et al.: ‘Quality assessment for a visual
and automatic license plate recognition’, Multimedia Tools Appl., 2014, 68,
(1), pp. 23–40
[20] Dule, E., Gokmen, M., Beratoglu, M.S.: ‘A convenient feature vector
construction for vehicle color recognition’. Proc. Int. Conf. Neural Network,
WSEAS, Lasi, Romania, 2010, pp. 250–255
[21] Dziech, W., Baran, R., Wiraszka, D.: ‘Signal compression based on zonal
selection methods’. Proc. of the Int. Conf. of Mathematical Methods in
Electromagnetic Theory, Kharkov, Ukraine, 2000, pp. 224–226
[22] Cao, M., Vu, A., Barth, M.J.: ‘A novel omni-directional vision sensing
technique for traffic surveillance’. IEEE Intelligent Transportation Systems
Conf., Seattle, USA, 2007, pp. 678–683
[23] Bertozzi, M., Broggi, A., Cellario, M.: ‘Artificial vision in road vehicles’,
Proc. IEEE, 2002, 90, (7), pp. 1258–1271
[24] Long, J., Shelhamer, E., Darrell, T.: ‘Fully convolutional networks for
semantic segmentation’. Proc. IEEE Conf. Computer Vision Pattern
Recognition, Boston, USA, 2015, pp. 3431–3440
[25] http://www.dailymail.co.uk/news/article-2930654/Know-enemy-
Incredibly-20-differentkindscameras-spying-motorists-spot-spotyou.html
[26] Rai, M., Maity, T., Yadav, R.K.: ‘Thermal imaging system and its real-time
applications: a survey’, J. Eng. Technol., 2017, 6, (2), pp. 290–303
[27] Rai, M., Husain, A.A., Maity, T., et al.: ‘Advance intelligent video
surveillance system (AIVSS): a future aspect’ (In Video Surveillance
IntechOpen, London, UK, 2018)
[28] Pang, C.C.C., Lam, W.W.L., Yung, N.H.C.: ‘A novel method for resolving
vehicle occlusion in a monocular traffic-image sequence’, IEEE Trans. Intell.
Transp. Syst., 2004, 5, (3), pp. 129–141
[29] Wang, Y.: ‘Joint random field model for all-weather moving vehicle
detection’, IEEE Trans. Image Process., 2010, 19, (9), pp. 2491–2501
[30] Liu, Y., Tian, B., Chen, S., et al.: ‘A survey of vision-based vehicle detection
and tracking techniques in ITS’. Proc. 2013 IEEE Int. Conf. on Vehicular
Electronics and Safety, Dongguan, China, 2013, pp. 72–77
[31] Zhang, J., Marszalek, M., Lazebnik, S., et al.: ‘Local features and kernels for
classification of texture and object categories: a comprehensive study’, Int. J.
Comput. Vis., 2007, 73, (2), pp. 213–238
[32] Bouwmans, T., Gonzalez, J., Shan, C., et al.: ‘Special issue on background
modelling for foreground detection in real-world dynamic scenes’, Mach. Vis.
Appl., 2014, 25, (5), pp. 1101–1103
[33] Siyal, M.Y.: ‘A neural vision-based approach for intelligent transportation
system’, IEEE ICIT’ 02, Bankok, Thailand, 2002, pp. 456–460
[34] Liu, Z., Huang, K., Tan, T.: ‘Cast shadow removal in a hierarchical manner
using MRF’, IEEE Trans. Circuits Syst. Video Technol., 2012, 22, (1), pp. 56–
66
[35] Cucchiara, R., Grana, C., Piccardi, M., et al.: ‘Improving shadow suppression
in moving object detection with HSV color information’. IEEE Proc. Int.
IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10
© The Institution of Engineering and Technology 2019
Conf. on Intelligent Transportation Systems, Oakland, USA, 2001, pp. 334–
339
[36] Xua, H., Xia, X., Guo, L., et al.: ‘A novel algorithm of moving cast shadow
suppression’. Proc. of the 18 Int. Conf. on Signal Processing, Beijing, China,
2006, pp. 1–4
[37] Yoneyama, A., Yeh, C.H., Kuo, C.C.J.: ‘Moving cast shadow elimination for
robust vehicle extraction based on 2D joint vehicle/shadow models’. Proc.
IEEE Conf. Advanced Video and Signal Based Surveillance, Miami, USA,
2003, pp. 229–236
[38] Iwasaki, Y., Kurogi, Y.: ‘Real-time robust vehicle detection through the same
algorithm both day and night’. Proc. of the Int. Conf. on Wavelet Analysis and
Pattern Recognition, Beijing, China, 2007, pp. 1008–1014
[39] Lou, J., Tan, T., Hu, W., et al.: ‘3-D model-based vehicle tracking’, IEEE
Trans. Image Process., 2005, 14, (10), pp. 1561–1569
[40] Kogut, G., Trivedi, M.: ‘A wide area tracking system for vision sensor
networks’. The 9th World Congress Intelligent Transport Systems, Chicago,
USA, 2002
[41] Lee, J.W., Kim, M.S., Kweon, I.S.: ‘A Kalman filter based visual tracking
algorithm for an object moving in 3-D’. Proc. Int. Conf. Intelligent Robots
and Systems, Pittsburgh, USA, 1995, pp. 355–358
[42] Costa, M.S., Shapiro, L.G.: ‘3-D object recognition and pose with relational
indexing’, Comput. Vis. Image Underst., 2000, 79, (3), pp. 64–407
[43] Tan, T.N., Baker, K.D.: ‘Efficient image gradient based vehicle localization’,
IEEE Trans. Image Process., 2000, 9, (11), pp. 1343–1356
[44] Muller, K., Smolic, A., Drose, M., et al.: ‘3-D construction of a dynamic
environment with a fully calibrated background for traffic scenes’, IEEE
Trans. Circuits Syst. Video Technol., 2005, 15, (4), pp. 538–549
[45] Cutler, R., Davis, L.S.: ‘Model-based object tracking in monocular image
sequences of road traffic scenes’, IEEE Trans. Pattern Anal. Mach. Intell.,
2000, 22, (8), pp. 781–796
[46] Ghosh, N., Bhanu, B.: ‘Incremental vehicle 3-D modeling from video’. Proc.
of the 18th Int. Conf. on Pattern Recognition, Hong Kong, China, 2006, pp.
272–275
[47] Meyer, D., Denzler, J., Niemann, H.: ‘Model based extraction of articulated
objects in image sequences for gait analysis’. Proc. IEEE Int. Conf. Image
Processing, Santa Barbara, USA, 1998, pp. 78–81
[48] Barron, J., Fleet, D., Beauchemin, S.: ‘Performance of optical flow
techniques’, Int. J. Comput. Vis., 1994, 12, (1), pp. 42–77
[49] Shaikh, S.H., Saeed, K., Chaki, N.: ‘Moving object detection using
background subtraction’ (Springer Briefs in Computer Science, New York,
2014)
[50] Chalidabhongse, T.H., Kim, K., Harwood, D.: ‘A perturbation method for
evaluating background subtraction algorithms’. Int. Workshop on Visual
Surveillance and Performance Evaluation of Tracking and Surveillance, Nice,
France, 2003
[51] Heikkila, J., Silven, O.: ‘A real-time system for monitoring of cyclists and
pedestrians’. Proc. of 2nd IEEE Workshop on Visual Surveillance, Fort
Collins, USA, 1999, pp. 74–81
[52] Cucchiara, R., Grana, C., Prati, A., et al.: ‘Probabilistic classification for
human behaviour analysis in transactions on systems’, Man Cybern., 2005,
35, pp. 42–54
[53] Benezeth, Y., Jodoin, P., Emile, B., et al.: ‘Comparative study of background
subtraction algorithms’, J. Electron. Imaging, Soc. Photo-Opt. Instrum. Eng.,
2010, 19, (3), pp. 1–12
[54] Stauffer, C., Grimson, W.E.L.: ‘Adaptive background mixture models for real-
time tracking’. Int. Conf. on Computer Vision and Pattern Recognition, Fort
Collins, USA, 1999
[55] Zivkovic, Z.: ‘Improved adaptive Gaussian mixture model for background
subtraction’. Int. Conf. on Pattern Recognition, Cambridge, UK, 2004
[56] Unzueta, L., Nieto, M., Cortes, A.: ‘Adaptive multi cue background
subtraction for robust vehicle counting and classification’, IEEE Trans. Intell.
Transp. Syst., 2012, 13, (2), pp. 527–540
[57] Yuan, W., Wang, J.: ‘Gaussian mixture model based on the number of moving
vehicle detection algorithm’. Proc. of 2012 IEEE Int. Conf. on Intelligent
Control Automatic Detection and High-End Equipment (ICADE), Beijing,
China, 2012, pp. 94–97
[58] Bouwmans, T., El-Baf, F., Vachon, B.: ‘Background modelling using mixture
of gaussians for foreground detection: a survey’, Recent Patents Comput. Sci.,
2008, 1, (3), pp. 219–237
[59] Jenifa, R., A., T, , Akila, C., et al.: ‘Rapid background subtraction from video
sequence’. IEEE Int. Conf. on Computing, Electronic and Electrical
Technologies (ICCEET), Kumaracoil, India, 2012, pp. 1077–1086
[60] Oliver, N.M., Rosario, B., Pentland, A.P.: ‘A Bayesian computer vision
system for modelling human interactions’, IEEE Trans. Pattern Anal. Mach.
Intell., 2000, 22, (8), pp. 831–843
[61] Bouwmans, T., Zahzah, E.: ‘Robust PCA via principal component pursuit: a
review for a comparative evaluation in video surveillance’, Computer Vision
and Image Understanding, 2014, 122, pp. 22–34
[62] Cutler, R., Davis, L.: ‘Robust real-time periodic motion detection, analysis
and applications’, IEEE Trans. Pattern Recognit. Mach. Intell., 2000, 13, pp.
129–155
[63] Wang, Y., K., Chen, S.: ‘A robust vehicle detection approach’. IEEE Conf. on
Advanced Video and Signal Based Surveillance, Tehran, Iran, 2005, pp. 117–
122
[64] Wang, X., Zhang, J.: ‘A traffic incident detection method based on wavelet
algorithm’. IEEE Workshop on Soft Computing in Industrial Applications,
Espoo, Finland, 2005, pp. 166–172
[65] Yin, M., Zhang, H., Meng, H., et al.: ‘An HMM based algorithm for vehicle
detection in congested traffic situation’. IEEE Intelligent Transportation
Systems Conf., Seattle, USA, 2007, pp. 736–741
9
[66] Fadlullah, Z.M., Tang, F., Mao, B., et al.: ‘State-of-the-art deep learning:
evolving machine intelligence toward tomorrow's intelligent network traffic
control systems’, IEEE Commun. Surv. Tutor., 2017, 19, (4), pp. 2432–2455
[67] Goodfellow, I., Bengio, Y., Courville, A.: ‘Deep learning’ (MIT Press,
Cambridge, Mass, USA, 2016)
[68] Carrio, A., Sampedro, C., Rodrigues-Ramos, A., et al.: ‘A review of deep
learning methods and applications for unmanned aerial vehicles’, J. Sens.,
2017, 1, pp. 1–13
[69] Wang, Z.: ‘The applications of deep learning on traffic identification’, 2016.
Available at https://www.blackhat.com/docs/us-15/materials/us-15-Wang-
The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf
[70] Liu, W., Zhang, M., Cai, Y.: ‘An ensemble deep learning method for vehicle
type classification on visual traffic surveillance sensors’, IEEE Access, 2017,
5, pp. 24417–24425
[71] Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘Imagenet classification with deep
convolutional neural networks’. Proc. of Advances in Neural Information
Processing Systems, Las Vegas, USA, 2012, pp. 1097–1105
[72] Girshick, R., Donahue, J., Darrell, T., et al.: ‘Rich feature hierarchies for
accurate object detection and semantic segmentation’. Proc. of the IEEE
Conf. on Computer Vision and Pattern Recognition, Columbus, USA, 2014,
pp. 580–587
[73] Karpathy, A., Toderici, G., Shetty, S., et al.: ‘Large-scale video classification
with convolutional neural networks’. Proc. of the IEEE Conf. on Computer
Vision and Pattern Recognition, Columbus, USA, 2014, pp. 1725–1732
[74] Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-
scale image recognition’, 2014. Available at https://arxiv.org/abs/1409.1556
[75] Szegedy, C., Liu, W., Jia, Y., et al.: ‘Going deeper with convolutions’. Proc.
of the IEEE Conf. on Computer Vision and Pattern Recognition, Boston,
USA, 2015, pp. 1–9
[76] He, K., Zhang, X., Ren, S., et al.: ‘Deep residual learning for image
recognition’. Proc. of the IEEE Conf. on Computer Vision and Pattern
Recognition, Las Vegas, USA, 2016, pp. 770–778
[77] Dosovitskiy, A., Fischer, P., Srringenberg, J.T., et al.: ‘Discriminative
unsupervised feature learning with convolutional neural networks’, CoRR,
2014, vol. abs/1406, no. 6909
[78] Kavukcuoglu, K., Sermanet, P., Boureau, Y., et al.: ‘Learning convolutional
feature hierarchies for visual recognition’, Adv. Neural. Inf. Process. Syst.,
2010, 23, pp. 1090–1098
[79] LeCun, Y., Boser, B., Denker, J.S., et al.: ‘Backpropagation applied to
handwritten Zip code recognition’, Neural Comput., 1989, 1, (4), pp. 541–551
[80] Freund, Y., Haussler, D.: ‘Unsupervised learning of distributions on binary
vectors using two layers networks’ Proc. 4th Int. Conf. on Neural Information
Processing Systems (NIPS'91), Denver, USA, 1991, pp. 912–919
[81] Sutskever, I., Martens, J., Hinton, G.: ‘Generating text with recurrent neural
networks’. Proc. of the 28th Int. Conf. on Machine Learning (ICML 11),
Bellevue, USA, 2011, pp. 1017–1024
[82] Liou, C.Y., Huang, J.C., Yang, W.C.: ‘Modeling word perception using the
Elman network’, Neurocomputing, 2008, 71, (1618), pp. 3150–3157
[83] Bourlard, H., Kamp, Y.: ‘Auto-association by multilayer perceptrons and
singular value decomposition’, Biol. Cybern., 1988, 59, (4), pp. 291–294
[84] Socher, R., Bengio, Y., Manning, C.: ‘Deep learning for NLP (without
magic)’. Tutorial Abstracts of ACL, Atlanta, USA, 2012, pp. 5–5
[85] Zhou, Y., Nejati, H., Do, T.T., et al.: ‘Image-based vehicle analysis using deep
neural network: A systematic study’. IEEE Int. Conf. on Digital Signal
Processing, Beijing, China, 2016
[86] Schmidhuber, J.: ‘Deep learning in neural networks: an overview’, Neural
Netw., 2015, 61, pp. 85–117
[87] Luckow, A., Cook, M., Ashcraft, N., et al.: ‘Deep learning in the automotive
industry: applications and tools’, CoRR, 2017, vol. abs/1705.00346
[88] Suhao, L., Jinzhao, L., Guoquan, L., et al.: ‘Vehicle type detection based on
deep learning in traffic scene’, Procedia Comput. Sci., 2018, 131, pp. 564–
572
10
[89] Redmon, J., Divvala, S., Girshick, R., et al.: ‘You only Look once: unified,
real-time object detection’. IEEE Conf. on CVPR, Las Vegas, USA, 2016
[90] Sermanet, P, Eigen, D, Zhang, X, et al.: ‘Overfeat: integrated recognition,
localization and detection using convolutional networks’. Advances in Neural
Information Processing Systems [S.1]: ICLR Press, Banff, Canada, 2014, pp.
1055–1061
[91] Russakovsky, O., Deng, J., Su, H., et al.: ‘Imagenet large scale visual
recognition challenge’, IJCV, 2015, 115, (3), pp. 211–252
[92] Cordts, M., Omran, M., Ramos, S., et al.: ‘The cityscapes dataset’. CVPR
Workshop on the Future of Datasets in Vision, Las Vegas, USA, 2015
[93] Cordts, M., Omran, M., Ramos, S., et al.: ‘The cityscapes dataset for semantic
urban scene understanding’. Proc. of the IEEE Conf. on Computer Vision and
Pattern Recognition (CVPR), Las Vegas, USA, 2016
[94] Lin, T. Y., Maire, M., Belongie, S., et al.: ‘Microsoft COCO: common objects
in context’, ECCV, 1, 4, 5, 7, 2014
[95] Geiger, A., Lenz, P., Urtasun, R.: ‘Are We ready for autonomous driving?’.
The KITTI Vision Benchmark Suite, Conf. on Computer Vision and Pattern
Recognition (CVPR), Providence, USA, 2012
[96] Geiger, A., Lenz, P., Stiller, C., et al.: ‘Vision meets robotics: the KITTI
dataset’, Int. J. Robot. Res. (IJRR), 2013, 32, (11), pp. 1231–1237
[97] Brostow, G. J., Fauqueur, J., Cipolla, R.: ‘Semantic object classes in video: A
high-definition ground truth database’, Pattern Recognit. Lett., 2009, 30, (2),
pp. 88–97
[98] Maddalena, L., Petrosino, A.: ‘A self organizing approach to background
subtraction for visual surveillance applications’, IEEE Trans. Image Process.,
2008, 17, (7), pp. 1168–1177
[99] Lai, A.H.S., Fung, G.S.K., Yung, N.H.C.: ‘Vehicle type classification from
visual-based dimension estimation’. Proc. of the IEEE Intelligent
Transportation Systems Conf., Oakland, USA, 2001, pp. 201–206
[100] Mithun, N.C., Rashid, N.U., Rahman, S.M.: ‘Detection and classification of
vehicles from video using multiple time-spatial images’, IEEE Trans. Intell.
Transp. Syst., 2012, 13, (3), pp. 1215–1225
[101] LeCun, Y., Bottou, L., Orr, G.B., et al.: ‘Efficient backpropagation in neural
networks’ (Tricks of the Trade, Springer, 1998), pp. 9–50
[102] Dong, Z., Wu, Y., Pei, M., et al.: ‘Vehicle type classification using a
semisupervised convolutional neural network’, IEEE Trans. Intell. Transp.
Syst., 2015, 16, (4), pp. 2247–2256
[103] Zhang, F.: ‘Car detection and vehicle type classification based on deep
learning’ (Jiangsu University, China, 2016)
[104] Zhang, F., Xu, X, Qiao, Y.: ‘Deep classification of vehicle makers and
models: the effectiveness of Pre-training and data enhancement’. IEEE Int.
Conf. on Robotics and Biomimetics (ROBIO), Zhuhai, China, 2015, pp. 231–
236
[105] Ambardekar, A., Nicolescu, M.: ‘Vehicle classification framework: a
comparative study’, EURASIP J. Image Video Process., 2014, 29, pp. 1–13
[106] Huang, S.C., Chen, B.H., Cheng, Y.J.: ‘An efficient visibility enhancement
algorithm for road scenes captured by intelligent transportation systems’,
IEEE Trans. Intell. Transp. Syst., 2014, 15, (5), pp. 2321–2332
[107] Huang, S.C., Chen, B.H., Wang, W.J.: ‘Visibility restoration of single hazy
images captured in real-world weather conditions’, IEEE Trans. Circuits Syst.
Video Technol., 2014, 24, (10), pp. 1814–1824
[108] He, K., Sun, J., Tang, X.: ‘Single image haze removal using dark channel
prior’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (12), pp. 2341–
2353
[109] Levin, A., Lischinski, D., Weiss, Y.: ‘A closed form solution to natural image
matting’. Proc. of the IEEE Conf. on Computer Vision and Pattern
Recognition, New York, USA, 2006, pp. 61–68
[110] Li, S., Ren, W., Zhang, J., et al.: ‘Fast single image rain removal via a deep
decomposition-composition network’, Comput. Vis. Pattern Recognit.
(CVPR), 2018, 186, pp. 48–57
IET Image Process., 2020, Vol. 14 Iss. 1, pp. 1-10
© The Institution of Engineering and Technology 2019