2017 6th MEDITERRANEAN CONFERENCE ON EMBEDDED COMPUTING
Real-Time Stopped Vehicle Detection
Based on Smart Camera
Boris A. Alpatov, Maksim D. Ershov
Department of Automation and Information Technologies in Control
Ryazan State Radio Engineering University (RSREU)
Ryazan, Russia
aitu@rsreu.ru
Abstract—The paper describes traffic control and road safety
problems and their solutions using image processing algorithms.
We have developed an algorithm to detect stopped vehicles
during traffic surveillance. Our algorithm is based on
background estimation in image sequences. This paper offers
some features of algorithm implementation in smart cameras and
experimental results.
Keywords-traffic surveillance; video analysis; background
subtraction; embedded systems; smart cameras
I. INTRODUCTION
Currently, efficient road traffic control and road safety of
all road users require traffic surveillance, estimation of traffic
stream parameters, immediate detection of dangerous situations
and their reporting to appropriate services [1].
Not so long ago such important and time-consuming tasks
were performed by people – controllers in the traffic control
centers, but now traffic control systems must be automated due
to the constant increase in the number of monitored road
sections.
Video detectors are more and more widely used to analyze
traffic stream [2]. Their advantages are:
x Vehicle detection on several traffic lanes with one
sensor.
x Collecting a large number of various traffic stream
parameters.
x Visual monitoring of vehicles.
If the data source is CCTV camera, there are two
implementations of data processing algorithms.
x Online processing is performed on the embedded
platform of CCTV camera and provides reducing the
amount of transmitted data and thus reducing the
requirements for communication links and computing
server [3].
x Offline processing is performed on the central
computing server and provides a significant reduction
in the requirements for CCTV camera specification [4].
The work has been supported by the grant for the Leading Scientific Schools
of the Russian Federation (NSh-7116.2016.8).
978-1-5090-6742-8/17/$31.00 ©2017 IEEE
(MECO), 11-15 JUNE 2017, BAR, MONTENEGRO
This paper describes the solution for traffic control system
– stopped vehicle detection. If a vehicle stops on the road, it
may be an obstruction for traffic and pose a serious threat. In
this case, a timely detection of such situation allows to warn
other road users and inform the emergency services in time.
Our stopped vehicle detection algorithm is designed for use
on the embedded platform of smart CCTV cameras. Thus, we
reduce the amount of data transmitted to the traffic control
system. However, it must be remembered that the computing
capability of such cameras is limited, and the following factors
should be taken into the account during image processing:
changing the weather and light conditions, different camera
angles and positions.
II. STOPPED VEHICLE DETECTION ALGORITHM
Stopped vehicle detection algorithm is based on
background subtraction technique [5]. Algorithm work can be
divided into the initialization stage and the stage of stopped
object detection.
Initialization stage is initial background estimation. For this
purpose, an averaging filter is applied: the average value of
brightness for a given number of frames is determined in each
pixel of the image [6]. Thus, the initial background estimation
is done by accumulating the information about the pixel
brightness for a sufficiently long period of time.
When detection algorithm is based on background
subtraction, one of the tasks is to update the background
estimation. To do so, our algorithm uses an exponential
filter [7]. With background estimation done, object detection
can be carried out by analyzing the module of the difference
between the current image and the background.
Our approach is based on a current background estimation
BGN and a queue of background estimations for various short
time intervals BGi (i = 0..N–1). Then we subtract the earliest
background estimation from the current background estimation
for stopped object detection.
Thus, the stopped object detection stage comprises the
following steps.
x Getting matrix BD by binarizing the differential image
D = |BGN – BG0|.
 2017 6th MEDITERRANEAN CONFERENCE ON EMBEDDED COMPUTING
x Processing of binary image BD.
x Information accumulation in matrix S about the
duration of assigning each pixel to the object.
x Getting matrix BS by binarizing matrix S.
x Objects marking and parameterization.
x Update of background estimations.
D is the module of the difference between the current
background estimation BGN and the background estimation for
the earliest period of time BG0. Binarization of D is carried out
by thresholding.
Next step is binary image BD postprocessing in order to
connect small segments. To do so, morphological operations of
opening and closing are applied. It is also possible to use a
predetermined mask for zeroing pixel values in the image BD in
areas not belonging to the roadway.
Matrix S is created in order to account for the temporary
detection threshold. It stores for each pixel the number of
frames in which this pixel is classified as belonging to a
stopped object. Update of the matrix S is performed for each
frame and based on the information in the image BD.
Binary image BS is the result of the matrix S thresholding.
The threshold is set according to required detection time.
Finally, marking and parametrization of selected objects is
performed basing on image BS. During the procedure, each
object is assigned with a unique number, coordinates and area
are calculated. Too small objects are rejected.
To update the current background estimation BGN, the
exponential filter input is supplied with current image IN. As a
result, BGN will accumulate information about stopped objects.
Update of earlier background estimations BGi (i = 0..N–1)
is based on the mask, which is binary image BD. For pixels that
belong to stopped objects, the brightness value is averaged over
all background estimations BGi (i = 0..N–1). After a
predetermined short period of time, background estimation
queue shifts, the earliest estimation is rejected (BG0 = BG1, …,
BGN–1 = BGN).
III. EMBEDDED IMPLEMENTATION OF ALGORITHM
This section describes AXIS products with embedded video
processing, as well as provided tools to develop applications
for video analytics. It will also consider the implementation of
the developed stopped vehicle detection algorithm.
A. Hardware Specification
The final implementation of the algorithm is set to work on
the smart camera embedded platform. We used network
cameras AXIS P1365 [8] and video encoders AXIS Q7424-R
[9] in our research.
These products enable urban surveillance and infrastructure
monitoring in different conditions.
(MECO), 11-15 JUNE 2017, BAR, MONTENEGRO
Smart camera AXIS P1365 delivers up to 60 fps in HDTV
1080p (up to 1920×1200 pixels). It provides noise reduction
and reduced motion blur in low light. The camera is based on
ARTPEC-5 platform, has 512 MB RAM, 256 MB Flash.
Video encoder AXIS Q7424-R features a rugged design
optimized to withstand vibrations, shocks, and extreme
temperatures. Video encoder delivers up to 30 fps in resolution
up to 720×576 pixels. The encoder can connect up to 4 analog
video cameras, but the embedded video processing is supported
only for one link. The encoder is based on ARTPEC-4
platform, has 512 MB RAM, 128 MB Flash.
B. Development Tools
Software implementation of the proposed algorithm is
written in C/C++ programming language. Video analytics
application for devices described above were developed using a
special software development kit (SDK) – AXIS Camera
Application Platform (ACAP) [10].
ACAP includes documentation, application programming
interface (API), sample projects and related compilers.
Applications on this platform are written in C and/or C++
programming languages, they are based on an operating system
with the Linux kernel and the GNU system libraries. Package
gdbserver enables remote debugging of applications installed
on the smart camera.
ACAP also contains the following third-party software
libraries.
x RAster Processing Primitives library (RAPP) [11]. The
library for image processing focused on computer
vision and video analytics applications. It contains the
efficient implementation of algorithms such as simple
thresholding, binary morphology, convolution,
geometric transformations, etc.
x Fixed Point library (Fixmath) [12]. The library for
working with fixed point numbers. On considered
platform, multiplication of fixed-point numbers is
performed two times faster, and division and algebraic
functions are 10-50 times faster than corresponding
operations with floating point numbers.
C. Developed Software
The developed software package consists of the following
components.
x Video stream analysis module using an embedded
processor of the camera.
x Data exchange module based on TCP/IP protocol using
an embedded processor of the camera.
x Configuration and results display module, which is a
stand-alone application written in C++ for Windows
operating system.
The video stream analysis module performs real-time video
processing basing on the stopped vehicle detection algorithm.
 2017 6th MEDITERRANEAN CONFERENCE ON EMBEDDED COMPUTING (MECO), 11-15 JUNE 2017, BAR, MONTENEGRO

The data exchange module is a TCP server, which saves

and delivers algorithm configuration, delivers the results (the
position of stopped vehicles).
The configuration and results display module has a
graphical user interface. It allows user to configure the video
stream analysis module, to receive the current image from the
camera and mark detected stopped vehicles. To get the
displayed information, the module refers to the data exchange
module located on the camera.
To perform real-time video processing, the image size is
reduced to 480×320 pixels (the original 1920×1200) and to
360×288 pixels (the original 720×576).
In addition, to perform the background estimation, the data
is presented as fixed-point numbers, because operations on
floating-point numbers are significantly more time-consuming
when running on the embedded processor of considered smart Figure 1. Detection of 3 stopped vehicles.
cameras. Wrapper class for Fixmath library functions enables
easy switching between different types of data.
IV. EXPERIMENTAL RESULTS
Experimental researches of the developed algorithm and
software were conducted with previously recorded actual
videos. Also, the algorithm has been tested on CCTV cameras
when surveilling different sections of the roads in real time.
The video monitoring was done under different weather
and light conditions. The length of video sequences is from 10
minutes to 8 hours. The size of processed images is from
360×288 to 1280×720 pixels.
When using the least productive platform (AXIS Q7424-R
on ARTPEC-4), processing speed was about 5 fps. Considering
the specifics of the task, this speed was sufficient for the timely
detection of stopped vehicles.
Results of experimental researches for different video Figure 2. Detection of 2 stopped vehicles.
sequences are shown in Table I. We use following notations.
x TP (True Positive) – true detection of a stopped TABLE I. RESULTS OF STOPPED VEHICLE DETECTION
vehicle.
Test Results
x FN (False Negative) – not detected stopped vehicle video TP FN FP
(skip of the object). 1 4 0 0
x FP (False Positive) – false detection of a stopped 2 3 0 0
vehicle (false alarm).
3 4 0 0
Thus, the true detection rate (ratio of the number of all
4 2 0 0
stopped objects to TP) is 100% at the error rate (ratio of the
number of all stopped objects to the sum of FN and FP) – 5 2 0 0
7.5%.
6 1 0 1
Fig. 1 presents an example of stopped vehicle detection on 7 1 0 0
a sunny day (stop duration – more than 1 minute). Also, the
lighting conditions on this video sequence change over time 8 1 0 0
due to the presence of clouds. 9 1 0 0
Fig. 2 presents an example of stopped vehicle detection in 10 3 0 0
the dark under the street lights (stop duration – more than 10
seconds). 11 22 0 1
 2017 6th MEDITERRANEAN CONFERENCE ON EMBEDDED COMPUTING (MECO), 11-15 JUNE 2017, BAR, MONTENEGRO

Test Results REFERENCES

video TP FN FP [1] V. M. Markuts, “Transport flows of highways and the city streets
12 27 0 0 (practical applications),” Tyumen, 2008, 108 p., in Russian.
[2] B. Coifmana, D. Beymer, P. McLauchlan, and J. Malik, “A real-time
13 8 0 0 computer vision system for vehicle tracking and traffic surveillance,”
Transportation Research Part C 6, 1998, pp. 271–288.
14 14 0 2
[3] B. Alpatov, P. Babayan, M. Ershov, and V. Strotov, “The
15 0 0 1 implementation of contour-based object orientation estimation algorithm
in FPGA-based on-board vision system,” Proc. of SPIE, vol. 10007,
16 0 0 0 100070A, pp. 1–8, October 2016.
17 0 0 2 [4] M. D. Ershov, “Application of image processing algorithms for traffic
stream parameters estimation,” Proc. of New information technologies
18 0 0 0 in scientific researches, Ryazan: RSREU, 2016, pp. 259–261, in
Russian.
V. CONCLUSION [5] Y. Benezeth, B. Emile, H. Laurent, and C. Rosenberger, “Review and
evaluation of commonly-implemented background subtraction
In the paper we described one of the tasks of traffic algorithms,” International Conference on Pattern Recognition, pp. 1–4,
surveillance – stopped vehicle detection. We introduced an December 2008.
algorithm to process video sequences from optical sensor [6] M. Piccardi, “Background subtraction techniques: a review,” Proc. of
IEEE International Conference on Systems, Man and Cybernetics,
performing traffic surveillance. The algorithm is based on pp. 3099–3104, 2004.
background subtraction and designed to detect and localize
[7] J. Heikkila, O. Silven, “A real-time system for monitoring of cyclists
stopped vehicles. and pedestrians,” 2nd IEEE Workshop on Visual Surveillance, pp. 74–
81, 1999.
Our algorithm is implemented on smart cameras. We have
considered the implementation based on embedded platform of [8] AXIS P1365 network camera. Support & Documentation,
https://www.axis.com/global/en/products/axis-p1365-mki/support-and-
AXIS cameras. documentation.
Experiments have confirmed the efficiency of stopped [9] AXIS Q7424-R video encoder. Support & Documentation,
vehicle detection algorithm and developed software. After the https://www.axis.com/global/en/products/axis-q7424-r-mki/support-and-
documentation.
analysis of the results, we can conclude that the developed
[10] AXIS Camera Application Platform, https://www.axis.com/global/en/
software package allows solving the considered problem in support/developer-support/axis-camera-application-platform.
real-time under different surveillance conditions. [11] RAPP library. Documentation, http://www.nongnu.org/rapp/doc/rapp.
[12] Fixmath library. Documentation, http://www.nongnu.org/fixmath/doc.