Traffic analysis of video surveillance using Faster-

RCNN
Imad EL MALLAHI Adnane Mohammed Mahrez Mohamed Adnane Mahraz
dept. of computer sciences dept. of computer sciences dept. of computer sciences
Sidi Mohemmed Ben Abdellah Sidi Mohemmed Ben Abdellah Sidi Mohemmed Ben Abdellah
University, faculty of sciences Dhar El University, faculty of sciences Dhar El University, faculty of sciences Dhar El
mahraz mahraz mahraz
Fez, Morocco Fez, Morocco Fez, Morocco
imade.elmallahi@usmba.ac.ma adnane.mehrez@usmba.ac.ma adnane_1@yahoo.fr

Hamid TAIRI
dept. of computer sciences
Sidi Mohemmed Ben Abdellah
University, faculty of sciences Dhar El
mahraz
Fez, Morocco
hamid.tairi@usmba.ac.ma

Abstract— This work focusses the issue of traffic analysis of
video surveillance using Faster-RCNN. The challenge here is
expressed as classifying and counting vehicles in Traffic. To
improve this proposed issue, we used the related work of
Faster R-CNN and detector vehicles. For the classification in
live traffic movement. The performance of the Faster R-CNN
was improved by many restrictions such as adaptive feature
pooling, anchors optimization, focal loss, and other mask
branch. For training the proposed model, we used UA-
DETRAC dataset of 10 hours of videos at 25 frames per
seconds (fps), with resolution of 960×540 pixels, this dataset
have more than 140 k frames in the UA-DETRAC dataset and
8250 vehicles that are manually annotated with DarkLabel.
Keywords— Video surveillance, Traffic estimation, Faster R-
CNN, Traffic detection, Convolutional neural network,
Surveillance Vehicle.
I. INTRODUCTION
Now, most cities in the world have many video
surveillance systems. They have fast growth, now they have
heterogeneous cameras with various resolutions [1]. today,
the Closed-circuit television works all times the week and
generates a huge amount of data, called Big and huge Data of
videos. Moreover, this video data can be used as a
foundation for the automated vehicle surveillance system.
There are many issues when working with Big Data traffic
surveillance. To realize an intelligent system of Traffic
vehicle surveillance, we have an efficient hard disk system
Frame Conv
Extracti Feature
layers Maps
ons
Input video
for storing, and moving in forward and back, and analyzing
videos. In this work, we concern the analyze videos for
Traffic surveillance, this research remains limited in terms of
real-time data analysis.
Some of the representative papers in this theme, which
used heterogeneous low-resolution data to estimate traffic
density and count vehicles.
many efforts have been made to improve vehicle
analysis, counting, classification and detection, satisfactory
results in specific tasks.
Most of the recent studies focus on adapting and
improving related work in vehicle frameworks such as
recurrent convolutional neural networks and SSD and YOLO
detectors. This includes new contributions at the architectural
level, ameliorating speed level, classification, and tracking
vehicle. In this study, we focused on a gap between recent
works and their applications in the real world. We began to
adopt the model of Faster R-CNN and improved the
sequential baseline performance via an auxiliary mask, next,
we have added the anchor's optimization, then we calculate
the loss and pooling. The proposed can be used to analyze,
and be count the traffic movement, the figure 1 represents the
Flow shart of proposed video surveillance using Faster-
RCNN . The experiments will be used by city officials to
improve the overall throughput of the intersection.
Region
Proposal
Network
Classification
counting vehicles
Region of
interest Final detection
pooling

Figure 1: Flowchart of proposed video surveillance using Faster-RCNN

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

II. CLASSIFICATION AND RECOGNITION OF TRAFFIC VIDEO
A. Related Works
The classification and detection the moving objects in
videos is a significant issue to resolve in building autonomus
video surveillance. Some algorithms have been implemented
to aid resolve this issue, many methodes for classifying and
detecting objects in videos such as background
subtraction.The differents methods to resolve this issue with
own designing traffic video analytic resolutions should
consider the type of algorithm implementation employed to
good fit the task at hand.
B. Objects detection in video surveillance
The fast convolutional neural networks (FCNN) for
object detection in video surveillance, it divided into two
principal’s classes: the first is single phase detectors and
second phase detectors. The single-phase detectors are
usually very fast and can be predict objects in video
bounding boxes with classes within a simple, and single
network. Conventional examples of the first and single-stage
detectors are SSD [2], and YOLO [ 3 ]. These methods can
be use particularly in cases when goal objects occupy a
considerable amount of global image. Some recent examples
of this methods using UA-DETRAC vehicle detection
dataset [4]. Moreover, this data has been used by Dmitriy
Anisimov and Tatiana Khanova [5] to constructed SSD-like
detector well be run for 40 frames per second while
maintaining favorable precision. In addition, the more
application are used of good speed-precision trade-of is
version 2 for YOLO method [6], the particular of vehicle
detection via anchors clustering, multi-layer feature fusion
strategy, and loss normalization. The R-CNN family of
detectors is used for the many representative method of two-
Figure 2: Some examples cameras of UA-DETRAC dataset
stage detectors that currently occupy leading places in
Cityscapes [7], the COCO [8], and benchmark datasets.
The comparaison between the first stage and second stage
detectors began with the prediction of regions and next
classify and refine each of them during the two stage.
The first paper of R-CNN [9] was employed a simple
approach: region was generated with many selective
researchs algorithms and next implemented the classification
of CNN. The global vitesse of the R-CNN is feeble due to
the selective research compute time and requirement to
execut classifier for each region. To resolve this issue, we
suggest a Fast R-CNN [10]. Instead of executiogn ing CNN
for each region. Modifing selective research in the proposed
Faster R-CNN [11] with CNN, called region suggested
network, also boosted accuracy, and vetesse of the detector
III. GOAL AND METHODOLOGY OF RESEARCH
The main goal, we are divide the issue in three sub
stages, first car detection, Next car tracking, and finally car
orientation estimation.
A. Dataset of detection the video surveillance
In this application, the number of surveillance cameras is
more than forty surveillance cameras, the most of
surveillance cameras are fixed above the crossroads to give a
global overview of the traffic situation in the boulevards the
city. In this dataset the most of camers have twenty-five
frames per second, with resolution of 960×540. Morever, the
streams of these videos are not very good due to firstly
hardware faults, then blurring, and finally compression
artefacts. The figure 1 represents the video from the one of
these cameras, and the figure 2 show the images from these
cameras of dataset.
 Figure 3: Frame in the UA-DETRAC dataset
we annotated these data with many polygons more than sixty
thousand instances across nine hundred eighty two frames
such as UA-DETRAC [12] or KITTI [13] . The from the chosen camera. All annotation relevant satage were
dissimilarity comes from multitude aspects:firstlly with the provided with the tool of COCO Annotator [14]. It is very
number of instances on a single frame, next with high difficult to annotate video sequences, because they have
variations in scale, then with viewing angle, and finally of many time consuming, also there are crowded scenes.
occlusions level. The challenge od this dataset resides even a Instead, we concentrate firstly on traffic situations, on
human eye is not able of labelling each instance on the of weather conditions, on time of day. In this paper, we also
image all time, figure 3 shows the frame in the UA- annotated every single vehicle that have distinguished with
DETRAC dataset. In this paper, we concetrate on a single big confidence, particularly in crowded traffic jams. the
camera that monitors one of the many problematic Table 1 give summarize the distribution of dataset.
crossroads. To maximize detection precision, and get a
minimum viable product for a deliberate choice. In this work,
TABLE 1 DISTRIBUTION OF DATASET
Mean instance per
Type of vehicle Number of instances
frame
Bus 1234 1.26
Truck 2415 2.46

Car 53083 4.06

Trolleybus 611 0.62

TRAM 1298 1298

VAN 2783 2.83

B. Faster R-CNN detector for detect the module
The base of our detection architecture is a widely
adopted two step Faster R-CNN [15] detector. The
main reasons for this choice. At first, two step detectors
are current related work in many detection benchmarks
[16-18 ] and have a good start point in all detection
task. Then, the next work may provided complex multi-
task learning. The extenction of Faster R-CNN model
for multiple stages is as easy as adding a new
prediction.
 We also improved Faster R-CNN baseline quality
with a caracteristic pyramid network (FPN) backbone
[19], a suplimentary mask branch has added [20], Then,
the some anchors shape optimization, after that the
focal loss [21], and finally adaptive caracteristic
pooling [22].
C. Mask branch of R-CNN
The provided in the Mask R-CNN work [23],
supplementary regression of the per-instance masks leads to
good accuracy. Thus, the primary optimization that we used
a additional mask branch. This branch executes in parallel
with the current Faster R-CNN branches and target to regress
a binary mask for every region of interest.
D. Optimization of anchors shape
This amelioration from the easy observation that the Faster
R-CNN anchors parameterization was provided for the
COCO dataset [ 20], but our suggested dataset is different.
IV. EXPIRIMENTATION
A. Numerical Result and experimentations
In this experimentation, we carried out on a HP EliteBook
x460 1130 G5 -Windows 10 -based on python 3.8
environment, which an Intel Core i5 - 8400 2.80 Giga Hz
CPU and Nvidia GeForce RTX 2160 GPU with eight Giga
Byte in memory, employing Darknet-53 as the convolutional
neural network on deep learning framework, with backbone
for YOLOv3 object detection in this approach. The chose of
the parameters for this algorithm are began with learning rate
to 0.0001, next we adopted the step of the learning strategy.
For the optimization, we adjust the parameters of the
architecture, we based on Stochastic Gradient Descent
(SGD) with a momentum of 0.9. In all experiments, and to
avoid the overfitting for this model, we used a weight decay
of 0.0005. Moreover, to train 40,000 batches, we employ the
same top parameters. To reduce the learning rate to 1/10 of
the initial value, we use also the 36,000 th and 46,000 th
batches of this architecture.
B. Comparisons the result after and before detection
of YOLO
As shown in Table 2, the recal and precesion is much
higher than Means No data Augmentation. To increase the
computational cost, we added an additional layer at the
output of the model, but the numerical results which are
made up of recall, and precision have the most interest on the
sensitivity calculation, and in Table 3, the recal and
precesion is much higher than YOLO.
Table 2: Numerical result of Means No
Augmentation and Video Corruption
Augmentation True False False Recal Precision
of data Positive Positive Negative
Video 3254 623 624 0.76 0.71
Corruption
Means No 1423 612 516 0.74 0.69
data
Augmentation
Table 3: Comparaison of YOLO and Proposed method
Detection Algorithm Recal Precision
Proposed method 0.80 0.79
YOLO 0.78 0.77
V. CONCLUSION
In this paper, we focussed on the issue of traffic analysis
of video surveillance using Faster-RCNN. We expressed as
classifying and counting vehicles in Traffic. To improve this
proposed issue, we used the related work of Faster R-CNN
both and detector vehicles. The performance of the Faster R-
CNN was improved by many restrictions such as adaptive
feature pooling, anchors optimization, focal loss, and other
mask branch. For training the proposed model, we used UA-
DETRAC dataset of 10 hours of videos at 25 frames per
seconds (fps), with resolution of 960×540 pixels, this dataset
have more than 140 k frames in the UA-DETRAC dataset
and 8250 vehicles that are manually annotated with
DarkLabel
REFERENCES
[1] Zhang W, et all. Table of contents. IEEE conference on computer
vision and pattern recognition. volum. 2017-Jan, 2017.
[2] Liu W, et all. SSD: single shot multibox detector. computer science
(including subseries lecture notes in artificial intelligence and lecture
notes in bioinformatics) . pp. 21–37, 2016.
[3] Redmon J, et all. You only look once: unified, real-time object
detection. IEEE conference on computer vision and pattern
recognition . 2016, p. 779–88.
[4] Wen L et all. UA-DETRAC: a new benchmark and protocol for
multi-object detection and tracking. 2015.
[5] Anisimov D, et all. Towards lightweight convolutional neural
networks for object detection. In: 2017 14 th IEEE international
conference on advanced video and signal based surveillance (AVSS).
2017, p. 1–8.
[6] Sang J, et all. An improved YOLOv2 for vehicle detection. Sensors.
2018;18(12):4272.
[7] Cordts M, et all. The cityscapes dataset for semantic urban scene
understanding. In: 2016 IEEE conference on computer vision and
pattern recognition (CVPR). 2016, p. 3213–23.
[8] Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár
P, Zitnick CL. Microsoft COCO: common objects in context. In:
data Lecture notes in computer science (including subseries lecture notes
in artificial intelligence and lecture notes in bioinformatics). vol. 8693
LNCS, 2014, p. 740–55.
[9] Girshick R, et all. Rich feature hierarchies for accu- rate object
detection and semantic segmentation. In: Proceedings of the IEEE
conference on computer vision and pattern recognition. 2012.
[10] Girshick R, et all. Fast R-CNN. In: Proceedings of the IEEE
international conference on computer vision 2015 Inter, 2015,
p.1440–8.
[11] Ren S, et all. Faster R-CNN: towards real-time object detection with
region proposal networks. IEEE Trans Pattern Anal Mach Intell.
2017;39(6):1137–49.
[12] Wen L, et all. UA-DETRAC: a new benchmark and protocol for
multi-object detection and tracking. 2015
[13] Geiger A, et all. Vision meets robotics: the KITTI dataset. Int J Rob
Res. 2013;32(11):1231–7
[14] Brooks J, et all. COCO annotator. 2019, https ://githu b.com/jsbro
ks/coco-annotator/.
[15] Redmon J, et all. YOLO9000: Better, faster, stronger. IEEE
conference on computer vision and pattern recognition . 2017, p.
6517–25.
[16] Cordts M, et all. The cityscapes dataset for semantic urban scene
understanding. In: 2016 IEEE conference on computer vision and
pattern recognition (CVPR). 2016, p. 3213–23.
[17] Lin TY, et all. Microsoft COCO: common objects in context. In:
Lecture notes in computer science (including subseries lecture notes
in artificial intelligence and lecture notes in bioinformatics). vol. 8693
LNCS, 2014, p. 740–55.
[18] Wen L, et all. UA-DETRAC: a new benchmark and protocol for
multi-object detection and tracking. 2015.
[19] Lin TY, et all. Feature pyramid networks for object detection. IEEE
conference on computer vision and pattern recognition (CVPR). vol.
2017 Jan, 2017, p. 936–44.
[20] He K, et all. Mask R-CNN. IEEE international conference on
computer vision (ICCV).vol. 2017 Oct, 2017, p. 2980–8.
[21] Wang X, et all. Focal loss dense detector for vehicle surveillance. In:
2018 international conference on intelligent systems and computer
vision (ISCV). vol. 2018 May, 2018, p. 1–5.
[22] Liu S, et all. Path aggregation network for instance segmentation. In:
2018 IEEE/CVF conference on computer vision and pattern
recognition. 2018, p. 8759–68.
[23] He K, et all.. Mask R-CNN. international conference on computer
vision (ICCV). vol. 2017 Oct, 2017, p. 2980–8.
IEEE.