AMRUTVAHINI COLLEGE OF ENGINEERING,SANGAMNER.

Department of Information Technology

A
project review
on

“AI BASED SURVEILLANCE SYSTEM FOR

CROWD MONITORING”
Presented by:
Awdhoot Kanawade
Shubham Band
Ganesh Mahale
Kanchan Khare

Guided by

Prof. Markad A.V.

1
 Contents
1. Introduction
2. Literature Survey
3. Problem Statement
4. Objective
5. Software tools
6. Proposed system and proposed outcomes
7. Detailed Design
8. Experimental Result
9. Conclusion
10. References
11. Project Plan 3.0

2
 Introduction
● The object detection and tracking is the important steps of computer vision
algorithm.The robust object detection is the challenge due to variations in the
scenes.

● Another biggest challenge is to track the object in the occlusion conditions.

Hence in this approach, the moving objects detection using TensorFlow object
detection API. Further the location of the detected object is pass to the object
tracking algorithm.

● A novel CNN based object tracking algorithm is used for robust object detection.
The proposed approach is able to detect the object in different illumination and
occlusion. The proposed approach achieved the accuracy of 90.88 percent on
self generated image sequences 4
• Usually, frame contains background and foregrounds information. In this
foreground object is represented by features points in the ROI and remaining
features are considered as background. In general, surveillance system
consists of two major steps such as moving object detection and motion
estimation. The first step is the object detection and it is influenced by the
background pixels information’s

• Background subtraction is the method which extracts the interested moving

object from the video frames. The background subtraction is affected by
mostly non-stationary background and illumination changes.

5
• To overcome this limitation, in this approach a novel and generalized Tensor
flow based object detection and CNN based object tracking algorithm has
been presented.These approaches are robustly detected and track the object
in complex scenes and complicated background conditions.

● Tracking objects is a fundamental problem in computer vision. Traditional

methods based on feature, such as those based on color or blobs movement ,
follow-up maintaining a simple model of the objective and adapting this model
over time. However, real situations in practice pose enormous challenges to
these techniques

5
 Literature Survey
Sr. Reference Name Another Name Year Conclusion
No

1. Online tracking by Hong, S., T. You, S. Xiv preprint output of the last layer of the
learning discriminative Kwak, and B. Han arXiv:1502.067 pre-trained CNN module is
saliency map with 96 cascade with the on-line SVM to
convolutional neural learn discriminative appearance
network models. The tracking is performed
using Bayesian network with
target specify saliency map.

2. transferring rich feature Wang, N., S. Li, A. Computing used pre-trained CNN model for
hierarchies Gupta, and D. Yeung Research online tracking. The CNN is used
for robust visual tracking Repository after parameter tuning to adjust
abs/1501.0458 the appearance of the object in
7, 2015 the scene and probability map is
created to instead of creating
labels 6
Sr. Reference Name Aauther Name Year Conclusion
No

3. Human tracking using Jialue Fan, Wei Xu, Proc. IEEE It use to consider human tracking as a
Convolutional Neural Ying Wu and Yihong transactions special case of object class tracking,
Network Gong on neural spatial and temporal structures have been
networks, learned during offline training.
vol.21,
NO.10, Oct
2010

4. Boosting object Feris, R., A. Datta, S. It present a novel approach to automatically

detection Pankanti, and M. T. IEEE create efficient and accurate object
performance in Sun Workshop on detectors to work well on specific video
crowded surveillance Applications surveillance cameras (specific-domain
videos In of Computer detectors), using samples acquired with the
Vision, pp. help of a more expensive, general-domain
427-432, detector (trained using images from multiple
2013. cameras).

7
 Problem Statement

• TO IMPLEMENT ROBUST MOVING OBJECT DETECTION AND TRACKING ALGORITHM USING

TENSORFLOW OBJECT DETECTION API AND NOVEL CNN TO TRACK OBJECTS IN COMPLEX
SCENES

3
 Objectives
• To overcome the issues with older object detection & tracking
algorithms using Tensorflow And CNN based approach

• To provide robust solution to detect and track object in

complex scenes having non steady background

• Accurately track objects throughout the frame

8
 Software Tools

● Python 3.6
● Tensorflow 2.0
● Darknet Yolo V3
● DeepSORT
● TKinter
● Any live video source CCTV / Drone etc.
● USB video receiver.

3
 Proposed system and proposed
outcomes
● Crowd monitoring cameras can be equipped with this technology to further
assist authorities to track down suspicious people

● Public CCTV systems can use this technology to further increase their
reliability furthermore shop cameras can be also equipped with this
technology which will come very useful in incidents of robbery.

● Outcome : Authorities will get sms/popup notification about any

suspect , so that they can take further action
8
Detailed Design
 Experimental Result

❖ Annotation

● In this process we label the images using the tool ‘lableimg’

● In this process we are making boundary box around the object which we want to detect
● By doing so we are showing the detector which part is to be detected
● Output of this process is text file which contains (x,y) coordinates of the bounding box
● But to use this data with YOLOV3 we have to convert it to yolo specific format ( starting
coordinate at bottom )

3
Original File

Converted File
❖ Training

● Training is process of making your model familiar with the objects we minted to track
● Due to limitation in local computing power we opted for cloud servers with graphics card to
train our data
● We used “google cloud servers “ with “Nvidia tesla t4 “ graphics cards
❖ Testing

● Testing is process of testing newly trained model against unseen data to get grasp of its
accuracy
● Testing was done on local machine with 480p camera stream and Nvidia gforce GTX 1050ti
(compute power 6.1 )
❖ GUI

● Gui is developed in TKinter python

● Gui consist of canvas to show video stream and different buttons
❖ Alerts

● Alerts were sent by integrating Pushbullet api in gui

● System captures location using IP address of the base and integrates it with google map api
link which when clicked can directly open map to the sent location.
● Screenshot of detection is captured and sent using pushbullet api
● Pushbullet client installed inside smartphone will display push notification to the respective
authorities
Alert sent to Headquarters Push notification on Alert sent to Ground
notification shade Control Crew
❖ Performance Evaluation

Training Error 0.001000

Testing Accuracy 60%+

FrameRate 12-13 Fps

❖ Publication and presentation details

● Presented at CSI Project competition 2020-21.

● Published in “International Journal for Scientific Research and
Development”
 Conclusion
1. In this project new approach to public safety is explained using artificial
intelligence , which can be very helpful in maintaining peace in public gatherings
, to help in defence operations and to help to deal with robberies in banks or
jewellery shops

28
 References
1. Shraddha Mane ,“Moving object detection and tracking Using Convolutional
Neural Networks ”,Proceedings of the Second International Conference on
Intelligent Computing and Control Systems (ICICCS 2018).

2. Hong, S., T. You, S. Kwak, and B. Han,“Online tracking by learning

discriminative saliency map with convolutional neural network”, arXiv preprint
arXiv:1502.06796

28
3. Wang, N., S. Li, A. Gupta, and D. Yeung,“Transferring rich feature
hierarchies for robust visual tracking”, Computing Research Repository
abs/1501.04587,2015.

4. Jialue Fan, Wei Xu, Ying Wu and Yihong Gong,” Human tracking using
Convolutional Neural Network,” in Proc. IEEE transactions on neural
networks, vol.21, NO.10, Oct 2010

5. Feris, R., A. Datta, S. Pankanti, and M. T. Sun, “Boosting object detection

performance in crowded surveillance videos”. In IEEE Workshop on
Applications of Computer Vision, pp. 427-432, 2013.
29
Project Plan 4.0 28
Thank You …