Save.

iot
(Abstract)

Problem Description:
Looking up at all the security issues in our country today, there needs to be an automated
surveillance system that can detect anomalies, fight, shoplifting, harassment of any sort, etc.
and report itself to the nearest police station via an app or a phone call. A system is required
that incorporates AI and computer vision to achieve the same.

Solution:

There are two parts to the solution:

Part 1: The software:

The proposed solution is a real-time violence detector based on deep-learning methods. The
proposed model consists of CNN as a spatial feature extractor and LSTM as a temporal relation
learning method with a focus on the three-factor (overall generality - accuracy - fast response
time).

Part 2: The hardware:

The proposed system is a single robot that scans a room by moving around it, randomly,
continually watching for the occurrence of any violence-related activity. Upon detecting a violent
activity, it sends out a notification to the user’s mobile phone, in which it includes a picture of the
detected activity. The user can now choose to view a live feed from the robot’s perspective and
control it manually, to find out more. ​A Raspberry Pi module along with a pi camera​, that is
connected to the WiFi will power the robot.

What exactly the hardware will be?

The bot will be equipped with an​ ultrasonic sensor ​to detect obstacles that come in its way. It
will also have a ​PIR (Passive Infrared Sensor)​ sensor mounted that will detect abnormal
activity and further activates the camera to check whether the activity matches a violent activity
or not.

Android app:
The robot exchanges data and commands from the user through an android application, using
Google’s Firebase platform. The robot and the app are connected to an online real-time
database, that can be read from and written to by robot devices.

Main features of this app will be:

1) It can be used for navigating the robot in the manual mode.
2) It can also be used to view the live stream from the bot’s perspective.
3) It can be used to view the timely updates in the form of images that will be captured by the
cam.
Integration of the software and hardware:
The final output will be a surveillance bot that can be controlled manually by an app or can be
deployed autonomously and will be linked to an online-server(firebase) on which the proposed
deep-learning model will be deployed and the real-time video will be captured by a raspberry pi
and a raspberry pi cam module setup.

There will be two ways of capturing the live feed of the camera module:

1)Power saving mode:

The power-saving mode will fire the camera only when it gets a signal from the PIR sensor. The
output from the PIR sensor is checked at every iteration of the main loop and if that is high, the
camera is fired and the activity is reported via the mobile app. The rest of the time, the camera
module will be off, thus saving power.

2)Regular mode:
The regular mode will have the camera module switched-on always and a loop will be
continuously checking on the live feed. If there is a detection of any kind of violence, there will
be a notification sent on the mobile app.

Value Proposition:
Taking care of important things and guarding houses has always been a necessary aspect of
human life. Earlier these activities were done by humans themselves but as time passed, the
CCTV’s were introduced which helped in having an eye over multiple locations from a single
position/place. But these CCTV’s are having a major disadvantage of them being stationary and
we need many of them to track over multiple locations. And moreover, a person needs to take
up the responsibility of monitoring this video feed. Also, these CCTV’s need installation and
maintenance and have no feature of reporting a suspicious/violent activity.

Data-flow​:
Architecture​:

Tech stack:
Main Software:
For the ​software part​, we will be using​ CNN and LSTM​ based out of ​Keras and Tensorflow
for violence detection and confirmation. The language used to code will be ​python​ and the data
set(required for training and testing the model) will be a custom data set, which will be a mixture
of pre-existing ones and those made by us.

Mobile app:
For the app, we will be using android studio.

Hardware:
For the hardware, we will be using Raspberry Pi and its camera module along with the
ultrasonic sensor and PIR sensor. We will be using the motor driver and DC motors for the
translation of the robot. The mount will be either styrofoam or 3D printed board.

Further works:
We would like to apply the same logic to a drone-based system that can detect violence from
aerial views. Further, we want to make a network of surveillance bots to cover a large area. In
this project, we would add the feature of auto-signaling and covering a large field of an area with
less power.