STEPS TO IMPLEMENT IT ON YOUR END – INTRUDER/BURGLARY DETECTION SYSTEM WITH FACIAL

RECOGNITION USING MACHINE LEARNING ALGORITHM

Hardware

Webcam

Google Home

VS Code

Microsoft VS Code

Python 64 bit

Microsoft Visual Studio - For installing the face recognition library

STEP 1: Install Libraries

install the following libraries for the code to work

Open CV: pip install opencv-python - to get video input and save image when a face is detected.

Face recognition: pip install face_recognition - to detect and identify faces from video input.

PyChromecast: pip install pychromecast - to cast the audio file to Google home, notifying the user of the
detected face.

Pickle: pip install pickle - to save and get the names in our dataset and their face encodings.

gTTS: pip install gtts - to generate and save the audio files to notify the user of the face which is
detected.

Install each of these libraries by opening a command prompt window and typing in the commands
mentioned above. For example, to install the opencv library, type in pip install opencv-python in the
command prompt window.

Note: Installing the face_recognition library is a bit hairy. Open a command prompt window where you
have installed Visual Studio (Not VS code) and type in the command. You will have to also have to install
the cmake library as a dependency to do this.

All the other libraries which are required are:

smtplib- for setting up a simple server for email

email.message- for sending an email to the user, notifying them on a face detection
http.server- for setting up a simple HTTP server to cast local audio files to Google Home.

multiprocessing- to run the HTTP server parallelly with the rest of the code, instead of manually creating
an HTTP server

STEP 2: Training Data

Data: Training and testing data

Make a directory for storing the images/training data of yourself. the programme has to recognize your
face. At least 5 images is necessary to have a more accurate result. Make subfolders for the intruder,
and put images of the intruder there too.

STEP 3: Code to generate face data and corresponding audio files

Generate the face data for each image in the directory where you have stored the images and save the
data. You will also generate and save audio files for each person, which will notify the user of the person
detected.

Note: To get the names of each person whose picture is in the known directory, you need to save the
picture in the name of the person – the pictures in the folder, rename them to your own name.

import pickle

import face_recognition

import os

import gtts

training_encodings = []

Names = []

image_dir = r" *Path to directory of known images* "

for root, dirs, files in os.walk(image_dir):

for file in files:

path = os.path.join(root,file)

name = os.path.splitext(file)[0]

person = face_recognition.load_image_file(path)

encoding = face_recognition.face_encodings(person)[0]

training_encodings.append(encoding)
if "_" in name:

name = name.split("_")[0]

Names.append(name)

print(name)

with open('train.pkl','wb') as f:

pickle.dump(Names,f)

pickle.dump(training_encodings,f)

print(Names)

for name in Names:

msg = name+"is at the door"

tts = gtts.gTTS(msg,lang="en",tld = "co.uk")

url = name+".mp3"

tts.save(url)

STEP 4: The Main Code !! – check .py file sent.

import smtplib

from email.message import EmailMessage

import face_recognition

import cv2

import pickle

import pychromecast

import http.server

import multiprocessing

train_encodings = []

Names = []

scale_factor = 0.25

cast = pychromecast.Chromecast(" *Your Google home's local IP address* ")

cast.wait()

print(cast.name)
mc = cast.media_controller

speak = True

mail = True

nameOld = ''

def caster(url):

global speak

if speak == True:

mc.play_media(url, 'audio/mp3')

mc.block_until_active()

mc.pause()

mc.play()

speak = False

def email_alert(subject, body, to):

global mail

if mail == True:

msg = EmailMessage()

msg.set_content(body)

msg['subject'] = subject

msg['to'] = to

user = " *Your email ID* "

msg['from'] = user

password = " *Your Google account's double authentication password* "

server = smtplib.SMTP('smtp.gmail.com',587)

server.starttls()

server.login(user,password)

with open(r'login.jpg', 'rb') as f:

image_data = f.read()

image_name = f.name

image_type = image_name.split(".")[1]
 msg.add_attachment(image_data, maintype='image', subtype=image_type, filename=image_name)

server.send_message(msg)

server.quit()

mail = False

def web_server():

httpd =
http.server.HTTPServer(server_address=('',8000),RequestHandlerClass=http.server.SimpleHTTPRequest
Handler)

httpd.serve_forever(poll_interval=0.5)

with open ('train.pkl','rb') as f:

Names = pickle.load(f)

train_encodings = pickle.load(f)

print(Names)

cam = cv2.VideoCapture(0)

font = cv2.FONT_HERSHEY_SIMPLEX

while __name__ == '__main__':

p = multiprocessing.Process(target=web_server, args=())

p.daemon = True

p.start()

r,img = cam.read()

img_small = cv2.resize(img,(0,0),fx = scale_factor,fy = scale_factor)

img_rgb = cv2.cvtColor(img_small,cv2.COLOR_BGR2RGB)

face_positions = face_recognition.face_locations(img_rgb,model = 'cnn')

if not face_positions:

continue

all_encodings = face_recognition.face_encodings(img_rgb,face_positions)

for (top,right,bottom,left),face_encoding in zip(face_positions,all_encodings):

name = "Unknown person"

matches = face_recognition.compare_faces(train_encodings,face_encoding)
 if True in matches:

image_index = matches.index(True)

name = Names[image_index]

email = "Log in by: "+name

url = "http:// *Your computer's local IP address* :8000/"+name+".mp3"

print(url)

if name != nameOld:

speak = True

nameOld = name

caster(url)

top = int(top//scale_factor)

left = int(left//scale_factor)

bottom = int(bottom//scale_factor)

right = int(right//scale_factor)

cv2.rectangle(img,(top,left),(bottom,right),(0,255,0),2)

cv2.putText(img,name,(left,top),font,0.75,(0,255,0),thickness=2)

cv2.imwrite("login.jpg",img)

email_alert("Update",email," *To email address* ")

In this code, you are doing 3 main things:

Load the training data into the code using the pickle.dump command

Acquiring data from the camera and comparing it with the face encodings using the
face_recognition.compare_faces(train_encodings,face_encoding) command. you need to save this
output in a variable, I'm calling it matches. This returns a list of true and false values, comparing the face
found in the camera to each face in our database.

To integrate this program with Google Home. The Google Home will notify the user of the name(s)
detected – download googlehome if you don’t have it.
STEP 5: Integration with Google Home

*This is an extra part of the main code. You can customize your output however you want it.

I wanted to integrate this code with Google Home. The goal was to make the Home notify the user of
the person detected in the camera. There are multiple libraries out there to cast an audio file to Google
or make it say something; the process remains the same. I considered using the googlehomepush library
to do this, however, the library wasn't compatible with the gtts version I had. So, I decided to use an
alternate library: pychromecast. As the name suggests, this library can cast an audio /video file to any
cast-enabled device. So, I have used the gtts library to generate the audio files for the customized
message for each user. For instance, if the name detected is "John", the automated voice would say
"john” at the door".

The googlehomepush library does the same, except it stores the generated audio files as cache.

import pychromecast

cast = pychromecast.Cast(" *Your Google home's local IP address* ")

cast.wait()

mc = cast.media_controller

def caster(url):

global speak

if speak == True:

mc.play_media(url, 'audio/mp3')

mc.block_until_active()

mc.pause()

mc.play()

speak = False

The url variable is normally the path to the audio file. However, local files need an HTTP server to be cast
to the Google Home. create one by typing in python -m http.server in a command prompt window. Note
that this needs to be done in the directory where your program is.