instructables

Build Your Own AI Personal Assistant and Smart Security Camera in Python

by abbasmj

In this instructable, I will show you how you can create two easy and practical projects using Face Recognition in Python.
The rst project will be Using Face Recognition as a Security Camera
The second project will be A personal Assistant using Face Recognition

This tutorial assumes you have some experience with python and installing python modules using PIP
Supplies:

Hardware Requirements:
A laptop/desktop with a webcam
Andriod Phone to receive a push noti cation (only for Step 2)

Software Requirements:
Python 3
External Python Libraries
face_recongition
google tts
playsound
opencv
pushbullet (only for Step 2)
todoist-api (only for Step 3)

Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 1
 https://youtu.be/StoU0O3o7vY

Step 1: Teach + Recognition of Face

In the rst step of the project, we will write the code to teach a face to the program and then see if it can correctly
identify/recognize the face that we taught it. To do this we will use the face_recogniztion library/module in python
Install Python Library

Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 2
 pip3 install face_recognition
pip3 install opencv-python

Teach our face

We need to rst teach a face to our program. We do this by loading an image ( make sure to have this image in the same
folder as the script/program) of a face that would like to recognize. Once we load the image we use the "face encoding"
function to get all the features of the face and save it a variable for later use.
import face_recogition
abbas_image = face_recognition.load_image_file("abbas.jpg")
abbas_face_encoding = face_recognition.face_encodings(abbas_image)[0]
known_face_encodings = [abbas_face_encoding,]
known_face_names = ["Abbas",]

Recognize our face

Once we have the encoding of a face that we want to recognize using the output of our webcam we can look at each
frame of the video and see if we can detect a face. Once we have detected a face we get the encoding of that face and
compare it to the encoding of the face that we want to recognize. If the encodings are similar then it is a match else the
face will be categorized as "Unknown". The code to do this look like this:
import cv2
video_capture = cv2.VideoCapture(0)
while True:
# Grab a single frame of video
ret, frame = video_capture.read()

# Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
rgb_frame = frame[:, :, ::-1]

# Find all the faces and face enqcodings in the frame of video
face_locations = face_recognition.face_locations(rgb_frame)
face_encodings = face_recognition.face_encodings(rgb_frame, face_locations)

# Loop through each face in this frame of video

for (top, right, bottom, left), face_encoding in zip(face_locations, face_encodings):
# See if the face is a match for the known face(s)
matches = face_recognition.compare_faces(known_face_encodings, face_encoding)

name = "Unknown"

# If a match was found in known_face_encodings, just use the first one.

if True in matches:
first_match_index = matches.index(True)
name = known_face_names[first_match_index]

# Draw a box around the face

cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

# Draw a label with a name below the face

cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
font = cv2.FONT_HERSHEY_DUPLEX
cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

# Display the resulting image

cv2.imshow('Video', frame)

# Hit 'q' on the keyboard to quit!

if cv2.waitKey(1) & 0xFF == ord('q'):
break

# Release handle to the webcam

video_capture.release()
cv2.destroyAllWindows()

Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 3
Now when you run your program your a window will pop up and you should see your face is detected and also correctly
identi ed. You can also exit the program by pressing 'q' on your keyboard

Step 2: Using Face Recognition As a Security Camera

Now that we are able to recognize a face using python let's use this to create a face-recognizing security camera that can
alert you when an unknown person is detected. We will reuse the code from step 1 for the face recognition part and for
the alarm/announcement, we will use Google Text To Speech (gTTs) and playsound library. We will also send an alert
noti cation on our smartphone using the Push Bullet API (only on Android)
Let's start by installing the necessary libraries using pip
pip3 install gTTs
pip3 install pushbullet.py
pip3 install playsound
pip3 install PyObjc

Adding an announcement
We will add this code in an else clause after the compare operation. Basically what we do is "if a match is not found then
we create a custom speech mp3 using google text to speech and play it on our computer's speaker using the playsound
library. The message will be played every 2s.

Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 4
 from gtts import gTTs
import datetime
import playsound

# Outside the while loop

last_time = datetime.datetime.now()

if True in matches:
first_match_index = matches.index(True)
name = known_face_names[first_match_index]
else:
current_time = datetime.datetime.now()
if current_time <= last_time + datetime.timedelta(seconds=2):
tts = gTTS('Intruder Alert ! Intruder Alert.', lang='en')
tts.save('intruder.mp3')
playsound.playsound('intruder.mp3')
last_time = current_time

Sending a noti cation ( Only on Android )

We will use the Push Bullet service to send noti cations to our phones. To do this we need to rst do the following:
1. Go to Push Bullet Website and Sign Up
2. Once signed in go to the settings tab and click on the button "Create an Access Token". We will use the
token in our python script ( see image)
3. Download Push Bullet App on your Android and Sign in
4. Add the following code to send a noti cation on your smartphone

from pushbullet import Pushbullet

# Outside the while loop
api_key = "YOUR_ACCESS_TOKEN"
pb = Pusbullet(api_key)

# In the else clause we added in the last step

pb.push_note("Intruder Detected ", "An unknown intruder was detected")

Now run the script and when an "Unknown" person is detected an alarm should be played on the your computer and
you should receive a push noti cation on your smartphone.

https://youtu.be/un39MrKpopg

Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 5
 Download

https://www.instructables.com/ORIG/F7M/9O3X/KX0ERUW5/F7M9O3XKX0ERUW5.py

Step 3: Personal Assistant Using Face Recongnition

Another use of Face Recognition could be to use it as a personal assistant. The inspiration for this use case came from my
laziness in checking my to-do list when I came to my home o ce desk and as a reason missing important deadlines.
Furthermore, I also wanted it to only tell me the list and not when someone else ( e.g wife or kids) walked in, and
therefore face recognition was a perfect t
The features of this assistant are the following:
1. Greets you
Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 6
 2. Tells you the latest weather
3. Reads you your incomplete task from your Todoist Todo list

The face recognition code will be the same as Step 1

Adding the greeting
This is easy. We just create a new function that will take the name of the user that was detected and create greeting
audio with gTTS and play in using the playsound library. Then we call the function from the "True in matches"
section in the while loop
def greet_user(name):
tts = gTTS(f'Hello {name}.', lang='en')
tts.save('greet.mp3')
playsound.playsound('greet.mp3')

Announcing the latest weather

For getting the latest weather information we will utilize the OpenWeatherMap API. To use the API we need to rst
generate an API key. To do this go to their website and signup. Once you have signed up and login click on your pro le
and go to the "My API keys" section ( See image ). Over here you should be able to generate a new API key to use. Once
you have the API key let's query the data. We will use the "requests" library to query the data and the "json" library to
convert it to python parseable format
import json
import requests

weather_api_key = "YOUR_API_KEY"
base_url = "http://api.openweathermap.org/data/2.5/weather?"
city_name = "YOUR_CITY_NAME"

def get_latest_weather():
raw_response = requests.get(base_url + "appid=" + weather_api_key + "&q=" + city_name)
data = raw_response.json()
current_temp = round(int(data["main"]["temp"]) - 273.15)
current_hum = round(int(data["main"]["humidity"]))
current_cond = data["weather"][0]["description"]
tts = gTTS(f'The current condition is {current_cond} with the temperature of {current_temp} degree celcius and current humidity at {current_hum} percent')
tts.save("current_weather.mp3")
playsound.playsound("current_weather.mp3")

Don't forget to call the "get_latest_weather" function from the "True in matches" section in the while loop

Reading you incomplete tasks from your todo list

For my to-do list, I didn't want to create a to-do list application but wanted to use an already available solution. That's
where I found Todoist. It's a complete to-do list management application with a nice web UI + ios and android
integration. It also has an open API so it integrates well with python also. To get data from the API as usual we need an
API key, which you can get by logging into the Todoist click on your pro le picture in the top-right hand corner, and then
selecting ‘Integrations’. You should nd your API Key there (see image). Next, we will create a new project from the web
app and add a few tasks to them. Now let's get this information into our python application
Start by installing the Todoist python package
pip3 install todoist-python

We will also need to get the "id" of the new project that we created. We can do this by running these commands from the
python repl
Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 7
 from todoist.api import TodoistAPI
api = TodoistAPI("YOUR_API_KEY")
api.sync()
print(api.state['projects'])

When you run this you might have a big list. Look for the 'name' of your project and once you have found it copy the "id"
to your python program (see image). Once you have all this you can query and get all the uncompleted tasks running
this code:
api_key = "YOUR API_KEY"
api = TodoistAPI(api_key)

def get_latest_todo_list():
api.sync()
all_data = api.projects.get_data(YOUR_PROJECT_ID)
todo_list = [items["content"] for items in all_data["items"]]
audio = f'''
You have {len(todo_list)} items in your to do list
'''
for item in todo_list:
audio = audio + item + ', '
tts = gTTS(audio)
tts.save("todo_list.mp3")
playsound.playsound("todo_list.mp3")

Don't forget to call the "get_latest_todo_list" function from the "True in matches" section in the while loop

https://youtu.be/_wdRAWZBHC8

Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 8
Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 9
 Download

https://www.instructables.com/ORIG/F0E/2CN7/KX1TX7A1/F0E2CN7KX1TX7A1.py

Build Your Own AI Personal Assistant and Smart Security Camera in Python: Page 10