5-8-9-18-21-34-LE3

VNR Vignana Jyothi Institute of Engineering and

Technology (Affiliated to J.N.T.U, Hyderabad)
Bachupally(v), Hyderabad, Telangana, India.

DRIVER’S DROWSINESS DETECTION

A course based project(Machine Learning) submitted in partial fulfilment of the

requirements for the award of the degree of
BACHELOR OF TECHNOLOGY

IN

COMPUTER SCIENCE AND ENGINEERING

Submitted by

ANURAAG CHILAKAMARRI (20071A0505)

BELIGINI SHIVA TEJA (20071A0508)
BENDUKURI RISHAB (20071A0509)
GANGISHETTY MANIDEEP (20071A0518)
GUMMADI MAHESH GANDHI CHOWDARY (20071A0521)
KONDAM SAI SRUTHIN REDDY (20071A0534)
KURAPATI JASHWANTH KUMAR (21075A0503)

Under the guidance of

Mr.P.Venkateswara
Rao Assistant Professor
Dept. of Computer Science and Engineering
5-8-9-18-21-34-LE3

VNR Vignana Jyothi Institute of Engineering and Technology

(Affiliated to J.N.T.U, Hyderabad)
Bachupally(v), Hyderabad, Telangana, India.

CERTIFICATE

This is to certify that C.Anuraag (20071A0505) , B.Shiva Teja

(20071A0508), B.Rishab (20071A0509), G.Manideep (20071A0518) ,
G.Mahesh Gandhi (20071A0521) and K.Sai Sruthin Reddy
(20071A0534), K.Jashwanth Kumar have completed their course based
project work (ML using Python ) at CSE Department of VNR VJIET,
Hyderabad entitled “DRIVER’S DROWSINESS DETECTION” in
partial fulfilment of the requirements for the award of B.Tech degree
during the academic year 2020-2021. This work is carried out under my
supervision and has not been submitted to any other University/Institute
for award of any degree/diploma.

Mr.P.Venkateswara Rao Nagini Sabbineni

Professor Associate Professor and Head

CSE Department CSE Department

VNRVJIET VNRVJIET
5-8-9-18-21-34-LE3

DECLARATION

This is to certify that our project report titled “ DRIVER’S DROWSINESS

DETECTION” submitted to Vallurupalli Nageswara Rao Institute of Engineering
and Technology in complete fulfilment of requirement for the award of Bachelor of
Technology in Computer Science and Engineering is a bona fide report to the work carried
out by us under the guidance and supervision of Mr.P.Venkateswara Rao, Assistant
Professor, Department of Computer Science and Engineering, Vallurupalli Nageswara Rao
Institute of Engineering and Technology. To the best of our knowledge, this has not been
submitted in any form to other university or institution for the award of any degree or
diploma.
5-8-9-18-21-34-LE3

ACKNOWLEDGEMENT

I would like to express my gratitude and appreciation to all those who gave me the
possibility to complete this report. Special thanks is due to our Professor,
Mr.P.Venkateswara Rao whose help, stimulating suggestions and encouragement
helped us in all time of fabrication process and in writing this report. I also sincerely
thank for the time spent proofreading and correcting our mistakes.

Behind every achievement lies an unfathomable sea of gratitude to those who

activated it, without which it would ever never have come into existence. To them I
lay the words of gratitude imprinting within us.

VNRVJIET has helped us transform ourselves from mere amateurs in the field of
Computer Science into skilled engineers capable of handling any given situation in
real time. I am are highly indebted to the institute for everything that it has given us.

We would like to express our gratitude towards the principal of our institute, Dr.
Challa Dhanunjaya Naidu and the Head of the Computer Science & Engineering
Department, Nagini Sabbineni for their kind co- operation and encouragement
which helped us complete the project in the stipulated time.
5-8-9-18-21-34-LE3

Scheme of Course Based Project

Name of the course : Course Based project (ML using Python)

Year / Semester : II-B.Tech I-semester
Project Title : Drivers drowsiness detection
Done by ANURAAG CHILAKAMARRI (20071A0505)
BELIGINI SHIVA TEJA (20071A0508)
: BENDUKURI RISHAB (20071A0509)
GANGISHETTY MANIDEEP (20071A0518)
GUMMADI MAHESH GANDHI CHOWDARY
(20071A0521)
KONDAM SAI SRUTHIN REDDY (20071A0534)
KURAPATI JASHWANTH KUMAR
(21075A0503)

Project Objectives :
 Driver drowsiness detection is a car safety technology which helps to save
the life of the driver by preventing accidents when the driver is getting
drowsy.
 The main objective is to first design a system to detect driver’s drowsiness
by continuously monitoring the eye.
 The system works in spite of driver wearing spectacles and in various
lighting conditions.
 To alert the driver on the detection of drowsiness by using buzzer or alarm.
 Traffic management can be maintained by reducing the accidents.
Project Outcomes : A training and testing model to identify drowsiness.
Description : Drivers face is continuously monitored using a video
camera. In order to detect the drowsiness the first step is
to detect the face using the series of frame shots taken by
the camera. Scores is incremented when eyes are closed
and vice versa when open. Based on the score, the alarm
beeps So that driver can be alerted.
Stages
involved:
Step 1 – Take image as input from a camera.
Step 2 – Detect the face in the image and create a Region of
Interest (ROI).
Step 3 – Detect the eyes from ROI and feed it to the classifier.
Step 4 – Classifier will categorize whether eyes are open or
closed.
Step 5 – Calculate score to check whether the person is
drowsy.
5-8-9-18-21-34-LE3

ABSTRACT
This system is a real time system which captures image continuously and measures the
state of the eye according to the specified algorithm and gives warning if required.

 ABOUT 68% OF THE ROAD ACCIDENTS HAPPEN DUE TO DRIVER

UNAWARENESS THAT IS DUE TO DRIVER DROWSINESS. THIS CAN
BE PREVENTED BY BUILIDING AN ACTIVE DRIVER DROWSINESS
DETECTION SYSTEM IMPELEMENTED IN THE AUTOMOBILES.
 OUR SYSTEM USES TECHNOLOGIES LIKE PYTHON, OPENCV,
TENSORFLOW, AND EXTRACTS THE EYES FROM THE PERSON VIDEO
FRAME BY USING HAAR EYE CLASSIFIER AND BY USING THE
TRAINED MODEL WARNS THE PERSON IF HE IS DROWSY BY
SOUNDING A BEEP ALERT.
 BY IMPLEMENTING OUR SYSTEM IN AUTOMOBILES WE CAN
REDUCE MAJORITY OF THE FATAL ROAD ACCIDENTS.

Car accident is the major cause of death in which around 1.3 million people die every
year. Majority of these accidents are caused because of distraction or the drowsiness of
driver. Construction of high-speed highway roads had diminished the margin of error
for the driver. The countless number of people drives for long distance every day and
night on the highway. Lack of sleep or distractions like the phone call, talking with the
passenger, etc may lead to an accident. To prevent such accidents we propose a system
which alerts the driver if the driver gets distracted or feels drowsy. Facial landmarks
detection is used with help of image processing of images of the face captured using the
camera, for detection of distraction or drowsiness. This whole system is deployed on
portable hardware which can be easily installed in the car for use. This project is aimed
towards developing a prototype of drowsiness detection system.
5-8-9-18-21-34-LE3

INDEX
Contents PageNo

1. Introduction........................................................................................ 10

2. Literature............................................................................................ 11

2.1 Convolution Neural Network (CNN)............................................................11

2.2 Extracting features........................................................................................11

2.3 Performance Metrics......................................................................................11

3. Requirements....................................................................................... 13

3.1 Software Requiements.......................................................................................13

3.2 Hardware Requirements.................................................................................15

4. Proposed System................................................................................. 16

5. .Model Implementation.......................................................................17

5.1 The Dataset...........................................................................................................17

5.2 Data Preprocessing..............................................................................................18

5.2.1 Importing the required libraries........................................................18

5.2.2 Load Eye dataset from kaggle............................................................18

5.2.3 Using image augmentation generator................................................19

5.2.4 Normalizing the images.......................................................................19

5.3 Features For The CNN........................................................................................20

5.4 Build A Classifier.................................................................................................21

5.4.1 Split data into training and validation sets.........................................21

5.4.2 Compiling the neural network.............................................................22

5.5 Training the model..............................................................................................23

6.Testing...................................................................................................24

6.1 Making Predictions..........................................................................................24

6.2 Retraining the model if desired accuracy is not reached..............................30

5-8-9-18-21-34-LE3

7.Conclusion............................................................................................32

7.1 Execution..................................................................................................32

7.2 Conclusion……………………………………………………………….34

7.3 Applications & Future Scope.................................................................34

7.4 Bibilography............................................................................35
5-8-9-18-21-34-LE3

LIST OF FIGURES
Contents Page No.

1.1 Drowsiness detection 10

2.3 Performance Metrics 12
3.1 Requirements 13
4. Proposed System 16
5.1 Dataset 17,18
5.5 Visualizing the learned model 23
6.3 Training loss and Accuracy 31
7.2 Open eye detection 32
7.3 Code execution 33
7.4 Closed eye detection 33

9
5-8-9-18-21-34-LE3

CHAPTER 1
INTRODUCTION
1.1 Introduction
With this Python project, we will be making a drowsiness detection system. A
countless number of people drive on the highway day and night. Taxi drivers,
bus drivers, truck drivers and people traveling long-distance suffer from lack of
sleep. Due to which it becomes very dangerous to drive when feeling sleepy.

The majority of accidents happen due to the drowsiness of the driver. So, to
prevent these accidents we will build a system using Python, OpenCV, and
Keras which will alert the driver when he feels sleepy.

1.1 Drowsiness Detection

Drowsiness detection is a safety technology that can prevent accidents that are
caused by drivers who fell asleep while driving.

The objective of this intermediate Python project is to build a drowsiness detection

system that will detect that a person’s eyes are closed for a few seconds. This
system will alert the driver when drowsiness is detected.

10
5-8-9-18-21-34-LE3

CHAPTER 2
LITERATURE
2.1 Convolution Neural Network (CNN)
 A convolutional neural network, or CNN, is a deep learning neural network
sketched for processing structured arrays of data such as portrayals.
 CNN are very satisfactory at picking up on design in the input image, such as
lines, gradients, circles, or even eyes and faces.
 This characteristic that makes convolutional neural network so robust for
computer vision.
 CNN can run directly on a underdone image and do not need any
preprocessing.
 A convolutional neural network is a feed forward neural network, seldom with
up to 20.
 The strength of a convolutional neural network comes from a particular kind of
layer called the convolutional layer.
 CNN contains many convolutional layers assembled on top of each other, each
one competent of recognizing more sophisticated shapes.
 With three or four convolutional layers it is viable to recognize handwritten
digits and with 25 layers it is possible to differentiate human faces.
 The agenda for this sphere is to activate machines to view the world as humans
do, perceive it in a alike fashion and even use the knowledge for a multitude of
duty such as image and video recognition, image inspection and classification,
media recreation, recommendation systems, natural language processing, etc.
2.2 Extracting features
 First we capture the frames from the video source and convert the picture into
grayscale.
 Next we are going to extract the features of the face using the haar frontal face
xml file downloaded from github.
 link:- https://github.com/opencv/opencv/blob/master/data/haarcascades_cuda/
haarcascade_frontalface_alt.xml
 Next we are going to extract the left eye and the right eye from the grayscale
image using the haar left eye and haar right eye classifier.
 link:-https://github.com/opencv/opencv/tree/master/data/haarcascades_cuda
 Next we extract the features of both the left and the right eye using maxpooling
layer in CNN.
2.3 Performance Metrics
The Confusion matrix is one of the most intuitive and easiest (unless of course,
you are not confused)metrics used for finding the correctness and accuracy of the
model. It is used for Classification problem where the output can be of two or
11
5-8-9-18-21-34-LE3

more types of classes. The Confusion matrix in itself is not a performance

measure as such, but almost all of the performance metrics are based on
Confusion Matrix and the numbers inside it.
Accuracy is a good measure when the target variable classes in the data are
nearly balanced.
Precision: Out of all the positive predicted, what percentage is truly positive. The
precision value lies between 0 and 1.
Recall: Out of the total positive, what percentage are predicted positive. It is the
same as TPR (true positive rate).
F1 Score: It is the harmonic mean of precision and recall. It takes both false
positive and false negatives into account. Therefore, it performs well on an
imbalanced dataset.

2.3 Performance Metrics

12
5-8-9-18-21-34-LE3

CHAPTER 3
REQUIREMENTS
Requirements analysis in systems engineering and software engineering,
encompasses those tasks that go into determining the needs or conditions to
meet for a new or altered product or project, taking account of the possibly
conflicting requirements of the various stakeholders, analyzing, documenting,
validating and managing software or system requirements.

Fig 3.1 Requirements

The requirement for this Python project is a webcam through which we will
capture images. You need to have Python (3.6 version recommended) installed
on your system, then using pip, you can install the necessary packages.

3.1 Software Requirements

Software requirements deal with defining software resource requirements and
prerequisites that need to be installed on a computer to provide optimal
functioning of an application. The software requirements for this project are:
 Python 3.6:
A Python version of 3.6 comes with the required packages that supports an
operating system of windows 7 to 10 along with 2GB RAM (4GB
preferable).
 Spyder:
The Spyder is a free and open source scientific environment written in
Python, for Python, and designed by and for scientists, engineers and data
analysts. It features a unique combination of the advanced editing, analysis,
debugging, and profiling functionality of a comprehensive development
13
5-8-9-18-21-34-LE3

tool with the data exploration, interactive execution, deep inspection, and
beautiful visualization capabilities of a scientific package.
 OpenCV:
OpenCV is a cross-platform library using which we can develop real-time
computer vision applications. It mainly focuses on image processing, video
capture and analysis including features like face detection and object
detection. In this tutorial, we explain how you can use OpenCV in your
applications.This tutorial has been prepared for beginners to make them
understand the basics of OpenCV library. We have used the Java
programming language in all the examples, therefore you should have a
basic exposure to Java in order to benefit from this tutorial.
 Tensor flow:
TensorFlow is an open-source end-to-end platform for creating Machine
Learning applications. It is a symbolic math library that uses dataflow and
differentiable programming to perform various tasks focused on training
and inference of deep neural networks. It allows developers to create
machine learning applications using various tools, libraries, and
community resources.
Currently, the most famous deep learning library in the world is Google’s
TensorFlow. Google product uses machine learning in all of its products to
improve the search engine, translation, image captioning or
recommendations.
 Keras:
P Keras is an open-source software library that provides a Python interface
for artificial neural networks. Keras acts as an interface for the TensorFlow
library.
Up until version 2.3, Keras supported multiple backends, including
TensorFlow, Microsoft Cognitive Toolkit, Theano, and PlaidML. As of
version 2.4, only TensorFlow is supported. Designed to enable fast
experimentation with deep neural networks, it focuses on being user-
friendly, modular, and extensible. It was developed as part of the research
effort of project ONEIROS (Open-ended Neuro-Electronic Intelligent
Robot Operating System), and its primary author and maintainer is
François Chollet, a Google engineer. Chollet is also the author of the
XCeption deep neural network model.
 Pygame:
Pygame is a Python-based library which is filled with different modules
(files containing code) that are used for developing 2D videogames and
other multimedia programs. Popular among millions of people across the
14
5-8-9-18-21-34-LE3

world, Pygame is suitable for kids learning to code who want to create
something exciting, unique and fun.

3.2 Hardware Requirements

The most common set of requirements defined by any operating system or
software application is the physical computer resources, also known as hardware,
The most common set of requirements defined by any operating system or
software application is the physical computer resources, also known as hardware,
A hardware requirements list is often accompanied by a hardware compatibility
list (HCL), especially in case of operating systems. An HCL lists tested,
compatible, and sometimes incompatible hardware devices for a particular
operating system or application.
 Architecture – All computer operating systems are designed for a
particular computer architecture. Most software applications are limited
to particular operating systems running on particular architectures.
Although architecture-independent operating systems and applications
exist, most need to be recompiled to run on a new architecture. See also
a list of common operating systems and their supporting architectures.
 Processing power – The power of the central processing unit (CPU) is a
fundamental system requirement for any software. Most software
running on x86 architecture define processing power as the model and
the clock speed of the CPU.
 RAM: 4GB and higher
 Processor: Intel i3 and above
 Hard Drive: Minimum 32 GB; Recommended 64 GB or more.
 Ethernet connection (LAN) OR a wireless adapter (Wi-Fi)
 Memory (RAM): Minimum 1 GB; Recommended 4 GB or above.

15
5-8-9-18-21-34-LE3

CHAPTER 4
PROPOSED SOLUTION
Our model uses machine learning algorithms(convolutional neural networks) to
predict whether the eyes are closed or open and can be implemented easily on
low powered devices such as raspberry pi making it an efficient system for
mass production.

 Take Image as Input from a Camera

 Exploring some features

 Detect Face in the Image and Create a Region of Interest (ROI)

 Detect the eyes from ROI and feed it to the classifier

 Classifier will Categorize whether Eyes are Open or Closed

 Calculate Score to Check whether Person is Drowsy

 A threshold is defined for example if score becomes greater than 15 that

means the person’s eyes are closed for a long period of time.

1. Proposed System

16
5-8-9-18-21-34-LE3

CHAPTER 5
MODEL IMPLEMENTATION

5.1 THE DATASET

The LendingClub is a peer-to-peer leading company that directly connects
borrowers and potential lenders/investors. The dataset used for this model is
created by us. To create the dataset, we wrote a script that captures eyes from a
camera and stores in our local disk. We separated them into their respective
labels ‘Open’ or ‘Closed’. The data was manually cleaned by removing the
unwanted images which were not necessary for building the model. The data
comprises around 7000 images of people’s eyes under different lighting
conditions. After training the model on our dataset, we have attached the final
weights and model architecture file “models/cnnCat2.h5”.
Now, you can use this model to classify if a person’s eye is open or closed.
Alternatively, if you want to build and train your own model, you can download
the dataset: Driver Drowsiness Dataset

17
5-8-9-18-21-34-LE3

5.1 Dataset

5.2 DATA PREPROCESSING

5.2.1 Importing the required libraries
We import cv2, os, load_model from keras.models, numpy as np,
mixer from pygame and time modules

5.2.2 Load Eye dataset from kaggle

The dataset used for this model is created by us. To create the dataset, we wrote
a script that captures eyes from a camera and stores in our local disk. We
separated them into their respective labels ‘Open’ or ‘Closed’. The data was
manually cleaned by removing the unwanted images which were not necessary
for building the model. The data comprises around 7000 images of people’s eyes
under different lighting conditions. After training the model on our dataset, we
have attached the final weights and model architecture file
“models/cnnCat2.h5”.

18
5-8-9-18-21-34-LE3

def generator(dir, gen=image.ImageDataGenerator(rescale=1./255),

shuffle=True,batch_size=1,target_size=(24,24),class_mode='categorical' ):

    return
gen.flow_from_directory(dir,batch_size=batch_size,shuffle=shuffle,color_mode='graysca
le',class_mode=class_mode,target_size=target_size)

BS= 32
TS=(24,24)
train_batch= generator('data/train',shuffle=True, batch_size=BS,target_size=TS)
valid_batch= generator('data/valid',shuffle=True, batch_size=BS,target_size=TS)

5.2.3 Using image augmentation generator

We are going to use the image augmentation generator from keras preprocessing.
The data generator will first normalize the images and then shuffle the images
randomly.
The images are converted to grayscale and then resized to (24,24) dimensions.
Then we convert the image to martix format.
This matrix is then normalized by dividing the elements of the matrix by 255.
def generator(dir, gen=image.ImageDataGenerator(rescale=1./255),
shuffle=True,batch_size=1,target_size=(24,24),class_mode='categorical' ):

    return
gen.flow_from_directory(dir,batch_size=batch_size,shuffle=shuffle,color_mode='graysca
le',class_mode=class_mode,target_size=target_size)

5.2.4 Normalizing the images

Image Scaling: Once we’ve ensured that all images are square (or have some
predetermined aspect ratio), it’s time to scale each image appropriately. We’ve
decided to have images with width and height of 24 pixels. We’ll need to scale
the width and height of each image by a factor of 0.4 (100/250). There are a wide
variety of up-scaling and down-scaling techniques and we usually use a library
function to do this for us.
Dimensionality reduction: We could choose to collapse the RGB channels into a
single gray-scale channel. There are often considerations to reduce other
dimensions, when the neural network performance is allowed to be invariant to
that dimension, or to make the training problem more tractable.
Data augmentation: Another common pre-processing technique involves
augmenting the existing data-set with perturbed versions of the existing images.
Scaling, rotations and other affine transformations are typical. This is done to
expose the neural network to a wide variety of variations. This makes it less
likely that the neural network recognizes unwanted characteristics in the data-set.
19
5-8-9-18-21-34-LE3

5.3 Features For The CNN

The construction of a convolutional neural network is a multi-layered feed-
forward neural network, made by assembling many unseen layers on top of each
other in a particular order.
It is the sequential design that give permission to CNN to learn hierarchical
attributes.
In CNN, some of them followed by grouping layers and hidden layers are
typically convolutional layers followed by activation layers.
The pre-processing needed in a ConvNet is kindred to that of the related pattern
of neurons in the human brain and was motivated by the organization of the
Visual Cortex.
The Model Architecture
The model we used is built with Keras using Convolutional Neural Networks
(CNN). A convolutional neural network is a special type of deep neural network
which performs extremely well for image classification purposes. A CNN
basically consists of an input layer, an output layer and a hidden layer which can
have multiple layers. A convolution operation is performed on these layers using
a filter that performs 2D matrix multiplication on the layer and filter.

The CNN model architecture consists of the following layers:

Convolutional layer; 32 nodes, kernel size 3

Convolutional layer; 32 nodes, kernel size 3
Convolutional layer; 64 nodes, kernel size 3
Fully connected layer; 128 nodes
The final layer is also a fully connected layer with 2 nodes. A Relu activation
function is used in all the layers except the output layer in which we used
Softmax.

Split data into training and validation sets

Compiling the neural network

20
5-8-9-18-21-34-LE3

5.4 BUILD A CLASSIFIER

5.4.1 Split data into training and validation sets
The portion of data that we don't train on will be used to help select model
parameters (this is known as model selection). Thus, this portion of data should
be called a validation set. Examining performance of various potential models
(i.e. models with different parameters) should be on validation set, while
evaluation of the final selected model should always be on test data.

import os
from keras.preprocessing import image
import matplotlib.pyplot as plt
import numpy as np
from keras.utils.np_utils import to_categorical
import random,shutil
from keras.models import Sequential
from keras.layers import Dropout,Conv2D,Flatten,Dense, MaxPooling2D,
BatchNormalization
from keras.models import load_model

def generator(dir, gen=image.ImageDataGenerator(rescale=1./255),

shuffle=True,batch_size=1,target_size=(24,24),class_mode='categorical' ):

    return
gen.flow_from_directory(dir,batch_size=batch_size,shuffle=shuffle,color_mode='graysca
le',class_mode=class_mode,target_size=target_size)

BS= 32
TS=(24,24)
train_batch= generator('data/train',shuffle=True, batch_size=BS,target_size=TS)
valid_batch= generator('data/valid',shuffle=True, batch_size=BS,target_size=TS)
SPE= len(train_batch.classes)//BS
VS = len(valid_batch.classes)//BS

21
5-8-9-18-21-34-LE3

5.4.2 Compiling the neural network

model = Sequential([
    Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(24,24,1)),
    MaxPooling2D(pool_size=(1,1)),
    Conv2D(32,(3,3),activation='relu'),
    MaxPooling2D(pool_size=(1,1)),
#32 convolution filters used each of size 3x3
#again
    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(1,1)),

#64 convolution filters used each of size 3x3

#choose the best features via pooling
   
#randomly turn neurons on and off to improve convergence
    Dropout(0.25),
#flatten since too many dimensions, we only want a classification output
    Flatten(),
#fully connected to get all relevant data
    Dense(128, activation='relu'),
#one more dropout for convergence' sake :)
    Dropout(0.5),
#output a softmax to squash the matrix into output probabilities
    Dense(2, activation='softmax')
])

model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])

model.fit_generator(train_batch,
validation_data=valid_batch,epochs=15,steps_per_epoch=SPE ,validation_steps=VS)

model.save('models/cnnCat2.h5', overwrite=True)

We are going to use adam compiler and we are going to use categorical cross entropy
for estimating the loss.

22
5-8-9-18-21-34-LE3

5.5 Training the model

5.5 Visualizing the learned model

We are going to set the batch size as 32 and we are going to set the No. of epochs as 15. And also our
learning rate is 0.01

model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])

model.fit_generator(train_batch,
validation_data=valid_batch,epochs=15,steps_per_epoch=SPE ,validation_steps=VS)

model.save('models/cnnCat2.h5', overwrite=True)

After training the model for 15 epochs, we are going to check for accuracy and if accuracy satisfies
the requirements, we are going to save the model.

23
5-8-9-18-21-34-LE3

CHAPTER 6
TESTING
6.1 MAKING PREDICTIONS

1. import cv2

2. import os

3. from keras.models import load_model

4. import numpy as np

5. from pygame import mixer

6. import time

7.

8. mixer.init()

9. sound = mixer.Sound('alarm.wav')

10.

11. face = cv2.CascadeClassifier('haar cascade files\

haarcascade_frontalface_alt.xml')

12. leye = cv2.CascadeClassifier('haar cascade files\

haarcascade_lefteye_2splits.xml')

13. reye = cv2.CascadeClassifier('haar cascade files\

haarcascade_righteye_2splits.xml')

14.

15. lbl=['Close','Open']

24
5-8-9-18-21-34-LE3

16.

17. model = load_model('models/cnncat2.h5')

18. path = os.getcwd()

19. cap = cv2.VideoCapture(0)

20. font = cv2.FONT_HERSHEY_COMPLEX_SMALL

21. count=0

22. score=0

23. thicc=2

24. rpred=[99]

25. lpred=[99]

26.

27. while(True):

28. ret, frame = cap.read()

29. height,width = frame.shape[:2]

30.

31. gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

32.

33. faces =
face.detectMultiScale(gray,minNeighbors=5,scaleFactor=1.1,minSize=(25,25))

34. left_eye = leye.detectMultiScale(gray)

25
5-8-9-18-21-34-LE3

35. right_eye = reye.detectMultiScale(gray)

36.

37. cv2.rectangle(frame, (0,height-50) , (200,height) , (0,0,0) ,

thickness=cv2.FILLED )

38.

39. for (x,y,w,h) in faces:

40. cv2.rectangle(frame, (x,y) , (x+w,y+h) , (100,100,100) , 1 )

41.

42. for (x,y,w,h) in right_eye:

43. r_eye=frame[y:y+h,x:x+w]

44. count=count+1

45. r_eye = cv2.cvtColor(r_eye,cv2.COLOR_BGR2GRAY)

46. r_eye = cv2.resize(r_eye,(24,24))

47. r_eye= r_eye/255

48. r_eye= r_eye.reshape(24,24,-1)

49. r_eye = np.expand_dims(r_eye,axis=0)

50. rpred = model.predict_classes(r_eye)

51. if(rpred[0]==1):

52. lbl='Open'

53. if(rpred[0]==0):

26
5-8-9-18-21-34-LE3

54. lbl='Closed'

55. break

56.

57. for (x,y,w,h) in left_eye:

58. l_eye=frame[y:y+h,x:x+w]

59. count=count+1

60. l_eye = cv2.cvtColor(l_eye,cv2.COLOR_BGR2GRAY)

61. l_eye = cv2.resize(l_eye,(24,24))

62. l_eye= l_eye/255

63. l_eye=l_eye.reshape(24,24,-1)

64. l_eye = np.expand_dims(l_eye,axis=0)

65. lpred = model.predict_classes(l_eye)

66. if(lpred[0]==1):

67. lbl='Open'

68. if(lpred[0]==0):

69. lbl='Closed'

70. break

71.

72. if(rpred[0]==0 and lpred[0]==0):

27
5-8-9-18-21-34-LE3

73. score=score+1

74. cv2.putText(frame,"Closed",(10,height-20), font, 1,

(255,255,255),1,cv2.LINE_AA)

75. # if(rpred[0]==1 or lpred[0]==1):

76. else:

77. score=score-1

78. cv2.putText(frame,"Open",(10,height-20), font, 1,

(255,255,255),1,cv2.LINE_AA)

79.

80. if(score<0):

81. score=0

82. cv2.putText(frame,'Score:'+str(score),(100,height-20), font, 1,

(255,255,255),1,cv2.LINE_AA)

83. if(score>15):

84. #person is feeling sleepy so we beep the alarm

85. cv2.imwrite(os.path.join(path,'image.jpg'),frame)

86. try:

87. sound.play()

88.

89. except: # isplaying = False

90. pass

28
5-8-9-18-21-34-LE3

91. if(thicc<16):

92. thicc= thicc+2

93. else:

94. thicc=thicc-2

95. if(thicc<2):

96. thicc=2

97. cv2.rectangle(frame,(0,0),(width,height),(0,0,255),thicc)

98. cv2.imshow('frame',frame)

99. if cv2.waitKey(1) & 0xFF == ord('q'):

100. break

101. cap.release()

102. cv2.destroyAllWindows()

Step 1 – Take Image as Input from a Camera

With a webcam, we will take images as input. So to access the webcam, we made an
infinite loop that will capture each frame. We use the method provided by OpenCV,
cv2.VideoCapture(0) to access the camera and set the capture object (cap). cap.read()
will read each frame and we store the image in a frame variable.

Step 2 – Detect Face in the Image and Create a Region of Interest (ROI)

To detect the face in the image, we need to first convert the image into grayscale as the
OpenCV algorithm for object detection takes gray images in the input. We don’t need
color information to detect the objects. We will be using haar cascade classifier to
detect faces. This line is used to set our classifier face = cv2.CascadeClassifier(‘ path to
our haar cascade xml file’). Then we perform the detection using faces =
29
5-8-9-18-21-34-LE3

face.detectMultiScale(gray). It returns an array of detections with x,y coordinates, and

height, the width of the boundary box of the object. Now we can iterate over the faces
and draw boundary boxes for each face.

for (x,y,w,h) in faces:

cv2.rectangle(frame, (x,y), (x+w, y+h), (100,100,100), 1 )
Step 3 – Detect the eyes from ROI and feed it to the classifier

The same procedure to detect faces is used to detect eyes. First, we set the cascade
classifier for eyes in leye and reye respectively then detect the eyes using left_eye =
leye.detectMultiScale(gray). Now we need to extract only the eyes data from the full
image. This can be achieved by extracting the boundary box of the eye and then we can
pull out the eye image from the frame with this code.

l_eye = frame[ y : y+h, x : x+w ]

l_eye only contains the image data of the eye. This will be fed into our CNN classifier
which will predict if eyes are open or closed. Similarly, we will be extracting the right
eye into r_eye.

Step 4 – Classifier will Categorize whether Eyes are Open or Closed

We are using CNN classifier for predicting the eye status. To feed our image into the
model, we need to perform certain operations because the model needs the correct
dimensions to start with. First, we convert the color image into grayscale using r_eye =
cv2.cvtColor(r_eye, cv2.COLOR_BGR2GRAY). Then, we resize the image to 24*24
pixels as our model was trained on 24*24 pixel images cv2.resize(r_eye, (24,24)). We
normalize our data for better convergence r_eye = r_eye/255 (All values will be
between 0-1). Expand the dimensions to feed into our classifier. We loaded our model
using model = load_model(‘models/cnnCat2.h5’) . Now we predict each eye with our
model
lpred = model.predict_classes(l_eye). If the value of lpred[0] = 1, it states that eyes are
open, if value of lpred[0] = 0 then, it states that eyes are closed.

Step 5 – Calculate Score to Check whether Person is Drowsy

The score is basically a value we will use to determine how long the person has closed
his eyes. So if both eyes are closed, we will keep on increasing score and when eyes are
open, we decrease the score. We are drawing the result on the screen using
cv2.putText() function which will display real time status of the person.

cv2.putText(frame, “Open”, (10, height-20), font, 1, (255,255,255), 1, cv2.LINE_AA )

A threshold is defined for example if score becomes greater than 15 that means the
person’s eyes are closed for a long period of time. This is when we beep the alarm
using sound.play()
30
5-8-9-18-21-34-LE3

6.2 Retraining the model if desired accuracy is not reached

While testing, if we are not getting accurate predictions, we can use metrics score
inbuild in keras modules and check for accuracy. If the accuracy is too low or if there
is over fitting, we can increase the batch size and also increase the number of epochs
and retrain the model

6.3 Training loss and accuracy

The validation loss and accuracy of our required model should look like the above
plot

31
5-8-9-18-21-34-LE3

CHAPTER 7
CONCLUSIO
N
7.1 Driver Drowsiness Detection Execution
Let’s execute drive drowsiness detection system and see the working of our ml project.
To start the project, you need to open a command prompt, go to the directory where
our main file “drowsiness detection.py” exists. Run the script with this command.

7.2 Open Eyes Detection

The score can be seen as 0 at starting. When the person closes his eyes, score starts
increasing and that can be seen closed eye detection pic. When score is reached to the
target score, It stars giving alarm, indicating driver that he is drowsy.

32
5-8-9-18-21-34-LE3

7.3 Code execution

7.4 Closed Eye Detection

33
5-8-9-18-21-34-LE3

7.5 Conclusion
In this Python project, we have built a drowsy driver alert system that you can
implement in numerous ways. We used OpenCV to detect faces and eyes using a
haar cascade classifier and then we used a CNN model to predict the status.
7.6 Applications & Future Scope
Applications:
 The proposed system is used to avoid various road accidents caused by drowsy
driving.
 This system is used for security purpose of a driver to caution the driver if any
fire accident or any gas leakage occur.
 Drowsy driver detection methods can form the basis of a system to potentially
reduce the number of crashes related to drowsy driving.
Future Scope
 In future we can implement drowsiness detection system in aircraft in order to
alert pilot.
 The alcoholic sensor is also used for drunk drivers.

34
5-8-9-18-21-34-LE3

CHAPTER 8
BIBILOGRAPHY

[1] CNN –
https://www.coursera.org/learn/machine-learning/home/welcome
[2] Computer vision-
https://www.udemy.com/course/python-for-computer-vision-with-opencv-and-deep-
learning/
[3] Machine learning:-Machine Learning Engineering Book by Andriy Burkov
[4] Introduction to Linear Regression Analysis
[5] Book by Douglas Montgomery, Elizabeth A. Peck, and G. Geoffrey Vining
[6] Learn Python the Hard Way - Shaw, Zed
[7] Dataset:-
https://www.kaggle.com/shehzadhanif/eyes-open-closed-dataset

35