FINAL REPORT

ON
“Driver’s Drowsiness Detection and Alarming System”

SUBMITTED TO

J.C. Bose University of Science and Technology, YMCA

IN THE PARTIAL FULFILLMENT OF THE REQUIREMENT FOR THE AWARD OF THE DEGREE OF

BACHELOR OF TECHNOLOGY
IN
COMPUTER SCIENCE AND ENGINEERING

Submitted to: Submitted by:

Mr. Pradeep Ravneet kaur (20CSE61)
Professor (ACEM) Sunder singh(20CSE82)

Department of Computer Science & Engineering

(December 2022)

1
 DECLARATION

We hereby declare that the project work entitled “Driver’s Drowsiness

Detection and Alarming System in Python” submitted to J.C. Bose
University of Science and Technology Faridabad, Haryana (India), is a record
of an original work done by us under the guidance of Mr. Pradeep Professor
in Computer Science and Engineering, ARAVALI COLLEGE OF
ENGINEERING AND MANAGEMENT, FARIDABAD, and this project
work is submitted in the partial fulfillment of their requirements for the award
of the degree of Bachelor of Technology in COMPUTER SCIENCE AND
ENGINEERING..

2
 ACKNOWLEDGEMENT

This project would not have taken shape, without the guidance provided by Mr.
Pradeep our Trainer who helped in the modules of our project and resolved all the
technical as well as other problems related to the project and, for always providing
us with a helping hand whenever we faced any bottlenecks, in spite of being quite
busy with their hectic schedules.
We would also like to thank our project supervisor Mr. Pradeep who gave us the
opportunity and provided us all the academic and conceptual support for our project.
Above all we wish to express our heartfelt gratitude to Ms. Sakshi Kumar, H.O.D,
CSE DEPARTMENT whose support has greatly boosted our self-confidence and
will go a long way on helping us to reach further milestones and greater heights.

3
ABSTRACT
Drowsiness of the drivers is the main cause of accidents in the world. Drowsiness
appears in situations of stress and fatigue unexpectedly and inconveniently. It may
be produced by sleep disorders, certain types of medications, and even boredom
because of driving for an extended period. The sleeping sensation reduces the level
of vigilante, producing dangerous situations and increases the probability of an
accident occurring. Those in the prime of their lives are particularly vulnerable, with
road injuries the leading cause of death among those 15-29 years old. The results in
terms of damage, injury, and death can be only as permanent. Drivers who are tired
and sleepy have delayed reactions and make bad decisions. Due to lack of sleep and
tiredness, drowsiness can occur while driving. The best way to avoid accidents
caused by drivers’ drowsiness is to detect drowsiness of the driver and warn him
before fall into sleep. To detect drowsiness many techniques like eye retina
detection, facial feature recognition has been used.
Here in this project, we propose a software for detecting driver drowsiness using eye
retina detection and pulse rate detection of the driver. In this report, we propose a
more accurate drowsiness detection method which is a hybrid approach of eye retina
detection and pulse pattern detection.
Driver’s drowsiness detection and alarming system is a car safety technology
software which helps to prevent accidents when the driver gets drowsy. Various
studies have suggested that around 20% of road accidents are fatigue related. A sleep
alarm is used in a vehicle for detecting the condition indicative of the onset of
sleepiness of a driver and for alerting the driver. An eye blink sensor is used to keep
track of the driver’s eyelid motion. If the predefined safety conditions are not met,
then the driver is alerted by producing an alarming sound from the inbuilt car
speakers primarily. It uses the concept of the Human-Computer Interaction and
Usability Test as it follows the maximization of the ease, efficiency, and safety of
the product and also to have well-designed software that has a sizeable impact on
learning time, performance speed, error rates, and personal satisfaction. Nowadays
most of the products of driver anti-sleep detection sold in the market is simply
earphone making intermittent noises, which is quite annoying and inefficient. As
such, there is a high demand for cheap and efficient driver sleep detection. Therefore,
we came up with an idea to develop a driver anti-sleep alarm system, which could
effectively meet this demand.

4
Table of Contents
INTRODUCTION.......................................................................................................................................................... 6
LITERATURE REVIEW: DROWSINESS ............................................................................................................... 7
STAGES OF DROWSINESS: ................................................................................................................................... 7
EXISTING DROWSINESS DETECTION TECHNIQUES: .................................................................................... 8
TECHNIQUES TO MONITOR DROWSINESS: ..................................................................................................... 8
STRATEGIES FOR DROWSINESS IDENTIFICATION: ...................................................................................... 9
PROBLEM STATEMENT: ......................................................................................................................................... 10
OBJECTIVE OF THE PROPOSED SYSTEM: .......................................................................................................... 11
SYSTEM ANALYSIS ................................................................................................................................................. 12
HARDWARE AND SOFTWARE REQUIREMENTS: ......................................................................................... 12
HARDWARE SPECIFICATION: ................................................................................................................................ 12
SOFTWARE SPECIFICATION: ................................................................................................................................. 12
CONSTRAINTS: ......................................................................................................................................................... 13
GENERAL CONSTRAINTS: ................................................................................................................................. 13
REQUIREMENTS: .................................................................................................................................................. 13
ANTICIPATED CONSTRAINTS:.......................................................................................................................... 14
SYSTEM DESIGN: ..................................................................................................................................................... 15
FLOW DIAGRAM: ................................................................................................................................................. 16
FEATURES OF THE PROPOSED SYSTEM: ....................................................................................................... 18
TESTING & TEST RESULTS .................................................................................................................................... 19
TESTING PLAN:..................................................................................................................................................... 19
TESTING STRATEGY: .......................................................................................................................................... 19
CONTENT TESTING: .............................................................................................................................................. 19
A. CAMERA ROTATION TEST ........................................................................................................................... 19
B. HEAD ROTATION TEST: ............................................................................................................................... 20
C. OPEN VS CLOSED EYES TEST: ...................................................................................................................... 20
SYSTEM IMPLEMENTATION ................................................................................................................................. 21
System Coding: ........................................................................................................................................................ 21
Module Snaps:.............................................................................................................................................................. 25
LIMITATIONS OF THE PROPOSED SYSTEM: ...................................................................................................... 26
CONCLUSION AND FUTURE SCOPE .................................................................................................................... 27
CONCLUSION: ....................................................................................................................................................... 27
FUTURE SCOPE:.................................................................................................................................................... 27
FUTURE ESCALATION: ........................................................................................................................................... 28
References: ................................................................................................................................................................... 29

5
INTRODUCTION
Road traffic injuries and deaths have a terrible impact on individuals, communities,
and countries. Driver exhaustion is a significant variable in an expansive number of
vehicle accidents. Late insights, assess that yearly 1,200 deaths and 76,000 injuries
can be credited to weariness related accidents. Road Accidents in countries cause
financial losses worth around billions every year. It can be seen there are around
2,400 road accidents consistently which is one death per every four hours. It has
been figured around 20% of car crashes with driver fatalities are due to driver’s
drowsiness. It was uncovered that driving execution quickly drop with expanded
tiredness which result in making more than 20% of all vehicle accidents. Less
attention heads the driver to being distracted and the likelihood of street accident
goes high. Drowsiness related accidents have all the earmarks of being more serious,
because of the higher speeds involved distraction and the driver being not able to
take any avoiding activity, or even brake, before the accident. Road accidents
involve massive costs too often overburdened health care systems occupy scarce
hospital beds consume resources and result in significant losses of productivity and
prosperity, with deep social and economic repercussions. According to the 2016
report of WHO, 1.24 million road traffic deaths occur every year. This makes it the
number one cause of death among those aged 15-29 years. This number is predicted
to increase to around 1.9 million by 2030 and to become the seventh leading cause
of death if no action is to be taken.
The improvement of innovations for recognizing or preventing tiredness of the
driver is a significant test in the field of accident preventing systems. Because of the
danger that that drowsiness presents on the road, strategies need to be created for
checking its influences. Loss of the awareness because of the tiredness causes a few
changes in the human's body and activities. These side effects and parameters
empower us to effectively measure the drowsiness level. The anti-sleep driving
alarm for people doing all night drives as well as security guards and others we have
to sit in one place for long periods of time without any stimulating interaction. The
newest high-tech way to stay awake is good whether you ever have to drive back
home after an exhausting day at work or just need to get something done and sleep
is not an option. This trusty sleep alarm will keep you at full alert and is always ready
to help if your head dozes off. This has the potential to save lives on the road. Long-
distance lorry drivers can fall asleep by driving too long hours due to the pressures
put on them to get the goods to their destination at certain times.

6
LITERATURE REVIEW: DROWSINESS

Also, a survey report and research provided by the National Highway Traffic Safety
Association (NHTS) on road accidents, about 795 people die from drowsy-driving
and 91,000 people die from motor vehicle crashes involving drowsy driving in 2017.
So, the drowsy driver is considered as one of the factors for road accidents.
Likewise, the researchers show that after 2 to 3 hours, the driver is exhausted and
the steering efficiency is also reduced. Likewise, in the early afternoon, after getting
lunch and at midnight, there is a larger risk. So, in simple terms, drowsiness is
defined as a disorder in which a person feels asleep during active hours.
 The term drowsiness is used here to refer to the state of reduced alertness,
usually accompanied by performance and psycho physiological changes, that
may result in loss of alertness or being asleep at the wheel.
 The term ―driver fatigue is also widely used to describe this condition,
especially on Police Accident Reports and in accident data files. Driver fatigue
is a significant factor in a large number of vehicle accidents.

STAGES OF DROWSINESS:
This project i.e. driver’s drowsiness detection and alarming system helped us to
research three types of people who are suffering from drowsiness: they are
categorized as awake, rapid eye movement (REM), and non-rapid eye movement
(NREM). Similarly, persons suffering from non-rapid eye movement are subdivided
into three stages. They are listed below:

Stage I The transition from awake to sleep

Stage II Light Sleep
Stage III Deep Sleep

First stage i.e. “The transition from awake to sleep” is considered as a drowsiness
state by researchers for analyzing the driver’s drowsiness. Drowsing when driving
is dangerous as it can lead to some serious cases too. The driver should be monitored
and alert when the driver is in the drowsy condition in order to avoid road injuries.

7
EXISTING DROWSINESS DETECTION TECHNIQUES:

 Carnegie-Mellon Research Institute developed the PERCLOS Systems.

PERCLOS (percentage closure) is defined as the measurement of the
percentage of time the pupils of the eyes are 80% or more occluded over a
specified time interval. It has been found that PERCLOS is a reliable
measure in detecting drowsiness.
 Head position metrics: Systems have been devised such that the head
position of the driver is detected and when the head leaves the headrest past
a certain threshold percentage, the system alerts the driver.

TECHNIQUES TO MONITOR DROWSINESS:

Some of the techniques used to monitor and alert are vehicle-based, behavioral-
based, and physiological-based.

 Vehicle-Based: In this type, different mechanical parts of the vehicle are

included, such as steering, the motion of the wheel, lane detection, speed of
the vehicle, and many. All these mechanical components are constantly
monitored and if any of the components have exceeded the threshold value,
the device determines that the driver is in a drowsy state. Some of the systems
like radar, GPS, infrared cameras to alert the driver and provide some
instruction not to drowse. A brand like Ford, Tesla, Infiniti, Volvo, and much
other company provides features to alert the driver. In this vehicle-Based
system, automatic identification of lanes provides human drivers with
assistance. This feature serves to alert a driver if the car is actually steering
off track.
 Behavioural-based: It is also one of the techniques by which the camera can
detect the driver’s drowsiness. In this behavioural-based different behaviour
of head such as the position of the head, eye closing or blinking, yawing, etc.
All of the behaviour is continuously tracked and observed by a camera.
Drowsiness can be measured through the distance between eyelids at three
stages. The average number of blinks per minute was taken from the estimate,
as long as the driver grows drowsier. The technique of detecting drowsiness
tracks the mouth and yawning behaviour along with closure and opening of
the eyes. The driver is alerted when any of those signs are identified and the
driver wakes up. And for facial recognition, the viola-jones object detection
algorithm may be used.

8
  Physiological-based: Physiological signals such as ECG (Electrocardiogram)
and EOG (Electrooculogram) are used in the physiological category. The
ECG is used to monitor the heart rate and also to check the various potential
conditions for drowsiness. And the EOG is used to measure the different
electrical functions of the brain. So, the system observes the heart rate, pulse
rate, and brain activity to detect the drowsiness and alert the driver.

STRATEGIES FOR DROWSINESS IDENTIFICATION:

Different strategies for drowsiness identification can be partitioned into two general
classifications.
 The techniques in the first gathering recognize the level of the tiredness
focused around the physiological changes in the body. Eye status, speech
properties, time interval between two yawning, head position, sitting carriage,
heart rate, and brain signals are simply a couple of illustrations of the
strategies in the first classification.
 Techniques in the second category estimate the driver's drowsiness level by
following these progressions. Steering angle, distance from the following
vehicle, lateral position of the vehicle, longitudinal speed, longitudinal
speeding up, and lane departure are utilized as a part of the technique of the
second classification.

9
PROBLEM STATEMENT:

Driver exhaustion/fatigue has been the main issue for countless mishaps due to
tiredness, tedious road condition, and unfavorable climate situations. Every year, the
National Highway Traffic Safety Administration (NHTSA) and World Health
Organization (WHO) have reported that approximately 1.35 million people die due
to vehicle crashes across the world. Generally, road accidents mostly occur due to
inadequate way of driving. These situations arise if the driver is addicted to alcohol
or in drowsiness. The maximum types of lethal accidents are recognized as a severe
factor of tiredness of the driver. When drivers fall asleep, the control over the vehicle
is lost.

There is a need to design smart or intelligent vehicle system through advanced

technology. Late insights assess that yearly 1,200 deaths and 76,000 injuries can be
credited to weariness-related accidents. Less attention leads the driver to be
distracted and the likelihood of a street accident goes high. Drowsiness-related
accidents have all the earmarks of being more serious, because of the higher speeds
involved distraction and the driver being not able to take any avoiding activity, or
even brake, before the accident.

The improvement of innovations for recognizing or preventing tiredness of the

driver is a significant test in the field of an accident preventing systems. Because of
the danger that that drowsiness presents on the road, strategies need to be created for
checking its influences. Loss of awareness because of tiredness causes a few changes
in the human body and activities. These side effects and parameters empower us to
effectively measure the drowsiness level. Every year, 1.25 million people around the
world die due to road crashes – a global problem that the World Health Organization
(WHO) says is both predictable and preventable. Most of the victims come from low
and middle-income countries.

A deep learning-based system is designed to avoid countless mishaps due to drowsy

drivers’ behavioral and psychological changes by focusing on driver’s eye
movements. In addition to monitoring the intensity of the collisions impacts during
road accidents, it is also required to keep records of the location for taking supportive
action. This system implements a mechanism to alert the driver on the condition of
drowsiness or daydreaming. A camera monitors the driver’ eye blinking, eye
closure, face detection, head posture, etc. with face landmark algorithm and
Euclidean distance in the behavioral-based approach.

10
OBJECTIVE OF THE PROPOSED SYSTEM:

Drowsiness is a serious concern when driving and can cause accidents because it
impairs the elements of human performance that are critical to safe driving: slower
reaction time, reduced vigilance, deficits in information processing. Driver fatigue
is a significant factor in a large number of vehicle accidents. The development of
technologies for detecting drowsiness at the wheel is a major challenge in the field
of accident avoidance systems. Because of the hazard that drowsiness presents on
the road, methods need to be developed for counteracting its affects.
The Drowsiness detection and alarming system deals with automatic driver
drowsiness detection based on visual information and Artificial Intelligence.
 The objectives of this project is to develop a system that can detect the
sleepiness of the driver and make alarms accordingly which aims to prevent
the drivers from drowsy driving and create a safer driving environment.
 The main aim of this is to develop a drowsiness detection system by
monitoring the eyes it is believed that the symptoms of driver fatigue can be
detected early enough to avoid a car accident. Detection of fatigue involves
the observation of eye movements and blink patterns. The analysis of face
images is a popular research area with applications such as face recognition,
and human identification security systems.
 This project is focused on the localization of the eyes, which involves looking
at the entire image of the eye, and determining the position of the eyes, by a
self-developed image-processing algorithm.
 The purpose of the drowsiness detection system is to aid in the prevention of
accidents passenger and commercial vehicles. The system will detect the early
symptoms of drowsiness before the driver has fully lost all attentiveness and
warn the driver that they are no longer capable of operating the vehicle safely.
This device will not, however, guarantee that the driver will be fully awakened
and that an accident will be avoided.
 It is simply a tool for improving driver safety; focusing primarily on long-haul
truck drivers, nighttime drivers, people driving long distances alone or people
suffering from sleep deprivation.
 The aim of this system is to locate, track, and analyze both the drivers face
and eyes to compute a drowsiness index, where this real-time system works
under varying light conditions (diurnal and nocturnal driving).

11
SYSTEM ANALYSIS
HARDWARE AND SOFTWARE REQUIREMENTS:

HARDWARE SPECIFICATION:

Processor: Intel Pentium 4 or later or compatible

Processor Speed: 2.4 GHZ

RAM: 256 MB or more

Hard disk: 10 GB or more

Mouse: optical mouse
Keyboard: Logitech keyboard
Web Camera : used to get the video feed.

SOFTWARE SPECIFICATION:

1.Operating System: Windows 7, 8, 9, 10, XP

2. Front-End: Python
Requirement Analysis
 Python: Python is the basis of the program that we wrote. It utilizes many of
the python libraries.
 Libraries:
 Numpy: Pre-requisite for Dlib
 Scipy: Used for calculating Euclidean distance between the eyelids.
 Playsound: Used for sounding the alarm
 Dlib: This program is used to find the frontal human face and estimate its
pose using 68 face landmarks.
 Imutils: Convenient functions written for Opencv.
 Opencv: Used to get the video stream from the webcam, etc.

12
CONSTRAINTS:
GENERAL CONSTRAINTS:

 Space: The system needs to be implemented in a space-efficient manner. It

must not interfere with the existing controls of the car.
 Power: There will be a limited power source so the system needs to designed
so that it can operate properly on limited power requirements.

REQUIREMENTS:

The driver drowsiness detection system described in this paper must comply with
the following main requirements:

• Algorithmically simple and easy to implement- We chose to rely on off-the-shelf

solutions for most stages, based on the popularity and success of the associated
algorithms (e.g., Viola-Jones face detector, Support Vector Machine classifier).

• Easily portable to different platforms- The application must run on a mobile

device (e.g., Android-based smartphone) mounted on the vehicle’s dashboard.
Ideally, it should be easily portable to other (e.g., iOS-based) mobile devices of
comparable size and computational capabilities.

• Computationally non-intensive- Since (near) real-time performance is required,

algorithms must be optimized to ensure continuous monitoring of driver’s state
without excessive burdening of the device’s main processor. As a side benefit,
battery consumption is reduced as well.

• Accuracy- One of the main challenges of designing such a system is related to the
fact that both type I and type II errors are highly undesirable, for different reasons:

1. type I errors (false positives) will annoy the driver and reduce their willingness
to use the system (due to excessive false alarms)
2. type II errors (false negatives) can have literally catastrophic consequences
and defeat the purpose of the entire system.

• Robustness- The system must be tolerant to modest amounts of lighting variations,

relative camera motion (e.g. due to poor road conditions), changes to the driver’s
visual appearance (even in the course of a session, e.g., by wearing/removing a hat
or sunglasses), camera resolution and frame rates, and different computational
capabilities of the device’s processors.

 Reliability-The solution should reliably detect drowsiness so that it can serve its
purpose as a system for promoting driver safety.
13
  Real-time response- The operation of a vehicle can involve relatively high
speeds, a system that cannot detect drowsiness and warn that driver promptly
can lead to serious consequences.
 Unobtrusive- It is very important that the solution is as transparent to the
driver as possible.
 Economical- Existing solutions to this problem are available today but the
effective ones are usually too expensive for widespread implementation.
 Flexible- To be effective, the solution should be designed so as to
accommodate for all types of users, in terms of physical attributes.

ANTICIPATED CONSTRAINTS:

Some of the anticipated constraints and limitations faced by the proposed system
include:

• Lighting conditions- Frequent and drastic change in darkness or brightness of a

scene, which may happen even during the shortest driving intervals, have been
proven to be a significant challenge for many computer vision algorithms.

• Camera motion- Poor road conditions as well as a more aggressive style of driving
can introduce significant amount of vibrations and discomfort to the driving
experience. Those vibrations can be passed onto the camera and cause distortion in
the images which can significantly skew the results and decrease the overall
performance of the system.

• Relative positioning of device- The camera must be positioned within a certain

range from the driver and within a certain viewing angle. Every computer vision
algorithm has a “comfort zone” in which it performs the best and most reliably. If
that comfort zone is left, performance can be dropped significantly.

• Hardware and software limitations- Typical mobile devices have one or two
processor cores, reduced working memory and tend to work on lower clock
frequencies, compared to their desktop counterparts. The reason for all of this is to
reduce the energy consumption but it creates a significant obstacle in designing this
type of system.

• Driver cooperation- Last, but certainly not least, all driver drowsiness detection
systems assume a cooperative driver, who is willing to assist in the setup steps, keep
the monitoring system on at all times, and take proper action when warned by the
system of potential risks due to detected drowsiness.

14
SYSTEM DESIGN:
A Drowsy Driver Detection System uses a specially designed deep learning based
algorithm that points directly towards the driver’s eyes in order to detect fatigue. The
development of technologies for detecting or preventing drowsiness at the wheel is
a major challenge in the field of accident avoidance systems. Because of the hazard
that drowsiness presents on the road, methods need to be developed for
counteracting its affects.
The aim of this is to develop a prototype drowsiness detection system. The focus
will be placed on designing a system that will accurately monitor the open or closed
state of the driver’s eyes in real-time. By monitoring the eyes, it is believed that the
symptoms of driver fatigue can be detected early enough to avoid a car accident.
 In such a case when fatigue is detected, a warning signal is issued to alert the
driver. The system deals with using information obtained for the binary
version of the image to find the edges of the face, which narrows the area of
where the eyes may exist.
 Once the face area is found, the eyes are found by computing the horizontal
averages in the area. Taking into account, the knowledge that eye regions in
the face present great intensity changes, the eyes are located by finding the
significant intensity changes in the face.
 Once the eyes are located, measuring the distances between the intensity
changes in the eye area determine whether the eyes are open or closed. A large
distance corresponds to eye closure.
 If the eyes are found closed the system draws the conclusion that the driver is
falling asleep and issues a warning signal. The system works under reasonable
lighting conditions. To overcome the limitations of current drowsiness
detection methods, this proposed project aims to develop a real-time, easy
implementable, nonintrusive, and accurate drowsiness detection system.
A sleep detection and driver alert apparatus has a compact housing that can be placed
in any automobile. It contains an eye blink sensor for monitoring the effects of early
impending sleep. This will alert the driver in case the system detects drowsiness
while the vehicle is moving, in case the driver does not respond to the alert
mechanisms, then the brakes are actuated to stop the vehicle in order to avoid a
collision. This can take place due to sleepiness or any medical emergency of the
driver. Thus, the main objective of this project is to avoid fatal accidents.

15
FLOW DIAGRAM:

In this project, we have used the non-intrusive method of detecting the drowsiness
of the driver using webcam by getting a video image of the driver’s mouth and
eyes with the help of image processing techniques. We will use Python and
OpenCV to build a system that can detect the closed eyes of drivers and alert them
if ever they fall asleep while driving. OpenCV will monitor and collect the
driver’s images via a webcam that was attached and feed them into the deep
learning model and then the model will classify the driver’s eyes as ‘open’ or
‘closed.’

 It uses state of eyes and position of irises monitored with the inputs like
frequency of eye blink and duration of eye closing.

16
  Template matching technic is used to match the open and close of eyes of the
driver with the drowsy conditions. One of the important conditions of
drowsiness is yawn.
 Using this method, the changes in driver’s head position, sagging movement
and state of the eyes are measured. The symptoms are compared with the
images in video and using human features. This software is used to monitor
the visual behavior of the driver’s vigilance.
 A certain threshold is established by eye-tracking in order to identify the
drowsy eye of a human.
 The alarm begins when the human is completely awakened when the eye
becomes lower than the threshold as defined. Then again, it continues to train
the people.
 Furthermore, the alert sound begins if the people are overwhelmed and are
not focused on driving and unable to trace their faces. There is an end button
at the end to close the camera mode.

This item has the potential to keep them awake or at least to tell them when they are
overtired and need to stop driving. This study intends to develop a device or a system
that will help the driver in minimizing road accidents

17
FEATURES OF THE PROPOSED SYSTEM:
This system has many features that make it unique and functional. These features
include:
 It is simply a tool for improving driver safety, focusing primarily on long-haul
truck drivers, night time drivers, people driving long distances alone or people
suffering from sleep deprivation.
 This system implements a mechanism to alert the driver on the condition of
drowsiness or daydreaming.
 A camera monitors the driver’ eye blinking, eye closure, face detection, head
posture, etc. with face landmark algorithm and Euclidean distance in the
behavioral-based approach.
 A sleep alarm is used in a vehicle for detecting the condition indicative of the
onset of sleepiness of a driver and for alerting the driver.
 An eye blink sensor is used to keep track of the driver’s eyelid motion and
provides eyelid distance tracking to detect the sleepiness.
 The system will detect the early symptoms of drowsiness before the driver
has fully lost all attentiveness and warn the driver that they are no longer
capable of operating the vehicle safely.
 The drowsiness detection system will detect that a person’s eyes are closed
for a few seconds and will alert the driver when it occurs. This device will
not, however, guarantee that the driver will be fully awakened and that an
accident will be avoided.
 Provides Daytime and night detection.
 Sound and flashing LED warning system to redraw driver’s attention
 Little inference and potential hazard to driver’s normal driving
 Drowsiness detection system provides Real time image processing more than
1 frame/second
 It has affordable cost of the application.
 Sleepiness detection is efficient and alarms will be generated only when
demanded.
 Enhanced algorithm to ensure the darkness detection ability

18
TESTING & TEST RESULTS

TESTING PLAN:

Any system, to be successful, must be thoroughly tested, and well managed test plan
should be prepared before actual testing is being performed. “Modules” have been
developed and need to be tested in a manner that can reduce occurring of defects as
low as possible.

TESTING STRATEGY:
CONTENT TESTING:

In order to fulfill all of the challenging prerequisites we have to test out limitations
of our chosen off-the-shelf components. Some of our preliminary tests were devised
to help us out with that. Errors in Project content can be as trivial as minor
typographical error as incorrect information, improper organization or validation of
intellectual property laws. Content Testing attempt to uncover this and many other
problems before the user encounter them
The goal of the devised set of experiments is to test the basic performance in various
ways. for differentiating between open and closed eyes we decided to go with a
proven two-class Support Vector Machine (SVM) classifier [7]. SVM is a non-
probabilistic binary linear classifier. It takes a set of input data and predicts, for each
given input, which of two possible classes forms the output. This is perfect since our
basic problem can be defined as two class problem: open eyes vs. closed eyes.

A. CAMERA ROTATION TEST: The aim of this test is to find out how does
movement of the camera and change in the viewing angle relative to the driver
influences the performance of used face/eye detection algorithm as well as the
consistency of its detection.
If we define an angle of 0 degrees to be when the camera is directly facing
the driver, we can conclude that the range in which it was reliably detecting
drivers eyes falls approximately within 25 degree viewing angle in both
direction from a driver. This result is very encouraging since this means that
we can position the camera of the system conveniently on car’s dashboard
(usually has available slot for attaching devices such as phones etc.) which is
within driver’s reach.

19
B. HEAD ROTATION TEST: In this test the camera stays directly facing the
driver while the driver rotates his head to the right, returns to the starting
position and then rotates the head to the left and back to center again. We are
again looking to see what is the angle of breaking for our head/eye detection
algorithm and what is its consistency across the frames.

C. OPEN VS CLOSED EYES TEST: To test the behavior of our chosen

classification algorithm we decided to go with the simple solution of using
cropped image regions containing eyes and to feed that set of pixels as an
input to the SVM classification algorithm. A special video containing 571
frames of a driver sitting in a drivers position is used. Driver is moving
minimally. Every frame is manually marked as containing open or closed
eyes. Open and closed eye samples are roughly the same in number, 304
containing open eyes and 267 containing closed eyes. 70 % of randomly
selected samples were used to train the SVM model on which the remaining
30% of the samples were tested. Using SVM algorithm, our results showed
96 % success rate. Thus, building a system in which an SVM eye model is
user specific is encouragingly good idea. It can simplify significantly our
system as well as provide a dose of robustness and reliability to it. Some of
the classification results are shown as:

20
SYSTEM IMPLEMENTATION

System Coding:
from scipy.spatial import distance as dist
from imutils.video import VideoStream
from imutils import face_utils
from threading import Thread
import numpy as np
import argparse
import imutils
import time
import dlib
import cv2
import os

def alarm(msg):
global alarm_status
global alarm_status2
global saying

while alarm_status:
print('call')
s = 'espeak "' + msg + '"'
os.system(s)

if alarm_status2:
print('call')
saying = True
s = 'espeak "' + msg + '"'
os.system(s)
saying = False

def eye_aspect_ratio(eye):
A = dist.euclidean(eye[1], eye[5])
B = dist.euclidean(eye[2], eye[4])

C = dist.euclidean(eye[0], eye[3])

ear = (A + B) / (2.0 * C)

return ear

21
def final_ear(shape):
(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"]
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye"]

leftEye = shape[lStart:lEnd]
rightEye = shape[rStart:rEnd]

leftEAR = eye_aspect_ratio(leftEye)
rightEAR = eye_aspect_ratio(rightEye)

ear = (leftEAR + rightEAR) / 2.0

return (ear, leftEye, rightEye)

def lip_distance(shape):
top_lip = shape[50:53]
top_lip = np.concatenate((top_lip, shape[61:64]))

low_lip = shape[56:59]
low_lip = np.concatenate((low_lip, shape[65:68]))

top_mean = np.mean(top_lip, axis=0)

low_mean = np.mean(low_lip, axis=0)

distance = abs(top_mean[1] - low_mean[1])

return distance

ap = argparse.ArgumentParser()
ap.add_argument("-w", "--webcam", type=int, default=0,
help="index of webcam on system")
args = vars(ap.parse_args())

EYE_AR_THRESH = 0.3
EYE_AR_CONSEC_FRAMES = 40
YAWN_THRESH = 25
alarm_status = False
alarm_status2 = False
saying = False
COUNTER = 0

print("-> Loading the predictor and detector...")

# detector = dlib.get_frontal_face_detector()

22
detector = cv2.CascadeClassifier("haarcascade_frontalface_default.xml") # Faster
but less accurate
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')

print("-> Starting Video Stream")

vs = VideoStream(src=args["webcam"]).start()
# vs= VideoStream(usePiCamera=True).start() //For Raspberry Pi
time.sleep(1.0)

while True:

frame = vs.read()
frame = imutils.resize(frame, width=450)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

# rects = detector(gray, 0)
rects = detector.detectMultiScale(gray, scaleFactor=1.1,
minNeighbors=5, minSize=(30, 30),
flags=cv2.CASCADE_SCALE_IMAGE)

# for rect in rects:

for (x, y, w, h) in rects:
rect = dlib.rectangle(int(x), int(y), int(x + w), int(y + h))

shape = predictor(gray, rect)

shape = face_utils.shape_to_np(shape)

eye = final_ear(shape)
ear = eye[0]
leftEye = eye[1]
rightEye = eye[2]

distance = lip_distance(shape)

leftEyeHull = cv2.convexHull(leftEye)
rightEyeHull = cv2.convexHull(rightEye)
cv2.drawContours(frame, [leftEyeHull], -1, (0, 255, 0), 1)
cv2.drawContours(frame, [rightEyeHull], -1, (0, 255, 0), 1)

lip = shape[48:60]
cv2.drawContours(frame, [lip], -1, (0, 255, 0), 1)

if ear < EYE_AR_THRESH:

COUNTER += 1

23
 if COUNTER >= EYE_AR_CONSEC_FRAMES:
if alarm_status == False:
alarm_status = True
t = Thread(target=alarm, args=('wake up sir',))
t.deamon = True
t.start()

cv2.putText(frame, "DROWSINESS ALERT!", (10, 30),

cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

else:
COUNTER = 0
alarm_status = False

if (distance > YAWN_THRESH):

cv2.putText(frame, "Yawn Alert", (10, 30),
cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
if alarm_status2 == False and saying == False:
alarm_status2 = True
t = Thread(target=alarm, args=('take some fresh air sir',))
t.deamon = True
t.start()
else:
alarm_status2 = False

cv2.putText(frame, "EAR: {:.2f}".format(ear), (300, 30),

cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
cv2.putText(frame, "YAWN: {:.2f}".format(distance), (300, 60),
cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF

if key == ord("q"):
break

cv2.destroyAllWindows()
vs.stop()

24
Module Snaps:

Non-drowsy Person: Non-drowsy state

Drowsy Person: Different aspect ratio at drowsy state

25
LIMITATIONS OF THE PROPOSED SYSTEM:
While researching and building a project, fatigue is observed using image
recognition and computer vision techniques.

 Keeping the user’s head down causes fatigue detection inaccuracy

 There is no dual user usability available
 The eye-detection algorithm which plays an important role in detecting
drowsiness creates a high degree of misunderstanding when tested with
different positions of eyes.
 It is only prevention, not a solution

26
CONCLUSION AND FUTURE SCOPE
CONCLUSION:

 A non-invasive system to localize the eyes and monitor fatigue was

developed.
 Information about the eyes position is obtained through various self-
developed image processing algorithms. During the monitoring, the system is
able to decide if the eyes are opened or closed. When the eyes have been
closed for too long, a warning signal is issued.
 In addition, during monitoring, the system is able to automatically detect any
eye localizing error that might have occurred. In case of this type of error, the
system is able to recover and properly localize the eyes.
 Image processing achieves highly accurate and reliable detection of
drowsiness.
 Image processing offers a non-invasive approach to detecting drowsiness
without the annoyance and interference.
 A drowsiness detection system developed around the principle of image
processing judges the driver’s alertness level on the basis of continuous eye
closures. With 80% accuracy, it is obvious that there are limitations to the
system.

FUTURE SCOPE:

 This project would have a bright future in the field where more safety
enchantments are made electro-mechanically in the automotive sector.
 It could be successfully applicable to lorries, trucks and other vehicles that are
being used to travel at night. Using this set-up, we can easily detect the fatigue
of the driver and can prevent the accidents due to fatigue of the driver.
 More concentrated research in this field could help upgrade technologies that
could reduce the accidents caused by the ignorance of the drivers around 80%.

27
FUTURE ESCALATION:
With the advent of technologies and innovations, the proposed system could be
further developed. The following are the new enhancements that can be introduced
to enhance the program and the accessibility of the end-user in the future.
 Both Android and IOS will be compliant with this program.
 The threshold can be set by users according to the size of their eyes
 The user can be traced in an emergency case with the access location
 More can be applied to alarm sound so that the user can select the sound
 If the user does not respond for a longer time, emergency call and message
will be sent automatically
 Use OpenGL to control the frame rate more accurately
 To achieve a higher accuracy at night
 Use parallel programming such as CUDA to make code faster and more
efficient
 Use bash script to enable our program to auto start after booting.
 Use parallel programming and multi thread to handle image capturing,
sending control signal, and running algorithm separately.
 Capture individual driver’s steering activity while drowsy
 Conduct additional simulator experiments to validate the algorithm, test
additional road conditions, and test a more diversified group of drivers to
understand the characteristics of impaired driving under the influence of
alcohol, motion sickness, stress, or inattention.
 Refine the Algorithm based on the road test data and conduct research on
warning systems integrated with the detection system.
 Incorporating motion capture and EEG facilities to our system. The motion
capture system will enable analyzing the upper torso movements. In addition,
the EEG will provide a ground-truth for drowsiness.

28
References:

 www.google.com

 www.slideshare.net

 www.youtube.com

 https://www.researchgate.net

 http://www.ieomsociety.org/brazil2020/papers/700.pdf

 https://dspace.mit.edu

 http://repository.rjt.ac.lk/bitstream

 https://arxiv.org/pdf/2012.05705

 www.udemy.com/topic/sleepdetectionsystem

 https://uwaterloo.ca/systems-design-engineering/current-undergraduate-
students/courses/workshop-projects/fourth2002/drowsiness-detection-
system#:~:text=The%20purpose%20of%20the%20drowsiness,of%20operati
ng%20the%20vehicle%20safely.

 https://data-flair.training/blogs/python-project-driver-drowsiness-
detection-system/

 https://www.analyticsvidhya.com/blog/2022/05/drowsiness-detection-
system/

29