The 8th IEEE International Conference on E-Health and Bioengineering - EHB 2020

Grigore T. Popa University of Medicine and Pharmacy, Web Conference, Romania, October 29-30, 2020

Drivers’ Drowsiness Detection and Warning Systems

for Critical Infrastructures
Ioana-Raluca Adochiei1, Oana-Isabela TIRBU2, Narcis - Iulian Adochiei, Matei Pericle-Gabriel1, Ciprian- Marius
Larco1, Stefan-Mircea Mustata1, Diana Costin3
DSIAM, Technical Military Academy Bucharest, Romania, edu_ioana_raluca@yahoo.com
 DMAECS, University Politehnica of Bucharest, Romania
 Faculty of Medicine, Grigore T. Popa University of Medicine and Pharmacy, Iasi, Romania,

diana.costin@umfiasi.ro

Abstract— Road traffic accidents, due to driver fatigue, tend
to inflict high mortality rates comparing with accidents involving
rested drivers. Currently there is an emerging automotive
industry trend towards equipping vehicles with various driver-
assistance technologies. Third parties also started producing
complementary systems, including ones that can detect the
driver’s degree of fatigue, but this growing field requires further
research and development.
The main purpose of this paper is the development and
implementation of a system capable to detecting and alert, in
real-time, the driver’s level of fatigue. A system like this is
expected to make the driver aware of the assumed danger when
his level of driving and taking decisions are reduced and is
indicating a sleep break as the necessary approach. By
monitoring the state of the human eyes, it is assumed that the
signs of driver fatigue can be detected early enough to prevent a
possible road accident, which could result in severe injuries or
ultimately, in fatalities. Hence, in this work the authors are
focused on the video monitoring of the driver face, especially on
his eyes position in time, when open or closed, using a machine
learning object detection algorithm, the Haar Cascade. Two pre-
trained Haar classifiers, a face cascade, and an eye cascade were
imported from the OpenCV GitHub repository. The OpenCV
library, as well as other required packages, were installed on a
BeagleBone Black Wireless development board. The software
implementation, in order to achieve the driver’s drowsiness
detection, was made through the Python software program. The
proposed system manages to alert if the eyes of the driver are
being kept closed for more than a certain amount of time by
triggering a set of warning lights and sounds. The large-scale
implementation of this type of system will drop the number of
road accidents caused by the drivers’ fatigue, thus saving
countless lives and bringing a reduction of the socio-economic
costs associated with these tragic events.
Keywords—Driver Fatigue Detection, Driver Monitoring,
Driver Warning, Haar Cascade,
I. INTRODUCTION
According to recent statistics and reports, globally, around
1.35 million people die every year as a result of road traffic
accidents, 20-50 million of humans being injured [1-2].
Drowsy driving is one of the main factors leading to road
accidents. Sleep is a physiological process vital for the
existence of the human being. Sleep deprivation is associated
with reduced responsiveness of the organism to different
stimuli from the surrounding environment, loss of alertness,
visual impairment, difficulties in processing information,
reduced short-term memory performance, as well as other
effects. In other words, by considering these facts, it can be
stated that under the influence of fatigue, drivers’ performance
decreases drastically, at the same time significantly increasing
the risk of losing control of the vehicles, with potentially
harmful consequences [3-4].
Fatigue can be classified as active fatigue, passive fatigue,
and sleep-related fatigue, the latter referring to the quality and
quantity of the sleep, changes in the circadian rhythm, and as
well other factors [5]. Active fatigue represents the mental
exhaustion determined by the active involvement in performing
a task, meaning prolonged cognitive activity. Individuals that
have an intense work schedule experience active fatigue.
Passive fatigue is caused by monotonous tasks. Even if the
individual is not tired, a monotonous secondary task will inflict
a distraction from the main task. Prolonged driving sessions
result in an unconscious loss of engagement of the driver.
Consequently, road accidents may occur not because the driver
is fatigued, but because he is distracted from the task of
driving. During driving, the duration of blinking and the
reaction times are positively correlated with passive fatigue [6].
One study [7] suggested that the level of driving
performance is lower on straight roads, as in the case of
highways, in comparison with the roads that have frequent
changes of direction, since driving on highways is monotonous
and leads to passive fatigue. The circadian rhythm represents
the internal biological cycle of 24 hours, during which the sleep
and wakefulness states are alternated. For adults, the minimum
energy available is recorded after midnight, between 02:00 –
04:00, and afternoon, between 13:00 and 15:00 [8]. If the
driving task is performed between these intervals of time, the
possibility of experiencing sleep-related fatigue is very high.
Sleep-related fatigue and its effects on driving performance are
described in [9]. The study suggests that the driver’s ability to
drive the vehicle decreases with increasing fatigue, the
necessary breaking length depending on the time of the day and
the distance already traveled. Since sleep-related fatigue is
accentuated during nighttime, between 20:00 and 05:00, and in
the morning, between 06:00 and 10:00, the efficiency of the
drivers is compromised between these time intervals. Other
factors that play a role in driver performance are age and the

IEEE 978-1-7281-8803-4/20/$31.00 ©2020 IEEE

Authorized licensed use limited to: University of Saskatchewan. Downloaded on June 14,2021 at 16:59:10 UTC from IEEE Xplore. Restrictions apply.
driving time [11]. As the duration of the driving task increases,
the errors in lane keeping and the reaction times will also
increase. Likewise, after long hours of work, driving
performance decreased significantly [12-13].
Taking all these facts into account, the development of
driver fatigue monitoring and warning systems is crucial in
effectively combating the drowsiness state of the drivers and
reducing the number of road traffic accidents. These systems
rely on measuring the driver variables that are sensitive to
fatigue.
As shown in Fig. 1, the methods of detecting driver fatigue
are divided into direct, indirect, and methods based on the
fusion of information [14-17].
Physiological
Signals
of the Driver
The system is composed of: a Logitech C920 HD Pro web
camera, which monitors the state of the driver's eyes, a
BeagleBone Black Wireless development board loaded with
the image processing algorithm, an electrical circuit, which
generates two types of warnings, and a power source.
A. WebCam
The Logitech C920 Pro HD web camera (Fig. 3) is the
input device of the driver fatigue detection and warning system
in critical infrastructures.
The camera is positioned in front of the driver and the
images acquired are further transmitted to BeagleBone Black
Wireless component for image processing.
EEG
EMG

EOG
Direct Methods

Eye Features
Phvisical
Manifestation of Head Tilt
Drivers' Fatigue
Yawn Fig. 3. The Logitech C920 Pro HD web camera

B. The BeagleBone Black Wireless development board

Lane Departure The BeagleBone Black Wireless (Fig. 4) is a low-cost,
opensource, community-supported single-board computer, with
Indirect Driving Steering
Methods Characteristics Behaviour built-in wireless networking capability and sufficient
processing power provided by the 1 GHz ARM Cortex A-8
Vehicle Speed processor, for real-time analysis, image collection, and analysis
Bayesian software such as OpenCV. It is suitable for physical computing
Networks and smaller embedded applications [18-19].
Dempster-
Methods Based
Shafer
on
Data Fusion Theory
Information
Fusion
Fuzzy Neural
Networks

Rough Set
Theory

Fig. 1. The methods of detecting driver fatigue

II. MATERIALS AND METHODS Fig. 4. The BeagleBone Black Wireless development board

The implementation of the Driver Drowsiness Detection C. Software

and Warning System consists of hardware and software In this paper, driver drowsiness was detected through two
components, described immediately below. Fig. 2 represents Haar cascades. The program was written in Python programing
the block diagram of the proposed system. language and the algorithm that underlies the process of
Vetter Logitech C92
determining the state of the driver’s eyes was imported from
BeagleBone DRIVER
Car Charger Black Wireless HD PRO the OpenCV library.
Haar feature-based cascade classifiers represent an efficient
method for object detection and was proposed by Paul Viola
Buzzer LEDs and Michael Jones in 2001, in their paper entitled “Rapid
Object Detection Using a Boosted Cascade of Simple
Features”. It is a machine learning approach, in which a
cascade function is trained using a multitude of positive
Fig. 2. The architecture of the Driver Drowsiness Detection and Warning images, which contain the target object, and negative images,
System which don’t contain the object of interest. The trained function
is then used to detect the target objects in other images [20-21].

Authorized licensed use limited to: University of Saskatchewan. Downloaded on June 14,2021 at 16:59:10 UTC from IEEE Xplore. Restrictions apply.
 III. RESULTS
The configuration of the driver fatigue detection and Fig. 8 shows the flowchart of the driver drowsiness
warning system in critical infrastructures (Fig. 5b) was detectionand warning algorithm
obtained based on the associated diagram made in the Fritzing Starting Video
electronics design software (Fig. 5a). Capturing

Color Image

Grayscale Image

a) b) Bilateral Filtering
Fig. 5. a)Fritzing diagram of the system layout without the web camera; b)
The hardware implementation of the driver drowsiness detection and warning
system Haar Cascade for
Next, Fig. 6 and 7 demonstrate the functionality of the Face Extraction
driver fatigue detection and warning system.
In Fig. 6, the eyes are open, so the warning lights and sound
are turned off but in Fig. 7, the eyes were closed for more than Driver’s Face
2 seconds, as we decided in our experiment, so the warning
lights and sound were turned on.
Number of
No
Detected Faces is
BeagleBone Black
Wireless
equal to 1?
Eyes are
Yes
Logitech C920 Open
HD PRO Haar Cascade for
Warning Lights and
Sound Eye Extraction
Turned Off

Driver’s Eyes

Number of Detected
Vetter Car
Charger Eyes is Equal to 2? No

Yes No Number of Detected Eyes

Fig. 6. The eyes are open and the warning lights and sound are turned off is Not Equal to 2 for
more than 2 seconds?
Yes
Eyes are
Closed Driver’s Eyes are
Logitech C920 BeagleBone Driver Fatigue Detected
HD PRO Black Wireless Open
Warning Lights and
Sound Turn On
Turned On
Warning Lights and
Sound

Fig. 8. The driver drowsiness detection and warning algorithm flow

Vetter Car IV. CONCLUSIONS

Charger A higher risk of road accidents is associated with drowsy
driving, this being a real threat to human existence. Active,
passive, and sleep-related fatigue can all be experienced by a
driver. Under the influence of drowsiness, a driver is not
Fig. 7. The eyes are closed and the warning lights and sound capable of safely reaching his destination. Recently, a

Authorized licensed use limited to: University of Saskatchewan. Downloaded on June 14,2021 at 16:59:10 UTC from IEEE Xplore. Restrictions apply.
multitude of manufacturers in the automotive industry, such as
Audi, BMW, Mercedes-Benz, Toyota, and Volvo, have
installed new driver safety and assistance features in their latest
vehicles, and improved the existing ones, to prevent road
accidents. Third parties also started developing and
manufacturing driver safety systems, such as driver drowsiness
detection and warning systems, but the intervention of the
research community is needed to improve the achieved
performances. Research efforts are currently underway in the
field of driver drowsiness detection and a series of specialized
studies show promising results, but significant advances are
still needed to develop a robust, accurate, real-time technique
capable of successfully operating in all possible scenarios.
Through the proposed system, drivers will be warned about
their state of drowsiness, having to take a break to rest, thus
preventing possible road accidents.
The characteristics of the proposed driver drowsiness detection
and warning system in critical infrastructures are as follows:
Ȉ Face and eyes extraction to determine the state of the
eyes, open or closed;
Ȉ Real-time image processing and driver fatigue detection;
Ȉ Warning the driver through lights and sound;
Ȉ The system does not distract the driver. The frequency of
the buzzer was chosen such that when turned on, would
not disturb the driver;
Ȉ Portability is ensured by the system's size and the power
source, represented by the car’s charging ports.
The integration of this technology in the current application
aims to improve and maintain road safety.
A future direction for improving the proposed driver
drowsiness detection and warning system would be the
implementation of other parameters that indicate the
drowsiness state of the driver, such as head tilt or forward tilt
of the body, which could be monitored for instance through an
inertial measurement unit. Other viable options include steering
behavior, which could be monitored through a steering wheel
angle sensor, steering wheel grip force, or physiological signals
such as ECG, EEG, EOG, or GSR. Even though the
physiological parameters would significantly boost up the
performance of the system, they are generally avoided, owing
to their level of invasiveness and the discomfort created to the
driver.
REFERENCES
[1] Wanghua Deng and Ruoxue Wu. Real-Time Driver-Drowsiness
Detection System Using Facial Features. IEEE Access, 7:118727–
118738, 2019.
[2] Bejinariu, Silviu-Ioan; Luca, Ramona; Costin, Hariton, Nature-inspired
Algorithms based Multispectral Image Fusion, 9th International
Conference and Exposition on Electrical and Power Engineering (EPE)
Location: Iasi, ROMANIA, Pages: 10-15, OCT 20-22, 2016
[3] R. Manoharan and S. Chandrakala. Android OpenCV Based Effective
Driver Fatigue and Distraction Monitoring System. In 2015 International
Conference on Computing and Communications Technologies (ICCCT).
IEEE, February 2015.
[4] F. Adochiei, R. Ciucu, I. Adochiei, S. D. Grigorescu, G. Clin Serian
and M. Casian, "A WEB Platform for Rendring and Viewing MRI
Volumes using Real-Time Raytracing Principles," 2019 (ATEE),
Bucharest, Romania, 2019, pp. 1-4.
[5] Rehmat Ullah, Asif Iqbal, Bok-Hee Lee, and Kyung Sup Kwak. Smart
Real-Time Video Surveillance Platform for Drowsiness Detection Based
Authorized licensed use limited to: University of Saskatchewan. Downloaded on June 14,2021 at 16:59:10 UTC from IEEE Xplore. Restrictions apply.
on Eyelid Closure. Wireless Communications and Mobile Computing,
2019:1–9, March 2019.
[6] Jennifer F. May and Carryl L. Baldwin. Driver Fatigue: The Importance
of Identifying Causal Factors of Fatigue when Considering Detection
and Countermeasure Technologies. Transportation Research Part F:
Traffic Psychology and Behaviour, 12(3):218–224, May 2009.
[7] Moritz Korber, Andrea Cingel, Markus Zimmermann, and Klaus Ben-¨
gler. Vigilance Decrement and Passive Fatigue Caused by Monotony in
Automated Driving. Procedia Manufacturing, 3:2403–2409, 2015.
[8] Pierre Thiffault and Jacques Bergeron. Monotony of Road Environment
and Driver Fatigue: a Simulator Study. Accident Analysis & Prevention,
35(3):381–391, May 2003.
[9] Sleep Disorders and Sleep Deprivation. National Academies Press,
September 2006.
[10] Adochiei F, Edu I, Schulz S, Costin H, Voss A, Changed neuronal
coupling in schizophrenia, 4rd International Conference on E-Health and
Bioengineering (EHB), pp. 1-4., 978-1-4799-2373-1/13/$31.00, 2013,
WOS:000346672900112
[11] Clement Bougard, St´ ephane Espi´ e, Bruno Larnaudie, S´ ebastien
Mous-´ say, and Damien Davenne. Effects of Time of Day and Sleep
Deprivation on Motorcycle-Driving Performance. PLoS ONE,
7(6):e39735, June 2012.
[12] Pierre Philip, Patricia Sagaspe, Nicholas Moore, Jacques Taillard,
Andre´ Charles, Christian Guilleminault, and Bernard Bioulac. Fatigue,
sleep restriction, and driving performance. Accident Analysis &
Prevention, 37(3):473–478, May 2005.
[13] T. L. Grigorie, I. R. Edu and F. C. Adochiei, "Inertial positioning
precision improvement through sensors denoising by using wavelet
functions tuned with DTF method", 6th International Conference on
Information, Intelligence, Systems and Applications (IISA),JUL 06-08,
2015, Corfu, GREECE.
[14] T. Akerstedt, B. Peters, A. Anund and G. Kecklund “Impaired Alertness
and Performance Driving Home from the Night Shift: a Driving
Simulator Study”. Journal of Sleep Research, 14(1):17–20, March 2005.
[15] N. Sharma and V K Banga. Drowsiness Warning System Using
Artificial Intelligence. World Academy of Science, Engineering and
Technology, 43:850–852, 01 2010.
[16] Narcis Adochiei, Valeriu David, Felix Adochiei, Ioan Tudosa, “ECG
Waves and Features Extraction Using Wavelet Multi-Resolution
Analysis”, 3rd International Conference on E-Health and Bioengineering
(EHB), NOV 24-26, 2011, Univ Med & Pharm, Iasi, ROMANIA,
[17] A. Pasarica, G. Andruseac, I. Adochiei, C. Rotariu, H. Costin, F.
Adochiei, “Remote Control of an Autonomous Robotic Platform Based
on Eye Tracking”, Advances in Electrical and Computer Engineering,
Publication Year: 2016, Page(s): 95 – 100, DOI:
10.4316/AECE.2016.04015,
[18] Mohammad Amin Assari and Mohammad Rahmati. Driver Drowsiness
Detection Using Face Expression Recognition. In 2011 IEEE
International Conference on Signal and Image Processing Applications
(ICSIPA). IEEE, November 2011.
[19] I. Edu, T. L. Grigorie, F. Adochiei, C. Rotaru, and N. Jula, "Inertial
sensor signals denoising with directed transfer function," 2015 9th
International Symposium on Advanced Topics in Electrical Engineering
(ATEE), Bucharest, 2015, pp. 945-948.Romania,
[20] Yaniel Sousa Barbosa, Alexandre Zambujo Brito, Luis Brito Palma,
Paulo Sousa Gil, and Rui Azevedo Antunes. BeagleBone Black for X8-
VB Quadcopter Attitude Control. In IECON 2019 - 45th Annual
Conference of the IEEE Industrial Electronics Society. IEEE, October
2019.
[21] Zahra Tabatabaie, Rahmita Wirza, Nur Udzir, Binti Udzir, Esameil
Kheirkhah, and Putra Malaysia. A Hybrid Face Detection System Using
Combination of Appearance-Based and Feature-Based Methods.
International Journal of Computer Science and Network Security, 9, 01
2009.
[22] Ioana-Raluca Adochiei, Felix Adochiei, Costin Cepisca, George Serian,
Bogdan Enache, Florin Argatu, Radu Ciucu, "Complex Embedded
System for Stress Quantification," 2019 (ATEE), Bucharest, Romania,
2019, pp. 1-4.