4444 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO.

10, OCTOBER 2020

A Comprehensive Review of Driver Behavior

Analysis Utilizing Smartphones
Teck Kai Chan, Cheng Siong Chin , Senior Member, IEEE, Hao Chen , Member, IEEE,
and Xionghu Zhong, Member, IEEE

Abstract— Human factors are the primary catalyst for traffic TABLE I
accidents. Among different factors, fatigue, distraction, drunken- FACTORS I NFLUENCING D RIVING B EHAVIOUR
ness, and/or recklessness are the most common types of abnormal
driving behavior that leads to an accident. With technological
advances, modern smartphones have the capabilities for driving
behavior analysis. There has not yet been a comprehensive review
on methodologies utilizing only a smartphone for drowsiness
detection and abnormal driver behavior detection. In this paper,
different methodologies proposed by different authors are dis-
cussed. It includes the sensing schemes, detection algorithms,
and their corresponding accuracy and limitations. Challenges
and possible solutions such as integration of the smartphone
behavior classification system with the concept of context-aware,
mobile crowdsensing, and active steering control are analyzed.
The issue of model training and updating on the smartphone and
cloud environment is also included.
Index Terms— Driving behavior analysis, detection algorithms,
drowsiness detection, abnormal driving detection, smartphone Among the different factors, it was found that fatigue,
telematics solution. distraction, drunkenness, and/or recklessness are the most
I. I NTRODUCTION common reasons for abnormal driving leading to acci-
dents [15], [16]. On the other hand, studies have also shown
T ECHNOLOGIES advances in automobiles have brought
humans great benefits and convenience with a high level
of mobility. However, such invention causes several social and
if passenger can provide timely warning to the driver, he is
less likely to cause any injury related collision [17]. Hence,
environmental problems such as traffic congestion, traffic acci- it is critical for active systems such as the Advanced Driver
dents, and environmental pollutions [1]. Undeniably, the loss Assistance System (ADAS) or Intelligent Driver Assistance
of life or even injuries caused by traffic accidents are the most Systems (IDAS) to provide accurate and timely warning to
undesirable problems to avoid. Yet, it has been discovered the driver in the event of potential danger in order to have
that the main catalyst of traffic accidents is due to human ample time to escape from unwanted situation.
factors [2]–[4]. Assistance systems have to categorize an unsafe driver
A driver’s behavior can be defined as the way a driver or detect abnormal driving patterns to enable such warning
responds to his or her existing driving state (e.g., the vehicle’s system. When a driver is drowsy, distracted, reckless, or drunk,
speed or distance to the vehicle in front) by performing a the driver usually exhibit certain changes in behavior or
certain action (e.g., accelerate or steer) [5]. However, different body movement, resulting in an abnormal driving pattern.
drivers differ in how they accelerate and brake pedals, how A drowsy driver usually exhibits symptoms such as rapid
they turn the steering wheel, and the distance they keep when and constant blinking, nodding or swinging their head, and
following a vehicle [6]. frequent yawning [18]. On the other hand, a distracted driver
Furthermore, such behavior is also susceptible to external may be caused by some unpredicted reasons that occur at
factors as shown in Table 1 [7]–[14]. random [19]. Such a driver has an increased response time to
external stimuli and may unintentionally slow down or have
Manuscript received March 25, 2019; revised June 30, 2019 and sudden longitudinal and transversal movements [16], [20].
September 1, 2019; accepted September 6, 2019. Date of publication
September 19, 2019; date of current version October 2, 2020. The A drunk driver intoxicated by alcohol usually has the habit of
Associate Editor for this article was M. Brackstone. (Corresponding author: sudden acceleration or deceleration with a delayed response.
Cheng Siong Chin.) A reckless driver can be considered similar to a drunk driver,
T. K. Chan and C. S. Chin are with the Faculty of Science, Agriculture and
Engineering, Newcastle University in Singapore, Singapore 599493 (e-mail: such that the driver may be affected by emotional or external
t.k.chan2@newcastle.ac.uk; cheng.chin@newcastle.ac.uk). factors that caused him/her to accelerate or decelerate the
H. Chen and X. Zhong are with the College of Computer Science and vehicle suddenly and violates the speed limit awake [16].
Electronic Engineering, Hunan University, Changsha 410082, China (e-mail:
haochen@hnu.edu.cn; xzhong@hnu.edu.cn). As seen in Fig. 1, there are six types of abnormal driving
Digital Object Identifier 10.1109/TITS.2019.2940481 patterns. Weaving can be described as serpentine driving or
1524-9050 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
Fig. 1. Different Types of Abnormal Driving Pattern. a) Weaving, b)
Swerving, c) Side slipping, d) Fast U-turn, e) Turning with Wide Radius,
f) Sudden Braking [21].
driving in S-shape. Swerving can be described as an abrupt
redirection when driving along a generally straight course.
Sideslipping occurs when a driver deviates from the normal
driving direction. Fast U-turn is a U-turn at high speed.
Turning with a wide radius is almost similar to a fast U-turn.
However, the vehicle would drive along a curve with a big
radius that can drift out of the lane. Sudden braking is similar
to an emergency brake, but a sudden brake is usually done
with no apparent reason [21].
Driver monitoring can be categorized into direct or indirect
driver monitoring. The former includes monitoring of heart
rate and/or driver body movement using different sensors. The
latter includes analysis of facial expression, pedal and steering
activities, and reactions to certain events [15], [22], [23].
As reviewed by Kaplan et al. [15], direct driver monitoring has
a high success rate, especially through the monitoring of the
driver’s electroencephalogram (EEG). However, the method is
not user-friendly as it possesses an extremely intrusive nature
where it is mandatory for the driver to wear an electrode
helmet while driving to collect data. Although there exists
a nonintrusive alternative such as placing electrodes on the
steering wheel or in the driver’s seat, it comes with lower
accuracy due to improper electrode contact. Furthermore,
personal differences, such as gender and personality, influence
the signals that complicate the detection process.
Therefore, the most viable and attractive solution would
be the use of indirect driving monitoring. Telematics boxes
have been introduced in the market dedicated to such purpose.
However, the main drawbacks of such systems are their asso-
ciated high cost and low customer acceptance that limit wide
and rapid platform deployment [15], [24]. On the other hand,
smartphones are cheap and easy to deploy for driver behavior
analysis. A modern smartphone is equipped with multiple sen-
sors that include an accelerometer, gyroscope, magnetometer,
microphone, cameras, thermometer, and Global Positioning
System (GPS). Thus, the smartphone has the potential of
monitoring not just the tracking of vehicles’ acceleration but
also the driver’s distraction and proximity to external obstacles
or objects. Moreover, the network capability of a smartphone
allows fast and reliable transmission of sensors’ data [25].
While there has been an increasing interest in driver behav-
ior analysis using smartphones, there has not yet been a com-
prehensive review in this area. Kaplan et al. [15] provided a
review focused on the detection of the driver’s drowsiness and
distraction through the visual and non-visual approach. How-
ever, the details on utilizing smartphone solution were limited.
Ferreira Jr., et al. [26] provided an investigation with differ-
ent Android smartphone sensors and classification algorithm.
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4445
Fig. 2. Categories of Methodology Reviewed.
However, the paper only investigated four different machine
learning algorithms, namely, Artificial Neural Network
(ANN), Support Vector Machine (SVM), Random Forest (RF),
Bayesian Network (BN), missing out on the latest and exciting
methodology proposed in the literature. On the other hand,
Handel et al. [27] focused on the area of smartphone-based
insurance telematics, where they only provided some form of
comparison between different smartphones for harsh braking
detection. Meiring and Myburgh [19] review paper focused on
the general picture of driving behavior analysis, and the types
of algorithms applied thus lacking emphasis on smartphone
solutions.
Additionally, Vlahogianni and Barmpounakis [28] provided
a general overview of driving behavior analysis using a
smartphone. It missed out the methodology proposed in the
literature. Although Engelbrecht et al. [29] provided a survey
of smartphone-based sensing in vehicles for intelligent trans-
portation system applications, it only provided a review on
a limited number of papers, and only a few were targeted
at driver behavior analysis using a smartphone. On the other
hand, Martinez et al. [7] focused on the types of machine
learning algorithms used in the characterization of driving
styles. disregarding the types of platform where the algorithms
were applied and their integration to the Advanced Driver
Assistance System (ADAS). Wahlstrom et al. [30] review was
comprehensive, but the paper focused on the smartphone’s role
as a measuring instrument and user-interactive services instead
of drivers’ behavior classification.
Hence, this paper aims to fill the current gaps by providing
contributions that includes the following.
• A comprehensive review on non-intrusive smartphone
solution for driving behavior analysis. Selected publica-
tions for review are based on the condition that they only
utilized smartphone in their application.
• This paper also includes the identification of different
limitations of different methodologies with an in-depth
discussion on the challenges of smartphone solution and
the possibility of integration with other technology.
The organization of the paper is as follow. A selection
of different publications targeted towards using a smartphone
for driving behavior analysis is reviewed in Section III and
Section IV. The list of methodologies can be seen in Fig. 2.
Followed by the discussion on the challenges of smartphone
solution. In Section VI, possible solutions are discussed, and
finally, the paper ends with a conclusion.
II. A PPLICATIONS OF S MARTPHONE
In the following section, discussion on driver behavior
analysis using a smartphone together with other hardware are
4446 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
Fig. 3. Proposed System by Lee and Chung [31].
presented. Details are provided to suggest why such an idea
may not always be feasible. Applications of a smartphone
in other domains are also discussed including the human
mobility analysis, activity recognition, structural monitoring,
transportation analytics and their applications in the healthcare
domain.
A. Driver Behavior Analysis Using Smartphone with other
Hardware
Lee and Chung [31] proposed a multi-sensory fusion
approach for driver’s drowsiness detection. The proposed
system combined the concept of PERcentage of CLOSure
of Eyelid (PERCLOS) through the smartphone’s 3-axis
accelerometer, front camera and with electrocardiography
(ECG), photoplethysmography (PPG), in-vehicle temperature
for drowsiness detection as seen in Fig. 3.
The Hue-Saturation-Value (HSV) color model approach is
used for face detection, and the extracted eye feature is used
for calculation of PERCLOS. As explained by the authors [31],
the ECG signal is used to calculate the heart rate and represents
the subject’s overall activity level, that is a good indicator
for drowsiness. On the other hand, PPG signals are used to
calculate the blood pressure as it was found that subjects who
worked long hours of overtime and lack of sleep have higher
blood pressure. The temperature and the 3-axis accelerometer
are included to predict the driver’s vigilance index. It was
suggested that sleep symptoms are more likely to happen in
the event of a warm environment, and the accelerometer can
be used to track the vehicle’s speed. Sensors data from the
ECG sensor, PPG sensor, and temperature sensor are sent
to the smartphone through the Bluetooth connection. The
transmitted and collected data by the smartphone were used as
the inputs for the Fuzzy Bayesian network. Each fuzzy variable
define the membership degree to the output state, which in
this case, the level of fatigue. The closer the output value is
to 1, the more tired is the driver. An output value of 0.6 to
0.75 indicates that the driver is in a partial sleep condition and
is advised to take a break. Any value above 0.75 will trigger
a warning alarm.
This system is comprehensive but the PPG sensor can be
considered as slightly intrusive. Moreover, the driver cannot
possibly place his finger on the sensor throughout the driving
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
journey to measure the PPG signals. While the use electrode
fabric to collect ECG signals is not as intrusive as collect-
ing EEG signals by wearing an electrode helmet, but ECG
signals may not be as accurate as using EEG signals for
drowsiness detection [15]. Furthermore, it is also unclear if
the electrode fabric will dramatically increase the price of
the system. Finally, the proposed system is also considered
to be under development as the design of the system requires
further improvement. This is because the placement of the
sensors may hinder with the normal operation of the steering
wheel.
Araujo et al. [32] proposed an application that is capa-
ble of advising and teaching the driver to follow efficient
driving patterns. Although the authors’ focus was to reduce
fuel consumption by placing more emphasis on coaching the
driver, the methodology also aims to detect dangerous driving
behavior such as high acceleration and high level of throttle
aggression. Secondly, even though most of the data were
collected from CAN bus, the authors’ proposal was a low-cost
solution called the Torque Pro (a smartphone application)
that collect the CAN bus data through a low-cost On-Board
Diagnostics-II (OBD-II) Bluetooth Adaptor. Therefore, this
solution is considered as relevant in this paper.
Araujo et al. [32] methodology begin with the collection
of data such as vehicle’s speed, acceleration, altitude, throttle
signal, instant engine fuel consumption, and engine rotations
over a 200s window. Features such as average, minimum, max-
imum, and duration of vehicle stoppage were then extracted
from the collected data.
Driving conditions (urban, highway, or combined) can be
easily classified using the vehicle’s speed data. A normalized
consumption metric was calculated using the result and the
manufacturer’s consumption metric. It was then converted into
a fuzzy number where a low number would indicate a good
fuel consumption. The authors then adopted a center-average
defuzzification to deliver a crisp value to the driver. Finally,
the driving hint for driving behavior improvement was evalu-
ated through fuzzy logic by taking the driving conditions and
fuel consumption into consideration. The hint with the highest
value was delivered to the driver.
Firstly, the methodology could not be assessed as it did
not include any accuracy measurement involving false positive
rate or false negative. As the driving condition is determined
through a linear discriminant, such a system cannot detect
speeding as it is considered as driving on the highway.
In the study, the system can only provide driving hint after
200s of data collection. While the window can change, it is
unclear on how high data transmission can affect the data
measurements or classification accuracy. As explained earlier,
the normalized consumption metric is derived by dividing the
fuel consumption by the manufacturer’s consumption metric
on the different driving condition. However, there are several
other different factors affecting the fuel consumption rate, such
as the vehicle’s age and conditions of air filters, which in
turn affects the classification results. Finally, each automobile
manufacturer has proprietary protocols posing difficulty to
look for a generic and cheap compatible OBD-II Bluetooth
adaptor [105].
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
Hong et al. [33] utilized a naïve Bayes classifier with 5-bin
discretization for classification of aggressive driving behavior.
In addition to smartphone sensors, Hong et al. [33] also
proposed the use of low-cost ODB-II Bluetooth reader as well
as an additional IMU to be mounted behind the steering wheel.
As explained by the authors, with the inclusion of these two
hardware, more features could be extracted, and classification
can be more accurate.
Proposed features include maximum, average, and standard
deviation of speed, speed change, longitudinal and lateral
acceleration from the smartphone’s GPS and accelerometer.
The ODB-II will provide the maximum, average, and standard
deviation of speed, speed change, engine RPM, engine RPM
change, throttle position, and throttle position change. Finally,
the IMU mounted on the steering wheel will be used to detect
turn events based on its z-axis acceleration change. To prevent
features from being dependent on the context of any specific
trip, Hong et al. [33] defined six specific driving conditions
that compare the driver’s driving style more independently of
the trips. These conditions consist of 1) Start: for 5 seconds
after a start, 2) Stop: for 5 seconds before a stop, 3) H-speed:
when the vehicle is traveling at speed above 50 km/hr, 4) Turn:
when the vehicle is turning, 5) B-turn: for 5 seconds just before
the vehicle makes a turn, 6) A-turn: for 5 seconds after the
vehicle makes a turn. In order to ensure all collected sensory
signals have a consistent orientation, a virtual recalibration
method is used to repeatedly adjust the 3D orientation of the
smartphone and IMU with respect to their orientation when
the vehicle stops moving.
Features were extracted from a set of driving data collected
over three weeks by 22 participants. It consists of 1017 trip
with a total duration of 542 hours. To obtain the ground
truth of whether a driver is aggressive, Hong et al. [33]
proposed the use of historical traffic violation records for the
past 3 years and a driving behavior questionnaire. The model
was then validated using a leave one out cross-validation
and achieved an accuracy of 90.5% when ground truth is
obtained using historical traffic violation records. The accuracy
then drops to 81% when the ground truth is obtained using
driving behavior questionnaire. While the accuracy of this
implementation is high when ground truth is obtained using
historical traffic violation records, historical traffic violation
records may be biased. This is because it does not distinguish
between the causes of accidents or tickets which may result in
nonrepresentative data being collected for aggressive driving
behavior. Secondly, each automobile manufacturer has propri-
etary protocols, and it may not be easy to look for a generic
and cheap compatible OBD-II Bluetooth reader [105].
B. Human Mobility Analysis
One particular interesting domain related to driver behavior
analysis is the human mobility analysis. A slight difference
between the two is the latter is interested in discovering
subgroups of vehicles and travels that can be characterized
by some common movement behavior [34]. Such study can
be carried out due to the widespread availability of GPS
technology in navigation devices, vehicles, or smartphone,
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4447
that allows tracking of an individual’s movement with a high
degree of accuracy and temporal frequency. These rich sources
of data that represent society-wide proxies of human mobile
activities are ideal for statistical analysis and inference which
can help in understanding and characterizing human mobility
patterns during their daily activities [34]–[37]. Such analysis
improves decision making and root cause analysis in many
mobility tasks such as monitoring and planning of traffic or
public transportation systems. And also in detecting extra-
ordinary events and forecasting or simulating traffic-related
phenomena [34], [38].
As demonstrated in [34], an extraordinary event can be
detected by analyzing collective trajectory patterns. Trajectory
pattern is a pattern which reflects an individual’s travel behav-
ior [39]. Therefore, extraordinary events such as concerts or
sports competitions are detected when the destination of many
individual trips are set toward a specific small area (the event
location) and subsequently set as the origin of return trips after
the event ends [34].
C. Application of Smartphone in Other Domains
A modern smartphone is equipped with multiple sensors that
include an accelerometer, gyroscope, magnetometer, micro-
phone, cameras, thermometer, and Global Positioning Sys-
tem (GPS). Thus, a smartphone is capable of other tasks
other than driver behavior analysis. The application of mobile
sensing has been applied in human activity recognition [40],
structural monitoring [41], transportation analytics [42] as well
as in the healthcare domain [43], [44].
In [40], several classifiers were trained using sensory data
collected from accelerometer and gyroscope to perceive user’s
motion state which includes descending stairs, ascending
stairs, walking, jogging and jumping and results have shown
that such application was a success.
In addition, authors [41] showed that acceleration data col-
lected using smartphone contained consistent and significant
indicators of the first three modal frequencies of the bridge.
The results can be more precise if acceleration data collected
from smartphones were combined. Therefore, proving the
hypothesis that crowdsensing with smartphones can be used
for structural health monitoring.
On the other hand, Lu et al. [42] propose a collaborative
framework for transportation service analytics which applied
spatiotemporal analytics on transportation infrastructural data
to detect anomalous transportation events. The detected anom-
alous events can trigger mobile sensing and thereby recognize
specific commuting-related activities of interest with much
higher accuracy and lower energy cost.
Kelly et al. [43] showed that health status could be estimated
using motion sensors within a smartphone. Another interest-
ing application of a smartphone is the ability to estimate
Parkinson’s disease severity [44] as the data collected from
an accelerometer can be used to differentiate between patients
with Parkinson’s disease and healthy people by measuring
their walking patterns. Although the smartphone is inexpensive
and readily available, they have a huge potential to solve
problems from our daily lives.
4448 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
TABLE II
F EATURES AND A NALYSIS M ETHODS AVAILABLE
FOR D ROWSINESS D ETECTION
III. D ROWSINESS D ETECTION
In this section, drowsiness detection proposed by different
authors are discussed in details. The proposed methodologies
are classified into visual detection methodology and sound
detection methodology.
A. Visual Detection Methodology
As mentioned earlier, when a driver is drowsy, he usu-
ally exhibits symptoms such as rapid and constant blinking,
nodding or swinging their head, and frequent yawning [18]
. In addition, there will also be physiological changes in a
driver [15]. Hence, drowsiness detection can be carried out by
analyzing different features, as shown in Table 2.
With the availability of only a smartphone, non-visual
features (EEG, ECG and EOG) are usually not applicable.
A common idea to detect a drowsy driver visually is using
the smartphone’s front camera to capture the image of the
driver’s face. Based on the captured image, further analysis
can be used to determine if the driver is drowsy. For such
methodology, a reliable and valid metric known as PERCLOS
is commonly adopted to determine the alertness of driver [15].
PERCLOS is the percentage of eye closure [45]. Figure 4
illustrates different states of an open eye. Theoretically, if a
driver is tired, duration of eye closure will be higher leading
to a higher PERCLOS. Therefore a threshold can be set
to indicate whether the driver is at a dangerous state of
drowsiness.
Such an idea was adopted in the system proposed by
Xu et al. [47]. Using the smartphone front camera and an
Android API, Xu et al. [47] captured the position of the
eye and zoomed the eye image to a size of 40 × 20 pixel
for consistency. It also eliminates redundant information at
the same time. A single hidden layer Artificial Neural Net-
work (ANN) with 16 hidden neurons was used to classify the
states of the eyes. A quadratic sum of weights penalty factor
was introduced into the error function to minimize underfitting
and overfitting.
Based on the classification results of eyes state, PERCLOS
was calculated as the percentage of the closed eye in the
number of frames captured and blink time was derived using
the time of the 1st and the last frame of consecutive closed
eyes. Blink rate was then determined by the total number of
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
Fig. 4. The different States of An Open Eye [46].
blinks over the total time of the recent 500 frames. As defined
by the authors [47], the driver is deemed to be severely
drowsy if PERCLOS exceeds 25% in the recent 500 frames
or a blink time duration of more than 2500ms. On the other
hand, the driver is deemed to have mild drowsiness when
blink time was more than 1000ms and blink rate was more
than 0.5, or the driver has been driving continuously for
4 hours. Using such threshold, authors [47] have reported
a detection rate (drowsy driver detection) of 95% accuracy
with a zero false positive and a 10% false negative rate.
While the detection rate, as reported by the authors [47]
is impressive, there are several limitations to the systems.
Firstly, to achieve a high detection rate, the system must
have a personalized set of training data (eye state image) for
different drivers. Secondly, training of backpropagation ANN
was not performed in the smartphone environment. Model
parameters have to be manually loaded into the smartphone
after training in an external environment using MATLAB. The
possibility of training the ANN in a smartphone environment
is also unclear. The required level of computation power can
be difficult to deploy in all smartphone [58]. While glasses
or skin complexion has little influence on this methodology,
light is a factor that can affect the performance; therefore,
applicability is limited at night. Such a system also requires
the fixed position of the phone in order to capture the facial
image. If the face of the driver cannot be detected, the system
is not useful.
Similarly, Dasgupta et al. [48] also adopted PERCLOS
for drowsiness detection. However, as mentioned by
Dasgupta et al. [48], a single image-based cue may not be
reliable enough. Therefore, to increase the reliability of the
system, the authors [48] suggested a three-stage detection
process. The three-stage detection framework begins with the
computing of PERCLOS, then the classification of voice to the
unvoiced ratio (VUR) based on the driver’s voice as speech
signals can be used as an indicator of drowsiness. Finally,
a reaction time test where a driver must touch the screen
of the smartphone within a time limit. Image captured was
first processed to increase the local contrast, enhancement of
details, and to provide an orientation correction if the driver’s
head had a tilt of more than 30 degrees. Detection of the
face was then executed using Haar-like features. Based on
the detected face, a Haar classifier trained with eye images
during day time and night time was used to detect the eyes.
A linear Support Vector Machine (SVM) was then trained to
classify between open and closed eyes. To compute the VUR,
Dasgupta et al. [48] use the microphone in a smartphone to
capture speech signals sampled at 20 kHz, and Singular Value
Decomposition (SVD) was done to remove external noise.
Again, SVM was adopted to return the voiced and unvoiced
length with Mel frequency cepstral coefficients (MFCC) as
input. VUR was then calculated by using the two outputs.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
The last test requested the driver to touch the screen of
the smartphone within 10s to test the alertness. In the event,
if PERCLOS over a 10s sliding window is more than 20%,
then the second stage that classifies the driver’s UVR is
activated. If the driver fails the voice test and the reaction test,
an alarm will be generated through the speaker. As a result,
an SMS will be sent to the emergency number registered.
Based on such system, Dasgupta et al. [48] achieved an
average detection rate of 93% as compared to a system using
only PERCLOS which obtained an average detection rate
of 88% based on four different test set. However, a system
using only PERCLOS achieved a higher detection rate in one
of the test set. Although Dasgupta et al. [48] claimed that
SVD could be performed to remove noise from the audio,
the paper lacks details on whether such system can extract the
driver’s voice from polyphonic audio that may contain other
speech present in the audio collected. To separate overlapping
of multiple sound events is a challenging task [49], [50], and
it is highly doubtful that using only an SVD is sufficient.
Therefore, if the radio is switched on, it may cause a false
negative indicating that the driver is alert. Subsequently, asking
the driver to touch the screen within 10s while driving is a form
of distraction. Moreover, training of the SVM model appears
to perform outside of the smartphone environment. Finally, the
position of the phone is fixed to capture the facial image.
Similarly, Chang et al. [51] also adopted the idea of PERC-
LOS in their drowsiness detection system, but at the same time,
they also track the distance of the vehicle in front that can give
an alarm if the distance gets too close. Chang et al. [51] used
the Haar-like features for face and eye detection. Based on the
detected region of interest, a skin color model was then applied
to the eye image to classify the eye state. As explained by the
authors [51], white pixel indicates the skin area, and black
pixel indicates the eyeball area. If the ratio of a black pixel
over white pixel drops below 0.8, then the eye is considered
as closed, and the driver is deemed as drowsy if the eye is
closed for more than 400ms. The vehicle in front of the driver
was also detected using Haar-like features, and the distance
between the vehicle was calculated according to the equation
given as
Wc Wp
D= ∗ (1)
Wr 2 ∗ tan(F OV /2)
where Wc is the real width value of the detected vehicle,
Wr is the width of the detected square, W p is the width
of image and FOV is the field of view of the rear camera.
If D is less than 7m, then an alarm is given. In this sys-
tem, Adaboost algorithm was used to train a face classifier
(using 7240 images), eye classifier (3500 images) and vehicle
classifier (using 5900 images). Audio was used but was only
considered for engine state classification and voice command.
Therefore, audio, in this case, is considered as irrelevant to
drowsiness detection and excluded from further discussion.
The accuracy of such a system was not reported in the
paper. The drawback of this system is a large amount of
different images is required to train each classifier. It is also
unknown how Wc can be obtained or estimated as it is
difficult to predict the actual width of the vehicle in front
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4449
of the driver. Moreover, the performance in regards to relative
position accuracy, detection of vehicles, and the number of
tracked objects can be affected by the limited processing
capabilities of smartphones [52], [53]. Finally, the smartphone
has to be fixed at a position to capture the facial image and
vehicle in front of the driver simultaneously.
On the other hand, Qiao et al. [54] captured facial features
as eye blinking, head shaking, and yawning for drowsiness
detection. The image was first processed using Gaussian Filter
for noise removal, followed by Mean Filter for image enhance-
ment. The detection of the face of the driver was carried out
using a Haar classifier and location of the eyes and mouth were
approximated using position approximation based on the size
of the driver’s face. As explained by Qiao et al. [54], an open
eye should have a black pixel since it has a larger visible
pupil. Therefore, after locating the eyes, the eye image was
converted to a grey image, and the ratio of the black pixel
was calculated. On the other hand, the movement of the head
was calculated based on the variances of the face’s centroid.
Finally, the contour of the mouth was computed using Canny
contour-finding algorithm. This contour was then measured,
which indicates a yawn. Finally, a threshold was set for all
collected information to determine if the eye was closed; the
head was shaking, or the driver was yawning. Detection of
a driver performing any of the actions will indicate that the
driver is tired. The system was tested on 500 images of an
eye blink, head shake and yawning and reported an accuracy
of 90.5%. Although the proposed system aims to lower the
chance of false negative, it has a higher false positive rate
as the proposed system will raise the alarm upon detecting
any of the indicators (eye closed, head shaking or yawning).
A driver that performed any of the action does not necessarily
indicate that he is tired, only through continuous performing of
the action, one can determine if the driver is tired. Moreover,
the position of the smartphone has to be fixed to capture facial
features.
For accurate eye detection without training data,
Li et al. [55] proposed the Progressive Locating Method
(PLM). The system aimed to eliminate the need for a
statistical method and to focus on the apparent and geometric
relationship within the facial region. It was to reduce the
uncertainty of eye detection due to the unrelated facial
region and avoid the high computational cost. As described
by Li et al. [55], the system consists of three major parts.
The first part was the detection of the driver’s face on the
image captured by the smartphone using a skin color model.
The second part of the system was the use of PLM algorithm
that consists of three smaller subparts. The first subpart
was to find the facial feature as a reference to detect the
coarse region where the right eye was, and in this case, Li et
al. [55] proposed using the driver’s mouth. Once the mouth
was detected, the bottom and right points were chosen as a
reference point to get the coarse eye region.
In the second subpart, the grey level projection was used to
locate the right eye region. Finally, an accurate eye coordinate
was measured by subsequent analysis. In the final part of the
system, the extracted eye image was analyzed. An open eye
size (height and dimension) was first determined and used as
4450 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
a threshold. If the eye size collected in a subsequent image
frame is smaller than the threshold value, then the eye is
considered close. If the system detected three closed eyes out
of five frames, then a drowsiness alert is given to warn the
driver. Although such a system is not affected by skin color
variation, it cannot work well in a dark environment. Presence
of glasses and long fringe can affect the detection accuracy
of the system. These are due to the reflection of light on
the surfaces of glass, and long fringe affects the distribution
of greyscale around the right eye region. Thus, such system
over-relies on the grey distribution of the facial region [56].
In addition, the position of the smartphone has to be fixed to
detect the face.
Zhang et al. [57] also proposed to use a smartphone camera
to capture facial features. However, instead of capturing just
the eye blinks and yawns, they include the extraction of
Blood Volume Pulse (BVP) for drowsiness detection. The idea
is based on imaging photoplethysmography (PPG) technique
to monitor changes of image sequence without concerning
the specific content. Thus, physiological processes recorded
in a video can be extracted and analyzed. In their work,
Zhang et al. [57] utilized the Second Order Blind Identification
(SOBI) extended-PPG for such purposes. A front camera of a
smartphone was set between a distance of 15cm to 80cm away
from the driver depending on the camera’s specification. The
captured image stream was then analyzed by a nine-channel
SOBI to estimate the underlying signals from different phys-
iological activities.
As signals were extracted in a random order, an identi-
fication algorithm was required for correct identification of
the signal that corresponds to the physiological response. The
analysis was then carried out to estimate the Heart Rate
Variability (HRV), blink duration, blink frequency, and yawn
frequency. Drowsiness was determined if any of the indicators
exceeds the threshold set by the authors [57]. Following
this implementation, the system reported a detection rate
of 91.5%. Although this system proposed a novel idea of
multi-information fusion utilizing only a single camera, there
are several limitations.
The proposed method cannot work well on poorly illumi-
nated or with illumination interference environment, in the
event when the driver’s head is tilted or shaking and also
on a bumpy road. Zhang et al. [57] also mentioned that
system could not be readily deployed without checking the
driver’s physiological characteristics as they differ from person
to person and the system will require further adjustment for
optimal performance. Finally, the system requires the fixed
position of a smartphone, and it is also not applicable for
drivers with small eyes, and for driver wearing a pair of
sunglasses.
B. Sound Detection Methodology
Xie et al. [58] proposed an interesting system, D3 -Guard
to detect drowsy driver using the smartphone microphone.
As the movement of the human body produce Doppler shift,
Xie et al. [58] proposed to detect nodding, yawning, and
operating of the steering wheel in a drowsy state. Doppler
shift refers to the change in frequency or wavelength of
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
audio signals about an observer who moves relative to the
signal source. The speakers first send out acoustic signals
at 20 kHz, and the microphone received the reflected sig-
nals. Band-pass filter and undersampling were then applied
to improve frequency resolution without distorting frequency
spectra before transforming the signal into the time-frequency
domain through FFT. Effective features from Doppler profiles
of audio signals for each drowsy driving action was extracted
and used as inputs of Long Short Term Memory (LSTM)
network. Two LSTM models were trained for this propose.
The first LSTM classify between normal state, nodding, and
yawning. The second LSTM classify between normal and
abnormal steering operation. Outputs of both LSTM are passed
to a NN to predict if the driver is drowsy. Based on such
a system, Xie et al. [58] reported an accuracy of 94% in
detecting drowsy driver.
However, the main limiting factor is a large amount of data
is required to train the classifier. Moreover, road conditions
also affect the accuracy of such a system. It was found that
when the driver needs to operate the steering wheel more often
on a crooked road, it will result in low detection accuracy.
Furthermore, it is also unclear if the presence of passenger will
affect the accuracy as passenger movement will also affect the
Doppler shift. In addition, it is unknown if LSTM or NN can
be trained in a smartphone environment.
C. Summary
This section presents the discussion on different method-
ologies proposed by different authors. The summary of the
proposed methodologies and their limitations can be found
in Table 3 and Table 4, respectively. Table 3 and 4 con-
tain information such as smartphone sensor used, pre/post-
processing procedure, the algorithm used, features extraction
and calculation, drowsiness detection accuracy, and the limi-
tations. In the pre/post-processing column, the rough idea of
how processing is provided and how the algorithm is used is
also indicated. For example, Median Filter was used for noise
removal. Features extraction are listed in the authors’ paper.
There are features that used in system and not just features
for the final classifier. As seen in Table 3, the main idea for
drowsiness detection is through the direct analysis of driver
characteristics such as his eyelid, head, or body movement in
a continuous fashion. Thus, the phone must be mounted in a
secured position to capture the driver’s face regardless of the
illumination condition.
IV. A BNORMAL D RIVING PATTERN D ETECTION
As mentioned earlier, a distracted, drunk, or reckless driver
may unintentionally slow down or have sudden longitudinal
and transversal movements [16], [20]. Such actions can be
detected by the different sensors of a smartphone. In the
following subsections, different types of methodologies are
discussed in details.
A. Thresholding Methodology
As evaluated by Fazeen et al. [59], extreme driving behavior
will quickly produce a higher value in the data collected by
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES 4451

TABLE III
D ETAILS ON P ROPOSED M ETHODOLOGIES FOR D ROWSINESS D ETECTION

the accelerometer. Therefore by setting a threshold; one can
quickly detect the abnormal driving behavior. Moreover, such
an idea can also be extended to detect road anomalies such as
speed bumps, pothole. The smartphone is first aligned with the
vehicle coordinate system at the y-axis of the accelerometer
with the moving direction of the vehicle to allow for such
detection. After the alignment, data from the x-axis accelerom-
eter can be used to detect turning or lane change, y-axis can be
used to detection acceleration or braking, and finally, the z-axis
can be used to detect vibrations or road anomalies. However,
Fazeen et al. [59] did not report the accuracy or detection rate
of abnormal driving but indicated an accuracy of 85.6% for
the overall road anomaly classification system. Such a simple
idea can be readily deployed into any smartphone and do not
require a high computational cost.
However, the threshold value defined by such a method is
only applicable to a specific type of vehicle, road, and phone
combination [60]. It is due to the different phone and car
possess different characteristics where the threshold value used
in car A with smartphone A cannot be used on car B with a
different smartphone.
Moreover, poor road conditions also affect the readings of
the sensors. Thus, it cannot accurately distinguish the differ-
ences in various driving behavioral patterns [61]. In addition,
it only works well in low speed and short distances cases due
to accelerometer’s accumulated error [62], [63]. Finally, this
system also requires the smartphone to be in a fixed position
and can produce inaccurate readings if the phone’s orientation
tilts because of a road bump [24], [64].
B. Pattern Matching Methodology
Another methodology rather similar to the setting of the
threshold is the use of Dynamic Time Warping (DTW) to
compare the similarity of the collected sensors’ signal with
the signals of aggressive maneuvers. Such methodology was
applied by [65]–[68].
Based on such an idea, Johnson and Trivedi [65] proposed
a system called MIROAD utilizing DTW on the sensor-fusion
output of accelerometer, gyroscope, and magnetometer to
classify the maneuvers performed by the driver. As explained
by the authors [65], the gyroscope gives a clearer indication
of turns, and by using accelerometer and magnetometer in
conjunction with the gyroscope, a more accurate reading of
device orientation can be obtained. Johnson and Trivedi [65]
then proposed to use an array consisting of readings from
the x-axis of the gyroscope, the y-axis of accelerometer
and x-axis of a magnetometer as they were found to be
the most accurate indicator of different driving maneuvers.

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4452 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020

TABLE IV
L IMITATIONS OF P ROPOSED M ETHODOLOGIES FOR D ROWSINESS D ETECTION

In order to detect the start and end time of an event, Simple passes a predefined threshold, then the event has started, and
Moving Average (SMA) of the rotational energy about the subsequent samples are concatenated until the values drop
x-axis was used via the x-axis of the gyroscope. If SMA below the threshold to give the end time. Based on these

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
collected samples, templates of different maneuvers (collected
in the same way) were compared using DTW. The smallest
distance value will indicate that the collected samples are
almost identical to the template of that category. Based on
the distance calculated, Johnson and Trivedi [65] proposed to
use K-Nearest Neighbor (K-NN) with K as 3 to determine the
type of event. Based on such a system, the overall detection
accuracy of events was at 91.9%%.
On the other hand, Eren et al. [66] also utilized the same
sensors for event detection. They first collected the safe
driving templates for each driving event (turning, acceleration,
slowing down and lane change) from five safe drivers by
manually selecting and labeling each safe driving habits based
on the collected data. For identification of event within the
selected portion of the signal, windowed samples collected
during driving were z-score normalized and matched with
each template to compute the similarity between them using
Dynamic Time Warping (DTW) algorithm. The classification
of driver behavior was performed through the comparison of
probabilities computed using Bayesian inference.
P(rn )P(s|rn )
P(rn |s) =
P(s)
where rn is the class of driving event. P(r1 ) denotes prior
unsafe probability and P(r2 ) denotes prior safe probability.
P(r1 ) was chosen as 0.2 and P(r2 ) was chosen as 0.8 based
on the estimation of ten different drivers on whether they were
safe or not. If P(r1 |s) is higher than P(r2 |s) than the driver
is classified as an unsafe driver. Based on such a system,
the authors [66] reported a detection rate of 93%.
Singh et al. [67] proposal was to detect brakes, and lateral
maneuvers applied to the car. As explained by the authors [67],
the brakes can produce a subtle measurement in the form of
acceleration signal that can be collected by the accelerometer
whereas gravity sensor and gyroscope data can be fused to
detect lateral maneuvers such as sudden left, sudden right, and
lane changes. In addition, through the use of crowdsensing,
congestion on the road can be detected when multiple smart-
phones detect the braking event in the same geographical area.
Such information can be used to distinguish braking due to
congestion and sudden braking due to harsh driving. In order
to allow matching of templates, different maneuvers profile
have to be collected. Before a drive, the phone axes were
aligned with the vehicle axes where the y-axis of the phone
must be in the same direction with the moving direction of the
car. Collected data from the accelerometer were first reoriented
to its original axes using Euler’s angle, and only the data from
the y-axis was kept for detection of brakes. SMA filter was
then applied to smooth the data to remove vibrations caused by
the car. The smoothened data were passed through a bandpass
filter to remove noise due to hardware sensitivity. Yaw angle
calculated from data collected by gravity sensor were fused
with data collected along the z-axis of the gyroscope to
compute the angular velocity. The angular velocity was also
passed to the SMA filter to smoothen the data. The data were
then matched individually with the templates signal and the
pair with the lowest similarity score determine the event of
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4453
the collected data. Based on this system, Singh et al. [67]
achieved an average detection accuracy of 94.6%.
Finally, Ali et al. [68] did an evaluation between DTW
and K-Nearest Neighbor (K-NN) utilizing the accelerometer,
gyroscope, and GPS in a smartphone. As explained by the
authors [68], the y and z axes of the accelerometer can easily
pinpoint the road bump present on the road while the x-axis
of the accelerometer can be used to detect aggressive driving
behavior. On the other hand, the z-axis of the gyroscope can be
used to detect aggressive maneuver. Finally, GPS can provide
the speed of the vehicle. To facilitate the collection of data,
the axes of the smartphone must be aligned with the axes of the
vehicle. The data is filtered through a low pass filter to remove
the presence of noise. A total of 540 driving events were
recorded over two months. These events belong to 9 different
categories namely, 1) normal right turn, 2) normal left turn,
3) normal right lane change, 4) normal left lane change,
5) aggressive right turn, 6) aggressive left turn, 7) aggressive
right lane change, 8) aggressive left lane change and 9) road
anomalies.
In the study, Ali et al. [68] compared the two algorithms for
two different task, 1) classification of normal and aggressive
(2)
driving as well as road anomalies and 2) classification of
normal and aggressive turn and lane turn. The difference
between the two tasks is that task 1 is more generalized
classification where the algorithm can only detect normal or
aggressive driving regardless of turns or lane change but taking
into consideration of road anomalies. On the other hand, task
2 classified the event into their respective categories without
considering road anomalies. Based on the results, K-NN was
able to perform better in task 1 as compared to DTW (98.67%
vs. 90.22). However, K-NN achieved a much lower average
accuracy in task 2 as compared to DTW (78.06% vs. 96.75%).
The main limitation of using DTW algorithm is the need
to collect templates of different maneuvers for similarity
calculation. Much effort is required to select and label each
driving habits manually. At the same time, it is also impossible
to collect all types of templates due to the unique driving
styles and behaviors of different drivers [69]. Moreover, DTW
is a computationally expensive operation since the algorithm
computes the similarity between two elements for all elements
in two signals. Furthermore, dangerous templates can only
be collected in a safe environment when no safety concern
complicates the data collection process. As such, template
matching appears to be unfeasible for a large and heteroge-
neous sensing environment when considering different types
of vehicles and devices [70]. A common drawback among the
four methodologies proposed is that they require the phone to
be in a fixed position.
Furthermore, in [65], they only considered turn events, and
other events such as speeding, hard brake were not considered.
Despite the fact that the overall accuracy is high, such a
system was not able to achieve high classification accuracy for
U-turn, which achieved only 77% accuracy. In [66], only ten
drivers were considered for computing the prior probability.
It is insufficient to represent all drivers. Moreover, the accu-
racy of the classifier is based on completed surveys by test
participants, where the results could have been influenced by
4454 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
subjectivity [29]. Whereas for [67], the detection accuracy
of events at high speed is not clear. As the authors [67]
mentioned that templates would be discarded if the recognition
rate for test samples is low, and another template will be
selected. Therefore, it is assumed that only one template is
used for each event. As such, if a non-representative template
is used, it will only be updated after some time and is
unclear how updating will take place. Although, the use of
crowdsensing allows collective sharing of data, the required
cost and time to implement crowdsensing-based applications
can be a limiting factor [71]. In [68], the study cannot be
considered as conclusive because there were several details
which were left out. Firstly, the authors [68] did not specify
the number of K (number of nearest neighbors) used, and there
was no study on the effects of K on the classification accuracy.
Secondly, the study did not include hard braking and speeding.
Chaovalit et al. [72] also proposed a template matching
system similar to [65]–[68]. As opposed to using DTW,
Chaovalit et al. [72] proposed to use Symbolic Aggregation
Approximation (SAX). In their system, the accelerometer
was used for considering movement along the lateral and
longitudinal axes that correspond to turning, lane changing,
braking, and acceleration. The raw data from the magne-
tometer was used as an indicator for the detection of driving
events in the lateral domain. Finally, the GPS data was
used to provide the location and speed data of the vehicle.
In the paper, it was unclear how representation samples of
different aggressive driving behavior were collected. It was
only mentioned that samples representing different aggressive
driving patterns (sudden acceleration, sudden brake, and other
aggressive driving behaviors) were available, and such data
were manually validated [72]. The idea of SAX was to
convert the data into a string. Before the string conversion,
piecewise aggregation was performed where samples during
a time window were averaged. The primary motivation of
SAX was to reduce computational cost [72]. During a drive,
streaming data was collected and converted to a string using
SAX and compared with the converted representative data. The
similarity is measured in terms of distance and is calculated
using min-dist [73].
If the similarity passed a pre-defined threshold, correspond-
ing driving behavior could be identified from the streaming
data. This system is easy to use and implement with a
lower computational cost as compared to the system proposed
by Eren et al. [66]. However, by using SAX, there is a
possibility that temporal information is averaged out in the
time window that may affect accuracy. Such a system is also
not applicable for real-life deployment as the detection rate of
aggressive events is only at 25%. In addition, it also requires
the smartphone to be in a fixed position.
C. Fuzzy Logic Methodology
Castignani et al. [24] proposed a fuzzy logic mechanism,
SenseFleet, to detect risky driving events that includes hard
acceleration, hard braking, aggressive steering, and over speed-
ing. Such mechanism consisted of a calibration phase to make
it adaptive and was proposed with the intention to overcome
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
the limitation of hard thresholding to detect different abnormal
driving behavior. The system utilized an accelerometer, gravity
sensor, a magnetic sensor as well as GPS to compute the
degree of a jerk, orientation rate, speed variation and bearing
variation for abnormal driving detection. The internal linear
accelerometer was used to compute the rate of change of
acceleration with respect to time that constitutes to a jerk.
In order to compute the vehicle’s direction variation, magnetic
and gravity sensors were used to compute the orientation
vector. Such vector consists of yaw, roll, and pitch that
characterize rotation around the different axes. All sensor
data were fused with GPS data to reduce noise due to
electromagnetic interference and device vibration. In addition,
the system also considered speed limits for each particular
road to detect speeding events. Events were also combined
with weather information and time of day to better determine
the riskiness of events and score the driver. The outputs were
then fed to a fuzzy system with a set of fuzzy rules. Each rule
evaluates a combination of different possible fuzzy values of
input variables and outputs a type of event (hard acceleration,
hard braking, aggressive steering or over speeding). However,
as different smartphone and vehicle have different components
and characteristic, a newly set up system requires calibration.
After calibrating, the system dynamically adjusts the fuzzy sets
for the variables. As explained by the authors [24], calibration
was done only to adjust the jerk and yaw rate since speed
variation, and bearing variation can be fixed regardless of
device or vehicle. Based on such a system, the paper [24]
reported a detection rate of at least 90% if there are at least
1500 calibration samples.
The main drawback of this system is the calibration
samples required. It can lead to further complications if
non-representative calibration samples are collected [29].
Usage of non-representative calibration samples can affect the
system reliability as each set of rules can only be applied to a
single driver. Although several solutions were provided, they
possessed several limitations. Castignani et al. [24] suggested
that the system can dynamically stop the calibration phase and
decide if the information is representative or not. However, it is
unknown how can it affect the system detection accuracy if the
system stops calibration with only a small set of data collected.
The second solution provided is to perform a periodic update
of the fuzzy rules. However, it would imply the system can be
inaccurate for the first few drives if the first calibration samples
collected are non-representative of the driver’s behavior. The
third solution is to combine information from different drivers
and consolidate the information to give a universal set of rules.
However, this solution will require a big amount of data to
provide a universal set of rules. Fuzzy logic also requires
the setting of various thresholds which is sensitive to the
noise [74]. Finally, such a system also requires the position of
the smartphone to be fixed.
Eftekhari and Ghatee [75] proposed a hybrid of Discrete
Wavelet Transformation (DWT) and Adaptive Fuzzy Inference
System (ANFIS) to recognize overall driving behaviors. Four
features relating to angular velocity, lateral acceleration, lon-
gitudinal acceleration, and angle variation were used in this
system. Before calculating these four features, a two-level
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
decomposition was applied on the time series data using
DWT. The features are then used as the input of ANFIS.
Six ANFIS were trained to measure the similarity and
dissimilarity between the collected data concerning safe,
semi-aggressive, and aggressive driving patterns. Three
decision-making mechanism was proposed. Among the three
mechanism, hierarchical ordering gave the highest classifi-
cation accuracy compared to majority voting and ratio of
similarity over dissimilarity. The system was evaluated on
twenty drivers who were asked to compute a questionnaire
to determine if they are a safe, semi-aggressive or aggressive
driver. Around eight mins of driving data were collected for
each of them with 2Hz sampling rate. Thus, resulting in a
total of 960 samples, where 70% were used for training and
30% for testing. Based on such implementation, Eftekhari and
Ghatee [75] reported an accuracy of 92%. However, it may not
be accurate to classify the driver based on a questionnaire.
Moreover, the system was only validated on a small number
of samples. In addition, such a system cannot be applied on
a smartphone where high computational power is needed to
run a large number of algorithms (DWT, 6 ANFIS, hierarchical
ordering). The position of the smartphone has to be fixed such
that the coordinate system of the smartphone and the vehicle
are aligned.
Subsequently, Eftekhari and Ghatee [76] proposed a new
system making use of fuzzy logic in combination with a neural
network for aggressive driving detection utilizing the smart-
phone’s gyroscope, accelerometer and magnetometer. This
system evaluated the driver’s driving behavior by determining
the similarity between each driver’s behavior and safe and
aggressive driving behavior. The system consists of four stages
that analyzed segments of sensor data where each segment
consisted of 3s data with six samples and 50% overlap (half
of the data used belong to the previous segment). In the
first stage, the energy of a segment, FGyr , was calculated via
the sum of squared values collected by the gyroscope in the
z-axis during the 6s. A threshold was set based on this energy
value and any exceedance will indicate a maneuvering action
including the lane change, turns and U-turns. In the second
stage, the maneuver type (lane change or turns or U-turns)
was determined using the peak of FGyr , angular variation
and duration of each maneuver as inputs to a single hidden
layer backpropagation NN with 10 neurons. Eftekhari and
Ghatee [76] proposed a length of 500 training inputs and
reported a detection accuracy of 89%. In the third stage, the
detected maneuver was assigned with a fuzzy number. This
number was calculated based on the peak of the variance
of the squared values measured by the accelerometer in the
lateral axis. In the final stage, the fuzzy number calculated in
the previous stage was used to evaluate the driving behavior.
The membership function was first determined by considering
750 maneuvers collecting from three aggressive and three safe
drivers.
The similarity between the calculated fuzzy number and
the fuzzy number of safe and unsafe driving was then tabu-
lated using the max-min composition. Finally, scores for the
maneuver was computed to give a safe and aggressive driving
score. This two scores with the driver anger level were then
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4455
used to classify the driving behavior of the driver. If the safe
driving score is lower than the aggressive driving score and
if the driver has an anger level of above 51, then the driver
is considered as aggressive. The system was tested on twenty
drivers with different smartphone and vehicles. As Eftekhari
and Ghatee [76] reported the results in the form of scores
for safe and unsafe driving with the classification of behavior,
there is no detection rate available.
The limitation of Eftekhari and Ghatee [76] system can
be said to be similar to [24] as the samples collected from
a small number of the driver cannot be used to represent
all the drivers. Hence, they have a high rate of false alarm
or missed detection. In addition, maneuvers consider in this
system were only lane change, turns and U-turns. Furthermore,
there are instances of drivers where their behavior cannot be
classified using such a system as their safe driving score was
the same as their aggressive driving score. The anger score
tabulated by the questionnaire is not representative of the anger
level throughout the journey. Assuming the questionnaire was
done before the drive, the driver may be calm, but due to an
unforeseen event, the anger level may rise during the drive
leading to system inaccuracy. Moreover, the position of the
smartphone has to be fixed such that the coordinate system of
the smartphone and the vehicle are aligned.
D. Non-Ensembled Learning Methodology
To address the limitation of using non-representative cali-
bration samples, Castignani et al. [70] extended his work by
proposing a Multivariate Normal Model (MVN) to detect risky
driving maneuvers and profile drivers as well as including
a periodic update of the system with the collection of new
samples. The largest advantage of such a system is it does not
require any labeled data for detection as the probability of an
observation that allows the assessment of driver behavior in a
continuous fashion.
Data collected by the accelerometer was again used to
calculate the jerk, and the data measured by the magnetometer
was used to calculate the average yaw rate and the angular
velocity. GPS used in this system was for measuring the speed,
speed variation as well as the bearing variation. Based on
empirical result, Castignani et al. [70] suggested that a feature
set comprising of only 1) product of speed variation and
standard deviation of jerk and 2) product of bearing variation
and average yaw rate is sufficient for the task but feature set
comprising of only 1) product of speed, speed variation and
standard deviation of jerk and 2) product of speed, bearing
variation and average yaw rate provides fewer false positives
and do not produce any high severity during calm driving.
The feature set used will then generate the first MVN model
using Maximum Likelihood Estimator (MLE). An MVN can
be defined by the multivariate Gaussian probability density
function given as
 1 1 −1
p(x|μ, )=  d exp(− 2 (x − μ)
T
(x − μ))
2π   2
d
2
(3)
4456 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020

where p(x|μ, ) is the multivariate Gaussian probability
density function, x is the features considered,
 d represents the
dimension of feature space and θ = (μ, ) represents the
parameters’ distribution. In this system, θ was dynamically
estimated. Using the probability density function with the
estimated θ , the likelihood of an observation can be calculated
and anomalies can be identified. After the training phase,
the system can periodically update the model’s parameter by
considering the new samples and also the old samples where
Castignani et al. [70] suggested to use 2900 old samples
and 1200 new samples based on their evaluation. To prevent
adding samples of low quality (non-representative samples),
Castignani et al. [70] added a criterion to calculate the ratio
of the determinant of new samples against the determinant of
the old samples. If it does not exceed the pre-defined threshold,
then the system does not perform the update. Similar to the
SenseFleet [24], once the system detects an anomaly, it will
then deduct the score of the driver with the consideration of
the weather, daylight and environmental information. It was
reported that such a system was able to detect 600,000 risky
maneuvers from 4800 drivers using iPhone 6 and Samsung
Galaxy S5 in a large scale implementation. The paper included
an evaluation of the average score, but it did not include a
comparative study between ground truth and detected anomaly.
Therefore, details such as detection rate, false position or false
negative were excluded.
As mentioned earlier, Castignani et al. [70] suggested to
use 2900 old samples and 1200 new samples for a system
update and this criterion was based on the highest recall value
of detection rate which was approximately 0.7. Since recall
is defined as a ratio of true positive over the addition of true
positive and false negative, there may be a need to increase
the recall of the detection rate even though such system is
robust against different car and phone combination. Similar to
their early work [24], the position of the smartphone has to
be fixed.
Yu et al. [21] utilized data from the accelerometer and
orientation sensor on a smartphone together with Support
Vector Machine (SVM) and NN to detect abnormal driving
behavior such as weaving, swerving, side slipping, fast U-turn,
turning with a wide radius and sudden braking. The proposed
system consists of two different phases, namely: an offline
and online phase. The offline phase consists of data collection,
data processing, feature extraction, and finally, model training.
During the data collection stage, the coordinate system of
the vehicle and smartphone placed were aligned by align-
ing the accelerometer’s y-axis along the moving direction
of the vehicle. To remove the noise present in the data
collected, a low pass filter known as the Dynamic Exponential
Smoothing Filter (DESF) was used to preserve effective fea-
tures of different driving behavior. Based on the processed
data, sixteen basic features were extracted to differentiate
the different driving behavior. To further improve the distinct
difference between the different types of driving behavior, the
authors [21] suggested using another 136 features. Sixteen of
them were squared of the basic features, and the rest were
the product of any two different features. However, SVM
only utilized 16 basic features for training while NN used all
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
152 features. Based on such implementation, SVM obtained
an accuracy of 95.4%, and NN obtained an accuracy of 96.9%.
As different features were used by different classes, a direct
comparison between them can be quite biased.
Although it is unclear how well SVM will perform with
152 features, it can be concluded that it may not be worthwhile
to increase the computational load with a small increment
of accuracy. While sudden braking is indeed an abnormal
driving behavior, a driver may use the sudden brake to avoid
potential collision ahead of him where the fault is not on
the driver. Therefore, there is a need to consider the differ-
ence behind the reason for sudden braking to prevent any
bias/demerit being given to the driver when the fault is not on
the driver. As explained by Yu et al. [21] side slipping occurred
when a driver is driving in a straight line but deviates from
a normal driving direction as seen in Fig. 2. However, such
behavior is almost similar to a lane change, and Yu et al. [21]
did not include a detailed explanation of how the system may
classify between the two scenarios. It was only mentioned that
data were collected in a real driving environment with a normal
driving behavior with few and small fluctuations in sensors’
readings due to abnormal driving. Therefore, it is unclear if
lane change was conducted during the data collection phase
that resulted in a false alarm caused by the system. Finally,
the model was trained outside of the smartphone environment
and the phone was required to be mounted in a fixed position.
Ahmed et al. [77] proposed a Random Forest (RF) to clas-
sify between undistracted and distracted drivers. Distractions
can be due picking up a phone to call, text or read and
Ahmed et al. [77] found that when drivers picked up their
phone to perform any of the actions, it will cause subtle
changes in the accelerometer and gyroscope measurement.
A total of 28 features were calculated, and 6 were extracted
using a filter based approach based on linear correlation and
information gain. These features were then fed into an RF
as inputs for driving behavior classification. Based on such a
system, the authors [77] reported an accuracy of 96%. While
this method is accurate in detecting if a driver has picked up
his phone, it has a poor accuracy classifying whether the driver
is reading, calling or texting on the phone. Ahmed et al. [77]
suggested that it was due to humans having different habits
when they read, call or text using a phone. This system
only considered distraction caused by driver picking up the
phone to perform certain actions but it is unclear if such a
system can detect driver distraction if he did not pick up his
phone. The method was also tested on a younger group of
participants which may have different results for other age
groups, particularly for an older driver. Furthermore, such a
method is also not useful in detecting distraction caused by
other scenarios such as eating or falling asleep.
Bejani and Ghatee [78] proposed the use of Convolutional
Neural Network (CNN) for feature extraction and classification
of the driving style. As CNN suffers the drawback of over-
fitting, the original algorithm was modified with an adaptive
regularized function where the authors [78] called CNNAR.
The idea of adaptive regularized is to achieve an adaptive
dropout and weight decay. To enable adaptive dropout, Bejani
and Ghatee [78] keep track of the ratio of mean square error on
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
the validation set over mean square error on the training set.
Based on this ratio and two predefined thresholds, keeping
probability is updated adaptively and passed to the dropout
layer. In weight decay, a controlling term is added to the error
function of CNN.
Err or new = M S E T rain + a| |w| |2F
where a is the scaling parameter and | |w| |2F is the Frobenius
norm of synaptic weights. Based on the pre-defined threshold
and the ratio of mean squared error on the validation set over
mean square error on the training set, the scaling parameter is
updated adaptively. Two CNNAR were trained, one for feature
extraction and one for driving style classification. The pro-
posed system was able to achieve 95% classification between
a dangerous driver and a safe driver. CNN is a computationally
expensive algorithm and would take a substantial amount of
time to train the model as compared to other machine learning
algorithms. The proposed methodology was only applied on
data collected previously, and it is also unclear if such a system
can actually be implemented in a smartphone.
On the other hand, Carvalho et al. [79] performed a driver
behavior profiling using three different types of Recurrent
Neural Network (RNN) namely, a simple RNN, Gated Recur-
rent Neural Network (GRU) and LSTM. Their study was based
on using only the accelerometer readings and was aimed to
classify between aggressive braking, aggressive acceleration,
aggressive left turn, aggressive right turn, aggressive left lane
change, aggressive right lane change and non-aggressive event.
To ensure both the smartphone and vehicle’s coordinate
systems are aligned, the smartphone was fixed on the car’s
windshield using a car mount and a rotation matrix. Data were
then collected from 4 car trip and the total driving duration
was approximately an hour that provides a total of 69 events.
The authors [79] then used 70% of the data as training data
and 30% of the data as testing data. Based on their result, GRU
and LSTM were the best performing RNN which achieved an
accuracy of above 95%. However, the accuracy of LSTM was
easily affected by the number of neurons.
Even though the results were good, the number of events
considered were only 69 from two experienced drivers.
In addition, two of the categories (aggressive left and right
lane change) were imbalanced as compared to other categories.
Moreover, data were collected during sunny weather, on a dry
road and paved with asphalt. It is unknown how wet weather
condition will affect the accuracy of the classifiers. Further-
more, the position of smartphone must be fixed throughout the
trip which may be affected by road anomalies such as pothole
or speed bump. Finally, model training and validation were
conducted offline, as it is unclear if model training can work
in a smartphone environment and if the good performance is
consistent for streaming data.
E. Ensembled Learning Methodology
To improve the detection accuracy, Bejani and Ghatee [80]
proposed the use of ensemble learning. To allow reliable data
collection, the smartphone must be placed in a specific position
followed by a calibration process to align the smartphone axes
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4457
with the car axes. The t-SNE algorithm was then applied on
raw data to extract features relevant to different labels. The
combination of C4.5 (a type of decision tree) and Radial Basis
Function (RBF) was applied to classify the type of maneu-
ver (U-turn, turns or lane change) using a set of piecewise
aggregated magnetometer data (different window of samples
(4) are averaged) as the input. A Multi-Layer Perceptron (MLP)
was then applied to determine the traffic situation (congested
or no congestion) based on the speed derived from GPS and
zero crossing rate of the data collected by the y-axis of the
accelerometer. MLP was again applied to classify the type
of cars into high and low sensitivity car by considering the
zero crossing rate, variance, minimum, maximum of data col-
lected by z-axis of the accelerometer. The fusion of K-nearest
neighbor (KNN), MLP and SVM with RBF were then applied
to determine if the maneuver was safe or dangerous under
different traffic condition. The results of three algorithms
were fused with a voting mechanism to produce the initial
evaluation. Based on the output of the algorithm, a Fuzzy
Inference System (FIS) was applied to determine the risk level
of the driver. The result then passed to another FIS to provide
suggestion if the maneuver was deemed as dangerous. Based
on this implementation, the authors [80] reported an accuracy
of 94%.
The main drawback of this system is the large number of
algorithms that make it complicated and difficult to implement
in a real system. There is much work involved to retrain all
the models as the system cannot be used in different cities due
to different traffic conditions. In addition, such methodology
also requires a system with a high level of computation power
to run all the algorithms. Moreover, all training procedure
was performed in a MATLAB environment which prompts the
question of how a user can update the system. Subsequently,
this system only takes into account of turns, U-turn and lane
change. Besides, ground truth obtained was based on expert
scoring of driving style that is subjected to subjective judg-
ments of other drivers or experts [81]. Furthermore, the use of
piecewise aggregation to smooth the magnetometer data may
cause some temporal information to be averaged out. Finally,
the smartphone also must be placed in a specific position for
proper calibration.
Xie et al. [82] also proposed a methodology utilizing
ensemble learning using data collected from accelerometer,
gyroscope, and GPS. In their system, data collected were
first filtered by a moving median and moving mean filter
for noise removal. Feature extraction was then performed
on the filtered data to form a feature vector of dimension
273 and z-scored normalized. Ensemble learning using KNN,
Logistic Regression (LR), Naïve Bayes (NB) and RF were
then applied where the output of each classifier was fused to
give the final output. Based on such a system, Xie et al. [82]
achieved an accuracy of weighted F1-score of 87%. However,
this system was tested only to one case of driver distraction
due to talking inside the car. An experienced driver is less
likely to be affected by talking to a passenger. In addition,
the dimension of the feature vector is very large, that requires
a high computational effort. Finally, such a methodology was
not tested in a smartphone environment.
4458 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
Guo et al. [83] proposed a novel algorithm combining
Autoencoder and Self-Organizing Map (AESOM) for driving
behavior clustering using only the GPS sensor in a smartphone.
As GPS data may be affected by noise or outlier, a software
is embedded within the GPS receiver to filter the data with
a propriety filtering algorithm automatically. Based on the
collected GPS data, a total of 10 input features related to
speed and acceleration were estimated. Features were then z-
scored normalized and fed into the autoencoder for extraction
of latent features and followed by the passing of latent features
to the SOM for clustering of different driving behavior. In their
paper, Guo et al. [83] did not report any detection accuracy
rate but showed the loss value of the autoencoder is between
0.03 to 0.08 and the clustering results of the different type of
driving behavior such as clustering acceleration behavior into
a slight, moderate and heavy. As a result, there is no way to
determine the accuracy of the algorithm. Moreover, validation
of detected abnormal driving behavior against the ground truth
was not performed. As such, this methodology lacks reliability.
The algorithm was also not applied in real time. Instead, it was
applied on data collected for a period of time. It is also unclear
how the new data collected is related to the clustering results
and how it is clustered into one category. Therefore, such
algorithm lacks practicality when it can only be applied to
a group of data for post-event analysis and cannot be used
for online behavior analysis. Finally, Guo et al. [83] did not
provide any details on the ground truth (abnormal driving)
obtained.
Daptardar et al. [84] proposed a Hidden Markov
Model (HMM) based driving event detection methodology.
Prior to the data collection, alignment of smartphone coor-
dinate system must be in sync with the vehicle coordinate
system such that the y-axis of the accelerometer is aligned
with the moving direction of the vehicle. Data collected were
then filtered with a low pass filter and Kalman filter to remove
noise due to road anomalies. Data collected from the z-axis
of the gyroscope was leveraged to detect the lateral driving
maneuvers (lane change and turns). The data collected from
the y-axis of the accelerometer was leveraged to detect the
longitudinal driving maneuvers (hard accelerating and hard
braking). Gyroscope data were linearly fitted to reduce dimen-
sionality and slopes of the linearly fitted data was calculated
as the input to the HMM model where the HMM detects the
occurrences of events. If input slopes are high, represented by
high gyroscope values followed or preceded by low slopes,
then an event had taken place. The output of the HMM was
then passed to an RF classifier to categorize the event as a lane
change or turn. On the other hand, a severity index was used
to detect hard accelerating and hard braking. First, the data
from the y-axis of the accelerometer was used to calculate the
jerk energy defined as the rate of change of acceleration. The
severity index was then calculated using the jerk energy given
as
I ndex = (RoV ∗ (J E Max − J E Min ))/100
where RoV is the rate of change of velocity, J E Max and
J E Min are the maximum and minimum jerk energy in the
event window. Samples of a particular window which produce
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
value above 50 will be classified as aggressive events. How-
ever, it was not mentioned how this value could be classified
into hard accelerating or hard braking, but it was reported that
such system achieved a detection accuracy of 95%.
In this methodology, Daptardar et al. [84] predefined 4 states
for the HMM considering slopes and acceleration but it is
highly difficult to generalize the states as the two parameters
are bound to change on different kinds of roads as well
as different locations [85]. Moreover, the hard threshold of
severity index also limits the system to be used in a specific
location with a specific phone and vehicle. It is because it
is unclear if the same threshold can be used for different
phones and car since the sensitivity of the accelerometer
and characteristics of cars are different. While the detection
accuracy is high, the system can only detect hard acceleration
and hard braking. Lastly, the smartphone must be placed in a
specific position where the coordinate system is aligned with
the vehicle’s coordinate system.
F. Unsupervised Methodology
On the other hand, Yang and Hansen [86] proposed the use
of unsupervised clustering approach to access driving behav-
ior using data retrieved from in-vehicle smartphone Inertia
Measurement Unit (IMU) and GPS sensors. Their method-
ology was specifically designed to grade the performance
of 5 maneuvers, left turn, right turn, gas hit, brake hit, and
normal driving. For this methodology, the data collected were
first filtered by the Median filter to remove noise followed
by resampling of data to have a uniform rate using linear
interpolation. A coordinate transformation was then performed
to convert the smartphone centered X, Y, Z accelerations to
vehicle centered acceleration. Global-referenced GPS bearing
directions was also converted to vehicle-referenced moving
angles. Subsequently, an alignment step was done to assign the
axis by referring to gravity and GPS information. To identify
the event start and end, GPS data was used. As suggested
by Yang and Hansen [86], peaks and troughs of GPS speed
derivative indicated the occurrences of gas-hit/brake-hit events
while unwrapped GPS bearing derivative indicated a right or
left turn. A total of 7 features (mean, variance, mean crossing
rate, peak frequency, spectral energy, entropy, and correlation)
were extracted from each axis of the accelerometer. Based on
the features, a cluster scheme was applied iteratively for three
times where each of them was labeled with a grade (A to D).
It is assumed that for an experienced driver, the majority of
the maneuvers will have grade A due to safe driving. The
results were only compared with a grading providing by the car
insurance. Therefore, results were inconclusive as they were
not verified against the ground truth. In addition, smartphone
location was only tested in one location and may not be
free-positioned as suggested by the authors [86].
G. Combination of Visual and Non-Visual Methodology
(5)
On the other hand, Bergasa et al. [20] proposed an appli-
cation called DriveSafe that used the computer vision and
pattern recognition techniques to detect abnormal driving
due to drowsiness and inattentive drivers. As explained by
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
Bergasa et al. [20], lane weaving and drifting can be used
to infer drowsiness. On the other hand, sudden longitudinal
and transversal movements can be used to infer distractions.
Since drowsiness is inferred by lane weaving and drifting,
Bergasa et al. [20] proposed to use the rear camera of a
smartphone to capture the image of the road ahead. Image
captured will be pre-processed by converting it into greyscale
and resizing it to a size of 320x240 pixel. An adaptive Canny
algorithm and Hough transform were used to capture the lane
markings on the processed image followed by using Kalman
filtering to track the lane. Lane drifting detection was based
on Lanex (fraction of lane exit), which is defined as the
fraction of a given time interval spend outside a virtual driving
lane. If Lanex exceeds 80%, a lane drifting has occurred. The
microphone was used to detect the switching on of blinking
indicator by tracking the sound generated when a driver turns
on the blinking indicator. Therefore, if a turn or lane change
is made without switching on the blinking indicator, then
an involuntary lane change has occurred. As explained by
Bergasa et al. [20], actions such as acceleration, braking and
turning are highly correlated to the data collected by the
accelerometer. Therefore, any violation of the threshold set
by the authors [20], one can merely detect the critical event
(sudden acceleration, braking, and turning). However, as mea-
surements from accelerometer were noisy, data collected were
required to be clean using a Kalman filter. The use of the GPS
was to estimate the vehicle speed and road curvature where
both values were used to estimate the centripetal acceleration
of the vehicle due to the road. This value was used to decouple
the lateral acceleration due to the road curvature from the
one caused by wrong driver movements. Through this system,
the authors [20] reported a precision value of 0.74 to 0.93.
As the details of the false negative are not shown, the detection
accuracy cannot be calculated.
There are several shortcomings to this application. The
application cannot detect abnormal driving behavior occur at
speed below 50km/h as the application will only be activated
when the vehicle exceeds the speed of 50km/h. Using only
the sound of switching on the blinking indicator is also
unreliable as there is a possibility of misdetection for safe
drivers. Although authors [20] mentioned that detection of
such sound is out of their scopes, the main challenge of
such sound detection is the detection has to be carried out
in a noisy environment if the radio is turned on or when
the passenger is talking. Threshold values were also simply
taken from [87] as Paefgen et al. [87] also used an iPhone
in their studies. It is unclear if the same threshold can be
used for another type/brand of the smartphone as the threshold
used cannot be considered as industrial standards [88] since
it varies for different driver and phone. As the system relies
on image processing technique to detect lane marking, it will
be a challenge for the system to detect fading lane markings.
In addition, the error rate will likely increase in the event
of challenging light conditions due to rainy days or at night.
Moreover, the position of the phone must be fixed throughout
the journey.
Jiang et al. [89] also proposed a system utilizing a camera
that analyzes driver behavior at an intersection or junction
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4459
(at a stop sign and traffic light). This system was proposed to
identify drivers who dash across the red traffic lights as well as
making unsafe turns. Measurements from the accelerometer,
gyroscope, GPS were fused with the image captured by the
rear camera of a smartphone to form a vision-based driving
event detection system. Prior to the drive, the phone was first
mounted with its coordinate system aligned with the vehicle
coordinate system by aligning the z-axis of the smartphone
with the moving direction (y-axis) of the vehicle. The rear
camera of the smartphone must be facing the road without any
obstruction. The system must also be equipped with a prior
map containing location information of the traffic lights. The
light or sign detector using the smartphone rear camera will
only be activated when the distance between the car and
the traffic lights or sign is within 30m, and this distance is
estimated using the Haversine formula. To extract the shape
of the traffic light or stop sign from the captured image,
Jiang et al. [89] applied the HSV color model. By taking
weather into consideration, color ranges have a different range
for a stop sign and traffic light during different weather
conditions. To differentiate between a stop sign and traffic
light, the shape of the detected shape is compared with the
shape of the traffic light and stop sign. As explained by the
authors [89], the traffic light has a shape of a circle while
the stop sign has an octagon shape. In order to increase the
processing speed, noisy images caused by vibrations were
filtered based on the threshold value set on the x-axis of the
accelerometer. The filtered image was then downsampled to a
smaller size by using a Gaussian pyramid. Finally, to detect
unsafe turns, authors [89] proposed to first detect a turn by
tracking the measurement from the gyroscope x-axis, lane
marking is then detected using the canny algorithm and Hough
transform. A turn is detected if the threshold of 0.5 rad/s is
crossed and a drifting event is then detected if the midpoint
of the lane marker crosses the centreline of the lane marker
detection window.
The precision value of such a system to detect stop
sign, red light, stop sign driving event, red light driving
event and lane drifting event was found to be approxi-
mating 0.8. However, overall, detection accuracy was not
reported.
As the system is a vision-based system, detection error will
increase in the event of challenging light conditions due to
rainy days or at night. Roadside color objects such as vehicles,
buildings also add to the false positive detection of red lights
and stop signs. On the other hand, fading lane markers will
affect the detection accuracy of the system. The system is
also not robust against road with a turn lane. If a driver drifts
into the turn lane and makes a turn at an intersection without
stopping when the traffic light is green, such action is detected
as a lane drifting event.
In addition, yellow light is not considered in their work that
may result in several false positive. In the event of a yellow
light, driving behaviors at intersections becomes complicated.
While drivers should stop the vehicle in advance, it may not
always be the best choice in every situation. If the vehicle is
close to the stop line at high speed, then it is much safer to
pass the intersection in this situation [90].
4460 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
Furthermore, if the prior map installed on the smartphone
is not updated with the latest information of the stop sign
and traffic lights, then the system will not be activated when
approaching an unmarked intersection. The threshold used to
detect turn may not be a value that can be used universally.
Finally, the position of the smartphone must be fixed through-
out the journey.
Saleh et al. [91] methodology also included the use of the
camera, but they utilized a Long Short Term Memory (LSTM)
for classification of driving behavior. The motivation was to
eliminate the manual handcrafting of features extraction as
LSTM can enable the exploitation of inherent dependency
and correlation between different sensor data captured at each
time step of real driving sessions. For their methodology,
Saleh et al. [91] proposed to use measurements collected by
the accelerometer, GPS and the rear camera of the smartphone.
As data were collected with the different sampling rate,
collected data with the lowest sampling rate were up-sampled
to match the highest frequency used for data collection using
Finite Impulse Response (FIR) filter. They were followed by
z-score normalization to produce data of mean 0 with standard
deviation 1. A rolling window of size 64 overall features with
an overlap of 50% was then done to obtain 9499 window
samples of which 70% was used for training and the rest
for validation. Although, the methodology was not tested
in real life application, Saleh et al. [91] methodology was
applied on the UAH-Drive set dataset which was introduced
for driving behavior analysis, and they reported an F1 score
of 0.91. It is unclear if training of model can be carried out
in the smartphone environment. Otherwise, the model must be
trained outside a smartphone environment and network para-
meter transferred to the phone, which may be inconvenient.
As hyperparameters are virtually independent and should be
optimized separately [92], it is also unclear if model training
can be carried out automatically for a new driver. In addition, a
large amount of training data was also used for such a system.
H. Sound Methodology
Finally, Xu et al. [93] proposed an interesting sys-
tem for early recognition of inattentive driving using a
smartphone microphone. The idea is the same as [58] which is
to utilize the Doppler shift of audio signals caused by human
action. As explained by the authors [93], such application is
suitable for identifying the four representatives of different
inattentive driving behavior (fetching forward, picking up
drops, turning back and eating and drinking) because they
have unique Doppler profile. Data for different inattentive
driving behaviors were collected by using a smartphone as
a transmitter and receiver device.
Audio signals were transmitted at 20 kHz to filter back-
ground noise and effects of people. Collected signals were
then transformed into the frequency domain using Sliding
window Overlap Fourier Transform (SOFT) and features were
extracted using PCA. These features were trained using binary
SVM for every pair of inattentive driving event. A voting
mechanism was then provided to score each identified event.
The number of votes will determine the category of driving
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
behavior. If the number of votes does not satisfy the condition
of the number of the classifier minus 1, then it classified as
other driving event and required further processing. Such a
driving event is considered as events that belong to inattentive
driving but are hard to detect atypical inattentive driving.
Therefore, Xu et al. [93] proposed to use the Self-Organizing
Map to separate such events from normal driving. For early
recognition of inattentive driving, Xu et al. [93] suggested
using the degree of completion of inattentive behavior at the
time, t, using the following equation.
t − t0 π
A= = (6)
t1 − t0 T
where t1 is the end of action and t0 is the start of action
based on historical data. A group of gradient model for-
est was set up for early detection of events. To take into
account of false alarm caused by road bumps and car stop,
Xu et al. [93] also included the use of an accelerometer to
detect road bumps and car stop. In the online phase, signals
were collected and transformed into Doppler profiles in a
similar fashion. Based on the classifier trained, the system will
identify if the collected signals are the behavior of inattentive
driving. Based on the proposed system, it achieved an accuracy
of 94%.
Although the system is comprehensive, much work is
required for the training and tuning of several different algo-
rithms which includes SVM, SOM and gradient model forest.
Secondly, the degree of completion of inattentive driving
behavior is only based on 8 drivers that may not be comprehen-
sive enough because different drivers have different reaction
time in different scenarios. Furthermore, while the authors
consider atypical inattentive driving events, they did not study
on the effect of atypical attentive driving events. Example,
an attentive driver chewing on gum or passenger who is eating.
Therefore, such events may cause a false alarm by the system.
I. Summary
Section IV presents the discussion on different method-
ologies proposed by different authors. The summary of the
proposed methodologies and their limitations can be found
in Table 5 and 6. Table 5 and 6 contain information such
as smartphone sensor used, pre/post-processing procedure,
algorithm used, features extraction and calculation, drowsi-
ness detection accuracy and the limitations. In the pre/post-
processing column, the rough idea of how processing is done is
provided and in the algorithm used column, how the algorithm
is used is also indicated, example: Median Filter for noise
removal. Features extracted and calculated are all features
listed in the authors’ paper. They are features that are used
in system and not just features for the final classifier.
In short, driving behavior classification can be broadly
classified into two categories, 1) the classification of individual
driving events and 2) the detection of aggressive driving events
over a long window and extending detection to an overall
driver risk assessment [29], [30]. However, the key question
is, how can the classification of individual driving events or
overall driver risk assessment bring value-added features to
the driver? Therefore, if a system is classifying individual
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES 4461

TABLE V
D ETAILS ON P ROPOSED M ETHODOLOGIES FOR S MARTPHONE -BASED D RIVING E VALUATION

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4462 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020

TABLE V
(Continued.) D ETAILS ON P ROPOSED M ETHODOLOGIES FOR S MARTPHONE -BASED D RIVING E VALUATION

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES 4463

TABLE V
(Continued.) D ETAILS ON P ROPOSED M ETHODOLOGIES FOR S MARTPHONE -BASED D RIVING E VALUATION

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4464 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020

TABLE VI
L IMITATIONS ON D IFFERENT P ROPOSED M ETHODOLOGIES

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES 4465

TABLE VI
(Continued.) L IMITATIONS ON D IFFERENT P ROPOSED M ETHODOLOGIES

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4466 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
TABLE VI
(Continued.) L IMITATIONS ON D IFFERENT P ROPOSED M ETHODOLOGIES
driving events, it may be more worthwhile to consider how
can a driver be warned of his dangerous behavior and how
such information can be disseminated to fellow road users or
the authorities. Whereas, for a system that provides overall
driver risk assessment, it may be more attractive for the driver
where recommendations can be given according to improve
their driving [29].
Among the systems proposed, different systems have dif-
ferent requirements. Some systems have an active and passion
mode thus requiring user to manually activate the evaluation
process [65]. Other systems can automatically start the evalua-
tion process when the smartphone is located on the dashboard
and automatically off the evaluation process when the location
of the smartphone changes [76]. Some systems may require
the smartphone to be mounted in a fixed position [59],
[65]–[68] while others do not but require an additional reori-
entation process [86].
V. C HALLENGES
Based on the methodologies proposed earlier, different types
of challenges faced when implementing smartphone driver
behavior analysis are consolidated and discussed in details in
this section.
A. Visual Detection Methods for Drowsiness Detection
As discussed in Section III-A, drowsiness detection is
commonly detected through facial expression by using the
smartphone camera. However, there are several limitations
for such methodology as seen in Table 7. Firstly such
methodology is easily affected by poor illumination. As such,
several authors [48], [31], [94], [95] had proposed the use
of near InfraRed (IR) illumination to provide illumination
for image capturing under poor lighting conditions. The use
of IR illumination is a simple and effective approach for
pupil detection based on differential infrared lighting scheme.
The high contrast between the pupils and the rest of the
face can significantly improve the eye-tracking robustness
and accuracy [96]. Moreover, such implementation can help
in the detection of eye state even if the driver is wearing
sunglass [96], [97]. Moreover, clinical research has also found
that near IR does not induce neurodegeneration [98]. Therefore
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
the use of such light is safe for drivers without any long-term
detrimental effects.
However, there are several limiting factors for this imple-
mentation. In order, for such implementation to be effective,
lighting conditions must be relatively stable, and drivers must
be close to the light source [94], [96]. In order to overcome
the influence of light on the image, Zhang et al. [97] proposed
active IR light of 850nm and combined with an 850nm narrow-
band filter to get the images under the infrared illumination.
However, cameras in a smartphone may not produce quality
images in the IR spectrum. In addition, an 850 nm emitter
that has a red glow can be distracting to drivers because the
human eye is more sensitive to this wavelength [99].
The second limitation of visual methodology for drowsiness
detection is possible false positive or false negative in several
scenarios. The use of PERCLOS may fail in the event when
the driver sleeps with his eyes open [15] and also in the event
when the driver is in a state of daze. As such, PERCLOS can
produce a false negative for these two conditions. Also, false
positive can be produced in cases where drivers close their
eyes for some time in an attempt to moisten their eyes [100].
In addition, tracking of mouth contours to detect yawn may
produce several false positive due to the driver talking or
singing [15].
Finally, visual detection methodology cannot work if it
cannot detect what it is set out to detect (i.e., failure to capture
an image of the driver’s eye due to long fringe or other
accessories or in the event driver tends to keep moving or
shifting his position). Unfortunately, there has not yet been a
solution in the event if the driver’s eye is covered by his fringe.
To mitigate the effects of driver’s movement, several cameras
or smartphones may be required [100]. However, that would
suggest an increase in hardware cost. In addition, the mounting
of several cameras or smartphones at eye level would obstruct
the driver’s vision.
B. Abnormal Driving Detection Methods
On the other hand, a non-visual approach that utilizes
smartphone’s sensors to detect abnormal driving behavior is
not without any drawbacks. As mentioned earlier, driving
behavior classification can be broadly classified into two
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
TABLE VII
C OMPARISON B ETWEEN C LASSIFICATION OF I NDIVIDUAL D RIVING E VENTS AND OVERALL D RIVING R ISK A SSESSMENT
categories, 1) the classification of individual driving events
and 2) the detection of aggressive driving events over a long
window and extending detection to an overall driver risk
assessment [29], [30].
The former is dedicated to identify specific aggressive
maneuvers and would first require the detection of a maneuver,
followed by classifying the aggressiveness of the maneu-
ver. Although Table 5 indicates that such implementation has
high accuracy, it is not a trivial task. The first critical issue to
address is the number of events to consider. As seen in Table 5,
different authors considered various types of maneuver. In such
cases, authors focus only on several specific maneuvers with-
out considering all possible maneuvers. Moreover, considering
only a specific maneuver without understanding its variants
can cause an incomplete evaluation [75]. It may not be possible
to consider every single type of maneuvers leading to the
question of how many events needs to be considered before
it is exhaustive. Finally, such implementation are targeted
towards the quality of maneuvers instead of the quality of
overall driving.
On the other hand, an overall driver risk assessment does
not require a large number of labeled data showcased in [70].
As driving style of each driver is unique, a unique set of
threshold may be required to gauge the level of abnormality in
their driving. Thus, the first question would be how can person-
alized labeled data be collected for supervised methodology in
a safe and trustable way. Subsequently, as the classification of
driving style is done after collecting a period of the signal,
it may be too late when abnormal driving was detected [101].
In addition, driving style can be comparable if there is a
set of reliable scoring system [102], [103] and if the test is
conducted repeatedly using the same vehicle type and road
situation [102].
In some overall driver risk assessment systems, a driver is
required to complete a questionnaire in order to evaluate the
driving risk. However, it may induce some form of biasness.
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4467
As seen in Amadoa et al. [104], questionnaires are completed
before the actual drive, It contains a vague reference point
problem since drivers do not have an objective input related
to their own performances.
Assessing drivers in terms of the expected performance on
a future task might be more related to their confidence in their
general driving skill, rather than the specific behavioral output.
In fact, a lower correspondence between subjective ratings
and actual performance and self-enhancement bias as found
in previous studies [104] could be seen. If the questionnaire
is done after the drive, drivers rely on their memory that
can be inaccurate due to impression management and self-
deception [33]. Thus, it is important to consider the issue
related to each type of implementation which are summarized
in Table 7.
The next limitation of using smartphone sensors for abnor-
mal driving behavior analysis is the varying level of accu-
racy due to different hardware specifications on a different
phone. A homogeneous precision and sampling rate of the
sensor data may not be guaranteed in every existing device.
A compromise has to be reached to accommodate most of
the existing hardware [25]. Furthermore, IMU in smartphones
are considered to be the lowest grade of sensors among the
commercial sensors [30], [105] and can exhibit significant
bias, scale factor, misalignment, and a high level of noise [28],
[29], [106]. Likewise, vibrations from the vehicle, road con-
dition, and environment can also contribute to noisy sensor
readings [107]. Additionally, measurements of a magnetometer
can be corrupted easily by electromechanical components or
can exhibit sharp variation due to sudden changes in phone
orientation [108], [109]. Furthermore, measurements provided
by a magnetometer are relative to the Magnetic North, not to
the Geographic North Pole. The difference between both is
the magnetic declination can vary with time and place [108].
Therefore, there is a need to preprocess the data before any
further analysis. An easy and perhaps the most common way
4468 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
(as seen in Table 6) to remove noise is by adopting low, high,
median or bandpass filter.
The use of GPS also presents several obstacles. Firstly,
the estimation of speed using GPS can be reliable and
accurate, but a significant amount of processing is required
for driving analytics [28]. Moreover, the accuracy of GPS
location data is inconsistent as it is affected by the current
location of the vehicle such as in a tunnel, basement, forest,
high-building as well as incorrect placement of phone within
the vehicle [110], [111]. In addition, the sampling rate of
the smartphone-based GPS is only about 1Hz, which is too
slow to detect and classify certain maneuvers accurately [110]
as compared to IMU which has sampling rates of 20Hz to
300 Hz [30]. Subsequently, GPS has high energy consumption.
The average power drain by a GPS while sampling at 1Hz
is approximately 370mW, whereas the average power drain
of sampling gyroscope, accelerometer, and magnetometer at
100Hz is only at around 60mW [110]. As such, the phone
must be charged constantly throughout the journey.
In addition, the use of GPS stirs privacy issue. Although
logistics fleet administrator can track how their fleet is being
used and how their drivers behave so that potential risk and
operational cost can be reduced [24]. But the drivers may feel
being micro-managed and monitored during their entire work
hours. In addition, insurance companies also face challenges
trying to convince users to use usage-based insurance due to
tracking issue [75].
This is because individual daily whereabouts is still a
private issue and tracking of the vehicle through GPS is a
privacy violation [75]. Furthermore, tracking the location of an
individual over a period can reveal his frequent destinations.
In a worst-case scenario, such information can be used to
predict the future destinations of an individual [112].
In another study [113], it was found that if a company
explicitly explain to the customer that fair information practice
will be followed, the customer is more likely to disclose their
personal information. As long as the user of a system is not
unwillingly identified, monitored or tracked by the system
and the right to reject being tracked in public or semi-public
spaces are given, it should be fine. The company should also
ensure that such information can only be shared (with user
consent) with the relevant entities for a specific time to avoid
tracking [112].
Subsequently, most of the methodologies and applications
require the driver to mount their smartphone in a fixed position
or dynamically reorienting its axes by real-time computation
of the Euler angles [111]. It is because measurements cannot
be used directly without knowing the position of the phone
inside the vehicle. Therefore, an alignment strategy is needed
to align the smartphone’s coordinate system with the vehicle
coordinate system [107]. Even if the device is perfectly fixed to
a phone mount, there are potential errors since the device itself
might not be able to determine if the vehicle is on a flat surface
or is tilted due to the slope of a road [114]. Further complicat-
ing the alignment issue is that the position of smartphones may
change daily or even during the journey due to user interaction
and vibration [25] , [30], [88]. Moreover, some of the positions
where the phone is mounted pose safety concerns. In one of the
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
literature, authors [115] mount the smartphone directly to the
steering wheel. While the system may detect steering actions
with higher accuracy, the smartphone directly hinders with
the deployment of the airbag. When the airbag is deployed,
it may fling the device towards the driver at high speed causing
grievous injuries.
An abnormal driving behavior detection system may not
be useful or even impractical if it is designed to operate
only in one particular position. If the smartphone is not in
a fixed position, small orientation changes can happen due
to vehicle moving over road slopes or speed bump leading
to measurement errors. While dynamic reorientation strategies
can rectify such issues and are easy to calculate, they produce
sensor data with significant errors due to the angles evolving.
Therefore, to implement an efficient reorientation strategy,
it is of utmost importance to consider an efficient approach
to assess when and how to correct the orientation of the
smartphone concerning the vehicle’s coordinate system [28].
Thus, if the device changes its position for some unknown
reason, the specific time interval of position change can be
detected and the new position of the phone can be recognized
before proceeding to detect subsequent events [28].
Another issue to consider is the type of feature extraction
to use for driver behavior analysis. While statistical features
are easy to calculate and extract, they are usually handcrafted
and have to be carefully examined to determine the level
of importance towards the end goal. It may lead to large
feature dimension, excessive training time, low prediction for
the model, and poor real-time performance [116].
On the other hand, unsupervised feature extraction method-
ology such as Principal Component Analysis (PCA) and
Independent Component Analysis (ICA) can automatically
extract feature and filters out any redundant sensor informa-
tion [117]. PCA performs data transformation to find principal
components that can best explain the data and achieve good
results when the input data follow a Gaussian distribution
and essential hidden features orthogonal to each other in the
vector space [117]. Whereas, ICA can extract independent hid-
den features from multivariate signals by assuming observed
signals are generated via linear transformations from source
signals [117]. However, both methods may not be able to
extract the representative features from driving behavior data
by linear transformations as vehicle dynamics and human
driving behavior involve nonlinear properties [116], [117].
On the other hand, wavelet transform is a powerful tool for
nonstationary signal analysis for representing various aspects
of the signal, such as trends, discontinuities, and repeated pat-
terns [118]. The wavelet transformation can convert the signal
into components of a different frequency while preserving
information related to the time domain [115]. However, given
that wavelet analysis provides a signal representation of higher
precision at low frequencies and less at high frequencies [119],
it can result in a loss of information if the high-frequencies
signal carry information about the driver behavior or signature
of a road anomaly [120].
On the other hand, the current state of the art, Deep Neural
Network (DNN) not only omits the process of handcraft-
ing features but increases the recognition accuracy [121].
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
In simple, DNN can be described as a NN with a larger
number of hidden layer where each layer is parametrized by
a set of weights and bias. The shallower layer can extract
low-level features to represent more specific driving behaviors
while deeper layer can extract higher-level features to represent
more abstract driving behavior by fusing various low-level
features [117]. However, one major shortcoming of DNN
is requirement of a large amount of data for training as
insufficient data reduces the network’s generalization ability
leading to over-fitting [121]. Thus, it may be worthwhile to
consider the solution provided in [78] to tackle the issue of
overfitting. Finally, for most of the methodologies proposed,
it is unknown if training and updating of the algorithm can be
done in a smartphone environment.
VI. P OSSIBLE S OLUTIONS
In this section, some possible solutions are provided and
discussed which may help to tackle some of the challenges
mentioned in the earlier section. This includes the inclusion
of context-aware system, mobile crowdsensing and active
steering control to improve driving safety.
A. Context-Aware Driving Behavior Detection System
A driver’s behavior can easily be influenced by any of the
factors listed in Table 1. However, effects of environmental
conditions, as well as car condition on driver behavior, were
usually excluded in the decision-making process even though
they may play a big part on the driver behaviours [80]. The
difference in traffic condition can be used as one example. The
difference in behavior can easily be spotted when driving on
a normal road and driving on a congested road where a driver
tends to brake more often in a congested road.
A possible framework to consider would be the context-
aware system. Context can be formally defined as the infor-
mation used to characterize the situation of entities (person,
place or object) that are considered relevant to the interaction
between a user and an application, including the user and
the application themselves [122]. Thus, in the Intelligent
Transportation System (ITS) domain, a driving context can
be considered as a set of facts or circumstances that sur-
round a driving activity. For example, but not restricted to,
surrounding objects, weather, traffic and demographic infor-
mation, vehicle’s states, driver’s psychological and physical
states [123]. Context-aware system is particularly desirable if
it can react specifically to their current location, time and other
environmental factors and adapt their behavior according to the
changing circumstances [122].
In the ITS domain, the context-aware system usually con-
sists of three phases in their architecture, which are, 1) Sensing
phase, 2) Reasoning phase and 3) Acting phase [124], [125].
In the sensing phase, the system collects different information
through multiple sensors (low-level contextual information).
The reasoning phase is responsible for performing reasoning
on low-level contextual information to deduce the behavior
of a driver (high-level contextual information). This phase
may include calculating corrective actions for other road users.
In the acting phase, the system is responsible for giving alarms
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4469
Fig. 5. Typical Mobile Crowdsensing Architecture [131].
and if corrective actions are calculated, such information is
also disseminated to other road users [124].
It was demonstrated in [80], by considering the car’s
sensitivity, evaluation of the driver’s style can be improved.
Thus, showing that driving context is useful in explaining
driving behavior and can improve the generalizability and
reliability of existing driving behavior models [126]. However,
the key limiting factor is the acquiring of low-level contextual
information during the sensing phase. A large number of
sensors have to be considered in order to retrieve a large
amount of context information, and a smartphone is unlikely
to have the capability to perform such a task. Secondly,
identifying the relevant context is a challenge as the type and
number could vary significantly with respect to the situation
and driver. Thus, it can become an intractable problem [126].
B. Mobile Crowdsensing for Information Dissemination
As discussed in the earlier section, one of the factors
influencing driver’s behavior is the environmental conditions.
By considering such factor, the accuracy of driver behavior
analysis can increase. One alternative solution other than the
context-aware system is through a paradigm known as mobile
crowdsensing.
Mobile crowdsensing is a new paradigm that utilizes a
large number of smartphones to collect and analyze data from
the environment through its sensors to benefit users through
widespread information dissemination [127]–[130]. The key
difference between context-aware and mobile crowdsensing
is their primary objective. The former is focused on the
retrieval of low-level contextual information to deduce high-
level contextual information to assist the driver. The latter
is focused on analyzing collected data and the sharing of
analytical results to other road users. The typical mobile
crowdsensing architecture can be seen in Fig. 5.
In the case of driver behavior analysis, mobile crowd-
sensing can be beneficial by sharing road and traffic condi-
tion [67], [129], [131]. As discussed in the earlier section,
Singh et al. [67] utilized mobile crowdsensing to confirm road
congestion. As described by the authors [67], congestion on
the road can be detected when multiple smartphones detect the
braking event in the same geographical area. Such information
can be used to distinguish braking due to congestion and
sudden braking due to harsh driving, thereby providing higher
analytical precision.
Similarly, the sharing of road surface information can
also increase the accuracy of driver behavior analysis. One
common methodology to detect aggressive driving is through
analyzing data collected from a smartphone’s accelerometer.
4470 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020

Fig. 6. Benchmark results for different android devices running DNN [135].

However, speed bumps and pothole can affect the analysis implement crowdsensing-based applications may be a limiting
result. Thus, if any driver who detected such anomaly can factor [71].
send this information together with its corresponding location
to any server or cloud platform. The next driver can download C. Active Steering Control for Improved Driving Safety
such information into the abnormal driving detection system.
Secondly, if a smartphone can make accurate classification
As this concept involves a large number of participants, road
of abnormal driving behavior, then it may be appropriate
surface information can be frequently updated, resulting in a
that such diagnostic is integrated into the ADAS and allows
more accurate road surface information database [131]. Using
steering control of the vehicle by the smartphone. Although
this database, any driver behavior analysis system can take
some implementations [47], [48], [51] provide passive feed-
the road anomalies into account and easily exclude behavioral
back for the drivers upon detecting traces of driver drowsi-
analysis during the period when the car is driving past the
ness, the driver may choose to ignore the warning given by
road anomalies.
the passive system [7]. Therefore, an active system where
Mobile crowdsensing also provide benefits in other areas.
the smartphone is given steering control may be a better
Firstly, the traveling route can also be dynamically updated
alternative. Upon detection of abnormal driving behavior,
using this information so that the best trip routes based on
the smartphone can take the current dynamics and kinematics
road quality can be recommended [132]. Secondly, mobile
model of the vehicle into consideration to control the lateral
crowdsensing can increase the efficiency of real-time driving
motion of a vehicle [134]–[139]. Also, a trajectory can be
recommendation through traffic information dissemination.
computed, and the steering control would then perform the
Similar to the concept of sharing road surface information,
maneuvering action based on the planned trajectory as closely
the corresponding location of traffic light can be shared.
as possible [140], [141]. Such implementation has the potential
System can then know in advance how far ahead is the traffic
to reduce the number of accidents. However, drivers may need
light and provide a gentle reminder for the driver to slow the
to be educated in how such system works as haptic guidance
car down. Finally, mobile crowdsensing can also promote road
can easily cause driver’s annoyance [7] and in a study carried
safety. The primary objective of a behavior analysis system
out by Katzourakis et al. [142], drivers may fail to understand
is to detect any abnormal driving behavior. Therefore, if the
the functionality of a lane departure assistance system leading
information of detected aggressive driver can be shared among
to inadvertent driver reactions.
other road users, other road users can take precaution in
advance and reduce the possibility of collision.
While such a concept brings great benefits, there are also D. Mobile/Cloud-Based Training and Updating of Algorithms
several drawbacks. The number of participants of a mobile As mentioned earlier, the two most challenging question
crowdsensing application may vary as it is based on voluntary for smartphone-based driver behavior analysis is “can the
participation and contribution which cannot be controlled or algorithms can be trained and updated in a smartphone envi-
planned [130], [133]. In addition, the required cost and time to ronment?” and “If no, is there any possible solution?” In this

Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
Fig. 7. Framework of Tensorflow lite [145].
subsection, the discussion of the solution towards these two
questions will be presented.
Although most of the proposed system did not mention
if training and updating of an algorithm can be done in a
smartphone, but in general, the modern smartphone is capable
of such a task. One good evidence for such claim is the Apple’s
iPhone and a simple example would the predictive keyboard
which demonstrates on-device training and updating. While
it can be argued that such application is trivial or uses a
simple model, but as published in Apple’s machine learning
journal [143], it is also possible to perform on-device Deep
Neural Neural (DNN) for face recognition. In a test carried out
by Ignatov et al. [144], a Samsung S3 that was manufactured
over more than 5 years ago is capable of running deep learning
solution although it is at a much slower rate as compared to
a newer phone. Figure 6 shows the portion of the benchmark
performance of different android devices running DNN. For
more information, reader is recommended to refer to [144] for
more details.
Therefore, it can be concluded that the smartphone is
capable of running a machine learning algorithm, leading to
the question of how can one load the model into a smartphone.
Fortunately, there are solutions cater towards this purpose such
as Tensorflow Lite from Google [145] and Core ML from
Apple [146]. As shown in Fig.7, such a solution can be easily
adopted.
However, there are still several issues that can restrict an
algorithm being deployed into a smartphone. Firstly, the model
may take up a large storage space and require significant
computational time on the GPU and/or CPU while sharing
resources with other running applications [143]. High com-
plexity model may even require the full computation power
of a smartphone and require other applications to be closed.
In addition, it may be possible that the temperature of the
device increases significantly due to running a computationally
intensive application. Finally, a single application needs to be
developed in different platforms (iOS, Android) to cate for
different groups of users.
Thus, if mobile training and updating of the algorithm
are not feasible, the next option would be to utilize cloud
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4471
computing. Cloud computing is a new way of how computing
technology is deployed where dynamically scalable and virtu-
alized resources are provided on demand [147], [148]. Cloud
resources are fully dedicated to a specific task and typically
use high-end GPUs with a large amount of memory avail-
able, thus removing any constraints relating to computational
resource [143]. Once the model is defined, the user can simply
request to train or update the model by uploading the new
data through an application and download the model when
the training is done that brings great convenience. However,
one factor to consider in regards to such a solution is the cost.
Such a solution may be expensive depending on the space and
resources used.
VII. C ONCLUSION
Data collected from the smartphone are a rich source of
information for analyzing driver behavior. Numerous method-
ologies proposed by different authors to detect different
kinds of abnormal driving were reviewed. While smartphone
solutions system has several advantages as compared to the
telematics boxes, there exist several challenges that must
be taken into consideration for an accurate driver behavior
classification.
One of the key challenges of using vision-based methodol-
ogy is the varying light condition that can greatly affect the
detection accuracy. The other key consideration for the sensor-
based methodology is how the noise and external factors such
as road conditions or vehicle electrical components can be
eliminated. It is imperative that such issues should be taken
into consideration in future works. In addition, future works
should also address the issue of mounting of the smartphone in
a fixed and specific position. Eliminating such a requirement
will allow noise caused by a sudden orientation change of the
smartphone to be eliminated. Furthermore, it brings greater
flexibility and convenience for the driver.
In this paper, some possible solutions were discussed that
include the integration of smartphone driver behavior analysis
together with the concept of context-aware, crowdsensing,
and steering control by smartphones. In addition, mobile or
cloud-based training and updating of algorithms were also
discussed. Although the solutions discussed may not be per-
fect, it has the potential to alleviate some of the issues faced.
Therefore, there is definitely room for further improvements
for a smartphone based behavior analysis system.
ACKNOWLEDGMENT
The authors would like to thank the reviewers for their
constructive comments which greatly improve the quality of
this manuscript.
R EFERENCES
[1] H. Y. Guo, Y. Ji, T. Qu, and H. Chen, “Understanding and modeling
the human driver behavior based on MPC,” in Proc. 7th IFAC Symp.
Adv. Autom. Control, Tokyo, Japan, Sep. 2013, pp. 133–138.
[2] A. B. Ellison, S. P. Greaves, and M. C. J. Bliemer, “Driver behaviour
profiles for road safety analysis,” Accident Anal. Prevention, vol. 76,
pp. 118–132, Mar. 2015.
[3] H. S. Kim, Y. S. Hwang, D. S. Yoon, W. G. Choi, and
C. H. Park, “Driver workload characteristics analysis using EEG data
from an urban road,” IEEE Trans. Intell. Transp. Syst., vol. 15, no. 4,
pp. 1844–1849, Aug. 2014.
4472 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
[4] Y. Liao, M. Wang, L. Duan, and F. Chen, “Cross-regional driver–
vehicle interaction design: An interview study on driving risk percep-
tions, decisions, and ADAS function preferences,” IET Intell. Transp.
Syst., vol. 12, no. 8, pp. 801–808, May 2018.
[5] B. Higgs and M. Abbas, “Segmentation and clustering of car-following
behavior: Recognition of driving patterns,” IEEE Trans. Intell. Transp.
Syst., vol. 16, no. 1, pp. 81–90, Feb. 2015.
[6] C. Miyajima et al., “Driver modeling based on driving behavior and
its evaluation in driver identification,” Proc. IEEE, vol. 95, no. 2,
pp. 427–437, Feb. 2007.
[7] C. M. Martinez, M. Heucke, B. Gao, D. Cao, and F.-Y. Wang, “Driving
style recognition for intelligent vehicle control and advanced driver
assistance: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 19, no. 3,
pp. 666–676, Mar. 2018.
[8] B. Shi, L. Xu, and W. Meng, “Applying a WNN-HMM based driver
model in human driver simulation: Method and test,” IEEE Trans.
Intell. Transp. Syst., vol. 19, no. 11, pp. 3431–3438, Nov. 2018.
[9] B. Zhu, J. Zhao, S. Yan, and W. Den, “Personalized lane-change
assistance system with driver behavior identification,” IEEE Trans. Veh.
Technol., vol. 67, no. 11, pp. 10293–10306, Nov. 2018.
[10] A. Bender, G. Agamennoni, J. R. Ward, S. Worrall, and E. M. Nebot,
“An unsupervised approach for inferring driver behavior from natural-
istic driving data,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 6,
pp. 3325–3336, Dec. 2015.
[11] J. Xu, J. Liu, X. Sun, K. Zhang, W. Qu, and Y. Ge, “The relationship
between driving skill and driving behavior: Psychometric adaptation of
the driver skill inventory in China,” Accident Anal. Prevention, vol. 120,
pp. 92–100, Nov. 2018.
[12] P. Stanojević, T. Lajunen, D. Jovanović, P. Sârbescu, and S. Kostadinov,
“The driver behaviour questionnaire in South-East Europe countries:
Bulgaria, Romania and Serbia,” Transp. Res. F, Traffic Psychol. Behav.,
vol. 53, pp. 24–33, Feb. 2018.
[13] M. Møller and S. Haustein, “Keep on cruising: Changes in lifestyle
and driving style among male drivers between the age of 18 and 23,”
Transp. Res. F, Traffic Psychol. Behav., vol. 20, pp. 59–69, Sep. 2013.
[14] W. Chu, C. Wu, C. Atombo, H. Zhang, and T. Özkan, “Traffic climate,
driver behaviour, and accidents involvement in China,” Accident Anal.
Prevention, vol. 122, pp. 119–126, Jan. 2019.
[15] S. Kaplan, M. A. Guvensan, A. G. Yavuz, and Y. Karalurt, “Driver
behavior analysis for safe driving: A survey,” IEEE Trans. Intell.
Transp. Syst., vol. 16, no. 6, pp. 3017–3032, Dec. 2015.
[16] J. Hu, L. Xu, X. He, and W. Meng, “Abnormal driving detection based
on normalized driving behavior,” IEEE Trans. Veh. Technol., vol. 66,
no. 8, pp. 6645–6652, Aug. 2017.
[17] A. Tawari, S. Martin, and M. M. Trivedi, “Continuous head movement
estimator for driver assistance: Issues, algorithms, and on-road evalu-
ations,” IEEE Trans. Intell. Transp. Syst., vol. 15, no. 2, pp. 818–830,
Apr. 2014.
[18] A. Sahayadhas, K. Sundaraj, and M. Murugappan, “Detecting driver
drowsiness based on sensors: A review,” Sensors, vol. 12, no. 12,
pp. 16937–16953, Dec. 2012.
[19] G. A. M. Meiring and H. C. Myburgh, “A review of intelli-
gent driving style analysis systems and related artificial intelli-
gence algorithms,” Sensors, vol. 15, no. 12, pp. 30653–30682,
Dec. 2015.
[20] L. M. Bergasa, D. Almería, J. Almazán, J. J. Yebes, and R. Arroyo,
“DriveSafe: An app for alerting inattentive drivers and scoring driving
behaviors,” in Proc. IEEE Intell. Veh. Symp., Dearborn, MI, USA,
Jun. 2014, pp. 240–245.
[21] J. Yu, Z. Chen, Y. Zhu, Y. Chen, L. Kong, and M. Li, “Fine-
grained abnormal driving behaviors detection and identification
with smartphones,” IEEE Trans. Mobile Comput., vol. 16, no. 8,
pp. 2198–2212, Aug. 2017.
[22] Y. Saito, M. Itoh, and T. Inagaki, “Driver assistance system with a
dual control scheme: Effectiveness of identifying driver drowsiness and
preventing lane departure accidents,” IEEE Trans. Human-Mach. Syst.,
vol. 46, no. 5, pp. 660–671, Oct. 2016.
[23] J.-L. Yin, B.-H. Chen, K.-R. Lai, and Y. Li, “Automatic danger-
ous driving intensity analysis for advanced driver assistance systems
from multimodal driving signals,” IEEE Sensors J., vol. 18, no. 12,
pp. 4785–4794, Jun. 2018.
[24] G. Castignani, T. Derrmann, R. Frank, and T. Engel, “Driver behavior
profiling using smartphones: A low-cost platform for driver moni-
toring,” IEEE Intell. Transp. Syst. Mag., vol. 7, no. 1, pp. 91–102,
Spring 2015.
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
[25] G. Castignani, R. Frank, and T. Engel, “Driver behavior profiling using
smartphones,” in Proc. 16th Int. IEEE Annu. Conf. Intell. Transp.
Syst. (ITSC), The Hague, The Netherlands, Oct. 2013, pp. 552–557.
[26] J. Ferreira, Jr., et al., “Driver behavior profiling: An investigation with
different smartphone sensors and machine learning,” PLoS One, vol. 12,
no. 4, Apr. 2017, Art. no. e0174959.
[27] P. Händel et al., “Insurance telematics: Opportunities and challenges
with the smartphone solution,” IEEE Intell. Transp. Syst. Mag., vol. 6,
no. 4, pp. 57–70, Winter 2014.
[28] E. I. Vlahogianni and E. N. Barmpounakis, “Driving analytics using
smartphones: Algorithms, comparisons and challenges,” Transp. Res.
C, Emerg. Technol., vol. 79, pp. 196–206, Jun. 2017.
[29] J. Engelbrecht, M. J. Booysen, G.-J. van Rooyen, and F. J. Bruwer,
“Survey of smartphone-based sensing in vehicles for intelligent trans-
portation system applications,” IET Intell. Transp. Syst., vol. 9, no. 10,
pp. 924–935, Dec. 2015.
[30] J. Wahlström, I. Skog, and P. Händel, “Smartphone-based vehicle
telematics: A ten-year anniversary,” IEEE Trans. Intell. Transp. Syst.,
vol. 18, no. 10, pp. 2802–2825, Oct. 2017.
[31] B.-G. Lee and W.-Y. Chung, “A smartphone-based driver safety
monitoring system using data fusion,” Sensors, vol. 12, no. 12,
pp. 17536–17552, Dec. 2012.
[32] R. Araújo, A. Igreja, R. de Castro, and R. E. Araújo, “Driving coach:
A smartphone application to evaluate driving efficient patterns,” in
Proc. IEEE Intell. Veh. Symp., Alcala de Henares, Spain, Jun. 2012,
pp. 1005–1010.
[33] J. H. Hong, B. Margines, and A. K. Dey, “A smartphone-based sensing
platform to model aggressive driving behaviors,” in Proc. SIGCHI
Conf. Hum. Factors Comput. Syst., Toronto, ON, Canada, Apr. 2014,
pp. 4047–4056.
[34] F. Giannotti et al., “Unveiling the complexity of human mobility by
querying and mining massive trajectory data,” VLDB J. Int. J. Very
Large Data Bases, vol. 20, no. 5, pp. 695–719, 2011.
[35] H. Barbosa et al., “Human mobility: Models and applications,”
Phys. Rep., vol. 734, pp. 1–74, Mar. 2018.
[36] L. Pappalardo, F. Simini, S. Rinzivillo, D. Pedreschi, F. Giannotti,
and A.-L. Barabási, “Returners and explorers dichotomy in human
mobility,” Nature Commun., vol. 6, Sep. 2015, Art. no. 8166.
[37] L. Pappalardo, S. Rinzivillo, Z. Qu, D. Pedreschi, and F. Giannotti,
“Understanding the patterns of car travel,” Eur. Phys. J. Special Topics,
vol. 215, pp. 61–73, Jan. 2013.
[38] F. Giannotti, M. Nanni, D. Pedreschi, C. Renso, and R. Trasarti,
“Mining mobility behavior from trajectory data,” in Proc. Int. Conf.
Comput. Sci. Eng., Vancouver, BC, Canada, Aug. 2009, pp. 948–951.
[39] D. Wang, Q. Liu, Z. Xiao, J. Chen, Y. Huang, and W. Chen,
“Understanding travel behavior of private cars via trajectory big
data analysis in urban environments,” in Proc. IEEE 15th Int. Conf
Dependable, Autonomic Secure Comput., 15th Int. Conf Pervasive
Intell. Comput., 3rd Int. Conf Big Data Intell. Comput. Cyber Sci.
Technol. Congr. (DASC/PiCom/DataCom/CyberSciTech), Orlando, FL,
USA, Nov. 2017, pp. 917–924.
[40] Y. Chen and C. Shen, “Performance analysis of smartphone-sensor
behavior for human activity recognition,” IEEE Access, vol. 5,
pp. 3095–3110, 2017.
[41] T. J. Matarazzo et al., “Crowdsensing framework for monitoring bridge
vibrations using moving smartphones,” Proc. IEEE, vol. 106, no. 4,
pp. 577–593, Apr. 2018.
[42] Y. Lu, A. Misra, W. Sun, and H. Wu, “Smartphone sensing meets
transport data: A collaborative framework for transportation service
analytics,” IEEE Trans. Mobile Comput., vol. 17, no. 4, pp. 945–960,
Apr. 2018.
[43] D. Kelly, K. Curran, and B. Caulfield, “Automatic prediction of
health status using smartphone-derived behavior profiles,” IEEE
J. Biomed. Health Informat., vol. 21, no. 6, pp. 1750–1760,
Nov. 2017.
[44] S. Wan, Y. Liang, Y. Zhang, and M. Guizani, “Deep multi-
layer perceptron classifier for behavior analysis to estimate Parkin-
son’s disease severity using smartphones,” IEEE Access, vol. 6,
pp. 36825–36833, 2018.
[45] D. Sommer and M. Golz, “Evaluation of PERCLOS based cur-
rent fatigue monitoring technologies,” in Proc. 32nd Annu. Int.
Conf. IEEE EMBS, Buenos Aires, Argentina, Aug./Sep. 2010,
pp. 4456–4459.
[46] B. Akrout and W. Mahdi, “A blinking measurement method for driver
drowsiness detection,” in Proc. 8th Int. Conf. Comput. Recognit. Syst.
(CORES), Miłków, Poland, May 2013, pp. 651–660.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
[47] L. Xu, S. Li, K. Bian, T. Zhao, and W. Yan, “Sober-drive:
A smartphone-assisted drowsy driving detection system,” in Proc. Int.
Conf. Comput. Netw. Commun. (ICNC), Honolulu, HI, USA, Feb. 2014,
pp. 398–402.
[48] A. Dasgupta, D. Rahman, and A. Routray, “A smartphone-based
drowsiness detection and warning system for automotive drivers,” IEEE
Trans. Intell. Transp. Syst., to be published.
[49] T. Hayashi, S. Watanabe, T. Toda, T. Hori, J. Le Roux, and K. Takeda,
“Duration-controlled LSTM for polyphonic sound event detection,”
IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 25, no. 11,
pp. 2059–2070, Nov. 2017.
[50] G. Lafay, M. Lagrange, M. Rossignol, E. Benetos, and A. Roebel,
“A morphological model for simulating acoustic scenes and its appli-
cation to sound event detection,” IEEE/ACM Trans. Audio, Speech,
Lang. Process., vol. 24, no. 10, pp. 1854–1864, Oct. 2016.
[51] K. Chang, B.-H. Oh, and K.-S. Hong, “An implementation of
smartphone-based driver assistance system using front and rear cam-
era,” in Proc. Int. Conf. Consum. Electron., Las Vegas, NV, USA,
Jan. 2014, pp. 280–281.
[52] F. De Ponte Müller, “Survey on ranging sensors and cooperative
techniques for relative positioning of vehicles,” Sensors, vol. 17, no. 2,
p. 127, Jan. 2017.
[53] D.-Y. Chen, G.-R. Chen, and Y.-W. Wang, “Real-time dynamic vehicle
detection on resourcelimited mobile platform,” IET Comput. Vis., vol. 7,
no. 2, pp. 81–89, Apr. 2013.
[54] Y. Qiao, K. Zeng, L. Xu, and X. Yin, “A smartphone-based driver
fatigue detection using fusion of multiple real-time facial features,”
in Proc. 13th IEEE Annu. Consum. Commun. Netw. Conf. (CCNC),
Las Vegas, NV, USA, Jan. 2016, pp. 1–6.
[55] Z. Li, G. Sun, F. Zhang, L. Jia, K. Zheng, and D. Zhao, “Smartphone-
based fatigue detection system using progressive locating method,” IET
Intell. Transp. Syst., vol. 10, no. 3, pp. 148–156, Apr. 2016.
[56] Y. Ji, S. Wang, Y. Lu, J. Wei, and Y. Zhao, “Eye and mouth state
detection algorithm based on contour feature extraction,” J. Electron.
Imag., vol. 27, no. 5, Feb. 2018, Art. no. 051205.
[57] C. Zhang, X. Wu, X. Zheng, and S. Yu, “Driver drowsiness detection
using multi-channel second order blind identifications,” IEEE Access,
vol. 7, pp. 11829–11843, Feb. 2019.
[58] Y. Xie, F. Li, Y. Wu, S. Yang, and Y. Wang, “D3-guard: Acoustic-based
drowsy driving detection using smartphones,” in Proc. IEEE Int. Conf.
Comput. Commun., Paris, France, Apr./May 2019, pp. 1225–1233.
[59] M. Fazeen, B. Gozick, R. Dantu, M. Bhukhiya, and M. C. González,
“Safe driving using mobile phones,” IEEE Trans. Intell. Transp. Syst.,
vol. 13, no. 3, pp. 1462–1468, Sep. 2012.
[60] Z. Ouyang, J. Niu, and M. Guizani, “Improved vehicle steering
pattern recognition by using selected sensor data,” IEEE Trans. Mobile
Comput., vol. 17, no. 6, pp. 1383–1396, Jun. 2018.
[61] Z. Chen, J. Yu, Y. Zhu, Y. Chen, and M. Li, “D3 : Abnormal driving
behaviors detection and identification using smartphone sensors,” in
Proc. 12th Annu. IEEE Int. Conf. Sens., Commun., Netw. (SECON),
Seattle, WA, USA, Jun. 2015, pp. 524–532.
[62] Q. Wang, Y. Gu, J. Liu, and S. Kamijo, “DeepSpeedometer: Vehicle
speed estimation from accelerometer and gyroscope using LSTM
model,” in Proc. IEEE 20th Int. Conf. Intell. Transp. Syst. (ITSC)
Workshop, Yokohama, Japan, Oct. 2017, pp. 214–219.
[63] Y. Gu, Q. Wang, and S. Kamijo, “Intelligent driving data recorder
in smartphone using deep neural network-based speedometer and
scene understanding,” IEEE Sensors J., vol. 19, no. 1, pp. 287–296,
Jan. 2019.
[64] A. Alasaadi and T. Nadeem, “UniCoor: A smartphone unified
coordinate system for ITS applications,” in Proc. IEEE 13th Int.
Conf. Mobile Ad Hoc Sensor Syst., Brasilia, Brazil, Oct. 2016,
pp. 290–298.
[65] D. A. Johnson and M. M. Trivedi, “Driving style recognition using a
smartphone as a sensor platform,” in Proc. 14th Int. IEEE Conf. Intell.
Transp. Syst., Washington, DC, USA, Oct. 2011, pp. 1609–1615.
[66] H. Eren, S. Makinist, E. Akin, and A. Yilmaz, “Estimating
driving behavior by a smartphone,” in Proc. Intell. Veh. Symp.,
Alcala de Henares, Spain, Jun. 2012, pp. 234–239.
[67] G. Singh, D. Bansal, and S. Sofat, “A smartphone based technique
to monitor driving behavior using DTW and crowdsensing,” Pervasive
Mobile Comput., vol. 40, pp. 56–70, Sep. 2017.
[68] A. H. Ali, A. Atia, and M. Sami, “Driving events recognition using
smartphone sensors,” Int. J. Ambient Comput. Intell., vol. 8, no. 3,
pp. 22–37, Jul. 2017.
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4473
[69] Z. Ouyang, J. Niu, Y. Liu, and J. Rodrigues, “Multiwave: A novel
vehicle steering pattern detection method based on smartphones,” in
Proc. IEEE ICC Ad-hoc Sensor Netw. Symp., Kuala Lumpur, Malaysia,
May 2016, pp. 1–7.
[70] G. Castignani, T. Derrmann, R. Frank, and T. Engel, “Smartphone-
based adaptive driving maneuver detection: A large-scale evaluation
study,” IEEE Intell. Transp. Syst., vol. 18, no. 9, pp. 2330–2339,
Sep. 2017.
[71] D. G. Costa, A. Damasceno, and I. Silva, “CitySpeed: A crowdsensing-
based integrated platform for general-purpose monitoring of vehicular
speeds in smart cities,” Smart Cities, vol. 2, no. 1, pp. 46–65, Feb. 2019.
[72] P. Chaovalit, C. Saiprasert, and T. Pholprasit, “A method for
driving event detection using SAX on smartphone sensors,” in Proc.
13th Int. Conf. ITS Telecommun., Tampere, Finland, Nov. 2013,
pp. 450–455.
[73] J. Lin, E. Keogh, S. Lonardi, and B. Chiu, “A symbolic representation
of time series, with implications for streaming algorithms,” in Proc. 8th
ACM SIGMOD Workshop Res. Issues Data Mining Knowl. Discovery,
San Diego, CA, USA, Jun. 2003, pp. 2–11.
[74] J. Xie, A. R. Hilal, and D. Kulić, “Driving maneuver classification:
A comparison of feature extraction methods,” IEEE Sensors J., vol. 18,
no. 12, pp. 4777–4784, Jun. 2018.
[75] H. R. Eftekhari and M. Ghatee, “Hybrid of discrete wavelet trans-
form and adaptive neuro fuzzy inference system for overall driving
behavior recognition,” Transp Res. F, Traffic Psychol. Behav., vol. 58,
pp. 782–796, Oct. 2018.
[76] H. R. Eftekhari and M. Ghatee, “A similarity-based neuro-fuzzy
modeling for driving behavior recognition applying fusion of smart-
phone sensors,” J. Intell. Transp. Syst., vol. 23, no. 1, pp. 72–83,
Jan. 2019.
[77] K. B. Ahmed, B. Goel, P. Bharti, S. Chellappan, and M. Bouhorma,
“Leveraging smartphone sensors to detect distracted driving activities,”
IEEE Trans Intell. Transp. Syst., vol. 20, no. 9, pp. 3303–3312,
Sep. 2019.
[78] M. M. Bejani and M. Ghatee, “Convolutional neural network with
adaptive regularization to classify driving styles on smartphones,” IEEE
Trans. Intell. Transp. Syst., to be published.
[79] E. Carvalho et al., “Exploiting the use of recurrent neural net-
works for driver behavior profiling,” in Proc. Int. Joint Conf. Neural
Netw. (IJCNN), Anchorage, AK, USA, May 2017, pp. 3016–3021.
[80] M. M. Bejani and M. Ghatee, “A context aware system for driving
style evaluation by an ensemble learning on smartphone sensors data,”
Transp. Res. C, Emerg. Technol., vol. 89, pp. 303–320, Apr. 2018.
[81] Q. Xue, K. Wang, J. J. Lu, and Y. Liu, “Rapid driving style recognition
in car-following using machine learning and vehicle trajectory data,”
J. Adv. Transp., vol. 2019, Jan. 2019, Art. no. 9085238.
[82] J. Xie, A. R. Hilal, and D. Kulic, “Driver distraction recognition based
on smartphone sensor data,” in Proc. IEEE Int. Conf. Syst., Man,
Cybern., Miyazaki, Japan, Oct. 2018, pp. 801–806.
[83] J. Guo, Y. Liu, L. Zhang, and Y. Wang, “Driving behaviour style
study with a hybrid deep learning framework based on GPS data,”
Sustainability, vol. 10, no. 7, p. 2351, Jul. 2018.
[84] S. Daptardar, V. Lakshminarayanan, S. Reddy, S. Nair, S. Sahoo, and
P. Sinha, “Hidden Markov model based driving event detection and
driver profiling from mobile inertial sensor data,” in Proc. SENSORS,
Busan, South Korea, Nov. 2015, pp. 1–4.
[85] A. Bhatt, V. Dave, Y. Panchamia, and P. Thakre, “Analyzing behavioral
attributes of drivers and implementing safe driving model,” in Proc.
IEEE Int. Conf. Veh. Electron. Saf. (ICVES), Vienna, Austria, Jun. 2017,
pp. 228–232.
[86] Z. Yang and J. H. L. Hansen, “Unsupervised driving performance
assessment using free-positioned smartphones in vehicles,” in Proc.
19th Int. IEEE Conf. Intell. Transp. Syst., Rio de Janeiro, Brazil,
Nov. 2016, pp. 1598–1603.
[87] J. Paefgen, F. Kehr, Y. Zhai, and F. Michahelles, “Driving behavior
analysis with smartphones: Insights from a controlled field study,” in
Proc. 11th Int. Conf. Mobile Ubiquitous Multimedia, London, U.K.,
Dec. 2014, Art. no. 36.
[88] J. Wahlström, I. Skog, P. Händel, B. Bradley, S. Madden, and
H. Balakrishnan, “Smartphone placement within vehicles,” IEEE Trans.
Intell. Transp. Syst., to be published.
[89] L. Jiang, X. Chen, and W. He, “Safecam: Analyzing intersection-
related driver behaviors using multi-sensor smartphones,” in Proc.
IEEE Int. Conf. Pervasive Comput. Commun., Sydney, NSW, Australia,
Mar. 2016, pp. 1–9.
4474 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 21, NO. 10, OCTOBER 2020
[90] Q. Wang, Y. Liu, J. Liu, Y. Gu, and S. Kamijo, “Critical areas
detection and vehicle speed estimation system towards intersection-
related driving behavior analysis,” in Proc. IEEE Int. Conf. Consum.
Electron. (ICCE), Las Vegas, NV, USA, Jan. 2018, pp. 1–6.
[91] K. Saleh, M. Hossny, and S. Nahavandi, “Driving behavior classi-
fication based on sensor data fusion using LSTM recurrent neural
networks,” in Proc. IEEE 20th Int. Conf. Intell. Transp. Syst. (ITSC),
Yokohama, Japan, Oct. 2017, pp. 1–6.
[92] K. Greff, R. K. Srivastava, J. Koutnìk, B. R. Steunebrink, and
J. Schmidhuber, “LSTM: A search space odyssey,” IEEE Trans. Neural
Netw. Learn. Syst., vol. 28, no. 10, pp. 2222–2232, Oct. 2017.
[93] X. Xu, J. Yu, Y. Chen, Y. Zhu, S. Qian, and M. Li, “Leveraging
audio signals for early recognition of inattentive driving with smart-
phones,” IEEE Trans. Mobile Comput., vol. 17, no. 7, pp. 1553–1567,
Jul. 2018.
[94] M. J. Flores, J. M. Armingol, and A. De La Escalera, “Driver drowsi-
ness detection system under infrared illumination for an intelligent
vehicle,” IET Intell. Transp. Syst., vol. 5, no. 4, pp. 241–251, Dec. 2011.
[95] L. M. Bergasa, J. Nuevo, M. A. Sotelo, and M. Vazquez, “Real-time
system for monitoring driver vigilance,” in Proc. IEEE Intell. Veh.
Symp., Parma, Italy, Jun. 2004, pp. 78–83.
[96] X. Liu, F. Xu, and K. Fujimura, “Real-time eye detection and tracking
for driver observation under various light conditions,” in Proc. Intell.
Veh. Symp., Versailles, France, Jun. 2002, pp. 344–351.
[97] F. Zhang, J. Su, L. Geng, and Z. Xiao, “Driver fatigue detection
based on eye state recognition,” in Proc. Int. Conf. Mach. Vis. Inform.
Technol. (CMVIT), Singapore, Feb. 2017, pp. 105–110.
[98] S. Romeo et al., “Fluorescent light induces neurodegeneration in the
rodent nigrostriatal system but near infrared LED light does not,” Brain
Res., vol. 1662, pp. 87–101, Mar. 201.
[99] Spectral Sensitivity of the Human Eye. Accessed: Sep. 13, 2019.
[Online]. Available: https://light-measurement.com/spectral-sensitivity-
of-eye/
[100] N. A. Stanton, A. Hedge, K. Brookhuis, E. Salas, and H. W. Hendrick,
Handbook of Human Factors and Ergonomics Methods. Bristol, PA,
USA: Taylor & Francis, 2004, pp. 257–258.
[101] B. Mandal, L. Li, G. S. Wang, and J. Lin, “Towards detection of bus
driver fatigue based on robust visual analysis of eye state,” IEEE Trans.
Intell. Transp. Syst., vol. 18, no. 3, pp. 545–557, Mar. 2017.
[102] B. Shi et al., “Evaluating driving styles by normalizing driving behavior
based on personalized driver modeling,” IEEE Trans. Syst., Man,
Cybern., Syst., vol. 45, no. 12, pp. 1502–1508, Dec. 2015.
[103] G. Castignani, R. Frank, and T. Engel, “An evaluation study of driver
profiling fuzzy algorithms using smartphones,” in Proc. 21st IEEE
Int. Conf. Netw. Protocols (ICNP), Goettingen, Germany, Oct. 2013,
pp. 1–6.
[104] S. Amado, E. Arıkan, G. Kaça, M. Koyuncu, and B. N. Turkan, “How
accurately do drivers evaluate their own driving behavior? An on-road
observational study,” Accident Anal. Prevention, vol. 63, pp. 65–73,
Feb. 2014.
[105] S. Kanarachos, S.-R. G. Christopoulos, and A. Chroneos, “Smartphones
as an integrated platform for monitoring driver behaviour: The role
of sensor fusion and connectivity,” Transp. Res. C, Emerg. Technol.,
vol. 95, pp. 867–882, Oct. 2018.
[106] C. Bo et al., “Detecting driver’s smartphone usage via nonintrusively
sensing driving dynamics,” IEEE Internet Things J., vol. 4, no. 2,
pp. 340–350, Apr. 2017.
[107] Y. Wang et al., “Determining driver phone use by exploiting smart-
phone integrated sensors,” IEEE Trans. Mobile Comput., vol. 15, no. 8,
pp. 1965–1981, Aug. 2016.
[108] J. Almazán, L. M. Bergasa, J. J. Yebes, R. Barea, and R. Arroyo,
“Full auto-calibration of a smartphone on board a vehicle using
IMU and GPS embedded sensors,” in Proc. IEEE Intell. Veh. Symp.,
Gold Coast, QLD, Australia, Jun. 2013, pp. 1374–1380.
[109] P. Nguyen, H. Nguyen, D. Nguyen, T. N. Dinh, H. M. La, and T. Vu,
“ParkSense: Automatic parking positioning by leveraging in-vehicle
magnetic field variation,” IEEE Access, vol. 5, pp. 25021–25033,
Nov. 2017.
[110] F. J. Bruwer and M. J. Booysen, “Comparison of GPS and MEMS sup-
port for smartphone-based driver behavior monitoring,” in Proc. IEEE
Symp. Series Comput. Intell., Cape Town, South Africa, Dec. 2015,
pp. 434–441.
[111] S.-R. G. Christopoulos, S. Kanarachos, and A. Chroneos, “Learning
driver braking behavior using smartphones, neural networks and the
sliding correlation coefficient: Road anomaly case study,” IEEE Intell.
Transp. Syst., vol. 20, no. 1, pp. 65–74, Jan. 2019.
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
[112] T. A. Butt, R. Iqbal, K. Salah, M. Aloqaily, and Y. Jararweh, “Privacy
management in social Internet of vehicles: Review, challenges and
blockchain based solutions,” IEEE Access, vol. 7, pp. 79694–79713,
Jun. 2019.
[113] A. Rohunen, J. Markkula, M. Heikkila, and J. Heikkila, “Open traffic
data for future service innovation: Addressing the privacy challenges
of driving data,” J. Theor. Appl. Electron. Commerce Res., vol. 9, no. 3,
pp. 71–89, Sep. 2014.
[114] L. Kang and S. Banerjee, “Practical driving analytics with smartphone
sensors,” in Proc. IEEE Veh. Network. Conf. (VNC), Torino, Italy,
Nov. 2017, pp. 303–310.
[115] R. Lotfi and M. Ghatee, “Smartphone based driving style clas-
sification using features made by discrete wavelet transform,”
Mar. 2018, arXiv:1803.06213. [Online]. Available: https://arxiv.
org/abs/1803.06213
[116] J. Chen, Z. C. Wu, and J. Zhang, “Driver identification based on
hidden feature extraction by using adaptive nonnegativity-constrained
autoencoder,” Appl. Soft Comput., vol. 74, pp. 1–9, Jan. 2019.
[117] H. Liu, T. Taniguchi, Y. Tanaka, K. Takenaka, and T. Bando,
“Visualization of driving behavior based on hidden feature extraction
by using deep learning,” IEEE Trans. Intell. Transp. Syst., vol. 18,
no. 9, pp. 2477–2489, Sep. 2017.
[118] C. Zhang, H. Wang, and R. Fu, “Automated detection of
driver fatigue based on entropy and complexity measures,” IEEE
Trans. Intell. Transp. Syst., vol. 15, no. 1, pp. 168–177,
Feb. 2014.
[119] M. Guerrieri, R. Mauro, G. Parla, and T. Tollazzi, “Analysis of
kinematic parameters and driver behavior at turbo roundabouts,”
J. Transp. Eng., A, Syst., vol. 144, no. 6, pp. 1–12, Mar. 2018.
[120] A. S. El-Wakeel, A. Noureldin, H. S. Hassanein, and N. Zorba,
“Utilization of wavelet packet sensor de-noising for accurate posi-
tioning in intelligent road services,” in Proc. 14th Int. Wireless Com-
mun. Mobile Comput. Conf. (IWCMC), Limassol, Cyprus, Jun. 2018,
pp. 1231–1236.
[121] Y. Zhang, J. Li, Y. Guo, C. Xu, J. Bao, and Y. Song, “Vehicle driving
behavior recognition based on multi-view convolutional neural network
with joint data augmentation,” IEEE Trans. Veh. Technol., vol. 68, no. 5,
pp. 4223–4234, May 2019.
[122] M. Baldauf, S. Dustdar, and F. Rosenberg, “A survey on context-
aware systems,” Int. J. Ad Hoc Ubiquitous Comput., vol. 2, no. 4,
pp. 263–277, 2007.
[123] P. Angkititrakul, C. Miyajima, and K. Takeda, “Impact of driving
context on stochastic driver-behavior model: Quantitative analysis of
car following task,” in Proc. IEEE Int. Conf. Veh. Electron. Saf.,
Istanbul, Turkey, Jul. 2012, pp. 163–168.
[124] S. Al-Sultan, A. H. Al-Bayatti, and H. Zedan, “Context-aware
driver behavior detection system in intelligent transportation sys-
tems,” IEEE Trans. Veh. Technol., vol. 62, no. 9, pp. 4264–4275,
Nov. 2013.
[125] D. Böhmländer, T. Dirndorfer, A. H. Al-Bayatti, and T. Brandmeier,
“Context-aware system for pre-triggering irreversible vehicle safety
actuators,” Accidents Anal. Prevention, vol. 103, pp. 72–84, Jun. 2017.
[126] A. Rakotonirainy and F. D. Maire, “Context-aware driving behavioural
model,” in Proc. Int. Tech. Conf. Enhanced Saf. Vehicles (ESV),
Washington DC, USA, Jun. 2005, pp. 1–6.
[127] C. Ma, Q. Zhu, S. Wu, and B. Liu, “Representation learning from
time labelled heterogeneous data for mobile crowdsensing,” Mobile
Inf. Syst., vol. 2016, Aug. 2016, Art. no. 2097243.
[128] W. Zamora, C. T. Calafate, J.-C. Cano, and P. Manzoni, “A survey on
smartphone-based crowdsensing solutions,” Mobile Inf. Syst., vol. 2016,
Oct. 2016, Art. no. 9681842.
[129] G. Singh, D. Bansal, S. Sofat, and N. Aggarwal, “Smart patrolling:
An efficient road surface monitoring using smartphone sensors and
crowdsourcing,” Pervasive Mobile Comput., vol. 40, pp. 71–88,
Sep. 2017.
[130] G. Merlino, S. Arkoulis, S. Distefano, C. Papagianni, A. Puliafito, and
S. Papavassiliou, “Mobile crowdsensing as a service: A platform for
applications on top of sensing clouds,” Future Gener. Comput. Syst.,
vol. 56, pp. 623–639, Mar. 2016.
[131] X. Li and D. W. Goldberg, “Toward a mobile crowdsensing system
for road surface assessment,” Comput., Environ. Urban Syst., vol. 69,
pp. 51–62, May 2018.
[132] A. S. El-Wakeel, A. Noureldin, H. S. Hassanein, and N. Zorba,
“iDriveSense: Dynamic route planning involving roads quality infor-
mation,” in Proc. IEEE Global Commun. Conf., Abu Dhabi,
United Arab Emirates, Dec. 2018, pp. 1–6.
CHAN et al.: COMPREHENSIVE REVIEW OF DRIVER BEHAVIOR ANALYSIS UTILIZING SMARTPHONES
[133] F. Montori, P. P. Jayaraman, A. Yavari, A. Hassani, and
D. Georgakopoulos, “The curse of sensing: Survey of techniques
and challenges to cope with sparse and dense data in mobile crowd
sensing for Internet of Things,” Pervasive Mobile Comput., vol. 49,
pp. 111–125, Sep. 2018.
[134] H. M. Fahmy, M. A. A. El Ghany, and G. Baumann, “Vehicle risk
assessment and control for lane-keeping and collision avoidance at low-
speed and high-speed scenarios,” IEEE Trans. Veh. Technol., vol. 67,
no. 6, pp. 4806–4818, Jun. 2018.
[135] C. M. Kang, S.-H. Lee, and C. C. Chung, “Multirate lane-keeping
system with kinematic vehicle model,” IEEE Trans. Veh. Technol.,
vol. 67, no. 10, pp. 9211–9222, Oct. 2018.
[136] S. Malan, M. Milanese, P. Borodani, and A. Gallione, “Lateral control
of autonomous electric cars for relocation of public urban mobility
fleet,” IEEE Trans. Control Syst. Technol., vol. 15, no. 3, pp. 590–598,
May 2007.
[137] M. Martinez-García, Y. Zhang, and T. Gordon, “Modeling lane keeping
by a hybrid open-closed-loop pulse control scheme,” IEEE Trans. Ind.
Informat., vol. 12, no. 6, pp. 2256–2265, Dec. 2016.
[138] W. Kim, Y. S. Son, and C. C. Chung, “Torque-overlay-based robust
steering wheel angle control of electrical power steering for a lane-
keeping system of automated vehicles,” IEEE Trans. Veh. Technol.,
vol. 65, no. 6, pp. 4379–4392, Jun. 2016.
[139] S.-J. Wu, H.-H. Chiang, J.-W. Perng, C.-J. Chen, B.-F. Wu, and
T.-T. Lee, “The heterogeneous systems integration design and imple-
mentation for lane keeping on a vehicle,” IEEE Trans. Intell. Transp.
Syst., vol. 9, no. 2, pp. 246–263, Jun. 2008.
[140] J. Ji, A. Khajepour, W. W. Melek, and Y. Huang, “Path planning and
tracking for vehicle collision avoidance based on model predictive
control with multiconstraints,” IEEE Trans. Ultrason. Eng., vol. 66,
no. 2, pp. 952–964, Feb. 2017.
[141] P. Petrov and F. Nashashibi, “Modeling and nonlinear adaptive control
for autonomous vehicle overtaking,” IEEE Trans. Intell. Transp. Syst.,
vol. 15, no. 4, pp. 1643–1656, Aug. 2014.
[142] D. I. Katzourakis, J. C. F. de Winter, M. Alirezaei, M. Corno, and
R. Happee, “Road-departure prevention in an emergency obstacle
avoidance situation,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 44,
no. 5, pp. 621–629, May 2014.
[143] Apple Machine Learning Journal. Accessed: Sep. 13, 2019. [Online].
Available: https://machinelearning.apple.com/
[144] A. Ignatov et al., “AI benchmark: Running deep neural networks
on Android smartphones,” Oct. 2018., arXiv:1810.01109. [Online].
Available: https://arxiv.org/abs/1810.01109
[145] Deploy Machine Learning Models on Mobile and IoT Devices.
Accessed: Sep. 13, 2019. [Online]. Available: https://www.tensorflow.
org/lite
[146] Core Machine Learning (ML). Accessed: Sep. 13, 2019. [Online].
Available: https://developer.apple.com/documentation/coreml
[147] K. Chine, “Learning math and statistics on the cloud, towards an EC2-
based Google docs-like portal for teaching/learning collaboratively with
R and scilab,” in Proc. 10th IEEE Int. Conf. Adv. Learn. Technol.,
Sousse, Tunisia, Jul. 2010, pp. 752–753.
[148] E. Hormozi, H. Hormozi, M. K. Akbari, and M. S. Javan, “Using of
machine learning into cloud environment (a survey): Managing and
scheduling of resources in cloud systems,” in Proc. 7th Int. Conf.
P2P, Parallel, Grid, Cloud Internet Comput., Victoria, BC, Canada,
Nov. 2012, pp. 363–368.
Teck Kai Chan received the B.Eng. and M.Phil.
degrees from Newcastle University upon Tyne
in 2015 and 2018, respectively. He is currently
pursuing the Ph.D. degree under the Singapore
Industrial Postgraduate Programme with Visenti Pte.
Ltd., and Newcastle University upon Tyne. He was
a Data Engineer with Visenti Pte Ltd., where he
was involved in the investigation of advanced infor-
mation fusion methods to coordinate the deployed
large-scale heterogeneous sensor network and the
development of models to achieve a reliable and
robust leak detection and localization performance. His current research
interests include water leakage detection and localization algorithms, signal
processing, machine learning, and handling of imbalanced dataset.
Authorized licensed use limited to: University of Pretoria. Downloaded on July 13,2023 at 19:44:20 UTC from IEEE Xplore. Restrictions apply.
4475
Cheng Siong Chin (M’01–SM’09) received the
B.Eng. degree in mechanical and production
engineering from Nanyang Technological Univer-
sity (NTU) in 2000, the M.Sc. degree in advanced
control and systems engineering from The Univer-
sity of Manchester in 2001, and the Ph.D. degree
in applied control engineering from the Research
Robotics Centre, NTU, in 2008. He is currently an
Associate Professor with Newcastle University in
Singapore, Singapore. His research interests include
the design and simulation of complex systems for
land and underwater vehicles in uncertain environment. He was the general
chair and the technical review committee member for various international
conferences. He served as a Lead Guest Editor for the Journal of Advanced
Transportation (Special Issue on Intelligent Autonomous Transport Systems
Design and Simulation).
Hao Chen (M’13) received the B.S. degree in chem-
ical engineering from Sichuan University, China,
in 1998, and the Ph.D. degree in computer science
from the Huazhong University of Science and Tech-
nology, China, in 2005. He is currently a Professor
with the College of Computer Science and Elec-
tronic Engineering, Hunan University, China. He has
published more than 60 articles in journals and
conferences, such as the IEEE T RANSACTIONS ON
PARALLEL AND D ISTRIBUTED S YSTEMS , the IEEE
T RANSACTIONS ON C OMPUTERS , IPDPS, IWQoS,
HiPC, and ICPP. His current research interests include parallel and distributed
computing, operating systems, cloud computing, and systems security. He is
a member of the ACM.
Xionghu Zhong (M’11) received the B.Eng. and
M.Sc. degrees from Northwestern Polytechnical
University, China, in 2003 and 2006, respectively,
and the Ph.D. degree from the Institute for Digi-
tal Communications, The University of Edinburgh,
in 2010. He was a Research Fellow with the School
of Computer Engineering and a Senior Research
Fellow with the School of Electrical and Elec-
tronic Engineering, Nanyang Technological Univer-
sity, Singapore. He was with Xylem Inc., as a Data
Scientist from 2017 to 2018. He is currently a
Professor with the College of Computer Science and Electronic Engineering,
Hunan University, China. His research interests include statistical signal
processing, target localization and tracking, nonparametric Bayesian modeling
and machine learning methods, and their applications to distant speech
enhancement and recognition, V2X communications, and water distribution
network monitoring.