Copyright @IFAC Analysis.

Design and Evaluation of

Human-Machine Systems. Kassel. Germany. 2001

DETECTION OF EVOLUTIONS IN THE DRIVER BEHAVIOUR USING

RELIABILITY CENTRED MAINTENANCE METHODS

Jc. Popieul (I), P. Simon (I), P. Loslever (I), A. Todoskoff(Z)

(I) Laboratoire d'Automatique et de Mecanique Industrielles et Humaines

Le Mont Houy, F-593/3 Valenciennes Cedex 9
Tel : 33 (0)32 7 51 1462, Fax : 33 (0)32751 1316
e-mail: Jean-Christophe.Popieu!r(i) univ-va!enciennes.fr

(2) Institut des Sciences & Techniques de l'lngenieur d'Angers (IST1A)

Qualite et Surete de Fonctionnement
62 avenue Notre-Dame du Lac, F-49000 Angers
rei : 33 (0)2 4! 365 7 57, Fax: 33 (0)241 3657 43

Abstract: This paper deals with the application of techniques coming from Reliability
Centred Maintenance to detect time evolutions of a driver's behaviour in a long
duration simulated driving task. It describes the problematic raised by such systems,
where a human operator is involved in the control loop, and proposes an approach of
solution using exploratory data analysis techniques. Copyright " 20011FAC

Keywords: vigilance, reliability, maintenance, exploratory data analysis, active

security, driver behaviour, classification, driving simulation.

state continuously modifies the standard

INTRODUCTION
The work presented in this paper deals with the
problem of failure detection in systems where a Human
operator has a permanent role in the control loop, such
as car driving for example. For most of these systems,
the complexity of performing an early detection and
diagnosis of faults has its origin in the difficulty of the
definition of a standard performance running area for
the system, allowing to study time evolutions inside
and outside of this area:
There is no available model (Human cognitive
activity) that can be used for the definition of this
area and of the variables that can characterise it,
The large variability of Human operator's
behaviour induces a wide area of standard
performance,
In the car driving example, the continuous
evolution of both the environment and the driver's
performance area.
As the lack of models does not allow an a priori
definition of this space, the proposed approach is based
on an exploratory technique. It aims at building a
working space, through an observation of the system
time evolutions, in which performance (regarding a
specific criterion such as security for example) can
continuously be assessed, and where trends can be
identified. To ensure sensitivity and reliability, this
space has to be defined by the best fitted set of
variables among those that can be measured
continuously on the system.
This approach takes its inspiration in industrial
maintenance strategies, and particularly in condition
directed maintenance (CDM) which consists in
monitoring equipment performance to recognise the
onset of failure, determine degradation rate and
forecast failure (Moubray, 1997).

253
 The final aim is to implement detection and diagnostic
devices that will be able to trigger specific tools giving
the best help to the operator, in a context of Human -
Machine cooperation.
2 PROBLEM CONTEXT
2.1 Driver's vigilance
Fatigue and driver's loss of vigilance have always been
recognized as an important cause of accidents. Many
research studies have been lead to try to assess the part
of total accident that could be a consequence of fatigue
or loss of vigilance. Estimates of vigilance or fatigue
related accidents vary in a wide range : 1.2-3.2%
(USA: Knippling and Wang, 1995) ; 2.5% (UK:
Maycock, 1995) ; 8.4% (France : Thomas and Attard,
1994) when vigilance or fatigue are considered as the
main factor and up to 54% if they are considered as an
important factor among others (France : Leger, 1994).
The aim of increasing public safety and reducing the
cost of this disease for society induced studies in
several fields . Two main areas are explored. The first
one aims at a better understanding of fatigue
mechanisms in relation with various factors : the length
of continuous work spells and daily duty periods, times
of rest, time of the day, travel length or duration, .. . in
order to propose recommendations to reduce the risk
(Brown, 1994 ; Sagberg, 1999). The second area aims
at proposing in-vehicle alerting devices to counteract
driver drowsiness. These devices can be divided in two
general categories (Verwey and Zaidel, 1999). The
first includes those designed for detection of
drowsiness followed by a warning and, in more
advanced concepts, followed by an intervention. These
devices are based on the continuous measurement of
indicators used for assessment of drowsiness (e.g.
EEG , behavioral activities (Roge et aI., 2001» or of
impaired driving performance (Verwey and Zaidel,
2000).
In the second category are the devices which are
designed for maintaining alertness by some form of
activation of the driver throughout the trip. This
approach is based on the hypothesis that an attention
demanding secondary task might increase the general
level of arousal and improve performance. This is the
solution that is in practice in train driving for many
years now. Operators have to react to the occurrence of
a signal that appears periodically throughout the trip by
pressing a switch or pulling a lever. However, studies
dealing with the comparison of different forms of such
systems have shown drivers could operate most of
them automatically at very low levels of alertness
(Fruhstorfer et aI., 1977) and that the only ones who
could really increase the alertness of train drivers were
those that require more cognitive processing. This
result has also been put forward in car driving by
Verwey and Zaidel (1999) who used game playing as a
254
secondary task. But these authors also conclude that
this complementary mental activity should be induced
at the appropriate times to assure maximal safety
benefits (it should not distract the driver in demanding
driving situations).
Due to this problem, most of the work in car driving
has been centered on devices from the first category.
All authors try to find out the ability of certain types of
"behaviors" for giving a reliable prediction of severe
drowsiness conditions. Behaviors have been
characterized by many different predictor variables
related to vehicle performance (control of speed,
steering and lateral position, TIC, TLC, .. . ), driver
physiological status (electroencephalogram, eye blinks
frequency and duration, slow eye movements, heart
rate, ... ), collateral activities (facial expression, body
and head movements, . .. ) etc. (see Todoskoff, 1999 for
a review).
Experiments made to assess the capacity of these
variables to be used as predictors deal generally with
one variable at a time (monovariable statistical test are
the most common used methods). If some of these
variables reveal to be usable in the tested experimental
conditions, their application to real driving conditions
rises many problems. The great variability of situations
in real driving conditions (changes in weather, lighting
contitions, trafic density, road surface, etc.) induces
much more changes on these variables than those that
are consequences of drowsiness. This can lead to a
lack of reliability of the prediction.
One way of solution is to work simultaneously on the
best suited set of variables that could be used as
predictors. As in an industrial diagnosis application,
working on several variables that evolve
simultaneously to indicate a specific situation gives a
more reliable result. This is the way that is being
explored using methods from the field of technical
diagnosis.
2.2 Reliability Centered Maintenance
RCM is a global maintenance approach that aims at
applying the most cost-effective and appropriate
maintenance technique to a device (called an "asset")
regarding the specific consequences of its functional
failure (system operation, safety, cost, ...) (Moubray,
1997). In our context, driver' s "failures" may have
disastrous consequences and are very difficult to
anticipate. In such a context, the best approach is to
assess continuously the state of a system, applying a
Condition Directed Maintenance (CDM) strategy,
leading to trigger maintenance actions only when
necessary (Smith, \993).
RCM and CDM classify failures in two sets :
functional or potential failures. Functional failures
occur when a device looses its ability to perform a
required function with respect to a performance level
 defined in its technical specifications. The device is
unavailable due to this failure (Zwingelstein, 1996).
Once a specific functional failure has been identified,
one or several physic conditions can generally be
defined in order to detect that this failure is about to
occur. The satisfaction of these physic conditions is
called a potential failure. Under these circumstances, it
is possible to react just before the functional failure
and thus to use the device at the maximum potential of
its components. The detection of the satisfaction of this
condition is mainly a classification problem working
with "indicators" (state variables) measured onto the
system.
When working with automated systems, the
"indicators" describing the physic condition can be
identified rather easily (vibratory, chemical, electrical,
... analyses). When a Human operator is involved,
these indicators are much more difficult to find.
As when working on complex systems, the main
problem is that the system can not be modelled easily
and thus :
variables that describe the system state are not a
priori known,
evolutions of these variables regarding the effects
of the studied factors on the system can not be
predicted,
eventual relationships among variables are not a
priori known .
As a consequence, it is very difficult to make
hypotheses on these variables in order to check them
with statistical test methods : very often, indicators are
multivariate and working only on a single variable
increases the risk of false alarm while working on
several that evolve simultaneously in a specific
situation gives a more reliable diagnosis.
Thus, the approach has to be exploratory and able to
put forward, among a large set of candidate variables,
those who are the most sensitive and discriminent to
the studied factors, the relationships among them and
their ability to be used in a working states classification
process (from normal system behaviour to fault or
failure).
To achieve these goals, the best suited method is the
Multiple Correspondence Analysis (Johnson and
Wichern, 1992 ; Pau, 1997). The main assets of this
exploratory data analysis method is that :
it fulfils the above goals,
it allows to use simultaneously quantitative and
qualitative variables,
it shows graphically linear or non linear
relationships among variables (through the use of
space modalities),
it shows graphically relationships between
"observations" as sets of points materialising
classes.
Thus, MCA allows a better knowledge of the data and
of their structure and, in this way, is a required first
step for an efficient use of classification methods.
255
This approach is illustrated on a system where a human
operator is highly involved: the car-driver system.
2.3 The Car-Driver system in Environment
The first step in this approach is to select candidate
variables among all those that can be measured onto
the system. The path that is proposed is to take into
account variables describing the interactions between
all the "components" of the system in its environment,
as if a behaviour model had to be built.
]n the field of car driving, as driving is a complex task
involving several regulation loops (control of the car,
tactic and strategic activities) (Donges, 1978), the set
of candidate variables has been restricted to those
describing the control activity which is the only one
that gives continuously observable information.
c~
Visual
.. perception . , :
PsychologlC.
phYSlOloglc and
behavlOfal aspects
~) - - --
Actions on Vehicle kinematics.
vehicle ...
commands ,
~
Fig. I. The Car-Driver system in Environment
Interactions between the "components" of car driving
can be classified into three main categories (figure I):
Interactions between the driver and the
environment. In this category are found mainly
visual perception of the driver but also
kinaesthetic feedback.
Interactions between the driver and the vehicle
through actions on the commands of the car
(steering wheel, pedals, ... ).
Interactions between the vehicle and the
environment where are found mainly cinematic or
dynamic information.
As the focus is on the problems raised by the Human
operator, a fourth category is to be considered though
it doesn't deal with interactions, it is what can be found
in the "psychological, physiological and behavioural
aspects" (e.g. driver's motivation, stress, self centred
gestures, body movements, .. . ) (Todoskoff, et ai,
1999).
Starting from a set of variables selected in these 4
categories comes the problem of putting forward the
most discriminant ones, but also the relationships
among them. As seen in the first part, Multiple
Correspondence Analysis (MCA) is a well suited
method to achieve this goal. It also allows to build
classes of "observations" that can correspond to
different system's working states, through its
multivariate and exploratory aspects (Escofier and
Pages, 1988 ; Greenacre, 1993).
2.4 First validation step
As a first validation step, the approach was tested on a
long monotonous driving context in the aim of
detecting evolutions of the driver's behaviour.
Experiments were carried out in a driving simulator,
first of all for safety considerations but also to get
complete control of the driving situations and to ease
the measurement of some data.
3 EXPERIMENTAnONS
3. J The SHERPA driving simulator
SHERPA - Simulateur Hybride d'Etude et de
Recherche de PSA pour l'Automobile - is a fixed base
driving simulator. The driving is performed in a Xantia
Citroen fully instrumented: dashboard, gearbox, pedals
(clutch, brake, accelerator) and steering wheel.
Almost all data measured on the car cabin or computed
in the dynamical model can be stored at 120 Hz in data
files during the simulation session.
3.2 Eye gaze automatic processing in SHERPA
Eye gaze of the driver is measured using an ASL 4000
(Applied Science Laboratories) eye tracker. It uses an
helmet for eye gaze measurement by the "corneal
vector" technique and an Ascension Flock of Birds
magnetic tracker for the measure of head movements.
Due to the number of subjects and the experimentation
duration (vigilance problems), a manual data extraction
is not realistic (far too much time consuming). Thus, an
original system named SA TMOS has been developed
to allow an automatic processing of eye gaze
measurement in the driving simulator.
The result of the data extraction is given as a list of
visual entities (fixations, smooth movements, saccades
and losses) with their duration and, for fixations &
smooth movements, the corresponding objects (inside
or outside of the car) and their respective distances
from the driver's eyes.
3.3 Experimental protocol
Two experiments were carried out. In the first one,
after a 90 km learning phase, 10 drivers had to drive a
160 km motorway travel (8 times the same lap made of
14 different stretches). During this test, variables were
recorded from three of the above categories :
Steering wheel angle for category 2 (interactions
between the driver and the vehicle),
Speed, lateral position and heading error of the
vehicle for category 3 (interactions between the
vehicle and the environment).
256
x, Y, Z positions of the head for category 4
("psychological, physiological and behavioural
aspects"),
Eye gaze measurement was left out due to the duration
of the experiment (1h20) (problems of eye tracker's
accuracy and driver tiredness for long periods of use).
In the second experiment, 10 subjects, after an 80 km
learning phase, had to drive a 336 km highway travel.
This travel was divided in four phases of 84 km,
themselves cut into three 24 km identical "laps" with a
starting and an ending stretch. Same variables as in the
first test were recorded except in the 1sI and 3rd phases
where the eye gaze of the driver was recorded as a
variable from the I sI category (interactions between the
driver and the environment).
4 RESULTS
4. J First experiment
To apply MCA upon a set of data, several preliminary
steps aiming at building the "analysis variables" have
to be achieved. The first one is the choice of the
statistical unit (time cutting). In order to be able to
make comparisons between subjects, the time cutting
building the statistical units is based on a spatial
criterion: the stretch. So each travel is a succession of
112 statistical units. For each statistical unit, data are
characterised by the arithmetic mean and standard
deviation.
The second step aims at obtaining homogeneous data.
It is based on a cutting of the ranges of evolutions of
each analysis variable into space modalities. Three
modalities are used, with an equal width cutting
criterion and fuzzy membership values in order to
reduce the information loss due to the transition from a
quantitative to a qualitative scale.
Given as an example, first factor plane hereunder
shows the trajectories of "mean subject" for the 112
stretches (Figure 2).
2 12"4
Fig. 2. First factor plane, modality points.
Projections of the 8 laps (triangles on figure 2) shows
clearly a time evolution (lap I to 8 going from right to
left on the first axis). This evolution is related to those
of the variables building the first axis which are the difference in driver behaviour evolution from one
standard deviations of head position, of lateral car subject to another puts forward the major problem
position and steering wheel angle but also mean values when dealing with man machine systems which is in
of the vehicle speed and longitudinal and vertical taking into account inter-individual differences when
positions of the head. developing new systems. This is especially true with
drowsiness among drivers (Verwey and Zaidel, 2000) .

4.2 Second experiment

The same approach was carried out on the 2nd, 3rd and
4th category variables with different characterisation
methods and showed almost the same results. A
specific analysis was performed on variables
describing the visual activity (l st category). The same
For each ooint . spatial cutting criterion is kept for time windows,
. stretch number (from 1 to 112)
." rr-------------~ • kind of stretch leading to a total number of windows of 72 (two
- fIrst lac (stretch 1 to 14) A (strai.ht line) phases with the eye tracker * 3 laps * 12 stretches).
<. > (left. ,",ht curve : R-3000m)
-last lao (strt1:dl ~N to.> 112)
Each visual entity is characterised by its duration and
«. » (Ieft. rioht curve: R-8OOm)
"-f .()~ O~ ,~
angular amplitude, which are then cut in three fuzzy
space modalities (same widths). For visual strategies,
Fig. 3. First farctor plane, statistical units. membership values describe the areas of interest
observed by the driver (among 10). Each time window
Figure 3 shows the projection of the 112 stretches being summarised by mean membership values.
building the 8 laps (8 times a 14 stretches lap). The
same evolution as on figure 2 can be observed. Mean axis 2

subject trajectory evolves from right to left of the first

axis between the I st and the II t h stretch. The second
axis shows differences induced by the geometry of the
road: bottom points correspond to curves while top
points correspond to the straight lines (second axis is
built by the steering wheel related characterisation
variables and lateral position).
Figure 4 hereunder is another analysis made on the 10
drivers considered individually.
1.5 .---,--------y--------,-- ------,--------,-- -- -
2

57 15 17 0.2 0.3
18
'~58
5 "154 16
52'\ 53 Fig. 5. First factor plane, time windows.
0.5 \ Driver 5 11 !
Driver 1
~ 13
51 ..
-.:r"l:2J8
1~
Cl
Given as an example, first factor plane here above
0
Drivers ,
... ... +
~~--
.., "
shows the 72 statistical units of one subject (figure 5).
The same phenomenon as previously can be observed:
3,4,6,8,9
:117 71 .25 Driver 10 the influence of time is revealed by the first factor axis.
·0.5
"G 2~4 First stretches are located on the left of the I st axis and
.~ DriVer 2
last ones on the right. When grouping the stretches of
the first and last lap as on figure 3, two independent
. 1 L-~.0~.5~----~0------~0~.5~-----7------~1~
. 5~~
sets can be distinguished, showing two distinct
behaviours of the driver.
Fig. 4. First factor plane, statistical units for the These behaviours can be characterised by the variables
analysis of the 10 drivers considered individually. building the 1st axis (eye blinks duration: increase with
time, amplitudes and duration of saccades: increase,
The same result can be observed, that is mainly a time fixations on landscape and sky: more frequent).
evolution here from left to right of the first axis, which
is still built by almost the same variables. What can be 5 CONCLUSION AND PROSPECTS
observed here is that for some drivers (I, 7, 10), this
evolution is far more important than for others. This

257
This paper presents a method based on principles
coming from RCM/CDM in the aim of characterising
the behaviour of systems where a Human operator is
included in the control loop. A first validation step of
this approach is presented through its use to study time
evolutions of the driver behaviour in a simulated
motorway monotonous driving context. The method
has shown its capacity to put forward time sensitive
variables and their joint evolutions allowing to
distinguish different system's running states.
A second step in the approach will be to use
classification methods such as discriminant analysis
(Hand, 1981), on the basis of the identified sensitive
variables, to automatically detennine the current
running state of the system.
6 ACKNOWLEDGEMENTS
The work presented in this paper has been realised in
the context of a partnership between PSA Peugeot
Citroen and the LAMIH, with the financial support of
the "Fondation MAIF" and the "Region Nord - Pas de
Calais".
7 REFERENCES
Brown, 1.0. (\994). Driver Fatigue, Human Factors,
36(2), pp 298-314.
Donges, E. (1978). A two level model of driver
steering behaviour, Human Factors, 20, 691-707.
Escofier, 8., Pages, J. (\ 988). Analyses factorie//es
simples et multiples. Dunod. Paris, France.
Fruhstorfer, H., Langanke, P., Meinzer, K., Peter, J.H. ,
Pfaff, U. (1977). Neurophysiological vigilance
indicators and operational analysis of a train
vigilance monitoring device: A laboratory and
field study. In Vigilance. Theory, operational
performance and physiological correlates,
Mackie, R.R. (Ed), pp 87-109. Plenum press, New
York.
Greenacre, MJ. (\993). Correspondence Analysis in
Practice. Academic Press. London.
Hand, DJ. (\ 981). Discrimination and classification,
Wiley. London.
Johnson, M., & Wichern, D.W. {I 992). Applied
Multivariate Statistical Analysis, Prentice Hall
International Editions.
Knipling, R.R. & WangJ .-S. (1995). Revised estimates
of the US drowsy driver crash problem size based
on general estimates systems case reviews.
Proceedings of the 39'h annual meeting of the
association for advancement of automotive
medicine, October.
258
Leger, D. (\994). The cost of sleep-related accidents: a
report for the National Commission on Sleep
Disorders Research. Sleep 17, pp 84-93.
Maycock, G. (\ 995). Driver sleepiness as a factor of
in car and HGV accidents. Technical report 169.
Crowthorne, UK Transport and Research
Laboratory,.
Moubray, J. (\ 997). Reliability-Centered Maintenance
lI. Butterworth-Heinemann. Oxford, UK.
Pau, L-F. (1997). Diagnostic des pannes dans les
systemes. CEPADUES Editions. Toulouse, France.
Roge, J., Pebayle T. & Muzet A. (2001). Variations of
the level of vigilance and of behavioural activities
during simulated automobile driving. Accident
Analysis and Prevention, 33, pp 181-186.
Sagberg, F. (\999). Road accidents caused by drivers
falling asleep. Accident Analysis and Prevention,
31, pp 639-649.
Smith, A.M . (\ 993). Reliability Centered
Maintenance. Mc Graw Hill. New York,.
Thomas, C. & Attard, S. (1994). Drowsiness and Fatal
accidents. Technical Report, Nanterre, France.
Centre Europeen d'Etude Socioeconomiques et
Accidentologiques des Risques (CEESAR).
Todoskoff, A. {I 999). Etude des evolutions
tempore//es du comportement du conducteur sur
autoroute : analyse multidimensionnelle de
signaux relatift au vehicule et aux mouvements de
tete sur simulateur. These de Doctorat de
l'Universite de Valenciennes, janvier.
Todoskoff, A., Dillies, M-A., Popieul, J-C. & Angue,
J-c. (1999). Characterization of driver'S behaviour
: a joint study in a car driving simulator and in real
driving situations. Proceedings of the Driving
Simulation Conference '99. Paris, France, July.
Verwey, W.B. & Zaidel, D.M. (1999). Preventing
drowsiness accidents by an alertness maintenance
device. Accident Analysis and Prevention, 31, pp
199-211, 1999.
Verwey, W.B. & Zaidel, D.M. (2000). Predicting
drowsiness accidents from personal attributes, eye
blinks and ongoing driving behaviour. Personality
and Individual Differences, 28, pp 123-142.
Zwingelstein, G. (1999). La maintenance basee sur la
fiabilite, guide pratique d'application de la RCM.
Hennes. Paris, France.