Prediction of Human Behaviour Using Artificial Neural

Networks

Zhicheng Zhang1, Frédéric Vanderhaegen2, and Patrick Millot2

1
Division of I&C and Electrical Systems, Framatome ANP,
Tour Areva, 92084 Paris La Defense Cedex, France
zhicheng.zhang@framatome-anp.com
2 CNRS UMR 8530, Laboratoire d'Automatique,

de Mécanique et d’Informatique industrielles et Humaines, University of Valenciennes,

Le Mont Houy, 59313 Valenciennes Cedex 9, France
{vanderhaegen, millot}@univ-valenciennes.fr

Abstract. This paper contributes to the analysis and prediction of deviate inten-
tional behaviour of human operators in Human-Machine Systems using Artifi-
cial Neural Networks that take uncertainty into account. Such deviate
intentional behaviour is a particular violation, called Barrier Removal. The ob-
jective of the paper is to propose a predictive Benefit-Cost-Deficit model that
allows a multi-reference, multi-factor and multi-criterion evaluation. Human
operator evaluations can be uncertain. The uncertainty of their subjective judge-
ments is therefore integrated into the prediction of the Barrier Removal. The
proposed approach is validated on a railway application, and the prediction
convergence of the uncertainty-integrating model is demonstrated.

1 Introduction
The study of human factors plays an increasingly important role in the design of new
complex Human-Machine Systems (HMS) or in the updating of older systems. Be-
haviours that deviate from a given prescription are interpreted as errors when the
behaviors or their consequences are not intentional, and as violations when they are
intentional [1]. Though human error has received close attention for about a century,
the study of intentional violations is still in its early stages [2]. The awareness of the
importance of taking safety violations into account in the risk analysis process in-
creased after the Chernobyl accident. According to J. Reason, five of the seven human
actions that led directly to the accident were deliberate deviations from written rules
and instructions, rather than slips, lapses or mistakes. As a matter of fact, though
violations have been mentioned in a number of contexts, research on violations is still
insignificant compared to research about slips, lapses, and rule & knowledge-based
mistakes.
This paper takes a connectionist approach that allows uncertainty to be taken into
account when analyzing and predicting the deviate intentional behavior (violations) of
the human operators in an HMS. The following section of the paper presents our pre-
dictive Benefit-Cost-Deficit (BCD) model. The third section proposes an uncertainty-
based prediction approach to Barrier Removal (BR) using Artificial Neural Networks.

D.S. Yeung et al. (Eds.): ICMLC 2005, LNAI 3930, pp. 770 – 779, 2006.
© Springer-Verlag Berlin Heidelberg 2006
 Prediction of Human Behaviour Using Artificial Neural Networks 771

In the fourth section, the validation of this approach on a railway simulator experi-
ment is described and the convergence of the uncertainty-based predictions is dis-
cussed. The final section presents our conclusions and offers perspectives for future
research.

2 A Three-Dimentional BCD Model

In an HMS, human operator actions are initially delimited by such boundaries as cost
acceptability, available resources and acceptable safety practices (Figure 1).

Functional limit of
safety acceptability

Prescribed limit of
Limit of
safety acceptability
finance acceptability

Migration Degree of liberty

margin for human action

Limit of available resources

Fig. 1. Human behavioural boundaries

How much these boundaries can migrate depends on the maximum functional lim-
its that users of a given system can accept in all safety. Human operators continually
manage a compromise between three joint and sometimes contradictory objectives:
performance objectives imposed by the organization or by the operators themselves,
safety objectives for the system and for the operator, and objectives related to the
cognitive and physiological costs of operator activities (e.g., workload, stress). These
objectives limit and bound the field of human actions. An action that crosses the set
boundaries can lead to loss of control, an incident, or even an accident. In this paper,
any action that crosses a boundary is called a Barrier Removal (BR). BR are specific
violations, made without any intention of subjectively damaging the HMS, and may
indicate that system designers and system users accept divergent degrees of risk. The
analysis of system user’s activities is three-dimensional: the reference-based, the BCD
factor-based and the criteria-based dimensions to identify and analyze the benefits,
costs and potential deficit generated by BR within a BCD framework.
772 Z. Zhang, F. Vanderhaegen, and P. Millot

2.1 Multi-reference BR Evaluation

There are often differences between the task prescribed by the designer and the actual
activity in its operational context, due to a variety of individual, technical and/or envi-
ronmental factors. For HMS designers, risk analysis is usually limited to assessing
safety risks, which is a mono-criterion process. Once the machine is operating on-site,
the validation process stops evolving and remains quite stable since the process was
the result of a common decision. However, this type of designer-based risk evaluation
is done independently of the users and is limited to the technical failures. Users, on
the other hand, control the risks associated with operational situations by evaluating
them after they are detected and by intervening in the piloted process to avoid their
occurrence or to limit their consequences. Thus, each operational BR is motivated by
several factors.

2.2 Multi-factor BR Evaluation

When deciding whether or not to remove a barrier, both the positive and negative
consequences should be taken into account [3], specifically:
– The expected benefit: Barrier Removal is a goal-driven behavior seen to offer an
immediate benefit that outweighs the cost.
– The immediate cost of removal: In order to remove a barrier, the human operator
must sometimes modify the material structure and/or the operational mode, which
usually leads to an increased workload and can have negative consequences on
productivity or quality.
– The potential deficit: Because removing a barrier introduces a potentially danger-
ous situation, such actions creates a potential deficit, due to the related risk.

2.3 Multi-criteria BR Evaluation

Evaluating operational BR is more or less a multi-criteria risk control process that

takes into account not only system safety criteria, but also economic criteria (e.g.,
production and quality) or social criteria (e.g., motivation or workload). Since it de-
pends on the variability of the operational situations to be controlled, as well as on
inter- and intra-individual differences, this risk control process is dynamic and vari-
able. Moreover, it can be used to evaluate a variety of elements, such as technical
failures, human and organisational errors, and violations, to name only a few.
During BR analysis, all three factors mentioned above (Benefit, Cost, potential
Deficit) are evaluated for each barrier class in terms of several performance criteria,
making it complicated to identify the removal status of a barrier directly and/or to
easily group similar BRs together. Clearly, determining the complex nonlinear rela-
tionships that exist between the different criteria is not easy, nor is identifying the
similaries/proximities of all BRs. In fact, BR analysis is a two-phase process. First, all
BRs must be classified in terms of the various performance criteria, and if possible,
the contributive BR criteria for a given HMS must be identified by looking for and
memorizing the similarities/proximities of all BRs. Second, the likelihood of the
new/changed barriers being removed must be predicted, according to the identified
criteria and the memorized similarities/proximities.
 Prediction of Human Behaviour Using Artificial Neural Networks 773

Artificial Neural Networks have the potential to accomplish the above tasks. A se-
ries of approaches for predicting Barrier Removal using ANN have already been
developed to anticipate or predict the removal of a given barrier in a given system
using the retained criteria by either considering a network according to one perform-
ance criterion (mono-performance) or by considering a network according to several
performance criteria (multi-performance) [4]. Based on these connectionist models
and methods (i.e. mono-performance ANN and multi-performance ANN), the uncer-
tainty of subjective human operator evaluations can be analyzed and processed,
allowing it to be integrated in the overall prediction methodology.

3 A Connectionist BCD Model and Prediction with Uncertainty

As stated above, BR is a safety-related violation. Its impact can be analyzed in terms
of benefits, costs, and potential deficits. In order to allow designers to integrate BR
into the risk analysis during the design phase or during re-design work, we have al-
ready proposed three Self-Organizing Map (SOM) predictive algorithms [5] used to
model the activation or the removal of barriers by the human operators. Designers
conceive their systems according to the pertinent regulations, standards and technical
guidelines, paying particular attentions to safety concerns. They equip their systems
with barriers in order to reduce human errors, limit failure propagation and/or protect
human operators from technical failures.
However, particular operational contexts require the definition of a series of con-
nectionist models and BR methods using ANN that allow the different contexts to be
dealt with and still obtain optimal results. Like an artificial neural network, the Self-
Organizing Map was originally designed for multidimensional data reduction with
topology-preserving properties [6]. The proposed connectionist methods were thus
validated through experimental manipulation designed to analyse and/or predict the
removal of a given barrier in a given system by integrating both the subjectivity and
the uncertainty of operator BCD evaluations and by considering both mono-
performance and multi-performance approaches.
Each evaluation of a BR factor incorporates a certain degree of uncertainty. Factor
evaluations with different uncertainty levels may have different numbers of subsets,
and thus weights should be allocated to each subset element, respectively. Different
weight allocations can be defined, for example:
– The lower the uncertainty level, the more representative the given value, making
the associated weight high.
– The lower the uncertainty level, the less numerous the values in the corresponding
subset.
– The closer a value is to the one evaluated by the human operator, the less its
weight differs from the evaluated one.
– The sum of the attributed weights is equal to 1.
Once the weight allocations have been defined, the subsets of all the factor evalua-
tions can be combined. Table 1 illustrates the final format of BR data for which un-
certainty has been factored in.
774 Z. Zhang, F. Vanderhaegen, and P. Millot

Table 1. The final format of BR data with uncertainty

Barrier Removal Factors

Benefit Cost Potential Deficit
Evaluation Uncertainty Evaluation Uncertainty Evaluation Uncertainty
Criterion 1 Variable 1 Variable 2 Variable 3 Variable 4 Variable 5 Variable 6
Criterion 2 Variable 7 Variable 8 Variable 9 Variable 10 Variable 11 Variable 12
Performance Criterion 3 Variable 13 Variable 14 Variable 15 Variable 16 Variable 17 Variable 18
criteria
Criterion 4 Variable 19 Variable 20 Variable 21 Variable 22 Variable 23 Variable 24

…… …… …… …… …… ……

As mentioned above, the human operators' evaluation of the BR factors incorpo-

rates a degree of uncertainty, and thus the uncertainty data must be pre-processed in
order to create cases or scenarios for which all the BCD values and their associated
uncertainties appear linearly for each criterion. For each case or scenario, an addi-
tional data element is required: the boolean value of the BR, which indicates whether
or not the corresponding barrier was removed. Figure 2 shows the basic uncertainty-
based BR prediction process.
Each predictive network has two phases: a learning phase and a prediction phase.
During the first phase, the learning needed for BR classification requires all the data
for a given case, and is accomplished via USOM, SSOM, and/or HSOM:
– In Unsupervised Self-Organizing Map (USOM) learning, the input data are the
subjective evaluations of benefit, cost and potential deficit in terms of the different
performance criteria;
– In Supervised Self-Organizing Map (SSOM) learning, the input data are the same
as those in Unsupervised SOM, but include a removal label for the corresponding
barrier;
– In Hierarchical Self-Organizing Map (HSOM) learning, the input data are the
same as those in Supervised SOM, except that network is formed by classifying
the data into parallel subsets, according to the personalities of the human opera-
tors. For example, experimental BR data can be grouped into several subsets re-
lated to the controllers' cultural background (e.g., ethnic characteristics).

During the second phase, barrier removal predictions are made, based on the iden-
tified criteria and the similarity/proximity data that was memorized during the learn-
ing process. For a known target value, the SSOM algorithms are used to classify the
data & to predict barrier removal: the input data are all the variables for a given case,
except the BR boolean value that has to be predicted. In addition, because different
people have different characteristics, the HSOM algorithms are used to group BR data
into subsets based on personality. Finally, for an unknown target value, the USOM
algorithms are used in order to classify the input data and identify two groups (i.e. the
group of removed barriers and the group of unremoved barriers), as is the case in data
mining, for example.
 Prediction of Human Behaviour Using Artificial Neural Networks 775

Variable n
Variable 1

Variable 2
Variable 3

Variable 4
…… …
Case 1
Case 2
Case 3
…

1
Classification Prediction for a
by learning given barrier
…

Np Input vectors Np+1

Weight vector for the SOM
Mapping Neighbours of a given cell

c
A
Output E
assessment B D

Fig. 2. The basic uncertainty-based BR prediction process

4 Case Study Using a Railway Simulator

This case study was performed on an experimental platform called TRANSPAL.

4.1 Platform Configuration

The TRANSPAL platform simulates train movements from depot to depot, via sev-
eral transformation stations at which human operators load/unload the products lo-
cated on a train stopped at the station platform. [7]. A human operator controls the
train traffic flow.
Several risks have been identified for this controlled process:
– Train derailment: if the corresponding switching device is not operated correctly
when a train is authorised to move, the train may derail.
– Shunting error: a train may be directed toward the wrong route.
– Train collision: trains may crash head-on or during an overtaking maneuver.
– Operator injury: an injury may occur because the human operators are not aware
that a train is entering or leaving a transformation area.
– Planning delay: the products on the trains may not be loaded/unloaded on time or
may be only partially loaded/unloaded.

In order to limit the risks due to control errors, several barriers exist in order to or-
ganize the traffic flow and the train routes, to prevent collisions or derailments, and to
keep operators in the transformation areas informed.
776 Z. Zhang, F. Vanderhaegen, and P. Millot

The proposed experiment was run in 3 phases:

– Phase 1: A 5-min familiarization period to allow participants to understand the
TRANSPAL process and interface;
– Phase 2: A 15-min experiment with all barriers in place (e.g., the signals at the
depots, the switching device and the transformation areas) including a presenta-
tion of the planning, the product handling on the platform and a performance
evaluation;
– Phase 3: A 10-min period during which participants may remove some barriers,
but must complete questionnaires assessing the importance of the BCD factors and
the uncertainty of their own evaluation for each barrier removed.

4.2 Results

Twenty experts from the European railway project, Urban Guided Transport Man-
agement System (UGTMS), participated in the three experimental phases. In the third
phase, they were asked to complete a questionnaire evaluating the advantages of re-
moving barriers, in terms of benefits, costs, and potential deficits, and evaluating the
level of the subjective certainty of their evalutations of these BCD factors. They were
asked to take four performance criteria into account:
– The quality of the planning.
– The production, based on the percentage of the product loaded/unloaded at the
stations.
– The traffic safety, based on possible collisions, derailments and injuries due to
incorrectly synchronized announcements of train movement at the transformation
stations.
– The human workload, based on the occupational rate (i.e. number of actions on the
interface).
The results focus on the perceived impact of the BR and were grouped into five
barrier families:
– The signals for trains entering the depots.
– The signals for trains exiting the depots.
– The signals for trains approaching a shunting device.
– The signals for trains entering and leaving the transformation stations.
– The signals for trains stopping at transformation areas.
Prior to predicting human actions based on their perceptions about barrier removal,
a learning step was required in order to determine the data distribution. The tested
algorithm exploits the SSOM approach, meaning that the input vectors of the neural
network were the barrier removal factors (i.e. the benefit, cost and potential deficit
associated with barrier removal for each performance criterion, the decision to respect
or remove the corresponding barrier, as well as the associated uncertainty level).
An example comparing prediction with uncertainty and prediction without un-
certainty. After constructing the vectors for sample data without uncertainty and
reconstructing the vectors for sample data with uncertainty, predictions with uncer-
tainty and without uncertainty were compared. Table 2 summarizes the comparative
 Prediction of Human Behaviour Using Artificial Neural Networks 777

Table 2. Comparison of prediction with and without uncertainty

10: learning ; 10: prediction

Prediction rate
Prediction Observation Variations
Without uncer- 9 Not removed 12 Not removed 9 cases out
tainty 21 Removed 18 Removed of 30 70%

With uncer- 81 Not removed 69 Not removed 33 cases out

tainty 108 Removed 120 Removed of 189 83%

results for the depot entrance signals in the mono-performance mode, based on data
for the productivity criterion.
As this table shows, the sample data for the first 10 subjects were used for the
learning phase, and the sample data for the last 10 subjects were used to identify the
variations between prediction and observation. The result of comparing the predic-
tion with uncertainty and the prediction without uncertainty (column “Prediction rate”
in the table) shows that the prediction rate taking the uncertainty of the human opera-
tor's evaluation into account is higher than the one not taking uncertainty into account.
Input data with uncertainty increased the number of studied cases because extrapola-
tions were made regarding the uncertainty values on the BCD factors. This paper does
not develop this extrapolation.
Convergence of prediction integrating uncertainty. In order to verify the accuracy
of predictions integrating the uncertainty of barrier removal factor evaluations,

0.9

0.8
Correct prediction rate

0.7

0.6

0.5

0.4

0.3

0.2
1 2 3 4 5 6 7 8 9 10 11 12
Cas
Case
Fig. 3. Convergence of prediction with uncertainty data
778 Z. Zhang, F. Vanderhaegen, and P. Millot

prediction convergence was studied. Figure 3 provides an example showing the im-
pact of the quantity of learning phase input vectors on the prediction rate. In this fig-
ure, several cases are defined: case 1 corresponds to a learning phase that integrates
the input vectors of 5 human experts and the prediction phase concerns the 15 other
experts; case 12 considers the input vectors of 16 human experts, and makes predic-
tions for the last 4 human experts. The prediction rate is determined by comparing the
prediction produced by the SSOM algorithm and the real behaviour of the human
experts.
The results show that the number of input vectors used for the learning phase has
an impact on the convergence of the prediction rate. The accurate prediction rate
converges toward 95% when the uncertainty evaluation is taken into account.

4 Conclusions
This paper has presented a method using Artificial Neural Networks to analyze and
predict BR based on evaluations of the uncertainty and subjectivity of the data. Rep-
resenting BR results within a BCD framework as a constraint network can provide
designers/users with tools that will allow them to predict the likelihood that
new/changed barriers will be removed.
Human operator evaluations incorporate a degree of uncertainty. Analyzing and
processing the uncertainty and subjectivity of human operator evaluations allows
these two elements to be integrated into the prediction. Given enough learning cases
and enough sample data, competitive neural networks can be configured, and the
SOM maps obtained from a learning set can be used to predict barrier removal for any
given barrier.
The experiment with 20 specialists from a European transportation project shows
that the prediction rate based on BR data that integrate uncertainty is higher than the
rate based on data that doesn't integrate uncertainty. The result with the uncertainty-
based BR data converges toward 95% of accurate predictions.
It should be noted that the results presented here constitute only a preliminary
analysis of uncertainty in BR prediction. Our approach is able to deal with not only
subjective data but also objective data if available. In the mean time, it can be used as
a statistical data mining method to aide in the identification of the most contributive
BR criteria.

Acknowledgements
The research presented in this paper was supported by the European framework pro-
ject, Urban Guided Transport Management System. The authors would like to thank
the 20 experts from the project consortium for their contributions to this project.

References
1. Reason, J.: Human error, Cambridge University Press, Cambridge, 1990.
2. Reason, J.: A system approach to organizational error. Ergonomics, 38, 8, 1708–1721,
1995.
 Prediction of Human Behaviour Using Artificial Neural Networks 779

3. Vanderhaegen, F.: Analyse et contrôle de l’erreur humaine. Hermes science, Paris, France,
2003.
4. Zhang, Z., Polet, P., Vanderhaegen, F., and Millot, P.: Artificial Neural Network for Viola-
tion Analysis. Reliability Engineering and System Safety, 84, 1, 3–18, 2004.
5. Zhang, Z., and Vanderhaegen, F.: A method integrating Self-Organizing Maps to predict
the probability of Barrier Removal. Chapter 30 in H. Bozdogan (ed.), Statistical Data Min-
ing and Knowledge Discovery, CRC Press, New York, July 2003.
6. Kohonen, T.: Self-Organizing Maps. Springer-Verlag, Third edition, Berlin, Heidelberg,
Germany, 2001.
7. Zhang, Z. : Fiabilité humaine: prédiction des violations par réseaux de neurones et applica-
tion aux systèmes de transport. PhD Thesis, Universite de Valenciennes, France, March
2004.