53 Workload assessment in

railway control
Sarah Nichols, Nikki Bristol and John R. Wilson
Institute for Occupational Ergonomics (IOE)
University ofNottingham, UK

Abstract

A new centralisation initiative in railway control is likely to have an impact on the

workload of railway signallers. Factors associated with this initiative that are
particularly likely ·to affect signaller workload include an increase in the
geographical area an individual controls and a change in the type of signalling
technology that is used. This paper presents a field study conducted in railway
signal boxes that aimed to identify the relationship between different types of load
placed upon a signaller, with different levels of technological support, and the
signaller's perceived workload. Initial results from this study are presented, along
with suggestions about how traditional mental workload measures can be used in a
field environment.

Introduction

As new technology is introduced into control room environments the type of tasks
required to be completed by the operator, and the amount of automated support
available is changing. This paper presents a study of the experiences of operators
in control rooms in the railway environment that use different levels of technology
and specifically considers the implications of the introduction of different types of
control room technology for signaller workload.
There is no one universally accepted definition of workload; workload has been
described as 'measurable quantities of information processing demands placed on
an individual by a task' (Sanders and McCormick, 1993) and it has been
considered to be a 'hypothetical construct that represents the cost incurred by a
human operator to achieve a particular level of performance' (Hart and
Staveland, 1988). No matter how it is defined, the primary consideration is
whether total system demands overload or underload a controller's information-

463
Sarah Nichols eta!.
processing capabilities. If either overload or underload occurs, then an operator
may not be capable of maintaining the appropriate level of performance.
In the current control environment there are discrete operational functions that
are managed by dedicated staff (signallers, electrical controllers and zone control)
from separate geographical locations. A large range of systems and generations of
technology are currently used. As part of the West Coast Route Modernisation,
Railtrack are planning to develop new centralised control centres, known as
Network Management Centres (NMCs). This will result in fewer control centres,
each covering a wider geographical area. NMCs will also house people with
different roles in the same physical location and operators will be increasingly
remote from the point of operational activity. In addition, NMCs will be
accompanied by the introduction of a new generation of signalling technology.
The majority of railway signallers currently use one of two types of technology
- NX Panel-based or VDU-based systems. An NX panel is a 'hard-wired' control
system, with a physical mimic of the track layout, signals and trains, indicated by
LED displays in an overview panel and operated by switch presses. VDU-based
systems represent the tracks and signals on a suite of computer monitors and the
signaller sets routes by using a tracker ball and keyboard. Automatic Route
Setting (ARS) technology is also incorporated within the VDU-based system -
this system automatically sets routes for passenger trains, although it can easily be
overridden by the signaller in order to set a preferred route pattern.
This paper considers the potential impacts of the introduction of NMCs for
signallers. These potential impacts include:
• Increase in geographical area controlled by individual signaller: The
introduction of new signalling technology could allow a signaller to control
a larger geographical area than previously.
• Wider use of ARS technology: The increase in use of ARS could allow
signallers to control a wider geographical area, and minimise the number of
manual route settings required from the signaller.
• Role integration: The physical location of people with different roles at the
same physical location will affect the mode of communication between
signallers and others - changing it from primarily via telephone to face-to-
face communication.
• Increase in distance between signal box and point of operational activity: At
present signallers may be located close to the point of operational activity,
and may even be able to actually see the trains and signals they are
controlling- this will inevitably not be the case with NMCs.
The aim of the research presented in this paper is to make an initial attempt at
establishing the impact of these factors, particularly the increase in geographical
area and use of ARS, on signaller workload.

464
 Workload assessment in railway control
Assessment ofworkload in railway control

At first, railway signalling appears to be a task that is based on the interaction

between the signaller and the signalling system (Panel, VDU-based etc.).
However, there are numerous other inputs into the signal box that affect the
signaller's task, including timetabling information, phone calls from other
signallers, zone controllers, train drivers and maintenance workers and views of
specific areas of track via CCTV monitors. As such it would be extremely
difficult and potentially unrealistic to simulate signalling to such an extent that a
realistic environment was experienced. Therefore an approach to assess the level
of workload in the actual signal box was developed.
After a number of visits to signal boxes, a number of different elements of the
signalling task were identified. These were broadly classified as elements of
'static load'- factors that affected signaller workload but were unlikely to change,
such as number of signals under an individual's control, track length etc; and
'dynamic load' - factors that changed regularly during a period of signal box
operation, such as number of trains under control, or number of phone calls
received. As railway control is a cognitive task that requires a great deal of
planning and interpretation on the part of the signaller, actions of the signaller,
such as route settings, or making phone calls were also included in this
classification of 'dynamic load' as they were felt to be valuable indicators of the
level of work being done by an operator in any one period.

Static Load Dynamic Load

Observed Perceived
Workload Workload

FUNCTION COMPLEXITY INDEX

Figure 1 Integration ofworkload measures

Static and dynamic load combined to form the observable actions in the
signaller's task. Perceived workload measures were also applied during and after
the observation period. The aim of the work was to compare the apparent level of
work being done with the perceived workload of the signaller, with the aim of
identifying trends, so that the potential impact of changes in static and dynamic

465
Sarah Nichols et a!.
load on signaller workload could be anticipated. This prediction tool was named
the 'Function Complexity Index'. Figure 1 illustrates this process.

Method

Fieldwork sites

The work presented in this paper describes visits to two sites. One was an NX
panel with 86 signals (automatic, semi-automatic and controlled), where· the
operator was required to manually set routes on a physical mimic control panel,
and the other was a VDU-based system with Automatic Route Setting (ARS) and
94 signals, where the operator set routes using a tracker ball and key presses.
Both signal boxes contained a single workstation; the NX panel was single-
manned, and the VDU-based box was double-manned, although only one signaller
controlled the system at any one time.

Measures applied

As one of the aims of this work was to establish appropriate measures of workload
in a field environment a number of different measures of workload and operator
performance were employed. In order to minimise disruption and allow data to be
obtained over a longer time period data was collected over ten-minute periods
throughout a shift. Measures of static load were obtained in pilot visits to sites so
collection of this data did not interfere with the observation sessions. During and
immediately after each ten-minute period the following measures were applied.

Self-report rating scales In order to obtain the signaller's subjective reports of

workload to obtain data to compare with observed task performance a short
measure of subjective workload to be applied during the ten-minute period of
observation was required. Initial pilot work (Cordiner et al., 2000) applied a
multidimensional measure of workload- the NASA Task Load Index (TLX) (Hart
and Staveland, 1988). However, this tool was found to be unsuitable for
application in this domain for a number of reasons. Firstly, the TLX evaluates
workload of specific tasks. Railway control hobs do not involve discrete tasks,
and simultaneous tasks and incidents can be ongoing for a number of hours. Staff
also had some difficulty understanding the terminology used to describe the six
constructs. Most importantly, obtaining workload ratings on six scales took
approximately three minutes - this length of time severely interfered with the
signaller's primary task and is not acceptable for on-line instantaneous reports of
workload.
Therefore a uni-dimensional measure of workload - the AFFTC (Air Force
Flight Test Center) Revised Workload Estimate (Ames and George, 1993)- was

466
 Workload assessment in railway control
applied during the ten minute period of observation. The respondent was required
to rate their workload from 1 (No system demands) to 7 (System unmanageable,
essential tasks undone). This scale has been developed to yield interval data, and
has been found to be sensitive across a wide variety of tasks and to correlate well
with other workload measures (Charlton, 1996). This measure was less intrusive
and could be completed by the operator in approximately thirty seconds, although
this time was reduced as signallers became familiar with the measure. In addition,
adapted questions from the TLX asking about levels of thinking and planning,
time pressures and physical fatigue were applied at the end of the observation
period.

Direct observation Pilot work and discussion with signallers identified a number
of observable tasks that were felt to be indicative of the dynamic workload being
experienced by the signaller. Examples of these included phone calls, route sets
and consultations of reference information such as timetables. The different
technologies being used by the signallers meant that the observed tasks differed
slightly between the two sites, but they were felt to reflect the same types of
cognitive process being completed by the signallers. The number of times each of
these tasks was performed during an observation period was recorded.

Video observation Observer Pro software (Noldus et al., 2000) was used to
retrospectively analyse the signaller's actions during the ten minute period of
observation. This allowed information that would have been too complex to
collect concurrently, such as gaze direction and posture to be collected.

Verbal protocol Trials were conducted into the potential of using verbal protocol,
either concurrently or retrospectively, to elicit information about the reasoning and
planning elements of the signaller's task. However, due to the complex nature of
the task and the high level of skill and experience of the signaller, this task was
difficult for the signallers to perform. Although retrospective verbal protocol
using video was more successful, the signallers did not feel comfortable
completing this task, and found it difficult to remember all of the details about the
situation being viewed.

Initial findings

This section presents findings from the observation data and subjective workload
estimates from the AFFTC. More detailed descriptions of the data obtained from
these and other field sessions can be found in Bristol (2000).
Figure 2 shows the comparison between number of route settings in each
observation session and subjective workload rating for that session. A few initial
points can be noted from these graphs - the overall number of route settings

467
Sarah Nichols et a!.
required with the VDU-based system is lower than for the NX panel - reflecting
the impact of ARS on the signalling task. In addition, the subjective workload
ratings appear to be generally lower for the VDU-based system compared to the
NXpanel.
NXPanel VDU-based system

7 7

l0 6 l 6
.s
-t! 5
J.. 4
:ol 5
0
)I
.. 4
.!!:
z
.!.
3 ·i 3
,!,
..::1 ..::1
J! 2 ci! 2
1 II I 1 I I
6 7 8 9 10 11 12 13 14 15 16 2 3 4 5 6 7 8 9 10 11 12
No. o I route settings No. of route settings

Figure 2 Comparison of route settings and subjective workload

In order to obtain data that could contribute towards a Function Complexity

Index, a trend should emerge that indicates a structured relationship between
number of demands placed on the operator and workload experienced. It can be
seen that there is little evidence of such a trend with route settings.
NXPanel VDU-based system

16-20 21-25 26-30 31-35 36-40 6-10 11-15 16-20 21-25 26-30
No. of behaviours No. of behaviours

Figure 3 Comparison of all behaviours and subjective workload

Figure 3 shows a comparison of all behaviours and subjective workload. It can

be seen that for the NX panel a trend does seem to be emerging, where more

468
 Workload assessment in railway control
behaviours seem to be associated with a higher level of workload. However, no
such trend is apparent for the VDU-based system - a system likely to be more
reflective of the type of technology to which we would wish to generalise in the
future.

Methodological issues

A number of methodological issues have emerged from this work. Firstly, the
different requirements placed upon the signallers when working with the different
systems have implications for the observability of their actions - it is much easier
to observe the actions of a signaller manually setting routes on an NX panel than
on a VDU-based system for example. This means that it is more time-consuming
to analyse interactions with VDU-based systems, as more emphasis has to be
placed on the retrospective video analysis.
There are also huge differences in the different data that may be collected at
different times of day and with different signallers. The railway signalling tasks
allows for a certain amount of individual choice on the part of the signaller in
terms of the order in which they complete tasks, and the number of times they
override or anticipate the ARS system. Therefore, actions that appear to indicate a
busier period of work for the operator to the naive observer, may in fact be
strategies employed by an individual signaller in order to aid planning, or maintain
an appropriate level of workload. Discussions with the signallers indicated that
there may also be an underload effect - of which such strategies could be an
indication - and further work has been completed to consider this issue (Bristol,
2000). Finally, on the several occasions that the signal boxes were visited by
researchers, there were no incidents or emergencies that appeared to significantly
impact on the signaller's workload. Therefore it is not clear whether data obtained
from normal running situations can be extrapolated to busy periods. It is of course
these busy periods when the issue of operator workload becomes critical.

Future work

The issues of operator workload - both overload and underload - must be

considered with the introduction of new signalling technology. However, it
appears that many existing tools, that have primarily been developed for use in a
simulated environment, are not appropriate for application in the field. If the
potential impact of new technology on workload is to be anticipated, then tools
that can be unobtrusively applied in the field are required.
There are a number of contrasting approaches that may be considered suitable.
The first is to continue with the approach outlined in this paper, but to collect
more detailed information about task activities, and increase the data pool by a
large amount so that statistical. analysis techniques, such as multiple regression,
can be applied. This approach would allow the development of a Function

469
Sarah Nichols et a/.
Complexity Index, but is highly dependent on the quality of a large amount of
data, and as such would be time-consuming. The second approach would be to
pursue qualitative analysis, and focus on the nature of the dynamic task elements,
rather than simply the number of each element. Thus, instead of counting the
number of phone calls made and received by the operator, the content of these
calls, and their relationship to the decisions made could be identified. This
approach places a heavier burden on the analysis, but may identify critical
indicators ofworkload.
It is vital that we better understand the workload involved in tasks such as
railway signalling, prior to the introduction of integrated control centres, so that
staff can maintain the high level of performance and have sufficient capacity to
deal with abnormal events and emergency situations.

References

Ames, L.L. and George, E.J. (1993). Revision and verification of a seven-point
workload scale (AFFTC-TIM-93-01). Air Force Flight Test Center: Edwards
AFB, CA.
Bristol, N. (2000). Developing a set of tools to measure changes in the workload
and underload of railway signal controllers. Unpublished MSc Thesis,
University ofNottingham.
Charlton, S.G. (1996). Mental Workload Test and Evaluation. In, T.G. O,Brien
and S.G. Charlton (1996) Handbook of human factors testing and evaluation.
Lawrence Erlbaum Associates: Mahwah, NJ.
Cordiner, L., Nichols, S. and Wilson, J.R. (1998) RELOAD: Summary ofworkload
assessment from field studies. Confidential IOE Report RAIL/00/012.
Hart, S.G. and Staveland, L.E. (1988) Development of the NASA task load index
(TLX): Results of empirical and theoretical research. In, P.A. Hancock and N.
Meshkati (Eds.) Human Mental Workload. North-Holland: Amsterdam.
Noldus, L.P.J.J., Trienes, R.J.H,. Hendriksen, A.H.M., Jansen, H. and Jansen,
R.G. (2000). The Observer Video-Pro: New software for the collection,
amangement and presentation of time-structured data from videotapes and
digital media files. Behaviour Research Methods, Instruments and Computers,
32, 197-206.
Sanders, M.S. and McCormick, E.J. (1993). Human Factors in Engineering and
Design. McGraw-Hill: New York.

470