Human Factors Methods

A Practical Guide for Engineering and Design

2nd Edition

NEVILLE A. STANTON
PAUL M. SALMON
LAURA A. RAFFERTY
GUY H. WALKER
CHRIS BABER
DANIEL P. JENKINS
206 HUMAN FACTORS METHODS HUMAN ERROR IDENTIFICATION AND ACCIDENT ANALYSIS METHODS 207

Reliability and Validity The Cognitive Reliability and Error Analysis Method (CREAM)

Kirwan (1992a, 1992b) describes a validation of nine HRA techniques and reports that, of the nine Background and Applications
techniques, HEART, THERP, absolute probability judgment (APJ) and JHEDI performed moderately
well. A moderate level of validity for HEART was reported. In a second validation study (Kirwan 1997a), CREAM (Hollnagel,1998) is an HEI/HRA method that was developed in response to an analysis of existing
HEART, THERP and JHEDI were subject to a validation study. The highest precision rating associated HRA approaches. It can be used both predictively, to predict potential human error, and retrospectively,
with the HEART technique was 76.67 per cent. Of 30 assessors using the HEART approach, 23 displayed to analyse and-quantify-error. According to Hollnagel (1998), CREAM enables the analyst to:
a significant correlation between their error estimates and the real HEPs. According to Kirwan (1997a,
1997b), the results demonstrate a level of empirical validity for the three techniques. identify those parts of the work, tasks or actions that require or dependuupon human cognition,
and which therefore may be affected by variations in cognitive reliability;
Tools Needed determine the conditions under which the reliability of cognition may be reduced, and where
therefoYettre-actions may constitute a source of risk;
The HEART approach can be applied using pen and paper.The associated HEART documentation is also provide an appraisal of the consequences of human performance on system safety, which can be
required (HEART generic categories, HEART EPCs, etc.). used in probabilistic risk assessment (PRA)/PSA; and
• develop and specify modifications that improve these conditions and hence serve to increase the
Flowchart reliability of cognition and reduce the risk.

START i
CREAM uses a model of cognition, the COCOM, which focuses on how actions are chosen and assumes
that the degree of control that an operator has over his or her actions is variable, and determines the
Analyse task
using HTA reliability of his or her performance;The COCOM describes four modes of control: scrambled control,
opportunistic control, tactical control and strafe is control. Acc iab to Noffnagel (1998), when the
i Take the first/nod
bottom level task
level of operator ntr_6 rises, so does the operators' performanceor reliability.
l The CREAM technique uses
step from the HTA a classification scheme consisting of a number of groups that describe the phenotypes (error modes) and
genotypes (causes) of the erroneous actions. The CREAM classification scheme is used by the analyst to
Assign a HEART
generic category to the predict and describe how errors could potentially occur. It allows the analyst to define the links between,
taskstep in question -
the causes and consequences of the error under analysis. Within the CREAM classification scheme, there
Assign a nominal human !. are three categories of causes (genotypes): individual, technological and organisational causes. A brief
i error probability(HEP) to'.
the task step in question) description of each genotype category is provided below:

Select any relevant error

I producing conditions Individual-related genotypes: specific cognitive functions, general person-related functions
7
i—.
(temporary) and general person-related functions (permanent).
Take the first/ next
Technology-related genotypes: equipment, procedures, interface (temporary) and interface
EPC (permanent).
Organisation-related genotypes: communication, organisation, training, ambient conditions and
Select any relevant error
producing conditions ! working conditions.
(EPC's)

The CREAM technique uses a number of linked classification groups. The first classification group
Are there
any more
describes the CREAM error modes, which are presented below:
N--<,
EPCs?

Timing — too early, too late, omission.

v Duration — too long, too short.
Calculate the final Sequence — reversal, repetition, commission, intrusion.
HEART HEP for the
task step In question Object — wrong action, wrong object.
Force — too much, too little.
Direction —Wrong direction.
Are there
any more Distance — too short, too far.
task steps? ,--
Speed — too fast, too slow.
N
These eight different error mode classification groups are then divided further into the four sub-groups:
STOP
 _111111111ow-

208 HUMAN FACTORS METHODS UMAN ERROR IDENTIFICATION AND ACCIDENT ANALYSIS METHODS 209

• Action at the wrong time - includes the error mode's timing and duration. Procedure and Advice (Prospective Analysis)
• Action of the wrong type - includes the error mode's force, distance, speed and direction. -
• Action at the wrong object - includes the error mode'object. Step 1: Task Analysis
• Action in the wrong place - includes the error mode'sequence. The first step in a CREAM analysis involves describing the task or scenario under analysis. It is
recommended that an HTA of the task or scenario under analysis is developed for this purpose. A number
In addition to the exploration of error modes (phenotypes) and causes of error (genotypes), the CREAM of data collection procedures may be used to collect the data required for the HTA, including interviews
technique also uses a set of CPCs that are used by the analyst to describe the context in the scenario/task with SMEs and observational study of the task or scenario under analysis.
under analysis. These are similar to the PSFs used by other HEI/HRA techniques. The CREAM common
performance conditions are presented in Table 6.23. Step 2: Context Description
CREAM is highly successful and as such a number of variations on the method have been developed. Once the task or scenario under analysis is described, the analyst should begin by first describing the
A recent adaptation of the CREAM technique is the CEAM (Communication Error Analysis Method) (Lee, context in which the scenario under analysis takes place. This involves describing the context using
Ha and Seo_ng, 2011) which focuses upon errors of communication. Kennedy et al. (2007) also identified the CREAM CPCs (Table 6.23). To do this, the analyst uses his or her subjective judgment to rate each
a s mp~lified version ofTHERP called the Accident Sequence Evaluation Programme (ASEP),which they CPC regarding the task under analysis. For example, if the analyst assumes that the operator has little
suggest is beneficial for the identification of high-level hazards. A frequently utilised version of CREAM experience or training for the task under analysis, then the CPC'Adequacy of training and experience'
has also been developed called DREAM. should be rated'limited experience/inadequate.
DREAM is an adaptation of the CREAM method designed for use within the driving domain. It was
developed in order to provide a series of codes capable of describing both genotype and phenotype Step 3: Specification of the Initiating Events
causality in car crashes (Ljung, 2010). The analyst then needs to specify the initiating events that will be subject to the error predictions.
Hollnagel (1998) suggests that PSA event trees can be used forthis step. However, since a task analysis has
Domain of Application already been conducted in step 1 of the procedure, it is recommended that this be used. The analyst(s)
should specify the tasks or task steps that are to be subject to further analysis.
Although the technique was developed for the nuclear power industry, it is a generic approachnd can
be applied in any of domain involving the operation of complex, dynamic systems. Step 4: Error Prediction
Once the CPC analysis has been conducted and the initiating events are specified, the analyst should
Table 6.23 Cream CPCs then determine and describe how an initiating event could potentially into an error occurrence.
To predict errors, the analyst constructs a modified consequent/antecedent matrix. The rows on
CPC name Level/descriptors the matrix show the possible consequents, while the columns show the possible antecedents. The
Adequacy of organisation. The quality ofthe roles and responsibilities ofteam members, additional support, communication analyst starts by finding the classification group in the column headings that correspond to the
systems, safety management systems, instructions and guidelines for externally orientated activities,
initiating event (e.g. for missing information, it would be communication). The next step is to find all
etc.:
very efficient/efficient/inefficient/deficient.
the rows that have been marked for this column. Each row should point to a possible consequent,
Working conditions. The nature ofthe physical working conditions such as ambient lighting, glare on screens, noise from
which in turn may be found amongst the possible antecedents. Hollnagel (1998) suggests that in
alarms, task interruptions, etc.: this way, the prediction can continue in a straightforward way until there are no further paths left.
advantageous/compatible/incompatible. Each error should be recorded along with the associated causes (antecedents) and consequences
Adequacy of MMI and The man—machine interface in general, including the information available on control panels, (consequents).
operational support. computerised workstations and operational support provided by specifically designed decision aids:
supportive/adequate/tolerable/inappropriate.
Step 5: Selection of Task Steps for Quantification
Availability of procedures/ Procedures and plans, including operating and emergency procedures, familiar patterns of response
plans. heuristics, routines, etc.: Depending upon the analysis requirements, a quantitative analysis may be required. If so, the analyst
appropriate/acceptable/inappropriate.. should select the error cases that require quantification. It is recommended that if quantification is
Number of simultaneous The number oftasks a person is required to pursue or attend to at the same time: required, all of the errors identified should be selected for quantification.
goals fewer than capacity/matching current capacity/more than capacity.

Available time The time available to carry out the task: Step 6: Quantitative Performance Prediction
adequate/temporarily inadequate/continuously inadequate.. CREAM has a basic and extended method for quantification purposes. Since this review is based upon
Time of day (Circadian Time at which the task is carried out, in particular whether or not the person is adjusted to the the predictive use of CREAM, the error quantification procedure is not presented. For a description of the
rhythm). current time: quantification procedure, the reader is referred to Hollnagel (1998).
daytime (adjusted)/night-time (unadjusted).

Adequacy of training and The level and quality oftraining provided to operators, such as familiarisation with new technology, Advantages
experience. refreshing old skills, etc. Also refers to operational experience:
adequate, high experience/adequate, limited experience/inadequate..
CREAM has the potential to be extremely exhaustive.
Crew collaboration quality. The quality of collaboration between the crew members, including the overlap between the official
and unofficial structure, level oftrust and the general social climate among crew members: Context is considered when using CREAM.
very efficient/efficient/inefficient/deficient.
 _111111111i'mo-

210 HUMAN FACTORS METHODS ;d JMAN ERROR IDENTIFICATION AND ACCIDENT ANALYSIS METHODS 211

• It is a clear, structured and systematic approach to error identification and quantification. Flowchart - Prospective Analysis
• It can be used both proactively to predict potential errors and retrospectively to analyse error
occurrence. j START
• It is not domain-specific and the potential for application in different domains is apparent.
• Its classification scheme is detailed and exhaustive, even taking into account system and
r

environmental (socio-technical) causes of error. Analyse task

• Sandin (2009) argues that the links in a DREAM analysis (and correspondingly a CREAM analysis) using HTA
L
illustrate causal relationships.
Take the first/next
Disadvantages { bottom level task
step from the HTA
t.
• To the novice analyst, CREAM appears complicated and daunting.
• The exhaustiveness of the classification scheme serves to make it larger and more resource- Assign a HEART
generic category to the
intensive than other methods.
task step in question
• It has not been used extensively. _ ____._ L
• It is apparent that the training and application time for the CREAM technique would be
I Assign a nominal human
considerable.
error probability (HEP) to i
• It does not offer remedial measures, i.e. ways to recover erroneous human actions are not the task step in question)
provided or considered.
I
• It appears to be very complicated in its application. Select any relevant error
• It would presumably require analysts with knowledge of HF and cognitive ergonomics. producing conditions
• Its application time would be high, even for very basic scenarios. (EPC's)

Related Methods j Take the first/ next

EPC
CREAM analyses are typically conducted on an HTA of the task or scenario under analysis. A number of
data collection procedures may be used during the development of the HTA, including interviews with W

SMEs and observational study of the task or scenario in question. CREAM is a taxonomy-based approach iSelect any relevant error
1 producing conditions
to HEI. Other taxonomic approaches include SHERPA (Embrey, 1986), HET (Marshall et al., 2003) and L (EPC's)
TRACEr (Shorrock and Kirwan, 2002).
I
-,

Approximate Training and Application Times Are there

N--<,' any more
Although there is no data regarding training and application times presented in the literature, it is EPC's?

estimated that the associated times will be high in both cases.

Reliability and Validity

Calculate the final
Validation data for the CREAM technique is limited. Hollnagel, Kaarstad and Lee (1999) report a 68.6 Y HEART HEP for the
per cent match between errors predicted and actual error occurrences and outcomes when using the L task step in question

CREAM error taxonomy.

V

Tools Needed Are there \

any more
ta`sk ste ps?J
At its simplest, CREAM can be applied using pen and paper only. A prototype software package has
also been developed to aid analysts (Hollnagel, 1998). Kennedy et al. (2007) identify a recent software ksep

development to aid in THEA analysis: ProtoTHEA. N

STOP
J