APAC AIG Investigation Analysis Workshop Manual

APAC AIG
Investigation Analysis
Worksop

8‐10 October 2018
Bangkok, Thailand

Workshop Topics

1. Introduction
2. The modern approach to safety
3. Human Factors
4. Investigation analysis framework
5. Sequence of events
6. Evidence tables
7. Contributing Factor analysis
8. ORLIO factor mapping
9. Safety Action and Recommendations
10. Summary

Topic 1
Introduction

Investigation Analysis
Dr Richard Batt
Australian Transport Safety Bureau
Capt. Nurcahyo ‘Ray’ Utomo

Indonesian national Transportation Safety Committee
Introduction
• Welcome!
• Thank you to the ICAO Asia and Pacific Regional Office,
and in particular the Regional Director Mr Arun Mishra,
for supporting the APAC AIG Investigation Analysis
Workshop
1
Introduction
• Safety and comfort
• Mobile phones and electronic devices
• Lunch and morning and afternoon tea provided
• Workshop facilitators
– Dr Richard Batt, ATSB Manager International
– Capt. Nurcahyo ‘Ray’ Utomo, NTSC Head of Aviation Sub
Committee
Annex 13
• Independence of investigations
– A State shall establish an accident investigation authority that is
independent from State aviation authorities and other entities
that could interfere with the conduct or objectivity of an
investigation
• Objective of the investigation
– The sole objective of the investigation of an accident or incident
shall be the prevention of accidents and incidents
– It is not the purpose of this activity to apportion blame or liability
2
The role of safety investigation
• The role of an independent safety investigation authority
is crucial to maintaining and enhancing aviation safety
• By carrying out aviation safety investigations in a
thorough and timely manner, a State/Administration can
identify areas of safety concern, leading to action by
government and aviation industry organisations
What does ‘Independent’ mean?

‘Independent’ refers to a number of different aspects
related to both the structure and functioning of the accident
investigation authority
3
Independence of structure
• Independence of structure refers to the organisational
context in which the accident investigation authority
operates
• In general, the investigation authority should be
independent from any other party whose interests could
conflict with those of the authority
• The authority should be separate from the national civil
aviation authority – the ‘regulator’
Independence of functioning
The accident investigation authority should not be
controlled or limited in its ability to,
• Initiate investigations
• Gather and analyse evidence
• Determine causes/contributing factors
• Issue safety recommendations
• Publically release investigation reports
4
ICAO Doc 9756
• Manual of Aircraft Accident and Incident Investigation
– Part 1 – Organization and Planning
– Part 2 – Procedures and Checklists
– Part 3 – Investigation
– Part 4 – Reporting
ICAO Secure Portal

https://portallogin.icao.int
ICAO Doc 9756 Part 4 - Reporting

• Analysis section of the Final Report
– The purpose of the analysis is to provide a logical link between
the factual information and the conclusions that provide the
answer to why the accident occurred
– The reasoning must be logical and may lead to the formulation
of hypotheses which are then discussed and tested against the
evidence
– It is important to state the reasons why a particular hypothesis
is accepted or rejected
5
Analysis
• Analysis is evidence-based
• Facts not opinions
The role of an investigation authority

• Aviation safety investigation authorities are information
processing organisations
– Gather safety information
– Analyse safety information
– Communicate and encourage others to act on safety information
• In many cases, effort and resources are applied in
decreasing amount to gather, analyse, and
communicate/act
6
Workshop overview
• Workshop topics
– The modern approach to safety
– Human Factors
– Investigation analysis framework
– Sequence of events
– Evidence tables
– Contributing Factor analysis
– ORLIO factor mapping
– Safety Action and Recommendations
Workshop overview
• A workshop format – you will be doing exercises in small
groups and reporting back to the full assembly
• Prepare your report on a laptop and save to a USB for
presentation
• A USB with resource materials is provided
7
Thank you
8

Topic 2
The modern approach to safety

Richard Batt
Overview
• Three key principles of the modern approach to safety
• Sources of safety data
• Investigating serious incidents
• Normal operations monitoring
• Equal importance of technical and non-technical aspects
• Person vs System approach
1
• An emphasis on a proactive approach
• Equal importance is given to technical and non-technical
aspects
• A recognition that in high-reliability systems, accidents
happen to systems, not people
Proactive safety
• Having and heeding good safety data is the key to
improving aviation safety
• Sources of safety data,
– Investigating accidents
– Investigating serious incidents
– Normal operations monitoring
2
Investigating serious incidents
• Investigating a ‘close call’ provides a ‘free lesson’ in
safety
• The same safety benefit can be achieved without paying
the economic and social cost associated with an
accident
Annex 13
Definition of Serious Incident
• An incident involving circumstances indicating that there
was a high probability of an accident.
• Note: The difference between an accident and a serious
incident lies only in the result.
3
Annex 13
Section 5.1.2
• The State of Occurrence shall institute an investigation
into the circumstances of a serious incident when the
aircraft is of a maximum mass of over 2,250 kg
ICAO Annex 13 – Attachment C
Examples of serious incidents
• Near collisions
• Near controlled flight into terrain (CFIT)
• Take‐offs or landings on a closed or engaged runway
• Fires and/or smoke
• Multiple systems failures
• Flight crew incapacitation
• Fuel emergency
4
Proposed changes to Attachment C
• State Letter 2018/34 includes a number of proposals for
amendment to Annex 13 arising from AIGP/3
• A proposed amendment to Attachment C is to include
guidance for States/Administrations to apply a simplified
version of the Aviation Risk Management Solutions
(ARMS) event risk classification (ERC) matrix when
determining if an incident was a serious incident
Examples of serious incidents

investigated by the ATSB
From 2013 to 2017, the ATSB investigated 68 serious
incidents involving high‐capacity regular public transport
operations
Max seating capacity >38 seats or max payload > 4,200kg
5
ATSB RPT serious incidents
• Flight crew incapacitation (15) • Runway event (4)
• Aircraft systems and/or engine • TCAS RA and close proximity
failure (8) in flight (4)
• Fire, smoke, electrical arc (5) • Loss of separation or
assurance (4)
• Ground event (6)
• Stickshaker event (2)
• Weather related (6)
• Other (9)
• EGPWS and descent below
minima (5)
ATSB Investigation AO-2010-081

Stickshaker activation
• Boeing 717, two stickshaker
activations on approach to land at
Kalgoorlie, WA
• Crew had entered the wrong aircraft
weight into the FMS
• Incorrect approach speed
6
Descent below approach path
• B777 low on approach to Melbourne,
Vic
• Tower controller asked the crew to
check their altitude and then
instructed the crew to go‐around
• Crew caught by surprise by the
aircraft’s automation

Loss of separation
• Two B737 aircraft on converging
tracks at FL39 near Ceduna, SA
• Procedural longitudinal separation
standard of 20 NM was infringed
• Contributing factors of controller
workload and experience
7
Airspeed indication failure on take-off
• A330 aircraft departing Brisbane, Qld, for Kuala Lumpur,
Malaysia
• During the take‐off roll the crew detected an airspeed
anomaly, including red speed (SPD) flags on both primary
flight displays (PFD)
• When airborne, the flight crew carried out actions for
unreliable airspeed indications and made a PAN call
Airspeed indication failure

• In accordance with published procedures, the flight crew
turned off the three air data reference systems (ADRs)
• This activated the aircraft’s backup speed scale (BUSS),
which provided a colour-coded speed scale derived from
angle of attack and other information, and altitude
derived from GPS data
• The flight crew also obtained groundspeed information
from ATC, and used the aircraft’s radar altimeter
8
• Normal landing gear extension could not be accomplished
with all three ADRs off
• The flight crew performed a landing gear gravity extension
before conducting an overweight landing
• A subsequent inspection identified that the pitot probe
covers were still fitted to the aircraft’s three pitot probes
after it landed
9
10
• Local engineering support crew placed covers on the
pitot probes after the previous landing
• There has been a problem at Brisbane Airport with pitot
tubes being blocked by insect nests in recent years
• Inspections during the aircraft’s turnaround did not
identify that the pitot covers were in place
• The investigation is continuing
Serious incidents
• How many are reported?
• How many are investigated?
11
Sources of safety data
• Investigating accidents
• Investigating serious incidents
• Normal operations monitoring
Normal operations monitoring

• Normal operations monitoring uses trained observers to
collect data about operator’s actions and the situations
they face in everyday ‘normal operations’
• LOSA – Line Operations Safety Audit
• ICAO – Normal Operations Safety Study
12
Normal operations monitoring
• Unobtrusive fly-on-the-wall observations
• Identifies threats and errors, and how they were managed
• Non-jeopardy (individual names not recorded)
• An audit of an organisation, not individuals or teams
Normal Operations Safety Study (NOSS)

• ICAO Document 9910
13
aspects
Technical and non-technical aspects

• Critical to safe operations
• Essential for a thorough safety investigation
14
Operational non-technical skills
The cognitive and interpersonal skills necessary for
efficient and safe operations,
• Cooperation and teamwork
• Leadership and managerial skills
• Communication skills
• Situational awareness
• Decision making
15
Aviation safety investigation
• The crucial first step in any aviation safety investigation
is a thorough operational and technical investigation to
establish what happened in the accident or incident, but
it is typically only by then looking at human factors - at
both the individual and organisational level - that we can
understand how and why the accident or incident
occurred.

aspects
16
Person vs System approach
• The Person approach
– Focuses on the errors and violations of individuals
– Remedial efforts directed at people at the ‘sharp end’
• The System approach
– Traces the contributing factors back into the system as a whole
– Remedial efforts directed at situations and organisations
Human error in context

• Human error cannot be entirely eliminated
• We need to understand how and why errors happen
• Many errors reflect systemic deficiencies
17
Limitations of the Person approach
Blaming individuals can be emotionally satisfying and
legally convenient, but achieves little because,
• Fallibility is part of the human condition
• You can’t change the human condition
• You can change the conditions in which humans work
Systemic deficiency
• If human error on the part of one or two individuals can
go unchecked within an organisation and result in a
significant breakdown of the workings of the system,
then the failure reflects a systemic deficiency, not just
human error
18
Active vs latent failures
• Active failures
– Errors that occur at the level of the frontline operator and whose
effects are felt almost immediately
• Latent failures
– Errors in the design, organization, training, or maintenance that
lead to operator errors and whose effects typically lie dormant in
the system for lengthy periods of time
– Latent failures are typically the consequence of management or
organizational processes
Multiple Contributing Factors

• Most accidents and incidents are due to a combination of
Contributing Factors
• How factors combine can be complex and dynamic
19
The Swiss cheese model (Reason)
CFIT accident Commercial

pressures
Fatigue
Poor design of
approach plate
CFIT
Inadequate TOD briefing
20
21
Organisational mindfulness
• No system can guarantee safety for once and for all
• Don’t take success as proof that nothing can go wrong
• Be wary of using success to justify the elimination of
what is seen as unnecessary effort and redundancy
• Accept appropriate redundancy so that staff are not
routinely placed in situations of overload

aspects
22
Thank you
23

Topic 3
Human Factors

Human Factors
Richard Batt
Overview
• What is Human Factors?
• Human error
• Some Human Factors topics and case studies
1
Why investigate?
• What happened?
Operational and technical investigation
• Why did it happen?

Human factors and organisational investigation
• What can we do about it?

Safety action
What happened
How and why it

happened
2
What is Human Factors?
Human factors is the multi-disciplinary science that
applies knowledge about the capabilities and
limitations of human performance to all aspects of the
design, manufacture, operation, and maintenance of
products and systems.
What is Human Factors?

Human factors considers the effects of physical,
psychological, and environmental factors on human
performance in different task environments, including
the role of human operators in complex systems.
3
Ergonomics
Perception
Memory Workload
Stress
CRM Automation
Attention
Human Information Processing
Physiological Factors Fatigue

Decision Making
4
Aviation accident factors
(Hobbs, 2004)
Human error
5
Human error
Human error is a natural part of life.
A key principle is that human error, although undesirable, is

nevertheless both frequent and widespread.
What are you likely to forget to do?
The same situations keep on provoking the same kinds

of errors, regardless of who is involved.
6
Myths about human error
Myth:
Human error occurs randomly.
Fact:
Human error is not random. It is systematically connected
to features of operator’s tools, tasks, and operating
environment.
Myth:
A highly experienced and motivated operator could not
have made an error during a familiar task.
Fact:
The best people can make the worst mistakes.
7
Myth:
If an operator can perform a task easily, they will never
make an error doing the task.
Fact:
Operators will periodically make errors as a consequence
of variations in task, environment, and individual factors.

Myth:
Operators who make errors during a familiar task reveal a
lack of skill, vigilance, or diligence.
Fact:
Skill, vigilance, and diligence are necessary but not
sufficient to prevent error.
8
Myth:
The types of human error that lead to serious accidents are
different to the types of human error that we see in
everyday life.
Fact:
The types of human error can be the same. Whether the
consequences are minor or disastrous depends on the
situation.
From Pilot Error to Human Error
Human error is not the conclusion of a safety investigation,

it is the starting point.
9
Investigating human error
• Just finding and highlighting people’s mistakes explains
nothing
• To understand human error you have to understand why
people’s actions and assessments made sense to them
at the time
• The principle of Local Rationality
10
Intention to comply
• Errors – unintentional
– The failure of planned actions to achieve a desired goal
– Information processing problem
• Violations - intentional
– Intentional deviations from SOPs, standards or rules
– Social / motivational problem
• Helps to identify what type of safety interventions may
be helpful – eg VFR into IMC
Violations
• Routine violation
– Normal way of operating, often condoned by management
• Situational violation
– Irregular deviation from procedures to get the job done
• Exceptional violation
– Individual satisfying other motives, eg to impress others
11
Classifying individual actions
Violation + Error = Potential disaster
12
The culpability continuum
10% 90%
Culpable Blameless
Sabotage System-induced violations

Substance abuse Negligent errors
Reckless violations System-induced errors
etc. etc.
The culpability continuum

Knowingly Pass History
Were the Unauthorised violating substitution of unsafe
NO NO NO YES
actions substance? safe operating test? acts?
as intended? procedures?
NO
YES
YES NO YES YES NO
Deficiencies
Were procedures in training &
Medical available, workable selection or
condition? intelligible and inexperience? Blameless
correct? Blameless error
error but
Were the NO YES corrective
consequences YES NO training or
as intended? NO YES System- counselling
Possible induced indicated
negligent error
System-
error
induced
YES violation
Possible
reckless
Substance violation
Substance abuse with
abuse without mitigation
Sabotage, mitigation
malevolent
damage,
suicide, etc.
13
Substitution test
Would you have done the same thing?
Could a different person (well motivated, equally

competent, and comparably qualified) have made the
same error under similar circumstances (determined by
their peers)
If ‘Yes’ the person who made the error is probably

blameless
Error management
We can’t change human nature, but we can change the
conditions under which people operate.
Hence, the focus of error management is on,

• Error reduction
what systemic factors provoke errors
• Error containment
detection and recovery (greater system tolerance)
14
Errors are like mosquitoes
You can swat them one by one,

but they still keep coming.
(Reason 1990)
Drain the swamp in which they breed

Conflicting
goals Poor
defences
Training Poor
deficiencies design
Inadequate
procedures
15
Learning from errors
If errors are not reported, possibly because staff think that
they will be punished, then the organisation will lose a
valuable source of safety information.
Therefore, need to foster a Just Culture in which people

are encouraged, even rewarded, for reporting unsafe acts.
Developing a Just Culture

• A Just Culture depends on,
– The trust of the workforce
– Knowing and agreeing on the difference between acceptable
and unacceptable behaviour
• A ‘No blame’ culture is neither feasible nor desirable.
• Some unsafe acts deserve sanctions.
16
Just Culture
Balancing safety and accountability
For those that run or

regulate organisations, the
incentive to have a Just
Culture is very simple.
Without it, you won't know
what is going on.
Sidney Dekker
Human error - Summary
We can’t eliminate human error, but we can,

understand
minimise
and control it.
17
Human Factors
• A selection of topics and case studies
– Human error
– Attention
– Workload
– Fatigue
– Stress
– Automation
– Decision making
– Spatial disorientation
Data entry errors

• An example of how a simple human error can have
potentially catastrophic consequences
• Whether the consequences are minor or disastrous
depends on the situation
18
Runway overrun and tailstrike
• A340 flight from Melbourne to Dubai, March 2009, with
18 crew and 257 passengers on board
• Departure at night with no moon
• During take-off the aircraft failed to accelerate sufficiently
• At the end of the runway, the crew applied TOGA power
• The aircraft sustained a tailstrike and overran the end of
the runway
19
20
Incorrect take-off weight
• The crew climbed the aircraft to a safe height and
declared an emergency
• The aircraft returned to land at Melbourne after
dumping fuel
• The take-off weight used to perform the take-off
performance calculations was 262.9 tonnes instead
of the planned 362.9 tonnes
Data entry error

• Data entry error occurred when the take-off weight
calculated by the aircraft’s flight management and
guidance system (FMGS) was manually entered into
the electronic flight bag (EFB)
• A single digit error, 262.9 entered instead of 362.9
• A required verbal check was omitted at a busy time in
the cockpit
• Two further opportunities to capture the error were
also not successful
21
Degraded take-off performance
• Acceleration information isn’t presented to the flight crew
• Crew experienced a wide range of performance, eg in
previous two months TOW varied from 150-370 tonnes
• Speeds vary with weight, no ability to do a
‘reasonableness’ check
• Dark night takeoff meant there were reduced visual cues
Future risk controls

• An automatic system check of the ‘reasonableness’
of the performance data entered into the FMGS
• Take-off performance monitoring systems
22
Attention
Attention
• The selection of information for further processing
• Attention is a limited resource that
– Prevents cognitive ‘overload’
– Ensures cognitive resources are available for particular tasks
23
Limits to attention
• Too much information
Information
• Too little information not detected
• Stress
• Low Arousal Information
• Fatigue is ‘shed’
24
Attention
• Inattentional blindness
Failure to perceive a fully-visible but unexpected object
because attention was directed elsewhere
‘Looked but failed to see’
• Attentional tunnelling / fixation

Allocation of attention to a particular information source or
task for a duration longer than optimal in the circumstances
Narrowing of attention is exacerbated by stress
The multitasking myth

“The myth of multitasking is that we
can - and in fact are expected to -
handle concurrent demands without
repercussions.”
The Multitasking Myth: Handling Complexity
in Real‐World Operations.
Loukopoulos, Dismukes, and Barshi (2009).
25
Workload
26
Effect of workload on performance
Both very low and very high levels of task load
can lead to degraded performance.
Yerkes-Dodson Inverted U Curve
27
Vigilance tasks
• Operator required to detect signals over a prolonged
period of time when the signals are intermittent and
infrequent
• Low level of stimulation
• Vigilance decrement - deterioration of sustained
attention over time
– eg a pilot monitoring and aircraft on autopilot in the cruise
Fatigue
28
Fatigue
• Fatigue affects everyone regardless of motivation, skill,
knowledge and training
• Individuals are not good at judging their own level of
fatigue
Determinants of fatigue
• Time awake
• Circadian rhythm
• Sleep inertia
29
Effect of fatigue on performance
• Fatigue can have a negative effect on
– Attention and reaction time
– Short term memory
– Decision making
– The timing of tasks
– The ability to judge distance, speed, and time
– The ability to perceive and interpret information
– Communication and coordination
Effect of fatigue on performance

The effect of 17 to 24 hours of sleep deprivation on
performance is equivalent to the impairment in
performance of a blood alcohol level of 0.05 to 0.10 BAC.
30
Fatigue management approaches
FRMS – Fatigue Risk Management System
ICAO fatigue management manuals
https://www.icao.int/safety/fatiguemanagement/Pages/Resources.aspx
31
ICAO fatigue management manuals
https://www.icao.int/safety/fatiguemanagement/Pages/Resources.aspx
32
Biomathematical fatigue models
• Examples of biomathematical fatigue models
– Boeing Alertness Model (BAM)
– Circadian Alertness Simulator (CAS)
– Fatigue Assessment Tool by InterDynamics (FAID)
– Fatigue Risk Index (FRI)
– System for Aircrew Fatigue Evaluation (SAFE)
– SAFTE-FAST
– Sleep / Wake Predictor (SWP)
Biomathematical fatigue models

• Be aware of the strengths and limitations of the models
– Data inputs and outputs
– Use as an investigation tool
• CASA Australia guidance manual
https://www.casa.gov.au/files/biomathematical-fatigue-models-guidance
33
Inflight loss of control
• Saab 340 on approach to Bathurst, NSW, June 2002
• Aircraft stall and loss of control - ICAO Serious Incident
• No injury or damage, very nearly catastrophic

• The aircraft stalled due to probable ice accretion on the
wings
• Flight crew
– Did not detect or adequately consider airframe or wing ice during
the descent
– Left the power setting unchanged after levelling out
– Did not notice increase in pitch or decrease in speed until just
prior to the stall
34
• Flight crew
– Had flown six sectors and had three sectors to go
– Had flown an additional sector that was not rostered
– Only had the opportunity for a snack during the day
• Flight crew report
– The PIC said that it had been a long day - he was up early and
during the night with his young children
– The copilot said that it had been a big day with a heavy
workload, and that she was tired
35
Stress
Possible causes of stress
• Task
• Environment
• Life events
Stress due to personal and work factors can interact.
36
Effects of stress
• Narrowing of attention (task stress)
- more focussed attention, can be good or bad
- reduced peripheral vision
• Distraction (environment / life stress)

- diverts attention from working memory
Change is stressful
37
Automation
Automation surprise
• What is it doing?
• Why is it doing that?
• What will it do next?
Automation as a poor team player,

• Strong minded
• A poor communicator
• Unpredictable
38
Mode error
• A mode error occurs when an operator loses track of
which mode the device is in, or confuses which actions
are appropriate in a particular mode
Mishandled go-around
• A320 on approach to Melbourne, Vic, in July 2007
• Missed approach initiated due to fog
• Crew did not perform the go-around procedure correctly
• Crew unaware of the aircraft's current flight mode
• The aircraft descended to within 38 ft of the ground
before climbing
39
A320 thrust lever positions
Thrust levers are also
mode selectors
‘TOGA tap’
40
A320 Flight Mode Annunciator (FMA)
Changed go-around procedure

Aircraft manufacturer’s procedure Operator’s changed procedure
41
Airbus Safety Magazine
Airbus recommendations for go-around

• Firewall it!
• Thrust levers are also mode selectors
• Know your FMA at all times
42
Decision making
Human information processing
43
Decision making
• Decision making involves
– Uncertainty
– Multiple options
– Risk
• Hence, an investment of cognitive resources
Decision making in practice

• Skill-based
– Emergency drills, memory item responses
• Rule-based
– Standard Operating Procedures (SOPs), Quick Reference
handbook (QRH)
• Knowledge-based
– Expertise, Recognition Primed Decision Making
44
Decision making processes
• Analytical, calculative
– Conscious effort, controlled processing
– GRADE, DECIDE, etc
• Naturalistic, intuitive
– Non-conscious automatic processing
– Recognition Primed Decision (RPD) making
Naturalistic decision making

• Describes how people use their experience to make
decisions in real-world settings
• Typically involves ill-structured problems, dynamic
environments, time pressure, and competing goals
• Emphasises that the way people size up situations is
more critical than the way they select between courses
of action
45
What affects decision making?
• Fatigue
• Workload
• Stress
• Poor team coordination and communication
• Organisational influences
• Most HF factors!
Pre-packaged decisions
• Pre-flight decision making and planning for emergencies
and abnormal situations
• Partial engine failure
46
Decision making biases
Three decision making biases and their relevance to
investigators,
• Confirmation bias
• Attribution bias
• Hindsight bias
Confirmation bias
• The tendency to seek or accept information that confirms
our expectations or assumptions, and to downplay or
discount evidence to the contrary
• Confirmation bias affects how information is perceived
and interpreted
47
Avoiding confirmation bias
• Actively seek disconfirming information
• In an operational setting,
– Encourage questioning and cross referencing, even when
everything seems to be going well
– If something feels wrong, then it probable is
• Investigators are just as susceptible to confirmation bias
as anyone else!
Attribution bias
• Fundamental Attribution Error
– In explaining other peoples’ behaviour, we tend to overestimate
the role of individual characteristics and underestimate the
impact of situations
• The reverse for explaining our own behaviour
• The Fundamental Attribution Error works against the
very basis of our systemic approach to safety
investigation
48
Hindsight bias
• The tendency to see events that have occurred as more
predictable than they in fact were before they took place
• Remember the principle of Local Rationality – what
people were doing made sense to them at the time
Spatial disorientation
49
Spatial disorientation
Collision with terrain

• Twin Squirrel helicopter at Lake Eyre, SA, in August 2011
• VFR night flight, filming for a television documentary
• No moon, no ground lights, no cloud
• Impacted terrain 3 km from departure point
• Pilot and two passengers fatally injured
50
Accident flight path
• Helicopter seen departing to the north-east when meant
to depart to the south
• The pilot probably selected an incorrect destination on
the helicopter's GPS units prior to departure
• One location the crew intended to visit, Cowarie Station,
was listed in the GPS as a waypoint and was in the
direction of the initial departure track
Accident flight path

• Helicopter levelled at 1,500 ft, entered a gentle
right turn, and began descending
• Descent and increasing bank for 38 seconds,
– pilot incapacitation?
– spatial disorientation?
51
52
Incapacitation or disorientation?
• Simulator trials by the US Army Aeromedical
Research Laboratory matched the observed flight
path if continual control adjustments were made
• Controls in fixed position produced different flight
paths
• Sudden and significant incapacitation unlikely
• Pilot probably became spatially disoriented
Factors that may have induced

spatial disorientation
• Limited perceptual cues
• Attention diverted by problem with track
• Abnormal event not expected
• Limited recent instrument flying
• No autopilot or stabilisation system
53
VFR dark night flight
• Visual flight in dark night conditions is effectively
the same as flight in instrument meteorological
conditions
• It may be possible to see for miles, but there
may be nothing to see
The final word
54
Two sides of the same coin
Human as Human as
hazard defence
• Slips • Adjustments
• Lapses • Compensations
• Mistakes • Recoveries
• Violations • Improvisations
Resilience
55
Take-home message
Considering Human Factors issues is an essential
part of any investigation
Thank you
56

Topic 4
Investigation analysis framework

Investigation analysis framework
Richard Batt
Overview
• The importance of investigation analysis
• Terminology
• Investigation analysis model
• Investigation pathway
1
To do a job well...
A good plan The right tools Sufficient resources
Investigation analysis
• Investigation analysis methods and tools
– Evidence tables
– Factor maps
2
Investigation
A process conducted for the purpose of accident prevention
which includes the gathering and analysis of information,
the drawing of conclusions, including the determination of
causes and/or contributing factors and, when appropriate,
the making of safety recommendations.
Annex 13 Definitions
The investigation process

• Initial intensive on-site phase, including gathering
perishable evidence
• ‘Second phase’ activities, including further information
gathering, analysis and report writing
3
• Analysis is an iterative process in which available data is
evaluated and converted into a series of arguments,
which then produce relevant findings
• Without sound analysis, the lessons from accidents will
not be learnt, and safety improvements will not be made
Investigation effort
4
To ensure the best result, investigation analysis requires,
• A structured set of methods and tools
• A team-based approach
• Relevant subject matter expertise
• Investigation analysis is a skill that needs initial and
recurrent training, and plenty of practice
5
terminology
Events and Conditions

• Event
– An event is something that happens at a specific point or points
in time
– An event may be something that didn’t happen - eg the pilot
didn’t lower the landing gear
• Condition
– A condition is something that exists for a period of time
– A condition may refer to something that didn’t exist - eg the
absence of a procedure
6
Contributing Factor
An event or condition that if had not been present then,
• The accident or incident would probably not have
occurred or
• The consequences would probably not have been as
serious or
• Another Contributing Factor would probably not have
occurred or existed
Cause
• The term Contributing Factor is preferred to the term
Cause
• The term Cause is more likely to suggest blame
• Most accidents and incidents are the result of a
combination of a number of factors
7
Probability expressions
Probability expressions
• Indicate relative likelihood, not precise
• Avoid ‘possible’ where possible
• A low likelihood is not the same as a lack of evidence
– If you cannot estimate with confidence then say so
8
model
Investigation analysis model

• The investigation analysis model used by the Australian
Transport Safety Bureau (ATSB) and the Indonesian
National Transportation Safety Committee (NTSC) is
based on the work of Professor James Reason
– Reason J (1997) Managing the Risks of Organisational Accidents
• The model has been adapted by Dr Michael Walker of the
ATSB
– Walker M and Bills K (2007) Analysis, Causality and Proof in
Safety Investigations
9
Investigation analysis model
Production goals
• Examples of production goals
– An operator flying people or cargo from one location to another
– A maintenance organisation conducting maintenance on an
aircraft
10
What happened
• Technical events
– The crankshaft failed
• Individual actions
– The pilot raised the flaps
Individual actions
• Observable behaviours performed by operational
personnel
• If individual actions increase risk, they are sometimes
termed ‘Unsafe acts’ or ‘Active failures’
• The neutral term ‘Individual action’ is preferred
11
Individual actions in context
Individual actions take place in the context of,
• Local conditions
• Risk controls
Local conditions
• Conditions associated with the immediate context or
environment in which individual actions occur
• Sometimes termed ‘Local hazards’ or ‘Local threats’
• Local conditions can increase the likelihood of individual
actions and/or technical events that increase risk
– Fatigue resulting in the pilot missing a checklist item
– High operating temperature leading to a mechanical failure
12
Types of local conditions
• Personal factors
• Knowledge, skills, experience
• Task demands
• Social environment
• Workspace environment
• Physical environment
• Weather conditions
Local conditions
• Personal factors
– Human physical limitations, health issues, fatigue, stress and
anxiety, alcohol and drugs, individual motivation and attitude,
pre-occupation
• Knowledge, skills, experience
– Task knowledge and skills, task experience and recency,
equipment knowledge and skills, language skills
13
Local conditions
• Task demands
– High workload, low workload, task completion pressure, time
pressure, distractions
• Social environment
– Peer pressure, interpersonal conflicts, diffusion of responsibility,
the effects of differences in age, culture, personality
Local conditions
• Workspace environment
– Workspace lighting, noise, temperature, humidity, air quality
• Physical environment
– Light conditions, whiteout, sun glare, runway or taxiway surface,
lack of environmental cues
• Weather conditions
– Visibility, wind, windshear, turbulence, icing conditions
14
Risk controls
• Measures put in place by an organisation to facilitate and
assure safe operations
• Risk controls prevent hazards resulting in losses
• Sometimes termed ‘defences’ or ‘barriers’
Types of risk controls

• Preventative
– Control measures put in place to minimise the likelihood of
undesirable events or conditions
• Recovery
– Control measures but in place to detect, correct, or minimise the
adverse effects of undesirable events or conditions
– ‘Last line of defence’ – warning systems and emergency
equipment/procedures
15
‘Bow-tie’ model
C
O
N
H S
A E
Z Q
Occurrence
A U
R E
D N
C
E
S
Preventive controls Recovery controls
Types of risk controls

• Equipment
• Facilities / infrastructure
• Procedures
• Training and assessment
• People management
16
Risk controls
• Equipment
– Displays and controls, workspace equipment, tools and
materials, detection and warning systems, protection and rescue
systems, automation
• Facilities / infrastructure
– Aerodrome lighting and signage, runway design, navigation aids
Risk controls
• Procedures
– Problems associated with the design, availability or consistency
of procedures, checklists or technical publications
• Training and assessment
– Problems with the design, delivery or availability of training -
includes initial training, recurrent training and training for both
normal and abnormal operations
17
Risk controls
• People management
– Problems with line supervision, rostering and scheduling, crew
pairing, staff selection, monitoring fitness for work
Organisational influences
• Conditions that establish, maintain or otherwise influence
the effectiveness of an organisation’s operational risk
controls
• Two main categories of organisational influences
– Internal influences
– External influences
18
Types of organisational influences
• Safety management processes
• Organisational characteristics
• Regulatory influences
• Safety management processes
– Policies and procedures for hazard identification, risk
assessment, change management, training needs analysis,
equipment selection, internal auditing, collection and analysis of
safety data
• Organisational characteristics
– Organisational structure, skills and experience of key personnel,
allocation of resources to safety processes, internal
communication, policies for organisational learning and
benchmarking
19
• Regulatory influences
– Problems related to regulatory material (Acts, Regulations,
advisory material), compliance monitoring (accreditation, audit,
inspection, intelligence gathering) and enforcement
Occurrence events
• An accident or incident may involve more than one
occurrence event
– Unstable approach followed by a runway overrun
– Bird strike leading to an engine failure
20
Investigation pathway
• A bottom up process
• Keep asking why
21
O
What could have been in place to minimise problems with
the risk controls?
R
Risk controls
What could have been in place to reduce the likelihood or
severity of problems at the operational level?
Why?
Local conditions
L What aspects of the local environment may have influenced

the individual actions or technical events?
Why?
Individual actions Technical events
I What individual actions

increased safety risk?
How did the equipment fail? Why?
Occurrence events
O What events best describe the occurrence? Why?
ORLIO levels of analysis

O Organisational Influence
Operator’s polices and procedures for fatigue risk management
R Risk Control
Flight crew roster
L Local Condition
Flight crew fatigue
I Individual Action
Pilot entered wrong data into the Flight Management System
O Occurrence Event
Breakdown of separation
22
Systemic investigations
• Emphasis on the identification of broader safety
issues
• Go beyond individual actions and local conditions,
and look at risk controls and organisational
influences
• Provide the best opportunity for making significant
improvements to safety
Stop rule
• How far do you go?
• Not all investigations will identify risk controls and
organisational influences as Contributing Factors
• When to stop depends on
– The likely safety benefit of further investigation
– The severity of the occurrence and the resources available
23
Stop rule
• Continue if still identifying Contributing Factors that are
significant and it is reasonable to expect the organisation
to address
• Stop if focussing on factors that no organisation could
reasonable be expected to address
Group exercise
• Based on the Uberlingen mid-air collision
• Working as a group, identify the relevant
– Individual actions
– Local conditions
– Risks controls
– Organisational influences
• At the end of the exercise, one member of each group
will report their results to the assembly
24
Uberlingen mid-air collision
On 1 July 2002 at 21:35, a Bashkirian Airlines Tupolev 154
passenger jet and a DHL Boeing 757 cargo jet collided in
mid-air over Uberlingen, Germany (near Lake Constance),
killing all 71 onboard both aircraft.
Uberlingen mid-air collision

The passenger jet was en route from Moscow, Russia, to
Barcelona, Spain. The cargo jet was flying from Bergamo,
Italy, to Brussels, Belgium.
25
Group exercise
• Review the information provided about the Uberlingen
mid-air collision
• Apply the investigation model
• Complete the coding sheet
26
Thank you
27
Topic 5
Sequence of events
Sequence of events
Presented by
Capt. Nurcahyo ‘Ray’ Utomo
NTSC ‐ Indonesia
Sequence of events analysis
• Definition:
Process of identifying, listing and reviewing
the events before, during, and after the
accident or incident
• The sequence of events should be
summarised in an event list, timeline, or some
other suitable form
• Answers the question “What happened?”
1
Sequence of events analysis
• Structures the data in a form relevant to the
occurrence
• Identifies potential relationships between
events
• Identifies gaps and discrepancies in the data
• Guides subsequent investigation analysis
Constructing a sequence of events
• Review all available data
• Identify key events and changes in situation
• Organise items in a table or other suitable
form
• Identify any features of note
• Review for missing data
2
Review Available Information
• Witness evidence
• Operational documentation
• Flight recorders
• Other recorded data
• Wreckage examination
• Normal sequence of events
Develop Sequence of Events List
• List the events and the time/date when they
occurred (and additional comments).
• Develop the initial list using a timeline chart.
3
Develop Sequence of Events List
• Select relevant point and work forward, or
• Start at final occurrence event and work
backward
Things to consider:
‐ Focus on aircraft performance then other events
‐ Start with more important events
‐ Start with more data sources
‐ Divide source of data with investigation team and
work together
Review for Contributing Factors

• The final output of the sequence of events
analysis should be a list of events which are
potential Contributing Factors
• The main types of events to look for are
occurrence events (including technical problems)
and individual actions which increase safety risk
• Identifying these events provides the platform on
Contributing Factor analysis
4
Sequence of events presentation
Sequence of events can be depicted in a number
of ways,
• Table
• Timeline
• Geographical overlay
• Animation
Sample: SOE table format
• Stickshaker activation, Boeing 717 at
Kalgoorlie, WA, on 13 October 2010
5
Sample: SOE table format
Mid air collision between Boeing 757 and
Tupolev 154 near Uberlingen, Germany, on 1
July 2002.
6
Sample: SOE timeline format
Runway overrun, Boeing 747 at Bangkok,
Thailand, on 23 September 1999.
7
Sample: SOE
timeline format
Sample: SOE geographical overlay
Collision with terrain, Xian MA60 at Kaimana,
Indonesia on 7 May 2011.
8
Case study: Ground collision
A B737 on landing collided with an ATR72 that was
entering the runway for take off.
While the Tower controller issued landing clearance
to the B737, the ATR pilot was communicating with
Ground controller.
The tower controller was handed over to another
controller.
The ATR pilot requested to the tower controller to
enter the runway via Rapid Exit Taxiway (Taxiway
Delta).
9
Case study: Ground collision
The conditional clearance for ATR pilot to enter the
runway consisted of 28 words, delivered in 11
seconds (the rate was about 152 words/minute).
The B 737 pilots understood the clearance. The ATR
pilot read back only the last part of the clearance.
The controller did some other task during the
critical moment.
• Was the time adequate for the B 373 go around
when they saw the ATR entering the runway?
• Time synchronisation several recorders to develop
sequence of events.
SOE table on both cockpits and tower cab
10
11
Sequence of events : Geographical overlay
Sequence of events: Animation
12
Sequence of events
• Developing a sequence of events list, timeline
or chart is essential for every investigation
• Use suitable format
• Start constructing the sequence of events at
the beginning of the investigation
• Review and update it regularly
Group exercise
• Based on an ATSB investigation of landing
below minima by two B737 aircraft at Mildura,
Vic, in June 2013
• Working as a group, identify the key
occurrence events and develop a sequence of
events table
• At the end of the exercise, one member of
each group will report their results to the
assembly
13
Landing below minima
• A Qantas B737 and a Virgin B737 were on
scheduled flights to Adelaide, SA
• Both aircraft diverted to Mildura, Vic, due to
weather
• On arrival at Mildura the weather was
significantly different than forecast, with
visibility reduced in fog
Landing below minima
• The flight crew of the Qantas aircraft
conducted an instrument approach and
landed below minima
• The flight crew of the Virgin aircraft also
conducted an instrument approach and
landed below minima in fog and with fuel
below the fixed reserve
14
Group exercise
• Review the extract from the ATSB report
• Develop a sequence of events table based on
the ‘Occurrence’ section of the report
• Each group will decide
– What table format to use
– What information to include
THANK YOU
KOMITE NASIONAL KESELAMATAN TRANSPORTASI
NATIONAL TRANSPORTATION SAFETY COMMITTEE
Transportation Building 3rd floor

Jl. Medan Merdeka Timur No. 5, Jakarta - 10110 INDONESIA
Telp. (021) 384 7601 ; Fax (021) 351 7606
Mobile phone (24hr): +62 812 126 55 155
Website : http://knkt.dephub.go.id/knkt
E-mail : knkt@dephub.go.id
15

Topic 6
Evidence tables

Evidence Tables
Richard Batt
Outline
• Nature of evidence
• Findings
• Elements of an Evidence table
• Too low on approach example
• Group exercise
1
What is evidence?
Evidence is information with some relevance to an
investigation hypothesis related to a potential Finding or
Contributing Factor
Sources of evidence
• Witness report
– Pilots, controllers, maintainers, cabin crew, ground crew, and
eyewitnesses
• Recorded data
– Aircraft FDR CVR and QAR, ATC data, system data (eg GPS),
photographs/video of event, met data, phone records
• Site measurements/observations
– Wreckage distribution, witness marks on ground or on parts/
components, control positions
2
Sources of evidence
• Test/examination report
– Fuel quality tests, toxicology, functional tests, simulation study
• Technical documentation
– Procedures manuals, training manuals, system descriptions,
maps/charts, regulations/orders
• Operational records
– Log books, technical logs, maintenance records, training
records, occurrence reports, weather observations
Sources of evidence
• Expert and general opinion
– Provided by operational or subject matter experts, or by other
relevant parties
• Audits/assessments
– Internal audits, external audits, risk assessments
• Statistical data
– Review of occurrence database, review of maintenance records,
opinion survey
3
Summarising and presenting evidence
• In written free text
– Can be open to different interpretations
– Can obscure deficiencies in information or logic
• In an evidence table
– Forces a structured approach that lists and evaluates each piece
of evidence and summarises the overall result
Findings
A Finding is a statement about a significant condition,
event, or circumstance in the accident sequence
ICAO Document 9756 Manual of Accident

and Incident Investigation Part IV – Reporting
4
Findings
• Interpret or summarise evidence
• Resolve ambiguity or uncertainty
• Not necessarily causal or indicative of a deficiency
• May relate to a positive individual action, risk control, or
condition that enhanced safety
• May established an intermediate step for a more
important Finding
Findings
• Significant events and factors that were investigated in
detail, but eliminated in analysis, should be stated as a
Finding
– Flight crew fatigue was not a factor in the accident
• Areas of ambiguity should be identified and stated
– The investigation was unable to establish whether the pilot-in-
command or the co-pilot was the pilot flying the aircraft at the
time of the accident
5
Findings
• Findings may be simple or complex
– Although the aircraft was equipped for instrument flight, the pilot
was not qualified for IFR flight
– The civil aviation authority’s safety oversight of the operator’s
procedures and operations was adequate/inadequate
ICAO Doc 9756 IV, Appendix 5 to Chapter 1 - Commonly used Findings
Examples of Findings
• The aircraft were squawking the assigned transponder
codes and there were no identified unserviceabilities with
the transponders of the aircraft
• Controller 2 applied prompt compromised separation
recovery actions after they had identified the loss of
separation
• Based on analysis of the available information, an
airworthiness issue was considered unlikely to be a
contributing factor to the accident
6
Evidence tables
• Use an Evidence Table to determine if a possible Finding
is supported by the available evidence
• Develop a separate Evidence Table for each hypothesis
Evidence tables
For each potential Finding,
• Identify relevant items of evidence
• Evaluate each item of evidence
• Evaluate the overall set of evidence
• Determine if the Finding is supported
7
Relevant items of evidence
Essential to consider all relevant items of evidence,
• Those that support the proposed Finding
• Those that oppose the proposed Finding
• Any other evidence that would normally be considered
relevant
Relevant items of evidence

Essential to consider,
• What items of evidence would be expected for the
proposed Finding but were not seen
• What items were observed but would not be expected
• Any alternatives to the proposed Finding that fit the
pattern of evidence
• Absence of evidence is not evidence of absence
8
Evidence Tables
An Evidence Table can be a simple paper or electronic
document, or incorporated into Safety Investigation
Information Management System (SIIMS) software
SIIMS
9
Elements of an Evidence Table
• Proposed finding
– The hypothesis that is being tested
• Item of evidence
– A brief description of the evidence itself, including the type of
evidence and the source
• Evaluation
– A brief summary of the strengths and weaknesses of the item of
evidence
Evaluating items of evidence

• Is the item of evidence relevant?
• Is the evidence credible? eg reliability, objectivity
• Are similar items of evidence consistent? eg multiple
witnesses
• List the strengths and weaknesses of the evidence
10
• Rating
– Strongly supports
– Supports
– No effect
– Opposes
– Strongly opposes
– Unsure

• Summary of evidence
– A brief summary of the overall strengths and weaknesses and
associations between the items of evidence
• Is proposed Finding supported?
– Yes / No
– Overall result must be consistent with the evaluation of the
individual items of evidence
11
Summary of evidence
Evaluate the overall pattern of evidence,
• Quality and consistency of the evidence
• Independence of the sources of evidence
• Extent to which opposing evidence can be explained
Review the Evidence Table

• Any relevant sources of evidence not considered?
• Be mindful of Confirmation Bias
– Particularly look for disconfirming evidence that could provide an
alternative explanation
12
Developing an Evidence Table
Example of an Evidence Table

Occurrence
– During approach to land the aircraft deviated significantly below
the glideslope as indicated by the Precision Approach Path
Indicator (PAPI) guidance system
13
Fitness for duty
• One area for the investigation to consider was the flight
crew’s fitness for duty
• A preliminary review of the evidence suggested that was
unlikely to be a factor
• Proposed Finding
– It is unlikely that there were any medical or physiological
conditions that impaired the crew’s performance during the
approach and landing
Fitness for duty

– Regulator’s annual medical records indicated no ongoing or
potential concerns
• Evaluation
– Records not always reliable indicators of existing problems, but
do include some medical test results
• Rating Strongly supports
Supports
– Supports No effect
Opposes
Strongly opposes
Unsure
14
Fitness for duty
– Interviews with pilots revealed no indications of ongoing or
recent medical problems likely to influence performance
• Evaluation
– Crews typically unlikely to volunteers such information during
investigations. No overt indications of problems during
interviews.
• Rating
– Supports
Fitness for duty

– Crew reported no concerns about medical fitness of other crew
members
• Evaluation
– Crews typically unlikely to volunteer such information during
investigation interviews
• Rating
– Supports
15
Fitness for duty
– Doctor who interviewed crew two days later was concerned
about pilot in command’s concentration
• Evaluation
– Probably not relevant as problems consistent with the trauma of
the accident and the pilot’s concern about the nature of the
interview. No problems encountered in previous or subsequent
interviews by investigators.
Alternative explanation
• Rating No corroborating evidence
– No effect
Fitness for duty

– Operator arranged for crew to undertake eyesight tests. No
problems identified.
• Evaluation
– Results not actually sighted firsthand. However, no reason to
doubt operator. Only covers one aspect of medical or
physiological conditions, but one which is central to the
occurrence.
• Rating
– Supports
16
Fitness for duty
– Review of CVR and ATC recordings indicated that at the time the
aircraft deviated below the approach path the crew were
distracted by another aircraft on a parallel approach
• Evaluation
– Item of evidence provides credible alternative explanation for
crew’s actions
• Rating
– Supports
Fitness for duty

• Summary of evidence
– No reliable, conflicting information. Enough information to justify
Finding. Other factors were identified to explain crew actions
during the occurrence.
• Is proposed Finding supported?
– Yes
It is unlikely that there were any medical or physiological

conditions that impaired the crew’s performance during the
approach and landing
17
Evidence Tables
• Evidence tables are analysis tools to assist the
investigation team, they are not included in investigation
reports
• Support Findings that will be reflected in the analysis
section of the final report
• Help to ensure that the final report will be focussed and
well organised
18
Investigation review
• All investigation reports should be thoroughly reviewed
by peers and management before being publically
released
• Well documented Evidence Tables clearly demonstrate
to reviewers that the investigation Findings, Conclusions,
and any Recommendations are properly supported by
the evidence
Group exercise
• Working as a group, think of a potential Finding that may
be relevant to an occurrence investigation.
• Complete an Evidence Table that supports the proposed
Finding. The Evidence Table should contain at least
three items of evidence (15 minutes).
• Repeat the exercise for another proposed Finding which
is not supported by the evidence (15 minutes).
19
Group exercise
• Complete two Evidence Tables, one in which the Finding
is supported and one in which the Finding is not
supported
Thank you
20

Topic 7

Richard Batt
Overview
• Definition of Contributing Factor
• Tests for Existence, Influence and Importance
• Example of Contributing Factor analysis
• Group exercise
1
Contributing Factor
occurred or
serious or
occurred or existed
Link-by-link approach
• The operator’s use of a modified checklist
– was a Contributing Factor to
• The pilot in command not arming the autobrake system
– was a Contributing Factor to
• The aircraft overrunning the runway
2
Cause
• The term Contributing Factor is preferred to the term
Cause
• The term Cause is more likely to suggest blame
• Most accidents and incidents are the result of a
combination of a number of factors
Example of a Probable Cause

• The non-instrument rated pilot's improper decision to
continue visual flight into instrument meteorological
conditions, which resulted in spatial disorientation and
subsequent in-flight collision with mountainous terrain.
3
Testing Contributing Factors
• Existence
– Did the Contributing Factor exist?
• Influence
– Did the Contributing Factor have an influence on the
occurrence or another Contributing Factor?
Existence + Influence = Contributing Factor
Importance
• If the factor didn’t have an influence, is it worth
investigating further anyway?
• Is it a potential safety issue?
4
Proposed Contributing Factor
Existence? No End
Yes
Influence? No Importance? No End
Yes Yes
Contributing Factor Potential safety issue
Examples of Contributing Factors

• Both controllers were experiencing a high workload due
to a arrange of factors, including traffic levels, weather
diversions, and the airspace configuration
• The regulatory requirements and advisory material
placed limited emphasis on the potential difficulties of
conducting VFR flight in dark night conditions
• The aerodrome procedures and airspace design did not
assure lateral or vertical separation between traffic flows
5
Examples of Contributing Factors
• The pilot unintentionally entered instrument
meteorological conditions and was unable to maintain
visual reference to the ground
• During cruise, the airspeed decreased to the extent that
control of the aircraft was lost
• Debris originating from the starter failure was not
contained by the starter casing and severed the number
one engine oil scavenge pipe
Testing Contributing Factors

• Similar process to testing the evidence for a proposed
Finding
• Similar use of an Evidence Table
• The tests for Existence and Influence only need to be
verified to the ‘probable’ level
6
Tests for existence
• Direct evidence
– A lack of fuel in an aircraft’s tanks and fuel lines as evidence
of fuel exhaustion
• Symptoms or effects
– The existence of smoke could be an indication of fire
• Sources or reasons
– A lack of sleep, long work hours, or working in the early
hours of the morning may suggest the existence of fatigue
7
Tests for existence
• Predictability
– Information about an aircraft’s initial fuel
load and estimated fuel burn could indicate
fuel exhaustion
• Alternative explanations
– The operator’s slurred speech may have
been due to the effects of fatigue rather
than alcohol
Tests for influence

• Logical connection
– A lack of maintenance lead to corrosion and component failure
• Characteristics of the problem
– Slips and lapses are commonly associated with distractions
• Sufficient magnitude
– Was the amount of glare great enough to significantly affect the
pilot’s visual performance
8
Tests for influence
• Presence of enhancers
– Dehydration and lack of recent nutrition may
exacerbate the influence of fatigue
• Alternative explanations for the problem
– The loss of engine power engine may have
been due to carburettor icing rather than a
mechanical problem
Test of importance
• Is there underlying safety issue that should identified to
improve safety, even though it did not contribute to the
occurrence event?
• Does not require an Evidence Table, but a justification is
still required
9
Tests for importance
• Prior existence
– If the factor has occurred or existed numerous times before, then
the potential future risk is greater
• Scope of future existence
– Does the factor potentially relate to a range of situations, eg
operation types, operators, geographical location etc
• Existing controls
– To what extent are there multiple controls in place to detect and
recover from the factor?
Tests for importance

• A new system or process?
– The introduction of new equipment, procedures or systems can
be a vulnerable time, and there is less known history of safe
operations available
• External interest
– Sometimes a high level of external interest or controversy
associated with a factor may indicate further investigation
10
Example of a Contributing Factor
• Occurrence
– During approach and landing in heavy rain, the aircraft landed
long and fast. Just before touchdown, the crew initiated and
then cancelled a go-around. The crew did not engage reverse
thrust and the aircraft subsequently overran the runway and was
substantially damaged.
Runway overrun
• A preliminary review suggested that the possibility of
pilot fatigue should be further investigated
• Proposed Contributing Factor
– The pilot in command’s performance was probably affected by a
moderate level of fatigue at the time of the occurrence
11
Fatigue - Test of existence
– The pilot’s FAID score at time of the occurrence based on work
duties was 70, which is undesirable for high risk occupations
• Evaluation
– FAID score was based on estimates of his managerial work
duties, so may not be accurate
• Rating Strongly supports
Supports
– Supports No effect
Opposes
Strongly opposes
Unsure

– Pilot reported no problems with sleep or any activities that would
have exacerbated fatigue in recent days or on day prior to flight
• Evaluation
– Pilot may not report all relevant information
• Rating
– No effect
12
– Pilot reported did not feel tired or fatigued during approach
• Evaluation
– Self-reports typically underestimate fatigue levels. Other pilot
reported experiencing some fatigue at time of occurrence.
• Rating
– Unsure

– Time of occurrence was 0215 in pilot’s normal sleep cycle
• Evaluation
– This time is generally associated with higher levels of fatigue
• Rating
– Supports
13
– Pilot had been awake for 21 hours without any sleep periods
• Evaluation
– Had reported 30 minute rest and a two hour rest during flight, but
did not sleep during these periods. Fatigue levels and accident
rates reported to be higher than normal with this number of hours
of wakefulness.
• Rating
– Supports

– No overt indications of fatigue on CVR or reports of other crew
• Evaluation
– Absence of these indicators is not strong evidence that fatigue
did not exist
• Rating
– No effect
14
• Summary
– Strong evidence to say pilot’s fatigue levels were moderate to
high. Pilot reported no problems with sleep in recent days – but
this does not preclude fatigue due to sleep/wake cycle in last 24
hours. Only conflicting evidence was pilot self-report, which are
not generally reliable.
• Existence
– Yes
Fatigue – Test of influence

– Error type (rapid decision to change situation) not known to be
strongly associated/consistent with fatigue.
• Evaluation
– Research in this area is not extensive.
• Rating
– Opposes
15
– There were other factors that could partly explain the captain’s
action.
• Evaluation
– These factors include pilot’s explanation of change in visibility at
the time, and fact that they had touched down.
• Rating
– Unsure

– The pilot was probably in a highly alerted state in the period just
prior to the error due to the weather conditions and the nature of
the final approach.
• Evaluation
– This alerted state had not existed for a long duration – probably
only started as passed over middle marker and hit heavy rain.
Does not rule out fatigue, but makes it less likely.
• Rating
– Opposes
16
• Summary
– No apparent evidence for influence in this case.
• Influence
– No
Fatigue was not a Contributing Factor
Fatigue – Test of importance

• Is proposed factor/issue important?
– Yes
• Justification
– Fatigue levels were undesirable high for person in safety-critical
role. Fatigue levels appear to be partly due to normal operations,
and partly due to pilot's role as management pilot. Operator has
many other management pilots also flying on the line, and who
also probably work long hours. Likely to be an underlying safety
issue involved.
17
Existence? No End
Yes
Yes Yes
18
Contributing Factors
• Important to identify both immediate Contributing Factors
and the deeper or systemic Contributing Factors
• All Contributing Factors are Findings but not all Findings
are Contributing Factors
• Some States list Contributing Factors separately from
Findings while other States indicate in the list of Findings
which were also Contributing Factors
Group exercise
• Working as a group, think of a potential Contributing
Factor that may be relevant to an occurrence
investigation
• Complete a Contributing Factor Evidence Table,
including the results for Existence and Influence, and
Importance if applicable (30 minutes)
19
Thank you
20

Topic 8
ORLIO factor mapping

Richard Batt
Overview
• Why factor mapping?
• ORLIO factor mapping
• Case example
• Group exercise
1
Why factor mapping?
• A picture is worth a thousand words
Why factor mapping?

• An aid to analysis
– Forced to critically examine logical connections
• An aid to communication
– Clearly depicts complex data relationships
Why?
Why?
Why?
Why?
2
• ORLIO factor mapping is based on the AcciMap (accident
map) technique developed by Jans Rasmussen
– Rasmussen J and Svedung I (2000) Proactive risk management
in a dynamic society
O
the risk controls?
R
Risk controls
Why?
Local conditions

Why?

Occurrence events
3
ORLIO levels of analysis
O Organisational Influence
Operator’s polices and procedures for fatigue risk management
R Risk Control
Flight crew roster
L Local Condition
Flight crew fatigue
I Individual Action
Pilot entered wrong data into the Flight Management System
O Occurrence Event
Breakdown of separation
Relationship between ORLIO elements

• The investigation pathway is bottom up
• The influence pathway is top down
Risk controls
Local conditions
Individual actions
Occurrence events
4
• Summarises the occurrence events and conditions, and
the interrelationships between them
– Investigation at a glance
• Depicts the context in which the accident or incident
occurred
– Remember Context is everything
• Highlights the factors that influenced individual actions
– Helps to reinforce a no-blame approach to investigation

• A graphical representation of the events and conditions
that came together to produce an organisational accident
5
• ORLIO levels of analysis are Organisational
influence
depicted from top to bottom

• Events are depicted by squares, Risk control
conditions by ovals
Local
condition
• Different colours can be used for
different level of analysis Individual action
Occurrence event

• Arrows depict paths of influence
Organisational
influence
Risk control Risk control
Local Local
condition condition
Individual action
6
• There can be one or more occurrence events
• The occurrence event sequence is depicted left to right
Occurrence event 1 Occurrence event 2 Occurrence event 3

• Start with the identified events and conditions
– Risk controls
7
• Determine paths of influence
• Factors may have
– more than one input
– more than one output
Organisational Local
Risk control Individual action
influence condition

• Factors may link across more than one level
• But organisational influences cannot link directly to
individual actions
Organisational
Rick control
influence
Local Local
condition condition
X
Individual action Individual action
8
• Work as a team, starting with Post-it notes or a whiteboard
– Flexible approach and easy to revise factor map

• Ensure labels for events and conditions are clear, short,
and specific
 The pilot had not received recurrent HUET training
 Lack of training – Not specific
 The pilot had received initial HUET training in 2014 and

although scheduled for recurrent training in subsequent years
he was on each occasion unable to attend – Too detailed
9
ORIO factor mapping
• No factor should be listed more than once
– If two factors are similar, see if they can be combined into one
more general factor
• Initial map may identify gaps in the influence pathways
that need to be filled in
Summary
• An ORLIO factor map is a useful tool for organising,
analysing and communicating information
• It can illustrate the interplay of systemic factors that lead
to an organisational accident
• And that’s how we improve safety!
10
Group exercise
• Based on a Beech 1900 in-flight loss of control
• Working as a group, identify the relevant
– Risk controls
• Construct an ORLIO map that shows the lines of
influence between the identified events and conditions
In-flight loss of control

• Shortly after takeoff, the flight crew of the Beech 1900
aircraft reported a runaway trim and were unable to
maintain control of the aircraft
• The aircraft impacted terrain and was destroyed
• The two crew and six passengers on board were fatally
injured
11
In-flight loss of control
• The accident flight was the first flight after maintenance
personnel replaced the forward elevator trim cable
Group exercise
• Review the excerpt from the investigation report
• Complete a coding sheet
• Prepare an ORLIO map using the template provided or
by some other means
12
Thank you
13

Topic 9
Safety Action and Recommendations

Safety action and recommendations
Richard Batt
Overview
• Safety Communication
• Proactive Safety Action
• Safety Recommendations
• Monitoring Safety Actions and Safety Recommendations
1
ICAO Guidelines
ICAO Doc 9756
Manual of Aircraft Accident and Incident
Investigation, Part IV – Reporting
Appendix 6 to Chapter 1
Guidelines on the identification, drafting
and follow-up of safety recommendations
Safety communication
• Safety communication can be formal or informal,
and include discussions, briefings, safety advisories,
and formal safety recommendations.
• The form and timing of safety communication is
influenced by the degree of risk associated with the
underlying safety issue.
2
Risk analysis of safety issues
• Apply and document a formal method when deciding
what level of safety communication is required
Proactive Safety Action

• Ideally an investigation report will not include any Safety
Recommendations
• By the time the investigation report is released, any
necessary Safety Action will already have been taken by
those organisations best placed to do so
– Operator
– Manufacturer
– Regulator
– Maintenance organisation
– Air navigation service provider
3
• The actions that organisations take in order to reduce
the risk of a future accident and incident
• In a safe system, organisations will take action to
address any safety deficiencies as soon as they are
identified
• Proactive Safety Action already taken by relevant
organisations should be documented in the investigation
report

• Airspeed indication failure on take-off when pitot covers
not removed before flight
4
• ATSB Preliminary Report acknowledged proactive Safety
Action already taken
– The operator provided a notice to engineers and flight crew
related to the fitment and removal of pitot covers at Brisbane
Airport
– The engineering support provider improved its procedures for
conducting turnarounds, including improved inspection,
documentation, and tool control
– The ground handling service provider provided all employees
with a ‘read and sign’ bulletin emphasising its arrival and
departure inspection procedures
Safety Recommendations
• The investigation authority will need to issue a
Recommendation if is considered that not all appropriate
safety action has/is being taken
• A Recommendation can be issued at anytime, not
necessary to wait until the Final Report
5
• The greatest benefit will be achieved by
Recommendations that address systemic safety issues
• Having too many recommendations can dilute their
overall importance
• Recommendations should be communicated to the
organisations that are best able to take action to mitigate
the risks, and that that have a mandate to take action
with the broadest impact
• One addressee for each Recommendation
– Makes it clear who is responsible for taking safety action
– Assists tracking and evaluating the response to the
Recommendation
6
Addressee for SRGCs
• For Safety Recommendations of Global Concern
(SRGC), the action addressee normally would be the
State civil aviation authority responsible for the
certification and oversight, in part, of the design,
manufacture, maintenance and/or operations of the
aircraft or facilities involved in the occurrence
A good Safety Recommendation should
• Be clearly and directly supported by the investigation
factual information and analysis
• Outline the safety issue that needs to be addressed, not
prescribe a solution
– The organisation that bears the risk is best placed to decide on
suitable safety action to mitigate the risk
– The accident investigation authority should not become part of
the solution that could possibly have unintended consequences
7
A good Safety Recommendation should
• Be written in clear and concise language
• Avoid using words which can easily result in no change
such as ‘review’ or ‘re-examine’
Mandated response
• ICAO Annex 13 (paragraph 6.10) requires that a State
that receives a Safety Recommendation shall within 90
days inform the proposing State of the preventive action
taken or under consideration, or the reasons why no
action will be taken
8
Monitoring progress
Annex 13 recommends that,
• The State issuing a Safety Recommendation should
record the response received (paragraph 6.11)
• The State receiving a Safety Recommendation should
monitor what action is taken (paragraph 6.12)
Summary
• Early and ongoing communication between the
independent investigation authority and all relevant
organisations is crucial
• Proactive Safety Action is preferred
• Recommendations are a last resort but sometimes
necessary
• Apply the principles for a good Safety Recommendation
9
Thank you
10

Topic 10
Summary

Investigation Analysis Summary
Richard Batt

aspects
1
Proactive safety
• Having and heeding good safety data is the key to
improving aviation safety
• Sources of safety data,
– Investigating accidents
– Investigating serious incidents
– Normal operations monitoring
Technical and non-technical aspects

• Critical to safe operations
• Essential for a thorough safety investigation
2
Aviation safety investigation
• The crucial first step in any aviation safety investigation
is a thorough operational and technical investigation to
establish what happened in the accident or incident, but
it is typically only by then looking at human factors - at
both the individual and organisational level - that we can
understand how and why the accident or incident
occurred.
Human error
Human error is a natural part of life.
A key principle is that human error, although undesirable, is

nevertheless both frequent and widespread.
3
Human error
We can’t eliminate human error, but we can,

understand
minimise
and control it.
Human Factors
• A selection of topics and case studies
– Human error
– Attention
– Workload
– Fatigue
– Stress
– Automation
– Decision making
– Spatial disorientation
4
Human Factors
Considering Human Factors issues is an essential
part of any investigation
Individual actions in context

Individual actions take place in the context of,
• Local conditions
• Risk controls
5
O
the risk controls?
R
Risk controls
Why?
Local conditions

Why?

Occurrence events
• Investigation analysis methods and tools
– Evidence tables
– Factor maps
6
Sequence of events
Sequence of events can be depicted in a number of ways
• Table
• Timeline
• Geographical overlay
• Animation
Evidence tables
• Use an Evidence Table to determine if a possible Finding
is supported by the available evidence
• Develop a separate Evidence Table for each hypothesis
7
Contributing Factor
occurred or
serious or
occurred or existed
Existence? No End
Yes
Yes Yes
8
• Arrows depict paths of influence
Organisational
influence
Risk control Risk control
Local Local
condition condition
Individual action
Safety Actions and Recommendations

• Early and ongoing communication between the
independent investigation authority and all relevant
organisations is crucial
• Proactive Safety Action is preferred
• Recommendations are a last resort but sometimes
necessary
• Apply the principles for a good Safety Recommendation
9
Summary
• Use a structured method
– Based on sound theory
– With effective tools
• Work as a team
• Train and practice to maintain skill levels
Thank you
10

APAC AIG Investigation Analysis Workshop Manual

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

APAC AIG Investigation Analysis Workshop Manual

Uploaded by

Copyright:

Available Formats

Capt. Nurcahyo ‘Ray’ Utomo

What does ‘Independent’ mean?

ICAO Secure Portal

ICAO Doc 9756 Part 4 - Reporting

The role of an investigation authority

Examples of serious incidents

ATSB Investigation AO-2010-081

ATSB Investigation AO-2011-144

Airspeed indication failure

Normal operations monitoring

Normal Operations Safety Study (NOSS)

Technical and non-technical aspects

The modern approach to safety

Human error in context

Multiple Contributing Factors

CFIT accident Commercial

The modern approach to safety

• Why did it happen?

• What can we do about it?

How and why it

What is Human Factors?

Physiological Factors Fatigue

Human error is a natural part of life.

A key principle is that human error, although undesirable, is

What are you likely to forget to do?

The same situations keep on provoking the same kinds

Myths about human error

Myths about human error

From Pilot Error to Human Error

Human error is not the conclusion of a safety investigation,

Violation + Error = Potential disaster

Sabotage System-induced violations

The culpability continuum

Could a different person (well motivated, equally

If ‘Yes’ the person who made the error is probably

Hence, the focus of error management is on,

You can swat them one by one,

Drain the swamp in which they breed

Therefore, need to foster a Just Culture in which people

Developing a Just Culture

• A ‘No blame’ culture is neither feasible nor desirable.

• Some unsafe acts deserve sanctions.

For those that run or

Human error - Summary

We can’t eliminate human error, but we can,

Data entry errors

Data entry error

Future risk controls

• Attentional tunnelling / fixation

The multitasking myth

Yerkes-Dodson Inverted U Curve

Effect of fatigue on performance

FRMS – Fatigue Risk Management System

ICAO fatigue management manuals

Biomathematical fatigue models

Inflight loss of control

Possible causes of stress

Stress due to personal and work factors can interact.

• Distraction (environment / life stress)

Automation as a poor team player,

Changed go-around procedure

Airbus recommendations for go-around

Human information processing

Decision making in practice