Safety Performance

Indicators
Stefano PROLA
Manager Safety and Flight Ops EUR IATA

Overview
 SMS > why SPIs
 Measuring Safety
 Examples

SMS Basics  Safety is dependent on this “System”     working SMS requires organizations to make data-based decisions and actions Dependent on the collection and analysis of correct information Must understand the system as a whole and the risks associated with it Shift of responsibility for safety risk mitigation Proactive safety: from theory to practice 3 29 October 2015 .

Fundamental Shift  Reactive to proactive management  Prescriptive to performance based management and oversight Proactive safety: from theory to practice 4 29 October 2015 .

Shift to Performance Based Management  New requirements  New roles  New needs Proactive safety: from theory to practice 5 29 October 2015 .

Performance Based Management  In context of operation • Domestic / International / Regional • Wide-body / Narrow-body • Bulk load / Container load • Cargo / Pax / Combi • Risk tolerance of the organization  Complexity of the operation  An alternative means to prescriptive compliance Proactive safety: from theory to practice 6 29 October 2015 .

Why?  Proactive management  Technology improvements  Vast majority of accidents / incidents related to technology / human interface  Improvements in the understanding of human factors  Drill into causal factors Proactive safety: from theory to practice 7 29 October 2015 .

system must continue to evolve Proactive safety: from theory to practice 8 29 October 2015 .SMS purpose  “Safety Management processes identify hazards with the potential to adversely affect safety” ~ ICAO SMM  Purpose is to improve safety performance  Also requirement for continuous improvement.

2 – Safety accountabilities 1.5 – SMS documentation Safety Risk Management 2.ICAO SMS Framework .1 – Hazard identification 2.2 – Risk assessment and mitigation Proactive safety: from theory to practice 9 29 October 2015 .3 – Appointment of key safety personnel 1.1 – Management commitment and responsibility 1.4 – Coordination of emergency response planning 1.Annex 19 Safety Policy and Objectives 1.

2 – The management of change 3.ICAO SMS Framework – Annex 19 Safety Assurance 3.3 – Continuous improvement of the SMS Safety Promotion 4.2 – Safety communication Proactive safety: from theory to practice 10 29 October 2015 .1 – Safety performance monitoring and measurement 3.1 – Training and education 4.

.

Measuring Safety Performance “You cannot manage what you don’t measure” How do you measure safety performance? .

related to. or in direct support of the operation of aircraft.ICAO  Safety Performance ~ a State’s or Service Provider’s safety achievement as defined by its safety performance targets and safety performance indicators .ICAO SMM Ed3 Effective 2013 Proactive safety: from theory to practice 13 29 October 2015 . are reduced and controlled to an acceptable level .Definitions  Safety ~ the state in which risks associated with aviation activities.

Definitions  Safety Performance Indicator ~ a data based safety parameter used for monitoring and assessing safety performance .ICAO SMM  Performance Measures ~ Metrics (or values) that are set as a target (usually a number or rate) in order to measure the level of operational performance being achieved .IATA Reference Manual for Audit Programs (IRM) Proactive safety: from theory to practice 14 29 October 2015 .

2. [SMS] (GM)  1 September 2016 “should” > “shall” Proactive safety: from theory to practice 15 29 October 2015 .IOSA Reference  ORG 3.1 The Operator should have processes for setting performance measures as a means to monitor the operational safety performance of the organization and to validate the effectiveness of safety risk controls.

Performance Measurement and Monitoring  Dependent on all SMS framework requirements met  Managing operational risks  Maintaining regulatory compliance Proactive safety: from theory to practice 16 29 October 2015 .

Measuring Safety Performance  Set measureable (SMART) safety objectives  Track performance  Compare against targets  Achievement of a target > improvement in performance Proactive safety: from theory to practice 17 29 October 2015 .

landing tail strikes.g. aircraft component/equipment failures) Cabin operations (inadvertent slide deployments) Ground handling (aircraft damages due to vehicles or equipment) Proactive safety: from theory to practice 18 29 October 2015 .IOSA examples  Flight operations (e.g.. fuel diversions due to fuel) Engineering and maintenance (in-flight engine shutdowns.. unsatisfactory     line or training evaluations) Operational control (e.

SMS 3rd phase Doc 9859 SMM “Develop high-consequence SPIs and associated targets and alert settings.”  Typically focused on number of serious accidents / incidents  High profile  Easy to measure  Reactive • Does not expose systemic issues or hazards Proactive safety: from theory to practice 19 29 October 2015 .

Proactive Shift  SPIs measure performance of safety controls  Shift focus to precursors of serious events Proactive safety: from theory to practice 20 29 October 2015 .

SMS 4th phase Doc 9859 SMM “Develop lower-consequence SPIs and associated targets/alert settings”  Enhance the safety data collection and processing system to include lower-consequence events. Proactive safety: from theory to practice 21 29 October 2015 .  Establish lower-consequence safety/quality indicators with target/alert level monitoring as appropriate.  Reach an agreement with the State oversight authority on lower-consequence safety performance.

Data  Must measure leading and lagging indicators  Leading indicator ~ Metrics that provide information on the current situation that may affect future performance  Lagging Indicator ~ Metrics that measure safety events that have already occurred including those unwanted safety events you are trying to prevent  Established for any operational or maintenance area Proactive safety: from theory to practice 22 29 October 2015 .

Example ~ Long Landing (leading)  Identify touchdown points of ALL flights Proactive safety: from theory to practice 23 29 October 2015 .

Example – Unstable approach paradox Extract from Data Report for Evidence-Based Training “While airline Standard Operating Procedures (SOP’s) mandate a go-around if an approach is unstable. 90% uneventful)  Missed approaches are rarely handled well Proactive safety: from theory to practice 24 29 October 2015 .”  Landings that follow an unstable approach are usually uneventful (98% land. data indicates that landing from an unstable approach may be less risky.

Summary  SPIs not driven to simply get a “good score”  Must improve overall safety performance  Process approach focusing on inputs and their direct impact on output(s)  Results to be regularly reviewed and communicated  Can be established for virtually any operational or maintenance area  Must be challenging and result in enhancing the SMS as a whole Proactive safety: from theory to practice 25 29 October 2015 .

IATA Activity  Creation of SPI Guidance Material • Sample SPIs • Supporting methodology  Assist organizations in developing the means to verify their safety performance against meaningful safety objectives  Facilitate understanding of the need to look beyond “events” Proactive safety: from theory to practice 26 29 October 2015 .

Thank you! Proactive safety: from theory to practice 27 29 October 2015 .