Process Safety Management Boot Camp Training

Oil & Gas Skills (OGS)

DNV GL © SAFER, SMARTER, GREENER

Module 23
Element 18 - Measurement and metrics

2 DNV GL © SAFER, SMARTER, GREENER

Measurement and metrics – what is it

 This element addresses which indicators to consider, how often to collect data, and what to do
with the information to help ensure a responsive and effective functioning of the PSM system
operation

3 DNV GL ©
Why measure?

4 DNV GL ©
Objectives of measuring

 KPIs allow to monitor the real-time performance of management system activities rather than
wait for accidents to happen or for infrequent audits to identify latent management system
failures
 Such performance monitoring allows problems to be identified and corrective actions to be
taken before a serious incident occurs
 In addition, depending on the type of KPI, it allows for benchmarking with sister companies or
with peers in the industry

5 DNV GL ©
Texas City Refinery - PSM deficiencies

 No process safety performance metrics:

Leadership & Commitment
Policy & Strategic Objectives
Organisation, Resources & Competence
Risk Evaluation & Management
Planning Standards & Procedures
Implementation & Monitoring
 Eight (8) previous incidents (serious releases of
Audit
– Executive management, refining management and
supervisory reporting line had no process safety
performance measures
– Performance contracts had no process safety
metrics included
– Use of inadequate methods to measure safety
conditions. The BP HSE MS (GHSER) did not
require explicit use of process safety performance
indicators
hydrocarbon from unit) not investigated

Management Review

6 DNV GL ©
Leading and lagging indicators

7 DNV GL ©
Leading vs. lagging

Leading
metrics or Lagging
indicators metrics or
indicators

8 DNV GL ©
Leading and lagging metrics / indicators
Ref. IOGP (International Association of Oil & Gas
Producers), ”Process Safety – Recommended Practice on
Key Performance Indicators”, Report. No. 456, November
2016

9 DNV GL ©
Definitions

 Leading metrics
– A forward looking set of metrics which indicate the performance of the key work processes,
operating discipline, or layers of protection that prevent incidents
– Leading indicators measure the inputs to the PSM system; these indicators “lead” process
safety events
– The basic premise is that proper inputs will lead to improved outputs – leading indicators
therefore provide a picture of future PSM performance
– Relevant to prevention barriers
– Can be used to monitor the strength of the barrier – by measuring the company’s performance
in maintaining robust risk controls

10 DNV GL ©
Definitions

 Lagging metrics
– A retrospective set of metrics that are measurements of consequence or results
– Lagging indicators show when a desired safety outcome has failed, or has not been achieved
– Lagging indicators are indicative of performance improvement and are the ultimate measure of
achievement by which stakeholders determine the organization’s level of success
– Relevant to mitigation/escalation barriers
– May be used to measure the number and size of holes in the barriers to assess the extent of
weaknesses, defects or failures in the risk control system

11 DNV GL ©
Group exercise –
Leading and lagging indicators

 Provide examples of leading and lagging process safety performance metrics

12 DNV GL ©
Example lagging metrics (OSHA)

 Injury and/or incident reports related to process safety:

– Near miss incidents reported that did or could have led to a loss of containment
– Recordable injuries and first-aid incidents due to loss of primary containment
– Number of incidents vs. number of incidents with formal reports
– Status of incident investigations

 Loss of primary containment:

– Number of incidents
– Severity of incident
– Whether there was primary or secondary containment
– Cause and location of the incident

Ref. OSHA Factsheet “The Use of Metrics in Process Safety Management (PSM) Facilities”

13 DNV GL ©
Example leading metrics (OSHA) (1)

 Management of change (MOC):  Process hazard analysis (PHA):

– Overdue MOCs – PHA actions open
– Approved MOCs – PHAs overdue
– Open MOCs – PHAs completed
– MOCs performed per period (e.g. each month) – Scheduled vs. completed PHAs
– Status of PHA/incident recommendations
 Preventative maintenance (PM): – Status of scheduled PHA revalidations
– Completion rates
– Open items
– Overdue safety critical PMs
– Number of inspections

Ref. OSHA Factsheet “The Use of Metrics in Process Safety Management (PSM) Facilities”

14 DNV GL ©
Example leading metrics (OSHA) (2)

 Mechanical Integrity (MI):  Safety action item. E.g. follow-up on PHA, incident
– Number of inspections scheduled (relief valve, investigation, MOC, or compliance audit
piping, pressure vessel, storage tank) recommendations. It may also include planned
inspections, tests, maintenance activities, training,
– Status of variance requests
or other safety-related activities
– Routine inspections
– Past action items
– Number of overdue work orders
– Initiated vs. completed items
 Training:
– Open action items
– Safety/refresher training completed
– Training planned vs. completed
– Training exceptions
– Contractor training.

Ref. OSHA Factsheet “The Use of Metrics in Process Safety Management (PSM) Facilities”

15 DNV GL ©
Best industry practice

16 DNV GL ©
Best practices

 UK Health and Safety Executive guidebook HSG 254 “Developing Process Safety Indicators” is a
good introduction. This describes not only examples of the indicators, but also a mechanism for
identifying what indicators are suitable for the organization in question:
http://www.hse.gov.uk/pubns/priced/hsg254.pdf
 API RP 754 “Process Safety Performance Indicators for the Refining and Petrochemical
Industries”. This standard was developed using an American National Standards Institute (ANSI)
multi-stakeholder process and was based on preliminary guidance on metrics from both API and
the CCPS
 International Association of Oil and Gas Producers (IOGP), “Process Safety – Recommended
Practice on Key Performance Indicators”, Report No. 456, November 2011. Supports RP 754’s
applicability for upstream activities and to recommend this standard to E&P companies worldwide
 CCPS “Process Safety Metrics - Guide for Selecting Leading and Lagging Indicators”, latest
revision April 2018. Building on API RP 754. Can be downloaded at:
http://www.aiche.org/ccps/knowledgebase/measurement.aspx

17 DNV GL ©
Four tier approach – OGP 456 (based on API RP 754)

 Tier approach – mirrors the safety triangle

 Tiers 1 and 2 are more lagging and cover asset

integrity (LOPC) major and less severe incidents –
referred to as Process Safety Event (PSE)
 Tiers 3 and 4 mostly provide leading measures. The
indicators are intended to be much more specific to a
company’s own management system and often will
be specific to a particular activity or to an individual
asset, facility or plant

18 DNV GL ©
Tier 1 – OGP 456 (based on API RP 754)

 A Tier 1 PSE is a loss of primary containment (LOPC) with the greatest consequence
 A Tier 1 PSE is an unplanned or uncontrolled release of any material, including non-toxic and non-flammable
materials (e.g. steam, hot condensate, nitrogen, compressed CO2 or compressed air), from a process that
results in one or more of the consequences listed below:
– An employee, contractor or subcontractor ‘days away from work’ injury and/or fatality
– A hospital admission and/or fatality of a third-party
– An officially declared community evacuation or community shelter-in-place
– A fire or explosion resulting in greater than or equal to $100,000 of direct cost to the Company
– A pressure relief device (PRD) discharge to atmosphere whether directly or via a downstream destructive
device that results in (1) liquid carryover, (2) discharge to a potentially unsafe location, (3) an onsite
shelter-in-place, or (4) public protective measures (e.g., road closure), as well as a PRD discharge quantity
greater than the threshold quantities in Appendix B in any one-hour
– A release of material greater than the threshold quantities described in Appendix B in any one-hour period

19 DNV GL ©
Tier 2 – OGP 456 (based on API RP 754)

 A Tier 2 Process Safety Event (PSE) is an LOPC with lesser consequence

 A Tier 2 PSE is an unplanned or uncontrolled release of any material, including non-toxic and non-flammable
materials (e.g. steam, hot condensate, nitrogen, compressed CO2 or compressed air), from a process that
results in one or more of the consequences listed below and is not reported in Tier 1:
– An employee, contractor or subcontractor recordable injury
– A fire or explosion resulting in greater than or equal to $2,500 of direct cost to the Company
– A pressure relief device (PRD) discharge to atmosphere whether directly or via a downstream destructive
device that results in (1) liquid carryover, (2) discharge to a potentially unsafe location, (3) an onsite
shelter-in-place, or (4) public protective measures (e.g., road closure) and a PRD discharge quantity greater
than the threshold quantities in Appendix B in any one-hour period
– A release of material greater than the threshold quantities described in Appendix B in any one-hour period

20 DNV GL ©
Tier 1 and Tier 2 KPIs

 The Tier 1 and Tier 2 KPIs count LOPC incidents that are reportable as PSEs, i.e. the incident
results in any of the consequences stated in RP 754 (and repeated in OGP 456)
 These KPIs can be compared and shared across industry, and used for benchmarking with peers –
providing Tier 1 and Tier 2 definitions a re strictly adhered to
 In this case normalization may be required. IOGP suggests to adopt a Tier 1 and 2 Process Safety
Event Rate (PSER):
" "
𝑇𝑖𝑒𝑟 𝑃𝑆𝐸𝑅 1 = " "
𝑥 1,000,000
" "
𝑇𝑖𝑒𝑟 𝑃𝑆𝐸𝑅 2 = " "
𝑥 1,000,000

21 DNV GL ©
Tier 3 – OGP 456 (based on API RP 754)

 A Tier 3 indicator records an operational situation, typically considered a near miss, which has
challenged the safety system by progressing through one or more barrier weaknesses to result in
an event or condition with
– Consequences that do not meet the criteria for a reportable Tier 1 or Tier 2 event
– No actual consequences, but the recognition that, in other circumstances, further barriers could
have been breached and a Tier 1 or Tier 2 event could have happened
 Tier 3 indicators are typically based on lagging outcomes, i.e. retrospective recording of
undesirable events or findings

22 DNV GL ©
Example Tier 3 KPIs

 Types of KPIs implemented at Tier 3 could include numerical data or other parameters related to:
– Demands on safety systems, e.g. pressure relief devices (see example)
– Safe operating limit excursions (see next slide)
– Primary containment inspection or testing results outside acceptable limits
– LOPC below Tier 2 thresholds
– Near misses with potential for LOPC
– Asset integrity/process safety audit findings indicating barrier weaknesses
– Non-compliances with asset integrity or process safety voluntary standards or legislation

23 DNV GL ©
Example Tier 3 KPI – Safe Operating Limit

24 DNV GL ©
Tier 4 – OGP 456 (based on API RP 754)

 A Tier 4 KPI represents performance of the individual risk control barriers, or its components,
within a facility’s management system, and operating discipline
 These KPIs are typically more leading and pro-active because they reflect activities of the
company directly associated with maintaining and improving its risk control barriers

25 DNV GL ©
Example Tier 4 KPIs

 Measures can be focused on barriers such as:

– Engineering and inherently safe design
– Equipment maintenance, inspection and testing
– Process hazard and major incident risk assessments
– Quality of, and adherence to, operating procedures
– Facility management of change
– Contractor capability and management
– Audit improvement actions
– Asset integrity and process safety initiatives
– Workforce and management training and development
– Technical competence assessment and assurance

26 DNV GL ©
Approach for selection and review of KPIs – OGP 456

 Selecting effective indicators is a challenge,

particularly leading Tier 3 and 4 KPIs which aim to
pro-actively improve process safety at facility level
 This requires companies and their facilities to
develop knowledge and understanding of the most
critical risk control barriers, whether the barriers are
facility-specific or apply to groups of similar facilities
or even apply across the whole company
 OGP 456 recommends a six step approach, starting
developing Tier 1 and Tier 2 KPIs first

27 DNV GL ©
Use of dashboards in a larger organization – OGP 456

For larger organisations, it

will be appropriate to
communicate the data to
various levels of management
by consolidating data in
different views and degrees
of detail

28 DNV GL ©
References on process safety metrics

 OGP (International Association of Oil & Gas Producers), ”Process Safety – Recommended Practice
on Key Performance Indicators”, Report. No. 456, November 2011
 Health & Safety Executive, “Developing process safety indicators - a step-by-step guide for
chemical and major hazard industries”, HSG 254, first edition, 2006
 ANSI/API RP 754, Process Safety Performance Indicators for the Refining and Petrochemical
Industries, Second Edition
 CCPS, “Guidelines for Process Safety Metrics”, ISBN: 978-0-470-57212-2, October 2009
 CCPS, “Process Safety Leading Indicators Industry Survey”,
http://www.aiche.org/sites/default/files/docs/pages/leading-indicator-survey_0.pdf

29 DNV GL ©
www.dnvgl.com

The trademarks DNV GL®, DNV®, the Horizon Graphic and Det Norske Veritas®
SAFER, SMARTER, GREENER are the properties of companies in the Det Norske Veritas group. All rights reserved.

30 DNV GL ©