The

T in the Three-Legged Stool of HPRCT

Steven S Prevette
Senior Quality Engineer, ASQ Fellow and Certified Quality Engineer

Human Performance, Root Cause, and Trending Conference

June 20, 2019
Content – Three Parts

1. The Three-Legged Stool, HP, RC, and T

2. Introduction to Statistical Process Control as a Trending Tool
3. SPC Dashboards – A way to show overall performance and
trends across organizations and topics

Keep going for information

on SRNL family colors.

2
Introduction

• I have spoken to small groups at HPRCT in 2014 and 2015

• This presentation reflects work through Fluor at
– Fernald Ohio
– Portsmouth Ohio
– Paducah KY
– Hanford WA
– Savannah River Site (Aiken SC)
– Savannah River National Laboratory
– Canadian Nuclear LaboratoriesKeep
(Chalk
going River, ON)
for information
on SRNL family colors.

3
Part 1 - Three Legged Stool

Root Trending
Cause Keep going for information
Human
on SRNL family colors.
Performance

4
Part 1 - Three Legged Stool – Looking Backwards

AN EVENT
HAS
HAPPENED

Root Trending
Cause Keep going for information
Human
on SRNL family colors.
Performance

5
Part 1 - Three Legged Stool – Looking Backwards

AN EVENT
HAS
HAPPENED

What
Happened?
Why? Root Trending
Cause Keep going for information
Human
on SRNL family colors.
Performance

6
Part 1 - Three Legged Stool – Looking Backwards

AN EVENT
HAS
HAPPENED

What
Happened?
Why? Root Trending
Cause Keep going for information
Human
on SRNL family colors.
Performance

Why Did Somebody do

That?

7
Part 1 - Three Legged Stool – Looking Backwards

AN EVENT
HAS
HAPPENED

Has it
Happened
Before?
What
Happened?
Why? Root Trending
Cause Keep going for information
Human
on SRNL family colors.
Performance

Why Did Somebody do

That?

8
Part 1 - Three Legged Stool – Looking Backwards

What to do to
AN EVENT Correct and
HAS Prevent
HAPPENED Recurrence?

Has it
Happened
Before?
What
Happened?
Why? Root Trending
Cause Keep going for information
Human
on SRNL family colors.
Performance

Why Did Somebody do

That?

9
Part 1 - A Quote From Dr. Deming

Dr. Deming: Any decision that management makes, that anybody makes for himself or for
other people, is prediction. The simplest plan is prediction, with a chance to be wrong. How
may I get home tonight? I predict that my automobile will start and run, or that the bus will
come, or that the train will come. I make plans. Those plans are predictions. Management is
prediction; our lives are prediction. We predict what will happen. We try to choose a course
of action that will react in favor of us. That's our aim. We predict the consequence of actions.

https://www.industryweek.com/quality/dr-deming-management-today-does-not-know-what-its-job-part-2

Or, more simply, “The Job of Management is Prediction”

Keep going for information

on SRNL family colors.

10
Part 1 - Three Legged Stool – Looking Forwards

What to do to
WHAT EVENT(S) Prevent New
CAN HAPPEN? Events?
What is Risk?

Prediction from
Past
Performance
Pre-Mortem
& Near
Misses Root Trending
Cause Keep going for information
Human
on SRNL family colors.
Performance

Observations of Work

11
Part 1 - Trending – Looking Forward

"Past performance is no guarantee of future results”

“Go where the puck will be, not where it is.” (Wayne Gretzky)
When analyzing past returns, it's best to generally ignore returns from the past 1-3 years and
focus on 10-year returns or even longer. Those longer returns are more indicative of the
stability and strength of what you are investing in.
https://www.thebalance.com/past-performance-is-no-guarantee-of-future-results-357862

Good investing advice, eh?

Also good in trending operational data!
“Stability and Strength” – How can
Keepwe “see”
going that?
for information
on SRNL family colors.

12
Part 1 - Trending – Looking Forward – Variation in Results

Dr. Shewhart realized: What we need to know if a system is

stable, and if it is stable, it is predictable. It follows “Common
Cause” variation. We can then evaluate expected performance
against goals, specs, acceptable risk, etc.

We also need to know if we have a change occurring, a “trend”

which was from outside of the system. It is “Special Cause”
variation. We need to ask “What Happened”?

Dr. Deming – 94% of causes areKeep

Common Cause, from within the
going for information
on SRNL family colors.
system, and the system is controlled by management.
https://blog.deming.org/2012/10/knowledge-of-variation/

13
Part 1 - Trending – Looking Forward – The Stable System vs. an Unstable System

System Status Management Decision

Stable: Good Bad

Trend: Good Bad
Corrective Actions on the most recent event is useful IF there is a
Trend – SPECIAL CAUSE

Corrective Actions on the most recent event is NOT useful if the

event is the result of a stable process – COMMON CAUSE

Corrective Actions on the most recent result that came from a

STABLE SYSTEM can make the situation WORSE
Keep going !for information
https://www.youtube.com/watch?v=fiJeRHQ-q7w
on SRNL family colors.

14
Part 1 - Trending – Looking Forward - Two Errors
• Actions taken to improve a process are different, depending on whether or not the
process is “stable”.

• Attempting to explain or correct for individual datum point changes in a stable

process will not improve performance. Knee-jerk reactions.
Will make a mountain out of a molehill.

• Missing the initial indication of a trend. Missing the opportunity to determine the
cause of the trend.
Will cause the molehill to grow into a mountain.

National Park Service

15
Part 1 - Trending – Looking Forward – The Stable System

BUT – stability and goodness are two independent issues.

I can be stable and predictable in having Occupational Deaths.

Hopefully we can agree “this is not good”

Two States:
Stable and predictably wrong
Not stable and heading in the wrong direction
Keep going for information
The actions needed to improve from these
on SRNL two states
family colors. are very
different

16
Concluding Part 1

We need an objective, repeatable method to:

Separate Signal from Noise
Separate Stable from Trend
One of these things is not like the other (link)
Alarm functions such as a Smoke Detector

Minimizing False Alarms

such as declaration that three points above average is a trend
Keep going for information
While Minimizing Failures to Detect
on SRNL family colors.

such as the launch decision for the Challenger

17
Part 2 – Statistical Process Control

One method for trending, which is objective, repeatable, visual and

easily understandable to the reader is:
Statistical Process Control

Developed by Dr. Shewart in 1930

Popularized by Dr. Deming and others
Still applicable today (just “google” it)
U.S. News and World Report included SPC
Keep going for information
on SRNL family colors.

18
Part 2 – Statistical Process Control – Lies, Damned Lies and Statistics

Injuries per Month

20

15

10

0
May-03

May-04

May-05
Mar-04
Mar-03

Nov-03

Nov-04

Mar-05
Jul-03

Jul-04
Jan-03

Jan-04

Jan-05
Sep-04
Sep-03

Keep going for information

on SRNL family colors.

19
Part 2 – Statistical Process Control – Lies, Damned Lies and Statistics

Injuries per Month Injuries per Month

12 month moving average
20 20

15 15

10 10

5 5

0 0
May-03

May-04

May-05
Mar-04
Mar-03

Nov-03

Nov-04

Mar-05
Jul-03

Jul-04
Jan-03

Jan-04

Jan-05
Sep-04
Sep-03

May-04

May-05
May-03
Mar-03

Nov-04

Mar-05
Nov-03

Mar-04
Jan-03

Jul-03

Jan-04

Jul-04

Jan-05
Sep-03

Sep-04
Keep going for information
on SRNL family colors.

20
Part 2 – Statistical Process Control – Lies, Damned Lies and Statistics

Injuries per Month Injuries per Month

12 month moving average
20 20

15 15

10 10

5 5

0 0
May-03

May-04

May-05
Mar-04
Mar-03

Nov-03

Nov-04

Mar-05
Jul-03

Jul-04
Jan-03

Jan-04

Jan-05
Sep-04
Sep-03

May-04

May-05
May-03
Mar-03

Nov-04

Mar-05
Nov-03

Mar-04
Jan-03

Jul-03

Jan-04

Jul-04

Jan-05
Sep-03

Sep-04
Injuries per Month - by the Color

14
12
10
8
6
4
2
0 Keep going for information
on SRNL family colors.

May-05
Mar-05
Jan-05

Apr-05
Feb-05

So, where are we, really?

What is our Prediction for future performance?

21
Part 2 – Statistical Process Control - Example

Injuries per Month - as a Control Chart

25
20
15
10
5
0
May-03

May-04

May-05
Nov-03

Mar-04

Mar-05
Mar-03

Nov-04
Jan-03

Jul-03

Jan-04

Jul-04

Jan-05
Sep-04
Sep-03

Keep going for information

The SPC “control chart”onallows us colors.
SRNL family to see that this
is a stable, but random process
Yes, Really – it was random numbers

22
Part 2 – Statistical Process Control – Predicting the Future

Injuries per Month - as a Control Chart

25
20
15
10
5
0
May-03

May-04

May-05
Nov-03

Nov-04

Mar-05
Mar-03

Mar-04
Jan-03

Jul-03

Jan-04

Jul-04

Jan-05
Sep-03

Sep-04
Keep going for information
Red Arrow = Worst Case on SRNL family colors.
Black Arrow = Central Tendency
Green Arrow = Best Case

23
Part 2 – Statistical Process Control – Actual example at a Facility

Keep going for information

on SRNL family colors.

24
Part 2 – Statistical Process Control – Decision Making

Error Typical Behavior Reaction Action

Reacting to ups and Comparisons point to Tampering and knee When stable, work on
downs (false alarms) point, to average, to jerk reactions, long-term history, fix
last year frustration the system

Failure to detect trend No criteria to separate Molehill grows into Use SPC to detect
trend from noise mountain trends accurately and
in time

25
Part 2 – Statistical Process Control – Definition of a Trend

• One point outside the control limits

• Two out of Three points two standard deviations above/below average
• Four out of Five points one standard deviation above/below average
• Seven points in a row all above/below average
• Ten out of Eleven points in a row all above/below average
• Seven points in a row all increasing/decreasing

Note: There are variations from author to author. This list is the
most conservative I have found at identifying trends. This list is
the same as in INPO 07-007

Dr. Wheeler uses 8 points in a row above/below average

Western Electric Rules (such as in MiniTab) uses 9

26
Part 2 – Statistical Process Control – Definition of a Trend

4 SPC Trend Rules - Shift versus Number of

Points Needed
Shift Needed (in Standard

3 3 Sigma (UCL)
Deviations)

2 2 Sigma

1 Sigma
1
STABLE,
NO TREND Same Side of Average Line
0
1 Point 2 of 3 4 of 5 7 10 of
Points Points Points 11
Points
Number of Points Needed

27
Part 2 – Statistical Process Control – Conceptual Trending and Context

• Results from SPC:

Should be reviewed for Context with involved parties
Results may trigger Conceptual Trending
• Conceptual Trending
Recognition of patterns from human memory of events
A pattern may then be analyzed using SPC
• Conceptual Trending and SPC may trigger Root Cause and Human
Performance analysis methods and vice-versa

28
Concluding Part 2

• SPC is an objective, well-documented method for Trending

• SPC provides the means to determine if a process is stable versus changing
• A Stable Process needs to be evaluated for is it stable where we want to be
• Trends can be identified with SPC techniques, and a determination of “what happened”
made
• May be integrated with
Conceptual Trending

Keep going for information

Root on SRNL family colors.
Trending
Cause
Human
Performance

29
Part 3 – SPC and the Dashboard

• We are generally faced with pulling together data from several sources, several ways of
looking at ‘performance’ and several organizations in order to get one “answer”

Keep going for information

on SRNL family colors.

30
Part 3 – SPC and the Dashboard

This is one I’d really like to see:

The Health of the Site is:

8.3
Turn around and go home

Keep going for information

on SRNL family colors.

31
Part 3 – SPC and the Dashboard - Rationale

If we segregate our performance results into the 4 trending bins we can apply
Red / Yellow / Green coloring:

Keep going for information

on SRNL family colors.

32
Part 3 – Early Prototype of the Canadian Nuclear Laboratories Cover Sheet

CNL SPC DASHBOARD

Indicator
(with link to DWM Chalk DWM
definition) CNL R&D River DWM PHAI Whiteshell Operations Engineering HSSE&Q

LAGGING Y G G G G G G G
INDICATORS
Event Free Day
Resets Y G G G G G G G

Event Free Day

Resets (Dept) G G G G G G G G

Events Exernally
Reported G G G G G G G G

Injuries (ImpAct
Reported) G G G G G G G G

Reactor Trip G G G

SL 1 and 2 Y G G G G G G G

LEADING Y G G G G Y G Y
INDICATORS
ImpAct Perform-
ance Health R Y Y Y Y Y R
Percent Self ID
G G G
Keep going
G
for information
G G G G
ImpAct
on SRNL family colors.
Observation Rate G G G G G G G Y
% Observations with
ImpAct generated G G G G G R G Y

33
Part 3 – Early Prototype of the Canadian Nuclear Laboratories Cover Sheet

CNL SPC DASHBOARD

Indicator
(with link to DWM Chalk DWM
definition) CNL R&D River DWM PHAI Whiteshell Operations Engineering HSSE&Q

LAGGING Y G G G G G G G
INDICATORS
Event Free Day
Resets Y G G G G G G G

Event Free Day

Resets (Dept) G G G G G G G G

Events Exernally
Reported G G G G G G G G

Injuries (ImpAct
Reported) G G G G G G G G

Reactor Trip G G G

SL 1 and 2 Y G G G G G G G

LEADING Y G G G G Y G Y
INDICATORS
ImpAct Perform-
ance Health R Y Y Y Y Y R
Percent Self ID
G G G
Keep going
G
for information
G G G G
ImpAct
on SRNL family colors.
Observation Rate G G G G G G G Y
% Observations with
ImpAct generated G G G G G R G Y

34
Part 3 – Early Prototype of a CNL Chart Sheet

Site - Event Free Day Resets

EFDR - Site Level EFDR - Sum of all Dept Resets
6 140

5 120

100
4
80
3 
60
2
40
1 20
0 0
Oct-13

Oct-14

Apr-15

Oct-13

Apr-14

Oct-14

Apr-15
Apr-13

Apr-14

Oct-15

Apr-13

Oct-15
Rate of Event Free Day Resets baseline is higher than 1.03 per month Improving trend, December below LCL.
Stable at a rate higher than goal. Y G

Organization Pareto (Site EFDR) Reasons for Site EFDR's

12 6
10 5
From 2015/01/01 To
8  4  From 2015/01/01 To
2015/12/31
6 3 2015/12/31
4 2
2 1
0
Keep going for information
0

reportable event
reactivity change
Switching/tagging/wr

11 - Improper conf of

6 - Doses exceeding
Record/Non Record

9 - Unplanned

15 - SL1 or SL2
Ops

DWM WL

DWM Chalk River

Other
DWM PHAI
HSSE

Engineering
R&D

3 - Unplanned
facility equipment

shutdown
ong component
5 - Fatality or

Action Levels
on SRNL family colors.
12 -
LTI

Site EFDR Goal is less than 12.4 per year, per 145-514020-PRO-001 - EVENT FREE DAY RESET (EFDR) Revision 5, or less than 1.03 per month

35
Part 3 - Example Dashboard at SRNL – Leading Indicators
Contractor Assurance System May 2019
Leading Indicators - STAR and HAS
Charts are updated through end of April 2019
Subject
Link to Chart Link to Chart Link to Chart Link to Chart
(Contains Link to Chart Definitions)
Management Total No. of % SRNL MFOs % SRNL MFOs Pareto of Unsat
SRNL MFO's with Actions Tier III Tier III Results
Observations (MFO)
% Managers MFO's by Level 2
No. of Mgr % MFO's with
Doing MFO's Org for 12
MFO's Trend Actions Trend
Trend Months
Manager MFO Details Types of MFO MFO Actions by
FA 22 Conops FA 27 WP&C
Actions 12 Funct. Area
MFO Actions MFO Actions
Months 12 Months
Corrective Action CTS Total Trend
Significance Cat Problem Type & Funct Area & Sig
1 & 2 Trend Sig Cat Pareto Cat Pareto
Management (CTS)
Work Planning
Conops FA 22
& Control FA 27 FA 22 Pareto FA 27 Pareto
Trend
Trend

Rad FA 11 Trend QA FA 08 Trend FA 11 Pareto FA 08 Pareto

STAR CTS Functional Area OSH FA 20 Trend

Maintenance FA
FA 20 Pareto FA10 Pareto
Details 10 Trend

Chemicals FA 25 Safety Doc FA 06

FA 25 Pareto FA 06 Pareto
Trend Trend

Security FY 18 Fire Protection

FA 18 Pareto FA 12 Pareto
Trend FA 12 Trend

36
Part 3 – Current Prototype Sheet from SRNL – Lagging
SRNL Refocus Indicator

REPORTABLE EVENTS RATES

400

365 Divided by Days Between

350

Today
300
250
200 Goal

Events
150 < 12
100
50
0

5/1/17

4/5/18

6/4/18

1/8/19

5/2/19
1/25/17

10/12/17
12/21/17
2/21/18

9/21/18
3/27/17

7/19/17
8/15/17

4/26/18

7/25/18

3/28/19
Last Reportable Event was on May 14, 2019
Stable worse than Goal. Chart updated as of 5/17/19.
Stoplight set to Green five days after last event. G Goal set to 1 per month (12). Y

Total Events, 12 months Total Events by Month

Past 12 months from Last Event (May 17, 2019) 14 (Sum of Reportable, II, PJR)
16 12
14
12 Reportable 10 Goal < 3.1
10 per month
II 8
8
6
4
PJR Keep
6 going for information
4
2
0
on
2
SRNL family colors.
Radiological
Con Ops/Con R&D

Safety

Facility/Equipment
Security

0
Jul-12

Jul-13

Jul-14

Jul-15

Jul-16

Jul-17

Jul-18
Jan-12

Jan-14

Jan-15

Jan-16

Jan-17

Jan-18

Jan-19
Jan-13
Failure

Chart stable on Goal, Green. May data through 5/17 Y

37
 10
12
14
10
15
20
25

0
2
4
6
8
0
5
EH03: ASSUMPTIONS A3B1: Skill based error

EH04: COMPLACENCY/ A4B1: Management

OVERCONFIDENCE Methods LTA

ET06: INTERPRETATION A4B4: Supervisory

REQUIREMENTS Methods LTA

EI01: UNFAMILIARITY A3B3: Knowledge Based

WITH TASK/FIRST TIME Error

ET01: TIME PRESSURE A4B3: Work Organization

(IN A HURRY) & Planning LTA
EI02: LACK OF
A5B4: Verbal
KNOWLEDGE (MENTAL

to May 1, 2019
Communication LTA
to May 1, 2019

MODEL)

(2nd Level)
A5B2: Written Comm.
EH05: MIND-SET
Content LTA

Discovery Dates From May 1, 2018

Discovery Dates From May 1, 2018

A2B6: Defective, Failed, or

EH02: HABIT PATTERNS
Incidents by Top 10 Causes

Contaminated
(2nd Level STAR CAT Codes)

EW06: UNEXPECTED
Incidents by Top 10 Error Precursors

II
II

A3B2: Rule Based Error

EQUIPMENT CONDITION

PJR
PJR

EW02:

ORP
ORP

A6B2: Training Methods

Evaluation of Incidents

CHANGES/DEPARTURE
LTA
FROM ROUTINE

0
1
2
3
4
5
10
15
20
25

0
5

FJ22: PREJOB
BRIEFING EH: HUMAN NATURE
FJ07: WORK
PACKAGE EI: INDIVIDUAL
QUALITY CAPABILITIES
FJ06: PROCEDURE
QUALITY ET: TASK DEMANDS
Causes and Human Performance for ORP, II, and PJR

FJ24: TASK
EW: WORK
PREVIEW
ENVIRONMENT

FW10: RIGOR
.

FW07:
PROCEDURE USE FJ: JOB-SITE
on SRNL family colors.
Part 3 – Current Prototype Sheet from SRNL – Pareto Chart Page

to May 1, 2019

to May 1, 2019

AND ADHERENCE CONDITIONS

(2nd Level)

FP11:
Keep going for information

MANAGEMENT FW: WORKER

OVERSIGHT BEHAVIOR
Discovery Dates From May 1, 2018

Discovery Dates From May 1, 2018

Errors and Flawed Defenses

FO01: HANDOFFS FO: ORGANIZATIONAL

Incidents by Human Perfomance

FACTORS
.

FJ08: EQUIPMENT FP: PLANT RESULTS

II

CONDITION
Incidents by Top 10 Flawed Defenses

AND PROCESS
II

IMPROVEMENTS
FJ05: WORKER
PJR
PJR

ORP
ORP

KNOWLEDGE, FL: LEADERSHIP

SKILL,… PRACTICES
38
Part 3 – SPC Improvement Cycle (left side) and Correction Cycle (right side)

Benchmarks, START: Gather Data Context,

Goals, Customer Severity
Expectations
From SPC:
Yes Stable? No
Correct
OKAY? Yes Direction?
Yes
No No
Apply Human Reinforce, Apply
Performance & Lessons Root Cause &
Cont. Improv. Learned Corrective Action
Keep going for information Management
Implement on SRNL family colors.
Change
Go back to
START

39
Example of Implementation of this Methodology at a Nuclear Laboratory

• Weekly Reports are being sent out

– Actions, Assessments, FAM Reviews, Verifications - Overdue and Coming Due
– Has lead to accountability, reminders, and improvement
– Overdue items are decreasing
• Existing Stoplight Chart was simplified
– Based upon Reportable Events now, 2 days Red, 3 days Yellow
– Video published
• Site Contractor Assurance Charts were drilled down to the Facility Level
– 61 Charts
• Facility specific charts developed (“Priming the Pump”)
– 40 Leading Indicators, 50 Lagging Indicators
– Initial goals established
– Statistical Trending implemented Keep going for information
on SRNL family colors.
• Facility Management team is verifying goals and choosing a critical few subset of 5 to 7
metrics

40
Lessons from CAS metrics implementation at another Nuclear Facility

• Good technical innovation, well accepted by the Government Client

• Seen as a product for the Government Client and senior management
• Lack of refinement of goals and prioritization
• Lack of visible actions to correct adverse trends and improve processes
not meeting goals
• Lack of visibility in the work place, discussion at safety meetings, staff
meetings

• “It would be beneficial line management and the communication team

collaborated to create multiple points of visibility of safety metrics for both
the stakeholders and personnel.”

41
A Few Thoughts on Goals

• We do need goals to go with our metrics, for comparison with when we are stable
– At least answer the question – do we need to improve? Yes or No . . .

• Management makes the decisions for actions that require $ and Resources, not the
statistician. Help them . . .
– Navy Ops Researcher advice – Do not wear the Admiral’s Hat

• I do believe in Dr. Deming’s 14 Points, including eliminate numerical targets

– Numerical targets, especially those with a means to achieve them will lead to unintended
consequences

• Directional / Aspirational Goals are valid. “We

Keepneed
goingtoforimprove upon this!”
information
on SRNL family colors.
• The combination of Goals, Metrics, SPC, and HPRCT will lead to continual improvement
and success

42
My Information

Steve Prevette,
American Society for Quality Fellow and Certified Quality Engineer
Steven.Prevette@srs.gov 803-952-9803
Steven.Prevette@cnl.ca
Steven.Prevette@siu.edu

prevettejs@gmail.com (Home)

Thanks for Attending!

Any Questions?

43
References

44