You are on page 1of 118

Slide 1

A Scientific Approach
to Determining Root Cause

ASQ Lean Six Sigma Conference


February 2011

Bev Daniels

The core principle of Six Sigma is Problem Solving of chronic problems, yet this is the
weakest area of the initiative. Analyze phase is best described as ‘then a miracle
occurs’
This presentation is intended to provide an introduction to a disciplined structured
approach to determining root cause. This method is based on sound scientific and
statistical principles.
Although we will only be able to cover enough material to provide an overview, there is
substantial extra materials and references/bibliography for further study.

What are some of the problem solving methods – not statistical tools – that you have
used?
Slide 2

Problem Solving Frameworks

ACT PLAN
PDSA

STUDY DO

8D - 8 Disciplines DMAIC
•Plan
•Define
•Form a Team
•Define the Problem •Measure
•Containment •Analyze
•Identify Root and Escape Cause •Improve
•Choose Permanent Solution •Control
•Implement & Validate Solution
•Preventive Measure for Systemic Causes
•Congratulate & Celebrate

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 2 IDEXX Laboratories, Inc.

This presentation focuses only on the “Analyze” phase. Also known as known as the
“Identify Root Cause” or Root Cause Analysis step of Problem Solving.
It does not address how to develop and validate solutions, implement controls or
measurement systems analysis.
Slide 3

Two System Sources of Problems

Physics
or
Geometry

Problem

Business
(People)
Processes

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 3 IDEXX Laboratories, Inc.

Problems can be broken down into two basic types: Physics based technical problems
or People based process problems. This distinction applies to the Problem or Effect and
not to the cause side of the causal system. These are operational definitions and as
such are somewhat loose and open to interpretation. But in general they will serve us
well in helping to determine the appropriate Problem Solving Strategies.

Problems are defined as an undesired situation whose cause is unknown.


Slide 4

Common Root Cause Approaches

Theory based
• Brainstorming & fishbone diagrams
• Scientific theories of how a specific factor or event
creates the problem (e.g. fault tree)
FMEA
Trial and error – often solution based
Experience based
5 Why
“Is, Is Not” (Kepner-Tregoe)

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 4 IDEXX Laboratories, Inc.

Theories include pet theories, favorite theories, agenda driven theories

Brainstorming often involves multi-voting and other consensus based ranking schemes
to select theories to test. “science is not a democracy” “public opinion polls have never
changed a law of physics”

FMEA has elements of Problem Solving but is not intended to be a diagnostic tool.
FMEA is focused on how multiple functions fail (complete, partial, intermittent or
unexpected) and what might cause each one as well as severity, occurrence and
detection ratings. In a very limited sense the concept function failure is useful in
defining the Problem and the detection rating is a result of a MSA. The listing of
causes is still limited to known causes or guesses.

Experience based: “the last time this happened”


5 Why and Kepner Tregoe present a holistic diagnostic strategy and are intended to be
augmented with sound experiments and knowledge of science.
Slide 5

Two Basic Approaches1,2,3


Cause to Effect Effect to Cause
• Conjecture : • Evidenced based
Brainstorming → fishbone • Disproves potential
diagrams → multi-voting causes
• Proves a cause creates • Considers all potential
an effect causes
• Swing for the fence • Iterative approach
• Divergent – random • Convergent
searches • Quick tight experiments
• One factor testing or • Focus is on how the
fractional factorials system fails
• Focus is on how the
system works
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 5 IDEXX Laboratories, Inc.

Cause to Effect: The literature has a lot of references to the various techniques, but
little rigorous explanation of how to use the techniques.

Effect to Cause: The literature has few references to the various techniques but tend to
be much more detailed and rigorous in their use. However, the references are very
disparate and there is very little in the literature that synthesizes the techniques in a
coherent and systematically useful manner.
Slide 6

“A bad system will defeat a good person every time”


– W. Edwards Deming

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 6 IDEXX Laboratories, Inc.

Cause to effect can work, it’s just not very efficient (low batting average).

It’s very seductive since it often promises a quick discovery of the cause or a solution.
What most people forget is that there are too often many iterations of the ‘quick’
approach and too often the solution doesn’t work.

It is essential to understand that there is a place for brainstorming (and the


accompanying creativity) but it isn’t in the ‘fishbone diagram’ manner of brainstorming
potential causes. That is not to say that input isn’t needed – it is. A cross functional
team that understands the science and the actual workflow/use/processes involved as
well as a solid understanding of statistical sound experimental techniques is absolutely
required. This team must also often get creative in how they structure their diagnostic
questions and more importantly how they structure the experiments that answer those
questions.
Slide 7

People vs. Physics Systems

Diagnostic Structures Two System Sources


of Problems
Y→ X
Physics
or
Geometry
Functional
Decomposition

Problem

Apollo
“Sequence of Events” Business
(People)
Processes
Lean & 5-Whys

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 7 IDEXX Laboratories, Inc.

Y→X is a 5-Why strategy that utilizes the principles of analytical studies (Deming) and
the “half-split”* technique to work backwards in an iterative progression from the
Problem statement to converge on the causal mechanism.

*”half-split” is similar to 5-Why. It is a proper name rather a precise count.


A 5 Why may have 2 levels or 20, but we still call it 5-Why.
A split may have 2, 3, 4 or more actual split categories.
Slide 8

5 essential elements to the 5 whys

1. The question “why did this problem occur” is a single


layer at a time question, truly like peeling an onion
2. The strategy of the questioning must be aligned with the
nature of cause and effect systems
3. Questions must be carefully structured to yield
conclusive answers
4. Each layer must involve a carefully constructed split of
the system such that the approach converges on the
causal mechanism
5. Sound experimental approaches are required to
properly answer the question.
6. Asking “why” 5 times is a rule of thumb, not a precise
requirement
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 8 IDEXX Laboratories, Inc.
Slide 9

A convergent process of elimination

Begins with highest level of immediate mutually exclusive


and exhaustive causal categories and focuses on
eliminating or disproving each category as containing the
root cause.

Primary focus is on how the system has failed to work.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 9 IDEXX Laboratories, Inc.

Categories contain related causes and causal systems.


Early in the diagnostic process the potential causes are grouped by their effect on The
Y. It is not necessary to know or list the individual factors at this point.
A factor can only exist in one category.
Slide 10

A ‘cause and effect’ tree diagram


Immediate
Cause

Immediate
Cause
The Ys CTQs Metrics

Immediate
Cause Structural Y Functional Y Customer Y $$

Structure Function Customer


• Dimensions • Performance • Requirements Business
• Properties tolerances • Opinion Case
Immediate
Cause • Esthetics

DEFECT FAILURE DIS-SATISFACTION


Immediate
Cause

The Xs : aka Inputs, Variables, Factors


ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 10 IDEXX Laboratories, Inc.

Although the individual factors are highly confounded in the begining, it is easy to
unconfound the single category that contains the causal mechanism.

Now let’s look at the tree in some detail…


Slide 11

Definitions

A defect is a static condition. . A nonconformance to a


specification of a structural dimension, feature or property
(such as found on a blueprint)

A failure is a dynamic condition: a failure of a performance


characteristic. The product or service doesn’t function as
intended.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 11 IDEXX Laboratories, Inc.
Slide 12

Failure modes

A failure mode is a way in which a failure can occur;


it is not a cause.
Individual performance characteristics may have multiple
failure modes.
These failure modes may be the result of different causes,
or may simply a single cause that results in different modes
depending on the state of the causal factor:

• Complete non function


• Partial function
• Intermittent function
• Unintended function

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 12 IDEXX Laboratories, Inc.
Slide 13

True defects

A ‘true’ defect eventually results in a failure.

Some defects may require time or other stresses to result


in a failure. (It requires a condition for failure)

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 13 IDEXX Laboratories, Inc.
Slide 14

Interaction of conditions & product

When product that has variation in some critical


characteristic is subjected to conditions that also vary,
failure will only occur when the conditions are “bad” and
the product is also at a “bad” level:

Stress Strength

Distribution of the
Distribution of the
Primary X for the
severity of the
product
conditions that the
product can see Parts that fail

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 14 IDEXX Laboratories, Inc.
Slide 15

The interaction diagram

Failed Analysis &


Parts Experiments
must be under
the “bad” or
Good
“high” stress
Parts High Strength Parts
condition to be
meaningful
“Low” “High”
Stress Stress
Condition Condition

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 15 IDEXX Laboratories, Inc.
Slide 16

Increasing “strength”

If the “stress”, or conditions for failure, cannot be


controlled, it is necessary to increase the “strength”
of the product...

Strength
Old New
Stress

No parts fail!

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 16 IDEXX Laboratories, Inc.
Slide 17

Test under worst case conditions

Once the conditions for failure are known and defined all
subsequent testing must take place under the same
conditions that can cause failure

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 17 IDEXX Laboratories, Inc.
Slide 18

“Root cause”

An actionable factor, or interaction of factors, that if


corrected or controlled, will prevent future occurrences.

Root causes are physical causes and are uniquely related


to the specific object which is experiencing the Problem.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 18 IDEXX Laboratories, Inc.

In people based processes the “root cause” is often a combination of a specific action
and a specific condition, that may result in a cascading series of actions and conditions.
(See the Apollo method by Dean Gano)
Slide 19

Other causes

Immediate cause: the factor that immediately causes the


Problem. Fixing this will not prevent reoccurrence, but may
enable containment rework/repair.

Intermediate causes: sequence of factors that lie


between the root cause and the immediate cause.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 19 IDEXX Laboratories, Inc.
Slide 20

Peripheral causes

Enabling cause: the defect exists and the enabling cause


allows it to continue. Addressing this may enable better
screening of the defect or Problem.

Systemic cause: an actionable factor that if corrected or


controlled will prevent future occurrences on similar
objects. (Systemic causes are typically business process
or system related)

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 20 IDEXX Laboratories, Inc.

Until the causal mechanism is determined and understood, discussion of these causes
are diversionary; addressing them will not solve the current Problem

Enabling causes can and should be addressed during the diagnostic process IF they
can improve the measurability of the Problem or enable better screening to provide
effective containment of the Problem.

Systemic causes cannot typically be effectively addressed until the causal mechanism
is known. Discussion of a systemic casue for an unknown physical cause is conjecture.
Slide 21

Sequence of causes

Enabling Causes

Systemic Root Intermediate Immediate


Cause Cause Cause(s) Cause
The Problem $

Condition
for Failure

Time Sequence

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 21 IDEXX Laboratories, Inc.
Slide 22

Understanding the causal mechanism

It is the understanding of the causal mechanism that


enables us to devise a viable solution to eliminate or
significantly reduce the Problem.

Enabling Causes

Systemic Root Intermediate Immediate


The Problem $
Cause Cause Cause(s) Cause

Condition
for Failure

Time Sequence

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 22 IDEXX Laboratories, Inc.
Slide 23

The Titanic: a simple example


CAUSE

The
Why did the Titanic
Titanic sink? Sinks

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 23 IDEXX Laboratories, Inc.
Slide 24

The Titanic: a simple example


CAUSE

Why did the Bulkheads Fill The


bulkheads fill w/Water, ship Titanic
with water? loses buoyancy Sinks

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 24 IDEXX Laboratories, Inc.

Don’t skip the ‘obvious’; if it’s a incontrovertible fact, document it; if it’s an assumption,
test to disprove.
Slide 25

The Titanic: a simple example


CAUSE

White Star requires


Smaller bulkheads to
ensure full size first
class cabins

Why did the Bulkheads Fill The


bulkheads fill w/Water, ship Titanic
with water? loses buoyancy Sinks

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 25 IDEXX Laboratories, Inc.

An enabling cause – the smaller bulkheads filled with water faster than larger bulkheads
Slide 26

The Titanic: a simple example


CAUSE

White Star requires


Smaller bulkheads to
ensure full size first
class cabins

How did Bulkheads Fill The


water enter w/Water, ship Titanic
the ship? loses buoyancy Sinks

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 26 IDEXX Laboratories, Inc.

How is often a better question than why…


How focuses us on the physics of the causal system.
Slide 27

The Titanic: a simple example


CAUSE

White Star requires


Smaller bulkheads to
Puncture of ensure full size first
steel plates class cabins

Seams of Bulkheads Fill The


the ship w/Water, ship Titanic
4 possible ways open up loses buoyancy Sinks
for water to
enter the ship Ripping or
tearing of
steel plates

Swamped

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 27 IDEXX Laboratories, Inc.

In order to answer this question, we need specific scientific proof.


Slide 28

The causal system of the sinking of the Titanic


CAUSE

White Star requires


Smaller bulkheads to
ensure full size first
class cabins

Seams of Bulkheads Fill The


Why did the the ship w/Water, ship Titanic
seams open up? open up loses buoyancy Sinks

Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 28 IDEXX Laboratories, Inc.

The Titanic’s sister ship had a side swipe accident in shallow waters and her seems
opened up. The thumbnail above is a picture of this failure.

Unfortunately, the side of the ship that was damaged by hitting the iceberg is the one on
the ocean floor and so we have no direct evidence that it too opened up. We do
however, have additional evidence that makes this factor the most likely cause of water
entering the ship.
Slide 29

The causal system of the sinking of the Titanic


CAUSE

White Star requires


Smaller bulkheads to
ensure full size first
class cabins

Rivet Seams of Bulkheads Fill The


Why did the Heads the ship w/Water, ship Titanic
rivets shear? Shear open up loses buoyancy Sinks

Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 29 IDEXX Laboratories, Inc.

Rivets found in the debris field were found in great numbers to have been sheared at
the rivet head. Shear forces are very easy to identify under metallurgical examination.
Eye witness – ear witness – accounts reported hearing ‘popping’ noises like ball
bearings or marbles hitting the floor at the time of the impact.

Sheared rivets are almost conclusive proof that the seams opened up rather than the
metal sheets being ‘torn’ open
Slide 30

The causal system of the sinking of the Titanic


CAUSE

White Star requires


Smaller bulkheads to
ensure full size first
class cabins

Why did the Rivet Seams of Bulkheads Fill The


Slag weakened
Heads the ship w/Water, ship Titanic
rivets shear? Rivet
Shear open up loses buoyancy Sinks

Metallurgical structure
Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 30 IDEXX Laboratories, Inc.

Metallurgical examination revealed the presence of extensive slag in the rivets which is
known to weaken the strength of metal.
But the rivets didn’t shear on their own
Slide 31

The causal system of the sinking of the Titanic


CAUSE

White Star requires


Side Swipes Iceberg; Smaller bulkheads to
Condition shear force applied ensure full size first
for Failure along side of ship class cabins

Rivet Seams of Bulkheads Fill The


Physical Slag weakened
Heads the ship w/Water, ship Titanic
Root Cause Rivet
Shear open up loses buoyancy Sinks

Metallurgical structure
Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 31 IDEXX Laboratories, Inc.

4 Whys (2 asked twice) to get to causal mechanism.


This is the classic strength-stress interaction.
We could go further and identify why slag was in the rivets – although this was a
scientific/manufacturing constraint at the time. But at this point we have enough specific
information about the causal mechanism to develop a viable solution. The seams need
to be held together by a stronger fastening system. This can be either rivets with no or
minimal slag or a different fastening system altogether. The key is that this is an
actionable factor that if controlled, eliminated or changed will prevent
occurrences in the future.
In this case actions to guard against the condition for failure are futile. Any sideswipe
can cause the rivets to shear; in fact this is how the Olympia failed.
Slide 32

The causal system of the sinking of the Titanic


CAUSE

White Star requires


Why did the Side Swipes Iceberg; Smaller bulkheads to
titanic hit the shear force applied ensure full size first
iceberg? along side of ship class cabins

Rivet Seams of Bulkheads Fill The


Slag weakened
Heads the ship w/Water, ship Titanic
Rivet
Shear open up loses buoyancy Sinks

Metallurgical structure
Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 32 IDEXX Laboratories, Inc.

Now we are dealing with the specific event. We are also crossing over from physics to
people based systems.

All physics based failures have a systemic cause in people based systems.
People create many of the specific conditions for failure* and people certainly create the
conditions for behavioral causes (actions).

*People didn’t create the atmospheric condition that made the iceberg practically
invisible until the last seconds, nature did. But people did create an environment that
led to reckless behavior in entering the ice field and trying to run as fast as possible…
Slide 33

The causal system of the sinking of the Titanic

Systemic Causes CAUSE

White Star requires


Lookouts See Iceberg Side Swipes Iceberg; Smaller bulkheads to
too late; shear force applied ensure full size first
swerve to avoid along side of ship class cabins

Rivet Seams of Bulkheads Fill The


Slag weakened
Heads the ship w/Water, ship Titanic
Rivet
Shear open up loses buoyancy Sinks

Metallurgical structure
Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 33 IDEXX Laboratories, Inc.
Slide 34

The causal system of the sinking of the Titanic

Systemic Causes CAUSE


A Thermal inversion
causing super
refraction and mirages
White Star requires
Lookouts See Iceberg Side Swipes Iceberg; Smaller bulkheads to
too late; shear force applied ensure full size first
swerve to avoid along side of ship class cabins

Rivet Seams of Bulkheads Fill The


Slag weakened
Heads the ship w/Water, ship Titanic
Rivet
Shear open up loses buoyancy Sinks

Metallurgical structure
Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 34 IDEXX Laboratories, Inc.
Slide 35

The causal system of the sinking of the Titanic

Systemic Causes CAUSE


A Thermal inversion
causing super
refraction and mirages
White Star requires
Lookouts See Iceberg Side Swipes Iceberg; Smaller bulkheads to
too late; shear force applied ensure full size first
swerve to avoid along side of ship class cabins

Titanic sailing too fast Rivet Seams of Bulkheads Fill The


Slag weakened
for conditions Heads the ship w/Water, ship Titanic
Rivet
Shear open up loses buoyancy Sinks

Metallurgical structure
Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 35 IDEXX Laboratories, Inc.
Slide 36

The causal system of the sinking of the Titanic

Systemic Causes CAUSE


A Thermal inversion
causing super
refraction and mirages
White Star requires
Lookouts See Iceberg Side Swipes Iceberg; Smaller bulkheads to
too late; shear force applied ensure full size first
swerve to avoid along side of ship class cabins

Titanic sailing too fast Rivet Seams of Bulkheads Fill The


Slag weakened
for conditions Heads the ship w/Water, ship Titanic
Rivet
Shear open up loses buoyancy Sinks

Captain ignores iceberg


warnings

Metallurgical structure
Open Seams on Olympia

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 36 IDEXX Laboratories, Inc.
Slide 37

The causal system of the sinking of the Titanic

Systemic Causes CAUSE


A Thermal inversion
causing super
refraction and mirages
White Star requires
Lookouts See Iceberg Side Swipes Iceberg; Smaller bulkheads to
too late; shear force applied ensure full size first
swerve to avoid along side of ship class cabins

Titanic sailing too fast Rivet Seams of Bulkheads Fill The


Slag weakened
for conditions Heads the ship w/Water, ship Titanic
Rivet
Shear open up loses buoyancy Sinks

Captain ignores iceberg


warnings

White Star pushes for Metallurgical structure


Open Seams on Olympia
record crossing time

EFFECT
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 37 IDEXX Laboratories, Inc.

This is a relatively simple example that really didn’t require sophisticated experimental
designs to answer any of the 5 Why questions. It does show the basic flow of causal
mechanism and the separation of systemic, enabling and physical causes.
There are a great deal of other factors that contributed to the Titanic being in the
presence of this massive iceberg during a thermal inversion. There are also factors that
made the death toll higher than it should have been.
It is a romantic notion to say that “no one thing caused the Titanic to sink” or that “there
were a cascade of unforeseeable events that doomed the Titanic to its tragic demise”.
Certainly these events cascaded to put the Titanic on a collision course with the iceberg.
But to give this series of events such importance is to give up control of our lives to the
fates. Nothing could be further from the truth. The Titanic sank because it experienced
a shear force and it had very weak rivets that could not withstand this force. If it hadn’t
hit the iceberg on April 14 1912, it would have hit something at some time and the
seams would have opened up then. Increasing the strength of the fastening system
would have saved the Titanic or future ships of similar size and this is the goal of
engineers and scientists.
Slide 38

The Effect to Cause diagnostic tree


Levels of Causes
3 2 1
Categories of Causes

Immediate
Cause

Immediate
Cause

Immediate
Cause Structural Y Functional Y

Structure Function
• Dimensions • Performance
• Properties tolerances
Immediate
• Esthetics
Cause

Immediate
Cause

Branch of Causes
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 38 IDEXX Laboratories, Inc.
Slide 39

Using the Effect to Cause tree diagram


All potential causes are not listed on the diagram.
• Only one level at a time is listed.
• Each level contains a mutually exclusive and
exhaustive categorization of the causes
• Only those categories of causes that are investigated are
listed.
• Investigation of each category is to determine if it contains
the primary cause or not.
• The next level is not listed until the previous level is fully
investigated and only the category containing the primary
cause remains
• Only those branches that are found to contain the primary
cause are investigated

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 39 IDEXX Laboratories, Inc.

This is accomplished by analyzing it’s affect on Y: how much does The Y


vary as this category varies. It is not an analysis of how much the category
itself varies.
Slide 40

Disprove

Progressive searches utilize experimental designs that


simultaneously test all alternative theories and seek to
disprove whole categories of causes, rather than to
prove a single factor or causal category is the root cause.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 40 IDEXX Laboratories, Inc.
Slide 41

A true Effect to Cause tree diagram

Immediate
Cause

Immediate
Cause

Immediate
Cause The Y Functional Y

Immediate
Cause

Factor
Immediate
Factor Factor
Cause
Factor 2nd Cause
Root Cause

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 41 IDEXX Laboratories, Inc.
Slide 42

A simple split exercise4


1 2 3 4 5 6

8 9 10 11 12 13 14 15 16 17

18 19

20 21 22 23 24 25 26

27 28 29 30 31 32

33 34 35 36 37 38 39 40

41 42 43

44 45 46 47 48 49 50 51 52 53 54

55 56

57 58 59 60 61 62 63

64 65 66 67 68 69 70 71 72

73 74 75 76 77 78 79

80 81

82 83 84 85 86 87 88 89 90 91 92

93 94 95

96 97 98 99 100 101 102 103

104 105 106 107 108 109

110 111 112 113 114 115 116

117 118

119 120 121 122 123 124 125 126 127 128

129

130 131 132 133 134 135

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 42 IDEXX Laboratories, Inc.

This example is a simple linear flow with no interactions and only one causal factor.
It also represents a deterministic approach (the failure rate is a binary response). If this
were a real world example, the failure rate will most likely be a consideration.
If there were a secondary cause (less effect than a primary cause, the investigator
would detect this as the failure rate would not go to zero when the primary cause was
determined and/or controlled.)
This causal system is amenable to the half split technique.
Other systems may require multi-way splits.
Slide 43

Split categories

Split categories must be exhaustive and mutually exclusive


All possible splits are either
• Functional
• Structural
• Temporal

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 43 IDEXX Laboratories, Inc.

Some causal systems will use hybrids of the 3 primary split categories
Slide 44

Functional causal categories

Functional Failure Modes


Identify which of the potential failure modes is most prevalent

Broad “use case” functional categories


• User
• Product
• Consumables, supplies
• Environment
• Use conditions

Specific functions or energy transfer paths

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 44 IDEXX Laboratories, Inc.
Slide 45

Structural causal categories

Location
• Within piece
• cavity to cavity
• station to station
• line to line
• plant to plant
• region to region
Components
• Sub-assemblies
• Components
• Raw materials
• Process (assembly or process methods)
Specific features, dimensions and/or properties
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 45 IDEXX Laboratories, Inc.
Slide 46

Temporal causal categories

Temporal
• (Within Piece), Piece to piece, lot to lot, vendor lot
to lot, month to month, season to season etc.
• Product use: during use, use to use
• Operator to Operator
• Within a process; step to step or operation to
operation.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 46 IDEXX Laboratories, Inc.
Slide 47

The need for speed

There may be multiple viable choices for causal categories,


particularly for the first level.

The best choice is often determined by what can be easily


or quickly tested.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 47 IDEXX Laboratories, Inc.
Slide 48

Diagnostic pairs5

The Effect to Cause approach is a progressive search


using diagnostic pairs.
A diagnostic pair is a causal category that contains two
distinctly different results in The Y.
Comparisons of the diagnostic pair will yield an actionable
causal factor.
The most effective diagnostic pair is one that is as
close in time and space as possible.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 48 IDEXX Laboratories, Inc.
Slide 49

Examples of diagnostic pairs5

• Good and bad parts


• Good and bad events within the same part
• Raw materials or components associated with good
and bad parts
• Manufacturing lines, processes or Customers
associated with good and bad events.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 49 IDEXX Laboratories, Inc.
Slide 50

Four questions: framing the Effect to Cause search6

1. What’s Wrong
What’s
Happening?
2. What’s Changed

Effectiveness
What’s
Different?
3. What’s Different
What’s

4. What’s Happening Changed?

What’s
Wrong?

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 50 IDEXX Laboratories, Inc.

What’s Wrong: The most common approach and works well with simple Problems and
obvious (easily observed) causes.
What’s Changed: The weakest question. It requires that the cause was known,
measured and recorded. It is also prone to post hoc, ergo propter hoc* errors.
What’s Different: Involves determining the differences between the diagnostic pairs.
When used with a convergent elimination strategy it is highly effective.
What’s Happening: This is the strongest question. It includes the other three
questions as appropriate and when coupled with a convergent elimination strategy is
the most effective approach for highly complex problems
Slide 51

Start with a good problem statement6

Simple: object, defect format


Singular: only 1 defect at a time
Use only verified facts in the statement
In terms of the effect not the cause

“An approximate answer to the


right question is worth a good deal
more than the exact answer to an
approximate problem.” John Tukey

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 51 IDEXX Laboratories, Inc.
Slide 52

The observational study

The beginning of the Scientific Process is OBSERVATION.


This requires observation of the actual process under
Normal Operating Conditions.

If The Problem occurs in the field, then product that


experiences the failure and product that doesn’t experience
the failure (but has had the same opportunity in time, usage,
conditions, etc.) must be retrieved and run in-house.

If this is not possible, then product must be run under


Customer conditions in-house to recreate the failures.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 52 IDEXX Laboratories, Inc.
Slide 53

Usefulness of the observational study

Understanding of the full range of variation in The Y


Understanding of any non-homogeneous variation which
will drive sampling schemes for any invasive experiments.
• Largest components of variation such as run to run, time to time,
vendor lot to lot, etc.
• Clustering of failures (a common occurrence with rare events,
defect rate < 5%)
Understanding of Normal Operating Conditions including
best and worst case.
Identification and separation of existing failure modes
The causal factor or many clues may actually be visible…

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 53 IDEXX Laboratories, Inc.
Slide 54

The observational study also lends itself to a less rigorous,


but quick assessment of the situation and the
appropriateness of several alternative causal categories.

It can quickly point to the strategy that will be the most


effective.

It’s amazing the things you can see


when you look!
Yogi Berra

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 54 IDEXX Laboratories, Inc.
Slide 55

Process & function maps

First level categories come from process maps, functional


maps and knowledge of the functional components.
How is The Y created or used?
What categories of Xs will cause variation in The Y?

The first level should not be constructed until the


process and/or function maps are created and
understood.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 55 IDEXX Laboratories, Inc.

It is always helpful for the team lead – and team members where feasible – to carry
around the part experiencing the Problem. Holding it, looking at it, playing with it, keep
the eladers focus on the Problem part and it’s function.
Slide 56

Types of Studies

Enumerative Analytic
Descriptive Predictive
• Estimation of a finite population or static
data set • Understand the causal mechanisms and
resulting performance of a system in
• Quantifies only the product or process in
order to make predictions about future
front of us.
performance
• Has no predictive usefulness for future
performance. • Used to improve products and processes
in the future

Statistics and statistical precision of the Tests of statistical significance are often
estimates have value. redundant
Used to determine what action should Proper structure is more important to
be taken on the population under study. our belief in the prediction than
statistical estimates of the precision or
accuracy of the analysis.
Examples: Census, customer surveys, Examples: Y-X problem solving studies,
acceptance sampling, factorial experiments, response surface
experiments
Critical Structural Element: Critical Structural Elements:
Randomness (representativeness) of Independence & Replication
the sample
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 56 IDEXX Laboratories, Inc.

Besides choosing an effective diagnostic strategy, we must also choose effective


experimental approaches. This is where understanding the difference between
statistical approaches that characterize a process and those that improve a process is
so critical.
Slide 57

Analytic Studies

Predictive studies performed in order to take action to


improve future performance.
The purpose of the study is to understand the causal
mechanisms of the system or process.
Most statistical tests of differences (ANOVA, t-tests,
homogeneity of variance), statistical summaries (averages
and standard deviations) and quantification of statistical
significance (p values) have little to no value in analytic
studies.
This is not to say that analytic studies do not have to be
statistically sound.
A solid understanding of the fundamental nature of variation
is absolutely essential to constructing the appropriate
experimental structure and interpreting the result
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 57 IDEXX Laboratories, Inc.
Slide 58

Structure Reduces Uncertainty

The greatest source of uncertainty in analytic studies is


the structure of the study.
The conditions and factors that will change in the
future must also change in the study or there can be
very little confidence in the prediction of future
results.
This uncertainty cannot be statistically quantified.
Conversely, study results that are replicated across the
full range of expected variation are far more reliable
than any statistical test of significance might indicate.

Structure is more important than statistical formulae


ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 58 IDEXX Laboratories, Inc.
Slide 59

Replication

Replication is the cornerstone of science

Without replication of results the study has no credibility

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 59 IDEXX Laboratories, Inc.
Slide 60

Six Sigma doesn’t eliminate physics

Science &
Engineering

6
Practical
Statistics
Knowledge

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 60 IDEXX Laboratories, Inc.
Slide 61

The half-split technique and the binary search4

The majority of the experimental tactics used in a


progressive search are focused on elimination of causal
categories.
The half split technique is a time honored approach to
progressively cut a system in half until the root cause is all
that is left.

Not all situations lend themselves to the half split technique


but will be suited for a variant of it; our experimental design
may split the causal systems into thirds or quadrants, etc.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 61 IDEXX Laboratories, Inc.

A classic example is The Dictionary Game: One player picks a word from the dictionary
and the other player tries to guess the word. The guessing player then attempts to
determine what the selected word is by asking questions that can only be answered
with a ‘yes’ or a ‘no’. The weakest approach is to ask very specific questions: “is it this
word or that word”. They may elect to ask such questions as “does it start with an ‘A’”
or “is it an animal, vegetable or mineral”, or perhaps even “is it a noun, verb, descriptor
or interjection”? While each of these questioning strategies has some ‘elimination’
power, they are weaker than simply asking “is the word in the first half of the dictionary”
and progressively cutting the part of the dictionary that contains the word in half until the
guessing player is down to the last remaining word. The player doesn’t need to know
what the word is, how to pronounce it or even how to spell it; they simply need to have
the dictionary in their hands.
Slide 62

Balance, Discipline and Structure

The biggest mistakes in formal or informal Problem Solving


are a lack of balance, discipline and/or structure

BALANCE: testing all levels that are relevant

STRUCTURE: create statistical significance and practical


importance

DISCIPLINE: execute the test protocol as designed

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 62 IDEXX Laboratories, Inc.
Slide 63

Balance

High and Low levels of the X (suspected cause)


Old and New: current method or part vs. the proposed
solution to the method or part.
Sample sizes for each level must be equivalent; a ratio of
no more than 3:4 is recommended if equal sample sizes
can’t be obtained.
Ensure that your data spans the full range of variation in
The Y

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 63 IDEXX Laboratories, Inc.
Slide 64

Structure
Sample size and independent replicates
Simultaneous testing of all alternative theories
• Use of appropriate experimental controls: ensure
that external extraneous changes are not missed or
misinterpreted.
• Inclusion of changes in “experimentally uncontrolled”
factors
• Randomization
Protocol:
• ensuring that the test conditions replicate
normal operating conditions
• Ensuring that worst case conditions are
correlated to normal conditions and are not
beyond actual worst case conditions
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 64 IDEXX Laboratories, Inc.
Slide 65

Discipline

Don’t alter the test plan to chase an “observation” (aka


shiny object) that may well be an isolated anomaly or
coincidence.
Don’t alter the test plan to be quicker or easier midstream
or without consultation
BE THERE – It’s amazing the things you can see if you
look
Don’t throw out data you don’t like. If the data is a
confirmed typo or experimental error you can remove from
the statistics (Annotate its’ existence). If it’s extreme value
without assignable error cause, leave it in.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 65 IDEXX Laboratories, Inc.
Slide 66

Look both ways before crossing the street!

The initial analysis phase


The X
concentrates on determining the
differences between the highest and
lowest values of The Y.
When evaluating suspect Xs it is
essential to ensure that the factor The Y
“Passing” “Failing”
accounts for both high and low Part Part
results in The Y.

Do not look only at the “bad parts”.


The Root Cause factor is responsible for both the “good”
and the “bad” results in The Y!
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 66 IDEXX Laboratories, Inc.
Slide 67

You have isolated the causal mechanism when…

In order to ensure that you have isolated the primary root


cause factor:
• It is essential that your data spans the full range of
variation in The Y
• All other factors should not be held constant.
• All experiments (invasive) should be run randomly.
• Ensure that there are multiple independent samples
(the number of data points does not equal the sample
size)
• You can turn it on and off at will

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 67 IDEXX Laboratories, Inc.
Slide 68

A turned dimension

The Problem:
What are the ‘categories’
A 3 Component assembly is
experiencing cracks in the clips
for the first level?
and some loose B Parts...
Part A
Top View
Part B

The Y

Cracked
Clip Clips

Clip

Side
View A B

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 68 IDEXX Laboratories, Inc.
Slide 69

Effect to Cause 1st level split: structural

A is too big

The Y
B is too big
Cracked
Clips
C is too small

C is too weak

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 69 IDEXX Laboratories, Inc.
Slide 70

First experiment
Part A
Top View

Failed assemblies were returned form the Part B

field.
A “post hoc” analysis was performed Clip

Since the center clips were never observed


to be cracked, the center parts were
compared to the end parts that exhibited
cracks
This was a “paired” test with the center and
end parts compared within each failed
assembly: These two parts form the
diagnostic pair.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 70 IDEXX Laboratories, Inc.
Slide 71

The Results

Part A Thickness Part B Thickness

Cracked Cracked
Center Center

0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11
Assembly Assembly

Part C Thickness Part C Tensile Strength

Cracked Cracked
Center ` Center

0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11
Assembly Assembly

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 71 IDEXX Laboratories, Inc.

This comparison can also be displayed on a Youden plot. A systemic difference will
display as a bias to a 45 degree 1:1 line
Slide 72

Part A is too thick

It was found that Part A was larger in the cracking end


clip area than in the non cracking center clip area.
In fact it was found that the Dimension did not have to be
out of spec to cause a cracked clip... The Y
Cli p

Side
View A B

Tk

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 72 IDEXX Laboratories, Inc.
Slide 73

The turning process


Part A is turned on a fixture that holds 8 parts.
It takes 10 hours to turn a complete Set of 8 parts.
Three different lathes are used exclusively for producing these parts.
There is only one vendor for the castings.

What approach would you use?


What are the split categories?
Part A
Top View
Part B

Clip

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 73 IDEXX Laboratories, Inc.
Slide 74

Turning categories of variation

Within Piece
Piece - Piece = Within Fixture
Set to Set
Machine to Machine
Operator to Operator
Time to Time
Shift to Shift
Vendor Batch to Batch

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 74 IDEXX Laboratories, Inc.
Slide 75

Effect to Cause 2nd level split: temporal & location


DR = 7.6
Part A Thickness

MSA
Cracked
Center

Within Piece
0 1 2 3 4 5 6 7 8 9 10 11
Assembly

Piece to Piece A is too big Clip

Clip A B

Lot to Lot
The
B is too big Y
Time to Time
Cracked
Operator to Clips
Operator C is too small

Lathe to Lathe

C is too weak
Vendor Lot to
Lot
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 75 IDEXX Laboratories, Inc.
Slide 76

Multi-Vari results
Data was taken from 3 sets, 3 readings per part (at each of the 3
clip locations), on all 8 parts off of a single machine. Two
separate operators were involved. The results were:

10 10
9 9
8 8
7 7 Left
6 6 Cent er
5 5 Right
4 4 LSL
3 3 USL
2 2
1 1
0 0
3

12

15

18

21

24

27
Which category causes the largest change in the Y?
Do you see anything unusual about the data?
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 76 IDEXX Laboratories, Inc.

With a multi-vari when the majority of the baseline variation in Y is observed, root cause
was active in that category and it is not necessary to proceed to test the other
categories
Slide 77

Effect to Cause 3rd level split?


DR = 7.6
Part A Thickness

MSA
Cracked
Center

0 1 2 3 4 5 6 7 8 9 10 11

Within Piece Assembly

Piece to Piece A is too big Clip

Clip A B

Lot to Lot
The Y
B is too big
Time to Time
Cracked
The centers are thin Operator to Clips
Operator C is too small
The ends are thick
How does this happen? Lathe to Lathe
10
9
8
7
10
9
8
7 Left
C is too weak
6 6 Center
5 5 Right
4 4 LSL
3 3 USL
2 2
1 1
0 0
3

12

15

18

21

24

27

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 77 IDEXX Laboratories, Inc.
Slide 78

Effect to Cause 3rd level split: process


DR = 7.6
MSA
Part Moving
Within Piece
Cutter Moving
Piece to Piece A is too big Clip

Clip A B

Lot to Lot
The Y
B is too big
Time to Time
Cracked
Operator to
Clips
Operator C is too small

Lathe to Lathe
10
9
8
7
10
9
8
7 Left
C is too weak
6 6 Center
5 5 Right
4 4 LSL
3 3 USL
2 2
1 1
0 0
3

12

15

18

21

24

27

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 78 IDEXX Laboratories, Inc.
Slide 79

New Process Change

A quick experiment on the clamping was tried


with the following results:

10 10
9 9 Left
8 8 Center
7 7
Right
6 6
New Left
5 5
New Center
4 4
3 3 New Right

2 2 LSL
1 1 USL
0 0
3

12

15
18

21

24

27

33

36
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 79 IDEXX Laboratories, Inc.
Slide 80

Final Effect to Cause


DR = 7.6
Part A Thickness

MSA
Clamping Cracked

Part Moving Center

Strength 0 1 2 3 4 5 6 7 8 9 10 11

Within Piece Assembly

Cutter moving

10 10
Piece to Piece A is too big Clip

9 9 Left
8 8
7 7
Center Clip A B
Right
6 6
New Left
5 5
New Center
4 4

Lot to Lot
3 3 New Right

2 2 LSL

The
1 1 USL
0 0

12

15
18

21

24

27

33

36
B is too big Y
Time to Time
Cracked
Operator to Clips
Operator C is too small

Lathe to Lathe
10
9
8
7
10
9
8
7 Left
C is too weak
6 6 Center
5 5 Right
4 4 LSL
3 3 USL
2 2
1 1
0 0
3

12

15

18

21

24

27

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 80 IDEXX Laboratories, Inc.

Notice the thumbnails of the critical analyses…although they are not easily read on the
diagram, they serve as ‘objective evidence’ that each level is closed with data, not
opinion.
Slide 81

Second Example

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 81 IDEXX Laboratories, Inc.
Slide 82

A prototype printer
A new low cost printer design. Each of 5 prototypes are exhibiting ~ 5%
misfeeds (multiple sheets pulled at a time resulting in a paper jam:
What analysis strategy & questions would you use to get to Root Cause?
Direction of Paper Travel

Paper Roller

Paper Stack
Retard Pad

Tray Lift Spring Paper Tray

Engineering’s List of Key Factors:


• Downward Force of Paper Roller
• Friction of Paper Roller Pad
• Lift Force of tray Lift Spring
• Friction of Retard Pad
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 82 IDEXX Laboratories, Inc.
Slide 83

The first question

What is the normal operating condition of the printer?


• This is an inexpensive desktop printer.
• The typical user is not going to be printing a large number of pages
• May have periods where there are several jobs at a single time
(prep for the day or a series of meetings)
• May have considerable time period between jobs (as they attend
meetings or do other work).
• Mostly cheap copier paper
• May not fill to a full stack each time

The experimental set-up must consider this.

The Y is the presence or absence of a misfeed/jam

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 83 IDEXX Laboratories, Inc.
Slide 84

The second question

What is the diagnostic pair?


In this case all five prototypes experience the same
level of jamming, so the pair cannot be instruments.
All we are left with is mis-feed events and non mis-feed
events.
The first experiment should be to determine what
differentiates the mis-feeding events from the non
mis-feeding events.
This is typically a time sequence type of investigation or
observational study.
This first study can also be used to determine if the jam
rate is actually 5%.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 84 IDEXX Laboratories, Inc.
Slide 85

Printer Effect to Cause 1st level split: temporal


Page to Page

Within Stack

Job Type:
Large or Small

Frequency:
Short or Long Time Paper Jams
Between Jobs

Stack to Stack

Paper Lot to Lot

Paper Type to
Type

Printer to Printer
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 85 IDEXX Laboratories, Inc.
Slide 86

Time sequence, pattern or categorical correlation?

The Multi-Vari approach would be to analyze the time


sequence of mis-feeds: do they happen in any particular
pattern?
Under any particular condition such as large print jobs (10+
pages in the job) or small print jobs (1-2 pages)?
Is there a difference between failure rates when running
many sequential jobs or when the time between jobs is
long?
Is there a difference the location in the stack or are the
jams randomly distributed within a stack?

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 86 IDEXX Laboratories, Inc.
Slide 87

The multi-vari result

In this particular case, all of the mis-feeds occurred


early in a ‘full’ stack of 200 sheets.
After the first 50 sheets are fed, there are no more mis-
feeds.
It doesn’t matter whether a small job or a large job is
run.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 87 IDEXX Laboratories, Inc.
Slide 88

Effect to Cause 2nd level split?

Page to Page

Within Stack

Job Type:
Large or Small

Frequency:
Short or Long
Paper Jams
What changes within the Time Between
Jobs
stack?
Direction of Paper Travel Stack to Stack
Paper Roller
Paper Stack
Paper Lot to Lot
Retard Pad
Tray Lift Spring
Paper Tray
Paper Type to
Type
5% Misfeeds
ASQ Lean Six Sigma Conference Printer to Printer Bev Daniels
February, 2011 88 IDEXX Laboratories, Inc.
Slide 89

What changes within the stack?

Direction of Paper Travel


Paper Roller
Paper Stack

Retard Pad
Tray Lift Spring
Paper Tray

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 89 IDEXX Laboratories, Inc.
5% Misfeeds
Slide 90

Effect to Cause 2nd level split

There are 3 factors that change within the stack :


• The angle at which the paper hits the retard pad
• The force exerted by the tray lift spring thru the stack to
the paper roller
• The angle of the top of the stack of paper

Direction of Paper Travel


Paper Roller
Paper Stack

Retard Pad
Tray Lift Spring
Paper Tray

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 90 IDEXX Laboratories, Inc.

5% Misfeeds
Slide 91

Effect to Cause 2nd level split: structural


Effective upward
Page to Page
force of Spring

Angle of Retard Pad Within Stack

Angle of Paper to Job Type:


Roller Large or Small

Frequency:
Short or Long
Paper Jams
Time Between
Jobs

Stack to Stack

Paper Lot to Lot

Paper Type to
Type

ASQ Lean Six Sigma Conference Printer to Printer Bev Daniels


February, 2011 91 IDEXX Laboratories, Inc.
Slide 92

Retard Pad

A 23 experiment is run on these 3 factors and it is found


that the angle of the retard pad is the root cause
factor…
The tray lift spring doesn’t maintain a single top of the
stack height – the thinner the stack, the lower the height.
This results in the paper contacting the retard pad at
different places.
Direction of Paper Travel
Paper Roller
Paper Stack

Retard Pad
Tray Lift Spring
Paper Tray
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 92 IDEXX Laboratories, Inc.

5% Misfeeds
Slide 93

Final Effect to Cause diagram


Effective upward
Page to Page
force of Spring

Angle of Retard Pad Within Stack

Angle of Paper to Job Type:


Roller Large or Small

Frequency:
Short or Long
Paper Jams
Time Between
Jobs

Stack to Stack

Paper Lot to Lot

Paper Type to
Type

ASQ Lean Six Sigma Conference Printer to Printer Bev Daniels


February, 2011 93 IDEXX Laboratories, Inc.
Slide 94

Backup and Supplemental Material

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 94 IDEXX Laboratories, Inc.
Slide 95

Repeatability and reproducibility7,8,9

Effectiveness of the Measurement system of The Y should be


confirmed with an appropriate MSA. (NOT a traditional gauge
R&R) The MSA should assess repeatability vs actual product
variation.

Problems that are intermittent or measurement systems with low


discrimination will require an effective increase in sample size

• Multiple runs for intermittent functional failures


• Multiple measurements of the same unit for low discrimination
• Actual increases in sample size if the discrimination is low
enough to behave like categorical data.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 95 IDEXX Laboratories, Inc.
Slide 96

SPC is not a diagnostic tool

These are operational definitions that are of limited applicability for


problem solving and are often misunderstood

Assignable Causes are those


that create a ‘sudden’ or
21
20
19
18

‘excursionary’ change in a 17
16
15
14

stable process; either in a 13


12
11
10
nonrandom pattern or beyond 9
8
7

predictable (historical) limits. 6


5
4
3
2
1
Common Causes are those 0

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
that create a stable process
that behaves randomly within
predictable limits.
An Assignable Cause isn’t necessarily easy to find and correct

A Common Cause isn’t necessarily difficult to find and improve


ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 96 IDEXX Laboratories, Inc.
Slide 97

Draw out the sample space

This approach requires us to understand exactly what


questions will be answered and what questions will not be
answered by any given experimental design.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 97 IDEXX Laboratories, Inc.
Slide 98

Exercise: cards10
The Rules:
• Each card has a letter on one side and a number on the
other
• If the card has a letter that is a vowel then the number
will be an even number
The Game: Given the following 4 cards which cards would
you flip to determine if the rule were true or not? You may
only flip 1 or 2 cards:

A 7 8 P
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 98 IDEXX Laboratories, Inc.
Slide 99

The Answer: A and 7

A 7 8 P

even odd vowel cons vowel cons even odd

Inconclusive Inconclusive Inconclusive Inconclusive


Supports Disproves Disproves Supports
The rule The rule The rule The rule
the Rule The rule The rule the Rule
doesn’t doesn’t doesn’t doesn’t
cover cover cover cover
consonants consonants consonants consonants

Anything that disproves a rule is much more


informative and therefore powerful Bev Daniels
ASQ Lean Six Sigma Conference
February, 2011 99 IDEXX Laboratories, Inc.
Slide 100

References
[1] Steiner, Stefan H., MacKay, R. Jock, “Strategies for Variability Reduction”, Quality Engineering, Volume
10, Issue 1, September 1997 , pp 125-136
[2] Kavuri, Surya N., Rengaswamy, Raghunathan, Venkatasubramanian, Venkat, “A Review of Process Fault
Detection and Diagnosis Part II: Qualitative Models and Search Strategies”, Computers and Chemical
Engineering, 27 (2003), pp. 313-326
[3] Smith, Gerald, F., “Determining the Cause of Quality Problems: Lessons From Diagnostic Disciplines”,
Quality Management Journal, 98 5, No. 2, 1998 ASQ
[4] Dale, H. C. A., “Fault Finding in Electronic Equipment”, Ergonomics, pp. 356-383, 1957
[5] Charles Higgins Kepner, Benjamin B. Tregoe, The New Rational Manager, Princeton Research Press,
1981
[6] Allen, John R., “Three Good Questions (and One Not So Good), The New Science of Fixing Things, 2006,
www.tnsft.com
[7] Youden, William John, “Graphical Diagnosis of Interlaboratory Test Results”, Industrial Quality Control,
May 1959, Vol. 15, No. 11
[8] Donald S. Ermer and Robin Yang E-Hok, “Reliable data is an Important Commodity”, The Standard, ASQ
Measurement Society Newsletter, Winter 1997, pp. 15-30.
[9] Donald J Wheeler, “An Honest Gauge R&R Study”, Manuscript 189, January 2009.
http://www.spcpress.com/pdf/DJW189.pdf
[10] Kida, Thomas, Don’t Believe Everything You Think: The 6 Basic Mistakes We Make in Thinking,
Promethius Books, 2006
Gano, Dean, L., “Effective Problem Solving – A New Way Of Thinking”, Apollo Associated Services, Inc.,
www.apollo-as.com
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 100 IDEXX Laboratories, Inc.
Slide 101

Bibliography
Seder, Leonard, “The Technique of Experimenting in the Factory”, Industrial Quality Control,
March 1948
Gano, Dean, Apollo Root Cause Analysis - A New Way Of Thinking, Apollonian Publications,
Distributed by BookMasters, Inc., 1999
Allen, John R., Hartshorne, David J., “The Art and Science of Fixing Things”, 2006,
www.tnsft.com
Steiner, Stefan H., MacKay, R. Jock, Statistical Engineering: An Algorithm for Reducing
Variation in Manufacturing Processes, ASQ Quality Press, 2005
Haviland, Paul R., “Analytical Problem Solving”, The Haviland Consulting Group,
www.fmeca.com
Leonard A. Seder, “Diagnosis With Diagrams – Part I”, Industrial Quality Control, January 1950,
pp. 11-19
Leonard A. Seder, “Diagnosis With Diagrams – Part II”, Industrial Quality Control, March 1950,
pp. 7-11
Mario Perez-Wilson, Multi Vari – A Pre-Experimentation Technique, Advanced Systems
Consultants, 1992
Robert D Zaciewski and Lou Nemeth, “The Multi-Vari Chart: An Underutilized Quality Tool”,
Quality Progress, October 1995, pp. 81-83

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 101 IDEXX Laboratories, Inc.
Slide 102

Types of Studies

Enumerative Analytic
Descriptive Predictive
• Estimation of a finite population or static
data set • Understand the causal mechanisms and
resulting performance of a system in
• Quantifies only the product or process in
order to make predictions about future
front of us.
performance
• Has no predictive usefulness for future
performance. • Used to improve products and processes
in the future

Statistics and statistical precision of the Tests of statistical significance are often
estimates have value. redundant
Used to determine what action should Proper structure is more important to
be taken on the population under study. our belief in the prediction than
statistical estimates of the precision or
accuracy of the analysis.
Examples: Census, customer surveys, Examples: Y-X problem solving studies,
acceptance sampling, factorial experiments, response surface
experiments
Critical Structural Element: Critical Structural Elements:
Randomness (representativeness) of Independence & Replication
the sample
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 102 IDEXX Laboratories, Inc.
Slide 103

Enumerative Studies

Studies of estimation that deal with a finite population.


Statistical estimations of the characteristics of the
population are made based on a sample data set in order
to determine what action to take on that population.
There is no intent to understand the causal mechanisms of
the process that produced the population.
Enumerative studies do not result in conclusions about
future results.
Many statistical tests are helpful in estimating the
characteristics of a population.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 103 IDEXX Laboratories, Inc.
Slide 104

Examples of Enumerative Studies

• Sample inspection of a lot of material in order to


determine if it sufficiently conforms to requirements
• Determination of the prevalence of a disease state
• Determination of the normal range of a biomarker or
chemical in a healthy animal
• Surveys: employee engagement, Customer Net
Promoter Score, US Census

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 104 IDEXX Laboratories, Inc.
Slide 105

Analytic Studies

Predictive studies performed in order to take action to


improve future performance.
The purpose of the study is to understand the causal
mechanisms of the system or process.
Most statistical tests of differences (ANOVA, t-tests,
homogeneity of variance), statistical summaries (averages
and standard deviations) and quantification of statistical
significance (p values) have little to no value in analytic
studies.
This is not to say that analytic studies do not have to be
statistically sound.
A solid understanding of the fundamental nature of variation
is absolutely essential to constructing the appropriate
experimental structure and interpreting the result
ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 105 IDEXX Laboratories, Inc.
Slide 106

Structure Reduces Uncertainty

The greatest source of uncertainty in analytic studies is


the structure of the study.
The conditions and factors that will change in the
future must also change in the study or there can be
very little confidence in the prediction of future
results.
This uncertainty cannot be statistically quantified.
Conversely, study results that are replicated across the
full range of expected variation are far more reliable
than any statistical test of significance might indicate.

Structure is more important than statistical formulae


ASQ Lean Six Sigma Conference Bev Daniels
February, 2011 106 IDEXX Laboratories, Inc.
Slide 107

Replication

Replication is the cornerstone of science

Without replication of results the study has no credibility

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 107 IDEXX Laboratories, Inc.
Slide 108

Types of Factors

Analytical studies rely on replication across the full


range of expected variation of all contributing factors
of the system.
Factors can be categorized in 3 ways based on their type
of ‘control’ in any study.

• Direct Control
• Indirect Control
• Uncontrolled

These groupings do not have any bearing on the amount of


contribution they have to the overall variation of the system.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 108 IDEXX Laboratories, Inc.
Slide 109

Direct Control

These are the factors that can and will be deliberately


manipulated, set, selected or otherwise controlled at
predetermined levels in the study.
These are typically:
• dimensions
• properties
• process input parameter settings
• chemical concentration
• use conditions or environmental factors
– speed, temperature, humidity, barometric pressure

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 109 IDEXX Laboratories, Inc.

Direct Control factors are also referred to as Main Effects


Slide 110

Indirect Control

These are typically natural groupings (or blocks) of


individual factors that:
• may not be precisely known or measureable
• are homogenous or ‘fixed’ at one level within a unique
set of a grouping
• have a different level within another unique set of the
grouping

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 110 IDEXX Laboratories, Inc.

Indirect Control factors are also referred to as “chunky type” or “background” factors.
This type of grouping of factors is also used in Components of Variation studies.
Slide 111

Indirect Control

These factors exist in natural groupings such as:


• raw material lots
• processing batches
• Sub-assembly or component lots, versions or serial
numbers
• equipment
• operators
• shifts or days.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 111 IDEXX Laboratories, Inc.

Indirect Control factors are also referred to as “chunky type” or “background” factors.
This type of grouping of factors is also used in Components of Variation studies.
Slide 112

Indirect Control

The variation of these groups is incorporated into the study


by selecting sets of each group that are uniquely different
from each other. e.g. using different raw material lots,
different process lots, different operators, etc.

If one or more factors in the group have a substantial


unexpected effect on the output, the group can be
deconstructed to determine which factor(s) are the major
contributors.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 112 IDEXX Laboratories, Inc.

Indirect Control factors are also referred to as “chunky type” or “background” factors.
This type of grouping of factors is also used in Components of Variation studies.
Slide 113

Uncontrolled

Very difficult or impossible to control

Controlled within some acceptable input tolerance

Unknown factors

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 113 IDEXX Laboratories, Inc.

Uncontrolled factors may also be referred to as “noise”, “nuisance”, “background” or


even “random” factors.
Slide 114

Very Difficult or Impossible to Control

Examples of these factors are:


• ambient temperature
• humidity
• barometric pressure
• Customer usage

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 114 IDEXX Laboratories, Inc.

Customer usage in this context involves selctign use conditions by selecting different
customers as in a field trial or Beta testing.
Use conditions, once identified can be directly controlled either as a direct control main
effect or in a grouping or ‘block’ by selecting several use conditions and controlling
each condition to a unique level.
Slide 115

Controlled Within Some Acceptable Input Tolerance

Process settings like


• temperature
• pressure
• concentration
These factors may be set anywhere within some tolerance
and can change during a run.
The change from the initial set point may be a deliberate
adjustment by an operator or it may be natural drifting of
the equipment during an operation.
The actual value applied to the product may vary from the
set-point.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 115 IDEXX Laboratories, Inc.

Applied energy is not uniformly distributed across the target area (among cavities, within
a chamber, across a web) and/or over time.
Mixtures are not homogenously distributed within the volume of material
Slide 116

Unknown

The variation of these factors is incorporated into the study


by running the experimental treatments randomly across
a sufficient time period to allow for these factors to vary
naturally.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 116 IDEXX Laboratories, Inc.

Uncontrolled factors may also be referred to as “noise”, “nuisance”, “background” or


even “random” factors.
Slide 117

All Factors

Incorporating all of these factors in your study or


experiment provides real confidence in the viability of the
resulting choices for future performance.

Replication yields confidence.

When all factors are properly incorporated their contribution


can be detected visually in an appropriate graphical display
and quantified by a components of variation analysis.

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 117 IDEXX Laboratories, Inc.

Uncontrolled factors may also be referred to as “noise”, “nuisance”, “background” or


even “random” factors.
Slide 118

References and Bibliography

1 Deming, W. Edwards, “On Probability as a Basis for Action”, The American Statistician, 1975,
Vol. 29, No. 4, pp146-152
2 Moen, Ronald D., Nolan, Thomas, W., Provost, Lloyd P., “Quality Improvement through Planned
Experimentation”, 2nd Edition, McGraw-Hill, 1999
3 Kida, Thomas, “Don’t Believe Everything You Think, Prometheus Books, 2006
4 Bauernfeind, R. H. “The Need for replication in Educational Research”, Phi Delta Kappan, No.
50, pp. 126-128, 1968
5 Sterne, Jonathon A. G., Smith, George Davey," Sifting the Evidence – What’s Wrong With
Significance Tests?”, British Medical Journal, Volume 322, January 2001, pp226-231
6 Johnson, Douglas H., Journal of Wildlife Management, “The Insignificance of Statistical
Significance Testing”, Vol. 63, Issue 3, pp. 763-772, 1999
7 Carver, Ronald P., “The Case Against Statistical Significance Testing”, Harvard Educational
Review, Vol 48, Issue 3, pp 378-399, 1978
8 Shaver, James P., “What Statistical Significance Testing Is and What It Is Not”, Journal of
Experimental Education, No.61, pp. 293-316, 1993
9 Tukey, John W., “A Quick, Compact, Two Sample Test to Duckworth’s Specifications”,
Technometrics, vol. 1, February 1959, pp. 31-48
10 Gans, D. J. (1981), “Corrected and Extended Tables for Tukey’s Quick Test”, Technometrics, 23,
pp 193-195
11 Westlake, W.J. (1971), “A One-Sided Version of the Tukey-Duckworth Test”, Technometrics, 13,
p 901-903

ASQ Lean Six Sigma Conference Bev Daniels


February, 2011 118 IDEXX Laboratories, Inc.

You might also like