Advancing Performance Excellence

www.asq.org DECEMBER 2003

2003 SALARY SURVEY

How Do You Stack Up?
Plus: Process 8 Essential How To Lose
Improvement Six Sigma Weight the
Know-How Tools Quality Way
STATISTICS
ROUNDTABLE
Eight Essential Tools
by Ronald D. Snee
T
he broad use of Six Sigma since
its introduction in the 1980s has
taught us much about how to
make the best use of Six Sigma tools.
Now we need to take a look back and
reflect on what we’ve learned, particu-
larly those things that make the use
of Six Sigma tools more effective.
As noted by philosopher George
Santayana, “Those who remain ignorant
of history are doomed to repeat it.”
In my experience, eight tools are
used most frequently in Six Sigma
projects—all eight use statistical
thinking and methods. The tools are:
process maps, cause and effect matri-
ces (CE), failure mode and effects
analysis (FMEA), measurement sys-
tem analysis (MSA), process capability
studies, multi-vari studies, design of
experiments (DOE) and process con-
trol plans.1
These eight tools are integrated with
the Six Sigma define, measure, ana-
FIGURE 1 Six Sigma Tool Linkage
Customers
e d
t ne Process
men
p r ove
Im
Process
map Control plan
MSA Capability
analysis
CE matrix
CE – cause and effect.
DOE – design of experiments.
FMEA – failure mode and effects analysis.
MSA – measurement system analysis.
SPC – statistical process control.
86 I DECEMBER 2003 I www.asq.org
lyze, improve and control (DMAIC) lack of team involvement is a waste of
process improvement approach. valuable knowledge and experience
Understanding how the output of one and should be avoided. Team involve-
tool can become the input for one or ment in model formulation, data col-
more other tools can help you enhance lection, interpretation of results and
implementation of process changes is
These proven important to the effective use of the
other tools as well.
statistical tools can
help you implement Good Measurement Systems
MSA generally focuses on the use of
Six Sigma more gage repeatability and reproducibility
effectively. studies with measurements being
made on 10 parts, samples or items by
your use of the tools. A schematic of one or more people in duplicate or
the key linkages of the tools is shown triplicate using one or more measur-
in Figure 1. Insight into the sequenc- ing devises.
ing and linkage of the tools speeds up Though I’ve seen practitioners use
the learning process and helps you as few as five parts, samples or items,
become productive more quickly. such small sample sizes should be
used with caution and viewed as a
Get Team Input minimum because they may not rep-
The results of creating a process map, resent the full range of variation in the
CE matrix or FMEA are most useful process. Measurement variation is fre-
when these tools are quently different at different levels of
used by a Black Belt the variable being measured, and a
(BB) or Green Belt change in variation may be missed if
(GB) and a project the parts, samples or items measured
team. All these people do not reflect the total range of varia-
need to be involved tion in the process.
because the tools are Those using the Six Sigma tools often
knowledge based, overlook that a complete MSA also
rather than data includes an evaluation of accuracy, lin-
based, and the results earity and stability over time as well as
SPC
are only as good as an assessment of variation. While
the knowledge of every MSA may not need a detailed
those who use them. assessment of all these characteristics,
BBs and GBs are you should try to determine whether a
Multi-vari DOE typically confident in more thorough evaluation in any of
their abilities and these areas is needed.
sometimes, in the The typical discussion of MSA gives
FMEA
interest of saving little advice on what to do when the
time, may create the measurement system variation is too
process map, CE high. One approach is to conduct
matrix and FMEA measurement method ruggedness
by themselves. Such studies by using DOE techniques and
highly fractioned factorial designs to
identify the variables causing the high
variation in the measurement system.2
When doing Six Sigma projects on
processes outside manufacturing, you
will often find there is no measurement
system in place, so you must create one
by establishing the measurement
method and data collection procedures.
You can then do an MSA to evaluate
the repeatability and reproducibility of
the measurement system. But beware:
Measurement systems in service
processes frequently categorize process
outputs by attributes—pass/fail or
poor/fair/good/excellent—rather than
assign hard measurements.3
Assess Process Capability
A key limitation of many capability
studies is that the computed capabili-
ty indexes (Cp, Cpk, Pk and Ppk) are
based on too small a sample. The sam-
ple size should be at least 30, though
more than 50 is preferable, because
capability indexes are a function of
the process standard deviation.
Standard deviations are highly vari-
able and require large sample sizes to
produce precise estimates of the capa-
bility indexes. For example, there is
large variation in capability indexes
with a sample size as large as 90. The
95% confidence intervals for P pk =
1.33 as a function of sample size are
1.02 - 1.76 for n = 30, 1.11 - 1.61 for n =
60 and 1.17 - 1.52 for n = 90.4
In the measure phase, a capability
study typically uses the baseline data to
establish the process baseline. The base-
line data can also be analyzed using run
charts, control charts and other graphi-
cal techniques to detect special cause
variation that, when addressed, can
lead to process improvements. A 24/7
operation, for example, will have three
or four teams of workers cover the dif-
ferent shifts. Analysis of the operation’s
baseline data might indicate the teams
are performing at different levels due to
the team members’ skills and experi-
ence. Bringing the performance level of
the lower performing teams up to that
FIGURE 2 Screening Experiments in Six Sigma
Cause and
Process map
effect matrix
Take action
• Control Xs
• Select operating ranges
of the higher performing teams may
result in process improvements. Thus,
improvements are identified in the
measure phase well before the project is
officially complete.
The Effectiveness of Multi-Vari
Studies
As one of the most important and
versatile Six Sigma tools, the multi-vari
study is useful in manufacturing and
nonmanufacturing environments. In a
typical multi-vari study, the process is
studied as it operates. The objective is to
find the input variables and the con-
trolled and uncontrolled (noise) process
variables that affect the process output
variables. Unfortunately, many people
forget a major purpose of the multi-vari
study is to identify important uncon-
trolled variables, such as team, day of
the week, season of the year or shift. It
is not uncommon to see multi-vari
studies with few or zero noise variables.
Another common error is not sam-
pling the process long enough to see
the total range of variation in process
operation and performance. If the sam-
pling timeframe is too short, your sta-
tistical analysis may find no, or few,
important variables due to the small
amount of variation in the data. When
few important variables are found, the
process should be re-evaluated so
Failure mode
Multi-vari
and effects
studies
analysis
List of
candidate
Xs
Two NO Design of experiments (DOE)
to five screening experiment
Xs?
YES
DOE
• Factorial
• Response surface
other important variables can be
added to the study and an extension of
the sampling time can be considered.
Identify Key Variables
A key strategy of Six Sigma projects
is to identify the three to six key vari-
ables (drivers) that drive the perfor-
mance of the process. Once these
variables are identified, you can opti-
mize and control the process.
Regression analysis is often used in
the analysis of multi-vari studies and
DOEs to identify the key variables.
It is not uncommon to see regression
analyses performed without consider-
ation of the amount of measurement
variation in the data. Measurement
variation can decrease the R-squared
and adjusted R-squared statistics and
mislead you into thinking you have a
poor model, when in fact the low R-
squared statistic is due to high mea-
surement variation. A low adjusted
R-squared statistic can be due to any
combination of the following: wrong
variables in the model or missing vari-
ables that are important, wrong func-
tional form for the model (for
example, linear relationships are
assumed when a curved response rela-
tionship is present) and high measure-
ment error.
Another thing to look out for when
QUALITY PROGRESS I DECEMBER 2003 I 87
STATISTICS
ROUNDTABLE

using regression analysis, particularly
in analysis of multi-vari studies, is the
presence of correlations among the
process variables, known as multi-
collinearity. Collinearity among the
predictor variables (Xs) in regression
models can lead to erroneous conclu-
sions. The regression coefficient vari-
ance inflation factor (VIF) measures
the effects of collinearity.
If the collinearity is large, the regres-
sion coefficients used to identify the
key drivers will be too large on the
average and may indicate a positive
effect when the true effect is negative
or vice versa. A useful guideline is VIF
< 5 indicates no problem and VIF > 10
indicates collinearity is a problem and
the model should be reformulated or
more data should be collected. A VIF
between five and 10 is a gray area in
which collinearity may or may not be
a problem. Computation and evalua-
tion of the VIF should be standard
practice in regression analysis.
There is also a way for you to increase
your chances of finding the critical few
variables. In my experience, practition-
creates more heat than light. Some BBs
and GBs, when doing regression analy-
sis or analysis of variance (ANOVA),
treat all the y values (process output) as
a single sample and test them for nor-
mality. The test often fails, with the con-
clusion being regression or ANOVA is
not appropriate.
Such a scenario results from a mis-
understanding of the true nature of
the normality assumption: The with-
in-group variation for ANOVA and
the residuals (y – y-predicted) for
regression analysis, not the raw data,
should be normally distributed.
Allowing the statistical software to do
the normality test without giving
careful thought to the role of the nor-
mality assumption is one reason prac-
titioners make this error.
Practitioners also often overlook that
regression analysis and ANOVA are
robust to departures from normality,
with nonhomogeneous variance being
a much more critical assumption. They
need to pay particular attention to the
testing of measurements, such as relia-
bility and lifetime and environmental
self and others make more effective
use of the tools. Recalling the admon-
ishment of Santayana, our goal should
be to make history by quickly making
large breakthroughs in process perfor-
mance, not repeat history.
REFERENCES
1. Forrest W. Breyfogle III, Implementing Six
Sigma—Smarter Solutions Using Statistical Methods,
second edition, John Wiley and Sons, 2003.
2. ASQ Chemical and Process Industries
Division, Trusting Measurement Results in the
Chemical and Process Industries, ASQ Quality
Press, 2001.
3. S.E. Windsor, “Attribute Gage R&R,” Six
Sigma Forum Magazine, August 2003, pp. 23-28.
4. ASQ Chemical and Process Industries
Division, Chemical Interest Committee, Quality
Assurance for the Chemical and Process Industries,
ASQ Quality Press, 1999.
5. George E.P. Box, William G. Hunter and J.
Stuart Hunter, Statistics for Experimenters, Wiley-
Interscience, 1978.
RONALD D. SNEE is principal of performance
excellence at Tunnell Consulting in King of
Prussia, PA. He has a doctorate in applied and
mathematical statistics from Rutgers
University. Snee is a Shewhart Medalist and
an ASQ Fellow.
Copyright 2003, Ronald D. Snee.

ers pay too little attention to the use of
screening designs when using DMAIC
to improve existing processes. The theo-
ry is that by the time they work their
way through the define, measure and
analyze phases, they have reduced the
potentially critical variables to five or
fewer and standard two-level factorial
designs will do the trick. But when they
get to the improve phase with six or
more variables, they should consider
the use of screening designs such as
two-level fractional factorial and
Plackett-Burman designs (see Figure 2,
p. 87).5 The key drivers identified by the
screening designs are used in factorial
and response surface experiments to
find the best region in which to operate
the process.
Test for Normality and Outliers
I have observed an overemphasis on
testing for normality and dealing with
outliers. These are important issues, but
the overemphasis on these two subjects
data, for normality because these mea-
surements are known not to follow a © 2003 American Society for Quality.
normal distribution. Subject matter Reprinted with permission.
knowledge should not be ignored
when using statistical methods.
When handling outliers, you may Please
get buried in the details and become comment
confused. In the final analysis the rele-
If you would like to comment on
vant question is, “Do the suspect
this article, please post your remarks
observations have any effect on the
on the Quality Progress Discussion
overall conclusions of the analysis?” If
Board at www.asq.org, or e-mail
so, you should perform a careful
them to editor@asq.org.
study of the pedigree of the data or
collect additional data. If not, you
should move on. But don’t forget the
suspect data; your suspicions may be
confirmed later, or the mystery may
go unsolved. Remember, when ana-
lyzing very large data sets, you sim-
ply will not have the time to do a
detailed investigation of every outlier
because there may be hundreds or
thousands of them.
I encourage you to try these ideas
and look for other ways to help your-

88 I DECEMBER 2003 I www.asq.org