Professional Documents
Culture Documents
Bill Rodebaugh
Director, Six Sigma
GRACE
slide 1
Objectives
The Hidden Factory Concept
− What is a Hidden Factory?
− What is a Measurement System’s Role in the Hidden
Factory?
Review Key Measurement System metrics including
%GR&R and P/T ratio
Case Study at W. R. GRACE
− Measurement Study Set-up and Minitab Analysis
− Linkage to Process
− Benefits of an Improved Measurement System
How to Improve Measurement Systems in an
Organization
slide 2
The Hidden Factory -- Process/Production
OK
Inputs Operation Inspect First Time
Correct
NOT
Rework OK
Hidden Factory
Scrap
slide 3
The Hidden Factory -- Measurement Systems
OK
Sample Lab Work Inspect Production
Inputs
NOT
Re-test OK
Hidden Factory
Waste
Time, cost, people
slide 4
The Hidden Factory Linkage
Production Environments generally rely upon in-
process sampling for adjustment
As Processes attain Six Sigma performance they begin
to rely less on sampling and more upon leveraging the
few influential X variables
The few influential X variables are determined largely
through multi-vari studies and Design of
Experimentation (DOE)
Good multi-vari and DOE results are based upon
acceptable measurement analysis
slide 5
Objectives
The Hidden Factory Concept
− What is a Hidden Factory?
− What is a Measurement System’s Role in the Hidden
Factory?
Review Key Measurement System metrics including
%GR&R and P/T ratio
Case Study at W. R. GRACE
− Measurement Study Set-up and Minitab Analysis
− Linkage to Process
− Benefits of an Improved Measurement System
How to Improve Measurement Systems in an
Organization
slide 6
Possible Sources of Process Variation
Observed Process Variation
slide 7
How Does Measurement Error Appear?
Frequency
10
0
30 40 50 60 70 80 90 100 110
Process
15
0
30 40 50 60 70 80 90 100 110
Observed
slide 8
Measurement System Terminology
Discrimination - Smallest detectable increment between two measured values
Accuracy related terms
− True value - Theoretically correct value
− Bias - Difference between the average value of all measurements of a sample and the
true value for that sample
Precision related terms
− Repeatability - Variability inherent in the measurement system under constant
conditions
− Reproducibility - Variability among measurements made under different conditions
(e.g. different operators, measuring devices, etc.)
Stability - distribution of measurements that remains constant and predictable over time for
both the mean and standard deviation
Linearity - A measure of any change in accuracy or precision over the range of instrument
capability
slide 9
Measurement Capability Index - P/T
Precision to Tolerance Ratio
. * σ MS
515
P/T = Usually expressed
as percent
Tolerance
Addresses what percent of the tolerance is taken up by
measurement error
Includes both repeatability and reproducibility
− Operator x Unit x Trial experiment
Best case: 10% Acceptable: 30%
Note: 5.15 standard deviations accounts for 99% of Measurement System (MS) variation.
The use of 5.15 is an industry standard.
slide 10
Measurement Capability Index - % GR&R
σ MS
%R & R = x 100
σ Observed Pr ocess Variation Usually expressed
as percent
slide 11
Objectives
The Hidden Factory Concept
− What is a Hidden Factory?
− What is a Measurement System’s Role in the Hidden
Factory?
Review Key Measurement System metrics including
%GR&R and P/T ratio
Case Study at W. R. GRACE
− Measurement Study Set-up and Minitab Analysis
− Linkage to Process
− Benefits of an Improved Measurement System
How to Improve Measurement Systems in an
Organization
slide 12
Case Study Background
Internal Raw Material, A1, is necessary for Final Product production
− Expensive Raw Material to produce – produced at 4 locations Worldwide
− Cost savings can be derived directly from improved product quality, CpKs
− Internal specifications indirectly linked to financial targets for production costs are used to
calculate CpKs
− If CTQ1 of A1 is too low, then more A1 material is added to achieve overall quality – higher
quality means less quantity is needed – this is the project objective
High Impact Six Sigma project was chartered to improve an important quality variable,
CTQ1
The measurement of CTQ1 was originally not questioned, but the team decided to study
the effectiveness of this measurement
− The %GR&R, P/T ratio, and Bias were studied
− Each of the Worldwide locations were involved in the study
Initial project improvements have somewhat equalized performance across sites. Small
level improvements are masked by the measurement effectiveness of CTQ1
slide 13
CTQ1 MSA Study Design (Crossed)
Op 1 Op 2 Op 3 6 analyses/site/sample
2 samples taken from each site
T1 T2 2*4 Samples should be representative
Each site analyzes other site’s sample.
Each plant does 48 analyses
6*8*4=196 analyses
slide 14
CTQ1 MSA Study Results (Minitab Output)
Components of Variation Response By Sample
120 890
%Contribution
100
%Study Var
80 840
Percent
%Tolerance
60
40 790
20
740
0
Gage R&R Repeat Reprod Part-to-Part Sample 1 2 3 4 5 6 7 8
840
50 UCL=52.45
790
R=16.05
0 LCL=0 740
0 Oper CB1 CB2 CB3 LC1 LC2 LC3 V1 V2 V3 W1 W2 W3
750
0 Sample 1 2 3 4 5 6 7 8
slide 15
CTQ1 MSA Study Results (Minitab Session)
Source DF SS MS F P
Sample 7 14221 2031.62 5.0079 0.00010
Operator 11 53474 4861.27 11.9829 0.00000
Operator*Sample 77 31238 405.68 1.4907 0.03177
Repeatability 96 26125 272.14
Total 191 125058
%Contribution
Source VarComp (of VarComp)
Total Gage R&R 617.39 90.11 Sample, Operator,
& Interaction are
Repeatability 272.14 39.72
Significant
Reproducibility 345.25 50.39
Operator 278.47 40.65
Operator*Sample 66.77 9.75
Part-To-Part 67.75 9.89
slide 16
CTQ1 MSA Study Results
Mean
P/T Equal Variances Differences
Site %GRR R-bar
Ratio within Groups
(Tukey Comp.)
94.3
All 116 16.05 No (0.004) Only 1,2 No Diff.
(78.6 – 100)*
38.9
Site 1 29 7.22 Yes (0.739) All Pairs No Diff.
(30.0 – 47.6)
91.0
Site 2 96 17.92 Yes (0.735) Only 1,2 Diff.
(70.7 – 100)
80.0
Site 3 79 20.37 Yes (0.158) All Pairs No Diff.
(60.8 – 94.8)
98.0
Site 4 120 18.67 Yes (0.346) Only 2,3 No Diff.
(64.8 – 100)
*Conf Int not calculated with Minitab, Based upon R&R Std Dev
slide 17
CTQ1 MSA Study Results (Minitab Output)
Dotplot of All Samples over All Sites
890
840
790
740
A
Site 1 Site 2 Site 3 Site 4
slide 18
CTQ1 MSA Study Results (Minitab Session)
Analysis of Variance for Site
Source DF SS MS F P
Site 3 37514 12505 26.86 0.000
Error 188 87518 466
Total 191 125032
Individual 95% CIs For Mean
Based on Pooled StDev
Level N Mean StDev -+---------+---------+---------+-----
Site 1 48 824.57 15.38 (---*---)
Site 2 48 819.42 22.11 (---*---)
Site 3 48 800.98 20.75 (---*---)
Site 4 48 840.13 26.58 (---*---)
-+---------+---------+---------+-----
Pooled StDev = 21.58 795 810 825 840
slide 19
CTQ1 MSA Study Results (Minitab Output)
X-bar R of All Samples for All Sites
100
Discrimination
CB1 CB2 CB3 LC1 LC2 LC3 V1 V2 V3 W1 W2 W3
Sample Range
Index is “0”,
however can
50 UCL=52.45
Most of the
Sample Mean
850 UCL=851.5
Mean=821.3
samples are
seen as “noise”
800
LCL=791.1
750
0
slide 20
CTQ1 MSA Study Results (Minitab Output)
X-bar R of All Samples
y for Site 4
70 W1 W2 W3
60 UCL=60.99
Sample Range
50
40
30
20 R=18.67
10
0 LCL=0
0
Xbar Chart by WO OP
900 W1 W2 W3
UCL=875.2
Sample Mean
850
Mean=840.1
800 LCL=805.0
MSA Study
Sample Mean
840
830
820
810
Mean=819.4
Results with
800
790
Mean = 819.4
LCL=785.7
780
0
1000
2002 Historical
1
11
Individual Value
1 1 1
1 1
Process
900 55 UCL=899.2
6 22 6 6
662 62
2 22
4 2 22
Results with
Mean=832.5
800 2
22 4 2
6 6 6
Mean = 832.5
5 LCL=765.8
1 1
1
700
Subgroup 0 100 200 300 400
slide 22
CTQ1 MSA Study Results – Process Linkage
Site 2 Example
y
100 LC1 LC2 LC3
0
R=17.92
LCL=0
Calc for Subgroup
0
150
1
1 1 2002 Historical
11
Process
1 1 1
Moving Range
11
100 1 1
1
Results with
UCL=81.95
50
2
R=25.08 Range = 25.08
Calc for pt to pt
2 2 2
0 22 222 2 2
LCL=0
Use Power and Sample Size Calculator with and without impact
of MS variation. Lack of clarity in process improvement work,
results in missed opportunity for improvement and continued
use of non-optimal parameters
slide 24
CTQ1 MSA Study Results – Process Linkage
Site 2 Example
2-Sample t Test 2-Sample t Test
Alpha = 0.05 Sigma = 22.23 Alpha = 0.05 Sigma = 6.67
Sample Target Actual Sample Target Actual
Difference Size Power Power Difference Size Power Power
2 2117 0.9000 0.9000 2 192 0.9000 0.9011
4 530 0.9000 0.9002 4 49 0.9000 0.9036
6 236 0.9000 0.9002 6 22 0.9000 0.9015
8 133 0.9000 0.9001 8 13 0.9000 0.9074
10 86 0.9000 0.9020 10 9 0.9000 0.9188
12 60 0.9000 0.9023 12 7 0.9000 0.9361
14 44 0.9000 0.9007 14 5 0.9000 0.9156
16 34 0.9000 0.9018 16 4 0.9000 0.9091
18 27 0.9000 0.9017 18 4 0.9000 0.9555
20 22 0.9000 0.9016 20 3 0.9000 0.9095
slide 26
Objectives
The Hidden Factory Concept
− What is a Hidden Factory?
− What is a Measurement System’s Role in the Hidden
Factory?
Review Key Measurement System metrics including
%GR&R and P/T ratio
Case Study at W. R. GRACE
− Measurement Study Set-up and Minitab Analysis
− Linkage to Process
− Benefits of an Improved Measurement System
How to Improve Measurement Systems in an
Organization
slide 27
Measurement Improvement in the Organization
slide 28
Measurement Improvement in the Organization
slide 29
Final Thoughts
slide 30