MSA For Attribute or Categorical Data IG PDF

MSA for Attribute or Categorical Data
MSA Example:
Attribute or
Categorical Data
All Rights Reserved, Juran Institute, Inc.
#### XXXX.#### vx

Definitions
Accuracy: Overall agreement of the measured value with the true value (which may be an
expert value). Bias plus precision.
Attribute Data: Discrete qualitative data.
Attribute Measurement System: Compares parts to a specific set of criteria and accepts the
item if the criteria are satisfied.
Bias: A systematic difference from the true value. Revealed in the differences in averages from
the true value.
Precision: Variation in the measurement process.
R&R: Repeatability and Reproducibility. Two elements of precision.
Repeatability: The variation observed when the same operator measures the same item
repeatedly with the same device.
Reproducibility: The variation observed when different operators measure the same parts using
the same device, sometimes it can be the same operator using different devices.
The list provides a quick reference for key terms used in Measurement System Analysis.
#### XXXX.#### vx

The Fundamental MSA Question
Is the variation (spread) of the measurement system

too large to study the current level of process
variation?
Product Variability
Process Variability
Variation
in the
measurement
process
(Observed Variability)
Total Variability
MSA for Continuous Processes 3 .PPT
Quantifying Variation
Like all processes, the measurement process has CTQs. The graph above lists
some of the most common CTQs used for the measurement process. MSA
quantifies the amount of variation for:
Accuracy
Repeatability
Reproducibility
Stability (this is typically covered in the Black Belt workshop)
Linearity (covered in Black Belt workshops)
#### XXXX.#### vx

Bias
True
Value or
Standard
Possible Causes of Bias

Sensor not properly
calibrated
Bias
Improper use of sensor

Unclear procedures
Observed
Average
Human limitations
Bias
Bias is the difference between the observed average of measurements and the true
average. Validating accuracy is the process of quantifying the amount of bias in the
measurement process. Experience has shown that bias and linearity are typically
not major sources of measurement error for continuous data, but they can be.
In service and transaction applications, evaluating bias most often involves testing
the judgment of people carrying out the measurements.
Example
A team wants to establish the accuracy of its process to measure defects in
invoices. First, they gather a standard group of invoices and have an expert
panel establish the type and number of defects in the group. Next, they have the
standard group of invoices measured by the normal measurement process.
Differences between averages the measurement process came up with, and what
the known defect level was from the expert panel represented the bias of the
measurement process.
#### XXXX.#### vx

Repeatability
Possible Causes of Poor

Repeatability
Equipment
Gage instrument needs
maintenance
The gage needs to be more
rigid
Repeatability
People
Environmental conditions
(lighting, noise)
Physical conditions (eyesight)
Repeatability
Repeatability is the variation in measurements obtained when one operator uses
the same measurement process for measuring the identical characteristics of the
same parts or items.
Repeatability is determined by taking one person, or one measurement device, and
measuring the same units or items repeatedly. Differences between the repeated
measurements represent the ability of the person or measurement device to be
consistent.
Possible causes of the lack of repeatability are listed on the slide.
#### XXXX.#### vx

Reproducibility
Mean of
the measurements
of Operator B
Mean of
the measurements
of Operator A
Possible Causes of Poor

Reproducibility
Measurement procedure is not
clear
Operator is not properly trained
in using and reading gage
Operational Definitions not
established
Reproducibility
Reproducibility
Reproducibility is very similar to repeatability. The only difference is that instead of
looking at the consistency of one person, you are looking at the consistency
between people.
Reproducibility is the variation in the average of measurements made by different
operators using the same measurement process when measuring identical
characteristics of the same parts or items.
Possible causes of poor reproducibility include: measurement process is not clear,
operator not properly trained in using the measurement system, and operational
definitions are not clear nor well established.
#### XXXX.#### vx

Attribute Measurement Systems Study
Discrete qualitative data
Go/no-go basis; or limited data categories
Compares parts to specific criteria for accept/not accept or
to be placed in category
Must screen for effectiveness to discern good parts from
bad
At least two appraisers and two trials each
If available, have Quality Master rate parts first
Listed here are the key highlights of conducting an MSA for attribute or categorical
data. The parts can be invoices, parts or reason codes for customer returns, for
example.
#### XXXX.#### vx

Attribute MSA Study
Attribute MSA Example

Appraiser A
Master
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
G
G
G
NG
G
G
NG
NG
G
G
G
G
G
G
NG
G
G
G
G
NG
Appraiser B
G
G
NG
NG
G
G
NG
NG
G
G
G
G
G
G
G
G
G
G
G
G
G
G
G
NG
G
G
NG
NG
G
G
G
G
NG
G
G
G
G
G
G
G
G
G
G
NG
G
NG
G
G
G
G
G
G
G
G
G
G
G
G
G
G
G
G
G
NG
G
G
NG
G
G
G
G
G
G
G
G
G
G
G
G
G
G = Good
NG = Not Good
This shows the results of 2 rounds using 2 appraisers, assessing the same 20
items.
#### XXXX.#### vx

Challenges of Continuous Process MSA
MSA study is an experiment

Requires two or more trials for calculating Repeatability
Needs a way to present the inspection units to the appraiser
multiple times
Is not possible within the continuous process
When conducting an MSA for a continuously running process, parts should be

taken off-line to conduct the MSA study.
#### XXXX.#### vx

Case Example: Visual Inspection of Glass
Packers
Catwalk
Cutter
Inspector
Glass
The example given here is that of visual inspection of glass.
10
#### XXXX.#### vx

Case Example: Challenges to Overcome
Bias to the standard could be evaluated on-line.

Repeatability and Reproducibility (R & R) could not be
evaluated on-line.
A method had to be devised to allow the inspectors to view
the same pieces of glass repeatedly.
The solution was an off-line conveyor which simulated the
on-line condition as closely as possible.
11
#### XXXX.#### vx

Case Example: Attribute MSA Method Employed
20 pieces of glass from the process that included both good

and bad samples were selected.
A team of people well versed in the quality standard
classified each piece of glass as either pass or fail.
All regular inspectors independently evaluated each piece
twice (in random order).
The inspectors used a log sheet to record the data.
Minitab was used to analyze the data.
There were two outcomes in this inspection or measurement process: pass or fail.
Twenty pieces, a team of inspectors, and two rounds (or trials) were used in the
MSA.
12
#### XXXX.#### vx

Case Example: Attribute MSA Study Data
Excerpt of
full data
for 20
inspectors
This slide shows the data in MINITAB. The Standard column documents the
correct or expert answer for each piece of glass.
13
#### XXXX.#### vx

Case Example: Attribute MSA Study Results
Date of study :
Reported by :
Name of product:
Misc:
Assessment Agreement
Within Appraisers
Appraiser vs Standard
100
95.0% C I
P ercent
90
90
80
80
Percent
Percent
100
9/13 and 9/19/07

Bill Sunderland and Bob Panatieri
Solar
Mainline Mark ing
70
60
95.0% C I
P ercent
70
60
50
50
40
40
l
t
n y o lly Tim E a r e v e ni e oh n n ie lin rry
n
n
a
To M
S t J ea J Ro C L
l
t
n y o lly T im E a r e v e ni e oh n n ie lin rry
a
n
n
To M
S t J ea J Ro C L
Appraiser
Appraiser
The graph on the left shows the agreement (or repeatability) of each appraiser
between Trial 1 and Trial 2.
The graph on the right shows the agreement of each appraiser with the Standard.
As shown on both graphs, the blue dots show the percent agreement and the
redlines are the 95% confidence intervals.
14
#### XXXX.#### vx

Case Example: Attribute MSA Study Results (continued)
This slide shows the Within Appraiser agreement.

For example, Larry scored 100% - Trial 1 and Trial 2 are in full agreement.
On the other hand, Allen scored only 50% agreement there is only a 50%
agreement between his Trial 1 and Trial 2 measurements. Or he disagrees with
himself half the time!
15
#### XXXX.#### vx

This slide shows the agreement of each Appraiser (across both trials) with the
Standard.
For example, Larry has 89% agreement with the Standard but, Allen only has 39%
agreement with the Standard.
16
#### XXXX.#### vx

This shows the level of agreement across all Appraisers. In this case, only 5.56%
agreement!
17
#### XXXX.#### vx

Case Example: Attribute MSA Study Conclusions
What could have caused the poor agreement?

What was done to improve consistency?
Given the results of the MSA study, what could have caused the poor agreement?
And what should be done to improve the measurement system?
The measurement system must be improved and tested again (with another MSA
study) to reach at least 90% agreement before the data can be used for baselining
process performance or further analysis.
18
#### XXXX.#### vx

MSA For Attribute or Categorical Data IG PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MSA For Attribute or Categorical Data IG PDF

Uploaded by

Copyright:

Available Formats

MSA for Attribute or Categorical Data

All Rights Reserved, Juran Institute, Inc.