Attribute Gage R&R

An Overview

Presented to ASQ Section 1302

August 18, 2011
by Jon Ridgway

Copyright: JC Quality Business Solutions LLC

Overview

What is an Attribute Gage R&R?

Why is it worth the work?
What are the caveats?
How do I perform it?
How do I understand the results?

Copyright: JC Quality Business Solutions LLC

My Attribute Gage R&R Experience

First Data
Credit Card mailings (packages)

Airlite Plastics:
Mold & Print Quality
Print Quality, Cup & Lid Decoration

Copyright: JC Quality Business Solutions LLC

WHY conduct a Gage R&R????

Some processes require subjective decision
making:
Inspection
Validation

Subjectivity creates the potential for variation

Measurement System variation impacts process
capability:
Type I Errors
Type II Errors

Copyright: JC Quality Business Solutions LLC

So, Why Conduct a Gage R&R?

To understand:
How likely Appraiser will agree with himself / herself:
WITHIN / Repeatability
How likely all Appraisers will agree with each other:
BETWEEN / Reproducibility

Understanding R&R allows you to:

Predict probability (%) of agreement / disagreement
Implement training to improve that probability
Reduce Type I and Type II Errors = $$$

Copyright: JC Quality Business Solutions LLC

MSA
The Foundation of everything in Quality is
measurement
Measure for two primary reasons:
To make a decision
As the basis for process improvement

Can we trust our measurement system to

give us reliable data?
CONFIDENCE
Copyright: JC Quality Business Solutions LLC

Ultimate purpose of the Attribute

Agreement Analysis
To determine if your measurement system
can distinguish between a good & bad part
Accuracy & Precision:
Accuracy:
Absence of bias, or agreeing with the standard.

Precision:
Ability of different Appraisers to reach the same
conclusion several times.

Copyright: JC Quality Business Solutions LLC

Accurate, But Not Precise

Copyright: JC Quality Business Solutions LLC

Precise, But Not Accurate

Copyright: JC Quality Business Solutions LLC

Which is Easier to Remedy?

Accurate, but not Precise

Precise, but not Accurate

Copyright: JC Quality Business Solutions LLC

Gage R&R Review

Measurement System Analysis (MSA)
1st R: Repeatability
2nd R: Reproducibility

Data in General:
Continuous / Variables
Attribute / Discrete

Copyright: JC Quality Business Solutions LLC

Attribute vs. Continuous

Attribute Data:
Categorical, named only, arbitrary scales
Also known as Discrete Data

Continuous Data:
Allows for infinitely finer sub-divisions
Also known as Variables Data

Copyright: JC Quality Business Solutions LLC

Basic Data Types

Nominal:
Literally, name
Represents categories

Ordinal:
Ordered or ranked data
Not scaled

Interval:
Measured / scaled data: Each position equidistant
0 can be relevant (temperature)

Ratio:
Numbers compared as multiples of one another
Copyright: JC Quality Business Solutions LLC

Hierarchy of Data Types

Nominal

Ordinal

Interval

Ratio

Classified Data

Quantified Data

DISCRETE / ATTRIBUTE

CONTINUOUS / VARIABLE

Non-parametric

Parametric

Copyright: JC Quality Business Solutions LLC

2 Main Attribute Gage R&R Types

1) Binary / Nominal
GO / NO GO
Data are Categorical and mutually exclusive
Kappa statistic is relevant

2) Ordinal
Rank, not categorical
Data are not mutually exclusive
Kendalls statistic more relevant than Kappa

Copyright: JC Quality Business Solutions LLC

Kappa Statistic
Proportion of agreement between
evaluators after chance agreement has
been removed:
Kappa = P observed P chance / P chance

Expressed as a number:
From 0 (expected by chance)
Up to +1 (complete agreement)
Copyright: JC Quality Business Solutions LLC

Kendalls Statistics
Two different Kendalls for different tests:
Kendalls Coefficient of Concordance:
Rankings without a known Standard

Kendalls Correlation Coefficient:

Rankings with a known Standard

Expressed as 0 (weaker agreement) to +1

(stronger agreement)

Copyright: JC Quality Business Solutions LLC

Kappa & Kendalls Summary

Kappa:
Nominal / Binary Only
Match or No Match

Kendalls Coefficient of Concordance:

Ordinal but not using a known Standard

Kendalls Correlation Coefficient:

Ordinal and using a known Standard

Copyright: JC Quality Business Solutions LLC

Attribute Gage R&R Considerations

Study Purpose
Destructiveness
Precision vs. Time
Binomial / Nominal vs. Ordinal

Copyright: JC Quality Business Solutions LLC

MSA Factors Impacting Variation

Gage
Appraiser
Method
Part
Environment

Copyright: JC Quality Business Solutions LLC

Controlling MSA Factors

Ideally:
1. Use the same Assessment Method
2. Require all Appraisers to assess the same
dimension / feature / sample
3. Conduct the study under the normal
assessment conditions

Copyright: JC Quality Business Solutions LLC

Controlling MSA Factors, Cont.

1) Appraisers: Select from group that
normally appraises the part.
2) Number of parts should cover the entire
range of variation.
3) More than one appraisal per Appraiser
should be done.
4) The presentation of the samples within
the Trial should be randomized.
Copyright: JC Quality Business Solutions LLC

Nominal / Binary Study

Two Appraisers
50 Parts
2 Trials

Copyright: JC Quality Business Solutions LLC

Copyright: JC Quality Business Solutions LLC

Copyright: JC Quality Business Solutions LLC

Output: Within

Copyright: JC Quality Business Solutions LLC

Output: Within vs. Standard

Copyright: JC Quality Business Solutions LLC

Date of study:
Reported by:
Name of product:
Misc:

Assessment Agreement

Within Appraisers

Appraiser vs Standard
95.0% C I
P ercent

100

95
Percent

95
Percent

95.0% C I
P ercent

100

90

90

85

85

80

80
Lee

Fred

Lee

Appraiser

Fred
Appraiser

Copyright: JC Quality Business Solutions LLC

Output: Between

Copyright: JC Quality Business Solutions LLC

Output: Between vs. Standard

Copyright: JC Quality Business Solutions LLC

Ordinal Case Study: Print Quality

Copyright: JC Quality Business Solutions LLC

What did we want to know?

Do all Appraisers of Print Quality:
Agree consistently with Themselves?
Agree consistently with Each Other?
Given our world, we have an ordinal system:
Accept
Accept but Adjust
Reject

Copyright: JC Quality Business Solutions LLC

How was it Done?

10 samples, Good & Bad

Random Order, Same for All
2 Trials per person
All people in the study
Environment

Copyright: JC Quality Business Solutions LLC

Ensure Gage R&R Consistency

Copyright: JC Quality Business Solutions LLC

Spanish Version

Copyright: JC Quality Business Solutions LLC

Vietnamese Version

Copyright: JC Quality Business Solutions LLC

Trial Order

My Checklist

Copyright: JC Quality Business Solutions LLC

Copyright: JC Quality Business Solutions LLC

Results
Sample QA1-1 QA1-2

QA2-1

QA2-2

QA4-1

QA4-2

10

Copyright: JC Quality Business Solutions LLC

Standard

2
3
2
3
2
2
3
3
1
1

False Alarms & Misses

Assess Fail when Standard = Pass:
False Alarm
Type I Error

Assess Pass when Standard = Fail:

Miss
Type II Error

Copyright: JC Quality Business Solutions LLC

False
Alarms

Results
Sample QA1-1 QA1-2

QA2-1

QA2-2

QA4-1

QA4-2

10

MISSES

Copyright: JC Quality Business Solutions LLC

Standard

2
3
2
3
2
2
3
3
1
1

Minitab 15

Four Results:
1.
2.
3.
4.

Within
Within vs. Standard
Between
Between vs. Standard

Copyright: JC Quality Business Solutions LLC

Check Here
Copyright: JC Quality Business Solutions LLC

Copyright: JC Quality Business Solutions LLC

Copyright: JC Quality Business Solutions LLC

Date of study :
Reported by :
Name of product:
Misc:

Assessment Agreement

Within Appraisers

Appraiser vs Standard
100

95.0% C I
P ercent

80

80

60

60

Percent

Percent

100

40

20

0
QA-1

95.0% C I
P ercent

40

20

QA-2
Appraiser

QA-4

0
QA-1

Copyright: JC Quality Business Solutions LLC

QA-2
Appraiser

QA-4

Copyright: JC Quality Business Solutions LLC

Copyright: JC Quality Business Solutions LLC

Two Big Lessons

You cant trust your data until it is proven

to be trustworthy.
A single, one-time Gage R&R study is
not enough

Copyright: JC Quality Business Solutions LLC

Questions?
Thank You!

Copyright: JC Quality Business Solutions LLC