You are on page 1of 28

Measurement System Analysis

Welcome to the seminar


On
Measurement System Analysis -
4th Edition
By

Vinay Kumbhar
Vinay S K 1

Measurement System Analysis

INTRODUCTION OF FACULTY

Name : Vinay S. K.
More than 10 years Experience in Automotive Industry,
Home appliances and Quality Management systems.
Areas of Work : ISO 9001, TS 16949, ISO 17025 (NABL), and
Core tools
(Manufacturing Industry, Calibration Lab, Consultancy
Service
Email – vinayskumbhar@gmail.com

Vinay S K 2

1
Measurement System Analysis

Please keep your


mobiles Switched off /
on Silent Mode…

Vinay S K 3

Measurement System Analysis

OBJECTIVE

To provide

 Basic concept of measurement system

 Detail application methodology of all statistical


studies for variable and attribute data

 Methods to meet ISO/TS-16949 requirements

Vinay S K 4

2
Measurement System Analysis

Today’s Agenda
 Brief Changes and overview of 4th edition.

 Objectives

 Introduction

 ISO/TS 16949 Requirements

 Measurement System Analysis

 Selection of Instrument ( Discrimination )

 Statistical Properties of Measurement System

 Bias , Linearity, Stability & Gauge Repeatability & Reproducibility

 Graphical Analysis

 Attribute data Study & Conclusion

Vinay S K 5

Measurement System Analysis

Brief Changes in 4th edition


 Calibration and relationship to external systems ;e.g., ISO 17025.

 Analysis of Results

 The use of GRR guidelines as threshold criteria alone is not an


acceptable practice for determining the acceptability of a
Measurement system.

 Individual graphical analysis recommended.

 Application of Measurement system needs to be considered to have


lower GRR.

 Bias – calculation methods changed.

 Chapter no 4 and 5 –clubbed and made one chapter no 4


Vinay S K 6

3
Measurement System Analysis

Over View of Changes in 4th edition


 Calibration System

A Calibration System consists of both the documented and


technical processes that assure measurement traceability is

disseminated to the “shop floor”.

– Critical for assuring accuracy, precision and reliability of


measurements

– Calibration Events provide the links or traceability to higher


level standards

Vinay S K 7

Measurement System Analysis

Vinay S K 8

4
Measurement System Analysis

Vinay S K 9

Measurement System Analysis

GRR Analysis And Target Pp


Pp (or Ppk) target value

– use when sufficient samples to represent the process are not available and
an existing process with similar process variation is not available or the
new process is expected to have less variability than an existing process

• Use the following TV in the GRR analysis:

Vinay S K 10

5
Measurement System Analysis

APQP And MSA


PLAN AND DEFINE

In the Plan and Define Program stage the team is to identify preliminary special
product and process characteristics.

The organization’s product quality planning team should build on this listing and
reach consensus through the evaluation of the technical information.

The technical information should come from:

– the drawings

– industry standards such as SAE

– Engineering specifications

This is to be documented on the appropriate Control Plan

Vinay S K 11

Measurement System Analysis

 Gages/testing equipment requirements should also


be identified at this time.

 The organization’s product quality planning team


should add these requirements to the Timing Chart.

 Progress should be monitored to assure that required


timing is Met

Vinay S K 12

6
Measurement System Analysis

After gauge Test Equipment identified


The team must assess that proposed gauge /test system design:

– Understand which traceable standard are appropriate for the required testing

– The required frequency that the standards are to be used to assess the accuracy
of the measurement system

– Cost of the testing

– An estimate of the proposed system accuracy, precision, repeatability,


reproducibility, etc based on similar parts and similar measurement systems.

– The time required for the testing

– The frequency of the testing

This is a joint effort between design, quality and the supplier

Vinay S K 13

Measurement System Analysis

 The characteristics to be measured should be agreed to as early in the


program as possible. This should help prevent late gauging changes.

 The gauging methods should be established up front.

 This should also help prevent late gauging changes.

 The gauging must be used on the prototype build. An early gage study can be
performed. While the full range of process variation will probable not be
represented, as preliminary study can be performed.

 The gage can be verified at the prototype stage to ensure that the required
features are being correctly measured. This will help eliminate late changes to
the gauging method.

 Before using a gage to measure the product, the gage

must have acceptable bias and linearity, and repeatability

Vinay S K 14

7
Measurement System Analysis

Introduction
A decision to adjust a manufacturing process not is commonly based on
measurement data. Measurement data are also used in analytical studies to know
the relation between two or more variables, which in turn will lead to a better
understanding of process.

Benefit of using measurement data will depend on the Quality of data. Much of the
variations in a set of measurements is due to the interaction between the
measurement system and its environment .If the interaction generates too much of
the variations then the quality of the data may be so low that the data are not
useful.

If the Quality of the data is not acceptable, then it must be improved. This is usually
accomplished by improving the measurement system, rather than by improving the
data themselves.

Vinay S K 15

Measurement System Analysis

Effects of Measurement Systems

Effect on Product Decisions :

After measuring the part one of the action that can be taken to determine
the status of the part.

wrong decisions will be some times made

Eg,, some times good parts will be called as bad parts – Producers risk –
Eg
false alarm

Some times bad parts will be called as good part – Customer’s risk -

Miss rate.

Vinay S K 16

8
Measurement System Analysis

Introduction…
Variation :
It is the difference between two things or an entity .

Measurement System Variation :

Variation present in the measurement system affects the process results

Total variation :

It is the total variation observed in the process outputs

A) There can be variation due to Process

B) There can be variation due to Measurement system

Vinay S K 17

Measurement System Analysis

Introduction

Variation due
to Interaction
Total
Variation Appraiser
Variation

Equipment
Part to MSV
Within Part Variation
Part
Variation Variation
Always Measurement System variation must be
smaller than manufacturing process variation

Vinay S K 18

9
Measurement System Analysis

Selection of Instruments

Based on its Discrimination / Resolution

Discrimination : Capability of an instrument to indicate even


small changes of the measured characteristic.

Discrimination Decision Rules :

 Resolution of 1/10th of tolerance or process spread

 Study gage discrimination during APQP and test before PPAP

 For Continuous Improvement , 1/10th of tolerance may be


inadequate , MSA recommends 1/10th of six sigma (Total
manufacturing standard deviation).

Vinay S K 19

Measurement System Analysis

Statistical Properties of Measurement Systems

Accuracy : Absolute correctness of measurements compared with a


known standard. It is quantified by evaluating bias.

Precision: Ability to the measurement system to reproduce or


duplicate readings. It is quantified by evaluating repeatability as
well as reproducibility.
Refer Illustration 1 :

Measurement System Variation can be characterized by

1. Location : Stability, Bias, & Linearity

2. Width or Spread : Repeatability & Reproducibility

Vinay S K 20

10
Measurement System Analysis

BIAS – WHAT IT IS ?

Ref.
Value BIAS= X bar - Ref.Value

X bar
Vinay S K 21

Measurement System Analysis

BIAS - STEPS
1. Capture reference value
• Use masters calibrated- traceable to
standards
• Measure by better instrument than the
instrument being measured
• By Layout inspection

2. Measure the part/ master by Min. 10 times

Vinay S K 22

11
Measurement System Analysis

BIAS - STEPS
3. Calculate BIAS for individual reading

4. Calculate ,  bias, t limits

APPENDIX C t distribution table

Vinay S K 23

Measurement System Analysis

ACCEPTANCE CRITERIA

Zero must lie between t Limits of


Bias distribution or
Bias should be within acceptable gage
error as defined in Calibration Proc.

Vinay S K 24

12
Measurement System Analysis

Bias :
 Difference between observed average of measurements
and reference value

 Reference value also known as accepted reference value


or master value, is a value that serves as an agreed upon
reference for the measured values

 A reference value can be determined by averaging


several measurements with a higher level of measuring
equipment

Vinay S K 25

Measurement System Analysis

BIAS DATA SHEET


Part No.:- Date:-
Part Name:- Characteristics:
Instrument Used for Ref:-
Instrument under study:

Trial Reference value Observed Value

Reference value = Observed Value =

Bias =

Vinay S K 26

13
Measurement System Analysis

 BIAS CASE STUDY

 The inside diameter of a coupling is measured fifteen


times repeatedly by the same operator. The data is
given below.

 The reference value is 6.00.


 Bias = Observed Average – Reference Value
 OD Values observed:
5.8, 5.7, 5.9, 5.9, 6.0,
6.1, 6.0, 6.1, 6.4, 6.3,
6.0, 6.1, 6.2, 5.6, 6.0

Vinay S K 28

Measurement System Analysis

BIAS CASE STUDY

Ref. Value = 6.00 mm


Number of samples, n = 15
Average = 6.0067 mm
Bias = 6.0067 – 6.0000 = 0.0067 mm
r= Rbar / d2* = (6.4-5.6)/3.55333 = 0.22514
d2* from Appendix C for 15 trials
bias =  r / Sq. root of n
0.22514 / sq. root of 15
0.22514 / 3.873 = 0.05813
t-Static= bias / bias
0.0067 / 0.05813 = 0.1217
Upper & Lower Limits of [ bias +&– [bias x t(v,1-L/2]
t-static@ 95% Confidence = v, the degree of freedom, = n-1 = 15-1 =14
for confidence level of 95% & L= 0.05, factor for C/L =(1-L/2)
= (1-0.05/2) = 0.975
from t-static table for v=14 & 95%confidence level value t
value for bias is 2.51
[0.0067+&-{ 0.05813 * 2.51}
EV %= 100(EV/TV)= EV=  r , TV = process variation or tol/6

t-static Upper limit = +0.1526


t-static Lower limit = -0.1395

CONCLUSION: As observed t-static is between upper & lower limits our Bias Study is acceptable.
However observed t-static is not exactly in the middle or very near to mid value some action has to
be initiated before next study is due.

Vinay S K 29

14
Measurement System Analysis

 CASE STUDY ON BIAS

Vinay S K 30

Measurement System Analysis

LINEARITY

Difference in Bias over the Operating range


of the Gage
+ve Bias
- ve Bias

X bar X bar 31

Vinay S K

15
Measurement System Analysis

Linearity - Steps

1. Collect 5 parts / masters over the


operating range of the gage through process
variation
2. Capture reference value for the 5 parts

3. Measure at least 12 times in a random


manner to cover all 5 parts

Vinay S K 32

Measurement System Analysis

Linearity – Steps

4. Calculate Average, BIAS and enter the data


in a Spread sheet

5. Draw Best fit line for the average, Lower


limit and Higher limit of Bias.

Vinay S K 33

16
Measurement System Analysis

Linearity – Acceptance criteria


ZERO LINE MUST BE CONTAINED WITHIN THE
TWO LIMIT LINES OR WITHIN ACC.CRITERIA OF
CAL ERROR
+ VE Upper t limits

0
Lower t limits
- VE

Is this MS acceptable ?

Vinay S K 34

Measurement System Analysis

LINEARITY STUDY – DATA SHEET

Date:
Instrument under study:-
Part  1 2 3 4 5
Ref val 
1
2
T 3
R 4
I
A 5
L 6
S
7
8
9
10
11
12
Observed Avg

Bias

Vinay S K 35

17
Measurement System Analysis

LINEARITY STUDY – Example

Vinay S K 36

Measurement System Analysis

Stability study – What?


Variation of Bias w.r.t time when measured over a
Same Master

T1

T2

T3

Vinay S K 37

18
Measurement System Analysis
Stability :
Stability (or drift) is the total variation in the measurements obtained with a measurement
system on the same master or parts when measuring a single characteristic over an
extended time period.

Procedure for Stability study:


 Decide the part and characteristics for the study.

 Identify the location of measurement on the part.

 Make this part as a master part and ensure that the part is preserved till the study is
completed.

 Decide the subgroup size (3 to 5), no. Of subgroup (20) & frequency (every day) of data
collection.

 As per the frequency, measure the master part 3 to 5 times and record the details in
the Xbar-R chart.

 After completion of data collection, calculate each subgroup avg and range.

Vinay S K 38

Measurement System Analysis

Procedure for Stability study ( Contd..) :

 Calculate the upper control limit & lower control limit for range ( R ).

 Calculate the upper control limit & lower control limit for X double bar.

 Calculate the standard deviation (Sigma)

 Calculate the Sigma X bar.

 Calculate the 't‘ value.

 After calculating the 't' value compared that value with 't' distribution
table value.

 If 't' calculated value is less than or equal to 't' table value


@95% confidence level. It means variation is accepted.

Vinay S K 39

19
Measurement System Analysis

 Stability -example

Vinay S K 40

Measurement System Analysis

Repeatability (equipment variation)


Repeatability is the variation in measurements obtained with
one measurement instrument when used several times by
one appraiser while measuring the identical characteristic
on the same part.

Reproducibility (appraiser variation)


Reproducibility is the variation in the average of the
measurements made by different appraisers using the same
measuring instruments when measuring the identical
characteristic on the same part.

Gage Repeatability and Reproducibility study


Should be carried out for all measurement systems referred in
control plan.

Vinay S K 41

20
Measurement System Analysis
Procedure for Gage R&R study:
 Identify the measurement system for which R&R study to be conducted.

 Decide the part and the characteristics to be measured.

 Select the instrument and ensure the instrument is calibrated and is within
the acceptance criteria.

 Collect 5 to 10 parts randomly from a lot of production. Ensure the parts


collected are representative of the total process variation.

 Identify the location on the part where the measurement has to be done
repeatedly.

 Number each part.

 Choose 2 or 3 operators and have each operator measure all parts 2 or 3


times. Enter these results in the data sheet.

 Ensure that the readings are not shown to appraisers.

 Calculate AV, EV, PV, R&R & TV as per the data sheet and calculation sheet.

Vinay S K 42

Measurement System Analysis

Procedure for Gage R & R study - ( Contd….)


 Interpret the results based on the acceptance criteria given below :

R&R % below 10% - M.S. Is acceptable.


R&R % between 10 & 30% - M.S. Can be acceptable under
Concession, based on the cost of
repair, importance on product
quality,etc..
R&R% greater than 30% - Not acceptable. M.S. Needs
Improvement.
NDC =/> 5
Notes:
If EV is greater than AV – Act on instrument & method
If AV is greater than EV – Act on appraisers method of
checking, training, make gage design
more sturdy, etc..
Refer Annexure 4 for the Data Sheet & Analysis Sheet

Vinay S K 43

21
Measurement System Analysis
Graphical Analysis :
It is used to analyze the results of a measurement system. It provides
insight in to the variation model and inter- relationship.

-Average Chart

Since the group of parts used in the study represents the process
variation. Approximately one half or more of the averages should fall
outside the control limits.

- If data shows this pattern then the measurement system should be


adequate to detect part to part variation and the measurement system
can provide useful information to analyzing and controlling the process.

- If less than half falls outside the control limits then measurement
system lacks adequate effective resolution or the sample does not
represent the expected process variation.

Vinay S K 44

Measurement System Analysis

Graphical Analysis
-Range Chart

ranges of the multiple readings by each appraiser on each part


is plotted on range control chart.

if all ranges are within control all appraisers are doing the same
job.

if one appraiser is out of control limit , the method used by


appraiser differs from other.

If all Appraisers have some out of control ranges, Measurement


system is inadequate as appraiser methods –techniques needs
improvement.

Vinay S K 45

22
Measurement System Analysis

Attribute data Measurement

An Attribute Gage :
 Compares each part to a specific set of limits and accepts the
part if the limits are satisfied.( It detects the Nonconforming
parts)
 Is designed to accept / reject a set of master parts
 Cannot indicate how good or bad a part is, only whether the
part is accepted or rejected ( Pass/Fail )

Vinay S K 46

Measurement System Analysis

Attribute Gage Study :


Miss: Part is bad. Decision given is good.
False Alarm: Part is good. Decision given is bad.
Procedure for long method:
• Decide the no. Parts to be selected, no. Of appraisers & no. Of trials
for conducting the study.
• Identify all parts by a unique number and inspect the part to decide
the master value. This has to be approved by a expert.
• Have each appraiser measure the parts randomly for the no. Of trials
decided.
• Record the results as “good / bad” or “go/no go” or “accept / reject”,
etc..

Vinay S K 47

23
Measurement System Analysis

Procedure ( Contd…)
• After completion of collecting the data calculate the probability of
miss and probability of false alarm.
• Based on the probability of miss and false alarm, obtain the bias (miss)
and bias (false alarm) from the table.
• Calculate the effectiveness and bias factor as per the calculation
sheet.
• Interpret the effectiveness results as defined below.
Eff is > 0.9 - M.S is effective.
Eff is betwn 0.7 – 0.9 - M.S is acceptable under Concession
based on the importance, cost of
repair, etc..
Eff is < 0.7 - M.S is not effective.

Vinay S K 48

Measurement System Analysis

Data collection sheet

SL. Master
Appraiser and trails
NO status
I II III IV V

1 2 3 1 2 3 1 2 3 1 2 3 1 2 3

Vinay S K 49

24
Measurement System Analysis

Data evaluation sheet


Sl.No. Parameters Appraiser
I II III IV V
1 Total no.of samples (N)
2 No. of Trials (n)
3 No of good samples (NG)
4 No of bad samples (NB)
5 No of Miss (NM)
6 No of False Alarm (NFA)
7 No of Good Decision (NGD)
8 Probability of Miss P(M) = NM / NB X n
9 Probability of False Alarm P (FA) = N
FA / NG X n

10 Effectiveness (E) = N GD / N X n
11 * Conclusion

* Refer table

Vinay S K 50

Measurement System Analysis


Long Method
Evaluation criteria for inspection capability studies involving attribute data.
Parameter Acceptable Marginal Unacceptable
E 0.9 or More 0.8 to 0.9 Less than 0.8
P (FA) 0.05 or Less 0.05 to 0.1 More than 0.1
P (Miss) 0.02 or Less 0.02 to 0.05 More than 0.05
B 0.80 to 1.20 0.50 to 0.80 or Less than 0.5 or
1.2 to 1.5 More than 1.5

Special cases in computing bias


P (FA) P (Miss) B Decision or action
0 More than 0 0 Unacceptable
More than 0 0 No value Use E,P(FA),and P(Miss) directly
0 0 No value This is same as E=1 since ,P(FA)= P(Miss)
acceptable
More than 0.5 0.5 or Less More than 1.5 Unacceptable
0.5 or Less More than 0.5 Less than 0.5 Unacceptable
More than 0.5 More than 0.5 No value Bias unimportant, study is
unacceptable.Based on P(Miss) and
P(FA) being More than 0.5

Vinay S K 51

25
Measurement System Analysis

 Constants for evaluating attribute data


P(FA) or P(MISS) B(FA) or B(MISS) P(FA) or P(MISS) B(FA) or B(MISS)
0.01 0.0264 0.17 0.2541
0.02 0.0488 0.18 0.2613
0.03 0.0681 0.19 0.2707
0.04 0.0863 0.20 0.2803
0.05 0.1040 0.21 0.2874
0.06 0.1200 0.22 0.2966
0.07 0.1334 0.23 0.3034
0.08 0.1497 0.24 0.3101
0.09 0.1526 0.25 0.3187
0.10 0.1758 0.26 0.3251
0.11 0.1872 0.27 0.3312
0.12 0.1989 0.28 0.3372
0.13 0.2107 0.29 0.3429
0.14 0.2227 0.30 0.3485
0.15 0.2323 0.31 0.3538
0.16 0.2444 0.32 0.3572

Vinay S K 52

Measurement System Analysis

0.33 0.3621
0.34 0.3668
0.35 0.3712
 Constants for evaluating attribute data
0.36 0.3739
0.37 0.3778
0.38 0.3814
0.39 0.3836
0.40 0.3867
0.41 0.3885
0.42 0.3910
0.43 0.3925
0.44 0.3945
0.45 0.3961
0.46 0.3970
0.47 0.3977
0.48 0.3984
0.49 0.3989
0.50 0.3989

Vinay S K 53

26
Measurement System Analysis

 List of Measurement Systems


SL. Part Param Instrum Manufact
No. No eter ent urer Variable Attribute
Max/
Min
Bias Line Stabili R&R
arity ty

Vinay S K 54

Measurement System Analysis

Conclusion :

MSA study has become important since it can be applied


under the following areas
 Criterion to accept new instrument
 Comparison of one measuring device against another
 Basis for evaluating a gage suspected of being deficient
 Comparison of equipment before and after repair
 Required element for calculating process variation
 Develop Gage Performance Curve which indicates the
probability of gage accepting a part of some true value
Limitation of standard are it is Difficult to use in
destructive testing , Some Processes ,Product
Characteristics & Tests have no defined industry or
National standards

Vinay S K 55

27
Measurement System Analysis

Any

QUESTIONS ???

Vinay S K 56

Measurement System Analysis

Thank You !

Vinay S K 57

28