Measurement error

The terminology and definitions used in different

industries

Roland Caulcutt
Caulcutt Associates
Measurement

How do people use measurements?

People use measurements to help them:

1. Control a process
2. Assess the capability of a process
3. Improve a process
4. Classify a product or service
5. Settle customer/supplier disputes

2
 The classification of products and
services
When there is no measurement error we can
distinguish between good and bad products,
with confidence.

Not Not
acceptable Acceptable acceptable

LSL USL Quality

3
Classification with measurement error
When there is measurement error we may be left in
doubt about the acceptability of the product or
service.

Uncertainty Uncertainty
????? ?????
Not Not
acceptable Acceptable acceptable

LSL USL Quality

4
Terminology

Accuracy
Bias
Trueness
Precision
Repeatability
Reproducibility
R&R
Measurement systems analysis
Measurement uncertainty

Unfortunately some of these terms have different

meanings in different organisations.
5
Measurement in the chemical industry
Lab A

19.5 20.0 20.5 21.0 Viscosity

Lab B
Poor precision

19.5 20.0 20.5 21.0 Viscosity

Lab C Bias/Trueness

19.5 20.0 20.5 21.0 Viscosity

Lab D

19.5 20.0 20.5 21.0 Viscosity

True value 6
Bias and trueness C

For centuries scientists spoke of bias and precision.

BS5497:1979 used the words bias and precision.

BS5725:1994 uses the words trueness and precision.

7
Repeatability and reproducibility

Precision can be split into two components:

Repeatability
Reproducibility.

Repeatability:
Can the person producing the measurement
repeat in the future what he/she did in the past?

Reproducibility:
Can a second person reproduce the
measurement made by the first person?
8
Definitions in the chemical industry C

Repeatability is a number that is unlikely to be

exceeded by the difference between two
measurements made under repeatability conditions.
(Repeatability = 2.82s)

Reproducibility is a number that is unlikely to be

exceeded by the difference between two
measurements made under reproducibility
conditions.
(Reproducibility = 2.82s)
9
Calculation of repeatability and
reproducibility C

Repeatability = 2.82(Repeatability sd)

Reproducibility = 2.82(Reproducibility sd)

Reproducibility sd =
[(Repeatability sd)2 + (Between labs sd)2]

10
A procedure for the chemical industry C

For each level:

send 2 samples
to each of 10 laboratories.

Use the 20 results to estimate repeatability and

reproducibility, at each level.

11
Measurement uncertainty C
Since about 1993 analytical chemists have
increasingly used measurement uncertainty to
assess measurement error.

Measurement uncertainty is an interval within which

the true value is believed to lie.
Measurement uncertainty = x + ks
x is the measured value, s is a standard deviation
k is a coverage factor.

12
Measurement in mechanical engineering
Smith

19.5 20.0 20.5 21.0 Diameter

Jones
Poor precision

19.5 20.0 20.5 21.0 Diameter

Brown Inaccuracy

19.5 20.0 20.5 21.0 Diameter

Lee

19.5 20.0 20.5 21.0 Diameter

True value 13
Repeatability in engineering E
Repeatability
is
the measurement variability
found
when QS9000
the same item is measured repeatedly
with a specific gauge
by
the same operator.

Repeatability is often called Equipment variation

14
Reproducibility in engineering E
Reproducibility
is
the additional measurement variability
found
when QS9000
the same item is measured repeatedly
with a specific gauge
by
different operators.

Reproducibility is often called Appraiser variation

15
R&R E
R&R
is
the total measurement variability
found when
the same item is measured repeatedly QS9000
with a specific gauge
by
different operators.

R & R depends on equipment variation and

appraiser variation.

16
Measurement variability E
Measurement variability is defined as the range within
which 99% of the repeat measurements can be
expected to lie.

99% of repeat measurements

will lie within a band, which
has a width of 5.15s.

99%
-2.575s Mean
2.575s

17
Calculation of R&R E
Repeatability = 5.15sE
sE = SD of repeat measurements by a typical operator.
Reproducibility = 5.15sA
sA = SD of operator true mean measurements.

R&R = 5.15sT
sT = SD of repeat measurements by different operators.

QS9000
18
R&R and product classification E
We can express the R&R as a percentage of the
tolerance.

%R&R = 100*R&R/(USL-LSL)

%R&R should not be more than 30% of tolerance.

It is desirable that %R&R is less than 10%. QS9000

If the purpose of measuring is process

improvement, we should express the R&R as a
percentage of the product variation.
19
 Can you demonstrate your quality? E
R&R = 100%

LSL USL

R&R = 30%
LSL USL

R&R = 10%
LSL USL

R&R = 0%
LSL USL
20
A procedure for engineering industry E

Select 10 components.
Ask assessor A to measure each component twice.
Ask assessor B to measure each component twice.
You may wish to use a third assessor.

Analyse the 40 (60) results, using MINITAB, to

estimate repeatability, reproducibility and R&R.

Express R&R as a % of:

1. Tolerance
2. Product variation
21
Repeatability differences
Engineering definition:
Repeatability is the spread of measurements made
under repeatability conditions.
(Repeatability = 5.15s)

Chemical definition:
Repeatability is a number that is unlikely to be
exceeded by the difference between two
measurements made under repeatability
conditions.
(Repeatability = 2.82s)
22
Reproducibility differences

Engineering definition:
The additional spread within measurements made
under reproducibility conditions.

Chemical definition:
Reproducibility is a number that is unlikely to be
exceeded by the difference between two
measurements made under reproducibility
conditions.

23
R&R

Engineering definition:
The total spread of measurements made under
reproducibility conditions.

The chemical industry does not use R&R.

R&R(Engineering) is similar to Reproducibility(Chemical)

24
 Variability
To understand the usefulness of your measurement process
you need to calculate the total variation and break this down
as follows:
Total variation

Product variation Measurement system variation

Short Long Repeatability Reproducibility

term term

Operator Operator by
part
interaction

25
Summary
Repeatability and reproducibility have different
meanings in different industries.

R&R should be less than 30% of tolerance, and

preferably less than 10%.

Using only 2 or 3 assessors can be a weakness of the

recommended procedure in QS9000.

Graphing the results may tell you much more

about the measurement system than you would
learn from the analysis.

26
Bibliography
QS 9000 Superseded by: PD ISO/TS 16949: 2009 Quality
management systems. Particular requirements for the
application of ISO 9001:2008 for automotive production and
relevant service part organizations. (Section 8 - Measurement
Analysis and Improvement)

BS ISO 5725-2: 1994 Accuracy (trueness and precision) of

measurement methods and results. Basic methods for the
determination of repeatability and reproducibility of a standard
measurement method.

BS EN ISO 4259: 2006 Petroleum products. Determination and

application of precision data in relation to methods of test.

Quantifying uncertainty in analytical measurement , Second

edition (2000) - Eurachem
27
Figure 1 Measurements by four operators

Smith

19.5 20.0 20.5 21.0 Diameter

Jones

19.5 20.0 20.5 21.0 Diameter

Brown

19.5 20.0 20.5 21.0 Diameter

Lee

19.5 20.0 20.5 21.0 Diameter