Professional Documents
Culture Documents
3.1.2 Precision/Repeatability/Reproducibility
5 30.11.2021
3.1 Static Characteristics of Instruments (4)
3.1.3 Tolerance
Tolerance is a term that is closely related to accuracy and defines the maximum
error that is to be expected in some value.
where qi is the measured quantity, qo is the output reading, and ao ... an, bo
... bm are constants.
Linear
input-output
relationship
Nonlinear
input-output
relationship
Nonlinear
input-output
relationship
The smaller the spread of the measurements, the more confidence we have in
the mean or median value calculated.
The median value tends toward the mean value as the number of
measurements increases.
The variance (Vs) of the set of measurements is defined formally as the mean
of the squares of deviations:
Mean = 406.5
, …
n is the number of measurements
where is the mean value of the data set
With , we have:
when n, 0
a level of
significance ()
= the level of
random chance
100(1-)% of the
area under 2 the
distribution curve
The 2 test then examines whether the calculated value of 2 is greater than
that would be expected for a Gaussian distribution according to some
specified levels of chance.
In this case, . is the number of bins minus 2, because data are manipulated
twice to obtain the and statistical values used in the calculation of n’i.
In this example, . 8 2 6
Error in a sum:
Error in a difference
Error in a product
Both 4 and are very much less than one in magnitude and thus terms in 43 are
negligible compared with other terms. Therefore, we have,
It is very unlikely that the errors in 3 and will both be at the maximum or minimum
value at the same time. A statistically better estimate of the likely maximum error 5
in product 6, provided that the measurements are uncorrelated, is given by:
Error in a quotient