Professional Documents
Culture Documents
STANDARD 9169
First edition
1994-l l-1 5
Reference number
IS0 9169:1994(E)
Foreword
IS0 (the International Organization for Standardization) is a worldwide
federation of national standards bodies (IS0 member bodies). The work
of preparing International Standards is normally carried out through IS0
technical committees. Each member body interested in a subject for
which a technical committee has been established has the right to be
represented on that committee. International organizations, governmental
and non-governmental, in liaison with ISO, also take part in the work. IS0
collaborates closely with the International Electrotechnical Commission
(IEC) on all matters of electrotechnical standardization.
0 IS0 1994
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced
or utilized in any form or by any means, electronic or mechanical, including photocopying and
microfilm, without permission in writing from the publisher.
m
International anization fo r Standa rdization
Case Postale 56 l CH-121 Geneve 20 l Switzerland
Printed in Switzerland
For measurement method stability surveillance under IS0 3534-l :I 993, Statistics - Vocabulary and sym-
routine measurement conditions, it may suffice to bols - Part I: Probability and general statistical
check the essential performance characteristics using terms.
simplified tests, the degree of simplification accepta-
ble being dependent on the knowledge of the invari- IS0 5725:1986, Precision of test methods - Deter-
ance properties of the performance characteristics mination of repeatability and reproducibility for a
previously gained by the procedures presented here. standard test method by inter-laboratory tests.
There is no fundamental difference between the in- IS0 6879: -31, Air quality - Performance character-
strumental (automatic) and the manual (e.g. wet- istics and related concepts for air quality measure-
chemical) procedures as long as the measured value ment methods.
I) The definition of method in IS0 6879:1983, 4.2.1.9 implies a specific instrumental setup.
2) This linearity may be enforced by a certain amount of postprocessing of the primary output variable.
3) To be published. (Revision of IS0 6879:1983)
Averaging of a continuous time series: The order of the objects or operations is then said to
be randomized. (See IS0 3534-l .)
@IA@) = -&I@
LI do @IT) . . . (2) 3.8 random variable: Variable which may take any
0
0
of the values within a specified set of values and with
which is associated a probability distribution.
In both cases, the original sample described by C^(Z)is [ISO 3534-l]
linked to a representative interval of time Z, whereas
;(A@), the result after application of the averaging 3.9 variance function: Variance of the output vari-
process, is made representative for the interval of able as a function of the air quality characteristic ob-
time A0 (just preceding O), the averaging time. served.
The averaging time, A@, is therefore the predefined 3.10 warm-up time: Minimum waiting time for an
and, by convention, common time interval for which instrument, after switching on, to meet predefined
the measured variable c^is made representative in the values of its performance characteristics stabilized in
sense that the square deviation of the original values, a non-operating condition.
attributed to time intervals z < A@, from c^ over A@
is a minimum. NOTES
NOTE 3 The averaging process can alternatively be car- 5 Pn practice, warm-up time can be determined by using
ried out by means of a special sampling technique (averag- the performance characteristic that is expected to need the
ing by sampling). longest time to stabilize.
6 In the case of manual procedures, the corresponding AiVi Difference of values of IVi
term is run-up time.
L Total number of measurements of in-
stability test
hot bl Parameters of the estimate function for Ni Number of values of the output variable
the calibration function at Ci
c, Weighted mean, with set of weights Normalization factor for the standard
SO
Ok deviation; in this International Standard
the value of so is assumed to be 1
DEP(c^), I First-order dependence of the meas-
ured value on the ith influence variable sbol sb, Estimate of the standard deviation of
at c instability (see IS0 6879) of the inter-
cept, and of the slope of the linear
D E w&v , First-order dependence of the intercept
calibration function, respectively
on the ith influence variable
Estimate of the standard deviation of
DWb,hv i First-order dependence of the slope on
instability at c
the ith influence variable
Estimate of the standard deviation of
DEP(xh i First-order dependence of the output
the experimentally determined cali-
signal on the ith influence variable
bration function (in units of the air
Drift (see IS0 6879) of the intercept of quality characteristic)
D(b,)
the linear calibration function
Estimate of the standard deviation of
Drift of the slope of the linear cali- the experimentally determined cali-
w, 1
bration function bration function (in units of the output
variable)
A
D(c) Drift of the measured value, c^,at c
si Estimate of the standard deviation of
F Statistic (cf. F-test) repeated xij at ci; j repetition index
iVi Value of Iy
tv;q q-quantile of the t-distribution with v or perfo rming a series of calibration experime nts in
degrees of freedom order to determine performance chara cteristics.
TC Test characteristic of Grubbs’ outlier NOTE 7 Under certain conditions it may be suitable to
test test only one step or a selected group of steps of the
measurement method. Under other conditions it may not
X Output variable be possible to include all steps of the measurement
method. It is recommended to include as many steps as
Value of X possible.
Estimate of x
5.2 Specification of performance
Estimate of output signal at cj characteristics to be tested
Mean of the set of output signals at ci The performance characteristics of the measurement
method shall be specified in order of their relevance
x*l,extr Output signal at ci with the highest ab- for the final assessment of accuracy. The descriptors
solute distance from q of the calibration function, i.e. intercept, PO,and slope,
DI, as well as their qualifying performance character-
4j jth output signal at ci istics are vital. Those performance characteristics for
Output signal after i time intervals at which prior knowledge is available, and those per-
4;if 4J;i
the lower and upper value of the air taining influence variables covered by randomization,
quality characteristic of the reference are of lesser importance and need not be determined.
material
5.3 Test conditions
Weighted mean of the whole set of
output signals within the calibration Perform the tests under explicitly stated conditions
experiment which must be representative of the operational
measurements. When testing for statistical perform-
ance characteristics, all specified influence variables
Bo, B1 Intercept and slope of the linear cali- shall remain constant. When testing for performance
bration function, respectively characteristics describing functional dependencies, all
influence variables shall remain constant except the
0 Time one under consideration.
V Number of degrees of freedom in the By convention, the statistical performance character-
calibration experiment istics used in this International Standard are estimated
Number of degrees of freedom for the throughout at the confidence level 1 - a = 0,95.
VP V2
numerator of the F-distribution, re-
spectively
0 Sample
P
Preparation
I
I
t
-I
1 output [
Calibration
1
signal [ --------------
function
I
i
- Measurement
----- Calibration
4) The functional and statistical performance characteristics may be calculated on a computer using the TurboPascal program
adjunct to ASTM Standard D 5280, available from ASTM, 1916 Race St., Philadelphia PA 19103-I 187, USA.
Ll *
u‘
.- ai
t 7;
.-ul .-0x3
L
taJ ki
>
2
2 3
a
f c
U
h 2
t
.-
-F;
z-
.-L
<
- \
c3
r l-
*2
0,l I
Xl
c2
.
Cl
m w -
Time, 8
Lag time D- Lag time --
A linear calibration function is determined by its slope NOTE 8 Repetitions performed under reproducibility
(sensitivity) and its intercept. Instability and the ef- conditions (see IS0 5725) require a random sample of the
fects of influence variables are described by their im- population of the influence variables to be examined
(randomization).
pacts on the slope (sensitivity) and intercept.
All ouput signals evaluated throughout these tests 6.2.1 .I Elimination of outliers
shall be obtained after the measuring system has
reached stabilized conditions. Usually, experience helps to identify potential outliers.
A less arbitrary way of detection of such potential
6.2.1 Calibration outliers is given by combination of this experience
with, e.g., Grubbs’ test [I]. However, it should be
A calibration experiment for the evaluation of per- clear that such a test identifies potential outliers. The
formance characteristics consists of at least ten re- underlying reasons may be statistical or due to sys-
peated measurements at a minimum of five different tem operation interferences. The latter presents suf-
values of the air quality characteristic. ficient grounds for elimination of the respective output
signal (confirmation as an outlier).
If drift occurs, the duration of the calibration exper-
iment shall be kept as short as possible. This may be Estimate the standard deviation sj at cj by:
accomplished by consecutive instrument readings at
a certain value of the air quality characteristic and after
a change in that value and stabilization, again con-
secutive instrument readings at that value, etc. (see
figure 3). This is only valid in the absence of hysteresis
or if hysteresis is negligible.
II
Output signal
b
u‘
.-
c
.-w Unsmoothed
ck
y 3.1
ki nq,
2
fU
h
.-
“d
s
.-L
a
c2
Cl
Time, 8
Averaging time -4 A& A@2- 1
4j jth time average over the interval of time A0 at the ith value of the air quality characteristic, generated by reference
material
AOi Intervals of time during which unsmoothed output signals shall not be submitted to the averaging procedure and,
thus, not be evaluated
At ci, take the ouput signal Xiextr with the highest ab- may be repeated; however, if more than 5 % of the
solute distance from the mean output signal xi. Derive number of output signals is rejected this way, this
the test characteristic calibration experiment is not valid.
TC = IxiIextr - ql/si . . . (4) If operational reasons are not found for TC exceeding
the critical value, the potential outlier may not be
where eliminated. In this case, verification of a basic test
cj -%j
assumption or prerequisite is recommended.
INi . . . (5)
and compare it with the tabulated value of Grubbs’ 6.2.1.2 Computation of the variance function
two-sided outlier test (see annex A) to be taken as
the critical value. A central tool for the estimation of relevant perform-
ance characteristics is the variance function. There-
If TC exceeds the critical value, check if it is due to fore, some guidelines for its computation, and the
operational reasons, and if so, reject it. This procedure computation of related parameters are given.
Compute for each of the values ci (i = 1 to M) of the Element Qt,- OnIis obtained by substituting [ with z and
air quality characteristic the variance So?of the output q with z or y’:
signals xii (j = 1 to Ni): 2
yi= log+; Zi= $
J-
. . - (10)
SO
a, =
[Qci,y)Q(z2,
z2)
- Q(z2, y) Q(z, z2J
..0
[ Q(z, z) Qg, 22)- (Q(z,z’$2]
-G
ii
.-d
=i
->
cn
0
A
cc
b,=i j Wi
Xii
(Ci
- C,) Compute the analytical function
bration function:
by inverting the cali-
b. = i& - b, Fw . . . (15)
6.2.1.5 Linearity
and
The hypothesis of linearity of the calibration function
c, = x, Ni *i ci / 7, Nk Ok (see figure 5) is examined using the F-test [6]:
i k
~~ = ~~
i j
Wi Xii / ~
k
Nk Ok . . . (16) F= ~ Ni cc)i(‘-~‘)* 1 Iv1
. . . (21
2
COi(Xij - ZJ /V*
In addition to the various standard deviations defined I
as descriptors for the mutual scattering of true values,
measured values and output signals, there arises a where
special scatter to be attributed to the estimation
process as a whole. v,=M-2
Sometimes the output signal is obtained after cor- If F does exceed the critical value, the hypothesis of
rection for the blank. When the blanks correspond to linearity must be rejected. The question whether
genuine zero samples, the corrected calibration func- non-linearity is substantial as compared to other un-
tion must pass through the origin. In this case the certainties may be tested by determining whether the
coefficient, b,, reduces to: following inequality criterion holds:
b1;trf =
cc Oi
i j
Xij Ci
. . . (18)
M:x
i=l
<I . . . (22)
2
c Nk Ok ck If the inequality criterion is met (see figure 5) the
k
subsequent performance characteristics can be cal-
culated. If the inequality criterion is not met, the de-
The standard deviation, sic, is invariant to the trans-
termination of performance characteristics must be
formation, but the number of degrees of freedom
terminated. For the latter situation, the following pro-
changes to:
cedure is recommended:
%f =
. . . (W a) Examine the quality of reference material samples
as a potential cause for non-linearity.
b) If, based on the result of this examination, the [3] by the estimate stx for the respective standard
problem cannot be solved, examine whether the deviation (cf. 6.2.1.3)
subrange in which the inequality criterion is ful-
filled contains the region of interest, or test for a
monotonic transformation with a monotonic first SXC
^
1
derivative to reduce the deviation from linearity. 9x
=- +I
bl
Ni ai
c) If the possibility of reducing the deviation from i
6.2.1.7.1 Repeatability
LDL = tv.o
I Ig5J $ + szx (s,. and six at C = 0) . . . (29)
Repeatability, r, is calculated using the variance func-
tion referring to the corresponding conditions (see
IS0 5725). 6.2.1.10 Upper limit of measurement
Calculate the smoothed variance, G*(C), (see 6.2.1.2) The upper limit of measurement is approximated by
and hence estimate the repeatability standard devi- the value of the air quality characteristic correspond-
ation by: ing to the maximum measured value confirmed by the
calibration process.
r = 4-o
I I975 Sr-J- 2 . . . (26) 6.2.2 Instability
where tV.Og75 is the tabulated value ty., -ar12 of the Performance characteristics are assumed not to
t-distribution for the two-sided test for the significance change with time. However, in practice they do. In
level a = 0,05 (see annex C) and for v degrees of particular, a change in the coefficients b. and b, of the
freedom (v = Min (Ni - 1)). calibration function may considerably influence the
accuracy of the measured value. The change in the
NOTE 9 The presence of the factor r 2 is due to the coefficients over a stated period of time (instability)
fact that Y and R, by definition, refer to the difference be- may have a systematic part (drift) and a random part
tween two single measurements. (dispersion). It is assumed that the value of drift is a
constant. The value of the dispersion standard devi-
ation is equal to or greater than the repeatability
6.2.1.7.2 Reproducibility standard deviation.
The performance test for reproducibility is described Drift and dispersion are derived from the linear re-
in IS0 5725. gression of the output variable with time, where the
time interval between successive output signals is the
6.2.1.8 Measurement resolution time interval of interest (figure6). Drift is equal to the
slope of the regression function and dispersion is
Measurement resolution at C = c may be estimated measured by the standard deviation of the residuals.
.
bY.
Calculate the variance, t*(O), at C = 0 from the vari- Use reference materials of C = cI and C = C, (c, in the
ance function (6.2.1.2). The repeatability standard de- lower and c, in the upper part of the range of
viation is then, according to 6.2.1.7: measurement; cl < cJ.
*
ai
z;
.-dL
d
t>
t2
z
Time, 8
i i 22 22
Cu Sl - Cl %
sbo = 2 2 . . . (35)
r
i
2 2
su - Sl
For C = c,, compute the corresponding values of pu sb, = 2 2
. . . (36)
and s,. cu - Cl
Finally, the contribution of dispersion to instability is obtained under field conditions? It is assumed that
expected to be: the impact of the influence variable on the measured
value can be adequately determined by tests at the
Sinst . ..(37) extremes (see figure 7).
AIIr-
IlrIE
III
b
L_I’ Time, 8
From
x =X(63)
IV 5 Me)
.-u‘
t
.-ln
c
0 1
c=- CM
aJ
t
U
d
2
r
U
A
t.-
-2
.-
:
Time, 0
Record for each of the values of C the difference in Using the limiting value of the performance charac-
output signal, AX, on going from one extreme test teristics taken into account (see 6.3.1), investigate the
value IV’ to the other. critical performance characteristic limiting the period
of unattended operation.
Compute the dependence, DEP, on the influence
variable, n/it at C = ck, k = I, u Investigate the most unfavourable operating con-
ditions to be expected.
DEP(&c I = * I&c ... (39)
i Perform the necessary maintenance operations.
The dependence of b. and b, on the influence variable Initiate the measuring system according to the oper-
is shown by: ating instructions at the most unfavourable operating
conditions with the measuring system warmed up or
c, DER-Gv;l~l
- q DEP(x),~;lc run up. Record the time elapsed until stabilization.
DEP(b,),I =
U
. . . (40)
Cu - Cl
Annex A
(normative)
Tabulated values of Grubbs’ two-sided outlier test for the significance level a = 0,05 are given in table A.I.
Table A.1
Tabulated value
Number of replicates
(critical value)
3 1,155
4 1,481
5 1,715
6 1,887
7 2,020
8 2,126
9 2,215
10 2,290
11 2,355
12 2,412
13 2,462
14 2,507
15 2,549
16 2,585
17 2,620
18 2,651
19 2,681
20 2,709
25 2,822
30 2,908
40 3,036
50 3,128
Annex B
(normative)
F-distribution
Table B.1
.
Number of degrees of freedom of variance in the numerator
Denominator
Vl
I v2
1 2 3 4 5 6 7 8 9 IO 11 12
I 40 1 4,08 1 3,23 1 2,84 1 2,61 1 2,45 ) 2,34 12,25 12,18 12,12 12,08
208 12,04 12,00
2,OO
50 4,03 3,18 2,79 2,56 2,40 2,29 2,20 2,13 2,07 2,03 I,99 I,95
60 4,00 3,15 2,76 2,53 2,37 2,25 2,17 2,lO 2,04 I,99 , I ,95 I,92
I 100 I 3,94 I 3,09 2,70 1 2,46 1 2,31 1 2,19 1 2,lO ( 2,03 1 I,97 1 I,93
I ,93 1 I,89 1 I,85
I 120
120 ) 3,92 ( 3,07 1 2,68 1 2,45 1 2,29 1 2,18 1 2,09 1 2,02 1 II,96
,96 1 II,91
,91 1 I,87 ( I,83
1,83
00
00 3,84
3,84 3,00
3,00 2,60
2,60 2,37
2,37 2,21 2,lO 2,Ol I,94 I,88 I,83 I,79 I,75
F v,,v,;o,95= IO”
where
A=
I,4287 - 0,681 (v* - v,)
v1v2
2 vp2 / (v, + v2) - 0,95
Annex C
(normative)
t-distribution
Tabulated values of the t-distribution for the significance level c1= 0,05 are given in table C.l .
Table C.l
Number of degrees of
One-sided case Two-sided case
freedom
V tv;l -a = 4;0,95 G;l -a/2 = tv;o,975
1 6,314 12,706
2 2,920 4,303
3 2,353 3,182
4 2,132 2,776
5 2,015 2,751
6 1,943 2,447
7 1,895 2,365
8 1,860 2,306
9 1,833 2,262
IO 1,812 2,228
II 1,796 2,201
12 1,782 2,179
13 1,771 2,160
14 1,761 2,145
15 1,753 2,131
16 1,746 2,120
17 1,740 2,110
18 1,734 2,101
19 1,729 2,093
20 1,725 2,086
30 1,697 2,042
40 1,684 2,021
60 1,671 2,000
00
00 1,645 1,960
Annex D
(informative)
Bibliography
Cl1 GRUBBS, F. E. and BECK, G. Extension of sample PI GREEN, J. R. and MARGERISON, D. Statistica/
sizes and percentages points for significance Treatment of Experimental Data. Elsevier,
tests of outlying observations. Technometrics, Amsterdam, 1978.
14, 1972, pp. 847-854.
I31 DRAPER, N. R. and SMITH, H. AppliedRegression
PI GARDEN, J. S., MITCHELL, D. G. and MILLS, W. Analysis. Wiley, New York, 1966.
N. Non-constant variance regression techniques
for calibration-curve-based analysis. Anal. L-51NATRELLA, M. G. Experimental statistics. In: NBS
Chem., 52, 1980, pp. 2310-2315. Handbook 91, 1966.
ICS 13.040.10
Descriptors: air, quality, air pollution, measurement, determination, measurement characteristics.