You are on page 1of 48

. .

4851903 0594509 493 11


N

929

INTERNATI~NAL ISO

STAN~AR' 5725-2

First edlticn
1994-12-15

...~,

~'1..t.,;¡&"::'j i
>
;,r",~-'\l:;;t~"~""~ .A
J:w .. j jo ~~ ¿~~¿ .0.0.::..

Accuracy (trueness and precision) of


measurement methods and resutts
Part 2;
Basic method for the determination of
repeatability and reproducibility of a standard
measurement method

EX8ctitude (justesse et fid$líté) des résuftats et méthodes de mesure -


Partie 2: Méthode dlfJ base pour la dérermination de la répétabi/ité et de
reproductíbiJité d'une méthod8 de mesure normalisée
[<Ji

MATERIAL SÓLO PARA FINES DE


INVESTIGACiÓN PROTEGIDO POR U\
LEY FEDERAL DE DERECHOS DE AUTOR

* CIl:~_MI PROHi8;DA SU REPRQDUCC¡Óf~ )

Reference number
lS0 5'725-2:1994IE)

Copyright bY.~le lHTERHATIOHAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


'. fue Jan 16 16:56:1Q 1996
. . 4651903 0594510 105 . .

ISO 5725-2:1994(E)

Contents
Page

Scope ..... ,............ " ..... ,....... ,...... ,........... ,,, .. ,..,.... ,............ ,.............. ,

2 Normative references ................ . .. ......... ,. .......... " ..... ,........... 2

J Definitíons ............................................ 2

" Estimates of the parameters in the basic model ...... " ...... " ..... 2

5 Requirements for a precision experiment ................................ 2

5.1 Layout of me experiment ..................................... ' ...... ' .......... 2

5.2 Recruítment of 1he 'aboratarias ...................... " .................... 3

5.3 Preparation of the materíals

6 Personnel ínvQlved in a precision experiment ........ ,..... 5

6.1 Panel ................ ,.............. ,..... ,...... ,.... " ...................... ,...... ,.... , 5

6.2 Statistical functions 5

6.3 Executive functions 5

6.4 Supervlsors ............ ' ........... ,...... ,............................................ 5

6.5 Operators 6

7 Statistical analysis of a precísion experiment 6

7.1 Preliminary consideraticns 6

7.2 Tabulation of the results and notation used 7

7.3 Scrutlny of results for conststenc\, and outfíers ..................... 9

7.4 Calculatfon of the general mean and variances ................... 13

1.5 Establishing a functlonal relationship between precision va!ues and

the mean level m ,............................................................. ". 14

7.6 Statistical analysis as a step-by-step procedur8 ................... 16

7.1 The report to, and the decisions te be taken by, the panel 20

8 Statistica! tables ..... ', ..... " ............... ,........................................ 21

() ISO 1994

Al! nghts reserved. Unless otherwise- specified. no par! 01 thls pubhcatlOn rnlly be reproduced

O(uti{¡zed 10 any fO(nl Q( b', any means, eleCtIOI1IC o' mechanu:;al. Includll19 phOtOCOpyll1g "na

microfilm, without permlss,on ,n wntlng from the pubhsher.

Internatlonal Organl,atlon lor Standardlzatton

Case Postale 56 • CH·1211 Genéve 2Q " Sw,tzefland

Pnnted ,n SWltzerfand

ji

Cop~right h~ the INTERNATIOHAL ORGANIZArION FOR STANDARDIZATION (ISO)


Tue Jan 1& 1& :5&: 10 1996
.. ~851903 059~511 O~1 ..

e ISO ISO 5725-2: 19941 E)

Annexes

A Symbols and abbrevlations used in ISO 6725 ............. 25

B Examples of the 51atís1ical analysis of prec1síon experiments Z7

B.l Example 1. Deterrnination of tr.e suifur content of coal (Se\leral

Jevels with no misslng or outlying data) ...... " ..................... 27

S.2 Example 2: Softeníng point of pitch (Severallevels with missing

data) ..................................................................................... 32

B.l Example 3: Thermometric titration of creosote oíl (Severallevels

with outlying data).................................... 36

e Bibliography 42

íii

Copyright by the INTERNATIOHAL ORGANIZArlON FOR STAHDARDIZATIOH (ISO)


fue Jan 16 16:56:1G 1996
. . 4851903 D59~512 ra8 . .

ISO 5725-2:1994[EJ tO ISO

-
,

Foreword
ISO Uhe Internatíonal Organization for Standardizationt is a worldwide
federation of national standards bodies (ISO member bodiesL The wor!<
of preparíng Intemational Standards is normally carríed out through ISO
technícal committees. Each member body interested in a subject for
which a technical committee has been establíshed has the nght te be
represented on that committee. International o rganizatlons, governmental
and non-governmental. ín liaison wíth ISO, also take part in the work. ISO
collaborates c!osely with me International Electrotechnical Commíssíon
(lEC) on all matters of electrotechnícal standardízatíon.
Draft International Stanpards a<lopted by the technical committees are
crrculated to the member bodies for votíng. Publícation as sn International
Standard requires approval by a1 leaSl 75 % of the member bodies casting
a vote.

Interna1ional Standard ISO 5725--2 was prepared by Technical Committee


ISOITC 69, Applications of sraristice/ methods, Subcommittee 6,se
Measuremenr methods and results.

ISO 5725 consists of the following parts, under the general title Accuracy
(trueness and precísion) of meBsurement methods and resu{ts:

- Pan 1: General principies and definitions

- Pan 2.' Basje method for the determínatíon of rep¡;¡atabilíty Bnd re­
producibility of a st~ndard measurement method

- Pan 3: Intermediate measures of the precision of él standard


measurement method

- Part 4. 8asü:; methods for the determinatíon of rhe truenE/SS of a


standard measurement method

Pan 5: Alternative methods for the determination of the precision


of a standard measurement method

- Pan 6: Use in practica of accuracv va/ues

Parts 1 to 6 of ISO 5725 together cancel and replace ISO 5725: 1986,
which has been extended to covar trueness (in addítíon to pmcision) and
intermedia,€! precisíon condítions (in addítion to repeatabilíty and repro­
ducibility condítions).
Annex A forms an íntegra I part of this part of ISO 5725. Annexes B and
C are tor information only.

IV

Copyright by the lliTERIiATIOIiAL ORGAIiIZATIOIi FOR STAIiDARD[ZATlOIi (ISO)


Tue Jan 16 16:56:10 1996
. . 4851903 0594513 914 . .

o ISO ISO 5725~2:'994{EJ

Introduction
0.1 ISO 5725 uses two terms u trueness" and "precision" to describe
N
the accuracy of a measurement method. "Trueness refers ta the close­
ness of agreement between the arithmetic mean of a large number Qf test
results 8nd tlle trua or accepted reference value. "Precision" refers to the
c10seness of agreement between test results.

0.2 General consideration of these quant¡ües le given in ISO 5725-1 and


so is not repeatad in this part of ISO 5725. ISO 5725-1 shoufd be rsed in
conjunctíon with all other parts of ISO 5725. íncluding this parto b~cause
it gives the undedying defi()jtIQns and general principies.

0.3 This part of ISO 5725 i5 concerned solely with estimating by mean::;
of the repeatabílity standard deviation and reproducibility standard devi­
atíon. Although other types 01 experiment (such as the split-1evel exper­
ímenO are usad in certain clrcumstances for the estimatíon of precision.
they are not dealt with ín this part of ISO 5725 but rather are the subject
01 ISO 5725-5 Nor does thís part 01 ISO 5725 considef any other meas­
ures of precisíon intermedia te between the two principal measures; those
are 1he subject of ISO 5725-3.

0.4 In certain circumstances, the data obtained from sn experiment


carried out to estima te precisíon are usad also to estimate trueness. The
estimation of trueness is not considered in thls oart of ISO 5725; aU as­
pects of the estimarían of trueneS$ ale ¡he subJect of ISO 5725-4.

CupYl'lght hy the INTERNATIOIiAL ORGANIZATION FOR STANDARDIZATION (ISO)

...., .
Tue Jan 16 16:58:57 1996

11 4SS1903 0594514 850 . .

INTERNATIONAL STANDARD e ISO ISO 5725-2;1994(E)

Accuracy (trueness and precision) of measurement


methods and results -

Part 2:
Basic method for the determination of repeatability and
reproducibility of a standard measurement method

1 Scope 1.3 It assumes that In tfie deslgn and performance


of the precísion experiment all the principies as !(lid
down in ISO 5725-1 have beeo observed. The basic
method uses the same number of test results in each
1.1 This part 01 ISO 5725 laboratory, with each laboratory analysiog the sama
levels of test sample: Le. a balanced urüform-Ievel
amplífíes the general principies ro be observad in experiment The basíc method applíes to procedures
designlog experiments tor the numerical eSh­
that have been standardized and are in regular use in
mation of the precisíon of measuremant methods a number of laboratoríes.
by means of a collaborative ínterlaboratory exper­
¡ment; NOTE 2 Worked examples are gíven lo demonstlate bal­
anead uniform sets of test results, although in on6 example
- provides a detailed. practical descriptlon of Ihe a variable numbar of replícates per cell were reported (un·
basic method tor routíne use in estímatíng the balanced desígn) and in ar,other sorne data were rnissing,
preclsion of measurement methods; This is because 6n experiment deslgned 10 be baianced can
turn out lo b€ unbalaflced. Stragglers and outilers are also
- provldes guídance to all personnel concerned wíth considered,
designíng, performing or analysing the results of
the tests lar estímating preclsioo, 1.4 The statlstical model of clause 5 of
ISO 5725-1: 1994 ís accepted as a suitable basís for
NOTE 1 Modlfícations to thls basic method for particular the interpretation and analysls of the test results. the
purposes are glVCIi in other parts of ISO 5725,
distributíon of which is approximately normal.

Annex B provides practical examples of estímating


the precision of measurement methods by exper­ 1.5 The baslc method, as described in this par! or
iment. ISO 6725, will (usually) estlmate 1he precision 01 a
measurernent methorl:

al wnen It IS requlrea to det<:;!lll1ine lile fepeélldbi¡¡ly


1.2 Thís part of ISO 5725 is concomed exclusively and reproducibility standard deviations as defined
with measurernent methods which yield measure­ in ISO 5725-1 :
ments on a continuous scale and give a single value
as the test resl.llt, although thís single valua may be b) when the materí<:lls to be usad are homogeneous,
the outcome of a calculation from a sel of observa­ or when the effects of heterogenslt'y' can be in­
tioos. cluded In the predsion values; and

COP'Iright b'l tite IHTEllNATIOI'IAL ORGAHIZATIOH rOll STAHDAllDIZATIOH <ISO)


fue Jan 16 16:5B:57 1996
. . 4851903 0594515 797 . .

ISO 5725-2:1994CE} Q ISO

el when the use of a balanced uníform-Ievel layout B is the laboratory component of bias under re­
is acceptable. peatability conditions:

e i5 the random error occurring in every


1.6 The sarne approach can be usad to make a measurement under repeatability condltlons.
preliminary esttmate of precision for measuremem
methods which hava not reached standardization or
are not in routine use. 4.2 Equatíons (2) to (6) of ISO 5725-1: 1994,
clause 5 are expressed in terms of the true standard
deviations of the populations considered. In practica.
2 Normative references the exact valuas of these standard deviations are not
k:nowl1¡ and estimates of precision valuas must be
The following standards cantain provlslons which, made from a relatively smaH sample of all the possible
through reference in this text, constitute provisions laboratories, and within those labarataríes from a
of thís part of ISO 5725. At the time of publicatian. the small sample of all the possíble test results.
edítions índicated were valid. Al! standards are subject
lO ravisian, and partias to agreements based on this
part of íSO 5725 are encouraged to investigate the 4.3 In statistical practice. where the trua value 01 a
possibilíty of applyíng the most recent edítions of the standard deviation, tl. is not known and is replaced by
standards índicated below. Members of lEC and ISO an estímate basad upon a sample, thél1 the symbol a
maintain registers of currently valid International is replaced by S to denote that it ís an estímate. Thls
Standards. has to be done in each of the equatíons (2) to {6l of
ISO 5725-1 :1994, giving:
ISO 3534-1: 1993, Statistics - Vocabular¡t and sym­
bols - Pan 1.' Probability ano general statísticai .r~ is the estímate 01 the between-Iaboratory
terms. varianee:

ISO 5725-1 :1994, Accuracv (trueness and precision) s~¡ is the estímate of the within-Iaboratory vari­
of measurement methods and resufts - Pan 1: anca;
'-- General principies and defrnitions.
8,2 is the arithmetic mean of s~ and is the estí­
mate of the repeatability variance; this arith­
metic mean is taken over all those
3 Dufinitions rabaratoríes taking part in the aCCUfacy ex­
periment which remain after outliers haya
For the purposes of this part of ISO 5725, the defi­ been excluded:
nitions given in ISO 3534-1 and in ISO 5725-1 apply.

The symbols usec in ISO 5725 are gíven in ann€x A.


si ís the estímate of the reproducibility vari­
ance:

.. (1)
4 Estimates of the parameters in the
basic model
5 Requirements for a precision
4.1 The procedures gilien In this pal1 of ¡SO 5725 experiment
are basad on the statistical modal given in dause 5
of ISO 5725-1 :1994 and elaborated upon in subclausa
1.2 of ISO 5725-1:1994. In panicular, these pro­ 5.1 Layout of the experiment
cedures are based on equations (2) to (S} of clause 5
of ISO 5725-' :1994.
5,1.1 In tha layout usad in ¡he basic method, sam­
The model is pies 1rom q batch es of materials, representing q díf­
y=m+B+e ferent levels of the test. are sent to p laboratories
which each obtain exactly n replícate test resuits un­
where, for the particular materíal tested. dar repeatability conditions at each of the q levels.
Thls type of experiment ís callad a balanced IJniform­
In is the general mean (expectatíonl; level 8xperiment.

Copyri~ht by the IMTERMATIOMAL ORGANIZATIOM FOR STANDARDIZATIOM (ISO)


fue Jan 16 16:58:57 1996
. . 4851903 D59451b 623 . .
e ISO ISO 5725-2:1994{El

5.1.2 The performance of these measuremants shall tween the day the samples are receíved and the
be organizad and instructíons issued as follaws. day the measurements are performed.

al Any preliminary checkin9 of equipment shall be hl AII samples shall be clearly labelled with the name
as speclfied in the standard method, of the experiment ami a sample identification.

b) Each group of It measuremenls belonging to one 5.1.3 In 5. í.2 and elsewhere in this part of
level shall be carriad out under repeatability con­ ISO 5725, reference is meda to the operator. For
ditions, ¡,e. wíthin a short interval of time and by some measuremants, there may in fact be a team of
the sama operator, and without any íntermediate operatars, each of whom performs so me specific part
recalibration 01 the apparatus unless this is en in­ of the procedure, In such a case, the team shall be
tegral part oi performíng a measurement. regarded as "the operato(" and any crlange In the
team shall be regarded as providing a differen1 .. op­
el It is essential that a group of n tests under re­ aratar" .
peatability conditions be performed independently
as it they were n tests on dífierent materials. As 5.1.4 In commercial practica, the test results may
a rule, how8'1er, the operator will know that be rcunded rather crudely, but in a precision exper·
he/she is testing identical matenal, but the point ¡ment test results shall be reported to at least one
should be stressed in the instructions that the more digit than specified in the standard method. If
whole purpose 01 the experiment is to determine the method doas not specify the number 01 digits, the
what differences in rasults can occur in actual rounding shall not be coarser than half the repeatabíl­
testing. If lt 16 feared that. despite this warning, ity standard deviatlon estímate. When precision may
previous results ma,! lnfluence subsequent test depend on the level m, di-lferent degrees of roul'lding
results and 1hus the repeatability varianee, ít may be needed for dlfferent levels.
should be considered whether to use n separata
samples at each 01 the q levels, coded in such a 5.2 Recruitment of the laboratories
way that the oparator will not know which are the
replícates tor a given level. However, such a pro­ 5.2.1 The general principies regarding recruítment
cedure could cause problems in ensuring that re­ of the laboratoríes to participate in an interlaborator'l
peatability conditions will apply between experiment are given in 6.3 of ISO 5725-1: 1994, In
raplleates, This would on(y be possible if the enlistlng the cooperation of the requisite number oi
measuremems were of sur,;h a natUfe tnat all the laboratories, their responsibllitles shall be clearly
qn measurements could be performed wíthin a
stated. An example 01 a suitable enlistment question­
short intsNal of time, naire is given in figure 1 .
d} It 15 not essential ,that all the q groups 01 11
5.2.2 Far ths purposes of this part Qf ISO 5725, a
measurements each be pe(formed strictly within
"Iaboratory" is considered to be a combin.3tion of the
a short interval; different groups of measurernents
operator, the equipment and the test slte, One test
may be carríed out on difierent days.
sita {al' laboratory in tha conventional sense) may thus
produce several "¡aboratorles" if it can provide several
e) Measurements of all q levels shall be peñormed
operators each with independent sets 01 equipment
. by one 8nd the sama operator and, in addition, the
and situations in which to perfoml the work.
n measurements at a gíven level shall be per­
formed using the same equipment throughout.
5.3 Preparation of the materials
fl If in the courss of the measurements sn oparatar
should becoma una'lailable, aoother operatar may 5.3.1 A dlscussion of the poínts that need to be
the mea sureme nts, províded that the considered when selectíng materíals ter use in a pre­
changa does not oecur within a group of n cision experiment 1s given in 6.4 of ISO 5725-'1 :1994,
measurements at one level but only occurs be­
tween two of the q groups_ Any such change shall 5.3.2 When deciding on the quanbties of material to
be reported with 1he resulta. be provided, allowance shatl be maae tor accidental
spillage or el-rors in obtaining some test results vvhích
g) A time limit shall be gíven within which all may necessitate using extra material. The amount 01
measurements shall be completed. This may be material prepared shall be sufficíent to cover the ex­
necessary to limit the time allowed to elapse be­ periment and allow an adequate stock in reserve.

Copyright INfERNATIONAL ORGANIZArlON FOR STANDARDIZAflON (ISO)


fue Jan 1G 199G
. . 46519D3 0594517 5bT . .
JSél 5725-2:1994(EI o ISO

Questionnaire for interlaboratory study


Title of measurement method (copy sttachedJ ........................................................ .

1. OUf laboratory is wílling to partlcipate in rhe precision experiment for this stan­
dard measurement method.

YES O NO O (tick appropriate box)

2. As a participant, we understand that:

a) all essentlal aoparatus. chemlcals and other requirements specified in the


method must be available in our laboratory when the programme begins;
U
b) specified "tímíng requírements such as starting date, arder 01 testing
specimens and finíshíng date of the programme must be rigidly met

el the method must be strictly adhered to;

d) samples must be handled in accordance with instructfons;

el a qualified aparatar must perform the measurements.

Having studied the method and having mede a fair appraisal of our capabilities and
facilities, we feel that we will be adequately preparad tor cooperative testing of this
method.

3. Comments
(Signed) ...................................................... .

(Companv or

laboratory} ................................................. .

Figure 1 - Enlístment questionnaire for Interlabofatory study

5.3.3 It should be considered whether it is desirable 5.3.5 For the samples a1 each leveL n separate con­
fo( so me laboratoríes to obtsin same prelíminary test tainers shall be usad tor each laboratory if there is any
results tor familiarizatíon with the measurement danger of the materíals deteriorating once the con­
method befare obtaining the officlal test result and, if tainer has besn opened (e.9. by oxidation. by losing
so. whether addítional material (not precisíon exper­ vOlatile components, or with hygroscoplc material). In
íment samples) should be provided ior thlS purpose. the case oí unstable materiais, speciol instructíons on
storage and treatment shall be spec1fied. Precautions
may be needed to ensure that samples remaín iden­
tical up to the time the measurements are mada. If
5.3.4 When a matenal has to be homogenized, this
the material to be measured consísts of a mixture ot
shall be done in the manner most appropriate for that
powders oí different relativa density or of different
material. When the material to be tested is not
grain sjze. sorne cara is needed because segregatian
homogeneous, it ¡s important to prepare the samples
may result from shakíng, tor examp!e durillg transporto
in tlle manner specifíed in the method, preferably
When reaction with the atmosphere may be ex­
startíng with cne batch 01 commercial material for
pacted, the specimens may be sealed into ampoules.
each level. In the case 01 unstable material s, specíal
either evacuated or filiad with an inert gas. For pero
ínstructions on storaga and treatment shall be speci­
ishable materíals such as food or blood samples, ii
fiad.

Copyri~ht hy the IMTERMATIOtlAL ORGAMIZATIOtI FOR STAMDARDIZATIOM (lsm


!ue Jan 1& 1&:58:57 199&
. . 4851903 0594518 4Tb . .
o ISO ISO 5725-2:1994IE)

may be necessary to sand them in a deep-frozen state al to contributa hísfher spedalized knowledge in de­
to the participating labarataríes with detaíled in­ signing the experiment;
structions for ihe procedure for thawing.
b) to analyse the data;
el to write a repDrt tor submission to the panel fol­
6 Personnel ¡nvolved in a precision lowinQ the instructions comained in 7 I
experiment
NOTE 3 The methods of operation withín differenl lab­ 6.3 Executive functions
orataries are not expected to be identlcal. Therefore the
contents oí thís clause are anly ¡ntended as a gUlde to be
modified as appropriate to cater for a particular situatioo. 6.3.1 The actual organization of the experiment
should be entrusted to a single laboratory. A member
of the 5t8ft of that laboratory should take full raspen­
6.1 Panel sibilí1y; he/she is called the executíve officer and is
appointed by the paneL
6.1.1 The parle: snould consist of experts familiar
wíth the measurement method and its applicatíon.
6.3.2 The tasks of the executive officer are:

6,1.2 The tasks of the panel ara: al to enlist the cooperation of the requisite number
of labaratorías and to ensure that supervisors are
al to plan anó coordinate the experiment; appointed;
b) to decide on the number of laboratoríes, lellels b) to organiza and supslVisa the preparation of the
and measurements to be made. and the number materials and samples and the dispatch of the
of signíficant figures to be required; samples; lor each le1l81, en adequate quantity ot
e) to appoint someone for the statístical funclions material should be set aSIda as a reserve sto.;;k;
(see 6.2);
e) te draft instructions covering aíl the poínts in 5.1.2
d) to appoint someone tar the executlve functions a) to h). and circulate them to the supervlsors
(see 6.3': early enough in advenca for them to raise any
comrnents or querias and to ensure that operators'
e) to conslder the instructions to be issued to the
selected are those who would normal1y carry out
laboratory supervisors in addition 10 the standard
such measurements in routine operat¡ons:
measurement method;
dl to design suítable forms for the operator to use
fl to decide whether some operators may be al­
as a working record and for the supervisor to re­
!owed to carry out ti few urlofficial rneasurements
port the test results to the requisite number of
in arder to regain experíence 01 the method after
significant figures (such forms may include the
a long imerval (such measurements shall never
name of the oparatar, the dates on wh¡ch sam­
be carried out on the affieíal collaboratíve sam­
pies wara received and measured, the equipment
pies);
usad and any other relevant ínformatíon);
gl to discuss the raport of the statistical analysis on
e} to deal with aoy queries from laboratones regard­
completion of the analysís of the test results;
ing the performance of the measurements:
h) to establísh final valuas for the repeatabilíty stan­
dard deviation and the reproducibility standard f) to see that an overall time schedule is maintained:
daviation; g) to callect the data forms and present them to the
i) lo dec,de if flJr1her actions are requlred to improve statistical experto
the standard for the measurement method or with
regard io laboratories whose test results nave
been rejected as outliers. 6.4 Supervisors

6.2 Statistical funetions 6.4.1 A staff member in each of the partícípating


laboratoríes should be made responsible for organiz­
At least one member of the panel should have 6)(­ ing the actual performance oi the measurements. in
perience in statistic81 design and analysis of e)(per­ keeping wíth instructions received from the executive
iments. His/her tasks are: officar. and for reporting the test resulls.

Cop~right b~ the INTERNATIONAL ORGANIZATION FOR STANDARDIZATION (ISO)


Tue Jan 1& 1&:58:57 199&
. . 4851903 05945~9 332 . .
ISO 5125-2:1994(E) o ISO

6.4.2 The tasks of the supervisor are 6.5.Z Because the object of the experiment is to
determine the precision obtainable by the general
al ensure tha1 tha aparatara setected are those who popula11on of operators workíng from the standard
would normally carry out such measurements in measurement method, in general the operators
routine operatíons; should not be given amplificatíons to the standard for
b) to hand out the samples to the operatorts) in the measuremen1 method. However, ít should be
keeping with the instructions of the executív€ of­ pOlnted out to the operators tnat the purpose of the
fieer (and to provida material tor famíliarization exercise is to discover the extent to which results can
experiments, if necessary); vary in practica, so that there will be less temptation
e) to supervise the execution of the measurements far them to disca rd or rework results that they feal are
(the supervisor shall not take par! in performing inconsistent.
the measurements);
dí 10 ensure that lhe operators carry out the requirad
6.5.3 Although normally the operators should re­
ceive no supplementary amplifícations to the standard
number af measurements;
measurement method. they should bé encouraged to
el lo ensure adherence to the sel timetable for per­ commant on the standard and, in particular, to stete
forming the measurements; whether the instructions contaíned in ¡t are sufficiently
f) to collec1 the test results recorded to the agreed unambiguous and elear.
number of decimal places. including any anom·
alies and dittículties eX.O€Iríenced. and comments 6.5.4 The tasks of the operators are:
made by the operators.
al to perform 1he measurements accordlng to the
6.4.3 The supervisor of each laboratory should write standard measurement method;
a full report which should contairr the following in1or­ b) to report any anomalies or difficulties experí­
mation: anced; it ¡s better to report a mistake than to ad­
just the test resu/ts because one or two missir¡¡:;
al the test results. enterad legibly by their origínator test results will not spoil 1he experiment end
on the forms províded, not transcribed or typed n1any índicate a deficiency in the standard;
(computer or testing machine printout may ba ac­
ceptable as an alternativa); e) te comment 01'1 the adequacy of the instructions
in the standard; operators should report any oc­
al the original observad values or readings (if any)
casions when they are unable to follow their in­
from which the test results were derivad, enterad
structions as this may also indicate a deficiency in
iegibiy by the operator on the forms provided, not
the standard.
transcriaed or typed;
el comments by the operators 01'1 the standard tor
the measurement method; 7 Statistical analysis of a precision
dI informatíon about irregulanties or disturbancss experiment
that may hava occurred during rhe measure­
ments, íncludíng any change of opera1or that may 7.1 Preliminary i:onsiderations
have occuned( together with a statement as to
which measurements were performed by which
operator, end the reasans tor any missing results; 7.1.1 The analysis 01 the data, whlch should be
considered as a statistica! problem to be salvad by a
el the dete(s) on which the samples were received; statistical experto ínvol1l8s three successive stages:
ti the date(s) on whieh each sample was measured;
ai critical examination of the data in older lo ¡dentiíy
g) information about the equipment used. if relevant; and treat outlíers or othef írregularities and to test
the sUltabilíty of the model;
h) sny othar relevant information.
b} computation of preliminary I/alues of precision and
6.5 Operators means tor each leve I separately;
e) establishment of final values of precísion and
6.5.1 In each laboratory the measuremenls sha/l be means, íncluding the establishment af a relation­
carried out by one oparatar selec1ed as being repre· ship between precision and the leve I m when the
sentative of those likely to perform the measure· analysis indícates that 5IJch a relatíonsh¡p may
mants in normal operations. existo

Cop~ri!Jht h~ the INTERNAT IOHAL ORGAH IZAr ION FOR STANDARD ¡ZAT IOH (ISO)
fue Jan 16 16:58:57 1996
• 48.51903 0594.520 054 •
QISO ISO 5125-2:1994(E)

7.1.2 The analysis first compules. for each level from the comparable entríes in the same tabla that
separately. estimates of they are considered irreconcilable wilh the other dala.
Experience has taught that outliers cannot always be
- the repeatabilíty variance s} avoíded and they have to be taken into consíderation
- the between-Iaboratory varianee .se in a similar way to the treatment of missing data.

the reproducibility variance s~ s; + sE 7.2.5 Outlying laboratories


- the mean m.
When severa I unexplained abnormal test results occur
7.1.3 lhe analysls incfudes a systematic applicatlon at different !evels within the same laboratory, then
of statistical tests for outllers. a great variety of which that laboratory may be considered to be an outlier,
are available irom the hterature and which could be having too high a within-Iaboratory variance and{or 100
usad for the purposes of thls part of ISO 5725. For larga a systematic error in the level of lts test results.
practical reasons. only a limited Ilumber of these It may then be reasonable to discard soma or 611 of
tests. as explained in 7.3, haya been incorporated. the data from such an outlying laboratory.

7.2 Tabulation 01 the results and notation lhis part of ISO 5725 does not provida a statistical
test by which suspected laboratories may be judged.
used The primary decísion should be the responsibi¡ity of
the stallstical expert, but all rejected !aboratories shall
7.2.1 Cells
be reported to the panel for further action.
Each combínalion of a laboratory and a level is called
a cel! of the precision experimem. In ths ideal case, 1.2.6 Erroneou-S data
the results of 6n experiment with p laboratorles and
q levels consist of a tabla wíth pq cells. each contaín­ Obviously erroneous data should be ínvestigated and
ing n reolícate test results that can all be usad for corrected or discarded.
computing the repeatability standard deviation and the
reproducibillty standard deviatlon. T;¡is ideal situatiofl 7.2.7 Balanced linifol'm·revel tast results
15 no1. however, always attained in practice. Depar­
tures occur owing to redundant data, missíng datá and The ideal case is p labaratories caHed
oUlliers. (í ~ 1. 2, ... p), each testlng q levels called j
(; = 1. 2. .... q) with n replicates at each level (6ach
7.22 Redundant data ij eombinationl. glving a total of pqn test results. Be­
cause of missing O.2.3} or deviating (7.2.4) test re­
Sornetirnes a laboratory may carr¡ out and rapart sufts, or outlying labaratories (7.2.5l or erroneous daTa
more than the 11 test results officially specified. In that 17.2.6), this ideal situation is no1 always attained. Un­
case. the supervisor sha" rsport why this was done der these conditions too notations given irl 7.2.8 to
and whích are the correet test results. If the answer 7.2.10 and the procedures of 7.4 aHow for dHfering
la that they are all equally valid. lhen a random se­ numbers of test results. Specimens oi recommended
lection should be made from those available test re· forms for the statístical analysis are gíven in figure 2.
sults to choose the planned number of test results for For convenience. they wlll be raferrad to slmply as
analysls. e
forms A. 8 and (of fígure2).

7.2.3 Míssing data


7.2.8 Original test results
In other cases, some 01 the test results may be
Ses fQrm A of figure 2, where
missing, for example because of loss of a sample or
6 mistake in performing the measurerl1ent. The
n(i is the number of test results in the cel! for
analysís recommended In 7.1 ia such that completely laboratory i at level j;
empty cells can simply be ignorad, while partly empty
ce lis can be taken into account by the standard com­ Y'}K is any one of these test results
putationaf procedurs. (k;: 1, 2, .... ll¡J);

7.2.4 Outliers Pi is the number of laboratones reporting at


least one test result tor level j (after elim·
These are entnes among the original test resules, or inaling any test results designated as
in the tables derivad from them. that deviate so muerl outliers or as erroneousl.

Copyright hy the INTERNATIONAL ORGANIZATION FOR STANDARDIZATIOH (ISO)


Tue Jan 16 16:58:57 1996
. . 4851903 OE~4521 190 . .
ISO 5725-2:1994(E) () ISO

Form A - Racommended form for the eollatlon of the orIginal da1a


Lavel lO07'
laboratory rO'!- ,
t
1 2 .. I .. j ., .­ i q­ 1 q
r ¡o¿ (1¡1 ¡H/U
ei2 f l
1 ,<
,
I
1f'·'Lcj.
·.
·,
/1:/I ~~-:- ,

,
l'
. f:/
./ /
-
·.
p ) I

Form B - Reeommended form for tila colla1ion of the mesns .J


Leve!
Laooratofy
1 2 .. .. j , , , ,
1
<7 1 I q
1
I
2
,
, i
I
í
Y"
I
p

Form e­ Recammend8d form far 1he eallatian of 1he measures of spread withlo eells
I.anl
LabOfilt1ory I I
1 2 .. .. j .. i , , q 1 q
1
2
.,

¡ $i)

, ,

Figure 2 - Recommended torms for the collation of results for analysis

7.2.9 Cell means (fomo B of figure 2) The cel! means should be recorded 10 one more sig­
nificant figure than the test result in form A.

These are derived from form A as follows:


1.2.10 Measures of cell spread {fofln e 01 figure 21
... (2) These are derived rrom form A (see 7.2.8) and iorm
B (see 7.2.9J as follows.

:opyrigllt by the IHTERHATIOHAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


fue Jan 1& 1&:58:57 1996
. . 48519Q3 0594522 927 . .
ISO 5725-2:1994(E)

For the general case, use the intracell standard devi­ oratories or values that appear to be inconsistent with
atloo all other laboratoríes or values may cr.ange the esti­
mates, 8nd decisions haya to be made with respect
'l. to these values. Two approaches are íntrcduc~d:
s·:
'/
¿(y,¡.t-Yi~ . " (3) fl, "r:" ,:. ("

/1:=1 a) graphical consistency technique: f,\\l\'\;}).}.

r
b) numerical outlier tests. _\\I\(,I¡..'. le G
oro equ!valently

7.3.1 Graphical consistency technique

Two measures called Mandel's h and k statístics are


usad. It may be notad that, as well as describing the
variability ef the measurement method. these help in
(4) laboratory el/sluation.
In uSlng these equatioflS, care shall be taken te retaln
a suftícient numbar of digits in the calculations; Le. 7.3.1.1 Calculate the between-Iaboratory conslst­
every intermediate value shall be calculated to at least ency statistic. h. for ea eh laboratory by dividing the
twice as many digits as in the original data. cell deviation (cell mean minu$ the grand mean for
that level) by the standard deviatíon among ce" means
NOTE 4 If a ce!! ij contains tINO test results. lhe IntracBl1 (fer that levell:
slandard del/latíon IS
... (5)
Íly = -r=============- . (6)
p/
There/ore, tor Slmplicíty. absoluta di/farancas can be usad
ínslead ot standard deviatíoflS íf all ce lis contain two test -:--'--::-:-¿ (y,j 7.Y;)2
, 1
results,

The standard deviatlon should be expressed to one in whieh. tor y,¡ sea 7.2.9. and for ~ see 7A.4.
more significant figure than the results in form A.
Plot tha lit! valuas for each cell in arder of laboratory,
For values of n,} less than 2, a dash should be inserted in groups for each le'lel land separately grouped for
in form C. the several levels examinad by each laboratory) (se9
figure B.7).

7.2.11 Correc:ted or rejected data


7.3.1.2 Calculate the within-Iaboratory consistency
statistic, k, by tirst calculating the poolad wlthin-cell
As sorne of the data may be corrected or rejected on
standard deviation
the basis of the tests mentioned in 7.1.3, 7.3.3 and
7.3.4, che values of Yi¡k' I1'J and PJ used for the final
determinations of precision and mean may be difter­
em from the values refernng to tne original test re·
sults as recorded in forms A. B and e
of figure 2.
Hence in reportíng the final values Tor precision and
trueness. i1 shall always be stated what data, if any, for aBeh leve!. and thsn calculate
have been corrected or discarded.
k¡j= . .. (7)

7.3 Serutiny of results for consistency and


outliers
for each laboratory within each level.
Ses refersnce [3].
Plat the k,¡ values for aaen cel! in arder of laboratop{.
From data collectcd on a number of levels, in grolJps for each level (and sepacately grouped lar
repeatabiiity ana reproducibiliW standard deviatlons the several levels examíned by eacn laboratory) (se e
are to be estimated. The presence 01 individual lab­ figure B.8).

Copyright by the INTERNATIONAL ORGANIZATION FOR 3TANDARDIZATION (130)


Tue Jan 16 16:58:57 1996
. . ~8S1903 059~523 863 . .
ISO 5725-2:1994(EI ISO

~ 7.3.1.3 Examination of the h and k plots may indicate 7.3.1.7 In additíon to these h and k graphs.
1hat specific Jabaratoríes exhíbít patterns of results histograms of cel! means and cel! ranges can reveal
that are markedly different trom the others in the the presence of. for example. two distinct popu­
study, This ís índicated by consístently high or low lations. Such a case would require sp9cial treatment
within-cell variatíon and/or extreme cell means across as the general underlyíng principie behind the meth·
many levels. If this occurs. the specific labaratory ods described here assumes a single unimodal popu­
should be contacted to try to ascertain the cause of lation.
the discrepant behaviour On the basis of the findings.
the statístícal expert could:
7.3.2 Numerical outller tachnique
al retaín the laboratory's data for the moment;
7.3.2.1 The following practice ís recomrnended for
b) ask the laboratory to rado the measurement (if
dealing with outliers.
feasiblel;
al The tests recommended in 7.3.3 and 7.3.4 are
e) remove the laboratory's data from the study. applied to iden¡ify stragglers or outliers:

7.3.1.4 Various patterns can appear in the h plots. - if the test statístic is less than or aquel to its
Al! laboratoríes can have both positive and negative It 5 % critical vslue, the item tested is accepted
va'ues at dífferent levels of the experimento Individual as correct;
laboratoríes may tend to give either al! positiva or alJ
negative /¡ valuas, and the number of laborataríes - if the test statistic is greater than íts 5 % crit,
giving negativa values is approximately equa! ro those ical value and less than or equal 10 ¡ts 1 %
glVlng posltive valuas. Neither of these patterns IS critlcal value, the ítem testad is callad a
unusual or requires investigation. although tila second straggler and is índicated by a single asterisk;
of these patterns may suggest that a common source
of laboratory bias exists. On the other hand, if all 1he - if the test statistic is greater than its 1 % crit~
h valuas for one laboratory are af one sign and the h ical value, the ítem is called a statistical outlier
valuas for the other laboratoríes are all of the other and ls indicated by a double asterisk.
sigo, then the reason should be 50ught. Ukewise. if
the h valuas·for a laboratory are extreme and appear b) It is next investigated whether the stragglers
to depend on the experimental level in sorne sys­ and/or statistícal outlíers can be explained by
tematic way. then the rsason should be sought. Unes soma technícal error. for example
are drawn on the h plots corresponding to the indica­
tors given in 8.3 (tables 6 and 7), These indicator lines - a slip in performing the measurement.
serve as guides when examíni'.1g patterns in the data.
- an error in computation,
7.3.1.5 If one laboratory stands out on the k plot as
- a simple clerical error in transcribing a test re­
having many large values, then the reason should be
sult, or
sought: this indlcates that it has a poorer repeatability
than !he other laborataríes. A laboratory could give - analysis of the wrong sample.
rise lo consistently small k values because of such
factors as excessive rounding of its data or an insen­ Where the error was one of the computation or
sitíve measurement scale. Lines are drawn on the k transcríption type, the suspeet result should be
plots correspondíng to the indícators given in 8.3 (ta­ raplaced by the correet value: where the error
bies 6 and 7}. These indicator Hnes serve as guides was fmm analysíng a wrong sample. the result
when examining patterns in the data. should be placad in íts correct cell. After sLlch
correctíon has been made. the examínatlon for
7.3.1.6 When I€ n h or k plot grouped by laboratory slragglers or outliers should be repeated. If the
sllggests that one laboralory has several h or k values explanation of the technical error is sueh that it
near the critica I value line, the correspondíng plot proves impossible to replace the suspeet test re­
grouped by level should be stlldíed, Often a valua that sult, then it should be discarded as a "genuine"
appears larga in a plot grouped by laboratory will turn outlier 1hat does not be!ong to the experiment
out to be reasonably consis1ent with olnar labora· proper.
torles tor the same level. If ít ís revealed as strongly
dif1erent from values tor the other laboratories, then e) When any straggler::¡ and/or statistical outliers re­
the reason should be sought. main that hsve not been explained or reiected as

10

2"1'~,'i~hL b\l CJ." ltiTERHATlOtiAL ORGAtiIZATIOti FOR STAHDARDIZATlOH (ISO)


rue Jan 16 16:58:57 1996
al 4851903 05945~4 7TT . .
10 ISO ISO 5725·2:1994(EJ

belongíng to an outlying laboratory, the stragglers Cochran'::; test has to be applied to form C of figure 2
are retained as correet i1ems and the statistical at each level separately.
outliers are discarded unless the statistician for
good reason decides to rataín them, 7.3.3.3 Cochran's critaríon applies Slríctly only when
all the standard deviatlons are derived from the same
d) When the data for a cell have been rsiectad for number (n) of test results obtaíned under repeatability
fQrm 8 of figure 2 under 1he aboye procedure, condl1íons, In actual cases, this number may vary
then the corresponding data shall be rejected for owing to missing or discarded data, This part of
forme of figure 2, and vice versa. ISO 5725 8ssumes, however, that in a properly or­
ganizad 9xperiment su eh vanations in the number of
1.3.2,2 The tests given in 7.3,3 and 7,3.4 are of two test results par cell will be limitad and can be ignorad,
types. Cochran's test is a test of the within-Iaboratory and therefore Cachran's critanon ís applied using for
variabilities and should be applied first, then any n the number of test results occurring in the majority
neeessary action should be taken, with repoatad tes1s of cells.
il necessar'l. The other test (Grubbs') is primarily a
test of between-Iaboratory variebility, and can al50 be
used (if ti> 2) where Cochran's test has raised sus· \7.3,3.4 Cochran's criteríon tests only the híghest
píclons as to whether the high within-Iaboratol)' varí­ : value in a set 01 standard daviatíons and io.; therefore
ation ís attríbutable to only one of the test results in 1 a ane-sided outlier test. Varlance heterogeneity may
~'--also, of course, manifest itself in some of the stan­
the cell,
dard devia1ions being comparatively too low. How­
aver, sma!1 values of standard deviation may be very
7.3.3 Cochran's test strongly influeneed by tha degree of rounding Qf the
original data and are for that reason not very relíable.
7.3.3.1 Thls pan of ISO 5725 assumes that between In addition, it seems unreasonable to reject t"e data
laboratorjes only 5mall differences exis1 in the within­ from a laboratory beca use it has accomplished a
laboratory variances, Experience, however, shows higher precision in its test results than the other I¿¡b­
that thís is not always the case" so that a test has oratories. Hence Cochran's criterían is consídered ad­
been included here to test the validity of this as­ equete,
sumption. Several tests could be used for this pur­
pose, but Cochrao's test has been chosen.
7.3.3.5 A critical examination of fonn e of figure 2
may sometimes reveal that the standard deviations
7.3.3.2 Given a set 01 p standard devíations Si' all for a particular laboratory are at an or al mast levels
computed from the same number (ll) of replicate re­ lower than those for other laborataríes. This may in·
sults, Cochran's test s1atis1ic, e, ís dicate that the laboratory works with a lower repeat­
abílity standard deviation than the other labarataríes,
e ... (8) which ir, turn may be caused either by bertar tech­
nique ano equipment or by a modified or incorrect
application of the standard maasuremenl method. If
thís occurs it should be reported to the panel, which
where Smax is Ihe highest standard devíation in 1he set. should then decide whether the poínt is worthy of a
more detailed inves1igation, (An example of this is
al If the test s1atistic ís less than or equa I to its 5 % taboratory 2 in the experime n1 detailed In B, 1.)
critical valuB, the ítem tested is accepted as cor­
ree!.
7,3.3,6 If the highest s1andard deviat'ron ís classed
bl If ¡he tes1 statísllc is greater than its 5 % critica! as an autlier, then the value should be omitted and
value and (ess then or equel te its 1 % critical Cochran's test repeated on the remaining va!ues. This
value, the ítem tested is called a straggler and is process can be repestad but í1 may fead 10 excessiv!';l
indicated by .si single asterlsk, rejeetíons wheo, as is sometimes the case, the
underlyíng assumption of normality is not sufficíently
el If the test statistic ís greater than its 1 % critical welJ approximated lO. The repeated applicauon of
valua, the ítem is callad a statistical outlíer and is Cochran's test is here proposad only as a he(pful tooi
índicated by a double asterisk. in view of the lack of a statístical test designed for
testing severa! outliers together, Cochran's test is not
Critical values for Cochran's test are gíven in 8.1 designed for this purpose and great cautlOn shourd be
(table 4). exercised in drawíng conclusions. When two or three

11

Copyright by the IMTERMATlOMAL ORGAMIZATIOH FOR StAHDARDIZATIOH (ISO)


Tue Jan 16 16:58:57 1996
.. ~8S19D3 OS9~525 63b . .
ISO 5725-2:1994(E} o ISO

laboratarías give results havíng hígh standard deví­ where


atíons, particularly if this is withín only one of the lev­
els, conclusians from Cochran'$ t~st should be
... (13)
examinad carefully. On the other hand, if severa I
stragglers 8nd¡or s'(atistical outliers are found a! dif­
ferent levels within one laboratory, this may be a and
5trong indicat¡on that the laboratory's within­
p 2
laboratory variance ís exceptionally high, and the
whole of the data from that labaratary should be re­ S;_l,P = L)~¡- ~- 1,/ ... (14)
i:;; 1
jectad.
and
7.3.4 Grubbs' test
" . (15)
7.3.4.1 One ouUying observation
Alternatively, to test the two smallest observations,
Given a set of data Xi for i = 1. 2, .". p, arranged in
compute the Grubbs' test statistic G:
ascending ardar, than to determina whether the larg­
es{ observ8,ion is 8n outlier usíng Grubbs' test, com­ , " (16)
pute the Grubb's statistic. GI)"
where
G!, == (.x¡, - x)/s ... (9)
f.'
2
where S;,2 ¿(Xi ­ X1,2) ... (17)
1=3

... (10) and

and (18)

s= ... (11) Critical values for Grubbs' test are given in 8.2
(lable 5).

To test the significance of the smallest observation,


compute the test statistic
7.3.4.3 Application of Grubbs' test
G, = ~'( X1)/S
When snalysing a precísion experiment, Grubbs' test
al i7 the test statist¡c is ress than equal to :ts 5 %

can be app!led (O the following.
critical value, the ítem tested is accepted as CO(­
rect. a) The cell al/eragas (form 8 of figure2) for a given
lever i in which case
t) If the test statistlc is greater than its 5 % crítica!
value and l~$S than or equal to its 1 % critlcal x, == Yt¡
value, the ítem tested ís called a straggler and is
índ¡cated by a single asterisk. and

el If tha test statistic 1$ greater ihdn its 1 % critical


value, the ítem ís called a statistical outlier and ¡s'
indicated by a double asterisk. where j is fíxed.

Taking the data at one level, apply the Grubbs'


7,3.4.2 Two outlying observations test for one outlying observation to cell means as
descríbed in 7.3.4.1. If a cefl mean is shown to be'l
To test whather the two largest observations may be an outlier by this test exclude it, and repeat the 1:
outlíers, compute the Grl.lbbs' test statistic G: test at the other extreme cell mean if the~
highest ís an outlier then look at the lowest with
. , . (12) the highest excludedl, but do not app1v the

12

COP'Jright b'J the II'lTERNATIOHAL ORGANIZATIOH FOR STAHDARDIZATIOH (ISO)


Tue Jan 16 16:58:57 1996
.. 4~51903 0594526 572 . .
ISO 572S-2:1994(EI

Grubbs' test tor two outlying obseNótions de· 7.4.4 Calculation of the general mean m
scribed in 7.3.42. It the Grubbs' test doss not
show a cel! mean to be an outlier, then apply the For level j, the general mean is
double-Grubbs' test described in 7.3.4.2.

bl A single result within a cel!, where Cochran's tes1


has shown the cell standard deviation to be sus­ ... (19)
pect

7.4 Calcuiation oí tne general mean and


variances
7.4.5 Calculation 01 varianees

Three varíances are calculalsd for each level. They are


7.4.1 Method of analvsis the repeatabílity variance, the between-Iabor-aíory
variance and the reproducibility variance.
The mathod of analysis adopted 'In lhls' parl Qf
ISO 5725 involves carrying out the estimation of m
and the precisíon for e8ch level separately_ The ¡esults
of the computation are expressed in atable for each 7.4.5.' The repeatabilíty varíance is
value of I
p

¿)n" l)s~
i=' _ _ __ __ (20)
7.4.2 Basie data f'

¿(III} 1)
The basic data needed for the computations are pre­ i= 1
sented in the three tables given in figure 2:

- tabla A cont8lfling the onginal test results;


7.4.5.2 The between-laboratory variance is
- table 8 containing the cel! means;
2 2
- tabla e containing. the rneasures of within-cell ,fdj: Srj
.. (21)
spread. lli

where

7.4.3 Non-emptv <:8115

As a consequence of the rule stated in 7.3.2.1 d), the


number of non-empty cells to be used in the compu­
latían wlll. for a specific level, always be the sama in (22)
tables B and e, An exception might occur if, owing to
missing data, él cel! in table A contains only a single
test result. which will entail en empty cell in table e and
but not in tabla B. In that case it ís possible

al to discard the solitary test result, which will lead f'


to empty cells in both tables B and e, or ñ¡, - '­
p-1 ~'1;j
L.. (23)
1= i
b) if this is considerad en unde'sirable 1055 of infor­
maban, to íosert a dash in table C.

The number of non-ernpty cells may be different tor These celculatlons are ¡Uustrated in the examples in
different levels. hence the index j in Pi' 8.1 and 8.3 in annex B.

13

Copy~ight hy the INTERNATIOHAL ORGANIZATION FOR STANDARDIZATIQM (ISO)


Tue Jan 1& 1&:58:57 1996
. . 4&51903 0594527 409 . .

ISO 5725-2: 1994fE) o ISO

7.4.5.3 For 1he particular case where all ni} == 11 = 2, 111: Ig Sr ¡; + d Ig m lar s, = Cm d); d .¡;.1 (an expo­
the simpler formulae may be used, gíving nential relationship)

It 1$ to be sxpected that in lhe majority of cases 6t


least one 01 these formulae will give a satisfactory fit.
If not the statístical expert carrying out the analysís
should seek an alternativa salutlo!l. To avold con­
and
fusiono the constants a, b, e, C and d occurríng in
r 2
these equations may be dist1nguished by subscripts,
2 1 "'"' _ ~ 2 Sr¡
St"i =P_ 1 .L.. (YI! - y) - 2 aro b,. ... for repeatability and aH' bll , ••• when consid­
I 1 ering reproducíbility, but these have been omiUed ín
thís clause again to simplify the notahons Also S}' has
These are illustrated by the example given in B.2.
bE*3n abbrevíated simply to s 108110'.1'1 a suffix for the
level j.
7.4.5.4 Where, owing 10 random effects, a negative
value for s~ is obtained from these calculatlons, the
valva should be assumed to be zero. 7.5.3 In general d > O so that relatíonshíps I and 1II
will lead to s = O for m = 0, which may seem un~
7.4.5.5 The reproducibility variance is acceptable from an experimental point of view. How­
Z 2 2

ever, when reporting ths precisíon data, it should be


SRJ '" srJ -t SLj
... (24) made clear that they apply only within the levels cov­
erad by the interlaboratory precisiol! experimento
7.4.6 Dependence af the variances upon m
7.5.4 For a O 8nd d = 1. all three relationships are
Subsequently, ít should be ínvestigated whether the
identical, so when a líes near zero andlor d fíes near
precísiofi depends upon m and, if so, the functíonal
unity, 1wo or all three of these relationships will yield
relationship should be determinad.
prac11cally eqUlvalent fits, anCi In such a case relation~
ship I should be preferred because it permits the toJ·
1.5 Estabfishing a functional relationship lowing simple statement.
between precision values and the mean
level m "Two tes1 results are considerad as suspec1 when
they differ by more than (100 h) %."
7.5.1 Ir cannat always be taken for granted that
In stinistícal terrnlnology, lhls is a statsment that the
there exists a regular functional relationship between
coefficient of varia1ion (100 s/m) is a constant for aH
precision and m. rn particular, where material
levels.
heterogeneity forms an inseparable part of the vari~
abilíty of the \9St results, there will be a functíonal
relatíonshíp only it this heterogeneity is a regular 7.5.5 li in a plot of 5) agaínst mj'or a plot of Ig Sj
function of the leve! m. With solid materials of differ­ against 19 mi' the set of points are found to líe rea·
ent composition and comíng from different produc1ion sonably close to a straight líns, a line drawn by hand
processes, a regular functional relationship ís in no may provida a satisfactory solution; but if for some
way certain. This point shourd be dectded befare the raason a numerical method of fitting is preferred, the
following procedufe is applíed. A1ternatively, separate procedure 01 7.5.6 is recommended for relationships
values of precision would haya te be Slstablished tor I and 11, and that of 7.5.8 for relationship 111.
each material investlgated.

7.5.6 From a statistical viewpoint, the fitting oI. a


1.5.2 The reasoning and computatlon procedures
presented in 7.5.3 to 7.5.9 apply both to repeatabillty
straíght !ine is cOíJ1plicated by 1he fact that both ti\
and
~ are estímates and thus subject to error. But as the
and reproducibiJity standard deviations, but are pre­
slopa b ís usually small (af the order of 0.1 or I&ss).
sented here for repeatability only in the ímerests of
brevity, Only three types 01 relationship wil! be con~
then errors in m have little influence and the errors in
estímating s predominate.
sidered:

1: $, = bm (a straight line through the origin) 7.5.6.1 A good estirnate of the parameters of the
regression líne requires a weíghted regression be­
11. Sr = a + bm (e straight line with a positiva inter­ cause the standard error 01 ,f is proportional 10 1he
capt) predícted value of Sj (~).

14

Copyright by the IHTERNATlOHAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


Tue Jan 16 16:58:57 1996
.. 4851~D3 0594528 345 ..
tl ISO ISO 5725-2:1994IEJ

The weightíng factors haya to be proportional to 7.5.6.4 For relatíonshlp 11, the mitial values Sal are the
l/(si. where ~ is the predicted repeatabirlty standard original values of s as obtained by the procedures
deviatíon fQr level j. However si
depends on par­ gjven in 7.4. These are used to calcutate
ameters that have yet to be calculated
WOj = 11(so/ U=:; 1, 2, .... q)
A mathematícally correet proeedure fer fíndíng esti­
mates cerresponding to the weighted least-squares and to calculate a, and b¡ as in 7.5.6.2.
of residuals may be comp¡¡cated. The following pro­
cedure, wliich has preved to be satisfactory in prac­ This leads to
tice, is recommended.

7.5.6.2 With weighting factor W¡ equal to 1/(~l, The computations are (hen repeatad with
where N '" O, 1, 2 ... for successive íterations. tlien Mi,) = l/{s,l
to produce
tlie calculated formulae are:
;2j=lJ:l + b¿~j

Tha sama procedure could now be repeated once


again with weighting factors WZ1 = 1/(8'1/ derived
from these equations, but this will only lead 10 unim~
portant changes. The step from WOj to Wlj is eHective
in eliminating. gross errors in the weíghts, and the
equations tar S2j shQuld be considerad as the final re­
sult.

7.5.7 The standard error of Ig s is ¡ndependent af s


and so an ur-weighted regression ot Ig 8 on Ig 16 m
appfQpriate.

7.5.8 For relationship 111, the computetíonal formula e


are:
Then for relatíonsllip i (s = bm), the value of b is glven
by T5 IT'J'

For relatíonship 11 (8 -= a +

a= ... (25)

and

... (26)
T4 ... ¿ (19 mj) (lg Sj)
i

7.5.6.3 For relationsliip 1, algebraic substitutíon fer and thencé


=
the weighting factors W, = , f(~}2 with ~ bm) leads
to the simpljfjed expression: T2 T3~ TI T4
e- ......::;.--F---';;--'- •.. (28)
- ii,-~
¿(sin¡) and
b "" .-:..J__- ­ ... (27)
q
... (29)
and no iteratíon is necessary.

15

Cop~right by the INTERNATIONAL ORGANIZATION FOR 3TANDARDIZATION (130l


Tue Jan 1ó 1ó:58:57 199ó
. . 4851903 0594529 281 . .

. ISO 5725-2:1994(El o ISO

',,-- 7.5.9 Exampres of fittirtg relationships 1, I! and 111 of 7.5.9.1 An example of fitting relatio0shlp I is given
7.5.2 to the same set of data are now given in 7.5.9.1 in table 1.

to 7.5.9.3. The data are taken from the case stvdy of

B.3 and have been used here only to iIIustrate the 7.5.9.2 An exampre 01 fitting retationship II is given

numerical procedure. They will be further discussed in tabla 2 (mi' si are as in 7.5.9.1).

in B.3.

7.5.9.3 An example of fitting relatíonship 111 IS gíven


in table 3.

Table 1 - Relationshlp 1: .r = bm
mi 3,94 8,28 14 18 15.59 2D,41
0,092
1 .
0,127
sI 0.179 0.337 0.393

~/mJ 0.0234 0.021 6 I 0,008 9 0,021 6 0.0193

~(Jjlmj) 0.094 B
= 0,019
b --q­ 5

s=bm 0,075 0,157 I 0,269 0,296 O,38S

Tabl~ 2 - Relationship 11: s = a + bm


Wo¡ 118 31 62 B,B 6.5

31 0,058 + 0,009 O m

S"u 0,093 0,132­ 0.185 0,197 j 0,240


W\j 116 57 29 26 17

El = 0,030 + 0,015 6 m
,
S2J 0,092 0,159 0,251 0,273 0,348
w 118 40 16 13 B
"
SJ 0.032 + 0,015 4 m

S3j 11 0,093 0,160 0,251 0,273 0,3413

NOTE - The values of the weighting factors are not crítical; íwo significant figures suffice.

1) The díHerence from s2 ís negligible,

Table 3 - Relationship 111: Ig S"" e + d Ig m

19 1ÍiJ
19 so,
I + 0.595
- 1,036
+ D,91S
- 0,747
+ 1,152
- 0,896
I + 1,193
0.472
I

!
'1 1.310

-0.406
19 s = ­ 1,506 5 + 0,772 Ig m
or & - 0,031 mO,}]

S 10,089 0,158 0,239 ! 0,257 [ 0,316

16

:op~right h~ the IHTERHATIONAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


flle Jan 1& 1&:58:57 199&
.. 4551903 059~530 TT3 . .
<O ISO ISO 5725-2:1994/E)

1.6 Statistical analysis as a step-by-step 7.6.6 Investigate whether there ís or may be some
procedure technical explanation for the stragglers and{or stat·
istical outlíers and, if possible, verfty su eh an expla­
NOTE 5 Figure 3 índicates in a slepwíse fashlon the pro· nation. Correet or discard. as required, those
cedure given in 7.6. stragglers andjor statistical outliers tha1 have been
satisfactorily explained, and apply corresponding ccr­
rections to the forms. If there are no stragglers or
7.6.1 Colreet all available test resul1s in ene form, statistical outlíers lef1 that nave not been explaíned,
10rm A of fjgure2 (sea 7.2). It is recommended that ignore steps 7.6.7 to 7.6.10 and proceee direCtly with
this form be arranged ¡nto p rows, indexed 7.6.11.
i = 1, 2, ... p (representing the p laboratorías that
have contributed data) and q columns, índaxed NOTE 6 A large number 01 stragglers and{or statistical
j = 1, 2, ", q (representing the q levels in ínereasing outliers may indícale a pronounced variance inhomogeneity
arder). or pronounced dífferences between laborataríes and may
lhereby cast doubt on the suitability of the measurement
In a uniform-Ievel experiment the test results within method. This should be reported lo the panel.
a cel! of form A need not be distínguished and may
be put in aoy desired crder. 7.6.7 If the distríbution of the unexpkllned stragglers
e
or statistical outliers in form B or does not suggest
any outlying laboratoríes {see 7.2.5L ignore step 7.6.8
7.6.2 Inspect form A tor any obvlous írregularities, and proceed directly with 7.6.9.
írwestígate ando if necessary, discard any obviously
erroneous data (tor example, data outside the range 7.6.8 If tne evidence against sorne suspected
of the measuring mstrument or data which are ím­ autlying laboratDríes is cOf"Isidered strong enough to
possible for technical reasons) and rsport to the panel. iustífy the rejectían of some or all the data from those
It is sometimes ímmediately evident that the test re­ laboratoríes, than discard the requisite dala and report
sults 01 a particular laboratory or in a particular cell líe to the panel.
at a level ínconsistent with the other da1a. Such obvi­
ously discordant data shall be discarded immedlatelv, The decision to rejeet some or al1 data from a partic­
but the fact shall be reported to the panel fer further ular laboratory is the responsibility of tlle stalistical
consíderation (see 7.7.1 ). expert carrying out the analysis, but shall be reported
to the panel tor further consideration (see 7.7.1).

7.6.3 From form A, corrected according to 7.6.2 7.6.9 lf any stragglers andlor statistical outliers re­
when needed, compute form B containing cel! means main that have not been explained or attributed to an
e
and form containing. measures of wíthin·ceil spread. outlying laboratory, discard tha statlstical ou1liers but
retain the stragglers.
When a cell in form A contains only a single test re­
sult, one of the options of 7.4.3 should be adoptad. 7.8.10 If in the previous steps any entry in form B
has been rejected. then the corresponding entry in
form C has lo be rejected also. and vice versa.
1.6.4 Prepare the Mandel h and k plots as descríbed
ln 7.3.1 and examine tham for consístency oi t.he 7.6.11 From the entries that have been retained as
ctata~ TheseRlots may indicate the suitabilíty Qf tlle correet in forms B and e, compute, by the procedures
data for further analysis, the pre,sence of any possíbie given in 7.4, for each leve l separately, the mean level
outlying values or outlying laboratories. However, :'la In) and tlle repeatabilit'{ and reproducibilíty standard
definite decisions are taken at 1his stage, but are de· deviations.
layed until completion of 7.6.5 to 7.6.9.
¡Ji.12 If the experímerlt only used a sll1gle level, or
if 11 has been decided that the repeatabiiiw and re­
7.6.5 Inspect forms B end e (see flgure 2) level by
producibility standard deviations should be given sep­
level fo r possible stragglers and/or statíst]cal outliers
arately for each level (see 7.5.1) and not as functions
[sea 7.3.2.1 al]. Apply the statistical tests given in 7.3
of the level, ignore steps 7.6.13 to 7,6,18 and proceed
te all suspect ítems, marking the stragglers wíth a
dírectly with 7.6.19.
single asterisk and the statistical outliers with a dou­
ble asterisk. lf there are no stragglers or statistícal NOTE 7 The following steps 7.6,1 3 lo 7.6.1 7 are applíed
oulliers, 19nore steps 7.6.6 tú 7.6.10 and pracead di­ 10 ;sr aOO separately, but lor oreviw they are wrilten out
SR
rectly with 7.6.11. only in terms al sr

11

Copyright by the lHTERHATlOHAL ORGAHIZATlOH FOR STAHDARDlZATIOH (ISO)


Tue Jan 16 16:5B:57 1996
. . 4dS1903 0594531 93T . .
ISO 5725·2:1994(E) e ISO

Ora", up forrn Á.

Oiscurd discordant OarQ.

Computa forms B and C.

Prepure Ma.ndeL's h und k plots.

No

Oiscc.rd correspondfng
entry in lorm 8 01" C.

Figure 3 - Flow diagram o, the principal steps in the statistical analysi:s (continued Q() page 19)

18

Cop~r¡9ht b~ the INTERNAtlONAL ORGANIZATION FOR STANDARDIZATION (ISO)


Tue Jan 16 16:58:57 1996
. . 4851903 0594532 a7b . .
ISO 5725-2:1994(EI

Compute. for eQcllleVEL sepnrntely.


usiog the pro<;edures givcn in 7.4:
- mean 11l;

- r~p~alah¡lity standard óevinllon s,;

- reproduclbiut., staMord deviation SI!.

Calculate Ihe vulue¡¡ oi S,, ano


SR to opply ID aH \lalU~5 ot m.

Ol)f(l;n Ihe linear relolionship


by applying the computatlonaL
Yes procedure given in 7.5.

Obtain Ihp linear t'elatiQnship


by tlpptying Ihe computationat
Pl'oceduf'e ¡¡iven io 7.5.

tstnbLish that retatiooship.

Report resll11s lo punet (1.71.

Figure 3 - Flow diagram of the principal steps in the statistical analvsis

19

Copyright by the IHTERHATIOHAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


Tue Jan 16 16:58:51 1996
. . 4851903 OS~4S33 702 . .

ISO 5725-2: 1994rE) OISO

7.6.13 PIQt sJ agamst n~ and Judge frOn! tnis plot panel. in thls repon me followíng informatíon should
whether s depends on m or not. If s is considered to be given:
depend on m, ignore step 7.6.14 and proceed with
7.6.15. If s is judged to be independent of m, proceed al a full account of the observatíolls received f[Qm
with step 7.6.14. If there should be doubt, it is best the operators and/or supervisors concerning the
to work out both cases and let the panel decide. standard tor the measurement method;
Thare exists no useful statistical test appropriate for b) a fuI! account of the laboratories that have bse/l
thís probJem, but ¡he technical experts familiar with reíected as outlying labarataries in steps 7_6.2 and
the measurement method should have sufficient ex­ 7.6.8. together wíth the reasons for their re-­
perience to take a decision. jection;

7.6.14 Use +LSJ= sr as the final value of the re­ c) a full account of any stragglers and/or statístical
outliers tha! were discovered, and whether these
peatability standard deviation. Ignore steps 7.6.15 to
7.6 18 and proceed directly wíth 7.6.19. were explained and corrected, or díscarded;
Mj , .~,. and ,'iN and an ac­
d) a form of the final rasults
1.6.15 Judge from the plot of 7.6.13 whether the count of the conclusions rsached in steps 7.6.13,
relalionship between s and m can be representad by 7.6.15 or 7.616. illustrated by one of the plots
a straight line and, if so, whether relationshíp I recommended in these steps;
(s hm) or relationship 1I (.1 = a + bm) is appropri<'lte
(sea 7.5.2). Determine the parameter b, or the two e) forms A, B and e (figure 2) usad in the statlstlcal
parameters a and b, by the procedure of 7.5.6. If the analysis. possibly as an annex.
linear reJatíonship is considered satisfactory, ignore
atep 7.6.16 afld proceed directfy wfth 7.6.17. If noto
prooeed with step 7.6.16. 1.7.2 Decisions 10 be taken by the panel

7.6.16 Plot Ig Sj against Ig fñj and judge from this The panel should then discuss this report and take
whe1har the ralationship between 19 s and 19 m can decisions conceming the following questions.
reasonably be representad by a stralght Ime. It thls is
al Are the discordant ri3sults, stragglers or outliers.
considerad satisfactory, fit the relationship 111
if any, due to defects in the descrjption of the
(Ig s = e + d 19 m) using me procedure given in 7.5.8.
standard for the measurement method?
7.6.17 11 a sa1isfactory relation has been estabrished b) What aotíon should be taken w[th respect to re­
in step 7.6.15 or 7.6.16, then the final values of Sr (or jected outlying ¡abaratarías?
SR) are the smoothed values obrained from this re­
el Do tlle r9sults of the outlying laboratories andlor
lationship tor given. values of m. Ignore step 7.6.18
the commen1s received from the operators and
and proceed with 7.6.19,
supervisors indicate the need to ímprove the
standard for tha meaSurement method? If so,
7.6.18 If no satisfactory relation has been estab­
what are the improvements required?
lished in step 7.6_15 or 7.6.16, the statistical expert
should decide whether some other relation between d) Do the results of the precíslon experiment justify
< Ano m GI'!f1 b~ established. N alternatilfel'l \,vhethe r the establishment of values of the P9peatabilit'i
the data are so irregular that the establishment of a standard deviation and reproducíbility standard
functíonal relationshíp ia considered to be ímpossible. deviation? If so, what are those valves, in what
form should they be published, and what is the
7.6.'19 Prepare a report showing the basic data and regian in which the precisíon data apply?
the results and conclusions from the statistical analy­
siso and present this to the panel. The graphícal pres­
entatíons of 7.3.1 may be useful in presenting thtt 7.7.3 Full report
consístency or variabilíty oE the results.
A report setting out the reasons for the wQrk and how
7.7 The report tD, l,lnd the decisions to be it was organized, including the rapart by the statisti­
taken by, the panel cían and setting out agreed conclusions. should be
prepared by the executive officer for approval by the
7.1.1 Report by the statistical expert panel. Sorne graohical presentation of consistencv or
variability is oiten useful. The report should be circu­
Havlng completad tha statistlcal analysis, the statisti­ lated to those responsible for authorizing tlle work
cal expert should wdte a report to be submitted te the and to cther interested parties.

20

:opyright hy the IMTERHATIOHAL ORGAHIZATIOH FOR STAMDARDIZATIOH (ISO)


fue Jan 16 16:58:57 1996
. . 4aS1903 0594534 ~49 ..
clSO ISO 5725-2:1994(E)

8 Statistical tables
8.1 Critical values for Cochran's test (see 7.3,3} are
given in table 4.

Table 4 - Critical valu8s for Cochran's test


n=2 n 3 11~4 ,,=5 n 6
p
1% 5% 1 % 5% 1 % 5% I 1% 5% 1 % 5%

2 - - 0,995 0,975 0,979 0,939 0,959


0,834
0,906
0,746
0,937 0,877
0,707
3 0.993 0,967 0,942 0,871 0,883 0,798 0,793
4 0,968 0.906 0,864 0,768 0,781 0,684 0.721 0,629 0,676 0,590

¡ 5
6
0,928
0,883
0,841
0,781
0,788
0,722
0,684
0,616
0,696
0,626
0,598
0,532
0.63.'3
0,564
0.544
0,480
0.589
0,520
0,50e
0,445
l 0,838 0,727 0,664 O,56'~ 0,568 0,480 0,508 0,431 0,466 0,397
8 0,794 0.680 0.615 0.516 0,521 0,438 0,463 0,391 0,423 0.360
9 0,754 0,638 0,573 0,478 0,481 0,403 0,425 0,358 0,387 0,329
10 0,71$ 0.602 0.536 0.445 0.447 0,373 0.393 0,331 0,357 0,303
11 0,684 0.570 0.504 0.417 0,418 0,348 0.366 0,308 0,332 0,281
12 Q,S53 0,541 0,475 0,392 0,392 0.326 0.343 0,288 0,310 0,262
13 0,624 0,515 ~.450 0,371 0,369 0,307 0.322 0,271 0,291 0,:243
14 0,599 0,482
• 0,427 0.352 0,349 0,291 0,304 0.255 0 274
1 0,232
15 0,575 , 0,471 0,407 0,335 0.332 0.276 0,288 0,242 0,259 0,220
16 0,553 0,452 0,388 0.319 0,316 0,262 0,274 0,230 0,246 0.208
17 0.532 0,434 0.372 0,305 0,301 0,250 0.261 0.219 0.234 0,198
18 0,514 0,418 0,356 0.293 0,288 0.240 0,249 0.209 0,223 0,1!3S
19 0,496 0.403 0,343 0,291 0,276 0,230 0,238 0,200 0,214 0.181
20 OA80 0,389 0,330 0,270 0,265 0,220 0,229 0.192 0,205 Q.174
21 DAS!: 0,377 0,318 Oj2o~ O.,"'",
,_...r,,", 0,212 0.220 0,185 0,197 0.167
22 0.450 0,365 . 0,307 0,252 0.246 0,204 0,212 0,178 0,189 0.160
23 0,437 0,354 0,297 0,243 0.238 0,197 0,204 0.172 0,182 0,155
24 0,425 0,343 0,287 0.235 0.230 0,191 0.197 0,166 0,176 0.149
25 0,413 0,334 0,278 0,228 0,222 0,185 0.190 0.150 0.170 0.144
26 OA02 0.325 0.270 0.221 0,215 0,179 0.184 0,155 0,164 0,140
27 0,391 0,316 0.262 0,216 0.209 0,173 0,179 0.150 0.159 0,135
28 0,382 0,308 0,255 0,2Q9 0,202 0,163 0.173 0.148 0,154 0.131
29 0,372 ' 0,300 0,248 0,203 0.196 0,164 0,168 0.142­ 0,150 0,127
30 0,363 0.293 0,241 0,198 0,191 0,159 0,164 0,138 0,145 0.124
31 0,355 0,286 0,235 0,193 0,186 0,155 0,159 0,134 0,141 0.120
32 0,347 0,280 0,229 0,188 0,181 0,151 0,155 0,131 0,138 0,117
33 0,339 0,273 0,224 0,184 0,177 0,147 0,151 0,127 0,134 0,114
34 0,332 0,267 0.218 0,179 0,172 0.144 0,147 0,124 0,131 0,111
35 0.325 0,262 0.213 0.175 0.168 0.140 0.144 0,121 0,127 0,108
0,318 0,208 1 0,172 0,165 0,137 0,140 0,116 0,124 0,106
36 0.256
37 0,312 0.251 0,204 0,168 0,161 0,134 0,137 0.116 0,121 0,103
38 0,306 0,246 0.200 0.154 0,157 0.131 0,134 0.113 0,119 0.101
39 0,300 0,242 0,196 0,161 0,154 0.129 0,131 0,111 0,116 0,099
40 0,294 0,237 0,192 0,158 0,151 0.126 0,128 0,108 0,114 0,097
p = number 01 laboratories at a given level

n = number ot tesl resulls per cel! [see 7.3,3.3)

21

Copyright by the IKTERNATIONAL ORGANIZArlON FOR STANDARDIZATION (ISO)


Tue Jan 16 16:58:57 1996
.. 4851903 059~535 585 ..

ISO 5725-2:1994(E) 10 ISO

"-.-. 8.2 CritiCal values for Grubbs' test (sea 7.3.4) are For the Grubbs' test tor two outlying observations.
given ir: table 5. Qutliers and stragglers give rise to values which are
smaller than the tabulated 1 % and 5 % critical values
For the GrubbS' test for one outlying observation, respectívely.
outliers and stragglers give rise to values whlch are
larger than the tabulated 1 % and 5 % critical values 8.3 Indicators tor Mandel's h and k statistícs (ses
respectively. 7.3.1) are given in tables 6 and 7.

Table 5 - Critical values far Grubbs' test


One la rgest or one smaUest Two largest or two smallest
p
Upper 1 % Upper 5 % Lower 1 % lower 5 %

3
1,155 1.155 -
4
h496 ¡.<!Si 0.000 O 0,0002
5
1.764 1,715 0,001 8
0,009 O
6
1,973 1,887 0,011 6
0,0349
7
2,139 2,020 0,0308
0,0708
B
2,274 2,126 0,0563
0.110 1

9
2,387 2,215 0.0851 0.1492

10
2,482 2,290 0.115 O 0.1864

11
2,564 2.355 0,144 8
Ot22i 3

12
2,636 2.412 0,1738 0.2537

13
2.699 2,462 0,2016 0,2836

14
2,755 2.507 0.228 O 0.311 2

15
2.806 2.549 0.253 O 0.3367
16
2,852 2.585 0.2767 0.360 :;

17
2,894 2,620 0,299 O
0,382 :2

18
2,932 2,651 0,320 O
0,4025

19
2,968 2,681 0,339 a
0,421 4

20
3,001 2,709 0,358 5
0,439 1

21
3.031 2.733 0.3761 0,455 e

22
3,060 2,758 0,3927 0,471 1

23
3,087 2,781 0,4085 0,4857

24
3.112 2.802 0,4234 0,4994

25
3,135 2,822 0,4376 0,5123

26
3.\57 2.841 0,451 o 0,5245

27
3,178 2,859 0.4638 0.536 o
28
3,199 2,876 0,4759 0,547 o
29
3,218 2,893 0,4875 0,5574
30
3,2313 2,908 0,4985 0,5672
31
3.253 2,924 0.5091 0,5766
32
3.270 2,938 0,5192 0,5856
33
3,286 2,952 0,5288 0.5941
34
3.301 2,965 0,5381 0.6023
35
3,316 2,979 0,5469 0,610 1
36
3,330 2,991 0,5554 0,6175
37
3,343 3,003 0,5636 0,6247
38
3,356 3,014 0,5714 0,6316
39
3.369 3,025 0.5789 0,636 :4
40
3,381 3,036 0,5862 0,6445
Reproduced, wíth the permission of the American Statistícal Assocíation. from re/arenea
(4) in annex C.

p =numbsr of laboratories at " given leve!

22

:opyrÍght by the lMTERMATlOMAL ORGAMIZATlON FOR STAMDARDIZATIOM (ISO)

'ue Jan 16 16 :58 :57 1996

. . 4851903 059453b 411 . .

ISO 5725.2:1994(E)

Table 6 - Indicators for Mandel's h and k statistics al the 1 % significance level


fe
p }¡ n
2 3 4 S S 7 8 9 10
3 1,15 1,71 1,64 1,58 1,53 1,49 1,46 1,43 1,41 1,39
4 1,49 1,91 1,77 1,67 1,60 1,55 1,51 1,48 1,45 1,43
5 1,72 2,05 1,85 1,73 1,65 1,59 1,55 1,51 1,48 1.46
6 1,87 2.14 1,90 i,n 1,68 1,62 1.57 1,53 1.50 1,47
7 1,98 2,20 1,94 1,79 1,70 1,63 1,58 1,54 1,51 1,48
8 2,06 2,25 1,97 1,81 1,71 1,6S 1.59 1.55 1,52 lAS
9 2,13 2,29 1,99 1,82 1,73 1,66 1,60 1,56 1.53 1,50
10 2,18 2,32 2,00 1.84 1,74 1,66 1,61 1,57 1.53 1.50
11 2,22 2,34 2.01 1.85 1.74 1.67 1.62 1,57 1,54 1,51
12 2.25 2,36 2,02 1.85 1.75 1.68 1.62 1,58 1,54 1.51
13 2.27 2,38 2,03 1,86 1.76 1,68 1,63 1,58 1.55 1,52
14 2.30 2,39 2,04 1,87 1,76 1,69 1.63 1,58 1,55 1.52
15 2.32 2.41 2,05 1,87 1,76 1,69 1,63 1.69 1,55 1.52
16 2,33 2,42 2,05 1,88 1,77 1,69 1,63 1.59 1,55 1,52
17 2,35 2.44 2,06 1,88 1,77 1,69 1,64 1,59 1,55 1,52
18 2,36 2,44 2.06 1,88 1,77 1,70 1,64 1,59 1,56 1,52
19 2.37 2.44 2.07 1.89 1,78 1,70 1.64 1,59 1.56 1,53
20 2,39 2.45 2,07 1,89 1J8 1.70 1,64 1,60 1.56 1,53
21 2.39 2,46 2,07 1.8S 1,78 1.70 1.64 1,60 1,56 1,53
22 2,40 2.46 2,08 1.90 1,78 1.70 1,65 1,60 1,56 1,53
23 2.41 2,47 2.08 1,80 1,78 1,71 1,65 1.60 ',56 1,63
24 2,42 2,47 2,08 1,90 1.79 1,71 1.65 1,60 1.56 1,53
25 2,42 2,47 2,08 1,90 1,78 1,71 1,65 1,60 1.56 1,53
26 2,43 2,48 2,09 1,90 1,79 1,71 1,65 1,60 1.56 1,53
27 2,44 2,48 2,09 1.90 1.79 1.71 1,66 1,60 1,56 1.53
28 2,44 2,49 2.09 1.91 1,79 1.71 1.65 ) 1.60 1,57 1,53
29 2.45 2.49 2,09 1.91 1,79 1,71 1,65 I
1.60 1,57 1.53
30 2,45 2,49 2,10 1,91 1.79 1.71 1.65 I 1,61 1.57 1,53
p = number 01 labora tones al a gtVtlfl level I

n = nurnber 01 replicales wlthin each laboralcry al that level


NOTE ­ Supp[¡ed by Dr. J. Mande! and published with hís permission.

23

Copyright by the IHIERHATIOHAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


Tue Jan 16 16:58:57 1996
. . 4851903 0594537 358 . .
ISO 5725-2:1994(E) <!;J ISO

Table 7 - Indicators for Mandel's h and k statistics at the 5 % signñicance level

p /¡
"
n
2
3
4
5
5
7
8
9
10

3
1,15 1,65 1,53 1,45 1,40 1.37 1,34 1,32 1,30 1,29
4
1,42 1,76 1,59 1,50 1,44 1,40 1,37 1,35 1,33 1.31
!5 1,57 1,81 1,62 1,53 1,46 1,42 1,39 1,36 1,34 1,32
IS 1,66 1,85 1,64 1,54 1,48 1,43 1,40 1,37 1,35 1,33
7
1,71 1,87 1,66 1.55 1.49 1,44 1,41 1,38 1,36 1,34
8
1.75 1.88 1,67 1.56 1.60 1,45 1,41 1,38 1.36 1,34
9
1,78 1,90 1.6B 1.57 1,50 1,45 1,42 1.39 1,36 1,35
10
1.80 1.90 1,68 1,57 1.50 1,46 1.42 1,39 1.37 1,35
11
1,82 1,91 1,69 1,58 1,51 1,46 1,42 1.39 1,37 1.35
12
1,83 1,92 1,69 1,58 1.51 1,46 1,42 1,40 1,37 1,35
13
1,84 1,92 1,69 1,58 1,51 1.46 1,43 1,40 1.37 1,35
14
1,85 1,92 1,70 1,59 1,52 1,47 1,43 1,40 1,37 1,35
15
1,86 1,93 1,70 1,59 1.52 1,47 1,43 1,40 1.38 1.36
16
1,86 1.93 1,70 1.59 1,52 1,47 1,43 1,40 1,38 1.36
17
1,87 1.93 1,70 1,59 1,52 1.47 1.43 1,40 1.38 1.36
18
1,88 1.93 1,71 1,59 1.52 1.47 1.43 1.40 1,38 1,36
19
1,88 1,93 1,71 1.59 1,52 1.47 1.43 1,40 1,38 1.36
: 20
1,89 1,94 1,71 1,59 1,52 1,47 1,43 1,40 1,38 1,36
21
1,89 1.94 1,71 1,60 1.52 1,47 1,44 1,41 1,38 1,36
22
1,89 1.94 1,71 1,60 1,52 1,47 1,44 1.41 1,38 1,36
23
1.90 1.94 1,71 1,60 1,53 1,47 1,44 1,41 1,38 1,36
24
1.90 1.94 1,71 1,60 1,53 1,48 1,44 1,41 1,38 1,38
25
1,90 1.94 ' 1,7i 1,60 1,53 1.48 1,44 1,4'1 1.. 38
t36
26
1,90 1.94 1,71 1,60 1,53 1.48 1.44 1.41 ¡ 1.38 1,36
27
1.91 1,94 1,71 1,60 1.53 1,48 1,44 1,41 1,38 1,36
2B 1,91 1,94 1.71 1,60 1,53 1,48 1,44 1.41 1,38 1,36
29
1.91 1.94 1,72 1,60 1,53 1,48 1,44 1.41 1,38 1,36
30
1,91 1,94 1,72 1,60 1,53 1,48 1,44 1,41 1,38 1.36
p = number of laboratoríes at a gíven leve!

1I = number of repUeates within each laboratory at that leve!

NOTE - Supp!í€d by Dr. J, Mande! and published with his permíssion.

24

~op~right b~ the IHTERHATIOHAL ORGANIZATION FOR 3TANDARDIZATION (130l

fue Jan lE> 16 :59:57 1996

• 4B51903 0594.538 2Cj4 _


lb> ISO ISO 5725-2:1994(EI

Annex A
{normative)

Symbols and abbreviations used in ISO 5725

a Intercept in the relatíonship k Mandel's within-Iaboratory COfisistency test


statistic
s=a+bm
LCL Lower control limit leither actíon límít or warning
A Factor used to calculate the uncer­
limitl
tainty of sn estímate
m Genaral mean of the test proper1';': level
h Slope in the re!atiooship

>""a+bm M Number of factors considered in imermedtate


precislon conditiofiS
B Component io a test result repre­
sen1íng the deviation of a laboratory N l'Jumber of iterations
from the general average (laboratory
n Number of test results obtained in ane labora­
component of bias)
tary at one level (Le. par cell)
Component of B representing all
p Number "f labaratarles partrcipating in the inter­
factors that do not change in inter­
laborato!'{ experiment
mediate precision conditions
P Probablhty
Componenta of B representing fac­
tors that vary in intermediate pre­ q Number of levels of the test property in the
cision conditions ínterlaboratory experiment
Intercept In the relatlonSillp r Repeatability limit
Ig s= c+dlg m Reproducibility limit
R
e, e'. e" Test statistics
RM Reference material
CClll' C'crrt' C"cnt Critical values tor statistícal tests
Estímate of a standard deviatlon
Critical differenca tor probability P i\
S Predicted standard devia1ion
Critical range for probability P
T Total or sum of sorne expression
Slope in the relationship
Ig s =: e + d Ig ni Number of test objects or groups

Component in a test result repre­ UCL Upper controllimit (either actíon límit or warning
senting the random error occurríng límítl
in every test result
W \'Yeighting factor used in calculating a weighted
f Critical range factor regression
p-quantile of the F-distribution with
VI and iI.
degrees of freedom
w Range of a set of test results

G GrtJbbs' test statistic x Diltum usad for Grubbs' test

h Mandej's betwsen-Iaboratory con­ y Test result


sistency test statistic

25

Copyright by the INTERNATIONAL ORGANIZATION FOR 3TANDANDIZATION (130)


tus Jan 16 16:58:57 1996
.. ~851903 0594539 120 . .
ISO 5725-2:1994(E) () ¡SO

Arithmetic mean of test results Symbols used as subscripts

Grand mean of test results C Calibration-dífferent

Significance tevel E Equipment-different

Type 1I error probabilíty Identífier for a particular laboratory

y Ratio of the reproducibility standard deviatíon to lO Identifier tor intermediate measures of


the repeatability standard deviation (URJ(J,) precision; in brackets, ídentifícation of
the type of intermediate situatíon
.d Laboratory bias
j ldentifier for a particular level
"
.<1 Estimate of .<1 (ISO 5725-2).
Identjfier for a group of tests or for a
Ó Bias of the measurement method
factor (ISO 5725-3)
"¿j Estímate of {;
k Identifier for a particular test result in a
1 Detectable difference between two laboratQry laboratory i at levelj
bl8ses Of the biases 01 tWD measurement
L Betweerrlaboratory (interlaboratory)
methods
m ldentifier for detectable bias
Ji. Trua valua or accepted reference value of a test
property M Betweerr-test-sampl€l
)1 Number of degrees of freedom O Operator-different
() Detectable ratio betw99n the repeatability stan­ p Probability
dard devíations of meihod B and method A
r Repeatability
(l Trua value of a standard deviation
R Aeproducibility
1: Componer¡t In a test result representing the
variation due to time sioce last calíbration T Time-different

tjJ Detectable ratio between the square roots of W W¡thln-Iaboratory (intralaboratory)


the between-Iaboratocy mean squares of
method B and method A 1,2,3. For test results, numbering in the order
of obtaíning them
:.r:;(v) p-quantile of the l-distribution with y degrees
of freedom (1 J. 121, (3), .. FOf test results, numbering in the arder
01 increasíng magnítude

26

;"p~right b~ tbe INTERNATIOI1AL ORGANIZATION FOR STAHDARDIZATIOI1 (ISO)


rue Jan lr, 16 :58 :S7 1996
. . 4851903 OS9~540 942 . .
e ISO ISO 5725-2:1994{E)

Annex B
(informative)

Examples of the statistical analysis of precision experiments

Table 8.1 - Original data: Sulfur content of coal


B.1 Example 1: Determination 01 the
sulfur content of coal (Several levels with Level j
Laboratory i
no missing or outlying data' 1 2 3 4
0,71 1,20 1.68 3,26
B.1.1 Background 0,71 1.18 1,70 3,26
1
0,70
I 1,23 1,68 3,20
al Measurement method 0,71 1,21 1.69 3,24
0.69 1.22 1,84 3.20
Determinauoo of the sulfur content ín coai wlth 0,67
2 1,21 1,64 3,20
test results expressed as a percentage by mass. 0,68 1,22 1,65 3,20

b) Source 0,68 1,28 1,61 3,37


3 0,65 1,31 1,61 3,36
Tomkins, S. S. Industrial and Engineering Chem­ 0,69 1,30 1,62 3,38
istry. (Sea reference [6J in annex CJ 0,57 1,23 1,68 3,16
I
4 0,66 1,18 1.66 3,22
el Deseription 0,66 1.20 1,66 3,23
0,70 1,31 1.64 3,20
Eight laboratarles participated in the experiment.
0,69 1,22 1,67 3,19
carrying out the analysis accordíng to a standard­
5 0,66 1.22 1.60 3.18
ized measurement method described In the 0,71 1,24 1,66 3,27
SQurce cited. Laboratory 1 reported four test re­ 0,69 - 1,68 3,24
suíts and laboratory 5 reportad four 01 five; the
0,73 1,39 l,7Q 3,27
other laboratarías all carried out three measure­
6 0,74 1,36 1.73 3,31
ments. 0,73 1.37 1,73 3,29

d) Graphic:al presentation 0,71 1,20 1,69 3,27


7 0,71 1,26 1,70 3,24
Mandel 's h and k statistics should be plotted, but 0,69 1,26 1,68 3,23
becaw;e in this exarnple they showed little of not8 OJO 1.24 1,67 3,25
they haya been omitted in arder to allow space tor 8 0,65 1,22 1.68 3.26
a dífferent example of the graphícal presentation 0,68 I 1.30 1,67 3,26
01 data. Mandel's plots are fully iIIustrated and
discussed in the example given in B.3. NOTE 8 For ¡he el(periment quoted in tabla B.l, the lab­
ora1ories were not instructed as te how many measure­
ments were to be made, only a minimum number. By the
recammended proccdures glven In tt"',¡s par! 01 ISO 5725, tor
8.1.2 Original data laboratoríes 1 and 5 a random selection should be made
from the values given in arder to reduce ail cells to exactly
Thase are given, as percentage by mass [% {mlm))' three test results. Hewever, in order to iIIustrale the com­
in tabla B.l in the format of form A of flgure 2 (sea putatronal proceduras for variable numbers 01 test results.
7.2.8) ond do not invite any specific remarks. all test results have beeo retained in thls example. The
reader may malee random selectioos to reduce {he number
Grdphlcai presenlaúons oi ll1esa data are glven in 01 test results 10 three ¡Ji each cell íi hc/she wlshes to venfy
tha1 such a procedure has relatively Iltrle effcct on the val­
ures B.1 to 8.4
U9S of ~, s, and SR'

27

Copyr igl.t b\l tbe INTERNAr IONAL ORGAN lZAflON FOR STANDARD ¡ZATlON (ISO)
Tue Jan 16 16:58:57 1996
.. ~8519D3 0594541 589 . .
1$0 5725-2: 1994rE) o ISO

6.1.3 Computation of can means lY;j) For lavel 1, largest value 01 s ís in laboratary 8:

The cell means are given, as a percentage by mass ri = 0,001 82: test value = 0,347
[% lmlmlJ. in table 6.2 in the format of form B of fig·
ure2 (see 7.2.9).
For level 2, largas! value af s is ¡n labora1ory 5:
8.1.4 Computation of standard deviations
(sy) r:s 2 0,006 36; test value = 0,287

The standard deviatíons are given, as a percentage by


mass ['ro (mlm)], in tabla 8.3 in the format of form e For level 3, largest valua of ~. is in labaratary 5:
of fig ure 2 (see 7.2. lO!.
ri = 0,00' 72; test value = 0,598
6.1.5 Scrutiny fer consistency and Dutliers

Cochrao's test with n = 3 tor p = 8 laboratoríes gives Fer level 4, largest value of s ís in laboratory 4:
critical valuas of 0,516 for 5 % and 0,615 tor 1 %.
I:s
2
= 0,004 63; test value = 0,310

Tab'a 8.2 - Cell means: Sulfur content of coal


Levelj
I
Laboratory l 1 2 3 4
I
Yl] I /l" YZ! /111
! Y!l "v ji,! nt.1

1 0,708 4 1,205 4 1,688 4 3,240 4


2 0,680 3 1,217 3 1,643 3 3,200 3
3 0.667 3 1.297 3 1,613 3 3,370 3
4 0,660 3 1.203 3 1.667 3 3,203 3
5 0,690 5 1.248 4 1,650 5 3,216 5
6 0,733 3 1.373 3 1,720 3 3,290 3
7 0,703 :; 1,240 :; 1,690 3 3,247 3
8 0.677 3 1,253 3 1,673 3 3,257 3

Table B,3 - Standard devlatlons: Sulfur content of coal


lI~val j

laboratory i 1 i 2 ! 3 4
I
slj nv &,¡ ti¡¡ slí lIij ,f
V nü

1 0,005 4 0,021 4 0,010 4 0,028 4


2 0,010 3 0,006 3 0,006 3 0.000 3
3 0,021 3 0,015 3 0,006 3 0,010 3
I
4 0.010 3 0,025 3 0,012 3 0,038 3
5 0,019 5 0,043 4 0.032 5 0,038 5
6 0,006 3 0,015 3 0.017 3 0,020 3
7 0,012 3 0,035 3 0,010 3 0,021 3
8 0,025 3 0.042 3 0,006 3 0,006 3

28

;opyr;ght hy the IHTERHATIOHAL ORGAHIZATIOH rOR STAHDARDIZATIOH (ISO)


'ue Jan 16 16 :58: 57 1996
. . 4851903 0594542 715 . .

olSO ISO 5725·2:1994(E)

1 i xx
X
X

2 xx

3 xx

4 XXX

5~ x XX

x
6~ xx

x
7~ x

8 x X X

0.65 0.70 Q,75 0,80 m. %


1
m,
Figure 8.1 - Sulfur content 01 coal, sampls 1

XxX)(

X
2~ xx

3 XI<X

xx x
""

51 h x

)<Xx
'"
,~ x !5

e xx x

1.1 1,2 1,3 1,4 1.5 m,%

m",(

Figure B.2 - SuHur content of coal. sample 2

29

Copyright by the IHTERHATIOHAL ORGANIZATIOH FOR STANDARDIZATIOH (ISO)


Tue Jan ló ló:51l:57 1996
.. 4aS~903 OS945~3 651 ..

ISO 5725-2:1994(E) e ISO

xxxx
1~
x
z{ xx

x
3{ xx

x
4{ x x

si x x ) xx

6~ x ~

? xxx

e{ ~

1.50 l.SS 1.60 1,65 1,70 1,75 1,80 1,8S (D, %

Figure B,3 - Sulfur contetlt of coal, sample 3

1{ x ~ Ix

z{ x
X

3 xxx

x
xx I
S xxx x x

6 xxx

7 x> x

8~ xx

3.0 ~,1 3,2 3,3 M 3.5 ),6 m,%

m.

Figure 8.4 - Sulfur content of coa!, sample 4


~

30

;opyright by the INTERHATIOKAL ORGANIZATION FOR STANDARDIZATION (ISO)


iue Jan 16 16:58:57 1996
. . 4651903 059454~ 598 . .
10 ISO ISO 5125-2:1994{E)

Th¡s indlcates that one cell in level 3 may be regarded


as a straggler, and there are no outliers. The straggler
is retalned in subsequent calculations.
[;3~ _~i -3:] [T;i_ T~) ]
Grubbs' tests were applíed to the cell means, giving = 0,000 460 3

the values shown in table BA. There are no single


stragglers or outliers. At levels 2 and 4, the high re­
. .~ :; si + s; ;:; o,obo 692 5

sults for laboratorles 3 and 6 are stragglers according '1'

to the double-high test; these were retained in the m= ":.L


T
= 0,690 44
3

analysis.
S,. = 0,015 24

Sil:; 0,026 32

8.1.6 Computation of mJ' Srj and SMj


Th€ calculations lor levels 2, 3 and 4 may be carned
The varillnces defrned in 7.4.4 and 7.4.5 are calculated out similarly to give the results shown in tabla 8.5,
as follows, uSlng level 1 as an example.
8.1.7 Oependence of precision on ni
Number of laboratorias, p = 8
An examinaÍlon oi the data in tabla 8.5 does not Indi­
T1 = Ln¡ ji; 18,642

cate any dependence and average values can be


T2
1:n¡ (y;)2 '" 12,883 7
used.

13 LJl¡ 27
8.1.8 Conclusions
T4
= r.n,2 = 95 The precision of the measurement method should be
quoted, as a percentage by mass, as
T5 '= L(n¡ 1)s¡2 = 0,004411

repeatabihty standard devlatlon, sr = 0,022


2
Sr == T"

-'1'-- = 0,000 232 2


·3 P reproducibility standard deviation, SR = 0,045

Table B.4 - Applícation of Grubbs' test to cel! means

Level
Single
low I Singl.
hlgh
Double
low I Double
high
Type oftest

1 1,24 1,80 0,53!) 0,298


:2 0,91 2,09 0,699 0,108 Grubbs' test
3 1,67 1.58 0,378 00469 statistics
4 0,94 2,09 ! 0,679 0.132
Stragglers 2,126 2(126 I 0.110 1 0.110 1 Grubbs' critical
Outliers
2,274 2,274
I 0,0563 0,0563 valUEts

Table B.5 - Computed values of mi' Sr] and SRi for sulfur content o, coa.
Level} f A

Pi /'ti., 1r¡ 5J1J

1 8 0,690 0,015 0,026


..,2. e 1,252 0,029 D,De1
<J 8 1.667 0,017 0,035
4 a 3,250 0,026 0.058

31

Cop~right b~ the IHIERHATIOHAL ORGAHIZArlOH FOR 3TAHDARDIZATIOH (130)


Tue Jan 16 16:58:57 1996
.. ~851903 059~5~5 424 . .
ISO 5725-2:1994IE) ID ISO

These values may be applied within a range order to provida for another type of graphical
0,69 % (m/m) to 3,25 % (mlm). They were deter­ presentation of data. Mandel's piats are fully
mined from a uniform-Ieve! experíment involving 8 illustrated and discussed in the example given in
laboratories covering that range of values, in which 8.3.
four stragglers were detectad 8nd retained.
8,2.2 Original data
These are presentad in table B.6. in degrees Celsius,
B_2 Example 2: Softening point of pitch in the format of form A of figure 2 (see 7.2.8).
(Severallevels with miss;ng data)
Table 8.6 - Original data: Softening point of
8.2.1 Background pítch (oC)

al Measurement metl1Qd Laboratory ,


I1 Lavel j
I 1 I 2 3 i <1
Tha determination of the sottening point of pitch
91,0 97,0 96,5 104.0
by ring and ball. 1
89,6 1 97,2 97,0 104,0

b) Source 2 i
89.7
89,8 I 98,5
97,2
97.2
97,0
102,6
103.6

Standard methods for testing tar and ¡ts products;


Pitch section; Method Serial No. PT3 using neutral
3
88.0
97,5 I 97,8
94,5 I
94,2
95,8
103,0
99,5
glycerine (reference (5J in annex el. 89.2 96,8 96,0 102,5
4
88,5 97,5 98.0 103.5
el Material 89,Q 97,2 98,2 101.0
5
90,0 - 98,5 100,2
Thls was selected from comrnercial batches of
88,5 97.8 99,~ 102.2
pitch collected and prepared as specified in the 6
ff
"Samples chapter of the pitch section of refer­
90,5 97,2 103,2 I 102.0
,,--. enea [5]. 88,9 96,6 98,2 102.8
7
88,2 97,S 99,0 102.2

dI Description B - 96,0 98,4 102,6


97,5 97,4 103.9
This was the <:Ietermínatlon of a propert'¡ involving 90.1 95):; 98,2 102,8
9
temperature measurement in degrees Celsius. 88,4 96,8 96,7 102.0
Sixteen laboratorles cooperated. It was ¡ntended 86,0 95,2 94,8 99.8
to measure four specimens at about 87,5 'C, 10
85,8 95,0 93.0 100.8
92,5 'C, 97,5 ·C and 102,5 ·C to cover the normal
87,6 93,2 93,6 98.2
commerciai ranga Qf products, but wrong materíal 11
84,4 I 93,4 93,9 97,8
was chosen tor level 2 with a mean temperature
88.2 I 95,8 95,8 101,7
oi about 96 oC which \-vas similar to level 3. Lab­ 12 I 101,2
oratory 5 applied the method incorrectly at 1irst 87,4
I 95.4 95,4
on the sample tor level 2 (the fírst one they
13
91.0
90,4
I 98.2
99,5
98,0
97,0
I• 105,6
104,5
measuredJ ano there was then insufficient ma­
terial remaining for more than one determination. 87,5 97,0 97,1 105,2
Laboratory 8 found that they did not have a sam­ 14
87,$ 95,5 96,6
I 101,8
pie for level 1 (they had two specimens for level
87,5 95.C 97.8 101,5
4). 15
87,6 95.2 99.2 100,9
88,8 95.0 97,2 9$,5
e} Grsphical presentations 16
85,0 93,2 97.8 I 99,8
Mandel's h and k statistics should be plotted, but NOTE - There are no obvious stragglers or statistícal
again in this example they have been omitted in , outliers,

32

:opyright by the INTERNATIOHAL ORGANIZATION FOR STANDARDIZATIOH (ISO)


'ue Jan 1616:58:57 1996
.. 4851903 D59454b 3bO ..
clS0 ISO 5725-2:1994(E}

B,2.3 Cell means 8.2,4 Absolute differenees within cetls


These are gil/en in table B,7, in degraes Celsius, in the In this example there are two test results per cel! and
format of form B of figure 2 (sea 7.2.9). the absolute difference can be used to represent the
variability. The absoluta dífterences wlthin cells, in
A graphical presentation of these data ¡s given in fig­ degrees Celsius, are given in table B,8, in the format
ureB.5. of form e of figure2 (see 7.2:10).

A graphical presentatían of these data ís given in fig­


ureB.6.

Table B.7 - Cell means: Softening point of pitc:h (cC)


Levelj
Laboratory i
1 2: I 3 4
1 90,30 97,10 96.15 104,00
2
3
89,75
87,75
97,85
96,16
97,10
95,00 ¡ 103,10
101.25
4 88.85 97.15 97,00 103,00
5 89,50 - 98,35 100,60
6 89,50 97,50 101,35 102,10
7 88.55 97,05 98,60 102.50
8 - 96,75 97,90 103,25
9 89,25 96,15 97,45 102,40
10 85,90 95,10 93,90 100.30
11 86.00 93,30 93,75 98,00
12 87,8Q 95,60 95.60 101,45
13 90,70 98,85 97.50 105,05
14 87.65 96,25 96,85 103,50
15 87,55 95,10 98,50 101.20
16 86,90 94,10 97,50 99,65
NOTE - The entry for i = 5. j 2 has been dropped (sea 7.4.3)

Table 8.8 AbsoJute differences within cells: Softening point of pitch ("C)
Level}
Laboratory i I 1 2
I
3 4
i

1 1,4 0,2 0,5 0,0


2 0,1 "\
'.-'
'j I 0,2 1,0
3 0,5 3,3 1,6 3,5
4 0,7 0,7 2,0 1,0
5
6
l.Q
2,0
-
0,6
0.3
3,7
0,8
0,2
7 0,7 0,9 0,8 0,6
8 1,5 1.0 1.3
9 1.7 1,3 1,5 0.8
10 0,2 0.2 1.8 1,0
11 3,2 0,2 0.3 0.4
12 0,8 0,4 0.4 0,5
13 0.6 1,3 1,0 1.1
14 0,3 1,5 0,5 3,4
15 0.1 0.2 1,4 0,6
16 I 3,8 1,6
I 0,6 0,3

33

Copyright b~ the IHTERHATIONAL ORGANIZATIOH FOR STAHDARDIZATION (ISO)


Tue Jan 16 16:58:57 1996
. . 4851903 0594547 2T7 . .

~o 5725-2:1994(E) <O ISO

B.2.5 Scrutiny for consistency and outliers Number of replicates. n = 2

Application oi Cochran's test leads to the va/ues of 11 = I:J¡ = 125,950 O


the test statístic e given in tabla B.9.
T2 = I:(y¡)i = 1 087,977 5
Th e critical values (sea 8.1} at the 5 % signiflcance
level are 0,471 for p = 15 and 0,452 for p 16 where 13 == (Jll ­ Yd 2 = 36,910 O
n 2. No stragglers are indicated.
2 T3
~'=- 1,2303
Grubbs' tests were applied to the cell means. No , 2p
single or double stragglers or outliers were found.
.2
~L
= [P12 - 11" ] _ J;
p(p _ 1) 2
= 1 557 5
'
B.2.6 Computation of mjl S,j and SRj

These are calculated as in 7.4.4 and 745 s~ sI + s; 2,7878

Using level 1 tor example, the calculatíons are as fol­


Jows. To ease tha arithmetic, 80,00 has been sub­
"
m =-ji
11
(add 80.00) = 88,396 6

tracted from all the data, Tile method tor tI = 2


s, = 1.1092

replícatas per cel! is used.


SR 1,6697

Number 01 laboratories, p 15
The values for all tour levels are given in tabla B.l1.

Table 8.9 - Values of Cochran's test statistic, e


Levelj 1 2 3 4

e 0,391 (15) 0,424(15) 0.434 (16) 0,380/16)

NOTE - Number of laboratories ís given in parentheses.

Table B.10 - Applicatíon o, Grubbs' test tQ tell means


Single Single Double­ Oouble
Lavel; n Type of test
low high low high
1: 15 1,69 1,56 0.546 0,002
2; 15 2,04 1,77 0.478 0,646 Grubbs' test
3; 16 1,76 2,27 0,548 0.566 statistics
4; 16 2.22 1,74 0,500 0.672
Stragglers
n,., 15 2.549 2,549 0,3367 0,3367
n = 16 2,585 2.585 0.3603 0,3603 Grubbs' critica I
Oullíers
!
I ""3Iues
11 ~ 15 2,806 2,806 0,253 O 0,253 O
11 = 16 2,852 2,852 0,2767 0,2767 J

34

:opyright by the IHTERHATIOHAL ORGAHIZArlOH FOR 8TAHDARDIZATIOH (180)


Tue Jan 16 16:58:57 1996
. . 4851903 0594548 133 . .
(.) ISO ISO 5725-2:1994(E}

Table 8.11 - Computed values of mJ' Sr! and SR] for $oftening point of piteh
!
Level¡ ~
m) (OCI sd s"J

1 15 88,40 1,109 1,670


2 15 96,27 0,926 1,597
3 16 97,07 0.993 2,010
4 16 ¡
101,96 1,004 1,915

:v:
15 9 ti
110 12 7 4
1~1116fS ) 6 z 11HI leveL 1,

16
n
9

~
14 ~
15 9 4
12 9 '2
111116 10 ) 1 131
level 2

i5 ro;­
;¡ 7
116 12 7 51,'
110111 3 4 2 1 1

80 90 100 110
Temperature, oc

Figure 8.5 - Softenir.g point of pltch: Ca!! maans

35

Copyright by the IHTERHATIOHAL ORGAHIZArlOH FOR STAHDARDIZATIOH (ISO)


Tue Jan 16 16:58 :57 1996
. . 4851903 0594549 07T . .

ISO 5125-2:1994(E) e ISO

~~LL~~~~____~~~~~________________~3~______""_____________ leve!2

2 3 S
Temperature. 'C

Figure B.6 - Softening point of pitch: Absolute differences within cells

6.2.7 Dependence of precision on m B.3 Example 3: Thermometric titration


01 creosote oil (Several levels with
A cursory examination of table 8_11 does not reveal
any marked depandence. except perhaps in repro­
outlying data)
ducibílity. The changes over the range of values of m,
if any al aH. are too small to be considerad sígnifícant. B.3.1 Background
Moreove-r, in view of the small ranga of values of m
and tlle natura of tha measurement, a dependence al $ource
on m is hardly 10 be expe~ed. It 58ems safe to con­
Standard methods 10r testing tar and its products;
elude that precislon does not depend on m in this
Creosote 011 5ection; Method Serial No. Co. 18
range, whích was stated as coverlng normal com­
(reference [5J in annex CJ.
mercial material, so that the means may be taken as
the final values tor repeatabílity and reproducibility b) Material
standard deviations.
Thís wss selected from cornmercial batches of
creosote oil collected and preparad as specífied in
B.2.8 Conclusions the "Sampies" chapter of the creosote oil sectíon
of reference [5].
For practical applicatíons, the preósion values- for the
measurement method can be considerad as inda­ e) De5eription
pendent of tM!> level 01 materíal, and are
This was a standard measurement method tor
repeatability standard dev¡atlon, Sr = 1,0 ·C chemical analysis lnvolving a thermornelríc
títration, with results expressed as a percentage
reproduciblllty standard deviation, Sil 1,8 oC by maS$, Nine ¡abaratarías partlcipated by rneas­
uring five specímens in duplicate, the specimens

36

:opyright by the IHTERHATIOMAL ORGAHIZATIO" FOR STAHDARDIZATIOH (ISO)


fue Jan 16 16:58:57 1996
• 4851"!)3 IJ.59l1SSD fl91 _
0,;1150 ISO 5725-2: 1994{E)

measured having been selected so as to cavar the 'P1e test resul1s for laboratory 1 were always higher,
normal ranga expected to be encountered in gen· and at sorne levers consicterably higher, than those 01
eral commercial applicatíon. These were chosen the other /aboratarías.
to líe at t1'1e approximate leve!s of 4, 8, 12, 16 and
20 ['Yo Cm/m)]. The usual practice would be to re­ The second test result for laboratory 6 at level 5 is
cord test results to only one decimal place, but for suspect; the vafue recordad would fít much battsr a1
this experiment operators were instructed to work level4.
to two decimal places,
These points are discussed further in B,3,5

8,3.2 Original data


8.3.3 Cell means
ThesE! are presentad in table B.12, as a percentage by
. These are given in table 8.13, as a percentage by
mass, in the format of form A of figure 2 (see 7.2.S).
mass, in the format 0110rm B of figure 2 (sea 72.9),

Table 8.12 - Original data: Thermometric titration of creosote oil


levelj
lllboratory i
J !
1
!
I
2 I 3 4 5
1 I
4.44 4,39 9,34 9,34 17,40 16,90 19,23 19,23 24,28 24,00
.2 4,03 4,23 8,42 8,33 14,42 14,50 16,06 16,22 20AO 19,91
3 3,70 3,70 7,60 7,40 13,60 13,60 14.50 15,10 19.30 19,70
4 4,10 4,10 8,93 8,SO 14,60 14,20 15,60 15,50 20.30 20,3Q
5 3,97 4,04 I 7,89 8,12 13,73 13,92 15,54 15,78 20.53 20,88
6 3.75 4,03 8.76 9,24 13,90 14,00
J, 16,42 16,58 18.56 16.58
7 3,70 3,80 B,OO 8,30 14.10 14,20 14,90 16,00 19,70 20,50
8 3,91 3,90 I 8.04 8,07 14,84 14,84 i 15,41 15,22 21,10 20.78
9 4,02 4,07 8,44 8,17 14,24 14,10 15,14 15,44 20,71 21,00

Tabla 8.13 - Ca" means; Tbermometric titration of creosote oíl


Level j
Laboratorv i I
1 2 3 I 4 5
1 4,416 9,340 17,150 u 19,230~~ 24,140*
2 4,130 8,375 14,460 16,140 20,155
3 3,700 7,500 13,600 14,800 19.500
4 4,100 8,B65 14,400 15,550 20.300
5 4,005 B,005 13,825 15,660 20,705
6 3,890 9,000 13,980 16,500 17,570
7 3,750 8,150 , 4,1 50 15,450 20,100
8 3,905 8,055 14,840 15,315 20,940
9 4,045 8,305 14,170 15,290 21,185
• Regarded as a 5traggler.
..... Regarded as a Slalistícal outlier

37

Copyright by the I"TER"ATIOHAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


Tue Jan 16 16:58:51 1996
.. 4~S1903 0594551 728 . .
ISO 5725-2:1994{E} ID ISO

'-.. 8.3.4 Absoluta dlfferences within cells At level 5, the absolute difference 1,98 gave a test
t
statistlc valve of 1.9S ¡6.166 3 =.: 0,636.
These are gíven In tabla B. 14, as Wjj' as a percentage
by mass, in the format of form e of figure 2 (sea For p 9, the critical values for Cochran's lest are
7.2.10). 0,638 for 5 %. and 0,754 for 1 %.

The value 1, 10 at level 4 is clearly a straggler, and the


8.3.5 Scrutiny for consistency and outliers value 1,98 at level 5 is so near the 5 % level as to be
a1so a possible straggler As these two values are sO'
Calculaban of Mandel's h and k consistency slatistics differen1 from all the others. and as their presence has
(sea 7.3.1) oave the values shown in figures 8.7 and inflated the divisor used in Cochran's test statistic,
B.8. Horizo~tal lines are shown corresponding to the they haya both been regarded as stragglers and
value of Manders indicators taken trom 8.3, marked with an asterisk. Tlle evidence agaínst them
so fer, however, cannot be regarded as sufficient for
The h graph (figure B.7) shows clearly that laboratory
1 obtained much higher test results than all otMar rejection. although Mandel's k plot (figure 8.8l al50
labaratarías a1 all levels. Such results require attention giV9S riS9 to suspicicn of these values.
on the part of the committee running the inlerlabora­ Applícalion of Grubbs' lests to the ceU means gives
tory study. If no explanatíons can be found for these the results shown in table B, 15.
test rasults. the members of the committee should
use their judgemsnt. based 01'1 additional and perhaps For levels 3 and 4, because the slOgle Grubbs test
non-statistical considerations, in declding whether 10 indica1es en outlier, the double Grubbs test is not ap­
inelude or exclude thís laboratory in the calculation of plied ises 7.3,1)
the precislon values.
The cell means for laboratar;, 1 in levels 3 and 4 are
The k graph (figure B.8l exhibits rather larga variability found to be outlíers. The cell mean for this laboratory
between replicate tes1 results for laboratories 6 for level 5 is al50 high. This is 31so clearly indicated
and 7. However, these test results do not seem so on Mandel's h plot (figure B.7).
severe as to require any speclal setlan beyond s
search for possible explanations and, íf necessary, On further enquiry, it was learned that at least one of
remedíal action for these test results. the samples for labaratory 6, level 5, might by mistake
have come from level 4. As the absolute difference
Appliéaüon of Cochran's test yields the following re­ for this celt was also suspect it was decided that thís
sults. pair of test results may also have to be rejected.
Wíthout the "help" 01 this paír of values, the test re­
At level 4, the absolute difference 1,10 gave a test
2 sult for laboratory 1 at level 5 is now definitely suspi­
statistic value of 1.10 /1,8149 = 0,667. cious.

Table B.14 - Cel! ranges: Therrnome1ríc titra1ion 01 ~reosote oH


Levelj
Laboratory i
1 2 I 3 4 5
1 0,05 0,00 0,60 0,00 0,28
...'" 0,20 0,09 o.aa 0,16 0,49
3 0,00 0,20 0,00 0,60 0,40
4 0,00 0,13 DAD 0,10 0.00
5 0,07 0.23 0.19 0,24 0.35
6 0,28 0,48 O.HI 0.16 1,98'
7 0,10 0.30 0.10 1.10" 0,80
8 0,01 0.03 0,00 0.19 0,32
9 I 0,05 0,27 0.14 0,30 0,95
lt
Regarded as a straggler.

38

:opyr;ght by the IHTERHATIOHAL ORGAHIZAtlOH FOR STAHDARDIZATIOH (ISO)


[ue Jan 16 16:58:57 1996
. . 4851903 059~552 664 ..
Q ISO ISO 5725-2:1994(E)

Becí:luse of these test results, it was dedded to reject Wíthout these test results, 1he Cochran's test statístíc
tha pair of test results from laboratary 6 for level 5 at level 4 was then compared with the critical value
because it was uncertaín what meteríal had been for 8 laborataries (O,680 at 5 %) :and this no longer
measured and to rejeet all the test resul1s from lab­ appeared as a straggler and was retained.
oratory 1 as coming from an outlying laboratory.

Table B.15 - Application ol Grubbs' test to cell mean~

Single Single Doubla Double


Level Type oftast
low high low high
1 1,36 1,95 0,502 0,356
:2 1,57 1,64 0,540 0,395
Grubbs' test
3 0,86 2,50 - - statístic5
4 0,91 2.47 - -
5 1,70 2,10 0,501 0,318
Stragglers 2,215 2,215 0,1492 0,1492 Grubbs' critica I
Outliers 2,387 2,387 0,0851 0,0851 values

.., 3

:i ---------
~ 2
E
.,
------
'" 1
'¡j

.
-o
c:
¿
O

- 1

2 _.___________________________________••____________________________________

3 7 8

Loboro.tory I

Figure B.1 - Titration of creosote oi!: Mandet's between·laboratory consistency statistic, h, grouped by
laboratoríes

39

Copyright by the IHTERHATIOHAL ORGAHIZATIOH FOR STAHDARDIZATIOH (ISO)


fue Jan 16 16:58:51 1996
.. 4!S1903 0594553 STO ..
ISO 5725-2:1994{E) o ISO

6~--------------------------------------------------------------,

- 1
:1 5 6 1 9

labora tory f

Figure B.8 THration of creosote oil: Mandel's within-laboratory consistency statistic, k, grouped by
laboratories

B.3.6 Computation of mí' Sr) and ~Rj Someone familiar with the requiremen1s lar a stan­
dard measurement method for creosote oil may be
The va!ues 01 ~, S¡j and Sflj computed without the test able to select the most suitable relationshíp.
results of laboratory 1 and the pair of test results from
laboratory 6, level 5, are given in table 8.16, as a per­
csntage by mas::;, calculated as in 7.4.4 and 7.4.5. B.3.8 Final values el precision

The final values, duly rounded, should be


B.3.7 Dependence of preeision on m
repeatability standard deviation, S, = O,019m
From table B_16. it seems dear that the standard de­
vlatlons teM 10 increase with higher values ot m. SO reproducibility standard devíatior.,
rt is likely that 1t might be permissible to establish sI( = 0,086 + O,030m or
some form of functional relationship. This view was
supported by a chemist familiar with the messure­ SR = O,078m ú .72
ment method. who was of the view that the precision
was likely te be dependent on the level.
B.3,9 Conclusions
The actual calculations for fitting a íunctíonal relation­
ship are not given hare as they have already been set There are no statistical reasons for preferring either
out in detail for Sr in 7.5.9. The valuas of Srj and Jil/) are one 01 the two equations for SR in 8.3.8. The panel
plotted against mi
in figure 8.9. should decide which one to use.

From figure 8.9 it ís evident that lhe valva for level 3 The reason for the outlying 1esr results of laboratory
is strongly divergant and could not be ImprQved by 1 should be investigated.
any alternatíveprocedures {see 7.5.2).
This 5eems to have been a rather unsatisfaclory pre­
For repeatability, a straight line through the origin cision experiment. One of the 9 laboratarías had to
seems adequate. be reiected as an outlier, and another laborator)' had
testad a wrong specimen. The material tor !evel 3
For reproducibility. al! three fines show adequate tit 5eems to have been wrongly selected, having almost
with the data, relationship 111 showing the best fit the same value as leve! 4 instead of Iying midway

40

:OI'~l'i!lJ.t h~ tl.e nUERNATIOMAL ORGAtlIZATION FOR STANDARDIZATlON (ISOl


rue Jan 16 16:58:51 1996
. . 4851903 0594554 437 . .
clSO ISO 5125-2:1994(E)

between levels 2 and 4. Moreover, the material for other material. 1t might be wor1hwhile to repeat this
leve I 3 seems to have been sQmewhat dífferent in experiment, taklng more cara over the selection of the
nature, perhaps beíng more homogeneous than 1he materíals for the different levels.

Table B.16 - Computad values 01 ~jl .':'j and sR} for thermometric titration of
creosote oi!

LevelJ P, m¡ sn '\'Ri

1 8 3,94 0.092 0.171


2 8 6,28 0.179 0,498
3 8 14,18 0.127 0,400
4 8 15,59 0,337 0,579
5 7 20,41 0,393 0,637

SR' 0.066· 0.030m

".
o~·t
SR,O,04m '''-..

0,6

0.5

0.4

0.3

0,2

0,1

a
o s 10 1S zo m

Figure 8.9 - Plot of sr) and Sil) Hgainst mJ o. the data from table 8.16, showing the functional relationships
fittad in 7.5 from these data

41

Copyrígllt by tJ¡e HlTEHNATlOHAL ORGANIZATIOI'I FOR STAI'IDAHDIZATION (1S0)


Tue Jan 16 16:58:57 1996
.. ~851903 0594555 373 . .

ISO 5725-2:1994{EI <O ISO

Annex e
{informatível

Bibliography

[1] ISO Guide 33:1989, Uses of cereitíed reference [8J ISO 3534-2: 1983, Statistícs Vocabular¡ and
materia/s. symbols - Pare 2: Statistícal quafity control.

[2] ISO Guide 35:1989, Certification of reference [9] ISO 3534-3:1985, Statistics - VocabulalY and
materia!s -- General and ststistical principIes. symbo/s - Pan 3: Design of experiments.

(3) ASTM E691-87, Scandard Practice for Conduct­ [10J ISO 5725-3:1994, Accuracy (trueness and pre­
ing an ¡nterlaborarory Study to Determine che cision) of measuremenf rnethods and resu/ts -
Precísíon of 8 Test Method. American Society Part 3: Intermedia te measures of the precision
for Testing and Materials, Philadelphia. PA, of a standard me8surement method.
USA.
[11] ISO 5725-4:1994, Accuracy (trueness and pre­
[4J GRUB6S, F.E. Qnd BECK. G. Extension of sample cisíon) of measurement methods and resulrs -
SiZS5 and percentage points tor sígnificance Part 4: Basle methods for the determination of
tests of outlying obseNations. Technometrics, the trueness of 8 standard measurement
14, 1972, pp. 847-854. method.

[5] "Standard Methods fer Testing Tar and its [12) ISO 5725-6:-1., AcCtlraey (trueness and pre-­
Products", 7th Ed. Stsndardisation ot Tar Prod­ ciSÍon) of measurement methods and results
ucts Tests Commi1tee, 1979. Pare 5: Alternative methods for the determj·
¡¡ation of the precisíon of a standard measure­
[6J TOMKINS, S.S. Industrial and Engineering Chem­ ment method.
istry (Analytical edition), 14, 1942, pp. 141-145.
[13J ISO 5725-6:1994, Accuracy (trueness and pre-­
[7] GRUBBS, F.E. Procedures for detecrlng oU11ying císíon) of measurement methods and results­
obS8Nations in sample:>. Technometrics, 11, Pare 6: Use in practice of accuracy values.
1969, pp. 1-21.

1) To be published.

42

;opyright by the I"TERNATIOMAL ORGANIZATIO" FOR 3TAMDARDIZATIOM (130)


[U" Jan lb 16:56:57 19%
.. 4851~03 OS9455b 20T . .
lO ISO

les 03.120.30
Descrlptors: measurem¡¡nt tests, tesl results, accuracy, reproduclbllir:y. statlsllcal analyslS.

Príce based 0/1 42

Copyright by the IHTERHATIOHAL ORGANIZArlON FOR STANDARDIZATION (180)


Tue Jan 16 16:513:57 1996

You might also like