NMKL Procedure No. 8, 4. Ed. (2008) : Ontents

Measurement of uncertainty in quantitative microbiological
NMKL PROCEDURE examination of foods

No. 8, 4. Ed. (2008) Page: 1 of 33
Version: 4
Date: September 2008
Approved: Ole Bjørn Jensen
MEASUREMENT OF UNCERTAINTY IN
QUANTITATIVE MICROBIOLOGICAL
EXAMINATION OF FOODS
CONTENTS
1. PREFACE
2. INTRODUCTION
3. DEFINITIONS
4. GENERAL CONSIDERATIONS
5. DESIGN FOR ESTIMATION OF MEASUREMENT UNCERTAINTY
6. ESTIMATION OF MEASUREMENT UNCERTAINTY
7. CONTROL OF THE ESTIMATED MEASUREMENT UNCERTAINTY
8. LITERATURE
ANNEX:
Comparison of estimations of measurement uncertainty based on different statistical models.
Nordic Committee on Food Analysis

No. 8, 4. Ed. (2008) Page: 2 of 33
Version: 4
1. PREFACE
The first NMKL Procedure on estimation of uncertainty for microbiological analyses was published already
in 1999. Professor Eystein Skjerve, at the Norwegian School of Veterinary Science was project leader and
arranged courses in the use of the procedure. Skjerve revised the procedure, and the 2. Ed. was published in
2002. In connection with quality assurance and accreditation of microbiological methods, according to EN
ISO /IEC 17025 (1), a calculated value for the uncertainty shall be given in connection with an analytical
result. In cases where it is not possible to estimate a value for the uncertainty, the laboratories shall be able
to identify sources to the uncertainty and provide a reasonable estimate of the proportion of the various con-
tributions.
One technique for estimating the uncertainty is to divide the method in question into its individual working
operations or steps. This can provide the analysts useful information about where the huge sources of error
are to be found and hence give better knowledge to reduce these, if necessary. A general procedure is de-
scribed in GUM (2). For microbiological methods this is described in Niemelä (4). This NMKL Procedure
does not describe estimation of measurement uncertainty for each individual step, however, estimates the
uncertainty based on a total/global assessment. In the previous editions of the NMKL Procedure No 8 the
measurement uncertainty is estimated from the extra variance in addition to the Poisson variance. The Pois-
son variance is the variance connected to the bacteria counting. The extra variance represents the variance in
connection with weighing, homogenisation and pipetting. This edition, which corresponds more with other
guidelines for measurement uncertainty [NMKL Procedure No 5, GUM, Norwegian Accreditation, Sector
Committee P9’s document, NA-S53, ISO / TS 19036] uses the standard deviation calculated from repeat-
ability and reproducibility data.
This 4th version of the NMKL Procedure replaces version 3, released in May 2008, which was only available
a few months. In version 3, the between-series variance (i.e., the variation between analysts, day-to-day and
batch to batch) was left out. Instead of issuing an addendum to Chapter 6, version 4 is released. In the esti-
mation of measurement uncertainty in NMKL Procedure No. 8, the method’s bias (uncertainty in relation to
the true value) it is not considered. It is informed, however, about the use of certified reference materials and
participation in proficiency testing schemes in order to check that the estimated measurement uncertainty is
satisfactory.
No. 8, 4. Ed. (2008) Page: 3 of 33
Version: 4
The revision of this procedure has been carried out by the following individuals:
Denmark: Erik Dahm, Danish Veterinary and Food Administration, Region North
Iceland: Snorre þórisson Rannsóknaþjónustan Sýni ehf/SYNY Laboratory service
Norway: Marianne Økland, National Veterinary Institute
Sweden: Lennart Larsson, LaVet
NMKL Secretary General: Hilde Skaar Norli, National Veterinary Institute (project leader)
A special thank is given to the Section of Feed and Food Microbiology at the National Veterinary Institute
for providing analytical data and to Joakim Engman, National Food Administration, Sweden for vital contri-
butions to this procedure.
The procedure is available from:

The office of NMKL Secretary General, c/o National Veterinary Institute,
P.Box 750 Sentrum,
N-0106 Oslo, Norway
E-mail: nmkl@vetinst.no, Web page: http://www.nmkl.org.
NMKL welcomes any comments and input on the procedure. Comments are requested to be forwarded to
the office of NMKL Secretary General.
©NMKL
No. 8, 4. Ed. (2008) Page: 4 of 33
Version: 4
2. INTRODUCTION
All quantitative determinations are burdened with measurement uncertainty. The term measurement uncer-
tainty must not be mixed up with error. Error is the difference between measured and true value, while the
measurement uncertainty is an expression for the spread, and is a quantitative expression for the quality of
an analytical result.
The “true” number of microorganisms in a sample is not usually known. Microorganisms can be changed,
multiply or die out in a sample or diluent. In qualitative and quantitative examinations of foods for the oc-
currence of specific microorganisms or groups of bacteria (“heterotrophic bacteria”, “coliform bacteria”),
there are several factors contributing to spread of the results, and to the uncertainty of the obtained analytical
results. These factors must be borne in mind when considering the results. It is important that the measure-
ment uncertainty stated is realistic and that the confidence intervals of the precision are not made too nar-
row.
Lately several certified reference materials are made available at the market, where the levels of microorgan-
isms are given. Certified reference materials and materials used in proficiency testing schemes / ring trials
are useful for estimating the measurement uncertainty of the laboratory. Further participation in proficiency
testing and use of control charts are important to control that the obtained measurement uncertainty of the
laboratory is relevant and satisfactory.
3. DEFINITIONS
The definitions are collected from NMKL Procedure No 5 (3), NMKL Procedure No 4 (6), NMKL Proce-
dure No 18 (5) and VIM (7).
Control charts
A control chart is a diagram where the results of analyses from known control samples are continuously
filled in. The control chart is connected to a certain method and matrix (5).
Error (of measurement)

Measured value minus a reference value (7).
No. 8, 4. Ed. (2008) Page: 5 of 33
Version: 4
Measurement uncertainty
Parameter, which is associated with the result of a measurement, and which characterizes the
dispersion of the values that could reasonably be attributed to the measurand (3).
Notes
1. The parameter can, for instance, be a standard deviation (or a given multiple thereof), or half-width of
an interval having a stated level of confidence.
2. Measurement uncertainty generally includes many components, which are characterized by their stan-
dard deviations estimated from a series of measurements, or from assumed probability distributions
based on experience or other information.
3. It is presumed that the measurement result is the best estimate of the value of the measurand, and that
all elements of uncertainty, including those, which depend on systematic effects (e.g. those which are
connected with corrections and reference standards), contribute to the dispersion.
Repeatability
Repeatability means that the analysis results are obtained using the analytical method on identical samples in
the same laboratory, using the same equipment within a short period of time (6).
Reproducibility
Reproducibility means that the results are obtained by using the analytical method on identical samples in
different laboratories and using different equipment (6).
Internal reproducibility means that the determination is carried out at different times, by different persons
and on different batches of reagents – but in the same laboratory (6).
Precision
Precision is the degree of agreement between independent analysis results obtained under specific circum-
stances (6).
Reference materials
Reference materials contain a specified or agreed amount of certain microorganisms or a mixture of micro-
organisms (5).
No. 8, 4. Ed. (2008) Page: 6 of 33
Version: 4
Certified reference material (CRM)

Reference material, accompanied by documentation issued by an authoritative body and providing one or
more specified property values with associated uncertainties and traceability, using valid procedures (7).
Standard deviation (SD)

The standard deviation is a measure for the spread around the mean in a collection of several results.
Comments: Expressed mathematically the standard deviation is the square root of the variance.
1 n
SD = ∑
n − 1 i =1
(x i − x ) 2
SD = standard deviation, xi = the values of the single results, x = the mean of the results, n = number of
measures.
Standard uncertainty, u(xi)

The uncertainty of a measurement result expressed as a standard deviation (3).
Combined standard uncertainty, u

Standard uncertainty of the result of a measurement when that result is obtained from the values of a number
of other quantities, equal to the positive square root of a sum of terms, the terms being variances or covari-
ances of these other quantities, weighed according to how the measurement result varies with these quantities
(3).
Expanded uncertainty, U
The combined standard uncertainty, uc, multiplied with a coverage factor, k. At a 95 % confidence level
(probability) k ≈ 2, at 99% confidence (probability) k ≈ 3.
U=k·u.
No. 8, 4. Ed. (2008) Page: 7 of 33
Version: 4
4. GENERAL CONSIDERATIONS
4.1 Traceability
To be able to use results from different laboratories in a sensible manner, the results must be comparable.
When it comes to e.g. weighing, the balance must be calibrated against a traceably calibrated set of weights.
This set of weights has been calibrated against a national set of weights, which in turn is calibrated against
the international kilogram prototype. The optimal situation would be to make all measurements that trace-
able, by using calibrated equipment, certified reference material with suitable matrix and a collaborative
validated method.
4.2 Reference materials, proficiency testing

One of the main problems faced by microbiologists in dealing with internal method validation is the lack of
relevant reference materials. Fortunately, more reference materials have been produced lately and there are a
number of freeze-dried materials and matrix related samples, often in connection with proficiency testing /
ring trials. In practice, the real food samples are dealt with rather than freeze-dried materials. Analysing ma-
terial from a freeze-dried ampoule necessarily differs from analysis of a food sample involving a series of
additional steps such as sampling, primary dilution etc. Nevertheless, participation in proficiency tests are
vital in evaluation of the analytical competence of the laboratory at field of interest and to review if the es-
timated measurement uncertainty is adequate. The use of reference materials is described in NMKL Proce-
dure No 18 and hence will not be given much attention in this procedure.
4.3 Official analytical methods

Lately several of the official microbiological methods are validated collaboratively, i.e. they are tested out at
several laboratories on different matrices at different levels of the agent. The requests to the method organi-
sations for including validation data in the methods are increasing. Validation data from a collaborative
study of the method’s performance can be used for estimating the measurement uncertainty at any labora-
tory.
No. 8, 4. Ed. (2008) Page: 8 of 33
Version: 4
4.4 Large variations between results

Most people working with microbiological analysis have learned that variations between results from analy-
ses of the same sample may be considerable. This large variation may be easily exemplified using plating
methods. Results may be obtained that most analysts would consider as improbable without any statistical
calculation, for example: widely deviating parallels, substantial deviation between dilutions, or surprising
combinations of results from a MPN analysis. The causes of these phenomena are often not apparent. It is
crucial for a laboratory to establish whether such variations are due to inaccuracies in laboratory procedures,
or whether such phenomena are simply to be expected when dealing with microorganisms.
When using a plating method, e.g. 88 and 68 colonies counted on two plates from a 10-3 - dilution, most ana-
lysts will view the agreement between the results to be satisfactory, and use them to estimate the «true»
sample level to be 78000/g (7.8·104/g). If repeated, the analysis is likely to yield a different result. If re-
peated on numerous occasions, the analysis will produce a certain dispersion of results. The characteristics
of this dispersion indicate to what extent any one-measurement result may be assumed to be close to the
«true» value. If the shape of the dispersion is unknown, we cannot estimate the uncertainty of single results.
Strictly speaking, the «true» value can only be determined by examining the entire lot, from which the sam-
ples were taken.
The variation observed if analyses are repeated may be due to a number of reasons. Firstly, a stochastic dis-
tribution will always exist with respect to the exact number of microorganisms present in the sample re-
ceived by a laboratory. Even when the sampling method is satisfactory, exactly similar values will never be
attained. The biological nature of the microorganisms is a further contributing factor to the variation. Fi-
nally, in carrying out an analysis, uncertainty and variation are being introduced through inaccuracies in the
method.
4.5 Microbial distribution in sample

Ideally, each sample should be selected in such a way that it represents the entire product or lot to be tested.
This may prove to be quite difficult when faced with large lots and solid foods. Liquid foods such as water
and milk are products involving small particle sizes, and as such, the only products from which samples are
likely to be representative for the whole lot or product. Solid foods like meat, cheese etc. may only partly be
relied upon to provide representative samples. Sampling should be optimised to ensure that the final sample
is suitable for quantification of the real content of microorganisms of the product or lot concerned.
No. 8, 4. Ed. (2008) Page: 9 of 33
Version: 4
The microbial load in a single sample cannot always be expected to be evenly distributed. Microorganisms
may be more numerous on the surface than inside the sample, and different strains of microorganisms may
be active in different areas of the sample.
4.6 Homogenisation
In order to obtain an even distribution of a food or matrix, the sample is usually homogenised. This applies
to both liquid and solid samples. The microorganisms in the final sample are more evenly distributed by
homogenising in a stomacher or other instrument. One should be aware, however, of the differences in effi-
ciency of such homogenisation procedures. Heterogeneous foods, those containing fatty particles or other
particles/substances, may also have heterogeneously distributed microorganisms even after homogenisation.
Microorganisms must be considered as "particles" and will, due to spherical effects and electrical charges
often bind to other particles in the sample. Standardising the homogenisation procedure may also prove dif-
ficult, subjective judgement often being used when considering what constitutes an acceptable level of ho-
mogenisation. Each laboratory should have established procedures for the homogenisation of each type of
food. For laboratories running large series on the same kinds of foods, these procedures may prove crucial
for the consistency between analytical results.
4.7 Single microbes or agglomerations?

Various microorganisms may appear as single particles or agglomerations (clusters) of particles. Such ag-
glomerations are naturally formed by proliferation of microbes in foods. Microbiological analysis takes this
into account, results often being expressed as Colony Forming Units (CFU) and not in numbers of «mi-
crobes». A Colony Forming Unit can be an agglomeration of bacteria stemming from one or more microbes
originally transmitted, and which have subsequently divided to produce a large number of microorganisms.
A detailed description of the evaluation of colonies is given in NMKL Report No 5 (9).
When sampling, it must be assumed that these agglomerations were originally present in the sample and that
any growth has occurred from these agglomerations. In the further preparation of the sample, some of these
agglomerations are assumed to break up, with the microorganisms or the reduced agglomerations behaving
rather as single particles in a liquid phase. This assumption is the basis for the majority of theories on quan-
tification of microorganisms.
No. 8, 4. Ed. (2008) Page: 10 of 33
Version: 4
In some foods, growth is believed to occur on a limited scale only, with microorganisms more often being
actually found as single particles. Drinking water with a low content of particles and nutrient salts may serve
as a representative example.
Various microorganisms exhibit wide variations of behaviour when in a liquid phase. Some come close to
the single particle ideal, whereas others typically agglomerate, especially those producing capsules or mu-
cous substances, for example Pseudomonas spp. It is possible to minimise their capability for agglomerating
by adding surface-active compounds into dilutions and media.
4.8 Microbial growth and death

Even when sampling takes place under controlled conditions and optimal homogenisation is ensured, the
problem of microbial growth remains. Many of the relevant organisms are able to proliferate in the product,
in the sample following sampling, and to a certain extent throughout the whole procedure until final inocula-
tion takes place.
4.9 The analysts - Procedural differences

Differences in the way procedures are being followed may contribute to considerable uncertainty in the final
result. Despite these factors being under good control, some inconsistencies will occur during weighing, pi-
petting etc. The mechanical handling of a sample may also contribute to a gradual change in sample charac-
teristics. Homogenisation and the microbes' ability to agglomerate have been mentioned previously. Fol-
lowing homogenisation, various dilution steps are often used, depending on the parameters to be analysed.
Each step may be instrumental in breaking up existing microbial agglomerations, resulting in an increase of
CFU numbers. Even a process as simple as pipetting may be highly effective in breaking up bacterial ag-
glomerations due to the strong suction exerted by narrow pipette-tips. In addition, mechanical handling may
also injure to the microbes, resulting in reduced vitality, and substances present in the diluent may also re-
duce microbial viability. Inexperienced analytists may place too much faith in the standardisation of results
from microbiological analyses. It is essential to have established a realistic acceptable limit for variations in
results. With reference to quality assurance of methods, small inaccuracies in weighing/pipetting have rela-
tively little significance compared to the potentially large effects of physical treatment of microbe aggre-
gates or temperature deviations, which can allow growth or death. Weighing/pipetting inaccuracy may give
rise to percent error, but inaccuracy in the sample storage and/or incubation temperature or variations in the
routines for dilution can give rise to log10-unit errors.
No. 8, 4. Ed. (2008) Page: 11 of 33
Version: 4
4.10 Reliability of colony counting

Subjective evaluations will often play a role when counting colonies on an agar plate. Methods to evaluate
the correspondence between an analyst’s repeated counting of a plate and two analysts’ counting of the same
plate should be employed regularly in order to tighten up the criteria for estimating single colonies. A simple
method is published in an earlier NMKL document (9), where it is recommended that the relative variation
for one analyst should not exceed 7.7 % and for two analysts, 18.2%. Error location and optimising of meth-
ods should also include testing of the analysts’ ability to reproduce counts.
5. DESIGN FOR ESTIMATION OF MEASUREMENT UNCERTAINTY
For establishing an estimate of the measurement uncertainty of an analysis at a laboratory, the laboratory’s
internal reproducibility should be used. Internal reproducibility means that a standard deviation is calculated
from results obtained by repeated analyses at the same laboratory at different times, by different persons and
on different batches of reagents. The internal reproducibility of the laboratory will be the standard uncer-
tainty. If, for some reason, a laboratory should not have the capacity to make an internal reproducibility
study, an estimate of the measurement uncertainty may be established based on results from the method’s
performance study. This presupposes that the method is validated collaboratively, and also that the labora-
tory verifies the method’s performance with respect to the repeatability. The following chapters contain ex-
amples of the mentioned possibilities for estimating measurement uncertainty based on
1. the standard deviation for the internal reproducibility from experiments at own laboratory
2. the standard deviation for the reproducibility obtained from a collaborative method performance
study
Alternative 1 is recommended.
In collaborative validation studies within microbiology, robust statistics are usually applied when evaluating
the results. This means that the median is used instead of the mean, and the so called recursive median, Sn of
Rousseeuw, is used instead of the classical standard deviation (13). The median is commonly used in order
to give the so called ”random shots”, deviating results which appear now and then, less influence on the total
result. Robust statistics can be also used to estimate the measurement uncertainty. As most analysts have
more experience with the use of classical statistics, the mean and standard deviation are described in this
procedure. This can be justified as many results are used when establishing measurement uncertainty, and
therefore random shots are of less consequence.
No. 8, 4. Ed. (2008) Page: 12 of 33
Version: 4
Within chemistry (NMKL Procedure No. 5), the relative standard deviation is usually applied for estimating
the measurement uncertainty. The relative standard deviation means standard deviation (the spread of the
results) divided by the concentration/level. Within microbiology, use of relative standard deviation is not
relevant as the relation between level and standard deviation (calculated on log10 values) is approximately
constant.
The following figures show the relation between the concentration levels and standard deviations of results
from the analysis of 681 parallels (1362 analyses) of different microbes, levels and matrixes estimated in the
following 3 ways:
a) non-logged results, where the standard deviation is calculated from cfu/g
b) log10 -transformed results, where the standard deviation is log10-transformed after the calculations
are carried out
c) log10 transformed results, where the standard deviation is calculated from the log10- values of the re-
sults
a) Relation between level and standard deviation in cfu/g

(i.e. non-logged data)
20000 R2 = 0,5638
15000
standard deviation in cfu/g
10000
5000
0
0 5000 10000 15000 20000 25000 30000 35000
-5000
level in cfu/g
No. 8, 4. Ed. (2008) Page: 13 of 33
Version: 4
b) Relation between level and standard deviation when the

results are log 10 transformed after the calculations
5
standard deviation (log 10 cfu/g)
4,5
4 R2 = 0,8051
3,5
3
2,5
2
1,5
1
0,5
0
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5
level (log 10 cfu/g)
c) Relation between level/concentration and standard deviation

(in log 10 cfu/g)
1,4
standard deviation (log 10 cfu/g)
1,2
0,8
0,6
0,4
0,2
0 R2 = 0,0449
0,00 0,50 1,00 1,50 2,00 2,50 3,00 3,50 4,00 4,50 5,00
level (log 10 cfu/g)
For Figure a) (original, non-logged data) there is an increasing spread of the standard deviation with increas-
ing levels. A trend line is included in the figure. However, the linear relation has to be rejected as the regres-
sion coefficient is low (R2 = 0.56). If the results are logged, after the calculations are carried out there is an
obvious linear relation between increasing level and standard deviation (figure b).
Quite often, standard deviations are calculated based on results in log10 cfu/g – the data are logged prior to
the calculations. Figure c), which illustrates results where the calculations are carried out on log10 values,
No. 8, 4. Ed. (2008) Page: 14 of 33
Version: 4
there is no increase in the standard deviation by increasing level, the relation is almost constant. This illus-
trates that it is important to know whether or not the results are logged-transformed prior to or after the cal-
culations. For the results from the analysis of the 681 parallels, 98% of the results are below 0.5 log10 cfu/g,
96% of the results are below 0.4 log10 cfu/g and 94% of the results are below 0.35 log10 cfu/g. Based on
these results (several 100s of parallels), the standard deviation between two parallels (the repeatability)
should be less than 0.4 log10 cfu/g (at 95% confidence). This can be used as a rule of thumb. However, the
standard deviation depends on the analysis (method, bacteria, materials), and, not least, on the analyst’s
technical skills.
No. 8, 4. Ed. (2008) Page: 15 of 33
Version: 4
6. ESTIMATION OF MEASUREMENT UNCERTAINTY
The measurement uncertainty is estimated from the standard deviation of internal reproducibility. For calcu-
lation of this standard deviation, analytical results obtained at reproducible conditions are used (see chapter
3 for definitions). The calculations are conducted on log10 transformed results. The results can be put in a
suitable table such as shown below:
Combinations/Series A;1;1 B;2;2 C;3;3

Analysts;
Batch;
Day
Replicate log10 log10 log10

1 xa11-1 xb22-1 xc33-1
2 xa11-2 xb22-2 xc33-2
3 xa11-3 xb22-3 xc33-3
4 xa114 xb22-4 xc33-4
9 xa11-9 xb22-9 xc33-9

10 xa11-10 xb22-10 xc33-10
In the following, it is described step-by-step how to calculate internal reproducibility (precision within and
between series), and thereby an estimate for the measurement uncertainty.
Step 1: Calculate the precision (the repeatability) within the series / combinations
In order to get a measure of the precision within the series (A, B and C) each of the obtained preci-
sion of the series are combined. This is done by calculating the mean (average) and the standard de-
viation for each combination of analyst / batch / day (A, B and C), and then combining the obtained
standard deviations.
The mean is the sum of the results divided on the number of replicates.
No. 8, 4. Ed. (2008) Page: 16 of 33
Version: 4
∑x i
x11−1 + x11− 2 + x11−3 ... + x11− 10
Mean, x = i =1
=
n 10
n = number of replicates (10 in this example)
n
2
∑ ( x 11 − 1 − x )
Standard deviation, s, is calculated for each combination: s = i =1
n −1
It is easy to do this in Excel. The results can be presented in a table:
Combination/Series A;1;1 B;2;2 C;3;3

Number of replicates n1 (10) n2 (10) n3 (10)
Mean, x x1 x2 x3
Standard deviation, s s1 s2 s3
The combined standard deviation of the precisions (the repeatability), Sr, to the series is calcu-
lated as follows:
( n1 − 1) s12 + (n2 − 1) s 22 + (n3 − 1) s32

Sr =
n1 + n2 + n3 − 3
If n1 = n2 = n3 (= 10 in this example), the formula becomes
s12 + s 22 + s32
Sr =
3
( "3" because it is 3 series (A, B and C), if it were 6 series / combinations the denominator should
have been 6.)
Step 2: Calculate the precision between the series /the combinations

To examine whether there is a major difference in the results between the series A, B and C, calcu-
late the mean of the series’ mean values and the standard deviation thereof. Then use the obtained
standard deviation to calculate the between-series variance.
x1 + x 2 + x3
The mean of a series mean, y =
3
n
2
∑ ( xi − y )
i =1
The standard deviation of the mean of the series’ mean, Sx = 3
No. 8, 4. Ed. (2008) Page: 17 of 33
Version: 4
The between-series variance is a measure of the spread of the results between the series, when the
repeatability of the series has been adjusted for.
S r2
S L2 = S x2 − (Sr is calculated in step 1, n = number of replicates (the same as in step 1))
n
Step 3: Calculate the standard deviation of the internal reproducible, SR = standard uncertainty, u,
The standard uncertainty, u, is the standard deviation of the internal reproducibility, SR, and is calcu-
lated by combining the standard deviation of the precision for each of the series (A, B and C) (step
1) and standard deviation between the series, which represents the day-to-day, batch-to-batch and
analyst-to-analyst variations.
u = SR = S r2 + S L2
Step 4: Estimate the measurement uncertainty, U.

In order to make the measurement uncertainty correspond to an interval containing a large
fraction of the expected variation in the results, the combined standard uncertainty is multi-
plied with a coverage factor. The coverage factor is usually 2, but can sometimes be 3. A
coverage factor of 2 corresponds to a confidence level of approx. 95%, and a coverage fac-
tor of 3 corresponds to a confidence level above 99%. It is recommended that a coverage
factor of 2 is used in the following manner: U = 2 · u
U, is half of the measurement uncertainty interval. In order to express the whole measurement uncertainty
interval for a measurement result, y (in log10 units), the following formula is usually used:
• y ± U,
• y log10 [y - U, y + U] or
• y cfu /g or ml [10y (log10)-U, 10 y(log10)+U] NB! not 10y ± 10U
No. 8, 4. Ed. (2008) Page: 18 of 33
Version: 4
6.1 Estimation of measurement uncertainty from internal reproducibility from

analytical resuls of one matrix, one level and several analysts
For estimation of measurement uncertainty from internal reproducibility, each of the current analysts (the
ones that are or will be performing the analysis) analyse 10 replicates (10 plates) for each microorganisms in
question (or group of microorganisms) for each matrix group of interest at different times. If only one ana-
lyst performs the analysis routinely, the analyst has to analyse a set of 10 replicates (10 plates of the same
culture medium) and then repeat the experiment on a later a date.
A way of estimating the measurement uncertainty is to use a sample, preferably a reference material of real
matrix, which are repeatedly analysed by different assigned analysts at the laboratory. The analysts perform
analyses on different days, in order to also include the day-to-day variation and batch to batch variation in
the estimate of the measurement uncertainty.
Example 1
For establishing a measurement uncertainty for determination of aerobic count (NMKL Method No. 184),
the laboratory has used a material from a proficiency test (chocolate pudding mix). Five analysts each ana-
lysed 10 replicates on different days. The results are given in table 1.
Table 1: Results from analysis of aerobic count in chocolate pudding mix, 5 analysts, 10 parallels on dif-
ferent days
Series:
Analysts,
Batch,
Day I II III IV V
Replicates log10 cfu/g
1 3.67 3.79 3.61 3.78 3.80
2 3.66 3.76 3.72 3.84 3.77
3 3.72 3.85 3.59 3.73 3.74
4 3.85 3.86 3.68 3.49 3.00
5 3.70 3.90 3.54 3.57 3.96
6 4.02 3.60 3.90 3.51 4.05
7 3.87 3.86 3.96 3.77 3.81
8 3.90 3.89 3.87 3.71 4.08
9 3.74 3.82 4.01 3.80 3.93
10 3.45 3.86 3.86 3.70 3.71
No. 8, 4. Ed. (2008) Page: 19 of 33
Version: 4
Follow the steps 1 - 4 to establish an estimate for the measurement uncertainty.
Step 1: The precision within in the series

Calculate the mean and the standard deviation of each series. Then combine the obtained standard
deviations.
For the series 1, the mean and the standard deviation is calculated from the results in column 2 in
Table 1, as follows:
n
∑x i
3.67 + 3.66 + 3.72 + ... + 3.45
The mean of the series I: x = i =1
= = 3.76 (log10 cfu/g)
n 10
The standard deviation of the series I:

n
( 3 . 67 − 3 . 76 ) 2 + ( 3 . 66 − 3 . 76 ) 2 + .... + ( 3 . 45 − 3 . 76 ) 2
2
∑ ( xi − x )
sr = i =1
n −1
= = 0 . 16
9
(log10 cfu/g)
Calculate the mean and the standard deviation in the same way for the other series in Table 1.
The mean values and the standard deviations of the results in Table 1, is given in Table 2.
Table 2: The mean and the standard deviation of the series I -V from Table 1.
Series: I II III IV V
X (log10 cfu/g) 3.76 3.82 3.78 3.69 3.79
sr (log10 cfu/g) 0.16 0.09 0.17 0.12 0.30
The Series V has relatively considerable spread compared to the other series (sr is about twice the
value as for the others). The example is authentic and the analyst running series V is under training
and not authorised to perform the analysis. As the person is not allowed to do contracting analysis,
Series V is excluded in the estimation of the measurement uncertainty.
The combined standard deviation of the precision (the repeatability) Sr, of the series is:
Sr12 + Sr22 + Sr32 + Sr42 0.162 + 0.092 + 0.172 + 0.122

Sr = = = 0.14 (log10 cfu/g)
4 4
Step 2: The precision between the series

To get a measure of the spread of the results between the series (the analysts, the days and the
batches) calculate the mean of the series means and its standard deviation. Thus, from this standard
No. 8, 4. Ed. (2008) Page: 20 of 33
Version: 4
deviation calculate the between-series variance.
The mean of the series’ I, II, II and IV mean values, y:

3.76 + 3.82 + 3.78 + 3.69
y= = 3.76 (log10 cfu/g)
4
The standard deviation of y, Sx:

4
∑ (x
i =1
i − y) 2
(3.76 − 3.76) 2 + (3.82 − 3.76) 2 + (3.78 − 3.76) 2 + (3.69 − 3.76) 2
Sx = =
n −1 3
= 0.054 (log10 cfu/g)
The standard deviation of the mean of the series mean values (Sx = 0.054 log10 cfu/g), is a measure
for the spread of the series’ mean values.
The between-series variance, SL2:

S r2 0.14 2
S L2 = S x2 − = 0.054 2 − = 0.00096 (log10 cfu/g)
n 10
The spread of the results between the series is small, the between-series variation, SL, is low and ne-
glected when the standard uncertainty is calculated as the standard deviation of the internal repro-
ducibility.
Step 3: Calculate the standard deviation of the internal reproducible, SR = standard uncertainty, u
Internal reproducibility, SR, is the sum of the variance of the repeatability (from step 1) and variance
of the between-series (from step 2) is as follows:
u = S R = S r2 + S L2 = 0.142 + 0.00096 = 0.14 (log10 cfu/g)
Step 4: Estimate the measurement uncertainty, U

If the results are to be given as a 95% confidence interval, the combined uncertainty is multiplied by
a coverage factor of 2, k = 2. Thus, the expanded uncertainty is:
U = 2 · u = 2 · (± 0.14 log10 cfu/g) = ± 0.28 log10 cfu/g
No. 8, 4. Ed. (2008) Page: 21 of 33
Version: 4
Excel spreadsheet is indeed suitable to make the above calculations and in providing the results in a dia-
gram. The diagram below shows the results in a coordinator system, where the x-axis is the number of paral-
lels and the Y-axis is the results in log10 cfu/g. The bold lines at the top and at the bottom show the expanded
uncertainty, U ± 0.28 log10 cfu/g. The line in the middle shows the mean of all the results.
Aerobic bacteria
4,20
4,10
4,00
3,90
log 10 cfu/g
3,80
3,70
3,60
3,50
3,40
3,30
1 2 3 4 5 6 7 8 9 10
number of parallels
Expression of the result:

Example of expression of result: If the obtained analytical result is 6900 cfu/g = 3.84 log10 cfu/g for aerobic
count, the result should be given in one of the following ways:
• 3.84 log10 cfu/g ± 0,28 log10 cfu/g,
• 3.84 log10 cfu/g [3.56 – 4.12] or
• 6900 cfu/g [103.56, 104.12] = 6900 cfu/g [3600, 13200] (NB! not 103.84 ± 100.28 = 6900 ± 2)
• 3.6 ·103 – 1.3·104 cfu/g
6.2 Estimation of measurement uncertainty based on internal reproducibility from

different matrixes and levels at the same time.
For the sake of convenience and resources, often one estimate for the measurement uncertainty is used for
different matrixes at different levels. The prerequisite is that the method is applicable for the matrixes and
levels of interest. In the following example it is shown how to establish one value for the measurement un-
certainty of for the determination of Staphylococcus aureus in different matrixes and levels.
Example 2: The laboratory uses NMKL Method No. 66, 4. Ed., 2003: ”Staphylococcus aureus. Determina-
tion in foods.” Two analysts are trained in the method. The matrixes analysed are cheese, lettuce and ham.
No. 8, 4. Ed. (2008) Page: 22 of 33
Version: 4
The content of S. aureus in the samples is enumerated by surface streaking of decimal dilutions of a speci-
fied quantity of the sample on the Baird-Parker with Rabbit plasma fibrinogen agar (BP+RPF). After incu-
bation colonies with typical and atypical appearance are counted.
Both analysts analyse 10 parallels with samples of cheese, lettuce and ham, respectively. The materials are
homogenised. The results are given in table 3.
Table 3. Two analysts’ results from analyses of 10 parallels of cheese, lettuce and ham.
Series /
Analyst 1 2
Matrix Cheese Lettuce Ham Cheese Lettuce Ham
Sample cfu/g Log10 cfu/g Log10 cfu/g Log10 cfu/g Log10 Cfu/g Log10 cfu/g Log10
1 6500 3.81 3500 3.54 7200 3.86 5800 3.76 3400 3.53 6200 3.79
2 6000 3.78 4000 3.60 7600 3.88 5400 3.73 2800 3.45 6500 3.81
3 5400 3.73 3000 3.48 7100 3.85 6200 3.79 3500 3.54 5900 3.77
4 5000 3.70 2800 3.45 6800 3.83 5500 3.74 2700 3.43 6100 3.79
5 5800 3.76 3600 3.56 7900 3.90 6200 3.79 2600 3.41 6300 3.80
6 6400 3.81 3800 3.58 7100 3.85 5300 3.72 3200 3.51 6000 3.78
7 6200 3.79 2700 3.43 7700 3.89 5100 3.71 3500 3.54 5800 3.76
8 6800 3.83 4300 3.63 7500 3.88 5900 3.77 3900 3.59 6400 3.81
9 6800 3.83 4500 3.65 7800 3.89 4900 3.69 3100 3.49 6300 3.80
10 5600 3.75 3400 3.53 6900 3.84 5700 3.76 3300 3.52 6400 3.81
Follow the steps 1 - 4 to establish an estimate for the measurement uncertainty.
Step 1: The precision within in the series

Calculate the mean and the standard deviation of each series. Then combine the obtained standard
deviations.
For Series I, Cheese, the mean and the standard deviation are calculated as follows:
n
∑x
i =1
i
3.81 + 3.78 + 3.73 + ... + 3.75
Mean, x: x = = = 3.78 i.e. (log10) cfu/g
n 10
Standard deviation:
n
( 3 . 81 − 3 . 78 ) 2 + ( 3 . 78 − 3 . 78 ) 2 + .... + ( 3 . 75 − 3 . 78 ) 2
2
∑ ( xi − x )
sr = i =1
n −1
= = 0 . 044 i.e. log 10 cfu/g
9
Corresponding calculations are carried out on log10-results of the other matrices for series I and II.
The results are given in table 4.
No. 8, 4. Ed. (2008) Page: 23 of 33
Version: 4
Table 4: The mean and the standard deviation of the results in table 3
Series/ I II
Analyst
Matrix Cheese Lettuce Ham Cheese Lettuce Ham
Mean
log10 cfu/g 3.78 3.54 3.87 3.75 3.50 3.79
sr, log10 cfu/g 0.044 0.076 0.023 0.034 0.056 0.016
For estimating the standard deviation of the repeatability for all series, Sr, combine the sr of each ma-
trix and series (given in table 4) as follows:
∑ sr
i =1
i
2
0.044 2 + 0.076 2 + 0.023 2 + 0.034 2 + 0.056 2 + 0.016 2

Sr = = = 0.042 (log10 cfu/g)
i 6
Step 2: The precision between the series

To get a measure of the spread of the results between the series (the analysts, the days and the
batches) calculate the mean value of the series means and its standard deviation for each type of ma-
trix (from table 4) as given in table 5.
Table 5: The mean and standard deviation between the series:
Matrix Cheese Lettuce Ham
x1 + x 2
Mean, (3.78+3.75)/2 = 3.76 (3.50 + 3.52)/2 (3.87+3.79)/2
2 = 3.52 = 3.83
(log10 cfu/g)
Standard deviation,
n
∑ (x i − x )
2 (3.78− 3.76)2 + (3.75− 3.76)2
= 0.023 0.031 0.053
s= i =1
n −1 1
(log10 cfu/g)
Then combine the standard deviations of the series means in order to get a measure for the spread of
their mean value:
n
∑s
i =1
2
i
0.023 2 + 0.0312 + 0.053 2
Sx = = = 0.035 (log10 cfu/g)
i 3
Then calculate the between-series variance, SL2, in order to get a measure for the spread of the re-
sults between the series:
No. 8, 4. Ed. (2008) Page: 24 of 33
Version: 4
S r2 0.042 2
S L = S x2 − = 0.035 2 − = 0.033 (log10 cfu/g)
10 10
Step 3: Calculate the standard deviation of the internal reproducible, SR = standard uncertainty, u
Internal reproducibility, SR, is the sum of the variance of the repeatability (from step 1) and variance
of the between-series (from step 2) is as follows:
u = S R = S r2 + S L2 = 0.042 2 + 0.033 2 = 0.054 (log10 cfu/g)
Step 4: Estimate the measurement uncertainty, U

The standard uncertainty is 0.054 log10 cfu/g , and the expanded uncertainty, U:
U = 2 · u = 2 · (± 0.054log10 cfu/g) = ± 0.11log10 cfu/g
Expression of the result:

If the obtained result is 3.74 log10 cfu/g, the result can be expressed either as:
• 3.74 log10 cfu/g ± 0.11 log10 cfu/g
• 3.74 log10 cfu/g [3.63 – 3.85]
If it is preferable to express the result in cfu/g in stead of log10 cfu/g, the result is transformed, 103.74 cfu/g =
5495 cfu/g, which rounded to nearest 100 would be 5500 = 5.5 · 103 cfu/g. Then the result can be expressed
as:
♦ 5500 cfu/g [103.74 – 0.11, 103.74 + 0.11] = 5500 cfu/g [4300, 7100]
♦ 4.3·103 – 7.1·103 cfu/g.
6.3 Estimation of measurement uncertainty based on results from collaborative

method validation.
Several official methods are collaboratively validated, i.e. the methods are tested out on several laboratories
at the same time to check the method’s performance. The participating laboratories follow carefully the
description of the specified method when analysing the forwarded samples. The samples are often pre-
sented as blind duplicates of different matrixes at different levels, i.e. two and two samples are alike, but
that do not the participants know. How to use results from a collaborative validation of NMKL Method
No 136 for estimation of the measurement uncertainty for Listeria monocytogenes in foods are shown in
the example below. In the enumeration procedure the initial suspension and/or its dilutions are surface
plated on a L. monocytogenes specific isolation medium, ALOA or LMBA or Chromogenic Listeria
No. 8, 4. Ed. (2008) Page: 25 of 33
Version: 4
Agar with equal operating principle as ALOA. After incubation presumptive L. monocytogenes colonies
are counted and confirmed using appropriate morphological and biochemical tests.
The collaborative validation showed that there were no significant statistical differences in the results either from
the qualitative or the quantitative part when using the different selective media. In the following example we
have chosen to use the results on ALOA for cheese, salmon and ham (food samples). Table 5 is an extraction
of the results of the collaborative validation study of NMKL Method No 136.
Table 5: Extract from table 2, annex 1 of NMKL Method No. 136. Number of participating laboratories and
the standard deviation of the reproducibility of the analyses of cheese, salmon and ham by ALOA.
Matrix ALOA
Cheese No of labs 14 14 12 13
sR 0.17 0.31 0.15 0.24
Salmon No of labs 14 14 12 13
sR 0.39 0.24 0.24 0.28
Ham No of labs 14 14 13 14
sR 0.19 0.31 0.17 0.15
sR: standard deviation of reproducibility
The estimation of the uncertainty, u, is calculated as follows:
u=
∑ (n − 1)sR
i i
2
=
∑ (n − 1)i
13 ⋅ 0.1 72 + 13 ⋅ 0.312 + 11 ⋅ 0.15 2 + 12 ⋅ 0.24 2 + 13 ⋅ 0.39 2 + 13 ⋅ 0.24 2 + ....... + 12 ⋅ 0.17 2 + 13 ⋅ 0.15 2

= 0.270
13 + 13 + 11 + 12 + 13 + 13 + 11 + 12 + 13 + 13 + 12 + 14
The combined standard uncertainty, u = 0.27 log10 cfu/g.
The expanded uncertainty, U = 2 · u = 0.54 log10 cfu/g.
0.54 log10 cfu/g is a rather high, but realistic estimation of measurement uncertainty, when it is estimated
from a method performance study. The high uncertainty is due to the fact that the spread from all the labora-
tories are included in the estimate. When using results from collaborative studies to estimate the laboratory’s
measurement uncertainty, the laboratory has to check that their own obtained values for the precision is not
poorer than the results obtained by the participants in the study. This can be performed by analysing several
parallels in order to check that the repeatability obtained at own laboratory lies within the levels for the re-
peatability (not the reproducibility) in the method validation and/or by participation in proficiency testing
schemes, which is discussed in chapter 6. Maybe the laboratory will conclude that it may pay to make an
No. 8, 4. Ed. (2008) Page: 26 of 33
Version: 4
effort to estimate the measurement uncertainty based on internal reproducibility rather than using validation
date from a collaborative study.
Expression of result:
Assume the obtained result is 1000 cfu/g = 3.0 log10 cfu/g. The result can be expressed in one of the follow-
ing options:
3.0 log10 cfu/g ± 0.54 log10 cfu/g
3.0 log10 cfu/g [2.46– 3.54]
If it is preferable to express the result in cfu/g in stead of log10 cfu/g, the result is transformed, 103 cfu/g =
3000 cfu/g. The result can the be expressed as
1000 cfu/g [103.00 – 0.54, 103.00 + 0.54] = 1000 cfu/g [288, 3467] = 1000 cfu/g [290, 3500]
2.9·102 – 3.5·102 cfu/g
No. 8, 4. Ed. (2008) Page: 27 of 33
Version: 4
7. CONTROL OF THE ESTIMATED MEASUREMENT UNCERTAINTY
Unfortunately, it is not possible to establish an estimate of the measurement uncertainty for forever all with-
out any form of further follow-ups. As every thing else one have to check that the obtained estimate is real-
istic, that the uncertainty is not over or under estimated. The control of the measurement uncertainty can
simplest be performed by
use of control charts, and
participation in proficiency testing schemes
7.1 Use of control charts - control of measurement uncertainty
Use of control charts, as described in NMKL Procedure No. 18: (”The use of reference materials, reference
strains and control charts in a food microbiological laboratory”) is a useful tool for maintaining the quality
on the analyses and to check if the estimated measurement uncertainty is realistic.
A control chart is a diagram where the results of analyses from known control samples are continuously
filled in. The control chart is connected to a certain method and matrix. On the control chart the mean, warn-
ing and action limits are marked. The warning limits corresponds to the mean ± the measurement uncer-
tainty with a coverage factor, k = 2, and the action limit corresponds to the measurement uncertainty with a
coverage factor of 3.
Upper and lower warning limits: Mean value ± 2 · u

Upper and lower action limits: Mean value ± 3 · u
Factor 2 and 3 correspond to 95% and 99% significance.
Example of such a control chart is elaborated for aerobic bacteria in chocolate pudding mix from example 5.2.
No. 8, 4. Ed. (2008) Page: 28 of 33
Version: 4
Control chart
4,4
4,2 action limit

warning limit
4
3,8
mean
log cfu/g
3,6
3,4 action limit

warning limit
3,2
2,8
0 2 4 6 8 10 12
time
Mean: 3.76 log10 cfu/g, upper warning limit: 4.04 log10 cfu/g; lower warning limit: 3.48 log10 cfu/g
upper action limit: 4,18 log10 cfu/g, lower action limit: 3.34 log10 cfu/g,
Here are also the results of the analyst under training included (Analyst V table 1). In one case the result is
below the lower action limit and at two occasions the result is above the upper warning limit (all 3 results
are performed by Analyst V). Analysis of the control sample should be run frequently and be plotted on the
control chart. If the result of the control sample lies within the warning limits, the measurement uncertainty
used for expressing the result is ok. If the result is outside both the warning and the action limit the labora-
tory has to follow up and evaluate whether the area of measurement uncertainty should be expanded. Is
however the results always far from the warning limits, it should be considered whether the estimated meas-
urement uncertainty is too big. The control charts are also useful to evaluate any general increasing tenden-
cies of the results. This should be considered even if the obtained results are within the warning limits. An
assessment of such a tendency can prevent sudden results above the warning limit.
No. 8, 4. Ed. (2008) Page: 29 of 33
Version: 4
7.2 Participation in proficiency testing schemes – control of the measurement

uncertainty
The established value for the measurement uncertainty can also be controlled by participation in proficiency
testing schemes. If the laboratory’s obtained results ± the measurement uncertainty includes the mean value
of the scheme (or ”the true value”), the measurement uncertainty estimated by the laboratory is satisfactory.
However, if the “true value” is just outside the confidence interval, the interval for the measurement uncer-
tainty should be expanded correspondingly. If it is far outside the limits, the laboratory has to check for cal-
culation error, sample mistake/problems, reagent, method and / or the routine of the laboratory.
In many of the proficiency testing schemes the z-score of the laboratory is given, which is the difference
between the result obtained by the laboratory and ”true value” (the median or mean of all the results) di-
vided on the standard deviation to all participants. Usually the results are considered satisfactory when the z-
score of the laboratory is less than ± 2, which corresponds to upper and lower warning level. Thus the profi-
ciency testing results of the laboratory can be plotted in a control chart for each analysis where the lower and
upper warning limit is 2 z-scores and the action limits are ± 3 z-scores.
No. 8, 4. Ed. (2008) Page: 30 of 33
Version: 4
8. LITERATURE
1. ILAC-G17 (2002): Introducing the concept of uncertainty of measurement in testing in association

with the application of the standard ISO/IEC 17025. See www.ilac.org.
2. GUM (1995): Guide to the expression of uncertainty in measurement (1995), ISBN 92-67-10188-9.
3. NMKL Procedure No 5, 2. Ed. 2003: Estimation and expression of measurement uncertainty of

chemical analysis.
4. Niemelä (2003): Uncertainty of quantitative determinations derived by cultivation of microorgan-

isms. MIKES Publication J4/2003.
5. NMKL Procedure No 18, 2006: The use of reference materials, reference strains and control charts
in a food microbiological laboratory.
6. NMKL Procedure No 4, 2. version 2005: Validation of chemical analytical methods.
7. VIM (2007) International Vocabulary of basic and general standard terms in Metrology. 2007, ISO
Guide 99, 3rd Ed., ISO Geneva 2007.
8. NMKL Procedure No 12, 2002: Guide on sampling for analysis of foods.
9. NMKL rapport nr 5, 2. utg., 1995: Handledning i kvalitetssäkring - för mikrobiologiska laboratorier.

(Available in Swedish and Finnish only)
10. NMKL rapport nr 1, 1983: Statistical evaluation of results from quantitative microbiological exami-
nation.
11. Norsk Akkreditering, Sektorkomité P9 sitt dokument, NA-S53: Måleusikkerhet ved mikrobiologiske
analyser. (Available in Norwegian only)
No. 8, 4. Ed. (2008) Page: 31 of 33
Version: 4
12. ISO/TS 19036 Microbiology of food and animal feeding stuffs -- Guidelines for the estimation of
measurement uncertainty for quantitative determinations.
13. ISO 16140: Microbiology of food and animal feeding stuffs -- Protocol for the validation of alterna-
tive methods.
No. 8, 4. Ed. (2008) Page: 32 of 33
Version: 4
ANNEX:
Comparison of estimations of measurement uncertainty based on dif-

ferent statistical models.
In the previous editions of NMKL Procedure No. 8, the Poisson distribution was used as the mathematical
model for the estimation of measurement uncertainty. Comments received on the procedure indicate that the
measurement uncertainty was underestimated. Would estimates of the measurement uncertainty be very dif-
ferent if the calculations were carried out according to the old and new version of NMKL Procedure No 8,
respectively?
In this Annex, an example from the previous version of NMKL Procedure No. 8 is used to compare the two
models. The example is based on the estimation of the number of heterotrophic bacteria in a food sample,
where data from 4 countable plates are used. The following formulas are used in the calculations:
Poisson distribution Normal distribution
n
Mean, X n
C1 + C2 + .. + Cn ∑C n
C ∑x i
x1 + x 2 + x 3 ... + x n
= 1
= i =1
=
V1 + V2 + .. + Vn n
∑V
V n n
n
1
x = single results (cfu/g) or log10 cfu/g
C = number of colonies
n = number of parallels
V = dilution
(use Excel – Mean (figure1, figure2, figure3,
figure4))
Standard deviation, SD 1 n
∑ ( xi − x )
2
C i =1
V n −1
(use Excel – STDEV(figure1, figure2, figure3,

figure4))
No. 8, 4. Ed. (2008) Page: 33 of 33
Version: 4
The results for heterotrophic bacteria are reproduced below as cfu/g and log10 cfu/g.
Results Poisson distribution Normal distribution Normal distribution
cfu/g at dilution cfu/g log10 cfu/g
-4
112 at 10 1120000 6.05
-4
95 at 10 1200000 6.08
12 at 10-5 950000 5.98
9 at 10-5 900000 5.95
112 + 95 + 12 + 9
10 + 10 − 4 + 10 − 5 + 10 − 5
−4
Mean value 228 6.015 ≈ 1.04 · 106

= = 103636 4 6
0.00022 = 1042500 ≈ 1.04 · 10
6
≈ 1.04 · 10
1
Standard deviation 228 = 68635 141038 0.059
0.00022
6.02 ± 0.12
Expanded measurement I cfu/g:
1.04 · 106 ± 2· 0.69 · 105
uncertainty 1.04 · 10 ± 2 · 0.14·10
6 6
= 1.04· 106 ± 2· (106.02-
= 1.04 · 106 ± 0.14 · 106
U = X ± 2 · SD 106.02-0.059)
1.04 · 106 ± 2· 0.13·106
Confidence interval cfu/g [0.90 · 106 , 1.2 · 106] [0.76 · 106 , 1.3 · 106] [0.78· 106 , 1.3 · 106]
In this example, the estimates of the expanded measurement uncertainties obtained by the calculations based
on Normal distribution, are exactly twice the expanded uncertainty calculated using the model from the
Poisson distribution. The lowest level of the confidence interval from Poisson, 0.90 · 106 cfu/g, is also the
lowest obtained result from the 4 counted plates. This leaves no room for results below the lowest measured
value, and therefore the measurement uncertainty could be considered being too narrow.

NMKL Procedure No. 8, 4. Ed. (2008) : Ontents

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

NMKL Procedure No. 8, 4. Ed. (2008) : Ontents

Uploaded by

Copyright:

Available Formats

Measurement of uncertainty in quantitative microbiological

NMKL PROCEDURE examination of foods

Nordic Committee on Food Analysis

The procedure is available from:

Error (of measurement)

Certified reference material (CRM)

Standard deviation (SD)

Standard uncertainty, u(xi)

Combined standard uncertainty, u

4.2 Reference materials, proficiency testing

4.3 Official analytical methods

4.4 Large variations between results

4.5 Microbial distribution in sample

4.7 Single microbes or agglomerations?

4.8 Microbial growth and death

4.9 The analysts - Procedural differences

4.10 Reliability of colony counting

5. DESIGN FOR ESTIMATION OF MEASUREMENT UNCERTAINTY

a) Relation between level and standard deviation in cfu/g

b) Relation between level and standard deviation when the

c) Relation between level/concentration and standard deviation

6. ESTIMATION OF MEASUREMENT UNCERTAINTY

Combinations/Series A;1;1 B;2;2 C;3;3

Replicate log10 log10 log10

9 xa11-9 xb22-9 xc33-9

It is easy to do this in Excel. The results can be presented in a table:

Combination/Series A;1;1 B;2;2 C;3;3

( n1 − 1) s12 + (n2 − 1) s 22 + (n3 − 1) s32

Step 2: Calculate the precision between the series /the combinations

Step 4: Estimate the measurement uncertainty, U.

6.1 Estimation of measurement uncertainty from internal reproducibility from

Follow the steps 1 - 4 to establish an estimate for the measurement uncertainty.

Step 1: The precision within in the series

The standard deviation of the series I:

Sr12 + Sr22 + Sr32 + Sr42 0.162 + 0.092 + 0.172 + 0.122

Step 2: The precision between the series

deviation calculate the between-series variance.

The mean of the series’ I, II, II and IV mean values, y:

The standard deviation of y, Sx:

The between-series variance, SL2:

u = S R = S r2 + S L2 = 0.142 + 0.00096 = 0.14 (log10 cfu/g)

Step 4: Estimate the measurement uncertainty, U

Expression of the result:

6.2 Estimation of measurement uncertainty based on internal reproducibility from

Follow the steps 1 - 4 to establish an estimate for the measurement uncertainty.

Step 1: The precision within in the series

0.044 2 + 0.076 2 + 0.023 2 + 0.034 2 + 0.056 2 + 0.016 2

Step 2: The precision between the series

u = S R = S r2 + S L2 = 0.042 2 + 0.033 2 = 0.054 (log10 cfu/g)

Step 4: Estimate the measurement uncertainty, U

Expression of the result:

6.3 Estimation of measurement uncertainty based on results from collaborative

The estimation of the uncertainty, u, is calculated as follows:

13 ⋅ 0.1 72 + 13 ⋅ 0.312 + 11 ⋅ 0.15 2 + 12 ⋅ 0.24 2 + 13 ⋅ 0.39 2 + 13 ⋅ 0.24 2 + ....... + 12 ⋅ 0.17 2 + 13 ⋅ 0.15 2

7. CONTROL OF THE ESTIMATED MEASUREMENT UNCERTAINTY

7.1 Use of control charts - control of measurement uncertainty

Upper and lower warning limits: Mean value ± 2 · u

4,2 action limit

3,4 action limit

7.2 Participation in proficiency testing schemes – control of the measurement

1. ILAC-G17 (2002): Introducing the concept of uncertainty of measurement in testing in association

3. NMKL Procedure No 5, 2. Ed. 2003: Estimation and expression of measurement uncertainty of

4. Niemelä (2003): Uncertainty of quantitative determinations derived by cultivation of microorgan-