Excel-calculation tool ISO 16140-3:2021 Version number: 1

(version: 20210319)

This Excel®-based program (workbook) can be used for the calculations as described in ISO 16140-3:2021
Microbiology of the food chain - Method validation - Part 3: Protocol for the verification of reference methods and validate

This free-for-use workbook was developed by members of ISO/TC 34/SC 9/WG 3 'Method validation' with the support of IS
It is based on an Excel-tool originally created by J.H.M. van Lieverloo (www.viaeterna.nl) for Unilever Innovation Centre W
The information is given after the exercise of all reasonable care and skill in its completion, preparation and issue, but is pr

The use of this workbook requires basic proficiency in Excel. Before using this workbook, make a copy for y

Separate worksheets are available for the various clauses within ISO 16140-3:2021 'Method verification':

5 - Quali - eLOD50 (Protocol 1) Qualitative methods - Technical protocol for verification: Calculation

5 - Quali - eLOD50 (Protocol 2) Qualitative methods - Technical protocol for verification: Calculation

5 - Quali - eLOD50 (Protocol 3) Qualitative methods - Technical protocol for verification: Calculation

6.1 - Quanti - SIR Quantitative methods - Technical protocol for verification: Calculatio

6.2 - Quanti - eBias Quantitative methods - Technical protocol for verification: Calculatio

Initially, all worksheets show data and results from corresponding examples in ISO 16140-3.
However, you can overwrite these by entering your own data.

In general: Enter your data into the yellow cells

Calculation results will be shown in blue cells
Evaluation of the results will be shown in green (accepted) or orange (not accepted):

Entering data should only be done in the yellow cells.

In addition, the cells identifying the (food) items 1 - 9 may be re-named.
All other cells should remain protected, in order not to accidently change any of the (hidden) formulae.

In order to improve this Excel®-based program (workbook), please report any comments by contacting the secretariat of IS
ence methods and validated alternative methods in a single laboratory

tion' with the support of ISO/TC 34/SC 9/WG 2 'Statistics' (WG 2 has verified this version as well).
ever Innovation Centre Wageningen B.V. (Wageningen, The Netherlands).
aration and issue, but is provided without liability in its application and use.

rkbook, make a copy for your own use.

or verification: Calculation and evaluation tool for verification of estimated LOD 50 (eLOD50) using Protocol 1 (ISO 16140-3, 5.5.1).

or verification: Calculation and evaluation tool for verification of estimated LOD 50 (eLOD50) using Protocol 2 (ISO 16140-3, 5.5.2).

or verification: Calculation and evaluation tool for verification using Protocol 3 (ISO 16140-3, 5.5.3).

for verification: Calculation and evaluation tool for verification of the intralaboratory reproducibility standard deviation (S IR) (ISO 1614

for verification: Calculation and evaluation tool for verification of the estimated bias (eBias) (ISO 16140-3, 6.2)

ange (not accepted): Accepted Not accepted

tacting the secretariat of ISO/TC 34/SC 9/WG 3 'Method validation' at: Laura.Mout@nen.nl
col 1 (ISO 16140-3, 5.5.1).

col 2 (ISO 16140-3, 5.5.2).

tandard deviation (S IR) (ISO 16140-3, 6.1)

 Microbiology of the food chain - Method validation - Part 3: Protocol for the verification of referenc
International Standard ISO 16140-3:2021
Cell colour coding:

5 Qualitative methods - Technical protocol for verification

Calculation and evaluation tool for verification of estimated LOD50 (eLOD50) using Protocol 1 (ISO 16140-3, 5.5

5.4.2 Inoculation of the test portions

Determination of the inoculum level (based on appropriate enumeration of the high-level inoculum, or by MPN

Implementation
verification
(Food) item 1
Determined low inoculum level LIL (cfu/test portion): 2.0

5.5.1 Determination of eLOD50 using protocol 1

Results per Inoculum level [number of positive (food) item test portions per inoculum level: enter 0, 1, 2, 3 or 4
Inoculum level of the test portions (Food) item 1
High inoculum 1
Intermediate inoculum 4
Low inoculum 3
Blank (uninoculated) 0
1430
eLOD50 (cfu/test portion) = 0,5 x LIL
Determined eLOD50 (cfu/test portion) 1.0

5.6 Acceptability limits (protocol 1)

The eLOD50 shall not be > 4 x LOD50 observed in the validation study

Published validation data of the method. If no validation data is available, assume an LOD 50 of 1 cfu/test portio

Observed LOD50 (cfu/test portion) 2.5

Acceptable eLOD50 (cfu/test portion) = 4 * LOD50 10.0

Acceptability limit evalution

Determined eLOD50 ≤ Acceptable eLOD50 Accepted

1 Table 6 - Determination of eLOD50 based on the number of positive results per level of contamination using p

Combi- High inoculation level Intermediate inoculation level

nation targeted 9 x LOD50 / test portion targeted 3 x LOD50 / test portion
1440 1/1 4/4
1430 1/1 4/4
1420 1/1 4/4
1410 1/1 4/4
1400 1/1 4/4
1340 1/1 3/4
1330 1/1 3/4
1320 1/1 3/4
1310 1/1 3/4
1300 1/1 3/4
1240 1/1 2/4
1230 1/1 2/4
1220 1/1 2/4
1210 1/1 2/4
1200 1/1 2/4
1140 1/1 1/4
1130 1/1 1/4
1120 1/1 1/4
1110 1/1 1/4
1100 1/1 1/4
1040 1/1 0/4
1030 1/1 0/4
1020 1/1 0/4
1010 1/1 0/4
1000 1/1 0/4

a
LIL = Low inoculation level.
b
Unreliable MPN result: MPN combination is very unlikely to occur. Experiment shall be repeated

End of this tab

for the verification of reference methods and validated alternative methods in a single laboratory

Enter values Calculated results Deviance

NOTE: initially, this excel sheet shows data from the EXAMPLE in Clause 5, but these can be replaced by your own data

using Protocol 1 (ISO 16140-3, 5.5.1)

he high-level inoculum, or by MPN according to Annex C)

(Food) item verification

(Food) item 2 (Food) item 3 (Food) item 4 (Food) item 5

noculum level: enter 0, 1, 2, 3 or 4 in each cell]

(Food) item 2 (Food) item 3 (Food) item 4 (Food) item 5

0 0 0 0
Enter results Enter results Enter results Enter results
Enter results Enter results Enter results Enter results

sume an LOD 50 of 1 cfu/test portion

Enter data Enter data Enter data Enter data

Not accepted Not accepted Not accepted Not accepted

per level of contamination using protocol 1

Low inoculation level Blank level

targeted 1 x LOD50 / test portion
4/4 0/1
 3/4 0/1
2/4 0/1
1/4 0/1
0/4 0/1
4/4 0/1
3/4 0/1
2/4 0/1
1/4 0/1
0/4 0/1
4/4 0/1
3/4 0/1
2/4 0/1
1/4 0/1
0/4 0/1
4/4 0/1
3/4 0/1
2/4 0/1
1/4 0/1
0/4 0/1
4/4 0/1
3/4 0/1
2/4 0/1
1/4 0/1
0/4 0/1

ment shall be repeated

a single laboratory

t these can be replaced by your own data

(Food) item verification

(Food) item 6 (Food) item 7 (Food) item 8 (Food) item 9

(Food) item 6 (Food) item 7 (Food) item 8 (Food) item 9

0 0 0 0
Enter results Enter results Enter results Enter results
Enter results Enter results Enter results Enter results

Enter data Enter data Enter data Enter data

Not accepted Not accepted Not accepted Not accepted

10 11 12

eLOD50 Lookup result

(cfucfu/test
per testportion
portion)
< 1,0 x LIL a not applied
 = 0,5 x LIL 0.5
= 0,7 x LIL 0.7
= 1,0 x LIL 1
= 1,5 x LIL 1.5
= 0,7 x LIL 0.7
= 1,0 x LIL 1
= 1,3 x LIL 1.3
= 1,7 x LIL 1.7
= 2,3 x LIL 2.3
= 1,1 x LIL 1.1
= 1,5 x LIL 1.5
= 1,9 x LIL 1.9
= 2,6 x LIL 2.6
= 3,7 x LIL 3.7
Unreliable MPN result b not applied
= 2,1 x LIL 2.1
= 2,8 x LIL 2.8
= 4,0 x LIL 4
= 6,3 x LIL 6.3
Unreliable MPN result b not applied
= 3,0 x LIL 3
= 4,3 x LIL 4.3
= 6,7 x LIL 6.7
= 14,0 x LIL 14
End of this tab
End of this tab
 Microbiology of the food chain - Method validation - Part 3: Protocol for the verification of reference m
International Standard ISO 16140-3:2021
Cell colour coding:

5 Qualitative methods - Technical protocol for verification

Calculation and evaluation tool for verification of estimated LOD50 (eLOD50) using Protocol 2 (ISO 16140-3, 5.5.2)

5.4.2 Inoculation of the test portions

Determination of the inoculum level (based on appropriate enumeration of the intermediate-level inoculum, or by

Implementation
verification
(Food) item 1
Determined low inoculum level LIL (cfu/test portion): 2.0

5.5.2 Determination of eLOD50 using protocol 2

Results per Inoculum level [frequency of positive duplicates per (food) item test portion and per inoculum level: ent
Inoculum level of the test portions (Food) item 1
High inoculum not applied
Intermediate inoculum 3
Low inoculum 1
Blank (uninoculated) 0
310
eLOD50 (cfu/test portion) = 1,4 x LIL
Determined eLOD50 (cfu/test portion) 2.8

5.6 Acceptability limits (protocol 2)

The eLOD50 shall not be > 4 x LOD50 observed in the validation study

Published validation data of the method. If no validation data is available, assume an LOD 50 of 1 cfu/test portion

Observed LOD50 (cfu/test portion) 2.5

Acceptable eLOD50 (cfu/test portion) = 4 * LOD50 10.0

Acceptability limit evalution

Determined eLOD50 ≤ Acceptable eLOD50 Accepted

1 Table 8 - Determination of eLOD50 based on the number of positive results per level of contamination using proto
 Combi- High inoculation level Intermediate inoculation level
nation
not applied targeted 3 x LOD50 / test portion
350 3/3
340 3/3
330 3/3
320 3/3
310 3/3
300 3/3
250 2/3
240 2/3
230 2/3
220 2/3
210 2/3
200 2/3
150 1/3
140 1/3
130 1/3
120 1/3
110 1/3
100 1/3
50 0/3
40 0/3
30 0/3
20 0/3
10 0/3

a
LIL = Low inoculation level.
b
Unreliable MPN result: MPN combination is very unlikely to occur. Experiment shall be repeated

End of this tab

otocol for the verification of reference methods and validated alternative methods in a single laboratory

Enter values Calculated results Deviance fields temporarily visible (only in alpha test version ISO, will
NOTE: initially, this excel sheet shows data from the EXAMPLE in Clause 5, but these can be replaced by your own data

LOD50) using Protocol 2 (ISO 16140-3, 5.5.2)

tion of the intermediate-level inoculum, or by MPN according to Annex C)

(Food) item verification

(Food) item 2 (Food) item 3 (Food) item 4 (Food) item 5 (Food) item 6

item test portion and per inoculum level: enter or select 0, 1 or 2 in each cell]
(Food) item 2 (Food) item 3 (Food) item 4 (Food) item 5 (Food) item 6
not applied not applied not applied not applied not applied
3
1
1
311 0 0 0 0
Positive blank Enter results Enter results Enter results Enter results
Repeat test Enter results Enter results Enter results Enter results

ble, assume an LOD 50 of 1 cfu/test portion

Enter data Enter data Enter data Enter data Enter data

Not accepted Not accepted Not accepted Not accepted Not accepted

esults per level of contamination using protocol 2 10

 Low inoculation level Blank level eLOD50
(cfu per test portion)
targeted 1 x LOD50 / test portion cfu/test portion
5/5 0/1 < 1,0 x LIL a
4/5 0/1 = 0,4 x LIL
3/5 0/1 = 0,7 x LIL
2/5 0/1 = 1,0 x LIL
1/5 0/1 = 1,4 x LIL
0/5 0/1 = 2,0 x LIL
5/5 0/1 = 0,7 x LIL
4/5 0/1 = 0,9 x LIL
3/5 0/1 = 1,2 x LIL
2/5 0/1 = 1,6 x LIL
1/5 0/1 = 2,3 x LIL
0/5 0/1 = 3,7 x LIL
5/5 0/1 Unreliable MPN result b
4/5 0/1 = 1,4 x LIL
3/5 0/1 = 1,8 x LIL
2/5 0/1 = 2,6 x LIL
1/5 0/1 = 4,1 x LIL
0/5 0/1 = 8,6 x LIL
5/5 0/1 Unreliable MPN result b
4/5 0/1 Unreliable MPN result b
3/5 0/1 = 2,9 x LIL
2/5 0/1 = 4,5 x LIL
1/5 0/1 = 9,4 x LIL

xperiment shall be repeated

 End of this tab

alpha test version ISO, will be invisible in final version)

d by your own data

(Food) item 7 (Food) item 8 (Food) item 9

(Food) item 7 (Food) item 8 (Food) item 9

not applied not applied not applied

0 0 0
Enter results Enter results Enter results
Enter results Enter results Enter results

Enter data Enter data Enter data

Not accepted Not accepted Not accepted

11 12
 eLOD50 Lookup result
(cfu per test portion)
cfu/test portion
< 1,0 x LIL a not applied
= 0,4 x LIL 0.4
= 0,7 x LIL 0.7
= 1,0 x LIL 1
= 1,4 x LIL 1.4
= 2,0 x LIL 2
= 0,7 x LIL 0.7
= 0,9 x LIL 0.9
= 1,2 x LIL 1.2
= 1,6 x LIL 1.6
= 2,3 x LIL 2.3
= 3,7 x LIL 3.7
Unreliable MPN result b not applied
= 1,4 x LIL 1.4
= 1,8 x LIL 1.8
= 2,6 x LIL 2.6
= 4,1 x LIL 4.1
= 8,6 x LIL 8.6
Unreliable MPN result b not applied
Unreliable MPN result b not applied
= 2,9 x LIL 2.9
= 4,5 x LIL 4.5
= 9,4 x LIL 9.4

End of this tab

 Microbiology of the food chain - Method validation - Part 3: Protocol for the verification of reference m
International Standard ISO 16140-3:2021
Cell colour coding:

5 Qualitative methods - Technical protocol for verification

Calculation and evaluation tool for verification using Protocol 3 (ISO 16140-3, 5.5.3).

5.4.2 Inoculation of the test portions with 3 cfu to 5 cfu per test portion
Determination of the inoculum level (based on appropriate enumeration of the inoculum, or by MPN according to

Implementation
verification
(Food) item 1
Determined inoculum level (cfu/test portion): 4.0

5.5.3 Use of protocol 3

Results [number of positive (food) item test portions: enter 0, 1, 2, 3, 4, 5, 6, 7 in each cell]
(Food) item 1
7 inoculated replicates 7
Blank (uninoculated) 0

5.6 Acceptability limits (protocol 3)

There shall be a minimum of six positive results out of the seven replicates tested

Acceptability limit evalution

Positive results ≥ 6 Accepted

End of this tab

ol for the verification of reference methods and validated alternative methods in a single laboratory

Enter values Calculated results Deviance

NOTE: initially, this excel sheet shows data from the EXAMPLE in Clause 5, but these can be replaced by your own data

the inoculum, or by MPN according to Annex C)

(Food) item verification

(Food) item 2 (Food) item 3 (Food) item 4 (Food) item 5 (Food) item 6
1.7 5.0 5.4 4.0 4.0
Inoculum > 5
Repeat experiment

, 7 in each cell]
(Food) item 2 (Food) item 3 (Food) item 4 (Food) item 5 (Food) item 6
6 7 7 4 7
0 0 0 0 1
Positive blank
Repeat experiment

tested

Accepted Accepted Not accepted Not accepted Not accepted

 End of this tab

d by your own data

(Food) item 7 (Food) item 8 (Food) item 9

(Food) item 7 (Food) item 8 (Food) item 9

Not accepted Not accepted Not accepted

End of this tab

 Microbiology of the food chain - Method validation - Part 3: Protocol for the verification of referenc
International Standard ISO 16140-3:2021
Cell colour c

6 Quantitative methods - Technical protocol for verification; calculation and lookup tool for co
Calculation and evaluation tool for verification of the intralaboratory reproducibility standard deviation (S IR)

6.1.6 Evaluation of results

The intralaboratory reproducibility (SIR) is calculated, based on a minimum of 10 laboratory samples, according
NOTE: Results of tests with low counts (e.g. indicative results) cannot be used for the calculation of S IR (see crit

Where SIR
i
n
YiA, YiB

Calculation of the Intralaboratory reproducibility (SIR)

Table 10 Number of samples used in calculations 10 -------

Laboratory sample (or test sample) number 1
Test portion number and duplicate ID 1A
Result per test portion expressed as cfu/g or ml 180
log10 cfu/g or ml 2.26

Table 11 SIR calculation

Absolute difference |YiA - YiB| log10 cfu/g or ml 0.0048
Squared difference |YiA - YiB|2 0.0000
Sum of squared differences 0.6020
Sum of squared differences / (2 * 10) 0.0301
√(Sum of squared differences / (2 * 10)) = SIR 0.17

6.1.7 Acceptability limit

The SIR shall be ≤ 2 x the lowest mean value of the S R of the (food) items used in the validation study
SIR : intralaboratory reproducibility standard deviation; S R : interlaboratory reproducibility standard deviation

From the published validation data of the method: Lowest mean SR 0.18

Acceptability limit evaluation (SIR ≤ 2 x SR) Accepted

End of this tab

verification of reference methods and validated alternative methods in a single laboratory

Cell colour coding: Enter data Calculated results Deviance

NOTE: initially, this excel sheet shows the valid data (Laboratory sample numbers 2,3,4,5,6,7,8,9,10,12

and lookup tool for comparing results with results of the verification
y standard deviation (S IR) (ISO 16140-3, 6.1)

atory samples, according to the formula given below.

calculation of S IR (see criteria for low counts in ISO 7218).

SIR is the intralaboratory reproducibility standard deviation

i is the index of the laboratory sample, i = 1 to n (n ≥ 10)
n is the number of samples
YiA, YiB are the log-transformed data, in log10(cfu/g) or log10(cfu/ml) from conditions A and B respectively

Scroll to the right if you wish to e

-------------------------------------------------------------------------------------- Minimum of 10 samples required -------------------

1 2 3 4 5 6
1B 2A 2B 3A 3B 4A 4B 5A 5B 6A 6B
182 410 620 640 330 690 570 780 640 620 1300
2.26 2.61 2.79 2.81 2.52 2.84 2.76 2.89 2.81 2.79 3.11

0.0048 0.1796 0.2877 0.0830 0.0859 0.3216

0.0000 0.0323 0.0828 0.0069 0.0074 0.1034
0.6020
0.0301
0.17

validation study
ility standard deviation

0.18

Accepted
bers 2,3,4,5,6,7,8,9,10,12) from the EXAMPLE in Table 10 and Table 11, but these can be replaced by your own data

pectively

the right if you wish to enter results of more samples Note: only results of the first 10 samples (v

s required --------------------------------------------------------------------------------- -------------------------------------

7 8 9 10 11 12
7A 7B 8A 8B 9A 9B 10A 10B 11A 11B 12A
870 1500 8600 6400 16000 5000 20000 32000
2.94 3.18 3.93 3.81 4.20 3.70 4.30 4.51

0.2366 0.1283 0.5051 0.2041

0.0560 0.0165 0.2552 0.0417
s of the first 10 samples (value entered in cell C20) will be used in calculations.

---------------------------------------------------------------------------------------------- Reserve samples -----------------------------------------------------------

12 13 14 15 16 17
12B 13A 13B 14A 14B 15A 15B 16A 16B 17A 17B
 End of this tab

-----------------------------------------------------------------------
18 19 20
18A 18B 19A 19B 20A 20B

End of this tab

 Microbiology of the food chain - Method validation - Part 3: Protocol for the verification of referenc
International Standard ISO 16140-3:2021
Cell colour coding:

6 Quantitative methods - Technical protocol for verification

Calculation and evaluation tool for verification of the estimated bias (eBias) (ISO 16140-3, 6.2)

6.2.5 Evaluation of results

Compare the results of the artificially contaminated (food) item to the results of the inoculum suspension teste
*This calculation sheet is based on the use of a 10-gram test portion inoculated with 1 ml of inoculum (also see

Contamination level Level 1

Laboratory sample (or batch) 1
Test portion number 1 2

Result per test portion in cfu/g or ml 74 177

Result per test portion in log10 cfu/g or ml 1.87 2.25
Mean result per laboratory sample in log10 cfu/g or ml 2.06
Mean result in log10 cfu per laboratory sample* 3.06

Concentration inoculum suspension in cfu/ml 1480

Concentration inoculum suspension in log10 cfu/ml 3.17

6.2.6 Acceptability limit

It is expected that, at each level, the absolute difference between the results of the artificially contaminated (f

Determined eBias 0.11

Acceptability limit (eBias ≤ 0,5 log10) Accepted

End of this tab

erification of reference methods and validated alternative methods in a single laboratory

Cell colour coding: Enter data Calculated results Deviance

NOTE: initially, this excel sheet shows data from the EXAMPLE in Table 13, but these can be replaced by your o

oculum suspension tested, using the method to be verified.

ml of inoculum (also see Table 13).

Level 2 Level 3
2 3
1 2 1 2

1450 1150 8500 11000

3.16 3.06 3.93 4.04
3.11 3.99
4.11 4.99

11250 195000
4.05 5.29

tificially contaminated (food) item in log 10 cfu/test portion and that of the inoculum suspension is equal to or less than 0,5 log 10.

0.06 0.30

Accepted Accepted
 End of this tab

3, but these can be replaced by your own data

equal to or less than 0,5 log 10.

End of this tab