Journal of Applied Bacteriology 1994,76, 86-94

An evaluation of the repeatability and reproducibility of a

surface test for the activity of disinfectants
Sally F. Bloomfield, M. Arthur, B. Van Klingerenl, W. Pullenl, J.T. Holah2 and Rebecca Elton“
Chelsea Department of Pharmacy, King’s College London, London, UK, ’National Institute of Public Health,
Bilthoven, The Netherlands and 2Campden Food and Drink Research Association, Chipping Campden,
Gloucestershire, UK
4429/01/93:accepted 26 July 1993

S.F. B L OOMF I EL D, M. ARTHUR, B. VAN KLIN GER EN , W . PU LLEN , J.T. H OLAH AND R . ELTON.
1994. A,collaborative study was carried out to determine the precision of a disinfectant surface
test method which is currently under consideration for development as a harmonized
European standard surface test. Results indicate that significant variation in microbicidal
effect occurs both within and between test laboratories despite careful standardization of test
conditions, b u t that the variability may be less than that associated with suspension tests.
Indications are that m u c h of this variability derives from random variations in the resistance
of t h e test strains from day to day and, most particularly, from test period to test period both
within as well as between laboratories. It is concluded that although the test may be
sufficiently reliable to be used as a standard method, adequate replication m u s t be specified to
distinguish borderline pass from borderline fail concentrations.

INTRODUCTION involving suspension and surface tests carried out under

conditions simulating practical use.
The British standard 5283 (Anon. 1986) defines disin-
Phase 1 tests for bactericidal and fungicidal activity have
fection as the destruction of harmful organisms (but not
been agreed and are now being published as ‘provisional
spores) to a level acceptable for a defined purpose. T h e
European Norms’ (Anon. 1993 a,b) whilst phase 2 suspen-
ultimate purpose of disinfectant testing is to establish
sion tests simulating practical conditions are in the final
whether products meet these requirements under practical
stages of preparation. These tests are based on the ‘Method
conditions-whether this be a critical hospital situation, the
of test for the antimicrobial activity of disinfectants in food
treatment of processing surfaces in food or pharmaceutical
hygiene’ (usually referred to as the European Suspension
industries or decontamination of the hands.
Test (EST)) (Anon. 1988) in which the concentration of
As ‘field trials’ under ‘use’ conditions are difficult and
disinfectant producing a 5 log reduction in the number of
expensive to perform, the approval of disinfectants, for the
viable organisms within 5 min is determined.
most part, is based on results of laboratory tests. For evalu-
Although suspension tests can be used to assess the
ation of disinfectants standard tests which are robust, rele-
activity of disinfectants under a range of conditions, they
vant to use conditions and internationally acceptable are
give no information about how products actually perform
therefore required to verify and compare activity.
on contaminated surfaces. A number of standard surface
A number of standard test methods have been developed
tests now operate in various European countries
and are official methods in different European countries.
(Reybrouck 1992) and, although these differ in experimen-
These are reviewed by Reybrouck (1992). In 1989, the
tal detail, the method is basically the same involving quan-
European committee CEN T C 216 was established to
titative determination of viable organisms recovered from a
produce harmonized European methods for disinfectants
contaminated dried surface before and after application of a
and antiseptics used in food hygiene, medicine, agriculture
disinfectant.
and veterinary practice. The committee resolved that
This paper describes an evaluation of test procedures
testing should comprise a number of phases: phase 1
currently under consideration for a European surface test
involving suspension tests to define minimum standards for
method. Studies in the Chelsea laboratory were used to
bactericidal, fungicidal and sporicidal activity and phase 2
assess the precision of the test and to compare it with that
Correspondence to :Dr S.F. Bloornjeld, Chelsea Department of Pharmacy, of the European suspension test studied under similar con-
King’s College London, Manresa Road, London SW3 6LX, CJK. ditions (Bloomfield and Looney 1992). A collaborative

study was also carried out to evaluate the repeatability and
reproducibility of the surface test within and between dif-
ferent laboratories.
M A T E R I A L S AND M E T H O D S
Media and reagents
Media and reagents were prepared as described in the EST
(Anon. 1988). Media bases were obtained from Oxoid and
L-histidine from Sigma. Other chemicals were obtained
from BDH. Neutralization medium containing (g 1- '):
polysorbate, 30; lecithin, 3; histidine, 1-0; Na,S,O,, 5;
diluent containing (g 1-'): tryptone, 1.0; NaCl, 8.5; and
water of standard hardness (WSH) were prepared as
described in the 1987 E S T (Anon. 1988).
Test surfaces
Test surfaces were 2 cm diameter stainless steel discs with a
grade 2B finish (CMB Engineering group, Worcester, UK)
and 2 cm squares of formica (Resopal U K Ltd, Maccles-
field, UK). Before each use the surfaces were washed in
detergent and rinsed in distilled water. T h e stainless steel
discs were stored in 70% alcohol until required, and then
flamed and placed in a sterile glass tray. Formica surfaces
were swabbed twice with 70% alcohol and placed in a
sterile glass tray overnight to dry.
Disinfectants and test solutions
Disinfectants used in this study were: ethyl alcohol, a solu-
bilized phenolic disinfectant containing 5 YOw/v chloroxyle-
no1 (Dettol, Reckitt & Colman Products, Hull, UK),
sodium hypochlorite solution (1&14% w/v available chlo-
rine, BDH), benzalkonium chloride (BAK) (McCarthys
Ltd, Romford, UK), chlorhexidine gluconate solution BP
(CHDNE) (ICI Chemicals). Stock solutions of BAK and
CHDNE containing 1 % w/v were prepared in WSH and
sterilized at 121°C for 20 min. Stock solutions and products
were diluted for testing in WSH. Test solutions were
freshly prepared on each day of testing. Concentrations are
stated as the w/v of active agent or v/v for alcohol.
Test organisms and test suspensions
T h e test organisms were Enterococcus faecium (ATCC
10541) and Pseudomonas aeruginosa (ATCC 15442). Test
suspensions were prepared as described in the E S T (Anon.
1988). Stock cultures of test organisms on tryptone soya
agar (TSA), prepared from freeze-dried cultures, were
stored in a refrigerator at 5°C and subcultured monthly.
For preparation of test suspensions subcultures grown at
DISINFECTANT SURFACE TEST METHODS 87
37°C for 18-24 h at least two but not more than three suc-
cessive times on TSA were used. T h e TSA subcultures
were suspended in diluent and the number of colony-
forming units (cfu) adjusted to 1-3 x lo9 ml-' with the
diluent; the number of units was estimated with a spectro-
photometer. T h e suspension was maintained at 20°C f 1°C
and used within 2 h.
Disinfectant testing method
Test suspensions and test solutions were equilibrated to
25°C before use. After inoculation of test surfaces with 0.1
ml of suspensions containing 10' cfu the test surfaces were
dried at 37°C in a fan-assisted incubator for 1 h. Samples
of disinfectant test solution or WSH (0.1 ml) were dropped
onto the surface to cover the test film. After a contact time
of 5 min the discs were placed in a 50-ml conical flask
containing 10 ml of neutralization medium together with
3.6 g (approximately 100) of glass beads (3 mm diameter)
and placed in a shaking water bath (200 strokes min-',
stroke amplitude 7.5 cm) at 25°C for 10 min. T h e discs
were placed face down over the glass beads so that the
beads were in contact with the inoculated surface. T h e neu-
tralization medium ( 1 ml) and 10-fold dilutions of it were
plated on TSA, incubated at 37°C for 24-48 h and the
number of cfu determined. The microbicidal effect (ME
value) was calculated by subtracting Nd, the log number of
cfu per carrier after action of the disinfectant, from N c , the
log number of cfu per carrier in the control test with WSH
instead of disinfectant. Disinfectant test concentrations
were chosen to give a quantifiable M E value between 0 and
6. All tests were carried out by a trained microbiologist
with experience of disinfectant testing. T h e neutralization
medium was validated by carrying out procedures as
described above with uninoculated test surfaces. After the
surfaces were transferred to neutralization media, 10 ml
samples of that medium, with and without the discs, were
inoculated with a known number of cfu of the test organism
and the viable counts were determined and compared.
Statistical analysis of results
A one-way analysis of variance of log Nc was carried out
and within and between day components of variation esti-
mated, together with 95% confidence limits. Analysis
assumptions were checked by diagnostic residual plots.
Hierarchical analysis of variance was performed for the ME
values for each test product, test organism and test surface.
Comparison of variability estimates was made with the
Bartlett's test or variance ratio tests.
88 S A L L Y F. BLOOMFIELD E T A L .
Electron microscopy
Samples for scanning electron microscopy were air-dried
overnight, mounted on specimen stubs, sputter-coated with
gold and examined and photographed at 7.2 kV with a Phil-
lips EM501B scanning electron microscope. Air drying
rather than conventional critical point drying was employed
to preserve, as far as possible, the original distribution and
density of the organisms on the stainless steel.
RESULTS
Evaluation of the surface test method
T h e test conditions (i.e. test organisms, test surfaces, etc.)
chosen for this study were based on proposals under con-
sideration by CEN T C 216 for a European surface test
together with a consideration of test conditions most appro-
priate to practical use. T h e test procedure (i.e. method of
inoculation and recovery of organisms from surfaces, etc.)
was selected on the basis of experience gained from studies
of the Dutch, Belgian, Deutsche Gesellschaft fur Hygiene
und Mikrobiologie (DGHM) and Association franGaise de
normalisation (AFNOR) surface test methods (Werner et
al. 1977; Van Klingeren 1983) and preliminary studies in
our own laboratory as outlined below. In selecting the test
procedure the primary aim was to achieve consistent results
whilst, as far as possible, retaining relevance to practical
conditions.
Formica and stainless steel were chosen as test surfaces.
These surfaces relate to a wide variety of practical applica-
tions and, probably because they are smooth and imper-
vious, are generally reported to give more consistent results
than some other types of surfaces (Werner et al. 1977).
The Gram-negative test organism was Ps. aeruginosa as
used in phase 1 and phase 2 suspension tests, but Ent.
faecium (used in phase 2 tests only) was used as the Gram-
positive organism rather than Staphylococcus aureus which
has a tendency to form clumps which may contribute to the
variability of the test result. One of the problems of using
Ps. aeruginosa for inoculation o f test surfaces as shown by
Van Klingeren (1983) is its susceptibility to the lethal
effects of drying. Preliminary experiments (not shown here)
indicated that by suspending the organisms in diluent con-
taining 0.1 % tryptone the survival of Ps. aeruginosa was
adequately maintained but disinfectant activity was unaf-
fected.
Contaminated surfaces were prepared by inoculating the
carriers with 0.1 ml drops of test suspension which then
spread to almost cover the 2 cm surface area. Electron
micrographs of stainless steel surfaces inoculated with lo8
cfu of Ent. faecium (Fig. 1) show that the cells have a ten-
Fig. 1 Scanning electron micrograph of a stainless steel surface
inoculated with 0.1 ml of a suspension containing lo8 cfu
Enterococcus faecium and dried overnight at room temperature. Bar
marker = 10 pm
dency to form aggregates but that these aggregates are well
separated and there is no evidence of formation of contin-
uous multilayers which might affect penetration of the dis-
infectant. Some standard test methods involve spreading
the inoculum over the carrier, but preliminary observations
with formica test surfaces of larger area suggested that this
did not achieve better surface distribution but was associ-
ated with increased loss of viability on drying and some
increase in resistance to disinfectants.
Preliminary investigations (not reported here) indicated
that drying at 37°C (as in the AFNOR test) for 60 min and
at 25°C (as in the DGHM, Dutch and Belgian test), which
required 150 min, produced similar recovery rates. T h e
former method was therefore chosen to maximize the
number of tests performed in a working day.
Results in Table 1 indicate that, with the prescribed
method for inoculation and drying of surfaces, the mean Nc
values achieved by inoculating the test surface with 10' cfu
were 7.63 and 7.67 for Ent. faecium and 6.45 and 6.55 for
Ps. aeruginosa for formica and stainless steel, respectively.
Table 1 shows, as expected, that the process of drying and
recovery of test organisms from surfaces was associated
with higher limits of error for Nc values compared with
those found previously for suspension tests (i.e. i 0 . 5 7 and
0.59 for the surface test compared with P & 0.14 in the
suspension test for Ps. aeruginosa (Bloomfield and Looney
1992)), but this was not associated with increased variability
of the M E values indicating, as found in previous studies
(Van Klingeren et al. 1981; Bloomfield and Looney 1992),
that the variance in Nc values contributes little to the varia-
tion in M E values. It was noted, as found by Van
Klingeren (1983), that the increased loss of viability on
 DISINFECTANT SURFACE TEST METHODS 89

Table 1 Mean microbicidal effect (ME values), initial log count (Nc) and 95% confidence limits for disinfectant products tested on
stainless steel and formica surfaces

Mean ME values from 15 replicate tests

(95% limits of error)

Initial Dettol CHDNE BAK

count 5% NaOCl 0.1% 0.2% Alcohol
Test organism Surface (Nc) v/v 500 ppm 1000 ppm w/v w/v 35% v/v 50% V/V
~

Enterococcus faecrum Stainless steel 7.67 1.33 1.68 2.43 0.54 1.27 1.02 4.24
1*+0.26 k0.87 f0.83 f0.40 f0.58 f 1.14 f0.42 f0.57
2t - f0.60 f0.59 f0.39 f0.50 f0.84 f0.33 f0.57
Formica 7.63 1.29 1.69 2.36 0.43 1.14 1.09 4.18
1 * f0.27 f 1.61 f0.72 f0.78 -t 0.24 f0.87 f0.38 f0.65
2t - k1.03 f0.56 & 0.74 f0.19 f0.61 f0.38 f0.65

Pseudomonas aeruginosa Stainless steel 6.55 2.95 1.86 4.08 1.36 1.50 1.14 2.54
1**0.59 f0.61 f0.43 f0.59 f1.24 f0.72 f0.76 f0.57
2t - k0.49 k0.33 f0.46 f0.98 f0.57 f0.54 f0.55
Formica 6.45 2.57 1.93 4.15 1.22 1.37 1.05 2.17
1*f0.57 +1.64
- f0.58 f0.45 f 1.19 +
- 1.13 f0.35 f0.62
2t - +1.12
- f0.44 f0.43 f0.80 0.90 f0.35 f0.62
* Calculated for a single ME value.
t Calculated for the mean of three ME values.
BAK, Benzalkonium chloride; CHDNE, chlorhexidine gluconate.

drying for Ps. aeruginosa compared with Ent. faecium was
associated with increased variability of the N c values.
Evaluation of disinfectant activity on test surfaces
Table 2 shows individual M E values obtained from three
replicate tests carried out on each of 5 d with Ps. aeruginosa
inoculated onto stainless steel surfaces. These are typical of
the results obtained and illustrate the within and between
day variability for a range of different products and test
organisms. The study was carried out with both Ps. aeru-
ginosa and Ent. faecium on stainless steel and formica sur-
faces from which the mean M E values and 95% limits of
error for both sets of data were calculated (Table 1) in
order to make further observations about the test method
and its precision.
Table 1 shows that there was little or no difference in
mean ME values and limits of error for the formica com-
pared with stainless steel surfaces, suggesting that both are
equally suitable and that one of the surfaces could be
chosen to represent both ‘in use’ situations,
Whereas previous results from suspension tests
(Bloomfield and Looney 1992) indicated higher limits of
error for M E values against Staph. aureus than those
obtained for Ps. aeruginosa and Candida albicans, results
reported here indicate that the variability for surface tests
was independent of the nature of the test organism. Where
Ps. aeruginosa was used as test organism there was concern
that the presence of substantial numbers of non-viable cells
on the test surface might affect the result. Table 1,
however, indicates that M E values (particularly for NaOCl
which is most sensitive to inactivation by organic soil) were
mostly higher for Ps. aeruginosa than for Ent. faecium. It
was also noted that the ‘rank order’ was similar for both
test organisms, with NaOCl 500 and 1000 ppm, alcohol
50% and Dettol 5% being more active and BAK 0.2%’
CHDNE 0.1% and alcohol 35% being less active.
The disinfectants used in this study were chosen to rep-
resent a comprehensive range of chemical types as used in
the UK, but whereas previous studies of suspension tests
with the same products (Bloomfield and Looney 1992) indi-
cated that the variability of M E values for NaOCl tended to
be greater than those for the other products and the varia-
bility of M E values for BAK tended to be less, results of
this study indicate that the precision of the surface test
method is independent of the nature of the product.
Repeatability of ME values within a single test
laboratory
Previous studies of suspension tests (Bloomfield and
Looney 1992) indicated that between day variabiiity of M E
values tended to be greater than within day variability.
90 S A L L Y F. BLOOMFIELD E r A L .

Table 2 Microbicidal effect (ME values) against Pseudomunus aeruginosu inoculated onto stainless steel surfaces produced by disinfectant
solutions in water of standard hardness at 5 min contact

Dettol CHDNE BAK

Test 5% NaOCl 0.1% 0.2% Alcohol
day v/v 500 ppm 1000 ppm WJV Wf V 35% VJV 50% VJV
~~~ ~ ~ ~ ~ ~ ~ ~~~~~~

1 Nc 6.44 6.48 6.48 6.44 6.44 6.48 6.48

ME 2.90 1.47 3.44 0.50 1.62 0.55 2.70
ME 2.54 1.66 3.66 0.33 1.78 0.48 2.56
ME 3.05 1.67 4.03 0.93 1.63 0.72 2.40

2 Nc 6.75 6.88 6.88 6.75 6.75 6.88 6.88

ME 2.6 1 I .79 4.23 1.55 2.07 1.34 2.42
ME 2.75 2.06 4.49 1.35 1.52 1.41 2.49
ME 2.61 1.64 4.26 1.23 1.64 1.33 2.44

3 Nc 6.25 6.87 6.25 6.25 6.25 6.87 6.87

ME 2.81 2.08 4.21 2.07 1.68 1.38 2.16
ME 2.92 1.97 4.1 1 2.67 1.82 1.34 2.74
ME 3.33 2.11 4.07 1.89 1.64 1.15 2.13

4 Nc 6.47 6.68 6.68 6.47 6.47 6.68 6.68

ME 3.47 1.89 4.2 1 1.28 1.34 1.35 2.43
ME 3.22 2.06 4.32 1.97 0.78 0.95 2.89
ME 3.22 1.92 4.05 1.64 1.24 1.63 3.04

5 Nc 6.08 6.59 6.59 6.08 6.08 6.59 6.59

ME 2.85 1.82 3.89 1.02 1.52 1.05 2.33
ME 3.04 1.87 4.23 0.78 1.27 1.32 2.60
ME 2.94 2.01 4.03 1.24 1.06 1.11 2.89
Nc, Initial count ; BAK, benzalkonium chloride ; CHDNE, chlorhexidine gluconate.

From the results of the surface tests it was calculated, as significantly different (95% confidence limits for a single
shown in Table 3, that for 419 of the total of 420 ME ME value calculated as k0.9) from the other two values.
values obtained, the ME values were consistent within a In 134 of the total of 140 d of testing (954%) there was no
given day; in one test only, one ME value was found to be significant difference between results obtained on different

Table 3 Repeatability and reproducibility of surface test results

Probability of obtaining a significantly different result

Between periods Between labs

Within days Between days within labs between periods
a b C a b C a b C a b C

Chelsea study 1 420 1 0.24% 140 6 4.2% - - - - - -

Chelsea study 2 - - - 369 25 6.8% 84 15 17'8% - - -

Chelsea/Bilthoven

Campden
] 108 0 0Y"
54

18
1

4 224%
1.9% -

-
-

-
~
-1
-
~
24 4 16.6%

a, No. of tests or trials; b, no. of tests or trials which were significantly different; c, % frequency occurrence of a different result.
 DISINFECTANT SURFACE TEST METHODS 91

days over the 5 d test period but on six occasions (4.2%),
one set of M E values was consistently different (95% con-
fidence limits for a mean of three M E values calculated as
f 1.3) from the sets of values obtained on the other 4 d.
As part of our evaluation of surface testing in the Chelsea
laboratory the test method was used to compare the activity
of a range of disinfectant products that are currently used
in the UK (Bloomfield et al. 1993). In this study disin-
fectants were also tested against Staph. aureus, but tests
were carried out only once a day on each of 3 (and in some
cases 5) d. For a significant proportion of products, the 3 or
5 d testing period was repeated one or more times, the
interval between test periods never being less than 1 month
and the whole study taking place over a 6-month period.
Typical sets of results collated from different study
periods for Dettol (Table 4) illustrate that, although consis-
tent M E values were mostly obtained within a particular
test period, significant differences quite often occurred
when the test period was repeated at a later time. A
summary of all the results for all the products from this
second study (Bloomfield et al. 1993) (Table 3) indicated,
as in the first study, that for 344 of the total of 369 M E
values determined (93-2%), there was no difference (95%
confidence limits for a single M E value calculated as f 1.5)
between the values obtained on separate days over a 3 or 5
d study period. On 25 (6.8%) occasions the individual M E
value obtained was significantly different from values
obtained on the other 2 (or 4) d. Where the study period
was repeated at a later date (there were 84 study periods in
all) the number of occasions on which the M E values from
one test period were consistently different (95YOconfidence
limits for a mean of 3 M E values calculated as f 1.1) from
the M E values in the other test period (s) was 15 (17.8%).
Repeatability and reproducibility of ME values within
and between laboratories
In order to further study reproducibility within and also
between laboratories, tests with NaOCl, alcohol and BAK
were carried out on two further occasions in the Chelsea
laboratory (an original test and repeat testing after 1

Table 4 Microbicidal effect (ME values) against test organisms inoculated onto stainless steel surfaces produced by Dettol 2.5% and 5.0%
v/v in water of standard hardness at 5 min contact as determined in different study periods

Dettol ME value on day Mean

Test Study concn ME
organism period (.I.) 1 2 3 4 5 value

Staphylococcus aureus 2 2.5% 4.6 2.8 3.3 4.5 4.1 3.8

4 2.5% 3.8 4.3 2.4 3.5
5 2.5% 1.8 1.7 2.0 1.9
6 2.5% 3.1 2.8 3.6 3.2

2 5.0% 4.9 4.6 4.8 4.5 3.6 4.5

3 5.0% 3.3 3.4 3.8 3.5
5 5.0% 4.2 3.6 3.9 3.9

Pseudomonas aeruginosa 2 2.5% 1.5 1.6 1.7 1.8 1.3 1.6

4 2.5% >6 >6 >6 >6
5 2.5% >6 >6 >6 >6

5.0% 2.9 2.6 3.3 2.3 2.8 2.9

5.0% 4.1 4.0 3.8 3.8 3.6 3.8
5.0% >6 >6 >6 >6
5.0% >6 >6 >6 >6

Enterococcus faecium 2 2.5% 5.3 3.9 4.4 3.8 4.1 4.3

5 2.5% >6 >6 >6 >6

1 5.0Y" 0.8 1.9 1.2 1.4 1.9 1.3

2 5.0% 4.9 5.7 5.0 4-1 4.2 4.8
5 5.0% 3.6 4.4 3.9 3.9
92 S A L L Y F. B L O O M F I E L D E T A L .

month) and also in laboratories at Bilthoven and Campden.
For the Chelsea original and Campden tests, three replicate
test were carried out on each of three successive days. For
the Chelsea repeat and Bilthoven tests, a single test was
carried out on each of three successive days.
T h e results of the Chelsea and Campden trials (Tables 3
and 5) indicate that, for a total of 108 tests, all the ME
values obtained were consistent (95% confidence limits for
a single ME value calculated as kO.9) within a given day
which is in agreement with the first Chelsea study.
From the results of the Chelsea (including the repeat)
and Bilthoven studies as shown in Tables 3 and 5 it was
also found that for 53 of a total of 54 d of testing, all the
ME values were consistent (95% confidence limits for a
single ME value calculated as & 1.5) within the particular 3
d period. For the Campden study, however, a consistently
different (95% confidence limits for a mean of 3 ME values
calculated as f 1.3) set of results was obtained in 4 of 18
(22%) d of testing. T h e higher variability between days as
determined in this study may relate to the fact that,
although the operator was a trained microbiologist, she had
less experience of disinfectant testing.
Where the results obtained from studies carried out in
different laboratories were compared (there were 24 studies
in all) the number of occasions on which the ME values
from one test laboratory were consistently different (95%
confidence limits for a mean of 3 ME values calculated as
& 1.1) from the ME values obtained in one or both of the
other test laboratories was 4 (16.6%).
DISCUSSION
One of the most important considerations in devising stan-
dard tests for the antimicrobial activity of disinfectants
must be their precision. Collaborative studies of standard
suspension test methods (Van Klingeren ef al. 1977, 1981;
Bloomfield and Looney 1992) indicate that the repeatability
and reproducibility of these tests is a matter for concern.
These studies indicate significant variability in ME values
not only between test laboratories but also within labor-
atories despite careful standardization of test conditions.
Introduction of a carrier surface into the test introduces
additional difficulties which might be expected to increase

Table 5 Microbicidal effect (ME values) against test organisms inoculated onto stainless steel surfaces produced by disinfectant solutions
in water of standard hardness at 5 min contact as determined in different testing laboratories

Pseudomonas aeruginosa Enterococcus faecium

ME value on day ME value on day

1 2 3 Mean ME 1 2 3 Mean ME

Chelsea?
NaOCl 1000 ppm 4.03 4.23 4.2 1 4.16 2.54 2.41 2.66 2.53
BAK 0.2'Y'u 1.62 2.07 1.68 1.79 0.51 1-03 1.18 0.9 1
Ethanol 50% 2.70 2.42 2.16 2.42 4.18 4.46 4.49 4.38

Chelsea repeat
NaOCl 1000 ppm 4.38 4.92 4.48 4.59 3.11 2.82 3.44 3.12
BAK 0.2% 2.27 1.73 1.53 1.84 2.93 2.77 2.65 2.78
Ethanol 50% 3.14 2.87 2.20 2.73 4.40 4.24 4.16 4.26

Bilthoven
NaOCl 1000 ppm 4.18 4.35 4.33 4.28 1.27 1.48 1.19 1.31
BAK 0.2% 1.58 2.2 1 2.59 2.12 2.28 3.02 2.1 1 2.47
Ethanol 50% 3.13 3.53 2.90 3.18 1.73 1.43 1.71 1.62

Campdent
NaOCl 1000 ppm 2.96 2.17 3.77 2.96 3.00 1.03 1.47 1.83
BAK 0.2% 0.66 1.59 2.30 1.51 0.60 2.30 1.50 1.43
Ethanol 50% * 2.58 1.54 2.5 1 2.21 1.40 1.70 1.40 1.50

* IMS was used in this study.

t Only the first result on each day was included.
RAK, Benzalkonium chloride.

the variability of the test result. One of the major problems
is achieving consistent recovery of survivors from surfaces.
Another problem is the substantial loss of viability in
drying the inoculum onto the surface.
It is surprising, however, that the results described in
this paper indicate that the precision of surface tests may be
better than that of suspension tests. In our previous study
of the suspension test in which closely similar test condi-
tions were used (Bloomfield and Looney 1992), it was
found that, for a single M E value determined on a single
day, limits of error varied from 1.03 to 2.51 considering
all the five products tested against Ps. aeruginosa and C.
albicans, up to as much as k3.38-4.1 for NaOCl, alcohol
and chlorhexidine against Staph. aureus. By contrast the
results of the surface tests as reported here indicate limits
of error of f 0 . 3 8 for 35% alcohol against Ent. faecium up
to not more than f l . 6 4 for Dettol against Ps. aeruginosa.
From similar studies of suspension and surface tests Van
Klingeren (1983) also reported that the precision of the
Dutch surface test was at least as good as that of the sus-
pension test.
In developing proposals for a unified European surface
test into a form which is acceptable for use as an analytical
standard it is important that as far as possible we identify
the major sources of test variability and develop the test
method accordingly. From results of the three studies
described in this paper a number of observations can be
made about the sources of variability. Whereas there has
previously been a tendency to assume that variability
between test laboratories originates from variations in test
procedure, the results of studies reported in this paper,
partly because they involve such a large number of repli-
cates not only between but also within laboratories, suggest
otherwise.
T h e results of the Chelsea and also the Campden studies
indicate that where replicate tests are carried out on the
same day within a laboratory with the same test suspension,
consistent M E values may be obtained, the majority of
which differ from one another by not more than 1 log. T h e
probability that one M E value will be significantly different
from the others is less than 1YO,and it seems likely that this
occasional event is due to operator error.
Results of all the Chelsea together with the Bilthoven
studies indicate that, with test suspensions prepared from
sequential daily subcultures, it is also possible for the most
part to obtain consistent M E values on different days, but a
single M E value which is significantly different from the
other values is sometimes obtained (frequency occurrence
up to about 7”/;,-although for the relatively limited
Campden study (18 trials) this was increased to 22%). As
the ME values obtained in this study are generally consis-
tent within a given day it is unlikely that these variations
are due to operator error; they are probably due to random
DISINFECTANT SURFACE TEST METHODS 93
changes in the resistance of the test suspension-but could
be due to errors in preparing the product test solution on
the particular day.
Where a sequential series of daily tests was repeated at a
later date, either in the same laboratory or in another labor-
atory, for the most part, again, the two sets of M E values
were consistent, but in this situation the probability of
obtaining a consistent but consistently different result
between test periods was about 1618% (as opposed to 8%
for results obtained within a test period).
These results indicate that significant variability in M E
values occurs as frequently between test periods within a
laboratory (17.8%) as between test periods in different
laboratories (16.6%). This suggests that the variability does
not result primarily from differences in test procedure
between different laboratories. Since the results are consis-
tent within each laboratory within each of the test periods,
it seems likely again that this variability results from
changes in the resistance of the test suspension.
When the results of the second Chelsea study, where a
number of different biocides were tested in a given study
period, were considered in chronological sequence, it was
noted that a consistent increase in resistance to one biocide
in one test period was not accompanied by a general
increase in resistance to the other biocides. This indicates
that there is little point in using an ‘internal standard’ to
identify changes in the resistance of the test strain.
It has been concluded from previous studies of standard
suspension tests that the tests are sufficiently reliable for
use as international standards since the transition from
active to inactive disinfectant concentrations takes place
within assessable limits and the variations described above
occur only with borderline concentrations. Because of the
significant variability for borderline concentrations,
however, it is important that products are assessed from a
number of replicate tests as required to achieve confidence
in the result. For surface tests the implications are particu-
larly serious. Whereas the majority of a range of
commonly-used products, when tested at their recommend-
ed use concentration in suspension tests (Bloomfield et al.
1991), consistently give a greater than 5 log reduction (i.e.
no detectable survivors), for many of these same products
tested in a surface test (Bloomfield et al. 1993), M E values
between 0.5 and 5.0 were obtained. I t has been suggested
within committee CEN T C 2 16 that where the European
surface test is used for approval of disinfectants a pass/fail
criteria of less than 5.0 may be acceptable for certain situ-
ations.
Estimates of 95% limits of error from this study suggest
that, for a single M E value determined within a given test
period in a given laboratory (and taking the worst case situ-
ation of Dettol against Ps. aeruginosa on a formica surface),
we can only say with confidence that the M E value is less
94 S A L L Y F. B L O O M F I E L D E T A L .
than the specified pass value if it differs from that value by
more than 1.64. If the number of replicate tests is increased
to three, then a fail value is indicated by a mean ME value
which differs from the specified pass value by 1.12. I n
order to account for between test period and between
laboratory variability, this critical difference value will be
substantially increased.
From this it is concluded that, in the short term, where
these tests are adopted internationally it will be important,
if the tests are not to be abused, to specify an adequate
number of replicates in order to determine whether the
product complies with the specified criteria. For the longer
term, however, there is a need to develop alternative
methods for maintaining test strains and preparing test sus-
pensions which achieve a more consistent resistance to bio-
cides and are, therefore, acceptable for use in analytical
standards. T h e ways in which the method of stock culture
preservation and the method of preparation of test suspen-
sions can affect the resistance of challenge inocula to anti-
microbial agents has been reviewed by Gilbert et al. (1987).
T h i s review suggests that by proper investigation it should
be possible to achieve greater consistency in the resistance
of test inocula to biocides.
ACKNOWLEDGEMENTS
T h i s work was funded by Reckitt and Colman Products.
W e also wish to thank Mr R. Hoare of Reckitt and Colman
Products for carrying out the statistical analyses and Dr A.
Brain of King’s College London for preparing the electron
micrograph.
REFERENCES
Anon. (1986) Terms Relating to Disinfectants. BS 5283. London :
British Standards Institution.
Anon. (1988) Method of Test f o r the Antimicrobial Activity of Dis-
infectants in Food Hygiene. DD 177. London: British Standards
Institution.
Anon. (1993a) Chemical Antiseptics and Disinfectants-Basic Bac-
tericidal Activity Test Method and Requirement. Provisional
European Norm, PrEN 1040. London : British Standards Insti-
tution.
Anon. (1993b) Chemical Antiseptics and Disinfectants-Basic Fun-
gicidal Activity Test Method and Requirement. Provisional Euro-
pean Norm. London : British Standards Institution (in press).
Bloomfield, S.F. and Looney, E. (1992) Evaluation of the repeata-
bility and reproducibility of European suspension test for anti-
microbial activity of disinfectants and antiseptics. Journal of
Applied Bacteriology 73, 87-93.
Bloomfield, S.F., Arthur, M., Looney, E., Begun, K. and Patel,
H. (1991) Comparative testing of disinfectants and antiseptic
products using proposed European suspension test methods.
Letters in Applied Microbiology 1, 233-237.
Bloomfield, S.F., Arthur, M., Begun, K. and Patel, H. (1993)
Comparative testing of disinfectants using proposed European
surface test methods. Letters in Applied Microbiology 17, 119-
125.
Gilbert, P., Brown, M.R.W. and Costerton, J.W. (1987) Inocula
for antimicrobial sensitivity testing : a critical review. Journal of
Antimicrobial Chemotherapy 20, 147-1 54.
Reybrouck, G. (1992) Evaluation of the antimicrobial activity of
disinfectants. In Principles and Practice of Disinfection, Sterilisa-
tion and Preservation, 2nd edn. Russell, A.D., Hugo, W.B. and
Ayliffe, G.A.J. pp. 114-133. Oxford: Blackwell Scientific Pub-
lications.
Van Klingeren, B. (1983) A Two-tier System for the Evaluation of
Disinfectants. Proefschrift, University of Utrecht, The Nether-
lands.
Van Klingeren, B., Leussink, A.B. and Van Wijngaarden, L.J.
(1977) A collaborative study on the repeatability and repro-
ducibility of the Dutch Standard-suspension-test for the evalu-
ation of disinfectants. Zentralblatt fur Bakteriologie,
Parasitenkunde, Infektionskrankheiten und Hygiene, I . Abt. Orig.
B164, 521-548.
Van Klingeren, B., Leussink, A.B. and Pullen, W. (1981) A Euro-
pean Collaborative Study on the Repeatability and the Repro-
ducibility of the Standard Suspension Test for the Evaluation of
Disinfectants in Food Hygiene. Report. no. 35901001. Bilthoven,
The Netherlands: National Institute of Public Health and
Environmental Hygiene.
Werner, H.P., Borneff, M. and Borneff, J. (1977) Development of
a new test method for surface disinfection procedures. Parts 1 -
6. Zentralblatt fur Bakteriologie, Parasitenkunde, Infektionsk-
rankheiten und Hygiene, I . Abt. Orig. B165, 1-94.