You are on page 1of 52

Systematic Literature Review of Oral Hygiene Practices for Intensive Care Patients Receiving Mechanical Ventilation

Background: Oropharyngeal colonization with pathogenic organisms contributes to the development

of ventilator-associated pneumonia in intensive care units. Although considered basic and potentially nonessential nursing care, oral hygiene has been proposed as a key intervention for reducing ventilatorassociated pneumonia. Nevertheless, evidence from randomized controlled trials that could inform best practice is limited.

Objective :To appraise the peer-reviewed literature to determine the best available evidence for
providing oral care to intensive care patients receiving mechanical ventilation and to document a research agenda for this important activity in optimizing patients outcomes.

Methods: Articles published from 1985 to 2006 in English and indexed in the CINAHL, MEDLINE,
Joanna Briggs Institute, Cochrane Library, EMBASE, and DARE databases were searched by using the key terms oral hygiene, oral hygiene practices, oral care, mouth care, mouth hygiene, intubated, mechanically ventilated, intensive care, and critical care. Reference lists of retrieved journal articles were searched for publications missed during the primary search. Finally, the Google search engine was used to do a comprehensive search of the World Wide Web to ensure completeness of the search. The search strategy was verified by a health librarian.

Results: The search yielded 55 articles: 11 prospective controlled trials, 20 observational studies, and
24 descriptive reports. Methodological issues and the heterogeneity of samples precluded metaanalysis.

Conclusions: Despite the importance of providing oral hygiene to intensive care patients receiving
mechanical ventilation, high-level evidence from rigorous randomized controlled trials or high-quality systematic reviews that could inform clinical practice is scarce.


The goals of this review were to evaluate peer-reviewed publications to determine the best available evidence for providing oral care to ICU patients receiving mechanical ventilation and to document a research agenda to improve patients outcomes. Method Approaches used to review the scientific literature range from a purposeful, systematic evaluation of rigorous studies to subjective overviews of descriptive articles.18(p53) Well-conducted systematic reviews can result in 3 major outcomes. First, increased power can be obtained by combining the effects of a number of smaller studies on the same topic when homogeneity allows meta-analysis. Second, systematic reviews to some extent enable the comparison of effects of studies with different designs.18(p53) Finally, a prospective and systematic review allows synthesis of the data and should assist in providing quality current evidence to guide clinical practice.19

Development of evidence-based guidelines is limited by the small number of randomized controlled trials and the variability of interventions studied.

Formulation of the review question requires extensive background research to enable an informed outcome. The question must accurately reflect the extent of the issue to be reviewed. Therefore, a comprehensive approach, including a wide-ranging search of the literature together with consultation with experts, including nurses, in the field of dental health and critical care resulted in the following review question: With respect to intensive care patients receiving mechanical ventilation, what is the best method for providing oral hygiene that will result in a reduction of colonization of dental plaque with respiratory pathogens?

Both experimental and nonexperimental study designs were included in the review. Because of the scarceness of review material on ICU patients receiving mechanical ventilation, articles that focused on specific oral care tools or solutions for the seriously ill also were included in the review.

This review considered studies that included patients in ICUs who were intubated and receiving mechanical ventilation. Also included were studies that proposed a link between oral hygiene and systemic diseases. The interventions of interest were those designed to affect dental plaque specifically and oral hygiene in general. The types of outcome measures considered were general and specific indicators of oral health:

Microbial counts

Plaque indices

Oral assessment scores

Validation of tools used in the provision of oral care

Articles were excluded if the study sample consisted of healthy participants or the study was done in a setting other than a critical care environment (eg, oncology).

Articles published from 1985 to 2006 in English and indexed in the following databases were searched: CINAHL, MEDLINE, Joanna Briggs Institute, Cochrane Library, EMBASE, DARE, and the World Wide Web search engine, Google. Key search terms used in the review were oral hygiene, oral hygiene practices, oral care, mouth care, mouth hygiene, intubated, mechanically ventilated, intensive care, and critical care. This search strategy was verified by a health librarian.

Full copies of articles considered to meet the inclusion criteria (on the basis of their title, abstract, and subject descriptors) were obtained for data synthesis. Articles identified through reference lists and bibliographic searches were considered for data collection depending on the titles. Articles were independently selected according to prespecified inclusion criteria by 3 reviewers, each with a minimum of a masters degree and certification in critical care. Discrepancies in the reviewers selections were resolved at meetings between the reviewers before the selected articles were included.

Until recently, one system used to grade levels of evidence was based on work by the US Agency for Healthcare Research and Quality. Because of the increasing awareness of the limitations of that system, however, the classification structure was revised by the Scottish Intercollegiate Guidelines Network. Therefore, the rating method used for categorization of levels of evidence found in this review was based on the revised system (Tables 1 and 2).20

View this table: In this window In a new window Table 1 Guide to the levels of evidence

View this table: In this window In a new window Table 2 Grades of recommendations

Previous Section Next Section Results

Although we found a number of references for the provision of oral hygiene in the management of oncology and other medical patients, most articles related to critical care were review articles. For the prospective randomized control trials we found, meta-analysis could not be used to synthesize the results because of variations in the methods of these studies. For example, in some studies, the populations assessed differed, and for those studies in which the populations were the same, the interventions were often dissimilar. These limitations were recognized in a recent meta-analysis on the use of chlorhexidine and the incidence of nosocomial pneumonia.21

Using the classification system developed by the Scottish Intercollegiate Guidelines Network, we reviewed 11 prospective controlled trials,3,4,13,14,2228 20 observational studies,15,2947 and 24 descriptive studies.21,4870 The 11 articles on prospective controlled trials are presented in Table 3. Summary tables of the observational studies (Table 4) and descriptive papers (Table 5) are available only on the American Journal of Critical Care Web site ( in the full-text view of this article. A review of food safety and food hygiene training studies in the commercial sector

M.B. Egan a , M.M. Raats a, , S.M. Grubb a , A. Eves b , M.L. Lumbers b , M.S. Dean a , M.R. Adams c a Food, Consumer Behaviour and Health Research Centre, University of Surrey, Guildford, Surrey GU2 7XH, UK b School of Management, University of Surrey, Guildford, Surrey GU2 7XH, UK c School of Biomedical and Molecular Sciences, University of Surrey, Guildford, Surrey GU2 7XH, UK Received 2 May 2005; received in revised form 2 August 2006; accepted 5 August 2006 Abstract This review summarises the methods and results of studies conducted worldwide on the eVectiveness of food safety and food hygiene

training in the commercial sector of the food industry. In particular it focuses on those studies that have tried to evaluate the eVectiveness of such training. Forty-six studies of food hygiene training are included which used some outcome measure to assess the eVectiveness of training. The short-term nature and variety of measures used limited the majority of studies. The need for the development of evaluation criteria of eVectiveness of food hygiene training is discussed. 2006 Elsevier Ltd. All rights reserved. Keywords: Food safety; Training; Evaluation; HACCP 1. Introduction Food safety remains a critical issue with outbreaks of foodborne illness resulting in substantial costs to individuals, the food industry and the economy (Kaferstein, Motarjemi, & Bettcher, 1997). Within England and Wales the number of food poisoning notiWcations rose steadily from approximately 15,000 cases in the early 1980s to a peak of over 60,000 cases in 1996 (Wheeler et al., 1999). This may be partly attributed to improved surveillance (GriYth, Mullan, & Price, 1995; Kaferstein & Abdussalam, 1999) but may equally reXect increased global trade and travel, changes in modern food production, the impact of modern lifestyles, changes in food consumption and the emergence of new pathogens (Collins, 1997; Tauxe, 1997). Recent years have seen a reversal in this trend but food poisoning remains a high priority for the public and government (Parliamentary OYce of Science & Technology, 2003). Mishandling of food plays a signiWcant role in the occurrence of foodborne illness. Improper food handling may be implicated in 97% of all foodborne illness associated with

catering outlets (Howes, McEwan, GriYths, & Harris, 1996). In two studies of general outbreaks of infectious intestinal disease (IID) in England and Wales the primary causes were related to poor food-handling practices (Djuretic, Ryan, & Wall, 1996; Evans et al., 1998). Improper practices responsible for microbial foodborne illnesses have been well documented (Bryan, 1988) and typically involve cross-contamination of raw and cooked foodstuVs, inadequate cooking and storage at inappropriate temperatures. Food handlers may also be asymptomatic carriers of food poisoning organisms (Cruickshank, 1990). Food handler training is seen as one strategy whereby food safety can be increased, oVering long-term beneWts to the food industry (Smith, 1994). A postal survey of manufacturing, retail and catering food businesses by Mortlock, Peters, and GriYth (2000) revealed that less than 10% had failed to provide some food hygiene training for staV. Less encouraging was the fact that less than 20% of managers * Corresponding author. Fax: +44 1483 689553. E-mail address: (M.M. Raats).M.B. Egan et al. / Food Control 18 (2007) 11801190 1181 were trained to supervisory level. This lack of training for food managers may restrict their ability to assess risks in their business and to assign appropriate hygiene training for their staV. The aim of this review is to analyse studies of food

hygiene training; in particular studies that have attempted to evaluate the eVectiveness of training. The studies will be evaluated principally on the outcome measures used in each study and their limitations for evaluating training eVectiveness discussed. 2. Background 2.1. Food hygiene legislation in the United Kingdom New food hygiene legislation has applied throughout the UK from 1st January 2006. Regulation 852/2004 (EC) of the European Parliament and Council on the Hygiene of Food StuVs now applies to all food businesses. The Food Hygiene (England) Regulations (2006); also come into force and separate but similar legislation will apply in Scotland, Wales and Northern Ireland. Article 5 states that: Food businesses operators shall put into place, implement and maintain a permanent procedure based on the principles of hazard analysis critical control points (HACCP). With regard to training Chapter XII states that food business operators are to ensure that: food handlers are supervised and instructed and/or trained in food hygiene matters commensurate with their work activity; that those responsible for the development and maintenance of the procedure referred to in Article 5 (1) of the Regulation or for the operation of the relevant guides have received adequate training in the application of HACCP principles, and compliance with any requirement of national law concerning training programmes for persons working in certain food sectors. Managers responsible for maintaining a food safety management system will require adequate training to enable them to carry out the statutory requirement. Accordingly a

new set of food safety qualiWcations will be launched in 2006 to help train managers and other staV in the essentials of food safety management systems. Furthermore Article 7 of Regulation (EC) 852/2004 on the hygiene of foodstuVs provides for the development of national Guides to Good Hygiene Practice and the Application of HACCP principles (known as Good Practice Guides). These guides are being developed by individual food sectors, in consultation with interested parties. Butchers shop licensing has been withdrawn across the UK from the end of 2005. Since 1st January 2006, all retail butchers are subject to the new EC hygiene regulations that apply to all other retail and catering businesses. 2.2. Hazard analysis and critical control points (HACCP) HACCP is an internationally recognised food safety assurance system that concentrates prevention strategies on known hazards; it focuses on process control, and the steps within that, rather than structure and layout of premises (Kirby, 1994; Worsfold & GriYth, 1995). HACCP establishes procedures whereby these hazards can be reduced or eliminated and requires documentation and veriWcation of these control procedures (Codex, 1997). Whilst HACCP has been widely adopted by the food manufacturing industry and the larger companies in the hospitality and catering sector, there have been concerns about implementation by smaller businesses. Barriers to the implementation of HACCP in small businesses have been identiWed which include lack of expertise, absence of legal requirements,

Wnancial constraints and attitudes (Ehiri, Morris, & McEwen, 1995; Taylor, 2001; Walker, Pritchard, & Forsythe, 2003; WHO, 1999). 2.3. Training and evaluation of training eVectiveness The Manpower Services Commission (1981) deWned training as a planned process to modify attitude or skill behaviour through learning experience to achieve eVective performance in an activity or range of activities. Evaluation is integral to the cycle of training, providing feedback on the eVectiveness of the methods used, checking the achievement of the objectives set by both the trainer and trainee and in assessing whether the needs originally identiWed have been met (Bramley, 1996). Criteria that may be used for evaluating the eVectiveness of a training programme include reaction to training, knowledge acquisition, changes in job-related behaviour and performance and improvements in organisational-level results (Kirkpatrick, 1967). Research on training eVectiveness has focused primarily on factors that are directly related to training content, design and implementation (Tannenbaum & Yuki, 1992). However other factors outside the training environment may inXuence the eVectiveness of any programme (Tracey, Tannenbaum, & Kavanagh, 1995). Despite general acceptance that training eVorts must be systematically evaluated, few studies have tried to identify the beneWts food hygiene training brings to the industry. This is illustrated by a survey of the US lodging industry where fewer than 10% of the hospitality companies conducted formal evaluations of their training programmes

(Conrade, Woods, & Ninemeir, 1994). 2.4. Transfer of knowledge into practice To be eVective food hygiene training needs to target changing those behaviours most likely to result in foodborne illness. Most food hygiene training courses rely heavily on the provision of information. There is an implied assumption that such training leads to changes in behaviour, based on the Knowledge, Attitudes and Practices (KAP) model. This model has been criticised for its limitations (Ehiri, Morris, & McEwen, 1997b; GriYth, 2000). It is accepted that knowledge alone is insuYcient to trigger preventive practices and that some mechanism is needed to motivate action and generate positive attitudes1182 M.B. Egan et al. / Food Control 18 (2007) 11801190 (Tones & Tilford, 1994). In an evaluation of food hygiene education Rennie (1994) concluded that knowledge alone does not result in changes in food handling practices. Various studies have shown that the eYcacy of training in terms of changing behaviour and attitudes to food safety is questionable (Mortlock, Peters, & GriYth, 1999). 3. Overview of studies 3.1. Mapping exercise The aim of the review was to identify criteria used by previous studies to evaluate the eVectiveness of food safety and hygiene training. Reports referring to training in the context of food safety training or food hygiene training in the commercial sector were considered relevant. Only those

studies written in the English language were included. DiVerent sources of published and unpublished research literature were searched to locate relevant papers. Searches were conducted on commercially available electronic databases including PsycINFO, Medline, ERIC, CINAHL, Social Science Citation Index, Science Direct, etc. These searches covered the full range of publication years available in each database at the time of searching. For all the databases the following search strategy was used: [{Food safety} or {Food hygiene}] and [{train*} or {teach*} or {course*} or {educat*}]. All citations identiWed by these searches were downloaded and when possible captured and compiled as a Reference Manager database. Further studies were identiWed through hand searching journals and references to publications in retrieved papers. 3.2. General characteristics of relevant studies included in review Studies were included in the review if they met two criteria: The study used some outcome measures to assess the eVectiveness of food hygiene training. The study was based in a commercial setting. A total of forty-six studies of food hygiene training were retained and included in this review. Full details of the included studies are given in Tables 15. The earliest study

in this review was undertaken in 1969 and the most recent in 2003. Fifteen studies (32%) were from the UK, twenty (43%) from the USA, two (4%) each from Canada, Italy and Malaysia and one (2%) each from Australia, Bahrain, New Zealand, Nigeria and Saudi Arabia. Thirty studies (65%) involved food handlers, 11 (24%) focused on food managers and one study involved both (Burch & Sawyer, 1991). The level of training was not speciWed in the majority of the studies, the UK studies generally used standard basic food hygiene courses. Twenty-two studies (48%) included a training intervention (Tables 1 and 35). Twenty-nine studies (63%) measured knowledge (Tables 1 and 35). Most studies addressed attitude, behaviour and work practices concerning food safety and food hygiene in some form, however the methods varied greatly. Only four of the studies (Ehiri et al., 1997b; Laverack, 1989; Reicks, Bosch, Herman, & Krinke, 1994; Tracey & Cardenas, 1996) make reference to a social cognition theoretical model as a basis for their study. 3.3. Training interventions Of the 22 studies involving a training intervention, 15 were from North America, Wve from the UK and one from Bahrain. North American training included courses for food handlers and food service managers. It also encompassed diVerent types of training such as home study, workshops and more formal courses. A number of those studies also compared the results of using diVerent methods of delivering training. Seventeen of the 22 studies used a knowledge measure to evaluate the eVectiveness of the

intervention, most commonly a pre- and post-test. 3.4. Study design and theoretical models Few of the reports speciWed their study design. For clarity we have attributed each to one of the Wve evaluation designs detailed by Ovretveit (1998). A brief description of each of the designs is given here: (i) Descriptive: Evaluator observes and selects features of the intervention, which he or she describes. Twentysix of the 46 studies reviewed fell into this category (Table 1). (ii) Audit: Evaluator compares what the service does with what it should or was intended to do, according to written standards or procedures. Three (Audit Commission, 1990; Holt & Henson, 2000; Morrison, CaYn, & Wallace, 1998) of the 46 reviewed fell into this category (Table 2). (iii) Beforeafter: Evaluator compares a group of participants before and after an intervention. Seven of the 46 studies reviewed fell into this category (Table 3). (iv) Comparative-experimentalist: Evaluator compares the outcomes of two groups undergoing diVerent interventions. Five (Costello, Gaddis, Tamplin, & Morris, 1997; Howes et al., 1996; Kirby & Gardiner, 1997; Nabali, Bryan, Ibrahim, & Atrash, 1986; Rinke, Brown, & McKinley, 1975) of the 46 studies reviewed fell into this category (Table 4). (v) Randomised controlled experimental: Evaluator compares one group that receives an intervention with another group that does not, but that is in all other possible respects the same. Five (Ehiri et al., 1997b;

Reicks et al., 1994; SoneV, McGeachy, Davison, McCargar, & Therien, 1994; Waddell & Rinke, 1985; Wright & Feun, 1986) studies fell into this category (Table 5).M.B. Egan et al. / Food Control 18 (2007) 11801190 1183 Table 1 Food hygiene training evaluation studies using a descriptive design Study and year Country Participants (number) Training intervention Knowledge Attitude, behaviour and working practices Al-Dagal (2003) Saudi Arabia Sanitarians (n D 82) None Questionnaire Self-reported practices Burch and Sawyer (1991) USA Managers (n D 13) and employees (n D 27) of 13 convenience stores None Sanitation quiz (8 questions) Researcher survey Angellilo et al. (2000) Italy Food handlers (n D 411) None Face-to-face interviews using structured questionnaire Self-reported hygiene practices Angellilo et al. (2001) Italy Food handlers (n D 290) in hospitals (n D 36) None Questionnaire Questionnaire Clayton et al. (2002) UK Food handlers (n D 137)

from 52 food SMEs None Not assessed Questionnaire, self-reported practices Cook and Casey (1979) USA Food service managers NIFI course, over a 5-week period Written examination Comparison of post-course sanitation inspection scores Hart et al. (1996) USA Beef demonstrators (n D 93) National Restaurant Association SERVSAVE programme Pre and post-training questionnaires Pre- and post-training questionnaires Hennum et al. (1983) USA Restaurants (n D 16) None Interview Interview and observation Hine et al. (2003) USA Restaurant managers (n D 500) None Not assessed Survey Johnston et al. (1992) New Zealand Managers of food service outlets (n D 300) None Not assessed Questionnaire Little et al. (2002) UK Take-away restaurants and sandwich bars None Not assessed Microbiological study

Manning and Snider (1993) USA Food handlers (n D 64) None Questionnaire Questionnaire, observation checklist McElroy and Cutter (2004) USA Participants (n D 1,448) in Statewide Food safety CertiWcation program (SFSCP) Food safety workshop (16hrs) Not assessed Self-reported changes in food safety behaviours assessed by questionnaire Oteri and Ekanem (1989) Nigeria Hospital food handlers (n D 161) None Not assessed Structured interview, observation of some practices Powell et al. (1997) UK StaV in 30 food premises CIEH basic certiWcate in food hygiene Basic food hygiene certiWcate examination Frequency inspection ratings Sumbingco et al. (1969) USA Food service employees (n D 11) of university residence halls

Programmed texts for two food service tasks Oral test Quality of work assessed, time for doing tasks measured Tebbutt (1986) UK Premises selling sliced cooked meats (n D 160) None Not assessed Microbiological sampling, questionnaire on cleaning and disinfection Tebbutt (1991) UK StaV in 89 restaurants None Multiple choice questions Premises assessed Tebbutt (1992) UK StaV in 75 premises producing high-risk foods None Multiple-choice questions Numerical scores for premises based on 20 variables Toh and Birchenough (2000) Malaysia Food hawkers (n D 100) from 15 sites None Structured on-site interview Thirteen attitude items

using a Likert scale Wade (1998) UK Hospitality managers (n D 27) None Not assessed Survey of hygiene management Walker et al. (2003) UK Food handlers (n D 444), from 104 small food businesses None Multiple-choice questions Not assessed Worsfold (1993) UK Members of the Womens Royal Voluntary Service (n D 93) Royal Society of Health Basic Food hygiene course Pre-course questionnaire End-of-course evaluation Worsfold and GriYth (2003) UK Small or medium sized businesses handling high-risk foods (n D 66) None Not assessed Semi-structured interview with manager; observation of hygiene practices Wyatt (1979) USA Managers or owners of food markets (n D 219)

None Questionnaire Questionnaire on attitudes, opinions, experiences and practices Zain and Naing (2002) Malaysia Food handlers (n D 430) None Questionnaire Questionnaire evaluating attitude and practice1184 M.B. Egan et al. / Food Control 18 (2007) 11801190 3.5. Outcome measures One of the aims of the review was to identify criteria for evaluating the eVectiveness of food safety and hygiene training. Evaluation of training is complex given the number of variables that may inXuence the outcome, including who is being trained, the level of training, motivation and cultural dimensions. Unfortunately few of the studies were Table 2 Food hygiene training evaluation studies using an audit design Study and year Country Participants (number) Training intervention Knowledge Attitude, behaviour and working practices Audit Commission (1990) UK Food premises (n D 5,000) None Not assessed Survey of premises and working practices Holt and Henson (2000) UK Manufacturers of ready to eat meat products (n D 24) None Not assessed Hygiene audit using

EFSIS protocol Morrison et al. (1998) Australia Food service operations (n D 19) None Not assessed Survey checklist us for practices observed and standards measured Table 3 Food hygiene training evaluation studies using a before-after design Study and year Country Participants (number) Training intervention Knowledge Attitude, behaviour and working practices Cotterchio et al. (1998) USA 3 groups of trainee restaurant managers (n D 96) Food manager training and certiWcation programme Not assessed Routine sanitary inspection scores compared pre- and post-training Kneller and Bierma (1990) USA Food service facilities (n > 400) None Not assessed Review of pre- and postcertiWcation inspection scores Laverack (1989) UK Food handlers IEHO Basic Food Hygiene

Course Pre- and post-training tests Questionnaire pre and posttraining Medeiros et al. (1996) USA Food safety educators (n D 45) and voluntary cooks (n D 136) Safe food handling for occasional cooks training programme Pre- and post-course test of 55 questions Self-declared behaviour checklist used at time of initial training Palmer et al. (1975) USA Food service managers in 31 takeout restaurants Manager training programme (2 2 h sessions) Not assessed Before and after survey of premises, total demerit score awarded Sparkman

et al. (1984) USA Food service workers (n D 23) Food service training manual, 3 h training session Pre- and post-test with 21 multiple-choice questions post-training On-the job performance evaluation with 30 observations Tracey and Cardenas (1996) USA Dining services division of two private colleges (n D 76) Two food-service safety training programmes Pre- and post-training tests based on course training materials Pre-training motivation assessed by survey, reactions to training surveyed immediately post-training Table 4

Food hygiene training evaluation studies using a comparative-experimentalist design Study and year Country Participants (number) Training intervention Knowledge Attitude, behaviour and working practices Costello et al. (1997) USA Employees of 6 quick service restaurants (n D 43) Two teaching methodslecture format or computer interactive method Questionnaire25 multiple-choice questions; pre- and post-training tests Not assessed Howes et al. (1996) Canada Food handlers (n D 69) Home study food-handler certiWcation course Pre- and post-instruction tests using 50 multiple-choice questions Pre-observation of 16 food safety practices; postobservation of two hand

washing practices Kirby and Gardiner (1997) UK StaV in 30 food premises CIEH basic certiWcate in food hygiene Not assessed Pre- and post-training hygiene audit for each premise Nabali et al. (1986) Bahrain Food service managers in 24 premises Manager training programme (2 2.5 days sessions) Pre- and post-course test of 50 questions Pre- and post-course inspection surveys of premises Rinke et al. (1975) USA Food production personnel in university residence halls (n D 60) Training program, presented

as live instruction or taped instruction Pre- and post-training testing Not assessedM.B. Egan et al. / Food Control 18 (2007) 11801190 1185 similar enough to allow any direct comparisons. Data were collected using a variety of research methods. These included self-completed questionnaires, face-to-face interviews, premises inspections, observation and microbiological sampling. We identiWed four outcome measures that could be used to compare the studies: knowledge; attitudes, behaviour and work practices; retraining and duration of eVects. Our evaluation is based on these measures. 4. Results 4.1. Knowledge Assessment of knowledge featured in 29 (63%) of the studies reviewed here (Tables 1 and 35). Questionnaires were used as the principal measure of knowledge. Generally these were of multiple-choice format with the number of questions varying from 8 (Tracey & Cardenas, 1996) to 55 (Medeiros et al., 1996), but some (e.g. Wright & Feun, 1986) providing no detail. Few of the studies detailed the questions used, referring only to the general topics covered. These included high-risk foods, foodborne pathogens, cross-contamination, personal hygiene, temperature control and cleaning. A number of the studies (Costello et al., 1997; Hart, Kendall, Smith, & Taylor, 1996; Laverack, 1989; Medeiros et al., 1996; Nabali et al., 1986; Reicks et al.,

1994; Taylor, 1996; Tracey & Cardenas, 1996; Wright & Feun, 1986) involved interventions using pre- and posttraining tests of knowledge. Nine studies (Costello et al., 1997; Hart et al., 1996; Howes et al., 1996; Medeiros et al., 1996; Nabali et al., 1986; Reicks et al., 1994; Sparkman, Briley, & Gillham, 1984; Tracey & Cardenas, 1996; Waddell & Rinke, 1985) found statistically signiWcant improvements in the test scores of the intervention groups, whilst a further two (Laverack, 1989; Wright & Feun, 1986) measured some improvement. Two studies (Ehiri et al., 1997b; Reicks et al., 1994) found a signiWcant diVerence between the intervention and control group. Only one intervention (Powell, Attwell, & Massey, 1997) measured no signiWcant diVerence in post-training scores. The results from those studies not involving any intervention also varied. These were frequently based on questionnaires and results ranged from good knowledge through to poor knowledge in critical aspects of food safety. Generally there was good awareness of common food pathogens (Al-Dagal, 2003; Angellilo, Viggiani, Greco, Rito, & the Collaborative group, 2001), but poor knowledge of temperature control, especially regarding reheating and cooling (Manning & Snider, 1993; Zain & Naing, 2002). In summary it is very diYcult to make any direct comparisons as the studies were all conducted in diVerent ways, involving diVerent tests. Of the 21 studies where a training intervention was included, four (Cotterchio, Gunn, CoYll,

Tormey, & Barry, 1998; Kirby & Gardiner, 1997; Palmer, Hatlen, & Jackson, 1975; SoneV et al., 1994) did not use any knowledge tests to evaluate the training. 4.2. Attitudes, behaviour and work practices Very few of the studies reviewed included any detailed investigation of attitude, a cognitive element that may inXuence food safety behaviour and practice. Again any direct comparison of results is diYcult because of the disparity of the measures used in the reported studies. The means of evaluating attitudes, behaviour and work practices fall into two broad categories, namely surveys or inspections of premises and structured questionnaires. Seventeen of the studies included a premises survey or observation of behaviour, and Wve studies (Cook & Casey, 1979; Cotterchio et al., 1998; Kneller & Bierma, 1990; Powell et al., 1997; Wright & Feun, 1986) used routine inspection scores. In some instances this was the sole measure of behaviour and hygiene practices. The exact range of the surveys varied but usually included inspection of physical facilities and assessments of cleaning procedures, personal hygiene and Table 5 Food hygiene training evaluation studies using a randomised controlled experimental design Study and year Country Participants (number) Training intervention Knowledge Attitude, behaviour and working practices Ehiri et al. (1997b) Scotland Intervention group (n D 188) and comparison group (n D 204) who receive no training REHIS elementary food

hygiene course Self-administered test of 20 questions Not assessed Reicks et al. (1994) USA Leaders of home study groups (n D 97) Food safety instruction (2 h lesson) Thirteen multiple choice questions, pre- and post-instruction Pre- and post-instruction evaluation of attitudes to food safety, using 5-point Likert scale SoneV et al. (1994) Canada StaV at 46 community based adult care facilities Training workshop plus manual, manual only or no intervention Not assessed Pre- and post-training assessment of staV practices Waddell and Rinke (1985) USA Food service employees

(n D 230) at large military hospital Computer assisted training (CAI) and lecture method of instruction (LMI) Pre- and post-test questionnaire, 33 questions Questionnaire to assess attitude to training using Likert scale Wright and Feun (1986) USA Food service managers (n D 54); study group (n D 27) and control group (n l D 27) NIFI training programme Pre- and post-tests used Pre-inspection of premises; two post-inspections soon after course1186 M.B. Egan et al. / Food Control 18 (2007) 11801190 temperature control. Quite often premises were assigned an inspection score, most usually when the study involve a preand post-training inspection. Questionnaires or interviews were used to document self-reported food hygiene practices and attitudes to food hygiene and training. Attitudes were measured most commonly using a 5-point Likert scale. Less frequently studies incorporated researcher observation of food safety practices on site. In one study (Howes et al., 1996) 16 practices

were observed prior to training but this was reduced to two practices post-training because of inherent diYculties with completing these observations. In three studies (Little, Barnes, & Mitchell, 2002; Tebbutt, 1986; Tebbutt, 1991) microbiological sampling was used as a measure of eVectiveness. A number of interesting results do emerge. The majority of food handlers and managers expressed a positive attitude to food safety but this was not supported by selfreported practices (e.g. Angellilo, Viggiani, Rizzio, & Bianco, 2000). Furthermore some studies have demonstrated the discrepancy between self-reported behaviour and observed or actual behaviour (Clayton, GriYth, Price, & Peters, 2002; Oteri & Ekanem, 1989). In studies using inspections/surveys of premises four of the Wve studies using routine inspection scores (Cook & Casey, 1979; Cotterchio et al., 1998; Kneller & Bierma, 1990; Wright & Feun, 1986) found a signiWcant improvement in post-training inspection scores. In the Cook & Casey study however the improved inspection score was not signiWcantly higher than that of control establishments. Other studies using inspections (e.g. Kirby & Gardiner, 1997) reported no signiWcant improvements. Furthermore there seemed to be no correlation between knowledge test scores and hygiene inspection scores (e.g. Cook & Casey, 1979; Powell et al., 1997). In one UK study (Little et al., 2002) the presence of a trained manager improved food safety procedures and in

one US study (Cotterchio et al., 1998) the mandatory attendance of managers resulted in improved inspection scores. A poor correlation emerged between microbiological examinations and visual inspections (Tebbutt, 1986, 1991). 4.3. Retraining and duration of eVects Retraining or refresher training featured in only four studies (Holt & Henson, 2000; Tebbutt, 1991, 1992; Worsfold & GriYth, 2003). Three studies (Holt & Henson, 2000; Tebbutt, 1992; Worsfold & GriYth, 2003) checked on the frequency of refresher training whilst the fourth study (Tebbutt, 1991) assessed the management attitude to retraining as part of an interview. In one UK study (Tebbutt, 1992), 41% of the businesses involved oVered very limited or no retraining whilst a more recent UK study (Worsfold & GriYth, 2003) reported very little refresher training being carried out. Thirteen studies included some measure of the impact of training over time. The time period used for those studies (Costello et al., 1997; Hart et al., 1996; Laverack, 1989; Medeiros et al., 1996; SoneV et al., 1994; Sparkman et al., 1984; Sumbingco, Middleton, & Konz, 1969; Worsfold, 1993; Wright & Feun, 1986) ranged from one week to six months. When an inspection was the measure, the time period increased up to Wve years (Cotterchio et al., 1998; Kneller & Bierma, 1990; Wright & Feun, 1986). Two US studies (Cotterchio et al., 1998; Kneller & Bierma, 1990) reported that improvements in inspection scores were sustained for 1824 months and only began to decline after

three years. However another study from the US reported a reduction in post-training performance after only eight weeks (Sparkman et al., 1984) whilst another found reduced inspection scores after six months (Wright & Feun, 1986). 4.4. HACCP training studies A search of the food safety literature identiWed only Wve studies that involved some evaluation of HACCP training, these included a HACCP training programme in the Lithuanian dairy industry (Boccas et al., 2001), a survey of HACCP implementation in Glasgow (Ehiri, Morris, & McEwen, 1997a) and an evaluation of a short HACCP course involving representatives from residential care homes (Worsfold, 1998). A lack of food hygiene knowledge by staV was identiWed as the greatest problem in a study of the application of HACCP in a Xight catering establishment (Lambiri, Mavridou, & Papadakis, 1995). Training was seen as critical in order to assess hazards and control food safety in the long-term. In a study of cleaning standards and practices in 1502 food premises in the UK (Sagoo, Little, GriYth, & Mitchell, 2003a), deWciencies were associated with premises that did not have management food hygiene training or hazard analysis. Education and training is crucial in implementing any HACCP system. This has been recognised previously by both the Codex (1997) and NACMCF (1998) and is now more relevant in

the UK with the introduction of new hygiene legislation. 5. Discussion and conclusions This review particularly focused on studies that attempted to evaluate the eVectiveness of food safety and hygiene training. Other reviews of eVective food safety training support many of the Wndings from the studies examined here. A number of reviews (Riben, Mathias, Campbell, & Wiens, 1994; Riben, Mathias, Wiens, et al., 1994; Mathias et al., 1994) undertook critical appraisals of the literature relating to food safety education in Canada, focusing on routine restaurant inspections and education of food handlers. They identiWed thirteen studies but many were weak, lacking in methodological detail and with poorly deWned outcomes. They concluded that training had an impact on examination scores and restaurant inspection scores in the short-term. It was impossible to deWne a particular educational intervention as most eVective due to diVerences inM.B. Egan et al. / Food Control 18 (2007) 11801190 1187 those used and lack of controls. In a follow up study, Mathias, Sizto, Hazelwood, and Cocksedge (1995) examined the eVects of inspection frequency and food handler education on restaurant inspection violations. Those restaurants with any staV having food handler education did signiWcantly better on the overall inspection score than those with staV who had no such education. In a further review of the eVectiveness of Canadian public health interventions in food safety 15 studies were examined and three categories of interventions identiWed

(Campbell et al., 1998). These were inspections, food handler training and community-based education. Once again there was some evidence that interventions can result in improved food safety, but the authors emphasize that because of diVerences in protocols many studies are not useful in establishing guidelines. This work was recently extended (Mann et al., 2001) and now includes 55 papers of which seven were rated as moderate and 48 rated as weak. The authors concluded that four of the seven studies provided good evidence to support the eVectiveness of the food safety interventions with positive results for the main outcome measured. Five of the seven studies included in the review focused on food handler training and or certiWcation, three of which (Cotterchio et al., 1998; Rinke et al., 1975; Waddell & Rinke, 1985) provided evidence for the eVectiveness of the intervention. Several other reviews have attempted to identify the key features of an eVective training programme (Sprenger, 1991; Rennie, 1994; Taylor, 1996). Training in the workplace is one such feature. Current training is often conducted away from the workplace and there may be diYculties in translating theory into improved food handling. Rennie (1994) concluded that the need for improvements in food handling practices might be better served by training in the workplace, allowing for practical reinforcement of the hygiene message. Taylor (1996) reiterated this, arguing that the impact of food handler training is minimal and would be more eVective if conducted in the workplace, where it can be job speciWc. She cites the minimal eVect of training courses on knowledge, attitudes and behaviour of food handlers as well as their inability to inXuence operational practices.

Another critical issue is that of eVective management training. In an evaluation of a fast food management training programme Jackson, Hatlen, and Palmer (1977), concluded that management training can be eVective if it is administered on a continuous basis, supported by the owners and includes frequent follow-up. Nabali et al. (1986), concluded from their study that training of managers was eVective in improving hygiene standards. A microbiological study of open, ready-to-eat, prepared salad vegetables from retail or catering premises by Sagoo, Little, and Mitchell (2003b), identiWed a direct relationship between food hygiene training of management, increased conWdence in the food business management and the presence of food safety procedures. In a study of ready-to-eat stuYng from retail premises in the northeast of England, Richardson and Stevens (2003) suggested that poor microbiological quality of product might be related to management food hygiene training and conWdence in management scores. Sprenger (1991) argues that prioritising the training of managers may be more important than that of basic food handlers. The numerous beneWts of management training include the ability of managers to inXuence premises hygiene, less turnover of managers and their impact on the training of staV. Whereas training food handlers has had minimal impact, training managers may be more cost eVective, premises hygiene is more within their remit and managers can self-inspect and train employees (Taylor, 1996). However a previous study of 300 professionally qualiWed catering

managers does not support many of these assertions (Taylor, 1994). The results suggested that trained managers did not put their theoretical knowledge into practice or alternatively did not possess the knowledge that their qualiWcation should have delivered. Ultimately the training did not result in the implementation of critical food safety practices in the workplace. A further issue to arise from US studies is that of mandatory training. Penninger and Rodman (1984) addressed this issue by determining the eVectiveness of both voluntary and mandatory food service managerial certiWcation training programs in a limited random study. Mandatory programmes were more successful in certifying managers than voluntary programs with 91% of mandatory agencies claiming improvement in inspection scores, compared to 33.3% of voluntary agencies. However these Wndings were based on a response rate of less than 35% of the agencies surveyed. In an evaluation of the Ohio Food service manager certiWcation course (Clingman, 1976), there was a 5.5% improvement in overall sanitation level for those restaurants whose managers had been certiWed. This compared to a 3.3% improvement for restaurants whose mangers were not certiWed. This study also noted that management turnover in non-certiWed manager establishments was 29.7%, whereas that in certiWed manager establishments was 19.5% over the study period. Similarly, the voluntary nature of the Minnesota Quality Assurance Programme for the Prevention of Foodborne Illness reduced its eVectiveness (Heenan & Synder, 1978). Burch and Sawyer (1991), also recommended mandatory training based on their Wnding that the

sanitary condition of stores was closely associated with the food safety knowledge of management. The importance of training food handlers is acknowledged by many as critical to eVective food hygiene yet there have been limited studies on the eVectiveness of such training. Many of the authors recommend approaches that may result in improved food handling practices. Rennie (1995) suggests that behavioural change would be more likely if the settings approach to health promotion were adopted in food premises. Studies involving an assessment of food hygiene training were included in this review. It is very diYcult to make any direct comparisons between studies because of the varied designs and outcome measures used. Within the various1188 M.B. Egan et al. / Food Control 18 (2007) 11801190 measures there was much variation and incomplete reporting of the details of the intervention methods and outcome measures. For example, few studies reported details of the questionnaires used. The majority of the studies were only short-term and while these can provide useful information, longer-term interventions and evaluations are needed to assess behavioural change. Another limitation of the studies involving interventions was the lack of information on costs or cost-eVectiveness, an issue often cited as a barrier to training. The limitations of measures used to assess training interventions are further discussed by Ehiri and Morris (1996).

They found the use of pre- and post-training test scores limited for evaluation purposes as they often measure knowledge of items not reXected in behaviour change. They also highlighted the lack of correlation between examination scores and improvements in food safety by reference to the studies of Luby, Jones, and Horan (1993) and Laverack (1989). The use of food hygiene inspection scores as a means of evaluation is limited by the lack of correlation between training and inspection scores. It is worth noting also that these studies were based solely in the commercial sector. Commercial food safety is very dependant on organisational structures and cannot easily be related to individual behaviour. EVorts to reduce the incidence of foodborne illness through interventions have had mixed results. The focus of interventions, in the commercial sector, has been on improving food handling practices. A primary aim and therefore a primary criterion for evaluation of any training is a change in behaviour towards less risky food handling practices. A related goal is to improve knowledge about food safety practices such as cross-contamination, temperature control and personal hygiene. Related to these is the issue of measuring outcomes. The principal conclusions of this review of the literature are: Current evidence for the eVectiveness of food hygiene training is limited. This review has shown that many of the studies on food hygiene training are limited both by a lack of methodological detail and of well deWned outcomes,

Comparisons between studies are restricted and it becomes impossible to deWne eVective interventions due to these diVerences. There is a need to identify meaningful performance indicators at an individual level that can be used to measure the eVectiveness of food hygiene training. Questionnaires are a convenient measure of knowledge and attitudes but direct observation has limited value. Reliable data from the workplace is essential to develop, implement and evaluate eVective food hygiene training, however information on food hygiene behaviours obtained by direct observation has limited value. Such observations are usually restricted to a small number of practices because of the variety and complexity of roles involved in food handling. StaV may also exhibit altered behaviours in the presence of the observer to present what is perceived to be a more desirable image. There are also practical considerations in relation to time and cost involved in such observations. Evaluation of training is essential and factors other than training content and design are important. It is necessary to look beyond the training context to understand how and why training does or does not work. Issues such as managerial support, the availability of equipment and tools, training and pre-training motivation can all inXuence the extent to which individuals react to the training experience. Training outcomes will also be inXuenced by a host of other factors, both organisational and individual. These may include cultural dimensions, legislation, environmental. Training of managers can be eVective in reducing food safety problems. The training of managers is seen by many as a necessary precursor to the implementation of realistic food safety practices within the workplace. If managers were trained to advanced levels they would then provide

basic training for food handlers in-house and make training more sector speciWc. The eVectiveness of training is very dependent on both management attitude and their willingness to provide the resources and systems for food handlers to implement good practice. Evidence from the literature suggests that food hygiene training as a means of improving food safety standards is limited by a lack of understanding of those factors contributing to successful outcomes. There is a need to develop training methods that are proven to change behaviour as well as imparting knowledge. Further research is needed on issues including course content, the site of training, duration of courses and refresher training. Such research needs to be clearly thought out, well designed with good baseline data to achieve worthwhile results. Acknowledgement This study has been carried out with Wnancial support from the Food Standards Agency. It does not necessarily reXect its views and in no way anticipates the Agencys future policy in this area. References Al-Dagal, M. M. (2003). Food safety knowledge and behaviour of Sanitarians of Riyadh Municipality (Saudi Arabia). Food Protection Trends, 23(2), 142148. Angellilo, I. F., Viggiani, N. M. A., Greco, R. M., Rito, D., & the Collaborative group (2001). HACCP and food hygiene in hospitals: knowledge, attitudes, and practices of food-service staV in Calabria, Italy. Infection Control and Hospital Epidemiology, 22(6), 363369.

Angellilo, I. F., Viggiani, N. M. A., Rizzio, L., & Bianco, A. (2000). Food handlers and foodborne diseases: knowledge, attitudes and reported behaviour in Italy. Journal of Food Protection, 63(30), 381385. Audit Commission (1990). Environmental Health Survey of Food Premises (Rep. No. 2). Boccas, F., Ramanauskas, A., Boutrif, E., Cavaille, P., Lacaze, J. M., & Powell, S. C. (2001). HACCP train-in-action program in the Lithuanian dairy industry. Food Control, 12(3), 149156. Bramley, P. (1996). Evaluating training eVectiveness: translating theory into practice. London: McGraw-Hill. Bryan, F. L. (1988). Hazard analysis critical control point: what the system is and what it is not. Journal of Environmental Health, 50(7), 400407.M.B. Egan et al. / Food Control 18 (2007) 11801190 1189 Burch, N. L., & Sawyer, C. A. (1991). Food handling in convenience stores. Journal of Environmental Health, 54(3), 2327. Campbell, M. E., Gardner, C. E., Dwyer, J. J., Isaacs, S. M., Krueger, P. D., & Ying, J. Y. (1998). EVectiveness of public health interventions in food safety: a systematic review. Canadian Journal of Public Health, 89(3), 197202. Clayton, D. A., GriYth, C. J., Price, P., & Peters, A. C. (2002). Food handlers beliefs and self-reported practices. International Journal of Environmental Health Research, 12(1), 2539. Clingman, C. (1976). Ohio evaluated eVects of food certiWcation training. Journal of Environmental Health, 38, 235236. Codex Alimentarius Commission (1997). Report of the 30th session of the Codex Commission on Food Hygiene, ALINORM 99/13 Rome: Codex Alimentarius Commission.

Collins, J. E. (1997). Impact of changing consumer lifestyles on the emergence of and re-emergence of foodborne pathogens. Emerging Infectious Diseases, 3(4), 471479. Conrade, G., Woods, R. H., & Ninemeir, J. D. (1994). Training in the US lodging industry: perception and reality. Cornell Hotel and Restaurant Administration Quarterly, 35(5), 1621. Cook, C. C., & Casey, R. (1979). Assessment of a food service management sanitation course. Journal of Environmental Health, 41(5), 281284. Costello, C., Gaddis, T., Tamplin, M., & Morris, W. (1997). Evaluating the eVectiveness of two instructional techniques for teaching food safety principles to quick service employees. Journal of Food Service Systems, 10(1), 4150. Cotterchio, M., Gunn, J., CoYll, T., Tormey, P., & Barry, M. A. (1998). EVect of a manager training program on sanitary conditions in restaurants. Public Health Reports, 113(4), 353358. Cruickshank, J. G. (1990). Food handlers and food poisoning: training programmes are best. British Medical Journal, 300(6719), 207208. Djuretic, T., Ryan, M. J., & Wall, P. G. (1996). The cost of inpatient care for acute infectious intestinal disease in England from 1991 to 1994. CDR Review, 6, R78R80. Ehiri, J. E., & Morris, G. P. (1996). Hygiene training and education of food handlers: does it work? Ecology of Food and Nutrition, 35(4), 243251. Ehiri, J. E., Morris, G. P., & McEwen, J. (1995). Implementation of HACCP in food businesses: the way ahead. Food Control, 6(6), 341 345. Ehiri, J. E., Morris, G. P., & McEwen, J. (1997a). A survey of HACCP implementation in Glasgow: is the information reaching the target?

International Journal of Environmental Health Research, 7(1), 7184. Ehiri, J. E., Morris, G. P., & McEwen, J. (1997b). Evaluation of a food hygiene training course in Scotland. Food Control, 8(3), 137147. Evans, H. S., Madden, P., Douglas, C., Adak, G. K., OBrien, S. J., Djuretic, T., et al. (1998). General outbreaks of infectious intestinal disease in England and Wales: 1995 and 1996. Communicable Disease and Public Health, 1(3), 165171. GriYth, C. J., Mullan, B., & Price, P. E. (1995). Food safety: implications for food medical and behavioural scientists. British Food Journal, 97(8), 2328. GriYth, C. J. (2000). Food safety in catering establishments. In J. M. Farber & E. C. Todd (Eds.), Safe handling of foods (pp. 235256). New York: Marcel Dekker. Hart, H., Kendall, P., Smith, K., & Taylor, L. (1996). Evaluation of a pilot food safety training program for beef demonstrators. Journal of Food Service Systems, 9, 129141. Heenan, T., & Synder, O. (1978). The Minnesota program for prevention of foodborne illness: an evaluation. Journal of Food Protection, 41(7), 556558. Hennum, G., Lawrence, D., & Snyder, O. P. (1983). Two years of mandatory foodborne illness prevention certiWcation training in Moorehead, Minnesota. Dairy and Food Sanitation, 3(6), 208214. Hine, S., Thilmany, D., Kendall, P., & Smith, K. (2003). Employees and food safety: is training important to food service managers? Journal of Extension, 41(1). Holt, G., & Henson, S. (2000). Quality assurance management in small

meat manufacturers. Food Control, 11(4), 319326. Howes, M., McEwan, S., GriYths, M., & Harris, L. (1996). Food handler certiWcation by home study: measuring changes in knowledge and behaviour. Dairy, Food and Environmental Sanitation, 16(11), 737744. Jackson, B., Hatlen, J., & Palmer, B. (1977). Evaluation of a fast food management training program: one year later. Journal of Food Protection, 40(8), 562565. Johnston, M. C., Arthur, J., & Campbell, I. (1992). Foodhandling practices of Dunedin caterers: a cause for concern. New Zealand Medical Journal, 105(938), 289291. Kaferstein, F., & Abdussalam, M. (1999). Food safety in the 21st century. Bulletin of the World Health Organization, 77(4), 347351. Kaferstein, F. K., Motarjemi, Y., & Bettcher, D. W. (1997). Foodborne disease control: a transnational challenge. Emerging Infectious Diseases, 3(4), 503510. Kirby, M. P., & Gardiner, K. (1997). The eVectiveness of hygiene training for food handlers. International Journal of Environmental Health Research, 7(3), 251258. Kirby, R. (1994). HACCP in practice. Food Control, 5(4), 230236. Kirkpatrick, D. L. (1967). Training and development handbook; a guide to human resource development. New York: McGraw-Hill. Kneller, P., & Bierma, T. (1990). Food Service CertiWcationmeasuring the eVectiveness of a state program. Journal of Environmental Health, 52(5), 292294. Lambiri, M., Mavridou, A., & Papadakis, J. (1995). The application of hazard analysis critical control point (HACCP) in a Xight catering establishment improved bacteriological quality of meats. Journal of the Royal Society of Health(February), 2630. Laverack, G. R. (1989). Content analysis of a food hygiene course and the

assessment of its eVectiveness in fulWlling the training needs of food handlers. Master of Science thesis. University of Manchester Medical School. Little, C. L., Barnes, J., & Mitchell, R. T. (2002). Microbiological quality of take-away cooked rice and chicken sandwiches: eVectiveness of food hygiene training of the management. Communicable Disease and Public Health, 5(4), 289298. Luby, S. P., Jones, J. L., & Horan, J. M. (1993). A large salmonellosis outbreak associated with a frequently penalized restaurant. Epidemiology and Infection, 110(1), 3139. Mann, V., DeWolfe, J., Hart, R., Hollands, H., LaFrance, R., Lee, M., et al. (2001). The eVectiveness of food safety interventions Canada: Kingston and Frontenac and Lennox and Addington Health Unit (pp. 1105). Manning, C. K., & Snider, O. S. (1993). Temporary public eating places: food safety knowledge, attitudes and practices. Journal of Environmental Health, 56(1), 2428. Manpower Services Commission (1981). Glossary of training terms (3rd ed.). London: HMSO. Mathias, R. G., Riben, P. D., Campbell, E., Wiens, M., Cocksedge, W., Hazlewood, A., et al. (1994). The evaluation of the eVectiveness of routine restaurant inspections and education of food handlers: restaurant inspection survey. Canadian Journal of Public Health, 85(Suppl. 1), S61S66. Mathias, R. G., Sizto, R., Hazelwood, A., & Cocksedge, W. (1995). The eVects of inspection frequency and food handler education on restaurant inspection violations. Canadian Journal of Public Health, 86(1), 4650. McElroy, D. M., & Cutter, C. N. (2004). Self-reported changes in food

safety practices as a result of participation on a statewide food safety programme. Food Protection Trends, 24(3), 150161. Medeiros, L. C., George, R. T., Burns, K., Chandler, C., Crusey, S., Fittro, J., et al. (1996). The safe food handling for occasional quantity cooks curriculum. Journal of Nutrition Education, 28(1), 3943. Morrison, P., CaYn, N., & Wallace, R. (1998). Small establishments present challenge for Australian food safety code. International Journal of Contemporary Hospitality Management, 10(3), 101106. Mortlock, M. P., Peters, A. C., & GriYth, C. (1999). Food hygiene and hazard analysis critical control point in the United Kingdom food industry: practices, perceptions and attitudes. Journal of Food Protection, 62(7), 786792.1190 M.B. Egan et al. / Food Control 18 (2007) 11801190 Mortlock, M. P., Peters, A. C., & GriYth, C. (2000). HACCP in the raw. Environmental Health Journal, 108(6), 186189. Nabali, H., Bryan, F. L., Ibrahim, J., & Atrash, H. (1986). Evaluation of training food service managers in Bahrain. Journal of Environmental Health, 48(6), 315318. National Advisory Committee on Microbiological Criteria for Foods (NACMCF) (1998). Microbiological safety evaluations and recommendations on fresh produce. Food Control, 9(6), 321347. Oteri, T., & Ekanem, E. E. (1989). Food hygiene behaviour among hospital food handlers. Public Health, 103(3), 153159. Ovretveit, J. (1998). Evaluating health interventions: an introduction to evaluation of health treatments, services policies and organisational interventions. Buckingham: Open University Press. Palmer, B. J., Hatlen, J. B., & Jackson, B. B. (1975). Implementation and evaluation of management training in a fast food restaurant chain. Journal of Environmental Health, 37(4), 364368.

Parliamentary OYce of Science and Technology (POST) (2003). Food poisoning. January. Number 193. Available from Penninger, H., & Rodman, V. (1984). Food Service Managerial CertiWcation: how eVective has it been? Dairy and Food Sanitation, 4(7), 260264. Powell, S. C., Attwell, R. W., & Massey, S. J. (1997). The impact of training on knowledge and standards of food hygienea pilot study. International Journal of Environmental Health Research, 7(4), 329334. Reicks, M., Bosch, A., Herman, M., & Krinke, U. B. (1994). EVectiveness of a food safety teaching strategy promoting critical thinking. Journal of Nutrition Education, 26(2), 97100. Rennie, D. M. (1994). Evaluation of food hygiene education. British Food Journal, 96(11), 2025. Rennie, D. M. (1995). Health education models and food hygiene education. Journal of the Royal Society of Health, (April), 7578. Riben, P. D., Mathias, R. G., Campbell, E., & Wiens, M. (1994). The evaluation of the eVectiveness of routine inspections and education of food handlers: critical appraisal of the literature. Canadian Journal of Public Health, 85(Suppl. 1), S56S60. Riben, P. D., Mathias, R. G., Wiens, M., Cocksedge, W., Hazlewood, A., Kirschner, B., et al. (1994). Routine restaurant inspections and education of food handlers. Recommendations based on critical appraisal of the literature and survey of Canadian jurisdictions on restaurant inspections and education of food handlers. Canadian Journal of Public Health, 85(Suppl. 1), S67S70. Richardson, I. R., & Stevens, A. M. (2003). Microbiological examination of ready-to-eat stuYng from retail premises in the north-east of England. The Get StuVed survey. Journal of Applied Microbiology, 94(4), 733737. Rinke, W. J., Brown, N. E., & McKinley, M. M. (1975). Two methods for

instructing personnel about food service sanitation. Journal of the American Dietetic Association, 67(4), 364367. Sagoo, S. K., Little, C. L., GriYth, C. J., & Mitchell, R. T. (2003a). Study of cleaning standards and practices in food premises in the United Kingdom. Communicable Disease and Public Health, 6(1), 34. Sagoo, S. K., Little, C. L., & Mitchell, R. T. (2003b). Microbiological quality of open ready-to-eat salad vegetables: eVectiveness of food hygiene training of management. Journal of Food Protection, 66(9), 15811586. Smith, R. (1994). Food hygiene training: the chance to create a coherent training policy. British Food Journal, 96(7), 4145. SoneV, R., McGeachy, F., Davison, K., McCargar, L., & Therien, G. (1994). EVectiveness of two training methods to improve the quality of food service in small facilities for adult care. Journal of the American Dietetic Association, 94(8), 869873. Sparkman, A. F., Briley, M. E., & Gillham, M. B. (1984). Validated food service training manual for supervisors with limited experience. Journal of the American Dietetic Association, 84(12), 14751478. Sprenger, R. A. (1991). Hygiene for management: a text for food hygiene courses. Doncaster, UK: HighWeld Publications. Sumbingco, S., Middleton, R., & Konz, S. (1969). Evaluating a programmed text for training food service employees. Journal of the American Dietetic Association, 54(4), 313317. Tannenbaum, S. I., & Yuki, G. A. (1992). Training and development in work organisations. Annual Review of Psychology, 43, 399441. Tauxe, R. V. (1997). Emerging foodborne disease, an evolving public health challenge. Emerging Infectious Diseases, 3(4), 425434. Taylor, E. (1994). Does management training ensure a safe food supply for the hospitality industry? Australian Journal of Hospitality Management, 1, 1315.

Taylor, E. (1996). Is food hygiene training really eVective? Environmental Health, (September), 275276. Taylor, E. (2001). HACCP in small companies: beneWt or burden? Food Control, 12(4), 217222. Tebbutt, G. M. (1986). An evaluation of various working practices in shops selling raw and cooked meats. Journal of Hygiene, 97, 8190. Tebbutt, G. M. (1991). Development of Standardized inspections in restaurants using visual assessments and microbiological sampling to quantify the risks. Epidemiology Infection, 107, 393404. Tebbutt, G. M. (1992). An assessment of food hygiene training and knowledge among staV in premises producing or selling high-risk foods. International Journal of Environmental Health Research, 2, 131137. The Food Hygiene (England) Regulations (2006). S.I. 14/2006. Available from Toh, P. S., & Birchenough, A. (2000). Food safety knowledge and attitudes: culture and environment impact on hawkers in Malaysia. Food Control, 11(6), 447452. Tones, B. K., & Tilford, S. (1994). Health education: eVectiveness eYciency and equity (2nd ed.). London: Chapman & Hall. Tracey, J. B., & Cardenas, C. G. (1996). Training eVectiveness: an empirical examination of factors outside the training context. Hospitality Research Journal, 20(2), 113123. Tracey, J. B., Tannenbaum, S. I., & Kavanagh, M. J. (1995). Applying trained skills on the job: the importance of the work environment. Journal of Applied Psychology, 80(2), 239252.

Waddell, K. P., & Rinke, W. J. (1985). EVectiveness of a computer-assisted instruction program for teaching sanitation. Journal of the American Dietetic Association, 85(1), 6267. Wade, J. A. (1998). Hygiene for caterersthe perennial problem. Hospitality Management, 17(1), 8387. Walker, E., Pritchard, C., & Forsythe, S. (2003). Food handlers hygiene knowledge in small food businesses. Food Control, 14(5), 339343. Wheeler, J. G., Seith, D., Cowden, J. M., Wall, P. G., Rodrigues, L. C., Tompkins, D. S., et al. (1999). Study of IID in England: rates in the community, presenting to general practice and reported to national surveillance. British Medical Journal, 318(7190), 10461055. WHO (1999). Strategies for implementing HACCP in small and/or less developed businesses: report of the WHO Consultation in collaboration with the Ministry of Health, Welfare and Sports, The Netherlands, The Hague, 1619 June 1999, WHO, Geneva. Worsfold, D. (1993). Food safety: an appraisal of a training programme. Journal of the Royal Society of Health, (December), 316319. Worsfold, D. (1998). Help with HACCP. Journal of the Royal Society of Health, 118(5), 295299. Worsfold, D., & GriYth, C. (1995). A generic model for evaluating consumer food safety behaviour. Food Control, 6(6), 357363. Worsfold, D., & GriYth, C. J. (2003). A survey of food hygiene and safety training in the retail and catering industry. Nutrition and Food Science, 33(2), 6879. Wright, J., & Feun, L. (1986). Food service manager certiWcation: an evaluation of its impact. Journal of Environmental Health, 49(1), 1215.

Wyatt, C. J. (1979). Concerns, experiences, attitudes and practices of food market managers regarding sanitation and safe food handling practices. Journal of Food Protection, 42(7), 555560. Zain, M. M., & Naing, N. N. (2002). Socio-demographic characteristics of food handlers and their knowledge, attitude and practice towards food sanitation: a preliminary report. Southeast Asian Journ