International Journal for Quality in Health Care 2009; Volume 21, Number 1: pp. 37 –43 10.

1093/intqhc/mzn049
Advance Access Publication: 6 November 2008

The cycle of change: implementing

best-evidence clinical practice
MARIKO CAREY 1,2, HEATHER BUCHAN3 AND ROB SANSON-FISHER4
1
The Cancer Council Victoria, Carlton, Australia, 2The University of Melbourne, Parkville, Australia, 3National Institute of Clinical Studies,
Melbourne, Australia, and 4University of Newcastle, Newcastle, Australia

Abstract
To improve health outcomes, effective and systematic mechanisms to foster the adoption of evidence-based guideline
recommendations into routine practice need to be identified. A cyclical process for achieving this objective involving three
key phases is suggested.
Phase 1. Writing actionable best-evidence guidelines that prioritize key recommendations while indicating the levels of
adoption needed for population health benefits to be accomplished.
Phase 2. Developing implementation plans for the priority guideline recommendations. These should systematically consider
skills training and accreditation; social influences including opinion leaders and patient influences; environmental factors;
monitoring and feedback; and incentives for clinical change.
Phase 3. Pilot testing the effectiveness of proposed approaches in producing the desired clinical changes. If implementation
requires system changes and evaluation at an organizational level, the use of alternative research designs to the randomized

Downloaded from by guest on November 16, 2015

controlled trial could be considered. The purpose evaluation would be to enable refinement of the implementation plans
before widespread dissemination.
Keywords: evidence-based practice, guidelines, guideline adherence, quality in healthcare

Introduction recommendations that require changes in individual

behaviour, while recognizing that system changes may also
Closing the gap between best evidence and existing clinical be needed to support these changes. A summary of the strat-
practice has the potential to improve health outcomes [1]. egies suggested, underlying theoretical constructs and empiri-
Evidence-based clinical guidelines aim to do this; however cal references is shown in Table 1.
guidelines alone may not result in the required change in clini-
cal practice [2]. We propose a three-phase process for achieving
these objectives. The first involves producing evidence-based Box 1 Key points for the implementation
guidelines that maximize the likelihood of implementation. of the three-phase model
The second relates to developing implementation plans that † Carefully consider how the guideline is presented
give guidance on improving uptake of priority recommen- and how recommendations are phrased
dations. The third component involves testing of the † Identify priority recommendations within the
implementation plan to enable refinement before widespread guideline
dissemination. Although the model is resource intensive, such † Produce guideline in conjunction with an
investment is warranted in order to translate evidence develop- implementation plan for priority recommendations
ment efforts into clinical benefit in priority health areas. An † Involve key stakeholders in production of
overview of key steps in the model is shown in Box 1. implementation plan
The need for an organizing framework to aid the appli- † Pilot test both the guideline and implementation
cation of theoretical concepts to the development of plan
evidence-implementation interventions has been identified † Refine guideline and implementation plan before
[3]. The current model suggests strategies targeting factors widespread dissemination
associated with behavioural change in a logical sequence.
Our focus is on improving uptake of guideline

Address reprint requests to: Mariko Carey, Tel: þ61 3 9635 5377; Fax: þ61 3 9635 5380; E-mail: mariko.carey@cancervic.org.au

International Journal for Quality in Health Care vol. 21 no. 1

# The Author 2008. Published by Oxford University Press in association with the International Society for Quality in Health Care;
all rights reserved 37
38

Carey et al.
Table 1 Strategies to promote guideline implementation: theoretical constructs and examples of application

Strategy Relevant constructs Key illustrative examples

............................................................................................................................................................................................................................................

Phase 1
Concrete and specific Knowledge, executability, Concrete and specific recommendations were more likely to be adopted by GPs than
recommendations decidability vague non-specific recommendations. Observational study. Grol et al. [19]
Identify priorities Goal-setting, action planning Of 228 primary care patients with cardiovascular disease risk factors who made an
Set targets for implementation Goal-setting action plan to identify behavioural change goals, 53% also reported making behavioural
change related to their action plan. Descriptive study. Handley et al. [11]
Present a rationale Beliefs, attitudes, perceived relative Recommendations compatible with current values were more likely to be adopted by
advantage GPs than those perceived as controversial or incompatible with values. Observational
study. Grol et al. [19]
Highlight clinical norms Normative beliefs, attitudes, An intervention to improve myocardial infarction care that involved using local medical
modelling/verbal persuasion opinion leaders to influence peers through small group discussions, informal
consultation and revisions of clinical protocols was compared with performance
feedback alone. Hospitals in both groups improved from baseline to follow up on
indicators of quality, however the improvement was greatest for those allocated to the
peer intervention. RCT Soumerai et al. [21]
Orient to the need of the end user Complexity Among the guideline characteristics most commonly endorsed to promote use by GPs
was ‘clarity, simplicity and availability of a short format’. Descriptive study of 391 GPs.
Watkins et al. [20]
Phase 2
Skills training Skills, knowledge, self-efficacy CME improves knowledge, skills, attitudes and patient outcomes. CME that is
interactive, uses multimedia, live media, and involves multiple exposures is more effective
than other types. Systematic review. Marinopoulos et al. [23]
Social influences Normative beliefs, attitudes, The use of local opinion leaders in hospital settings can be effective in promoting
modelling, verbal persuasion evidence-based practice. Systematic review of 12 studies. Doumitt et al. [24]
Environmental influences Cues to action, environmental Guideline adherence improved due to the implementation of a computerized clinical
triggers decision aid that gave clinicians real time recommendations for venous
thromboembolism prophylaxis. Time series study. Durieux et al. [26]
Patient-mediated Knowledge, skills and attitudes of Patient request for a new drug and patient acceptability were cited as contributing to
patients decisions to prescribe a new drug in 20% of cases. Descriptive study. Prosser et al. [28]

Downloaded from by guest on November 16, 2015

 Putting evidence into practice

Phase 1: Producing actionable guidelines

Breakthrough collaborative model intervention that involved a series of iterative plan, do,
studies examining provider group-level incentives demonstrated partial or positive effects
Audit and feedback are effective strategies for improving care, particularly when baseline

study, act cycles was found to be effective in improving care for chronic heart failure.
Although guidelines are a way of helping clinicians keep

Five out of six studies examining physician level incentives, and seven out of nine
adherence to the recommended practice is low. Systematic review of 118 studies.
up-to-date with best evidence, their utility rests upon their
capacity to effect implementation [4]. While much effort has
been invested in developing rigorous processes for appraising
the evidence used to construct evidence-based guideline rec-
ommendations [5], until recently there has been less focus on
other attributes of guidelines which may affect clinical
behaviour. A study of 51 lung cancer guidelines appraised
using the AGREE (Appraisal of Guidelines Research and
on quality indicators. Systematic review. Petersen et al. [36]

Evaluation) instrument found that implementation issues were

inadequately addressed [6]. Leading international guideline
Quasi experimental, controlled study. Asch et al. [44]

developers are now exploring mechanisms that may improve

the implementability of guidelines. Strategies have included
use of the GLIA (GuideLine Implementability Appraisal)
instrument [11] and the use of questionnaires and consul-
tation processes that seek feedback on local applicability [8].
Some guideline recommendations may be more difficult to
implement because of the intrinsic nature of the required
practice changes [9]. The evidence-based behaviour may be
unconventional, require acquisition of new skills or equip-
Key illustrative examples

ment, or require systems to change in ways that are expensive

or difficult. Although some of these matters may not be able
Jamtvedt et al. [30]

to be modified, guideline developers should be cognizant of

Downloaded from by guest on November 16, 2015

how recommendations are phrased and presented. The fol-
lowing mechanisms, which might optimize the actionability
of the guidelines, should be considered [10].

Recommendations should specifically describe

the desired clinical behaviour
Positive/negative reinforcement;

Positive/negative reinforcement

The guideline should describe the desired behaviour in con-

goal-setting; skill development

Perceived advantages; beliefs;

crete and specific terms. This would include details about the
specific recommended clinical action, who should undertake
the action, to which group of patients it should be applied,
Relevant constructs

and under what circumstances [5, 7]. The GLIA tool [7]
refers to concepts of ‘decidability’ (i.e. the precise circum-
stances under which something should be done), and
‘executability’ (i.e. exactly what should be done under the cir-
trialability

cumstances defined).

Guidelines should identify priorities

Pilot testing with iterative refinement

Prioritization of actions is an important feature of action

plans, a tool commonly used to attain goals [11]. To increase
guideline effectiveness, developers should prioritize those
recommendations that exhibit important divergence from
of implementation strategies

best practice; which are sufficiently potent to produce signifi-

cant health benefits if broadly adopted; and for which there
are accurate outcome measures of provider adherence.
Currently there is limited research on how these priority rec-
ommendations should be presented within the guideline
Incentives
Feedback

format. Educational research suggests that presenting priority

Strategy

Phase 3

recommendations first in any document, emphasizing their

importance, and re-iterating them in a summary may increase
recall and hence potentially implementation [12].

39
Carey et al.
Targets for adoption should be defined
Goal-setting is a key component of many behavioural change
theories [13, 14]. Combinations of short-term and longer-
term goals may be more effective than long-term goals alone
[13]. Therefore, guidelines should specify a series of incre-
mental population-level targets for clinically significant
change in uptake of priority recommendations, and indicate a
reasonable time frame in which this level of change could be
expected. Advice on specific indicators that may be used to
measure whether targets have been achieved should be
included [7].
Recommendations should specify benefits
associated with change
A number of theories including Operant [15], Social
Cognitive [14] and Diffusion of Innovations [16] postulate
that the way people behave is influenced by their beliefs
about the likely outcomes and the perceived value attached
to those outcomes. Therefore, guidelines should include
rationales for adoption in order to increase the likelihood of
implementation. An evaluation of the implementation of
guidelines issued by the UK National Institute for Clinical
Excellence concluded that practice had changed faster in
areas where practitioners and managers saw clear advantages
to adoption [17].
Compatibility with existing clinical
norms should be highlighted
Social influences play an important role in the adoption of
new practices [16] and behaviours that are perceived to be
normative are more likely to be adopted [18]. If the practice
is one that is readily endorsed by a majority of the target
group, emphasis of this may be beneficial. A study of Dutch
general practitioners found that guideline recommendations
perceived as controversial and incompatible with current
values were less likely to be adopted than those without this
attribute [19].
Guideline presentation should be oriented
to the needs of end users
New practices that are difficult to understand or use may be
less likely to be adopted [16]. Consequently guidelines that are
oriented to the needs of the end user in terms of their presen-
tation and by provision of tools to facilitate use may be more
easily adopted. General practitioner (GPs) prefer guidelines
that are presented in a short format or flowcharts [20]. Tools
for application such as patient education materials and guide-
line summaries may aid implementation; while presentation
and formatting can be used to make priority recommendations
easy to identify [5, 7]. As a result it is recommended that
careful attention is paid to the way in which guidelines are pre-
sented, in particular, the provision of a compelling and com-
prehensive rationale for practice change. Guideline developers
should also consider the identification of priorities and targets
40
for implementation, as well as the resources for guideline
users that could accompany the guideline document.
Phase 2: Designing implementation plans
Providing strategies for implementation in guidelines [7] and
suggestions about overcoming organizational barriers to
implementation may also be helpful [5]. These may require a
mix of individual, organizational and general system strategies.
Because different guideline recommendations focus on differ-
ent aspects of clinical behaviour or system performance,
each priority recommendation should be accompanied by a
detailed plan that provides step-by-step suggestions for
implementation. These plans are most likely to be successful
if there is early identification and involvement of key stake-
holders from the area of practice where the need for change
is identified (such as clinicians, managers and policy makers
and representatives from professional associations involved in
accreditation and professional development). Specialists such
as behavioural scientists, epidemiologists, health economists
and legal experts may also be needed to help design
implementation strategies. Plans should consider six key areas:
requirement for new skills; social influences such as opinion
leaders; environmental factors that may promote adoption;
Downloaded from by guest on November 16, 2015
patient-mediated influences; monitoring and feedback; and
incentives. Organizations that have a commitment to imple-
menting the guidelines should then identify a leadership team
who can take responsibility for assessing how the plan can be
used to inform quality improvement initiatives at the local
level. The strategy of engaging local leaders has been used
successfully to promote improvements in care across a variety
of areas, e.g. for acute myocardial infarction [21].
Assessing whether new skills are required
Like knowledge, appropriate level of skill is considered a
necessary pre-condition for behaviour change [14]. If a rec-
ommendation requires that clinicians learn new skills or
improve existing skills, then skills assessment, training and
accreditation may be required. This is likely to be required
for specialty procedural skills, such as robot-assisted surgery
[22]. Consultation with clinical experts and professional
bodies may enable implementation plans to provide appropri-
ate strategies for assessment of existing skills in the work-
force. Depending on the complexity of the skill, strategies
may include self-assessment, peer assessment, external
assessment, or credentialing by a professional body.
If formal training and accreditation are required, the plans
should provide information on types of training likely to be
effective and means of accreditation. A recent review has
found evidence that continuing medical education (CME)
improves knowledge, skills, attitudes, clinical behaviour, and
patient outcomes [23]. Providing opportunities for learners
to see a new skill performed (modelled), giving opportunities
for practice, and providing constructive performance feed-
back builds both skills and confidence to perform a new
behaviour [14]. Reviews of CME effectiveness have found

that educational approaches which are interactive, use multi-
media and live media and involve multiple exposures appear
more effective in changing clinical performance [23].
Collaboration between implementation plan developers and
professional associations would facilitate the development of
appropriate training packages.
Using social influences to enhance
implementation
Behavioural change theories [14, 16, 18] recognize the role
of perceptions of the social environment in influencing beha-
viour. A Cochrane review indicates that opinion leaders can
effectively promote evidence implementation [24], but there
are still uncertainties about how best to maximize effective-
ness. A recent systematic review on innovation in health [25]
similarly found evidence to support the role of opinion
leaders and champions (those committed to the adoption of
a new practice) in effecting change. Individuals who have
strong social ties both within and between organizations may
be particularly influential in promoting adoption of a new
practice [25]. Adoption is greater when those who are the
focus of the intervention are similar in terms of background
to the change agent [25]. Drawing on best evidence,
implementation plans should provide practical strategies for
identifying, engaging, and using opinion leaders or other
change agents to promote implementation.
Using environmental factors to promote adoption
The environment plays an important role in influencing
behaviour [14, 15]. Systems can support or deter a clinical
behaviour. Automated reminders have been used to improve
adherence to venous thromboembolism prophylaxis guide-
lines [26]. Similarly, implementation of organizational pro-
cedures and protocols can change practice and reduce
adverse events [27]. Implementation plans should assess how
systems can operate to maximize the probability of clinical
change at the organizational level (e.g. by development of
policies and procedures) or at the regional level (e.g. diabetes
registers). Plans should outline methods for effectively adapt-
ing to existing systems or setting up new systems to promote
implementation.
Using patient-mediated influences
to increase adoption
Patient preferences and expectations are a form of social
influence that can have a powerful effect on care. For
example, where GPs made a decision to prescribe a new
drug, the decision was reported to be influenced by patient
requests in 22% of cases, and by patient acceptability in 20%
of cases [28]. Strategies to improve patient health literacy,
involvement in treatment decision-making, and self-
management of health conditions may assist patients to have
a positive impact on their own health care [29]. Consequently
implementation plans should identify effective strategies
Putting evidence into practice
to influence patient attitudes and knowledge about best
evidence practice.
Role of audit and feedback in adoption
Feedback on performance is an important feature of beha-
vioural change theories that encompass goal-setting and skill
development [13, 14]. Early feedback allows refinement of
goals and detection of knowledge or skill deficits that may
hinder goal attainment, as well as enhance motivation [13].
Systematic reviews have shown that performance feedback is
an effective means of changing provider practice [30].
Therefore feedback on the extent of implementation of pri-
ority recommendations should be provided to clinicians and
to health care organizations.
If given non-punitively, feedback may be most effective
given at the individual rather than group level [31]. The
impact of feedback is illustrated by Tibballs’ study [32] of
handwashing among intensive care staff. Staff washed their
hands both before and after patient contact 4.3% of the time
when covertly observed. This rose to 22.6% during a period
of overt observation and to 55.2% when feedback was
added to the overt observation.
Feedback may provide comparisons with past perform-
ance, peers, benchmarks, or gold standards. The type of
Downloaded from by guest on November 16, 2015
comparative data provided with feedback should be con-
sidered within the implementation plan. This may depend, in
part, on the level at which feedback is provided, e.g. to indi-
vidual clinicians, wards, practices, hospitals, or regional
health services. Feedback is likely to be more effective when
given close to the time of patient contact [33] and on a fre-
quent basis [30]. Therefore, the feedback schedule, as well as
practical issues such as cost, should be considered when
examining the feasibility of different data collection methods.
Using incentives to increase adoption
Positive consequences of behaviour encourage adoption of
specific behaviours, while negative consequences discourage
adoption [15]. Desire to improve patient care, professional
pride and peer influences may act as powerful intangible
motivators [34]. Intangible incentives such as these may be
more effective where the type of clinical performance
required is difficult to accurately quantify [35]. These should
be the first choice for trialling with an implementation plan.
Over reliance on external tangible incentives such as financial
rewards may damage internal motivators such as profession-
alism [35].
There is some evidence that pay for performance strat-
egies for the individual clinician may change the care pro-
vided [36]. A recent randomized study comparing
interventions to improve dentists’ use of sealant treatments
to prevent cavities in newly erupted molars found that pro-
viding a fee for service increased sealant use by 10%.
Education alone had no significant effect on practice [37].
Not all studies, however, have found evidence for the effec-
tiveness of financial incentives [38]. Several factors have been
identified that may influence the effectiveness of such
41
Carey et al.
approaches: the size of the incentive, who it is directed at,
and how the targets for incentive payments are set [39].
Using fixed performance targets might reward those insti-
tutions with the highest baseline performance rather than
those who demonstrated the greatest improvement [38].
Given this literature, the implementation plan should
identify ways that both intangible and tangible incentives may
promote implementation. If internal motivators are found to
be ineffective then external incentives should be considered.
Careful consideration should be given to the way in which
both the targets and the incentives are formulated. Punitive
strategies should be considered only when the relevant clini-
cal behaviour may result in serious patient harm.
Phase 3: Evaluating the effectiveness of
guideline dissemination through
implementation plans
Stakeholder opinions about implementation approaches are
commonly used to indicate the likely effectiveness of
implementation strategies. However in spite of positive
reviews by providers this information does not always predict
appropriate practice change [40, 41]. Preferably evaluation
should measure if the guideline produces change in actual
clinical performance rather than reported guidelines intention
to change [3]. Allowance should be made for the time
required for change to occur. If the desired change in clinical
performance does not occur, there is an opportunity to
refine either the format of the guidelines or the proposed
implementation strategy.
Evaluations should be done in a cost-effective and rigor-
ous manner. As implementation often requires systems
change, the effectiveness of the combined guidelines and
implementation plans may be tested on large units such as
hospitals, rather than on individual clinicians. If so, the use
of randomized controlled designs to test effectiveness is
likely to present difficulties in terms of preventing contami-
nation between experimental conditions, cost and logistics
[42]. The multiple baseline design is a feasible alternative for
evaluating system-level interventions. This design requires a
minimum of two organizations (or other units) to which the
intervention (in this case, the implementation plan) is applied
at different time points [43]. Adherence to the clinical rec-
ommendation is monitored for at least three time points
before the implementation guide is released in each organiz-
ation so that baseline performance can be determined.
Monitoring continues after the release of the implementation
plan and guideline to determine their effect on performance.
Steps in the implementation plan may be added sequentially,
allowing the impact of the addition of each further strategy
to be assessed.
To enable timely evaluation and dissemination of
implementation plans, initial monitoring of the priority rec-
ommendations in one or two organizations could commence
before finalizing the plans for testing. The results of the
evaluation should enable detection of any problems with the
42
implementation strategies, or aspects of the guidelines them-
selves that need clarification. Hence, data from the evaluation
can be used iteratively to improve the guidelines and the
implementation plans.
Conclusion
Guideline production and dissemination do not necessarily
result in the widespread adoption of best-evidence clinical
care. Guideline developers should give more attention to
the coordinated and systematic approaches required for
implementation. We suggest that three overlapping phases
are required. Firstly, the guidelines should be presented in a
way that identifies priorities and maximizes the likelihood of
implementation. Secondly, mechanisms designed to maximize
implementation should be identified concurrently with the
guideline development. Such implementation plans should
involve testable strategies that systematically address knowl-
edge, skills, motivational and environmental factors that
influence clinical performance. Thirdly, pilot testing to evalu-
ate the effectiveness of implementation plans should routi-
nely occur. This would allow for improvement in the plans
as well as the accumulation of evidence about how to
implement best-evidence care successfully.
Downloaded from by guest on November 16, 2015
References
1. National Institute of Clinical Studies. Evidence-practice Gaps
Report, Vol. 2. 2005, Melbourne, Australia. http:www.nhmrc.
gov.au/nics/asp/index.asp?page=materials/materials_type_-
articles&cid=5212&id=405.
2. Grimshaw JM, Thomas RE, MacLennan G. Effectiveness and
efficiency of guideline dissemination and implementation strat-
egies. Health Technol Assess 2004;8.
3. Eccles M, Grimshaw J, Walker A et al. Changing the behaviour
of healthcare professionals: the use of theory in promoting the
uptake of research findings. J Clin Epidemiol 2005;58:107 –12.
4. Shaneyfelt TM. Building bridges to quality. JAMA 2001;286:
2600 –01.
5. Cluzeau FA, Burgers JS, Brouwers M et al. Development and
validation of an international appraisal instrument for assessing
the quality of clinical practice guidelines: the AGREE project.
Qual Saf Health Care 2003;12:18–23.
6. Harpole LH, Kelley MJ, Schreiber G et al. Assessment of the
scope and quality of clinical practice guidelines in lung cancer.
Chest 2003;123(Suppl. 1):7S –20S.
7. Shiffman RN, Dixon J, Brandt C et al. The GuideLine
Implementability Appraisal (GLIA): development of an instru-
ment to identify obstacles to guideline implementation. BMC
Med Inform Decis Mak 2005;5:23. doi:10.1186/1472-6947-5-23.
8. Marshall C. Summary report: assessing the implementability of
guidelines. Report to the National Institute of Clinical Studies,
September 2006. http://www.nhmrc.gov.au/nics/asp/index.
asp?page=materials/materials_type_article&cid=5212&id=544
(25 August 2008, date last accessed).

9. Eccles MP, Grimshaw JM. Selecting, presenting and delivering
clinical practice guidelines: are there any ‘magic bullets? Med J
Aust 2004;180 (Suppl. 6):S52 –S54.
10. Davis D, Goldman J, Palda VA. Handbook on Clinical Practice
Guidelines. Canadian Medical Association, 2007.
11. Handley M, MacGregor K, Schillinger D et al. Using action
plans to help primary care patients adopt healthy behaviours: a
descriptive study. J Am Board Fam Med 2006;19:224–31.
12. Ley P. Memory for medical information. Br J Soc Clin Psychol
1979;18:245– 55.
13. Locke EA, Latham GP. Building a practically useful theory of
goal setting and task motivation. Am Psychol 2002;57:705– 17.
14. Bandura A. Social Foundations of Thought and Action. Englewood
Cliffs, NJ, USA: Prentice-Hall, 1986.
15. Gleitman H (ed). Psychology, 3rd edn. NY: W.W. Norton &
Company, Inc., 1991.
16. Rogers EM. Diffusion of Innovations, 5th edn. New York, NY:
Free Press, 2003.
17. Sheldon T, Cullum N, Dawson D et al. What’s the evidence that
NICE guidance has been implemented? Results from a national
evaluation using time series analysis, audit of patient’s notes,
and interviews. Br Med J 2004;329:999.
18. Azjen I. From intentions to actions: a theory of planned beha-
viour. In: Kuhl J, Beckman J (eds). Action-Control: From Cognition
to Behaviour. Heidelberg: Springer, 1985, 11–39.
19. Grol R, Dalhuijsen J, Thomas S et al. Attributes of clinical
guidelines that influence use of guidelines in general practice:
observational study. Br Med J 1998;317:858– 61.
20. Watkins C, Harvey I, Langley C et al. General practitioners’ use
of guidelines in the consultation and their attitudes to them. Br
J Gen Pract 1999;49:11– 5.
21. Soumerai SB, McLaughlin TJ, Gurwitz JH et al. Effect of local
medical opinion leaders on quality of care for acute myocardial
infarction. JAMA 1998;279:1358 –63.
22. Kypson AP, Nifong LW, Chitwood WR Jr. Robot-assisted
surgery: training and re-training surgeons. Int J Med Robot
2004;1:70 –6.
23. Marinopoulos SS, Dorman T, Ratanawongsa N et al.
Effectiveness of continuing medical education. Evid Rep Technol
Assess 2007;14;1 – 69.
24. Doumitt G, Gattellari M, Grimshaw J et al. Local opinion
leaders: effects on professional practice and health care out-
comes. Cochrane Database Syst Rev 2007, Issue 1. Art No.:
CD000125. doi: 10.1002/14651858.CD000125.pub3.
25. Greenlagh T, Robert G, Bate P. How to spread good ideas. A
systematic review of the literature on diffusion, dissemination
and sustainability of innovations in health service delivery and
organization. Report for the National Co-ordinating Centre for NHS
Service Delivery and Organisation R&D (NCCSDO), April 2004.
26. Durieux P, Nizard R, Ravaud P et al. A clinical decision support
system for prevention of venous thromboembolism: effect on
physician behaviour. JAMA 2000;283:2816 –21.
27. Couzigou C, Lamory J, Salmon-Ceron D et al. Short peripheral
venous catheters: effect of evidence-based guidelines on
Putting evidence into practice
insertion, maintenance and outcomes at a university hospital.
J Hosp Infect 2005;59:197–204.
28. Prosser H, Almond S, Walley T. Influences on GPs’ decision to
prescribe new drugs: the importance of who says what. Fam
Pract 2003;20:61–8.
29. Coulter A, Ellins J. Effectiveness of strategies for informing,
educating, and involving patients. BMJ 2007;335:24 –7.
30. Jamtvedt G, Young JM, Kristofferson DT et al. Audit and feed-
back: effects on professional practice and health care outcomes.
Cochrane Database Syst Rev 2006;2:CD000259.
31. Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical
practice guideline adherence: making feedback actionable.
Implement Sci 2006;1:9.
32. Tibballs J. Teaching hospital medical staff to handwash. Med J
Aust 1996;164:395 –98.
33. Davis D, Taylor-Vaisey A. Translating guidelines into practice: a
systematic review of theoretic concepts, practical experience and
research evidence in the adoption of clinical practice guidelines.
Can Med Assoc J 1997;157:408 –16.
34. Spooner A, Chapple A, Roland M. What makes British general
practitioners take part in a quality improvement scheme?
J Health Serv Res Policy 2001;6:145–50.
35. Marshall M, Harrison S. Its about more than money: financial
Downloaded from by guest on November 16, 2015
incentives and internal motivation. Qual Saf Health Care
2005;14:4 –5.
36. Petersen LA, Woodward LD, Urech T et al. Does
pay-for-performance improve the quality of health care? Ann
Intern Med 2006;145:265–72.
37. Clarkson JE, Turner S, Grimshaw JM et al. Changing clinicians’
behavior: a randomized controlled trial of fees and education.
J Dent Res 2008;87:640 –44.
38. Rosenthal MB, Frank RG, Zhonghe L et al. Early experience
with pay-for-performance. JAMA 2005;294:1788 – 92.
39. Rowe JW. Pay-for-performance and accountability: related themes
in improving health care. Ann Intern Med 2006;695–699.
40. Fretheim A, Oxman D, Flottorp S. Improving prescribing of
antihypertensive and cholesterol-lowering drugs: a method for
identifying and addressing barriers to change. BMC Health Serv
Res 2004;4:23. doi: 10.1186/1472-6963-4-23.
41. Fretheim A, Oxman AD, Havelsrud K et al. Rational prescrib-
ing in primary care (RaPP): a cluster randomized trial or a tai-
lored intervention. PLoS Med 2006;3:pe134. doi: 10.1371/
journal.pmed.0030134.
42. Sanson-Fisher RW, Bonevski B, Green L et al. Limitations of
the randomized controlled trial in evaluating population-based
health interventions. Am J Prev Med 2007;33:155 –61.
43. Hawkins NG, Sanson-Fisher RW, Shakeshaft A et al. The mul-
tiple baseline design for evaluating population-based research.
Am J Prev Med 2007;33:162 –68.
44. Asch SM, Baker DW, Keesey J et al. Does the collaborative
model improve care for chronic heart failure. Med Care
2005;43:667 –75.
Accepted for publication 2 October 2008
43