Downloaded from http://ebm.bmj.com/ on November 23, 2017 - Published by group.bmj.
com
Evidence-Based Medicine Online First, published on November 23, 2017 as 10.1136/ebmed-2017-110854
EBM Learning
10.1136/ebmed-2017-110854
1 consider essential reading for anyone starting out on an
Nuffield Department of Primary
Care Health Sciences, University
of Oxford, Oxford, UK
2
Centre for Evidence-Based
Medicine, University of Oxford,
Oxford, UK
Correspondence to:
Dr David Nunan, Nuffield
Department of Primary Care
Health Sciences, University of
Oxford, Oxford, OX2 6GG, UK; ​
david.​nunan@​phc.​ox.​ac.u
​ k
202 Evid Based Med December 2017 | volume 22 | number 6 |
Copyright Article author (or their employer) 2017. Produced by BMJ Publishing Group Ltd under licence.
Ten essential papers for the practice of evidence-
based medicine
David Nunan,1,2 Jack O Sullivan,1,2 Carl Heneghan,1,2
Annette Pluddemann,1,2 Jeffrey Aronson,2 Kamal Mahtani1,2
Abstract
In this article we signpost readers to 10 papers we
evidence-based medicine journey. We have considered
papers consisting a mix of old and new, seminal and
cutting-edge that offer insight into what evidence-based
medicine is, where it came from, why it matters and
what it has achieved. This is balanced against some of
the common criticisms of evidence-based medicine and
efforts to tackle them. We have also highlighted papers
acknowledging the importance of teaching and learning
of the principles of evidence-based medicine and how
health professionals can better use evidence in clinical
decisions with patients.
Introduction
As an introduction to evidence-based practice, we as
a group of evidence-based researchers, clinicians and
editors have collated the top 10 papers we consider most
helpful when starting on the journey of evidence-based
medicine (EBM). We have based our selection on our
experience of teaching a wide range of individuals and
describe why we consider each paper to be important.
EBM: what it is and what it is not
In 1996, the BMJ published an editorial by Sackett and
others, in which they defined EBM as ‘the conscientious,
explicit and judicious use of current best evidence in
making decisions about the care of individual patients’.1
By emphasising the essential components of EBM, they
made it clear that evidence, values and expertise play
similar roles in clinical decision making.
EBM: a new approach to teaching the practice
of medicine
Although the 1996 paper by Sackett et al clarified
what EBM is and what it is not, the term and concept
had already been introduced in a 1992 paper by the
Evidence-Based Medicine Working Group.2 A paradigm
shift in medical practice was proposed, where the exam-
ination of evidence from clinical research is given equal
place in clinical decision making as that of biological
reasoning, clinical experience and intuition. As a result,
physicians needed to develop new skills, including effec-
tive searching skills and the application of evidence rules
to evaluate the clinical literature. Thus, EBM was born.
The scandal of poor medical research
Over 20 years ago, Altman published his now seminal
paper in the BMJ, telling us in his view how most
medical research was  not good and was probably
wrong.3 Altman wrote that much research was ‘seriously
flawed through the use of inappropriate designs, unrep-
resentative samples, small samples, incorrect methods of
analysis and faulty interpretation’.
Echoing the realisation by Sackett and colleagues
that much of medical practice lacked evidence of effec-
tiveness and that much research was inadequate, Altman
concluded: ‘We need less research, better research, and
research done for the right reasons’. Now, 22 years later,
the call to arms has been repeated in the EBM Manifesto
for Better Healthcare.4
Assessing the quality of research
A cornerstone of evidence-based practice is the ability
to assess the quality of the evidence and the research
that underpins it—often easier said than done. Glasziou
and colleagues published an editorial in the BMJ, aimed
at helping clinicians and researchers to assess research.5
They suggested five general principles:
1. Different types of research are needed to answer
different types of clinical questions.
2. Irrespective of the type of research, systematic re-
views are necessary.
3. Adequate grading of the quality of evidence goes
beyond the categorisation of research design.
4. Assessment of the benefit to harm balance should
draw on a variety of types of research.
5. Clinicians need efficient search strategies for
identifying reliable clinical research.
Empirical evidence of bias. Dimensions of
methodological quality associated with
estimates of treatment effects in controlled
trials
The relevance of the methods used in clinical trials to the
results that emerge was highlighted in this seminal paper
by Schulz and colleagues.6 They found that knowledge
of treatment allocation inflated the effect of an inter-
vention by an average of 41%. Failure to blind group
allocation could overestimate the intervention effect by
17%. This paper was influential in defining the effects
of systematic bias on research outcomes and in showing
why critical appraisal matters.
What is the evidence that postgraduate
teaching in EBM changes anything?
A systematic review
With growing acceptance of the importance of
evidence-based practice, it soon followed that doctors
need skills to appraise, interpret and apply research
findings to their clinical practice. Most medical schools
worldwide now include some element of EBM teaching.
But what is the evidence that teaching EBM affects
anything?
Downloaded from http://ebm.bmj.com/ on November 23, 2017 - Published by group.bmj.com
This systematic review of 23 studies showed that
stand-alone teaching improved knowledge, but not
skills, attitude, or behaviour.7  Clinically integrated
teaching improved all four of these domains. The
authors proposed a hierarchy of evidence-based health-
care teaching and learning activities:
Level 1—interactive and clinically integrated
activities.
Level 2(a)—interactive but classroom-based activities.
Level 2(b)—didactic but clinically integrated
activities.
Level 3—didactic, classroom or standalone teaching.
Clarke and colleagues have since published an over-
view of systematic reviews, which supports these find-
ings and highlights the need to implement effective
teaching strategies.
EBM manifesto for better healthcare
The manifesto was a response to systematic bias,
wastage, error and fraud in research relevant to patient
care. Jointly published in the BMJ4 and Evidence-
Babsed  Medicine, the manifesto is an invitation to
contribute towards better evidence by creating a list of
priorities and sharing the lessons from achievements
already made. The manifesto steps necessary to develop
trusted evidence were refined through consultation and
requires the evidence-based community to focus atten-
tion on strategies that could most improve the quality
of healthcare.
EBM: a commentary on common criticisms
This was the first systematic appraisal of some common
criticisms of EBM8 Following  a systematic database
search and feedback from seminars (delivered by David
Sackett), Straus and McAlister identified three limitations
unique to EBM, including limited time and resources, the
need to develop new skills and a paucity of evidence
that EBM is effective.
They said that many of what they called ‘pseudolim-
itations’ and criticisms of EBM often stem from misper-
ceptions or misrepresentations, for example, that EBM is
an ivory tower concept and that only randomised trials
or systematic reviews constitute evidence. They cited
evidence from frontline clinicians that refuted the first
claim and showed that the question determines the best
type of evidence to answer it, thus disproving the second
criticism.
They concluded that  clinicians should have better
access to evidence in practice and that the way evidence
was described and shared with patients needed to be
improved. They also pointed out the lack of evidence of
the impact of EBM on healthcare and patient outcomes
that needs addressing.
General practitioners’ perceptions of the route
to EBM: a questionnaire survey
This paper demonstrated rapid adoption of the ethos
and philosophy of EBM by primary care doctors but
also identified some of the early barriers to its imple-
mentation.9 These included awareness of the available
resources and lack of time. Access to available tech-
nologies was also a major problem; for example, only
Evid Based Med December 2017 | volume 22 | number 6 |
EBM Learning
around 20% of general practitioners at that time had
access to the Internet and key bibliographic databases
such as Medline.
Most of those surveyed had some understanding of
the technical terms used in EBM, however, more than
two-thirds felt unable to explain the meaning of these
terms. Respondents thought that the best way to move
towards EBM was by using evidence-based guidelines or
protocols developed by colleagues.
Evidence-based guidelines or collectively
constructed ‘mindlines?’ Ethnographic
study of knowledge management in
primary care
This paper was one of the first empirical assessments of
how general practitioners use findings from scientific
research in their daily practice and decision making.10
Using a mixed methods approach, Gabbay and le May
found that few practitioners went through the steps
associated with the traditional model of evidence-
based healthcare (eg, the 5 A’s), including accessing
newly published knowledge.
They observed that practitioners preferred short-
cuts and relied on ‘mindlines’ (‘collectively reinforced,
internalised guidelines’). These mindlines were devel-
oped over time not through reading of literature, but
predominantly by their experiences and interactions
with colleagues, opinion leaders, pharmaceutical repre-
sentatives, patients and other sources of knowledge.
However, they stressed that practitioners were
professionally responsible for ensuring that mind-
lines are underpinned by research evidence, and that
‘knowledge of key opinion leaders, from medical or
nursing school onwards, is based on research and
experiential evidence and wherever appropriate follow
the evidence-based healthcare model’.
Conclusion
Our list is not designed to be exhaustive; you may disa-
gree with our top 10, as we certainly did. We, however,
found it useful to discuss the papers that we think matter
and are essential to developing and understanding the
use of evidence in healthcare. We hope that this list will
evolve and would welcome suggestions to enhance it.
Contributors  DN drafted the manuscript. All authors
made edits and agreed on the final manuscript.
Funding  DN has received expenses and fees for his
media work. He holds grant funding from the NIHR
School of Primary Care Research and the Royal
College of General Practitioners. On occasion, he
receives expenses for teaching EBM. CH has received
expenses and fees for his media work. He holds grant
funding from the NIHR, the NIHR School of Primary
Care Research, The Wellcome Trust and the WHO. On
occasion, he receives expenses for teaching EBM and is
also paid for his GP work in NHS out of hours.
Disclaimer  The views expressed in this commentary
represent the views of the authors and not necessarily
those of their host institution, the NHS, the NIHR or the
Department of Health.
203
 Downloaded from http://ebm.bmj.com/ on November 23, 2017 - Published by group.bmj.com
EBM Learning
Competing interests  None declared.
Provenance and peer review  Commissioned; internally
peer reviewed.
© Article author(s) (or their employer(s) unless
otherwise stated in the text of the article) 2017. All
rights reserved. No commercial use is permitted unless
otherwise expressly granted.
References
1. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based
medicine: what it is and what it isn't. BMJ 1996;312:71–2.
2. Evidence-Based Medicine Working Group. Evidence-based
medicine. A new approach to teaching the practice of medicine.
JAMA 1992;268:2420–5.
3. Altman DG. The scandal of poor medical research. BMJ
1994;308:283–4.
204 Evid Based Med December 2017 | volume 22 | number 6 |
4. Heneghan C, Mahtani KR, Goldacre B, et al. Evidence based
medicine manifesto for better healthcare. BMJ 2017;357:j2973.
5. Glasziou P, Vandenbroucke JP, Chalmers I. Assessing the
quality of research. BMJ 2004;328:39–41.
6. Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence
of bias. Dimensions of methodological quality associated
with estimates of treatment effects in controlled trials. JAMA
1995;273:408–12.
7. Coomarasamy A, Khan KS. What is the evidence that
postgraduate teaching in evidence based medicine changes
anything? A systematic review. BMJ 2004;329:1017.
8. Straus SE, McAlister FA. Evidence-based medicine: a
commentary on common criticisms. CMAJ 2000;163:837–41.
9. McColl A, Smith H, White P, et al. General practitioner's
perceptions of the route to evidence based medicine: a
questionnaire survey. BMJ 1998;316:361–5.
10. Gabbay J, le May A. Evidence based guidelines or collectively
constructed “mindlines?” Ethnographic study of knowledge
management in primary care. BMJ 2004;329:1010–3.
 Downloaded from http://ebm.bmj.com/ on November 23, 2017 - Published by group.bmj.com

Ten essential papers for the practice of

evidence-based medicine
David Nunan, Jack O Sullivan, Carl Heneghan, Annette Pluddemann,
Jeffrey Aronson and Kamal Mahtani

Evid Based Med published online November 23, 2017

Updated information and services can be found at:

http://ebm.bmj.com/content/early/2017/11/23/ebmed-2017-110854

These include:

References This article cites 10 articles, 7 of which you can access for free at:
http://ebm.bmj.com/content/early/2017/11/23/ebmed-2017-110854
#BIBL
Email alerting Receive free email alerts when new articles cite this article. Sign up in the
service box at the top right corner of the online article.

Notes

To request permissions go to:

http://group.bmj.com/group/rights-licensing/permissions

To order reprints go to:

http://journals.bmj.com/cgi/reprintform

To subscribe to BMJ go to:

http://group.bmj.com/subscribe/