ARTICLE IN PRESS

Current Anaesthesia & Critical Care (2004) 15, 227–234

www.elsevier.com/locate/cacc

RESEARCH METHODOLOGY

How to perform a systematic review$

H.H.G. Handolla,, A.F. Smithb

a
Teesside Centre for Rehabilitation Sciences, University of Teesside, The James Cook University Hospital,
Marton Road, Middlesbrough TS4 3BW, UK
b
Department of Anaesthetics, Royal Lancaster Infirmary, Ashton Road, Lancaster LAI 4RP, UK

KEYWORDS Abstract Through the identification, critical appraisal and summary of evidence,
Systematic reviews; systematic reviews play a pivotal role in making sense of the underlying evidence on
Evidence based which to base healthcare decisions. In this article we outline how to perform a
medicine; Cochrane review, namely a systematic review produced with the support of the
Critical appraisal; Cochrane Collaboration and published in The Cochrane Library. As well as describing
Cochrane the basic principles and methodology of doing a Cochrane review, we identify some
Collaboration of the challenges involved and highlight some issues of more specific relevance to
reviews in anaesthesia.
r 2004 Elsevier Ltd. All rights reserved.

Introduction information.2 Systematic reviews differ from tradi-

The importance of evidence-based medicine (EBM),
which has been defined as ‘the conscientious,
explicit, and judicious use of current best evidence
in making decisions about the care of individual
patients’,1 is increasingly recognized. Sound re-
search evidence is required to underpin clinical
governance and facilitate a critical, evaluative
healthcare culture. Central to EBM is the location,
assessment, summary and interpretation of the
most reliable available evidence from primary
studies. This is the key role played by systematic
reviews, which are fundamental to making sense of
the enormous and rising quantity and complexity of
$
DOI of original article 10.1016/j.cacc.2003.12.002
Corresponding author. Tel.: +44-131-242-3499; fax: +44-131-
242-6467.
E-mail address: h.handoll@ed.ac.uk (H.H.G. Handoll).
tional narrative reviews in that they address a
clearly defined question, and use systematic
and explicit methods to identify, select and
critically appraise all relevant research, to
collect and analyse data from the primary
studies included in the review, and to present
results and draw conclusions.2,3 They are often
termed meta-analyses when statistical techniques
have been used to pool the results of the included
studies.
Although systematic reviews are performed in
many contexts, this article will illustrate the
principles by which they are produced by referring
to the processes of the Cochrane Collaboration.
This is an international, not-for-profit organization,
established in 1992, that aims to help people
make well-informed decisions about healthcare
by the preparation, maintenance and dissemination
of good-quality systematic reviews of the effects

0953-7112/$ - see front matter r 2004 Elsevier Ltd. All rights reserved.
doi:10.1016/j.cacc.2004.08.002
 ARTICLE IN PRESS
228
of healthcare interventions (http://www.cochrane.
org). The Cochrane Collaboration was named after
Archie Cochrane, a British professor of Public
Health, who in the late 1970s became interested
in the application of rigorous evaluative methods
for healthcare interventions. He favoured rando-
mised controlled trials (RCTs), where random
methods are used to allocate patients to the
intervention groups under comparison, because
they are less susceptible to bias and thus are more
likely to provide reliable evidence.4,5 Subsequently,
he challenged the medical profession to compile
critical summaries ‘by speciality or subspeciality,
adapted periodically, of all relevant randomised
controlled trials’.6 The Collaboration was formed to
meet this need and has grown steadily since its
inception. Issue 4, 2003 of the Cochrane Database
of Systematic Reviews (CDSR) contains over 1800
completed reviews. These often reflect the inter-
ests of the volunteer clinicians, academics and
others who prepare the reviews, but most areas of
healthcare are now represented, including anaes-
thesia. The CDSR is one component of The Cochrane
Library, an electronic publication that is available
on the internet and on CD-ROM (see Appendix A for
details on access). The reviews, termed Cochrane
reviews, have a standardized format and are
prepared using purpose-designed software (Review
Manager) provided by the Collaboration.
Preparation and upkeep of Cochrane reviews are
the main activities of 49 international Collaborative
Review Groups (CRGs). Of these, the Cochrane
Anaesthesia Review Group (CARG) is responsible for
supporting the production of Cochrane reviews in
anaesthesiology; specifically, the areas of anaes-
thesia, peri-operative medicine, intensive care
medicine, pre-hospital medicine, resuscitation
and emergency medicine. Reflecting the late
establishment of CARG (in 2000) compared with
some other CRGs and/or overlapping scopes, the
support and, crucially, editorial input for some
Cochrane reviews on anaesthesia are shared with
other CRGs. Currently (November 2003) CARG has
10 published reviews, 36 protocols and nearly 40
registered titles.
In this article, we outline how to perform a
Cochrane review, namely a systematic review pro-
duced with the extensive support provided by the
Cochrane Collaboration and published in the CDSR.
Stages in the production of a Cochrane
review
The eight key steps of systematic review produc-
tion are listed in Table 1. The process may seem
H.H.G. Handoll, A.F. Smith
Table 1 Stages of a systematic review.
1. The title—framing the review question
2. Forming the review team
3. The protocol—deciding on and defining
inclusion and exclusion criteria, and methods
4. Exhaustive search for material
5. Appraisal and quality assessment of material
6. Data extraction
7. Summarizing and synthesis (combining the
results)
8. Interpretation
unfamiliar and complex initially. However, it can be
helpful to think about a systematic review as one
would a clinical trial. Such an investigation would
have a well-defined question, a protocol setting out
in advance what the investigators intend to do, and
statements of how the results would be analysed. A
systematic review follows the same principles and,
in fact, the term secondary research has been
applied to systematic reviewing, in contrast to
primary research such as clinical trials.
While the general principles and methodology
apply to all systematic reviews of healthcare
interventions, we highlight the various ways in
which doing a review under the aegis of the
Cochrane Collaboration assist the delivery of a
relevant and reliable product. Key aspects of
relevance are examining questions and using out-
comes that matter to people making choices in
healthcare, enabling broad access and keeping up
to date. Key to reliability are the various ways of
minimizing bias, such as through scientific rigour,
transparent processes, broad participation and
avoiding conflicts of interest, and an emphasis on
a quality peer-reviewed product that has met
rigorous standards.
The title
Would-be Cochrane reviewers must first propose
their title to CARG. If the title is approved, it will
be registered and listed under the Group’s activ-
ities on the Library. However, this is more than a
formality since defining the research question is
crucial to any research study. Even the formal
registration of a title for a Cochrane review can
result in several benefits. These include ensuring
relevance—that the question is worth asking;
enabling delivery—checking that the scope of the
review is realistic and achievable; and avoiding
duplication—for there is enough to be reviewed
without repeating what already has been done or is
underway. In cases of duplication, there may be
 ARTICLE IN PRESS
How to perform a systematic review
scope for meaningful contribution to already
existing work—CARG will mediate in such circum-
stances.
A good question specifies the intervention to be
studied, the sort of population, the setting and the
outcomes to be used. Not only does this help focus
the reviewers’ minds on what is most important, it
also helps guide the search strategy for articles (see
locating the evidence below). For instance, the
question ‘Is propofol better than thiopentone?’ is
broad and unwieldy. The question ‘Does propofol
lead to fewer unplanned hospital admissions than
thiopentone in women undergoing day case laparo-
scopic sterilization?’ is much more precise.
The review team
There is no doubt that collaborative efforts
enhance the end product and, in our experience,
the process can be more instructive and enjoyable.
As well as protecting against various sources of bias
and error that can result from single person efforts,
the perspectives of others—especially consumers,
and from different professions and countries—in-
terested in the topic are invaluable. If requested,
the editorial team of CARG are often able to
suggest potential co-reviewers. In reality, practical
considerations usually limit the membership, both
in terms of numbers (generally 2–4 are manage-
able) and the specific expertise of the review team.
Input and feedback from a broad spectrum of
people at various stages of the review process,
especially at the editorial and external review of
the protocol, is a key strength of the Cochrane
process.
The protocol
Once the title has been approved, reviewers are
invited to submit a protocol for their review. The
basic format of a Cochrane protocol (displayed in
Table 2) helps reviewers to set out the essential
background and define the aims and intended
methods of the review. The prior specification in
the protocol of the objectives and related hypoth-
eses, the selection criteria (study design—usually
RCTs, participants, interventions and outcomes),
and methods of the systematic review is a key
element of quality control and strengthens the
conclusions of the eventual review.
The methods include a description of the
intended search strategy for the identification of
relevant studies; CARG’s Trial Search Co-ordinator
routinely advises reviewers at this stage. In addi-
tion to feedback on draft protocols, reviewers are
assisted by the provision of a comprehensive
guidebook on preparing systematic reviews (The
Cochrane Reviewers’ Handbook)7 and the avail-
229
Table 2 Sections of a Cochrane protocol*.
Title
Background
Objectives
Criteria for considering studies for this review
Types of studies
Types of participants
Types of interventions
Types of outcome measures
Search strategy for identification of studies
Methods of the review
References
*
These form part of the eventual review.
ability of specific software (Review Manager).
There may also be an opportunity to attend a
Cochrane workshop, such as ‘Developing a Protocol
for a Review’. More specific help from CARG
includes a comprehensive ‘Tips for reviewers’
document which details the support and resources
available from the CARG editorial office. Essen-
tially, the interaction with the editorial team at
this stage brings great benefits to both reviewers
and editors, allowing the development of a proto-
col that makes sense, is methodologically sound,
and feasible. A summary of the editorial review
process for titles, protocols, completed reviews
and updates may be found on the CARG website
(see Appendix B).
Locating the evidence
A comprehensive, unbiased search for all relevant
primary studies is fundamental since serious bias
may result if trials are missed through inadequate
search strategies.8 This need will not be met by a
quick trawl through MEDLINE. Searching of a
number of electronic bibliographic databases is
essential, and a proper search strategy must be
developed. CARG’s Trials Search Co-ordinator can
provide expert advice on this. Further material can
be located by checking the reference lists at the
end of retrieved articles, by hand-searching
through journals which tend to publish relevant
studies, and through trials registries. The Cochrane
Collaboration has, through substantial and continu-
ing efforts, including systematic retrospective and
prospective handsearching of hundreds of journals,
amassed the world’s largest database of controlled
trials (the Cochrane Central Register of Controlled
Trials), published in The Cochrane Library (see
Appendix A). A core activity of the editorial base of
CARG is the identification of trials and the
incorporation of those that fall within the scope
of the CARG into a specialized register for use
 ARTICLE IN PRESS
230
within systematic reviews in anaesthesia. As well as
benefiting from these, reviewers are encouraged to
conduct their own searches, and in particular to
draw on their specialist knowledge and contacts
and attendance at conferences.
Assessing the evidence
Basing the selection of studies for inclusion into the
review on clearly stated ‘a priori’ and consistent
criteria is fundamental to systematic reviews.3,8 As
in clinical trials, deviations from the protocol
should be kept to a minimum; those which do
become necessary should be described and justi-
fied. Another key aspect of systematic reviews is
the formal critical appraisal of the methodological
quality of the included studies. This is necessary to
limit bias in conducting the systematic review, gain
insights into the validity of individual studies, and
assist the interpretation of findings. The validity of
a study is the extent to which its design and
conduct are likely to prevent systematic errors or
bias.
Quality assessment, however, is an imperfect
science. It is frequently hampered by the inade-
quate reporting of trials; the application of
standards for reporting trials such as the CONSORT
statement9 should help in the future. Many quality
checklists and scoring schemes have been devised,
some of which are quite complex, but there is only
limited empirical evidence of a relationship be-
tween trial outcomes and a very few of the specific
criteria generally used to assess the risk of
systematic bias. The main finding is that trials with
inadequate concealment of allocation tend to
result in larger estimates of effect than RCTs with
adequately concealed allocation.10 In the latter
trials, the method of randomization is such that the
trial investigators have no foreknowledge of, and
thus potential influence on, allocation of trial
participants to the intervention groups under
comparison. Other key influences on validity are
blinding of trial investigators and outcome asses-
sors, and how dropouts (losses to follow-up) and
treatment non-compliers (including protocol viola-
tions) are handled in the analyses. One important
related concept is intention-to-treat analysis,
where the outcomes of trial participants are
analysed according the group to which they were
allocated at randomization.
Despite the lack of a precise knowledge of the
consequences of various methodological deficien-
cies, the insights gained through a critical and
structured evaluation of trial methodology provide
some guides to the trustworthiness of the evidence.
Quality assessment also enables the systematic
examination of the effects of various aspects of
H.H.G. Handoll, A.F. Smith
trial methodology (such as the assessor blinding) on
the results of a review. To assist reviewers, the
CARG has a scoring scheme that covers aspects of
internal validity and applicability. The latter
includes the adequate description of trial inclusion
criteria, participants, interventions and outcomes.
Independent quality assessment undertaken
by at least two reviewers is essential to assure
objectivity.
Extracting the data
The collection of clinically relevant outcomes, as
opposed to surrogate or intermediate outcomes,
and specifying these including identifying primary
outcomes in the protocol are stressed by the CARG.
Also emphasized is the use of a data extraction
form to facilitate the collection of outcome
measures and results, and characteristics of the
study and study participants in a standardized and
reproducible way from primary study documenta-
tion. Such forms are designed to provide a quick
overview of each study as well as to assist the
transfer of information into various sections of a
Cochrane review. CARG, which provides a standard
example that can be readily adapted, encourages
reviewers to submit a copy of their data extraction
form with their protocol. Again, it is recommended
that data extraction using pre-piloted forms is done
independently by at least two reviewers, with an
explicit procedure for dealing with discrepancies.
Reviewers are encouraged to request further data
and information on trial methodology from trialists
as necessary.
Rigorous extraction of the results of a trial can be
very revealing but also frustrating. There is a great
variation in what trialists report, or fail to report,
in terms of actual results. Common failings include
the presentation of means without standard devia-
tions for continuous outcomes, and percentages
without the numbers from which these were
calculated.
Presenting and analysing the results
Drafting a list of comparisons and associated
outcomes at the protocol stage can help to focus
on what is needed to address the aims of the
review. This is encouraged by CARG and again
serves to strengthen review findings through the
pre-specification of hypotheses, including sub-
group analyses, and outcome measures. This ‘table
of comparisons’ will usually need some adjustment
at the review stage, reflecting the availability of
studies and data. Nonetheless, it serves as a basis
for presenting the results in the review text,
tables and, if appropriate and possible, in the
analyses. The basic structure of Cochrane reviews
 ARTICLE IN PRESS
How to perform a systematic review
and use of Review Manager software greatly assists
this process.
A systematic review may present either a
descriptive (qualitative) account of the included
evidence or a quantitative synthesis, or both.
Where the study interventions and results are
similar enough to warrant it, pooling of data
(meta-analysis) is conducted. Qualitative accounts,
still produced using rigorous methodology, are
usually preferable where data are sparse and/or
of poor quality.
There has been a growing recognition that this
stage of the review can be quite challenging.
Support is available from the CARG editorial team,
which includes a statistical editor, and the provision
of examples and feedback on draft reviews from
CARG. This stage is also covered in detail in the
Cochrane Reviewers’ Handbook.7
Discussing the review findings and drawing
conclusions
The discussion should contain a summary of the
main findings and reviewers’ comments on the
methodological limitations of the included studies
and of their own review, if applicable. It also
provides the opportunity to set the results in the
context of other work and to bring out the
implications for clinical practice and future re-
search. In the interpretation of results, one
important distinction that is frequently made is
that between ‘no evidence of an effect’ and
‘evidence of no effect’. Guidance on interpreting
the results and drawing appropriate conclusions
based on a careful appraisal of the evidence is
available and, in the authors’ experience, often a
vital aspect of editorial feedback. There is a
general emphasis on providing conclusions that
focus on the systematically appraised and summar-
ized evidence in the review; and essentially, to
inform rather than advise. Also, it is important to
realize that in many areas, despite meticulous
searching and analysis, results can be inconclusive
(usually due to lack of primary evidence). Although
this may be disappointing, it is still a valuable
finding as it provides a sound foundation for future
research.
Completing the process
Completed draft reviews are submitted by the lead
reviewers to the CARG editorial base for editorial
processing. The draft review is generally sent to
the editor and external referees, often specialists
in the review topic, who commented on the
protocol, as well as the CARG’s statistical editor
and one or more consumer representatives. The
input from members of the CARG consumer group is
231
particularly valuable in terms of keeping the focus
on things that matter to patients. Overall, the
rigorous editorial process, including the require-
ment of a structured response from the reviewers,
helps to ensure that only those reviews meeting the
CARG standards are included in the Group’s
module. This latter is submitted on a quarterly
basis for publication in The Cochrane Library.
Afterwards: maintaining reviews and responding
to comments and criticisms
Electronic publication of Cochrane reviews enables
corrections and changes to be made to reviews in
the light of new evidence, new review methodol-
ogy, comments and criticisms and so on. There is
great emphasis in the Collaboration on keeping
reviews up to date and the editorial policy of the
CARG reflects this. Most reviews are updated every
1 or 2 years to ensure their continuing relevance.
Reflections from undertaking Cochrane
reviews in anaesthesia
Different reviews present different challenges and
raise different issues, but there are some common
themes. Some of the common problems that hinder
or may even prevent the completion of Cochrane
reviews are listed in Table 3. Many can be avoided
by proper preparation, good time management and
taking advantage of the training resources avail-
able to Cochrane reviewers. It is important to
realize that to get a good-quality product on which
people making healthcare decisions can rely re-
quires considerable and usually shared effort.
Table 4 presents brief accounts of some of the
issues concerning context, mainly raised in our
three Cochrane reviews11–13 on anaesthesia, that
have some general implications for systematic
reviews in this area.
Conclusions
Performing a systematic review, as with all properly
conducted research, is a serious undertaking that
requires commitment and diligence to see it
through. Both of us have found that performing
systematic reviews, greatly assisted by the support
provided by the Cochrane Collaboration, has been
instructive and rewarding. In particular it brings an
appreciation of methodological quality in all forms
of research. Conducting a systematic review gen-
erally gives reviewers a deep understanding of their
subject often over and above the basic results. The
Collaboration’s global focus and its involvement of
 ARTICLE IN PRESS
232 H.H.G. Handoll, A.F. Smith

Table 3 Common problems hindering completion of Cochrane reviews.

Problems Potential solutions

Overambitious scope  Timely editorial input on the scope and seeking and accepting advice on what is
involved from the start should help to avoid this.
 A redefined scope may be considered but the question posed by the review needs to
remain relevant.
 Unless very experienced with dedicated support and resources, it is best to tackle one
well-defined question (review) at a time.
Unrealistic timescale  As above. Good time management is important in all research and it is important not to
underestimate the time required to perform a review. In particular, allowance needs to
be made for communication with and contributions from co-reviewers and CARG, and
for revisions in response to editorial feedback.
 Often, it is more effective to allocate yourself a good block of time to accomplish a
stage of the review. And if it is a case of ‘getting round to it’, the answer is ‘just do
it!’.
Inexperience  This is not a problem unless reviewers do not take advantage and heed of the extensive
training resources available.
 With the help of CARG, inexperienced reviewers can usually be ‘matched’ with more
experienced reviewers.
 Would-be reviewers will draw greater benefit from their taking time to learn the
necessary skills to perform a review. These skills and insights derived from a
methodical and critical approach extend beyond the review itself.
Isolation  This should not be a problem in this electronic age.
 Timely and regular communication is key here. CARG can advise on and usually help
solve many of the typical problems that hinder the progress of reviews. An email can
often make the difference.
Language  The main language of the Collaboration is English, and much of the evidence
considered is also in English. However, CARG, whose editorial base is in Denmark, and
the Cochrane Collaboration as a whole, is emphatically an international organization
with international perspectives and the inclusion of evidence from non-English
language reports is encouraged.
 Where possible, additional allowance and help will be given to those whose first
language is not English but there will be no differences in the quality standards applied
ultimately.
Misperception of  While input from CARG is supportive and far more generous than that of any journal,
purpose of editorial the responsibility for the review lies with the reviewers. CARG aims to ensure that only
process good-quality and valid protocols and reviews are published and its editorial processes
reflect this.
 Fulfilling the requirements of the editorial processes is an important part of
completing any Cochrane review.

consumers have brought us valuable new perspec-
tives and its rigorous yet informal style creates a
pleasant working environment. Further, the in-
creasing availability of The Cochrane Library means
that our work has the potential of reaching more
people than specialist journals and thus likely to be
of greater use in the short and, when kept up to
date, long term.
Acknowledgements
We thank Jane Cracknell for her helpful feedback
on this article. Both authors are active Cochrane
reviewers and have received editorial support from
CARG. Andrew Smith is a founding editor of CARG.
The views in this article are those of the authors
and not necessarily of CARG.
Appendix A. The Cochrane Library,
contents and access
The Cochrane Library, which is published on a
quarterly basis, is acknowledged to be a key
resource for information on the effectiveness of
healthcare interventions. As well as including the
CDSR, CENTRAL and full information about the
Cochrane Collaboration, it contains five other
databases of relevance to evidence based medicine.
 ARTICLE IN PRESS
How to perform a systematic review 233

Table 4 Some background issues on anaesthesia reviews.

When evaluating anaesthesia interventions, it is very important to consider their context. Here we consider some
aspects:

Getting the ‘full picture’

Reviews should focus on clinically important outcomes and, indeed, outcomes that matter to patients. Trials
often record/report surrogate or intermediate outcomes such as peri-operative hypotension, body temperature
and arterial blood gases. These can be relatively quickly and easily measured and are taken as being predictive of
important clinical outcomes. However, the relationship is usually not an exact one and some conditions, such as
operative hypotension, may be remedied to reduce the risk of a serious clinical event occurring.12 Thus the
results of surrogate outcomes may be misleading.
Adverse effects should always be sought but otherwise confining outcome to the simple ascertainment of
anaesthesia (essentially pain relief) is still unlikely to provide the full picture. Trials and reviews failing to assess/
report the success of the procedure, such as reduction of fractures11 and/or dislocations, for which anaesthesia is
required are thus potentially misleading.
Ascertainment of outcome also needs to be within an appropriate time frame. While tempting to focus on very
short-term outcomes, such as pain during fracture manipulation, it is important to collect long-term outcomes
given that inadequate anaesthesia could have long-term consequences; for instance, resulting from inadequate
fracture reduction.11

Confounding
On the whole, trials of anaesthesia are especially vulnerable to confounding, which is where factors other than
the interventions under test influence the trial results. In particular, lack of comparability in care programmes,
such as an imbalance in the choice of surgical technique and/or important differences in postoperative care can
hinder/prevent a reliable appraisal of the effectiveness of the anaesthesia interventions under test.
Strict control of confounders is the hall mark of explanatory trials. Such trials determine whether an intervention
works (its efficacy) under ideal conditions in a well-defined and co-operative patient group. In contrast,
pragmatic trials aim to test effectiveness, which is the extent to which a specific intervention, when used under
ordinary circumstances, does what it is intended to do. Trials often fall somewhere in between these two study
designs.

Applicability and relevance

Healthcare is generally not static and awareness of context is always important when considering the implications
for current practice and future research. For example, day-case anaesthesia had changed considerably during the
20 plus year time period covered by the trials in our review of premedication for anxiety in adult day surgery.13
Techniques, drugs and more especially the organization of day-case units have changed, as well as the population
characteristics. However, though some early trials may not be appropriate to current practice in the UK, they still
may be pertinent to practice in other countries. For instance, not all countries in the world are able to afford the
drugs commonly used for day-case anaesthesia in affluent nations. In addition, a useful role played by a
systematic review can be as an archive of all the studies published on a particular issue. The ‘rediscovery’ of some
‘historic’ treatment is also not unknown.

Whose agenda?
There is growing awareness and now some empirical evidence of the influence of industry on the research agenda
and research findings.14 It is thus prudent, when forming review questions and appraising trial findings to bear this
in mind. For example, to question whether a comparison was marketing-driven rather than addressing a key
clinical issue or whether there was some partiality in the reporting of study results, such as emphasis on a
particularly favourable outcome. Similar reservations should probably apply to research in general.

All UK NHS health professionals have full and
free access through the National electronic
Library for Health (NeLH) gateway/site: http://
www.nelh.nhs.uk. We highly recommend this
website. In addition, some countries (by
November 2003: Australia, England and Wales,
Finland, Ireland, Norway and Spain) have
bought a license to allow the residents of the
whole country to use The Cochrane Library without
charge.
 ARTICLE IN PRESS
234 H.H.G. Handoll, A.F. Smith

Appendix B. The Cochrane Anaesthesia
Review Group
Readers interested in conducting a Cochrane re-
view on anaesthesia are encouraged to contact the
Cochrane Anaesthesia Review Group as well as
looking at the website to see the current activities
of this group and for further details on what is
involved:
Jane Cracknell
Review Group Coordinator,
Department of Anaesthesiology,
Bispebjerg University Hospital,
Bispebjerg Bakke 23,
DK 2400 Copenhagen NV,
Denmark.
Tel.: +45-35-31-30-14; fax: +45-35-31-33-39;
E-mail: carg@cochrane.dk
Web: http://www.cochrane-anaesthesia.suite.dk.
References
1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson
WS. Evidence based medicine: what it is and what it isn’t.
BMJ 1996;312(7023):71–2.
2. Mulrow CD. Rationale for systematic reviews. BMJ
1994;309(6954):597–9.
3. Mulrow CD. The medical review article: state of the science.
Ann Intern Med 1987;106(3):485–8.
4. Sackett DL, Wennberg JE. Choosing the best research design
for each question. BMJ 1997;315(7123):1636.
5. Sibbald B, Roland M. Understanding controlled trials. Why
are randomised controlled trials important? BMJ
1998;316(7126):201.
6. Cochrane AL. 1931–1971: a critical review, with particular
reference to the medical profession. In: Teeling-Smith G,
Wells N, editors. Medicines for the Year 2000. London: Office
of Health Economics, 1979. pp. 1–11.
7. Clarke M, Oxman AD, (editors). Cochrane Reviewers’ Hand-
book 4.2.0 [updated March 2003]. http://www.cochrane.
dk/cochrane/handbook/handbook.htm (accessed 12
November 2003).
8. Egger M, Smith GD. Bias in location and selection of studies.
BMJ 1998;316(7124):61–6.
9. Moher D, Schulz KF, Altman DG. The CONSORT statement:
revised recommendations for improving the quality of
reports of parallel-group randomised trials. Lancet
2001;357(9263):1191–4.
10. Kunz R, Vist G, Oxman AD. Randomisation to protect against
selection bias in healthcare trials (Cochrane Methodology
Review). In: The Cochrane Library, Issue 3. Chichester, UK:
Wiley, 2003.
11. Handoll HHG, Madhok R, Dodds C. Anaesthesia for treating
distal radial fracture in adults (Cochrane Review). In:
The Cochrane Library, Issue 4. Chichester, UK: Wiley,
2003.
12. Parker MJ, Handoll HHG, Griffiths R, Urwin SC. Anaesthesia
for hip fracture surgery in adults (Cochrane review). In: The
Cochrane Library, Issue 4. Chichester, UK: Wiley 2003.
13. Smith AF, Pittaway AJ. Premedication for anxiety in adult
day surgery (Cochrane Review). In: The Cochrane Library,
Issue 4. Chichester, UK: Wiley, 2003.
14. Abbasi K, Smith R. No more free lunches. BMJ
2003;326(7400):1155–6.