Because Cochrane Reviews address questions about the effects of health care, they focus

primarily on randomized trials and randomized trials should be included if they are feasible
for the interventions of interest (MECIR Box 3.3.b). Randomization is the only way to
prevent systematic differences between baseline characteristics of participants in different
intervention groups in terms of both known and unknown (or unmeasured) confounders
(see Chapter 8), and claims about cause and effect can be based on their findings with far
more confidence than almost any other type of study. For clinical interventions, deciding who
receives an intervention and who does not is influenced by many factors, including
prognostic factors. Empirical evidence suggests that, on average, non-randomized studies
produce effect estimates that indicate more extreme benefits of the effects of health care than
randomized trials. However, the extent, and even the direction, of the bias is difficult to
predict. These issues are discussed at length in Chapter 24, which provides guidance on when
it might be appropriate to include non-randomized studies in a Cochrane Review.

C10:  Including randomized trials (Mandatory)

Include randomized trials as
eligible for inclusion in the
review,  if it is feasible to
conduct them to evaluate the
interventions and outcomes of
interest.
Randomized trials are the best study design for
evaluating the efficacy of interventions. If it is feasible
to conduct them to evaluate questions that are being
addressed by the review, they must be considered
eligible for the review. However, appropriate exclusion
criteria may be put in place, for example regarding
length of follow-up.

The decision of whether non-randomized studies (and what type) will be included is decided
alongside the formulation of the review PICO. The main drivers that may lead to the
inclusion of non-randomized studies include: (i) when randomized trials are unable to address
the effects of the intervention on harm and long-term outcomes or in specific populations or
settings; or (ii) for interventions that cannot be randomized (e.g. policy change introduced in
a single or small number of jurisdictions) (see Chapter 24). Cochrane, in collaboration with
others, has developed guidance for review authors to support their decision about when to
look for and include non-randomized studies (Schünemann et al 2013).

=== CHAPTER 24 ===

Cochrane Reviews of interventions have traditionally focused mainly on systematic reviews

of randomized trials because they are more likely to provide unbiased information about the
differential effects of alternative health interventions than NRSI. Reviews of NRSI are
generally undertaken when the question of interest cannot be answered by a review of
randomized trials. Broadly, we consider that there are two main justifications for including
NRSI in a systematic review, covered by the flow diagram shown in Figure 24.1.a:
1. To provide evidence of the effects (benefit or harm) of interventions that can feasibly
be studied in randomized trials, but for which available randomized trials address the
review question indirectly or incompletely (an element of the GRADE approach to
assessing the certainty of the evidence, see Chapter 14, Section 14.2) (Schünemann et al
2013). Such non-randomized evidence might address, for example, long-term or rare
outcomes, different populations or settings, or ways of delivering interventions that
better match the review question.
2. To provide evidence of the effects (benefit or harm) of interventions that cannot be
randomized, or that are extremely unlikely to be studied in randomized trials. Such non-
randomized evidence might address, for example, population-level interventions (e.g. the
effects of legislation; (Macpherson and Spinks 2008) or interventions about which
prospective study participants are likely to have strong preferences, preventing
randomization (Li et al 2016).

A third justification for including NRSI in a systematic review is reasonable, but is unlikely
to be a strong reason in the context of a Cochrane Review:

3. To examine the case for undertaking a randomized trial by providing an explicit

evaluation of the weaknesses of available NRSI. The findings of a review of NRSI may
also be useful to inform the design of a subsequent randomized trial (e.g. through the
identification of relevant subgroups).

Two other reasons sometimes described for including NRSI in systematic reviews are:

4. When an intervention effect is very large.

5. To provide evidence of the effects (benefit or harm) of interventions that can feasibly
be studied in randomized trials, but for which only a small number of randomized trials
is available (or likely to be available).
Randomization (RCT, Quasi, and Stratified):
http://www.bandolier.org.uk/booth/glossary/RCT.html

Randomized trials are the preferred design for studying the effects of healthcare interventions
because, in most circumstances, a high-quality randomized trial is the study design that is
least likely to be biased. All Cochrane Reviews must consider the risk of bias in individual
primary studies, whether randomized trials or NRSI (see Chapter 7, Chapter 8 and Chapter
25). Some biases apply to both randomized trials and NRSI. However, some biases are
specific (or particularly important) to NRSI, such as biases due to confounding or selection of
participants into the study (see Chapter 25). The key advantage of a high-quality randomized
trial is its ability to estimate the causal relationship between an experimental intervention
(relative to a comparator) and outcome. Review authors will need to consider (i) the strengths
of the design features of the NRSI that have been used (such as noting their potential to
estimate causality, in particular by inspecting the assumptions that underpin such estimation);
and (ii) the execution of the studies through a careful assessment of their risk of bias. The
review team should be constituted so that it can judge suitability of the design features of
included studies and implement a careful assessment of risk of bias.