Professional Documents
Culture Documents
Checklist For Quasi
Checklist For Quasi
EXPERIMENTAL STUDIES
(NON-RANDOMIZED
EXPERIMENTAL STUDIES)
JBI is an international research organisation based in the Faculty of Health and Medical Sciences at the
University of Adelaide, South Australia. JBI develops and delivers unique evidence-based information,
software, education and training designed to improve healthcare practice and health outcomes. With over
70 Collaborating Entities, servicing over 90 countries, JBI is a recognised global leader in evidence-based
healthcare.
© JBI, 2020. All rights reserved. JBI grants use of these Critical Appraisal Checklist for Quasi-Experimental Studies - 2
tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
JBI CRITICAL APPRAISAL CHECKLIST FOR
QUASI-EXPERIMENTAL STUDIES
Reviewer______________________________________ Date_______________________________
© JBI, 2020. All rights reserved. JBI grants use of these Critical Appraisal Checklist for Quasi-Experimental Studies - 3
tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
© JBI, 2020. All rights reserved. JBI grants use of these Critical Appraisal Checklist for Quasi-Experimental Studies - 4
tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
EXPLANATION FOR THE CRITICAL APPRAISAL TOOL
FOR QUASI-EXPERIMENTAL STUDIES
How to cite: Tufanaru C, Munn Z, Aromataris E, Campbell J, Hopp L. Chapter 3: Systematic reviews of
effectiveness. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available
from https://synthesismanual.jbi.global
Critical Appraisal Tool for Quasi-Experimental Studies (Experimental Studies without random
allocation)
1. Is it clear in the study what is the ‘cause’ and what is the ‘effect’ (i.e. there is no
confusion about which variable comes first)?
Ambiguity with regards to the temporal relationship of variables constitutes a threat to the internal validity
of a study exploring causal relationships. The ‘cause’ (the independent variable, that is, the treatment or
intervention of interest) should occur in time before the explored ‘effect’ (the dependent variable, which is
the effect or outcome of interest). Check if it is clear which variable is manipulated as a potential cause.
Check if it is clear which variable is measured as the effect of the potential cause. Is it clear that the ‘cause’
was manipulated before the occurrence of the ‘effect’?
© JBI, 2020. All rights reserved. JBI grants use of these Critical Appraisal Checklist for Quasi-Experimental Studies - 5
tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
Control groups offer the conditions to explore what would have happened with groups exposed to other
different treatments, other than to the potential ‘cause’ (the intervention of interest). The comparison of
the treated group (the group exposed to the examined ‘cause’, that is, the group receiving the intervention
of interest) with such other groups strengthens the examination of the causal plausibility. The validity of
causal inferences is strengthened in studies with at least one independent control group compared to
studies without an independent control group. Check if there are independent, separate groups, used as
control groups in the study. [Note: The control group should be an independent, separate control group, not
the pre-test group in a single group pre-test post-test design.]
5. Were there multiple measurements of the outcome both pre and post the
intervention/exposure?
In order to show that there is a change in the outcome (the ‘effect’) as a result of the
intervention/treatment (the ‘cause’) it is necessary to compare the results of measurement before and
after the intervention/treatment. If there is no measurement before the treatment and only measurement
after the treatment is available it is not known if there is a change after the treatment compared to before
the treatment. If multiple measurements are collected before the intervention/treatment is implemented
then it is possible to explore the plausibility of alternative explanations other than the proposed ‘cause’
(the intervention of interest) for the observed ‘effect’, such as the naturally occurring changes in the
absence of the ‘cause’, and changes of high (or low) scores towards less extreme values even in the
absence of the ‘cause’ (sometimes called regression to the mean). If multiple measurements are collected
after the intervention/treatment is implemented it is possible to explore the changes of the ‘effect’ in time
in each group and to compare these changes across the groups. Check if measurements were collected
before the intervention of interest was implemented. Were there multiple pre-test measurements? Check
if measurements were collected after the intervention of interest was implemented. Were there multiple
post-test measurements?
© JBI, 2020. All rights reserved. JBI grants use of these Critical Appraisal Checklist for Quasi-Experimental Studies - 6
tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.
8. Were outcomes measured in a reliable way?
Unreliability of outcome measurements is one threat that weakens the validity of inferences about the
statistical relationship between the ‘cause’ and the ‘effect’ estimated in a study exploring causal effects.
Unreliability of outcome measurements is one of different plausible explanations for errors of statistical
inference with regards to the existence and the magnitude of the effect determined by the treatment
(‘cause’). Check the details about the reliability of measurement such as the number of raters, training of
raters, the intra-rater reliability, and the inter-raters reliability within the study (not to external sources).
This question is about the reliability of the measurement performed in the study, it is not about the validity
of the measurement instruments/scales used in the study. [Note: Two other important threats that weaken
the validity of inferences about the statistical relationship between the ‘cause’ and the ‘effect’ are low
statistical power and the violation of the assumptions of statistical tests. These other threats are not
explored within Question 8, these are explored within Question 9.]
© JBI, 2020. All rights reserved. JBI grants use of these Critical Appraisal Checklist for Quasi-Experimental Studies - 7
tools for research purposes only. All other enquiries
should be sent to jbisynthesis@adelaide.edu.au.