Critical Review of Epidemiologic Studies

Ephrem M
Outline for critiquing epidemiologic studies

A. Collection of data
– What was the context of the study?
– What were the objectives of the study?
– What was the primary exposure of interest?
Was this accurately measured?
– What was the primary outcome of interest?
Was this accurately measured
– What type of study was conducted?
 Outline cont..
– Describe the sources of the study population, process of
subject selection, sample size, and ratio of proposition to
comparison subjects.

– Could there have been bias in the selection of the study

subjects?
How likely was this bias?
– Could there have been bias in the collection of information?
– How likely was this bias?

– What provisions were made to minimize the influence of

confounding factors prior to the analysis of the study?
Were these provisions sufficient?
 Outline cont..
B. Analysis of data
– What methods were used to control confounding
during data analysis?
Were these methods sufficient?

– What measures of association were reported in

this study?
– What measures of statistical stability were
reported in this study?
 Outline cont..
C. Interpretation of data
– What were the major results of this study?
– How is the interpretation of these results affected by
selection bias, information bias, and confounding?

Discuss both the direction and magnitude of any bias

–
What were the authors’ main conclusions? Were they
justified by the findings?

– To what larger population can the results of this study be

generalized?
 A. COLLECTION OF DATA

1. What was the context of the study?

• This qes deals with the investigators’ motivation of

the study.

• E.g observations that stimulated the study may have

come from
– laboratory research
– clinical reports
– other epidemiological studies.
 What was the context of the study?
Cont…

• The answer to this question gives the reader a sense

of the particular contribution made by the study at
hand.

• For example:
– the study may be the first of its kind
– it may be replicating results of an earlier study in
a different population, with larger numbers, or by
using more refined methods.
 2. What were the objectives of the study?

• deals with the purpose of the study/ hypothesis being

tested.

• Examples of hypotheses
-unprotective receptive anal intercourse increases the
risk of HIV-1 transmission among homosexual men
 What were the objectives of the
study? Cont…
• Whenever possible, the hypothesis should quantify
the relationship between the exposure and disease.

• For example, a reduction of 1000 ppm or more of

lead in soil accessible to children will decrease their
blood lead levels by at least 3 mg/dL.

• Specific hypothesis are easier than nonspecific ones

to test and refute
3. What was the primary exposure of interest?
Was this accurately measured?

The exposure of interest is the factor that either

causes, prevents, or treats the outcome
4. What was the primary outcomes of interest?
Was this accurately measured?

• consider whether the outcome was accurately defined

and measured

• look for measures taken to improve the accuracy of

the outcome data

• E.g use of a combination of signs and symptoms,

tests to define cases of disease etc.
 5. What type of study was conducted?

• Which design was used?

• Important to know which design was used because

certain designs are more prone to bias than others
6. Describe the source of the study population, process
of subject selection, sample size,

• This allows the reader to evaluate many important

topics like:

– the potential for selection bias and confounding

– the statistical power of the study
– generalizability of the results.

• The source of the study population is the underlying

population from which the subjects were drawn.
.
7. Could there have been bias in the selection of the
study subjects? How likely was this bias?

• Selection bias arises from systematic differences in

selecting the study groups, or in following the selected
groups.

• These differences lead to an effect estimate among

subjects in the study that is different from the estimate
obtainable from the entire underlying population.

• Losses greater than 20%, especially if they occur among

different types of subjects than those who remain, call the
validity of the study into question.
9. What provisions were made to minimize the influence of
confounding factors prior to the data analysis?
Were these provisions sufficient?

• When evaluating a study, the reader should review

the information on confounder that was collected.

• The methods to control for confounding prior to data

analysis include:
– randomization,
– restriction,
– matching, and
– use of the same source population for the
compared groups.
 B. ANALYSIS OF DATA

10. What methods were used to control confounding

during the data analysis?
Were these methods sufficient?
• During analysis, the main ways to control
confounding are standardization, stratification,
matched analysis, and multivariate analysis.
 11. What measures of association were
reported in this study?
What measures of statistical stability were reported in
this study?

• Hypothesis testing and interval estimation are used to

assess the precision of study findings and to evaluate
the role of chance in epidemiologic studies.

• P values and confidence intervals are the two main

ways to assess the role of chance in epidemiologic
research.
13. How is the interpretation of these results affected
by information bias, selection bias, and confounding?

Discuss both the direction and magnitude of any bias

The reader should gather all of the relevant information

from the other sections of the critique to determine the
net effect of any information bias, selection bias, and
confounding that may be present in the study.

• A small amount of bias or confounding will usually not

have a major impact on the study results, but

• a large amount of bias or residual confounding could

completely alter the findings.
 How is the interpretation of these results
affected cont…
•The impact of bias and confounding will also depend
on the true size of the association.

•They can account entirely for a weak association, but

they are unlikely to account entirely for a strong one.

It may be useful to conduct a “sensitivity analysis” in

order to determine the magnitude of the bias.
This analysis determines the robustness of the results by
examining the extent to which they are affected by
assumption about the amount of bias.
15. Did the discussion section adequately
address the limitations of the study?

A self-critical examination of the study methods and

findings should be included in the discussion section
of the article
16. What were the authors’ main conclusions? Were
they justified by the findings?

• The conclusion are a brief summary of the findings

and their implications

• In addition the authors sometimes include

recommendations for future research and public
health policy

• All conclusions should be supported by the study

findings
 To what larger population can the results
of this study be generalized?
• The evaluation of generalizability or “external
validity” requires review of:
– the study methods –
Eg was restriction used to control for confounding?
– the composition of the study population
Eg- were minorities included?
– subject matter knowledge such as the biological
basis of the association
Eg. would the same results be expected among men
and women?
Second Assignment: Critically Review one Research Article
based on criteria of Critical Review of Epidemiologic
Studies and submit the document (group Assignment)