BIAS IN

MEDICAL RESEARCH

James M. Shultz MS PHD

Director, DEEP Center, Miami FL
BIAS IN MEDICAL RESEARCH

BIAS: DEFINITION

BIAS IS
SYSTEMATIC DEPARTURE
FROM THE TRUTH
BIAS IN MEDICAL RESEARCH

SELECTION
BIAS

Persons included in the study

are unrepresentative
due to sampling or selection factors.
BIAS IN MEDICAL RESEARCH

SELECTION BIAS Persons included in the study

are unrepresentative
due to sampling or selection factors.

MEASUREMENT BIAS Flaws in the process

of gathering research information
distort and bias the information.

HAWTHORNE EFFECT The presence of investigators

distorts the responses
and the data collected .

OBSERVER BIAS The observer s prior knowledge

of the subject s treatment condition
biases the observations.
BIAS IN MEDICAL RESEARCH

EXPERIMENTER Desired outcomes are conveyed

EXPECTANCY to study subjects - who then
change their behavior
based on this information.

LEAD-TIME BIAS The early detection interval

on a screening test is misinterpreted
as an increase in survival.

RECALL BIAS In a case-control study, cases

have better recall of past exposures
than controls.

CONFOUNDING Unanticipated variables

obscure the true
cause-and-effect relationship.
BIAS IN MEDICAL RESEARCH

SELECTION BIAS

NON-PARTICIPATION

Persons who agree to participate in a study differ

from persons who decline to participate

ATTRITION

Persons who stay in the study until the end differ

from persons who drop out during the study

* COHORT Study: Major source of bias

BIAS IN MEDICAL RESEARCH

SELECTION BIAS

INAPPROPRIATE SAMPLING

Selection of non-representative persons limits the

ability to generalize findings to the larger population

Health club members enrolled in a CVD study

Hospitalized patients

BERKSON S BIAS

Over-matching control subjects with cases in a

case-control study
BIAS IN MEDICAL RESEARCH

MEASUREMENT
BIAS

Flaws in the process

of gathering research information
distort and bias the information.
BIAS IN MEDICAL RESEARCH

MEASUREMENT BIAS

ASKING LEADING QUESTIONS

HAWTHORNE EFFECT
The presence of investigators distorts
the responses and the data collected

Include a control or
SOLUTION: placebo group
BIAS IN MEDICAL RESEARCH

OBSERVER
BIAS

The observer s prior knowledge

of the subject s treatment condition
biases the observations.
BIAS IN MEDICAL RESEARCH

OBSERVER BIAS

INVESTIGATORS KNOW THE

TREATMENT ASSIGNMENTS OF
THE SUBJECTS A PRIORI

Double blinding
SOLUTION: (masking)
BIAS IN MEDICAL RESEARCH

EXPERIMENTER
EXPECTANCY

Desired outcomes are conveyed

to study subjects - who then
change their behavior
based on this information.
BIAS IN MEDICAL RESEARCH

EXPERIMENTER EXPECTANCY

STUDY SUBJECTS ARE TIPPED OFF

TO THE STUDY GOALS OR DESIRED
OUTCOMES

SYNONYM:
PYGMALIAN EFFECT

Double blinding
SOLUTION: (masking)
BIAS IN MEDICAL RESEARCH

LEAD-TIME
BIAS

The early detection interval

on a screening test is misinterpreted
as an increase in survival.
BIAS IN MEDICAL RESEARCH

LEAD-TIME BIAS

SCREENING TESTS ALLOW EARLY

DETECTION AND DIAGNOSIS BUT
THE EARLIER DETECTION INTERVAL
CANNOT BE CLAIMED AS AN
EXTENSION OF SURVIVAL TIME.

* SCREENING TEST: Major source of bias

Compute true survival

SOLUTION: time
BIAS IN MEDICAL RESEARCH

RECALL BIAS

In a case-control study,
cases have better recall
of past exposures
than controls.
BIAS IN MEDICAL RESEARCH

RECALL BIAS

CASES HAVE DISEASE AND CAN

RECALL MORE DETAILS ABOUT
PAST EXPOSURES THAN CONTROLS
WHO ARE DISEASE-FREE.

* CASE-CONTROL: Major source of bias

Multiple confirmatory
SOLUTION: information sources
BIAS IN MEDICAL RESEARCH

LATE-LOOK
BIAS

The earliest and most severe

cases are lost to follow-up.
BIAS IN MEDICAL RESEARCH

LATE-LOOK BIAS

INITIAL DISEASE CASES DIE PRIOR

TO INTERVIEW OR SURVEILLANCE

NEW DISEASE INVESTIGATION:

Major source of bias

SOLUTION: Stratify cases by severity

BIAS IN MEDICAL RESEARCH

CONFOUNDING

Unanticipated variables obscure

the true cause-and-effect relationship.
BIAS IN MEDICAL RESEARCH

CONFOUNDING

THE TRUE RISK FACTOR FOR THE

DISEASE OF INTEREST IS RELATED
TO A CONFOUNDER VARIABLE.

UNCRITICAL ANALYSES MAY APPEAR

TO SHOW A RELATIONSHIP BETWEEN
THE CONFOUNDER AND THE DISEASE.

Stratify data analyses on

SOLUTION: levels of the confounder
BIAS IN MEDICAL RESEARCH

BIAS
SAMPLE PROBLEMS
BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

An investigator from a pharmaceutical company takes

time to introduce himself to subjects who have been
randomized to receive his company s new medication
the focus of the experimental trial.

He congratulates them for their good fortune

in being able to use this new medication and
suggests they will have a very favorable experience.

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

EXPERIMENTER EXPECTANCY
BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A pharmaceutical company conducts

a trial of a new medication.

One alarming observation that introduces

a source of bias is the high rate of drop-out
found for the group receiving the control medication
(33% drop-out on the control medication,
7% drop-out on the new medication).

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

SELECTION BIAS (ATTRITION)

BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A consulting firm works with a hospital network to improve
customer care. Patient satisfaction is continuously monitored.

Satisfaction ratings increase significantly at one network hospital

that is visited by a very supportive and friendly team of consultants
-- during the time the team is observing and monitoring on-site.

Satisfaction is unchanged at the other network hospitals.

However, as soon as the consultants depart,

patient satisfaction drops rapidly back to baseline levels.

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

HAWTHORNE EFFECT
 BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A pharmaceutical company conducts a trial

of a promising new anti-hypertensive medication.
Subjects are sorted fairly to receive the company s new
angiotensin-2 receptor blocker (experimental group)
or a proven antihypertensive medication (control group).

Data from the study suggest that the new medication

lowers blood pressure significantly compared to the
control medication. However, the fact that clinicians knew
which medication each patient was taking may have
influenced their measurements.
What bias is operating here?
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

OBSERVER BIAS
 BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A pharmaceutical company conducts a trial

of a promising new medication
that could generate substantial profits.

Company executives carefully scrutinize the

performance of the clinicians working in the study.

Knowing they are under observation, these

clinicians are less likely to report adverse side-effects
for the patients receiving the company s new medication.

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

HAWTHORNE EFFECT
 BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A study is conducted to assess race/ethnicity in

relation to etiology of dementia in the elderly.

Two independent groups of geriatric psychiatrists review

clinical assessments and imaging results for all subjects.
One group knows the subjects race; the other does not.

The review group that knows the subjects race reports

significantly more hypertensive etiology for the subset of
African American subjects than the other review group.

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

OBSERVER BIAS
BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A study is conducted at UCSF to explore the relationship between
consumption of a caffeinated energy drink by medical students
in relation to residency placement in highly-selective residencies.

The study finds that a higher proportion of students drinking the

energy beverage successfully match in neurosurgery residencies.

However, there is a possible source of bias:

more males than females apply to neurosurgery and
more males than females drink the energy beverage.

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

CONFOUNDING
BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A prospective cohort study revealed

a potentially causal relationship
between elevated serum cholesterol (>240 mg/dl)
and onset of diabetes type 2 (RR=1.91).

However, when the investigators reanalyzed the data,

stratifying study subjects into two groups
(obese and non-obese) based on BMI,
no significant relationship between cholesterol and diabetes
was apparent for either group
(Obese: RR = 1.09; Non-obese: RR = 0.98)

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

CONFOUNDING
BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A prospective cohort study revealed

a potentially causal relationship
between cigarette smoking
and onset of liver cirrhosis (RR=1.91).

However, when the investigators reanalyzed the data,

stratifying study subjects into two groups
based on habitual alcohol consumption (high/low),
no significant relationship between smoking and cirrhosis
was apparent for either group
(High Alcohol Use: RR = 1.09; Low Alcohol Use: RR = 0.98)

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

CONFOUNDING
BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A prospective cohort study revealed

a potentially causal relationship
between alcohol consumption (>240 mg/dl)
and onset of lung cancer (RR=1.91).

However, when the investigators reanalyzed the data,

stratifying study subjects into two groups,
cigarette smokers and nonsmokers,
no significant relationship between alcohol use and lung cancer
was apparent for either group
(Smokers: RR = 1.09; Nonsmokers: RR = 0.98)

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Confounding

D. Observer Bias

E. Selection Bias

CONFOUNDING
 BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

A new screening test is introduced for a rapidly-

progressive cancer for which no treatment is available.
Survival following clinical diagnosis is typically 6 months.

With the new screening test, patients are detected,

on average, 6 months earlier. Life expectancy after
initial detection by screening test is about one year.

Developers boldly claim that the screening test doubles

the survival time of patients with this deadly cancer.

What bias is operating here?

BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Lead-time Bias

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Lead-time Bias

D. Observer Bias

E. Selection Bias

LEAD-TIME BIAS
BIAS IN MEDICAL RESEARCH

BIAS DEFINITION PROBLEMS

Two companies in the same industry have almost

identical workforce characteristics. Both companies
contract with the same occupational medicine provider.
Company A mandates annual employee physicals.
Company B does not.
The provider analyzes the cardiovascular disease
experiences of the two workforces. Demographics and
risk factors are comparable for both companies.
Company A has a lower incidence rate for myocardial
infarction and a lower CVD death rate.
What bias is operating here?
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Lead-time bias

D. Observer Bias

E. Selection Bias
BIAS IN MEDICAL RESEARCH

WHAT BIAS IS OPERATING HERE?

A. Experimenter Expectancy

B. Hawthorne Effect

C. Lead-time Bias

D. Observer Bias

E. Selection Bias

LEAD-TIME BIAS