Boards & Beyond:

Biostatistics & Epidemiology Slides

Color slides for USMLE Step 1 preparation
from the Boards and Beyond Website

Jason Ryan, MD, MPH

2022 Edition

Boards & Beyond provides a virtual medical school curriculm used

by students around the globe to supplement their education and
prepare for board exams such as USMLE Step 1.

This book of slides is intended as a companion to the videos for

easy reference and note-taking. Videos are subject to change
without notice. PDF versions of all color books are available via the
website as part of membership.

Visit www.boardsbeyond.com to learn more.

Copyright © 2022 Boards and Beyond

All rights reserved.

i
ii
 Table of Contents
Statistics...................................................................1
Hypothesis Testing..............................................6
Tests of Significance......................................... 11
Correlations......................................................... 14
Study Designs..................................................... 16
Risk Quantification........................................... 20
iii
Sensitivity/Specificity..................................... 25
Predictive Values............................................... 30
Diagnostic Tests................................................. 33
Bias.......................................................................... 36
Clinical Trials...................................................... 41
Evidence-Based Medicine............................. 46
iv
 Statistics
Statistical Distribution

Random Blood Glucose Healthy Subjects

Statistics 90
87
101
115
112
101
90
87
101
115 90
112 87
101 101
115
112
101
90
87
101
115
112
101
110 92 110 92 110 92 110 92
Jason Ryan, MD, MPH 105 85 105 85 105 85 105 85
93 79 93 79 93 79 93 79
92 100 92 100 92 100 92 100
95 99 95 99 95 99 95 99
88 86 88 86 88 86 88 86
112 102 112 102 112 102 112 102

Statistical Distribution Central Tendency

Normal or Gaussian Distribution
• Center of normal distribution
• Three ways to characterize:
• Mean: average of all numbers
• Median: middle number of data set when all lined up in order
• Mode: most commonly found number
No.
Subjects

Blood Glucose Level

Shutterstock

Mean and Mode Median

• Six blood pressure readings: • Odd number of data elements in set
• 90, 80, 80, 100, 110, 120 • 80-90-110
• Mean = (90+80+80+100+110+120)/6 = 96.7 • Middle number is median = 90
• Mode is most frequent number = 80 • Even number of data elements
• 80-90-110-120
• Halfway between middle pair is median = 100
• Must put data set in order to find median

Shutterstock
Shutterstock

1
Central Tendency Central Tendency
Skewness
• Positive or negative based on location of tail
Mean
Median
Mode
Negative Skew Positive Skew
Mode is always highest point Mode
If distribution even, mean/median=mode
Mode Median
No. Median
Subjects No.
No. Mean
Mean Subjects
Subjects

Blood Glucose Level

Blood Glucose Level Blood Glucose Level

Central Tendency Dispersion

Key Points

• If distribution is equal: mean=mode=median

• Mode is always at peak
• In skewed data:
• Mean is always furthest away from mode toward tail
• Median is between Mean/Mode
• Mode is least likely to be affected by outliers
• Adding one outlier changes mean, median
• Only affects mode if it changes most common number
• One outlier unlikely to change most common number 100mg/dl 100mg/dl

Shutterstock

Dispersion Measures Standard Deviation

• Used to describe dispersion of data in a data set
• Standard deviation σ = Σ(x-x)2
• Variance n-1
• Z-score
x-x = difference between data point and mean
Σ(x-x) = sum of differences
Σ(x-x)2 = sum of differences squared
n = number of samples

2
Standard Deviation Standard Deviation
σ = Σ(x-x)2
n-1 • Percentage within standard deviation
Group 1 Difference Group 2 Difference
(mean=10) from mean Squared (mean=10) from mean Squared
9 -1 1 5 -5 25
8 -2 4 6 -4 16
9 -1 1 9 -1 1
10 0 0 10 0 0
11 1 1 12 2 4
12 2 4 13 3 9
10 0 0 15 5 25
10 0 0 14 4 16
11 96 -1σ +1σ -2σ +2σ -3σ +3σ

σ = 11 = 1.24 68% 95% 99%

σ = 96 = 3.7
7 7

Standard Deviation Sample Question

• Fasting glucose is measured in 200 medical students
• The mean value is 100 mg/dL with a standard deviation of 5
mg/dL
• The values are normally distributed
• How many students have a glucose > 110 mg/dL?
• 110 is two standard deviations away from mean
• 2.5% of students are in this range (1/2 of 5%)
• 2.5% of 200 = 5 students
3σ 2σ 1σ
68% 68%
2.5% 2.5%
95%
95%
99% 99%
- +2
2σ σ

95%

Variance Z score
• Measure of dispersion in a data set • Describes a single data point
• Related to standard deviation • How far a data point is from the mean
• Average degree to which each point differs from the mean • Z score of 0 is the mean
• Z score of +1 is 1SD above mean
Standard Deviation Variance • Z score of -1 is 1SD below mean
σ = Σ(x-x)2 σ2 = Σ(x-x)2
n-1 n-1 2σ 1σ
3σ
0
-1 +1
-2 +2
-3 +3

3
Z score True Population Mean
Example
• Suppose test grade average (mean) = 79 • Suppose 100 samples taken to estimate population mean
• Standard deviation = 5 glucose level
• Your grade = 89 • Total population = 1M individuals
• Your Z score = (89-79)/5 = +2 • How close is sample mean to true population mean?
• Standard error of the mean
• Confidence intervals

2σ 1σ
3σ
0
-1 +1
-2 +2
-3 +3

Standard Error of the Mean Confidence Intervals

• How far is dataset mean from true population mean • Range in which 95% of repeated measurements would be
• Standard deviation divided by square root of n expected to fall
• Small standard deviation → less SEM → closer to true mean • 95% chance true population mean falls within this range
• More samples (n) → less SEM → closer to true mean • Related to standard error of the mean (SEM)

σ
SEM = CI95% = Mean +/- 1.96*(SEM)
n

Confidence Intervals Confidence Intervals

Example
• n = 16 • Don’t confuse standard deviation with confidence intervals
• Mean = 10 • Standard deviation is for a dataset
• SD = 4 • Suppose we have ten samples
CI95% = Mean +/- 1.96*(SEM) • These samples have a mean and standard deviation
• SEM = 4/sqrt(16) = 4/4 = 1 • 95% of samples fall between +/- 2SD
• CI = 10 + 1.96*(1) = 10 + 2 • This is descriptive characteristic of the samples
• 95% of repeated means fall between 8 and 12 • Confidence intervals
• Upper confidence limit = 12 • This does not describe the samples in data set
• An inferred value of where the true mean lies for population
• Lower confidence limit = 8

4
95%
• This value often confusing
• Read carefully: What are they asking for?
• Data set or true population mean?
• Range in which 95% of measurements in a dataset fall
• Mean +/- 2SD
• Range in which true population mean likely falls?
• Confidence interval of the mean
• Mean +/- 1.96*SEM

Shutterstock

5
 Hypothesis Testing
Hypothesis Testing
• Critical element of medical research
• Determination of whether results are meaningful
• Are observed study findings real or due to chance?
Hypothesis Testing • Allows extrapolation of medical testing results to general
population
• All studies involve a subset of the total population
Jason Ryan, MD, MPH • Hypothesis testing: do study findings represent reality in the total
population?
• Usually involves comparison between different groups
• Are differences between groups real differences or due to
chance?

Hypothesis Testing Hypothesis Testing

Example Example
• MERIT-HF Trial • MIzyme protein level may be elevated in myocardial
• 3,991 patients with systolic heart failure infarction
• Patients treated with metoprolol or placebo • Can this level be used to detect myocardial infarction in ED?
• Primary endpoint: all-cause mortality • Samples of MIzyme level obtain in 100 normal subjects
• Placebo group: all-cause mortality 11.0% • Samples of MIzyme level obtain in 100 subjects with
• Metoprolol group: all-cause mortality 7.2%
• Are these true differences or due to chance?
myocardial infarction
• Is metoprolol associated with lower mortality or is this due to • Mean level in normal subjects: 1 mg/dl
chance? • Mean level in myocardial infarction patients: 10 mg/dl
• Hypothesis for testing: metoprolol is associated with lower • Can this test be used to detect myocardial infarction in the
mortality general population?

Hypothesis Testing Hypothesis Testing

Example
• Is the mean value of MIzyme in myocardial infraction • Mathematically calculates probabilities (5%, 50%)
subjects truly different? • Probability the two means truly different
• Or was the difference in our experiment simply due to • Probability the difference is due to chance in our
chance? experiment
• Hypothesis for testing: MIzyme level is higher in myocardial • Math is complex (don’t need to know)
infarction patients • Probabilities by hypothesis testing depend on:
• Difference between means normal/MI
• Scatter of data
• Number of subjects tested

Shutterstock Shutterstock

6
Hypothesis Testing Hypothesis Testing
Size of difference between groups Scatter
Magnitude of difference in means influences likelihood Key Point: Scatter of data points
that difference between means is due to chance influences the likelihood that there is
a true difference between means

d d
d d

Normal MI Normal
MI Normal MI Normal MI
MIzyme level
MIzyme level MIzyme level MIzyme level

Hypothesis Testing Hypothesis Testing

Number of samples
Key Point: Number of data
points influences the • Two possibilities for MIzyme levels in normal and MI
likelihood that there is a true
difference between means patients
• #1: MIzyme DOES NOT distinguish between normal/MI
• Levels are the same in both groups
d
dd
d ddd
d • #2: MIzyme DOES distinguish between normal/MI
d d dd
d • Levels are not the same in both groups
• Null hypothesis (H0) = #1
• Alternative hypothesis (H1) = #2

Normal MI Normal MI
MIzyme level MIzyme level

Hypothesis Testing Hypothesis Testing

Four possible outcomes
• In reality, either H0 or H1 is correct • #1: There is a difference in reality and our experiment
• In our experiment, either H0 or H1 will be deemed correct detects it
• The alternative hypothesis (H1) is found true by our study
• Hypothesis testing determines likelihood our experiment • #2: There is no difference in reality and our experiment also
matches with reality finds no difference
• The null hypothesis (H0) is found true by our study
• #3: There is no difference in reality but our study finds a
difference
• This is an error! Type 1 (α) error
• #4: There is a difference in reality but our study misses it
• This is also an error! Type 2 (β) error

7
Hypothesis Testing Hypothesis Testing
% Likelihood based on:
% Likelihood based on: Difference between means normal/MI
Difference between means normal/MI Scatter of data
Scatter of data Reality Number of subjects tested
Number of subjects tested
H1 H0

Experiment
Reality
H1 H0 H1 Power α
Experiment

H1 H0 β H0 Correct
H0
Power = Chance of detecting difference
α = Chance of seeing difference that is not real
β = chance of missing a difference that is really there
Power = 1- β

P-value P-value
• Used to accept or reject a hypothesis • Represents chance that null hypothesis is correct
• Calculated based on differences between groups • No difference between means
• Magnitude of difference between groups • If p < 0.05 we usually reject the null hypothesis
• Scatter of data • Difference in means is “statistically significant”
• Number of samples

P-value P-value
Example
• Represents chance of “false positive” finding • MERIT-HF Trial
• No difference in reality • 3,991 patients with systolic heart failure
• Study finds a difference that is not real • Placebo group: all-cause mortality 11.0%
• Must be low (< 0.05) for findings to be positive • Metoprolol group: all-cause mortality 7.2%
• Similar but different from α (significance level) • P-value = 0.00009
• α set by study design • Reject null hypothesis that differences are due to chance
• P value calculated by comparison of groups
• Accept alternative hypothesis that metoprolol reduces
mortality
• “Positive study”

8
Power Power Calculation
• Chance of finding a difference when one exists • Studies try to maximize power to detect a true difference
• Also called rejecting the null hypothesis (H0) • In study design, you have little/no control over:
• Power is increased when: • Scatter of data
• Increased sample size • Difference between means
• Large difference of means • You DO have control over number of subjects
• Less scatter of data • Number of subjects chosen to give a high power
• Commonly used power goal is 80%
• This is called a power calculation

Detecting a Difference Statistical Errors

Type 1 (α) error
• Study 1 • False positive
• Group A: 10% • Finding a difference or effect when there is none in reality
• Group B: 20%
• P-value: 0.001 • Rejecting null hypothesis (H0) when you should not have
• “Study detected a difference” • Example: researchers conclude a drug benefits patients, but
• Study 2 it does not
• Group A: 10%
• Group B: 20%
• P-value: 0.25
• “Study did not detect a difference”

Pixy.org/Public Domain

Type 1 Errors Statistical Errors

Causes Type 2 (β) error
• Example: researchers conclude a drug benefits patients, but • False negative
it does not • Finding no difference/effect when there is one in reality
• Random chance • Accepting null hypothesis (H0) when you should not have
• Most studies: chance of alpha error 5% • Example: Researchers conclude a drug does not benefit patients (p > 0.05)
• One out of 20 times → error • Subsequent study finds that it does
• Improper research techniques

Shutterstock

9
Statistical Errors
Type 2 (β) error
• Common cause: too few patients
• Need lots of patients to have sufficient power
• Especially when differences are small
• Significant p-value (< 0.05) more likely with:
• Large difference between groups
• Large number of patients

10
 Tests of Significance
Comparing Groups
• Many clinical studies compare groups
• Often find differences between groups
• Different mean ages

Tests of Significance • Different mean blood levels

• Are differences real or due to chance
Jason Ryan, MD, MPH • Key element of hypothesis testing
• Accept or reject null hypothesis

Shutterstock

P-value Key Point

• Calculated based on differences between groups • P-value for group comparison calculated based on group
• Represents chance that null hypothesis is correct data
• No difference between group values • Magnitude of difference between groups
• If p < 0.05 we usually reject the null hypothesis • Scatter of data points
• Reject hypothesis of no difference between groups
• Accept alternative hypothesis of difference between groups • Number of data points
• Difference between groups is “statistically significant” • Don’t need to know the math
• Just understand principle

Shutterstock

Comparing Groups Data Types

Determination of P values
• Three key tests • Quantitative data/variables:
• T-test • Have numerical values
• 1, 2, 3, 4
• ANOVA
• Categorical data/variables:
• Chi-square • High, medium, low
• Positive, negative
• Yes, No

Public Domain

11
Data Types T-test
• Quantitative variables often reported as number • Compares two mean quantitative values
• Mean age was 62 years old • Yields a p-value
• Categorical variables often report as percentage
• 40% of patients take drug A
• 20% of patients are heavy exercisers

Public Domain Public Domain

T-test
• A researcher studies plasma levels of sodium in patients
with SIADH and normal patients. The mean value in SIADH
patients is 128 mg/dl. The mean value in normal patients is
136 mg/dl.
• Common questions:
• Which test to compare the means? (t-test)
• What p-value indicates significance? (< 0.05)
T-test
• A researcher studies plasma levels of sodium in patients
with SIADH and normal patients. The mean value in SIADH
patients is 128 mg/dl. The mean value in normal patients is
136 mg/dl.
• A p-value of 0.01 is reported. What does this mean?
• 1% probability that results are due to chance
• 1% probability that there is no true difference in means

T-test ANOVA
• A researcher studies plasma levels of sodium in patients • Analysis of variance
with SIADH and normal patients. The mean value in SIADH • Used to compare more than two quantitative means
patients is 128 mg/dl. The mean value in normal patients is • Consider:
136 mg/dl. • Plasma level of creatinine determined in non-pregnant, pregnant, and post-
partum women
• Three means determined
• The p-value is 0.20 (non-significant) - why might that be the • Cannot use t-test (two means only)
case? • Use ANOVA
• Need more patients • Yields a p-value like t-tests
• Increase sample size → increase power to detect differences

Shutterstock

12
Chi-square Confidence Intervals
• Compares two or more categorical variables • In scientific literature, means are reported with a
• When asked to choose statistical test for a dataset: beware confidence interval
of percentages • For 10 study subjects mean glucose was 90 +/- 4
• Often categorical data • 95% confidence interval = mean +/- 1.96 * SD / n = 90 +/-
• Always ask yourself whether data is quantitative or categorical
2.5
• If the study subjects were re-sampled
• Mean result would fall between 87.5 and 92.5 for 95% of
re-samples

Shutterstock

Confidence Intervals Confidence Intervals

Group Comparisons Group Comparisons
• If ranges do not overlap: statistically significant difference • Many studies report differences between groups
• Group 1 mean: 10 +/- 5; Group 2 mean: 30 +/-4 • Example: difference in systolic blood pressure between two
• Confidence intervals do not overlap groups = 10 mmHg
• Significant difference between means
• Similar to p < 0.05 for comparison of means • Can calculate confidence intervals
• Group 1 mean: 10 +/- 5; Group 2 mean: 8 +/-4 • If range includes zero, no statistically significant difference
• Confidence intervals overlap • Example:
• Difference may be significant or insignificant • Mean difference between two groups is 10.0 mmHg +/- 15.0 mmHg
• Depends on degree of overlap • Includes zero
• No significant difference between groups
• Similar to p > 0.05

Odds and Risk Ratios Real World Example

• Some studies report odds or risk ratios with confidence
intervals
• If range includes 1.0 then exposure has no significant
impact disease/outcome
• Example:
• Risk of lung cancer among chemical workers studied
• Risk ratio = 1.4 +/- 0.5
• Confidence interval includes 1.0
• Chemical work not significantly associated with lung cancer
• Similar to p > 0.05

Wiviott SD et al. Dapagliflozin and Cardiovascular Outcomes in Type 2 Diabetes.

N Engl J Med. 2019 Jan 24;380(4):347-357. doi: 10.1056/NEJMoa1812389. Epub 2018 Nov 10. PMID: 30415602.

13
 Correlations
Correlation Coefficient
Pearson Coefficient

Correlations

Lifespan
Jason Ryan, MD, MPH

Pack-years of smoking

Correlation Coefficient Correlation Coefficient

Pearson Coefficient Pearson Coefficient
• Measure of linear correlation between two variables
• Represents strength of association of two variables
• Number from -1 to +1
• Closer to 1, stronger the relationship
• (-) number means inverse relationship
Lifespan

• More smoking, less lifespan

• (+) number means positive relationship
• More smoking, more lifespan
• 0 means no relationship
Pack-years of smoking

Correlation Coefficient Correlation Coefficient

Pearson Coefficient Pearson Coefficient

Direction of Relationship
Strength of Relationship

d d
d

r = -0.5 r = +0.5 r=0

r = +0.5 r = +0.9 Negative Positive No relationship
(stronger relationship)

14
Correlation Coefficient Coefficient of Determination
Pearson Coefficient r2
• Studies will report relationships with correlation coefficient • Sometimes r2 reported instead of r
• Example: • Always positive
• Study of pneumonia patients • Indicates % of variation in y explained by x
• WBC on admission evaluated for relationship to LOS
• r = +0.5
• Higher WBC → Higher LOS
• Sometimes p value is also reported
• P < 0.05 indicates significant correlation
• P > 0.05 indicates no significant correlation

r2 = 0.6 r2 = 1
(60% variation y explained by x) (100% variation y explained by x)

15
 Study Designs
Epidemiology Studies
• Studies of populations
• Examine association of exposure with disease
• Many real world examples
Study Designs • Hypertension → stroke
• Smoking → lung cancer
• Toxic waste → leukemia
Jason Ryan, MD, MPH
• Different from clinical/research studies
• No control over exposure in epidemiology studies
• Researchers control exposure in clinical studies

Public Domain

Types of Studies Cross-Sectional Study

Determine association of exposure with disease Prevalence Study
• Cross-sectional study • Patients studied based on being part of a group
• Cohort study (prospective/retrospective) • New Yorkers, women, tall people
• Case-control study • Frequency of disease and risk factors identified at the same
time
• How many have lung cancer?
• How many smoke?
• Snapshot in time
• Patients not followed for months/years
• Main outcome of study is prevalence
• 50% of New Yorkers smoke
• 25% of New Yorkers have lung cancer

Shutterstock

Piqsels

Cross-Sectional Study Cross-Sectional Study

Prevalence Study Prevalence Study
• Easy and quick to perform • New Yorkers were surveyed to determine whether they
• May have more than one group smoke and whether they have a morning cough. The study
• 50% men have lung cancer, 25% of women have lung cancer found a smoking prevalence of 50%. Among responders,
• But groups not followed over time (e.g., years)
25% reported morning cough.
• Major disadvantage: can’t determine causal relationships
• How much smoking increases risk of lung cancer (RR) • Note the absence of a time period
• Odds of getting lung cancer in smokers vs. non-smokers (OR) • Patients not followed for 1-year, etc.
• Likely questions:
• Type of study? (cross-sectional)
• What can be determined? (prevalence of disease)

Shutterstock

16
Cross-Sectional Study Cross-Sectional Study
Prevalence Study Prevalence Study
• Researchers discover a gene that they believe leads to the
• Using a national US database, rates of lung cancer were development of diabetes. A sample of 1000 patients is
determined among New Yorkers, Texans, and Californians. Lung randomly selected. All patients are screened for the gene.
cancer prevalence was 25% in New York, 30% in Texas, and 20% Presence or absence of diabetes is determined from a
in California. The researchers concluded that living in Texas is patient questionnaire. It is determined that the gene is
associated with higher rates of lung cancer. strongly associated with diabetes.
• Key points: • Key points:
• Presence of different groups could make you think of other study types
• Note lack of time frame
• But note lack of time frame
• Study is just a fancy description of disease prevalence
• Patients not selected by disease or exposure (random)
• Just a snapshot in time

Case Series Descriptive Studies

• Purely descriptive study (similar to cross-sectional) • Case reports
• Often used in new diseases with unclear cause • Case series
• Multiple cases of a condition combined/analyzed • Cross-sectional studies
• Patient demographics (age, gender)
• Symptoms
• May identify clues about etiology
• No control group

Shutterstock

Cohort Study Cohort Study

Incidence Study Incidence Study
• Compares group with exposure to group without exposure Disease
(diabetes)
• Exposure determined before outcome is known
Exposed
• Did exposure change likelihood of disease? (obese)
• Prospective: monitor groups over time going forward No Disease

• Retrospective: look back in time at groups over time Cohort

• Usually done for common diseases (e.g., diabetes)
Disease
• Easy to find cases in different groups (diabetes)
• Can establish incidence of disease in groups Unexposed
(non-obese)

No Disease

17
Cohort Study Cohort Study
Incidence Study Incidence Study
• Main outcome measure is relative risk (RR)
• How much does exposure increase risk of disease • A group of 100 New Yorkers who smoke were identified based on
• Example results a screening questionnaire at a local hospital. These patients were
• 50% smokers get lung cancer within 5 years
• 10% non-smokers get lung cancer within 5 years
compared to another group that reported no smoking. Both
• RR = 50/10 = 5 groups received follow-up surveys asking about development of
• Smokers 5 times more likely to get lung cancer lung cancer annually for the next 3 years. The incidence of lung
cancer was 25% among smokers and 5% among non-smokers.
• Likely questions:
• Type of study? (prospective cohort)
• What can be determined? (relative risk)

Pixabay/Public Domain

Cohort Study Retrospective Cohort Studies

Incidence Study
• A group of 100 New Yorkers who smoke were identified
based on a screening questionnaire at a local hospital. These
patients were compared to another group that reported no
smoking. Hospital records were analyzed going back 5 years
for all patients. The incidence of lung cancer was 25%
among smokers and 5% among non-smokers.
• Likely questions:
• Type of study? (retrospective cohort)
• What can be determined? (relative risk)
• Challenging to identify
• Especially to distinguish from cross-sectional studies
• Patients identified “over a 5-year period”
• Cross-sectional study: outcome is prevalence of disease
• How many patients at clinic over a 5-year period have hypertension
• Retrospective cohort: outcome is incidence of disease
• Patients with exposure (e.g., smoking) identified
• How many patients developed hypertension over 5 years

Cohort Study Case-Control Study

Incidence Study
• Disadvantage: does not work with rare diseases • Compares group with disease to group without
• Imagine: • Looks retrospectively over time for exposure or risk factors
• 100 smokers, 100 non-smokers • Opposite of cohort study
• Followed over 1 year
• Zero cases of lung cancer both groups • Better for rare diseases
• In rare diseases need LOTS of patients for LONG time
• Easier to find cases of lung cancer
• Then compare to controls without lung cancer

Shutterstock

Public Domain

18
Case-Control Study Case-Control Study
• Patients not followed over time for onset of disease
Exposed • Cannot determine incidence of disease
Disease • Cannot determine relative risk
Unexposed (cases)
• Main outcome measure is odds ratio
Compare
rates of exposure
• Odds of disease exposed/odds of disease unexposed

Exposed
No Disease
Unexposed (controls)

Shutterstock

Case-Control Study Matching

• A group of 100 New Yorkers with lung cancer were
identified based on a screening questionnaire at a local
hospital. These patients were compared to another group
that reported no lung cancer. Both groups were questioned
about smoking within the past 10 years. The prevalence of
smoking was 25% among lung cancer patients and 5%
among non-lung cancer patients.
• Likely questions:
• Type of study? (case-control)
• What can be determined? (odds ratio)
• Selection of control group (matching) key to getting good
study results
• Controls should be as close to disease patients as possible
• Ideally, only difference between groups is presence or
absence of disease
• This limits confounding in results

Shutterstock

Randomized Trials How to Identify Study Types?

• Don’t confuse with case-control
• Patients identified by disease like case-control
• Exposure controlled by researchers Cross-sectional Cohort Case-Control

• Exposure assigned randomly Members of group Selection by exposure Selection by disease status
Exposure/outcome same time status Odds ratio
Snapshot in time Risk ratio
Prevalence Incidence

Shutterstock

19
 Risk Quantification
Risk of Disease
• Determined from epidemiology studies
• Cohort studies and case-control studies
• Smoking increases risk of lung cancer X percent
Risk Quantification • Exercise decreases risk of heart disease Y percent

Jason Ryan, MD, MPH

Picpedia

The 2 x 2 Table The 2 x 2 Table

Uses

• Derives from data from cohort or case control studies • Relative risk (risk ratio)
• Patients with disease and exposure = A • Odds ratio
• Patients with disease and no exposure = C • Attributable risk
• Patients without disease and exposure = B • Number needed to harm
• Patients without disease and no exposure = D

Disease
+ -
Exposure

+ A B
- C D

Relative Risk Relative Risk

• Ratio of risk in exposed group to risk in unexposed group • Risk of disease with exposure vs non-exposure
• Established from a cohort study • RR = 5
• Smokers 5x more likely to get lung cancer than nonsmokers
• Risk in exposed group = A/(A+B)
• Ranges from zero to infinity
• Risk in unexposed group = C/(C+D) • RR = 1 → No increased risk from exposure
• Relative risk = risk exposed/risk unexposed • RR > 1 → Exposure increases risk
• RR < 1 → Exposure decreases risk
Disease Disease
+ - + -
Exposure

Exposure

+ A B + A B
- C D - C D

20
Relative Risk Relative Risk
Example 1
• 10% smokers get lung cancer
Disease • 10% nonsmokers get lung cancer
+ - • RR = 1
Exposure

+ A B
- C D

Incidence Exposed A/(A+B)

RR = =
Incidence Unexposed C/(C+D)

Relative Risk Relative Risk

Example 2 Example 3
• 50% smokers get lung cancer • 10% smokers get lung cancer
• 10% nonsmokers get lung cancer • 50% nonsmokers get lung cancer
• RR = 5 • RR = 0.2
• Smoking protective!

Relative Risk Odds Ratio

• A group of 1000 college students is evaluated over ten years. • Case control studies
Two hundred are smokers and 800 are non-smokers. Over • Odds of exposure-disease/odds exposure-no-disease
the 10-year study period, 50 smokers get lung cancer • Ranges from zero to infinity
compared with 10 non-smokers. • OR = 1 → Exposure equal among disease/no-disease
• OR > 1 → Exposure increased among disease/no-disease
+ Disease
- • OR < 1 → Exposure decreased among disease/no-disease
Exposure

+
-

RR = A/(A+B) =
C/(C+D)

21
Odds Ratio Risk Ratio versus Odds Ratio
• Risk ratio is the preferred metric
Disease • Easy to understand
• Tells you how much exposure increase risk
+ -
• Not valid in case-control studies
Exposure

+ A B • RR is different depending on number cases you choose

- C D

OR = A/C = A*D
B/D B*C

Risk Ratio versus Odds Ratio Risk Ratio versus Odds Ratio
Suppose we find 100 cases and 200 controls Now suppose we find 200 cases and 200 controls
RR = 50/100 = 2.0 RR = 100/150 = 1.6
50/200 100/250

Lung Cancer Lung Cancer

+ - + -
Smoking

Smoking

+ 50 50 + 100 50
- 50 150 - 100 150
100 200 200 200

Risk Ratio versus Odds Ratio Risk Ratio versus Odds Ratio
• Risk ratio is dependent on number of cases and controls
OR does not change with case number • Invalid to use risk ratio in case-control studies
• Must use odds ratio instead
+ - + -
+ 50 50 + 100 50
- 50 150 - 100 150
100 200 200 200

OR = 50/50 = 3.0 OR = 100/100 = 3.0

50/150 50/150
c

Needpix.com

22
Rare Disease Assumption Rare Disease Assumption
• OR = RR
• Most exposed/unexposed have no disease (-)
• Few disease (+) among exposed/unexposed OR = A/C = A*D
B/D B*C

RR = A/(A+B) = A/B = A*D

C/(C+D) C/D B*C

OR = RR
When B>>A and D>>C
c

Rare Disease Assumption Attributable Risk

• Allows use of a case-control study to determine RR • Suppose 1% incidence lung cancer in non-smokers
• Commonly accepted number is prevalence < 10% • Suppose 21% incidence in smokers
• Case-control studies easier and less expensive • Attributable risk = 20%
• But odds ratio is a weak association • Added risk due to exposure to smoking
• Classic question:
• Description of case-control study
• RR reported
• Is this valid?
• Answer: only if disease is rare

Picpedia

Attributable Risk Attributable Risk

Disease
+ - Study 1 Study 2
Exposure

Risk exposed = 50% Risk exposed = 10%

+ A B Risk unexposed = 25%
Relative Risk = 2.0
Risk unexposed = 5%
Relative Risk = 2.0
- C D Attributable Risk = 25% Attributable Risk = 5%

AR = A/(A+B) – C/(C+D) RR = A/(A+B)

C/(C+D)

23
Attributable Risk Percentage Attributable Risk
• Percent of disease explained by risk factor
• Ratio of attributable risk to risk in exposed
• Suppose attributable risk for smoking and lung cancer 25% Study 1 Study 2
Risk exposed = 50% Risk exposed = 10%
• Suppose risk in smokers is 50% Risk unexposed = 25% Risk unexposed = 5%

• Indicates 50% of lung cancers explained by smoking Relative Risk = 2.0

Attributable Risk = 25%
Relative Risk = 2.0
Attributable Risk = 5%
• Can be calculated directly from attributable risk or relative Attributable Risk % = 50% Attributable Risk % = 50%

risk
ARP = AR = RR – 1 ARP = AR = RR – 1
Re RR Re RR

Number Need to Harm Attributable Risk

• Number of patients exposed for one episode of disease to
occur on average Study 1 Study 2

• Number who need to smoke for one case of lung cancer to Risk exposed = 50%
Risk unexposed = 25%
Risk exposed = 10%
Risk unexposed = 5%
develop Relative Risk = 2.0
Attributable Risk = 25%
Relative Risk = 2.0
Attributable Risk = 5%
• Equal to reciprocal of attributable risk Attributable Risk % = 50% Attributable Risk % = 50%
NNH = 4 NNH = 20
• If attributable risk to smoking is 20%, then NNH is 1/0.2 = 5
• Similar to number needed to treat calculated from clinical
trials NNH = 1
NNH = 1 AR
AR

24
 Sensitivity/Specificity
Incidence and Prevalence
• Incidence of diabetes: 1,000 new cases diabetes per year
• Prevalence of diabetes: 100,000 cases at one point in time
in a population
Sensitivity and Specificity
Jason Ryan, MD, MPH

Incidence and Prevalence Incidence and Prevalence

• Incidence rate = new cases / population at risk • For chronic diseases: prevalence >> incidence
• Determined for a period of time (e.g. one year) • For rapidly fatal diseases: incidence ~ prevalence
• Population at risk = total population – people with disease
• 40,000 people • New primary prevention programs:
• 10,000 with disease • Both incidence and prevalence fall
• 1,000 new cases per year • New drugs that improve survival
• Incidence rate = 1,000 / (40k-10k) = 1,000 cases/30,000 • Incidence unchanged
• Prevalence rate = number of cases / population at risk • Prevalence increases
• Entire population at risk

Blue Diamond Gallery

Diagnostic Tests Diagnostic Tests

• Used to identify individuals with and without disease Blood Glucose Levels
• Gold standard = best available test
Normal Subjects
• New tests compared to gold standard Diabetics
90 115 90 140 115 140
• Described by test performance versus gold standard 87 112 87 132 112 132
• Key metrics: sensitivity and specificity 101
110
101
92
101
110
110
105
101
176
110
105
105 85 105 127 180 127
93 79 93 170 199 170
92 100 92 140 100 140
95 99 95 160 143 160
88 86 88 112 168 112
112 102 112 160 102 160

Pixabay.com

25
Diagnostic Tests Diagnostic Tests

Normal Subjects Diabetics Normal Subjects Diabetics

No. No.
Subjects Subjects

Blood Glucose Level Blood Glucose Level

Diagnostic Tests Sensitivity

Disease
Disease + -
+ -
+ TP FP
Test

+ TP FP
Test

- FN TN
- FN TN

Sensitivity = TP
TP + FN

Sensitivity Sensitivity
Sensitivity = TP Sensitivity = TP
TP + FN TP + FN
Normal Subjects Diabetics Normal Subjects Diabetics

No. No.
Subjects Subjects

Blood Glucose Level Blood Glucose Level

Very sensitive Not very sensitive

26
Specificity Specificity
Specificity = TN
TN + FP
Disease
Normal Subjects Diabetics
+ -
+ TP FP
Test

No.
- FN TN Subjects

Specificity = TN Blood Glucose Level

TN + FP Very specific

Specificity Sample Question

Specificity = TN
TN + FP
• The results below are obtained from a study of test X on
patients with and without disease A. What is the sensitivity
Normal Subjects Diabetics
of test X?

No.
Subjects Disease A
+ -
Test X

+ 25 10
- 75 10
Blood Glucose Level

Not very specific

Sample Question Key Point

• The results below are obtained from a study of test X on • High sensitivity = good at ruling OUT disease
patients with and without disease A. What is the specificity • High specificity = good at ruling IN disease
of test X? • SnOUT and SpIN

Disease A
+ -
Test X

+ 25 10
- 75 10

Shutterstock

27
Finding Rare Disease Sensitivity and Specificity
• Screen with a SENSITIVE test • Use sensitive tests when you don’t want to miss cases
• Most people will be negative • Captures many true positives (at the cost of false positives)
• Result is reliable because test is sensitive • Screening of large populations
• Follow up (+) screening tests with a SPECIFIC test • Severe diseases
• Sift through all the false/true positives • Use specific tests after sensitive tests
• Confirmatory tests

Blue Diamond Gallery

Sensitivity & Specificity Sensitivity & Specificity

• Midpoint cutoff maximizes sensitivity/specificity • Degree of overlap limits maximum combined
sensitivity/specificity

Normal Diabetics Normal Diabetics Normal Diabetics

Blood Glucose Level Blood Glucose Level Blood Glucose Level

Key Point Sensitivity/Specificity

• Sensitivity and specificity are characteristics of the test
• Remain constant for any prevalence of disease Test X
Sensitivity 80%
Specificity 50%

Group 1 Group 2
Prevalence = 80% Prevalence = 20%
Disease Disease
+ - + -
+ 64 10 + 16 40
Test

Test

- 16 10 - 4 40

80 20 20 80

Shutterstock

28
Sensitivity/Specificity Sensitivity and Specificity
• “A test is negative in 80% of people who do not have the
Group 1 Group 2 disease.”
Prevalence = 80% Prevalence = 20% • True negatives; specificity

+
Disease
- +
Disease
-
• “A test is positive in 50% of the people who do have the
+ 64 10 + 16 40 disease.”
Test

Test
- 16 10 - 4 40 • True positives; sensitivity

Sens = 64/80 = 80% Sens = 16/20 = 80% Disease

Spec = 10/20 = 50% Spec = 40/80 = 50%
+ -
+ TP FP

Test
- FN TN

29
 Predictive Values
Predictive Values
• For diagnostic tests, what doctors and patients want to
know is:
• I have a positive result; what is the likelihood I have this disease?

Predictive Values • I have a negative result; what is the likelihood I don’t have this disease?
• Sensitivity and specificity do not answer these questions
Jason Ryan, MD, MPH • Need to use positive and negative predictive values

Positive Predictive Value Negative Predictive Value

Disease Disease
+ - + -
+ TP FP + TP FP
Test

Test

- FN TN - FN TN

PPV = TP Specificity = TN NPV = TN

TP + FP TN + FP TN + FN

Sample Question Key Point

• A test has a sensitivity of 80% and a specificity of 50%. The • Predictive values are dependent on the prevalence of
test is used in a population where disease prevalence is disease
40%. What is the positive predictive value? • Sensitivity and specificity are independent of prevalence of
Disease A disease
+ -
+
Test X

32 30
- 8 30

40 patients 60 patients 100 patients

PPV = TP = 32 = 52%
TP + FP 62
Shutterstock

30
Positive Predictive Value Negative Predictive Value

Test X Test X
Sensitivity 80% Sensitivity 80%
Specificity 50% Specificity 50%

Group 1 Group 2 Group 1 Group 2

Prevalence = 80% Prevalence = 20% Prevalence = 80% Prevalence = 20%
Disease Disease Disease Disease
+ - + - + - + -
+ 64 10 + 16 40 + 64 10 + 16 40
Test

Test

Test

Test
- 16 10 - 4 40 - 16 10 - 4 40

80 20 20 80 80 20 20 80

PPV = 64 = 86% PPV = 16 = 29% NPV = 10 = 38% NPV = 40 = 91%

74 56 26 44

Key Point Cutoff Point

Diagnostic Tests
PPV = TP
• PPV is higher when prevalence is higher
• NPV is high when prevalence is lower Moving cutoff this way lowers PPV
TP + FP

Normal Diabetics

No.
Subjects

Blood Glucose Level

(-) test (+) test

Shutterstock

Cutoff A Cutoff B
Cutoff Point
PPV = TP TP = 10 TP = 15 Diagnostic Tests
FP = 5 FP = 10
TP + FP PPV = 10/15 PPV = 15/25 NPV = TN
= 66% = 60%
Moving cutoff this way increases NPV
TN + FN
Normal Subjects
Diabetics

Normal Diabetics
No.
Subjects No.
Subjects

Blood Glucose Level

Blood Glucose Level

(-) test (+) test

B A

31
Sample Question Predictive Values and
Sensitivity/Specificity
• The American Diabetes Association proposes lowering the
• Predictive values vary with prevalence and
cutoff value for the fasting glucose level that indicates
sensitivity/specificity
diabetes. How will this change affect sensitivity, specificity,
• Factors that increase PPV
PPV, and NPV? Normal Subjects • High sensitivity = more TP
Diabetics
• Sensitivity: Increase • High specificity = less FP
• Specificity: Decrease • High prevalence = more TP
• PPV: Decrease
• NPV: Increase • Factors that increase NPV
No.
Subjects
• High sensitivity = less FN
• High specificity = more TN
• Decreased prevalence = less FN

32
 Diagnostic Tests
Diagnostic Tests
Special Topics
• Accuracy/Precision
• ROC Curves
• Likelihood ratios
Diagnostic Tests
Jason Ryan, MD, MPH

Shutterstock

Accuracy and Precision Accuracy and Precision

• Describe quality of measurements used as part of • More precise tests have smaller standard deviations
diagnostic test • Less precise tests have larger standard deviations
• Accuracy (validity): how closely data matches reality
• Precision (reliability): how closely repeated measurements Number or tests with result

match each other Test B

• Can have accuracy without precision (or vice versa)

Test A

10mg/dl
Precise and accurate Precise not accurate Accurate not precise Not accurate or precise

Accuracy and Precision ROC Curves

Receiver Operating Characteristic
• Random measurement errors: reduce precision • Cutoff value for positive tests determines
• Random error: some measurements okay, others bad sensitivity/specificity
• Accuracy may be maintained but lots of data scatter
• Which cutoff value maximizes sensitivity/specificity?
• Systemic errors reduce accuracy
• Imagine every BP measurement off by 10 mmHg due to wrong cuff size • ROC curves answer this question
ROC Curve
• Systemic error in data set (non-random error)
• Precision maintained but accuracy poor

33
ROC Curve ROC Curve

Normal Diabetics
Cutoff (mg/dL) Sensitivity Specificity
(%) (%)
100 80 20

110 65 42
No.
Subjects 120 54 63

130 42 75

Blood Glucose Level

ROC Curves ROC Curves

• Straight line from bottom left to top right is a bad test
• Closer the curve is to a right angle, the better the test
(True Positive Rate)
Sensitivity (%)

1-Specificity (%)
(False positive rate)

ROC Curves ROC: Area Under Curve

• Point closest to top left corner is the best cutoff to maximize • Useless test has 0.5 (50%) area under curve
sensitivity/specificity • Perfect test has 1.0 (100%) area under curve
• More area under curve = better test
• More ability to discriminate healthy from disease

34
Likelihood Ratios Likelihood Ratios

Post-test Post-test
Probability Probability LR + = Sensitivity
(-) Test (+) Test 1 - Specificity

0% 100%
LR - = 1 - Sensitivity
Specificity
Pretest
Probability Likelihood ratios tell us how much Characteristics of test like sensitivity/specificity
probability shifts with (+) or (-) test Do not vary with prevalence of disease

Likelihood Ratios Likelihood Ratios

• Serum ferritin as a screening test for iron deficiency in
LR Interpretation
children
> 10 Large increase in probability
1 No change in probability Serum Ferritin (ng/dL) Likelihood Ratio
< 15 51.8
< 0.1 Large decrease in probability
15 to 24 8.8

25-34 2.5

45-100 0.5

>100 0.08

Screening for Iron Deficiency in Early Childhood Using Serum Ferritin

in the Primary Care Setting Hannah Oatley et al. Pediatrics Dec 2018, 142 (6)

Term: “Likelihood”
• What is likelihood of disease in a person with (+) test?
• Positive predictive value
• What is likelihood of disease in a person with (-) test?
• Negative predictive value
• What is the positive likelihood ratio?
• Calculated from sensitivity/specificity
• What is the negative likelihood ratio?
• Calculated from sensitivity/specificity

Shutterstock

35
 Bias
Bias
• Systematic error in a study
• Group selection
• Measurement
Bias • Outcome assessment
• Exposure assessment
Jason Ryan, MD, MPH • Confounding

Wikimedia Commons

Selection Bias Sampling Bias

• Errors in selection or retention of study groups • Subtype of selection bias
• Usually used as a general term • Patients not representative of actual practice
• Many subtypes • Results not generalizable to clinical practice
• Example: average age many heart failure trials = 65
• Average age actual heart failure patients = 80+
• Study results may not apply

Picmpedia/org

Attrition Bias Berkson’s Bias

• Subtype of selection bias • Subtype of selection bias
• Occurs in prospective studies • Hospitalized patients chosen as treatment or control arm
• Patients lost to follow-up unequally between groups • May have more severe symptoms
• Suppose 100 smokers lost to follow-up due to death • May have better access to care
• Study may show smoking not harmful • Alters results of study
• Example:
• Study of hospitalized pneumonia patients
• Shows many patients have history of stroke
• Pneumonia associated with stroke
• Similar study in community: no association

Wikipedia/Public Domain
Pixabay/Public Domain

36
Nonresponse Bias Prevalence Bias
Participation Bias Neyman Bias
• Subtype of selection bias • Subtype of selection bias
• Occurs with survey and questionnaire studies • Exposure occurs long before disease assessment
• Non-responders not included • Patients exposed who die quickly not included
• Patients who respond may represent a selected group • Prevalence of disease based on select group of survivors

Exposure Assessment

Shutterstock

Length-time Bias Lead-Time Bias

• Patients with severe disease do not get studied because they • Screening test identifies disease earlier
die • Survival appears longer when it is not
• Example: analysis of HIV+ patients shows the disease is • Example:
asymptomatic • Average time from detection of breast lump to death = 5 years
• May overestimate survival because severe cases missed • Screening test identifies cancer earlier
• Time from detection to death = 7 years
• Example: screening program identifies only slow-growing
• Avoided by controlling for disease severity
tumors • Expected survival based on stage at detection
• Screening programs may appear more effective

OpenClipArt Public Domain

Measurement Bias Recall Bias

• Sloppy research technique • Form of measurement bias
• Blood pressure measured incorrectly in one arm • Inaccurate recall of past events by study subjects
• Protocol not followed • Common in survey studies
• Avoided by standardized data collection • Example:
• Objective, previously-tested methods • Parents of disabled children asked about lifestyle during pregnancy
• Carefully planned ahead of time • Pregnancy occurred many years ago
• Poor recall leads to inaccurate findings
• Avoided by minimizing recall timeframe

Wikipedia/Public Domain

Shutterstock

37
Observer Bias Procedure Bias
• Form of measurement bias • One group receives procedure (e.g., surgery) and another
• Investigators know exposure status of patient does not
• Examples: • More care and attention given to procedure patients
• Cardiologist interprets EKGs knowing patients have CAD • Avoided by blinding (masking)
• Pathologist reviews specimens knowing patients have cancer • Care team unaware which patients had procedure
• Avoided by blinding • Also avoided by using placebo
• Sometimes sham surgery performed

Public Domain

PxHere

Confounding Bias Stratified Analysis

Eliminates Confounding Bias
• Unmeasured factor confounds study results
• Example: Lung Cancer
• Alcohol users more like to develop lung cancer than non-users

Alcohol Use
+ -
+ 50 50
• Smoking much more prevalent among alcohol users
- 10 90
• Smoking is true cause of more cancer
• Smoking is a confounder of results RR = 5

Smokers Non-Smokers
Lung Cancer Lung Cancer
Alcohol Use

Alcohol Use
+ - + -
+ 15 35 + 15 35

- 15 35 - 15 35

RR = 1 RR = 1

Controlling for Confounders Ways to Reduce Bias

• Randomization • Randomization - limits confounding and selection bias
• Ensures equal variables in both arms • Matching of groups
• Matching • Blinding
• Case-control studies
• Careful selection of control subjects • Crossover studies
• Goal is to match case subjects as closely as possible
• Choose patients with same age, gender, etc.

Public Domain

38
Crossover Study Crossover Study
• Subjects randomly assigned to a sequence of treatments
• Group A: placebo 8 weeks –> drug 8 weeks
• Group B: drug 8 weeks –> placebo 8 weeks Placebo Drug
Washout
• Subjects serve as their own control Period
• Avoids confounding (same subject!) Group 1

• Drawback is that effect can “carry over”

• Avoid by having a “wash out” period Group 2
Washout
Period
Drug Placebo

Effect Modification Effect Modification

Stratified Analysis
• Not a type of bias (point of confusion)
DVT
• Occurs when third factor alters effect + -

Drug A
+
• Consider:
50 50

- 10 90
• Drug A is shown to increase risk of DVT
RR = 5
• To cause DVT, Drug A requires Gene X
• Gene X is an effect modifier
Gene X (+) Gene X (-)
DVT DVT
+ - + -
Drug A

Drug A
+ 25 25 + 15 35

- 5 45 - 15 35

RR = 5 RR = 1

Confounding vs. Effect Modification Confounding vs. Effect Modification

Example

• Confounding: • Patients taking drug A have increased rates of lung cancer

• A 3rd variable distorts the effect on outcome • Drug A is taken mostly by smokers
• Smoking and alcohol
• Alcohol appears associated with cancer (positive) • Breakdown data into smokers and non-smokers:
• Real effect of exposure on outcome distorted by confounder • NO relationship between Drug A and cancer
• Smoking is the real cause
• Effect modification: • Drug A has no effect
• A 3rd variable maintains effect but only in one group • This is confounding
• There is a real effect of exposure on outcome
• Effect requires presence of 3rd variable

39
Confounding vs. Effect Modification Hawthorne Effect
Example

• Patients taking drug A have increased rates of lung cancer • Type of measurement bias
• Drug A activates gene X to cause cancer • Study patients improve because they are being studied
• Breakdown data into gene X (+) and (-) • Patients or providers change behavior based on being
• Relationship exists between Drug A and cancer but only in gene X (+) studied
• Drug A does have effect (different from confounding) • Common in studies of behavioral patterns
• But drug A requires another factor (gene X) • Example:
• This is effect modification (not a form of bias) • Physicians know patients surveyed about vaccination status
• Physicians vaccinate more often
• Example:
• Patients being studied for exercise capacity
• Patients exercise more often

Pygmalion Effect Pygmalion vs. Hawthorne

Observer-expectancy effect
• Researcher believes in efficacy of treatment • Pygmalion effect
• Influences outcome of study • Provider believes in treatment
• Influences results to be positive
• Example: • Pygmalion unique to investigator driving positive benefit
• Creator of a new surgical device overseeing study
• Creator assesses outcomes positively • Hawthorne Effect
• Subjects/investigators behave differently because of study

Shutterstock

40
 Clinical Trials
Clinical Trials
• Observational studies: no control over exposure
• Cohort, case-control
• Experimental studies: researchers control exposure
Clinical Trials • Goal is to determine benefit of therapy
• Drugs
Jason Ryan, MD, MPH • Surgery

Shutterstock

Clinical Trial Features Control

• Control • One group receives therapy
• Randomization • Other group no therapy (control group)
• Blinding • Compared changes in therapy group to control group

PxHere

PicPedia.org

Randomization Table 1
• Subjects randomly assigned to treatment or control • Non-significant p values indicated randomization was
• All variables other than treatment should be equal successful
• Should eliminate confounding
• All potential confounders (age, weight, blood levels) should be equal in both
Intervention Control p value
arms
Male (%) 49% 51% NS
• Limits selection bias Age (mean) 64 65 NS
• Patients cannot choose to be in drug arm of study
Diabetes (%) 10 11 NS
• Table 1 in most studies demonstrates randomization Systolic BP (mean) 121 119 NS

41
Blinding Clinical Trials
• Treatment subjects given therapy/drug • Best evidence of efficacy comes from randomized,
• Control subjects given placebo controlled, blinded studies
• Subjects unaware if they are getting treatment or not • Why not do these for everything?
• Single blind: subjects unaware • Takes a long time
• Double blind: subjects and providers unaware • By end of study, new treatments sometimes have emerged
• Triple blind: subjects, providers, data analysts unaware • Costs a lot of money

Kolijoriverhouse/Wikipedia
Public Domain

Parachute Example Data from Clinical Trials

• No clinical data exists showing parachutes are effective • Drug X → 30% mortality over 3 years
compared to placebo • Placebo → 50% mortality over 3 years
• Several ways to report this:

Absolute Risk Reduction = 50% - 30% = 20%

Relative Risk Reduction = 50% - 30% = 40%

50%

Number Needed to Treat Number Needed to Treat

NNT
• Drug X → 30% mortality over 3 years
• Placebo → 50% mortality over 3 years
• Number need to treat to prevent one outcome

Number Needed to Treat = 1 = 1 = 5

ARR 0.2

Wiviott SD et al. Dapagliflozin and Cardiovascular Outcomes in Type 2 Diabetes.

N Engl J Med. 2019 Jan 24;380(4):347-357. doi: 10.1056/NEJMoa1812389. Epub 2018 Nov 10. PMID: 30415602.

42
Clinical Trial Results Clinical Trial Results
Interpretation Outcomes
• Is the result clinically meaningful? • Intervention group compared to control group
• Drug reduces blood pressure by 2 mmHg (p < 0.05) • What outcome (endpoint) will be assessed?
• Average HTN patient is 15 mmHg above goal
• Major element of interpreting study results
• Was the population representative of actual practice?
• Patients described in Table 1 – are they similar to actual practice?
• Does the drug improve a meaningful outcome?
• Drug reduces biologic activity of cancer cells
• No change in mortality between groups

Clinical Trial Clinical Trial

Outcomes Outcomes
• Continuous – outcome exists on a continuum • Soft: quality of life, reduction in pain
• Blood pressure, total cholesterol, weight • Not directly harmful
• Mean values compared between groups • Hard: hospitalization, stroke, myocardial infarction, death
• Categorical – outcome exists in a category (yes/no) • Directly harmful
• Death
• Hospitalization
• Stroke
• Myocardial infarction
• Blood pressure < 140 mmHg
• Compare percentage of patients with outcome in each group

Shutterstock

Surrogate Outcomes Clinical Trial

Outcomes
• Soft outcomes that are predictive of hard outcomes • Combined endpoint
• Systolic blood pressure → stroke • Often used in trials with categorical outcomes (death, MI, stroke)
• Hemoglobin a1c level → diabetes complications • Imagine death as endpoint in trial of 2000 patients
• Advantages • After two years: 5 deaths total – no significant difference between groups
• Easier to obtain • Imagine death, hospitalization and stroke as endpoint in trial of 2000 patients
• After two years: 375 endpoints – significant difference between groups
• Disadvantages
• May lead to erroneous findings • Many trials used combined endpoints to allow faster studies

Shutterstock

43
Clinical Trial Negative Clinical Trials
Outcomes
• Primary endpoint • Difference between groups not statistically significant
• Outcome study is designed to evaluate • Must consider power of study
• Endpoint used to determine power of trial • Power = chance of finding a difference when one exists
• Trial power determines sample size needed • Or chance of rejecting no difference because there really is one
• Secondary endpoint • Power increases with sample size
• Outcomes of interest • Small studies are “underpowered”
• Not used for power calculation or sample size determination • Cannot “detect small differences”
• Interpret with caution • P value will be nonsignificant for small differences

Shuttersto
ck

Non-Inferiority Trials Non-Inferiority Trials

• New treatment compared to standard of care • Study designed by choosing difference in outcomes that is
• Goal is to show new treatment has similar outcomes to acceptable (∆)
standard of care • Null hypothesis: between group differences > ∆ (one group
• Used when a placebo group may be unethical superior)
• Standard treatment well-established and effective • Alternative hypothesis: between group differences ≤ ∆ (non-
• Or when improvement with new treatment may be small inferiority)
• P-value < 0.05 = reject null hypothesis = new treatment non-
inferior

Non-Inferiority Trials Kaplan-Meier Curves

• Apixaban versus dalteparin • Time-to-event curves
• Recurrent thromboembolism: • Proportion of patients without event over time
• 5.6% in apixaban group
• 7.9% in dalteparin group
• P < 0.001 for noninferiority

April 23, 2020 N Engl J Med 2020; 382:1599-1607

44
Hazard Ratios Intention to Treat Analysis
• Probability of events in treatment group compared to • Subjects analyzed according to the group they were
control group originally assigned
• < 1 = event less likely in treatment group • Regardless of whether they received treatment or not
• > 1 = event more likely in treatment group • Not affected by crossover or dropout
• Patients in control group may require treatment (crossover)
• Still analyzed as members of the control arm
• Patient in treatment group may be unable to comply with
treatment (crossover)
• Still analyzed as members of treatment arm
• Patients in treatment arm may dropout of study
• Still analyzed as members of treatment arm

Wiviott SD et al. Dapagliflozin and Cardiovascular Outcomes in Type 2 Diabetes.

N Engl J Med. 2019 Jan 24;380(4):347-357. doi: 10.1056/NEJMoa1812389. Epub 2018 Nov 10. PMID: 30415602.

Meta Analyses New Drug Approval

• Pools data from several studies together • Clinical trials conducted in phases
• Increases number of subjects and controls • Phase 1
• Increases statistical power • Small number of healthy volunteers
• Safety, toxicity, pharmacokinetics
• Limited because pooled studies often differ
• Selection criteria • Phase 2
• Exact treatment used • Small number of sick patients
• Efficacy, dosing, side effects
• Often placebo controlled, often blinded

Shutterstock

New Drug Approval Phase 4

• Phase 3 • Post-marketing study
• Large number of sick patients • After drug is on the market and being used
• Many patients, many centers
• Randomized trials • Monitor for long-term effects
• Drug efficacy determined vs. placebo or standard care
• After phase 3, drug may be approved by FDA

45
 Evidence-Based Medicine
Evidence-Based Medicine
• Caring for patients using best-available research
Evidence-Based • Four basic elements:
• Formulating a clinical question

Medicine
• Identifying best available evidence
• Assessing validity of evidence
• Applying the evidence in practice
Jason Ryan, MD, MPH

Public Domain

Clinical Questions Bad Clinical Question

• Should be focused • “Do ACE inhibitors work for hypertension?”
• Should be answerable from research literature • Vague
• PICO model • No population
• What is the patient population? • No specific outcome
• What intervention is being considered?
• What is the comparison intervention or population?
• What outcomes are important?

Pixabay/Public Domain

Good Clinical Question Types of Evidence

Population
• Primary resources
• Case reports/series First Available

“Among obese adult women with hypertension

• Observational studies
• Randomized clinical trials (best)
is lisinopril more effective than HCTZ for the
prevention of heart disease?”
• Systematic reviews/meta-analysis
• Compilation of primary studies
Intervention Comparison
• Society guidelines
• Written based on multiple sources
Outcome • Primary data, systematic reviews
• Clinical expertise, patient preferences Last Available

46
Types of Evidence Evaluating Evidence
• Internal validity
• Was the research conducted properly?
Stronger
• Are the conclusions correct?
Less Bias Meta-analysis RCT
• Is there bias?
Systematic RCT Review • Are results due to chance?
Randomized Controlled Trial

Cohort Study
Observational
Case Control Study

Case Report/Case Series

Animal Research
Weaker
More Bias

Shutterstock

Evaluating Evidence Evidence-Based Medicine

• External validity • Must also apply clinical expertise and patient’s wishes
• Does the research apply to patients not in study?
• Are study patients similar to real world patients?
• Is the intervention similar to real world interventions?
• Does this apply to the patient in my clinical question? Best Evidence

Patient
Clinical
Expertise
EBM Wishes

Shutterstock

47