Professional Documents
Culture Documents
Binary or Chi-square test: McNemar’s chi-square test: Fisher’s exact test: compares
categorical compares proportions between compares binary outcome between proportions between independent
two or more groups correlated groups (e.g., before and groups when there are sparse data
(e.g. after) (some cells <5).
fracture,
yes/no) Relative risks: odds ratios
or risk ratios Conditional logistic McNemar’s exact test:
regression: multivariate compares proportions between
regression technique for a binary correlated groups when there are
Logistic regression: sparse data (some cells <5).
outcome when groups are correlated
multivariate technique used
(e.g., matched data)
when outcome is binary; gives
multivariate-adjusted odds
ratios GEE modeling: multivariate
regression technique for a binary
outcome when groups are correlated
(e.g., repeated measures)
Comparing more than two
groups…
Continuous outcome (means)
Are the observations independent or correlated?
Outcome Alternatives if the normality
Variable independent correlated assumption is violated (and
small sample size):
Continuous Ttest: compares means Paired ttest: compares means Non-parametric statistics
(e.g. pain between two independent groups between two related groups (e.g., Wilcoxon sign-rank test:
the same subjects before and non-parametric alternative to the
scale, after)
ANOVA: compares means paired ttest
cognitive
function) between more than two
independent groups Repeated-measures Wilcoxon sum-rank test
ANOVA: compares changes (=Mann-Whitney U test): non-
Pearson’s correlation over time in the means of two or parametric alternative to the ttest
more groups (repeated
coefficient (linear measurements)
correlation): shows linear Kruskal-Wallis test: non-
correlation between two parametric alternative to ANOVA
continuous variables Mixed models/GEE
modeling: multivariate
regression techniques to compare Spearman rank correlation
Linear regression: changes over time between two or coefficient: non-parametric
multivariate regression technique more groups; gives rate of change alternative to Pearson’s correlation
ANOVA example
Mean micronutrient intake from the school lunch by school
S1a, n=28 S2b, n=25 S3c, n=21 P-valued
Calcium (mg) Mean 117.8 158.7 206.5 0.000
SDe 62.4 70.5 86.2
Iron (mg) Mean 2.0 2.0 2.0 0.854
SD 0.6 0.6 0.6
Folate (μg) Mean 26.6 38.7 42.6 0.000
SD 13.1 14.5 15.1
Zinc (mg) Mean 1.9 1.5 1.3 0.055
SD 1.0 1.2 0.4
a
School 1 (most deprived; 40% subsidized lunches). FROM: Gould R, Russell J,
Barker ME. School lunch menus
b
School 2 (medium deprived; <10% subsidized). and 11 to 12 year old children's
food choice in three secondary
c
School 3 (least deprived; no subsidization, private school). schools in England-are the
nutritional standards being met?
d
ANOVA; significant differences are highlighted in bold (P<0.05). Appetite. 2006 Jan;46(1):86-92.
ANOVA
(ANalysis Of VAriance)
Idea: For two or more groups, test
difference between means, for
quantitative normally distributed
variables.
Just an extension of the t-test (an
ANOVA with only two groups is
mathematically equivalent to a t-test).
One-Way Analysis of Variance
H 0 : μ1 μ 2 μ 3
Recall, we have already used an “F-test” to check for equality of variances If F>>1 (indicating
unequal variances), use unpooled variance in a t-test.
The F-distribution
The F-distribution is a continuous probability distribution that depends
on two parameters n and m (numerator and denominator degrees of
freedom, respectively):
http://www.econtools.com/jevons/java/Graphics2D/FDist.html
The F-distribution
A ratio of variances follows an F-distribution:
2
between
2
~ Fn ,m
within
10 10 10
y1 j y
j 1
y 2j y 3j
j 1
4j The group means
y1 y 2
j 1
y 3
j 1 y 4
10 10 10 10
10 10 10
10
( y 2 j y 2 ) 2 (y (y
(y y 4 ) 2
2
1j y1 ) 2
3j y 3 ) 4j
j 1 j 1 j 1 j 1 The (within)
10 1 10 1 10 1 10 1 group variances
Sum of Squares Within (SSW),
or Sum of Squares Error (SSE)
10 10 10
(y
10 2
(y 1j y1 ) 2
j 1
2j y 2 ) (y j 1
3j y 3 ) 2
(y
j 1
4j y 4 ) 2
The (within)
j 1
group variances
10 1 10 1 10 1 10 1
10 10
10 10
(y 1j
2
y1 ) + ( y 2 j y 2 ) 2 + ( y 3 j y 3 ) + 2
(y
j 1
4j y 4 ) 2
j 1 j 3
j 1
4 10
i 1 j 1
( y ij y i ) 2 Sum of Squares Within (SSW)
(or SSE, for chance error)
Sum of Squares Between (SSB), or
Sum of Squares Regression (SSR)
4 10
Overall mean of
all 40 y
i 1 j 1
ij
observations
(“grand mean”) y
40
10 x (y
i 1
i y ) 2 (SSB). Variability of the
group means compared to
the grand mean (the
variability due to the
treatment).
Total Sum of Squares (SST)
i 1 j 1
( y ij y ) 2 observation from the overall
mean. (numerator of
variance of Y!)
Partitioning of Variance
4 10 4 4 10
( y
i 1 j 1
ij y i ) 2
+10x ( y i y ) 2 = ( y ij y ) 2
i 1 i 1 j 1
Mean Sum
Source of Sum of of Squares
variation d.f. squares F-statistic p-value
Total 39 2257.1
Step 3) Fill in the ANOVA table
Total 39 2257.1
INTERPRETATION of ANOVA:
How much of the variance in height is explained by treatment group?
R2=“Coefficient of Determination” = SSB/TSS = 196.5/2275.1=9%
Coefficient of Determination
2 SSB SSB
R
SSB SSE SST
The amount of variation in the outcome variable (dependent
variable) that is explained by the predictor (independent variable).
Beyond one-way ANOVA
Often, you may want to test more than 1
treatment. ANOVA can accommodate
more than 1 treatment or factor, so long as
they are independent. Again, the variation
partitions beautifully!
TSS = SSB1 + SSB2 + SSW
ANOVA example
Table 6. Mean micronutrient intake from the school lunch by school
S1a, n=25 S2b, n=25 S3c, n=25 P-valued
Calcium (mg) Mean 117.8 158.7 206.5 0.000
SDe 62.4 70.5 86.2
Iron (mg) Mean 2.0 2.0 2.0 0.854
SD 0.6 0.6 0.6
Folate (μg) Mean 26.6 38.7 42.6 0.000
SD 13.1 14.5 15.1
Mean 1.9 1.5 1.3 0.055
Zinc (mg)
SD 1.0 1.2 0.4
a
School 1 (most deprived; 40% subsidized lunches). FROM: Gould R, Russell J,
Barker ME. School lunch menus
b
School 2 (medium deprived; <10% subsidized). and 11 to 12 year old children's
food choice in three secondary
c
School 3 (least deprived; no subsidization, private school). schools in England-are the
nutritional standards being met?
d
ANOVA; significant differences are highlighted in bold (P<0.05). Appetite. 2006 Jan;46(1):86-92.
Answer
Step 1) calculate the sum of squares between groups:
Mean for School 1 = 117.8
Mean for School 2 = 158.7
Mean for School 3 = 206.5
Total 74 489,179
**R2=98113/489179=20%
School explains 20% of the variance in lunchtime calcium
intake in these kids.
ANOVA summary
A statistically significant ANOVA (F-test)
only tells you that at least two of the
groups differ, but not which ones differ.
• Scheffe (adjusts p)
significant)
Procedures for Post Hoc
Comparisons
If your ANOVA test identifies a difference between group
means, then you must identify which of your k groups
differ.
If you did not specify the comparisons of interest
(“contrasts”) ahead of time, then you have to pay a price
for making all kCr pairwise comparisons to keep overall
type-I error rate to α.
Arrange p-values:
6 9 7 10 5 2 8 4 3
Holm: .001<.0056; .002<.05/8=.00625; .006<.05/7=.007; .01>.05/6=.0083; therefore,
new drug only significantly different than standard drugs 6, 9, and 7.
Hochberg: .3>.05; .25>.05/2; .08>.05/3; .05>.05/4; .04>.05/5; .01>.05/6; .006<.05/7;
therefore, drugs 7, 9, and 6 are significantly different.
Practice problem
b. Your patient is taking one of the standard drugs that was
shown to be statistically less effective in minimizing
motion sickness (i.e., significant p-value for the
comparison with the experimental drug). Assuming that
none of these drugs have side effects but that the
experimental drug is slightly more costly than your
patient’s current drug-of-choice, what (if any) other
information would you want to know before you start
recommending that patients switch to the new drug?
Answer
The magnitude of the reduction in minutes of nausea.
If large enough sample size, a 1-minute difference could
be statistically significant, but it’s obviously not clinically
meaningful and you probably wouldn’t recommend a
switch.
Continuous outcome (means)
Are the observations independent or correlated?
Outcome Alternatives if the normality
Variable independent correlated assumption is violated (and
small sample size):
Continuous Ttest: compares means Paired ttest: compares means Non-parametric statistics
(e.g. pain between two independent groups between two related groups (e.g., Wilcoxon sign-rank test:
the same subjects before and non-parametric alternative to the
scale, after)
ANOVA: compares means paired ttest
cognitive
function) between more than two
independent groups Repeated-measures Wilcoxon sum-rank test
ANOVA: compares changes (=Mann-Whitney U test): non-
Pearson’s correlation over time in the means of two or parametric alternative to the ttest
more groups (repeated
coefficient (linear measurements)
correlation): shows linear Kruskal-Wallis test: non-
correlation between two parametric alternative to ANOVA
continuous variables Mixed models/GEE
modeling: multivariate
regression techniques to compare Spearman rank correlation
Linear regression: changes over time between two or coefficient: non-parametric
multivariate regression technique more groups; gives rate of change alternative to Pearson’s correlation
Non-parametric ANOVA
Kruskal-Wallis one-way ANOVA
(just an extension of the Wilcoxon Sum-Rank (Mann
Whitney U) test for 2 groups; based on ranks)
Binary or Chi-square test: McNemar’s chi-square test: Fisher’s exact test: compares
categorical compares proportions between compares binary outcome between proportions between independent
two or more groups correlated groups (e.g., before and groups when there are sparse data
(e.g. after) (some cells <5).
fracture,
yes/no) Relative risks: odds ratios
or risk ratios Conditional logistic McNemar’s exact test:
regression: multivariate compares proportions between
regression technique for a binary correlated groups when there are
Logistic regression: sparse data (some cells <5).
outcome when groups are correlated
multivariate technique used
(e.g., matched data)
when outcome is binary; gives
multivariate-adjusted odds
ratios GEE modeling: multivariate
regression technique for a binary
outcome when groups are correlated
(e.g., repeated measures)
Chi-square test
for comparing proportions
(of a categorical variable)
between >2 groups
I. Chi-Square Test of Independence
When both your predictor and outcome variables are categorical, they may be cross-
classified in a contingency table and compared using a chi-square test of
independence.
A contingency table with R rows and C columns is an R x C contingency table.
Example
Asch, S.E. (1955). Opinions and social
pressure. Scientific American, 193, 31-
35.
The Experiment
A Subject volunteers to participate in a
“visual perception study.”
Everyone else in the room is actually a
conspirator in the study (unbeknownst
to the Subject).
The “experimenter” reveals a pair of
cards…
The Task Cards
Yes 20 50 75 60 30
No 80 50 25 40 70
Yes 47 47 47 47 47
No 53 53 53 53 53
Chi-Square test
(observed - expected)2
2
expected
2 (20 47) 2 (50 47) 2 (75 47) 2 (60 47) 2 (30 47) 2
4
47 47 47 47 47
(80 53) 2 (50 53) 2 (25 53) 2 (40 53) 2 (70 53) 2
85
53 53 53 53 53
The expected
value and
variance of a chi-
square:
E(x)=df
Var(x)=2(df)
Chi-Square test
(observed - expected)2
2
expected
2 (20 47) 2 (50 47) 2 (75 47) 2 (60 47) 2 (30 47) 2
4
47 47 47 47 47
(80 53) 2 (50 53) 2 (25 53) 2 (40 53) 2 (70 53) 2
85
53 53 53 53 53
Rule of thumb: if the chi-square statistic is much greater than it’s degrees of freedom,
indicates statistical significance. Here 85>>4.
Chi-square example: recall data…
8 435 453
5 3
ptumor / cellphone .014; ptumor / nophone .033
352 91
(pˆ1 p
ˆ2) 0 8
Z ;p .018
( p )(1 p ) ( p )(1 p ) 453
n1 n2
(.014 .033) .019
Z 1.22
(.018)(.982) (.018)(.982) .0156
352 91
Same data, but use Chi-square test
Brain tumor No brain tumor
Own 5 347 352
Don’t own 3 88 91
8 435 453
8 352
ptumor .018; pcellphone .777 Expected value in
453 453 cell c= 1.7, so
ptumor xpcellphone .018 * .777 .014 technically should
Expected in cell a .014 * 453 6.3; 1.7 in cell c; use a Fisher’s exact
here! Next term…
345.7 in cell b; 89.3 in cell d
(R-1 )*(C-1 ) 1*1 1 df
2 (8 - 6.3) 2 (3 - 1.7) 2 (89.3 - 88) 2 (347 - 345.7) 2
1 1.48
6.3 1.7 89.3 345.7
NS
note :Z 2 1.22 2 1.48
Caveat
**When the sample size is very small in
any cell (expected value<5), Fisher’s
exact test is used as an alternative to
the chi-square test.
Binary or categorical outcomes
(proportions)
Are the observations correlated? Alternative to the chi-
Outcome square test if sparse
Variable independent correlated cells:
Binary or Chi-square test: McNemar’s chi-square test: Fisher’s exact test: compares
categorical compares proportions between compares binary outcome between proportions between independent
two or more groups correlated groups (e.g., before and groups when there are sparse data
(e.g. after) (np <5).
fracture,
yes/no) Relative risks: odds ratios
or risk ratios Conditional logistic McNemar’s exact test:
regression: multivariate compares proportions between
regression technique for a binary correlated groups when there are
Logistic regression: sparse data (np <5).
outcome when groups are correlated
multivariate technique used
(e.g., matched data)
when outcome is binary; gives
multivariate-adjusted odds
ratios GEE modeling: multivariate
regression technique for a binary
outcome when groups are correlated
(e.g., repeated measures)