ANOVA - Analysis of Variance

ANOVA - Analysis of Variance

Extends independent-samples t test

ANOVA - Analysis of Variance

Extends independent-samples t test Compares the means of groups of independent observations

ANOVA - Analysis of Variance

Extends independent-samples t test Compares the means of groups of independent observations
Dont be fooled by the name. ANOVA does not compare variances.

ANOVA - Analysis of Variance

Extends independent-samples t test Compares the means of groups of independent observations
Dont be fooled by the name. ANOVA does not compare variances.

Can compare more than two groups

ANOVA Null and Alternative Hypotheses

Say the sample contains K independent groups

ANOVA Null and Alternative Hypotheses

Say the sample contains K independent groups

ANOVA tests the null hypothesis H0: 1 = 2 = = K

That is, the group means are all equal

ANOVA Null and Alternative Hypotheses

Say the sample contains K independent groups

ANOVA tests the null hypothesis H0: 1 = 2 = = K

That is, the group means are all equal

The alternative hypothesis is H1: i j

for some i, j

or, the group means are not all equal

Implants were placed in a manikin using placement guides of various widths. 15 implants were placed using each guide. Error (discrepancies with a reference implant) was measured for each implant.

Mean Implant Height Error (mm)

0.23

0.24

0.25

0.26

0.27

Example: Accuracy of Implant Placement

Mean Error by Guide Width

4mm

6mm Guide Width

8mm

Mean Implant Height Error (mm)

The overall mean of the entire sample was 0.248 mm. This is called the grand mean, and is often denoted by X . If H0 were true then wed expect the group means to be close to the grand mean.

0.23

0.24

0.25

0.26

0.27

Example: Accuracy of Implant Placement

Mean Error by Guide Width

4mm

6mm Guide Width

8mm

Mean Implant Height Error (mm)

The ANOVA test is based on the combined distances from X . If the combined distances are large, that indicates we should reject H0.

0.23

0.24

0.25

0.26

0.27

Example: Accuracy of Implant Placement

Mean Error by Guide Width

4mm

6mm Guide Width

8mm

The Anova Statistic

To combine the differences from the grand mean we
Square the differences Multiply by the numbers of observations in the groups Sum over the groups

SSB 15 X 4 mm X 15 X 6 mm X 15 X 8mm X
where the X * are the group means.

SSB = Sum of Squares Between groups

The Anova Statistic

To combine the differences from the grand mean we
Square the differences Multiply by the numbers of observations in the groups Sum over the groups

SSB 15 X 4 mm X 15 X 6 mm X 15 X 8mm X
where the X * are the group means.

SSB = Sum of Squares Between groups

Note: This looks a bit like a variance.

How big is big?

For the Implant Accuracy Data, SSB = 0.0047 Is that big enough to reject H0? As with the t test, we compare the statistic to the variability of the individual observations. In ANOVA the variability is estimated by the Mean Square Error, or MSE

MSE Mean Square Error

The Mean Square Error is a measure of the variability after the group effects have been taken into account.
0.5

Implant Height Error by Guide Width

Implant Height Error (mm)

1 xij X j 2 MSE N K j i
where xij is the ith observation in the jth group.

0.1

0.2

0.3

0.4

4mm

6mm Guide Width

8mm

MSE Mean Square Error

The Mean Square Error is a measure of the variability after the group effects have been taken into account.
0.5

Implant Height Error by Guide Width

Implant Height Error (mm)

1 xij X j 2 MSE N K j i
where xij is the ith observation in the jth group.

0.1

0.2

0.3

0.4

4mm

6mm Guide Width

8mm

MSE Mean Square Error

The Mean Square Error is a measure of the variability after the group effects have been taken into account.
0.5

Implant Height Error by Guide Width

Implant Height Error (mm)

1 xij X j 2 MSE N K j i
Note that the variation of the means seems quite small compared to the variance of observations within groups

0.1

0.2

0.3

0.4

4mm

6mm Guide Width

8mm

Notes on MSE
If there are only two groups, the MSE is equal to the pooled estimate of variance used in the equalvariance t test.

ANOVA assumes that all the group variances are equal. Other options should be considered if group variances differ by a factor of 2 or more.

ANOVA F Test
The ANOVA F test is based on the F statistic

SSB (K 1) F MSE
where K is the number of groups.

Under H0 the F statistic has an F distribution, with K-1 and N-K degrees of freedom (N is the total number of observations)

Implant Data: F test p-value

To get a p-value we compare our F statistic to an F(2, 42) distribution.

F(2,42) distribution

2 F

Implant Data: F test p-value

To get a p-value we compare our F statistic to an F(2, 42) distribution.
In our example

F(2,42) distribution

.0047 2 F .211 .0467 42

00.211

2 F

Implant Data: F test p-value

To get a p-value we compare our F statistic to an F(2, 42) distribution.
In our example

F(2,42) distribution

.0047 2 F .211 .0467 42

The p-value is
P = .81

PF (2,42) .211 0.81

00.211

2 F

ANOVA Table
Results are often displayed using an ANOVA Table
Sum of Squares Between Groups Within Groups Total .005 .466 .470 Mean Square .002 .011

df 2 42 44

F .211

Sig. .811

ANOVA Table
Results are often displayed using an ANOVA Table
Sum of Squares Between Groups Within Groups Total .005 .466 .470 Mean Square .002 .011

df 2 42 44

F .211

Sig. .811

Pop Quiz!: Where are the following quantities presented in this table? Sum of Squares Between (SSB) Mean Square Error (MSE) F Statistic p value

ANOVA Table
Results are often displayed using an ANOVA Table
Sum of Squares Between Groups Within Groups Total .005 .466 .470 Mean Square .002 .011

df 2 42 44

F .211

Sig. .811

Sum of Squares Between (SSB)

Mean Square Error (MSE)

F Statistic

p value

ANOVA Table
Results are often displayed using an ANOVA Table
Sum of Squares Between Groups Within Groups Total .005 .466 .470 Mean Square .002 .011

df 2 42 44

F .211

Sig. .811

Sum of Squares Between (SSB)

Mean Square Error (MSE)

F Statistic

p value

ANOVA Table
Results are often displayed using an ANOVA Table
Sum of Squares Between Groups Within Groups Total .005 .466 .470 Mean Square .002 .011

df 2 42 44

F .211

Sig. .811

Sum of Squares Between (SSB)

Mean Square Error (MSE)

F Statistic

p value

ANOVA Table
Results are often displayed using an ANOVA Table
Sum of Squares Between Groups Within Groups Total .005 .466 .470 Mean Square .002 .011

df 2 42 44

F .211

Sig. .811

Sum of Squares Between (SSB)

Mean Square Error (MSE)

F Statistic

p value

Post Hoc Tests

NHANES I data, women 40-60 yrs old. Compare cholesterol between periodontal groups. The ANOVA shows good evidence (p = 0.002) that the means are not all the same.
Sum of Squares Between 33383 Groups Within 4417119 Groups Total 4450502 df 3 2007 2010 Mean Square 11128 2201 F 5.1 Sig. .002

Which means are different? Can directly compare the subgroups using post hoc tests.

Least Significant Difference test

N Healthy Gingivitis Periodontitis Edentulous 802 490 347 372 Mean 221.5 223.5 227.3 232.4 Std. Deviation 46. 2 45.3 48.9 48. 8

The most simple post hoc test is called the Least Significant Difference Test. The computation is very similar to the equalvariance t test.

Sum of Squares Between 33383 Groups Within 4417119 Groups Total 4450502

df 3 2007 2010

Mean Square 11128 2201

F 5.1

Sig. .002

Compute an equal-variance t test, but replace the pooled variance (s2) with the MSE.

Least Significant Difference Test: Examples

N Healthy Gingivitis Periodontitis Edentulous 802 490 347 372 Mean 221.5 223.5 227.3 232.4 Std. Deviation 46. 2 45.3 48.9 48. 8

Compare Healthy group to Periodontitis group:

221.5 227.3 T 1.92 22011 802 1 347

p 2 P(t1147 1.92) 0.055

Sum of Squares Between 33383 Groups Within 4417119 Groups Total 4450502

df 3 2007 2010

Mean Square 11128 2201

F 5.1

Sig. .002

Compare Gingivitis group to Periodontitis group: 223.5 227.3 T 1.15 22011 490 1 347

p 2 P(t835 1.15) 0.25

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons.

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons. For example, if the data contain 4 groups, then 6 different pairwise comparisons can be made
Healthy Gingivitis

Periodontitis

Edentulous

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons. For example, if the data contain 4 groups, then 6 different pairwise comparisons can be made
Healthy Gingivitis

Periodontitis

Edentulous

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons. For example, if the data contain 4 groups, then 6 different pairwise comparisons can be made
Healthy Gingivitis

Periodontitis

Edentulous

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons. For example, if the data contain 4 groups, then 6 different pairwise comparisons can be made
Healthy Gingivitis

Periodontitis

Edentulous

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons. For example, if the data contain 4 groups, then 6 different pairwise comparisons can be made
Healthy Gingivitis

Periodontitis

Edentulous

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons. For example, if the data contain 4 groups, then 6 different pairwise comparisons can be made
Healthy Gingivitis

Periodontitis

Edentulous

Post Hoc Tests: Multiple Comparisons

Post-hoc testing usually involves multiple comparisons. For example, if the data contain 4 groups, then 6 different pairwise comparisons can be made
Healthy Gingivitis

Periodontitis

Edentulous

Post Hoc Tests: Multiple Comparisons

Each time a hypothesis test is performed at significance level , there is probability of rejecting in error. Performing multiple tests increases the chances of rejecting in error at least once.

Post Hoc Tests: Multiple Comparisons

Each time a hypothesis test is performed at significance level , there is probability of rejecting in error. Performing multiple tests increases the chances of rejecting in error at least once. For example:
if you did 6 independent hypothesis tests at the = 0.05

Post Hoc Tests: Multiple Comparisons

Each time a hypothesis test is performed at significance level , there is probability of rejecting in error. Performing multiple tests increases the chances of rejecting in error at least once. For example:
if you did 6 independent hypothesis tests at the = 0.05 If, in truth, H0 were true for all six.

Post Hoc Tests: Multiple Comparisons

Each time a hypothesis test is performed at significance level , there is probability of rejecting in error. Performing multiple tests increases the chances of rejecting in error at least once. For example:
if you did 6 independent hypothesis tests at the = 0.05 If, in truth, H0 were true for all six. The probability that at least one test rejects H0 is 26%

Post Hoc Tests: Multiple Comparisons

Each time a hypothesis test is performed at significance level , there is probability of rejecting in error. Performing multiple tests increases the chances of rejecting in error at least once. For example:
if you did 6 independent hypothesis tests at the = 0.05 If, in truth, H0 were true for all six. The probability that at least one test rejects H0 is 26% P(at least one rejection) = 1-P(no rejections) = 1-.956 = .26

Bonferroni Correction for Multiple Comparisons

The Bonferroni correction is a simple way to adjust for the multiple comparisons.
Bonferroni Correction Perform each test at significance level .

Multiply each p-value by the number of tests performed.

The overall significance level (chance of any of the tests rejecting in error) will be less than .

Example: Cholesterol Data post-hoc comparisons

Group 1 Healthy Healthy Healthy Group 2 Gingivitis Periodontitis Edentulous

Mean Difference (Group 1 Group 2) -2.0

-5.8 -10.9

Least Significant Difference p-value .46

.055 .00021

Bonferroni p-value 1.0 .330 .00126

Gingivitis
Gingivitis Periodontitis

Periodontitis
Edentulous Edentulous

-3.9
-8.9 -5.1

.25
.0056 .147

1.0
.0336 .88

Example: Cholesterol Data post-hoc comparisons

Group 1 Healthy Healthy Healthy Group 2 Gingivitis Periodontitis Edentulous

Mean Difference (Group 1 Group 2) -2.0

-5.8 -10.9

Least Significant Difference p-value .46

.055 .00021

Bonferroni p-value 1.0 .330 .00126

Gingivitis
Gingivitis Periodontitis

Periodontitis
Edentulous Edentulous

-3.9
-8.9 -5.1

.25
.0056 .147

1.0
.0336 .88

Conclusion: The Edentulous group is significantly different than the Healthy group and the Gingivitis group (p < 0.05), after adjustment for multiple comparisons