Overview of ANOVA and

ANCOVA
By Brian W. Sloboda

18 - 1
 Preview of ANOVA and
Some Preliminaries
 Analysis of variance (ANOVA) is a method for
testing the hypothesis that three or more
population means are equal.

 For example:
H0: µ1 = µ2 = µ3 = . . . µk
H1: At least one mean is different
 ANOVA Methods
Require the F-Distribution
1. The F- distribution is not symmetric; it is skewed to
the right.
2. The values of F can be 0 or positive; they cannot
be negative.
3. There is a different F-distribution for each pair of
degrees of freedom for the numerator and
denominator.
F-Distribution
 Definition
One-way analysis of variance (ANOVA) is a method of
testing the equality of three or more population means
by analyzing sample variances. One-way analysis of
variance is used with data categorized with one
treatment (or factor), which is a characteristic that
allows us to distinguish the different populations from
one another.
Relationship Between F Test Statistic / P-Value
 One-Way ANOVA Assumptions
1. The populations have approximately normal
distributions.
2. The populations have the same variance 2
(or the standard deviation  ).
3. The samples are simple random samples.
4. The samples are independent of each other.
5. The different samples are from populations
that are categorized in only one way.
 Procedure for testing
H0: µ1 = µ2 = µ3 = . . µ0

1. Use R, Minitab, Excel, SAS Studio or SPSS to obtain

results.

2. Identify the p-value from the output from the

software package.

3. Form a conclusion based on these

criteria:
 Procedure for testing
H0: µ1 = µ2 = µ3 = . . µ0

If the p-value  , reject the null

hypothesis of equal means and conclude
that at least one of the population means
is different from the others;
If the p-value > , fail to reject the null
hypothesis of equal means.
 Caution
When we conclude that there is sufficient
evidence to reject the claim of equal population
means, we cannot conclude from ANOVA that
any particular mean is different from the others.
There are other tests that can be used to
identify the specific means that are different,
and some of them are discussed in this
presentation.
Caution: EXCEL does not do these tests. You
will need to create your test from scratch!!!!
 Example:
Use the chest deceleration measurements
listed below and a significance level of  =
0.05 to test the claim that the three samples
come from populations with means that are all
equal.

Chest Deceleration Measurements (in g) from Car Crash Tests Mean

Small 44 43 44 54 38 43 42 45 44 50 44.7
Cars
Medium 41 49 43 41 47 42 37 43 44 34 42.1
Large 32 37 38 45 37 33 38 45 43 42 39.0

I like to thank Roya Amjadi from Federal Highway Administration in US Department of

Transportation for providing these data
 Example:
Requirements are satisfied: distributions are
approximately normal (normal quantile plots);
population variances appear to be about the
same; simple random samples; independent
samples, not matched; categorized according
to a single factor of size. (I did not show
these)
H0: 1 = 2 = 3
H1: At least one of the means is different
from the others
significance level is  = 0.05 (Type I error)
 Example:
Use EXCEL to obtain ANOVA results for the
Data given earlier

ANOVA
Source of
Variation SS df MS F P-value F critical
Between
Groups 162.8667 2 81.43333 4.094413 0.027986 3.354131
Within
Groups 537 27 19.88889

Total 699.8667 29       

 Example:
Step 2: Look at the p-value from the EXCEL
output which is 0.028 when rounded

Step 3: Because the p-value of 0.028 is less

than  = 0.05, we reject the null
hypothesis of equal means.

Interpretation of the Results: There is sufficient

evidence to warrant rejection of the claim that
the three samples come from populations with
means that are all equal.
 Identifying Means That Are Different

After conducting an analysis of variance

test, we might conclude that there is
sufficient evidence to reject a claim of
equal population means, but we cannot
conclude from ANOVA that any particular
mean is different from the others.
 Identifying Means That Are Different
Informal methods for comparing means

1. Use the same scale for constructing

boxplots of the data sets to see if one or
more of the data sets are very different from
the others.
2. Construct confidence interval estimates of
the means from the data sets, then compare
those confidence intervals to see if one or
more of them do not overlap with the others.
 Bonferroni Multiple
Comparison Test- A Common Approach

Step 1. Do a separate t test for each pair of

samples, but make the adjustments
described in the following steps.

Step 2. For an estimate of the variance σ2 that

is common to all of the involved populations,
use the value of MS(error).
 Bonferroni Multiple
Comparison Test
Step 2 (cont.) Using the value of MS(error),
calculate the value of the test statistic, as
shown below. (This example shows the
comparison for Sample 1 and Sample 4.)

x1  x4
t
1 1
MS (error )    
 n1 n4 
 Bonferroni Multiple
Comparison Test
Step 2 (cont.) Change the subscripts and
use another pair of samples until all of the
different possible pairs of samples have
been tested.

Step 3. After calculating the value of the

test statistic t for a particular pair of
samples, find either the critical t value or
the p-value, but make the following
adjustment:
 Bonferroni Multiple
Comparison Test
Step 3 (cont.) p-value: Use the test statistic t
with df = N – k, where N is the total number
of sample values and k is the number of
samples. Find the p-value the usual way,
but adjust the p-value by multiplying it by
the number of different possible pairings of
two samples.

(For example, with four samples, there are

six different possible pairings, so adjust the
p-value by multiplying it by 6.)
 Bonferroni Multiple
Comparison Test

Step 3 (cont.) Critical value: When finding

the critical value, adjust the significance
level α by dividing it by the number of
different possible pairings of two samples.

(For example, with four samples, there are

six different possible pairings, so adjust the
significance level by dividing it by 6.)
 Recap of One Way ANOVA
 One-way analysis of variance (ANOVA)

Calculations with Equal Sample Sizes

Calculations with Unequal Sample

Sizes

 Identifying Means That Are Different

Bonferroni Multiple Comparison Test

What is the Two Way ANOVA?

The analysis of variance procedure

introduced in the last part of the
presentation is referred to as one-way
analysis of variance because the data are
categorized into groups according to a
single factor (or treatment).

Now we introduce the method of two-way

analysis of variance, which is used with data
partitioned into categories according to two
factors.
 Two-Way
Analysis of Variance
In a nutshell, two-way ANOVA involves two factors.
The data are partitioned into subcategories
called cells.
 Definition
There is an interaction between
two factors if the effect of one of
the factors changes for different
categories of the other factor.
 More on Interactions
• There is an interaction between A and B if the
difference in means for the different levels of
factor A changes as the level of factor B changes.
• If there are interactions, the main effects no longer
have a clear interpretation. Need to examine the
means of all combinations of levels of A and B
(e.g., by using an interaction plot).
 14

12

10

Mean Response
8 Low A

6 High A

0
Low B High B

14

12

10
Mean Response

8 Low A

6 High A

0
Low B High B
 F tests for the Two-Way ANOVA with
Replication

• Test for the difference between the levels of the

main factors A and B
SS(A)/(a-1) SS(B)/(b-1)
MS(A) MS(B)
F= F=
MSE MSE SSE/(n-ab)

Rejection region: F > F,a-1 ,n-ab F > F, b-1, n-ab

• Test for interaction between factors A and B

MS(AB) SS(AB)/(a-1)(b-1)
F=
MSE

Rejection region: F > Fa-1)(b-1),n-ab

Example:
Let’s explore the data we used earlier in this
presentation by calculating the mean for each
cell and by constructing a graph but adding
Foreign and Domestic .

Small Cars Medium Large

 
Foreign 44 41 35
  54 49 42
  43 47 32
Domesti
54 43 37
c
  43 44 38
  42 37 33
Example:

The individual cell means are shown in the

next table Those means vary from a low of
36.0 to a high of 47.0, so they appear to vary
considerably.

Small Medium Large

Foreign 47.0 45.7 39.7

Domestic 43.0 38.0 36.0
Example:
The next figure is an interaction graph, which
shows graphs of those means, and that figure
has two very notable features:
Example:
Larger means: Because the line segments
representing foreign cars are higher than the
line segments for domestic cars, it appears
that foreign cars have consistently larger
measures of chest deceleration.
Interaction: Because the line segments
representing foreign cars appear to be
approximately parallel to the line segments for
domestic cars, it appears that foreign and
domestic cars behave the same for the
different car size categories, so there does not
appear to be an interaction effect.
Example:

In general, if a graph results in line segments

that are approximately parallel, we have
evidence that there is not an interaction
between the row and column variables.
These observations are largely subjective.
Let’s proceed with a more objective method of
two-way analysis of variance.
Procedure for
Two-Way ANOVAS
 Requirements

1. For each cell, the sample values come from a

population with a distribution that is approximately
normal.
2. The populations have the same variance 2.
3. The samples are simple random samples.
4. The samples are independent of each other.
5. The sample values are categorized two ways.
6. All of the cells have the same number of sample
values.
 Two-Way ANOVA
Two-Way ANOVA calculations are quite
involved and tedious , so we will use a software
package, e.g, R, SAS Studio, EXCEL etc is
being used.
 Procedure for Two-Way ANOVA

Step 1: Interaction Effect - test the null hypothesis that

there is no interaction

Step 2: Row/Column Effects - if we

conclude there is no interaction
effect, proceed with these two
hypothesis tests
Row Factor: no effects from row
Column Factor: no effects from
column
 Example:

Given the chest deceleration measurements in

the table a few slides ago, use two-way
analysis of variance to test for an interaction
effect, an effect from the row factor of type of
car (foreign, domestic), and an effect from the
column factor of car size (small, medium,
large). Use a 0.05 significance level.
 Example:
Requirements are satisfied: sample values from
normally distributed population; variances vary
considerably but not extremely, we will keep
this in mind; simple random samples;
independent samples, not matched; categorized
two ways (foreign-domestic, small-medium-
large; all cells has same number of samples (3)
Here are the results from EXCEL:
 Results from ANOVA Output using EXCEL

ANOVA
Source of Variation SS df MS F P-value F crit
0.45983
Sample 14.22222 1 14.222220.583144 4 4.747225
0.00907 3.88529
Columns 348.1111 2 174.0556 7.136674 8 4
3.88529
Interaction 14.77778 2 7.3888890.302961 0.744114 4
Within 292.6667 12 24.38889

Total 669.7778 17       

Note: In EXCEL use Two Way ANOVA with Replication

18 - 1
 Example:

For MS(Error) use Within in EXCEL table. The p-

value is 0.744, so fail to reject the null
hypothesis of no interaction between the two
factors
Step 2: Row/Column Effects
H0: There is no effect from the row factor (the
row means are equal)
H0: There is no effect from the column factor (the
column means are equal)
 Example:
MS(type) 117.556
Row Factor: F   5.44
MS(error) 21.611

p-value = 0.038 < 0.05, so reject the null

hypothesis of no effects from the type of car

MS(size) 77.389
Column Factor: F   3.58
MS(error) 21.611

p-value = 0.060 > 0.05, so fail to reject the null

hypothesis of no effects from the size of car
 Randomized Blocks vs. Two-Way
ANOVA
• The randomized block design is a special case of two-way
ANOVA in which the blocks are the second factor and the
number of replications is 1.
• However, in analyzing a randomized blocks design, we
assume that there are no interactions.
• Also, in a randomized block design, blocking is specifically
performed to reduce variation and there is no interest in the
block effect itself. In the general two-way design, the effect
of both of the factors is of interest.
• In EXCEL this is Two Way ANOVA without Replication
 Two Way ANOVA Without
Replication Using EXCEL
ANOVA
Source of
Variation SS df MS F P-value F critical
Rows 224.4444 5 44.88889 4.617143 0.019169 3.325835
Columns 348.1111 2 174.0556 17.90286 0.000496 4.102821
Error 97.22222 10 9.722222

Total 669.7778 17       

Note: That the Two Way ANOVA without Replication only has Rows and Columns
and no interaction effects.
 Advantages of Two-Way ANOVA
• When interested in studying the effects of two
factors, two-way designs offer great advantages
over several single-factor studies.
• Researchers want to determine the influence of dietary
minerals on blood pressure.
• Rats receive diets prepared with varying amounts of
calcium and varying amounts of magnesium, but with
all other ingredients of the diets the same. There are
three levels of calcium (low, medium and high) and three
levels of magnesium.
 Recap
In this section we learned about Two Way
ANOVA with Replication

 Two- way analysis of variance (ANOVA)

 Special case: One observation per cell and
no interaction (This is two way ANOVA
without Replication which can be done in
EXCEL as shown)
What is ANCOVA?
 Analysis of Covariance (ANCOVA)
• A procedure for comparing treatment means that
incorporates information on a quantitative
explanatory variable, X, sometimes called a
covariate.

•ANCOVA is a combination of ANOVA with a

regression.

•EXCEL does not do ANCOVA!

23-49
 Example: Calf Weight Gain
• An animal scientist wishes to examine the impact of a pair of new dietary
supplements on calf weight gain (response).
• Three treatments are defined: standard diet, standard diet + supplement Q,
and standard diet + supplement R.
• All new calves from a large herd are available for use as study units. She
selects 30 calves for study. Calves are randomized to the three diets at
random (completely randomized design).
• Initial weights are recorded, then calves are placed on the diets. At the
end of four weeks the final weight is taken and weight gain is computed.
• Simple analysis of variance and associated multiple comparisons
procedures indicate no significant differences in weight gain between the
two supplementary diets, but big differences between the supplemental
diets and the standard diet.
• Is this the end of the story?

23-50
 ANOVA Results

Average Weight Gain

(Response g/day)

xx
x xx x x xx Standard
Diet
xxx
xxx x xx + Supplement Q

xx x
x x xx x x x + Supplement R

Simple ANOVA of a one-way classification would suggest no difference

between Supplements Q and R but both different from Standard diet.

23-51
 Initial Weights

Initial Weight

xx x
x xx x x xx Standard
Diet
xx
x xxxx x xx + Supplement Q

x xx
xx xx x x x + Supplement R

Plotting of the initial weights by group shows that the groups were not equal
when it came to initial weights.

23-52
 Weight Gain to Initial Weight
Standard Diet
Weight (kg)

2
wF
2
w gain
1 2
1
wF wi
w gain 1
wi
age
If animals come into the study at different ages, they have different initial weights and
are at different points on the growth curve. Expected weight gains will be different
depending on age at entry into study.

23-53
 Regression of Initial Weight to
Weight Gain

Weight
Gain If we disregard the
2
(g/day) age of the animal
(Y)
w gain but instead focus on
the initial weight,
we see that there is
1 a linear relationship
w gain
Initial between initial
Weight weight and the
1 2 (x)
wi wi weight gain
expected.

23-54
 Covariates
Initial weight in the previous example is a covariate.

A covariate is a disturbing variable (or known as the confounder), that is, it is known
to have an effect on the response. Usually, the covariate can be measured but
often we may not be able to control its effect through blocking.

In the EXAMPLE, had the animal scientist known that the calves were very
variable in initial weight (or age), she could have:
• Created blocks of 3 or 6 equal weight animals, and randomized
treatments to calves within these blocks.
• This would have entailed some cost in terms of time spent sorting the
calves and then keeping track of block membership over the life of the
study.
• It was much easier to simply record the calf initial weight and then use
analysis of covariance for the final analysis.
• In many cases, due to the continuous nature of the covariate, blocking is
just not feasible.

23-55
 Expectations under Ho
Under Ho: no treatment
If all animals had come in with the same initial
effects.
weight, All three treatments would produce the
same weight gain.

Expected
Weight
Gain
(g/day)
(Y)

Initial
Weight
(x)
Average Weight Animal

23-56
 Different Initial Weights
If the average initial weights in the treatment
Under Ho: no treatment
groups differ, the observed weight gains will
effects.
be different, even if treatments have no
effect.

WGR
WGs
WGQ

Expected
Weight
Gain cc c
(g/day) qq r rr Initial
c cc c c cc
(Y) q qqqq q qq rr rr r r r Weight
(x)

23-57
 Observed Responses under HA
Suppose now that different supplements actually do increase weight gain.
This translates to animals in different treatment groups following different, but
parallel regression lines with initial weight.
+ Supplement Q

+ Supplement R
r
rr r Standard Diet
WGR q r rr r
q rr
WGQ qq q
q q c
q q c cc
WGs q c cc c Under HA: Significant
c c Treatment
effects

Weight
Gain
(g/day) cc c
(Y) qq r rr Initial
c cc c c cc
q qqqq q qq rr rr r r r Weight
(x)
What difference in weight gain is due to Initial weight and what is due to Treatment?
23-58
 Simple one-way classification ANOVA (without
Observed Group Means accounting for initial weight) gives us the
wrong answer!
Weight
Gain
(g/day) + Supplement Q
(Y)
+ Supplement R
r rr
yr q
rr
r r r
Standard Diet
rr
yq qq q
q
q q c c
q q q c c
c c
yc c c
cc

cc c
qq r rr Initial
c cc c c cc
Unadjusted
q qqqq q qq rr rr r r r Weight
treatment means (x)

23-59
 A Priori Assumptions
The covariate is related to the response, and can account for variation in the
response.
Check with a scatterplot of Y vs. X.

The covariate is NOT related to the treatments.

If Y is related to X, then the variance of the treatment differences is
increased relative to that obtained from an ANOVA model without X, which
results in a loss of precision.

The treatment’s regression equations are linear in the covariate.

Check with a scatterplot of Y vs. X, for each treatment. Non-linearity can be
accommodated (e.g. polynomial terms, transforms), but analysis may be
more complex.

The regression lines for the different treatments are parallel.

This means there is only one slope in the Y vs. X plots. Non-parallel lines can
be accommodated, but this complicates the analysis since differences in
treatments will now depend on the value of X.

23-60
 Formulation Strength Thickness
1 46.5 13

Example 1
1
45.9
49.8
14
12
Four different formulations of an industrial glue are 1 46.1 12
being tested. The tensile strength (response) of the glue 1 44.3 14
is known to be related to the thickness as applied. Five 2 48.7 12
observations on strength (Y) in pounds, and thickness 2 49.0 10
(X) in 0.01 inches are made for each formulation. 2 50.1 11
2 48.5 12
Here: 2 45.2 14
• There are t=4 treatments (formulations of glue). 3 46.3 15
• Covariate X is thickness of applied glue. 3 47.1 14
• Each treatment is replicated n=5 times at different 3 48.9 11
values of X. 3 48.2 11
3 50.3 10
4 44.7 16
4 43.0 15
4 51.0 10
4 48.1 12
4 46.8 11

23-61
 Formulation Profiles

52.0

48.0
Strength
(Y)
44.0

40.0
16 15 10 12 11
Thickness (X)

Form_1 Form_2 Form_3 Form_4

23-62
 data glue;
SAS Studio Program input Formulation Strength Thickness;
datalines;
1 46.5 13
1 45.9 14
1 49.8 12
1 46.1 12
1 44.3 14
2 48.7 12
2 49.0 10
2 50.1 11
2 48.5 12
2 45.2 14
3 46.3 15
3 47.1 14
3 48.9 11
3 48.2 11
3 50.3 10
The basic model is a combination of 4 44.7 16
regression and one-way 4 43.0 15
4 51.0 10
classification. 4 48.1 12
4 46.8 11
;
run;
proc glm;
class formulation;
model strength = thickness formulation
/ solution ;
lsmeans formulation / stderr pdiff;
run;

23-63
Output: Use Type III SS to test significance of each variable
MSE

Source DF Squares Mean Square F Value Pr > F

Model 4 66.31065753 16.57766438 10.17 0.0003
Error 15 24.44684247 1.62978950
Corrected Total 19 90.75750000 Regression on
thickness is
R-Square Coeff Var Root MSE Strength Mean significant.
0.730636 2.691897 1.276632 47.42500 No formulation
differences.

Source DF Type I SS Mean Square F Value Pr > F

Thickness 1 63.50120135 63.50120135 38.96 <.0001
Formulation 3 2.80945618 0.93648539 0.57 0.6405

Source DF Type III SS Mean Square F Value Pr > F

Thickness 1 53.20115753 53.20115753 32.64 <.0001
Formulation 3 2.80945618 0.93648539 0.57 0.6405

Divide by MSE
Standard
to get mean
Parameter Estimate Error t Value Pr > |t|
squares.
Intercept 58.93698630 B 2.21321008 26.63 <.0001
Thickness -0.95445205 0.16705494 -5.71 <.0001
Formulation 1 -0.00910959 B 0.80810401 -0.01 0.9912
Formulation 2 0.62554795 B 0.82451389 0.76 0.4598
Formulation 3 0.86732877 B 0.81361075 1.07 0.3033
Formulation 4 0.00000000 B . . .

23-64
 Least Squares Means
(Adjusted Formulation means computed at the
average value of Thickness [=12.45])
The GLM Procedure
Least Squares Means

Strength Standard LSMEAN

Formulation LSMEAN Error Pr > |t| Number

1 47.0449486 0.5782732 <.0001 1

2 47.6796062 0.5811616 <.0001 2
3 47.9213870 0.5724527 <.0001 3
4 47.0540582 0.5739134 <.0001 4

Least Squares Means for effect Formulation

Pr > |t| for H0: LSMean(i)=LSMean(j)

Dependent Variable: Strength

i/j 1 2 3 4
1 0.4574 0.3011 0.9912
2 0.4574 0.7695 0.4598
3 0.3011 0.7695 0.3033
4 0.9912 0.4598 0.3033

23-65
 End of Presentation

• Thank you for attending the session

today!!
• For quick questions, please contact
me at bsloboda@email.phoenix.edu.

18 - 1