Analysis of Variance

Analysis of Variance
girishchadha.gc@gmail.com
JV65UCK2AH
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited.
This file is meant for personal use by girishchadha.gc@gmail.com only.
Sharing or publishing the contents in part or full is liable for legal action.
Learning Objectives
In this chapter, you learn:
• The basic concepts of experimental design
• How to use one-way analysis of variance to test for differences among
the means of several groups
JV65UCK2AH
• How to use two-way analysis of variance and interpret the interaction
effect
• How to perform multiple comparisons in a one-way analysis of variance
and a two-way analysis of variance
General ANOVA Setting
• Investigator controls one or more factors of interest
• Each factor contains two or more levels
• Levels can be numerical or categorical
• Different levels produce different groups
JV65UCK2AH
• Think of each group as a sample from a different
population
• Observe effects on the dependent variable
• Are the groups the same?
• Experimental design: the plan used to collect the data
Completely Randomized Design
• Experimental units (subjects) are assigned randomly to groups

• Subjects are assumed homogeneous
• Only one factor or independent variable
• With two or more levels
JV65UCK2AH
• Analyzed by one-factor analysis of variance (ANOVA)
One-Way Analysis of Variance
• Evaluate the difference among the means of three or
more groups
Examples: Number of accidents for 1st, 2nd, and 3rd shift
JV65UCK2AH Expected mileage for five brands of tires
• Assumptions
• Populations are normally distributed
• Populations have equal variances
• Samples are randomly and independently drawn
Hypotheses of One-Way ANOVA
• H0 : μ1 = μ2 = μ3 =  = μc
• All population means are equal
• i.e., no factor effect (no variation in means among groups)
JV65UCK2AH
•
H• 1At: Not
leastall
oneofpopulation
the population
mean ismeans are equal
different
• i.e., there is a factor effect
• Does not mean that all population means are different
(some pairs may be the same)
One-Way ANOVA
H0 : μ1 = μ2 = μ3 =  = μc
H1 : Not all μ j are equal
The Null Hypothesis is True
All Means are the same:
JV65UCK2AH
(No Factor Effect)
μ1 = μ 2 = μ 3
One-Way ANOVA
(continued)
H0 : μ1 = μ2 = μ3 =  = μc
H1 : Not all μ j are equal
The Null Hypothesis is NOT true
JV65UCK2AH
At least one of the means is different
(Factor Effect is present)
or
μ1 = μ2  μ3 μ1  μ2  μ3
Proprietary content. ©Great Learning.
Sharing All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable for use
legal or distribution
action. prohibited.
Partitioning the Variation
• Total variation can be split into two parts:
SST = SSA + SSW

JV65UCK2AH
SST = Total Sum of Squares

(Total variation)
SSA = Sum of Squares Among Groups
(Among-group variation)
SSW = Sum of Squares Within Groups
(Within-group variation)

Sharing All Rights
action. prohibited.
Partitioning the Variation
SST = SSA + SSW
Total Variation = the aggregate variation of the individual

JV65UCK2AH data values across the various factor levels (SST)
Among-Group Variation = variation among the factor

sample means (SSA)
Within-Group Variation = variation that exists among

the data values within a particular factor level (SSW)

Sharing All Rights
action. prohibited.
Partition of Total Variation
Total Variation (SST)
JV65UCK2AH
Variation Due to Variation Due to Random

= Factor (SSA) + Error (SSW)

Sharing All Rights
action. prohibited.
Total Sum of Squares
SST = SSA + SSW
c nj
SST =  ( Xij − X) 2
JV65UCK2AH
Where: j=1 i=1
SST = Total sum of squares

c = number of groups or levels
nj = number of observations in group j
Xij = ith observation from group j
X = grand mean (mean of all data values)
Sharing All Rights
action. prohibited.
Total Variation
(continued)
2 2 2
SST = ( X 11 − X ) + ( X 12 − X ) +    + ( X cn − X )
c
JV65UCK2AH
Response, X
Group 1 Group 2 Group 3

Sharing All Rights
action. prohibited.
Among-Group Variation
SST = SSA + SSW
c
SSA =  n j ( X j − X)2
JV65UCK2AH
j=1
Where:
SSA = Sum of squares among groups
c = number of groups
nj = sample size from group j
Xj = sample mean from group j
X = grand mean (mean of all data values)
Sharing All Rights
action. prohibited.
(continued)
c
SSA =  n j ( X j − X)2
j=1
SSA
Variation Due to
MSA =
JV65UCK2AH
Differences Among Groups
c −1
Mean Square Among =
SSA/degrees of freedom
i j
Sharing All Rights
action. prohibited.
(continued)
SSA = n1 (X1 − X) + n 2 (X 2 − X) +    + n c (X c − X)
2 2 2
JV65UCK2AH
Response, X
X3
X2 X
X1

Sharing All Rights
action. prohibited.
Within-Group Variation
SST = SSA + SSW
c nj
SSW =   ( Xij − X j ) 2
JV65UCK2AH
j=1 i=1
Where:
SSW = Sum of squares within groups
nj = sample size from group j
Xj = sample mean from group j
Xij = ith observation in group j
Sharing All Rights
action. prohibited.
(continued)
c nj
SSW =   ( Xij − X j )2
j=1 i=1
SSW
Summing the variation
MSW =
JV65UCK2AH
within each group and then
adding over all groups n−c
Mean Square Within =
SSW/degrees of freedom
μj
Sharing All Rights
action. prohibited.
(continued)
SSW = (X11 − X1 ) + (X12 − X 2 ) +    + (Xcn c − Xc )

2 2 2
JV65UCK2AH Response, X
X3
X2
X1

Sharing All Rights
action. prohibited.
Obtaining the Mean Squares
The Mean Squares are obtained by dividing the various
sum of squares by their associated degrees of freedom
SSA Mean Square Among

MSA = (d.f. = c-1)
JV65UCK2AH c −1
SSW
MSW = Mean Square Within
n−c (d.f. = n-c)
SST
MST = Mean Square Total
n −1 (d.f. = n-1)
Sharing All Rights
action. prohibited.
One-Way ANOVA Table
Source of Degrees of Sum Of Mean Square F

Variation Freedom Squares (Variance)
Among SSA FSTAT =

c-1 SSA MSA =
JV65UCK2AH Groups c-1
MSA
Within SSW
n-c SSW MSW = MSW
Groups n-c
Total n–1 SST
n = sum of the sample sizes from all groups
df = degrees of freedom
Sharing All Rights
action. prohibited.
One-Way ANOVA
F Test Statistic
H0: μ1= μ2 = … = μc
H1: At least two population means are different
• Test statistic
MSA
JV65UCK2AH FSTAT =
MSW
MSA is mean squares among groups
MSW is mean squares within groups
• Degrees of freedom
• df1 = c – 1 (c = number of groups)
• df2 = n – c (n = sum of sample sizes from all populations)

Sharing All Rights
action. prohibited.
Interpreting One-Way ANOVA
F Statistic
• The F statistic is the ratio of the among estimate
of variance and the within estimate of variance
• The ratio must always be positive
• df1 = c -1 will typically be small
JV65UCK2AH • df2 = n - c will typically be large
Decision Rule:
◼ Reject H0 if FSTAT > Fα, 
otherwise do not reject
H0 0 Do not Reject H0
reject H0
Fα
Sharing All Rights
action. prohibited.
One-Way ANOVA
F Test Example
You want to see if three Club 1 Club 2 Club 3

different golf clubs yield 254 234 200
different distances. You 263 218 222
randomly select five 241 235 197
JV65UCK2AH measurements from trials on an 237 227 206
automated driving machine for 251 216 204
each club. At the 0.05
significance level, is there a
difference in mean distance?

Sharing All Rights
action. prohibited.
One-Way ANOVA Example: Scatter
Plot
Distance
Club 1 Club 2 Club 3 270
254 234 200 260 •
263 218 222 ••
241 235 197
250 X1
240 •
JV65UCK2AH
237 227 206 • ••
251 216 204 230
• X
220 ••
X2 •
210
x1 = 249.2 x 2 = 226.0 x 3 = 205.8
•• X3
200 •
•
x = 227.0 190
1
2 3
Sharing All Rights
legal or Club
distribution
action. prohibited.
One-Way ANOVA Example
Computations
Club 1 Club 2 Club 3 X1 = 249.2 n1 = 5
254 234 200 X2 = 226.0 n2 = 5
263 218 222
X3 = 205.8 n3 = 5
241 235 197
JV65UCK2AH 237 227 206 n = 15
X = 227.0
251 216 204 c=3
SSA = 5 (249.2 – 227)2 + 5 (226 – 227)2 + 5 (205.8 – 227)2 = 4716.4
SSW = (254 – 249.2)2 + (263 – 249.2)2 +…+ (204 – 205.8)2 = 1119.6
MSA = 4716.4 / (3-1) = 2358.2 2358.2

FSTAT = = 25.275
MSW = 1119.6 / (15-3) = 93.3 93.3
Sharing All Rights
action. prohibited.
One-Way ANOVA Example Solution
H0: μ1 = μ2 = μ3 Test Statistic:

H1: μj not all equal
MSA 2358.2
 = 0.05 FSTAT = = = 25.275
MSW 93.3
df1= 2 df2 = 12
JV65UCK2AH
Critical Decision:
Value:
Reject H0 at  = 0.05
Fα = 3.89
 = .05 Conclusion:
There is evidence that
0 Do not Reject H 0
at least one μj differs
reject H0
FSTAT = 25.275 from the rest
Fα = 3.89
Sharing All Rights
action. prohibited.
One-Way ANOVA
Excel Output
SUMMARY
Groups Count Sum Average Variance
Club 1 5 1246 249.2 108.2
Club 2 5 1130 226 77.5
JV65UCK2AH
Club 3 5 1029 205.8 94.2
ANOVA
Source of
SS df MS F P-value F crit
Variation
Between
4716.4 2 2358.2 25.275 0.0000 3.89
Groups
Within
1119.6 12 93.3
Groups
Total 5836.0 14
Sharing All Rights
action. prohibited.
ANOVA Assumptions
• Randomness and Independence
• Select random samples from the c groups (or randomly
assign the levels)
JV65UCK2AH
• Normality
• The sample values for each group are from a normal
population
• Homogeneity of Variance
• All populations sampled from have the same variance
• Can be tested with Levene’s Test

Sharing All Rights
action. prohibited.
Factorial Design:
Two-Way ANOVA
• Examines the effect of

• Two factors of interest on the dependent variable
• e.g., Percent carbonation and line speed on soft drink
bottling process
JV65UCK2AH • Interaction between the different levels of these
two factors
• e.g., Does the effect of one particular carbonation level
depend on which level the line speed is set?

Sharing All Rights
action. prohibited.
Two-Way ANOVA
(continued)
• Assumptions
• Populations are normally distributed

JV65UCK2AH
• Populations have equal variances
• Independent random samples are drawn

Sharing All Rights
action. prohibited.
Two-Way ANOVA
Sources of Variation
Two Factors of interest: A and B
r = number of levels of factor A
JV65UCK2AH
c = number of levels of factor B
n’ = number of replications for each cell
n = total number of observations in all cells
n = (r)(c)(n’)
Xijk = value of the kth observation of level i of
factor A and level j of factor B
Sharing All Rights
action. prohibited.
Two-Way ANOVA
Sources of Variation (continued)
SST = SSA + SSB + SSAB + SSE Degrees of

Freedom:
SSA r–1
Factor A Variation
JV65UCK2AH
SST SSB c–1
Factor B Variation
Total Variation
SSAB
Variation due to interaction (r – 1)(c – 1)
between A and B
n-1
SSE rc(n’ – 1)
Random variation (Error)
Sharing All Rights
action. prohibited.
Two-Way ANOVA Equations
Total Variation: r c n
SST =  ( Xijk − X) 2
i=1 j=1 k =1
JV65UCK2AH
Factor A Variation: r
SSA = cn  ( Xi.. − X)
 2
i=1
Factor B Variation: c
SSB = rn ( X. j. − X)2
j=1
Sharing All Rights
action. prohibited.
(continued)
Interaction Variation:
r c
JV65UCK2AH SSAB = n ( Xij. − Xi.. − X.j. + X)2
i =1 j=1
Sum of Squares Error:

r c n
SSE =  ( Xijk − Xij. ) 2
i =1 j =1 k =1

Sharing All Rights
action. prohibited.
(continued)
r c n
where:  X
i=1 j=1 k =1
ijk
X= = Grand Mean
c n
rcn
 X
j=1 k =1
ijk
JV65UCK2AH Xi.. = = Mean of ith level of factor A (i = 1, 2, ..., r)
cn
r n
 X ijk
X. j. = i=1 k =1
= Mean of jth level of factor B (j = 1, 2, ..., c)
rn
n
Xijk
Xij. = 
r = number of levels of factor A
= Mean of cell ij
k =1 n
c = number of levels of factor B
n’ = number of replications in each cell
Sharing All Rights
action. prohibited.
Mean Square Calculations
SSA
MSA = Mean square factor A =
r −1
SSB
girishchadha.gc@gmail.com MSB = Mean square factor B =
JV65UCK2AH
c −1
SSAB
MSAB = Mean square interactio n =
(r − 1)(c − 1)
SSE
MSE = Mean square error =
rc(n'−1)
Sharing All Rights
action. prohibited.
Two-Way ANOVA:
The F Test Statistics
F Test for Factor A Effect
H0: μ1..= μ2.. = μ3..= • • = µr..
MSA Reject H0 if
H1: Not all μi.. are equal FSTAT =
MSE FSTAT > Fα
JV65UCK2AH F Test for Factor B Effect
H0: μ.1. = μ.2. = μ.3.= • • = µ.c.
MSB Reject H0 if
H1: Not all μ.j. are equal FSTAT =
MSE FSTAT > Fα
F Test for Interaction Effect

H0: the interaction of A and B is
equal to zero
MSAB
H1: interaction of A and B is not FSTAT = Reject H0 if
MSE FSTAT > Fα
zero
Sharing All Rights
action. prohibited.
Two-Way ANOVA
Summary Table
Source of Sum of Degrees of Mean
F
Variation Squares Freedom Squares
MSA MSA
Factor A SSA r–1
= SSA /(r – 1) MSE
JV65UCK2AH
MSB MSB
Factor B SSB c–1
= SSB /(c – 1) MSE
AB MSAB MSAB
SSAB (r – 1)(c – 1)
(Interaction) = SSAB / (r – 1)(c – 1) MSE
MSE =
Error SSE rc(n’ – 1)
SSE/rc(n’ – 1)
Total SST n–1
Sharing All Rights
action. prohibited.
Features of Two-Way ANOVA
F Test
• Degrees of freedom always add up
• n-1 = rc(n’-1) + (r-1) + (c-1) + (r-1)(c-1)
• Total = error + factor A + factor B + interaction
JV65UCK2AH
• The denominators of the F Test are always the same
but the numerators are different
• The sums of squares always add up
• SST = SSE + SSA + SSB + SSAB
• Total = error + factor A + factor B + interaction

Sharing All Rights
action. prohibited.
Examples:
Interaction vs. No Interaction
◼ Interaction is present:
• No interaction: line
segments are parallel some line segments
not parallel
JV65UCK2AH Factor B Level 1
Mean Response
Mean Response
Factor B Level 1
Factor B Level 3
Factor B Level 2
Factor B Level 2
Factor B Level 3
Factor A Levels Factor A Levels

Sharing All Rights
action. prohibited.
Summary
In this chapter we discussed
• The one-way analysis of variance
• The logic of ANOVA
• ANOVA assumptions
JV65UCK2AH
• F test for difference in c means
• The two-way analysis of variance
• Examined effects of multiple factors
• Examined interaction between factors

Sharing All Rights
action. prohibited.

Analysis of Variance

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Analysis of Variance

Uploaded by

Copyright:

Available Formats

Analysis of Variance

• Experimental units (subjects) are assigned randomly to groups

• Analyzed by one-factor analysis of variance (ANOVA)

SST = SSA + SSW

SST = Total Sum of Squares

This file is meant for personal use by girishchadha.gc@gmail.com only.

SST = SSA + SSW

Total Variation = the aggregate variation of the individual

Among-Group Variation = variation among the factor

Within-Group Variation = variation that exists among

This file is meant for personal use by girishchadha.gc@gmail.com only.

Variation Due to Variation Due to Random

This file is meant for personal use by girishchadha.gc@gmail.com only.

SST = Total sum of squares

Group 1 Group 2 Group 3

Group 1 Group 2 Group 3

SSW = (X11 − X1 ) + (X12 − X 2 ) +    + (Xcn c − Xc )

Group 1 Group 2 Group 3

SSA Mean Square Among

Source of Degrees of Sum Of Mean Square F

Among SSA FSTAT =

Total n–1 SST

This file is meant for personal use by girishchadha.gc@gmail.com only.

You want to see if three Club 1 Club 2 Club 3

This file is meant for personal use by girishchadha.gc@gmail.com only.

MSA = 4716.4 / (3-1) = 2358.2 2358.2

H0: μ1 = μ2 = μ3 Test Statistic:

This file is meant for personal use by girishchadha.gc@gmail.com only.

• Examines the effect of

This file is meant for personal use by girishchadha.gc@gmail.com only.

• Populations are normally distributed

This file is meant for personal use by girishchadha.gc@gmail.com only.

SST = SSA + SSB + SSAB + SSE Degrees of

Sum of Squares Error:

This file is meant for personal use by girishchadha.gc@gmail.com only.

F Test for Interaction Effect

This file is meant for personal use by girishchadha.gc@gmail.com only.

Factor A Levels Factor A Levels

This file is meant for personal use by girishchadha.gc@gmail.com only.

This file is meant for personal use by girishchadha.gc@gmail.com only.

You might also like