PSYC 2060

Research and Quantitative Methods in

Psychology

Session 6

1
 One-way ANOVA
Examples:

2
 Factorial ANOVA
Factorial design: A research design in which each level of an
independent variable (IV, a.k.a. factor in ANOVA contexts) is
paired with each level of every other IV. A factorial design thus
includes all the combinations of the IVs’ levels. Examples:

Factors: 4 hours of 7 hours of

• Amount of sleep (2 levels) sleep sleep
• Age (2 levels) Ages 20 - 25 Group 1 Group 3
Ages 60 - 65 Group 2 Group 4
DV = arithmetic calculation performance

Factors:
• Diet (3 levels)
• Exercise (3 levels)
DV = weight reduction
3
 ANOVA terminology
Examples IV 1
level 1 level 2 level 3
• 2 x 3 ANOVA: two IVs, one IV 2 level 1
with 2 levels and one with 3 level 2

levels IV 1
level 1 level 2 level 3 level 4
• 3 x 4 ANOVA: two IVs, one IV 2 level 1
level 2
with 3 levels and one with 4 level 3
levels
IV 3
• 3 x 2 x 2 ANOVA: three IVs,
one with 3 levels and two IV 1
with 2 levels
IV 2
A cell is a combination of one level of an
IV with one level of every other IV.
4
 Factorial ANOVA (example)

• The dependent
variable (DV) is
word memory
performance
• The IVs: 2
1
– Anxiety
– Word type
3 4
• A between
participants design
• 2 x 2 ANOVA model

5
 Factorial ANOVA (example)
• Main effect: the overall effect of each IV on the DV:
 the overall effect of anxiety on memory across both types
of words
 the overall effect of word type on memory across both
anxious and non-anxious individuals.
 Examples:

6
 Factorial ANOVA (example)
• Interaction effect: the effects of one of the IVs are not the same
across all levels of another IV.
• Examples:

7
 Factorial ANOVA (example)
The null hypotheses:
• no differences in marginal
means for anxiety (i.e., no
main effect of anxiety)
Marginal mean = • no differences in marginal
13.43 means for word type (i.e., no
main effect of word type)
• no interaction effects
Cell mean = 14.75 Cell mean = 12.10 between anxiety and word
type.

A cell mean is the mean for a

Marginal mean = combination of one level of an IV with
11.43 one level of every other IV.

A marginal mean is the mean for one

level of an IV across all levels of the
Cell mean = 9.65 Cell mean = 13.20 other IV (s).
Grand mean = Marginal mean = Marginal mean =
12.43 12.20 12.65 8
 Sources of variability
Sources of
variabilities in a
one-way between-
subjects ANOVA:
Source:
prism.pharm.
nottingham.a
SStotal = SSbetween + SSerror
c.uk

Group means Grand mean

IV1 Sources of
Error variabilities in a
(within group) IV2 2 × 2 between-
subjects ANOVA:
SStotal = SSIV1 + SSIV2 + SSinteraction
Interaction
+ SSerror
9
 Factorial ANOVA (example)
Source SS df MS F p
Anxiety 4.05 1 4.05 0.285 0.595
Word type 80 1 80 5.633 0.020
Anxiety * Word type 192.2 1 192.2 13.534 0.000
Error 1079.3 76 14.20
Total 1355.55

Formulas for df, MS, and F

Source SS df MS F
Factor A (# of levels = a) SSA a–1 SSA/dfA MSA/MSWithin
Factor B (# of levels = b) SSB b–1 SSB/dfB MSB/MSWithin
AxB SSAxB (a-1)(b-1) SSAxB/dfAxB MSAxB/MSWithin

Within (Error) SSWithin N – a*b SSWithin/dfWithin

Total N-1

10
 Factorial ANOVA (example)
Source SS df MS F p
Anxiety 4.05 1 4.05 0.285 0.595
Word type 80 1 80 5.633 0.020
Anxiety * Word type 192.2 1 192.2 13.534 0.000
Error 1079.3 76 14.20
Total 1355.55

11
 Factorial ANOVA (example)
Source SS df MS F F*critical p
Anxiety 4.05 1 4.05 0.285 3.98 0.595
Word type 80 1 80 5.633 3.98 0.020
Anxiety * Word type 192.2 1 192.2 13.534 3.98 0.000
Error 1079.3 76 14.20
Total 1355.55
* Approximate value from statistical table

This ANOVA provided three significance tests:

• the main effect of anxiety - comparing the marginal means
of anxious versus non-anxious individuals;
• the main effect of word type - comparing the marginal
means of neutral versus negative words; and
• the interaction effect between anxiety and word type –
comparing the effects of an IV across different levels of the
other IV.
12
 Factorial ANOVA (example)
Source SS df MS F p
Anxiety 4.05 1 4.05 0.285 0.595
Word type 80 1 80 5.633 0.020
Anxiety * Word type 192.2 1 192.2 13.534 0.000
Error 1079.3 76 14.20
Total 1355.55

• The main effect of anxiety

on recall was not significant,
F(1,76) = 0.285, p = .595
• The main effect of word
type on recall was
significant; Overall, the
recall on negative words
was better than that on
neutral words, F(1,76) =
5.633, p = .020.
Note: Generally, on such mean plots, the distances Source: Dancey et al.
between the “mid-points” represent the main effects only
when the cells concerned are equal in group sizes. 13
 Factorial ANOVA (example)
Source SS df MS F p
Anxiety 4.05 1 4.05 0.285 0.595
Word type 80 1 80 5.633 0.020
Anxiety * Word type 192.2 1 192.2 13.534 0.000
Error 1079.3 76 14.20
Total 1355.55

• There was a significant

interaction between anxiety
and word type, F(1,76) =
13.534, p < .001
• For anxious individuals, recall
appeared to be better for
negative words relative to
neutral words; whereas the
non-anxious individuals showed
the opposite pattern (though
the difference magnitude
appeared to be small for the
non-anxious individuals). Source: Dancey et al.
14
 Factorial ANOVA
When an interaction effect is statistically significant, we should (a) first
study and test the nature of the interaction effect (as further discussed
below) and then (b) ascertain whether the main effects are still of
interest*
Example: The main effect of expectation is Expect alcohol

an average of the various degrees of

differences in aggression (across “expect Do not expect alcohol
alcohol” and “do not expect alcohol”)
under the various alcohol conditions
(King et al., 2011)

*Some researchers argue that when IVs interact, examining their main effects
is not needed or not productive. Howell (2010): it is not reasonable to
automatically exclude interpretation of main effects in the presence of any
interaction between the IVs concerned. It depends on whether the main effects
can be meaningfully interpreted.
NOTE: Main effects need to be included anyway when reporting F tests’ results.
15
 Interaction effect
• When an interaction effect is statistically significant, the effect
of the IV of interest at each level of the other IV(s) should be
examined for interpreting the results - using two
complementary approaches (see the slides below): profile plots
of cell means
Examples of
and post-hoc simple effects
testing of simple
effects
• A simple effect is
the effect of one
of the IVs at a Source: Dancey et al.

particular level of
another IV.

16
 Profile plot of cell means
(a) “parallel-line” patterns suggesting the absence of interaction effects

Source: Howell
Source: Dancey et al.

(b) “non-parallel-line” patterns suggesting the presence of interaction effects

Source: Dancey et al. Source: Howell

17
 Ordinal and disordinal interaction
• Ordinal interactions: an interaction for which the relative
ranks for the levels of one IV are the same at all levels of
the other IV
• Disordinal interactions: an interaction for which the
relative ranks for the levels of one IV differ across the
levels of the other IV.

Ordinal interactions (example) Disordinal interactions (example)

18
 Post-hoc testing of simple effects
• For this example, depending on the research questions, we can examine the
simple effects (a) of word type at different anxiety levels or (b) of anxiety for
different word types - see the next two slides.

• The overall Type I error rate should be controlled, e.g., by applying Bonferroni
correction:
 The adjusted alpha level ’ =  FW / K
 FW : familywise Type I error rate
K : the number of comparisons to be performed
 An individual comparison is then taken as statistically significant only if the
no-correction p value is less than ’.
19
Post-hoc testing of simple effects

20
Post-hoc testing of simple effects

21
 Factorial ANOVA (example)
There was a significant interaction Source: Dancey et al.

between anxiety and word type,

F(1,76) = 13.534, p < .001. For
anxious individuals, recall was
significantly better for negative
words compared with neutral
words, t(76) = 4.280, p < .001 (after
Bonferroni correction); whereas for
non-anxious individuals, there was
no significant difference in recall
between the two types of words,
t(76) = 0.923, p > .999 (after
Bonferroni correction).

NOTE: While profile plots aid the interpretation of interaction effects, conclusions on
presence of interactions (at population level) should be based on statistical tests.
22
 Factorial ANOVA
See the notes above: When an interaction effect is statistically significant, we should first study and
test the nature of the interaction effect to ascertain whether the main effects are still of interest …

When there are no statistically significant interaction effects between the

IVs and an IV’s main effect of interest is statistically significant:
How many levels
2 levels the IV has? > 2 levels

No further post-hoc testing is Examine the data as a one-way ANOVA

needed to understand the main for the IV of interest and apply a post
effect’s nature hoc test to compare the means across
different pairs of the IV’s levels
Example: Example (assuming A is the IV of interest):

Source: Howell 23
Source: Dancey et al.
 Effect size
ETA squared:
• h2 = SSgroups / SStotal
• the percentage of variability of the DV that can be
attributed to a main or interaction effect
• a biased measure – it tends to overestimate the
population effect size

Omega squared: a = the number

of levels of an IV
a less biased measure
compared with h2 a and c = the
numbers of
levels of the
two interacting
IVs (A & C)
Source: Howell

24
 Effect size

Total 1355.55

Word type:
• ETA squared = 80/1355.55 = 0.059 h2 = SSgroups / SStotal
• Omega squared
= 80 – (1)(14.201) / (1355.55 + 14.201)
= 0.048
Anxiety * word type:
• ETA squared = 192.2/1355.55 = 0.142
• Omega squared
= 192.2 – (1)(1)(14.201) / (1355.55 + 14.201)
= 0.130
25
 Effect size
For comparison of two (cell or marginal) means in ANOVA, the MS error
provides an appropriate standardizer of Cohen’s d as an effect size measure.

Source: Howell

26
 Assumptions
• The dependent variable (DV) is measured at the
interval or ratio level
• Each of the independent variables (IVs) consists of
two or more categorical levels
• Observations are all independent of one another
• The DV is normally distributed for each group
population of DV scores (i.e., for each combination of
the levels of the IVs)
• All group populations of DV scores have the same
variance (homogeneity of variance) – NB: ensure that
serious heterogeneity of variance and unequal sample
sizes do not occur together (Howell).
27