Experimental design

Experiments vs. observational

studies

Manipulative experiments: The only way to prove the

causal relationships
BUT
Spatial and temporal limitation of manipulations
Side effects of manipulations
Example of side effects –
exclosures for grazing
 Exclosures have significantly
higher density of small rodents

????????????
The poles of fencing are perfect
perching sites for birds of pray
 Laboratory, field, natural trajectory (NTE), and natural
snapshot experiments (Diamond 1986)
Lab Field NTE NSE
Regulation of Highest Medium/low None None
indep. variables
Site matching Highest Medium Medium/low Lowest
Ability to follow Yes Yes Yes No
trajectory
Maximum Lowest Lowest Highest Highest
temporal scale
Maximum Lowest Low Highest Highest
spatial scale
Scope (range of Lowest Medium/low Medium/hig Highest
manipulations) h
Realism None/low High Highest Highest
Generality None Low High High

NTE/NSE - Natural Trajectory/Snapshot Experiment

 Observational studies
(e.g. for correlation between
environment and species, or
estimates of plot characteristics)
Random vs. regular sampling plan
 Take care

Even if the plots are located randomly, some of

them are (in a finite area) close to each other,
and so they might be “auto-correlated”
Regular pattern maximizes the distance between
neighbouring plots
Regular design - biased results, when there is some regular
structure in the plot (e.g. regular furrows), with the same
period as is the distance in the grid - otherwise, better design
providing better coverage of the area, and also enables use
of special permutation tests.
 Manipulative experiments
frequent trade-off between feasibility and
requirements of correct statistical design and power of
the tests

To maximize power of the For the feasibility and

test, you need to maximize realism, you need plots of
number of independent some size, to avoid the
experimental units edge effect
 Important -
treatments randomly
Completely randomized design assigned to plots

Typical analysis: One way ANOVA

 Regular patterns of individual treatment type
location are often used, they usually maximize
possible distance and so minimize the spatial
dependence of plots getting the same treatment

Similar danger as for

regular sampling pattern -
i.e., when there is inherent
periodicity in the
environment – usually
very unlikely
When randomizing, your treatment
allocation could be also e.g.:

Regular pattern
helps to avoid
possible
“clumping” of the
same treatment
plots
 Randomized complete blocks

ENVIRONMENTAL GRADIENT

Block 1 Block 2 Block 3 Block 4

For repeated measurements - adjust the blocks (and even the

randomization) after the baseline measurement
ANOVA, TREAT x BLOCK interaction is the error term

TREAT BLOCK RESPO1 RESPO2

1 1 5 5
2 1 6 6
3 1 4 4
1 2 7 5
2 2 9 5
3 2 8 4
1 3 3 5
2 3 5 7
3 3 2 4
1 4 6 4
2 4 7 6
3 4 5 5
1 5 8 4
2 5 11 5
3 5 9 6
 2
1

0
1
If the block has
a strong
8 explanatory
6
power, the RCB
RESPO1

design is
4 stronger than
2
E
TR AT: completely
G1
_:1

G
E
TR
2
_
A
:2
T: randomized one
E
TR AT:
0
G1
_:1 G2
_:2 G3
_:3 G4
_:4 G5
_:5 G3
_:3

BLOCK

df MS df MS
Effect Effect Error Error F p-level
TREAT 2 6.066667 8 0.4 15.16667 0.001897
BLOCK 4 17 8 0.4 42.5 1.97E-05

TREAT 2 6.066667 12 5.933333 1.022472 0.389016

 7
.5

If the block has

7
.0
no explanatory
6
.5
power, the RCB
6
.0
design is weak
5
.5
RESPO2

5
.0

4
.5
TR
E AT:
G_
1:1
4
.0 TR
E AT:
G_
2:2
TR
E AT:
3
.5
G_
1:1 G_
2:2 G_
3:3 G_
4:4 G_
5:5 G_
3:3

BLOCK

df MS df MS
Effect Effect Error Error F p-level
TREAT 2 2.4 8 0.816667 2.938776 0.110435
BLOCK 4 0.166667 8 0.816667 0.204082 0.929067

TREAT 2 2.4 12 0.6 4 0.046656

 Reminder – Covariates
• Use of covariates (covariables) and
Analysis of Covariance – another possibility
how to filter out „noise“ and decrease the
unexplained variability
Latin square design
In most cases rather weak test if
analyzed as Latin square (i.e.
column and row taken as factors in
incomplete three way ANOVA)

Again, useful to avoid clumping of

the same treatment
 Most frequent errors - pseudoreplications

1.1. Most frequent errors – Pseudoreplications

Cited 4000+ times
Note, B. is in fact not a
pseudoreplication, if the
analysis reflects correctly
the hierarchical design of
the data
Oksanen, L
Logic of experiments in ecology: is
pseudoreplication a pseudoissue?
OIKOS 94 : 27-38

Hurlbert divides experimental ecologist into 'those who do not see

any need for dispersion (of replicated treatments and controls) and
those who do recognize its importance and take whatever measures
are necessary to achieve a good dose of it'. Experimental ecologists
could also be divided into those who do not see any problems with
sacrificing spatial and temporal scales in order to obtain replication,
and those who understand that appropriate scale must always have
priority over replication.
Factorial designs

Completely randomised
F for testing effects in various
combination of fixed and random
factors in two-way ANOVA

Tested Both fixed A-fixed, Both random

effect B-random
A MSA/MSerror MSA/MSAxB MSA/MSAxB

B MSB/MSerror MSB/MSerror MSB/MSAxB

AxB MSAxB/MSerror MSAxB/MSerror MSAxB/MSerror

Reminder Type I and Type III SS
• If the design is balanced, you don’t need to
care
• In non-balanced designs –
• Type I – sequential – the order of predictors
IS important
• Type III, Type VI
What is the interaction?
 Log transformation and the
interaction
If the interaction = 0 - we expect the pure additivity

Effect of A+B = Effect of A + Effect of B

Null model for interaction: Xijk = m + ai + bj + eijk
(fertilization increases height by 5cm)

Model with interaction: Xijk = m + ai + bj + gij + eijk

Often biologically more feasible null model: pure multiplicativity

(fertilization increases height by 20%)
Null model for interaction: Xijk = m . ai . bj . eijk
Then: log(Xijk) = log(m . ai . bj . eijk) = log m + log ai + log bj + log eijk
If you log-transform in a factorial
ANOVA
• Think more about the meaning of the
interaction, the distributional properties of
response variable are (usually) less
important
FIXED AND RANDOM FACTORS
F for testing effects in various
combination of fixed and random
factors in two-way ANOVA

Tested Both fixed A-fixed, Both random

effect B-random
A MSA/MSerror MSA/MSAxB MSA/MSAxB

B MSB/MSerror MSB/MSerror MSB/MSAxB

AxB MSAxB/MSerror MSAxB/MSerror MSAxB/MSerror

 Fertilization experiment in three countries
Difference of meaning of the test, depending on
whether the country is factor with fixed or random
effect
COUNTRY FERTIL NOSPEC
1 CZ 0.000 9.000
2 CZ 0.000 8.000
3 CZ 0.000 6.000
4 CZ 1.000 4.000
5 CZ 1.000 5.000
6 CZ 1.000 4.000
7 UK 0.000 11.000
8 UK 0.000 12.000
9 UK 0.000 10.000
10 UK 1.000 3.000
11 UK 1.000 4.000
12 UK 1.000 3.000
13 NL 0.000 5.000
14 NL 0.000 6.000
15 NL 0.000 7.000
16 NL 1.000 6.000
17 NL 1.000 6.000
18 NL 1.000 8.000
 Country is a fixed factor (i.e., we are
interested in the three plots only)
Summary of all Effects; design: (new.sta)
1-COUNTRY, 2-FERTIL
df MS df MS
Effect Effect Error Error F p-level
1 2 2.16667 12 1.055556 2.05263 .171112
2 1 53.38889 12 1.055556 50.57895 .000012
12 2 26.05556 12 1.055556 24.68421 .000056

Country is a random factor (i.e., the three plots are

considered as a random selection of all plots of
this type in Europe - [to make Brussels happy])
Summary of all Effects; design: (new.sta)
1-COUNTRY, 2-FERTIL
df MS df MS
Effect Effect Error Error F p-level
1 2 2.16667 12 1.05556 2.05263 .171112
2 1 53.38889 2 26.05556 2.04904 .288624
12 2 26.05556 12 1.05556 24.68421 .000056
Nested design („split-plot“)
Two explanatory variables, Treatment and Plot,
Plot is random factor nested in Treatment.
Accordingly, there are two error terms, effect of
Treatment is tested against Plot, effect of Plot
against residual variability:
F(Treat)=MS(Treat)/MS(Plot)
F(Plot)=MS(Plot)/MS(Resid) [often not of interest]
 Split plot (main plots and split plots - two error levels)

Plot 1 Plot 2 Plot 3

N C N
C N
C
P P P

Plot 4 Plot 5 Plot 6

N
P C P N
C C
N P
ROCK PLOT TREA RESP
1 1 1 5 ROCK is the MAIN PLOT factor,
1 1 2 8
1 1 3 6 PLOT is random factor nested in
1 2 1 6
1 2 2 8 ROCK, TREATMENT is the within
1 2 3 6 plot (split-plot) factor.
1 3 1 2
1 3 2 3
1 3 3 3
2 1 1 5 Two error levels:
2 1 2 6
2
2
1
2
3
1
5
5
F(ROCK)=MS(ROCK)/MS(PLOT)
2 2 2 4
2 2 3 3 F(TREA)=MS(TREA)/MS(PLOT*TREA)
2 3 1 5
2 3 2 7
2 3 3 6

df MS df MS
Effect Effect Error Error F p-level
ROCK 1 0.055556 4 8.944445 0.006211 0.940968
PLOT 4 8.944445 0 0
TREA 2 3.166667 8 0.611111 5.181818 0.036018
ROCK*PLOT
ROCK*TREA 2 0.722222 8 0.611111 1.181818 0.355068
PLOT*TREA 8 0.611111 0 0
3way
Following changes in time
Non-replicated BACI (Before-after-control-
impact)
 Analysed by two-way ANOVA
factors: Time (before/after) and Location (control/impact)
Of the main interest: Time*Location interaction (i.e., the
temporal change is different in control and impact
locations)

13
13
12
12
11 11

10 10
CD

PB

9 9

8 8

7 7
TIME: TIME:
BEFO RE BEFO RE
6 T6
IME: TIME:
CONTR IMPACT AFTER CONTR IMPACT AFTER

LOCATION LOCATION
In fact, in non-replicated BACI, the test is based on
pseudoreplications.
Should NOT be used in experimental setups

In impact assessments, often the best possibility

(The best need not be always good enough.)
Replicated BACI - repeated measurements

Usually analysed by
“univariate repeated
T0 Treatment T1 T2
measures ANOVA”.
Control This is in fact split-plot,
where TREATment is
Impact the main-plot effect,
time is the within-plot
Control
effect, individuals (or
experimental units) are
Impact
nested within a
treatment.
Impact

Of the main interest is

Control
interaction
TIME*TREAT
TRE T1 T214 T3
1 5 6 7
1 6 13 5 8
1 5 12 7 7
2 4 7 11
11
2 6 8 12
2 5 10 9 15
9
Height

8
7
6
5 TR
E :
G_
1:1
4 TR
E :
T1 T2 T3 G_
2:2

TIME

df MS df MS
Effect Effect Error Error F p-level
1 1 24.5 4 2.111111 11.60526 0.027111
2 2 35.72222 8 0.944444 37.82353 8.37E-05
12 2 12.16667 8 0.944444 12.88235 0.003151