HURLBERT THE EXPERIMENTAL APPROACH

ABSTRACT 5 COMPONENTS TO AN EXPERIMENT

Pseudoreplication • Hypothesis
• (In manipulative experiments) Defined as o Of primary importance
the use of inferential statistics to test for ▪ If it is not good, the
treatment effects with data from experiment will be of little
experiments where: value
o either treatments are not • Experimental design
replicated (though samples may be) o The logical structure of the
equab
probability for
or experiment
independence of samples hinge
on
o replicates are not statistically o Based on the objectives, and the
I

various treatments employed

each be on the
.

to
sample
independent objectives should describe or specify
• In ANOVA terminology: the:
o The testing for treatment effects with ▪ Nature of experimental units
.J D.
or S E
. for SUBSAMPLES
.
are not important error term inappropriate to the to be employed (ex. the type
hypothesis being considered of glassware)
• Refers to not a problem with experimental ▪ The number and kinds of
PROBLEMS & SOUN :
design, but to a particular combination of treatments
① Nondemonic
intrusion experimental design (or sampling) and ▪ Properties or responses of
> Interspersion statistical analysis which is inappropriate the experimental units that
② Bias for testing the hypothesis of interest will be measured (ex.
and interspersion
>Randomization accuracy and precision)
Nondemonic intrusion o It should specify:
③ Random error
>Replication • The impingement of chance events on an ▪ How treatments are assigned
experiment in progress to the available experimental
• As safeguard against this and other units
preexisting gradient, INTERSPERSION is ▪ The number of replicates per
considered an obligatory feature of a good treatment
design. ▪ Physical arrangement of
experimental units
• Randomization and interspersion can ▪ Temporal sequence in which
sometimes be in conflict with one another, treatments are applied
especially in small experiments. ▪ Measurements made on
Comprehension of this conflict can be aided different experimental units
by: • Experimental execution
o Distinguishing pre-layout o Success requires avoiding systematic
(conventional) and layout-specific error and minimizing random error
alpha (the probability of a type I ▪ Systematic error renders an
error) experiment invalid or
▪ Type I error = when the null inconclusive
hypothesis is rejected, when it o Errors in experimental execution are
is in fact true often subtler and more likely to
• When it was happen than design errors + they
concluded that there are more difficult to detect
is a significant • Statistical analysis
difference, when there o To increase the clarity, conciseness,
really is none and objectivity with which results are
presented and interpreted
 • Interpretation
• Interpretation and statistical analysis are the
least critical aspects of experimentation, as
the data can be reanalyzed.
• The only complete remedy for design or
execution errors is repetition of the
experiment.
MENSURATIVE EXPERIMENTS
• Involve only the making of measurements at
one or more points in space or time
• Space or time is the only experimental
variable or treatment.
• Usually do not involve the imposition of an
external factor, thus, every experimental
units are treated identically
QUESTION:
EXAMPLE:
• Objective: to determine the decomposition
rate of maple leaves in a 1-m isobath
COMPARATIVE MENSURATIVE EXPERIMENTS
• Measures a property of the system at two
points within it and asking whether there is
a real difference
EXAMPLE:
• Objective: to determine the decomposition
rate of maple leaves in a 1 m isobath vs. 10
m isobath
• Hypothesis: the decomposition rate will be
different
IMPORTANT:
• The dispersion of the replicate samples in a
manner appropriate to the hypothesis being
tested is the most critical aspect of a
mensurative experiment.
PSEUDOREPLICATION IN MENSURATIVE
EXPERIMENTS
• Often a consequence of the actual physical
space over which samples are taken, or
• Measurements being made smaller than
the inference space implicit in the
hypothesis being tested
o Inference space is the space in which
the samples occupy. It should be
congruent with the sampling domain,
which is based on the objectives of
the experiment.
MANIPULATIVE EXPERIMENT
• Involves two or more treatments
• The different experimental units receive
different treatments and the assignment of
these treatments can be randomized
Difference with Mensurative Experiments:
• Mensurative: comparing photosynthetic
rates of naturally established oak and maple
trees
o Treatment variable: species
o Randomized assignment of
treatments to locations WOULD
NOT BE POSSIBLE
• Manipulative: comparing decomposition
rate of maple and oak leaves
o Treatment variable: species
o Randomized assignment of
treatments to locations IS POSSIBLE
CRITICAL FEATURES OF A CONTROLLED
EXPERIMENT
Critical Features of Experimental Design:
1. Controls
o “Any treatment against which one or
more other treatments is to be
compared”
o May be an:
▪ Untreated treatment
• No imposition of an
experimental variable
▪ Procedural treatment
▪ Simply a different treatment
o In experimentation with biological
systems, controls are required
because these systems exhibit
temporal change.
o Controls for temporal change and
procedure effect
 o Regulation of the conditions under
which the experiment is conducted
▪ The homogeneity of
experimental units
▪ Precision of treatment
procedures
▪ Regulation of the physical
environment in which the
experiment is conducted
o ^ However, the above definition is
unfortunate usage, as the adequacy
of true controls is independent of
this regulation of physical
conditions, thus the validity of the
experiment is unaffected by this
regulation.
__
• Replication and randomization both
improve estimation and permit testing.
2. Replication
o Controls for the stochastic factor,
which is the variability among
replicates arising from:
▪ The material itself
▪ The experimenter
▪ Nondemonic intrusion
o Reduces the effects of random error
and increases precision
o With respect to testing, its main
purpose is:
▪ supply an estimate of error
by which the significance of
these comparisons is to be
judged
3. Randomization
o Controls for potential experimenter
bias in the assignment of
experimental units to treatments and
the execution of the experimental
procedure
o Increases accuracy of estimates
o With respect to testing, its main
purpose is:
▪ To guarantee the validity of
the significance test based on
the error estimate from
replication
Why is this important?
• Randomization confers validity because it
guarantees that, on the average, errors are
independently distributed.
o The lack of independence of errors
prohibits us from knowing alpha, the
probability of a type I error.
4. Interspersion
o Controls for regular spatial variation
in properties of the experimental
unit
▪ The physical layout of the
experiment and how
experimental units should be
distributed in space
o Most of the time, when you have
randomization, you also have
interspersion, but that is NOT always
the case.
▪ Interspersion is the most
critical concept, and
randomization is simply a
way of achieving
interspersion → eliminate
bias → allow accurate
specification of the alpha
o For preliminary assessment of the
adequacy of experimental designs,
interspersion is more practical to
consider than randomization.
Sources of Confusion in Manipulative Experiments:
 MODES OF SPATIAL INTERSPERSION AND
SEGREGATION
A-1 COMPLETELY RANDOMIZED DESIGN
• Most basic and straightforward way of
assigning treatments to experimental units
• Not frequently employed in ecological field
experiments (when experimental units are
large)
• Has a good chance of producing treatments
that are segregated rather than spatially
interspersed
A-2 RANDOMIZED BLOCK DESIGN
• Commonly used design in ecological field
experiments
• Reduces the probability of chance
segregation of treatments
• Helps prevent pre-existing gradients and
nondemonic intrusion
A-3 SYSTEMATIC DESIGN
• Achieves a very regular interspersion of
treatments but runs the risk that the spacing
interval coincides with the period of some
periodically varying property of the
experimental area
• In both systematic and randomized block
designs, we can base the assignment
process NOT ON THE LOCATIONS but on
the internal properties of the experimental
units prior to imposition of treatments.
o Risk: spatially segregated treatments
The magnitude of this risk decreases with
•
increasing number of replicates.
B-1 AND B-2 SIMPLE AND CLUMPED
SEGREGATION
• Simple segregation
o Only one layer of treatments, but the
placement of a treatment group is
adjacent to the same group
• Rarely employed in ecological field
experiments
• More commonly found in laboratory
experiments
• Dangers lead to spurious treatment effects.
Causes:
o Pre-existing difference in the
“locations” of two treatments
o Non-demonic intrusion
▪ Difference between locations
can become greater during
the experiment
independently of any true
treatment effect
B-3 ISOLATIVE SEGREGATION
• Poses all dangers of simple segregation but
in a more extreme form
• Further increase the likelihood of a spurious
treatment effect, as within-treatment
variances are less likely to be increased
B-4 PHYSICALLY INTERDEPENDENT REPLICATES
• Example:
o 4 aquaria in each set sharing
common heating, aeration, filtration,
etc.
• The replicates are dependent on each
other.
RANDOMIZATION VS. INTERSPERSION
• POSSIBLE SOLUTION (Cox)
1. Reject highly segregated layouts and
rerandomize
o Risk: precludes the probability of
knowing the exact value of alpha
Pre-layout and Layout-specific Alpha
• αPL as Aph
o Conventional alpha units are segregated
· if experimental
o The probability, averaged over all -
degree
to some
possible layouts of a given
experiment, of making a type I error
well-intersported
aus perimental units are
 • αLS
o the probability of making a type I
error if the layout was used
o will usually be less than or greater
than the pre-layout alpha
• When not adhering to strict randomization
procedures, the αPL will have marginal error.
However, it does not mean that it cannot be
done. It is up to experimenter to determine
whether knowing the aPL exactly is better
than having an idea of a possible upper
bound to aLS.
Biased estimation of treatment effects
• The greatest bias in estimating a treatment
effect will result from some particular
nonsystematic design and not from a
systematic one.
PSEUDOREPLICATION IN MANIPULATIVE
EXPERIMENTS
• If treatments are spatially or temporally
segregated (B1-3), if all replicates of a
treatment are interconnected (B-4), or if
“replicates” are only samples from a single
experimental unit (B-5), then it is
pseudoreplication.
o At best, they can only demonstrate a
difference between locations
SIMPLE PSEUDOREPLICATION
• Involves only a single replicate per
treatment
• The validity of using unreplicated treatments
depends on the experimental units being
identical at the time of manipulation and
after manipulation, except as there is a
treatment effect.
o The lack of significant difference
prior to manipulation cannot be
interpreted as evidence of
identicalness, as it can only be a
consequence of a small number of
samples taken from each unit.
• Two experimental units are different in
every measurable property
o Thus, if we increase the number of
samples taken from each unit, our
chances of finding a significant
premanipulation difference will
increase (alpha will approach 1.00)
▪ THIS IS NOT WHAT WE NEED.
• Replicability is a false issue. It pertains to
the similarity that can be obtained once the
experimental design is repeated.
o The question to be asked is NOT:
▪ Are experimental units
sufficiently similar for one to
be used per treatment?
o Rather, it is:
▪ Given the observed or
expected variability among
experimental units, how
many should be assigned to
each treatment?
TEMPORAL PSEUDOREPLICATION
• Multiple samples from each experimental
unit are taken sequentially over several
dates, which are then taken to represent
replicated treatments
SACRIFICIAL PSEUDOREPLICATION
• Results when an experimental design:
o Involves true replication of
treatments but data are pooled prior
to statistical analysis, or
o Where two or more samples taken
from each experimental unit are
treated as independent replicates
(the concept of subsamples)
• Example:
o In any field situation, two replicate
plots or ponds in the same
treatment are NOT identical, so a
significance test of the difference is
not relevant.
CHI-SQUARE AND PSEUDOREPLICATION
Employing a Chi-square analysis pn subsamples
rather than replicates.
IMPLICIT PSEUDOREPLICATION
• When authors seem to regard paired and
non-overlapping 95% confidence intervals
as equivalent to significance tests, and if
 they offer no specific disclaimer that their
data is inadequate in assessing treatment
effects

QUESTIONS
• Can mensurative experiments have control
treatments?
o Mostly, no.

DISCUSSION ABOUT QUIZ #1

1. To increase the precision and accuracy of
the results, it is better to increase the
number of replicates in an experiment than
to increase the number of subsamples.

Factors that can affect precision and accuracy:

• Replicates
• Subsamples

HEADING

HEADING
KJASKDJSA

HEADING
Sdd