(eBook PDF) Experimental Design for

the Life Sciences 4th Edition

Visit to download the full and correct content document:
https://ebooksecure.com/download/ebook-pdf-experimental-design-for-the-life-scienc
es-4th-edition/
Preface

Why you should read this book

A little effort (this is a short book) will help you design better experiments, and design­
ing better experiments will make your life richer and happier and possibly easier! First
of all, collecting data to test theories is the very essence of science—no matter what
sort of scientist you aim to be, you really need to be able to design good experiments
yourself and/or evaluate other scientists’ experiments. So don’t fool yourself by
pretending experimental design doesn’t matter to you. We are confident that whether
you have never designed an experiment before or whether you have been doing it for
some time, there are ideas in this book that will improve your confidence and ability to
design and evaluate data collection. Don’t fool yourself into thinking that experimental
design is an abstruse exercise in applied maths that is beyond you. Flick through this
book, there is barely an equation, and we have spent four editions honing our explan­
ations to make them as clear as possible—without a shadow of doubt, if you read
this book, then you will understand it and you will be a better scientist for it. At the
opposite extreme, be wary of thinking that experimental design is so simple that this
book isn’t worth your time. If that were true then the world would not be littered with
student lab reports with low grades because the student made basic mistakes in ex­
perimental design, and manuscripts by professional scientists that will never become
published scientific papers for the same reason. If you read this book then for the rest
of your scientific life you will design experiments that give clearer answers to more
interesting questions, using less of your time, fewer resources, and simpler statistics.
Not only that, but you will get more from every scientific paper you read and every
scientific talk you listen to. The upshot of these benefits is that you will be a better
scientist and a happier person. Given all that, the question really is why on Earth should
you not read this book?

Why not to read this book

If you want a book on statistics this is not it. We do explore links between experimental
design and statistical analysis throughout the book, but this book has very little to
teach you about formal statistics. If you want an encyclopaedic tome covering every
possible experimental design, then we cannot offer that either: that would be a book
at least ten times the size. What we do offer is to give you an understanding of the
underlying philosophy common to all experimental designs, and a clear introduction
to terminology—these will help you to read books and papers that offer complex
designs, but here we only look in depth at relatively simple designs. But ‘simple’ does
not mean ‘toy’—the latest issues of the very best scientific journals have articles
based on experiments no more complicated that the ones we cover in this book, and
 viii Preface

most scientists will go through their whole career without having to delve into more
complicated designs.

Why we felt the need to produce yet another edition of

the book
In truth, some typos aside, there is nothing that was downright wrong in any of our
previous editions. With each new edition we have been responsive to feedback from
readers and from our own development as scientists to come up with improvements
in how we communicate the basic ideas of experimental design. This revision has
ook Title Stage Supplier Date
gn for the Life Sciences Revise 1 been
Thomson much08more
Digital sweeping
Apr 2016 that the previous two. We have extensively reorganized
and rewritten so as to emphasize linkages between different aspects of experimen­
tal design more effectively. Thus we think this book should function both to give you
quick answers when you have a very focused question and to give you a solid overview
3.1 Experimental manipulation versus natural variation 37
of the overarching concepts of experimental design. From that perspective, we hope
e amount of ice cream sold inBook
Customer England
Title in a given Stage that you readofDate
In a survey
Q 3.1 Supplier the
road-book at least once from cover to cover, and then keep it to dip into
OUP
ccidents. Experimental Design for the Life Sciences
This is unlikely to be a direct effect: that, Revise 1
killed badgers it08was
Thomson Digital
found
Apr 2016

m leads to muscle cramp or loss of consciousness that thosequickly withas andlevels

higher when of required.
intestinal parasites had lower
variable is at work: when the weather is warm then
3.1 Experimental manipulation versus natural variation 37
bodyweights: can we safely
nd separately) inexperienced sailors hire boats and conclude that intestinal
Learning features
There is a correlation between the amount of ice cream sold in England in a given Q 3.1 In a survey of road-
s dive intoweekrivers and
and the lakes.
number of drowning accidents. This is unlikely to be a directparasites
effect: that, cause weight loss or
killed badgers it was found
say, heavy consumption of ice cream leads to muscle cramp or loss of consciousness that those with higher levels of
t it is difficult to be confident that the pattern we is there
among swimmers. More likely a third variable is at work: when the weather is warm then
a plausible
intestinal third
parasites variable
had lower
bodyweights: can we safely
system ismore really due to the factors that we have effect
ice creams are sold but also (and separately) inexperienced sailors hire boats andthat could
conclude thatexplain
intestinal this
people who are not strong swimmers dive into rivers and lakes.
observation? parasites cause weight loss or
n with some other unmeasured variable. The only
In short, third variables mean that it is difficult to be confident that the pattern we is there a plausible third variable

ms with third
have observed in an unmanipulated system is really due to the factors that we have
variables
measured andBook istoto
Title
not due carrywith
correlation out experimental
some Stage
other unmeasured variable. The only
effect that could explain this
Supplier
observation? Date
Key definitions
way to be certain of removing problems with third variables is to carry out experimental
Experimental Design for the Life Sciences Revise 1 Thomson Digital 08 Apr 2016
There is a lot of jargon associated with experimental design and
manipulations.

Reverse causation
r The second problemBook Title
of correlational studies is reverse causation. ThisStage
can occur when Supplier
Reverse causation is Date statistical analysis. We have not tried to avoid this. Indeed, we
we see a relationship between factors A and B. It means mistakenly assuming that fac­ mistakenly concluding that
studies is reverse causation.
Experimental Design for theThis can occur whenRevise 1Reverse
Life Sciences causation
Thomson Digital is
08variable
Apr 2016
the broad design of your
tor A influences study
factor variable A influences
B, when in fact it is change in B that drives change in A.
themselves tomistakenly
B when actually it is B that
concluding that
have deliberately tried to introduce you to as much of the jar­
rs A and B. ItFormeans mistakenly
example, imagine assuming
a survey shows that those whothat fac­
consider regularly
influences A.

ct it is change
use recreational drugs also consider themselves to have financial worries. It might be
intoBconclude
those
tempting that drives
conditions
that arechange
a drug stillis likely
habit into
likely to A.befinancial
cause more problems.
variable A influences variable
representative
The reversethan the conditions in a lab. gon as we can. By being exposed to this jargon, we hope that it
causation explanation is that people who have financial problems are more
Thus, itof
is usually safer to generalize
B when
from a field study than likely actually it is B that
a lab study. Generalization is
ws thatthe
lecting those
broad who
designconsider themselves
your study to regularly will become second nature to you to use—and understand—it.
influences A.
than average to turn to drugs (perhaps as a way to temporarily escape their financial
a subject that is at the heart of statistical methodology, so we explore it a little further
r themselves to have financial worries. It might be
worries). In this case, we’d consider the first explanation to be the more likely; but the
reverse
bit is likely operating
to cause
those conditions are still likely to be more representative than the conditions in a lab.
incausation
Statistics Box 3.1.
explanation
financial
Thus,
is at least plausible. Indeed, both mechanisms could be
it is usuallyproblems.
simultaneously. The from
safer to generalize reverse
a field study than a lab study. Generalization is
This should also make reading more advanced texts and scien­
Sometimes, our research question means that a lab study is impractical. If we are
ple who have
the number
Let’s consider another example of reverse causation. There is a correlation between
a subject that is at the heart of statistical methodology, so we explore it a little further
financial
interested
of storksin problems
differences
nesting areofmore
between
on the chimneys
in Statistics Box 3.1.
likelyin and
the farmhouse
a Dutch sexes effort expended in feeding the young in
the number tific papers on experimental design less daunting. However, to
of children
aps as a way to intemporarily
the the family
nest using living in thefinches,
zebra house. Thisthen
escape sounds
researchtheir
bizarre until
itfinancial
would beyou think
very that larger
adifficult
lab studytoiscreate foraging
If weconditions
Sometimes, our question means that
families tend to live in larger houses with more chimneys available as nest sites for the
in an families
aviarylead
impractical. are
make negotiating the minefield of jargon more straightforward,
the first explanation towhere
storks. Large interested thethe conditions
beintodifferences
morebetween
likely;
more opportunities
were
the
but
for nesting
close
sexesthe enough
instorks
storks; effort to those naturally experienced by
expended
don’t lead to in feeding the young in

east plausible.
wild birds.
large families!
Indeed,
Reverse causation both
is
In using
the nest the wild
zebraparents
mechanisms
notwhere
a universal
will itbring
finches, then
could
problem of were
wouldback
be
correlational
a great
be very difficultdiversity
studies. Sometimes
to createof food conditions
foraging types gathered we have increased the number of definitions of key terms pro­
from inaanwide
aviaryvariety the
of conditions
microhabitats close enough
sometimes tohundreds
those naturally experienced
of metres by This
distant.
reverse causation can be discounted as a plausible explanation. In our correlational
study would
wild birds. In the wild parents will bring back a great diversity of food types gathered
bestreamer
of male tail impossible to faithfully
length and recreate
mating success describedin theitlab.
earlier, wouldThe
be danger is that any aviary vided and increased the level of detail in all definitions. Each key
from a wide variety of microhabitats sometimes hundreds of metres distant. This
f reverse causation.
experiments
extremely There
difficult thatishow
to imagine a
youcorrelation
theconducted between
on
number of matingsthis would
a male beindiscounted
gets later a season by Devil’s Advocates as
would be impossible to faithfully recreate in the lab. The danger is that any aviary
could affect his tail length when we catch him at the beginning of the season. However, word or phrase is emboldened at the point where we first use it,
of being
e chimneysimagine
athat entirely
Dutch
experiments
artefactual,
farmhouse
we had instead that you
caught
due to
and the
andconducted
measured
your
the
asking
number
onmales
this at
would
birds to operate in a foraging situation far
beofdiscounted
the end the season. by Devil’s Advocates as
In thatremoved
ouse. Thishissounds
case itbeingfrom
might that
entirely
be quite which
artefactual,
plausible they
that
bizarre until you think that larger
the have
duenumber been
to your evolutionarily
of asking
matings birds
a maletogets
hormone levels. The change in hormone levels might then affect the way that the
adapted
operate
affects to cope
in a foraging with. far
situation However,
and is given a clear definition in a box nearby.
if your research
removed question
from is they
that which about how
have egg
been laying affects
evolutionarily theto
adapted distribution of reserves of
cope with. However,
with moremale’s
chimneys
calcium inavailable
different as nest
research question
parts of sites
is about howfor
females’ eggthe
tail develops during the season in such a way that his tail length comes to depend
if your bodies, laying affects
then the distribution
it would be easieroftoreserves
argue of
that this
on the number of matings that he gets. In that case we would observe the relationship
pportunities for
is less
between tail
calcium
nesting in
likelyand
length to different
storks; parts
storks
be influenced
matings that
of by
females’
don’t
we expect, the bodies,to
lead then
but itenvironmental
would be
it would be easierand
conditions,
nothing to do with
to argue
hencethat this
a laboratory
is less
the reason that likely to
we believe
unlikely to youexperiment
be influenced
(females
mightofbe
and you may know
by the
preferring long
reasonable.
no example So,
of such
environmental
tails). Such a pathwayconditions,
as ever,
a thing the most
happening.
might seem and hence a laboratory
experiment might be reasonable. So, as ever, the most appropriate approach depends
appropriate
However, you approach depends
Statistics boxes
on your research question.
rsal problem of correlational
can be sure that the Devil’s
on your studies.
Advocate
research Sometimes
will find such arguments highly compelling unless
question.

ed as a plausible explanation. In our correlational To emphasize the important link between good experimental
and mating success described
StAtiSticS
StAtiSticS
earlier,
Box
Box
it would from
Generalizing
3.1 Generalizing
3.1
be
froma study
a study design and statistics we now include a number of statistical
he number of matings a male gets later in a season
catch
An important issue in the interpretation of a study is how widely applicable the re­
An important issue in the interpretation of a study is how widely applicable the re­
him at the beginning sults of theof theareseason.
study However,
likely to be. That is, how safely can you generalize from the
boxes. These boxes should help you to see how thinking about
sults of the study are likely to be. That is, how safely can you generalize from the10:42 AM
03-RuxtonandColegrave-Chap03.indd 37 09/04/16

and measured the particular set of subjects used in your study to the wider world? Imagine that we
males
particular setatofthe end of
subjects usedthe
in season.
your study to the wider world? Imagine that we
measured representative samples of UK adult males and females and found that
design helps you think about statistics and vice versa. We have
le that the number of matings
measured a male
representative gets
samples affects
of UK adult
taller.males
Neitherand
we females and
elsefound
are that
males
males
in
inthen
our sample
our sample
particularly
were
were
interested
on average
on
in our
5 cm
average
specific 5the
cm taller.
nor anyone
Neither
we arewe nor anyone else are
added the boxes at points where we think that keeping the
ormone levels might affect the way thatindividuals, rather interested in what

n in such a way thatthink

his tail
we can conclude
particularly
aboutlength comes
the population
more
interested in widely
towhich
from depend
from study
our specific of the sample.
individuals, ratherThis
webecomes easy if we
are interested
our sample was drawn. Our aim was to cre­
in what statistics in mind is particularly useful and include pointers in
we can conclude more widely from study of the sample. This becomes easy if we
ets. In that casethink
weate
would observe
a representative
about the thefrom
sample
population relationship
of UK adults. If we have done this well, then we should
which our sample was drawn. Our aim was to cre­
feel confident about generalizing our results to this wider population. Can we gen­
the main text to each box.
at we expect, but
ate aitrepresentative
would be nothing
sample ofto UKdo with
adults. If we have done this well, then we should
eralize even wider than this: say to European adults or human adults in general?
preferring longfeel confident
tails). about
Suchurge
We would generalizing
a pathway
a great might
deal of ourseem
caution results to this
here. If your wider
interest waspopulation.
in European Can we gen­
adults,
eralize even
then wider have
you should thandesigned
this: sayyour
to study
European
in the adults or to
first place human
obtain aadults in general?
representa­
no example of such a thing happening. However, you
We would urge of
tive sample a great dealrather
European of caution here.
than only If yourHowever,
UK adults. interestproviding
was in European
you add anadults,
e will find such arguments highly
thenappropriate
you should have compelling
amount of cautionyour
designed unless
to your interpretation
study in the firstthen it may
place be appropriate
to obtain a representa­
to use your knowledge of biology to explore how your results can be extrapolated
tive sample of European rather than only UK adults. However, providing you add an
more widely than your original target population. In this specific case, the specific
appropriate amount of caution to your interpretation then it may be appropriate
difference of 5 cm would probably have little relevance beyond your specific target
to use your knowledge
population of since
of UK adults, biology tocountries
other explore will
how your
differ results can
significantly be extrapolated
in factors (e.g.
moreracial
widely
mix,than yourare
diet) that original target
highly likely population.
to affect height. In this specific case, the specific
 BALB/c males, then even the most cynical critic would probably be happy to accept
94 6 Sample size, power, and efficient design
that it is very likely to be harmful to three-and-a-half-months-old BALB/c males, and
probably
workto allaadult
with malessubset
more limited of that genotype.
of that population,Whether it is also likely
say three-months-old to apply to females
male BALB/c
Customer Book Title Stage Supplier Date
mice, then technically any results of our study should only be applied to the population
or other genotypes is a matter of biological judgement and interpretation. At first sight,
OUP Experimental Design for the Life Sciences
that they represent (other Revise 1
three-months-old BALB/cThomson Digital
mice). Of course, how 08any
far Apr 2016
you might find such advice worrying. What use is a study that only applies to a subset
results can be generalized outside of the specific population studied depends on the Preface ix
of thebiology
Customer population? InBook
of the system.fact, we biologists
IfTitle find that the make these
food additive kinds to
is harmful
Stage of generalization
Date beyond the
three-months-old
Supplier
OUP BALB/c
specifics males, study
their then
ofExperimental even
Design
all the
ofLifemost
for the
the cynical We
time.
Sciences criticare
would
Reviseprobably
usually
1 be happy
happy
Thomson Digitalto accept
to assume that a study
08 Apr 2016
that it is very likely to be harmful to three-and-a-half-months-old BALB/c males, and
done on a random sample of British males may often tell us something useful about
Self-test questions probably to all adult males of that genotype. Whether it is also likely to apply to females
male humans 3.1 Experimental
in general, or that amanipulation
substance thatversus natural variation 37 well
or other genotypes is a matter of biological judgement andinduces tumours
interpretation. At firstin rabbits may
sight,
3.1 Experimental manipulation versus natural variation 37
Customer
indicate
youthat
mightitfind
would
such do so worrying.
advice in humans.WhatThe Book
use iskey Title
is to
a study remember
that only appliesthat such extrapolationS
to a subset
The more you think about experimental design,
There is athe easier
correlation between the amountanof
isThere is of the population?
aOUP
ice cream
correlation
assumption sold In fact,
between the
of yours.
biologists
inamount
England
You inmake
Experimental
of ice cream
should asold
be
these
given
in kinds
Design
England
ready to
of
in afor generalization
giventhe
persuade Life
Q 3.1 the
beyond
In Sciences
aInDevil’s
Q 3.1
survey
the
Advocate
a survey of
of Re
road- (based
road-
specifics
week and the number of their study
of drowning all ofThis
accidents. the time. We
is unlikely are
to be usually
a direct happy
effect: that, killed badgers
to assume that a itstudy
was found

designing robust experiments becomes.week

To and
getthe
you thinking
number of drowning accidents.
say, Thisdone
onheavy is unlikely
general
consumption
on of iceto
biological
a random be
cream a direct
knowledge)
leads
sample to muscleeffect:
of Britishthat
malesor that,
crampthe lossindividuals
mayofoften
killed
that
consciousness
tell us you
that badgers
those
something have
useful
it levels
wasofto
chosen
with higher
intestinal parasites about
had lower
found
study can
among swimmers. More likely a third variable is at work: when the weather is warm then
Q 6.4 You areofplanning
say, heavy consumption a leadsreasonably
ice cream to muscle cramp
be
male humans expectedor loss
in general, toorof consciousness
tell
that you something
a substance thattumours
about
that induces those with
thebodyweights:
population
in higher
rabbits that
canmay
we levels
well
safely of care
you really
while you are reading this book, we include a number
among swimmers. toof
study More self-test
measure
likely agrowth
third variable
about.
people whoindicate
is at work:
But
are notin that swimmers
when it would do
general
strong thebe so
weather
aware
dive into
more ice creams are sold but also (and separately) inexperienced sailors hire boats and
inrivers
humans.
isand
thatwarmthere
lakes. then
intestinal
The key is to remember that
might be a trade-off
parasites
conclude
between
parasites cause
had lower
that intestinal
such extrapolation
narrowing
weight loss or your
rates in bacterial cultures In short, is an variables
third assumption
mean of yours.
that You should
it is difficult be readythat
to be confident to the bodyweights:
persuade
pattern the
we Devil’s can we
is thereAdvocate
a plausible safely
(based
third variable
questions in every chapter. Often theremore
will ice
not be aareclear
creams right
sold but
experiencing different
14
also (and separately)
pool
have inexperienced
ofonexperimental
2
observed in Starting
an unmanipulated
general biological
sailors
subjects
with hire
tothat
a well-defined
system is really
knowledge)
boats
increase
due to the
the
and
standardization
hypothesis
factors
individualsthat
conclude
we you
that
and
effect
havehave
that
thatimprove
chosen
could explainpower,
intestinal
to study can
this and your
observation?
measured and not due to correlation with some other unmeasured variable. The only
people environmental
who are not strong Q swimmers
6.4 You are planning ability
aredive into
a to generalize
rivers and lakes.
reasonably from to
be expected that study.
tell you something about the populationcause
parasites that youweight
really care
loss or
or wrong answer to a question, but suggested answersstresses.
tostudy
alltoYou
measure growthway to be certain of removing problems with third variables is to carry out experimental
about.
In short, third variables
considering mean
increasing
rates in thethat
bacterial There
it is difficult
cultures Qmay
to
manipulations. beBut
2.1
in general
also
confident
Suggest
bethat
sometimes awarethe
some
thata
be there
pattern might
trade-off we
2)
be a trade-off
between
is there
Patients
between
increasingnarrowing
a plausible
nearer sample
the door
yoursize
third in order
arevariable
seen tot
first on
questions can be found at the back of the
have book.
number of
observed in cultures that you
experiencing
an unmanipulated different improve
Customer
system
pool of experimental subjects to increase standardization and improve power, and your
hypotheses
power
is really
Reverse due and
causation to the that
the could
improvement
Book
factors to power
Title we
that
ability to generalize from that study.
havemore effect
through
of the that
using could
more
doctor’s explain
uniform
Stage
time andthis
experimental
Supplier
attention.
environmental stresses. You are observation?
measure, butdue
thisto
would mean OUPThe explain the ofobservation
Experimental that
Design forcausation.
the Lifeonly
Sciences Revise 1 is beThomson Digital
measured and not correlation the material.
increasingwith some other That is,alsosometimes
unmeasured beincreasing
variable. The your sample size can
size inonly
order to achieved by
second problem correlational studies is reverse This can occur when Reverse causation
considering There may sometimes a trade-off between increasing sample
your experiment would need to people
we see drive
a relationship faster
between and B. It means mistakenly3)
onAimprovement
factors the Patients
assuming that fac­respond
mistakenly positively
concluding thatto increased l
number of cultures that you increasing between-subject
way to be certain of removing problems with third
tor A
improve
variables
influences
power
factor isand
B, when to the
carry
in fact outvariability.
it is change
to power This
experimental through
in B that drives change in might
A.
using more
associated
occur
variablebecause
uniform
with
Aexperimental increasing your
influences variable
spread into a second
measure,incubator.
but this would meanForjourney to work than on the way B whenbeing
actually nearer
it is B that the ward entr
material.
example, imagineThat
a survey is,shows
sometimes
that thoseincreasing
who consideryour sample
themselves size can only be achieved by
to regularly
manipulations. sample size requires you to use domestic chicks from two different suppliers rather
Take-home messages your experiment would needuseto home. influences A.
What effect would you expect increasing
recreational between-subject
drugs also consider themselves variability. This worries.
to have financial might It occur
might bebecause increasing your
spread into a second incubator.
these changesWhat
to have on the than
tempting one; orasize
to conclude two different
that a drug incubators
habit is likely to cause financialTesting
for yourfrom
problems. specific
petri
The hypotheses
dishes
reverse instead helps
of one; give
or focus to y
measure
14 2
effect would you expectStarting sample
with requires
well-defined you to use
hypothesis domestic chicks two different suppliers rather
Reverse causation
power of your study?
these changes to have on the
causation explanation is that people who have financial problems are more likely
your
than plants
average
than one;intothree
to turn different
or two different
drugs (perhaps batches
incubators
as a way
ofescape
forrather
your
to temporarily
actually
than
petri dishes finding
all in theout
instead
their financial same
of something
one; measure interesting. Do
orsession.
To help to consolidate your thinking, weThe
end most sections with
power of your study? worries).
QCustomer
second problem of correlational studies Inyour
isSuggest
2.1 reverse
plants
this case,
some we’dinconsider
three different
causation. the first batchesnearer
explanation
2) Patients
This can occur
rather
to beyouthan
thethe
Book
whenmoreall
think inare
the
likely;
Title
door same
something
butseen session.
the first
Reverse like:
on is round and Stage
the doctor’s
causation end up
reverse causation
hypotheses thatexplanation
could is at least plausible. Indeed, both mechanisms could be
more of the doctor’s time and attention.our
a take-home message. By including these throughout
we see the text
a relationship between factors A and
explain B.
OUP
operating
the
Anything
It simultaneously.
means
observation Anything
that
wewecan
mistakenly do toreduce
reduce
assuming
Experimental
can do to that variation
fac­
Design
variation for the
amongst
‘Chimps
amongst
mistakenly
Life
areour Sciences
samples
really
samples
concluding
will will increase1
that
animals,Revise
increase
interesting so I’ll go
tor A influences factor B, when in fact it is change B that
instorks drives change A. farmhouse the
people drive faster onpower
the powerofofour 3)
Let’s consider another example of reverse
ourstudy. Patients
study. respond
causation. There positively
is a correlation variable
between A influences
to increased variable
levels of activity and human c
numberCustomer of ain Book Title to be Stage Supplier d
we hope to give you an immediate opportunity to evaluate your journey the of
to work than on the way
OUP
of children in the
nesting on the chimneys
Experimental
family living in the house.
Dutch
associated with being
Design
This sounds bizarre
and
untilfor
andthere
youthe Life
think
are
the number
nearer
Sciences
that
bound
the ward
Blarger
when entrance.
actually
lots of interesting
itRevise
is B that
1 Thomson Dig
For example, imagine a survey shows that those who consider themselves to regularly
home. families tend to live in larger houses with more chimneys available as nest sites influences
understanding of a section before moving on. 6.4.2 Dealing withTesting specific
variation Wedesign
hypotheses
through agree helps for the
that focus toA.your study
givechimpanzees canand
beincreases
very interes
your c
use recreational drugs also consider themselves
storks. Largeto have
families leadfinancial worries.
to more opportunities It might
for nesting be
storks; storks don’t lead to
6.4.2 Dealing with variation through design
of actually finding out something interesting. Don’t get yourself into a position
them
A pilot study isapproach an exploration definedfor a while. However, is to use what you see in t
large families!
tempting to conclude that a drug habit is likely to An cause financial
alternative
Reverse causation is not a universal
problems. to dealing
youproblem The
think something
with reverse
of correlationallike:
sources
studies. Sometimes
of variation measure
6.2 Factors affect
of the
themstudy and system ateThis clear questions. Once asyou have your questio
causation explanation is that people who Anhave
reverse alternative can explicitly
financial
causation beapproach
problems
discounted include as toa them are
plausiblein our
dealing more
study.
with
explanation. likely ourmight
Indefined involve
correlational sources something
6.2of
simple
variation
Factors
asis to
affecting themeasure
power of a
conducted
measuring before
thelength
additional the
and ‘Chimpsmain
factors are
for thereally interesting
individualsearlier, used animals,
in beour so I’llso
study gothat
down they tocanthe zoo, videoYou the ct
study of
them
male
and
tail streamer
explicitly include
mating
them
success
in our potheses
described
study. This that
it would
might might involve answer something the question.
as simple as
than average to turn to drugs (perhaps asextremely a way to temporarily escape their financial
Boxes and online supplementary material worries). In this case, we’d consider the first
body
measuring
could affectBOX
explanation
oftodata
difficult
be included
6.1
his tailtheBOX
length
to
incollection
imagine
An
howanalysis.
our
additional
6.1
when
be
the number
weAnexample
the
catch
andin
example
of matings
Inthere
the
him at theof
factors
more
of
areaof
case male
boundourgets
statistical
things
statistical
for
beginning
likely; the
but
of the
the
tolater
food
power
individuals
season.
in a season
supplement
be
you
lots
powerwould
calculation:
However, used
study, wedata
of interesting
calculation:
expect
in
mightindecide
returning
our to
study
100 hours of footage.’
returning
observe
so that if your
they hyp
can
orderthatto refine
rather than research
only carrying out the study on three-months-old mice we will use mice
Inbe
to
thatincluded
casethat
aims hens’
imagine that we had instead to hens’
in
differ
and our eggs
in age,
caught eggs
analysis.
data-collection
and
but we
measured
thatwill Inkeep
the
We agree that chimpanzees
theacase
males
note
at
you
of
the
ofthe
end
aour
of
would
age.
the
food
cannotseason.
A more
be very
expect
supplement
complex
interesting
to see toifwatch,
study,
procedure the
might
so by all means
we hypothesis
might decide we
reverse causation explanation is at least plausible. anIndeed,
it might be quiteboth
plausible mechanisms the number ofcould
matings be
male gets affects
We have aimed to produce a book that operating
can be simultaneously.
read from cover Aofpilot study isinvolve
thattechniques.
his hormone
the male’s
study rather
system
exploration
classifying
In Chapter
levels. The
than Aonly
changegood inall
them for ainto
individuals
4hormone
we
carrying used
pilot
while.
themight
levels
out exampleathen
the
However,
numberdata
of theof
affect
study you
use categories,
age
wayneed
effects
the
on
what you see
of GM
that theto
three-months-old feed such
collect
on egg
inas this exploratory
young,
to
shelleither
mice mid-confirm
thickness
we
pilot study to
will use ormice
refu
In Chapter
dle-, and
tail develops 4
toold-aged,
during we
the season
think about used
ate
prior clearthe
to aallocating
invariation
such wayand example
questions.
his them
thatreplication Once
tail length of
to the
you
treatment
comes
(see effects
have
Boxto4.1).depend your
groups, of
Let’s return GM
question
andtothen feedin on
mind,
ensuringto
this example egg
you shell
should thickne
try to fo
conducted study
thebefore
will
the maximize
main the design your study and collect your data. This ap
to cover, and so have tried to keep unnecessary details
Let’s consider anotherout example of reverse body of
onthat
data
between
differ
number
causation. of
to think
that
benefits
in
half
tail length of
collection
age,
matings
There that
ofabout
now the
think
in your
and
but we
he gets.
isabout
a
individuals
matingsmain
Inwill
potheses
correlation
variation
that wedata-
keep
that case
in each
statistical that
and amight
we would note
between observe
replication
category
power. For end
theofup
answer
expect, but it would be nothing to do with
the
the
(see
sakein theage.
the
of twoA
question.
relationship
Box
this more
4.1).
food
example You complex
Let’s
we then need
arereturn
treatments. going procedure
These
totoas-make
to this example might
predictions
involve classifyingsumeother)that allthe things
individuals youbewould
feed containing into the hypotheses,
aexpect
GMnumber
grain toreally
observe
of and
age
does predictions
ifhave
your in hypothesis
categories,
an effect on9, issowere
such
the muchastrue
thick- more
young,and/or likely
thint
mid-
the number of storks nesting on the chimneys order to
therefine of(and
now several
a Dutch believefarmhouse
wethink
research about designs
statistical will
and the discussed
number
power. in much
For the more
sake detail Chapter
seem of this example we are going to a
we
of the main text. However, in areas where we feel that more
reason that (females preferring long tails). Such a pathway might
collection phase and you helpwould you notan
aims and dle-, toand
data-collection
unlikely you and ness
old-aged,
you of egg
may know prior
of shells,
no to
example causing
allocating
of such a expect
average
thing themtoreduction
collected
happening. seeHowever,
toif the with hypothesis
of
treatment
you nomm.
0.4 Orwere
specific to put
groups, true.
it inYou
plan and for then
terms
then need This
its use. to decid
ensuring sh
of children in the family living in the house. This
techniques.
sounds
avoidsume
A good
can be sure bizarre
pilot
that the that
pitfalls.
that a the until feed
statistician
Devil’s Advocate data you might
will findyou think
containing use,
suchneed that
the
arguments thelarger
population
to collect GM grain
mean
to eitherunless
highly compelling really
of GM does
eggs is have
different an
from effect
confirm or refute your predictions. Only then the on the thicc
details or more examples will lead to a fuller how youendare going theto usefood data, the likelihood of y
familiesunderstanding
tend to live in larger housesof a with studymorethat
will half
chimneys
ness
maximize ofthe the
of individuals
available
egg
population shells,
mean design
instandard
asofcausing
nest each
yoursites
category
an
eggs.
study for Ofthe
average
and collectreduction
course, our upinvestigator
your
in
data. of
two 0.4
Thisdoes mm.
not
approach
treatments.
know Or to putspecific
of this,
having itThese
in termque
(and several other) designs willusualbe anddiscussed
right sort of data islikely
low.
to in much more detail in Chapter 9, so we
benefits of your main data-and is engaged in their
hypotheses, quest to determine
predictions is much whether
more GM feed affects
to eggs
produce shell conclusions
concept we have included supplementary storks. Large families
boxes. Where leadthere
to moreisopportunities for
that
collection phase and help
nesting storks;
a statistician storks
might
you To do this they have measured
don’t
use, the lead population mean of GM is firm
different from that
thickness. Given five GM eggs and five standard eggs. Un-
collected with no specific planthe for its importance
use. This should of a befocused
no surprise; research
if you don
large families! avoid pitfalls. population mean
surprisingly, theof standard
average thicknesses eggs. of Ofthe course,
two groups our are investigator
not identical (godoes back not know th
(to our mind) really interesting material that willcausation
not beisstrictly not a universal problemand is toengaged
Box 4.1 if this
how you are going to
does surprise
instudies.
their usual you)
spending
quest
use data, the even a
likelihoodfew
and the investigator carries out an appropri-
ofminutes
you collecting making unfocused
a suitable amoun
Reverse of correlational right sort of Sometimes
data is low. to determine whether GM feed affects sh
06-RuxtonandColegrave-Chap06.indd 94 ate statistical test to examine whether wasted time that
the difference couldthe
between betwobettersamples spent
25/02/16 2:02collectin
PM

relevant to all readers, we have not included

reverse that material
causation in the03-RuxtonandColegrave-Chap03.indd
can be discounted as a plausible thickness.
37 explanation. To do this
that they have collected
In they
Given ourthe have
correlational
is bigimportancemeasured
enough to of a focused
suggest
five GM
that GM
eggs and
research
feed really
We disagree. You should be striving to address th
five 09/04/16
question,
does have
standard
you 10:42 AM eggs. U
anmight be thinkin
See
surprisingly, Section the 2.1 averageof
spending the even a
thicknesses few minutes making
of the two unfocused observations of the chimps is
study of male tail streamer length and mating success described
adverse effect. Inearlier,
statistical itjargon,
would thebe investigator wantsgroups
to know whetherare notthese identical (go ba
book itself, but lodged it on an associated website (www.oxford­
extremely difficult to imagine how the number of to
supplementary
Boxgroups
matings 4.1a ifarethis
male
material
wasted
significantly
gets does time
laterdifferent
surprise
in
thatfrom
a
you
could
seasonyou)one can,
be better
another,
and
notsuggesting
thespentas many
investigator thatquestions
collecting thedata
populations
carries
as you
to answer
out scientific
can. que
an approp
Da
See for a more
Section in-depth
2.1 ofthat thethey represent comparison
We disagree. have differentYou should are
means. not the famous
beIfstriving because
to address
investigator finds the theyinteresting
a most
significant solved
dif- more
scientific probqu
textbooks.co.uk/orc/ruxton4e/). We point to this
could affect his tailsupplemen­
length when we catch supplementary himbetween
at theate beginning
statistical
exploratory
ference, then
material of test andto
the
they
you
examine
season.
conclude
can, not as thatmany
whether
However,GM feed
they
the difference
affectsasshell
questions
solved youthickness;
important can. Darwin,
between
if they
problems.
the two sampl
find no and Watson an
Einstein,So take the tim
imagine that we had instead caught for a more in-depththat
and measured the males at the
confirmatory they
comparison have
studies.
significant collected
are end
Go
difference, to: they
not famous is big
of the because
conclude enough that to
there suggest
is no evidence that
season.they solved more problems than anyone else, but b2
that GM
GM feed
feed really
affects does have
tary material at appropriate points throughout the book.
06-RuxtonandColegrave-Chap06.indd 94
between exploratory and shell thickness.
www.oxfordtextbooks.co.uk/
adverse
questions are. We are not promising that you’ll
In that case it might be quite plausible that the number
confirmatory studies. Go to:
ofeffect.
matings In
they astatistical
solvedgets
male jargon,
important affects the investigator
problems. So take thewants time to to thinkknow aboutwhether
what the the inte
orc/ruxton4e/. Let us imagine that the investigator
questions are. We follow
are
gets
not one this
promising advice,
a statistically
thatbut you’ll the
significant chances
become
result, andwill increase.
as famous as Einstein
his hormone levels. The change in hormone levelsgroups
www.oxfordtextbooks.co.uk/ might are
then
comes significantly
to affect
the conclusion thedifferent
way
that GM that from
feed the has an effect another, on shell suggesting
thickness. However, that the populatio
orc/ruxton4e/. follow this advice, but the chances will increase.
that
male’s tail develops during the season in such a way thatthey
the represent
hisresearcher
tail length doeshave different
not stop
comes tothere. means.
Instead,
depend beingIfvery
thethorough,
investigator finds
they come backa significant d
the next day and repeat the experiment inAs a general
the same ruleway asyour experiment shou
Chapter outlines on the number of matings that he gets. In that caseference,
weday thenobserve
would they conclude
before. The shellAs
the thatrule
relationship
a general
measurements
GMyour
that
exactly
feed affects
experiment shell
should
they did
thickness;
be
clear hypothesis about your research sy
they collect in this second
the
designediftothey
experiment
find
test at lea
between tail length and matings that we expect, significant
but will difference,
it would
differ bethose
from they
clear
nothing
of conclude
thehypothesis
to
day do with
before, that there
about
because your is no evidence
of theresearch
effects ofsystem. Athat GM
period
random variation offeed affec
unfocused
vation can help vation
you identify can help you
interesting identify
research interesting
questions, but a resear
study bas
shell thickness.
(there are likely to be many factors that affect the thickness of an egg shell). This in
Most chapters begin with an outline of the
thereason
mainthat points covered
we believe (femalesby that chapter.
preferring long tails). These
Such a pathway might seem
turn will affecttirely on unfocused
the mean tirely
size of the on unfocused
observation
two groups. will rarely
Again, observation
answer
the investigatorsuch will rarely
doesquestions
their answ
definitive
unlikely to you and you may know of no example of suchLet aus imagine
thing that the
happening. investigator
However, you gets a statistically significant result, a
outlines should help you to prepare for can
what is ahead. The outlines will also
be sure that the Devil’s Advocate will find such
provide
comes is to
notthe
arguments
an
statistics and finds a significant difference between the groups. However, even this
conclusion
highly
enough that
compelling
for our intrepid GM feed
unless
researcher, andhas
they an
are effect
back the on shellday
following thickness.
do- Howev
2.1.1 An example of moving from a question to hypotheses,
easy way for you to dip in and out of the book during subsequent readings.
the researcher does
ing exactly the not
same
and then
affects the means.
stop there.
experiment
totoday
However,
2.1.1 Instead,
again. Again,
an experimental
An being
the exact
example veryofthorough,
measures
design
(by chance) the five
movingthey
differ, and this
GM shells all happen to be
come
from ba
a que
the next day and repeat the experiment
and theninto
exactly the same way as
an experimental they did t
design
slightly thicker than the average of their population due to random variation, while
day before. The
The shellslastmeasurements
shell section might havethat seemedtheya little abstract,
collect so letsecond
in this us try anexperime
example. I
the standard are all thinner than would be expected, having negative devia-
tions.from
will differ
thatthat
This means
those
youthere
of the isdayveryThe
have collectedlittlealast
before, section
number
difference of
between
because might
carcasses
of the have
of
themeans
effectsseemed
black
of grouse
the
oftwo
(aagame
randomlittlevariati
abstr
bird) as

Chapter summaries a and

sample groups,
(there difference
are likelybetween
study
to be many
looking
when at their mortality
that
the investigator
factors that
does therisk
youaffect
have collected due to
statistics
the thickness
heflying
a number into
of an of
finds no
egg
the groups, so today he finds no evidence that GM feed has
man-made structures s
carcasses
significant o
shell). This
turn will affect the mean size ofathe
any effect.
study
twolooking
groups.atAgain,
their mortality risk duedoes
the investigator to flyin
th
Every chapter finishes with a summary of the most important points covered in
statisticsthat
This
andis a finds
strangeaset of circumstances,
significant so let’s get
difference this clear. the
between The investigator has
groups. However,
done nothing wrong in this third experiment, the hens were all randomly chosen,
even t
03-RuxtonandColegrave-Chap03.indd 37 is not enough for our intrepid researcher, and they are back the 09/04/16
following day
10:42 AMd
chapter. By reading through this list you should be able to reassure yourself that you
and the experiment was carried out correctly. The fact that the GM shells are all
ing exactly the same experiment again. Again, the exact measures differ, and t
have got everything out of the chapter that you can, or go back and read sections
affects the means. However, today (by chance) the five GM shells all happen to
02-RuxtonandColegrave-Chap02.indd 14
slightly thicker than the average of their population due to random variation, wh
again that are not yet clear in your mind.
the standard shells are all thinner than would be expected, having negative dev
tions. This means that there is very little difference between the means of the tw
02-RuxtonandColegrave-Chap02.indd 14
sample groups, and when the investigator does the statistics he finds no significa
difference between the groups, so today he finds no evidence that GM feed h
06-RuxtonandColegrave-Chap06.indd 83

any effect.
This is a strange set of circumstances, so let’s get this clear. The investigator h
done nothing wrong in this third experiment, the hens were all randomly chose
and the experiment was carried out correctly. The fact that the GM shells are
x Preface

Flow chart
The exact process of designing an experiment will vary considerably between studies.
Nevertheless, there are key stages in the design process that will apply to most, if not
all, studies. The flow chart at the end of this book is intended to summarize and guide
you through the main stages of designing an experiment. We have indicated at each
point in the chart the sections of the book that are most relevant.
Acknowledgements from
the fourth edition

In the 15 years we have been involved with this book, we have had explicit feedback
from countless students and colleagues; and have had our thinking on experimental
design sharpened by countless others. We don’t attempt even a partial list; but if you
spot your influence anywhere in this work be assured of our profound thanks. Even
if you can’t spot your direct influence be aware that we are both really grateful for
everyone who has forced us to think harder about experimental design and how to
explain our ideas. We are grateful for the support we have had from editorial staff at
OUP (most recently Jessica White) who have always been supportive when required
without ever feeling heavy-handed; the production staff (most recently Tomas Furby)
have always been a model of quiet competence. Julian Thomas is a copy editor who
combines thoroughness with a collegiate, open, and interactive approach. We have
both been privileged to work at institutions that have fostered the academic freedom
that has allowed us to devote so much time to this endeavour. We have both been
blessed by friends and family that have been tolerant of the demands of this book but
always there to provide compelling reasons not to let it swallow our lives entirely.
Contents

1 Why you should care about design1

1.1 Why experiments need to be designed1

1.2 The costs of poor design3

1.2.1 Time and money3
1.2.2 Ethical issues3

1.3 The relationship between experimental design

and statistics4

1.4 Why good experimental design is particularly

important to life scientists5
1.4.1 Random variation5
1.4.2 Confounding factors6
1.4.3 Simpson’s paradox7

1.5 Subjects, experimental units, samples, and terminology9

Summary11

2 Starting with a well-defined hypothesis13

2.1 Why your study should be focused: questions,

hypotheses, and predictions13
2.1.1 An example of moving from a question to hypotheses,
and then to an experimental design14
2.1.2 An example of multiple hypotheses16
2.1.3 Where do (good) ideas come from in the first place?18

2.2 Producing the strongest evidence with which

to challenge a hypothesis20
2.2.1 Indirect measures20
2.2.2 Considering all possible outcomes of an experiment21
2.2.3 Satisfying sceptics22

2.3 Controls23
2.3.1 Different types of control23
xiv Contents

2.3.2 Making sure the control is as effective as possible26

2.3.3 The ethics of controlling27
2.3.4 Situations where a control is not required27

2.4 The importance of a pilot study and preliminary data28

2.4.1 Making sure that you are asking a sensible question28
2.4.2 Making sure that your techniques work29
Summary31

3 Selecting the broad design of your study32

3.1 Experimental manipulation versus natural variation32

3.1.1 An example hypothesis that could be tackled by
either manipulation or correlation32
3.1.2 Arguments for doing a correlational study33
3.1.3 Arguments for doing a manipulative study35
3.1.4 Situations where manipulation is impossible38

3.2 Deciding whether to work in the field or the laboratory41

3.3 In vivo versus in vitro studies43

3.4 There is no perfect study43

Summary44

4 Between-individual variation, replication,

and sampling46

4.1 Between-individual variation46

4.2 Replication47

4.3 Selecting a sample57

4.3.1 Simple random sampling57
4.3.2 Remember, you want a representative sample58
4.3.3 Stratified sampling59
4.3.4 Cluster sampling61
4.3.5 Convenience sampling63
4.3.6 Self-selection64
Summary65
 Contents xv

5 Pseudoreplication67

5.1 Explaining what independence and pseudoreplication are67

5.2 Common sources of pseudoreplication70

5.2.1 A shared enclosure70
5.2.2 The common environment71
5.2.3 Relatedness72
5.2.4 A pseudoreplicated stimulus72
5.2.5 Individuals are part of the environment too72
5.2.6 Pseudoreplication of measurements through time73
5.2.7 Species comparisons and pseudoreplication73

5.3 Dealing with non-independence74

5.3.1 Independence of replicates is a biological issue74

5.4 Accepting that sometimes replication is impractical76

5.5 Pseudoreplication, third variables, and

confounding variables77

5.6 Cohort effects, confounding variables, and

cross-sectional studies78
Summary79

6 Sample size, power, and efficient design81

6.1 Selecting the appropriate number of replicates81

6.1.1 Educated guesswork82
6.1.2 Formal power analysis82

6.2 Factors affecting the power of an experiment82

6.3 Working out the power of a planned study84

6.3.1 Determining the likely effect size85
6.3.2 How much variation?86
6.3.3 Experimental design87
6.3.4 How many replicates?87
6.3.5 Imaginary experiments88
xvi Contents

6.4 Improving the power of a study91

6.4.1 Reducing random variation92
6.4.2 Dealing with variation through design94
6.4.3 Increasing the effect size95
6.4.4 Consider all other ways of increasing power before
increasing sample sizes97

6.5 Comparing the power of different planned studies98

Summary100

7 The simplest type of experimental design:

completely randomized, single-factor101

7.1 Completely randomized single-factor designs102

7.2 Randomization102
7.2.1 Randomizing study subjects102
7.2.2 Randomizing other aspects of your study103
7.2.3 Haphazard allocation104
7.2.4 Balanced and unbalanced allocation105

7.3 Factors with more than one level106

7.4 Advantages and disadvantages of complete randomization107

Summary108

8 Experiments with several factors

(factorial designs)109

8.1 Randomized designs with more than one factor109

8.2 Interactions110

8.3 Confusing levels and factors117

8.4 Split-plot designs (sometimes called split-unit designs)119

8.5 Latin square designs121

8.6 Thinking about the statistics123

Summary125
 Contents xvii

9 Beyond complete randomization: blocking

and covariates127

9.1 The concept of blocking on a particular variable128

9.2 Blocking on individual characters, space, and time131

9.3 The advantages and disadvantages of blocking132

9.4 Paired designs133

9.5 How to select blocks133

9.6 Covariates134

9.7 Interactions between covariates and factors138

Summary140

10 Within-subject designs141

10.1 What do we mean by a within-subject design?141

10.2 The advantages of a within-subject design142

10.3 The disadvantages of a within-subject design142

10.3.1 Period effects142
10.3.2 Carry-over effects144

10.4 Isn’t repeatedly measuring the same individual

pseudoreplication?146

10.5 With multiple treatments, within-subject experiments

can take a long time147

10.6 Which sequences should you use?147

10.7 Within-subject designs and randomized

block designs148

10.8 It is possible to design experiments that mix

within-subject and between-subject effects149
Summary150
xviii Contents

11 Taking measurements152
11.1 Calibration153

11.2 Inaccuracy and imprecision154

11.2.1 Subsampling: more woods or more trees?157

11.3 Sensitivity and specificity159

11.4 Intra-observer variability162

11.4.1 Describing the problem162
11.4.2 Tackling the problem163
11.4.3 Repeatability163
11.4.4 Remember, you can be consistent but still consistently wrong164

11.5 Inter-observer variability166

11.5.1 Describing the problem166
11.5.2 Tackling the problem166

11.6 Deciding how to measure167

11.6.1 Defining categories167
11.6.2 How fine-scale to make measurements of continuous variables168
11.6.3 Observer bias and blinding168
11.6.4 Allocation concealment171
11.6.5 Floor and ceiling effects172
11.6.6 Observer effects173

11.7 Pitfalls to avoid when recording data174

11.7.1 Don’t try to record too much information at once175
11.7.2 Beware of shorthand codes175
11.7.3 Keep more than one copy of your data175
11.7.4 Write out your experimental protocol formally and
in detail, and keep a detailed field journal or lab book176
11.7.5 Don’t overwork176
11.7.6 Take care to check computers and automated data collection176
Summary177

Sample answers to self-test questions 179

Flow chart on experimental design 191
Bibliography 196
Index 201
Why you should care
about design 1
• We begin this chapter by explaining the benefits of learning how to design
good scientific studies (Section 1.1), and the potential costs of failing to do so
(Section 1.2).
• We then address the relationship between design of a study and statistical
analysis of the data collected in the study (Section 1.3).
• Much of good experimental design comes down to dealing with the challenges of
random variation and confounding factors, so we next introduce these concepts
(Section 1.4).
• In any study, we generally measure only a sample from the wider population
that we are interested in. We discuss how life scientists refer to the ‘things’ in
a sample as an introduction to how we deal with the specialist terminology of
experimental design throughout the book (Section 1.5).

1.1 Why experiments need to be designed

When some life scientists see the phrase ‘experimental design’, it can either ease them
gently into a deep sleep or cause them to run away screaming. For many, experimen-
tal design conjures up unhappy memories of mathematics or statistics lessons, and is
generally thought of as something difficult that should be left to statisticians. Wrong
on both counts! Designing simple but effective experiments doesn’t require difficult
maths. Instead, experimental design is more about common sense, biological insight,
and careful planning. Having said that, it does require a certain type of common sense,
and there are some basic rules. In this book, we hope to steer you relatively painlessly
towards thinking more effectively about designing experiments.
So why are many life scientists so averse to thinking about design? Part of the reason
is probably that it is easy to think that time spent designing experiments would be
better spent actually doing experiments. After all, the argument goes, we are biologists
2 1 Why you should care about design
Independent data points
come from unconnected
individuals. If the measured
value from one individual
gives no clue as to which
of the possible values the
measurement of another
individual will produce, then
the two measurements are
independent.
Q 1.1 If we wanted to
measure the prevalence of
both left-handedness and
religious practices among prison
inmates, what population would
we sample from?
Q 1.2 If we find that two
people in our sample have
been sharing a prison cell for the
last 12 months, will data from
them be independent?
so let’s concentrate on the biology and leave the statisticians to worry about the design
and analysis. This attitude has given rise to two myths that you can hear from the
greenest student or the dustiest professor.
Myth 1: It does not matter how you collect your data, there will always be a
statistical ‘fix’ that will allow you to analyse it.
It would be wonderful if this was true, but it is not. There are a large number of statisti-
cal tests out there, and this can lead to the false impression that there must be one
for every situation. However, all statistical tests make assumptions about your data
that must be met before the test can be meaningfully applied. Some of these assump-
tions are very specific to the particular test. If you cannot meet these, there may be a
substitute test that assumes different characteristics of your data. But you may find
that this alternative test only allows you to use your data to answer a different scien-
tific question to the one that you originally asked. This alternative question will almost
certainly be less interesting than your original (otherwise why weren’t you asking it in
the first place?). Further, there are some basic assumptions that apply to many of the
most commonly used statistical tests, and you ignore these at your peril. For instance,
statistical tests generally assume that your data consist of what statisticians refer to
as independent data points (more on this in Chapter 5). If your data don’t meet this
criterion then there is much less that statistics can do to help you. Careful design will
allow you to avoid this fate. It’s also generally true that well-designed experiments re-
quire simpler statistical methods to analyse them than less-well-designed experiments.
So time spent carefully designing your experiment might save you a lot of time later in
the study trying to master more complex statistical techniques than you really needed.
The group of experimental subjects that we use in our experiment (called the
sample) needs to be representative of the wider set of individuals in which we are
interested (called the population). One key way to achieve this is to ensure that each
individual selected is not linked in some way to another individual in the sample, i.e.
they are independent. For example, if we were surveying human food preferences,
then gathering food-preference data from five members of the same family would not
produce five independent data points. We would expect that members of the same
family are more likely to share food preferences than two unconnected individuals,
since family members often have a long history of eating together. Similarly, gathering
data from the same person on five separate occasions certainly does not provide five
independent data points, since a person’s preference on one occasion is likely to be a
good guide to their preferences a little while later.
Myth 2: If you collect lots of data something interesting will come out, and you’ll
be able to detect even very subtle effects.
It is always reassuring to have a notebook full of data. If nothing else, it will convince
your supervisor that you have been working hard. However, quantity of data is really no
substitute for quality. A small quantity of carefully collected data, which can be easily
analysed with powerful statistics, has a good chance of allowing you to detect interest-
ing biological effects. In contrast, no matter how much data you have collected, if it is
 1.2 The costs of poor design 3

of poor quality, it will be unlikely to shed much light on anything. More painfully, it will
probably have taken far longer and more resources to collect than a smaller sample of
good data.

Designing effective experiments needs thinking about biology more than it

does mathematical calculations. Careful experimental design at the outset
can save a lot of sweat and tears when it comes to analysing your data.

1.2 The costs of poor design

1.2.1 Time and money

Any experiment that is not designed in an effective fashion will at best provide limited
returns on the effort and resources invested, and at worst will provide no returns at all.
It is obvious that if you are unable to find a way to analyse your data, or the data that
you have collected does not enable you to answer your question, then you have wasted
your time and also any materials. However, even if you don’t make mistakes of this mag-
nitude, there are other ways that a poorly designed experiment might be less efficient.
It is a common mistake to assume that an experiment should be as big as possible; but
if you collect more data than you actually need to address your question effectively,
you waste time and money. At the other extreme, if your experiment requires the use of
expensive consumables, or is extremely time consuming, there is a temptation to make
it as small as possible. However, if your experiment is so small that it has little chance
at all of allowing you to detect the effects that you are interested in, you have saved
neither time nor money, and you will probably have to repeat the experiment, this time
doing it properly. Problems like these can be avoided with a bit of careful thought, and
in later chapters (particularly Chapter 6) we will discuss ways of doing so.
Similarly, it is not uncommon for people to collect as many different measurements
on their samples as possible without really thinking about why they are doing so. At
best this may mean you spend a great deal of time collecting data that you have no
use for, and at worst may mean that you don’t collect the information that is critical
for answering your question, or that you do not give sufficient time or concentration to
collecting the really important information. Don’t be over-ambitious: better you get a
clear answer to one question than a guess at the answers to three questions.

While it is always tempting to jump into an experiment as quickly as possible,

time spent planning and designing an experiment at the outset will save time
and money (not to mention possible embarrassment) in the long run.

1.2.2 Ethical issues

If the only issue at stake in ill-conceived experiments was wasted effort and resourc-
es, that would be bad enough. However, life science experiments have the additional
Another random document with
no related content on Scribd:
The Project Gutenberg eBook of At home in the
Smokies
 This ebook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this ebook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

Title: At home in the Smokies

a history handbook for Great Smoky Mountains
National Park, North Carolina and Tennessee

Author: Wilma Dykeman

James Stokely

Creator: United States. National Park Service

Release date: December 10, 2023 [eBook #72365]

Language: English

Original publication: Washington DC: National Park Service,

1984

Credits: Carla Foust, Steve Mattern, Thiers Halliwell and the

Online Distributed Proofreading Team at
https://www.pgdp.net

*** START OF THE PROJECT GUTENBERG EBOOK AT HOME IN

THE SMOKIES ***
At Home In the Smokies
Handbook 125
 At Home
In the Smokies
A History Handbook for
Great Smoky Mountains National Park
North Carolina and Tennessee

Produced by the
Division of Publications
National Park Service

U.S. Department of the Interior

Washington, D.C. 1984
Using This Handbook
This theme handbook, published in this new edition on the 50th
anniversary of Great Smoky Mountains National Park, tells the story
of the people who settled and lived in the mountains along the
Tennessee and North Carolina border. Part 1 gives a brief
introduction to the park and its historical sites. In Part 2, Wilma
Dykeman and Jim Stokely present the history of the region from the
early Cherokee days to the establishment of the park in 1934 and
the renewed interest in the past in the 1970s; this text was first
published by the National Park Service in 1978. Part 3 gives a brief
description of the major historical buildings you can see in the park.
For general information about the park and its wildlife, see Handbook
112.
National Park Handbooks, compact introductions to the great natural
and historic places administered by the National Park Service, are
published to support the National Park Service’s management
programs at the parks and to promote understanding and enjoyment
of the parks. This is Handbook 125.
Library of Congress Cataloging in Publication Data
Main entry under title: At home in the Smokies. (National park
handbook; 125)
Rev. ed. of: Highland homeland/Wilma Dykeman and Jim Stokely.
1978. Includes index.
Supt. of Docs, no.: I 29.9/5:125
1. Great Smoky Mountains (N.C. and Tenn.)—Social life and
customs. 2. Great Smoky Mountains (N.C. and Tenn.)—History. 3.
Great Smoky Mountains National Park (N.C. and Tenn.)—Guide-
books. 4. Cherokee Indians—History.
I. Dykeman, Wilma. Highland homeland. II. United States. National
Park Service. Division of Publications. III. Series: Handbook (United
States. National Park Service. Division of Publications); 125.
F443.G7A8 1984 976.8’89 84-600108
ISBN 0-912627-22-0
Part 1 Recapturing the Past 4
Smoky Mountain Heritage 7
Part 2 Highland Homeland 12
By Wilma Dykeman and Jim Stokely
Homecoming 17
Rail Fences 30
Land of the Cherokees 35
The Pioneers Arrive 49
Rifle Making 60
A Band of Cherokees Holds On 63
From Pioneer to Mountaineer 73
Spinning and Weaving 94
The Sawmills Move In 97
Birth of a Park 107
The Past Becomes Present 121
Handicrafts 132
Coming Home 137
Part 3 Guide and Adviser 146
Traveling in the Smokies 148
Oconaluftee 150
Cades Cove 152
Other Historic Sites in the Park 154
Related Nearby Sites 156
Armchair Explorations 157
Index 158
 Part 1

Recapturing the Past

Joseph S. Hall
Aden Carver of Oconaluftee was a carpenter, stone
mason, millwright, deacon, and preacher. He was
more versatile than some men but representative of
many who worked hard and enjoyed their lives in the
Smokies.
 Smoky Mountain Heritage
Seemingly endless ridges, forests, mountain streams, waterfalls, and
wildlife attract hundreds of thousands of travelers each year to Great
Smoky Mountains National Park on the Tennessee-North Carolina
border. Many are drawn by a long procession of wildflowers and
shrubs bursting into bloom in the spring and by the colorful foliage of
the hardwoods in the fall. Thousands hike the park’s many trails,
which range from short spurs to the 110 kilometers (70 miles) of the
Appalachian Trail that runs through the park. Also attracting wide
interest are the park’s historical sites and the lifeways of the
mountain people. They are pleasant surprises in the midst of all of
nature’s richness. They are physical ties with our ancestors, many of
whom traveled from their homelands across the sea to build new
homes in the relatively unexplored continent of North America.
The National Park Service has preserved some of the historic
structures in Great Smoky Mountains National Park so that we, and
future generations, can better understand how our forefathers lived.
By walking through and closely examining their finely crafted—and
crudely crafted—log houses, barns, and other farm buildings we gain
a new respect for their diligence and perseverance. The hours spent
hewing massive beams, preserving foods for winter use, and making
clothes from scratch are nearly incomprehensible in our age of
machines and computers. The mountainous terrain demanded hard
work, and the isolation fostered a zealous independence. The land
truly molded a resourcefulness and hardiness in the Smokies
character.
The story of these mountain people and communities is told in Part 2
of this handbook by Wilma Dykeman and Jim Stokely, who can look
out on the expanse of the Great Smokies from their family home in
Newport, Tennessee. Their engaging story of the Smokies is
illustrated with historic photographs that largely come from the park’s
files. Although the identities of many of the photographers are
unknown (see page 160), we are no less indebted to them. They
have helped to preserve the history and folkways of the Great
Smokies people, who played a part in molding and defining our
national character.