4.

2 Experiments
Observational Studies vs. Experiments
An observational study observes individuals and
records behavior, but DOES NOT attempt to
impose a treatment in order to manipulate a
response.
Ex: Watching the behavior of animals in the wild

An experiment deliberately imposes some
treatment on individuals to measure their
responses.
Used to measure the cause and effect relationship
between two variables


Lurking Variables and Confounding
A lurking variable is variable that is NOT among
the explanatory and response variables in a study,
but may influence the response.
Explanatory variables (typically X) attempts to explain
that changes in the response variable (typically Y).

Confounding occurs when two variables are
associated in such a way that their effects on a
response variable cannot be distinguished from each
other.
The Language of Experiments
Treatments- A specific condition applied to
the individuals in an experiment

Expermental units- Smallest collection of
individuals to which treatments are applied

Subjects- Term used when Exp. Units are
humans.
Why Experiments over Observational
Studies?
Experiments can give good evidence for
causation.

However, it is important to discuss what is a
good experiment and what is a bad experiment.

Factors and Levels
Sometimes, experiments study the joint effect of
several factors

The factors of an experiment make up all of the
explanatory variables
( X = Explanatory variable= Factor)

Each treatment is formed by combining a
specific value (called a level) of each of the
factors. (Look at Example 4)
How to Experiment Badly
Making the experiments too simple
Example #5- Does caffeine affect pulse rate?

This simplified experiment doesnt analyze
students who drink caffeine and those who
dont.

This is a BAD experiment due to the amount of
confounding in the experiment.
How to Experiment Well
A good experiment compares two or more
treatments and their effects on the experimental
units

Random assignment means each exp. unit is
assigned to treatments at random
Using Table D
Random # Generator
**Names in a hat (equal size slips & well
mixed)** --- BEST TO USE ON AP EXAM!
E
x
p
e
r
i
m
e
n
t
s

The Randomized Comparative Experiment
Definition:
In a completely randomized design, the treatments are
assigned to all the experimental units completely by chance.
Some experiments may include a control group that receives
an inactive treatment or an existing treatment.
Experimental
Units
Random
Assignment
Group
1
Group
2
Treatment
1
Treatment
2
Compare
Results
Ex: SAT Prep- In class or Online?
Due to local budget cuts, a local high school is
wants to determine if students can take the SAT
Prep course online versus in the classroom
without affecting SAT scores. 50 students have
agreed to participate to take the SAT Prep class
either online or with a teacher in the classroom.

Design a completely randomized experiment to
compare student SAT scores given either online or
classroom instruction.
Principles of Experimental Design
1. Control for lurking variables that might affect
the response. Use a comparative design and
ensure that the only difference between the
groups is the treatment administered.

2. Random assignment: Use impersonal
chance to assign experimental units to
treatments. This eliminates hidden bias and
lurking variables while creating a roughly
equivalent group.
Prinicples of Experimental Design
3. Replication: Using enough subjects/units in
each group so that any differences in the effects of
the treatments can be distinguished from the
chance differences between the groups.


**These are the founding principles in
Experimental Design! You must know
these!**

Placebos
A placebo is a dummy pill/drug that looks and
tastes like the treatment pill/drug, but contains
NO active ingredients

A placebo effect occurs when a subject has a
measurable response to a dummy treatment.

We often use blind experiments to test whether
or not a placebo has an effect on a subject.
Single blind v Double blind
Single blind experiments occur when either
the subject OR the people interacting with them
are unaware of the treatment being received.

Double blind experiments occur when BOTH
the subject AND the people interacting with
them are unaware of the treatment being
received.
Why use a blinded experiment?
To test a subjects response to a placebo
We need to measure people who THINK they are
receiving the drug versus those who are
ACTUALLY receiving the drug.
To eliminate any bias from results
This could be in selection of the subject
Also experimenter bias The one conducting the
experiment could possibly tamper with important
experimental data

Statistical significance
Occurs when an observed effect is so large that it
would rarely happen by chance.

If something is statistically significant from a
well designed experiment, it DOES imply
causation.
E
x
p
e
r
i
m
e
n
t
s

Blocking Design
Definition:
A block is a group of experimental units that are known before
the experiment to be similar in some what that is expected to
affect the response of the treatments.

In a randomized block design, the random assignment of
experimental units to treatments is carried out separately
within each block.
Experimental
Units
Block
1
Block
2
Random
Assignment
Random
Assignment
Treatment 2
Treatment 1
Treatment 2
Treatment 1
Compare
Results
Compare
Results
Blocking
A specific control that can be placed on an
experiment.

We can now draw separate conclusions about
each block or even make comparisons between
blocks (Block 1 v Block 2)

Moral of Blocking (and even Exp. Design)
Control what you can
Block what you cant control
Randomize to create comparable groups


Matched pairs design
A type of randomized
block design.
Used to compare two
units or subjects by
matching up similar
ones in pairs.
Ex: Pre-tests versus
Post-tests
Ex: Left hand strength
versus Right hand
strength


Blocking v. Matched Pairs
Blocking is used in order to eliminate the bias
from a variable that is likely to affect the
response.
Does Gender matter when comparing SAT scores
of students who took the SAT prep course online
v. in the classroom?
Matched pairs is used to compare two responses
on the same unit/individual
Do you have more strength in your left hand v.
your right hand?