Experimental

Research Designs
Experimental Design
 Advantages
 Best establishes cause-and-effect
relationships
 Disadvantages
 Artificiality of experiments
 Feasibility
 Unethical
Causality
 Temporal precedence
 Covariation between IV and DV
 Eliminate alternative explanations
Types of Experimental Designs
 Simple True Experimental
 Complex True Experimental
 Quasi-Experimental
Types of Experimental Designs
 Simple True Experimental
 Complex True Experimental
 Quasi-Experimental
Simple True Experimental
 Characteristics
 Types
 Variations
Characteristics of True Designs
 Manipulation (treatment)
 Randomization
 Control group

Characteristics of simple true designs

 One IV with 2 levels (T, C)
 One DV
Types
 Randomized posttest control group design
 Randomized pretest-posttest control
group design
Randomized posttest control group design

R T Post
R C Post
Randomized pretest-posttest control group
design

R Pre T Post
R Pre C Post
Advantages & Disadvantages
 Advantages of pretest design
 Equivalency of groups
 Can measure extent of change
 Determine inclusion
 Assess reasons for and effects of mortality
 Disadvantages of pretest design
 Time-consuming
 Sensitization to pre-test
Solomon four-group design
R Pre T Post
R Pre C Post
R T Post
R C Post
Variations
 Independent groups (between groups)
 Repeated measures (within groups)
Repeated Measures Design
 Advantages:
 Fewer subjects needed (less costly)
 Sensitive to finding statistical differences
 Disadvantages:
 Order effect (practice, fatigue, carry-over)
Dealing with Order Effects
 Counterbalancing
 n!
 Latin squares
Latin Squares
1 2 3 4
Row 1 A B D C
(60) (0) (120) (180)
Row 2 B C A D
(0) (180) (60) (120)
Row 3 C D B A
(180) (120) (0) (60)
Row 4 D A C B
(120) (60) (180) (0)
Dealing with Order Effects
 Counterbalancing
 n!
 Latin
squares
 Randomized blocks
 Time interval between treatments
Variations
 Independent groups (between) vs.
repeated measures (within) designs

Consider external validity when

deciding which design to use.
Types of Experimental Designs
 Simple True Experimental
 Complex True Experimental
 Quasi-Experimental
 Pre-Experimental
Characteristics of True Designs
 Manipulation (treatment)
 Randomization
 Control group

Characteristics of simple true designs

 One IV with 2 levels (T, C)
 One DV
Complex True Experimental
 Randomized matched control group
design
 Increased levels of IV
 Factorial design
 Multiple DVs
Complex True Experimental
 Randomized matched control group
design
 Increased levels of IV
 Factorial design
 Multiple DVs
Randomized matched control group
design
M R T Post
M R C Post

• Used in small samples

∀ ↑ cost in time & money
Complex True Experimental
 Randomized matched control group
design
 Increased levels of IV
 Factorial design
 Multiple DVs
Increased Levels of IV
 Provides more complete information about
the relationship between the IV & DV
 Detects curvilinear relationships
 Examines effects of multiple treatments
 Reward Amount
$0 $1 $2 $3

DV
Performance level (% complete)

Amount of reward promised ($) IV

Increased Levels of IV

DV
Performance level (% complete)

Amount of reward promised ($) IV

Complex True Experimental
 Randomized matched control group
design
 Increased levels of IV
 Factorial design
 Multiple DVs
Factorial Design
 >1 IV (factor)
 Simultaneously determine effects of 2 or
more factors on the DV (real world)
 Between Factor vs. Within Factor
 ID’d by # of factors and levels of factors

2X2
Do differing exercise regimens (hi,
med, lo intensity) have the same effect
on men as they do on women?

 3 X 2 (Exercise Regimen X Gender)

2 factors
 Exercise Regimen – 3 levels
 Gender – 2 levels
 Between factors
 DV?
 Experimental IVs or Participant IVs?
 Gender
Male Female
High
Exercise
Intensity

Medium
Low
Do strength gains occur at the same rate in men
as they do in women over a 6 mo. training period?
Measurements are taken at 0, 2, 4, 6 mo.
 2 X 4 (Gender X Time)
? factors
 Time – 4 levels
 Gender – 2 levels
 Between or within factors?
 DV?
 Experimental IVs or Participant IVs?
 Time
0 mo. 2 mo. 4 mo. 6 mo.
Gender

Male

Female
Cell means, Margin means
Main Effects, Interactions

Time
0 mo. 2 mo. 4 mo. 6 mo.
Gender

Male 50 70 90 130 85

Female 30 60 75 90 64

40 65 83 110 74

Cell means
Margin means Grand mean
Interaction & Main Effects
 Interaction - combined effect of the factors
on the dependent variable
 Main effect – the deviation of two or more
treatment means from the grand mean
 Time
0 mo. 2 mo. 4 mo. 6 mo.
Gender

Male 50 70 90 130 85

Female 30 60 75 90 64
40 65 83 110 74
 Parallel lines indicate no Is there
a main
interaction. effect?
Interaction of Exercise Intensity and Gender

70
65
VO2 Max (ml/kg/min)

60
55 Male
50 Female
45
40
35
High Medium Low
Exercise Intensity
 Is there
a main
Interaction of Exercise Intensity and Gender effect?

70

65
VO2 Max (ml/kg/min)

60

55 High
Medium
50
Low
45

40

35
Male Female
Gender
 Non-parallel lines indicate an
Is there
interaction. a main
Interaction of Gender and Time
effect?
140

120
Weight Lifted (lbs.)

100

80 Male
60 Female

40
20

0
0 mo. 2 mo. 4 mo. 6 mo.
Time
 Is there
a main
Interaction Between Gender and Time effect?
140
120
Weight Lifted (lbs.)

100
0 mo.
80 2 mo.
60 4 mo.
6 mo.
40

20
0
Male Female
Gender
Interpretation
 Always interpret the interaction first
(graphical)
 If no significant interaction, interpret main
effects
 Advantages of factorial designs:
 Greater protection against Type I error
 More efficient
 Can examine the interaction
 Disadvantages:
 ↑ subject # for between factor designs

Consider external validity when

deciding which design to use.
IV A: Exposure to Violence – violent vs.
nonviolent video
IV B: Gender – male vs. female
B1
DV: # ads recalled (0-8)
B2
9
B
1 2
1 1 5 3
A 5
2 9 5 7
5 5 1

1 2

A
IV A: Exposure to Violence – violent vs.
nonviolent video
IV B: Gender – male vs. female
B1
DV: # ads recalled (0-8)
B2
9

A: Yes
B: No 5

AxB: Yes 1

1 2

A
Complex True Experimental
 Randomized matched control group
design
 Increased levels of IV
 Factorial design
 Multiple DVs
Do strength gains occur at the same rate in men
as they do in women over a 6 mo. training period?
Measurements are taken at 0, 2, 4, 6 mo.

Time
0 mo. 2 mo. 4 mo. 6 mo.
Gender

Male 50 70 90 130 85

Female 30 60 75 90 64
40 65 83 110 74
Types of Experimental Designs
 Simple True Experimental
 Complex True Experimental
 Quasi-Experimental
Characteristics of True Designs
 Manipulation (treatment)
 Randomization
 Control group

 Less control
 More real-world
 Program evaluation
Randomized posttest control group design

R T Post
R C Post
Randomized pretest-posttest control group
design

R Pre T Post
R Pre C Post
Quasi-experimental Designs
 One group posttest-only design
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Quasi-experimental Designs
 One group posttest-only design
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Randomized posttest control group design

R T Post
R C Post
One group posttest-only design
(One shot study)
T Post

No control of IV threats
Use?
Quasi-experimental Designs
 One shot study
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Randomized pretest-posttest control group
design

R Pre T Post
R Pre C Post
One group pretest-posttest design

Pre T Post

•History
•Maturation
•Testing Use control group
•Instrument decay
•Regression
Quasi-experimental Designs
 One shot study
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Randomized posttest control group design

R T Post
R C Post
Non-equivalent control group design

(Static group comparison design)

T Post
C Post

•Selection bias
Quasi-experimental Designs
 One shot study
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Randomized pretest-posttest control group
design

R Pre T Post
R Pre C Post
Non-equivalent control group
pretest-posttest design
Pre T Post
Pre C Post

•Can check selection bias

Quasi-experimental Designs
 One shot study
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Time series
Pre Pre Pre Pre T Post Post Post Post
Quasi-experimental Designs
 One shot study
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Quasi-experimental Designs
 One shot study
 One group pretest-posttest design
 Non-equivalent control group design
 Non-equivalent control group pretest-posttest
design
 Time series
 Single subject designs (Case study)
 Developmental designs
Developmental Research Designs

Longitudinal Cross Sectional

 Powerful (within  Less time consuming

subject)  Cohorts problem
 Time consuming
 Attrition
 Testing effect
Choosing a Research Design
 Best addresses the problem
 Ethics
 Cost in time and money
 Validity (internal & external)