Business

Research Methods

William G. Zikmund

Chapter 12:
Experimental Research
 Experiment

• A research investigation in which conditions are

controlled
• One independent variable is manipulated
(sometimes more than one)
• Its effect on a dependent variable is measured
• To test a hypothesis
 Basic Issues of Experimental
Design
• Manipulation of the Independent Variable
• Selection of Dependent Variable
• Assignment of Subjects (or other Test
Units)
• Control Over Extraneous Variables
The experimenter has some degree of
control over the independent variable.
The variable is independent because its
value can be manipulated by the
experimenter to whatever he or she
wishes it to be.
 Experiment Treatment

Alternative manipulations of the

independent variable being
investigated
 Independent Variable

• The experimenter controls independent

variable.
• The variable’s value can be
manipulated by the experimenters to
whatever they wish it to be.
 Manipulation of Independent
Variable
• Classificatory Vs. continuous variables
• Experimental and control groups
• Treatment levels
• More than one independent variable
 Experimental Treatments
• The alternative manipulations of the
independent variable being investigated
 Dependent Variable

• Its value is expected to be dependent

on the experimenter’s manipulation
• Criterion or standard by which the
results are judged
 Dependent Variable

• Selection
– e.g... sales volume, awareness, recall,
• Measurement
 Test Units
• Subjects or
entities whose
response to the
experimental
treatment are
measured or
observed.
 Two Types of Experimental
Error
• Constant errors
• Random errors
 Field versus
Laboratory Experiments
Controlling Extraneous Variables
• Elimination of extraneous variables
• Constancy of conditions
• Order of presentation
• Blinding
• Random assignment
 How May an Experimenter control for
Extraneous Variation?
• Eliminate Extraneous Variables
• Hold Conditions Constant
• Randomization
• Matching Subjects
Establishing Control
 Demand Characteristics
• Experimental procedures that intentionally
hint to subjects something about the
experimenter’s hypothesis
 Demand Characteristics

• Guinea pig effect

• Hawthorne effect
Field Vs. Laboratory Experiment
Laboratory Experiment Field Experiment

Artificial-Low Realism Natural-High Realism

Few Extraneous Many Extraneous

Variables Variables
High control Low control

Low Cost High Cost

Short Duration Long Duration

Subjects Aware of Subjects Unaware of
Participation Participation
 Control Groups
Isolate
extraneous
variation
 When does an Experiment have
Internal Validity?
Internal Validity - The ability of an
experiment to answer the question whether
the experimental treatment was the sole
cause of changes in a dependent variable

Did the manipulation do what it was supposed

to do?
 Factors Influencing Internal
Validity
• History
• Maturation
• Testing
• Instrumentation
• Selection
• Mortality
 Isolating Extraneous Variation
with a Control Group
• History Effects
• Maturation Effects
• Mortality Effects
Type of Extraneous Variable Example

History - Specific events in the A major employer

environment between the Before closes its plant in
and After measurement that are test market area
beyond the experimenter’s control

Maturation - Subjects change Subjects become

during the course of the experiment tired

Testing - The Before measure alerts Questionnaire

or sensitizes subject to nature of about the traditional
experiment or second measure. role of women
triggers enhanced
awareness of women
in an experiment.
Instrument - Changes in New questions about
instrument result in response bias women are interpreted
differently from earlier
questions.

Selection - Sample selection Control group and

error because of differential experimental group is
selection comparison groups self-selected group
based on preference for
soft drinks

Mortality - Sample attrition; some Subjects in one group

subjects withdraw from experiment of a hair dying study
marry rich widows and
move to Florida
How can Internal Validity
Increase?
Increasing Internal Validity
• Control group

• Random assignment

• Pretesting and posttesting

• Posttest only
What are the Different Basic
Experimental Designs?
 Quasi-Experimental Designs
• One Shot Design (After Only)
• One Group Pretest-Posttest
• Static Group Design
One Shot Design (After Only)
X O1
One Group Pretest-Posttest
O1 X O2
 Static Group Design
Experimental Group X O1
Control Group O2
Three Good Experimental Designs

• Pretest - Posttest Control Group Design

• Posttest Only Control Group

• Solomon Four Group Design

Pretest-Posttest Control Group Design

Experimental Group R O1 X O2

Control Group R O3 X O4
Posttest Only Control Group

Experimental Group R X O1

Control Group R O2
 One-Shot Design
Internal Validity Problems
• History • Instrumentation
– weak – not relevant
• Maturation • Selection
– weak – weak
• Testing • Mortality
– not relevant – weak
One-Group Pretest-Posttest
Internal Validity Problems
• History • Instrumentation
– weak – weak
• Maturation • Selection
– weak – controlled
• Testing • Mortality
– weak – controlled
 Static-Group Design
Internal Validity Problems
• History • Instrumentation
– controlled – controlled
• Maturation • Selection
– possible source of – weak
concern • Mortality
• Testing – weak
– controlled
 Pretest-Posttest Control
Internal Validity Problems
• History • Instrumentation
– controlled – controlled
• Maturation • Selection
– controlled – controlled
• Testing • Mortality
– controlled – controlled
Solomon Four-Group Design
Internal Validity Problems
• History • Instrumentation
– controlled – controlled
• Maturation • Selection
– controlled – controlled
• Testing • Mortality
– controlled – controlled
 Posttest-Only Control
Internal Validity Problems
• History • Instrumentation
– controlled – controlled
• Maturation • Selection
– controlled – controlled
• Testing • Mortality
– controlled – controlled
Solomon Four Group Design

Experimental Group 1: R O1 X O2
Control Group 1: R O3 O4
Experimental Group 2: R X O5
Control Group 2: R X O6
Advanced Experimental Designs are
More Complex

• Completely randomized
• Randomized block design
• Latin square
• Factorial
Completely Randomized Design
• An experimental design that uses a random
process to assign subjects (test units) and
treatments to investigate the effects of only
one independent variable.
Completely Randomized Designs
Control: Experimental Experimental
no music treatment: treatment:
slow music fast music

Average minutes 16 18 12
shopper spends
in store
 Independent Variable A
Level 1 Level 2 Level 3

Group A Group B Group C

Completely Randomized Design
With a pretest posttest
Group A R O1 X1 O2

Group A R O3 X2 O4

Group A R O5 X3 O6
Completely Randomized Design
With a posttest
Group A R X1 O1

Group B R X2 O2

Group C R X3 O3
 Randomized Block Design

• An extension of the completely randomized

design in which a single extraneous variable
that might affect test units’ response to the
treatment has been identified and the effects
of this variable are isolated by blocking out
its effects.
 Randomized Block Design
Independent Variables

Control: Experimental Experimental

no music treatment treatment:
slow music fast music
Blocking variable

Mornings and
afternoons

Evening hours
 Factorial Design

• An experiment that investigates the

interaction of two or more variables on a
single dependent variable.
 Independent Variable 1

No Music Slow Music Fast Music

Independent Variable 2

No Music
cart signs

Grocery
cart signs
Factorial Design -- Roller Skates

Package Design
Price Red Gold

$25 Cell 1 Cell 4

$30 Cell 2 Cell 5
$35 Cell 3 Cell 6
 Effects
• Main effect • Interaction effect
• The influence of a • The influence on a
single independent dependent variable by
variable on a combinations of two
dependent variable. or more independent
variables.
 2 x 2 Factorial Design
Ad A Ad B

Men 65
Main Effects
> of Gender

Women 65

70 60
>

Main Effects of Ad
 100
Interaction Between Gender and
Advertising Copy
90
Wo
80 me
n
70
Believability

60 Men

50

40

30

20

10
Ad A Ad B
 Independent Variable 1

Level 1 Level 2

Level 1 Group A Group B

Independent
Variable 2

Level 2 Group D
Group C
2 x 2 Factorial with a Pretest
Posttest

Group A R O1 X11 O2

Group B R O3 X21 O4

Group C R O5 X12 O6

Group D R O7 X22 O8
2 x 2 Factorial Design with a
Posttest Measure

Group A R X11 O1

Group B R X21 O2

Group C R X12 O3

Group D R X22 O4
 A Test Market Experiment
on Pricing
Sales in Units (thousands)
Regular Price Reduced Price Cents-Off Coupon
$.99 $.89 Regular Price

Test Market A, B, or C 130 145 153

Test Market D, E, or F 118 143 129
Test Market G, H, or I 87 120 96
Test Market J, K, or L 84 131 99
Mean X1=104.75 X2=134.75 X1=119.25
Grand Mean
X=119.58
 Latin Square Design

• A balanced, two-way classification scheme

that attempts to control or block out the
effect of two or more extraneous factors by
restricting randomization with respect to the
row and column effects.
 Order of Usage

1 2 3

1 A B C
SUBJECT

2 B C A
3 C A B
TEST MARKETING

Not just trying But scientific

something testing
out

Controlled experimentation
 Test Marketing

Not just trying But scientific

something testing
out

Controlled experimentation
 Test Marketing

• An experimental procedure that provides an

opportunity to test a new product or a new
marketing plan under realistic market
conditions to measure sales or profit
potential.
 Functions of
Test Marketing

IDENTIFY AND
ESTIMATE CORRECT
OUTCOMES WEAKNESSES
IN PLANS
A Lengthy and Costly Procedure

When not
$$$$$
to Test?

Loss of How Long

Secrecy Should a
Test Last?
 Popular Test Markets
• Pittsfield, • Eau Claire,Wisconsin
Massachusetts • Wichita, Kansas
• Charlotte, North • Tulsa, Oklahoma
Carolina • Omaha, Nebraska
• Columbus, Ohio • Grand Junction.
• Little Rock, Arkansas Colorado
• Wichita Falls, Texas
• Evansville, Indiana
• Odessa-Midland, Texas
• Cedar Rapids, Iowa
 Selecting a Test Market

• Population size
• Demographic composition
• Lifestyle considerations
• Competitive situation
• Media
• Self-contained trading area
• Overused markets - secrecy
 Control Method of Test
Marketing
• Small city
• Low chance of being detected
• Distribution is forced (guaranteed)
 The Advantages of Using the
Control Method of Test Marketing
• Reduced costs
• Shorter time period needed for reading test
market results
• Increased secrecy from competitors
• No distraction of company salespeople from
regular product lines
Some Problems Estimating Sales
Volume
• Over-attention
• Unrealistic store conditions
• Reading competitive environment
incorrectly
• Incorrect volume forecasts
– Adjusted data
– Penetration and repeat purchase rate

• Time lapse
 High Tech Test Markets

Electric
Test
Markets
Simulated
Test
Markets Virtual-reality
Simulated
Test Markets