11-1

Chapter 11

Experiments
and
Test Markets

McGraw-Hill/Irwin © 2006 The McGraw-Hill Companies, Inc., All Rights Reserved.

 11-2

Learning Objectives

Understand . . .
• uses for experimentation
• advantages and disadvantages of the
experimental method
• seven steps of a well-planned experiment
• internal and external validity with
experimental research designs
 11-3

Learning Objectives

• three types of experimental designs and the

variations of each
• functions and types of test marketing used
in experimenting with new marketing
products and services
 11-4

Causal Evidence

Agreement between
IVs and DVs

Time order of occurrence

Extraneous variables
did not influence DVs
 11-5

Evaluation of Experiments

Advantages Disadvantages
• Ability to manipulate • Artificiality of labs
IV • Non-representative
• Use of control group sample
• Control of extraneous • Expense
variables • Focus on present and
• Replication possible immediate future
• Field experiments • Ethical limitations
possible
 11-6

Exhibit 11-1 Experimentation

in the Research Process
 11-7

Conducting an Experiment

Specify treatment levels

Specify treatment levels
Control environment
Choose experimental design

Select and assign participants

Pilot-test, revise, and test

Collect data

Analyze data
 11-8

Exhibit 11-2 Experiment of

Placement of Benefits Module
 11-9

Selecting and Assigning

Participants

Random
Matching
assignment
 11-10

Random Assignment
 11-11

Exhibit 11-3
Quota Matrix Example
 11-12

Measurement Options

Observation

Physiological
Physiological Paper-and-
Paper-and-
measures
measures pencil
pencil tests
tests
Options
Options
Scaling Self-
Scaling
techniques administered
techniques
instruments
 11-13

Validity in Experimentation

Internal External
 11-14

Threats to Internal Validity

Maturation History

Experimental
Testing
mortality Threats

Statistical
Instrumentation
regression

Selection
 11-15

Additional Threats to
Internal Validity

Diffusion
Diffusion of
of treatment
treatment

Compensatory
Compensatory equalization
equalization

Compensatory
Compensatory rivalry
rivalry

Resentful
Resentful disadvantaged
disadvantaged

Local
Local history
history
 11-16

Threats to External Validity

Reactivity of
testing on X

Interaction of
selection and X

Other
reactive factors
 11-17

Experimental
Research Designs

Pre-experiments

True experiments

Field experiments
 11-18

After-Only Case Study

X O

Pre-experiment
 11-19

One Group Pretest-

Posttest Design

O1 X O2

Pre-experiment
 11-20

Static Group Comparison

X O1
O2

Pre-experiment
 11-21

Pretest-Posttest Control
Group Design

R O1 X O2
R O3 O4

True experiment
 11-22

Posttest-Only Control
Group Design

R X O1
R O2

True experiment
 11-23

Nonequivalent Control
Group Design

O1 X O2
O3 O4

Field experiment
 11-24

Separate Sample Pretest-

Posttest Design

R O1 (X)
R X O2

Field experiment
 11-25

Group Time Series Design

R O1 O2 O3 X O4 O5 O6
R O7 O8 O9 O10 O11 O12

Field experiment
 11-26

Test Market Selection

Over-testing

Control
Control of
of
distribution
distribution Representative
Representative

Criteria
Criteria
Isolation Multiple
Multiple
Isolation
locations
locations

Media
coverage
 11-27

Types of Test Markets

Standard
Standard

Controlled
Controlled

Electronic
Electronic

Simulated
Simulated

Virtual
Virtual

Web-enabled
Web-enabled
 11-28

Exhibit 11-5
Test Market Cities
 11-29

Key Terms

• Blind • Experiment
• Control group • Experimental treatment
• Controlled test market • External validity
• Dependent variable • Field experiment
• Double-blind • Hypothesis
• Environmental control • Independent variable
• Internal validity
 11-30

Key Terms

• Matching • Test market

• Operationalized – Electronic test market
• Quota matrix – Simulated test market
– Standard test market
• Random assignment
– Virtual test market
• Replication
• Treatment levels
• Web-enabled test market
 11-31

Exhibit 11-6