Chapter 9

Conducting
Marketing
Experiments

©Cengage. All rights reserved.

 LEARNING OUTCOMES

1. Know basic characteristics of research experiments

2. Design an experiment using the basic issues of
experimental design
3. Know ways of maximizing the validity of
experiments, including the minimization of demand
characteristics
4. Weigh the trade-off between internal &external
validity
5. Recognize the appropriate uses of test-marketing
6. Avoid unethical experimental practices

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 3
The Characteristics of Experiments
• Subjects
• The sampling units for an experiment, usually human
respondents who provide measures based on the
experimental manipulation.
• Experimental Conditions
• One of the possible levels of an experimental
(independent) variable manipulation.
• Ex. Placebo & aspirin, dependent variable?
• Variables included in the statistical analysis as a way of
controlling or accounting for variance due to that
variable:
• Blocking variables – categorical variables ex. Gender & pain relief
• Covariate – continuous variable ex. Age & pain relief
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 5
Effects
• Main Effects
• The experimental difference in dependent
variable means, between the different levels of
any single experimental variable.
• Interactions
• Differences in dependent variable means due to
a specific combination of independent variables.

Scale: 0 = “would not shop” to 300 = “definitely would shop”

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 7
EXHIBIT Color and Lighting Experiment
9.1-9.3

Scale: 0 = “would not shop” to 300 = “definitely would shop”

©Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 8
4 Basic Issues in Experimental Design
1. Manipulation of Independent Variable
• Experimental treatment: the way an
experimental variable is manipulated.
• Categorical variables: described by class or quality
• Continuous variables: described by quantity (level)
– Ex. Color, brightness of light
• Experimental Group
• A group of subjects to whom an experimental
treatment is administered.
• Control Group
• A group of subjects to whom no experimental
treatment is administered.
– Ex. Split cable ad test
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 10
Experimental Design (cont’d)
• Several experimental treatment levels
(different values of independent) may be used.
• Ex. Different levels of advertising in split cable test
• More than one independent variable may be
examined.
• Cell: a specific treatment combination associated
with an experimental group.
– Computation of the number of cells in an experiment:
K = (T1)(T2)..(Tm)
– Ex. Color & lighting test with 3 colors & 3 levels of light.

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 11
Experimental Design (cont’d)
2. Selection and Measurement of the
Dependent Variable
• Selecting dependent variables that are relevant
and truly represent an outcome of interest is
crucial.
• Ex. Crystal Pepsi test market – trial
• Choosing the right dependent variable is part
of the problem definition process.
• Thorough problem definition will help the
researcher select the most important dependent
variable(s).
– Ex. TV ad test market – brand awareness, recall,
preference or sales?
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 12
Experimental Design (cont’d)
3. Selection and Assignment of Test Units
• Test units: the subjects or entities whose responses to
treatment are measured or observed.
• Sample Selection And Sampling Errors
• Systematic or non-sampling error
– Subject selection, experimental design, and unrecognized extraneous
variables
– Ex. Snacks & exam performance
• Overcoming sampling errors
– Randomization – ex. Snack & exam test 8am
– Matching – ex. Gender
1pm
– Repeated measures – ex. Color & lighting
– Extraneous Variables
7pm
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 14
Experimental Design (cont’d)
4. Extraneous Variables
• Experimental Confound
• When there is an alternative explanation beyond the
experimental variables for any observed differences in
the dependent variable.
– Ex. Fish stick taste test; scores of 78 vs. 39; n= 400
• Once a potential confound is identified, the validity of
the experiment is severely questioned.
• Sources
– Sampling error
– Systematic error ex. Snack & exam test – time
– Later-identified extraneous variables ex. Fish stick taste test
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 15
Demand Characteristics and
Experimental Validity
• Demand Characteristic
• An experimental design element or procedure
that unintentionally provides subjects with
hints about the research hypothesis.
• Ex. Informing subjects about hypothesis before
color & lighting study
• Demand Effect
• Occurs when demand characteristics actually
affect the dependent variable.
• Ex. Interviewer facial expressions

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 16
Reducing Demand Characteristics
• Experimental disguise
• Placebo – an experimental deception involving
a false treatment.
• Placebo effect – the corresponding effect in a
dependent variable that is due to the psychological
impact that goes along with knowledge that a
treatment has been administered.
• Isolate experimental subjects
• Use a “blind” experimental administrator
• Administer only one experimental
condition per subject
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 17
Establishing Control
• Constancy of Conditions
• Subjects in all experimental groups are
exposed to identical conditions except for the
differing experimental treatments.
• Ex. Beer taste testing
• Counterbalancing
• Attempts to eliminate the confounding effects
of order of presentation by varying the order of
presentation (exposure) of treatments to
subject groups.
• Ex. Color & lighting test
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 18
Basic versus Factorial Experimental
Designs
• Basic Experimental Designs
• A single independent variable and a single
dependent variable.
• Factorial Experimental Designs
• Allows for an investigation of the interaction to
two or more independent variables.
• Ex. Synergy of price & advertising

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 19
Laboratory and Field Experiments
• Laboratory Experiment
• A situation in which the researcher has more
complete control over the research setting and
extraneous variables.
• Ex. Ad test in simulated environment
• Field Experiments
• Research projects involving experimental
manipulations that are implemented in a
natural environment.
• Ex. Split cable ad test

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 20
Advantages of Between-Subjects
Designs
• Within-Subjects Design
• Involves repeated measures because with each
treatment the same subject is measured.
• Ex. Light & Color study
• Between-Subjects Design
• Each subject receives only one
treatment combination.
• Usually advantageous although they
are usually more costly.
• Validity is usually higher.
• Why?
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 22
Internal Validity
• Internal Validity
• The extent that an experimental variable is truly
responsible for any variance in the dependent
variable.
• Does the experimental manipulation truly cause
changes in the specific outcome of interest?
– Ex. Fish stick taste testing
• Manipulation Checks
• A validity test of an experimental manipulation to
make sure that the manipulation does produce
differences in the independent variable.
• Ex. Comparing two price points for a car
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 23
Factors Affecting Internal Validity
• History
• Maturation
• Testing
• Instrumentation
• Selection
• Mortality

Study Hint: M-H-I-S-T-M

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 24
Internal Validity (cont’d)
• History Effect
• Occurs when some change other than the
experimental treatment occurs during the
course of an experiment that affects the
dependent variable.
• Ex. Competition changes strategy
• Cohort Effect
• A change in the dependent variable that occurs
because members of one experimental group
experienced different historical situations than
members of other experimental groups.
– Ex. Financial incentives & performance
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 25
Internal Validity (cont’d)
• Maturation Effects
• Effects that are a function of time and the
naturally occurring events that coincide with
growth and experience.
• Ex. Compensation program & productivity 1yr
• Testing effects
• A nuisance effect occurring when the initial
measurement or test alerts or primes subjects in
a way that affects their response to the
experimental treatments.
• Only occur in before & after tests
– Ex. health tests before & after an exercise intervention.
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 26
Internal Validity (cont’d)
• Instrumentation Effect
• A change in the wording of questions, a change in
interviewers, or a change in other procedures causes a
change in the dependent variable.
• Ex. Bored interviewers
• Selection
• The selection effect is a sample bias that results from
differential selection of respondents for the
comparison groups, or a sample selection error.
• Ex. Gender skew in comparison
• Mortality Effect
• Occurs when some subjects withdraw from the
experiment before it is completed.
• Ex. High vs. Low Sales supervision
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 27
 Extraneous Variables
Example Extraneous Variable

A major employer closes its plant in

test market area.

Subjects become tired during the

experiment.

A questionnaire about the traditional

role of women
triggers enhanced awareness of .
females in an experiment.

9–28
©Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 Extraneous Variables (continued)
Example Extraneous Variable

New questions about women are

interpreted differently from earlier
questions.

Control group and experimental

group is self-selected group based on
preference for soft drinks

Subjects in one group of a hair dying

study marry rich widows and move to
Florida

9–30
©Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
External Validity
• External Validity
• The accuracy with which experimental results
can be generalized beyond the experimental
subjects.
• Student surrogates: Atypical?
• Trade-Offs Between Internal and External
Validity
• Artificial laboratory experiments usually are high
in internal validity, while naturalistic field
experiments generally have less internal validity,
but greater external validity.
• Ex. Ad testing 32
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part.
Using Test-Marketing
• Forecasting new product success
• Testing the marketing mix
• Identifying product weaknesses

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 34
Estimating Sales Volume: Some
Problems
• Over-attention
• Ex. TV ads, sales efforts
• Unrealistic store conditions
• Ex. Eye level stocking, extra facings
• Reading competitive environment
incorrectly
• Test-market sabotage: intentional attempts to
disrupt the results of a test-market being
conducted by another firm.
• Ex. P&G’s Millstone coffee & Starbuck’s test market
• Time lapse
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 36
Advantages and Disadvantages of
Test-Marketing
• Advantages
• Real-world setting
• Easily communicated results
• Disadvantages
• Cost
• - Ex. Hidden Valley Ranch test market
• Time
• Loss of secrecy

©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 37
Ethical Issues in Experimentation
• Debriefing experimental subjects
• Communicating the purpose of the experiment
• Explaining the researcher’s hypotheses about
the nature of consumer behavior
• Goal is to return to state prior to study
– Ex. Smoking study
• Attempts to interfere with a competitor’s
test-marketing efforts
• Such acts as changing prices or increasing
advertising to influence (confound) competitors’
test-marketing results are ethically questionable.
• Ex. Hidden Valley Ranch dressing
©2013 Cengage
©Cengage Learning.
Learning.
All Rights
All Rights
Reserved.
Reserved.
May not
Maybenot
scanned,
be scanned,
copied
copied
or duplicated,
or duplicated,
or posted
or posted
to a publicly
to a publicly
accessible
accessible
website,
website,
in whole
in whole
or in part.
or in part. 39
©Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 40