Topic 2

Experimental method in
consumer behavior research
 Agenda
• Correlational vs. causal relationship
• How to design an experiment
• 2X2 factorial design
 Correlational vs. causal relationship
• Correlational relationship: A correlates with B
• Causal effect: A causes B

Spurious correlations: Margarine linked to divorce? - BBC News

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Hormone Replacement Treatment
reduces heart disease?

Hormone replacement
treatment
Higher social-economic
groups with healthier diet
and exercise habits

Reduced heart disease

Bush, Trudy L., et al. "Estrogen use and all-cause mortality: preliminary results from the
Lipid Research Clinics Program Follow-Up Study." Jama 249.7 (1983): 903-906.
https://pubmed.ncbi.nlm.nih.gov/11297189/

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Another example of mis-attributed
causal effect
• Young children who sleep with the light on are
much more likely to develop myopia in later
life. (Nature, 1999)
child myopia
Parent myopia x (Nature, 2000)

leaving a light on

Zadnik, Karla; Jones, Lisa A.; Irvin, Brett C.; Kleinstein, Robert N.; Manny, Ruth E.; Shin, Julie A.; Mutti, Donald O.
(2000). "Vision: Myopia and ambient night-time lighting". Nature. 404 (6774): 143–144.
Gwiazda, J.; Ong, E.; Held, R.; Thorn, F. (2000). "Vision: Myopia and ambient night-time lighting". Nature.
404 (6774): 144.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Conditions for a causal relationship
• Causal relationship means that something has
happened, or is happening, based on something that
has occurred or is occurring.
– 1) The cause preceded the effect.
– 2) The cause was related to the effect.
– 3) We can find no plausible alternative explanation for the
effect other than the cause.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 How to test a causal relationship?
• Using experiments
– In an experiment, a researcher manipulates one or
more variables, while holding all other variables
constant.
– By noting how the manipulated variables affect a
response variable, the researcher can test
whether a causal relationship exists between the
manipulated variables (IV, independent variables)
and the response variables (DV, dependent
variables).

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Elements in experimental design
• Independent variable
• Dependent variable
• Manipulation
• Control
• Random assignment

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Independent variable
• Independent variable (IV)
– The “cause”, or “treatment”, or what is manipulated.
– The factor that researchers are most interested in.
– Examples
• Causal relationship: Playing classical music → Increased sales in the
store
– IV: whether or not to play classical music
• Causal relationship: Drinking Coke → Obesity
– IV: the amount of coke consumed per day
– The independent variable in an experiment can only have
discrete values.
• Binary: Yes vs. No
• Multiple levels: Coke consumption 100ml vs. 200ml vs. 300ml per day

1.9
Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Dependent variable
• Dependent variable (DV)
– The “effect”.
– Its value depends on independent variables.
– Examples
• Causal relationship: Playing classical music → Increased sales in
the store
– DV: actual sales in the store
• Causal relationship: Drinking Coke → Obesity
– DV: actual change of weight

– Measuring the dependent variable

• Self-reported attitudes or behavioral intention.
• Actual purchase/consumption/donation etc. behavior.

1.10
Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Manipulation
• Manipulation describes the process by which an
experimenter purposefully change the independent
variables (IVs).
– By setting up different conditions, the
experimenter can observe whether the value of
DV vary to different conditions.
– IV suggests the difference among different
conditions.
– Manipulation suggests how to introduce the
difference to different conditions.

1.11
Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 IV vs. Manipulation (an example)
• An experiment is conducted to test whether or not
playing classical music can increase sales in the store.
– IV: whether or not to play classical music
– Manipulation: there are different ways to manipulate
Option 1: playing classical music in Store A and not
playing classical music in Store B (a store very similar to
Store A) during each store’s peak hours for two weeks.

Option 2: playing classical music in a store during its

peak hours on Mon and Wed; and not playing classical
music in the same store during its peak hours on Tue and
Thurs.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
Answer the questions on the following slides:
Q1 Suppose that you want to study whether
a TOEFL coaching program actually helps
students to score higher on the TOEFL, so
you gather data on a random sample of
students who have attended the program.

Suppose you find that 95% of the sample

scored higher on their TOEFL after attending
the program than before attending.
Moreover, suppose you calculate that the
mean of the improvements in TOEFL scores
was a substantial 50 points.

Can you be certain that the TOEFL coaching program caused

these students to improve on the test? Explain why.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
Q2 Suppose that researchers want to study whether pets provide
therapeutic benefits for their owners. Specifically, they decide to
investigate whether heart attack patients who own a pet tend to
recover more often than those who do not.

They randomly select a sample of heart attack patients from a

large hospital, ask who want to have pets, give them pets, and
then check up on them after one year. The researchers find that
92% of those with pets are still alive while only 64% of those
without pets have survived.
Can the researchers claim that this
experiment strongly suggests that pets
play a significant role in the survival of
heart attack patients?

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Control
• A control is used in experiments to minimize the
effect of variables which are not the interest of the
study.
– To control for possible effects of other variables so that
differences in DV can be attributed directly to IV.
• Control the “noises” brought by research participants (RP,
individual differences).
• Control the “noises” brought by experiment stimuli (i.e., materials
presented to participants).

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Noises from RPs: how to control
• Balance out individual differences
– E.g., Assign equal % of female in each condition.
• “Standardize” participant status
– E.g., tell all participants not to eat for 3 hours before
experiments; use a hypothetical brand name.
• Statistical analysis: measure individual differences and later
include them in data analysis as covariates.
– E.g., individual differences can be gender, brand
attitudes, purchase history, etc.
• Random assignment!

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Random assignment
• The experimenter cannot decide which participants
will be exposed to which conditions on a subjective
basis.
• Participants are assigned to different groups using
randomization, such as by flipping a coin, or using a
random number generator.
• The purpose is to control for possible individual
differences.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
What else can check whether a research result
is true or falsified: Replications

Link 1

Link 2

We contacted the authors to request the materials needed to conduct a replication. We ran two
identical replications. The first was run on MTurk using the same criteria the original authors
specified. The second was run on MTurk using a new(ish) feature that screens for only high-
quality “CloudResearch Approved Participants”. [97] Data Replicada #10

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
Noises from stimuli: how to control
• Use hypothetical brand names to exclude the
possible noises from existing brands.
• Use different products in different
experiments conducted to test the same effect,
to rule out the possibility that the effect is
only observed for a particular product
(category).

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
Q3 Suppose that a study is conducted to
determine whether taking regular doses of
Vitamin C increases one’s resistance to
catching the common cold. The researchers
form two groups – subjects in one group are
given regular doses of vitamin C while those
in the other group are not. Subjects are
assigned to the treatment group (getting the
vitamin) based on the flip of a coin.
The health of the subjects is monitored over the winter and the
researchers find that 56% of subjects in the treatment group caught at
least one cold while 82% of the other subjects caught at least one cold.

Can the researchers claim that taking regular doses of vitamin C

increases one’s resistance to catching the common cold in the winter?
Explain why.
Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Review of experimental design
(IV, DV, and manipulation)
• By manipulating independent variables, the
experimenter creates the conditions to
observe the possible causal effect of
independent variables on dependent variables.

A B
causes

• A is the independent variable, also called a factor;

• B is the dependent variable;
• How A varies across different conditions is a manipulation.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 How to describe experimental
elements (an example)
• Assuming researchers want to conduct an
experiment to test the effect of Vitamin C on the
chances of catching a common cold:
– IV: whether the patient takes Vitamin C or a placebo.
– DV: the probability of catching a common cold.
– Manipulation: there are two groups of RPs. One group
takes regular doses of Vitamin C everyday and the other
takes a placebo (e.g., sugar pills) everyday.
– Control for gender, age, general health conditions, etc.
– Patients are randomly assigned to either group and it is a
double-blinded experiment.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
An exercise for experimental design
• The research question: Will consumers think Häagen-Dazs
tastes better just because it is Häagen-Dazs?
– In other words, will consumers evaluate a product only based on
its brand name? How to design an experiment to test it?

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 About 2X2 factorial design
• What is a 2X2 factorial design?
• Why do we need a 2X2 factorial design?
• How to interpret results of a 2X2 factorial
design?

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
Full factorial design in experiments
• A full factorial experiment is an experiment
whose design consists of two or more factors,
each with discrete values or "levels", and the
experiment takes on all possible combinations of
these levels across all such factors.

• Such an experiment allows the experimenter to

study the main effect of each factor on DV as
well as the interaction effects on DV.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
The 2X2 full factorial design (an example)
Time in Instruction

Factors: Independent variables

Setting

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
The 2X2 full factorial design (an example)
Time in Instruction
In-class 1 hour/week 4 hours/week

Levels:
Setting

Subdivisions of factors
Pull-out

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
The 2X2 full factorial design (an example)
Time in Instruction
In-class 1 hour/week 4 hours/week

I hour per week, 4 hour per week,

in-class instruction in-class instruction
Setting
Pull-out

I hour per week, 4 hour per week,

pull-out instruction pull-out instruction

A 2(rows) x 2 (columns) design, 4 (i.e., 2x2) groups/conditions.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
How to describe a full factorial design?
• 2 factors, each factor has 3 levels

• 3 factors, each factor has 2 levels

• 2 factors, one has 2 levels and the other 3 levels

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
How to describe a full factorial design?
• 2 factors, each factor has 3 levels
– It is a 3X3 factorial design.
– A1B1, A1B2, A1B3, A2B1, A2B2, A2B3, A3B1, A3B2, A3B3
• 3 factors, each factor has 2 levels
– It is a 2X2X2 factorial design.
– A1B1C1, A1B1C2, A1B2C1, A1B2C2, A2B1C1, A2B1C2,
A2B2C1, A2B2C2
• 2 factors, one has 2 levels and the other 3 levels
– It is a 2X3 factorial design.
– A1B1, A1B2, A1B3, A2B1, A2B2, A2B3

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
How to analyze a full-factorial design?
• Main effects
• Interaction effects

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 What is a main effect?
• A “main effect” is the effect of one of the
factors on DV, neglecting the effects of all
other factors.
– Here “neglecting” means “averaging” across
groups defined by other factors.
• A 2X2 design: A1B1, A1B2, A2B1, A2B2
– To calculate A’s main effect, treat the four groups as two big
groups A1 and A2.
– Whether the difference is significant or not will be
determined by statistical analysis (an ANOVA test).

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 What is an interaction effect?
• A significant interaction effect suggests that the
effect of a factor (IV1) depends on the other
factor (IV2), or, the effect of a factor varies to
different levels of the other factor.

• Using a full factorial design to test an interaction

effect
– For example, using 2 (A: A1 vs. A2) X 2 (B: B1 vs. B2) to
test whether the effect of A on DV is different
between B1 and B2 levels.
• A1B1, A1B2, A2B1, A2B2

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
Interpreting results of a 2X2 factorial design

• On the next slide are three possible results

that researchers may find from the 2 (male vs.
female) X 2 (new medicine vs. placebo) study.
– Let’s assume DV difference that is larger than 30
reaches significance.
• That means, when the difference between
MeanCondition A and MeanCondition B is larger than 30, we assume
that the two group means are significantly different from
each other.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
How do we know whether or not there
is a significant difference?
Only a significant difference supports an effect. Whether there is a significant
difference between Meancondition 1 and Meancondition 2 needs to be tested with a
statistical analysis (e.g., ANOVA), and the analysis result indicates whether or not
the difference is significant. In the example below, because p = 0.027 < 0.05 (a
threshold value), the difference is significant.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Results (1)

Effect
Medicine
150 Placebo
130
110
110 100
90

70
50
50 40
30

10

Male Female

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Results (1): No interaction,
only main effect of medicine

Effect Main effect of medicine:

Medicine Placebo
150 (1) DV in the medicine groups:
130
(100+110)/2 = 105
110 (2) DV in the placebo groups:
110 100 (40+50)/2 = 45
90

70
50 Main effect of gender:
50 40 (1) DV in the male groups:
30 (100+40)/2 = 70
(2) DV in the female groups:
10
Male Female
(110+50)/2 = 80

It shows the main effect of medicine is significant: the new medicine is effective in curing the
disease. The gender effect is 70 vs. 80, not significant. Moreover, the medicine is equally
effective for both gender, suggesting no interaction effect (male: 100 vs. 40; female: 110 vs. 50).

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Results (2)
Medicine doesn't have a main effect and
Effect gender has a main effect;
140 Given anything taken, male is easier to
recover than female.
Medicine Placebo
120
120
110

100

80

60
50
40
40

20

Male Female

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Results (3)
Effect
140

120 Medicine has a main effect and gender also

120 has a main effect.
The new medicine is only effective for males
100
Medicine Placebo
80

60
60
50
40
40

20

Male Female

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Results (2): No interaction,
only main effect of gender

Effect
140
Main effect of gender:
(1) DV in the male groups:
120
120
110
Medicine Placebo (120+110)/2 = 115
(2) DV in the placebo groups:
100 (50+40)/2 = 45
80

60
50 Main effect of medicine:
40 (1) DV in the medicine groups:
40
(120+50)/2 = 85
20
(2) DV in the placebo groups:
(110+40)/2 = 75
0

Male Female
It shows the main effect of gender: no matter what was taken in, male can recover better than
female. The medicine effect is 85 vs. 75, not significant.

Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Results (3): with interaction effect,
and two main effects
140
Effect • Main effect of gender (90 vs.
120 45): male can recover better
120
than female regardless of what
100
Medicine Placebo was taken in.
80 • Main effect of medicine
60 (means: 85 vs. 50): the new
60
50 medicine is generally effective
40
40 regardless of the gender.
20
• The interaction effect: the new
medicine is effective only for
0 male
Male Female
– 120 vs. 60, medicine has an effect
– 50 vs. 40, medicine has no effect

When interaction effects are significant, main effects are not meaningful. That is, although
main effect of medicine is significant here, but the medicine is effective only for male.
Correlational vs. causal relationship| how to design an experiment | 2X2 factorial design
 Wrap-up
• Distinguish between a correlational and a
causal relationship.
• Elements in experimental design
– IV, DV, manipulations, control.
• How to interpret results from a 2 X 2 factorial
design experimental data.
 Read before the next class (available on BB):
P.375-381 (stops at “Roadmap for future research”)

• Fundamental motives • This paper discussed

– (1) evading physical harm
– (2) avoiding disease
– (3) making friends
– (4) attaining status,
– (5) acquiring a mate,
– (6) keeping that mate
– (7) caring for family
(1) how, why, and when
these fundamental
motives might shape
behavior, and
(2) how consumer
choices may help fulfill
one or more of these
evolutionary needs.