Professional Documents
Culture Documents
Research—Understanding When,
Where, and How to Enhance Realism
ANDREA C. MORALES
ON AMIR
LEONARD LEE
In this article, we consider why employing realistic experimental designs and measur-
ing actual behavior is important and beneficial for consumer research. More specifi-
cally, we discuss when, where, and how researchers might go about doing this in
order to increase the veracity and believability of their work. We analyze the choice of
independent variables (IVs) along the experimental-realism dimension, ranging from
artificial to realistic, and the choice of dependent variables (DVs) along the
behavioral-measures dimension ranging from hypothetical intention to actual behav-
ior. Importantly, we also map various goals of consumer research along these two
dimensions to highlight when it is most appropriate to enhance the realism and be-
havioral measures of an experiment. Using a number of illustrative examples from re-
search in the extant literature, we specifically highlight how consumer researchers
can increase experimental realism and utilize actual-behavior measures in their ex-
periments in order to improve both the fidelity of the research and the likelihood that
the research provides insight into real consumer behavior.
Andrea C. Morales (acmorales@asu.edu) is Lonnie L. Ostrom Chair in As indicated in my comments to the authors, I like the prop-
Business and professor of marketing at the W.P. Carey School of Business, osition, and the main result is interesting. But is it true?
Arizona State University, Tempe, AZ 85287. On Amir (oamir@ucsd.edu) is —Comments from an anonymous JCR reviewer
associate professor of marketing at the Rady School of Management, University
of California, San Diego, La Jolla, CA 92093. Leonard Lee (leonard.lee@nus. Consumer behavior research is a broad field bringing to-
edu.sg) is associate professor of marketing and Dean’s Chair at NUS Business gether diverse research methodologies. What binds these
School, National University of Singapore, BIZ 1, 8-20, Mochtar Riady myriad methodologies together is the drive to really under-
Building, 15 Kent Ridge Drive, Singapore 119245, Singapore. Order of author- stand the behavior of individuals as they interact with one
ship was determined by degree of Star Wars fandom, and any disagreements
were resolved implicitly through passive-aggressive editing of the manuscript,
another and their environment in a consumption context.
leaving the illusion of unchallenged solidarity. The authors thank the JCR edi- Somewhat ironically, understanding consumer behavior
tors, Darren Dahl, Eileen Fischer, Gita Johar, and Vicki Morwitz, for the invita- through experimental research does not always involve ex-
tion and unique opportunity to have this article included in the “Tutorials in amining the actual behavior of consumers. More often, in
Consumer Research” series, as well as for their feedback across multiple fact, it involves manipulating aspects of a stylized, artifi-
rounds. The authors also thank an anonymous JCR reviewer for graciously giv- cial scenario and measuring consumer responses to these
ing them permission to use his or her comments at the start of the article.
scenarios, responses that typically reflect consumers’
This tutorial was invited by editors Darren Dahl, Eileen Fischer, Gita thoughts and theories about the scenarios, and not their
Johar, and Vicki Morwitz. actual behavior. This approach can contribute to concerns—
such as that of the reviewer above—over whether the re-
Advance Access publication March 13, 2017
search is true, or at the very least, valid. In this work, we
C The Author 2017. Published by Oxford University Press on behalf of Journal of Consumer Research, Inc.
V
All rights reserved. For permissions, please e-mail: journals.permissions@oup.com Vol. 44 2017
DOI: 10.1093/jcr/ucx048
465
466 JOURNAL OF CONSUMER RESEARCH
examine the importance and benefits of utilizing realistic discuss the breadth and usefulness of available alternatives
manipulations and measuring actual behavior, and discuss for enhancing the realism (IVs) and behavioral measures
when, where, and how researchers might go about doing (DVs) of consumer research experiments in more detail in
this in order to increase the veracity and believability of later sections, but first, we delve into the question of when
their work. We analyze the choice of independent variables it is most appropriate to do so.
(IVs) along the experimental-realism dimension, ranging
from artificial to realistic, and the choice of dependent var-
manipulations and utilizing behavioral measures, as op- consequential, in order for the experiments to demonstrate
posed to artificial manipulations and nonbehavioral mea- the principles that the researchers proposed. Since the goal
sures, is most pertinent. Note, however, that this is not an of the research was to develop an alternative theory of de-
exhaustive list and other research goals exist that are not cision making (that applied generally across domains and
discussed here. contexts)—or in the researchers’ words, a new “theory of
The first goal that an experimental consumer research mind”—the researchers needed to show that the basic te-
paper may have is to develop a new theory and/or test the nets of expected utility theory could not explain choice pat-
experiments that greater reliance on emotional reactions of a product (i.e., beer) prior to consumption would influ-
during decision making results in greater preference con- ence their preference for the product. In these studies, the
sistency, inferring preferences from actual choices. In both experimental realism and behavioral nature of the mea-
of these papers, the behavior involved making a choice— sures allowed the researchers to convince the readers that a
to purchase in the former, and to indicate preference in the very surprising or counterintuitive result exists. Less realis-
latter. Eliciting one’s theory about the effects or one’s in- tic IVs and nonbehavioral measures would have weakened
tention to act would have provided less compelling evi- the impact of these findings.
products on a computer screen. For example, Morales to take their measurements for the suit, the researchers
(2005) and Castro, Morales, and Nowlis (2013) both used were able to make the lab experience more realistic for par-
real consumer packaged goods to create physical product ticipants. Consequently, we contend that participant evalu-
displays inside the lab, just like those that would be found ations of the (larger-than-usual sized) suit should be closer
inside a grocery store. Using actual, physical products as to what they would be in an actual consumption experience
stimuli rather than pictures of products can also have far- than if they had only imagined purchasing the suit.
reaching implications on participants’ responses, as prior Ideally, experimental realism is at its maximum when the
More generally, we contend that anything that makes the gets stuck (Kristofferson et al. 2017), how much food to
IVs of the experimental design more similar to the actual consume under certain conditions (Garg, Wansink, and
consumption experience will increase the experimental re- Inman 2007; McFerran et al. 2010; Scott et al. 2008), and
alism of the experiment and therefore also enhance the nat- how much time to wait before giving up on a task (Zhao,
uralism of the responses. As noted above, this could be as Lee, and Soman 2012). It is important to note that behavior
simple as using actual, physical products instead of pic- is even broader than these and sometimes even more sub-
tures (in cases where the research focuses on brick-and- tle. One may consider physical measures such as facial ex-
consequences and therefore meet the criteria of being a be- measures). The majority of experimental work in consumer
havior. For example, as discussed above, having partici- research falls into one of the following three bins: field ex-
pants expend effort to measure motivation or commitment periments, experiments in the field, and lab experiments.
(i.e., by clicking on a mouse or keyboard) is a behavior, Though field experiments, by definition, are naturally
whereas having participants rate their level of motivation is higher in realism for the IVs (experimental realism) and
not. Similarly, the same classification exists for writing behavior for DVs (behavioral measures), it is possible to
recommendations to friends as opposed to rating the likeli- increase both dimensions in all three types of designs. To
Therefore, this study was high on experimental realism (IV natural as possible), but higher than lab experiments, and
¼ realistic), but low on behavioral measures (DV ¼ not ac- as high as a nonfield experiment possibly can be in terms
tual behavior). Such a study in which the DV is not a be- of realism. In addition, although the experimental designs
havioral measure is subject to issues associated with all of such studies can be carefully orchestrated to hold as
self-reported measures (Nisbett and Wilson 1977), and many factors constant as possible, a key drawback of this
generalizing from these measures to strategy or policy im- type of experiment is that the researcher ultimately has less
plications still requires a bit of a leap of faith. However, control over the environment in which the study is con-
on the light-gray floor carpet under the station: in half the eye movements pointing to attention and information expo-
sessions, the researchers laid a short black runner on the sure (Pieters and Wedel 2007; Teixeira et al. 2012), voice
floor that covered only the area under the station, whereas intonations (Cavanaugh et al. 2017; Duke and Amir 2017;
in the remaining sessions, they laid a long black runner Lieberman et al. 2017) and facial expressions (Teixeira
that extended from under the station to the opposite end of et al. 2012; Ward and Broniarczyk 2011) to assess affective
the room. Participants were informed that the food samples states or emotions, heart rate or GSR to measure emotional
were not yet available, and after waiting for a few minutes, arousal (Mehta, Zhu, and Cheema 2012; Noseworthy, Di
be substantial and well worth the investment. The ability to more realistic experimental manipulations and measuring
generalize from the results is even stronger the more realis- actual behavior can help test these theories and provide a
tic the IVs and context are. In the aforementioned exam- greater understanding of real consumer behavior.
ples, if the context includes actual movie tickets, actual
marketing collateral for the movie, or an email to genu- REFERENCES
inely invite a friend for an outing, the generalization step
would be minimized. Aaker, Jennifer L. and Angela Y. Lee (2001), “‘I’ Seek Pleasures
Coleman, Nicole Verrochi and Patti Williams (2013), “Feeling Lee, Leonard and Dan Ariely (2006), “Shopping Goals, Goal
Like My Self: Emotion Profiles and Social Identity,” Journal Concreteness, and Conditional Promotions,” Journal of
of Consumer Research, 40 (August), 203 – 222. Consumer Research, 33 (1), 60–70.
Dhar, Ravi and Stephen M. Nowlis (1999), “The Effect of Time Lee, Leonard, Shane Frederick, and Dan Ariely (2006), “Try It,
Pressure on Consumer Choice Deferral,” Journal of You’ll Like It: The Influence of Expectation, Consumption,
Consumer Research, 25 (4), 369–84. and Revelation on Preferences for Beer,” Psychological
Duke, Kristen and On Amir (2017), “Guilt Accounting Theory: Science, 17 (12), 1054–8.
Separating Choice and Behavior Reduces Guilt During and Lee, Leonard and Claire I. Tsai (2014), “How Price Promotions
Pocheptsova, Anastasiya, On Amir, Ravi Dhar, and Roy F. Teixeira, Thales, Michel Wedel, and Rik Pieters (2012), “Emotion-
Baumeister (2009), “Deciding Without Resources: Resource Induced Engagement in Internet Video Advertisements,”
Depletion and Choice in Context,” Journal of Marketing Journal of Marketing Research, 49 (2), 144–59.
Research, 46 (3), 344–55. Tversky, Amos and Daniel Kahneman (1986), “Rational Choice
Russell, Cristel, Dale Russell, Andrea Morales, and Jean-Marc and the Framing of Decisions,” Journal of Business, 59 (4),
Lehu (2017), “Hedonic Contamination: Exposure to S251–78.
Advertising Taints Subsequent Entertainment Experiences,” Ward, Morgan K. and Susan M. Broniarczyk (2011), “It’s Not Me,
Journal of Advertising Research, 57(1). It’s You: How Gift Giving Creates Giver Identity Threat as a