Organizational Behavior and Human Decision Processes 161 (2020) 3–19

Contents lists available at ScienceDirect

Organizational Behavior and Human Decision Processes

journal homepage: www.elsevier.com/locate/obhdp

Nudging: Progress to date and future directions☆

John Beshears a, *, Harry Kosowsky b
a
Harvard Business School and National Bureau of Economic Research, Soldiers Field, Boston, MA 02163, United States
b
National Bureau of Economic Research, 1050 Massachusetts Avenue, Cambridge, MA 02138, United States

A R T I C L E I N F O A B S T R A C T

Keywords: Nudges influence behavior by changing the environment in which decisions are made, without restricting the
Nudge menu of options and without altering financial incentives. This paper assesses past empirical research on nudging
Choice architecture and provides recommendations for future work in this area by discussing examples of successful and unsuccessful
Behavioral economics
nudges and by analyzing 174 articles that estimate nudge treatment effects. Researchers in disciplines spanning
Behavioral science
the behavioral sciences, using varied data sources, have documented that many different types of nudges succeed
in changing behavior in a wide range of domains. Nudges that automate some aspect of the decision-making
process have an average effect size, measured by Cohen’s d, that is 0.193 larger than that of other nudges.
Our analyses point to the need for future research to pay greater attention to (1) determining which types of
nudges tend to be most impactful; (2) using field and laboratory research approaches as complementary methods;
(3) measuring long-run effects of nudges; (4) considering effects of nudges on non-targeted outcomes; and (5)
examining interaction effects among nudges and other interventions.

1. Introduction The insight that small changes to the decision-making environment

In 2008, Richard H. Thaler and Cass R. Sunstein published Nudge:
Improving Decisions about Health, Wealth, and Happiness. The authors
argued that managers and policy makers can help individuals make
wiser choices by subtly altering the features of the environments in
which those individuals make decisions. For example, changing the
language used to describe the options in a menu, the format in which the
options are presented, or the process by which the options are selected
can alter individuals’ choices in domains ranging from health care to
personal finance to environmental conservation. Importantly, to qualify
as “nudges,” these “choice architecture” strategies do not mandate or
forbid options, and they do not meaningfully change the financial in­
centives associated with various options. Rather, nudges tap into the
psychology of decision making and gently guide individuals to different
outcomes. In this paper, written a little more than ten years after the
publication of Nudge, we provide an assessment of the academic litera­
ture evaluating the impact of nudge techniques, and we highlight
challenges that this literature will need to confront going forward.
can significantly alter choices has its foundations in a long literature in
behavioral economics. Going back at least as far as the work of Herbert
A. Simon on bounded rationality (Simon, 1955) and the work of Daniel
Kahneman and Amos Tversky on heuristics and biases (Kahneman &
Tversky, 1972, 1979; Tversky & Kahneman, 1973, 1974, 1981), social
scientists have recognized that individuals face limitations on their
ability to process information. Because of these limitations, individuals
form many judgments using mental shortcuts that can lead to systematic
decision-making errors. Subsequent work in behavioral economics
incorporated these psychological phenomena into theoretical models of
individual decisions (e.g., Bordalo, Gennaioli, & Shleifer, 2012; Bush­
ong, Rabin, & Schwartzstein, 2019; Gabaix, 2014; Koszegi & Rabin,
2006; Laibson, 1997; O’Donoghue & Rabin, 1999, 2001; Shefrin &
Thaler, 1988; Thaler & Shefrin, 1981). Additional work in this area also
documented the empirical relevance of such psychological factors for
economic outcomes in both laboratory and field settings (e.g., Angele­
tos, Laibson, Repetto, Tobacman, & Weinberg, 2001; Beshears & Milk­
man, 2011; Busse, Pope, Pope, & Silva-Risso, 2015; Camerer, Babcock,

☆
This article is an invited submission. It is part of a supplemental issue on “Healthy Habits” edited by Katherine L. Milkman, Dilip Soman, and Kevin G. Volpp and
supported by WW. This supplemental issue collects papers by participants in the Roundtable Discussion on Creating Habit Formation for Healthy Behaviors,
organized in late 2019 by the Wharton-Penn Behavior Change for Good Initiative (BCFG) and the Penn Center for Health Incentives and Behavioral Economics
(CHIBE).
* Corresponding author.
E-mail addresses: jbeshears@hbs.edu (J. Beshears), kosowsky@nber.org (H. Kosowsky).

https://doi.org/10.1016/j.obhdp.2020.09.001
Received 4 March 2020; Received in revised form 7 August 2020; Accepted 20 September 2020
Available online 10 December 2020
0749-5978/© 2020 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
J. Beshears and H. Kosowsky
Loewenstein, & Thaler, 1997; Camerer & Lovallo, 1999; Chetty, Looney,
& Kroft, 2009; DellaVigna & Malmendier, 2006; Fehr & Goette, 2007;
Lacetera, Pope, & Sydnor, 2012; Lerner, Small, & Loewenstein, 2004;
Loewenstein & Prelec, 1992; Malmendier & Tate, 2005; Milkman &
Beshears, 2009; Odean, 1998, 1999).1
Building on this large literature that investigates deviations from the
neoclassical economic model of decision making, the idea that choice
architecture strategies can and should be used to influence behavior
relies on two further implications of the behavioral economics
perspective. First, the impact of psychological biases on behavior im­
plies that individuals’ choices cannot necessarily be relied upon, in the
spirit of Afriat (1967), to reflect individuals’ normatively relevant
preferences. This gap between revealed and normative preferences
provides a rationale, beyond standard market failure considerations, for
well-intentioned managers and policy makers to intervene in in­
dividuals’ decision making (Beshears, Choi, Laibson, & Madrian,
2008a). Second, the role of psychological factors in decision making
represents not only a challenge but also an opportunity for improving
economic outcomes. Managers and policy makers can augment their
traditional toolkit of financial incentives with nudges that harness psy­
chological factors in the service of promoting wiser decisions. The
literature on nudges is thus a natural extension of prior work in
behavioral economics, but it is also distinct from past research in that it
focuses on how to apply behavioral economics ideas to important
practical problems with the goal of impacting outcomes.
For one example of how to apply behavioral economics ideas in this
way, consider automatic enrollment in retirement savings plans. Based
on the insights that people are sometimes inattentive (Gabaix, 2019),
tend to procrastinate when it comes to taking actions with short-run
costs but long-run benefits (Laibson, 1997; O’Donoghue & Rabin,
1999), are often reluctant to switch away from an option because of an
aversion to giving up its benefits (Samuelson & Zeckhauser, 1988), and
are attracted to options that are perceived to be the social norm (Cialdini
& Goldstein, 2004) or are perceived to be endorsed by a trusted au­
thority (McKenzie, Liersch, & Finkelstein, 2006), a manager who hopes
to increase employee savings in a firm’s retirement plan may wish to
change employees’ default enrollment status in the plan from non-
participation to participation (Thaler, 1994). This nudge changes the
enrollment process from an opt-in mechanism (which implements a
default contribution rate of zero to the plan for employees who do not
actively elect to participate) to an opt-out mechanism (which imple­
ments a default contribution rate that is strictly positive for employees
who do not actively elect an alternative). This change in the default
dramatically increases the fraction of employees contributing to the plan
(Beshears et al., 2006, 2008b, 2018; Choi et al., 2002, 2004; Madrian &
Shea, 2001). Better yet, retirement savings can be further boosted by
automatically escalating employees’ contribution rates at future points
in time, such as the beginning of each year (Thaler & Benartzi, 2004).
Many public and private organizations have embraced choice ar­
chitecture approaches to influencing decisions. Adoption of nudge
techniques by these organizations has been driven in part by the quan­
titative evidence demonstrating the potential for nudges like automatic
enrollment to cheaply and effectively change behavior. Adoption has
also been bolstered by the argument that nudges do not restrict the
choice set of well-informed individuals but do help individuals who
would otherwise have difficulty selecting and implementing beneficial
options from the choice menu (Camerer, Issacharoff, Loewenstein,
O’Donoghue, & Rabin, 2003; Thaler & Sunstein, 2003, 2008).
Continuing the savings example, automaticity and defaults are now
widely applied in the context of retirement plans. In the United States,
1
Thaler’s “Anomalies” column in the Journal of Economic Perspectives played
an important role in establishing and disseminating these ideas. See Rabin
(1998), DellaVigna (2009), and Bazerman and Moore (2012) for reviews of this
literature.
4
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
the Pension Protection Act of 2006 promoted the adoption of automatic
enrollment in defined contribution plans (Beshears, Choi, Laibson,
Madrian, & Weller, 2010), and a 2016 survey conducted by the Plan
Sponsor Council of America (2018) found that 60% of the 401(k) plans
in the sample used automatic enrollment. In several states, including
California, Illinois, and Oregon, certain employers that do not sponsor
qualified retirement plans are required to automatically enroll their
employees in state-administered Individual Retirement Accounts (Cen­
ter for Retirement Initiatives, 2019). Automatic enrollment also features
prominently in national-level retirement systems in the United
Kingdom, New Zealand, and Turkey. Beyond the domain of savings, the
Organisation for Economic Co-operation and Development reports that
202 organizations around the world apply nudge tactics to public policy
(OECD Research, 2018), and multinational firms such as Google, Merck,
Swiss Re, and Deloitte have internal groups of employees responsible for
incorporating choice architecture techniques into organizational
processes.
The enthusiasm for nudge tactics among practitioners has been
accompanied by burgeoning interest in nudge tactics among scholarly
researchers. In this paper, we use two approaches to discuss the schol­
arly literature on nudging to date and to highlight important challenges
for research in this area to consider in the future. First, we describe in­
dividual research articles that serve as examples illustrating key issues.
This discussion is far from a comprehensive literature review, but it
serves to make our points concrete. Second, we build and analyze a data
set of 174 articles that estimated nudge treatment effects. These articles
comprise the full set of articles in Elsevier’s Scopus database that (1)
presented new data evaluating the effect of a nudge, (2) cited one of
three seminal works on nudging (Camerer et al., 2003; Thaler & Sun­
stein, 2003, 2008), and (3) had at least ten citations as of July 2019. This
data set represents only a fraction of the universe of research on nudging
and is not a perfectly representative sample, but the selection criteria
suggest that the data set is unlikely to generate a misleading impression
of the portion of the literature receiving citations.
We begin our assessment of the empirical literature on nudging by
exploring nudge treatment effect estimates across academic disciplines,
domains of application, research settings (field observation versus other
approaches), and types of nudges (those that automate some aspect of
the decision-making process versus those that do not, as well as nudge
categories defined by Beshears and Gino (2015)). For each category
along each of these dimensions, we calculate the fraction of treatment
effect estimates associated with the category and perform a p-curve
analysis (Simonsohn, Nelson, & Simmons, 2014) for estimates that fall in
the category. For all categories that have enough treatment effect esti­
mates to permit meaningful analysis, we find that nudges have a sta­
tistically reliable effect on behavior. Thus, whether we consider the
literature on nudging overall or category by category, the finding that
nudges change outcomes is not purely an artifact of statistically unsound
research practices or publication bias in favor of statistically significant
estimates. However, this result does not rule out the possibility that our
database captures a set of nudge treatment effect estimates that reflects
some degree of publication bias. Comparisons of average treatment ef­
fects across categories are subject to the caveat that the severity of
publication bias may vary by category, potentially causing average
treatment effects in our sample to be differentially inflated relative to
true average treatment effects.
We discuss five issues that future research on nudging should devote
more attention to. First, we recommend that researchers investigate
which types of nudges tend to have larger effects on outcomes. Such
guidance would be useful to managers and policy makers who must
select from a large number of possible nudge techniques when seeking to
change behavior. In exploratory regressions analyzing our data set of
nudge treatment effect estimates, we find suggestive evidence that
nudges that automate some aspect of the decision-making process tend
to have larger and more robust effects than nudges that do not automate
some aspect of the decision-making process, but further work is needed
J. Beshears and H. Kosowsky
to examine this issue. Second, researchers should increasingly use field-
based and laboratory-based approaches as complementary methods to
investigate why and in which situations nudges change outcomes. Third,
researchers should place greater emphasis on studying the extent to
which nudges lead to cumulative long-run effects on outcomes. Nudges
may have long-run effects because they induce changes in habits,
because they prompt investments in durable capital (e.g., physical
capital or organizational capital as embedded in systems and processes),
or because they are applied repeatedly over time. Fourth, researchers
should put more effort into measuring the effects of nudges on non-
targeted outcomes, as such unintended consequences can partially or
even completely offset the intended effects of nudges on targeted out­
comes. Fifth, in applications, nudges often represent only one part of a
multi-pronged approach to changing behavior, so researchers should
increase focus on the interaction effects among nudges and traditional
interventions such as financial incentives. Depending on the circum­
stances, various interventions may be substitutes or complements for
one another, and the distinction is critical for designing effective pack­
ages of interventions.
Overall, the literature on nudging has been a success, as the empir­
ical evidence indicates that applying behavioral science in this way to
solve managerial problems and to advance policy objectives can indeed
change behavior. Still, we advocate a more ambitious approach for the
next generation of nudging research. Instead of merely asking whether
choice architecture strategies can change the action that a person takes
in a particular situation (the resounding answer is that they can), future
research should go much further by asking which types of choice ar­
chitecture strategies are most impactful at changing outcomes in an
enduring and comprehensive fashion in important contexts.
The paper proceeds as follows. In Section 2, we describe the con­
struction of our data set of nudge treatment effect estimates. In Section
3, we use the data set and a series of example articles to assess the nudge
literature to date. We explain our recommendations for future nudge
research in Section 4. Section 5 contains our general discussion and
conclusions.
2. Data set construction
In this section, we briefly describe the process for constructing our
data set capturing information on past empirical research on nudging.
The Online Appendix provides further details.
2.1. Selection of articles
To build our data set, we first identified all of the articles in Scopus,
Elsevier’s abstract and citation database, that cite at least one of three
foundational works on nudging: “Regulation for Conservatives: Behav­
ioral Economics and the Case for ‘Asymmetric Paternalism’” (Camerer
et al., 2003), “Libertarian Paternalism” (Thaler & Sunstein, 2003), and
Nudge: Improving Decisions about Health, Wealth, and Happiness (Thaler &
Sunstein, 2008). In order to focus on articles that have some degree of
influence on the scholarly conversation, we limited the list of articles to
those that had at least ten citations in Scopus as of July 1, 2019. We then
examined each article in the resulting list of 1052 articles to determine
whether it reported at least one treatment effect estimate for a nudge
intervention based on novel data (as opposed to a re-analysis of previ­
ously reported data). In judging whether or not an intervention qualifies
as a nudge, we relied on the definition articulated by Thaler and Sun­
stein (2008, p. 6): a nudge is an intervention that changes “people’s
behavior in a predictable way without forbidding any options or
significantly changing their economic incentives. To count as a mere
nudge, [an] intervention must be easy and cheap to avoid. Nudges are
not mandates.” This process narrowed the list to 174 articles.
Fig. 1 shows the distribution of publication year for the 174 articles
in our data set. The book Nudge, which was cited more frequently in our
data set than the other two foundational works on nudging, was first
5
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
published in 2008, so the publication year for articles in our data set is
concentrated in the years following 2008. The distribution tails off in the
later years because a later publication date leaves less time for an article
to accumulate at least ten citations by July 1, 2019.
2.2. Determination of the number of nudge treatment effect observations
The unit of observation in our data set is the nudge treatment effect
estimate. A given article may include multiple experimental conditions,
measure multiple outcome variables, and work with multiple pop­
ulations in multiple settings. Our data set includes separate observations
to capture the distinct nudge treatments embedded in a single article.
For example, an article comparing two nudge treatments to a control
group (using one outcome variable and one study population in one
research setting) is represented by two observations in our data set.
When an article examines multiple outcome variables, populations, or
settings, we follow the authors of the article in determining how to re­
cord the results. If the authors report results aggregated across these
dimensions, our data set records an observation for the aggregated
treatment effect. Otherwise, our data set records multiple observations
for the disaggregated treatment effects. This process generated 965
nudge treatment effect observations. See the Online Appendix for
details.
2.3. Variables recorded in the data set
For each observation in our data set, we recorded the following
variables:
• What academic discipline is associated with the article containing
the treatment effect estimate (e.g., economics)?
• In what domain of application was the treatment effect estimated (e.
g., health care)?
• In which type of research setting was the treatment effect estimated
(e.g., field experiment)?
• What is the nature of the outcome variable (e.g., a summary measure
of a series of actions in a field setting)?
• Did the nudge treatment automate some aspect of the individual’s
decision-making process?
• To which categories and subcategories of nudges does the treatment
belong, in the taxonomy of Beshears and Gino (2015)?
• Did the researchers collect follow-up data to measure the treatment
effect over a longer time horizon?
• If the researchers collected follow-up data, did the treatment effect
persist over the longer time horizon?
• Did the researchers also measure the treatment effect of the nudge on
an outcome variable that could offset the treatment effect on the
focal outcome variable recorded in this observation? If so, was there
such a variable in the same domain as the focal outcome variable?
Was there such a variable in a different domain from the focal
outcome variable?
• Did the article test whether there is an interaction effect between the
nudge and another intervention?
• Is the outcome variable continuous or dichotomous?
• What was the size of the treatment effect, as measured by Cohen’s d?
• For the test of the hypothesis that the treatment effect was zero, was
the p-value less than 0.10? Was the p-value less than 0.05? What was
the exact p-value?2
• Did the nudge backfire?
The Online Appendix describes the details of these variables, but
2
We create three variables to capture information about the p-value because
some papers report that the p-value was less than 0.10 or less than 0.05 without
reporting the exact p-value.
J. Beshears and H. Kosowsky
40
35
30
Number of articles
25
20
15
10
0
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Year of publication
Fig. 1. Distribution of articles in the data set by year of publication.
several variables require further commentary here.
The variables capturing the research setting and the nature of the
outcome variable are closely related to each other. In our analysis, we
focus on the research setting and not the nature of the outcome variable.
To develop a taxonomy of nudges, Beshears and Gino (2015) build on
the idea, popularized by Kahneman (2011), that humans have two basic
modes of thinking that they use to make decisions. System 1 thinking is
fast and intuitive, but it is prone to errors because it relies on mental
shortcuts that can sometimes be led astray. System 2 thinking is slow
and deliberative, but it is more likely to reach well-considered conclu­
sions. Beshears and Gino (2015) argue that nudges can alter an in­
dividual’s decisions by triggering system 1 (that is, by eliciting an
intuitive reaction that leads to a different choice), by engaging system 2
(that is, by prompting a momentary pause during which the individual
engages in a more reflective cognitive process), or by bypassing both
systems (that is, by removing the individual from some aspect of the
decision-making process).3 Each of these categories of nudges has sub­
categories, which we describe in Sections 3.4.2–3.4.4. We allow a given
nudge to fall in more than one category or subcategory to recognize that
a nudge may operate through multiple mechanisms. Note that the set of
nudges that automate some aspect of the decision-making process is
larger than the set of nudges that bypass both systems because some
nudges that trigger system 1 and some nudges that engage system 2 also
involve automaticity. For example, a nudge might automate the process
of filling out a registration form for a service, but the individual might
still be required to submit the form to sign up for the service. This nudge
triggers system 1 by simplifying the process of implementing the deci­
sion to sign up. The nudge does not bypass both systems because it has
not removed the individual from the process of making the decision to
sign up. Thus, the variable recording whether the nudge involves
automaticity is not redundant with the variable recording the nudge
category.
To calculate Cohen’s d for a continuous outcome variable, we divide
the estimated treatment effect size in the natural units of the outcome
variable by the mean of the standard deviation of the outcome variable
across the experimental conditions. When the outcome variable is
( √̅̅̅̅̅
dichotomous, we use the arcsine transformation 2*arcsine p1 − 2*
3
See Johnson et al. (2012), Ly, Mažar, Zhao, and Soman (2013), and Halpern
(2015) for alternative schemes for categorizing nudges.
6
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
√̅̅̅̅̅ )
arcsine p2 to calculate Cohen’s d, where p1 and p2 are the proportions
of successes in the two experimental groups (see, for example, Chernev,
Böckenholt, & Goodman, 2015).4 Many articles do not report sufficient
information to calculate Cohen’s d, so out of the 965 nudge treatment
effect observations in our data set drawn from 174 research articles,
Cohen’s d is non-missing for only 507 observations drawn from 101
articles. We perform an ordinary least squares regression for which the
outcome variable is an indicator for having a missing Cohen’s d. The
explanatory variables are (a) either an indicator for whether the nudge
involves automaticity or indicators for nudge categories in the Beshears
and Gino (2015) taxonomy; (b) indicators for academic discipline; (c)
indicators for domain of application; (d) an indicator for whether the
research setting involves field observation (either via a field experiment
or via observational analysis of a natural experiment involving field
data); and (e) an indicator for whether the outcome variable is dichot­
omous. The indicators for academic discipline are jointly statistically
significant, as are the indicators for domain of application, suggesting
that scholarly norms within a subfield are important determinants of
whether researchers report the necessary information for calculating
Cohen’s d. The indicator for whether the outcome variable is dichoto­
mous is also statistically significant, but this pattern is not surprising
because the data requirements for calculating Cohen’s d for dichoto­
mous variables are less stringent. The indicator for whether the nudge
involves automaticity and the indicator for whether the research setting
involves field observation are not significant, and the indicators for
nudge categories are not statistically different from each other.
Finally, it is natural to ask whether the analysis recorded in a given
observation in our data set was pre-registered. We follow the method­
ology of Schäfer and Schwarz (2019) to determine whether the analyses
captured in our data set were pre-registered, and we find that none of
them were. Thus, it is important to keep in mind that the effect size
estimates in our data set may overstate true effect sizes (DellaVigna &
Linos, 2020; Schäfer & Schwarz, 2019). Furthermore, differences in ef­
fect size estimates across the nudges in our data set may be due to dif­
ferential overstatement of true effect sizes, perhaps driven by
differential severity of publication bias. However, for some important
comparisons of nudge treatment effect estimates (e.g., the comparison of
nudges that involve automaticity versus those that do not), there is little
reason to expect differential overstatement of true effect sizes. In
4
When information on effect size in percentage points is not missing but
information on the baseline proportion of successes to total observations is
missing, we assume that the effect is symmetric around a proportion of 0.5.
J. Beshears and H. Kosowsky
addition, the possibility of overstated effect sizes does not call into
question the conclusion that nudges affect behavior. When we conduct
p-curve analyses (described in Section 3), the evidence indicates that the
conclusion that nudges impact outcomes is not purely the result of
publication bias.
3. Assessing the empirical literature on nudging
In Tables 1–4, we analyze our data set by breaking down the nudge
treatment effect estimates into categories along several different di­
mensions: academic discipline, domain of application, research setting,
and type of nudge. For each category that we consider, we report the
percentage of nudge treatment effect estimates in the data set that are
associated with that category. In this calculation, each observation re­
ceives a weight proportional to the inverse of the total number of ob­
servations associated with the same article. The calculation thereby
weights each article equally. We prefer this weighting scheme because it
does not place greater weight on an article merely because it measures a
larger number of outcome variables.
Within a category that we consider, we also report the mean Cohen’s
d, the percentage of effects with p < 0.05, and the percentage of effects
with p < 0.10. Here, each observation within a category receives a
weight proportional to the inverse of the total number of non-missing
observations in that category associated with the same article.
We also perform a p-curve analysis (Simonsohn et al., 2014) for each
of the categories that we consider. A p-curve analysis is a statistical test
that is applied to a collection of null hypothesis statistical tests. In our
case, the inputs to the p-curve analysis are the p-values from the tests of
the null hypothesis that the nudge treatment effect is zero. If the null
hypothesis were true for all of the nudge treatment effects in the cate­
gory under consideration, we would expect the p-values less than 0.05 to
be uniformly distributed over the interval from zero to 0.05. The p-curve
analysis formally tests the hypothesis that the distribution is uniform. If
this hypothesis is rejected and the nudge treatment effect p-values less
than 0.05 are close to zero more frequently than they are close to 0.05,
we conclude that there is evidence for the existence of a nudge treatment
effect not equal to zero.5 In order to weight articles equally in the p-
curve analysis and to use independent p-values as inputs to the p-curve
algorithm, we implement the following procedure. Among observations
within a category, we randomly select one observation per article and
enter the resulting subset of p-values into the p-curve algorithm.6 We
record whether the algorithm rejects (with p < 0.05) the null hypothesis
that the collection of p-values represents a body of research that does not
have evidential value. We repeat this procedure for 49 additional in­
dependent random draws of one observation per article, and we report
the percentage of these 50 random draws for which the algorithm rejects
the null hypothesis of no evidential value.
In Tables 1–4, we always report the percentage of nudge treatment
effect estimates associated with a given category. However, we only
report the results of the other calculations when the category is associ­
ated with at least ten research articles. In this section, we primarily
discuss the percentages of nudge treatment effect estimates associated
with various categories because we wish to establish that many different
types of nudges have been shown to be effective in a wide range of ac­
ademic disciplines, domains of application, and research settings. We
defer cross-category comparisons of Cohen’s d, the likelihood of an
5 3.4.1. Nudges that do and do not use automaticity
If the hypothesis of a uniform distribution is rejected and the nudge treat­
ment effect p-values less than 0.05 are close to 0.05 more frequently than they
are close to zero, we suspect that the collection of results is tainted by unsound
research practices that allow the researchers to manipulate p-values to be just
below the 0.05 threshold.
6 7
For the p-curve algorithm, we use the code available at http://www.p-c
urve.com/ (accessed January 30, 2020). When a research article reports a p-
value of zero, we enter a p-value of 0.001 into the algorithm.
7
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
effect with p < 0.05, and the likelihood of an effect with p < 0.10 to
Section 4, in which we report the results of regression analyses. Every
category for which we conduct the p-curve analysis has 86% or more of
its associated p-curves rejecting the null hypothesis of no evidential
value, with most categories having 100% of their associated p-curves
rejecting that null hypothesis. Thus, we do not discuss the p-curve results
category by category. We simply note here that the presence of nudge
treatment effects is statistically robust.
3.1. Nudge research by academic discipline
Table 1 shows that the literature evaluating nudge interventions is
truly multidisciplinary. The most represented academic discipline in our
data set is psychology, which is associated with 24.1% of nudge treat­
ment effect estimates. Economics is the second-most represented disci­
pline, with 21.3% of estimates. The category encompassing marketing
and consumer behavior follows close behind with 18.4%. Medicine,
environmental science, and public health also have substantial repre­
sentation. In each of the categories with at least ten associated research
articles, mean Cohen’s d falls in the range 0.20–0.53, and the percentage
of effects with p < 0.05 (p < 0.10) falls in the range 49–70% (55–70%).
3.2. Nudge research by domain of application
In Table 2, we see that the effect of nudge interventions on behavior
has also proved to be robust across many domains of application. In our
data set, 33.9% of the nudge treatment effect estimates are applied to
health-related decisions, and 24.1% are applied to decisions related to
the environment. Financial decision-making and prosocial behavior are
also well represented. Among domains associated with at least ten
research articles, mean Cohen’s d falls in the range 0.21–0.50, and the
percentage of effects with p < 0.05 (p < 0.10) falls in the range 52–70%
(60–76%).7
3.3. Nudge research by research setting
Table 3 splits our data set into two types of research settings:
observation in the field (via field experiments or observational analyses
of natural experiments) and observation in other environments (via
laboratory experiments, online experiments, or surveys). These two
types of settings are roughly equally prevalent, with observation in the
field accounting for 44.0% and observation in other environments ac­
counting for 56.0% of the nudge treatment effect estimates. Among
treatment effect estimates associated with field observation, mean
Cohen’s d is 0.32, and the percentage of effects with p < 0.05 (p < 0.10)
is 66.0% (70.0%). Among treatment effect estimates not associated with
field observation, mean Cohen’s d is 0.48, and the percentage of effects
with p < 0.05 (p < 0.10) is 60.1% (65.3%).
3.4. Nudge research by type of nudge
In Table 4, we present statistics summarizing nudge treatment effect
estimates associated with different types of nudges. Because one of the
issues that we highlight in Section 4 is the importance of determining
which types of nudges are particularly impactful relative to others, this
subsection goes into detail regarding our definitions of different cate­
gories and subcategories of nudges.
First, we divide nudge treatments into those that do and those that do
not automate some aspect of an individual’s decision-making process.
Nearly one-quarter of nudge treatment effect estimates fall in an uncate­
gorized domain of application. Mean Cohen’s d in this group is 0.554, and the
percentage of effects with p < 0.05 (p < 0.10) is 59.1% (63.0%).
J. Beshears and H. Kosowsky Organizational Behavior and Human Decision Processes 161 (2020) 3–19

Table 1
Nudge research by academic discipline.
Percentage of effects associated Mean Percentage of effects Percentage of effects Percentage of p-curves with
with this discipline Cohen’s d with p < 0.05 with p < 0.10 evidential value (p < 0.05)

All disciplines 100% 0.405 61.0% 65.6% 100%

Economics and finance 21.8% 0.201 62.4% 66.1% 100%
Economics 21.3% 0.201 61.3% 65.2% 100%
Finance 0.6% – – – –
Environmental science 8.6% 0.480 65.4% 67.1% 100%
Marketing / consumer 18.4% 0.377 48.7% 54.8% 100%
behavior
Medicine 11.5% 0.377 64.1% 68.3% 100%
Psychology and cognitive 27.0% 0.526 62.4% 69.1% 100%
science
Psychology 24.1% 0.531 60.9% 68.4% 100%
Cognitive Science 2.9% – – – –
Public health 6.3% 0.435 69.5% 69.5% 98%
Miscellaneous 6.3% 0.490 66.0% 70.0% 100%
Computer science 0.6% – – – –
Engineering 0.6% – – – –
Law 1.2% – – – –
Management 1.2% – – – –
Political science 0.6% – – – –
Public administration 0.6% – – – –
Transportation 1.7% – – – –

Percentage of effects associated with this discipline is a weighted mean, with an observation’s weight proportional to the inverse of the number of observations
associated with the same article. Mean Cohen’s d and percentage of effects with p < 0.05 (p < 0.10) are weighted means, with an observation’s weight proportional to
the inverse of the number of non-missing observations associated with the same article and with the discipline named in the leftmost column. For the p-curve analysis,
we randomly sample one observation per article and calculate the p-value for the null hypothesis that the random sample does not have evidential value. We repeat this
procedure for 50 independent random samples and report the fraction of samples for which p < 0.05. We only report mean Cohen’s d, the percentage of effects with p <
0.05, the percentage of effects with p < 0.10, and the percentage of p-curves with evidential value if the discipline is associated with at least ten articles.

Table 2
Nudge research by domain of application.
Percentage of effects associated Mean Percentage of effects with Percentage of effects with Percentage of p-curves with evidential
with this domain Cohen’s d p < 0.05 p < 0.10 value (p < 0.05)

Environment 24.1% 0.421 66.6% 71.3% 100%

Finance 7.4% 0.316 70.1% 76.1% 98%
Health 33.9% 0.416 59.3% 62.5% 100%
Exercise 1.7% – – – –
Health care 10.9% 0.286 57.3% 59.5% 96%
Healthy eating 19.5% 0.504 59.7% 62.0% 100%
Miscellaneous 1.7% – – – –
health
Prosocial behavior 6.9% 0.213 51.8% 66.4% 86%
Miscellaneous 27.7% 0.504 59.0% 62.2% 100%
Crime / criminal 2.3% – – – –
justice
Development 0.6% – – – –
Education 1.1% – – – –
Labor 0.6% – – – –
Other 23.1% 0.554 59.1% 63.0% 100%

Percentage of effects associated with this domain is a weighted mean, with an observation’s weight proportional to the inverse of the number of observations associated
with the same article. Mean Cohen’s d and percentage of effects with p < 0.05 (p < 0.10) are weighted means, with an observation’s weight proportional to the inverse
of the number of non-missing observations associated with the same article and with the domain named in the leftmost column. For the p-curve analysis, we randomly
sample one observation per article and calculate the p-value for the null hypothesis that the random sample does not have evidential value. We repeat this procedure for
50 independent random samples and report the fraction of samples for which p < 0.05. We only report mean Cohen’s d, the percentage of effects with p < 0.05, the
percentage of effects with p < 0.10, and the percentage of p-curves with evidential value if the domain is associated with at least ten articles.

An example of a nudge that involves automation is changing the default
enrollment status in a defined contribution retirement savings plan from
non-participation to participation. If an individual does not actively
indicate a desire to contribute to the plan or not contribute to the plan,
this nudge automatically implements a strictly positive contribution
amount on behalf of the individual. Previous research has documented
that this nudge generates large increases in retirement plan participation
rates (Beshears et al., 2006, 2008b; Choi et al., 2002, 2004; Madrian &
Shea, 2001).
Table 4 reveals that 15.3% of the nudge treatment effect estimates in
our data set involve a nudge that uses automaticity. Among this subset of
treatment effect estimates, mean Cohen’s d is 0.521, and the percentage
of effects with p < 0.05 (p < 0.10) is 72.7% (78.6%). Among the subset of
treatment effect estimates involving nudges that do not use automa­
ticity, mean Cohen’s d is 0.385, and the percentage of effects with p <
0.05 (p < 0.10) is 58.2% (62.7%). While the effects of nudges are sta­
tistically robust regardless of whether or not they automate some aspect
of the decision-making process, there is suggestive evidence that auto­
maticity produces larger effects. We explore this possibility in more
depth in Section 4.1.
3.4.2. Nudges that trigger system 1
We next examine nudges that operate by triggering system 1—that
is, nudges that change behavior by invoking a fast, intuitive reaction. In

8
J. Beshears and H. Kosowsky Organizational Behavior and Human Decision Processes 161 (2020) 3–19

Table 3
Nudge research by research setting.
Percentage of effects Mean Percentage of effects Percentage of effects Percentage of p-curves with
associated with this setting Cohen’s d with p < 0.05 with p < 0.10 evidential value (p < 0.05)

Field observation (field experiment 44.0% 0.320 66.0% 70.0% 100%

or observational study)
Laboratory experiment, online 56.0% 0.481 60.1% 65.3% 100%
experiment, or survey

Percentage of effects associated with this setting is a weighted mean, with an observation’s weight proportional to the inverse of the number of observations associated
with the same article. Mean Cohen’s d and percentage of effects with p < 0.05 (p < 0.10) are weighted means, with an observation’s weight proportional to the inverse
of the number of non-missing observations associated with the same article and with the setting named in the leftmost column. For the p-curve analysis, we randomly
sample one observation per article and calculate the p-value for the null hypothesis that the random sample does not have evidential value. We repeat this procedure for
50 independent random samples and report the fraction of samples for which p < 0.05.

Table 4
Nudge research by type of nudge.
Percentage of effects associated Mean Percentage of effects Percentage of effects Percentage of p-curves with
with this category Cohen’s d with p < 0.05 with p < 0.10 evidential value (p < 0.05)

Nudges that use automaticity 15.3% 0.521 72.7% 78.6% 100%

Nudges that do not use 84.7% 0.385 58.2% 62.7% 100%
automaticity
Nudges that trigger system 1 72.1% 0.468 63.2% 66.7% 100%
By arousing emotions 29.3% 0.326 54.9% 59.7% 100%
By harnessing biases 20.0% 0.515 64.3% 65.0% 100%
By simplifying the process 24.4% 0.539 71.4% 73.9% 100%
Nudges that engage system 2 41.6% 0.346 60.9% 65.1% 100%
By encouraging joint – – – – –
evaluation
By creating opportunities 22.4% 0.329 65.0% 68.7% 100%
for reflection
By prompting planning 1.9% – – – –
By inspiring broader 14.7% 0.396 49.5% 54.2% 100%
thinking
By increasing 1.2% – – – –
accountability
By emphasizing – – – – –
disconfirming evidence
By using reminders 2.6% – – – –
Nudges that bypass both 13.8% 0.546 69.9% 77.1% 100%
systems
By setting the default 13.2% 0.546 68.8% 76.3% 100%
By making automatic 0.6% – – – –
adjustments

Percentage of effects associated with this category is a weighted mean, with an observation’s weight proportional to the inverse of the number of observations
associated with the same article. Note that percentages add up to more than 100% because a given nudge can belong to more than one category and more than one
subcategory. Mean Cohen’s d and percentage of effects with p < 0.05 (p < 0.10) are weighted means, with an observation’s weight proportional to the inverse of the
number of non-missing observations associated with the same article and with the category named in the leftmost column. For the p-curve analysis, we randomly
sample one observation per article and calculate the p-value for the null hypothesis that the random sample does not have evidential value. We repeat this procedure for
50 independent random samples and report the fraction of samples for which p < 0.05. We only report mean Cohen’s d, the percentage of effects with p < 0.05, the
percentage of effects with p < 0.10, and the percentage of p-curves with evidential value if the discipline is associated with at least ten articles.

Table 4, we see that 72.1% of the treatment effect estimates in our data
set involve a nudge that triggers system 1, at least in part (recall that a
given nudge may simultaneously trigger system 1, engage system 2, and
bypass both systems). Mean Cohen’s d for these treatment effect esti­
mates is 0.468, and the percentage of effects with p < 0.05 (p < 0.10) is
63.2% (66.7%).
Beshears and Gino (2015) list three different techniques for trig­
gering system 1: arousing emotions, harnessing biases, and simplifying
the decision-making process.
3.4.2.1. Nudges that arouse emotions. The first technique for triggering
system 1 is to arouse emotions. For example, Beshears, Choi, Laibson,
Madrian, and Zeldes (2014) study framing effects in annuity purchases.
Life annuities transform a lump sum payment into a steady stream of
monthly income that lasts for the rest of the annuitant’s life. When a
hypothetical annuity purchase decision is framed as a choice between
more versus less guaranteed income, survey respondents annuitize more
of their wealth compared to when the decision frame emphasizes the
implications of annuities for flexibility and control over asset allocation
and the timing of spending. Of course, having “more guaranteed in­
come” and having “less flexibility” are two ways of framing the same
fundamental feature of annuities, namely the steady stream of payments
that they provide. Emphasizing one lens or the other invokes different
emotional reactions that sway an individual’s annuity purchase de­
cisions. Table 4 shows that 29.3% of the treatment effect estimates in our
data set are associated with a nudge that arouses emotions.
3.4.2.2. Nudges that harness biases. The second technique for triggering
system 1 is to harness biases. Nudges that harness biases change
behavior by tapping into psychological processes that are known to in­
fluence decision making in a systematic fashion. For example, Beshears,
Dai, Milkman, and Benartzi (2019) test a retirement savings nudge that
harnesses the “fresh start effect,” the tendency of individuals to initiate
the pursuit of virtuous goals at moments that represent the beginning of
a new time period (Dai, Milkman, & Riis, 2014). When individuals
received mailings offering the opportunity to increase retirement plan

9
J. Beshears and H. Kosowsky
contributions at a future point in time that was framed as a new
beginning (the recipient’s birthday, New Year’s, or the first day of
spring), they contributed more compared to when the future point in
time was framed as a control temporal landmark and compared to when
the future point in time was discussed without reference to a temporal
landmark. Table 4 shows that 20.0% of the treatment effect estimates in
our data set are associated with a nudge that harnesses biases.
3.4.2.3. Nudges that simplify the process. The third technique for trig­
gering system 1 is to simplify the process by which decisions are made.
By making it easy to select a particular option from the choice menu, a
nudge can take advantage of the tendency of system 1 to gravitate to­
wards options that involve little up-front effort to implement. For
example, Beshears, Choi, Laibson, and Madrian (2013) apply this tech­
nique to increase retirement savings plan participation. The typical plan
enrollment process is somewhat complex because it often involves
selecting both a contribution rate and an asset allocation for those
contributions. When the process is simplified to be a “yes or no” choice
of whether to enroll with a pre-selected contribution rate and pre-
selected asset allocation, plan participation rates increase by 10–20
percentage points. Here, simplification makes people more likely to take
action. In Table 4, we see that 24.4% of the treatment effect estimates in
our data set are associated with a nudge that simplifies the decision-
making process.
3.4.3. Nudges that engage system 2
We now turn to nudges that operate by engaging system 2—that is,
nudges that change behavior by prompting individuals to initiate a more
methodical decision-making process than would have otherwise taken
place. Table 4 shows that 41.6% of nudge treatment effect estimates in
our data set involve a nudge that engages system 2. Mean Cohen’s d for
these treatment effect estimates is 0.346, and the percentage of effects
with p < 0.05 (p < 0.10) is 60.9% (65.1%).
Beshears and Gino (2015) identify seven techniques for engaging
system 2, and five of these techniques are represented in our data set:
creating opportunities for reflection, prompting planning, inspiring
broader thinking, increasing accountability, and using reminders.8
3.4.3.1. Nudges that create opportunities for reflection. The first tech­
nique for engaging system 2 is to create opportunities for reflection.
Individuals have limited attention, so in some cases it is possible to
promote wise decision making by simply encouraging individuals to
pause for a moment to consider the options available to them. For
example, consider the case of a company that was looking to increase the
number of its employees who were receiving maintenance prescription
medications for chronic conditions, such as high cholesterol, by mail
delivery instead of by in-person pick-up at the pharmacy. Mail delivery
is more cost-effective both for the employer and for the employees. The
company changed from an opt-in policy, under which employees could
elect to receive mail delivery but would otherwise obtain prescriptions
by in-store pick-up, to an active choice policy, under which employees
were eligible for the prescription drug plan only if they indicated affir­
matively whether they wished to use the mail delivery service or wished
to pick up prescriptions at the pharmacy. The financial incentives
associated with mail delivery and in-store pick-up were unchanged, but
the active choice policy called attention to the decision at hand and
increased the percentage of employees choosing home delivery from 6%
to 42%, creating cost savings for the employer and its employees of
approximately $700,000 per year (Beshears, 2016c; Beshears, Rooney,
& Sanford, 2016a, 2016b; Beshears et al., in press-b). In Table 4, we see
that 22.4% of the treatment effect estimates in our data set are
8
The two remaining techniques involve encouraging joint evaluation and
emphasizing disconfirming evidence.
10
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
associated with a nudge that creates opportunities for reflection.
3.4.3.2. Nudges that prompt planning. Other nudges engage system 2 by
prompting individuals to create plans for implementing valuable actions
in the future. The process of thinking through the details of how to enact
plans embeds those plans more firmly in memory, raises awareness of
possible logistical hurdles (which are then more likely to be resolved),
and creates a personal commitment from which deviation is undesirable,
and all of these factors increase the likelihood of following through on
intentions (Beshears, Milkman, & Schwartzstein, 2016). Consistent with
this reasoning, when an employer offers free workplace influenza
vaccination clinics to employees, prompting employees to write down
the date and time when they plan to visit a clinic increases vaccination
rates (Beshears, 2016a, 2016b; Milkman, Beshears, Choi, Laibson, &
Madrian, 2011). Similar results obtain in the domain of preventive
cancer screenings (Dai et al., 2012; Milkman, Beshears, Choi, Laibson, &
Madrian, 2013). Table 4 indicates that 1.9% of the treatment effect es­
timates in our data set are associated with a nudge that prompts
planning.
3.4.3.3. Nudges that inspire broader thinking. Some nudges engage sys­
tem 2 by encouraging individuals to reconsider the lens through which
they are viewing a problem and to contemplate the consequences of
different possible actions within a broader context. A broader decision-
making frame tends to incorporate factors that might not otherwise be
considered, such as long-run implications and possible unintended
consequences. For example, McKenzie and Liersch (2011) demonstrate
that encouraging employees at a company to think about what their
future retirement savings plan balance will be causes 41% of them to
report greater interest in saving more, whereas encouraging employees
to think about their current plan balance causes only 27% of them to
report greater interest in saving more. In Table 4, we see that 14.7% of
the treatment effect estimates in our data set are associated with a nudge
that inspires broader thinking.
3.4.3.4. Nudges that increase accountability. Another technique for
engaging system 2 is to increase the extent to which individuals feel
accountable for their actions. Even if individuals are holding themselves
accountable to some internally generated standard of behavior, and not
being held accountable to a standard imposed by an external party, their
desire to be consistent with their self-professed values and to honor their
promises to themselves can change their behavior. In an experiment
conducted at a hotel, guests who were invited during the check-in pro­
cess to commit to reusing bathroom towels and were given a symbolic
pin to wear for making such a commitment were more likely to reuse
towels than guests who were invited to make a generic commitment to
environmentally friendly behavior or who were not given a symbolic pin
(Baca-Motes, Brown, Gneezy, Keenan, & Nelson, 2013). See Aleksovska,
Schillemans, and Grimmelikhuijsen (2019) for a review of the literature
on accountability. Table 4 shows that 1.2% of the treatment effect es­
timates in our data set are associated with a nudge that increases
accountability.
3.4.3.5. Nudges that use reminders. A final technique for engaging sys­
tem 2 is to remind individuals of the opportunity to take a particular
action. When individuals intend to implement a behavior but have not
yet found a convenient moment for following through, or when in­
dividuals have simply forgotten of their intentions, a reminder can bring
their intentions back to the top of mind and prompt them to take action.
For example, Altmann and Traxler (2014) show that among patients of a
dentist who were due for a check-up, a postcard reminder increased both
the likelihood of making an appointment and the likelihood of
completing an appointment by approximately ten percentage points. For
another example, placing a free workplace influenza vaccination clinic
in a location that employees pass regularly in the course of their day-to-
J. Beshears and H. Kosowsky
day activities increases vaccination rates relative to placing the clinic in
a nearby location that employees do not pass regularly, as encountering
the clinic on the way to some other activity reminds employees to obtain
a vaccination (Beshears, Choi, Laibson, Madrian, & Reynolds, 2016). In
Table 4, we see that 2.6% of the treatment effect estimates in our data set
are associated with a nudge involving a reminder.
3.4.4. Nudges that bypass both systems
When a nudge bypasses both systems, it takes individuals’ actions as
inputs and changes how those actions translate into outcomes. Table 4
shows that 13.8% of nudge treatment effect estimates in our data set
involve a nudge that bypasses both systems. Mean Cohen’s d for these
treatment effect estimates is 0.546, and the percentage of effects with p
< 0.05 (p < 0.10) is 69.9% (77.1%).
Beshears and Gino (2015) identify two subcategories of nudges that
bypass both systems: nudges that set the default and nudges that make
automatic adjustments.
3.4.4.1. Nudges that set the default. In Section 1, we discussed automatic
enrollment in employer-sponsored retirement savings plans, an example
of a nudge that sets the default. By changing the default contribution
rate from zero to a strictly positive percentage of pay, automatic
enrollment changes what happens when an employee does not actively
indicate a preferred contribution rate. Without automatic enrollment,
such an employee does not become a plan participant; under automatic
enrollment, such an employee does start contributing to the plan. The
nudge bypasses both systems in the sense that many individuals
passively accept the default without using either system 1 thinking or
system 2 thinking to consider their options, and automatic enrollment
translates this action (or, more accurately, inaction) into plan partici­
pation instead of non-participation as the outcome. Past research has
documented that automatic enrollment generates large increases in
savings plan participation rates (Beshears et al., 2006, 2008b; Choi et al.,
2002, 2004; Madrian & Shea, 2001). In a meta-analysis of nudges that
set the default, Jachimowicz, Duncan, Weber, and Johnson (2019) find
that these nudges influence outcomes across a wide range of settings. In
Table 4, we see that 13.2% of the treatment effect estimates in our data
set are associated with a nudge that sets the default.
3.4.4.2. Nudges that make automatic adjustments. Whereas a nudge that
sets the default changes the outcome that is implemented when an in­
dividual does not actively select an option, a nudge that makes auto­
matic adjustments changes the outcome that is implemented when an
individual does make an active choice. For example, consider a firm that
offers a traditional before-tax retirement savings account to its em­
ployees. When employees contribute money out of their paychecks to
such an account, the contributions are tax-deductible in the year they
are made, and subsequent withdrawals from the account are taxable.
Now consider what happens if the firm introduces a “Roth” savings ac­
count as one of its retirement offerings. Contributions to a Roth account
are not tax-deductible in the year they are made, but withdrawals from
the account are not taxable. If employees use rules of thumb such as
“save 10% of income” and ignore the tax treatment of savings, the Roth
account increases savings relative to the before-tax account because a
given percentage of gross income contributed to a Roth account, which
is not taxed at withdrawal, translates into greater after-tax savings than
the same percentage of gross income contributed to a before-tax ac­
count, which is taxed at withdrawal. Consistent with this idea, Beshears,
Choi, Laibson, and Madrian (2017b) analyze eleven companies that
introduced Roth retirement accounts and find no evidence that these
accounts changed total contributions. Thus, the Roth accounts bypassed
both systems because many employees seemed not to change their sys­
tem 1 thinking or system 2 thinking; instead, they elected to contribute
the same percentage of gross income, without regard for the tax treat­
ment of contributions. The Roth accounts made automatic adjustments
11
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
to employee outcomes in the sense that the same percentage of gross
income contributed to a Roth account translated into higher effective
(after-tax) savings. Table 4 indicates that 0.6% of the treatment effect
estimates in our data set are associated with a nudge that makes auto­
matic adjustments.
4. Future challenges for the empirical nudge literature
Having documented that there is statistically robust evidence in
favor of the efficacy of nudges across multiple disciplines, domains of
application, research settings, and types of nudges, we turn to a dis­
cussion of challenges that the empirical nudge literature must confront if
it is to further expand its influence on managerial practice and policy
design.
4.1. Comparing categories of nudges based on their impact
To begin our exploration of the categories of nudges that tend to have
the largest impact on outcomes, we use our data set of nudge treatment
effect estimates and study the correlates of a nudge’s effect size as
measured by Cohen’s d. Recall that we cannot calculate Cohen’s d for all
of the observations in our data set because the necessary information is
not always available, but we can calculate Cohen’s d for 507 treatment
effect estimates reported in 101 articles.
Columns 1 and 2 of Table 5 report the results from ordinary least
squares regressions with Cohen’s d as the outcome variable. In column 1,
the explanatory variable of interest is an indicator for whether the nudge
automates some aspect of the decision-making process. The control
variables are indicators for the academic discipline of the article from
which the treatment effect estimate is drawn (see Table 1), indicators for
the domain of application (see Table 2), an indicator for whether the
research setting involves field observation (either via a field experiment
or via observational analysis of a natural experiment involving field
data), and an indicator for whether the outcome variable for which
Cohen’s d is calculated is dichotomous (because the calculation of
Cohen’s d differs for continuous versus dichotomous variables; see
Section 2.3). In the regression, each observation has a weight propor­
tional to the inverse of the number of observations in the regression
associated with the same article. This procedure gives each article equal
weight in the regression. Standard errors are clustered by article. We
find that nudges involving automaticity are associated with a Cohen’s
d that is 0.193 larger (p < 0.05). To put the magnitude of this estimate in
context, Fig. 2 shows predicted values and associated 95% confidence
intervals from the regression holding all right-hand-side variables fixed
at their means except for the indicator for whether the nudge involves
automaticity, which takes a value of either zero or one. With other
variables held at their means, the predicted Cohen’s d for nudges that
involve automaticity is approximately 50% larger than the predicted
Cohen’s d for nudges that do not involve automaticity.
J. Beshears and H. Kosowsky Organizational Behavior and Human Decision Processes 161 (2020) 3–19

Table 5
Regression analysis of effect size and statistical significance.
Cohen’s Cohen’s Indicator for p < Indicator for p < Indicator for p < Indicator for p <
d d 0.05 0.05 0.10 0.10

Nudge uses automaticity 0.193* – 0.139* – 0.155** –

(0.083) – (0.066) – (0.058) –
Nudge triggers system 1 – 0.476** – 0.828** – 0.802**
– (0.095) – (0.126) – (0.150)
Nudge engages system 2 – 0.421** – 0.764** – 0.779**
– (0.135) – (0.138) – (0.160)
Nudge bypasses both – 0.658** – 0.815** – 0.915**
systems – (0.204) – (0.182) – (0.191)
Research setting uses − 0.113 − 0.107 0.006 0.028 0.002 0.012
field observation (0.100) (0.100) (0.063) (0.065) (0.064) (0.065)
Dichotomous outcome − 0.125 − 0.103 – – – –
(0.093) (0.095) – – – –
Indicators for disciplines yes yes yes yes yes yes
Indicators for domains yes yes yes yes yes yes

p-value for hypothesis “triggers system 1” = “engages – 0.484 – 0.368 – 0.727

system 2”
p-value for hypothesis “triggers system 1” = “bypasses both – 0.269 – 0.926 – 0.365
systems”
p-value for hypothesis “engages system 2” = “bypasses both – 0.140 – 0.710 – 0.279
systems”

R-squared 0.187 0.177 0.068 0.062 0.070 0.061

Number of papers 101 101 174 174 174 174
Number of observations N = 507 N = 507 N = 965 N = 965 N = 965 N = 965

This table reports the results of ordinary least squares (OLS) regressions with the outcome variable in the column heading and the explanatory variables in the leftmost
column. When a nudge belongs to more than one category, the variables recording whether the nudge triggers system 1, whether the nudge engages system 2, and
whether the nudge bypasses both systems take fractional values (1/2 and 1/2, or 1/3 and 1/3 and 1/3). The second, fourth, and sixth regressions omit a constant term.
An observation’s weight in the regression is proportional to the inverse of the number of observations associated with the same article. Standard errors are clustered by
article. The symbols +, *, and ** indicate statistical significance at the 10%, 5%, and 1% levels, respectively.

0.9
p < 0.05
0.8

0.7
Predicted Cohen's d

0.6

0.5

0.4

0.3

0.2

0.1

0
Nudges that do not use automaticity Nudges that use automaticity
Fig. 2. Predicted Cohen’s d for nudges that do and do not use automaticity. This figure shows predicted values and 95% confidence intervals from an ordinary least
squares (OLS) regression for which the outcome variable is Cohen’s d. The right-hand-side variables are an indicator for whether the nudge uses automaticity,
indicators for academic discipline, indicators for domain of application, an indicator for whether the research setting involves field observation, and an indicator for
whether the outcome variable is dichotomous. An observation’s weight in the regression is proportional to the inverse of the number of observations associated with
the same article. Standard errors are clustered by article. For the predictions, all right-hand-side variables are held fixed at their means except for the indicator for
whether the nudge uses automaticity, which takes a value of either zero or one.

12
J. Beshears and H. Kosowsky
In column 2 of Table 5, we replace the indicator for whether the
nudge automates some aspect of the decision-making process with
variables capturing the extent to which a nudge falls into the three major
categories identified by Beshears and Gino (2015), namely nudges that
trigger system 1, nudges that engage system 2, and nudges that bypass
both systems.9 The control variables, weighting procedure, and clus­
tering of standard errors are the same as in column 1. Nudges that bypass
both systems tend to have a higher Cohen’s d than nudges that trigger
system 1 and nudges that engage system 2, but these differences are not
statistically significant.
Columns 3 and 4 of Table 5 use the same regression specifications as
columns 1 and 2, respectively, except for two changes. First, the
outcome variable is an indicator for whether the estimated treatment
effect is statistically significant at the 5% level. Second, we drop the
indicator for whether the treatment effect is estimated for a dichoto­
mous variable, which was only relevant for the Cohen’s d regressions
because the calculation of Cohen’s d differed for continuous versus
dichotomous variables. Columns 5 and 6 of Table 5 are identical to
columns 3 and 4, respectively, except the outcome variable is an indi­
cator for whether the estimated treatment effect is statistically signifi­
cant at the 10% level. Columns 3–6 indicate that nudges involving
automaticity are associated with a 13.9 percentage point (15.5 per­
centage point) higher likelihood of having a treatment effect that is
statistically significant at the 5% (10%) level, but there are not sys­
tematic differences in the likelihood of a statistically significant treat­
ment effect across the three major categories of nudges.
The results in Table 5 are robust to alternative specifications. The
results in columns 1 and 2 are nearly identical if we winsorize Cohen’s
d at the 1st and 99th percentiles. Also, for our baseline regressions,
Cohen’s d is treated as missing and dropped from the sample if the
estimated nudge treatment effect was the opposite of the predicted di­
rection and either statistically significant or marginally statistically
significant. However, the results in columns 1 and 2 are similar if we
include those observations in the regression sample either coded as
having a positive value for Cohen’s d or coded as having a Cohen’s d of
zero. In addition, when the nudge treatment effect is estimated for a
dichotomous outcome variable and the mean of the outcome variable for
the control group is unknown, our baseline regressions assume that the
treatment effect is symmetric around 0.5 for the purposes of calculating
Cohen’s d. The results in columns 1 and 2 are similar if we instead drop
these observations from the regression sample. We have also explored
alternative methods of weighting observations. If we assign each
observation a weight proportional to the inverse of the squared standard
error of the Cohen’s d estimate, some observations receive extremely
high weights, but if we winsorize those weights at the 20th and 80th
percentiles, the results in columns 1 and 2 are similar.10 We view equal
9 in the empirical literature on nudging. As indicated in Table 3, 44.0% of
We do not simultaneously include the indicator for whether a nudge in­
volves automaticity as a right-hand-side variable because it is moderately
highly correlated with the three new variables. Because a given nudge can fall
into more than one of the three major categories, we do not use indicator
variables for the categories. If a nudge falls into only one category (e.g., nudges
that trigger system 1), the variable associated with that category takes a value
of one, while the variables associated with the other two categories take values
of zero. If a nudge falls into two categories, the variables associated with those
categories take a value of 1/2, while the variable associated with the remaining
category takes a value of zero. If a nudge falls into all three categories, all three
variables take a value of 1/3. The regression omits a constant term, so it models
a nudge that spans two or three categories as being a simple average of those
categories. As a robustness check, we have estimated regressions that instead
use indicator variables for the three categories, allowing multiple indicators to
take a value of one for the same nudge, and our qualitative conclusions are
unchanged.
10
To calculate these weights, we sometimes must calculate standard errors
based on reported p-values. When a reported p-value is zero, we replace it with
0.001 for this calculation.
13
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
weighting of the nudge treatment effect observations in our data set as
inappropriate because of the disproportionately high weight assigned to
articles that include a large number of small-sample studies. Nonethe­
less, the results in Table 5 are similar if we pursue this strategy, except
that the coefficient estimate on the indicator for using automaticity in
column 1 is no longer statistically significant or marginally statistically
significant (even though the coefficient estimate of 0.160 is close to the
coefficient estimate of 0.193 in Table 5). Finally, in columns 3–6 of
Table 5, logistic regressions deliver the same conclusion as ordinary
least squares regressions.
Overall, the results in Table 5 suggest that nudges involving auto­
maticity have a larger impact, both as measured by Cohen’s d and as
measured by the likelihood of a statistically significant effect, than
nudges that do not involve automaticity. This finding aligns with prior
theorizing that automating some aspect of the decision-making process,
without relying on individuals to act differently, tends to be more
powerful for driving changes in outcomes than encouraging individuals
to change their decision-making process, which requires attention and
effort (Beshears & Gino, 2015). Of course, our conclusions here are
tentative because they are based only on correlational evidence. While
we control for some potential confounding factors, the type of nudge
that was applied in a given setting was not randomly assigned, so the
relationships that we find may be driven by omitted variables, such as
how difficult it is to change individuals’ behavior in that setting.
Moreover, while it is likely that certain types of nudges are consistently
more impactful than other types of nudges regardless of context, a
critical ingredient for impact may also be the match between the type of
nudge and the nature of the behavior to be changed.
It is also important to note that nudges may be beneficial even if they
do not exert an effect on the targeted outcome variable. Nudges that
simplify the choice environment sometimes have this property. For
example, in an experimental study of mutual fund investment decisions,
a nudge that simplified the presentation of information regarding the
funds did not change participants’ investment choices, but participants
needed less time to reach decisions of the same quality, suggesting that
the nudge was valuable (Beshears, Choi, Laibson, & Madrian, 2011).
Finally, when judging the desirability of a nudge, it is not only the
nudge’s effect size that is important, but also the cost of implementing
the nudge. We return to this point in Section 4.5.
Notwithstanding the caveats above, the evidence in Table 5 suggests
that there are important differences in impact across various categories
of nudges. Developing a better understanding of this heterogeneity
would be a valuable pursuit for the empirical literature on nudging.
4.2. Using field research and laboratory research as complementary
methods
Field research and laboratory research have both played major roles
the nudge treatment effects in our data set were estimated using field
experiments and the observational analysis of field data, and 56.0% of
effects were estimated using laboratory-style methods (laboratory ex­
periments, online experiments, and surveys). We argue that research on
nudging is most compelling when it uses field and laboratory approaches
as complementary methods for understanding why and under what
circumstances nudges influence behavior.
Field and laboratory approaches are complementary because they
build on each other. Laboratory techniques are often deployed more
quickly and easily, so they are well suited for exploring the viability of
new nudge interventions that have not been tested previously. Research
in the field can then build on laboratory findings by focusing on nudges
that have shown promise in laboratory settings. This step is important
because even when evidence from laboratory experiments documents
that a given nudge can alter individual decisions, evidence from the field
may reveal that the size of the effect of the nudge diminishes in field
settings, perhaps to the point where it is too small to be of practical use
J. Beshears and H. Kosowsky
or too small to detect. In some cases, the discrepancy between the lab­
oratory and field results may be due to the fact that laboratory experi­
ments can amplify the effect of a nudge by stripping away relevant
contextual factors, which would have otherwise also influenced the
decision at hand. Field research, on the other hand, incorporates those
contextual factors, and those factors may drown out the effect of the
nudge by drawing attention away from the stimuli by which the nudge is
delivered or by triggering alternative ways of thinking about the choice
that compete with the decision-making process promoted by the nudge.
To be clear, it is not a critique of laboratory techniques to point out
that they tend to strip away contextual factors and are less suited for
measuring the size of nudge effects in field settings. The objective of
laboratory-style research is often to demonstrate the existence of an
effect, not to estimate the size of an effect, and the demonstration of an
effect in the absence of contextual factors strengthens the case for the
generalizability of the effect across contexts. Furthermore, when field
data indicate that a nudge has a small effect or no effect (or even has the
opposite of the intended effect) in a particular context, it is often
logistically challenging or prohibitively costly to collect additional field
data that might shed light on the psychological mechanism behind the
result, in part because many nudges deployed in field settings operate
through multiple mechanisms simultaneously (Hauser, Gino, & Norton,
2018). Laboratory approaches, which tend to have more carefully
controlled experimental manipulations and more targeted measurement
of key constructs, are valuable for understanding the decision-making
processes that underlie variation in nudge effects across domains and
for developing predictions regarding the situations in which a nudge will
or will not have the desired effect. We emphasize that the combination of
field and laboratory approaches is necessary for providing guidance to
practitioners who use nudges to address managerial and public policy
problems.
The literature on financial returns aggregation and investment
portfolio decisions is one example of the importance of using both field
and laboratory methods. Some laboratory studies demonstrate that
showing individuals aggregated returns, such as returns aggregated over
a long time horizon instead of a short time horizon, increases in­
dividuals’ willingness to invest in risky assets (Gneezy & Potters, 1997;
Thaler, Tversky, Kahneman, & Schwartz, 1997). These findings are
consistent with the hypothesis that individuals are myopically loss
averse—they evaluate the outcomes of risky investments within a nar­
row frame, such as a short time window, and when they decide whether
or not to invest in a risky asset, a possible loss of a given size within this
frame is weighted more heavily than a gain of the same size within this
frame (Benartzi & Thaler, 1995). Under this hypothesis, aggregating
returns widens the frame, and for most common risky asset return dis­
tributions, a wider frame reduces the likelihood of experiencing a loss,
which in turn increases willingness to invest in risky assets. Despite the
laboratory evidence indicating that an aggregation nudge can change
risk-taking behavior, Beshears, Choi, Laibson, and Madrian (2017a)
show that the nudge has no effect in a setting that uses mutual funds
instead of laboratory gambles and a one-year experimental period
instead of a short laboratory session. This evidence suggests that
embedding the nudge within a field context drowns out the effect of the
nudge. To better understand the mechanisms underlying this result,
Beshears et al. (2017a) conduct follow-up laboratory experiments and
document that changes to the return distribution or to the amount of
time between the moment when a portfolio is chosen and the moment
when returns are viewed eliminates the effect of the nudge. These
findings do not invalidate the hypothesis that myopic loss aversion
drives portfolio choices, but they do suggest that nudges based on this
hypothesis are unlikely to impact portfolio choice in field settings.
The literature on nudges that provide information about peer
behavior is another example of the importance of using both field and
laboratory methods. Field experiments have demonstrated that telling
individuals that a particular behavior is common among their peers can
make those target individuals more likely to engage in the behavior
14
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
themselves. This result has been documented in domains including
residential energy conservation (Allcott, 2011), contributions to an on­
line community (Chen, Harper, Konstan, & Li, 2010), towel reuse in
hotels (Goldstein, Cialdini, & Griskevicius, 2008), voting (Gerber &
Rogers, 2009), job choice (Coffman, Featherstone, & Kessler, 2017),
food consumption (Sparkman & Walton, 2017), tax compliance (Halls­
worth, List, Metcalfe, & Vlaev, 2017; Bott, Cappelen, Sørensen, &
Tungodden, 2020), and fare evasion at train stations (Ayal, Celse, &
Hochman, in press). In the context of an employer-sponsored retirement
savings plan, however, Beshears, Choi, Laibson, Madrian, and Milkman
(2015) show that telling certain non-participating individuals about the
high participation rates of their peers reduces the target individuals’
likelihood of enrolling in the plan, and that this effect is stronger when
the participation rate among peers is (plausibly exogenously) higher.
The effect is driven by individuals who have low incomes compared to
their peers, suggesting that the nudge backfires because it interacts with
individuals’ concerns regarding relative economic standing, triggering
feelings of discouragement and thereby lowering plan enrollment rates.
Additional work, including work in laboratory environments, docu­
ments that peer information nudges may have no effect or may backfire
because individuals’ preferences may not depend on perceived social
norms, because disclosing the low prevalence of an undesirable behavior
may unintentionally make the behavior seem acceptable, because the
reference group whose behavior is reported may be interpreted as dis­
similar from the target individuals, or because individuals may misre­
member the peer information in self-serving ways (Bicchieri & Dimant,
2019; Dimant, van Kleef, & Shalvi, 2020).11 Thus, the combination of
field and laboratory evidence provides a nuanced understanding of
when and how managers and policy makers might successfully use peer
information nudges.
4.3. Studying the long-run effects of nudges
In our data set of nudge treatment effects, 17 out of the 174 articles
collect follow-up data to estimate at least one treatment effect over a
longer time horizon than the initial time horizon used to examine the
impact of the nudge;12 24 out of the 174 articles estimate at least one
treatment effect for an outcome variable that measures the cumulative
impact of a series of actions in a field setting (one type of outcome
variable that captures a long-run effect); and 36 out of the 174 articles
fall in either of the first two categories. Thus, at most 21% of articles
attempt to assess the long-run effect of a nudge. Future empirical
research on nudging should devote more attention to whether and how
nudges exert long-run effects, as nudging is more compelling as a
managerial and policy tool if it can generate long-lasting changes in
outcomes instead of only short-run effects.
One particularly intriguing area for future research is the mecha­
nisms by which nudges can help individuals develop habits that lead to
sustained changes in behavior. Habits are patterns of behavior charac­
terized by repeated automatic performance of an action or set of actions
in response to a routinely occurring cue to act (Wood & Runger, 2016).
Unfortunately, in many situations, encouraging individuals to create
entirely new routines for engaging in a desirable behavior is unlikely to
succeed in promoting habit formation because such new routines are
frequently disrupted by competing demands on individuals’ time and
11
The possibility of backfiring is not unique to nudges that provide infor­
mation about peer behavior. See, for example, Brown, Johnstone, Haščič, Vong,
and Barascud (2013), Ascarza, Iyengar, and Schleicher (2016), and Bolton,
Dimant, and Schmidt (2020).
12
See the Online Appendix for details on our method for determining whether
the researchers collected follow-up data. For five of the 17 articles, there is
ambiguity, but we include these articles in our count to be conservative. Out of
the other 12 articles, only two find no evidence of a treatment effect that
persists over the longer time horizon.
J. Beshears and H. Kosowsky
attention (Beshears, Lee, Milkman, Mislavsky, & Wisdom, 2019). Suc­
cess may require devising routines that dovetail with existing recurring
events in individuals’ lives, but this hypothesis deserves further study.
Given the challenges of helping individuals develop beneficial
habits, another promising direction for future research is to explore how
nudges can prompt individuals to make one-time, up-front investments
that generate a long series of future changes in outcomes. For example,
Brandon et al. (2017) document that a nudge showing individuals how
their home energy consumption compares to their neighbors’ con­
sumption induces investment in energy-saving technologies, such as
energy-efficient appliances. These one-time investments lead to persis­
tent reductions in energy use.
Of course, even if a nudge has a short-run effect but not a long-run
effect, it may nonetheless be valuable if it can be applied repeatedly
and continues to have a short-run effect each time it is applied. Some
past work has identified nudges that, upon repeated application, have an
effect each time they are applied (see, e.g., Allcott & Rogers, 2014;
Altmann & Traxler, 2014; Beshears et al., 2013), but additional work on
this issue would be valuable.
4.4. Measuring the effect of nudges on non-targeted outcomes
Out of the 174 articles in our data set of nudge treatment effects, only
12 measure the effect of the nudge on an outcome variable that could
offset the treatment effect on the focal outcome variable and that is in
the same domain as the focal outcome variable. Only three articles
measure the effect of the nudge on an outcome variable that could offset
the treatment effect on the focal outcome variable and that is in a
different domain from the focal outcome variable. There are strong
theoretical reasons to believe that nudges can impact outcomes other
than the targeted outcomes. Nudges often operate by changing the
choice architecture of one decision-making setting to prompt individuals
to change their behavior in that setting. Nudges do little to shift the
fundamental costs and benefits of various possible courses of action, so
outside of the setting with altered choice architecture, individuals may
engage in behavior that compensates for the changes induced by the
nudge.
Investigations of nudge effects on non-targeted outcomes do not al­
ways find evidence of compensatory behavior. Beshears, Choi, Laibson,
Madrian, and Skimmyhorn (2019) study the effects of automatic
enrollment into an employer-sponsored retirement savings plan on both
targeted and non-targeted outcome variables. Consistent with prior
research, they find that automatic enrollment increases contributions to
the savings plan, the targeted outcome. One concern with automatic
enrollment is that it might also have the unintended consequence of
increasing financial distress if individuals have more retirement savings
but diminished financial resources for repaying debt. However,
Beshears, Choi, Laibson, et al. (2019) do not find that automatic
enrollment increases financial distress or credit card and other non-
secured debt.13
In other situations, however, nudges have effects on non-targeted
outcomes that undermine their effects on targeted outcomes. For
example, Wisdom, Downs, and Loewenstein (2010) study interventions
designed to decrease caloric intake at a fast-food sandwich chain. One
nudge intervention decreased the convenience of ordering high-calorie
sandwiches and thereby reduced the likelihood with which partici­
pants selected high-calorie sandwiches. An unintended consequence was
that the nudge simultaneously increased the caloric content of the side
dishes and drinks selected by participants, perhaps due to participants’
13
Beshears, Choi, Laibson, et al. (2019) do find suggestive evidence that
automatic enrollment increases auto debt and first mortgage debt. Because
increases in these types of secured debt are associated with asset purchases,
these findings do not necessarily imply decreases in household net worth, but
future research should investigate these issues further.
15
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
feelings that their virtuous sandwich choices should be rewarded with
indulgent side dish and drink choices. The increase in caloric intake
from side dishes and drinks entirely offset the decrease in caloric intake
from sandwiches.
The dearth of attempts to gauge the effects of nudges on non-targeted
outcomes is a glaring omission in the empirical nudge literature, as such
unintended consequences can partially offset, entirely eliminate, or even
reverse the benefits that nudges deliver on a targeted dimension. In
addition to documenting effects on non-targeted outcomes, future
research on nudging should attempt to understand the circumstances
under which nudging does and does not induce compensatory behavior
that undermines the intended impact of the nudge.
4.5. Placing a nudge in the context of other nudges and other
interventions
Managers and policy makers often deploy multiple interventions
simultaneously. Empirical research on nudging must therefore help
managers and policy makers understand how a given nudge fits within a
broader constellation of strategies for changing behavior. Towards this
end, Benartzi et al. (2017) point out that an important metric by which
nudges should be judged is their impact relative to their cost. These
authors show across four policy domains that relative to traditional in­
terventions such as financial incentives, nudges tend to deliver greater
impact on targeted behaviors per dollar spent. Going forward, evalua­
tions of nudge interventions should place greater emphasis on calcu­
lating the costs associated with nudges in order to facilitate comparisons
for managers and policy makers who are choosing among interventions.
Furthermore, it is important for managers and policy makers to have
evidence on the interactive effects among nudge interventions and other
interventions. Out of the 174 articles in our data set of nudge treatment
effect estimates, 48 articles (27.6%) feature at least one test of the
interaction of a nudge with some other intervention (whether the other
intervention is a nudge or not). More research along these lines is needed
to provide guidance as to the optimal mix of interventions for achieving
managerial and policy objectives.
In some cases, nudges act as complements to other interventions. For
example, a nudge that streamlines the process of applying for financial
aid for attending college is complementary to existing financial aid
programs and can increase college enrollment (Bettinger, Long, Oreo­
poulos, & Sanbonmatsu, 2012). In other cases, nudges are substitutes for
other interventions. For example, in employer-sponsored retirement
savings plans, both automatic enrollment and employer matching con­
tributions (contributions from the employer that are deposited in
employee accounts contingent upon employees’ own contributions) are
intended to increase plan participation. These two interventions are
likely to be substitutes. In the absence of automatic enrollment, an
employer match of $0.25 per dollar of employee contributions is esti­
mated to increase plan participation rates by approximately 10 per­
centage points relative to not offering a match (Papke, 1995; Basset,
Fleming, & Rodrigues, 1998).14 When a plan automatically enrolls
employees, an employer match of $0.25 per dollar of employee contri­
butions is estimated to increase plan participation rates by no more than
5–6 percentage points relative to not offering a match (Beshears, Choi,
Laibson, & Madrian, 2010), an increase that is substantial but smaller
than the increase in the absence of automatic enrollment.
An understanding of the extent to which interventions serve as
complements or substitutes for each other is a valuable input to mana­
gerial and policy decisions. For example, in the case of retirement sav­
ings plans, employers with automatic enrollment may wish to direct
financial resources away from employer matching contributions, a
14
Estimates of this treatment effect vary widely, with some estimates as low as
5 percentage points (Engelhardt & Kumar, 2007) and others as high as 33
percentage points (Even & Macpherson, 2005).
J. Beshears and H. Kosowsky
substitute for automatic enrollment, towards other programs that pro­
mote retirement security, such as employer contributions to employee
accounts that are not contingent on employee contributions.
5. General discussion and conclusions
In this paper, we discussed examples of past research on nudging and
analyzed a data set of 174 articles in the empirical literature on nudging
in order to assess progress to date in this literature and to highlight key
challenges for future research on the topic. We documented that many
types of nudges, as studied by scholars in several different academic
disciplines examining data in field and laboratory settings, have suc­
ceeded in changing behavior in a wide range of domains of application.
We argued that future research on nudging should place greater
emphasis on (1) seeking to identify the types of nudges that tend to be
most impactful, (2) using field-based methods and laboratory-based
methods as complementary approaches, (3) examining the long-run ef­
fects of nudges, (4) considering the effects of nudges on non-targeted
outcomes, and (5) studying nudges within the context of a suite of
nudges and other interventions designed to change a particular set of
behaviors. Many organizations have started to use nudging as a tech­
nique for changing behavior, but in order for nudging to take its place
alongside traditional interventions (such as financial incentives) in the
standard toolkit of managers and policy makers, research on nudging
will need to devote much more attention to the five issues above.
As discussed in Section 4, results from the handful of research articles
that have begun to address the five issues outlined above suggest that
some nudges that are currently considered powerful methods for
changing behavior may prove to be less effective than previously
thought. For example, nudges that have a large short-run impact may
have a negligible long-run impact, necessitating repeated exposure to
the nudge in order to generate a meaningful cumulative impact. Nudges
that succeed in changing a targeted behavior may simultaneously induce
offsetting changes in non-targeted behaviors.
Even when nudges are deemed effective after they have been eval­
uated in a more comprehensive fashion, it is important to note that their
impact on outcomes is often modest. Consider two of the most widely
known nudges: automatic enrollment in an employer-sponsored retire­
ment savings plan and personalized reports comparing the recipient’s
energy consumption to the energy consumption of the recipient’s
neighbors. When Beshears, Choi, Laibson, et al. (2019) evaluated the
implementation of automatic enrollment at a large employer, they
estimated that automatic enrollment increased cumulative retirement
plan contributions during the first four years after an employee’s hire
date by only 4.1% of first-year annualized salary, a measurable change
but not one that on its own can eliminate the risk that a household will
experience a drop in its standard of living at retirement (Munnell, Hou,
& Sanzenbacher, 2018). Similarly, Allcott (2011) estimated that
personalized reports containing information about neighbors’ electricity
utilization reduced energy consumption by 2%, a notable effect but not
one that on its own can reduce carbon emissions to the point where the
consequences of global climate change are entirely mitigated.
The observation that the effects of nudges tend to be modest does not
imply that psychological factors are unimportant in determining
behavior. To the contrary, the fact that it is difficult to change behavior
in, for example, the retirement savings domain is consistent with a
pervasive role for psychological factors—a nudge that boosts savings in
one narrow context is easily swamped by the operation of psychological
factors driving overspending in all other contexts that an individual
encounters. Nor does the observation that the effects of nudges tend to
be modest imply that managers and policy makers should abandon the
choice architecture approach to influencing behavior. No single policy
or intervention should be expected to resolve a major societal problem
on its own—substantial progress requires many policies and in­
terventions with modest effects all pushing in the right direction.
Furthermore, nudges may have modest effects, but they also have small
16
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
costs of implementation. On the basis of impact per unit of cost incurred,
nudges are in fact highly valuable tools for changing behavior (Benartzi
et al., 2017).
The modest effect sizes of most nudges do imply, however, that it
would be unwise to ignore the possibility of implementing other types of
interventions merely because a nudge has been deployed to address a
given problem (Bhargava & Loewenstein, 2015; Hagmann, Ho, & Loe­
wenstein, 2019; Loewenstein & Chater, 2017). In moving beyond
nudges, of course, managers and policy makers must be prepared to
accept interventions that fall short of the nudge definition of influencing
behavior without limiting choice or meaningfully changing financial
incentives. In some cases, even outright prohibition of certain options
may be appropriate. Consider the case of restrictions on pre-retirement
withdrawals from retirement savings accounts. In the United States,
such withdrawals are widely available in exchange for payment of a 10%
tax penalty, and they are available penalty-free under special circum­
stances. In other countries, such as the United Kingdom, such with­
drawals are prohibited entirely except in extreme situations (Beshears,
Choi, Hurwitz, Laibson, & Madrian, 2015). If the population includes
both individuals who have self-control problems that cause them to
undersave and individuals who do not have self-control problems
(Augenblick, Niederle, & Sprenger, 2015; Beshears et al., 2020), simu­
lations indicate that a policy prohibiting pre-retirement withdrawals
leads to large welfare gains for the individuals with self-control prob­
lems and very small welfare losses for the individuals without self-
control problems, a tradeoff that a policy maker may be willing to
make (Beshears, Choi, Clayton, et al., 2019).
When managers and policy makers contemplate the range of possible
interventions they might implement, they should consider how the in­
terventions affect consumer welfare, a challenging question when in­
dividuals’ actions do not necessarily reflect their best interests (see
Allcott & Kessler, 2019; Beshears et al., 2008a; Bernheim & Rangel,
2009; Bernheim, Fradkin, & Popov, 2015). A further challenge is that
while nudges are expressly intended as tools for improving individual
welfare, managers and policy makers who embrace this role for nudges
must recognize that profit-maximizing firms may have an incentive to
exploit individuals’ biases for their own gain (see Baker & Wurgler,
2013; Beshears, Gino, Lee, & Wang, 2016; Heidhues & Koszegi, 2018;
and Malmendier, 2018). The designs of interventions should anticipate
and account for the ways in which such firms may undermine efforts to
improve individual welfare. A complete discussion of formal welfare
analysis and strategic interactions with profit-maximizing firms is
beyond the scope of this paper, but we briefly highlight a key ethical
tension that emerges when contrasting nudge interventions and tradi­
tional interventions such as financial incentives. On one hand, tradi­
tional interventions may be attractive because they are transparent in
their attempts to influence individuals’ decisions, whereas nudge in­
terventions, especially those that trigger system 1 or bypass both sys­
tems, influence individuals’ decisions in ways that those individuals may
not fully recognize and understand. On the other hand, nudge in­
terventions may be attractive because they help individuals who would
otherwise have difficulty making wise decisions and because they do not
restrict the choices of individuals who make wise decisions on their own
(Camerer et al., 2003; Thaler & Sunstein, 2003, 2008). Traditional in­
terventions, in contrast, often restrict the choices of the latter group in
order to help the former group. This tension is one that managers, policy
makers, and society at large must grapple with.
In summary, the empirical literature on nudging has established that
choice architecture techniques can succeed in changing behavior in
many managerial and policy-relevant settings. This paper has outlined
future directions that research in this area should pursue in order to
make nudges part of the standard toolkit of managers and policy makers.
Acknowledgments
This research was supported by the National Institutes of Health
J. Beshears and H. Kosowsky
(grant P30AG034532), the Pershing Square Fund for Research on the
Foundations of Human Behavior, and Harvard Business School. We
thank Max Bazerman, James Choi, Francesca Gino, Brian Hall, David
Laibson, George Loewenstein, Brigitte Madrian, Deepak Malhotra,
Kathleen McGinn, Katherine Milkman, Mario Small, three anonymous
reviewers, and participants in the WW Roundtable Discussion on
Creating Habit Formation for Healthy Behaviors, hosted by the Center
for Health Incentives and Behavioral Economics and the Behavior
Change for Good Initiative, for helpful comments. We are grateful for the
research assistance of Alicia Zhang. Beshears has received additional
grant support from the TIAA Institute and the National Employment
Savings Trust (NEST); is a TIAA Institute Fellow; has received research
data from Alight Solutions, Voya Financial, and the Commonwealth
Bank of Australia; and is an advisor to and equity holder in Nutmeg
Saving and Investment, a robo-advice asset management company. See
his website for a complete list of outside activities. The views expressed
here are those of the authors and do not reflect the views or position of
any agency of the federal government, Harvard University, or the Na­
tional Bureau of Economic Research.
Appendix A. Supplementary material
Supplementary data to this article can be found online at https://doi.
org/10.1016/j.obhdp.2020.09.001.
References
Afriat, S. N. (1967). The construction of utility functions from expenditure data.
International Economic Review, 8, 67–77.
Aleksovska, M., Schillemans, T., & Grimmelikhuijsen, S. (2019). Lessons from five
decades of experimental and behavioral research on accountability: A systematic
literature review. Journal of Behavioral Public Administration, 2(2), 1–18.
Allcott, H. (2011). Social norms and energy conservation. Journal of Public Economics, 95,
1082–1095.
Allcott, H., & Kessler, J. B. (2019). The welfare effects of nudges: A case study of energy
use social comparisons. American Economic Journal: Applied Economics, 11, 236–276.
Allcott, H., & Rogers, T. (2014). The short-run and long-run effects of behavioral
interventions: Experimental evidence from energy conservation. American Economics
Review, 104, 3003–3037.
Altmann, S., & Traxler, C. (2014). Nudges at the dentist. European Economic Review, 72,
19–38.
Angeletos, G.-M., Laibson, D., Repetto, A., Tobacman, J., & Weinberg, S. (2001). The
hyperbolic consumption model: Calibration, simulation, and empirical evaluation.
Journal of Economic Perspectives, 15(3), 47–68.
Ascarza, E., Iyengar, R., & Schleicher, M. (2016). The perils of proactive churn
prevention using plan recommendations: Evidence from a field experiment. Journal
of Marketing Research, 53, 46–60.
Augenblick, N., Niederle, M., & Sprenger, C. (2015). Working over time: Dynamic
inconsistency in real effort tasks. Quarterly Journal of Economics, 130, 1067–1115.
Ayal, S., Celse, J., & Hochman, G. (2020). Crafting messages to fight dishonesty: A field
investigation of the effects of social norms and watching eye cues on fare evasion.
Organizational Behavior and Human Decision Processes (in press). https://doi.org/
10.1016/j.obhdp.2019.10.003.
Baca-Motes, K., Brown, A., Gneezy, A., Keenan, E. A., & Nelson, L. D. (2013).
Commitment and behavior change: Evidence from the field. Journal of Consumer
Research, 39, 1070–1084.
Baker, M., & Wurgler, J. (2013). Behavioral corporate finance: an updated survey. In G.
M. Constantinides, M. Harris, & R. M. Stulz (Eds.), Handbook of the economics of
finance (Vol. 2, Part A, pp. 357-424). Amsterdam: Elsevier Press.
Bassett, W. F., Fleming, M. J., & Rodrigues, A. P. (1998). How workers use 401(k) plans:
The participation, contribution, and withdrawal decisions. National Tax Journal, 51,
263–289.
Bazerman, M. H., & Moore, D. A. (2012). Judgment in managerial decision making (8th
ed.). Hoboken, NJ: Wiley.
Benartzi, S., Beshears, J., Milkman, K. L., Sunstein, C. R., Thaler, R. H., Shankar, M., …
Galing, S. (2017). Should governments invest more in nudging? Psychological Science,
28, 1041–1055.
Benartzi, S., & Thaler, R. H. (1995). Myopic loss aversion and the equity premium puzzle.
Quarterly Journal of Economics, 110, 73–92.
Bernheim, B. D., Fradkin, A., & Popov, I. (2015). The welfare economics of default
options in 401(k) plans. American Economic Review, 105, 2798–2837.
Bernheim, B. D., & Rangel, A. (2009). Beyond revealed preference: Choice-theoretic
foundations for behavioral welfare economics. Quarterly Journal of Economics, 124,
51–104.
Beshears, J. (2016a). Evive Health and workplace influenza vaccinations. Harvard
Business School case 916-044.
Beshears, J. (2016b). Evive Health and workplace influenza vaccinations. Harvard
Business School teaching note 916-049.
17
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
Beshears, J. (2016c). Express Scripts: Promoting prescription drug home delivery (A) and
(B). Harvard Business School teaching note 916-047.
Beshears, J., Choi, J. J., Clayton, C., Harris, C., Laibson, D., & Madrian, B. C. (2019).
Optimal illiquidity. Working paper.
Beshears, J., Choi, J. J., Hurwitz, J., Laibson, D., & Madrian, B. C. (2015). Liquidity in
retirement savings systems: An international comparison. American Economic Review
Papers and Proceedings, 105, 420–425.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2006). Retirement saving: Helping
employees help themselves. Milken Institute Review, 8(3), 30–39.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2008a). How are preferences
revealed? Journal of Public Economics, 92, 1787–1794.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2008b). The importance of default
options for retirement saving outcomes: Evidence from the United States. In
S. J. Kay, & T. Sinha (Eds.), Lessons from pension reform in the Americas (pp. 59–87).
Oxford: Oxford University Press.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2010). The impact of employer
matching on savings plan participation under automatic enrollment. In D. A. Wise
(Ed.), Research findings in the economics of aging (pp. 311–327). Chicago, IL:
University of Chicago Press.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2011). How does simplified
disclosure affect individuals’ mutual fund choices? In D. A. Wise (Ed.), Explorations
in the economics of aging (pp. 75–96). Chicago, IL: University of Chicago Press.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2013). Simplification and saving.
Journal of Economic Behavior and Organization, 95, 130–145.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2017a). Does aggregated returns
disclosure increase portfolio risk taking? Review of Financial Studies, 30, 1971–2005.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2017b). Does front-loading
taxation increase savings? Evidence from Roth 401(k) introductions. Journal of
Public Economics, 151, 84–95.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2018). Behavioral household
finance. In B. D. Bernheim, S. DellaVigna, & D. Laibson (Eds.), Handbook of
behavioral economics – Foundations and applications 1 (Vol. 1, pp. 177–276).
Amsterdam: Elsevier Press.
Beshears, J., Choi, J. J., Harris, C., Laibson, D., Madrian, B. C., & Sakong, J. (2020).
Which early withdrawal penalty attracts the most deposits to a commitment savings
account? Journal of Public Economics, 183, article 104144.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2020). Active choice, implicit
defaults, and the incentive to choose. Organizational Behavior and Human Decision
Processes (in press).
Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C., & Milkman, K. L. (2015). The effect
of providing peer information on retirement savings decisions. Journal of Finance, 70,
1161–1201.
Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C., & Reynolds, G. I. (2016). Vaccination
rates are associated with functional proximity but not base proximity of vaccination
clinics. Medical Care, 54, 578–583.
Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C., & Skimmyhorn, W. L. (2019).
Borrowing to save? The impact of automatic enrollment on debt. Working paper.
Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C., & Weller, B. (2010). Public policy
and saving for retirement: The “autosave” features of the Pension Protection Act of
2006. In J. J. Siegfried (Ed.), Better living through economics (pp. 274–290).
Cambridge, MA: Harvard University Press.
Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C., & Zeldes, S. P. (2014). What makes
annuitization more appealing? Journal of Public Economics, 116, 2–16.
Beshears, J., Dai, H., Milkman, K. L., & Benartzi, S. (2019). Using fresh starts to nudge
increased retirement savings. Working paper.
Beshears, J., & Gino, F. (2015). Leaders as decision architects: Structure your
organization’s work to encourage wise choices. Harvard Business Review, 93(5),
52–62.
Beshears, J., Gino, F., Lee, J., & Wang, S. (2016). T-Mobile in 2013: The Un-Carrier.
Harvard Business School case 916-043.
Beshears, J., Lee, H. N., Milkman, K. L., Mislavsky, R., & Wisdom, J. (2019). Creating
exercise habits using incentives: The tradeoff between flexibility and routinization.
Working paper.
Beshears, J., & Milkman, K. L. (2011). Do sell-side stock analysts exhibit escalation of
commitment? Journal of Economic Behavior and Organization, 77, 304–317.
Beshears, J., Milkman, K. L., & Schwartzstein, J. (2016). Beyond beta-delta: The
emerging economics of personal plans. American Economic Review Papers and
Proceedings, 106, 430–434.
Beshears, J., Rooney, P., & Sanford, J. (2016a). Express Scripts: Promoting prescription
drug home delivery (A). Harvard Business School case 916-026.
Beshears, J., Rooney, P., & Sanford, J. (2016b). Express Scripts: Promoting prescription
drug home delivery (B). Harvard Business School case 916-040.
Bettinger, E. P., Long, B. T., Oreopoulos, P., & Sanbonmatsu, L. (2012). The role of
application assistance and information in college decisions: Results from the H&R
block FAFSA experiment. Quarterly Journal of Economics, 127, 1205–1242.
Bhargava, S., & Loewenstein, G. (2015). Behavioral economics and public policy 102:
Beyond nudging. American Economic Review Papers and Proceedings, 105, 396–401.
Bicchieri, C., & Dimant, E. (2019). Nudging with care: The risks and benefits of social
information. Public Choice.
Bolton, G., Dimant, E., & Schmidt, U. (2020). When a nudge backfires: Combining (im)
plausible deniability with social and economic incentives to promote behavioral
change. CESifo Working Paper No. 8070.
Bordalo, P., Gennaioli, N., & Shleifer, A. (2012). Salience theory of choice under risk.
Quarterly Journal of Economics, 127, 1243–1285.
Bott, K. M., Cappelen, A. W., Sørensen, E.Ø., & Tungodden, B. (2020). You’ve got mail: A
randomized field experiment on tax evasion. Management Science, 66, 2801–2819.
J. Beshears and H. Kosowsky
Brandon, A., Ferraro, P. J., List, J. A., Metcalfe, R. D., Price, M. K., & Rundhammer, F.
(2017). Do the effects of social nudges persist? Theory and evidence from 38 natural
field experiments. Working paper.
Brown, Z., Johnstone, N., Haščič, I., Vong, L., & Barascud, F. (2013). Testing the effect of
defaults on the thermostat settings of OECD employees. Energy Economics, 39,
128–134.
Bushong, B., Rabin, M., & Schwartzstein, J. (2019). A model of relative thinking.
Working paper.
Busse, M. R., Pope, D. G., Pope, J. C., & Silva-Risso, J. (2015). The psychological effect of
weather on car purchases. Quarterly Journal of Economics, 130, 371–414.
Camerer, C., & Lovallo, D. (1999). Overconfidence and excess entry: An experimental
approach. American Economic Review, 89, 306–318.
Camerer, C., Babcock, L., Loewenstein, G., & Thaler, R. (1997). Labor supply of New York
City cabdrivers: One day at a time. Quarterly Journal of Economics, 112, 407–441.
Camerer, C., Issacharoff, S., Loewenstein, G., O’Donoghue, T., & Rabin, M. (2003).
Regulation for conservatives: Behavioral economics and the case for “asymmetric
paternalism”. University of Pennsylvania Law Review, 151, 1211–1254.
Center for Retirement Initiatives. (2019). State-facilitated retirement savings programs: A
snapshot of program design features. Washington, DC: Georgetown University.
Chen, Y., Harper, F. M., Konstan, J., & Li, S. X. (2010). Social comparisons and
contributions to online communities: A field experiment on MovieLens. American
Economic Review, 100, 1358–1398.
Chernev, A., Böckenholt, U., & Goodman, J. (2015). Choice overload: A conceptual
review and meta-analysis. Journal of Consumer Psychology, 25, 333–358.
Chetty, R., Looney, A., & Kroft, K. (2009). Salience and taxation: Theory and evidence.
American Economic Review, 99, 1145–1177.
Choi, J. J., Laibson, D., Madrian, B. C., & Metrick, A. (2002). Defined contribution
pensions: Plan rules, participant decisions, and the path of least resistance. In
J. M. Poterba (Ed.), Tax policy and the economy (Vol. 16, pp. 67–114). Cambridge,
MA: MIT Press.
Choi, J. J., Laibson, D., Madrian, B. C., & Metrick, A. (2004). For better or for worse:
Default effects and 401(k) savings behavior. In D. A. Wise (Ed.), Perspectives on the
economics of aging (pp. 81–121). Chicago, IL: University of Chicago Press.
Cialdini, R. B., & Goldstein, N. J. (2004). Social influence: Compliance and conformity.
Annual Review of Psychology, 55, 591–621.
Coffman, L. C., Featherstone, C. R., & Kessler, J. B. (2017). Can social information affect
what job you choose and keep? American Economic Journal: Applied Economics, 9,
96–117.
Dai, H., Milkman, K. L., & Riis, J. (2014). The fresh start effect: Temporal landmarks
motivate aspirational behavior. Management Science, 60, 2563–2582.
Dai, J., Milkman, K. L., Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2012).
Planning prompts as a means of increasing rates of immunization and preventive
screening. Public Policy and Aging Report, 22(4), 16–19.
DellaVigna, S. (2009). Psychology and economics: Evidence from the field. Journal of
Economic Literature, 47, 315–372.
DellaVigna, S., & Linos, E. (2020). RCTs to scale: Comprehensive evidence from two
nudge units. Working paper.
DellaVigna, S., & Malmendier, U. (2006). Paying not to go to the gym. American Economic
Review, 96, 694–719.
Dimant, E., van Kleef, G. A., & Shalvi, S. (2020). Requiem for a nudge: Framing effects in
nudging honesty. Journal of Economic Behavior & Organization, 172, 247–266.
Engelhardt, G. V., & Kumar, A. (2007). Employer matching and 401(k) saving: Evidence
from the health and retirement study. Journal of Public Economics, 91, 1920–1943.
Even, W. E., & Macpherson, D. A. (2005). The effects of employer matching in 401(k)
plans. Industrial Relations, 44, 525–549.
Fehr, E., & Goette, L. (2007). Do workers work more if wages are high? Evidence from a
randomized field experiment. American Economic Review, 97, 298–317.
Gabaix, X. (2014). A sparsity-based model of bounded rationality. Quarterly Journal of
Economics, 129, 1661–1710.
Gabaix, X. (2019). Behavioral inattention. In B. D. Bernheim, S. DellaVigna, & D. Laibson
(Eds.), Handbook of behavioral economics – Foundations and applications 2 (Vol. 2, pp.
261–343). Amsterdam: Elsevier Press.
Gerber, A. S., & Rogers, T. (2009). Descriptive social norms and motivation to vote:
Everybody’s voting and so should you. Journal of Politics, 71, 178–191.
Gneezy, U., & Potters, J. (1997). An experiment on risk taking and evaluation periods.
Quarterly Journal of Economics, 112, 631–645.
Goldstein, N. J., Cialdini, R. B., & Griskevicius, V. (2008). A room with a viewpoint:
Using social norms to motivate environmental conservation in hotels. Journal of
Consumer Research, 35, 472–482.
Hagmann, D., Ho, E. H., & Loewenstein, G. (2019). Nudging out support for a carbon tax.
Nature Climate Change, 9, 484–489.
Hallsworth, M., List, J. A., Metcalfe, R. D., & Vlaev, I. (2017). The behavioralist as tax
collector: Using natural field experiments to enhance tax compliance. Journal of
Public Economics, 148, 14–31.
Halpern, D. (2015). Inside the nudge unit: How small changes can make a big difference. New
York, NY: Random House.
Hauser, O. P., Gino, F., & Norton, M. I. (2018). Budging beliefs, nudging behavior. Mind
& Society, 17, 15–26.
Heidhues, P., & Koszegi, B. (2018). Behavioral industrial organization. In B. D. Bernheim,
S. DellaVigna, & D. Laibson (Eds.), Handbook of behavioral economics – Foundations
and applications 1 (Vol. 1, pp. 517–612). Amsterdam: Elsevier Press.
Jachimowicz, J. M., Duncan, S., Weber, E. U., & Johnson, E. J. (2019). When and why
defaults influence decisions: A meta-analysis of default effects. Behavioural Public
Policy, 3, 159–186.
18
Organizational Behavior and Human Decision Processes 161 (2020) 3–19
Johnson, E. J., Shu, S. B., Dellaert, B. G. C., Fox, C., Goldstein, D. G., Häubl, G., …
Weber, E. U. (2012). Beyond nudges: Tools of a choice architecture. Marketing
Letters, 23, 487–504.
Kahneman, D. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.
Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of
representativeness. Cognitive Psychology, 3, 430–454.
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk.
Econometrica, 47, 263–292.
Koszegi, B., & Rabin, M. (2006). A model of reference-dependent preferences. Quarterly
Journal of Economics, 121, 1133–1165.
Lacetera, N., Pope, D. G., & Sydnor, J. R. (2012). Heuristic thinking and limited attention
in the car market. American Economic Review, 102, 2206–2236.
Laibson, D. (1997). Golden eggs and hyperbolic discounting. Quarterly Journal of
Economics, 112, 443–478.
Lerner, J. S., Small, D. A., & Loewenstein, G. (2004). Heart strings and purse strings:
Carryover effects of emotions on economic decisions. Psychological Science, 15,
337–341.
Loewenstein, G., & Chater, N. (2017). Putting nudges in perspective. Behavioural Public
Policy, 1, 26–53.
Loewenstein, G., & Prelec, D. (1992). Anomalies in intertemporal choice: Evidence and
an interpretation. Quarterly Journal of Economics, 107, 573–597.
Ly, K., Mažar, N., Zhao, M., & Soman, D. (2013). A practitioner’s guide to nudging.
University of Toronto Rotman School of Management Research Report Series.
Madrian, B. C., & Shea, D. F. (2001). The power of suggestion: Inertia in 401(k)
participation and savings behavior. Quarterly Journal of Economics, 116, 1149–1187.
Malmendier, U. (2018). Behavioral corporate finance. In B. D. Bernheim, S. DellaVigna,
& D. Laibson (Eds.), Handbook of behavioral economics – Foundations and applications
1 (Vol. 1, pp. 277–380). Amsterdam: Elsevier Press.
Malmendier, U., & Tate, G. (2005). CEO overconfidence and corporate investment.
Journal of Finance, 60, 2661–2700.
McKenzie, C. R. M., & Liersch, M. J. (2011). Misunderstanding savings growth:
Implications for retirement savings behavior. Journal of Marketing Research, 48,
S1–S13.
McKenzie, C. R. M., Liersch, M. J., & Finkelstein, S. R. (2006). Recommendations implicit
in policy defaults. Psychological Science, 17, 414–420.
Milkman, K. L., & Beshears, J. (2009). Mental accounting and small windfalls: Evidence
from an online grocer. Journal of Economic Behavior and Organization, 71, 384–394.
Milkman, K. L., Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2013). Planning
prompts as a means of increasing preventive screening rates. Preventive Medicine, 56,
92–93.
Milkman, K. L., Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2011). Using
implementation intentions prompts to enhance influenza vaccination rates.
Proceedings of the National Academy of Sciences of the United States of America, 108,
10415–10420.
Munnell, A. H., Hou, W., & Sanzenbacher, G. T. (2018). National Retirement Risk Index
shows modest improvement in 2016. Center for Retirement Research at Boston College
Brief, 18–21.
O’Donoghue, T., & Rabin, M. (1999). Doing it now or later. American Economic Review,
89, 103–124.
O’Donoghue, T., & Rabin, M. (2001). Choice and procrastination. Quarterly Journal of
Economics, 116, 121–160.
Odean, T. (1998). Are investors reluctant to realize their losses? Journal of Finance, 53,
1775–1798.
Odean, T. (1999). Do investors trade too much? American Economic Review, 89,
1279–1298.
OECD Research (2018). Behavioural insights and public policy: Institutions applying BI
to public policy around the world. OECD website, https://www.oecd.org/gov/reg
ulatory-policy/behavioural-insights.htm. Accessed January 21, 2020.
Papke, L. E. (1995). Participation in and contributions to 401(k) pension plans. Journal of
Human Resources, 30, 311–325.
Plan Sponsor Council of America. (2018). 60th annual survey of profit sharing and 401(k)
plans. Chicago, IL: Plan Sponsor Council of America.
Rabin, M. (1998). Psychology and economics. Journal of Economic Literature, 36, 11–46.
Samuelson, W., & Zeckhauser, R. (1988). Status quo bias in decision making. Journal of
Risk and Uncertainty, 1, 7–59.
Schäfer, T., & Schwarz, M. A. (2019). The meaningfulness of effect sizes in psychological
research: Differences between sub-disciplines and the impact of potential biases.
Frontiers in Psychology, 10.
Shefrin, H. M., & Thaler, R. H. (1988). The behavioral life-cycle hypothesis. Economic
Inquiry, 26, 609–643.
Simon, H. A. (1955). A behavioral model of rational choice. Quarterly Journal of
Economics, 69, 99–118.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer.
Journal of Experimental Psychology: General, 143, 534–547.
Sparkman, G., & Walton, G. M. (2017). Dynamic norms promote sustainable behavior,
even if it is counternormative. Psychological Science, 28, 1663–1674.
Thaler, R. H. (1994). Psychology and savings policies. American Economic Review Paper
and Proceedings, 84, 186–192.
Thaler, R. H., & Benartzi, S. (2004). Save more tomorrow: Using behavioral economics to
increase employee saving. Journal of Political Economy, 112, S164–S187.
Thaler, R. H., & Shefrin, H. M. (1981). An economic theory of self-control. Journal of
Political Economy, 89, 392–406.
Thaler, R. H., & Sunstein, C. R. (2003). Libertarian paternalism. American Economic
Review Papers and Proceedings, 93, 175–179.
Thaler, R. H., & Sunstein, C. R. (2008). Nudge: Improving decisions about health, wealth,
and happiness. New Haven, CT: Yale University Press.
J. Beshears and H. Kosowsky Organizational Behavior and Human Decision Processes 161 (2020) 3–19

Thaler, R. H., Tversky, A., Kahneman, D., & Schwartz, A. (1997). The effect of myopia
and loss aversion on risk taking: An experimental test. Quarterly Journal of Economics,
112, 647–661.
Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and
probability. Cognitive Psychology, 5, 207–232.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases.
Science, 185, 1124–1131.
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of
choice. Science, 211, 453–458.
Wisdom, J., Downs, J. S., & Loewenstein, G. (2010). Promoting healthy choices:
Information versus convenience. American Economic Journal: Applied Economics, 2(2),
164–178.
Wood, W., & Runger, D. (2016). Psychology of habit. Annual Review of Psychology, 67,
289–314.

19