Journal of Economic Psychology 33 (2012) 665–676

Contents lists available at SciVerse ScienceDirect

Journal of Economic Psychology

journal homepage: www.elsevier.com/locate/joep

Review

Behavioral economic engineering

Gary E. Bolton a,⇑, Axel Ockenfels b,⇑
a
Smeal College of Business, Penn State University, University Park, PA 16802, USA
b
University of Cologne, Department of Economics, Germany

a r t i c l e i n f o a b s t r a c t

Article history: Economic engineering is the science of designing real-world institutions and mechanisms
Available online 14 September 2011 that align individual incentives and behavior with the underlying goals. This paper dis-
cusses why behavioral economic engineering is a promising research field, how behavioral
JEL classification: phenomena may affect economic engineering, and the role of theory and laboratory exper-
C9 iments for behavioral economic engineering in practice. We provide examples, many from
D0 our own work.
PsycINFO classification:
Ó 2011 Elsevier B.V. All rights reserved.
3000
3600

Keywords:
Economic engineering
Behavioral economics
Experimental economics

Contents

1. Introduction: Why messiness matters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666

2. The need for behavioral economic engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667
2.1. It derives from the messiness of behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667
2.2. Institutional versus behavioral messiness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667
2.3. Examples of behavioral engineering in practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
3. The role of theory and laboratory experiments in economic engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 672
3.1. Theoretical models as maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 672
3.2. Experiments as maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 672
4. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674
Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 676

‘‘A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity
of complete fools’’

Douglas Adams (A Hitchhiker’s Guide to the Galaxy, 1979)

⇑ Corresponding authors.
E-mail addresses: gbolton@psu.edu (G.E. Bolton), ockenfels@uni-koeln.de (A. Ockenfels).
URLs: http://www.personal.psu.edu/faculty/g/e/geb3/ (G.E. Bolton), http://ockenfels.uni-koeln.de/ao (A. Ockenfels).

0167-4870/$ - see front matter Ó 2011 Elsevier B.V. All rights reserved.
doi:10.1016/j.joep.2011.09.003
666 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676

‘‘The proof of the pudding is in the eating.’’

Cervantes in his Don Quixote (1605)

1. Introduction: Why messiness matters

Economic engineering is the science of designing real-world institutions and mechanisms that align individual incentives
and behavior with the underlying goals. Actively designed economic mechanisms figure prominently in modern economic
life; examples include spectrum and security auctions, energy markets, emission trading rules, procurement and supply
chain contracts, worker compensation plans, matching and online markets (Chen & Ledyard, 2006; Milgrom, 2004; Roth,
2002a, 2008, among others, provide surveys). The designs chosen affect the price for mobile phone services, for electricity
and for consumer goods, the taxpayers’ cost of the financial crisis, motivation and satisfaction at work, the probability of
receiving an organ when needed, the degree of (dis-) trust in markets, and the success of climate politics.
Mechanisms matter because they affect incentives, and decision makers respond to incentives. They do not always do so
in a rational or selfish way, though; yet behavior often responds in a systematic way. This is the starting point of research in
what we call ‘‘behavioral economic engineering’’. This research field aims at designing mechanisms based on sound behav-
ioral research: a mechanism that works under the assumption of rational and selfish behavior may not work well with hu-
man behavior.
A major motivation for studying behavioral economic engineering is increasing disappointment with how economic sci-
ence contends with real-world challenges. In a notable speech delivered at Princeton in September 2010, Ben Bernanke ad-
dressed the implications of the financial crisis for economic research. He distinguished between economic science,
engineering and management. According to Bernanke, the financial crisis was especially a failure of economic engineering
and economic management.
‘‘Underpinning any practical scientific or engineering endeavor, such as a moon shot, a heart transplant, or the construction of a
skyscraper are: first, fundamental scientific knowledge; second, principles of design and engineering, derived from experience
and the application of fundamental knowledge; and third, the management of the particular endeavor, often including the coor-
dination of the efforts of many people in a complex enterprise while dealing with myriad uncertainties. Success in any practical
undertaking requires all three components. For example, the fight to control AIDS requires scientific knowledge about the causes
and mechanisms of the disease (the scientific component), the development of medical technologies and public health strategies
(the engineering applications), and the implementation of those technologies and strategies in specific communities and for indi-
vidual patients (the management aspect). Twenty years ago, AIDS mortality rates mostly reflected gaps in scientific understand-
ing and in the design of drugs and treatment technologies; today, the problem is more likely to be a lack of funding or trained
personnel to carry out programs or to apply treatments. With that taxonomy in hand, I would argue that the recent financial
crisis was more a failure of economic engineering and economic management than of what I have called economic science. The
economic engineering problems were reflected in a number of structural weaknesses in our financial system. [. . .] And although I
don’t think the crisis by any means requires us to rethink economics and finance from the ground up, it did reveal important
shortcomings in our understanding of certain aspects of the interaction of financial markets, institutions, and the economy’’
Bernanke goes onto discuss what he sees as a gap between our models of behavior and actual behavior:
‘‘. . . the crisis has raised some important questions that are already occupying researchers and should continue to do so. As I
have discussed today, more work is needed on the behavior of economic agents in times of profound uncertainty. . .’’
Paul Krugman (along with a number of other economists, journalists and politicians around the world) makes a similar
point in an essay published as a 2009 NYT-article after the financial crisis peaked:
‘‘. . . what’s almost certain is that economists will have to learn to live with messiness. That is, they will have to acknowledge the
importance of irrational and often unpredictable behavior, face up to the often idiosyncratic imperfections of markets and
accept that an elegant economic ‘‘theory of everything’’ is a long way off.’’
This paper discusses the need for economic engineering in terms of the behavioral messiness institutional (and mecha-
nism) design confront. The need is evident from current research: There is now a large catalogue of systematic behavioral
deviations away from the full rationality and narrow self-interest assumptions most standard economic theory makes.
The research effort that went into developing these models reflects many years of painstaking work. The effort that will
go into incorporating the more nuanced findings how humans actually behave will likely take as long, and even then there
will likely remain substantial limitations to our theoretical understanding. Supplementing this effort with an engineering
approach aims to bridge the gap between theory and practice, providing careful analytical vetting of new institutional de-
signs, informed by up-to-date science. What we learn from both successful and failed design efforts in a messy world can be
important for discovering promising research topics, developing new theory and empirical methods, and understanding how
different scientific approaches may complement each other (see Ostrom, 1990; Roth, 2002a, for pioneering work along these
lines).
 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676 667

Section 2 discusses the nature of behavioral messiness in institutional design, the potential value an engineering ap-
proach can have in dealing with it, and examples of behavioral economic engineering. Section 3 discusses the role of theory
and experiments. In most current research, the way we think of theory and experiment pivots on the testing of descriptive
hypotheses. Engineering, while rooted in descriptively valid models, is ultimately prescriptive in goal. To this end, theory and
experiment provide ‘maps’, with the advantages and limitations this term implies. Section 4 makes concluding remarks.

2. The need for behavioral economic engineering

2.1. It derives from the messiness of behavior

Engineering offers a method of delving into the messiness of human behavior and social interaction prior to an institu-
tion’s launch on a large social scale. The potential social benefit from grappling with the behavioral factor is nicely illustrated
by the development of the F100 jet engine for the famous F-15 fighter jet. In 1970, the US Air Force commissioned Pratt and
Whitney to develop and build the F100 to specifications beyond the capabilities of any plane of the time. One of the major
hurdles in the engineering development involved unanticipated human reactions to the introduction of the engine. As Dre-
wes (2005) explains
‘‘The performance of the F100, when it worked, was so vastly superior to any predecessor that it changed combat tactics and the
training regime. . . . Neither the Air Force nor Pratt anticipated this change . . . Perhaps no one could have fully appreciated the
capabilities and new opportunities provided by the F-15 until fighter pilots could actually get behind the stick . . .’’
Yet the greater sophistication in combat tactics also caused a higher than anticipated frequency of engine breakdowns.
Behavioral changes from new machinery are not uncommon. The aeronautic engineers working on the F100 could (and
eventually would) fix this one. The real problem in the story turned out to be in the economic engineering: Pratt and Whit-
ney’s contract with the Air Force stipulated a fixed price in exchange for an engine that met a fixed set of specifications – and
the engine met every specification stipulated. The development contract did not allow for any flexibility if an unanticipated
problem was uncovered. The engine was put into full production, and the planes experienced major problems.
As a result of this unhappy experience, the US Air Force reworked its sourcing practices starting in 1979, introducing
greater flexibility into contracts, to provide room for unanticipated circumstances, together with dual sourcing and tit-
for-tat play, to control the incentive problems that greater flexibility might facilitate. Crocker and Reynolds (1993) study
the entire episode. Their analysis includes a number of lessons for procurement contracting:
‘‘The first concerns the emphasis by policymakers . . . on firm-fixed pricing in development contracts to constrain seller oppor-
tunism and to contain cost overruns. While more complete contracts certainly suffice to mitigate ex post redistributive efforts by
contractors, our analysis indicates that such benefits may be dwarfed by the costs of drafting truly complete agreements, par-
ticularly in complex exchange environments. A second, and related, point is that procurement officers should be granted the lat-
itude to craft agreements on a case-by-case basis, where the design of a particular contract would depend on the specifics of
both the product and the contractors. . . . A final implication concerns the potential benefits accruing from the use of dual sourc-
ing. While other authors have been concerned with the cost and pricing effects of split awards, dual sourcing may also generate
resource savings in the drafting of contracts by permitting the implementation of less complete agreements. The discipline pro-
vided by the existence of an alternative supplier mitigates the necessity for a complete, and costly, contractual document to con-
strain future contractor behavior’’
These important lessons are now common place in procurement practice. But trial and error was a very expensive way to
learn them. At the time, the guiding theory behind the practice focused on seller opportunism overlooking (messy) problems
in the development process. The objective behind behavioral economic engineering is to catch problems in the robustness of
new economic institutions and mechanisms prior to their engagement on a large social scale, where failure can be far cost-
lier.1 In fact, with regard to procurement, there is now an emerging behavioral operations management literature that, among
other things, aims to engineer better procurement auctions; see for example Engelbrecht-Wiggans, Haruvy, and Katok (2007).
Economic theory typically abstracts away from ‘‘messy details’’ often defended on the grounds that these details are rel-
atively inconsequential, and so we can abstract away from them with little loss of accuracy. But if we reflect on this for a
moment, we see this is unlikely to be true. For one, we know that many theoretical results in economics are sensitive to
the details of institutions and behavior. Even small details of the mechanism, small changes in the information available,
or small deviations from rationality or self-interest, or common knowledge thereof, can dramatically change predictions –
even before we consider other real-world complexities that come with a specific context. For example, different bargaining
theories arrive at sometimes very different conclusions when attempting to explain the same underlying bargaining problem
(the set of feasible outcomes and outside options) – depending on the action space they assume (the institutional and com-

1
Similar conclusions have been drawn from the spectrum auction history. VanDamme (2001), for instance, writes in the context of the Dutch 3G spectrum
auction that ‘‘Just as constructing physical infrastructure is professional work that takes time, so is market engineering and the constructing of proper auction rules to
obtain desirable outcomes. The latter, however, does not yet seem to be appreciated in the Netherlands as the recent experience with the planned auction of frequencies
for commercial radio stations has clearly shown. This case shows how powerful lobbies of vested interests can be and how populistic (and foolish) the Dutch
parliament.’’
668 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676

munication structure), on the beliefs bargainers have (the informational structure), or on the preferences of the bargainers
(fully self interested or social preferences – the behavioral structure). Moreover, most social interaction have a history or
future, and the theory of repeated interaction often does not yield unique predictions and recommendations.
In order to say that an institution is useful and robust in practice, we need first acknowledge the existence of messiness,
and then to better understand the nature of this messiness, and how it affects and is affected by institutional design. This is
the major challenge of economic engineering. The following sections provide further examples of some of those challenges.

2.2. Institutional versus behavioral messiness

Most of the economic engineering literature focuses on complexities arising from non-convexities, complementarities,
incompleteness and other complicating issues (see the surveys in Kittsteiner & Ockenfels, 2006; Klemperer, 2004; Milgrom,
2004; Müsgens & Ockenfels, 2006; Roth, 2002a, 2008). One area of research, for example, addresses problematic incentives
for the timing of interactions: the flexibility in choosing the timing of the transactions and the resulting endogenous dynam-
ics of the economic and social interactions can critically affect the success of mechanisms. Nevertheless, standard game the-
ory often assumes simple, exogenous structures of the information flows. In a series of market engineering papers, involving
theoretical, simulation and experimental studies, Roth and his coauthors analyzed and developed mechanisms in various
matching markets that address problems partly arising from incentives to act earlier than others (see, e.g., Mongell & Roth,
1991; Roth, 1990, 1991; Roth & Peranson, 1999; Roth & Xing, 1994; examples for complementary experimental studies
are Chen & Sönmez, 2006; Kagel & Roth, 2000; a survey is provided by Roth, 2008). For instance, competition for people
and positions in entry-level jobs for American doctors led to earlier and earlier dates of appointment, to the point that stu-
dents were being hired before useful information about their performance was available, and before the students themselves
could develop informed career preferences. When this induced yet another crisis in the market in the mid 1990s, Roth
started to design what would be very successful matching algorithms to stabilize the market. One of Roth’s first challenges
nicely illustrates the naturally occurring messiness that can thwart existing theory: Theoretical matching mechanisms were
not robust to the incentives and behavior of doctor couples who need a pair of positions, as was the case in the American
doctor market of the 1990s – a circumstance unanticipated by the theory. See Roth (2002a) for a survey of how this and other
challenges in matching markets can be successfully addressed utilizing the scientific toolbox.
Another example comes from markets that must deal with incentives to act later than others. In particular, dynamic trading
often leads to very late activities. For example, ‘sniping’ – bidding in an auction at the last moment – hampers price discovery
and market efficiency on eBay. While this was not, at the time, recognized in theory, careful design choices can avoid sniping.
For example, ‘‘soft close auctions’’ can eliminate the incentives to bid late in online auctions, and ‘‘activity rules’’, which re-
quire a bidder to be ‘active’ (that is to be the current high bidder or to submit new bids), can prevent bidders from holding
back.2
Other examples stem from the engineering history of spectrum auctions, where new rules and mechanisms were de-
signed, tested and implemented to more or less successfully cope with various economic and computational complexities
(Cramton, Shoham, & Steinberg, 2006; Klemperer, 2004; Milgrom, 2004). There is also a related literature on the design
of electricity markets, which are inherently incomplete, imperfectly competitive and plagued by various other complications
(e.g., Cramton & Stoft, 2008; Grimm, Ockenfels, & Zoettl, 2008; Müsgens, Ockenfels, & Peek, 2011; Wilson, 2001; Ockenfels,
2009b). On a more general note, the process of engineering mechanisms for infrastructure industries, including telecommu-
nication and energy markets, is related (and in fact sometimes identical) to the challenges faced by regulators. An early ac-
count of the regulator’s difficulties to deal with messiness comes from Kahn (1979) in a paper called ‘‘Application of
Economics to an imperfect world’’:
‘‘. . . the world to which he [the economist] would apply his principles is excruciatingly imperfect and resistant; and the compass
he needs is one that would help him thread his way through the thickets of second best. The really challenging job is deciding not
what the ultimate economically rational equilibrium should look like, but what is economically rational in an irrational world,
and how best to get from here to there. That, too, turns out to be a kind of frontier; and life on it is full of excitement.’’
These examples deal mostly with messy problems introduced by various frictions in the economic environment. Yet, the
current paper’s main focus (while related) is on behavioral engineering – problems that result from behavioral messiness.
Economic and psychological experiments teach us that decision makers do not always respond to incentives set by institu-
tions in a rational or selfish way, as assumed by standard economic theory (see, e.g., Camerer, 2003 for a survey). But nor is
behavior wildly chaotic or completely irrational. Rather, people follow their own, bounded rationality, characterized by sys-
tematic patterns and limitations of motivation, affection, cognition and adaptation (Selten, 1998).
Sometimes it is argued that a combination of market forces, learning and evolution render bounds of rationality irrele-
vant. But such is often not the case. For one, because of limits of arbitrage and because of complementarities in strategic
interaction, even small deviations from rationality or self-interest by a few decision makers may strongly influence behavior

2
Sniping is also a concern in other markets. Two examples: The German stock exchange, where the exact moment the auction will end is randomly selected
in order to prevent bidders from waiting until the very last second before submitting their final bids. Online negotiation sites that promise dispute resolution
(such as e-commerce disputes and traditional litigation) via electronic and standardized communication also suffer from late bidding. See Ockenfels and Roth
(in press) for a survey concerning online auctions.
 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676 669

of all market participants. Examples include Fehr and Tyran’s (2005, 2008) work on money illusion, Kreps and Wilson’s
(1982) work on the impact of little ‘craziness’ on reputation building, Nagel’s (1995) studies of the guessing game, and Fehr
and Schmidt’s (1999) and Bolton and Ockenfels’ (2000) work on the interaction of institutions and preferences for fairness.
Moreover, the intuition that experts behave more rationally than subjects in a laboratory has been often rejected (see, e.g.,
Bolton, Ockenfels, & Thonemann, 2010; Fehr, Fischbacher, & Tougareva, 2002, and the references therein).
Second, almost all economic transactions involve bilateral bargaining. As Camerer (2003, p. 151) puts it:
‘‘Bargaining . . . is possibly the most basic activity in economic life. Even in thick, competitive markets where traders closely
resemble ‘price takers’ of economic theory, haggling occurs over time of delivery, repairs, sidepayments, and quality, as well
as price.’’
Bilateral interaction, however, often leaves significant room for outcomes being affected by social cognition, other-
regarding preferences and trust, as well as by mistakes and noise (see, e.g., Roth, Prasnikar, Okuno-Fujiwara, & Zamir,
1991; Fehr, Klein, & Schmidt, 2007, for laboratory experiments).
This is related to ideas in new institutional economics as laid out in Williamson (1979). He argues that many exchanges
involve transaction-specific investments that essentially lock-in the participants (the F100 engine episode described in Sec-
tion 2.1 is an example). For instance, once an agreement to make a transaction is struck in the marketplace, any commitment
of money or time to making the deal happen is a transaction-specific investment, in that the value of the time and money
cannot be fully retrieved unless the deal goes through. Effectively, the relationship between a buyer and seller becomes one
of bilateral monopoly (which, of course, is also true for any formal contractual relationship for transactions over longer peri-
ods). Bilateral monopoly, however, creates the potential for opportunistic behavior on the part of the participants. Here,
‘‘opportunistic behavior’’ often encompasses more than problems with incentives and cannot easily be handled by formal
contracts. Masten (1988) writes:
‘‘Transactors are characterized by their cleverness . . . in circumventing rules, discovering loopholes, or otherwise exploiting stra-
tegic advantages. Using contracts to try to induce cooperative behavior from a non cooperative actor is like trying to pick up
mercury.’’
The aim then, according to Masten, is to ‘‘design institutional arrangements to minimize the frictions inhibiting cooperative
exchange.’’ Behavioral economic engineering aims to discover how actual (social) behavior interacts with the institutional
design.
The good news from the literature is that behavior can often be described in a systematic and testable fashion. In the last
two decades there has been enormous progress, through experiments and modeling, in understanding how social behavior
affects strategic behavior, shaping outcomes. It is now widely acknowledged, including in economics, that some human deci-
sion makers care not only about their own material payoff, but also about the distribution of payoffs in the reference group,
about how the distribution was generated, and about reciprocity. Agents can be driven by a concern for reciprocal fairness,
distributive fairness or altruism, and they may behave trustworthy or trusting even when economic incentives dictate other-
wise. Social preferences interact with institutions in non-trivial ways. For example, depending on the institutional and infor-
mational environment, social agents can change the incentives for selfish agents, and vice versa: positive reciprocity may
induce selfish agents to be cooperative, negative reciprocity may deter selfish agents from behaving aggressively. Selfish
agents can induce conditional cooperative agents to behave selfishly. Less research, however, has taken a more prescriptive,
engineering approach, addressing the questions of how mechanisms can be designed such that competitive and cooperative
motives are balanced in an efficient way.

2.3. Examples of behavioral engineering in practice

Work in a number of disciplines can help us better understand the role of behavior in institutional design. Here we give a
brief and highly selective account of the literature.
Recent, well-cited work that incorporates behavioral sciences into economic engineering deals with individual saving
decisions for retirement, something that requires both complex calculations and willpower. It is well known that people pro-
crastinate about joining savings plan, and that it is possible to increase participation rates simply by switching the default
option for new workers (see, e.g., Choi, Laibson, & Madrian, 2009). Other examples are given in a popular book by Sunstein
and Thaler (Sunstein & Thaler, 2008), who show how decision biases involving education, personal finance, health care,
mortgages, etc. can be debiased by an accurate ‘‘choice architecture’’ designed to ‘‘nudge’’ the decision maker into beneficial
directions. Bazerman (2005) gives a number of examples on how psychological decision research can (and did) affect the
marketplace and legal judgment. As he argues in a paper titled ‘‘Conducting influential research: the need for prescriptive
implications’’:
‘‘. . . too often, prescription is not [. . .] on the radar screen for other social sciences. As a graduate student in the late 1970s, I was
trained to be descriptive; prescription was for consultants, not for serious researchers. I now believe that this attitude is wrong,
not only for the field of management but for society as a whole. Today, the most important management research provides novel
descriptive findings that can be turned into useful prescriptions.’’
670 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676

A more prescriptive approach has also been adopted in the behavioral operations management literature. For example, a
good deal of work has been directed at examining judgment in the newsvendor game, a foundational paradigm for the prob-
lem of inventory stocking in the face of uncertain demand. The robust finding is that judgment in this game tends to be
biased away from optimality, regardless of whether played by students or procurement managers, although the precise nat-
ure of the biases vary widely by individual (Bolton et al., 2010; Moritz, 2008; Schweitzer & Cachon, 2000) A uniting theme
behind these variations is a tendency to draw conclusions on the basis of too little sampling. A proposed engineering fix that
does well in the lab is to insist that individuals place a fixed orders for multiple rounds, the precise number chosen to provide
adequate sampling (Bolton & Katok, 2008; Lurie & Swaminathan, 2007).
There is also a large literature on cognitive engineering combining cognitive science and human factors to analyze hu-
man–computer interaction and engineering human–system interfaces (see Roth, Patterson, & Mumaw, 2002 for the role
of cognitive engineering in promoting efficient decision making). Related to these are the marketing and consumer research
literatures, places where practice and fundamental research directly aim to profit from each other (Kotler & Armstrong,
2007). Finally, there is increasing interest in management science to promote ‘‘Evidence Based Management’’, an attempt
to elevate ‘‘professional decisions away from personal preferences and unsystematic experience toward those based on the best
available scientific evidence’’ (see Rousseau, 2006 and the references therein). While the last two research fields do not di-
rectly focus on institutional design, the idea of bridging fundamental behavioral research and a more prescriptive approach
link them to behavioral economic engineering.
Most of the examples mentioned above, however, do not address the limits of rationality and self-interest in interactive
and social behavior in organizations and markets. Rousseau (2006) describes one early exception using psychological knowl-
edge to improve real, interactive markets through optimizing feedback:
‘‘The executive director of a health care system with twenty rural clinics notes that their performance differs tremendously
across the array of metrics used. This variability has nothing to do with patient mix or employee characteristics. After interview-
ing clinic members who complain about the sheer number of metrics for which they are accountable (200 indicators sent
monthly, comparing each clinic to the 19 others), the director recalls a principle from a long-ago course in psychology: human
decision makers can only process a limited amount of information at any one time. With input from clinic staff, a redesigned
feedback system takes shape. The new system uses three performance categories – care quality, cost, and employee satisfaction
– and provides a summary measure for each of the three. Over the next year, through provision of feedback in a more interpret-
able form, the health system’s performance improves across the board, with low-performing units showing the greatest
improvement. In this example a principle (human beings can process only a limited amount of information) is translated into
practice (provide feedback on a small set of critical performance indicators using terms people readily understand).’’
Plott (1994, 1997) is one of the early pioneers of using experimental research to inform policy makers on market design
and auction issues in a more sophisticated and systematic way. Plott’s experiments have contributed to a wide range of pol-
icy issues, including the problems of allocating landing rights at major airports and telecommunication spectrum, as well as
regulation and anti-trust issues. (See also the influential work by, and surveyed in Ledyard (1993) and Chen and Ledyard
(2006).) Increasingly, this line of laboratory engineering work has been complemented by theoretical models that describe
how behavioral phenomena may affect institutional design. For example, Crawford, Kugler, Neeman, and Pauzer (2009) de-
rive the optimal sealed-bid auction design assuming ‘‘level-k’’ thinking on the bidder side. They show, among other things,
that in a first-price auction, level-k bidding changes the optimal reserve price. Also in the auction context, Ockenfels and Sel-
ten (2005) show that an equilibrium model based on a simple ex-post rationality principle correctly predicts different effects
of (theoretically irrelevant) feedback mechanisms. And the work by Ariely, Ockenfels, and Roth (2005), Roth and Ockenfels
(2002), and Ockenfels and Roth (2006) shows how naive and sophisticated behavior in online auctions interact in ways that
can be robustly controlled for by adjustments in the auction rules.
In addition to the market and auction literature, there is an evolving engineering literature dealing with the promotion of
cooperation. This literature is partly based on the enormous progress made in the theory of mechanism design in the last
decades. As Hugo Sonnenschein wrote already in 1983:
‘‘Economic research now almost routinely takes up the problem of whether or not there are institutions that will enforce coop-
erative behavior, institutions that will get the sidewalks and streetlights built in precisely the amount that is socially optimal.
We now consider the possibility of ‘‘designing cooperation,’’ just as engineers have been concerned with the problem of design-
ing electric switches. We are very much at this stage of basic research, more physics than real engineering, but I want to argue
that our success has been real, and I want to emphasize that this achievement has been at the very top of what has happened in
economics during the last couple of decades. To my mind there is no hope that economists will speak ‘‘with one tongue’’ until we
understand the economics of incentives and the possibility of designing cooperative behavior.’’3

3
See Cramton and Stoft (2010) for promoting an ’engineering negotiations’ approach for the (so far failed) international climate negotiations. Another
intriguing example is Roth, Sönmez, and Ünver (2004), who propose that kidney exchange be organized by integrating cycles and chains of transplants. There
was quite some messiness in kidney exchange caused by the need for simultaneous exchange, resulting in a redesign around pairwise exchanges. More efficient
chains of exchange were integrated when it became possible to do exchanges non-simultaneously, corresponding to the behavioral fact that people who agree
to give a kidney mostly do so even after their patient gets a kidney.
 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676 671

The engineering literature dealing with conflict and cooperation is also based on progress in the psychology and econom-
ics of peoples’ social preferences for considerations such as fairness, reciprocity and altruism. The work opens the door for
studies probing how these concerns shape and are shaped by institutional design. For instance, Chen, Harper, Konstan, and Li
(2010) find in an intriguing paper that social comparison processes can be effectively used to design optimal personalized
social information schemes that increase the level of public goods provision. Specifically, they designed a feedback scheme
in the online community MovieLens and show that users below the median demonstrate a 530% increase in the number of
monthly movie ratings, while those above the median decrease their ratings by 62%.
Ockenfels, Sliwka, and Werner (2011) find that the transparency of social reference point violations within a large multi-
national firm’s bonus system significantly affects employees’ satisfaction and performance. They provide a model of the
underlying social comparison concerns that captures a number of phenomena in the data, including the well-documented
effects in personnel economics and personnel psychology of ‘‘leniency bias’’ and ‘‘centrality bias’’. In the present context,
subjective performance ratings are often too ‘‘lenient’’, i.e. that supervisors often avoid giving negative ratings, and that rat-
ings tend to be too ‘‘compressed’’, i.e. supervisors typically do not sufficiently differentiate between high and low performers.
The model, laboratory results and the naturally occurring field data in Ockenfels et al. all suggest that the transparency of
bonuses in the firm is a major source of undesirable social comparison effects, and so hints at possible engineering solutions
(that were partly implemented and are being evaluated; see also Greiner, Ockenfels, & Werner, in press).
Designing feedback, channeling social information and diminishing the leniency bias also come up in a study by Bolton,
Greiner, and Ockenfels (2011), who focus on improving a particular feedback system, eBay, through complementary and
comparative study of field data garnered from a variety of Internet markets, field data from eBay, laboratory experiments,
economic theory, and review of earlier research.4 More specifically, Bolton, Ockenfels, and Ebeling (2011) provide data on
endogenous feedback giving phenomena that are not well understood and little addressed by existing reputation theory.
One important finding is that frequency and content of feedback giving are highly reciprocal in nature, and that this has both
potentially positive effects on the system (more feedback giving) as well as negative effects (discouraging negative reports).5 For
instance, Bolton et al. find that only 1.4% of all feedback on eBay is negative, supplying little discrimination between sellers.
Resnick and Zeckhauser (2002) and Dellarocas and Wood (2008) find 0.6% negative feedback, an even smaller share. On the
other hand, there is evidence that the actual share of dissatisfied buyers is much higher (Dellarocas & Wood, 2008). A major
reason for the negligible share of negative feedback is retaliation: sellers tend to retaliate negative feedback by giving negative
feedback to buyers, thus deterring negative feedback in the first place. This gaming in the production of reputation information
significantly reduces the informativeness of feedback information, hampering the ability of a reputation system to facilitate
trust and trade efficiency.
However, as Bolton et al. demonstrate, reciprocity can be guided by changing the way feedback information flows through
the market system, leading to more accurate reputation information, more trust and more efficient trade.6 Specifically, their
data show that, compared to the simple two-sided feedback system traditionally implemented by eBay, both ‘blindness’ in feed-
back giving and one-sidedness in a detailed seller rating system increase the informativeness of the feedback presented to buy-
ers.7 This result is behaviorally robust in the sense that it is based on complementary evidence from very different sources. It is
in line with fundamental theoretical and laboratory research on social behavior and reputation building that gradually adds
institutional details. The results are also in line with various naturally occurring field data (Bolton et al.), as well as with con-
trolled field experiments (such as in Resnick, Zeckhauser, Swanson, & Lockwood, 2006; Bolton & Ockenfels, 2010) that ‘lose con-
trol in a controlled way’ in order to gap the bridge between field, laboratory and thought experiments.
Bolton, Greiner, and Ockenfels (2011) engineering analyses guided eBay in its decision to change its reputation system,
and preliminary data confirm the expected effects from the new field system. While, given the messiness (the relative lack of
control and measurability) of a system like eBay, we cannot be absolutely certain the system works for the right reasons, we
can be rather confident of it given that the key principles have been identified in more controlled environments. The engi-
neering work also points to important and promising challenges for standard economics research that are at the heart of al-
most any (online or offline) reputation system; e.g., gaps in our understanding of the strategic value and motivational side of
feedback provision, the implied limits of cooperation, trust and competition in markets and society, and how these can be
shaped by institutional design (Ockenfels & Resnick, in press).8

4
Surveys on the economics of Internet auctions in general, including design aspects, are provided by Bajari and Hortaçsu (2004), Ockenfels, Reiley, and
Sadrieh (2007), and Greiner et al. (in press).
5
A highly controlled framed field experiment in Bolton and Ockenfels (2010) supports these findings.
6
A major challenge in solving marketplace trust problems has to do with the need to take into account adverse platform effects due to side-effects or
disruption of path dependencies caused in the migration to a new system. We will, however, not discuss this issue here.
7
Retaliatory feedback might be eliminated by not letting sellers evaluate buyers as suggested by, e.g., Güth, Mengel, and Ockenfels (2006), or by having a
‘blind period’ in which trading partners can ‘simultaneously’ leave feedback on each other as suggested by, e.g., Klein et al. (2007). Also, clever incentive
schemes, either based on economics (Miller, Resnick, & Zeckhauser, 2005) or social psychology (Rashid et al., 2006), may overcome the public goods problem
and promote full provision of all relevant feedback information. And modern authentication technologies or ‘entry fees’ may eliminate manipulative changes of
online identities (see Friedman & Resnick, 2001; Ockenfels, 2003).
8
EBay is a convenient starting point for this research because it allows researchers to quantify some of the benefits and problems with naturally occurring
systems. We also note that eBay’s feedback system is part of a larger mix of (imperfect) policies and rules that interact to promote trust and trade efficiency.
Currently, only a few papers address this interaction. One is Güth et al. (2006), who investigate the joint effectiveness of buyer insurance (which is part of eBay’s
so-called Purchase Protection Program) and eBay’s feedback forum.
672 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676

3. The role of theory and laboratory experiments in economic engineering

Currently, laboratory research is most associated with testing theories for descriptive validity. How then do we think of
the role of the laboratory, and of theory, for the purpose of behavioral engineering, where the questions are prescriptive in
nature? The answer, we think, is ‘as maps’.

3.1. Theoretical models as maps

One might conclude from the above discussion (along with other objections commonly voiced) that current theory is
pretty useless for practical purposes. But we would say that this conclusion is wrong. Economic theory provides orientation
in a messy world. The insights from both auction and matching theory have been critical to the development of real-world,
engineered auctions and matching algorithms (see the literature cited above). Theory provides a useful starting point to de-
velop and communicate intuition, and to provide conceptual insights – and it does so precisely because it abstracts away
from messiness.
One important lesson from economic engineering is that dealing with real-world messiness clarifies the role of theory for
understanding the real world. For example, what precisely do we mean when we say the theoretical models provide insight?
We think of theoretical models as ‘maps’. The most useful models are simplifications of the actual landscape, calling our
attention to the most important features which, of course, depend on the objective – you use a different kind of map for hik-
ing than for driving. Even very good road maps have inaccuracies and omissions.9 They regularly fail to alert us that the road is
washed out or fail to note the location of the nearest repair shop when the car breaks down. Yet in spite of these limitations,
road maps are incredibly useful. They provide a starting point for planning the trip and get us towards our destination; albeit,
we have to be ready to make adjustments along the way. Sometimes that means supplementing the road map with other kinds
of information, such as the advice of local people or alternative maps of the landscape.
Two insights offered by social preference theory helped us (together with Ben Greiner; Bolton, Greiner, et al., 2011) to
identify the broader pattern of reciprocal behavior behind the feedback retaliation observed in eBay’s feedback system. First,
reciprocal behavior is most prominent in non-competitive interactions such as bilateral bargaining and gift exchange. Ebay’s
feedback system is associated with after-auction circumstances, where the seller and winning buyer are locked into a bilat-
eral relationship. Hence it was natural to view retaliation as reciprocal behavior. Second, the act of any kind of ‘‘gift giving’’ is
sufficient to trigger reciprocity. If feedback retaliation is a form of negative reciprocity for a negative valued gift, then we
would expect positive valued gifts to induce a positive reciprocal response. In fact, in exchanges where both sides give feed-
back, positive feedback is overwhelmingly reciprocated with positive feedback (about 98% of the time). Recognizing this was
important because it suggested that methods of suppressing retaliatory feedback might also reduce all feedback giving be-
cause much of it is reciprocally based. If the overall reduction was substantial enough, it might reduce the credibility of the
system to traders who had come to view ‘‘silence’’ (no feedback given) as a negative signal. It is important to note that social
preference models have been formally worked out to explain only pecuniary, reciprocal exchanges (see Cooper & Kagel, in
press, for a review). But the larger insight nevertheless can be applied successfully to the exchange of reputation information.

3.2. Experiments as maps

As argued before, theory alone is not sufficient to guide engineering. As Roth (2002a) put it:
‘‘Dealing with complications requires not only careful attention to the institutional details of a particular market, it also requires
new tools, to supplement the traditional analytical toolbox of the theorist.’’
We believe that experiments will play a central role in this toolbox. Much of what has been said regarding theory also
holds for experiments: Laboratory experiments too are not always robust (e.g., context, framing, subject pool and other ef-
fects). However, experiments too provide maps, and can provide orientation in a messy world – especially in combination
with the maps theory offers. This combination of maps provides a broader gauge of robustness:
Engineering is to a large extent about robustness to real world context. Economic models, on the other hand, typically do
not address this kind of robustness. Rather, the main measures for model predictions are existence, uniqueness and effi-
ciency of equilibrium behavior. Yet, for design purposes, a mechanism is irrelevant if it is not robust to behavioral assump-
tions, regardless of whether an equilibrium exists, is unique or efficient.10 Experiments are useful first because they address
the robustness of the theory’s findings against behavioral complexities. This holds in particular in dynamic settings, where out-

9
As Roth (2002b) put it: ‘‘Since we know that approximations are not precisely true, it is easy not to be impressed by evidence that they are not.’’ The
exchange between Bolton and Ockenfels (2006) and Engelmann and Strobel’s (2004) comment on Bolton and Ockenfels (2000) provides an illustration for our
point.
10
While we focus on behavioral robustness, robustness as a criteria is, of course, not new. A related, theoretical literature, for instance, follows the Wilson
doctrine: ‘‘Game theory has a great advantage in explicitly analyzing the consequences of trading rules that presumably are really common knowledge; it is deficient to
the extent that it assumes other features to be common knowledge, such as one agent’s probability assessment about another’s preferences or information. I foresee the
progress of game theory as depending on successive reductions in the base of common knowledge required to conduct useful analyses of practical problems. Only by
repeated weakening of common knowledge assumptions will the theory approximate reality.’’ (see Wilson, 1987; see Bergemann & Morris, 2005, and the references
therein, for theoretical developments along these lines.)
 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676 673

of-equilibrium behavior, path dependencies, and transitional problems may arise, e.g., when new mechanisms are imple-
mented. It is likely that, in the laboratory, characteristics such as social context, transparency of rules, and complexity and dy-
namic stability become important to the success of a mechanism. In this sense, the laboratory serves as a wind-tunnel for both
behavior and design.
A car, with its thousands of electronic modules, is a complicated system. The same is true for mechanisms in firms and
markets, also actively designed. The difference is that the car was test-bedded by engineers who investigated the impact of
(the failure of) many elements, including flawed human decision making. We do not permit a car on the street without these
kinds of robustness tests. But as it stands now, because there is hardly any economic literature explaining how the blueprints
of economic theory can be transformed in reliable real-world mechanisms, there are no analogous standards for robustness
checks for firms and markets. Laboratory experiments can play a central role in providing the needed robustness checks.
Going back to our example of designing eBay’s feedback system: Standard reputation theory typically assumes that rep-
utation information is perfectly accurate and complete. Under these conditions, we can expect to see perfect reputation
building, and perfect trust, among market actors (Wilson, 1985). Thus, in principle, Internet feedback systems can work.
One stream of our own earlier fundamental research that proved useful in designing feedback systems investigates the per-
formance and limits of such ‘‘perfect’’ feedback systems in the laboratory.11 Specifically, our laboratory experiments test the
behavioral limits of these systems, providing an upper bound on what can be expected from field systems where conditions are
less than perfect. An important manipulation in these studies is the pattern of interaction among trading partners allowed in the
market. In markets with direct trading networks, each trader transacts with the same partner repeatedly. In markets with indi-
rect trading networks, traders rotate partners and do little or no repeated business with one another. Three main sets of findings
emerge from these studies. First, in terms of observed behavior, there is a performance gap between the two kinds of networks:
Feedback systems are less effective in promoting trust in indirect trading networks compared to direct trading networks, even
under conditions in which, in theory, they should perform the same (Bolton, Katok, & Ockenfels, 2004, 2005; Bolton & Ockenfels,
2009a; Bolton, Ockenfels, & Ebeling, 2011). The second set of findings partially explains the first: Traders whose trust is betrayed
tend to lose trust, both in the market in general as well as in the specific person who was not trustworthy. These disappointed
traders are less likely to engage in future trades even with those traders with good reputations. Thus, an untrustworthy trader
does more damage the more traders he comes into contact with. In addition, a trusting buyer in a feedback market generates
valuable feedback information for others – information that has little future value for the generating buyer in an environment,
where there is little repeated trading with the same partner. As a result, there is a tendency for buyers to hold back their feed-
back when they anticipate they will have little future trading relations with the same seller. The latter observation was further
studied in Bolton, Loebbecke, and Ockenfels (2008), yielding the third main finding: When there is either competition for trad-
ing partners or opportunities to form long-term trading relationships, both trust and trustworthiness improve under the feed-
back system; in fact, the performance gap between the two kinds of networks is effectively erased (see also Huck, Lünser, &
Tyran, 2007). We can think laboratory experiments such as these as models of a social institution (or mechanisms). For the most
part, laboratory models, like the ones above, have been used to test the predictions of theoretical models, and to identify sys-
tematic deviations. To achieve theoretical tests with high internal validity, the laboratory models are designed to closely match
the theoretical model’s specification of the activity rules, uncertainties and payoffs associated with the institution or
mechanism.
However, the usefulness of this kind of laboratory work, motivated by theory only, is restricted to behavioral complex-
ities. In our feedback system example, because the reputation systems are perfect in the sense of always delivering accurate
and reliable reputation information, they show the limits of a feedback system’s effectiveness due to ‘behavioral defects’
such as insufficient forward looking and out-of-equilibrium dynamics. This way, these studies establish a behavioral map
that, superimposing the theory map, is useful in giving orientation in a messy world. However, neither theory nor the lab
testing of the theory can detect how behavior responds to institutional details not addressed by theory but are relevant in
the field. That is, neither laboratory experimenting nor armchair theorizing can substitute for investigating real-world insti-
tutional messiness. Engineering studies can benefit from laboratory experiments that investigate the robustness of mecha-
nisms against complexities in the institutional environment that are observed in the field. Theoretical models inform this
enterprise as well, since they identify important causal factors and promising new institutional designs. But an engineering
study need also grapple with the messy institutional details that might derail an otherwise fine design, or may suggest the
design, even if successful, will have damaging side effects. The messy details that turn out to matter to the success or failure
of a design can also be viewed as gaps in existing scientific knowledge. So engineering studies can aid basic science by pointing
to places where our understanding of important phenomenon that have yet to receive sufficient theoretical treatment.
Because of the messy details and gaps in scientific knowledge, an engineering lab model requires us to supplement exist-
ing scientific knowledge with subjective judgment. Judgments must be made about which messy details might matter, and
conjectures must be made to fill scientific gaps. The necessary judgments for these tasks can be developed from at least two
sources. One is from the experience-educated assessments of practitioners. For example, managers or other players in the
institution may have insight into the connection between the target problem and other aspects of the system. The second
source is from inspection of institutions that are similar to some extent to the system being investigated. In particular,

11
Keser (2003) investigates ’imperfect’ laboratory feedback systems. See Buskens and Raub (2010) and Ockenfels and Raub (2010) and the references cited
therein for related literature in sociology.
674 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676

institutions with features like a proposed design might provide insight into both connections with other aspects of the sys-
tem as well as how well the proposal might work. In all cases, it is information that we subjectively believe relevant but have
no formal, objective studies for confirmation.
Ideally, no implementation would take place until all issues were subjected to formal and thorough scientific study. But
this process typically requires long periods of time, too long to make waiting before implementing a feasible strategy. Engi-
neering models are a relatively fast method of producing rigorous data, making for a more informed decision. An engineering
model can provide a rigorous test that incorporates both what we know from economic research as well as the messy fea-
tures of the environment that we suspect might blunt the effect of the basic design or produce undesirable side effects. And
at a relatively low cost in circumstances where there may be a high cost to failing in the field. For institutional designs that
are novel, there may be no other way of getting data that speaks to all the issues prior to implementation. See Plott (1994),
Chen and Ledyard (2006) and Ockenfels (2009a) for more on the use of the laboratory in economic engineering research.
Our engineering work with eBay in Boltan, Greiner, et al. (2011) illustrates how an engineering laboratory model can
come together. The model abstracts out the aspects of the system we believe important to the retaliatory feedback, along
with other issues that might turn out to be important to the larger market system. For example, the auction component
is modeled as a private valuation, sealed bid second priced auction. In fact, eBay auctions often involve elements of common
valuation and are known to have important strategic differences with the sealed bid format (Ockenfels & Roth, in press). But
these differences we judged insignificant to the feedback problem.
We modeled the feedback system eBay was using at the time as a soft close procedure to enable and capture the retal-
iation phenomenon. In fact, if we had been exclusively concerned with (suppression of) retaliatory feedback behavior, we
might have chosen to model only the feedback stage of the market. But our analysis of other Internet markets and discussion
with eBay managers suggested to us that changing retaliation and reciprocal feedback in general might change the overall
performance of the market. Here there is little theoretical knowledge to guide us. What we know from theory is that a mar-
ket reputation system that provides accurate information on the history of traders can, in equilibrium, support profitable
trade (Kandori, 1992). But economic theory says far less on what promotes the production of accurate feedback information,
or what effect distortions will have on the system. So we chose to model the entire system, including bidding, product pric-
ing and sales volume and the total feedback in the system. Bidding, pricing and sales volume speak to the overall efficiency of
the market. EBay managers felt that efficiency was probably lower because of retaliation, although at the time there was not
hard evidence to prove this. Monitoring total feedback enabled us to check whether the various methods proposed to fix the
feedback problem might also reduce the amount of feedback, and if so whether this was related to market performance. Here
again, there was little formal analysis to back this concern. But inspection of another Internet market, far smaller in scope
and depth than eBay, that had recently changed its feedback system to address retaliation issues suggested that the concern
about a negative influence on feedback quantity was merited.
As it turns out, our experiment suggests that different methods of suppressing retaliation do indeed influence feedback
quantity and market efficiency in differential ways. This was important information for the eventual selection of changes to
the eBay feedback system. But it also contributes to the formal evidence we have about these systems, evidence that
researchers can use to reach a deeper, formal understanding of such challenges.

4. Conclusion

Engineering puts scientific insight to a prescriptive test. Economic engineering is ‘directed’ research in realistic contexts.
Economic research, theory or lab experiments, are always done in some specific context; e.g., under assumptions of common
knowledge. Economic engineering is typically done in more complex contexts, but the contexts that are the ultimate objec-
tives of social science research.
In the engineering enterprise, theory and experiments provide useful maps and give engineers orientation in a messy
world. Circular feedback between these maps and the real-world can, on the one hand, benefit practitioners and society,
and on the other hand, benefit basic research as the usefulness of models and methods is learned from both successful
and failed design efforts. Economic engineering is also an avenue for discovering critical challenges for rich research ques-
tions. These characteristics of economic engineering complement more fundamental economic research, and so make the
two approaches a natural couple for advancing science and society.

Acknowledgements

We thank the members of the DFG-research group ‘‘Economic Engineering of Firms and Markets’’ in Cologne, Ben Greiner
and our other co-investigators in various behavioral and engineering research projects, as well as Hartmut Kliemt and two
anonymous referees for much stimulus. Bolton and Ockenfels gratefully acknowledge the support of the Deutsche
Forschungsgemeinschaft.

References

Ariely, D., Ockenfels, A., & Roth, A. E. (2005). An experimental analysis of ending rules in internet auctions. RAND Journal of Economics, 36(4), 891–908.
Bajari, P., & Hortaçsu, A. (2004). Economic insights from internet auctions. Journal of Economic Literature, 42(2), 457–486.
 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676 675

Bazerman, M. (2005). Conducting influential research: The need for prescriptive implications. Academy of Management Review, 30(1), 25–31.
Bergemann, D., & Morris, S. (2005). Robust mechanism design. Econometrica, 73(6), 1771–1813.
Bernanke, B. (2010). On the implications of the financial crisis for economics. In Remarks by the chairman board of governors of the federal reserve system
at a conference in Princeton, New Jersey. <http://www.federalreserve.gov/newsevents/speech/bernanke20100924a.pdf>.
Bolton, G., & Ockenfels, A. (2010). Does laboratory trading mirror behavior in real world markets? Fair bargaining and competitive bidding on Ebay. Working
paper, University of Cologne.
Bolton, G. E., Greiner, & B., Ockenfels, A., (2011). Engineering trust – Reciprocity in the production of reputation information. Working paper.
Bolton, G., Ockenfels, A., & Thonemann, U. (2010). Managers and students as newsvendors. Working paper.
Bolton, G. E., Ockenfels, A., & Ebeling, F. (2011). Information value and externalities in reputation building. International Journal of Industrial Organization, 29,
23–33.
Bolton, G. E., & Katok, E. (2008). Learning-by-doing in the newsvendor problem: A laboratory investigation of the role of the experience. Manufacturing and
Service Operations Management, 10, 519–538.
Bolton, G. E., Katok, E., & Ockenfels, A. (2004). How effective are online reputation mechanisms? An experimental investigation. Management Science, 50(11),
1587–1602.
Bolton, G. E., Katok, E., & Ockenfels, A. (2005). Cooperation among strangers with limited information about reputation. Journal of Public Economics, 89,
1457–1468.
Bolton, G. E., Loebbecke, C., & Ockenfels, A. (2008). Does competition promote trust and trustworthiness in online trading? An experimental study. Journal of
Management Information Systems, 25(2), 145–169.
Bolton, G. E., & Ockenfels, A. (2000). ERC – A theory of equity, reciprocity and competition. American Economic Review, 90(1), 166–193.
Bolton, G. E., & Ockenfels, A. (2006). Measuring efficiency and equity motives: A comment on ‘Inequality aversion, efficiency, and maximin preferences in
simple distribution experiments’. American Economic Review, 96(5), 1906–1911.
Bolton, G. E., & Ockenfels, A. (2009a). The limits of trust. In K. Cook, C. Snijders, & V. Buskens (Eds.), ETrust (pp. 15–36). New York: Russell Sage.
Buskens, V., & Raub, W. (2010). Rational choice research on social dilemmas. In R. Wittek, T. A. B. Snijders, & V. Nee (Eds.), Handbook of rational choice social
research. New York: Russell Sage Foundation.
Camerer, C. F. (2003). Behavioral game theory: Experiments in strategic interaction. Princeton, NJ: Princeton University Press.
Chen, Y., & Ledyard, J. (2006). Mechanism design experiments. Working paper.
Chen, Y., Harper, M., Konstan, J., & Li, S. X. (2010). Social comparisons and contributions to online communities: A field experiment on MovieLens. American
Economic Review, 100, 1358–1398.
Chen, Y., & Sönmez, T. (2006). School choice: An experimental study. Journal of Economic Theory, 127(1), 202–231.
Choi, J., Laibson, D., & Madrian, B. (2009). Reducing the complexity costs of 401 (k) participation through quick enrollment (TM). In D. A. Wise (Ed.),
Developments in the economics of aging. Chicago: University of Chicago Press.
Cooper, D. J., & Kagel, J. H. (in press). Other regarding preferences: A selective survey of experimental results. In: J. H. Kagel, A. E. Roth (Eds.), Handbook of
experimental economics (Vol. 2). Princeton, NJ: Princeton University Press.
Cramton, P., & Stoft, S. (2010). International climate games: From caps to cooperation. Research paper, Global Energy Policy Center.
Cramton, P., Shoham, Y., & Steinberg, R. (2006). Combinatorial auctions. Cambridge: MIT Press.
Cramton, P., & Stoft, S. (2008). Forward reliability markets: Less risk, less market power, more efficiency. Utilities Policy, 16, 194–201.
Crawford, V. P., Kugler, T., Neeman, Z., & Pauzer, A. (2009). Behaviorally optimal auction design: Examples and observations. Journal of the European
Economic Association, 7(2–3), 377–398.
Crocker, K. J., & Reynolds, K. J. (1993). The efficiency of incomplete contracts: An empirical analysis of air force engine procurement. The RAND Journal of
Economics, 24, 126–146.
Dellarocas, C., & Wood, C. A. (2008). The sound of silence in online feedback: Estimating trading risks in the presence of reporting bias. Management Science,
54(3), 460–476.
Drewes, R. W. (2005). The air force and the great engine wars. Honolulu: University Press of the Pacific.
Engelbrecht-Wiggans, R., Haruvy, E., & Katok, E. (2007). A comparison of buyer-determined and price-based multiattribute mechanisms. Marketing Science,
26(5), 629–641.
Engelmann, D., & Strobel, M. (2004). Inequality aversion, efficiency, and maximin preferences in simple distribution experiments. American Economic
Review, 94(4), 857–869.
Fehr, E., Fischbacher, U., & Tougareva, E. (2002). Do high stakes and competition undermine fairness? Evidence from Russia. IEW working paper No. 120,
University of Zurich.
Fehr, E., Klein, A., & Schmidt, K. M. (2007). Fairness and contract design. Econometrica, 75(1), 121–154.
Fehr, E., & Schmidt, K. M. (1999). A theory of fairness, competition, and cooperation. Quarterly Journal of Economics, 114, 817–868.
Fehr, E., & Tyran, J.-R. (2005). Individual irrationality and aggregate outcomes. Journal of Economic Perspectives, 19(4), 43–66.
Fehr, E., & Tyran, J.-R. (2008). Limited rationality and strategic interaction – The impact of the strategic environment on nominal inertia. Econometrica, 76(2),
353–394.
Friedman, E., & Resnick, P. (2001). The social cost of cheap pseudonyms. Journal of Economics and Management Strategy, 10(2), 173–199.
Greiner, B., Ockenfels, A., & Werner, P. (in press). Wage transparency and performance: A real-effort experiment. Economics Letters.
Grimm, V., Ockenfels, A., & Zoettl, G. (2008). Strommarktdesign: Zur Ausgestaltung der Auktionsregeln an der EEX. Zeitschrift für Energiewirtschaft, 32(3),
147–161.
Güth, W., Mengel, F., & Ockenfels, A. (2006). An evolutionary analysis of buyer insurance and seller reputation in online markets. Working paper, University
of Cologne.
Huck, S., Lünser, G. K., & Tyran, J. R. (2007). Competition fosters trust. CEPR Discussion Paper.
Kagel, J. H., & Roth, A. E. (2000). The dynamics of reorganization in matching markets: A laboratory experiment motivated by a natural experiment. Quarterly
Journal of Economics, 115(1), 201–235.
Kahn, A. E. (1979). Applications of economics to an imperfect world. The American Economic Review, 69(2), 1–13.
Kandori, M. (1992). Social norms and community enforcement. Review of Economic Studies, 59, 63–80.
Keser, C. (2003). Experimental games for the design of reputation management systems. IBM Systems Journal, 42(3), 498–506.
Kittsteiner, T., & Ockenfels, A. (2006). Market design: A selective review. Zeitschrift für Betriebswirtschaft, 5, 121–143.
Klein, T. J., Lambertz, C., Spagnolo, G., & Stahl, K. O. (2007). Reputation building in anonymous markets: Evidence from eBay. Working paper. University of
Mannheim.
Klemperer, P. (2004). Auctions: Theory and practice. Princeton: Princeton University Press.
Kotler, P., & Armstrong, G. (2007). Principles of marketing. New Jersey: Pearson, Prentice Hall.
Kreps, D. M., & Wilson, R. (1982). Reputation and imperfect information. Journal of Economic Theory, 27, 253–279.
Ledyard, J. O. (1993). The design of coordination mechanisms and organizational computing. Journal of Organizational Computing, 3(1), 121–134.
Lurie, N. H., & Swaminathan, J. M. (2007). Is timely information always better? The effect of feedback frequency on performance and knowledge acquisition.
Working paper, Georgia Institute of Technology.
Masten, S. (1988). A legal basis for the firm. Journal of Law, Economics and Organization, 4(1), 181–198.
Milgrom, P. (2004). Putting auction theory to work. Cambridge: Cambridge University Press.
Miller, N., Resnick, P., & Zeckhauser, R. (2005). Eliciting honest feedback: The peer prediction method. Management Science, 51(9), 1359–1373.
Mongell, S., & Roth, A. E. (1991). Sorority rush as a two-sided matching mechanism. American Economic Review, 81(3), 441–464.
676 G.E. Bolton, A. Ockenfels / Journal of Economic Psychology 33 (2012) 665–676

Moritz, B. (2008). In Behavior, cognition and individual differences in the newsvendor problem, presentation at the 3rd annual behavioral operations conference,
July 23–25, 2008, Edmonton, Canada.
Müsgens, F., & Ockenfels, A. (2006). Marktdesign in der Energiewirtschaft. In W. Franz, H. J. Ramser, & M. Stadler (Eds.), Umwelt und Energie,
Wirtschaftswissenschaftliches Seminar Ottobeuren (pp. 247–272). Tübingen: Mohr Siebeck.
Müsgens, F., Ockenfels, A., & Peek, M. (2011). Economics and design of balancing power markets in Germany. Working paper.
Nagel, R. (1995). Unraveling in guessing games: An experimental study. American Economic Review, 85(5), 1313–1326.
Ockenfels, A. (2003). Reputationsmechanismen auf Internet-Marktplattformen. Zeitschrift für Betriebswirtschaft, 73(3), 295–315.
Ockenfels, A. (2009a). Marktdesign und Experimentelle Wirtschaftsforschung. Perspektiven der Wirtschaftspolitik, 10(1), 31–53.
Ockenfels, A. (2009b). Empfehlungen für das Auktionsdesign für Emissionsberechtigungen. Zeitschrift für Energiewirtschaft, 2, 105–114.
Ockenfels, A., & Raub, W. (2010). Rational und fair. In Gert Albert, Steffen Sigmund (Eds.), Kölner Zeitschrift für Soziologie und Sozialpsychologie (Vol. 50, pp.
119–136).
Ockenfels, A., Reiley, D., & Sadrieh, A. (2007). Online auctions. In T. J. Hendershott (Ed.), Handbooks in information systems (pp. 571–628). North Holland:
Elsevier.
Ockenfels, A., & Resnick, P. (in press). Negotiating reputations. In: G. Bolton & R. Croson (Eds.), The Oxford handbook of conflict resolution. Oxford University
Press..
Ockenfels, A., & Roth, A. E. (in press). Ending rules in internet auctions: Design and behavior. Oxford Market Design Handbook.
Ockenfels, A., Sliwka, D., & Werner, P. (2011). Bonus payments and reference point violations. Working paper.
Ockenfels, A., & Roth, A. E. (2006). Late and multiple bidding in second price internet auctions: Theory and evidence concerning different rules for ending an
auction. Games and Economic Behavior, 55, 297–320.
Ockenfels, A., & Selten, R. (2005). Impulse balance equilibrium and feedback in first price auctions. Games and Economic Behavior, 51, 155–170.
Ostrom, E. (1990). Governing the commons: The evolution of institutions for collective action. Cambridge: Cambridge University Press.
Plott, C. R. (1994). Market architectures, institutional landscapes and testbed experiments. Economic Theory, 4(1), 3–10.
Plott, C. R. (1997). Laboratory experimental testbeds: Application to the PCS auction. Journal of Economics and Management Strategy, 6, 605–638.
Rashid, A., Ling, K., Tassone, R. D., Resnick, P., Kraut, R., & Riedl, J. (2006). Motivating participation by displaying the value of contribution. In: Proceedings of
ACM CHI 2006 (pp. 955–958).
Resnick, P., & Zeckhauser, R. (2002). Trust among strangers in internet transactions: Empirical analysis of eBay’s reputation system. Advances in Applied
Microeconomics, 11, 127–157.
Resnick, P., Zeckhauser, R., Swanson, J., & Lockwood, L. (2006). The value of reputation on eBay: A controlled experiment. Experimental Economics, 9, 79–101.
Roth, A. E. (1990). New physicians: A natural experiment in market organization. Science, 250, 1524–1528.
Roth, A. E. (1991). A natural experiment in the organization of entry level labor markets: Regional markets for new physicians and surgeons in the UK.
American Economic Review, 81(3), 415–440.
Roth, A. E. (2002a). The economist as engineer: Game theory, experimental economics and computation as tools of design economics. Econometrica, 70(4),
1341–1378.
Roth, A. E. (2002b). Slides prepared for Al’s ‘‘Experimental Economics’’ class. <http://www.courses.fas.harvard.edu/%7Eec2040/>.
Roth, A. E. (2008). What have we learned from market design? Economic Journal, 118(527), 285–310.
Roth, E. M., Patterson, E. S., & Mumaw, R. J. (2002). Cognitive engineering: Issues in user-centered system design. In J.J. Marciniak (Ed.), Encyclopedia of
software engineering (2nd ed.). New York: John Wiley and Sons.
Roth, A. E., & Ockenfels, A. (2002). Last-minute bidding and the rules for ending second-price auctions: Evidence from eBay and Amazon auctions on the
internet. American Economic Review, 92(4), 1093–1103.
Roth, A. E., & Peranson, E. (1999). The redesign of the matching market for American physicians: Some engineering aspects of economic design. American
Economic Review, 89(4), 748–779.
Roth, A. E., Prasnikar, V., Okuno-Fujiwara, M., & Zamir, S. (1991). Bargaining and market behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo. American
Economic Review, 81(5), 1068–1095.
Roth, A. E., Sönmez, T., & Ünver, M. U. (2004). Kidney exchange. Quarterly Journal of Economics, 119(2), 457–488.
Roth, A. E., & Xing, X. (1994). Jumping the gun: Imperfections and institutions related to the timing of market transactions. American Economic Review, 84(4),
992–1044.
Rousseau, D. (2006). Is there such a thing as ‘‘Evidence-based management?’’. Academy of Management Review, 31(2), 256–269.
Schweitzer, M. E., & Cachon, G. P. (2000). Decision bias in the newsvendor problem with a known demand distribution: Experimental evidence. Management
Science, 46(3), 404–420.
Selten, R. (1998). Features of experimentally observed bounded rationality. European Economic Review, 42, 413–436.
Sonnenschein, H. (1983). The economics of incentives: An introductory account. Nancy L. Schwartz Memorial Lecture, J.L. Kellogg School of Management,
Northwestern University, Chicago.
Sunstein, R., & Thaler, R. (2008). Nudge: Improving decisions about health, wealth, and happiness. London, New Haven: Yale University Press.
vanDamme, E. (2001). The Dutch UMTS-auction in retrospect. CPB Report 2001/2 (pp. 25–30).
Williamson, O. E. (1979). Transaction-cost economics: The governance of contractual relations. Journal of Law and Economics, 22(2), 233–261.
Wilson, R. (1985). Reputations in games and markets. In Alvin Roth (Ed.), Game theoretic models of bargaining (pp. 27–62). Cambridge University Press.
Wilson, R. (1987). Game-theoretic analyses of trading processes. In: Truman Bewley (Ed.), Advances in economic theory: Fifth world congress (pp. 33–70).
Cambridge: Cambridge University Press.
Wilson, R. (2001). Architecture of power markets. Econometrica, 70(4), 1299–1340.

Further reading

Bolton, G. E., & Ockenfels, A. (2009b). Testing and modeling fairness motives. In Michael. Bourmann & Bernd. Lahno (Eds.), Perspectives in moral sciences,
contributions from philosophy, economics, and politics in honour of Hartmut Kliemt (pp. 199–206). Frankfurt: Frankfurt School Verlag.
Greiner, B., Ockenfels, A., & Sadrieh, A. (in press). Internet auctions. Oxford Handbook of the Digital Economy.
Greiner, B., & Ockenfels, A. (2009). Vom labor ins feld: Die Ökonomik des vertrauens. In Jens Becker, Christoph Deutschmann (Ed.), Kölner Zeitschrift für
Soziologie und Sozialpsychologie (Vol. 49, pp. 219–242).