Dynamics of bid optimization

in online advertisement auctions

Christian Borgs Jennifer Chayes Omid Etesami∗

Microsoft Research Microsoft Research U.C. Berkeley
Redmond, WA Redmond, WA Berkeley, CA
borgs@microsoft.com jchayes@microsoft.com etesami@cs.berkeley.edu

Nicole Immorlica Kamal Jain Mohammad Mahdian†

Microsoft Research Microsoft Research Yahoo! Research
Redmond, WA Redmond, WA Santa Clara, CA
nickle@microsoft.com kamalj@microsoft.com mahdian@yahoo-inc.com

Categories and Subject Descriptors
F.2 [Theory of Computation]: ANALYSIS OF AL-
GORITHMS AND PROBLEM COMPLEXITY
Keywords
Advertisement Auctions, Bidding Algorithms
ABSTRACT
We consider the problem of online keyword advertising
auctions among multiple bidders with limited budgets,
and study a natural bidding heuristic in which advertis-
ers attempt to optimize their utility by equalizing their
return-on-investment across all keywords. We show that
existing auction mechanisms combined with this heuris-
tic can experience cycling (as has been observed in many
current systems), and therefore propose a modified class
of mechanisms with small random perturbations. This
perturbation is reminiscent of the small time-dependent
perturbations employed in the dynamical systems liter-
ature to convert many types of chaos into attracting mo-
tions. We show that the perturbed mechanism provably
converges in the case of first-price auctions and experi-
mentally converges in the case of second-price auctions.
Moreover, the point of convergence has a natural eco-
nomic interpretation as the unique market equilibrium
in the case of first-price mechanisms. In the case of
second-price auctions, we conjecture that it converges
to the “supply-aware” market equilibrium. Thus, our
results can be alternatively described as a tâtonnement
process for convergence to market equilibrium in which
prices are adjusted on the side of the buyers rather than
∗
Work performed while author was an intern at Mi-
crosoft Research.
†
Work performed while author was a postdoc at Mi-
crosoft Research.
Copyright is held by the author/owner(s).
WWW2007, May 8–12, 2007, Banff, Canada.
. interested advertisers with remaining budget, typically
the sellers. We also observe that perturbation in mech-
anism design is useful in a broader context: In general,
it can allow bidders to “share” a particular item, lead-
ing to stable allocations and pricing for the bidders, and
improved revenue for the auctioneer.
1. INTRODUCTION
Online search engine advertising is become an increas-
ingly important and costly component of the market-
ing and sales strategies of many businesses. The cor-
responding auctions are the main source of revenue for
many search engines and other Internet-related busi-
nesses. It is therefore of tremendous interest to under-
stand and analyze the behavior of these auction sys-
tems, and to try and ensure that the system functions
smoothly. In this paper, we consider the advertisement
auction system as a whole and from a dynamic perspec-
tive. We first define a simple and natural bidding heuris-
tic for budget-limited advertisers based on equalizing
the “return-on-investment” (ROI) across keywords. We
then observe that, when used by a set of advertisers,
multiple copies of this heuristic may induce cycling be-
havior into the system. We propose circumventing this
undesirable effect by introducing random perturbations,
and see that this modified system converges to the mar-
ket equilibrium (provably for first-price auctions and
experimentally for second-price auctions). Thus our re-
sults may alternatively be interpreted as providing a
tâtonnement process for convergence to market equilib-
rium in which prices are adjusted on the side of the
buyers rather than the sellers.
Online search engine advertising is typically sold via
keyword auctions (see, for example, Google’s AdWords,
Yahoo’s Search Marketing, and MSN’s AdCenter). Each
prospective advertiser chooses a set of keywords relevant
to his products, and for each keyword submits a bid rep-
resenting an estimate of his utility for a click when that
word is displayed. He also submits a maximum bud-
get which must be respected for the chosen time period.
When each keyword appears, it is auctioned among all
using a first-price or second-price auction mechanism
(see [5, 1, 11, 16] for a comparison of these approaches).
As bidders have limited budgets, the bid optimization
problem they face is essentially a discrete separable re-
source allocation problem [7]. One of the most popu-
lar metrics to assess the efficiency of various investment
strategies is marginal “return-on-investment,” which in
this context can be taken as the derivative of the util-
ity with respect to the price. (See Section 3 for precise
definitions.) Here we use an easily computable approx-
imation to this quantity, namely the ratio rather than
the derivative. For a particular advertiser, we define
the ROI of a keyword at a given bid to be the ratio of
the utility of this word to the price of the word, both
at the given bid. In the bidding heuristic we consider,
each budget-constrained advertiser bids an amount such
that his ROI is equal across all keywords. Such heuris-
tics are common in practice and have been proposed in
other theoretical contexts as well [15].
Assume that the above bidding heuristic is employed
by a set of advertisers. Two questions immediately
arise. First, does there exist an underlying mechanism
which causes these algorithms to converge? Second, if a
convergent mechanism does exist, to what does it con-
verge? In particular, how does this system impact rev-
enue for the search engine provider? It is important
to note that we consider these questions in light of the
bidding dynamics defined by the specified heuristics, as-
suming all bidders adhere to these heuristics and use
them truthfully regardless of the optimality of such a
strategy. In particular, we do not study the properties
of these systems in a strategic equilibrium.1
The first question, namely the existence of a conver-
gent mechanism, is more than just a theoretical ques-
tion. Indeed, what appears to be chaotic cycling be-
havior has been observed in actual search engine auc-
tions [11].2 Moreover, for straightforward mechanisms
used in conjunction with the ROI bidding heuristic, we
can easily construct two-bidder examples which exhibit
cycling, with the allocation oscillating between the bid-
ders. These observations and examples are not surpris-
ing in light of the general phenomenon of heteroclinic
cycles that can occur in both continuous [6] and dis-
crete [14] dynamic systems with symmetry, sometimes
leading to cycling chaos [3, 13].
In order to overcome this, we introduce an online
random bid perturbation into our algorithm. In some
sense, this perturbation is reminiscent of the small time-
dependent perturbations employed in the dynamical sys-
tems literature to convert many types of chaos into at-
1
A major difficulty in studying this setting as a strate-
gic game is the repeated nature of the game. Folklore
theorems show that repeated games (such as this one)
have a plethora of equilibria, thereby making equilib-
rium analysis (without any restriction on the set of avail-
able strategies) unsuitable for predicting the behavior
of the system. In this work, we are taking a different
route: we fix a particular bidding strategy (whose vari-
ants are used in practice) and analyze the equilibrium
this strategy.
2
For an alternative justification of observed cycling pat-
terns see [17].
tracting motions [12]. In mechanism design, perturba-
tion has been proposed previously as a solution to spite-
ful bidding (bidding strategies which attempt to drive
out competition by exhausting their budgets) [10]. Our
results further motivate the introduction of perturba-
tions to mechanism design as a technique for smoothing
the dynamics of the system and permitting bidders to
“share” items in arbitrary ratios.
Indeed, in the case of a first-price auction, we prove
that the introduction of random perturbations causes
the mechanism to converge. This is by far the most
technically complex part of the paper. We conjecture
that the random perturbations will also eliminate cy-
cling behavior and lead to convergence of an analogous
second-price auction, a conjecture which is supported
by simulations in Section 5. Furthermore, we can prove
that, in the case of the perturbed first-price auction, the
prices (and hence revenue) of our system converges to
the unique market equilibrium. As a side note, this also
gives an algorithm for computing the market equilib-
rium in our setting (incidentally, the algorithm is quite
similar to that of Devanur et al. [4] for computing mar-
ket equilibria), as well as a tâtonnement process for con-
vergence to market equilibrium in which prices are ad-
justed on the side of the buyers rather than the sellers.
All of our results are supported by simulations, which
we discuss in Section 5.
2. MODEL
Search engines often display advertisements alongside
search results when a user performs a search. These
advertisements appear in a dedicated area of the search
results page, each one in a particular fixed subarea, or
slot. An online advertisement auction is a mechanism
for selling these slots based on the keyword which the
user provided to the search engine.
We consider a setting in which m advertisers bid for
the advertising slots of n keywords. Each keyword j has
l slots and appears qj (t) times on day t (by “day” we
mean some fixed unit of time; it does not necessarily
have to be 24 hours). Advertiser i has a value vij for
each click received when his advertisement is displayed
on keyword j. Note that while advertisers value clicks,
our auction is actually selling impressions, or the chance
to appear in a keyword slot. We can convert the values
per click to an expected value per impression uijk by
taking the product of vij with the probability cijk that
advertiser i receives a click when displayed in slot k of
keyword j. This probability is called the click-through-
rate. We assume these click-through-rates factor, that
is, there exist βij for each bidder i and keyword j, and
αk for each slot k (independent of the advertiser and
keyword)3 such that cijk = βij αk . Thus the per impres-
sion bid uijk for the k’th slot can be written as αk uij
for some uij . We number slots in order of decreasing
click-through-rate so α1 ≥ α2 ≥ . . . ≥ αl and without
loss of generality assume α1 = 1.
Each advertiser submits a bid bij for each keyword
3
The assumption that the click-through-rate can be de-
composed in this way is a reasonable assumption and is
used in practice.
representing the amount he is willing to pay for one im-
pression in slot 1 of keyword j (i.e., uij above). By
extension, we assume he is willing to pay αk bij for an
impression in slot k of keyword j.4 Advertisers addi-
tionally submit a daily budget Bi indicating the maxi-
mum amount they are willing to spend in a given day.
Although in general these parameters may be adjusted
at arbitrary times, for simplicity we assume they are
updated at most daily and in the beginning of the day.
Upon a search for a particular keyword j, the adver-
tisement auction then selects up to l advertisers i1 , . . . , il
and assigns them to slots 1, . . . , l, respectively. It then
computes a price pjk for each advertiser ik ∈ {i1 , . . . , il }.
The auction guarantees no bidder is charged more than
his bid nor exceeds his budget. Furthermore, no bidder
is awarded more than one slot per search query. We
focus our attention on two particular auction mecha-
nisms quite common in practice. The first is a first-
price mechanism in which advertisers are awarded slots
in a priority order determined by their bids. Advertis-
ers are then charged a price equal to the minimum of
their bid and remaining budget. The second mechanism
is a generalization of the second-price mechanism. The
allocation rule of this mechanism is identical to that of
the first-price mechanism, but the pricing scheme is dif-
ferent. Each advertiser is now charged a price equal to
the minimum of his remaining budget and the bid of
the advertiser in the next slot. The pseudocode of these
two mechanisms appears in Figure 1.
For our theoretical results, we simplify the model in
the following ways. First, we study a setting in which
there is only one slot per keyword. The single-slot set-
ting is rich enough to capture the chaotic behavior our
results circumvent and thus suffices to illustrate our
main points.5 Second, we consider a continuous-time
version of the auction: For each keyword j, there are
a constant number qj of searches each day, and these
searches are evenly spaced throughout the day. We as-
sume qj ’s are large and therefore we can model this
process as one in which all keywords arrive continuously
at a uniform rate throughout the day. The daily budget
of advertiser i is Bi , and the total utility of advertiser i
for showing his ad on keyword j throughout the entire
day is uij (thus, his utility for being shown during an α
fraction of the day is P αuij ). Without loss of generality
we will assume Bi ≤ j uij .
3. BID OPTIMIZATION HEURISTICS
In this section we describe a natural bidding heuristic
for optimizing the utility of the advertisers. We consider
the following abstraction of the bid optimization prob-
lem for advertiser i. We want to specify a bid bij on each
4
Note we could just have easily described our results for
a setting where advertisers submit a bid per click if we
assume the click-through-rates of advertisers and slots
are known or estimated.
5 6
In fact, it is straightforward to generalize our con-
vergence result (Theorem 1) to the multi-slot setting
(essentially the only thing that needs to be changed is
Equation 1). However, the point to which the system
converges can no longer be characterized as a market
equilibrium.
keyword j. We assume that if advertiser i bids bij on
keyword j then his day-long charge and net utility (i.e.,
total value minus total charge) on that keyword is given
by Pj (bij ) and Uj (bij ) respectively.6 The P optimization
problem is now to choose {bij } such that j Uj (bij ) is
P
maximized subject to j Pj (bij ) ≤ Bi . Through the
use of Lagrangian relaxation, we see that a necessary
condition for the optimality of bids b∗ij is the existence
of a constant λ (the Lagrangian multiplier) such that
for all j with Uj (b∗ij ) > 0,
d Uj /d Pj |bij =b∗ij = λ
if such derivatives exist. This derivative is known as
the marginal return-on-investment (marginal ROI) and
measures how the net utility of an advertiser changes as
he modifies his investment. Thus, for an optimal set of
bids {b∗ij }, we know advertiser i has the same marginal
ROI at b∗ij across all keywords. This marginal ROI is
exactly the Lagrangian multiplier λ above.
The marginal ROI is usually difficult to estimate,
and is even undefined when Pj or Uj are discontinuous.
Thus, it is useful to approximate the marginal ROI of
keyword j at bid b by the ROI of keyword j at that
bid, where ROI is defined as ROIj (b) = Uj (b)/Pi (b).
This suggests one method for optimizing the bids of the
advertiser: set the bids bij such that ROIj (bij ) approx-
imately equals some constant ROI for all j.
If the prices were fixed and known to the advertiser,
determining an optimal biding vector would be a sim-
ple calculation. Suppose the price of the kth slot for
keyword j is pjk . We further introduce an artificial slot
l + 1 with price zero and utility zero indicating that the
advertiser does not appear in any slot on that keyword.
A bidding strategy is now a selection of affordable slots
sj ∈ {1, . . . , l + 1} for each keyword j, where a selection
is affordable if the sum of prices is at most the budget
of the advertiser. This problem is a natural extension
of the knapsack problem [8] and has a similar FPTAS.
In fact, the idea of the ROI heuristic is similar to the
well-known 2-approximation for knapsack. It tries to
maintain the invariant that for some constant R = ROI,
R ∈ (ujsj /pjsj , uj(sj +1) /pj(sj +1) ] for all keywords j, and
searches for the maximum possible R subject to the bud-
get constraint. Thus, if the advertiser has budget left
over at the end of the day, he finds the keyword j with
minimum ujsj /pjsj and chooses slot sj +1 for keyword j
on the following day. Otherwise, if he ran out of budget
early, he finds the keyword j with maximum ujsj /pjsj
and chooses slot sj −1 for that keyword on the following
day.
An alternative way to implement the ROI heuristic
is through a tâtonnement-like process, where the ad-
vertiser iteratively incrementing bids on keywords with
relatively large ROI and decrementing bids on keywords
with relatively small ROI by small increments. The ad-
Note that we assume the charge and net utility of ad-
vertiser i for keyword j is a function of his bid for key-
word j alone and does not depend on the bids of i for
other keywords. Although this is not strictly true, it is
a reasonable approximation and serves to develop our
intuition for our heuristic.
 First-Price Mechanism
Let S be the set of bidders {i : si ≤ Bi }.
For k = 1 to l do
Let i = argmaxi∈S (bij ),
Set S = S − {i},
Assign i to slot k,
Charge i price min(αk bij , Bi − si ).
Figure 1: Pseudocode for the first and second-price auctions, respectively. The parameter si is the
current total daily charge of advertiser i.
vantage of this method is that it requires the minimal
amount of information. In particular, it does not even
need to know the price of the slots above and below the
current slot. It is easy for an advertiser to calculate the
ROI for each keyword in hindsight at the end of the day.
Based on this idea, we consider the following ROI-based
heuristic bidding algorithm for advertiser i.
Algorithm 1. On each day t, all bids of advertiser
i are determined by a single parameter Ri (t) ∈ (0, 1].7
The parameter Ri (t) is adjusted based on the perfor-
mance of advertiser i’s bids on the previous day. Start-
ing from an arbitrary Ri (0) ∈ (0, 1] for day t = 0, ad-
vertiser i sets
8 $0.5e² , and b bids $0.5. Bidder a is going to win all
< Ri (t)e−² if i runs out of money
Ri (t+1) = before the end of day t
: end of the day, but he is going to decrease his bid for
min(Ri (t)e² , 1) otherwise
where ² > 0 is a small constant. Finally, he sets the bid
bij (t) of keyword j to
bij (t) = Ri (t)uij .
Note since Ri (t) ∈ (0, 1], bij (t) ≤ uij .
Before discussing the dynamics of this algorithm, let
us note that an added advantage of the above bidding
heuristic is that it can be adapted to cases where an ad-
vertiser only knows her budget and the relative utility
of various keywords (i.e., the ratio of uij ’s), and not the
exact value of the utilities. In this case, the bidding al-
gorithm for advertiser i can initially set her largest util-
ity to Bi and the other utilities according to the spec-
ified ratios, and then adjust these values by changing
Ri (t). This is useful in practice since for an advertiser
estimating the ratio of the values of various keywords is
a considerably simpler task than estimating the exact
utilities.
4. DYNAMICS OF THE SYSTEM
In Section 3 we defined a heuristic for bidding in an
advertisement auction. In order to better understand
the properties of a system where bidders are using such
a heuristic, we need to analyze the interplay of bidding
7
This parameter is related to the target return-on-
investment by Ri (t) = 1/(ROI + 1) where ROI is the
target return on investment of advertiser i.
Second-Price Mechanism
Let S be the set of bidders {i : si ≤ Bi }.
For k = 1 to l do
Let i = argmaxi∈S (bij ),
Set S = S − {i},
Assign i to slot k,
Charge i price min(αk max 0
bi0 j , Bi − si ).
i ∈S
algorithms of various bidders. One might wonder if such
a system could ever stabilize, and whether the resulting
prices would be logical in some sense (i.e., be simulta-
neously “reasonable” for the advertisers and generate
sufficient revenue for the search engine). In fact, the
following example shows that the combination of the
first-price auction with the ROI heuristic may result in
an unstable situation with low prices.
Example 1. Suppose there is just one keyword with
one slot and 1000 impressions. There are two advertis-
ers a and b, each advertiser with a budget of $500 and
a utility of $1 for each impression of the keyword. Con-
sider the first-price auction mechanism. Assume a bids
the impressions until he runs of the budget around the
tomorrow to $0.5, since he ran out of budget today. On
the other hand, b is going to bid $0.5e² on the following
day. Thus, a and b will interchange roles. This way the
allocation of the impressions alternates between a and b
daily.
It is easy to see that the above example works for the
second price mechanism as well. The results of Section 5
confirm that such examples arise in a variety of plausible
scenarios, resulting in oscillating allocations and damp-
ened revenue. We avoid such situations by applying
a random perturbation to the bids of the advertisers in
determining the allocation, as defined below. In this sec-
tion we study variants of the first and second-price auc-
tions with perturbations. We prove that the perturbed
first-price auction, coupled with multiple copies of the
bid optimization algorithm presented in Section 3, con-
verges to a fixed allocation and set of prices correspond-
ing to the market equilibrium. We conjecture a similar
result for the perturbed second-price auction, support-
ing our conjecture with simulation results in Section 5.
4.1 Perturbations
In order to get rid of situations like the one explained
in Example 1, we modify the auction mechanism to
slightly perturb the bids before running the auction,
thereby giving the bidder with a smaller bid some chance
of winning if his bid is close to the largest bid. The
perturbations are defined as follows. On each day t, ad-
vertiser i bids a value bij (t) for the day-long possession
of keyword j. When a search on keyword j occurs, we
perturb the bids as follows:
b0ij = bij (t) exp(−ηi ),
where ηi is a uniformly random number in [0, δ]8 , in-
dependently generated for each bidder/query pair, and
δ > 0 is a constant. The auction mechanisms are run
exactly as described in Section 2, but the allocation is
determined according to the perturbed bids b0ij (t).
Perturbations essentially allow advertisers to bid such
that they share the keyword in any portion they please.
That is, fixing the bids of other advertisers on a partic-
ular keyword, a given advertiser can choose to receive
in expectation any fraction α of the day-long proces-
sion of the keyword by adjusting his bid appropriately.
Note that such a sharing property can not be achieved
by introducing a randomized tie-breaking rule; applying
the perturbation to the bids themselves is significantly
more powerful. Notice how this affects the advertisers
in Example 1.
Example 2. Again, consider the scenario from the
previous example. However, now suppose the bids are
perturbed as described above and notice the instability we
observed before won’t happen. Indeed a and b share the
impressions almost equally in expectation, and so nei-
ther bidder runs out of budget. Therefore, they will in-
crease their bids until their bids get close to $1 at which
time both the price and allocations remain stable. In
this case the perturbation both removed the cycling and
improved auctioneer’s revenue by a factor of two.
4.2 Convergence to Equilibria
We now discuss our main theoretical results, namely
the convergence properties of our perturbed mechanisms
with multiple bid optimization algorithms. Throughout
the remainder of this section, we assume there is just
one slot per keyword.9
We consider both perturbed first-price and perturbed
second-price auctions. In each of these auctions, the
allocation rule awards the keyword slot to the bidder
with the highest perturbed bid b0ij . The winning ad-
vertiser is then charged a price equal to the minimum
of his remaining budget and unperturbed bid bij in the
case of the first-price auction10 , or the minimum of his
remaining budget and the perturbed bid of the closest
competitor in the case of the second-price auction. Once
the spending of an advertiser during a day reaches his
daily budget, he is withdrawn from all further auctions
during that day.
We now state our principal result. Namely, we prove
that in a perturbed first-price auction where bidders bid
according to the ROI heuristic, Algorithm 1 of Section 3,
both the prices and the daily utilities of the advertisers,
8
The choice of the distribution for perturbation is essen-
tially arbitrary, and our results hold for other reason-
able perturbation models (e.g., Gaussian perturbations)
as well.
9
It is not hard to see that Theorem 1 holds for the
multi-slot case with essentially the same proof.
10
Note that our results hold if the pricing rule charges
the winning bidder his perturbed bid b0ij as well.
and hence the revenue of the auctioneer, converge to
that of the market equilibrium in the sense of Arrow
and Debreu [2] when goods correspond to the ad spaces
and the money (see Appendix A).
More formally, let si (t) ∈ [0, Bi ] denote the spending
of advertiser i on day t. Let τi (t) ∈ [0, 1] denote the
moment during day t when advertiser i spends all his
budget (or 1 if he does not spend all his budget). Finally
let ri (t) denote the spending rate of advertiser i in the
beginning of the day before anyone runs out of budget.
In other words,
X
n Z δ Y
bij (t)
ri (t) = Pr[bij (t)e−x > bi0 j (t)e−ηi0 ]dx
δ 0 ηi0
j=1 i0 6=i
(1)
Note that the rate of spending only increases as other
advertisers run out of budget, and therefore we have
si (t) ≥ ri (t)τi (t). We first show these parameters con-
verge, namely, that after some time no advertiser runs
out of budget early and each advertiser either spends
most of his budget or is bidding nearly his utility on all
keywords. The proof of the following theorem appears
at the end of this section.
Theorem 1. Given utilities uij , budgets Bi , and con-
stants δ > 0 and γ > 0, there exist constants ² > 0 and
t0 < ∞, such that for all t ≥ t0 and all i, we have
1. τi (t) ≥ 1 − γ, and
2. si (t) ≥ (1 − γ)Bi or Ri (t) ≥ 1 − γ.
Here ² and t0 can be chosen as ² = Θ(γ min{1, δ/C 2P })
and t0 = (2 log C)/²−log(mini Ri (0)) with C = maxi ( j uij /Bi ).
The above theorem allows us to characterize the equi-
librium of our system. Let Li (t) = Bi − si (t) be the un-
used portion of advertiser i’s budget at the end of day
t. Then the following theorem holds.
Theorem 2. Given δ > 0 and γ > 0, let t = t(δ, γ) ≥
t0 , where t0 is defined as in Theorem 1. Let pj (t) be the
maximum price at which keyword j is sold in day t, and
let xij (t) be the fractional daily allocation of word j to
advertiser i on day t. As δ, γ go to zero, the price vec-
tor pj (t) converges to that of the market equilibrium,
and the total utilities Pof the advertisers including their
unused budgets, Li + j uij xij (t), converge to the util-
ities of an equilibrium allocation.
Notice that convergence of the price
P vector implies
also convergence of the total revenue i pi for the auc-
tioneer. The proof of Theorem 2, which makes substan-
tial use of the stability results in Theorem 1, is deferred
to Appendix A.
Proof of Theorem 1. We first show Statement 1,
i.e. that after some finite time nobody runs out of bud-
get early. More precisely, we will show that for every
0 < λ < 1, ² small enough and t ≥ Tλ (where Tλ is a
constant depending on λ), we have τi (t) ≥ 1 − λ for all
1 ≤ i ≤ n. Let k(t) be the first advertiser who finishes
his budget on day t. The proof of Statement 1 follows
from the following two claims.
 Claim 1. If τk(t) (t − 1) < 1, then
² lemma and our condition on ², we have
τk(t) (t) ≥ min(e τk(t) (t − 1), 1).
Claim 2. If τk(t) (t − 1) = 1, then τk(t) (t) ≥ 1 − λ,
provided ² is chosen in such a way that 2C²e² ≤ λδ.
To see that these two claims imply Statement 1 of
the theorem, set τmin (t) = mini τi (t). Claims 1 and 2
− λ, e² τmin (t − 1)). We
together imply τmin (t) ≥ min(1P
know that τmin (1) ≥ mini Bi /( j uij ) = 1/C. There-
fore for t ≥ Tλ = ²−1 log(C(1 − λ)), we have τmin (t) ≥
1 − λ, as required.
Proof of Claim 1. Throughout this proof, let k =
k(t). If τk (t) = 1, then the claim is true. Assume
τk (t) < 1. Note that since τk (t − 1) < 1, Rk (t) =
Rk (t − 1)e−² and for i 6= k, Ri (t) ≥ Ri (t − 1)e−² . Con-
sider an imaginary scenario in which on day t, R̂i (t) =
Ri (t − 1)e−² for all bidders i. By (1), the spending
rate r̂k (t) of bidder k in the imaginary scenario is at
least that of the real scenario (r̂k (t) ≥ rk (t)). Further-
more, r̂k (t) = rk (t − 1)e−² since advertisements in the
imaginary scenario are sold to advertisers with the same
probabilities as day t − 1 and at a price e−² times the
price of day t − 1. Therefore, we have
rk (t − 1)τk (t − 1) ≤ Bk = τk (t)rk (t) ≤ τk (t)rk (t − 1)e−²
which implies Claim 1.
In order to prove Claim 2, we first prove the following
lemma.
Lemma 1. For all t and all i, we have |ri (t) − ri (t −
1)| ≤ (2C²e² /δ)Bi .
Proof. Note that Ri (t) ≤ Ri (t − 1)e² and Ri0 (t) ≥
−² 0
Ri0 (t − 1)e for i 6= i. Consider an imaginary scenario
in which on day t, R̂i (t) = Ri (t − 1)e2² and R̂i0 (t) =
Ri0 (t−1) for i0 6= i. Then R̂i (t) ≥ e² Ri (t) and R̂i (t)/R̂i0 (t) ≥ nor (3) holds, we saw in the above paragraph that Ri (t) =
Ri (t)/Ri0 (t), which implies that now r̂i (t) ≥ ri (t)e² . We
0
couple the perturbed bids b̂i0 j (t) of the imaginary sce-
nario with the perturbed bids b0i0 j (t − 1) of day t − 1
in such a way that b̂0i0 j (t) = b0i0 j (t − 1) if i0 6= i and
Pr[b̂0ij (t) 6= b0ij (t − 1)] = 2²/δ. Namely, we set
 next section for evidence of this in simulation results).
b0ij (t − 1) if b0ij (t − 1) ≥ b̂ij (t) exp(−δ)
b̂0ij (t) =
b0ij (t − 1)eδ otherwise
As the ratio of b̂ij (t) to bij (t − 1) is e2² , it is easy to
see that this coupling results in the desired probability.
Thus, even if advertiser i wins all auctions in which
b̂0ij (t) 6= b0ij (t−1) (which happens at most a 2²/δ fraction
of the times), we have
2² X 2²
r̂i (t) ≤ ri (t − 1) + uij e2² ≤ ri (t − 1) + CBi e2²
δ j δ market equilibrium (see [9]). A supply-aware equilib-
Using that r̂i (t) ≥ ri (t)e² , this implies ri (t) ≤ ri (t−1)+
(2C²e² /δ)Bi . The matching upper bound on ri (t − 1) in
terms of ri (t) is proved by exchanging the roles of t and
t − 1.
Proof of Claim 2. Let k = k(t). By the previous
rk (t) ≤ rk (t − 1) + λBk ≤ Bk (1 + λ) = rk (t)τk (t)(1 + λ)
where we used the assumption τk (t − 1) = 1 to conclude
that rk (t−1) ≤ Bk . This gives τk (t) ≥ 1/(1+λ) ≥ 1−λ,
proving the claim.
Now we will prove Statement 2. Note that ri (t) ≥
Bi (1−γ) implies si (t) ≥ Bi (1−γ) (this is because either
si (t) = Bi or τi (t) = 1 in which case si (t) ≥ ri (t)).
Therefore, it is enough to show that for all t ≥ 2Tλ −
log(mini Ri (0)) and all i, one of the following holds:
ri (t) ≥ (1 − γ)Bi , (2)
Ri (t) ≥ e−² (3)
so long as ² is less than γ. We first prove the following
claim.
Claim 3. For 2Cλ ≤ γ, 4C²e² ≤ γδ, and (t − 1) ≥
Tλ , we have si (t − 1) − ri (t) ≤ γBi .
Proof. By Statement 1, τmin (t − 1)P ≥ (1 − λ), and
therefore si (t − 1) ≤ ri (t − 1)(1 − λ) + λ j uij ≤ ri (t −
1) + γBi /2 provided 2Cλ ≤ γ. Moreover, by Lemma 1
and our condition on ², we have ri (t−1) ≤ ri (t)+γBi /2.
Therefore si (t − 1) ≤ ri (t) + γBi .
The proof of Statement 2 now follows by backwards
induction. First suppose neither (2) nor (3) holds on day
t and t−1 ≥ Tλ . We will show neither inequalities holds
on day t − 1. Indeed, by the above claim, si (t − 1) ≤
ri (t)+γBi < Bi and hence Ri (t) = min(Ri (t−1)e² , 1) ≥
Ri (t−1). Therefore (3) did not hold on day t−1 as well,
which implies that Ri (t) = Ri (t − 1)e² . Now using an
argument similar to Claim 1, we can show that ri (t) ≥
ri (t − 1)e² . It follows that (2) did not hold on day t − 1
either.
For the base case, notice that as long as neither (2)
Ri (t−1)e² and so for t ≥ 2Tλ −log(mini Ri (0)), inequal-
ity (3) will hold.
The above result shows that the prices in a perturbed
first-price mechanism converge. We believe that a simi-
lar result holds for a perturbed second price auction (see
However, our proof technique fails for the second price
auction. Given the convergence result, in Theorem 2
(whose proof is presented in Appendix A) we show that
for the first price auction, the prices converge to the
market equilibrium prices. For the second-price auc-
tion, assuming our conjecture on the convergence of the
system, we can similarly show that the prices tend to
approximate equilibria for a new notion of market equi-
librium, called the self-competition-free or supply-aware
rium for a market with additive utilities is a regular
market equilibrium for a modified setting in which the
utility of each buyer for each item is capped to the util-
ity they derive by buying the entire supply of the item.
The simulations in Section 5 support our intuitions.
 1 7200

0.9
7000

0.8
6800

0.7

6600
0.6

6400
0.5

6200
0.4 1st price
2nd price
0.3 6000 1st Perturbed
1st/2nd price without perturbation
1st/2nd price with perturbation 2nd Perturbed

0.2
0 50 100 150 200 250 300 350 400 450 500
5800
0 100 200 300 400 500

Figure 2: Change in bids, Examples 1 and 2 Figure 4: Change in efficiency, random instance

1000 5400

900 5200

800 5000

700 4800

600 4600

500 4400

400 4200
1st price 1st price
300 2nd price 4000 2nd price
1st Perturbed 1st Perturbed
200 2nd Perturbed 3800 2nd Perturbed

100 3600
0 100 200 300 400 500 0 100 200 300 400 500

Figure 3: Change in revenue, Examples 1 and 2 Figure 5: Change in revenue, random instance

5. SIMULATIONS keyword. The graph of the bid of the first advertiser

In this section, we present the results of simulating as a function of time for each of the four mechanisms
the bid optimization algorithm of Section 3 for vari- is shown in Figure 2 (the second advertiser has similar
ous auction mechanisms. In particular, we compare the bids). As we see in this figure, in unperturbed mech-
behavior of the bid optimization algorithm in the equi- anisms, the bids of the advertisers grow only to $0.50,
librium for the first and second-price auctions with and and after that remain constant, whereas in perturbed
without perturbation. mechanisms, the bids grow to $1. The revenue of the
Parameters of the simulation: We have imple- mechanisms are compared in Figure 3.11 Since the utili-
mented the simulation program in Matlab. In all our ties in this example are equal, the efficiency of all mech-
simulations, we assume that αk = 1/k (i.e., click-through anisms are constant over time.
rates of different slots follow a power law with expo- A larger example: We have simulated the bid opti-
nent −1). We assume that throughout the day, each mization algorithm with different mechanisms on larger
keyword is searched for 1000 times, and these searches instances generated at random. Figures 6, 4 and 5 show
occur in a random order. At the end of each day, the the changes in the bids on two keywords, and the effi-
bid optimization algorithm is run to update the bids of ciency and the revenue of the auctions (per day) as a
each advertiser. For most simulations, the parameters function of the day for an instance with n = 20 bid-
² (determining the aggressiveness of the bid optimiza- ders, m = 10 keywords, and one slot per keyword. In
tion algorithm in changing bids) and δ (determining the this instance, each advertiser bids on each keyword with
extent of the perturbations for perturbed mechanisms)
11
are set to 0.01 and 0.1, respectively. The decrease in the revenue of the perturbed second-
A small example: We start by showing the out- price auction (compared to the first-price) is due to the
come of the simulation for the instance explained in fact that after a short while, the randomness in the sys-
tem could cause the bid of one of the advertisers to be
Examples 1 and 2 for 500 days. In this instance, there
slightly more than the other, resulting in the advertiser
are two advertisers and one keyword with one ad slot. running out of budget earlier than the other advertiser,
Each advertiser has a utility of $1 and a daily budget of and the other advertiser getting the remaining ad spaces
$500. Both advertisers start by bidding $0.20 on each in that day for free.
probability 1/3, and the value of the bids are drawn
uniformly at random from [0, 1]. The daily budgets of
the advertisers are 3000, 3000/2, 3000/3, . . . , 3000/20.12
As Figure 6 shows, the mechanisms with perturbation
avoid having bids that are almost equal and frequently
change order, whereas in mechanisms without perturba-
tion, such situations are common. This can be observed
from the diagram of efficiency in Figure 4, where it can
be observed that the efficiency of the allocation on odd-
numbered days are significantly lower than the efficiency
of the mechanism on even-numbered days.
Random instances: We have simulated the bid op-
timization algorithm with each of the four auction mech-
anisms on a set of 150 randomly generated instances to
measure the average behavior of the algorithm in dif-
ferent auctions. The instances are generated similar to
the way described in the previous example, with 10 bid-
ders, 5 keywords, and 3 slots per keyword. We have
simulated the auctions for 300 days, and measured the
following parameters: the convergence of system, and
the efficiency and the revenue of the auction.
Convergence. To measure the convergence, we check
the properties required in the statement of Theorem 1,
and compute the fraction of bidders for whom both of
these properties are satisfied at the end of the simula-
tion. We say we have perfect convergence if these con-
ditions (for γ = 0.1) are satisfied for all bidders and
good convergence if they are satisfied for 90% (i.e., all
but at most one) of the bidders after 1000 steps. Fig-
ure 7 shows the distribution of the number of converged
bidders, and Figure 8 compares the percentage of the
times perfect or good convergence is achieved on the
four mechanisms. In this figure, mechanisms 1, 2, 3,
and 4 represent the first price, the second price, the
perturbed first price, and the perturbed second price
mechanisms, respectively. These figures confirm our re-
sult that perturbed mechanisms are significantly more
likely to converge to an equilibrium.
Revenue and Efficiency. The comparison of the
revenue and the efficiency of the mechanisms reveals
that in this set of instances, the revenue and the ef-
ficiency of the perturbed mechanisms are consistently
(between 79% and 97% of the times) more than the un-
perturbed mechanisms. However, the difference is small
(between 1.5% and 5% on average).
Acknowledgments: We would like to thank Max Chick-
ering, Uri Feige, and Chris Meek for many fruitful dis-
cussions regarding the systems proposed in this paper.
6. REFERENCES
[1] G. Aggarwal, A. Goel, and R. Motwani. Truthful
auctions for pricing search keywords. In Proceedings of
the 7th ACM Conference on Electronic Commerce,
2006.
[2] K. Arrow and G. Debreu. Existence of an equilibrium
for a competitive economy. Econometrica, 22:265–290,
1954.
[3] M. Dellnitz, M. Field, M. Golubitsky, A. Hohmann,
and J. Ma. Cycling chaos. Intern. J. Bifur. & Chaos,
12
The choice of budgets as a power law distribution with
exponent −1 is motivated by the classical observation
that income distribution often follows such a power law.
5:1243–1247, 1995.
[4] N.R. Devanur, C.H. Papadimitriou, A. Saberi, and
V.V. Vazirani. Market equilibrium via a
primal-dual-type algorithm. In Proceedings of the 43rd
Symposium on Foundations of Computer Science,
2002.
[5] J. Feng, H.K. Bhargava, and D. Pennock. Comparison
of allocation rules for paid placement advertising in
search engines. In Proceedings of the 5th International
Conference on Electronic Commerce, 2003.
[6] M.J. Field. Equivariant dynamical systems. Trans.
Amer. Math. Soc., 259:185–205, 1980.
[7] T. Ibaraki and N. Katoh. Resource Allocation
Problems: Algorithmic Approaches. MIT Press,
Cambridge, MA, 1988.
[8] O.H. Ibarra and C.E. Kim. Fast approximation
algorithms for the knapsack and sum of subset
problems. Journal of the ACM, 22(4):463–468, 1975.
[9] K. Jain and K. Talwar. Truth revealing market
equilibria. Manuscript, 2004.
[10] C. Meek, D. Chickering, and D. Wilson. Stochastic and
contingent-payment auctions. In 1st Workshop on
Sponsored Search Auctions, 2005.
[11] M. Ostrovsky, B. Edelman, and M. Schwarz. Internet
advertising and the generalized second price auction:
Selling billions of dollars worth of keywords. In 2nd
Workshop on Sponsored Search Auctions, 2006.
[12] E. Ott, C. Grebogi, and J.A. Yorke. Controlling chaos.
Physics Review Letters, 64, 1990.
[13] A. Palacios. Cycling chaos in one-dimensional couple
interated maps. Intern. J. Bifur. & Chaos,
12m:1859–1868, 2002.
[14] A. Palacios. Heteroclinic cycles in coupled systems of
difference equations. J. Difference Eqs & Appl.,
9:671–686, 2002.
[15] P. Rusmevichientong and D. Williamson. An adaptive
algorithm for selecting profitable keywords for
search-based advertising services. In Proceedings of the
7th ACM Conference on Electronic Commerce, 2006.
[16] H. Varian. Position auctions. Manuscript, 2006.
[17] X.M. Zhang and J. Feng. Price cycles in online
advertising auctions. In Proceedings of the 26th
International Conference on Information Systems
(ICIS), 2005.
APPENDIX
A. PROOF OF THEOREM 2
We start by recalling some standard definitions, as
applied to our setting. Given the prices pj for keywords,
an optimal allocation xij for advertiser i is any solution
to the following linear program:
X
maximize Li + uij xij
j
X
subject to Li + pj xij = Bi
j
∀j : xij ≥ 0
Li ≥ 0.
Here xij is the fractional amount of word j assigned to
the advertiser i, and Li is the amount of money un-
spent by i. A price vector is called a market equilibrium
price vector if there exist allocations xij that satisfy the
following two conditions:
• At the given price vector, xij is an optimal alloca-
 Bids for keyword 6 in 1st price auction Bids for keyword 6 in 2nd price auction
0.45 0.45

0.4 0.4

0.35 0.35

0.3 0.3

0.25 0.25

0.2 0.2

0.15 0.15

0.1 0.1

0.05 0.05

0 0
0 100 200 300 400 500 0 100 200 300 400 500

Bids for keyword 6 in perturbed 1st price auction Bids for keyword 6 in perturbed 2nd price auction
0.45 0.45

0.4 0.4

0.35 0.35

0.3 0.3

0.25 0.25

0.2 0.2

0.15 0.15

0.1 0.1

0.05 0.05

0 0
0 100 200 300 400 500 0 100 200 300 400 500

Figure 6: Change in the bids in a random instance

1st price 2nd price

100 100

80 80

60 60

40 40

20 20

0 0
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

perturbed 1st price perturbed 2nd price

100 100

80 80

60 60

40 40

20 20

0 0
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

Figure 7: Distribution of the number of converged bidders

 Perfect convergence Good convergence
1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0
1 2 3 4 1 2 3 4

Figure 8: Distribution of the number of converged bidders

X X X
tion for each advertiser i. pj + Li ≤ (1 + ρ(δ, γ)) Bi .
P j i i
• For each keyword j, we have i xij = 1 (recall that
the supply of each keyword was assumed to be 1). The prices and allocation of our algorithm must satisfy
The next theorem follows from the classical results in these constraints. Consider the convex region specified
the economic literature (see, for example, Arrow and by these relaxed constraints. As δ and γ go to zero,
Debreu[2]) by considering the market with one com- the constraints approach those of Theorem 3, implying
modity for each keyword and an additional commodity that the price and utility vectors converge to the unique
termed “money”. The proof of this theorem is deferred equilibrium price and utility vectors, respectively. This
to the full version of the paper. completes the proof of Theorem 2.

Theorem 3. There exists an equilibrium price vec-

tor. Moreover, the market equilibrium prices are unique,
and can be characterized as the set specified by the fol-
lowing convex program.
P
Li + j0 uij 0 xij 0 uij
∀i, j : ≥ (4)
Bi pj
P
Li + j 0 uij 0 xij 0
∀i : ≥1 (5)
Bi
X
∀j : xij ≤ 1 (6)
i
X X X
pj + Li ≤ Bi (7)
j i i

∀i, j : xij ≥ 0 (8)

∀j : pj ≥ 0. (9)

Now let us return to the proof of Theorem 2. We show

that as δ and γ approach zero, the constraints in The-
orem 3 becomes satisfied. In fact, constraints (5), (6),
(8), and (9) are always satisfied: constraint (5) is satis-
fied because no advertiser buys any keyword at a higher
price than his utility, and the other three constraints are
satisfied because Algorithm 1 always computes a feasi-
ble allocation and non-negative prices. Therefore, the
only constraints that we need to check are (4) and (7).
But it is easy to use Theorem 1 to show that these con-
straints are satisfied approximately, i.e., there is a value
ρ(δ, γ) that approaches zero as δ and γ approach zero
so that:
P
Li + j0 uij 0 xij 0 uij
∀i, j : ≥ (1 − ρ(δ, γ))
Bi pj