This action might not be possible to undo. Are you sure you want to continue?

# Chapter 3

**Investing in the future: amortized
**

analysis of operations (on data

stru
tures)

3.1 What is amortized analysis?

In previous
ourses you have probably had o
asion to analyze the running time of a single

exe
ution of an algorithm. Typi
ally the analysis attempts to answer one, or both, of the

following questions:

**what performan
e is guaranteed - how long will the algorithm run on the most diÆ
ult
**

input of size n. This kind of analysis is usually
alled worst

ase

analysis.

**what performan
e is expe
ted - how long will the algorithm run on the average if all
**

possible inputs of size n are equally likely to o
ur. This kind of analysis is usually

alled average
ase analysis.

**For example, a straightforward implementation of Qui
ksort has a worst
ase running time
**

of O(n2 ) and an average-
ase running time of O(n log n).

Impli
it in these analyses is the assumption that when a person uses a program to solve a

parti
ular problem it is often a one-time aair. And even if she runs the program repeatedly

**the data used by the algorithm in two dierent exe
utions bear no relation to ea
h
**

other, and

**the algorithm retains no memory of any previous run.
**

In
ontrast, when an algorithm makes use of an Data Type it is rather to be expe
ted

that

**the algorithm will make a sequen
e of (dierent) requests on the Data Type;
**

15

16

CHAPTER 3.

AMORTIZED ANALYSIS

the data
ontained in the Data Type will stay more or less the same between operations,

and

**previous operations in
uen
e the
urrent state of the data stru
ture used by the Data
**

Type.

**After all, it is the purpose of some operations to
hange the
ontent, and sometimes even
**

the internal organization, of the data stru
ture. In this
ontext, then, it is appropriate to

worry not so mu
h about the running time of a single operation as about the running time

of a sequen
e of operations, in whi
h an o
asional lengthy operation may be overshadowed

by long sequen
es of short operations. Let's
all the running time of an operation its a
tual

ost. In a worst
ase analysis of an algorithm the a
tual
ost of an operation is repla
ed by

its worst
ase
ost. In
ontrast

amortized analysis aims at providing an amortized
ost per operation in su
h a way that at

any time the total amortized
ost of the sequen
e of operations performed so far

is not less than the a
tual
ost in
urred.

In the following se
tions we will explore several methods for
omputing upper bounds

on the amortized
ost of an operation. I emphasized the words \upper bound" be
ause I

want you to be aware that that's all we're
omputing. So do not be surprised, or worried,

if dierent methods give you dierent bounds. That is the advantage of having dierent

methods at your disposal: you pi
k the one that gives you the best result - in this
ase the

lowest upper bound. I will illustrate the methods on an example, the pw-sta
k, whi
h may

seem to be arti
ial at rst glan
e, but in a
tuality appears, in various guises, in a number

of algorithms. Se
tion 3.5 presents one su
h guise.

**3.2 A basi
onstru
t: the pop-while sta
k
**

The pop-while sta
k, for short pw-sta
k (read pee-wee-sta
k), augments the usual data stru
ture \sta
k" with the operation pop while. The interfa
e is therefore:

f

interfa
e PW Sta
k

publi
void Push(Obje
t x)

publi
Obje
t Top()

publi
Obje
t Pop()

publi
boolean IsEmpty()

In addition there is the operation

**- pushes obje
t x on top of the sta
k ;
**

- returns the value of the top element of S;

- pops the top element o the sta
k, and returns its value;

- returns true if the sta
k is empty, false otherwise.

**publi
Obje
t Pop-While(b) - pops obje
ts o the sta
k, one by one, as long as
**

boolean variable b remains true.

**Returns the value of the last element popped.
**

Running time proportional to number of elements popped.

Copyright

17

2001 by Avraham. A. Melkman

**Whereas the standard sta
k operations run in O(1) time, the pop-while operation
an
**

on
eivably pop the
ontents of the entire sta
k and has therefore a running time that is

proportional to the size of the sta
k (at that time), in the worst
ase.

Suppose now that the algorithm that employs the pw-sta
k exe
utes a sequen
e of n

operations in all,
onsisting of some mixture of the permissible operations. What then is the

total running time? A
rude analysis goes as follows: if the sequen
e
ontains O(n) pushes

the sta
k may
ontain as many as O(n) obje
ts; a single pop-while
ould therefore take as

long as O(n). If the sequen
e
ontains k pop-whiles it looks like the running time
ould be

as bad as O(kn), whi
h for k = O(n) is O(n2 )! In a
tuality, however, the running time is

mu
h less.

Theorem 3.2.1

Any sequen e of n operations on the pw-sta k runs in time O(n).

**The following two se
tions give ea
h a dierent proof of this theorem. Sin
e the running
**

time is bounded by a linear fun
tion of the number of individual pushes and pops that are

performed, we
an, and will, simplify the analysis by assigning ea
h a
ost of one unit (a

pw-unit) and analyze the running time in terms of these abstra
t
osts.

**3.3 The
harge-now-pay-later method
**

The basis of the
harge method is to re
ord the
ost of ea
h individual operation, the
harge,

and to sum up all re
orded
harges at the end of the sequen
e. There are two ways in whi
h

harges
an be re
orded.

1.

The harge-per-operation method,

**whi
h lists the
harge for ea
h operation separately,
**

mu
h as it is on your
redit
ard statement. Let's see how this works with the pw-sta
k.

Suppose there were k pushes ea
h
osting one pw-unit, and ` pops and pop-whiles of

whi
h the i-th operation popped ni items (ni = 1 for a straight

pop), and therefore

P

P

osted ni pw-units. Then the total of the re
orded
harges is k + `i=1 ni . But `i=1 ni

k be
ause no more items
an be popped o the sta
k then were pushed on. Thus the

total number of
harges does not ex
eed 2k. This proves Theorem 3.2.1 sin
e k n.

2.

The distributed- harge method,

**in whi
h the
harge for ea
h operation is distributed
**

among all, or some, of the elements that were ae
ted by the operation. At the end of

the sequen
e all
harges atta
hed to individual elements are
olle
ted. In the pw-sta
k

example,

suppose p items are in use, and that item number i is pushed mi times, with

Pp

i=1 mi = k . Then the
harges
olle
ted by that item do not ex
eed 2mi , be
ause the

item
annot P

be popped more often than it is pushed. Hen
e the a
umulated
harges

are at most pi=1 2mi = 2k, the same bound as obtained by method 1.

18

CHAPTER 3.

AMORTIZED ANALYSIS

**3.4 The vou
her method
**

In
ontrast to the previous method, the vou
her method attempts to derive an upper bound

on the
ost of a sequen
e by prepaying the
osts of the operations. The idea is to allot ea
h

type of operation a (possibly dierent) standard number of vou
hers, its amortized
ost.

For some operations this standard number of vou
hers is more than what is needed to pay

for their
ost; the remainder of the vou
hers is then stored in appropriate pla
es in the data

stru
ture. For other operations the standard number of vou
hers is not suÆ
ient to pay for

their
ost, and they are supposed to pi
k up additional vou
hers from data stru
ture. For

this method to work, therefore, it must be shown that there are always a suÆ
ient number

of vou
hers to pay for ea
h operation, no matter how
ostly.

Let me illustrate, again, with the pw-sta
k. Suppose ea
h push is outtted with two

vou
hers; one vou
her is used to pay for the
ost of the operation itself, the other
an be

thought of as a vou
her left with the obje
t just stored (on top of the plate). The pop and

pop-while operations, on the other hand, are sent on their way with no vou
hers at all!

The
ost of these operations is payed for with vou
hers that were left in the sta
k. Namely,

ea
h obje
t to be popped
arries with it the vou
her left there by the push operation that

pla
ed it on the sta
k. This shows that, no matter how many obje
ts are popped o by a

pop(-while) operation, there are suÆ
ient vou
hers to pay for its
ost. We have shown,

therefore, that the a
tual
ost of ea
h operation is
ompletely
overed by a
ombination of

the vou
hers given to the operation and the ones "stored" in the sta
k. Consequently the

sum of the
osts of the operations does not ex
eed the sum of the vou
hers provided to the

sequen
e. The latter is simply twi
e the number of pushes, 2k, the same bound as derived

in the previous se
tion.

**3.5 Interlude: an appli
ation to nding the
onvex hull
**

Given a set of points P in the plane, imagine that a nail is driven (perpendi
ularly) into

ea
h point. Now take a rubber band, stret
h it so that it surrounds all nails, and let go.

The form that the rubber band takes on is a
onvex polygon, C (P ), en
losing a shape in the

plane that is
alled the
onvex hull of P . (Sin
e the
onvex hull is
ompletely determined

by C (P ) the latter itself is often referred to as the
onvex hull). Figure 3.5.1 is a bird's eye

view of su
h a set of nails and the taut rubber band.

In this se
tion we will look at an algorithm for nding C (P ), given P . Spe
i
ally, we

will nd the sequen
e of its verti
es as it is traversed in, say,
ounter
lo
kwise order. In

gure 3.5.1 that would be, for example p2 ; p5 ; p1 ; p6 ; p9 . Note that we
an start this sequen
e

at any of the verti
es of the
onvex hull. To summarize, we want to solve the following

problem

Instan
e: A set = f

Find: The verti
es of

P

p1 ; : : : pn

g of points in the plane.

( ) in ounter lo kwise sequen e.

C P

Copyright

19

2001 by Avraham. A. Melkman

p1

p3

p5

p7

p6

p4

p8

p2

p9

**Figure 3.5.1: A set of points, and its
onvex hull
**

The algorithm proposed by R. Graham for solving this problem
omprises two stages:

I. Renumber the points so that

1.

**p1 is the point with least x-
oordinate (and in
ase of ties, with least y -
oordinate
**

among them); this point is guaranteed to be on the hull.

2. p2 ; : : : ; pn are sorted by polar angle with respe
t to p1 in
ounter
lo
kwise fashion.

**In gure 3.5.2 you
an see this renumbering for the points in gure 3.5.1. This stage
**

is not
heap, sin
e it involves a sorting of n obje
ts, whi
h takes O(n log n)
omparisons. Moreover, the angle
omparison of two points needed in 2 appears to be rather

expensive, sin
e (naively) it seems that one has to use the inverse tangent to
ompute

these angles.

**Let's take a moment to address this last issue, an important one for planar
omputational geometry.
**

Noti
e that the a
tual values of the angles are irrelevant- we only want to label them a
ording to

in
reasing angle. Now, the angle that 3 makes with 1 is greater than the angle that 2 makes with

1 if and only if 3 is to the left of 1 2 , the dire
ted line from 1 to 2 . This
an be done as follows:

ompute

1

1

1

= 1 2 2

1

3

3

Then = the area of the triangle 1 2 3, whereas () 0 if and only if 3 is to the left of the

dire
ted line 1 2.

Prove that is invariant under translation and rotation. We may therefore assume,

without loss of generality, that 1 = 0 and 2 = ( 2 0); what is the value of () in this
ase?

p

p

p

j

p

p

p p

j

p

x

y

x

y

x

y

p p p

sign

>

p

p

p p

Finger exer ise:

p

p

x ;

sign

**Let's skip now to the se
ond stage, be
ause there appears what I'm really interested
**

in: a use of the pw-sta
k. Observe that if one \
onne
ts the dots" after stage I the

20

CHAPTER 3.

AMORTIZED ANALYSIS

p9

p8

p1

p7

p6

p5

p3

p4

p2

**Figure 3.5.2: Renumbering the points, and
onne
ting the dots
**

sequen
e < p1 ; : : : ; pn ; p1 > des
ribes a simple polygon; this polygon, however, is not

ne
essarily
onvex, see gure 3.5.2 (it is, however, a star-shaped polygon: one in whi
h

one spe
ial point, in this
ase the pivot,
an "see" the whole polygon). Whi
h is why

we need the next stage. Observe also that p1 ; p2 , and pn are guaranteed to be verti
es

of C (P ).

**II. Exe
ute the following "Graham S
an":
**

Build the hull, by mar
hing around the star-shaped polygon, adding edges when we

make a left turn, and ba
k-tra
king when we make a right turn.

push(p1 ); push(p2 ); push(p3 );

i=4;

while

the sta
k
ontains the verti
es of C(

f

order, with pi

1

i n do

pop-while(pi not left of

push(pi );

i++;

endwhile.

Here

pt

f

g

p1 ; : : : ; pi

**on top and p1 on bottom
**

pt

1

g) in ounter lo kwise

1 pt );

represents the vertex at the top of the sta k, and pt

1

the one just below the

Copyright

2001 by Avraham. A. Melkman

21

top.

I will leave it to you to prove the
orre
tness of this s
an, i.e., to prove the invariant

(guidan
e: note that the angle of pi is greater than any that
ame before, so that it ne
essarily

is outside the
onvex hull assembled so far). My point here is that the pop-while
onstru
t

does appear in a
tual algorithms.

**3.6 The potential method
**

3.7 A simple appli
ation: dynami
ve
tors

3.8 Summary

Amortized analysis deals with the running time of sequen
es of operations. When we say

that the amortized
ost (running time) of an operation on an abstra
t data stru
ture is

O (log n) we mean that, even though the
ost of an individual operation may be mu
h larger

than O(log N ), any sequen
e of m operations runs in a total time bounded by m log n. Even

if the sequen
e was
leverly arranged by a endish enemy to be the most diÆ
ult to exe
ute.

It is therefore a kind of worst-
ase analysis - a worst sequen
e s
enario. In
ontrast, when

we say that the worst-
ase
ost of an operation is O(log N ) we mean that the running time

of ea
h and every operation is bounded by a
onstant times log n. Unless the algorithm that

employs these operations is real-time, an amortized
ost of O(log N ) is just as good as a

worst-
ase
ost of O(log N ), be
ause they guarantee the same overall performan
e.

3.9 Key on epts