Amortized Analysis

Algorithm and Complexity

Dr. Ritu Garg

1
 Amortized Analysis

What is Amortized Analysis ?

In amortized analysis, the time required to perform
a sequence of operations is averaged over all the
operations performed.

 No involvement of probability
Average performance on a sequence of operations,
even some operation is expensive.
Guarantee average performance of each operation
among the sequence in worst case.

Amortized analysis is not just an analysis tool, it is also a way of

thinking about designing algorithms.

2
 Amortized Analysis

Methods of Amortized Analysis

 Aggregate Method: we determine an
upper bound T(n) on the total sequence
of n operations. The cost of each will
then be T(n)/n.
 Accounting Method: we overcharge
some operations early and use them
to as prepaid charge later.
 Potential Method: we maintain credit
as potential energy associated with
the structure as a whole.

3
 Amortized Analysis

1. Aggregate Method

Show that for all n, a sequence of n operations take

worst-case time T(n) in total

In the worst case, the average cost, or amortized cost ,

per operation is T(n)/n.

The amortized cost applies to each operation, even

when there are several types of operations in the
sequence.

4
 Amortized Analysis

Aggregate Analysis: Stack Example

3 ops:

Multi-
Push(S,x) Pop(S)
pop(S,k)
Worst-case O(min(|S|,k)
O(1) O(1)
cost: = O(n)

Amortized cost: O(1) per operation

5
 Amortized Analysis

……. Aggregate Analysis: Stack Example

Sequence of n push, pop, Multipop operations
 Worst-case cost of Multipop is O(n)
 Have n operations
 Therefore, worst-case cost of sequence is O(n2)

Observations
 Each object can be popped only once per time that it’s
pushed
 Have <= n pushes => <= n pops, including those in
Multipop
 Therefore total cost = O(n)
 Average over n operations => O(1) per operation on
average

Notice that no probability involved

6
 Amortized Analysis
2. Accounting Method
Charge i th operation a fictitious amortized cost ĉi,
where $1 pays for 1 unit of work (i.e., time).
 Assign different charges (amortized cost ) to different
operations
 Some are charged more than actual cost
 Some are charged less
This fee is consumed to perform the operation.
Any amount not immediately consumed is stored in the
bank for use by subsequent operations.
The bank balance (the credit) must not go negative!
n n
We must ensure that
for all n.
∑ci ≤ ∑
cˆi
i =1 i =1

Thus, the total amortized costs provide an upper

bound on the total true costs.

7
 Amortized Analysis
….. Accounting Method: Stack Example

3 ops:

Push(S,x) Pop(S) Multi-pop(S,k)

•Assigned
2 0 0
cost:
•Actual cost: 1 1 min(|S|,k)
Push(S,x) pays for possible later pop of x.
8
 Amortized Analysis

….. Accounting Method: Stack Example

When pushing an object, pay $2

 $1 pays for the push
 $1 is prepayment for it being popped by either pop
or Multipop
 Since each object has $1, which is credit, the
credit can never go negative
 Therefore, total amortized cost = O(n), is an upper
bound on total actual cost

9
 Amortized Analysis

….. Accounting Method: Binary Counter

Introduction
k-bit Binary Counter: A[0..k−1]

−1 A[i ] ⋅ 2i
x = ∑ik=0

INCREMENT(A)
1. i ← 0
2. while i < length[A] and A[i] = 1
3. do A[i] ← 0 ⊳ reset a bit
4. i←i+1
5. if i < length[A]
6. then A[i] ← 1 ⊳ set a bit

10
 Amortized Analysis

….. Accounting Method: Binary Counter

Consider a sequence of n increments. The
worst-case time to execute one increment is
Θ(k). Therefore, the worst-case time for n
increments is n · Θ(k) = Θ(n⋅ k).

WRONG! In fact, the worst-case cost for n

increments is only Θ(n) ≪ Θ(n⋅ k).

Let’s see why. Note: You’d be correct

if you’d said O(n⋅ k).
But, it’s an overestimate.

11
 Amortized Analysis

….. Accounting Method: Binary Counter

Ctr A[4] A[3] A[2] A[1] A[0] Cost
0 0 0 0 0 0 0
1 0 0 0 0 1 1
2 0 0 0 1 0 3
Total cost of n operations 3 0 0 0 1 1 4

A[0] flipped every op n 4 0 0 1 0 0 7

5 0 0 1 0 1 8
A[1] flipped every 2 ops n/2
6 0 0 1 1 0 10
A[2] flipped every 4 ops n/22 7 0 0 1 1 1 11
A[3] flipped every 8 ops n/23 8 0 1 0 0 0 15
… … … … … 9 0 1 0 0 1 16

A[i] flipped every 2i ops n/2i 10 0 1 0 1 0 18

11 0 1 0 1 1 19
12
 Amortized Analysis

….. Accounting Method: Binary Counter

 lg n 
n
Cost of n increments= ∑  i 
i =1  2 
∞
1
< n ∑ i = 2n
i =1 2

= Θ(n)

Thus, the average cost of each increment operation is

Θ(n)/n = Θ(1).

13
 Amortized Analysis

….. Accounting Method: Binary Counter

Charge an amortized cost of $2 every time a bit is set from 0 to 1
• $1 pays for the actual bit setting.
• $1 is stored for later re-setting (from 1 to 0).
At any point, every 1 bit in the counter has $1 on it… that pays for
resetting it. (reset is “free”)

Example:

0 0 0 1$1 0 1$1 0

0 0 0 1$1 0 1$1 1$1 Cost = $2

0 0 0 1$1 1$1 0 0 Cost = $2

14
 Amortized Analysis

….. Accounting Method: Binary Counter

INCREMENT(A)
1. i ← 0
2. while i < length[A] and A[i] = 1
3. do A[i] ← 0 ⊳ reset a bit
4. i←i+1
5. if i < length[A]
6. then A[i] ← 1 ⊳ set a bit

When Incrementing,
 Amortized cost for line 3 = $0
 Amortized cost for line 6 = $2

Amortized cost for INCREMENT(A) = $2

Amortized cost for n INCREMENT(A) = $2n =O(n)

15
 Amortized Analysis

Accounting Method vs. Aggregate Method

Accounting method:
First assign amortized cost per operation
Check that they are valid .
Then compute cost of entire sequence of
operations

Aggregate Method :
First analyze entire sequence
Then calculate amortized cost per operation

16
 Amortized Analysis

3. Potential Method
IDEA: View the bank account as the
potential energy (as in physics) of the
dynamic set.

FRAMEWORK:
Start with an initial data structure D0.
Operation i transforms Di–1 to Di.
The cost of operation i is ci.
Define a potential function Φ : {Di} → R,
such that Φ(D0 ) = 0 and Φ(Di ) ≥ 0 for all i.
The amortized cost ĉi with respect to Φ is
defined to be ĉi = ci + Φ(Di) – Φ(Di–1).

17
 Amortized Analysis

….. Potential Method

Like the accounting method, but think of the

credit as potential stored with the entire data
structure.
 Accounting method stores credit with
specific objects while potential method
stores potential in the data structure as a
whole.
 Can release potential to pay for future
operations
Most flexible of the amortized analysis
methods .

18
 Amortized Analysis

….. Potential Method~

ĉi = ci + Φ(Di) – Φ(Di–1)

potential difference ∆Φi

 If ∆Φi > 0, then ĉi > ci. Operation i

stores work in the data structure for later
use.
 If ∆Φi < 0, then ĉi < ci. The data
structure delivers up stored work to help
pay for operation i.

19
 Amortized Analysis
….. Potential Method
The total amortized cost of n operations is
n n

∑cˆ = ∑( c
i =1
i
i =1
i + Φ( Di ) − Φ( Di −1 ) )

Summing both sides telescopically.

n
= ∑ ci + Φ ( Dn ) − Φ ( D0 )
i =1
n
≥ ∑ci since Φ(Dn) ≥ 0 and Φ(D0 ) = 0.~
i =1

20
 Amortized Analysis

….. Potential Method: Stack Example

Define: φ(Di) = #items in stack Thus, φ(D0)=0.

Plug in for operations:

Push: ĉi = ci + φ(Di) - φ(Di-1)
= 1 + j - (j-1)
=2
Pop: ĉi = ci + φ(Di) - φ(Di-1)
= 1 + (j-1) - j
=0
Multi-pop: ĉi = ci + φ(Di) - φ(Di-1)
= k’ + (j-k’) - j k’=min(|S|,k)
=0
21
 Amortized Analysis

….. Potential Method: Binary Counter

Define the potential of the counter after the ith operation
by Φ(Di) = bi, the number of 1’s in the counter after the ith
operation.
Note:
• Φ(D0 ) = 0,
• Φ(Di) ≥ 0 for all i.
Example:

0 0 0 1 0 1 0
(~ 0 0 0 1$1 0 1$1 0 Accounting method)

22
 Amortized Analysis

….. Potential Method

Assume ith INCREMENT resets ti bits (in line 3).

Actual cost ci = (ti + 1)
Number of 1’s after ith operation: bi = bi–1 – ti + 1
The amortized cost of the i th INCREMENT is
ĉi = ci + Φ(Di) – Φ(Di–1)
= (ti + 1) + (1 − ti)
=2
Therefore, n INCREMENTs cost Θ(n) in the worst case.

23
 Amortized Analysis

….. Dynamic Tables

Implementing a table (e.g., hash table) for dynamic

data, want to make it small as possible

Problem: if too many items inserted, table may be too

small
Idea: allocate more memory as needed

24
 Amortized Analysis

….. Dynamic Tables: Why Dynamic Tables?

In some applications:
 We don't know how many objects will be stored in a table.

Similarly, if many objects are deleted from the table:

 It may be worthwhile to reallocate the table with a smaller size .

This problem is called

 Dynamically Expanding and Contracting a table.

25
 Amortized Analysis

….. Dynamic Tables:

Using amortized analysis we will show that :
 The amortized cost of insertion and deletion is O(1).
 Even though the actual cost of an operation is large when it triggers an
expansion or a contraction.

We will also show how to guarantee that:

 The unused space in a dynamic table never exceeds a constant fraction of
the total space.

Operations on Dynamic Table

 TABLE-INSERT: Inserts into the table an item that occupies a single slot.
 TABLE-DELETE: Removes an item from the table & frees its slot.
Load Factor: For empty its 1 and for non empty its given below
Number of items stored in the table
α (T ) =
size(number of slots ) of the table
26
 Amortized Analysis

….. Insertion-Only Dynamic Tables:

Assumption:
 Table is allocated as an array of slots.

A table fills up when:

 All slots have been used.
 Equivalently, when its load factor becomes 1.

Table-Expansion occurs when:

 An item is to be inserted into a full table.
A Common Heuristic:
 Allocate a new table that has twice as many slots as the old one.

Hence, insertions are performed if only :

1 / 2 ≤ α(T) ≤ 1

27
 Amortized Analysis

….. Insertion-Only Dynamic Tables:

Table Insert:

TABLE-INSERT (T, x)
1. if size[T] = 0 then
2. allocate table[T] with 1 slot table[T] : pointer to block
3. size[T] ← 1 of table storage
4. if num[T] = size[T] then num[T] : number of
5. allocate new-table with 2.size[T] slots
6. copy all items in table[T] into new-table items in the table
7. free table[T] size[T] : total number of
8. table[T] ← new-table[T] slots in the table
9. size[T] ← 2.size[T]
10. insert x into table[T] Initially, table is empty,
11. num[T] ← num[T] + 1 so num[T] = size[T] = 0
12. end
1 / 2 ≤ α(T) ≤ 1

28
 Amortized Analysis

● Let ci = cost of ith insert

● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1

27
 Amortized Analysis

● Let ci = cost of ith insert

● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2

28
 Amortized Analysis

● Let ci = cost of ith insert

● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3

29
 Amortized Analysis

● Let ci = cost of ith insert

● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4

30
 Amortized Analysis

Let ci = cost of ith insert

ci = i if i-1 is exact power of 2, 1 otherwise
Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5

31
 Amortized Analysis

● Let ci = cost of ith insert

● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6

32
 Amortized Analysis

● Let ci = cost of ith insert

● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Insert(7) 8 1 7

33
 Amortized Analysis

● Let ci = cost of ith insert

● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Insert(7) 8 1 7
Insert(8) 8 1 8
34
 Amortized Analysis

● Let ci = cost of ith insert 1

2
● ci = i if i-1 is exact power of 2, 1 otherwise 3
4
● Example: 5
6
■ Operation Table Size Cost 7
Insert(1) 1 1 1
8
Insert(2) 2 1 + 1 2
9
Insert(3) 4 1 + 2
Insert(4) 4 1
Insert(5) 8 1 + 4
Insert(6) 8 1
Insert(7) 8 1
Insert(8) 8 1
Insert(9) 16 1 + 8
35
 Amortized Analysis

1. Aggregate Method

 Table is initially empty.

 Observe:
o i-th operation causes an expansion only when i-1
is an exact power of 2.

i if i is an exact power of 2
ci = 
1 otherwise

38
 Amortized Analysis

…Aggregate Method

 Therefore the total cost of n TABLE-INSERT

operations
n  lg n  lg n

∑c
i =1
i = n+ ∑2
j =0
j
< n + ∑ 2 = n + 2n = 3n
j =0
j

i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ...
ci 1 2 3 1 5 1 1 1 9 1 1 1 1 1 1 1 17 1 1 1 ...

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ...
Expansion 1 2 4 8 16
cost

 The amortized cost of a single operation is 3n/n=3 = O(1)

39
 Amortized Analysis
… Accounting Method: Dynamic Tables

Assign the following amortized costs

– Table-Expansion : 0
– Insertion of a new item : 3
Insertion of a new item
a) 1 (as an actual cost) for inserting itself into the table
b) 1 (as a credit) for moving itself in the next expansion
c) 1 (as a credit) for moving another item (in the next
expansion) that has already moved in the last expansion

40
 Amortized Analysis

…Accounting Method : Dynamic Tables

Size of the table: M

Immediately after an expansion (just before the insertion)
num[T] = M/2 and size[T] = M where M is a power of 2.
Table contains no credits

41
 Amortized Analysis

…Accounting Method : Dynamic Tables

1st insertion

(a) $1 for
(c)
(b) insertion

2nd insertion

42
 Amortized Analysis

…Accounting Method : Dynamic Tables

M/2th Insertion

Thus, by the time the table contains M items and

is full
– each item in the table has $1 of credit to pay for its
move during the next expansion
43
 Amortized Analysis
……. Potential Method: Dynamic Tables
Define a potential function Φ that is
– 0 immediately after an expansion
– builds to the table size by the time table becomes full
Next expansion can be paid for by the potential.
One possible Φ is:
Φ(T) = 2*num[T] –size[T]
Immediately after an expansion
size[T] = 2*num[T] ⇒ Φ(T) = 0
Immediately before an expansion
size[T] = num[T] ⇒ Φ(T) = num[T]
The initial value for the potential is 0

44
 Amortized Analysis

……. Potential Method: Dynamic Tables

Definition of Φ:
Since the table is at least half full
(i.e. num[T] ≥ size[T] / 2)
Φ(T) is always nonnegative.
Thus, the sum of the amortized cost of n TABLE-INSERT
operations is an upper bound on the sum of the actual
costs.

45
 Amortized Analysis

……. Potential Method: Dynamic Tables

Analysis of i-th Table Insert:

numi : num[T] after the i-th operation

sizei : size[T] after the i-th operation

Φi : Potential after the i-th operation

Initially we have numi = sizei = Φi = 0

Note that, numi = numi-1 +1 always hold.

46
 Amortized Analysis
……. Potential Method: Dynamic Tables
Insertion without Expansion:
sizei = sizei – 1 ; ci = 1

Insertion with Expansion:

sizei = 2.sizei – 1 and sizei-1 =numi – 1 =numi - 1

47
 Amortized Analysis
……. Potential Method: Dynamic Tables
Adding Delete Operation:
TABLE-DELETE: Remove the specified item from the
table. It is often desirable to contract the table.
In table contraction, we would like to preserve two
properties
 Load factor of dynamic table is bounded below by a
constant
 Amortized cost of a table operation is bounded above
by a constant
We assume that the cost can be measured in terms of
elementary insertions and deletions
48
 Amortized Analysis
……. Potential Method: Dynamic Tables
A natural strategy for expansion and contraction
• Double the table size when an item is inserted into a
full table
• Halve the size when a deletion would cause
α (T )< 1 / 2
1
This strategy guarantees ≤ α(T ) ≤1
2

Unfortunately, it can cause the amortized cost of an

operation to be quite large
49
 Amortized Analysis
……. Potential Method: Dynamic Tables
Worst-Case for α(T) ≥ ½
Consider the following worst case scenario
– We perform n operations on an empty table
where n is a power of 2
– First n/2 operations are all insertions , cost a
total of Θ(n)
at the end: we have num[T] = size[T] = n/2
– Second n/2 operations repeat the sequence
IDDI
that is I D D I I D D I I D D I ...

50
 Amortized Analysis
……. Potential Method: Dynamic Tables
Worst-Case for α(T) ≥ ½
Example: n=16

In the second n/2 operations

– The first INSERT cause an expansion
– Two further DELETEs cause contraction
– Two further INSERTs cause expansion ... and so on
Hence there are n/8 expansions and n/8 contractions
The cost of each expansion and contraction is ≈ n/2
51
 Amortized Analysis
……. Potential Method: Dynamic Tables
Worst-Case for α(T) ≥ ½
Thus the total cost of n operations is Θ(n2) since
– First n/2 operations : 3n
– Second n/2 operations : (n/4)*(n/2)=n2/8
The amortized cost of an operation is Θ(n)
The difficulty with this strategy is
– After an expansion, we do not perform enough
deletions to pay for a contraction
– After a contraction, we do not perform enough
insertions to pay for an expansion

Amortized Analysis 52
 Amortized Analysis
……. Potential Method: Dynamic Tables
Improving Expansion – Contraction

We can improve upon this strategy by allowing α(T)

to drop below ½
We continue to double the table size when an item
is inserted into a full table
But, we halve the table size (perform contraction)
when a deletion causes α(T) < ¼ rather than α(T)
<½,
Therefore, ¼ ≤ α(T) ≤ 1

53
 Amortized Analysis
……. Potential Method: Dynamic Tables
Improving Expansion – Contraction
Hence after an expansion, α(T) = ½ , thus at least
half of the items in the table must be deleted
before a contraction can occur.
Similarly, after a contraction α(T) = ½ , thus the
number of items in the table must be doubled by
insertions before an expansion can occur.

54
 Amortized Analysis
……. Potential Method: Dynamic Tables
Define the potential function as follows
• Φ(T) = 0 immediately after an expansion or
contraction
• Recall that, α(T) = ½ immediately after and
expansion or contraction,
therefore the potential should build up to num[T]
as α(T) increases to 1 or decreases to ¼
• So that the next expansion or contraction can be
paid by the potential.

55
 Amortized Analysis
……. Potential Method: Dynamic Tables
Description of New Φ: One such Φ is
 1
2num[T ] − size[T ] if α (T) ≥
Φ(T ) =  2
size[T] 1
 − num[T ] if α (T) <
 2 2

• Φ(T) = 0 when α = ½
• Φ(T) = num[T] when α = ¼

• Φ (T)= 0 for an empty table (num[T] = size[T]=0)

• Φ is always nonnegative
56
 Amortized Analysis
……. Potential Method: Dynamic Tables

Operations are:
– TABLE-INSERT
– TABLE-DELETE

57
 Amortized Analysis
……. Potential Method: Dynamic Tables
Table Insert:
 αi-1 ≥ ½
 Analysis is identical to that for table expansion so ĉi = 3.
 αi-1 < ½ and αi < ½
Expansion may not occur (ĉi = 1 , sizei = sizei-1 )

58
 Amortized Analysis
……. Potential Method: Dynamic Tables
Table Insert:
 αi-1 < ½ but αi ≥ ½

Thus, amortized cost of a TABLE-INSERT operation is at most 3.

59
 Amortized Analysis
……. Potential Method: Dynamic Tables
Table Delete:
 αi-1 < ½ (No contraction) :sizei = sizei-1

 αi-1 < ½ (Contraction) ĉi =numi + 1 , sizei /2= sizei-1 /4 = numi-

1= numi + 1

60
 Amortized Analysis
….. Dynamic Tables

 Thus, the amortized cost of a TABLE-DELETE operation is at

most 2
 Since the amortized cost of each operation is bounded above by
a constant
 The actual time for any sequence of n operations on a Dynamic
Table is O(n)

25
 Amortized Analysis

62
 Amortized Analysis

2017/15/05 Amortized Analysis 63