17.

Amortized analysis

Hsu, Lih-Hsing
 Computer Theory Lab.

 The time required to perform a

sequence of data structure operations
in average over all the operations
performed.
 Average performance of each
operation in the worst case.

Chapter 17 P.2
 Computer Theory Lab.

 For all n, a sequence of n operations

takes worst time T(n) in total. The
amortize cost of each operation is
T ( n) .
n

Chapter 17 P.3
 Computer Theory Lab.

Three common techniques

 aggregate analysis
 accounting method
 potential method

Chapter 17 P.4
 Computer Theory Lab.

17.1 The aggregate analysis

 Stack operation
- PUSH(S, x)
- POP(S)
- MULTIPOP(S, k)

MULTIPOP(S, k)
1 while not STACK-EMPTY(S) and k  0
2 do POP(S)
3 kk–1

Chapter 17 P.5
 Computer Theory Lab.

Action of MULTIPOP on a stack S

top  23
17
6
39
10 top  10
47 47
  
initial MULTIPOP(S,4) MULTIPOP(S,7)

Chapter 17 P.6
 Computer Theory Lab.

 Analysis a sequence of n PUSH, POP,

and MULTIPOP operation on an
initially empty stack.
 O(n2)

 O(n) (better bound)

 The amortize cost of an operation is

O( n )
 O(1).
n
Chapter 17 P.7
 Computer Theory Lab.

Incremental of a binary counter

Chapter 17 P.8
 Computer Theory Lab.

INCREMENT
INCREMENT(A)
1 i0
2 while i < length[A] and A[i] = 1
3 do A[i]  0
4 ii+1
5 if i < length[A]
6 then A[i]  1
Chapter 17 P.9
 Computer Theory Lab.

Analysis:
 O(n k) (k is the word length)
 Amortize Analysis:
log n   n   1
  2i   n  2i
i 0 i 0
 2n

O ( n)
 the amortize cost is  O(1)
n
Chapter 17 P.10
 Computer Theory Lab.

17.2 The accounting method

 We assign different charges to different
operations, with some operations charged
more or less than the actually cost. The
amount we charge an operation is called its
amortized cost.

Chapter 17 P.11
 Computer Theory Lab.

 When an operation’s amortized cost exceeds

its actually cost, the difference is assign to
specific object in the data structure as credit.
Credit can be used later on to help pay for
operations whose amortized cost is less than
their actual cost.

Chapter 17 P.12
 Computer Theory Lab.

 If we want analysis with amortized costs to

show that in the worst cast the average cost
per operation is small, the total amortized
cost of a sequence of operations must be an
upper bound on the total actual cost of the
sequence.
 Moreover, as in aggregate analysis, this
relationship must hold for all sequences of
operations.

Chapter 17 P.13
 Computer Theory Lab.

 If we denote the actual cost of the ith

operation by ci and the amortized cost of the
ith operation by ĉi , we require
n n
 cˆi   ci
i 1 n 1
for all sequence of n operations.
 The total credit stored in the data structure is
the difference between the total actual cost,
or n
ˆi  c .
n
c
i 1 i 1 i

Chapter 17 P.14
 Computer Theory Lab.

Stack operation
PUSH 1 PUSH 2

POP 1 POP 0

MULTIPOP min{k,s} MULTIPOP 0

 Amortize cost: O(1)

Chapter 17 P.15
 Computer Theory Lab.

Incrementing a binary counter

01 1 01 2

10 1 10 0

 Each time, there is exactly one 0 that is

changed into 1.
 The number of 1’s in the counter is never
negative!
 Amortized cost is at most 2 = O(1).

Chapter 17 P.16
 Computer Theory Lab.

17.3 The potential method

 initial data structure D0 .
 Di : the data structure of the result after applying the i-th
operation to the data structure Di1 .
 Ci : actual cost of the i-th operation.

Chapter 17 P.17
 Computer Theory Lab.

A potential function  maps each


data structure Di to a real number
( Di ) , which is the potential associated
with data structure Di .

The amortized cost ci of the i-th
operation with respect to potential 
is defined by cˆ  c   ( D )   ( D ) .
i i i i 1

Chapter 17 P.18
 Computer Theory Lab.

n n

 cˆ   (c  ( D )  ( D
i 1
i
i 1
i i i 1 ))
n
  ci   ( Dn )   ( D0 )
i 1
n n
 ci   ci
 If ( Di )  ( D0 ) then i 1 i 1 .
 If ( Di )  ( Di 1 ) then the potential
increases.

Chapter 17 P.19
 Computer Theory Lab.

Stack operations
 ( Di )  the number of objects in the
stack of the ith operation.
 ( D0 )  0
 ( D )  0
i

Chapter 17 P.20
 Computer Theory Lab.

PUSH

( Di )  ( Di 1 )  ( s  1)  s  1

ci  ci  ( Di )  ( Di 1 )  1  1  2

Chapter 17 P.21
 Computer Theory Lab.

MULTIPOP

k'  min{ k , s }
( Di )  ( Di 1 )   k'
ci  ci  ( Di )  ( Di 1 )  k'  k'  0

Chapter 17 P.22
 Computer Theory Lab.

POP

ci  0

 The amortized cost of each of these

operations is O(1).

Chapter 17 P.23
 Computer Theory Lab.

Incrementing a binary counter

 ( Di )  the number of 1’s in the
counter after the ith operations = bi .
 t i = the ith INCREMENT operation
resets t i bits.
 ( D )  b  b  t  1
i i i 1 i

  ( Di )   ( Di 1 )  (bi 1  ti  1)  bi 1  1  ti
 cˆi  ci  ( Di )  ( Di1 )  (ti  1)  (1  ti )  2
 Amortized cost = O(1)
Chapter 17 P.24
 Computer Theory Lab.

Even if the counter does not start at

zero:
n n
 ci  cˆi  ( Dn )  ( D0 )
i 1 i 1
n
  2  bn  b0  2n  bn  b0
i 1

Chapter 17 P.25
17.4 Dynamic tables
 Computer Theory Lab.

17.4.1 Table expansion

 TABLE-INSERT
 TABLE-DELETE (discuss later)
1
 load-factor  ( T ) (  ( T )  )
2
num[ T ]
 ( T )  (load factor)
size[ T ]

Chapter 17 P.27
 Computer Theory Lab.

TABLE_INSERT
TABLE_INSERT(T, x)
1 if size[T] = 0
2 then allocate table[T] with 1 slot
3 size[T]  1
4 if num[T] = size[T]
5 then allocate new-table with 2  size[T] slots
6 insert all items in table[T] in new-table
7 free table[T]
8 table[T]  new-table
9 size[T]  2  size[T]
10 insert x into table[T]
11 num[T]  num[T] + 1
Chapter 17 P.28
 Computer Theory Lab.

Aggregate method:

i if i-1 is an exact power of 2

ci  
1 otherwise
n  lg n 
 ci  n  2  n  2n  3n
j

i 1 j 1

 amortized cost = 3

Chapter 17 P.29
 Computer Theory Lab.

Accounting method:
 each item pays for 3 elementary
insertions;
1. inserting itself in the current table,
2. moving itself when the table is
expanded, and
3. moving another item that has already
been moved once when the table is
expanded.

Chapter 17 P.30
 Computer Theory Lab.

Potential method:
(T )  2  num[T ]  size[T ]
 sizei  sizei 1 (not expansion)
cˆi  ci   i   i 1
 1  (2  numi  sizei )  (2  numi 1  sizei 1 )
 1  (2  numi  sizei )  (2(numi  1)  sizei )
3

Chapter 17 P.31
 Computer Theory Lab.

Example:
sizei  sizei 1  16 , numi  13

cˆi  ci   i   i 1
 1  (2 13  16)  (2 12  16)
 1  (2 13  16)  (2 12  16)
3

Chapter 17 P.32
 Computer Theory Lab.

 sizei / 2  sizei 1  numi  1 (expansion)

cˆi  ci   i   i 1
 numi  (2  numi  sizei )  (2  numi 1  sizei 1 )
 numi  (2  numi  (2  numi  2))
 (2(numi  1)  (numi  1))
 numi  2  (numi  1)
3

Chapter 17 P.33
 Computer Theory Lab.

Example:
sizei / 2  sizei 1  numi  1  16
cˆi  ci   i   i 1
 17  (2 17  32)  (2 16  16)
 17  (2 17  (2 17  2))  (2 16  16)
 17  2  16
3
 Amortized cost = 3
Chapter 17 P.34
 Computer Theory Lab.

17.4.2 Table expansion and

contraction
 To implement a TABLE-DELETE operation,
it is desirable to contract the table when
the load factor of the table becomes too
small, so that the waste space is not
exorbitant.

Chapter 17 P.35
 Computer Theory Lab.

Goal:
 The load factor of the dynamic table is
bounded below by a constant.
 The amortized cost of a table
operation is bounded above by a
constant.
1
 Set load factor 
2

Chapter 17 P.36
 Computer Theory Lab.

n
 The first 2 operations are inserted. The
n
second 2 operations, we perform I, D, D, I,
I, D, D, …
 Total cost of these n operations is ( n 2
).
Hence the amortized cost is ( n ).
1
 Set load factor  4 (as TABLE_DELETE)
(after the contraction, the load factor
become 1 )
2

Chapter 17 P.37
 Computer Theory Lab.

 1
 2 num [T ]  size[T ] if  (T ) 
 (T )   2
size[T ] 1
  num[T ] if  (T ) 
 2 2

Chapter 17 P.38
 Computer Theory Lab.

 Initial

num0  0
size0  0
0  1
0  0

Chapter 17 P.39
 Computer Theory Lab.

TABLE-INSERT
1
 if  i1  , same as before.
2

1
 if  i1 
2
1
if i 
2
ci  ci   i   i 1
sizei sizei 1
1(  numi )  (  numi 1 )
2 2
sizei sizei
1(  numi )  (  ( numi  1 ))
2 2
Chapter 17 0 P.40
 Computer Theory Lab.

Example:

sizei  sizei 1  16 , numi  6

16 16
ci  ci   i   i 1  1  (  6 )  (  5 )
2 2
0

Chapter 17 P.41
 Computer Theory Lab.

1
 If  i 
2
cˆi  ci   i   i 1
sizei 1
 1  (2numi  sizei )  (  numi 1 )
2
sizei 1
 1  (2(numi 1  1)  sizei 1 )  (  (numi 1 ))
2
3
 3numi 1  sizei 1  3
2
3
 3 i 1 sizei 1  sizei 1  3
2
3 3
 sizei 1  sizei 1  3
2 2
Chapter 17 3 P.42
 Computer Theory Lab.

Example:

sizei  sizei 1  16 , numi  8

ci  ci   i   i 1
16
 1  ( 2  8  16 )  (  7 )
2
3
Amortized cost of Table insert is O(1).

Chapter 17 P.43
 Computer Theory Lab.

TABLE-DELETE
1
 If  i1 
2
 i does not cause a contraction (i.e.,
sizei  sizei 1

ci  ci   i   i 1
sizei sizei 1
1(  numi )  (  numi 1 )
2 2
sizei sizei
1(  numi )  (  ( numi  1 ))
2 2
2
Chapter 17 P.44
 Computer Theory Lab.

Example:

sizei  sizei 1  16 , numi  6

16 16
ci  ci   i   i 1  1  (  6 )  (  7 )
2 2
2

Chapter 17 P.45
 Computer Theory Lab.

  i causes a contraction
ci  numi  1 ( actual cost )
sizei sizei 1
  numi  1
2 4

Chapter 17 P.46
 Computer Theory Lab.

cˆi  ci   i   i 1
sizei sizei 1
 ( numi  1)  (  numi )  (  numi 1 )
2 2
 ( numi  1)  (( numi  1)  numi )
 (( 2numi  2)  ( numi  1))
1

Chapter 17 P.47
 Computer Theory Lab.

Example:
sizei sizei 1
  numi  1  4
2 4
ci  ci   i   i 1
8 16
 ( 3  1)  (  3 )  (  4 )
2 2
1

Chapter 17 P.48
 Computer Theory Lab.

1
 if  i1  (Exercise 18.4.3)
2
 Amortized cost O(1).

Chapter 17 P.49