Design and Analysis of

Algorithms

Chp 4
Solving Recurrences
 Recurrent Algorithms
BINARY – SEARCH
For an ordered array A, finds if x is in the array A[lo…hi]

Alg.:BINARY-SEARCH (A, lo, hi, x)

1 2 3 4 5 6 7 8

2 3 5 7 9 10 11 12
if (lo > hi)
return FALSE
mid
mid  (lo+hi)/2 lo hi

if x = A[mid]
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
Recurrences 2
 Analysis of BINARY-SEARCH

Alg.:BINARY-SEARCH (A, lo, hi, x)

if (lo > hi) constant time: c1
returnFALSE constant time: c2
mid  (lo+hi)/2
ifx = A[mid] constant time: c3
return TRUE
if ( x < A[mid] )
same problem of size n/2
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
same problem of size n/2
BINARY-SEARCH (A, mid+1, hi, x)

• T(n) = c + T(n/2)
– T(n) – running time for an array of size n
Recurrences 3
 Recurrences and Running Time
• Recurrences arise when an algorithm contains
recursive calls to itself

• What is the actual running time of the algorithm?

• Need to solve the recurrence
– Find an explicit formula of the expression (the generic
term of the sequence)

Recurrences 4
 Example Recurrences
• T(n) = T(n-1) + n Θ(n2)
– Recursive algorithm that loops through the input to
eliminate one item
• T(n) = T(n/2) + c Θ(lgn)
– Recursive algorithm that halves the input in one step
• T(n) = T(n/2) + n Θ(n)
– Recursive algorithm that halves the input but must
examine every item in the input
• T(n) = 2T(n/2) + 1 Θ(n)
– Recursive algorithm that splits the input into 2 halves
and does a constant amount of other work
Recurrences 5
 Methods for Solving Recurrences

• Iteration method

• Recursion tree method

Recurrences 6
Some Simple Summation Formulas
n
n( n  1)
• Arithmetic series:  k  1  2  ...  n 
k 1 2
n
x n 1  1
• Geometric series:

k 0
x  1  x  x  ...  x 
k 2 n

x 1
 x  1

1
– Special case: x < 1: x
k 0
k

1 x
n
1 1 1
• Harmonic series: 
k 1 k
 1 
2
 ... 
n
 ln n

• Other important formulas:  lg k  n lg n

k 1

n
1
 k p  1p  2 p  ...  n p 
k 1 p 1
n p 1

7
 The Iteration Method
T(n) = c + T(n/2)
T(n) = c + T(n/2) T(n/2) = c + T(n/4)
= c + c + T(n/4) T(n/4) = c + T(n/8)
= c + c + c + T(n/8)
Assume n = 2k  k = lgn
T(n) = c + c + … + c + T(1)  = k . C + T(1)

k times
So
T(n) = clgn + T(1)
= Θ(lgn)
Recurrences 8
 Iteration Method – Example
T(n) = n + 2T(n/2) Assume: n = 2k
T(n) = n + 2T(n/2) T(n/2) = n/2 + 2T(n/4)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
… = in + 2iT(n/2i)
= kn + 2kT(1)
= nlgn + nT(1) = Θ(nlgn)

Recurrences 9
 Iteration Method – Example
T(n) = n + T(n-1)

T(n) = n + T(n-1)
= n + (n-1) + T(n-2)
= n + (n-1) + (n-2) + T(n-3)
… = n + (n-1) + (n-2) + … + 2 + T(1)
= n(n+1)/2 - 1 + T(1)
= n2+ T(1) = Θ(n2)

Recurrences 10
  0 n0
s ( n)  
c  s (n  1) n  0
• s(n) =
c + s(n-1)
c + c + s((n-1)-1)
c + c + s(n-2)
2c + s(n-2)
2c + c + s(n-3)
3c + s(n-3)
…

kc + s(n-k) = ck + s(n-k)

Recurrences 11
  0 n0
s ( n)  
c  s (n  1) n  0
• So far for n >= k we have
– s(n) = ck + s(n-k)
• What if k = n?
– s(n) = cn + s(0) = cn

Recurrences 12
  0 n0
s ( n)  
c  s (n  1) n  0
• So far for n >= k we have
– s(n) = ck + s(n-k)
• What if k = n?
– s(n) = cn + s(0) = cn
• So

• Thus in general
– s(n) = cn
 0 n0
s ( n)  
c  s (n  1) n  0
Recurrences 13
  0 n0
s ( n)  
n  s (n  1) n  0
• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
=…
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

Recurrences 14
  0 n0
s ( n)  
n  s (n  1) n  0
• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
=…
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
n

i
i  n  k 1
 s (n  k )
Recurrences 15
  0 n0
s ( n)  
n  s (n  1) n  0
• So far for n >= k we have

i
i  n  k 1
 s (n  k )

Recurrences 16
  0 n0
s ( n)  
n  s (n  1) n  0
• So far for n >= k we have

i
i  n  k 1
 s(n  k )

• What if k = n?

Recurrences 17
  0 n0
s ( n)  
n  s (n  1) n  0
• So far for n >= k we have

i
i  n  k 1
 s (n  k )

• What if k = n?
n n
n(n  1)
i
i 1
 s (0)  i
i 1
 0 
2

Recurrences 18
  0 n0
s ( n)  
n  s (n  1) n  0
• So far for n >= k we have

i
i  n  k 1
 s(n  k )

• What if k = n?
n n
n(n  1)
i
i 1
 s ( 0)  i
i 1
 0 
2
• Thus in general n(n  1)
s ( n) 
2
Recurrences 19
  c n 1
 n
T (n)  2T
   c n 1
  2 
• T(n) =
2T(n/2) + c
2(2T(n/2/2) + c) + c
22T(n/22) + 2c + c
22(2T(n/22/2) + c) + 3c
23T(n/23) + 4c + 3c
23T(n/23) + 7c
23(2T(n/23/2) + c) + 7c
24T(n/24) + 15c
…
2kT(n/2k) + (2k - 1)c
Recurrences 20
  c n 1
 n
T (n)  2T
   c n 1
  2 
• So far for n > 2k we have
– T(n) = 2kT(n/2k) + (2k - 1)c
• What if k = lg n?
– T(n) = 2lg n T(n/2lg n) + (2lg n - 1)c
= n T(n/n) + (n - 1)c
= n T(1) + (n-1)c
= nc + (n-1)c = (2n - 1)c

Recurrences 21
The recursion-tree method

Convert the recurrence into a tree:

– Each node represents the cost incurred at that level
of recursion
– Sum up the costs of all levels

Used to “guess” a solution for the recurrence

Recurrences 22
W(n) = 2W(n/2) + n2
Example 1

• Subproblem size at level i is: n/2i

• Subproblem size hits 1 when 1 = n/2i  i = lgn
• Cost of the problem at level i = (n/2i)2, No. of nodes at level i = 2i
• Total cost:

lg n 1 lg n 1 i  i
n2 1 1 1
W (n)   i  2 W (1)  n     n  n     O( n) n 2
lg n 2 2
 O ( n)  2n 2
i 0 2 i 0  2  i 0  2  1 1
2
 W(n) = O(n2)

Recurrences 23
W(n) = 2W(n/2) + n2
Example 1
• Subproblem size at level i is: n/2i
• Subproblem size hits 1 when 1 = n/2i  i = lgn
• Cost of the problem at level i = (n/2i)2, No. of nodes at level i = 2i
• Total cost:
lg n 1
n2
W (n)   i  2 lg n W (1)
i 0 2
lg n 1 i  i
1 1
 n2   
i 0  2 
 n  n 2
    O ( n) 
i 0  2 

1
n 2
 O ( n)  2 n 2
1 1
2
 W(n) = O(n2)

Recurrences 24
 Example 2
Solve T(n) = T(n/4) + T(n/2)+ n2

Recurrences 25
 Example 2
Solve T(n) = T(n/4) + T(n/2)+ n2

Recurrences 26
 Example 2
Solve T(n) = T(n/4) + T(n/2)+ n2

Recurrences 27
 Example 2
Solve T(n) = T(n/4) + T(n/2)+ n2

Recurrences 28
 Example 2
Solve T(n) = T(n/4) + T(n/2)+ n2

Recurrences 29
 Example 2
Solve T(n) = T(n/4) + T(n/2)+ n2

Recurrences 30
 Example 3

T(n) = 2T(n/2) + cn

Recurrences 31
Using the recursion tree to visualize a recurrence.

Recurrences 32