Substitution Method

• Quick Sort Worst case: pivot is smallest/largest element all the time.
T(n) = T(n-1) + cn
T(n-1) = T(n-2) + c(n-1)
T(n-2) = T(n-3) + c(n-2)
…
T(n-k) = T(n-k-1) + c(n-k)

T(n) = T(n-1) + cn ➔ T(n) = T(n-2) + c(n-1) + cn

T(n) = T(n-3) + c(n-2) + c(n-1) + cn ➔ T(n) = T(n-k) + c(n + n-1 + … n-k)

Let us suppose that : n − k = 1  k = n − 1

n 18
T(n) = T(1) + c  i = O(n ) 2

i =1
Substitution Method
Solve the recurrence relation given below.
1 if n = 1

T ( n) =  n
3  T ( 4 ) + n otherwise

Solution:
n
(1) T (n) = 3  T ( ) + n
4
n n n
(2) T ( ) = 3  T ( 2 ) +
4 4 4
n n n
(3) T ( 2 ) = 3  T ( 3 ) + 2 and so on
4 4 4
19
n n n
(4) T ( k −1 ) = 3  T ( k ) + k −1
4 4 4
 Substitution Method
n
Now substitute the value of T ( ) from (2) to (1)
4
 n n
(5) T (n) = 33T ( 2 ) +  + n
 4 4
n 3
= 3 T ( 2 ) + ( )  n + n
2

4 4
n
substitute the value of T ( 2 ) from (3) to (5) equation, we have
4
 n n 3
T (n) = 3 3  T ( 3 ) + 2  + ( )n + n
2

 4 4  4
n 3 k −1 3 k −2 3 3 0
T ( n) = 3  T ( k ) + ( ) n + ( ) +     + ( ) n + ( ) n
k

4 4 4 4 4
Let us suppose that : n = 4 k  log 4 n = k 20
n  3 3 2 3 k −1 
T (n) = 3  T ( ) + n 1 + ( ) + ( ) + .... + ( ) 
k

n  4 4 4 
Substitution Method
n  3 3 2 3 k −1 
T (n) = 3  T ( ) + n 1 + ( ) + ( ) + .... + ( ) 
k

n  4 4 4 
 3 k 
 1 − ( ) 
4 k −1 1− xk
T ( n) = 3  T (1) + n 1  (
k
)  1 + x + x + ... + x
2
= 1 ( )
 3  1− x
1−
 4 
3k
T (n) = 3  1 + 4n(1 − k )
k
 T (1) = 1
4
3 k
3 n −
k
3 k
T (n) = 3 + 4n  (1 − k ) = 3 + 4n  (1 − ) = 3 k + 4n(
k k
)
4 n n
= 3 k + 4( n − 3 k ) = 1  3k + 4 n − 4  3k

= 4 n − 33 k
= 4n − 3  3 log n
4 21

T ( n) = 4n − 3  n Hence T (n)   (n)

log 34
Recursion Tree Method
• Although substitution method can provide a sufficient
proof that a solution to a recurrence is correct, sometimes
difficult to give a good guess.

• Draw a recursion tree

• straight forward way to devise a good guess.

• In recursion tree, nodes represent costs of sub-problems in

the set of recursive function invocations.

• Sum up processing done at each level of the tree

• Then sum all per-level costs to determine the total cost of

all levels of the recursion.
22
• Useful when recurrence describes running time of divide
and conquer algorithms.
Recursion Tree Method
• Using recursion tree to generate a good guess, we can often
tolerate a small amount of sloppiness since we have to
verify it later on.

• If we are careful when drawing out a recursion tree and

summing costs, then we can use a recursion tree as a direct
proof of a solution to any recurrence of any problem.

23
Recursion Tree Method
Example:
Solve the following recurrence using recurrence tree method

(1) if n = 1
T ( n) = 
n
3.T ( ) + (n 2 ) if otherwise
 4
Solution: The above recurrence can be written in the form

1 if n = 1
T ( n) = 
n
 3.T ( ) + cn 2 if otherwise
 4
Assumption: We assume that n is exact power of 4.
24
The recurrence tree is given in the next slide
Recursion Tree Method
c.n 22
c.n
T(n) 22
T(n)==3.T(n/4)+c.n
3.T(n/4)+c.n

T(n/4)
T(n/4) T(n/4)
T(n/4) T(n/4)
T(n/4)

c.n 22
c.n

c.(n/4)2 2
c.(n/4) 2
c.(n/4)2 c.(n/4)2 c.(n/4)
c.(n/4)2

25
T(n/16)
T(n/16)T(n/16)
T(n/16)T(n/16)
T(n/16) T(n/16)
T(n/16)T(n/16)
T(n/16)T(n/16)
T(n/16) T(n/16)
T(n/16)T(n/16)
T(n/16)T(n/16)
T(n/16)
 Recursion Tree Method
c.n2 c.n2
Suppose n=4k

c.(n/4)2 c.(n/4)2 c.(n/4)2 (3/16).c.n2

2
c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)
(3/16)2.c.n2

26
T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k)
 Recursion Tree Method
c.n2 c.n2
Suppose n = 4k

c.(n/4)2 c.(n/4)2 c.(n/4)2 (3/16).c.n2

2 2 2 2 2
c(n/16)2 c(n/16)2 c(n/16) c(n/16)2 c(n/16)2 c(n/16) c(n/16) c(n/16) c(n/16)
(3/16)2.c.n2

(3/42)k-1 cn2
k k k k k k k k 27
T(n/4 ) T(n/4 ) T(n/4 ) T(n/4 ) T(n/4 ) T(n/4 ) T(n/4 ) T(n/4 )
(nlog43)
T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)
Recursion Tree Method
Total computation cost = Cost of Children
+ Cost of tree excluding children
= Cost of Child x total number of Children
+ Cost of all levels excluding children level
= total number of Children + sum of costs at each
level excluding children level.
4 k
= n  k = log 4 n
T (n) = 3 + cost at Levels above child level
log n4

log 4 3  3 0 3 1 3 k −1  2
T (n) = (n ) + ( 2 ) + ( 2 ) +     +( 2 )  cn
 4 4 4 
1 16 2
T ( n )  ( n log 4 3
)+( )cn = (n
2 log 4 3
) + cn 28
3 13
1− ( )
16
Hence : T (n) = (n 2 )
Recursion Tree Method
T(n) = 2T(n/2) + n2.

29
Recursion Tree Method

30
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

31
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

T(n)

32
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

n2

T(n/4) T(n/2)

33
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

n2

(n/4)2 (n/2)2

T(n/16) T(n/8) T(n/8) T(n/4)

34
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

n2

(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(1)
35
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

n2 n2
(n/4)2 (n/2)2

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(1)
36
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

n2 n2
(n/4)2 (n/2)2 5 n2
16
(n/16)2 (n/8)2 (n/8)2 (n/4)2

(1)
37
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

n2 n2
(n/4)2 (n/2)2 5 n2
16
(n/16)2 (n/8)2 (n/8)2 (n/4)2 25 n 2
256

…
(1)
38
Recursion Tree Method

Solve T(n) = T(n/4) + T(n/2) + n2:

n2 n2
(n/4)2 (n/2)2 5 n2
16
(n/16)2 (n/8)2 (n/8)2 (n/4)2 25 n 2
256

…
(1)
Total = 2 5( 5 2
n 1 + 16 + 16 + 16 ( ) ( )
5 3
+ ) 39
= (n2) geometric series
Master Method
• The master method applies to recurrences of the
form
𝒏
𝑻 𝒏 = 𝒂𝑻 +𝒇 𝒏
𝒃

where 𝒂 ≥ 𝟏, 𝒃 > 𝟏 and 𝒇(𝒏) is asymptotically

positive.
Master Method
Master Method (alternative)
𝒏 where 𝒂 ≥ 𝟏,
𝑻 𝒏 = 𝒂𝑻 + Θ(𝒏𝒌 𝒍𝒐𝒈𝒑 𝒏) , 𝒃 > 𝟏,
𝒃
𝐤 ≥ 𝟎 and
𝑝∈𝑅
• Case 1: 𝒂 > 𝒃𝒌
𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 )
• Case 2: 𝒂 = 𝒃𝒌
• 𝑝 < −1 𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 )
• 𝑝 = −1 𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 log 2 𝑛)
• 𝑝 > −1 𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 log 𝑝+1 𝑛)
• Case 3: 𝒂 < 𝒃𝒌
• 𝑝<0 𝑇 𝑛 = Θ(𝑛𝑘 )
• 𝑝≥0 𝑇 𝑛 = Θ(𝑛𝑘 log 𝑝 𝑛)
 𝑛
Example 1: 𝑇(𝑛) = 2𝑇( ) + 𝑛 log 𝑛
2
𝒏
𝒂 = 𝟐, 𝒃 = 𝟐, 𝒌 = 𝟏, 𝒑 = 𝟏 𝑻 𝒏 = 𝒂𝑻 + Θ(𝒏𝒌 𝒍𝒐𝒈𝒑 𝒏) ,
𝒃
where 𝒂 ≥ 𝟏,
• Case 1: 𝒂 > 𝒃𝒌 𝒃 > 𝟏,
𝒌, 𝟏 𝐤 ≥ 𝟎 and
𝒂? 𝒃 , 𝟐=𝟐 𝑇 𝑛 = Θ(𝑛 log𝑏 𝑎
) 𝑝∈𝑅
• Case 2: 𝒂 = 𝒃𝒌
𝐶𝑎𝑠𝑒 2 with p>-1 so, • 𝑝 < −1, 𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 )
𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 log 𝑝+1 𝑛) • 𝑝 = −1, 𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 log 2 𝑛)
• 𝑝 > −1, 𝑇 𝑛 = Θ(𝑛log𝑏 𝑎 log 𝑝+1 𝑛)
𝑇 𝑛 = Θ(𝑛log2 2 log1+1 𝑛) • Case 3: 𝒂 < 𝒃𝒌
• 𝑝 < 0, 𝑇 𝑛 = Θ(𝑛𝑘 )
• 𝑝 ≥ 0, 𝑇 𝑛 = Θ(𝑛𝑘 log 𝑝 𝑛)
𝑇 𝑛 = Θ(𝑛 log 2 𝑛)