5.

Solving recurrences
 Time Complexity Analysis of Merge Sort

 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Q. How to prove
 that the run-time of merge sort is O(n log n)?
A.

2
 Time Complexity Analysis of Merge Sort

 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Q. How to prove
 that the run-time of merge sort is O(n log n)?
A. We have seen several methods:
 Recursion tree
 Substitution (by induction)

3
 Proof by Recursion Tree

 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging
#nodes(merge time):

T(n) n


T(n/2) T(n/2) 2(n/2)

T(n/4) T(n/4) T(n/4) T(n/4) 4(n/4)

log2n
...
T(n / 2k) 2k (n / 2k)

...
T(2 T(2 T(2 T(2) T(2 T(2 T(2 T(2 n/2 (2)
) ) ) ) ) ) )
T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)

merge time: n log2n

 Proof by Induction/Substitution (when n is power of 2)

Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n.

assume n is a power of 2
 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Pf. (by induction

 on n)

5
 Proof by Induction/Substitution (when n is power of 2)

Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n.

assume n is a power of 2
 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Pf. (by induction

 on n)
 Base case: n = 1.
 Induction hypothesis: T(n) = n log2 n.
 Step: show that T(2n) = 2n log2 (2n). Now for 2n (not n+1 as we are used to) !

Q. How do we proof this step?

6
 Proof by Induction/Substitution (when n is power of 2)

Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n.

assume n is a power of 2
 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Pf. (by induction

 on n)
 Base case: n = 1.
 Induction hypothesis: T(n) = n log2 n.
 Step: show that T(2n) = 2n log2 (2n). Now for 2n (not n+1 as we are used to) !

T (2 n )  2T (n )  2 n
 2 n log 2 n  2 n Induction hypothesis


 2 n log 2 (2 n )

7
 Proof by Induction/Substitution (when n is power of 2)

Claim. If T(n) satisfies this recurrence, then T(n) = n log2 n.

assume n is a power of 2
 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Pf. (by induction

 on n)
 Base case: n = 1.
 Induction hypothesis: T(n) = n log2 n.
 Step: show that T(2n) = 2n log2 (2n). Now for 2n (not n+1 as we are used to) !

T (2 n )  2 T (n )  2 n
log 2 ( 2 n )  1  log n
 2 n log 2 n  2n
 2 n log 2 (2 n )  1   2 n
 2 n log 2 (2 n )

8
 Proof by Induction/Substitution (with rounding)

Claim. If T(n) satisfies the following recurrence, then T(n)  n log n.

log2n
 0 if n  1

T( n )   T  n / 2    T  n / 2    n otherwise
 merging
 solve left half solve right half

Pf. (by induction on n)


 Base case: n = 1.
 Define n1 = n / 2, n2 = n / 2.
 Hypothesis: assume true for 1, 2, ... , n–1.
 Step:

9
 Proof by Induction/Substitution (with rounding)

Claim. If T(n) satisfies the following recurrence, then T(n)  n log n.

log2n
 0 if n  1

T( n )   T  n / 2    T  n / 2    n otherwise
 merging
 solve left half solve right half

Pf. (continued)

 Step:

T (n)  T ( n 1 )  T (n 2 )  n
 n 1 log n 1   n 2  log n 2   n Induction hypothesis

 n  log n 

10
 Proof by Induction/Substitution (with rounding)

Claim. If T(n) satisfies the following recurrence, then T(n)  n log n.

log2n
 0 if n  1

T( n )   T  n / 2    T  n / 2    n otherwise
 merging
 solve left half solve right half

Pf. (continued)

 Step:

T (n)  T ( n 1 )  T (n 2 )  n
 n 1 log n 1   n 2  log n 2   n
 n 1  log n 1   n 2  log n 1   n
 n  log n 1   n

 n  log n 

11
 Proof by Induction/Substitution (with rounding)

Claim. If T(n) satisfies the following recurrence, then T(n)  n log n.

log2n
 0 if n  1

T( n )   T  n / 2    T  n / 2    n otherwise
 merging
 solve left half solve right half

Pf. (continued)

 Step:
n1  n / 2 
T (n)  T ( n 1 )  T (n 2 )  n  2  log n 
/2 
n 1 log n 1   n 2  log n 2   n
 log n 
  2 /2
 log n 1   log n   1
 n 1  log n 1   n 2  log n 1   n
 log n 1    log n   1
 n  log n 1   n
 n (  log n   1 )  n Because right side is an integer,

 n  log n  rounding to nearest integer is OK.

12
 Time Complexity Analysis of Merge Sort

 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Q. How to prove
 that the run-time of merge sort is O(n log n)?
A. We have seen several methods:
 Recursion tree
 Substitution (by induction)

13
 Time Complexity Analysis of Merge Sort

 0 if n  1

T( n )   2 T (n / 2 )  n otherwise

 sorting both halves merging

Q. How to prove
 that the run-time of merge sort is O(n log n)?
A. We have seen several methods:
 Recursion tree
 Substitution (by induction)

We don’t like proofs. Can’t you give

us a general rule for the complexity
of recursive functions?

14
 General Recursion Tree

  (1) if n  1

for a  1, b  1, T( n )   aT (n / b)  f ( n ) otherwise
 1 4 2 43 123
 recursive calls combining



15
 General Recursion Tree

  (1) if n  1

for a  1, b  1, T( n )   aT (n / b)  f ( n ) otherwise
 1 4 2 43 123
 recursive calls combining
#nodes(time):

T(n) f(n)
a


T(n/b) T(n/b) T(n/b) a f(n/b)

a a a

16
 General Recursion Tree

  (1) if n  1

for a  1, b  1, T( n )   aT (n / b)  f ( n ) otherwise
 1 4 2 43 123
 recursive calls combining
#nodescombining:

T(n) f(n)
a


T(n/b) T(n/b) T(n/b) a f(n/b)

a a a

-1+logbn
T(n/b2 T(n/b2 T(n/b2 T(n/b2 a2
) ) ) ) f(n/b2)
...
T(n / bk) ak f(n /
bk )
...
T(1) T(1) T(1) T(1) T(1) T(1 T(1) T(1) a
log b n
f(1)
)
 1  log b n
 n 
n operations in the leaves   
log b n log b a
a  n
log b a k
a f k 
k 0 b 
 Master Method

  (1) if n  1

for a  1, b  1, T( n )   aT (n / b)  f ( n ) otherwise
 1 4 2 43 123
 recursive calls combining

 1  log b n
 n 
So, T ( n )   n log b a
  
k
a f k  where
 k 0 b 

 first term is cost of all n log a subproblems of size 1, and

b

 second term cost for combining in each level.

Three common cases:

 Running time dominated by cost at leaves
 Running time evenly distributed throughout the tree
 Running time dominated by cost at root

Consequently, to solve the recurrence, we need only to characterize the

dominant term, n log a or f (n )
b

18
 Master Method

Given a recurrence of the form T( n )  aT (n / b)  f ( n )

We can distinguish three common cases:

1. Running time dominated  by cost at leaves: for an ε > 0
if f ( n )  O n log a   
b
then T ( n )   n
log b a

2. Running time evenly distributed throughout the tree:
if f ( n )   n  then T ( n )   n log n 
log a
b log a b

3. Running time dominated by cost at root: for an ε > 0

if f ( n )   n log a   
b
then T ( n )    f ( n ) 
If f(n) satisfies regularity
condition: a f(n/b) ≤ c f(n)
for some c < 1
(polynomials always do)

The master method cannot solve every recurrence of this form.

19
 Summary Master Method

 Extract a, b, and f(n) from a given recurrence

 Determine n log a
b

 Compare f(n) and n log a asymptotically

b

 Determine appropriate Master Method case and apply:

1. Running time dominated by cost at leaves:

if f ( n )  O n log a   
b
then T ( n )   n log b a
 for an ε > 0

2. Running time evenly distributed throughout the tree:

if f ( n )   n  then T ( n )   n log a log n 
log a
b b

3. Running time dominated by cost at root:

if f ( n )   n log a   
b then T ( n )    f ( n )  for an ε > 0
Case 3: Only If f(n) satisfies regularity condition:
a f(n/b) ≤ c f(n) for some c < 1
(polynomials always do)

20
 Examples

 Extract a, b, and f(n) from a given recurrence

 Determine n log a
b

 Compare f(n) and n log a asymptotically

b

 Determine appropriate Master Method case, and apply

Example. Analyze Merge Sort using the Master Method:

T ( n )  2T ( n / 2 )   ( n )
a  2 , b  2; n n
log b a log 2 2
n
Q. What is dominant? (leaves, equal, root node)

21
 Examples

 Extract a, b, and f(n) from a given recurrence

 Determine n log a
b

 Compare f(n) and n log a asymptotically

b

 Determine appropriate Master Method case, and apply

Example. Analyze Merge Sort using the Master Method:

T ( n )  2T ( n / 2 )   ( n )
a  2 , b  2; n n
log b a log 2 2
n
f (n)   (n)
this is case 2, f (n)   n  log b a
, so

T (n)   n  log b a
log n 

22
 Examples

Binary-search(A, p, r, s):
q(p+r)/2
if A[q]=s then return q
else if A[q]>s then
Binary-search(A, p, q-1, s)
else Binary-search(A, q+1, r, s)

Q. Analyze complexity of binary search using the master method (1 min)

A. Analysis:
T (n)  T (n / 2)  1

23
 Examples

Binary-search(A, p, r, s):
q(p+r)/2
if A[q]=s then return q
else if A[q]>s then
Binary-search(A, p, q-1, s)
else Binary-search(A, q+1, r, s)

Q. Analyze complexity of binary search using the master method (1 min)

A. Analysis:
T (n)  T (n / 2)  1
a  1, b  2 ; n  n   (1)
log b a log 2 1 0
n
f ( n )   (1)
This is case 2, f ( n )   n log b a
 , so:
T (n)   n  log b a

log n   log n 

24
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  9T ( n / 3 )  n

A. Analysis:
a  9 , b  3;  ...
log 3 9
n
f (n)   (n)

Q. What is dominant? (leaves, equal, root node)

25
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  9T ( n / 3 )  n

A. Analysis:
a  9 , b  3; n
log 3 9 2
n
f (n)   (n)

This is case 1, f ( n )  O n  , so:

log b a  

T (n)   n  log b a
   n  2



26
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  3T ( n / 4 )  n log n

A. Analysis:
a  3, b  4 ; n
log 4 3 0 . 793
n
f ( n )   ( n log n )

Q. What is dominant? (leaves, equal, root node)

27
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  3T ( n / 4 )  n log n

A. Analysis: WARNING: is not a polynomial

Check regularity condition.
a  3, b  4 ; n
log 4 3 0 . 793
n
f ( n )   ( n log n )

This is case 3, f ( n )   n  , so:

log b a  

T ( n )    f ( n )     n log n 

Check regularity condition:


af ( n / b )  3 ( n / 4 ) log( n / 4 )  ( 3 / 4 ) n log n  cf ( n )

OK, for example for c=3/4 (and this c < 1)

28
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  2T ( n / 2 )  n log n

A. Analysis:
a  2 , b  2; n
log 2 2
n
f ( n )   ( n log n )

Q. What is dominant? (leaves, equal, root node)

29
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  2T ( n / 2 )  n log n
I’ll be
back!
A. Analysis:
a  2 , b  2; n
log 2 2
n
f ( n )   ( n log n )

This is not case 3, f ( n )   n , for c>0, nor one of the others!

log b a  

Back to substitution method and induction proof (try n log2n ).



Because nc for c>0 is (log n),

so n1+c is (n log n)
so n log n is not (n1+c)

30
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  4T ( n / 2 )  n
3

A. Analysis:
a  4 , b  2;  ...
log 2 4
n
f (n)   (n )
3

Q. What is dominant? (leaves, equal, root node)

31
 Examples

Q. Use the master method to solve the following recurrence relation:

T ( n )  4T ( n / 2 )  n
3

A. Analysis:
a  4 , b  2; n
log 2 4 2
n
f (n)   (n )
3

This is case 3, f ( n )   n  , so:

log b a  

T (n)    f (n)   n   3

Check regularity condition:


af ( n / b )  4  n / 2   ( 4 / 8 ) n  cf ( n )
3 3

OK, for example for c=3/4 (and this c < 1)

32