Divide-&-Conquer Algorithms

& Recurrence Relations:

Selected Exercises
 Exercise 10

Find f( n ) when n = 2k, where f satisfies the recurrence

relation f( n ) = f( n/2 ) + 1, with f( 1 ) = 1.

Copyright © Peter Cappello 2

 Exercise 10 Solution

We are asked for the value of f.

We are given that

f( n ) = f( n / 2 ) + 1, with f( 1 ) = 1, where n = 2k.

f( 2k ) = f( 2k - 1 ) + 1, with f( 20 ) = 1.

Answer: f( 2k ) = k + 1. (Proof would be inductive.)

Copyright © Peter Cappello 3
 Master Theorem

Let f be an increasing function that such that

f( n ) = a.f( n / b ) + c.nd

whenever n = bk, where a, b, k  Z+ & c > 0 & d ≥ 0:

If ( a < bd ) then f( n ) is O( nd )

If ( a = bd ) then f( n ) is O( ndlogn )

If ( a > bd ) then f( n ) is O( nlogba ).

Copyright © Peter Cappello 4
 Exercise 10 Solution (2)
• If we are interested only in the asymptotic growth rate,
use the Master Theorem.
• For f( n ) = f( n / 2 ) + 1
= 1f( n / 2 ) + 1n0
a=1
b=2
d=0
So, a = bd
So, f(n) is O( ndlog n ) = O( log n ).
Copyright © Peter Cappello 5
 Merge Sort

Procedure mergesort( List a )

{
assert a != null && !a.isEmpty();
int n = a.size();
If ( n == 1 ) return a;
List L1 = new List containing a( 0 ), . . . , a( n/2 );
List L2 = new List containing a( n/2 + 1 ), …, a( n );
return merge( mergesort( L1 ), mergesort( L2 );
}
Copyright © Peter Cappello 6
Give a recurrence relation, f( n ), the time to run
merge sort on a list of size n.

Copyright © Peter Cappello 7

 Solution
A Divide&Conquer Recurrence Relation

f( n ) = 2 f( n / 2 ) + cn1, where c is some constant.

Give a big-O estimate of the time to run merge sort on a

List of size n.

Copyright © Peter Cappello 8

Give a big-O estimate of the time to run merge sort on a

List of size n.

• f( n ) = 2 f( n / 2 ) + cn1.

• a = 2; b = 2; d = 1.

• Since 2 = 21, f( n ) is O( n1 log n ).

Copyright © Peter Cappello 9
 Strassen’s Matrix Product Algorithm

• Let C = A x B, where A, B, & C are n x n matrices.

• Do this recursively, in terms of n/2 x n/2 matrices.
Done conventionally, this results in:
8 n/2 x n/2 matrix products and 4 n/2 x n/2 matrix additions.
(Show on board)
• The time to perform the arithmetic is modeled by the following
recurrence equation:
T( n ) = 8T( n/2 ) + 4( n/2 )2 (explain)
• Applying the Master Theorem, where a = 8, b = 2, d = 2, we
obtain a big-O estimate: O( nlogb(a) ) = O( nlog2(8) ) = O( n3 ).
Copyright © Peter Cappello 10
• Strassen showed how to do this with:

7 n/2 x n/2 matrix products and 18 n/2 x n/2 matrix additions.

• The time to do the arithmetic is modeled by recurrence:

T( n ) = 7T( n/2 ) + 18( n/2 )2

• Apply the Master Theorem, where a = 7, b = 2, d = 2, to obtain a

big-O estimate:

O( nlogb(a) ) = O( nlog2(7) ) ≈ O( n2.81).

Copyright © Peter Cappello 11

 End

Copyright © Peter Cappello 12

 Exercise 20 (a)

Set up a divide-&-conquer recurrence relation for the # of modular

multiplies (MM) required to compute

an mod m, where a, m, n  Z+, using the recursive algorithm from

Example 3 in Section 4.4.

Let b = a mod m.

The algorithm then is:

– an mod m = ( an/2 mod m )2 mod m, for even n (1 MM)

– an mod m = ([(a(n-1)/2 mod m)2 mod m] . b) mod n,

for odd n (2 MMs)
Copyright © Peter Cappello 13
 Exercise 20 (a) Solution

The algorithm is:

an mod m = ( an/2 mod m )2 mod m, for even n (1 MM)

an mod m = ([(a(n-1)/2 mod m)2 mod m] . b) mod n, for odd n (2 MMs)

Let f( n ) denote the # of multiplies.

For even n, f( n/2 ) + 1 MM is used.

For odd n, f( n/2 ) + 2 MMs are used.

Worst case, f( n ) = f( n/2 ) + 2

(The recurrence is used in a big-O estimate of run time. It suffices to

know that the additive term is a constant.)
Copyright © Peter Cappello 14
 Exercise 20 (b)

Using f( n ) = f( n/2 ) + 2, construct a big-O estimate for

the # of modular multiplies used to compute an

mod m using the recursive algorithm.

Copyright © Peter Cappello 15

 Exercise 20 (b) Solution

Use the Master Theorem: f( n ) = a.f( n/b ) + c.nd

f( n ) = f( n/2 ) + 2

= 1f( n/2 ) + 2n0

a = 1, b = 2, d = 0.

So, a = bd

So, f( n ) is O( ndlog n ) = O( log n ).

Copyright © Peter Cappello 16

 Exercise 30

Exercise 30 assumes that f satisfies the conditions of the Master Theorem.

Use Exercise 29 to show that, for f(n) = a.f(n/b) + c.nd

if a = bd, then f(n) is O( nd log n ).

Exercise 29 gives us:

If a = bd & n = bk

then f(n) = f(1)nd + cnd logbn.

Copyright © Peter Cappello 17

 Exercise 30 Solution

To show:
f(n) = f(1)nd + cnd logbn is O( nd log n ).
So:
– f(n) growth clearly is dominated by cnd logbn.
– c is a constant.
– logbn differs from log2n by only a constant factor.
( log2b ) . logbn = log2( b logbn ) = log2n.
– cnd logbn therefore is O( nd log n ).
Copyright © Peter Cappello 18
 Theorem 1
Let f be an increasing function such that
f(n) = a.f(n/b) + c
whenever b | n, where a ≥ 1, b >1 are integers, & c > 0 is real.
If ( a > 1 ) then f(n) is O( nlogba )
If ( a = 1 ) then f(n) is O( logn ).
When n = bk where k > 0 is integer, and a > 1,
f(n) = C1 nlogba + C2,

where C1 = f(1) + c/(a – 1) and C2 = – c/(a – 1).

Copyright © Peter Cappello 19