You are on page 1of 39

COMP2230 Introduction to Algorithmics Lecture 3

A/Prof Ljiljana Brankovic

Lecture Overview
Analysis of Algorithms - examples Recurrence Relations continued -Text, Section 2.4

Next Lecture:
Data Structures Text, Chapter 3

Example 1
Show that for any real a and b, b > 0 (n+a)b = (nb) Big O: Show that there are constants C1 and n0 such that (n+a)b C1 nb, for all n > n0 (n+a)b 2b nb for all n > a Big : Show that there are constants C2 and n0 such that (n+a)b C2 nb, for all n > n0 (n+a)b (1/2)b nb for all n > 2|a|

Example 2
When is f(2n) = (f(n) )? f(n) = n2

Big O: (2n)2 4 n2, for all n > 0 Big : (2n)2 n2, for all n > 0

f(n) = 2n

Big O: 22n C1 2n, for all n > n0 ??? 2n 2n C1 2n ??? NO!!! There is no constant C1 such that 2n C1 for all n > n0.

Solving Recurrence Relations


Iteration (substitution): Example from last weeks lecture C(n) = n + C(n/2), n > 1 C(1) = 0 Solution for n=2 k, for some k. C(2k) = 2k + C(2k-1) = 2k + 2k-1 + C(2k-2) = 2k + 2k-1 + + 21 + C(20) = 2k + 2k-1 + + 21 + C(1) = 2k + 2k-1 + + 21 + 0 = 2k+1 2 = 2n-2 = (n).

(as C(1) = 0)

Solving Recurrence Relations


Solving the same recurrence for any n: C(n) = n + C(n/2), n > 1 C(1) = 0 Solution for 2k-1 n < 2k , for some k: Since C(n) is an increasing function (prove it!): C (2k-1) C(n) < C (2k ) C(2k ) = 2k+1 -2, C(2k-1) = 2k -2 C(n) < C(2k ) = 2k+1 -2 < 4n, thus C(n) = O(n) C(n) = C(2k-1) = 2k 2 > n-2 = n/2, thus C(n) = (n) Thus C(n) = (n).

Main Recurrence Theorem (also known as Master Theorem)


Let a, b and k be integers satisfying a 1, b 2 and k 0. In the following, n/b denotes either n/b or n/b. In the case of the floor function the initial condition T(0) =u is given; and in the case of the ceiling function, the initial condition T(1) = u is given.

Recurrence Theorem (also known as Master Theorem)


Upper Bound

If T(n) aT(n/b) + f(n) and f(n) =O(n k) then

T(n) =

O(n k) O(n k log n) O(n log a)


b

if a < b k if a = b k if a > b k

The same theorem holds for and notations as well.

Recurrence Theorem (also known as Master Theorem)


Lower Bound

If T(n) aT(n/b) + f(n) and f(n) =(n k) then (n k)

T(n) =

(n k log n) (n
logba)

if a < b k if a = b k if a > b k

Recurrence Theorem (also known as Master Theorem)


Tight Bound

If T(n) = aT(n/b) + f(n) and f(n) = (n k) then

T(n) =

(n k) (n k log n) (n log a)
b

if a < b k if a = b k if a > b k

Example 1
Solve cn = n + cn/2, n > 1; c1 = 0 We have a = 1, b = 2, f(n) = n and k=1. Since a < b k, cn = (n k) and since k = 1 , cn = (n). If T(n) = aT(n/b) + f(n) and f(n) = (n k) then

T(n) =

(n k) (n k log n) (n logba)

if a < b k if a = b k if a > b k

Example 2
Solve cn = n + 2cn/2 , n > 1; c1 = 0 We have a = 2, b = 2, f(n) = n and k=1. Since a = b k , cn = (n k log n) and since k = 1 , cn = (n log n). If T(n) = aT(n/b) + f(n) and f(n) = (n k) then

T(n) =

(n k) (n k log n) (n logb a)

if a < b k if a = b k if a > b k

Solve cn = n + 7cn/4

Example 3

We have a = 7, b = 4, f(n) = n and k=1. Since a > b k, cn = (n log4 7 ) If T(n) = aT(n/b) + f(n) and f(n) = (n k) then

T(n) =

(n k) (n k log n) (n logb a)

if a < b k if a = b k if a > b k

Solve cn n

+ cn/2 + cn/2

Example 4

If cn is nondecreasing then cn/2 cn/2 and we have cn n 2 + 2cn/2 We have a = 2, b = 2, f(n) = n2 and k=2. Since a < b k, cn = O(n k) and since k=2, cn = O(n 2) If T(n) = aT(n/b) + f(n) and f(n) = (n k) then

T(n) =

(n k) (n k log n) (n logb a)

if a < b k if a = b k if a > b k

Other ways for solving recurrences


Guess and prove by mathematical induction Match with familiar forms homogeneous/ non-homogeneous forms

Examples
Solve the following recurrences:

1. T(n) = T(n-1) + nr 2. T(n) = 2 T(n-1) + nr


(Note: The solution will depend on the value of the parameter r and the initial conditions, which both need to be specified)

Example
0 T(n) = 2 + 4T(n/2) otherwise if n = 0

Restrict n to be a power of 2: n = 2k Look at the expansion


n 20 21 22 T(n) 2 2+42 2 + 4 2 + 42 2

Example
Pattern seems to suggest:
T(n) = T(2k) = i=0k 2 4i = 2 (4k+1 - 1)/(4-1) = (8n2 - 2)/3 = O(n2)

Proof by mathematical induction: Base case: k = 0, n=20


T(1) = (8 - 2)/3 = 2 basis holds.

Assumption: True for k. Proof that it is true for k+1: T(2k+1 ) = 2 + 4T(2k ) = 2 + 4 (822k 2)/3 = (6 + 8 22k+2 -8)/3 = (8 22k+2 - 2)/3

We assumed n is a power of 2 This gives a conditional bound T(n) is in O(n2 | n is a power of 2) Can we remove this condition? We can use smoothness (FoA p.89)

basically show that well-behaved functions which have bounds at certain points, have the same bounds at all other points

Smoothness
A function f is smooth if: f is non-decreasing over the range [N, +)
eventually non-decreasing f(bn) is O(f(n))

For every integer b 2 Let f(n) be smooth Let t(n) be eventually non-decreasing t(n) is (f(n)) for n a power of b implies t(n) in (f(n))

Smoothness
Formally, a function f defined on positive integers is smooth if for any positive integer b 2, there are positive constants C and N, depending on b, such that f(bn) Cf(n) and f(n) f(n+1) for all n N.

Smoothness
Lemma 1: If t is a nondecreasing function, f is a smooth function, and t(n) = (f(n)) for n a power of b, then t(n) = (f(n)).

Smoothness
Which functions are smooth?
log and polynomial functions are smooth eg. log(n) , log2(n), n3 - n, n log(n) superpolynomial functions are not eg. 2n , n! Using this, we can use guess-and-check a bit more loosely: prove for easier cases, and use smoothness for the rest

Proof of the Main Recurrence Theorem


Let a, b and k be integers satisfying a 1, b 2 and k 0. In the following, n/b denotes either n/b or n/b. In the case of the floor function the initial condition T(0) =u is given; and in the case of the ceiling function, the initial condition T(1) = u is given. We first need to show for both floor and ceiling T(n) is well defined for all n. We use

Proof of the Main Recurrence Theorem


Next we need to show that
m +1 m +1 x y m i i x y = ,x y x y i =0 m

We use mathematical induction to show this. Base Case: m=0 x0y0 = (x1 y1)/(x-y), that is, 1 = 1 Inductive Assumption: m=k
k
i=0

xk-iyi = (xk+1 yk+1)/(x-y)

Proof of the Main Recurrence Theorem


Inductive Step: m=k+1
k+1
i=0

xk+1-iyi = (xk+2 yk+2)/(x-y) xk+1-iyi =


k
i=0

k+1
i=0

xk+1-iyi + x0yk+1 =

k
i=0

xxk-iyi + yk+1 =

=x(xk+1 yk+1)/(x-y) +yk+1 = (xk+2 xyk+1 +xyk+1 yk+2)/(x-y) = = (xk+2 yk+2)/(x-y)

Proof of the Main Recurrence Theorem


We are now ready to formulate a lemma that we shall latter use to prove the Main Recurrence Theorem. Lemma 2: If n>1 is a power of b and a bk the solution of recurrence relation T(n) = aT(n/b) + cnk is T(n) = C1nlogba + C2nk for some constants C1 and C2, where C2>0 for a<bk and C2<0 for a>bk. If n>1 is a power of b and a = bk the solution of recurrence relation T(n) = aT(n/b) + cnk is T(n) = C3nk + C4nk logbn for some constants C3 and C4>0.

Proof of the Main Recurrence Theorem


Proof:
Suppose that n = bm. Then m = logb n, nk = (bm)k=(bk) m and am = (blog a) m = nlog a Then we have: T(n) = aT(n/b) + cnk T(n) = T(b m) = aT(b m-1) +c(bk) m = a[aT(b m-2) +c(bk) m-1] + c(bk) m = a2T(b m-2) +c[a(bk) m-1+(bk)m] = a2[aT(b m-3) +c(bk) m-2] +c[a(bk) m-1+(bk)m] = a3T(b m-3) +c[a2(bk) m-2+a(bk)m-1 +(bk)m] ... m =a mT(b0)+c a m-i(bk)i
b b

i=1

Proof of the Main Recurrence Theorem


Proof (contd):
Thus T(n) = a mT(b0)+c
m

m
i=1

am-i(bk)i

If a bk, using T(n)

x
i =0

m i

x m +1 y m +1 y = ,x y x y
i

we get

= a mT(1) + c[((bk) m+1-am+1)/(bk-a)-am] = a mT(1) + c(bk) m+1/(bk-a) + c[(-am+1)/(bk-a)-am] = a mT(1) + c(bk) m+1/(bk-a) amcbk/(bk-a) = [T(1)- cbk/(bk-a)] a m + [cbk/(bk-a)] (bk) m = [T(1)- cbk/(bk-a)] nlogb a + [cbk/(bk-a)] (bk)m = C1 nlogb a + C2 (bk)m

where C1 = T(1)- cbk/(bk-a) and C2 = cbk/(bk-a). Note that if a < bk then C2 > 0 and if if a > bk then C2 < 0.

Proof of the Main Recurrence Theorem


Proof (contd): Recall T(n) = amT(b0)+c If a = bk , and we get T(n) = amT(1) + c
m
i=1

m
i=1

a m-i(bk )i

am

= amT(1) + cmam = C3nk + C4nk logbn where C3 = T(1) and C4 = c>0. Note that if a < bk then C2 > 0 and if if a > bk then C2 < 0.

Proof of the Main Recurrence Theorem

Lemma 2 proves the main Recurrence Theorem for case where n is a power of b. We then apply Lemma 1 (smooth lemma) to generalize the proof for any n (see text page 63 and 64 for details).

Homogeneous Recurrences
Characteristic equation of the recurrence: a0tn + a1tn-1 + a2tn-2 + ..... aktn-k = 0 We are looking for tis. The recurrence is: Linear, as it does not contain ti2, etc. Homogeneous, as the linear combination of tis is equal to 0. With constant coefficients, as ais are constants.

Homogeneous Recurrences
Guess: tn = xn, for an (as yet) unknown x a0xn + a1xn-1 + a2xn-2 + ..... akxn-k = 0 (a0xk + a1xk-1 + a2xk-2 + ..... akx0)xn-k = 0 Trivial solution x=0 is not interesting for us. Thus, a0xk + a1xk-1 + a2xk-2 + ..... akx0 = 0

Homogeneous (cont)
p(x) is the Characteristic Polynomial; any polynomial of degree k has exactly k roots (not necessarily all distinct). Let r1 r2 .... rk be the roots of this polynomial. If roots are distinct, the general solution to the recurrence is k tn = ci rin where ci are some constants that can be determined from initial conditions.
i =1

p(x) = a0xk + a1xk-1 + a2xk-2 + ..... akx0 = 0

Example
T(n) = 1 2T(n-1) n 1 otherwise T(n) = xn, T(n-1) = xn-1

T(n) - 2T(n-1) = 0, Solve xn - 2xn-1 = 0: (x - 2)xn-1 = 0, T(n) = c12n, Big-O: T(n) is O(2n)

x 2 = 0, T(1) = 1 = c121,

x=2 c1 = 1/2

Another example
Fibonacci Sequence n fn = fn-1 + fn-2 f0 0 f1 1 f2 1 f3 2 otherwise f4 3 f5 5 f6 8 f7 13 ... ... if n=0 or n=1

Example - Fibonacci (cont)


Recurrence? Characteristic equation? Roots? General form? Constants? fn ?
fn - fn-1 - fn-2 = 0 x2 - x - 1 = 0

x1 = (1 + 5)/2, x2 = (1 - 5)/2 fn = c1 .[(1 + 5)/2]n + c2 .[(1 - 5)/2]n f0 = c1 + c2 = 0, c = c1 = - c2 f1 = (c/2) (1 + 5 1 + 5) = c 5 = 1 C = 5/5 fn = 5/5 {.[(1 + 5)/2]n - [(1 - 5)/2]n}

What if roots are not all distinct ?


If r1, , rk are k distinct roots with multiplicities m1 to mk, respectively, then the general solution is:

t n = cij n j ri n
i =1 j = 0

k mi 1

Example
n T(n) = 5tn-1-8tn-2+4tn-3 otherwise if n = 0, 1 or 2

The recurrence: tn - 5tn-1+8tn-2-4tn-3 = 0 Characteristic Polynomial: X3 -5x2 + 8x -4 = (x 1) (x -2)2 Roots: r1 = 1, multiplicity m1 = 1, and r2 = 2, multiplicity m2 = 2. General solution: tn = c11n + c22n + c3n2n From initial conditions we get c1 = -1, c2 = 2 and c3 = -(1/2)

You might also like