Recurrences
Solving recurrences
Methods for solving recurrences: 1. Substitution method. With substitution you need to guess the result and prove it by induction. 2. Iteration method Recursion-tree method (Master method) Iteration is constructive, i.e. you figure out the result; somewhat tedious because of summations. 4
4 Solving recurrences by induction (the Substitution method)
Idea: Make a guess for the form of the solution and prove by induction. Can be used to prove both upper bounds O() and lower bounds (). Lets solve T(n) = 2T(n/2) + n using substitution Guess T(n) _ cn log n for some constant c (that is, T(n) = O(n log n)) Proof: _ Base case: we need to show that our guess holds for some base case (not necessarily n = 1, some small n is ok). Ok, since function constant for small constant n. _ Assume holds for n/2: T(n/2) _ cn 2 log n 2 (Question: Why not n 1?) Prove that holds for n: T(n) _ cn log n T(n) = 2T(n/2) + n _ 2(c n 2 log n 2 )+n = cn log n 2 +n = cn log n cn log 2 + n = cn log n cn + n So ok if c _ 1 Similarly it can be shown that T(n) = (n log n) Exercise! Similarly it can be shown that T(n) = T(bn 2 c) + T(dn 2 e) + n is _(n lg n). Exercise! The hard part of the substitution method is often to make a good guess. How do we make a good (i.e. tight) guess??? Unfortunately, theres no recipe for this one. Try iteratively O(n3), (n3),O(n2), (n2) and so on. Try solving by iteration to get a feeling of the growth. 5
5 Solving Recurrences with the Iteration/Recursion-tree Method
In the iteration method we iteratively unfold the recurrence until we see the pattern. The iteration method does not require making a good guess like the substitution method (but
it is often more involved than using induction). Example: Solve T(n) = 8T(n/2) + n2 (T(1) = 1) T(n) = n2 + 8T(n/2) = n2 + 8(8T( n 22 ) + (n 2 )2) = n2 + 82T( n 22 ) + 8(n2 4 )) = n2 + 2n2 + 82T( n 22 ) = n2 + 2n2 + 82(8T( n 23 ) + ( n 22 )2) = n2 + 2n2 + 83T( n 23 ) + 82(n2 42 )) = n2 + 2n2 + 22n2 + 83T( n 23 ) =... = n2 + 2n2 + 22n2 + 23n2 + 24n2 + . . . Recursion depth: How long (how many iterations) it takes until the subproblem has constant size? i times where n 2i = 1 ) i = log n What is the last term? 8iT(1) = 8log n T(n) = n2 + 2n2 + 22n2 + 23n2 + 24n2 + . . . + 2log n1n2 + 8log n =
logXn1 k=0 2kn2 + 8log n
= n2
logXn1 k=0
2k + (23)log n Now
Plog n1
k=0 2k is Plog n1 k=0 2k =
a geometric sum so we have
_(2log n1) = _(n) (23)log n = (2log n)3 = n3 T(n) = n2 _(n) + n3 = _(n3) 6
5.1 Recursion tree
A different way to look at the iteration method: is the recursion-tree, discussed in the book (4.2). we draw out the recursion tree with cost of single call in each noderunning time is sum of costs in all nodes if you are careful drawing the recursion tree and summing up the costs, the recursion tree is a direct proof for the solution of the recurrence, just like iteration and substitution Example: T(n) = 8T(n/2) + n2 (T(1) = 1)
(n/2)2 (n/2)2 (n/4)2 (n/4)2 n2 n2 (n/2)2 (n/2)2 T(n/4) n2 T(n/4) T(n)
1)
T(n/2) T(n/2) 2) n2 T(1) T(1) T(1) 8(n/2) = 2n 8 (n/4) = 2 n
22 2222
log n 8 T(1) + + + +
3) log n)
T(n) = n2 + 2n2 + 22n2 + 23n2 + 24n2 + . . . + 2log n1n2 + 8log n 7
6 Master Method
We have solved several recurrences using substitution and iteration. we solved several recurrences of the form T(n) = aT(n/b) + nc (T(1) = 1). Merge-sort ) T(n) = 2T(n/2) + n (a = 2, b = 2, and c = 1). It would be nice to have a general solution to the recurrence T(n) = aT(n/b) + nc. We do! T(n) = aT
n
b
+ nc a _ 1, b _ 1, c > 0 + T(n) =
8>< >:
_(nlogb a) a > bc _(nc logb n) a = bc _(nc) a < bc Proof (Iteration method) T(n) = aT
n
b
+ nc = nc + a
_n
b
_c + aT _
n b2
__
= nc +
a
bc
nc + a2T
_
n b2
= nc +
a
bc
nc + a2
__
n b2 _c +
aT
_
n b3
__
= nc +
a
bc
nc +
a
bc _2 nc +
a3T
_
n b3
= ... = nc +
a
bc
nc +
a
bc
_2 nc + a
bc
_3 nc + a
bc
_4 nc + ... + a
bc _logb n1 nc +
alogb nT(1)
= ncPlogb n1
k=0 a bc _k +
alogb n = ncPlogb n1
k=0 a bc
_k + nlogb a
Recall geometric sum
Pnk
=0 xk = xn+11 x1 =
_(xn) 8 a < bc a < bc , a bc < 1 )
Plogb n1
k=0 a bc _k _ P+1 k=0 a bc
_k = 1
1(
bc a
) = _(1) a < bc , logb a < logb bc = c T(n) = ncPlogb n1
k=0 a bc
_k + nlogb a
= nc _(1) + nlogb a = _(nc) a = bc a = bc , a bc = 1 )
Plogb n1
k=0 a bc _k = Plogb n1 k=0 1
= _(logb n) a = bc , logb a = logb bc = c T(n) =
Plogb n1
k=0 a bc
_k + nlogb a
= nc_(logb n) + nlogb a = _(nc logb n) a > bc a > bc , a bc > 1 )
Plogb n1
k=0 a bc
_k = _ _ a
bc _logb n
=_
_
alogb n (bc)logb n
=_
_
alogb n nc
T(n) = nc _
_
alogb n nc
+ nlogb a = _(nlogb a) + nlogb a = _(nlogb a) Note: Book states and proves the result slightly differently (dont read it).
7 Changing variables
Sometimes reucurrences can be reduced to simpler ones by changing variables Example: Solve T(n) = 2T( p n) + log n Let m = log n ) 2m = n ) p n = 2m/2 T(n) = 2T( p n) + log n ) T(2m) = 2T(2m/2) + m Let S(m) = T(2m) T(2m) = 2T(2m/2) + m ) S(m) = 2S(m/2) + m ) S(m) = O(mlogm) ) T(n) = T(2m) = S(m) = O(mlogm) = O(log n log log n) 9
8 Other recurrences
Some important/typical bounds on recurrences not covered by master method: Logarithmic: _(log n) Recurrence: T(n) = 1 + T(n/2) Typical example: Recurse on half the input (and throw half away) Variations: T(n) = 1 + T(99n/100) Linear: _(N) Recurrence: T(n) = 1 + T(n 1) Typical example: Single loop Variations: T(n) = 1+2T(n/2), T(n) = n+T(n/2), T(n) = T(n/5)+T(7n/10+6)+n
Quadratic: _(n2) Recurrence: T(n) = n + T(n 1) Typical example: Nested loops Exponential: _(2n) Recurrence: T(n) = 2T(n 1) 10
