Professional Documents
Culture Documents
Recur
Recur
Solving recurrences
• The analysis of integer multiplication
from Lecture 1 required us to solve a
recurrence.
• Recurrences are a major tool for
analysis of algorithms
• Today: Learn a few methods.
• Lecture 3: Divide and Conquer
algorithms which are analyzable by
recurrences.
L2.2
Recall: Integer Multiplication
L2.3
Substitution method
The most general method:
1. Guess the form of the solution.
2. Verify by induction.
3. Solve for constants.
Example: T(n) = 4T(n/2) + 100n
• [Assume that T(1) = (1).]
• Guess O(n3) . (Prove O and separately.)
• Assume that T(k) ck3 for k < n .
• Prove T(n) cn3 by induction.
L2.4
Example of substitution
T ( n) 4T ( n / 2) 100n
4c(n / 2)3 100n
(c / 2) n3 100n
cn3 ((c / 2) n3 100n )
desired – residual
cn 3 desired
whenever (c/2)n3 – 100n 0, for
example, if c 200 and n 1.
residual
L2.5
Example (continued)
• We must also handle the initial conditions,
that is, ground the induction with base
cases.
• Base: T(n) = (1) for all n < n0, where n0
is a suitable constant.
• For 1 n < n0, we have “(1)” cn3, if we
pick c big enough.
L2.7
A tighter upper bound!
IDEA: Strengthen the inductive hypothesis.
• Subtract a low-order term.
Inductive hypothesis: T(k) c1k2 – c2k for k < n.
T ( n) 4T ( n / 2) 100n
4(c1 (n / 2) 2 c2 (n / 2)) 100n
c1n 2 2c2 n 100n
c1n 2 c2 n (c2 n 100n)
c1n 2 c2 n if c > 100.
2
Pick c1 big enough to handle the initial conditions.
L2.8
Recursion-tree method
• A recursion tree models the costs (time) of a
recursive execution of an algorithm.
• The recursion tree method is good for
generating guesses for the substitution
method.
• The recursion-tree method can be unreliable,
just like any method that uses ellipses (…).
• The recursion-tree method promotes intuition,
however.
L2.9
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
L2.10
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
T(n)
L2.11
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
T(n/4) T(n/2)
L2.12
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2
L2.13
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
(n/4)2 (n/2)2
(1)
L2.14
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
(n/4)2 (n/2)2
(1)
L2.15
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
5 n2
(n/4) 2 (n/2) 2
16
(n/16)2 (n/8)2 (n/8)2 (n/4)2
…
(1)
L2.16
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
5 n2
(n/4) 2 (n/2) 2
16
25 n 2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
256
…
…
(1)
L2.17
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2 n2
5 n2
(n/4) 2 (n/2) 2
16
25 n 2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
256
…
…
(1) Total = n2
5 5
1 16 16
2
5 3
16
= (n2) geometric series
L2.18
Appendix: geometric series
n 1
1 x
1 x x2 xn for x 1
1 x
2 1
1 x x for |x| < 1
1 x
Return to last
slide viewed.
L2.19
The master method
L2.20
Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)
…
…
#leaves = ah
= alogbn nlogba(1)
(1)
= nlogba
L2.21
Three common cases
Compare f (n) with nlogba:
1. f (n) = O(nlogba – ) for some constant > 0.
• f (n) grows polynomially slower than nlogba
(by an n factor).
Solution: T(n) = (nlogba) .
L2.22
Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)
…
C ASE 1:
1: The
The weight
weight increases
…
CASE increases
geometrically
geometrically from
from the
the root
root to
to the
the nlogba(1)
(1) leaves.
leaves. The
The leaves
leaves hold
hold aa constant
constant
fraction
fraction of
of the
the total
total weight.
weight. (nlogba)
L2.23
Three common cases
Compare f (n) with nlogba:
L2.24
Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)
…
…
CCASEASE 2:
2: (k
(k == 0)
0) The
The weight
weight
isis approximately
approximately the the same
same on
on
nlogba(1)
(1)
each
each ofof the
the log
logbbnn levels.
levels.
(nlogbalg n)
L2.25
Three common cases (cont.)
Compare f (n) with nlogba:
3. f (n) = (nlogba + ) for some constant > 0.
• f (n) grows polynomially faster than nlogba (by
an n factor),
and f (n) satisfies the regularity condition that
a f (n/b) c f (n) for some constant c < 1.
Solution: T(n) = ( f (n) ) .
L2.26
Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)
…
C ASE 3:
3: The
The weight
weight decreases
…
CASE decreases
geometrically
geometrically from
from the
the root
root to
to the
the nlogba(1)
(1) leaves.
leaves. The
The root
root holds
holds aa constant
constant
fraction
fraction of
of the
the total
total weight.
weight. ( f (n))
L2.27
Examples
L2.28
Examples
Ex. T(n) = 4T(n/2) + n3
a = 4, b = 2 nlogba = n2; f (n) = n3.
CASE 3: f (n) = (n2 + ) for = 1
and 4(cn/2)3 cn3 (reg. cond.) for c = 1/2.
T(n) = (n3).
L2.29
Conclusion
L2.30