You are on page 1of 18

Recurrence Relations

CS 3110

Textbook readings:
Chapter 4

CS 3110 — Recurrence Relations

1/12

Overview What are recurrence relations? What is their relationship to algorithm design? Methods to solve them: Substitution (induction in disguise) Recursion trees Master Theorem CS 3110 — Recurrence Relations 2/12 .

What Are Recurrence Relations? A recurrence relation expresses the value of a function f for an argument n in terms of the values of f for arguments less than n. Examples: f (n) = 1 if n ≤ 1 2f (n/2) + n if n > 1 f (n) = 5 if n ≤ 10 f (n/10) + f (n − 1) + f (n/2) if n > 10 CS 3110 — Recurrence Relations 3/12 .

q + 1. r) 5 Merge(A. p. r) CS 3110 — Recurrence Relations 4/12 . p. p. r) 1 if r > p 2 then q ← ⌊ p+r ⌋ 2 3 Merge-Sort(A. q) 4 Merge-Sort(A. Example: (Merge-Sort) Merge-Sort(A.Recurrence Relations and Recursive Algorithms The running time of a recursive algorithm is easily expressed using a recurrence relation. q.

Recurrence Relations and Recursive Algorithms The running time of a recursive algorithm is easily expressed using a recurrence relation. p. q) 4 Merge-Sort(A. r) Running time: T (n) = O(1) if n ≤ 1 2T (n/2) + O(n) if n > 1 CS 3110 — Recurrence Relations 4/12 . Example: (Merge-Sort) Merge-Sort(A. q + 1. q. p. p. r) 5 Merge(A. r) 1 if r > p 2 then q ← ⌊ p+r ⌋ 2 3 Merge-Sort(A.

Recurrence Relations and Optimization Problems Given: Set T of n tasks t1 . K) = the minimal cost of assigning the tasks in T to the contractors in K. K \ {ki })) if T = ∅ CS 3110 — Recurrence Relations 5/12 . k2 . tn Set K of n contractors k1 . A recurrence relation: C = c(T. . . K) = 0 if T = ∅ min1≤i≤n (ci1 + c(T \ {t1 }. . t2 . . . kn Contractor ki charges cij dollars for performing task tj Assign one task to every contractor Goal: Minimize the total cost C. . . K) c(T. . c(T.

Solving Recurrence Relations Given two algorithms for the same problem. Running times: Algorithm 1: T1 (n) = 2T1 (n/2) + O(n lg n) Algorithm 2: T2 (n) = 3T2 (n/2) + O(n) Which one is faster? CS 3110 — Recurrence Relations 6/12 .

Solving Recurrence Relations Given two algorithms for the same problem.) CS 3110 — Recurrence Relations 6/12 . Running times: Algorithm 1: T1 (n) = 2T1 (n/2) + O(n lg n) Algorithm 2: T2 (n) = 3T2 (n/2) + O(n) Which one is faster? We need closed forms of T1 (n) and T2 (n). (Expressions for T1 (n) and T2 (n) that are not recurrence relations.

(Expressions for T1 (n) and T2 (n) that are not recurrence relations.) To “solve” a recurrence relation means to derive such a closed form from the recurrence relation. CS 3110 — Recurrence Relations 6/12 .Solving Recurrence Relations Given two algorithms for the same problem. Running times: Algorithm 1: T1 (n) = 2T1 (n/2) + O(n lg n) Algorithm 2: T2 (n) = 3T2 (n/2) + O(n) Which one is faster? We need closed forms of T1 (n) and T2 (n).

Methods for Solving Recurrence Relations Substitution method: Make a guess Verify the guess using induction Recursion trees: Visualize how the recurrence unfolds May lead to a guess to be verified using substitution If done carefully. may lead to an exact solution Master theorem: “Cook-book” solution to a common class of recurrence relations CS 3110 — Recurrence Relations 7/12 .

Substitution Method Three steps: 1. Verify the guess for all n ≤ n0 . Prove that the guess is correct assuming that it can be verified for all n less than some n0 . (inductive step) 3. Make a guess 2. (base case) CS 3110 — Recurrence Relations 8/12 .

Verify the guess for all n ≤ n0 . Prove that the guess is correct assuming that it can be verified for all n less than some n0 .Substitution Method Three steps: 1. Make a guess 2. (inductive step) 3. (base case) Why do we switch the two parts of the inductive proof? CS 3110 — Recurrence Relations 8/12 .

Make a guess 2. (base case) Why do we switch the two parts of the inductive proof? Our guess is vague. CS 3110 — Recurrence Relations 8/12 .Substitution Method Three steps: 1. Verify the guess for all n ≤ n0 . Prove that the guess is correct assuming that it can be verified for all n less than some n0 . (inductive step) 3.

Prove that the guess is correct assuming that it can be verified for all n less than some n0 . we do not care. The inductive step may work only for certain values of c. and we do not want to know. Make a guess 2. The base case usually works for any value of c. T (n) = O(n lg n)) We do not know c. Verify the guess for all n ≤ n0 .Substitution Method Three steps: 1. Example: T (n) ≤ cn lg n (that is. CS 3110 — Recurrence Relations 8/12 . (base case) Why do we switch the two parts of the inductive proof? Our guess is vague. (inductive step) 3.

black magic. Recursion trees help us to visualize how the recurrence unfolds. . prove loose upper and lower bounds and tighten them. .How To Make a Guess Experience. If everything fails. CS 3110 — Recurrence Relations 9/12 . inspiration. .

Recursion Trees T (n) = 3T (n/4) + n2 n2 Cost n2 n 2 4 n 2 4 n 2 4 3 16 n2 n 2 16 n 2 16 n 2 16 n 2 16 n 2 16 n 2 16 n 2 16 n 2 16 n 2 16 3 2 16 n2 T (1) T (1) T (1) O(nlog4 3 ) Cost of the tree = sum of the costs of the levels T (n) = Θ(n2 ) CS 3110 — Recurrence Relations 10/12 .

for some ǫ > 0. for some c < 1 and all n ≥ n0 . (ii) If f (n) = Θ(nlogb a ). then T (n) = Θ(f (n)).Master Theorem Theorem: (Master Theorem) Let a ≥ 1 and b > 1. (iii) If f (n) = Ω(nlogb a+ǫ ). CS 3110 — Recurrence Relations 11/12 . for some ǫ > 0. then T (n) = Θ(nlogb a lg n). then T (n) = Θ(nlogb a ). and let T (n) be given by the following recurrence: T (n) = aT (n/b) + f (n) (i) If f (n) = O(nlogb a−ǫ ). and af (n/b) ≤ cf (n). let f (n) be a function over the positive integers.

There are three methods to solve recurrence relations: Substitution method Recursion trees Master theorem CS 3110 — Recurrence Relations 12/12 . Recurrence relations are useful for expressing the running times of recursive algorithms and the costs of optimal solutions to optimization problems.Summary A recurrence relation expresses the value of a function f for an argument n in terms of the values of f for arguments less than n.