You are on page 1of 8

Recurrence Relations Recursive Algorithm Recursive algorithm invokes (makes reference to) itself repeatedly until a certain condition matches

Examples: Computing factorial function Tower of Hanoi puzzle Digits in binary representation Non-recursive algorithm is executed only once to solve the problem. Steps in mathematical analysis of recursive algorithms: • Decide on parameter n indicating input size. • Identify algorithm’s basic operation • Set up a recurrence relation and initial condition(s) for T(n)-the number of times the basic operation will be executed for an input of size n o Alternatively count recursive calls. • Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution Recurrence relations result naturally from the analysis of recursive algorithms, specifying the algorithm's run time; solving recurrence relations yields a closed-end formula for calculation of run time. Why are recurrences good things? 1. Many natural functions are easily expressed as recurrences:

2. It is often easy to find a recurrence as the solution of a counting problem. Solving the recurrence can be done for many special cases as we will see, although it is somewhat of an art.

For example, the recursive n! algorithm of: int factorial (int n) { if (n == 1) return 1; else return factorial (n-1) * n; } The running time, T(n), can be defined as:

We always want to solve these recurrence relations by getting an equation for T. where T appears on just the left side of the equation. t(n) = t(n-1) + 1 (eqn 1) t(1) = 1 (eqn 2) Plugging in: t(2) = t(2-1) + 1 (by eqn 1) = t(1) + 1 = 1 + 1 = 2 (by eqn 2) t(3) = t(2) + 1 = 2 + 1 = 3 Now we guess that t(n) = n. where T appears on both the left and right sides of the equation. This is the one we will discuss here. One is to plug in small values of n and deduce a pattern.T(1) = 1 T(n) = T(n-1) + 4 for all n>1 Eg. which is correct . The recurrence relation is an intermediate step in the process of determining the big-O running time of an algorithm. T(n) = 2 * T(n=2)+n is an example of a recurrence relation A Recurrence Relation is any equation for a function T. Examples 1. A closed-form is a function like t(n) = nlog n (which is the answer for the recurrence relation above). How do you guess a solution? There are two ways. three methods of solving recurrence relation • • • Forward Substitution Backward Substitution Homogeneous Equation method Forward Substitution Method Consider a recurrence relation such as: t(n) = 2t(n/2) + n t(1) = 0 This is not a "closed-form". After determining the recurrence relation we need to find a closed-form. Solving Recurrence Relations We study here. The simplest way to solve easy recurrence relations it to guess a solution and prove it by induction.

If it doesn't work you need to change your guess. Proof Claim : For the recurrence relation t(n)= n Proof (by induction) Basis: We claim that t(1) = 1.After you guess the closed form for a recurrence relation you need to prove it correct by induction.H. Inductive Step: We must show that t(n) = n(n+1)/2. Proof Claim: For this recurrence relation.. t(n) = n(n+1)/2 Proof (by induction): Basis: We claim that t(1) = 1(2)/2 = 1. 3.n + 2n)/2 = (n2 + n)/2 = n(n+1)/2 Therefore the claim is true. which is the value given by the recurrence relation so the claim is true for n = 1. Inductive Hypothesis: Assume t(n-1) = (n-1)*n/2 for some n.. + n =n(n+1)/2. which is the value given by the recurrence relation Inductive Hypothesis: Assume t(n-1) = (n-1) for some n Inductive Step: We must show that t(n) = n t(n)= t(n-1)+1 =( n-1) + 1 by inductive hypothesis =n Therefore the claim is true 2. t(n) = t(n/2) + n t(1) = 1 . The proof will work out if and only if your guess is correct (assuming you are doing inductive proofs correctly). t(n) = t(n-1) + n = (n-1)*n/2 + n by I. t(n) = t(n-1) + n t(1) = 1 Plugging in: t(2) = t(1) + 2 = 1+2=3 t(3) = t(2) + 3 =3+3=6 t(4) = t(3) + 4 = 6 + 4 = 10 This one is actually tricky unless you notice that the sum is 1 + 2 + . = (n*n .

if n = 1. Plugging in: t(2) = t(1) + 2 = 1 + 2 = 3 t(4) = t(2) + 4 = 3 + 4 = 7 t(8) = t(4) + 8 = 7 + 8 = 15 t(16) =t(8) + 16 = 15 + 16 = 31 ------^ Notice that in the 3rd column we find that t(n) = (n-1) + n = 2n-1. t(n) = 2t(n/2) + 1. t(1) = 1 2. 3. t(1) = 0 3.. t(1) = 1 2.When dealing with n/2 you may consider only values that are powers of 2 in this class. t(2) = 1 Backward Substitution Method T(n) = 1 . if n > 0 Solution: STEP 1: Apply enough substitutions to find a pattern i = 1 T(n) = 2 T(n − 1) + 1 i = 2 T(n) = 2[2T(n − 2) + 1] + 1 = 22T(n − 2) + 2+ 1 i = 3 T(n) = 22 [2T(n − 3) + 1] + 2 + 1 = 23 T(n − 3) + 22 + 2 + 1 i = 4 T(n) = 23 [2T(n − 4) + 1] + 22 + 2 + 1 = 24T(n − 4) + 23 + 22 + 2 + 1 …. t(n) = t(n/2) + n. Give Proof yourself Practice Problems: Guess solutions for the following recurrence relations. t(n) = t(n-1) + n. t(n) = t(n/2) + n. t(n) = 2*t(n/2) + 3n. This is correct. 1. t(1) = 1 For each of the recurrence relations below. 2T(n − 1) + 1. find the solution 1. t(1) = 1. STEP 2: Guess what the recurrence is after i substitutions: T(n) = 2iT(n − i) + (2i + 2i-1 + 2i-2 + …. t(n) = t(n-1) + 2. + 22 + 2 + 1) .

Solution STEP 1: Apply enough substitutions to find a pattern i = 1 T(n) = T(n − 1) + c i = 2 T(n) = T(n − 2) + 2c i = 3 T(n) = T(n − 3) + 3c i = 4 T(n) = T(n − 4) + 4c STEP 2: Guess the recurrence after i substitutions: T(n) = T(n − i) + ic STEP 3: We terminate when the recurrence reaches a base case . T(n) = 2T(n/2) + n if n = 3. T(n − 1) + c.for this problem. when n − i = 3 or i = n − 3. + 22 + 2 + 1) = (2i – 1)/ 2 -1 STEP 3: Solve for i when the recurrence reaches a base case: For this problem. STEP 4: Substitute back into the recurrence guessed in STEP 2: T(n) = T(3) + (n − 3)c T(n) = 7 + (n − 3)c Hence solved.Sum of (2i + 2i-1 + 2i-2 + …. when n − i = 1 (since T(1) is the base case) or i = n − 1. the Open form of the equation. STEP 4: Substitute the value for i back into the equation. Given a Recurrence. T(1) = 1 T(n) = 2T(n/2) + n Backward Substitution Find T(n) in terms of i by using Backward Substitution and looking for a pattern. if n > 3 = 2n – 1 . T(n) = 2n−1T(1) + 2n−1 − 1 = 2n−1 + 2n−1 − 1 T(n) Example 2 Recurrence: T(n) = 7.

.. ..bk are real numbers •bk ≠ 0.= 2[2T(n/22) + n/2] + n = 22T(n/22) + n + n = 22T(n/22) + 2n After ith substitution T(n) = 2iT(n/2i) + in When reaching the base case. Closed Form: T(n) = 2iT(n/2i) + in = 2log2 nT(n/2log2 n) + (log2 n)n = nT(n/n) + nlog2 n = nT(1) + nlog2 n T(n) = n + nlog2 n Note: logb x = y ↔ by = x Solving Linear homogeneous recurrence of degree k tn = b1tn-1 + b2tn-2 +. So set 1 equal to the reduction of n in terms of i and solve for i.. T(1) will need to equal the recursive call..b2. Base: T(1) = T(n/2i) 1 = n/2i 2i = n log2 n = i T(1) = T(n/2log2n) = T(n/n) = T(1) Use T(n) in terms of i from and substitute i .. Reduce algebraically until you get a Closed form for T(n).+bktn-k •b1.

a2 = -b2.bktn-k = 0 or a0tn + a1tn-1 + a2tn-2 +. Look for solutions of form tn = xn where x is a constant... tk-1 = Ck-1 Why important? • often occur in determination of run-time of algorithms. Sequence uniquely determined by recurrence relation and k initial conditions t0 = C0.e. • can be systematically solved.. Otherwise..... equation satisfied if and only p(x) = a0xk+a1xk-1+. without guesswork Solving linear homogeneous recurrences The recurrence can be rewritten as tn .+ ak = 0 characteristic equation p(x) is characteristic polynomial p(x) = where ri may be complex numbers.no interest to us... t1= C1. Substituting: a0 xn +a1xn-1 +.b1tn-1 .+ak ) = 0 Satisfied if xn-k=0 . xn-k (a0xk + a1xk-1 +.+akxn-k = 0 i. Degree k : tn is expressed in terms of previous k terms of sequence.... These ri are only solutions of the equation p(x)=0.ak = -bk Note that any linear combination of solutions is itself a solution.b2tn-2 . Homogeneous : no terms that are not multiples of tj ’s..+aktn-k = 0 where a0 =1...a1 = -b1.... Therefore tn = .Linear : sum of constant multiples of the previous terms of the sequence. . ..

fn-1 .ck . All the ri distinct only solutions of this form.x .. Constants can be determined from k initial conditions by solving system of k linear equations in k unknowns.c2.fn2 =0 Characteristic polynomial: x2 .x – 1=0 Roots: r1 =(1√5 )/2 . r2 =(1√5 )/2 General solution: fn = c1r1n c2r2n n=0 : f0 = c1 c2 0 n=1 : f1 = c1r1 c2r2 1 Solving: Thus: fn c1 = 1/√5.1 Characteristic equation: x2 .satisfies the recurrence for any choice of constants c1. c2 =1.. Therefore: tn = 4n − (−1)n . Example Fibonacci numbers: f0 =0 f1 =1 fn = fn-1 fn-2 for n >1 Rewriting: fn . c2=1/√5 1/√5 ( (1+√5) n /2) – (1-√5) n /2 ) Another example t0 = 0 t1 = 5 tn = 3tn−1 + 4tn−2 if n >1 Rewriting: tn − 3tn−1 − 4tn−2 = 0 Characteristic equation: x2 − 3x − 4 = (x +1)(x − 4) Roots: r1 = −1.. General solution: tn = c1(−1)n + c24n Initial conditions: c1 + c2 = 0 when n=0 −c1 + 4c2 = 5 when n=1 Solving: c1 = −1.. r2 = 4.