Module #20 - Recurrences
Recurrence Relations
Module #20 - Recurrences
Recurrence Relations
A recurrence relation (R.R., or just
recurrence) for a sequence {an} is an
equation that expresses an in terms of one or
more previous elements
a0, , an1 of the sequence
sequence, for all nn0.
i.e., just a recursive definition, without the base
cases.
cases
A particular sequence (described nonrecursively) is said to solve the given
recurrence relation if it is consistent with the
definition of the recurrence.
A given recurrence relation may have many
solutions.
Module #20 - Recurrences
Recurrence Relation Example
Consider the recurrence relation
an = 2an11 an22 (n2).
(n2)
Which of the following are solutions?
Y
Yes
an = 3n
an = 2n
No
Yes
an = 5
Module #20 - Recurrences
Example Applications
Recurrence relation for growth of a bank
account with P% interest p
per g
given
period:
Mn = Mn11 + (P/100)Mn11
Growth of a population in which each
organism yields 1 new one every period
starting 2 time periods after its birth.
Pn = Pn1 + Pn2 (Fibonacci
(Fib
i relation)
l ti )
Module #20 - Recurrences
Solving Compound Interest RR
Mn = Mn1 + (P/100)Mn1
= (1 + P/100) Mn11
= r Mn1
(let r = 1 + P/100)
= r (r
( Mn2)
= rr(r Mn3) and so on to
= rn M0
Module #20 - Recurrences
Recurrence relations
Many counting problems can be solved
with recurrence relations
Example: The number of bacteria
d bl
doubles
every 2 hours.
h
If a colony
l
begins with 5 bacteria, how many will be
presentt in
i n hours?
h
?
Let an=2an-1 where n is a positive
integer with a0=5
6
Module #20 - Recurrences
Recurrence relations
A recurrence relation for the sequence
{an} is an equation that expresses an in
terms of 1 or more of the previous terms of
the sequence, i.e., a0, a1, , an-1
n 1, for all
integers n with nn0 where n0 is a
nonnegative integer
A sequence is called a solution of a
recurrence relation if its terms satisfy the
recurrence relation
7
Module #20 - Recurrences
Recursion and recurrence
A recursive algorithm provides the solution of
a problem of size n in terms of the solutions
of one or more instances of the same
problem of smaller size
When we analyze the complexity of a
g
, we obtain a recurrence
recursive algorithm,
relation that expresses the number of
p
required
q
to solve a p
problem of
operations
size n in terms of the number of operations
q
to solve the p
problem for one or more
required
instance of smaller size
8
Module #20 - Recurrences
Example
Let {an} be a sequence that satisfies
the recurrence relation
an = an-1 an-2 for n = 2, 3, 4,
a0= 3 and a1= 5,
what are a2 and a3?
Using the recurrence relation,
a2=a1-a0=5-3=2 and a3=a2-a1=2-5=-3
9
Module #20 - Recurrences
Example
Determine whether the sequence {an},
}
where an=3n for every nonnegative
integer n
n, is a solution of the recurrence
relation
an=2a
2 n-1 an-2 for
f n=2,
2 3
3, 4
4,
Suppose an=3n for every nonnegative
integer n. Then for n
2ann-11-ann-22=2(3(n-1))-3(n-2)=3n=a
( ( )) ( )
n.
n
10
Module #20 - Recurrences
Modeling with recurrence relations
Compound interest: Suppose that a person
d
deposits
it $10
$10,000
000 iin a savings
i
accountt att a
bank yielding 11% per year with interest
compounded annually
annually. How much will it be in
the account after 30 years?
Let Pn denote the amount in the account after
n years. The amount after n years equals the
amount in the amount after n-1 yyears p
plus
interest for the n-th year, we see the
sequence {Pn} has the recurrence relation
Pn=Pn-1+0.11Pn-1=(1.11)Pn-1
11
Module #20 - Recurrences
Modeling with recurrence
relations
The initial condition P0=10,000, thus
P1=(1
(1.11)P
11)P0
P2=(1.11)P1=(1.11)2P0
P3=(1.11)P
(1 11)P2=(1.11)
(1 11)3P0
Pn=(1.11)Pn-1=(1.11)nP0
We can use mathematical induction to
establish its validity
12
Module #20 - Recurrences
Modeling with recurrence relations
We can use mathematical induction to
establish its validity
Assume Pn=(1.11)n10,000. Then from the
recurrence relation
l ti and
d th
the iinduction
d ti
hypothesis
Pn+1=(1.11)Pn=(1.11)(1.11)n10,000
=(1.11)
(
)n+110,000
,
n=30, P30=(1.11)3010,000=228,922.97
13
Tower of Hanoi Example
Module #20 - Recurrences
Problem: Get all disks from peg 1 to peg 2.
Only move 1 disk at a time.
Never set a larger disk on a smaller one.
Peg #1
Peg #2
Peg #3
14
Module #20 - Recurrences
Hanoi Recurrence Relation
Let Hn = # moves for a stack of n disks.
Optimal
O i l strategy:
Move top n1 disks to spare peg. (Hn1
moves)
Move bottom disk. ((1 move))
Move top n1 to bottom disk. (Hn1
moves)
Note:
Hn = 2Hn1 + 1
15
Module #20 - Recurrences
Solving Tower of Hanoi RR
Hn = 2 Hn1 + 1
= 2 (2 Hn2 + 1) + 1
= 22 Hn2 + 2 + 1
= 22(2 Hn3 + 1) + 2 + 1
= 23 Hn3 +
22 + 2 + 1
= 2n1 H1 + 2n2 + + 2 + 1
= 2n1 + 2n2 + + 2 + 1 (since H1 = 1)
n 1
= 2i
i =00
= 2n 1
Module #20 - Recurrences
Solving Tower of Hanoi RR
Hn = 2 Hn1 + 1
17
Module #20 - Recurrences
Finding Recurrence Relation
Find a recurrence relation and give
initial conditions for the number of
bit strings of length n that do not
have two consecutive 0s. How
many such
h bit strings
ti
are th
there off
length 5?
18
Module #20 - Recurrences
Another R.R. Example
Find a R.R. & initial conditions for the number
of bit strings of length n without two
consecutive 0s.
We can solve this by breaking down the
strings to be counted into cases that end in 0
and in 1.
For each endingg in 0,, the previous
p
bit must be 1,, and
before that comes any qualifying string of length n2.
For each string ending in 1, it starts with a qualifying
string of length nn11.
Thus, an = an1 + an2. (Fibonacci
recurrence.))
The initial conditions are: a0 = 1 (), a1 = 2 (0 and 1).
(n2 bits) 1 0
(n1 bits) 1
Module #20 - Recurrences
Codeword Enumeration
Consider a string of decimal digits a valid
codeword if it contains an even number
of 0 digits. For example, 1230407869 is
valid,, whereas 120987045608 is not
valid. Let an be the number of valid ng codewords. Find a recurrence
digit
relation for an .
20
Module #20 - Recurrences
Yet another R.R. example
Give a recurrence (and base cases) for
the number of n-digit decimal strings
containing an even number of 0 digits.
Can break down into the following
cases:
Any valid string of length n1 digits, with any
digit 1-9 appended.
Any invalid string of length n1 digits, + a 0.
an = 9ann11 + (10n1 ann11) = 8ann11 +
10n1.
Base cases: a0 = 1 (), a1 = 9 (1-9).
Module #20 - Recurrences
Catalan Numbers
Find a recurrence relation for Cn , the
number of ways
y to parenthesize
p
the
product of n+1 numbers, x0, x1,, xn, to
p
y the order of multiplication.
p
For
specify
example, C3 = 5.
22
Catalan Numbers
Module #20 - Recurrences
C3 = 5 b
because
there are five ways to parenthesize x0 x1 x2
x3
3 tto determine
d t
i th
the order
d off multiplication:
lti li ti
((x0x1)x2)x3
( 0 ( 1 2)) 3
(x0(x1x2))x3
(x0x1)(x2x3)
x0((x1x2)x3)
x0(x1(x2x3))
23
Catalan Numbers
Module #20 - Recurrences
Solution:
S
l i
one operator remains
i outside
id allll
parentheses.
Thi fi
This
finall operator
t appears b
between
t
ttwo off the
th
n + 1 numbers, say, xk and xk+1 .
There are Ck C nk 1 ways to insert
parentheses
24
Catalan Numbers
Module #20 - Recurrences
Because this
B
hi fi
finall operator can appear b
between
any two of the n + 1 numbers, it follows that
Cn = C0Cn1 + C1Cn2 ++C
Cn2C
2C1 + Cn1C0
n 1
=
C C
0
n - i -1
i =0
Note that the initial conditions are C0 = 1 and C1 =1
25
Module #20 - Recurrences
Solving Recurrences
General Solution Schemas
A linear homogeneous recurrence of
degree
g
k with constant coefficients ((kLiHoReCoCo) is a recurrence of the
form
an = c1an1 + + ckank,
where the ci are all real,, and ck 0.
The solution is uniquely determined if k
initial conditions a0a
ak1
provided
k 1 are provided.
26
Module #20 - Recurrences
Solving Recurrences
linear : Involves sums of p
previous terms
ank
homogeneous : No terms with product of
type ank anl
recurrence of degree k: k terms such that
non zero coefficient of ank
constant coefficients : ck are constants
t t
27
Module #20 - Recurrences
Solving LiHoReCoCos
Basic idea: Look for solutions of the
form an = rn, where r is a constant.
This requires the characteristic
equation:
rn = c1rn1 + + ckrnk, i.e.,
rk c1rk1 ck = 0
The solutions (characteristic roots) can
yield an explicit formula for the
sequence.
28
Module #20 - Recurrences
Solving 2-LiHoReCoCos
Consider an arbitrary 2-LiHoReCoCo:
an = c1ann11 + c2ann22
It has the characteristic equation (C.E.):
r2 c1r c2 = 0
Thm. 1: If this CE has 2 roots r1r2, then
an = 1r1n + 2r2n for n0
for some constants 1, 2.
29
Module #20 - Recurrences
Example
Solve the recurrence an = an1 + 2an2
given the initial conditions a0 = 2,, a1 = 7.
g
Solution:
30
Module #20 - Recurrences
Example
Solve the recurrence an = an1 + 2an2 given
the initial conditions a0 = 2,, a1 = 7.
Solution: Use theorem 1:
We have c1 = 1, c2 = 2
The
Th characteristic
h
t i ti equation
ti is:
i r2 r 2 = 0
Solve it:
(1) (1) 2 4(1)(2) 1 9
r=
2(1)
so, r = 2 or r = 1.
So,, an = 1 2n + 2 ((1))n.
(Using the
quadratic
formula here.)
ax
x =
1 3
= 2 or 1.
2
+ bx
b + c = 0
b
b 2 4 ac
2a
Example Continued
Module #20 - Recurrences
To find 1 and 2, just solve the equations for the
initial conditions a0 and a1:
a0 = 2 = 120 + 2 ((1)
1)0
a1 = 7 = 121 + 2 (1)1
Simplifying,
p y g, we have the p
pair of equations:
q
2 = 1 + 2
7 = 21 2
which we can solve easily by substitution:
2 = 21; 7 = 21 (21) = 31 2;
9 = 31; 1 = 3;; 2 = 1.
Using 1 and 2, our final answer is: an = 32n
(1)n
Check: {an0} = 2, 7, 11, 25, 47, 97
Module #20 - Recurrences
Another Example
p
Solve the recurrence relation
an=7an-1-10an-2 for n2 with a0=2 and
a1=1 byy characteristic equations.
q
Module #20 - Recurrences
Another Example
p
Let an=rn.
The C.E. is r2-7r+10=0. There are two
distinguish
g
root r=2 and r=5.
So, the two basic solution is 2n and 5n,
and assume
ass me an=12n+25n.
From a0=2, we have 1+2=2.
From a1=1, we have 21+52=1.
Thus,
Th we can gett 1=33 andd 2=-1.
1 So,
S
an=32n-5n.
Module #20 - Recurrences
Proof of Theorem 1
Proof that an = 1r1n+2r2n is always a
solution:
We know r12 = c1r1 + c2 and r22 = c1r2 + c2.
Now we can show the proposed sequence satisfies the
recurrence an = c1an1 + c2an2:
c1an1 + c2an2 = c1(1r1n1+2r2n1) + c2(1r1n2+2r2n2)
= 1r1nn22(c1r1+c2) + 2r2nn22(c1r2+c2)
= 1r1n2r12 + 2r2n2r22 = 1r1n + 2r2n = an.
Can complete the proof by showing that for
any initial conditions, we can find
corresponding
s
s.
But it turns out this goes through only if r1r2.
Module #20 - Recurrences
Example
Find an explicit formula for the
Fibonacci numbers.
36
Module #20 - Recurrences
The Case of Degenerate Roots
Now, what if the C.E. r2 c1r c2 = 0
has only
y 1 root r0?
Theorem 2: Then,
an = 1r0n + 2nr0n, for all n0,
n0
for some constants 1, 2.
Module #20 - Recurrences
Degenerate Root Example
Solve an = 6an19an2 with a0=1, a1=6.
The C.E.
C E is: r26r+9=0
6r+9 0.
Note that b24ac = (6)2419 = 3636
= 00.
Therefore, there is only one root,
namely
l
b/2a = (6)/2 = 3.
k-LiHoReCoCos
Module #20 - Recurrences
Consider a k-LiHoReCoCo:
It
Itss C
C.E.
E is: k k
k i
r ci r
=0
an = ci an i
i =1
i =1
Th
Theorem 33: If this
hi has
h k distinct
di i roots ri,
then the solutions to the recurrence are of
the
h fform:
k
an = i ri
i =1
for all n0,
n0 where the i are constants.
constants
Module #20 - Recurrences
Degenerate k-LiHoReCoCos
Suppose there are t roots r1,,rt with
multiplicities m1,,mt. Then:
mi 1
n
j
an = i , j n ri
i =1 j = 0
for all n0,, where all the are constants.
Module #20 - Recurrences
The Case of Degenerate Roots
Now, what if the C.E. r2 c1r c2 = 0
has only
y 1 root r0?
Theorem 2: Then,
an = 1r0n + 2nr0n, for all n0,
n0
for some constants 1, 2.
Ex: an = 6an1 9an 2 , a0 = 1, a1 = 6
41
k-LiHoReCoCos
Module #20 - Recurrences
Consider a k-LiHoReCoCo:
Its
I C.E.
C E iis: k k
k
k i
r ci r = 0 a =
c
a
n
i
n i
i =1
i =1
Thm.3: If this has k distinct roots ri, then
the solutions to the recurrence are of
the form:
k
an = i ri
i =1
f allll n0,
for
0 where
h
th
the i are constants.
t t
42
Module #20 - Recurrences
Example
Ex:
a n = 6a n1 11a n 2 + 6a n3 ,
a0 = 2, a1 = 5, a 2 = 15
43
Module #20 - Recurrences
Degenerate k-LiHoReCoCos
Suppose there are t roots r1,,rt with
multiplicities
p
m1,,,m
, t. Then:
mi 1
n
j
an = i , j n ri
i =1 j = 0
for all n0, where all the are constants.
44
Module #20 - Recurrences
Example
Ex:
a n = 3a n1 3a n 2 a n 3 ,
a0 = 1, a1 = 2, a 2 = 1
45
Module #20 - Recurrences
LiINReCoCos
Linear inhomogeneous RRs with
constant coefficients may
y ((unlike
LiHoReCoCos) contain some terms
( ) that depend
p
only
y on n ((and not on
F(n)
any ais). General form:
an = c1an11 + + ckankk + F(n)
The associated homogeneous recurrence relation
(associated LiHoReCoCo).
Module #20 - Recurrences
Solutions of LiINReCoCos
A useful theorem about LiINReCoCos:
If an = p(n) is any particular solution to the
LiINReCoCo:
k
an = ci an i + F (n)
i =1
Then all of its solutions are of the form:
an = p(
p(n)) + h(n),
( )
where an = h(n) is any solution to the associated
homogeneous RR
k
an = ci an i
i =1
Module #20 - Recurrences
LiINReCoCo Example
Find all solutions to an = 3an1+2n.
Which solution has a1 = 3?
Notice this is a 1-LiINReCoCo. Its associated
1-LiHoReCoCo is an = 3ann11, whose solutions
are all of the form an = 3n. Thus the solutions
to the original problem are all of the form
an = p(n)
( ) + 3n. So, all
ll we needd to do
d is
i find
fi d
one p(n) that works.
Module #20 - Recurrences
Trial Solutions
If the extra terms F(n) are a degree-t
polynomial in n, you should try a general
degree t polynomial as the particular solution
degree-t
p(n).
This case: F(n) is linear so try an = cn + d.
cn+d = 3(c(n1)+d) + 2n
(for all
n)
(2c+2)n + (2d3c) = 0 (collect terms)
So c = 1 and d = 3/2.
So an = n
n 3/2 is a solution
solution.
Check: an1 = {5/2, 7/2, 9/2, }
Module #20 - Recurrences
Finding
g a Desired Solution
From the previous, we know that all
general solutions to our example are of
the form:
an = n 3/2 + 3
3n.
Solve this for for the given case, a1 =
3:
3 = 1 3/2 + 31
= 11/6
The answer is an = n
n 3/2 + (11/6)3n.
Module #20 - Recurrences
Double Check Your Answer!
Check the base case, a1=3:
an = nn 3/2 + (11/6)3n
a1 = 1 3/2 + (11/6)31
= 2/2 3/2 + 11/2 = 5/2 + 11/2 = 6/2 = 3
Check the recurrence, an = 3an1+2n:
nn 3/2 + (11/6)3n = 3[
3[(n1)
(n 1) 3/2 + (11/6)3n1]+2n
= 3[n 1/2 + (11/6)3n1] + 2n
= 3n 3/2 + (11/6)3n + 2n = n 3/2 + (11/6)3n
Module #20 - Recurrences
Divide & Conquer R.R.s
Main points so far:
Many types of problems are solvable by
reducing a problem of size n into some
number a of independent subproblems
subproblems,
each of size n/b, where a1 and b>1.
The
Th titime complexity
l it tto solve
l such
h
problems is given by a recurrence
relation:
l ti
T(n) = aT(n/b) + g(n)
52
Module #20 - Recurrences
Divide+Conquer Examples
Binary search: Break list into 1 subproblem (smaller list) (so a=1) of size
n/2
(so b=2).
So T(n) = T(n/2)+c (g(n)=c constant)
Merge sort: Break list of length n into 2
sublists (a=2), each of size n/2 (so
b 2) then
b=2),
th merge th
them, iin g(n)
( ) = (n)
( )
time.
So
S T(n)
T( ) = 2T(n/2)
2T( /2) + cn (roughly,
(
hl for
f some c))
53
Module #20 - Recurrences
Recurrence relations
Play an important role in many aspects
of algorithms
g
and complexity
p
y
Can be used to
analyze the complexity of divide-and-conquer
divide and conquer
algorithms (e.g., merge sort)
Solve dynamic programming problems (e.g.,
(e g
scheduling tasks, shortest-path, hidden Markov
model))
Fractal
54
