Discrete Structures

Topic 21 – Recurrence Relations & Complexity

of Recursive Algorithms (Ch. 8.1)
CMPS 211 – Fall 2016 – American University of Beirut

* Extracted from Discrete Mathematics and It’s Applications book slides

1
 Recurrence Relations

2
Recurrence Relations
} Recall Recursive Definitions:
} BASIS STEP: Specify the value of the function at zero, i.e.,: f(0)(first
element in the domain)
} RECURSIVE STEP: Give a rule for finding f(n) based on the value of
f(i) for i < n

} A recurrence relation for the sequence {an} is an equation

that expresses an in terms of one or more of the previous
terms of the sequence, namely, a0, a1, …, an-1, for all integers
n with n ≥ n0, where n0 is a nonnegative integer
} A sequence is called a solution of a recurrence relation if its terms
satisfy the recurrence relation
} The initial conditions for a sequence specify the terms that
precede the first term where the recurrence relation takes effect

3
Rabbits Riddle
} Example: A young pair of rabbits (one of each gender)
is placed on an island. A pair of rabbits does not breed
until they are 2 months old. After they are 2 months
old, each pair of rabbits produces another pair each
month.

} What is the number of pairs of rabbits on the island

after n months, assuming that rabbits never die?

4
Modeling the Rabbits Population Growth

5
Rabbits Solution
} Solution: Let fn be the the number of pairs of rabbits after n
months
} There are f1 = 1 pairs of rabbits on the island at the end of the
first month
} We also have f2 = 1 because the pair does not breed during the
first month
} To find the number of pairs on the island after n months, add
the number on the island after the previous month, fn-1, and
the number of newborn pairs, which equals fn-2, because each
newborn pair comes from a pair at least two months old
} Consequently the sequence {fn } satisfies the recurrence
relation fn = fn-1 + fn-2 for n ≥ 3 with the initial conditions f1
= 1 and f2 = 1
} The number of pairs of rabbits on the island after n months is
given by the nth Fibonacci number
6
The Tower of Hanoi
} In the late nineteenth century, the French mathematician Édouard
Lucas invented a puzzle consisting of three pegs on a board with disks
of different sizes

} Initially all of the disks are on the first peg in order of size, with the
largest on the bottom

} Rules: You are allowed to move the disks one at a time from one peg to
another as long as a larger disk is never placed on a smaller
} Goal: Using allowable moves, end up with all the disks on the second
peg in order of size with largest on the bottom

7
The Tower of Hanoi (cont.)
} How many moves are needed to move a Tower of Hanoi of
size n from peg 1 to peg 2?
} Solution:
} Let {Hn} denote the number of moves needed
} We need to set up a recurrence relation for the sequence {Hn}
} Begin with n disks on peg 1
} We can transfer the top n −1 disks, following the rules of the puzzle,
to peg 3 using Hn−1 moves
} Now we can directly transfer the largest disk from peg 1 to peg 2 in 1
move
} Finally, we transfer the n −1 disks from peg 3 to peg 2
using Hn−1 additional moves.

} Therefore, the total number of moves needed to transfer

a tower of Hanoi from Peg 1 to Peg 2 is:
Hn = 2Hn−1 + 1

} For a tower with 1 single disk, 1 single move is needed to move

the tower from peg 1 to peg 2, and therefore
H1= 1

8
The Tower of Hanoi (cont.)
} Now we know that moving a Tower of Hanoi of size n requires
Hn = 2Hn−1 + 1
} We need to find a closed form equation for Hn (i.e doesn’t depend on Hn-1)
} To do that, we can use an iterative approach (to solve this recurrence
relation) by repeatedly expressing Hn in terms of the previous terms of
the sequence
Hn = 2Hn−1 + 1
= 2(2Hn−2 + 1) + 1 = 22 Hn−2 +2 + 1
= 22(2Hn−3 + 1) + 2 + 1 = 23 Hn−3 +22 + 2 + 1
⋮
= 2n-1H1 + 2n−2 + 2n−3 + …. + 2 + 1
= 2n−1 + 2n−2 + 2n−3 + …. + 2 + 1 because H1= 1
= 2n − 1 using the formula for the sum of the n i a(r n+1 −1)
terms of a geometric series ∑ ar =
r −1
,r ≠1
i=0

9
The Tower of Hanoi (cont.)
} There was a myth created with this puzzle
} Monks in a tower in Hanoi are transferring 64 gold disks from one peg
to another following the rules of the puzzle
} They move one disk each day
} When the puzzle is finished, the world will end

} Using this formula for the 64 gold disks of the myth,

264 −1 = 18,446, 744,073, 709,551,615
days are needed to solve the puzzle,
which is more than 500 billion years

} Reve’s puzzle (proposed in 1907 by Henry Dudeney) is similar but has

4 pegs
} There is a well-known unsettled conjecture for the minimum number of
moves needed to solve this puzzle

10
Counting Bit Strings
} Find a recurrence relation for the number of bit strings of
length n without two consecutive 0s

} Solution: Let an denote the number of bit strings of length n

without two consecutive 0s
} To obtain a recurrence relation for {an }, note that the number of
bit strings of length n that do not have two consecutive 0s is
} the number of such bit strings ending with a 0
} plus the number of such bit strings ending with a 1

11
Counting Bit Strings
} The bit strings of length n ending with 1 without two consecutive 0s are the bit
strings of length n −1 with no two consecutive 0s with a 1 at the end
Hence, there are an−1 such bit strings

} The bit strings of length n ending with 0 without two consecutive 0s are the bit
strings of length n −2 with no two consecutive 0s with 10 at the end
Hence, there are an−2 such bit strings

} We conclude that an = an−1 + an−2

12
Counting Bit Strings (cont.)
} The initial conditions are:
} a1 = 2, since both the bit strings 0 and 1 do not have consecutive 0s
} a2 = 3, since the bit strings 01, 10, and 11 do not have consecutive 0s, while 00 does
} For n ≥ 3
} an = an−1 + an−2

} How many such bit strings are there of length five?

} Solution: To obtain a5 , we use the recurrence relation three times to find that:
} a3 = a2 + a1 = 3 + 2 = 5
} a4 = a3 + a2 = 5+ 3 = 8
} a5 = a4 + a3 = 8+ 5 = 13

• Note that {an } satisfies the same recurrence relation as the

Fibonacci sequence
• Since a1 = f3 and a2 = f4 , we conclude that an = fn+2

(You can use the Tree Diagram to reach the same results)

13
 Complexity of Recursive Algorithms

14
Complexity of Recursive Algorithms
} Similar to iterative algorithms, we wish to find a way to
compute the time complexity of recursive algorithms
} However, counting the number of operations can be
hard to estimate for recursive algorithms
} To overcome this, express the number of operations as
a recurrence relation, and then solve it to get a closed
form
} Using the iteration method
} Using the Master Theorem

15
 Iteration Method

16
Complexity of Recursive Factorial
} Give a recurrence relation for the complexity of the
recursive factorial method
procedure factorial(n: nonnegative integer)
if n = 0 then
return 1
else
return n∙factorial(n − 1)
{output is n!}

} Solution: We express the number of comparisons for input n as

T(n)
} For the base case, we need 1 comparison
} For the recursive case, we need 1 + T(n-1) comparisons

17
Complexity Recursive Factorial (cont.)
} We need to solve the following recurrence relation using
the iteration method
T(n) = T(n-1) + 1, and T(0) = 1

T(n) = T(n-1) + 1
= T(n-2) + 2
= T(n-3) + 3
⋮
= T(0) + n
= n +1
T(n) is Q(n)

18
Complexity of Recursive Binary Search
} Give a recurrence relation for the complexity of the recursive binary search
procedure binary search(A: a1,a2,…, an , i, j, x : integers, 1≤ i ≤ j ≤n)
m := ⌊(i + j)/2⌋
if i > j then
return 0
else if x = am then
return m
else if x < am then
return binary search(A, i,m−1,x)
else
return binary search(A, m+1,j,x)
{output is location of x in a1, a2,…,an if it appears, 0 otherwise}

} Solution: Consider the worst-case (i.e., element not in the list)

} Express the number of operations for input n as T(n), where n is the size of the array
to be searched
} For the base case, we need 1 comparison
} For the recursive case, we need 3 + T(n/2) comparisons

19
Complexity of Recursive Binary Search
(cont.)
} We need to solve the following recurrence relation using the iteration
method
T(n) = T(n/2) + 3, and T(0) = 1

T(n) = T(n/2) + 3 T(n/2) = 3 + T(n/4)

= 3 + 3 + T(n/4)
T(n/4) = 3 + T(n/8)
= 3 + 3 + 3 + T(n/8)

For simplicity, Assume n = 2k, and therefore k = lgn

T(n) = 3 + 3 + … + 3 + T(0)
k+1 times
= 3 (k + 1) + T(0) = 3 (lgn +1) + 1 = 3 lgn + 4
T(n) is Q(lgn)

20
Complexity of the Recursive Merge Sort
} Give a recurrence relation for the complexity of the recursive merge
sort

procedure mergesort(L = a1, a2,…,an )

if n > 1 then
m := ⌊n/2⌋
L1 := a1, a2,…,am
L2 := am+1, am+2,…,an
L := merge(mergesort(L1), mergesort(L2 ))
{L is now sorted into elements in increasing order}

} Solution: Express the number of operations for input n as T(n), where n

is the size of the array to be sorted
} For the base case, we need 1 comparison
} For the recursive case, we need 1 + 2T(n/2) + f(n) comparisons, where f(n) is
the number of comparisons needed for merging

21
Complexity of the Recursive Merge Sort
(cont.)
} What is f(n), i.e., the number of comparisons needed for merge?

procedure merge(L1, L2 :sorted lists)

L := empty list
while L1 and L2 are both nonempty
remove smaller of first elements of L1 and L2 from its list;
put at the right end of L
if this removal makes one list empty
then remove all elements from the other list and append them to L
return L {L is the merged list with the elements in increasing order}

} Solution:
} In the worst case, we would have to compare every element in L1 with every
element in L2
} L1 and L2 are of size n/2 each
} So f(n) = n -1 in the worst case (last element won’t need to be compared)

22
Complexity of the Recursive Merge Sort
(cont.)
} We need to solve the following recurrence relation using the iteration method
T(n) = 2T(n/2) + n -1 + 1 = 2T(n/2) + n , and T(1) = 1
} Solution:

T(n) = n + 2T(n/2)
= n + 2(n/2 + 2T(n/4)) T(n/2) = n/2 + 2T(n/4)
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
⋮
= in + 2iT(n/2i) Assume: n = 2k à k = lgn
⋮
= kn + 2kT(1)
= nlgn + nT(1)
= nlgn + n
T(n) is Q(nlgn)
23
Common Recurrence Relation Solutions
} T(n) = T(n-1) + 1 T(n) is O(n)

} T(n) = T(n-1) + n T(n) is O(n2)

} T(n) = T(n/2) + c T(n) is O(lgn)

} T(n) = T(n/2) + n T(n) is O(n)

} T(n) = 2T(n/2) + c T(n) is O(n)

} T(n) = 2T(n/2) + n T(n) is O(nlgn)

} T(n) = 2T(n-1) + c T(n) is O(2n)

24 *Credits to Monica Nicolescu, University Of Nevada
Example
} Solve the following recurrence:
T(n) = n + T(n-1)

} Solution:

T(n) = n + T(n-1)
= n + (n-1) + T(n-2)
= n + (n-1) + (n-2) + T(n-3)
= n + (n-1) + (n-2) + … + 2 + T(1)
= n(n+1)/2 - 1 + T(1)
= n2 + T(1)
T(n) is Q(n2)

25
 Master Theorem

26
The Master Theorem
} The Master Theorem is a “cookbook” recipe that can be
used to solve recurrences of the form: 𝑇(𝑛)=𝑎𝑇(𝑛/𝑏)+𝑓(𝑛)
} such that a ≥ 1, b > 1, and f(n) > 0
} where a is the number of recursive calls,
} b is the number of chunks the original input is split into
} f(n) is the total time needed for operations during the recursive
calls

} Notes:
} The Master Theorem gives an estimate of the solution of the
recurrence relation using asymptotic notations (i.e., big-O
notation)
} The Master Theorem assumes that all recursive calls operate on
equal size subsets of the original input
} The Master Theorem has different variations based on the
function f(n)
27
The Master Theorem
} Let T(n) be an increasing function that satisfies the
recurrence relation 𝑇(𝑛)=𝑎𝑇(𝑛/𝑏)+cnd, and such that:
} a≥1
} b>1
} c>0
} d≥0
} Then

28
Complexity of Recursive Binary Search
} Give a recurrence relation for the complexity of the recursive binary search
and solve it using the Master Theorem

procedure binary search(A: a1,a2,…, an , i, j, x : integers, 1≤ i ≤ j ≤n)

m := ⌊(i + j)/2⌋
if i > j then
return 0
else if x = am then
return m
else if x < am then
return binary search(A, i,m−1,x)
else if x > am then
return binary search(A, m+1,j,x)
{output is location of x in a1, a2,…,an if it appears, 0 otherwise}

} Solution:
} We know that the recurrence relation for the complexity of binary search
is: T(n) = T(n/2) + 3
} Using the Master Theorem, we have: a = 1, b = 2, c = 3, d = 0
è1 = 20 è a = bd è T(n) is O(lgn)

29
Complexity of the Recursive Merge Sort
} Give a recurrence relation for the complexity of the recursive
merge sort
procedure mergesort(L = a1, a2,…,an )
if n > 1 then
m := ⌊n/2⌋
L1 := a1, a2,…,am
L2 := am+1, am+2,…,an
L := merge(mergesort(L1), mergesort(L2 ))
{L is now sorted into elements in increasing order}

} Solution:
} We know that the recurrence relation for the complexity of merge sort is:
T(n) = 2T(n/2) + n
} Using the Master Theorem, we have: a = 2, b = 2, c = 1, d = 1
è2 = 21 è a = bd è T(n) is O(nlgn)

30
Example
} Use the Master Theorem to solve the recurrence
T(n) = 2T(n/2) + n2

} a = 2, b = 2, c =1, d = 2
} Compare a with bd
i.e.,: 2 with 22

Þ a < bd Þ Case 1
Þ T(n) is O(n2)

31
Example
} Use the Master Theorem to solve the recurrence
T(n) = 2T(n/2) + sqrt(n)

} a = 2, b = 2, c = 1, d = 0.5
} Compare a with bd
i.e.,: 2 with 20.5 = 1.41

Þ a > bd Þ Case 3
Þ logba = log22 =1

Þ T(n) is O(n)

32
Example
} Use the Master Theorem to solve the recurrence
T(n) = 2T(n/2) + 3n2 - 5n + 6lgn - 4

} a = 2, b = 2, c =3, d = 2
} Compare a with bd
i.e: 2 with 22

Þ a < bd Þ Case 1
Þ T(n) is O(n2)

33
Example
} Use the Master Theorem to solve the recurrence
T(n) = 3T(n/4) + nlgn

Doesn’t Apply since nlgn is

not a polynomial function

34
The recursion-tree method

Convert the recurrence into a tree:

} Each node represents the cost incurred at various
levels of recursion

} Sum up the costs of all levels

Used to “guess” a solution for the recurrence

35
 Example 1
W(n) = 2W(n/2) + n2

} Subproblem size at level i is: n/2i

} Subproblem size hits 1 when 1 = n/2i Þ i = lgn
} Cost of the problem at level i = n2/2i No. of nodes at level i = 2i
} Total cost: lg n 1 2
n lg n 2
lg n 1
1
i
2 1
i
2 1
W ( n) 2 W (1) n n n O ( n) n O ( n)
i 0 2i i 0 2 i 0 2 1 1
2
36 Þ W(n) = O(n2)
Example 2
T(n) = 3T(n/4) + cn2

• Subproblem size at level i is: n/4i

• Cost of a node at level i = c(n/4i)2
• Number of nodes at level i = 3i.
• Cost at level i is 3i c(n2/42i) = c(3/16)in2.
• Subproblem size hits 1 when 1 = n/4i Þ i = log4n
• Last level has 3log4n = nlog43 nodes.
37
Example 2…
T(n) = 3T(n/4) + cn2

• Total cost:
log4 n 1 i i
3 log4 3 3 1
T ( n) cn 2 n cn 2 n log4 3 cn 2 n log4 3 O(n 2 )
16 16 3
i 0 i 0
1
Þ T(n) = O(n2) 16

38
 Any Questions?

39