Recursion

Merge Sort. The idea of the Merge Sort is to split a list into two halves, recursively sort
each half, then merge the two sorted sublists.
procedure merge sort(a[1, ....n])
Input: An array of numbers a[1, ..., n]
Output: A sorted version of this array
if n > 1 then


merge merge sort(a[1, ..., bn/2c]), merge sort(a[bn/2c + 1, ..., n])
To merge two ordered lists, we can use the following algorithm for the subroutine merge
used in the merge sort algorithm.
procedure merge(L1 , L2 : sorted lists)
L := empty list
while L1 and L2 are both nonempty
(remove smallest between first elements of L1 and L2 , place it at the right end of L)
if (this removal makes a list empty) then (remove all elements from other list and append
them to L)
return L
(L is the merged list with elements in increasing order).
Example. Merge the two lists 2, 3, 5, 6 and 1, 4.
Solution.
First List Second List Merged List
2356
14
2356
4
1
356
4
12
4
123
56
56
1234
123456
Two sorted lists with m and n elements respectively can be merged into a sorted list using
no more than m + n 1 comparisons.
The number of comparisons in the worst case for an array of n elements in the merge sort
is




T (n) = T bn/2c + T dn/2e + n 1
where n 1 is the maximum number of comparisons for merging. This has complexity
(n log n).
We will see how to solve a version of this under the assumption that n is a power of 2:
(
1
if n = 1
T (n) =
2 T (n/2) + n if n = 2k , k 1

Theorem 1 (The Master Theorem (First Part)). Let a > 1 and b > 1 be constants, and
let f (n) be a function. Suppose that T (n) is defined on the nonnegative integers by the
recurrence
T (n) = a T (n/b) + f (n)
If f (n) is (nlogb a ), then T (n) is (nlogb a log n).
With a = 2, b = 2, and f (n) = n, we have nlogb a = nlog2 2 = n. Since f (n) is (nlogb a ),
T (n) is (n log n) by the Master Theorem.
Example. Find the complexity of T (n) = 4 T (n/2) + n2 .
Solution. a = 4, b = 2, and logb a = log2 4 = 2. Since f (n) = n2 and nlog2 4 = n2 , we know
f (n) is (nlog2 4 ). Thus, T (n) is (n2 log n).
Quick Sort. The idea of Quick Sort is to pick an element in the array to serve as a pivot.
Split the array into two parts: one with elements larger than the pivot, and the other with
elements smaller than the pivot. Recursively repeat the algorithm for both halves of the
original array.
Example. Sort the list 4, 2, 3, 7, 9, 1, 6
Solution. We use the first element of the array as the pivot. Choosing 4 as the pivot, we
obtain the sublists 2, 3, 1 and 7, 9, 6. Using 2 as the pivot for the first sublist gives the sublists
1 and 3, and using 7 as the pivot for the second sublist gives the sublists 6 and 9. Altogether,
we have
1 2 3
4
6 7 9
Suppose that the list is already in the correct order. Then using the first element of a sublist
as the pivot will create a sublist containing all elements but the first one. This means that
the maximum number of comparisons for quick sort on n elements is
(n 1) + (n 2) + + 1 =

n(n 1)
2

Thus, in the worst case, quick sort is (n2 ).

The efficiency of the quick sort algorithm depends heavily on which element is chosen as
the pivot. Randomly choosing a pivot point is recommended if the data to be sorted is not
random. As long as the pivot is chosen randomly, the quick sort has complexity (n log n).
Tower of Hanoi. There are 3 pegs and n disks of increasing radius. The disks are all on a
single peg, with the largest disk on the bottom of the stack. The goal is to move all disks
from the first peg to the third. We can only move one disk at a time. We can never place a
larger disk above a smaller one. An algorithm to solve this problem is the following:
move the n 1 smallest disks to a different peg
move the largest disk
move the n 1 smallest disks onto the largest disk

This lets us move n disks.

Let Tn be the number of moves that will transfer n disks from one peg to another. In
general, the total number of moves needed is
Tn = Tn1 + 1 + Tn1 = 2Tn1 + 1
Since the total number of moves needed to move only 1 peg is T1 = 1, we have
(
2Tn1 + 1 if n > 1
Tn =
1
if n = 1
We can prove that Tn = 2n 1:
Proof. Base case: Let n = 1. Then Tn = 1 = 21 1.
Induction hypothesis: Let n be an arbitrary positive integer, and assume that Tn = 2n 1.
We must show that Tn+1 = 2n+1 1:
Tn+1 = 2Tn + 1
= 2(2n 1) + 1
= 2(2n ) 2 + 1
= 2n+1 1
Therefore, Tn = 2n 1 and Tn is (2n ).

Theorem 2 (Master Theorem). a > 1 and b > 1 are constants, f (n) is a function, T (n) is
defined on the nonnegative integers by the recurrence T (n) = aT (n/b) + f (n).
i) If f (n) is O(nlogb a ) for some constant  > 0, then T (n) is (nlogb a ).
ii) If f (n) is (nlogb a ), then T (n) is (nlogb a log n).
iii) If f (n) is (nlogb a+ ) for some constant  > 0, and if af (n/b)

cf (n) for some
constant c < 1 and all sufficiently large n, then T (n) is f (n) .
Example. Let T (n) = 9T (n/3) + n. Here, f (n) = n, a = 9, and b = 3. Then nlogb a =
nlog2 9 = n2 . Since f (n) = n is O(n2 ), we can apply the first part of the Master Theorem:
T (n) is (n2 ).
Example. Let T (n) = T (2n/3) + 1. Here, f (n) = 1, a = 1, and b = 3/2. Then nlogb a =
nlog3/2 1 = 0. Since f (n) = 1 is (n0 ), we can apply the second part of the Master Theorem:
T (n) is (log n).
Example. Let T (n) = 4T (n/2) + n3 . Here, f (n) = n3 , a = 4, and b = 2. Then nlogb a =
nlog2 4 = n2 . Since f (n) = n3 is (n3+ ) and
4(n/2)3 = (1/2)n3
we can apply the first part of the Master Theorem: T (n) is (n3 ).
Definition (First-Order Recurrences). A first-order recurrence is a relation of the form
(
d
if n = k
T (n) =
cT (n 1) + f (n) if n > k

We can usually solve such recurrences using back substitution.

Example. Let
(
1
if n = 1
T (n) =
2T (n 1) + 1 if n > 1
Then
T (n) = 2T (n 1) + 1


= 2 2T (n 2) + 1 + 1


= 4 2T (n 3) + 1 + 2 + 1


= 8 2T (n 4) + 1 + 4 + 2 + 1
..
.
= 2k T (n k) + 2k1 + 2k2 + 21 + 20
..
.
= 2n1 T (1) + 2n2 + + 21 + 20
n1
X
=
2j
j=0
n

= 2 1
Definition (Second-Order Recurrences). A second-order recurrence is a relation of the form

if n = k
d1
T (n) = d2
if n = k + 1

c T (n 1) + c T (n 2) + f (n) if n > k
1
2
If f (n) = 0, then we say the recurrence is homogeneous.
An example of a second-order recurrence: Fibonacci numbers.
To solve a homogeneous second-order recurrence, we take
T (n) = c1 T (n 1) + c2 T (n 2)
and obtain the equation
T (n) c1 T (n 1) c2 T (n 2) = 0
We then consider the equation
x 2 c1 x c2
called the characteristic equation, and find its roots x1 and x2 . Then,
i) if x1 6= x2 , the general solution is
T (n) = Ax1 n + Bx2 n
ii) if x1 = x2 , the general solution is
T (n) = Ax1 n + Bnx2 n

where A and B are found by using the initial conditions.

Example. Solve the recurrence T (n) = 2T (n 1) + 3(n 2) if n 2, T (0) =, and T (1) =.
Solution. The characteristic equation is x2 2x 3 = (x 3)(x + 1) = 0, so the characteristic
roots are x1 = 3 and x2 = 1. Then T (n) = A(3)n + B(1)n .
T (0) = A + B = 2
T (1) = 3A B = 2
Adding the two previous equations gives 4A = 4, so A = 1. Then B = 1, so T (n) =
3n + (1)n .
Example. Solve the recurrence for the Fibonacci numbers: fn = fn1 + fn2 if n 2,
f0 = 0, and f1 = 1
Solution. The characteristic equation is x2 x 1 = 0, so the characteristic roots are given
by
p

(1) (1)2 4(1)(1)

1 5
x1 , x2 =
=
2(1)
2
 n
 n
Then fn = A 1+2 5 + B 12 5 .
f0 = A + B = 0

1 5
1+ 5
A+
B=1
f1 =
2
2

Multiplying the first of the two previous equations by 1+2 5 , we obtain the new system of
equations
(

1+ 5
1+ 5
A
+
B=0
2
2
1+ 5
1 5
A+ 2 B =1
2
Subtracting the above second equation from the first gives

1 5
1+ 5
B
B = 1
2
2
1
This can be solved to find B =
. Then A = 15 , so we have
5
!n
!n
1
1+ 5
1
1 5
fn =

2
2
5
5
Example. Solve the recurrence T (n) = 4T (n 1) 4T (n 2) if n 2, T (0) = 1, and
T (1) = 0.
Solution. The characteristic equation is x2 4x 4 = (x 2)2 = 0, so the characteristic
roots are x1 = x2 = 2. Then T (n) = A(2n ) + Bn(2n ).
T (0) = A = 1
T (1) = 2A + 2B = 0
We find that B = 1, so T (n) = 2 n2n .
n