CS 316: ALGORITHMS

Lecture 4
Analysis Of Recursive Algorithms
Assoc. Prof. Ensaf Hussein
Dr. Salwa Osama
Department Of Computer Science

09/26/2023
 OUTLINE
Divide and Conquer Technique
• Merge Sort
Analysis of recursive algorithms
• Substitution
• Iteration Method – Iteration Tree
• Master Method
Recursive Algorithms
• Factorial
• Fibonacci Sequence
• Tower of Hanoi
DIVIDE AND CONQUER ALGORITHM
DESIGN STRATEGY
 DIVIDE AND CONQUER
Three steps are involved:
• Divide the problem into several subproblems, perhaps
of equal size
• Subproblems are solved, typically recursively
• The solutions to the subproblems are combined to get
a solution to the original problem
 DIVIDE AND CONQUER (CONTD.)

Problem of size n

Subproblem 1 Subproblem 2

Solution to Solution to
subproblem 1 subproblem 2

Don’t assume
always breaks up Solution to the
into 2, could be > 2 original probelm
subproblems
DIVIDE AND CONQUER (CONTD.)
Example 1: Divide array into two halves,
recursively sort left and right halves, then merge
two halves  Mergesort
Example 2: Partition array into items that are
“small” and items that are “large”, then
recursively sort the two sets  Quicksort
MERGE SORT
MERGESORT

8 2 9 4 5 3 1 6
Sorting Problem: Sort a sequence of n
elements into non-decreasing order.
Divide: Divide the n-element sequence to be
sorted into two subsequences of n/2 elements
each
Conquer: Sort the two subsequences
recursively using merge sort.
Combine: Merge the two sorted
subsequences to produce the sorted answer.
MERGESORT(CONTD.)

Algorithm Merge-Sort(A, p, r):

if p < r then
q¬ (p+r)/2
Merge-Sort(A, p, q)
Merge-Sort(A, q+1, r)
Merge(A, p, q, r)

A: 2 3 8 9 1 4 5 7

P q r
 MERGESORT(CONTD.)
Divide:
Merge:
8 3 2 9 7 1 5 4

q
8 3 2 9 7 1 5 4
q 2 9 7 1 q 5 4
8 3

8 3 2 9 7 1 5 4

3 8 2 9 1 7 4 5

2 3 8 9 1 4 5 7

1 2 3 4 5 7 8 9
 p q r
MERGING IDEA
1 2 3 4 5 6 7 8

2 4 5 7 1 2 3 6

• Copy each of the two sorted sub-sets into two temp sets
• Choose the smaller of the two top elements of the two
temp sets
•Remove it and place it in the output set
• Repeat the process until one set is empty
• Take the remaining and place them onto the output set
A1 A[p, q]
A[p, r]

A2 A[q+1, r]

11
PSEUDOCODE OF THE MERGE PROCEDURE
MERGE (A, p, q, r ) p q r
1 2 3 4 5 6 7 8

//Compute n1 and n2 2 4 5 7 1 2 3 6

1. n1 ← q − p + 1
n1 n2
2. n2 ← r − q
3. Create arrays L[1 . . n1 + 1] and R[1 . . n2 + 1]
4. FOR i ← 1 TO n1
5. DO L[i] ← A[p + i − 1]
6. FOR j ← 1 TO n2 L 2 4 5 7 
7. DO R[j] ← A[q + j ]
8. L[n1 + 1] ← ∞ R 1 2 3 6 
9. R[n2 + 1] ← ∞
...
PSEUDOCODE OF THE MERGE PROCEDURE
MERGE (A, p, q, r ) p q r
1 2 3 4 5 6 7 8

... 2 4 5 7 1 2 3 6

10. i ← 1 k n1 n2
11. j ← 1
12. FOR k ← p TO r
13. DO IF L[i ] ≤ R[ j] i
14. THEN A[k] ← L[i]
15. i←i+1
2 4 5 7 
16. ELSE A[k] ← R[j] L
17. j←j+1
r

R 1 2 3 6 

j
ANALYSIS OF MERGE SORT
ALGORITHM Mergesort(A, p, r )

IF p < r
THEN q = FLOOR[(p + r)/2]

MERGE_Sort (A, p, q) T(n/2)

MERGE_Sort (A, q + 1, r) T(n/2)

MERGE (A, p, q, r) θ(n)

T(n) = 2T(n/2) + θ(n)

 GENERAL PLAN FOR ANALYSIS OF
RECURSIVE ALGORITHMS
 Decide on a parameter indicating an input’s size
 Identify the basic operation
 Check whether the number of times the basic
operation. is executed may vary on different
inputs of the same size. (If it may, the worst,
average, and best cases must be investigated
separately.)
 Set up a recurrence relation with an appropriate
initial condition expressing the number of times
the basic operation is executed.
 Solve the recurrence (or, at the very least,
establish its solution’s order of growth).
RECURRENCES
An equation that describes a function in
terms of its value on smaller inputs.

T(n) = T(n-1) + n

Recurrences arise when an algorithm contains

recursive calls to itself

17
 EXAMPLE RECURRENCES
T(n) = T(n-1) + n Θ(n2)
• Recursive algorithm that loops through the input to
eliminate one item
T(n) = T(n/2) + c Θ(lgn)
• Recursive algorithm that halves the input in one step
T(n) = T(n/2) + n Θ(n)
• Recursive algorithm that halves the input but must
examine every item in the input
T(n) = 2T(n/2) + 1 Θ(n)
• Recursive algorithm that splits the input into 2 halves
and does a constant amount of other work
18
BINARY-SEARCH: RECURRENCE
For an ordered array A, finds if x is in the array A[lo…hi]
BINARY-SEARCH (A, lo, hi, x)
if (lo > hi) 1 2 3 4 5 6 7 8
return FALSE 2 3 5 7 9 10 11 12
mid  (lo+hi)/2
lo mid hi
if x = A[mid]
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x) 19
 BINARY-SEARCH: RECURRENCE
BINARY-SEARCH (A, lo, hi, x)
if (lo > hi)
return FALSE constant time: c1
mid  (lo+hi)/2 constant time: c2
if x = A[mid]
return TRUE constant time: c3
Because of the IF
condition only

if ( x < A[mid] )
one of the two
calls will be
executed

BINARY-SEARCH (A, lo, mid-1, x) same problem

if ( x > A[mid] ) of size n/2
BINARY-SEARCH (A, mid+1, hi, x) same problem
of size n/2
T(n) = c + T( n / 2 ) 20
METHODS FOR SOLVING RECURRENCES
Iteration method

Recursion tree method

Master method

21
1. ITERATION METHOD
The basic idea is to expand the recurrence and express
it as a summation of terms dependent only on n and the
initial conditions.
Example:
Consider the recurrence
T(n) = T(n-1) +1 , and T(1) = θ(1).
Solution:
Expanding the above terms
T(n-1) = T(n-2) + 1 thus,
22
T(n) = (T(n-2)+1) +1 =T(n-2) + 2
T(n-2) = T(n-3) + 1. thus,
T(n) = (T(n-3) +1) + 2 = T(n-3) + 3.
T(n-3) = T(n-4) + 1. thus,
T(n) = T(n-4) +4 .
...
T(n) = T(n-k) + k.
when k =n-1, T(n-k)= T(1)= θ(1).
Thus, T(n)=θ(1)+(n-1)=θ(n)
Hence, T(n)=θ(n)
23
EXAMPLE(2) Binary-search recurrence

T(n) = c + T(n/2) T(n/2) = c + T(n/4)

= c + c + T(n/4) T(n/4) = c + T(n/8)
= c + c + c + T(n/8) = 3c + T(n/23)
...
= k c + T(n/2k) T(1) will be reached
when n = 2k
k = log2n
= c log2n + T(1)
= Θ(lgn)

24
 n+3n/4+32n/42+33n/43+34T(n/44)
EXAMPLE(3) n+3n/4+32n/42+33n/43+……+3kT(n/4k)
T(n/4k) will be T(1) By taking n = 4k
k = log4n
2. THE RECURSION-TREE METHOD
Is a pictorial representation of an iteration method
which is in the form of a tree, where at each level
nodes are expanded.
Idea:
Each node represents the cost of a single
subproblem.
Sum up the costs with each level to get level cost.
Sum up all the level costs to get total cost.
Particularly suitable for divide-and-conquer
recurrence.
26
Best used to generate a good guess.
 RECURSION TREE FOR T(N)=3T(N /3)+(N)

T(n) cn
T(n) = 3T(n/3) + cn
T(n/3) T(n/3) T(n/3) T(n/3) = 3T(n/9)+c(n/3)
T(n/9) =3T(n/27)+c(n/9)

T(n) cn

c(n/3) c(n/3) c(n/3)

T(n/9) T(n/9) T(n/9) T(n/9)T(n/9)T(n/9) T(n/9)T(n/9)T(n/9)

 Cost per level
Level # node cn cn
0 1
c(n/3) c(n/3) 3cn/3
c(n/3)
1 3

c(n/9) c(n/9) c(n/9) c(n/9) c(n/9) c(n/9) c(n/9) c(n/9) c(n/9)

32cn/9
2 3 2

3icn/3
28
i

i 3i

T(1)T(1)T(1) T(1)T(1)T(1) 3h T(1)

h 3h
Total T(n)

28

T(n) = 3hT(1) +
Size of problem at leaves :

n/3h = 1 → 3h = n → h = log3n

T(n) = 3h * T(1) +

T(n) = 3log3n * T(1) +

T(n) = n T(1) + cn. h

T(n) = n T(1) + cn . Log3 n

T(n) = θ(n log n)

29
 RECURSION TREE FOR T(N)=4T(N /2)+(N)

T(n) cn
T(n/2) T(n/2) T(n/2) T(n/2)
T(n) = 4T(n/2) + cn
T(n/2) = 4T(n/4)+c(n/2)
T(n/4) =4T(n/8)+c(n/4)
cn

c(n/2) c(n/2) c(n/2) c(n/2)

T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4) T(n/4)
Level # node cn cn
0 1

1 4
c(n/2) c(n/2) c(n/2) c(n/2) 4cn/2

C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) C(n/4) 42cn/4
2 4 2

i 4i
4icn/2i = 2icn

T(1)T(1)T(1) T(1)T(1)T(1) 4h T(1)

h 4h
Size of problem at leaves :
T(n) = 4h * T(1) +
n/2h = 1 → 2h = n → h = log2n
T(n) = 4log2n * T(1) + n.
4log2n = nlog24 = n2
T(n) = n2 T(1) + n.
T(n) = n2 T(1) + n.
Geometric series s= a+ar+ar2+…+ark-1 = a=1, r = 2
1(2h – 1) / (2-1) = 2h – 1 = 2log2n – 1 = n – 1
T(n) = n2 + n(n-1)
32
T(n) = θ(n2)
REFERENCES

Introduction to Algorithms, Second Edition by Thomas H.

Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford
Stein, The MIT Press © 2001
Chapter 4.3