CS 321-Analysis of Algorithms

Lecture 03

Instructor: Asim Rehan

arehan@numl.edu.pk

National University of Modern Languages, Islamabad

1
Recurrences
 The expression:
 c n 1

T ( n)  
2T  n   cn n  1
  2 
is a recurrence.
 Recurrence: an equation that describes a function in
terms of its value on smaller functions

2
 Recurrence Examples

 0 n0  0 n0
s ( n)   s ( n)  
c  s (n  1) n  0 n  s (n  1) n  0

n 1 
 c  c n 1
 
T ( n)   T ( n)  
2T    c n  1
 n  n
  2  aT    cn n  1
 b
3
Methods to solve recurrence
 Substitution method
 Iteration method
 Recursion tree method
 Master Theorem

4
Substitution
 The substitution method
 the “making a good guess method”
 Guess the form of the answer, then use induction to find
the constants and show that solution works

 Comes with experience

5
Solving Recurrences
 Another option is what the book calls the “iteration
method”
 Expand the recurrence
 Work some algebra to express as a summation
 Evaluate the summation
 We will show several examples

6
  0 n0
s ( n)  
 s(n) = c  s (n  1) n  0
c + s(n-1)
c + c + s(n-2)
2c + s(n-2)
2c + c + s(n-3)
3c + s(n-3)
…
kc + s(n-k) = ck + s(n-k)
7
  0 n0
s ( n)  
c  s (n  1) n  0

 So far for n >= k we have

 s(n) = ck + s(n-k)
 What if k = n?
 s(n) = cn + s(0) = cn

8
  0 n0
s ( n)  
 So far for n >= k we have c  s (n  1) n  0
 s(n) = ck + s(n-k)
 What if k = n?
 s(n) = cn + s(0) = cn
 So
 0 n0
s ( n)  
 Thus in general c  s (n  1) n  0
 s(n) = cn

9
  0 n0
s ( n)  
n  s (n  1) n  0

 s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
10
 s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
 0 n0
= n + n-1 + n-2 + s(n-3) s ( n)  
= n + n-1 + n-2 + n-3 + s(n-4) n  s (n  1) n0
= …
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
=
n

i
i  n  k 1
 s(n  k )

11
 So far for n >= k we have  0 n0
s ( n)  
n  s (n  1) n  0
n

i
i  n  k 1
 s(n  k )

12
 So far for n >= k we have

 0 n0
s ( n)  
n

i
i  n  k 1
 s(n  k ) n  s (n  1) n  0

 What if k = n?

13
 So far for n >= k we have
 0 n0
s ( n)  
n n  s (n  1) n  0
i
i  n  k 1
 s(n  k )

 What if k = n?
n n
n 1
i
i 1
 s ( 0)   i  0  n
i 1 2

14
 So far for n >= k we have
n

i
i  n  k 1
 s(n  k )
 What if k = n?

 Thus in general
n n
n 1

i 1
i  s ( 0)   i  0  n
i 1 2

n 1
s ( n)  n
2
15
Recurrence Relation

16
 Find the Recurrence relation of Factorial and
Fibonacci series.

 if (n==1)
 return 1;
 return n * factorial(n-1);
 T(n) = T(n-1) + k

17
Divide and Conquer
The divide-and-conquer paradigm
Aside: What is a paradigm? One that serves as a pattern or model.
 Divide the problem into a number of subproblems
 Conquer the subproblems by solving them
recursively. If small enough, just solve directly
without recursion
 Combine the solutions of the subproblems into the
solution for the original problem

18
Merge Sort
 Divide the n-element sequence to be sorted in half
(or just about, if n is odd)
 Conquer – sort the two subsequences
 Combine – merge the two sorted subsequences

19
Analyzing Divide-and-Conquer Algorithms
Let T(n) be the running time on a problem of size n.

If problem is small enough, then solution takes

constant time (this is a boundary condition, which will
be assumed for all analyses)

It takes time to divide the problem into sub-problems at

the beginning. Denote this work by D(n).

20
Analyzing Divide-and-Conquer Algorithms

 It takes time to combine the solutions at the end.

Denote this by C(n).

 If we divide the problem into ‘a’ problems, each

of which is ‘1/b’ times the original size (n), and
since the time to solve a problem with input size
‘n/b’ is T(n/b), and this is done ‘a’ times, the total
time is:

T(n) = (1), n small (or n  c )

T(n) = aT(n/b) + D(n) + C(n)

21
Merge Sort

22
Merge

23