Unit I (Design and Analysis of Algorithm)

Design and Analysis of Algorithms
Lecture-1
Manoj Mishra (Assistance Professor)

Department of Computer Science and Engineering
United College of Engineering and Research,
Prayagraj
1
Syllabus
Unit-I:
Introduction: Algorithms, Analyzing Algorithms, Complexity of
Algorithms, Growth of Functions, Performance
Measurements, Sorting and Order Statistics - Shell Sort, Quick
Sort, Merge Sort, Heap Sort, Comparison of Sorting
Algorithms, Sorting in Linear Time.
Unit-II:
Advanced Data Structures: Red-Black Trees, B–Trees,
Binomial Heaps, Fibonacci Heaps, Tries, Skip List.
2
Syllabus
Unit-III:
Divide and Conquer with Examples Such as Sorting, Matrix
Multiplication, Convex Hull and Searching. Greedy Methods
with Examples Such as Optimal Reliability Allocation,
Knapsack, Minimum Spanning Trees – Prim’s and Kruskal’s
Algorithms, Single Source Shortest Paths - Dijkstra’s and
Bellman Ford Algorithms.
3
Syllabus
Unit-IV:
Dynamic Programming with Examples Such as Knapsack. All
Pair Shortest Paths – Warshal’s and Floyd’s Algorithms,
Resource Allocation Problem. Backtracking, Branch and
Bound with Examples Such as Travelling Salesman Problem,
Graph Coloring, n-Queen Problem, Hamiltonian Cycles and
Sum of Subsets.
Unit-V:
Selected Topics: Algebraic Computation, Fast Fourier
Transform, String Matching, Theory of NP Completeness,
Approximation Algorithms and Randomized Algorithms
4
Text books
1. Thomas H. Coreman, Charles E. Leiserson and Ronald L.

Rivest, “Introduction to Algorithms”, Printice Hall of India.
2. E. Horowitz & S Sahni, "Fundamentals of Computer

Algorithms",
5
Course Outcome
CO1 Design new algorithms, prove them correct, and analyze
their asymptotic and absolute runtime and memory
demands.
CO2 Find an algorithm to solve the problem (create) and prove
that the algorithm solves the problem correctly (validate).
CO3 Understand the mathematical criterion for deciding
whether an algorithm is efficient, and know many
practically important problems that do not admit any
efficient algorithms.
CO4 Apply classical sorting, searching, optimization and graph
algorithms.
CO5 Understand basic techniques for designing algorithms,
including the techniques of recursion, divide-and-conquer,
6
and greedy.
Definition of Algorithm
An algorithm is any well-defined computational

procedure that takes some value, or set of values, as input
and produces some value, or set of values, as output.
An algorithm is thus a sequence of computational

steps that transform the input into the output.
An algorithm is a set of steps to accomplish or

complete a task that is described precisely enough
that a computer can run it.
7
Characteristics of an algorithm
 Input: Algorithm must contain 0 or more input values.

Output: Algorithm must contain 1 or more output
values.
Finiteness: Algorithm must complete after finite
number of steps.
Definiteness: Each instruction of the algorithm must
be defined precisely or clearly.
Effectiveness: Every instruction must be basic i.e.
simple instruction.
8
Algorithm vs Program
Algorithm Program
•Simple English like language, •Predefined syntax and grammar,
with no predefined syntax. compatible to a programming
•Person writing algorithm must language.
have domain knowledge. •Developed and made by
•Critical usage in Design Phase of programmer
SDLC. •Used in Coding phase of SDLC.
•H/W and OS independent. •H/W and OS dependent.
•Analysis is done to ensure its •Testing is done to check its
efficiency. correctness.
9
Pseudo Code
Pseudo code gives a high-level description of an algorithm
without the ambiguity associated with plain text but also
without the need to know the syntax of a particular
programming language.
It is an artificial and informal language that helps
programmer to develop a program.
It is text based detailed design tool.
Pseudo Code is an intermediatary between an algorithm
and a program.
10
Pseudo Code Vs Algorithm: Example
Algorithm: Insertion Sort:

Input: A list L of integers of length n
Output: A sorted list L1 containing those integers present in L
Step 1: Keep a sorted list L1 which starts off empty
Step 2: Perform Step 3 for each element in the original list L
Step 3: Insert it into the correct position in the sorted list L1.
Step 4: Return the sorted list
Step 5: Stop
11
Pseudo Code Vs Algorithm: Example
Pseudo Code : Insertion Sort
for j=2 to A.length
Key=A[j]
//Insert A[j] into sorted sequence A[1…j-1
i=j-1
while(i>0 && A[i]> key)
A[i+1]=A[i]
i=i-1
A[i+1]=key
12
Pseudo code conventions
We use the following conventions in our pseudo code.

 Indentation is used to indicate a block.
 The looping constructs while, for, and repeat-until
and the if-else conditional construct have
interpretations similar to those in C.
 The symbol “//” indicates that the remainder of the
line is a comment.
 A multiple assignment of the form ij  e assigns to
both variables i and j the value of expression e; it
should be treated as equivalent to the assignment j  e
followed by the assignment i  j .
13
Pseudo code conventions
 We access array elements by specifying the array name

followed by the index in square brackets. For example,
A[i] indicates the ith element of the array A.
 The notation “..” is used to indicate a range of values
within an array. Thus, A[1..j] indicates the sub-array of
A consisting of the j elements A[1], A[2],…………,A[j].
 The Boolean operators “and” and “or” are used.
 A return statement immediately transfers control back
to the point of call in the calling procedure.
14
Design Strategy of Algorithm
 Divide and Conquer

 Dynamic Programming
 Branch and Bound
 Greedy Approach
 Backtracking
 Randomized Algorithm
15
Analysis of algorithms
Analysis of algorithms is the determination of the

amount of time and space resources required to execute it.
Time complexity: T he amount of time required by an

algorithm is called time complexity.
Space complexity: T he amount of space/memory

required by an algorithm is called space complexity.
16
Analysis of algorithms
The main concern of analysis of algorithms is the

required time or performance. Generally, we perform the
following types of analysis −
Worst case − The maximum number of steps taken on

any instance of size n.
Best case − The minimum number of steps taken on any
instance of size n.
Average case − An average number of steps taken on any
instance of size n.
17
Running time of an algorithm
 Running time of an algorithm is the time taken by the

algorithm to execute it successfully.
 The running time of an algorithm on a particular input
is the number of primitive operations or "steps"
executed.
 It is convenient to define the notion of step so that it is
as machine independent as possible.
 It is denoted by T(n), where n is the input size.
18
Insertion Sort
Ex. Sort the following elements by insertion sort:- 4, 3, 2,

10, 12, 1, 5, 6.
19
Insertion Sort Algorithm
Insertion_Sort(A)
1. n length[A]
2. for j2 to n
1. aA[j]
2. //Insert A[j] into the sorted sequence A[1 .. j-1].
3. i j-1
4. While i > 0 and a < A[i]
1. A[i+1]A[i]
2. i i-1
5. A[i+1]  a
20
Insertion Sort Algorithm Analysis
Insertion_Sort(A)
Instruction cost times
n length[A] c1 1
for j2 to n c2 n
aA[j] c3 n-1
//Insert A[j] into the sorted 0 n-1
sequence A[1 .. j-1].
i j-1 c4 n-1
𝑛
∑𝑗=
While i > 0 and a < A[i] c5 2 𝑡𝑗𝑗
c6 𝑛
∑𝑗=
A[i+1]A[i] 2 (𝑡𝑗𝑗-1)
i i-1 c7 𝑛
∑𝑗= (𝑡𝑗𝑗-1)
2
A[i+1]  a c8 n-1
21
Time complexity of Insertion sort
T(n) = c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5.∑𝑗= 𝑛 𝑡

2 𝑗
+ c6.∑𝑛𝑗=2 (𝑡𝑗𝑗-1) + c7 𝑗=2 (𝑡𝑗𝑗-1) + c8.(n-1) ………..(1)
.∑𝑛
Here, there will be three case.
(1) Best case
(2) Worst case
(3) Average case
22
Best case: This case will be occurred when data is already

sorted.
In this case, value of t j will be 1. Put the value of t j in
equation (1) and find T(n). Therefore,
𝑛
T(n) = c1.1 + c2.n + c 3.(n-1) + c 4.(n-1) + c 5.∑ 𝑗= 2 1
+ c6.∑ 𝑗𝑛=2 (1-1) + c7.∑𝑗𝑛=2 (1-1) + c8.(n-1)
= (c2+c3+c4+c5+c8).n + (c1- c3 - c4 - c5 - c8)
= an + b
Clearly T(n) is in linear form, therefore,
T(n) = θ(n)
23
Worst case: This case will be occurred when data is in reverse sorted
order.
In this case, value of tj will be j. Put the value of tj in equation (1) and
find T(n). Therefore,
T(n) = c1.1 + c2.n + c3.(n-1) + c 4.(n-1) + c 5.∑ 𝑗𝑛=2 𝑗
+ c6.∑ 𝑗𝑛=2 (j-1) + c7.∑𝑛𝑗=2 (𝑗𝑗-1) + c .(n-1)
8
= c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. (n+2).(n-1)/2
+ c6.n(n-1)/2+ c7. n(n-1)/2+ c8.(n-1)
= (c5+c6+c7).n2/2+ (c2+c3+c4+(c5 – c6- c7)/2+c8).n +
(c1- c3 - c4 - c5 - c8)
= an2 + bn + c
Clearly T(n) is in quadratic form, therefore, T(n) = θ(n2)
24
Average case: This case will be occurred when data is in any order
except best and worst case.
In this case, value of tj will be j/2. Put the value of tj in equation (1) and
find T(n). Therefore,
T(n) = c1.1 + c2.n + c3.(n-1) + c 4.(n-1) + c 5.∑ 𝑗𝑛=2 (𝑗𝑗/2)
+ c6.∑ 𝑗𝑛=2 (j/2-1) + c7.∑𝑛𝑗=2 (𝑗𝑗/2-1) + c .(n-1)
8
= c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. (n+2).(n-1)/4
+ c6.(n-2)(n-1)/4+ c7. (n-2)(n-1)/4+ c8.(n-1)
= (c5+c6+c7).n2/4+ (c2+c3+c4+(c5 – 3c6- 3c7)/4+c8).n +
(c1- c3 - c4 - c5/2 + c6/2+ c7/2 - c8)
= an2 + bn + c
Clearly T(n) is in quadratic form, therefore, T(n) = θ(n2)
25
Lecture-2

Prayagraj
26
Some other sorting algorithms
 Selection Sort
27
Selection_sort(A)
n ←length[A]
for i←1 to n-1
min ← i
for j←i+1 to n
if(A[j] < A[min])
min ←j
Interchange A[i] ↔ A[min]
Time complexity, T(n) = θ (n2)

28
Bubble Sort
29
Bubble _Sort(A)
30
Divide and Conquer approach
The divide-and-conquer paradigm involves three steps at each level
of the recursion:
Divide the problem into a number of sub-problems that are smaller

instances of the same problem.
Conquer the sub-problems by solving them recursively. If the sub-

problem sizes are small enough, however, just solve the sub
problems in a straightforward manner.
Combine the solutions to the sub-problems into the solution for

the original problem.
31
Analysis of Divide and Conquer based algorithm
When an algorithm contains a recursive call to itself, we

can often describe its running time by a recurrence
equation or recurrence, which describes the overall
running time on a problem of size n in terms of the
running time on smaller inputs. We can then use
mathematical tools to solve the recurrence and provide
bounds on the performance of the algorithm.
32
Analysis of Divide and Conquer based algorithm
A recurrence equation for the running time of a divide-
and-conquer algorithm uses the three steps of the basic
paradigm.
The recurrence equation for the running time of a divide-

and-conquer algorithm is the following:-
T(n) = aT(n/b) + D(n) + C(n), if n > c

= θ(1) , otherwise
Where, n is the size of original problem. a is the number
of sub-problems in which the original problem divided at
an instant. Each sub-problems has size n/b. c is a small
integer.
33
Merge Sort
34
Merge Sort Algorithm
35
Merge Sort Algorithm
36
Merge Sort Algorithm Analysis
The recurrence equation for the running time of

merge sort algorithm will be
T(n) = 2T(n/2) + θ(1)+ θ(n), if n > 1

= θ(1) , otherwise
It can be modified as:-

T(n) = 2T(n/2) + θ(n) , if n > 1
= θ(1) , otherwise
37
When we solve recurrence equation, modify it

as:-
T(n) = 2T(n/2) + cn , if n > 1
= d , otherwise
Here, c and d are some constants.
38
Iterative method:
T(n) = 2T(n/2) + cn
= 2(2T(n/4) + cn/2) + cn
= 2 2 T(n/4) + 2cn
= 2 2 (2T(n/8)+cn/4) + 2cn
= 2 3 T(n/8)+ 3cn
=………………….
= 2 kT(n/2k) + kcn
= nT(1) + c nlog(n) (Let n = 2k )
= dn + cnlog(n) (since T(1) = d)
= θ(nlog(n))
Therefore, T(n) = θ(nlog(n)) 39

Lecture-3

Department of Computer Science and
Engineering United College of Engineering and
Research, Prayagraj
40
Asymptotic Notations
 The notations we use to describe the asymptotic
running time of an algorithm are defined in terms of
functions whose domains are the set of natural
numbers N={ 0, 1, 2, … … … … } .
 We will use asymptotic notations to describe the
running times of algorithms.
 Following notations are used to define the running
time of algorithms.
 notation
2. O-notation
3. Ω-notation
4. o-notation
5. ω-notation
41
notation ( Theta notation )
 For a given function g(n), it is denoted by (g(n)).
 It is defined as following:-
(g(n)) = { f(n) ! ∃positive constants c1, c2 and n 0
such that
0 ≤ c1g(n) ≤f(n) ≤ c2g(n), ∀ n ≥ n0 }
 This notation is
said to be tight
bound.
 If f(n) ∈ (g(n)) then
f(n) = (g(n)) 42
notation ( Big-oh notation )
 For a given function g(n), it is denoted by (g(n)).
(g(n)) = { f(n) ! ∃ positive constants c and n 0 such
that
0 ≤ f(n) ≤ cg(n), ∀ n ≥ n0 }
 This notation is said

to be upper bound.
 If f(n) ∈ (g(n)) then
f(n) = (g(n))
 If f(n) = (g(n)) then
f(n) = (g(n))
43
Ωnotation ( Big-omega notation )
 For a given function g(n), it is denoted by Ω(g(n)).
Ω(g(n)) = { f(n) ! ∃ positive constants c and n 0 such
that
0 ≤ cg(n) ≤f(n), ∀ n ≥ n0 }
 This notation is said

to be lower bound.
 If f(n) ∈ Ω(g(n)) then
f(n) = Ω(g(n))
 If f(n) = (g(n)) then
f(n) = Ω(g(n))
44
 f(n) = (g(n)) iff f(n) = Ω(g(n)) and f(n) = (g(n))
notation ( little-oh notation )
 The asymptotic upper bound provided by O-
notation may or may not be asymptotically tight.
 o-notation denotes an upper bound that is not
asymptotically tight.
 For a given function g(n), it is denoted by (g(n)).
(g(n)) = { f(n) ! for any positive constants c, there
exists a constant n0 such that
0 ≤ f(n) < cg(n), ∀ n ≥n0 }
45
𝜔notation ( little-omega notation )
 The asymptotic lower bound provided by 𝛀-
notation may or may not be asymptotically tight.
 𝜔-notation denotes an upper bound that is not
asymptotically tight.
 For a given function g(n), it is denoted by 𝜔(g(n)).
𝜔 (g(n)) = { f(n) ! for any positive constants c, there
exists a constant n0 such that
0 ≤ cg(n) < f(n), ∀ n ≥n0 }
46
Example: Show that (1/2)n2 - 3n = θ(n2).
Solution: Using definition of θ-notation,
c1g(n) ≤ f(n) ≤c2g(n), ∀ n ≥n0
In this question, f(n) = (1/2)n2 - 3n and g(n) = n2 , therefore
c1 n 2 ≤(1/2)n2 - 3n ≤c2 n 2, ∀ n ≥n0
We divide above by n2, we get
c1 ≤(1/2)- (3/n) ≤c2 , ∀ n ≥n 0 … … … … … … . .(1)
Now, we have to find c1, c2 and n0, such that equation (1) is satisfied.
Consider, left part of (1), c1 ≤(1/2)- (3/n) … … … … . (2)
The value of c1 will be positive value less than or equal to the minimum value
of (1/2)- (3/n). Minimum value of (1/2)- (3/n) = 1/14. Therefore, c1 =
1 /14 . This value of c1 will satisfyequation(2)forn≥7.
Here, c1 = 1 /14 and n ≥7 which satisfy (2). 47
Consider, right part of (1), (1/2)- (3/n) ≤c2 , … … … … . (3)
The value of c2 will be positive value greater than or equal
to the maximum value of (1/2)- (3/n). Maximum value of
(1/2) - (3/n) = 1/2. Therefore, c2 = 1/2 . This value of c2
will satisfy equation (3) for n ≥1.
Here, c2 = 1/ 2 and n ≥1 which satisfy (3).
Therefore, for c1 = 1/14 , c2 = 1/2 and n0 = 7, equation (1)
is satisfied.
Hence by using definition of θ-notation ,
(1/2)n 2 - 3n = θ(n2).
It is proved.
48
Example: Show that 2n+5 = O(n2).
Solution: Using definition of O-notation,
f(n) ≤cg(n) , ∀ n ≥n0
In this question, f(n) = 2n+5 and g(n) = n2 , therefore
2n+5≤cn 2 ∀ n ≥n 0 We divide
above by n2, we get
(2/n)+(5 /n 2) ≤c , ∀ n ≥n0… … … (1)
Now, we have to find c and n0, such that equation (1) is
satisfied.
49
The value of c will be positive value greater than or equal to
the maximum value of (2/n)+(5/n 2 ) .
Maximum value of (2/n)+(5/n 2 ) = 7.
Therefore, c = 7.
Clearly equation (1) is satisfied for c = 7 and n ≥1.
Hence by using definition of O-notation ,
2n+5 = O(n2).
It is proved.
50
Example: Show that 2n 2 +5n+6 = 𝛀(n).
Solution: Using definition of 𝛀 -notation,
cg(n) ≤f(n) , ∀ n ≥n0
In this question, f(n) = 2n2+5n+6 and g(n) = n , therefore
cn ≤2n2+5n+6 , ∀ n ≥n0
We divide above by n, we get
c ≤2n +5 +(6/n) , ∀ n ≥n0 ………(1)
Now, we have to find c and n0, such that equation (1) is
always satisfied.
51
The value of c will be positive value less than or equal to the
minimum value of 2n + 5 + (6/n) .
Minimum value of 2n + 5 + (6/n) = 12.
Therefore, c = 12.
Clearly equation (1) is satisfied for c = 12 and n ≥2.
Hence by using definition of 𝛀 -notation ,
2n 2 +5n+6 = 𝛀 (n).
It is proved.
52
Example: Show that 2n2 = o(n3).
Solution: Using definition of o-notation,
f(n) < cg(n) , ∀ n ≥n 0
Here, f(n) = 2n2, and g(n) = n3. Therefore,
2n2 < cn3 , ∀ n ≥n 0
(2/n) < c , ∀ n ≥n 0 … … … … (1)
for c = 1, there will be n0 = 3, which satisfy (1).
for c = 0.5, there will be n0 = 7, which satisfy (1).
Therefore, for every c, there exists n0 which satisfy (1).
Hence 2n2 = o(n3). 53
Example: Show that 2n2 ≠o(n2).
Solution: Using definition of o-notation,
f(n) < cg(n) , ∀ n ≥n 0
2n2 < cn2 , ∀ n ≥n 0
2 < c, ∀ n ≥n 0 … … … … (1)
Clearly for c = 1, inequality (1) does not satisfy.
Therefore, for every c, there does not exist n0 which satisfy
(1). Hence 2n2 ≠o(n2).
54
Example: Show that 2n2 = 𝜔(n).
Solution: Using definition of 𝜔 -notation,
cg(n) < f(n) , ∀ n ≥n 0
Here, f(n) = 2n2, and g(n) = n. Therefore,
cn < 2n2 , ∀ n ≥n 0
We divide above by n, we get
c < 2n , ∀ n ≥n 0 … … … … (1)
for c = 1, there will be n0 = 1, which satisfy (1).
for c = 10,there will be n0 = 6, which satisfy (1).
Therefore, for every c, there exists n0 which satisfy (1).
Hence 2n2 = 𝜔(n). 55
Example: Show that 2n2 ≠𝜔 (n2).
Solution: Using definition of 𝜔 -notation,
cg(n) < f(n) , ∀ n ≥n 0
cn2 < 2n2 , ∀ n ≥n 0
c < 2, ∀ n ≥n 0 … … … … (1)
Clearly for c = 3, there does not exists n0, which satisfy (1).
Therefore, for every c, there does not exist n 0 which satisfy
(1). Hence 2n2 ≠𝜔 (n2).
56
Lecture-4
Manoj Mishra(Assistance Professor)

Prayagraj
57
Example: Show that using definition of notations
(a) 3n3-10n+50 = θ(n3)
(b) 5n2-100n ≠ θ(n3)
(c) 3n3-10n+50 = O(n3)
(d) 5n2-100n ≠ O(n)
(e) 3n3-10n+50 = 𝛀(n3)
(f) 5n2-100n ≠ 𝛀(n3)
58
Limit based method to compute notations for a function
𝑓 𝑛
First compute lim = c.
𝑛→∞ 𝑔 𝑛
(1) If c is a constant such that 0< c < ∞, then f(n) = θ(g(n)).
(2) If c is a constant such that 0 ≤ c < ∞, then f(n) = O(g(n)).
(3) If c is a constant such that 0< c ≤ ∞, then f(n) = 𝛀(g(n)).
(4) If c is a constant such that c = 0, then f(n) = o(g(n)).
(5) If c is a constant such that c = ∞, then f(n) = 𝜔(g(n)).
59
Exercises
1. Let f(n) and g(n) be asymptotically non-negative

functions. Using the basic definition of ‚θ-notation,
prove that max(f(n), g(n)) = θ(f(n)+g(n)).
2. Show that for any real constants a and b, where b > 0,

(n+a)b = θ(nb)
3. Solve the followings:-

(a) Is 2n+1 = O(2n) ?
(b) Is 22n = O(2n) ? 60
Exercise(cont.)
7.Arrange the following in ascending order of growth
or rank the following functions by order of growth.
n3, (3/2)n, 2n, n2, log(n), 22n, loglog(n), n!, en.
8.Let f(n) and g(n) be two asymptotically positive
functions. Prove or disprove the following:-
(a) f(n) = O(g(n)) implies g(n) = O(f(n)).
(b) f(n) + g(n) = θ(min(f(n),g(n))).
(c) f(n) = O(g(n)) implies lg(f(n)) = O(lg(g(n))), where
lg(g(n)) ≥1 and f(n) ≥ 1 for all sufficiently large n.
(d)f(n) = O(g(n)) implies 2f(n) = O(2g(n)).
(e) f(n) = O(((f(n))2)
61
AKTU questions
1. Take the following list of functions and arrange them in
ascending order of growth rate. That is, if function g(n)
immediately follows function f(n) in your list, then it
should be the case that f(n) is O(g(n)). f1(n) = n2.5, f2(n) =
√2n, f3(n) = n + 10, f4(n) = 10n, f5(n) = 100n, and
f6(n) = n2 log n
2. Rank the following by growth rate: n, 2lg √n, log n, log

(logn), log2n, (lgn)lgn, 4, (3/2)n, n!
62
Lecture-5
Recurrence Relation

Prayagraj
63
Recurrence Relation
•A Recurrence is an equation or inequality that

describes a function in terms of its value on smaller
input.
•The recurrence describes the running time of an
algorithm that divides a problem of size n into a
number of sub-problems.
64
65
66
Recurrence relation
Recurrence equations will be of the following form:-
(1) T(n) = aT(n/b) + f(n)

(2) T(n) = T(n-1) + n
(3) T(n) = T(n/3) + T(2n/3) + n
(4) T(n) = T(n-1) + T(n-2)
Some approaches to sole recurrence relations

(1) Iterative method
(2) Substitution method
(3) Recurrence Tree
(4) Master theorem method
67
Substitution method
The substitution methodfor solving recurrences comprises

two steps:
1. Guess the form of the solution.
2.Use mathematical induction to find the constants and show

that the solution works.
68
Back Substitution Method
Example: Solve the given recurrence

relation using substitution method:
T(n) = 1, n=0
T(n – 1) + 1, n > 0
69
Solution
70
Example
Solve the given recurrence relation

using substitution method:
T(n) = 1, n=0
T(n – 1) + n, n > 0
71
Solution
72
Example
Solve the given recurrence relation

using substitution method:
T(n) = 1, n=1
T(n/2) + n, n > 1
73
Solution
74
Substitution method
Example: Find the upper bound of following recurrence

relation T(n) = 2T(⌊n/2⌋) + n. ………..(1)
Solution: (Do yourself)
75
Substitution method
Example: Find the upper bound of following recurrence
relation T(n) = 2T(⌊n/2⌋ + 17) + n
Solution: ( Do yourself)
76
Substitution method
Example: Solve the following recurrence relation
T(n) = 2T(⌊√n⌋ ) + lg n
Solution: ( Do yourself)
77
Substitution method
Example: Solve the following recurrence relation T(n) = T(⌊n/2⌋ )
+ T(⌈ n/2 ⌉ ) + 1
Solution: (Do yourself)
78
Lecture-6
Recurrence Relation

Prayagraj
79
Recurrence tree method
In a recursion tree, each node represents the cost of a
single sub-problem somewhere in the set of recursive
function invocations. We sum the costs within each level of
the tree to obtain a set of per-level costs, and then we sum
all the per-level costs to determine the total cost of all
levels of the recursion.
A recursion tree is best used to generate a good guess,

which you can then verify by the substitution method.
80
Example
Solve the given recurrence relation using

recurrence tree method:
T(n) = 1, n=0
T(n – 1) + n, n > 0
81
Solution
82
Example
Solve the given recurrence relation using

recurrence tree method:
T(n) = 1, n=1
T(n/2) + n, n > 1
83
Solution
84
Example: Solve the following recurrence equation
T(n) = 3T(⌊n/4⌋) + θ(n2)
Solution: (Do Yourself)
85
Example: Solve the following recurrence relation using recurrence
tree method
T(n) = T(n/3) + T(2n/3) + θ(n)
Solution: (Do Yourself)
86
Exercise
(1)Draw the recursion tree for T(n) = 4 T(⌊n/2⌋) + cn, where
c is a constant and provide a tight asymptotic bound on its
solution. Verify your bound by the substitution method.
(2)Use a recursion tree to give an asymptotically tight
solution to the recurrence T(n) = T(n-a) + T(a) + cn, where a
≥1 and c > 0 are constants.
(3)Use a recursion tree to give an asymptotically tight
solution to the recurrence T(n) = T(αn) + T((1- α)n) + cn,
where α is a constant in the range 0 < α < 1 and c > 0 is also
a constant.
87
Lecture-7
Recurrence Relation

Research, Prayagraj
88
Master Theorem
Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and let
T(n) be defined on the nonnegative integers by the
recurrence
T(n) = aT(n/b) + f(n);
Where we interpret n/b to mean either ⌊n/b⌋ or ⌈n/b⌉. Then
T(n) has the following asymptotic bounds:
1. If f(n) = 𝑂(nlogba − ϵ) for some constant ϵ> 0, then
T(n) = θ(nlogba).
2. If f(n) =θ(nlog ba) , then T(n) = θ(nlog balg n).
3. If f(n) = 𝛺(nlogba + ϵ) for some constant ϵ> 0, and if
af(n/b) ≤ cf(n) for some constant c < 1 and all sufficiently
large n, then T(n) = θ(f(n).
89
Solution Steps using Master theorem
•General form of recurrence relation is T(n) = =aT(n/b) + f(n)
where a>= 1, b > 1 and f(n) = Θ(nk logpn)
From above we find two values as:
(1) logba (2) k
Based on above two values there are three cases to solve recurrence relation:
Case I
If logba > k then solution is Θ(nlogba )
Case II
If logba = k,
(i) if p > -1 then solution will be Θ(nk logp+1 n)
(ii) if p = -1 then Θ(nk log logn)
(iii) ifp < -1 then Θ (nk )
Case III
if logba < k,
(i) p>= 0 then solution is Θ(nk logp n)
(ii) p < 0 then solution is Θ (nk )
90
Example
1- Solve the following recurrence relation :

(i)T(n) = 2T(n/2) + 1
(ii)T(n) = 4T(n/2) + n
(iii)T(n) 8T(n/2) + n
(iv) T(n) = 9T(n/3) +1
2- Solve the following recurrence relation:
(i)T(n) = 2T(n/2) + n
(ii)T(n) = 2T(n/2) + n/logn
(iii) T(n) = 2T(n/2) + n/ log2 n
3- Solve the following recurrence relation:
T(n) = T(n/2) + n2
91
Master Theorem Method
Example: Solve the following recurrence relations using
master theorem method
(a) T(n) = 9T(n/3) + n
(b) T(n) = T(2n/3) + 1
(c) T(n) = 3T(n/4) + n log n
Solution:
(a) Consider T(n) = 9T(n/3) + n
In this recurrence relation, a = 9, b= 3 and f(n) = n.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog 39 = n2
Clearly, n𝑙𝑜𝑔 𝑏𝑎 > f(n) , therefore case1 can be applied.
Now determine ϵ such that f(n) = 𝑂(n2−ϵ) . Here ϵ = 1.
Therefore case 1 will be applied.
Hence solution will be T(n) = θ(n2).
92
Solution:
(b) Consider T(n) = T(2n/3) + 1
In this recurrence relation, a = 1, b= 3/2 and f(n) = 1.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog3/2 1 = 0
Clearly, f(n) =θ(nlog ba) , therefore case 2 will be applied.
Hence solution will be T(n) = θ(log n).
93
Solution:
(c) Consider T(n) = 3T(n/4) + n log n
In this recurrence relation, a = 3, b= 4 and f(n) = n log n.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog 43 = n0.793
Clearly, n𝑙𝑜𝑔 𝑏𝑎 < f(n) , therefore case 3 can be applied.
Now determine ϵ such that f(n) =𝛺(n0.793+ϵ) . Here ϵ = 0.207.
Now, af(n/b) ≤ cf(n) imply that 3f(n/4) ≤ cf(n)
⇒ 3(n/4)log (n/4) ≤ c n log n
⇒ (¾)log(n/4) ≤ c log n
Clearly above inequality is satisfied for c = 3/ 4. Therefore
case 3 will be applied.
Hence solution will be T(n) = θ(n log n).
94
Master theorem method
Example: Solve the following recurrence relation
T(n) = 2T(n/2) + n log n
Solution: Here, a = 2 , b=2 and f(n) = n log n.
nlogba = nlog 22 = n
If we compare nlog a and f(n), we get f(n) is greater than nlog a
b b
. Therefore, case 3 may be applied.
Now we have to determine ϵ >0 which satisfy f(n) = 𝛺
(nlogba+ϵ), i.e. n logn = 𝛺(n1+ϵ). Clearly there does not exist
any ϵ which satisfy this condition. Therefore case 3 can not
be applied. Other two cases are also not satisfied. Therefore
Master theorem can not be applied in this recurrence
relation. 95
Generalized Master theorem
Theorem: If f(n) = θ(n log ba lgkn), where k ≥ 0, then the
solution of recurrence will be T(n) = θ(nlogba lgk+1n).
Now, consider the previous example:- T(n) =

2T(n/2) + n log n
Solve it using aboe theorem,
Here a =2, b= 2, and k = 1. T herefore, the solution of this
recurrence will be
T(n) = θ(nlog 2 lg1+1n)
2
= θ(n lg2n)
Hence, T(n) = θ(n lg2n)
96
Recurrence relation
Exercise
1.Use the master method to give tight asymptotic bounds for
the following recurrences:-
(a) T(n) = 8T(n/2) + θ(n2)
(b) T(n) = 7T(n/2) + θ(n2)
(c) T(n) = 2T(n/4) + 1
(d) T(n) = 2T(n/4) + √n
2.Can the master method be applied to the recurrence
4T(n/2) + n2 log n ? Why or why not? Give an asymptotic
upper bound for this recurrence.
97
Lecture-8

Research, Prayagraj
98
Recurrence relation
3. Give asymptotic upper and lower bounds for T(n) in each
of the following recurrences. Assume that T(n) is constant
for n ≤2. Make your bounds as tight as possible, and justify
your answers.
(a) T(n) = 2T(n/2) + n4
(b) T(n) = T(7n/10) + n
(c) T(n) = 16T(n/4) + n2
(d) T(n) = 2T(n/4) + √n
(e) T(n) = T(n-2) + n2
(f) T(n) = 7T(n/3) + n2
(g) T(n) = 3T(n/3 -2) + n/2
99
Recurrence relation
4. Give asymptotic upper and lower bounds for T(n) in each of the
following recurrences. Assume that T(n) is constant for sufficiently small
n. Make your bounds as tight as possible, and justify your answers.
(a) T(n) = 4T(n/3) + n lgn
(b) T(n) = 3T(n/3) + n/lgn
(c) T(n) = 2T(n/2) + n/lgn
(d) T(n) = T(n/2) + T(n/4) + T(n/8) + n
(e) T(n) = T(n-1) + 1/n
(f) T(n) = T(n-1) + lg n
(g) T(n) = T(n-2) + 1/lg n
(h) T(n) = √n T(√n) + n
100
Lecture-9
Recurrence Relation

Prayagraj
101
AKTU Examination Questions
1. Solve the recurrence T (n) = 2T(n/2) + n 2 + 2n+ 1
2.Solve the recurrence using recursion tree method:
T (n) = T (n/2) + T (n/4) + T (n/8) + n
3. Use a recursion tree to give an asymptotically tight solution
to the recurrence T (n) = T (αn) + T ((1 - α)n) + cn, where α is a
constant in the range 0 <α< 1 and c > 0 is also a constant.
4.The recurrence T (n) = 7T (n/3) + n2 describes the running
time of an algorithm A. Another competing algorithm B has a
running time of S (n) = a S (n/ 9) + n2. What is the smallest
value of ‘a’ such that A is asymptotically faster than B?
5. Solve the recurrence relation by substitution method
T(n)= 2T(n/2) + n
102
6. Show that the solution to T (n) = 2T (⌊n/2⌋ + 17) + n is
O(nlgn).
7. Solve the recurrence: T (n) = 50 T (n/49) + log n!
8.Solve the following recurrence using Master method:
T (n) = 4T (n/3) + n2
9.Find the time complexity of the recurrence relation
T(n) = n +T(n/10)+T(7n/10)
10.Solve the following By Recursion Tree Method
T(n) = n + T(n/5)+T(4n/5)
11.The recurrence T (n) =7T (n/2) +n 2 describe the running
time of an algorithm A. A competing algorithm A has a running
time of T’ (n) =aT’ (n/4) +n2. What is the largest integer value
for a A’ is asymptotically faster than A? 103
Lecture-10
Manoj Mishra(Assistance Professor)

Research, Prayagraj
104
Merge sort
Merge sort is defined as a sorting algorithm that

works by dividing an array into smaller subarrays,
sorting each subarray, and then merging the sorted
subarrays back together to form the final sorted
array.
•we can say that the process of merge sort is to divide
the array into two halves, sort each half, and then
merge the sorted halves back together. This process is
repeated until the entire array is sorted.
105
How does Merge Sort work?
Merge sort is a recursive algorithm that continuously splits the

array in half until it cannot be further divided i.e., the array has
only one element left (an array with one element is always
sorted). Then the sorted subarrays are merged into one sorted
array.
106
Example
Lets consider an array arr[] = {38, 27, 43, 10}

Initially divide the array into two equal halves:
Solution:
107
Solution Contd..
These subarrays are further divided into two halves. Now they
become array of unit length that can no longer be divided and
array of unit length are always sorted.
108
Solution Contd..
These sorted subarrays are merged together, and we
get bigger sorted subarrays.
109
Solution Contd..
This merging process is continued until the sorted array is
built from the smaller subarrays.
110
Algorithm: Merge Sort
Algorithm MergeSort(l,h)
{
If l < h
{
mid = (l + h) /2 ;
MergeSort(l,mid);
MergeSort(mid + 1, h);
Merge(l,mid,h);
}
}
111
Complexity Analysis of Merge Sort:
Time Complexity: O(N log(N)), Merge Sort is a recursive

algorithm and time complexity can be expressed as following
recurrence relation.
T(n) = 2T(n/2) + θ(n)
The above recurrence can be solved either using the
Recurrence Tree method or the Master method. It falls in case
II of the Master Method and the solution of the recurrence is
θ(Nlog(N)). The time complexity of Merge Sort is θ(Nlog(N))
in all 3 cases (worst, average, and best) as merge sort always
divides the array into two halves and takes linear time to
merge two halves.
Auxiliary Space: O(N), In merge sort all elements are copied
into an auxiliary array.
112
Applications of Merge Sort:
Sorting large datasets: Merge sort is particularly well-suited for
sorting large datasets due to its guaranteed worst-case time
complexity of O(n log n).
External sorting: Merge sort is commonly used in external
sorting, where the data to be sorted is too large to fit into
memory.
Custom sorting: Merge sort can be adapted to handle different
input distributions, such as partially sorted, nearly sorted, or
completely unsorted data.
113
Advantage s and Disadvantages of Merge Sort
Advantages of Merge Sort:
Stability: Merge sort is a stable sorting algorithm, which means it maintains
the relative order of equal elements in the input array.
Guaranteed worst-case performance: Merge sort has a worst-case time
complexity of O(N logN), which means it performs well even on large
datasets.
Parallelizable: Merge sort is a naturally parallelizable algorithm, which
means it can be easily parallelized to take advantage of multiple processors
or threads.
Drawbacks of Merge Sort:
Space complexity: Merge sort requires additional memory to store the
merged sub-arrays during the sorting process.
Not in-place: Merge sort is not an in-place sorting algorithm, which means it
requires additional memory to store the sorted data. This can be a
disadvantage in applications where memory usage is a concern.
Not always optimal for small datasets: For small datasets, Merge sort has a
higher time complexity than some other sorting algorithms, such as
insertion sort. This can result in slower performance for very small
114
datasets.
Heapsort
Heap
 The (binary) heap data structure is an array object

that we can view as a nearly complete binary tree.
 Each node of the tree corresponds to an element

of the array. The tree is completely filled on all
levels except possibly the lowest, which is filled
from the left up to a point.
115
Heapsort
Max-heap Array
116
Heapsort
Index: If i the index of a node, then the index of parent and its
child are the following:-
Parent(i) = ⌊i/2⌋
Left(i) = 2i
Right(i) = 2i+1
Note: Root node has always index 1 i.e. A[1] is root element.
Heap-size: Heap-size is equal to the number of elements in the heap.
Height of a node: The height of a node in a heap is the
number of edges on the longest simple downward path from the
node to a leaf.
Height of heap: The height of the heap is equal to the height of its
root.
117
Heapsort
Types of heap
There are two kinds of binary heaps:
(1) max-heaps (2) min-heaps
Max-heap: The heap is said to be max-heap if it satisfy the
max-heap property.
The max-heap property is that the value at the parent node
is always greater than or equal to value at its children.
Min-heap: The heap is said to be min-heap if it satisfy the
min-heap property.
The min-heap property is that the value at the parent node
is always less than or equal to value at its children.
118
Heapsort
Heap sort algorithm consists of the following two sub-
algorithms.
(1)Max-Heapify: It is used to maintain the max-heap
property.
(2)Build-Max-Heap: It is used to construct a max-heap for
the given set of elements.
119
Heapsort
Max-Heapify Algorithm
Action done by max-heapify algorithm is shown in the following
figures:-
120
Heapsort
Max-Heapify Algorithm
121
Heapsort
Time complexity of Max-Heapify Algorithm
The running time of max-heapify is determined by the
following recurrence relation:-
T(n) ≤T(2n/3)+ θ(1)
Here n is the size of the sub-tree rooted at node i.
Using master theorem, the solution of this recurrence
relation is
T(n) = θ(lg n)
122
Lecture-11

Prayagraj
123
Heapsort
Build-Max-Heap Algorithm
Example: Construct max-heap corresponding to the
following elements
4, 1, 3, 2, 16, 9, 10, 14, 8, 7.
Solution:
124
Heapsort
Build-Max-Heap Algorithm (cont.)
125
Heapsort
Build-Max-Heap Algorithm (cont.)
The running time of this algorithm is

T(n) = O(n lg n)
But this upper bound is not asymptotically tight.
Now, we shall calculate tight upper bound.
126
Heapsort
Time complexity of Build-Max-Heap Algorithm
We can derive a tighter bound by observing that the time for MAX-
HEAPIFY to run at a node varies with the height of the node in the tree,
and the heights of most nodes are small.
Our tighter analysis relies on the properties that an n-element heap has
height ⌊lg n⌋ and at most ⌈n/2h+1⌉ nodes of any height h.
If h is the height of the sub-tree then running time of max-

heapify is O(h).
Therefore, total cost of build-max-heap is
T(n) =
127
Heapsort
Time complexity of Build-Max-Heap Algorithm
Now, T(n) =
=
Since
Therefore,
T(n) = O( 2n)
= O(n)
128
Heapsort Algorithm
Example: Sort the following elements using heapsort
5, 13, 2, 25, 7, 17, 20, 8, 4.
Solution: ( Do Yourself)
129
Heapsort Algorithm
Time complexity: The HEAPSORT procedure takes time

O(nlgn), since the call to BUILD-MAX-HEAP takes time
O(n) and each of the n-1 calls to MAX-HEAPIFY takes time
O(lgn).
130
Heapsort Algorithm
Exercise
1. What are the minimum and maximum numbers of
elements in a heap of height h?
2. Show that an n-element heap has height ⌊lgn⌋.
3. Is the array with values 23, 17, 14, 6, 13, 10, 1, 5, 7, 12 a
max-heap?
4. Show that there are at most ⌈n/2h+1⌉nodes of height h in
any n-element heap.
5. What is the running time of HEAPSORT on an array A of
length n that is already sorted in increasing order? What
about decreasing order?
131
Heapsort Algorithm
1. Sort the following array using Heap-Sort techniques—

5,8,3,9,2,10,1,35,22
2.How will you sort following array A of elements using
heap sort: A = (23, 9, 18, 45, 5, 9, 1, 17, 6).
132
Lecture-12

Prayagraj
133
Quicksort
Quicksort, like merge sort, applies the divide-and-

conquer paradigm.
Quicksort uses three steps to sort the elements.
 Divide
 Conquer
 Combine
134
Quicksort
Algorithm
It divides the large array into smaller sub-arrays. And then quicksort
recursively sort the sub-arrays.
Pivot
1. Picks an element called the "pivot".
Partition
2.Rearrange the array elements in such a way that the all values lesser
than the pivot should come before the pivot and all the values greater
than the pivot should come after it.
This method is called partitioning the array. At the end of the partition
function, the pivot element will be placed at its sorted position.
Recursive
3.Do the above process recursively to all the sub-arrays and sort the
elements.
135
Quicksort
Example: Sort the following elements using quicksort
2, 8, 7, 1, 3, 5, 6, 4
Solution: Here, we pick the last element in the list as a pivot
1 2 3 4 5 6 7 8
2 8 7 1 3 5 6 4
2 8 7 1 3 5 6 4
2 8 7 1 3 5 6 4
2 8 7 1 3 5 6 4
2 1 7 8 3 5 6 4
2 1 3 8 4 5 6 4
136
Quicksort
1 2 3 4 5 6 7 8
2 1 3 8 7 5 6 4
2 1 3 4 7 5 6 8
Partition completed in first pass
2 1 3 4 7 5 6 8
2 1 3 4 7 5 6 8
2 1 3 4 7 5 6 8
2 1 3 4 7 5 6 8
2 1 3 4 7 5 6 8
137
Partition completed in second pass
Quicksort
1 2 3 4 5 6 7 8
2 1 3 4 7 5 6 8
2 1 3 4 7 5 6 8
1 2 3 4 5 7 6 8
1 2 3 4 5 6 7 8
Partition completed in third pass
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
Process
138
completed
1 2 3 4 5 6 7 8
Quicksort Algorithm
Quicksort(A, p, r)
1 if p < r
2 q = PARTITION(A, p, r)
3 QUICKSORT(A, p, q-1)
4 QUICKSORT(A, q+1, r)
To sort an entire array A, the initial call is QUICKSORT(A, 1,

length[A]).
139
Quicksort Algorithm
140
Quicksort Algorithm Analysis
Time Complexity:
The running time of quicksort algorithm is

T(n) = T(n1) + T(n2) + θ(n) ……………….(1)
Here n1 and n2 are the size of sub-problems.
The running time of quicksort depends on whether the

partitioning is balanced or unbalanced, which in turn depends
on which elements are used for partitioning.
If the partitioning is balanced, the algorithm runs
asymptotically as fast as merge sort.
If the partitioning is unbalanced, however, it can run
asymptotically as slowly as insertion sort.
141
Worst-case partitioning
The worst-case behavior for quicksort occurs when the
partitioning routine produces one sub-problem with n-1
elements and one with 0 elements.
Let us assume that this unbalanced partitioning arises in

each recursive call. Therefore,
T(n) = T(0) + T(n-1) + θ(n)

T(n) = T(n-1) + θ(n) (since T(0) = θ(1) )
142
Worst-case partitioning
After solving this recurrence relation, we get
T(n) = θ(n2)
Therefore the worst-case running time of quicksort is no better

than that of insertion sort.
Moreover, the θ(n2) running time occurs when the input array
is already completely sorted—a common situation in which
insertion sort runs in O(n) time.
143
Best-case partitioning
If PARTITION produces two sub-problems, each of size no
more than n / 2, since one is of size ⌊n /2⌋ and one of size
⌈n/2⌉-1 . In this
case, quicksort runs much faster. The recurrence for the
running time is then
T(n) = 2T(n/2) + θ(n)
After solving this recurrence relation, we get
T(n) = θ(n lgn)
Note: By equally balancing the two sides of the partition at

every level of the recursion, we get an asymptotically faster
algorithm.
144
Balanced partitioning
The average-case running time of quicksort is much closer to
the best case than to the worst case.
Suppose that the partitioning algorithm always produces a 9-
to-1 proportional split. In this case, running time will be
T(n) = T(9n/10) + T(n/10) + θ(n)
This relation is equivalent to the following
T(n) = T(9n/10) + T(n/10) + cn
We can solve this recurrence using recurrence tree method.
145
Recurrence tree will be
146
Therefore, the solution of recurrence relation will be
T(n)≤cn + cn + cn+……………..+cn
= cn(1+lg10/9n)
= cn + cnlg 10/9n
Therefore, T(n) = O(nlgn)
147
Exercise
1. Sort the following elements using quicksort
13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11.
2.What is the running time of QUICKSORT when all elements
of array A have the same value?
3.Show that the running time of QUICKSORT is‚ θ(n2) when
the array A contains distinct elements and is sorted in
decreasing order.
4.Suppose that the splits at every level of quicksort are in the
proportion 1-α to α, where 0 < α ≤ 1/2 is a constant. Show that
the minimum depth of a leaf in the recursion tree is
approximately lgn/ lgα and the maximum depth is
approximately lgn/ lg(1-α) . (Don’t worry about integer round-
off.) 148
A randomized version of quicksort
149
A randomized version of quicksort
The running time of randomized quick sort will be

T(n) = O(nlgn)
150
Lecture-13

Prayagraj
151
Sorting in Linear Time
Comparison Sort
In a comparison based sort, we use only comparisons between
elements to gain order information about an input sequence
<a 1, a2,……………., an>. That is, given two elements
ai and aj , we perform one of the tests ai < aj, ai ≤aj , ai=aj , ai
≥aj, or ai > aj to determine their relative order.
The decision-tree model

We can view comparison sorts abstractly in terms of decision
trees.
A decision tree is a full binary tree that represents the
comparisons between elements that are performed by a
particular sorting algorithm operating on an input of a given
152
size.
In a decision tree, we annotate each internal node by i:j for
some i and j in the range 1≤i, j ≤ n, where n is the number of
elements in the input sequence. We also annotate each leaf by
a permutation < 𝑟(1), 𝑟(2), 𝑟(3), … … … … , 𝑟(n) >.
 The execution of the sorting algorithm corresponds to

tracing a simple path from the root of the decision tree
down to a leaf.
 Each internal node indicates a comparison ai ≤ aj . The left
sub-tree then dictates subsequent comparisons once we
know that ai ≤ aj , and the right sub-tree dictates subsequent
comparisons knowing that ai > aj .
 When we come to a leaf, the sorting algorithm has
established the ordering <a𝑟(1), a 𝑟(2) ,…………., a𝑟(n)>. 153
The decision tree operating on three elements is the
following:-
154
Theorem:
Any comparison sort algorithm requires 𝛺(nlg n) comparisons
in the worst case.
Proof: The worst-case number of comparisons for a given
comparison sort algorithm equals the height of its decision
tree.
Consider a decision tree of height h with l reachable leaves
corresponding to a comparison sort on n elements. Because
each of the n! permutations of the input appears as some
leaf, we have n! ≤ l . Since a binary tree of height h has no
more than 2h leaves, therefore
n! ≤l ≤2h ⇒ h ≥ lg(n!)
Therefore h = 𝛺(nlgn)
155
Counting sort
 Counting sort assumes that each of the n input elements
is an integer in the range 0 to k, for some integer k.
 When k = O(n), the sorting algo. runs in θ(n) time.
 Counting sort determines, for each input element x, the

number of elements less than x. It uses this information to
place element x directly into its position in the output
array.
156
Counting sort
Example: Sort the following elements using counting sort
2, 5, 3, 0, 2, 3, 0, 3
Solution:
157
Counting sort
158
Counting sort
 Time complexity of counting sort is

T(n) = θ(n+k)
 If k ≤ n, then T(n) = θ(n)
159
Lecture-14
Manoj Mishra (Assistane Professor)

Prayagraj
160
Stable sorting
A sorting algorithm is said to be stable if two objects with
equal keys appear in the same order in sorted output as they
appear in the input array to be sorted.
Some stable sorting algorithms

1. Counting sort
2. Insertion sort
3. Merge Sort
4. Bubble Sort
Some unstable sorting algorithms

1. Selection sort
2. Quicksort
161
3. Heap sort
Radix sorting
 This sorting algorithm assumes all the numbers must be of
equal number of digits.
 This algorithm sort all the elements on the basis of digits

used in the number.
 First sort the numbers on the basis of least significant digit.

Next time on the basis of 2nd least significant digit. After it
on the basis of 3rd least significant digit. And so on. At the
last, it sort on the basis of most significant digit.
162
Radix sorting
Example: Sort the following elements using radix sort
326, 453, 608, 835, 751, 435, 704, 690.
Solution: In these elements, number of digits is 3. Therefore, we
have to used 3 iterations.
163
Radix sorting
164
Radix sorting
Time complexity of radix sort is
T(n) = θ(d(n+k))
Note: When d is constant and k=O(n), we can make radix

sort run in linear time i.e T(n) = θ(n).
165
Radix sorting
Time complexity of radix sort is
T(n) = θ(d(n+k))
Note: When d is constant and k=O(n), we can make radix

sort run in linear time i.e T(n) = θ(n).
166
Bucket sorting
 Bucket sort assumes that the input is generated by a
random process that distributes elements uniformly and
independently over the interval [0,1).
 Bucket sort divides the interval [0,1) into n equal-sized

subintervals, or buckets, and then distributes the n input
numbers into the buckets. Since the inputs are uniformly
and independently distributed over [0,1), we do not expect
many numbers to fall into each bucket. To produce the
output, we simply sort the numbers in each bucket and
then go through the buckets in order, listing the elements
in each.
167
Bucket sorting
Example: Sort the following elements using bucket sort
0.78, 0.17, 0.39, 0.26, 0.72, 0.94, 0.21, 0.12, 0.23, 0.68
168
Bucket sorting
Time complexity T(n) = θ(n)
169
Linear search
 It is a simple search algorithm that search a target value

in an array or list.
 It sequentially scan the array, comparing each array item
with the search element.
 If search element is found in an array, then the index of
an element is returned.
 The time complexity of linear search is O(n).
 It can be applied to both sorted and unsorted array.
170
Binary search
 Binary search is the most popular Search algorithm. It is

efficient and also one of the most commonly used
techniques that is used to solve problems.
 Binary search works only on a sorted set of elements. To

use binary search on a collection, the collection must first
be sorted.
 When binary search is used to perform operations on a

sorted set, the number of iterations can always be reduced
on the basis of the value that is being searched.
171
Binary search process
172
Binary search
Binary-search(A, n, x)
l=1
r=n
while l ≤ r
do
m = ⌊(l + r) / 2⌋
if A[m] < x then
l=m+1
else if A[m] > x then
r=m−1
else
return m
return unsuccessful
Time complexity T(n) = O(lgn)

173
AKTU Previous Year Questions
1. How analyze the performance of an algorithm in different cases?
2. Derive the time complexity of Merge sort.
3. Solve the recurrence i) T (n) =3T (n/4) + cn2 using recursion tree method. ii)
T (n) = n + 2T (n/2) using Iteration method. (Given T(1)=1)
4. Write Merge sort algorithm and sort the following sequence {23, 11, 5, 15,
68, 31, 4, 17} using merge sort.
5. What do you understand by stable and unstable sorting? Sort the following
sequence {25, 57, 48, 36, 12, 91, 86, 32} using heap sort.
6. What is recurrence relation? How is a recurrence relation solved using
Master’s theorem?
7. What is asymptotic notation? Explain Omega(Ω) notation.
8. Write an algorithm for counting sort. Illustrate the operation of counting
sort on the following array: A = {4,0,2,0,1,3, 5,4,1,3,2,3}.
9. Solve the following recurrence relations:-
(a) T(n) = T(n-1)+n4
(b) T(n) = T(n/4)+T(n/2) + n2
10.Write an algorithm for insertion sort? Find the time complexity in all the
174
case.
Comparison of Sorting
Sorting is a fundamental operation in computer science and plays a crucial role in a wide range of
applications. There are many sorting algorithms available, each with its own advantages and
disadvantages. Here's a comparison of some commonly used sorting algorithms based on various
factors:
1- Time Complexity:
Best Case: The time complexity in the best-case scenario when the input data is already sorted.
Average Case: The time complexity in an average-case scenario.
Worst Case: The time complexity in the worst-case scenario.
Bubble Sort:
Best Case: O(n)
Average Case: O(n^2)
Worst Case: O(n^2)
Insertion Sort:
Best Case: O(n)
Worst Case: O(n^2)
Selection Sort:
Best Case: O(n^2)
Worst Case: O(n^2)
175
Comparison of Sorting
Merge Sort:
Best Case: O(n log n)
Average Case: O(n log n)
Worst Case: O(n log n)
Quick Sort:
Worst Case: O(n^2) - but rarely occurs with good pivot selection
Heap Sort:
Worst Case: O(n log n)
Radix Sort:
Best Case: O(n * k)
Average Case: O(n * k)
Worst Case: O(n * k)
176
2- Space Complexity:
The amount of additional memory required for
sorting.
Bubble Sort, Selection Sort, Insertion Sort:
O(1) - They are in-place sorting algorithms and do not
require additional memory.
Merge Sort, Quick Sort:
O(n) - They require additional memory for temporary
storage.
Heap Sort:
O(1) - It's an in-place sorting algorithm but involves
restructuring the heap.
Radix Sort:
O(n + k) - It depends on the range of the input data.177
3- Stability:
Whether the sorting algorithm preserves the relative
order of equal elements.
Bubble Sort, Insertion Sort, Merge Sort, and Radix Sort:
Stable - They preserve the order of equal elements.
Selection Sort, Quick Sort, and Heap Sort:
Not stable - They may change the order of equal
elements.
178
4-Use Cases:
Different sorting algorithms are suitable for different
scenarios.
Bubble Sort, Insertion Sort, and Selection Sort:
Small data sets or nearly sorted data.
Merge Sort and Quick Sort:
General-purpose sorting for larger data sets.
Heap Sort:
In-place sorting with better average-case performance than
Bubble, Insertion, or Selection Sort.
Radix Sort:
When sorting integers with a fixed number of digits.
179
Advantages and Disadvantages:
•Each algorithm has its unique advantages and
disadvantages in terms of implementation complexity,
stability, and performance.
•The choice of a sorting algorithm depends on the

specific requirements of the problem, the size of the
data, and whether stability is a concern. It's essential
to consider these factors when selecting the
appropriate sorting algorithm for a given task.
180
Thank you.
181

Unit I (Design and Analysis of Algorithm)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Unit I (Design and Analysis of Algorithm)

Uploaded by

Copyright:

Available Formats

Design and Analysis of Algorithms

Manoj Mishra (Assistance Professor)

1. Thomas H. Coreman, Charles E. Leiserson and Ronald L.

2. E. Horowitz & S Sahni, "Fundamentals of Computer

An algorithm is any well-defined computational

An algorithm is thus a sequence of computational

An algorithm is a set of steps to accomplish or

 Input: Algorithm must contain 0 or more input values.

Algorithm: Insertion Sort:

We use the following conventions in our pseudo code.

 We access array elements by specifying the array name

 Divide and Conquer

Analysis of algorithms is the determination of the

Time complexity: T he amount of time required by an

Space complexity: T he amount of space/memory

The main concern of analysis of algorithms is the

Worst case − The maximum number of steps taken on

 Running time of an algorithm is the time taken by the

Ex. Sort the following elements by insertion sort:- 4, 3, 2,

T(n) = c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5.∑𝑗= 𝑛 𝑡

Best case: This case will be occurred when data is already

Manoj Mishra (Assistance Professor)

Time complexity, T(n) = θ (n2)

Divide the problem into a number of sub-problems that are smaller

Conquer the sub-problems by solving them recursively. If the sub-

Combine the solutions to the sub-problems into the solution for

When an algorithm contains a recursive call to itself, we

The recurrence equation for the running time of a divide-

T(n) = aT(n/b) + D(n) + C(n), if n > c

The recurrence equation for the running time of

T(n) = 2T(n/2) + θ(1)+ θ(n), if n > 1

It can be modified as:-

When we solve recurrence equation, modify it

Therefore, T(n) = θ(nlog(n)) 39

Manoj Mishra (Assistance Professor)

 This notation is said

 This notation is said

Manoj Mishra(Assistance Professor)

(1) If c is a constant such that 0< c < ∞, then f(n) = θ(g(n)).

(2) If c is a constant such that 0 ≤ c < ∞, then f(n) = O(g(n)).

(3) If c is a constant such that 0< c ≤ ∞, then f(n) = 𝛀(g(n)).

(4) If c is a constant such that c = 0, then f(n) = o(g(n)).

(5) If c is a constant such that c = ∞, then f(n) = 𝜔(g(n)).

1. Let f(n) and g(n) be asymptotically non-negative

2. Show that for any real constants a and b, where b > 0,

3. Solve the followings:-

2. Rank the following by growth rate: n, 2lg √n, log n, log

Manoj Mishra (Assistance Professor)

•A Recurrence is an equation or inequality that

(1) T(n) = aT(n/b) + f(n)

Some approaches to sole recurrence relations

The substitution methodfor solving recurrences comprises

1. Guess the form of the solution.

2.Use mathematical induction to find the constants and show

Example: Solve the given recurrence

Solve the given recurrence relation

Solve the given recurrence relation

Example: Find the upper bound of following recurrence

Solution: (Do yourself)

Solution: (Do yourself)

Manoj Mishra (Assistance Professor)

A recursion tree is best used to generate a good guess,

Solve the given recurrence relation using