Reference books

• Introduction to Algorithms by Thomas H. Cormen, PHI

• Fundamentals of Computer Algorithms by Ellis Horowitz, Sartaj
Sahni, S. Rajshekhran.
• Fundamentals of Algorithms by Gills Brassard, PHI
• Introduction to Design and Analysis of Algorithms by Anany Levitin,
Pearson
• Foundations of Algorithms by Shailesh R. Sathe, Penram
• Design and Analysis of Algorithms by Dave and Dave, Pearson

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu
Prerequisites

• Knowledge of Basic Mathematics

• Data Structures
• Programming Language like C,C++, Python etc.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 Motivation

• Why do we Study Algorithm?

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 • Heart of technology
• Emphasizes the skills needed to design, Implement
solution to problems
• Gives a solid grounding in problem solving
• Employment focused course with a balance of theory
and practice
• Provide preparation for a number of industry standard
• Placement oriented subject.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 What is an algorithm?

• An algorithm is a finite set of instruction that, if

followed, accomplishes a particular task.
OR
• “An algorithm is any well-defined computational
procedure that takes some value, or set of values, as
input and produces some value, or set of values as
output.” [Thomas H. Cormen, Charles E. Leiserson,
Ronald L. Rivest, Clifford Stein]
OR
• It is a combination of sequence of finite steps(one or
more operation) to solve the problem.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 Informal definition of an algorithm
used in a computer

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

What is an algorithm?

• An algorithm is a finite set of instruction that if followed accomplishes

a particular task
• All algorithms must satisfy the following criteria:
• Input: Zero or more quantities supplied
• Output: At least one quantity is produced
• Definiteness: Each instruction is clear and unambiguous
• Finiteness: If we trace out the instructions of an algorithm, then for all cases,
the algorithm terminates after a finite number of steps
• Effectiveness: Every instruction must be a very basic so that it can be carried
out in principal by a person using only pen and pencil

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

Goals of Algorithm Study
• To develop framework for instructing computer to perform tasks.

• To introduce notion of algorithm as means of specifying how to solve

a problem.

• To introduce and appreciate approaches for defining and solving very

complex tasks in terms of simpler tasks.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

Application of Algorithms
• Efficiently accessing and retrieving the large amount of information
over the internet.
• Electronic commerce : Securing credit card, passwords, bank
statement details.
• Manufacturing and other commercial sites like an Oil company , US
president Election, ISP.
• Solution of biological problems like identifying genes in Human DNA.
• Finding shortest route over the Network.
• Matrix chain Multiplication and many such mathematical problem
solving.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 117 Graph
110
216
problems
246

199 182 • Coloring problem

• Shortest path
170
121 231
315
• Traveling
79 142
191
242
148 78
136 salesman
191 126 120
178 149 89
116 234
170
51 112 79 131 109
163 73
90 143 72 63 86
53
105 27
59 58
135
116

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 Finding the largest integer
among five integers

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 Defining actions in FindLargest algorithm

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 FindLargest refined

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 Generalization of FindLargest

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

Algorithm Development Cycle
• Identify problem statement
• Identify constraints
• Design : Express logic or strategy by Algorithm/
Flowchart/ Case Diagram
• Divide and conquer
• Greedy method
• Dynamic programming
• Branch and Bound
• Backtracking
• Validation
• Analysis
• Implementation
• Testing and Debugging

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 Analysis of algorithms

• Analysis includes 1. Determine Resource consumption

2. Performance Comparison
Resource Consumption includes
• Time
• Space
• Other factors like Registers, Network bandwidth, power
consumption(Battery) etc.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

Analysis of algorithms
The theoretical study of computer-program
performance and resource usage.
What’s more important than performance?
• modularity
• correctness
• maintainability
• functionality
• robustness
• user-friendliness
• programmer time
• simplicity
• extensibility
• reliability
5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu
 Analysis of algorithms
Why study algorithms performance?
Need of Analysis
• The most straightforward reason for analyzing an algorithm is to discover
its characteristics in order to evaluate its suitability for various
applications or compare it with other algorithms for the same application.

• Determination of the amount of resources (such as time and storage)

necessary to execute them.

•Performance often draws the line between what is feasible and what is
impossible.

• Speed is fun!
5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu
 • How to do Analysis?
Time taken depends on 2 factors
1. Machine or Hardware dependent
2. Platform or Software dependent

Types of Analysis
1. Apriori Analysis : Machine and Platform independent
2. Apostriori Analysis : Machine and Platform dependent

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu 22

How to compare Algorithm?
Execution time: Not a good measure as execution time
varies to a particular computer.

Number of statements executed: Not a good measure,

since no. of statements varies with the programming
language.

Ideal Solution: Assume that we expressed running time of

given algo as a function of the input size n (i.e. f(n)) and
compare these different functions corresponding to running
time.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu 23

Time Complexity Is a Function
Specifies how the running time depends on the size of the input.

A function mapping

“size” of input

“time” T(n) executed .

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu 24

Definition of Time
• # of seconds (machine, implementation
dependent).

• # lines of code executed.

• # of times a specific operation is performed

(e.g., addition).

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu 25

Theoretical analysis of time efficiency

Time efficiency is analyzed by determining the number of repetitions of

the basic operation as a function of input size.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu 26

Behaviour of Algorithm

w.r.to. inputs
• Best case
• Average case
• Worst case
Note: Choosing what kind of input to be consider when analyzing an
algorithm have a significant impact on the behaviour of Algorithm.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu 27

Best-case, average-case, worst-case

• Worst case: W(n) – maximum over inputs of size n

• Defines the input for which the algorithm takes long time

• Best case: B(n) – minimum over inputs of size n

• Defines the input for which the algorithm takes lowest time

• Average case: A(n) – “average” over inputs of size n

• NOT the average of worst and best case
• Under some assumption about the probability distribution of all possible inputs of
size n, calculate the weighted sum of expected C(n) (numbers of basic operation
repetitions) over all possible inputs of size n.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

 Machine-independent time

What is insertion sort’s worst-case time?

• It depends on the speed of our computer,

• relative speed (on the same machine),
• absolute speed (on different machines).

BIG IDEA:
• Ignore machine-dependent constants.
• Look at growth of T(n) as n ─> ∞
“Asymptotic Analysis”

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

Asymptotic Analysis
•Asymptotic analysis of an algorithm, refers to defining the mathematical
boundation/framing of its run-time performance.

•Asymptotic analysis refers to computing the running time of any operation in

mathematical units of computation.

•Asymptotic analysis are input bound i.e., if there's no input to the algorithm it
is concluded to work in a constant time. Other than the "input" all other
factors are considered constant.

• Using asymptotic analysis, we can very well conclude the best case,
average case and worst case scenario of an algorithm.

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

Asymptotic Notation
Notations for describing the behavior of Algorithm and determining
the rate of growth of Function.

Growing an input from some value k to ∞ is Asymptotic growth.

Notations are: 1. Big-Oh (O)

2. Big-Omega(Ω)
3. Theta (Θ)
4. Small-Oh(o)
5. Small Omega(w)

5/7/2022 Prepared by: Asst. Prof. Lokesh Sahu

What is rate of growth ?

• The rate at which the running time increases as a function of input is

called rate of growth.

• Total cost = cost_of_car + cost_of _cycle

• Total cost = cost_of _car (approximation)
Rate of growth
Big – Oh (O) Notation
• Given nonnegative functions f and g defined on positive integers,
we write f(n) = O (g(n)) if and only if there is positive constant C and
integer n0 such that f(n)≤ C . g(n) where C>0, n0 ≥ 1 ,for all n ≥ n0.
Examples

• Ex 1: Find upper bound for f(n) = 3n + 8

Sol: 3n+8 <= 4n , for all n>=8
3n+8 = O(n) with c=4 and n0 = 8

• Ex 2: Find upper bound for f(n) = n^2+1

Sol: n^2 + 1 <= 2n^2, for all n>=1
n^2 + 1 = O(n^2) with c=2 and n0 = 1

• Ex 4: Find upper bound for f(n) = n

Sol: O(n) with c=1 and n0 = 1
Big – Omega ( Ω ) Notation
• Given nonnegative functions f and g defined on positive
integers, we write f(n) = Ω (g(n)) if and only if there is positive
constant C and integer n0, f(n)  C * g(n) whenever nn0
Examples

• Find the lower bound for f(n) = 5n2

Sol: there exist c, n0 such that: 0 <= cn2 <= 5n2
cn2<=5n2
c=1, and n0 = 1
So, 5n2 = Ω(n2) with c = 1, and n0 = 1
Theta ( Θ ) Notation
• Given nonnegative functions f and g defined on positive integers, we
write f(n) = Θ (g(n)) if and only if there is positive constant C and
integer n0
C1* g(n)  f(n)  C2* g(n) whenever n  n0

Design and Analysis of Algorithms Chapter 2.1

Examples of Asymptotic Notations
Q1: f(n) = n, g(n) = n^2
Which of the following is true
a. f(n) = O(g(n))
b. f(n)=Ω(g(n))
c. f(n)=θ(g(n))
d. none
Ans: a

Q2: f(n) = n^2, g(n) = 2^n. what is the relationship

between f(n) and g(n)?
a. O
b. Ω
c. θ
d. None
Ans: a
True/false

1) (n + k)m = O(nm) where k is constant::

:: True
2) 2n+1 = O(2n)
:: true
3)22n = O(2n)
:: false
Frequency Count
A counts that denotes how many times the particular statement is executed

Ex1 calculate frequency count for:

For(i=0;i<n;i++)
Sum=sum + I;
Ans: O(n)

Ex2 calculate frequency count for:

For(i=0;i<n;i++)
For(j=0;j<n;j++)
a=a+2;
Ans: O(n^2)
Ex 3. calculate frequency count for:

i=1
Ex 4: calculate frequency count for:

While(i<=n) For(i=0;i<n;i++)
For(j=0;j<n;j++)
{ for(k=0;k<n;k++)

x=x+1; x=x+1;
Ans: O(n^3)
i=i+1;
}
Ans: 3n+2 = O(n)
• Ex 4: calculate frequency count for:

main() int i, j, k = 0;
for (i = n / 2; i <= n; i++) {
{ for (j = 2; j <= n; j = j * 2) {
k = k + n / 2;
i=n; }
While(i>=1) }
{ O(n)
O(nLogn)
i=i/2; O(n^2)
} O(n^2Logn)
}
Ans: O(log n)
• Ex 4: calculate frequency count for:

Main()
{
For( i= 1 to n)
{
For( j=1 to n )
{
For (k=1 to 165)
{
X=Y+Z;
}}}}

• Ans: O(n^2)
Space Complexity
• Space Complexity of an algorithm is the total space taken by the
algorithm with respect to the input size. Space complexity includes
both Auxiliary space and space used by input.
• Auxiliary Space is the extra space or temporary space used by an
algorithm.
Example
• int add (int n){
• if (n <= 0){
• return 0;
• }
• return n + add (n-1);
• }
1. add(4)
2. -> add(3)
3. -> add(2)
4. -> add(1)
5. -> add(0)
Each of these calls is added to call stack and takes up actual memory.
So it takes O(n) space.
Example-2
• int main(){
• int n, i, sum = 0;
• scanf("%d", &n);
• int arr[n];
• for(i = 0; i < n; i++)
• {
• scanf("%d", &arr[i]);
• sum = sum + arr[i];
• }
• printf("%d", sum);
• }
Space Requirement: the array consists of n integer elements. So, the space occupied by
the array is 4 * n. Also we have integer variables such as n, i and sum. Assuming 4 bytes for
each variable, the total space occupied by the program is 4n + 12 bytes. Since the highest
order of n in the equation 4n + 12 is n, so the space complexity is O(n) or linear.