BCOL306 Design & Analysis of Algorithm

Course Objectives
 To design and analyze the performance of algorithms by mathematical logic
building
 To evaluate instances by using Divide and Conquer techniques.
 To apply problem solving techniques such as Greedy and Dynamic
Programming strategy.
 To demonstrates Backtracking and Branch-Bound algorithmic design strategies
 To identifies and analyze NP-Hard and NP-Complete algorithms
 To study Parallel and String Matching algorithms.

Ms. Savitri B. Patil DAA 1

Course Outcomes:
 Describe concepts of Algorithmic characteristics
 Interpret various problem solving techniques using algorithmic
types
 Explain performance of algorithms using mathematical formulas
 Demonstrate design strategies for solving various applications
 Analyze complexity of problems for different computing areas

Ms. Savitri B. Patil DAA 2

 Introduction
 Divide and Conquer
 Greedy and dynamic programming
 Backtracking and Branch and bound
 Complexity theory
 Parallel and advanced algorithms
 Horowitz and Sahani, "Fundamentals of Computer Algorithms“
 Thomos H. Corman, Charls E. Leiserson, Ronald E. Rivest, Clifford

Stein, “Introduction to Algorithms”, Third Edition,Prentice Hall India

Learning Pvt. Ltd.

Ms. Savitri B. Patil DAA 3
Examination Scheme
 TAE-20 CAE-20
 TAE 1 Problem based CAE I – 1 and 2 Unit
 TAE 2 Presentation CAE II – 3 and 4 Unit
 TAE 5 Poster designing CAE III – 5 and 6 Unit
 ESE-60
 INTR PR -25
 EXTR PR -25

Ms. Savitri B. Patil DAA 4

What is an algorithm?

 Algorithm is a set of steps to complete a task.

 A sequence of unambiguous instructions for solving

problem
 For example,
 Task: to make a cup of tea.
 Algorithm:

· add water and milk to the kettle,

· boil it, add tea leaves,
· Add sugar, and then serve it in cup.

Ms. Savitri B. Patil DAA 5

Cont..

Euclidian algorithm Middle School Procedure

GCD (60, 24) 60 = 2 . 2 . 3. 5

a =bxq+r
24 = 2. 2. 2. 3
60 = 24 x 2 + 12
2. 2 .3 =12
24 = 12 x 2 + 0

12 = 0

Ms. Savitri B. Patil DAA 6

 What is Computer algorithm?
 “a set of steps to accomplish or complete a task that is

described precisely enough that a computer can run it’’.

 Described precisely: very difficult for a machine to know

how much water, milk to be added etc. in the above tea
making algorithm.

 These algorithms run on computers or computational

devices. For example, GPS in our smartphones, Google
hangouts.

 GPS uses shortest path algorithm. Online shopping uses

cryptography which uses RSA algorithm.
Ms. Savitri B. Patil DAA 7
Characteristics of an algorithm:-

1. Must take an input.

2. Must give some output(yes/no, value etc.)
3. Definiteness –each instruction is clear and unambiguous.
4. Finiteness –algorithm terminates after a finite number of
steps.
5. Effectiveness –every instruction must be basic i.e. simple
instruction.

Ms. Savitri B. Patil DAA 8

Design and Analysis of Algorithm Process

Ms. Savitri B. Patil DAA 9

 The Need for Analysis
 Algorithms are often quite different from one another, though the
objective of these algorithms are the same. For example, we know
that a set of numbers can be sorted using different algorithms.
Number of comparisons performed by one algorithm may vary with
others for the same input. Hence, time complexity of those
algorithms may differ. At the same time, we need to calculate the
memory space required by each algorithm.

Ms. Savitri B. Patil DAA 10

Algorithm Specification
 Name of the algorithm
 Comments

 Data types

 Assignment statements

 Expression

 If statement

 Switch statement

 Looping statements

 Input/output statements

 Functions

Ms. Savitri B. Patil DAA 11

Analysis of Algorithms
 Analysis of algorithms is the determination of the amount of time and
space resources required to execute it.

 Usually, the efficiency or running time of an algorithm is stated as a

function relating the input length to the number of steps, known as
time complexity, or volume of memory, known as space complexity.

Ms. Savitri B. Patil DAA 12

Time Complexity

It’s a function describing the amount of time required to run an algorithm

in terms of the size of the input. "Time" can mean the number of
memory accesses performed, the number of comparisons between
integers, the number of times some inner loop is executed, or some other
natural unit related to the amount of real time the algorithm will take.

T(n) = Cop.C(n)

- time taken by basic operation to execute

- number of times the operation needs to be executed

Ms. Savitri B. Patil DAA 13

Space complexity
 It’s a function describing the amount of memory an algorithm takes in
terms of the size of input to the algorithm. We often speak of "extra"
memory needed, not counting the memory needed to store the input
itself.

 Space complexity is sometimes ignored because the space used is

minimal and/or obvious, however sometimes it becomes as important
an issue as time.

 S(p) = C + Sp
Fixed part: Instruction, Variables, space for constants
and array size
Variable part: whose size is dependent upon the
particular problem instance being solved
Ms. Savitri B. Patil DAA 14
 Analysis of algorithm is the process of analyzing the problem-solving
capability of the algorithm in terms of the time and size required (the
size of memory for storage while implementation). However, the
main concern of analysis of algorithms is the required time or
performance. Generally, we perform the following types of analysis −

 Worst-case − The maximum number of steps taken on any instance

of size a.

 Best-case − The minimum number of steps taken on any instance of

size a.

 Average case − An average number of steps taken on any instance of

size a.

Ms. Savitri B. Patil DAA 15

 To solve a problem, we need to consider time as well as space
complexity as the program may run on a system where memory is
limited but adequate space is available or may be vice-versa. In this
context, if we compare bubble sort and merge sort. Bubble sort does
not require additional memory, but merge sort requires additional
space. Though time complexity of bubble sort is higher compared to
merge sort, we may need to apply bubble sort if the program needs to
run in an environment, where memory is very limited.

Ms. Savitri B. Patil DAA 16

Growth of Functions ( Asymptotic notations)

 It is a way to describe the characteristics of a function

in the limit.
 It describes the rate of growth of functions.
 Focus on what’s important by abstracting away low-

order terms and constant factors.

 It is a way to compare “sizes” of functions:
Big O
Notation Name Example(s)
# Odd or Even number,
O(1) Constant
# Look-up table (on average)
# Finding element on sorted array with 
O(log n) Logarithmic
binary search
O(n) Linear # Find max element in unsorted array,
# Sorting elements in array with 
O(n log n) Linearithmic
merge sort
# Duplicate elements in array **(naïve)**,
O(n2) Quadratic
# Sorting array with bubble sort
O(n3) Cubic # 3 variables equation solver
O(2n) Exponential # Find all subsets
# 
O(n!) Factorial
Find all permutations of a given set/string
Asymptotic Notations
 Execution time of an algorithm depends on the instruction set,
processor speed, disk I/O speed, etc. Hence, we estimate the
efficiency of an algorithm asymptotically.
 Time function of an algorithm is represented by T(n), where n is the
input size.
 Different types of asymptotic notations are used to represent the
complexity of an algorithm. Following asymptotic notations are used
to calculate the running time complexity of an algorithm.
 O − Big Oh
 Ω − Big omega
 θ − Big theta
 o − Little Oh
 ω − Little omega
CONTINUED..

 O: Asymptotic Upper Bound

 ‘O’ (Big Oh) is the most commonly used notation. A function f(n) can
be represented is the order of g(n) that is O(g(n)), if there exists a
value of positive integer n as n0 and a positive constant c such that −
 f(n)⩽c.g(n)
 for n>n0in all case
 Hence, function g(n) is an upper bound for function f(n), as g(n)
grows faster than f(n).
CONTINUED..

 Ω: Asymptotic Lower Bound

 We say that f(n)=Ω(g(n))
 when there exists constant c that f(n)⩾c.g(n)for all sufficiently large
value of n. Here n is a positive integer. It means function g is a lower
bound for function f; after a certain value of n, f will never go below
g.
 θ: Asymptotic Tight Bound
 We say that f(n)=θ(g(n))
 when there exist constants c1 and c2 that c1.g(n)⩽f(n)⩽c2.g(n)for all
sufficiently large value of n. Here n is a positive integer.
 This means function g is a tight bound for function f.
Time complexity

#include <stdio.h>
int main()
{
    printf("Hello World");
}

#include <stdio.h>
void main()
{
    int i, n;
    for (i = 1; i <= n; i++) { 3C n+1 3(n+1)+n = 3n+3+n
        printf("Hello Word !!!\n"); 1C n 4n+3 =O(n)
    }
}
Sum(a,b){
return a+b ; or c=a+b; cost 2 no of times 1
return c cost 1 no of time 1
} O(1)

list_Sum(A,n){
total =0    cost 1 no of times 1        
for i=0 to n-1     cost 2 no of times n+1
sum = sum + A[i]   cost 2 no of times n
return sum         cost 1 no of times 1
}
1+2n+2+2n+1 = 4n+4 = O(n)
   
 for i=0 to n-1 cost = 2 no of times n+1
for j=0 to n-1 2 n(n+1)
sum= sum+a[i] 2 nn
return sum 1 1

2(n+1)+2(nn+n)+2(nn)+1
2n+2+2n.n+2n+2n.n+1
4n2+4n+3
O(n2)

-drop all lower order terms

-drop all constant terms
i) a =a+b c1
ii) for i=0 to n-1 or for i=1 to n n+1
x= y+z c2 n

iii) for i=0 to n-1 n+1

c=d+e c3 n

c1+(n+1)+c2n+(n+1)+c3n
 If else statement

if ( )
{
O(n)
}

else
{
O(n2)
}
for (i=1; i<n; i=i*2)
Statement;

i= 1 1x2=2 n=8 n=10

i=2 2x2 =22 i=1, 2, 22, 23 =3C
i= 22 22 x2 =23 i=1, 2, 22, 23, 25 =4C
i= 23 23x2=24
….

…..
2k log28 =3 log210 =3.3
log223 = 3log22 =3

Assume i>=n
i=2k 2k>=n we can write 2k=n
k=log n O(log n)
Frequency Count Method
  s/c Frequency Total
Function ArraySum(A , n)  0 0  0
Sum=0; 1 1 1
for(i=0; i<n; i++) 2 n+1 2n+2
{
sum=sum+A[i]; 2 n 2n
}
return sum; 1 1 1
End Function
(n) 4n+4
Frequency Count Method
  Step Count
int sum(int n)
{
int sum=0; 1
for(i=0; i<n; i++) 1+(n+1)+n
sum+=i*i*i 4n
return sum; 1
}
O(n) 6n+4
1
1
1+(n+1)+n
2n+2
n
2n
n

1
1
1
6n+7 =>O(n)
  i P
p=0; 1 0+1=1
for(i=1;p<=n; i++) 2 0+1+2=3
p=p+i; 3 0+1+2+3=4
return p; 4 0+1+2+3+4=5
} . p>n
. k(k+1)/2
k (k2 +k)/2>n

k2>n
k>n =O(n)
  i
for(i=n; i>=1; i=i/2)
{
Statement; n=6
} i= 6 stat-1
i=3 stat-2
i=1 stat-3
i=0
  i
for(i=1; i*i<=n; i++) 1*1
{ 2*2 i*i >=n
Statement; 3*3 i2 >=n
} 4*4 i >= n
. O(n)
.
.
.
i2
  i j No of
times
for (i=0; i< n; i++) 0 0 0
1 0 1
for (j=0; j < i; j++)
statements 1x
2 0 2
1
2x
1+2+3+……n 3
0 3
1
n(n+1)/2 =(n2 +n)/2 2
3
4x
(n2)
for( ; m!=n; )
{
if(m>n)
m=m-n; M=24 n=12
12 12
else
m=16 N=2 m=34 n=12
n=n-m; M=8 N=1
14 2 22 12
7 1
} 12 2 10 12
6 1
10 2 10 2
5 1
8 2 8 2
4 1
6 2 6 2
3 1
4 2 4 2
2 1
2 2 2 2
1 1
Test(n)
{
if(n>5)
printf(“%d”, n);
else
{
for(i=0; i<n; i++)
printf(“%d”, i);
}
Space complexity
int add (int x, int y, int z)
{
int sum= x+y+z;
return sum;
}

void add(int A[], int n)

{
int sum;
for(i=0; i<n; i++)
sum=sum+A[i];
print(%d”, sum);
Algorithm add(a, b, n)
for(i=0; i<n; i++) n+1
for(j=0; j<n; j++) n(n+1)
c[i][j]=a[i][j]+b[i][j]; n. n
end
O(n2 )

Variables
a - n.n
b - n.n
c - n.n
n , i, j = 3n2+3 O(n2)
Algorithm add(a, b, n)
for(i=0; i<n; i++)
for(j=0; j<n; j++)
c[i][j]=0;
for(k=0; k<n; k++)
c[i][j]=c[i][j]+a[i][k]+b[k][j];
end
O(n3 )

Variables
a - n.n
b - n.n
c - n.n
n , i, j k,= 3n2+4 O(n2)
 Imagine a classroom of 50 students in which you gave your pen to
one person. Now, you want that pen.

 The ways to find the pen and what the O order is?

 Going and asking each student individually

 Now I divide the class into two groups, then ask: “Is it on the left
side, or the right side of the classroom?” Then I take that group and
divide it into two and ask again, and so on. Repeat the process till
you are left with one student who has your pen. 

 You go and ask the first person of the class, if he has the pen. Also,
you ask this person about other 49 people in the classroom if they
have that pen and so on,