Lec - 2 - Analysis of Algorithms (Part 2)

Lecture 2
Analysis of Algorithms
(Part 2)
Big O Notation
• Big-O is an Asymptotic Notation for the ceiling
of growth for a given function. It provides us
with an asymptotic upper bound for the growth
rate of the runtime of an algorithm.
• f(n) = O(g(n)) that is, f of n is Big–O of g

of n if there exist positive constants c and
𝒏𝟎 such that f(n) ≤ cg(n) for all 𝒏 ≥ 𝒏𝟎 .
• It means that for large amounts of data, f(n)

will grow no more than a constant factor than
g(n).
Analysis of Algorithms 2
Big O Notation
• Example: Show that 30n+8 is O(n).
• Here we have 𝑓 𝑛 = 30n+8, and 𝑔 𝑛 = 𝑛

• We need to prove that 𝟑𝟎𝒏 + 𝟖 ≤ 𝒄𝒏 for some constant c.
• Let, c = 31. Then we have
30𝑛 + 8 ≤ 31𝑛 for all 𝑛 ≥ 8.
• Therefore,
30𝑛 + 8 = O 𝑛 with c = 31 and 𝑛0 = 8 (Proved)
Big O Notation
• Note 30n+8 isn’t less than
n anywhere (n>0). cn =
31n 30n+8
Value of function →
• It isn’t even less than 31n
everywhere.
30n+8
n
• But it is less than 31n = O(n)
everywhere to
the right of n=8. n>n0=8 →
Increasing n →
Big O Notation
• Example: Is 3𝑛 + 2 = O 𝑛 ?
• Here we have 𝑓 𝑛 = 3𝑛 + 2, and 𝑔 𝑛 = 𝑛

• We need to prove that 𝟑𝒏 + 𝟐 ≤ 𝒄𝒏 for some constant c.
• We notice that when c = 4, we have 3𝑛 + 2 ≤ 4𝑛 for all 𝑛 ≥ 2.

• Therefore,
𝑓 𝑛 = O 𝑔(𝑛)
Or, 3𝑛 + 2 = O 𝑛
• with c = 4 and 𝑛0 = 2
Big O Notation
• Example: Is 𝑛2 + 𝑛 = 𝑂 𝑛3 ?
• Here we have 𝑓 𝑛 = 𝑛2 + 𝑛, and 𝑔 𝑛 = 𝑛3

• Notice that if 𝑛 ≥ 1, we have 𝑛 ≤ 𝑛3 .
• Also, notice that if 𝑛 ≥ 1, we have n2 ≤ 𝑛3
• Therefore,
𝑛2 + 𝑛 ≤ 𝑛3 + 𝑛3 = 2𝑛3
• We have just shown that
𝑛2 + 𝑛 ≤ 2𝑛3 for all 𝑛 ≥ 1
• Thus, we have shown that 𝑛2 + 𝑛 = 𝑂 𝑛3 with 𝐜 = 𝟐 𝐚𝐧𝐝 𝒏𝟎 = 𝟏
Big O Notation
Big O visualization
O(g(n)) is the set
of functions
with smaller or
same order of
growth as g(n)
Big Ω Notation
― The asymptotic lower bound
― The function 𝐟 𝐧 = Ω(𝐠(𝐧)) if and only if there exist positive constants c
and n0 such that f(n) ⩾ c.g(n) for any n > n0 .
― read as “f of n is omega of g of n”
Big Ω Notation
• Example: 3𝑛 + 2 = Ω 𝑛 ?
• Here we have 𝑓 𝑛 = 3𝑛 + 2, and 𝑔 𝑛 = 𝑛

• Notice that for any 𝑛 ≥ 1, we have 3𝑛 + 2 ≥ 𝑛.
• Therefore,
𝑓 𝑛 = Ω 𝑔(𝑛)
or 3𝑛 + 2 = Ω 𝑛
• with c = 1 and 𝑛0 = 1
Big θ Notation
― The asymptotic tight bound
― The function 𝐟 𝐧 = 𝛉(𝐠(𝐧)) if and only if there exist positive
constants c1 , c2 and n0 such that c1 . 𝑔 𝑛 ≤ f 𝑛 ≤ c2 . 𝑔(𝑛) for any n > n0 .
― read as “f of n is theta of g of n”
Big θ Notation
• Example: n2 + 5n + 7 = θ(n2 ) ?
• Here we have 𝑓 𝑛 = n2 + 5n + 7, and 𝑔 𝑛 = n2

• When 𝑛 ≥ 1, we have
n2 + 5n + 7 ≤ n2 + 5n2 + 7n2
≤ 13n2
• Again, when n ≥ 1,
𝑛2 ≤ n2 + 5n + 7
• Thus, 𝑓 𝑛 = θ 𝑔(𝑛) or n2 + 5n + 7 = θ(n2 ) with 𝒏𝟎 = 𝟏, 𝒄𝟏 = 𝟏 and 𝒄𝟐 =
𝟏𝟑
Big θ Notation
• Subset relations between order-of-growth sets.
R→R
O( f ) ( f )
•f
( f )
Other Notations (Self-Study)
• Little o Notation
• Little Omega Notation (ω)
Common Rates of Growth
According to the Big O notation, we have five different categories
of algorithms. Their running time complexities are given as:
Faster algorithms
• Constant : O(1)
• Logarithmic : O(logn)
• Linear : O(n)
• Log-linear : O(n logn)
• Polynomial : O(𝒏𝒌 ) where k > 1
• Exponential : O(𝒌𝒏 ) where k > 1

Slower algorithms
Properties of Asymptotic Notations
General Properties:
• If f(n) is O(g(n)) then a*f(n) is also O(g(n)) ; where a is a constant.
• Example:
f(n) = 2n²+5 is O(n²)
then 7*f(n) = 7(2n²+5)
= 14n²+35 is also O(n²)
• Similarly, this property satisfies both Θ and Ω notation. We can say

If f(n) is Θ(g(n)) then a*f(n) is also Θ(g(n)); where a is a constant.
If f(n) is Ω (g(n)) then a*f(n) is also Ω (g(n)); where a is a constant.
Reflexive Properties:
• If f(n) is given then f(n) is O(f(n)).
• Example: f(n) = n²
Then f(n) = O(n²) i.e O(f(n))
• Similarly, this property satisfies both Θ and Ω notation. We can say

If f(n) is given then f(n) is Θ(f(n)).
If f(n) is given then f(n) is Ω (f(n)).
Transitive Properties :
• If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) = O(h(n)) .
• Example: if f(n) = n , g(n) = n² and h(n)=n³

n is O(n²) and n² is O(n³) then n is O(n³)
• Similarly this property satisfies for both Θ and Ω notation. We can say
If f(n) is Θ(g(n)) and g(n) is Θ(h(n)) then f(n) = Θ(h(n)) .
If f(n) is Ω (g(n)) and g(n) is Ω (h(n)) then f(n) = Ω (h(n))
Symmetric Properties :
• If f(n) is Θ(g(n)) then g(n) is Θ(f(n)) .
• Example: f(n) = n² and g(n) = n² then f(n) = Θ(n²) and g(n) =

Θ(n²)
• This property only satisfies for Θ notation.
Transpose Symmetric Properties :

• If f(n) is O(g(n)) then g(n) is Ω (f(n)).
• Example: f(n) = n , g(n) = n² then n is O(n²) and n² is Ω (n)
• This property only satisfies for O and Ω notations.
Some More Properties :
1. If f(n) = O(g(n)) and f(n) = Ω(g(n)) then f(n) = Θ(g(n))
2. If f(n) = O(g(n)) and d(n)=O(e(n))
then f(n) + d(n) = O( max( g(n), e(n) ))
Example: f(n) = n i.e O(n)

d(n) = n² i.e O(n²)
then f(n) + d(n) = n + n² i.e O(n²)
3. If f(n)=O(g(n)) and d(n)=O(e(n))

then f(n) * d(n) = O( g(n) * e(n) )
Example: f(n) = n i.e O(n)

d(n) = n² i.e O(n²)
then f(n) * d(n) = n * n² = n³ i.e O(n³)
General Rules for (Iterative) Algorithm Analysis
RULE 1- FOR LOOPS:
• The running time of a for loop is at most the running time of the
statements inside the for loop (including tests) times the number of
iterations.
(Taken from Chapter 2 of ‘Data Structures & Algorithm Analysis in C’ by Mark A. Weiss.)
RULE 2- NESTED FOR LOOPS:
• Analyze these inside out. The total running time of a statement inside a
group of nested for loops is the running time of the statement
multiplied by the product of the sizes of all the for loops.
• As an example, the following program fragment is 𝑂(𝑛2 ):
for( i=0; i<n; i++ )

for( j=0; j<n; j++ )
k++;
RULE 3- CONSECUTIVE STATEMENTS:
• These just add (which means that the maximum is the one that
counts). As an example, the following program fragment, which has
𝑂(𝑛) work followed by 𝑂(𝑛2 ) work, is also 𝑂 (𝑛2 ):
for( i=0; i<n; i++)

a[i] = 0;
for( i=0; i<n; i++ )
for( j=0; j<n; j++ )
a[i] += a[j] + i + j;
RULE 4- lF/ELSE:
• For the fragment
if( cond )
S1
else
S2
• the running time of an if/else statement is never more than the

running time of the test plus the larger of the running times of S1 and
S2.
Worst-case, Average-case, Best-case Analysis
• Worst-case running time
✓ This denotes the behaviour of an algorithm with respect to
the worst possible case of the input instance.
✓ The worst-case running time of an algorithm is an upper

bound on the running time for any input.
✓ Therefore, having the knowledge of worst-case running time

gives us an assurance that the algorithm will never go beyond
this time limit.
• Average-case running time
✓ The average-case running time of an algorithm is an estimate
of the running time for an ‘average’ input.
✓ It specifies the expected behaviour of the algorithm when the

input is randomly drawn from a given distribution.
✓ Average-case running time assumes that all inputs of a given

size are equally likely.
• Best-case running time
✓ The term ‘best-case performance’ is used to analyse an algorithm
under optimal conditions.
✓ For example, the best case for a simple linear search on an array
occurs when the desired element is the first in the list.
✓ However, while developing and choosing an algorithm to solve a

problem, we hardly base our decision on the best-case
performance.
✓ It is always recommended to improve the average performance and

the worst-case performance of an algorithm.
Related Readings
➢ Introduction to Algorithms (CLRS)
• Chapter 3 (3.1)
➢ Fundamentals of Computer Algorithms (Horowitz, Sahni, Rajasekaran)

• Chapter 1 (1.3.4)
• https://yourbasic.org/algorithms/big-o-notation-explained/
• Asymptotic Notations:
https://drive.google.com/file/d/1PaF_NqJeqykBVT63RcSJRtm_2iUFpz
MB/view?usp=sharing
Try Yourself!
Q1. Show that, Q3. Consider the following two functions:
a) 3n2 + 4n − 2 = 𝚯(n2) 𝑛3 for 0 ≤ 𝑛 ≤ 10000
𝑔1 𝑛 = ቊ 2
b) 3logn + 100 = O(logn) 𝑛 𝑓𝑜𝑟 𝑛 ≥ 10000
𝑛 𝑓𝑜𝑟 0 ≤ 𝑛 ≤ 100
𝑔2 𝑛 = ቊ 3
Q2. Consider the following three claims 𝑛 𝑓𝑜𝑟 𝑛 ≥ 100
1) 𝐧 + 𝟓 𝟐 = 𝚯(𝐧𝟐 ) Which of the following statements are true?
2) 𝟐𝐧 + 𝟏 = 𝐎(𝟐𝐧 )
(A) g1 n is O(g 2 n )
3) 𝟐𝟐𝐧 + 𝟏 = 𝐎(𝟐𝐧 )
(B) g 2 n is O(n)
Which of these claims are correct ?

(C) g 2 n is O(g1 n )
(A) 1 and 2 (D) g1 n is O(n3 )

(B) 1 and 3 Q4. Show that,
𝒑
(C) 2 and 3
෍ 𝒌𝒏𝒌 = 𝚯 𝒏𝒑
(D) 1, 2, and 3 𝒌=𝟏
[p is a positive integer]
Iterative Algorithm Analysis
➢ Analyze the following algorithms.
Or,
➢ Find out the order of growth/time complexity/big-O of the following

algorithms.
Alg()
{
int i;
for i = 1 to n
{
printf(“Hello!”);
}
}
Solution: For this algorithm, the running time mainly depends on the
number of times the loop is executed.
Since the loop will be executed n times, this algorithm is 𝑶(𝒏).
Alg()
{
int i = 0, s = 0;
while (s < n)
{
i++;
s = s + i;
}
}
Solution: For this algorithm, the running time mainly depends on the number
of times the loop is executed.
Here, i is incremented by 1 and s is incremented by i after each iteration.
Let, the loop will be executed k times (i.e. after k iterations, 𝑠 ≥ 𝑛).
Now,
i= 1 2 3 4 5 6 … k
s= 1 3 6 10 15 21 … n
From the table above, we get the following equation,

1 + 2 + 3 + 4 + ⋯+ 𝑘 = 𝑛
𝑘(𝑘+1)
Or, =𝑛
2
1 1 (when, n is large, k is also
Or, 𝑘 2 + 𝑘 = 𝑛 large. In such case, we can
2 2 keep the most dominant term
Or, 𝑘 2 ≈ 2𝑛 𝒌𝟐 and ignore other smaller
terms and constants.)
Or, 𝑘 = √2√𝑛
Therefore, this algorithm is 𝑶(√𝒏).
Alg()
{
int i;
for (𝒊 = 1; 𝒊𝟐 < 𝒏; 𝒊++)
{
}
}
number of times the loop is executed.
Here, we can write the condition check 𝒊𝟐 < 𝒏 as 𝒊 < √𝒏.
So the loop will be executed √𝒏 times and this algorithm is 𝑶(√𝒏).
Alg()
{
int i, j, k;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊++){
for (𝒋 = 𝟏; 𝒋 ≤ 𝒏; 𝒋++){
for (𝒌 = 𝟏; 𝒌 ≤ 𝟏𝟎𝟎; 𝒌++){
}
}
}
}
Alg()
{
int i, j, k;
for (𝒊 = 1; 𝒊 ≤ 𝒏; 𝒊++){
for (𝒋 = 𝟏; 𝒋 ≤ 𝒊; 𝒋++){
for (𝒌 = 𝟏; 𝒌 ≤ 𝟏𝟎𝟎; 𝒌++){
}
}
}
}
number of times the innermost loop (loop k) is executed.
Here,
When i = 1 2 3 4 … n
Loop j is
1 times 2 times 3 times 4 times … n times
executed
Loop k is 1 x 100 2 x 100 3 x 100 4 x 100 n x 100
…
executed times times times times times
From the table above, we get,

Running time = (1 × 100) + (2 × 100) + (3 × 100) + … + (𝑛 × 100)
= 100 × 1 + 2 + 3 + ⋯ + 𝑛
𝑛(𝑛+1)
= 100 ×
2
= 50𝑛2 + 50𝑛
= 𝑶 𝒏𝟐
Therefore, this algorithm is 𝑶 𝒏𝟐
Related Materials
➢ Time Complexity Analysis Of Iterative Programs:
https://www.youtube.com/watch?v=FEnwM-iDb2g
➢ Data Structures & Algorithm Analysis in C (Mark A. Weiss)

• Chapter 2
➢ Introduction to Algorithms (CLRS)

• Chapter 2
• Asymptotic Notations:
https://drive.google.com/file/d/1PaF_NqJeqykBVT63RcSJRtm_2iUFpzMB/view
?usp=sharing

Lec - 2 - Analysis of Algorithms (Part 2)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lec - 2 - Analysis of Algorithms (Part 2)

Uploaded by

Copyright:

Available Formats

Lecture 2

• f(n) = O(g(n)) that is, f of n is Big–O of g

• It means that for large amounts of data, f(n)

• Here we have 𝑓 𝑛 = 30n+8, and 𝑔 𝑛 = 𝑛

• Here we have 𝑓 𝑛 = 3𝑛 + 2, and 𝑔 𝑛 = 𝑛

• We notice that when c = 4, we have 3𝑛 + 2 ≤ 4𝑛 for all 𝑛 ≥ 2.

• Here we have 𝑓 𝑛 = 𝑛2 + 𝑛, and 𝑔 𝑛 = 𝑛3

• Here we have 𝑓 𝑛 = 3𝑛 + 2, and 𝑔 𝑛 = 𝑛

• Here we have 𝑓 𝑛 = n2 + 5n + 7, and 𝑔 𝑛 = n2

• Log-linear : O(n logn)

• Polynomial : O(𝒏𝒌 ) where k > 1

• Exponential : O(𝒌𝒏 ) where k > 1

• Similarly, this property satisfies both Θ and Ω notation. We can say

• Similarly, this property satisfies both Θ and Ω notation. We can say

• Example: if f(n) = n , g(n) = n² and h(n)=n³

• Example: f(n) = n² and g(n) = n² then f(n) = Θ(n²) and g(n) =

• This property only satisfies for Θ notation.

Transpose Symmetric Properties :

• Example: f(n) = n , g(n) = n² then n is O(n²) and n² is Ω (n)

• This property only satisfies for O and Ω notations.

Example: f(n) = n i.e O(n)

3. If f(n)=O(g(n)) and d(n)=O(e(n))

Example: f(n) = n i.e O(n)

for( i=0; i<n; i++ )

for( i=0; i<n; i++)

• the running time of an if/else statement is never more than the

✓ The worst-case running time of an algorithm is an upper

✓ Therefore, having the knowledge of worst-case running time

✓ It specifies the expected behaviour of the algorithm when the

✓ Average-case running time assumes that all inputs of a given

✓ However, while developing and choosing an algorithm to solve a

✓ It is always recommended to improve the average performance and

➢ Fundamentals of Computer Algorithms (Horowitz, Sahni, Rajasekaran)

Which of these claims are correct ?

(A) 1 and 2 (D) g1 n is O(n3 )

➢ Analyze the following algorithms.

➢ Find out the order of growth/time complexity/big-O of the following

Since the loop will be executed n times, this algorithm is 𝑶(𝒏).

From the table above, we get the following equation,

Therefore, this algorithm is 𝑶(√𝒏).

Here, we can write the condition check 𝒊𝟐 < 𝒏 as 𝒊 < √𝒏.

So the loop will be executed √𝒏 times and this algorithm is 𝑶(√𝒏).

From the table above, we get,

Therefore, this algorithm is 𝑶 𝒏𝟐

➢ Data Structures & Algorithm Analysis in C (Mark A. Weiss)

➢ Introduction to Algorithms (CLRS)

You might also like