ALGORITHMIC

ANALYSIS
ASYMPTOTIC GROWTH FUNCTIONS
Agenda: How to compare two algorithms?
• Complexity measures
• Time
• Time is more important than space constraint unless space increase is exponential.
• Space
Machine
• Only taking CPU time of execution is not feasible because: dependence
• CPU execution time changes with type of machine, instruction set etc.
• Same program on same computer can have different times of execution due
to different size of inputs.
Input
dependence
Approximations to be made
• For every unit of computation give a fixed unit time.
• Eg: Maximum addition time for 10 bit numbers is x units. Based on this find
how many units of time does a function take.
• For different sizes of inputs, measure the computation time.
• Eg: Size of a input takes 20 units of time. Size of b input takes 30 units of time.
Measuring time complexity (in terms of
steps) as a function of input size
• Example of time complexity function f(n) = 12 n3 + 15 n logn – 5 n logn
n is a measure
of the input
size

• Eg: For finding the max. of 4 numbers, f (n) = 5 n – 3.

n=4

• Why not take all 4 numbers as a string and sort? eg: 5#2#3#12
• In such a case the number of inputs is 1.
• Does the time complexity reduce?
Concept of Input Size
• In the previous case, the complexity of the sorting algorithm would be
measured proportional to the length of the input.

• For a computation x = a + b, if we assume one step of time is taken,

then the underlying assumption is that sizes of a, b and x are bound
by a fixed constant.

• Therefore, an addition of 2 numbers of 1 million size will not have

the same time complexity as addition of any other two numbers.
Concept of Input Size
• Therefore, addition of two arbitrary size numbers is not of constant
time, but a function of maximum constant size of the numbers.
• Eg: Add 12345678 and 23456789 on a 32 bit CPU.

Carry
12345678 For n such
operations,
+ 23456789 f(n) = 2(n – 1) + 1
Concept of Range of Size
• Eg:
• If input numbers vary from 10 to 20 then the range of size of input is known.
• If it is known that numbers from 1 to 100 need to be sorted, then the range of
the maximum input size is known.
• Therefore, maximum complexity of the algorithm can be predetermined.
• i.e. for adding 100 nos. if complexity f(n) = 2n - 1 for 1 ≤ n ≤ 100, then f(n) is a
constant for maximum limit of 100. The algorithm will not take more time
than the maximum limit for 100 numbers.
• However, the analysis needs to be made for the range within 1 ≤ n ≤
100.
Worst case analysis for all ‘n’ inputs
• Consider f(n) = 305 n2 + 3n + 2
g(n) = n3 + 8n2 + 8n + 9
h(n) = 3000 n logn + 9
Take n = 1 to 3 and find which function yields the least value.
What if n tends to a very large value??

Thus, depending on the range of inputs, the complexity becomes

important.
Worst case analysis for all ‘n’ inputs
• Also, the constants of each function f(n), g(n), h(n) signify
approximated values for various processes of machines and are
machine dependent.

• Therefore, all algorithms for specific orders need to be compared only

when n  large value or infinity ideally.

• This is termed as Asymptotic Complexity Analysis

 Asymptotic Complexity Analysis
g(n)

f(n) After n> n0, f(n) ≤ g(n) asymptotically.

Both functions are always positively growing

function(n)

functions.

n0 n
f( n ) = n3 + 1999n + 1337 gives f( n ) = n3
Asymptotic Growth Functions
• Three types:
• Upper bounding function (O)
• Lower bounding function (Ω)
• Order function (Θ)
The O(f(n)) notation (Asymptotic upper
bounding)
A function g(n) = O(f(n)) if there are constants c and n0, both greater
than 0 such that,
0 ≤ g(n) ≤ cf(n) for all n ≥ n0

i.e. given g(n) we try to find another function that upper bounds it in a
certain range.
O(f(n))
Let g(n) = 502n2+ 3n – 7
f(n) = 715 n2 + 100n + 10
Find c and n0 such that for all n ≥ n0, 0 ≤ g(n) ≤ cf(n)

Solution:
Take c = 1; n0 = 2…, the condition 0 ≤ g(n) ≤ cf(n) would get satisfied.

Let f(n) = n3 + 200n2 – 5 or take f(n) = 0.3 n3 + 200n2 – 5 ;

Even then, for some value of c and n0 the condition will be satisfied
O(f(n))
• Implication is that constants of the equations in f(n) and g(n)
correspond to variables of the machine on which the algorithms are
tested.
• However, if the function is asymptotically upper bounded, it remains
the same across all machines.

• Therefore, in the previous case, g(n) = O(n3) [Read as g(n) is upper

bounded by f(n) with an order of n3]
O(f(n))
Let g(n) = 502 n2 + 3n – 7
f(n) = n2 – 7n – 15
Is g(n) = O(f(n))?

Answer: Yes..
Therefore, g(n) = O(n2)

Hence, upper bounding functions give a measure of maximum bounds of

a function
The Ω(f(n)) notation (Asymptotic lower
bounding)
A function g(n) = Ω(f(n)) if there are constants c and n0, both greater
than 0 such that,
0 ≤ cf(n) ≤ g(n) for all n ≥ n0

i.e. given g(n) we try to find another function that lower bounds it in a
certain range.

Alternatively, if g(n) upper bounds f(n), then the vice versa is true.
The Θ(f(n)) notation (Order function)
A function g(n) = Θ(f(n)) if there are constants c1, c2 and n0, all greater
than 0 such that,
0 ≤ c1f(n) ≤ g(n) ≤ c2f(n) for all n ≥ n0

Eg: Let f(n) = 70 n2 – 3n + 5; g(n) = n2

Then, f(n) = O(n2) = Ω(n2) and both functions are called asymptotically
equivalent.
TIME COMPLEXITY OF SIMPLE LOOPS
THE FOR LOOP

for (i = 0; i < n; i++) Runs (n) times

{ Degree of time complexity polynomial inside
loop is 1.
statements; Runs (n) times Thus, time complexity is O(n1) i.e. O(n)

for (i = n; i > 0; i - -) Runs (n) times

{ O(n)
statements; Runs (n) times

}
THE FOR LOOP
Runs (n) times
for (i = 0; i < n; i=i+2)
O(n)
{
statements; Runs (n/2) times

Runs (n) times

for (i = 0; i < n; i=i++)
{
Runs (n x n) times O(n2)
for(j=0; j < n; j++)
{
Runs (n x n) times
statements;
}
}
THE FOR LOOP
Calculation:
for (i = 0; i < n; i=i++) i j No. of
{ executions
of
statements
for(j = 0; j < i; j++)
0 0 0
{
1 0 1
statements; 1
} 2 0 2
1
} 2
n 0….n n

Number of executions: (1 + 2 + 3 + …n) = n(n + 1)/2

= n2 + n / 2 = O(n2)
THE FOR LOOP Calculation:
i p
1 0+1
p = 0; 2 0+1+2
for(i = 1; p <= n; i++) 3 0+1+2+3
k 0 + 1 + 2 + 3+..k
{
p = p+i;
} Loop runs for k times until p > n

Number of executions: k(k + 1)/2 > n

This implies; k2 + k / 2 > n
This implies; k2 > n
Therefore, its O
THE FOR LOOP
Calculation:
for(i = 1; i < n; i = i*2) i

{ 1
1*2 = 2
statements; 2*2 = 22
} 22*2 = 23
2k

Assume: i >= n
As i = 2k
2k >= n
k=O
THE FOR LOOP

for(i = n; i >= 1; i = i/2)

{
statements;
}
FIND THE TIME COMPLEXITY OF THE LOOP
THE FOR LOOP

for(i = 0; i*i < n; i ++)

{
statements;
}
FIND THE TIME COMPLEXITY OF THE LOOP
THE FOR LOOP
p = 0;
for(i = 1; i < n; i = i*2)
{ p = log n
p++;
}
for(j = 1; j < p; j = j*2)
{
log p = O(log(log n))
statements
}
THE FOR LOOP
for(i = 0; i < n; i ++) n
{
for(j = 1; j < n; j=j*2) n n logn

{
statement;
}
}

Final order O(n log n)

SUMMARY
for(i = 0; i < n; i ++)…………O(n)
for(i = 0; i < n; i +2)………….O(n)
for(i = n; i > 1; i --)……………O(n)
for(i = 0; i < n; i = i*2)…………O
for(i = 0; i < n; i = i*3)…………O
for(i = 0; i < n; i = i/2)…………O
PYTHON CODING
 ANAGRAM DETECTION EXAMPLE
One string is an anagram of another if the second is simply a rearrangement of the first. For example, 'heart' and 'earth' are anagrams.
SOLUTION 1:
O(N^2)
SOLUTION 2:
O(N^2) OR
O(N LOG N)
SOLUTION 3:
O(N)