Dr.

Tarun Biswas
Dept. of Computer Science & Engineering
IIIT Ranchi
➢ The topic “Analysis of Algorithms” is primarily concerned with
determining the time (complexity) and memory (space)
requirements of an algorithm.

➢ The time complexity (or simply, complexity) of an algorithm is

measured as a function of the problem size, e.g.,
▪ The complexity of an algorithm to sort n elements may be
given as a function of n.
▪ The complexity of an algorithm to multiply an m×n matrix
and an n×p matrix may be given as a function of m, n, and p.

➢Way to estimate the time complexity:

▪ Operation counts.
▪ Step counts.
2
➢ Operation counts:
▪ Example [Max Element]: An algorithm that returns the position of the
largest element in the array a[0:n-1].

int max(int [] a, int n)

{
if (n < 1) return -1; // no max
int positionOfCurrentMax = 0;
for (int i = 1; i < n; i++)
if (a[positionOfCurrentMax] < a[i]) positionOfCurrentMax = i;
return positionOfCurrentMax;
}

▪ When n ≤ 1, the for loop is not entered. So no comparisons between

elements of a are made.
▪ When n > 0, the time complexity can be estimated by determining the
number of comparisons made between elements of the array a.

3
➢ Operation counts:
▪ When n > 1, each iteration of the for loop makes one
comparison between two elements of a, and the total number
of element comparisons is n-1.
▪ Therefore, the number of element comparisons is max{n-1, 0}.

▪ Other comparisons (for example, each iteration of the for loop

is preceded by a comparison between i and n) are not included
in the estimate.
▪ Initializing positionOfCurrentMax and incrementing the for
loop index i are also not included in the estimate.

▪ The algorithm has the nice property that the operation count is
precisely determined by the problem size.
4
➢ Operation counts:
▪ Example [Sequential Search]: An algorithm that searches a[0:n-1] for the
first occurrence of x.

int sequentialSearch(int [] a, int n, int x)

{
int i;
for (i = 0; i < n && x != a[i]; i++);

if (i == n) return -1; // not found

else return i; // found at i
}

▪ The number of comparisons between x and the elements of a is not

uniquely determined by the problem size n.
▪ For example, if n = 100 and x = a[0], then only 1 comparison is made.
▪ However, if x is not equal to any of the a[i]’s, then 100 comparisons are
made.
5
➢ Operation counts:
▪ A search is successful when x is one of the a[i]s. All other searches
are unsuccessful.

▪ For unsuccessful search, the number of comparisons is n.

▪ For successful searches, the best comparison count is 1, and the

worst is n.

▪ For the average count, assume that all array elements are distinct
and that each is searched for with equal frequency. The average
count for a successful search is

6
➢ Step counts:
▪ We attempt to account for the time spent in all parts of the
algorithm. As was the case for operation counts, the step count is a
function of the problem size.
▪ A step is any computation unit that is independent of the problem
size. Thus 10 additions can be one step; 100 multiplications can
also be one step; but n additions, where n is the problem size,
cannot be one step.
▪ To determine the step count of an algorithm, we first determine
1. the number of steps per execution (s/e) of each statement and
2. the total number of times (i.e., frequency) each statement is
executed.
▪ Combining above these two quantities gives us the total
contribution of each statement to the total step count. We then add
the contributions of all statements to obtain the step count for the
entire algorithm.
7
➢ Step counts:
Best-case Step Counts (when x = a[0]):

Worst-case Step Counts (when x is not in the list):

8
➢ Step counts:
Step Counts when x =a [j]:

▪ We assume that the n values in a are distinct and that in a successful

search, x has an equal probability of being any one of these values.
▪ Under these assumptions the average step count for a successful search
is the sum of the step counts for the n possible successful searches
divided by n.

9
➢ Step counts:
Step Counts when x =a [j]:

▪ To obtain this average, we first obtain the step count for the case
x = a[j] where j is in the range [0, n − 1]. Now we obtain the average
step count for a successful search:

10
➢ Step counts:
• Now suppose that successful searches occur only 80 percent of the
time and that each a[i] still has the same probability of being
searched for.
• The average step count for sequentialSearch is

.8 ∗ (average count for successful searches) + .2 ∗ (count for an

unsuccessful search)
= .8(n + 7)/2 + .2(n + 3)
= .6n + 3.4

11
➢ When the input sizes is large enough, only the order of growth
of the running time is relevant, i.e., asymptotic efficiency of
algorithms. Asymptotic means approaching a value or curve
arbitrarily closely.

➢ We are concerned with how the running time of an algorithm

increases with the size of the input in the limit, as the size of
the input increases without bound.

➢ Usually, an algorithm that is asymptotically more efficient

will be the best choice for all but very small inputs.

➢ Lets start by defining several types of “asymptotic notation.”

12
Big-Oh Notation:
O(g(n)) = {f(n) :  positive
constants c and n0, such that
n  n0, we have 0  f(n)  cg(n) }

➢ g(n) is an asymptotically upper

bound for f(n).

➢ Example: f(n) =7n-2 is in O(n).

▪ Need c > 0 and n0  1 such that 7n-2  c×n for n  n0.
▪ This is true for c = 7 and n0 = 1 and g(n) = n.

13
 Big-Oh Notation: O(g(n)) = {f(n) :  positive constants c and n0,
such that n  n0, we have 0  f(n)  cg(n) }

➢ Example: f(n) = 3n3 + 20n2 + 5 is in O(n3).

▪ Need c > 0 and n0  1 such that 3n3 + 20n2 + 5  c × n3 for n  n0
▪ This is true for c = 4 and n0 = 21.
➢ Example: f(n) = 3logn + 5 is in O(log n).
▪ Need c > 0 and n0  1 such that 3 log n + 5  c × log n for n  n0
▪ This is true for c = 8 and n0 = 2.

➢ Technically, we can also claim:

▪ f(n)=7n-2 is in O(n2) for c = 1 and n0 = 7 and g(n) = n2.
▪ f(n)=3n3 + 20n2 + 5 is in O(n4) for c =1 and n0=7 and g(n) = n4.

14
Big-Oh rule
➢ If is f(n) a polynomial of degree d, then f(n) is O(nd), i.e.,
1. Drop lower-order terms
2. Drop constant factors
➢ Take functions f(n) & g(n), consider only the most significant
term and remove constant multipliers:
 f(n) = 5n+3 → g(n) = n
 f(n) = 7n+.5n2+2000 → g(n) = n2
 f(n) = 300n+12+nlogn → g(n) = n log n

➢ Use the smallest possible class of functions

 Say “2n is in O(n)” instead of “2n is in O(n2)”
➢ Use the simplest expression of the class
 Say “3n + 5 is in O(n)” instead of “3n + 5 is in O(3n)”

15
Big-Oh and growth rate
➢ The Big-Oh notation gives an asymptotically upper bound on
the growth rate of a function.
➢ The statement “f(n) is O(g(n))” means that the growth rate of
f(n) is no more than the growth rate of g(n).
➢ We can use the Big-Oh notation to rank functions according
to their growth rate.

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more Yes No
f(n) grows more No Yes

16
Big-Oh Analysis Example:
The following algorithm computes prefix averages.
Algorithm prefixAverages1(X, n)
Input array X of n integers
Output array A of prefix averages of X #Steps
A  new array of n integers 1
for i  0 to n − 1 do n
s  X[0] n
for j  1 to i do 1 + 2 + …+ (n − 1)
s  s + X[j] 1 + 2 + …+ (n − 1)
A[i]  s / (i + 1) n
return A 1
Total steps = (n2 + 2n +2)

Thus, the algorithm prefixAverages1 runs in O(n2) time.

17
Big-Oh Analysis Example:
The following algorithm computes prefix averages.
Algorithm prefixAverages2(X, n)
Input array X of n integers
Output array A of prefix averages of X #Steps
A  new array of n integers 1
s0 1
for i  0 to n − 1 do n
s  s + X[i] n
A[i]  s / (i + 1) n
return A 1
Total steps = (3n +3)

Thus, the algorithm prefixAverages2 runs in O(n) time.

18
Big-Omega Notation:
(g(n)) = {f(n) :  positive constants
c and n0, such that n  n0, we have
0  cg(n)  f(n)}
➢ g(n) is an asymptotically lower
bound for f(n).

➢Example: f(n) = 3n3 + 20n2 + 5 is

in (n3).
▪ Need c > 0 and n0  1 such that, c×n3  3n3 + 20n2 + 5 for n  n0.
▪ This is true for c = 1 and n0 = 1 and g(n) = n3.

➢ Example: f(n) = 3n3 + 20n2 + 5 is also in (n2) and (n).

19
Big-Omega Notation:  (g(n)) = {f(n) :  positive constants c
and n0, such that n  n0, we have 0  cg(n)  f(n)}

When we say that the running time of an algorithm is (g(n)), we

mean that no matter what particular input of size n is chosen for
each value of n, the running time on that input is at least a
constant times g(n), for sufficiently large n.

20
Theta Notation:
(g(n)) = {f(n) :  positive constants
c1, c2, and n0, such that n  n0, we
have 0  c1g(n)  f(n)  c2g(n)}

➢ g(n) is an asymptotically tight

bound for f(n).
➢ f(n) and g(n) are nonnegative, for
large n.

➢For any two functions f(n) and g(n), we have f(n) = (g(n)) if
and only if f(n) = O(g(n)) and f(n) = (g(n)).

21
Theta Notation:
(g(n)) = {f(n) :  positive constants c1, c2, and n0, such that n 
n0, we have 0  c1g(n)  f(n)  c2g(n)}

➢Example: f(n) = 3n3 + 20n2 + 5 is in (n3).

▪ Need c1, c2 > 0 and n0  1 such that,
c1 × n3  3n3 + 20n2 + 5  c2 × n3 for n  n0.
▪ This is true for c1 = 1, c2 = 4 and n0 = 21 and g(n) = n3.

➢Therefore, f(n) = 3n3 + 20n2 + 5 is in O(n3) as well as (n3).

22
Small-Oh Notation:
o(g(n)) = {f(n):  c > 0,  n0 > 0 such that  n  n0, we have
0  f(n) < cg(n)}.

➢ Differences between Big-oh and small-oh?

▪ f(n) = O(g(n)), the bound 0  f(n)  cg(n) holds for some
constant c > 0, but
▪ in f(n) = o(g(n)), the bound 0  f(n) < cg(n) holds for all
constants c > 0.
➢ Therefore, f(n) becomes insignificant relative to g(n) as n
approaches infinity:
lim [f(n) / g(n)] = 0
n→

➢ g(n) is an upper bound for f(n) that is not asymptotically tight.

23
 Small-Omega Notation:
ω (g(n)) = {f(n):  c > 0,  n0 > 0 such that  n  n0, we have
0  cg(n) < f(n)}.

➢Differences between Big-omega and small-omega?

▪ f(n) = (g(n)), the bound 0  cg(n)  f(n) holds for some
constant c > 0, but
▪ in f(n) = ω(g(n)), the bound 0  cg(n) < f(n) holds for all
constants c > 0.
➢Therefore, f(n) becomes arbitrarily large relative to g(n) as n
approaches infinity:
lim [f(n) / g(n)] = 
n→

➢ g(n) is an lower bound for f(n) that is not asymptotically tight.

Dr. Pratyay Kuila, NIT Sikkim 24
➢ We say that f(n) is asymptotically smaller than g(n) if
f(n)= o(g(n)), and f(n) is asymptotically larger than g(n) if f(n)
= ω(g(n)).
➢ “The running time of algorithm A is at least O(n2),” is
meaningless.
➢ Technically, it is an abuse to say that the running time of an
algorithm is O(n2), since for a given n, the actual running time
varies.
➢ When we say “the running time is O(n2),” we mean that there is
a function f(n) that is O(n2) such that for any value of n, no
matter what particular input of size n is chosen, the running
time on that input is bounded from above by the value c.n2.
25
Comparison of Functions:

fg  ab
➢ f (n) = O(g(n))  a  b
➢ f (n) = (g(n))  a  b
➢ f (n) = (g(n))  a = b
➢ f (n) = o(g(n))  a < b
➢ f (n) = ω (g(n))  a > b

26
Limits
➢lim [f(n) / g(n)] = 0 ⇒ f(n) ∈ o(g(n))
n→

➢lim [f(n) / g(n)] <  ⇒ f(n) ∈ O(g(n))

n→

➢0 < lim [f(n) / g(n)] <  ⇒ f(n) ∈ (g(n))

n→

➢0 < lim [f(n) / g(n)] ⇒ f(n) ∈  (g(n))

n→

➢lim [f(n) / g(n)] =  ⇒ f(n) ∈ ω (g(n))

n→

➢lim [f(n) / g(n)] undefined ⇒ cannot say

n→

27
Properties:
Transitivity
f(n) = (g(n)) & g(n) = (h(n))  f(n) = (h(n))
f(n) = O(g(n)) & g(n) = O(h(n))  f(n) = O(h(n))
f(n) = (g(n)) & g(n) = (h(n))  f(n) = (h(n))
f(n) = o (g(n)) & g(n) = o (h(n))  f(n) = o (h(n))
f(n) = w(g(n)) & g(n) = ω(h(n))  f(n) = ω(h(n))

Reflexivity Symmetry
f(n) = (f(n)) f(n) = (g(n)) iff g(n) = (f(n))
f(n) = O(f(n))
f(n) = (f(n)) Complementarities
f(n) = O(g(n)) iff g(n) = (f(n))
f(n) = o(g(n)) iff g(n) = ω((f(n))

28
➢ What do you mean by n = O(n2)? How do we interpret such
formulas?

29
➢ What do you mean by n = O(n2)? How do we interpret such
formulas?
▪ When the asymptotic notation stands alone (that is, not
within a larger formula) on the right-hand side of an equation
(or inequality), as in n = O(n2), the equal sign to mean set
membership: n ∈ O(n2).

30
➢ What do you mean by n = O(n2)? How do we interpret such
formulas?
▪ When the asymptotic notation stands alone (that is, not
within a larger formula) on the right-hand side of an equation
(or inequality), as in n = O(n2), the equal sign to mean set
membership: n ∈ O(n2).

➢ What do you mean by the formula 2n2 + 3n + 1 = 2n2 + (n)?

How do we interpret such formulas?

31
➢ What do you mean by n = O(n2)? How do we interpret such
formulas?
▪ When the asymptotic notation stands alone (that is, not
within a larger formula) on the right-hand side of an equation
(or inequality), as in n = O(n2), the equal sign to mean set
membership: n ∈ O(n2).

➢ What do you mean by the formula 2n2 + 3n + 1 = 2n2 + (n)?

How do we interpret such formulas?
▪ The formula 2n2 + 3n + 1 = 2n2 + (n) means that 2n2 + 3n +
1 = 2n2 + f(n), where f(n) is some function in the set (n). In
this case, we let f(n) = 3n + 1, which indeed is in (n).

32
➢ What about the formula 2n2 + (n) = (n2)?

33
➢ What about the formula 2n2 + (n) = (n2)?
▪ No matter how the anonymous functions are chosen on
the left of the equal sign, there is a way to choose the
anonymous functions on the right of the equal sign to
make the equation valid.
▪ Thus, our example means that for any function
f(n)∈(n), there is some function g(n) ∈ (n2) such that
2n2 + f(n) = g(n) for all n. In other words, the right-hand
side of an equation provides a rough level of detail than
the left-hand side.

34
Q1: Let f(n) = 3n2 + 4n +5. Which of the following (a-f)
is/are correct?
(a) f(n) = Θ(n2)
(b) f(n) = O(n2)
(c) f(n) = O(n3)
(d) f(n) =  (n2)
(e) f(n) = Θ(n3)
(f) f(n) =  (n)

Q2: Which one of the following is not correct?

(a) f(n) = (g(n)) iff g(n) = (f(n))
(b) f(n) = O(g(n)) iff g(n) = (f(n))
(c) f(n) = o(g(n)) iff g(n) = ω((f(n))
(d) f(n) = O(g(n)) iff g(n) = (f(n))
35
Q1: Let f(n) = 3n2 + 4n +5. Which of the following (a-f)
is/are correct?
(a) f(n) = Θ(n2)
(b) f(n) = O(n2)
(c) f(n) = O(n3)
(d) f(n) =  (n2)
(e) f(n) = Θ(n3)
(f) f(n) =  (n)

Q2: Which one of the following is not correct?

(a) f(n) = (g(n)) iff g(n) = (f(n))
(b) f(n) = O(g(n)) iff g(n) = (f(n))
(c) f(n) = o(g(n)) iff g(n) = ω((f(n))
(d) f(n) = O(g(n)) iff g(n) = (f(n))
36
37
38
References:
1. Horowitz, Sartaj Sahni, S. Anderson-Freed, Fundamentals of Data
Structures in C, Universities Press, 2nd Ed.
2. T. H. Cormen, C. E. Leiserson, R. L. Rivest, Introduction to Algorithms,
Prentice hall, 2nd Ed.

39