CS 148 – Designs & Analysis of Algorithm

ASYMPTOTIC ANALYSIS

 The purpose of asymptotic analysis is to

examine the behavior of an algorithm for
large input size.
 If T(n) is the running time for an input of
size n, we would want to know the
behavior or growth rate of T(n) for very
large values of n.

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC ANALYSIS

 The asymptotic behavior of an algorithm is

often compared to some standard
mathematical function, such as n2, n lg n etc.
 The relationship or similarity of behavior is
often expressed by a special notation
which is called asymptotic notation.

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC NOTATIONS

 Ο (Big Oh)
 Ω (Big Omega)
 Θ (Theta)

CS 148 – Designs & Analysis of Algorithm

 O-NOTATION: DEFINITION

 Big O is a characterization scheme that

allows to measure properties of algorithms
such as time and space complexity.
 It is useful to set the prerequisites of
algorithms and to develop and design
efficient algorithms in terms of time and
space complexity.
 O-NOTATION: DEFINITION

 If f(n) is running time of an

algorithm, and g(n) is some
standard growth function such
that for some positive constants
c and integer n0,

f(n) ≤ c · g(n) for all n ≥ n0

then f(n) = Ο(g(n))

 It follows that for n< n0 f(n) may

lie above or below g(n), but for
all n ≥ n0, f(n) falls consistently
below g(n).

CS 148 – Designs & Analysis of Algorithm

 O-NOTATION: DEFINITION

What does it mean?

If f(n) = O(n2), then:
- f(n) can be larger than n2 sometimes, but…
- I can choose some constant c and some value n0 such
that for every value of n larger than n0 : f(n) < cn2
- That is, for values larger than n0, f(n) is never more than
a constant multiplier greater than n2
- Or, in other words, f(n) does not grow more than a
constant factor faster than n2.
 O-NOTATION: ASYMPTOTIC UPPER BOUND

 If f(n) = Ο(g(n)), then the

function g(n) is called
asymptotic upper bound of f(n).
 Since the worst-case running
time of an algorithm is the
maximum running time for any
input, it would follow that g(n)
provides an upper bound on
the worst running time.
 The notation Ο(g(n)) does not
imply that g(n) is the worst
running time; it simply means
that worst running time would
never exceed upper limit
determined by g(n).

CS 148 – Designs & Analysis of Algorithm

 O-NOTATION
 The following functions are often encountered in
computer science Big Oh analysis:
 T(n) = O(1)
 This is called constant growth.

 T(n) does not grow at all as a function of n, it is a constant.

 It is pronounced "Big Oh of one."

 Ex. array access has this characteristic. A[i] takes the same

time independent of the size of the array A.

 T(n) = O(lg(n)).
 This is called logarithmic growth.

 T(n) grows proportional to the base 2 logarithm of n.

 It is pronounced "Big Oh of log n."

 Ex. binary search

 O-NOTATION
 T(n) = O(n)
 This is called linear growth.

 T(n) grows linearly with n.

 It is pronounced "Big Oh of n."

 Ex. looping over all the elements in a one-dimensional array

would be an O(n) operation.

 T(n) = O(n log n).
 This is called "n log n" growth.

 T(n) grows proportional to n times the base 2 logarithm of n.

 It is pronounced "Big Oh of n log n."

 Ex. Merge Sort has this characteristic. In fact, no sorting

algorithm that uses comparison between elements can be

faster than n log n.
 O-NOTATION

 T(n) = O(nk).
 This is called polynomial growth.

 T(n) grows proportional to the k-th power of n.

 We rarely consider algorithms that run in time O(n ) where k is

k

greater than 5, because such algorithms are very slow.

 Ex. selection sort is an O(n ) algorithm.
2

 T(n) = O(2n)
 This is called exponential growth.

 T(n) grows exponentially.

 It is pronounced "Big Oh of 2 to the n."

 Exponential growth is the most-feared growth pattern in

computer science; algorithms that grow this way are basically

useless for anything but very small problems.
 O-NOTATION
 The growth patterns in the order of increasing "size”
Notation Name
O(1) constant
O(log log n) double logarithmic
O(log n) logarithmic
O(logkn), k > 1 polylogarithmic
O(nk), 0 < k < 1 fractional power
O(n) linear
O(n log * n) n log-star n
O(n log n) = O(log n!) linearithmic, loglinear or
quasilinear
O(n2) quadratic
O(nk), k > 1 polynomial or algebraic
O(kn), k > 1 exponential
O(n!) factorial
O(n  n!) n x n factorial
 O-NOTATION: BASIC METHOD

Example 1:
Using basic definition, we show that 3n2 + 10n = Ο(n2)
Consider, 10 ≤ n for n ≥ 10
10n ≤ n2 for n ≥ 10
3n2 + 10n ≤ 3n2 + n2 for n ≥ 10
= 4n2
3n2 + 10n ≤ c·n2 for n ≥ n0 where c=4 and n0 = 10
Therefore, it follows from the basic definition that
3n2 + 10n = Ο(n2)

CS 148 – Designs & Analysis of Algorithm

 O-NOTATION: BASIC METHOD
Example 2:
In the preceding example it was shown that
3n2 + 10n ≤ c · n2 for c=4, n0 = 10.
We now show that the relation holds true for a different
value of c and corresponding n0.
Consider n ≤ n2 for n ≥ 1
10n ≤ 10n2 for n ≥ 1
3n2 + 10n ≤ 3n2 + 10n2 for n ≥ 1
3n2 + 10n ≤ 13n2 for n ≥ 1
Or, 3n2 + 10n ≤ c · n2, for n ≥ n0 where c=13, n0 =1
Therefore, by basic definition,
3n2 + 10n = Ο(n2)

CS 148 – Designs & Analysis of Algorithm

 O-NOTATION: COMPARISON OF GROWTH RATES

 It can be seen that the

function cg(n) = 4n2
(c=4) overshoots the
function f(n) = 3n2 + 10n
for n0 = 10.
 The function cg(n) =
13n2 ( c = 13 ) grows
faster than f(n) = 3n2 +
10n for n0 = 1

CS 148 – Designs & Analysis of Algorithm

 O-NOTATION: BASIC METHOD
Example 3:
Show : f(n) = 2n7 – 6n5 + 10n2 – 5 = O(n7)

Solution:
f(n) < 2n7 – 6n5 + 10n2
≤ 2n7 – 6n7 + 10n7
= 18n7
thus, with c=18 we have shown that f(n) = O(n7)

- Any polynomial is big-Oh of its term of highest

degree.
- We are also ignoring constants.

CS 148 – Designs & Analysis of Algorithm

 O-NOTATION: SET BUILDER
 Consider the functions, say f1(n), f2(n), f3(n),…, fk(n) for which g(n) is the
asymptotic upper bound. By definition,
f1(n) ≤ c1 g(n) for n ≥ n1
f2(n) ≤ c2 g(n) for n ≥ n2
f3(n) ≤ c3 g(n) for n ≥ n3
………………………….
fk(n) ≤ ck g(n) for n ≥ nk
where c1, c2, c3,…, ck are constants and n1, n2, n3,…, nk are positive
integers. The functions f1(n), f2(n) , f3(n) ,…, fk(n) are said to belong to
the class O(g(n)). In set notation, the relation is denoted by
O(g(n)) = {f1(n), f2(n) , f3(n) ,…, fk(n)}
 Alternatively, using set-builder notation, if
O(g(n)) = {f(n): there exist positive constant c and nk such that
f(n) ≤ c · g(n), for all n ≥ n0 }
then,
f(n) є O(g(n))

CS 148 – Designs & Analysis of Algorithm

 Ω -NOTATION : DEFINITION

If f(n) is running time of an

algorithm, and g(n) is some
standard growth function
such that for some positive
constants c, positive
integer n0,

c · g(n) ≤ f(n) for all n ≥ n0

then f(n) = Ω(g(n))

CS 148 – Designs & Analysis of Algorithm

 Ω -NOTATION: ASYMPTOTIC LOWER BOUND

 If f(n) = Ω(g(n)), then the function

g(n) is called asymptotic lower
bound of f(n).
 Since the best-case running time
of an algorithm is the minimum
running time for any input, it
would follow that g(n) provides a
lower bound on best running
time.
 The notation Ω(g(n)) does not
imply that g(n) is the best
running time; it simply means
that best running time would
never be lower than g(n).

CS 148 – Designs & Analysis of Algorithm

 Ω -NOTATION: BASIC METHOD

Example 1:
Using basic definition, we show that n2 - 10n = Ω(n2)
For, n ≥ n / 2 for n ≥ 0
n – 10 ≥ n / (2 x 10) for n ≥ 10
= n / 20
n2 - 10n ≥ n2 / 20 for n ≥ 10
n2 - 10n ≥ c·n2 for n ≥ n0 where c=1 / 20 and n0 = 10
Therefore, by basic definition,
n2 - 10n = Ω(n2).

CS 148 – Designs & Analysis of Algorithm

 Ω -NOTATION: COMPARISON OF GROWTH RATES

Example 1

For n > 10, the

function cg(n) = n2 / 20
(c = 1/20) consistently
fall below the function
f(n) = n2 – 10n.

CS 148 – Designs & Analysis of Algorithm

 Ω -NOTATION: BASIC METHOD
Example 2:

3n2 - 25n = Ω(n2)

For, n ≥ n / 2 for n ≥ 0
n – 25/3 ≥ 3n / (2 x 25) for n ≥ 9
= 3n / 50 for n ≥ 9
3n2 - 25n ≥ 9n2 / 50 for n ≥ 9
3n2 - 25n ≥ c·n2 for n ≥ n0 where c=9 / 50 and n0 = 9
Therefore, by definition,
3n2 - 25n = Ω(n2).

CS 148 – Designs & Analysis of Algorithm

 Ω -NOTATION: COMPARISON OF GROWTH RATES

Example 2

For n ≥ 9, the
function 9n2 / 50
falls below the
function 3n2 - 25n.

CS 148 – Designs & Analysis of Algorithm

 Ω -NOTATION: SET BUILDER
Consider the functions, say f1(n), f2(n), f3(n),…, fk(n) for which g(n) is
the asymptotic lower bound. By definition,
f1(n) ≥ c1 g(n) for n ≥ n1
f2(n) ≥ c2 g(n) for n ≥ n2
f3(n) ≥ c3 g(n) for n ≥ n3
………………………….
fk(n) ≥ ck g(n) for n ≥ nk
where c1, c2, c3,…, ck are constants and n1, n2, n3,…, nk are positive
integers. The functions f1(n), f2(n) , f3(n) ,…, fk(n) are said to belong
to the class Ω(g(n)). In set notation, the relation is denoted by
Ω(g(n)) = {f1(n), f2(n) , f3(n) ,…, fk(n)}
Alternatively, using set-builder notation, if
Ω(g(n)) = {f(n): there exist positive constant c and nk such that
f(n) ≤ c · g(n), for all n ≥ n0 }
then,
f(n) є Ω(g(n))

CS 148 – Designs & Analysis of Algorithm

 Θ -NOTATION : DEFINITION

If f(n) is running time of an

algorithm, and g(n) is some
standard growth function such
that for some positive
constants c1, c2 and positive
integer n0,

0 < c2 · g(n) ≤ f(n) ≤ c1 · g(n)

for all n ≥ n0

then f(n) = θ(g(n))

CS 148 – Designs & Analysis of Algorithm

 Θ -NOTATION : SET BUILDER

 There can be a several functions for which the g(n) is

asymptotic tight bound. All such functions are said to
belong to the class θ(g(n)).
 Using set builder notation, if
θ(g(n)) = {f(n): there exist positive constants c1, c2
and positive integer n0 such that
c1g(n) ≤ f(n) ≤ c2g(n)
then,
f(n) є θ(g(n))

CS 148 – Designs & Analysis of Algorithm

 Θ-NOTATION: BASIC METHOD
Example:
We show that 5n2 - 19n = θ(n2)
First consider the upper bound. As explained before,
5n2 - 19n ≤ 5n2 for n ≥ 0
5n2 - 19n ≤ c1·n2 for n ≥ n1, where c1 = 5 and n1 = 0 ………(1)
Next, consider the lower bound,
n ≥ n / 2 for n ≥ 0
n-19/5 ≥ 5n / (2 x 19) for n ≥ 4
= 5n / 38 for n ≥ 4
5n2 - 19n ≥ 25n2 / 38 for n ≥ 4
5n2 - 19n ≥ c2·n2 for n ≥ n2 where c2 = 25 / 38 and n2 = 4 ……(2)
From (1) and (2) it follows,
0 < c2 · n2 ≤ 5n2 - 19n ≤ c1 · n2 for n ≥ n0, where n0=4, c1=5 and
c2=25/38.

Therefore, 5n2 - 19n = θ(n2).

CS 148 – Designs & Analysis of Algorithm

 Θ -NOTATION: COMPARISON OF GROWTH RATES

The graph shows

the asymptotic
upper and lower
bounds for the
function
f(n) = 5n2 - 19n

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC NOTATION CONSTANT RUNNING TIME
If running time T(n)=c is a constant, i.e. independent of input
size, then by convention, the asymptotic behavior is denoted by
the notation
Ο(c) = Ο(1), θ(c) = θ(1), Ω(c) = Ω(1)
 If c is a constant then using basic definition it can be shown that
Ο( c·f(n) ) = Ο( f(n) )
θ( c·f(n) ) = θ( f(n) )
Ω( c·f(n) ) = Ω( f(n) )
 Example:
(i) Ο( 1000n ) = Ο( n ),
(ii) θ( 7lgn ) = θ ( lg n ),
(iii) Ω( 100 n! ) = Ω( n! )

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC NOTATION RELATIONSHIPS
 Theorem: If f(n)= θ(g(n)) then
f(n)= Ω(g(n)), and f(n)= Ο(g(n))
Conversely, if f(n)= Ω(g(n)), and f(n)= Ο(g(n)) then f(n)= θ(g(n))
 Example
1. Since, n(n-1)/2 = θ(n2), therefore it follows that
n(n-1)/2 = Ω(n2)
n(n-1)/2 = Ο(n2)
2. It can be shown that
5n2 + 1 = Ω(n2)
and 5n2 + 1 = Ο(n2)
Therefore, 5n2 + 1 = θ(n2)

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC SET NOTATIONS RELATIONSHIP

 if f(n) є Ο( f(n) ) and f(n) є Ω( f(n) ) then f(n) є θ( f(n) )

where θ(g(n)) = Ο(g(n)) ∩ Ω(g(n))
 Venn diagram

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC SET NOTATIONS EXAMPLE

 The relationship among the Ο, Ω, and θ notations can

be expressed using set notation.

 It follows that θ(g(n)) = Ο(g(n)) ∩ Ω(g(n))

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC SET NOTATIONS EXAMPLE

Venn diagram:

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC SET NOTATIONS ORDER THEOREM

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC SET NOTATIONS ORDER THEOREM

CS 148 – Designs & Analysis of Algorithm

 ASYMPTOTIC SET NOTATIONS ORDER THEOREM

CS 148 – Designs & Analysis of Algorithm

 Ο-NOTATION AND Ω-NOTATION

o-notation and ω-notation are like < and >.

ο(g(n))= { f(n) : for any constant c > 0, there
is a constant n0 > 0 such that 0 ≤ f(n) < cg(n)
for all n ≥ n0 }
 Ex. 2n2= o(n3) (n0= 2/c)

ω(g(n))= { f(n) : for any constant c > 0, there

is a constant n0 > 0 such that 0 ≤ cg(n) < f(n)
for all n ≥ n0 }
 √n = ω(lg n) (n0= 1+1/c)

CS 148 – Designs & Analysis of Algorithm

 ANALOGY TO ARITHMETIC OPERATORS

f n   O g n   ab
f n    g n   ab
f n    g n   ab
f n   o g n   ab
f n     g n   ab

CS 148 – Designs & Analysis of Algorithm

VISUALIZATION OF ASYMPTOTIC GROWTH
o(f(n))
O(f(n))

(f(n))
f(n)

(f(n))

(f(n))

n0
 logarithms:
 logb(xy) = logbx + logby
 logb (x/y) = logbx - logby
 logbxa = alogbx
 logba = logxa/logxb

 exponentials:
 a(b+c) = abac
 abc = (ab)c
 ab /ac = a(b-c)
 www.eshikshak.co.in