Algorithms and Data

Structures I
Lecture 3:
Growth of functions;
Asymptotic notations: Θ, O, o, Ω, and ω.

A. M., IASA “Igor Sikorsky KPI”

 Worst-case and average-case
analysis
• Usually, only the worst-case running time is analyzed which is the
longest running time for any input of size n because:
– the worst-case running time of an algorithm gives us an upper bound on
the running time for any input;
– for some algorithms, the worst case occurs very often;
– the “average case” is often roughly as bad as the worst case.
• In some cases, average-case running time of an algorithm is
analyzed. The problem is to define “average” input for a particular
problem.
 Order of growth
• One more simplifying abstraction
– only the rate of growth, or order of growth, of the running time is
analyzed.
• We ignore constants and the lower order items.
• For insertion sort, when we only consider the factor of n2 from the
leading term.
• So, insertion sort has a worst-case running time of Θ(n2) (theta of n-
squared).
 Growth of Functions
• Comparing algorithms
– Using order of growth
– Exact running time is not usually worth the effort
– If inputs are large enough, the effects of the input size itself
dominates.
 Asymptotic notation
• For insertion sort:
– The worst-case running time
T(n) = an2+bn+c
T(n) ∈ Θ(n2)
(Θ applies to functions)

• Informally:
the order of growth of the fastest growing term
 Θ-notation
• For a given function g(n), we denote by Θ(g(n)) the set of
functions:
Θ(g(n)) = {f(n): there exist positive constants c1, c2, and n0 such that
0 ≤ c1 g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0}

• Example
g(n) = n, f(n) = 2n+5, c1 = 1, c2 = 3, n0 = 5
0 ≤ n ≤ 2n+5 ≤ 3n, for n ≥ n0

• Exercise
g(n) = n2, f(n) = 2n2+3n+5, find c1, c2, n0, so that:
0 ≤ c1n2 ≤ 2n2+3n+5 ≤ c2n2, for n ≥ n0.
Θ-notation (cont.)

(Cormen et. al., 2009)

 Θ-notation (cont.)
• A function f(n) belongs to the set g(n) if there exist positive constants
c1 and c2 such that it can be “sandwiched” between c1g(n) and
c2g(n), for sufficiently large n.
• Because g(n) is a set, we could write “f(n) ∈ Θ(g(n))” to indicate that
f(n) is a member of Θ(g(n)).
• Frequently, it can be written as “f(n) = Θ(g(n))” to express the same
notion.
 Θ-notation examples
• Let’s prove: 0.5n2 - 3n ∈ Θ(n2)
c1n2 ≤ 0.5n2 - 3n ≤ c2n2,
c1 ≤ 0.5 – 3/n ≤ c2 ,
c1 = 1/14, c2 = 1/2, and n0 = 7.

• We can prove that for f(n) = an2 + bn + c, where a ≠ 0, f(n) ∈

Θ(n2):
c1 = a/4, c2 = 7a/4, and n0 = 2 max(|b| / a, √(|c|/a)).

• For any polynomial p(n) of degree d, p(n) ∈ Θ(nd).

 O-notation
• O-notation is used for an asymptotic upper bound.
• O-notation gives an upper bound on a function, to within a constant
factor.

• For a given function g(n), we denote by O(g(n)) the set of functions:

O(g(n)) = {f(n): there exist positive constants c and n0 such that
0 ≤ f(n) ≤ c g(n) for all n ≥ n0}

• Note that f(n) ∈ Θ(g(n)) implies f(n) = O(g(n)), since Θ-notation is a

stronger notion than O-notation. Θ(g(n)) ⊆ O(g(n)).

• Examples
f(n)=n2+n ∈ O(n3); f(n)=n3 + 5∈ O(n3 log n); f(n) = n5 ∈ O(2n);
f(n)=n3+2n2 + 5 ∈ O(n3)
O-notation (cont.)

(Cormen et. al., 2009)

 Ω-notation
• Ω-notation provides an asymptotic lower bound.

• For a given function g(n), we denote by Ω(g(n)) the set of

functions
Ω(g(n)) = {f(n): there exist positive constants c and n0 such that
0 ≤ c g(n) ≤ f(n) for all n ≥ n0}

• f(n)=n2+n ∈ Ω(n); f(n)=n3+5∈ Ω(n log n); f(n) = 2n ∈

Ω(n5); f(n)=n2+3n ∈ Ω(n2)
Ω-notation (cont.)

(Cormen et. al., 2009)

 o-notation
• The asymptotic upper bound provided by O-notation may or may not
be asymptotically tight.
• The bound 2n2 ∈ O(n2) is asymptotically tight, but the bound 2n ∈
O(n2) is not.
• o-notation is used to denote an upper bound that is not
asymptotically tight.

• o(g(n)) is formally defined as the set:

o(g(n)) = {f(n): for any positive constant c > 0, there exists a constant
n0 > 0 such that 0 ≤ f(n) < c g(n) for all n ≥ n0}
 o-notation (cont.)

f ( n)
lim =0
n →∞ g ( n)

• Examples
2n ∈ o(n2)
2n2 ∉ o(n2)
 ω-notation
• ω-notation is used to denote a lower bound that is not
asymptotically tight.

• ω(g(n)) is formally defined as the set:

ω(g(n)) = {f(n): for any positive constant c > 0, there exists a
constant n0 > 0 such that 0 ≤ c g(n) < f(n) for all n ≥ n0}
 ω-notation (cont.)

f ( n)
lim =∞
n →∞ g ( n)

• Examples
n2 / 2 ∈ ω(n)
n2 / 2 ∉ ω(n2)
 Notations: summary
Notation Similar to

f(n) ∈ O(g(n)) a≤b

f(n) ∈ Ω(g(n)) a≥b

f(n) ∈ Θ(g(n)) a=b

f(n) ∈ o(g(n)) a<b

f(n) ∈ ω(g(n)) a>b

 Bibliography
• T. H. Cormen, C. E. Leiserson, R. L. Rivest, C. Stein, Introduction to
Algorithms, Third Edition, The MIT Press, 2009, Cambridge,
Massachusetts/London, England, 1292 p.