DESIGN and ANALYSIS for ALGORITHMS

LECTURE 2

Techniques of Evaluation

1 / 15 Lecture 2 Techniques of Evaluation

 Landau’s notation

In general it is impossible to compute precisely the cost of an

algorithm in terms of time or space storage.
We will try to find distributions giving the minimal , the
average or the maximal cost.
Usually it is preferable to find the asymptote to which the cost
tends when the size of data becomes large.

2 / 15 Lecture 2 Techniques of Evaluation

 O(”order of” ) notation

Consider a function f (n) that is non-negative for all integers

n ≥ 0. We say that f (n) is in order of g(n) and we write
f (n) = O(g (n)), if there exists an integer n0 and a constant
c > 0 such that for all integer n ≥ n0 , f (n) ≤ cg (n).
Informally , we say that O(g ) is the set of all functions which
grow faster than f.

3 / 15 Lecture 2 Techniques of Evaluation

 O(”order of” ) notation(cont’d)

we also denote it by f (n) ∈ O(g (n)) or f (n) is O(g (n))

if g (n) = c ( constant) then f (n) = O(1)
Example : For f (n) = n + 6,
We have
f (n) = O(n),
f (n) = O(n2 )

4 / 15 Lecture 2 Techniques of Evaluation

 Properties of O

a) constant factors may be ignored. For all k > 0 , kf is O(f ). For instance cn2
and bn2 are both O(n2 ).
b) If f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)) then
f1 (n) + f2 (n) = O(max(g1 (n), g2 (n))). for instance if f1 (n) = n2 + 1 and
f2 (n) = 1 then f1 (n) + f2 (n) = O(n2 )
c) If f (n) = f1 (n) + f2 (n) in which f1 (n) and f2 (n) are both non-negative for all
n ≥ 0 such that limn→∞ ff2 (n)(n)
= L for some limit L ≥ 0, then f (n) = O(f1 (n)).for
1
instance if f1 (n) = n5 + 4 and f2 (n) = n2 , then f1 (n) + f2 (n) = O(n5 ).
d) If f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)), then f1 (n) × f2 (n) = O(g1 (n) × g2 (n)).
for instance if f1 (n) = 4n2 and f2 (n) = 3 log n then f1 (n) × f2 (n) = O(n2 log n).

e) If f1 (n) = O(g1 (n)) and g2 (n) is a function whose value is non-negative for
n ≥ 0, then f1 (n) × g2 (n) = O(g1 (n) × g2 (n)). for instance if f1 (n) = O(n2 ) and
g2 (n) = e n then f1 (n) × g2 (n) = O(n2 e n ).

5 / 15 Lecture 2 Techniques of Evaluation

 Properties of O(cont’d)

f) If f (n) = O(g (n)) and g (n) = O(h(n)) then f (n) = O(h(n)). f (n) = O(2n2 ) but
2n2 = O(n2 ) then f (n) = O(n2 ).
g) the growth rate of a polynomial is determined by the degree of its leading term
If f is a polynomial of degree d, then f is O(nd )
e.g: a0 + a1 n + . . . + ap np is O(np )

h) For every integer k ≥ 1, logk n = O(n).

6 / 15 Lecture 2 Techniques of Evaluation

 Tight bound

Consider a function f (n) = O(g (n)) . If for every function

h(n) such that f (n) = O(h(n)) it is also true that
g (n) = O(h(n)), then we say it is a tight asymptotic bound on
f (n).
if f (n) = n2 + 21 then f (n) = O(n3 ) and f (n) = O(n2 ) but f is
a polynomial then O(n2 ) is tighter.

7 / 15 Lecture 2 Techniques of Evaluation

 Ω Notation
Consider a function f (n) that is non-negative for all integers n ≥ 0.
We say that ”f(n) is omega g(n)”, which we write f (n) = Ω(g (n)),
if there is an integer n0 and a constant c > 0 such that for all
integers n ≥ n0 , f (n) ≥ cg (n).

Example: if f (n) = 3n2 − 8n + 8 then f (n) = Ω(n2 )

8 / 15 Lecture 2 Techniques of Evaluation
 Ω and Θ Notations

Consider a function f (n) that is non-negative for all integers n ≥ 0.

We say that ”f(n) is theta g(n)” which we write as
f (n) = Θ(g (n)), if and only if f (n) is O(g (n)) and f (n) is Ω(g (n)).

Example: if f (n) = 3n2 − 8n + 8 then f (n) = Θ(n2 )

9 / 15 Lecture 2 Techniques of Evaluation

 some useful formulae

a) The sum of first n r th powers as the (r + 1)th power:

n
n(n + 1) 1
∑ i = 2 = 2 n2 + O(n)
i=1
n
n(n + 1)(2n + 1) 1
∑ i2 = 6
= n3 + O(n2 )
3
i=1
in general
n
1
∑ i p = p + 1 np+1 + O(np )
i=1
b) It is proven
√ that n! can be computed with the stirling’s formula as follows :
n! = 2πn( ne )n [1 + O( n1 )] = O(nn )
c) the nth harmonic
Hn = 1 + 21 + 13 + . . . + n1
and it is proved that Hn = log (n) + γ + O( n1 ) where γ is a real number called
Euler’s constant γ ≈ 0.579
k+1 −1
d) Geometric series : ∑ki=0 ar i = a( r r −1 )
k
e) Arithmetic -Geometric Series : ∑ i2i = (k − 1)2k+1 + 2
i=1

10 / 15 Lecture 2 Techniques of Evaluation

 homogeneous linear recurrences

Homogeneous linear recurrences with constant coefficients are

recurrences of the form:

a0 tn + a1 tn−1 + . . . + ak tn−k = 0 (1)

the characteristic equation of equation(1) is

a0 x k + a1 x k−1 + . . . + ak = 0 (2)

if equation(2) has k distinct root roots r1 , r2 , . . . , rk then

k
tn = ∑ ci rin
i=1

Example: Given the equation recurrence tn = 8tn−1 − 15tn−2

with t0 = 0 and t1 = 4, we have tn = 2.5n − 2.3n

11 / 15 Lecture 2 Techniques of Evaluation

 homogeneous linear recurrences(cont’d)

Proposition
Suppose that the roots of characteristic equation (2) are not all
distinct. If m is the multiplicity of a root r , then tn = r n , tn = nr n ,
tn = n2 r n , . . ., tn = nm−1 r n are all possible solution of equation
(2). The general solution is then a linear combination of these
terms and the terms contributed by other roots of the
characteristic equation.

Example: If Tn is defined by the following recurrent relation

tn = 5tn−1 − 8tn−2 + 4tn−3 , n ≥ 3
subject to t0 = 0, t1 = 1 and t2 = 2.
then
tn = 2n+1 − n2n−1 − 2.

12 / 15 Lecture 2 Techniques of Evaluation

 homogeneous linear recurrences (cont’d)

Proposition
Consider (xn )n≥0 a linear recurrent sequence defined by equation
(1). If the roots r1 , r2 , . . . , rk of the equation (2) are simple ( means
each has multiplicity 1), then

|xn | = O(maxi=1,...,k (|ri |)n )

Example: With the Fibonacci sequence defined as follows:

(
n if n =0 or n = 1
Fn = (3)
Fn−1 + Fn−2 if n ≥ 2
We have Fn = O(Φn )

13 / 15 Lecture 2 Techniques of Evaluation

 homogeneous linear recurrences (cont’d)

Proposition 1.5
Consider (xn )n≥0 a linear recurrent sequence defined by equation
(1). If ρ is the leading root of the equation 2 (ρ is the leading root
of 2 if for all root r of equation (2), |r | < |ρ|) then there exists a
real constant A such that :
limn→∞ xn = limn→∞ Aρ n means xn = O(ρ n )

For instance fibonacci sequence is O(Φn )

14 / 15 Lecture 2 Techniques of Evaluation

 Your To do List

Read Section 5( Evaluations) in Chapter 1: Introduction

15 / 15 Lecture 2 Techniques of Evaluation