Lecture 3

Asymptotic Notation
&
Divide-and-Conquer

Dr. Md. Ali Hossain

Associate Professor, CSE, RUET

4 -1
 Engineered for

Analysis of Algorithms Tomorrow

 How good is the algorithm?

•
Correctness
•
Time efficiency
•
Space efficiency

 Does there exist a better algorithm?

•
Lower bounds
•
Optimality
 Engineered for

Asymptotic Analysis Tomorrow

 To compare two algorithms with running times f(n) and g(n), we

need a rough measure that characterizes how fast each function
grows.

 Hint: use rate of growth

 Compare functions in the limit, that is, asymptotically!

 (i.e., for large values of n)
 Engineered for

Asymptotic Notation Tomorrow

 O notation: asymptotic “less than”:


f(n)=O(g(n)) is read as “ f of n is a big oh of g of n”


implies: f(n) ≤ c*g(n) [ where c is a positive constant and n ≥

n0]

  Ω notation: asymptotic “greater than”:

 f(n)= Ω (g(n)) implies: f(n) ≥ c*g(n) [for all n ≥ n0]

 θ notation: asymptotic “equality”:

 f(n)= θ (g(n)) implies: : c1*g(n) ≤ f(n) ≤ c2*g(n) for all n ≥ n0

 Engineered for Tomorrow

Asymptotic Notations

O-notation
 Engineered for Tomorrow

Examples

f(n)=3n2+n
= 3n2+n2 n2 ≤ cn2 ; c ≥ 1 ; c = 1 and n0= 1

=4n2


f(n)<=c*g(n)

3n2+n<=4n2=o(n2)

Where n>=n0 and n=1

 Engineered for Tomorrow

Asymptotic Notations (Cont.)

Ω

- notation

Ω(g(n)) is the set of functions

with larger or same order of
growth as g(n)
 Engineered for

Examples
Tomorrow

f(n)=3n2+n
= 3n2+n
=3n2


f(n)>=c*g(n)

3n2+n>=3n2= Ω(n2)

Where n<=n0 and n=1

 Engineered for

Asymptotic Notations (Cont.) Tomorrow

•
θ -notation

θ(g(n)) is the set of functions

with the same order of growth as
g(n)
 Engineered for

Examples
Tomorrow

C2g(n)<=f(n)<=c1g(n) for all n>=n0

3n2+n<=f(n)<=3n2+n2
3n2<=f(n)<=4n2
Where c2=3, c1=4 and n=1
Therefore, 3n2+n ∈ θ(n2)
 Engineered for

Max Min Algorithm

Tomorrow

Straight Forward: Improvement:

StraightMaxMin(a, n, max, min) StraightMaxMin(a, n, max, min)

//Set max to the maximum and min to the minimum //Set max to the maximum and min to the
of a[1 : n]. minimum of a[1 : n].
{ {
max:= min:= a[1]; max:= min:= a[1];
for i : = 2 to n do for i : = 2 to n do
{ {
if (a[i] >max) then max:= a[i]; if (a[i] >max) then max:= a[i];
if (a[i] <min) then min:= a[i]; else if (a[i] <min) then min:= a[i];
} }
} }
Required Comparisons: Required Comparisons:
Beset case: 2*(n-1) Beset case: (n-1)
Worst case: 2*(n-1) Worst case: 2*(n-1)
 MaxMin Algorithm
Recursively Max Min algorithm:
MaxMin(i, j, max, min)
{
     if (i=j) then max := min := a[i]; //Small(P)
     else if (i=j-1) then // Another case of Small(P)
          {
                if (a[i] < a[j]) then max := a[j]; min := a[i];
                else max := a[i]; min := a[j];
          }
     else
     {
           // if P is not small, divide P into sub-problems. Find where to split the set.
           mid := ( i + j )/2;     // Solve the sub-problems.
           MaxMin( i, mid, max, min );
           MaxMin( mid+1, j, max1, min1 );
           // Combine the solutions.
           if (max < max1) then max := max1;
           if (min > min1) then min := min1;
     }
}
4 -12
 A simple example
 Finding the maximum and minimum of a set S of n
numbers

4 -13
 Time complexity
T(n)=2T(n/2)+2T(n/2)+2 if n>=2
=1 if n=2
=0 if n=1

 Calculation of T(n):  3n/2 -2 is smaller

than 2n-2
Assume n = 2k,
 This optimal in
T(n) = 2T(n/2)+2 number of
= 2(2T(n/4)+2)+2 comparisons
= 4T(n/4)+4+2
:
=2k-1T(2)+2k-2
=3n/2-2
4 -14
Merge Sort

4 -15
Algorithm: Merge Element

4 -16
Tree of Calls of Merge Sort

4 -17
Merge Sort (Example) Cont..
Merge Sort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
MergeSort (Example) Cont..
 Merge

14 23 45 98 6 33 42 67
 Merge

14 23 45 98 6 33 42 67
 Merge

14 23 45 98 6 33 42 67

6
 Merge

14 23 45 98 6 33 42 67

6 14
 Merge

14 23 45 98 6 33 42 67

6 14 23
 Merge

14 23 45 98 6 33 42 67

6 14 23 33
 Merge

14 23 45 98 6 33 42 67

6 14 23 33 42
 Merge

14 23 45 98 6 33 42 67

6 14 23 33 42 45
 Merge

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67
 Merge

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98