Discrete Structures

Algorithm

Dr. Peter Amoako-Yirenkyi

Mathematics Department/College of Science

Kwame Nkrumah University of Science and Technology

December 11, 2018

Introduction
Brief History on Algorithm
Introduction
Brief History on Algorithm
Introduction
Brief History on Algorithm
• The term algorithm is a corruption of the name Al-Khowarizmi.
Introduction
Brief History on Algorithm
• The term algorithm is a corruption of the name Al-Khowarizmi.
• Al-Khowarizmi a mathematician of the ninth century, whose book on
Hindu numerals is the basis of modern decimal notation.
Introduction
Brief History on Algorithm
• The term algorithm is a corruption of the name Al-Khowarizmi.
• Al-Khowarizmi a mathematician of the ninth century, whose book on
Hindu numerals is the basis of modern decimal notation.
• Originally, the word algorism was used for the rules for performing
arithmetic using decimal notation.
Introduction
Brief History on Algorithm
• The term algorithm is a corruption of the name Al-Khowarizmi.
• Al-Khowarizmi a mathematician of the ninth century, whose book on
Hindu numerals is the basis of modern decimal notation.
• Originally, the word algorism was used for the rules for performing
arithmetic using decimal notation.
• Algorism evolved into the word algorithm by the eighteenth century.
Introduction
Brief History on Algorithm
• The term algorithm is a corruption of the name Al-Khowarizmi.
• Al-Khowarizmi a mathematician of the ninth century, whose book on
Hindu numerals is the basis of modern decimal notation.
• Originally, the word algorism was used for the rules for performing
arithmetic using decimal notation.
• Algorism evolved into the word algorithm by the eighteenth century.
• With the growing interest in computing machines, the concept of an
algorithm was given a more general meaning, to include all definite
procedures for solving problems, not just the procedures for
performing arithmetic.
Introduction

What is an Algorithm ?
Introduction
Definition
Introduction
Definition

An algorithm is a finite sequence of

precise instructions for performing a
computation or for solving a
problem.
Algorithm
Problem
Algorithm
Problem
Describe how one may find a maximum value in a finite sequence of
integers.
Algorithm
Problem
Describe how one may find a maximum value in a finite sequence of
integers.
Solution
Algorithm
Problem
Describe how one may find a maximum value in a finite sequence of
integers.
Solution
1. Set the temporary maximum equal to the first integer in the sequence.
Algorithm
Problem
Describe how one may find a maximum value in a finite sequence of
integers.
Solution
1. Set the temporary maximum equal to the first integer in the sequence.
2. Compare the next integer in the sequence to the temporary maximum,
and if it is larger than the temporary maximum, set the temporary
maximum equal to this integer.
Algorithm
Problem
Describe how one may find a maximum value in a finite sequence of
integers.
Solution
1. Set the temporary maximum equal to the first integer in the sequence.
2. Compare the next integer in the sequence to the temporary maximum,
and if it is larger than the temporary maximum, set the temporary
maximum equal to this integer.
3. Repeat the previous step if there are more integers in the sequence.
Algorithm
Problem
Describe how one may find a maximum value in a finite sequence of
integers.
Solution
1. Set the temporary maximum equal to the first integer in the sequence.
2. Compare the next integer in the sequence to the temporary maximum,
and if it is larger than the temporary maximum, set the temporary
maximum equal to this integer.
3. Repeat the previous step if there are more integers in the sequence.
4. Stop when there are no integers left in the sequence. The temporary
maximum at this point is the largest integer in the sequence.
Algorithm
Algorithm
Finding the Maximum Element in a Finite Sequence.
Algorithm
Algorithm
Finding the Maximum Element in a Finite Sequence.

procedure max (a1 , a2 , ..., an : integers)

Algorithm
Algorithm
Finding the Maximum Element in a Finite Sequence.

procedure max (a1 , a2 , ..., an : integers)

max := a1
Algorithm
Algorithm
Finding the Maximum Element in a Finite Sequence.

procedure max (a1 , a2 , ..., an : integers)

max := a1
for i := 2 to n
Algorithm
Algorithm
Finding the Maximum Element in a Finite Sequence.

procedure max (a1 , a2 , ..., an : integers)

max := a1
for i := 2 to n
if max < ai then max := ai
Algorithm
Algorithm
Finding the Maximum Element in a Finite Sequence.

procedure max (a1 , a2 , ..., an : integers)

max := a1
for i := 2 to n
if max < ai then max := ai
return max{max is the largest element}
Algorithm
Properties of Algorithm
Algorithm
Properties of Algorithm
Algorithm
Properties of Algorithm
• Input from a specified set,
Algorithm
Properties of Algorithm
• Input from a specified set,
• Output from a specified set (solution),
Algorithm
Properties of Algorithm
• Input from a specified set,
• Output from a specified set (solution),
• Definiteness of every step in the computation,
Algorithm
Properties of Algorithm
• Input from a specified set,
• Output from a specified set (solution),
• Definiteness of every step in the computation,
• Correctness of output for every possible input,
Algorithm
Properties of Algorithm
• Input from a specified set,
• Output from a specified set (solution),
• Definiteness of every step in the computation,
• Correctness of output for every possible input,
• Finiteness of the number of calculation step,
Algorithm
Properties of Algorithm
• Input from a specified set,
• Output from a specified set (solution),
• Definiteness of every step in the computation,
• Correctness of output for every possible input,
• Finiteness of the number of calculation step,
• Effectiveness of each calculation step,
Algorithm
Properties of Algorithm
• Input from a specified set,
• Output from a specified set (solution),
• Definiteness of every step in the computation,
• Correctness of output for every possible input,
• Finiteness of the number of calculation step,
• Effectiveness of each calculation step,
• Generality for a class of problems.
Searching Algorithm
Searching Algorithm
Example
Searching Algorithm
Example
A program that checks the spelling of words searches for them in a
dictionary, which is just an ordered list of words. Problems of this kind
are called searching problems.
Searching Algorithm
Example
A program that checks the spelling of words searches for them in a
dictionary, which is just an ordered list of words. Problems of this kind
are called searching problems.

Definition
Searching problems are problems of locating an element in an
ordered list.
Searching Algorithm
Some Searching Methods

1. The Linear search

Searching Algorithm
Some Searching Methods

1. The Linear search

2. The Binary Search

Searching Algorithm
Some Searching Methods

1. The Linear search

2. The Binary Search

Searching Algorithm
Algorithm
The Linear Search

procedure linear search(x : integer , a1 , a2 , . . . , an : distinct integers)

Searching Algorithm
Algorithm
The Linear Search

procedure linear search(x : integer , a1 , a2 , . . . , an : distinct integers)

i := 1
Searching Algorithm
Algorithm
The Linear Search

procedure linear search(x : integer , a1 , a2 , . . . , an : distinct integers)

i := 1
while (i ≤ n and x 6= ai )
Searching Algorithm
Algorithm
The Linear Search

procedure linear search(x : integer , a1 , a2 , . . . , an : distinct integers)

i := 1
while (i ≤ n and x 6= ai )
i := i + 1
Searching Algorithm
Algorithm
The Linear Search

procedure linear search(x : integer , a1 , a2 , . . . , an : distinct integers)

i := 1
while (i ≤ n and x 6= ai )
i := i + 1
if i ≤ n then location := i
Searching Algorithm
Algorithm
The Linear Search

procedure linear search(x : integer , a1 , a2 , . . . , an : distinct integers)

i := 1
while (i ≤ n and x 6= ai )
i := i + 1
if i ≤ n then location := i
else location := 0
Searching Algorithm
Algorithm
The Linear Search

procedure linear search(x : integer , a1 , a2 , . . . , an : distinct integers)

i := 1
while (i ≤ n and x 6= ai )
i := i + 1
if i ≤ n then location := i
else location := 0
return location{location is the subscript of the term that equals x ,
or is 0 if x is not found}
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
j := n{j is right endpoint of search interval}
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
j := n{j is right endpoint of search interval}
while i < j
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
j := n{j is right endpoint of search interval}
while i < j
m := [(i + j)/2]
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
j := n{j is right endpoint of search interval}
while i < j
m := [(i + j)/2]
if x > am then i := m + 1
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
j := n{j is right endpoint of search interval}
while i < j
m := [(i + j)/2]
if x > am then i := m + 1
else j := m
if x = ai then location := i
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
j := n{j is right endpoint of search interval}
while i < j
m := [(i + j)/2]
if x > am then i := m + 1
else j := m
if x = ai then location := i
else location := 0
Binary Search Algorithm
Algorithm

binary search(x : integer, a1 , a2 , . . . , an : increasing integers)

i := 1{i is left endpoint of search interval}
j := n{j is right endpoint of search interval}
while i < j
m := [(i + j)/2]
if x > am then i := m + 1
else j := m
if x = ai then location := i
else location := 0
return location{location is the subscript i of the term ai equal to x or 0
if x is not found }
Searching Algorithm

Remark
If the terms in a sequence are ordered, a binary search algorithm is more
efficient than linear search.
What is Sorting?
Definition

Sorting refers to ordering data in an increasing or

decreasing fashion according to some linear
relationship among the data items.
Some Sorting Methods

1. The Bubble Sort

Some Sorting Methods

1. The Bubble Sort

2. The Insertion Sort

The Bubble Sort Algorithm
Algorithm
The Bubble Sort Algorithm
Algorithm

procedure bubble sort (a1 , . . . , an : real numbers with n ≥ 2)

The Bubble Sort Algorithm
Algorithm

procedure bubble sort (a1 , . . . , an : real numbers with n ≥ 2)

for i := 1 to n − i
The Bubble Sort Algorithm
Algorithm

procedure bubble sort (a1 , . . . , an : real numbers with n ≥ 2)

for i := 1 to n − i
for j := 1 to n − i
The Bubble Sort Algorithm
Algorithm

procedure bubble sort (a1 , . . . , an : real numbers with n ≥ 2)

for i := 1 to n − i
for j := 1 to n − i
if aj > aj+1 then interchange aj and aj+1
The Bubble Sort Algorithm
Algorithm

procedure bubble sort (a1 , . . . , an : real numbers with n ≥ 2)

for i := 1 to n − i
for j := 1 to n − i
if aj > aj+1 then interchange aj and aj+1
{a1 , . . . , an is in increasing order }
Insertion Sort Algorithm
Algorithm
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
while aj > ai
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
while aj > ai
i := i + 1
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
for k := 0 to j − i − 1
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
for k := 0 to j − i − 1
aj−k := aj−k−1
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
for k := 0 to j − i − 1
aj−k := aj−k−1
ai := m
Insertion Sort Algorithm
Algorithm

procedure insertionsort (a1 , a2 , . . . , an : real numbers with n ≥ 2)

for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
for k := 0 to j − i − 1
aj−k := aj−k−1
ai := m
{a1 , . . . , an is in increasing order}
What is Greedy Algorithm?
Definition

Greedy Algorithm is an algorithm which selects the

best choice at each step.
Greedy Algorithm
Problem
Suppose we have a group of proposed talks with preset start and end
times. Devise a greedy algorithm to schedule as many of these talks as
possible in a lecture hall, under the assumptions that:
1. once a talk starts, it continues until it ends
Greedy Algorithm
Problem
Suppose we have a group of proposed talks with preset start and end
times. Devise a greedy algorithm to schedule as many of these talks as
possible in a lecture hall, under the assumptions that:
1. once a talk starts, it continues until it ends
2. no two talks can proceed at the same time
Greedy Algorithm
Problem
Suppose we have a group of proposed talks with preset start and end
times. Devise a greedy algorithm to schedule as many of these talks as
possible in a lecture hall, under the assumptions that:
1. once a talk starts, it continues until it ends
2. no two talks can proceed at the same time
3. a talk can begin at the same time another one ends
Greedy Algorithm
Problem
Suppose we have a group of proposed talks with preset start and end
times. Devise a greedy algorithm to schedule as many of these talks as
possible in a lecture hall, under the assumptions that:
1. once a talk starts, it continues until it ends
2. no two talks can proceed at the same time
3. a talk can begin at the same time another one ends
4. talk j begins at time sj (where s stands for start)
Greedy Algorithm
Problem
Suppose we have a group of proposed talks with preset start and end
times. Devise a greedy algorithm to schedule as many of these talks as
possible in a lecture hall, under the assumptions that:
1. once a talk starts, it continues until it ends
2. no two talks can proceed at the same time
3. a talk can begin at the same time another one ends
4. talk j begins at time sj (where s stands for start)
5. talk j ends at time ej (where e stands for end)
Greedy Algorithm for Scheduling Talks
Greedy Algorithm for Scheduling Talks
Algorithm
Greedy Algorithm for Scheduling Talks
Algorithm
Greedy Algorithm for Scheduling Talks
Algorithm

procedure schedule(s1 ≤ s2 ≤ . . . ≤ sn : start times of talks, e1 ≤

e2 ≤ . . . en : ending times of talks)
Greedy Algorithm for Scheduling Talks
Algorithm

procedure schedule(s1 ≤ s2 ≤ . . . ≤ sn : start times of talks, e1 ≤

e2 ≤ . . . en : ending times of talks)
sort talks by finish time and reorder so that e1 ≤ e2 ≤ . . . ≤ en
Greedy Algorithm for Scheduling Talks
Algorithm

procedure schedule(s1 ≤ s2 ≤ . . . ≤ sn : start times of talks, e1 ≤

e2 ≤ . . . en : ending times of talks)
sort talks by finish time and reorder so that e1 ≤ e2 ≤ . . . ≤ en
S := ∅
Greedy Algorithm for Scheduling Talks
Algorithm

procedure schedule(s1 ≤ s2 ≤ . . . ≤ sn : start times of talks, e1 ≤

e2 ≤ . . . en : ending times of talks)
sort talks by finish time and reorder so that e1 ≤ e2 ≤ . . . ≤ en
S := ∅
for j := 1 to n
Greedy Algorithm for Scheduling Talks
Algorithm

procedure schedule(s1 ≤ s2 ≤ . . . ≤ sn : start times of talks, e1 ≤

e2 ≤ . . . en : ending times of talks)
sort talks by finish time and reorder so that e1 ≤ e2 ≤ . . . ≤ en
S := ∅
for j := 1 to n
if talk j is compatible with S then
Greedy Algorithm for Scheduling Talks
Algorithm

procedure schedule(s1 ≤ s2 ≤ . . . ≤ sn : start times of talks, e1 ≤

e2 ≤ . . . en : ending times of talks)
sort talks by finish time and reorder so that e1 ≤ e2 ≤ . . . ≤ en
S := ∅
for j := 1 to n
if talk j is compatible with S then
S := S ∪ {talk j}
Greedy Algorithm for Scheduling Talks
Algorithm

procedure schedule(s1 ≤ s2 ≤ . . . ≤ sn : start times of talks, e1 ≤

e2 ≤ . . . en : ending times of talks)
sort talks by finish time and reorder so that e1 ≤ e2 ≤ . . . ≤ en
S := ∅
for j := 1 to n
if talk j is compatible with S then
S := S ∪ {talk j}
return S{S is the set of talks scheduled}
Halting Problem

What is Halting Problem?

Halting Problem
Definition
Halting problem is the problem of determining, from
a description of an arbitrary computer program and
an input, whether the program will finish running or
continue to run forever.
Halting Problem

If H(P,P) = ”halts,”
P as program then loop forever

Input Program Output Program

Program P H(P,I) H(P,P) K(P)

P as input If H(P,P) = ”loops forever,”

then halt

Showing that the Halting Problem is Unsolvable.

Complexity
In general, we are not so much interested in the time and space
complexity for small inputs.
Complexity
In general, we are not so much interested in the time and space
complexity for small inputs.
Example
Assume two algorithms A and B that solve the same class of problems.
Complexity
In general, we are not so much interested in the time and space
complexity for small inputs.
Example
Assume two algorithms A and B that solve the same class of problems.
The time complexity of A is 5, 000n, the one for B is [1.1n ] for an input
with n elements.
Complexity
In general, we are not so much interested in the time and space
complexity for small inputs.
Example
Assume two algorithms A and B that solve the same class of problems.
The time complexity of A is 5, 000n, the one for B is [1.1n ] for an input
with n elements.
For n = 10, A requires 50,000 steps, but B requires only 3, so B seems
to be superior to A.
Complexity
In general, we are not so much interested in the time and space
complexity for small inputs.
Example
Assume two algorithms A and B that solve the same class of problems.
The time complexity of A is 5, 000n, the one for B is [1.1n ] for an input
with n elements.
For n = 10, A requires 50,000 steps, but B requires only 3, so B seems
to be superior to A.

For n = 1000, however, A requires 5, 000, 000 steps, while B requires

2.5 · 1041 steps.

The difference in time complexity between linear and binary search is

meaningless for a sequence with n = 10, it is gigantic for n = 230 .
Complexity
Complexity
This means that algorithm B cannot be used for large inputs, while
algorithm A is still feasible.
Complexity
This means that algorithm B cannot be used for large inputs, while
algorithm A is still feasible.
So what is important is the growth of the complexity functions.
Complexity
This means that algorithm B cannot be used for large inputs, while
algorithm A is still feasible.
So what is important is the growth of the complexity functions.
Complexity
This means that algorithm B cannot be used for large inputs, while
algorithm A is still feasible.
So what is important is the growth of the complexity functions.
Complexity
This means that algorithm B cannot be used for large inputs, while
algorithm A is still feasible.
So what is important is the growth of the complexity functions.

Remark
The growth of time and space complexity with increasing input size n is a
suitable measure for the comparison of algorithms.
Complexity

Table 1: Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

Complexity

Table 1: Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5000n [1.1n ]
Complexity

Table 1: Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5000n [1.1n ]
10 50,000 3
Complexity

Table 1: Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5000n [1.1n ]
10 50,000 3
100 500,000 13,781
Complexity

Table 1: Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5000n [1.1n ]
10 50,000 3
100 500,000 13,781
1,000 5,000,000 2.5 · 1041
Complexity

Table 1: Comparison: time complexity of algorithms A and B

Input Size Algorithm A Algorithm B

n 5000n [1.1n ]
10 50,000 3
100 500,000 13,781
1,000 5,000,000 2.5 · 1041
1,000,000 5 · 109 4.8 · 1041392
The Growth of Functions
The growth of functions is usually described using the big-O notation.

Definition
Let f and g be functions from the integers or the real numbers to the
real numbers.
We say that f (x ) is O(g (x )) if there are constants C and k such that
|f (x )| ≤ C |g (x )| whenever x > k.
The Growth of Functions
The growth of functions is usually described using the big-O notation.

Definition
Let f and g be functions from the integers or the real numbers to the
real numbers.
We say that f (x ) is O(g (x )) if there are constants C and k such that
|f (x )| ≤ C |g (x )| whenever x > k.

If we want to show that f (x ) is O(g (x )), we only need to find one pair
(C , k ) (which is never unique).
The Growth of Functions
The idea behind the big-O notation is to establish an
upper boundary for the growth of a function f (x ) for large x.
The Growth of Functions
The idea behind the big-O notation is to establish an
upper boundary for the growth of a function f (x ) for large x.
The Growth of Functions
The idea behind the big-O notation is to establish an
upper boundary for the growth of a function f (x ) for large x.

This boundary is specified by a function g (x ) that is usually much

simpler than f (x ). We accept the constant C in the requirement
f (x ) ≤ C · g (x ) whenever x > k, because C does not grow with x.
We are only interested in large x, so it is OK if f (x ) > C · g (x ) for x ≤ k.
The Growth of Functions
The idea behind the big-O notation is to establish an
upper boundary for the growth of a function f (x ) for large x.

This boundary is specified by a function g (x ) that is usually much

simpler than f (x ). We accept the constant C in the requirement
f (x ) ≤ C · g (x ) whenever x > k, because C does not grow with x.
We are only interested in large x, so it is OK if f (x ) > C · g (x ) for x ≤ k.
The Growth of Functions
Example
Show that f (x ) = x 2 + 2x + 1 is O(x 2 ).
The Growth of Functions
Example
Show that f (x ) = x 2 + 2x + 1 is O(x 2 ).
For x > 1 we have:
The Growth of Functions
Example
Show that f (x ) = x 2 + 2x + 1 is O(x 2 ).
For x > 1 we have:
x 2 + 2x + 1 ≤ x 2 + 2x 2 + x 2
The Growth of Functions
Example
Show that f (x ) = x 2 + 2x + 1 is O(x 2 ).
For x > 1 we have:
x 2 + 2x + 1 ≤ x 2 + 2x 2 + x 2
⇒ x 2 + 2 x + 1 ≤ 4x 2
The Growth of Functions
Example
Show that f (x ) = x 2 + 2x + 1 is O(x 2 ).
For x > 1 we have:
x 2 + 2x + 1 ≤ x 2 + 2x 2 + x 2
⇒ x 2 + 2 x + 1 ≤ 4x 2
Therefore, for C = 4 and k = 1:
The Growth of Functions
Example
Show that f (x ) = x 2 + 2x + 1 is O(x 2 ).
For x > 1 we have:
x 2 + 2x + 1 ≤ x 2 + 2x 2 + x 2
⇒ x 2 + 2 x + 1 ≤ 4x 2
Therefore, for C = 4 and k = 1:
f (x ) ≤ Cx 2 whenever x > k.
The Growth of Functions
Example
Show that f (x ) = x 2 + 2x + 1 is O(x 2 ).
For x > 1 we have:
x 2 + 2x + 1 ≤ x 2 + 2x 2 + x 2
⇒ x 2 + 2 x + 1 ≤ 4x 2
Therefore, for C = 4 and k = 1:
f (x ) ≤ Cx 2 whenever x > k.
⇒ f (x ) is O(x 2 ).
The Growth of Functions
Question
If f (x ) is O(x 2 ), is it also O(x 3 )?
The Growth of Functions
Question
If f (x ) is O(x 2 ), is it also O(x 3 )?

Yes. x 3 grows faster than x 2 , so x 3 grows also faster than f (x ).

The Growth of Functions
Question
If f (x ) is O(x 2 ), is it also O(x 3 )?

Yes. x 3 grows faster than x 2 , so x 3 grows also faster than f (x ).

Therefore, we always have to find the smallest simple function g (x )

for which f (x ) is O(g (x )).
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
• log n
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
• log n
• n
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
• log n
• n
• n log n
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
• log n
• n
• n log n
• n2
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
• log n
• n
• n log n
• n2
• n3
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
• log n
• n
• n log n
• n2
• n3
• 2n
The Growth of Functions
”Popular” functions g (n) are n log n, 1, 2n , n2 , n!, n, n3 , log n
Listed from slowest to fastest growth:
• 1
• log n
• n
• n log n
• n2
• n3
• 2n
• n!
The Growth of Functions
Definitions
1. A problem that can be solved with polynomial
worst-case complexity is called tractable.
The Growth of Functions
Definitions
1. A problem that can be solved with polynomial
worst-case complexity is called tractable.
2. Problems of higher complexity are called
intractable.
The Growth of Functions
Definitions
1. A problem that can be solved with polynomial
worst-case complexity is called tractable.
2. Problems of higher complexity are called
intractable.
3. Problems that no algorithm can solve are called
unsolvable.
Useful Rules for Big-O

1. For any polynomial f (x ) = an x n + an−1 x n−1 + · · · + a0

where a0 , a1 , . . . , an are real numbers, f (x ) is O(x n ).
Useful Rules for Big-O

1. For any polynomial f (x ) = an x n + an−1 x n−1 + · · · + a0

where a0 , a1 , . . . , an are real numbers, f (x ) is O(x n ).
2. If f1 (x ) is O(g1 (x )) and f2 (x ) is O(g2 (x )), then
(f1 + f2 )(x ) is O(max(g1 (x ), g2 (x )))
Useful Rules for Big-O

1. For any polynomial f (x ) = an x n + an−1 x n−1 + · · · + a0

where a0 , a1 , . . . , an are real numbers, f (x ) is O(x n ).
2. If f1 (x ) is O(g1 (x )) and f2 (x ) is O(g2 (x )), then
(f1 + f2 )(x ) is O(max(g1 (x ), g2 (x )))
3. If f1 (x ) is O(g (x )) and f2 (x ) is O(g (x )), then
(f1 + f2 )(x ) is O(g (x )).
Useful Rules for Big-O

1. For any polynomial f (x ) = an x n + an−1 x n−1 + · · · + a0

where a0 , a1 , . . . , an are real numbers, f (x ) is O(x n ).
2. If f1 (x ) is O(g1 (x )) and f2 (x ) is O(g2 (x )), then
(f1 + f2 )(x ) is O(max(g1 (x ), g2 (x )))
3. If f1 (x ) is O(g (x )) and f2 (x ) is O(g (x )), then
(f1 + f2 )(x ) is O(g (x )).
4. If f1 (x ) is O(g1 (x )) and f2 (x ) is O(g2 (x )), then (f1 f2 )(x )
is O(g1 (x )g2 (x )).