Chapter 3: Algorithms

Trần Hòa Phú

Ngày 20 tháng 10 năm 2023

 Objectives
Algorithms
The Growth of functions
Complexity of Algorithms

Trần Hòa Phú Chapter 3: Algorithms

 Algorithms
Definition
An algorithm is a finite sequence of precise instructions for
performing a computation or for solving a problem.

Trần Hòa Phú Chapter 3: Algorithms

 ALGORITHM 1
Finding the Maximum Element in a Finite Sequence
procedure max(a1, ..., an : distinct integers)
max := a1
for i := 2 to n
if max < ai then max := ai
return max {max is the largest element }.

Trần Hòa Phú Chapter 3: Algorithms

 ALGORITHM 2
The Linear Search Algorithm
procedure linearsearch (x : integer, a1, ..., an : distinct
integers)
i := 1
While (i ≤ n and x ̸= ai )
i := i + 1
if i ≤ n then location := i
else location:=0
return location
Trần Hòa Phú Chapter 3: Algorithms
 ALGORITHM 3
The Binary Search Algorithm
Example To search 19 in the list
1 2 3 5 6 8 10 12 13 15 16 18 19 20 22
Split the list
1 2 3 5 6 8 10 12 13 15 16 18 19 20 22
19 > 10?, Yes. split the right list
12 13 15 16 18 19 20 22
19 > 16?, Yes. split the right list
18 19 20 22
19 > 19?, No. split the left list
18 19
19 > 18? Yes. split the right list
Trần Hòa Phú Chapter 3: Algorithms
procedure binarysearch (x : integer, a1 , ..., an : increasing integers)
i := 1{i is left endpoint of search interval }
j := n{j is right endpoint of search interval }
While i < j
m := [(i + j)/2]
if x > am then i := m + 1
else j := m
if x = ai then location := i
else location := 0
return location (location is the subscript i of the term ai equal to x ,
or 0 if x is not found)

Trần Hòa Phú Chapter 3: Algorithms

 ALGORITHM 4
Bubble Sort
Example Use the bubble sort to put 3,2,4,1,5 into increasing order.

Trần Hòa Phú Chapter 3: Algorithms

Trần Hòa Phú Chapter 3: Algorithms
Trần Hòa Phú Chapter 3: Algorithms
procedure bubblesort(a1, ..., an : real numbers with n ≥ 2)
for i = 1 to n − 1
for j := 1 to n − i
if aj > aj+1 then interchange aj and aj+1
{a1, a2, ..., an is in increasing order }

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
bubblesort(a1 , ..., an : real numbers with n ≥ 2)
for i = 1 to n − 1
for j := 1 to n − i
if aj > aj+1 then interchange aj and aj+1
{a1 , a2 , ..., an is in increasing order }
If input = 3,2,4,7,1,6,5. Find the order of the elements in the list
after the first pass (i=1).
A. 2,3,4,1,5,6,7
B. 2,3,1,4,5,6,7
C. 2,3,4,1,6,5,7
D. 2,3,1,4,6,5,7
Trần Hòa Phú Chapter 3: Algorithms
 ALGORITHM 5
Insertion Sort
Example
Use the insertion sort to put the elements of the list
4, 3, 2, 10, 12, 1, 5, 6 in increasing order.

Trần Hòa Phú Chapter 3: Algorithms

Trần Hòa Phú Chapter 3: Algorithms
Trần Hòa Phú Chapter 3: Algorithms
procedure insertion sort (a1, ..., an : real numbers with
n ≥ 2)
for j := 2 to n
i := 1
while aj > ai
i := i + 1
m := aj
for k := 0 to j − i − 1
aj−k := aj−k−1
ai := m
{a1, ..., an is in increasing order }

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Apply the insertion sort to the sequence 5, 3, 4, 9, 6. What
is the result obtained after the iteration when j = 3?
A. 3,4,5,9,6
B. 3,5,4,6,9
C. 3,4,5,6,9
D. None of the other choices is correct

Trần Hòa Phú Chapter 3: Algorithms

 ALGORITHM 6
Greedy Algorithm
Optimization problems :
Finding a route between two cities with smallest total
mileage
Determining a way to encode messages using the fewest
bits possible
Algorithm: select the best choice at each step instead of
considering of all sequences of steps

Trần Hòa Phú Chapter 3: Algorithms

Bài toán: Đổi tiền

Ví dụ: Cho 5 loại tiền giấy có mệnh giá lần lượt là 1 ngàn, 2 ngàn,
5 ngàn, 10 ngàn.
Hỏi số tờ tiền ít nhất cho tương ứng với 28 ngàn là bao nhiêu?
Thuận toán Tham Lam:
- Đổi 1 tờ 10 ngàn. Do đó còn 18 ngàn.
- Đổi 1 tờ 10 ngàn. Còn 8 ngàn.
- Đổi 1 tờ 5 ngàn. Còn 3 ngàn.
- Đổi 1 tờ 2 ngàn. Còn 1 ngàn.
- Đổi 1 tờ 1 ngàn.
Câu hỏi: Tối ưu mỗi bước có tối ưu toàn cục?

Trần Hòa Phú Chapter 3: Algorithms

Trần Hòa Phú Chapter 3: Algorithms
 The Growth of Functions
Big-O Notation
Definition
Let f and g be functions Z → R or R → R.
We say that f (x ) is O(g(x )) if there are constants C and k
such that
|f (x )| ≤ C |g(x )| ∀x > k

Trần Hòa Phú Chapter 3: Algorithms

Trần Hòa Phú Chapter 3: Algorithms
Example Show that 5x + 3 is O(x ).
Chứng minh.
We have
|5x + 3| ≤ 5|x | + 3 ≤ 5|x | + 3|x | = 8|x | ∀x > 1 (1)

Trần Hòa Phú Chapter 3: Algorithms

Example Show that 5x + 3 is O(x 2).
Chứng minh.
We have
|5x +3| = |5||x |+3 ≤ 5|x ||x |+3|x |2 = 8|x 2| ∀x ≥ 1 (2)

Trần Hòa Phú Chapter 3: Algorithms

Theorem
Let an x n + an−1x n−1 + ... + a1x + a0 is O(x n )
Example 1 8x 10 + 5x 8 + x 7 + 1 is O(x 10)
1 8
Example 2 x − 64 + 2x 2 is O(x 8)
100

Trần Hòa Phú Chapter 3: Algorithms

Example Show that 1 + 2 + ... + n is O(n2).
Chứng minh.
We have
1 + 2 + 3 + ... + n ≤ n + n + n + ... + n = n.n = n2 ∀n ≥ 1
(3)

Trần Hòa Phú Chapter 3: Algorithms

1 ≤ log n ≤ n ≤ n log n ≤ n2 ≤ 2n ≤ n! n≥5

Trần Hòa Phú Chapter 3: Algorithms

The Growth of Combinations of Functions
Theorem
Suppose that
f1(x ) is O(g1(x )) and f2(x ) is O(g2(x )). Then
(f1 + f2 )(x ) is O(max(|g1 (x )|, |g2 (x )|)) (4)
Example 1 The function 2x + log x is O(x ).
Example 2 The function log x + 2x is O(2x )
Example 3 The function x 2 + x log x is O(x 2)

Trần Hòa Phú Chapter 3: Algorithms

Theorem
Suppose that f1(x ) is O(g1(x )) and f2(x ) is O(g2(x )).
Then (f1f2)(x ) is O(g1(x )g2(x )).
Example (x 5 + 3x + 1)(x 3 − 2) is O(x 8)
Example (x + x 3)(x log x + x 2) is O(x 5)

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Determine whether each of these functions
is O(x )
a) f (x ) = 10
b) f (x ) = 3x + 7
c) f (x ) = x 2 + x + 1
d) f (x ) = 5 log x
Trần Hòa Phú Chapter 3: Algorithms
Solution
a. 10 is O(1), O(x )
b. 3x + 7 is O(x )
c. x 2 + x + 1 is O(x 2), not O(x )
d. 5 log x is O(log x ), O(x )

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Determine whether each of these functions is O(x 2)
a) f (x ) = 17x + 11
b) f (x ) = x 2 + 1000
c) f (x ) = x log x
x4
d) f (x ) =
2
e) f (x ) = 2x
x 3 + 2x
f) f (x ) =
2x + 1
Trần Hòa Phú Chapter 3: Algorithms
Solution
a. 17x + 11 is O(x ), is O(x 2)
b. x 2 + 1000 is O(x 2)
c. x log x is O(x 2)
x4
d. is O(x 4), not O(x 2)
2
e. 2x is O(2x ), not O(x 2)
x 3 + 2x
f. is O(x 2)
2x + 1

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Give a big-O estimate for each of these functions
a. (n2 + 8)(n + 1)
b. (4n5 + 6n − 4)(7n2 − 3)
c. 12 + 22 + ... + n2
d. 1.2 + 2.3 + ... + (n − 1)n

Trần Hòa Phú Chapter 3: Algorithms

Solution
a. (n2 + 8)(n + 1) is O(n3)
b. (4n5 + 6n − 4)(7n2 − 3) is O(n7)
c. 12 + 22 + ... + n2 is O(n3)
d. 1.2 + 2.3 + ... + (n − 1)n is O(n3)

Trần Hòa Phú Chapter 3: Algorithms

 Big-Omega
Definition
We say that f (x ) is Ω(g(x )) if there are positive constants C and k
such that
|f (x )| ≥ C |g(x )|
whenever x > k.
Theorem
f (x ) = Ω(g(x )) ⇔ g(x ) = O(f (x ))

Trần Hòa Phú Chapter 3: Algorithms

Example Show that 3x 2 + x + 5 is Ω(x 2)?
Chứng minh.
We have
|3x 2 + x + 5| = 3x 2 + x + 5 ≥ 3x 2 ∀x ≥ −5 (5)

Trần Hòa Phú Chapter 3: Algorithms

 Big-Theta
Definition
We say that f (x ) is Θ(g(x )) if f (x ) is O(g(x )) and f (x ) is
Ω(g(x )). It means there are C1 , C2 , k such that

C1 |g(x )| ≤ f (x ) ≤ C2 |g(x )) ∀x > k (6)

When f (x ) is Θ(g(x )), we say that f is big-Theta of g(x ), that
f (x ) is of order g(x ), and that f (x ) and g(x ) are of the same order.

Trần Hòa Phú Chapter 3: Algorithms

Theorem
Let an x n + an−1x n−1 + ... + a1x + a0 is Θ(x n )
Example x 5 − 10x + 42 is Θ(x 5)
1
Example x 8 − 20x 3 is Θ(x 8)
2

Trần Hòa Phú Chapter 3: Algorithms

 Complexity of Algorithms
How can the efficiency of an algorithm be analyzed?
Time Complexity: the number of operations used by the
algorithm.
Space Complexity: amount of computer memory
required to implement the algorithm
Computational Complexity = Time Complexity + Space
Complexity

Trần Hòa Phú Chapter 3: Algorithms

Trần Hòa Phú Chapter 3: Algorithms
Trần Hòa Phú Chapter 3: Algorithms
Worst-Case Complexity: the largest number of operations
needed to solve the given problem using this algorithm on
input of specified size.
Average-Case Complexity: the average number of
operations used to solve the problem over all possible
inputs of a given size is found in this type of analysis.

Trần Hòa Phú Chapter 3: Algorithms

Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Consider the linear search algorithm
procedure linearsearch (x : integer, a1 , ..., an : distinct integers)
i := 1
While (i ≤ n and x ̸= ai )
i := i + 1
if i ≤ n then location i := i
else location:=0
return location
Given the sequence 3 1 5 7 4 6. How many comparisions for
searching x = 7 ?
Trần Hòa Phú Chapter 3: Algorithms
Exercise procedure binary search (x : integer, a1 , ..., an : increasing
integers)
i := 1{i is left endpoint of search interval }
j := n{j is right endpoint of search interval }
While i < j
m := [(i + j)/2]
if x > am then i := m + 1
else j := m
if x = ai then location := i
else location := 0
return location
How many comparisons that are used if we use this algorithm to
search for x=3 in the list [1, 3, 4, 5, 6, 8, 9, 12, 17, 20, 26]?
Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Give the best big-O complexity of the following algorithm.
procedure giaithuat(a1, .., an : integers)
count:= 0
for i= 1 to n do
if ai > 0 then count := count + 1
print (count)

Trần Hòa Phú Chapter 3: Algorithms

 Exercise

Trần Hòa Phú Chapter 3: Algorithms

 Exercise

Trần Hòa Phú Chapter 3: Algorithms

 Exercise

A. O(n) B. O(n2 ) C. O(log n) D. O(n log n)

Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Give a big O estimate for the number of multiplications and
additions used in the following algorithm
procedure Horner(c, a0, a1, ..., an : real numbers)
y := an
for i := 1 to n
y := y ∗ c + an−i
√
A. O(n) B. O(1) C.O( n) D. O(log n)

Trần Hòa Phú Chapter 3: Algorithms

 Exercise

A. O(n) B.O(n2 ) C.O(n log n) D. O(1)

Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Let f be a function such that f (x ) = 3x 2 log x + 8x .
Find the least positive integer n such that f (x ) is O(x n )?
A.1 B.2 C.3 D.4

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Given the algorithm
procedure SUM(n: int)
t:=0
for i:=1 to n do
for j:=1 to n do
t := t ∗ i + j
return t
How many addtions and multiplications are used if n = 11?
A. 363 B.121 C.484 E.242

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Determine whether each of these functions is
a. 5 log x is O(x )
x 3 + 2x
b. is O(x 2)
2x + 1
c. x 2 + x 4 is O(x 3)
d. 2x is O(x 2)
Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Which of the following statements are true?
i) 5n = O(3n )
ii) (n2 + 1)(n + 1) = O(n2)
iii) n3 log(n) = O(n4)
A. all of i,ii, iii
B. i and iii
C. iii
D. i and ii
Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Find the least integer n such that f (x ) = O(x n )
(x 3 + 7x 2 + 3)2
f (x ) =
(2x + 1)2
A.1 B.3 C.4 D.2 E.None

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Given the Binary search algorithm as below. If input = 2, 4, 5, 7, 8, 9, 10, 13
and x = 6. After the second time of dividing into sublists, the sublist to be
considered is ....
A.7,8,9 B.5,7 C.5,7,8 D.2,4,5
procedure binary search (x : integer, a1 , ..., an : increasing integers)
i := 1{i is left endpoint of search interval }
j := n{j is right endpoint of search interval }
While i < j
m := [(i + j)/2]
if x > am then i := m + 1
else j := m
if x = ai then location := i
else location := 0
Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Given the following algorithm
procedure LS(x , a1, ..., an : integer)
i:=1
while (i ≤ n and x > ai )
i:=i+1
How many comparisions used in the algorithm with input
x = 10 and the sequence 2, 4, 6, 8, 10, 12?
A.11 B.10 C.12 D.6

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Which functions below is NOT O(n4)
i) f (n) = 1 + 23 + 33 + ... + (n + 2)3
ii) f (n) = (n + 4)5 − (n − 2)5
iii) f (n) = (n log n − 1)4
A. only i B. only ii C. only iii D. Both i,iii

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Let f (x ) = x 2 log x and g(x ) = x log(x 2).
Choose the correct statements.
i) f (x ) = O(g(x ))
ii) g(x ) = O(f (x ))
A. i
B. ii
C. Both i and ii
D. None of the other choices is correct

Trần Hòa Phú Chapter 3: Algorithms

 Exercise
Consider the following statements as given below
1) x 3 − x is O(x )
x5 − x3 + 1 2
2) is O(x )
x 3 − 2x
3) 12 + 22 + ... + n2 is O(n2)
4) 2(log x )3 − 2 is O(x ).
How many correct statements are there?
A.0 B.1 C.2 D.3
Trần Hòa Phú Chapter 3: Algorithms
 Exercise
Let f be a function such that f (x ) = 3x 2 log x + 8x .
Find the least positive integer n such that f (x ) is O(x n )?
A.1 B.2 C.3 D.4

Trần Hòa Phú Chapter 3: Algorithms