Chapter 3. Algorithms

Algorithms The Growth of Functions Complexity of Algorithms
Chapter 3. Algorithms
Mai Văn Duy
Ngày 17 tháng 9 năm 2022
Mai Văn Duy Chapter 3. Algorithms

Contents
1. Algorithms

Contents
1. Algorithms
2. The Growth of Functions

Contents
1. Algorithms
2. The Growth of Functions
3. Complexity of Algorithms

Example.
• Finding the Maximum Element in a Finite Sequence

{8, 4, 11, 3, 10}.

Example.

{8, 4, 11, 3, 10}.
• To search for 19 in the list
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22

Example.

{8, 4, 11, 3, 10}.
• To search for 19 in the list
1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22
• ...

Definition
An algorithm is a finite sequence of precise instructions for

performing a computation or for solving a problem.

Pseudocode
Pseudocode provides an intermediate step between an

English language description of an algorithm and an
implementation of this algorithm in a programming
language. The steps of the algorithm are specified using
instructions resembling those used in programming
languages.

PROPERTIES OF ALGORITHMS
• Input. An algorithm has input values from a specified

set.


set.
• Output. From each set of input values an algorithm
produces output values from a specified set (solution).


set.
• Definiteness. The steps of an algorithm must be
defined precisely.


set.
defined precisely.
• Correctness. An algorithm should produce the correct
output values for each set of input values.


set.
defined precisely.
• Finiteness. An algorithm should produce the desired
output after a finite (but perhaps large) number of
steps for any input in the set.


set.
defined precisely.
• Effectiveness. It must be possible to perform each step
of an algorithm exactly and in a finite amount of time.


set.
defined precisely.
• Effectiveness. It must be possible to perform each step
of an algorithm exactly and in a finite amount of time.
• Generality. The procedure should be applicable for all
problems of the desired form.
Finding the Maximum Element in a Finite

Sequence.

Example.
{8, 4, 11, 3, 10}

The Linear Search Algorithm

Example: Search for 9 in the list 2, 3, 4, 5, 6, 8, 9, 11:

The Binary(nhị phân) Search Algorithm

Example: Search for 19 in the list

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22

Sorting
Sorting is putting the elements into a list in which the

elements are in increasing order.

Sorting
Sorting is putting the elements into a list in which the

elements are in increasing order.
For instance, sorting the list 7, 2, 1, 4, 5, 9 produces the list
1, 2, 4, 5, 7, 9. Sorting the list d, h, c, a, f (using
alphabetical order) produces the list a, c, d, f, h.

The Bubble Sort

The Bubble Sort
Note: At the first stage, the largest element will be put in

deepest position. Similarly, at the second stage, one does
this for the largest element of the remain...

Example. Use the bubble sort to put 3, 2, 4, 1, 5 into

increasing order.

The Insertion Sort

Note: Begin with the second element. At the first stage

(j = 2), one find the right position of a2 in the first 2
elements of the sequence. If a2 ≤ a1 then one insert a2
into the front of a1 (a1 now is labeled as a2 ). In the
general, one find the right position of aj in list of the first j
elements and inserts it into this position (the remain must
be put 1 step backward).

Example. Use the insertion sort to put the elements of the

list 3, 2, 4, 1, 5 in increasing order.

Big-O Notation

Big-O Notation

Example.
Show that f (x ) = x 2 + 2x + 1 is O (x 2 ).


Big-O theorem
1. Let f (x ) = an x n + an−1 x n−1 + ... + a1 x + a0 where

a1 , a2 , ....an−1 , an are real numbers. Then, f (x ) is O (x n ).

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))
3. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 f2 )(x ) = O (g1 g2 (x ))

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))
3. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 f2 )(x ) = O (g1 g2 (x ))

Big-O theorem

2. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 + f2 )(x ) = O (max (|g1 (x )|, |g2 (x )|))
3. f1 (x ) = O (g1 (x )) ∧ f2 (x ) = O (g2 (x ))
=⇒ (f1 f2 )(x ) = O (g1 g2 (x ))
COROLLARY. Suppose that f1 (x ) and f2 (x ) are both

O (g (x )). Then (f1 + f2 )(x ) is O (g (x )).

The growth of functions commonly used in big-O

estimates.

Example.
1. Give a big-O estimate for

f (n) = 3n log(n!) + (n2 + 3) log n, where n is a positive
integer.

Example.

f (n) = 3n log(n!) + (n2 + 3) log n, where n is a positive
integer.
f (x ) = (x + 1) log(x 2 + 1) + 3x 2 .

Solution.

Quizz

Quizz

Big-Omega and Big-Theta Definition
• f (x ) = Ω(g (x )) if ∃C > 0, k : |f (x )|≥C |g (x )|, ∀x > k;


• f (x ) = Θ(g (x )) if [f (x ) = O (g (x ))] ∧ [f (x ) = Ω(g (x ))].




Example. Show that f (x ) = 1 + 2 + ... + n is Θ(n2 )


n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2


n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )


n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )
Notes.
1. f (x ) is Ω(g (x )) if and only if g (x ) is O (f (x )).


n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )
Notes.
2. If f (x ) = Θ(g (x )) then f(x) is of order g(x) or f(x) and
g(x) are of the same order (cùng bậc).


n(n + 1) n2
f (x ) = 1 + 2 + ... + n = =⇒ ≤ f ( x ) ≤ n2
2 2
=⇒ f (x ) = Ω(n2 ) ∧ f (x ) = Θ(n2 ) =⇒ f (x ) = Θ(n2 )
Notes.
2. If f (x ) = Θ(g (x )) then f(x) is of order g(x) or f(x) and
g(x) are of the same order (cùng bậc).
a1 , a2 , ....an−1 , an are real numbers and an 6= 0. Then,
f (x ) is Θ(x n ).

Introduction
• Computational complexity = Time complexity +

space complexity (not be considered here!).

Introduction

• Time complexity can be expressed in terms of the
number of operations used by the algorithm.

Introduction

1. Worst-case complexity (tells us how many operations
an algorithm requires to guarantee that it will produce
a solution.).

Introduction

1. Worst-case complexity (tells us how many operations
an algorithm requires to guarantee that it will produce
a solution.).
2. Average-case complexity (the average number of
operations used to solve the problem over all possible
inputs of a given size).

Example.
Describe the time complexity (Worst-case) of the

algorithm for finding the largest element in a set.

Example.

→ Number of comparisons f (n) = 2(n − 1) + 1.

Example.

→ Number of comparisons f (n) = 2(n − 1) + 1.Thus,

f (x ) = Θ (n).

Quizz

Chapter 3. Algorithms

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 3. Algorithms

Uploaded by

Copyright:

Available Formats

Algorithms The Growth of Functions Complexity of Algorithms

Mai Văn Duy

Ngày 17 tháng 9 năm 2022

Mai Văn Duy Chapter 3. Algorithms

Mai Văn Duy Chapter 3. Algorithms

Mai Văn Duy Chapter 3. Algorithms

Mai Văn Duy Chapter 3. Algorithms

• Finding the Maximum Element in a Finite Sequence

Mai Văn Duy Chapter 3. Algorithms

• Finding the Maximum Element in a Finite Sequence

Mai Văn Duy Chapter 3. Algorithms

• Finding the Maximum Element in a Finite Sequence

Mai Văn Duy Chapter 3. Algorithms

An algorithm is a finite sequence of precise instructions for

Mai Văn Duy Chapter 3. Algorithms

Pseudocode provides an intermediate step between an

Mai Văn Duy Chapter 3. Algorithms

• Input. An algorithm has input values from a specified

Mai Văn Duy Chapter 3. Algorithms

• Input. An algorithm has input values from a specified

Mai Văn Duy Chapter 3. Algorithms

• Input. An algorithm has input values from a specified

Mai Văn Duy Chapter 3. Algorithms

• Input. An algorithm has input values from a specified

Mai Văn Duy Chapter 3. Algorithms

• Input. An algorithm has input values from a specified

Mai Văn Duy Chapter 3. Algorithms

• Input. An algorithm has input values from a specified

Mai Văn Duy Chapter 3. Algorithms

• Input. An algorithm has input values from a specified

Finding the Maximum Element in a Finite

Mai Văn Duy Chapter 3. Algorithms

{8, 4, 11, 3, 10}

Mai Văn Duy Chapter 3. Algorithms

The Linear Search Algorithm

Mai Văn Duy Chapter 3. Algorithms

Example: Search for 9 in the list 2, 3, 4, 5, 6, 8, 9, 11:

Mai Văn Duy Chapter 3. Algorithms

The Binary(nhị phân) Search Algorithm

Mai Văn Duy Chapter 3. Algorithms

Example: Search for 19 in the list

Mai Văn Duy Chapter 3. Algorithms

Sorting is putting the elements into a list in which the

Mai Văn Duy Chapter 3. Algorithms

Sorting is putting the elements into a list in which the

Mai Văn Duy Chapter 3. Algorithms

The Bubble Sort

Mai Văn Duy Chapter 3. Algorithms

The Bubble Sort

Note: At the first stage, the largest element will be put in

Mai Văn Duy Chapter 3. Algorithms

Example. Use the bubble sort to put 3, 2, 4, 1, 5 into

Mai Văn Duy Chapter 3. Algorithms

The Insertion Sort

Mai Văn Duy Chapter 3. Algorithms

Note: Begin with the second element. At the first stage

Mai Văn Duy Chapter 3. Algorithms

Example. Use the insertion sort to put the elements of the

Mai Văn Duy Chapter 3. Algorithms

Mai Văn Duy Chapter 3. Algorithms

Mai Văn Duy Chapter 3. Algorithms

Mai Văn Duy Chapter 3. Algorithms

Mai Văn Duy Chapter 3. Algorithms