Ch4.

Algorithms

School of Electronics Engineering

Prof. Jeongsik Choi
 Contents

4.1 Introduction

4.2 Examples of Algorithms

4.3 Analysis of Algorithms

4.4 Recursive Algorithms

Prof. Jeongsik Choi 2

 Introduction
Algorithm
A step-by-step method of solving some problems

Typically refers to a solution that is executed by a computer

Ex. steps to find the maximum of three numbers 𝑎, 𝑏, 𝑐

1. 𝑙𝑎𝑟𝑔𝑒 = 𝑎

2. If 𝑏 > 𝑙𝑎𝑟𝑔𝑒, then 𝑙𝑎𝑟𝑔𝑒 = 𝑏

3. If 𝑐 > 𝑙𝑎𝑟𝑔𝑒, then 𝑙𝑎𝑟𝑔𝑒 = 𝑐

Prof. Jeongsik Choi 3

 Introduction
Characteristics of algorithms

Input and output

Precision – Steps are precisely stated

Determinism – Results of each step of execution are unique

Finiteness – Algorithm terminates

Correctness – Output is correct

Generality – Algorithm applies to a set of inputs

Prof. Jeongsik Choi 4

 Introduction
Pseudocode
A plain language that describes the steps in an algorithm

[Pseudocode] [C code] [Python code]

Prof. Jeongsik Choi 5

Introduction

Prof. Jeongsik Choi 6

Introduction

Prof. Jeongsik Choi 7

 Examples of Algorithms
Common algorithms
Searching
✓ Ex. Find the pattern “math” in the text “discrete mathematics”

Sorting
✓ Ex. Sort numbers stored in array [1, 3, 2, 6, 10, 1, 20]

Randomized algorithms
✓ Ex. Shuffle numbers stored in array [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Prof. Jeongsik Choi 8

Examples of Algorithms: Searching

Prof. Jeongsik Choi 9

 Examples of Algorithms: Searching
Example 4.2.2
Shows a trace of Text Search algorithm where we are searching for
the pattern “001” in the text “010001”

Prof. Jeongsik Choi 10

Examples of Algorithms: Sorting

Prof. Jeongsik Choi 11

Examples of Algorithms: Shuffle

Prof. Jeongsik Choi 12

 Analysis of Algorithms
Time and space complexity of an algorithm
Time complexity
✓ Time required to execute the algorithm

Space complexity
✓ (memory) space required to execute the algorithm

Performance of the algorithm evaluated by the time and space

complexities for the inputs of size 𝑛

Prof. Jeongsik Choi 13

Analysis of Algorithms

Prof. Jeongsik Choi 14

 Analysis of Algorithms
Different types of complexity

Best-case time/space

Worst-case time/space

Average-case time/space

Prof. Jeongsik Choi 15

 Analysis of Algorithms
Upper and lower bound of a function
Let 𝑓 and 𝑔 be functions with domain 𝐍

𝑓 𝑛 =𝑂 𝑔 𝑛 if ∃𝐶1 > 0 s.t. 𝑓 𝑛 ≤ 𝐶1 𝑔 𝑛 for ∀𝑛 ≥ 𝑁1

✓ 𝑓(𝑛) is of order at most 𝑔 𝑛
✓ 𝑓(𝑛) is big oh of 𝑔(𝑛)

𝑓 𝑛 =Ω 𝑔 𝑛 if ∃𝐶2 > 0 s.t. 𝑓 𝑛 ≥ 𝐶2 𝑔 𝑛 for ∀𝑛 ≥ 𝑁2

✓ 𝑓(𝑛) is of order at least 𝑔 𝑛
✓ 𝑓(𝑛) is omega of 𝑔(𝑛)

Prof. Jeongsik Choi 16

 Analysis of Algorithms
Upper and lower bound of a function (cont.)
If 𝑓 𝑛 = 𝑂 𝑔 𝑛 , 𝑔 is an asymptotic upper bound for 𝑓

If 𝑓 𝑛 = Ω 𝑔 𝑛 , 𝑔 is an asymptotic lower bound for 𝑓

If 𝑓 𝑛 = 𝑂 𝑔 𝑛 and 𝑓 𝑛 = Ω 𝑔 𝑛 ,

𝑓 𝑛 = Θ(𝑔 𝑛 )

𝑔 is an asymptotic tight bound for 𝑓

Prof. Jeongsik Choi 17

 Analysis of Algorithms
Example 4.3.3
𝑓 𝑛 = 60𝑛2 + 5𝑛 + 1

Prof. Jeongsik Choi 18

 Analysis of Algorithms
Theorem 4.3.4
Let 𝑝 𝑛 = 𝑎𝑘 𝑛𝑘 + 𝑎𝑘−1 𝑛𝑘−1 + ⋯ + 𝑎1 𝑛 + 𝑎0 be a polynomial with
𝑎𝑘 ≠ 0 and 𝑎𝑖 ≥ 0 for 𝑖 = 0, 1, 2 … , 𝑘 − 1
𝑝 𝑛 = Θ(𝑛𝑘 )

Prof. Jeongsik Choi 19

 Analysis of Algorithms
Example 4.3.5 lg 𝑛 = log 2 𝑛
𝑓 𝑛 = 2𝑛 + 3 lg 𝑛

Example 4.3.6
𝑓 𝑛 = log 𝑏 𝑛

Prof. Jeongsik Choi 20

 Analysis of Algorithms
Example 4.3.7
𝑓 𝑛 = 1 + 2+ ⋯+ 𝑛

Example 4.3.8
𝑓 𝑛 = 1𝑘 + 2𝑘 + ⋯ + 𝑛𝑘 with some 𝑘 ∈ 𝐍

Prof. Jeongsik Choi 21

 Analysis of Algorithms
Example 4.3.9
Show that lg 𝑛! = Θ 𝑛 lg 𝑛

Prof. Jeongsik Choi 22

 Analysis of Algorithms
Example 4.3.10
Show that if 𝑓 𝑛 = Θ 𝑔 𝑛 and 𝑔 𝑛 = Θ ℎ 𝑛 then 𝑓 𝑛 = Θ ℎ 𝑛

Prof. Jeongsik Choi 23

 Analysis of Algorithms
Definition 4.3.11
If an algorithm requires 𝑡(𝑛) units of time to terminate in the best
case for an input size 𝑛 and
𝑡 𝑛 =𝑂 𝑔 𝑛

we say that the best-case time required by the algorithm is 𝑂 𝑔 𝑛

Prof. Jeongsik Choi 24

 Analysis of Algorithms
Example 4.3.13
Find a theta notation in terms of 𝑛 for the number of times the
statement 𝑥 = 𝑥 + 1 is executed

Prof. Jeongsik Choi 25

 Analysis of Algorithms
Example 4.3.14
Find a theta notation in terms of 𝑛 for the number of times the
statement 𝑥 = 𝑥 + 1 is executed

Prof. Jeongsik Choi 26

Analysis of Algorithms

Prof. Jeongsik Choi 27

Analysis of Algorithms

Prof. Jeongsik Choi 28

 Recursive Algorithms
Recursive function
A function that invokes itself
Show that this algorithm
returns 𝑛!

Prof. Jeongsik Choi 29

Recursive Algorithms
𝑤𝑎𝑙𝑘 1 , 𝑤𝑎𝑙𝑘 2 , … , are
Fibonacci sequence

Prof. Jeongsik Choi 30

Prof. Jeongsik Choi 31
 Recursive Algorithms
Example 4.4.7
If 𝑓1 = 1, 𝑓2 = 2, 𝑓𝑛 = 𝑓𝑛−1 + 𝑓𝑛−2 (𝑛 ≥ 2), use induction to show that
𝑛

෍ 𝑓𝑘 = 𝑓𝑛+2 − 1 , ∀𝑛 ≥ 1
𝑘=1

Prof. Jeongsik Choi 32