L00

Introduction
SUTD ISTD 50.004 Algorithm
Chong Kai Fong Ernest
Cyrille Pierre Joseph Jegourel, Soh De Wen, Wang Yixiao
SUTD

SUTD ISTD 50.004 Algorithms 1

Summary
– Why should we study “Algorithms”?
Correctness, speed, real-life examples.
– Grading, course material, and teaching team.
– Basic definitions of “Algorithms” and time complexity.
T(n), n, computational problem, algorithm, input space, output space, input
instance, etc.
– Asymptotic complexity (Θ, O, Ω)
Notation, how to tell the asymptotic complexity of functions.
– The exact definitions.
Definitions of Θ, O, Ω; a useful theorem on complexity.

SUTD ISTD 50.004 Algorithms 2

Why should we study algorithms?

1, You want your algorithm to terminate with a

correct output.
2, Speed Matters!
• Speed is time.
• Speed is money.
• Speed is productivity.
……

SUTD ISTD 50.004 Algorithms 3

Why should we study algorithms?

• You are a consultant of a company.

• One day, your manager comes to you…

Why does my stock analysis

program run so slowly?
It needs about 157 years to
complete the calculation!

Manager

SUTD ISTD 50.004 Algorithms 4

Why should we study algorithms?

• You are a consultant of a company.

• One day, your manager comes to you…

Because you used a brute force

algorithm!
You should use a divide and
conquer algorithm instead.

Manager

You
SUTD ISTD 50.004 Algorithms 5
Why should we study algorithms?

• You are a consultant of a company.

• One day, your manager comes to you…

After one day of re-coding,

the program can complete any computation in 10 minutes.

• 157 years → 10 minutes

• Save more time for a happy life ☺
SUTD ISTD 50.004 Algorithms 6
Why should we study algorithms?

• You are a website developer of a company.

• One day, your manager comes to you…

Why does our website run so slow?

It takes more than 10 seconds to
search for a keyword on the website!
It brings a very bad user experience!

Manager

SUTD ISTD 50.004 Algorithms 7

Why should we study algorithms?

• You are a website developer of a company.

• One day, your manager comes to you…

32% of users will walk away from

a brand they like after a single
bad user experience.
86% of users will pay more for
better user experience.

Manager

You
SUTD ISTD 50.004 Algorithms 8
Why should we study algorithms?

• You are a website developer of a company.

• One day, your manager comes to you…

After one day of re-coding,

the program can complete any computation in 0.2 seconds….

• 10 seconds → 0.2 seconds

• A better website for happier customers ☺
SUTD ISTD 50.004 Algorithms 9
Why should we study algorithms?
My own experience as a parametric designer:
You can even create your own position because
of faster algorithms.

We want to create a position for

you since your algorithm runs
much faster than others!

I will think about it …

You
SUTD ISTD 50.004 Algorithms 10
What is an algorithm?
• An algorithm is any well-defined computational
procedure that takes some value, or set of values,
as input and performs some action and/or
produces some value, or set of values, as output.
• An algorithm is thus a sequence of computational
steps that transforms the input.
• Algorithms, like computer software, hardware,
user interfaces, etc., is a kind of technology.

SUTD ISTD 50.004 Algorithms 11

What is an algorithm?

• An algorithm is a procedure for solving problems.

e.g. how do you sort a cart of books

in increasing order of the volume
number? (i.e. volume 1, volume 2,
volume 3…)

- Brute force algorithm: Find

volume 1, then find volume 2,
then volume 3, etc.
- Clever algorithm: divide the
books into two halves, sort the
first half, sort the second half,
merge both halves.

SUTD ISTD 50.004 Algorithms 12

What is an algorithm?

• An algorithm is a procedure for solving problems.

e.g. How to find the best travelling

time between any two stations?

- Brute force algorithm: manually

find the travelling tie between
every two stations.
- Clever algorithm: just record the
travelling time between
consecutive stations, then use
Dijkstra’s shortest path algorithm.

SUTD ISTD 50.004 Algorithms 13

SUTD ISTD 50.004 Algorithms 14
What is a correct algorithm?
• An algorithm is said to be correct if, for every
input, it halts with the correct output returned,
or the correct action performed.
• Incorrect algorithms may not halt on some
inputs, or it might halt with an incorrect output
returned or an incorrect action performed.
• Incorrect algorithms can sometimes be useful if
we can control their error rate. However, we shall
normally only be concerned with correct
algorithms.
SUTD ISTD 50.004 Algorithms 15
Algorithm Efficiency
• Definition: the resources needed to run an
algorithm on a computer.
• “Resources” include: time, computer’s memory,
computer’s hardware, etc.
• The differences between algorithms can be much
more significant than differences due to
hardware and software.

SUTD ISTD 50.004 Algorithms 16

What we just briefly covered:
• Algorithm?
• Correct Algorithm?
• Good/Bad Algorithm? (Algorithm Efficiency?)
• Why should we study algorithms?

SUTD ISTD 50.004 Algorithms 17

Course material
All course materials can be found on eDimension.

Textbook
• Introduction to Algorithms
– by Cormen, Leiserson, Rivest, and Stein (CLRS)
• Very very useful, almost the whole world is using this
book
• For pre-reading and revision, I won’t go through the
book in class
• But most of my materials are all based on this book
• Highly recommend to read the whole chapter of the
course related topics.
SUTD ISTD 50.004 Algorithms 18
Grading
• 6 weekly homework: 15%
– Each homework will take at most 2 hours of your time.
• 2 Problem sets: 20%
– More difficult than homework sets, and will take at most 24
hours.
– due week 5 and week 11.
• 2 Tests: 30%
– The first test is in week 6.
• Project 2D: 5%
– For ISTD students only.
• Final Exam: 30%
– Week 14
• Course Evaluation Survey: Bonus 2%
SUTD ISTD 50.004 Algorithms 19
Closest deadlines:
• Homework Set 1 due next week (31 Jan, Monday, 1pm, online
submission).
• Homework Set 2 due following week (7 Feb, Monday, 1pm, online
submission).

SUTD ISTD 50.004 Algorithms 20

The teaching team
Instructors
• Chong Kai Fong Ernest
• Email: ernest_chong@sutd.edu.sg

• Cyrille Pierre Joseph Jegourel

• Email: cyrille_jegourel@sutd.edu.sg

• Soh De Wen
• Email: dewen_soh@sutd.edu.sg

• Wang Yixiao
• Email: yixiao_wang@sutd.edu.sg

Teaching Assistants:
Li Zihao (Masters): zihao_li@sutd.edu.sg
Anirudh Shrinivason (Undergraduate): anirudh_shrinivason@mymail.sutd.edu.sg
Daniel Low Yu Hian (Undergraduate): daniel_low@mymail.sutd.edu.sg
Huang He (Undergraduate): he_huang@mymail.sutd.edu.sg
Jin Ziqi (Undergraduate): ziqi_jin@mymail.sutd.edu.sg
Leong Yun Qin Melody (Undergraduate): melody_leong@mymail.sutd.edu.sg
Sun Kairan (Undergraduate): kairan_sun@mymail.sutd.edu.sg

SUTD ISTD 50.004 Algorithms 21

L01.01
Complexity, Asymptotic notation
50.004 Introduction to Algorithm
Dr. Wang Yixiao
ISTD/DAI, SUTD

SUTD ISTD 50.004 Algorithms 22

Basic definitions
• What is a computational problem?
Map an input (e.g. x) to an output (e.g. f(x)).

SUTD ISTD 50.004 Algorithms 23

Basic definitions
• e.g. let say, f(x) = shortest distance from Orchard
to x, then
– If x = “Expo”, f(x) = shortest distance from Orchard to
Expo
– If x = “Changi”, f(x) = shortest distance from Orchard to
Changi
• So, given x, we want to COMPUTE f(x).
• A good algorithm can compute f(x) quickly.

SUTD ISTD 50.004 Algorithms 24

 Basic definitions
• What is an input space?
– Set of possible input (e.g. input space = {expo, changi, Tanah
Merah})
• What is an output space?
– Set of possible output (e.g. output space = any positive number)
• What is an input instance?
– A particular input of a problem instance (e.g. x = expo)
1min
Expo
2 min
Changi 3min
TanahMerah
4 min
input space
5min
SUTD ISTD 50.004 Algorithms ... output space 25
 Basic definitions
• What is an input space?
– Set of possible input (e.g. input space = {expo, changi, Tanah
Merah})
• What is an output space?
– Set of possible output (e.g. output space = any positive number)
• What is an input instance?
– A particular input of a problem instance (e.g. x = expo)

input instance x
= Expo
f (x) = 21min

input space output space

SUTD ISTD 50.004 Algorithms 26

Exercise 1
• Define the input + output space, and give an example of
an input instance for the following problems
1. Integer multiplication
2. Find if a given integer k is in a list of n integers [k1,k2,…,kn]
3. Sort a list of n integers in increasing order

• What is the “size” of the input instance in each case?

“Bigger” question = more work required to answer

SUTD ISTD 50.004 Algorithms 27

Exercise 1 Answers
• Define the input + output space, and give an example of
an input instance for the following problems
1. Input Space: the set of 2-dimensional integer vectors; Output
Space: the set of integers; An input instance: [1, 2]
2. Input Space: the set of integers; Output Space: {Yes, No}; An
input instance: [10].
3. Input Space: the set of all the possible lists of n integers;
Output Space: the set of all the lists of n integers in an
increasing order; An input instance: [n, n-1, n-2, … 1].
• What is the “size” of the input instance in each case?
2, 1, n

SUTD ISTD 50.004 Algorithms 28

Algorithm
• Procedure for solving a computational problem.
• Usually a finite sequence of operations
– described in flow charts
– described in structured English
– Written in pseudocode or real code

SUTD ISTD 50.004 Algorithms 29

Property of algorithm

f (x) | x | = "size" of problem instance = n

x y T (n) = number of steps to solve
A
the problem as a function of its size

SUTD ISTD 50.004 Algorithms 30

A Good Algorithm
• Correct
• Fast
– T(n) should increase “slowly” as n grows.

SUTD ISTD 50.004 Algorithms 31

Algorithmic time complexity T(n)
Example P: Is a certain number k in a list of n numbers?
[k1 , k2 ,¼, kn ]ü
ý yes/no
kþ

• Algorithm A1: check for each element ki of the list if ki = k

• How much “time” (i.e. T(n)) does A1 takes?
(input + loop + output)

• T(n) is the exact number of steps it takes to run A1 (e.g.

1002 times, 3045 times)

SUTD ISTD 50.004 Algorithms 32

Asymptotic complexity
• The asymptotic complexity describes T(n), as n
grows to infinity.

• In this course, we talk about THREE types of

asymptotic complexity.
– Θ (Big Theta)
– O (Big O)
– Ω (Big Omega)

SUTD ISTD 50.004 Algorithms 33

Before we continue…
Go through the following concepts in your head:
• What is T(n)?
• What is n?
• What is a computational problem?
• What is an algorithm?

SUTD ISTD 50.004 Algorithms 34

Asymptotic complexity
– Θ (Big Theta) means “grows asymptotically = ”
Grows as fast as …
– O (Big O) means “grows asymptotically <=“
Grows at most as fast as …
– Ω (Big Omega) means “grows asymptotically >=“
Grows at least as fast as …

SUTD ISTD 50.004 Algorithms 35

Asymptotic complexity
– Θ (Theta) means “grows asymptotically equal ”
– For example
n = Q(n )
2 2

– (when n increases, 𝑛2 grows in the same level of speed

as 𝑛2 )
– For example
0.1n2 -100n1.9 + 5 = Q(n 2 )

• F(x) = Θ (G(x)) means “F grows (equally) as G, when

x grows to infinity”
SUTD ISTD 50.004 Algorithms 36
Asymptotic complexity
– O (Big O) means “grows asymptotically <=“
– For example
n = O(n
2 1000
)

– For example
2n3 +100n2 + 5 = O(n3000000 )

• F(x) = O(G(x)) means “F grows at most as fast as G,

when x grows to infinity”

SUTD ISTD 50.004 Algorithms 37

Asymptotic complexity
– Ω (Omega) means “grows asymptotically >=“
– For example
n 9999
= W(1)

– For example
2n 999999 +100n33 + 5 = W(n2 )

• F(x) = Ω (G(x)) means “F grows at least as fast as G,

when x grows to infinity”

SUTD ISTD 50.004 Algorithms 38

Three Key Tricks
• Think about at what speed levels a given function
grows, instead of exactly how fast it grows.
• Pay attention to the item that grows at the
highest speed level, since it will dominate the
growing speed of the whole function.
• What you time may change the level of growing
speed of an item, but what you plus or minus
doesn’t influence an item’s level of growing
speed.

SUTD ISTD 50.004 Algorithms 39

Quick Exercises

(1.01) = ?(x log x)

x 10

x log log x = ?(x ) 1.5

x + x(log x) = ?(x )
2 2 2

Hints, asymptotic increasing order (quick to slow)

log(log x), logx, x, x2, x10, x99999, 2x, xx

SUTD ISTD 50.004 Algorithms 40

Exact definitions
f (n) = O(g(n)) Û
$D > 0, n0 such that
f (n) £ D g(n) for n ³ n0

Intuition: I am g(n) and you are f(n). If I am faster than you, then
no matter how far I am behind you now, one day I will surpass you.

f(n) and g(n) should be asymptotically non-negative. That means

when n is large enough, f(n) and g(n) should all ≥ 0

SUTD ISTD 50.004 Algorithms 41

Exact definitions
f (n) = O(g(n)) Û
$D > 0, n0 such that
f (n) £ D g(n) for n ³ n0

f (n) = W(g(n)) Û
$D > 0, n0 such that
f (n) ³ D g(n) for n ³ n0

Intuition: I am g(n) and you are f(n). If you are faster than me, then
no matter how far I am ahead of you now, one day you will surpass
me.
SUTD ISTD 50.004 Algorithms 42
Exact definitions

f (n) = Q(g(n)) Û
$D1, D2 > 0, n0 such that
D1 g(n) ³ f (n) ³ D2 g(n) for n ³ n0

SUTD ISTD 50.004 Algorithms 43

For a given function g(n), we denote by O(g(n)) the set of functions:

For a given function g(n), we denote by Ω(g(n)) the set of functions:

For a given function g(n), we denote by θ(g(n)) the set of functions:

SUTD ISTD 50.004 Algorithms 44

According to the definition, there are constant C1, C2, and a big enough 𝑛0 so that
when n ≥ 𝑛0 , 0 ≤ C1*g(n) ≤ f(n) ≤ C2*g(n).
This means there is constant C1 and a big enough 𝑛0 so that when n ≥ 𝑛0 , 0 ≤
C1*g(n) ≤ f(n), which is the definition of f(n) = Ω(g(n)).
This also means there is a constant C2 and a big enough 𝑛0 so that when n ≥ 𝑛0 , 0
≤ f(n) ≤ C2*g(n), which is the definition of f(n) = O(g(n)).

And for the “vice versa”:

Since f(n) = Ω(g(n)), there is constant C1 and a big enough 𝑛0 so that when n ≥ 𝑛0 ,
0 ≤ C1*g(n) ≤ f(n).
Since f(n) = O(g(n)), there is a constant C2 and a big enough 𝑛1 so that when n ≥ 𝑛1 ,
0 ≤ f(n) ≤ C2*g(n).
Let 𝑛2 = max (𝑛0 , 𝑛1 )
Thus, there are constant C1, C2, and a big enough constant 𝑛2 so that when n ≥ 𝑛2 ,
0 ≤ C1*g(n) ≤ f(n) ≤ C2*g(n), which is the definition of f(n) = θ(g(n)).

SUTD ISTD 50.004 Algorithms 45

Exact definitions
Examples
f (n) = 100n +1000
g(n) = n 2 D=1
𝑛0 = 200
so, f (n) = O(g(n))

f (n) = 0.001n D=0.001

g(n) = n 0.5 +1000 𝑛0 = 10000
so, f (n) = W(g(n))

D1=3
f (n) = 2n +100n -1000
2
D2=1
g(n) = n 2 𝑛0 = 100
so, f (n) = Q(g(n))
SUTD ISTD 50.004 Algorithms 46
Complexity of algorithms

In simple and practical words used in the industry

Θ(logn): great algorithm
Θ(n), Θ(n logn), Θ(n2): good algorithm
Θ(n3): hard to say….
Θ(2n): bad algorithm
SUTD ISTD 50.004 Algorithms 47
Asymptotic complexity in practice
Practically
Data Size in μs “Practical”
ineffective
Algorithm
algorithm

The time needed to solve a problem

SUTD ISTD 50.004 Algorithms 48

Summary
– Why should we study “Algorithms”?
Correctness, speed, real-life examples.
– Grading, course material, and teaching team.
– Basic definitions of “Algorithms” and time complexity.
T(n), n, computational problem, algorithm, input space, output space, input
instance, etc.
– Asymptotic complexity (Θ, O, Ω)
Notation, how to tell the asymptotic complexity of functions.
– The exact definitions.
Definitions of Θ, O, Ω; a useful theorem on complexity.

SUTD ISTD 50.004 Algorithms 49