Course Introduction

“Big-O”
Analyzing Efficiency
⬣ Measure both time and space
efficiency
⬣ Platform/hardware independent
⬣ High level description of
algorithm
⬣ Performance scales in relation
to the input sizes
⬣ The complexity of an algorithm
is how efficient the algorithm is
in terms of input sizes
High Level Description
⬣ Primitive operations:
⬣ Assign Value
⬣ Arithmetic operation
⬣ Comparing two entities
⬣ Method call or return from
method
⬣ Access element or follow a
reference

⬣ Primitive operations execute

in constant time
⬣ Count the operations
Measuring Efficiency as
a Function
⬣ f(n) represents the function of
primitive operations on an
input of size n
⬣ There are three cases:
⬣ Worst-Case - the algorithm running with
the worst set of data, worst
performance
⬣ Best-Case - the algorithm running with
the best designed set of data, fastest
performance
⬣ Average-Case - somewhere between
and difficult to compute
⬣ We want the most accurate
worst-case analysis for Big-O
Big-O Notation
⬣ Denoted O(f(n)) where n is the size of our input(s), f(n) is a function
representing the scaling of the algorithm with n
⬣ Typically represents an upper bound, but for this class we want the
most accurate upper bound
4
⬣ Example: Everything in this course is O(n4), but this is not a very
helpful description of performance
⬣ Other Measures: o(n), Ω(n), Ө(n), etc

Big-O Review
Common Complexities (Closer look)

Complexity O(n3) O(2n)

Big-O Notation
Name
Constant O(1)

Logarithmic O(log(n)) O(n) O(n log(n))

4
Linear O(n)

Log-Linear O(nlog(n))

Quadratic O(n2) O(n2)

Cubic O(n3) O(1)
Exponential O(an), a constant O(log(n))

Big-O Review
Example: 100 operations with 1000 inputs
Algorithm
N = 1000 O(2n) O(n3)
Complexity
O(1) constant

O(log(n)) 0.09966
4 O(n2)
O(n) 10

O(nlog(n)) 99.66

O(n2) 10,000 O(n log(n))

O(n3) 100,000,000 O(n)
O(log(n))
O(2n) 1.07 × 10299 O(1)

Big-O Review
Big-O Conventions

⬣ Drop Constant Factors:

O(5n) → O(n) O(0.5n2) → O(n2)

⬣ Drop Lower Order Terms: 4

O(n2 + 1000n - 3) → O(n2) O(3n + 2log(n) + nlog(n)) → O(nlog(n))

Big-O Review
O(1) – Constant Complexity

⬣ Performance does not scale at all with input size.

⬣ Example: Given an array of length n, returning the first element is
O(1).
4

Can always 5 1 3 2 4
immediately
access first
1 4 2 9 4 2 1 3 3 7
element.

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4

5 1 3 2 4

1 4 2 9 4 2 1 3 3 7

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4
5
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4
6
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7
5

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
9 4

5 1 3 2 4

1 4 2 9 4 2 1 3 3 7
7

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4
11
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7
16

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4 15
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7

20

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4 15
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7

22

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4 15
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7

23

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4 15
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7
26

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4 15
5 1 3 2 4

1 4 2 9 4 2 1 3 3 7

29

Big-O Review
O(n) – Linear Complexity

⬣ Performance scales proportionally with the input size.

⬣ Example: Given an array of length n, summing up all elements in
the array.
4 15
Second array
took twice as 5 1 3 2 4
long when
doubling array 1 4 2 9 4 2 1 3 3 7
length.
36

Big-O Review
O(log(n)) – Logarithmic Complexity

⬣ Performance scales logarithmically with input size.

⬣ The base doesn’t matter due to change of base (m constant):
log2(n)
_______
logm(n) = = Clog2(n) → O(logm(n)) → O(log(n))
log2(m) 4
⬣ Example: Given a sorted array of length n, performing a binary
search.

Big-O Review