You are on page 1of 424

# Why?

## • You don’t need deep knowledge of algorithms & data

structures to be a software engineer

## • However, they are fundamental to a real

understanding of what is happening when you write
some code.
What is an algorithm?
What is an algorithm?

## A list of actions, to be followed in order, to achieve some

goal
Agenda
1. Calculating Runtimes (Big O)
2. Algorithms
A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures
C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

## I. Binary Search Tree

J. Hash Table

K. Set

L. Graph

M. Heap
Runtimes
Collection<String> animals = new ArrayList<>();
// ….
animals.contains("dog");

Vs

## Collection<String> animals = new Set<>();

// ….
animals.contains("dog");
Runtimes
You’ll need some Algebra to fully understand Big
O and asymptotic analysis.

## If your algebra is rusty, there are a lot of online

resources to review

## Udacity College Algebra

Fibonacci Sequence

F20 = 6765
F50 = 12586269025
F100 = 354224848179261915075
F500 =
139423222456169788013972438287040728
395007025658769730726410896294832557
1622863290691557658876222521294125
Fibonacci Sequence
def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)

Vs

def FibList(n):
create an array F[0…n]
F[0] = 0
F[1] = 1
for i from 2 to n:
F[i] = F[i - 1] + F[i - 2]
return F[n]
How fast is this?

def FibList(n):
create an array F[0…n]
F[0] = 0
F[1] = 1
for i from 2 to n:
F[i] = F[i - 1] + F[i - 2]
return F[n]
T(n)

def FibList(n):
create an array F[0…n] 1
F[0] = 0 1
F[1] = 1 1
for i from 2 to n: n *
F[i] = F[i - 1] + F[i - 2] 1
return F[n] 1
Common runtimes
O(1) Constant

O(log n) Logarithmic

O(n) Linear

## O(n log n) Linearithmic

O(n3) Cubic

O(2n) Exponential

https://www.desmos.com/calculator/ty01lmuivl
O(n)

def FibList(n):
create an array F[0…n] O(n)
F[0] = 0 O(1)
F[1] = 1 O(1)
for i from 2 to n: O(n) *
F[i] = F[i - 1] + F[i - 2] O(1)
return F[n] O(1)

## O(n) + O(1) + O(1) + O(n) * O(1) + O(1) = ???

Recursive

def FibRecurse(n):
if n <= 1: O(1)
return n O(1)
else: O(1)
return FibRecurse(n - 1) + FibRecurse(n - 2) O(???)
Recursive
4

def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Recursive
4

3 2

def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Recursive
4

3 2

2 1

def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Recursive
4

3 2

2 1

1 0

def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Recursive
4

3 2

2 1 1 0

1 0

def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Recursive
4

3 2

2 1 1 0

1 0

def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Recursive
5

4 3

3 2 2 1

2 1 1 0 1 0

1 0

def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Recursive
5

4 3

3 2 2 1

2 1 1 0 1 0

1 0

O(2n)
def FibRecurse(n):
if n <= 1:
return n
else:
return FibRecurse(n - 1) + FibRecurse(n - 2)
Common runtimes
Common runtimes
Agenda
1. Calculating Runtimes (Big O)
2. Algorithms
A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures
C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

## I. Binary Search Tree

J. Hash Table

K. Set

L. Graph

M. Heap
Searching

3 4 5 7 1 8 9 6 2
Searching

3 4 5 7 1 8 9 6 2

## How many steps does it take to check if the

number 6 is in this collection?
Linear Search

6?

3 4 5 7 1 8 9 6 2
Linear Search

6?

3 4 5 7 1 8 9 6 2
Linear Search

6?

3 4 5 7 1 8 9 6 2
Linear Search

6?

3 4 5 7 1 8 9 6 2
Linear Search

6?

3 4 5 7 1 8 9 6 2
Linear Search

6?

3 4 5 7 1 8 9 6 2
Linear Search

6?

3 4 5 7 1 8 9 6 2
Linear Search

6?

3 4 5 7 1 8 9 6 2
Searching

1 2 3 4 5 6 7 8 9

## What about if the collection is sorted?

How many steps to find if number 6 is in this
collection?
Binary Search

6?

1 2 3 4 5 6 7 8 9
Binary Search

6?

1 2 3 4 5 6 7 8 9
5 is less than 6, so everything to the left of 5 can be ignored
Binary Search

6?

1 2 3 4 5 6 7 8 9
Binary Search

6?

1 2 3 4 5 6 7 8 9
7 is greater than 6, so everything to the right of 7 can be ignored
Binary Search

6?

1 2 3 4 5 6 7 8 9
Binary Search

1 2 3 4 5 6 7 8 9

In this case, the search took 3 steps, rather than the 9 steps using linear
search.

## In a collection of 1 million elements, linear search will take time

proportional to 1 million, binary search will take time proportional to 20
Sorting

3 4 5 7 1 8 9 6 2
Selection Sort

3 4 5 7 1 8 9 6 2
Selection Sort

3 4 5 7 1 8 9 6 2
Selection Sort

1 4 5 7 3 8 9 6 2
Selection Sort

1 4 5 7 3 8 9 6 2
Selection Sort

1 4 5 7 3 8 9 6 2
Selection Sort

1 2 5 7 3 8 9 6 4
Selection Sort

1 2 5 7 3 8 9 6 4
Selection Sort

1 2 3 7 5 8 9 6 4
Selection Sort

1 2 3 7 5 8 9 6 4
Selection Sort

1 2 3 4 5 8 9 6 7
Selection Sort

1 2 3 4 5 8 9 6 7
Selection Sort

1 2 3 4 5 8 9 6 7
Selection Sort

1 2 3 4 5 8 9 6 7
Selection Sort

1 2 3 4 5 6 9 8 7
Selection Sort

1 2 3 4 5 6 9 8 7
Selection Sort

1 2 3 4 5 6 7 8 9
Selection Sort

1 2 3 4 5 6 7 8 9
Selection Sort

1 2 3 4 5 6 7 8 9
Selection Sort

1 2 3 4 5 6 7 8 9
Selection Sort

1 2 3 4 5 6 7 8 9

## What is the complexity of selection sort?

Insertion Sort

3 4 5 7 1 8 9 6 2
Insertion Sort

3 4 5 7 1 8 9 6 2
Insertion Sort

3 4 5 7 1 8 9 6 2
Insertion Sort

3 4 5 7 1 8 9 6 2
Insertion Sort

1
3 4 5 7 8 9 6 2
Insertion Sort

1
3 4 5 7 8 9 6 2
Insertion Sort

1 3 4 5 7 8 9 6 2
Insertion Sort

1 3 4 5 7 8 9 6 2
Insertion Sort

1 3 4 5 7 8 9 6 2
Insertion Sort

6
1 3 4 5 7 8 9 2
Insertion Sort

6
1 3 4 5 7 8 9 2
Insertion Sort

1 3 4 5 6 7 8 9 2
Insertion Sort

2
1 3 4 5 6 7 8 9
Insertion Sort

2
1 3 4 5 6 7 8 9
Insertion Sort

1 2 3 4 5 6 7 8 9
Insertion Sort

1 2 3 4 5 6 7 8 9

## What is the complexity of insertion sort?

Bubble Sort

3 4 5 7 1 8 9 6 2
Bubble Sort

3 4 5 7 1 8 9 6 2
Bubble Sort

3 4 5 7 1 8 9 6 2
Bubble Sort

3 4 5 7 1 8 9 6 2
Bubble Sort

3 4 5 1 7 8 9 6 2
Bubble Sort

3 4 5 1 7 8 9 6 2
Bubble Sort

3 4 5 1 7 8 9 6 2
Bubble Sort

3 4 5 1 7 8 9 6 2
Bubble Sort

3 4 5 1 7 8 6 9 2
Bubble Sort

3 4 5 1 7 8 6 9 2
Bubble Sort

3 4 5 1 7 8 6 2 9
Bubble Sort

3 4 5 1 7 8 6 2 9
Bubble Sort

3 4 5 1 7 8 6 2 9
Bubble Sort

3 4 5 1 7 8 6 2 9
Bubble Sort

3 4 1 5 7 8 6 2 9
Bubble Sort

3 4 1 5 7 8 6 2 9
Bubble Sort

3 4 1 5 7 8 6 2 9
Bubble Sort

3 4 1 5 7 8 6 2 9
Bubble Sort

3 4 1 5 7 6 8 2 9
Bubble Sort

3 4 1 5 7 6 8 2 9
Bubble Sort

3 4 1 5 7 6 2 8 9
Bubble Sort

3 4 1 5 7 6 2 8 9
Bubble Sort

3 4 1 5 7 6 2 8 9
Bubble Sort

3 4 1 5 7 6 2 8 9
Bubble Sort

3 1 4 5 7 6 2 8 9
Bubble Sort

3 1 4 5 7 6 2 8 9
Bubble Sort

3 1 4 5 7 6 2 8 9
Bubble Sort

3 1 4 5 7 6 2 8 9
Bubble Sort

3 1 4 5 6 7 2 8 9
Bubble Sort

3 1 4 5 6 7 2 8 9
Bubble Sort

3 1 4 5 6 2 7 8 9
Bubble Sort

3 1 4 5 6 2 7 8 9
Bubble Sort

3 1 4 5 6 2 7 8 9
Bubble Sort

3 1 4 5 6 2 7 8 9
Bubble Sort

1 3 4 5 6 2 7 8 9
Bubble Sort

1 3 4 5 6 2 7 8 9
Bubble Sort

1 3 4 5 6 2 7 8 9
Bubble Sort

1 3 4 5 6 2 7 8 9
Bubble Sort

1 3 4 5 6 2 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 5 2 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 4 2 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 3 2 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9
Bubble Sort

1 2 3 4 5 6 7 8 9

## What is the complexity of bubble sort?

Merge Sort

3 4 5 7 1 8 9 6 2
Merge Sort
3 4 5 7 1 8 9 6 2
Merge Sort
3 4 5 7 (magic)
1 2 6 8 9
Merge Sort
3 4 5 7 1 2 6 8 9
Merge Sort
3 4 5 7 2 6 8 9

1
Merge Sort
3 4 5 7 6 8 9

1 2
Merge Sort
4 5 7 6 8 9

1 2 3
Merge Sort
5 7 6 8 9

1 2 3 4
Merge Sort
7 6 8 9

1 2 3 4 5
Merge Sort
7 8 9

1 2 3 4 5 6
Merge Sort
8 9

1 2 3 4 5 6 7
Merge Sort
9

1 2 3 4 5 6 7 8
Merge Sort

1 2 3 4 5 6 7 8 9
What is the complexity of the merge?
Merge Sort

Merge Sort

## How do we sort the sub lists?

Merge Sort!
Merge Sort
1 8 9 6 2

3 4 5 7
Merge Sort
1 8 9 6 2

3 4 5 7
Merge Sort
1 8 9 6 2

5 7

3 4
Merge Sort
1 8 9 6 2

5 7

3 4
Merge Sort
1 8 9 6 2

5 7

3 4
Merge Sort
1 8 9 6 2

5 7

3
Merge Sort
1 8 9 6 2

5 7

3 4
Merge Sort
1 8 9 6 2

3 4 5 7
Merge Sort
1 8 9 6 2

3 4

5 7
Merge Sort
1 8 9 6 2

3 4

5 7
Merge Sort
1 8 9 6 2

3 4

5 7
Merge Sort
1 8 9 6 2

3 4

5
Merge Sort
1 8 9 6 2

3 4

5 7
Merge Sort
1 8 9 6 2

3 4 5 7
Merge Sort
1 8 9 6 2

3 4 5 7
Merge Sort
1 8 9 6 2

4 5 7

3
Merge Sort
1 8 9 6 2

5 7

3 4
Merge Sort
1 8 9 6 2

3 4 5
Merge Sort
1 8 9 6 2

3 4 5 7
Merge Sort
3 4 5 7 1 8 9 6 2
Merge Sort
3 4 5 7

1 8 9 6 2
Merge Sort
3 4 5 7

1 8 9 6 2
Merge Sort
3 4 5 7

6 2
1 8 9
Merge Sort
3 4 5 7

6 2
1 8 9
Merge Sort
3 4 5 7

6 2
9
1 8
Merge Sort
3 4 5 7

6 2
9
1 8
Merge Sort
3 4 5 7

6 2
9
1 8
Merge Sort
3 4 5 7

6 2
9
8

1
Merge Sort
3 4 5 7

6 2
9

1 8
Merge Sort
3 4 5 7

6 2
1 8 9
Merge Sort
3 4 5 7

6 2
1 8 9
Merge Sort
3 4 5 7

6 2
8 9

1
Merge Sort
3 4 5 7

6 2
9

1 8
Merge Sort
3 4 5 7

6 2

1 8 9
Merge Sort
3 4 5 7

1 8 9 6 2
Merge Sort
3 4 5 7

1 8 9

6 2
Merge Sort
3 4 5 7

1 8 9

6 2
Merge Sort
3 4 5 7

1 8 9

6 2
Merge Sort
3 4 5 7

1 8 9

2
Merge Sort
3 4 5 7

1 8 9

2 6
Merge Sort
3 4 5 7

1 8 9 2 6
Merge Sort
3 4 5 7

1 8 9 2 6
Merge Sort
3 4 5 7

8 9 2 6

1
Merge Sort
3 4 5 7

8 9 6

1 2
Merge Sort
3 4 5 7

8 9

1 2 6
Merge Sort
3 4 5 7

1 2 6 8
Merge Sort
3 4 5 7

1 2 6 8 9
Merge Sort
3 4 5 7 1 2 6 8 9
Merge Sort
3 4 5 7 1 2 6 8 9
Merge Sort
3 4 5 7 2 6 8 9

1
Merge Sort
3 4 5 7 6 8 9

1 2
Merge Sort
4 5 7 6 8 9

1 2 3
Merge Sort
5 7 6 8 9

1 2 3 4
Merge Sort
7 6 8 9

1 2 3 4 5
Merge Sort
7 8 9

1 2 3 4 5 6
Merge Sort
8 9

1 2 3 4 5 6 7
Merge Sort
9

1 2 3 4 5 6 7 8
Merge Sort

1 2 3 4 5 6 7 8 9
Merge Sort

1 2 3 4 5 6 7 8 9
What is the complexity of mergesort?
Merge Sort
(The numbers are cleaner with an even n)

## How many layers - and how much work in each layer?

3 4 5 7 1 8 9 6
3 4 5 7 1 8 9 6
3 4 5 7 1 8 9 6
3 4 5 7 1 8 9 6
Merge Sort
(The numbers are cleaner with an even n)

## How many layers - and how much work in each layer?

O(n)
3 4 5 7 1 8 9 6
O(n/2) O(n/2) O(n)
O(log n)

3 4 5 7 1 8 9 6
O(n/4) O(n/4) O(n/4) O(n/4) O(n)
3 4 5 7 1 8 9 6
O(n/16) O(n/16) O(n/16) O(n/16) O(n/16) O(n/16)O(n/16) O(n/16) O(n)
3 4 5 7 1 8 9 6
Quick Sort

3 4 5 7 1 8 9 6 2
Quick Sort

Pivot

3 4 5 7 1 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

3 4 5 7 1 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

3 4 5 7 1 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

3 4 5 7 1 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

3 4 5 7 1 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

3 1 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

3 1 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 5 7 4 8 9 6 2
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 2 7 4 8 9 6 5
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 3 2 7 4 8 9 6 5
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 2 3 7 4 8 9 6 5
Less Greater
Unknown
Than Than
Quick Sort

Pivot

1 2 3 7 4 8 9 6 5
Less Greater
Unknown
Than Than
Quick Sort

Correct Location

1 2 3 7 4 8 9 6 5

## Can be sorted separately

Quick Sort

Pivot

1 2 3 7 4 8 9 6 5
Quick Sort

Correct Location

1 2 3 7 4 8 9 6 5
Quick Sort

Pivot

1 2 3 7 4 8 9 6 5
Quick Sort

Correct Location

1 2 3 7 4 8 9 6 5
Quick Sort

Pivot

1 2 3 7 4 8 9 6 5
Quick Sort

Pivot

1 2 3 4 7 8 9 6 5
Quick Sort

Pivot

1 2 3 4 7 8 9 6 5
Quick Sort

Pivot

1 2 3 4 7 8 9 6 5
Quick Sort

Pivot

1 2 3 4 6 7 9 8 5
Quick Sort

Pivot

1 2 3 4 6 5 7 8 9
Quick Sort

Correct Location

1 2 3 4 6 5 7 8 9
Quick Sort

Pivot

1 2 3 4 6 5 7 8 9
Quick Sort

Pivot

1 2 3 4 6 5 7 8 9
Quick Sort

Pivot

1 2 3 4 6 5 7 8 9
Quick Sort

Correct Location

1 2 3 4 6 5 7 8 9
Quick Sort

Pivot

1 2 3 4 6 5 7 8 9
Quick Sort

Pivot

1 2 3 4 5 6 7 8 9
Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9
Quick Sort

Pivot

1 2 3 4 5 6 7 8 9
Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9
Quick Sort

Pivot

1 2 3 4 5 6 7 8 9
Quick Sort

Pivot

1 2 3 4 5 6 7 8 9
Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9
Quick Sort

Pivot

1 2 3 4 5 6 7 8 9
Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9
Quick Sort

1 2 3 4 5 6 7 8 9
Quick Sort

1 2 3 4 5 6 7 8 9

## What is the complexity of quicksort?

Quick Sort

1 2 3 4 5 6 7 8 9

## What is the complexity of quicksort?

O(n2)?? Why did you teach me that?
Quick Sort

## Worst case: already sorted input

1 2 3 4 5 6 7 8 9
Quick Sort

## Expected on random input: O(n log n)

8 4 9 7 5 1 3 6 2
If our pivot is roughly in the middle
Quick Sort

## Expected on random input: O(n log n)

8 4 9 7 5 1 3 6 2
If our pivot is roughly in the middle

Quick Sort

## Expected on random input: O(n log n)

8 4 9 7 5 1 3 6 2
If our pivot is roughly in the middle

## We can shuﬄe an array in O(n) time

Comparison-based sorting

## For comparison-based sorting O(n log n)

is the best possible time

## If we know more about our data, we can

sometimes do better
Counting Sort
If we know that our collection only contains integers in a given
(small) range, counting sort can be applied.

## In this case, we have numbers between 1 and 10.

3 4 5 2 2 5 9 6 2
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 0
5: 0 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 0 7: 0
3: 1 8: 0
4: 0 9: 0
5: 0 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 0 7: 0
3: 1 8: 0
4: 1 9: 0
5: 0 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 0 7: 0
3: 1 8: 0
4: 1 9: 0
5: 1 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 1 7: 0
3: 1 8: 0
4: 1 9: 0
5: 1 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 2 7: 0
3: 1 8: 0
4: 1 9: 0
5: 1 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 2 7: 0
3: 1 8: 0
4: 1 9: 0
5: 2 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0
2: 2 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 1
2: 2 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 1
2: 3 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

1: 0 6: 1
2: 3 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

1: 0 6: 1
2: 3 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

1: 0 6: 1
2: 3 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2
1: 0 6: 1
2: 2 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2
1: 0 6: 1
2: 1 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2
1: 0 6: 1
2: 0 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2
1: 0 6: 1
2: 0 7: 0
3: 1 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3
1: 0 6: 1
2: 0 7: 0
3: 0 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3
1: 0 6: 1
2: 0 7: 0
3: 0 8: 0
4: 1 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4
1: 0 6: 1
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4
1: 0 6: 1
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 2 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5
1: 0 6: 1
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 1 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5
1: 0 6: 1
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5
1: 0 6: 1
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 1
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 0
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 0
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 0
5: 0 10: 0
Counting Sort
3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9
1: 0 6: 0
2: 0 7: 0
3: 0 8: 0
4: 0 9: 0
5: 0 10: 0

## What is the complexity of counting sort?

Agenda
1. Calculating Runtimes (Big O)
2. Algorithms
A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures
C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

## I. Binary Search Tree

J. Hash Table

K. Set

L. Graph

M. Heap
Array
‘h' ‘e’ ‘l’ ‘l’ ‘o’

## • Size must be specified ahead-of-time

Array
‘h' ‘e’ ‘l’ ‘l’ ‘o’

## • O(1) writing to a given index

Array
‘h' ‘e’ ‘l’ ‘l’ ‘o’

## • O(n) “inserting” an element

‘h' ‘e’ ‘l’ ‘l’ ‘o’

## • Grows and shrinks as needed

‘h' ‘e’ ‘l’ ‘l’ ‘o’

## • O(1) to reach the next index given an entry

‘h' ‘e’ ‘l’ ‘l’ ‘o’

## ‘m’ ‘e’ ‘l’ ‘l’ ‘o’

Insert to tail O(1)

## ‘m’ ‘e’ ‘l’ ‘l’ ‘o’

Insert to tail O(1)

## ‘m’ ‘e’ ‘l’ ‘l’ ‘o’ ‘w’

Insert to tail O(1)

## ‘m’ ‘e’ ‘l’ ‘l’ ‘o’ ‘w’

‘h' ‘e’ ‘l’ ‘l’ ‘o’

• Each entry holds a value and a pointer to the next entry and the previous entry

## • O(1) to reach the previous index given an entry

Delete from tail O(1)

## ‘h' ‘e’ ‘l’ ‘l’ ‘o’

Delete from tail O(1)

## ‘h' ‘e’ ‘l’ ‘l’ ‘o’

Delete from tail O(1)

## ‘h' ‘e’ ‘l’ ‘l’

ArrayList
AKA Dynamic Array

## • Contiguous space in memory

• Automatically resize

## • O(n) removing an element

Manipulating Arraylists
Appending to an array list

## ‘h' ‘e’ ‘l’ ‘l’ ‘o’

Manipulating Arraylists
Appending to an array list

## Create a new array, double the size of the previous one

This takes O(n) time

## ‘h' ‘e’ ‘l’ ‘l’ ‘o’

Manipulating Arraylists
Appending to an array list

## ‘h' ‘e’ ‘l’ ‘l’ ‘o’ ‘‘

Manipulating Arraylists
Appending to an array list

## What is the complexity of appending to an

arraylist?
Manipulating Arraylists
Appending to an array list

## If you start with an array size of 1, and you append

n times. How many times have you resized?
Manipulating Arraylists
If you start with an array size of 1, and you append
n times. How many times have you resized?

## Re-ordering this addition can give us n + n/2 + n/4 + … + 8 + 4 + 2+ 1

n/2
n n/8
= n n
n/4 n/16

n + n/2 + n/4 + … + 8 + 4 + 2+ 1 2n

## The cost of appending n times is O(n). So the cost of appending once

is amortised as O(1)
Stack

## • A collection of elements which works in Last In First

Top 6 Out (LIFO) order

## • O(1) “pop” which removes an element from the top and

returns it

4
• O(1) “peek” which returns the element at the top
without removing

5
Stack Manipulation
Push

Top 6

5
Stack Manipulation
Push

Top 7

5
Stack Manipulation
Pop

Top 7

5
Stack Manipulation
Pop

Top 6

5
Stack Manipulation
Pop

Top 6

5
Stack

## • A collection of elements which works in Last In First

Top 6 Out (LIFO) order

## • O(1) “pop” which removes an element from the top and

returns it

4
• O(1) “peek” which returns the element at the top
without removing

## Of the data structures you know so far, which

would be most appropriate to implement a stack?
Queue

## • A collection of elements which works in

First In First Out (FIFO) order

5 4 6 the tail

## • O(1) “dequeue” which removes an

element from the head and returns it

• O(1) “peek” which returns the element at
Queue Manipulation
Enqueue

5 4 6

Queue Manipulation
Enqueue

7 5 4 6

Queue Manipulation
Dequeue

7 5 4 6

Queue Manipulation
Dequeue

7 5 4 6

Queue Manipulation
Dequeue

7 5 4 6

Queue Manipulation
Dequeue

7 5 4

Queue

## • A collection of elements which works in

First In First Out (FIFO) order

5 4 6 the tail

## • O(1) “dequeue” which removes an

element from the head and returns it

• O(1) “peek” which returns the element at

## Of the data structures you know so far, which

would be most appropriate to implement a queue?
Tree
Root 5

## 1 2 3 • A node has 0 or more “children”, each of

which is itself a tree

Binary Tree
Root 5

## 1 2 • A tree where each node can only have 0,

1, or 2 children.
Binary Tree Traversal
Root 1

2 3 4,6,2,5,1,3

1,2,4,6,5,3

## Post Order (Left,Right,Root)

6,4,5,2,3,1

4 5

6
Binary Tree Traversal
Depth-first
Root 1

## Explore the tree, going as deep as

2 3
possible down one branch before trying
other branches.

## In this case, a depth first search would

visit:

1, 2, 4, 6, 5, 3
4 5

6
Binary Tree Traversal
Depth-first
Root 1

## Explore the tree, going as deep as

2 3
possible down one branch before trying
other branches.

visit:

1, 2, 4, 6, 5, 3
4 5

## Of the data structures you know so

far, which would be most appropriate
6
to use during a depth-first search?
Binary Tree Traversal
Root 1

## Explore the tree, exploring all children of

2 3
the current node before moving to the
next level

## In this case, a breadth first search would

visit:

1, 2, 3, 4, 5, 6
4 5

6
Binary Tree Traversal
Root 1

## Explore the tree, exploring all children of

2 3
the current node before moving to the
next level

visit:

1, 2, 3, 4, 5, 6
4 5

## Of the data structures you know so

far, which would be most appropriate
6
to use during a breadth-first search?
Binary Search Tree
Root 5

## • A binary tree, where every left descendent of

a node must have a value less than the
node, and every right descendent of a node
3 9 must have a value greater than the node

## • Allows for O(log n) lookup for a value

• O(log n) insertion

## 1 6 22 • Allows for eﬃcient searching of ranges (e.g.

give me all nodes with values between 5 and
9)

15
BST Traversal
Searching for “6”

Pointer 5

3 9

1 6 22

15
BST Traversal
Searching for “6”

Pointer 5

6 is greater than 5, so
3 9
we move right

1 6 22

15
BST Traversal
Searching for “6”

3 9 Pointer

6 is less than 9, so we
1 6 22
move left

15
BST Traversal
Searching for “6”

3 9

6 is equal to 6. Found it

1 6 22

Pointer
15
BST Balancing
1

## This is a valid BST, what is the 4

complexity of searching in this
tree?
BST as a Map
If we associate both a key and a value with
“jon”:
29 each node, we can use a BST as a map.

## If we want to see how old a given person is,

we can search the BST in O(log n) time
“liam”: (using alphabetical order).
“cat”:
20 30
Other features of the BST still apply. E.g. we
can query for everyone who’s name begins
with “j”
“amy”: “kate”: “pete”:
19 30 21

“mary”
: 18
Hashing
“Hello there” 4357752745

1886414098

## Identical objects should always result in the same hash value

Diﬀerent objects would ideally result in a diﬀerent hash values (but can’t always)
Hash Table
Let’s say we can hash a string into a number between 0 and 7

## (we could do this by using any hash function

and then using modulus to restrict it to 0-7)

## And we create an 8 element array

Hash Table

Using our hash function and this array, we can create a map

## We hash “jon” and it gives us 882751264

882751264 % 8 = 0
Hash Table

## {key: “jon”, value: 29}

Using our hash function and this array, we can create a map

## We hash “jon” and it gives us 882751264

882751264 % 8 = 0
Hash Table

Hash Table

## {key: “pete”, value: 21}

{key: “cat”, value: 20}
{key: “jon”, value: 29}

Hash Table

## {key: “pete”, value: 21}

{key: “cat”, value: 20}
{key: “jon”, value: 29}

## We can insert a few more values

And because the same key will always hash to the same
index, we can retrieve our values in O(1) time
Hash Collisions

## {key: “pete”, value: 21}

{key: “cat”, value: 20}
{key: “jon”, value: 29}

Hash Collisions

## {key: “pete”, value: 21}

{key: “cat”, value: 20}
{key: “jon”, value: 29}

## There are two common ways of dealing with

this. Linear probing and Separate chaining.
Linear Probing

## {key: “liam”, value: {key: “pete”, value: 21}

30}
{key: “cat”, value: 20}
{key: “jon”, value: 29}

With linear probing, we’d just insert our new data in the next
empty index after the one we hashed to.

When fetching data, if we don’t find the key we want in the first
index, we’d have to search through the rest of the array until we find
an empty slot.

This means, if your array is quite full. Inserting and fetching could be
O(n). You must ensure your array doesn’t get too full.
Linear Probing

## {key: “liam”, value: {key: “pete”, value: 21}

30}
{key: “cat”, value: 20}
{key: “jon”, value: 29}

## We can resize the array the same as with a dynamic array/arraylist

However we will have to re-hash every element for the new array
size. This means that some inserts will be O(n). It is amortised
O(1).

## Deletion needs to be handled carefully, as if you remove an

element in the middle of a sequence of items which all hash to the
same index, those which follow the deleted element may become
inaccessible
Separate Chaining

{key: “cat”,
{key: “jon”, {key: “pete”,
value: 20}
value: 29} value: 21}

## In separate chaining, each index actually contains a linked list of

elements which hash to that index
Separate Chaining

{key: “cat”,
{key: “jon”, {key: “pete”,
value: 20}
value: 29} value: 21}

{key:
“liam”,
value: 30}

## In case of a hash collision, we just append to the list

Separate Chaining

{key: “cat”,
{key: “jon”, {key: “pete”,
value: 20}
value: 29} value: 21}

{key:
“liam”,
value: 30}

Worst case is O(n) lookup and set (if we have a bad hash function
and everything is in a single bucket).

## With a reasonable implementation, and a good hash function - this

gives O(1) set and lookup
HashMap vs TreeMap

with a BST

## With a BST we get additional functionality, including max and

min, range queries, and previous/next
Set
1 400
9
22
7

O(1) time
Set
1 400
9
22
7

O(1) time

Set
1 400
9
22
7

## We could also create a TreeSet using a BST. The would

have O(log n) insert and check. It would also allow for
range queries, min/max, etc.
Graph
jon

pete
cat

liam amy

## Used to represents geographical maps, social

networks, circuits, etc.
Graph Usage
jon

pete
cat

liam amy

## How would you calculate the minimum number of links

between cat and amy? What is the complexity of this?
Graph
London 15

Belfast
55
Hangzhou 392

134
Beĳing Paris
315

The edges may have weights, for example to indicate cost or distance

## Common algorithms exist for search for the min/

max edge weights between two nodes (e.g.
Djikstra’s Algorithm, A*)
Graph Representation

## cat jon liam amy pete

jon
cat 0 0 1 0 0
jon 0 0 1 1 1 pete
cat
liam 1 1 0 1 0
amy 0 1 1 0 0
liam amy
pete 0 1 0 0 0

## • Removing an edge takes O(1) time

• Checking for an edge takes O(1) time
• Sparse graphs consume a lot of space
• Adding a vertex is O(V2) time
Graph Representation

jon

pete
cat

liam amy

## amy jon liam

node.
• Saves space in a sparse graph
• Queries of edge existing between two nodes
Pete amy is O(V)
• Adding an edge is O(1)
• Adding a vertex is O(1)
Heap
100

19 36

17 3 1

2 7

## A heap is a tree in which the value of each node is

greater than or equal to its descendants

## • Find the maximum value in O(1) time

• Remove maximum value in O(log n)
• Insert in O(log n)

## Can also be a min-heap in which the rule is reversed and

the minimum is at the root of the heap.
Heap Manipulation
Remove max
100

19 36

17 3 1

2 7
Heap Manipulation
Remove max
100

19 36

17 3 1

2 7

## Copy a leaf to the root

Heap Manipulation
Remove max
7

19 36

17 3 1

2 7

## Copy a leaf to the root

Heap Manipulation
Remove max
7

19 36

17 3 1

2 7

## Remove the leaf

Heap Manipulation
Remove max
7

19 36

17 3 1

## Remove the leaf

Heap Manipulation
Remove max
7

19 36

17 3 1

## Compare to largest child. Is it larger than our value?

Heap Manipulation
Remove max
7

19 36

17 3 1

Swap them
Heap Manipulation
Remove max
36

19 7

17 3 1

Swap them
Heap Manipulation
Remove max
36

19 7

17 3 1

## Compare to largest child. Is it larger than our value?

Heap Manipulation
Remove max
36

19 7

17 3 1

No? Finished
Heap Manipulation
Remove max
36

19 7

17 3 1

## What is the complexity of remove max?

Heap Manipulation
Inserting
36

19 7

17 3 1

2
Heap Manipulation
Inserting
36

19 7

17 3 1

2 22

## Add new element as leaf

Heap Manipulation
Inserting
36

19 7

17 3 1

2 22

## Compare with parent, is parent smaller?

Heap Manipulation
Inserting
36

19 7

17 3 1

2 22

## Yes? Swap with parent

Heap Manipulation
Inserting
36

19 7

22 3 1

2 17

## Yes? Swap with parent

Heap Manipulation
Inserting
36

19 7

22 3 1

2 17

## Compare with parent, is parent smaller?

Heap Manipulation
Inserting
36

19 7

22 3 1

2 17

## Yes? Swap with parent

Heap Manipulation
Inserting
36

22 7

19 3 1

2 17

## Yes? Swap with parent

Heap Manipulation
Inserting
36

22 7

19 3 1

2 17

## Compare with parent, is parent smaller?

Heap Manipulation
Inserting
36

22 7

19 3 1

2 17

No? Finished
Heap Manipulation
Inserting
36

22 7

19 3 1

2 17

## What is the complexity of insert?

Heap Implementation
36

22 7

19 3

36 22 7 19 3

A heap can be implemented using an array, with the top of the heap at the 0th index

Priority Queue

## A heap can be used to implement a “priority queue”

This pairs each element with a “priority” and elements are returned in priority order
Agenda
1. Calculating Runtimes (Big O)
2. Algorithms
A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures
C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

## I. Binary Search Tree

J. Hash Table

K. Set

L. Graph

M. Heap
Books
The Algorithm Design Manual - Steven Skiena
Introduction to Algorithms - CSRL

Courses
MIT Introduction to Computer Science and Programming
Harvard CS50
Coursera Data Structures and Algorithms Specialization

Videos

## Quicksort Folk Dance

Steven Skiena Algorithm Lectures