0 ratings0% found this document useful (0 votes)

21 views424 pagesAlgorithms study aid

Sep 02, 2018

© © All Rights Reserved

PDF, TXT or read online from Scribd

Algorithms study aid

© All Rights Reserved

0 ratings0% found this document useful (0 votes)

21 views424 pagesAlgorithms study aid

© All Rights Reserved

You are on page 1of 424

structures to be a software engineer

understanding of what is happening when you write

some code.

What is an algorithm?

What is an algorithm?

goal

Agenda

1. Calculating Runtimes (Big O)

2. Algorithms

A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures

C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

J. Hash Table

K. Set

L. Graph

M. Heap

Runtimes

Collection<String> animals = new ArrayList<>();

animals.add("cat");

animals.add("fish");

// ….

animals.contains("dog");

Vs

animals.add("cat");

animals.add("fish");

// ….

animals.contains("dog");

Runtimes

You’ll need some Algebra to fully understand Big

O and asymptotic analysis.

resources to review

Fibonacci Sequence

F20 = 6765

F50 = 12586269025

F100 = 354224848179261915075

F500 =

139423222456169788013972438287040728

395007025658769730726410896294832557

1622863290691557658876222521294125

Fibonacci Sequence

def FibRecurse(n):

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Vs

def FibList(n):

create an array F[0…n]

F[0] = 0

F[1] = 1

for i from 2 to n:

F[i] = F[i - 1] + F[i - 2]

return F[n]

How fast is this?

def FibList(n):

create an array F[0…n]

F[0] = 0

F[1] = 1

for i from 2 to n:

F[i] = F[i - 1] + F[i - 2]

return F[n]

T(n)

def FibList(n):

create an array F[0…n] 1

F[0] = 0 1

F[1] = 1 1

for i from 2 to n: n *

F[i] = F[i - 1] + F[i - 2] 1

return F[n] 1

Common runtimes

O(1) Constant

O(log n) Logarithmic

O(n) Linear

(n2) Quadratic

O(n3) Cubic

O(2n) Exponential

https://www.desmos.com/calculator/ty01lmuivl

O(n)

def FibList(n):

create an array F[0…n] O(n)

F[0] = 0 O(1)

F[1] = 1 O(1)

for i from 2 to n: O(n) *

F[i] = F[i - 1] + F[i - 2] O(1)

return F[n] O(1)

Recursive

def FibRecurse(n):

if n <= 1: O(1)

return n O(1)

else: O(1)

return FibRecurse(n - 1) + FibRecurse(n - 2) O(???)

Recursive

4

def FibRecurse(n):

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Recursive

4

3 2

def FibRecurse(n):

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Recursive

4

3 2

2 1

def FibRecurse(n):

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Recursive

4

3 2

2 1

1 0

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Recursive

4

3 2

2 1 1 0

1 0

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Recursive

4

3 2

2 1 1 0

1 0

def FibRecurse(n):

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Recursive

5

4 3

3 2 2 1

2 1 1 0 1 0

1 0

def FibRecurse(n):

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Recursive

5

4 3

3 2 2 1

2 1 1 0 1 0

1 0

O(2n)

def FibRecurse(n):

if n <= 1:

return n

else:

return FibRecurse(n - 1) + FibRecurse(n - 2)

Common runtimes

Common runtimes

Agenda

1. Calculating Runtimes (Big O)

2. Algorithms

A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures

C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

J. Hash Table

K. Set

L. Graph

M. Heap

Searching

3 4 5 7 1 8 9 6 2

Searching

3 4 5 7 1 8 9 6 2

number 6 is in this collection?

Linear Search

6?

3 4 5 7 1 8 9 6 2

Linear Search

6?

3 4 5 7 1 8 9 6 2

Linear Search

6?

3 4 5 7 1 8 9 6 2

Linear Search

6?

3 4 5 7 1 8 9 6 2

Linear Search

6?

3 4 5 7 1 8 9 6 2

Linear Search

6?

3 4 5 7 1 8 9 6 2

Linear Search

6?

3 4 5 7 1 8 9 6 2

Linear Search

6?

3 4 5 7 1 8 9 6 2

Searching

1 2 3 4 5 6 7 8 9

How many steps to find if number 6 is in this

collection?

Binary Search

6?

1 2 3 4 5 6 7 8 9

Binary Search

6?

1 2 3 4 5 6 7 8 9

5 is less than 6, so everything to the left of 5 can be ignored

Binary Search

6?

1 2 3 4 5 6 7 8 9

Binary Search

6?

1 2 3 4 5 6 7 8 9

7 is greater than 6, so everything to the right of 7 can be ignored

Binary Search

6?

1 2 3 4 5 6 7 8 9

Binary Search

1 2 3 4 5 6 7 8 9

In this case, the search took 3 steps, rather than the 9 steps using linear

search.

proportional to 1 million, binary search will take time proportional to 20

Sorting

3 4 5 7 1 8 9 6 2

Selection Sort

3 4 5 7 1 8 9 6 2

Selection Sort

3 4 5 7 1 8 9 6 2

Selection Sort

1 4 5 7 3 8 9 6 2

Selection Sort

1 4 5 7 3 8 9 6 2

Selection Sort

1 4 5 7 3 8 9 6 2

Selection Sort

1 2 5 7 3 8 9 6 4

Selection Sort

1 2 5 7 3 8 9 6 4

Selection Sort

1 2 3 7 5 8 9 6 4

Selection Sort

1 2 3 7 5 8 9 6 4

Selection Sort

1 2 3 4 5 8 9 6 7

Selection Sort

1 2 3 4 5 8 9 6 7

Selection Sort

1 2 3 4 5 8 9 6 7

Selection Sort

1 2 3 4 5 8 9 6 7

Selection Sort

1 2 3 4 5 6 9 8 7

Selection Sort

1 2 3 4 5 6 9 8 7

Selection Sort

1 2 3 4 5 6 7 8 9

Selection Sort

1 2 3 4 5 6 7 8 9

Selection Sort

1 2 3 4 5 6 7 8 9

Selection Sort

1 2 3 4 5 6 7 8 9

Selection Sort

1 2 3 4 5 6 7 8 9

Insertion Sort

3 4 5 7 1 8 9 6 2

Insertion Sort

3 4 5 7 1 8 9 6 2

Insertion Sort

3 4 5 7 1 8 9 6 2

Insertion Sort

3 4 5 7 1 8 9 6 2

Insertion Sort

1

3 4 5 7 8 9 6 2

Insertion Sort

1

3 4 5 7 8 9 6 2

Insertion Sort

1 3 4 5 7 8 9 6 2

Insertion Sort

1 3 4 5 7 8 9 6 2

Insertion Sort

1 3 4 5 7 8 9 6 2

Insertion Sort

6

1 3 4 5 7 8 9 2

Insertion Sort

6

1 3 4 5 7 8 9 2

Insertion Sort

1 3 4 5 6 7 8 9 2

Insertion Sort

2

1 3 4 5 6 7 8 9

Insertion Sort

2

1 3 4 5 6 7 8 9

Insertion Sort

1 2 3 4 5 6 7 8 9

Insertion Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

3 4 5 7 1 8 9 6 2

Bubble Sort

3 4 5 7 1 8 9 6 2

Bubble Sort

3 4 5 7 1 8 9 6 2

Bubble Sort

3 4 5 7 1 8 9 6 2

Bubble Sort

3 4 5 1 7 8 9 6 2

Bubble Sort

3 4 5 1 7 8 9 6 2

Bubble Sort

3 4 5 1 7 8 9 6 2

Bubble Sort

3 4 5 1 7 8 9 6 2

Bubble Sort

3 4 5 1 7 8 6 9 2

Bubble Sort

3 4 5 1 7 8 6 9 2

Bubble Sort

3 4 5 1 7 8 6 2 9

Bubble Sort

3 4 5 1 7 8 6 2 9

Bubble Sort

3 4 5 1 7 8 6 2 9

Bubble Sort

3 4 5 1 7 8 6 2 9

Bubble Sort

3 4 1 5 7 8 6 2 9

Bubble Sort

3 4 1 5 7 8 6 2 9

Bubble Sort

3 4 1 5 7 8 6 2 9

Bubble Sort

3 4 1 5 7 8 6 2 9

Bubble Sort

3 4 1 5 7 6 8 2 9

Bubble Sort

3 4 1 5 7 6 8 2 9

Bubble Sort

3 4 1 5 7 6 2 8 9

Bubble Sort

3 4 1 5 7 6 2 8 9

Bubble Sort

3 4 1 5 7 6 2 8 9

Bubble Sort

3 4 1 5 7 6 2 8 9

Bubble Sort

3 1 4 5 7 6 2 8 9

Bubble Sort

3 1 4 5 7 6 2 8 9

Bubble Sort

3 1 4 5 7 6 2 8 9

Bubble Sort

3 1 4 5 7 6 2 8 9

Bubble Sort

3 1 4 5 6 7 2 8 9

Bubble Sort

3 1 4 5 6 7 2 8 9

Bubble Sort

3 1 4 5 6 2 7 8 9

Bubble Sort

3 1 4 5 6 2 7 8 9

Bubble Sort

3 1 4 5 6 2 7 8 9

Bubble Sort

3 1 4 5 6 2 7 8 9

Bubble Sort

1 3 4 5 6 2 7 8 9

Bubble Sort

1 3 4 5 6 2 7 8 9

Bubble Sort

1 3 4 5 6 2 7 8 9

Bubble Sort

1 3 4 5 6 2 7 8 9

Bubble Sort

1 3 4 5 6 2 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 5 2 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 4 2 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 3 2 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Bubble Sort

1 2 3 4 5 6 7 8 9

Merge Sort

3 4 5 7 1 8 9 6 2

Merge Sort

3 4 5 7 1 8 9 6 2

Merge Sort

3 4 5 7 (magic)

1 2 6 8 9

Merge Sort

3 4 5 7 1 2 6 8 9

Merge Sort

3 4 5 7 2 6 8 9

1

Merge Sort

3 4 5 7 6 8 9

1 2

Merge Sort

4 5 7 6 8 9

1 2 3

Merge Sort

5 7 6 8 9

1 2 3 4

Merge Sort

7 6 8 9

1 2 3 4 5

Merge Sort

7 8 9

1 2 3 4 5 6

Merge Sort

8 9

1 2 3 4 5 6 7

Merge Sort

9

1 2 3 4 5 6 7 8

Merge Sort

1 2 3 4 5 6 7 8 9

What is the complexity of the merge?

Merge Sort

Merge Sort

Merge Sort!

Merge Sort

1 8 9 6 2

3 4 5 7

Merge Sort

1 8 9 6 2

3 4 5 7

Merge Sort

1 8 9 6 2

5 7

3 4

Merge Sort

1 8 9 6 2

5 7

3 4

Merge Sort

1 8 9 6 2

5 7

3 4

Merge Sort

1 8 9 6 2

5 7

3

Merge Sort

1 8 9 6 2

5 7

3 4

Merge Sort

1 8 9 6 2

3 4 5 7

Merge Sort

1 8 9 6 2

3 4

5 7

Merge Sort

1 8 9 6 2

3 4

5 7

Merge Sort

1 8 9 6 2

3 4

5 7

Merge Sort

1 8 9 6 2

3 4

5

Merge Sort

1 8 9 6 2

3 4

5 7

Merge Sort

1 8 9 6 2

3 4 5 7

Merge Sort

1 8 9 6 2

3 4 5 7

Merge Sort

1 8 9 6 2

4 5 7

3

Merge Sort

1 8 9 6 2

5 7

3 4

Merge Sort

1 8 9 6 2

3 4 5

Merge Sort

1 8 9 6 2

3 4 5 7

Merge Sort

3 4 5 7 1 8 9 6 2

Merge Sort

3 4 5 7

1 8 9 6 2

Merge Sort

3 4 5 7

1 8 9 6 2

Merge Sort

3 4 5 7

6 2

1 8 9

Merge Sort

3 4 5 7

6 2

1 8 9

Merge Sort

3 4 5 7

6 2

9

1 8

Merge Sort

3 4 5 7

6 2

9

1 8

Merge Sort

3 4 5 7

6 2

9

1 8

Merge Sort

3 4 5 7

6 2

9

8

1

Merge Sort

3 4 5 7

6 2

9

1 8

Merge Sort

3 4 5 7

6 2

1 8 9

Merge Sort

3 4 5 7

6 2

1 8 9

Merge Sort

3 4 5 7

6 2

8 9

1

Merge Sort

3 4 5 7

6 2

9

1 8

Merge Sort

3 4 5 7

6 2

1 8 9

Merge Sort

3 4 5 7

1 8 9 6 2

Merge Sort

3 4 5 7

1 8 9

6 2

Merge Sort

3 4 5 7

1 8 9

6 2

Merge Sort

3 4 5 7

1 8 9

6 2

Merge Sort

3 4 5 7

1 8 9

2

Merge Sort

3 4 5 7

1 8 9

2 6

Merge Sort

3 4 5 7

1 8 9 2 6

Merge Sort

3 4 5 7

1 8 9 2 6

Merge Sort

3 4 5 7

8 9 2 6

1

Merge Sort

3 4 5 7

8 9 6

1 2

Merge Sort

3 4 5 7

8 9

1 2 6

Merge Sort

3 4 5 7

1 2 6 8

Merge Sort

3 4 5 7

1 2 6 8 9

Merge Sort

3 4 5 7 1 2 6 8 9

Merge Sort

3 4 5 7 1 2 6 8 9

Merge Sort

3 4 5 7 2 6 8 9

1

Merge Sort

3 4 5 7 6 8 9

1 2

Merge Sort

4 5 7 6 8 9

1 2 3

Merge Sort

5 7 6 8 9

1 2 3 4

Merge Sort

7 6 8 9

1 2 3 4 5

Merge Sort

7 8 9

1 2 3 4 5 6

Merge Sort

8 9

1 2 3 4 5 6 7

Merge Sort

9

1 2 3 4 5 6 7 8

Merge Sort

1 2 3 4 5 6 7 8 9

Merge Sort

1 2 3 4 5 6 7 8 9

What is the complexity of mergesort?

Merge Sort

(The numbers are cleaner with an even n)

3 4 5 7 1 8 9 6

3 4 5 7 1 8 9 6

3 4 5 7 1 8 9 6

3 4 5 7 1 8 9 6

Merge Sort

(The numbers are cleaner with an even n)

O(n)

3 4 5 7 1 8 9 6

O(n/2) O(n/2) O(n)

O(log n)

3 4 5 7 1 8 9 6

O(n/4) O(n/4) O(n/4) O(n/4) O(n)

3 4 5 7 1 8 9 6

O(n/16) O(n/16) O(n/16) O(n/16) O(n/16) O(n/16)O(n/16) O(n/16) O(n)

3 4 5 7 1 8 9 6

Quick Sort

3 4 5 7 1 8 9 6 2

Quick Sort

Pivot

3 4 5 7 1 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

3 4 5 7 1 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

3 4 5 7 1 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

3 4 5 7 1 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

3 4 5 7 1 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

3 1 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

3 1 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 5 7 4 8 9 6 2

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 2 7 4 8 9 6 5

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 3 2 7 4 8 9 6 5

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 2 3 7 4 8 9 6 5

Less Greater

Unknown

Than Than

Quick Sort

Pivot

1 2 3 7 4 8 9 6 5

Less Greater

Unknown

Than Than

Quick Sort

Correct Location

1 2 3 7 4 8 9 6 5

Quick Sort

Pivot

1 2 3 7 4 8 9 6 5

Quick Sort

Correct Location

1 2 3 7 4 8 9 6 5

Quick Sort

Pivot

1 2 3 7 4 8 9 6 5

Quick Sort

Correct Location

1 2 3 7 4 8 9 6 5

Quick Sort

Pivot

1 2 3 7 4 8 9 6 5

Quick Sort

Pivot

1 2 3 4 7 8 9 6 5

Quick Sort

Pivot

1 2 3 4 7 8 9 6 5

Quick Sort

Pivot

1 2 3 4 7 8 9 6 5

Quick Sort

Pivot

1 2 3 4 6 7 9 8 5

Quick Sort

Pivot

1 2 3 4 6 5 7 8 9

Quick Sort

Correct Location

1 2 3 4 6 5 7 8 9

Quick Sort

Pivot

1 2 3 4 6 5 7 8 9

Quick Sort

Pivot

1 2 3 4 6 5 7 8 9

Quick Sort

Pivot

1 2 3 4 6 5 7 8 9

Quick Sort

Correct Location

1 2 3 4 6 5 7 8 9

Quick Sort

Pivot

1 2 3 4 6 5 7 8 9

Quick Sort

Pivot

1 2 3 4 5 6 7 8 9

Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9

Quick Sort

Pivot

1 2 3 4 5 6 7 8 9

Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9

Quick Sort

Pivot

1 2 3 4 5 6 7 8 9

Quick Sort

Pivot

1 2 3 4 5 6 7 8 9

Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9

Quick Sort

Pivot

1 2 3 4 5 6 7 8 9

Quick Sort

Correct Location

1 2 3 4 5 6 7 8 9

Quick Sort

1 2 3 4 5 6 7 8 9

Quick Sort

1 2 3 4 5 6 7 8 9

Quick Sort

1 2 3 4 5 6 7 8 9

O(n2)?? Why did you teach me that?

Quick Sort

1 2 3 4 5 6 7 8 9

Quick Sort

8 4 9 7 5 1 3 6 2

If our pivot is roughly in the middle

Quick Sort

8 4 9 7 5 1 3 6 2

If our pivot is roughly in the middle

Quick Sort

8 4 9 7 5 1 3 6 2

If our pivot is roughly in the middle

…

Comparison-based sorting

is the best possible time

sometimes do better

Counting Sort

If we know that our collection only contains integers in a given

(small) range, counting sort can be applied.

3 4 5 2 2 5 9 6 2

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 0

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 0 7: 0

3: 1 8: 0

4: 0 9: 0

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 0 7: 0

3: 1 8: 0

4: 1 9: 0

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 0 7: 0

3: 1 8: 0

4: 1 9: 0

5: 1 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 1 7: 0

3: 1 8: 0

4: 1 9: 0

5: 1 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 2 7: 0

3: 1 8: 0

4: 1 9: 0

5: 1 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 2 7: 0

3: 1 8: 0

4: 1 9: 0

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 0

2: 2 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 1

2: 2 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 1

2: 3 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 1

2: 3 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 1

2: 3 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

1: 0 6: 1

2: 3 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2

1: 0 6: 1

2: 2 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2

1: 0 6: 1

2: 1 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2

1: 0 6: 1

2: 0 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2

1: 0 6: 1

2: 0 7: 0

3: 1 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3

1: 0 6: 1

2: 0 7: 0

3: 0 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3

1: 0 6: 1

2: 0 7: 0

3: 0 8: 0

4: 1 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4

1: 0 6: 1

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4

1: 0 6: 1

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 2 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5

1: 0 6: 1

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 1 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5

1: 0 6: 1

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5

1: 0 6: 1

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 1

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 0

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 0

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 0

5: 0 10: 0

Counting Sort

3 4 5 2 2 5 9 6 2

2 2 2 3 4 5 5 6 9

1: 0 6: 0

2: 0 7: 0

3: 0 8: 0

4: 0 9: 0

5: 0 10: 0

Agenda

1. Calculating Runtimes (Big O)

2. Algorithms

A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures

C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

J. Hash Table

K. Set

L. Graph

M. Heap

Array

‘h' ‘e’ ‘l’ ‘l’ ‘o’

Array

‘h' ‘e’ ‘l’ ‘l’ ‘o’

Array

‘h' ‘e’ ‘l’ ‘l’ ‘o’

Single Linked List

‘h' ‘e’ ‘l’ ‘l’ ‘o’

Single Linked List

‘h' ‘e’ ‘l’ ‘l’ ‘o’

Single Linked List

‘h' ‘e’ ‘l’ ‘l’ ‘o’

Single Linked List Manipulation

Head Tail

Single Linked List Manipulation

Deleting from head O(1)

Head Tail

Single Linked List Manipulation

Deleting from head O(1)

Head Tail

Single Linked List Manipulation

Deleting from head O(1)

Head Tail

Single Linked List Manipulation

Insert to head O(1)

Head Tail

Single Linked List Manipulation

Insert to head O(1)

Head Tail

Single Linked List Manipulation

Insert to tail O(1)

Head Tail

Single Linked List Manipulation

Insert to tail O(1)

Head Tail

Single Linked List Manipulation

Insert to tail O(1)

Head Tail

Double Linked List

‘h' ‘e’ ‘l’ ‘l’ ‘o’

• Each entry holds a value and a pointer to the next entry and the previous entry

Double Linked List Manipulation

Delete from tail O(1)

Head Tail

Double Linked List Manipulation

Delete from tail O(1)

Head Tail

Double Linked List Manipulation

Delete from tail O(1)

Head Tail

ArrayList

AKA Dynamic Array

• Automatically resize

Manipulating Arraylists

Appending to an array list

Manipulating Arraylists

Appending to an array list

This takes O(n) time

Manipulating Arraylists

Appending to an array list

Manipulating Arraylists

Appending to an array list

arraylist?

Manipulating Arraylists

Appending to an array list

n times. How many times have you resized?

Manipulating Arraylists

If you start with an array size of 1, and you append

n times. How many times have you resized?

n/2

n n/8

= n n

n/4 n/16

n + n/2 + n/4 + … + 8 + 4 + 2+ 1 2n

is amortised as O(1)

Stack

Top 6 Out (LIFO) order

returns it

4

• O(1) “peek” which returns the element at the top

without removing

5

Stack Manipulation

Push

Top 6

5

Stack Manipulation

Push

Top 7

5

Stack Manipulation

Pop

Top 7

5

Stack Manipulation

Pop

Top 6

5

Stack Manipulation

Pop

Top 6

5

Stack

Top 6 Out (LIFO) order

returns it

4

• O(1) “peek” which returns the element at the top

without removing

would be most appropriate to implement a stack?

Queue

First In First Out (FIFO) order

5 4 6 the tail

element from the head and returns it

Tail Head

• O(1) “peek” which returns the element at

the head without removing

Queue Manipulation

Enqueue

5 4 6

Tail Head

Queue Manipulation

Enqueue

7 5 4 6

Tail Head

Queue Manipulation

Dequeue

7 5 4 6

Tail Head

Queue Manipulation

Dequeue

7 5 4 6

Tail Head

Queue Manipulation

Dequeue

7 5 4 6

Tail Head

Queue Manipulation

Dequeue

7 5 4

Tail Head

Queue

First In First Out (FIFO) order

5 4 6 the tail

element from the head and returns it

Tail Head

• O(1) “peek” which returns the element at

the head without removing

would be most appropriate to implement a queue?

Tree

Root 5

which is itself a tree

Binary Tree

Root 5

1, or 2 children.

Binary Tree Traversal

Root 1

2 3 4,6,2,5,1,3

1,2,4,6,5,3

6,4,5,2,3,1

4 5

6

Binary Tree Traversal

Depth-first

Root 1

2 3

possible down one branch before trying

other branches.

visit:

1, 2, 4, 6, 5, 3

4 5

6

Binary Tree Traversal

Depth-first

Root 1

2 3

possible down one branch before trying

other branches.

visit:

1, 2, 4, 6, 5, 3

4 5

far, which would be most appropriate

6

to use during a depth-first search?

Binary Tree Traversal

Breadth-first

Root 1

2 3

the current node before moving to the

next level

visit:

1, 2, 3, 4, 5, 6

4 5

6

Binary Tree Traversal

Breadth-first

Root 1

2 3

the current node before moving to the

next level

visit:

1, 2, 3, 4, 5, 6

4 5

far, which would be most appropriate

6

to use during a breadth-first search?

Binary Search Tree

Root 5

a node must have a value less than the

node, and every right descendent of a node

3 9 must have a value greater than the node

• O(log n) insertion

give me all nodes with values between 5 and

9)

15

BST Traversal

Searching for “6”

Pointer 5

3 9

1 6 22

15

BST Traversal

Searching for “6”

Pointer 5

6 is greater than 5, so

3 9

we move right

1 6 22

15

BST Traversal

Searching for “6”

3 9 Pointer

6 is less than 9, so we

1 6 22

move left

15

BST Traversal

Searching for “6”

3 9

6 is equal to 6. Found it

1 6 22

Pointer

15

BST Balancing

1

complexity of searching in this

tree?

BST as a Map

If we associate both a key and a value with

“jon”:

29 each node, we can use a BST as a map.

we can search the BST in O(log n) time

“liam”: (using alphabetical order).

“cat”:

20 30

Other features of the BST still apply. E.g. we

can query for everyone who’s name begins

with “j”

“amy”: “kate”: “pete”:

19 30 21

“mary”

: 18

Hashing

“Hello there” 4357752745

1886414098

Diﬀerent objects would ideally result in a diﬀerent hash values (but can’t always)

Hash Table

Let’s say we can hash a string into a number between 0 and 7

and then using modulus to restrict it to 0-7)

Hash Table

Using our hash function and this array, we can create a map

882751264 % 8 = 0

Hash Table

Using our hash function and this array, we can create a map

882751264 % 8 = 0

Hash Table

Hash Table

{key: “cat”, value: 20}

{key: “jon”, value: 29}

Hash Table

{key: “cat”, value: 20}

{key: “jon”, value: 29}

And because the same key will always hash to the same

index, we can retrieve our values in O(1) time

Hash Collisions

{key: “cat”, value: 20}

{key: “jon”, value: 29}

Hash Collisions

{key: “cat”, value: 20}

{key: “jon”, value: 29}

this. Linear probing and Separate chaining.

Linear Probing

30}

{key: “cat”, value: 20}

{key: “jon”, value: 29}

With linear probing, we’d just insert our new data in the next

empty index after the one we hashed to.

When fetching data, if we don’t find the key we want in the first

index, we’d have to search through the rest of the array until we find

an empty slot.

This means, if your array is quite full. Inserting and fetching could be

O(n). You must ensure your array doesn’t get too full.

Linear Probing

30}

{key: “cat”, value: 20}

{key: “jon”, value: 29}

However we will have to re-hash every element for the new array

size. This means that some inserts will be O(n). It is amortised

O(1).

element in the middle of a sequence of items which all hash to the

same index, those which follow the deleted element may become

inaccessible

Separate Chaining

{key: “cat”,

{key: “jon”, {key: “pete”,

value: 20}

value: 29} value: 21}

elements which hash to that index

Separate Chaining

{key: “cat”,

{key: “jon”, {key: “pete”,

value: 20}

value: 29} value: 21}

{key:

“liam”,

value: 30}

Separate Chaining

{key: “cat”,

{key: “jon”, {key: “pete”,

value: 20}

value: 29} value: 21}

{key:

“liam”,

value: 30}

Worst case is O(n) lookup and set (if we have a bad hash function

and everything is in a single bucket).

gives O(1) set and lookup

HashMap vs TreeMap

with a BST

min, range queries, and previous/next

Set

1 400

9

22

7

O(1) time

Set

1 400

9

22

7

O(1) time

Set

1 400

9

22

7

have O(log n) insert and check. It would also allow for

range queries, min/max, etc.

Graph

jon

pete

cat

liam amy

networks, circuits, etc.

Graph Usage

jon

pete

cat

liam amy

between cat and amy? What is the complexity of this?

Graph

London 15

Belfast

55

Hangzhou 392

134

Beĳing Paris

315

The edges may have weights, for example to indicate cost or distance

max edge weights between two nodes (e.g.

Djikstra’s Algorithm, A*)

Graph Representation

Adjacency Matrix

jon

cat 0 0 1 0 0

jon 0 0 1 1 1 pete

cat

liam 1 1 0 1 0

amy 0 1 1 0 0

liam amy

pete 0 1 0 0 0

• Checking for an edge takes O(1) time

• Sparse graphs consume a lot of space

• Adding a vertex is O(V2) time

Graph Representation

Adjacency Matrix

jon

pete

cat

liam amy

• A collection of linked lists. One linked list per

node.

• Saves space in a sparse graph

• Queries of edge existing between two nodes

Pete amy is O(V)

• Adding an edge is O(1)

• Adding a vertex is O(1)

Heap

100

19 36

17 3 1

2 7

greater than or equal to its descendants

• Remove maximum value in O(log n)

• Insert in O(log n)

the minimum is at the root of the heap.

Heap Manipulation

Remove max

100

19 36

17 3 1

2 7

Heap Manipulation

Remove max

100

19 36

17 3 1

2 7

Heap Manipulation

Remove max

7

19 36

17 3 1

2 7

Heap Manipulation

Remove max

7

19 36

17 3 1

2 7

Heap Manipulation

Remove max

7

19 36

17 3 1

Heap Manipulation

Remove max

7

19 36

17 3 1

Heap Manipulation

Remove max

7

19 36

17 3 1

Swap them

Heap Manipulation

Remove max

36

19 7

17 3 1

Swap them

Heap Manipulation

Remove max

36

19 7

17 3 1

Heap Manipulation

Remove max

36

19 7

17 3 1

No? Finished

Heap Manipulation

Remove max

36

19 7

17 3 1

Heap Manipulation

Inserting

36

19 7

17 3 1

2

Heap Manipulation

Inserting

36

19 7

17 3 1

2 22

Heap Manipulation

Inserting

36

19 7

17 3 1

2 22

Heap Manipulation

Inserting

36

19 7

17 3 1

2 22

Heap Manipulation

Inserting

36

19 7

22 3 1

2 17

Heap Manipulation

Inserting

36

19 7

22 3 1

2 17

Heap Manipulation

Inserting

36

19 7

22 3 1

2 17

Heap Manipulation

Inserting

36

22 7

19 3 1

2 17

Heap Manipulation

Inserting

36

22 7

19 3 1

2 17

Heap Manipulation

Inserting

36

22 7

19 3 1

2 17

No? Finished

Heap Manipulation

Inserting

36

22 7

19 3 1

2 17

Heap Implementation

36

22 7

19 3

36 22 7 19 3

A heap can be implemented using an array, with the top of the heap at the 0th index

Priority Queue

This pairs each element with a “priority” and elements are returned in priority order

Agenda

1. Calculating Runtimes (Big O)

2. Algorithms

A. Searching (Linear/Binary)

B. Sorting (Selection/Insertion/Bubble/Merge/Quick/Counting)

3. Data Structures

C. Array

D. Lists(Single/Double)

E. ArrayList

F. Stack

G. Queue

H. Tree

J. Hash Table

K. Set

L. Graph

M. Heap

Recommended Reading

Books

The Algorithm Design Manual - Steven Skiena

Introduction to Algorithms - CSRL

Courses

MIT Introduction to Computer Science and Programming

Harvard CS50

Coursera Data Structures and Algorithms Specialization

Videos

Steven Skiena Algorithm Lectures

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.