You are on page 1of 75

CS61BL Summer 2016

Lecture 5
Balanced BSTs
A B-Tree is a perfectly balanced tree in which each node can have up to (B-1)
entries and up to B children. Each internal node has one more child than it has
entries.
Balanced BSTs: 2-3-4 Tree
A 2-3-4 Tree is a B-Tree of order 4.

20 40 70

10 26 55 88 90

1 3 4 17 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree
● Entry find(K k):
○ At each node, check for a key matching k.
○ If not found, move down to the appropriate child by comparing k against the keys in the node.
○ Continue until k is found, or at a leaf not containing k.

find(3)

20 40 70

10 26 55 88 90

1 3 4 17 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree
● Entry find(K k):
○ At each node, check for a key matching k.
○ If not found, move down to the appropriate child by comparing k against the keys in the node.
○ Continue until k is found, or at a leaf not containing k.

find(3)

20 40 70

10 26 55 88 90

1 3 4 17 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree
● insert(K k, V v): (inserting key value pair)
○ Walk down the tree like finding k.
○ When encountering a 3-key-node, kick the middle value up as shown: 20
○ This ensures there’s room when we reach a leaf (and room to kick up)
○ When reaching a leaf, add the key to it’s ordered position.
1 30 40 70

20 40

1 30 70
Balanced BSTs: 2-3-4 Tree
● insert(K k, V v): (inserting key value pair)
○ Walk down the tree like finding k.
○ When encountering a 3-key-node, kick the middle value up as shown:
○ This ensures there’s room when we reach a leaf (and room to kick up)
○ When reaching a leaf, add the key to it’s ordered position.

insert(8)

20 40 70

10 18 26 55 88 90

1 3 4 17 19 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree

insert(8)

40

20 70

10 18 26 55 88 90

1 3 4 17 19 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree

insert(8)

40

20 70

10 18 26 55 88 90

1 3 4 17 19 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree

insert(8)

40

20 70

3 10 18 26 55 88 90

1 4 17 19 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree

insert(8)

40

20 70

3 10 18 26 55 88 90

1 4 8 17 19 22 39 41 66 77 89 99
Balanced BSTs: 2-3-4 Tree

insert(8)

40
Newly created 3-node;
will be broken up next
insert through this path 20 70

3 10 18 26 55 88 90

1 4 8 17 19 22 39 41 66 77 89 99
Balanced BSTs: Runtime
● Between 2h and 4h leaves in a 2-3-4 tree.
● Thus n >= 2h and log(n) >= h, meaning h = O(log(n))
● Thus insert is in O(log(n))
Balanced BSTs: Red-Black
● B-Trees can be hard to code, especially the remove() operation (omitted)
● Red-Black trees are binary search trees that maintain the balanced-ness
guarantee of 2-3-4 trees.
● Maintain certain invariants through rotations:
○ A node is either red or black.
○ The root & leaves are black.
○ If a node is red, both of its children are black.
○ Every path from a given node to any of its leaf descendants must contain the same number of
black nodes.
● Convertible to 2-3-4 trees and vice versa
● Used in , !
Balanced BSTs: Red-Black conversion
20 40 70

10 18 26 55 88 90
40

20 70

10 26 55 88

nil 18 nil nil nil nil nil 90

nil nil nil nil


Balanced BSTs: In Lab
● Implementations!
● 2-3 Trees ↔ Left-Leaning Red-Black Tree
○ Slightly simpler version, still maintains same asymptotic benefits
Priority Queues & Heaps
(quick break)
● Used to order elements according to their priority
● Optimized operations for getting the head of the queue (which is either the
min or max element depending on how the elements are ordered)
● Implemented using a priority heap

Inserts the specified element into this priority queue.

Retrieves, but does not remove, the head of this queue, or returns null if this queue is empty.

Retrieves and removes the head of this queue, or returns null if this queue is empty.
Min (Max) Heap
A min (max) heap is a tree with the following two
1
properties/invariants:

1. All children of a node hold larger (smaller)


3 5
elements than the node does.
○ The root (top element) is the min (max) element.

7 4 6 8
Min (Max) Heap
A min (max) heap is a tree with the following two
1
properties/invariants:

1. All children of a node hold larger (smaller)


3 5
elements than the node does.
○ The root (top element) is the min (max) element.
2. It is a complete tree (every level is completely
filled except for the lowest level, and the 7 4 6 8
lowest level is filled from left to right).
○ Balanced tree (helpful for runtime of operations)
Min (Max) Heap
A min (max) heap is a tree with the following two
1
properties/invariants:

1. All children of a node hold larger (smaller)


3 5
elements than the node does.
○ The root (top element) is the min (max) element.
2. It is a complete tree (every level is completely
filled except for the lowest level, and the 7 4 6 8
lowest level is filled from left to right).
○ Balanced tree (helpful for runtime of operations)
1 3 5 7 4 6 8
○ Can be represented using an array (where first index
of array is unused)
What do you think? Min (Max) Heap
Is there any relationship between two nodes , where
≠ , is not a descendant of and is not a 1
descendant of ?

3 5

7 4 6 8

1 3 5 7 4 6 8
What do you think? Min (Max) Heap
Is there any relationship between two nodes , where
≠ , is not a descendant of and is not a 1
descendant of ? No.

For a binary heap, if a node is at index of the array, 3 5


what indices are its children/parent located at?

7 4 6 8

1 3 5 7 4 6 8
What do you think? Min (Max) Heap
Is there any relationship between two nodes , where
≠ , is not a descendant of and is not a 1
descendant of ? No.

For a binary heap, if a node is at index of the array, 3 5


what indices are its children/parent located at?
● children:
● parent:
7 4 6 8
Indices for a -ary heap (where each node has
children rather than the two children of a binary heap)?
1 3 5 7 4 6 8
What do you think? Min (Max) Heap
Is there any relationship between two nodes , where
≠ , is not a descendant of and is not a 1
descendant of ? No.

For a binary heap, if a node is at index of the array, 3 5


what indices are its children/parent located at?
● children:
● parent:
7 4 6 8
Indices for a -ary heap (where each node has
children rather than the two children of a binary heap)?
1 3 5 7 4 6 8
● children: …
● parent:
Binary Heap:
1. Place the element in the next open spot in the array (leftmost open spot in the
last row of the heap).
2. Swap elements upwards (by comparing the added element with parent
elements) until the heap invariants are satisfied.

7
Binary Heap:
1. Place the element in the next open spot in the array (leftmost open spot in the
last row of the heap).
2. Swap elements upwards (by comparing the added element with parent
elements) until the heap invariants are satisfied.

7 7

5
Binary Heap:
1. Place the element in the next open spot in the array (leftmost open spot in the
last row of the heap).
2. Swap elements upwards (by comparing the added element with parent
elements) until the heap invariants are satisfied.

7 7 5

5 7
Binary Heap:
1. Place the element in the next open spot in the array (leftmost open spot in the
last row of the heap).
2. Swap elements upwards (by comparing the added element with parent
elements) until the heap invariants are satisfied.

7 7 5 5

5 7 7 3
Binary Heap:
1. Place the element in the next open spot in the array (leftmost open spot in the
last row of the heap).
2. Swap elements upwards (by comparing the added element with parent
elements) until the heap invariants are satisfied.

7 7 5 5 3

5 7 7 3 7 5
Binary Heap:

What happens if we call


7 5 and then ?
Binary Heap:

What happens if we call


7 5 and then ?

7 5

4
Binary Heap:

What happens if we call


7 5 and then ?

3 3

7 5 4 5

4 7
Binary Heap:

3
3
What happens if we call
7 5 and then ?
4 5

7 1
3 3

7 5 4 5

4 7
Binary Heap:

3
3 3
What happens if we call
7 5 and then ?
4 5 1 5

7 1 7 4
3 3

7 5 4 5

4 7
Binary Heap:

3
3 3
What happens if we call
7 5 and then ?
4 5 1 5

7 1 7 4
3 3
1

7 5 4 5
3 5

4 7
7 4
Binary Heap:
1. Swap the root with the last element in the array (rightmost filled spot in the
last row of the heap).
2. Swap elements downwards (by comparing the new root element with its
children) until the heap invariants are satisfied.

5 7
Binary Heap:
1. Swap the root with the last element in the array (rightmost filled spot in the
last row of the heap).
2. Swap elements downwards (by comparing the new root element with its
children) until the heap invariants are satisfied.

3 7

5 7 5 3
Binary Heap:
1. Swap the root with the last element in the array (rightmost filled spot in the
last row of the heap).
2. Swap elements downwards (by comparing the new root element with its
children) until the heap invariants are satisfied.

3 7 5

5 7 5 3 7
Binary Heap:
1. Swap the root with the last element in the array (rightmost filled spot in the
last row of the heap).
2. Swap elements downwards (by comparing the new root element with its
children) until the heap invariants are satisfied.

3 7 5 5

5 7 5 3 7 7
Binary Heap:
1. Swap the root with the last element in the array (rightmost filled spot in the
last row of the heap).
2. Swap elements downwards (by comparing the new root element with its
children) until the heap invariants are satisfied.

3 7 5 5 7

7
5 7 5 3 7 7 5
Binary Heap:
1

3 5

7 4

What happens if we
call twice?
Binary Heap:
1 1

3 5 3 5

7 4 7 4

What happens if we
call twice?
Binary Heap:
1 1 4

3 5 3 5 3 5

7 4 7 4 7 1

What happens if we
call twice?
Binary Heap:
1 1 4 3

3 5 3 5 3 5 4 5

7 4 7 4 7 1 7

What happens if we
call twice?
Binary Heap:
1 1 4 3

3 5 3 5 3 5 4 5

7 4 7 4 7 1 7

What happens if we 3
call twice?

4 5

7
Binary Heap:
1 1 4 3

3 5 3 5 3 5 4 5

7 4 7 4 7 1 7

What happens if we 3 7
call twice?

4 5 4 5

7 3
Binary Heap:
1 1 4 3

3 5 3 5 3 5 4 5

7 4 7 4 7 1 7

What happens if we 3 7 4
call twice?

4 5 4 5 7 5

7 3
Worst-case runtime of heap operations

Operation Description Runtime for binary heap Runtime for -ary heap

1. Place the element in the next


open spot in the array.
2. Swap elements upwards (by
comparing the new element with
its parent) until valid heap.

Access the root

1. Swap the root with the last


element in the array.
2. Swap elements downwards
(by comparing the new root with
its children) until valid heap.
Worst-case runtime of heap operations

Operation Description Runtime for binary heap Runtime for -ary heap

1. Place the element in the next 1. ϴ


open spot in the array. 2. ϴ
2. Swap elements upwards (by
comparing the new element with Total: ϴ
its parent) until valid heap.

Access the root

1. Swap the root with the last


element in the array.
2. Swap elements downwards
(by comparing the new root with
its children) until valid heap.
Worst-case runtime of heap operations

Operation Description Runtime for binary heap Runtime for -ary heap

1. Place the element in the next 1. ϴ 1. ϴ


open spot in the array. 2. ϴ 2. ϴ
2. Swap elements upwards (by
comparing the new element with Total: ϴ Total: ϴ
its parent) until valid heap.

Access the root

1. Swap the root with the last


element in the array.
2. Swap elements downwards
(by comparing the new root with
its children) until valid heap.
Worst-case runtime of heap operations

Operation Description Runtime for binary heap Runtime for -ary heap

1. Place the element in the next 1. ϴ 1. ϴ


open spot in the array. 2. ϴ 2. ϴ
2. Swap elements upwards (by
comparing the new element with Total: ϴ Total: ϴ
its parent) until valid heap.

Access the root ϴ ϴ

1. Swap the root with the last


element in the array.
2. Swap elements downwards
(by comparing the new root with
its children) until valid heap.
Worst-case runtime of heap operations

Operation Description Runtime for binary heap Runtime for -ary heap

1. Place the element in the next 1. ϴ 1. ϴ


open spot in the array. 2. ϴ 2. ϴ
2. Swap elements upwards (by
comparing the new element with Total: ϴ Total: ϴ
its parent) until valid heap.

Access the root ϴ ϴ

1. Swap the root with the last 1. ϴ


element in the array. 2. ϴ
2. Swap elements downwards
(by comparing the new root with Total: ϴ
its children) until valid heap.
Worst-case runtime of heap operations

Operation Description Runtime for binary heap Runtime for -ary heap

1. Place the element in the next 1. ϴ 1. ϴ


open spot in the array. 2. ϴ 2. ϴ
2. Swap elements upwards (by
comparing the new element with Total: ϴ Total: ϴ
its parent) until valid heap.

Access the root ϴ ϴ

1. Swap the root with the last 1. ϴ 1. ϴ


element in the array. 2. ϴ 2. ϴ
2. Swap elements downwards
(by comparing the new root with Total: ϴ Total: ϴ
its children) until valid heap.
Heapify
● Naive approach of creating a heap of size N
○ calls to , each of which takes ϴ
○ Total runtime: ϴ
● Can we do better?
○ Yes, we can heapify an array into an heap.
○ Total runtime: ϴ
Heapify
7 4 6 8 1 3 5
1. Start with an input array.
2. Begin at the middle index of the array (first 1
node with children).
3. Do the following at index :
a. Check that the th element is smaller (larger) than its 4 3
children to satisfy the heap invariant.
b. If necessary, swap the th element with its smallest
(largest) child and repeat step 3 for the child index that 8 7 6 5
the element was swapped into.
4. If the th index is not the root, repeat step 3 for
index . 1 4 3 8 7 6 5
Heapify 7 4 6 8 1 3 5

4 6

8 1 3 5
Heapify 7 4 6 8 1 3 5

4 6

8 1 3 5
Heapify 7 4 3 8 1 6 5

7 7

4 6 4 3

8 1 3 5 8 1 6 5
Heapify 7 4 3 8 1 6 5

7 7

4 6 4 3

8 1 3 5 8 1 6 5
Heapify 7 4
1 3 8 1
4 6 5

7 7 7

4 6 4 3 1 3

8 1 3 5 8 1 6 5 8 4 6 5
Heapify 7 4
1 3 8 1
4 6 5

7 7 7

4 6 4 3 1 3

8 1 3 5 8 1 6 5 8 4 6 5
Heapify 7
1 4
7 3 8 7
1
4 6 5

7 7 7

4 6 4 3 1 3

8 1 3 5 8 1 6 5 8 4 6 5

7 3

8 4 6 5
Heapify 7
1 4 3 8 1
7 6 5

7 7 7

4 6 4 3 1 3

8 1 3 5 8 1 6 5 8 4 6 5

1 1

7 3 4 3

8 4 6 5 8 7 6 5
Heapify 7
1 4 3 8 1
7 6 5

7 7 7

4 6 4 3 1 3

8 1 3 5 8 1 6 5 8 4 6 5

1 1

7 3 4 3

8 4 6 5 8 7 6 5
Worst-Case Heapify Runtime Proof Legend
height of tree

level 0 level # # of nodes height of total work


in level node* for level

level 1
... ...

... level h-1

... level h
Worst-Case Heapify Runtime Proof Legend
height of tree

level 0 level # # of nodes height of total work


in level node* for level

level 1
... ...

... level h-1

... level h
Worst-Case Heapify Runtime Proof Legend
height of tree

level 0 level # # of nodes height of total work


in level node* for level

level 1
... ...

... level h-1

... level h
Worst-Case Heapify Runtime Proof Legend
height of tree

level 0 level # # of nodes height of total work


in level node* for level

level 1
... ...

... level h-1

... level h *Why do we care about the height of the nodes?


This is equal to the max # of swaps to move the
node into the correct position (at the bottom).
Worst-Case Heapify Runtime Proof Legend
height of tree

level 0 level # # of nodes height of total work


in level node* for level

level 1
... ...

... level h-1

... level h *Why do we care about the height of the nodes?


This is equal to the max # of swaps to move the
node into the correct position (at the bottom).
Worst-Case Heapify Runtime Proof Legend
height of tree

level 0 level # # of nodes height of total work


in level node* for level

level 1
... ...

... level h-1

... level h *Why do we care about the height of the nodes?


This is equal to the max # of swaps to move the
node into the correct position (at the bottom).
Worst-Case Heapify Runtime Proof Legend
height of tree

level # # of nodes height of total work


in level node* for level

*Why do we care about the height of the nodes?


This is equal to the max # of swaps to move the
node into the correct position (at the bottom).
Worst-Case Heapify Runtime Proof Legend
height of tree

level # # of nodes height of total work


in level node* for level

How? WolframAlpha :3

*Why do we care about the height of the nodes?


This is equal to the max # of swaps to move the
node into the correct position (at the bottom).
Worst-Case Heapify Runtime Proof Legend
height of tree

level # # of nodes height of total work


in level node* for level

How? WolframAlpha :3

Recall that .

ϴ
*Why do we care about the height of the nodes?
This is equal to the max # of swaps to move the
node into the correct position (at the bottom).
Project 2 Open Q&A