Heaps & Priority Queues

Application of Almost Complete

Binary Trees: Heaps
A heap (or min-heap to be precise) is an almost
complete binary tree where
Every node holds a data value (or key)
The key of every node is ≤ the keys of the children

Note:
A max-heap has the same definition except that the
Key of every node is >= the keys of the children
Example of a Min-heap

5
8 20

15 11 30 27

33 16 18 12
 Operations on Heaps

Delete the minimum value and return it.

This operation is called deleteMin.
Insert a new data value

Applications of Heaps:
• A heap implements a priority queue, which is a queue
that orders entities not a on first-come first-serve basis,
but on a priority basis: the item of highest priority is at
the head, and the item of the lowest priority is at the tail

• Another application: sorting, which will be seen later

 DeleteMin in Min-heaps
The minimum value in a min-heap is at the root!
To delete the min, you can’t just remove the data
value of the root, because every node must hold a
key
Instead, take the last node from the heap, move its
key to the root, and delete that last node
But now, the tree is no longer a heap (still almost
complete, but the root key value may no longer be
≤ the keys of its children
Illustration of First Stage of deletemin

5
8 20 8 20

15 11 30 27 15 11 30 27

16 18 12 33 16 18 12
33
12 12
8 20 20
8
15 11 30 27 30
15 11 27
33 16 18 16 18
33
 Restore Heap

To bring the structure back to its

“heapness”, we restore the heap
Swap the new root key with the smaller
child.
Now the potential bug is at the one level
down. If it is not already ≤ the keys of its
children, swap it with its smaller child
Keep repeating the last step until the “bug”
key becomes ≤ its children, or the it becomes
a leaf
 Illustration of Restore-Heap
12 8
8 20 12 20

15 11 30 27 15 11 30 27

33 16 18 33 16 18

8
11 20

15 12 30 27

33 16 18 Now it is a correct heap

 Heaps (Ref.
www.cse.unt.edu/~rada/CSCE3110/Lectures/Heaps.ppt )

A heap is a binary tree T that stores a key-

element pairs at its internal nodes
It satisfies two properties:
• MinHeap: key(parent)  key(child)
• [OR MaxHeap: key(parent)  key(child)]
4
• all levels are full, except
the last one, which is
5 6

left-filled
15 9 7 20

16 25 14 12 11 8
 What are Heaps Useful for?
To implement priority queues
Priority queue = a queue where all elements
have a “priority” associated with them
Remove in a priority queue removes the
element with the smallest priority
insert
removeMin
Heap or Not a Heap?
 Heap Properties
A heap T storing n keys has height h = log(n + 1),
which is O(log n)

5 6

15 9 7 20

16 25 14 12 11 8
 Heap Insertion
Insert 6
 Heap Insertion
Add key in next available position
 Heap Insertion
Begin Unheap
Heap Insertion
 Heap Insertion
Terminate unheap when
reach root
key child is greater than key parent
 Heap Removal
Remove element
from priority queues?
removeMin( )
 Heap Removal
Begin downheap
Heap Removal
Heap Removal
 Heap Removal
Terminate downheap when
reach leaf level
key parent is greater than key child
 Building a Heap

build (n + 1)/2 trivial one-element heaps

build three-element heaps on top of them

 Building a Heap

 downheap to preserve the order property

 now form seven-element heaps

Building a Heap
Building a Heap
 Heap Implementation
Using arrays
Parent = k ; Children = 2k , 2k+1
Why is it efficient?

[1] [1] 6 [1] 30

6

[2] 12 [3] 7 [2] 9 [3] 7 [2]

31
[5] [6] [4]
[4] 18 19 9 10
 Overview
Binary trees
Perfect
Complete
Heaps
Priority queues
 Perfect Binary Tree
For binary tree with height h
All nodes at levels h–1 or less have 2 children (full)

h=1

h=2 h=3 h=4

 Complete Binary Trees
For binary tree with height h
All nodes at levels h–2 or less have 2 children (full)
All leaves on level h are as far left as possible

h=1

h=3

h=2
 Complete Binary Trees
Note: definition in
book is incorrect

h=4
 Heaps
Two key properties
Complete binary tree
Value at node
Smaller than or equal to values in subtrees
Example heap
XY X
XZ

Y Z
 Heap & Non-heap Examples
5

5 8

5 45 6 45

5 2 2

6 22 6 22 6 22

25 8 45 25 8 45 25

Heaps Non-heaps
 Heap Properties
Key operations
Insert ( X )
getSmallest ( )

Key applications
Heapsort
Priority queue
 Heap Operations – Insert( X )
Algorithm
1. Add X to end of tree
2. While (X < parent)
Swap X with parent // X bubbles up tree

Complexity
# of swaps proportional to height of tree
O( log(n) )
 Heap Insert Example
Insert ( 20 )

10 10 10

30 25 30 25 20 25

37 37 20 37 30

1) Insert to 2) Compare to parent, 3) Insert

end of tree swap if parent key larger complete
 Heap Insert Example
Insert ( 8 )

10 10 10 8

30 25 30 25 8 25 10 25

37 37 8 37 30 37 30

1) Insert to 2) Compare to parent, 3) Insert

end of tree swap if parent key larger complete
 Heap Operation – getSmallest()
Algorithm
1. Get smallest node at root
2. Replace root with X at end of tree
3. While ( X > child )
Swap X with smallest child // X drops down tree
4. Return smallest node

Complexity
# swaps proportional to height of tree
O( log(n) )
 Heap GetSmallest Example
getSmallest ()

8 30 10

10 25 10 25 30 25

37 30 37 37

1) Replace root 2) Compare node to 3) Repeat swap

with end of tree children, if larger swap if needed
with smallest child
 Heap GetSmallest Example
getSmallest ()

8 37 10 10

10 25 10 25 37 25 30 25

30 37 30 30 37

1) Replace root 2) Compare node to 3) Repeat swap

with end of tree children, if larger swap if needed
with smallest child
 Heap Implementation
Can implement heap as array
Store nodes in array elements
Assign location (index) for elements using formula
 Heap Implementation
Observations
Compact representation
Edges are implicit (no storage required)
Works well for complete trees (no wasted space)
 Heap Implementation
Calculating node locations
Array index i starts at 0
Parent(i) =  ( i – 1 ) / 2 
LeftChild(i) = 2  i +1
RightChild(i) = 2  i +2
 Heap Implementation
Example
Parent(1) =  ( 1 – 1 ) / 2  =  0 / 2  = 0
Parent(2) =  ( 2 – 1 ) / 2  =  1 / 2  = 0
Parent(3) =  ( 3 – 1 ) / 2  =  2 / 2  = 1
Parent(4) =  ( 4 – 1 ) / 2  =  3 / 2  = 1
Parent(5) =  ( 5 – 1 ) / 2  =  4 / 2  = 2
 Heap Implementation
Example
LeftChild(0) = 2  0 +1 = 1
LeftChild(1) = 2  1 +1 = 3
LeftChild(2) = 2  2 +1 = 5
 Heap Implementation
Example
RightChild(0) = 2  0 +2 = 2
RightChild(1) = 2  1 +2 = 4
 Heap Application – Heapsort
Use heaps to sort values
Heap keeps track of smallest element in heap

Algorithm
1. Create heap
2. Insert values in heap
3. Remove values from heap (in ascending order)

Complexity
O( nlog(n))
 Heapsort Example
Input
11, 5, 13, 6, 1
View heap during insert, removal
As tree
As array
Heapsort – Insert Values
Heapsort – Remove Values
 Heapsort – Insert in to Array 1
Input
11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 11 11
 Heapsort – Insert in to Array 2
Input
11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 5 11 5

Swap 5 11
 Heapsort – Insert in to Array 3
Input
11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 13 5 11 13
 Heapsort – Insert in to Array 4
Input
11, 5, 13, 6, 1

Index = 0 1 2 3 4

Insert 6 5 11 13 6

Swap 5 6 13 11

…
 Heapsort – Remove from Array 1
Input
11, 5, 13, 6, 1

Index = 0 1 2 3 4

Remove root 1 5 13 11 6

Replace 6 5 13 11

Swap w/ child 5 6 13 11
 Heapsort – Remove from Array 2
Input
11, 5, 13, 6, 1

Index = 0 1 2 3 4

Remove root 5 6 13 11

Replace 11 6 13

Swap w/ child 6 11 13
Heap Application – Priority Queue
Queue
Linear data structure Enqueue Dequeue
First-in First-out (FIFO)
Implement as array / linked list

Priority queue
Elements are assigned priority value
Higher priority elements are taken out first
Equal priority elements are taken out in FIFO order
Implement as heap
 Priority Queue
Properties
Lower value = higher priority
Heap keeps highest priority items in front
Complexity
Enqueue (insert) = O( log(n) )
Dequeue (remove) = O( log(n) )
For any heap
 Heap vs Binary Tree
Binary tree
Keeps values in sorted order
Find any value
O( log(n) ) for balanced tree
O( n ) for degenerate tree (worst case)
Heap
Keeps smaller values in front
Find minimum value
O( log(n) ) for any heap
Can also organize heap to find maximum value
Keep largest value in front