Heap

Topics
• Priority Queues
• Heaps
• Heapsort

2
 Priority Queue
A data structure implementing a set S of elements, each
associated with a key, supporting the following
operations:
insert(S, x) : insert element x into set S
max(S) : return element of S with largest key
extract_max(S) : return element of S with largest key
and remove it from S
increase_key(S, x, k) : increase the value of element x’ s key to
new value k
(assumed to be as large as current value)

3
 Background: Binary Trees
root
Has a root at the topmost level
Each node has zero, one or two Parent(x)
children
A node that has no child is called a
leaf x
For a node x, we denote the left leaf leaf
child, right child and of x as
left(x), right(x) and the parent left(x)right(x)
parent(x), respectively.
leaf
Height (Depth) of a Binary
Tree
The number of edges on the longest path from the
root to a leaf.

Height = 4
 Background: Complete Binary
Trees
A complete binary tree is the tree
Where a node can have 0 (for the leaves) or 2 children and
All leaves are at the same depth

height no. of nodes

0 1
1 2

2 4

No. of nodes and height 3 8

A complete binary tree with N nodes has height O(logN)
d 2d
A complete binary tree with height d has, in total, 2d+1-1 nodes
Proof: O(logN) Height
Proof: a complete binary tree with N nodes has height
of O(logN)
1. Prove by induction that number of nodes at depth d is 2d
2. Total number of nodes of a complete binary tree of depth d is 1 + 2 +
4 +…… 2d = 2d+1 - 1
3. Thus 2d+1 - 1 = N
4. d = log(N+1)-1 = O(logN)
Side notes: the largest depth of a binary
tree of N nodes is O(N)
 Heap
• Implementation of a priority queue
• An array, visualized as a nearly complete binary tree
• Max Heap Property: The key of a node is ≥ than the keys
of
its children
(Min Heap defined analogously)

1
2
16 3
14 10 1 2 3 4 5 6 7 8 9
5 6 7
4 10
8 7 9 3 1614 10
8 9 8 7 9 3 2 4 1

10
2 4 1 8
(Binary) Heap
Heaps are “almost complete binary trees”
All levels are full except possibly the lowest level
If the lowest level is not full, then nodes must be packed to the left

Pack to the left

 1 4

2 5 2 5

4 3 6 1 3 6

A heap Not a heap

Heap-order property: the value at each node is less than or equal to the
values at both its descendants --- Min Heap

It is easy (both conceptually and practically) to perform insert and

deleteMin in heap if the heap-order property is maintained
Structure properties
Has 2h to 2h+1-1 nodes with height h
The structure is so regular, it can be represented in an array and no
links are necessary !!!

Use of binary heap is so common for priority queue implemen-tations, thus

the word heap is usually assumed to be the implementation of the data
structure
 Heap Properties

Heap supports the following operations efficiently

Insert in O(logN) time

Locate the current minimum in O(1) time
Delete the current minimum in O(log N) time
 Heap as a Tree
root of tree: first element in the array, corresponding to i =
1
parent(i) =i/2: returns index of node's parent
left(i)=2i: returns index of node's left child
right(i)=2i+1: returns index of node's right
child
1
2
16 3
14 10 1 2 3 4 5 6 7 8 9
5 6 7
4 10
8 7 9 3 1614 10
8 9 8 7 9 3 2 4 1

10
2No4pointers
1 required! Heig ht of a binary heap is O(lg
5
 Heap Operations
build_max_heap : produce a max-heap from an
unordered array

max_heapify : correct a single violation of the

heap
property in a subtree at its root

insert, extract_max, heapsort

14
 Insertion
Algorithm
1. Add the new element to the next available position at the lowest
level
2. Restore the min-heap property if violated
General strategy is percolate up (or bubble up): if the parent of the
element is larger than the element, then interchange the parent and
child.

1 1 1

2 5 2 5 2 2.5
swap
4 3 6 4 3 6 2.5 4 3 6 5

Insert 2.5 Percolate up to maintain

the heap property
Insertion Complexity
7

9 8
A heap!
17 16 14 10

20 18

Time Complexity = O(height) = O(logN)

Max_heapify

•Assume that the trees rooted at left(i) and right(i)

are max-heaps

•If element A[i] violates the max-heap property, correct

violation by “trickling” element A[i] down the tree,
making the subtree rooted at index i a max-heap

17
 Max_heapify (Example)

Node 10 is the left child of node 5 but is drawn to the right for
18

convenience
Max_heapify (Example)

19
 Max_heapify (Example)

Time=? O(log n)
20
 Max_Heapify Pseudocode
l = left(i)
r = right(i)
if (l <= heap-size(A) and A[l] > A[i])
then largest = lelse largest = i
if (r <= heap-size(A) and A[r] > A[largest]) then
largest = r
if largest = i
then exchange A[i] and A[largest]
Max_Heapify(A, largest)

21
 Build_Max_Heap(A)
Converts A[1…n] to a max heap

Build_Max_Heap(A):
for i=n/2 downto 1
do Max_Heapify(A, i)

Why start at n/2?

Because elements A[n/2 + 1 … n] are all leaves of the

tree 2i > n, for i > n/2 + 1

Time=? O(n log n) via simple analysis

22
 Build_Max_Heap(A) Analysis
Converts A[1…n] to a max heap

Build_Max_Heap(A):
for i=n/2 downto 1
do
Max_Heapify(
A, i)

Observe however that Max_Heapify takes O(1) for

time for nodes that are one level above the leaves, and
in general, O(l) for the nodes that are l levels above the
leaves. We have n/4 nodes with level 1, n/8 with level
above
2, the leaves.
23
 Build_Max_Heap(A) Analysis
Converts A[1…n] to a max heap

Build_Max_Heap(A):
for i=n/2 downto 1
do
Max_Heapify(
A, i)

Total amount of
work in the for
loop can be
summed as:
n/4 (1 c) + n/8 (2
c) + n/16 (3 c)
Build-Max-Heap Demo

25
Build-Max-Heap Demo

26
Build-Max-Heap

27
deleteMin: First Attempt

Algorithm
1. Delete the root.
2. Compare the two children of the root
3. Make the lesser of the two the root.
4. An empty spot is created.
5. Bring the lesser of the two children of the empty spot to the empty
spot.
6. A new empty spot is created.
7. Continue
 Example for First Attempt
1

2 5 2 5

4 3 6 4 3 6

2 1
5 3 5

4 3 6 4 6

Heap property is preserved, but completeness is not preserved!

 deleteMin
1. Copy the last number to the root (i.e. overwrite the
minimum element stored there)
2. Restore the min-heap property by percolate down (or
bubble down)
Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

32
Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the
array!

33
Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the
array!
4. Discard node n from heap
(by decrementing heap-size variable)

34
Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the array!
4. Discard node n from heap
(by decrementing heap-size variable)
5. New root may violate max heap property, but its
children are max heaps. Run max_heapify to fix
this.

35
Heap-Sort
Sorting Strategy:

1. Build Max Heap from unordered array;

2. Find maximum element A[1];

3. Swap elements A[n] and A[1]:

now max element is at the end of the array!
4. Discard node n from heap
(by decrementing heap-size variable)
5. New root may violate max heap property, but its
children are max heaps. Run max_heapify to fix
this.
6. Go to Step 2 unless heap is empty.
36
Heap-Sort Demo
Swap A[10] and
A[1]

Max_heapify(A,1)

37
 Heap-Sort Demo

Swap A[9] and

A[1]

38
Heap-Sort Demo

39
 Heap-Sort Demo

Swap A[8] and

A[1]

40
Heap-Sort

Running time:
after n iterations the Heap is empty
every iteration involves a swap and a max_heapify
operation; hence it takes O(log n) time

Overall O(n log n)

41
 Priority Queue: Motivating
Example
3 jobs have been submitted to a printer in the order A, B, C.
Sizes: Job A – 100 pages
Job B – 10 pages
Job C -- 1 page

Average waiting time with FIFO service:

(100+110+111) / 3 = 107 time units
Average waiting time for shortest-job-first service:
(1+11+111) / 3 = 41 time units

A queue be capable to insert and deletemin?

Priority Queue
 Priority Queue
Priority queue is a data structure which allows at least two
operations
insert
deleteMin: finds, returns and removes the minimum elements in the
priority queue

deleteMin Priority Queue insert

Applications: external sorting, greedy algorithms

Possible Implementations

Linked list
Insert in O(1)
Find the minimum element in O(n), thus deleteMin is O(n)
Binary search tree (AVL tree, to be covered later)
Insert in O(log n)
Delete in O(log n)
Search tree is an overkill as it does many other operations
Eerr, neither fit quite well…
References

MIT courseware
Introduction to algorithms by Thomas H. Cormen