Lectures 7-8.

Heapsort
Introduction for Heapsort

¢ Heapsort
u Running time: O(n lg n)
ê Like merge sort
u Sorts in place: only a constant number of array elements
are stored outside the input array at any time
ê Like insertion sort

¢ Heap
u A data structure used by Heapsort to manage information
during the execution of the algorithm
u Can be used as an efficient priority queue
Perfect Binary Tree

¢ For binary tree with height h

u All nodes at levels h–1 or less have 2 children (full)

h=0

h=1 h=2 h=3

Complete Binary Trees

¢ For binary tree with height h

u All nodes at levels h–2 or less have 2 children (full)
u All leaves on level h are as far left as possible

h=0

h=2

h=1
Complete Binary Trees

h=3
Heaps
¢ Two key properties
u Complete binary tree
u Value at node
ê Smaller than or equal to values in subtrees
ê Greater than or equal to values in subtrees

¢ Example max-heap
uY£X
X
uZ£X

Y Z
Heap and Non-heap Examples

5
5

8
5 45

6 45
5 2
2
6 22 6 22
6 22
25 8 45 25
8 45 25
Min-heaps Non-heaps
Binary Heap
¢ An array that can be viewed as a nearly
complete binary tree
u Each tree node corresponds to an array element
that stores the value in the tree node
u The tree is completely filled on all levels except possibly the
lowest, which is filled from the left up to a point
u Array A has two attributes
ê length[A]: number of elements in the array
ê heap-size[A]: number of elements in the heap stored within A
ê heap-size[A] £ length[A]
u max-heap and min-heap
Max-heap
 Length and Heap-Size

11 7

11 7

Length = 10
Heap-Size = 7?
Heap Computation
¢ Given the index i of a node, the indices of its parent,
left child, and right child can be computed simply:

u PARENT (i) : return ëi / 2û

u LEFT (i) : return 2i
u RIGHT (i) : return 2i + 1
 Heap Computation
1
parent(i) = floor(i/2)
0 16 left-child(i) = 2i
2 3 right-child(i)= 2i +1
1 14 10
4 5 6 7
2 8 7 9 3

8 9 10
1 2 3 4 5 6 7 8 9 10
3 2 4 1 16 14 10 8 7 9 3 2 4 1
Heap Property

¢ Heap property
u The property that the values in the node must satisfy
¢ Max-heap property, for every node i other than the
root
u A[PARENT(i)] ³ A[i]
u The value of a node is at most the value of its parent
u The largest element in a max-heap is stored at the root
u The subtree rooted at a node contains values
no larger than value of the node itself
Heap Height

¢ The height of a node in a heap

u The number of edges on the longest simple downward path
from the node to a leaf
¢ The height of a heap is the height of its root
u The height of a heap of n elements is Q(lg n)
ê Exercise 6.1-2 on page 153
Heap Procedures

¢ MAX-HEAPIFY
u Maintains the max-heap property
u O(lg n)
¢ BUILD-MAX-HEAP
u Produces a max-heap from an unordered input array
u O(n)
¢ HEAPSORT
u Sorts an array in place
u O(n lg n)
Maintaining the Heap Property

¢ MAX-HEAPIFY
u Inputs: an array A and an index i into the array
u Assume the binary trees rooted at LEFT(i) and RIGHT(i) are
max-heaps, but A[i] may be smaller than its children
ê violate the max-heap property
u MAX-HEAPIFY lets the value at A[i] “float down” in the
max-heap
MAX-HEAPIFY

Extract the indices of LEFT and RIGHT

children of i

Choose the largest of A[i], A[l], A[r]

Float down A[i] recursively

Example of MAX-HEAPIFY (1/3)

16

4 10
4 < 14
14 7 9 3

2 8 1
Example of MAX-HEAPIFY (2/3)

16

14 10

4 7 9 3
4<8

2 8 1
Example of MAX-HEAPIFY (3/3)

16

14 10

8 7 9 3

2 4 1
MAX-HEAPIFY

Iterative version Recursive version

Running Time of MAX-HEAPIFY

¢ Q(1) to find out the largest among

A[i], A[LEFT(i)], and A[RIGHT(i)]
¢ Plus the time to run MAX-HEAPIFY on a
subtree rooted at one of the children of node i
u The children’s subtrees each have size
at most 2n/3 (why?)
ê the worst case occurs when the last row of the tree is
exactly half full
ê 2h/2 + 2h – 1 = n -> 2h ≃ 2n/3

¢ T(n) £ T(2n/3) + Q(1)

u By case 2 of the master theorem
u T(n) = O(lg n) 7/11 = 0.63
Heapify Example
Building a Max-Heap

¢ Observation: A[(ën/2û+1)..n] are all leaves of the tree

u Exercise 6.1-7 on page 154
u Each is a 1-element heap to begin with
¢ Upper bound on the running time
u O(lg n) for each call to MAX-HEAPIFY, and call n times à O(n lg n)
ê Not tight
Building a
Max-Heap
Cost for Build-MAX-HEAP
¢ Heap-properties of an n-element heap
u Height = ëlg nû
u At most én/2h+1ù nodes of any height h
ê Exercise 6.3-3 on page 159

ëlg n û ëlg n û h ¥
é n ù h
å
h =0
êê 2h+1 úú O(h) = O(n å 2h ) = O(nå 2h ) = O(n)
h =0 h =0

¥ 1
Ignore the constant ½ h 2 =2
å h
=
h =0 2 (1 - 1 ) 2
2

¥
x
å kx k =
k =0 (1 - x) 2 (for |x| < 1)
 Heapsort

¢ Using BUILD-MAX-HEAP to build a max-heap on the input array A[1..n],

where n=length[A]
¢ Exchange the maximum element A[1] and A[n]
u Then discard node n from the heap by decrementing heap-size(A)
¢ A[2..n-1] remain max-heaps, but A[1] may violate
u call MAX-HEAPIFY(A, 1) to restore the max-heap property
for A[1..n-1]
¢ Repeat the above process from n down to 2
¢ Cost: O(n lg n)
u BUILD-MAX-HEAP: O(n)
u Each of the n-1 calls to MAX-HEAPIFY takes time O(lg n)
Heapsort Algorithm
Example: Heapsort (1/8)
1
16
2 3
14 10
4 5 6 7
8 7 9 3

8 9 10
2 4 1 16 14 10 8 7 9 3 2 4 1
Example: Heapsort (2/8)
1
14
2 3
8 10
4 5 6 7
4 7 9 3

8 9 10
2 1 16
Example: Heapsort (3/8)
1
10
2 3
8 9
4 5 6 7
4 7 1 3

8 9 10
2 14 16
Example: Heapsort (4/8)
1
9
2 3
8 3
4 5 6 7
4 7 1 2

8 9 10
10 14 16
Example: Heapsort (5/8)
1
7
2 3
4 3
4 5 6 7
1 2 8 9

8 9 10
10 14 16
Example: Heapsort (6/8)
1
4
2 3
2 3
4 5 6 7
1 7 8 9

8 9 10
10 14 16
Example: Heapsort (7/8)
1
1
2 3
2 3
4 5 6 7
4 7 8 9

8 9 10
10 14 16
Example: Heapsort (8/8)
1
1
2 3
2 3
4 5 6 7
4 7 8 9
8 9 10
10 14 16
1 2 3 4 7 8 9 10 14 16
Heapsort Algorithm
Priority Queues

¢ A priority queue is a data structure for maintaining a set

of elements, each with an associated value called a key.
u max-priority queues and min-priority queues

¢ A max-priority queue supports the operations:

u Maximum(A) returns the maximum element/key
u Extract-Max(A) removes and returns the maximum element/key
u Increase-Key(A, i, key) increases value of ith node to key
u Insert(A, key) inserts key into A
 Extract-Max
Q(1)

O(lg n)
 Increase-Key

O(lg n)
Example: increase key (1/4)
1
16
2 3
14 10
4 5 6 7
8 7 9 3

8 9 10
2 4 1 increase 4 to 15
Example: increase key (2/4)
1
16
2 3
14 10
4 5 6 7
8 7 9 3

8 9 10
2 15 1
Example: increase key (3/4)
1
16
2 3
14 10
4 5 6 7
15 7 9 3

8 9 10
2 8 1
Example: increase key (4/4)
1
16
2 3
15 10
4 5 6 7
14 7 9 3

8 9 10
2 8 1
 Insert-Max

O(lg n)
 Practice Problems
¢ Show the resulting heap after insert 20 into the following
heap