Priority Queues

Binary and Binomial Heaps Supports the following operations.

■ Insert element x.
■ Return min element.
■ Return and delete minimum element.
■ Decrease key of element x to k.

Applications.
These lecture slides are adapted
from CLRS, Chapters 6, 19. ■ Dijkstra’s shortest path algorithm.
■ Prim’s MST algorithm.
■ Event-driven simulation.
■ Huffman encoding.
■ Heapsort.
■ ...

Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne 2

Priority Queues in Action Priority Queues

Heaps
Operation Linked List Binary Binomial Fibonacci * Relaxed
Dijkstra’s Shortest Path Algorithm
make-heap 1 1 1 1 1
PQinit() insert 1 log N log N 1 1
for each v ∈ V find-min N 1 log N 1 1
key(v) ← ∞
PQinsert(v) delete-min N log N log N log N log N
union 1 N log N 1 1
key(s) ← 0
decrease-key 1 log N log N 1 1
while (!PQisempty())
v = PQdelmin() delete N log N log N log N log N
for each w ∈ Q s.t (v,w) ∈ E is-empty 1 1 1 1 1
if π(w) > π(v) + c(v,w)
PQdecrease(w, π(v) + c(v,w))
Dijkstra/Prim
O(|V|2) O(|E| log |V|) O(|E| + |V| log |V|)
1 make-heap
|V| insert
|V| delete-min
|E| decrease-key
3 4
 Binary Heap: Definition Binary Heap: Properties
Binary heap. Properties.
■ Almost complete binary tree. ■ Min element is in root.
– filled on all levels, except last, where filled from left to right ■ Heap with N elements has height = log2 N.
■ Min-heap ordered.
– every child greater than (or equal to) parent

06 06
N = 14
Height = 3
14 45 14 45

78 18 47 53 78 18 47 53

83 91 81 77 84 99 64 83 91 81 77 84 99 64

5 6

Binary Heaps: Array Implementation Binary Heap: Insertion

Implementing binary heaps. Insert element x into heap.
■ Use an array: no need for explicit parent or child pointers. ■ Insert into next available slot.
– Parent(i) = i/2 ■ Bubble up until it’s heap ordered.
– Left(i) = 2i – Peter principle: nodes rise to level of incompetence
– Right(i) = 2i + 1

06 06
1
14 45 14 45
2 3

78 18 47 53 78 18 47 53
4 5 6 7

83 91 81 77 84 99 64 83 91 81 77 84 99 64 42 next free slot

8 9 10 11 12 13 14
7 8
 Binary Heap: Insertion Binary Heap: Insertion
Insert element x into heap. Insert element x into heap.
■ Insert into next available slot. ■ Insert into next available slot.
■ Bubble up until it’s heap ordered. ■ Bubble up until it’s heap ordered.
– Peter principle: nodes rise to level of incompetence – Peter principle: nodes rise to level of incompetence

06 swap with parent 06 swap with parent

14 45 14 45

78 18 47 53 78 18 47 42

83 91 81 77 84 99 64 42 83 91 81 77 84 99 64 53
42

9 10

Binary Heap: Insertion Binary Heap: Decrease Key

Insert element x into heap. Decrease key of element x to k.
■ Insert into next available slot. ■ Bubble up until it’s heap ordered.
■ Bubble up until it’s heap ordered. ■ O(log N) operations.
– Peter principle: nodes rise to level of incompetence
■ O(log N) operations.

06 stop: heap ordered 06

14 42 14 42

78 18 47 45 78 18 47 45

83 91 81 77 84 99 64 53 83 91 81 77 84 99 64 53

11 12
 Binary Heap: Delete Min Binary Heap: Delete Min
Delete minimum element from heap. Delete minimum element from heap.
■ Exchange root with rightmost leaf. ■ Exchange root with rightmost leaf.
■ Bubble root down until it’s heap ordered. ■ Bubble root down until it’s heap ordered.
– power struggle principle: better subordinate is promoted – power struggle principle: better subordinate is promoted

06 53

14 42 14 42

78 18 47 45 78 18 47 45

83 91 81 77 84 99 64 53 83 91 81 77 84 99 64 06

13 14

Binary Heap: Delete Min Binary Heap: Delete Min

Delete minimum element from heap. Delete minimum element from heap.
■ Exchange root with rightmost leaf. ■ Exchange root with rightmost leaf.
■ Bubble root down until it’s heap ordered. ■ Bubble root down until it’s heap ordered.
– power struggle principle: better subordinate is promoted – power struggle principle: better subordinate is promoted

53 exchange with left child 14 exchange with right child

14 42 53 42

78 18 47 45 78 18 47 45

83 91 81 77 84 99 64 83 91 81 77 84 99 64

15 16
 Binary Heap: Delete Min Binary Heap: Heapsort
Delete minimum element from heap. Heapsort.
■ Exchange root with rightmost leaf. ■ Insert N items into binary heap.
■ Bubble root down until it’s heap ordered. ■ Perform N delete-min operations.
– power struggle principle: better subordinate is promoted ■ O(N log N) sort.
■ O(log N) operations. ■ No extra storage.

14 stop: heap ordered

18 42

78 53 47 45

83 91 81 77 84 99 64

17 18

Binary Heap: Union Priority Queues

Union. Heaps
■ Combine two binary heaps H1 and H2 into a single heap. Operation Linked List Binary Binomial Fibonacci * Relaxed
■ No easy solution. make-heap 1 1 1 1 1
– Ω(N) operations apparently required
insert 1 log N log N 1 1
■ Can support fast union with fancier heaps.
find-min N 1 log N 1 1
delete-min N log N log N log N log N
H1 H2
union 1 N log N 1 1
14 11
decrease-key 1 log N log N 1 1
delete N log N log N log N log N
is-empty 1 1 1 1 1
78 18 53 62

41 91 81 77 84 99 64

19 20
 Binomial Tree Binomial Tree
Binomial tree. Useful properties of order k binomial tree Bk.
■ Recursive definition: ■ Number of nodes = 2k.
B0 Bk
■ Height = k.
Bk+1
■ Degree of root = k.
■ Deleting root yields binomial
Bk-1
trees Bk-1, … , B0.
B1 B0
Bk-1 B2
Proof. Bk
■ By induction on k.

B0 B1 B2 B3 B4 B0 B1 B2 B3 B4
21 22

Binomial Tree Binomial Heap

A property useful for naming the data structure. Binomial heap. Vuillemin, 1978.
k 
■ Bk has   nodes at depth i. ■ Sequence of binomial trees that satisfy binomial heap property.
i 
– each tree is min-heap ordered
– 0 or 1 binomial tree of order k

4
 =6
2
depth 0 6 3 18

depth 1
8 29 10 44 37

depth 2
30 23 22 48 31 17

depth 3
45 32 24 50

depth 4 B4 55
B4 B1 B0
23 24
 Binomial Heap: Implementation Binomial Heap: Properties
Implementation. Properties of N-node binomial heap.
■ Represent trees using left-child, right sibling pointers. ■ Min key contained in root of B0, B1, . . . , Bk.
– three links per node (parent, left, right) ■ Contains binomial tree Bi iff bi = 1 where bn⋅ b2b1b0 is binary
■ Roots of trees connected with singly linked list. representation of N.
– degrees of trees strictly decreasing from left to right ■ At most log2 N + 1 binomial trees.
heap ■ Height ≤ log2 N.

6 3 18 6 3 18
6 3 18
nt
are
P
37 37 8 29 10 44 37
29 10 44 29

Ri
ft

gh
Le

t
30 23 22 48 31 17 N = 19
48 31 17 48 10 # trees = 3
height = 4
45 32 24 50
binary = 10011
50 50 31 17 44

55
Binomial Heap Leftist Power-of-2 Heap B4 B1 B0
25 26

Binomial Heap: Union Binomial Heap: Union

Create heap H that is union of heaps H’ and H’’.
■ "Mergeable heaps."
■ Easy if H’ and H’’ are each order k binomial trees.
6 3 18
– connect roots of H’ and H’’
– choose smaller key to be root of H
8 29 10 44 37

30 23 22 48 31 17
15 7 12
6 45 32 24 50

28 33 25
55

8 29 10 44
+ 41

30 23 22 48 31 17
1 1 1
1 0 0 1 1
45 32 24 50
19 + 7 = 26 + 0 0 1 1 1
55 1 1 0 1 0
H’ H’’
27 28
 Binomial Heap: Union Binomial Heap: Union 12

18

6 3 18 6 3 18

8 29 10 44 37 8 29 10 44 37

30 23 22 48 31 17 30 23 22 48 31 17
15 7 12 15 7 12

45 32 24 50 45 32 24 50

28 33 25 28 33 25
55 55

+ 41 + 41

29 30

3 12 3 3 12

7 37 18 15 7 37 7 37 18

25 28 33 25 25

41

6 3 18 6 3 18

8 29 10 44 37 8 29 10 44 37

30 23 22 48 31 17 30 23 22 48 31 17
15 7 12 15 7 12

45 32 24 50 45 32 24 50

28 33 25 28 33 25
55 55

+ 41 + 41

12 12

18 18

31 32
 3 3 12 3 3 12

15 7 37 7 37 18 15 7 37 7 37 18

28 33 25 25 28 33 25 25

41 41

6 3 18 6 3 18

8 29 10 44 37 8 29 10 44 37

30 23 22 48 31 17 30 23 22 48 31 17
15 7 12 15 7 12

45 32 24 50 45 32 24 50

28 33 25 28 33 25
55 55

+ 41 + 41

3 12 6 3 12

15 7 37 18 8 29 10 44 15 7 37 18

28 33 25 30 23 22 48 31 17 28 33 25

41 45 32 24 50 41

33
55 34

Binomial Heap: Union Binomial Heap: Delete Min

Create heap H that is union of heaps H’ and H’’. Delete node with minimum key in binomial heap H.
■ Analogous to binary addition. ■ Find root x with min key in root list of H, and delete
■ H’ ← broken binomial trees
Running time. O(log N) ■ H ← Union(H’, H)
■ Proportional to number of trees in root lists ≤ 2( log2 N + 1).
Running time. O(log N)

3 6 18

8 29 10 44 37

1 1 1 30 23 22 48 31 17 H
1 0 0 1 1
+ 0 0 1 1 1 45 32 24 50
19 + 7 = 26
1 1 0 1 0
55
35 36
 Binomial Heap: Delete Min Binomial Heap: Decrease Key
Delete node with minimum key in binomial heap H. Decrease key of node x in binomial heap H.
■ Find root x with min key in root list of H, and delete ■ Suppose x is in binomial tree Bk.
■ H’ ← broken binomial trees ■ Bubble node x up the tree if x is too small.
■ H ← Union(H’, H)
Running time. O(log N)
Running time. O(log N) ■ Proportional to depth of node x ≤ log2 N .

6 18 3 6 18

8 29 10 44 37 depth = 3 8 29 10 44 37

30 23 22 48 31 17 H 30 23 22 48 31 17 H

45 32 24 50 x 32 24 50

55 55
H’ 37 38

Binomial Heap: Delete Binomial Heap: Insert

Delete node x in binomial heap H. Insert a new node x into binomial heap H.
■ Decrease key of x to -∞. ■ H’ ← MakeHeap(x)
■ Delete min. ■ H ← Union(H’, H)

Running time. O(log N) Running time. O(log N)

3 6 18 x

8 29 10 44 37

30 23 22 48 31 17 H H’

45 32 24 50

55
39 40
 Binomial Heap: Sequence of Inserts Priority Queues
Insert a new node x into binomial heap H.
■ If N = .......0, then only 1 steps. 3 6 x

■ If N = ......01, then only 2 steps.

29 10 44 37
Heaps
■ If N = .....011, then only 3 steps.
Operation Linked List Binary Binomial Fibonacci * Relaxed
■ If N = ....0111, then only 4 steps. 48 31 17
make-heap 1 1 1 1 1
50
insert 1 log N log N 1 1

Inserting 1 item can take Ω(log N) time. find-min N 1 log N 1 1

■ If N = 11...111, then log2 N steps. delete-min N log N log N log N log N
union 1 N log N 1 1
But, inserting sequence of N items takes O(N) time! decrease-key 1 log N log N 1 1
■ (N/2)(1) + (N/4)(2) + (N/8)(3) + . . . ≤ 2N delete N log N log N log N log N
■ Amortized analysis. N is-empty 1 1 1 1 1
■ Basis for getting most operations ∑ nn = 2 − NN − N1−1
n=1 2 2 2
down to constant time. ≤ 2
just did this
41 42