Minimum Spanning Trees

• Spanning Tree
– A tree (i.e., connected, acyclic graph) which contains
Design and Analysis of all the vertices of the graph
• Minimum Spanning Tree
Algorithms – Spanning tree with the minimum sum of weights
8 7
b c d
4 9
2
Minimum Spanning Trees (MST) 11
a i 4 14 e
7 6
8 10
Chapter 23
• Spanning forest g
1
g 2
f

Thanks to George Bebis – If a graph is not connected, then there is a spanning

tree for each connected component of the graph

1 2

Applications of MST Example

8
– Find the least expensive way to connect a set of Problem b c
7
d
9
cities, terminals, computers, etc. • A town has a set of houses 4
2
and a set of roads a 11 i 4 14 e
7 6
• A road connects 2 and only 8 10
h g f
2 houses 1 2
• A road connecting houses u and v has a repair
cost w(u, v)
Goal: Repair enough (and no more) roads such
that:
1. Everyone stays connected
i.e., can reach every house from all other houses
2. Total repair cost is minimum

3 4

3 4

Minimum Spanning Trees Properties of Minimum Spanning Trees

• A connected, undirected graph: • Minimum spanning tree is not unique
– Vertices = houses, Edges = roads

• A weight w(u, v) on each edge (u, v)  E

Find T  E such that: b

8
c
7
d
• MST has no cycles – see why:
4 9
1. T connects all vertices 2 – We can take out an edge of a cycle, and still have
a 11 i 4 14 e
the vertices connected while reducing the cost
2. w(T) = Σ(u,v)T w(u, v) is 8
7 6
10
h
1
g 2
f • # of edges in a MST:
minimized
– |V| - 1

5 6

5 6

1
 Growing a MST – Generic Approach Generic MST algorithm
• Grow a set A of edges (initially 1. A ← 
empty) 2. while A is not a spanning tree
• Incrementally add edges to A
do find an edge (u, v) that is safe for A
8 7
b c d 3.
such that they would belong 4 9
2
to a MST a 11 i 4 14 e 4. A ← A  {(u, v)} b
8
c
7
d
7 6 4 9
8 10 2
h g f 5. return A
Idea: add only “safe” edges 1 2 a 11 i 4 14 e
7 6
8 10
– An edge (u, v) is safe for A if and h
1
g 2
f

only if A  {(u, v)} is also a subset • How do we find safe edges?

of some MST
7 8

7 8

Finding Safe Edges Definitions

8 7 V-S 8 7
Let’s look at edge (h, g)
b c d b c d
• 4 9 4 9
2 2
– Is it safe for A initially? a 11 i 4 14 e • A cut (S, V - S) S a 11 i 4 14 e S
7 6 7 6
8 10 is a partition of vertices V- S  8 10
 V- S
• Later on: S h
1
g 2
f h
1
g 2
f
into disjoint sets S and V - S
– Let S  V be any set of vertices that includes h but not
g (so that g is in V - S)
• An edge crosses the cut

– In any MST, there has to be one edge (at least) that (S, V - S) if one endpoint is in S
connects S with V - S and the other in V – S
– Why not choose the edge with minimum weight (h,g)?

9 10

9 10

Definitions (cont’d) Theorem

• A cut respects a set A b
8
c
7
d • Let A be a subset of some MST (i.e., T), (S, V - S) be a
4
2
9 cut that respects A, and (u, v) be a light edge crossing
of edges  no edge S a 11 i 4 14 e S (S, V-S). Then (u, v) is safe for A .
in A crosses the cut
7 6
V- S  8 10 Proof:
 V- S
h
1
g 2
f • Let T be an MST that includes A
• An edge is a light edge – edges in A are shaded
S

crossing a cut  its weight is minimum over all • Case1: If T includes (u,v), then
edges crossing the cut it would be safe for A
u
• Case2: Suppose T does not include
– Note that for a given cut, there can be > 1 light
the edge (u, v)
edges crossing it • Idea: construct another MST T’ v
V-S
that includes A  {(u, v)}
11 12

11 12

2
 Theorem - Proof Theorem – Proof (cont.)
• T contains a unique path p between u and v T’ = T - {(x, y)}  {(u, v)}
• Path p must cross the S
Have to show that T’ is an MST: S

x
cut (S, V - S) at least x • (u, v) is a light edge
once: let (x, y) be that edge  w(u, v) ≤ w(x, y) u y
u y
p
• Let’s remove (x,y)  breaks p • w(T ’) = w(T) - w(x, y) + w(u, v)
v
T into two components. v ≤ w(T) V-S
V-S
• Adding (u, v) reconnects the components • Since T is a spanning tree
T’ = T - {(x, y)}  {(u, v)} w(T) ≤ w(T ’)  T’ must be an MST as well
13 14

13 14

Theorem – Proof (cont.) Prim’s Algorithm

Need to show that (u, v) is safe for A: • The edges in set A always form a single tree
• Starts from an arbitrary “root”: VA = {a}
i.e., (u, v) can be a part of an MST
• At each step:
• A  T and (x, y)  T 
8 7
b c d
S – Find a light edge crossing (VA, V - VA) 4 9
2
(x, y)  A  A T’ x – Add this edge to A a 11 i 4 14 e
7 6
– Repeat until the tree spans all vertices 8 10
• A  {(u, v)}  T’ u y h
1
g 2
f

p
• Since T’ is an MST
v
 (u, v) is safe for A V-S

15 16

15 16

How to Find Light Edges Quickly?

How to Find Light Edges Quickly? (cont.)
Use a priority queue Q: 8 7
• After adding a new node to VA we update the weights of all
b c d
• Contains vertices not yet 4 9 the nodes adjacent to it
2
e.g., after adding a to the tree, k[b]=4 and k[h]=8
included in the tree, i.e., (V – VA) a 11 i 4 14 e
– VA = {a}, Q = {b, c, d, e, f, g, h, i} 8
7 6
10 • Key of v is  if v is not adjacent to any vertices in VA
h g f
• We associate a key with each vertex v: 1 2
8 7
key[v] = minimum weight of any edge (u, v) b c d
4 9
connecting v to VA 2
a 11 i 4 14 e
7 6
8 10
Key[a]=min(w1,w2) w1
h g 2
f
w2 1
a

17 18

17 18

3
 PRIM(V, E, w, r)
1. Q← 
2. for each u  V
3. do key[u] ← ∞
4. π[u] ← NIL
5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0

Prim’s Algorithm
7. while Q  
8. do u ← EXTRACT-MIN(Q)
9. for each v  Adj[u]
10. do if v  Q and w(u, v) < key[v]
11. then π[v] ← u
12. DECREASE-KEY(Q, v, w(u, v))

19 20

19 20

Example
  
b
8
c
7
d 0  
4 9
 2  Q = {a, b, c, d, e, f, g, h, i}
a 11 i 4 14 e VA = 
7 6
8 10 Extract-MIN(Q)  a
h g 2
f
1
  

4
  
 [b] = a
Example/Dry Run 4
b
8

 2
c
7
d
9

key [b] = 4
key [h] = 8  [h] = a
a 11 i 4 14 e
8
7 6
10 4 8
h g 2
f Q = {b, c, d, e, f, g, h, i} VA = {a}
1
8  
Extract-MIN(Q)  b
21 22

21 22

Example Example
4 
8  key [c] = 8  [c] = b 4 8 7 key [h] = 7  [h] = i
8 7 8
b c d key [h] = 8  [h] = a - unchanged b c
7
d key [g] = 6  [g] = i
4 9 9
 2  4
8 8 2 2  7 46 7
a 11 i 4 14 e 11
a i 4 14 e
7 6 Q = {c, d, e, f, g, h, i} VA = {a, b} 6
Q = {d, e, f, g, h} VA = {a, b, c, i}
8 10 7
Extract-MIN(Q)  f
8
h g f Extract-MIN(Q)  c h g f
10
1 2
8   1 2
key [d] = 7  [d] = c 87 
6 4
4 8 
7 key [g] = 2  [g] = f
8 7 key [f] = 4  [f] = c 4 8 7

4
b c d
9 key [i] = 2  [i] = c b
8
c
7
d key [d] = 7  [d] = c unchanged
2 2
  9
11 14
4
2 2 
10 key [e] = 10  [e] = f
a i 4 e
a 11 i 14 e
8
7 6
7 4 8 2
4 7 10 2 7
10 7 6
8 10
h g 2
f Q = {d, e, f, g, h, i} VA = {a, b, c} Q = {d, e, g, h} VA = {a, b, c, i, f}
1 h g f
 
8 4
Extract-MIN(Q)  i
1
6
2
4
Extract-MIN(Q)  g
23
7 2 24

23 24

4
 Example Example
4 8 7 key [h] = 1  [h] = g
8 7
b c d 7 10 1
4 9
2 2 10 Q = {d, e, h} VA = {a, b, c, i, f, g} 4 8 7
a 11 i 14 e
4
Extract-MIN(Q)  h 8 7
8
7 6
10
b c d
9 9
key [e] = 9  [e] = f
4
2 2 10
h
1
g 2
f 9
a 11 i 4 14 e
7 2 4 7 10
1
7 6 Q = {e} VA = {a, b, c, i, f, g, h, d}
4 8 7 8 10
8 7
Q = {d, e} VA = {a, b, c, i, f, g, h} h g f Extract-MIN(Q)  e
b c d 1 2
4 9 Extract-MIN(Q)  d 1 2 4 Q =  VA = {a, b, c, i, f, g, h, d, e}
2 2 10
a 11 i 4 14 e
7 6
8 10
h g 2
f
1
1 2 4
25 26

25 26

PRIM(V, E, w, r)
1. Q← 
Total time: O(VlgV + ElgV) = O(ElgV)
2. for each u  V
3. do key[u] ← ∞ O(V) if Q is implemented
as a min-heap
4. π[u] ← NIL
5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV)

7. while Q   Min-heap

Analysis 8. do u ← EXTRACT-MIN(Q)
Executed |V| times
Takes O(lgV)
operations:
O(VlgV)
9. for each v  Adj[u] Executed O(E) times total
10. do if v  Q and w(u, v) < key[v] Constant O(ElgV)

11. then π[v] ← u Takes O(lgV)

12. DECREASE-KEY(Q, v, w(u, v))

27 28

27 28

Using Fibonacci Heaps Prim’s Algorithm

• Prim’s algorithm is a “greedy” algorithm
• Depending on the heap implementation, running time
could be improved! – Greedy algorithms find solutions based on a sequence
of choices which are “locally” optimal at each step.

• Nevertheless, Prim’s greedy strategy produces a

globally optimum solution!
– See proof for generic approach (i.e., slides 12-15)

29 30

29 30

5
 A different instance of the
generic approach
S
(instance 1)

(instance 2)
u

tree1
v

Kruskal’s Algorithm • A is a forest containing connected

V-S

u
components
– Initially, each component is a single
vertex v
tree2
• Any safe edge merges two of
these components into one
31 – Each component is a tree 32

31 32

Kruskal’s Algorithm Kruskal’s Algorithm

• How is it different from Prim’s algorithm? • Start with each vertex being its b
8
c
7
d
9
– Prim’s algorithm grows one own component
4
2
a 11 i 14 e
tree all the time • Repeatedly merge two 4
7 6
– Kruskal’s algorithm grows tree1 components into one by
8 10
h g 2
f
multiple trees (i.e., a forest) 1
choosing the light edge that
connects them We would add
at the same time. edge (c, f)
u
– Trees are merged together • Which components to consider
using safe edges at each iteration?
v
– Since an MST has exactly |V| - 1 tree2
– Scan the set of edges in
monotonically increasing order by
edges, after |V| - 1 merges,
weight
we would have only one component
33 34

33 34

KRUSKAL(V, E, w) Example
1. A←  1. Add (h, g) {g, h}, {a}, {b}, {c}, {d}, {e}, {f}, {i}
for each vertex v  V
8 7
2. b c d
9
2. Add (c, i) {g, h}, {c, i}, {a}, {b}, {d}, {e}, {f}
4
2 3. Add (g, f) {g, h, f}, {c, i}, {a}, {b}, {d}, {e}
3. do MAKE-SET(v) a 11 14 e 4.
i 4 Add (a, b) {g, h, f}, {c, i}, {a, b}, {d}, {e}
4. sort E into non-decreasing order by w 8
7 6
10 5. Add (c, f) {g, h, f, c, i}, {a, b}, {d}, {e}

5. for each (u, v) taken from the sorted list h

1
g 2
f 6. Ignore (i, g) {g, h, f, c, i}, {a, b}, {d}, {e}
7. Add (c, d) {g, h, f, c, i, d}, {a, b}, {e}
6. do if FIND-SET(u)  FIND-SET(v) 1: (h, g) 8: (a, h), (b, c) 8. Ignore (i, h) {g, h, f, c, i, d}, {a, b}, {e}
7. then A ← A  {(u, v)} 2: (c, i), (g, f) 9: (d, e) 9. Add (a, h) {g, h, f, c, i, d, a, b}, {e}
4: (a, b), (c, f) 10: (e, f)
UNION(u, v)
10. Ignore (b, c) {g, h, f, c, i, d, a, b}, {e}
8. 6: (i, g) 11: (b, h) 11. Add (d, e) {g, h, f, c, i, d, a, b, e}
9. return A 7: (c, d), (i, h) 14: (d, f) 12. Ignore (e, f) {g, h, f, c, i, d, a, b, e}
{a}, {b}, {c}, {d}, {e}, {f}, {g}, {h}, {i} 13. Ignore (b, h) {g, h, f, c, i, d, a, b, e}
14. Ignore (d, f) {g, h, f, c, i, d, a, b, e}
35 36

35 36

6
 Implementation of Kruskal’s Algorithm Operations on Disjoint Data Sets
8 • MAKE-SET(u) – creates a new set whose only
• Uses a disjoint-set data
7
b c d
4
2
9 member is u
structure (see Chapter a 11 i 4 14 e • FIND-SET(u) – returns a representative element
7 6
21) to determine whether 8 10
from the set that contains u
h g 2
f
1
an edge connects – Any of the elements of the set that has a particular
We would add
vertices in different edge (c, f) property
components – E.g.: Su = {r, s, t, u}, the property is that the element be
the first one alphabetically
FIND-SET(u) = r FIND-SET(s) = r
– FIND-SET has to return the same value for a given set
37 38

37 38

Operations on Disjoint Data Sets KRUSKAL(V, E, w)

1. A← 
• UNION(u, v) – unites the dynamic sets that
2. for each vertex v  V
contain u and v, say Su and Sv O(V)
3. do MAKE-SET(v)
– E.g.: Su = {r, s, t, u}, Sv = {v, x, y} 4. sort E into non-decreasing order by w O(ElgE)
UNION (u, v) = {r, s, t, u, v, x, y} 5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
• Running time for FIND-SET and UNION
7. then A ← A  {(u, v)} O(lgV)
depends on implementation. 8. UNION(u, v)
• Can be shown to be α(n)=O(lgn) where α() is a 9. return A
Running time: O(V+ElgE+ElgV)=O(ElgE) – dependent on
very slowly growing function (see Chapter 21) the implementation of the disjoint-set data structure
39 40

39 40

KRUSKAL(V, E, w) (cont.) Kruskal’s Algorithm

1. A← 
• Kruskal’s algorithm is a “greedy” algorithm
2. for each vertex v  V
O(V)
3. do MAKE-SET(v) • Kruskal’s greedy strategy produces a globally
4. sort E into non-decreasing order by w O(ElgE) optimum solution
5. for each (u, v) taken from the sorted list O(E) • Proof for generic approach
S

6. do if FIND-SET(u)  FIND-SET(v) x
applies to Kruskal’s
7. then A ← A  {(u, v)} O(lgV) u y
8. UNION(u, v) algorithm too
9. return A
v
- Running time: O(V+ElgE+ElgV)=O(ElgE) O(ElgV) V-S
- Since E=O(V2), we have lgE=O(2lgV)=O(lgV) 41 42

41 42

7
 Problem 1 Problem 1 (cont.)
• (Exercise 23.2-3, page 573) Compare Prim’s algorithm
with and Kruskal’s algorithm assuming:
(b) dense graphs
In this case, E=O(V2)
(a) sparse graphs:
In this case, E=O(V)
Kruskal:
Kruskal:

O(ElgE)=O(V2lgV2)=O(2V2lgV)=O(V2lgV)
O(ElgE)=O(VlgV)
Prim:
Prim: - binary heap: O(ElgV)=O(V2lgV)
- binary heap: O(ElgV)=O(VlgV) - Fibonacci heap: O(VlgV+E)=O(VlgV+V2)=O(V2)
- Fibonacci heap: O(VlgV+E)=O(VlgV)
43 44

43 44

Problem 2 Problem 2 (cont.)

1. A← 
2. for each vertex v  V
(Exercise 23.2-4, page 574): Analyze the O(V)
running time of Kruskal’s algorithm when 3. do MAKE-SET(v)
weights are in the range [1 … V] 4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)} O(lgV)
8. UNION(u, v)
9. return A
- Sorting can be done in O(E) time (e.g., using counting sort)
45 - However, overall running time will not change, i.e, O(ElgV)46

45 46

Problem 3 Problem 4
• Suppose that some of the weights in a connected • (Exercise 23.2-2, page 573) Analyze Prim’s
graph G are negative. Will Prim’s algorithm still algorithm assuming:
work? What about Kruskal’s algorithm? Justify
your answers. (a) an adjacency-list representation of G
– Yes, both algorithms will work with negative weights.
Review the proof of the generic approach; there is no O(ElgV)
assumption in the proof about the weights being
positive.
(b) an adjacency-matrix representation of G
O(ElgV+V2)

47 48

47 48

8
 PRIM(V, E, w, r) PRIM(V, E, w, r)
1. Q←  1. Q← 
Total time: O(VlgV + ElgV) = O(ElgV) Total time: O(VlgV + ElgV+V2) = O(ElgV+V2)
2. for each u  V 2. for each u  V
3. do key[u] ← ∞ O(V) if Q is implemented 3. do key[u] ← ∞ O(V) if Q is implemented
as a min-heap as a min-heap
4. π[u] ← NIL 4. π[u] ← NIL
5. INSERT(Q, u) 5. INSERT(Q, u)
6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV) 6. DECREASE-KEY(Q, r, 0) ► key[r] ← 0 O(lgV)

7. while Q   Executed |V| times

Min-heap 7. while Q   Executed |V| times
Min-heap
operations: operations:
8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV) 8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV)
9. for each v  Adj[u] Executed O(E) times 9. for (j=0; j<|V|; j++) Executed O(V2) times total
10. do if v  Q and w(u, v) < key[v] Constant O(ElgV) 10. if (A[u][j]=1) Constant
11. then π[v] ← u Takes O(lgV) 11. if v  Q and w(u, v) < key[v]
12. DECREASE-KEY(Q, v, w(u, v)) 12. then π[v] ← u Takes O(lgV) O(ElgV)
49
13. DECREASE-KEY(Q, v, w(u, v)) 50

49 50

Problem 5 Problem 6
• Find an algorithm for the “maximum” spanning • (Exercise 23.1-8, page 567) Let T be a MST of
tree. That is, given an undirected weighted a graph G, and let L be the sorted list of the
graph G, find a spanning tree of G of maximum edge weights of T. Show that for any other MST
cost. Prove the correctness of your algorithm. T’ of G, the list L is also the sorted list of the
– Consider choosing the “heaviest” edge (i.e., the edge edge weights of T’
associated with the largest weight) in a cut. The
generic proof can be modified easily to show that this
approach will work.
– Alternatively, multiply the weights by -1 and apply
either Prim’s or Kruskal’s algorithms without any
modification at all! T, L={1,2} T’, L={1,2}

51 52

51 52