Kruskal's Algorithm and Prim's Algorithm

Tahir Iqbal
Bahria University, Lahore Campus
Decision 1 Overview

1:: Algorithms 2:: Graphs and networks

Sorting and bin What is a graph and how they 3:: Algorithms on graphs
packing. represent things. What algorithms do I need to
be able to apply?
4:: Route inspection
5:: The Travelling Salesman
Find the shortest route 6:: Linear Programming
Find the shortest route
which travels along all How to find an optimal
which visits all places.
roads solution graphically

7:: The simplex algorithm

8:: Critical path analysis
How to find an optimal
solution algebraically. How to plan a project.
Minimum Spanning Trees
A minimum spanning tree is a spanning tree such that
the total length of its arcs (edges) is as small as possible.

A spanning tree is a subgraph, which includes all the

vertices and is a tree A minimum
spanning tree is
A subgraph of G is a graph , each of whose vertices
belongs to G and each of whose edges belongs to G. sometimes called
a minimum
A tree is a connected graph with no cycles.
connector.
A graph is connected if all its vertices are connected.

Two vertices are connected if there is a path between

them.
A path is a walk in which no vertex is visited more than
once.
A walk is a route through a graph along edges from one vertex to the
next. next.
3.1 Kruskal’s Algorithm
Kruskal’s Algorithm can be used to find a minimum spanning tree.
1. Sort all the arcs into ascending order of weight.
2. Select the arc of least weight to start the tree.
3. Consider the next arc of least weight.
• If it would form a cycle with the arcs already selected reject it.
• If it does not form a cycle, add it to the tree.
• If there is a choice of equal arcs then consider each in turn.
4. Repeat step 3 until all vertices are connected.
Uses of minimum spanning trees
• Cluster Analysis.
• Real-time face tracking and verification (i.e. locating Kruskal’s Algorithm is
human faces in a video stream). sometimes called the “greedy
• Protocols in computer science to avoid network cycles. algorithm” because it gobbles
• Entropy based image registration. up the best (least weight) arcs
• Max bottleneck paths. first. It considers the arcs, not
• Dithering (adding white noise to a digital recording in the vertices.
order to reduce distortion).
[Link]
Example of using Kruskal’s Algorithm

Use Kruskal’s algorithm to find the MST for the below network.
D

4 8 Kruskal’s Algorithm can be used to

6
find a minimum spanning tree.
E C
6 1. Sort all the arcs into
5 ascending order of weight.
5
7 B 2. Select the arc of least weight
A Next to start the tree.
3. Consider the next arc of
least weight.
• If it would form a cycle with
Order of the arcs is. the arcs already selected
DE(4), AE(5), BC(5), AD(6), BD(6), AB(7), CD(8), reject it.
• If it does not form a cycle,
add it to the tree.
Start with DE • If there is a choice of equal
D
D arcs then consider each in
All vertices are connected turn.
4 so this is a minimum 4. Repeat step 3 until all
E 6 C spanning tree. vertices are connected.
E C
6
5
5 Its weight is 20.
B
A B
A
Test your understanding
Use Kruskal’s algorithm to find the MST for the below network.
A
6 6 Order of the arcs is.
B D
1 6 BC(1), IJ(1), GI(2),
2 2 18 CG(2), BE(2), CD(2), Next
6 C KL(3), EF(4), AB(6),
E 2 J
AD(6), AC(6), EC(6),
4 11 G 1
8 JL(8), FH(10),FG(11),
F 2
IH(12), IK(16), DJ(18),
22 L
10 I GH (22), HK(25) AA
12 6 6
16 3 BB 6 DD
H
25
Start with BC 1
2 2
K 6
CC
EE 2 JJ
All vertices are connected so this is a minimum 4 1
GG 8
spanning tree. 2
FF
Its weight is 41. LL
10 II

Note – there are several different MST’s for this 3

HH
question, depending which order you take the equal
KK
weight edges in.
Test your understanding – past exam question

1
Exercise 3A
Pearson Decision 1, Page 56
3.2 Prim’s Algorithm
1) Choose any vertex to start the tree.
2) Select an arc of least weight that joins a vertex that is already in the tree to a vertex
that is not yet in the tree.
If there is a choice of arcs of equal weight, choose randomly.
3) Repeat step 2 until all the vertices are connected.

What is the main difference between Prim’s and Kruskal’s algorithm?

Prim’s considers vertices, whereas Kruskal’s considers edges.

Why might you use one rather than the other?

To use Kruskal you have to sort all the edges into order first, this could
be time consuming so Prim’s may be faster unless the edges are
already sorted. Prim’s is usually faster if you have a graph with high
ratio of edges to vertices.
Prim’s Algorithm - example
D 1) Choose any vertex to start
the tree.
4 8 2) Select an arc of least weight
that joins a vertex that is
E 6 C already in the tree to a vertex
6
5 that is not yet in the tree.
5 If there is a choice of arcs of
7 B equal weight, choose
A randomly.
3) Repeat step 2 until all the
Start anywhere – we will start vertices are connected.
at vertex A.
Add AE (5)
Next
Add ED (4) D
D
Add DB (6) 4
Add BC (5) EE 6 CC
5
All vertices are now connected 5
BB
so the minimum spanning tree AA

is weight 20.
Test your understanding
Use Prim’s algorithm to find the MST for the below network.
A
6 6 Starting at vertex A
B D
1 6 Choose AB (6)
2 2 18 Add BC (1) Next
6 C Choose BE(2)
E 2 J
Choose CD (2)
4 11 G 1
8 Add CG (2)
F 2
Add GI (2)
22 L
10 I Add IJ (1) A A
12 Add EF(4) 6
16 B B
3 D D
H
25
Add JL (8) 1
2 2
K Add LK (3)
C C
Add FH (10) E E 2 J J
4 1
G G 8
All vertices are connected so this is a minimum F F 2
spanning tree. I L L
10 I
Its weight is 41.
3
H H
K K
Test your understanding – past exam question

1
Exercise 3B
Pearson Decision 1, Page 59
3.3 Applying Prim’s algorithm to a distance matrix
As we have seen, we can represent graphs using a distance matrix.
A B C D 14 D
1) Choose any vertex to start the B
A - 8 10 - 7
tree. 8 23
2) Delete the row in the matrix for B 8 - 23 14
C
A
the chosen vertex. C 10 23 - 7 10
3) Number the column in the D - 14 7 -
matrix for the chosen vertex
4) Put a ring round the lowest 1) Choose any vertex to start
undeleted entry in the the tree.

between the methods

Look for similarities
2) Select an arc of least weight
numbered columns (If there is
that joins a vertex that is
an equal choice, choose already in the tree to a vertex
randomly) that is not yet in the tree.
5) The ringed entry becomes the If there is a choice of arcs of
next arc to be added to the equal weight, choose
tree. randomly.
6) Repeat 2,3,4 and 5 until all rows 3) Repeat step 2 until all the
are deleted. vertices are connected.

Prim’s for a matrix Prim’s for a graph

3.3 Applying Prim’s algorithm to a matrix - eg
1 2 3 4
1) Choose any vertex to
A B C D Next start the tree.
2) Delete the row in the
A - 8 10 - matrix for the chosen
vertex.
B 8 - 23 14 D
B 3) Number the column
C 10 23 - 7 7 in the matrix for the
8 chosen vertex
D - 14 7 - 10 C 4) Put a ring round the
A lowest undeleted
entry in the
Start at A. numbered columns
(If there is an equal
Cross through Row A and number Column A
choice, choose
The 1st arc is AB, put a ring around it. randomly)
Delete Row B and number column B 5) The ringed entry
The 2nd arc is AC, put a ring around it becomes the next arc
Delete Row C and number column C to be added to the
The 3rd arc is DC, put a ring around it. tree.
Delete Row D and number column D 6) Repeat 2,3,4 and 5
Finish because all rows are deleted. until all rows are
deleted.
Test your understanding – past exam question

1
Number of comparisons with Prim’s Algorithm
1 5 3 2 4
1) Select the first vertex, then you have to select the
A B C D E smallest from the 4 remaining values in column A.
A - 12 11 10 23 • Compare B with C and select the smallest
• Compare the smallest of {B,C} with D
B 12 - 17 9 21 • Compare the smallest of {B,C,D} with E
4 − 1 = 3 comparisons.
C 11 17 - 8 7
D 10 9 8 - 18 2) Now we select vertex D, and, we have to compare
the 3 remaining items in column D and the 3
E 23 21 7 18 -
remaining items in A so we have a further 6 − 1 = 5
comparisons.
Total of 3 + 5 + 5 + 3 = 16
comparisons. 3) Now select vertex C, and we have 3 columns all
with 2 items remaining, that’s 6 − 1 = 5
comparisons.

4) Now select vertex E, and we have 4 columns all

with 1 item remaining, that’s 4 − 1 = 3 comparisons.

5) Now select vertex B.

Order of Prim’s Algorithm

Calculate how many comparisons would be required for an 𝑛 × 𝑛

distance matrix, hence state the order of Prim’s algorithm.
Select the first vertex, then you have to select the
1 2 … n smallest from the 𝑛 − 1 remaining values in the first
column. 𝑛 − 1 − 1 comparisons
1 -
At each stage the number of columns increases by
2 - one and the number of values to consider in those
columns decreases by one.
⋮ ⋱ 2nd stage has 𝑛−2 ×2 −1
3rd stage has 𝑛−3 ×3 −1
n - (𝑛 − 1)th stage has ((𝑛 − 𝑛 − 1 ) × 𝑛 − 1 − 1)
𝑛th stage has no comparisons
𝑛−1 𝑛−1 𝑛−1 𝑛−1
The next bit needs
𝑇𝑜𝑡𝑎𝑙 = ෍ 𝑛 − 𝑟 × 𝑟 − 1 = 𝑛 ෍ 𝑟 − ෍ 𝑟2 − ෍ 1 Core Pure 1, Chapter 3
𝑟=1 𝑟=1 𝑟=1 𝑟=1

1 1 3𝑛2 𝑛 − 1 − 𝑛 𝑛 − 1 2𝑛 − 1 − 6 𝑛 − 1
=𝑛 𝑛 − 1 𝑛 − 𝑛 − 1 𝑛 2𝑛 − 1 − 𝑛 − 1 =
2 6 6
𝑛3 −7𝑛+6
= , therefore the order of Prim’s algorithm is 𝑛3
6
Exercise 3C
Pearson Decision 1, Page 63

Answer templates…
[Link]
3.4 using Dijkstra’s algorithm to find the shortest path
Dijkstra’s can be used to find the shortest path through a network.
1) Label the start vertex, S with the final label, 0.
2) Record a working value at every vertex, Y, which is directly connected to the vertex, X,
which has just received its final label.
- Working value at Y = final value at X + weight of arc XY
- If there is already a working value at Y, it is only replaced if the new value is smaller.
- Once a vertex has a final label it is not revisited and its working values are no longer
considered.
3) Look at the working values at all vertices without final labels. Select the smallest working
value. This now becomes the final label at that vertex. (If two vertices have the same
smallest working value either may be given its final label first.)
4) Repeat 2-3 until the destination vertex T, receives its final label.
5) To find the shortest path, trace back from T to S. Given that B already lies on the route,
include arc AB whenever final label of B – final label of A = weight of arc AB.

Uses of Dijkstra’s Algorithm

• Finding the shortest/quickest driving route to travel from A to B.*
• Internet Protocol Routing such as “Open Shortest Path First”.
• Telecommunication networks to find the least cost path to route communications.
• Modelling the spread of viruses to determine how fast spread will occur.
* the road application requires some modification as blindly applying Dijkstra would require you to consider all possible routes from say
Newton Aycliffe to Darlington, including going via Mosco!
Dijkstra’s Algorithm - notation
To make your working clear you
always replace the vertices with
boxes like this: Vertex
Order of
labelling
Final label

Working values
You will always be provided with
answer templates in the exam.
B 2 B 2 E
E
Same

4 3 3 4 3 14
3 14

C S C T
S 8 8 12
12 T

12 12 3 9 3
3 9 3
D F
5
Same 5
D F
Applying Dijkstra’s Algorithm – eg.
Order of
Vertex labelling
Final label 1) Label the start vertex, S with the final label, 0.
2) Record a working value at every vertex, Y,
Working values which is directly connected to the vertex, X,
which has just received its final label.
- Working value at Y = final value at X + weight of arc
XY
- If there is already a working value at Y, it is only
replaced if the new value is smaller.
BB 2 3 22 EE 3 5 - Once a vertex has a final label it is not revisited and
5 its working values are no longer considered.
3 3) Look at the working values at all vertices without
44 33
final labels. Select the smallest working value. This
33 1414 now becomes the final label at that vertex. (If two
vertices have the same smallest working value either
may be given its final label first.)
SS 1 0 88 CC 4 7 1212 TT 7 18 4) Repeat 2-3 until the destination vertex T, receives
8 7 8 19 19 18 its final label.
5) To find the shortest path, trace back from T to S.
Given that B already lies on the route, include arc AB
1212 33 99 33 whenever final label of B – final label of A = weight of
arc AB.
DD 5 10 FF 6 15
55
16 15 Working backwards… You must include this step!
12 10
T-F-D-C-B-S
Reversing
S-B-C-D-F-T
Test your understanding – past exam question

Answer template
on next slide.

1
Test your understanding – past exam question

1
Test your understanding – past exam question
b) Find a route for Avinash to
travel from S to T in the shortest
time. State, with a reason,
whether this route is a unique
solution.

On a particular day Avinash must

include C in his route.

c) Find a route of minimal time

1
from S to T that includes C, and
state its time.
Exercise 3D
Pearson Decision 1, Page 71

Answer templates…
[Link]

Note – in the contents of some versions of the Pearson text book it erroneously states that
section 3.5 - Floyd’s algorithm is required for AS Level. It isn’t.
3.5 Floyd’s Algorithm (A2 content only)
We have used Dijkstra’s algorithm to find the shortest path between 2 nodes in a
network. Using Floyd’s algorithm we can find the shortest path between any pair
of vertices in the network.
1. Complete an initial distance table for the network. If there is no direct route between
2 vertices label the distance as infinity (∞)
2. Complete an initial route table by making every entry the same as the label at the top
of the column
3. In the first iteration, copy the first row and the first column values of the distance
table into a new table. Shade these values
4. Consider each unshaded position in turn. Compare the value in this position in the
previous table with the sum of the corresponding shaded values.
• If 𝑋 + 𝑌 ≥ 𝑍 then copy 𝑍 into the new table (i.e. there is no change – you keep
the smallest value)
• If 𝑋 + 𝑌 < 𝑍 then copy 𝑋 + 𝑌 into the new table and write A in the
corresponding position in the route table. Once all areas of the unshaded region
have been considered the first iteration is complete.
5. For the second iteration copy the second row and second column values of the
distance table into a new table. Shade these values
6. Repeat step 4 with the new unshaded values. This time any changes write B in the
new route table
7. Continue until you have complete an iteration for all vertices (n iterations for n
vertices)
 Floyds Algorithm - Example
[Textbook] The distance graph shows the direct distances, by B
4
road, between four towns A, B, C and D, in miles. The road from A
D to A is a one way road as shown by the arrow. 7
a) Use Floyd’s algorithm to produce a table of shortest distances. C
You should give the distance table and route table for each 1 9
iteration.
a) Initial tables
A B C D A B C D D
A - 4 7 ∞ A A B C D
B 4 - ∞ 9 B A B C D
C 7 ∞ - ∞ C A B C D Compare BC with the
sum of corresponding
D 1 9 ∞ - D A B C D values in the first row
First Iteration and column. The sum
is less than the
A B C D A B C D existing value so
A replace it.
- 4 7 ∞ A A B C D
As the value has
B 4 - ∞
11 9 B A B CA D changed, replace C
C with A in the route
7 ∞
11 - ∞ C A BA C D table. Continue this for
D 1 95 ∞
8 - D A BA A
C D all unshaded values
Floyds Algorithm - Example
Second Iteration For the second
iteration shade the 2nd
A B C D A B C D row and column.
A - 4 7 ∞
13 A A B C D
B Compare values like
before. AD has a value
B 4 - 11 9 B A B A D more than the sum of
C 7 11 - ∞
20 C A A C D
B the corresponding
shaded cells so it is
D 1 5 8 - D A A A D replaced.
As the value has been
Third Iteration replaced, replace D in
the route table with B.
A B C D A B C D Continue for the rest
A - 4 7 13 A A B C D
B of the values.

B 4 - 11 9 B A B A D
C 7 11 - 20 C A A C D
B For the third iteration
shade the 3rd row and
D 1 5 8 - D A A A D column. Compare
values like before. This
time there are no
changes
Floyds Algorithm - Example
Fourth Iteration
A B C D A B C D
For the four iteration
A - 4 7 13 A A B C D
B shade the 4th row and
column. Compare
B 4 - 11 9 B A B A D values like before. This
C 7 11 - 20 C A A C D
B time there are no
changes.
D 1 5 8 - D A A A D
After changes in this
Final Tables iteration it gives us our
final tables
A B C D A B C D
A - 4 7 13 A A B C D
B
B 4 - 11 9 B A B A D
C 7 11 - 20 C A A C D
B
D 1 5 8 - D A A A D
Floyds Algorithm - Example
b) Find the route of minimum length from C to D.

A B C D A B C D
A - 4 7 13 A A B C D
B
B 4 - 11 9 B A B A D
C 7 11 - 20 C A A C D
B
D 1 5 8 - D A A A D

To find the route from C to D look at row C and column D

This gives us B meaning we have to go through B

Now look at row C and column B

This gives us A meaning we have to go through A

Now look at row C and column A

This gives us A meaning they are directly connected

So the quickest route from C to D is CABD. Look at the value in the distance table for
the length of the route. In this case 20 miles.
Floyds Algorithm - Example
[Textbook] 8 departure gates in an airport, linked by travellators and
escalators, are modelled using a network.
7 iterations of Floyd’s algorithm are applied to the network, resulting in
the following distance and route tables.
a) Apply the final iteration of Floyd’s algorithm to give the final
distance and route tables.
Distance Table Route Table
A B C D E F G H A B C D E F G H
A - 2 5 7 3 11 9 4 A A B B C E G C E
B 6 - 3 5 9 9 7 10 B D B C C D G C E
C 3 5 - 2 6 6 4 7 C D D C D D G G E
D 1 3 6 - 4 12 10 5 D A A B D A G C E
E ∞ ∞ ∞ ∞ - ∞ ∞ 1 E A B C D E F G H
F 7 1 4 6 2 - 8 3 F D B B C E F C E
G 9 3 6 8 4 2 - 5 G F F F F F F G F
H 2 4 7 1 5 7 5 - H D D D D D G G H
 Distance Table Route Table
A B C D E F G H A B C D E F G H
A - 2 5 [5]
7 3 11 9 4 A A B C
B [H] E G C E
B 6 - 3 5 9 9 7 10 B D B C C D G C E
C 3 5 - 2 6 6 4 7 C D D C D D G G E
D 1 3 6
- 4 12 10 5 D A A B D A G C E
∞ [5]
E [3] ∞ [2]
∞ [8] ∞ [6]
∞ - [8] ∞ 1 E [H]
A [H]
B [H] D
C [H] E [H]
F [H]
A H
F [5]
7 1 4 [4]6 2 - 8 3 F [H]
D B B [H]
C E F C E
G [7]
9 3 6 [6]8 4 2 - 5 G [H]
F F F [H]
F F F G F
H 2 4 7 1 5 7 5 - H D D D D D G G H
Iteration complete
b) Floyd needs to get from gate D to gate F. State the minimum time
needed to make this journey and determine the route he should take.
Minimum time from D to F = 12 minutes
Check row D, column F, goes via G
Check row D, column G, goes via C
Check row D, column C, goes via B
Check row D, column B, goes via A
Check row D, column A, goes directly to A, so route is D-A-B-C-G-F
Test your understanding
Test your understanding
1 2
4

6 2
2
2

4
4 3