Professional Documents
Culture Documents
CSC 302
SK Hafizul Islam
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 1 / 40
Agenda I
3 Suggested Readings
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 2 / 40
Spanning Tree
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 3 / 40
Spanning Tree
Figure: Graph Figure: Spanning Figure: Spanning Figure: Spanning Figure: Spanning
tree tree tree tree
Figure: Spanning Figure: Spanning Figure: Spanning Figure: Spanning Figure: Spanning
tree tree tree tree tree
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 4 / 40
Spanning Tree
3 3 3 3
7
6
7
7
4 6 4
5
5
2
2 2 2 2
7
6 4 6
5
5
2 2
Figure: Spanning Figure: Spanning Figure: Spanning Figure: Spanning Figure: Spanning
tree tree tree tree tree
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 5 / 40
Spanning Tree
If a graph is a complete graph with n vertices, then total number of spanning trees is
n(n−2) .1
A complete graph is a graph where every pair of vertices is joined by an edge, thus the
number of edges in a complete graph with n vertices is n C2 .
For a cycle/circular graph with n vertices, then total number of spanning trees is n. A
cycle/circular graph is a graph that contains only one cycle.
1
https://en.wikipedia.org/wiki/Cayley%27s_formula
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 6 / 40
Minimum Spanning Tree: Kruskal’s algorithm
1 2
h g f
The greedy strategy manages a set of edges A, maintaining the following loop invariant:
Prior to each iteration, T is a subset of some minimum spanning tree.
At each step, we determine an edge (u, v ) that we can add to A without violating this
invariant, in the sense that T ∪ {(u, v )} is also a subset of a minimum spanning tree.
We call such an edge a safe edge for T , since we can add it safely to T while maintaining
the invariant.
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 7 / 40
Minimum Spanning Tree: Kruskal’s algorithm
Algorithm: MST-KRUSKAL(G, w )
1 T := ϕ
2 for each vertex v ∈ V do
3 MAKE-SET(v )
4 end
5 Sort the edges of G into nondecreasing order by weight w
6 for each edge (u, v ) ∈ E , taken in nondecreasing order by weight do
7 if (FIND-SET(u) ̸=FIND-SET(v )) then
8 T := T ∪ {(u, v )}
9 UNION(FIND-SET(u),FIND-SET(v ))
10 end
11 end
12 Return T
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 8 / 40
Minimum Spanning Tree: Kruskal’s algorithm
The running time of this algorithm depends on how we implement the disjoint-set data struc-
ture. We assume that we use the disjoint-set implementation with the union-by-rank and
path-compression heuristics.
The time to sort the edges in line 5 is O(|E |log|E |).
The for loop of lines 6-11 performs O(|E |) FIND-SET and UNION operations on the
disjoint-set forest. The for loop of lines 2-4 performs O(|V |) MAKE-SET operations.
Therefore, total time for these two for loops is O |V |+|E | α(|V |), where α is the very slowly
growing function. Because we assume that G is connected, we have |E | ≥ (|V | − 1), and
so the disjoint-set operations take (|E |α(|V |)) time. Moreover, since α(|V |) = log(|V |) =
log(|E |), the total running time of Kruskal’s algorithm is O(|E |log|E |).
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 9 / 40
Minimum Spanning Tree: Kruskal’s algorithm
2 3
Initialization: S1 = {a}, S2 = {b}, S2 = {c},
a b c S4 = {d}, and S5 = {e}.
Sort the edges of G into nondecreasing order by
6
8
7 weight w .
5
(a, b): w (a, b) = 2.
d e (b, c): w (b, c) = 3.
9 (b, e): w (b, e) = 5.
(a, d): w (a, d) = 6.
(c, e): w (c, e) = 7.
(b, d): w (b, d) = 8.
(d, e): w (d, e) = 9.
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 10 / 40
Minimum Spanning Tree: Kruskal’s algorithm
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 11 / 40
Minimum Spanning Tree: Kruskal’s algorithm
2 3
a b c
6
5
d e
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 12 / 40
Minimum Spanning Tree: Prim’s Algorithm
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 13 / 40
Minimum Spanning Tree: Prim’s Algorithm
Algorithm: MST-PRIM(G, w , r )
1 V is the set of vertices of G
2 E is the set of edges of G
3 for (each vertex u ∈ V ) do
4 key[u]:=∞ ▷ Initialize
5 par[u]:=NIL ▷ Initialize
6 color[u]:=white ▷ Initialize
7 end
8 key[r ]:=0 ▷ Start at root node
9 par[r ]:=NIL
10 Q := V ▷ Put all the vertices in a min-priority queue Q
11 while (Q is not empty) do
12 u :=EXTRACT-MIN(Q)
13 for (each v adjacent to u) do
14 if (color[v ]=white and w (u, v ) <key[v ]) then
15 par[v ]:=u
16 DECREASE-KEY(v ,key [v ])
17 key[v ]:= w (u, v ) ▷ new lightest edge
18 end
19 end
20 color[u]:=black
21 end
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 14 / 40
Minimum Spanning Tree: Prim’s Algorithm
If we implement Q as a binary min-heap, time complexity for lines 3-7 is O(|V |).
The body of the while loop executes O(|V |) times, and since each EXTRACT-MIN oper-
ation takes O(lg|V |) time, the total time for all calls to EXTRACT-MIN is O(|V |lg|V |).
The for loop in lines 11-16 executes O(|E |) times.
The assignment in line 14 involves an implicit DECREASE-KEY operation on a binary
min-heap that supports in O(lg|V |) time.
Thus total time for Prim’s algorithm is O(|V |lg|V | + |E |lg|V |) = O(|E |lg|V |).
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 15 / 40
Minimum Spanning Tree: Prim’s Algorithm
23.2 The algorithms of Kruskal and Prim 635
8 7 8 7
b c d b c d
4 9 4 9
2 2
(a) a 11 i 4 14 e (b) a 11 i 4 14 e
7 6 7 6
8 10 8 10
h g f h g f
1 2 1 2
8 7 8 7
b c d b c d
4 9 4 9
2 2
(c) a 11 i 4 14 e (d) a 11 i 4 14 e
7 6 7 6
8 10 8 10
h g f h g f
1 2 1 2
8 7 8 7
b c d b c d
4
SK Hafizul Islam (Department of CSE, IIIT Kalyani) 9 Algorithms - I 4 October 31, 2022 9 16 / 40
2 2
(c) a 11 i 4 14 e (d) a 11 i 4 14 e
Minimum Spanning
8
Tree: Prim’s
10
Algorithm
8
7 6 7 6
10
h g f h g f
1 2 1 2
8 7 8 7
b c d b c d
4 9 4 9
2 2
(e) a 11 i 4 14 e (f) a 11 i 4 14 e
7 6 7 6
8 10 8 10
h g f h g f
1 2 1 2
8 7 8 7
b c d b c d
4 9 4 9
2 2
(g) a 11 i 4 14 e (h) a 11 i 4 14 e
7 6 7 6
8 10 8 10
h g f h g f
1 2 1 2
8 7
b c d
4 9
2
(i) a 11 i 4 14 e
7 6
8 10
h g f
1 2
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 17 / 40
Shortest Path Problem
A shortest path from vertex u to vertex v is then defined as any path p with weight
w (p) = δ(u, v ).
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 18 / 40
Shortest Path Problem
Dijkstra algorithm
Solves single-source shortest path problem
Solves only the problems with nonnegative weights, i.e.,
Bellman-Ford algorithm
Solves single-source shortest path problem
Applicable to problems with arbitrary weights (both positive and negative)
Floyd-Warshall algorithm
Applicable to problems with arbitrary weights
Solves a more general all pair shortest path problem
Floyd-Warshall and Bellman-Ford algorithms solve the problems on acyclic graphs with
negative weight.
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 19 / 40
Shortest Path Problem: Negative-weight Cycle
646 Chapter 24 Single-Source Shortest Paths
a b
–4
3 –1
h i
3 4 2
∞ ∞
s c 6 d g
5 8
0 5 11 –∞ –8 3
–3
∞
2 e 3 f 7
j
–∞ –∞
–6
Figure 24.1 Negative edge weights in a directed graph. The shortest-path weight from source s
If the graph G contains no negative
appears withinweight cycles
each vertex. Because vertices ereachable from cycle
and f form a negative-weight thereachable
source from s,s, then for
they have shortest-path weights of 1. Because vertex g is reachable from a vertex whose shortest-
all v ∈ V , the shortest-path weight
path weight is 1, it, too, has a shortest-path weight of 1. Vertices such as h, i, and j are not a negative
δ(s, c) remains well defined, even if it has
reachable from s, and so their shortest-path weights are 1, even though they lie on a negative-weight
value. cycle.
If the graph G contains a negative weight cycle reachable from s, however, shortest-path
Some shortest-paths algorithms, such as Dijkstra’s algorithm, assume that all
weights are not well defined. edge weights in the input graph are nonnegative, as in the road-map example. Oth-
ers, such as the Bellman-Ford algorithm, allow negative-weight edges in the in-
Only one path from s to a: p = put ⟨s,
grapha⟩,and δ(s,
producea) = wanswer
a correct (s, a)as long
= 3. as no negative-weight cycles are
reachable from the source. Typically, if there is such a negative-weight cycle, the
Only one path from s to b: p = ⟨s, a,canb⟩,
algorithm detectδ(s, b) =
and report w (s, a) + w (a, b) = 3 + (−4) = −1.
its existence.
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Cycles Algorithms - I October 31, 2022 20 / 40
Shortest Path Problem: Negative-weight Cycle
646 Chapter 24 Single-Source Shortest Paths
a b
–4
3 –1
h i
3 4 2
∞ ∞
s c 6 d g
5 8
0 5 11 –∞ –8 3
–3
∞
2 e 3 f 7
j
–∞ –∞
–6
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 22 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
Maintain an estimate d[v ] of the length δ(s, v ) of the shortest path for each vertex v .
Always d[v ] ≥ δ(s, v ) and d[v ] equals the length of a known path (d[v ] = ∞ if we have
no paths so far).
Initially d[s] = 0 and all the other d[v ] values are set to ∞.
The algorithm will then process the vertices one by one in some order.
Here “processing a vertex u” means finding new paths and updating d[v ] for all the
vertices v adjacent to u if necessary. The process by which an estimate is updated is called
relaxation.
The processed vertex’s estimate will be validated as being real shortest distance, i.e., d[v ] =
δ(s, v ).
When all vertices have been processed, d[v ] = δ(s, v ) for all v ∈ V .
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 23 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
Finding new paths: When processing a vertex u, the algorithm will examine all vertices
v adjacent to u. For each vertex v adjacent to u, a new path from s to v is found (path
from s to u + new edge).
Relaxation: If the length of the new path from s to v is shorter than d[v ], then update
d[v ] to the length of this new path.
Whenever we set d[v ] to a finite value, there exists a path of that length. Therefore,
d[v ] ≥ δ(s, v ). .
If d[v ] = δ(s, v ), then further relaxations cannot change its value.
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 24 / 40
There is a (shortest) path from to with length .
Single-Source Shortest Path: Dijkstra’s
Algorithm
There is a path from to with length .
How to Implementing the Idea of Relaxation
Combining
Consider an edgethis
from a vertex
path from u to to
v whosewiththe
is edge
weight w (u, v ). Suppose
, wethat
obtain
we have already ? E
anotherupath
processed so thatfrom
we know to d[u] with
= length
δ(s, u) and also computed a. current estimate for d[v ]. ? E
Then
If There is a (shortest)
path from
v with length
? , then
E
s towe replace
u with length the
d[u] old path < ? ? ? D
withThere
the isnew
a path from s to
shorter path
d[v ] . Hence we update
< ? ? ? D
Combining this path from s to u with the edge (u, v ), we obtain another path from s to v
d[u] +w (u,
with length v )
we replace the old path ⟨s, · · · , w , v ⟩ with the new shorter path
pred[v ] = u
B B
s w d[v]
v
u
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 25 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
Algorithm: Relax(u, v )
1 if ((d[u] + w (u, v ) < d[v ]) then
2 d[v ] := d[u] + w (u, v )
3 pred[v ] := u
4 end
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 26 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 27 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 28 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
INSERT(u, key [u]): Insert u with the key value key [u] in Q.
u = EXTRACT-MIN(): Extract the item with the minimum key value in Q.
DECREASE-KEY(u, new key ): Decrease the key value of u to new key .
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 29 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
Algorithm: DIJKSTRA(G, w , s)
1 for each u ∈ V do
2 d[u] := ∞
3 color[u] := white
4 end
5 d[s] := 0
6 pred[s] :=NIL
7 Q := V ▷ Put all the vertices in a queue Q
8 while (Q is not empty) do
9 u :=EXTRACT-MIN(Q)
10 for (each v adjacent to u) do
11 if (d[u] + w (u, v ) < d[v ]) then
12 d[v ] := d[u] + w (u, v ) ▷ Relax
13 DECREASE-KEY(Q,v ,d[v ])
14 pred[v ] := u
15 end
16 end
17 color([u] := black
18 end
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 30 / 40
Dijkstra’s Algorithm
Single-Source Shortest Path: Dijkstra’s Algorithm
Example:
b inf 1 inf c
7
8
0 3 2
4 5
s
2
inf inf
a 5 d
Algorithms - I
W W W W October 31, 2022 31 / 40
Dijkstra’s Algorithm
Single-Source Shortest Path: Dijkstra’s Algorithm
Example:
b 7 1 inf c
7
8
0 3 2
4 5
s
2
2 inf
a 5 d
*
( ' and ( and
Step 1: Adj[s] = {a, b}, Step work on
1: aAsand b and up- ' Priority
, work on Queue
date information. update information. v a b c d
v s a b c d
d[v ] 2 7 ∞ ∞
d[v ] 0 2 7 ∞ ∞ s a b c d
pred[v ] NIL s s NIL NIL *
+
0 " "
color [v ] B W W W W *
3 nil s s nil nil
) Algorithms
-I
b 5 1 10 c
7
8
0 3 2
4 5
s
2
2 7
a 5 d
priority queue. Adj[a] = {b, c, d},queue.
priority work onAsb,
c, d , work
v on b, , c
* ( * ( *
d
' ) )
d[v ] 0 2 5 10 7
0
*
+ .
pred[v ] NIL s a a a
nil s a a a
color [v ] B B W W W
*
- I
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms October 31, 2022 33 / 40
Dijkstra’s Algorithm
Single-Source Shortest Path: Dijkstra’s Algorithm
Example:
b 5 1 6 c
7
8
0 3 2
4 5
s
2
2 7
a 5 d
(
Step 3: After Step 2, b hasStep the minimum
3: After Step key in 2, thehas the Priority
minimum Queue
key in the
*
( ' ) , work on ' , ) and
priority queue. Adj[b] = {a, priority queue.
c}, work onAs a, c and
v c d
update information. update information.
d[v ] 6 7
v s a b c d
s a b c d
d[v ] 0 2 5 6 7 * + -
0
pred[v ] NIL s a b a *
color [v ] B B B W W 3 nil s a b a
) Algorithms
-I
b 5 1 6 c
7
8
0 3 2
4 5
s
2
2 7
a 5 d
priority queue. Adj[c] = {d},
oritywork on dAs
queue. and update , work on and vupdate
* * *
d
)
d[v ] 0 2 5 6 7
0
*
+
pred[v ] NIL s a b a
color [v ] B B B B W nil s a b a
*
- I
Algorithms
b 5 1 6 c
7
8
0 3 2
4 5
s
2
2 7
a 5 d
minimum in the
Queue: = ϕ.
ority queue.
on cAs , work on and update
* *
So we are done.
information. information.
v s a b c d
s a b c d
d[v ] 0 2 5 6 7
0
*
+
pred[v ] NIL s a b a
nil s a b a
*
color [v ] B B B B B
B B B B B
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 36 / 40
)
Single-Source Shortest Path: Dijkstra’s Algorithm
Dijkstra’s Algorithm
Shortest Path Tree: , where
A=
{(pred[v
? E
B ? E
b 5 1 6 c
0 3
s
2
2 7
a 5 d
Example:
v s a b c d
0 2
s a b c d
d[v ] 50 6 7 G
nil
pred[v ] NIL s B
a sb aa b a
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 37 / 40
Single-Source Shortest Path: Dijkstra’s Algorithm
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 38 / 40
Suggested Readings
Chapter 21-25, Thomas H. Cormen, Charles E. Lieserson, Ronald L. Rivest and Clifford
Stein, Introduction to Algorithms, 3rd Edition, MIT Press/McGraw-Hill, 2009.
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 39 / 40
Thank You
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 31, 2022 40 / 40