This action might not be possible to undo. Are you sure you want to continue?

15-211: Fundamental Data Structures and Algorithms Rose Hoberman April 8, 2003

Announcements

Announcements

Readings: • Chapter 14 HW5: • Due in less than one week! • Monday, April 14, 2003, 11:59pm

15-211: Fundamental Data Structures and Algorithms

3

Rose Hoberman April 8, 2003

Today

• • • • • More Graph Terminology (some review) Topological sort Graph Traversals (BFS and DFS) Minimal Spanning Trees After Class... Before Recitation

15-211: Fundamental Data Structures and Algorithms

4

Rose Hoberman April 8, 2003

Graph Terminology

**Paths and cycles
**

• A path is a sequence of nodes v1, v2, …, vN such that (vi,vi+1)∈E for 0<i<N – The length of the path is N-1. – Simple path: all vi are distinct, 0<i<N

**• A cycle is a path such that v1=vN – An acyclic graph has no cycles
**

15-211: Fundamental Data Structures and Algorithms

6

Rose Hoberman April 8, 2003

Cycles

BOS SFO DTW PIT JFK

LAX

15-211: Fundamental Data Structures and Algorithms

7

Rose Hoberman April 8, 2003

**More useful definitions
**

• In a directed graph:

• The indegree of a node v is the number of distinct edges (w,v)∈E. • The outdegree of a node v is the number of distinct edges (v,w)∈E. • A node with indegree 0 is a root.

15-211: Fundamental Data Structures and Algorithms

8

Rose Hoberman April 8, 2003

**Trees are graphs
**

• A dag is a directed acyclic graph. • A tree is a connected acyclic undirected graph. • A forest is an acyclic undirected graph (not necessarily connected), i.e., each connected component is a tree.

15-211: Fundamental Data Structures and Algorithms

9

Rose Hoberman April 8, 2003

Example DAG

Undershorts Socks Watch Pants Shirt Belt Tie Jacket

15-211: Fundamental Data Structures and Algorithms

Shoes

a DAG implies an ordering on events

10

Rose Hoberman April 8, 2003

Example DAG

Undershorts Socks Watch Pants Shirt Belt Tie Jacket

15-211: Fundamental Data Structures and Algorithms

Shoes

**In a complex DAG, it can be hard to find a schedule that obeys all the constraints.
**

11

Rose Hoberman April 8, 2003

Topological Sort

Topological Sort

• For a directed acyclic graph G = (V,E) • A topological sort is an ordering of all of G’s vertices v1, v2, …, vn such that... Formally: for every edge (vi,vk) in E, i<k. Visually: all arrows are pointing to the right

15-211: Fundamental Data Structures and Algorithms

13

Rose Hoberman April 8, 2003

Topological sort

• There are often many possible topological sorts of a given DAG • Topological orders for this DAG :

1 2 4 6 7 5

• • • •

1,2,5,4,3,6,7 2,1,5,4,7,3,6 2,5,1,4,7,3,6 Etc.

3

**• Each topological order is a feasible schedule.
**

15-211: Fundamental Data Structures and Algorithms

14

Rose Hoberman April 8, 2003

**Topological Sorts for Cyclic Graphs?
**

1 2 3

Impossible!

• If v and w are two vertices on a cycle, there exist paths from v to w and from w to v. • Any ordering will contradict one of these paths

15-211: Fundamental Data Structures and Algorithms

15

Rose Hoberman April 8, 2003

**Topological sort algorithm
**

• Algorithm

– Assume indegree is stored with each node. – Repeat until no nodes remain:

• Choose a root and output it. • Remove the root and all its edges.

• Performance

– O(V2 + E), if linear search is used to find a root.

15-211: Fundamental Data Structures and Algorithms

16

Rose Hoberman April 8, 2003

**Better topological sort
**

• Algorithm:

– Scan all nodes, pushing roots onto a stack. – Repeat until stack is empty:

• Pop a root r from the stack and output it. • For all nodes n such that (r,n) is an edge, decrement n’s indegree. If 0 then push onto the stack.

**• O( V + E ), so still O(V2) in worst case, but better for sparse graphs. • Q: Why is this algorithm correct?
**

15-211: Fundamental Data Structures and Algorithms

17

Rose Hoberman April 8, 2003

Correctness

• Clearly any ordering produced by this algorithm is a topological order But... • Does every DAG have a topological order, and if so, is this algorithm guaranteed to find one?

15-211: Fundamental Data Structures and Algorithms

18

Rose Hoberman April 8, 2003

Quiz Break

Quiz

• Prove:

– This algorithm never gets stuck, i.e. if there are unvisited nodes then at least one of them has an indegree of zero.

• Hint:

– Prove that if at any point there are unseen vertices but none of them have an indegree of 0, a cycle must exist, contradicting our assumption of a DAG.

15-211: Fundamental Data Structures and Algorithms

20

Rose Hoberman April 8, 2003

Proof

• See Weiss page 476.

15-211: Fundamental Data Structures and Algorithms

21

Rose Hoberman April 8, 2003

Graph Traversals

Graph Traversals

•Both take time: O(V+E)

15-211: Fundamental Data Structures and Algorithms

23

Rose Hoberman April 8, 2003

Use of a stack

• It is very common to use a stack to keep track of:

– nodes to be visited next, or – nodes that we have already visited.

• Typically, use of a stack leads to a depth-first visit order. • Depth-first visit order is “aggressive” in the sense that it examines complete paths.

15-211: Fundamental Data Structures and Algorithms

24

Rose Hoberman April 8, 2003

**Topological Sort as DFS
**

• do a DFS of graph G • as each vertex v is “finished” (all of it’s children processed), insert it onto the front of a linked list • return the linked list of vertices • why is this correct?

15-211: Fundamental Data Structures and Algorithms

25

Rose Hoberman April 8, 2003

Use of a queue

• It is very common to use a queue to keep track of:

– nodes to be visited next, or – nodes that we have already visited.

• Typically, use of a queue leads to a breadthfirst visit order. • Breadth-first visit order is “cautious” in the sense that it examines every path of length i before going on to paths of length i+1.

15-211: Fundamental Data Structures and Algorithms

26

Rose Hoberman April 8, 2003

Graph Searching ???

• Graph as state space (node = state, edge = action) • For example, game trees, mazes, ... • BFS and DFS each search the state space for a best move. If the search is exhaustive they will find the same solution, but if there is a time limit and the search space is large... • DFS explores a few possible moves, looking at the effects far in the future • BFS explores many solutions but only sees effects in the near future (often finds shorter solutions)

15-211: Fundamental Data Structures and Algorithms

27

Rose Hoberman April 8, 2003

Minimum Spanning Trees

15-211: Fundamental Data Structures and Algorithms

28

Rose Hoberman April 8, 2003

Problem: Laying Telephone Wire

Central office

15-211: Fundamental Data Structures and Algorithms

29

Rose Hoberman April 8, 2003

Wiring: Naïve Approach

Central office

Expensive!

15-211: Fundamental Data Structures and Algorithms

30

Rose Hoberman April 8, 2003

Wiring: Better Approach

Central office

**Minimize the total length of wire connecting the customers
**

15-211: Fundamental Data Structures and Algorithms

31

Rose Hoberman April 8, 2003

**Minimum Spanning Tree (MST)
**

(see Weiss, Section 24.2.2)

A minimum spanning tree is a subgraph of an undirected weighted graph G, such that • it is a tree (i.e., it is acyclic) • it covers all the vertices V

– contains |V| - 1 edges

• the total cost associated with tree edges is the minimum among all possible spanning trees • not necessarily unique

15-211: Fundamental Data Structures and Algorithms

32

Rose Hoberman April 8, 2003

Applications of MST

• Any time you want to visit all vertices in a graph at minimum cost (e.g., wire routing on printed circuit boards, sewer pipe layout, road planning…) • Internet content distribution

– $$$, also a hot research topic – Idea: publisher produces web pages, content distribution network replicates web pages to many locations so consumers can access at higher speed – MST may not be good enough!

• content distribution on minimum cost tree may take a long time!

• Provides a heuristic for traveling salesman problems. The optimum traveling salesman tour is at most twice the length of the minimum spanning tree (why??)

15-211: Fundamental Data Structures and Algorithms

33

Rose Hoberman April 8, 2003

How Can We Generate a MST?

9

a

5

b

6 4 5

9

2 4

d

5

a

5

b

6 4 5

2 4

d

5

c

e

c

e

15-211: Fundamental Data Structures and Algorithms

34

Rose Hoberman April 8, 2003

Prim’s Algorithm

Initialization a. Pick a vertex r to be the root b. Set D(r) = 0, parent(r) = null c. For all vertices v ∈ V, v ≠ r, set D(v) = ∞ d. Insert all vertices into priority queue P, using distances as the keys

9

a

5

b

6 4 5

2 4

d

5

e a b c d 0 ∞ ∞ ∞ ∞ e

35

Vertex Parent e -

c

15-211: Fundamental Data Structures and Algorithms

Rose Hoberman April 8, 2003

Prim’s Algorithm

While P is not empty: 1. Select the next vertex u to add to the tree u = P.deleteMin() 2. Update the weight of each vertex w adjacent to u which is not in the tree (i.e., w ∈ P) If weight(u,w) < D(w), a. parent(w) = u b. D(w) = weight(u,w) c. Update the priority queue to reflect new distance for w

15-211: Fundamental Data Structures and Algorithms

36

Rose Hoberman April 8, 2003

Prim’s algorithm

e d b c a

9

a

5

b

6 4 5

0 ∞ ∞ ∞ ∞

Vertex Parent e b c d Vertex Parent e b e c e d e

2 4

d

5

c

e

d b c a 4 5 5 ∞

The MST initially consists of the vertex e, and we update the distances and parent for its adjacent vertices

15-211: Fundamental Data Structures and Algorithms

37

Rose Hoberman April 8, 2003

Prim’s algorithm

d b c a

9

a

5

b

6 4 5

4 5 5 ∞

Vertex Parent e b e c e d e

2 4

d

5

c

e a c b 2 4 5

Vertex Parent e b e c d d e a d

15-211: Fundamental Data Structures and Algorithms

38

Rose Hoberman April 8, 2003

Prim’s algorithm

a c b

9

a

5

b

6 4 5

2 4 5

2 4

Vertex Parent e b e c d d e a d

d

5

c

e c b 4 5

Vertex Parent e b e c d d e a d

Rose Hoberman April 8, 2003

15-211: Fundamental Data Structures and Algorithms

39

Prim’s algorithm

c b

9

a

5

b

6 4 5

4 5

2 4

d

5

Vertex Parent e b e c d d e a d Vertex Parent e b e c d d e a d

c

e b 5

15-211: Fundamental Data Structures and Algorithms

40

Rose Hoberman April 8, 2003

Prim’s algorithm

b

9

a

5

b

6 4 5

5

2 4

d

5

Vertex Parent e b e c d d e a d

c

e

The final minimum spanning tree

Vertex Parent e b e c d d e a d

Rose Hoberman April 8, 2003

15-211: Fundamental Data Structures and Algorithms

41

**Running time of Prim’s algorithm (without heaps)
**

Initialization of priority queue (array): O(|V|) Update loop: |V| calls • Choosing vertex with minimum cost edge: O(|V|) • Updating distance values of unconnected vertices: each edge is considered only once during entire execution, for a total of O(|E|) updates Overall cost without heaps: O(|E| + |V| 2) When heaps are used, apply same analysis as for Dijkstra’s algorithm (p.469) (good exercise)

15-211: Fundamental Data Structures and Algorithms

42

Rose Hoberman April 8, 2003

**Prim’s Algorithm Invariant
**

• At each step, we add the edge (u,v) s.t. the weight of (u,v) is minimum among all edges where u is in the tree and v is not in the tree • Each step maintains a minimum spanning tree of the vertices that have been included thus far • When all vertices have been included, we have a MST for the graph!

15-211: Fundamental Data Structures and Algorithms

43

Rose Hoberman April 8, 2003

Correctness of Prim’s

• This algorithm adds n-1 edges without creating a cycle, so clearly it creates a spanning tree of any connected graph (you should be able to prove this). But is this a minimum spanning tree? Suppose it wasn't. • There must be point at which it fails, and in particular there must a single edge whose insertion first prevented the spanning tree from being a minimum spanning tree.

15-211: Fundamental Data Structures and Algorithms

44

Rose Hoberman April 8, 2003

Correctness of Prim’s

• Let G be a connected, undirected graph • Let S be the set of edges chosen by Prim’s algorithm before choosing an errorful edge (x,y)

x

y

**• Let V' be the vertices incident with edges in S • Let T be a MST of G containing all edges in S, but not (x,y).
**

15-211: Fundamental Data Structures and Algorithms

45

Rose Hoberman April 8, 2003

Correctness of Prim’s

w

• Edge (x,y) is not in T, so there must be a path in T from x to y since T is connected. • Inserting edge (x,y) into T will create a cycle • There is exactly one edge on this cycle with exactly one vertex in V’, call this edge (v,w)

15-211: Fundamental Data Structures and Algorithms

v x y

46

Rose Hoberman April 8, 2003

Correctness of Prim’s

• Since Prim’s chose (x,y) over (v,w), w(v,w) >= w(x,y). • We could form a new spanning tree T’ by swapping (x,y) for (v,w) in T (prove this is a spanning tree). • w(T’) is clearly no greater than w(T) • But that means T’ is a MST • And yet it contains all the edges in S, and also (x,y) ...Contradiction

15-211: Fundamental Data Structures and Algorithms

47

Rose Hoberman April 8, 2003

Another Approach

• Create a forest of trees from the vertices • Repeatedly merge trees by adding “safe edges” until only one tree remains • A “safe edge” is an edge of minimum weight which does not create a cycle

9

a

5

b

6 4 5

2 4

d

5

forest:

{a}, {b}, {c}, {d}, {e}

c

15-211: Fundamental Data Structures and Algorithms

e

48

Rose Hoberman April 8, 2003

Kruskal’s algorithm

Initialization a. Create a set for each vertex v ∈ V b. Initialize the set of “safe edges” A comprising the MST to the empty set c. Sort edges by increasing weight

9

a

5

b

6 4 5

2 4

d

5

c

15-211: Fundamental Data Structures and Algorithms

e

**F = {a}, {b}, {c}, {d}, {e} A=∅ E = {(a,d), (c,d), (d,e), (a,c), (b,e), (c,e), (b,d), (a,b)}
**

49

Rose Hoberman April 8, 2003

Kruskal’s algorithm

For each edge (u,v) ∈ E in increasing order while more than one set remains: If u and v, belong to different sets U and V a. add edge (u,v) to the safe edge set A = A ∪ {(u,v)} b. merge the sets U and V F = F - U - V + (U ∪ V) Return A • Running time bounded by sorting (or findMin) • O(|E|log|E|), or equivalently, O(|E|log|V|) (why???)

15-211: Fundamental Data Structures and Algorithms

50

Rose Hoberman April 8, 2003

Kruskal’s algorithm

9

a

5

b

6 4 5

2 4

d

5

E=

{(a,d), (c,d), (d,e), (a,c), (b,e), (c,e), (b,d), (a,b)}

c

e A ∅ {(a,d)} {(a,d), (c,d)} {(a,d), (c,d), (d,e)} {(a,d), (c,d), (d,e), (b,e)}

51

Rose Hoberman April 8, 2003

**Forest {a}, {b}, {c}, {d}, {e} {a,d}, {b}, {c}, {e} {a,d,c}, {b}, {e} {a,d,c,e}, {b} {a,d,c,e,b}
**

15-211: Fundamental Data Structures and Algorithms

**Kruskal’s Algorithm Invariant
**

• After each iteration, every tree in the forest is a MST of the vertices it connects • Algorithm terminates when all vertices are connected into one tree

15-211: Fundamental Data Structures and Algorithms

52

Rose Hoberman April 8, 2003

Correctness of Kruskal’s

• This algorithm adds n-1 edges without creating a cycle, so clearly it creates a spanning tree of any connected graph (you should be able to prove this). But is this a minimum spanning tree? Suppose it wasn't. • There must be point at which it fails, and in particular there must a single edge whose insertion first prevented the spanning tree from being a minimum spanning tree.

15-211: Fundamental Data Structures and Algorithms

53

Rose Hoberman April 8, 2003

Correctness of Kruskal’s

K

e

• • • •

S

T

Let e be this first errorful edge. Let K be the Kruskal spanning tree Let S be the set of edges chosen by Kruskal’s algorithm before choosing e Let T be a MST containing all edges in S, but not e.

15-211: Fundamental Data Structures and Algorithms

54

Rose Hoberman April 8, 2003

Correctness of Kruskal’s

Lemma: w(e’) >= w(e) for all edges e’ in T - S Proof (by contradiction): • Assume there exists some edge e’ in T - S, w(e’) < w(e) • Kruskal’s must have e considered e’ before e • However, since e’ is not in K (why??), it must have been discarded because it caused a cycle with some of the other edges in S. • But e’ + S is a subgraph of T, which means it cannot form a cycle ...Contradiction

K

S

T

15-211: Fundamental Data Structures and Algorithms

55

Rose Hoberman April 8, 2003

Correctness of Kruskal’s

• Inserting edge e into T will create a cycle • There must be an edge on this cycle which is not in K (why??). Call this edge e’ • e’ must be in T - S, so (by our lemma) w(e’) >= w(e) • We could form a new spanning tree T’ by swapping e for e’ in T (prove this is a spanning tree). • w(T’) is clearly no greater than w(T) • But that means T’ is a MST • And yet it contains all the edges in S, and also e ...Contradiction

15-211: Fundamental Data Structures and Algorithms

56

Rose Hoberman April 8, 2003

Greedy Approach

• Like Dijkstra’s algorithm, both Prim’s and Kruskal’s algorithms are greedy algorithms • The greedy approach works for the MST problem; however, it does not work for many other problems!

15-211: Fundamental Data Structures and Algorithms

57

Rose Hoberman April 8, 2003

That’s All!

Sign up to vote on this title

UsefulNot useful- Algorithms to Find Linear Geodetic Numbers and Linear Edge Geodetic Numbers in Graphs
- Graph Algorithms
- Graphs i
- Ada Lab File
- Graph
- Prim_s_Algo
- Rohit DS Algorithms
- discrete structure
- Chern_off_new
- lec1
- Operations Research
- Algorithmic Graph
- 10-Unit10
- 12-graph
- Document 1
- Depth First Search Alg
- rwa
- Graph Algorithms
- RAHUL-DST
- 10.1.1.43.6948
- Graphs Dec17 2012
- Study of Neural Network Algorithm for Straight-Line Drawings of Planar Graphs
- 13 Directed Graphs
- Mathieu Dutour Sikiric- Exhaustive Combinatorial Enumeration
- Graphs
- Di Graphs
- Presentation
- Touring Problems - A Mathematical Approach - Vipul Naik
- Week08 Tut
- A09 Greedy Algorithms
- 22348266 Graphs Topo Intro

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue reading from where you left off, or restart the preview.

scribd