Breadth First Search Depth First search

BFS (G,S): # S is source vertex Depth first search (G):

For each vertex v in G:
For each vertex v in g except S: v.status=unseen
v.status=unseen v.predecessor= none
v.distance= infinity Time=0
v.predecessor= none While there is unseen vertex v:
DFS(v)
s.status=visited
Enqueue(Q ,S) # add s to Queue DFS(v):
While Q not empty: v.status=visited
u=Deque (Q) # remove first element from queue v.start_time=time++
For each unseen neighbor v of u: While there is unseen neighbor U of v: {
v.status=visited v.predecessor=U
v.distance=u.distance+1 DFS(U)}
v.predecessor=u v.status=finished
Enqueue(Q,v) v.finish_time=time++
u.status=finished

Output: breadth first tree Output: forest with tree, back, forward, cross edges

Runtime= O(n+m) Runtime= O(n+m)

Runtime for connected graphs=O(m)

Uses: Uses:
Find shortest path to source vertex Topological sort by outputting vertices in decreasing order
of finish time
Works on unweighted graphs, dags and undirected
graphs Find SCCS:
1.Do a DFS on G and record finish times.
2.Do a DFS on G_reversed, processing vertices in order of
decreasing finish times from 1.
3. Output each DFS forest in 2 as a strongly connected
component of G

Works on unweighted graphs, dags and undirected graphs

Greedy Proofs

Progress argument Exchange argument

This algorithm progresses as quickly as any other
algorithm. For every k, the first k jobs that the greedy
solution picks are the k shortest jobs. Suppose the greedy
solution ends up picking the L shortest jobs whose total
time is less than or equal to T. It couldn’t add the (L+1)th
job since this would bring total time over T. So any set L+1
jobs must have total time > T. Therefore the greedy
algorithm is optimal
For the sake of contradiction , assume that there is some
optimal sequence of jobs, S_o, that finished earlier than
the sequence produced by our greedy algorithm, S_g. Let
g_i and o_i be the first sets that differ between S_o and
S_g. We know g_i won’t overlap with prior jobs in S_o.
Due to the ordering of the greedy solution, g_i finishes
before or at the same time as o_i. So we can replace o_i
with g_i and still have an optimal solution. We can continue
this way until S_g=S_o. So S_o cannot finish before S_g
since it is equal to S_g. This contradicts the assumption,
and therefore there is no sequence that finishes before the
greedy sequence, so S_g is the best possible sequence.
Huffman coding

While there is more than 1 symbol:

Combine the two lowest frequency symbols into one
Split combined symbols into two in reverse order, making the combined leaf the parent of two leaves that were
separated.

Runtime: O(n *logn)

Properties: as depth of leaves decrease, frequency of symbols decreases. Every node has two children. Full binary tree

Cut property If E is the lightest edge between a subset of vertices and the rest of the graph, then every MST
contains E.

Cycle property If edge E is the maximum weight edge of a cycle, no MST contains E

Other MST -If e is any edge of minimum weight in G, it is part of some MST
Properties -If G contains a unique edge of min weight, it is part of every MST.
-If e is part of some MST of G, it is an edge of min weight across some cut of G
- Multiplying every weight by positive constant, adding a constant, or changing to 2^w wont change
MST, but changes shortest path from source bc may greatly increase the weight of paths that use lots
of edges

Finding Minimum spanning tree( non negative and distinct weights)

Kruskal Prim Cycle property

Kruskal(G,w): Prim( G, w, root): Repeatedly delete

A = empty forest For each u in G: the heaviest weight
For each vertex v in G: u.min=infinity edge in every cycle
Make-Set(v) #u.min =min weight of edge connecting u to tree
Sort the edges of G in nondecreasing order u.predecessor=empty
of weight root.min=0
For each edge (u,v): Create Q# min heap of vertices keyed by min vals
If Find-Set(u) !=Find-Set(v):
Add edge (u,v) to A While Q not empty:
Union(u,v) u=extractmin(Q)
For each v adjacent to u:
Make-Set(v): Create a new set whose only If v in Q and weight(u,v)<v.min:
member is pointed to by v. Note that for this v.predecessor=u
operation v must already be in a set. v.min= weight(u,v)

Find-Set(v): Returns a pointer to the set

containing v.
Union(u, v): Unites the dynamic sets that
contain u and v into a new set

Runtime: O[m log(n)] Runtime: O[m log(n)]

Dijkstra’s algorithm -find the shortest path from a source to all vertices in a non-negative weight directed graph

Dijkstra (G,s): # s is source vertex

For each vertex v in G:

v.distance= infinity
v.predecessor= none
s.distance=0
S=Empty set
Create Q= min heap of vertices keyed by distance values
While Q not empty:
u=extractmin (Q)
S=S +u # add u to set
For each vertex v adjacent to u:
If v.distance>u.distance+weight(u,v):
v.distance=u.distance+weight(u,v)
v.predecessor=u

Runtime: O[ (n+m)log(n)]

Dynamic programming example

Weighted activity selection : A set s of n activities where each activity a_i has start time s_i and finish time f_i and weight
w_i. Find a maximum weight set of non overlapping activities

Sort the jobs in increasing order of finish time.

Compute array P[1..n] where P[i] is largest index j<i such that f_j < s_i (finishes before i starts).
T is max weight you can choose from first i activities.

(bottom-up)- solve smallest problem then build up Memoization( top down)

T[0]=0 Set each T value =-1

For i=1 to i=n:
T[i]=Max{wi+T[P(i)], T[i-1]} WAS(i):
If T[i]=-1:
If i=0:
T[i]=0
Else:
T[i]=Max{w_i+WAS(P(i)), WAS(i-1)}
Return T[i]

proof: proof:

Time complexity= # subproblems* time to complete solution to each subproblem

Counting Number of paths

Optimal Static binary search tree

Given n keys (sorted) k1<k2<kn with probability of each key being access p1<p2<pn
Build binary search tree that minimizes average access time

Super complicated????????? can u simplify????

Runtime: n^3

Find all pairs shortest path(the minimum distance from any -Find shortest path from source with negative edge
node to all other nodes in the graph) with negative edge weights
weights (assume no neg cycles) -Negative Cycle detection

Floyd-Warshall -explain more????????? Bellman-Ford

dijk is weight of shortest path from vertices i to j for which Bellman-Ford (G,w, s): # s is source vertex
all intermediate vertices are in the set {1...k} For each vertex v in G:
v.distance= infinity
v.predecessor= none
Floyd-Warshall( Weighted_adjacency_matrix W): s.distance=0
n=W.rows For i=1 to V-1
D0=W # D is matrix of shortest path weights For each edge (u,v) in E:
For k=1 to n: If v.distance>u.distance+weight(u,v):
Let Dk = dijk be a new n*n matrix v.distance=u.distance+weight(u,v)
For i=1 to n: v.predecessor=u
For j=1to n:
dijk =min(dijk-1, dikk-1+dkjk-1) For each edge (u,v) in E:
If v.distance>u.distance+weight(u,v):
Return Dn
Return “negative weight cycle reachable from s”
Return “NO negative weight cycle reachable from s”

Runtime = O(n^3) O(VE)

Regression- minimize number of lines used and squared errors

error=sum (mxi+b-yi)^2 , sum of distances between points and line

opt(i)=cost of points (x1,y1) up to (xi, yi)

reg(i,j)=least squared error for points (xi+1,yi+1) up to (xj, yj)

opt(n)=mini<nopt(i)+C+ reg(i,n) # calculates min cost

Compute bottom-up or top-down

Runtime : O(n^2)

Sequence alignment

Add more here

Graph Representations
Algorithm Time Complexity Time Complexity Time Description & Common Use
Creating Graph Visiting ALL Complexity
Neighbors Pair Adjacent?
(of vertex v)
Adjacency List (AL) O(m+n) O(deg(v)) O(deg(v)) Includes all vertices & then which vertices
have edges to them from each vertex.
More commonly used of the two graph
representations.
Adjacency Matrix 2 O(n) O(1) Commonly Used with Floyd Warshall’s All
O(n )
(AM) Pairs Shortest Path Algorithm

Graph Traversal Algorithms

Algorithm Applications Time Complexity Output/Description

Depth First Search -Topological Sort O(n+m) -DFS Forest (# trees based on order
(DFS) Algorithm (DAGs processed)
only) n = vertices, -start/finish times for each vertex
m = edges -edge types
-Strongly Connected
Component Algorithm
O(m)
(connected graph,
m>= n-1)

Breadth First Search Shortest Path O(n+m)

(BFS) (non-weighted graphs
only)

Topological Sort Useful for DAGs only. O(n+m) 1. Run DFS

Algorithm 2. Arrange vertices in decreasing order of
finish time
Decreasing order of finish time list is
the topological sort.

Strongly Connected O(n+m) 1. Run DFS, record finish times

Component (SCC) T
2. Create transposed graph G (reversing
Algorithm
all edges)
T
3. Run DFS on G picking unseen
vertices in decreasing order of finish time
for previous DFS.
Output vertices of each DFS call are
one SCC.

Graph Optimization Algorithms

Algorithm Purpose (Output) Style Time Complexity Directed/ Negative
Undirected Edges
? OK?
Binary Search Divide & O(log n) (when balanced)
Conquer
O(height)

Kruskal’s Algorithm Minimum Cost Spanning Greedy O(m log n) Undirected O (+/-)
Tree (MST) ONLY

-Repeatedly select min cost **If sorting easy bc of some

edge unless it forms a cycle -MST often used for network feature of weights allowing
until design problems easier sorting then Kruskals
l E’=|V|-1 might be better vs Prim’s

Prim’s Algorithm Minimum Cost Spanning Greedy O(m log n) Undirected OK (+/-)
Tree (MST) 2 ONLY
(w/min heap, otherwise O(n ))
-Select min cost edge
-Select min edge connected
to current tree unless it forms
a cycle & repeat until
E’=|V|-1

Dijkstra’s Algorithm Single Source Shortest Greedy 2 Both NO

O(n ) ( w/array) or
Path (SSSP) Directed & (+)
Undirected
-Identify Source
-Label starting distances for
O(m log n) (w/min heap)
each node. Neighbors = -SSSP often used for
edge weight; rest = infinity. network routing problems **Array better for max edges,
- Select shortest path vertex use min heap for more sparse
graphs
& perform relaxation on its
neighbors
- Repeat for next shortest

Floyd-Warshall's All Pairs Shortest Path Dynamic 3 Both OK (+/-)

Programming
O(n )
Algorithm -a good choice for computing Directed &
(DP) Undirected BUT, no
paths between all pairs of
vertices in dense graphs negative
cycles

Bellman Ford’s Single Source Shortest Dynamic O(n2 + mn) OK (+/-)

Algorithm Path (SSSP) Programming
(DP)
O(n3) worse case - complete
graph