BITS, PILANI – K. K.

BIRLA GOA CAMPUS

Design & Analysis of Algorithms

(CS F364)

Lecture No. 8

1
 All-Pairs Shortest Path
G = (V, E) be a graph
G has no negative weight cycles
Vertices are labeled 1 to n
If (i, j) is an edge its weight is denoted by wij
Optimal Substructure Property
Easy to Prove
 Recursive formulation - 1
(𝒎)
𝐋𝐞𝐭 𝒍𝒊𝒋 be the minimum weight of any path from
vertex i to vertex j that contains atmost m edges.
Then,
(𝟎) 𝟎, 𝒊𝒇 𝒊 = 𝒋
𝒍𝒊𝒋 = ቊ
∞, 𝒊𝒇 𝒊 ≠ 𝒋

(𝒎) 𝒎−𝟏 𝒎−𝟏

𝒍𝒊𝒋 = 𝒎𝒊𝒏(𝒍𝒊𝒋 , 𝒎𝒊𝒏 {𝒍𝒊𝒌 + 𝒘𝒌𝒋 })
𝟏≤𝒌≤𝒏
minimum over all predecessors k of j
 Recursive formulation - 2
Floyd-Warshall Algorithm
(𝒌)
𝐋𝐞𝐭 𝒅𝒊𝒋be the weight of the shortest path from
vertex i to vertex j,
for which all the intermediate vertices are in the set
{1,2,….,k}
Then
(𝒌)
𝒅𝒊𝒋 = wij if k = 0
(𝒌) (𝒌−𝟏) (𝒌−𝟏) (𝒌−𝟏)
𝒅𝒊𝒋 = min (𝒅𝒊𝒌 , 𝒅𝒊𝒌 + 𝒅𝒌𝒋 ) if k ≥ 1
We have tacitly used the fact that an optimal path
through k does not visit k twice.
 Optimal Binary Search Tree
We have n keys (represented as k1,k2,…,kn) in sorted order (so
that k1<k2<…<kn),
and we wish to build a binary search tree from these keys.
For each ki ,we have a probability pi that a search will be for ki.
In contrast, some searches may be for values not in ki , and so we
also have n+1 “dummy keys” d0,d1,…,dn
In particular, d0 represents all values less than k1, and
dn represents all values greater than kn, and for i = 1,2,…,n-1, the
dummy key di represents all values between ki and ki+1.
For each di ,we have a probability qi that a search will be for di
The dummy keys are leaves (external nodes), and the data keys
mean internal nodes.
 Optimal Binary Search Tree
• Every search is either successful or unsuccessful and
so we have
𝒏 𝒏

෍ 𝒑𝒊 + ෍ 𝒒𝒊 = 𝟏
𝒊=𝟏 𝒊=𝟎
Because we have probabilities of searches for each key
and each dummy key, we can determine the expected
cost of a search in a given binary search tree T.
Let us assume that the actual cost of a search is the
number of nodes examined,
i.e., the depth of the node found by the search in T,
plus1.
 Optimal Binary Search Tree
Then the expected cost of a search in T is

E[ search cost in T]

= σ𝑛𝑖=1(𝑑𝑒𝑝𝑡ℎ 𝑇 (𝑘𝑖 ) + 1)𝑝𝑖 + σ𝑛𝑖=0(𝑑𝑒𝑝𝑡ℎ 𝑇 𝑑𝑖 +

1)𝑞𝑖

= 1 + σ𝑛𝑖=1 𝑑𝑒𝑝𝑡ℎ 𝑇 (𝑘𝑖 )𝑝𝑖 + σ𝑛𝑖=0 𝑑𝑒𝑝𝑡ℎ 𝑇 𝑑𝑖 𝑞𝑖 (1)

Where depthT denotes a node’s depth in the tree T

Example k2 k2

k1 k4 k1 k5

d0 d1
d0 d1 d5
k3 k5 k4

d2 d3 d4 d5 d4
k3
Figure (a) Figure (a) costs 2.80

i 0 1 2 3 4 5
d2 d3

pi 0.15 0.10 0.05 0.10 0.20

Figure (b)

qi 0.05 0.10 0.05 0.05 0.05 0.10 Figure (b) costs 2.75
 Optimal Binary Search Tree
We start with a problem regarding binary search trees
in an environment in which the probabilities of
accessing elements and gaps between elements is
known.
Goal:
We want to find the binary search tree that minimizes
the expected number of nodes probed on a search.
 Optimal Binary Search Tree
Brute Force Approach:
Exhaustive checking of all possibilities.
Question
What is the number of binary search trees on n keys?
Answer

The number of BST is atleast Ω(2n)

• Greedy Strategy???
1. Put the most frequent key at the root, and then
recursively builds the left and right subtrees.
2. The balanced makes the height the smallest.
 Dynamic Programming Solution
The Dynamic Program for the optimal search tree follows
the same pattern we have seen multiple times now.
Step1 : Optimal Substructure Property
Exercise
Step2 : Recursive Formulation
We pick our subproblem domain as finding an Optimal BST
containing the keys ki ,…,kj , where i ≥1, j ≤ n, and
j ≥ i-1. (It is when j = i-1 that there are no actual keys; we
have just the dummy key di-1.)
Let us define e[i, j] as the expected cost of searching an
Optimal BST containing the keys ki ,…, kj
Ultimately, we wish to compute e[1,n].
 Optimal Binary Search Tree
When j = i-1
Then we have just the dummy key di-1
The expected search cost is e[i, i-1] = qi-1
When j ≥ 1, we need to select a root kr from among
ki ,…,kj
and then make an Optimal BST with keys ki ,…,kr-1 as its
left subtree
and an Optimal BST with keys kr+1 ,…,kj as its right
subtree.
 Optimal Binary Search Tree
What happens to the expected search cost of a subtree
when it becomes a subtree of a node?
The depth of each node in the subtree increases by 1.
So, by equation (1) the excepted search cost of this
subtree increases by the sum of all the probabilities in
the subtree.
For a subtree with keys ki ,…,kj let us denote this sum of
probabilities as
𝒋 𝒋

𝒘 𝒊, 𝒋 = ෍ 𝒑𝒍 + ෍ 𝒒𝒊
𝒍=𝒊 𝒍=𝒊−𝟏
 Optimal Binary Search Tree
Thus, if kr is the root of an optimal subtree containing
keys ki ,…,kj , we have
e[i, j]= pr + (e[i, r-1]+w(i, r-1)) + (e[r+1, j]+w(r+1, j))
Noting that w (i, j) = w(i,r-1)+ pr +w(r+1,j)
We rewrite e[i, j] as
e[i, j]= e[i, r-1] + e[r+1, j] + w(i, j)
The recursive equation as above assumes that we know
which node kr to use as the root.
We choose the root that gives the lowest expected
search cost
 Optimal Binary Search Tree
Final recursive formulation:
𝑞𝑖−1 , 𝑖𝑓 𝑗 = 𝑖 − 1
𝑒 𝑖, 𝑗 = ቐ min { 𝑒 𝑖, 𝑟 − 1 + 𝑒 𝑟 + 1, 𝑗 + 𝑤(𝑖, 𝑗)}, 𝑖𝑓 𝑖 ≤ 𝑗
𝑖≤𝑟≤𝑗

The e[i, j] values give the expected search costs in

Optimal BST.
To help us keep track of the structure of Optimal BST, we
define root[i, j], for 1≤ i ≤ j ≤ n, to be the index r for
which kr is
the root of an Optimal BST containing keys ki ,…,kj
 Longest Common Subsequence (LCS)
Given two sequences
X = x1, x2, …, xm
Y = y1, y2, …, yn
find a maximum length common subsequence
(LCS) of X and Y
Example
X = A, B, C, B, D, A, B
Subsequences of X:
– A subset of elements in the sequence taken in order
For example, A, B, D, B, C, D, B, etc.
 Example
Example
X = A, B, C, B, D, A, B
Y = B, D, C, A, B, A
B, C, B, A is a longest common subsequence of
X and Y (length = 4)
B, D, A, B is also a longest common
subsequence of X and Y (length = 4)
B, D, A, however is not a LCS of X and Y
 Brute Force Solution
Let length of X be m & length of Y be n
Brute Force
For every subsequence of X,
check whether it’s a subsequence of Y
Question? How many subsequences are there of X?
There are 2m subsequences of X
Question?
What is time required to check for each subsequence?
Each subsequence takes O(n) time to check
Scan Y for first letter, then scan for second, & so on
Therefore, Running time: O(n2m) Exponential
 Making the choice
X = A, B, Z, D
Y = Z, B, D
Choice : include one element into the common
sequence (D) and solve the resulting
subproblem
 Notations
• Given a sequence X = x1, x2, …, xm
The i-th prefix of X, for i = 0, 1, 2, …, m is
Xi = x1, x2, …, xi

• c[i, j] = the length of a LCS of the sequences

Xi = x1, x2, …, xi and Yj = y1, y2, …, yj
 Recursive Solution
Case 1: xi = yj
Example
Xi = <D, B, Z, E>
Yj = <Z, B, E>

c[i, j] = c[i - 1, j - 1] + 1

• Append xi = yj to the LCS of Xi-1 and Yj-1

• Must find a LCS of Xi-1 and Yj-1
 Recursive Solution
Case 2: xi  yj
Example
Xi = A, B, Z, G & Yj = A, G, Z
c[i, j] = max { c[i - 1, j], c[i, j-1] }
• Must solve two subproblems
1. find a LCS of Xi-1 and Yj:
Xi-1 = A, B, Z and Yj = A, G, Z
2. find a LCS of Xi and Yj-1:
Xi = A, B, Z, G and Yj-1 = A, G
 Recursive Solution
0 if i = 0 or j = 0
c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj
 Overlapping Subproblems
To find a LCS of X and Y
We may need to find the LCS between
X and Yn-1 and that of Xm-1 and Y
Both the above subproblems has the subproblem
of finding the following:
LCS of Xm-1 and Yn-1
Subproblems share subsubproblems
 Optimal Substructure
Optimal Substructure Property
Easy to prove that in both the cases
Optimal solution to a problem includes optimal
solutions to subproblems
Cut-Paste Argument
 Computing the Length of the LCS
0 if i = 0 or j = 0
c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj
0 1 2 n
yj: y1 y2 yn
0 xi: 0 0 0 0 0 0
1 x1 0 first

2 x2 0 second
0
0
m xm 0 c[m, n]
 Computing the table
0 if i = 0 or j = 0
c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj

Along with c[i, j] we also compute and record b[i, j] which

tells us what choice was made to obtain the optimal value
If xi = yj
b[i, j] = “ ”

Else, if c[i - 1, j] ≥ c[i, j-1]

b[i, j] = “  ”

Else
b[i, j] = “  ”
 Pseudo Code for LCS
1. for i ← 1 to m
2. do c[i, 0] ← 0
3. for j ← 0 to n
4. do c[0, j] ← 0
5. for i ← 1 to m
6. do for j ← 1 to n
7. do if xi = yj
8. then c[i, j] ← c[i - 1, j - 1] + 1
9. b[i, j ] ← “ ”
10. else if c[i - 1, j] ≥ c[i, j - 1]
11. then c[i, j] ← c[i - 1, j]
12. b[i, j] ← “↑”
13. else c[i, j] ← c[i, j - 1]
14. b[i, j] ← “←”
15.return c and b Running time: O(mn)
 Example
X = A, B, C, B, D, A, B 0 if i = 0 or j = 0
Y = B, D, C, A, B, A c[i, j] = c[i-1, j-1] + 1 if xi = yj
max(c[i, j-1], c[i-1, j]) if xi  yj
0 1 2 3 4 5 6
yj B D C A B A
If xi = yj
0 xi
b[i, j] = “ ” 0 0 0 0 0 0 0

Else if c[i - 1, j] ≥ c[i, j-1] 1 A 0 0 0 0  1 1 1
  
b[i, j] = “  ”
2 B 0 1 1 1 1 2 2
else    
3 C 0 1 1 2 2 2 2
b[i, j] = “  ”
4 B 0   
1 1 2 2 3 3
5 D 0     
1 2 2 2 3 3
6 A 0    
1 2 2 3 3 4
7 B 0    
1 2 2 3 4 4
 Constructing a LCS

Start at b[m, n] and

follow the arrows yj B D C A B A

0 xi
0 0 0 0 0 0 0

When we
1 A 0 0 0 0  1 1 1
encounter a “ “ in   
b[i, j] 2 B 0 1 1 1 1 2 2
 xi = yj is an 3 C 0    
1 1 2 2 2 2
element of the LCS
4 B 0   
1 1 2 2 3 3
LCS is BCBA 5 D 0     
1 2 2 2 3 3
6 A 0    
1 2 2 3 3 4
7 B 0    
1 2 2 3 4 4