Algorithms-I CSC 302 (July-Dec2022 IIITK) L41 - L42

Algorithms - I
CSC 302
SK Hafizul Islam
Department of CSE, IIIT Kalyani
October 19, 2022
SK Hafizul Islam (Department of CSE, IIIT Kalyani) Algorithms - I October 19, 2022 1 / 43
Agenda I
1 Dynamic Programming
Matrix-chain multiplication
Basics
Statement of the problem
Naive approach
Dynamic Programming approach
Optimal Substructure
Recursive definition of optimal solution
Find optimal cost
Construct an optimal solution
An Example
Dynamic Programming Algorithm
Recursive Algorithm
Memoization to avoid exponential time complexity
2 Longest common subsequence

Agenda II
Naive Approach
Dynamic Programming Approach
Recursive Algorithm
3 Suggested Readings
Input: We are given a sequence (chain) ⟨A1 , A2 , · · · , An ⟩ of n matrices to be multiplied.

Output: We wish to compute the product
T = A1 A2 · · · An (1)
with minimum number of scalar multiplications.

Consider three matrices A1 = [·]10×100 , A2 = [·]100×5 , and A3 = [·]5×50 .
If we multiply like ((A1 A2 )A3 ), the computation cost is 7500 scalar multiplications. we
perform
The computation cost for T = (A1 A2 ) is 10 × 100 × 5 = 5000 scalar multiplications.
The computation cost for (T )A3 is 10 × 5 × 50 = 2500 scalar multiplications.
If we multiply like (A1 (A2 A3 )), the computation cost is 75,000 scalar multiplications. we
perform
The computation cost for T = (A2 A3 ) is 100 × 5 × 50 = 25000 scalar multiplications.
The computation cost for A1 (T ) is 10 × 100 × 50 = 50000 scalar multiplications.
Matrix multiplication is recursively define

A1 A2 · · · An = A1 (A2 (A3 · · · (An−1 An )))
Matrix multiplication is associative
A1 A2 A3 = (A1 A2 )A3 = A1 (A2 A3 )
so parenthenization does not change the result.
A product of matrices is fully parenthesized if it is either a single matrix or the product
of two fully parenthesized matrix products, surrounded by parentheses.
For the chain of matrices ⟨A1 , A2 , A3 , A4 ⟩, we can fully parenthesize the product A1 A2 A3 A4
in five distinct ways:
(A1 (A2 (A3 A4 )))
(A1 ((A2 A3 )A4 ))
((A1 A2 )(A3 A4 ))
((A1 (A2 A3 ))A4 )
(((A1 A2 )A3 )A4 )
How we parenthesize a chain of matrices can have a dramatic impact on the cost
of evaluating the product.
We can multiply two matrices A and B only if they are compatible: the number of columns
of A must equal the number of rows of B.
The time to compute C = A ∗ B, where C = [cij ]p×r , A = [aij ]p×q , and B = [bij ]q×r , is
dominated by the number of scalar multiplications in cij = aik ∗ bkj , which is pqr .
We are given a sequence (chain) ⟨A1 , A2 , · · · , An ⟩ of n matrices, where for i = 1, 2, · · · , n,

matrix Ai has dimension to be multiplied pi−1 ×pi , fully parenthesize the product A1 A2 · · · An
in a way that minimizes the number of scalar multiplications.
Our goal is only to determine the “multiplication sequence (parenthenization)” that min-
imizes the number of scalar multiplications in computing.
Matrix-chain multiplication: Naive approach
P(n): Denote the number of alternative parenthesizations of a sequence of n matrices.

Then
n−1
(P
k=1 P(k)P(n − k) if n ≥ 2
P(n) =
1 if n = 1
When n ≥ 2, a fully parenthesized matrix product is the product of two fully parenthesized
matrix subproducts, and the split between the two subproducts may occur between the kth
and (k + 1)st matrices for any k = 1, 2, · · · , n − 1.
T (n) = Ω(2n ) [Use substitution method]
Conclusion: Trying all possible parenthesizations is a bad idea.
Matrix-chain multiplication: Dynamic Programming
Characterize the structure of an optimal solution.

Recursively define the value of an optimal solution.
Compute the value of an optimal solution.
Construct an optimal solution from computed information.
Step 1: The structure of an optimal solution (parenthesization)

For 1 ≤ i ≤ j ≤ n, determine the multiplication sequence Ai···j := Ai Ai+1 Ai+1 · · · Aj .
For any optimal multiplication sequence, at the last step you are multiplying two matrices
Ai···k and Ak+1···j for some k, i.e.,
Ai···j = (Ai Ai+1 · · · Ak )(Ak+1 Ak+2 · · · Aj )
For k = 5, A3···6 = (A3 (A4 A5 ))(A6 ) = A3···5 A6···6

How do we decide where to split the chain (what is k)?

For k = 5, A3···6 = (A3 (A4 A5 ))(A6 ) = A3···5 A6···6

(Search all possible values of k?)
How do we parenthesize the subchains Ai···k and Ak+1···j

For k = 5, A3···6 = (A3 (A4 A5 ))(A6 ) = A3···5 A6···6

(Search all possible values of k?)
How do we parenthesize the subchains Ai···k and Ak+1···j
(Problem has optimal substructure property that Ai···k and Ak+1···j must be optimal so we can
apply the same procedure recursively)

Optimal Substructure Property: If final “optimal” solution of Ai···j involves splitting into
Ai···k and Ak+1···j at final step then parenthesization of Ai···k and Ak+1···j in final optimal
solution must also be optimal for the subproblems “ standing alone”.
If there were a less costly way to parenthesize Ai Ai+1 · · · Ak , then we could substitute that
parenthesization in the optimal parenthesization of Ai Ai+1 · · · Aj to produce another way to
parenthesize Ai Ai+1 · · · Aj whose cost was lower than the optimum: a contradiction.
Step 2: Recursively define the value of an optimal solution

we define the cost of an optimal solution recursively in terms of the optimal solutions to
subproblems.
We choose our subproblems the problems of determining the minimum cost of parenthesizing
Ai Ai+1 · · · Ak for 1 ≤ i ≤ j ≤ n.
Let m[i, j] denote the minimum number of multiplications needed to compute Ai···j .
The optimum cost can be described by the following recursive definition
(
Mini≤k<j m[i, k] + m[k + 1, j] + pi−1 pk pj if i < j
m[i, j] =
0 if i = j
There are only j − i possible values of k, i.e., k = i, i + 1, · · · j − 1, so we can check them all
and find the one which returns a smallest cost.
Step 3: Compute the optimal cost in a bottom-up fashion

Compute each entry m[i, j] for i ≤ j of the Table: m[1 · · · n, 1 · · · n].
Remember that to calculate m[i, j] we must have already evaluated all of the values m[i, k]
and m[k + 1, j].
Step 4: Constructing an optimal solution

The m[i, j] values give the costs of optimal solutions to subproblems, but they do not provide
all the information we need to construct an optimal solution.
Maintain an array s[1 · · · n, 1 · · · n], where s[i, j] denotes a value k for optimal splitting in
computing Ai Ai+1 · · · Aj in an optimal parenthesization. That is, s[i, j] = k such that m[i, j] =
m[i, k] + m[k + 1, j] + pi−1 pk pj .
The array s[1 · · · n, 1 · · · n] can be used recursively to recover the “multiplication sequence
(parenthenization)”.
An Example
Matrix A1 A2 A3 A4
Dimension p0 × p1 p1 × p2 p2 × p3 p3 × p4
Dimension 5×4 4×6 6×2 2×7
An Example
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
We can compute m[i, j] for i < j
m[i, j] 1 2 3 4
1
2
3
4
An Example
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
We can compute m[i, j] for i < j We can compute s[i, j] for i < j
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 1
2 2
3
3
4
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4
1 0
2 0
3 0
4 0
m[i, i] := 0 for i = 1, 2, 3, 4
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 1
2 0 2
3 0
3
4 0
m[i, i] := 0 for i = 1, 2, 3, 4
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4
1 0 120
2 0
3 0
4 0
Computing m[1, 2] by definition

m[1, 2] = Min1≤k<2 m[1, k] + m[k + 1, 2] + p0 pk p2

= m[1, 1] + m[2, 2] + p0 p1 p2
= 0 + 0 + 5 × 4 × 6 = 120
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 1 1
2 0 2
3 0
3
4 0
4

s[1, 2] := k = 1 for optimal parenthesiza-
tion

m[1, 2] = Min1≤k<2 m[1, k] + m[k + 1, 2] + p0 pk p2

= m[1, 1] + m[2, 2] + p0 p1 p2
= 0 + 0 + 5 × 4 × 6 = 120
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4
1 0 120
2 0 48
3 0
4 0

m[2, 3] = Min2≤k<3 m[2, k] + m[k + 1, 3] + p1 pk p3

= m[2, 2] + m[3, 3] + p1 p2 p3
= 0 + 0 + 4 × 6 × 2 = 48
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 1 1
2 0 48 2 2
3 0
3
4 0
Computing m[2, 3] by definition s[2, 3] := k = 2 for optimal parenthesiza-

tion

m[2, 3] = Min2≤k<3 m[2, k] + m[k + 1, 3] + p1 pk p3

= m[2, 2] + m[3, 3] + p1 p2 p3
= 0 + 0 + 4 × 6 × 2 = 48
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4
1 0 120
2 0 48
3 0 84
4 0

m[3, 4] = Min3≤k<4 m[3, k] + m[k + 1, 4] + p2 pk p4

= m[3, 3] + m[4, 4] + p2 p3 p4
= 0 + 0 + 6 × 2 × 7 = 84
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 1 1
2 0 48 2 2
3 0 84
3 3
4 0

tion

m[3, 4] = Min3≤k<4 m[3, k] + m[k + 1, 4] + p2 pk p4

= m[3, 3] + m[4, 4] + p2 p3 p4
= 0 + 0 + 6 × 2 × 7 = 84
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4
1 0 120 88
2 0 48
3 0 84
4 0

m[1, 3] = Min1≤k<3 m[1, k] + m[k + 1, 3] + p0 pk p3

= Min m[1, 1] + m[2, 3] + p0 p1 p3 , m[1, 2] + m[3, 3] + p0 p2 p3

= Min 0 + 48 + 5 × 4 × 2, 120 + 0 + 5 × 6 × 2

= Min 48 + 40, 120 + 60 = 88
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 88 1 1 1
2 0 48 2 2
3 0 84
3 3
4 0

m[1, 3] = Min1≤k<3 m[1, k] + m[k + 1, 3] + p0 pk p3
tion

= Min m[1, 1] + m[2, 3] + p0 p1 p3 , m[1, 2] + m[3, 3] + p0 p2 p3

= Min 0 + 48 + 5 × 4 × 2, 120 + 0 + 5 × 6 × 2

= Min 48 + 40, 120 + 60 = 88
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4
1 0 120 88
2 0 48 104
3 0 84
4 0

m[2, 4] = Min2≤k<4 m[2, k] + m[k + 1, 4] + p1 pk p4

= Min m[2, 2] + m[3, 4] + p1 p2 p4 , m[2, 3] + m[4, 4] + p1 p3 p4

= Min 0 + 24 + 4 × 6 × 7, 48 + 0 + 4 × 2 × 7

= Min 24 + 168, 48 + 56 = 104
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 88 1 1 1
2 0 48 104 2 2 2
3 0 84
3 3
4 0

m[2, 4] = Min2≤k<4 m[2, k] + m[k + 1, 4] + p1 pk p4
tion

= Min m[2, 2] + m[3, 4] + p1 p2 p4 , m[2, 3] + m[4, 4] + p1 p3 p4

= Min 0 + 24 + 4 × 6 × 7, 48 + 0 + 4 × 2 × 7

= Min 24 + 168, 48 + 56 = 104
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4
1 0 120 88 158
2 0 48 104
3 0 84
4 0

m[1, 4] = Min1≤k<4 m[1, k] + m[k + 1, 4] + p0 pk p4

= Min m[1, 1] + m[2, 4] + p0 p1 p4 , m[1, 2] + m[3, 4] + p0 p2 p4 , m[1, 3] + m[4, 4] + p0 p3 p4

= Min 0 + 104 + 5 × 4 × 7, 120 + 24 + 5 × 6 × 7, 88 + 0 + 5 × 2 × 7

= Min 104 + 140, 120 + 24 + 210, 88 + 70 = 158
Matrix A1 A2 A3 A4
Dimension 5×4 4×6 6×2 2×7
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 88 158 1 1 1 3
2 0 48 104 2 2 2
3 0 84
3 3
4 0

m[1, 4] = Min1≤k<4 m[1, k] + m[k + 1, 4] + p0 pk p4
tion

= Min m[1, 1] + m[2, 4] + p0 p1 p4 , m[1, 2] + m[3, 4] + p0 p2 p4 , m[1, 3] + m[4, 4] + p0 p3 p4

= Min 0 + 104 + 5 × 4 × 7, 120 + 24 + 5 × 6 × 7, 88 + 0 + 5 × 2 × 7

= Min 104 + 140, 120 + 24 + 210, 88 + 70 = 158

m[i, j] 1 2 3 4
1 0 120 88 158
2 0 48 104
3 0 84
4 0
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 88 158 1 1 1 3
2 0 48 104 2 2 2
3 0 84
3 3
4 0
How to Recover the Multiplication Sequence (optimal parenthesization)?

If s[1, n] = k1 , then (A1 A2 · · · Ak1 )(Ak1 +1 · · · An )
If s[1, k1 ] = k2 , then (A1 A2 · · · Ak2 )(Ak2 +1 · · · Ak1 )
If s[k1 + 1, n] = k3 , then (Ak1 +1 · · · Ak3 )(Ak3 +1 · · · An )
.. ..
. .
Do this recursively until the multiplication sequence is determined.

m[i, j] 1 2 3 4
1 0 120 88 158
2 0 48 104
3 0 84
4 0
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 88 158 1 1 1 3
2 0 48 104 2 2 2
3 0 84
3 3
4 0

If s[1, 4] = 3, so k = 3, then A1 A2 A3 A4 = (A1 A2 A3 )(A4 )
MCM: Dynamic Programming-An Example

m[i, j] 1 2 3 4
1 0 120 88 158
2 0 48 104
3 0 84
4 0
MCM: Dynamic Programming-An Example
m[i, j] 1 2 3 4 s[i, j] 2 3 4
1 0 120 88 158 1 1 1 3
2 0 48 104 2 2 2
3 0 84
3 3
4 0

If s[1, 4] = 3, so k = 3, then A1 A2 A3 A4 = (A1 A2 A3 )(A4 )
If s[1, 3] = 1, so k = 1, then A1 A2 A3 A4 = ((A1 )(A2 A3 ))(A4 )
Optimal number of scalar multiplication is m[1, 4] = 158.
MCM: Dynamic Programming-Algorithm
Algorithm: Matrix-Chain(p, n) tabular, bottom-up method

1 Let m[1 · · · n, 1 · · · , n] and s[1 · · · n − 1, 2 · · · , n] be new tables
2 for i ← 1 to n do
3 m[i, i] := 0
4 end
5 for ℓ ← 2 to n do
6 for i ← 1 to n − ℓ + 1 do
7 j := i + ℓ − 1 ▷ ℓ is the chain length
8 m[i, j] := ∞
9 for k ← i to j − 1 do
10 q := m[i, k] + m[k + 1, j] + pi−1 pk pj
11 if (q < m[i, j]) then
12 m[i, j] := q
13 s[i, j] := k
14 end
15 end
16 end
17 end
Hafizul Islam ·(Department
18SKReturn(m[1 · · n, 1 · · · ,ofn],CSE, · · · nKalyani)
s[1IIIT − 1, 2 · · · , n]) Algorithms - I October 19, 2022 26 / 43
Algorithm: Matrix-Chain(p, n) tabular, bottom-up method T (n) = O(n3 )

1 Let m[1 · · · n, 1 · · · , n] and s[1 · · · n − 1, 2 · · · , n] be new tables S(n) = Θ(n2 )
3 m[i, i] := 0
4 end
5 for ℓ ← 2 to n do
6 for i ← 1 to n − ℓ + 1 do
7 j := i + ℓ − 1 ▷ ℓ is the chain length
8 m[i, j] := ∞
10 q := m[i, k] + m[k + 1, j] + pi−1 pk pj
12 m[i, j] := q
13 s[i, j] := k
14 end
15 end
16 end
17 end
Hafizul Islam ·(Department
18SKReturn(m[1 · · n, 1 · · · ,ofn],CSE, · · · nKalyani)
s[1IIIT − 1, 2 · · · , n]) Algorithms - I October 19, 2022 26 / 43
Constructing an optimal solution
Algorithm: Print-Optimal-Sequence(s, i, j) s[i, j] is global to this recursive pro-

cedure.
1 if (i = j) then
To compute A1 A2 · · · An , call
2 Print Ai Print-Optimal-Sequence(s, 1, n).
3 end
4 else
5 Print “(”
6 Print-Optimal-Sequence(s, i, s[i, j])
7 Print-Optimal-Sequence(s, s[i, j] + 1, j)
8 Print “)”
9 end
0 Return(m[1 · · · n, 1 · · · , n], s[1 · · · n − 1, 2 · · · , n])
Algorithm: Recursive-Matrix-Chain(p, i, j)
1 if (i = j) then
2 Return 0
3 end
4 m[i, j] := ∞
6 q :=Recursive-Matrix-Chain(p, i, k)+Recursive-Matrix-Chain(p, k + 1, j)+pi−1 pk pj
7 end
9 m[i, j] := q
10 end
11 Return(m[1, j])
( Pn−1
1 + k=1 T (k) + T ((n − k) + 1) if n > 1
T (n) =
1 if n = 1
T (n) = Ω(2n )
MCM: Overlapping subproblems
15.3 Elements of dynamic programming 385
1..4
1..1 2..4 1..2 3..4 1..3 4..4
2..2 3..4 2..3 4..4 1..1 2..2 3..3 4..4 1..1 2..3 1..2 3..3
3..3 4..4 2..2 3..3 2..2 3..3 1..1 2..2
Algorithm: Memoized-Matrix-Chain(p)
1 Let m[1 · · · n, 1 · · · n] be anew table
3 for j ← i to n do
4 m[i, j] := ∞
5 end
6 end
7 Return Lookup-Chain(m, p, 1, n)
Algorithm: Lookup-Chain(m, p, i, j)
1 if (m[i, j] < ∞) then
2 Return m[i, j]
3 end
4 if (i = j) then
5 m[i, j] := 0
6 end
7 else
9 q :=Lookup-Chain(m, p, i, k)+Lookup-
Chain(m, p, k + 1, j)+pi−1 pk pj
11 m[i, j] := q
12 end
13 end
14 end
15 Return m[i, j]
Algorithm: Lookup-Chain(m, p, i, j) T (n) = O(n3 )

1 if (m[i, j] < ∞) then
2 Return m[i, j]
3 end
4 if (i = j) then
5 m[i, j] := 0
6 end
7 else
9 q :=Lookup-Chain(m, p, i, k)+Lookup-
Chain(m, p, k + 1, j)+pi−1 pk pj
11 m[i, j] := q
12 end
13 end
14 end
15 Return m[i, j]
Longest common subsequence
Input: Two sequences

X = ⟨x1 , x2 , · · · , xm ⟩
Y = ⟨y1 , y2 , · · · , yn ⟩
Output: Longest common subsequence (LCS) of X and Y .
The maximum length sequence of characters that appears from left-to-right (but not
necessary a continuous string) in both the sequences
Given a sequence X = ⟨x1 , x2 , · · · , xm ⟩, another sequence Y = ⟨y1 , y2 , · · · , yn ⟩ is a sub-

sequence of X if there exists a strictly increasing sequence ⟨i1 , i2 , · · · , ik ⟩ of indices of X
such that for all j = 1, 2, · · · , k, we have xij = yj .
X = ⟨A, B, C , B, D, A, B⟩, Y = ⟨B, C , D, B⟩ is a subsequence of X with corresponding index
sequence ⟨2, 3, 5, 7⟩.
Given two sequences X and Y , we say that a sequence Z is a common subsequence of

X and Y if Z is a subsequence of both X and Y .
X = ⟨A, B, C , B, D, A, B⟩ and Y = ⟨B, D, C , A, B, A⟩, the sequence Z = ⟨B, C , A⟩ is a
common subsequence of both X and Y .
But Z = ⟨B, C , A⟩ is not a longest common subsequence (length = 3).
The sequence ⟨B, C , B, A⟩, which is also common to both X and Y (length = 4)
The sequence ⟨B, D, A, B⟩ is subsequence of X and Y sequence of length = 4.
Therefore, ⟨B, C , B, A⟩ and ⟨B, D, A, B⟩ are the LCS of X and Y since X and Y have no
common subsequence of length 5 or greater.
LCS: Naive Approach
In a brute-force approach to solving the LCS problem, we would enumerate all subsequences
of X and check each subsequence to see whether it is also a subsequence of Y , keeping
track of the longest subsequence we find.
Each subsequence of X corresponds to a subset of the indices {1, 2, 3, · · · , m}.
X has 2m subsequences, this approach requires exponential time.
LCS: Dynamic Programming Approach
Step 1: Characterizing a longest common subsequence

The LCS problem has an optimal-substructure property
Theorem 1 (Optimal substructure of an LCS)

Let X = ⟨x1 , x2 , · · · , xm ⟩ and Y = ⟨y1 , y2 , · · · , yn ⟩ be sequences, and let Z = ⟨z1 , z2 , · · · , zk ⟩
be any LCS of X and Y .
If xm = yn , then zk = xn = ym and Zk−1 is an LCS of Xm−1 and Yn−1 .
If xm ̸= yn , then zk ̸= xm implies that Z is an LCS of Xm−1 and Y .
If xm ̸= yn , then zk ̸= yn implies that Z is an LCS of X and Yn−1 .
Step 3: Recursive solution

If xm = yn , we must find an LCS of Xm−1 and Yn−1 . Appending xm = yn to this LCS yields
an LCS of X and Y
If xm ̸= yn , then we must solve two subproblems: finding an LCS of Xm−1 and Y and finding
an LCS of X and Yn−1 . Whichever of these two LCSs is longer is an LCS of X and Y .
Let c[i, j] to be the length of an LCS of the sequences Xi and Yj .
If either i = 0 or j = 0, one of the sequences has length 0, and so the LCS has length 0. The
optimal substructure of the LCS problem gives the recursive formula
Step 3: Recursive solution

If xm = yn , we must find an LCS of Xm−1 and Yn−1 . Appending xm = yn to this LCS yields
an LCS of X and Y
If xm ̸= yn , then we must solve two subproblems: finding an LCS of Xm−1 and Y and finding
an LCS of X and Yn−1 . Whichever of these two LCSs is longer is an LCS of X and Y .
Let c[i, j] to be the length of an LCS of the sequences Xi and Yj .
If either i = 0 or j = 0, one of the sequences has length 0, and so the LCS has length 0. The
optimal substructure of the LCS problem gives the recursive formula

0
 if i = 0 or j = 0,
c[i, j] = c[i − 1, j − 1] + 1 if i > 0, j > 0, and xi = yj ,

max c[i, j − 1], c[i − 1, j] if i > 0, j > 0, and xi ̸= yj

Step 3: Computing the length of an LCS T (n) = Θ(mn)

Algorithm: LCS-Length(X , Y )
1 Let b[1 · · · m, 1 · · · n] and c[0 · · · m, 0 · · · n] be anew tables
2 for i ← 1 to m do
3 c[i, 0] := 0
4 end
5 for j ← 1 to n do
6 c[0, j] := 0
7 end
8 for i ← 1 to m do
9 for j ← 1 to n do
10 if (xi = yj ) then
11 c[i, j] := c[i − 1, j − 1] + 1
12 b[i, j] := “ ↖ ”
13 end
14 else
15 if (c[i − 1, j] ≥ c[i, j − 1]) then
16 c[i, j] := c[i − 1, j]
17 b[i, j] := “ ↑ ”
18 end
19 else
20 c[i, j] := c[i, j − 1]
21 b[i, j] := “ ← ”
22 end
23 end
24 end
25 end
26 Return (c, b)
Step 4: Constructing an LCS

Algorithm: Print-LCS(b, X , i, j)
1 if (i = 0) or (j = 0) then
2 Return null
3 end
4 if (b[i, j] = “ ↖ ” then
5 Print-LCS(b, X , i − 1, j − 1)
6 Print xi
7 end
8 else
9 if (b[i, j] = “ ↑ ”) then
10 Print-LCS(b, X , i, j)
11 Print-LCS(b, X , i, j − 1)
12 end
13 end
T (n) = (m + n)
15.4 4:
Step Longest common subsequence
Constructing an LCS 395
j 0 1 2 3 4 5 6
i yj B D C A B A
0 xi 0 0 0 0 0 0 0
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2
4 B 0 1 1 2 2 3 3
5 D 0 1 2 2 2 3 3
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
15.4 4:
Step Longest common subsequence
Constructing an LCS 395
j 0 1 2 3 4 5 6
X = ⟨A, B, C , B, D, A, B⟩
i yj B D C A B A
Y = ⟨B, D, C , A, B, A⟩
LCS of X and Y is
0 xi 0 0 0 0 0 0 0
Z = ⟨B, C , B, A⟩
1 A 0 0 0 0 1 1 1
2 B 0 1 1 1 1 2 2
3 C 0 1 1 2 2 2 2
4 B 0 1 1 2 2 3 3
5 D 0 1 2 2 2 3 3
6 A 0 1 2 2 3 3 4
7 B 0 1 2 2 3 4 4
Longest common subsequence: Recursive Algorithm
Algorithm: LCS-Length-Recur(X , Y , m, n)
1 if m = 0||n = 0 then
2 Return 0
3 end
4 else if X [m] = y [n] then
5 Return 1+LCS-Length-Recur(X , Y , m − 1, n − 1)
6 end
7 else
8 Return max LCS-Length-Recur(X , Y , m, n − 1),LCS-Length-Recur(X , Y , m − 1, n)
9 end
1
https://www.enjoyalgorithms.com/blog/longest-common-subsequence
Longest common subsequence: Recursive Algorithm
Algorithm: LCS-Length-Recur(X , Y , m, n)
1 if m = 0||n = 0 then
2 Return 0
3 end
4 else if X [m] = y [n] then
5 Return 1+LCS-Length-Recur(X , Y , m − 1, n − 1)
6 end
7 else
8 Return max LCS-Length-Recur(X , Y , m, n − 1),LCS-Length-Recur(X , Y , m − 1, n)
9 end
Total number of subsequences of X and Y are 2m and 2n , respectively1 .

Time complexity is 2m+n .
Space complexity is O(max(m, n)).
1
https://www.enjoyalgorithms.com/blog/longest-common-subsequence
Suggested Readings
Chapter 15, Thomas H. Cormen, Charles E. Lieserson, Ronald L. Rivest and Clifford Stein,
Introduction to Algorithms, 3rd Edition, MIT Press/McGraw-Hill, 2009.
Chapter 05, E. Harowitz, S. Sahani, and S. Rajasekaran, Fundamentals of Computer
Algorithms, 2nd Edition, University Press, 2008.
Chapter 12, Michael T. Goodrich and Roberto Tamassia, Algorithm Design and Appli-
cations, Wiley, 2014.
Thank You

Algorithms-I CSC 302 (July-Dec2022 IIITK) L41 - L42

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Algorithms-I CSC 302 (July-Dec2022 IIITK) L41 - L42

Uploaded by

Copyright:

Available Formats

Algorithms - I

Department of CSE, IIIT Kalyani

October 19, 2022

2 Longest common subsequence

Input: We are given a sequence (chain) ⟨A1 , A2 , · · · , An ⟩ of n matrices to be multiplied.

with minimum number of scalar multiplications.

Matrix multiplication is recursively define

We are given a sequence (chain) ⟨A1 , A2 , · · · , An ⟩ of n matrices, where for i = 1, 2, · · · , n,

P(n): Denote the number of alternative parenthesizations of a sequence of n matrices.

Characterize the structure of an optimal solution.

Step 1: The structure of an optimal solution (parenthesization)

Ai···j = (Ai Ai+1 · · · Ak )(Ak+1 Ak+2 · · · Aj )

For k = 5, A3···6 = (A3 (A4 A5 ))(A6 ) = A3···5 A6···6

Step 1: The structure of an optimal solution (parenthesization)

Ai···j = (Ai Ai+1 · · · Ak )(Ak+1 Ak+2 · · · Aj )

For k = 5, A3···6 = (A3 (A4 A5 ))(A6 ) = A3···5 A6···6

Step 1: The structure of an optimal solution (parenthesization)

Ai···j = (Ai Ai+1 · · · Ak )(Ak+1 Ak+2 · · · Aj )

For k = 5, A3···6 = (A3 (A4 A5 ))(A6 ) = A3···5 A6···6

Step 1: The structure of an optimal solution (parenthesization)

Step 2: Recursively define the value of an optimal solution

Step 3: Compute the optimal cost in a bottom-up fashion

Step 4: Constructing an optimal solution

Computing m[1, 2] by definition

Computing m[1, 2] by definition

Computing m[2, 3] by definition

Computing m[2, 3] by definition s[2, 3] := k = 2 for optimal parenthesiza-

Computing m[3, 4] by definition

Computing m[3, 4] by definition s[3, 4] := k = 3 for optimal parenthesiza-

Computing m[1, 3] by definition

Computing m[1, 3] by definition s[1, 3] := k = 1 for optimal parenthesiza-

Computing m[2, 4] by definition

Computing m[2, 4] by definition s[2, 4] := k = 2 for optimal parenthesiza-

Computing m[1, 4] by definition

Computing m[1, 4] by definition s[2, 4] := k = 3 for optimal parenthesiza-

We can compute m[i, j] for i < j

How to Recover the Multiplication Sequence (optimal parenthesization)?

We can compute m[i, j] for i < j

How to Recover the Multiplication Sequence (optimal parenthesization)?

We can compute m[i, j] for i < j

How to Recover the Multiplication Sequence (optimal parenthesization)?

Algorithm: Matrix-Chain(p, n) tabular, bottom-up method

Algorithm: Matrix-Chain(p, n) tabular, bottom-up method T (n) = O(n3 )

Constructing an optimal solution

Algorithm: Print-Optimal-Sequence(s, i, j) s[i, j] is global to this recursive pro-

1..1 2..4 1..2 3..4 1..3 4..4

3..3 4..4 2..2 3..3 2..2 3..3 1..1 2..2

Algorithm: Lookup-Chain(m, p, i, j) T (n) = O(n3 )

Input: Two sequences

Given a sequence X = ⟨x1 , x2 , · · · , xm ⟩, another sequence Y = ⟨y1 , y2 , · · · , yn ⟩ is a sub-

Given two sequences X and Y , we say that a sequence Z is a common subsequence of

Step 1: Characterizing a longest common subsequence

Theorem 1 (Optimal substructure of an LCS)

Step 3: Recursive solution

Step 3: Recursive solution

Step 3: Computing the length of an LCS T (n) = Θ(mn)

Step 4: Constructing an LCS

Total number of subsequences of X and Y are 2m and 2n , respectively1 .

You might also like