Matrix-chain multiplication

I sequence (chain) hA1 , A2 , . . . , An i of n matrices to be

multiplied
I matrix multiplication is associative; all parenthesizations yield
the same product
I (A1 (A2 (A3 A4 ))), (A1 ((A2 A3 )A4 )), ((A1 A2 )(A3 A4 ))),
((A1 (A2 A3 ))A4 ), (((A1 A2 )A3 )A4 )
I different way of parenthesizing a chain of matrices yield
different cost evaluating product
I multiply two matrices A and B if they are compatible; number
of columns of A must equal the number of rows of B
I A: p × q and B: q × r → C : p × r
I number of scalar multiplications → pqr dominating
I cost in terms of number of scalar multiplications

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication

I hA1 , A2 , A3 i of dimensions 10 × 100, 100 × 5 and 5 × 50

I ((A1 A2 )A3 ) → 10 · 100 · 5 = 5000 for C = A1 A2 with
dimension 10 × 5
I C multiply with A3 results into 10 · 5 · 50 = 2500
multiplications - total 7500 multiplications
I (A1 (A2 A3 )) needs 100 · 5 · 50 = 25000 plus
10 · 100 · 50 = 50000 multiplications - total 75000
multiplications
I first way of parenthesization 10 times faster
I Given a chain of n matrices parenthesize the product in such a
way that minimizes the number of scalar multiplications
I determine an order for multiplying matrices that has lowest
cost

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication

1. if columns[A] 6= rows[B]
2. then error ”incompatible dimensions”
3. else for i ← 1 to rows[A]
4. do for j ← 1 to columns[B]
5. do C [i, j] ← 0
6. for k ← 1 to columns[A]
7. C [i, j] ← C [i, j] + A[i, k] · B[k, j]
8. return C

I goal is to determine order for multiplying matrices that has

the lowest cost
I exhaustively checking all possible parenthesizations does not
yield an efficient algorithm

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication

I P(n) number of alternative parenthesizations of a sequence of

n matrices
I n = 1 one matrix and one way to fully parenthesize the matrix
product
I n ≥ 2 fully parenthesized matrix product is the product of two
fully parenthesized matrix subproducts
I split between the two subproducts may occur between the kth
and (k + 1)st matrices; yields the recurrence

1 if n = 1
P(n) = Pn−1
k=1 P(k)P(n − k) if n≥2

I the solution to the recurrence is Ω(2n )

I number of solution - exponential in n so brute-force method is
not suitable
M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design
Matrix-chain multiplication: DP approach

I find optimal substructure to construct solution to the problem

I evaluating the product Ai...j → Ai Ai+1 · · · Aj (i < j)
I split the product between Ak and Ak+1 for some k (i ≤ k ≤ j)
I compute Ai...k and Ak+1...j , then multiply these products for
final product Ai...j
I cost parenthesization is the cost of computing Ai...k plus cost
of computing Ak+1...j plus multiplying them together
I for optimal substructure, say optimal parenthesization of
Ai Ai+1 · · · Aj splits the product between Ak and Ak+1
I then the parenthesization of subchain Ai Ai+1 · · · Ak within
this optimal parenthesization of Ai Ai+1 · · · Aj must be an
optimal parenthesization of Ai Ai+1 · · · Ak

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication: DP approach

I this is true because

I if there were a less costly way to parenthesize Ai Ai+1 · · · Ak
substituting that parenthesization in the optimal
parenthesization of Ai Ai+1 · · · Aj would produce another
parenthesization of it whose cost was lower than the
optimum: a contradiction
I similarly, parenthesization of subchain Ak+1 Ak+2 · · · Aj in the
optimal parenthesization of Ai Ai+1 · · · Aj must be an optimal
parenthesization of Ak+1 Ak+2 · · · Aj
I build an optimal solution to an instance of matrix-chain
multiplication problem by splitting the problem into two
subproblems and finding optimal solutions to subproblem
instances and then combine these solutions

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication: DP approach

I define the cost of optimal solution recursively in terms of

optimal solutions to subproblems
I determine the minimum cost of a parenthesization of
Ai Ai+1 · · · Aj for 1 ≤ i ≤ j ≤ n
I m[i, j] the minimum number of scalar multiplications needed
to compute the matrix Ai...j for the full problem; the cost of a
cheapest way to compute A1...n would be m[1, n]
I if i = j only one matrix - no scalar multiplication so
m[i, i] = 0 for i = 1, 2, . . . , n
I to compute m[i, j] when i < j
I optimal parenthesization splits the product Ai Ai+1 · · · Aj
between Ak and Ak+1 i ≤ k < j

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication: DP approach

I m[i, j] equal to minimum cost for computing subproducts

Ai...k and Ak+1...j plus cost of multiplying these two matrices
together
I Ai dimension pi−1 × pi so computing product Ai...k Ak+1...j
takes pi−1 pk pj scalar multiplications

m[i, j] = m[i, k] + m[k + 1, j] + pi−1 pk pj

I optimal parenthesization must use one of k, check for all

possible k (k = i, i + 1, . . . , j − 1) to find the best
I recursive definition for minimum cost of parenthesizing the
product Ai Ai+1 · · · Aj

0 if i = j
m[i, j] = min
i≤k<j {m[i, k] + m[k + 1, j] + pi−1 pk pj } if i < j

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication: DP approach

I m[i, j] give the costs of optimal solutions to subproblems

I to keep track of how to construct an optimal solution
I define s[i, j] - value of k at which the product Ai Ai+1 · · · Aj
can be split to obtain an optimal parenthesization
I s[i, j] = k such that m[i, j] = m[i, k] + m[k + 1, j] + pi−1 pk pj
I it takes exponential time to compute minimum cost m[1, n]
for multiplying A1 A2 · · · An
I for each choice of i and j (1 ≤ i ≤ j ≤ n), this algorithm may
encounter each subproblem many times in different branches
of its recursion tree → overlapping subproblems
I so computer optimal cost using DP approach: tabular
bottom-up approach

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication: DP approach

I Ai has dimensions pi−1 × pi (i = 1, 2, . . . , n)

I p = hp0 , p1 , . . . , pn i length[p] = n + 1
I m[1..n, 1..n] stores m[i, j] and s[1..n, 1..n] records index of k
achieved the optimal cost in computing m[i, j]
I m[i, j] is cost of computing product of j − i + 1 matrices;
depends on the costs of computing products of fewer matrices
than j − i + 1, i.e.,
I Ai...k is product of k − i + 1 < j − i + 1 matrices and Ak+1...j
is product of j − k < j − i + 1 matrices
I table m is filled in a manner that corresponds to solving the
parenthesization problem on matrix chains of increasing length

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

matrix-chain-order(p)

1. n ← length[p] − 1
2. for i ← 1 to n
3. do m[i, i] ← 0
4. for l ← 2 to n #l chain length in increasing order
5. do for i ← 1 to n − l + 1
6. do j ← i + l − 1
7. m[i, j] ← ∞
8. for k ← i to j − 1
9. do q ← m[i, k] + m[k + 1, j] + pi−1 pk pj
10. if q < m[i, j]
11. then m[i, j] ← q
12. s[i, j] ← k
13. return m and s
M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design
Matrix-chain multiplication

I m[i, i] ← 0 initialization - cost for chain of length 1

I then for l = 2, compute m[i, i + 1] using recurrence, ... for
l = 3 compute m[i, i + 2]
I m[i, j] cost depends on m[i, k] and m[k + 1, j] already
computed (for smaller length)
I for n = 6 i ≤ j only upper portion (above main diagonal) of m
is used
matrix dimension
A1 30 × 35
A2 35 × 15
A3 15 × 5
A4 5 × 10
A5 10 × 20
A6 20 × 25

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication

j→
i↓ 1 2 3 4 5 6
1 0 15750 7875 9375 11875 15125
2 0 2625 4375 7125 10500
I m table 3 0 750 2500 5375
4 0 1000 3500
5 0 5000
6 0
j→
i↓ 2 3 4 5 6
1 1 1 3 3 3
2 2 3 3 3
I s table
3 3 3 3
4 4 5
5 5

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication

I m[i, j] is computed using pi−1 pk pj for k = i, i + 1, . . . , j − 1

I running time Ω(n3 ) and requires Θ(n2 ) for m and s
I more efficient than the exponential time method of
enumerating all possible parenthesization and checking each
one


 m[2, 2] + m[3, 5] + p1 p2 p5



 = 0 + 2500 + 35 · 15 · 20 = 13000
m[2, 3] + m[4, 5] + p1 p3 p5

m[2, 5] = min

 = 2625 + 1000 + 35 · 5 · 20 = 7125
m[2.4] + m[5, 5] + p1 p4 p5




= 4375 + 0 + 35 · 10 · 20 = 11375


= 7125

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

Matrix-chain multiplication

I s[i, j] records the value of k such that optimal

parenthesization of Ai Ai+1 · · · Aj splits the product between
Ak and Ak+1
I final matrix multiplication in computing A1..n optimally is
A1..s[1,n] As[1,n]+1..n
I earlier matrix multiplications can be computed recursively
I s[1, s[1, n]] determines the last matrix multiplication in
computing A1..s[1,n] and s[s[1, n] + 1, n] determines the last
matrix multiplication in computing As[1,n]+1..n]
I recursive procedure to print an optimal parenthesization of
hAi , Ai+1 , . . . , Aj i

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design

print-optimal-parens(s, i, j)

1. if i = j
2. then print Ai
3. else print (
4. print-optimal-parens(s, i, s[i, j])
5. print-optimal-parens(s, s[i, j] + 1, j)
6. print )

I for print-optimal-parens(s, 1, 6) prints the parenthesization

((A1 (A2 A3 ))((A4 A5 )A6 ))

M. A. Zaveri, SVNIT, Surat Algorithm Analysis and Design