Matrix Chain Multiplication

• Given some matrices to multiply, determine the best

order to multiply them so you minimize the number of
single element multiplications.
• i.e. Determine the way the matrices are parenthesized.

• First off, it should be noted that matrix multiplication is

associative, but not commutative. But since it is
associative, we always have:

• ((AB)(CD)) = (A(B(CD))), or any other grouping as long

as the matrices are in the same consecutive order.

• BUT NOT: ((AB)(CD)) = ((BA)(DC))

• How to compute A1 A2 ... An where Ai is a
matrix for every i.
• Example: A1 A2 A3 A4
( A1( A2 ( A3 A4 ))) ( A1(( A2 A3 ) A4 ))
(( A1 A2 )( A3 A4 )) (( A1( A2 A3 )) A4 )
((( A1 A2 ) A3 ) A4 )
Formula for finding
possible combination

2n-1Cn-1 / n
Where n is the number of matrices
Example: A1 A2 A3 A4
n=4
2 x (4-1) C4-1 / 4 = 6 C3 / 4 = 5
Example:
A1 is a 10  100 matrix, A2 is a 100  5 matrix,

Then A1 A2 takes
10 x 100 x5 = 5000 times
Cost of matrix multiplication:

Let A be a p  q matrix
B be a q  r matrix.
Then the cost(number of multiplication) is
pqr
MATRIX MULTIPLY
MATRIX MULTIPLY(A, B)
1 if columns[A]  column[B]
2 then error “incompatible dimensions”
3 else for i 1 to rows[A]
4 do for j  1 to columns[B]
5 do c [ i , j ]  0
6 for k 1 to columns[A]
7 do c [ i , j ]  c [ i , j ]  A[ i , k ] B[ k , j ]
8 return C
Example:
A1 is a 10  100 matrix, A2 is a 100  5 matrix,
and A3 is a 5  50 matrix
Then (( A1 A2 ) A3 )
takes 10  100  5  10  5  50  7500 time.
However ( A1( A2 A3 ))
takes 100  5  50  10  100  50  75000 time.
The matrix-chain
multiplication problem:
• Given a chain  A1 , A2 ,..., An  of n matrices,
where for i=0,1,…,n, matrix Ai has
dimension pi-1pi, fully parenthesize the
product A1 A2 ... An in a way that
minimizes the number of scalar
multiplications.
Step 1: The structure of an
optimal parenthesization

Optimal

(( A1 A2 ... Ak )( Ak 1 Ak  2 ... An ))

Combine
Step 2:A Recursive Solution
+ Let m[i, j] = the minimum number of multiplications needed to compute
Ai…j
+ Consider the subproblem of parenthesizing
Ai…j = Ai Ai+1  Aj for 1  i  j  n
p i-1 p kp j
= Ai…k Ak+1…j for i  k < j
m[i, k] m[k+1,j]

+ Assume that the optimal parenthesization splits the product Ai Ai+1  Aj at k
(i  k < j)
m[i, j] = m[i, k] + m[k+1, j] + pi-1pkpj
min # of multiplications min # of multiplications # of multiplications
to compute Ai…k to compute Ak+1…j to compute Ai…kAk…j
Step 2: A recursive solution

• Define m[i, j]= minimum number of

scalar multiplications needed to compute
the matrix Ai.. j  Ai Ai 1 .. A j
• goal m[1, n]
• m[ i , j ] 
 0 i j

min ik  j {m[i, k ]  m[k  1, j ]  pi 1 pk p j } i  j
Step 3:Computing the Optimal Costs
0 if i = j
m[i, j] = min {m[i, k] + m[k+1, j] + pi-1pkpj} if i < j
ik<j

2 3 n
n
m[1, n] gives the optimal
solution to the problem

j
Compute rows from bottom to 3
top and from left to right 2
1
i
 Example min {m[i, k] + m[k+1, j] + pi-1pkpj}
1 2 3
Compute A1  A2  A3
2 2
3 0
+ A1: 10 x 100 (p0 x p1) 7500 25000
2 1
+ A 2: 100 x 5 (p1 x p 2) 0
5000
+ A 3: 5 x 50 (p2 x p 3) 1
0
m[i, i] = 0 for i = 1, 2, 3
m[1, 2] = m[1, 1] + m[2, 2] + p0p1p2 (A1A2)
= 0 + 0 + 10 *100* 5 = 5,000
m[2, 3] = m[2, 2] + m[3, 3] + p1p2p3 (A2A3)
= 0 + 0 + 100 * 5 * 50 = 25,000
m[1, 3]=min m[1, 1] + m[2, 3] + p0p1p3 = 75,000 (A1(A2A3))
m[1, 2] + m[3, 3] + p0p2p3 = 7,500 ((A1A2)A3)
26
MATRIX_CHAIN_ORDER
MATRIX_CHAIN_ORDER(p)
1 n  length[p] –1
2 for i  1 to n
3 do m[i, i]  0
4 for l  2 to n
5 do for i  1 to n – l + 1
6 do j  i + l – 1
7 m[i, j]  
8 for k  i to j – 1
9 do q  m[i, k] + m[k+1, j]+ pi-1pkpj
10 if q < m[i, j]
11 then m[i, j]  q
12 s[i, j]  k
13 return m and s
3
O( n )
Complexity:
Example:
A1 30  35  p0  p1
A2 35  15  p1  p2
A3 15  5  p2  p3
A4 5  10  p3  p4
A5 10  20  p4  p5
A6 20  25  p5  p6
the m and s table computed by
MATRIX-CHAIN-ORDER for n=6
m[2,5]=
min{
m[2,2]+m[3,5]+p1p2p5=0+2500+351520=13000,
m[2,3]+m[4,5]+p1p3p5=2625+1000+35520=7125,
m[2,4]+m[5,5]+p1p4p5=4375+0+351020=11374
}
=7125
Step 4: Constructing an
optimal solution
MATRIX_CHAIN_MULTIPLY
MATRIX_CHAIN_MULTIPLY(A, s, i, j)
1 if j > i
2 then x  MCM ( A, s, i , s[ i , j ])
3 y  MCM ( A, s, s[ i , j ]  1, j )
4 return MATRIX-MULTIPLY(X,Y)
5 else return Ai

• example: (( A1( A2 A3 ))(( A4 A5 ) A6 ))