Lecture 12: Chain Matrix Multiplication

CLRS Section 15.2

Outline of this Lecture

Recalling matrix multiplication.

The chain matrix multiplication problem.

A dynamic programming algorithm for chain matrix multiplication.

Recalling Matrix Multiplication

Matrix: An



dimensional array


 









 

#

#



$#




matrix




..


..


%"

&

..




!


which has

rows and

-


"

$#

..


,
3


/

43

(
(

')(


' (




matrix:
,
1

.2/

1
,

 

columns.


Example: The following is a



"

"


is a two-

Recalling Matrix Multiplication

The product


matrix is a 

for






of a 
 matrix
 matrix given by


 

   




 




 and




and a 

 

 .

Example: If



' (

/
1


!


,



' (
.

*
,

then





43

43

.
+
0

+
3

43

')(


0
3

*
5

Remarks on Matrix Multiplication

If



is defined,

may not be defined.



Quite possible that



Multiplication is recursively defined by







 






  




 



 




Matrix multiplication is associative , e.g.,



















 


so parenthenization does not change result.

Direct Matrix multiplication



Given a 
 matrix
way of multiplying


for




 

 and

and a 
 matrix , the direct



is to compute each


 

   




 

 .

Complexity of Direct Matrix multiplication:

  

Note that has   entries and each entry takes



  
time to compute so the total procedure takes
time.





Direct Matrix multiplication of

Given a 
 matrix , a 
 matrix and a 




matrix , then
can be computed in two ways








and 
:
The number of multiplications needed are:



  





     

  
 





 
   

When 


,

,




,
+

 

 

and










.

, then


.
5

A big difference!
Implication: The multiplication sequence
(parenthesization) is important!!

The Chain Matrix Multiplication Problem

Given




dimensions 


555




corresponding to matrix sequence
,
,
, 
555
 





 ,
where
has dimension 
determine the multiplication sequence that minimizes
the number of scalar multiplications in computing



   . That is, determine how to parenthisize

the multiplications.

















     



     


Exhaustive search:

 





+





Question: Any better approach?















 



 
 

.
Yes DP







Developing a Dynamic Programming Algorithm

Step 1: Determine the structure of an optimal solution
(in this case, a parenthesization).
Decompose the problem into subproblems: For

  
each pair
, determine the multiplication
 


   
 


 that minimizes
sequence for
the number of multiplications.
Clearly,


 

is a 



matrix.

Original Problem: determine sequence of multiplica



tion for
 .

Developing a Dynamic Programming Algorithm

Step 1: Determine the structure of an optimal solution
(in this case, a parenthesization).





High-Level Parenthesization for
For any optimal multiplication sequence, at the last
 

 and   
 
step you are multiplying two matrices

for some . That is,



 





 

  




 





 



5

Example
 



Here

 






 





 



5


,

Developing a Dynamic Programming Algorithm

Step 1 Continued: Thus the problem of determining the optimal sequence of multiplications is broken
down into 2 questions:

How do we decide where to split the chain


(what is )?

(Search all possible values of )

How do we parenthesize the subchains

 

 and   
  ?
(Problem has optimal substructure property that
 

 and   
  must be optimal so we can apply the same procedure recursively)

10

Developing a Dynamic Programming Algorithm

Step 1 Continued:
Optimal Substructure Property: If final optimal so 

 

 involves splitting into
 and   
 
lution of
 

 and   
 
at final step then parenthesization of
in final optimal solution must also be optimal for the
subproblems standing alone:



 

If parenthisization of
was not optimal we could
replace it by a better parenthesization and get a cheaper
final solution, leading to a contradiction.


Similarly, if parenthisization of 
 was not optimal we could replace it by a better parenthesization
and get a cheaper final solution, also leading to a contradiction.


11

Developing a Dynamic Programming Algorithm

Step 2: Recursively define the value of an optimal
solution.
As with the 0-1 knapsack problem, we will store the
solutions to the subproblems in an array.



 "

For
, let 
denote the minimum
 


number of multiplications needed to compute
.
The optimum cost can be described by the following
recursive definition.

12

Developing a Dynamic Programming Algorithm

Step 2: Recursively define the value of an optimal
solution.
 

   
 


 
 


  
 
 

 



Proof: Any optimal sequence of multiplication for

is equivalent to some choice of splitting



 



 



 



for some , where the sequences of multiplications for
 

 and   
  also are optimal. Hence
   

  

 

 







 
5

13

Developing a Dynamic Programming Algorithm


Step 2 Continued: We know that, for some
   

  

 

 







 
5

We dont know what is, though



But, there are only !
possible values of so we
can check them all and find the one which returns a
smallest cost.

Therefore



 



  

 











 





14

Developing a Dynamic Programming Algorithm

Step 3: Compute the value of an optimal solution in a
bottom-up fashion.
  



 
Our Table:

. 


only defined for
.

The important point is that when we use the equation

  

to calculate
 
 

 

 

 

 







 

 

we must have already evaluated

and


For both cases, the
length of the
 

 corresponding

. Hence, the algorithm
matrix-chain are both less than
should fill the table in increasing order of the length of the matrixchain.
That is, we calculate in the order

 
  
  !  !  "# $%&

'   (
)**+, - &
  
)/.0*1
..
.

*2#  &

&
15

Dynamic Programming Design Warning!!

When designing a dynamic programming algorithm
there are two parts:
1. Finding an appropriate optimal substructure property and corresponding recurrence relation on table items. Example:

   


     
           
  


2. Filling in the table properly.

This requires finding an ordering of the table elements so that when a table item is calculated
using the recurrence relation, all the table values
needed by the recurrence relation have already
been calculated.
In our example this means that by the time 
   
is calculated all of the values 
and 
 
were already calculated.


 "


16



Example for the Bottom-Up Computation


Example: Given a chain of four matrices
,
 


 + ,
 1 ,  
, , 
and
, with 
  0
 

. Find 
.

+




S0: Initialization

m[i,j]

3
j

0

4

A1
p0

A2
p1

2

A3
p2

7

A4
p3

,
and

p4

17

Example Continued
Stp 1: Computing









By definition
 
 
&

 "

$














 

3
5

m[i,j]

3
j

120

1

0

4

A1
p0

A2
p1

2

A3
p2

7

A4
p3

p4

18

Example Continued
Stp 2: Computing

# 



$

By definition
  
 


$

 

# 




 




 


 


  

.
+
5

m[i,j]

3
j

120

1

48

0

4

A1
p0

A2
p1

2

A3
p2

7

A4
p3

p4

19

Example Continued
 #

Stp3: Computing


 #

 


 




By definition
  
 

+












+


  

.
+
5

m[i,j]

3
j

120

1

48

84

0

4

A1
p0

A2
p1

2

A3
p2

7

A4
p3

p4

20




Example Continued

 

Stp4: Computing

 



 

  "




By definition
 
 


 &



 #
$# 











 
 




 

 

5

m[i,j]

3
j

88

2

120

1

48

84

0

4

A1
p0

A2
p1

2

A3
p2

7

A4
p3

p4
21

Example Continued
#

Stp5: Computing


#

 

$

$#&




By definition
 
 

+



$# 


43
+

 #








+


   

 



  




5

m[i,j]

3
j

88

2

104

120

1

48

84

0

4

A1
p0

A2
p1

2

A3
p2

7

A4
p3

p4
22




Example Continued


St6: Computing



 









By definition
 
 




+

 


.
,











+




 #


+

#











   
  


  









5

m[i,j]

158

3
j

2

88

2

104

120

1

48

84

0

4

A1
p0

A2
p1

2

A3
p2

7

A4
p3

p4

We are done!
23




Developing a Dynamic Programming Algorithm

Step 4: Construct an optimal solution from computed
information extract the actual sequence.


%


 "

Idea: Maintain an array

, where
de5
5
5
5

 



notes for the optimal splitting in computing
 

 
 
  
  . The array  
can be used re5 5
5 5
cursively to recover the multiplication sequence.
How to Recover the Multiplication Sequence?

 

  
         




 
 





 
  




...








 

  
    
 
        

        


 
   

 
  
      

...

Do this recursively until the multiplication sequence is

determined.
24

Developing a Dynamic Programming Algorithm

Step 4: Construct an optimal solution from computed
information extract the actual sequence.
Example of Finding the Multiplication Sequence:

1
 
1 %
1 
Consider
. Assume that the array
5 5
5 5
has been computed. The multiplication sequence is
recovered as follows.

 1  
 

        

 

  

 
  





,

Hence the final multiplication sequence is








 
 










5

25

The Dynamic Programming Algorithm

 

Matrix-Chain(
 
 )
for ( 
to  )
for (

to  )

   

 



for (
to


  

 


;
 

  
   ; 

for (

to


if (




return



 






 
















 

 

 



 ;
;

and ; (Optimum in




Complexity: The loops are nested three deep.

Each loop index takes on

Hence the time complexity is

values.


 

. Space complexity


26

Constructing an Optimal Solution: Compute


 "

The actual multiplication code uses the

value to
determine how to split the current sequence. Assume
that the matrices are stored in an array of matrices

 

  
, and that
is global to this recursive pro5 5
cedure. The procedure returns a matrix.
Mult(

if (








is now



 

  

  

;
 , where  is
 

;

 


is now
  ; multiply matrices 
return

else return

To compute

 


 

and

;
 

 , call Mult(

%

).
27

Constructing an Optimal Solution: Compute

Example of Constructing an Optimal Solution:




Compute
.


. Assume
Consider the example
earlier,
where

1 
1 
that the array
has been computed. The
5 5
5 5
multiplication sequence is recovered as follows.
  

 

 
 

 
  

 


Mult   
   
   


  
 

 
  


Mult    
   
 
  
 

 

    

Mult   
   

   




 
    
  
  

     
Mult       
      

     

               


Mult


Hence the product is computed as follows







 
 










5

28