High Quality
Open the downloaded document, and select print from the file menu (PDF reader required).
University College, DublinMathematical Physics DepartmentMaster of Computational Science Degree 2004-2005
NUMERICAL ALGORITHMS
Dr Derek O’Connor
Laboratory Exercise No. 6:
Complexity of Matrix Operations.
The standard mathematical definition of matrix multiplication,
C
=
AB
, where
A
is
m
×
n
,
B
is
n
×
p
, and
C
is
m
×
p
is as follows :
c
ij
=
n
k
=1
a
ik
×
b
kj
, i
= 1
,
2
,...,m, j
= 1
,
2
,...,p.
(1)Do the following1. Using this definition, derive an expression,
N
ops
(
m,n,p
), for the number of floating-point
additions
and
multiplications
needed to form the matrix product
C
=
AB
,2. Derive a similar expression
N
ops
(
m,n,p,q
) for
D
=
ABC
, where
A
is
m
×
n
,
B
is
n
×
p
, and
C
is
p
×
q
, and
D
is
m
×
q.
Although matrix multiplication is associative, i.e.,
A
(
BC
) = (
AB
)
C,
show that
N
ops
(
A
(
BC
))
=
N
ops
((
AB
)
C
)
,
in general
.
(2)That is,
N
ops
(
m,n,p,q
) depends on the order in which the product
ABC
is formed or
parenthesized
.3. Write a
Matlab
function
C = function MatMult(A,B)
that implements the definition above.Test and compare this function with
Matlab
’s
C = A*B
for random square matrices of size
n
= 250
,
500
,
1000
.
4. Find 3 sets of values
{
m,n,p,q
}
that demonstrate clearly that the inequality (2) above is truein general. Use both your function
MatMult
and
Matlab
’s
C = A*B
in this demonstration andcompare the results.5. Calculate the
mflops/sec
for each of the tests above. Remember to give the machine parameterswith these rates.
note:
When timing the operations above use
Matlab
’s
cputime
. Here is the
help
for this function :
CPUTIME returns the CPU time in seconds that has been usedby the MATLAB process since MATLAB started.For example:t=cputime; your_operation; cputime-treturns the cpu time used to run your_operation.The return value may overflow the internal representationand wrap around.See also ETIME, TIC, TOC, CLOCK
derek o’connor – October 10, 2008.
Lab. Exer. 6. Complexity of Matrix Operations 6.2
Solution Notes for Lab. Exercise No. 6.Analysis
Standard matrix multiplication,
C
=
AB
, where
A
is
m
×
n
,
B
is
n
×
p
, and
C
is
m
×
p
is as follows :
c
ij
=
n
k
=1
a
ik
×
b
kj
, i
= 1
,
2
,...,m, j
= 1
,
2
,...,p.
There are
m
×
p
elements
c
ij
and each requires the summation
a
i
1
×
b
1
j
+
a
i
2
×
b
2
j
+
···
+
a
in
×
b
nj
,
whichrequires
n
mults and
n
−
1 adds. Hence we get a total of 2
mnp
−
mp
=
O
(
mnp
) operations.The matrix triple multiplication operation
D
=
ABC
, where
A
is
m
×
n
,
B
is
n
×
p
, and
C
is
p
×
q,
and
D
is
m
×
q,
is defined in terms of the matrix pair multiplication above. This gives two possible orders of multipliction :
D
1
= (
AB
)
C
or
D
2
=
A
(
BC
)
.
Mathematically,
D
1
and
D
2
are identical, but computationally they are not. Using
O
(
mnp
) for matrix pairmultiplication we have
N
ops
(
A
(
BC
)) =
N
ops
(
R
=
BC
)) +
N
ops
(
D
=
AR
) =
O
(
npq
) +
O
(
mnq
)
N
ops
((
AB
)
C
) =
N
ops
(
R
=
AB
)) +
N
ops
(
D
=
RC
) =
O
(
mnp
) +
O
(
mpq
)We will drop the
O
(
−
) formalism and simple say that
N
ops
(
A
(
BC
)) =
N
1
=
npq
+
mnq
and
N
ops
((
AB
)
C
) =
N
2
=
mnp
+
mpq.
It is very difficult to say in general when these two functions have different or equal values. A crude way of getting some idea is to run this program
%=============== [kl,ke,kg, Equal] = MNPQ(low, high)===============%% When do R1 = A*(B*C) and R2 = (A*B)*C have different op counts?% N1 = npq+mnq N2 = mnp+mpq% The matrices A,B,C are m*n, n*p, p*q.%===================================================================%function [kl,ke,kg, Equal] = MNPQ(low, high)%===================================================================%Less = zeros((high-low+1)^4,4); Equal = zeros((high-low+1)^4,4);Great = zeros((high-low+1)^4,4); kl = 0; ke = 0; kg = 0;for m = low:highfor n = low:highfor p = low:highfor q = low:highN1 = n*p*q + m*n*q;N2 = m*n*p + m*p*q;if N1 < N2kl = kl + 1;Less(kl,:) = [m n p q];elseif N1 == N2ke = ke + 1;Equal(ke,:) = [m n p q];elsekg = kg + 1;Great(kg,:) = [m n p q];end;end;end;end;end;%-------------------------- End of MNPQ(low, high) ------------------%
derek o’connor – October 10, 2008.
Lab. Exer. 6. Complexity of Matrix Operations 6.3
Running this program for
low = 1
and
high = 5
gives a total of 5
4
= 625 4-tuples (
m,n,p,q
) of which 290give
N
1
< N
2
, 45 give
N
1
=
N
2
, and 290 give
N
1
> N
2
.
Here are the 45 for which
N
1
=
N
2
.
1 1 1 1 2 1 1 2 3 1 1 3 4 1 1 4 5 1 1 51 1 2 2 2 2 1 1 3 2 2 3 4 2 2 4 5 2 2 51 1 3 3 2 2 2 2 3 3 1 1 4 3 3 4 5 3 3 51 1 4 4 2 2 3 3 3 3 2 2 4 4 1 1 5 4 4 51 1 5 5 2 2 4 4 3 3 3 3 4 4 2 2 5 5 1 11 2 2 1 2 2 5 5 3 3 4 4 4 4 3 3 5 5 2 21 3 3 1 2 3 3 2 3 3 5 5 4 4 4 4 5 5 3 31 4 4 1 2 4 4 2 3 4 4 3 4 4 5 5 5 5 4 41 5 5 1 2 5 5 2 3 5 5 3 4 5 5 4 5 5 5 5
The main point here is that for most 4
−
tuples (
m,n,p,q
) we have
N
1
=
N
2
,
which prompts the question :How do we decide on
A
(
BC
) or (
AB
)
C
? The obvious answer is to calculate
N
1
and
N
2
before we do thecomputations.
Calculating
A
k
.
The na¨ıve calculation
R
=
A
×
(
A
×
(
··· ×
A
(
A
×
A
))
···
) requires (
k
−
1)
n
3
ops. If
k
= 2
p
there is a better way :
A
2
=
A
×
A, A
4
=
A
2
×
A
2
, ...,A
2
p
=
A
2
p
−
1
×
A
2
p
−
1
,
which requires
p
multiplications or
pn
3
= log
2
kn
3
ops. The
Matlab
program is trivial
R
=
A
;
for i
=
1
:
log2
(
k
)
,
R
=
R
∗
R
;
end
;The na¨ıve program is
R
=
A
;
for i
=
2
:
k
,
R
=
A
∗
R
;
end
;
Exercise
6.0.2 : Re-write the first program to handle the general case, i.e., when
k
is not a power of 2.
⊔⊓
Matrix-Chain Multiplication.
Calculating
A
1
A
2
···
A
k
where
A
i
is an
m
i
−
1
×
m
i
matrix is not an easy extension of the 3-matrix case.Consider
A
1
A
2
A
3
A
4
. This can be
parenthesized
in 5 different ways :((
A
1
A
2
)(
A
3
A
4
)) (
A
1
((
A
2
A
3
)
A
4
)) (
A
1
(
A
2
(
A
3
A
4
))) (((
A
1
A
2
)
A
3
)
A
4
)) ((
A
1
(
A
2
A
3
))
A
4
)
.
Let
C
(
k
) be the number of ways to parenthesize the matrix-chain
A
1
A
2
···
A
k
. Let us put the first parenthesesbetween
A
i
−
1
and
A
i
: (
A
1
A
2
...A
i
−
1
)(
A
i
...A
k
)
.
There are
C
(
i
) ways to parenthesize the left part and
C
(
k
−
i
) ways to parenthesize the right part. Now any parenthesization of the left part may be combinedwith any parenthesization of the right part and so there are
C
(
i
)
C
(
k
−
i
) ways of doing this. Now
i
can haveany value between 1 and
k
−
1. Hence we must sum
C
(
i
)
C
(
k
−
i
) for all
i
to get
C
(
k
) =
k
−
1
i
=1
C
(
i
)
C
(
k
−
i
)
.
The number of parenthesizations of the matrix chain
A
1
A
2
···
A
k
is the
Catalan Number
C
(
k
) =1
k
2
k
−
2
k
−
1
= Ω(4
k
/k
2
)
.
Catalan Numbers
n
1 2 3 4 5 6 7 8 9 10 15
C
(
k
) 1 1 2 5 14 42 132 429 1430 4862 2674440
derek o’connor – October 10, 2008.
Add a Comment