Professional Documents
Culture Documents
Introduction
Throughout this presentation I have chosen to use a symbolic matrix notation. This choice
was not made lightly. I am a strong advocate of index notation, when appropriate. For
example, index notation greatly simplifies the presentation and manipulation of differential
geometry. As a rule-of-thumb, if your work is going to primarily involve differentiation
with respect to the spatial coordinates, then index notation is almost surely the appropriate
choice.
In the present case, however, I will be manipulating large systems of equations in which
the matrix calculus is relatively simply while the matrix algebra and matrix arithmetic is
messy and more involved. Thus, I have chosen to use symbolic notation.
.
.
.
CE 8361
Spring 2006
Convention 2
When it is useful to explicitly attach the matrix dimensions to the symbolic notation, I will
use an underscript. For example, A , indicates a known, multi-column matrix with m rows
mn
and n columns.
A superscript T denotes the matrix transpose operation; for example, AT denotes the
transpose of A. Similarly, if A has an inverse it will be denoted by A1 . The determinant
of A will be denoted by either |A| or det(A). Similarly, the rank of a matrix A is denoted by
rank(A). An identity matrix will be denoted by I, and 0 will denote a null matrix.
Matrix Multiplication
(3)
n
X
(4)
aik bkj
k=1
for all i = 1, 2, . . . , m,
j = 1, 2, . . . , p.
n
X
(5)
(6)
aik xk
k=1
for all i = 1, 2, . . . , m. Similarly, let y be m 1, then the typical element of the product
zT = yT A
is given by
zi =
n
X
aki yk
(7)
(8)
k=1
m X
n
X
ajk yj xk
(9)
(10)
j=1 k=1
CE 8361
Spring 2006
(11)
CT = BT AT
(12)
then
Proof: The typical element of C is given by
cij =
n
X
aik bkj
(13)
k=1
n
X
ajk bki
(14)
k=1
Hence,
CT = BT AT
(15)
q.e.d.
Proposition 3 Let A and B be n n and invertible matrices. Let the product AB be given
by
C = AB
(16)
then
C1 = B1 A1
(17)
CB1 A1 = ABB1 A1 = I
(18)
Proof:
q.e.d.
Partioned Matrices
Frequently, I will find it convenient to deal with partitioned matrices 1 . Such a representation,
and the manipulation of this representation, are two of the relative advantages of the symbolic
matrix notation.
Definition 4 Let A be m n and write
"
A=
B C
D E
(19)
Much of the material in this section is extracted directly from Dhrymes (1978, Section 2.7).
CE 8361
Spring 2006
(20)
A
A
A
A
A
A
A
A
A
11
12
21
12
22
21
12
22
11
11
1
1
A1 =
(21)
1
1
A1
A
A
A
A
A
A
A
A
A
21
11
12 22
21
22
21 11
12
22
Proof: Direct multiplication of the proposed A1 and A yields
A1 A = I
(22)
q.e.d.
Matrix Differentiation
In the following discussion I will differentiate matrix quantities with respect to the elements
of the referenced matrices. Although no new concept is required to carry out such operations,
the element-by-element calculations involve cumbersome manipulations and, thus, it is useful
to derive the necessary results and have them readily available 2 .
Convention 3
Let
y = (x),
where y is an m-element vector, and x is an n-element
y
y1
1
x1
x
y2 y22
y
x1
x2
=
.
.
..
x ..
ym
x1
ym
x2
(23)
vector. The symbol
y
1
xn
y2
xn
..
.
ym
xn
(24)
will denote the m n matrix of first-order partial derivatives of the transformation from x
to y. Such a matrix is called the Jacobian matrix of the transformation ().
Notice that if x is actually a scalar in Convention 3 then the resulting Jacobian matrix
is a m 1 matrix; that is, a single column (a vector). On the other hand, if y is actually a
scalar in Convention 3 then the resulting Jacobian matrix is a 1 n matrix; that is, a single
row (the transpose of a vector).
Proposition 5 Let
y = Ax
(25)
Much of the material in this section is extracted directly from Dhrymes (1978, Section 4.3). The
interested reader is directed to this worthy reference to find additional results.
CE 8361
Spring 2006
(26)
n
X
aik xk
(27)
k=1
it follows that
yi
= aij
xj
for all i = 1, 2, . . . , m,
(28)
j = 1, 2, . . . , n. Hence
y
=A
x
(29)
y = Ax
(30)
q.e.d.
Proposition 6 Let
where y is m 1, x is n 1, A is m n, and A does not depend on x, as in Proposition 5.
Suppose that x is a function of the vector z, while A is independent of z. Then
x
y
=A
z
z
(31)
n
X
aik xk
(32)
k=1
(33)
but the right hand side of the above is simply element (i, j) of A x
. Hence
z
y
y x
x
=
=A
z
x z
z
(34)
q.e.d.
Proposition 7 Let the scalar be defined by
= yT Ax
(35)
= yT A
x
(36)
5
CE 8361
Spring 2006
and
= xT AT
y
(37)
wT = yT A
(38)
= wT x
(39)
= wT = yT A
x
(40)
Proof: Define
and note that
Hence, by Proposition 5 we have that
(41)
= xT AT
y
(42)
q.e.d.
Proposition 8 For the special case in which the scalar is given by the quadratic form
= xT Ax
(43)
= xT A + AT
x
Proof: By definition
=
n X
n
X
aij xi xj
(44)
(45)
j=1 i=1
=
akj xj +
aik xi
xk
j=1
i=1
n
(46)
= xT AT + xT A = xT AT + A
x
(47)
q.e.d.
CE 8361
Spring 2006
(48)
= 2xT A
x
(49)
(50)
y
x
= xT
+ yT
z
z
z
Proof: We have
=
n
X
xj yj
(51)
(52)
j=1
xj
=
+ yj
zk
zk
zk
j=1
(53)
y x
y
x
=
+
= xT
+ yT
z
y z x z
z
z
(54)
q.e.d.
Proposition 11 Let the scalar be defined by
= xT x
(55)
x
= 2xT
z
z
(56)
(57)
= xT AT
+ yT A
z
z
z
(58)
7
CE 8361
Spring 2006
Proof: Define
wT = yT A
(59)
= wT x
(60)
w
x
= xT
+ wT
z
z
z
(61)
y
=
+
= x T AT
+ yT A
z
y z x z
z
z
(62)
q.e.d.
Proposition 13 Let the scalar be defined by the quadratic form
= xT Ax
(63)
= xT A + AT
(64)
z
z
Proof: This is an obvious application of Proposition 12. q.e.d.
Proposition 14 For the special case where A is a symmetric matrix and
= xT Ax
(65)
= 2xT A
(66)
z
z
Proof: This is an obvious application of Proposition 13. q.e.d.
Definition 5 Let A be a mn matrix whose elements are functions of the scalar parameter
. Then the derivative of the matrix A with respect to the scalar parameter is the m n
matrix of element-by-element derivatives:
a
a12
11
1n
a
a21 a22
2n
a
A
(67)
= .
.
.
..
..
..
am1
am2
amn
CE 8361
Spring 2006
(69)
(70)
A1
A 1
= A1
A
(71)
A1
rearranging the terms yields
q.e.d.
References
Dhrymes, Phoebus J., 1978, Mathematics for Econometrics, Springer-Verlag, New
York, 136 pp.
Golub, Gene H., and Charles F. Van Loan, 1983, Matrix Computations, Johns
Hopkins University Press, Baltimore, Maryland, 476 pp.
Graybill, Franklin A., 1983, Matrices with Applications in Statistics, 2nd Edition,
Wadsworth International Group, Belmont, California, 461 pp.