APPENDIX C

DIFFERENTIATION WITH
RESPECT TO A VECTOR
The rst derivative of a scalar-valued function f (x) with respect to a vector
x = [x
1
x
2
]
T
is called the gradient of f (x) and dened as
f (x) =
d
dx
f (x) =

f/x
1
f/x
2

(C.1)
Based on this denition, we can write the following equation.

x
x
T
y =

x
y
T
x =

x
(x
1
y
1
+x
2
y
2
) =

y
1
y
2

= y (C.2)

x
x
T
x =

x
(x
2
1
+x
2
2
) = 2

x
1
x
2

= 2x (C.3)
Also with an M N matrix A, we have

x
x
T
Ay =

x
y
T
A
T
x = Ay (C.4a)

x
y
T
Ax =

x
x
T
A
T
y = A
T
y (C.4b)
where
x
T
Ay =
M

m=1
N

n=1
a
mn
x
m
y
n
(C.5)
Applied Numerical Methods Using MATLAB

, by Yang, Cao, Chung, and Morris

Copyr ight 2005 John Wiley & Sons, I nc., ISBN 0-471-69833-4
471
472 DIFFERENTIATION WITH RESPECT TO A VECTOR
Especially for a square, symmetric matrix A with M = N, we have

x
x
T
Ax = (A +A
T
)x
if A is symmetric
2Ax (C.6)
The second derivative of a scalar function f (x) with respect to a vector x =
[x
1
x
2
]
T
is called the Hessian of f (x) and is dened as
H(x) =
2
f (x) =
d
2
dx
2
f (x) =

2
f/x
2
1

2
f/x
1
x
2

2
f/x
2
x
1

2
f/x
2
2

(C.7)
Based on this denition, we can write the following equation:
d
2
dx
2
x
T
Ax = A +A
T
if A is symmetric
2A (C.8)
On the other hand, the rst derivative of a vector-valued function f(x) with
respect to a vector x = [x
1
x
2
]
T
is called the Jacobian of f (x) and is dened as
J(x) =
d
dx
f(x) =

f
1
/x
1
f
1
/x
2
f
2
/x
1
f
2
/x
2

(C.9)