You are on page 1of 7

Jim Lambers

MAT 419/519
Summer Session 2011-12
Lecture 3 Notes
These notes correspond to Section 1.3 in the text.

Positive and Negative Definite Matrices and Optimization
The following examples illustrate that in general, it cannot easily be determined whether a symmetric matrix is positive definite from inspection of the entries.
Example Consider the matrix 

A=

1 4
4 1 

.

Then
QA (x, y) = x2 + y 2 + 8xy
and we have
QA (1, −1) = 12 + (−1)2 + 8(1)(−1) = 1 + 1 − 8 = −6 < 0.
Therefore, even though all of the entries of A are positive, A is not positive definite. 2
Example Consider the matrix 

A=

1 −1
−1
4 

.

Then
QA (x, y) = x2 + 4y 2 − 2xy = x2 − 2xy + y 2 + 3y 2 = (x − y)2 + 3y 2
which can be seen to be always nonnegative. Furthermore, QA (x, y) = 0 if and only if x = y and
y = 0, so for all nonzero vectors (x, y), QA (x, y) > 0 and A is positive definite, even though A does
not have all positive entries. 2
Example Consider the diagonal matrix


1 0 0
A =  0 3 0 .
0 0 2
Then
QA (x, y, z) = x2 + 3y 2 + 2z 2 ,
which can plainly be seen to be positive except when (x, y, z) = (0, 0, 0). Therefore, A is positive
definite. 2
1

y) = at2 y 2 + 2bty 2 + cy 2 = (at2 + 2bt + c)y 2 . .  d2 . then we use a change of variable x = ty and obtain QA (x. n. 5. . . If y 6= 0. .. .. indefinite if and only if di > 0 for some indices i. . . ϕ00 (t) = 2a. . negative semidefinite if and only if di ≤ 0 for i = 1. . positive definite if and only if di > 0 for i = 1.  . 0  · · · 0 dn then A is: 1. 0) = ax2 . a a We conclude that A is positive definite if and only if a > 0 and det(A) > 0. The minimum value is ϕ(t∗ ) = a(−b/a)2 + 2b(−b/a) + c = c − b2 1 = det(A).. n. . If y = 0. 2. positive semidefinite if and only if di ≥ 0 for i = 1. then we have QA (x. This leads to the following theorem. n. . . negative definite if and only if di < 0 for i = 1. We now consider a general 2 × 2 symmetric matrix   a b . 2. . . 2. 3. . Theorem A 2 × 2 symmetric matrix A is 2 . 4. we obtain the single critical point t∗ = −b/a and determine that it is a strict global minimizer of ϕ(t). Thus QA (x.  . 2. so we must certainly have a > 0 in order for A to be positive definite. y) is positive definite if and only if ϕ(t) = at2 + 2bt + c is positive. and negative for other indices. 0 as follows: if A is an n × n diagonal matrix  0 ··· 0 . .The preceding example can be generalized  d1   0 A=  . 2. y) = ax2 + 2bxy + cy 2 . . A= b c This matrix induces the quadratic form QA (x.  . n. 1 ≤ i ≤ n. From ϕ0 (t) = 2at + 2b. y) = QA (ty.

Example Let f (x. and let Ak be the submatrix of A obtained by taking the upper left-hand corner k × k submatrix of A. A is positive definite if and only if ∆k > 0 for k = 1. the kth principal minor of A. combined with mathematical induction. 2. A is negative definite if and only if (−1)k ∆k > 0 for k = 1. indefinite if and only if det(A) < 0 A similar argument. 2. Note that the last two statements in the theorem are not “if and only if” statements. . negative definite if and only if a < 0 and det(A) > 0 3. . We have ∇f (x. 2. A is negative semidefinite if (−1)k ∆k > 0 for k = 1. it is actually indefinite. 1. if   1 1 1 A =  1 1 1 . 2. let ∆k = det(Ak ). then A is not necessarily positive semidefinite. 2z + y − x). Furthermore. n. 2. 1. 2y − x + z. n − 1 and ∆n = 0. . . n − 1 and ∆n = 0. 0. 3. positive definite if and only if a > 0 and det(A) > 0 2.1. 0) yields only the trivial solution (0. z) = (0. . Then 1. leads to the following generalization. . For example. . −2) < 0. . z) =  −1 −1 1 2 3 . We also have   2 −1 −1 2 1 . . A is positive semidefinite if ∆k > 0 for k = 1. . . 0). . Theorem Let A be an n × n symmetric matrix. z) = x2 + y 2 + z 2 − xy + yz − xz. if A has nonnegative principal minors. . but (1. . y. 4. z) = (2x − y − z. . 0. −2) · A(1. Furthermore. We now show how the preceding discussion can be applied to find minimizers and maximizers of functions. so A is not positive semidefinite. . y. y. Hf (x. y. Solving ∇f (x. 1 1 21 All of the principal minors are nonnegative. n.

0) is the only critical point. y) = . ∆2 = 2. x = y and x = 0 for a critical point. It can also be shown by direct computation of the minors that Hf (x. That is. using the fact that ex > 0 for all x. 0. 2 Example Let 2 f (x. 0 0 2 and therefore   3 −2 0 2 0 . Therefore. We have ∇f (x. (0. 4 . −ex−y + ey−x ). y. z). z) = ex−y + ey−x + ex + z 2 . −ex−y + ey−x . y). which yields the critical points (x. y) = ex−y + ey−x . z) = (ex−y − ey−x + 2xex . 0. y) = (ex−y − ey−x . We have 2 ∇f (x. 2 Example Let f (x. x) for all x ∈ R. Therefore. ∆2 = (ex−y + ey−x )2 − (−ex−y − ey−x )2 = 0. z). z) is positive definite on all of R3 . Hf (x. y. y. 0) is a strict global minimizer of f (x. We then have  x−y  2 2 e + ey−x + 4x2 ex + 2ex −ex−y − ey−x 0 Hf (x. y. 0) is a strict global minimizer of f (x. Hf (0. 0. x) is positive semidefinite. 2 Example Let f (x.which can be confirmed to be positive definite by examinination of the principal minors: ∆1 = 2. −ex−y − ey−x ex−y + ey−x which yields ∆1 = ex−y + ey−x > 0. and ∆3 = 4. y. ∆3 = 4. in view of ∆1 = 3. 0. 0) =  −2 0 0 2 which is positive dfinite. the critical point (0. 2z) which yields the conditions z = 0. x) a global minimizer of f (x. We also have   ex−y + ey−x −ex−y − ey−x Hf (x. It follows that the critical point (0. ∆2 = 3. making (x. y) = ex−y + ex+y . z) =  −ex−y − ey−x ex−y + ey−x 0  . y.

W (t) = f (x∗ + tw). We now consider the implications of an indefinite Hessian at a critical point. Therefore. If we define Y (t) = f (x∗ + ty). Therefore. strict local maximizer of f (x) if Hf (x∗ ) is negative definite. and a strict local maximizer along a curve contained in the graph of f that passes through x∗ in the direction of w. there are no global minimizers. By continuity of the second partial derivatives. 5 . which leads to the following definition. Definition A saddle point for f (x) is a critical point x∗ for f (x) such that there are vectors y.We have ∇f (x. strict local minimizer of f (x) if Hf (x∗ ) is positive definite. If Hf (x∗ ) is indefinite. w · Hf (x∗ )w < 0. 2. w · Hf (x∗ + tw)w < 0 for |t| < . t = 0 is a strict local minimizer of Y (t) and a strict local maximizer of W (t). w for which t = 0 is a strict local minimizer for Y (t) = f (x∗ + ty) and a strict local maximizer for W (t) = f (x∗ + tw). This theorem can be proved by using the continuity of the second partial derivatives to show that Hf (x) is positive definite for x sufficiently close to x∗ . Then x∗ is a: 1. The result of the preceding discussion is summarized in the following theorem. 2 We now consider how the Hessian can be used to establish the existence of a local minimizer or maximizer. Let x∗ be an interior point of D that is a critical point of f (x). and then applying the multi-variable generalization of Taylor’s Formula. Suppose that f (x) has continuous second partial derivatives on a set D ⊆ Rn . then there exist vectors y. Furthermore. It follows that x∗ is a strict local minimizer along a curve contained in the graph of f that passes through x∗ in the direction of y. which yields no critical points. y) = (ex−y + ex+y . then Y 0 (0) = W 0 (0) = 0. while Y 00 (0) > 0 and W 00 (0) < 0. Theorem Suppose that f (x) has continuous first and second partial derivatives on a set D ⊆ Rn . there exists an  > 0 such that y · Hf (x∗ + ty)y > 0. −ex−y + ex+y ). This gives the graph of f the appearance of a saddle near x∗ . w such that y · Hf (x∗ )y > 0. let x∗ be an interior point of D that is a critical point of f (x). as ∂f /∂x can never be zero.

Furthermore. y) has no global minimizer on minimizer. because its principal minors are positive. Hf (0. We see that Hf (2. lim f (x. We also have   6x −12 Hf (x. which yields the critical point (0. 2 6 . 2 R2 . 0) is a strict global minimizer. y) = +∞. lim f (x. which is positive semidefinite. but Hf (0. but it can be seen from a graph that (0. so (0. 0). That is. as ∆1 = 0 and ∆2 = −144. Hf (2. 1) is positive definite. 0) is not. and decreases along the y-direction. 0) is the zero matrix. We have ∇f (x. y) increases from (0. 0) is the zero matrix. then x∗ is a saddle point of x∗ . We then have   12x2 0 Hf (x. 0 −12y 2 Therefore Hf (0. 0) is a saddle point. As in the previous example. 1) = 12 −12 −12 48  . maximizer or saddle point of the function. y) = . 0) is indefinite. Example Let f (x. 0) along the x-direction. y) = −∞. then it cannot be concluded that the critical point is necessarily a minimizer. We can conclude. 2 Example Let f (x.Theorem If f (x) is a function with continuous second partial derivatives on a set D ⊆ Rn . Hf (0. f (x. However. y) = x4 − y 4 . y) = (4x3 . 1). −12 48x and therefore  Hf (0. We have ∇f (x. so (0. y) = (3x2 − 12y. 0) = 0 −12 −12 0   . y) = . 1) is a strict local It should be emphasized that if the Hessian is positive semidefinite or negative semidefinite at a critical point. x→∞ x→−∞ so f (x. 0) is neither a local minimizer nor maximizer. y) = x3 − 12xy + 8y 3 . −12x + 24y 2 ). −4y 3 ). which yields the critical points (0. if x∗ is an interior point of D that is also a critical point of f (x). 0) and (2. however. y) = x4 + y 4 . and if Hf (x∗ ) is indefinite. that (2. Example Let f (x.

Exercise 9 5. Chapter 1. Chapter 1. Exercise 2 2. Exercise 19 7 . Chapter 1. Chapter 1. Exercise 11 6. Exercise 7 4.Exercises 1. Chapter 1. Exercise 6 3. Chapter 1.