Positive Definite Matrices (正定矩陣)

Recall: 在微積分中，檢驗函數 f : [a, b] ⟶ ℝ 的 critical points 是否為 maximum/minimum

常用的方法有兩種：一階導數檢驗法與二階導數檢驗法‧
二階導數檢驗法：Suppose f ∈ C 2 (twice differentiable and f′′ is continuous) and f′(c) = 0,
(i) If f′′(c) < 0, then f has a local maximum at x = c;
(ii) If f′′(c) > 0, then f has a local minimum at x = c;

但在多變數函數的情況該怎麼辦？例如 f : ℝn ⟶ ℝ, f ∈ C 2. 怎麼去找出及檢驗 f 的
maximum/minimum？在考慮多變數函數時，顯然一階導數檢驗法無法使用，因此只能修正
二階導數檢驗法：
Step 1: Solve
∂f ∂f ∂f
(x) = (x) = ⋯ = (x) = 0,
∂x1 ∂x 2 ∂xn
Find the critical points.
Step 2: 利用 positive/negative definite matrices 來檢驗 maximum/minimum‧

定義：

Definition 1: A = (aij) ∈ Mn×n(ℝ).

1≤i, j≤n

(1) A is called positive definite (正定) if for every nonzero vector v ∈ ℝn,
n
v T Av =
∑
aij vi vj > 0.
i, j=1

(2) A is called negative definite (負定) if for every nonzero vector v ∈ ℝn,
n
T
∑
v Av = aij vi vj < 0.
i, j=1

(3) A is called positive semidefinite (半正定) if for every vector v ∈ ℝn,

n
v T Av =
∑
aij vi vj ≥ 0.
i, j=1

(4) A is called negative semidefinite (半負定) if for every vector v ∈ ℝn,

n
T
∑
v Av = aij vi vj ≤ 0.
i, j=1

(1 2)
2 1
Example 2: (1) A = . Then







 (1 2) (v2)
2 1 v1
v T Av = (v1 v2) = 2v12 + 2v1v2 + 2v22

( 2 )
2
1 3
= 2 v1 + v2 + v22 ≥ 0.
2

and v T Av = 0 ⟺ v1 = v2 = 0. This implies that A is positive definite.

(−1 1 )
1 −1
(2) A = . Then

(−1 1 ) (v2)
1 −1 v1
v T Av = (v1 v2) = v12 − 2v1v2 + v22

= (v1 − v2 )2 ≥ 0.
and v T Av = 0 ⟺ v1 = v2. This implies that A is positive semidefinite.

但用這個定義似乎太複雜，有沒有快一點的方法？底下介紹兩種檢驗的方法：固有值
(eigenvalue) 法與行列式 (determinant) 法‧

固有值法：

Definition 3: Given A ∈ Mn×n(ℝ). λ is an eigenvalue of A if det(A − λ In ) = 0.

Theorem 4: Suppose A is symmetric. Then

(1) A is positive definite if and only if all eigenvalues of A is positive;
(2) A is positive semidefinite if and only if all eigenvalues of A is nonnegative;
(3) A is negative definite if and only if all eigenvalues of A is negative;
(4) A is negative semidefinite if and only if all eigenvalues of A is nonpositive.

(1 2)
2 1
Example 5: (1) A = . Then

( 1 2 − λ)
2−λ 1
det(A − λ I2 ) = det = λ 2 − 4λ + 3 = (λ − 1)(λ − 3).

A has eigenvalues 1 and 3. Thus, A is positive definite.

(−1 1 )
1 −1
(2) A = . Then

( −1 1 − λ)
1 − λ −1
det(A − λ I2 ) = det = λ 2 − 2λ = λ(λ − 2).

A has eigenvalues 0 and 2. Thus, A is positive semidefinite.

行列式法（正(負)定）：

Theorem 6: Suppose A is symmetric. Then A is positive definite if and only if all of the upper
left submatrices of A have positive determinant, i.e.,
Δ1 = det (a11) > 0,

( 21 a 22)
a11 a12
Δ2 = det a > 0,

⋮
a11 a12 ⋯ a1n
a 21 a 22 ⋯ a 2n
Δn = det > 0.
⋮ ⋮ ⋱ ⋮
an1 an2 ⋯ ann

Theorem 7: Suppose A is symmetric. Then A is negative definite if and only if the upper left
submatrices of A satisfy
a11 a12 ⋯ a1n
a 21 a 22 ⋯ a 2n
(−1)k Δk = (−1)k det > 0,
⋮ ⋮ ⋱ ⋮
ak1 ak 2 ⋯ akk

即行列式必須為負正負正負正... 的形式 (Δ1 < 0, Δ2 > 0, Δ3 < 0, Δ4 > 0, ...)‧

行列式法（半正(負)定）：

Notation 8: Consider A = (aij) ∈ Mn×n(ℝ), for 1 ≤ k ≤ n, 1 ≤ i1 < i2 < ⋯ < ik ≤ n, denote

1≤i, j≤n

ai1i1 ai1i2 ⋯ ai1ik

ai2 i1 ai2 i2 ⋯ ai2 ik
Δi1,i2,...,ik = (−1)k det .
⋮ ⋮ ⋱ ⋮
aik i1 aik i2 ⋯ aik ik

Theorem 9: Suppose A is symmetric. Then

(1) A is positive semidefinite if and only if for all 1 ≤ k ≤ n, 1 ≤ i1 < i2 < ⋯ < ik ≤ n,
Δi1,i2,...,ik ≥ 0.
(2) A is negative semidefinite if and only if for all 1 ≤ k ≤ n, 1 ≤ i1 < i2 < ⋯ < ik ≤ n,
(−1)k Δi1,i2,...,ik ≥ 0.

但為什麼 semidefinite 會特別複雜？舉個例子來看就知道了‧

 1 0 0
(0 0 1)
Example 10: A = 0 0 0 . Then

Δ1 = 1 > 0, Δ2 = 0, Δ3 = 0.
但 A 的 eigenvalues 為 1, 0, −1，是個 indefinite matrix.

例題：

底下用上述介紹的三個方法來檢驗矩陣的正負定：

2 −1 0
( 0 −1 2 )
Example 11: (1) A = −1 2 −1 . Then A is symmetric.

Method 1: 定義
v T Av = 2v12 + 2v22 + 2v32 − 2v1v2 − 2v2 v3
= (v1 − v2 )2 + (v2 − v3)2 + v12 + v32 ≥ 0.
and v T Av = 0 ⟺ v1 = v2 = v3 = 0. Then A is positive definite.
Method 2: 固有值法
det(A − λ I3) = − (λ − 2)(λ 2 − 4λ + 2).
The eigenvalues of A are 2, 2 + 2, 2 − 2 , all positive. Thus, A is positive definite.
Method 3: 行列式法
Δ1 = 2 > 0

(−1 2 )
2 −1
Δ2 = det = 3 > 0,

2 −1 0
( 0 −1 2 )
Δ3 = det −1 2 −1 = 4 > 0.

Then A is positive definite.

2 −1 −1
(−1 −1 2 )
(2) B = −1 2 −1 . Then B is symmetric.

Method 1: 定義
v T B v = 2v12 + 2v22 + 2v32 − 2v1v2 − 2v2 v3 − 2v1v3
= (v1 − v2 )2 + (v2 − v3)2 + (v1 − v3)2 ≥ 0.
and v T B v = 0 ⟺ v1 = v2 = v3. Then B is positive semidefinite.
Method 2: 固有值法
det(B − λ I3) = − λ(λ − 3)2.
The eigenvalues of A are 0, 3, 3, all nonnegative. Thus, A is positive semidefinite.
 Method 3: 行列式法
Δ1 = Δ2 = Δ3 = 2 > 0
Δ1,2 = Δ1,3 = Δ2,3 = 3 > 0,
2 −1 −1
(−1 −1 2 )
1,2,3
Δ = det −1 2 −1 = 0 = 0.

Then B is positive semidefinite.

二階導數檢驗法：

Theorem 12: Suppose f : ℝn ⟶ ℝ with f ∈ C 2. c ∈ ℝn satisfies

∂f ∂f ∂f
(c) = (c) = ⋯ = (c) = 0.
∂x1 ∂x 2 ∂xn
Let
∂2 f ∂2 f ∂2 f
2
(x) ∂x1∂x 2
(x) ⋯ ∂x1∂x n
(x)
∂x1

∂2 f ∂2 f ∂2 f
2 (x) (x) ⋯ (x)
D f (x) = ∂x 2 ∂x1 2
∂x 2 ∂x 2 ∂x n

⋮ ⋮ ⋱ ⋮
∂2 f ∂2 f ∂2 f
∂x n ∂x1
(x) ∂x n ∂x 2
(x) ⋯ (x)
∂x n2

Then
(1) If D 2 f (c) is positive definite, then f has a local minimum at c;
(2) If D 2 f (c) is negative definite, then f has a local maximum at c;
(3) If f has a local minimum at c, then D 2 f (c) is positive semidefinite;
(4) If f has a local maximum at c, then D 2 f (c) is negative semidefinite.

Example 13: f (x, y) = x 4 + y 4 − 4x y + 1. Then f has three critical points (1,1), (0,0), (−1, − 1).
Consider

( −4 12y 2)
12x 2 −4
D 2 f (x, y) = .

(−4 12 )
12 −4
(i) (1,1)：For D 2 f (1,1) = ,

Δ1 = 12 > 0, Δ2 = 128 > 0.

Then f (1,1) = − 1 is a local minimum.
 (−4 0 )
0 −4
(ii) (0,0)：For D 2 f (0,0) = ,

Δ1 = 0, Δ2 = − 16 < 0.
Then f (0,0) 不是 maximum 也不是 minimum.

(−4 12 )
12 −4
(iii) (−1, − 1)：For D 2 f (−1, − 1) = ,

Δ1 = 12 > 0, Δ2 = 128 > 0.

Then f (−1, − 1) = − 1 is a local minimum.