Applied Linear Algebra

Vectors and Quadratic

Forms 2
Overview

Students will learn about:

 Eigenvalues and Eigenvectors

 Methods of Finding of Eigenvalues
 Quadratic Forms
 Definiteness of a Quadratic Forms
 Quadratic Forms with Linear Constraints
 Eigenvalues and Eigenvectors
 Many applied problems, especially in dynamic economics, involve
the powers of a square matrix A.
 If the dimensions of A are very large and x is a given nonzero
vector, then computing or even worse, , is usually a
major problem.
 But suppose there happens to be a scalar λ, with the special
property that

 In this case, we would have

and in general,
 Many of the properties of A and can be deduced by finding
the pairs (λ,x) , that satisfy the equation above.
Eigenvalues and Eigenvectors

 A nonzero vector x that solves the equation above is called an

eigenvector, and the associated λ is called an eigenvalue.
 Zero solutions are not very interesting, of course, because
for every scalar λ.
 In optimization theory, in the theory of difference and differential
equations, in statistics, in population dynamics, and in many other
applications of mathematics, there are important arguments and
results based on eigenvalues.
Eigenvalues and Eigenvectors

EIGENVALUES AND EIGENVECTORS

 If A is an n x n matrix, then a scalar λ is an eigenvalue of A if
there is a nonzero vector x in R n such that Ax   x.
 Then x is an eigenvector of A (associated with λ).

 It should be noted that if x is an eigenvector associated with the

eigenvalue λ, then αx is another eigenvector for every scalar α≠0.
 Eigenvalues and eigenvectors are also called characteristic
roots (values) and characteristic vectors, respectively.
 How to Find Eigenvalues
 The eigenvalue equation can be written as ( A   I )  x  0,where I
denotes the identity matrix of order n.
 This homogeneous linear system of equations has a solution x≠0 if
the coefficient matrix has determinant equal to 0.
 Letting p ( )  A   I , we have the equation

 This is called the characteristic equation (or eigenvalue equation)

of A.
 How to Find Eigenvalues

 If the components of the vector x are then the system can be

written as

 An eigenvector associated with λ is a nontrivial solution of the

system.
 How to Find Eigenvalues

 EXAMPLE 12 Find the eigenvalues and the associated

eigenvectors of the matrices

 Solution: (a) The characteristic equation is

 This has solutions -2 and 3, which are the eigenvalues of A.

 How to Find Eigenvalues

 For the two equations of system both reduce to

 Choosing we have
 The eigenvectors associated with are, therefore,
 If we put t=- 3s, we can equivalently represent the eigenvectors as

 For λ=3, system implies that so the eigenvectors are

 1
x  s   , s  0.
 1
How to Find Eigenvalues

 The characteristic equation is

which has the complex roots

 The eigenvectors turn out to be

with non-zero s and t.

 How to Find Eigenvalues

 EXAMPLE 13 Find all the eigenvalues and those eigenvectors that

are associated with the real eigenvalues of the matrices

 Solution: (a) The characteristic equation is

which has λ = 1 as its only real root. (There are two complex
eigenvalues)
How to Find Eigenvalues

 The eigenvectors associated with λ= 1 satisfy the system,

which becomes

 This gives the eigenvectors

How to Find Eigenvalues

 (b) The characteristic equation is

 Thus, are the eigenvalues.

How to Find Eigenvalues

 For the eigenvectors are

 For the eigenvectors are all the vectors of the form

with s and t real, not both s and t equal to 0.

 Diagonalization

 We begin by noting a simple and useful result.

 Let A and P be n x n matrices with P invertible. Then

 This is true because the two matrices have the same

characteristic polynomial:

where we made use of rule for determinants, and the fact that
 Diagonalization
 An nxn matrix A is diagonalizable if there exists an invertible n x n
matrix P and a diagonal matrix D such that
 As we know the eigenvalues of a diagonal matrix are the diagonal
elements.
 Hence, if A is diagonalizable, then

are the eigenvalues of A.

Two questions arise:

 Which square matrices are diagonalizable?
 If A is diagonalizable, how do we find the matrix P ?
Diagonalization

THEOREM (DIAGONALIZABLE MATRICES)

 An n x n matrix A is diagonalizable if it has a set of n linearly
independent eigenvectors
 In that case,

where P is the matrix with as its columns, and

are the corresponding eigenvalues.
 Diagonalization

 EXAMPLE 15 Verify the Theorem above for

 Solution: From Example earlier, the eigenvalues are

 For the matrix P we can choose

whose inverse is

 Now direct multiplication shows that

Diagonalization

NOTE
 A matrix P is called orthogonal if
 If are the n column vectors of P, then
are the row vectors of the transposed matrix, P'.
 The condition P'P =I then reduces to the n2 equations

if
 We conclude that P is orthogonal if all have length
1 and are mutually orthogonal.
 Diagonalization

Many of the matrices encountered in economics are symmetric. For

symmetric matrices we have the following important result.
THEOREM (THE SPECTRAL THEOREM)
If the matrix is symmetric, then:
 All the n eigenvalues are real.
 Eigenvectors that correspond to different eigenvalues are orthogonal.
 There exists an orthogonal matrix P such that

 The columns of the matrix P are eigenvectors of unit

length corresponding to the eigenvalues
Quadratic Forms

 Many applications of mathematics make use of the following

special kind of double sum.
 A quadratic form in n variables is a function Q of the form

where the aij are constants.

 Quadratic Forms
 Suppose we put
 Then it follows from the definition of matrix multiplication that

 Of course, so we can write

 If we replace then the new numbers
become equal without changing
 Thus, we can assume that for all i and j, which means
that the matrix A is symmetric.
 Then A is called the symmetric matrix associated with Q, and Q is
called a symmetric quadratic form.
 Quadratic Forms

 EXAMPLE 17 Write
in matrix form with A symmetric.
 Solution: We first write Q as follows:

 Then where
DEFINITENESS OF A QUADRATIC FORM

 A quadratic form , as well as its associated

symmetric matrix A, are said to be positive definite, positive
semidefinite, negative definite, or negative semidefinite
according as

for all
 The quadratic form Q(x) is indefinite if there exist vectors x*
and y* such that
 Thus an indefinite quadratic form assumes both negative and
positive values.
DEFINITENESS OF A QUADRATIC FORM

 In order to prove that a quadratic form is positive semidefinite,

we have to prove that for all choices of x.
 How can this be done?
 Consider first the case n =2. Then the quadratic form is

 We claim that:
 DEFINITENESS OF A QUADRATIC FORM

 The results for quadratic forms in two variables can be generalized. In

order to do so we need a few new concepts.
 To study the sign of quadratic forms we need some special types of
minor.
 Let be any n x n matrix.
 An arbitrary principal minor of order r is the determinant of the
matrix obtained by deleting all but r rows and r columns in A with the
same numbers.
 In particular, a principal minor of order r always includes exactly r
elements of the main (principal) diagonal.
 We call the determinant |A| itself a principal minor.. (No rows or
columns are deleted.) A principal minor is called a leading principal
minor of order r if it consists of the first (''leading") r rows
and columns of |A|.
DEFINITENESS OF A QUADRATIC FORM

 EXAMPLE 18 Write down the principal and leading principal

minors of

 Solution: By deleting row 2 and column 2 in A we get ,

which has determinant equal to the number itself.
 Deleting row 1 and column 1 in A we get .
 The principal minors of A are therefore
 The leading principal minors are
DEFINITENESS OF A QUADRATIC FORM

 The principal minors of B are |B| itself, and

while the leading principal minors are

 DEFINITENESS OF A QUADRATIC FORM

 Suppose A is an arbitrary n x n matrix. The leading principal minors are

 These determinants are obtained from the elements in |A| according to

the pattern suggested by the following arrangement:
 DEFINITENESS OF A QUADRATIC FORM

THEOREM Consider the quadratic form

with the associated symmetric matrix

 Let Dk be a leading principal minors and let k denote a arbitrary
prin-cipal minor of order k. Then we have:
 DEFINITENESS OF A QUADRATIC FORM

 EXAMPLE 19 Determine the definiteness of

 Solution: It makes sense to check the leading principal minors first,

in case the matrix turns to be definite rather than merely semidefinite.
 a) The associated symmetric matrix A is given in Example 17,
and its leading principal minors are

 We conclude that Q is positive definite.

 DEFINITENESS OF A QUADRATIC FORM

 B) The associated symmetric matrix is .

 Here the leading principal minors are

 It follows that the conditions in part (a) of Theorem are not
satisfied, nor are those in (b) or (c).
 In order to check the conditions in (d), we must examine all the
principal minors of A.
 As described in Example 11, the 3x3 matrix A has three principal
minors of order 1, whose values
1(1) , 1(2) and 1(3)
are the diagonal elements.
DEFINITENESS OF A QUADRATIC FORM

 These satisfy

 There are also three second-order principal minors which satisfy

 Hence
DEFINITENESS OF A QUADRATIC FORM

 Finally,
 Thus for all principal minors in the
matrix A associated with the quadratic form Q.
 It follows from (d) that Q is negative semidefinite.

 The definiteness of a quadratic form can often be determined

more easily from the signs of the eigenvalues of the associated
matrix.
 DEFINITENESS OF A QUADRATIC FORM

THEOREM
 Let be a quadratic form, where the matrix A is
symmetric, and let be the (real) eigenvalues of A.
Then:
 DEFINITENESS OF A QUADRATIC FORM

 EXAMPLE 20 Use the Theorem above to determine the

definiteness of the quadratic form in Example 12(b).
 Solution: The associated symmetric matrix is

 The characteristic equation simplifies to so

the eigenvalues are 0, -2, and -10.
 Theorem tells us the quadratic form is negative semidefinite.