Ch 19 The matrix eigenvalue problem

Let us consider linear homogeneous equations of the special form

a11x1 + a12x2 + a13x3 + ... + a1nxn = x1

a21x1 + a22x2 + a23x3 + ... + a2nxn = x2
Eq 1
a31x1 + a32x2 + a33x3 + ... + a3nxn = x3
⋮ ⋮
an1x1 + an2x2 + an3x3 + ... + annxn = xn

where the coefficients aij on the left side are assumed to be given. The unknowns are x1, x2, ..., xn, and
an additional parameter . There are (n+1) variables but n equations. Thus we know from the start
that one variable will be independent, i.e. one may assign an arbitrary value to one of the variables.
We have already given an example of such equations at the end of Ch17. Here we will discuss Eq 1 in
more detail.

Eq 1 may be written as

(a11 ─ )x1 + a12x2 + a13x3 + ... + a1nxn = 0

a21x1 + (a22 ─ )x2 + a23x3 + ... + a2nxn = 0 Eq 2
a31x1 + a32x2 + (a33 ─ )x3 + ... + a3nxn = 0
⋮ ⋮
an1x1 + an2x2 + an3x3 + ... + (ann ─ )xn = 0

Non-trivial solutions of these homogeneous linear equations exist if and only if the determinant of
the coefficients is zero.

(𝑎11 − ) 𝑎12 ⋯ 𝑎1𝑛

𝑎21 (𝑎22 − ) ⋯ 𝑎2𝑛
| ⋮ ⋮ |=0 Eq 3
⋮
𝑎𝑛1 𝑎𝑛2 ⋯ (𝑎𝑛𝑛 − )

When expanded, the determinant on the left gives a polynomial of degree n in . By finding the roots
of this polynomial we obtain the allowed values of  which will permit non-trivial solutions of Eq 2.
There will be n roots: 1, 2, ..., n, including the possibility that some roots may repeat. This is
always the first step in solving Eq 2. In the second step, for each of these  values, we solve Eq 2 for
the variables x1, x2, ..., xn. Note that for each  we will obtain a different set of values for x1, x2, ..., xn.
The linear homogeneous equation system Eq 1 can be written in matrix notation as follows. One
forms a n×n matrix A from the coefficients in Eq 1, and defines a n×1 column vector x of the
unknowns

… 𝑎1𝑛 𝑥1
𝑎11 𝑎12
… 𝑎2𝑛 𝑥2
𝑎21 𝑎22
𝑨=( ⋮ ⋮ ), 𝒙 = 𝑥3 Eq 4
⋮
𝑎𝑛1 𝑎𝑛2 … 𝑎𝑛𝑛 𝑥
( 𝑛)

Left side of the equations in Eq 1 can be expressed as a n×1 column vector by the product, Ax,
according to the rules of matrix multiplication, and this vector is equal to the vector x.

Ax =  x Eigenvalue-eigenvector equation of matrix A Eq 5

This matrix equation is very special; because in general, when one multiplies two matrices (here A
and x) one expects to find a resultant matrix that is different from the matrices involved in the
product. Yet, according to Eq 5, the resultant matrix of the product Ax is proportional to the column
matrix x involved in the product,  being the proportionality constant.

Please note that in Eq 5, the matrix A is the known part. The unknowns are  and x. The allowed
values of the scalar , and the associated vectors x all depend on what A is. Hence the central
quantity of Eq 5 is the coefficient matrix A.

The determinant in Eq 3 is called the characteristic determinant of the matrix A, and Eq 3 itself is
called the characteristic equation of the matrix A. The set of  values obtained from Eq 3 are called
the eigenvalues or characteristic values of the matrix A. For each value of , say i, solution of the
linear system Eq 2 gives a corresponding vector xi. There are n values of , and the same number of
vectors x. These vectors are called eigenvectors of the matrix A.

Before we discuss several examples, let us note a simple but important property of the eigenvalue-
eigenvector Eq 5. If a vector x is an eigenvector of A with eigenvalue  (i.e. if x satisfies Eq 5), then
any multiple of it, kx, also satisfies Eq 5 with the same eigenvalue, where k is an arbitrary constant.
Thus (remember that multiplication of a matrix by a scalar is commutative; i.e. kA = Ak),

Eq 6
A(kx) = kAx = kx = (kx)
Hence, eigenvectors of the matrix A are defined only within an arbitrary factor k. This is not
surprising since Eq 5 is equivalent to Eq 2, and as remarked above there are (n+1) variables in Eq 2,
but n equations with the consequence that one of the (nonzero) variables x1, x2, ..., xn is independent
and hence may be assigned any arbitrary value. The constant k in Eq 6 corresponds to the value of
the independent variable chosen.

e.g.1 Find the eigenvalues of the square matrix

8 −8 −2
𝑨 = (4 −3 −2) Eq 7
3 −4 1

Eigenvalues are found by solving the characteristic equation (Eq 3) of matrix A:

(8 − 𝜆) −8 −2
| 4 (−3 − 𝜆) −2 | = 0
3 −4 (1 − 𝜆)

Expansion of the determinant gives

─3 + 62 ─ 11 + 6 = ─ ( ─ 1)(  ─ 2)(  ─ 3) = 0

Hence, eigenvalues of A are: 1 = 1, 2 = 2, and 3 =3.

Note that trace of matrix A is: 8 – 3 + 1 = 6, and sum of its eigenvalues is: 1 + 2 + 3 =1 +2 + 3 = 6.
This is a general property valid for any square matrix A of order n

tr A = 1 + 2 + 3 + ... + n Eq 8

Another useful property is

det A = 1 2 3 ... n Eq 9

In our example, determinant of matrix in Eq 7 is 6, and it is equal to (1)(2)(3) = 6.

Please note from Eq 9 that if det A is zero, we immediately know that (at least) one of the
eigenvalues of A is zero; in this case, expansion of the characteristic determinant of A will not contain
a constant additive term, i.e, the nth degree polynomial in  that results from Eq 3 will have  as a
factor.
e.g.2 Find the eigenvalues of

6 −8 −2
𝑩 = (4 −5 −2 ) Eq 10
3 −4 −1

Before we construct the characteristic determinant, we notice that first row of B is twice row 3, and
therefore det B is zero. Hence we conclude that one of the eigenvalues will be zero. Let us continue
with Eq 3 for matrix B

(6 − 𝜆) −8 −2
| 4 (−5 − 𝜆) −2 | = −𝜆3 + 𝜆 = −𝜆(𝜆2 − 1) = 0
3 −4 (−1 − 𝜆)

As expected, there is no additive constant term in the polynomial, and  is a factor. Eigenvalues of B
are: 1= -1, 2 = 0, 3 = 1. Sum of eigenvalues is zero which is the same as tr B.

e.g.3 Find the eigenvectors of matrix A in Eq 7.

Eigenvectors are found from Eq 2, which for our matrix A is

(8 ─ )x1 ─ 8x2 ─ 2x3 = 0 Eq 11

4x1 + (─3─ )x2 ─ 2x3 = 0
3x1 ─ 4x2 + (1 ─ )x3 = 0

The first step in such a question is to find the eigenvalues because non-trivial solutions of these
equations can be found only if  is one of the eigenvalues. We already found the eigenvalues in e.g.1;
they are: 1 = 1, 2 = 2, and 3 =3. For each of these eigenvalues we solve Eq 11 to obtain the
corresponding eigenvectors.

For  = 1 = 1, Eq 11 becomes

(1) 7x1 ─ 8x2 ─ 2x3 = 0

(2) 4x1 ─4x2 ─ 2x3 = 0
(3) 3x1 ─ 4x2 = 0
We eliminate x3 from Eq (2) by: eq2  eq2 + (-1)eq1

(1) 7x1 ─ 8x2 ─ 2x3 = 0

(2) ─3x1 +4x2 = 0
(3) 3x1 ─ 4x2 = 0

Equations 2 and 3 are the same, so we discard Eq (3). There are 3 variables but 2 equations so that
one of the variables can be assigned any arbitrary value. We choose x2, and assign x2=3a where a is
an arbitrary number. Then from Eq (2), x1=4a, and from Eq (1), x3=2a. We index each eigenvector
with the index of the eigenvalue used in solving Eq 11. So the index is 1 for this solution, and the
eigenvector x1 is

4𝑎 4
𝒙1 = (3𝑎) = 𝑎 (3)
2𝑎 2

The arbitrary constant a corresponds to k in Eq 6; its value can not be determined from the
eigenvalue-eigenvector equation Eq 5 (which means Eq 3 plus Eq 2). It is customary to fix the value of
a by an additional condition called the “normalization” condition: eigenvectors are forced to be unit
vectors.

xTx = 1 Normalization condition for the eigenvectors Eq 12

Applying the normalization condition to x1

1
𝒙1𝑇 𝒙𝟏 = 𝑎2 (16 + 9 + 4) = 29𝑎2 = 1  𝑎 =
√29

Either sign may be selected for the square root, we have chosen the plus sign. Hence the normalized
eigenvector x1 corresponding to the eigenvalue 1 = 1 is

14
𝒙1 = (3)
√29 2

We repeat the above procedure for finding the eigenvectors of A corresponding to 2 and 3.
For 2 =2, Eq 11 is

(1) 6x1 ─ 8x2 ─ 2x3 = 0

(2) 4x1 ─5x2 ─ 2x3 = 0
(3) 3x1 ─ 4x2 ─ x3 = 0
Eq (1) and Eq (3) are the same, and we discard Eq (3). Using Eq (2) to eliminate x3 from Eq (1) by:
eq1  eq1 + (-1)eq2

(1) 2x1 ─ 3x2 = 0

(2) 4x1 ─5x2 ─ 2x3 = 0

Setting x2 = 2b where b is an arbitrary constant, one gets x1 = 3b from Eq (1), and then x3 = b from
Eq (2). The eigenvector corresponding to 2 =2 is thus

3𝑏 3 3
1
𝒙2 = (2𝑏) = 𝑏 (2)  𝒙2 = (2) normalized eigenvector
√14
𝑏 1 1

The eigenvector x3 corresponding to 3=3 is found similarly from Eq 11. I leave the details to you. You
should obtain

2𝑐 2 2
1
𝒙3 = ( 𝑐 ) = 𝑐 (1)  𝒙3 = (1) normalized eigenvector
√6
𝑐 1 1

where c is an arbitrary constant.

e.g.4 Consider the matrix B of e.g.2 (Eq 10)

6 −8 −2
𝑩 = (4 −5 −2 )
3 −4 −1

Which of the following vectors are eigenvectors of B? For those that are eigenvectors, state the
corresponding eigenvalues.

1 8 −2 3
𝒙1 = (2), 𝒙2 = (6), 𝒙𝟑 = ( 1 ), 𝒙𝟒 = ( 2)
1 4 1 1

The eigenvalue-eigenvector equation is Bx = x. We test this equation for each of the given vectors.
For x1
 6 −8 −2 1 −12 1 1
𝑩𝒙𝟏 = (4 −5 −2 ) (2) = ( −8 ) = −12 (8/12) ≠ 𝑐𝑜𝑛𝑠𝑡 (2)
3 −4 −1 1 −6 6/12 1

Hence x1 is not an eigenvector of B. For x2

6 −8 −2 8 −8 8
𝑩𝒙𝟐 = (4 −5 −2 ) (6) = (−6) = (−1) (6)
3 −4 −1 4 −4 4

Therefore x2 is an eigenvector of B with eigenvalue =-1. For x3

6 −8 −2 −2 −22 −2 −2
𝑩𝒙𝟑 = (4 −5 −2 ) ( 1 ) = (−15) = (11) (−15/11) ≠ 𝑐𝑜𝑛𝑠𝑡 ( 1 )
3 −4 −1 1 −11 −1 1

Hence x3 is not an eigenvector of B. For x4

6 −8 −2 3 0 3
𝑩𝒙𝟒 = (4 −5 −2 ) ( 2 ) = (0) = (0) (2)
3 −4 −1 1 0 1

Thus x4 is an eigenvector of B with eigenvalue =0.

Eigenvalues of positive integer powers of A

Eigenvectors of a square matrix A of order n are also eigenvectors of any nonnegative integer power
of A. Thus given that x is an eigenvector of A with eigenvalue , i.e.

Ax = x

we can easily see that x is also an eigenvector of, for example, A2.

𝑨2 𝒙 = 𝑨(𝑨𝒙) = 𝑨(𝜆𝒙) = 𝜆(𝑨𝒙) = 𝜆(𝜆𝒙) = 𝜆2 𝒙

Thus x is an eigenvector of A2 with eigenvalue 2. This property may be generalized to all positive
integer powers of matrix A:

Amx = m x, m = 1,2,3,... Eq 13

e.g.5 Consider the matrix A of Eq 7. What are the eigenvalues of A3?

8 −8 −2 3 74 −104 10
𝟑
𝑨 = (4 −3 −2) = (40 −51 −2)
3 −4 1 33 −52 13
Eigenvalues of A were found in e.g.1: 1 = 1, 2 = 2, and 3 =3. From Eq 13, eigenvalues of A3 are
13=1, 23=8, and 33=27. Note that tr A3 = 74-51+13 = 36, and this is equal to 1+8+27=36, as a partial
check. The corresponding eigenvectors of A3 are the same as those of A found in e.g.3.

We next show that any polynomial expression in square matrix A has the same eigenvectors as those
of A. Let

B = c0I + c1A + c2A2 + c3A3 + ... + cmAm Eq 14

where I is the unit matrix, and the coefficients ci are constants. All matrices in this expression are
square of order n. Due to the property Eq 13, and the fact that multiplication of a matrix with a scalar
is commutative (i.e. Ai=Ai), one has

Bx = c0Ix + c1Ax + c2A2x + c3A3x + ... + cmAmx = (c0 + c1 + c22 + c33 + ... + cmm)x

Hence

Bx = bx, b = c0 + c1 + c22 + c33 + ... + cmm Eq 15

Thus, an eigenvector x of matrix A with eigenvalue  is also an eigenvector of matrix B in Eq 14; the
eigenvalue of matrix B is b, obtained from Eq 15.

e.g.6 Taking the matrix A in Eq 7, what are the eigenvalues of the matrix B below?

108 −136 −4
2
B = 2I - 3A + 5A = ( 58 −64 −14)
46 −68 14

Eigenvalues of A are: 1 = 1, 2 = 2, and 3 =3. According to Eq 15, for 1 = 1, b1 = (2-3+5)=4; for 2 = 2,

b2 = (2-6+20)=16, and for 3 = 3, b3 =(2-9+45)=38. Checking: tr B = 108-64+14=58, and sum of
eigenvalues of B: 4+16+38=58. Eigenvectors of B are the same as those of A as found in e.g.3.

As a special case of the polynomial in Eq 14, consider the matrix

A’ = c0I + A Eq 16

The matrix A’ differs from A only in the diagonal elements: all diagonal elements of A are shifted by
the same amount c0 in forming A’. From Eq 15, the eigenvalues of A’ are ’ =  + c0. In other words,
when diagonal elements of a square matrix A are shifted by c0, (all of) its eigenvalues are shifted by
the same amount.

As an example of Eq 16, consider the matrices used in e.g.1 and e.g.2. The matrix A in Eq 7, and
matrix B in Eq 10 are reproduced below.

8 −8 −2 6 −8 −2
𝑨 = (4 −3 −2) , 𝑩 = (4 −5 −2 ) Eq 17
3 −4 1 3 −4 −1

Comparison of the two matrices reveals that they are related to each other as in Eq 16:

B = A ─ 2I Eq 18

If we designate eigenvalues of B by ’, and those of A by , we have

’ =  ─ 2 Eq 19

As remarked above, det B = 0, implying that one of the three vaues of ’ is 0, and hence solving the
characteristic equation of B is slightly easier than that of A.

Now consider a problem in which we are asked to find the eigenvalues of A, as in e.g.1. If by shifting
the diagonal elements of A we can find a matrix B as in Eq 18, such that det B = 0, our task becomes
somewhat simplified because then we may calculate the eigenvalues of B first, and then use Eq 19 to
obtain eigenvalues of A.

In e.g.2, we found ’1= -1, ’2 = 0, ’3 = 1 so that eigenvalues of A are =’ + 2; hence, 1=-1+2=1,
2=0+2=2, and 3=1+2=3. Note once again that eigenvectors of A and B are identical.

Symmetric matrices

In the majority of real-life applications, the square matrix A of order n in the eigenvalue-eigenvector
equation, Eq 5, is a symmetric matrix. A square matrix A is said to be symmetric if the transposed
matrix is equal to itself:

AT = A symmetric matrix Eq 20

Graphically, this relation means that in a symmetric matrix, the off-diagonal elements in the lower
triangle of a square matrix A are mirror images through the diagonal line, of those in the upper
triangular part. Or, in terms of elements, aji = aij for all i,j=1,2,...,n.
The significance of symmetric matrices lies in the following two properties:

i) All eigenvalues of a symmetric matrix are real numbers.

ii) Eigenvectors of a symmetric matrix form an orthogonal set of vectors.

These properties are not necessarily true for non-symmetric square matrices.

e.g.7 Find the eigenvalues and eigenvectors of the matrix below.

1 3 0
𝑨 = (3 1 3)
0 3 1

This is a symmetric matrix. As always, solution of an eigenvalue-eigenvector problem starts by finding

the eigenvalues of A. It should be easy to note that the matrix A may be expressed as in Eq 16:

1 3 0 0 3 0 1 0 0
(3 1 3 ) = (3 0 3 ) + (0 1 0)  A = A’ + I
0 3 1 0 3 0 0 0 1

Eigenvalues of A are related to those of A’ by:  = ’ + 1. The first and third columns of A’ are
identical, hence det A’ = 0. One of the eigenvalues of A’ is zero, and therefore we prefer to work with
A’, and then switch to A. Eigenvectors of A’ and A are the same, and their eigenvalues are related as
shown above.

The characteristic equation (Eq 3) of matrix A’ gives: -’(’2─18) = 0. Hence, 𝜆1′ = −3√2, 𝜆′2 = 0,
𝜆′3 = 3√2. For each eigenvalue, the corresponding eigenvector is obtained from

−𝜆′𝑥1 + 3𝑥2 =0
3𝑥1 − 𝜆′𝑥2 + 3𝑥3 =0
3𝑥2 − 𝜆′𝑥3 =0

You should verify that the normalized eigenvectors are:

1 1
𝐹𝑜𝑟 𝜆1′ = −3√2, 𝒙1 = (−√2)
2
1

1 1
𝐹𝑜𝑟 𝜆′2 = 0, 𝒙2 = ( 0)
√2 −1

1 1
𝐹𝑜𝑟 𝜆′3 = 3√2, 𝒙3 = (√2)
2
1
Now we turn to our actual problem: eigenvalues and eigenvectors of A. Eigenvalues of A are: 𝜆1 =
1 − 3√2, 𝜆2 = 1, and 𝜆3 = 1 + 3√2. The corresponding eigenvectors are the same as those of A’,
given above.

Let us consider all scalar products of two eigenvectors 𝒙𝑇𝑖 𝒙𝑗 with ij. For the three eigenvectors
above, one finds

1
𝒙1𝑇 𝒙2 = [(1)(1) + (√2)(0) + (1)(−1)] = 0
2√2

1
𝒙1𝑇 𝒙3 = [(1)(1) + (−√2)(√2) + (1)(1)] = 0
4

1
𝒙𝑇2 𝒙3 = [(1)(1) + (0)(√2) + (−1)(1)] = 0
2√2

Thus the three eigenvectors form an orthogonal set. The property of orthogonality of the
eigenvectors originates from the fact that matrix A is symmetric, and is generally true for any
symmetric matrix of order n.