3 Matrices and systems of equations

3.1 Systems of equations

Many engineering problems result in a system of equations for un-
known quantities x
1
, x
2
, x
3
, x
4
, say.
Variables x
j
(j = 1, 2, 3, 4) represent, for example, production totals,
electrical currents, stresses, etc.
Often these equations will be linear, i.e., no variable is either raised to
a power (other than one!) or multiplied by one of the other variables.
We will be interested in techniques which can be eciently extended
to problems with many unknowns, as engineering applications often
demand (although in lectures, problems etc, we will rarely go beyond
three variables).
39
3.1.1 One equation for one unknown
Simplest case is of only one equation and one unknown. An important
step in solving systems of equations will be solving a single equation.
For example, suppose we are asked to solve:
2x = 6.
Clearly the solution is x = 3.
In general if a and b are two given numbers, we might be asked to nd
the values of x for which
ax = b (3.1)
is satised. There are three cases.
40
The three possible outcomes when solving ax = b (equation (3.1)) are:
1. If a = 0 then we can divide (3.1) by a to give x = b/a. This is the
only value of x that satises (3.1) and there is a unique solution.
2. If a = 0 and b = 0, (3.1) is not satised for any value of x no
numbers x satisfy 0 x = b (= 0). Hence there is no solution to
(3.1).
3. If a = 0 and b = 0, then any value of x satises (3.1) because
0 x = 0 for any number x. Hence there are innitely many solutions
to (3.1).
We will return to this situation time and time again.
41
3.1.2 A 2 2 system (2 equations in 2 unknowns)
Simple example: system of two equations
3x
1
+4x
2
= 2 (3.2)
x
1
+2x
2
= 0 (3.3)
for the unknowns x
1
and x
2
.
Many ways to solve this simple system of equations we describe
one that is easily generalised to much larger systems of linear equations.
Step 1
Eliminate x
1
from (3.3) by replacing
(3.3) 3(3.3)(3.2)
to give
3x
1
+4x
2
= 2 (3.4)
2x
2
= 2. (3.5)
42
Step 2
Solve (3.5) (this is easy to do divide both sides by 2). This gives
x
2
= 1. (3.6)
Note: (3.5) is a 1 1 system, as in (3.1).
Step 3
Substitute the result from Step 2 back into (3.4). This gives
3x
1
4 = 2 3x
1
= 6 x
1
= 2. (3.7)
The solution to equations (3.2) and (3.3) is therefore
x
1
= 2 x
2
= 1.
Check
3 2 +4 (1) = 2
2 +2 (1) = 0.
43
3.1.3 A 3 3 system
Similar procedure can be used to solve a system of three linear equa-
tions for three unknowns x
1
, x
2
, x
3
.
For example, suppose we wish to solve
2x
1
+3x
2
x
3
= 5 (3.8)
4x
1
+4x
2
3x
3
= 3 (3.9)
2x
1
3x
2
+ x
3
= 1 (3.10)
to obtain x
1
, x
2
, x
3
. There are more steps involved because there are
more equations and more unknowns.
44
Step 1
Eliminate x
1
from equations (3.9) and (3.10) by subtracting multiples
of (3.8).
Replace
(3.9) (3.9)2(3.8)
(3.10) (3.10)(3.8)
to leave the system
2x
1
+3x
2
x
3
= 5 (3.11)
2x
2
x
3
= 7 (3.12)
6x
2
+2x
3
= 6 (3.13).
45
Step 2
Eliminate x
2
from (3.13). To do this, we subtract an appropriate mul-
tiple of (3.12) from (3.13).
(If we subtract a multiple of (3.11) from (3.13) instead, then in the
process of eliminating x
2
from (3.13) we reintroduce x
1
!)
Replace
(3.13) (3.13) 3 (3.12)
to leave
2x
1
+3x
2
x
3
= 5 (3.14)
2x
2
x
3
= 7 (3.15)
5x
3
= 15 (3.16).
Step 3
Solve the 11 system (3.16) to nd x
3
. In this case there is a unique
solution to (3.16):
x
3
=
15
5
= 3.
46
Step 4
Substitute this result back into (3.15) to give x
2
from
2x
2
3 = 7 2x
2
= 4 x
2
= 2.
Step 5
Substitute the values we have found for x
2
and x
3
back into (3.14) to
give
2x
1
+6 3 = 5 2x
1
= 2 x
1
= 1.
Finally the solution to the system (3.8), (3.9), (3.10), is
x
1
= 1 x
2
= 2 x
3
= 3.
This solution is unique.
This method is Gaussian Elimination, and can be applied to any
system where the number of equations equals the number of unknowns.
47
3.1.4 Another 3 3 system
We apply Gaussian elimination to the equations
x
1
+ x
2
+2x
3
= 1 R
1
2x
1
x
2
x
3
= 1 R
2
x
1
+4x
2
+7x
3
= 2 R
3
using somewhat more compact notation.
Step 1
Eliminate x
1
from the last two equations
R
2
R
2
2R
1

R
3
R
3
R
1
x
1
+ x
2
+2x
3
= 1 R

1
3x
2
5x
3
= 1 R

2
3x
2
+5x
3
= 1 R

3
Step 2
Eliminate x
2
from the last equation (use R

2
so x
1
doesnt come back!)
R

3
R

3
+R

x
1
+ x
2
+2x
3
= 1 R

1
3x
2
5x
3
= 1 R

2
0 = 0 R

3
48
Step 3
The last equation is a 1 1 system which is easily solved. In this
case the equation
0 = 0 R

3
is satised for any value of x
3
and there are innitely many solutions.
Let
x
3
= t
where t is an arbitrary number.
Step 4
Substitute x
3
back in the second equation R

2
to get
3x
2
5x
3
= 3x
2
5t = 1 x
2
=
1
3

5t
3
.
Step 5
Substitute x
2
and x
3
back in the rst equation R

1
to get
x
1
+ x
2
+2x
3
= x
1
+
1
3

5t
3
+2t = 1 x
1
=
2
3

t
3
.
49
Conclusion For any given value of t,
x
1
=
2
3

t
3
x
2
=
1
3

5t
3
x
3
= t
is a solution of the original set of equations and there are an innite
number of such solutions.
Exercise Check this by direct substitution in the original equations.
50
3.2 Matrices
Many engineering problems involve a large number of unknowns and
a large number of equations to be solved simultaneously. These can
number in the hundreds or even the thousands in practice. Matrices.
are a convenient way of representing and manipulating such problems.
For example, the equations
3x
1
+4x
2
= 2 (3.17)
x
1
+2x
2
= 0 (3.18)
solved previously may be written more compactly by introducing
A =
_
3 4
1 2
_
matrix of coecients
x =
_
x
1
x
2
_
vector of unknowns
b =
_
2
0
_
vector of right-hand sides.
51
x and b are examples of a column vector, i.e., a matrix with a single
column.
Then we write the system of equations as
Ax = b (3.19)
or
_
3 4
1 2
__
x
1
x
2
_
=
_
2
0
_
. (3.20)
In books, vectors are written in bold type
In handwriting, a

or a.
52
For example, the 3 3 system
2x
1
+3x
2
x
3
= 5 (3.8)
4x
1
+4x
2
3x
3
= 3 (3.9)
2x
1
3x
2
+ x
3
= 1 (3.10)
we solved previously may be written as
Ax = b, (3.21)
where
A =
_
_
_
2 3 1
4 4 3
2 3 1
_
_
_, x =
_
_
_
x
1
x
2
x
3
_
_
_ and b =
_
_
_
5
3
1
_
_
_.
53
More generally, if we have a system of m equations in n unknowns
a
11
x
1
+ a
12
x
2
+ . . . + a
1n
x
n
= b
1
a
21
x
1
+ a
22
x
2
+ . . . + a
2n
x
n
= b
2
.
.
.
.
.
.
a
m1
x
1
+ a
m2
x
2
+ . . . + a
mn
x
n
= b
m
_

_
(3.22)
these may be written in matrix form as
Ax = b, (3.23)
where
A =
_
_
_
_
_
a
11
a
12
. . . a
1n
a
21
a
22
. . . a
2n
.
.
.
a
m1
a
m2
. . . a
mn
_
_
_
_
_
, x =
_
_
_
_
_
x
1
x
2
.
.
.
x
n
_
_
_
_
_
and b =
_
_
_
_
_
b
1
b
2
.
.
.
b
m
_
_
_
_
_
. (3.24)
A is an mn matrix (m = number of rows, n = number of columns)
x is an n 1 matrix (column vector)
b is an m1 matrix (column vector)
54
Note that a
ij
= element of A at the intersection of the i-th row and
the j-th column. The rst subscript refers to the row, the second to
the column.
3.2.1 Order of a matrix
The order of a matrix is its size, e.g., the order of A in (3.24) is mn.
55
3.3 Matrix algebra
3.3.1 Equality
Two matrices are equal if they have the same order and if their corre-
sponding elements are equal.
3.3.2 Addition and subtraction
If A and B are of the same order then we may add corresponding
elements to obtain
A+B =
_
_
_
a
11
+b
11
a
12
+b
12
. . . a
1n
+b
1n
.
.
.
a
m1
+b
m1
a
m2
+b
m2
. . . a
mn
+b
mn
_
_
_
or subtract B from A to obtain
AB =
_
_
_
a
11
b
11
a
12
b
12
. . . a
1n
b
1n
.
.
.
a
m1
b
m1
a
m2
b
m2
. . . a
mn
b
mn
_
_
_.
56
3.3.3 Example
If
A =
_
1 2
0 4
_
and B =
_
0 3
1 4
_
then
A+B =
_
1 +0 2 +3
0 +1 4 +4
_
=
_
1 5
1 8
_
and
AB =
_
1 0 2 3
0 1 4 4
_
=
_
1 1
1 0
_
.
3.3.4 Example
If
A =
_
1 2
0 4
_
and B =
_
0 3 2
1 4 1
_
then A+B and AB do not make sense because A and B have dierent
orders.
57
3.3.5 Multiplication of matrices
Multiplication of one matrix by another is more involved than you might
expect.
Example
_
1 1
2 1
__
0 1
3 2
_
=
_
3 1
3 4
_
_
0 1
3 2
__
1 1
2 1
_
=
_
2 1
7 1
_
Note: For matrices A and B, AB is not in general equal to BA.
58
Example
A =
_
4 1 6
3 2 1
_
is a 2 3 matrix and B =
_
_
_
1 1
1 2
1 0
_
_
_ is a 3 2 matrix.
A + B is not dened, nor is A B (because A and B have dierent
orders).
AB =
_
4 1 6
3 2 1
_
_
_
_
1 1
1 2
1 0
_
_
_ =
_
11 6
6 7
_
BA =
_
_
_
1 1
1 2
1 0
_
_
_
_
4 1 6
3 2 1
_
=
_
_
_
7 3 7
10 5 8
4 1 6
_
_
_
Note: AB = BA.
Note: AB is a 2 2 matrix, and BA is a 3 3 matrix.
59
Let
A be an mn matrix
B be an p q matrix
Then AB exists if n = p.
The result AB is an mq matrix. Thus
A B = C
m n p
. .
equal
q mq
If n = p, AB does not exist i.e. number of columns of A must equal
number of rows of B.
For example, if A is a 3 2 matrix and B is a 2 2 matrix then AB is
a 3 2 matrix but BA does not make sense.
60
3.3.6 Rule for matrix multiplication
When C = AB exists, the element c
ij
in the i
th
row and j
th
column of
C is obtained by taking the product of the i
th
row of A with the j
th
column of B.
c
ij
=

r
a
ir
b
rj
61
3.3.7 Zero matrix, 0
This is a matrix whose elements are all zero. For any matrix A,
A+0 = A.
We can only add matrices of the same order, therefore 0 must be of
the same order as A.
3.3.8 Multiplication of a matrix by a scalar
If is a scalar (i.e., a number) we dene
A =
_
_
_
a
11
a
12
. . . a
1n
.
.
.
a
m1
a
m2
. . . a
mn
_
_
_,
i.e., we multiply every element of A by to obtain A.
For example,
3
_
1 2
0 1
_
=
_
3 6
0 3
_
.
62
3.3.9 Further properties
If is a scalar and A, B and C are matrices then, provided all the
products exist:
(A)B = (AB) = A(B);
A(BC) = (AB)C
so we may write each of these products unambiguously as ABC;
(A+B)C = AC +BC;
C(A+B) = CA+CB;
In general AB = BA, even if both AB and BA exist;
AB = 0 does not necessarily imply that A = 0 or B = 0;
A0 = 0.
63
Example
AB =
_
0 1
0 0
__
3 0
0 0
_
=
_
0 0
0 0
_
= 0
but neither A nor B is the zero matrix.
It follows that AB = AC does not necessarily imply that B = C be-
cause
AB = AC A(B C) = 0
and as A and (B C) are not necessarily 0, B is not necessarily equal
to C.
64
Example
AB =
_
0 1
0 0
__
0 0
1 0
_
=
_
1 0
0 0
_
and
AC =
_
0 1
0 0
__
1 2
1 0
_
=
_
1 0
0 0
_
= AB
but B = C.
65
3.4 Special matrices
3.4.1 Square matrices
A square matrix is one where
no. of rows = no. of columns.
Example
_
_
_
1 2 3
3 1 2
2 3 1
_
_
_ and
_
1 1
0 4
_
are square matrices while
_
1 2 3
2 3 1
_
and
_
_
_
1 1
1 0
0 4
_
_
_
are not.
66
3.4.2 The identity matrix
The identity matrix is a square matrix whose elements are all zero,
except those on the leading diagonal, which are unity (= 1). (The
leading diagonal is the one from the top left to the bottom right.)
The identity matrix is usually denoted by I (or sometimes by I
n
if there
is a need to stress that it has the order n n). For example
I
3
=
_
_
_
1 0 0
0 1 0
0 0 1
_
_
_.
The identity matrix has the properties that
AI = A and IA = A
for any matrix A, and
Ix = x
for any vector x.
67
3.4.3 The transpose of a matrix
The transpose the mn matrix A is an n m matrix denoted by A
T
and obtained by interchanging the rows and columns of A.
Examples
If
A =
_
3 2 1
4 5 6
_
, B =
_
1
4
_
, C =
_
_
_
1 2 3
0 5 1
2 4 7
_
_
_
then their transposes are
A
T
=
_
_
_
3 4
2 5
1 6
_
_
_, B
T
=
_
1 4
_
, C
T
=
_
_
_
1 0 2
2 5 4
3 1 7
_
_
_.
Note: If D = EF then
D
T
= (EF)
T
= F
T
E
T
(3.25)
68
3.4.4 The inverse of a matrix
If A is a square matrix, then its inverse matrix is denoted by A
1
and
is dened by the property that
A
1
A = AA
1
= I.
Not every square matrix has an inverse.
Although an inverse matrix needs to satisfy both A
1
A = I and
AA
1
= I, if we can show one of these equations, then then the other
must follow, although we will not show that in this module. In other
words, it is enough to show that A
1
A = I or AA
1
= I to know that
A
1
is the inverse of A.
We will show how to calculate inverse matrices later in the module.
69
The inverse, if it exists, is very useful. For example, if we can nd
A
1
then we can solve the system Ax = b because
Ax = b A
1
Ax = A
1
b
Ix = A
1
b
x = A
1
b.
Thus there is a unique solution to
Ax = b,
given by x = A
1
b.
If D = EF then
D
1
= (EF)
1
= F
1
E
1
(3.26)
provided the inverses exist. This is easy to prove
DD
1
= EFF
1
E
1
= EIE
1
= EE
1
= I.
70
3.4.5 Orthogonal matrices
A matrix A which is such that
A
1
= A
T
is said to be an orthogonal matrix. Another way of saying this is that
AA
T
= A
T
A = I.
Example
A =
_
_
1

2
1

2
1

2
_
_
, A
T
=
_
_
1

2

1

2
1

2
1

2
_
_
,
and
AA
T
=
_
_
1

2
1

2
1

2
_
_
_
_
1

2

1

2
1

2
1

2
_
_
=
_
1 0
0 1
_
.
So AA
T
= I. That is, A
T
is the inverse of A.
71
3.4.6 Symmetric and anti-symmetric matrices
A square matrix A is said to be symmetric if
A = A
T
.
Example
_
_
_
_
_
1 0 2 3
0 3 4 7
2 4 1 6
3 7 6 2
_
_
_
_
_
is a symmetric matrix.
Note: the element a
ij
= a
ji
. That is, the element in the ith row and jth
column is the same as the element in the jth row and the ith column.
The matrix is symmetric about the leading diagonal.
72
A square matrix A is anti-symmetric if
A = A
T
.
Note: the element a
ij
= a
ji
. In particular a
11
= a
11
, a
22
= a
22
etc. Hence a
11
= 0, a
22
= 0 etc. That is, all elements on the leading
diagonal are zero.
Example
_
_
_
0 1 5
1 0 1
5 1 0
_
_
_.
is an antisymmetric matrix
73
3.5 Determinants of square matrices
3.5.1 A general 2 2 system
Consider a general 2 2 system:
ax
1
+bx
2
= b
1
(3.27)
cx
1
+dx
2
= b
2
. (3.28)
To solve this system, proceed as before.
Step 1 Eliminate x
1
from (3.28) by replacing (3.28) with
a(3.28)c(3.27):
ax
1
+bx
2
= b
1
(3.29)
(ad bc)x
2
= ab
2
cb
1
(3.30)
Step 2 Solve (3.30) for x
2
. This is a 1 1 system remember the
three possibilities:
74
(a) If ad bc = 0, we may solve (3.30) to give
x
2
=
ab
2
cb
1
ad bc
.
We may then carry out. . .
Step 3 Substitute the result from Step 2 back into (3.29). This
gives
ax
1
+b
_
ab
2
cb
1
ad bc
_
= b
1
and so
ax
1
=
adb
1
abb
2
ad bc
and hence
x
1
=
db
1
bb
2
ad bc
.
(Note we have divided through by a. If a = 0 we may not do this,
but it can be shown that the result still holds in this case.)
75
(b) If ad bc = 0 and the right-hand side of (3.30) is non-zero, i.e.,
ab
2
cb
1
= 0,
then there is no solution to (3.30), and hence no solution to the
2 2 system (3.27), (3.28).
(c) If
ad bc = 0
and
ab
2
cb
1
= 0
then any value of x
2
will satisfy (3.30). We may then carry out. . .
Step 3 Substitute each value of x
2
back into (3.29) to give a
corresponding value of x
1
.
There are thus innitely many solutions to the 22 system (3.27),
(3.28) in this case.
76
3.5.3 The determinant of a 2 2 system
The quantity D = ad bc is clearly important, and is called the deter-
minant of the system (3.27), (3.28). It is denoted by

a b
c d

or det
_
a b
c d
_
.
The solution of the system (3.27), (3.28) is unique provided D = 0.
If D = 0 then there are either
- no solutions (i.e., the equations are inconsistent), or
- innitely many solutions (i.e., the equations are not independent)
77
3.5.4 The inverse of a 2 2 matrix
If A is the 2 2 matrix
_
a b
c d
_
its determinant is D = ab cd.
If D = 0, A has an inverse A
1
given by
A
1
=
1
D
_
d b
c a
_
.
If D = 0, the matrix A has no inverse.
78
Example
Find the inverse of the matrix A =
_
1 5
2 3
_
.
Step 1 Calculate the determinant
det A = 1 3 5 2 = 13.
Since det A = 0 the inverse exists.
Step 2 Compute the inverse A
1
=
1
13
_
3 5
2 1
_
.
Check:
AA
1
=
1
13
_
1 5
2 3
__
3 5
2 1
_
=
1
13
_
3 10 5 5
6 6 10 3
_
=
_
1 0
0 1
_
= I
79
3.5.6 The determinant of a 3 3 matrix
These ideas can be extended from 2 2 matrices to 3 3, 4 4
matrices and beyond.
Again the determinant dictates whether the matrix is singular (has
no inverse) or nonsingular (has an inverse).
Given a 3 3 matrix
A =
_
_
_
a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33
_
_
_,
we dene its determinant to be
det A = a
11
det
_
a
22
a
23
a
32
a
33
_
a
12
det
_
a
21
a
23
a
31
a
33
_
+a
13
det
_
a
21
a
22
a
31
a
32
_
.
We already know how to calculate each of the 2 2 determinants.
80
3.5.7 Example

1 2 3
1 3 5
1 5 12

= 1

3 5
5 12

1 5
1 12

+3

1 3
1 5

= (36 25) 2(12 5) +3(5 3)

= 11 14 +6 = 3
Note that the denition above may be thought of as
det A = a
11
det
_
_
_
a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33
_
_
_
a
12
det
_
_
_
a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33
_
_
_
+a
13
det
_
_
_
a
11
a
12
a
13
a
21
a
22
a
23
a
31
a
32
a
33
_
_
_
81
3.5.8 Chessboard determinant
det A may be obtained by expanding about any row or column, using
the chessboard pattern of signs and the cover up method.
_
_
_
+ +
+
+ +
_
_
_
82
3.5.9 Example
Let A =
_
_
_
1 3 0
2 6 4
1 0 2
_
_
_. Here are three ways to calculate det A.
(a) Expand about the rst row to give
det A = +1

6 4
0 2

2 4
1 2

+0

2 6
1 0

= 12 24 +0 = 12.
(b) Expand about the middle row to give
det A = 2

3 0
0 2

+6

1 0
1 2

1 3
1 0

= 12 +12 12 = 12.
(c) Expand about the last column to give
det A = 0

2 6
1 0

1 3
1 0

+2

1 3
2 6

= 0 12 +0 = 12.
83
3.5.10 Properties of determinants
The properties described below follow from the denition of the deter-
minant (although proving them can be tricky for high-order matrices).
They apply to deteminants of all orders, but are illustrated by 2 2
determinants.
(i) The determinant of any matrix is equal to the determinant of its
transpose, i.e. det(A) = det(A
T
). Therefore any results about de-
terminants that deal with rows are also true for columns.
(ii) If all elements of one row of a matrix are multiplied by a constant
, the determinant of the new matrix is times the determinant of
the original matrix. For example

a b
c d

= ad bc
= (ad bc)
=

a b
c d

.
84
The same holds if one column of a matrix is multiplied by a
constant .
Application If a factor appears in each element of a row or
column of a determinant it can be taken out as a factor: e.g.

2 12
1 3

= 2

1 6
1 3

= 2 3

1 2
1 1

= 2 3 (1 1 2 1) = 6.
Application If all elements of a row or column of a determinant
are zero, the value of the determinant is zero. For example

0 0
c d

= 0 d 0 c = 0.
Application If A is an n n matrix,
det(A) =
n
det A.
This follows on applying rule (ii) n times, once for each row.
(iii) Interchanging any two rows or any two columns of a determinant
changes the sign of the determinant. For example, interchanging
rows gives

c d
a b

= cb ad
= (ad bc)
=

a b
c d

.
Application If any two rows are identical, the determinant is zero.
Similarly if any two columns are identical then the determinant is
zero.
Application (of (ii) and (iii) together) If any row is a multiple of
any other row, the determinant is zero. (Similarly if any column
is a multiple of any other column, the determinant is zero.) For
example

c d
c d

c d
c d

= 0.
(iv) The value of a determinant is unchanged by adding to any row
(or column) a constant multiple of any other row (or column). For
example if Row 1 (Row 1 + Row 2),

a +c b +d
c d

= (a +c)d (b +d)c
= ad bc
=

a b
c d

.
(v) If A and B are square matrices of the same order then
det(AB) = det Adet B.
Application Let A = I and let B be any matrix with nonzero
determinant. Then, because AB = IB = B, we have
det(AB) = det B
= det Adet B = det I det B
and therefore det I = 1.
Example
Evaluate

1 1 2
0 2 4
3 0 6

1 1 2
0 2 4
3 0 6

= 2

1 1 2
0 1 2
3 0 6

(since Row 2 has factor of 2)

= 2 3

1 1 2
0 1 2
1 0 2

(since Row 3 has factor of 3)

= 2 3

0 1 0
0 1 2
1 0 2

(subtract Row 3 from Row 1)

= 2 3

0 2
1 2

= 2 3 2 = 12
85
3.6 Row reduction methods
Technique introduced in 3.1 for solving systems of equations (Gaus-
sian elimination) can be applied using matrices as follows.
Important to note that the full equations need not be written out at
each stage, only the numerical coecients are required.
These coecients can be written in matrix form. Solve the following
system of equations for x
1
, x
2
and x
3
:
3x
1
x
2
+ 2x
3
= 3
2x
1
+ x
2
+ x
3
= 2
x
1
3x
2
= 5.
In matrix form this is Ax = b, where
A =
_
_
_
3 1 2
2 1 1
1 3 0
_
_
_, x =
_
_
_
x
1
x
2
x
3
_
_
_, b =
_
_
_
3
2
5
_
_
_.
86
Step 1 Write down the augmented matrix of coecients and right-
hand sides
_
_
_
3 1 2 : 3
2 1 1 : 2
1 3 0 : 5
_
_
_.
Then, as in Gaussian elimination, we make terms below the leading
diagonal equal to zero. Note that we need not write down equations
for x
1
, x
2
, x
3
at each stage, we can simply deal with the numerical
coecients in the augmented matrix.
Carrying out the elimination process gives . . .
Step 2 Remove terms below leading diagonal in rst column.
R
1
R
1
R
2
3R
2
2R
1
R
3
3R
3
R
1
_

_
_
_
_
3 1 2 : 3
0 5 1 : 12
0 8 2 : 12
_
_
_
Step 3 Remove terms below leading diagonal in second column.
R
1
R
1
R
2
R
2
R
3
5R
3
+8R
2
_

_
_
_
_
3 1 2 : 3
0 5 1 : 12
0 0 18 : 36
_
_
_
87
Step 4 Re-write each row as an equation for x
1
, x
2
and x
3
. The last
row gives an equation for x
3
:
18x
3
= 36.
Solving this equation gives x
3
= 2.
Step 5 Substitute x
3
into the equation from row 2:
5x
2
x
3
= 12 5x
2
= 12 +2 = 10
x
2
= 2.
Step 6 Substitute x
2
and x
3
into the equation from row 1:
3x
1
x
2
+2x
3
= 3 3x
1
= 1 2
x
1
= 1.
The solution is therefore x
1
= 1, x
2
= 2, x
3
= 2.
Elementary row operations
The operations in Steps 2 and 3 are examples of elementary row
operations (EROs).
There are three types of elementary row operations:
1. interchange any two rows;
This is equivalent to swapping the order of any two equations.
2. multiply any row by any nonzero constant;
This is equivalent to multiplying both sides of a given equation by
the constant.
3. add a multiple of one row to another.
This is equivalent to adding a multiple of one equation to another.
88
It is important to distinguish EROs from the broader range of
operations that may be applied to determinants. For example,
operations to columns are not allowed when solving a system of
equations by EROs.
3.8 Inversion by GaussJordan method
Row reduction methods can be used to nd the inverse of a matrix.
We demonstrate by example.
Again only EROs may be applied: column operations are not allowed.
Example
Calculate the inverse of A =
_
_
_
1 1 3
2 1 1
1 3 5
_
_
_.
Step 1 Consider the augmented matrix
(A : I) =
_
_
_
1 1 3
.
.
. 1 0 0
2 1 1
.
.
. 0 1 0
1 3 5
.
.
. 0 0 1
_
_
_.
The method is to perform elementary row operations on this augmen-
ted matrix to reduce it to the form (I : B). The 3 3 matrix B is
then A
1
, the inverse of A. (Proof not required.)
89
Step 2 Clear the terms below leading diagonal of left-hand matrix.
_
_
_
1 1 3
.
.
. 1 0 0
2 1 1
.
.
. 0 1 0
1 3 5
.
.
. 0 0 1
_
_
_
R
2
R
2
2R
1

R
3
R
3
R
1
_
_
_
1 1 3
.
.
. 1 0 0
0 1 5
.
.
. 2 1 0
0 2 2
.
.
. 1 0 1
_
_
_
R
3
R
3
+2R
2

_
_
_
1 1 3
.
.
. 1 0 0
0 1 5
.
.
. 2 1 0
0 0 8
.
.
. 5 2 1
_
_
_
90
Step 3 Clear the terms above leading diagonal of left-hand matrix
_
_
_
1 1 3
.
.
. 1 0 0
0 1 5
.
.
. 2 1 0
0 0 8
.
.
. 5 2 1
_
_
_
R
2
8R
2
5R
3

R
1
8R
1
+3R
3
_
_
_
8 8 0
.
.
. 7 6 3
0 8 0
.
.
. 9 2 5
0 0 8
.
.
. 5 2 1
_
_
_
R
1
R
1
+R
2

_
_
_
8 0 0
.
.
. 2 4 2
0 8 0
.
.
. 9 2 5
0 0 8
.
.
. 5 2 1
_
_
_
R
1
R
1
/8

R
2,3

1
8
R
2,3
_
_
_
_
_
1 0 0
.
.
.
1
4
1
2

1
4
0 1 0
.
.
.
9
8
1
4
5
8
0 0 1
.
.
.
5
8

1
4

1
8
_
_
_
_
_
A
1
=
_
_
_
_
_
1
4
1
2

1
4

9
8
1
4
5
8
5
8

1
4

1
8
_
_
_
_
_
Exercise: Check that AA
1
= I.
91
3.9 Iterative schemes
Gaussian elimination is often the method of choice for solving a linear
system of n equations in n unknowns, but:
it requires a large number of algebraic manipulations, and
it is unforgiving of any errors made in the elimination procedure.
Iterative methods are an alternative. They generate successive ap-
proximations to the solution following a rst guess.
Successive applications of the method give more and more accurate
approximations to the solution, until desired level of accuracy is rea-
ched.
92
3.9.1 Jacobi Iteration
Also known as method of simultaneous displacements.
Consider a system of n equations:
a
11
x
1
+ a
12
x
2
+ + a
1n
x
n
= b
1
a
21
x
1
+ a
22
x
2
+ + a
2n
x
n
= b
2
.
.
.
a
n1
x
1
+ a
n2
x
2
+ + a
nn
x
n
= b
n
.
_

_
(3.33)
Suppose:
(i) the solution to this system is unique (i.e., there is only one choice
of x
1
, . . . , x
n
that solves (3.33));
(ii) the diagonal coecients a
11
, a
22
, . . . , a
nn
are non-zero.
93
We can then rewrite each equation of the system (3.33) as follows:
(i) rearrange the rst equation to express x
1
in terms of the other x
j
s;
(ii) rearrange the second equation to express x
2
in terms of the other
x
j
s;
and so on.
Hence, this procedure gives
x
1
=
1
a
11
(b
1
a
12
x
2
a
13
x
3
a
1n
x
n
)
x
2
=
1
a
22
(b
2
a
21
x
1
a
23
x
3
a
2n
x
n
)
.
.
.
x
n
=
1
a
nn
(b
n
a
n1
x
1
a
n2
x
2
a
n(n1)
x
n1
)
_

_
(3.34)
94
If an initial approximation to the solution to (3.33) is made (e.g., by
guesswork) and then substituted into the right-hand side of (3.34), the
new values for x
1
, . . . , x
n
are an improved approximation to the solution.
If no initial guess is obvious we may take
x
1
= 0, x
2
= 0, . . . , x
n
= 0
as the initial guess.
This procedure can be repeated indenitely, so that the rened appro-
ximation is itself substituted back into (3.34) to get a better approxi-
mation yet.
95
3.9.2 Example
Solve the system of equations
20x
1
+ x
2
x
3
= 17
x
1
10x
2
+ x
3
= 13
x
1
+ x
2
+ 10x
3
= 18
(3.35)
by Jacobi iteration.
Solution: First rewrite the equations as
x
1
=
1
20
(17 x
2
+ x
3
)
x
2
=
1
10
(13 x
1
x
3
)
x
3
=
1
10
(18 + x
1
x
2
)
i.e.,
x
1
= 0.85 0.05x
2
+ 0.05x
3
x
2
= 1.3 + 0.1x
1
+ 0.1x
3
x
3
= 1.8 + 0.1x
1
0.1x
2
.
(3.36)
Since there is no obvious rst guess, try
x
1
= 0, x
2
= 0, x
3
= 0.
96
Then the rened guess is
x
1
= 0.85, x
2
= 1.3, x
3
= 1.8.
To improve on this, substitute back into (3.36) to give
x
1
= 0.85 + 0.05 1.3 + 0.05 1.8
= 1.005
x
2
= 1.3 + 0.1 0.85 + 0.1 1.8
= 1.035
x
3
= 1.8 + 0.1 0.85 0.1 (1.3)
= 2.015
97
Repeat process ad nauseam . . . .
initial rst 2nd 3rd
guess approx approx approx
x
1
0 0.85 1.005 1.0025
x
2
0 -1.3 -1.035 -0.9980
x
3
0 1.8 2.015 2.004
4th 5th 6th
approx approx approx
x
1
1.0001 0.99997 1.0000
x
2
-0.99935 -0.999985 -1.0000
x
3
2.00005 1.999945 2.0000
The solution appears to be converging to
x
1
= 1 x
2
= 1 x
3
= 2.
We can check that this is the solution by substituting these values into
the original equations (3.35).
98
3.9.3 Convergence of iterative schemes
Sometimes, Jacobi iteration fails to give an answer, even after many
iterations. In this case, the scheme is said to diverge.
On the other hand, the scheme converges if the solution can be de-
termined to any desired level of accuracy provided enough iterations
are performed.
99
3.10 Eigenvalues and eigenvectors
Eigenvalue problems arise in many situations, for example,
calculating the natural frequences of oscillation of a vibrating system;
nding principal axes of stress and strain;
calculating the oscillations of an electrical circuit.
An eigenvalue problem takes the form
Find all the values of for which the equation
Ax = x, (3.37)
has a nonzero solution x, where A is an n n matrix and x is
an n 1 matrix (column vector).
Such values of are called eigenvalues of the matrix A and the cor-
responding vectors x are the eigenvectors.
100
Equation (3.37) may be written as
Ax = Ix
Ax Ix = 0
(AI)x = 0. (3.38)
If det(AI) = 0, the matrix AI may be inverted, and the unique
solution to (3.38) is
x = (AI)
1
0 = 0.
However, nonzero solutions x will exist if
det(AI) = 0. (3.39)
There are innitely many eigenvectors corresponding to any given
eigenvalue. This is because: if x is an eigenvector of A corresponding to
the eigenvalue and c is a (nonzero) scalar thencx is also an eigenvector
of A.
To see this, note that A(cx) = cAx = cx = (cx).
101
If A is the 2 2 matrix
A =
_
a b
c d
_
then (3.39) is a quadratic equation for .
We can see this by calculating
|AI| =

_
a b
c d
_

_
1 0
0 1
_

a b
c d

=
2
(a +d) +(ad bc).
So for to be an eigenvalue of A,

2
(a +d) +(ad bc) = 0.
Use standard formula for solving a quadratic to obtain eigenvalues.
102
In general, if A is an nn matrix then det(AI) is a polynomial of de-
gree n in , called the characteristic polynomial. The characteristic
equation is
characteristic polynomial = 0,
i.e.,

n
+c
n1

n1
+c
n2

n2
+ +c
1
+c
0
= 0. (3.40)
Coecients c
0
, c
1
, . . . , c
n
can be found once the elements of A are
known.
The characteristic polynomial has n roots
1
,
2
, . . . ,
n
. Some of
these roots may be equal, in which case they are said to be repeated
roots.
103
3.10.1 Example
Find the eigenvalues and the corresponding eigenvectors of the matrix
A =
_
3 2
4 3
_
.
Step 1 Find the characteristic polynomial.
det(AI) = det
_
3 2
4 3
_
= (3 )(3 ) +8
=
2
9 +8 =
2
1
Step 2 Solve the characteristic equation.
The roots of
2
1 = 0 are = 1 and = 1.
Step 3 For each eigenvalue nd the corresponding eigenvector.
104
= 1
Write out
Ax = x
in full to give
_
3 2
4 3
__
x
y
_
= 1
_
x
y
_
=
_
x
y
_
,
where x =
_
x
y
_
. Hence
3x 2y = x x = y
4x 3y = y x = y.
Any vector of the form

_
1
1
_
is an eigenvector (where = 0 is an arbitrary constant).
105
= 1
Write out
Ax = x
in full to give
_
3 2
4 3
__
x
y
_
= 1
_
x
y
_
=
_
x
y
_
Hence
3x 2y = x 2x = y
4x 3y = y 2x = y.
Any vector of the form

_
1
2
_
is an eigenvector (where = 0 is an arbitrary constant).
106
3.10.2 Example
Find the eigenvalues and eigenvectors of
A =
_
_
_
1 2 1
0 1 2
1 0 3
_
_
_.
Step 1 Find the characteristic polynomial. The determinant det(AI)
is, expanding about the rst column,

1 2 1
0 1 2
1 0 3

= (1 )

1 2
0 3

2 1
1 2

= (1 )[(3 )(1 ) 2 0]
+[2 2 1 (1 )]
= (1 )(
2
+2 3) +( +3)
= (1 )( +3)( 1) +( +3)
= ( +3)(
2
+2) = ( 2)( +3).
107
Step 2 Solve the characteristic equation
( 2)( +3) = 0.
The solutions are 0, 2 and 3.
Step 3 For each eigenvalue, nd the corresponding eigenvector.
108
= 0
Write out
Ax = x
in full to give
_
_
_
1 2 1
0 1 2
1 0 3
_
_
_
_
_
_
x
y
z
_
_
_ = 0
_
_
_
x
y
z
_
_
_
Equivalently,
x +2y +z = 0 (1)
y +2z = 0 (2)
x 3z = 0 (3)
(3) x = 3z
(2) y = 2z
109
Substitute into (1):
x +2y +z = 3z 4z +z = 0.
So (1) is redundant (consistent)
Use these to write
x = 3
y = 2,
where z = .
Then eigenvector corresponding to = 0 is
_
_
_
3
2

_
_
_ =
_
_
_
3
2
1
_
_
_
for any = 0.
110
= 2
Ax = x
_
_
_
1 2 1
0 1 2
1 0 3
_
_
_
_
_
_
x
y
z
_
_
_ = 2
_
_
_
x
y
z
_
_
_
Equivalently,
x +2y +z = 2x
y +2z = 2y
x 3z = 2z
or
x +2y +z = 0 (1)
y +2z = 0 (2)
x 5z = 0 (3)
(3) x = 5z
(2) y = 2z
111
In (1): x +2y +z = 5z +4z +z = 0. So (1) is redundant
Therefore, if z = ,

_
_
_
5
2
1
_
_
_
is an eigenvector corresponding to = 2, for any = 0.
112
= 3
Ax = x
_
_
_
1 2 1
0 1 2
1 0 3
_
_
_
_
_
_
x
y
z
_
_
_ = 3
_
_
_
x
y
z
_
_
_
Equivalently,
x +2y +z = 3x
y +2z = 3y
x 3z = 3z
or
4x +2y +z = 0 (1)
4y +2z = 0 (2)
x = 0 (3)
113
(3) x = 0
(2) y = z/2
and (1) is redundant
Therefore

_
_
_
0

1
2
1
_
_
_
is an eigenvector corresponding to = 3, for any = 0.
114