EN2090 – Engineering Mathematics 2: Linear Algebra

Dr Phil Anderson
Room E/3.10
andersonpi1@cf.ac.uk

Contents
EN2090 – Engineering Mathematics 2: Linear Algebra ............................................................. 1
Contents.............................................................................................................................. 1
Why learn about matrices and linear algebra? ...................................................................... 3
Outline syllabus ...................................................................................................................... 3
Section 1: Review of basic matrix properties ............................................................................ 4
Aims of this section............................................................................................................. 4
Objectives of this section.................................................................................................... 4
1.1 Basic matrix properties ................................................................................................... 4
1.2 Transpose of a Matrix ..................................................................................................... 5
Worked Example 1.1........................................................................................................... 5
1.3 Special matrices .............................................................................................................. 5
1.4 Matrix addition and subtraction ..................................................................................... 6
Worked Example 1.2........................................................................................................... 6
1.5 Matrix multiplication ....................................................................................................... 7
Worked Example 1.3........................................................................................................... 7
1.6 Scalar product and unit vectors ...................................................................................... 8
Worked example 1.4........................................................................................................... 9
1.7 The determinant of a square matrix ............................................................................. 10
Worked Example 1.5......................................................................................................... 10
Some properties of determinants ........................................................................................ 11
1.8 The inverse of a square matrix ...................................................................................... 11
Worked Example 1.5 Inverse of a 22 matrix ............................................................... 12
Worked Example 1.6 Inverse of a 33 matrix............................................................... 12
1.9 Summary of Section 1 .................................................................................................... 13
1.10 Problems....................................................................................................................... 14
1.11 Solutions ....................................................................................................................... 14
Section 2: Solution of Linear Simultaneous Equations ............................................................ 16
Aims of this section........................................................................................................... 16
Objectives of this section.................................................................................................. 16
 2.1 Introduction .................................................................................................................. 16
Worked example 2.1......................................................................................................... 17
2.2 Solution of homogeneous equations ( A x = 0 ) ......................................................... 17
Worked example 2.2......................................................................................................... 18
2.3 Solution of A x = b by matrix inversion .................................................................. 18
Worked example 2.3......................................................................................................... 18
2.4 Solution of A x = b using Gaussian elimination ...................................................... 19
Worked example 2.4......................................................................................................... 20
2.5 General methodology for Gaussian elimination ........................................................... 22
Worked example 2.5......................................................................................................... 22
Worked example 2.6......................................................................................................... 24
2.6 Summary........................................................................................................................ 26
2.7 Problems......................................................................................................................... 27
Section 3 Eigenvalues, Eigenvectors and their Applications.................................................. 28
Aims of this section........................................................................................................... 28
Objectives of this section.................................................................................................. 28
3.1 Introduction ................................................................................................................... 28
3.2 Calculating the Eigenvalues ........................................................................................... 29
Worked example 3.1......................................................................................................... 29
Worked example 3.2......................................................................................................... 30
3.3 Useful properties of the eigenvalues ............................................................................ 31
3.4 Introduction to Eigenvectors ......................................................................................... 32
3.5 Calculating Eigenvectors................................................................................................ 32
Worked example 3.3......................................................................................................... 32
Worked example 3.4......................................................................................................... 33
Worked example 3.5......................................................................................................... 34
3.6 Application to the solution of coupled differential equations ...................................... 36
3.7 A note on repeated eigenvalues ................................................................................... 37
3.8 Summary........................................................................................................................ 37
3.9 Problems......................................................................................................................... 39
Why learn about matrices and linear algebra?
Matrices have widespread use in Engineering. The aim of this part of the course is to build
on the basic introduction to matrices from Engineering Mathematics 1. The outline syllabus
is given below.

We will begin with a review of the basic properties before introducing more advanced
methods of solving linear simultaneous equations.

The other main topic is eigenvalues and eigenvectors, which has applications across the
whole of Engineering, some examples being
• Frequency and mode analysis of mechanical or electrical vibrations
• Principal stress and principal axis analysis in rigid body mechanics
• Stability analysis of general dynamical systems

Outline syllabus

Section 1: Review of basic matrix properties

Section 2: Advanced methods for solving linear simultaneous equations
Section 3: Eigenvalues and Eigenvectors
Section 1: Review of basic matrix properties
Aims of this section
To review all of the basic matrix concepts introduced in Engineering mathematics 1

Objectives of this section

After completing this section you should be able to
• Describe different types of matrices
• Perform simple matrix addition, subtraction and multiplication
• Calculate the scalar product of two N dimensional vectors
• Evaluate the unit vector for any given vector
• Evaluate the determinants of 2x2 and 3x3 matrices
• Evaluate the inverses of 2x2 and 3x3 matrices

1.1 Basic matrix properties

A matrix is a rectangular array of elements. The order of a matrix is the number of rows by
number of columns (m  n). The element a is in row i and column j (note: rows first, then
ij
columns).

A square matrix has m=n  a11 a12 a13  a1 j  a1n 

 
A column vector has n = 1  a21 a22 a23  a2 j  a2 n 
(i.e. one column, order m a a32 a33  a3 j  a3n 
 1).  31 
      
A row vector has m = 1
a ai 2 ai 3  aij  ain 
 i1 
(i.e. one row, order 1  n).
      
a am2 am3  am j  amn 
 m1
1.2 Transpose of a Matrix
If matrix A has elements aij , its transpose (denoted AT) has elements aji (i.e. interchange
rows and columns). A square matrix is symmetric if it equals its transpose (i.e. A = AT), skew-
symmetric if A = -AT.

In this module we will only deal with square matrices (order N  N), column vectors (order N
 1) and row vectors (order 1  N). The integer N is the dimensionality of the vector space in
which we operate.

Worked Example 1.1

 0 1 − 1 is a 3  3 matrix with transpose AT =  0
A=
2 1
  
2 − 3 1   1 − 3 0
1 0 4   −1 1 4
  

1
B = 
2 3 1 2 3
 has transpose BT =   = B, i.e. B is symmetric
2 5 0 2 5 0
3  3 6 
 0 6  0

 1 
x =   is a 3  1 column vector with transpose xT = (1 − 2 3)
−
 2
 3 
 
 2 
y = (2 0 − 1) is a 1  3 row vector with transpose yT =  
 0 
 − 1
 

Note: the transpose of a column vector is a row vector (and vice versa).

1.3 Special matrices

We have already mentioned symmetric and skew-symmetric matrices.

 2 0 0
e.g. C =  
A diagonal matrix has elements equal to zero apart
0 − 3 0
from along its diagonal. 0 0 7
 
 1 0 0
 
The identity (or unit) matrix I is diagonal, with all I = 0 1 0
diagonal elements equal to unity. 0 0 1
 

 1 4 3
e.g. D =  
 0 − 3 5 
A triangular matrix has non-zero elements only in its
 0 0 1
lower or upper corners.  

0 0 0
0 =  
The null (or zero) matrix 0 has all its elements equal 0 0 0
0 0 
to zero.  0

(Note: these are shown for 33 matrices but can apply to any order).

1.4 Matrix addition and subtraction

Two matrices can be added (or subtracted) provided they have the same order, e.g. a 33
matrix can be added to a 33 matrix, a 31 column vector can be added to a 31 column
vector, a 13 row vector can be added to a 13 row vector. We simply add (or subtract) the
corresponding elements of the two matrices, i.e. if C = A  B then the elements of C are cij =
aij  bij.

Worked Example 1.2

 0 1 − 1 1 3  1 
 , x =   , y = (2 0 − 1)
2
If A =  , B = 
2 − 3 1  2 5 0  − 2
1 0 4  3 6   3 
  0  

then find: A + B, A – B, xT + y, yT – x

 0 + 1 1 + 2 − 1 + 3  1 3 2 
A+B=     ,
2 + 2 − 3 + 5 1+ 0  = 4 2 1 
1+ 3 0 + 0 4 + 6   4 0 10 

  −1 −1 − 4
A–B=  
 0 −8 1 
 − 2 0 − 2
 

xT + y = (1 − 2 3) + (2 0 − 1) = (3 − 2 2) ,

 2 −1   1 
yT - x =    
 0 − −2  =  2 
 − 1 − 3  − 4
   

1.5 Matrix multiplication

A matrix product (e.g. C = AB) is only defined if the number of columns in the first matrix A
equals the number of rows in the second B.

N
If C = A B then the elements of the product C are:
cij =  aik bkj
k =1

Note: m  k matrix multiplied by k  n matrix gives m  n matrix

Only the following multiplications are allowed using N dimensional square matrices and
vectors (row or column):

• NN matrix multiplied by NN matrix = another NN matrix

• NN matrix multiplied by N1 column vector = N1 column vector
• 1N row vector by N1 column vector = 11 matrix (i.e. a number)
• N1 column vector multiplied by 1N row vector = NN matrix

Note that whilst A  B = B  A, in general AB  BA (i.e. addition of two matrices is

commutative, multiplication is non-commutative).

Worked Example 1.3

 0 1 − 1 1 2 3  1 
If A =  , B =   , x =   , then find the following:
2 − 3 1  2 5 0  − 2
1 0  3   3 
 4  0 6  

xTx, Ax, AB
  1 
x Tx =  
(1 −2 3) − 2  = (1  1) + (− 2  −2 ) + (3  3) = 14
 3 
 

 0 1 − 1 1   0  1 + 1  −2 + − 1  3   − 5 
Ax =       
 2 − 3 1  − 2  =  2  1 + − 3  −2 + 1  3 =  11 
1 0 4  3   1  1 + 0  −2 + 4  3   13 


 0 1 − 1  1
AB = 
2 3
 
2 − 3 1  2 5 0
1 0 4   3 6 
 0

  0 1 + 1 2 +  −1 3 0  2 + 1 5 +  −1 0 0  3 + 1 0 + −1 6 

 
=   2 1 +  −3  2 + 1 3  2  2 +  −3  5 + 1 0 2  3 +  −3  0 + 1 6 
 11 +  0  2 +  4  3 1 2 + 0  5 +  4  0 1 3 + 0  0 + 4  6 


 −1 5 −6 
 
=  −1 −11 12 
 13 2 27 
 

1.6 Scalar product and unit vectors

For the row vector y = ( y1 y2  y N ) then:

yyT = y12 + y22 +  + y N2

The magnitude of y is defined as:

y = yy T = y12 + y22 +  + y N2 .

The same applies to a column vector x, except its magnitude is defined to be:

x = x T x = x12 + x22 +  + xN2 .

The unit vector of any vector z (row or column) is defined as:

z
zˆ =
z

i.e. ẑ is parallel to z but has unit magnitude ( z = 1).

Note: the product abT (for row vectors) or aTb (for column vectors) is defined as the scalar
(or “dot”) product of the two vectors a and b.

Two vectors are said to be orthogonal if abT = 0 (for row vectors) or if aTb = 0 (for column
vectors). The scalar product of a vector with itself enables the magnitude of the vector to be
calculated.

Worked example 1.4

 − 2  2
Calculate the magnitudes and unit vectors of x =   and y =   . Also verify that x and
 3  0
 1   4
   
y are orthogonal.

x  − 2
x= x x= T
(− 2) + (3) + (1) = 14
2 2 2
→ xˆ = = 1  
 3 
x 14  
 1 

( 2) + ( 0 ) + ( 4 ) = 20 = 2 5
2 2 2
y = yTy =

1
y 1  
→ yˆ = = 0
y 5  
 2
  2
 
xT y = ( −2 3 1)  0  = ( −2  2 ) + ( 3  0 ) + (1 4 ) = 0
 4
 

Since xTy = 0, x and y must be orthogonal.

1.7 The determinant of a square matrix

The determinant of a square matrix A is a number, written det (A ) or A .

a11 a12
22:
det (A ) = = a11a22 − a12 a21
a21 a22

a11 a12 a13

33: a a23 a a a a
= a21 a22 a23 = a11 22 − a12 21 23 + a13 21 22
a32 a33 a31 a33 a31 a32
a31 a32 a33

etc. for 44 and higher (done by computer in practice).

• Any row can be used to find det(A) but we usually use the first.
• If det(A) = 0 then the matrix A is said to be singular.

Worked Example 1.5

−1 2 0
1 1 3 1 3 1
3 1 1 = −1 −2 +0 = −3 − 8 + 0 = −11
−1 2 2 2 2 −1
2 −1 2
Some properties of determinants
We will illustrate these properties
a b 
using the 22 matrix A (opposite): A= , det (A ) = ad − bc
 c d 
If two rows (or columns) are swapped,
c d 
det(A) changes sign. B= , det (B ) = bc − ad
 a b 

The determinant of the transpose

a c 
A =
T
, det A T = ad − bc ( )
equals that of the original matrix. b d 
 a row (or column) by a constant k  ka b 
means determinant  k also C= , det (C) = k (ad − bc )
 kc d 
At least one row (or column)
a b
duplicated gives determinant = 0. D= , det (D) = akb − bka = 0
 ka kb 
Adding to any row (or column) a
multiple of another row (or column)  a b 
gives same determinant. E= , det (E ) = ad − bc
 ka + c kb + d 

1.8 The inverse of a square matrix

−1
The inverse of a square matrix A is written A . It is defined such that A −1 A = AA −1 = I
(where I is the identity matrix), and is found to be

adj(A ) where adj(A ) is matrix called the adjoint of A

A −1 =
det (A )
Worked Example 1.5 Inverse of a 22 matrix

1 2
A= 
3 4
 4 − 2 − 2 1 
→ det (A ) = −2, adj(A ) =   → A −1 =  
−3 1   1.5 − 0.5 

− 2 1  1 2   1 0 
Check:
A −1 A =   =  (as required)

 1.5 − 0.5  3 4   0 1 

Worked Example 1.6 Inverse of a 33 matrix

 1 − 2 1
 
A =  −1 1 0 → det (A ) = 1 (1) + 2  (− 1) + 1  (3) = 2
 − 2 −1 1
 

The inverse of a NN matrix is another NN + − +

matrix, here a 33 matrix. Next write down a 33  
place sign matrix, whose signs alternate.  − + − 
+ − +
 
The minor at each position ij is found by striking
out the ith row and jth column and finding the  + (1) − (− 1) + (3) 
determinant of what’s left. Next, the cofactor  
matrix C is found by inserting minors.  − (− 1) + (3) − (− 5)
 + (− 1) − (1) + (− 1) 
 

The adjoint of A is simply the transpose of the cofactor matrix, hence

  1 1 − 1  0.5 0.5 − 0.5 
adj(A ) C 1
T   
A −1 = = =  1 3 − 1 =  0.5 1.5 − 0.5 
det (A ) det (A ) 2    
 3 5 − 1  1.5 2.5 − 0.5 
In practice, and particularly for larger matrices, complex operations such as finding the
inverse are made straightforward using software packages such as Matlab, Mathcad or
Mathematica. See below for an example in Matlab using the inv() command.

>> A=[1 -2 1; -1 1 0; -2 -1 1]

A=

1 -2 1
-1 1 0
-2 -1 1

>> inv(A)

ans =

0.5000 0.5000 -0.5000

0.5000 1.5000 -0.5000
1.5000 2.5000 -0.5000

1.9 Summary of Section 1

This section has covered the relevant parts of matrix analysis in preparation for the next two
sections on matrix applications.

You should now be able to

• Define a matrix and be able to label (and locate) its elements
• Perform basic matrix algebra, including addition and multiplication
• Describe some special square matrix types
• Calculate the unit vector of any given vector
• Calculate the scalar product of any two vectors
• Calculate the determinant of any 22 or 33 matrix
• State and apply the properties of a determinant
• Calculate the inverse of any 22 or 33 matrix
1.10 Problems
1  3 1  2 1
1.1 Given x =  , y = (1 − 3), A =  , B =   , calculate
 2  − 1 1  0 6
(a) yx, (b) xy, (c) xTx, (d) Ax, (e) By, (f) AB and (g) BA

 2  1 0 1  − 3 1 2
     
1.2 Given u =  0 , v = (4 1 2), C =  2 1 1 , D =  − 1 1 0  , calculate
 − 2 1 0 2  3 2 1
     
T
(a) vu, (b) u u, (c) Cu, and (d) CD

1.3 Calculate the magnitudes and unit vectors of x, y, u and v in P1.1 and P1.2.

1.4 Write down two vectors that are orthogonal to x and two vectors orthogonal to u.

1.5 Calculate the determinants of matrices A, B, C and D.

1.6 Demonstrate the various determinant properties listed in section 1.2 applied to
det(C).

1.7 Calculate the inverses of matrices A, B, C and D. Verify your results by performing
the check AA −1 = I , etc. for B, C and D.

1.11 Solutions
 1 − 3  5
1.1 (a) yx = −5, (b) xy =  , (c) x T x = 5, (d) Ax =   , (e) doesn’t exist, since
 2 − 6 1
 6 9  5 3
no. of columns in B  no. rows in y, (f) AB =   , (g) BA =  
 − 2 5  − 6 6

 0  0  0 3 3
     
1.2 (a) vu = 4, (b) uTu = 8, (c) Cu =  2  = 2 1  , (d) CD =  − 4 5 5 
 − 2   − 1  3 5 4
     

 1  1
x  5 1 1 u 1  
1.3 x x=5 → x=
T
ˆ = =  , u u = 8 → uˆ =
T
= 0
5  2 
 5  2  8 2 
 5
 − 1

yy T = 10 → yˆ =
y
=
1
(1 − 3) , vv T = 21 → vˆ = v = 1 (4 1 2)
10 10 21 21
 1  − 1
2  − 2    
1.4   and   are orthogonal to x;  0  and  0  are orthogonal to u.
 − 1  1  1  − 1
   

1.5 det (A ) = 4, det (B ) = 12, det (C) = 1, det (D) = −12

0 1 1 1 2 1
1.6
 
( )
Swap cols 1 and 2 → 1 2 1 = −1 ; C =  0 1 0  → det C T = 1 = det (C);
T

1 1 2
0 1 2  
1 0 1
Multiply row 3 by 5 → 2 1 1 = 5 = 5 det (C) ;
5 0 10
2 0 1
Replace col 1 with (col 1 + col 3) → 3 1 1 = 1 = det (C)
3 0 2
1 1 − 1  0.25 − 0.25  1  6 − 1
1.7 A −1 =  = , B −1 =  
4 1 3   0.25 0.75  12  0 2 
 2 0 − 1  −1 − 3 2
−1   1 
C = −3 1 1 , D −1 =  −1 9 2
 −1 12 
 0 1   5 − 9 2

Section 2: Solution of Linear Simultaneous Equations
Aims of this section
To study the use of matrix methods to solve a system of linear simultaneous equations, with
emphasis on Gaussian elimination.

Objectives of this section

After completing this section you should be able to
• Describe homogeneous and non homogeneous simultaneous equations
• Describe the nature of their solutions
• Solve non homogenous equations using matrix inversion
• Solve non homogeneous equations using Gaussian elimination

2.1 Introduction
One of the main practical applications of matrices is in the solution of sets of linear
equations. Consider the following system of n linear simultaneous equations (“linear”
means no powers of x greater than 1).

a11 x1 + a12 x2 +  + a1n xn = b1

a21 x1 + a22 x2 +  + a 2 n xn = b2
   
an1 x1 + a n 2 x2 +  + ann xn = bn

We next write these equations in matrix form, giving

 a11 a12  a1n   x1   b1 

    
 a21 a22  a2 n   x2   b2 
  =
       
     
 an1 an 2  ann   xn   bn 

or, equivalently Ax = b
A is a n×n square matrix, x and b are n×1 column vectors.
Worked example 2.1
The following two examples show how to cast systems of two and three linear simultaneous

equations into matrix form: Ax = b

2x + y = 5 2 1  x   5 
→    =  
x − 2y = −5  1 − 2  y   − 5

x + y + z = 6 1 1 1   x   6 
    
x + 2 y + 3z = 14 →  1 2 3   y  =  14 
x + 4 y + 9 z = 36 1 4 9   z   36 
    

−1 −1
To solve A x = b , find the inverse of A (i.e. A ), multiply both sides by A and use
−1 −1
the fact that A A = AA = I (where I is the identity), i.e.

A −1 A x = I x = x = A −1b

A −1 exists (i.e. A is not singular, det (A )  0 ), then the solution of

Hence, provided that
−1
A x = b is x = A b (note the order of multiplication).

2.2 Solution of homogeneous equations ( A x = 0 )

Homogeneous equations are of the form A x = 0 (i.e. b = 0 ).

Recall the inverse of A is A −1 = adj(A ) / det (A ).

There are two cases to consider.

det (A )  0 and b = 0 If det (A )  0 then A exists (  0 ) and solution of

−1

−1
A x = 0 is x = A 0 , i.e. the only solution is the trivial solution x = 0 .
det (A ) = 0 and b = 0 Conversely, there are non-trivial solutions (i.e. x  0 ) of
A x = 0 when det (A ) = 0 .
This important result underpins the theory of eigenvalues and eigenvectors.

Worked example 2.2

Find the value of k for which this set of homogeneous equations have non-trivial (i.e. non-
zero) solutions. Find the solutions.

x + 5 y + 3z = 0
5 x + y − kz = 0
x + 2 y + kz = 0

Writing as A x = 0 , det (A ) = 0 → 1(k + 2k ) − 5(5k + k ) + 3(10 − 1) = 0

i.e. k = 1.

There are an infinite number of non-zero solutions, i.e. x =  , y = −2 and z = 3

( is any number, which could also be zero).

2.3 Solution of A x = b by matrix inversion

Non homogeneous equations are of the form A x = b with b  0 . In this case with
b  0 , if det (A )  0 then A
−1
exists and (as we’ve seen) there is one unique
solution, given by x = A −1b . More subtly, if det (A ) = 0 there is either no solution or
an infinite number of solutions.

Worked example 2.3

Find the solution to the second set of equations of worked example 2.1

1 1 1   x   6 
Writing in the form A x = b gives     
 1 2 3   y  =  14 
1 4 9   z   36 
    
 det (A ) = 2  0 , there is one unique solution given by x = A b
−1
Since

1 6 −5 1 𝑥
𝑨−𝟏 = (−6 8 −2) → (𝑦)
2 𝑧
2 −3 1

1 6 −5 1 6 1
= (−6 8 −2) (14) = (2)
2
2 −3 1 36 3

𝑥=1
i.e. 𝑦 = 2
𝑧=3

2.4 Solution of A x = b using Gaussian elimination

Matrix inversion is easy for solving two or three simultaneous equations but it becomes
unwieldy for four (or more) equations. It is more efficient to use an elimination method, e.g.
Gaussian elimination.

To illustrate its use, consider this simple example, where we need to solve:

x − 2y = −5 (1)
2x + y = 5 ( 2)

Subtract eqn (2) from 2 × eqn (1), giving a new eqn (2):

x − 2y = −5 (1)
− 5y = − 15 ( 2)
Eqn (2) now only has one unknown (y), so is used to find y = 3. On inserting the solution
for y into eqn (1), there is now only one remaining unknown (x) and rearranging eqn (1)
gives x = 2 y − 5 = 1.
Here we have actually reduced the original matrix equation Ax = b into a new form

U x = d where U is an upper-triangular matrix.

1 − 2 1 − 2
A=  → U= 
 2 1   0 5 
The general process of Gaussian elimination uses the row operations:

• Multiply an equation (i.e. row) by a constant

• Interchange any two equations (i.e. rows)
• Add or subtract one equation (i.e. row) from another

These transform the original equation set A x = b into U x = d , i.e.

 a11 a12  a1n   x1   b1   u11 u12  u1n   x1   d1 

         
 a21 a22  a2 n   x2   b2   0 u  u 2 n   x2   d 2 
  = → 22
=
                 
           
 an1 an 2  ann   xn   bn   0 0  unn   xn   d n 
original matrix upper-triangular matrix

Row n (Rn) now has only one unknown (xn), thus xn = d n / unn .

Rn−1 is then used to find xn −1 (using xn ), etc. for Rn−2  R1 until all the x1  xn
are found

Worked example 2.4

Gaussian elimination reduces a system of non homogeneous equations A x = b into the
upper-triangular form shown below. Now solve for x
R1 2 1 −1 0 1   x1   0  5 unknowns
    
R2 0 3 0 1 1   x2   − 2  4 unknowns
R3 0 0 1 −2 2   x3  =  4  3 unknowns
    
R4 0 0 0 1 − 1   x4   − 1  2 unknowns
0 1   x5   1 
R5  0 0 0 1 unknown
From R5 , x5 = 1
From R ,
4 x4 − x5 = −1 → x4 = 0
From R ,
3 x3 − 2 x4 + 2 x5 = 4 → x3 = 2
From R ,
2 3x2 + x4 + x5 = −2 → x2 = −1
From R ,
1 2 x1 + x2 − x3 + x5 = 0 → x1 = 0
Note: each row yields one extra solution until the full solution is found.

2.5 General methodology for Gaussian elimination

From A x = b we form the augmented matrix B by combining A with b.

 a11 a12  a1n   x1   b1   a11 a12  a1n b1 

      
 21 22
a a  a 2 n   x2   b2  a a  a2 n b2 
= → B =  21 22
              
       
 an1 an 2  ann   xn   bn   an1 an 2  ann bn 
Next, we use the elementary row operations

• Multiply a row by a constant

• Interchange any two rows
• Add or subtract one row from another

in any order to reduce the first column of B to zero (apart from the first element a11 ), then
reduce the second column of B to zero apart from the first two elements, etc., until B is
converted into an upper-triangular matrix. This is best illustrated by worked example.

Worked example 2.5

Use Gaussian elimination to solve the following:

(a ) x + y + z = 6 (b) 2 x − y − z = 11
x + 2 y + 3z = 14 x + 2 y − 3z = 3
x + 4 y + 9 z = 36 3x + 2 y + z = − 5
Writing each set of equations as A x = b , there will be a unique solution if
det (A )  0 , so remember to check this first.
(a) det (A ) = 2 (i.e.  0) so there will be one unique solution.

R1 1 1 1 6
The augmented matrix (with rows labelled) is  
R2 1 2 3 14 
1 4 9 36 
R3 

R1 1 1 1 6
Replace R2 with R2 − R1, thus forming  
R2 0 1 2 8
1 36 
R3  4 9
R1 1 1 1 6
Replace R3 with R3 − R1, thus forming  
R2 0 1 2 8
0 30 
R3  3 8

R1 1 1 1 6
Replace R3 with R3 − 3R2 , thus forming  
R2 0 1 2 8
0 0 2 6 
R3 
Finally, re-write in the form U x = d , with U an upper-triangular matrix (you can skip this
step when you get more familiar with the method).

1 1 1 x  6
    
 0 1 2  y  =8
0 0 2 z  6
    

2 z = 6 → z = 3;
Therefore,
y + 2z = 8 → y = 8 − 2z = 2
x + y + z = 6 → x = 6− y − z =1
Hence, the equations are solved. This is often a lot less painful than when using matrix
inversion, particularly when there are > 3 equations.
(b) det (A ) = 30 (i.e.  0) so there will be one unique solution.

R1  2 −1 −1 11
The augmented matrix is  
R2 1 2 −3 3
3 − 5 
R3  2 1

R2 → 2 R2 − R1
R1  2 −1 −1 11
 
R2  0 1 −1 −1
R2 → R2 / 5 3
R3  2 1 − 5 

R3 → 2 R3 − 3R1
R1  2 −1 −1 11 
 
R2 0 1 −1 −1 
0 7 − 43 
R3  5

R3 → R3 − 7R2 ; R1  2 −1 −1 11
 
R3 → R3 / 12
R2  0 1 −1 −1
0 0 1 − 3 
R3 

 z = −3; y − z = −1 → y = −4; 2 x − y − z = 11 → x = 2

Worked example 2.6

Use Gaussian elimination to solve the following sets of equations:

(a ) x + 2 y + 3z = 10 ( b) 2 x + 3 y + 4 z = 1
−x + y + z = 0 x + 2 y + 3z = 1
+ y − z = 1 x + 4 y + 5z = 2
( c) w + 2 x + 3y + z = 5
2w + x + y + z = 3
w + 2x + y = 4
x + y + 2z = 0

(a) det (A ) = −7 (i.e.  0), i.e. one unique R1  1 2 3 10 

solution. The augmented matrix is shown opposite. To  
solve, use the process: R2  −1 1 1 0
 0 −1 1 
R3  1

R2 → R1 + R2 ; R3 → R2 − 3R3 R1 1 2 3 10 
 
R2 0 3 4 10 
0 0 7 
R3  7

 7 z = 7 → z = 1; 3 y + 4 z = 10 → y = 2;
x + 2 y + 3z = 10 → x = 3

(b) det (A ) = −2 (i.e.  0), so one unique R1 2 3 4 1

solution. The augmented matrix is:  
R2 1 2 3 1
To solve, use the process: 1 2 
R3  4 5

R2 → 2 R2 − R1; R3 → 2 R3 − R1; R1 2 3 4 1
 
R3 → 5R2 − R3, thus leading to the solutions R2 0 1 2 1
0 2 
x = − 12 , y = 0, z = 12 R3  0 4

(c) det (A ) = 12 (i.e.  0), so one unique solution. To solve, fist form the augmented
matrix and solve using the process:
 𝑅2 → 2𝑅1 − 𝑅2 ; 𝑅3 → 𝑅1 − 𝑅3 ; 𝑅4 → 3𝑅4 − 𝑅2 ; 𝑅4 → 𝑅3 + 𝑅4

R1 1 2 3 1 5 R1 1 2 3 1 5
   
R2 2 1 1 1 3 R2 0 3 5 1 7
1 →
R3 2 1 0 4 R3 0 0 2 1 1
   
R4 0 1 1 2 0  R4 0 0 0 6 − 6 

The solution is w = 1, x = 1, y = 1, z = −1

2.6 Summary

This section has covered the solution of linear simultaneous equations.

Homogeneous equations have the form A x = 0 . There are only non-zero solutions for x
(an infinite number of them) when det (A ) = 0 .

Non homogenous equations have the form A x = b (with b  0), and have a single
unique solution for x when det (A )  0 . The solution is obtained by matrix inversion or
(often more conveniently) by elimination methods (e.g. Gaussian elimination).

Gaussian elimination proceeds by reducing the set of equations A x = b (using

elementary row operations) to U x = d , where U is an upper-triangular matrix; the
solution x is found by simple back substitution.

After completing this section you should be able to use Gaussian elimination to find the
solution of A x = b , applied to any number of linear simultaneous equations.
2.7 Problems
2.1 Determine the values of the constants p, q and r for which the following sets of
homogeneous equations have non-trivial (i.e. non-zero) solutions.

3x + y + 2 z = 0 x + y − rz = 0
3 px + 2 y = 0
(a) (b) 4 x + 2 y − qz = 0 (c) rx − 3 y + 11z = 0
2 x + py = 0
2 x − y + 3qz = 0 2 x + 4 y − 8z = 0

2.2 Use Gaussian elimination to solve these sets of simultaneous equations (remember to
check that a unique solution exists before proceeding – see the worked examples in the
lecture notes for guidance).

2x + y = 3 3x + 2 y = −1 8 x + 5 y = 29.5
(a) (b) (c)
x − y =1 7x + 3y = 6 − 3x + 9 y = 27

2.3 Use Gaussian elimination to solve these sets of simultaneous equations (remember to
check that a unique solution exists before proceeding).

x + 3y + 2z = 1 y+z =6 1.17 x + 2.64 y + 7.41z = 1.27

(a) 2 x + y + 4 z = 2 (b) 3 x − y + z = −7 (c) 3.37 x + 1.22 y + 9.64 z = 3.91
3x − y + 5 z = 1 x + y − 3 z = −13 4.10 x + 2.89 y + 3.37 z = 4.63

2.4 Use Gaussian elimination to solve

4w − x − z = −4
− w + 4x − y = 1
− x + 4y − z = 4
− w − y + 4 z = 10
Section 3 Eigenvalues, Eigenvectors and their Applications

Aims of this section

To introduce the eigenvalues and eigenvectors of a matrix, and their methods of calculation.

Objectives of this section

After completing this section, you should be able to

• Understand why non-trivial solutions of the eigenvalue equation Ax = x exist

only if det (A − I ) = 0
• Derive the characteristic equation and be able to solve it to find the eigenvalues of a
given square matrix
• Use the eigenvalues to derive unit eigenvectors of a square matrix
• Demonstrate that the eigenvectors of a symmetric matrix are orthogonal
• Apply eigenvalue analysis to the solution of coupled differential equations

3.1 Introduction

In many areas of Engineering we meet a set of non homogeneous, linear simultaneous

equations of the form Ax = x . Their trivial solution is x = 0 ; of more practical
importance are their non-zero solutions x  0 .

If x  0 and A is an n × n matrix, there are generally n values of the scalar quantity , the
eigenvalues of A. Each eigenvalue is associated with a non-zero solution x (which is an n × 1
column vector); these solutions x are the eigenvectors of A (generally there are n of them).

To find the eigenvalues and eigenvectors of A, we recast Ax = x into a system of

homogeneous equations using

Ax = x = I x → ( A − I ) x = 0
i.e.
 a11 a12  a1n   x1   x1   a11 −  a12  a1n   x1   0 
         
 21
a a  a 2n   2 
x  2
x  21a a −   a 2 n   x2   0 
 
22
=  → 22
=
                 
           
 n1
a a n2  a nn   n 
x  n
x  n1
a a n2  a nn −    xn   0 
As we found in section 2.2, the new system of homogeneous equations ( A − I ) x = 0
has non-zero solutions ( x  0 ) only if det (A − I ) = 0 , i.e.

a11 −  a12  a1n

a a22 −   a2 n
det (A − I ) = 21 =0
  
an1 an 2  ann − 

Direct evaluation of the determinant gives an nth order polynomial in , called the
characteristic equation of A, whose solutions are the n eigenvalues.

i.e. det (A − I ) = c0 + c1 + c2 2 + c33 + cn −1n −1 + cn n = 0

The coefficients cn of the characteristic equation are determined by the elements of A.

The process of finding the eigenvalues (and eigenvectors) of a matrix is usually
straightforward and is best illustrated by worked examples.

3.2 Calculating the Eigenvalues

Worked example 3.1

Find the characteristic equation  2 −1  and 3 2
and eigenvalues of A=  B= 
 − 1 2   1 1 
A is a 2×2 matrix, so its characteristic equation is a quadratic, with two solutions (i.e.
eigenvalues) found by solving:

2 −  −1
det (A − I ) = = (2 −  )(2 −  ) − 1 = 0
−1 2 − 
 2 − 4 + 3 = 0 → ( − 1)( − 3) = 0 →  = 1 and  = 3
B is also a 2×2 matrix, whose characteristic equation is again a quadratic, yielding two
eigenvalues, found by solving:

3− 2
det (A − I ) = = (3 −  )(1 −  ) − 2 = 0 → 2 − 4 + 1 = 0
1 1− 
This doesn’t factorise, so use − b  b 2 − 4ac 4  12
= = = 2 3
2a 2
Worked example 3.2
 1 1 − 2  2 0 1
Find the characteristic    
A =  −1 2 1  and B =  − 1 4 − 1
equation and
 0 1 − 1   −1 0 
eigenvalues of   2

The characteristic equation of a 3×3 matrix like A will be a cubic equation yielding three
eigenvalues, found by setting det (A − I ) = 0 , i.e.

1−  1 −2
−1 2 −  1 = (1 −  ) ( − ( 2 −  )(1 +  ) − 1) − (1 +  ) + 2 = 0
0 1 −1 − 

→ − (1 −  ) ( 2 −  )(1 +  ) + 1 + (1 −  ) = 0

→ (  − 1) (  2 −  − 2 ) = 0 → (  − 1)(  − 2 )(  + 1) = 0

Hence, the eigenvalues of A are  = −1, 1 and 2 . Likewise, for B:

2− 0 1
( )
− 1 4 −  − 1 = (2 −  ) 2 − 4 + 2 + (2 −  ) = ( − 2)( − 1)( − 3) = 0
−1 2 −
Hence, the eigenvalues of B are  = 1, 2 and 3.
3.3 Useful properties of the eigenvalues

Suppose A is an n×n matrix with elements aij and eigenvalues 1 ,  2 ,  3   n . The

trace of A is the sum of its diagonal elements.

We’ll state the following useful eigenvalue properties without proof:

n n
•
  i =  aii = trace(A )
Sum of eigenvalues is the trace of A, i.e.
i =1 i =1
n
• Product of eigenvalues is the determinant of A, i.e.
 i = det (A )
i =1
• The eigenvalues of A T
are also 1 ,  2 ,  3   n
• The eigenvalues of A −1 are 1 1 1 1
, , 
1  2  3  n

• If k is any scalar, the eigenvalues of kA are k1 , k 2 , k 3  k n

• The eigenvalues of A + kI are 1 + k ,  2 + k ,  3 + k   n + k

3.4 Introduction to Eigenvectors

If A is an n×n matrix with eigenvalues 1 ,  2 ,  3   n , then for each eigenvalue there

will be an eigenvector (an n×1 column vector), i.e.
eigenvalue  has eigenvector x , eigenvalue  has eigenvector x 2 , etc.
1 1 2

The calculation of eigenvectors is best done by worked examples.

3.5 Calculating Eigenvectors

Worked example 3.3

Calculate the eigenvectors for matrix A in worked example 3.1.

A is a 2×2 matrix with 2 eigenvalues and 2 eigenvectors. Since, by definition Ax = x ,

first substitute the eigenvalue  = 1 to find x , i.e.
1 1
 2 − 1 x1   x1  2 x1 − x2 = x1
Ax1 = 1x1 →    = 1  →
 − 1 2  x2   x2  − x1 + 2 x2 = x2

x1 = x2 , so that if x1 =  (where  is any scalar), so

Both of these equations imply that
too is x =  and there are an infinite number of eigenvectors x , all with components
2 1
in a fixed ratio x = x =  .
1 2
To get around this problem, we deal only with the unit eigenvector, of which there is only
one (the method of finding unit vectors is discussed in detail in section 1.5). Hence, for x
1
we need to find x̂1 , etc. for x̂ 2 .

 
→ x1T x1 = (  )  = 2 2
x1 =  
 
1    1 1  2 
1
x1
→ xˆ 1 = T =  =     1  (final answer)
x1 x1 2  
  2 1  2 
To find x 2 (and hence x̂ 2 ), next we substitute the eigenvalue  2 = 3

 2 − 1 x1   x1  2 x1 − x2 = 3 x1
Ax 2 =  2 x 2 →    = 3  →
 − 1 2  x2   x2  − x1 + 2 x2 = 3 x2
Again writing x1 =  , both equations agree that x2 = − x1 = − , so

  1    1  1   
1
x2 2
x2 =   → ˆ2
x = =  =   1 
−  T
x2 x2 2  −  2  − 1  − 2

Worked example 3.4

Calculate the unit eigenvectors of matrix B in worked example 3.2.

B is a 3×3 matrix with 3 eigenvalues and 3 eigenvectors. For 1 = 1:

 2 0 1   x1   x1  2 x1 + x3 = x1
    
Bx1 = 1x1 →  −1 4 − 1  x2  = 1 x2  → − x1 + 4 x2 − x3 = x2
 −1 0   x3   x3  − x1 + 2 x2 = x3
 2
Using row 1, if x1 =  then x3 = − ; using row 3, 2 x2 = x1 + x3 = 0
(note: no further information can be gained from row 2; it just gives consistency regarding
what we’ve just found for x1 , x2 and x3 ).

     1   12 
  x1 1   1  
 x1 =  0  → xˆ 1 = =  0 =  0   0 
− 
T
x1 x1 2  −  2  − 1  − 1 
       2

We now repeat this process for the other two eigenvalues  2 = 2 and  3 = 3 to find the
other two eigenvectors x 2 and x 3 .
 2 0 1   x1   x1  2 x1 + x3 = 2 x1
    
Bx 2 =  2 x 2 →  − 1 4 − 1  x2  = 2 x2  → − x1 + 4 x2 − x3 = 2 x2
 −1 0   x3   x3  − x1 + 2 x2 = 2 x3
 2
Using row 1, if x1 =  then x3 = 0 ; using row 3, x2 =  / 2 .
   2   2   25 
  1  x 1   1
 x 2 =  12   =    → xˆ 2 = T2 =      5
 0 2  0 x2 x2 5   0  0
       
  2 0 1   x1   x1  2 x1 + x3 = 3x1
    
Bx 3 =  3x 3 →  − 1 4 − 1  x2  = 3 x2  → − x1 + 4 x2 − x3 = 3x2
 −1 0   x3   x3  − x1 + 2 x2 = 3x3
 2
Using row 1, if x1 =  then x3 =  too; using row 3, x2 = 2 .
      1 
6
  x 1  
 x 3 =  2  → xˆ 3 = T3 =  2   
2

  x3 x3 6   6
1 
    6

Worked example 3.5

1 0 4
Calculate the eigenvalues and (unit) eigenvectors of  
A = 0 2 0
3 1 − 3 

A is a 3×3 matrix with 3 eigenvalues and 3 eigenvectors. Its characteristic equation is found
by setting det (A − I ) = 0 , i.e.

1−  0 4
0 2− 0 = −(1 −  )(2 −  )(3 +  ) − 12(2 −  ) = 0
3 1 −3−
( )
→ (2 −  ) 2 + 2 − 15 = 0 → ( − 2)( − 3)( + 5) = 0

Hence, the eigenvalues are 1 = −5,  2 = 2 and  3 = 3.

 1 0 4   x1   x1  x1 + 4 x3 = −5 x1
    
Ax1 = 1x1 →  0 2 0   x2  = −5 x2  → 2 x2 = − 5 x2
 3 1 − 3  x  x  3x1 + x2 − 3x3 = −5 x3
  3   3
If x =  , row 1 gives x = − 3 x = − 3  ; row 3 gives x = 0 .
1 3 2 1 2 2
     2   2 
  1    13 2
 x1 =  0  =  0  → x1 x1 = 4 (2 0 − 3)  0  =
T 1

 − 3   2  − 3   − 3  4
 2     

 2   213 
x 1  
 xˆ 1 = T1 =  0  0 
x1 x1 13   − 3   − 3 
   13 

 1 0 4   x1   x1  x1 + 4 x3 = 2 x1
    
Ax 2 =  2 x 2 → 0 2 0   x2  = 2 x2  → 2 x2 = 2 x2
 3 − 3   x3   x3  3x1 + x2 − 3x3 = 2 x3
 1

If x1 =  , row 1 gives x3 = 14 x1 = 14  ; row 3 gives x2 = 5 x3 − 3 x1 = − 74  .

   4   4   4 
66 
 7  1  x 1  
 x 2 =  − 4   =  − 7  → xˆ 2 = T2 =  − 7    −
7

 1  4   x x
2 2
66    
66
1 
 4       66 

 1 0 4   x1   x1  x1 + 4 x3 = 3x1
    
Ax 3 =  3x 3 → 0 2 0   x2  = 3 x2  → 2 x2 = 3x2
 3 − 3   x3   x3  3x1 + x2 − 3x3 = 3x3
 1

If x1 =  , row 1 gives x3 = 12 x1 = 12  ; row 3 gives x2 = 0 .

   2   2   25 
  1  x 1  
 x 3 =  0  =  0  → xˆ 3 = T3 =  0  0 
 1  2    x3 x3 5   1 
2        5
3.6 Application to the solution of coupled differential equations

Consider the specific example shown of two equal masses m tied to identical springs of
force constant k (we neglect the effects of gravity).

We will now use eigenvalue analysis to study the natural oscillations (or normal modes) of
this simple system. These ideas can be extended to much more complex dynamical systems.

The displacements x1 and x2 of each mass satisfy the coupled equations

m d 2 x1 m d 2 x2
= −2 x1 + x2 , = x1 − 2 x2
k dt 2 k dt 2

We seek oscillatory solutions so we write the displacements as x1 = X 1e jt and

x2 = X 2e jt . Then the displacement equations become:

 2 − 1  X 1   X1  m2
    =   , i.e. AX = X, with  =
 −1 2   X 2   X2  k

The characteristic equation is then 2 − 4 + 3 = 0 , giving eigenvalues of 1 = 1 and

 2 = 3. These correspond to the frequencies of the natural oscillations (i.e. normal
modes) of the two-mass system, here given by

k i k 3k
i = → 1 = and 2 =  3 1
m m m
The normal modes of oscillation are the corresponding eigenvectors.

For the lower frequency mode at 1 , the eigenvector is

 1  1 For this mode x1 = x2 , i.e.
  masses always move in the
2  1
same direction.

For the higher frequency mode at 2 = 3 1 , the eigenvector is

1  1 For this mode, x1 = − x2 ,
 
2  −1 i.e. masses always move in
opposite directions.

You can see from this simple example why the set of eigenvalues of a matrix A is also often
called its spectrum. Note also that, since the system matrix involved here is symmetric, the
resulting eigenvectors are orthogonal (hence the term “normal” mode).

3.7 A note on repeated eigenvalues

For all of the examples up to now, the eigenvalues 1 ,  2 ,  3   n of a matrix A have

been distinct (i.e. all different), and there has been no problem identifying a unique (unit)
eigenvector with each. Matrix A is then said to have a full set of independent eigenvectors.

However, a matrix can have repeated eigenvalues (of multiplicity m). It is sometimes
possible to construct two independent eigenvectors corresponding to the same eigenvalue,
thus completing the full set. However, this is not always possible and it is important to note
that if a matrix has repeated eigenvalues then a full set of independent eigenvectors may
not exist for that matrix.

3.8 Summary

This section has covered the topic of eigenvalues and eigenvectors.

The eigenvalue equation is Ax = x , for which non-trivial solutions exist only if

det (A − I ) = 0 .

Evaluating det (A − I ) = 0 results in the characteristic equation (a polynomial in ),

which is solved to find the eigenvalues  i of A.
The eigenvectors x are found by substituting the eigenvalues back into the eigenvalue
equation, i.e. Ax i =  i x i
The eigenvectors are conveniently expressed as unit vectors xˆ i = x i /x Ti x i
The eigenvalues of a symmetric matrix are always real, and its eigenvectors are orthogonal
(hence forming a suitable vector basis).

After completing this section you should be able to find the eigenvalues and unit
eigenvectors of any 22 or 33 matrix.
3.9 Problems
3.1 Calculate the eigenvalues and (unit) eigenvectors of the following matrices.

1 0   2 3  4 − 2  cos  sin  
(a)   (b)   (c)   (d)  
1 2   3 2 1 1   − sin  cos  

3.2 For the matrices of 3.1, show that

(a) the sum of the eigenvalues equals the trace of each matrix, and
(b) the product of eigenvalues equals the determinant of each matrix.

3.3 For any symmetric matrices in 3.1, show that the eigenvectors are mutually orthogonal.

3.4 Calculate the eigenvalues and (unit) eigenvectors of the following matrices.

2 0 2  2 −1 0 2 7 0  10 −2 4
       
(a)  0 4 0  (b)  − 1 2 − 1 (c)  1 3 1  (d)  − 20 4 − 10 
2 5   0 − 1 2  5 8   − 30 − 13 
 0   0  6

3.5 For the matrices of 3.4, show that

(a) the sum of the eigenvalues equals the trace of each matrix, and
(b) the product of eigenvalues equals the determinant of each matrix.

3.6 For any symmetric matrices in 3.4, show that the eigenvectors are mutually orthogonal.

3.7 Three balls of equal mass m are joined by identical springs of force constant k (similar to
that described in section 3.5). Determine the natural frequencies and normal modes of this
system, and sketch the resulting displacements for each mode.

Repeat if the middle ball is replaced with one of mass 2m.