College of Technical Engineering - (CTE)

Civil Engineering Department

Graduate Program Study – B.Sc. in Civil Engineering

Useful Notes

In

MATRIX ALGEBRA

Compiled By

Dr. Hmida Mohamed Karkuri

October, 2012
1 Solving Sets of Equations

A variety of engineering problems may be solved through network analysis, in

addition to determining the potentials in certain electrical networks, such as finding
the stresses in a building frame or bridge truss, computing the flow rates in a
hydraulic system with interconnected branches, or estimating the concentrations of
reactants subject to simultaneous chemical reactions. The mathematical problem in
all these cases reduces to solving a set of equations simultaneously. We shall also
find when we study the numerical methods of solving boundary-value problems and
partial-differential equations, that these also require the solution of sets of equations,
often very large in number.

The solution of sets of equations is much more difficult when the equations are
nonlinear. Fortunately the majority of applications involve only linear equations,
although when such systems are large, careful choice of methods is required to avoid
overly long procedures and to preserve maximum accuracy. In this section we shall
first discuss some of the basic ways of solving sets of linear equations, and later take
up the harder problem of nonlinear equations.

1.1 Matrix Notation

Our discussion will be facilitated by some of the concepts and notation of matrix
algebra. Only the more elementary ideas will be needed. A matrix is a rectangular
array of numbers in which not only the value of the number is important but also its
position in the array. The size of the matrix is described by the number of its rows
and columns. A matrix of n rows and m columns is said to be n × m. The elements of
the matrix are generally enclosed in brackets, and double-subscripting is the common
way of indexing the elements. The first subscript always denotes the row and the
second denotes the column in which the element occurs. Capital letters are used to
refer to matrices. For example,

a 11 a 12 .....a 1m 
 
a 21 a 22 .....a 2m 
. 
A     a ij , i = 1, 2, …, n, j = 1, 2, …, m. (1.1)
. 
. 
 
a n1 a n 2 .....a nm 

Enclosing the general elements, aij, in brackets is another way of representing matrix
A, as shown above. Two matrices of the same size may be added or subtracted. The
sum of:

A a ij  and B b ij  (1.2)

2
is the matrix whose elements are the sum of the corresponding elements of A and B,


C  A  B  a ij  b ij  c ij    (1.3)

Similarly we get the difference of two equal-sized matrices by subtracting

corresponding elements. If two matrices are not equal in size, they cannot be added or
subtracted. Two matrices are equal if and only if each element of one is the as the
corresponding element of the other. Obviously, equal matrices must be of the same
size. Some examples will help make this clear.
If
 3 2 1  0 3 2
A  and B (1.4)
 4 0 2  4 2 1 

we say that A is 2 × 3 because it has two rows and three columns. B is also 2 × 3.
Their sum, C, is also 2 × 3:

3 5 1
C  AB  (1.5)
0 2 3

The difference, D, of A and B is:

 3 1  3 
D A B (1.6)
8 2 1 

Multiplication of two matrices is defined as follows, when A is n × m and B is m × r:

 a 11 b 11  a 12 b 21  ...  a 1m b m1  ... a 11 b 1r  ...  a 1m b mr  

 
 a 21 b 11  a 22 b 21  ...  a 2 m b m1  ...  a 21 b 1r  ...  a 2 m b mr 
 
 a  b    c 
ij ij ij 
.
 ,
 . 
 . 
 
 a n1 b 11  a n 2 b 21  ...  a nm b m1  ...  a n1 b 1r  ...  a nm b mr 

m
c ij  
k 1
a ik b kj , i = 1, 2, …. , n, j = 1, 2, … , r. (1.7)

It is simplest to select the proper elements if one counts across the rows of A with the
left hand while counting down the columns of B with the right. Unless the number of
columns of A equals the number of rows of B (so the counting comes out even), the

3
matrices cannot be multiplied. Hence if A is n × m, B must have m rows or else they
are said to be “ non-conformable for multiplication “ and their product is undefined.
In general AB ≠ BA, so the order of factors must be preserved in matrix
multiplication. If a matrix is multiplied by a scalar (a pure number), the product is a
matrix, each element of which is the scalar times the original element. We can write:

If kA = C, c ij = kaij , (1.8)

A matrix with only one column, n × 1 in size, is termed a column vector, and one of
only one row, 1 × m in size, is called a row vector. When the unqualified term
“vector” is used, it nearly always means a column vector. Frequently the elements of
vectors are only singly subscripted.

Some examples of matrix multiplications are:

 1 2   4  y1 
x   2 
 2 0
B   0 2  , y   y 2  .
4
Suppose A  
3   
, ,

 1 2
 4  1   y 3 
 1 

 4 0 6 
 2 12 
A. B    B . A   2 4 6  ;
 
;
 13  3   9 14  3 

 0 2 y1  4 y 2 
Ax   ; Ay   .
 5    y1  2 y 2  3 y 3 

Since A is 2 × 3 and B is 3 × 2, they are conformable for multiplication and their

product is 2 × 2. When we form the product of B . A, it is 3 × 3. Observe that not only
is AB ≠ BA ; AB and BA are not even the same. The product of A and the vector x
(a 3 × 1 matrix) is another vector, one with two components.

Similarly, Ay has two components. We cannot multiply B times x and B times y ;

they are non-conformable.

The product of the scalar number 2 and A is

 4 8 0
2A   
 2 4 6 

4
Since a vector is just a special case of a matrix, a column vector can be multiplied by
a matrix, so long as they are conformable in that the number of columns of the matrix
equals the number of elements (rows) in the vector. The product, in this case will be
another column vector. The size of a product of two matrices, the first n × m and the
second n × r, is m × r. An n × m matrix times an n × 1 vector gives an m × 1
product.

The general relation for Ax = b is

No of cols.
bi  
k 1
a ik x k , i = 1, 2, …. , No of rows. (1.9)

Tow vectors, each with the same number of components, may be added or subtracted.
Two vectors are equal if each component of one equals the corresponding component
of the other.

This definition of matrix multiplication permits us to write the set of linear equations,

a11x1 + a12x2 + … + a1nxn = b1,

a21x1 + a22x2 + … + a2nxn = b2,
.
.
.
an1x1 + an2x2 + … + annxn = bn, (1.10)

much more simply in matrix notation, as Ax = b, where

a 11 a 12 ..... a 1m  x 1  b 1 
     
a 21 a 22 ..... a 2 m  x 2  b 2 
.  .  . 
A   , x   , b  . (1.11)
.  .  . 
.  .  . 
     
a n1 a n 2 ..... a nm   x n  b n 

For example,

 3 2 4  14 
 12 0 . x = 7 
   
 1 3 2   2 

is the same as the set of equations

5
 3x1 + 2x2 + 4x3 = 14,
x1 + 2x2 = -7,
- x1 + 3x2 + 2x3 = 2,

A very important special case is the multiplication of two vectors. The first must be a
row vector if the second is a column vector, and each must have the same number of
components. For example,

 4
 1 3 4  .   1     11 
 3 

gives a “matrix” of one row and one column. The result is a pure number, a scalar.
This product is called the scalar product of the vectors, also called the inner product.

Certain square matrices have special properties. The diagonal elements are the line of
elements aii from upper left to lower right of the matrix. When the diagonal elements
are each equal to unity while all off-diagonal elements are zero, the matrix is said to
be the identity matrix of order n. The usual symbol for such a matrix is In, and it has
properties similar to unity. For example, the order-four identity matrix is

 1 0 0 0
 0 1 0 0 
  I4 .
 0 0 1 0
 
 0 0 0 1

The subscript is omitted when the order is clear from the context.

A vector that has all its elements equal to zero except one element, which has a value
of unity, is called a unit vector. There are three distinct unit vectors for order-three
vectors; they are
1  0  0
 0 , 1 , and  0 .
     
 0   0   1 

If all the elements above the diagonal are zero, the matrix is called lower triangular;
it is called upper triangular when all the elements below the diagonal are zero. For
example, these order-three matrices are lower and upper triangular:

6
  1 0 0  1 3 3

L  4 6 0  , U   0  1 0 
 2 1  4   0 0 1 

Tridiagonal matrices are those which have nonzero elements only on the diagonal
and in the positions adjacent to the diagonal; they will be of special importance in
certain partial-differential equations. An example of a tridiagonal matrix is

4 2 0 0 0
 1 4 1 0 0 

 0 1 4 1 0
 
 0 0 1 4 1
 0 0 0 2  4 

The transpose of a matrix is the matrix that results when the rows are written as
columns (or, alternatively, when the columns are written as rows). The symbol AT is
used for the transpose of A.

Example
 3 1 4   3 0 1
A   0 2  3  ; A    1 2
T
1 
 1 1 2   4  3 2 

It should be obvious that the transpose of AT is just A itself.

When a matrix is square, a quantity called its trace is defined. The trace of a square
matrix is the sum of the elements on its main diagonal. For example, the traces of the
above matrices are:

tr ( A ) = 3 + 2 + 2 = 7 ; tr ( AT ) = 3 + 2 + 2 = 7

It should be obvious that the trace remains the same if a square matrix is transposed.

We present here some additional examples of arithmetic operations with matrices.

 3  . 
2 3 6
1
  .
 3 4   9 12

 1 3 2   1 0 2   0 3 4 
 1 0 4    4 1  3    3 1 1  ;
     

7
  2 1   3  2   1 3 
 0 4    4 1     4  5  .
 
  
 7 2   0  2   7 4 

 1  1
2 0 1   3  6 1 
3  .  2    but  .  2  = not defined.
 2 6  7  
3 2   1 
 1   

1 3  0 3   3 6  0 3  1 3 6 3
 2 1 .  5 
but  1 1  .  2 1  1  4
   1 1   1      

Division of a matrix by another matrix is not defined, but we shall discuss the inverse
of a matrix later in this section.

The determinant of a square matrix is a number. For a 2 × 2 matrix, the determinant

is computed by subtracting the product of the elements on the minor diagonal (from
upper right to lower left) from the product of terms on the major diagonal. For
example

 3 1
A  , det (A) = (3)(4) – (-1)(2) = 14;
2 4 

det (A) is the usual notation for the determinant of A. Sometimes the determinant is
symbolized by writing the elements of the matrix between vertical lines (similar to
representing the absolute value of a number).

The general rule that applies in all cases is to expand in terms of the minors of some
row or column. The minor of any term is the matrix of lower order formed by striking
out the row and column in which the term is found. The determinant is found by
adding the product of each term in any row or column by the determinant of its
minor, with signs alternating + and -. We expand each of the determinants of the
minor until we reach 2 × 2 matrices.

Example (1):

 3 0 1 2
 4 1 3  2 
Given A  
 0 2 1 3 
 
 1 0 1 4

8
 1 3 2 4 3 2 4 1 2 4 1 3
det (A) = 3 2 1 3 - 0 0 1 3 + (-1) 0 2 3 -2 0 2 1
0 1 4 1 1 4 1 0 4 1 0 1

 1 3 2 3 2 1 
= 3  (1)  (3)  (2) 
 1 4 0 4 0 1 

 2 3 0 3 0 2
+ (-1)  (4)  (1)  (2) 
 0 4 1 4 1 0

 2 1 0 1 0 2
- 2  (4)  (1)  (3) 
 0 1 1 1 1 0

= 3{(1)(-7) – (3)(8) + (-2)(2)} + (-1){(4)(8) – (1)(-3) + (-2)(-2)} -2{(4)(2) –

(1)((1) + (3)(-2)}
= 3(- 7 - 24 – 4 ) + (-1)(32 + 3 + 4) – 2(8 – 1 – 6)
= 3(- 35) + (-1)(39) – 2(1) = - 146.

In computing the determinant, the expansion can be about the elements of any row or
column. To get signs, give the first term a plus sign if the sum of its column number
and row number is even; give it a minus if the sum is odd, with alternating sings
thereafter. (For example, in expanding about the elements of the third row we begin
with a plus; the first element a31 has 3 + 1 = 4, an even number,) Judicious selection
of rows or columns with many zeros can hasten the process, but this method of
calculating determinants is a lot of work if the matrix is of a large size.

In fact, the evaluation of a determinant can perhaps best be done by adapting the
Gauss elimination procedure. Its utility derives from the fact that the determinant of a
triangular matrix (either upper- or lower-triangular) is just the product of its diagonal
elements. This is easily seen, in the case of an upper- triangular matrix, by expansion
by minors of the first column at each step.

Example (2):

1 4 2 3 1 4 2 3 1 4 2 3 1 4 2 3
2 2 0 4 0 6 4  2 0 6 4 2 0 6 4 2
= = = =
3 0 1 2 0  12 5  7 0 0 3 3 0 0 3 3
1 2 2 3 0 2 4 6 0 0 8
3
 163 0 0 0 8

= (1)(-6)(-3)(-8) = -144.

9
1.2 Matrix Inversion
While division of matrices is defined, the matrix inverse gives the equivalent result. If
the product of two square matrices is the identity matrix, the matrices are said to be
inverses. If AB = I, we write B = A-1; also A = B-1. Inverses commute on
multiplication, which is not true for matrices in general: AB = BA = I. Not all square
matrices have an inverse. Singular matrices do not have an inverse, and these are of
extreme importance in connection with the coefficient matrix of a set of equations, as
discussed above.

The inverse of a matrix can be defined in terms of the matrix of the minors of its
determinant, but this is not a useful way to fined an inverse. Gaussian elimination can
be adapted to provide a practical way to invert a matrix. The procedure is to augment
the given matrix with the identity matrix of the same order. One then reduces the
original matrix to the identity matrix by elementary row transformations, performing
the same operations on the augmentation columns. When the identity matrix stands as
the left half of the augmented matrix, the inverse of the original stands as the right
half. It should be apparent that this is equivalent to solving a set of equations with n
different right-hand sides; each of the right-hand sides is a unit vector, in which the
position of the element whose value is unity changes from row 1 to row 2 to row 3 …
to row n.

Example (3):

Find the inverse of

 1 1 2
A 3 0 1
 
 1 0 2 

Augment A with the identity matrix and then reduce:

 1 1 2 1 0 0  1 1 2 1 0 0 
3 0 1 0 1 0   0 3  5  3 1 0 
 
 1 0 2 0 0 1   0 1 0  1 0 1 

 1 1 2 1 0 0   1 1 0 1  65 
2
5
 
(1)  0 1 0  1 0 1  ( 2) 0 1 0 1 0 1 
 

 0 0  5 0 1  3 
 0 0 1 0  15 53 


01
 1 0 0 0  15 
2
5
 
 0 1 0 1 0 1 
0 0 1 0  15 53 

(1)
Interchange the third and second rows before eliminating from the third row,
(2)
Divide the third row by -5 before eliminating from the first row.

We confirm the fact that we have found the inverse by multiplication:

 1 1 2   0 52  15  1 0 0 
3 0 1    
 1 0 1  =  0 1 0 
  
 1 0 2   0 1 3  0 0 1 
 5 5

The inverse of the coefficient matrix provides a way of solving the set of equations
Ax = b because, when we multiply both sides of the relation by A-1, we get

A-1 Ax = A-1 b,
x = A-1 b.
The second equation follows because the product A-1 A = I, the identity matrix, and
Ix = x. If we know the inverse of A, we can solve the system for any right-hand side b
simply by multiplying the b vector by A-1. This would seem like a good way to solve
systems of equations, and finds frequent references to it.

1.2.1 Elimination Method

The first method we shall study for the solution of a set of equations is just an
enlargement of the familiar method of eliminating one unknown between a pair of
simultaneous equations. It is generally called Gaussian elimination and is the basic
pattern of a large number of methods that can be classed as direct methods. (This is to
distinguish them from indirect, or iterative, methods, which we shall discuss later).

The discussion continues using the overhead projector slides:

Exercise (1):
Consider the following simple example of three equations, and solve them using the
Gaussian elimination method

3x1 - x 2  2x 3  12,
x1  2x 2  2x 3  11,
2x1 - 2x 2 - x 3  2

00
Exercise (2):
Solve the system Ax = b, with multiple values of b, by Gaussian elimination:

3 2 1 2  0  1  2
1 4 0 2 0  1 2
A   , 1
b   , b  2
  , 3
b  
2 1 2 1  1   3 0
       
1 1 1 3  1   4 0

1.2.2 The LU decomposition

A modification of the elimination method, called the LU decomposition method (also
named Crout reduction, or, another discoverer, Cholesky), is frequently used in
computer programs. In it the matrix of coefficients A is transformed into the product
of two matrices L and U, where L is a lower triangular and U is an upper triangular
matrix with 1’s on its diagonal. Any matrix can be written as a product of a lower
triangular and an upper triangular matrix in an infinity of ways. For example, the
following system of equations.

2x1  4x 2  2x 3  - 2,
2x1  8x 2 - 4x 3  2x 4  - 4,
- 1x1  3x 3 - 2x 4  2,
- 4x 2  3x3  x4  3

which will be in a matrix notation as:

 2 4  2 0
 2 8 4 2 
A  
 1 0 3  2
 
 0 4 3 1 

That is:

 1 0 0 0   2 4 2 0 
 1 1 0 0   0 4 2 2 
L   , U   
  1/ 2 1/ 2 1 0   0 0 3 3 
   
 0 1 1/ 3 1   0 0 0 2

02
Then by a strange twist, our example we may verify

 1 0 0  0  2 4 2 0
 1 1 0  0  0 4 2 2
LU     
  1/ 2 1/ 2 1  0  0 0 3 3 
   
 0 1 1/ 3 1   0 0 0 2

 2 4 2 0 
 2 8 4 2 
    A ( !! )
 1 0 3  2 
 
 0  4 3 1 
The matrix A has been expressed as the product of matrices, one lower triangular and
one upper triangular. This is known as the LU factorization of A.

1.2.3 Gauss and Gauss-Jordan Methods

While the procedure of the previous section is satisfactory for hand calculations on
small systems, there are several objections that we should eliminate before we write a
computer program to perform Gaussian elimination. In a large set of equations, and
that is the situation we must prepare for, the multiplications will give very large and
unwieldy numbers that may overflow the computer’s registers. We therefore will
eliminate the first coefficient in the ith row by subtracting ai1/a11 times the first
equation from the ith equation. (this is equivalent to making the leading coefficient 1
in the equation that retains that leading term.) We use similar ratios of coefficient in
eliminating coefficients in the other columns.

We must also guard against dividing by zero. Observe that zeros may be created in
the diagonal positions even if they are not present in the original matrix of
coefficients. A useful strategy to avoid (if possible) such zero divisors is to rearrange
the equations so as to put the coefficient of largest magnitude on the diagonal at each
step. This is called pivoting. Complete pivoting may require both row and column
interchanges. This is not frequently done. Partial pivoting, which places a coefficient
of larger magnitude on the diagonal by row interchanges only, will guarantee a
nonzero divisor if there is a solution to the set of equations, and will have the added
advantage of giving improved arithmetic precision. The diagonal elements that result
are called pivoting elements.

To solve a system of linear equations

1. Augment the n × n coefficient matrix with the vector of right-hand sides to

form a n × ( n + 1) matrix.

03
 2. Interchange rows if necessary to make the value of a11 the largest magnitude of
any coefficient in the first column.
3. Create zeros in the second through nth rows in the first column by subtracting
ai1/a11 times the first row from the ith row.
4. Repeat steps (2) and (3) for the second through the (n – 1)st rows, putting the
largest-magnitude coefficient on the diagonal by interchanging rows
(considering only rows j to n), and the subtracting aij/ajj times the jth row from
the ith row so as to create zeros in all positions of the jth column below the
diagonal. At the conclusion of the step, the system is upper-triangular.
5. Solve for xn from the nth equation by

Xn = an,n+1 / ann, (1.12)

6. Solve for xn-1, xn-2, …. , x1 from the (n – 1)st through the first equation in turn
by
j  i 1
n
a i , n 1  a ij x j
xi  . (1.13)
a ii

Example (1):
Using the Gaussian elimination method
2x1  4x 2  2x 3  - 2,
2x1  8x 2 - 4x 3  2x 4  - 4,
- 1x1  3x 3 - 2x 4  2,
- 4x 2  3x3  x4  3

Answer:
Step k = 0. We begin with the original system of equations in matrix form this
system is denoted by A(0) x = b(0), where of course

 2 4  2 0  2 
 2 8 4 2   2 
 0
A    , b  0
  .
 1 0 3  2  3 
   
 0 4 3 1   3.

Step k = 1. We use the first equation to eliminate x1 from all other equations.

04
  Pivot
2x1  4x 2  2x 3  - 2,  Pivot Equation
R 2  1 R 1 4x 2 - 4x 3  2x 4  - 2,
R 3    12  R 1 2x 2  3x 3 - 2x 4   4,
R 4   0 R 1 - 4x 2  3x 3  x 4  3

The numbers 1, -1/2 and 0 are called the multipliers. The row used to clear the
column is known as the pivot equation. The coefficient of the pivot equation in the
column being cleared is the pivot. In matrix form this system is denoted by A(1) x =
b(1), where

 2 4  2 0  2 
 0 4 4 2   2 
1
A   , 1
b  
 0 2 2  2  4 
   
0 4 3 1   3.

Step k = 2. We use the new second equation to eliminate x2 from the third and fourth
equations.

2 x1  4x 2  2x 3  - 2,
4 x 2 - 2 x 3  2x 4  - 2,
R 3   12  R 2 3x 3 - 3x 4   3,
R 4   1 R 2 x3  x 4  1

This system is denoted by A(2) x = b(2), where

2 4  2 0  2 
0 4 2 2   2 
 2
A    , b  2
 
0 0 3 3   3 
   
0 0 1 1  1. 

Step k = 3. We use the new third equation to eliminate x3 from the fourth equations.

05
 2 x 1  4x 2  2x 3  - 2,
4 x 2 - 2 x 3  2x 4  - 2,
3x 3 - 3x 4   3,
R 4  13 R 3 2x4  2

This system is denoted by A(3) x = b(3), where

2 4 2 0  2 
0 4 2 2   2 
3
A    , 3
b  
0 0 3 3   3 
   
0 0 0 2  0.

This system is said to be in upper triangular form. It can now be solved. We work out
x4, and then substitute into the third equation to work out x 3. Then into the second
equation to work out x2, and so on.

x4  2
2
1
x3  1
3
  3  3x 4   13   3   1    0
x2  1
4
  2  2x 3  2x 4   14   2  2  0   2  1     1
x1  1
2
  2  4x 2  2x 3   12   2  4   1   2  0    1
This is called back substitution.

We can organize these computations more efficiently:

 By placing the coefficients in an array (rather than writing down x1, x2, .. etc
many times).
 By storing the multipliers in the space created by the corresponding new 0’s.

This is much the same way as a good computer program would be written. With this
new storage scheme we write successive stages as

 2 4 2 0  2
 
 A  0
b 0
 
2
 1
8 4
0 3
2  4
 2  3
 
 0 4 3 1 3

06
  2 4 2 0  2
 
 A 1 1
b   1
  1/ 2 2
4 2
2
2  2
 2  4
 
 0 4 3 1 3
 2 4 2 0  2
 
 A  2
b  2
   1
  1/ 2 1/ 2
4 2
3
2  2
 3  3
 
 0 1 1 1 1 
 2 4 2 0  2
 
 A 3 3
b  
1

  1/ 2
4
1/ 2
2
3
2  2
 3  3
 
 0 1 1/ 3 2 2

Now as back substitution is done the new solution overwrites the entries up the last
column from bottom to top. This gives the final solution in the space saving format

 2 4 2 0 1
 
 A 3

x  
1
  1/ 2
4
1/ 2
2
3 3
2 1
0
 
 0 1 1/ 3 2 1

____________________________________________________________________

07