Adama University Department of Electrical Engineering

Chapter 3 System of Linear Equations and Matrices

3.1 introduction to vectors and matrices

A real N-dimensional vector X is an ordered set of N real numbers and usually written in
the coordinate form ( ) here are called components of x.
The set consisting of all N-dimensional vectors is called N-dimensional space.

A matrix is a rectangular array of numbers that is arranged systematically in rows and

columns. A matrix having M rows and N columns is called an MxN (read “M by N”) matrix.
The capital letter A denotes a matrix, and lowercase subscripted letter denotes one of
the numbers forming the matrix.

 a11 a12 ...... a1N 

a a22 ...... a2 N 
 21
A     row _ i
  
a ...... aMN 
 M1 aM 2


column _ j

Some special properties of vectors and Matrices

The identity matrix of order N is the square matrix given by
[ ] Where {
An NxN matrix A is called non-singular or invertible if there exists an NxN matrix B such that

If no such matrix B can be found, A is said to be singular.

An NxN matrix [ ] is called upper triangular provided that the elements satisfy
whenever The NxN matrix [ ] is called lower triangular provided that
whenever

3.2 Existence and Uniqueness of Solutions

Matrix algebra is used to solve a system of simultaneous linear equations.
A general set of m linear equations and n unknowns,
a11x1  a12 x2    a1n xn  c1
a21x1  a22 x2    a2n xn  c2
……………………………………
…………………………………….
Computational Methods Page 1
Adama University Department Of Electrical Engineering

am1 x1  am2 x2  ........  amn xn  cm

Can be rewritten in the matrix form as

 a11 a12 . . a1n   x1   c1 

a    
 21 a 22 . . a 2 n   x 2   c 2 
       
    
      
a m1 a m 2 . . a mn   x n  c m 
Denoting the matrices by A , X  , and C  , the system of equation is AX   C  , where
A is called the coefficient matrix, C  is called the right hand side vector and X  is called
the solution vector. Sometimes AX   C  systems of equations are written in the
augmented form. That is
a a12 ...... a1n  c1 
 11 
 a21 a22 ...... a2 n  c2 
 A C      
 
   
am1 am 2 ...... amn  cn 

A system of equations can be consistent or inconsistent. What does that mean?

A system of equations AX   C  is consistent if there is a solution, and it is inconsistent if
there is no solution. However, a consistent system of equations does not mean a unique
solution, that is, a consistent system of equations may have a unique solution or infinite
solutions.

How can one distinguish between a consistent and inconsistent system of equations?
A system of equations AX   C  is consistent if the rank of A is equal to the rank of the
augmented matrix AC 
A system of equations AX   C  is inconsistent if the rank of A is less than the rank of
the augmented matrix AC  .
But, what do you mean by rank of a matrix?
The rank of a matrix is defined as the order of the largest square sub matrix whose
determinant is not zero.

Example 1
What is the rank of
 3 1 2
A  2 0 5
5 1 7
?
Computational Methods Page 2
Adama University Department Of Electrical Engineering

Solution
The largest square sub matrix of [ A] is of order 3 and that is [ A] itself. Since det( A)  0 ,
the rank of [ A] is less than 3. The next largest square sub matrix would be a 2  2 matrix.
One of the square sub matrices of
3 1 [ A] is
B   
 2 0

And det(B)  2  0 . Hence the rank of [ A] is 2. There is no need to look at other 2  2

sub matrices to establish that the rank of [ A] is 2.

Example2
How do I now use the concept of rank to find if?
 25 5 1  x1  106.8 
 64 8 1  x   177.2 
  2   
144 12 1  x3  279.2
Is a consistent or inconsistent system of equations?

Solution
The coefficient matrix is
 25 5 1
A   64 8 1
144 12 1
And the right hand side vector is
106.8 
C   177.2 
279.2
The augmented matrix is
 25 5 1  106.8 
B   64 8 1  177.2 
144 12 1  279.2
Since there are no square sub matrices of order 4 as [B] is a 3  4 matrix, the rank of [B] is
at most 3. So let us look at the square sub matrices of [B] of order 3; if any of these square
sub matrices have determinant not equal to zero, then the rank is 3. For example, a sub of
the augmented matrix [B] is
 25 5 1 matrix
[D]   64 8 1
144 12 1

Computational Methods Page 3

Adama University Department Of Electrical Engineering

det(D)  84  0 .
Hence the rank of the augmented matrix [B] is 3. Since [ A]  [ D] , the rank of [ A] is 3.
Since the rank of the augmented matrix [B] equals the rank of the coefficient matrix [ A] ,
the system of equations is consistent.
Example 3
Use the concept of rank to
25 5 1  x1  106.8  find if
64 8 1  x   177.2 
   2  
89 13 2  x3  280.0

Is consistent or inconsistent.
Solution
The augmented matrix is
25 5 1 : 106.8 
B  64 8 1 : 177.2 
89 13 2 : 280.0
Since there are no square sub matrices of order 4  4 as the augmented matrix [B] is a 4  3
matrix, the rank of the augmented matrix [B] is at most 3. So let us look at square sub
matrices of the augmented matrix (B) of order 3 and see if any of the 3  3 sub matrices
have a determinant not equal to zero. For example, a square sub matrix of order 3  3 of [B]
25 5 1
D  64 8 1
89 13 2
det(D) = 0
So it means, we need to explore other square sub matrices of the augmented matrix [B]
 5 1 106.8 
E    8 1 177.2 
13 2 280.0
det(E0  12.0  0 .
So the rank of the augmented matrix [B] is 3.
Now we need to find the rank of the coefficient matrix [ A] .
25 5 1
A  64 8 1
89 13 2
det( A)  0
So the rank of the coefficient matrix [ A] is less than 3. A square sub matrix of the
coefficient matrix [ A] is

Computational Methods Page 4

Adama University Department Of Electrical Engineering

J   
5 1

8 1
det(J )  3  0
So the rank of the coefficient matrix [ A] is 2.
Since the rank of the coefficient matrix [ A] is less than the rank of the augmented matrix
[B] , the system of equations is inconsistent. Hence, no solution exists for [ A][ X ]  [C ] .

If a solution exists, how do we know whether it is unique?

In a system of equations [ A][ X ]  [C ] that is consistent, the rank of the coefficient matrix
[ A] is the same as the augmented matrix [ A C ] . If in addition, the rank of the coefficient
matrix [ A] is same as the number of unknowns, then the solution is unique; if the rank of
the coefficient matrix [ A] is less than the number of unknowns, then infinite solutions exist.

[A] [X] = [B]

Consistent System if Inconsistent System if

rank (A) = rank (A.B) rank (A) < rank (A.B)

Unique solution if Infinite solutions if

rank (A) = number of unknowns rank (A) < number of unknowns

Figure 5.2. Flow chart of conditions for consistent and inconsistent system of equations.

Example 4
We found that the following system of equations
 25 5 1  x1  106.8 
 64 8 1  x   177.2 
   2  
144 12 1  x3  279.2
Is a consistent system of equations. Does the system of equations have a unique solution or
does it have infinite solutions?
Solution
The coefficient matrix is
 25 5 1
A   64 8 1
144 12 1
And the right hand side is
106.8 
C   177.2 
279.2

Computational Methods Page 5

Adama University Department Of Electrical Engineering

While finding out whether the above equations were consistent in an earlier example, we
found that the rank of the coefficient matrix (A) equals rank of augmented matrix AC 
equals 3. The solution is unique as the number of unknowns = 3 = rank of (A).
Example 5
We found that the following system of equations
25 5 1  x1  106.8 
64 8 1  x   177.2 
   2  
89 13 2  x3  284.0
Is a consistent system of equations. Is the solution unique or does it have infinite solutions.
Solution
While finding out whether the above equations were consistent, we found that the rank of
the coefficient matrix [ A] equals the rank of augmented matrix  AC  equals 2
Since the rank of [ A]  2 < number of unknowns = 3, infinite solutions exist.

Consistent systems of equations can only have a unique solution or infinite solutions. Can
a system of equations have more than one but not infinite number of solutions?
No, you can only have either a unique solution or infinite solutions. Let us suppose
[ A] [ X ]  [C ] has two solutions [Y ] and [Z ] so that
[ A] [Y ]  [C ]
[ A] [Z ]  [C ]
If r is a constant, then from the two equations
rAY   rC 
1  r AZ   1  r C
Adding the above two equations gives
rAY   1  r AZ   rC   1  r C 
ArY   1  r Z   C 
Hence, rY   1  r Z  is a solution to AX   C 
Since r is any scalar, there are infinite solutions for [ A][ X ]  [C ] of the form
rY   1  r Z 

3.3 Methods of solution of Linear Equations

3.3.1 Gaussian Elimination
How is a set of equations solved numerically?
One of the most popular techniques for solving simultaneous linear equations is the
Gaussian elimination method. The approach is designed to solve a general set of n
equations and n unknowns

a11x1  a12x2  a13x3  ...  a1n xn  b1

a21x1  a22x2  a23x3  ...  a2n xn  b2
. .
Computational Methods Page 6
Adama University Department Of Electrical Engineering

. .
. .
an1x1  an 2 x2  an3 x3  ...  ann xn  bn

Gaussian elimination consists of two steps:

Forward Elimination of Unknowns:
In this step, the unknown is eliminated in each equation starting with the first equation.
This way, the equations are reduced to one equation and one unknown in each equation.
Back Substitution: In this step, starting from the last equation, each of the unknowns is
Found.

Forward Elimination of Unknowns:

In the first step of forward elimination, the first unknown, x1 is eliminated from all rows
below the first row. The first equation is selected as the pivot equation to eliminate x1 . So,
to eliminate x1 in the second equation, one divides the first equation by a11 (hence called
the pivot element) and then multiplies it by a 21 . This is the same as multiplying the first
equation by a21 / a11 to give
a a a
a 21 x1  21 a12 x2  ...  21 a1n xn  21 b1
a11 a11 a11
Now, this equation can be subtracted from the second equation to give
 a   a  a
 a 22  21 a12  x2  ...   a 2 n  21 a1n  xn  b2  21 b1
 a11   a11  a11

Or
 x2  ...  a2 n xn  b2
a22

Where,
a21
  a22 
a22 a12
a11

a21
a2 n  a2 n  a1n
a11
This procedure of eliminating x1 is now repeated for the third equation to the n th equation
to reduce the set of equations as
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
 x2  a23
a22  x3  ...  a2 n xn  b2
 x2  a33
a32  x3  ...  a3 n xn  b3
. .
. .
Computational Methods Page 7
Adama University Department Of Electrical Engineering

an 2 x2  an 3 x3  ...  ann

 xn  bn
This is the end of the first step of forward elimination. Now for the second step of forward
elimination, we start with the second equation as the pivot equation and a22  as the pivot
element. So, to eliminate x2 in the third equation, one divides the second equation by a22 
(the pivot element) and then multiply it by a32  . This is the same as multiplying the second
equation by a32 / a22
 and subtracting it from the third equation. This makes the coefficient
of x2 zero in the third equation. The same procedure is now repeated for the fourth
equation till the n th equation to give

a11x1  a12 x2  a13 x3  ...  a1n xn  b1

 x2  a23
a22  x3  ...  a2 n xn  b2
a33  x3  ...  a3n xn  b3
. .
. .
an3 x3  ...  ann
 xn  bn

The next steps of forward elimination are conducted by using the third equation as a pivot
equation and so on. That is, there will be a total of n  1 steps of forward elimination. At
the end of n  1 steps of forward elimination, we get a set of equations that look like

a11x1  a12 x2  a x  ...  a x  b

13 3 1n n 1

 x2  a23
a22  x3  ...  a2 n xn  b2
 x3  ...  a3n xn  b3
a33
. .
. .
n 1 n 1
ann xn  bn

Back Substitution:
Now the equations are solved starting from the last equation as it has only one unknown.
bn( n 1)
x n  ( n 1)
a nn
Then the second last equation, that is the (n  1) th equation, has two unknowns: x n and
xn1 , but x n is already known. This reduces the (n  1) th equation also to one unknown.
Back substitution hence can be represented for all equations by the formula

bii 1   aiji 1 x j

n

j i 1
xi 
aiii 1 For i  n 1, n  2,,1

Computational Methods Page 8

Adama University Department Of Electrical Engineering

And
bn( n 1)
x n  ( n 1)
a nn
Example 1
Use Naïve Gauss elimination to solve
20 x1  15x2  10 x3  45
 3x1  2.249 x2  7 x3  1.751
5x1  x2  3x3  9
Solution
Working in the matrix form
 20 15 10  x1   45 
 3  2.249 7   x  1.751
   2  
 5 1 3   x 3   9 
=

Forward Elimination of Unknowns

First step
Divide Row 1 by 20 and then multiply it by –3, that is, multiply Row 1 by  3 / 20  0.15 .
20 15 10 45 0.15 Gives Row 1 as
 3  2.25  1.5  6.75
Subtract the result from Row 2
 3  2.249 7 1.751
  3  2.25  1.5  6.75
0 0.001 8.5 8.501

To get the resulting equations as

20 15 10   x1   45 
 0 0.001 8.5  x  8.501
   2  
 5 1 3   x 3   9 
=
Divide Row 1 by 20 and then multiply it by 5, that is, multiply Row 1 by 5 / 20  0.25
20 15 10 45 0.25 Gives Row 1 as
5 3.75 2.5 11.25
Subtract the result from Row 3
15 3 9
 5
3.75 2.5 11.25
0  2.75 0.5  2.25
To get the resulting equations as
Computational Methods Page 9
Adama University Department Of Electrical Engineering

20 15 10   x1   45 
 0 0.001 8.5  x   8.501 
   2  
 0  2.75 0.5  x3   2.25
=
Second step
Now for the second step of forward elimination, we will use Row 2 as the pivot equation
and eliminate Row 3: Column 2.
Divide Row 2 by 0.001 and then multiply it by –2.75, that is, multiply Row 2
 2.75 / 0.001  2750 .
0 0.001 8.5 8.501 2750 Gives Row 2 as
0  2.75  23375  23377.75
Subtract the result from Row 3
0  2.75 0.5  2.25
 0  2.75  23375  23377.7
0 0 23375.5 23375.45
To get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5   x 2   8.501 

 0 0 23375.5  x3  23375.45
=
This is the end of the forward elimination steps.

Back substitution
We can now solve the above equations by back substitution. From the third equation,
23375.5x3  23375.45
23375.45
x3 
23375.5
 0.9999978
Substituting the value of x3 in the second equation
0.001x2  8.5x3  8.501
8.501  8.5 x3
x2 
0.001
8.501  8.5  0.9999978

0.001
 1.0182
Substituting the value of x3 and x 2 in the first equation,
20 x1  15x2  10 x3  45
45  15 x2  10 x3
x1 
20
45  15  1.0182  10  0.9999978

20
Computational Methods Page 10
Adama University Department Of Electrical Engineering

 0.9863511

Hence the solution is:

 x1   0.98635 
[ X ]   x 2    1.0182 
 x3  0.9999978
Compare this with the exact solution of
 x1  1
X    x2   1
 
 x3  1

Are there any pitfalls of the Naïve Gauss elimination method?

Yes, there are two pitfalls of the Naïve Gauss elimination method.

Round-off error: The Naïve Gauss elimination method is prone to round-off errors. This is
true when there are large numbers of equations as errors propagate. Also, if there is
subtraction of numbers from each other, it may create large errors.

Division by zero: It is possible for division by zero to occur during the beginning of the n  1
steps of forward elimination.
For example:
5x2  6 x3  11
4 x1  5x2  7 x3  16
9 x1  2 x2  3x3  15
Will result in division by zero in the first step of forward elimination as the coefficient of x1
in the first equation is zero as is evident when we write the equations in matrix form.
0 5 6  x1  11
4 5 7  x   16
  2   
9 2 3  x3  15

But what about the equations below: Is division by zero a problem?

5x1  6 x2  7 x3  18
10 x1  12 x2  3x3  25
20 x1  17 x2  19 x3  56
Written in matrix form,

Computational Methods Page 11

Adama University Department Of Electrical Engineering

 5 6 7   x1  18 
10 12 3   x   25
  2   
20 17 19  x3  56
There is no issue of division by zero in the first step of forward elimination. The pivot
element is the coefficient of x1 in the first equation, 5, and that is a non-zero number.
However, at the end of the first step of forward elimination, we get the following equations
in matrix form
5 6 7   x1   18 
0 0  11  x     11
  2   
0  7  9   x3   16
Now at the beginning of the 2nd step of forward elimination, the coefficient of x2 in
Equation 2 would be used as the pivot element. That element is zero and hence would
create the division by zero problems. So it is important to consider that the possibility of
division by zero can occur at the beginning of any step of forward elimination.

What are some techniques for improving the Naïve Gauss elimination method?
One method of decreasing the round-off error would be to use more significant digits, that
is, use double or quad precision for representing the numbers. However, this would not
avoid possible division by zero errors in the Naïve Gauss elimination method.
To avoid division by zero as well as reduce (not eliminate) round-off error, Gaussian
elimination with partial pivoting is the method of choice.

How does Gaussian elimination with partial pivoting differ from Naïve Gauss elimination?
The two methods are the same, except in the beginning of each step of forward elimination;
a row switching is done based on the following criterion. If there are n equations, then
there are n  1 forward elimination steps.
a
At the beginning of the k th step of forward elimination, one finds the maximum of kk ,
a
a k 1,k , …………, nk .Then if the maximum of these values is a pk in the p th row, k  p  n ,
then switch rows p and k . The other steps of forward elimination are the same as the
Naïve Gauss elimination method. The back substitution steps stay exactly the same as the
Naïve Gauss elimination method.

Example 2
In the previous example, we used Naïve Gauss elimination to solve
20 x1  15x2  10 x3  45
 3x1  2.249 x2  7 x3  1.751
5x1  x2  3x3  9
The solution found was:

Computational Methods Page 12

Adama University Department Of Electrical Engineering

 x1   0.625 
X    x2    1.5 
 x3  0.99995

This is different from the exact solution of

 x1  1
X    x2   1
 x3  1
Find the solution using Gaussian elimination with partial pivoting

Solution
 20 15 10  x1   45 
 3  2.249 7   x  1.751
   2  
 5 1 3   x 3   9 
=
Forward Elimination of Unknowns
Now for the first step of forward elimination, the absolute value of the first column
20  3 5
elements below Row 1 is , , or 20, 3, 5.
So the largest absolute value is in the Row 1. So as per Gaussian elimination with partial
pivoting, the switch is between Row 1 and Row 1 to give

 20 15 10  x1   45 
 3  2.249 7   x  1.751
   2  
 5 1 3   x 3   9 
=

Divide Row 1 by 20 and then multiply it by –3, that is, multiply Row 1 by  3 / 20  0.15 .
20 15 10 45 0.15 Gives Row 1 as
 3  2.25  1.5  6.75
Subtract the result from Row 2
 3  2.249 7 1.751
  3  2.25  1.5  6.75
0 0.001 8.5 8.501
To get the resulting equations as

Computational Methods Page 13

Adama University Department Of Electrical Engineering

20 15 10   x1   45 
 0 0.001 8.5  x  8.501
   2  
 5 1 3   x3   9 
=
Divide Row 1 by 20 and then multiply it by 5, that is, multiply Row 1 by 5 / 20  0.25 .
20 15 10 45 0.25 Gives Row 1 as
5 3.75 2.5 11.25

Subtract the result from Row 3

5 1 3 9
 5 3.75 2.5 11.25
0  2.75 0.5  2.25
To get the resulting equations as
20 15 10   x1   45 
 0 0.001 8.5  x   8.501 
   2  
 0  2.75 0.5  x 3   2.25
=
This is the end of the first step of forward elimination.

Now for the second step of forward elimination, the absolute value of the second column
elements below Row 1 is
0.001  2.75
, or 0.001, 2.75
So the largest absolute value is in Row 3. So Row 2 is switched with Row 3 to give
20 15 10   x1   7 
 0  2.75 0.5  x   2.25
   2  
 0 0.001 8.5  x 3   8.501 
=
Divide Row 2 by –2.75 and then multiply it by 0.001, that is, multiply Row 2 by
0.001/  2.75  0.00036363 .
0  2.75 0.5  2.25 0.00036363 Gives Row 2 as
0 0.00099998  0.00018182 0.00081816

Subtract the result from Row 3

0 0.001 8.5 8.501
 0 0.00099998  0.00018182 0.00081816
0 0 8.50018182 8.50018184
To get the resulting equations as

Computational Methods Page 14

Adama University Department Of Electrical Engineering

20 15 10   x1   45 
 0  2.75 0.5  x    2.25 
   2  
 0 0 8.50018182  x 3  8.50018184
=
Back substitution:
8.50018182 x3  8.50018184
8.50018184
x3 
8.50018182
=1

Substituting the value of x3 in Row 2

 2.75x2  0.5x3  2.25
 2.25  0.5 x2
x2 
 2.75
 2.25  0.5  1

 2.75
1
Substituting the value of x3 and x 2 in Row 1
20 x1  15x2  10 x3  45
45  15 x2  10 x3
x1 
20
45  15  1  10  1

20
1
So the solution is,
 x1  1
X    x2  1
 x3  1
=
Can we use Naïve Gauss elimination methods to find the determinant of a square matrix?
One of the more efficient ways to find the determinant of a square matrix is by taking
advantage of the following two theorems on a determinant of matrices coupled with Naïve
Gauss elimination.

Theorem 1:
Let [ A] be a n  n matrix. Then, if [B] is a n  n matrix that results from adding or
subtracting a multiple of one row to another row, then det( A)  det(B) (The same is true for
column operations also).

Computational Methods Page 15

Adama University Department Of Electrical Engineering

Theorem 2:
Let [ A] be a n  n matrix that is upper triangular, lower triangular or diagonal, then
det( A)  a11  a22  ...  aii  ...  ann
n
  aii
i 1
This implies that if we apply the forward elimination steps of the Naïve Gauss elimination
method, the determinant of the matrix stays the same according to Theorem 1. Then since
at the end of the forward elimination steps, the resulting matrix is upper triangular, the
determinant will be given by Theorem 2.

Example 3
Find the determinant of
 25 5 1
[ A]   64 8 1
144 12 1
Solution
Using Naïve Gauss elimination method on [ A] gives,
25 5 1 
B   0  4.8  1.56

 0 0 0.7 

According to Theorem 2
det( A)  det(B)
 25  ( 4.8)  0.7
 84.00

What if I cannot find the determinant of the matrix using the Naïve Gauss elimination
method, for example, if I get division by zero problems during the Naïve Gauss elimination
method?
Well, we can apply Gaussian elimination with partial pivoting. However, the determinant of
the resulting upper triangular matrix may differ by a sign. The following theorem applies in
addition to the previous two to find the determinant of a square matrix.

Theorem 3:
Let [ A] be a n  n matrix. Then, if [B] is a matrix that results from switching one row with
another row, then det(B)   det( A) .

Example 4
Find the determinant of
Computational Methods Page 16
Adama University Department Of Electrical Engineering

 10  7 0
[ A]   3 2.099 6

 5  1 5
Solution
The end of the forward elimination steps of Gaussian elimination with partial pivoting, we
would obtain
10  7 0 

[B]   0 2.5 5 
 0 0 6.002

det B  10  2.5  6.002

 150.05
Since rows were switched once during the forward elimination steps of Gaussian
elimination with partial pivoting,
det  A   det(B)
 150.05

3.3.2 LU Decomposition Method

For a non-singular matrix A on which one can successfully conduct the Naïve Gauss
elimination forward elimination steps, one can always write it as

A  LU 
Where:
L = Lower triangular matrix
U  = Upper triangular matrix
Then if one is solving a set of equations
[ A][ X ]  [C ] ,
Then,
LU X   C  As [ A]  LU 
Multiplying both sides by L ,
1

L1 LU X   L1 C 

I U X  = L1 C  as L1 L  [ I ]
U X   L1 C  as I U   [U ]
Let
L1 C   Z 
Then
LZ   C  (1)

Computational Methods Page 17

Adama University Department Of Electrical Engineering

And
U X   Z  (2)
So we can solve Equation (1) first for [Z ] by using forward substitution and then use
Equation (2) to calculate the solution vector X  by back substitution.

How do I decompose a non-singular matrix [ A] , that is, how do I find A  LU?
If forward elimination steps of the Naïve Gauss elimination methods can be applied on a
nonsingular matrix, then A can be decomposed into LU as
 a11 a12  a1n 
a a 22  a2n 
[ A]   21 
     
 
 a n1 a n 2  a nn 
1 0  0 u11 u12  u1n 
 1  0  0 u22  u2 n 
  21  
          
  
  n1  n 2  1  0 0  unn 
The elements of the U  matrix are exactly the same as the coefficient matrix one obtains
at the end of the forward elimination steps in Naïve Gauss elimination.
The lower triangular matrix L  has 1 in its diagonal entries. The non-zero elements on the
non-diagonal elements in L  are multipliers that made the corresponding entries zero in
the upper triangular matrix U  during forward elimination.
Let us look at this using the same example as used in Naïve Gaussian elimination.

Example 1
Use LU decomposition to solve
20 x1  15x2  10 x3  45
 3x1  2.249 x2  7 x3  1.751
5x1  x2  3x3  9
Solution
Working in the matrix form,
[ A][ X ]  [C ]
 20 15 10  x1   45 
 3  2.249 7   x  1.751
   2  
 5 1 3   x 3   9 
=
Find the LU decomposition of the matrix
A  LU 

Computational Methods Page 18

Adama University Department Of Electrical Engineering

1 0 0 u11 u12 u13 

  21 1 0  0 u22 u23 

  
 31  32 1  0 0 u33 
The U  matrix is the same as found at the end of the forward elimination of Naïve Gauss
elimination method, that is
20 15 10 
U    0 0.001 8.5 

 0 0 23375.5
To find  21 and  31 , find the multiplier that was used to make the a 21 and a 31 elements
zero in the first step of forward elimination of the Naïve Gauss elimination method. It was
3
 21   0.15
20
5
 31   0.25
20
To find  32 , what multiplier was used to make a32 element zero? Remember a 32 element
was made zero in the second step of forward elimination. The A matrix at the beginning
of the second step of forward elimination was
20 15 10 
 0 0.001 8.5
 
 0  2.75 0.5
So,
 2.75
 32   2750
0.001

Hence,
 1 0 0
L   0.15 1 0

 0.25  2750 1

Now that L  and U  are known, solve LZ   C 

 1 0 0  z1   45 
 0.15 1 0  z 2  = 1.751

 0.25  2750 1  z 3   9 
This provides the three equations,
z1  45
 0.15z1  z 2  1.751
0.25z1   2750z 2  z3  9

Computational Methods Page 19

Adama University Department Of Electrical Engineering

Forward substitution starting from the first equation gives

z1  45
Substituting the value of z1 into the second equation,
z 2  1.751  0.15z1  1.751  0.15(45)  8.501
Substituting the values of z1 and z 2 into the third equation,
z3  9  0.25z1   2750z 2
 9  0.2545   27508.501
 23375.5
Hence,
 z1   45 
[ ]  z  =  8.501 
 2  
 z 3  23375.5

Now solve U X   Z 

20 15 10   x1   45 
 0 0.001 8.5  x   8.501 
  2  
 0 0 23375.5  x 3  23375.5
=
This provides the three equations for X 

From the third equation,

Substituting the value of in to the second equation,

( )

Substituting the values of and in to the first equation,

( ) ( )

The solution vector is,

 x1  1
x   1
 2
 x 3  1
How do I find the inverse of a square matrix using LU decomposition?
A matrix B  is the inverse of A if
AB  I   BA.
Computational Methods Page 20
Adama University Department Of Electrical Engineering

How can we use LU decomposition to find the inverse of the matrix? Assume the first
column of B  (the inverse of A ) is [b11 b12 ... ... bn1 ] , Then from the above definition of
T

an inverse and the definition of matrix multiplication:

b11  1
b  0
A 21    
   
   
bn1  0
Similarly the second column of B  is given by
 b12  0
b  1
A 22    
   
   
bn 2  0
Similarly, all columns of B  can be found by solving n different sets of equations with the
column of the right hand side being the n columns of the identity matrix.
Example 3
Use LU decomposition to find the inverse of
 25 5 1
A   64 8 1
144 12 1
Solution
Knowing that
A  LU 
 1 0 0 25 5 1 
 2.56 1 0  0  4.8  1.56
  
5.76 3.5 1  0 0 0.7 
We can solve for the first column of [ B]  A by solving for
1

 25 5 1 b11  1
 64 8 1 b   0
   21   
144 12 1 b31  0
First solve,
LZ   C  ,
That is
 1 0 0  z1  1
2.56 1 0  z   0
  2   
5.76 3.5 1  z 3  0
To give
z1  1
Computational Methods Page 21
Adama University Department Of Electrical Engineering

2.56 z1  z 2  0
5.76 z1  3.5z 2  z3  0

Forward substitution starting from the first equation gives

z1  1
z2  0  2.56z1
 0  2.561
 2.56
z3  0  5.76 z1  3.5z 2
 0  5.761  3.5 2.56
 3.2
Hence,
 z1   1 
Z    z 2    2.56
 z 3   3.2 
Now solve
U X   Z 
that is
25 5 1  b11   1 
 0  4.8  1.56 b    2.56
   21   
 0 0 0.7  b31   3.2 
   

25b11  5b21  b31  1

 4.8b21  1.56b31  2.56
0.7b31  3.2
Backward substitution starting from the third equation gives
3.2
b31 
0.7
 4.571
 2.56  1.56b31
b21 
 4.8
 2.56  1.56(4.571)

 4.8
 0.9524
1  5b21  b31
b11 
25
1  5(0.9524)  4.571

25
 0.04762

Computational Methods Page 22

Adama University Department Of Electrical Engineering

Hence the first column of the inverse of A is

b11   0.04762 
b    0.9524
 21   
b31   4.571 
Similarly by solving
 25 5 1 b12  0
 64 8 1 b   1
   22   
144 12 1 b32  0
b12   0.08333
b    1.417 
 22   
b32    5.000 
And solving
 25 5 1 b13  0
 64 8 1 b   0
   23   
144 12 1 b33  1
Gives
b13   0.03571 
b    0.4643
 23   
b33   1.429 
Hence
 0.04762  0.08333 0.03571 
A   0.9524 1.417  0.4643
1

 4.571  5.000 1.429 

Can you confirm the following for the above example?
AA1  I   A1 A
3.3.3 Gauss-Siedel Method
Why do we need another method to solve a set of simultaneous linear equations?
In certain cases, such as when a system of equations is large, iterative methods of solving
equations are more advantageous. Elimination methods, such as Gaussian elimination, are
prone to large round-off errors for a large set of equations. Iterative methods, such as the
Gauss-Seidel method, give the user control of the round-off error.
Given a general set of n equations and n unknowns, we have
a11x1  a12 x2  a13 x3  ...  a1n xn  c1
a21x1  a22 x2  a23 x3  ...  a2n xn  c2

Computational Methods Page 23

Adama University Department Of Electrical Engineering

. .
. .
an1 x1  an 2 x2  an3 x3  ...  ann xn  cn

If the diagonal elements are non-zero, each equation is rewritten for the corresponding
unknown, that is, the first equation is rewritten with x1 on the left hand side, the second
equation is rewritten with x 2 on the left hand side and so on as follows

 c  a x  a x ....  a x
1 12 1 13 3 1n n
x 1
a 11

c 2  a 21 x1  a 23 x3   a 2 n x n
x2 
a 22
.
.
c n 1  a n 1,1 x1  a n 1, 2 x 2   a n 1,n  2 x n  2  a n 1,n x n
x n 1 
a n 1,n 1
c n  a n1 x1  a n 2 x 2    a n ,n 1 x n 1
xn 
a nn

These equations can be rewritten in a summation form as

n
c1   a1 j x j
j 1
j 1
x1 
a11
n
c2   a2 j x j
j 1
j2
x2 
a 22
.
.
.
n
c n 1  a
j 1
n 1, j xj
j  n 1
x n 1 
a n 1,n 1
n
c n   a nj x j
j 1
jn
xn 
a nn
Hence for any row i ,

Computational Methods Page 24

Adama University Department Of Electrical Engineering

n
ci   aij x j
j 1
j i
xi  , i  1,2,, n.
aii
Now to find xi ’s, one assumes an initial guess for the xi ’s and then uses the rewritten
equations to calculate the new estimates. Remember, one always uses the most recent
estimates to calculate the next estimates, xi . At the end of each iteration, one calculates
the absolute relative approximate error for each xi as
x inew  x iold
a  100
i
x inew
new old
x
Where xi is the recently obtained value of i , and xi is the previous value of i .
x
When the absolute relative approximate error for each xi is less than the pre-specified
tolerance, the iterations are stopped.
The gauss-siedel method strictly converges if the coefficient matrix is strictly diagonally
dominant; otherwise the Gauss-Seidel method may or may not converge.

N.B: A matrix A of dimension NxN is said to be strictly diagonally dominant provided that
| | ∑ | | For k=1, 2,… ,N.

This means that in each row of the matrix the magnitude of the element on the main
diagonal must exceed the sum of the magnitudes of all other elements in the row.

Example 1
Find the solution to the following system of equations using the Gauss-Seidel method.
12 x1  3x2  5x3  1
x1  5x2  3x3  28
3x1  7 x2  13x3  76
Use
 x1  1
 x   0 
 2  
 x3  1
As the initial guess.

Solution
The coefficient matrix
12 3  5
A   1 5 3 
 3 7 13 
Is diagonally dominant as
a11  12  12  a12  a13  3   5  8
Computational Methods Page 25
Adama University Department Of Electrical Engineering

a22  5  5  a21  a23  1  3  4

a33  13  13  a31  a32  3  7  10
Hence, the solution should converge using the Gauss-Seidel method.
Rewriting the equations, we get
1  3 x 2  5 x3
x1 
12
28  x1  3x3
x2 
5
76  3x1  7 x2
x3 
13
Assuming an initial guess of
 x1  1
 x   0 
 2  
 x3  1
Iteration #1
Substituting the guess values into the first equation
1  30  51
x1 
12
 0.50000
Substituting the new value of and the remaining guess values into the second equation
28  0.50000  31
x2 
5
 4.9000
Substituting the new values of , , and the remaining guess values into the third
equation
76  30.50000  74.9000
x3 
13
 3.0923
The absolute relative approximate error for each then is
0.50000  1
a 1  100
0.50000
 100.000%
4.9000  0
a 2  100
4.9000
 100.000%
3.0923  1
a 3   100
3.0923
 67.662%
The maximum absolute relative approximate error is 100.000%
Iteration #2
The estimate of the solution vector at the end of Iteration #1 is
Computational Methods Page 26
Adama University Department Of Electrical Engineering

 x1  0.50000
 x    4.9000 
 2  
 x3   3.0923 
Substituting the values from Iteration #1 into the first equation
1  34.9000  53.0923
x1 
12
 0.14679
Substituting the new value of and the remaining values from Iteration #1 into the
second equation
28  0.14679  33.0923
x2 
5
 3.7153
Substituting the new value of , and the remaining values from Iteration #1 into the
second equation
76  30.14679  73.7153
x3 
13
 3.8118
At the end of second iteration, the absolute relative approximate error is
0.14679  0.50000
a 1  100
0.14679
 240.61%
3.7153  4.9000
a 2  100
3.7153
 31.889%
3.8118  3.0923
a 3   100
3.8118
 18.874%
The maximum absolute relative approximate error is 240.61%. This is greater than the
value of 100.00% we obtained in the first iteration. Is the solution diverging? No, as you
conduct more iteration, the solution converges as follows.

Iteration x1 a 1 % x2 a 2 % x3 a 3 %
1 0.50000 100.00 4.9000 100.00 3.0923 67.662
2 0.14679 240.61 3.7153 31.889 3.8118 18.874
3 0.74275 80.236 3.1644 17.408 3.9708 4.0064
4 0.94675 21.546 3.0281 4.4996 3.9971 0.65772
5 0.99177 4.5391 3.0034 0.82499 4.0001 0.074383
6 0.99919 0.74307 3.0001 0.10856 4.0001 0.00101

This is close to the exact solution vector of

Computational Methods Page 27

Adama University Department Of Electrical Engineering

 x1  1 
 x    3
 2  
 x3  4

Computational Methods Page 28