LINEAR SYSTEMS OF EQUATIONS

AND
MATRIX COMPUTATIONS
 1. DIRECT METHODS FOR SOLVING LINEAR SYSTEMS OF EQUATIONS

1.1 SIMPLE GAUSSIAN ELIMINATION METHOD

Consider a system of n equations in n unknowns,

a11x1 + a12x2 + …. + a1nxn = y1

a21x1 + a22x2 + …. + a2nxn = y2
… … … … …
an1x1 + an2x2 + …. + annxn = yn

We shall assume that this system has a unique solution and proceed to describe the
simple “Gaussian Elimination Method”, (from now on abbreviated as GEM),Page 2 of 11
for finding the solution. The method reduces the system to an upper triangular system
using elementary row operations (ERO).

Let A(1) denote the coefficient matrix A.

 a 11( 1 ) a 12( 1 ) ..... a 1( 1n ) 

 
 (1 )
a 21 (1 )
a 22 ..... a 2( 1n) 
 ..... ...... ..... ...... 
A = 
(1)

 ..... ...... ...... ...... 
 
 a n( 11 ) a n( 12) ...... (1 )
a nn 

where aij(1) = aij

Let
 y1(1) 
 (1) 
 y2 
y(1) =  M 
 
 y (1) 
 n 

where yi(1) = yi

We assume a11(1) ≠ 0

Then by ERO applied to A(1) , (that is, subtracting suitable multiples of the first
row from the remaining rows), reduce all entries below a11(1) to zero. Let the
resulting matrix be denoted by A(2).

Ri +mi(11)R1
A(1) 
→ A(2)

2
 ai(11)
where mi(11) = − ; i > 1.
a11(1)

Note A(2) is of the form

 a11(1) a12(1) ... ... a1(n1) 
 
 0 a22 ( 2)
... ... a2( 2n) 
 0 a ( 2) ... ... a3( n2) 
A(2) =  32
 M M M M M 
 ( 2) ( 2) 
 0 an 2 ... ... ann 
Notice that the above row operations on A(1) can be effected by premultiplying
A(1) by M(1) where

 1 0 0 L 0 0
 (1) 
 m21 
(1)
M = m31  (1)
I n −1 
 
 M 
 (1) 
 mn1 
(In-1 being the n-1 × n-1 identity matrix).
i.e.
M(1) A(1) = A(2)

Let
y(2) = M(1) y(1)
i.e.
R i + m i 1 R1
y (1)   → y ( 2 )
Then the system Ax = y is equivalent to

A(2)x = y(2)

Next we assume

(2)
a22 ≠0

and reduce all entries below this to zero by ERO

Ri +mi(22)
A(2) → A(3) ;

3
 ai(22)
m ( 2)
i2 = − ( 2) ; i>3
a22
Here

1 0 0 L 0
 
0 1 0 L 0
 0 m ( 2) 
M ( 2) = 32

 0 m42 
( 2)
I n −2
 
M M 
 0 m ( 2) 
 n2 

and
M(2) A(2) = A(3) ; M(2) y(2) = y(3) ;

and A(3) is of the form

 a11(1) a12(1) ... ... a1(n1) 

 
 0 (2)
a22 ( 2)
a23 ... a2( 2n) 
A( 3 ) = 0 0 ( 3)
a33 ... a3(3n) 


 M M M ... M 
 (3) 
 0 0 an( 33) ... ann 

We next assume a33(3)

≠ 0 and proceed to make entries below this as zero. We
thus get M , M , …. , M(r) where
(1) (2)

1 0 L 0 
 
0 1 L 0 0 r ×( n − r ) 
M M M M 
 
0 0 L 1
M (r ) = 
0 0 L mr( +r )1r 
 
0 0 L mr( +r )2 r I n −r 
M M M M 
 
0 0 L mnr( r ) 
 

4
  a11(1) ... ... ... ... a1(n1) 
 
 0 ( 2)
a22 ... ... ... a2( rn) 
 
M 0 arr( r ) ... ... arn( r ) 
M (r )
A (r )
=A ( r +1)
=
 M M 0 ar( r++1r1+) 1 ... ar( +r 1+n1) 
 
 M M M ... ... ... 
 0 anr( r ++11) ( r +1) 
 0 0 ... ann 
M(r) y(r) = y(r+1)

At each stage we assume arr( r ) ≠ 0 .

Proceeding thus we get,

M(1), M(2), …. , M(n-1) such that

M(n-1) M(n-2) …. M(1) A(1) = A(n) ; M(n-1) M(n-2) …. M(1) y(1) = y(n)
 a11(1) a12(1) L a1(n1) 
 
 ( 2)
a22 L a2( 2n) 
A( n ) = 
where  
 O 
 ( n) 
 ann 
which is an upper triangular matrix and the given system is equivalent to

A(n)x = y(n)

Since this is an upper triangular, this can be solved by back substitution; and
hence the system can be solved easily.

Note further that each M(r) is a lower triangular matrix with all diagonal entries as
1. Thus determinant of M(r) is 1 for every r. Now,

A(n) = M(n-1) …. M(1) A(1)

Thus

det A(n) = det M(n-1) det M(n-2) …. det M(1) det A(1)

det A(n) = det A(1) = det A since A = A(1)

5
Now A(n) is an upper triangular matrix and hence its determinant is a11(1) a22
(2) (n)
L ann .
Thus det A is given by

det A = a11(1) a22

(2) ( n)
L ann

Thus the simple GEM can be used to solve the system Ax = y and also to
evaluate det A provided aii( i ) ≠ 0 for each i.
Further note that M(1), M(2), …. , M(n-1) are lower triangular, and nonsingular as
their det = 1 and hence not zero. They are all therefore invertible and their
inverses are all lower triangular, i.e. if L = M(n-1) M(n-2) …. M(1) then L is lower
triangular and nonsingular and L-1 is also lower triangular.

Now LA = LA(1) = M(n-1) M(n-2) …. M(1) A(1) = A(n)

Therefore A = L-1 A(n)

Now L-1 is lower triangular which we denote by L and A(n) is upper triangular
which we denote by U, and we thus get the so called LU decomposition

A = LU

of a given matrix A – as a product of a lower triangular matrix with an upper

triangular matrix. This is another application of the simple GEM. REMEMBER IF
AT ANY STAGE WE GET aii(1) = 0 WE CANNOT PROCEED FURTHER WITH
THE SIMPLE GEM.

EXAMPLE:

Consider the system

x1 + x2 + 2x3 = 4
2x1 - x2 + x3 = 2
x1 + 2x2 =3

Here

 1 1 2 4
   
A =  2 −1 1 y = 2
1 2 0 3
   

6
 1 1 2 1 1 2 
  R 2 − 2 R1  
= 2 −1 → −3 − 3  = A (2)
(1)
A 1 0
1 0 
R 3 − R1 0 − 2 
 2  1

a11(1) = 1 ≠ 0
(1)
m21 = −2
(1)
m31 = −1
(2)
a22 = −3 ≠ 0

 1 0 0 4  4 
     
M (1)
=  − 2 1 0 y (1 )
=  2  →  − 6  = y (2) = M (1 )
y (1 )
 − 1 0 1 3  −1
     

1
R3 + R 2 1 1 2 
3
 
→ −3 − 3  = A ( 3) = −3
(2) (3)
A 0 a33
0 − 3 
 0

(2)
m31 =1 3

1 0 
 0  4 
 
M (2)
= 0 1 0 y (3)
= M (2)
y (2)
= − 6
   − 3
0 1 1  
 3 

Therefore the given system is equivalent to A(3)x = y(3) ,i.e.,

x1 + x2 + 2x3 = 4

-3x2 - 3x3 = -6

- 3x3 = -3

Back Substitution

x3 = 1

-3x2 - 3 = - 6 ⇒ -3x2 = -3 ⇒ x2 = 1

7
x1 + 1 + 2 = 4 ⇒ x1 = 1

Thus the solution of the given system is,

 x1   1 
   
x =  x 2  = 1
 x  1
 3  
The determinant of the given matrix A is

a11(1) a22 a33 = (1)(−3)(−3) = 9.

(2) (3)

Now

1 0 0
(M )  
( −1)
(1)
=  2 1 0
1 0 1
 

1 0 0 

(M ( ) )
( −1)
= 0 0
2
1
 
0 − 1 1
 3 

L = M(2) M(-1)

L-1= ( M (2) M (1) ) = ( M (1) ) (M )

−1 −1 (2) −1

 0 
 1 0 0  1 0
 
=  2 1 0  0 1 0
 1 0 1  
  0 − 1 1
 3 

1 0 0 

L = L(-1) =  2 1 0
 
1 − 1 1
 3 

1 1 2 
 
U=A (n)
=A (3)
=  0 − 3 − 3
 0 0 − 3
 

8
Therefore A = LU i.e.,

1 0 
1 1 2  0 1 1 2 
 
A=  2 −1 1  =  2 1 0  0 − 3 − 3
1 2 0    0 0 − 3
  1 − 1 1  
 3 

is the LU decomposition of the given matrix A.

We observed that in order to apply simple GEM we need arr( r ) ≠ 0 for each
stage r. This may not be satisfied always. So we have to modify the simple
GEM in order to overcome this situation. Further, even if the condition arr( r ) ≠ 0 is
satisfied at each stage, simple GEM may not be a very accurate method to use.
What do we mean by this? Consider, as an example, the following system:

(0.000003) x1 + (0.213472) x2 + (0.332147) x3 = 0.235262

(0.215512) x1 + (0.375623) x2 + (0.476625) x3 = 0.127653
(0.173257) x1 + (0.663257) x2 + (0.625675) x3 = 0.285321

Let us do the computations to 6 significant digits.

Here,

 0 .000003 0 .213472 0 .332147 

 
A(1) =  0 .215512 0 .375623 0 .476625 
 0 . 173257 0 .625675 
 0 . 663257

 0 . 235262 
 
y (1)
=  0 . 127653  a11(1) = 0.000003 ≠ 0
 0 . 285321 
 

(1)
a21 0.215512
(1)
m21 =− (1)
=− = −71837.3
a11 0.000003

(1)
a31 0.173257
(1)
m31 =− (1)
=− = −57752.3
a11 0.000003

9
  1 0 0  0.235262 
   
M (1)
=  − 71837 .3 1 0 ; y (2) (1)
=M y(1)
=  − 16900.5 
 − 57752 . 3 1   − 13586.6 
 0  

 0.000003 0.213472 0.332147 

 
A(2) = M(1) A(1) =  0 − 15334 .9 − 23860 .0 
 − 12327 .8 − 19181 .7 
 0

(2)
a22 = −15334.9 ≠ 0

(2)
a32 −12327.8
(2)
m32 =− =− = −0.803905
(2)
a22 −15334.9

1 0 0
 
M(2) =  0 1 0
 0 − 0.803905 1 
 

 0 .235262 
 
y (3)
=M (2) (2)
y =  − 16900 . 5 
 − 0 . 20000 
 

 0.000003 0.213472 0.332147 

 
A(3) = M(2) A(2) =  0 − 15334 .9 − 23860 .0 
 − 0.50000 
 0 0

Thus the given system is equivalent to the upper triangular system

A(3)x = y(3)

Back substitution yields,

x3 = 0.40 00 00
x2 = 0.47 97 23
x1 = -1.33 33 3

10
This compares poorly with the correct answers (to 10 digits) given by

x1 = 0.67 41 21 46 94
x2 = 0.05 32 03 93 39.1
x3 = -0.99 12 89 42 52

Thus we see that the simple Gaussian Elimination method needs modification in
order to handle the situations that may lead to arr( r ) = 0 for some r or situations as
arising in the above example. In order to do this we introduce the idea of Partial
Pivoting in the next section.

11