MTH2212 – Computational Methods and

Statistics

Solution of Linear System of Equations

Lecture 1:
Gaussian Elimination
 Objectives

 Introduction
 Gaussian Elimination
 Pitfalls in Gauss Elimination Method
 Pivoting
 Scaling

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 2

 Introduction

 We’ll see how to solve a set of n linear equations with n

unknowns.
 If n ≤ 3, the small set of linear equations can be solved
analytically.
 Simultaneous linear equations arise in a variety of
engineering problems context i.e. trusses, reactors, electric
circuits,…
 Broadly there are two types of methods to solve the linear
equations: Direct methods and Iterative methods.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 3

 Gaussian Elimination

 Solving a set of n linear equations:

a11 x1  a12 x2  a13 x3  ...  a1n xn  b1 (1a)

a x  a x  a x  ...  a x  b (1b)
 21 1 22 2 23 3 2n n 2
.

.
.
a x  a x  a x  ...  a x  b (1c)
 n1 1 n2 2 n3 3 nn n n

 Solution consists of two phases:

1. Forward elimination of unknowns
2. Back substitution

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 4

 Gaussian Elimination

1. Forward elimination of the unknowns

 Eliminate the first unknown, x1, from the second through the nth equations.
Multiply equation (1) by a21/a11
a21 a a a
a21 x1  a12 x2  21 a13 x3  ...  21 a1n xn  21 b1 (2)
a11 a11 a11 a11

 Subtract equation (2) from equation (1b) to get

a21 a21 a21 a21 (3)
(a22  a12 ) x2  (a23  a13 ) x3  ...  (a2 n  a1n ) xn  b2  b1
a11 a11 a11 a11

or '
a22 x2  a23
'
x3  ...  a2' n xn  b2'

where the prime indicates the new values for the elements.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 5

 Gaussian Elimination

 Similarly equation (1) can be multiplied by a31/a11 and the result subtracted
from the third equation.
 Repeating the procedure for the remaining equations give the following
modified forms:

a11 x1  a12 x2  a13 x3  ...  a1n xn  b1

 (4a)
 a '
x
22 2  a '
x
23 3  ...  a '
x
2n n  b '
2
(4b)
.

.
.
 a '
x  a '
x  ...  a '
x  b '
 n2 2 n3 3 nn n n
(4c)

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 6

 Gaussian Elimination

 The second unknown x2 is eliminated by multiplying through the next

equation by a’32/a’22 and subtracting the result from the following
equation.
 The procedure is continued using the remaining pivot equations. At the
end of the elimination stage, the original system of equations is
transformed to an upper triangular system:
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
 a '
x  a '
x  ...  a '
x  b '
 22 2 23 3 2 n n 2
(5a)
.
 (5b)
.
.
 ( n 1)
ann xn  bn( n 1)


(5c)

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 7

 Gaussian Elimination

2. Back Substitution
 The last unknown xn is now given by:

bn( n 1)
xn  ( n 1) (6)
ann

 This value of xn is then back-substituted into equation (n-1) to find xn-1.

This procedure can be repeated to evaluate the values of the remaining
x’s.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 8

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 9
 Example 1

Use Gauss elimination to solve:

3x1 – 0.1x2 – 0.2x3 = 7.85 (a)

0.1x1 + 7x2 – 0.3x3 = -19.3 (b)

0.3x1 – 0.2x2 + 10x3 = 71.4 (c)

use 6 significant figures in your computation.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 10

 Solution of Example 1

1. Forward elimination
 Multiply 1st equation (a) by 0.1/3 and subtract from 2nd equation (b) to eliminate
x1 from the latter equation:
7.00333x2 – 0.293333x3 = -19.5617

 Then multiply 1st equation (a) by 0.3/3 and subtract from 3rd equation (c) to
eliminate x1 from the latter equation:
-0.190000x2 + 10.0200x3 = 70.6150

 The system of equations is now:

3x1 – 0.1x2 – 0.2x3 = 7.85 (a’)
7.00333x2 – 0.293333x3 = -19.5617 (b’)
-0.190000x2 + 10.0200x3 = 70.6150 (c’)

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 11

 Solution of Example 1

 Now let’s eliminate x2 from latest 3rd equation (c’) by multiplying equation (b’)
by -0.190000/7.00333 and subtract the result from equation (c’):

10.0200x3 = 70.0843
 The system of equations is now reduced to an upper triangular form:

3 x1 – 0.1 x2 – 0.2 x3 = 7.85 (a’’)

7.00333 x2 – 0.293333 x3 = -19.5617 (b’’)

10.0200 x3 = 70.0843 (c’’)

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 12

 Solution of Example 1

2. Back Substitution
 Solve last equation (c’’) to find x3:
x3 = 70.0843/10.0200 = 7.00003

 Substitute x3 in equation (b’’) and solve for x2 :

7.00333x2 – 0.293333 (7.00003) = -19.5617
 x2 = -2.50000

 Finally substitute values of x2 and x3 in equation (a’’) to find x1 :

3x1 – 0.1(-2.5000) – 0.2(7.00003) = 7.85

 x1 = 3.00000

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 13

 Solution of Example 1

3. Check your results

 Substitute the values of x1 , x2 and x3 in the original system of equations.

3 (3.00000) – 0.1 (-2.50000) – 0.2 (7.00003) = 7.84999

0.1 (3.00000) + 7 (-2.50000) – 0.3 (7.00003) = -19.30000

0.3 (3.00000) – 0.2 (-2.50000) + 10 (7.00003) = 71.40030

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 14

 Some Pitfalls of Gauss Elimination Method

1. Division by zero

 Caused by coefficient value equals zero or very close to zero

 This can be solved by using pivoting technique.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 15

 Some Pitfalls of Gauss Elimination Method

2. Round-off errors
 This may be due to some of the following:
 Large number of equations to be solved due to the fact that every
result is dependent on previous results.
 Errors in early steps will tend to propagate.

 Round-off errors may be solved by:

 Using more significant figures.
 Using fractions instead of decimals.
 Always substitute your answers back into the original system of
equations to check for occurrence of substantial errors.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 16

 Pivoting

This involves the following steps:

1. Determine the largest coefficient available in the column below

the pivot element.

2. Switch the rows so that the largest element is the pivot element.
This is known as partial pivoting.

3. If columns as well as rows are searched for the largest element

and then switched, the process is called complete pivoting.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 17

 Example 2

Solve the following system of equations

0.0003 x1 + 3.0000 x2 = 2.0001 (1)

1.0000 x1 + 1.0000 x2 = 1.0000 (2)

1. Using naïve Gauss elimination

2. Using partial pivoting

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 18

 Solution of example 2

1. Firstly, let’s solve these equations the same way we did in example 1:

Forward elimination:
− Multiply equation (1) by 1/0.0003 to get

x1 + 10,000x2 = 6667
− Subtract the resulting equation from equation (2) to get
-9999x2 = -6666  x2 = 2/3

Back substitution:
0.0003x1 + 3.0000x2 = 2.0001  x1 = 2.0001 – 3(2/3)
0.0003

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 19

 Solution of example 2

 Now let’s check the effects of number of significant digits

on the result of x1 due to x2:

Significant
figures x2 x1 εx1(%)
3 0.667 -3.33 1099
4 0.6667 0.0000 100
5 0.66667 0.30000 10
6 0.666667 0.330000 1
7 0.6666667 0.3330000 0.1

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 20

 Solution of example 2

2. Secondly, let’s solve these equations by applying partial pivoting:

1.0000 x1 + 1.0000 x2 = 1.0000 (1)
0.0003 x1 + 3.0000 x2 = 2.0001 (2)

Forward elimination:
− Multiply equation (1) by 0.0003 to get

0.0003 x1 + 0.0003 x2 = 0.0003

− Subtract the resulting equation from equation (2) to get
2.9997x2 = 1.9998  x2 = 2/3

Back substitution:
x1 + x2 = 1  x1 = 1 – (2/3)
1

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 21

 Solution of example 2

 This case is much less sensitive to the number of significant

figures on the result of x1 due to x2:

Significant
figures x2 x1 εx1(%)
3 0.667 0.333 0.1
4 0.6667 0.3333 0.01
5 0.66667 0.33333 0.001
6 0.666667 0.333333 0.0001
7 0.6666667 0.3333333 0.00001

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 22

 Scaling

 Scaling is necessary when different units are used in the

same system of equations.

 Scaling is used to:

1. Standardize the size of the coefficients in the system of
equations.
2. Minimize the round-off errors caused by having much larger
coefficients than others.

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 23

 Example 3

Use three significant figures to solve

2 x1 + 100000 x2 = 10000 (1)

x1 + x2 = 2. (2)

1. Naïve Gauss elimination

2. Gauss elimination with scaling and pivoting
3. Gauss elimination without scaling and with pivoting

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 24

 Solution of example 3

1. Gauss elimination:
− Forward elimination
2 x1 + 100000 x2 = 10000 (1)
- 50000 x2 = -5000 (2)
− Back substitution
x2 = 0.10 and x1 = 0.00

2. Gauss elimination with scaling and pivoting:

− Scaling
0.00002 x1 + x2 = 0.1 (1)
x1 + x2 = 2 (2)

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 25

 Solution of example 3

− Pivoting
x1 + x2 = 2 (1)
0.00002 x1 + x2 = 0.1 (2)

− Forward elimination
x1 + x2 = 2
x2 = 0.1

− Back substitution
x1 = 1.90 and x2 = 0.10

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 26

 Solution of example 3

3. Gauss elimination without scaling and with pivoting of the

original equations:
− Pivoting
x1 + x2 = 2.
2 x1 + 100000 x2 = 10000 (1)
− Forward elimination
x1 + x2 = 2
100000 x2 = 10000
− Back substitution
x1 = 1.90 and x2 = 0.10

Dr. M. Hrairi MTH2212 - Computational Methods and Statistics 27