COMPUTATIONAL METHODS

&
STATISTICS
MTH 2212 -2011/2012 SEM 1
LECTURER
DR ZAHARAH WAHID
EXT 4514, ROOM E1-4-813
1

Gauss-Siedel Method

Commonly used iterative method

Employs initial guesses & then

iterates
to obtain estimate of the solutions

Gauss-Seidel Method

Advantages of the Gauss-Seidel Method

Algorithm for the Gauss-Seidel Method

Pitfalls of the Gauss-Seidel Method

Gauss-Seidel Method
An iterative method.
Basic Procedure:
Algebraically solve each linear equation for xi

Assume an initial guess solution array

Solve for each xi and repeat

Use absolute relative approximate error after each iteration

to check if error is within a pre-specified tolerance.
-

Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.

Elimination methods such as Gaussian Elimination and LU

Decomposition are prone to prone to round-off error.

Also: If the physics of the problem are understood, a close initial

guess can be made, decreasing the number of iterations needed.

Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:

a11 x1 a12 x2 a13 x3 ... a1n xn b1

a21 x1 a22 x2 a23 x3 ... a2n xn b2
.
.
.

.
.
.

an1 x1 an 2 x2 an 3 x3 ... ann xn bn

If: the diagonal elements are

non-zero

Rewrite each equation solving

for the corresponding unknown

ex:
First equation, solve for x1
Second equation, solve for x2

Gauss-Seidel Method
Algorithm
Rewriting each equation

c a12 x 2 a13 x3 a1n x n

x1 1
a11
x2

c2 a21 x1 a23 x3 a2 n xn
a22

xn 1
xn

From Equation 1

From equation 2

cn 1 an 1,1 x1 an 1, 2 x2 an 1,n 2 xn 2 an 1,n xn

From equation n-1

an 1,n 1

cn an1 x1 an 2 x2 an ,n 1 xn 1
ann

From equation n

Gauss-Seidel Method
Algorithm
General Form of each equation
n

c1 a1 j x j
x1

j 1
j 1

a11

cn 1
xn 1

a 22

j 1
j n 1

n 1, j

xj

an 1,n 1

c n a nj x j

c2 a2 j x j
x2

a
n

j 1
j2

xn

j 1
j n

a nn

 a11 x1 a12 x2 a13 x3 b1

a21 x1 a 22 x2 a23 x3 b2
a x a x a x b
31 1
32 2
33 3
3
If the diagonal elements are all non-zero then:

b1 a12 x2 a13 x3
x1
a11

x2

b2 a21 x1 a23 x3
a22

x3

b3 a31 x1 a32 x2
a33

With an initial guess of

i 1
1

i 1
2

i 1
3

10

x1
x
2
x3

b1 a12 x a13 x

a11
i
2

i
3

i 1
1

i
3

i 1
1

i 1
2

b2 a21 x
a23 x

a22
b3 a31 x
a32 x

a33

0
0
0

Gauss-Seidel Method
Algorithm
General Form for any row i
n

ci aij x j
xi

j 1
j i

aii

, i 1,2, , n.

How or where can this equation be used?

Particularly well suited for large numbers of equations
11

Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]

Important: Remember to use the

most recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.

x1
x
2

xn -1
xn

12

Use rewritten equations to solve for

each value of xi.

Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error

x inew x iold

100
new
xi

So when has the answer been found?

The iterations are stopped when the absolute relative
approximate error is less than a prespecified tolerance for all
unknowns.

13

Example: Surface Shape

Detection
To infer the surface shape of an object from images taken of a
surface from three different directions, one needs to solve the
following set of equations
0.9701
0.2425 0.9701

0.2357 0.2357 0.9428

0.2425

x1 247
x 248
2

x 3 239

The right hand side values are the light intensities from the
middle of the images, while the coefficient matrix is
dependent on the light source directions with respect to the
camera. The unknowns are the incident intensities that will
determine the shape of the object.
Find the values of x1, x2, and x3 use the Gauss-Seidel
method.
14

Example: Surface Shape

Detection
The system of equations is:

0
0.9701 x1 247
0.2425

0
0.2425 0.9701 x 2 248

0.2357 0.2357 0.9428 x3 239

Initial Guess:
x1
10
Assume an initial guess of x 10
2

x3

15

10

Example: Surface Shape Detection

Rewriting each
equation

0.9701

0.9701

0.2357 0.2357 0.9428

0.2425

0
0.2425

247 0 x 2 (0.9701) x3
x1
0.2425

x1 247
x 248
2

x 3 239

248 0 x1 (0.9701) x3
x2
0.2425

239 (0.2357) x1 (0.2357) x 2

x3
0.9428
16

Example: Surface Shape Detection

Iteration 1
x1
10

Substituting initial guesses

x

10
2


into the equations
247 0 10 0.9701 10
x1
1058.6
0.2425

x3

248 0 1058.6 0.9701 10

x2
1062.7
0.2425
239 0.2357 1058.6 0.2357 1062.7
x3
783.81
0.9428
17

10

Example: Surface Shape Detection

At the end of the first
iteration
xinew xiold
x
1058
.
56

a i

100
1
new
xi
x 1062.7
2

1058.6 10
x3 783.81
a 1
100 99.055%
1058.6

1062.7 10

100 99.059%
1062.7

a 3
18

783.81 10 100 101.28%

783.81

The maximum
absolute relative
approximate error
is 101.28%

Example: Surface Shape

Detection
Iteration 2
Using

x1 1058.6
x 1062.7
2

x 3 783.81

247 0 1062 .7 0.9701 783.81

x1
2117 .0
0.2425

19

x2

248 0 2117 .0 0.9701 783.81

2112 .9
0.2425

x3

239 0.2357 2117 .0 0.2357 2112 .9

803.98
0.9428

Example: Surface Shape

Detection
Finding the absolute relative approximate error for the second
iteration

a 1
a 2
a 3

2117 .0 1058.6 100 150.00%

2117 .0

2112.9 1062.7 100 150.30%

2112 .9

At the end of the first

iteration
x1 2117 .0
x 2112 .9
2

x3 803.98

803.98 783.81

100 197.49% The maximum absolute

relative approximate error
803.98
is 197.49%

20

ERROR
Perfect accuracy in most computational
processes is impossible
Error = true value- approximate value
Absolute error = |true value- approximate value|
The relative error is a measure of the
error in relation to the size of
true being sought
21

ERROR

22

Example: Surface Shape

Detection
Repeating more iterations, the following values are
obtained
a 1 %
x1
x2
x3
a 2 %
Iteration

a 3 %

1058.6

99.055

1062.7

99.059

783.81

101.28

2117.0

150.00

2112.9

150.30

803.98

197.49

4234.8

149.99

4238.9

149.85

2371.9

133.90

8470.1

150.00

8466.0

150.07

3980.5

159.59

16942

149.99

16946

149.96

8725.7

145.62

33888

150.00

33884

150.01

16689

152.28

Notice: The absolute relative approximate errors are not

decreasing.
23

Gauss-Seidel Method:
Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.

Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n

aii aij
j 1
j i

24

for all i

and

aii aij
j 1
j i

for at least one i

Gauss-Seidel Method: Pitfall

Diagonally Dominant: In other words.
For every row: the element on the diagonal needs to be equal
than or greater than the sum of the other elements of the
coefficient matrix
For at least one row: The element on the diagonal needs to be
greater than the sum of the elements.
What can be done? If the coefficient matrix is not originally
diagonally dominant, the rows can be rearranged to make it
diagonally dominant.
25

Example: Surface Shape

Detection
Examination of the coefficient matrix reveals that it is not
diagonally dominant and cannot be rearranged to become
diagonally dominant
0
0.9701
0.2425

0
0
.
2425

0
.
9701

0.2357 0.2357 0.9428

This particular problem is an example of a system of linear
equations that cannot be solved using the Gauss-Seidel
method.
Other methods that would work:
1. Gaussian elimination
Decomposition
26

2. LU

Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
But this is the same set of
equations used in example #2,
which did converge.

3 7 13
A 1 5 3
12 3 5
12 3 5
A 1 5 3
3 7 13

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equations can form a diagonally dominant matrix.

27

Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations

x1 x 2 x3 3
2 x1 3 x 2 4 x3 9

x1 7 x 2 x3 9
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?

28

Gauss-Seidel Method:
Example 2
Given the system of equations

12x 1 3x 2 - 5x 3 1

x 1 5x 2 3x 3 28
3x1 7x2 13x3 76

The coefficient matrix is:

12 3 5
A 1 5 3
3 7 13

With an initial guess of

x1
x
2
x3

29

1
0
1

Will the solution converge using the

Gauss-Siedel method?

Gauss-Seidel Method:
Example 2
Checking if the coefficient matrix is diagonally dominant
12 3 5
A 1 5 3
3 7 13

a11 12 12 a12 a13 3 5 8

a 22 5 5 a 21 a 23 1 3 4
a33 13 13 a31 a32 3 7 10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

30

Gauss-Seidel Method:
Example 2
Rewriting each equation
12 3 5
1 5 3

3 7 13

a1 1
a 28
2
a3 76

1 3 x 2 5 x3
x1
12

28 x1 3 x3
x2
5
76 3 x1 7 x 2
x3
13
31

With an initial guess of

x1

x
2

x3

x1

x2

1
0
1

1 3 0 51
0.50000
12

28 0.5 31
4.9000
5

76 3 0.50000 7 4.9000
x3
3.0923
13

Gauss-Seidel Method:
Example 2
The absolute relative approximate error

0.50000 1.0000
a 1
100 100.0%
0.50000

a
a

4.9000 0

100 100.00%
4.9000
3.0923 1.0000

100 67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%

32

Gauss-Seidel Method:
Example 2
After Iteration #1
x1 0.5000
x 4.9000
2

x3 3.0923
Substituting the x values into the
equations

1 3 4.9000 5 3.0923
x1
0.14679
12

x2

28 0.14679 3 3.0923
3.7153
5

x3
33

76 3 0.14679 7 4.900
3.8118
13

After Iteration #2

x1 0.14679
x 3.7153
2

x3 3.8118

Gauss-Seidel Method:
Example 2
Iteration #2 absolute relative approximate error
a 1

a 3

0.14679 0.50000
100 240.62%
0.14679

3.7153 4.9000

100 31.887%
3.7153

3.8118 3.0923

100 18.876%
3.8118

The maximum absolute relative error after the first iteration is 240.62%

This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
34

Gauss-Seidel Method:
Example 2
Repeating more iterations, the following values are obtained
Iteration
1
2
3
4
5
6

a1
0.50000
0.14679
0.74275
0.94675
0.99177
0.99919

a 1
67.662
240.62
80.23
21.547
4.5394
0.74260

a2
4.900
3.7153
3.1644
3.0281
3.0034
3.0001

x1 0.99919
The solution obtained

3
.
0001
2

x3 4.0001
x1

is close to the exact solution of x 2
x3
35

1
3
4

a3
2

100.00
31.887
17.409
4.5012
0.82240
0.11000

3.0923
3.8118
3.9708
3.9971
4.0001
4.0001

67.662
18.876
4.0042
0.65798
0.07499
0.00000

Gauss-Seidel Method:
Example 3
Given the system of equations
3x1 7 x2 13 x3 76

x1 5 x2 3 x3 28
12 x1 3 x2 5 x3 1
With an initial guess of
x1
x
2
x3
36

1
0
1

Rewriting the equations

76 7 x2 13x3
x1
3

28 x1 3 x3
x2
5

1 12 x1 3 x 2
x3
5

Gauss-Seidel Method:
Example 3
Conducting six iterations, the following values are obtained
Iteration

a1

1
2
3
4
5
6

21.000
196.15
1995.0
20149
2.0364105
2.0579105

a 1 %

A2

95.238
0.80000
110.71
14.421
109.83
116.02
109.90
1204.6
109.89
12140
109.89 1.2272105

a 2 %

a3

a 3 %

100.00
94.453
112.43
109.63
109.92
109.89

50.680
462.30
4718.1
47636
4.8144105
4.8653106

98.027
110.96
109.80
109.90
109.89
109.89

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?

37

38

Gauss-Seidel Method:
Example 3
Given the system of equations
3x1 7 x2 13 x3 76

x1 5 x2 3 x3 28
12 x1 3 x2 5 x3 1
With an initial guess of
x1
x
2
x3
39

1
0
1

Rearrange the equations

12 x1 3 x2 5 x3 1

x1 5 x2 3x3 28
3 x1 7 x2 13x3 76

Method:example2
Iteration #1

1 3 x 2 5 x3
x1
12

40

x1

With an initial guess of

x1

x
2

1 3 0 51
x3

12

0.50000

28 x1 3 x3
x2
5

28 0.5 31
x2
4.9000
5

76 3x1 7 x 2
x3
13

76 3 0.50000 7 4.9000
x3
3.0923
13

1
0
1

Gauss-Seidel Method:
Example 4
Rewriting the equations

76 7 x2 13 x3
x1
3

1 3 0 51
x1
0.50000
12

Relaxation
28 x1 3 x3
x2
5
1 12 x1 3 x 2
x3
5
41

Gauss-Seidel Method:
Example 4
Rewriting the equations

76 7 x2 13 x3
x1
3

28 x1 3 x3
x2
5
1 12 x1 3 x 2
x3
5

42

Gauss-Seidel Method: cont..

Iteration #1

1 3 x 2 5 x3
x1
12
28 x1 3 x3
x2
5
76 3x1 7 x 2
x3
13

43

76 3 0.6 7 3.093
x3
4.088
13