You are on page 1of 35

# Gauss-Siedel Method

Major: All Engineering Majors Authors: Autar Kaw (Modified by P. Goel for IDC103)

http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM Undergraduates
http://numericalmethods.eng.usf.edu

3/25/2010

Gauss-Seidel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
An iterative method. Basic Procedure :
-Algebraically solve each linear equation for xi -Assume an initial guess solution array -Solve for each xi and repeat -Use absolute relative approximate error after each iteration to check if error is within a pre-specified tolerance.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Why?
Obviously it isn t possible to obtain better than machine precision for a solution of SLEs (using Gaussian Elimination and LU Decomposition). Actually, the situation is worse for large systems: it isn t possible to get close to machine precision in direct methods.

The Gauss-Seidel Method allows the user to control round-off error. In fact, iterative methods can be used to improve the solution obtained by direct methods.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:

## a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + ... + a2n xn = b2
. . . . . .

non-zero

## Rewrite each equation solving

for the corresponding unknown

## an1 x1 + an 2 x2 + an 3 x3 + ... + ann xn = bn

ex:
First equation, solve for x1 Second equation, solve for x2

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
Rewriting each equation

x1 =

## c1 - a12 x 2 - a13 x3 KK - a1n x n a11

c2 - a21 x1 - a23 x3 KK - a2 n xn a22 M M M cn -1 - an -1,1 x1 - an -1, 2 x2 KK - an -1,n -2 xn -2 - an -1, n xn an -1,n -1 ann

From Equation 1

x2 =

From equation 2

xn -1 = xn =

## From equation n-1

cn - an1 x1 - an 2 x2 - KK - an , n -1 xn -1

From equation n

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form of each equation

c1 - a1 j x j x1 =
j =1 j 1

cn -1 xn -1 =

j =1 j n -1

a
n

n -1, j

xj

a11

an -1,n -1

c2 - a2 j x j x2 =
j =1 j2

c n - a nj x j xn =
j =1 jn

a 22

a nn
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form for any row i

ci - aij x j xi =
j =1 j i

aii

, i = 1,2,K , n.

## How or where can this equation be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X] Use rewritten equati ons to solve for each value of x i. Important: Remember to use the most recent value of xi. Which means to apply values calculated to the calculations remaining in the current iteration.

x1 x 2 M xn -1 xn

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error

a i =

new i

-x
new i

old i

100

## So when has the answer been f ound?

The iterations are stopped when the absolute relative approximate error is less than a prespecified tolerance for all unknowns.

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 1

The system of equati ons

## Initial Guess: Assume an initial guess of

a1 1 a = 2 2 a 3 5

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 1

Rewriting each equation

106.8 - 5a 2 - a 3 a1 = 25
177.2 - 64a1 - a 3 a2 = 8
279.2 - 144a1 - 12a 2 a3 = 1

## 25 5 1 a1 106 .8 64 8 1 a = 177 .2 2 144 12 1 a3 279 .2

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 1

Applying the initial guess and solving for ai
a1 1 a = 2 2 a 3 5
Initial Guess

a1 =

## 177.2 - 64(3.6720 ) - (5) a2 = = -7.8510 8

a3 = 279.2 - 144(3.6720 ) - 12(- 7.8510 ) = -155.36 1

When solving for a2, how many of the initial guess values were used?

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 1

Finding the absolute relative approximate error

a i

## xinew - xiold = 100 new xi

3.6720 - 1.0000 x100 = 72.76% 3.6720

a 1 =

## a1 3.6720 a = - 7.8510 2 a 3 - 155 .36

The maximum absolute relative approximate error is 125.47%

a 3 =

## - 155.36 - 5.0000 x100 = 103.22% - 155.36

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 1

Using

Iteration #2
a1 = 106.8 - 5(- 7.8510 ) - 155.36 = 12.056 25
177.2 - 64(12.056 ) - 155.36 = -54.882 8
the values of ai are found:

## a1 3.6720 a = - 7.8510 2 a 3 - 155.36

from iteration #1

a2 =

a3 =

## 279.2 - 144(12.056 ) - 12(- 54.882 ) = -798.34 1

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 1

Finding the absolute relative approximate error 12.056 - 3.6720 At the end of the second iteration a 1 = x100 = 69.543% 12.056 a 12.056
a 2 = - 54.882 - (- 7.8510) x100 = 85.695% - 54.882

a = - 54.882 2 a3 - 798.54

a 3 =

## The maximum absolute relative approximate error is 85.695%

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 1

Repeating more iterations, the following values are obtained
Iteration

## a1 3.6720 12.056 47.182 193.33 800.53 3322.6

a 1 %
72.767 69.543 74.447 75.595 75.850 75.906

## a2 7.8510 54.882 255.51 1093.4 4577.2 19049

a 2 %
125.47 85.695 78.521 76.632 76.112 75.972

## a3 155.36 798.34 3448.9 14440 60072 249580

a 3 %
103.22 80.540 76.852 76.116 75.963 75.931

1 2 3 4 5 6 Notice

The relative errors are not dec reasing at any significant rate

a 1 0.29048 Also, the solution is not converging to the true s olution of a 2 = 19.690 a 3 1.0857
http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Pitfall

What went wrong?
Even though done correctly, the answer is not converging to the correct answer This example illustrates a pitfall of the Gaus s-Siedel method: not all systems of equations will converge.

Is there a fix?

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Pitfall

GAUSS-SEIDEL CONVERGENCE THEOREM: If A is diagonally dominant, then the Gauss-Seidel method converges for any starting vector x. A sufficient, but not necessary condition.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:

aii aij
j =1 j i

for all i

and

j =1 j i

## for at least one i

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Pitfall

Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other c oefficients in that row and at least one row with a diagonal coefficient greater than the sum of the other coefficients in that row. Which coefficient matrix is diagonally dominant?
2 5.81 34 [A ] = 45 43 1 123 16 1
124 34 56 [ B] = 23 53 5 96 34 129

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 2

Given the system of equations The coefficient matrix is:

12 x1 + 3 x2- 5 x3 = 1

x1 + 5 x2 + 3x3 = 28
3x1 + 7 x2 + 13x3 = 76
With an initial guess of

12 3 - 5 [A] = 1 5 3 3 7 13
Will the solution converge using the Gauss-Siedel method?

x1 1 x = 0 2 x3 1

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 2

Checking if the coefficient matrix is diagonally dominant
12 3 - 5 [A] = 1 5 3 3 7 13

## a11 = 12 = 12 a12 + a13 = 3 + - 5 = 8

a 22 = 5 = 5 a 21 + a 23 = 1 + 3 = 4
a33 = 13 = 13 a31 + a32 = 3 + 7 = 10

The inequalities are all true and at least one row is strictly greater than: Therefore: The solution should converge using the Gauss-Siedel Method

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 2

Rewriting each equation
12 3 - 5 a1 1 1 5 3 a = 28 2 3 7 13 a3 76

## With an initial guess of

x1 1 x = 0 2 x 3 1
x1 = 1 - 3(0) + 5(1) = 0.50000 12

1 - 3x 2 + 5 x3 x1 = 12 28 - x1 - 3x3 x2 = 5

## 28 - (0.5) - 3(1) x2 = = 4.9000 5

76 - 3(0.50000 ) - 7(4.9000 ) x3 = = 3.0923 13
http://numericalmethods.eng.usf.edu

76 - 3x1 - 7 x 2 x3 = 13

## Gauss-Seidel Method: Example 2

The absolute relative approximate error

a
a

## 4.9000 - 0 = 100 = 100.00% 4.9000

3.0923 - 1.0000 = 100 = 67.662% 3.0923

The maximum absolute relative error after the first iteration is 100%
http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 2

After Iteration #1
x1 0.5000 x = 4.9000 2 x3 3.0923

## Substituting the x values into the equations

x1 =

After Iteration #2
x1 0.14679 x = 3.7153 2 x3 3.8118

x3 =

## 76 - 3(0.14679 ) - 7(4.900 ) = 3.8118 13

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 2

Iteration #2 absolute relative approximate error
a 1 = 0.14679 - 0.50000 100 = 240.61% 0.14679

## 3.8118 - 3.0923 = 100 = 18.874% 3.8118

The maximum absolute relative error after the fi rst iteration is 240.61%

This is much larger than the maxi mum absolute relative error obtai ned in iteration #1. Is this a problem?
http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 2

Repeating more iterations, the following values are obtained
Iteration 1 2 3 4 5 6 a1 0.50000 0.14679 0.74275 0.94675 0.99177 0.99919

a 1 %
100.00 240.61 80.236 21.546 4.5391 0.74307

## a2 4.9000 3.7153 3.1644 3.0281 3.0034 3.0001

a 2 %
100.00 31.889 17.408 4.4996 0.82499 0.10856

## a3 3.0923 3.8118 3.9708 3.9971 4.0001 4.0001

a 3 %
67.662 18.876 4.0042 0.65772 0.074383 0.00101

x1 0.99919 The solution obtained x = 3.0001 is close to the exact solution of 2 x3 4.0001

x1 1 x = 3 . 2 x 3 4

http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 3

Given the system of equations

3 x1 + 7 x2 + 13 x3 = 76

## Rewriting the equations

x1 + 5 x2 + 3x3 = 28
12 x1 + 3x2 - 5 x3 = 1
With an initial guess of

76 - 7 x2 - 13 x3 x1 = 3

x1 1 x = 0 2 x3 1

28 - x1 - 3x3 x2 = 5

1 - 12 x1 - 3 x 2 x3 = -5
http://numericalmethods.eng.usf.edu

## Gauss-Seidel Method: Example 3

Conducting six iterations, the following values are obtained
Iteration a1

a 1 %

A2

a 2 %
100.00 94.453 112.43 109.63 109.92 109.89

a3

a 3 %
98.027 110.96 109.80 109.90 109.89 109.89

1 2 3 4 5 6

## 21.000 196.15 1995.0 20149 2.0364105 2.0579105

95.238 0.80000 110.71 14.421 109.83 116.02 109.90 1204.6 109.89 12140 109.89 1.2272105

## 50.680 462.30 4718.1 47636 4.8144105 4.8653106

The values are not converging. Does this mean that the Gauss-Seidel method cannot be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not diagonally dominant But this is the same set of equations used in example #2, which did converge.

3 7 13 [A] = 1 5 3 12 3 - 5
12 3 - 5 [ A] = 1 5 3 3 7 13

If a system of linear equations is not diagonally dominant, check to see if rearranging the equati ons can form a di agonally dominant matrix.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Not every system of equations can be rearranged to have a diagonally dominant coefficient matrix.
Observe the set of equati ons

x1 + x 2 + x3 = 3
2 x1 + 3 x 2 + 4 x3 = 9

x1 + 7 x 2 + x3 = 9
Which equation(s) prevents this set of equation from having a diagonally dominant coefficient matrix?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method -Algorithm for the Gauss-Seidel Method -Pitfalls of the Gauss -Seidel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method

Questions?

http://numericalmethods.eng.usf.edu

Additional Resources
For all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit http://numericalmethods.eng.usf.edu/topics/gauss_seid el.html

THE END
http://numericalmethods.eng.usf.edu