Chapter 11

Iterative Methods for Linear and

Nonlinear Equations

“These notes are only to be used in class presentations”

1
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Solution of Nonlinear Systems of Equations

• Fixed Point Iteration:

f1 ( x1 , x2 , x3 ,  , xn )  0  x1  g1 ( x1 , x2 , x3 ,  , xn )
f 2 ( x1 , x2 , x3 ,  , xn )  0  x2  g 2 ( x1 , x2 , x3 ,  , xn )

f n ( x1 , x2 , x3 ,  , xn )  0  xn  g n ( x1 , x2 , x3 ,  , xn )

2
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 • Fixed Point Iteration:

x1( k 1)  g1 ( x1( k ) , x2( k ) , x3( k ) ,  , xn( k ) )

x2( k 1)  g 2 ( x1( k ) , x2( k ) , x3( k ) ,  , xn( k ) ) Resembles Jacobi

iteration

xn( k 1)  g n ( x1( k ) , x2( k ) , x3( k ) ,  , xn( k ) )

x1( k 1)  g1 ( x1( k ) , x2( k ) , x3( k ) , , xn( k)1, xn( k ) )

x2( k 1)  g 2 ( x1( k 1) , x2( k ) , x3( k ) , , xn( k)1, xn( k ) ) Resembles
Gauss-Seidel

xn( k 1)  g n ( x1( k 1) , x2( k 1) , x3( k 1) ,, xn( k11) , xn( k ) )
3
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Convergence Criterion:

Let D={(x1, x2,…… xn )} ai≤xi≤bi for each i=1,2,…n. Suppose G

is a continuous function and G(x)Є D whenever x Є D, then G
has a fixed point in D.
In addition, suppose all component functions of G have
continuous partial derivatives and a constant K<1 exists with

for each j=1,….n

then fixed point iteration method converges.

4
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Example 11.5:
Use fixed-point iteration method to determine the roots of
x1  x2  x2  0.25
8 x12  16 x2  8 x1x2  5
with initial guesses of x1(0) = x2(0) =0. Compute 3 iterations
with Gauss-Seidel Scheme.

iter. no x1 x2

0 0 0

1 0.2500

2 0.4955 0.2522

3 0.5000 0.2500
5
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Example 11.6:
Use fixed-point iteration method to determine the roots of

x1  10 x1  x22  8  0
2

x1 x22  x1  10 x2  8  0
with D{ (x1 , x2) \ 0≤ x1, x2≤ 1.5}. Compute 3 iterations with
Gauss-Seidel Scheme and calculate εa in each iteration by
using maximum magnitude norm.

iter. no. x1 x2 εa
0 0 0
1 0.8 0.88
2 0.9414 0.967 0.1462
3 0.9821 0.99 0.0411
6
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
• Newton-Raphson method
f1 ( x1 , x2 , x3 ,  , xn )  0
f 2 ( x1 , x2 , x3 ,  , xn )  0

f n ( x1 , x2 , x3 ,  , xn )  0

( k 1) (k ) f ( x (k ) )
Newton Raphson equation: x x 
f ( x ( k ) )

 f1 f1 f1 

Newton Raphson equation in matrix form  x 
x2 xn 
 1 
( k 1) (k ) F (k )  f 2 f 2

f 2 
X X  J   x1 x2 xn 
J (k )      
Jacobian
 f f n 
matrix
J ( k ) X ( k 1)  J ( k ) X ( k )  F ( k )  n
f n
 
 x1 x2 xn 
7
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 General solution of Newton Raphson method

(k ) (k )
 f1 f1 f1  ( k 1)  f1 f1 f1  (k ) (k )
 x   x1    x1   f1 ( x1 , x2 , x3 , , xn ) 
x2 xn   
 x x2 xn     
 1   1 
 f 2 f 2 f 2     f 2 f 2 f 2     
  x2    x2   f 2 ( x1 , x2 , x3 , , xn ) 
 x1 x2 xn     x1 x2 xn    
  
               
 f f n f n    f f n f n     
 n    xn   n    xn   f n ( x1 , x2 , x3 , , xn )
 x1 x2 xn   x1 x2 xn 

8
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Solve this set of linear equations at each iteration:

J (k ) X (k 1)  J (k ) X (k )  F (k )
J ( k ) { X ( k 1)  X ( k ) }   F ( k )
Rearrange:
J ( k ) X ( k )   F ( k )
(k )
 f1 f1 f1 
 x1 
(k )
 f1 ( x1 , x2 , x3 ,, xn ) 
(k )
 x 
x2 xn     
 1     
 f 2 f 2

f 2 
  x2   f ( x , x , x ,  , xn ) 
 x1 x2 xn       2 1 2 3 

         
 f n f n f n       
   xn   f n ( x1 , x2 , x3 ,, xn ) 
 x1 x2 xn 
( k 1) (k ) (k )
X X  X 9
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Example 11.7:
Use Newton-Raphson method to determine the roots of

2 x12  x22  4.32

x12  x22  0
with initial guesses of x1(0) = x2(0) =1. Compute two iterations.

iter.no x1 x2
0 1 1
1 1.2200 1.2200
2 1.2002 1.2002

10
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Solution:  f1 f1 
 x x2  4 x1 2 x2 
f1 ( x1, x2 )  2 x12  x22  4.32  0 J  1 
 f 2 f 2  2 x1  2 x2 
f 2 ( x1, x2 )  x12  x22  0  x1 x2 

1st iteration
x1( 0)  x2( 0)  1
4 x1( 0) 2 x2( 0)  x1( 0)   f1 ( x1( 0) , x2( 0) ) 
 (0) (0)   (0) 
  (0) 
2 x1  2 x2  x2  ( 0)
 f 2 ( x1 , x2 )
4 2  x1(0)  1.32 

 2  2  ( 0 )   0  , x1
( 0)
 0.22 , x 2  0.22
(0)

  x2   
x1(1)  x1(0)  x1( 0)  1.22
x2(1)  x2(0)  x2( 0)  1.22
11
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 2nd iteration
x1(1)  x2(1)  1.22
4 x1(1) 2 x2(1)  x1(1)   f1 ( x1(1) , x2(1) ) 
 (1) (1)   (1) 
  (1) 
2 x1  2 x2  x2  (1)
 f 2 ( x1 , x2 )

 4.88 2.44  x1(1)   0.1452 

2.44  2.44      0  , x1
(1)
  x 2  0.0198
(1)

 x2  
(1)
 
x1( 2)  x1(1)  x1(1)  1.2002
x2( 2)  x2(1)  x2(1)  1.2002

x1true  x2true  1.2

12
Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Example 11.8:
Use Newton-Raphson method to determine the roots of

x1  x1 x2  10
2

x2  3 x1 x22  57
with initial guesses of x1(0) =1.5, x2(0) =3.5.
Iterate until εa ≤ 0.001 . Use maximum magnitude norm to
calculate εa .

iter. no. x1 x2 εa
0 1.5 3.5
1 2.036 2.8439
2 1.9987 3.0023 0.0528
3 2.0000 2.999999≈3 0.0008
13

Copyright © 2006 The McGraw-Hill Companies, Inc. Permission required for reproduction or display.