Numerical Methods

(CENG-2084)
Chapter 4
Solutions of Systems of Nonlinear
Equations

3/23/2020 1
 Systems
Systemsof
of Nonlinear Equations
Nonlinear Equations
•Locate the roots of a set of simultaneous nonlinear equations:

f1 ( x1 , x2 , x3 ,, xn )  0
f 2 ( x1 , x2 , x3 , , xn )  0

f n ( x1 , x2 , x3 ,, xn )  0
Example:
x12  x1 x2  10  f1 ( x1 , x2 )  x12  x1 x2  10  0
x2  3x1 x2 2  57  f 2 ( x1 , x2 )  x2  3x1 x2 2  57  0
3/23/2020 2
• First-order Taylor series expansion of a function with more than one variable:
f1(i ) f1( i )
f1(i 1)  f1( i )  ( x1( i 1)  x1( i ) )  ( x2( i 1)  x2( i ) )  0
x1 x2
f 2(i ) f 2(i )
f 2(i 1)  f 2(i )  ( x1( i 1)  x1( i ) )  ( x2( i 1)  x2( i ) )  0
x1 x2
• The root of the equation occurs at the value of x1 and x2 where f1(i+1) =0 and f2(i+1) =0
Rearrange to solve for x1(i+1) and x2(i+1)

f1(i ) f1(i ) f1(i ) f1(i )

x1(i 1)  x2(i 1)   f1(i )  xi  x2 ( i )
x1 x2 x1 x2
f 2(i ) f 2(i ) f 2(i ) f 2(i )
x1(i 1)  x2(i 1)   f 2(i )  xi  x2 ( i )
x1 x2 x1 x2

 f1(i ) f1(i )   f1(i ) f1( i ) 

   f1(i )   x1 
 x1 x2   x1( i 1)  x2   x1(i ) 
f 2 (i )   x2 (i 1) 
     f f 2(i )   x2( i ) 
 f 2( i )  2(i )   2(i )
f
 x x2   x x2 
 1  1
• Since x1(i), x2(i), f1(i), and f2(i) are all known at the ith iteration, this represents a set of
two linear equations with two unknowns, x1(i+1) and x2(i+1)
• You may use several techniques to solve these equations 3
 General Representation for the solution of
Nonlinear Systems of Equations
 f1 f1 f1   Jacobian Matrix
f1 ( x1 , x2 , x3 ,, xn )  0  x 
x2 xn 
 1 
f 2 ( x1 , x2 , x3 , , xn )  0  f 2 f 2

f 2 
Solve this set of linear
J   x1 x2 xn 
      
equations at each iteration:
 f f n f n 
 n  
f n ( x1 , x2 , x3 ,, xn )  0  x1 x2 xn  [ J i ]{ X i 1}  {Fi }  [ J i ]{ X i }

 f1 f1 f1   f1 f1 f1 

 x   x   f ( x , x , x ,  , x )    x1 
xn     x 
1 1 1 2 3 n
x2   x2 xn  
 1   1 
 f 2 f 2 f 2      f f 2 f 2   
  x2   f 2 ( x1 , x2 , x3 ,, xn )   2   x2 
 x1 x2 xn           x1 x2 xn    
               
 f f n f n         f f n f n    
 n    xn   f n ( x1 , x2 , x3 , , xn )  n    xn 
 x1 x2 xn  i i 1 i  x1 x2 xn  i i

{X}(i+1) values are found by solving this system of linear equations.

These iterations will continue until the convergence is reached;
which will happen when the approximate relative error (defined on the vector of values) falls below the desired error tolerance εtol 4
3/23/2020
 Solution of Nonlinear Systems of Equations

Solve this set of linear equations at each iteration: [ J i ]{ X i 1}  {Fi }  [ J i ]{ X i }

Rearrange: [ J i ]{ X i 1  X i }  {Fi }
[ J i ]{X i 1}  {Fi }

 f1 f1 f1 

 x    x1   f1 ( x1 , x2 , x3 ,  , xn ) 
x2 xn    
 1 
 f 2 f 2 f 2     
 x2   f 2 ( x1 , x2 , x3 , , xn ) 
 x1 x2 xn        
  
      
 f f n f n       
 n   xn   f n ( x1 , x2 , x3 ,  , xn )
 x1 x2 xn  i i 1 ii

3/23/2020 5
 Notes on the solution of
Nonlinear Systems of Equations
n non-linear equations f1 ( x1 , x2 , x3 ,, xn )  0  f1
 x
f1

f1 
n unknowns to be solved: x2 xn 
 1 
f 2 ( x1 , x2 , x3 , , xn )  0  f 2 f 2

f 2 
Jacobian J   x1 x2 xn 
      
Matrix:  f f n f n 
f n ( x1 , x2 , x3 ,, xn )  0  n
 x1 x2
 
xn 

• Need to solve this set of Linear Equations

[ J i ]{X i 1}  {Fi }

in each and every iterative solution of the original Non-Linear system of equations.

• For n unknowns, the size of the Jacobian matrix is n2 and the time it takes to solve
the linear system of equations (one iteration of the non-linear system) is
proportional to O(n3)
• There is no guarantee for the convergence of the non-linear system. There may also
be slow convergence.
• In summary, finding the solution set for the Non-Linear system of equations is an
extremely compute-intensive task.
3/23/2020 6
 Iterative method
Fixed -Point iteration for Functions of Several Variables
Procedures used for solving non- linear problem by iterative method
• the 𝑖 𝑡ℎ equation is solved for 𝑥𝑖, the system is changed into the fixed-point
problem
• Using the initial guess 𝑥2 , 𝑥3 … solve for 𝑥1
• Using the values of 𝑥1 from step ii and 𝑥3 , 𝑥4, … solve for 𝑥2
• Using the value of x1 from step ii and that of x2 from step iii solve for x3
• Repeat this procedure until the required accuracy is achieved. /interval is
smaller than some specified tolerance/
Example : solve the system of nonlinear equation by iterative method
1
3𝑥1 − cos 𝑥2 𝑥3 − = 0
2
2 2
𝑥1 − 81 𝑥2 + 0.1 + 𝑠𝑖𝑛𝑥3 + 1.06 = 0
−𝑥1 𝑥2
10𝜋 − 3
𝑒 + 20𝑥3 + =0
3

3/23/2020 7
 Solution
If the 𝑖𝑡ℎ equation is solved for 𝑥𝑖 ,
the system is changed into the
fixed-point problem
1 1
𝑥1 = cos 𝑥2 𝑥3 +
3 6
1
𝑥2 = 𝑥1 2 + 𝑠𝑖𝑛𝑥3 + 1.06
9
− 0.1
1 −𝑥 𝑥 10𝜋 − 3
𝑥3 = − 𝑒 1 2 −
20 60

3/23/2020 8
 Solution
If the 𝑖𝑡ℎ equation is solved for 𝑥𝑖 , To approximate the fixed point𝑡p, we
choose 𝑥 0 = (0.1,0.1, −0.1) . The
the system is changed into the
sequence of vectors generated by
fixed-point problem (𝑘)
1 (𝑘−1) (𝑘−1)
1
1 1 𝑥1 = cos 𝑥2 𝑥3 + ,
𝑥1 = cos 𝑥2 𝑥3 + 3 6
3 6 1
1 𝑥2 (𝑘) = (𝑥1 (𝑘−1) )2 + sin 𝑥3 (𝑘−1) + 1.06
𝑥2 = 𝑥1 2 + 𝑠𝑖𝑛𝑥3 + 1.06 − 0.1 9
9 − 0.1,
1 −𝑥 𝑥 10𝜋 − 3 1 −𝑥 (𝑘−1) 𝑥 (𝑘−1) 10𝜋 − 3
𝑥3 = − 𝑒 1 2 − 𝑥3 (𝑘) =− 𝑒 1 2 −
20 60 20 60

3/23/2020 9
 Solution
If the 𝑖𝑡ℎ equation is solved for 𝑥𝑖 , To approximate the fixed point𝑡p, we
choose 𝑥 0 = (0.1,0.1, −0.1) . The
the system is changed into the
sequence of vectors generated by
fixed-point problem (𝑘)
1 (𝑘−1) (𝑘−1)
1
1 1 𝑥1 = cos 𝑥2 𝑥3 + ,
𝑥1 = cos 𝑥2 𝑥3 + 3 6
3 6 1
1 𝑥2 (𝑘) = (𝑥1 (𝑘−1) )2 + sin 𝑥3 (𝑘−1) + 1.06
𝑥2 = 𝑥1 2 + 𝑠𝑖𝑛𝑥3 + 1.06 − 0.1 9
9 − 0.1,
1 −𝑥 𝑥 10𝜋 − 3 1 −𝑥 (𝑘−1) 𝑥 (𝑘−1) 10𝜋 − 3
𝑥3 = − 𝑒 1 2 − 𝑥3 (𝑘) =− 𝑒 1 2 −
20 60 20 60

𝒌 𝑿𝟏 (𝒌) 𝑿𝟐 (𝒌) 𝑿𝟑 (𝒌) /𝑿𝟏 (𝒌) − 𝑿𝟏 (𝒌−𝟏) / /𝑿𝟐 (𝒌) − 𝑿𝟐 (𝒌−𝟏) / /𝑿𝟑 (𝒌) − 𝑿𝟑 (𝒌−𝟏) /
0 0.1 0.1 -0.1

1 0.49998333 0.00944115 -0.52310127 0.399983333 0.09055885 0.4231

2 0.49999593 2.5568E-05 -0.52336331 1.26014E-05 0.009415582 0.00026

3 0.50000000 1.2337E-05 -0.52359814 4.06505E-06 1.3231E-05 0.00023

4 0.50000000 3.4168E-08 -0.52359847 2.28887E-11 1.23026E-05 3.3E-07

3/23/2020 10
5 0.50000000 1.6487E-08 -0.52359877 6.9541E-12 1.76809E-08 3.1E-07
One way to accelerate convergence of the fixed-point iteration is to use the latest estimates x1
(k), … , xi-1 (k) instead of x1(k-1), … , xi-1(k-1) to compute xi(k), as in the Gauss-Seidel
method for linear systems.
The component equations then become
(𝑘)
1 (𝑘−1) (𝑘−1)
1
𝑥1 = 𝑐𝑜𝑠 𝑥2 𝑥3 +
3 6
(𝑘)
1
𝑥2 = (𝑥1 (𝑘−1) )2 + 𝑠𝑖𝑛 𝑥3 (𝑘−1) + 1.06 − 0.1
9
1 −𝑥 (𝑘−1)𝑥 (𝑘−1) 10𝜋 − 3
𝑥3 (𝑘) = − 𝑒 1 2 −
20 60
So the convergence was indeed accelerated for this problem by using the Gauss-Seidel
method. However, this method does not always accelerate the convergence.

𝒌 𝑿𝟏 (𝒌) 𝑿𝟐 (𝒌) 𝑿𝟑 (𝒌) /𝐗 (𝒌) −𝐗 (𝒌−𝟏) /

0 0.1000000 0.1000000 -0.1000000
1 0.49998333 0.02222979 -0.52304613 0.423046
2 0.49997747 2.8154E-05 -0.52359807 0.022202
3 0.50000000 3.7622E-08 -0.52359877 2.81E-05
4
3/23/2020 0.50000000 5.028E-11 -0.52359878 3.76E-08 11
 Example:- Solve the nonlinear system by Newton’s method
1
3x1 − cos x2 x3 − =0
2
x1 2 − 81 x2 + 0.1 2 + sinx3 + 1.06 = 0
10π − 3
e−x1 x2 + 20x3 + =0
3
Newton’s method will be used to obtain this approximation using the initial
approximation is 𝑥 (0) = 0.1,0.1, −0.1 𝑎𝑛𝑑
𝑭(𝒙𝟏 , 𝒙𝟐 , 𝒙𝟑 ) = (𝑓1 (𝑥1 , 𝑥2 , 𝑥3 ), 𝑓2 (𝑥1 , 𝑥2 , 𝑥3 ), 𝑓3 (𝑥1 , 𝑥2 , 𝑥3 ))𝑡 ,
Where
1
𝑓1 𝑥1 , 𝑥2 , 𝑥3 = 3x1 − cos x2 x3 −
2
2 2
𝑓2 𝑥1 , 𝑥2 , 𝑥3 = x1 − 81 x2 + 0.1 + sinx3 + 1.06
−x1 x2
10π − 3
𝑓3 𝑥1 , 𝑥2 , 𝑥3 = e + 20x3 +
3
And
The Jacobian matrix 𝑱(𝒙) for this system
3 𝑥3 sin𝑥2 𝑥3 𝑥2 sin𝑥2 𝑥3
𝐽 𝑥1 , 𝑥2 , 𝑥3 = 2𝑥1 −162(𝑥2 + 0.1) cos𝑥3
−𝑥2 e−x1x2 −𝑥1 e−x1x2 20

3/23/2020 12
 Thus, at the 𝑘𝑡ℎ step, the linear system 𝑱(𝐱 𝐤−𝟏 )𝐲 𝐤−𝟏 = −𝐅(𝐱 𝐤−𝟏 )
𝒚𝟏 𝒌−𝟏
𝒌−𝟏 = −(𝑱(𝒙𝟏 𝒌−𝟏 , 𝒙𝟐 𝒌−𝟏 , 𝒙𝟑 𝒌−𝟏 ))−𝟏 𝑭(𝒙𝟏 𝒌−𝟏 , 𝒙𝟐 𝒌−𝟏 , 𝒙𝟑 𝒌−𝟏 )
𝒚𝟐
𝒚𝟑 𝒌−𝟏
1
𝑥3 sin𝑥2 𝑥3 𝑦1 3x1 − cos x2 x3 −
3 𝑥2 sin𝑥2 𝑥3 2
2𝑥1 −162(𝑥2 + 0.1) cos𝑥3 𝑦2 2 2
= − x1 − 81 x2 + 0.1 + sinx3 + 1.06
−x1 x2 𝑦3
−𝑥2 e −𝑥1 e−x1x2 20 −x1 x2
10π − 3
e + 20x3 +
3
Taking 𝑥1 (0) = 0.1, 𝑥2 (0) = 0.1 𝑎𝑛𝑑 𝑥3 (0) = −0.1 as initial approximation, the above
equation reduces to
3 9.99𝑥10−4 −9.99𝑥10−4 𝑦1 1.9995
0.2 −32.4 0.995 ∗ 𝑦2 = 2.2698
−0.099 −0.099 20 𝑦3 −8.46
Solving this linear system of equation, results in
𝑦1 = 0.39987, 𝑦2 = − 0.080533 𝑎𝑛𝑑 𝑦3 = −0.42121.

3/23/2020 13
Use this results, the next approximation for the unknown variables can be calculated as
𝒙𝟏 𝟏 𝒙𝟏 𝟎 𝒚𝟏 𝟎
𝒙𝟐 𝟏 = 𝒙𝟐 𝟎 + 𝒚𝟐 (𝟎)
𝟏 𝟎 𝒚𝟑 𝟎
𝒙𝟑 𝒙𝟑

𝒙𝟏 𝟏 = 𝒙𝟏 𝟎 + 𝒚𝟏 𝟎 = 𝟎. 𝟏 + 𝟎. 𝟑𝟗𝟗𝟖𝟕 = 𝟎. 𝟒𝟗𝟗𝟖𝟕
𝒙𝟐 𝟏 = 𝒙𝟐 𝟎 + 𝒚𝟐 𝟎 = 𝟎. 𝟏 − 𝟎. 𝟎𝟖𝟎𝟓𝟑𝟑𝟏 = 𝟎. 𝟎𝟏𝟗𝟒𝟔𝟔
𝒙𝟑 𝟏 = 𝒙𝟑 𝟎 + 𝒚𝟑 𝟎 = −𝟎. 𝟏 − 𝟎. 𝟒𝟐𝟏𝟐𝟏 = −𝟎. 𝟓𝟐𝟏𝟓𝟐𝟎

𝒌 𝑿𝟏 (𝒌) 𝑿𝟐 (𝒌) 𝑿𝟑 (𝒌) /𝐗 (𝒌) −𝐗 (𝒌−𝟏) /

0 0.1000000 0.10000000 -0.10000000
1 0.4998700 0.02222979 -0.52304613 0.423046
2 0.50004593 2.8154E-05 -0.52359807 0.022202
3 0.50000034 3.7622E-08 -0.52359877 1.54x10-5
4 0.50000000 0.00000000 -0.52359878 1.24-08
5 0.50000000 0.00000000 -0.52359878 0.0000000

The example illustrates that Newton's method can converge very rapidly once an
approximation is obtained that is near the true solution.
3/23/2020 14
 Thank You

3/23/2020 15