Solving Equations by Iterative Methods

A Case Study on the Root-Finding Problem: Kepler’s Law of Planetary Motion

The root-finding problem is one of the most important computational problems,

arising in a wide variety of practical applications in physics, chemistry, biosciences,
engineering, etc.

Determination of any unknown appearing implicitly in scientific, or engineering

formulas gives rise to a root-finding problem.
A simple application

According to Kepler, a planet revolves around the sun in an elliptic orbit:

The position (x, y) of the planet at time t can be determined by:
The eccentric anomaly can be computed from Kepler’s equation of motion:

where M is the mean anomaly.

Thus, to find E we solve the nonlinear equation:

The solution of such an equation is the subject of this lesson.

The Root-Finding Problem is the problem of finding a root of the equation , where is a
function of a single variable . Specifically, the problem is stated as:

The number such that is called a root of the equation or a zero of the function .

The function can be algebraic or trigonometric or a combination of both.

Analytical versus Numerical Methods

Except for some very special functions, it is not possible to find an analytical expression for
the root, from where the solution can be exactly determined. This is true for even
commonly arising polynomial functions.

A polynomial of degree has the form:

The Fundamental Theorem of Algebra states that a polynomial of degree has at least
one zero.
We learn early in school how to find the root of a quadratic equation: using the analytical
formula:

Unfortunately, such analytical formulas do not exist for polynomials of degree 5 or

greater. This fact was proved by Abel and Galois in the 19th century.

Thus, most computational methods for the root-finding problem are iterative in nature.
Solution of Equations by Iteration
Find solutions of a single equation

A solution of (1) is a number x = s such that f(s) = 0.

To solve (1) when there is no formula for the exact solution

available, we can use an approximation method, such as an iteration
method.
In the iteration method, we start from an initial guess (which may be
poor) and compute step by step (in general better and better)
approximations of an unknown solution of (1).

The procedure stops when two successive values of are equal – to

the required degree of accuracy.
Understanding iteration methods and their underlying ideas will
allow us to select judiciously the appropriate software from among
different software packages that employ variations of such methods
and not just treat the software programs as “black boxes.”

In general, iteration methods are easy to program because the

computational operations are the same in each step—just the data
change from step to step—and, more importantly, if in a concrete
case a method converges, it is stable in general.
A solution of (2) is called a fixed point of g.

This is a solution of (1), since from we can return to the original form

From (1) we may get several different forms of (2). The behavior of
corresponding iterative sequences may differ, in particular, with respect to
their speed of convergence – some of them may not converge at all.
Method of Successive Substitution (MOSS) or Fixed-Point Iteration

The method of successive substitution is a numerical method for solving a nonlinear

equation for the unknown.

Steps:
• by some algebraic step, rewrite a nonlinear function into the form
𝑥= 𝑓 (𝑥 )
• choose an initial guess, and evaluate to yield .
• continue the iteration:
𝑥𝑘+1 = 𝑓 ( 𝑥 𝑘 ) 𝑘=0,1 , 2 , 3 , …

until the result no longer changes to within a specified tolerance, i.e. after iterations
where:
|𝑥 𝑚+1 − 𝑥 𝑚|≤ 𝜖
Example

Set up an iteration process for the equation:

2
𝑥 − 3 𝑥 +1=0
and solve for accurate to 3 decimal places (3D).
 𝑓 ( 𝑥 )=𝑥 2 − 3 𝑥+1
𝑓 ( 𝑥 )= 0 2
0=𝑥 −3 𝑥 +1

The equation may be written:

1
𝑥=𝑔 (𝑥 ) 𝑥= (𝑥 ¿¿ 2+1)¿
3
𝑥5 =0.384 𝑥6 =0.382 𝑥7 =0.382

Since to 3 decimal places, then .

If we choose ,

which diverge s and won’t lead to a root.

Our equation may also be written (divide by ):

which seems to approach the larger solution.

If we choose , we obtain the sequence:

𝑥5 =2.618

Since to 3 decimal places, then .

We conclude that convergence
depend on the fact that, in a
neighborhood of a solution, the
curve of g(x) is less steep than the
straight line y = x and this
condition is sufficient for
convergence.
Example 2
3 − 2𝑥
Problem Use MOSS to find that solves the equation 𝑥 +2 𝑥+2 −10 𝑒 =0

Solution
2
3 − 2𝑥
𝑥 +2 𝑥+2 −10 𝑒 =0 [ 𝑔 ( 𝑥 ) =0 ]
We may choose to rewrite the equation as:

√ ( )
3
1 𝑥 +2 𝑥 +2 [𝑥 = 𝑓 ( 𝑥 )]
𝑥= − ln
2 10
and for the purpose of iteration:

√ ( )
3
1 𝑥 𝑘+ 2 𝑥 𝑘+ 2 [ 𝑥𝑘 +1= 𝑓 ( 𝑥 𝑘 ) ]
𝑥𝑘+1 = − ln
2 10
 √ ( )
3
𝑘=0: 1 𝑥 + 2 𝑥 0 +2
0
𝑥1= − ln
2 10

Initial guess: Let

√ ( )
3
1 ( − 0.5 ) +2 ( −0.5 ) +2
𝑥1= − ln =1.1037
2 10

𝑘=1 :

√ 1
𝑥2 = − ln
2 ( 10 )
( 1.1037 )3 +2 ( 1.1037 ) +2
=0.54242
𝑘=2 :

√ ( )
3
1 ( 0.54242 ) +2 ( 0.54242 ) +2
𝑥3 = − ln =0.75022
2 10

𝑘=3 :

√ ( )
3
1 ( 0.75022 ) +2 ( 0.75022 ) +2
𝑥 4= − ln =0.68404
2 10

𝑘=4 :

√ ( )
3
1 ( 0.68404 ) +2 ( 0.68404 ) +2
𝑥5 = − ln =0.70621
2 10
 √ ( )
3
1 ( 0.70621 ) +2 ( 0.70621 ) +2
𝑥6 = − ln =0.69890
2 10

√ ( )
3
1 ( 0.69890 ) +2 ( 0.69890 )+2
𝑥7 = − ln =0.70133
2 10

√ ( )
3
1 ( 0.70133 ) +2 ( 0.70133 )+2
𝑥8 = − ln =0.70052
2 10
 √ ( )
3
1 ( 0.70621 ) +2 ( 0.70621 ) +2
𝑥6 = − ln =0.69890
2 10

√ ( )
3
1 ( 0.69890 ) +2 ( 0.69890 )+2
𝑥7 = − ln =0.70133
2 10

√ ( )
3
1 ( 0.70133 ) +2 ( 0.70133 )+2
𝑥8 = − ln =0.70052
2 10
 √ ( )
3
1 ( 0.70052 ) +2 ( 0.70052 ) +2
𝑥 9= − ln =0.70079
2 10

√ ( )
3
1 ( 0.70079 ) + 2 ( 0.70079 ) +2
𝑥10 = − ln =0.70070
2 10

√ ( )
3
1 ( 0.70070 ) + 2 ( 0.70070 ) +2
𝑥11 = − ln =0.70073
2 10
 √ ( )
3
1 ( 0.70073 ) + 2 ( 0.70073 ) +2
𝑥12= − ln =0.70072
2 10

√ ( )
3
1 ( 0.70072 ) + 2 ( 0.70072 )+ 2
𝑥13 = − ln =0.70072
2 10

Since the current estimate is equal to the previous one, the iteration is stopped and the
answer reported as as the root of the given equation.

It takes 13 iterations to arrive at the correct answer: the method converges slowly.
The convergence is highly dependent on how ones defines .

For instance, in the previous problem, if we rearranged the equation:

to be:

we will find that the method will diverge, and we will not obtain a root.
One sufficient condition for convergence is that the slope of 𝑓(𝑥) is between 1 and -1