MA 572 Numerical Analysis

Lecture 4
(August 6, 2021)

MA572 1 / 15
Rootfinding for nonlinear equations

To illustrate the idea of an iterative method for solving equation f (x) = 0, let us consider the
following equation:
1
f (x) ≡ a − = 0, (1)
x
for a given a > 0.

Let x = 1/a be an approximate solution of the equation. In other words, we want to find a root α
very near to 1/a. At the point (x0 , f (x0 )), draw the tangent line to the graph of y = f (x), i.e.,
we assume x0 to be an initial guess. Let x1 be the point at which the tangent line intersects the
x-axis.

It should now be an improved approximation of the root α.

MA572 2 / 15
Rootfinding for nonlinear equations

MA572 3 / 15
Rootfinding for nonlinear equations

To obtain an equation for x1 , from the slopes of the tangent line, we get

f (x0 ) − 0
f 0 (x0 ) = .
x0 − x1

Now from (1), we have f 0 (x0 ) = 1/x02 .

Therefore

1 a − x1
0
=
x02 x0 − x1
⇒ x0 − x1 = ax02 − x0
⇒ x1 = x0 (2 − ax0 ).

Now the next approximation x2 can be obtained by a similar procedure as

x2 = x1 (2 − ax1 ).

MA572 4 / 15
Rootfinding for nonlinear equations

Continuing in this way, the general iteration formula can be obtained as

xn+1 = xn (2 − axn ), n ≥ 0. (2)

A more appropriate form for theoretical purposes can be written by introducing the scaled residual

rn = 1 − axn . (3)

Consequently, (2) becomes

xn+1 = xn (1 + rn ), n ≥ 0. (4)

For the error

1 rn
en = − xn = . (5)
a a

Having used an iterative method, now the task is to analyze the convergence of the method, its
speed and its dependence on x0 .

MA572 5 / 15
Rootfinding for nonlinear equations

We have

rn+1 = 1 − axn+1 = 1 − axn (1 + rn )

= 1 − (1 − rn )(1 + rn ).

This implies
rn+1 = rn2 . (6)

Further,
n
rn = r02 . (7)

From (5), the error en converges to zero as n → ∞ if and only if rn converges to zero. From (7),
rn converges to zero if and only if |r0 | < 1 or equivalently −1 < 1 − ax0 < 1, i.e.,

2
0 < x0 < . (8)
a

MA572 6 / 15
Rootfinding for nonlinear equations

In order that xn converges to 1/a, it is necessary that x0 be chosen to satisfy (8).

To examine the speed of convergence when (8) is satisfied, we obtain formulas for the error and
the relative error. For the speed of convergence when (8) is satisfied, we have

rn+1 r2 e 2 a2
en+1 = = n = n
a a a
⇒ en+1 = aen2 . (9)

Further

en 2
 
en+1
= en2 a2 =
1/a 1/a
⇒ Rel(xn+1 ) = [Rel(xn )]2 , n ≥ 0. (10)

MA572 7 / 15
Rootfinding for nonlinear equations

Rel(xn ) denotes the relative error in xn . Based on equation (9), we can say that en converges to
zero quadratically. To illustrate how rapidly the error decreases, let us take Rel(x0 ) = 0.1. Then,

Rel(x1 ) = (0.1)2 = 10−2 , Rel(x2 ) = 10−4 , Rel(x3 ) = 10−8 , Rel(x4 ) = 10−16 .

This shows that each iteration doubles the number of significant digits.

Definition
A sequence of iterates {xn |n ≥ 0} is said to converge with order p ≥ 1 to a point α if

|α − xn+1 | ≤ c|α − xn |p , n ≥ 0 (11)

for some c > 0. If p = 1, the sequence is said to converge linearly to α. In that case, we require
c < 1. The constant c is called the rate of linear convergence of xn to α.

MA572 8 / 15
Rootfinding for nonlinear equations

By this definition, the example considered has order of convergence 2, which is also called
quadratic convergence.

This definition of order is not always convenient for some linearly convergent iterative methods.
By induction on (11) with p = 1, we get

|α − xn | ≤ c n |α − x0 |, n ≥ 0 (12)

This shows directly the convergence of xn to α. For some iterative methods, we can show (12)
directly whereas (11) may not be true for any c < 1. In such a case, the method will still be said
to converge linearly with a rate of c.

MA572 9 / 15
Rootfinding of nonlinear equation

Recall
Intermediate Value Theorem: Let f (x) be continuous on the finite interval a ≤ x ≤ b, and let ζ
be any real number that lies between f (a) and f (b). Then there exists a real number ξ with
a < ξ < b such that
f (ξ) = ζ.

One question that comes to our mind is that what does this have to do with rootfinding problem?
The answer is: Basically, the Intermediate Value Theorem provides a means for identifying
intervals which enclose the real zeros of continuous functions.

In this case
All that is required is to find an interval such that the values of the function at the endpoints are
of opposite signs. It is to be noted that the magnitudes of these endpoints do not matter.

As long as one endpoint is positive and the other is negative, zero is somewhere between the
values and at least one zero of the function is guaranteed to exist on the interval.

MA572 10 / 15
Rootfinding of nonlinear equation

Let us visualize this situation. Consider the function f (x) = x 3 + 2x 2 − 3x − 1. Let us have the
following values:

f (−3) = −1, f (−2) = 5,

f (−1) = 3, f (0) = −1,
f (1) = −1, f (2) = 9.

From above, we observe that each change in the sign of the function value indicates an interval
that contains a real zero. Therefore, this function clearly has three simple real zeros, one each in
the intervals: (−3, −2), (−1, 0) and (1, 2).

Mathematically we can state

Assume that f (x) is continuous on a given interval [a, b] and that it also satisfies

f (a)f (b) < 0. (13)

Since the function f (x) changes sign on [a, b], f (x) must have at least one root in [a, b].

MA572 11 / 15
Bisection Method

By Intermediate Value Theorem, we have located an interval that contains a zero of a continuous
function. Next objective is to systematically shrink the size of that root enclosing interval.

The simplest and most natural way to accomplish a reduction in interval size is to cut the interval
in half. Once this is done, we next determine which half contains a root by again using the
Intermediate Value Theorem. Then we repeat the process on that half and go on doing that.

This method is known as bisection method which insists on bisecting an interval at every step.

Bisection method is one of the most familiar methods used for rootfinding. It has both
advantages and disadvantages as compared to other methods of rootfinding.

MA572 12 / 15
Bisection Method

As understood from the basic idea of bisection method, it is clear that the method generates a
sequence of root enclosing intervals.

For notational convenience, let (an , bn ) be the enclosing interval during the n-th iteration of the
method.

Let αn denote the midpoint of the interval [an , bn ]:

an + bn
αn = .
2

αn can be used not only as one of the next enclosing interval but also as an approximation to the
location of the exact root α.

If αn is an accurate enough approximation, the iteration is terminated. Otherwise Intermediate

Value Theorem is again used to determine which of the intervals, (an , αn ), or (αn , bn ) contains
the root and becomes (an+1 , bn+1 ). The entire process is then repeated on that subinterval.

MA572 13 / 15
Bisection Method

Usually [a, b] is chosen to contain only one root α but the following algorithm for the bisection
method will always converge to some root α in [a, b] because of (12).

As mentioned, the simplest numerical method for finding a root is to repeatedly halve the interval
[a, b], keeping the half on which f (x) changes sign.

Algorithm:
Bisect (f , a, b, root, ).
1 Define c := (a + b)/2.
2 If b − c ≤ , then accept root := c and exit.
3 If sign(f (a)) · sign(f (b)) ≤ 0, then a := c, otherwise b := c.
4 Return to step 1.

MA572 14 / 15
Bisection Method

The major advantages of bisection method are

1 It is guaranteed to converge provided f is continuous on [a, b] and (12) is satisfied.
2 A reasonable error bound is available. Methods that at every step give upper and lower
bounds on the root α are called enclosure methods.

MA572 15 / 15