You are on page 1of 17

Roots of Equations

1.1 Introduction
The following problem may be used as an introduction to the problem of root finding. An electrical cable
is suspended from two towers that are 50 meters apart. The cable is allowed to dip 10 meters in the
middle. How long is the cable? We know that the curve assumed by a suspended cable is a catenary
(see Figure 0.0).
-20 -10 0 10 20
0
10
20
30
40
x
y
Figure 0.0. Cable suspended between two towers (left and right in the figure).
When the y-axis passes through the lowest point, we can assume an equation of the form
y = k cosh(xf k). Here k is a parameter to be determined. The conditions of the problem are that
y(25) = y(0) + 10. Hence
k cosh
25
k
= k + 10.
From this equation, k can be determined by the methods discussed in this chapter. The result is
k = 32.79. The question now is how can we find this value and what are the procedures to calculate it.
Another example for such kind of problems is the following missile-intercept problem. The movement of
an object in the x y plane is decried by the parametrized equations
x
1
(t) = t and y
1
(t) = 1 -
-t
.
A second object moves according to the equations
x
2
(t) = 1 - cos(a) t and y
2
(t) = sin(a) t - 0.1 t
2
.
Is it possible to choose a value for a so that both objects will be in the same place at some time?
2012 G. Baumann
When we set the x and y coordinates equal to each other, we get the system
t = 1 - cos(a) t and 1 -
-t
= sin(a) t - 0.1 t
2
that needs to be solved for the unknown a and t. If real values exist for these unknowns that satisfy the
two equations, both objectives will be in the same place at some value t. But even though the problem is
a rather simple one that yields a small system, there is no obvious way to get the answer, or even to see
if there is a solution. However, if we graphically represent the two curves we observe that there is an
intersection which means a solution (see Figure 0.0)
a
0.5 1.0 1.5 2.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Figure 0.0. Two objects are crossing on a common point.
The numerical solution of a system of nonlinear equations is one of the more challenging tasks in
numerical analysis and, as we will see, no completely satisfactory method exists for it. To understand the
difficulties, we start with what at first seems to be a rather easy problem, the solution of a single equation
in one variable
f(x) = 0.
The values of x that satisfy this equation are called the zeros or roots of the function f . In what is to follow
we will assume that f is continuous and sufficiently differentiable where needed.
1.2 Simple Root Finding Methods
To find the roots of a function of one variable is straightforward enoughjust plot the function and see
where it crosses the x-axis. The simplest methods are in fact little more than that and only carry out this
2 Lecture_006.nb
2012 G. Baumann
where it crosses the x-axis. The simplest methods are in fact little more than that and only carry out this
suggestion in a systematic and efficient way. Relying on the intuitive insight of the graph of the function f ,
we can discover many different and apparently viable methods for finding the roots of a function of one
variable.
Suppose we have two values a and b, such that f (a) and f (b) have opposite signs. Then, because it is
assumed that f is continuous, we know that there is a root somewhere in the interval [a, b]. To localize it,
we take there is a root somewhere in the interval and compute f (c). Depending on the sign of f (c), we
can then place the root in one of the two intervals [a, c] or [c, b]. We can repeat this procedure until the
region in which the root is known to be located is sufficiently small. The algorithm is known as the bisec-
tion method. This method is based on the intermediate-value theorem which is shown in Figure 0.0. In
this figure the graph of a function that is continuous on the closed interval [a, b] is shown. The figure
suggests that if we draw any horizontal line y = k, where k is between f (a) and f (b), then that line will
cross the curve y = f (x) at least once over the interval [a, b].
x
y
f(a)
f(b)
a b
k
x
Figure 0.0. Graph of a function with continuous behavior in the interval [a, b].
Stated in numerical terms, if f is continuous on [a, b], then the function f must take on every value k
between f (a) and f (b) at least once as x varies from a to b. For example, the polynomial
px_ := x
7
- x + 3
has a value of 3 at x = 1 and a value of 129 at x = 2. Thus it follows from the continuity of p that the
equation 3 - x + x
7
== k has at least one solution in the interval [1, 2] for every value of k between 3
and 129. This idea is stated more precisely in the following theorem.
Theorem 0.0. Intermediate-Value Theorem
If f is continuous on a closed interval [a, b] and k is any number between f (a) and f (b), inclusive, then
there is at least one number x in the interval [a, b] such that f (x) = k.
Although this theorem is intuitively obvious, its proof depends on a mathematically precise development
of the real number system, which is beyond the scope of this text.
A variety of problems can be reduced to solving an equation f (x) = 0 for its roots. Sometimes it is possi-
ble to solve for the roots exactly using algebra, but often this is not possible and one must settle for
decimal approximations of the roots. One procedure for approximating roots is based on the following
Lecture_006.nb 3
2012 G. Baumann
decimal approximations of the roots. One procedure for approximating roots is based on the following
consequences of the Intermediate-Value Theorem.
Theorem 0.0. Root Approximation
If f is continuous on [a, b], and if f (a) and f (b) are nonzero and have opposite signs, then there is at least
one solution of the equation f (x) = 0 in the interval (a, b).
This result, which is illustrated in Figure 0.0, can be proved as follows.
x
y
f(a)>0
f(x)=0
f(b)<0 a
b
0
Figure 0.0. Graph of a function f (x) allowing a root in the interval [a, b].
Proof: Since f (a) and f (b) have opposite signs, 0 is between f (a) and f (b). Thus by the Intermediate-
Value Theorem there is at least one number x in the interval [a, b] such that f (x) = 0. However, f (a) and
f (b) are nonzero, so that x must lie in the interval (a, b), which completes the proof.
Example 0.0. Root finding of a Polynomial
The equation
x
5
- x + 8 x
2
+ 1 = 0
can not be solved algebraically because the left hand side has no simple factors.
Solution 0.1. If we graph p(x) = x
5
- x + 8 x
2
+ 1 (Figure 0.0), then we are led to the conjecture that there
is one real root and that this root lies inside the interval [-2.1, 2]. The existence of a root is also con-
firmed by Theorem 0.0, since p(-2.1) = -2.46101 and p(2) = 63 have opposite signs. Approximate this
root to two decimal-place accuracy.
4 Lecture_006.nb
2012 G. Baumann
-2.20 -2.15 -2.10 -2.05 -2.00 -1.95 -1.90
-5
0
5
x
y
[-2.2, -1.9]
Figure 0.0. Graph of a function f (x) allowing a root in the interval [a, b].
The polynomial is defined by
px_ := x
5
+ 8 x
2
- x + 1
The following sequence of intervals shrinks the interval length in such a way that the conditions of Theo-
rem 0.0 are satisfied. The intervals are given in curled brackets in the second argument of Map[].
Map[{p[1]], p[2]]} &,
{{-2.1, 2}, {-2.1, -1.8}, {-2.06, -2.05}, {-2.059, -2.056}, {-2.0585, -2.058}}] //
TableForm[, TableHeadings - {{}, {"x", "p(x)"}}] &
x p(x)
-2.46101 63
-2.46101 9.82432
-0.0879704 0.464937
-0.031969 0.135084
-0.00402787 0.0238737
The table shows that the interval is chosen in such a way that the signs of the polynomial p(x) changes.
However, the exact value of the root can be determined by
FindRoot px 0, x, -3.1
x -2.05843
Lecture_006.nb 5
2012 G. Baumann
Stating that the real value of x = -2.0584 is the intersection of the polynomial p(x) with the horizontal x-
axis.
1.2.1 The Bisection Method
The bisection method is very simple and intuitive, but has all the major characteristics of other root-
finding methods. The simplest numerical procedure for finding a root is to repeatedly halve the interval
[a, b], keeping the half on which f (x) changes sign. This procedure is called the bisection method. It is
guaranteed to converge to a root.
The bisection method is very simple and uses the ideas introduced above that the product of the function
at two different locations distinguishes three cases. If we have positive values there is no change in sign
and thus no root, if we have a negative sign the two values are different in sign and we will have a root, if
the result is zero we found the root itself. In general the following steps are used:
Step1 : Choose the lower and upper boundary of an interval including the root. This means f (x
u
) f (x
l
) < 0.
Step2 : Estimate the root by the arithmetic mean.
Step3 : Make the following calculations to determine in which subinterval the root lies:
If f (x
l
) f (x
r
) < 0 the root lies in the lower interval. Therefore, set x
u
= x
r
and return to step 2.
If f (x
l
) f (x
r
) > 0 the root lies in the upper interval. Therefore, set x
l
= x
r
and return to step 2.
If f (x
l
) f (x
r
) = 0 the root equals x
r
; terminate the computation.
To be more precise in our definition, suppose that we are given an interval [a, b] satisfying f (a) f (b) < 0
and an error tolerance e > 0. Then the bisection method consists of the following steps:
Define c = (a + b) f 2.
If b - c e, then accept c as the root and stop.
If sign(f (b)) sign(f (c)) 0, then set a = c. Otherwise, set b = c. Return to step 1.
These algorithmic steps are implemented in the following lines
6 Lecture_006.nb
2012 G. Baumann
bisection[f_, a_, b_] := Block_]c, e = 10
-5
, ain = a, bin = b, m = 1, results = {}),
While_0 === 0,
(+ first step find the midpoint +)
c =
(ain + bin)
2
;
(+ second step select the root+)
If[(bin - c) < e, Return[results]];
AppendTo[results, {m, c}];
(+ third step select the interval +)
If[(f /. x -> bin) (f /. x - c) <= 0, ain = N[c], bin = N[c]];
m = m + 1
_
_
The application of the function to a polynomial shows us the following results
bisection]x
6
- x - 1, 1, 1.4 // TableForm[, TableHeadings - {{}, {"m", "c"}}] &
m c
1 1.2
2 1.1
3 1.15
4 1.125
5 1.1375
6 1.13125
7 1.13438
8 1.13594
9 1.13516
10 1.13477
11 1.13457
12 1.13467
13 1.13472
14 1.13474
15 1.13473
where m is the iteration step and c represents the approximation of the root at iteration step m. The
graphical representation of the function shows that there is in fact an intersection with the x axis.
Lecture_006.nb 7
2012 G. Baumann
Plot]x
6
- x - 1, {x, 1, 1.4}
1.0 1.1 1.2 1.3 1.4
-1
0
1
2
3
4
5
Figure 0.0. Graph of the function f (x) = x
6
- x - 1 allowing a root in the interval [1, 1.3].
In general, an iteration produces a sequence of approximate solutions; we will denote these iterates by
x
[0]
, x
[1]
, x
[2]
, .... These sequence of approximations is shown dynamically in the following Figure 0.0
1.0 1.1 1.2 1.3 1.4
-1
0
1
2
3
4
5
Figure 0.0. Sequence of approximations of the root for the function f (x) = x
6
- x - 1.
The difference between the various root-finding methods lies in what is computed at each step and how
the next iterate is chosen.
8 Lecture_006.nb
2012 G. Baumann
x
y
f(x
0
)
f(x
1
)
x
0
x
1
x
4
x
2
x
3
Figure 0.0. The bisection method. After three steps the root is known to lie in the interval [x
3
, x
4
].
To estimate the error bound of the bisection method we can proceed as follows. Let a
n
, b
n
and c
n
denote
the nth computed value of a, b, and c, respectively. Then easily we get
b
n+1
- a
n+1
=
1
2
(b
n
- a
n
) for n 1
and it is straightforward to deduce that
b
n
- a
n
=
1
2
n-1
(b - a) for n 1
where b - a denotes the length of the original interval with which we started. Since the root a is in either
the interval [a
n
, c
n
] or (c
n
, b
n
], we know that
a - c
n
c
n
- a
n
= b
n
- c
n
=
1
2
(b
n
- a
n
).
This is the error bound for c
n
that is used in the second step of the bisection algorithm. Combining it with
our estimation, we obtain the further bound
a - c
n

1
2
n
(b - a).
This shows that the iterates c
n
converges to a as n .
To see how many iterations will be necessary, suppose we want to have
a - c
n
e.
This will be satisfied if
1
2
n
(b - a) e.
Lecture_006.nb 9
2012 G. Baumann
Taking logarithms of both sides, we can solve this to give
n
log|
b-a
e
]
log(2)
.
For the example we discussed above the number of iterations for an accuracy of 10
-5
should be found
within
n
log|
1
0.00001
]
log(2)
= 16.6096.
Thus we need about n = 16 iterations which is in agreement with the calculation.
There are several advantages to the bisection method. The principal one is that the method is guaran-
teed to converge. In addition, the error bound, given is guaranteed to decrease by one-half with each
iteration. Many other numerical methods have variable rates of decrease for the error, and these may be
worse than the bisection method for some equations. The principal disadvantage of the bisection method
is that it generally converges more slowly than most other methods. For functions f (x) that have a continu-
ous derivative, other methods are usually faster. These methods may not always converge; when they do
converge, however, they are almost always much faster than the bisection method.
1.2.2 Method of False Position
Suppose we have two iterates x
0
and x
1
that encloses the root. We can then approximate f (x) by a
straight line in the interval and find the place where this line cuts the x-axis. We take this as the new
iterate
x
2
= x
1
-
(x
1
- x
0
) f (x
1
)
f (x
1
) -f (x
0
)
.
When this process is repeated, we have to decide which of the three points x
0
, x
1
, or x
2
, to select for
starting the next iteration. There are two plausible choices. The first, we retain the last iterate and one
point from the previous ones so that the two new points enclose the solution (Figure 0.0). This is the
method of false position.
10 Lecture_006.nb
2012 G. Baumann
x
y
f(x
0
)
f(x
1
)
x
0
x
1
x
2
x
3
Figure 0.0. The method of false position. After the second iteration, the root is known to lie in the interval
(x
3
, x
0
).
The formula for the false position algorithm is based on the similarity of the two triangles involved in the
iteration. Using the triangles generated by the straight line connecting the upper and lower value of the
function in the interval [x
n
, x
n-1
] we can write down the relation
f (x
n
)
x
n+1
- x
n
=
f (x
n-1
)
x
n+1
- x
n-1
This equation is equivalent to
(x
n+1
- x
n-1
) f (x
n
) = f (x
n-1
) (x
n+1
- x
n
)
which is written by collecting terms as
x
n+1
(f (x
n
) - f (x
n-1
)) = x
n-1
f (x
n
) - x
n
f (x
n-1
)
which is equivalent to
x
n+1
=
x
n-1
f (x
n
)
f (x
n
) - f (x
n-1
)
-
x
n
f (x
n-1
)
f (x
n
) - f (x
n-1
)
If we add and subtract on the right hand side x
n
we find
x
n+1
= x
n
+
x
n-1
f (x
n
)
f (x
n
) - f (x
n-1
)
- x
n
-
x
n
f (x
n-1
)
f (x
n
) - f (x
n-1
)
= x
n
+
x
n-1
f (x
n
)
f (x
n
) - f (x
n-1
)
+
-x
n
f (x
n
) + x
n
f (x
n-1
) - x
n
f (x
n-1
)
f (x
n
) - f (x
n-1
)
= x
n
+
x
n-1
f (x
n
)
f (x
n
) - f (x
n-1
)
+
-x
n
f (x
n
)
f (x
n
) - f (x
n-1
)
Lecture_006.nb 11
2012 G. Baumann
= x
n
+
(x
n-1
- x
n
) f (x
n
)
f (x
n
) - f (x
n-1
)
= x
n
-
(x
n
- x
n-1
) f (x
n
)
f (x
n
) - f (x
n-1
)
The successive iterates of the false position method are then simply computed by
x
n+1
= x
n
-
(x
n
- x
n-1
) f (x
n
)
f (x
n
) - f (x
n-1
)
.
We use this form because it involves one less function evaluation and one less multiplication than the
original relation (0.0) we started from.
The algorithm for the secant method consists of three steps:
Generate the approximated root by the derived iteration formula
Check if the error requirements are satisfied; if yes stop and return the value
If sign(f (a)) sign(f (c)) 0, then set a = c. Otherwise, set b = c. Return to step 1.
The following lines are an implementation of the secant method
falsePositionMethod[f_, a_, b_] :=
Block]]c, e = 10
-5
, ain = a, bin = b, cold = b, k = 0, results = {}),
While[0 === 0,
k = k + 1;
(+ first step find the approximation +)
c = bin - (f /. x -> bin) (bin - ain) / ((f /. x -> bin) - (f /. x -> ain));
(+ second step select the root and terminate +)
If[Abs[(cold - c)] < e, Return[results], cold = c];
AppendTo[results, {k, c}];
(+ third step select the interval +)
If[(f /. x -> ain) (f /. x -> c) <= 0, ain = N[c], bin = N[c]]
]

The application of the secant method shows the iteration steps


12 Lecture_006.nb
2012 G. Baumann
falsePositionMethod]x
6
- x - 1, 1, 2 // TableForm[, TableHeadings - {{}, {"k", "c"}}] &
k c
1
63
62
2 1.19058
3 1.11766
4 1.14056
5 1.1328
6 1.13537
7 1.13451
8 1.1348
9 1.1347
10 1.13473
11 1.13472
The same example was used previously as an example for the bisection method. The results are given
above. The last iterate equals the root a rounded to 5 significant digits. The false position method con-
verge only a little bit faster than the bisection method. But as the iterates become closer to a, the speed
of convergence increases.
1.2.3 Secant Method
The secant method and the false position method are known as straight-line approximations to the given
function y = f (x). Assume that two initial guesses to the root a are known and denoted by x
0
and x
1
. They
may occur on opposite side of a or on the same side of a. The two points (x
0
, f (x
0
)) and (x
1
, f (x
1
)), on the
graph of y = f (x), determine a straight line, called a secant line. This line is an approximation to the graph
of y = f (x) and its root x
2
is an approximation of a (see Figure 0.0).
To derive a formula for x
2
, we proceed in a manner similar to that used to derive the false position
formulas: Find the equation of the line and then find its root x
2
. The equation of the line is given by
y = p(x) = f (x
1
) + (x - x
1
)
f (x
1
) - f (x
0
)
x
1
- x
0
.
Solving p(x
2
) = 0, we obtain
x
2
= x
1
- f (x
1
)
x
1
- x
0
f (x
1
) - f (x
0
)
Having found x
2
, we can drop x
0
and use x
1
, x
2
as a new set of approximate values for a. this leads to an
improved value x
3
; and this process can be continued indefinitely.
Doing so, we obtain the general iteration formula
x
n+1
= x
n
- f (x
n
)
x
n
- x
n-1
f (x
n
) - f (x
n-1
)
for n 1.
This is the secant method. It is called a two-point method, since two approximate values are needed to
obtain an improved value. The bisection method is also a two-point method, but the secant method will
almost always converge faster than bisection.
Lecture_006.nb 13
2012 G. Baumann
Figure 0.0 illustrates how the secant method works and shows the difference between it and the method
of false position. From this example we can see that now the successive iterates are no longer guaran-
teed to enclose the root.
x
y
f(x
0
)
f(x
1
)
x
0
x
1
x
2
x
3
Figure 0.0. The secant method.
The algorithm for the secant method consists of three steps:
generate the approximated root by the derived iteration formula
change the boundary values a = b and b = c.
check if the error requirements are satisfied; if yes stop and return the value, if not return to step 1.
The following lines are an implementation of the secant method
secantMethod[f_, a_, b_] :=
Block]]c, e = 10
-5
, ain = a, bin = b, cold = b, k = 0, results = {}),
While[0 === 0,
k = k + 1;
(+ first step find the approximation +)
c = bin - (f /. x -> bin) (bin - ain) / ((f /. x -> bin) - (f /. x -> ain));
(+ second step select the root and terminate +)
ain = N[bin];
bin = N[c];
(+ third step select the root and terminate +)
If[Abs[(cold - c)] < e, Return[results], cold = c];
AppendTo[results, {k, c}];
]

The application of the secant method shows the iteration steps


14 Lecture_006.nb
2012 G. Baumann
secantMethod]x
6
- x - 1, 1, 2 // TableForm[, TableHeadings - {{}, {"k", "c"}}] &
k c
1
63
62
2 1.03067
3 1.17569
4 1.12368
5 1.13367
6 1.13475
7 1.13472
The same example was used previously as an example for both the bisection and false position method.
The results are given in the table above. The last iterate equals to the root a rounded to 5 significant
digits. Contrary to the bisection method the secant method converge very rapidly. When the iterates
become closer to a, the speed of convergence increases in a way which needs less steps.
Example 0.0. Secant and False Position Method
The function
f x_ := x
2

x
- 1
has a root in the interval [0, 1] since f (0) f (1) < 0.
Solution 0.2. The results for all three methods discussed so far, the bisection, the false position, and
secant methods, are demonstrated in the following. The function has a root near x 0.7 as shown in the
following Figure 0.0.
0.0 0.2 0.4 0.6 0.8 1.0
-1.0
-0.5
0.0
0.5
1.0
1.5
x
f
(
x
)
Figure 0.0. Graph of the function f (x) = x
2

x
- 1 for x [0, 1].
All methods start with two points x
0
= 0 and x
1
= 1. The following tables show the steps needed to derive
the root.
First the bisection method is applied to the problem
Lecture_006.nb 15
2012 G. Baumann
bisection[f[x], 0, 1] // TableForm[, TableHeadings - {{}, {"k", "c"}}] &
k c
1
1
2
2 0.75
3 0.625
4 0.6875
5 0.71875
6 0.703125
7 0.710938
8 0.707031
9 0.705078
10 0.704102
11 0.703613
12 0.703369
13 0.703491
14 0.70343
15 0.703461
16 0.703476
Next we use the false position method
falsePositionMethod[f[x], 0, 1] // TableForm[, TableHeadings - {{}, {"k", "c"}}] &
k c
1 1 -
-1

2 1.8816
3 0.420725
4 0.941745
5 0.589956
6 0.78112
7 0.660269
8 0.730769
9 0.68747
10 0.713283
11 0.697609
12 0.707023
13 0.701331
14 0.704759
15 0.70269
16 0.703937
17 0.703184
18 0.703638
19 0.703364
20 0.70353
21 0.70343
22 0.70349
23 0.703454
24 0.703476
25 0.703462
16 Lecture_006.nb
2012 G. Baumann
Finally the secant method is used.
secantMethod[f[x], 0, 1] // TableForm[, TableHeadings - {{}, {"k", "c"}}] &
k c
1 1 -
-1

2 0.569456
3 0.797357
4 0.685539
5 0.701245
6 0.703524
7 0.703467
The results for the different methods show that the bisection method needs the expected number of
iterations. However, the false position method needs more steps than expected. If we look at the results
generated during the iteration we observe that the root is approached. But during the first few iteration
steps there is some oscillation around the root which makes the convergence not direct. Contrary to the
secant method the false position method converge quite fast to the true root and does not show oscilla-
tions.
By using techniques from calculus and some algebraic manipulation, it is possible to show that the
iterates x
n
satisfy
a - x
n+1
= (a - x
n
) (a - x
n-1
)_
-f '' (x
n
)
2 f ' (x
n
)
.
The unknown number x
n
is between x
n
and x
n-1
, and the unknown number x
n
is between the largest and
the smallest of the numbers a, x
n
, and x
n-1
. The error formula closely resembles the Newton error
formula which is discussed in the next section. This kind of formula should be expected, since the secant
method can be considered as an approximation of Newton's method, based on the difference quotient
f ' (x
n
)
f (x
n
) - f (x
n-1
)
x
n
- x
n-1
.
Check as an exercise that the use of this expression in Newton's formula (0.XXX) will yield (see next
subsection)
x
n+1
= x
n
- f (x
n
)
x
n
- x
n-1
f (x
n
) - f (x
n-1
)
for n 1.
Lecture_006.nb 17
2012 G. Baumann

You might also like