Numerical Methods/Equation Solving

1 Solution of Algebraic and Tran- 1.2 Initial Approximation

scendental Equations
An equation of the type f (x) = 0 is either algebraic or of them have in common the requirement that we need
transcendental.
E.g, these equations are algebraic.
2 7
2x = 5; x + x = 1; x = x(1 + 2x).
and these are transcendental
The last point about the interval is one of the most useful
properties numerical methods use to find the roots. All
to make an initial guess for the root. Practically, this is
easy to do graphically. Simply plot the equation and make
a rough estimate of the solution. Analytically, we can
usually choose any point in an interval where a change
of sign takes place. However, this is subject to certain
conditions that vary from method to method.

√
1.3 Convergence
x
sin x = 0; e = π; tan x = x.
A numerical method to solve equations will be a long pro-
While roots can be found directly for algebraic equations
cess. We would like to know, if the method will lead to a
of fourth order or lower, and for a few special transcen-
solution (close to the exact solution) or will lead us away
dental equations, in practice we need to solve equations of
from the solution. If the method, leads to the solution,
higher order and also arbitrary transcendental equations.
then we say that the method is convergent. Otherwise,
As analytic solutions are often either too cumbersome the method is said to be divergent.i.e, in case of linear
or simply do not exist, we need to find an approximate and non linear interpolation convergence means tends to
method of solution. This is where numerical analysis 0.
comes into the picture.
1.4 Rate of Convergence
1.1 Some Useful Observations
Various methods converge to the root at different rates.
• The total number of roots an algebraic equation can That is, some methods are slow to converge and it takes
have is the same as its degree. a long time to arrive at the root, while other methods can
lead us to the root faster. This is in general a compromise
• An algebraic equation can have at most as many pos- between ease of calculation and time.
itive roots as the number of changes of sign in f (x)
. For a computer program however, it is generally better
to look at methods which converge quickly. The rate of
• An algebraic equation can have at most as many convergence could be linear or of some higher order. The
negative roots as the number of changes of sign in higher the order, the faster the method converges.
f (−x) .
If ei is the magnitude of the error in the i th iteration,
• In an algebraic equation with real coefficients, com- ignoring sign, then the order is n if eei+1n is approximately
i
plex roots occur in conjugate pairs constant.
• If f (x) = a0 xn +a1 xn−1 +a2 xn−2 +...+an−1 x+ It is also important to note that the chosen method will
an with roots α1 , α2 , ..., αn then the following hold converge only if ei+1 < ei .
good:
∑
• αi = −a 1
1.5 Bisection Method
∑i a0
• αi αj = aa20
∏ i<j This is one of the simplest methods and is strongly based
• i αi = (−1)n aan0
on the property of intervals. To find a root using this
• If f (x) is continuous in the interval [a, b] and method, the first thing to do is to find an interval [a, b]
f (a)f (b) < 0 then a root must exist in the inter- such that f (a) · f (b) < 0 . Bisect this interval to get a
val (a, b) point (c, f (c)) . Choose one of a or b so that the sign of

1
2 1 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS

f (c) is opposite to the ordinate at that point. Use this as 1.6 False Position Method
the new interval and proceed until you get the root within
desired accuracy. The false position method (sometimes called the regula
falsi method) is essentially same as the bisection method
-- except that instead of bisecting the interval, we find
Example Solve 4x4 + 3x3 + 2x − 7 correct up to 2 where the chord joining the two points meets the X axis.
decimal places. The roots are calculated using the equation of the chord,
i.e. putting y = 0 in
f (x) = 4x4 + 3x3 + 2x − 7
⇒ f (2) · f (3) < 0 ⇒ ϵ = 1
2 10
−2
⇒i= 2.3010
0.3010 = 8 y − f (x ) = f (x1 ) − f (x0 ) (x − x )
0 0
⇒ x1 = 2.5 x1 − x0
The rate of convergence is still linear but faster than that
f (x1 ) = 5.625 > 0 of the bisection method. Both these methods will fail if f
has a double root.
⇒ x2 = 2.25 ⇒ ...x8 = 2.09

1.6.1 Example
1.5.1 Error Analysis
Consider f(x)=x2 −1. We already know the roots of this
The maximum error after the i th iteration using this pro- equation, so we can easily check how fast the regula falsi
cess will be given as method converges.
For our initial guess, we'll use the interval [0,2].
|b − a| Since f is concave upwards and increasing, a quick sketch
ϵi =
2i of the geometry shows that the chord will always inter-
⇒i≥ [log(b−a)−log ϵi ] sect the x-axis to the left of the solution. This can be
log 2
confirmed by a little algebra.
ϵi+1
As the interval at each iteration is halved, we have =
1
ϵi We'll call our nth iteration of the interval [an, 2]
2 . Thus this method converges linearly.
The chord intersects the x-axis when
If we are interested in the number of iterations the Bisec-
tion Method needs to converge to a root within a certain
tolerance than we can use the formula for the maximum (22 − 1) − (a2n − 1)
error. −(a2n − 1) = (x − an )
2 − an
Rearranging and simplifying gives
Example How many iterations do you need to get
the root if you start with a = 1 and b = 2 and the
tolerance is 10−4 ?
1 + 2an
x=
The error ϵi needs to be smaller than 10−4 . Use the for- 2 + an
mula for the maximum error:
Since this is always less than the root, it is also an₊₁
−i −4
ϵi = 2 · (2 − 1) < 10 The difference between an and the root is en=an−1, but
−i
ϵi = 2 < 10−4

Solve for i using log rules 1 + 2an an − 1 1

en+1 = −1= = en
2 + an an + 2 an + 2

log10 2−i < log10 10−4 This is always smaller than en when an is positive. When
an approaches 1, each extra iteration reduces the error by
−i · log10 2 < −4 two-thirds, rather than one-half as the bisection method
4 would.
i> = 13.29 = 14
log10 2
The order of convergence of this method is 2/3 and is
Hence 14 iterations will ensure an approximation to the linear.
root accurate to 10−4 . Note: the error analysis only gives In this case, the lower end of the interval tends to the root,
a bound approximation to the error; the actual error may and the minimum error tends to zero, but the upper limit
be much smaller. and maximum error remain fixed. This is not uncommon.
1.7 Fixed Point Iteration (or Staircase method or x = g(x) method or Iterative method) 3

1.7 Fixed Point Iteration (or Staircase

method or x = g(x) method or Iterative
−2 < f ′ (x) < 0
method)
Note that this convergence will only happen for a certain
If we can write f(x)=0 in the form x=g(x), then the point range of x. If the first estimate is outside that range then
x would be a fixed point of the function g (that is, the no solution will be found.
input of g is also the output). Then an obvious sequence
to consider is Also note that although this is a necessary condition for
convergence, it does not guarantee convergence. In the
error analysis we neglected higher powers of en, but we
can only do this if en is small. If our initial error is large,
xn+1 = g(xn )
the higher powers may prevent convergence, even when
If we look at this on a graph we can see how this could the condition is satisfied.
converge to the intersection. If |g′(x)|<1 is true at the root, the iteration sequence will
Any function can be written in this form if we define converge in some interval around the root, which may be
x=g(x), though in some cases other rearrangements may smaller than the interval where |g′(x)|<1. If |g′(x)| isn't
prove more useful(must satisfy the condition |g(x)|<1). smaller than one at the root, the iteration will not converge
This method is useful for finding a positive root to the to that root.
infinite series also.
Fixed Point Theorem (Statement) 1.7.2 Example
If g is continuous on[a,b] and a ≤ g(x) ≤ b for all x in
Lets consider f (x) = x3 + x − 2 , which we can see has
[a,b], then g has at least a fixed point in [a,b].
a single root at x=1. There are several ways f(x)=0 can
Further, suppose g'(x) is continuous on (a,b) and that a be written in the desired form, x=g(x).
positive constant c exists with |g'(x)| ≤ c <1, for all x in
The simplest is
(a,b). Then there is a unique fixed point α of g in [a,b].
Also, the iterates xn+1 = g(xn) n≥0 will converge to α 1): xn+1 = xn + f (xn ) = xn + 2xn − 2
3

for any choice of x0 in [a,b]. In this case, g ′ (x) = 3x2 + 2 , and the convergence con-
dition is
1.7.1 Error analysis

We define the error at the nth step to be 1 > |g ′ (x)| = 3x2 + 2, −1 > 3x2

Since this is never true, this doesn't converge to the root.

en = xn − x where x = g(x) 2)An alternate rearrangement is

Then we have
xn+1 = 2 − x3n

en+1 = xn+1 − x This converges when

= g(xn ) − x
= g(x + en ) − x
= −x + g(x) + en g ′ (x) + · · · 1 1
1 > |g ′ (x)| = | − 3x2 |, x2 < , |x| < √
= en g ′ (x) + · · · since g(x) = x 3 3

So, when |g′(x)|<l, this sequence converges to a root, and Since this range does not include the root, this method
the error will be approximately proportional to n. won't converge either.
Because the relationship between en₊₁ and en is linear, we 3)Another obvious rearrangement is
say that this method converges linearly, if it converges at
all.
√
When g(x)=f(x)+x this means that if xn+1 = 3 2 − xn

In this case the convergence condition becomes

xn+1 = xn + f (xn )
1  2

−2
√
converges to a root, x, of f then (2 − xn )− 3  < 1, (2 − xn ) < 33 , |xn −2| > 27
3
4 1 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS

Again, this region excludes the root.

4)Another possibility is obtained by dividing by x2 +1

2
xn+1 =
x2n + 1

In this case the convergence condition becomes

4|x|
< 1, 4|x| < (1 + x2 )2
(1 + x2 )2

Consideration of this inequality shows it is satisfied if x>1,

so if we start with such an x, this will converge to the root.
Clearly, finding a method of this type which converges is Tangent is in red; the function itself in blue.
not always straightforwards.
The tangent through the point (xn, f(xn)) is
1.8 Newton-Raphson
y − f (xn )
In numerical analysis, Newton’s method (also known = f ′ (xn )
x − xn
as the Newton–Raphson method or the Newton–Fourier
method) is an efficient algorithm for finding approxima- The next approximation, xn₊₁, is where the tangent line
tions to the zeros (or roots) of a real-valued function. As intersects the axis, so where y=0. Rearranging, we find
such, it is an example of a root-finding algorithm.
Any zero-finding method (Bisection Method, False Posi-
tion Method, Newton-Raphson, etc.) can also be used to xn+1 = xn − f (xn )
find a minimum or maximum of such a function, by find- f ′ (xn )
ing a zero in the function’s first derivative, see Newton’s
method as an optimization algorithm. 1.8.2 Error analysis

Again, we define the root to be x, and the error at the nth

1.8.1 Description of the method
step to be
The idea of the Newton-Raphson method is as follows:
one starts with an initial guess which is reasonably close to
the true root, then the function is approximated by its tan- en = x n − x
gent line (which can be computed using the tools of cal-
Then the error at the next step is
culus), and one computes the x-intercept of this tangent
line (which is easily done with elementary algebra). This
x-intercept will typically be a better approximation to the
f (x) + en f ′ (x) + 12 e2n f ′′ (x) + · · ·
function’s root than the original guess, and the method en+1 = en −
can be iterated. Suppose f : [a, b] → R is a differentiable f ′ (x) + en f ′′ (x) + · · ·
function defined on the interval [a, b] with values in the where we've written f as a Taylor series round its root, x.
real numbers R. The formula for converging on the root Rearranging this, and using f(x)=0, we get
can be easily derived. Suppose we have some current ap-
proximation xn. Then we can derive the formula for a
better approximation, xn+1 by referring to the diagram ( )
en+1 = en − en f ′ (x) + 12 en f ′′ (x) /f ′ (x) + · · ·
on the right. We know from the definition of the deriva- ′′
= − ff ′ (x)
(x) 2
en + ···
tive at a given point that it is the slope of a tangent at that
point. where we've neglected cubic and higher powers of the
We can get better convergence if we know about the func- error, since they will be much smaller than the squared
tion’s derivatives. Consider the tangent to the function: term, when the error itself is small.
Near any point, the tangent at that point is approximately Notice that the error is squared at each step. This means
the same as f('x) itself, so we can use the tangent to ap- that the number of correct decimal places doubles with
proximate the function. each step, much faster than linear convergence.
1.9 Higher order methods 5

This sequence will converge if which is always true.

This method converges to the square root, starting from
 ′′  any positive number, and it does so quadratically.
 f (x) 2  ′
f (x)
 
 f ′ (x) en  < |en |, |en | <
f ′′ (x)
If f′ isn't zero at the root, then there will always be a range 1.9 Higher order methods
round the root where this method converges.
There are methods that converge even faster than
If f′ is zero at the root, then on looking again at (1) we Newton-Raphson. e.g
see that we get

f (xn ) 1 f ′′ (xn )f 2 (xn )

en+1 = en /2 xn+1 = xn − ′
+
f (xn ) 2 f ′ 3 (xn )
and the convergence becomes merely linear.
which converge cubicly, tripling the number of correct
Overall, this method works well, provided f does not have digits at each iteration, which is 50% faster than Newton-
a minimum near its root, but it can only be used if the Raphson.
derivative is known.
However, if iterating each step takes 50% longer, due to
the more complex formula, there is no net gain in speed.
1.8.3 Example For this reason, methods such as this are seldom used.
Main Page - Mathematics bookshelf - Numerical Meth-
Let’s consider f(x)=x2 -a. Here, we know the roots ex- ods
actly, so we can see better just how well the method con-
verges.
We have

( )
f (xn ) x2 − a x2 + a 1 a
xn+1 = xn − ′
= xn − n = n = xn +
f (xn ) 2xn 2xn 2 xn
This method is easily implemented, even with just pen
and paper, and has been used to rapidly estimate square
roots since long before Newton.
The nth error is en=xn-√a, so we have

√
(en + a)2 +a √
en+1 = √
2( a+e )
− a
√ n
√
2a+2en a+e2n
= 2(
√
a+e )
− a
√ √n
√
2( a+en ) a+e2n
= √
2( a+en )
− a
e2
= √ n
2( a+en )

If a=0, this simplifies to en/2, as expected.

If a>0, en₊₁ will be positive, provided en is greater than
-√a, i.e provided xn is positive. Thus, starting from any
positive number, all the errors, except perhaps the first
will be positive.
The method converges when,

 
 e2 
|en+1 | =  √ n  < |en |
2( a + en ) 
so, assuming en is positive, it converges when

( √ )
en < 2 en + a
6 2 TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES

2 Text and image sources, contributors, and licenses

2.1 Text
• Numerical Methods/Equation Solving Source: https://en.wikibooks.org/wiki/Numerical_Methods/Equation_Solving?oldid=3173376
Contributors: Carandol, DavidCary, Sajo, Ralbury, Jguk, Hagindaz, Thenub314, Herbythyme, Billymac00, Mike.lifeguard, Recent Runes,
Mike’s bot account, IJstLrndAlotAboutStargateAtWP, Wutsje, Jitse Niesen, QuiteUnusual, Adrignola, Reyk, Pyruss, Syum90, Ah3kal,
Npsasikumar and Anonymous: 35

2.2 Images
• File:Newton_iteration.png Source: https://upload.wikimedia.org/wikipedia/commons/f/f0/Newton_iteration.png License: Public do-
main Contributors: Transferred from en.wikipedia to Commons. Original artist: Olegalexandrov at English Wikipedia

2.3 Content license

• Creative Commons Attribution-Share Alike 3.0