Newton-Raphson Method

If you've ever tried to find a root of a complicated function algebraically, you may have
had some difficulty. Using some basic concepts of calculus, we have ways of numerically
evaluating roots of complicated functions. Commonly, we use the Newton-Raphson
method. This iterative process follows a set guideline to approximate one root,
considering the function, its derivative, and an initial x-value.

You may remember from algebra that a root of a function is a zero of the function. This
means that at the "root" the function equals zero. We can find these roots of a simple
function such as: f(x) = x2-4 simply by setting the function to zero, and solving:

f(x) = x2-4 = 0
(x+2)(x-2) = 0
x = 2 or x = -2

The Newton-Raphson method uses an iterative process to approach one root of a

function. The specific root that the process locates depends on the initial, arbitrarily
chosen x-value.

Here, xn is the current known x-value, f(x n) represents the value of the function at x n, and
f'(xn) is the derivative (slope) at x n. xn+1 represents the next x-value that you are trying to
find. Essentially, f'(x), the derivative represents f(x)/dx (dx = delta-x). Therefore, the term
f(x)/f'(x) represents a value of dx.

The more iterations that are run, the closer dx will be to zero (0). To see how this works,
we will perform the Newton-Raphson method on the function that we investigated
earlier, f(x) = x2-4. Below are listed the values that we need to know in order to complete
the process.

Theoretically, we could execute an infinite number of iterations to find a perfect

representation for the root of our function. However, this is a numerical method that we
use to lessen the burden of finding the root, so we do not want to do this. Therefore we
will assume that the process has worked accurately when our delta-x becomes less than
0.1. This value of precision should be specific to each situation.

The Newton-Raphson method does not always work, however. It runs into problems in
several places. First, consider the above example. What would happen if we chose an
initial x-value of x=0? We would have a "division by zero" error, and would not be able to
proceed. You may also consider operating the process on the function f(x) = x 1/3, using an
inital x-value of x=1. Do the x-values converge? Does the delta-x decrease toward zero (0)?

So, how does this relate to chemistry? Consider the van der Waals equation found in the
Gas Laws section of this text. Assuming that we have a set number of moles of a set gas,
not under ideal conditions, we can use the Newton-Raphson method to solve for one of
the three variables (temperature, pressure, or volume), based on the other two. To do
this, we need to use the van der Waals equation, and the derivative of this equation, both
seen below.

As you can see, the Van der Waals equation is quite complex. It is not possible to solve it
algebraically, so a numerical method must be used. The Newton-Raphson Method is the
easiest and most dependable way to solve equations like this, even though the equation
and its derivative seem quite intimidating.

Depending on the conditions under which you are attempting to solve this equation,
several of the variables may be changing. So, it may be necessary to use partial
derivatives. For the purposes of this example, we are assuming that pressure,
temperature, and volume are the only things changing, and that these values are all
functions of time. This avoids the use of a partial derivative; we simply differentiate all
variables with respect to time, as shown above. Some algebraic manipulation of the
equation and/or its derivative may be needed depending on the specific problem to be
solved. It is assumed that all of the variables but one are specified; that variable is used in
the expression for "xn+1" that Newton's method uses. Performing Newton's method on this
equation successfully would give a value of that variable which gives a solution when the
other variables are held constant at the values you specified.
 The Secant Method

One drawback associated with Newton's method is the requirement that there be some
mechanism to compute the derivative of the function. One may circumvent this
requirement and still adhere to the conceptual ideas that generated Newton's method.
Recall that description of Newton's method involved replacing the function f(x) with a line
which approximated the function in some sense. In particular, Newton's method
approximated the

function with a line that passed through the point  and had the

slope  .

Instead of defining the approximating line via the derivative  , the secant method

defines the line by requiring that it pass through  , values which are
readily available since they were used in the previous iteration.

  
Figure :   The secant method approximates the function f(x) by a line which passes through

the points  and  . The next approximation,  , is taken

to be the x-intercept of the approximating secant line.
    To formulate the mathematical definition of the secant method, begin with the secant
line

approximation passing through  and  :

    The x-intercept will define the next approximation  . To find the x-intercept, set
y=0 and solve for x. Doing this, we find

  

    Note the similarity with the equation which defines Newton's method. The latter part of

Equation 4.7 may be viewed as an approximation to  . That is, the secant
method may be viewed as a modification of Newton's method, where the modification
takes the form of the approximation

    Two initial approximations are needed for the secant method. These approximations do
not need to bracket a root. In other words, the secant method as described below is not a
bracketing method, although variations on the bracketing requirements leads to alternate
bracketing methods such as the ``Regula-Falsi'' (false-position) method.
 Fixed Point Iteration

When attempting to solve the equation f(x) = 0, it would be wonderful if we could rewrite
the equation in a form which gives explicitly the solution, in a manner similar to the
familiar solution method for a quadratic equation. While this does not occur for the vast
majority of equations we must solve, we can always find a way to re-arrange the equation
f(x) = 0 in the form:

(6.3)

Finding a value of x for which x = g(x) is thus equivalent to finding a solution of the equation f(x) =
0.

The function g(x) can be said to define a map on the real line over which x varies, such

that for each value of x, the function g(x) maps that point to a new point, on the real
line. Usually this map results in the points x and being some distance apart. If there is no
motion under the map for some x = xp, we call xp a fixed point of the function g(x). Thus we
have xp = g(xp), and it becomes clear that the fixed point of g(x) is also a zero of the
corresponding equation f(x) = 0.

Suppose we are able to choose a point x0 which lies near a fixed point, xp, of g(x), where of
course, we do not know the value of xp (after all, that is our quest here). We might
speculate that under appropriate circumstances, we could use the iterative scheme:

xn+1 = g(xn) (6.4)

where , and we continue the iteration until the difference between successive xn is
as small as we require for the the precision desired. To that level precision, the final value of xn
approximates a fixed point of g(x), and hence approximates a zero of f(x).

Figure: Fixed point iteration for the very simple case where g(x) is a linear function of x. In this
figure the line y = g(x) has been chosen to have a positive slope less than one and its iteration
started from the value x0. Similarly, the line has been chosen to have a positive slope

greater than one and its iteration started from the value . The convergent behavior of the g(x)

fixed point iteration is quite different from that for which diverges away from the fixed point

at xp. The divergence for is caused solely by the fact that the slope of is greater
than the slope of the line y = x.

The conditions for which the fixed point iteration scheme is convergent can be understood
by inspection of figure 6.3. In this case we consider the determination of the zero of the
simple function:

(6.5)
A straightforward choice for the function g(x) with which to do fixed point iteration is:

(6.6)

This example is particularly simple since we can solve f(x) = 0 analytically and find the fixed point of
g(x), xp = ma/(m-1). It is easy to verify that g(xp) = xp, confirming that xp is indeed a fixed point. The
fixed point iteration sequence is shown for two choices of the slope, m, both positive. The curve y

= g(x) has m < 1 and the curve has m > 1. It is clear that the m < 1 case results in
monotonic convergence to the fixed point xp, so that the fixed point is strongly attractive in this
case. The m > 1 case illustrates monotonic divergence away from the fixed point xp, so that the
fixed point is strongly repellent in this case.

While this simple linear case may seem special, it displays the behaviour which applies in
general to a continuous mapping function, g(x). In order to understand the reasons for the
difference in behaviour for the two cases m < 1 and m > 1, we need to follow the iteration
sequence in some detail. Once given the starting value x0, we compute g(x0), the
corresponding point on the y = g(x) curve. We then move along the line to
intersect the y = x line, and there read the value of x, and use this as the next iteration
value for x. Examination of the m < 1 iteration sequence in figure 6.3 shows that each
motion along the arrows of the iteration sequence leads towards the intersection point of
y = x and y = g(x), thus assuring convergence. A similar examination of the m > 1 case
shows that each motion along the arrows of the iteration sequence leads away from the
intersection point at xp, thus assuring divergence. While the point xp remains a fixed point

of , it is an unstable fixed point in the sense that starting arbitrarily close to the fixed
point still results in an iterative path that leads away from the fixed point. The terms
attractor and repeller then naturally describe the fixed point xp for the maps associated
with m < 1 and m > 1 respectively.
 Figure 6.3: Fixed point iteration for a general function g(x) for the four cases of interest.
Generalizations of the two cases of positive slope shown in figure 6.3 are shown on the left, and
illustrate monotonic convergence and divergence. The cases where g(x) has negative slope are
shown on the right, and illustrate oscillating convergence and divergence. The top pair of panels
illustrate strong and weak attractors, while the bottom pair of panels illustrate strong and weak
repellers.

We have considered iteration functions, like g(x), which have positive slopes in the
neighborhood of the fixed point, and shown that these lead to either monotonic
convergence or monotonic divergence. When g(x) has negative slope in the neighborhood
of the fixed point, the result is oscillating convergence or divergence, with convergence
requiring |m| < 1. The iteration sequences for all four cases are shown in 6.3 for more
general g(x). The conditions leading to convergence are unchanged from those derived for
the linear case as long as the neighborhood of the fixed point considered is small enough.
BIBLIOGRAPHY

 http://www.shodor.org/unchem/math/newton/
 http://www.mathcs.emory.edu/ccs/ccs315/ccs31
5/node18.html
 http://pathfinder.scar.utoronto.ca/~dyer/csca57/
book_P/node34.html