You are on page 1of 30

# Chapter 2

Numerical Solution of Nonlinear Equations of One Variable

In this chapter, we study methods for ﬁnding approximate solutions to the equation f (x) = 0, where f is a real-valued function of a real variable. Some classical examples include the equation x − tan x = 0 that occurs in the diﬀraction of light, or Kepler’s equation x − b sin x = 0 used for calculating planetary orbits. Other examples include transcendental equations such as f (x) = ex + x = 0 and algebraic equations such as x7 + 4x5 − 7x2 + 6x + 3 = 0.

2.1

Bisection Method

The bisection method is simple, reliable, and almost always can be applied, but is generally not as fast as other methods. Note that, if y = f (x), then f (x) = 0 corresponds to the point where the curve y = f (x) crosses the xaxis. The bisection method is based on the following direct consequence of the Intermediate Value Theorem. THEOREM 2.1 Suppose that f ∈ C [a, b] and f (a)f (b) < 0. Then there is a z ∈ [a, b] such that f (z ) = 0. (See Figure 2.1.) The method of bisection is simple to implement as illustrated in the following algorithm: ALGORITHM 2.1 (The bisection algorithm) INPUT: An error tolerance � OUTPUT: Either a point x that is within � of a solution z or “failure to ﬁnd a sign change” 1. Find a and b such that f (a)f (b) < 0. (By Theorem 2.1, there is a

39

40

Applied Numerical Methods

y

y = f (x)

+ a

z

+ b

x

FIGURE 2.1: Example for the Intermediate Value Theorem applied to roots of a function. z ∈ [a, b] such that f (z ) = 0.) (Return with “failure to ﬁnd a sign change” if such an interval cannot be found.) 2. Let a0 = a, b0 = b, k = 0. 3. Let xk = (ak + bk )/2. 4. IF f (xk )f (ak ) > 0 THEN (a) ak+1 ← xk , (b) bk+1 ← bk . ELSE (a) bk+1 ← xk , END IF 5. IF (bk − ak )/2 < � THEN Stop, since xk is within � of z . (See the explanation below.) ELSE (a) k ← k + 1. END IF END ALGORITHM 2.1. Basically, in the method of bisection, the interval [ak , bk ] contains z and bk − ak = (bk−1 − ak−1 )/2. The interval containing z is reduced by a factor

(b) ak+1 ← ak .

Numerical Solution of Nonlinear Equations of One Variable of 2 at each iteration.

41

Note: In practice, when programming bisection, we usually do not store the numbers ak and bk for all k as the iteration progresses. Instead, we usually store just two numbers a and b, replacing these by new values, as indicated in Step 4 of our bisection algorithm (Algorithm 2.1).

y

f (x) = ex + x

+ −1
FIGURE 2.2:

z

x

Graph of ex + x for Example 2.1.

Example 2.1 f (x) = ex + x, f (0) = 1, f (−1) = −0.632. Thus, −1 < z < 0. (There is a unique zero, because f � (x) = ex + 1 > 0 for all x.) Setting a0 = −1 and b0 = 0, we obtain the following table of values. k 0 1 2 3 4 ak bk xk −1 0 −1/2 −1 −1/2 −3/4 −3/4 −1/2 −0.625 −0.625 −0.500 −0.5625 −0.625 −0.5625 −0.59375

Thus z ∈ (−0.625, −0.5625); see Figure 2.2. The method always works for f continuous, as long as a and b can be found such that f (a)f (b) < 0 (and as long as we assume roundoﬀ error does not cause us to incorrectly evaluate the sign of f (x)). However, consider y = f (x) with f (x) ≥ 0 for every x, but f (z ) = 0. There are no a and b such that f (a)f (b) < 0. Thus, the method is not applicable to all problems in its present form. (See Figure 2.3 for an example of a root that cannot be found by bisection.) Is there a way that we can know how many iterations to do for the method of bisection without actually performing the test in Step 5 of Algorithm 2.1?

+1 (1 − 0) < 10 This example illustrates the preferred way of stopping the method of bisection. if the method of bisection is programmed.1 in matlab. THEOREM 2. log(2) and test k > N .1 would never hold in practice. and the algorithm would never stop. or k = 19. in the algorithm. then |z − xk | < �. b] and f (a)f (b) < 0. Thus. it is preferable to compute an integer N such that N> log((b − a)/�) − 1. if � were chosen too small (such as smaller than the distance between machine numbers near the solution z ). Namely. One reason is because integer comparisons (comparing k to N .2 How many iterations are required to reduce the error to less than 10−6 if a = 0 and b = 1? −6 Solution: We need 2k1 .1. Simply examining how the widths of the intervals decrease leads us to the following fact.42 Applied Numerical Methods y y = f (x) x FIGURE 2. Another reason is because. 2 (bk − ak ) = (b − a)/2 Example 2.2 Suppose that f ∈ C [a. the comparison in Step 5 of Algorithm 2.3: Example of when the method of bisection cannot be applied. 2k+1 > 106 . The following is an example of programming Algorithm 2. rather than testing the length of the interval directly as in Step 5 of Algorithm 2. 2k+1 k+1 Thus. or doing it implicitly in a programming language loop. . if 1 < �. such as the matlab loop for k=1:N) are more eﬃcient than ﬂoating point comparisons. then |xk − z | ≤ b−a .

root = a + (b-a)/2. For example. func) returns the % result of the method of bisection. % In either case. eps. return end if (b < a) disp(’Error: b < a’) success = false.a). eps. and with function defined by y = f(x).m is available in Matlab’s working % directory. root = a + (b-a)/2. return end % Set N to be the smallest integer such that N iterations of bisection .success] = bisect_method (a. and success to be set to % ’true’. fa=feval(f. success = true. in the absence of excessive roundoff % error. 1e-10. success is also set to ’false’ if the tolerance cannot be met. handle incorrect arguments -if (fa*fb > 0) disp(’Error: f(a)*f(b)>0’). and the midpoint of the present % interval is returned in the variable root. 2. fb=feval(f. % tolerance eps. success = false. with starting interval [a. a message is printed. return end if (eps <=0) disp(’Error: eps is less than or equal to 0’) success = false. issuing % [root. ’xsqm2’) % from Matlab’s command window will cause an approximation to % the square root of 2 that.Numerical Solution of Nonlinear Equations of One Variable 43 function [root. with the following contents: % function [y] = xsqm2(x) % y = x^2-2. b. % return % Then.b). success] = bisect_method (a. error=b-a. b]. root = (a+b)/2.success] = bisect (1. % suppose an m-file xsqm2. % % success is set to ’false’ if f(a) and f(b) do not have the same % sign. % First. f) % % [root. b. has absolute error of at most 10^{-16} % to be returned in the variable root.

matlab’s editor does indentation automatically. else b=x. disp(sprintf(’ %12. There are statements to catch errors in the input arguments. To a large extend. if (error > eps) disp(’Error: epsilon is too small for tolerance to be met’). end error=b-a. disp(’ Error Estimate ’).m contains this program within the working directory or within matlab’s search path. within the matlab system. for i=1:N x= a + (b-a)/2. it is wise to use a uniform style in the comments. In fact. The comments at the beginning of the program state precisely how the function is used. (With % additional analysis of the minimum possible relative error % (according to the distance between floating point % numbers). success = false. x)). check to see if the tolerance was actually met. unreasonable values of epsilon can be determined % before the loop on i. if(fx*fa > 0) a=x.4e’. and automatically highlights comments and syntax elements such as “if” and “for” in diﬀerent . etc.4e %12. and one issues the command help bisect method from the matlab command window. When developing such computer programs. if the ﬁle bisect method.1 -disp(’ -----------------------------’). avoiding unnecessary work. 1. indentation of “if” blocks and “for” loops.1 ) % This is where we actually do Algorithm 2. fx=feval(f.44 Applied Numerical Methods % suffices to meet the tolerance -N = ceil( log((b-a)/eps)/log(2) .x). all of these comments (those lines starting with “%”) prior to the ﬁrst non-comment line are printed to the command window. disp(’ -----------------------------’). 2. return end This program includes practical considerations beyond the raw mathematical operations in the algorithm Observe the following.) error = (b-a)/2. error. root = a + (b-a)/2. end % Finally.

the script run bisect method.0000e-016 >> Observe that N is not available within the environment calling bisect method.4063e+000 1. as well as the package (if any) to which the program belongs.” is an example of a matlab function . These ﬁles.0000e-001 1.Numerical Solution of Nonlinear Equations of One Variable 45 colors. but is not reproduced here.2500e-001 1. In such a ﬁle.m in the version posted on the web page for the book. the only quantities from the command line that are available while the operations within the m-ﬁle are being done are those passed on the left.4375e+000 3.1e-2. consider the following dialog in the matlab command window.2500e+000 1. quantities that are to be returned must appear in the list in brackets on the left of the “=.2500e-002 1. The above implementation. This is done for bisect method.5625e-002 1. clear a = 1 . that is.4219e+000 root = 1. for brevity. For example.success] = bisect_method(1.5000e+000 2. termed matlab scripts Matlab!script.4141 success = 1 >> N ??? Undefined function or variable ’N’. and the only quantities available to the command environment (or other function) from which the function is called are the ones in the bracketed list on the left.0000e-016 >> [root. an m-ﬁle that begins with a function statement.3750e+000 6. >>eps eps = 1. This contrasts with matlab m-ﬁles that do not begin with a function statement. For example.m might contain the following lines.” while quantities that are input must appear in the list in parentheses on the right of the statement.2.5000e-001 1. It is also a good idea to identify the author and date programmed.1250e-002 1. stored in a matlab “m-ﬁle. >> eps = 1e-16 eps = 1. In a function m-ﬁle. and eps is not available within bisect method.’xsqm2’) N = 6 ----------------------------Error Estimate ----------------------------5.

5000e-001 1.1000 N = 3 ----------------------------Error Estimate ----------------------------5. b.5000e+000 2. We end our present discussion of matlab programs with a note on the use of the symbol “=. =” means “the value stored in the memory locations represented by .3750e+000 root = 1.” In statements entered into the command line and in m-ﬁles. >> run_bisect_method a = 1 b = 2 eps = 0.” In quantities printed by the matlab system. >> clear >> a ??? Undefined function or variable ’a’.0000e-001 1. eps. = means “store the computed contents to the left of the = into the variable represented by the symbol on the right.success] = bisect_method(a.2500e+000 1.4375 success = 1 >> a a = 1 >> b b = 2 >> eps eps = 0.46 Applied Numerical Methods b = 2 eps = 1e-1 [root.1000 >> The reader is invited to use matlab’s help system to explore the other aspects of the function bisect method. Observe now the following dialog in the matlab command window. ’xsqm2’) The clear command removes all quantities from the environment.2500e-001 1. >> b ??? Undefined function or variable ’b’.

We see that this iteration converges to z = 1. xk+1 = g (xk ) Example 2. . >> x = -0. and will see it again when we study systems of linear and nonlinear equations.1 z ∈ G is a ﬁxed point of g if g (z ) = z . 2. the approximations might not be close enough for our purposes to the intended value.5 .Numerical Solution of Nonlinear Equations of One Variable 47 the printed symbol is approximately equal to the printed quantity. If f (x) = g (x) − x. or even due to error in conversion from the internal binary form to the printed decimal form. The ﬁxed-point iteration method is deﬁned by the following: For x0 ∈ G. then a ﬁxed point of g is a zero of f . 2.2 The Fixed Point Method The so-called “ﬁxed point method” is a really general way of viewing computational processes involving equations. ﬁxed point iteration becomes g (x) = xk+1 = 1 (xk + 1). systems of equations. · · · . and equilibria. x2 = 3/4. . due to roundoﬀ error or other errors. x1 = 1/2. For example. 2 and the ﬁrst few iterates are x0 = 0. DEFINITION 2. It is also seen in more advanced studies of systems of diﬀerential equations.3 Suppose for k = 0. 1. x4 = 15/16.1 REMARK 2. We can study ﬁxed point iteration with this g with the following matlab dialog.4 If f is as in Example 2. We introduce it here. a corresponding g is g (x) = −ex . Example 2.1 on page 41. 2 Then. 1 (x + 1). Note that this is signiﬁcantly diﬀerent from the meaning that a mathematician attaches to the symbol. starting with x0 = 0. . x3 = 7/8. .

but the convergence does not appear to be quadratic.2 g satisﬁes a Lipschitz condition on G if there is a Lipschitz constant L ≥ 0 such that |g (x) − g (y )| ≤ L|x − y | for all x. a ﬁxed point of g ? Fixed-point iteration does not always converge.1) If g satisﬁes (2. g is said to be a contraction on the set G.5452 >> x = -exp(x) x = -0.) We observe a convergence in which the approximation appears to alternate about the limit. · · · . Consider g (x) = x2 . x3 = 256.exp(x) by simply pressing the up-arrow button on the keyboard. In .5649 >> Applied Numerical Methods (Here. if g � is continuous and |g � (x)| ≤ L for every x. Although it is tempting to pose problems as ﬁxed point iteration. the ﬁxed point iterates do not always converge. DEFINITION 2. then ﬁxed point iteration converges. Basically. (2. For diﬀerentiable functions. so x1 = 4. whose ﬁxed points are x = 0 and x = 1. We talk about convergence of ﬁxed point iteration in terms of Lipschitz constants.5797 >> x = -exp(x) x = -0. x2 = 16. then xk+1 = x2 k . then g satisﬁes a Lipschitz condition with Lipschitz constant L.5601 >> x = -exp(x) x = -0. a common way of thinking about Lipschitz constants is in terms of the derivative of g .6065 >> x = -exp(x) x = -0. An important question is: when does {xk }∞ k=0 converge to z . we can recall the expression x = . y ∈ G.5000 >> x = -exp(x) x = -0.5712 >> x = -exp(x) x = -0. if L < 1 (or if |g � | < 1).48 x = -0. For instance.1) with 0 ≤ L < 1. it is not hard to show (using the mean value theorem) that. If x0 = 2.

0] + [0.125. 0. 1/2]. 6 120 and suppose we wish to ﬁnd a Lipschitz constant for g over the interval [−1/2.Numerical Solution of Nonlinear Equations of One Variable fact.125. and ﬁxed point iteration converges for any x. k = 1.125.e. 1. 1/2]2 + [−1/2. we have g (x) = − 1 1 g � ([−1/2. Then. 1/384] 2 24 ⊆ [−0. then. Example 2. in such instances. we need to know that g maps G into itself.0. we state conditions under which the convergence is faster than linear. Thus.. Since g � (x) = −x2 /2 + x4 /24. to be assured that ﬁxed point iteration converges.e.) This is embodied in the following theorem.. if x ∈ G then g (x) ∈ G) and g satisﬁes a Lipschitz condition with 0 ≤ L < 1 (i. since |g � (x)| ≤ maxy∈[−0. If g is a contraction for all real numbers x. if G must be restricted (such as if g is not a contraction everywhere or if g is not deﬁned everywhere).00261] |y | = 0. g is a contraction on G). · · · 1−L L |xk − xk−1 |. 1/2]4 2 24 1 1 = − [0. 2.125. k = 1. 0. there is a unique z ∈ G such that z = g (z ). We will proceed by an interval evaluation of g � over [−1/2. 1/16] = [−1/8. 1/2]) ∈ − [−1/2. · · · converges to z . Two possibilities are with the following two theorems. · · · |xk − z | ≤ 1−L (2.002605] ⊆ [−0.00261]. THEOREM 2. and the sequence determined by x0 ∈ G. k = 0. xk+1 = g (xk ). (That is. g satisﬁes a Lipschitz condition with Lipschitz constant 0. 2. 1/4] + [0. |xk+1 − z | = |g (xk ) − g (z )| ≤ L|xk − z |.2) (2. 0] + [0.3) x5 x3 + .3 (Contraction Mapping Theorem in one variable) Suppose that g maps G into itself (i. 1/2]. the domain G can be taken to be the set of all real numbers. with error estimates |xk − z | ≤ Lk |x1 − x0 |. 49 so ﬁxed point iteration is linearly convergent with convergence factor C = L. then the hypotheses of the contraction mapping theorem are automatically satisﬁed.) On the other hand. 2.125.5 Suppose . (Later.

0 ≤ L < 1. In particular. Then g maps G into itself. − .4. Thus.1 = .6 Let x 1 + . Thus. 2] to the unique ﬁxed point z = g (z ). = − . |g (3/2) − 3/2| = 1/12 ≤ 1/4. 2] 2 2 [1. and |g (c) − c| ≤ (1 − L)ρ. 2 4 g (x) = 1 Thus. THEOREM 2. |g � (x)| ≤ 1 x∈[− 1 2. it may be relatively easy to verify that |g � | < 1. and Theorem 2. Of course. and |g � (z )| < 1. c + ρ]. we essentially do one iteration of ﬁxed point iteration and compare the change to the size of the region. − = 2 2 4 2 2 4 � � 1 1 . 2]) ∈ [− 1 2 . g � (x) = 1/2 − 1/x2 . 2]? We will use Theorem 2. 2]) ∈ − − 2 2 [1. . Therefore. 2]. g is a contraction on [1.3 to show convergence. Furthermore. Thus. 4 ] for every x ∈ [1. 2]. 2] with interval arithmetic. 4] � � � � � � � � 1 1 1 1 1 1 .3 that the ﬁxed point iteration converges for any starting point x0 ∈ [1.4. in Theorem 2. Then g is a contraction in a suﬃciently small interval about z . Evaluating g � (x) over [1. by Proposition 2.4] max |x| = 1 2 for every x ∈ [1. after which we may actually try ﬁxed point iteration to see if it stays in the domain and converges. 2]. 2].4. Suppose that g is a contraction on G with Lipschitz constant L. + −1. In fact. g maps [1. g � (x) is continuous in an interval about z .50 Applied Numerical Methods THEOREM 2. we obtain � � 1 1 1 1 1 � . we can conclude from Theorem 2.4 Let ρ > 0 and G = [c − ρ. the iterates will converge. letting ρ = 1/2 and c = 3/2. and g maps this interval into itself. provided x0 is picked suﬃciently close to z .5 Assume that z is a solution of x = g (x). Example 2. = g ([1. 2 x Can we show that the ﬁxed point iteration xk+1 = g (xk ) converges for any starting point x0 ∈ [1. since g � (x) ∈ g � ([1. 2] into [1.

for any x0 ∈ R. if 0 ≤ g � (x) ≤ L < 1 and the ﬁxed point iterates stay within the domain G. . On the other hand. y g (x2 )+ g (x1 )+ g (a)+ y=x y = g (x) + a FIGURE 2. xk 4 4. This latter situation is illustrated in Figure 2. if −1 < −L ≤ g � (x) ≤ 0 and ﬁxed point iterates stay within G.2615 15 4.3298 4. where z = 4 + 1 3 sin 2z . . then the ﬁxed point iterates xk converge monotonically to z .7 Let g (x) = 4 + 51 1 3 sin 2x and xk+1 = 4 + � 1 3 sin 2xk . Thus. � � � 2 �2 � |g (x)| = � cos 2x� �≤ 3 3 14 4.2309 . with the iterates xk alternately less than and greater than the ﬁxed point z = g (z ). .4: + x1 ++ x2 z + x b Example of monotonic convergence of ﬁxed point iteration.4. . . Then g : G → G and g is a contraction on R.Numerical Solution of Nonlinear Equations of One Variable Example 2. the following values are obtained. k 0 1 2 . the iterations xk+1 = g (xk ) will converge to z . Observing that for all x shows that g is a contraction on all of R. so we can take G = R.2615 It is not hard to show that. For x0 = 4. then ﬁxed point iteration converges.

Also. so g is monotonically increasing.3]/5 = [0.258. in this case.) Example 2.6 Assume that the iterations xk+1 = g (xk ) converge to a ﬁxed point z . 2.2 2. Inspecting the values in the following table. g maps G into G. so the exact range of g over [1. since g � (z ) = 4 5 �= 0.4. (It is assumed that g ∈ C q (G). 2.3) = 2.3 then implies that there is a unique ﬁxed point z ∈ G. k 0 1 2 3 4 xk 2. 0. there is a linear rate of convergence. Then the sequence {xk } converges to z with order q .3] is the interval [1. Recall if lim xk = z and |xk+1 − z | ≤ c|xk − z |α . (Indeed. � � � 2x � � � |g (x)| = � � 5 � ≤ 0. (We specify that c < 1 for α = 1. since x occurs only once in the expression for g � . Furthermore. where G contains z . the exact range of g � in this case. notice that the convergence is not fast. assume that q is the ﬁrst positive integer for which g (q) (z ) �= 0 and if q = 1 then |g � (z )| < 1. the range of g � is 2/5[1. that is. Since g � (x) = 2x/5. we say k→∞ THEOREM 2.92 < 1 for x ∈ G.) There are conditions under which the convergence of ﬁxed point iteration is faster than linear. Furthermore.) Theorem 2.3].258] ⊂ [1.3]. g (1) = 7/5 and g (2.095 Example 2.52 Applied Numerical Methods {xk } converges to z with rate of convergence α. 2.9 Let g (x) = x2 + 4 x 2 + = 2 x 2x . an interval evaluation gives 2[1. 2. It is easy to see that the ﬁxed point is z = 2. 2.168 2.4.140 2.92].3] > 0.116 2. In addition.8 Let g (x) = x2 + 6 5 and G = [1. 2.

By Theorem 2. how many iterations would be required to obtain the ﬁxed point z with |xk − z | < 10−16 ? Can this number of iterations be computed before actually doing the iterations? We can use the bound |xk − z | ≤ Lk |x1 − x0 | 1−L Example 2. it can be veriﬁed that. and 2 1 − .00002 2. The mean value theorem gives xk+1 − z = g � (ck )(xk − z ). the convergence is quadratic.2 2. 2 x2 4 . xk > xk+1 > 2 when xk > 2. ﬁxed point iterations diverge unless x0 = 2. so g (2) = 12.2. Starting with x0 = 2. Thus. and we are in the same situation as if x0 > 2. x2 > x1 > x0 > 2. the ﬁxed point iterations converge only if x0 = 2.6. the sequence xk+1 = g (xk ) converges to z = 2. for any x0 ∈ (0. . Now consider the convergence rate. Also.11 Consider again g from Example 2. g � (x) = 3 2 x .00000000 3 4 x − 4. We have that g � (x) = so g � (2) = 0.Numerical Solution of Nonlinear Equations of One Variable 53 be as in Example 2. Indeed. {xk } is a monotonically decreasing sequence bounded by 2 and hence is convergent. It can be shown that if 0 < x0 < 2. Thus. then x1 > 2. if x0 < 2. 8 3 � There is a unique ﬁxed point z = 2. That is.8. g �� (x) = k 0 1 2 3 xk 2. However. xk < 0. g (x) = Example 2.10 Let from the Contraction Mapping Theorem (on page 49).00909 2. Similarly. after which xk+1 > 2. and we cannot conclude linear convergence. ∞). then x1 > x0 > 2. the convergence is rapid. for some k . If x0 > 2. x3 so g �� (2) �= 0.6. · · · . and as indicated in the following table.

ﬁnd z such that f (z ) = 0.92)k < 10−16 Thus. study Figure 2.54 Applied Numerical Methods but the smallest bound we know on |g � (ck )| (and � hence the � smallest L in the formula) is L = 0. IEEE double precision accuracy.7. 2. Newton’s iteration for ﬁnding approximate solutions to this problem has the form f (x k ) for k = 0. 2. (2. f (xk )) is followed to the x-axis. the tangent line to the curve at point (xk . Thus. and |x1 − x0 | = 0.92).4 · (0.92) 0. Another conceptually useful way of deriving Newton’s method is using Taylor’s formula. |xk − z | ≤ Solving for k gives k > −16 log(25) ≈ 617. 2 .2)2 + 6 /5.4) xk+1 = xk − � f (xk ) REMARK 2.4 · (0.2 Newton’s method is a special ﬁxed-point method with g (x) = x − f (x)/f � (x).92)k .032. Newton’s method is quadratically convergent. We also compute x1 = (2.92k 0. and is therefore fast when compared to a typical linearly convergent ﬁxed point method.032 = 0. Newton’s method may diverge if x0 is not suﬃciently close to a root z at which f (z ) = 0. To ﬁnd xk+1 .3 Newton’s Method (Newton-Raphson Method) We now return to the problem: given f (x). Therefore. log(0. To see this.5 illustrates the geometric interpretation of Newton’s method. 618 iterations would be required to achieve.92 0. x = xk − f (xk )/f � (xk ) = xk+1 . However. roughly. 1 − 0. The tangent line is y − f (xk ) = f � (xk )(x − xk ). at y = 0. 1.6. We have 0 = f (z ) = f (xk ) + f � (xk )(z − xk ) + (z − xk )2 �� f (ξk ). · · · . Figure 2.

6: Examples of divergence of Newton’s method. g �� (z ) �= 0.5: x Illustration of two iterations of Newton’s method. Thus. the sequence diverges. the quadratic convergence follows from Theorem 2. f � (z ) (f � (z ))2 and. we would expect xk+1 to be closer to z than xk . Thus. f (xk+1 )) + + + xk xk+1xk+2 FIGURE 2. f (xk ) Observe that g (z ) = z . g � (z ) = 0 = 1 − f � (z ) f (z )f �� (z ) + . when xk+1 = xk − f (xk )/f � (xk ). the sequence oscillates.6 by analyzing Newton’s method as a ﬁxed point iteration. Consider f (x k ) xk+1 = xk − � = g (xk ). where ξk is between z and xk . f (xk )) (xk+1 . assuming that (z − xk )2 is small. . On the left. on the right. f � (xk ) Hence.6.Numerical Solution of Nonlinear Equations of One Variable 55 y (xk . z ≈ xk − f (x k ) . usually. y y + z + x k + xk +1 + xk +2 x +x + x k k+1 z x FIGURE 2. The quadratic convergence rate of Newton’s method can be inferred from Theorem 2.

5 1 -0.545396 -0.4 The Univariate Interval Newton Method A simple application of the ideas behind Newton’s method and the Mean Value Theorem leads to a mathematically rigorous computation of the zeros of a function f .567143 -0.606244 -0.567143 -0.0 -0.625 3 -0. In particular.500474 -0.59375 5 -0. f (x) (2. THEOREM 2.5) Solving (2.566895 20 -0.578125 10 -0. Compare bisection.567148 xk (Newton’s) -1.537883 -0.5) for z . Then the Mean Value Theorem (page 5) gives 0 = f (ˇ x) + f � (ξ )(z − x ˇ ). then applying the fundamental theorem of interval arithmetic (page 27) gives z=x ˇ− f (ˇ x) f � (ξ ) f (ˇ x) ∈x ˇ− � = N (f .567143 k xk (Fixed-Point) -1.566987 -0. simple ﬁxed-point iteration.56 Applied Numerical Methods Example 2. .5625 4 -0.567143 -0.367879 -0. x.568429 -0.12 Let f (x) = x + ex . x ˇ ). x.75 2 -0. • Newton’s method: xk+1 = xk − (xk − 1)exk f (x k ) (xk + exk ) = xk − = . x ˇ). � x k f (xk ) (1 + e ) 1 + e xk • Fixed-Point (one form): xk+1 = −exk = g (xk ). xk (Bisection) a = −1.567143 2. and Newton’s method. b = 0 0 -0. suppose x = [x.567143 -0. Let x ˇ be any point (such as the midpoint of x).7 Any solution z ∈ x of f (x) = 0 must also be in N (f . (2.0 -0.692201 -0. and suppose that there is a z ∈ x with f (z ) = 0. x] is an interval.6) We thus have the following.

2. x. “existence and uniqueness veriﬁed” ← “false. A formal algorithm for the interval Newton method is as follows. . Furthermore. OUTPUT: Either 1.” 1. x ˇ) the univariate interval Newton operator . “solution does not exist within the original x”. ALGORITHM 2. this x∗ is unique. (b) IF x ˇ �∈ x THEN RETURN. Then there is an Suppose f ∈ C (x) = C ([x.” 4. x x∗ ∈ x such that f (x∗ ) = 0. THEOREM 2.8 ˇ ∈ x. The interval Newton operator forms the basis of a ﬁxed-point type of iteration of the form ˇk ) for k = 1. k ← 1. x. In addition to providing bounds on any solutions within a given region. but provides a way to use ﬂoating point arithmetic (with upward and downward roundings) to provide rigorous upper and lower bounds on exact solutions. “solution does not exist” ← “false. x ˇ) ⊆ x. x]. . (a) x ˇ ← (x + x)/2. DO WHILE k <= N . and one of: (a) “existence and uniqueness veriﬁed” and “tolerance met.2 (The univariate interval Newton method) INPUT: x = [x. x]). 2. and N (f . and a stopping tolerance �. . a new interval x∗ such that any x∗ ∈ x with f (x∗ ) = 0 has x∗ ∈ x∗ .” or (d) “existence and uniqueness veriﬁed” but “tolerance not met. f : x ⊂ R → R. . the interval Newton method has the following property. We now discuss existence and uniqueness properties of the interval Newton method. x The interval Newton method is similar in many ways to the traditional Newton–Raphson method of Section 2. xk .” (c) “solution does not exist. .3 (page 54).Numerical Solution of Nonlinear Equations of One Variable 57 We call N (f .” 3. a maximum number of iterations N . xk+1 ← N (f .” (b) “existence and uniqueness not veriﬁed. or 2.

x∗ ← x ii. we need to check for this possibility.) Otherwise. the midpoint is computed approximately. the machine must evaluate f with interval arithmetic to take account of rounding error. RETURN. The interval Newton method generally becomes stationary. Although f is evaluated at a point in the expression x ˇ− f (ˇ x) f � (x) ˜.” i. x ˇ).2. ˜ ∩ x = ∅ (that is. “tolerance met” ← “false.” 6. if x ˜ ≤ x) THEN (d) IF x ˜ ≥ x and x “existence and uniqueness veriﬁed ← “true. RETURN. the end points of x can be proven to not change.2. if x ˜ ≤ x or x ˜ ≥ x) THEN (e) IF x (f ) IF w(˜ x) < � THEN ˜.58 Applied Numerical Methods ˜ ← N (f . x. to avoid problems arising from slow convergence.” ii. and it occasionally occurs (when the interval is very narrow). Thus. In Step 4a of Algorithm 2. the results are not mathematically rigorous. END ALGORITHM 2. . (That is. that the machine approximation lies outside the interval. Notes: 1. we start with the computations with the degenerate interval [ˇ x. i.) However. “solution does not exist” ← “true. (That is. 2. it is good general programming practice to enforce an upper limit on the total number of iterations of any iterative process. ˜ ⊆ x (that is. x ˇ]. 3. RETURN. under certain assumptions on the machine arithmetic. tolerance met ← “true. END DO 5.” iii. etc. (g) x ← x ∩ x for N (f . END IF (h) k ← k + 1. x (c) x ˇ ). x.

w(N (f .) Apply the interval Newton method x ← N (f . 1. 1. 1. x ˇ)) = O(w(x) ).09 × 10−1 5.41421355929452. x ˇ).5.00 × 10−1 4. x ˇ ← ﬂ ((x + x)/2).35 × 10−2 2. x. 1.41421356237309. for the initial width w(x) suﬃciently small.41421356237310] [1. Suppose further that f � is a ﬁrst order or higher order interval extension of f in the sense of Theorem 1. page 222 in [20].71 × 10−16 4.Numerical Solution of Nonlinear Equations of One Variable 59 Similar to the traditional Newton–Raphson method.37499999999999.9 (on page 28).9 (Quadratic convergence of the interval Newton method) Suppose f : x → R. 1. the interval Newton method exhibits quadratic convergence.13 appear in Table 2. w(xk ) δk = max {maxy∈xk {|y |}.41421356237310] [1.70 × 10−9 4.51 × 10−4 4. Then.00 × 100 1.1. x.41421356594718] [1.33 × 10−15 1. and suppose there is an x∗ ∈ x such that f (x∗ ) = 0. Here. (This is common knowledge. 1. k 0 1 2 3 4 5 6 Convergence of the interval Newton method with δk 5.1: f (x) = x2 − 2. We will illustrate this quadratic convergence with Example 2.9 is a special case of Theorem 6. and ρk = maxy∈f (xk ) {|y |}.41421356237309.41406249999999. 2.43750000000001] [1.00000000000000] [1.71 × 10−16 ρk 2.33 × 10−15 1. 1} is a scaled version of the width w(xk ).77 × 10−4 1.41421356237309.01 × 10−8 1. to f (x) = x2 − 2. although Theorem 2.33 × 10−15 xk [1.41441761363637] [1.) An example of a speciﬁc theorem along these lines is THEOREM 2. We will not give a proof of Theorem 2.3.13 (Taken from [22].41421356237310] . 2] and x ˇ = 1.00000000000000. The results for Example 2. 2 TABLE 2.71 × 10−16 4. suppose f ∈ C (x) and f � ∈ C (x). The displayed decimal intervals have been rounded out from the corresponding binary intervals. starting with x = [1.9 here.

methods that converge more rapidly than the method of bisection. In the secant method. such as we explain in Section 6. Problems as complicated as this are not uncommon. in which one supplies the input x. have been developed. In “black box” functions. and we will treat f as a “black-box” function. because techniques of automatic diﬀerentiation (or “computational diﬀerentiation”). f (x∗ ) = 0) does not have access to the internal workings.60 Applied Numerical Methods 2. so that f � cannot be easily computed. but that do not require evaluation of f � . However. and the output f (x) is returned. xk − xk−1 e2x . f � (xk ) is approximated by f � (xk ) ≈ f (xk ) − f (xk−1 ) . In this section.2.5 The Secant Method Under certain circumstances. f � cannot be directly computed and the secant method is useful.14 Suppose we wish to ﬁnd a zero of � � a f (x) = e−ax − g cos x + sin x + ln x . f may have a continuous derivative. Automatic diﬀerentiation is now an option for many problems of this type. applying Newton’s method to this problem was quite diﬃcult because it would have been diﬃcult and time-consuming to calculate f � (xk ) at each time step. In such cases. we will assume that f � cannot be computed. This is less true now than in the past. and are used in practice. page 215. 1 + x2 h(x) = and a is a constant. (1 + x + x2 ) 2 where . but it may not be possible to explicitly compute it. in certain situations. but the user (or the designer of the method for ﬁnding points x∗ . Example 2. there are still various situations involving black box functions f . However. Prior to widespread use of automatic diﬀerentiation. f is evaluated by some external procedure (such as a software system provided by someone other than its user). x g (x) = 1 + 3x2 + 5x + h(5 + ex + cos x). are useful. have become more widely available. such as applying the shooting method to solution of boundary-value problems (see the discussion in Chapter 10).

7: x Geometric interpretation of the secant method. f (xk )) is followed to the x-axis. f (xk−1 )) (xk . since f (xk−1 ) is known from the previous iteration. to obtain xk+1 .7) If f (xk ) and f (xk+1 ) have opposite signs. then. However. f (xk )) + xk+1 FIGURE 2.7). (see ﬁgure 2.10 (Convergence of the secant method) Let G be a subset of R containing a zero z of f (x). f (xk−1 )) and (xk . we need starting values x0 and x1 . Assume f ∈ C 2 (G) and there exists an M ≥ 0 such that M= 2 min |f � (x)| x∈G max |f �� (x)| x∈G . For the secant method. Let x0 and x1 be two initial guesses to z and let K� (z ) = (z − �. . THEOREM 2. there must be an x∗ between xk and xk+1 for which f (x∗ ) = 0. only one evaluation of the function f is required at each iteration. Geometrically. y (xk−1 . the secant to the curve through (xk−1 . z + �) ∈ G. Interestingly.Numerical Solution of Nonlinear Equations of One Variable The secant method thus has the form xk+1 � xk − xk−1 . = xk − f (x k ) f (xk ) − f (xk−1 ) � 61 (2. the convergence rate of the secant method is faster than linear but slower than quadratic. as with the bisection method.

618. namely at http://interval. M √ Note that (1 + 5)/2 ≈ 1. For Newton’s method |xk − z | ≤ q 2 with q < 1.62 Applied Numerical Methods δ and δ < 1.5000 . x1 ∈ K� (z ).louisiana. a fractional order of convergence between 1 k and 2. namely. and returns the roots of that polynomial. explained in [26]. appear in the graduate version of this book [1]. Let x0 . and available from the web page for the book. One common computation is ﬁnding all of the roots of a polynomial equation p(x) = 0.edu/Classical-and-Modern-NA/ A step of the interval Newton method is implemented with the matlab function i newton step no fp. The matlab function roots accepts an array containing the coeﬃcients of the polynomial.8660i -0.0. 2. Then. we might have the following matlab dialog.1.5000 + 0. >> c = [1. Additional techniques for root-ﬁnding. x4 .8660i >> c = [1 5 4] c = 1 5 4 >> r = roots(c) r = -4 -1 >> The ﬁrst computation computes approximations to the roots of the polynomial √ p(x) = x2 + x + 1. are available from the web page for the graduate version of this book. the iterates x2 . For example. while the second . approximations to −1/2 ± 3/2i.m. This function uses intlab (see page 35) for the interval arithmetic and for automatically computing derivatives of f .1] c = 1 1 1 >> r = roots(c) r = -0. where � = M · · · remain in K� (z ) and converge to z with error |xk − z | ≤ 1 ( 1+√5 )k δ 2 . at http://www. such as ﬁnding complex roots and ﬁnding all roots of polynomials.siam.6 Software The matlab function bisect method we presented. as well as a matlab function for Newton’s method.org/books/ot110. x3 .

NETLIB (at http://www. An important problem in population biology is analyzing the solution behavior of such dynamical . a programming language such as Fortran or C++. Here.) Various examples. There is a general function verifynlss in intlab for ﬁnding Generally. It is usually advantageous to use polished programs within the chosen system (matlab. as well as explanations of the underlying algorithms.5671 >> (Compare this with Example 2. etc.1. It may also be eﬃcient in certain cases to program directly the simple methods described in this chapter. f (x) is the density-dependent growth rate of the population. consider the general population model given by dx = f (x)x = (b(x) − d(x))x. See [26] for an introduction to some of the techniques. one can take advantage of these. are available within the matlab help system for roots and fzero. C. etc. for computing roots of polynomial equations and other equations. Software for ﬁnding veriﬁed bounds on all solutions is also available. if one is certain of their convergence in the context of their use. we focus on a model from population biology.5) x = -0. if the function f has special properties. x(t) is the population density at time t. To this end. matlab also contains a function fzero for ﬁnding zeros of more general equations f (x) = 0.-0. in developing specialized packages. A matlab dialog with its use is >> x = fzero(’exp(x)+x’.8) Here. dt (2. Thus. intlab has the function verifypoly for ﬁnding certiﬁed bounds on the roots of polynomials.7 Applications The problem of ﬁnding x such that f (x) = 0 arises frequently when trying to solve for equilibrium solutions (constant solutions) of diﬀerential equation models in many ﬁelds including biology. 2. engineering and physics. However.org/ contains various software packages in Fortran.Numerical Solution of Nonlinear Equations of One Variable 63 computation computes the roots of the polynomial p(x) = x2 + 5x + 4.netlib.). ﬁnding a root of an equation is a computation done as part of an overall modeling or simulation process. namely x = −4 and x = −1. The function b(x) is the densitydependent birth rate and d(x) is the density-dependent death rate.

i. these solutions satisfy dx/dt = 0 which is equivalent to x = 0. % x_star is set to the approximate solution to f(x) = 0 % if iteration stopped successfully. and f_prime (also a character % string giving an m-file name) as the derivative of f. This implies that the growth rate is given by f (x) = e−x − 2x. x = x0. with the following contents: % function [y] = xsqm2(x) % y = x^2-2. Clearly. ’xsqm2’. and x_star % is set to x0 otherwise.64 Applied Numerical Methods models.. with the following % function [y] = xsqm2_prime(x) % y = 2*x.e. values x which make the growth rate equal zero. and iteration % stops unsuccessfully if maxitr iterations have been done % without stopping successfully or if a zero derivative % is encountered. and % success = 0 if iteration stopped unsuccessfully. 1e-10.x). eps.success] = newton(x0. i. % iteration stops successfully if |f(x)| < eps. if abs(fval) < eps. % On return: % success = 1 if iteration stopped successfully. . ’xsqm2_prime’.m is available in Matlab’s working % directory. 20) % from Matlab’s command window will cause an approximation to the square % root of 2 to be stored in x_star. issuing % [x_star. success = 0. f. assume the birth rate is of Ricker type b(x) = e−x and the mortality rate is linear function given by d(x) = 2x.maxitr) % does iterations of Newton’s method for a single variable. A ﬁrst step in such analysis is often ﬁnding the equilibrium solutions. function [x_star.e. % For example.success] = newton (x0. % return % and an m-fine xsqm2_prime is also available. success = 1.eps. Using Newton Method given by the following programming algorithm in matlab. maxitr) % % [x_star.success] = newton(1. f (a character string giving % an m-file name) as function. f_prime. To ﬁnd the unique positive equilibrium we need to solve the equation e−x − 2x = 0. the trivial solution is an equilibrium solution of this population model usually referred to as the extinction equilibrium and f (x) = 0. suppose an m-file xsqm2.f_prime.. fval = feval(f. % using x0 as initial guess. for i=1:maxitr.f. To focus on a concrete example. % return % contents: % Then.5.

Let f and x be as in Problem 1. x_star = x. x.8 Exercises 1. end.9f ’.1 with pencil and paper. Consider the method of bisection applied to f (x) = arctan(x).Numerical Solution of Nonlinear Equations of One Variable disp(sprintf(’ %10. with initial interval x = [−4. fval)). i.9. if fpval == 0. 65 and the following matlab dialog >> y=inline(’exp(-x)-2*x’) >> yp=inline(’-exp(-x)-2’) >> [x_star.fval / fpval. fpval = feval(f_prime. (a) Are the hypotheses under which the method of bisection converges valid? If so.351733711 Thus.000000000 0. 2. end. return.351689332 0. x_star =x0.40) we obtain the following table of iterations for the solution 1 2 3 4 5 0.1].333333333 0. until k = 5.yp.0f %15. . 5.9f %15. i.351733711 0.1e-10. x_star = x0. x = 0.351733711 is the unique positive equilibrium of this model.y. then how many iterations would it take to obtain the solution to within an absolute error of 10−2 ? (b) Apply Algorithm 2.9f %15.9f ’.x). x.success]=newton(0. disp(sprintf(’ %10. fval)). x = x . end. 2. arranging your computations carefully so you gain some intuition into the process.0f %15.

375.39375. Repeat Problem 2. (a) Use a digital computer with double precision arithmetic to do iterations of Newton’s method. This function has a unique zero z = 0. 1. f (ak ). 2]. f (bk ).0.66 Applied Numerical Methods (a) Modify bisect method so it prints ak .35. 1. but with f (x) = x2 − 2 and initial interval x = [1. For each �. (b) Find the minimum number of iterations n needed so that the absolute error in the n-th approximation to the root is correct to 10−4 . (b) What do you observe for the diﬀerent starting points? What is |g � | at each starting point. and how might this relate to the behavior you observe? 6. Also. x] and a suitable ﬁxed point iteration function g (x) to accomplish this. starting with x = 5. 1. starting with x0 = 0. Consider f (x) = arctan(x). (a) Find an interval x = [x. 1. (a) Perform 10 iterations of the ﬁxed point method xk+1 = g (xk ). bk . Find an approximation to 1000 4 correct to within 10−5 using the ﬁxed point iteration method. 1. Iterate until one of the following occurs: 1 . 7. Hint: Deleting the semicolon from the end of a matlab statement causes the value assigned to the left of the statement to be printed. � = 10−64 . Verify all conditions of the contraction mapping theorem.3875. use the ﬁxed-point iteration method (with the g you determined in part (a)) to determine this positive real root accurate to within 10−4 . � = 10−16 . � = 10−32 . Use the program for the bisection method in Problem 2 to ﬁnd an ap1 proximation to 1000 4 which is correct to within 10−5 .3921875. while a statement consisting only of a variable name causes that variable name to be printed. 5. It is desired to ﬁnd the positive real root of the equation x3 + x2 − 1 = 0. � = 10−4 . and f (xk ) for each step. and � = 10−128 . If you want more neatly printed quantities.5. and x = 0. i. 1. � = 10−8 . compute the k at which the algorithm should stop.1. study the matlab functions disp and sprintf. (b) Try to solve f (x) = 0 with � = 10−2 . 4. x = −1. 8. 1.390625. x = 1. x = −5. 1. Consider g (x) = x − arctan(x). What behavior do you actually observe in the algorithm? Can you explain this behavior? 3. ii. so you can see what is happening.3.4. 1.

starting with x0 = 2. x0 = 4. for the function f (z ) = z 2 + 1.7i. (a) Let f be as in Problem 8 of this set. Hint: The free intlab toolbox. (ii) Explain the behavior you observe in terms of the graph of f . (β ) Do you think we could choose x0 = p exactly in practice? 9. 10. • an operation exception occurs. Let f (x) = x2 − a. and with various intervals that contain zero. if x0 > p. b > 0 and a �= b. and x0 = 64. a] (with x ˇ = 0) and other intervals of the form [−a. with starting guess z0 = 0. 0].2 + 0. 67 (i) Describe the behavior you observe. (c) Explain your results in terms of the shape of the graph of f and in terms of the convergence theory in this section. {1. x0 = 16. 1. but with the secant method instead of Newton’s method.3}. using complex arithmetic. a > 0. Experiment with the interval Newton method for this problem. where f (x) = x + ex . is recommended for this problem. (iii) Evidently. and f � (zk ). x0 = 32. Do three steps of Newton’s method. b]. (c) Iterate the interval Newton method to ﬁnd as narrow bounds as possible on the solution proven to exist in part 10b.5. then Newton’s method diverges. f (zk ). (b) Use the interval Newton method to prove that there exists a unique solution to f (x) = 0 for x ∈ [−1. form a table of 15 iterations of Newton’s method. (b) For a = 2. 11. (Note: Newton’s method with .Numerical Solution of Nonlinear Equations of One Variable • |f (x)| ≤ 10−10 . Explain what you have found. Repeat Exercise 8a. or • 20 iterations are completed. then Newton’s method converges. and if x0 < p. Although you may use a computer program. page 66. you should show intermediate results. Try some intervals of the form [−a. etc. 1. including zk . (d) Compare your analysis here to the analysis in Example 2.9 on page 52. (a) Write down and simplify the Newton’s method iteration equation for f (x) = 0. (α) What would happen if x0 = p exactly? Illustrate what would happen on a graph of f .0}. (Use pairs of starting points {0. there is a point p such that.0.) 12. mentioned on page 35. x0 = 8.

see Exercise 5 on page 322. in Section 8.68 Applied Numerical Methods complex arithmetic can be viewed as a multivariate Newton method in two variables.) .2.