How to terminate Newton’s method

MC225 - Numerical & Computational Methods  Assume that f ∈ C 1 , the current estimate xk is near the actual
Lecture 5 root r ∗ , and f 0 (x) 6= 0 in the neighborhood of r ∗ . From the Mean
Value Theorem, we get
f (xk ) − f (r ∗ ) = f 0 (α)(xk − r ∗ ),
Aditya Tatu
f (xk )
xk − r ∗ = 0
f (α)
for some α ∈ [xk , r ∗ ]. Using the Newton’s iteration,
xk+1 = xk − ff 0(xk)
(xk ) to substitute f (xk ) is the previous expression,
gives
f 0 (xk )
r ∗ − xk = (xk+1 − xk ) .
f 0 (α)
Since we have assumed that f 0 6= 0 around the root (i.e., the
February 1, 2022. function is not too flat near the root). we get
r ∗ − xk = M(xk+1 − xk )
where M is a finite constant. Also note that as xk approaches
r ∗ , f 0 (xk ) → f 0 (r ∗ ) and M → 1.
 Thus one can use |xk+1 − xk | as an approximate measure of the
absolute error for stopping the Newton’s iterations.

Convergence order and rate What does this mean?

 If r∗is the actual root, and xk and xk+1 are two consecutive For xk ∼ r ∗ , |xk+1 − r ∗ | = C |xk − r ∗ |2 .
estimates, how are |r ∗ − xk | and |r ∗ − xk+1 | related?  Taking log10 on both sides, and ignoring log10 (C ), gives
 Let xk be the current estimate of the root r ∗ of the function f . log10 |xk+1 − r ∗ | ∼ 2 log10 (|xk − r ∗ |)
2
 Near xk , f (x) = f (xk ) + f 0 (xk )(x − xk ) + f 00 (α) (x−x2
k)
, for
some α ∈ [x, xk ].  The number log10 |xk+1 − r ∗ | can be interpreted as the number
 Assuming xk is near r ∗ , substitute x = r ∗ in the above equation of digits of accuracy. Thus, if the initial estimate in the Newton’s
∗ 2 method is close to the root, the number of digits of accuracy
to get f (xk ) + f 0 (xk )(r ∗ − xk ) + f 00 (α) (r −x
2
k)
= 0.
0 ∗ ∗ doubles every iteration.
 Thus, |f (xk ) + f (xk )(r − xk )| ≤ c1 (r − xk )2 , c1k ≥ 0.
k
 For the bisection method, we previously had obtained
 From the Newton’s method, we have xk+1 = xk − ff 0(x k)
(xk ) . |xk+1 − r ∗ | = 12 |xk − r | . Thus the Newton’s method offers a much
f (xk )
 Subtracting r ∗ on both sides gives xk+1 − r ∗ = xk − r ∗ − f 0 (xk ) , faster convergence.
|f 0 (xk )(xk −r ∗ )−f (xk )| 0 ∗
|f (xk )+f (xk )(r −xk )|  Formally, given that a sequence {xk } converges to x, if
and thus, |xk+1 − r ∗| = |f 0 (xk )| = |f 0 (xk )| .
 Assuming near the root, 1
≤ c2k , 0 < c2k < ∞, gives |xk+1 − r ∗ |
|f 0 (xk )| lim = c, c > 0, p > 0
k→∞ |xk − r ∗ |p
|xk+1 − r ∗ | ≤ c1k c2k |r ∗ − xk |2 = C k |xk − r ∗ |2 we say that the sequence converges with a rate of convergence c
 Given that xk → r ∗ , we get with order of convergence p.
|xk+1 − r ∗ |  Note that for p = 1, c < 1. In this case, we say that the
lim =C >0 sequence has a linear convergence.
k→∞ |xk − r ∗ |2
f 00 (α)
 For p = 2, we say the sequence has quadratic convergence.
 Since c1k = 2 ,α ∈ [r ∗ , xk ] and c2k = 1
|f 0 (xk )| , if
f 00 (α)
xk → r ∗, C = f 0 (α) .