Numerical Analysis (MATH-254)

Fall 2021
Lecture No. 13 (Error/Convergence Analysis: Order of Convergence of
Functional Iteration Schemes)

Focus

SU
- Order of Convergence

- Linear and Quadratic Convergence

K
- Examples: Linear and Quadratic Convergence by using definition method

Definition (Order of Convergence of Functional Iteration Schemes):

Suppose that the sequence {xN }∞
M
N =0 converges to a root α, and let eN =
α − xN denote the error of the N th iterate. If two positive constants β > 0
and R ≥ 1 exist, and
|α − xN +1 | |eN +1 |
ah
lim = lim = β,
N →∞ |α − xN |R N →∞ |eN |R

then the sequence is said to converge to α with order of convergence R. The

number β is called the asymptotic error constant.
Sh

Remark: The asymptotic error constant also affects the speed of conver-
gence of the functional iteration schemes but it is not as important as the
order of convergence. The following two cases of order of convergence are
given special attention.
r.

Case-I (Linear Convergence): If the order of convergence is 1 (R = 1 and

β < 1) then the convergence will be linear.
D

Case-II (Quadratic Convergence): If the order of convergence is 2 (R =

2) then the convergence will be quadratic.

Explanatory Example: See Burden and Faires 2.4:pg.79-80

1
 Example 2.45 Show that the following sequence (Functional Iteration Scheme)
 
1 N
xn+1 = xn 1 + 2 , n ≥ 0,
2 xn
√ √
for computing N will converge quadratically to N .

SU
Example 2.46 Find the values of a and b so that the rate of convergence of
the iterative scheme
 
N
xn+1 = axn + b , n ≥ 0,
x2n

K
1
for computing N 3 becomes quadratic or higher.

Next Lecture: Theorems and Lemmas for Speed of Convergence, Speed/Rate

M
of Convergence of Bisection and Fixed-point Methods
ah
Sh
r.
D

2
 Numerical Analysis (MATH-254)
Lecture No. 14 (Theorems and Lemmas for Speed of Convergence,
Speed/Rate of Convergence of Bisection and Fixed-point Methods)

Focus

- Theorems about Order/Speed of Convergence

SU
- Speed of Convergence of Bisection Method

- Speed of Convergence of Fixed-point Method

K
Lemma 2.3 (Linear Convergence): Let g ∈ C[a, b] and suppose that
g(x) ∈ [a, b], ∀x ∈ [a, b]. Suppose that g 0 (x) is ctn on (a, b) with

|g 0 (x)| ≤ k < 1, ∀x ∈ (a, b).

M
If g 0 (α) 6= 0, then for any x0 ∈ [a, b], then the sequence of approximations
obtained from the functional iteration scheme
ah
xN = g(xN −1 ), N ≥ 1

converges only linearly to the unique fixed-point α ∈ [a, b].

Sh

Example 2.47R: Consider the iterative scheme

xN +1 = 0.4 + xN − 0.1x2N , N ≥ 0.

Will this scheme converge to the fixed-point α = 2? If yes, find its rate of
convergence by (a) Convergence Lemma, and (b) by using definition of order
r.

of convergence.
D

Lemma 2.4 (Quadratic Convergence): Let α be a solution of the equa-

tion x = g(x). Suppose that g 0 (α) = 0, g 00 (x) is ctn on an open interval
(a, b) containing α, and g 00 (α) 6= 0. Then there exists a δ > 0 such that
for x0 ∈ [α − δ, α + δ], the sequence of approximations obtained from the
functional iteration scheme {xN }∞ N =0 defined by the iteration

xN = g(xN −1 ), N ≥ 1

3
 converges at least quadratically to α.

General Convergence Theorem: Let g(x) be a function that is n times

continuously differentiable on an interval [a, b]. Furthermore, assume that
g(x) ∈ [a, b], ∀x and |g 0 (x)| ≤ k < 1, ∀x ∈ (a, b). If the unique fixed-point
α ∈ [a, b] satisfies

SU
g 0 (α) = g 00 (α) = · · · = g (n−1) (α) = 0, g (n) (α) 6= 0

then for any x0 ∈ [a, b], Fixed-point (functional) Iteration scheme

xN = g(xN −1 ), N ≥ 1

K
converges to α. The order of convergence is n and the asymptotic error con-
 (n)
 g (α) 
stant is  n! .

M
Relevant Questions: Examples 2.48, 2.49, 2.54, Qs(R). 18, 19,

Example 2.49R:

(a) Find the values of k1 and k2 such that the functional iterative scheme
ah
k2
xN +1 = k1 x2N + − 5, N ≥ 0,
xN
Sh

converges quadratically to α = 1.

(b) Find the order of convergence of the functional iterative scheme

xN x2N + 3k
xN +1 = , k > 0,
3x2N + k
r.

√
as it converges to the fixed-point α = k.
D

Speed of Convergence of Bisection Method: Prove/Show that the

functional iteration formula of the bisection method converges linearly, with
asymptotic error constant 12 .

Example 2.50R: Let α be the root of the equation f (x) = 0 in the interval
[x, α], where f is a continuous function. Show that the sequence generated

4
 by the bisection method for approximating α converges only linearly.

Speed of Convergence of Fixed-point Method: Prove/Show that the

functional iteration formula of the  fixed-point method converges linearly, with
 g00 (α) 
asymptotic error constant  2! .

Example 2.51R: If α and β are roots of the nonlinear equation x2 +ax+b =

SU
0, then show that the following iterative scheme
 
axN + b
xN +1 = − = g1 (xN ), N ≥ 0,
xN

K
will converge near x = α if |α| > |β| and the other scheme
 
b
xN +1 = − = g2 (xN ), N ≥ 0
xN + a

will converge near x = α if |α| < |β|. M

Next Lecture: Speed of Convergence of Newton’s and Secant Methods
ah
Sh
r.
D

5
 Numerical Analysis (MATH-254)
Lecture No. 15 (Speed of Convergence of Newton’s and Secant Methods )

Focus

- Speed of Convergence of Newton’s Method

SU
- Speed of Convergence of Secant Method

A. Order of Convergence of Newton’s Methods

K
Exercise: Determine the Order of Convergence of Newton’s Method for a
Simple Root.

Example 2.52R Find the multiplicity of a root α = π of tan x = 0. Find

M
the order of convergence of Newton’s method to this root by using (a) Con-
vergence Lemmas and (b) definition of order of convergence.

Example 2.53R Given nonlinear equation x2 ex = 0.

ah
(a) Find the order of convergence of Newton’s method to the root α = 0.

(b) Find the order of convergence of the First Modified Newton’s method to
Sh

the root α = 0.

(c) Find first 3 approximations of the root by using x0 = 0.1 by (i) Newton’s
method and (ii) First Modified Newton’s method. Compare the error of ap-
proximation for each method and explain the results.
r.

Example 2.55: Choose a constant λ to ensure the rapid convergence of the

iterative scheme
D

λxn + x−2 n +1
xn+1 = , n≥0
λ+1
to the root lying in the interval [1.4, 1.5], using Newton’s method by comput-
ing x2 , if x0 = 1.46.

Example 2.58 If x = α is a root of multiplicity m of f (x) = 0, then show

6
 that the rate of convergence of the Newton’s method is linear.

Example 2.59 If x = α is a root of multiplicity m of f (x) = 0, then show

that the rate of convergence of the first modified Newton’s method is at least
quadratic.

Example 2.60 If x = α is a root of multiplicity m = 5 of f (x) = 0, then

SU
show that the rate of convergence of the first modified Newton’s method is
at least quadratic.

Exercise The nonlinear equation f (x) = tan x = 0 has a root α = π. De-

velop the best numerical method for computing this root, then use it to find

K
x2 using x0 = 0.5. Find the rate of convergence of the developed formula.

Related Exercises: Examples: 2.56, 2.57,

M
B. Order of Convergence of Secant Method
Exercise: Prove that for secant method en+1 ∝ en en−1 . Hence, show that
the order of convergence is approximately 1.618 i.e., convergence of Secant
ah
method is super linear.

Next Lecture: Newton’s method for finding approximate solution of sys-

Sh

tems of NL algebraic equations

∗∗
Last updated on September30, 2021
r.
D