Chapter 2:

Solutions of
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions
Chapter 2: Solutions of Equations in One
The Bisection
Method
Variable
Fixed-Point
Iteration

Newton’s Method Peter W. White

Error Analysis for white@tarleton.edu
Iterative Methods

Department of Mathematics
Tarleton State University

Fall 2018 / Numerical Analysis

 Chapter 2:
Solutions of Overview
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions Introduction: Root Finding vs. Solutions
The Bisection
Method

Fixed-Point
Iteration
The Bisection Method
Newton’s Method

Error Analysis for

Iterative Methods Fixed-Point Iteration

Newton’s Method

Error Analysis for Iterative Methods

 Chapter 2:
Solutions of Population Example
Equations in One
Variable
Dr. White I Malthusian model of population is P 0 (t) = λP(t) with
P(0) = P0 . This has solution: P(t) = P0 eλt .
Introduction: Root
Finding vs. I If immigration is permited at a constant rate ν, then
Solutions

The Bisection
the model is p0 (t) = λP(t) + ν =⇒
ν λt
P(t) = P0 eλt +
Method


e −1 .
Fixed-Point λ
Iteration I Question: suppose that P is measured in thousands
Newton’s Method
of individuals, t is years, P0 = 1, 000, ν = 435 and
Error Analysis for
Iterative Methods P(1) = 1, 564, then find λ.
I That is, find λ where
435  λ 
1, 564 = 1, 000eλ + e −1
λ
 Chapter 2:
Solutions of Population Example (Continued)
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions In this equation, λ appears both inside and outside a
The Bisection
Method
transcendental function. Because of this, it is not
Fixed-Point
currently possible to solve for λ. The methods in this
Iteration
chapter are designed to approximate solutions of
Newton’s Method
problems that can not be solved exactly.
Error Analysis for
Iterative Methods Note: If F (x) = f (x) − a, then solving f (x) = a for x is
equivalent to finding roots (or zeros) of F . That is,
f (p) = a if and only if F (p) = 0.
 Chapter 2:
Solutions of Overview
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions Introduction: Root Finding vs. Solutions
The Bisection
Method

Fixed-Point
Iteration
The Bisection Method
Newton’s Method

Error Analysis for

Iterative Methods Fixed-Point Iteration

Newton’s Method

Error Analysis for Iterative Methods

 Chapter 2:
Solutions of Intermediate Value Theorem
Equations in One
Variable
Dr. White Theorem 1
Introduction: Root Let f be a continuous function on the interval [a, b] with
Finding vs.
Solutions L ∈ R between f (a) and f (b). Then there exist c ∈ [a, b]
The Bisection such that f (c) = L.
Method

Fixed-Point
Iteration
Corollary 2
Newton’s Method Let f be a continuous function on the interval [a, b] with
Error Analysis for f (a)f (b) < 0. Then there exists p ∈ (a, b) such that
Iterative Methods
f (p) = 0.
 Chapter 2:
Solutions of Bisection Algorithm or Binary-search Method
Equations in One
Variable
Dr. White
To find an approximation to the solution of f (x) = 0 given
Introduction: Root the continuous function f on the interval [a, b], where f (a)
Finding vs.
Solutions and f (b) have opposite signs:
The Bisection INPUT endpoints a, b; tolerance TOL; max number of
Method

Fixed-Point
iterations N0 .
Iteration OUTPUT approximate solution p or message of failure.
Newton’s Method Step 1: Set i = 1 and FA = f (a),
Error Analysis for
Iterative Methods
Step 2: While i ≤ N0 do steps 3-6,
Step 3: Set p = a + (b − a)/2 and FP = f (p),
Step 4: If FP = 0 or (b − a)/2 < Tol then
OUTPUT(p) and Stop.
Step 5: Set i = i + 1.
Step 6: If FA · FP > 0 then a = p, FA = f (p),
else set b = p.
Step 7: OUTPUT("Method failed, maximum iterations
reached") and Stop.
 Chapter 2:
Solutions of Example of Bisection Method
Equations in One
Variable
Dr. White

Introduction: Root Example 3

Finding vs.
Solutions Find the square root of 10 accurate to 10−2 . Hint: find the
The Bisection
Method
positive root of f (x) = x 2 − 10.
Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
 Chapter 2:
Solutions of Homework
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions
Homework assignment section 2.1, due: TBA
The Bisection
Method

Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
 Chapter 2:
Solutions of Overview
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions Introduction: Root Finding vs. Solutions
The Bisection
Method

Fixed-Point
Iteration
The Bisection Method
Newton’s Method

Error Analysis for

Iterative Methods Fixed-Point Iteration

Newton’s Method

Error Analysis for Iterative Methods

 Chapter 2:
Solutions of Definition and Example
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs. Definition 4
Solutions

The Bisection
Fixed Point: The number p is a fixed point for a given
Method function g if g(p) = p.
Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
Example 5
Determine any fixed points of the function g(x) = x 2 − 6
 Chapter 2:
Solutions of Sufficient Conditions for Existence
Equations in One
Variable
Dr. White
Theorem 6
Introduction: Root
Finding vs.
Solutions 1. If g ∈ C[a, b] and g(x) ∈ a, b for all x ∈ [a, b], then g
The Bisection has at least one fixed point in [a, b].
Method

Fixed-Point 2. If, in addition, g 0 (x) exists on (a, b) and a positive

Iteration
constant k < 1 exists with
Newton’s Method

Error Analysis for

Iterative Methods
|g 0 (x)| ≤ k , for all x ∈ (a, b),

then there is exactly one fixed point in [a, b].

 Chapter 2:
Solutions of Example
Equations in One
Variable
Dr. White

Introduction: Root Example 7

Finding vs.
Solutions Show that g(x) = cos(x) has exactly one fixed point in
The Bisection
Method
[0, π/3].
Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
 Chapter 2:
Solutions of Fixed Point Iteration
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs. Suppose we generate a sequence {pn } that converges to
Solutions

The Bisection
p with pn+1 = g(pn ) where g is continuous, then p is a
Method fixed point of g.
Fixed-Point
Iteration Proof.
Newton’s Method

Error Analysis for

Iterative Methods
 
p = lim pn+1 = lim g(pn ) = g lim pn = g(p).
n→∞ n→∞ n→∞
 Chapter 2:
Solutions of Homework
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions
Homework assignment section 2.2, due: TBA
The Bisection
Method

Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
 Chapter 2:
Solutions of Overview
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions Introduction: Root Finding vs. Solutions
The Bisection
Method

Fixed-Point
Iteration
The Bisection Method
Newton’s Method

Error Analysis for

Iterative Methods Fixed-Point Iteration

Newton’s Method

Error Analysis for Iterative Methods

 Chapter 2:
Solutions of Definition and Example
Equations in One
Variable
Dr. White

Introduction: Root Definition 8

Finding vs.
Solutions
Newton’s Method: Suppose f ∈ C 1 (I), where I is an
The Bisection open interval and {pn } be a sequence in I converging to p
Method
f (pn )
Fixed-Point with pn+1 = pn − 0 . Then p is a root (or zero) of f .
Iteration f (pn )
Newton’s Method

Error Analysis for

Iterative Methods

Example 9
Determine any roots of the function f (x) = x 2 − 6
 Chapter 2:
Solutions of Convergence
Equations in One
Variable
Dr. White
Theorem 10
Introduction: Root
Finding vs. Let f ∈ C 2 [a, b]. If p ∈ (a, b) such that f (p) = 0 and
Solutions
f 0 (p) 6= 0, then there exists a δ > 0 such that Newton’s
The Bisection
Method method generates a sequence {pn }∞ n=1 converging to p
Fixed-Point for any initial approximation p0 ∈ [p − δ, p + δ].
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
Note: nothing is said about how small δ might be, or the
rate of convergence. In practice, for most reasonable
problems, Newton’s method will either quickly converge
or it will be obvious that it will not converge.
 Chapter 2:
Solutions of Secant Method
Equations in One
Variable
Dr. White
Question: What if f 0 (x) is not readily available?
Introduction: Root
Finding vs.
Solutions
In this case replace f 0 (pn ) with an approximation. If pn is
The Bisection
Method close to pn−1 , then
Fixed-Point
Iteration f (pn ) − f (pn−1 )
f 0 (pn ) ≈ .
Newton’s Method
pn − pn−1
Error Analysis for
Iterative Methods
The Secant method assumes that two initial
approximations, p0 and p1 , are given, then for n = Z+ ,

f (pn )(pn − pn−1 )

pn+1 = pn − .
f (pn ) − f (pn−1 )
 Chapter 2:
Solutions of Example
Equations in One
Variable
Dr. White

Introduction: Root Example 11

Finding vs.
Use the Secant −4
√ method with a tolerance of 10 to
Solutions

The Bisection
Method
approximate 6.
Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
 Chapter 2:
Solutions of Homework
Equations in One
Variable
Dr. White
Read: The Method of False Position at the end of the
Introduction: Root
Finding vs. section and then:
Solutions

The Bisection
Method
Homework assignment section 2.3, due: TBA
Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods
 Chapter 2:
Solutions of Overview
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions Introduction: Root Finding vs. Solutions
The Bisection
Method

Fixed-Point
Iteration
The Bisection Method
Newton’s Method

Error Analysis for

Iterative Methods Fixed-Point Iteration

Newton’s Method

Error Analysis for Iterative Methods

 Chapter 2:
Solutions of Order of Convergence
Equations in One
Variable
Dr. White

Introduction: Root Definition 12

Finding vs.
Solutions Suppose {pn } is a sequence that converges to p, with
The Bisection pn 6= p for all n. If there exists positive constants λ and α
Method

Fixed-Point
with
Iteration |pn+1 − p|
lim = λ,
Newton’s Method n→∞ |pn − p|α
Error Analysis for
Iterative Methods then {pn } converges to p of order α, with asymptotic
error constant λ.

An iterative technique of the form pn+1 = g(pn ) is said to

be of order α if the sequence converges to the solution
p = g(p) of order α.
 Chapter 2:
Solutions of Order of Convergence (cont.)
Equations in One
Variable
Dr. White

Introduction: Root
If α = 1 (and λ < 1), the sequence is linearly
Finding vs.
Solutions
convergent.
The Bisection
Method
If α = 2, the sequence is quadratically convergent.
Fixed-Point
Iteration

Newton’s Method

Error Analysis for Example 13

Iterative Methods 1
Let pn = . Find the order of convergence to zero.
2n
 Chapter 2:
Solutions of Theorem
Equations in One
Variable
Dr. White

Introduction: Root
Finding vs.
Solutions
Theorem 14
The Bisection
Let C ∈ C[a, b] be such that g(x) ∈ [a, b], for all x ∈ [a, b].
Method Suppose, in addition, that g 0 is continuous on (a, b) and
Fixed-Point
Iteration
that a positive constant k < 1 exists with |g 0 (x)| ≤ k , for
Newton’s Method all x ∈ (a, b). If g 0 (p) 6= 0, then for any number p0 6= p in
Error Analysis for [a, b], the sequence pn+1 = g(pn ) converges only linearly
Iterative Methods
to the unique fixed point p in [a, b].
 Chapter 2:
Solutions of Theorem
Equations in One
Variable
Dr. White
Theorem 15
Introduction: Root
Finding vs.
Let p be a solution of the equation x = g(x). Suppose
Solutions
that g 0 (p) = 0 and g 00 is continuous with |g 00 (x)| < M on
The Bisection
Method an open interval I containing p. Then there exists a δ > 0
Fixed-Point such that, for p0 ∈ [p − δ, p + δ], the sequence defined by
Iteration
pn+1 = g(pn ) converges at a rate at least quadratically to
Newton’s Method
p. Moreover, for sufficiently large n,
Error Analysis for
Iterative Methods
M
|pn+1 − p| < |pn − p|2 .
2
 Chapter 2:
Solutions of Homework
Equations in One
Variable
Dr. White
Read: The part about Multiple Roots at the end of the
Introduction: Root
Finding vs. section and then:
Solutions

The Bisection
Method
Homework assignment section 2.4, due: TBA
Fixed-Point
Iteration

Newton’s Method

Error Analysis for

Iterative Methods