Roots of Non-Linear Equations (Recap):

f(x) = 0
f may be a function belonging to any class: algebraic, trigonometric,
hyperbolic, polynomials, logarithmic, exponential, etc.

Five types of methods can broadly be classified:

 Graphical method
 Bracketing methods: Bisection, Regula-Falsi
 Open methods: Fixed point, Newton-Raphson, Secant, Muller
 Special methods for polynomials: Bairstow’s
 Hybrid methods: Brent’s

1
Bracketing Methods
 Intermediate value theorem: Let f be a continuous fn on [a, b]
and let f(a) < s < f(b), then there exists at least one x such that
a < x < b and f(x) = s.
 Bracketing methods are application of this theorem with s = 0
 Nested interval theorem: For each n, let In = [an, bn] be a
sequence of (non-empty) bounded intervals of real numbers
such that 𝐼1 ⊃ 𝐼2 ⊃ ⋯ 𝐼𝑛 ⊃ 𝐼𝑛+1 ⊃ ⋯ and lim 𝑎𝑛 − 𝑏𝑛 = 0,
𝑛→∞
then ‫∞ځ‬
𝑛=1 𝐼𝑛 contains only one point.
 This guarantees the convergence of the bracketing methods to the root.
 In bracketing methods, a sequence of nested interval is generated
such that each interval follows the intermediate value theorem with s
= 0. Then the method converges to the root by the one point specified
by the nested interval theorem. Methods only differ in ways to
generate the nested intervals.
2
Bisection Method
 Principle: Choose an initial interval based on intermediate value
theorem and halve the interval at each iteration step to generate
the nested intervals.
 Initialize: Choose a0 and b0 such that, f(a0)f(b0) < 0. This is done
by trial and error.
 Iteration step k:
 Compute mid-point mk+1 = (ak + bk)/2 and functional value f(mk+1)
 If f(mk+1) = 0, mk+1 is the root.
 If f(ak)f(mk+1) < 0: ak+1 = ak and bk+1 = mk+1; else, ak+1 = mk+1 and
bk+1 = bk
 After n iterations: size of the interval dn = (bn – an) = 2-n (b0 – a0),
stop if dn ≤ ε
 Estimate the root (x = α say!) as: α = mn+1 ± 2-(n+1) (b0 – a0)
3
Regula-Falsi or Method of False Position
 Principle: In place of the mid point, the function is assumed to be
linear within the interval and the root of the linear function is chosen.
 Initialize: Choose a0 and b0 such that, f(a0)f(b0) < 0. This is done by
trial and error.
 Iteration step k:
 A straight line passing through two points (ak, f(ak)) and (bk, f(bk)) is
𝑥−𝑎𝑘 𝑏𝑘 −𝑎𝑘
given by: =
𝑓 𝑥 −𝑓 𝑎𝑘 𝑓 𝑏𝑘 −𝑓 𝑎𝑘
𝑏𝑘 −𝑎𝑘
 Root of this equation at f(x) = 0 is: 𝑥 = 𝑚𝑘+1 = 𝑎𝑘 − 𝑓 𝑎𝑘
𝑓 𝑏𝑘 −𝑓 𝑎𝑘
 If f(mk+1) = 0, mk+1 is the root.
 If f(ak)f(mk+1) < 0: ak+1 = ak and bk+1 = mk+1; else, ak+1 = mk+1 and bk+1 =
bk
 After n iterations: size of the interval dn = (bn – an), stop if dn ≤ ε
𝑏𝑛 −𝑎𝑛
 Estimate the root (x = α say!) as: α = 𝑎𝑛 − 𝑓 𝑎𝑛
𝑓 𝑏𝑛 −𝑓 𝑎𝑛
4
Example Problem
 𝐴 𝑏𝑎𝑙𝑙 𝑜𝑓 𝑟𝑎𝑑𝑖𝑢𝑠 𝑅 5.5 𝑐𝑚 𝑚𝑎𝑑𝑒 𝑤𝑖𝑡ℎ 𝑎 𝑚𝑎𝑡𝑒𝑟𝑖𝑎𝑙 𝑜𝑓
𝑠𝑝𝑒𝑐𝑖𝑓𝑖𝑐 𝑔𝑟𝑎𝑣𝑖𝑡𝑦 0.6 𝑖𝑠 𝑝𝑎𝑟𝑡𝑖𝑎𝑙𝑙𝑦 𝑖𝑚𝑚𝑒𝑟𝑠𝑒𝑑 𝑖𝑛 𝑤𝑎𝑡𝑒𝑟.
Find the di𝑠𝑝𝑙𝑎𝑐𝑒𝑚𝑒𝑛𝑡 𝑥 𝑏𝑒𝑙𝑜𝑤 𝑡ℎ𝑒 𝑤𝑎𝑡𝑒𝑟 𝑠𝑢𝑟𝑓𝑎𝑐𝑒.
𝑓 𝑥 = 𝑥3 − 0.165𝑥2 + 0.399 ∙ 10−3 = 0, x is in m.

A. Solve by Bisection Method

 Initial guess: xL = 0 m; xU = 2R = 0.11 m Functions.xlsx
 11 iterations to converge to x = 0.062 m
B. Solve by Regula Falsi Method
 Initial guess: xL = 0 m; xU = 2R = 0.11 m Functions.xlsx
 4 iterations to converge to x = 0.062 m 5
Bracketing Methods:
 𝐴𝑑𝑣𝑎𝑛𝑡𝑎𝑔𝑒𝑠
 Convergence to a root is guaranteed (may not get all
the roots, though!)
 Simple to program
 Computation of derivative not needed
 Disadvantages
 Slow convergence
 For more than one roots, may find only one solution
by this approach.
6
Open Methods: Fixed Point
 Problem: f(x) = 0, find a root x = α such that f(α) = 0
 Re-arrange the function: f(x) = 0 to x = g(x)
 Iteration: xk+1 = g(xk)
𝑥𝑘+1 −𝑥𝑘
 Stopping criteria: ≤𝜀
𝑥𝑘+1

 Convergence: after n iterations,

 At the root: α = g(α) or α - xn+1 = g(α) - g(xn)
𝑔 𝛼 −𝑔 𝑥𝑛
 Mean Value Theorem: = 𝑔′ 𝜉 for some 𝜉 ∈ 𝛼, 𝑥𝑛
𝛼−𝑥𝑛
𝑒𝑛+1
(α - xn+1) = gʹ(ξ)(α - xn) or en+1 = gʹ(ξ) en or = 𝑔′ 𝜉
𝑒𝑛
 Condition for convergence: │gʹ(ξ)│ < 1
𝑒𝑛+1
 As 𝑥𝑛 → 𝛼, = 𝑔′ 𝛼 = constant
𝑒𝑛
7
Open Methods: Fixed Point

y = g(x)
y=x y = x y = g(x)

x3 x2 x1 x0 x3 x2 x1 x0
Root α Root α

8
Example Problem
 𝐴 𝑏𝑎𝑙𝑙 𝑜𝑓 𝑟𝑎𝑑𝑖𝑢𝑠 𝑅 5.5 𝑐𝑚 𝑚𝑎𝑑𝑒 𝑤𝑖𝑡ℎ 𝑎 𝑚𝑎𝑡𝑒𝑟𝑖𝑎𝑙 𝑜𝑓
𝑠𝑝𝑒𝑐𝑖𝑓𝑖𝑐 𝑔𝑟𝑎𝑣𝑖𝑡𝑦 0.6 𝑖𝑠 𝑝𝑎𝑟𝑡𝑖𝑎𝑙𝑙𝑦 𝑖𝑚𝑚𝑒𝑟𝑠𝑒𝑑 𝑖𝑛 𝑤𝑎𝑡𝑒𝑟.
Find the di𝑠𝑝𝑙𝑎𝑐𝑒𝑚𝑒𝑛𝑡 𝑥 𝑏𝑒𝑙𝑜𝑤 𝑡ℎ𝑒 𝑤𝑎𝑡𝑒𝑟 𝑠𝑢𝑟𝑓𝑎𝑐𝑒.
𝑓 𝑥 = 𝑥3 − 0.165𝑥2 + 0.399 ∙ 10−3 = 0, x is in m.

A. Solve by Fixed Point Method

𝑥3+0.399∙10−4
 Rearrange: 𝑥 = 𝑠. 𝑡. 𝑔′ (𝑥) < 1
0.165𝑥
 Initial guesses: x = 0.001, 0.01, 0.003 m DO NOT
CONVERGE Functions.xlsx
 Initial guess: x = 0.08 m, it takes 5 iterations to converge
to x = 0.062 m 9
Example Problem
A. Solve by Fixed Point Method
 Rearrange (alternate):
3
𝑥 = 0.165𝑥 2 − 0.399 ∙ 10−4 𝑠. 𝑡. 𝑔′ (𝑥) < 1
 Initial guesses: x = 0.001, 0.01, 0.003, 0.08 m all
CONVERGE Functions.xlsx

10
Open Methods: Newton-Raphson
 Problem: f(x) = 0, find a root x = α such that f(α) = 0
 Principle: Approximate the function as a straight line
having same slope as the original function at the point of
iteration.

f(x0)

f(x1)

x0
y = f(x)
x1 x2

11
 Open Methods: Newton-Raphson
 Problem: f(x) = 0, find a root x = α such that f(α) = 0
 Iteration Step 0: Taylor’s Theorem at x = x0
𝑓 𝑥1 = 𝑓 𝑥0 + 𝑥1 − 𝑥0 𝑓 ′ 𝑥0 + 2!1 𝑥1 − 𝑥0 2 𝑓 ′′ 𝑥0 + ⋯ + 𝑛!1 𝑥1 − 𝑥0 𝑛

𝑓 𝑛 𝑥0 + 𝑛+1
1
!
𝑥1 − 𝑥0
𝑛+1 𝑛+1
𝑓 𝜉 , for some 𝜉 ∈ 𝑥0 , 𝑥1
 Iteration Step k:
𝑓 𝑥𝑘+1 = 𝑓 𝑥𝑘 + 𝑥𝑘+1 − 𝑥𝑘 𝑓 ′ 𝑥𝑘 + 2!1 𝑥𝑘+1 − 𝑥𝑘 2
𝑓 ′′ 𝑥𝑘 + ⋯ +
𝑛
1
𝑛!
𝑥𝑘+1 − 𝑥𝑘 𝑓 𝑛 𝑥𝑘 + 1
𝑛+1 !
𝑥𝑘+1 − 𝑥𝑘 𝑛+1
𝑓 𝑛+1 𝜉 , for some 𝜉 ∈ 𝑥𝑘 , 𝑥𝑘+1

 Assumptions: Neglect 2nd and higher order terms and assume that the root is
arrived at the (k+1)th iteration, i.e., f(xk+1) = 0
𝑓 𝑥
 Iteration Formula: 𝑥𝑘+1 = 𝑥𝑘 − ′ 𝑘
𝑓 𝑥𝑘
𝑥𝑘+1 −𝑥𝑘
 Start with x = x0 (Guess!) Stopping criteria: ≤𝜀
𝑥𝑘+!

12
Open Methods: Newton-Raphson
 Convergence: Taylor’s Theorem after n iterations,
0 = 𝑓 𝛼 = 𝑓 𝑥𝑛 + 𝛼 − 𝑥𝑛 𝑓 ′ 𝑥𝑛 + 2!1 𝛼 − 𝑥𝑛 2 𝑓 ′′ 𝜉
for some 𝜉 ∈ 𝑥𝑛 , 𝛼
𝑓 𝑥𝑛 ′′
2 𝑓 𝜉
 Re-arrange: + 𝛼 − 𝑥𝑛 = − 1
𝛼 − 𝑥𝑛
𝑓 ′ 𝑥𝑛 2! 𝑓 ′ 𝑥𝑛

1 2 𝑓 ′′ 𝜉 1 2 𝑓 ′′ 𝜉
or 𝛼 − 𝑥𝑛+1 = − 𝛼− 𝑥𝑛 or 𝑒𝑛+1 = − 𝑒𝑛 ′
2! 𝑓 ′ 𝑥𝑛 2! 𝑓 𝑥𝑛

𝑒𝑛+1 𝑓 ′′ 𝛼
 As 𝑥𝑛 → 𝛼, = − 1
= constant
𝑒𝑛 2 2 𝑓′ 𝛼

13
Open Methods: Newton-Raphson

Advantages:
y = f(x)
Faster convergence f(x0)
(quadratic) x1
x0
Disadvantages: f(x1)

Need to calculate
derivate

Newton-Raphson
method may get stuck!
14
Open Methods: Secant
 Principle: Use a difference approximation for the slope
or derivative in the Newton-Raphson method. This is
equivalent to approximating the tangent with a secant.
 Problem: f(x) = 0, find a root x = α such that f(α) = 0

f(x0)

f(x1)
f(x2)
x0 x1 x2
y = f(x)
x3
15
Open Methods: Secant
 Problem: f(x) = 0, find a root x = α such that f(α) = 0

 Initialize: choose two points x0 and x1 and evaluate f(x0) and f(x1)
𝑓 𝑥𝑘 −𝑓 𝑥𝑘−1
 Approximation: 𝑓′ 𝑥𝑘 ≈ , replace in Newton-
𝑥𝑘 −𝑥𝑘−1
Raphson
𝑥𝑘 −𝑥𝑘−1
 Iteration Formula: 𝑥𝑘+1 = 𝑥𝑘 − 𝑓 𝑥𝑘
𝑓 𝑥𝑘 −𝑓 𝑥𝑘−1
𝑥𝑘+1 −𝑥𝑘
 Stopping criteria: ≤𝜀
𝑥𝑘+1

16
Open Methods: Secant

Advantages:

Fast convergence
(slightly less than
y = f(x)
quadratic) f(x0)
x1
Overcomes the x0
disadvantage of having f(x1)
to calculate derivate

Secant method may

also get stuck!
17
Open Methods: Secant
 Convergence: Newton’s polynomial,
0 = 𝑓 𝛼 = 𝑓 𝑥𝑛 + 𝛼 − 𝑥𝑛 𝑓 𝑥𝑛 , 𝑥𝑛−1 + 12 𝛼 − 𝑥𝑛 𝛼 − 𝑥𝑛−1 𝑓 ′′ 𝜉

for some 𝜉 ∈ 𝑥𝑛−1 , 𝑥𝑛 , 𝛼

𝑓 𝑥𝑛 −𝑓 𝑥𝑛−1
Divided difference:𝑓 𝑥𝑛 , 𝑥𝑛−1 =
𝑥𝑛 −𝑥𝑛−1

 Re-arrange:
𝑓 𝑥𝑛 1 𝑓′′ 𝜉
+ 𝛼 − 𝑥𝑛 = 𝛼 − 𝑥𝑛+1 = − 𝛼 − 𝑥𝑛 2
𝛼− 𝑥𝑛−1
𝑓 𝑥𝑛−1 ,𝑥𝑛 𝑓 𝑥𝑛−1 ,𝑥𝑛

𝑓′′ 𝜉
𝑒𝑛+1 = − 1
2
𝑒𝑛 𝑒𝑛−1 ′ ′ , 𝜉 ′ ∈ 𝑥𝑛−1 , 𝑥𝑛 [using mean value theorem]
𝑓 𝜉

𝑒𝑛+1 𝑓′′ 𝛼
 As 𝑥𝑛 → 𝛼, = − 1
= constant
𝑒𝑛 𝑒𝑛−1 2 𝑓′ 𝛼

18
Open Methods: Secant
 Derivative of the Newton-Raphson method
is evaluated numerically using difference
approximation.
 Numerical methods for estimation of
derivative of a function will be covered in
detail later.
 Rest of the method is same.

19
Open Methods: Muller
 Principle: fit a quadratic polynomial through three
points to approximate the function.
 Problem: f(x) = 0, find a root x = α such that f(α) = 0

f(x0)
f(x1)
f(x2)

x0 x1 x2
y = f(x)

x3
20
Open Methods: Muller
 Initialize: choose three points x0, x1, x2 and evaluate f(x0), f(x1),
f(x2). Denote f(xk) = fk
 Fit a quadratic polynomial through three points xk, xk-1, xk-2 as:
p(x) = a(x - xk)2 + b(x - xk) + c
 Equations:
p(xk) = fk = c
p(xk-1) = fk-1 = a(xk-1 - xk)2 + b(xk-1 - xk) + fk
p(xk-2) = fk-2 = a(xk-2 - xk)2 + b(xk-2 - xk) + fk
 Define Divided Differences:
𝑓 𝑥𝑘 −𝑓 𝑥𝑘−1
 1st Divided Difference: 𝑓 𝑥𝑘 , 𝑥𝑘−1 =
𝑥𝑘 −𝑥𝑘−1
𝑓 𝑥𝑘 ,𝑥𝑘−1 −𝑓 𝑥𝑘−1 ,𝑥𝑘−2
 2nd Divided Difference: 𝑓 𝑥𝑘 , 𝑥𝑘−1 , 𝑥𝑘−2 =
𝑥𝑘 −𝑥𝑘−2
21
Open Methods: Muller
p(xk) = fk = c
p(xk-1) = fk-1 = a(xk-1 - xk)2 + b(xk-1 - xk) + fk
p(xk-2) = fk-2 = a(xk-2 - xk)2 + b(xk-2 - xk) + fk

 Express as:
𝑓𝑘 − 𝑓𝑘−1
= 𝑓 𝑥𝑘 , 𝑥𝑘−1 = 𝑎 𝑥𝑘−1 − 𝑥𝑘 + 𝑏
𝑥𝑘 − 𝑥𝑘−1
𝑓𝑘 − 𝑓𝑘−2
= 𝑓 𝑥𝑘 , 𝑥𝑘−2 = 𝑎 𝑥𝑘−2 − 𝑥𝑘 + 𝑏
𝑥𝑘 − 𝑥𝑘−2
 Solutions:
𝑓 𝑥𝑘 , 𝑥𝑘−1 − 𝑓 𝑥𝑘 , 𝑥𝑘−2
𝑎=
𝑥𝑘−1 − 𝑥𝑘−2
𝑏 = 𝑓 𝑥𝑘 , 𝑥𝑘−2 + 𝑎 𝑥𝑘 − 𝑥𝑘−2
22
 Properties of Divided Differences
𝑓 𝑥𝑘 ,𝑥𝑘−1 −𝑓 𝑥𝑘−1 ,𝑥𝑘−2
2nd Divided Difference: 𝑓 𝑥𝑘 , 𝑥𝑘−1 , 𝑥𝑘−2 = 𝑥𝑘 −𝑥𝑘−2

𝑓 𝑥𝑘 , 𝑥𝑘−1 − 𝑓 𝑥𝑘 , 𝑥𝑘−2
𝑎=
𝑥𝑘−1 − 𝑥𝑘−2
𝑓𝑘 − 𝑓𝑘−1 𝑓𝑘 − 𝑓𝑘−2
−
𝑥𝑘 − 𝑥𝑘−1 𝑥𝑘 − 𝑥𝑘−2
=
𝑥𝑘−1 − 𝑥𝑘−2 +𝑥𝑘−1 𝑓𝑘−1 − 𝑥𝑘−1 𝑓𝑘−1
𝑥𝑘 𝑓𝑘 − 𝑥𝑘 𝑓𝑘−1 − 𝑥𝑘−2 𝑓𝑘 + 𝑥𝑘−2 𝑓𝑘−1 − 𝑥𝑘 𝑓𝑘 + 𝑥𝑘 𝑓𝑘−2 + 𝑥𝑘−1 𝑓𝑘 − 𝑥𝑘−1 𝑓𝑘−2
=
𝑥𝑘 − 𝑥𝑘−1 𝑥𝑘−1 − 𝑥𝑘−2 𝑥𝑘 − 𝑥𝑘−2
𝑥𝑘−1 − 𝑥𝑘−2 𝑓𝑘 − 𝑥𝑘−1 − 𝑥𝑘−2 𝑓𝑘−1 − 𝑥𝑘 − 𝑥𝑘−1 𝑓𝑘−1 + 𝑥𝑘 − 𝑥𝑘−1 𝑓𝑘−2
=
𝑥𝑘 − 𝑥𝑘−1 𝑥𝑘−1 − 𝑥𝑘−2 𝑥𝑘 − 𝑥𝑘−2
𝑓𝑘 − 𝑓𝑘−1 𝑓𝑘−1 − 𝑓𝑘−2
−
𝑥𝑘 − 𝑥𝑘−1 𝑥𝑘−1 − 𝑥𝑘−2 𝑓 𝑥𝑘 , 𝑥𝑘−1 − 𝑓 𝑥𝑘−1 , 𝑥𝑘−2
= = = 𝑓 𝑥𝑘 , 𝑥𝑘−1 , 𝑥𝑘−2
𝑥𝑘 − 𝑥𝑘−2 𝑥𝑘 − 𝑥𝑘−2

23
 Note: Properties of Divided Differences
 1st Divided Difference:
𝑓𝑘 − 𝑓𝑘−1 𝑓𝑘−1 − 𝑓𝑘
𝑓 𝑥𝑘 , 𝑥𝑘−1 = = = 𝑓 𝑥𝑘−1 , 𝑥𝑘
𝑥𝑘 − 𝑥𝑘−1 𝑥𝑘−1 − 𝑥𝑘
 2nd Divided Difference:
𝑓 𝑥𝑘 ,𝑥𝑘−1 −𝑓 𝑥𝑘−1 ,𝑥𝑘−2 𝑓 𝑥𝑘 ,𝑥𝑘−1 −𝑓 𝑥𝑘 ,𝑥𝑘−2 𝑓 𝑥𝑘 ,𝑥𝑘−2 −𝑓 𝑥𝑘−1 ,𝑥𝑘−2
𝑓 𝑥𝑘 , 𝑥𝑘−1 , 𝑥𝑘−2 = = =
𝑥𝑘 −𝑥𝑘−2 𝑥𝑘−1 −𝑥𝑘−2 𝑥𝑘 −𝑥𝑘−1

𝑓 𝑥𝑘 , 𝑥𝑘−1 , 𝑥𝑘−2 = 𝑓 𝑥𝑘−1 , 𝑥𝑘 , 𝑥𝑘−2 = 𝑓 𝑥𝑘−2 , 𝑥𝑘−1 , 𝑥𝑘 = 𝑓 𝑥𝑘 , 𝑥𝑘−2 , 𝑥𝑘−1

We shall use these properties for the Theory of Approximation!

24
 Open Methods: Muller
 Principle: fit a quadratic polynomial through three points to
approximate the function.
 Problem: f(x) = 0, find a root x = α such that f(α) = 0
 Initialize: choose three points x0, x1, x2 and evaluate f(x0), f(x1), f(x2)
 Fit a quadratic polynomial through three points xk, xk-1, xk-2 as:
p(x) = a(x - xk)2 + b(x - xk) + c
 Constants: c = f(xk); b = f[xk, xk-1] + (xk - xk-1)f[xk, xk-1, xk-2]; a = f[xk,
xk-1, xk-2]
𝑏± 𝑏2 −4𝑎𝑐 2𝑐
 Iteration step k: 𝑥𝑘+1 = 𝑥𝑘 − or 𝑥𝑘+1 = 𝑥𝑘 −
2𝑎 𝑏± 𝑏2 −4𝑎𝑐
𝑥𝑘+1 −𝑥𝑘
 Stopping criteria: ≤𝜀
𝑥𝑘+1
25
Open Methods: Muller
 Convergence: Analysis is similar to Secant method, using
Newton’s Polynomial.
1 𝑓 ′′′ 𝜉
𝑒𝑛+1 = − 𝑒 𝑒𝑛−1 𝑒𝑛−2 ′ ′ ,
6 𝑛 𝑓 𝜉
𝜉∈ 𝑥𝑛−2 , 𝑥𝑛−1 , 𝑥𝑛 , 𝛼 , 𝜉 ′ ∈ 𝑥𝑛−1 , 𝑥𝑛
𝑒𝑛+1 1 𝑓 ′′′ 𝛼
as 𝑥𝑛 → 𝛼, = − = constant
𝑒𝑛 𝑒𝑛−1 𝑒𝑛−2 6 𝑓′ 𝛼

26