Topic 7: Numerical Differentiation

• We like to estimate the value of f '(x) for a given

function f(x).
• The derivative represents the rate of change of a
dependent variable with respect to an independent
variable.
• The difference approximation is
∆y f ( xi + ∆x) − f ( xi )
=
∆x ∆x
• If ∆x is allowed to approach zero, the difference
becomes a derivative:
dy f ( xi + ∆x) − f ( xi )
= lim
dx ∆x→0 ∆x
Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM
Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM
 Numerical Differentiation

• The Taylor series expansion of f(x) about xi is

f ( xi +1 ) ≈ f ( xi ) + f ′( xi )( xi +1 − xi )
• From this:
f ( xi +1 ) − f ( xi ) f ( xi +1 ) − f ( xi )
f ′( xi ) ≈ =
xi +1 − xi h

• This formula is called the first forward divided

difference formula and the error is of order O(h).

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 • Or equivalently, the Taylor series expansion of f(x)
about xi can be written as
f ( xi −1 ) ≈ f ( xi ) + f ′( xi )( xi −1 − xi )
• From this:
f ( xi −1 ) − f ( xi ) f ( xi ) − f ( xi −1 )
f ′( xi ) ≈ =
xi −1 − xi h

• This formula is called the first backward divided

difference formula and the error is of order O(h).

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 • A third way to approximate the first derivative is to
subtract the backward from the forward Taylor series
expansions:
f ( xi +1 ) = f ( xi ) + f ′( xi )h
−
f ( xi −1 ) = f ( xi ) − f ′( xi )h
_________________________
f ( xi +1 ) − f ( xi −1 ) = 2 f ′( xi )h

• This yields to
f ( xi +1 ) − f ( xi −1 )
f ′( xi ) =
2h
• This formula is called the centered divided difference
formula and the error is of order O(h2).

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 Forward

Backward

Centered

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 Example

See the example and the solution in the book

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 Finite Difference Approximation of Higher Derivatives

• The forward Taylor series expansion for f(xi+2) in terms

of f(xi) is
f ′′( xi )
f ( xi + 2 ) ≈ f ( xi ) + f ′( xi )(2h) + ( 2h) 2
2

• Combine equations:
f ′′( xi )
f ( xi + 2 ) = f ( xi ) + f ′( xi )(2h) + ( 2h) 2
2
−
 f ′′( xi ) 2 
2  f ( xi +1 ) = f ( xi ) + f ′( xi )h + h 
 2 
_______________________________________
f ( xi + 2 ) − 2 f ( xi +1 ) = − f ( xi ) + f ′′( xi )h 2
Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM
 • Solve for f ''(xi):

f ( xi + 2 ) − 2 f ( xi +1 ) + f ( xi )
f ′′( xi ) =
h2

• This formula is called the second forward finite divided

difference and the error of order O(h).
• The second backward finite divided difference which
has an error of order O(h) is

f ( xi ) − 2 f ( xi −1 ) + f ( xi − 2 )
f ′′( xi ) =
h2

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 • The second centered finite divided difference which has
an error of order O(h2) is

f ( xi +1 ) − 2 f ( xi ) + f ( xi −1 )
f ′′( xi ) =
h2

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 Numerical Differentiation
• High accurate estimates can be obtained by retaining
more terms of the Taylor series.

HIGH-ACCURACY DIFFERENTIATION
FORMULAS
• The forward Taylor series expansion is:
f ' ' ( xi ) 2
f ( xi +1 ) ≈ f ( xi ) + f ′( xi )( xi +1 − xi ) + h
2
• From this, we can write
f ( xi +1 ) − f ( xi ) f ' ' ( xi )
f ′( xi ) ≈ − h
h 2

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 • Substitute the second derivative approximation into the
formula to yield:
f ( xi + 2 ) − 2 f ( xi +1 ) + f ( xi )
f ( xi +1 ) − f ( xi ) h 2
f ′( xi ) ≈ − h
h 2

• By collecting terms:

− f ( xi + 2 ) + 4 f ( xi +1 ) − 3 f ( xi )
f ′( xi ) ≈
2h
• Inclusion of the 2nd derivative term has improved the
accuracy to O(h2).
• This is the forward divided difference formula for the
first derivative.
Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM
 FIGURE Forward Formulas

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 FIGURE Backward Formulas

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 FIGURE Centered Formulas

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 Example

See the example and the solution in the book

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 Example

Estimate f '(1) for f(x) = ex + x using the centered formula

of O(h4) with h = 0.25.

Solution

• From Table 23.3:

− f ( xi + 2 ) + 8 f ( xi +1 ) − 8 f ( xi −1 ) + f ( xi − 2 )
f ′( xi ) =
12h

xi − 2 = xi − 2h = 1 − 0.5 = 0.5
xi −1 = xi − h = 1 − 0.25 = 0.75
xi = 1
xi +1 = xi + h = 1 + 0.25 = 1.25
+ 2 = xi + 2hIndustrial
Dr MuhammadxiAl-Salamah, = 1 + 0.5Engineering,
= 1.5 KFUPM
 • In substituting the values:

− f (1.5) + 8 f (1.25) − 8 f (0.75) + f (0.5)

f ′( xi ) =
12(0.25)
− 5.982 + 8(4.740) − 8(2.867) + (2.149)
=
3
= 3.717

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 RICHARDSON EXTRAPOLATION

• The Richardson extrapolation formula:

4 k −1 D j +1,k −1 − D j ,k −1
D j ,k ≅
4 k −1 − 1
• The solution is terminated when

D1,k − D2,k −1
εa = 100% < ε s
D1,k

• We can use any formula to compute the values of Dj,1.

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM

 Example

Estimate f '(1) for f(x) = ex + x using Richardson

extrapolation. For the initial values, use the centered
formula of O(h4) with h = 0.25. Let εs = 0.001%.

Solution

k 1 2 3
j h Dj,1 Dj,2 |εa| Dj,3 |εa|
1 0.25 3.717925 3.718372 0.003 3.718281 0.0002
2 0.125 3.718260 3.718287
3 0.0625 3.718280

Dr Muhammad Al-Salamah, Industrial Engineering, KFUPM