This chapter is a free translation of Chapter 1 of the book

Metodos numericos en ciencia e ingenieria (in Spanish) by

Guillermo Marshall. The translation has been undertaken by
Ph.D. Andrea Gonzalez and Prof. Bombardelli, for students of
the Department of Civil and Environmental Engineering of the
University of California, Davis.

FINITE DIFFERENCE METHOD

1.1

FIRST ORDER APPROXIMATION OF DERIVATIVES

The basic concept of the finite difference method relies in the approximation of the
derivative in a point by the quotient of differences in a small interval, transforming the
differential problem in an algebraic problem. For example, the derivative u' of the function
u(x) is approximated using the finite difference method as the secant of the tangent (see
Figure 1).

Figure 1: First derivative approximation

Defining the interval or step h as h=x2-x1 in any point of the interval x1 x x2, u' is
approximated as:

u' x

u x 2 u x 1
h

, x1 x x2

(0)

Expression (1) is an approximation for almost all the points in the interval (x1,x2), however
Rolle theorem shows that for at least one point there is an strict equality.
If we establish that:

u' x 1

u x 2 u x1
h

(0)

equation (2) represents a forward difference. On the other hand, if we establish that:

u' x 2

u x 2 u x1
h

(0)

equation (3) represents a backward difference. Figure 2 shows a center difference

approximation of u' (x) at x0 = (x1+x2)/2.

Figure 2: Centered finite differences approximation of the first order derivative.

u' x 0

u x 2 u x1
h

(0)

Considering some geometric properties, equation (4) represents a better approximation than
equations (2) or (3) because the first equation presents a better alignment with the tangent
line.
In order to demonstrate the previous statement it is necessary to study the precision of each
approximation. This study consists in the analysis of the error associated to the replacement
of the differential equation by a finite difference approximation and what parameters
influence this error. We will start with the centered difference approximation of u' (x). For
convenience Figure 3 shows the centered scheme, where x0 is the middle point between x1
and x2. Using the Taylors series expansion of u(x1) and u(x2) at u(x0)

u x 1 u x 0

h
1 h
u' x 0
2
2! 2

u x 2 u x 0

h
1 h
u' x 0
2
2! 2

u' ' x 0

u' ' x 0

1 h
u ' ' ' 1
3! 2

x1 1 x0

(0)

1 h
u' ' ' 2
3! 2

x0 2 x2

(0)

Figure 3: Centered difference approximation of u'(x)

Adding equation (5) and (6) and dividing by h the expression ended up as
3

1
u x 2 u x 1 u ' x 0 1 1 h 2 u ' ' ' 1 u ' ' ' 2
h
3! 2

(0)

Taking the absolute value and replacing u'''(1)+u'''(2) by 2 Max|u'''(x)| it is finally obtain
1
u x 2 u x 1 1 h 2 max u ' ' ' x
h
24

(0)

If u'''(x) is bounded between x1 and x2, the error associated to u'(x 0) using a centered
difference method is limited for a constant multiplied by h 2 or is order h2 (O(h2)). If the
value of h is reduced to a half, the error associated to this method is reduced by a factor of
4.
As we already mentioned, the forward and backward difference approximation of u'(x) are
less precise that the centered difference method. Using Figure 1 and equation (2), the
Taylors series expansion at x1 of u(x2) is

u x 2 u x1 h u' x1

1 2
h u' '
2!

x1 x2

(0)

Therefore:
1
u x 2 u x 1 u ' x 1 1 h max u ' ' x
h
2

(0)

The forward difference method (2) is O(h) only if u''(x) is limited between x1 and x2. If we
reduce the value of h to a half the error associated is reduced by a factor 2. The same result
is obtained for the precision analysis of the backward method.
Until now we have derived approximations of u'(x) using O(h) and O(h2) methods using
geometric considerations. We will present a method to derive systematically
approximations of u'(x) using the following expression:

u' x1 u x1 u x 2

( 0)

where and are coefficient to be determined. Expanding u(x2) in Taylors series at x1 and
replacing in (11) we ended up with:
u ' x 1 u x 1 h u' x 1

h 2
u' '
2

(0)

Making equal the coefficients in (11) and (12) we ended up with +=0 and h=1;
therefore =1/h and =-1/h. Replacing these values in (11) we get expression (2). The
precision of the forward difference method is O(h) for these values of and . It is required
that u'(x1) in (12) must be exact for the constant term and linear in u.
Therefore any difference scheme can be built in this way. In the undetermined coefficient
method the approximation of u'(x) is written as a weighted sum of the u(x i) . The
coefficients have to be determined considering that the method must be exact for the
biggest grade polynomial in u(xi).
A three points approximation for u'(x1) using the undetermined coefficient scheme it is
presented. Assuming u(x1), u(x2) and u(x3) are known at x1, x2=x1+h and x3=x2+h, the
approximation of u'(x1) is written as:
u' x1 u x1 u x 2 u x 3

Expanding u(x2) and u(x3) in Taylors series at x1 and replacing in (13)

(0)

 h 2

h 3
4h 3
2 h 2 u ' ' x 1
u ' ' ' 1
u' ' ' 2
3!
3
2

u ' x 1 u x 1 h 2 h u ' x 1

(0)
where x1 1 , 2 x2. Making equal the coefficients of equation (14) with equation (2) it
is obtain that ++=0, +2=1/h and +4=0, which implies that u'(x 1) is exact at the
constant, linear and quadratic term. The solution to the previous equations is =-3/2h,
=2/h and =-1/2h. Replacing these values in (13)
u' x1

1
3 u x1 4 u x 2 u x 3
2h

(0)

which is a forward difference approximation of three points O(h2).

If we want to built a centered difference approximation using four points and the highest
order possible for u'(x0) using x0=0, x1=-2h, x2=-h, x3=h and x4=2h and the values of u(x1),
u(x2), u(x3) and u(x4), the approximation of u'(x0) will be written as:
u' x 0 u x 1 u x 2 u x 3 u x 4

(0)

Expanding u(x1), u(x2), u(x3) and u(x4) in Taylors series at x0, replacing at (16), making the
coefficients equal to equation (2) and solving the 4 by 4 system of equations for the four
unknowns, the solution is =(12 h) -1, =-(2/3 h)-1 ,=(2/3 h)-1 and =-(12 h)-1. Replacing in
(16)
u' x 0

1
u x 1 8u x 2 8u x 3 u x 4
12h

(0)

which is four points centered difference approximation of O(h4).

1.2

HIGH ORDER DERIVATIVE APPROXIMATIONS

In order to obtain an approximation using finite differences of a second order derivative, we

will use the fact that u''(x)= (u'(x)) '. We will require at least values of u, for instance at the
points x1, x2 and x3 as it is shown in Figure 4. Calling the approximate value of the first
derivative at the points xm1 and xm2 u'(xm1) and u'(xm2), where xm1 and xm2 represents the half
value between x1 and x2, and x2 and x3 respectively, we will have:

u ' x m1
then

u x 3 u x 2
u x 2 u x1
, u' x m 2
h
h

(0)

u x 3 u x 2 u x 2 u x1

h
h

u' ' x

u ' x m 2 u x m1
h

u' ' x

u x 1 2u x 2 u x 3
h2

(0)

(0)

Depending on the point where the equation (20) is evaluated, the approximation will be
forward, backward or centered.

Figure 4: Second order derivative approximation.

The precision analysis of u''(x) using the centered approximation is analogous to the
analysis for u'(x). The approximation of u''(x2) will require the Taylor expansion of u(x 1)
and u(x3) at x2.

u x1 u x 2 h u' x 2

h2
h3
h 4 'v
u' ' x 2
u' ' ' x 2
u 1 x1 1 x2
2!
3!
4!

(0)

u x 3 u x 2 h u' x 2

h2
h3
h 4 'v
u' ' x 2
u' ' ' x 2
u 2
2!
3!
4!

(0)

x2 2 x3

Adding up equations (21) and (22)

1
h 2 'v

u
x

2
u
x

u
x

u
'
'
x

u 1 u ' v 2
1
2
3
2
2
4!
h

(0)

or
1
u x 1 2u x 2 u x 3 u ' ' x 2 M
h2

where M

(0)

1
max u ' v x
2
h

Equation (24) shows that the centered difference approximation is O(h2) if u'v(x) is limited.
Using the forward or backward difference method the order of the approximation will be
reducer to O(h).
Abramovitz and Stegun (1977) present a variety of formulas to made approximations of
high order for the derivatives. Godunov and Ryabenki (1964) and Isaacson and Keller
(1966) show a rigorous treatment to the problem of numerical approximations.
1.3 FINITE DIFFERENCE EQUATIONS
Linear differential equations are combination of derivatives of first and second order, order
zero and independent terms. The approximation of these equations using the finite
difference technique is called finite difference equations. As an introduction to the subject
we will present the approximation of a linear homogenous differential equation of first and
second order.
A first order equation example is given by the following initial value problem:
u' x a u x 0

x0

(0)

with u(x0)=u0 and a equal to a constant. A forward finite difference approximation for u'(x)
is given by:

u x 2 u x1
a u x1 0
h

(0)

This equation can be written as:

u x 2 1 ah u x 1 0

(0)

Dividing the x axis in (n+1) equal magnitude segments, numerating these segments as
0,1,2n and calling u(x1)=u(xn)=un and u(x2)=u(x2+h)=un+1, equation (27) is written as:
u n 1 (1 ah )u n 0

(0)

Equation (28) is a finite difference expression of first order because it requires the value of
u at a single point to solve it. In particular, it requires satisfying the initial condition. A

general solution of equation (26) is for example given by u n=c n, where is the solution of
the characteristic equation:
1 ah 0

(0)

1 ah

(0)

Using expression (30) and the initial condition, the solution of equation (28) is given by:
u n u 0 1 ah

(0)

It is observed that the exact solution for equation (28) has different behaviour depending on
the value of (1-ah):
(1-ah) > 1
(1-ah) =1
0<(1-ah)<1
(1-ah)=0
-1<(1-ah)<0
(1-ah)=-1
(1-ah)<-1

: The solution given by (31) increases monotonically.

: The solution is constant and equal to u0.
: The solution decreases monotonically
: The solution jumps from the initial value u0 to zero for any value of n.
: The solution decreases but oscillates.
: The solution keeps the same vale but it alternates signs.
: The solution amplitude increases but it alternates signs.

It is clear that the solution of the finite difference equation disagree with the behavior of the
differential equation, depending on the value of (1-ah). This behavior is related to the
numerically stability concept of the differential equation.
An example of a second order differential equation is given by the following contour
problem:
u' ' x u' x 0

(0)

And the following boundary conditions u(0)=1 and u(1)=0, where >0. The exact solution
to this equation is:
e e x
u x
e 1

(0)

Expression (32) represents a boundary layer problem (we suggest to draw the exact solution
for =1,0.1 and 0.01). Using centered difference approximation for u'(x) and u''(x), the
following expression is obtained:

u x 1 2u x 2 u x 3
u x u x1
3
0
2
2h
h

(0)

Using the same convention as the previous example, equation (34) can be written as:

1 p u n 1 2u n 1 p u n 1 0

(0)

where p= h/2. Expression (35) represents a second order finite difference equation and it
requires two different points to be solved. In particular equation (35) must satisfy the
boundary conditions stated at (32). In order to find the general solution for equation (35)
using a solution of the type u n=c n. Replacing this solution in (35) a characteristic equation
is obtained.

1 p 2 2 1 p 0

(0)

Expression (36) has two roots 1 and 2, and if they are different they are two particular
solutions of (35). The general solution is a linear combination of both roots:
u n c1 1n c 2 n2

(0)

where c1 and c2 depend on the boundary conditions. In this example both roots are real:
1=1 and 2=1-p2. Independently of the boundary conditions the solution of the finite
difference equation is:
1 p

1 p

u n c1 c 2

(0)

We expect that the solution given by expression (38) has a similar behavior than the
solution to the differential equation. However this is only true when p is less than 1. If p is
greater than 1 the exact solution will have an oscillatory behavior because it changes when
n is even or odd.
A question pops up: how can we avoid the oscillations? The answer is simple: using the
backward or forward finite difference approximation of u'(x) (depending of the sign of ).
References:

Abramowitz, M. and Stegun, I.A. (1972). Handbook of mathematical functions. Dover.

Godunov, S.K. and Ryabenki,V.S.(1964). Theory of difference schemes. North
Holland.
Isaacson, E. and Keller, H.B. (1968). Analysis of numerical methods. Mc-Graw Hill.