Chapter 3

APPROXIMATION METHODS FOR FIRST

ORDER DIFFERENTIAL EQUATIONS

When an exact solution to a first order differential

equation cannot be achieved using one of the
methods developed in chapter 2, it is still possible in
many cases, to obtain estimates of the solution using
numerical approximation techniques. These
methods yield numerical values of y(x) at selected
values of x. In this chapter we present some of the
most successful of these methods.
We restrict our development to estimating the
solution to a differential equation that can be written
in the form
dy
y   f ( x, y ) (2.1b)
dx

Formally, the solution to such a differential equation

is
y ( x )   f  x , y ( x ) dx  C (3.1)

159
160 Chapter 3

In order to determine the constant of integration C, a

value y(x0)  y0 must be given, and it must be
possible to determine C if an approximate value of
y(x) is to be obtained at some x  x0. Setting x = x0
in eq. 3.1, C is given by

C  y0    f  x, y( x) dx x  x0
(3.2)

3.1 – Taylor Series Approximation

The Taylor series representation of y(x) about the
point x0, when y(x0) is well defined, is

y ( x)   y ( n ) ( x0 )( x  x0 ) n 
n 0
(3.3)
 dy  1  d2y 
y ( x0 )    ( x  x0 )   2  ( x  x0 )2  ...
 dx  x  x0 2!  dx  x  x
0

The Taylor series methods involve approximating

y at some value of x by a Taylor series in powers of
(x – x0) truncated at some power (x – x0)M. Such a
truncated Taylor series is called a Taylor sum of
order M.
Using the notation
 dk y 
 k   y ( k ) ( xn ) (3.4)
 dx  x  x n

the M th order Taylor sum, expanded about the point

x0 is
Approximation Methods for First Order Differential Equations 161

y( x) 
y ( x0 ) y ( M ) ( x0 ) (3.5)
y ( x0 )  y ( x0 )( x  x0 )  ( x  x0 )2  ..  ( x  x0 )M
2! M!

With y(x0) specified, y’(x0) is obtained from the

differential equation and the higher derivatives are
found from derivatives of the differential equation.
That is, from the differential equation, we set x = x0
to obtain
y ( x0 )  f ( x0 , y0 ) (3.6a)
Then, taking the derivative of the differential
equation, we obtain
 df   f f dy   f f 
y ( x0 )         f (3.6b)
 dx  xy xy0  x y dx  x0 , y0  x y  x0 , y0
0

and from the derivative of y”(x), we obtain

 d  f f  
y ( x0 )     f  
 dx  x y   x , y
0 0

   f f    f f  
  f  y '  f  (3.6c)
 x  x y  y  x y   x , y
0 0

   f f    f f  
   f  f  f 
 x  x y  y  x y   x , y
0 0

With
   f f    2 f  2 f f f 
   f    2  f   (3.7a)
  x   x  y   x0 , y0   x  x y y x  x , y
0 0

and
162 Chapter 3

   f f     2 f  2 f  f  
2

  f   f   (3.7b)
 y  x y   x , y  xy y 2  y  
0 0 x0 , y0

we have
 2 f 2  f
2
 2 f f f  f  
2

y ( x0 )   2  f 2f  f   (3.8)

 x y 2 xy y x  y  
x0 , y0

As we see, finding these higher derivatives very

quickly becomes a cumbersome task. Our goal in
this treatment is to avoid the calculation of these
high order derivatives. As such, to approximate the
value of y(xN) we will restrict our approximation to
the second order Taylor sum. That is, we will
approximate y(xN) by

y ( k ) ( x0 ) y ( x0 )
y( xN )   ( xN  x0 )k  y( x0 )  y ( x0 )( xN  x0 )  ( xN  x 0 ) 2
k 0 k! 2!
(3.9)
Single iteration Taylor sum method
The single iteration Taylor sum method leading to
an approximate value of y at a point x1 is obtained
from eq. 3.9.
Example 3.1 – Single iteration Taylor
approximation
We consider the differential equation
dy
y   xy (3.10)
dx

from which
Approximation Methods for First Order Differential Equations 163

dy  d ( xy )
y     y(1  x 2 ) (3.11)
dx dx

Eq. 3.10 can be integrated straightforwardly to

obtain
2
y  Ce x /2
(3.12a)

If we constrain this solution by y(1) = e1/2 = 1.64872,

then C = 1, and the complete solution is
2
y( x)  e x /2
(3.12b)

From this, the exact value of y at x = 1.5, is y(1.5) =

3.08022.
The second order Taylor sum approximation of
y(x) expanded about
x0 = 1 is
y (1)
y ( x )  y(1)  y (1)  x  1   x  1 2 (3.13)
2!

With y(1) = 1.64872, we refer to eqs. 3.11 to obtain

y (1)   xy  x 1, y 1.64872  1.64872 (3.14a)

and
y (1)   y(1  x 2 ) x 1, y 1.64872  3.29744 (3.14b)

Therefore, the second order single iteration Taylor

sum approximation is
164 Chapter 3

1
y(1.5)  1.64872  1.64872 1.5  1  3.29744 1.5  1  2.88526
2

2!
(3.15)
which differs from the exact value of 3.08022 by
approximately 6.33%. □

Rather than taking a higher order Taylor sum,

which involves computing high order derivatives,
we can improve the accuracy of the results by using
a multiple iteration Taylor sum approximation of
order 2.
Two-iteration Taylor sum method

To approximate the value of y(x2) by two

iterations, let x1 be the midpoint between x0 and x2.
It is not necessary to choose the point between x0 and
x2 so that the two intervals are of equal width.
However, it is customary to do so. Therefore, for
multiple iteration Taylor sums, we choose

h  xℓ 1  xℓ (3.16)

to have the same value for all ℓ.

For a two-iteration method, the first iteration
yields a value of y(x1) from a Taylor sum expanded
about x0. Referring to eq. 3.9, we obtain
Approximation Methods for First Order Differential Equations 165

y ( x0 )
y ( x1 )  y( x0 )  y ( x0 )( x1  x0 )  ( x1  x0 )2 (3.17a)
2!

We determine the values of y   x1  and y   x1  as

described above in eqs. 3.5 through 3.7. From these

values, we then approximate y(x2) from
y ( x1 )
y ( x2 )  y ( x1 )  y ( x1 )( x2  x1 )  ( x2  x1 )2 (3.17b)
2!

Example 3.2 – Multiple iteration Taylor sum

method

We consider the differential equation discussed in

ex. 3.1,
dy
y   xy (3.10)
dx

with y(1) = e1/2 = 1.64872.

Using a two-iteration Taylor approximation to
estimate y(1.5), we divide
[1, 1.5] into two intervals defined by {x0, x1, x2} =
{1, 1.25, 1.5}. The first iteration will yield an
approximate value of y(1.25) by a second order
Taylor sum expanded about x = 1. Referring to eq.
3.9, we obtain
166 Chapter 3

y (1)
y (1.25)  y (1)  y (1)( x  1)  ( x  1)2 
2!
(3.18)
1
1.64872  1.64872 1.25  1  3.29744 1.25  1  2.16395
2

2!

To approximate y(1.5), we determine

y (1.25)   xy  x 1.25, y  2.16395  2.70494 (3.19a)

and

y (1.25)   y  xy    y  x 2 y   5.54512 (3.19b)

x 1.25, y  2.16395

These results are then used to approximate y(1.5) as

y (1.25)
y (1.5)  y (1.25)  y (1.25) 1.5  1.25  1.5  1.25 2 
2!
3.01348
(3.20)

It is straightforward to determine that this

approximate value is within 2.17% of the exact
value of y(1.5) = 3.08022. □
3.2 – Runge-Kutta Methods

Runge-Kutta methods for approximating the

solution to

y   f ( x, y ) (2.1b)
Approximation Methods for First Order Differential Equations 167

avoids the task of taking high order partial

derivatives of f(x, y) that arise in approximating yN
by a Taylor sum method.
With y(x0) given in the statement of the problem,
the Lth order Runge-Kutta approximation to yN =
y(xN) is given by

y N  y0   w1 R1  w2 R2  ...  wL RL  (3.21)

where the weights wm and the R-parameters Rm are

determined from f(x, y) at specified values of x and y
and do not involve any derivatives of f(x, y). They
are determined using the fact that f(x, y) is the slope
of the solution y(x) at a specified point (x, y). The
reader interested in the derivations of the various
order Runge-Kutta schemes is referred to Cohen, H.
1992, pp. 713, and 714.
To illustrate the form of the Runge-Kutta methods,
we present a few first, second and third order
Runge-Kutta methods. These methods are rarely (if
ever) used in practice. We present them to give the
reader an idea of the use of the Runge-Kutta method.

First order Runge-Kutta method

With
168 Chapter 3

h  xN  x0 (3.22)

the first order Runge-Kutta approximation is defined

by

y ( xN ) ≃ y( x0 )  w1 R1  y( x0 )  hf ( x0 , y0 ) (3.23)

From this, we can take w1 = 1 and

R1  hf ( x0 , y0 ) (3.24)

Second order Runge-Kutta methods

The Heun second order Runge-Kutta

approximation is defined by
1
y N  y0   R1  R2  (3.25)
2

so that w1 = w2 = 1/2, with

R1  hf  x0 , y0  (3.26a)

and

R2  hf  x0  h, y0  R1  (3.26b)

The midpoint second order Runge-Kutta method is

defined by w1 = 2,
w2 = 1 with
Approximation Methods for First Order Differential Equations 169

R1  hf ( x0 , y0 ) (3.26a)

and

R2  hf  x0  12 h, y0  12 R1  (3.27)

Therefore, the second order midpoint Runge-Kutta

approximation is

y N  y0  R2 (3.28)

The Ralston second order Runge-Kutta method is

defined by w1 = 1/3,
w2 = 2/3 with

R1  hf ( x0 , y0 ) (3.26a)

and

R2  hf  x0  34 h, y0  34 R1  (3.29)

Therefore, the second order Ralston Runge-Kutta

approximation is
1
y N  y0   R1  2 R2  (3.30)
3

Third order Runge-Kutta method

170 Chapter 3

A standard third order Runge-Kutta method is

defined by
1
y N  y0   R1  4 R2  R3  (3.31)
6

with

R1  hf ( x0 , y0 ) (3.26a)

R2  hf ( x0  12 h, y0  12 R1 ) (3.32a)

and

R3  hf  x0  h, y0  R1  2 R2  (3.32b)

Fourth order Runge-Kutta method

When one refers to “the Runge-Kutta method”,

this phrase almost always means a particular fourth
order scheme, referred to as the standard or classical
or common Runge-Kutta method.
To approximate yN by the standard Runge-Kutta
method, the values of the four Runge-Kutta
parameters are

R1  hf ( x0 , y0 ) (3.26a)
Approximation Methods for First Order Differential Equations 171

R2  hf  x0  12 h, y0  12 R1  (3.33a)

R3  hf  x0  12 h, y0  12 R2  (3.33b)

and

R4  hf  x0  h, y0  R3  (3.33c)

Then, yN is then given by

1
y N  y0 
6
 R1  2R2  2R3  R4  (3.34)

Single iteration Runge-Kutta approximation of

order 4

Just as with the single iteration Taylor sum, we

find a value of yN from y0 using a single iteration of
eq. 3.34.

Example 3.3 – Single step Runge-Kutta method

As in previous examples, we consider the

differential equation

y   xy (3.10a)

with y(1) = e1/2 = 1.64872.

172 Chapter 3

To determine y(1.5) using a single step, we take h

= 0.5 and define

R1  hx0 y0  .5e1/ 2  0.82436 (3.35a)

 
R2  h  x0  12 h  y0  12 R1   .5 1  12 .5 e1/ 2  12 0.82436  1.28806
(3.35b)

 
R3  h  x0  12 h   y0  12 R2   .5 1  12 .5 e1/ 2  12 1.28806  1.43297
(3.35c)

and

 
R4  h  x0  h  y0  R3   .5 1  .5 e1/ 2  1.43297  2.31127
(3.35d)

Then, from eq. 3.34,

1
y (1.5) ≃ y0   R  2R2  2 R3  R4  
6 1
1
e1/ 2 
6
 0.82436  2  1.28806  2  1.43297  2.31127  3.07834
(3.36)

which differs from the exact value of 3.08022 by

0.06%. This is a very accurate result considering
that h is fairly large. □

Multiple iteration Runge-Kutta method

Approximation Methods for First Order Differential Equations 173

As with the method of approximation by a Taylor

sum, to apply a multiple iteration Runge-Kutta
method, we divide [x0, xN] into several small
y   xy intervals. then apply the Runge-Kutta method
of eq. 3.34 over the interval [x0, x1] to find y1. From
this, we apply the Runge-Kutta scheme to the
interval [x1, x2] to find y2, and so on. We find yN by
applying the Runge-Kutta method over the interval
[xN–1, xN].

Example 3.4 – Multiple iteration Runge-Kutta

method

As before, we consider

y   xy (3.10a)

subject to y(1) = e1/2 = 1.64872.

To apply a two iteration Runge-Kutta method, we
take h = 0.25 and first determine y(1.25) by
calculating

R1  hx0 y0  0.41218 (3.37a)

R2  h( x0  12 h )( y0  21 R1 ) 
(3.37b)
 
.25 1  12 .25 e1/ 2  .25  .41218  0.52167
174 Chapter 3

R3  h( x0  12 h )( y0  12 R2 ) 
(3.37c)
 
.25 1  12 .25 e1/ 2  21 .52167  0.53706

and

 
R4  h( x0  h )( y0  R3 )  .25 1  .25 e1/ 2  .53706  0.68306
(3.37d)

from which
h
y (1.25)  y0   R  2 R2  2R3  R4  
6 1
.25
e1/ 2 
6
 .41218  2  .52167  2  .53706  .68306  2.18417
(3.38)

which agrees well with the exact value of 2.18420.

For the second iteration, this value of y(1.25) is
then taken to be y0 and y(1.5) is yN. Then, we
generate the second set of Runge-Kutta parameters

R1  hx1 y1  .25  1.25  2.18417  0.68255 (3.39a)

R2  h  x1  12 h  y1  12 R1  
(3.39b)
.25 1.25  12 .25  2.18417  12 .68255  0.86812

R3  h  x1  12 h  y1  12 R2  
(3.39c)
.25 1.25  12 .25  2.18417  12 .86812  0.90002
Approximation Methods for First Order Differential Equations 175

and
R4  h  x1  h  y1  R3  
(3.39d)
.25 1.25  .25  2.18417  .90002  1.15657

Then
y (1.5)  y1  16  R1  2 R2  2 R3  R4  
2.18417  16  .68255  2  .86812  2  .90002  1.15657  3.08007
(3.40)

which differs from the exact value of 3.08022 by

0.005%. □

3.3 – Finite Difference Methods

In the context of this text, a finite difference is the

difference between two values of a function, y(x2) –
y(x1) at distinctly different points. By this we mean
that (x2 – x1) and (y2 – y1) are “measurably large”.
This is distinguished from the infinitesimal
differences (differentials) dy and dx.
The finite difference approximations to
derivatives are developed from the Taylor series
expansion for y. Dividing the domain [x0, xN] into N
intervals of equal (measurably large) width h, the
Taylor series for
y(xk+1)  yk+1 is given by
176 Chapter 3

h2
yk 1 ≃ yk  yk  h  y " ... (3.41)
2! k

Euler finite difference approximation

Referring to eq. 3.41, we truncate the Taylor series

to obtain the Taylor sum of order O(h), this becomes

yk 1 ≃ yk  yk  h (3.42a)

from which
yk 1  yk
yk  (3.42b)
h

This is the Euler finite difference approximation

of y  xk  . Then, the Euler finite difference
approximation to a first order differential equation at
xk

yk  f ( xk , yk ) (3.43)

is obtained by replacing y   xk  by its Euler finite

difference to obtain
yk 1  yk
≃ f ( xk , yk ) (3.44a)
h

which we write as
Approximation Methods for First Order Differential Equations 177

yk 1 ≃ yk  hf ( xk , yk ) (3.44b)

Referring to the analysis for various Runge-Kutta

schemes, we see that the Euler finite difference
approximation of yk+1 – yk is the Runge-Kutta
parameter R1 in a multiple iteration Runge-Kutta
approximation.
To determine y(xN), we divide [x0, xN] into several
intervals defined by the set of points {x0, x1, …, xN}.
We then set k = 0, 1, …, N–1 in eq. 3.44b to obtain

y1 ≃ y0  hf ( x0 , y0 ) (3.45a)

y2 ≃ y1  hf ( x1 , y1 ) (3.45b)

y3 ≃ y2  hf ( x2 , y2 ) (3.45c)

i
i

y N ≃ yN 1  hf ( xN 1 , y N 1 ) (3.45d)

Example 3.5 – Two step Euler finite difference

solution

To approximate the solution to the differential

equation
178 Chapter 3

y   xy (3.10)

subject to y(1) = e1/2 = 1.64872, at x = 1.5, we take h

= 0.25 to obtain

y1  y(1.25)  y0  hx0 y0  e1/ 2  0.25  1  e1 / 2  2.06090 (3.46a)

Then
y2  y (1.5)  y1  hx1 y1  2.06090  0.25  1.25  2.06090  2.70493
(3.46b)

which is a highly inaccurate approximation,

differing from the exact value of 3.08022 by
12.2%. □

Improved Euler finite difference approximation

If we could integrate

y   f ( x, y ) (2.1b)

over the interval [xk, xk+1], we would obtain exact

value of y(xk+1)  yk+1 as

f  x, y( x ) dx
xk 1
yk 1  yk   (3.47)
xk

But since y(x) in the argument of f is not known, the

x dependence of the integrand is not known.
Approximation Methods for First Order Differential Equations 179

Therefore, this integral cannot be determined

exactly.
To approximate the integral from xk to xk+1, we fit
f(x, y) to a polynomial such that the value of f(x, y) at
x = xk is fk  f(xk, yk) and at x = xk+1, f(x, y) is fk+1 
f(xk+1, yk+1). Since (xk, fk) and (xk+1, fk+1) are only two
points used to define this polynomial, we can only
approximate f(x, y) by a straight line. Thus, for xk 
x  xk+1, we take

f ( x, y )   x   (3.48)

forming a trapezoid shown in fig. 3.1.

Figure 3.1
Area under a trapezoid

This method of approximating a function is called

polynomial interpolation (over 2 points in this
discussion). With this interpolated form of f(x, y),
180 Chapter 3

the approximation to the integral in eq. 3.47 is the

area under the trapezoid (the shaded area in figure
3.1). This is the area of the rectangle of width h and
height fk plus the area of the triangle of base xk+1 – xk
= h and height fk+1 – fk. Thus,

f  x, y( x ) dx ≃ Atrapezoid  hf k  12 h  f k 1  f k   12 h  f k  f k 1 
xk 1
 xk

(3.49)

This result is also obtained by integrating

f  x , y ( x ) dx ≃   x   dx  Atrapezoid
xk 1 xk 1
 xk xk
(3.50)

with the values of  and  found from

f k   xk   (3.51a)

and

f k 1   xk 1   (3.51b)

from which
f k 1  f k f f
  k 1 k (3.52a)
xk 1  xk h

and
f k xk 1  f k 1 xk f x f x
  k k 1 k 1 k (3.52b)
xk 1  xk h
Approximation Methods for First Order Differential Equations 181

With eq. 3.49, the finite difference approximation

to eq. 3.47 is

yk 1  yk  12 h  f k  f k 1   yk  12 h  f ( xk , yk )  f ( xk 1 , yk 1 ) (3.53)

Since yk+1 in f(xk+1, yk+1) is not known, we must

approximate it in some reasonable way. For
example, if we estimate this value of yk+1 by the
Taylor expansion of order O(h), we write

yk 1  y ( xk  h)  yk  hyk  yk  hf k (3.54)

Then, yk+1 is approximated by

yk 1 ≃ yk  12 h  f  xk , yk   f  xk 1 , yk  hf k   (3.55)

Eq. 3.55 is known as the improved Euler finite

difference approximation. We see from eqs. 3.225
and 3.26 that this is identical to the second order
Runge-Kutta approximation called the Heun method.

Example 3.6 – Improved Euler finite difference

solution

We again consider

y   xy (3.10)
182 Chapter 3

with y(1) = e1/2 = 1.64872. With h = 0.25, eq. 3.55

becomes

y1  y0  12 h  f  x0 , y0   f  x1 , y0  hf0   
y0  12 h  x0 y0  x1  y0  hx0 y0    (3.56a)
 
e1/ 2  12 (.25)  e1 / 2  1.25 e1/ 2  .25e1 / 2   2.17683

Then,

y2  y1  12 h  x1 y1  x2  y1  hx1 y1   
.25
2.17683  1.25  2.17683  1.5  2.17683  .25  1.25  2.17683  
2 
3.05266
(3.56b)

This result is within 0.9% of the exact value, which

is a significant improvement over the approximation
found in example 3.5. □

(3.184b)

Problems
Differential Initial Exact
equation condition Solution

I y   3x2 y y (1)  e y( x)  e x
3

II y   xy y (1)  e1/ 2 y( x)  e x
2
/2
Approximation Methods for First Order Differential Equations 183

III y  3 x y y (1)  e 2
3/2
y( x)  e2 x

IV 1 y (1)  e1/ 2 y( x)  e x/2

y  y
4 x

Table 3.7
Differential equations and their solutions
for problems 3 – 20, 25 – 29 and 32

1. y(x) satisfies the differential equation

y '  e xy with y ( 0)  1

Find the Taylor sum for y(x), expanded around x = 0, of order

(a) 2 (b) 3

Estimate the value of y(0.5) to 3 decimal places using each of these

Taylor sums.

2. For the differential equation of problem 1, find an approximation to

y(0.3) using a two step
(a) Taylor sum of order 2
(b) Classic 4th order Runge-Kutta method
(c) Euler finite difference method
(d) Improved Euler finite difference method

Each of the problems below (3 through 26 and 28), will refer to one or more
of the differential equations in table 3.7 by Roman numeral. Since the exact
solution to each differential equation is given in the table, the reader should
determine the accuracy of the result in each problem in terms of a %Error as
defined in eq. 3.123.

3. For differential equation I, approximate the value of y(1.5) using a

(a) single step Taylor sum
(b) (b) two step Taylor sum about x = 1, of order (x–1)2.

4. For differential equation IV, approximate the value of y(1.5) using a

(a) single step Taylor sum
(b) two step Taylor sum about x = 1, of order (x–1)2.
184 Chapter 3

5. For differential equation III, use a multi-step Taylor sum of order O(h2),
with h = 0.25, to estimate the value of y(1.75).

6. If y(x) satisfies differential equation III, find the approximate value of

y(1.50) using a single step of
(a) the second order Heun method.
(b) the second order midpoint Runge-Kutta method.
(c) the second order Ralston Runge-Kutta method.
(d) the fourth order classical Runge-Kutta method.

7. If y(x) satisfies differential equation III, find the approximate value of

y(1.50) using a two step
(a) Heun second order method.
(b) fourth order classical Runge-Kutta method.

8. For differential equation II, find the approximate value of y(1.50) using
(a) a single step classical fourth order Runge-Kutta method
(b) a two step classical fourth order Runge-Kutta method

9. For differential equation IV, find an approximate value of y(1.50) using

(a) a single step Euler finite difference approximation.
(b) a two step Euler finite difference approximation.

10. For differential equation III, find an approximate value of y(1.30) using
(a) a single step Euler finite difference approximation.
(b) a three step Euler finite difference approximation.

11. For differential equation IV, find an approximate value of y(1.50) using
(a) a single step improved Euler finite difference method.
(b) a two step improved Euler finite difference method.

12. For differential equation III, find an approximate value of y(1.30) using
(a) a single step improved Euler finite difference method.
(b) a two step improved Euler finite difference method.

13. For differential equation III, find an approximate value of y(1.30) using
(a) a two step Euler finite difference method.
(b) a two step improved Euler finite difference method.