Numerical Methods for Differential Equations

Differential equations frequently occur in

mathematical models that arise in many branches of
engineering. When the differential equations cannot be
solved readily solved analytically, it is convenient to
solve them numerically. Nowadays, this can be usually
be achieved very inexpensively to high accuracy and
with a reliable bound on the error between the analytical
solution and its numerical approximation. In this section,
we present some numerical methods used for solving the
system of ODEs.

A. Standard Form of ODE

Almost common system of ODEs can be reformed
in the standard form as
y  f (t , y ) (1)
where
 y1 (t ) 
 
y     are the dependent variable vector,
 y (t ) 
 n 
 f1 (t , y ) 
 
f (t , y )     are the gradient vector,
 f (t , y ) 
 n 
and t is the independent variable.
A more general form as provided by MATLAB is
M (t , y ) y  f (t , y ) (2)
where M (t , y ) is a nonsingular mass matrix.

With either form, we must formulate the ODEs a system

of first order equations. The usual way to do this is to
introduce new independent variables and transform the
original ODE problem to the equivalent system of the
first order ODEs (state-space form).

Example 1.
Numerically solve the Kepler’s equations describing the
motion of one body around another of equal mass
located at the origin under the influence of gravity.
u v
u   , v  
r3 r3

where r  u 2  v 2 and u, v are the coordinates of the

moving body relative to another body fixed at the origin.
With initial values

1 e
u (0)  1  e, u(0)  0, v (0)  0, v(0) 
1 e

To numerically solve above equations, the equivalent

first order ODEs must be formulated. One choice is to
introduce variables y1  u and y2  v for the unknowns
and introduce y3  u and y4  v for their first derivatives.
Thus the first order system becomes
y  f (t , y )

 y3 

 y1   
 y   y 4 
 2   y 
or    1 3
 y3   r 
 y4   y2 
 r 3 

where r  y12  y22

and the initial conditions are

1 e
y1 (0)  1  e, y2 (0)  0, y3 (0)  0, y4 (0) 
1 e
B. Uniqueness of Solutions
There are many physical problems that do not have
solutions for certain values of parameters. Also some
may have more than one solution. Clearly we will have
trouble computing a solution that does not exist, and if
there is more than one solution then we will have trouble
computing the right one (the desired solution).

Example 2.
(y) 2  1  0
 No solution

Example 3.
Consider a heavy mass pendulum system with light
and rigid rod hanging vertically from a friction-less
pivot. The angle that the pendulum makes with the
vertical satisfies the ODE

   sin( )  0
Suppose that the pendulum is hanging vertically so that
the initial angle  (0)  0 and that we thump the mass to
give an initial velocity  (0) . If the velocity is small, the
pendulum will swing back and forth. If the velocity is
large, the pendulum will swing over the top and it will
whirl around the pivot forever. There is a very special
solution that occurs as the solutions change from
oscillatory to increasing. Physically, it corresponds to an
initial velocity that causes the pendulum to approach and
then come to rest vertically and upside down. Clearly,
this solution is unstable: an arbitrary small change to the
initial velocity gives rise to a solution that is very
different. In other words, this situation is ill-condition.

 (0)  2.5

 (0)  1.5

Time (s)
Example 4.
Consider the ODE
y  y  cos(t )
One may plot the direction (slope) field as
C. Computational Errors
Digital systems like calculators and computers
hardly make a mistake, since they follow the
programmed order faithfully. Nonetheless, we often
encounter some numerical errors in the computing results
made by digital systems, mostly coming from
representing the numbers in finite bits, which is an
intrinsic limitation of digital world. If you let the
computer compute something without considering what
is called the finite-word-length effect, you might obtain a
weird answer. In this section we will see how the
computer represents and stores the numbers. Then we
think about the cause of computational error in order not
to be deceived by unintentional mistakes of the computer
and, it is hoped, to be able to take some measures against
them.
 MATLAB uses the IEEE 64-bit floating-point
number system to represent all numbers. It has a word
structure consisting of the sign bit, the exponent field,
and the mantissa field as follows:

Each of these fields expresses S,E, and M of a number in

the way described below.

Sign bit
S = b63 = 0 for positive numbers
1 for negative numbers

Exponent field

E = (b62b61b60 ・ ・ ・ b52):
adopting the excess 1023 code
E = Exp − 1023 = {0, 1, . . . , 211 − 1 = 2047} − 1023
= {−1023,−1022, . . . ,+1023,+1024}

Mantissa field
M = (b51b50 . . . b1b0):

This equivalent to about 16 digits decimal resolution.

Some MATLAB examples:

Example 5.
>> (3/2-1)*2-1
ans =
0
>> (4/3-1)*3-1
ans =
-2.2204e-016

Example 6.
Solving the two linear algebraic equations
17x1 + 5x2 = 22 and 1.7x1 + 0.5x2 = 2.2

>> A=[17 5; 1.7 0.5]; b=[22; 2.2]; A\b

Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 3.265362e-018.
ans =
-1.0588
8.0000
1. Numerical Techniques using One-Step Methods

One-Step methods use only data in current step,

tn , y (tn ), f (tn , y (tn )) , to estimate the next step solution
y (tn  h) where h  t .

1.1 Euler’s Method

When talking about the numerical solutions to ODEs,
everyone starts with the (explicit) Euler’s method, since
it is easy to understand and simple to program. Even
though its low accuracy keeps it from being widely used
for solving ODEs, it gives us a basic concept of
numerical solution for a differential equation simply and
clearly.
Let’s consider a first-order differential equation:
y  f (t , y ) (3)
with initial condition
y (t0 )  y 0 (4)
Consider the derivative of the function y(t)
dy
 f (t , y )
dt (5)
This derivative can be expressed by
dy y (t  h )  y (t )
 lim
dt h 0 h (6)
Thus for small h
dy y  yn
 n 1
dt t tn h
(7)
This can be arranged to give the Euler’s method
estimation.
y n 1  y n  h  f (tn , y n ) (8)

Example 7. Numerically solve the differential equation

dy
t
dt
for y(10) subject to the initial condition that y(0) = 0 with
h = 0.1.
Employing the Euler’s method, we can write
yn 1  yn  htn
since tn  nh , the above equation becomes

yn 1  yn  nh 2
Writing out the first few equations
y1  y0  0

y2  y1  h 2  y0  h 2

y3  y2  2h 2  y0  3h 2
…
 The Euler’s method leads to solution
n(n  1)h 2
y n  y0 
2
n 2h2
(compare with the exact solution yn  y0  )
2
For the given step size h = 0.1,
y(10) occurs at n = 10/0.1 = 100 thus
100  99  0.12
y (10)  y100 0 = 49.5
2
(comparing with exact solution = 50.0)

50

45

40

35

30

25

20

15

10

0
0 1 2 3 4 5 6 7 8 9 10
Example 8. Consider the problem (Ref. 1)
t 4  6t 3  12t 2  14t  9
y  y 
2

(1  t ) 2
with y(0) = 2
(1  t )(2  t )
(The exact solution is y (t )  )
1 t

The exact solution (solid line)

Three set of results from Euler’s method:
with the step sizes h = 0.2 (8 steps)
h = 0.1 (16 steps)
h = 0.05 (32 steps)
1.2 Implicit Euler’s Method
A similar method to eq.(8) is to use the backward
estimation as
y n  y n 1  h  f (tn 1 , y n 1 ) (9)
This leads to an implicit Euler’s method as
y n 1  y n  h  f (tn 1 , y n 1 ) (10)
Unlike the explicit Euler in Eq.(8), the implicit Euler’s
formula Eq.(10) contain the next step data in the right-
hand side and make the computation more complicate.
To solve this kind of equations, we commonly deal with
solving the nonlinear algebraic equations at every step
but not usually a very large increase. (The Newton’s
method is often employed.)

Example 9. Numerically solve the following system of

ODEs using explicit and implicit Euler’s methods with
the step size of h = 0.004.
 y1 (t )   8003 1999   y1 (t ) 
    y (t ) 
y  (
 2   t ) 23988 6004  2 
subject to the initial conditions
1 
y (0)   
4
The numerical solutions using 12 steps of size h = 0.004
are shown in the Table.

Since the problem is called a stiff system (eigenvalues

are -7 and -14,000), the implicit Euler’s method yields
more accurate solution.
1.3 Heun’s Method (Trapezoidal method)
An implicit one-step method with second-order is the
Heun’s method which consider the solving of Eq.(1) by
integration of both sides of the equation.
tn 1

 f (t , y )dt
tn 1
y (t ) t 
n
tn
(11)
tn 1

y (tn 1 )  y (tn )   f (t , y )dt

tn
with y (t0 )  y 0 (12)

If we assume that the value of the (derivative) function

f (t , y ) is linear within the time step [tk, tk+1), it becomes
h
y n 1  y n  [f (tn , y n )  f (tn 1 , y n 1 )] (13)
2
But the right-hand side of the equation has yn+1, which is
unknown at step tn (implicit method). To resolve this
problem, one technique is to use the explicit Euler
formula to estimate yn+1 as
y n 1  y n  hf (tn , y n ) (14)
Then Eq.(13) reads
h
y n 1  y n  [f (tn , y n )  f (tn 1 , y n  hf (tn , y n ))] (15)
2
This is Heun’s method. It is a kind of predictor-and-
corrector method in that it predicts the value of yn+1 by
Eq. (14) at tn and then corrects the predicted value by Eq.
(15) at tn+1. The truncation error of Heun’s method is
O(h2) (proportional to h2), while the error of Euler’s
method is O(h).

Example 10.
Consider the same problem as in Example 8.
t 4  6t 3  12t 2  14t  9
y  y 
2

(1  t ) 2
with y(0) = 2
(1  t )(2  t )
(The exact solution is y (t )  )
1 t

Heun’s Method:
with step size h = 0.4 (4 steps)
= 0.2 (8steps)

t
1.4 Runge-Kutta Method (Trapezoidal method)
Although Heun’s method is a little better than the
Euler’s method, it is still not accurate enough for most
real world problems. The Runge-Kutta method can have
higher order of truncation error is one of the most widely
used methods for solving differential equations. Its
algorithm is described below.
Considering the second order Runge-Kutta method,
the algorithm is expressed by
y n 1  y n  a1k 1  a2k 2 (16)
where k 1  hf (tn , y n )
k 2  hf (tn  b1h, y n  b2k 1 )

Expand the yn+1 using the Taylor series, one gets

h2
y n 1  y n  hf (tn , y n )  f (tn , y n )   (17)
2
From the chain rule, one can write
df i f i f i
  fi (18)
dt t yi
So that

h2  f f 
y n 1  y n  hf (tn , y n )   t  y f    (19)
2  
Recall Eq.(16)
y n 1  y n  a1hf (tn , y n )  a2 hf (tn  b1h, y n  b2 hf (tn , y n ))
(20)
with the last term can be expanded as
f
f (tn  b1h, y n  b2 hf (tn , y n ))  f (tn , y n )  b1h
t tn , y n

f
b2 hf (tn , y n ) (21)
y tn , y n

By substitution and comparison of Eq.(19) and (21), one

finds
f
y n 1  y n  a1hf (tn , y n )  a2 hf (tn , y n )  a2b1h 2
t tn , yn

f
 a2b2 h 2f (tn , y n )
y tn , yn

with term hf  a1  a2 = 1

f
with term h2  b1a2 = 0.5
t tn , y n

f
with term h 2f (tn , y n )  b2 a2 = 0.5
y tn ,y n
There is one free unknown, one may set
a1  a2  0.5 and b1  b2  1 (22)
Therefore the second order Runge-Kutta algorithm
becomes
h
y n 1  y n  f1  f2  (23)
2
where f1  f (tn , y n )
f 2  f (tn 1 , y n  hf (tn , y n ))
This is, in fact, equivalent to Heun’s method. Another
popular algorithms in the second order Runge-Kutta is
known as the modified Euler method (when b1=0.5).

A more accurate and widely used Runge-Kutta algorithm

is the forth order which can be derived in similar fashion
as second order and can be represented by
h
y n 1  y n  f1  2f2  2f3  f4  (24)
6
where
f1  f (tn , y n )
f 2  f (tn  h / 2, y n  f 1 h / 2)
f3  f (tn  h / 2, y n  f 2 h / 2)
f 4  f (tn 1 , y n  f 3 h)
Example 11.
Numerically solve the differential equation
y  y  1
with initial condition y(0) = 0 using step size h = 0.5
2. Numerical Techniques using Multi-Step Methods

Multi-Step methods use not only data in current step, tn,

yn, fn, but also data in past steps, tn-1, yn-1, fn-1, tn-2, yn-2, fn-
2, etc., to estimate the next step solution yn+1. There are
many multi-step methods available but the following will
list two popular methods.

2.1 Adams-Bashforth Method

Adams-Bashforth method is an example of an explicit
linear four-step method:
h
y n 1  y n  (55f n  59fn 1  37f n  2  9f n 3 )
24

2.2 Adams-Moulton Method

Adams-Moulton method is an example of an implicit
linear three-step method:
h
y n 1  y n  (9f n 1  19f n  5f n 1  f n  2 )
24
3. Predictor-Corrector Methods

Predictor-Corrector methods use the two different one-

step or multi-step methods to predict the next step
solutions then compute the error and make the correction
on the obtained solution. Again, there are many available
predictor-corrector methods but the following will list
two popular ones.

3.1 Adams-Bashforth-Moulton Method

Adams-Bashforth-Moulton method use explicit linear
four-step method for predictor and use modified implicit
linear three-step method for corrector as:
(with modification formulas)

h
Predictor  p n 1  y n  (55f n  59f n 1  37fn  2  9f n 3 )
24
251
Modifier  m n 1  p n 1  (c n  p n )
270
Corrector 
h
c n 1  y n  (9f (tn 1 , m n 1 )  19fn  5f n 1  9f n  2 )
24
and
19
y n 1  c n 1  (cn 1  p n 1 )
270
3.2 Hamming Method
Hamming method use explicit linear four-step method
for predictor and use modified implicit linear three-step
method for corrector (but different from Adams-
Bashforth-Moulton method) as:
(with modification formulas)

4h
Predictor  p n 1  y n 3  (2f n  f n 1  2f n  2 )
3
112
Modifier  m n1  p n 1  (c n  p n )
121
Corrector 
1
c n 1  [(9y n  y n  2  3h(f (tn 1 , m n 1 )  2f n  f n 1 )]
8
and
9
y n 1  c n 1  (c n 1  p n 1 )
121
4. MATLAB Application

Syntax
[T,Y] = solver (odefun, tspan, y0)
[T,Y] = solver (odefun, tspan, y0, options)
[T,Y,TE,YE,IE] = solver (odefun, tspan, y0, options)
sol = solver (odefun,[t0 tf], y0 ...)

where solver is one of [ode45, ode23, ode113, ode15s,

ode23s, ode23t, or ode23tb]
Arguments
The following table describes the input arguments to the
solvers.
- odefun
A function handle that evaluates the right side of the
differential equations. All solvers solve systems of
equations in the form or problems that involve a mass
matrix. The ode23s solver can solve only equations with
constant mass matrices. ode15s and ode23t can solve
problems with a mass matrix that is singular, i.e.,
differential-algebraic equations (DAEs).
 - tspan
A vector specifying the interval of integration, [t0 tf].
The solver imposes the initial conditions at t0 and
integrates from t0 to tf.
To obtain solutions at specific times (all increasing or all
decreasing), use tspan = [t0, t1 ,..., tf].
For tspan vectors with two elements [t0 tf], the solver
returns the solution evaluated at every integration step.
For tspan vectors with more than two elements, the solver
returns solutions evaluated at the given time points. The
time values must be in order, either increasing or
decreasing.
Specifying tspan with more than two elements does not
affect the internal time steps that the solver uses to
traverse the interval from [t0 tf]. All solvers in the ODE
suite obtain output values by means of continuous
extensions of the basic formulas. Although a solver does
not necessarily step precisely to a time point specified in
tspan, the solutions produced at the specified time points
are of the same order of accuracy as the solutions
computed at the internal time points.
Specifying tspan with more than two elements has little
effect on the efficiency of computation, but for large
systems, affects memory management.
 - y0
A vector of initial conditions.

- options
Structure of optional parameters that change the default
integration properties. This is the fourth input argument.
[T, Y] = solver (odefun, tspan, y0, options)
You can create options using the odeset function.

T  Column vector of time points

Y  Solution array. Each row in y corresponds to the
solution at a time returned in the corresponding row of t.
Nonstiff Solvers

ode45  a pair of one-step explicit Runge-Kutta

(Dormand-Prince pair)
ode23  a pair of one-step explicit Runge-Kutta
(Bogacki and Shampine pair)
ode113 a variable order Adams-Bashforth-Moulton

Stiff Solvers

ode15s  backward differentiation formulas (muli-step)

(Gear's method)
Ode23s  Runge-Kutta with modified Rosenbrock

Ode23t  Runge-Kutta (Trapezoidal)

In general, if the stiffness of the problem is not provided,

it is recommended to start with ode45 then comparing
with ode15s and other kinds of solvers to guarantee the
highest accuracy of the solutions.
Example 12.
Determine the top floor motion of a given three-story
building subjected to the ground motion of
xg (t )  0.01sin(20t ) . Assume the building is started from
rest and floor stiffness k = 2.0 MN/m.
m  100 ton
x3
3.0 2k
m  100 ton
x2
3.0 2k m  100 ton
x1
4.0 1.5k

xg

From Newton’s law, the building motion is described by

the second order differential equations as
Mx(t )  Kx(t )  M 1 xg (t )

becomes
1 0 0   x1  7 4 0   x1  1
0 1 0  105  x    4 8 4   106  x    1  105 x
   2    2  g
     
0 0 1   x3  0 4 4   x3  1
Write the first-order differential equations using state-
space form using
 y1   x1 
 y   x 
 2  2
 y   x 
y   3   3
 y4   x1 
 y5   x2 
   
 y6   x3 
get
0 M 1K  1
y    y    xg
I 0  0
or

 y1  0 0 0 70 40 0   y1  1 
 y  0 0 0 40 80 40   y2  1 
 2     
 y3  0 0 0 0 40 40   y3  1 
          4sin(20t )
 y4   1 0 0 0 0 0   y4   0 
 y5  0 1 0 0 0 0   y5  0
      
 y6  0 0 1 0 0 0   y6   0 
with initial conditions @ t=0 (from rest)
 y1  0 
 y  0 
 2  
 y3  0 
  
 y 4  0 
 y5  0
   
 y6  0 
Implement into MATLAB and use ode45 to numerically
solve above problem from t=0 to t=60 s
MATLAB M-File
clear
global A B
M=eye(3)*1e5;
K=[7 -4 0; -4 8 -4; 0 -4 4]*1e6;
A=[zeros(3) -inv(M)*K; eye(3) zeros(3)];
B=[4 4 4 0 0 0]';
t1=cputime;
[T,Y]=ode45(@building3,[0 60],zeros(1,6));
tcom=cputime-t1; disp(tcom)
disp(max(Y(:,6))
plot(T,Y(:,6));
title('Top Floor Movement')
ylabel('Displacement (m)');
xlabel('Time (s)')

Function Building3 (use to compute y )

function dy = building3(t,y)
global A B
dy=zeros(6,1);
dy=A*y+B*sin(20*t);
  we obtain the solution as followings:
max roof displacement x3 = 0.1112 m
computation time = 1.27 s

Top Floor Movement

0.15

0.1

0.05
Displacement (m)

-0.05

-0.1

-0.15

-0.2
0 10 20 30 40 50 60
Time (s)
 Beside the ability of solving the ODE by numerical
techniques. The MATLAB can be used to analytically
solve the ODE as well. The following will briefly
demonstrate this powerful tool.

Solve the following ODEs:

1) y' + y - 2 = 0

2
x /2
2) y' + xy = e

1
3) y   y  3 x 2 3
y
x

4) y   4 y  8 x  2 x
2

5) y   2 y   3 y  4e
2x
Example of MATLAB M-File

clear
DE='D2y+2*Dy-3*y=4*exp(2*t)'; %INPUT OF EQ.
t1=cputime;
y=dsolve(DE);
t2=cputime;
pretty(y); disp(t2-t1)

Results (y = f(t)) on Core 2 Duo @ 2.20 GHz

1) (2*exp(t)+C1)/exp(t) ; 0.05s

2) (t+C1)*exp(-1/2*t^2); 0.11s
3) [ 1/(-6*t+C1)^(1/2)/t]
[ -1/(-6*t+C1)^(1/2)/t] ; 0.09s
4) -1+1/2*t-2*t^2+C1*exp(2*t)+C2*exp(-2*t)
0.14s
5) -1/4*exp(t)+9/20*exp(-3*t)+4/5*exp(2*t);
0.18s
Plot of the symbolic function --> ezplot(y)

5 -1/4 exp(t)+9/20 exp(-3 t)+4/5 exp(2 t)

x 10

10

-6 -4 -2 0 2 4 6
t
References
1 An Introduction to Numerical Analysis, Suli and
Mayers, Cambridge University Press (2003)
2 Solving odes with MATLAB, Shampine, Gladwell
and Thomson, Cambridge University Press (2003)
3 An Introduction to Ordinary Differential Equations,
Jame Robinson, Cambridge University Press (2004)
4 An Introduction to Programming and Numerical
Methods in MATLAB, Otto and Denier, Springer
(2005)
5 Applied numerical methods using MATLAB, Yang,
Cao and Chung, Wiley (2005)