NCTU Department of Electrical and Computer Engineering 2015 Spring Course

<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

4. Runge-Kutta Formula For Differential Equations

To solve the differential equations numerically, the most useful formula is called
Runge-Kutta formula which has been widely used in numerical analysis.

For a dynamic system without input, it is generally expressed as the following

first-order differential equation:
x& (t ) = f (t , x(t )) , x(t0)=α (4-1)

where t=t0 is the initial time and x(t0)=α is the initial condition. The problem to solve
x(t) in (4-1) for t>t0 is called the initial value problem, or IVP in brief. For example,
the following equation
x& (t ) = − x(t ) + t , x(0)=1 (4-2)
is an IVP and its solution can be obtained in closed form as below:
x(t ) = 2e − t + t − 1 , t≥0 (4-3)
However, if (4-1) is more complicated, such as
x& (t ) = − x 2 (t ) + t ⋅ sin(t ) , x(0)=1 (4-4)
then it is impossible to find the solution x(t) in closed form. Hence, it is required to
solve (4-4) in a numerical method.

The simplest numerical method is called the Euler formula, which was propsed
by Euler in 1768. With the use of fixed grid size ∆t=h, the grid points along t are
denoted as t0, t1, t2,… in order, where
ti = t0 + i ⋅ h , i=1,2,3,…… (4-5)
and the value x(t) at ti is defined as
x i = x(t i ) , i=0,1,2,3,…… (4-6)

where the initial value x0=x(t0)=α is known. According to the definition of derivative,
we have
x(t + ∆t ) − x(t )
x& (t ) = lim , (4-7)
∆t →0 ∆t
which implies the derivative of x(t) at t=ti can be approximately expressed as
x(t i + ∆t ) − x(t i ) x(t i +1 ) − x(t i ) x i +1 − x i
x& (t i ) ≈ = = (4-8)
∆t h h
Substituting (4-8) into (4-1) at t=ti leads to

4-1
 NCTU Department of Electrical and Computer Engineering 2015 Spring Course
<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

xi +1 − xi
≈ f (t i , xi ) , x0=α (4-9)
h
and then
x i +1 = x i + h ⋅ f (t i , xi ) , x0=α (4-10)

which is the famous Euler formula. Clearly, if we try to achieve a solution more
precisely, the grid size h should be chosen as small as possible. However, reducing the
grid size h is inefficient since the cost of calculation time may increase tremendously.

Now, let’s introduce Taylor’s expansion to explain the error caused by Euler
formula (4-10). According to Taylor’s expansion, a function x(t) continuous at t=ti can
be expressed as
x(t ) = a0 + a1 (t − t i ) + a 2 (t − t i ) + L + a n (t − t i ) + L
2 n
(4-11)
whose higher order derivatives are
x& (t ) = a1 + 2a 2 (t − t i ) + 3a3 (t − t i ) + L + na n (t − t i )
n −1
+L
2
(4-12)

&x&(t ) = 2! a 2 + 3 ⋅ 2a3 (t − t i ) + 4 ⋅ 3a 4 (t − t i ) + L
2

(4-13)
+ L + n(n − 1)a n (t − t i )
n −2
+L

&x&&(t ) = 3! a3 + 4 ⋅ 3 ⋅ 2a 4 (t − t i ) + 5 ⋅ 4 ⋅ 3a5 (t − t i ) + L
2

(4-14)
+ L + n(n − 1)(n − 2 )a n (t − t i )
n −3
+L

From the above equations, we have a 0 = x(t i ) and a k = x (t i ) for k=1,2,…,∞.

1 (k )
k!
Thus, (4-11) can be rewritten as
&x&(t i ) x (n ) (t i )
x(t ) = x(t i ) + x& (t i )(t − t i ) + (t − t i ) + L + (t − t i )n + L
2
(4-15)
2! n!
Let t=ti+1, we have
&x&(t i )
x(t i +1 ) = x(t i ) + x& (t i )(t i +1 − t i ) + (t i +1 − t i )2 + L
2!
(4-16)
x (n ) (t i )
+L+ (t i +1 − t i ) + L
n

n!
i.e.,
f ′(t i , xi ) 2 f (n −1) (t i , xi ) n
xi +1 = xi + f (t i , xi )h + h +L+ h +L
2! n! (4-17)
= xi + f (t i , xi )h + O h 2 ( )
where

4-2
 NCTU Department of Electrical and Computer Engineering 2015 Spring Course
<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

f ′(t i , xi ) 2 f (n −1) (t i , xi ) n
O h2 =( ) 2!
h +L+
n!
h +L (4-18)

Comparing (4-17) with (4-10), we know that the term O h 2 ( ) is the error of Euler
formula, which is proportional to h2.

In order to reduce the error to h3, we further modify Euler form (4-10) as below:
x i +1 − x i
= β 0 f (t i , x i ) + β 1 f (t i + γh, x i + δh ) , (4-19)
h
where β0, β1, γ and δ are variables to be determined. Since f (t , x(t )) depends on x(t)
and t, its Taylor’s expansion at x(t)=xi and t=ti can be expressed as
f (t , y (t )) = a 0 + a t (t − t i ) + a x ( x(t ) − xi ) + a tt (t − t i ) + a tx ( x(t ) − xi )(t − t i )
2

+ a xx ( x(t ) − xi ) + a ttt (t − t i ) + a ttx (t − t i ) ( x(t ) − xi )

2 3 2

+ atxx (t − t i )( x(t ) − xi ) + a xxx ( x(t ) − xi ) + L

2 3
(4-20)

where all the coefficients a0, at, ax, att, atx … are constant. The partial derivatives of
f (t , x(t )) are
∂f (t , x(t ))
f t (t , x(t )) =
∂t
= a t + 2att (t − t i ) + a tx (x(t ) − xi ) + 3attt (t − t i )
2
(4-21)
+ 2attx (t − t i )(x(t ) − xi ) + a txx ( x(t ) − xi ) + L
2

∂f (t , x(t ))
f x (t , x(t )) =
∂x(t )
= a x + a tx (t − t i ) + 2a xx (x(t ) − xi ) + a ttx (t − t i )
2
(4-22)
+ 2a txx (t − t i )(x(t ) − xi ) + 3a xxx (x(t ) − xi ) + L
2

∂ 2 f (t , x(t )) ∂f t (t , x(t ))
f tt (t , y (t )) = =
∂t 2 ∂t (4-23)
= 2att + 3 ⋅ 2attt (t − t i ) + 2a ttx (x(t ) − xi ) + L

∂ 2 f (t , x(t )) ∂f x (t , x(t ))
f tx (t , y (t )) = =
∂x(t )∂t ∂t (4-24)
= a tx + 2attx (t − t i ) + 2atxx ( x(t ) − xi ) + L

∂ 2 f (t , x(t )) ∂f x (t , x(t ))
f xx (t , x(t )) = =
(∂x(t )) 2
∂x(t ) (4-25)
= 2a xx + 2atxx (t − t i ) + 3 ⋅ 2a xxx ( x(t ) − xi ) + L
Clearly, all the coefficients can be derived from the above partial derivatives and
expressed as

4-3
 NCTU Department of Electrical and Computer Engineering 2015 Spring Course
<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

a 0 = f (t i , xi ) , a t = f t (t i , xi ) , a x = f x (t i , xi ) ,

f tt (t i , xi ) , a tx = f tx (t i , xi ) , a xx = f xx (t i , xi ) , ….
1 1
a tt =
2! 2!
Substituting them into (4-20) yields

f (t , x(t )) = f (t i , xi ) + f t (t i , xi )(t − t i ) + f x (t i , xi )( x(t ) − xi ) + f tt (t i , xi )(t − t i )

1 2

2!
+ f tx (t i , xi )(t − t i )( x(t ) − xi ) + f xx (t i , xi )( x(t ) − xi )
1 2

2!
(4-26)
+ f ttt (t i , xi )(t − t i ) + f xxx (t i , xi )( x(t ) − xi ) + L
1 3 1 3

3! 3!
Let t=ti+γh and x(t)=xi+δh, then
f (t i + γh , x i + δh )

= f (t i , x i ) + hγf t (t i , x i ) + hδf x (t i , x i ) + h γ f tt (t i , x i )
1 2 2
(4-27)
2
+ h 2 γδf tx (t i , x i ) + h δ f xx (t i , x i ) + O h 3
1 2 2
2
( )
Hence, (4-19) can be rearranged and rewritten as
h2
xi +1 = xi + h[(β 0 + β 1 ) f (t i , xi )] + [2 β 1γf t (t i , xi ) + 2 β1δf x (t i , xi )]
2!

[ ]
3
3β 1γ 2 f tt (t i , xi ) + 6 β 1γδf tx (t i , xi ) + 3β 1δ 2 f xx (t i , xi )
h
+ (4-28)
3!
+ O h4( )
If we want to take the precision to h2, from (4-17) and (4-28) we have
f ′(t i , xi ) 2
xi +1 ≈ xi + f (t i , xi )h +
2!
h + O h3 ( )
≈ xi + h[(β 0 + β 1 ) f (t i , xi )] (4-29)

+
h2
2!
[2β1γf t (t i , xi ) + 2β1δf x (t i , xi )] + O h 3 ( )
Since x& (t i ) = f (t i , x i ) , the term f ′(t i , xi ) can be changed into the following form:

f ′(t i , xi ) = f t (t i , xi ) + f x (t i , xi )x& (t i )
(4-30)
= f t (t i , xi ) + f x (t i , xi ) f (t i , xi )
As a result, from (4-29) and (4-30) we obtain
h2
f (t i , xi )h + ( f t (t i , xi ) + f x (t i , xi ) f (t i , xi ))
2!
(4-31)
h2
= h[(β 0 + β 1 ) f (t i , xi )] + [2 β 1γf t (t i , xi ) + 2 β 1δf x (t i , xi )]
2!

4-4
 NCTU Department of Electrical and Computer Engineering 2015 Spring Course
<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

which leads to
β 0 + β1 = 1 (4-32)

2 β 1γf t (t i , x i ) + 2 β1δf x (t i , x i ) = f t (t i , x i ) + f x (t i , x i ) f (t i , x i ) (4-33)

From (4-33), wehave
2β1γ = 1 (4-34)
2 β 1δ = f (t i , x i ) (4-35)

Since there are four variables β0, β1, γ and δ in three equations (4-32), (4-34) and
(4-35), the choice of these variables is not unique and commonly they are selected as

β 0 = β 1 = , γ = 1, δ = f (t i , x i )
1
(4-36)
2
That means the numerical solution (4-28) is chosen as

f (t i , x i ) + f (t i + h , x i + hf (t i , xi )) ,
h h
x i +1 = x i + (4-37)
2 2
which is called the second-order Runge-Kutta formula. For the convenience of
programming, the formula is often rearranged as below:

x i +1 = x i +
1
(k 0 + k1 ) (4-38)
2
where
 k 0 = h ⋅ f (t i , xi )
 (4-39)
 k1 = h ⋅ f (t i + h , xi + k 0 )
In case that a higher precision is required, we oftem employ the higher order
Runge-Kutta formula. For example, if a precision of h4 is needed, then the
fourth-order Runge-Kutta formula must be used, which is often given as

x i +1 = x i +
1
(k 0 + 2k1 + 2k 2 + k 3 ) (4-40)
6
where
  k 
f (t i , xi ),
h
 k 0 = h ⋅ k1 = h ⋅ f  t i + , x i + 0 
 2 2
 (4-41)
 k 
k 3 = h ⋅ f (t i , xi + k 2 )
 k2 = h ⋅ h
f  t i + , xi + 1 ,
  2 2

This formula has been widely applied to a lot of applications in engineering due to its
simplicity and acceptable accuracy.

Next, let’s use an example to show the programming of the fourth-order

4-5
 NCTU Department of Electrical and Computer Engineering 2015 Spring Course
<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

Runge-Kutta formula in MATLAB. Consider the following equation:

x& (t ) = f (x(t ), t ) = − x(t ) + 1 , x(0)=0.5 (4-42)
and find the solution x(t) for t≥0, which can be solved as below:
x(t ) = −0.5e − t + 1 (4-43)
Now let’s apply the fourth-order Runge-Kutta formula to verify (4-43) with the step
size h=0.01. The precision is then in the order of h4=10−8. From (4-40), we have

x i +1 = x i +
1
(k 0 + 2k1 + 2k 2 + k 3 ) (4-44)
6
where
k 0 = h ⋅ f (t i , xi ) = h(− xi + 1) = − h( xi − 1)
 h k    k    k 
k1 = h ⋅ f  t i + , xi + 0  = h −  xi + 0  + 1 = − h xi + 0 − 1
 2 2   2   2 
 h k    k    k 
k 2 = h ⋅ f  t i + , xi + 1  = h −  xi + 1  + 1 = − h xi + 1 − 1
 2 2   2   2 
k 3 = h ⋅ f (t i , xi + k 2 ,) = h(− ( xi + k 2 ) + 1) = − h( xi + k 2 − 1)
The programming in MATLAB is given as below:
===================================================
>> % Fourth order Runge-Kutta method
>> x0=0.5; h=0.01; % initial condition x(0)=0.5 and step size 0.01sec
>> k0=-h*(x0-1); k1=-h*(x0+k0/2-1); k2=-h*(x0+k1/2-1); k3=-h*(x0+k2-1);
>> x(1)=x0+(k0+2*k1+2*k2+k3)/6;
>> e(1)=x(1)-(-0.5*exp(-h)+1); % numerical error at t=h
>> t(1)=h;
>> for n=1:599 % total simulation time 6 sec
>> k0=-h*(x(n)-1); k1=-h*(x(n)+k0/2-1);
>> k2=-h*(x(n)+k1/2-1); k3=-h*(x(n)+k2-1);
>> x(n+1)=x(n)+(k0+2*k1+2*k2+k3)/6;
>> t(n+1)=(n+1)*h; % t-axis
>> e(n+1)=x(n+1)-(-0.5*exp(-t(n+1))+1); % numerical error
>> end
>> plot(t,x); xlabel(‘t’); ylabel(‘x(t)’)
1

0.95

0.9

0.85

0.8
x(t)

0.75

0.7

0.65

0.6

0.55

0.5
0 1 2 3 4 5 6
t

4-6
 NCTU Department of Electrical and Computer Engineering 2015 Spring Course
<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

>> plot(t,e); xlabel(‘t’); ylabel(‘e(t)’)

-11
x 10
0

-0.2

-0.4

-0.6
e(t)

-0.8

-1

-1.2

-1.4

-1.6
0 1 2 3 4 5 6
t

===========================================================

From the above numerical results of error function e(t), it is true that the
precision is around 10−10 which is indeed less than h4(10−8). In MATLAB, some
instructions are provided to solve the differential equations, such as ode23 and ode45.
In fact, these instructions are also implemented by the Runge-Kutta method. Now,
let’s use the instruction ode23 to solve the system (4-42) and use the instruction ode45
to solve the system described as below:
x& (t ) = f (t , x(t )) = − x(t ) + 0.5sin(sin10t ) , x(0)=0.5 (4-45)
whose solution can not be expressed in closed form. The programming in MATLAB
is given in the following, which includes two m-files. The file first1.m is written for
(4-43) and solved by ode23. The file first2.m is written for (4-45) and solved by
ode45. Clearly, the result of (4-45) can not be expressed in closed form since it is
more complicated than the result of (4-43).

=========================================
Create m-file: first1.m
function dx=first1(t,x)
dx=-x+1;
=========================================
Create m-file: first2.m
function dx=first2(t,x)
dx=-x+0.5*sin(sin(10*t));
=========================================
>> % key in the following instructions
>> [t,x]=ode23(@first1,[0:0.01:6],0.5)
>> plot(t,x); xlabel(‘t’); ylabel(‘x(t)’)

4-7
 NCTU Department of Electrical and Computer Engineering 2015 Spring Course
<Dynamic System Simulation and Implementation> by Prof. Yon-Ping Chen

0.95

0.9

0.85

0.8
x(t)

0.75

0.7

0.65

0.6

0.55

0.5
0 1 2 3 4 5 6
t

>> % key in the following instructions

>> [t,x]=ode45(@first2,[0:0.01:6],0.5)
>> plot(t,x); xlabel(‘t’); ylabel(‘x(t)’)
0.6

0.5

0.4

0.3
x(t)

0.2

0.1

-0.1
0 1 2 3 4 5 6
t

=====================================================

4-8