Numerical Methods for Partial

Differential Equations
CAAM 452
Spring 2005
Lecture 4
1-step time-stepping methods: stability, accuracy
Runge-Kutta Methods,
Instructor: Tim Warburton
 Today
• Recall AB stability regions and start up issues
• Group analysis of the Leap-Frog scheme

• One-step methods
• Example Runge-Kutta methods:
– Modified Euler
– General family of 2nd order RK methods
– Heun’s 3rd Order Method
– The 4th Order “Runge-Kutta” method
– Jameson-Schmidt-Turkel
• Linear Absolute Stability Regions for the 2nd order family RK
• Global error analysis for general 1-step methods (stops slightly short of
a full convergence analysis)
• Warning on usefulness of global error estimate
• Discussion on AB v. RK
• Embedded lower order RK schemes useful for a posteriori error
estimates.

CAAM 452 Spring 2005

 Recall: AB2 v. AB3 v. AB4
• These are the margins of absolute stability for the AB methods:

• Starting with the yellow AB1 (Euler-Forward) we see that as the order of
accuracy goes up the stability region shrinks.
• i.e. we see that to use the higher order accurate AB scheme we are required
to take more time steps.
Q) how many more? CAAM 452 Spring 2005
 Recall:
Requirements Starting Requirements
u0  u  0 
AB1: 1 solution level for start
un1  un  dt  f  un  
u1  u  dt  2 solution levels for start

AB2: u0  u  0 

un1  un  dt  f  un   f  un1  
3 1
2 2 
AB3: u2  u  2dt  3 solution levels for start

u1  u  dt 
u0  u  0 
dt
un1  un   23 f n  16 f n1  5 f n2 
12 CAAM 452 Spring 2005
 cont

• So as we take higher order version of the AB

scheme we also need to provide initial values at
more and more levels.

• For a problem where we do not know the solution

at more than the initial condition we may have to:
– Use AB1 with small dt to get the second restart level
– Use AB2 with small dt to get the third restart level
– March on using AB3 started with the three levels
achieved above. AB2

AB1 AB3 CAAM 452 Spring 2005

 Recall: Derivation of AB Schemes

• The AB schemes were motivated by considering

the exactly time integrated ODE:
tn 1

u  tn1   u  tn    f u t   dt
tn

• Which we approximated by using a p’th order

polynomial interpolation of the function f
tn 1

u  tn1   u  tn    I f u t   dt
tn
p

CAAM 452 Spring 2005

 Leap Frog Scheme

• We could also have started the integral at: tn1

tn 1

u  tn1   u  tn1    f u t   dt
tn 1

• And used the mid point rule:

u  tn1   u  tn1   2dtf  u  tn  

• Which suggests the leap frog scheme:

un1  un1  2dtf  un 

CAAM 452 Spring 2005

 Volunteer Exercise un1  un1  2dtf  un 

1) accuracy: what is the local truncation error?

2) stability: what is the manifold of absolute linear

stability (try analytically) in the nu=dt*mu plane?

2a) what is the region of absolute linear-stability?

CAAM 452 Spring 2005
 cont un1  un1  2dtf  un 

3) How many starting values are required?

4) Do we have convergence?

5) What is the global order of accuracy?

6) When is this a good method?

CAAM 452 Spring 2005
 One Step Methods
• Given the difficulties inherent in starting the higher order AB
schemes we are encouraged to look for one-step methods
which only require un to evaluate un 1

• i.e. un1  un  dt  un , tn ; dt 

• Euler-Forward is a one-step method:

un1  un  dtf  un     un , tn ; dt  : f  un 
• We will consider the one-step Runge-Kutta methods.
• For introductory details see:
– “An introduction to numerical analysis”, Suli and Mayers, 12.2
(p317) and on
– Trefethen p75-
– Gustafsson,Kreiss and Oliger p241-
CAAM 452 Spring 2005
 Runge-Kutta Methods

• The Runge-Kutta are a family of one-step methods.

• They consist of s stages (i.e. require s evaluations

of f)

• They will be p’th order accurate, for some p.

• They are self starting !!!.

CAAM 452 Spring 2005

 Example Runge-Kutta Method
(Modified Euler)
• Modified Euler: a  dtf  un , tn 
 a
b  dtf  un  , tn  
dt 
 2 2
un1  un  b
• Note how we only need one starting value.
• We can also reinterpret this through “intermediate”
values: dt
uˆ1  un  f  un , tn 
2
un1  un  dtf  uˆ1 , tn1/ 2 
• This looks like a half step to approximate the mid-
interval u and then a full step.
• This is a 2-stage, 2nd order, single step method.
CAAM 452 Spring 2005
 Linear Stability Analysis

• As before we assume that f is linear in u and

independent of time
• The scheme becomes (for some given mu):
dt dt
uˆ1  un  f  un , tn  uˆ1  un  un
2 2
un1  un  dtf  uˆ1 , tn1/ 2  un1  un  dt uˆ1

• Which we simplify (eliminate the uhat variable):

dt
uˆ1  un  un  dt  
2

2 un1  un  dt un  un
2
un1  un  dt uˆ1
CAAM 452 Spring 2005
 cont

• We gather all terms on the right hand side:

  dt   
2

un1  1  dt    un
 2 

• [ Note: the bracketed term is exactly the first 3

terms of the Taylor series for exp(dt*mu), more on
that later ]
• We also note for the numerical solution to be
bounded, and the scheme stable, we require:
 dt  
2

1  dt   1
2
CAAM 452 Spring 2005
 cont
• The stability region is the set of nu=mu*dt in the complex
plane such that:
2
1   1
2

• The manifold of marginal stability can be found (as in the

linear multistep methods) by fixing the multiplier to be of unit
magnitude and looking for the corresponding values of nu
which produce this multiplier.

• i.e. for each theta find nu such that

2
1    ei
2
CAAM 452 Spring 2005
 cont

• We can manually find the roots of this quadratic:

2
1    ei
2

• To obtain a parameterized representation of the

manifold of marginal stability:

  1  1  2 1  ei 

CAAM 452 Spring 2005

 Plotting Stability Region for
Modified Euler

  1  1  2 1  ei 
CAAM 452 Spring 2005
 Checking Modified Euler
at the Imaginary Axis
• As before we wish to check how much of the
imaginary axis is included inside the region of
absolute stability.
• Here we plot the real part of the “+” root

CAAM 452 Spring 2005

 Is the Imaginary Axis in
the Stability Region ?
• We can analytically zoom in by choosing
nu=i*alpha (i.e. on the imaginary axis).
• We then check the magnitude of the multiplier:
2 2 2 2 2
    2
 4
1    1  i   1     2  1 
2 2  2  4
• So we know that the only point on the imaginary
axis with multiplier magnitude bounded above by 1
is the origin.
• Modified Euler is not suitable for the advection
equation.

CAAM 452 Spring 2005

 General 2 stage RK family

• Consider the four parameter family of RK schemes

of the form:

k1  f  un , tn 
k2  f  un   dtk1 , tn   dt 
un1  un  dt  ak1  bk2 
• where we will determine the parameters
(a,b,alpha,beta) by consideration of accuracy.

• [ Euler-Forward is in this family with a=1,b=0

CAAM 452 Spring 2005

 cont

• The single step operator in this case is:

k1  f  un , tn 
k2  f  un   dtk1 , tn   dt 
un1  un  dt  ak1  bk2 
 un1  un    un , tn 
where   un , tn   af  un , tn   bf  un   dtf  un , tn  , tn   dt 

CAAM 452 Spring 2005

 cont

• We now perform a truncation analysis, similar to

that performed for the linear multistep methods.

• We will use the following fact:

du
 f u t  , t 
dt
d 2u d f f du f f
 2  f u t  , t      f
dt dt t u dt t u
d 3u d  f f 
 3    f   ...
dt dt  t u 
 2 f 2 f   2 f 2 f  f  f f 
 t 2  tu f    ut  u 2 f  f  u  t  u f 
     
CAAM 452 Spring 2005
 cont (accuracy)

• We expand Phi in terms of powers of dt using the

bivariate Taylor’s expansion
  u  tn  , tn   af  u , tn   bf  u   dtf  u , t n  , t n   dt 
 
f 
 
 f f   O dt 3
 af  b  dt   dtf
 t u

  
 
   dt  2 2
 f
  dt   dt  f
 2
f

  dtf  2 2
 f 
 
 2! t 2
tu 2! u 2 

• where: f  f  u  tn  , tn 

CAAM 452 Spring 2005

 cont

• We construct the local truncation error as:

Tn   u  tn  dt   u  tn    dt  u  tn  , tn 
 dt 2 
dt
 dt  f   ft  ff u    f tt  2 f tu f  f uu f  f u  f t  f u f   
2

 2 3! 
 
dt af  b  f   dtf t   dtff u 
  dt 
2

f tt   dt ff tu 
2   dtf  2
 
f uu    O  dt 4 
 
  2 2  

• Now we choose a,b,alpha,beta to minimize the

local truncation error.
• Note – we use subindexing to represent partial
derivatives.
CAAM 452 Spring 2005
 cont

• Consider terms which are the same order in dt in the local

truncation error:
 dt 2 
dt
Tn  dt  f   ft  ff u    tt tu uu
f  2 f f  f f 2
 f u  f t  f u f  
 2 3! 
 
dt af  b  f   dtft   dtffu 
  dt 
2

f tt   dt ff tu 
2   dtf  2
 
fuu    O  dt 4 
 
  2 2  
• Condition 1: 1  a  b  0

• Condition 2: 1  ft  ffu   b  dtft   dtffu   0f  b  b  1

2 2

• Under these conditions, the truncation is order 3 so the

method is 2nd order accurate. It is not possible to further
eliminate the dt^3 terms by adjusting the parameters. CAAM 452 Spring 2005
 Case: No Explicit t Dependence in f
  u  tn  , tn   bf  u   dtf  u  
 f   dtf   f 
2 2

 b  f   dtf
u

2! u 2   O  dt 3

 
d 2u f d 3u  2 f 2  f 
2
du
 f u t   2  f , 3  2 f   f
dt dt u dt u  u 

Tn  u  tn1   u  tn   dt  u  tn  ; dt 
 dt 2  
     2
 
 fuu f  f u f    dt b  f   dtff u  f uu    O  dt 4 
dt dt
 dt  f  ffu  2 2

   
2 3!   2  
1
 b  1,  
2
It is easier to generalize to higher order RK in this case when there is no explicit time dependence in f.
CAAM 452 Spring 2005
 Second Example Runge-Kutta:
Heun’s Third Order Formula
• Traditional version • In terms of intermediate
variables:
a  dtf  un , tn 
 a dt 
b  dtf  un  , tn  
dt
uˆ1  un  f  un , tn 
3
 3 3
2dt
 2b 2dt  uˆ2  un  f  uˆ1 , tn1/ 3 
c  dtf  un  , tn   3
 3 3  1
un1  un   f  un , tn   3 f  uˆ2 , tn2 / 3  
1 4
un1  un   a  3c 
4
This is a 3rd order, 3 stage single step explicit Runge-Kutta method.

CAAM 452 Spring 2005

 Again Let’s Check the Stability Region
dt
uˆ1  un  f  un , tn 
3
2dt
uˆ2  un  f  uˆ1 , tn1/ 3 
3
1
un1  un   f  un , tn   3 f  uˆ2 , tn2 / 3  
4 dt
uˆ1  un   un
3
2dt 
  un  un 
dt
uˆ2  un 
With f=mu*u reduces to a 3  3 
single level recursion with a dt   2dt  
  un  un   
dt
very familiar multiplier: un 1  un   un  3  un 
4  3  3 
dt   2dt  dt   
 un     3   1   1      un
4  3  3 
   dt    dt  
2 3

 1   dt    un
 2 3  CAAM 452 Spring 2005

 Stability of Heun’s 3rd Order Method

• Each marginally stable mu*dt is such that the

multiplier is of magnitude 1, i.e.

2 3 i
1    e
2 6
• This traces a curve in the nu=mu*dt complex plane.
• Since we are short on time we can plot this using
Matlab’s roots function…

CAAM 452 Spring 2005

Stability Region for RK (s=p)

rk3

rk2

rk4

CAAM 452 Spring 2005

 Heun’s Method and The Imaginary Axis

• This time we consider points on the imaginary axis

which are close to the origin:
  i rk3
3 2
 2

1  i  i
2 6
2 2
    2
  3
 1       
 2   6 
 4
 6
 1 
12 36
  3  1.73
• And this is bounded above by 1 if CAAM 452 Spring 2005
 Observation on RK linear stability

• For the s’th order, s stage RK

we see that the stability region
grows with increasing s:

• Consequently we can take a

larger time step (dt) as the
order of the RK scheme
increase.

• On the down side, we require

more evaluations of f
CAAM 452 Spring 2005
 Popular 4th Order Runge-Kutta Formula

• Four stages:
a  dtf  un , tn 
b  dtf  un  a / 2, tn1/ 2 
c  dtf  un  b / 2, tn1/ 2 
d  dtf  un  c, tn1 
1
un1  un   a  2b  2c  d 
6
see: http://web.comlab.ox.ac.uk/oucl/work/nick.trefethen/1all.pdf p76 for details
of minimum number of stages to achieve p’th order.

CAAM 452 Spring 2005

 Imaginary Axis (again)

• With the obvious multiplier we obtain:

  i
4 2
 2
 3

1     
2 6 24
4 2
 2
 3
 6 8
1  i  i   1 
2 6 24 72 242
• For stability we require: rk4

 6
 8
  8   2 i.e.   2 2  2.83
72 242
CAAM 452 Spring 2005
Imaginary Axis Stability Summary

2.83 for the 4th Order “Runge-Kutta” method

1.73 for Heun’s 3rd Order Method

0 for modified Euler

CAAM 452 Spring 2005

 Bounding the Global Error in Terms of the Local
Truncation Error
Theorem: Consider the general one-step method
un1  un  dt  un , tn ; dt 

• and we assume that Phi is Lipschitz continuous

with respect to the first argument (with constant L )
• i.e. for   u, t  ,  v, t   D   u, t  : t0  t  tmax , u  u0  C
we have:
  u, t ; dt     v, t ; dt   L u  v

• Then assuming u  tn   u  t0   Cn  1, 2,.., N it follows that

un  u  t n  
L

T L tn t0 
e 
 1 , n  0,1,..., N where T  max Tn
0 n N 1

CAAM 452 Spring 2005

 cont

Proof: we use the definition of the local truncation

error: Tn   u  tn  dt   u  tn    dt  u  tn  , tn 
to construct the error equation:
u  tn1   un1  u  tn   un  dt    u  tn  , tn     un , tn    Tn

we use the Lipschitz continuity of Phi:

u  tn1   un1  u  tn   un  dtL u  tn   un  Tn
tidying:
u  tn1   un1  1  dtL  u  tn   un  Tn

CAAM 452 Spring 2005

 proof cont
u  tn1   un1  1  dtL  u  tn   un  Tn
 1  dtL  1  dtL  u  tn1   un1  Tn1   Tn

mn
 1  dtL  u  t0   u0   Tnm 1  dtL 
n 1 m

m 0
mn
  Tnm 1  dtL 
m

m 0

1  dtL   1
n 1

   
mn
 max Tm      max Tm
m
1 dtL
1 m
m 0
1 m n 1  dtL  1


T
dtL

1  dtL n1  1 
T
dtL
 
e n1dtL  1 

T
dtL

e L tn1 t0   1 
CAAM 452 Spring 2005
 proof summary

• We now have the global error estimate:

u  tn1   un1 
T
dtL
 
e L tn1 t0   1

• Broadly speaking this implies that if the local truncation error

is h^{p+1} then the error at a given time step will scale as
O(h^p):
u  tn1   un1  O  h p 

• Convergence follows under restrictions on the ODE which

guarantee existance of a unique C1 solution and stable
choice of dt.

CAAM 452 Spring 2005

 Warning About Global Error Estimate

• It should be noted that the error estimate is of

almost zero practical use.

u  tn1   un1 
T
dtL
 
e L tn1 t0   1

• In the full convergence analysis we pick a final time

t and we will see that exponential term again.
• Convergence is guaranteed but the constant can
be extraordinarily large for finite time:

L

1 L t t0 
e 1 
CAAM 452 Spring 2005
 A Posteriori Error Estimate

• There are examples of RK methods which have

embedded lower order schemes.
• i.e. after one full RK time step, for some versions it
is possible to use a second set of coefficients to
reconstruct a lower order approximation.
• Thus we can compute the difference between the
two different approximations to estimate the local
truncation error committed over the time step.

• google: runge kutta embedded

• Numerical recipes in C:
– http://www.library.cornell.edu/nr/bookcpdf/c16-2.pdf
CAAM 452 Spring 2005
 My Favorite s Stage
Runge-Kutta Method
• There is an s stage Runge-Kutta method of
particular simplicity due to Jameson-Schmidt-
Turkel, which is of interest when there is no explicit
time dependence for f
u  un
for m=0:s-1
dt
u  un  f u 
sm
end
un1  u
CAAM 452 Spring 2005
 RK v. AB

• When should we use RK and when should we use

AB?

rk3

rk2

rk4

CAAM 452 Spring 2005

 Class Cancelled on 02/17/05

• There will be no class on Thursday 02/17/05

• The homework due for that class will be due the

following Thursday 02/24/05

CAAM 452 Spring 2005