Time-stepping techniques

Unsteady flows are parabolic in time

use time-stepping methods to

advance transient solutions step-by-step or to compute stationary solutions

time
future

Initial-boundary value problem

zone of influence

present

past

u = u(x, t)

domain of
dependence

space

Time discretization

u
t

+ Lu = f

in (0, T )

time-dependent PDE

Bu = 0

on (0, T )

boundary conditions

u = u0

in at t = 0

initial condition

0 = t0 < t1 < t2 < . . . < tM = T

Consider a short time interval (tn , tn+1 ) , where

u0 u0 in
tn+1 = tn + t

Given un u(tn ) use it as initial condition to compute un+1 u(tn+1 )

Space-time discretization
Space discretization:

finite differences / finite volumes / finite elements

Unknowns:

time-dependent nodal values / cell mean values

ui (t)

Time discretization:

(i) before or (ii) after the discretization in space

The space and time variables are essentially decoupled and can be discretized
independently to obtain a sequence of (nonlinear) algebraic systems
A(un+1 , un )un+1 = b(un )
Method of lines (MOL)

L Lh

duh
+ Lh uh = fh
dt
FEM approximation

yields an ODE system for ui (t)

on (tn , tn+1 )

uh (x, t) =

n = 0, 1, . . . , M 1

N
P

j=1

semi-discretized equations

uj (t)j (x),

uni u(xi , tn )

Galerkin method of lines

R

Weak formulation
d
dt (u, v)

u
t

+ Lu f v dx = 0,

+ a(u, v) = l(v), v V

Differential-Algebraic Equations

d
dt (uh , vh )

t (tn , tn+1 )

v V,

+ a(uh , vh ) = l(vh ), vh Vh

MC du
dt + Au = b

t (tn , tn+1 )

where MC = {mij } is the mass matrix and u(t) = [u1 (t), . . . , uN (t)]T
Matrix coefficients
Mass lumping

mij = (i , j ),

MC ML = diag{mi },

aij = a(i , j ),
mi =

P
j

due to the fact that

P
j

j 1.

bi = l(i )

mij = (i ,

P
j

j ) =

i dx

In the 1D case FDM=FVM=FEM + lumping

Two-level time-stepping schemes

Lumped-mass discretization

First-order ODE system

ML du
dt + Au = b,

du
dt

+ R(u, t) = 0

u(tn ) = un ,

R(u, t) = ML1 [Au b]

for t (tn , tn+1 )
n = 0, 1, . . . , M 1

Standard scheme (finite difference discretization of the time derivative)


un+1 un 
n+1 n+1
n n
+ R(u
,t
) + (1 )R(u , t ) = 0
t

01

where t = tn+1 tn is the time step and is the implicitness parameter

=0

forward Euler scheme

explicit,

O(t)

= 1/2

Crank-Nicolson scheme

implicit,

O(t)2

=1

backward Euler scheme

implicit,

O(t)

Fully discretized problem

Consistent-mass discretization

R(u, t) = MC1 [Au b]

MC du
dt + Au = b,

[MC + tA]un+1 = [MC (1 )tA]un + t bn+

where

bn+ = bn+1 + (1 )bn ,

0 1,

n = 0, . . . , M 1

In general, time discretization is performed using numerical methods for ODEs

du(t) = f (t, u(t))

dt
Initial value problem
on (tn , tn+1 )
u(tn ) = un

Exact integration

R tn+1
tn

un+1 = un + f (, u( ))t,

du
dt

dt = u

n+1

u =

(tn , tn+1 )

R tn+1
tn

f (t, u(t)) dt

by the mean value theorem

Idea: evaluate the integral numerically using a suitable quadrature rule

Example: standard time-stepping schemes

Numerical integration on the interval (tn , tn+1 )
f

FE

111111111
000000000
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111

tn

tn+1

left endpoint

BE

111111111
000000000
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111

tn

tn+1

right endpoint

CN

111111111
000000000
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111

tn

tn+1

LF

111111111
000000000
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111

trapezoidal rule

tn

tn+1

midpoint rule

Forward Euler

un+1 = un + f (tn , un )t + O(t)2

Backward Euler

un+1 = un + f (tn+1 , un+1 )t + O(t)2

Crank-Nicolson

un+1 = un + 12 [f (tn , un ) + f (tn+1 , un+1 )]t + O(t)3

Leapfrog method

un+1 = un + f (tn+1/2 , un+1/2 )t + O(t)3

Properties of time-stepping schemes

Time discretization

tn = nt,

t =

Accumulation of truncation errors

p
loc
= O(t)

T
M

M=

T
t

n = 0, . . . , M 1

p1
glob
= M loc

= O(t)

Remark. The order of a time-stepping method (i.e., the asymptotic rate at

which the error is reduced as t 0) is not the sole indicator of accuracy
The optimal choice of the time-stepping scheme depends on its purpose:
to obtain a time-accurate discretization of a highly dynamic flow problem
(evolution details are essential and must be captured) or
to march the numerical solution to a steady state starting with some
reasonable initial guess (intermediate results are immaterial)
The computational cost of explicit and implicit schemes differs considerably

Explicit vs. implicit time discretization

Pros and cons of explicit schemes
easy to implement and parallelize, low cost per time step
a good starting point for the development of CFD software
small time steps are required for stability reasons, especially
if the velocity and/or mesh size are varying strongly
extremely inefficient for solution of stationary problems unless
local time-stepping i. e. t = t(x) is employed
Pros and cons of implicit schemes
stable over a wide range of time steps, sometimes unconditionally
constitute excellent iterative solvers for steady-state problems
difficult to implement and parallelize, high cost per time step
insufficiently accurate for truly transient problems at large t
convergence of linear solvers deteriorates/fails as t increases

Example: 1D convection-diffusion equation

u
t

u
+ v u
x = d x2

u(0) = u(1) = 0,

in (0, X) (0, T )
u|t=0 = u0

Lagrangian representation
where

d
dt

u
f (t, u(t)) = v u
x + d x2

2u
du(x(t), t)
=d 2
dt
x

pure diffusion equation

is the substantial derivative along the characteristic lines

Initial profile is convected at speed v

and smeared by diffusion if d > 0
d=0

=v

t=T
v

du(x(t), t)
=0
dt

For the pure convection equation u is

constant along the characteristics

t=0

dx(t)
dt

d=0
d>0

X x

Example: 1D convection-diffusion equation

Uniform space-time mesh
X
N,

i = 0, . . . , N

tn = nt,

t =

T
M,

n = 0, . . . , M

u0i = u0 (xi )

Fully discretized equation

01

unknown

x =

x

tn+1

t

tn

t

t 1
n

known

xi = ix,

un un+1 ,

x

un+1
= uni + [fhn+1 + (1 )fhn ]t
i
x 1
x
x +1
0
Central difference / lumped-mass FEM
"
#
n+1
n+1
n+1
n+1
n+1
n+1
n
u
u
2ui + ui+1
ui1
ui ui
= v i+1
+ d i1
t
2x
(x)2


uni1 2uni + uni+1
uni+1 uni1
+d
+ (1 ) v
2x
(x)2
i

a sequence of tridiagonal linear systems

i = 1, . . . , N,

X x

n = 0, . . . M 1

Example: 1D convection-diffusion equation

Standard scheme (two-level)
n+
n+
n+
n+
ui+1
ui1
ui1
2uin+ + ui+1
un+1
uni
i
+v
=d
t
2x
(x)2

uin+ = un+1
+ (1 )uni ,
i

01

Forward Euler ( = 0)

un+1
= h(uni1 , uni , uni+1 )
i

Backward Euler ( = 1)

n+1
n
un+1
= h(un+1
i1 , ui , ui+1 )
i

Crank-Nicolson ( = 12 )

n+1
n
n
n
un+1
= h(un+1
i
i1 , ui1 , ui , ui+1 , ui+1 )

n+1
t
t

Leapfrog time-stepping

n+1

n+1
t

, uni , uni+1 )
un+1
= h(uni1 , un1
i
i

n+1
t
t

BE

CN

un+1
= un1
+ 2tfhn
i
i

uni+1 uni1
uni1 2uni + uni+1
un+1
un1
i
i
+v
=d
2t
2x
(x)2
(explicit, three-level)

FE

LF

Fractional-step scheme
Given the parameters (0, 1), = 1 2, and [0, 1] subdivide the time
interval (tn , tn+1 ) into three substeps and update the solution as follows
Step 1.

un+ = un + [f (tn+ , un+ ) + (1 )f (tn , un )]t

Step 2.

un+1 = un+ + [(1 )f (tn+1 , un+1 ) + f (tn+ , un+ )] t

Step 3.

un+1 = un+1 + [f (tn+1 , un+1 ) + (1 )f (tn+1 , un+1 )]t

Properties of this time-stepping method

second-order accurate in the special case = 1

2
2

coefficient matrices are the same for all substeps if =

12
1

combines the advantages of Crank-Nicolson and backward Euler

Predictor-corrector and multipoint methods

Objective: to combine the simplicity of explicit schemes and robustness of
implicit ones in the framework of a fractional-step algorithm, e.g.,
1. Predictor

u
n+1 = un + f (tn , un )t

forward Euler

2. Corrector

un+1 = un + 21 [f (tn , un ) + f (tn+1 , u

n+1 )]t

Crank-Nicolson

un+1 = un + f (tn+1 , u
n+1 )t

backward Euler

or

Remark. Stability still leaves a lot to be desired, additional correction steps

usually do not pay off since iterations may diverge if t is too large
Order barrier: two-level methods are at most second-order accurate, so
extra points are needed to construct higher-order integration schemes
Adams methods

tn+1 , . . . , tnm ,

Runge-Kutta methods

tn+ [tn , tn+1 ],

m = 0, 1, . . .
[0, 1]

Adams methods
f

Derivation: polynomial fitting

Truncation error:

past

present future

= O(t)p
glob

for polynomials of degree p 1 which

interpolate function values at p points
Adams-Bashforth methods

tn 2

(explicit)

p=1

un+1 = un + tf (tn , un )

p=2

un+1 = un +

t
n
n
2 [3f (t , u )

p=3

un+1 = un +

t
n
n
12 [23f (t , u )

Adams-Moulton methods

tn 1

tn

tn+1

forward Euler
f (tn1 , un1 )]
16f (tn1 , un1 ) + 5f (tn2 , un2 )]

(implicit)

p=1

un+1 = un + tf (tn+1 , un+1 )

p=2

un+1 = un +

t
n+1
, un+1 )
2 [f (t

p=3

un+1 = un +

t
n+1
, un+1 )
12 [5f (t

backward Euler
+ f (tn , un )]

Crank-Nicolson

+ 8f (tn , un ) f (tn1 , un1 )]

Adams methods
Predictor-corrector algorithm
1. Compute u
n+1 using an Adams-Bashforth method of order p 1
2. Compute un+1 using an Adams-Moulton method of order p with
predicted value f (tn+1 , u
n+1 ) instead of f (tn+1 , un+1 )
Pros and cons of Adams methods
methods of any order are easy to derive and implement
only one function evaluation per time step is performed
error estimators for ODEs can be used to adapt the order
other methods are needed to start/restart the calculation
time step is difficult to change (coefficients are different)
tend to be unstable and produce nonphysical oscillations

Runge-Kutta methods
Multipredictor-multicorrector algorithms of order p
p=2

u
n+1/2 = un +

t
n
n
2 f (t , u )

un+1 = un + tf (tn+1/2 , u
n+1/2 )
p=4

forward Euler / predictor

midpoint rule / corrector

u
n+1/2 = un +

t
n
n
2 f (t , u )

forward Euler / predictor

u
n+1/2 = un +

t
n+1/2
,u
n+1/2 )
2 f (t

backward Euler / corrector

u
n+1 = un + tf (tn+1/2 , u
n+1/2 )
un+1 = un +

t
n
n
6 [f (t , u )

midpoint rule / predictor

+ 2f (tn+1/2 , u
n+1/2 )

+ 2f (tn+1/2 , u
n+1/2 ) + f (tn+1 , u
n+1 )]

Simpson rule
corrector

Remark. There exist embedded Runge-Kutta methods which perform extra

steps in order to estimate the error and adjust t in an adaptive fashion

General comments
Pros and cons of Runge-Kutta methods
self-starting, easy to operate with variable time steps
more stable and accurate than Adams methods of the same order
high order approximations are rather difficult to derive; p function
evaluations per time step are required for a pth order method
more expensive than Adams methods of comparable order
Adaptive time-stepping strategy

t t

makes it possible to achieve the desired accuracy at a relatively low cost

Explicit methods: use the largest time step satisfying the stability condition
Implicit methods: estimate the error and adjust the time step if necessary

Automatic time step control

Objective: make sure that

||u ut || T OL

Local truncation error

1.

ut = u + t2 e(u) + O(t)4

2.

umt = u + m2 t2 e(u) + O(t)4

(prescribed tolerance)

Heuristic error analysis

e(u)

umt ut
t2 (m2 1)

Remark. It is tacitly assumed that the error at t = tn is equal to zero.

Estimate of the relative error
2

||ut umt ||
t
= T OL
||u ut ||
t
m2 1

Adaptive time stepping

t2

t2 (m2 1)
= T OL
||ut umt ||

Richardson extrapolation: eliminate the leading term

m2 ut umt
4
+
O(t)
u=
m2 1

fourth-order accurate

Practical implementation
Automatic time step control can be executed as follows
Given the old solution un do:
1. Make one large time step of size mt to compute umt
2. Make m small substeps of size t to compute ut
3. Evaluate the relative solution changes ||ut umt ||
4. Calculate the optimal value t for the next time step
5. If t t, reset the solution and go back to step 1
6. Set un+1 = ut or perform Richardson extrapolation
Note that the cost per time step increases substantially (umt may be as
expensive to obtain as ut due to slow convergence at large time steps).
On the other hand, adaptive time-stepping enhances the robustness of the
code, the overall efficiency and the credibility of simulation results.

Evolutionary PID controller

Another simple mechanism for capturing the dynamics of the flow:
Monitor the relative changes en =

||un+1 un ||
||un+1 ||

of an indicator variable u

If en > reject the solution and repeat the time step using t =

en tn

Adjust the time step smoothly so as to approach the prescribed tolerance

tn+1 =

en1
en

k P 

T OL
en

k I 

e2n1
en en2

k D

tn

Limit the growth and reduction of the time step so that

tmin tn+1 tmax ,
Empirical PID parameters

kP = 0.075,

kI = 0.175,

tn+1
L
tn
kD = 0.01

Pseudo time-stepping
Solutions to boundary value problems of the form Lu = f represent the
steady-state limit of the associated time-dependent problem
u
+Lu = f,
t

u(x, 0) = u0 (x)

in

where u0 is an arbitrary initial condition. Therefore, the numerical solution

can be marched to the steady state using a pseudo time-stepping technique.
can be interpreted as an iterative solver for the stationary problem
the artificial time step represents an adjustable relaxation parameter
evolution details are immaterial t should be as large as possible
the unconditionally stable backward Euler method is to be recommended
explicit schemes can be used in conjunction with local time-stepping
it is worthwhile to perform an adaptive time step control (e.g., PID)