Numerical Methods for Partial

Differential Equations
CAAM 452
Spring 2005
Lecture 5
Summary of convergence checks for LMM and
one-step time stepping
Instructor: Tim Warburton
 Today

• I will summarize the steps we took to examine the

stability, consistency and thus convergence of the
AB and RK schemes

CAAM 452 Spring 2005

 AB
m s
• The general AB scheme reads: un1  un  dt   m f  un1m 
m 1

• To check linear stability we assume that f is linear in u (we

can also achieve a similar goal by assuming f is Lipschitz)
m s
• Assuming: f  un    un  un1  un  dt    m un1m
m 1

• We know that all the solutions of this recurrence relationship

are bounded provided the corresponding stability
m s
polynomial:
z  z  dt    m z s m
s s 1

m1
has roots which satisfy the root condition (all roots bounded
by 1, and if they have magnitude 1 must be simple roots).
CAAM 452 Spring 2005
 AB cont

• We must then check the roots of:

m s
z z
s s 1
 dt    m z s m
m 1

• For roots with |z|<1 we are sure that the method is stable.
What we wish to know is what values of mu*dt guarantee
this condition. We can think of this formula defining a map
between the (complex) plane of roots to the (complex) plane
of mu*dt

• We must determine the image of the unit circle in the root

plane in the mu*dt plane.

CAAM 452 Spring 2005

 AB cont
m s
• Given the root equation: z  z  dt    m z sm
s s 1

m 1

• We determine the map in a trivial manner:

m s
z s  z s 1
z s  z s 1  dt    m z s m  dt   ms
m 1
 m

m1
z s m

• Example AB3: z3  z 2
 :
(23 z 2 -16 z  5) /12

CAAM 452 Spring 2005

 AB3 Stability

z  rei 

z3  z2
 :
(23 z 2 -16 z  5) /12
CAAM 452 Spring 2005
 AB Accuracy

• Next we perform a local truncation error analysis.

• That is we estimate what correction at each step of

the solution is required for the exact solution of the
ODE to be a solution to the numerical method:
m s
u  tn1   u  tn   dt   m f  u  tn1m    Tn
m 1

CAAM 452 Spring 2005

 cont

• We now use Taylor expansions to express the

truncation error in terms ofmsu and its derivatives at
tn: u  tn1   u  tn   dt   m f  u  tn1m    Tn
m 1

• We found that Tn scaled as:

s 1
d u *
Tn  Cdt s 1

dt s 1  t n  for some constant C and t n   tn , tn 1 

*

• Finally we apply Dahlquist’s equivalence theorem,

which requires consistency and stability 
convergence and corollary which gives us global
error estimates: u  tn   un  Ct s ,u ,...dt s
n,

CAAM 452 Spring 2005

 RK Analysis

• A similar analysis is required for the Runge-Kutta

schemes.

• The stability analysis was straightforward as we

found that the recurrence for a linear f was:

 m s  m 
un1     un
 m 0 m ! 
• For stability we required: m s
m

m 0 m !
1

• We examined this by setting m s

m

m 0 m !
 e i

CAAM 452 Spring 2005

RK3 Stability
and let Matlab do the work.

rei 

2 3
1      rei
2 3!
CAAM 452 Spring 2005
 RK Accuracy

• The error estimate process for RK schemes is a

little more complicated, but still boils down to dt
expansions of the one-step operator.

• Again, once we have consistency and stability we

have convergence.

CAAM 452 Spring 2005

 Implicit Schemes

• Later on in the semester we will discuss implicit

schemes.

• Then we will also discuss the Butcher block

formulation of RK methods.

CAAM 452 Spring 2005

 After Time Stepping… Back to PDE’s

• We have now proposed two quite distinct but

reliable ways to advance an ODE approximately in
time.

• But our original goal was to solve a PDE

u u
c
t x

• And not just for one mode.

CAAM 452 Spring 2005

 u u
Sample Points in Time Space c
t x

• We chose to sample the ODE at discrete points on the time axis.

• We can also make the same choice for the spatial representation
at each discrete time level.

tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3

• We follow this choice with the dilemma of how to reconstruct an

approximation to the derivative of discretely sampled data.

CAAM 452 Spring 2005

 Choice of Stencil

tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3

tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3
tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3
tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3
tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3
And many more combinations
CAAM 452 Spring 2005
 Use Taylor’s Series

• In much the same spirit as time-domain we can

expand the solution at each grid point in terms of u
and its derivatives as a power series in dx (the
space between spatial grid points)

• Examples:
du dx 2 d 2u dx 3 d 3u *
3  m
u  xm1   u  xm   dx  xm   2  m
x  x
dx 2! dx 3! dx
du dx 2 d 2u dx 3 d 3u **
3  m 
u  xm1   u  xm   dx  xm   2  m
x  x
dx 2! dx 3! dx

CAAM 452 Spring 2005

 Left Differencing

tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3

• Using the data points on this stencil we wish to

compute an approximation to the gradient of u at xm

du dx 2 d 2u dx 3 d 3u *
3  m
u  xm1   u  xm   dx  xm   2  m
x  x
dx 2! dx 3! dx

• Dividing and rearranging we find:

du u  xm   u  xm1  dx d 2u dx 2 d 3u *
 xm   2  m 3  m
 x  x
dx dx 2! dx 3! dx
CAAM 452 Spring 2005
 Left Differencing

tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3

du u  xm   u  xm1  dx d 2u dx 2 d 3u *
 xm   2  m 3  m
 x  x
dx dx 2! dx 3! dx

This gives us an obvious one-sided linear interpolation formula

of the derivative.

We will later on decide whether it is suitable for time-stepping with

AB or RK.

CAAM 452 Spring 2005

 Left Difference Operator  

• We define the following difference operator

(appalling but fairly standard notational choice):
um  um1
 um 
dx
• Where um represents the function sampled at the
m’th data point.

• The difference formula shows

du dx d 2u dx 2 d 3u *
3  m
 um   xm   2  m
x  x
dx 2! dx 3! dx
i.e. delta- is a first order approximation of the derivative operator
CAAM 452 Spring 2005
 Right Difference Operator  

tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3

um1  um
  um 
dx

du dx d 2u dx 2 d 3u *
3  m
 um   xm   2  m
x  x
dx 2! dx 3! dx

i.e. delta+ is a first order approximation of the derivative operator

CAAM 452 Spring 2005

 Central Differencing Operator  0

tn x
xm3 xm2 xm1 xm xm1 xm 2 xm3

• Subtracting the forward and backward expansions:

du dx 2 d 2u dx3 d 3u dx 4 d 4u dx5 d 5u *
3  m 4  m 5  m
u  xm1   u  xm   dx  xm   2  m
x  x  x  x
dx 2! dx 3! dx 4! dx 5! dx
du dx 2 d 2u dx3 d 3u dx 4 d 4u dx5 d 5u **
3  m 4  m 5  m 
u  xm1   u  xm   dx  xm   2  m
x  x  x  x
dx 2! dx 3! dx 4! dx 5! dx

• We obtain:
3 3
du 2dx d u
3  m  
u  xm1   u  xm1   2dx  xm   x  O dx 5

dx 3! dx
CAAM 452 Spring 2005
 cont

um1  um1 du dx 2 d 3u
 m 3  m  
 0um :  x  x  O dx 4

2dx dx 3! dx
• i.e. the central differencing operator is second order
accurate in dx

• Note: for the error term to be bounded we require

that the third derivative be bounded.

CAAM 452 Spring 2005

 Approximating the Second Derivative

• We can create an approximation of the second

derivative:
 um  um1 
 um :   um   
2

dx


 um1  um   um  um1 

dx 2 dx 2


 um1  2um  um1 
dx 2
 du dx 2 d 2u dx 3 d 3u dx 4 d 4u * 
 u  xm   dx dx  xm   2! dx 2  xm   3! dx 3  xm   4! dx 4  x m  
1  
 2 2u  xm  
dx 
du dx 2 d 2u dx 3 d 3u dx 4 d 4u ** 
4  m 
u  xm   dx  xm   2  m
x  3  m
x  x 
 dx 2! dx 3! dx 4! dx 
CAAM 452 Spring 2005
 cont

 du dx 2 d 2u dx 3 d 3u dx 4 d 4u * 
 u  xm   dx dx  xm   2! dx 2  xm   3! dx 3  xm   4! dx 4  xm  
1  
 um  2 
2
2u  xm  
dx  2 2 3 3 4 4 
du dx d u dx d u dx d u
4  m 
u  xm   dx  xm   2  m
x  3  m
x  x 
**

 dx 2! dx 3! dx 4! dx 
d 2u
 2  xm   O  dx 2 
dx

CAAM 452 Spring 2005

 Delta Operators

du dx d 2u dx 2 d 3u *
 xm   2  m 3  m
 um  x  x
dx 2! dx 3! dx
du dx d 2u dx 2 d 3u *
 xm   2  m 3  m
  um  x  x
dx 2! dx 3! dx
2
d u
 2u m 
dx 2  m
x  O  dx 2

du dx 2 d 3u
 xm   3  m  
 0u m  x  O dx 4

dx 3! dx

CAAM 452 Spring 2005

 Using the delta’s: Example 1

du dx 2 d 3u
3  m
 0um   xm   x  O  dx  4
We use: dx 3! dx
 dx 2 d 2  du
 1  2 
3! dx  dx
 xm   O  dx 4



du dx 2 d 3u
3  m  
 0um   xm   x  O dx 4

dx 3! dx
 dx 2 2  du
 1     xm   O  dx  4

 3!  dx

CAAM 452 Spring 2005

 Example 2

• We can use the difference operators to eliminate

the third order spatial derivative:
 dx 2 d 2  du
 0um   1  2 
3! dx  dx
 x m   O  dx 4




du
dx
 xm   
dx 2 2
3!
 2 2

  O  dx   0  O  dx  u m  O  dx 
4

du dx 2 2

dx
 m
x 
3!
  u
0 m  O  dx 4

du  dx 2 2 
 xm     0    0  um  O  dx 4 
dx  3! 
CAAM 452 Spring 2005
 Example 2 cont

• We can expand out the difference operator

du  dx 2 2 
 xm     0    0  um  O  dx 4 
dx  3! 
 dx 2 2   um1  um1 
 1     O  dx 4 
 3!  2dx
 um1  um1  1
   um2  um  2um1  2um1  um  um2   O  dx 4 
2dx 3!2dx
1
  um2  8um1  8um1  um2   O  dx 4 
12dx
• And we have a super difference formula for the
derivative which is 4th order accurate!
• What is the stencil for this formula ? (volunteer) CAAM 452 Spring 2005
 Example 2 cont

• We can think of this difference formula as a matrix

operator acting on the point data values aligned in
a vector:
du 1
 m
x    um2  8um1  8um1  um2   O  dx 4 
dx 12dx

• Becomes

CAAM 452 Spring 2005

 For 10 Points on a Periodic Interval

 u1   0 8 1 1 8   u1 
u   8 0 8 1 1  u 
 2   2 
 u3   1 8 0 8 1   u3 
    
   1 8 0 8 1  
d   1  1 8 0 8 1  
  
dx   12dx 

1 8 0 8 1
   O  dx 4

 
   1 8 0 8 1  
    
   1  8 0 8 1  
   1 1 8 0 8   
     
 u10   8 1 1 8 0   u10 

Notice: the matrix is skew-symmetrix  imaginary eigenvalues

CAAM 452 Spring 2005
 Diagonalize

• We now use this formula in the PDE which we

evaluate at the 10 spatial sample points:
 u1   u1   0 8 1 1 8   u1 
u  u   8 0 8 1 1  u 
  2   2   2 
 u3   u3   1 8 0 8 1   u3 
      
 u 4  u 4  1  8 0 8 1   u4 
d  u5  d  u5  c  1 8 0 8 1   u5 
 c       O  dx 
4

dt  u6  dx  u6  12dx  1 8 0 8 1   u6 
 u7   u7   1 8 0 8 1   u7 
      
u
 8 u
 8  1  8 0 8 1   u8 
u  u   1 1 8 0 8   u9 
 9
 9
  
 u10   u10   8 1 1 8 0  u10 

This is an identity (although we are a little fuzzy about the O(dx^4) term)
CAAM 452 Spring 2005
 Some Notation

• We will represent the set of data values at the

spatial points as a vector:
 u1 
u 
 2
 u3 
 
 u4 
  u5 
u 
 u6 
 u7 
 
 u8 
u 
 9 
 u10 

CAAM 452 Spring 2005

 cont

• The system matrix is an approximation of the

derivative operator so we will denote it as D
 0 8 1 1 8 
 8 0 8 1 1 
 
 1 8 0 8 1 
 
 1  8 0 8 1 
1  1 8 0 8 1 
D  
12dx  1 8 0 8 1 
 1 8 0 8 1 
 
 1 8 0 8 1 
 1 1 8 0 8 
 
 8 1 1 8 0 

CAAM 452 Spring 2005

 PDE  System of ODE’s
-> Decoupled ODE’s
d 
u  cDu  O  dx 4 

• So we can express the PDE as:
dt
d  
• Let’s make a semi-discrete scheme: u  cDu
dt

• As a theoretical exercise we can diagonalize the

skew symmetric D operator D  SS 1 with Λ a
diagonal matrix.
d  1 
u  cSS u
• Then: dt
d    1 
 v  cv where v  S u
dt
CAAM 452 Spring 2005
 cont

• The system of 10 ODE’s:

d  
v  cv
dt

• are now totally decoupled, so we can apply our

extensive knowledge of treating scalar ODE’s to
each of the components of vhat.

• All we have to do is make sure that the eigenvalues

of the cD matrix are squeezed into the time-
stepping region by choosing small enough dt
CAAM 452 Spring 2005
 cont

• By linearity of the right hand side we can be sure that time

stepping (say using AB or RK) the original ODE will be fine
 
as long as   cdt max  i fits in the relevant stability
1i 10
region…

• We now need to solve: d    

u  f  u   cDu
dt

• Noting our studies of the stability regions, the fact that all the
eigenvalues sit on the imaginary axis rules out
– Euler-Forward (unconditionally unstable)
– AB2 and RK2 (marginally stable)
CAAM 452 Spring 2005
 cont

• In this case we can use Gerschgorin’s theorem to

bound the magnitude of the eigenvalues.
 0 8 1 1 8 
 8 0 8 1 1 
 
 1 8 0 8 1 
 
 1 8 0 8 1 
1  1 8 0 8 1 
D  
12dx  1 8 0 8 1 
 1 8 0 8 1 
 
 1 8 0 8 1 
 1 1 8 0 8 
 
 8  1 1 8 0 
• The Gerschgorin disk for each row is centered at
zero with radius: c  8  1  1  8   3c
12dx 2dx
CAAM 452 Spring 2005
 Cont (dt estimate)

• The skew symmetry tells us the eigenvalues are on

the imaginary axis so we know that if:

 3cdt 3cdt 
  2dx i, 2dx i 

• is in the stability region we have sufficient

conditions for stability of the ODE.

• This is a pretty good estimate, as the eigenvalues

given by Matlab for the 10x10 matrix dxD are:

CAAM 452 Spring 2005

 Comment on Gerschgorin Bound
>> [v,dxD] = test4th(10); >> 12*dxD
>>
>> v ans =

v= 0 8 -1 0 0 0 0 0 1 -8
-8 0 8 -1 0 0 0 0 0 1
0 - 1.36603956377562i 1 -8 0 8 -1 0 0 0 0 0
0 - 1.17011114634479i 0 1 -8 0 8 -1 0 0 0 0
0 - 0.94222308910582i 0 0 1 -8 0 8 -1 0 0 0
0 - 0.62520425034077i 0 0 0 1 -8 0 8 -1 0 0
0 - 0.00000000000000i 0 0 0 0 1 -8 0 8 -1 0
0 + 0.00000000000000i 0 0 0 0 0 1 -8 0 8 -1
0 + 0.62520425034077i -1 0 0 0 0 0 1 -8 0 8
0 + 0.94222308910582i 8 -1 0 0 0 0 0 1 -8 0
0 + 1.17011114634479i
0 + 1.36603956377562i

• The Gerschgorin estimate was pretty good in this

case (we suggested eigenvalues in i*[-1.5,1.5] )
CAAM 452 Spring 2005
 Note on Those Eigenvalues
• What should the eigenvalues of the derivative operator ideally tend to
in the limiting of decreasing dx?
• Example (10, 20, 80 points with the periodic length 2*pi):
>> [dxDeigs, dxD] = test4th(10); >> [dxDeigs, dxD] = test4th(20); >> [dxDeigs, dxD] = test4th(80);
>> dx = 2*pi/10; >> dx = 2*pi/20; >> dx = 2*pi/80;
>> sort(dxDeigs/dx) >> >> e = sort(dxDeigs/dx);
>> e = sort(dxDeigs/dx); >>
ans = >> e(1:10) >> e(1:10)

0.0000 - 0.0000i ans = ans =

0.0000 + 0.0000i
0.0000 - 0.9950i -0.0000 0.0000 - 0.0000i
0.0000 + 0.9950i -0.0000 0.0000 + 0.0000i
0 - 1.4996i -0.0000 - 0.9997i -0.0000 - 1.0000i
0 + 1.4996i -0.0000 + 0.9997i -0.0000 + 1.0000i
0 - 1.8623i 0.0000 - 1.6233i -0.0000 - 1.6639i
0 + 1.8623i 0.0000 + 1.6233i -0.0000 + 1.6639i
-0.0000 - 2.1741i -0.0000 - 1.9901i -0.0000 - 2.0000i
-0.0000 + 2.1741i -0.0000 + 1.9901i -0.0000 + 2.0000i
0.0000 - 2.9290i 0.0000 - 2.9997i
0.0000 + 2.9290i 0.0000 + 2.9997i
CAAM 452 Spring 2005
 cont

• From Fourier analysis we know that the field u, if sufficiently

smooth has a Fourier representation. i 2 x
u   uˆ e L


• Taking the first derivative we achieve:
d  i 2 x
 2 i 2 x
e L
 u
ˆ e L
dx   
 L
i 2 x
du 2
  uˆ e L
dx  L

• Also, each Fourier mode is an eigenfunction of the

differentiation operator with eigenvalue
2
  for all   
L CAAM 452 Spring 2005
 2
cont   for all   
L
• By the completeness of the Fourier
modes, we would hope the
eigenvalues of the discrete >> [dxDeigs, dxD] = test4th(80);
approximation differentation operator >> dx = 2*pi/80;
would converge to 0, 1, 2, 3,... >> e = sort(dxDeigs/dx);
>>
>> e(1:10)

• However, we see an oddity ans =

0.0000 - 0.0000i
0.0000 + 0.0000i
• There is the +/-1.6639 in there. -0.0000 - 1.0000i
-0.0000 + 1.0000i
-0.0000 - 1.6639i
• More about this later. -0.0000 + 1.6639i
-0.0000 - 2.0000i
-0.0000 + 2.0000i
0.0000 - 2.9997i
0.0000 + 2.9997i
CAAM 452 Spring 2005
 Using RK4
 n
a  dtf  u   cdtDu
 n
  n 
b  dtf  u  a / 2   cdtD  u  a / 2 
n 
 n  n 
c  dtf  u  b / 2   cdtD  u  b / 2 

  n 
d  dtf  u  c   cdtD  u  c 
n 

n1 n 1    
u  u   a  2b  2c  d 
6
To avoid confusion with subindices for the vector, we have
used super indices to represent the time level.

CAAM 452 Spring 2005

Using AB4

CAAM 452 Spring 2005

 That’s a Scheme

• Without much trouble we have created a stable finite

difference scheme for the advection equation – given the
local truncation errors are small we might expect that the
scheme is high order too.

• We will examine the accuracy and conditions for

convergence of this kind of finite difference scheme next
lecture.

• However, I did steer the choice of spatial difference

discretization towards a skew symmetric operator 

• I will motivate the choices next lecture.

CAAM 452 Spring 2005
 Homework 2 (due 02/03/2005)
Q1) Implement the 6 stage, 5th order Runge-Kutta from:

http://www.library.cornell.edu/nr/bookcpdf/c16-2.pdf

Use equations: 16.2.4 , 16.2.5, 16.2.6 with coefficients at the top of table
717 (Cash-Karp coefficients provided on the website)
du
Q1a) Solve  cos  t  u 0  t  1
dt
u  0  1
Q1b) Estimate dtmax by concocting a Lipschitz constant for the right hand
side of the ODE.

Q1c) Solve 4 times, using (i) dt=dtmax (ii) dt=dtmax/2 (iii) dt=dtmax/5 (iv)
dt=dtmax/6At each time step estimate the error (using 16.2.7) and
compare with actual error (plot both on a graph, use log scale if
necessary).
CAAM 452 Spring 2005