Chapter 1.

Partial Differential Equations (PDEs)

Required Readings:
Chapter 2 of Tannehill et al (text book)
Chapter 1 of Lapidus and Pinder (Numerical Solution of Partial Differential Equations in Science and
Engineering - web link)
Supplementary Reading: P1-P20 of Durran book.

Before we look at numerical methods, it is important to understand the types of equations we will be dealing with.

1. Differences between PDE's and ODE's

1) PDE's contain >1 independent variable, e.g.,

F ( x, y, u , ∂u / ∂x, ∂ 2u / ∂x∂y,....) = 0

whereas ODEs (Ordinal Differential Equations) contain 1 independent variable:

du
= F ( x, u ) .
dx

1
2) In an ODE, a specification of (x, u), for the above example, yields a unique value of du/dx, because there is
only one direction (x) that one can move.

For a similar PDE,

F(x, u, ux, uy, y)= 0,

specifying (x, y, u) only relates ux to uy (through the PDE equation), but does not uniquely determine either.

2
 For a PDE, the solution surfaces pass through a curve in a 3-D space, rather than a curve through a point in
the ODE.

3) ODE's have nice convergence proofs when solved using iterative methods. Therefore, ODE's can be solved
quickly because convergence is guaranteed. You will see a lot of black box ODE solvers in standard libs
such as LINPAK, IMSL, but not so many PDE's.

2. Properties of PDE's

1) Order – the order of the highest partial derivative present,

∂u ∂ 3u
e.g., =a 2 is 3rd order
∂t ∂ x∂y

We'll focus primarily on 1st and 2nd order equations

3
 Note that high-order equations may often be written in terms of a low-order systems of equations, e.g.,

∂w
wxx + wyy = wxz ( wx ≡ )
∂x

can be written as

ux + vy = uz ,
u = wx ,
v = wy .

Another example – the shallow water equations

∂u ∂h
= −g
∂t ∂x ∂ 2u 2 ∂ u
2
=> = −c where c = gH .
∂h ∂u ∂t 2 ∂x 2
= −H
∂t ∂x

The order has important implications because of the number of boundary conditions required and the
classification of the equation in the canonical or standard form.

2) Linearity – In simplest terms, nonlinearity implies a feedback.

4
Linear example: If you are a linear eater, the amount you eat does not affect your appetite. For linear
equations, the actual solution, such as the amplitude of the sine wave solution, does not affect the behavior
of the equation - i.e., there is no feedback.

Nonlinear example: If you are nonlinear eater, the more you eat, the more you can eat and the heavier you
become (your properties change), and the more you need to eat.

The linearity property is crucial for solving PDE's – it determines the techniques we use, etc.
Linear equations are much easier to solve, especially analytically.
Properties of nonlinear equations are often discussed after they are linearized.

Definition: An operator L( ) is linear if

L(α u + β v) = α L(u ) + β L(v)

where α and β are constant. This is a universal test!

Note that, since analytical solutions are often available to linear equations, we tend to linearize complex
systems to gain a better understanding of them, at least in the vicinity where the linearization occurs.

∂ 2 u ∂ 2u
Example: L(u ) = 2 + 2 is linear.
∂x ∂y

∂ 2u
Verify that L(u ) = u 2 is not linear.
∂x

5
 ∂u ∂u
Example: + c = 0 (c > 0 & constant)
∂t ∂x

is a linear equation. Solutions of a linear equation can be superimposed.

Equation

∂u ∂u
+u =0
∂t ∂x

is nonlinear. Solutions of a nonlinear equation generally can not be superimposed, i.e., the sum of 2 solutions
to the equations does not yield a correct third one.

To be mathematically rigorous, we can use the following:

Consider,
∂u ∂u
a (ξ ) + b(ξ ) = c(ξ ) .
∂x ∂y

If a, b, c = constant or ξ=ξ(x, y), it is Linear.

If ξ=ξ(u, ∂u/∂x, ∂u/∂y, un(n>1)), i.e., if one of the coefficients is a function of u, or derivatives of u, it
is Nonlinear.

6
3. Classification of second-order PDE's

(Reading Assignment: Sections 1.1.1, 1.2.1, 1.2.2 in Lapidus and Pinder).

There are three standard types for PDE's:

Hyperbolic
Parabolic
Elliptic

Consider a linear second-order PDE with 2 independent variables

(can be generalized to >2 cases):

∂ 2u ∂ 2u ∂ 2u ∂u ∂u
A 2 +B +C 2 + D + E + Fu + G = 0, (1)
∂x ∂x∂y ∂y ∂x ∂y

where A, B, …, G are constants or functions of (x, y). It turned out that this equation is

Hyperbolic if B2 – 4AC > 0,

Parabolic if B2 – 4AC = 0, (2)
Elliptic if B2 – 4AC < 0.

We will discuss more later to see why.

Note that the definition depends on only the highest-order derivatives in each independent variable.

Example: utt – c2 uxx = 0 (wave eq.) H

ut = c uxx (Diffusion eq.) P

7
 uxx + uyy = 0 (Laplace eq.) E

In order to understand this classification, we need to look into a certain aspect of PDE's known as the
characteristics.

4. Canonical or standard forms of PDE's

4.1. Three Canonical or Standard Forms of PDE's

Every linear 2nd-order PDE in 2 independent variables, i.e., Eq.(1) can be converted into one of three
canonical or standard forms, which we call hyperbolic, parabolic or elliptic.

Written in new variables ξ and η, the three forms are:

uξξ - uηη + …. = 0
or uξη + … = 0 H (3a)

uξξ + … = 0 P (3b)

uξξ + uηη + …. = 0 E (3c)

In this canonical form, at least one of the second order terms is not present.

We will see that hyperbolic PDE has two real characteristic curves, the parobolic PDE has one real
characteristic curve, and the elliptic PDE has no real characteristic curve.

Examples utt – c2 uxx = 0 (wave eq.) H

8
 ut = c uxx (Diffusion eq.) P
uxx + uyy = 0 (Laplace eq.) E

are already in the canonical forms.

The classification of some equations may depend on the value of the coefficients – need to use criteria in (2)
to determine. E.g.,

y uxx + uyy = 0 Elliptic for y > 0 and hyperbolic for y < 0.

4.2. Canonical Transformation

Consider again the general linear second-order PDE with 2 independent variables:

∂ 2u ∂ 2u ∂ 2u ∂u ∂u
a 2 +b +c 2 +d + e + fu + g = 0, (4)
∂x ∂x∂y ∂y ∂x ∂y

Introduce transform

ξ = ξ (x,y), η = η(x, y), (5)

which essentially transforms the PDE from coordinate system (x, y) to coordinate system (ξ, η).

Using the chain rule =>

[since u(x, y) = u(ξ(x, y), η(x, y))]

u x = u ξ ξx + u η η x

9
 u y = u ξ ξy + u η η y

uxx = uξξ ξx2 + 2uξηξxηx + uηη ηx2 + … (6)

uxy = uξξ ξxξy + uξη(ξxηy+ ξyηx) + uηη ηxηy + …
uyy = uξξ ξy2 + 2uξηξyηy + uηη ηy2 + …

The terms not including any 2nd-order derivative are not explicitly written out.

Substituting the derivatives in (6) for those in (4) yields

a uxx + b uxy + c uyy = A uξξ + B uξη + C uηη + …, (7)

where

A = a ξx 2 + b ξx ξy + c ξy 2

B = 2aξxηx + b(ξxηy+ ξyηx) + 2cξyηy (8)

C = a ηx2 + b ηxηy + c ηy2

From (8), we can obtain (show it for yourself!)

B2 – 4AC = (b2 – 4ac) (ξxηy - ξyηx)2. (9)

Note, B2 – 4AC and b2 – 4ac always have the same sign, as long as ξxηy - ξyηx ≠ 0!

This is why the only the coefficients of the second order derivative terms matter, and transformations from
one coordinate system to another does not change the sign of the criterion.

10
Therefore, nonsingular (where ξxηy - ξyηx ≠ 0) coordinate transformation does not change the type of PDE.

∂ (ξ ,η ) ξ x η x
Note that in (9), = = ξ xη y − η xξ y is the Jacobian of transformation therefore it can not be zero.
∂ ( x, y ) ξ y η y

Otherwise there will not be a one-to-one mapping between the two coordinate systems, in another word, the
transformation becomes singular.

Consider the case of b2 – 4ac > 0, i.e., the hyperbolic case, let's show that Eq.(4) can be reduced to a
canonical form as in (3a). Let consider the case of uξη + … = 0.

To achieve this form, we require that A and C given in (8) vanish, i.e.,

a ξx 2 + b ξx ξy + c ξy 2 = 0 (10a)
a ηx2 + b ηxηy + c ηy2 = 0 (10b)

Let
λ1 = ξx/ξy and λ2 = ηx/ηy, (11)

We find Eqs.(10a,b) can be satisfied when

a λ12 + b λ1 + c = 0 (12a)
a λ22 + b λ2 + c = 0 (12b)

11
 Obviously, the solutions of λ1 and λ2 are

−b ± b 2 − 4ac
λ1,2 = (13)
2a

Therefore, we see that if b2 – 4ac > 0, we can find 2 real roots of λ so that (10) is satisfied and the general
2nd-order PDE (4) can be transformed into the standard form like uξη + … = 0 which is hyperbolic.

4.3. Characteristic Equations and Characteristic Curves

Notice that equations in (11) are actually two 1st-order PDE's. They can be re-written as

ξx − λ 1 ξy = 0 (14a)
η x − λ2 η y = 0 (14b)

We will see that from them we can obtain two sets of characteristic curves.

The number of real characteristics of a PDE can actually determine its type.

Concept of Characteristics

Classification of PDE's is actually based on the mathematical concept of characteristics.

Characteristics are lines (in 2D problems, defined in terms of the number of independent variables) or
surfaces (in 3D problems) along which certain properties remain constant or certain derivatives may be
discontinuous.

12
Such lines or surfaces are related to the directions in which "information" can be transmitted in physical
problems governed by PDE's.

Because of this property, many methods developed before the digital computers for solving PDE's are based
on the characteristics and compatibility equations. The latter describes the conservation property of the
'information' along the characteristics.

• Equations (single or system) that admit wave-like solutions are known as hyperbolic.
• Those admitting solutions for damped waves are called parabolic.
• If the solutions are not wave-like, they are called elliptic.

It is important to know which type we are dealing with in order to choose the numerical method, the
boundary conditions, etc. Further, different physical interpretations are attached to different types, as we can
see from the above discussion.

Characteristic Equations of 1st-order PDE's

Let's go back and look at 1st-order PDE's in the following general form:

A ux + B u y = C (15)

Solution u represents a curved surface in a 3D space [u = u(x, y) ].

It can be shown that vector

G
F = ( A, B, C )

13
 G
is tangent to the surface where A, B and C are the components of vector F .

Diagram of a surface in a 3D space:

G
Because the downward normal of the surface at a given point P(x,y) is N = (u x , u y , −1) (consult your
G G
calculus text book) and F ⋅ N = Au x + Bu y − C = 0 .

Therefore, the PDE can be geometrically interpreted as the requirement that any solution surface
through point P must be tangent to the coefficient vector (A, B, C).

Question: For A, B and C that are all constant, what do the solution surfaces look like?

We also know, from u=u(x, y),

14
 du = u x dx + u y dy (16)

In both (15) and (16), ux and uy can take on more than one value but still satisfy these equations.
G G
This is because for a given vector F = ( A, B, C ) , there can be many vector N = (u x , u y , −1) that is
G G G
perpendicular to F . F ⋅ N = 0 is the only requirement for u to satisfy Eq.(15). Similar can be said of Eq.(16).

Therefore, ux and uy are non-unique (c.f., the diagram for three-D surface). This is important and we will
use this property to obtain the characteristic and compatibility equations.

Using the terminology of linear algebra, we actually have two linearly dependent equations.

Write these two equations in a matrix form:

⎛ A B ⎞ ⎛ ux ⎞ ⎛ C ⎞
⎜ dx dy ⎟ ⎜ u ⎟ = ⎜ du ⎟ (17)
⎝ ⎠⎝ y ⎠ ⎝ ⎠

⎛ ux ⎞
For ⎜ ⎟ to have more than one possible solution, the determinant of the coefficient matrix needs to be zero,
⎝ uy ⎠
i.e.,

A B dx dy
= 0. => = (18)
dx dy A B

15
Recall from linear algebra that, if a square coefficient matrix for a set of n linear equations has a vanishing
determinant, then a necessary condition for finite solutions to exist is that when the RHS is substituted for
any column of the coefficient matrix, the resulting determinant also vanish (c.f., the Cramer's rule for solving
linear systems of equations). Therefore we have

C B du dy
= 0 => = (20)
du dy C B

A C du dx
= 0 => = (21)
dx du C A

dx dy du
or = = (22)
A B C

They actually represent two independent ODE's. They fully determine our system apart from B.C. and I.C.
conditions, and they can be solved much more easily than the original PDE.

The equation

dy B
= (23)
dx A

is called the characteristic equation, and if A and B = constant, we have a family of parallel lines. Given the
initial and boundary conditions, we can obtain the solution to our equation.

16
For example, when A=1, B=β, C=0 (remember at least one of them has to be zero in one of the canonical
forms), and let x Æ t, y Æ x, we have

∂u ∂u
+β = 0, β > 0 and const. (24)
∂t ∂x

The characteristic equation is

dx
= β => x = βt + const. (const. to be determined from I.C.) (25)
dt

The solution represents a family of parallel lines in x-t space.

From (20) or (21), we have

du = 0 - which is called the Compatibility Equation. (26)

It says that u is conserved along the characteristic lines (c.f., earlier discussion of the properties of
characteristics).

Note that for general cases, the compatibility equation can only be obtained with the aid of the characteristic
equation. It is therefore said to be only valid alone the characteristics.

Diagram:

17
 t=0 x

4.4. Method of Characteristics (MOC)

Reading: Section 6.2 of Textbook.

For the previous problem, if we know the I.C. and B.C., we can use the Method of Characteristics (MOC) to
find the exact solution at any point (x,t) in the solution space.

u(x1, t1) = u(x0, 0) = f(x0) = I.C. (26)

Since x1 - β t1 = x0 + β 0 Æ x0 = x1 - β t1, the general solution is

u(x1, t1) = u(x0, 0) = f(x1 - β t1) (27)

18
where the functional form of f is specified by the I.C.

There general solution is therefore

u(x, t) = u(x0, 0) = f(x - β t) (28)

– it says that the solution of u at x and time t is equal to the value of initial function at location x-βt.

In the above example, du = 0, u=const along the characteristic lines, it's a case of pure advection.

In general cases, the characteristic lines are not straight lines and du ≠ 0 so it may have to be integrated
numerically along the characteristic lines.

Still the integration of this equation is usually much easier than that of the original PDE (the procedure to
obtain the characteristic and compatibility equations can be non-trivial however).

Having discussed the characteristic equations for 1st-order PDE, let's go back to section 2). Write down
Eq.(14a) again here:

ξx − λ 1 ξy = 0 (29)

Compared to Eq.(15), A =1, B= -λ1, C=0, therefore according to (22), we have

dx dy dξ
= = . (30)
1 −λ1 0

19
From ηx − λ2 ηy = 0, we get

dx dy dη
= = . (31)
1 −λ2 0

From them, we obtain the characteristics and compatibility equations:

dy
= −λ1 and dξ = 0 (32)
dx
dy
= −λ2 and dη = 0 (33)
dx

Therefore we see that when b2 – 4 ac >0, 2 real roots can be found so that Eqs. (10) are satisfied, the original
2nd-order PDE can be converted to its canonical form, at the same time, two sets of characteristic curves
exist.

When λ1 and λ2 are constant, these characteristics are straight lines which correspond to constant coordinate
lines (ξ=const, η=const) in the new coordinate

ξ=const along y=λ1 x + C1

η=const along y=λ2 x + C2 (34)

One can see the coordinate transformation from (x, y) to (ξ, η) not only simplifies the original 2nd-order
PDE, but also simplify the characteristics. In a sense, the compatibility equations are the corresponding
characteristic equations in the new coordinate.

20
 We have shown that one of the two canonical form of hyperbolic equation, i.e., the form of uξη + … = 0, can
be obtained. To obtain the alternative canonical form,

uξ ξ − uηη + ... = 0 (35)

we can use a linear combination of ξ and η:

ξ = (ξ + η ) / 2 and η = (ξ − η ) / 2 . (36)

In another word, we can perform the another transform from (ξ, η) coordinate system to (ξ ,η ) and convert
equation uξη + … = 0 into the form Eq. (35). Of course, we can also perform a transform directly from the
original equation in (x, y) coordinate, by requiring that B = 0 in Eq.(7).

Similar analysis can be performed for the parabolic and elliptic cases, where, respectively, one and zero
characteristic curve exists.

Reading: Section 2.3 of Text.

Section 1.1.2 of Durran.

4.5. Domain of Dependence for second-order PDE's

Reading: Section 2.3.1 of Textbook.

Consider 2nd-order wave equation

21
 utt − c 2u yy = 0 , (37)

on the interval - ∞ < x < ∞ with initial condition

u(x, 0) = f(x) and ut(x,0) = g(x) (38)

With coordinate transform discussed earlier, (37) can be converted to

uξη = 0 (39)

where ξ = x + ct and η= x – ct.

(39) can be easily integrated twice, with respect to each of the new independent variables, we obtain

uξ = C '(ξ )

u (ξ ,η ) = ∫ C '(ξ )dξ + D (η ) = C (ξ ) + D(η )

which can be rewritten in terms of x and t as

u ( x, t ) = C ( x + ct ) + D( x − ct ) (40)

where C and D are arbitrary functions to be determined from initial conditions. When the functional form of
C and D are determined from the I.C. The resulting solution is

f ( x + ct ) + f ( x − ct ) 1 x +ct
u ( x, t ) = + ∫ g ( x ')dx ' (40a)
2 2c x −ct

22
which is called the D'Alembert solution (see web link).

Diagram of Domain of Dependence.

t D.O.I

(x0 ,t 0)

1/c -1/c

D.O.D

t=0
(x0 - ct 0) (x0 + ct0 ) x

From (40), we can see that solution u at a point (x0, t0) depends only on the initial data contained in the
interval

x0 − ct0 ≤ x ≤ x0 + ct0 ,

i.e., the solution is only dependent on the condition in a domain bounded by the two characteristic lines
through point (x0, t0).

23
 The 1st part of solution (40) represents propagation of signals along the characteristic lines and the 2nd part
the effect of data within the closed interval at t = 0.

We call this domain the Domain of Dependence (DOD).

General properties of hyperbolic PDE's:

• They have limited domain of dependence (DOD)

• Disturbances outside the DOD cannot influence the solution at a particular point
• Shows why hyperbolic equations usually describe initial value problems.
• I.C. cannot be specified on a characteristic line (only) – otherwise the problem is ill-posed, i.e., a
unique solution can not be found.

Domain of Influence

The characteristic lines encompass a region outside which signal at (x0, t0) cannot influence at a later time.
Furthermore, the signal can only propagate a finite distance in a finite time. The domain defined by

x0 + ct0 ≤ x ≤ x0 − ct0 for t > t0 ,

outside which point (x0, t0) cannot affect is called the Domain of Influence of this particular point.

The time-matching nature of hyperbolic equations is clear.

DOD of Diffusion Equations

Now consider 2nd-order diffusion (heat transfer) equation, a parabolic equation

24
 Tt = K Txx (41)

where K is the diffusion coefficient or thermal conductivity. Note that there is only one 2nd-order derivative
term in the equation – it's already in the canonical form of parabolic equations.

Its analytical solution is

T(x,t) = T(x,0) exp( -K k2 t ) (42)

where k is wavenumber. t Æ ∞ , TÆ 0 for non-zero wavenumbers.

The characteristic equation for this equation is

dt
=0 (43)
dx

25
=> the DOD is the entire domain below a given time t,
all points are diffused simultaneously, at a rate dependent on local gradient of T (c.f., next figure).

Comments:

• Solution depends on entire time history, is still a time matching problem but irreversible
• No MOC for diffusion equations
• Parabolic equations represent a smoothing process

Elliptic Problem

• Elliptic equations have no real characteristics along which signal might propagate.
• They are always boundary value problems
• They involves no time matching

E.g., uxx + uyy = 0 (Laplace's Eq.)

26
 uxx+ uyy = f(x,y) (Possion's Eq.)

They are 'diagnostic' equations, one disturbance introduced into any part of the domain is 'felt' at all other
points instantaneous.

Not a suitable problem for distributed memory computers, because of the DOD is the entire domain.

Example: In a compressible fluid, pressure waves propagate at the speed of sound.

The linearized equations are

∂u ' ∂u ' 1 ∂p '

+U + =0 (44a)
∂t ∂x ρ ∂x
∂p ' ∂p ' ∂u '
+U + ρ c2 =0 (44b)
∂t ∂x ∂x

Signals propagates at speed c – a finite propagation speed.

Now, if we make the fluid incompressible,

∂u
i.e., dρ/dt = 0 => = 0,
∂x

this is equivalent to setting c = ∞ in (44b). Therefore, in incompressible fluid, disturbance is 'felt'

instantaneously in the entire domain.

Let's see what equations we have to solve now.

27
For an impressible system:
G
∂V G G 1
+ V ⋅ ∇ V = − ∇p (45a)
∂t ρ
G
∇ ⋅V = 0 (45b)

Take ∇⋅ of (45a), and make use of (45b), we get

G G
∇ 2 p = − ρ∇ ⋅ (V ⋅ ∇V ) (46)

which is an elliptic equation.

Now we see the connection between the type of equations and physical property of the fluid they describe.

28
5. Systems of First-order Equations

Reading: Section 2.5 of Textbook.

In fluid dynamics, we more often deal with a system of coupled PDE's.

a) Definition and methods based on the eigenvalue of coefficient matrix

Often, high-order PDE's can be rewritten into equivalent lower-order PDE's, and vice versa.

∂v ∂w
E.g., −c =0 (5.1a)
∂t ∂x
∂w ∂v
−c =0 (5.1b)
∂t ∂x

∂ ∂
(5.1a ) + c (5.1b) =>
∂t ∂x

∂ 2u 2 ∂u
2
−c =0 (5.2)
∂t 2 ∂x 2

which is a 2nd-order wave equation (e.g., wave propagation along a string).

We can determine the PDE's type using the b2 – 4ac criterion and find the characteristic and compatibility
equations using the method discussed earlier.

For systems of equations, we give an alternative but equivalent definition and method for obtaining
characteristic and compatibility equations.

29
We write the system in a vector form:
G G
∂u ∂u
+A =0 (5.3)
∂t ∂x

where

G ⎛v⎞ ⎛ 0 −c ⎞
u = ⎜ ⎟ and A = ⎜ ⎟.
⎝ w⎠ ⎝ −c 0 ⎠

Definition:

• If the eigenvalues of A are real and distinct (n of them for nth-order matrix), the equation is hyperbolic.
• If number of real eigenvalues is >0 but < n, it's parabolic.
• If they are all complex, it's elliptic.

Note that symmetric matrix has real eigenvalues. As in the above case.

Why: If all eigenvalues are real, bounded matrix T and T-1 exist so that

T-1 A T = D,

where D is a diagonal matrix with real eigenvalues dii. Matrix T actually consists of, in each column, the
eigenvectors corresponding to the discrete eigenvalues.

30
 G G
Let u = T v , then
G G G G
∂v ∂v ∂v −1 ∂v
T + AT = 0 ⇒ + T AT = 0 ⇒
∂t ∂x ∂t ∂x
G G
∂v ∂v ∂vi ∂v
+ D = 0, i.e., + dii i = 0 for i = 1, n
∂t ∂x ∂t ∂x

which are n decoupled individual compatibility equations.

Reading: Section 2.5 of Textbook

Eigenvalues of the previous problem can be found from

−λ −c
| A - λ I | = 0 => = 0 => λ1 = c, λ2 = -c.
−c −λ

c and – c are the actual wave propagation speed of the wave equations

dx dx
= + c and = −c
dt dt

which are actually the characteristic equations.

31
a) Method using the auxiliary equations

A more general method for obtaining characteristic and compatibility equations for problem in (5.1) is to
make use of two auxiliary equations:

dv = vt dt + vx dx
dw = wt dt + wx dx

and write them in a matrix form:

⎛1 0 0 −c ⎞ ⎛ vt ⎞ ⎛ 0 ⎞
⎜0 ⎜ ⎟
⎜ −c 1 0 ⎟⎟ ⎜ vx ⎟ ⎜⎜ 0 ⎟⎟
=
⎜ dt dx 0 0 ⎟ ⎜ wt ⎟ ⎜ dv ⎟
⎜ ⎟⎜ ⎟ ⎜ ⎟
⎝0 0 dt dx ⎠ ⎜⎝ wx ⎟⎠ ⎝ dw ⎠

Setting determinant of the coefficient matrix, | | = 0 => dx/dt = ± c as before!

Find the compatibility equation on your own (hit: replace one of the columns of the coefficient matrix by the
RHS, and set the determinant to zero).

Note: This method is more general and is most often used to find the characteristic and compatibility
equations, for both single equations and equation systems. Make sure that you know how to apply this
method. The physical interpretation of this method is not as clear as the matrix method, however.

32
Example Problem

The 1-D linear inviscid shallow water equations can be written as

∂u ∂u ∂h
+U +g =0 (1)
∂t ∂x ∂x
∂h ∂h ∂u
+U +H =0 (2)
∂t ∂x ∂x

where U and H are the mean flow speed and unperturbed water depth, respectively, and u, and h are the
corresponding deviations of total flow speed and water depth from U and H.

1) Classify the system in terms of the canonical types;

2) Find the characteristic equations of the system;

Equations (1) and (2) can be written as

u t + U u x + 0 ht + g h x = 0 (3)
0 u t + H u x + ht + U h x = 0 (4)

We make use of two auxiliary equations

dt ut + dx ux + 0 ht + 0 hx = du (5)
0 ut + 0 ux + dt ht + dx hx = dh (6)

Write (3)-(6) in a matrix form,

33
 ⎡1 U 0 g ⎤ ⎛ ut ⎞ ⎛ 0 ⎞
⎜ ⎟
⎢0 H 1 U ⎥ ⎜ u x ⎟ ⎜⎜ 0 ⎟⎟
⎢ ⎥ = (7)
⎢ dt dx 0 0 ⎥ ⎜ ht ⎟ ⎜ du ⎟
⎢ ⎥⎜ ⎟ ⎜ ⎟
⎣0 0 dt dx ⎦ ⎜⎝ hx ⎟⎠ ⎝ dh ⎠

For the reason of linear dependence, we require the coefficient matrix [ ] to be zero:

1 U 0 g
H 1 U U 0 g
0 H 1 U
= dx 0 0 + dt H 1 U =0
dt dx 0 0
0 dt dx 0 dt dx
0 0 dt dx

∴ dxdtU − (dx) 2 + dtUdx + Hg (dt ) 2 − U 2 (dt ) 2 = 0

Reorganize the above equations, we have

(dx) 2 − 2Udxdt + (U 2 − Hg )(dt ) 2 = 0

Divide the equation by (dt)2, we obtain

(dx / dt ) 2 − 2U (dx / dt ) + (U 2 − Hg ) = 0 ,

which is a quadratic algebraic equations for (dx/dt). Its solutions are

34
 2U ± 4U 2 − 4(U 2 − Hg )
dx / dt = = U ± gH . (8)
2

From (8), we obtain two families of characteristics equations:

x = (U + gH ) t + C1 and x = (U − gH ) t + C2 (10)

where C1 and C2 are the integration constants. Varying the values of C1 and C2 lead to two families of
characteristics curves. We can recognize that gH is the phase speed of shallow water external gravity waves,
which propagate in two directions. U is the advective flow speed. U ± gH are the Doppler shifted phased speeds.

Because we can find two real characteristics equations for this system of two 1st order equations, the system is
hyperbolic.

3) In the case that real characteristics exist, find the corresponding compatibility equations;

To find the compatibility equations, we replace one of the columns of the coefficient matrix in (7) by the right-
hand-side vector. The determinant of this matrix also has to be zero. This leads to

1 U 0 0
H 1 0 U 0 0
0 H 1 0
= dx 0 du + dt H 1 0 =0
dt dx 0 du
0 dt dh 0 dt dh
0 0 dt dh

∴− Hdudt − dxdh + dtUdh = 0 .

35
Divide the equation by (-dt), we have

Hdu + ( dx / dt )dh − Udh = 0 .

Substituting for (dx/dt) using (8) yields (note characteristics equations are used when deriving the
compatibility equations):

Hdu ± gH dh = 0 , therefore

d ( Hu ± gH h) = 0 .
Hu ± gH h = C3,4 (11)

are the two compatibility equations. Hu ± gH h are call the Riemann invariant that are conserved along the
characteristics curves.

4) Suggest an application for characteristics and compatibility equations.

Along characteristics curves, PDE can often be reduced to ODE, which is much easier to solve. In our
example, two quantities are found (from compatibility equations) to be conserved along the characteristics lines.
They can be traced along the lines to the initial time and the boundary, where the known solutions are used to
determine the integration constants. Therefore the solution in the interior domain can be found by solving the two
conservation equations. The method of finding a solution of PDE by using characteristics and compatibility
equations in combination with initial and boundary conditions is called the Method of Characteristics (MOC).

36
An alternative solution to this problem is given below:

Rewrite (1) and (2) in a vector-matrix form:

∂V ∂V ⎡u ⎤ ⎡U g ⎤
+A = 0, where V = ⎢ ⎥ and A = ⎢ ⎥. (12)
∂t ∂x ⎣h⎦ ⎣H U ⎦
U −λ g
Find the eigenvalues of A by requiring = 0,
H U −λ
therefore the two eigenvalues are λ1,2 = U ± gH .
Because we found two (the same number as the number of 1st order equations) real eigenvalues , the system is
hyperbolic.

Now find the eigenvectors corresponding to eigenvalues λ1 and λ2 by solving

AX = λ1 X , AX = λ2 X . (13)

The solutions of (13), i.e., the two eigenvectors are

⎡ 1 ⎤ ⎡ 1 ⎤
X1 = ⎢ ⎥ 2 ⎢
, X = ⎥.
⎣⎢ H / g ⎦⎥ ⎢⎣ − H / g ⎥⎦
⎡ 1 1 ⎤ −1
⎡1 g/H ⎤
Therefore we have matrix T = ⎢ ⎥ and its inverse T = 1/ 2 ⎢ ⎥ (show it for yourself) so
⎣⎢ H / g − H / g ⎥⎦ ⎢⎣1 − g / H ⎥⎦
that

37
 −1 ⎡λ1 0 ⎤ ⎡U + gH ⎤
0
T AT = D where D = ⎢ ⎥=⎢ ⎥.
⎣ 0 λ2 ⎦ ⎢⎣ 0 U − gH ⎥⎦

−1 ∂ T −1V −1 ∂ T −1V
We multiple equation (12) by T , so that + T AT = 0 therefore
∂t ∂x
∂ V ⎡U + gH 0 ⎤ ∂ V
+⎢ ⎥ =0 (14)
∂ t ⎢⎣ 0 U − gH ⎥⎦ ∂ x

where
⎡1 g / H ⎤ ⎡u ⎤ ⎡u + g / H h ⎤
 −1
V = T V = 1/ 2 ⎢ ⎥ ⎢ ⎥ = 1/ 2 ⎢ ⎥ . (15)
⎢⎣1 − g / H ⎥⎦ ⎣ h ⎦ ⎢⎣u − g / H h ⎥⎦

We rewrite (14) into a system of equations:

(
∂ u + g/Hh )+ U+ (
∂ u + g/Hh ) = 0,
∂t
( gH ) ∂x (16)
∂ (u − g / H h ) (
∂ u − g/Hh ) = 0.
+ (U − gH )
∂t ∂x

We know and can easily show that for (16), the compatibility equations are
( ) ( )
d u ± g / H h = 0 or u ± g / H h = C1,2 along characteristics equations x = U ± gH t + C3,4 .
These are the same as what we found with the first method. Mathematically it's more elegant. The two equations in
(16) are now decoupled and can be solved independent of each other.

38
6. Initial and Boundary Conditions

We will devote an entire section later on to boundary conditions – for now, we will look at general
conditions.

I.C. and B.C. are

- needed to obtain unique solutions

- physically and/or computationally motivated

Initial Condition - Specification of the dependent variable(s) and/or its (their) time derivative(s) at same
initial time.

Boundary Condition – Specification of dependent variable(s) at a domain boundary. Given in a general form
for 2nd-order PDE's
G
G ∂u ( x , t )
α u( x, t ) + β =γ
∂n
∂u
where is the gradient of u in the direction normal to the boundary. α and β are constant coefficients.
∂n

- Dirichlet or 1st B.C. β =0 Æ value of variable specified

- Neumann or 2nd B.C. α = 0, gradient of value specified

- Robin or 3rd B.C., neither α nor β is zero. - A linear combination of the above two.

39
 Note for Possion's equation ∇ 2ϕ = ζ , if gradient boundary condition is specified at all boundaries, the
solution is unique only up to an arbitrary constant – additional condition has to be used to determine this
constant for a physical problem.

7. Concept of Well-posedness

The governing equations and the associated auxiliary conditions (I.C. and B.C.) are said to be well-posed
mathematically if:

- the solution exists

- the solution is unique
- the solution depends continually upon the auxiliary conditions - a small change in auxiliary
conditions results in small change(s) in the solution (the future state is predictable – an
important issue for the atmosphere).

Existence – usually there isn't a problem for CFD – it can be, however, in cases where singularities exist
somewhere in the domain.

Uniqueness – this can really be a problem in fluid flow problems – We can show uniqueness for simple
problems only.

Consider an example – how do we show solution is unique?

Look at the diffusion equation:

ut = Κ uxx (K>0, 0≤ x ≤ L )

40
 I.C. u(x,0) = f(x)

B.C. u(x=0, t) = u(x=L, t) = 0

This is a Well-posed Linear problem.

To show that a solution is unique, let's make a counter-hypothesis that 2 solutions exist:

u1 and u2, i.e., the solution is non-unique.

If u3 ≡ u1 – u2¸ then u3 satisfies

(u3)t = K (u3)xx

u3(x,0) = 0 I.C. Note the difference from the original I.C.

u3(0,t) = u3(L,0)=0 B.C.

Let's define an "energy" or variance for this system:

L 1 2
E (t ) = ∫ u dx u ∈ real
0 2

E is "positive definite" and is zero if and only if u=0 in the entire interval [0, L].

To derive the energy equation for our problem, multiple the PDE by u:

41
 u ( ut – K uxx ) = 0

(u2/2)t = K (u ux)x – K (ux)2

Integrate from 0 to L gives

∂E L
= − K ∫ (u x ) 2 dx
∂t 0

=> energy decreases with time at a rate that can be computed from u.

Now,
L 1 2
E3 (t ) = ∫ u3 dx
0 2

From I.C. u3 = 0 => E3 = 0 at t =0. Since E3 can not go negative, and it has an initial value of zero, it has to
remain zero for all t. For E3 to be zero, u3 has to be zero for all x and t.

Therefore u3=0 => u1 = u2 => the solution is unique!

Continuous Dependence on Auxiliary Conditions

A small or bounded change in the I.C. or B.C. should lead to small or bounded changes in the solution.

This doesn't necessarily apply to chaotic systems, where a small change in the I.C. can lead to very large
difference in the solution – in such cases, the solution can still be continuous and usually is.

42
A Classic Example of Discontinuous Solution Near the Boundary

Look at Laplace's equation:

∂ 2u ∂ 2u
+ =0 − ∞ < x < ∞, y ≥ 0
∂x 2 ∂y 2

For this 2nd-order PDE, we need two boundary conditions.

We specify the conditions at y=0:

u(x,0) = 0

uy(x,0) = sin(nx)/n n>0.

Note n here is a coefficient in the boundary condition.

Using the method of separation of variables, we can show the solution to be

u(x,y) = [sin(nx) sinh( ny )]/n2

Is there continuous dependence on the B.C.?

Now, from our solution, we have

uy(x,0) = sin(nx) cosh(0) / n = sin(nx)/n,

43
so this works. But, does it work for all values of n?

We see that for the second B.C., there is no problem:

uy(x,0)→ 0 as n → ∝.

Looking at the first B.C.,

1 e ny − e − ny
u ( x, y ) = 2 sin(nx)
n 2

As n → ∝., the above → eny/n2 which grows without bound even for small y!

On the other hand, we have B.C. u(x,0) = 0, so the continuity with the boundary data is lost – the problem is
ill posed.

Actually, this problem requires the solution of the PDE on an open domain.

44
Summary of Chapters 0 and 1 – What you should know:

Basic computer architectures, CPUS types, memory hierarchy, common computer terminologies.
Current trend in moving toward distributed memory massively parallel systems
Superscalar, pipelining, vectorization and parallelization issues, Amdahl's Law
Code optimization issues

Canonical forms of second-order PDE's

Classification of first-order, second-order PDE's and systems of first-order PDE's
Methods for finding characteristic and compatibility equations and their solutions
Use method of characteristics to solve simple problems
Concept of domain of dependence and domain of influence
Basic types of I.C. and B.C.
Know something about the well-posedness of PDE systems

45
Chapter Two. Finite Difference Methods

2.1. Introduction

2.1.1. The Concept of Finite Difference Method

In FDM, we represent continuous fluid flow problems in a discrete manner, when the fluid continuum is replaced
by a mesh of discrete points. The same is true for the time variable.

FDM are the simplest of all approximations, and involve a mapping:

Discretization
PDE Æ System of algebraic equtions

Calculus Æ algebra

Derivative Æ difference

We focus on the following:

- Properties of FDM
- Derivation via several methods
- Physical interpretation in terms of characteristics
- Application to selected problems

First, we lay down a convention for notion:

1
Time level - superscript n - ρn ~ ρ at time level n

Δt = time interval = tn+1 – tn.

Most times, we use constant Δt. Occasionally, Δt changes with time.

n-1 ~ past
n ~ present
n+1 ~ future

t = n Δt where n = number of time steps = 0, 1, 2, 3, ….., N

T = N Δt = final time.

Spatial Location – subscript i, j, k, for x, y, and z.

Δx – constant grid interval

xi = i Δx

2
Note: Discretization Î information loss – the greater the number of points, the more accurate will be the
representation. See Figure.

3
2.1.2. Quantitative Properties of Numerical Algorithms

The governing equations (PDE's) have certain properties, and their computational counterparts should also do so.

1) Conservation – Typically the governing equations are written as conservation laws (which means that the
integral properties over a closed volume don't change with time).

E.g., the mass conservation equation

∂ρ G
= −∇ ⋅ ( ρV ) .
∂t

If we integrate this over a closed box

∂ G
∂t ∫Ω ∫Ω
ρ dV = − ∇ ⋅ ( ρV )dv = 0

Mathematically, we can also write this as

∂ρ G G
= − ρ∇ ⋅ V − V ⋅ ∇ρ
∂t

Will the numerical solution obey these rules? Not necessarily.

G
Consider the situation where ρ and V are defined at separate points, … this is how the continuity equation is really
derived:

4
 G
The mass within the zone changes due the mass fluxes
G through the sides. To get ρ V at a ρ point in this case (we
are using a staggered grid), we have to average V to ρ point, which smears out gradients!

Consider an alternative structure:

To calculate the fluxes through the sides of the grid cells shown above (which is a non-staggered Arakawa A-grid,
by the way – we will come to it later), we have to perform different averaging, which result in different
conservation properties of the numerical scheme.

For numerical solution to obey conservation, you must be very careful how you set up the grid, formulate the terms
in the FD form, and solve the equations!!

5
2) Positivity – Physically positive quantities (mass, energy, water vapor) cannot become negative. This is not
guaranteed with numerical solutions, however. Care must be taken to prevent negative values from being
generated. Schemes that do so are called positive-definite schemes. A more general type is the monotonic
schemes that also ensure positive definiteness, because they cannot generate new extrema that are not found in
the original field.

3). Reversibility – Says that the equations are invariant under the transform t Æ - t. This is important for pure
transport problems, but clearly not appropriate for diffusion problems. Reversibility is actually hard to achieve
even for simple advection/transportation due to unavoidable numerical errors.

4). Accuracy – Accuracy generally involves Computer precision, Spatial or temporal resolution, and algorithm
robustness, etc.

Some of the most accurate schemes don't satisfy the above properties!!

6
2.2. Methods for Obtaining FD Expressions

There are several methods, and we will look at a few:

1) Taylor series expansion – the most common, but purely mathematical.

2) Polynomial fitting or interpolation – the most general ways. Taylor series method is a subset of this method.
Interpolation takes us back to the M.O.C. and thus has a more physical interpretation.

3) Control volume approach – also called finite volume (FV) – we solve the equations in integral rather than
differential form. Popular in engineering where complex geometries and coordinate transformations are
involved. For Cartesian grids, simplest FV methods Æ FD.

We will look at only the first two approaches.

2.2.1. Taylor Series Expansion Method

Recall the definition of a derivative:

∂u u ( x0 + Δx, y0 ) − u ( x0 , y0 )
= lim
∂x x0 , y0
Δx →0 Δx

The Taylor series approach works backwards – want to approximate ∂u/∂x by a discrete difference, i.e., for finite
Δx.

Given u(x0, y0), we can write a Taylor series expansion for u(x0+Δx, y0), as

1
 ∂u (Δx) 2 ∂ 2u (Δx)3 ∂ 3u (Δx) n ∂ nu
∞
u ( x0 + Δx) = u ( x0 ) + Δx + + + ... = ∑
∂x x0 , y0 2! ∂x 2 x0 , y0
3! ∂x 3 x0 , y0 n =0 n! ∂x n

This expression is exact if we retain all terms!

The grid mesh or stencil looks like:

Here, ui+1,j = u( x0+Δx, y0)

ui,j+1 = u( x0, y0+Δy)
etc.

If we solve for ∂u/∂x from the Taylor series, we have

∂u u ( x0 + Δx, y0 ) − u ( x0 , y0 ) Δx ∂ 2u (Δx) 2 ∂ 3u
= − − + ... (1)
∂x x0 , y0 Δx 2! ∂x 2 x0 , y0
3! ∂x 3 x0 , y0

2
The first term on the RHS is simply the slope of the function u(x,y), using the current and the points to the right.

∂u u ( x0 + Δx, y0 ) − u ( x0 , y0 )
Therefore, = + O(Δx) .
∂x x0 , y0 Δx

One can also use the value at x0 − Δx instead to get

∂u u ( x0 , y0 ) − u ( x0 − Δx, y0 ) Δx ∂ 2u (Δx) 2 ∂ 3u
= + − + ... (2)
∂x x0 , y0 Δx 2! ∂x 2 x0 , y0
3! ∂x 3 x0 , y0

Both (1) and (2) provide an expression for ∂u/∂x, but numerically the answers will be different.

3
 (1) is called a forward difference
(2) is called a backward difference

Consider a 1-D example:

If we have the equation

∂u ∂u
+ c = 0, where c > 0,
∂t ∂x

we might want to use

∂u u − ui −1
+c i ≈ 0,
∂t Δx

which is called Upwind or Upstream Difference. Alternatively, in

∂u u −u
+ c i +1 i ≈ 0,
∂t Δx

Downstream Difference is used.

4
Upstream difference is better than downstream difference for this problem, because for this pure advection problem,
signals move from upstream (left) to downstream (right). The value of u at point i at a future time should be
influenced by the values of u upstream, not downstream.

Think of it in terms of characteristics:

No information is coming from the +x direction. This of course depends on the sign of c. If C<0, then upstream
means the left side of the current point.

Note that upstream or upstream-biased schemes are usually better choices in CFD (obviously we are talking about
hyperbolic equations that represent propagations!).

We will see later, that 1st-order downstream difference is absolutely unstable for the above problem.

We can get another discrete approximation to ∂u/∂x by adding (1) and (2):

5
 ∂u ui +1 − ui −1 (Δx) 2 ∂ 3u
= − + ... (3)
∂x x0 , y0 2 Δx 3! ∂x 3 x0 , y0

This is called Centered Difference. Note, it doesn't even use value of u at the current point i. It approximates the
slope using two neighboring points:

Note that this will not be accurate if u has very sharp gradient (e.g., has a shape of ∧). Note also that the extra
terms in (1), (2) and (3) are different – they control the order of accuracy of the scheme.

We can build many different approximations to the derivatives through linear combinations of various T.S.
expansions.

6
For high-order derivatives, we follow the same approach.

Consider ∂2u/∂x2. The Taylor series gives

∂u (Δx) 2 ∂ 2u (Δx)3 ∂ 3u
u ( x + Δ x ) = u ( x ) + Δx + + + ... (4)
∂x 2! ∂x 2 3! ∂x 3

∂ 2u ∂u
We can solve for from (4), but we don't want to have the unknown .
∂x 2 ∂x
We can replace it with one of the earlier approximations to it, or make use of the following:

∂u (Δx) 2 ∂ 2u (Δx)3 ∂ 3u
u ( x − Δx) = u ( x) − Δx + − + ... (5)
∂x 2! ∂x 2 3! ∂x3

∂ 2u
and add (4) and (5) and solve for :
∂x 2

∂ 2u u ( x + Δx) − 2u ( x) + u ( x − Δx)
= + O(Δx 2 )
∂x 2
( Δx ) 2

∂ 2u
which is a centered difference for 2 .
∂x

7
Truncation Error

The high-order terms (H.O.T.) of O(Δxn) are called the Truncation Error, and are a measure of the error associated
with representing a PDE by a truncated T.S. – we can't retain all terms, so we neglect the terms of order O(Δx2) and
above, for example.

PDE = FDE + τ

where τ is the Truncation Error.

If we change the nature of the H.O.T., by using a different approximation, the accuracy of the F.D. expression will
also change.

It is important to understand the impact of τ, and this is the subject of our computer problem #4.

From the above, it's clear that we want τ Æ 0 when PDE = FDE. Otherwise, we have a problem (consistency)!

Let Δx Æ 0, then τ should Æ 0 => our discrete system approaches continuum and our FDE Æ PDE.

Order of the F.D. Approximation.

The power to which the leading discrete interval in τ is raised is called the Order of the F.D. Approximation.

ui − ui −1
Example: is first-order accurate in space
Δx

8
 Δx ∂ 2 u
because τ = + ... = O(Δx1 )
2! ∂x 2

ui +1 − 2ui + ui −1
is second order accurate.
Δx 2

Comments on τ:

(1) If there are several independent variables, each has a truncation error, e.g., O(Δx2+ Δt), we say it's first order
in time and second order in space.

(2) The order of a scheme also depends on the local properties of the function, it may be much less than the
formal or theoretical order near sharp gradients. Recall that Taylor series are valid only for smooth and
continuous functions.

(3) Typically, we prefer higher-order scheme because τ is smaller. However, τ is not necessarily related to
accuracy because, for a given Δx, a first-order scheme may give more accurate results because the coefficient
is smaller. What does τ tells us is how the error will change as we change the resolution. Higher order τ
decreases faster when we decreases Δx.

e.g., τ = κ (Δx)3 versus τ = κ Δx.

∂ 4u
Remember that τ includes κ, which might be 4 , etc and it matters.
∂x

(4) Truncation error is cumulative – adds up per time step.

9
 (5) Truncation error is usually much larger than machine round-off error. Also for most problems, τ(space) >>
τ(time), because solution often evolves smoothly in time but have rapid changes in space. Time step size is
often not as large as we wish because of stability constraint. For meteorological models, we usually want to
increase time step size and decrease grid spacing.

General Method for Deriving FD Expressions

Let's write a generalized form of the Taylor series as

∞
⎛ ∂ mu ⎞ (Δx) m
ui +1 = ∑ ⎜ m ⎟
m = 0 ⎝ ∂x ⎠i m!

Now, suppose we want to use a 3-point stencil at i-1, i, i+1. Then, we can write a generic expression (PDE = FDE
+ τ) as:

∂u
= aui −1 + bui + cui +1 + O(Δx) m
∂x

where a, b and c are unknown constants to be determined and m is the order of the approximation. (General rule: If
F.D. spans n points, you can derive an n-1 order F.D. scheme).

Using our Taylor series for ui-1, ui and ui+1, we can write

10
 aui −1 + bui + cui +1 =
∂u (Δx) 2 ∂ 2u (Δx)3 ∂ 3u
a (ui − Δx + − + ...)
∂x 2! ∂x 2 3! ∂x3
+ bui
∂u (Δx) 2 ∂ 2u (Δx)3 ∂ 3u
c(ui + Δx + + + ...)
∂x 2! ∂x 2 3! ∂x3
∂u ( Δx ) 2 ∂ 2 u (Δx)3 ∂ 3u
= (a + b + c)ui + (− a + c)Δx + (a + c) + (− a + c ) + ... (6)
∂x 2! ∂x 2 3! ∂x3

∂u
Since we want (6) to have the form of + O(Δx) m , therefore we set
∂x
(a + b + c) =0,
(-a + c)Δx =1
a + c=0.

From them we can find b =0, c = - a = 1/(2Δx), therefore

∂u ui +1 − ui −1
= + O(Δx) 2
∂x 2Δx

which is a second-order centered difference scheme.

This method can be used in a general manner for symmetric or non-symmetric difference formula. You just pick
which points you want to use.

11
2.2.2. Polynomial Fitting

This is the second, most general method for generating finite difference expression. Here, we assume that the
solution to the PDE can be approximated by a polynomial, and that the values at the mesh points at exact. We thus
differentiate the polynomial to obtain expression for various derivatives.

Assume that u(x) = a x2 + b x + c:

Goal: Find a, b and c. Note that the grid spacing need not be uniform.

Applying the polynomial to those three points gives

ui-1 = a x2i-1 + b xi-1 + c

ui = a x2i + b xi + c
ui+1 = a x2i+1 + b xi+1 + c

12
Solve for a, b, c, we obtain

⎡ ( x − xi )( x − xi +1 ) ⎤ ⎡ ( x − xi −1 )( x − xi +1 ) ⎤
u ( x) = ui −1 ⎢ +
⎥ i⎢u ⎥
⎣ ( xi −1 − xi )( xi −1 − xi +1 ) ⎦ ⎣ ( xi − xi −1 )( xi − xi+1 ) ⎦
⎡ ( x − xi )( x − xi −1 ) ⎤
+ ui +1 ⎢ ⎥
⎣ ( xi +1 − xi )( xi +1 − xi −1 ) ⎦

Note: u(xi) = ui for i, i+1, i-1 (verify yourself).

The above formula is often called a Lagrange Interpolation Polynomial and can be generalized to any order.

If the true solution u is a polynomial having the same degree as that used in the interpolation, then the F.D.
expression will be exact … as will be the derivatives.

∂u
Now, let's compute the derivative by differentiating the expression for u(x).
∂x xi

First, let us define Δx ≡ xi+1 - xi = xi – xi-1 => 2 Δx = xi+1 – xi-1, therefore

∂u ui +1 − ui −1
=
∂x 2Δx
∂ u ui +1 − 2ui + ui −1
2
=
∂x 2 Δx 2

13
 ∂ 3u
=0 because we used a 2nd order polynomial. To obtain FD formulation for third order derivative,
∂x3
we have to use 3rd- or higher- order polynomial.

The 2nd order case can be interpreted physically as the derivative of derivative, or the difference between two
slopes

∂ ⎛ ∂u ⎞ ∂ 2u
⎜ ⎟=
∂x ⎝ ∂x ⎠ ∂x 2
or

ui +1 − ui ui − ui −1
−
Δx Δx
Δx

i.e., the difference between the slope centered at i+1/2 and slope centered at i-1/2.

14
 • It turns out the a 2nd-order polynomial is usually adequate

• Higher-order approximations tend to introduce noises into the solution because the higher-order derivatives
may be large, particularly near sharp gradients.

• Beyond 2nd-order, the polynomial method is not identical to the Taylor series method.

2.2.3. Interpolation and Characteristics

Interpolation is at the heart of virtually all numerical techniques. e.g., 1-D advection equation:

∂u ∂u
+c =0 (c > 0and constant)
∂t ∂x

Let's draw a space-time diagram, using discrete points in space and time:

Recall that dx/dt = c and du=0 along the characteristic curves.

15
 Goal: Determine uin+1 = f (uin , uin−1 , uin+1 , etc) .

We recognize uin+1 = value of u along characteristic curve at the previous time level, i.e., u*n , i.e.,

uin+1 = u*n .

In general, u* won't coincide with grid points but we know only values at the grid points. So we need interpolation
from the grid points to *.

We are free to use as many points as we want, more points Æ higher order.

Indeed, the interpolation used determines the order of accuracy of the solution.

With simple linear interpolation, we get (see figure)

δx ⎛ δx⎞ n
u* = uin−1 + ⎜1 − ⎟ ui .
Δx ⎝ Δx ⎠
Now, in one time step Δt, a particle moves a distance δx at speed c, i.e.,

δx = c Δt.

Substituting, we have

cΔt n cΔt n
u*n = uin+1 = ui −1 + uin − ui
Δx Δx
or

16
 uin+1 − uin uin − uin−1
+c =0
Δt Δx

This is a familiar first-order upstream method using a forward-in-time scheme!

As we will see later, this scheme is strongly damping, therefore inaccurate. This is because linear interpolation
always under-estimate the peak values in the solution.

This scheme yields exact solution if c = constant and Δt = Δx/c. For general problem this condition is hard to meet,
however.

If we new include point i+1 in our interpolation, we end up fitting a parabola to the points and obtain
n +1 α α2
u =u
n
* i =u −
n
i (u − u ) +
n
i +1
n
i −1 (uin+1 − 2uin + uin−1 )
2 2
cΔt
where α = .
Δx

This scheme is known as the Lax-Wendroff or the Crowley scheme in 1-D (see Tremback et al 1987 for example
of using high-order interpolation to obtain high-order Crowley schemes).

Notice that the last term on R.H.S. is an approximation to a second-order derivative which represents a diffusion
process. This term is not present in the original advection equation. It is needed, however, to keep the scheme
stable (by damping unstable modes). Without this term, the scheme becomes forward-in-time and centered-in-
space. We will see later in the section of stability analysis that such as scheme is absolutely unstable (i.e., the
numerical solution will grow without bound – in computer problems, you will quickly run into floating-point
overflow error).

17
Discussion on Stability Condition / Constraint on Time Step Size

It turns out, as we will show later, that we can only use interpolation not extrapolation at time n for numerical
stability.

The criterion of interpolation is that

cΔt
< 1 , i.e.,
Δx

|δx| < |Δx|

or particle cannot move more than one grid interval per time step.

This is known as the stability constraint, and we can write it as

Δt < Δx/c.

It is a limit on the time step size in terms of Δx and c. All explicit schemes have stability limitations (all of which
are not the same). The faster signal moves, the smaller is the time step size that can be used. Half Δx Æ half Δt. In
3D and for a fixed domain size, doubling resolution in all three dimensions increases the total computation by a
factor (2)4 = 16!

18
2.3. Quantitative Properties of Finite Difference Schemes
2.3.1. Consistency, Convergence and Stability of F.D. schemes

Reading: Tannehill et al. Sections 3.3.3 and 3.3.4.

Three important properties of F.D. schemes:

Consistency – An F.D. representation of a PDE is consistent if the difference between PDE and FDE, i.e., the
truncation error, vanishes as the grid interval and time step size approach zero,

i.e., when lim(PDE − FDE) = 0 .

Δ→0

Comment:

• Consistency deals with how well the FDE approximates the PDE.

Stability – For a stable numerical scheme, the errors from any source will not grow unboundedly with time.

Comments:

• A concept that is applicable only to marching (time-integration) problems.

• Generally we are much more concerned with stability than consistency.
• Some hard work is often needed to establish analytically the stability of a scheme.

Convergence – It means that the solution to a FDE approaches the true solution to the PDE as both grid interval
and time step size are reduced.

1
Lax's Equivalence Theorem

For a well-posed, linear initial value problem, the necessary and sufficient condition for convergence is that
the FDE is stable and consistent.

The theorem has been proved for initial value problems governed by linear PDE's (Richtmyer and Morton
1967).

We will discuss the three concepts one by one.

2.3.2. Consistency

Consistency means PDE − FDE → 0 when ΔxÆ 0 and ΔtÆ 0.

Clearly consistency is the necessary condition for convergence.

Example:

Consider a 1-D diffusion equation:

∂u ∂ 2u
= K 2 ( K > 0 and constant)
∂t ∂x

We use the forward-in-time and centered-in-space (FTCS) scheme:

2
 uin+1 − uin uin−1 − 2uin + uin+1
=K
Δt Δx 2

To show consistency, we need to determine the truncation error τ.

Using Taylor series expansion method,

∂u (Δt ) 2 ∂ 2u (Δt )3 ∂ 3u
uin+1 = uin + Δt + + + ...
∂t 2! ∂t 2 3! ∂t 3
∂u (Δx) 2 ∂ 2u (Δx)3 ∂ 3u
ui ±1 = ui ± Δx +
n n
± + ...
∂x 2! ∂x 2 3! ∂x 3

Substituting into the FDE, we have

∂u Δt ∂ 2u ⎡ ∂ 2u ⎤
+ + O(Δt ) = K ⎢ 2 + O(Δx 2 ) ⎥ + ...
2

∂t 2 ∂t 2
⎣ ∂x ⎦

therefore

Δt ∂ 2u
τ= + O(Δt 2 + Δx 2 ) .
2 ∂t 2

τ Æ 0 when ΔxÆ 0 and Δt Æ 0 => the scheme is consistent.

A counter example: The Dufort-Frankel method for the same diffusion equations:

3
 uin+1 − uin−1 (uin+1 + uin−1 ) − (uin+1 + uin−1 )
=K .
2 Δt Δx 2

It's a centered-in-time scheme that is 2nd-order accurate in both space and time.

We can find again the truncation error (do it yourself!)

K ( Δx ) 2 ∂ 4 u ⎛ Δ t ⎞ ∂ u ( Δt ) ∂ u
2 2 2 3
τ= −K⎜ ⎟ − + H .R.T .
12 ∂x 4 ⎝ Δx ⎠ ∂t
2
6 ∂t 3

We can see that lim τ → 0 except for the second term.

Δx ,Δt →0

Δt
If lim = 0 then the scheme is consistent therefore Δt must approaches zero faster then Δx.
Δx ,Δt →0 Δx

Δt
If they approaches zero at the same rate, then lim = β , then
Δx ,Δt →0 Δx

∂ 2u
lim τ = − K β 2
, 2
Δx ,Δt →0 ∂t 2

our equation becomes

∂u 2 2 ∂ u
2
∂ 2u
+K β =K 2
∂t ∂t 2 ∂x

4
Thus, we are solving the wrong equation. In fact this equation is hyperbolic instead of parabolic.

Δt
Note that if ~ 1 you might see spurious waves in your solution, due to the hyperbolic nature of the "new" PDE.
Δx

2.3.3. Convergence

General Discussion

Definition is given earlier. Symbolically, it is

lim uin = u ( x, t ) .
Δx ,Δt →0

Convergence is generally hard to prove, especially for nonlinear problems. The Lax's Theorem we presented earlier
is very helpful in understanding the convergence for linear systems, and is often extended to nonlinear systems.

We will also discuss numerical convergence and methods for measuring solution accuracy later.

We will first show a convergence proof for a diffusion problem. Certain concept introduced will be useful later.

Convergence proof for a 1-D diffusion problem

Consider

5
 ∂u ∂ 2u
= K 2 ( K > 0 and constant) for 0 ≤ x ≤ L
∂t ∂x

which has an initial condition:

∞
kπ x
u ( x, t = 0) = ∑ ak sin( ) = f ( x) .
k =1 L

The B.C. is

u (0, t ) = u ( L, t ) = 0 .

This is a well-posed, linear initial value problem (notice the I.C. satisfies the B.C. as well).

First, let's find the analytical solution to the PDE.

Because the problem in linear, we need only to examine the solution for a single wavenumber k, and can assume a
solution of the form:

kπ x
uk ( x, t ) = Ak (t )sin( )
L

Here Ak is the amplitude and sin gives the spatial structure and the final solution should be the sum of all wave
components. Note that this solution satisfies the boundary conditions.

Substituting the solution into the PDE, we obtain an ODE for the amplitude Ak:

6
 ⎛πk ⎞
2
dAk (t )
= −⎜ ⎟ KAk
dt ⎝ L ⎠

⎛πk ⎞
2
d ln( Ak )
Æ = −K ⎜ ⎟
dt ⎝ L ⎠

Ak (t ) = Ak (0)exp ⎡ − K (π k / L ) t ⎤ .
2
Æ
⎣ ⎦

It says that the amplitude of the solutions for all wave numbers decreases with time.

From the I.C., Ak(0) = ak, so we have

kπ x
uk ( x, t ) = ak exp ⎡ − K (π k / L ) t ⎤ sin(
2
)
⎣ ⎦ L

and
u ( x, t ) = ∑ uk ( x, t )
k

which is the analytical solution to the original diffusion equation.

A numerical approximation to the diffusion equation should converge to this solution as Δx, Δt Æ 0.

Consider the forward-in-time centered-in-space (FTCS) scheme we derived earlier.

7
Goal: Show that uit Æ u(x,t) as Δx, Δt Æ 0.

First, find the numerical solution.

This time, we use the FDE and substitute a discrete Fourier series into the equation.

Let the I.C. be given by

J
kπ xi
f ( xi ) = u = ∑ ak sin(
0
i ) for i= 0, 1, 2, ….., J
k =0 L

where J+1 = total number of grid points used to represent the initial condition.

The coefficient ak is given by a discrete Fourier transform:

2 J kπ xi
ak = ∑
J i =0
f ( xi )sin(
L
) for k=0, 1, 2, …., J.

Note 1: L = JΔx. As Δx Æ 0, JÆ ∞, the discrete Fourier series becomes continuous and ak → ak .

Note 2: The number of harmonics or Fourier wave components that can be represented is a function of the number
of grid points (J+1), which is the number of degrees of freedom. Spectral methods represent fields in terms of
spectral components, whose amplitudes are solved for.

kπ
The wavenumber for the wave components is in the above equations.
L
Recall that wavelength

8
 2π 2π 2L
λ= = =
w.n. (π k / L) k

where k is the number (index) of wave components.

Longest wavelength = ∞ , corresponding to wavenumber zero (k=0).

Next longest wave = 2L (k =1 )

.
.
.

Shortest wave = 2L/J = 2JΔx/J = 2Δx.

Comments:

A 2Δx wave is the shortest wave that can be resolved on any grid and it takes at least 3 points to represent a
wave.

9
 2Δx waves often have some special properties. They are also represented most poorly by numerical methods –
recall that smooth fields are more accurately represented by a finite number of grid points.

As in the continuous case, we examine only one wavenumber k (the solution is the sum of all waves), so for our
discrete problem, assume a solution of the form

⎛ kπ xi ⎞
uin = Ak (n)sin ⎜ ⎟
⎝ L ⎠

(It satisfies B.C.) and n is the time level.

We also have from I.C.

Ak (0) = ak .

Substituting this into the FDE

uin+1 − uin uin−1 − 2uin + uin+1

=K ,
Δt Δx 2

and letting

10
 ⎛ kπ xi ⎞
Si = sin ⎜ ⎟,
⎝ L ⎠

we have

Ak (n + 1) Si − Ak (n) Si KAk (n)

= [ Si +1 − 2 Si + Si −1 ] .
Δt (Δx) 2

Since xi = i Δx, xi+1 = (i+1) Δx therefore

⎛ kπ ( x + Δx) ⎞
Si +1 = sin ⎜ ⎟.
⎝ L ⎠

Using standard trigonometric identities, we can write the above in the form of a recursion relation:

⎡ ⎛ π k Δx ⎞ ⎤
Ak (n + 1) = Ak (n) ⎢1 − 4 μ sin 2 ⎜ ⎟⎥
⎣ ⎝ 2L ⎠⎦

K Δt
where μ = .
(Δx) 2

⎡ ⎛ π k Δx ⎞ ⎤
If we let M(k) ≡ ⎢1 − 4 μ sin 2 ⎜ ⎟ ⎥ , then we have
⎣ ⎝ 2L ⎠⎦

Ak (n + 1) = M (k ) Ak (n) .

11
Write it out for n =0, 1, 2, …., n:

Ak (1) = M (k ) Ak (0) = M (k )ak

Ak (2) = M (k ) Ak (1) = [ M (k )]2 ak
.
.
.
Ak (n) = M (k ) Ak (n − 1) = [ M (k )]n ak

we therefore have the solution of u for wave mode k:

⎛ kπ xi ⎞
(uin ) k = Ak (n) Si = ak [ M (k )]n sin ⎜ ⎟.
⎝ L ⎠

Definition: M(k) is known as the amplification factor, and if | M(k) | ≤ 1, the solution will not grow in time as n Æ
∞. This had better to be the case because the amplitude of the analytical solution is supposed to always decrease
with time.

For our problem we can see that | M(k) | ≤ 1 means

⎛ π k Δx ⎞
1 − 4 μ sin 2 ⎜ ⎟ ≤ 1.
⎝ 2L ⎠

If we take the maximum possible value of sin2( ) = 1, Æ

−1 ≤ 1 − 4 μ ≤ 1

12
 Æ μ ≤ 1/2.

This condition needs to be met for all k to prevent solution growth. Based on the definition of μ, the condition
becomes

(Δx) 2
Δt ≤ .
2K

This imposes an upper bound on the Δt that can be used for a given value of Δx, and such a condition is unknown
as the Stability Constraint.

Now we have our solution, let's check convergence for single mode k.

By definition of convergence, we take

n
⎡ 4K 2 ⎛ π k Δx ⎞ ⎤ ⎛ kπ xi ⎞
lim (u ) = lim ak ⎢1 − Δt
n
sin ⎜ ⎟⎥ sin ⎜ ⎟
(Δx) 2
i k
Δx ,Δt →0 Δx , Δ t → 0
⎣ ⎝ 2L ⎠⎦ ⎝ L ⎠

4K ⎛ π k Δx ⎞
or if we let f(Δx) ≡ sin 2 ⎜ ⎟,
(Δx) 2
⎝ 2L ⎠

⎛ kπ xi ⎞
lim (uin ) k = lim ak [1 − Δt f (Δx) ] sin ⎜
n
⎟.
Δx ,Δt →0 Δx , Δ t → 0
⎝ L ⎠

13
It can be shown that if f(x) is a complex-valued function of a real argument, say Δx, such that

lim f (Δx) = a , then,

Δx →0

lim [1 ± Δt f (Δx)]n = e ± at (we will show this later).

Δx ,Δt →0

sin( y )
We know that lim = 1 , therefore
y →0 y

2
⎡ ⎛ π k Δx ⎞ ⎤
2 ⎢ sin ⎜ ⎟
K (π k ) 2L ⎠ ⎥ ⎛πk ⎞
2

lim f (Δx) = lim ⎝ ⎥ = K⎜

⎢ ⎟ =a
Δx →0 Δx →0 L2 ⎢ ⎛ π k Δx ⎞ ⎥ ⎝ L ⎠
⎢⎣ ⎜⎝ 2 L ⎟⎠ ⎥⎦

Also lim ak = ak , therefore

Δx →0

⎡ ⎛ π k ⎞ ⎤ ⎛ π kx j
2
⎞
lim (u ) = ak exp ⎢ − K ⎜
n
i k ⎟ t ⎥ sin ⎜ ⎟.
Δx ,Δt →0
⎣⎢ ⎝ L ⎠ ⎦⎥ ⎝ 2 L ⎠

Interestingly, this solution is identical to our analytical solution derived earlier! Therefore the numerical solution
converges to the PDE solution when Δx, Δt Æ 0.

14
Finally, we show here (noting that nΔt = t)

n(n − 1) n(n − 1)(n − 2)

lim (1 ± f Δt ) n = 1 ± fnΔt + ( f Δt ) 2 + ( f Δt )3 + ...
Δx →0 2! 3!
a2 ⎛ 1 ⎞ a 3 ⎛ 1 ⎞⎛ 2 ⎞
= 1 ± at + ⎜1 − ⎟ (nΔt ) 2 ± ⎜1 − ⎟⎜ 1 − ⎟ (nΔt )3 + ...
2! ⎝ n ⎠ 3! ⎝ n ⎠⎝ n ⎠
(at ) 2 (at )3
= 1 ± at + ± + ...
2! 3!
= e ± at

2.3.4. Numerical Convergence

Reading: Fletcher (handout), Sections 4.1.2, 4.2.1, 4.4.1.

Numerical Convergence

Convergence is often hard to demonstrate theoretically.

True analytical solution is hard or even impossible to find is one of the reasons.

We can, however, find out the convergence of a given scheme numerically.

We compute solutions at successively higher resolutions and see how the error changes with the resolution.

Does τ Æ 0 when Δx Æ 0?

15
 And how fast τ decreases?

The procedure can be very expensive (remember the cost factor increase as Δ doubles).

A typical measure of error is the L2 norm or RMS error:

[ui Δx − ui ]2
L2 = ∑
n

where u is a true solution or a 'converged' numerical solution when exact solution is not available.

Example: 1-D diffusion equation using FTCS scheme:

Δt ∂ 2u (Δx) 2 ∂ 4u
τ= −K + ....
2 ∂t 2 12 ∂x 4

Making use of

∂u ∂ 2u ∂ 2u ∂ ⎛ ∂ 2u ⎞ ∂ 2 ⎛ ∂u ⎞ 2 ∂ u
4
=K 2 ⇒ 2 =K ⎜ 2 ⎟=K 2⎜ ⎟=K .
∂t ∂x ∂t ∂t ⎝ ∂x ⎠ ∂x ⎝ ∂t ⎠ ∂x 4

K (Δx) 2 ⎛ 1 ⎞ ∂ 4u
we have τ= ⎜ s − ⎟ 4 + O(Δx + Δt ) .
4 2

2 ⎝ 6 ⎠ ∂x

K Δt
where s =
(Δx) 2

16
Table 4.1 (from Fletcher) shows the error reduction with Δx for two
values of s.

The above figure shows plots of log10(err) as a function

of log10(Δx).

17
Recall that

τ = A (Δx)n Æ

log τ = log A + n log Δx.

This is a straight line in log-log diagram with a slope of n and intercept A.

Thus the slope of the line gives the rate of convergence.

In the above figure, we see that when s = 1/6, the error line has a steeper slope and the error is smaller for all Δx.
This is because for this value of s, the first term in τ drops out and the scheme because 4th-order accurate. The
scheme is second-order accurate for all other values of s.

Note that you can choose Δx and Δt such that s=1/6 only when K is constant in the entire domain.

In cases where no exact solutions is available, a so-called 'grid-convergence' or reference solution is often sought
and this solution can be used in the place of true solution in the estimating the solution error.

18
 An example from Straka et al (1993).

It shows a reference solution obtained at Δx=25 m, for a density current resulting from a dropping cold bubble.

19
Figure (from Straka et al 1993). Graph of θ' L2 norms (°C) from self-convergence tests with the compressible
reference model (REFC). The bold solid line labeled with 'self-convergence solutions' represents the L2 norms for
spatial truncation errors of solutions made with Δt = constant and varying grid spacings. The L2 norms were
computed against a 25.0 m reference solution. The bold dashed lines labeled with, for example, '200.0 m solutions'
represent L2 norms for temporal truncation errors of solutions made with Δx =constant (e.g. 200.0 m) and varying
time steps. The reference solutions for these computations were made using a time step consistent with Δt = 12.5 s
times a constant in each of the cases. The solid fines labeled O(l) and O(2) represent first- and second-order
convergence, respectively.

20
Richardson Extrapolation

As the grid becomes very fine, the error behaves much like that predicted from the leading terms in τ. Further
refinements are expensive, so we use another technique to improve the solution – Richardson Extrapolation.

Consider two numerical solutions obtained at Δxa and Δxb.

With FTCS scheme (assuming s≠1/6),

K (Δxa ) 2 ⎛ 1 ⎞ ∂ 4u
τa = ⎜ s − ⎟ 4 + O(Δxa + Δt )
4 2

2 ⎝ 6 ⎠ ∂x

K (Δxb ) 2 ⎛ 1 ⎞ ∂ 4u
τb = ⎜ s − ⎟ 4 + O(Δxb + Δt )
4 2

2 ⎝ 6 ⎠ ∂x

Find a linear combination of the two solutions, ua and ub

uc = a ua + b ub

where a + b =1 and a and b are chosen so that the leading terms in τa and τb cancel and the scheme becomes 4th-
order accurate (Of course this assumes that ∂ 4u ∂x 4 is the same in both case, which is reasonably assumption only
at relatively high resolutions when the solution is well resolved).

If Δxb = Δxa/2, then

4a + b = 0
with a + b = 1 Æ
a = -1/3, b = 4/3 and uc = -1/3 ua + 4/3 ub.

21
2.3.5. Stability Analysis

Reading: Tannehill et al. Section 3.6.

Stability – For a stable numerical scheme, the errors in the initial condition will not grow unboundedly with
time.

In this section, we discuss the methods for determining the stability of F.D. schemes. This is very important when
designing a F.D. scheme and for understanding its behavior.

There are several methods:

• Energy method
• von Neumann method
• Matrix method (for systems of equations)
• Discrete perturbation method (will not discuss)

Note: Stability refers to the F.D.

Does not involve B.C. or I.C.
Refers to time-matching problems only

The energy method

Read Durran Section 2.2 (handout, see web link)

1
This method is used much less often than the von Neumann method.

It's attractive because it works for nonlinear problem and problems without periodic B.C.

The key is to show that a positive definite quantity like ∑ (u

i
n 2
i ) is bounded for all n.

∂u ∂u
We illustrate this method using the upstream-forward scheme for wave (advection) equation +c = 0:
∂t ∂x

uin+1 − uin u n − uin−1

+c i =0
Δt Δx

Let μ = cΔt/Δx Æ

uin+1 = (1 − μ )uin + μuin−1

Squaring both sides and summing over all grid points:

∑ (u
i
i ) = ∑ [(1 − μ ) 2 (uin ) 2 + 2(1 − μ ) μuinuin−1 + μ 2 (uin−1 ) 2 ]
n +1 2

i
(1)

Assuming periodic B.C. Æ

∑ (u
i
i) = ∑ (uin−1 ) 2
n 2

2
and using the Schwarz inequality (which says that for two vectors U and V,| U ⋅ V | ≤ | U | ⋅ | V | ),

∑u u
i
n n
i i −1 ≤ ∑ (u
i
n 2
i ) ∑ (u
i
n 2
) = ∑ (uin ) 2 .
i −1
i

If μ (1 − μ ) ≥ 0 , all coefficients of RHS terms in (1) are positive and we have

∑ (u
i
i ) ≤ [(1 − μ ) 2 + 2(1 − μ ) μ + μ 2 ] ∑ (uin ) 2 = ∑ (uin ) 2 ,
n +1 2

i i

i.e., the L2 norm at n+1 is no greater than that at n, therefore the scheme is stable!

The condition μ (1 − μ ) ≥ 0 gives

μ = cΔt/Δx ≤ 1

which is the stability condition for this scheme. As we discussed earlier, it says that waves cannot propagate more
than one grid interval during one Δt in order to maintain stability.

von Neumann method

Read Tannehill et al, Section 3.6.1.

In a sense, we have already used this method to find stability of the 1-D diffusion equation – when we were
proving the convergence of the solution using FTCS scheme.

We found then

3
 un+1 = [ M(t) ]n+1 u0

where M(t) = amplification factor.

If M(t) ≤ 1 by some measure (M can be a matrix or a complex number), then un+1 ≤ un and the solution cannot
grow in time – the scheme is stable.

Essentially, von Neumann method expands the F.D.E. in a Fourier series, finds the amplification factor and
determines under what condition the factor is less than or equal to 1 for stability.

Assumptions:

1. The equation has to be Linear with constant coefficients.

2. It is assumed that the solution is periodic.

With this method, the dependent variable is decomposed into a complex (or a real) Fourier series:

u ( x, y , z , t ) = ∑ U (k , l , m)exp[i(kx + ly + mz − ωt )]
k ,l ,m
(2)

where U is the complex amplitude and ω = ωR + i ωΙ is the complex frequency.

In fact ωR gives the wave propagation speed and ωΙ gives the growth and decaying rate.

e − iωt = e − i[ωR +iωI ]t = e − iωRt eωI t

e − iωRt - phase function of Fourier components

eωI t - growth or decay rate

4
If ωI >0, the solution will grow exponentially in time.

Example 1: 1-D diffusion equation with FTCS scheme.

uin+1 − uin = μ (uin−1 − 2uin + uin+1 ) (3)

K Δt
where μ = .
(Δx) 2

Let's examine a single wave k:

uin = U k ei ( kx −ωt )
uin+1 = U k ei ( kx −ωt )e − iωΔt
uin±1 = U k ei ( kx −ωt )e± ik Δx

Substitute the above into (3) Æ

U k ei ( kx −ωt ) (e − iωΔt − 1) = μU k ei ( kx−ωt ) [eik Δx − 2 + e− ik Δx ] Æ

U k ei ( kx −ωt ) [e − iωΔt − 1 − 2 μ (cos k Δx − 1)] = 0 Æ

U k ei ( kx −ωt ) [e − iωΔt − 1 + 4 μ sin 2 (k Δx / 2)] = 0

5
For non-trivial solution, we require

e − iωΔt − 1 + 4 μ sin 2 (k Δx / 2) = 0 Æ

e − iωΔt = 1 − 4 μ sin 2 (k Δx / 2)

Here, e − iωΔt is actually the amplification factor, the same as the M discussed earlier.

λ ≡ e − iωΔt = un+1/un - the amplification factor.

For stability we require

|λ|≤1Æ

−1 ≤ 1 − 4 μ sin 2 (k Δx / 2) ≤ 1 Æ

μ ≤ 1/2 as before!

In practice, when μ=1/2, the solution (amplification factor) switches between –1 and +1 for 2Δx waves
( sin 2 (k Δx / 2) = 1 ) very other step, which is unrealistic. The standard requirement is therefore

μ ≤ 1/4.

Therefore, do not naively think diffusion terms in a numerical model does not cause numerical instability!
When integrated stably, the diffusion term in CFD models tends to stabilize the solution by killing off/damping
small scale waves, but when stability condition is not met, it tem itself will cause problem!

6
Read Pielke (1984) section 10.1.2 (handout, see web link).
2.3.6. Implicit Methods

Read Tannehill et al, second part of section 3.4.1.

So far, we have dealt with only explicit schemes which have the form of

u n+1 = f (u n , u n−1 ,...) .

With these schemes, the future state at each grid point is only dependent on the current and past time levels,
therefore the solution can be obtained directly or explicitly. c.f., explicit functions such as y = x2.

Implicit scheme involves variables of the future time level at more than one grid point (often resulting from finite
difference of variable(s) at the future time level). Mathematically it can be expressed as

u n+1 = f (u n+1 , u n , u n−1 ,...) .

This is analogous to implicit functions such as x = sin(x). As one can imagine, implicit schemes are more difficult
to solve. Usually matrix inversion is involved.

We will first look at the stability property of an implicit scheme.

Example. Consider the 1-D diffusion equation ut = K uxx again.

It is approximated by the following F.D. scheme:

7
 δ + t ui = K [αδ xxuin+1 + (1 − α )δ xxuin ] (4)

(Note: shorthand notations for F.D. are used. See Appendix. e.g., δ + t u = (u n+1 − u n ) / Δt ).

• When α = 0, the scheme is explicit, and is the FTCS scheme discussed earlier.
• When α = 1/2, it is implicit and is called Crank-Nicolson scheme.
• For other values of α, it is a general implicit scheme.

We can show that

∂ 4u ⎡ ⎛1 ⎞ (Δx) ⎤
2
τ = K 4 ⎢ K Δt ⎜ − α ⎟ − ⎥ + O ( Δx 4 + Δ t 2 )
∂x ⎣ ⎝2 ⎠ 12 ⎦

(show if for yourself!).

We can see that when α = 1/2, it is 2nd-order accurate in time and space. Otherwise, it's first-order in time – which
is expected for un-centered time-differencing scheme (when α=1/2, the right hand side is an averaged between the
current and future time levels valid at n+1/2. Relative to this RHS, the LHS time difference becomes centered in
time. We know that the simplest centered difference scheme is second-order accurate).

When [ ] = 0, the scheme becomes fourth-order in space.

Let's perform stability analysis on (4) using von Neumann method.

u nj = U k ei ( kx −ωt ) = U k e − iωnΔt eikjΔx ≡ U k λ n eikjΔx (5)

8
Here λ ≡ e − iωΔt . Note that we are now using j as the grid point index.

Substitute (5) into (4) Æ

U k eikjΔx (λ n+1 − λ n ) = μU k eikjΔx ⎡⎣αλ n+1 + (1 − α )λ n ⎤⎦ (eik Δx − 2 + e− ik Δx ) Æ

Dividing U k eikjΔx λ n on both sides, and rearranging Æ

λ − 1 = −4 μ sin 2 (k Δx / 2)[αλ + (1 − α )]

1 − 4(1 − α ) μ sin 2 (k Δx / 2)
λ=
1 + 4αμ sin 2 (k Δx / 2)

Look at several cases:

Case I: α = 0, λ = 1 − 4μ sin 2 (k Δx / 2) Æ μ ≤ 1/2 as before. The scheme is conditionally stable.

Case II: α = 1/2 (Crank-Nicolson)

1 − 2 μ sin 2 (k Δx / 2)
| λ |= ≤ 1 for all values of μ,
1 + 2 μ sin 2 (k Δx / 2)

therefore the scheme is absolutely or unconditionally stable.

Case III: α =1, the time difference is backward, relative to the RHS terms.

9
 1
| λ |= ≤ 1 , again for all values of μ,
1 + 4 μ sin 2 (k Δx / 2)

therefore the scheme is also absolutely stable. However, this scheme is only first-order accurate in time, as
discussed earlier (consistent with the time difference scheme being un-centered).

In general, when 0 ≤ α < 1/2, it is required that μ ≤ 1/(2 - 4α ), therefore the scheme is conditionally stable. When
1/2 ≤ α ≤ 1, the scheme is unconditionally stable (it is sometimes referred to as the forward-biased scheme).

In the ARPS, the implicit diffusion scheme is an option for treating the vertical turbulent mixing terms. This
treatment is necessary in order to remove the severe stability constraint from these terms when vertical mixing is
strong inside the planetary boundary layer (PBL). The latter occurs when the PBL is convectively unstably and the
non-local PBL mixing is invoked with the Sun and Chang (1986) parameterization. Parameter alfcoef in arps.input
corresponds to 1-α here (see handout).

Finally, we note that for multi-time level schemes, there is usually multiple solutions for the amplification factor λ.
some of them might represent spurious computational modes due to the use of extra (artificial) initial conditions.

The expression of λ can be too complicated so that a graphic plotting is needed to understand its dependency on
wave number k. |λ| has to be no greater than 1 for all possible waves. The shortest wave resolvable on a grid has a
wavelength of 2 Δ, and the longest is 2L, where L is the domain width.

Tridiagonal Solver

10
1-D implicit method often leads to tridiagonal systems of linear algebraic equations.

(In the ARPS, this appears twice – once when sound waves are treated implicitly in the vertical direction and once
when the vertical turbulence mixing is treated implicitly).

For example, Eq. (4) can be rewritten as

uin+1 − uin uin−+11 − 2uin+1 + uin++11 uin−1 − 2uin + uin+1

=αK + (1 − α ) K Æ (6)
Δt Δx 2 Δx 2

It can be rearranged into

ΔtK n+1 ΔtK

uin+1 − α (ui −1 − 2uin+1 + uin++11 ) = (1 − α ) 2 (uin−1 − 2uin + uin+1 ) + uin Æ
Δx 2
Δx

ΔtK n+1 ΔtK ΔtK

−α u + (1 + 2α 2 uin+1 ) − α 2 uin++11 = din .
2 i −1
Δx Δx Δx

uin−1 − 2uin + uin+1

where din = Δt (1 − α ) K + uin .
Δx 2

αΔtK 2αΔtK
Let Ai = Ci = − , Bi = (1 + ) , we then have
Δx 2
Δx 2
n +1 n +1
i i −1 + Bi ui
Au + Ciuin++11 = din , (7)

for i = 1, 2, …, N-1, assuming the boundaries are at i =0 and N.

11
If we have Dirichlet boundary conditions, i.e., u at i=0 and N are known, then for i =1, the equation becomes

B1u1n+1 + C1u2n+1 = d1n − A1u0n+1 (8)

and for i = N, the equation is

AN −1u Nn+−12 + BN −1u Nn+−11 = d Nn −1 − CN −1u Nn+1 . (9)

For i = 2, 3, …., N-2, the equation remains of the form in Eq.(7).

If we write the equations (7-9) in a matrix form, we have

⎡ B1 C1 ⎤ ⎡ u1 ⎤ ⎡ D1 ⎤
⎢A B2 C2 ⎥ ⎢ u ⎥ ⎢ D ⎥
⎢ 2 ⎥ ⎢ 2 ⎥ ⎢ 2 ⎥
⎢ . . . ⎥ ⎢ . ⎥ ⎢ . ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ Ai Bi Ci ⎥ ⎢ ui ⎥ = ⎢ Di ⎥ (10)
⎢ . . . ⎥ ⎢ . ⎥ ⎢ . ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ AN −2 BN −1 CN −2 ⎥ ⎢u N −2 ⎥ ⎢ DN −2 ⎥
⎢ BN −1 ⎥⎦ ⎢u ⎥ ⎢ D ⎥
⎣ AN −1 ⎣ N −1 ⎦ ⎣ N −1 ⎦

where

Di = din for i=2, …, N-2,

D1 = d1n − A0u0n+1 ,
DN −1 = d Nn −1 − CN u Nn+1 .

12
If we have Neumann boundary conditions, i.e., we know the gradient of u at the boundaries which in discretized
form are u1 − u0 = L and u N − u N −1 = R . Plug these relations into Eq.(7) for i=1 and i=N-1, we obtain equations
similar to (8) and (9):

( A1 + B1 )u1n+1 + C1u2n+1 = d1 + A1L (11)

AN −1u Nn+−12 + ( BN −1 + C N −1 )u Nn+−11 = d N −1 − C N −1R . (12)

In this case, the final coefficients in (10) are different for the first and last equation.

Since in each except for the first and last row of the coefficient matrix, only three elements are non-zero and the
non-zero elements of the matrix are aligned along the diagonal axis, this system is called tridiagonal system of
equations. It can be solved efficiently using Thomas Algorithm.

The procedure consists of two parts. First, Eq. (7) is manipulated into the following form:

⎡1 C '1 ⎤ ⎡ u1 ⎤ ⎡ D '1 ⎤
⎢ 1 C '2 ⎥ ⎢ u ⎥ ⎢ D' ⎥
⎢ ⎥ ⎢ 2 ⎥ ⎢ 2 ⎥

⎢ . . . ⎥ ⎢ . ⎥ ⎢ . ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ 1 C 'i ⎥ ⎢ ui =
⎥ ⎢ D 'i ⎥ (13)
⎢ . . . ⎥ ⎢ . ⎥ ⎢ . ⎥
⎢ ⎥ ⎢ ⎥ ⎢ ⎥
⎢ 1 C 'N −2 ⎥ ⎢ N −2 ⎥ ⎢ N −2 ⎥
u D '
⎢⎣ 1 ⎥⎦ ⎢u ⎥ ⎢ D ' ⎥
⎣ N −1 ⎦ ⎣ N −1 ⎦

in which the sub-diagonal coefficients A are eliminated and the diagonal coefficients are normalized. For the first
equation

13
 C1 D1
C '1 = , D '1 = . (14a)
B1 B1

For the general equations:

Ci Di − Ai D 'i −1
C 'i = , D 'i = . (14b)
Bi − AC
i 'i −1 Bi − AC
i 'i −1

Equations in (14) represent a forward sweep step (see figure below). It is followed by a backward substitution step
that finds solution ui from (13). The solution is:

u N −1 = D 'N −1
(15)
ui = D 'i − ui +1C 'i for i from N − 2 to 1.

14
Note both (14) and (15) involve reduction, the algorithm is inherently non-parallelizable. Fortunately, for multi-
dimensional problems, multiple systems of equations often need to be solved, and one can exploit parallelism along
other dimensions (e.g., j instead of i direction).

Read Section 4.3.3 of Tannehill et al and Appendix A.

2.3.7. Stability Analysis for Systems of Equations

When we are dealing with a system of equations, we can also apply the von Neumann method to find the stability
property of a given F.D. scheme.

As with single equations, von Neumann can only be used for linear systems of equations. For nonlinear systems,
linearization has to be performed first.

Without going into details, we point out that a system of linear equations can be expressed in a matrix form like
G G
∂u ∂u
+ [ A] = 0 (16)
∂t ∂x
G
The equation is first discretized using certain F.D. scheme, u can be written in terms of a discrete Fourier series
and the wave component is then substituted into the discrete equation to obtain something like:
G G
U kn+1 = [ M (Δt , Δx)]U kn (17)
G
where U kn is the amplitude vector for wave k at time level n, and [M] is called the amplification matrix.

The scheme is stable when the maximum absolute eigenvalue of [M] is no greater than 1.

15
Why the maximum absolute eigenvalue?

Because as you saw earlier (in Chapter 1) that a system of equation like (17) can be transformed into a system of
decoupled equations, and the eigenvalues
G of [M] become the amplification factors for each of the new dependent
variables, vi (the element of vector V ), i.e., we can obtain from (17)
G G
Vkn+1 = [ N ] Vkn ,

where [N] is a diagonal matrix with the eigenvalues of [M] as its diagonal elements.
G G G G G G
{[T]-1 [M] [T] = [N] Æ [T ]−1U kn+1 = [T ]−1[ M ][T ][T ]−1U kn Æ Vkn+1 = [ N ] Vkn where Vkn+1 = [ N ] −1 U kn . Therefore
G
(vi ) nk +1 = λi (vi ) nk where vi is an element of vector V .}

Since the system is stable only when all dependent variables remain bounded, the absolute value of the maximum
eigenvalue has to be no greater than 1.

Read section 3.6.2 of Tannehill et al.

16
Appendix

Shorthand notations for discrete/finite difference operators and discretization identities

Notations:

Aj + n x + Aj −n x
A nx =
2
Aj + n / 2 − Aj −n / 2
δ nx x =
n Δx
Aj +1 − Aj
δ+x x =
Δx
Aj − Aj −1
δ−x x =
Δx

Identities:

x
δ2x A ≡ δ x Ax ≡ δ x A

x
Aδ x B ≡ δ x ( A x B ) − Bδ x A

A xδ x A ≡ δ x ( A2 / 2)

1
2.4. Multi-Dimensional Problems

Reference: Tannehill et al 4.3.9-4.2.11.

C.F. Fletcher 8.1, 8.2, 8.5.

For multi-dimensional (MD) problems, one can use

1) Direct extension of 1-D operators. It's the most straightforward method but may not have the best stability
property. There can be problems with neglecting cross directive terms in the Taylor expansion.

2) Direction splitting method – we build up a MD problem by successive 1-D passes through the grid in alternating
coordinate directions. Each time solving a 1-D problem.

3) Full MD methods designed specifically for MD problems.

We will only discuss the first two methods.

Consider 2-D diffusion equation on a regular x-y domain:

∂u ∂ 2u ∂ 2u
= K( 2 + 2 ) ( K > 0 and constant) . (1)
∂t ∂x ∂y

29
2.4.1. Direct extension of FTCS method

δ + t uijn = K [δ xxuijn + δ yyuijn ] (2)

It is consistent andτ = O(Δt , Δx 2 , Δy 2 ) .

We can find out (show it for yourself), using Neumann stability analysis (assuming uijn = U k λ n ei ( kx +ly ) ) that the
stability condition is
−1
1 ⎛ 1 1 ⎞
Δt ≤ +
2 K ⎜⎝ Δx 2 Δy 2 ⎟⎠
. (3)

Δ2
For Δx = Δy, Δt ≤ , which is twice as restrictive as that for 1-D problem!
4K

2.4.2. Direct extension of general method

uijn+1 − uijn
= α K [δ xxuijn+1 + δ yy uijn+1 ] + (1 − α ) K [δ xxuijn + δ yyuijn ] (4)
Δt

Reorganizing Æ

[1 − αΔtK (δ xx + δ yy )]uijn+1 = [1 − (1 − α )ΔtK (δ xx + δ yy )] uijn (5)

30
One can find out that the stability condition is
−1
1 ⎛ 1 1 ⎞
Δt ≤ ⎜ + 2⎟ for 0 ≤ α < 1/ 2 (again more restrictive than the
(2 − 4α ) K ⎝ Δx 2
Δy ⎠

corresponding 1-D case), and unconditionally stable for1 ≥ α ≥ 1/ 2 .

When α ≠ 0 , the above scheme is implicit, as in the 1-D case.

This system of equations is more difficult to solve, however, due to the involvement of unknowns at more than 3
grid points – in fact, five grid points (five unknown ) are involved for this 2-D problem. This can be more clearly
seen if (5) is rewritten into the following form:

auin, +j 1−1 + buin, +j +11 + cuin, +j 1 + auin++1,1 j + buin−+1,1 j = dijn (6)

assuming K is constant and Δx=Δy in (5).

In matrix form, we can write (5) as

G G
{[ I ] − α [ A]}U n+1 = {[ I ] − (1 − α )[ A]}U n (7)
G
where [I] is an identity matrix and [A] is block tridiagonal. U is a vector consisting of u at all grid points.

31
This system cannot be solved as efficiently as the tridiagonal system from 1-D problem. Similar system arises from
the discretization of elliptic equation ∇ 2u = F . We will discuss methods for solving it at a later time.

2.4.3. Directional Splitting Method

Goal: We look for ways to avoid having to solve the block tridiagonal matrix – we want to get back to a single
tridiagonal matrix.

a) Alternating Direction Implicit (ADI) method

32
One of the best known way of doing this is the alternating direction implicit (ADI) scheme due to Peaceman and
Rachford.

The basic idea is to write the single full time step as a sum of two half steps, each representing a single coordinate
direction:

uijn+1/ 2 − uijn
= K [δ xxuijn+1/ 2 + δ yyuijn ]
Δt / 2
u − uijn+1/ 2
n +1
ij
= K [δ xxuijn+1/ 2 + δ yy uijn+1 ] (8)
Δt / 2

Æ [1 − sδ xx ] uijn+1/ 2 = [1 + sδ yy )] uijn (9a)

[1 − sδ yy ] uijn+1 = [1 + sδ xx )] uijn+1/ 2 (9b)

where s = KΔt/2.

The left hand side of the equations can be written in the form of

n +1/ 2
Au
i i −1 + Biuin+1/ 2 + Ciuin++11/ 2 and Aj u nj −+11 + B j u nj +1 + C j u nj ++11

therefore they form two systems of tridiagonal equations. (9a) is first solved for all j indices then (9b) is solved for
all i indices.

Stability: The amplification factor of each full step is simply the product of the amplification factor of the two
individual steps.

33
We can see

u n+1/ 2 = λa u n
u n+1 = λb u n+1/ 2

therefore u n+1 = λa λb u n = λu n .

Stable if | λ |=| λa λb |≤ 1 .

We can easily show that

1 − 2 μ sin 2 (l Δy / 2)
λa =
1 + 2 μ sin 2 (k Δx / 2)
1 − 2 μ sin 2 (k Δx / 2)
λb =
1 + 2 μ sin 2 (l Δy / 2)

therefore | λ |≤ 1 for all μ ! The scheme is absolutely stable.

Comment: ADI is unconditionally stable for the 2D diffusion equation, but conditionally stable in 3D. The
condition is K x Δt / Δx 2 ≤ 1, K y Δt / Δy 2 ≤ 1 and K z Δt / Δz 2 ≤ 1 .

To overcome the conditionally stability problem with the above 3D version of ADI, Douglas and Gunn (1964)
developed a general method for deriving ADI that are unconditionally stable for all dimensions. The method is
called approximate factorization.
This is discussed in section 4.2.10 of Tennehill or section 8.2.2 of Fletcher.

34
b) Local 1-D or Fractional Step Method

There are many ways to split MD problems into a series of 1D problems. So far, we have been splitting the FDE.
One can also split the PDE, into a pair of equations for 2D case, with each of them being a local 1D equation. This
method was developed by Soviet mathematicians in the early seventies (see Yanenko 1971).

In a sense, the method splits equation

∂u ∂ 2u ∂ 2u
= Kx 2 + K y 2 ( K x , K y > 0 and constant)
∂t ∂x ∂y

into two equations:

1 ∂u ∂ 2u
= Kx 2
2 ∂t ∂x
1 ∂u ∂ 2u
= Ky 2 .
2 ∂t ∂y

They can be solved using an explicit scheme:

uijn+1/ 2 = (1 + 0.5 K x Δtδ xx )uijn

uijn+1 = (1 + 0.5 K y Δtδ yy )uijn+1/ 2 .

When Δx = Δy this scheme is stable for μ ≤ 1/ 2 , which is only half as restrictive as our full one-step 2-D explicit
scheme (FTCS which has μ ≤ 1/ 4 ).

35
When the implicit Crank-Nicolson scheme is used to solve those two equations, i.e.,

(1 − 0.5 K x Δtδ xx )uijn+1/ 2 = (1 + 0.5 K x Δtδ xx )uijn

(1 − 0.5 K y Δtδ yy )uijn+1 = (1 + 0.5 K y Δtδ yy )uijn+1/ 2 .

This scheme is stable for all μ, and we solve two tridiagonal systems per time step.

In practice, we often use the xyyx ordering to avoid directional bias.

The direction splitting method can also be applied to hyperbolic equations, which is the subject of our next chapter.

36
Chapter 3. Finite Difference Methods for Hyperbolic Equations

3.1. Introduction

Most hyperbolic problems involve the transport of fluid properties. In the equations of motion, the term describing
the transport process is often called convection or advection.

E.g., the 1-D equation of motion is

du ∂u ∂u 1 ∂p
= +u =− + v∇ 2u . (1)
dt ∂t ∂x ρ ∂x

∂u
Here the advection term u term is nonlinear.
∂x

We will focus first on linear advection problem, and move to nonlinear problems later.

From (1), we can see the transport process can be expressed in the Lagrangian form (in which the change of
momentum u along with a particle, du/dt, is used) and the Eulerian form. With the former, advection term does not
explicitly appear. Later in this course, we will also discuss semi-Lagrangian method for solving the transport
problems. In this chapter, we discuss only the Eulerian advection equation.

3.2. Linear convection – 1-D wave equation

3.2.1. The wave equations

3-1
The classical 2nd-order hyperbolic wave equation is

∂ 2u 2 ∂u
2
=c . (2)
∂t 2 ∂x 2

The equation describes wave propagation at a speed of c in two directions.

The 1st-order equation that has properties similar to (2) is

∂u ∂u
+c = 0. (3)
∂t ∂x

Note that Eq.(2) can be obtained from Eq.(3), by taking a time derivative of (3) and resubstituting (3) into the new
equation.

For a pure initial value problem with initial condition

u(x, 0) = F(x), −∞ < x < ∞ ,

the exact solution to (3) is u(x,t) = F(x-ct), which we have obtained earlier using the method of characteristics. We
know that the solution represents a signal propagating at speed c.

3.2.2. Centered in time and space (CTCS) FD scheme for 1-D wave equation

We apply the centered in time and space (CTCS) scheme to Eq.(2):

3-2
 uin+1 − 2uin + uin−1 2 ui +1 − 2ui + ui −1
n n n
− c = 0. (4)
Δt 2 Δx 2

We find for this scheme,

τ = O(Δt 2 + Δx 2 ) .

Performing von Neumann stability analysis, we can obtain a quadratic equation for amplification factor λ:

1/ 2
⎛ k Δx ⎞ ⎛ k Δx ⎞ ⎡ 2 2 ⎛ k Δx ⎞ ⎤
λ± = 1 − 2μ sin ⎜
2 2
⎟ ± 2μ sin ⎜ ⎟ ⎢ μ sin ⎜ ⎟ − 1⎥
⎝ 2 ⎠ ⎝ 2 ⎠⎣ ⎝ 2 ⎠ ⎦

cΔt
where μ = which is the fraction of zone distance moved in Δt at speed c.
Δx

⎛ k Δx ⎞
Let θ = μ sin ⎜ ⎟ , we have
⎝ 2 ⎠

λ± = 1 − 2θ 2 ± 2θ [θ 2 − 1]1/ 2 .

We want to see under what condition, if any, | λ± |≤ 1 .

We consider two possible cases.

Case I: If θ ≤1, then λ is complex:

3-3
 λ± = 1 − 2θ 2 ± i 2θ [1 − θ 2 ]1/ 2

Æ | λ± |2 = (1 − 2θ 2 ) 2 + 4θ 2 [1 − θ 2 ] = 1

Therefore, when θ ≤ 1, the amplification factor is always 1, which is what we want for pure advection!

⎛ k Δx ⎞
θ ≤ 1 Æ p 2 sin 2⎜ ⎟ ≤1
⎝ 2 ⎠

We want the above to be true for all k, therefore p2 ≤ 1 has to be satisfied for all value of sin2().

cΔt
p2 ≤ 1 Æ p = ≤ 1,
Δx

which is the same as the condition we obtained earlier using energy method for FTUS scheme.

Case II:

If θ ≥ 1, λ is real:

λ± = 1 − 2θ 2 ± 2θ [θ 2 − 1]1/ 2 Æ

| λ± |2 = (1 − 2θ 2 ± 2θ [θ 2 − 1]1/ 2 ) 2 ,

you can show for yourself that | λ± |> 1 therefore the scheme is unstable.

3-4
3.2.3. Courant-Friedrichs-Lewy (CFL) Stability Criterion

Let’s consider the stability condition obtained above using the concept of domain of dependence.

Recall from earlier discussion, the solution at (x1, t1) depends on data in the interval [x1 – at1, x1 + at1], and the
D.O.D. is the area enclosed by the two characteristics lines (note here a instead of c is the advection speed)

Based on the following discretization stencil,

we can construct a numerical domain of dependence below:

3-5
Case I: When the numerical DOD is smaller than the PDE's DOD (which usually happens when Δt is large), the
numerical solution cannot be expected to converge to the true solution, because the numerical solution is not using
part of the initial condition, e.g., the initial values in the intervals of A and B. The true solution, however, is
definitely dependent on the initial values in these intervals. Different initial values there will result in different true
solutions, while the numerical solution remains unaffected by their values. We therefore cannot expect the
solutions to match.

The numerical solution must then be unstable. Otherwise, the Lax's Equivalence theorem is violated.

The above situation occurs when Δt / Δx > 1/ c Æ unstable solution. This agrees with the result of our stability
analysis.

Case II: When Δt / Δx = 1/c, the PDE DOD coincides with the numerical DOD, the scheme is stable.

3-6
Case III: When Δt / Δx < 1/c, the PDE DOD is contained within the numerical DOD:

the numerical solution now fully depends on the initial condition. It is possible for the scheme to be stable. In the
case of CTCS scheme, it is indeed the case.

cΔt
Definition: = σ = Courant number
Δx

The condition that σ ≤ 1 for stability is known as the Courant-Friedrichs-Lewy (CFL) stability criterion.

The CFL condition requires that the numerical domain of dependence of a finite difference scheme include the
domain of dependence of the associated partial differential equation.

Satisfaction of the CFL condition is a necessary, not a sufficient condition for stability.

3-7
E.g., the second-order centered-in-time and fourth-order centered-in-space scheme for a 1-D advection equation
requires σ ≤ 0.728 for stability whereas the D.O.D condition requires that σ ≤ 2.

Example 2: The forward-in-time, centered-in-space scheme is absolutely unstable, even if the CFL condition is
satisfied.

The DOD concept explains why implicit schemes can be unconditionally stable – it is because their numerical
DOD always contains the PDE's DOD

e.g., the second-order in time and space implicit scheme for wave equation (2):

c2
δ tt u = [δ xxu n+1 + 2δ xxu n + δ xxu n−1 ] .
n

is stable for all σ.

The numerical DOD is:

3-8
The numerical DOD covers the PDE's DOD.

Read Durran sections 2.2.2 and 2.2.3, which discuss the CFL criterion using the forward-in-time upstream-in-space
(also called upwind) scheme.

3-9
3.3. Phase and Amplitude Errors of 1-D Advection Equation

Reading: Duran section 2.4.2. Tannehill et al section 4.1.2.

The following example F.D. solutions of a 1D advection equation show errors in both the wave amplitude and
phase.

3-11
In this section, we will examine the truncation errors and try to understand their behaviors.

3.3.1. Modified equation

The 1D advection equation is

∂u ∂u
+c = 0. (5)
∂t ∂x

Upwind or Donor-Cell Approximation

We have discussed earlier the stability of the forward-in-time upstream-in-space approximation to the 1D
advection equation, using the energy method. The FDE is

uin+1 − uin uin − uin−1

+c =0 (6)
Δt Δx

Here we assume c>0, therefore the scheme is upstream in space.

We can find from (6) that

∂u ∂u Δt c Δx ( Δt ) 2 c ( Δx ) 2
+c = − utt + u xx − uttt − u xxx + O(Δx3 + Δt 3 ) (7)
∂t ∂x 2 2 6 6

and the right hand side is the truncation error. The scripts x and t denote partial derivative.

3-12
An analysis of τ can reveal a lot about the expected behavior of the numerical solution, and to investigate, we
develop what is known as the Modified Equation. In this method, we rewrite τ so as to illustrate the anticipated
error types.

Dispersion Error – occurs when the leading terms in τ have odd-order derivatives. They are characterized by
oscillations or small wiggles in the solution, mostly in the form of moving waves.

In the above example, the thick line is the true solution and thin line is the numerical solution.

It's called dispersion error because waves of different wavelengths propagate at different speed (i.e., wave speed =
a function of k) in the numerical solution due to numerical approximations – causing dispersion of waves. For the
PDE, all Fourier components described by Eq.(5) should move at the same speed, c.

Dissipation Error – occurs when the leading terms in τ have even-order derivatives. They are characterized by a
loss of wave amplitude. The effect is also called artificial viscosity , which is implicit in the
numerical solution.

3-13
The combined effect of dissipation and dispersion is often called diffusion.

To isolate these errors, we derive the Modified Equation, which is the PDE that is actually solved when a FD
scheme is applied to the PDE.

The modified equation is obtained by replacing time derivatives in the truncation error by the spatial derivatives.

Let's do this for Eq.(7).

To replace utt in right hand side of (7), we take ( Eq.(7) )t [note here we use the FDE not the PDE] Æ

Δt cΔ x (Δt ) 2 c ( Δx ) 2
utt + cu xt = − uttt + u xxt − utttt − u xxxt + ... (8)
2 2 6 6

and – c ( Eq(7) )x

cΔt c 2 Δx c ( Δt ) 2 c 2 ( Δx ) 2
−cutx − c u xx =
2
uttx − u xxx + utttx + u xxxx + ... (9)
2 2 6 6

3-14
then add (8) and (9) Æ

⎛ −uttt c ⎞ ⎛c c2 ⎞
utt = c u xx + Δt ⎜
2
+ uttx + O (Δt ) ⎟ + Δx ⎜ u xxt − u xxx + O(Δx) ⎟ . (10)
⎝ 2 2 ⎠ ⎝2 2 ⎠

Similarly, we can obtain other time derivatives, uttt found in (7) and (10) and uttx and uxxt found in (10). They are
(see Table 4.1 of Dannehill et al):

uttt = −c3u xxx + O(Δx + Δt ) ,

uttx = c 2u xxx + O(Δx + Δt ) , (11)
u xxt = −cu xxx + O(Δx + Δt ) .

Combining (7), (10) and (11) Æ

cΔt c(Δx) 2
ut + cu x =
2
(1 − μ ) u xx −
6
( 2 μ 2 − 3μ + 1) u xxx + O(Δx 3 , Δx 2Δt , Δt 2Δx, Δt 3 ) (12)

cΔt
where μ = .
Δx

Eq.(12) is the modified equation, which shows the error terms relative to the original PDE.

∂ 2u
Note that the leading term has as form of K which, for 1- μ > 0, represent the dissipation process and therefore
∂x 2
the dominant error is of a dissipation nature.

3-15
Note that if we had used CTCS scheme δ2t + c δ2xu = 0, then the leading error term in the modified equation would
be

c(Δx) 2 2 ∂ 3u
⎡ μ − 1⎤⎦ 3 . (12)
6 ⎣ ∂x

It contains the third (odd) order derivative, and the dominant error is of the dispersive nature.

Returning to the upstream scheme, we find that when μ = c Δt/Δx = 1, the scheme is exact!! The coefficient of the
cΔt
leading error term, (1 − μ ) , is called the artificial viscosity, and when μ ≠ 1, causes severe damping of the
2
computational solution (see figure shown earlier). In fact, the Doner-Cell is well known for its strong damping.

Read Tannehill et al, Section 4.1.2.

3-16
3.3.2. Quantitative Estimate of Phase and Amplitude Errors

Reading: Sections 4.1.2 – 4.1.12 of Tannehill et al.

Sections 2.5.1 and 2.5.2 of Durran.

By examining the leading order error in the modified equation, we can find the basic nature of the error. To
estimate the error quantitatively, we use either analytical (as part of the stability analysis) or numerical method. We
will first look at the former.

With the stability analysis, we were already examining the amplitude of waves in the numerical solution. For a
linear advection equation, we want the amplification factor to be 1, so that the wave does not grow or decay in
time. The von Neumann stability analysis actually also provides the information about propagation (phase) speed
of the waves. Any difference between the numerical phase speed and true phase speed is the phase error.

Going back to the figure we showed at the beginning of this section (section 3.3) and reproduced in the following,
we can see that the first-order FTUS scheme has strong amplitude error but little phase error, while the 2nd-order
CTCS scheme has large phase error but small amplitude error. The 4th-order CTCS scheme has a smaller phase
error than its 2nd-order counterpart.

3-17
Amplitude Error

Recall that in the Neumann stability analysis, the frequency ω can be complex, and if it is, the waves will either
decay or grow in amplitude – which is entirely computational for a pure advection problem. This is so because

if ω is real,

| λ | = |exp(−iωΔt ) | = |cos(ωΔt ) + i sin(ωΔt ) | =1 Æ no change in amplitude.

3-18
if ω is complex, i.e., ω = ωR + i ωI ,

| λ | = |exp(−iωΔt ) | = |exp(−iωR Δt )exp(ωI Δt ) | =| exp(ωI Δt ) |≠ 1 .

When ωΙ > 0, | λ | > 1, the wave grows and when ωΙ < 0, | λ | < 1 the wave decays (is damped). | λ | is the
amplitude change per time step and | λ |Ν is the total amplitude change after N steps.

If, e.g., | λ | = 0.95, then after 100 steps, the amplitude becomes 5.92 x 10-3!

Remember for PDE ux + c ux = 0, the frequency ω is always real. Assuming wave solution u = U exp[ i (kx - ωt) ],
you can find ω = kc, which is called the dispersion relation in wave dynamics. For the current problem the phase
speed of waves is ω/k = c which is the same for all wave components. Therefore the analytic waves are non-
dispersive.

Phase Error

For convenience, let's define

ωa = frequency of the analytical solution (PDE)

ωd = frequency of discrete solution (FDE)

then
λa ≡ exp(−iωa Δt ), λd ≡ exp(−iωd Δt ).

Recall that if z = x + i y ( i = −1 ), we can use the polar form and write

3-19
 z = | z | exp( i θ ) or z = | z | (cosθ + i sinθ )

where | z | = ( x2 + y2 )1/2 is called the modulus of z.

Thus,

λa = | λa | exp( i θa ) = exp( i θa ) ( because | λa | = 1 for advection problem).

We define θa = the phase change per time step of the analytic solution

= - ωa Δt ~ frequency × time step size

For the finite difference solution, ω will, in general, be complex, i.e., ω has an imaginary part:

ωd = (ωd)R + i (ωd)I

∴ λd = exp[ (ωd)I Δt ] exp[ -i (ωd)R Δt ] = | λd | exp( i θd )Æ (13)

θd ≡ − (ωd)R Δt is the phase change per time step associated with the F.D. scheme.
|λd| is not necessarily 1 here.

Im{λd }
From (13), we can see that θ d = atan . (14)
Re{λd }

θ d −(ωd ) R Δt kcd cd
Taking the radio, = = =
θa −ωa Δt kca ca

3-20
for the same wave number, and the ratio tells us about the relative phase error.

If cd / ca < 1, the F.D. solution lags the analytic solution (moves slower)
If cd / ca > 1, the F.D. solution leads the analytic solution (moves faster)
If cd / ca = 1, the F.D. solution has no phase error

a) First-order upwind scheme

Let's now apply these notations of phase and amplitude error to the first-order upwind (donor-cell) scheme.

uin+1 − uin uin − uin−1

+c =0 (15)
Δt Δx

Using Neumann method, assume that uin = U λdn exp(ikxi ) , you can show for yourself that

λd = 1 - μ + μ cos( kΔx) – i μ sin( kΔx ) (16)

where μ = cΔt/Δx is the Courant number.

| λd |2 = 1 + 2μ (μ - 1) [1- cos( kΔx) ]

since 1-cos( kΔx) ≥ 0,

| λd | ≤ 1 when μ ≤ 1.

Look at the 2Δx wave, kΔx = (2π/w.l.) Δx = 2π/(2Δx) Δx = π

3-21
 | λd |2 = 1 + 2μ (μ - 1) [1+1] = 1 + 4μ ( μ - 1). (17)

When μ = 1, | λd | = 1, there is no amplitude error.

When μ = 0.5, | λd | = 0, the 2 Δx wave is completed damped in one time step!!

For a 4Δx wave and when μ = 0.5, | λd | = 0.5 Æ 4Δx waves are damped by half in one time step!

Therefore, the upwind advection scheme is strongly damping. It should not be used except for some special reason.

The following figure shows the amplification modulus for the upwind scheme plotted for different values of μ
(υ in the figure), the Courant number.

3-22
β in the figure is our kΔx, and πΔx/L ≤ kΔx ≤ π. The lower and upper limits of kΔx correspond to 2L and 2Δx
waves, respectively. L is the length of the computational domain.

This is so because the shortest wave supported is 2Δx in wavelength, Æ

kΔx = 2π / (2Δx) Δx = π

The longest wave supported by a domain of length L is 2L in wavelength Æ

kΔx = 2π/(2L) Δx = πΔx/L

kΔx Æ 0 when L Æ ∞.

For example, for a 4Δx wave, kΔx = π/2

From the figure, we see that:

When μ = 1, the amplification factor is 1, there is no amplitude error for all values of kΔx ( β ), i.e., for all waves.

When μ > 1, the amplification factor is > 1 for all kΔx except for wave number zero. The amplification factor is the
largest for the shortest wave (kΔx = π), implying that the 2Δx wave will grow the fastest when μ >1, in another
word, the 2Δx wave is most unstable.

This is why we see grid-scale noises when the solution blows up!

3-23
When μ < 1, all waves are stable but are significantly damped. Again, the amplitude error is larger for shorter
waves (larger kΔx). For Courant number of 0.75, the amplitude of the 2Δx wave is reduced by half after one single
time step. The error is even bigger when μ = 0.5.

From the above, we can see that the numerical solution is poorest for the shortest waves, and as the wavelength
increases, the solution becomes increasingly accurate. This is so because longer waves are sampled by a large
number of grid points and are, therefore, well resolved.

2Δx wave is special in that it is often the most unstable when stability criterion is violated, and when the solution is
stable it tends to be most inaccurate.

For general cases, it is impossible to ensure μ =1 everywhere unless the advection speed is constant. Therefore,
strong damping is inevitable with the upwind scheme. You will see severely smoothed solution when using this
scheme.

The damping behavior of the upwind scheme can also be understood from the modified equation (13) discussed
∂ 2u
earlier. The leading error term is of the form K 2 which represents dissipation/diffusion.
∂x

Now, let's examine the dispersion (phase) error of the upwind scheme.

According to earlier definitions and (16)

θa = - ωa Δt = - c kΔt = - μ kΔx

3-24
 Im{λd } − μ sin(k Δx)
θ d = atan = atan
Re{λd } 1 − μ + μ cos(k Δx)

θd
From the above, the ratio of the numerical to analytic phase speed, , can be calculated.
θa

In the following figure this ratio is plotted as a function of β (kΔx) for Courant number μ (υ in the figure) = 0.25,
0.5 and 0.75.

3-25
We can see that there is no phase error (corresponding to the unit circle) when μ =0.5.

All waves are slowed down when μ < 0.5. All waves are accelerated when 0.5 < μ < 1.0.

Again the phase error is larger for short waves (larger β, i.e., kΔx). The error is greatest for the 2Δx wave.

For μ=0.25, θd Æ 0 when kΔx Æ π, i.e., 2Δx wave does not move at all!

Because the F.D. phase speed is dependent on wavenumber k, the numerical solution is dispersive, whereas the
analytical solution is not.

From the above discussions, we see that when using the upwind scheme the waves that move too slow are also
strongly damped.

The upwind advection scheme is actually a monotonic scheme – it does not generate new extrema (minimum or
maximum) that are not already in the field. For a positive field such as density, it will not generate negative values.
We will discuss more about the monotonicity of numerical schemes later.

Note that for practical problems, c can change sign in a computational domain. In that case, which point (on the
right or left) to use in the spatial difference depends on the local sign of c:

uin+1 − uin u n − uin−1

+c i =0 when c>0
Δt Δx
uin+1 − uin uin+1 − uin
+c =0 when c<0 (18)
Δt Δx

3-26
Using the following definitions:

c+= ( c + |c| )/2, c- = ( c - |c| )/2,

the upwind scheme in (19) can be written into a single expression

Δt + n
uin+1 = uin + ⎡⎣c (ui − uin−1 ) + c − (uin+1 − uin ) ⎤⎦ (19)
Δx

Substituting c+ and c- into (18) yields

n +1 c(uin+1 − uin−1 ) Δt Δx | c | (uin+1 − 2uin + uin−1 )

u = u + Δt
n
+ . (20)
2Δx Δx 2
i i
2

One can see that the 2nd term on RHS is the advection term in centered difference form and the 3rd term has a
form of diffusion. If one uses forward-in-time centered-in-space scheme to discretize equation (5), one will get a
FDE like (2) except for the 3rd term on RHS. The scheme is known as the Euler explicit scheme, and the stability
analysis tells us that it is absolutely unstable. So it should never be used. Apparently, the 'diffusion term' included
in the upwind scheme stabilizes the upwind scheme – it is achieved by damping the otherwise growing short
waves.

The included 'diffusion term' also introduces excessively damping to the short waves, as seen earlier. One possible
remedy is to attempt to remove this excessive diffusion through one or several corrective steps. This is exactly
what is done in Smolarkiewicz (1983, 1984) scheme, which is rather popular in the field of meteorology.

Because Smolarkiewicz scheme is based on the upwind scheme, it maintains the positive definiteness of the
advected fields therefore is a good choice for advecting mass and water variables.

3-27
References:

Smolarkiewicz, P. K., 1983: A simple positive definite advection scheme with small implicit diffusion. Mon. Wea.
Rev., 111, 479-486.

Smolarkiewicz, P. K., 1984: A fully multidimensional positive definite advection transport algorithm with small
implicit diffusion. J. Comput. Phys., 54, 325-362.

3-28
b). Leapfrog scheme for advection

In this section, we examine a perhaps most commonly used scheme in atmospheric models – the leapfrog centered
advection scheme.

Here leapfrog refers to finite difference in time – the frog leaps over time level n from n-1 to n+1 – it is a name for
the second-order centered difference in time.

Leapfrog scheme is usually used together with centered difference in space – and the latter can be of 2nd or higher
order.

The leapfrog scheme gives us second order accuracy in time.

The PDE is

uin+1 − uin−1 un − un
+ c i +1 i −1 = 0 (21)
2 Δt 2 Δx

τ = O(Δx 2 , Δt 2 )

and (show it yourself)

λ± = −iμ sin(k Δx) ± [1 − μ 2 sin 2 (k Δx)]1/ 2 (22)

If 1 − μ 2 sin 2 (k Δx) ≥ 0, then | λ± | ≡ 1 and there is no amplitude error for all waves. This is the most attractive
property of the leapfrog scheme.

3-29
In (22), we see that there are two roots for λ - one of them is actually non-physical and is known as the
computational mode.

Which one is computational and how does it behave?

Let's look at the positive root λ+ first:

λ+ = | λ+ | exp( -i β+ )

where β+ = - θd (θd is the phase change in one time step for the discretized scheme, as defined earlier).

If μ ≤ 1, | λ+ | = 1.

λ+ = cos( β + ) − i sin( β + ) = −ia + [1 − a 2 ]1/ 2

where a = μ sin(k Δx) , therefore

β + = sin −1[ μ sin( k Δx)] .

Now consider the negative root λ− :

λ− = | λ- | exp( - i β- )

If μ ≤ 1, | λ- | = 1.

3-30
 λ− = cos( β − ) − i sin( β − ) = −ia − [1 − a 2 ]1/ 2

with the aid of the following schematics, we can see that

β − + β + = −π Æ −β− = π + β+ .

We see that the phase of the negative root is the same as that of the positive root shifted by π then multiplied by –1.

3-31
What does all this mean then?

For a single wave k, we can write the solution as a linear combination of these two modes (since both modes are
present):

uin = ( Aλ+n + Bλ−n )eikxi

= [ Ae−inβ+ + Bein(π + β+ ) ]eikxi (23)
= [ Ae−inβ+ + B(−1)n einβ+ ]eikxi

where A and B are the amplitude of these two modes at time 0.

Which root corresponds to the computational mode then? The negative one, the one that give rises to the second
term in (23), because of the following observations:

(1) The computational mode changes sign every time step. The period of oscillation is 2Δt.

(2) It has a phase opposite to the physical mode, therefore it propagates in the opposite direction from the physical
mode.

(3) Because of the 2Δt period, the computational mode can be damped effectively using a time filter, which will be
discussed in next section.

(4) The presence of the computational mode is due to the use of three time levels, which requires two initial
conditions instead of one – the first and second time step integrations start from time level –1 and 0
respectively, which are two different initial conditions.

3-32
 In practice, we usually have only one initial condition – we often start the time integration by using forward-in-
time scheme for the first step, i.e., for the first step, we do

ui1 − ui0 ui0+1 − ui0−1

= −c
Δt 2Δx

and for the second, we do

ui2 − ui0 ui1+1 − ui1−1

+c = 0.
2Δt 2Δx

cΔt n
An additional note: when uin+1 = uin−1 − (ui +1 − uin−1 ) is used to integrate the advection equation, we can
Δx
experience the grid separation problem, as show schematically below:

3-33
Due to the layout of the computational stencil, the solution at cross points never knows what's going on at the dot
points. As the solution march forward in time, the solutions at neighboring points can split away from each other.
This problem is also related to the use of three time levels, and can be alleviated by the use of Asselin time filter.
An artificial spatial smoothing term of the form of K∂2u/∂x2 will also help. In practice, other forcing terms in the
equation can also couple the solutions together.

3-34
3-35
c) Asselin Time Filter

The Asselin (also called Robert-Asselin) time filter (Robert 1966; Asselin 1972) is designed to re-couple of the
splitting solutions in time and damp the computational mode found with the leapfrog scheme and others.

It is a two-step process:

(1) u is integrated to time level n + 1 using the regular leapfrog scheme,

ui*n+1 = uin−1 − μ (ui*+n1 − ui*−n1 ) (24)

where * indicates values that have not been 'smoothed'.

(2) a filter is then applied to three time levels of data

uin = ui*n + ε (ui*n+1 − 2ui*n + uin−1 ) . (25)

Note that the term in second term in (15) is a finite difference version of ∂2u/∂t2 - the diffusion in time which tends
to damp high-frequency oscillations.

If we use (25) in (24), we can do a stability analysis and examine the impact of the time filter on solution accuracy:

λ± = −ia + ε ± [b − a 2 ]1/ 2 (26)

where a = μ sin(k Δx) and b = (1- ε)2 [compare (22) with (26)].

3-36
If b – a2 ≥ 0 (note that this stability condition has also changed), we have

| λ |2 = (ε2 + b) ± 2ε ( b – a2)1/2.

We can plot this to determine its effect on the solution.

We will find that:

(1) amplitude error is introduced by the time filter;

(2) the time filter reduces the time integration scheme from second-order accurate to first-order accuracy only
(3) the filter makes the stability condition more restrictive (can use smaller Δt now).

3-37
3-38
We want to use as small a ε as possible. Typically ε = 0.05 to 0.1.

The leapfrog (2nd or 4th-order) centered difference scheme combined with the Asselin filter is used in the ARPS
for the advective process (more complex monotonic advection schemes are also available for scalar advection).

Reference:

Robert, A. J., 1966: The integration of a low order spectral form of the primitive meteorological equations. J.
Meteor. Soc. Japan, 44, 237-245.

Asselin, R., 1972: Frequency filter for time integration. Monthly Weather Review, 100, 487-490.

Reading: Durran Section 2.3.5.

3-39
d) Adam-Bashforth schemes

Second-order Adam-Bashforth Scheme

uin+1 − uin ⎡ 3 un − un 1 u n−1 − uin−−11 ⎤

= −c ⎢ i +1 i −1 − i +1
Δt ⎣ 2 2 Δx 2 2Δx ⎥⎦

• The RHS is a linear extrapolation of δ2xu from n-1 and n to n+1/2, so that the scheme is "centered" in time at
n+1/2.
• Second order in time and space
• Stability analysis shows that

1⎡ 3 ⎤
1/ 2
⎛ 9 ⎞
λ± = ⎢1 − is ± ⎜1 − s 2 − i s ⎟ ⎥
2 ⎢⎣ 2 ⎝ 4 ⎠ ⎥⎦

where s = μ sin(k Δx) .

Note that this scheme is also a 3 time level scheme and 3 time level schemes always have two modes – one
physical and one computational. We can see that for the physical mode (+ case), λ Æ 1 as s Æ 0, and for the
computational mode (minus sign case), λ Æ 0 as s Æ 0.

If s << 1, we can show that

| λ+ | ≈ (1 + s4/4 )1/2

3-40
 | λ− | ≈ 0.5 s (1 + s2 )1/2

(You can show it by performing binomial expansion).

Clearly | λ+ | > 1 for s ≠ 0 therefore the scheme is absolutely unstable.

However, for small enough values of s (i.e., Courant number), because s is raised to the 4th power, | λ+ | can be
close enough to 1 so that the growth rate is small enough for the scheme to be still usable (especially computational
diffusion is added to the equation).

One can estimate the growth rate in terms of e-folding time – i.e., the time taken for a wave to growth by a
factor of e.

However, it is the higher-order Adam-Bashforth (AB) schemes that we are more interested in. The higher-order
AB scheme can be obtained by extrapolating the right hand side of the equation (i.e., F in ut = F) to time level
n+1/2, as we do for the 2nd-order AB scheme, but using high-order (e.g., 2nd-order) polynomials, which will also
involve more time levels.

Third-order Adam-Bashforth Scheme

The 3rd-order AB scheme thus obtained has the form of

uin+1 − uin c
= − [23δ 2 xu n − 16δ 2 xu n−1 + 5δ 2 xu n−2 ]
Δt 12

• It involves data at four time levels – require more storage space.

• And it has two computational modes and one physical mode.

3-41
• The computational modes are strongly damped, however, unlike the leapfrog scheme, so there is no need for
time filtering.

3-42
• Most accurate results are obtained for μ near stability limit. This is not true for the leapfrog 4th-order
centered in space scheme. That solution is more accurate for μ < 0.5 where certain cancellation between time
and space truncation errors occurs.

3-43
3-44
• Durran (1991 MWR) shows that 3rd-order AB time difference combined with 4th-order spatial difference is
a good choice – it is in general more accurate than the commonly used leapfrog 4th-order centered-in-space
scheme.

3-45
e) Other schemes

There are many other schemes for solving the advection equation. In the following are some of them, given
together with brief discussions on their important properties.

3-46
Euler explicit (Euler refers to forward in time)

uin+1 − uin uin+1 − uin

+c = 0, c > 0 - forward-in-time, downstream in space
Δt Δx
uin+1 − uin uin+1 − uin−1
+c =0 - forward-in-time, centered in space
Δt 2Δx

• Both schemes are absolutely unstable. You can show it for yourself.
• They are of no use.

Lax Method

uin+1 − (uin+1 + uin−1 ) / 2 uin+1 − uin−1

+c =0
Δt 2Δx

• 1st-order in time, 2nd-order in space.

• Stable when |μ| ≤ 1.
• Large dissipation error.
• Significant leading phase error - waves propagate faster.
2Δx waves twice as fast when μ=0.2.

Lax-Wendroff

uin+1 − uin uin+1 − uin−1 c 2 Δt uin+1 − 2uin + uin−1

= −c +
Δt 2Δx 2 (Δx) 2

3-47
 • Effectively an Euler explicit (FTCS) scheme plus a diffusion term.

Its derivation is interesting – it's based the Taylor series expansion in time first:

1
uin+1 = uin + Δtut + (Δt ) 2 utt + O(Δt 3 )
2

and use ut = - c ux and utt = c2 uxx to rewrite it as

1
uin+1 = uin − cΔtu x + c 2 (Δt ) 2 u xx + O(Δt 3 ) .
2

It is then discretized in space.

• Stable when |μ| ≤ 1

• Amplitude (dissipation) error for short waves
• Mostly lagging phase error, for short waves. Leading phase error for shortest waves when μ is near 0.75.

We have actually obtained this scheme before based on characteristics and second order interpolation. See Section
2.3.

MacCormack (an example of two-step predictor-corrector method)

uin+1 − uin
Predictor: (uin+1 )* = uin − cΔt
Δx

3-48
 n +1 1⎡ n n +1 * (uin+1 )* − (uin−+11 )* ⎤
Corrector: u = ⎢ui + (ui ) − cΔt ⎥
Δx
i
2⎣ ⎦

• Combination of upwind and downwind steps

• Intermediate prediction is used in the second corrector step
• In the corrector step, the time difference is 'backward in time'
• For linear advection equation, this scheme is equivalent to (you can show this by substituting the 1st eq. into
the 2nd), therefore its properties are the same as, the Lax-Wendroff scheme.

Euler Implicit (Euler refers to forward in time)

uin+1 − uin u n+1 − uin−+11

+ c i +1 =0
Δt 2 Δx

• 1st-order in time and 2nd-order in space.

• Unconditionally stable.
• Relatively small dissipation error, only for intermediate wave lengths.
No dissipation error for longest and shortest waves.
• Significant lagging phase error for short waves.
• Need to solve a coupled system of equations.
Tridiagonal in 1-D. Block trigiagonal in 2-D.

Time-centered Implicit (Trapezoidal)

3-49
 uin+1 − uin c ⎡ uin++11 − uin−+11 uin+1 − uin−1 ⎤
+ ⎢ + =0
Δt 2 ⎣ 2Δx 2Δx ⎥⎦

• 2nd-order in both time and space.

• Absolutely stable.
• No dissipation error for all waves (similar to leapfrog scheme
which is also 2nd-order accurate in time)
• Significant lagging phase error for short waves, similar to Euler implicit.

Matsuno (forward-backward two-step) Scheme

(uin+1 )* − uin un − un
+ c i +1 i −1 = 0
Δt 2Δx
n +1
ui − ui n
(ui +1 ) − (uin−+11 )*
n +1 *
+c =0
Δt 2Δx

• 1st-order in time, second order in space

• Stable when μ ≤ 1.
• Relatively large dissipation and phase error

Leapfrog Fourth-order Centered-in-Space Scheme

uin+1 − uin−1 ⎡ 4 uin+1 − uin−1 1 uin+ 2 − uin−2 ⎤

+ c⎢ − =0
2Δt ⎣ 3 2Δx 3 4Δx ⎥⎦

• 2nd-order in time and 4th-order in space

3-50
 • Stable for μ ≤ 0.728 (more restrictive than 2nd-order)
• No dissipation error without time filter
• Also contains computational mode, as all three-time level schemes do
• Smaller phase error than 2nd-order centered-in-space counterpart
• Leapfrog scheme can be combined with centered spatial
difference schemes of even higher order

Second and third-order Rouge-Kutta Scheme

One possible form (there is more than one form that is second-order accurate) of 2nd-order Rouge-Kutta scheme
with centered spatial difference is

Δt uin+1 − uin−1
u =u − c
* n

2Δx
i i
2
n +1 ui +1 − ui*−1
*
ui = ui − Δtc
n

2Δx

This scheme is absolutely unstable although the stability is weak. It can be combined with upwind biased advection
to yield a stable time integration scheme, however. Wicker and Skamarock (1998) discusses applying such a
scheme to solve a compressible system of equations.

One possible form of 3rd-order Rouge-Kutta scheme with centered spatial difference is

Δt uin+1 − uin−1
u =u − c
* n

2Δx
i i
3

3-51
 Δt ui*+1 − ui*−1
u =u − c
** n

2Δx
i i
2
n +1 ui +1 − ui**−1
**
ui = ui − Δtc
n

2Δx

The scheme involves three steps, therefore three evaluations of the advection term. The increased cost is somewhat
offset by its better stability property. As discussed by Durran (page 68-69), for oscillation equations, it is stable for
kΔt < 1.73 while the leapfrog scheme requires kΔt < 1 therefore the time step size can be 1.73 times larger. Perhaps
more attractively, when the 3rd-order Rouge-Kutta scheme is combined with high-order spatial difference for the
advection term, the maximum stable Courant number is not as much reduced as in the leapfrog case. The following
table is from Wicker and Skamarock (2002).

The ability for the scheme to be combined with high odd order spatial difference and be used in split-explicit time
integration for compressible system of equation, and allowing relatively large time step size is attractive. Third-
order Rouge-Kutta scheme is used in the new Weather Research and Forecast (WRF, http://wrf-model.org) model.
See Durran Section 2.3.6, and Wicker and Skamarock (1998, 2002).

3-52
3-53
3-54
3-55
3-56
List of commonly used time difference schemes and their basic properties (from Durran):

3-57
3-58
3-59
3-60
3.3.3. Practical Measures of Dissipation and Dispersion Errors

Takacs (1985 Mon. Wea. Rev.) proposed a practical measure for estimating dissipation and dispersion errors based
on numerical solutions. The methods divide the total mean square error into two parts, one indicative of dissipation
error and one the dispersion error.

The total mean square error is given as

1 N
τ= ∑
N i
(ua − ud ) 2 . (27)

ua is the analytical solution and ud the numerical (discrete) solution.

It can be rewritten as (show it yourself):

τ = σ 2 (ua ) + σ 2 (ud ) − 2 ρσ (ua )σ (ud ) + (ua − ud ) 2 (28)

1 N 1 N
where σ 2 (ua ) = ∑
N i
(u a − u a ) 2
, σ 2
(u d ) = ∑
N i
(ud − ud ) 2 are the variance of the ua and ud, respectively.

1 N cov(ua , ud )
cov(ua , ud ) = ∑ (ua − ua )(ud − ud ) is the co-variance between ua and ud and ρ = is the correlation
N i σ (ua )σ (ud )
coefficient.

(28) can be rewritten as

3-61
 τ = [σ (ua ) − σ (ud )]2 + (ua − ud ) 2 + 2(1 − ρ )σ (ua )σ (ud ) (29)

Takacs definite the first two terms of the RHS of (29) as the dissipation error and the third term as the dispersion
error, i.e.,

τ DISS = [σ (ua ) − σ (ud )]2 + (ua − ud )2 (30a)

τ DISP = 2(1 − ρ )σ (ua )σ (ud ) (30b)

We can see that when two wave patterns differ only in amplitude but not in phase, their correlation coefficient ρ
should be 1. According to (30a), τDISP = 0. That's a reasonable result.

3-62
3.4. Monotonicity of Advection Schemes
3.4.1. Concept of Monotonicity

When numerical schemes are used to advect a monotonic function, e.g., a monotonically decreasing function of x,
the numerical solutions do not necessarily preserve the mononotic property – in fact, most of the time they do not,
and the errors tend to be large near sharp gradient. This is illustrated in the following:

A few example solutions are given below:

3-1
3-2
Monotonic numerical schemes are ones which, given an initial distribution which is monotonic before advection,
produce a monotonic distribution after advection.

A consequence of this property is that monotonic schemes neither create new extrema in the solution nor amplify
existing extrema.

S.K. Godunov (1959) showed that no schemes having greater than first-order accuracy in space can be monotonic
by construction (i.e., without using some artificial modification to ensure monotonicity). The highly dissipative
upstream scheme is the classic example of a monotonic scheme.

Monotonic schemes are widely used in computational fluid dynamics because they do not allow the Gibbs
Phenomenon to occur. This phenomenon results from attempting to represent a sharp gradient or discontinuity by a
truncated number of waves, and always produces "undershoots and overshoots" relative to the amplitude of the
initial distribution.

• These oscillations typically appear in the "wake" of a traveling wave which exhibits a sharp gradient, but do
not necessarily grow in time.
• They are short waves that become noises in the solution – the damping of them results in smoothing of
numerical solution.
• The oscillations can cause positive-definite fields, such as mass and water, to turn negative.

The Gibbs phenomenon is illustrated in the figure below, which shows how a square wave is represented by
various numbers of waves in a Fourier expansion. Even if 100 terms are retained in the expansion, small over- and
under-shoots remain. Monotonic schemes do not allow such oscillations to occur, i.e., one can think of the
oscillations being removed by very selective damping.

3-3
Spectral methods use truncated spectral series to represent variable fields – they are particularly suspect to the
Gibbs errors.

Monotonic schemes are often constructed by examining local features of the advected field, and adjust the
advective fluxes of certain high-order schemes explicitly so that no new extrema is created in the solution.

3.4.2. Two basic classes of monotonic schemes

One is called the Flux-corrected transport (FCT) scheme, original proposed by Boris and Book (1973) and
extended to multiple dimensions by Zalesak (1979).

With this scheme, the advective fluxes are essentially a weighted average of a lower-order monotonic scheme
(usually 1st-order upwind) and a higher-order non-monotonic scheme. The idea is to use the high-order scheme as
much as one can without violating the monotonicity condition. Details can also be found in Section 5.4 of Durran's
book. In the ARPS, the FCT scheme is available as an option for scalar advection – it is three to four times as
expensive as a regular 1st or second advection, however.

3-4
The other class is the so-called flux limiter method. With this method, the advective fluxes of a high-order scheme
is directly modified (limited by a limiter) and the goal is that the total variation of the solution does not increase in
time and this property is usually referred to as total variation diminishing (TVD).

The total variation of a function φ is defined as

N −1
TV (φ ) = ∑ | φ j +1 −φ j |
j −1

A TVD scheme ensures that TV(φn+1) ≤ TV(φn ).

Sweby (1984) presented a systematic derivation of the flux limiter for this class (see also Durran Section 5.5.1).

With both methods, the flux correction or limiting is done grid point by grid point – in effect, the coefficients of the
finite difference schemes are solution dependent therefore they are often called non-linear schemes.

Recommended Reading: Sections 5.2.1, 5.2.2., 5.3-5.5 of Durran.

Summarizing comments:

By now, you should have realized that no scheme is perfect, although some is better than the others. When we
design or choose a scheme, we need to look at a number of properties, including accuracy (in terms of amplitude
and phase), stability (implicit schemes tends to be more stable), computational complexity (implicit schemes cost
more to solve per step), monotonicity (can we tolerate negative water generation?), and conservation properties etc.
You need also consider the problem at hand – e.g., does it contain sharp gradient that is important to your solution?
What is your target computer?
The computational and storage requirement are other factors to consider.

3-5
3.5. Multi-Dimensional Advection

Reading: Durran section 3.2.1. Smolarkievicz (1982 MWR).

Similar to the diffusion or heat transfer equations, there are three general approaches for solving multi-dimensional
advection equations, namely:

1) Fully multi-dimensional methods

2) Direct extensions of 1-D schemes
3) Directional splitting methods

We will look at each in the following.

3.5.1. Direct Extension

Many 1-D advection schemes can be directly extended to multiple dimensions.

Multi-dimensional extension of 1-D explicit schemes often have a more restrictive stability condition.

We will look at the 2-D leapfrog centered scheme first.

For equation

∂u ∂u ∂u
+ cx + cy =0, (31)
∂t ∂x ∂y

3-6
the leapfrog centered discretization is

c x ∆t n c ∆t
umn+, 1j − umn−, 1j = − (um+1, j − umn −1, j ) − y (umn , j +1 − umn , j −1 ) (32)
∆x ∆y

τ = O ( ∆ x 2 , ∆ y 2 , ∆t 2 )

Let the individual wave component be

umn , j = λ n exp[i (km∆x + lj∆y ] (33)

where k and l are wave number in x and y directions, respectively.

Substituting (33) into (32) and solve for λ, you can obtain (do it yourself):

1/ 2
⎡ c ∆t c ∆t ⎤ ⎧ ⎡ c ∆t c ∆t ⎤⎫
λ± = −i ⎢ x sin(k ∆x) + y sin(l ∆y ) ⎥ ± ⎨1 − ⎢ x sin(k ∆x) + y sin(l ∆y ) ⎥ ⎬ (34)
⎣ ∆x ∆y ⎦ ⎩ ⎣ ∆x ∆y ⎦⎭

Similar to the 1-D case, if

⎡ c ∆t c ∆t ⎤
1 − ⎢ x sin(k ∆x) + y sin(l ∆y ) ⎥ ≥ 0, (35)
⎣ ∆x ∆y ⎦

then | λ± | ≡ 1, the scheme is stable (and has no amplitude error).

3-7
Inequality (35) is satisfied when

c x ∆t c ∆t
sin(k ∆x) + y sin(l ∆y ) ≤ 1. (36)
∆x ∆y

Let's consider the simpler case of ∆x = ∆y = d, and rewrite

cx = us cos(θ), cy = us sin(θ),

where us is the flow speed, (36) then becomes

us ∆t
cos(θ )sin(k ∆x) + sin(θ )sin(l ∆y ) ≤ 1 . (37)
d

Since we want (37) to be satisfied for all possible waves, we choose the most stringent case of sin(k∆x) = 1 and
sin(l∆y) = 1, (37) the becomes

us ∆t
cos(θ ) + sin(θ ) ≤ 1 .
d

The maximum value of cos(θ ) + sin(θ ) is 2 which occurs when θ = π/4, the result is the stability condition for
2-D advection equation in the case of ∆x = ∆y:

us ∆t us ∆t
2 ≤ 1 or ≤ 0.707 (38)
d d

3-8
i.e., the Courant number has to be less than 0.707, instead of 1 as we get for 1-D case.

The reason that ∆t has to be about 30% smaller is explained by the following diagram:

As seen from the figure, for a wave propagating from SW to NE, the effective distance between two grid points is
d/ 2 instead of d. A wave signal cannot propagate more than one (effective) grid interval with this explicit
second-order leapfrog-centered scheme for stability.

3-9
Similar reduction of time step size occurs for most other explicit schemes, including the upwind scheme.

3.5.2. Fully Multi-Dimensional Method

Not all direction extensions of 1-D schemes are stable, unfortunately.

Consider the Lax-Wendroff (also called Crowley) scheme we derived earlier using both second-order interpolation
method (section 2.3 of Chapter 2) and the Taylor series expansion method (section 3.2.5 of this chapter):

1-D Lax-Wendroff or Crowley scheme:

uin+1 − uin uin+1 − uin−1 c 2 ∆t uin+1 − 2uin + uin−1

= −c + (39)
∆t 2∆x 2 ( ∆x ) 2

The scheme is table when |µ| ≤ 1.

Using the notion of finite-difference operators, (39) becomes

(c∆t ) 2
u n +1
i = u − c∆tδ 2 xu +
n
i
n
δ xxu n . (40)
2

Direct extension of (40) into 2-D is:

(cx ∆t ) 2 (c y ∆t ) 2
u n +1
i = u − ∆t (cxδ 2 xu + c yδ 2 y u ) +
n
i
n n
δ xxu +
n
δ yyu n . (41)
2 2

3-10
It turns out that (41) is absolutely unstable. This is because the cross-derivative terms are neglected!

To see it, we need to go back to original derivation of the Lax-Wendroff scheme:

1
u n+1 = u n + ∆tut + (∆t ) 2 utt + O(∆t 3 ) (42)
2

Use ut = -cx ux - cy uy

and utt = -cx utx - cy uty = cx2 uxx + cy2 uyy + 2cxcy uxy ,

and replace the spatial derivatives with the corresponding finite differences, (42) becomes

(cx ∆t ) 2 (c y ∆t ) 2
un +1
i = u − ∆t (cxδ 2 xu + c yδ 2 y u ) +
n
i
n n
δ xxu +
n
δ yyu n + (cx c y ∆t 2 )δ xyu n (43)
2 2

Clearly, the last term on the RHS is additional, compared to (41).

Note that we can also obtain (43) using the characteristics method plus quardratic interpolation, as long as all terms
in the second-order 2-D polynomial are retained.

Equation (43) is an example of fully multidimensional scheme, which is different from the direct extension of 1-D
counterpart.

Smolarkiewicz (1982 MWR) discuss the MD Crowley scheme in details (handout).

3-11
3.5.3. Directional Splitting

It turned out that by using directionally splitting method (i.e., applying 1-D scheme in one direction at a time), the
effect of cross-derivative terms can also be retained and a stable scheme result, with the Lax-Wendroff scheme.

The algorithms is

( c x ∆t ) 2
(uin, +j 1 )* = uin, j − cx ∆tδ 2 xuin, j + δ xxuin, j (44a)
2
(c y ∆t ) 2
ui , j = (ui , j ) − c y ∆tδ 2 y (ui , j ) +
n +1 n +1 * n +1 *
δ yy (uin, +j 1 )* (44b)
2

In this case, we preserve the stability of each step and λ = λx λy.

With the above scheme, we have

Advantages:

1. 1-D advection is straightforward – properties of schemes are well understood.

2. The time step constraint is not as severe as for true multi-dimensional problems.

Disadvantages:

1. We implicitly assume that features move obliquely to the grid may be represented as a series of orthogonal steps
in the coordinate directions:

3-12
 In an implicit scheme, where the time step can be large, these errors can be substantial.

2. The biggest disadvantage is that splitting can introduce an O(∆t2) error in the form of a spurious source term. To
see this, consider the 2-D advection being solved using directional splitting upstream advection:

ut + Uu x + Vu y = 0 (45)

where U=U(x,y) >0 and V(x,y) >0 .

Writing this as a direct extension of the upwind scheme in 1-D, we have

u n+1 = u n − ∆tU δ − xu − ∆tV δ − yu . (46)

The directional splitting version is

3-13
 u * = u n − ∆tU δ − xu (47a)
u n+1 = u * − ∆tV δ − yu * (47b)

Substitute (47a) into (47b), we obtain a single step scheme

u * = u n − ∆tU δ − xu − ∆tV δ − y u + UV ∆t 2δ − xuδ − y u + V ∆t 2δ − xuδ − yU . (48)

We can see that the last term on RHS of (48) is actually spurious and is not zero when U is not constant.

3-14
Chapter 4. Nonlinear Hyperbolic Problems

4.1. Introduction

Reading: Durran sections 3.5-3.6. Mesinger and Arakawa (1976) Chapter 3 sections 6-7.
Supplementary reading: Tannehill et al sections 4.4 and 4.5 – Inviscid and viscous Burgers equations.

Nonlinear problems creates two important problems in CFD:

1. They generate nonlinear instability.

2. New waves can be generated in nonlinear problems via nonlinear wave interaction.

The stability analysis we discussed in the previous Chapter refers to linear stability and linear instability – because
they do not require nonlinearity in the equation.

The above two issues are specific to nonlinear equations.

Many processes in the atmosphere can be nonlinear – many physical processes, such as phase changes are
nonlinear. In the Navier-Stokes equations, the most significant nonlinear term is the advection term.

The simplest equation including nonlinear advection is the Burges Equation:

∂u ∂u
Inviscid: +u =0 (Hyperbolic) (1a)
∂t ∂x

4-1
 ∂u ∂u ∂ 2u
Viscous: +u =υ 2 (Parobolic) (1b)
∂t ∂x ∂x

∂u ∂ ⎛ u 2 ⎞
We can rewrite the advection term u = ⎜ ⎟ - the nonlinearity is often called quadratic nonlinearity.
∂x ∂x ⎝ 2 ⎠

There is a fundamental difference between the inviscid Burger's equation (1a) and the linear advection equation,
∂u ∂u
+ c = 0 , we discussed in last chapter, where c is a constant.
∂t ∂x

1. In the linear problem, all points on the wave move at the same speed, c, the shape of the wave remain
unchanged:

For a nonlinear equation (1b), the wave advects itself such the local speed depends on the wave amplitude and the
shape of the wave change in time:

4-2
The process is called nonlinear steepening, and eventually results in shock waves and overturning if the flow is
inviscid. In this case, the characteristics coalesce into a group where multiple values of u exist for a given x.

2. Nonlinear problems create new waves modes. This was evident in the previous problem where we start with a
single sine wave and ended up with a step-like function. Clearly the step function can be not represented by a

4-3
single wave Æ new waves have been generated! For nonlinear problems, the principle of superposition does not
apply!

To illustrate this, consider u = sin( kx ). Plug it into the advection term Æ

∂u
u = k sin(kx)cos(kx) = k sin(2kx) / 2 .
∂x

Now the system contains a new wave – sin(2kx), whose wave number is 2k, and wavelength is L=π/k, half of the
wave length of the original 2π/k.

The new wave can interact with itself and the original one, the process goes on and on and an entire spectrum of
waves will result! This process is the source of aliasing error, to be discussed soon.

Despite of its nonlinear, Burger's equation has analytical solutions.

For the inviscid case, one of the examples is:

if u(x, t=0) = - U tanh(kx)

then u(x, t) = - U tanh[ k(x - ut) ].

Note that the solution is an implicit function of u, and it has to be solved iteratively for the value of u.

For the viscous case, an example is:

4-4
if u(x, t=0) = - U tanh(kx)

then the steady state solution is

u(x, t)= - U tanh( ux/2υ ).

Here, dissipation of energy within the shock is exactly balanced by the conversion of kinetic energy from infinity.

References for exact solutions:

Platzman, 1964: Tellus, 4, 422-431.

Solution techniques

Many solution techniques discussed earlier for linear advection equation can be used for Burgers equation. We will
not discuss them in details here, but we will look the behavior of the solutions:

4-5
4-6
Because of the nonlinear steepening, the solution contains sharp gradient near the step – numerical schemes tend to
perform poorly near sharp gradient, and most schemes, especially high order ones, generates small scale
oscillations near the sharp gradient – monotonic schemes are particularly good at dealing with sharp gradient,
because they are designed to prevent overshoot and undershoot from being generated.

With conventional schemes, there is a tendency for the small-scale noises to grow quickly and eventually destroy
the solution or cause instability. Such instability occurs only in nonlinear problems, and was first discussed by the
developer of NGM (Nested Grid Model, an early operational numerical weather prediction model of U.S.),
Norman Phillips (1959), and the instability is called Nonlinear Instability.

4.2. Nonlinear Instability

Linear instability occurs when the linear stability criteria is violated, usually when Δt is too large.

Nonlinear instability occurs when waves shorter than 2Δx are generated and feed energy spuriously into the
wavelengths near but larger than 2Δx. The energy buildup becomes catastrophic.

The generation of waves with wavelength < 2Δx is a consequence of aliasing (c.f., p.35-42. Mesinger and Arakawa
1976. Read it!).

Aliasing:

Consider a function u = sin(kx).

We know that the shortest wave that can be represented by a grid has a wavelength of 2Δx, and a wave number of
kmax = 2π/(2Δx) = π/Δx Æ the largest wave number is kmax= π/Δx.

4-7
 ∂u
We saw earlier for the nonlinear advection term u
∂x

∂u
u = k sin(kx)cos(kx) = k sin(2kx) / 2 .
∂x

If k = kmax, then the new wave has a wave number of 2kmax, corresponding to a wavelength of (2Δx)/2 = Δx - too
short to be represented on the grid!

Therefore, nonlinear interaction between waves can generate waves that are unresolvable by the original grid!

Then what happens to these unresolvable waves? They are spuriously presented, or aliased, as resolvable waves!

Consider a wave with W.L. = 4/3Δx (<2Δx). With only three grid points to represent one wavelength, it cannot tell
it apart from the 4Δx wave. In fact, the grid misrepresents it as 4Δx wave!

4-8
Consider now a general case of a function u that contains harmonic components:

u = ∑ un Æ
u

nonlinear term will be of the form

sin( k1x) sin(k2x) = [ cos( k1-k2 ) x – cos( k1+k2) x ]/2

Æ two new waves, k1 ± k2, are created!

Even if the calculation is started with all wavelengths ≥ 2Δx, waves < 2Δx will be generated, through nonlinear
interaction.

To generalize, let's write, knowing kmax Δx= π,

4-9
 cos( kx) = cos[ 2kmax – (2kmax-k) ] xi
2π xi 2π 2π xi 2π
= cos cos( − k ) xi + sin sin( − k ) xi .
Δx Δx Δx Δx

2π xi 2π iΔx 2π xi
Since xi = i Δx, and i is integer, sin = sin = 0 , cos = 1Æ
Δx Δx Δx

cos(kxi ) = cos{[2kmax − k ]xi } .

Knowing only those values at the grid points, we cannot distinguish between wavenumber k and 2kmax-k, thus, if k
> kmax (W.L. < 2Δx), then k is really misrepresented as (or aliased as)

k* = 2kmax –k.

Thus, the aliased wave k* is less than kmax by an amount equal to the amount by which k was greater then kmax:

4-10
Back to our example, let W.L. = 4/3 Δx, this is aliased as

k* = 2π/(2Δx) – 2π/(4/3Δx) = 2π/(4Δx) Æ 4Δx wave

– the same as we saw earlier by the graphic means.

Note that the waves generated by aliasing are always near 2Δx – energy start to pile up in the form of short wave
noises. In the next section, we will look at ways to control such pileup.

4-11
4.3. Controlling Nonlinear Instability
4.3.1. Consequences of N.L. Instability

If a flow contains many modes, it is useful to examine the distribution of energy (a measure of the amplitude of the
modes) as a function of wavenumber:

E=∑
2
u' k
.
2
In a numerical simulation, aliasing occurs near 2Δx Æ energy is shifted to small scales and the short waves grow
with time Æ nonlinear instability.
4.3.2. Filter Method

Phillips (1959) showed that catastrophic growth of wave disturbances can be prevented in a 2-level geostrophic
model, by periodically applying a spectral filter, which eliminates waves shorter than or equal to 4Δx.

The method decomposes the solution into Fourier modes (waves / harmonics), and recomposes them without hence
eliminating the shortest waves.

4-12
Orszag (1971) later showed that it is sufficient to eliminate only waves equal to or shorter than 3Δx (see hand out).

The use of spectral filter is very expensive in grid point model. Doing it in spectral models is straightforward,
however, since the solution is already in the spectral form.

4-13
4.3.3. Spatial Smoothing or Damping

In this case, we apply, at chosen intervals (often every time step), a spatial smoother similar in form to the term in
our parabolic diffusion equation.

We want the smoothing to be selective, so that only the short (aliased) waves get damped.

Filter types:

Low-pass: allows low-frequency or long wavelength waves to pass through

High-pass: allows high-frequency or short wavelength waves to pass through
Band-pass: allows intermediate waves to pass through

What is desired here:

4-14
There exist many types of filters. Let look at one that is commonly used, the 1-2-1 or Shapiro filter:

u j = u j + 0.5υ (u j +1 − 2u j + u j −1 ) (2)

where the u j is the value after smoothing.

To see what the smoother does, we need to look at the response function σ defined by

u =σu .

- all a filter does is changing the wave amplitude (a well-designed filter should not change the phase). Here σ might
be a function of k, Δx, ν etc., much like | λ | earlier.

The method for obtaining σ is very similar to the method for von Neumann stability analysis.

Let u = A exp(ikx j ). Plug it into (2) Æ

u = [1 − υ (1 − cos(k Δx))] A exp(ikx j ) Æ

σ = [1 − υ (1 − cos(k Δx))] -- response function of filter (2).

See Figure.

4-15
One can create multi-dimensional smoothers by successive applications of 1-D smoothers, one can also design
fully MD ones.

4.3.4. Smoothing via numerical diffusion

This method damps the aliased waves by adding a smoothing or diffusion term to the prognostic equations (called
computational mixing term in the ARPS – which also helps to suppress small scale noises created by dispersion
and physical processes. Actually, ARPS uses advective formulations that conserve the advected quantities and their
variances – therefore nonlinear instability due to aliasing is reasonably controlled even without the smoothing).

Consider for example of the CTCS case for linear advection:

4-16
 n −1
⎡ −u ⎤
⎢ δ u ⎥
δ 2t u + u δ 2 xu = α ⎢ xx ⎥ , (3)
⎢ −δ xxxxu ⎥
⎢ ⎥
⎣δ xxxxxxu ⎦

the right RHS terms are called zero, 2nd, 4th and 6th order numerical diffusion / smoothing, respectively. Note that
the diffusion term is evaluated at time level n-1 – this makes the time integration forward in time relative to this
term – remember that forward-in-time is (conditionally) stable for diffusion term but centered-in-time scheme is
absolutely unstable.

We can find the response function to be

⎧ 1
⎪ [2 − 2cos(k Δx)]/ Δx 2
⎪
| λ | = 1 − 2αΔt ⎨
2
(4)
⎪ [6 − 8cos(k Δx) + 2cos(2k Δx)]/ Δx
4

⎪⎩[20 − 30cos( k Δx) + 12cos(2k Δx) − 2cos(3k Δx)]/ Δx 6

and they are plotted in the following figure. We can see that this term selectively damps shorter waves, and the
higher order schemes are more selective, which is desirable. Given
| λ |, you can estimate the amplitude change due to the diffusion for different wavelength after given number of
time steps.

4-17
From Xue (2001, Mon. Wea. Rev.)

4-18
4.3.5. Lagrangian or Semi-Lagrangian Formulation

The cause of nonlinear instability is the nonlinear advection term in momentum equations. If we can get rid of this
term, we can eliminate the instability!

This can be achieved by solving the advection problem in a Lagrangian or Semi-Lagrangian framework.

With Lagrangian methods, the pure advection problem is

du
=0
dt

i.e., u is conserved along the trajectory, which is also the characteristic curve (dx/dt = c) in this case.

In the purely Lagrangian method, the grid points move with the flow, and the grid can become severely deformed.

Semi-Lagrangian method is based on a regular grid – it finds the solution at grid points by finding the values of u at
the departure points – the location where the parcels come from. Spatial interpolation is usually needed to find the
value at the departure point. We will cover this topic in more details later.

4-19
4.3.6. Use of conservation to control nonlinear instability

Recall that aliasing acts to feed energy into small-scale components. It is possible to control (not prevent) aliasing
by forcing the total energy or other physical properties (e.g., enstrophy – squared voroticity) to be conserved – just
as the continuous system does.

If such constraints are satisfied, the energy spectrum cannot grow without bound!

Consider 2-D advection in a non-divergent flow:

∂A ∂A ∂A
+u +v =0 (5a)
∂t ∂x ∂y
∂u ∂v
+ =0 (5b)
∂x ∂y

We can write the advection-form equation (5a) in a flux-divergence form:

∂A ∂ (uA) ∂ (vA)
+ + =0 (6)
∂t ∂x ∂y

What is conserved for this system of equations?

We first the domain integration of the first moment of A to be:

4-20
 ∂⎡ ∂ (uA) ∂ (vA)
∫∫ Adxdy ⎤ = − ∫∫ dxdy − ∫∫ dxdy
∂t ⎣ ⎦ ∂x ∂y (7)
− ∫ [(uA) L − (uA) R ]dy − ∫ [(vA)T − (vA) B ]dx = 0

for a periodic domain. For non-periodic domain, we can see that the change in the domain integration of A equals
to the net flux through the lateral boundaries – there is no interior source or sink in A.

We say the domain integral of the first moment of A is conserved by this system of equations.

Let's now look at the conservation of the second moment of A, i.e., A2:

Multiply (6) by AÆ

∂A ∂ (uA) ∂ (vA)
A +A +A = 0Æ
∂t ∂x ∂y
∂ ⎛ A2 ⎞ ∂ (uAA) ∂ (vAA) ∂A ∂A
⎜ ⎟+ + − uA − vA =0 (8)
∂t ⎝ 2 ⎠ ∂x ∂y ∂x ∂y

Multiply (5b) by A Æ

∂ ⎛ A2 ⎞ ∂A ∂A
⎜ ⎟ + uA + vA =0 (9)
∂t ⎝ 2 ⎠ ∂x ∂y

(8) + (9) Æ

4-21
 ∂A2 ∂ (uA2 ) ∂ (vA2 )
+ + =0 (10)
∂t ∂x ∂y

we have a conservation equation for A2 in the flux divergence form too!

For a periodic domain, we have

∂⎡
⎣ ∫∫ A2 dxdy ⎤⎦ = 0
∂t

therefore the second-moment of A is also conserved by the continuous system.

What about the discrete equations? Do they also conserve these quantities? Not all discrete forms do. We will show
one conservative example in the following.

Conservation for the Discrete System

Consider the case of staggered Arakawa C-grid:

4-22
 v

u A u Δy

v
Δx
and the following second-order FD formulation:

δ 2t A + δ x (uA x ) + δ y (vA y ) = 0 (11a)

δ xu + δ y v = 0 (11b)

Q: Does this system conserve A and A2?

Note that A x is defined at u point and A y at v point, we denote A x = B and A y = C Æ

∑δ
ij
x (uA x ) = (u1B1 − u0 B0 ) / Δx + (u2 B2 − u1B1 ) / Δx + ....

+ (u N −1 BN −1 − u N −2 BN −2 ) / Δx + (u N BN − u N −1BN −1 ) / Δx = 0

with periodic B.C.

4-23
The same is true to the flux in y direction – therefore A is conserved.

Conservation of A2 is a little more complicated to show. We will make use of two identities (you can check them
out for yourself):

x
δ x ( P xQ) = Pδ xQ + (δ x P )Q (12a)

P xδ x P = δ x ( P 2 / 2) (12b)

Multiply (11a) by AÆ

Aδ 2t A = − Aδ x (uA x ) − Aδ y (vA y ) (13)

Look at only the 1st term on RHS of (13):

- Aδ x (uA x )

Let P = A, Q = uA x , (12a) becomes

x
δ x ( A xuA x ) = Aδ x (uA x ) + (δ x A)(uA x ) Æ

x
Aδ x (uA x ) = δ x ( A xuA x ) − uδ x ( A2 / 2) (14)

4-24
The 2nd term on RHS of (14) is not in the flux form. Using (12a) again, let P = A2/2, Q = u
x x
Æ δ x ( A2 / 2 u ) = ( A2 / 2)δ xu + δ x ( A2 / 2)u

therefore
x x
uδ x ( A2 / 2) = δ x ( A2 / 2 u ) − ( A2 / 2)δ xu (15)

Now (14) becomes

x
Aδ x (uA x ) = δ x ( A xuA x ) − δ x ( A2 / 2 u ) + ( A2 / 2)δ xu (16)

- only the last term is not in the flux form.

For the y direction, we can also get

y
Aδ y (vA y ) = δ y ( A y vA y ) − δ y ( A2 / 2 v) + ( A2 / 2)δ y v (17)

(16) + (17) Æ

Aδ x (uA x ) + Aδ y (vA y ) = .... + ( A2 / 2)(δ xu + δ y v)

the last term is zero because of (11b)!

Therefore

4-25
 ∑ Aδ
ij
2t A=0Æ ∑(A A
ij
n n +1
− An−1 An ) = 0 Æ

∑(A A
ij
n n +1
) = ∑ ( A An )
ij
n −1

An An+1 is not exactly A2 due to the temporal discretization – we say A2 is quasi-conserved!

Comments on Conservations:

• Conservation is generally a good thing to have in a model – can be used to check the
correctness of code – if you know your scheme conserves, check if the domain integral changes
in time.
• Don't want to use schemes that are known to conserve poorly.
• It is not always possible to conserve all conservative quantities of the continuous system,
however.

Nonlinear advection schemes that conserve more quantities

Arakawa derived and compared several methods for dealing with the nonlinear advection of a barotropic vorticity
equation

∂ζ ∂ζ ∂ζ
+u +v =0 (18)
∂t ∂x ∂x

∂ψ ∂ψ
with ζ = ∇ 2ψ ; u = − ; v=
∂y ∂x

where ζ is vorticity and ψ is the streamfunction.

4-26
(18) can be rewritten as

∂ζ ∂ψ ∂ζ ∂ψ ∂ζ
− + =0 (19)
∂t ∂y ∂x ∂x ∂y

where the advection can be written as a Jacobian:

∂ψ ∂ζ ∂ψ ∂ζ
J (ψ , ζ ) = − (20)
∂x ∂y ∂y ∂x

Arakawa came up with seven different forms of discretization for the Jacobian (called Arakawa Jacobians), some
conserve total energy and enstrophy (ζ2) (the PDE conserves both).

The following figures show total kinetic energy, total enstrophy and the kinetic energy spectrum, as function of
time, using different Jacobians. Formulation No. 7 (J7) is the only one that conserves both total energy and total
enstrophy.

4-27
4-28
4-29
Read Mesinger and Arakawa (1976) GARP Report, section 7 of Chapter 3.

4-30
Chapter 4. Nonlinear Hyperbolic Problems

4.4 System of Hyperbolic Equations – Shallow Water Equation model

Ref: Chapter IV of Mesinger and Arakawa (1976) GARP Report

4.4.1. Introduction

It is assumed that you are familiar with the shallow water equations and associated theories. If not, consult Holton
or Haltiner and Williams book.

The following is a set of linear 1D shallow water equations:

∂u ' ∂u ' ∂φ '

+u + =0 (21a)
∂t ∂x ∂x
∂φ ' ∂φ ' ∂u '
+u +Φ =0 (21b)
∂t ∂x ∂x

u = constant base state flow

Φ = gH = g × mean depth of the water = constant
u Æ u' = perturbation velocity
φ = gh' = perturbation geopotential height

Issues to consider with respect to numerical solution

4-1
1) More than 1 variable
2) Equations coupled
3) Can support multiple physical modes
4) There are more possibilities of grid layout (see figure below)

4.4.2. The differential solution

Performing standard analysis by assuming

ψ = Ψ exp[i (kx − ωt )] (22)

gives ω = k (u ± Φ ) (23)

which is called the dispersion relation.

4-2
From (23) Æ

ω
c= =u ± Φ .
k

In the phase speed, there are slow mode represented by u (advection) and fast mode given by Φ (surface gravity
waves). Since c is constant, the waves are non-dispersive.

Group velocity

∂ω
cg = =u ± Φ
∂k

represents the speed of wave energy propagation.

What about the characteristics (we have seen this before – see example problem given at the end of Chapter 1).
Make use of the auxiliary equations, we have the following equations in matrix form:

⎡1 u 0 1 ⎤ ⎛ ut ⎞ ⎛ 0 ⎞
⎜ ⎟
⎢0 Φ 1 u ⎥ ⎜ u x ⎟ ⎜⎜ 0 ⎟⎟
⎢ ⎥ = (24)
⎢ dt dx 0 0 ⎥ ⎜ φt ⎟ ⎜ du ⎟
⎢ ⎥⎜ ⎟ ⎜ ⎟
⎣0 0 dt dx ⎦ ⎜⎝ φx ⎟⎠ ⎝ dφ ⎠

Setting the determinant of the coefficient matrix to zero gives

4-3
 2
⎛ dx ⎞ dx
⎜ ⎟ − 2u + (u 2 − Φ ) = 0 Æ
⎝ dt ⎠ dt

dx
=u ± Φ
dt

which are the characteristics equations.

The compatibility equations can be found to be

φ dx
u± = constant along =u ± Φ . (25)
Φ dt

Eq. (25) can be rewritten as

⎡∂ ∂ ⎤⎛ φ ⎞
⎢⎣ ∂ t +(u + Φ )
∂ x ⎥⎦ ⎜⎝
u+ ⎟=0
Φ⎠
(26a)

⎡∂ ∂ ⎤⎛ φ ⎞
( )
⎢⎣ ∂ t + u − Φ ∂ x ⎥⎦ ⎜ u −
⎝ Φ⎠
⎟=0 (26b)

which are two decoupled equations describing wave disturbances 'advected' by the respective propagation speeds.
u ± φ / Φ are known as the Riemann invariants, as said before.

Equations (26) can also be obtained using matrix method (see Chapter 1).

4-4
4.4.3. Discretization for the Shallow Water Equations

4.4.3.1. Forward-backward scheme

We know that FTCS is unstable for pure advection equations, and this is also true to the shallow water equations.

But, we can obtain a stable scheme if we use backward scheme for the second equation. Let's look at the simper
case of u =0, i.e., there is not mean flow:

δ + t u + δ 2 xφ n = 0
(27)
δ + tφ + Φδ 2 xu n +1
=0

Since forward scheme is used for the first equation and backward scheme used for the second, the overall scheme
is called forward-backward scheme. We can show that it is conditionally stable.

Stability Analysis

Assume that

u nj = Aλ n exp(ikx j )
(28)
φ jn = Bλ n exp(ikx j )

Note here A and B could be complex so as to allow possible phase difference between u and φ.

Plug (28) into (27) Æ

4-5
 Δt ik Δx − ik Δx
(λ n+1 − λ n ) A + Bλ n (e − e )=0
2Δx
(29)
Δt ik Δx − ik Δx
(λ n+1 − λ n ) B + ΦAλ n+1 (e − e )=0
2Δx

or
Δt
(λ − 1) A + iB sin(k Δx) = 0
Δx
(30)
Δt
(λ − 1) B + iΦA λ sin(k Δx) = 0
Δx

or
⎛ Δt ⎞
⎜ λ −1 i sin(k Δx) ⎟ ⎛ A ⎞ ⎛ 0 ⎞
Δx ⎜ ⎟ ⎜ ⎟
⎜ ⎟⎜ ⎟ = ⎜ ⎟ (30')
⎜ iΦ Δt λ sin(k Δx) λ −1 ⎟⎜ B ⎟ ⎜ 0⎟
⎜ ⎟⎝ ⎠ ⎝ ⎠
⎝ Δx ⎠

(30') is a simultaneous linear system of equations for A and B. It has non-trivial solutions if and only if the
determinant of the coefficient matrix equals to zero. Æ

λ 2 − λ[2 − Φa 2 ] + 1 = 0 [where a = Δt / Δx sin(k Δx) ]

Φa 2 − 2 ± (2 − Φa 2 ) 2 − 4
λ± = (31)
2
If the radical is negative, then | λ± |≡ 1 . I.e., if

4-6
 (2 − Φa 2 ) 2 ≤ 4

| 2 − Φa 2 |≤ 2

−2 ≤ 2 − Φa 2 ≤ 2

2 − Φa 2 ≤ 2 is always satisfied, in addition,

Φa 2 ≤ 4 Æ

2Δx
Δt ≤ , for it to be valid for all k, we require
Φ sin(k Δx)

2 Δx Δt Φ
Δt ≤ or μ = ≤2 (32)
Φ Δx

which is the stability condition! Here Φ is the disturbance propagation speed in the absence of base-state
advective flow.

When the mean flow is non-zero, and when the advection terms are treated using the upstream-in-space, forward-
in-time scheme, the condition should be

Δx
Δt ≤ .
|u |+ Φ /2

4-7
Note the factor of 2 in the condition – the use of forward-backward scheme actually allows a Courant number of 2
relatively to the gravity wave to be used! This is due to the fact the backward scheme is actually kind of 'implicit'
scheme.
4.4.3.2. Centered-in-time (leapfrog) Center-in-Space (CTCS) scheme

δ 2t u + u δ 2 xu + δ 2 xφ = 0
δ 2tφ + u δ 2 xφ + Φδ 2 xu = 0

(here we assume a non-staggered grid)

Similar stability analysis leads to:

Δx
Δt ≤ (33)
|u |+ Φ

which is twice as restrictive as that for forward-backward scheme. Also it contains a computational mode.

Grid Splitting

The above CTCS scheme used non-staggered grid.

When using non-staggered grid for the above equations, we can run into the grid-splitting problem. We discussed
this issue in the past.

One way of avoiding grid splitting is to use staggered grid – in which different variables are located at different
points of a grid mesh.

4-8
Let's stagger u and φ (h in the figure) in the following way:

Our FD equation using CTCS scheme is then

δ 2t u + u δ 2 xu + δ xφ = 0 at u point
(34)
δ 2tφ + u δ 2 xφ + Φδ xu = 0 at φ point

Note the key difference in the third term of each equation from the previous non-staggered CTCS scheme. Also the
equations are solved at different grid point.

Stability analysis will show that the stability condition is

Δx
Δt ≤
| u | +2 Φ

which, for zero mean flow case, is twice as restrictive as the non-staggered version (because of the factor of 2 in
front of phase speed Φ ).

4-9
However, since the pressure gradient force and velocity divergence terms are differenced over one Δx interval
instead of over 2Δx, and these are terms responsible for the gravity wave propagation, the solution should be more
accurate, since the effective grid spacing is half as large.

4.4.3.3. Treatment of insignificant fast modes

(Reading: Durran Chapter 7 – Physically insignificant fast waves)

We obtained earlier the phase speed of shallow water waves:

c = u ± gH

it contains two modes. The slower advective mode and the faster gravity wave (GW) mode:

u ~ 10m / s

gH ~ 10 × 10000 ~ 200m / s for external gravity waves

| u | gH for many problems.

Gravity waves are not important in global coarse-resolution (effective grid spacing > 100 km) NWP models in
which the resolutions are usually too coarse to resolve them adequately anyway.

GWs are often important for mesoscale (~100 km in scale) atmospheric flows. For mesoscale NWP models, often,
compressible equations are used which support fast sound waves – so sound wave play a similar role as the gravity
waves in large scale model in limiting the time step size (when using explicit schemes).

4-10
When the fast mode is not important, we don’t want it to be the one that limits the time step size.

There are in general two ways to deal with this problem – one is to treat the terms responsible for the fast modes
implicitly, and the other uses different time step sizes for fast and slow modes and the method is called mode
splitting method. ARPS, MM5 and WRF models use the latter to deal with fast sound waves (hence the large and
small time steps, dtbig and dtsml you find in arps.input).

4.4.4.4. Semi-implicit method

Since the PGF term in u equation and the velocity divergence term in φ equation are responsible for gravity waves,
we can treat them implicitly, so that hopefully the gravity wave mode no longer limit the time step size.

Again we look at the non-staggered case:

2t
δ 2t u + u δ 2 xu + δ 2 xφ = 0
2t
(35)
δ 2tφ + u δ 2 xφ + Φδ 2 xu = 0

The time averages makes the scheme implicit. Since only some of the terms are treated implicitly, the scheme is
called semi-implicit.

Stability of the system – only the advective velocity u appears in the stability condition therefore much larger time
step can be used (see Durran 7.2.3; Mesinger and Arakawa Chapter 4 section 6).

Analysis shows that the fast mode in the numerical solution is actually slowed down – i.e., there is a lagging phase
error with this mode – it is okay if this mode is consider unimportant, like the sound waves in the atmosphere or
the gravity waves in large-scale models.

4-11
Solution procedure for (35)

1) Computer φn+1 for all j by eliminating un+1 from the 2nd equation using the first:

ΦΔt 2 n+1
φ n +1
− ⎡φ − 2φ jn+1 + φ jn++21 ⎤⎦ = f ,
2 ⎣ j −2
4 Δx
j

with the right hand side being known.

2) Two effectively decoupled tridiagonal systems of equations (for 1D problems) have to be solved, one for even j
and one for odd j (can lead to grid splitting).

3). Once φ n+1 is known, we can plug it into u equation to obtain un+1.

4) If a staggered grid is used, then only one tridiagonal system of equations has to be solved. The total amount of
calculation is about the same as the non-staggered case because twice any many grid points are now involved.

5) For 2D or 3D problems, the semi-implicit scheme results in a Helmholtz equation that can't be as easily solved
as the 1D tridiagonal equation.

Tapp and White is one of the first to use semi-implicit method in a compressible mesoscale model of the UK
Met Office (Tapp and White 1976 QJRMS).

Derivation of f in the above equation:

4-12
 2t
δ 2t u + u δ 2 xu + δ 2 xφ = 0
2t
u n+1 = u n−1 − 2Δtu δ 2 xu − 2Δtδ 2 xφ
2t
δ 2 xu n+1 = δ 2 xu n−1 − 2Δtu δ 2 x (δ 2 xu ) − 2Δtδ 2 x (δ 2 xφ )

2t
δ 2tφ + u δ 2 xφ + Φδ 2 xu = 0
1
δ 2tφ + u δ 2 xφ + Φ ⎡⎣(δ 2 xu ) n+1 + (δ 2 xu ) n−1 ⎤⎦ = 0
2

φ n+1 = φ n−1 − 2Δtu δ 2 xφ − ΔtΦ ⎡⎣(δ 2 xu ) n+1 + (δ 2 xu ) n−1 ⎤⎦

= φ n−1 − 2Δtu δ 2 xφ − ΔtΦ ⎡⎢δ 2 xu n−1 − 2Δtu δ 2 x (δ 2 xu ) − 2Δtδ 2 x (δ 2 xφ ) + (δ 2 xu ) n−1 ⎤⎥

2t

⎣ ⎦

φ n+1 − (Δt )2 Φδ 2 x (δ 2 xφ )n+1

= φ n−1 − 2Δtu δ 2 xφ − ΔtΦ ⎡⎣ 2δ 2 xu n−1 − 2Δtu δ 2 x (δ 2 xu ) − Δtδ 2 x (δ 2 xφ ) n−1 ⎤⎦

φ n+1 − (Δt )2 Φδ 2 x (δ 2 xφ )n+1

= φ n−1 − 2Δt ⎡⎣u δ 2 xφ + Φδ 2 xu n−1 ⎤⎦ + (Δt ) 2 ⎡⎣ 2u δ 2 x (δ 2 xu ) + δ 2 x (δ 2 xφ ) n−1 ⎤⎦

f = φ n−1 − 2Δt ⎡⎣u δ 2 xφ + Φδ 2 xu n−1 ⎤⎦ + (Δt ) 2 ⎡⎣ 2u δ 2 x (δ 2 xu ) + δ 2 x (δ 2 xφ ) n−1 ⎤⎦ in the above.

φ j + 2 − 2φ j + φ j −2
Note that δ 2 x (δ 2 xφ ) ≡ and terms without superscript are evaluated at time level n.
(2Δx) 2

4-13
4.4.4.5. Mode-splitting Method

For info on mode-splitting method for compressible model, see Klemp and Wilhemson (1978) and Durran Section
7.3.2.

Skamarock and Klemp (1982) discuss that stability issues associated with the mode-splitting methods as applied to
compressible system of equations.

References:

Klemp, J. B., and R. B. Wilhelmson, 1978: The simulation of three-dimensional convective storm dynamics. J.
Atmos. Sci., 35, 1070-1096.

Skamarock, W. C., and J. B. Klemp, 1992: The stability of time-split numerical methods for the hydrostatic and
nonhydrostatic elastic equations. Mon. Wea. Rev., 120, 2109-2127.

4.4.4. The Arakawa Grids

(p.47 in Mesinger and Arakawa 1976)

Arakawa (Arakawa and Lamb 1977) introduced a variety of staggered grids when trying to find the most accurate
method for handling geostrophic adjustment process, which we know relies on inertia gravity waves. Inertia
gravity waves are dispersive, they disperse ageostrophic energy.

To describe inertia GW, we need to include rotational effect into the shallow water equations:

4-14
 ∂u ∂h
+ g − fv = 0
∂t ∂x
∂v ∂h
+ g + fu = 0
∂t ∂y
∂h ⎛ ∂u ∂v ⎞
+H⎜ + ⎟=0
∂t ⎝ ∂x ∂y ⎠

∂
For 1-D version of this problem, i.e. for = 0 case, the dispersion equation for the exact solution is
∂y

ω = ( f 2 + k 2 gH )1/ 2 .

Arakawa defined 5 different grids, all of which has the same number of dependent variables per unit area – so that
the computational time is about the same.

4-15
4-16
For each of the above grid, the finite difference equation can be written as

4-17
We want to find the numerical dispersion relations and compare them with the exact solution. For 1-D problem,
the dispersion relations are (note ν here is our ω, d = Δx, the time derivative terms are not differenced, i.e., remain
in their continuous form):

They are plotted in the following figure:

4-18
The phase speed and group velocities for each of these grids can be plotted together with the exact solution:

4-19
We can see that for the 1-D problem, B and C grid perform the best.

A and D are not good at all. Energy of waves shorter than 4Δx propagates in the wrong direction.

E is reasonable good.

4-20
For 2-D problem, the ω /f is plotted in the following:

Exact Solution B grid C grid

We can see C grid is closest to the exact solution given in (A), and B grid is not as good in 2-D, especially along
the diagonal direction in the plot.

4-21
4.5. Boundary Conditions for Hyperbolic Equations

(ref. Chapter 8, Durran)

4.5.1. Introduction

In numerical models, we have to deal with two types of boundary conditions:

a) Physical

ƒ e.g., ground (terrain), coast lines, the surface of a car when modeling flow around a moving car.

ƒ internal boundaries / discontinuities

b) Artificial / Numerical

ƒ must impose them to limited integration domain, but they should act as if they don't exist
ƒ the boundary should be transparent to "disturbances" moving across the boundary
ƒ there can be different kinds of forcing at the boundaries, e.g., lifting by mountain slope and heating at the
surface
ƒ these boundaries should be well-posed mathematically
ƒ often we have to over-specify the boundary condition, e.g., when a grid is one-way nested inside the coarser
grid
ƒ it has been shown that no well-posed lateral b.c. exists for the shallow water equations or for the Navier-
Stokes equations
ƒ still a lot of debate in this area. B.C. are often critical because they can exert enormous control over the
interior solution

4-1
As you might suspect, B.C. for hyperbolic problems are closely related to the theory of characteristics –
information propagates along characteristic paths.

Consider the well-posed problem in a 1-D domain x ≥ 0 (only one b.c. at x=0, but to solve the problem numerically,
we have to place a computational boundary somewhere at x>0).

∂u ∂u
+c =0 (36)
∂t ∂x

I.C.: u(x,0) = f (x)

B.C.: u(0,t) = g(x)

And f(0) = g(0) for consistency between the I.C. and B.C.

From our earlier discussion, we know that

du = 0 along dx/dt = c, i.e., x = ct + β

where β is a constant to be determined by I.C.

Look at an x-t diagram:

4-2
Consider the characteristic passing through (x1 , t1):

u(x1 , t1) = u(x2, 0) = f( x2 ) = f( x1 – ct1 )

For any (x, t) such that x – ct ≥ 0, that solution can be related to the I.C. f(x), i.e.,

u(x, t) = f(x- ct) for x ≥ ct.

Consider now point (x3, t3). In this case, using the MOC, we see that

u(x3, t3) = u(0, t*) = g( t3 – x3/c) Æ

solution has dependency on the B.C. g(t) and not the I.C. Thus in general,

4-3
 u(x, t) = g(t - x/c) if x < ct.

Now, if we have to impose a boundary condition at x = L, the problem becomes ill-posed because we've over-
specified the solution at x = L, i.e., no condition is required there!

It is unlikely that the solution given by f(x1-ct1), for example, will match whatever condition we impose at x=L. The
problem is that, in the general case, the B.C. depends on the solution, which isn't known at x = L a priori! What
happens if we have a whole spectrum of waves that propagate at difference speeds? We can't supply a B.C. for
each one!

4.5.2. Number of Boundary Conditions

For the previous 1-D advection problem, we need only one B.C. Now let's look at the 1-D shallow water equations
in the absence of mean flow:

∂u ' ∂φ '
+ =0 (37a)
∂t ∂x
∂φ ' ∂u '
+Φ =0 (37b)
∂t ∂x

0≤ x ≤ L.

Recall the characteristic form of the system for Φ = constant

∂A ∂A
+ Φ =0 (38a)
∂t ∂x

4-4
 ∂B ∂B
− Φ =0 (38b)
∂t ∂x

where A = u + φ / Φ and B = u − φ / Φ .

Clearly, there are 2 pure advection equations in the Riemann invariants A and B, with wave speeds of ± Φ . We
have separated the waves, or eigenvalues, and in general, the number of boundary conditions equals to the number
of eigenvalues. This doesn't tell what the B.C. should be, however. Just how many.

In practice, the number of B.C. also depends on the particular grid structure used.

In our case,

λ1 = + Φ Æ right moving wave Æ must specify L.H. boundary condition

λ2 = − Φ Æ left moving wave Æ must specify R.H. boundary condition

Note that we specify the B.C. from where the wave originates, but not to where it's going!

4.5.3. Sample B.C. and Wave Reflection

For limited area models that contain artificial lateral boundaries, we desire to let incident waves pass through
without reflection, i.e., as if the boundary wasn’t there at all. This is the behavior for the exact or differential
solution.

4-5
 t = t1 t = t2

Consider 1-D linear advection:

∂u ∂u
+c =0 c >0 and constant, −∞ ≤ x ≤ ∞ , t ≥ 0 . (39)
∂t ∂x

To look at reflection at the boundary, we need consider only the spatial derivative.

∂u
+ cδ 2 xu = 0 −∞ ≤ x ≤ L (40)
∂t

This equation describes a right-moving wave. If there is reflection at x=L, the reflected wave must be
computational in origin, because the physical equation doesn't support left-moving wave.

Our center-in-space discretization connot be applied at x=L (since it needs u at L+1), so something else must be
done. Note that no B.C. should be speficied at x=L, except due to the fact that computer has limited memory and
computing power so you can't make the computational domain infinitely large.

4-6
Approximations to the PDE at x=L:

1. uL = 0 Fixed or rigid boundary

2. uL = uL-1 Zero gradient (about uL-∆x/2)
3. uL = 2uL-1 - uL-2 Linear extrapolation

One can also use special forms of the PDE, e.g., upstream at the boundary:

∂u u Ln − u Ln−1
= −c upstream
∂t L ∆x
∂u 3u Ln − 4u Ln−1 + 4u Ln−2
= −c second-order upstream
∂t L 2∆x

Question: What happens to a wave when these B.C. are applied to the semi-discrete (semi because time remains in
derivative form therefore there is no time discretization error – we focus on error caused by spatial discretization)
equation (40)?

Assume solutions of the form

u = A exp[i (kx j − ωt )] , Plug it into Eq.(40), we get

i ( kx j −ωt ) i ( kx j −ωt ) eik ∆x − e − ik ∆x

−iω Ae + cAe =0 Æ
2 ∆x

c sin(k ∆x) c sin( pd )

ωd = = ( pd = k ∆x)
∆x ∆x

4-7
Now, for the PDE we have

k ∆x pc
ωe = kc = c= e
∆x ∆x

where pe = exact value of k∆x. We now want to compare these two frequencies.

Note that, in the F.D. solution, two values of k∆x correspond to a single ωd, whereas in the exact solution, ωe is
linear in k∆x.

4-8
We therefore say that the F.D. solution is non-monochromatic, i.e., there is more than one wavelength per
frequency.

We can gain insight into the behavior of the solution by creating a monochromatic form, like we did in the leapfrog
scheme when we wrote β1 = f(β2) =β. In that spirit, we write:

u nj = [ Aeipj + Bei (π − p ) j ]e − iωt 0≤ p ≤π /2

where the 2nd term takes care of 'reflection'.

The second term is the computational mode in space. Note that the slope of the curve for p> π/2 is opposite to that
for p≤ π/2. Slope = ∂ω / ∂k = group velocity – we will return to this shortly.

ωd ωd c sin( pd )
Phase speed: cd = = = >0
k pd / ∆x pd

which is the same for both modes ( 0 ≤ p ≤ π / 2 ).

Note that, for small p, cd ∼ c .

We now start to see that phase speed isn't a good indicator of wave reflection, because it does not represent the
propagation of wave energy. In the above case, the phase speed is always positive – so it has no way of indicating
reflection.

∂ωd
Group velocity: cg = = c cos( pd )
∂ ( pd / ∆x)

4-9
 cg > 0 for pd = p (0 ≤ p ≤ π / 2)

cg< 0 for pd= π - p.

If cg < 0, this means that energy is propagating in a direction opposite to the cg of the exact solution (which is
positive), such negatively propagating waves may be a result of reflection at the boundary (it can also be because
of other reasons).

We can interpret reflection in terms of group velocity - the reflected waves (or more accurately wave packets)
transport energy in the opposite direction upon reflection.

It is possible to determine the amplitude, r, of the reflected wave. See Matsuno, JMSJ, 44, 145-157 (1966).

Let A = amplitude of incident wave = 1.0

B(=r) = amplitude of reflected (computational mode) wave

(notation is from the monochromatic solution)

u j = e − iωt ⎡⎣eipj + rei (π − p ) j ⎤⎦

Example: Zero gradient lateral boundary condition

uL = uL-1

without loss of generality, we can let L=0 Æ

u0 = u-1

4-10
With j=0 and j= -1 in our solution, we have

1 + r = e − ip + re − i (π − p )

where 'd' on p has been dropped, and the e − iωt 'cancels' via linear independence for a single frequency ω.

r (1 + eip ) = e− ip − 1 Æ
e −ip − 1 e − ip / 2 (e− ip / 2 − eip / 2 ) −i sin( p / 2)
r= = ip / 2 −ip / 2 ip / 2 = e −ip
1+ e ip
e (e +e ) cos( p / 2)

= -[cos(p ) − i sin( p )] i tan( p / 2) Æ

| r | = tan (p/2) where p = k ∆x = pd ( 0 ≤ p ≤ π / 2 )

Note: The reflected amplitude depends upon both k of the incident wave and ∆x.

4∆x wave ( p = π/4 ) Æ | r | = 1.0 Æ complete reflection without change in amplitude.

32 ∆x wave (p=π/16) Æ | r | = 0.0982 Æ relatively weak reflection

Æ | r | ~ 1/ (incident wavelength).

Short waves are reflected much more than long waves – it is understandable – infinitely long waves, i.e., constant
field, should pass through a zero gradient boundary freely.

4-11
4.5.4. Refelction for the Shallow Water Eqautions – Radiation Boundary
Condition

Let's consider a semi-discrete shallow water equations system:

∂u
+ u δ 2 xu + Φδ 2 xφˆ = 0
∂t
∂φˆ
+ u δ 2 xφˆ + Φδ 2 xu = 0
∂t

Here, we let φˆ = φ / Φ so that the two equations are completely symmetric. This is to make our discussion easier.

The system describes waves moving in the + or – directions depending upon the relationship of u and Φ1/2. If u <
0 and | u | > Φ1/2, then the wave will go to the left.

Often we want to allow such waves to pass through an artificial lateral boundary while minimizing the reflection.

Because we have 2 waves in this system, there will be 2 physical modes and 2 computational modes – one for each
wave. Using the same analysis as before, we have

Physical Modes:

u+ Φ
(u ,φˆ)1 = (u ,φ )1 ei ( k1x −ω1t ) with ω1 = sin(k1∆x)
∆x
u− Φ
(u ,φˆ) 2 = (u ,φ ) 2 ei ( k2 x −ω2t ) with ω2 = sin(k2 ∆x)
∆x

4-12
where u andφ are the amplitude of u and φˆ , respectively.

Computational Modes (the "image" about p=π/2)

u+ Φ π
(u ,φ )3 = (u,φ )3 ei ( k3 x −ω3t ) with ω3 = sin(k3∆x) and k3 = − k1
∆x ∆x
u− Φ π
(u ,φ ) 4 = (u ,φ ) 4 ei ( k4 x −ω4t ) with ω4 = sin(k4 ∆x) and k4 = − k2
∆x ∆x

Note: ω3 = ω1 and ω4 = ω2, i.e., 2 horizontal wavenumbers corresponding to the same value of frequency.

Let's now look at the directions of phase speed and group velocity of these modes.

Phase Speed

Physical modes: cp > 0 for both modes if u > Φ1/2

cp > 0 for one and < 0 for the other if 0<u <Φ1/2

Computational Modes: Same as above

Group Velocity

cg1 = - cg3 = ( u + Φ1/2) cos( k1 ∆x)

cg2 = - cg4 = ( u - Φ1/2) cos( k2 ∆x)

where 0 ≤ k1,2∆x ≤ π/2 for a monochromatic solution. Thus there are always 2 group velocities > 0 and two < 0.

4-13
Example: Consider what happens at an outflow boundary when Φ1/2 > u > 0. Here, we want to specify a boundary
condition at x = L.

Now, we know that

u = u1 + u2 + u3 + u4
φˆ = φˆ 1 + φˆ 2 + φˆ 3 + φˆ 4

From the analytical solution, we know that

ω = k ( u +Φ1/2 ) Æ
cp = cg,

i.e., the energy and phase always propagate in the same direction.

Going back to our 4 solutions, we find that for Φ1/2 > u > 0 (this case only),

Mode 1: Phase > 0, group > 0 Æ physical mode

Mode 2: Phase < 0 group < 0 Æ physical mode
Mode 3: Phase > 0 group < 0 Æ computational mode
Mode 4: Phase < 0, group > 0 Æ computational mode

Now, mode 1 is the physical incident wave, and if it gets reflected, its energy will come back in modes 2 and 3
because their group velocities are negative. The reflection won’t show up as mode 4.

As a result, we can analyze the reflection in terms of 1, 2 and 3 only:

4-14
 u = (eik1x + reik3 x )e − iω1t + R eik2 x e − iω2t
φˆ = (eik1x + reik3 x )e −iω1t − R eik2 x e− iω2t (41)

reflected reflected
incident
computational physical
physical
mode mode
mode
(amplitude=r)

Why the minus sign on the last term in the φ equation?

Recall that the Riemann invariant of the right-moving wave is u1,3 + φ1,3/Φ1/2 = u1,3 + φˆ 1,3 Æ
u + φˆ = 2(eik1x + reik3 x )e − iω1t ,
1,3 1,3

which is the physical mode 1 with its computational counterpart 3. The minus sign is needed to make sure that the
above 2 equations when added together form a Riemann invariant that does not involve left-ward propagation
waves - those that are supported by the other characteristic equation, i.e, ∂ (u − φˆ) / ∂t + (u − Φ )∂ (u − φˆ) / ∂x = 0 .

Now, let's examine the reflection properties for various boundary conditions applied to this particular semi-discrete
form of the shallow water equations.

First, it's important to recognize that the solutions at the boundary must be continuous Æ frequency of the incident
and reflected waves must be identical at the boundary. So, if ω1 = ω2 (ω1 = ω3 already), we have,

u + Φ1/2
k2 ≈ k1 .
u − Φ1/2

4-15
[Here we assumed that k∆x is small (good for high resolution) so thatsin( k ∆x) ≈ k ∆x ].

Therefore, k2 > k1, or the reflected wave is always smaller in scale than the incident wave. (Note that at a critical
layer where Φ1/2 = u , the analysis is not valid and a different approach has to be taken.)

Now, let's apply the above analysis to different types of boundary conditions.

Based on earlier analysis, we rewrite (41) as

π
i( − k1 ) x
u = [e ik1 x
+ re ∆x
+ R eik2 x ]e − iωt (ω1 = ω2 = ω3 = ω )
= [eik1x + r (−1) j eik1x + R eik2 x ]e − iωt ( x = j ∆x )

Similarly we have

φ = [eik x + r (−1) j eik x − R eik x ]e−iωt

1 1 2

Case I: uL = 0, i.e., we have the rigid wall.

∂φ
uL = 0 Æ =0 (if u=0, i.e., can't have u ≠ 0if u=0 at the boundary)
∂x L

Let L =0 without loss of generality:

4-16
 -1 0

therefore u0 = 0, φ0 = φ-1 .

For the first equation, we have

1+r+R=0

For the second, we have

1 + r − R = [e − ik1∆x − re − ik1∆x − R e− ik2∆x ]

Assuming that e x ≈ 1 + x , we have

1 + r − R = 1 − ik1∆x − r + rik1∆x − R + Rik2∆x

r = i∆x(−k1 + rk1 + Rk2 ) / 2

Equating the real and imaginary parts, we have

r=0
R = -1

4-17
Æ k1 = - k2.

We showed earlier that

u + Φ1/2
k2 = k1
u − Φ1/2

which agrees with current result when u =0.

In summary, we found total reflection in the physical mode, which is the expected condition for a rigid lateral
boundary. This is also undesirable if the true problem does not actually have a boundary here. When there is indeed
a wall, we often use the so-called mirror boundary condition (as in ARPS for B.C. option one), which finds it basis
in our analysis.

Case II: Wave radiating lateral boundary condition

At the boundary L, we use

∂u u − u L −1
+ (u + c* ) L =0
∂t ∆x

to replace the momentum equation. For φ, use the governing equation itself with one-sided difference:

∂φ φ −φ u − u L−1
+ u L L −1 + Φ L = 0.
∂t ∆x ∆x

4-18
In the above, c* is an estimate of the dominant wave speed at the lateral boundary (can be set constant, or computed
as was done by Orlanski 1976).

Doing a reflection analysis, we find (show for yourselves) that

(u − Φ1/2 )(Φ1/2 − c* )
R= (for reflected physical mode)
(u + Φ1/2 )(Φ1/2 + c* )
r=0 (for computational mode)

[See Klemp and Lilly, 1978, JAS, p78]

If we do a good job of estimating c*, then we can make R~0.

Again, the reflection is in the physical mode, and that is not good. Keep in mind that this analysis assumes that
k∆x 1.

In complicated models, one typically replaces the normal momentum equation at the lateral boundary with

∂u ∂u
+ (u + c* ) = 0 (42)
∂t ∂x

It neglects pressure gradient force responsible for the wave propagation, the effect can be thought as being
accounted for in c*).

In (42), the sign of u + c* determines inflow or outflow boundary. Other equations are then solved using their
original governing equations, with one-sided difference when necessary. This type of condition is called
Sommerfeld radiation condition. Orlanski (1976) applies equation like (42) to all variables, but in practice, it does
not work very well – it leads to over-specification of conditions and enhancement of the computational mode.

4-19
ARPS has options for four variations for radiation lateral boundary conditions – all follow Sommerfeld condition,
the difference being the way c* is determined and whether (42) is applied in large or small time steps.

Klemp and Lilly (1978) gives more details on the analysis of radiation lateral boundary conditions.

It should be noted that, in terms of characteristics, our shallow water system is

∂A ∂A
+ (u + Φ1/ 2 ) =0
∂t ∂x
∂B ∂B
+ (u − Φ1/ 2 ) =0
∂t ∂x

with A = u + φ/Φ1/2 and B = u - φ/Φ1/2 , the Riemann invariants.

∂u ∂u
The radiation condition + (u + c* ) = 0 is equivalent to setting to zero the amplitude of B. So, the radiation
∂t ∂x
B.C. can be regarded as a condition written in terms of the characteristics, and that's why a PGF does not occur. In
fact, if B=0, at the lateral boundary, then

A + B = A + 0 = 2u

∂ (2u ) ∂ (2u )
∴ + (u + Φ1/ 2 ) =0
∂t ∂x

which is the Radiation Condition we used.

4-20
References:

Orlanski, I., 1976: A simple boundary condition for unbounded hyperbolic flows. J. Comput. Phys., 21, 251-269.

Klemp, J. B., and D. K. Lilly, 1978: Numerical simulation of hydrostatic mountain waves. J. Atmos. Sci., 35, 78-
107.

4.5.4. Other Boundary Conditions

Rayleigh Damping (Sponge)

Here we include a regional (zone) next to the lateral boundary in which zero-order (Rayleigh) damping is applied
to the prognostic variables, to 'absorb' incident wave energy and damp possibly reflected energy.

δ 2t u = −r ( x)(u − u ) n−1 = −(u − u )n−1 / τ ( x)

4-21
where r is the Rayleigh damping coefficient and τ the corresponding e-folding time of damping. The smaller r is,
the longer it takes to damp.

Rayleigh damping is usually needed when the lateral boundary conditions are over-specified, such as the case of
externally forced boundary (e.g., when a grid is forced by the solution of another model, the coarse grid solution of
the same model or by analysis – the case of one-way nesting). ARPS uses Rayleigh damping with externally forced
boundary option.

Viscous sponge

It takes the form of second-order diffusion

δ 2t u = K δ xxu n−1

In this case, short waves are selectively damped. It does not damp long reflected waves effectively, however. It is
often used in combination with the Rayleigh damping, as in the ARPS.

Top boundary condition

In atmospheric models, the top boundary of the computational domain often has to be placed at a finite height –
creating an artificial top boundary. Vertically propagating, e.g., internal gravity, waves can reflect off the boundary
and interact with flow below – creating problems. One example of radiation top boundary condition is that of
Klemp and Durran (1983). See also Durran section (8.3.2).

Reference:
Klemp, J. B., and D. R. Durran, 1983: An upper boundary condition permitting internal gravity wave radiation in
numerical mesoscale models. Mon. Wea. Rev., 111, 430-444.

4-22
4.5.5. Additional References on Boundary Conditions (most significant ones with *. Most recent
references not included)

Abarbanel, S. S., and E. M. Murman, 1982: Stability of two-dimensional hyperbolic initial boundary value problems for explicit and
implicit schemes. Journal of Computational Physics, 48, 160-167.
Austin, D. Ian, 1982: Longshore boundary conditions for numerical wave models. International Journal of Numerical Methods in
Fluids, 2, 263-276.
Bennett, A. F., 1976: Open boundary conditions for dispersive waves. Journal of the Atmospheric Sciences, 33, 176-182.
Bennett, A. F., 1981: The ill-posedness of open ocean models. Journal of Physical Oceanography, 11, 1027-1029.
*Bougeault, Philippe, 1983: A non-reflective upper boundary condition for limited-height hydrostatic models. Monthly Weather
Review, 111, 420-429.
Carpenter, K. M., 1982: Note on the paper "Radiation conditions for the lateral boundaries of limited-area numerical models" by M. J.
Miller and A. J. Thorpe (Q. J., 107, 615-628). Quarterly Journal of the Royal Meteorological Society, 108, 717-719.
*Davies, H. C., 1976: A lateral boundary formulation for multi-level prediction models. Quarterly Journal of the Royal Meteorological
Society, 102, 405-418.
Elvius, T., and A. Sundstrom, 1973: Computationally efficient schemes and boundary conditions for a fine-mesh barotropic model
based on the shallow-water equations. Tellus, 25, 132-156.
Engquist, B., and A. Majda, 1977: Absorbing boundary conditions for the numerical simulation of waves. Mathematics of
Computation, 31, 629-651.
Engquist, B., and A. Majda, 1981: Numerical radiation boundary conditions for unsteady transonic flow. Journal of Computational
Physics, 40, 91-103.
Goldberg, M., 1977: On a boundary extrapolation theorem by Kreiss. Mathematics of Computation, 31, 469-477.
Goldberg, M., and E. Tadmor, 1981: Scheme-independent stability criteria for difference'approximations of hyperbolic
initial-boundary value problems. II. Mathematics of Computation, 36, 603-626.
Gustafsson, Bertil, 1982: The choice of numerical boundary conditions for hyperbolic systems. Journal of Computational Physics, 48,
270-283.
Hack, J. J., and W. H. Schubert, 1981: Lateral boundary conditions for tropical cyclone models. Monthly Weather Review, 109,
1404-1420.
Halpern, Laurence, 1982: Absorbing boundary conditions for the discretization schemes of the one-dimensional wave equation.
Mathematics of Computation, 38, 415-429.
Han, T. Y., J. C. S. Meng and G. E. Innis, 1983: An open boundary condition for imcompressible stratified flows. Journal of
Computational Physics, 49, 276-297.

4-23
Israeli, M., and S. A. Orszag, 1981: Approximation of radiation boundary conditions. Journal of Computational Physics, 41, 115-135.
*Klemp, J. B., and D. K. Lilly, 1978: Numerical simulation of hydrostatic mountain waves. Journal of the Atmospheric Sciences, 35,
78-107.
Klemp, J. B., and R. B. Wilhelmson, 1978: The simulation of three-dimensional convective storm dynamics. Journal of the
Atmospheric Sciences, 35, 1070-1096.
*Klemp, J. B., and D. R. Durran, 1983: An upper boundary condition permitting'internal gravity wave radiation in numerical
mesoscale models. Monthly Weather Review, 111, 430-444.
Kurihara, Y., and M. A. Bender, 1983: A numerical scheme to treat the open lateral boundary of a limited area model. Monthly
Weather Review, 111, 445-454.
Leslie, L. M., and R. K. Smith, 1977: On the choice of radial boundary conditions for numerical models of sub-synoptic vortex flows
in the atmosphere, with application to dust devils. Quarterly Journal of the Royal Meteorological Society, 103, 499-510.
*Lilly, D. K., 1981: Wave-permeable lateral boundary conditions for convective cloud and storm simulations. Journal of the
Atmospheric Sciences, 38, 1318-1316.
Madsen, N. K., and R. F. Sincovec, 1983: A general numerical boundary condition method for hyperbolic systems. (Submitted to the
Journal of Computational Physics.)
*Matsuno, T., 1966: False reflection of waves at the boundary due to the use of finite differences. Journal of the Meteorological
Society of Japan, 44, 145-157.
Miller, M. J., and A. J. Thorpe, 1981: Radiation conditions for the lateral boundaries of limited-area numerical models. Quarterly
Journal of the Royal Meteorological Society, 107, 615-628.
Nitta, T., 1962: The outflow boundary condition in numerical time integration of advective equations. Journal of the Meteorological
Society of Japan, 40, 13-24.
Nitta, T., 1964: On the reflective computational wave caused by the outflow boundary condition. Journal of the Meteorological
Society of Japan, 42, 274-276.
Oliger, J., 1974: Fourth order difference methods for the initial boundary-value problem for hyperbolic equations. Mathematics of
Computation, 28, 15-25.
Oliger, J., 1976: Hybrid difference methods for the initial boundary-value problem for hyperbolic equations. Mathematics of
Computation, 30, 724738.
*Oliger, J., and A. Sundstrom, 1976: Theoretical and practical aspects of some initial-boundary value problems in fluid dynamics.
Stanford University, STAN-CS-76-578.
*Orlanski, 1., 1976: A simple boundary condition for unbounded hyperbolic flows. Journal of Computational Physics, 21, 251-269.
Osher, S., and S. Chakravarthy, 1983: Upwind schemes and boundary conditions with applications to Euler equations in general
geometries. Journal of Computational Physics, 50, 447-481.

4-24
Pearson, R. A., 1974: Consistent boundary conditions for numerical models of systems that admit dispersive waves. Journal of the
Atmospheric Sciences, 31, 1481--1489.
Pearson, R. A., and J. L. McGregor, 1976: An open boundary condition for models of thermals. Journal of the Atmospheric Sciences,
33, 447-455.
*Perkey, D. J., and C. W. Kreitzberg, 1976: A time-dependent lateral boundary scheme for limited-area primitive equation models.
Monthly Weather Review, 104, 744-755.
Raymond, W. H., and H. L. Kuo, 1984: A radiation boundary condition for multi-dimensional flows. Quarterly Journal of the Royal
Meteorological Society, 110, 535-551.
Rudy, D. H., and J. C. Strikwerda, 1980: A nonreflecting outflow boundary condition for subsonic Navier-Stokes calculations. Journal
of Computational Physics, 36, 55-70.
Sloan, D. M., 1983: Boundary conditions for a forth order hyperbolic difference scheme. Mathematics of Computation, 41, 1-11.
Sundstrom, A., 1977: Boundary conditions for limited-area integration of the viscous forecast equations. Beitrage zur Physik det
Atmosphare, 50, 218-224.
Sundstrom, A., and T. Elvius, 1979: Computational problems related to limited-area modeling. Numerical Methods in Atmospheric
Models, Volume II, GARP Publications Series, No. 17.
Tadmor, Eitan, 1983: The unconditional instability of inflow-dependent boundary conditions in difference approximations to
hyperbolic systems. Mathematics of Computation, 41, 309-319.

4-25
Chapter 5. Methods for Solving Elliptic Equations

References: Tannehill et al Section 4.3.

Fulton et al (1986 MWR).

Recommended reading:
Chapter 7, Numerical Methods for Engineering Application. J. H. Perziger, 2nd Ed.

5.1. Introduction

In fluid dynamics, elliptic problems often arise. Two examples are given below.

Example 1:

When solving the incompressible equations, we have

G
∂V G G ⎛ p⎞ G G
+ V ⋅ ∇V = −∇ ⎜ ⎟ + F + K ∇ 2V (1a)
∂t ⎝ ρ0 ⎠
G
where F is a body force and K is a constant viscosity.
G
∇ ⋅V = 0 (1b)

dθ
= 0. (1c)
dt

5-1
The pressure has to be obtained from a diagnostic elliptic equation, obtained by taking ∇ ⋅ ( ) of the momentum
equation (1):
G G G G
( ) ( ) (
∇ 2 p = −∇ ⋅ ρ0V ⋅ ∇V + ∇ ⋅ ρ0 F + ∇ ⋅ ρ0 K ∇ 2V ) (2)

Equation (2) is a Possion equation, which is a special case of elliptic equations. It is a boundary condition problem
(no time integration therefore no I.C. needed).

Example 2:

When we solve 2-D equations in vorticity-streamfunction form, we define

∂ψ ∂ψ
u≡ , v≡−
∂y ∂x

∂v ∂u
where ψ is the streamfunction, and vorticity ζ = − = ∇ 2ψ . From the equations in example 1, we can obtain
∂x ∂y
G
prognostic vorticity equation (assuming ∇ ⋅ V = 0 ):

dζ
= K ∇ 2ζ (3)
dt

and diagnostic elliptic equation

∇ 2ψ = ζ . (4)

5-2
Equation (2) is integrated forward for one time step first then (3) is solved to obtain ψ which is then used to
calculated u and v. The new u and v are then used to advect the vorticity field to obtain ζ at the next time level.

Other examples of elliptic equations include the omega equation for diagnosing vertical motion in quasi-
geostrophic synoptic dynamics, and the equation for diagnosing potential function from divergence. Due to the
non-time matching nature of elliptic equations, we often refer to them as diagnostic equations.

5.2. Methods for Solving Elliptic Equations

5.2.1. Introduction

Many methods exist, some of them are available as 'canned' programs in standard libraries, such as IMSL.
Commonly used methods can be divided into the following classes:

a. Iterative (relaxation) methods

b. ADI method
c. Multi-grid methods
d. Direct Methods
- FFT
- Gaussian elimination or block cyclic reduction
e. Preconditioned conjugate residual method (e.g, Skamarock et al 1997, MWR)
f. many more

We will only look at the first three.

5-3
Consider a simple form of an elliptic, the Possion equation:

∂ 2u ∂ 2u
+ = ρ ( x, y ) 0 ≤ x ≤ a , 0 ≤ y ≤ b . (5)
∂x 2 ∂y 2

Boundary conditions are crucial in elliptic problems (elliptic problems are boundary conditions problems in
nature). As discussed previously, the B.C. can be written in a general form:

⎛ ∂u ⎞
α uB + β ⎜ ⎟ =γ (6)
∂
⎝ ⎠B
n

where α, β are constants, α2 + β2 ≠ 0. Depending on the value of α and β, we have Neumann, Dirichlet and Robin
boundary conditions.

One of the most common forms of discretization for elliptic problems is the so-called 5-point formula which is
based on second-order centered difference:

δ xxu + δ yyu = ρ . (7)

Let the B.C be u Bn = f i ,nj .

Rearranging terms, our F.D. equation can be written as

ui , j = cx (ui +1, j + ui −1, j ) + c y (ui , j +1 + ui , j −1 ) − C ρi , j (8)

5-4
 Δy 2 Δx 2 ΔxΔy
cx = , c = , C = .
2(Δx 2 + Δy 2 ) 2(Δx 2 + Δy 2 ) 2(Δx 2 + Δy 2 )
y

We can also write this system in a matrix form as

G G
[ A] U = B (9)

which can be solved using one of the standard methods for linear system of equations. However, because [A] is a
sparse matrix, it's generally more efficient to use methods designed for solving such sparse matrix problems.

5.3.2. Iterative Methods

Strategy: Make an initial guess for ui,j at the zeroth iteration, and then apply a correction to the guess based on a
local error. Once the difference between successive iterations falls below a set tolerance, the iteration is considered
converged and the solution found.

Method I: Jacobi Iteration

Let uiv, j = vth = solution after v iterations. If we write

residual = Rν = ∇ 2u v − ρ

or Riv, j = cx (u v i +1, j + u v i −1, j ) + c y (u v i , j +1 + u v i , j −1 ) − C ρi , j − u v i , j (10)

5-5
then the solution is converged if Riv, j = 0 . In practice, we can't make R exactly zero using iteration methods, but we
want || Riv, j || < ε. However, we can always define a new estimate of the solution uiv,+j1 such that the residual is
temporarily reduced to zero at (i, j):

cx (u v i +1, j + u v i −1, j ) + c y (u v i , j +1 + u v i , j −1 ) − C ρi , j − u v +1i , j = 0 . (11)

To achieve that, we need, from (11) – (10):

uiv,+j1 = uiv, j + Riv, j (12)

where Riv, j is computed from (10).

(12) is the formula to obtain solution of u through iterations. The iterations are needed because even though the
residual R has been reduced to zero by (12) at point (i, j), when u at the neighboring points are adjusted in a similar
manner, the residual at (i, j) is messed up (because the equations at each point are not independent)! If one
recompute R using new values of u at the neighboring points, it is no longer zero. But, we do this iteratively, i.e.,
perform the adjustment repeated, hopefully we can eventually reduce the residual at all points to below certain
threshold!

This method can be shown to converge all the time, but unfortunately at a slow rate.

Method II: Guass-Seidel or Sequential Relaxation

This method is the same as the Jacobi method, except for the definition of the residual which is now given by

5-6
 Riv, j = cx (u v i +1, j + u v +1i −1, j ) + c y (u v i , j +1 + u v +1i , j −1 ) − C ρi , j − u v i , j . (13)

The circled terms are different from the previous formula.

For a loop of the form:

DO j =2, ny-1
DO I =2, nx-1

this essentially means that the newest values of u are used in the calculation of R.

Implications:

• Ri,j is calculated from the newest values of u, old values of u do not need to be stored – storage requirement
is reduced.

• Because Ri,j depends on the latest values of u, the algorithm is sequential and therefore cannot be vectorized.

Method III: Successive or Sequential Over-relaxation (SOR)

SOR is essentially the Gauss-Seidel method augmented by a relaxation parameter α:

uiv, j = uiv,−j1 + α Riv, j . (14)

α > 1 is usually used to improve convergence.

5-7
How do we determine the optimal value of α?

It depends on the particular elliptic PDE being solved and its coefficients. Unfortunately, the convergence rate is
very sensitive to the value of α.

Without going into the heavy theory behind, we simply note here that maximum rate of convergence is obtained
from the smaller root of

α opt
2 2
tm − 16α opt + 16 = 0 (15)

where α opt = optimal over-relaxation parameter

tm = cos(π/M ) + cos(π/N ) (16)

where M and N are the number of grid intervals in the two directions. This formula says that the α is mostly
dependent on the grid size.

Solving (15) Æ

8 − 4 4 − tm2
α opt = . (17)
tm2

If M and N are large, then

5-8
 π ⎛ 1
1/ 2
1 1 ⎞
α opt ≈ − ⎜ 2+ 2⎟ . (18)
2 2 2⎝M N ⎠

For a square domain (M=N)

M=N α(1) α(2)

10 1.53 1.80
100 1.94 1.98
1000 1.994 1.998

5.3.3. Alternate Direction Implicit (ADI) Method

We encountered this method when discussing methods for solving multi-dimensional heat transfer equations. With
this method, we solve the elliptic equation one direction at a time, by doing the following:

cx uiv++1,1/j 2 − uiv,+j1/ 2 + cx uiv−+1,1/j 2 = C ρi , j − c y (u v i , j +1 + u v i , j −1 ) (19a)

c y uiv,+j1+1 − uiv,+j1 + c y uiv,+j1−1 = C ρi , j − c y (uiv−+1,1/j 2 + uiv++1,1/j 2 ) (19b)

Each of them is a tridiagonal system of equations. Repeat the steps until convergence.

5-9
Discussion – The memory and operation counts for a 2D problem on N x N grid (see Fulton et al 1986).

Methods Memory Operation Comments

Gaussian elimination O(N4) O(N4) error accumulation problem

Jacobi and G-S O(N2) O(N4) Flexible, easy to code

SOR O(N2) O(N3 )
ADI O(N2) O(N2lnN)

Fast/Direct Solvers O(N2) Limited to simple problems

Cyclic reduction with constant coefficients
matrix decomposition
or combination of two
after using FFT

Multi-grid method O(N2) Suitable for more general problems

5-10
5.3.4. Multigrid Methods for Elliptic Equations

Multi-grid represents a particular strategy for elliptic problems, within which a variety of methods can be used.
They were developed in the 1970's by Achi Brandt. Because of its flexibility and operational counts on the same
order as the much more restrictive direct methods, it has become rather popular.

The methods work by approximating a problem on multiple overlapping grids with widely varying mesh sizes and
cycling between these approximations, using relaxation to reduce the error on the scale of each grid. The efficiency
is optimal – the operational count is proportionally to the number of unknowns.

Reading: Fulton et al. (1986, MWR).

Analysis of Error Reduction with Relaxation Methods

Let's analyze the error of G-S relaxation:

Assume Δx=Δy, cx = cy= 1/4, C=1/4, (13) becomes

1 v 1
(u i +1, j + u v +1i −1, j + u v i , j +1 + u v +1i , j −1 ) − u v i , j − ρi , j = Riv, j (20)
4 4

uiv,+j1 = u v i , j + Riv, j (21)

(20) - (21) gives

1 v 1
(u i +1, j + u v +1i −1, j + u v i , j +1 + u v +1i , j −1 ) − u v+1i , j − ρi , j = 0 (20a)
4 4

5-11
Assume v is the true (discrete) solution, the error e = v – u and

1 1
(vi +1, j + vi −1, j + vi , j +1 + vi , j −1 ) − vi , j − ρi , j = 0 (22)
4 4

(22) – (20a) Æ

1
,j =
einew (ei +1, j + e newi −1, j + ei , j +1 + e newi , j −1 ) (23)
4

We see that the error at (i, j) is a simple average of the current errors at four surrounding points, which suggests
that small scale errors can be reduced by the averaging faster due to cancellation. Smooth error field is slow to
reduce!

Let's quantify the result by using Fourier analysis. Let

ei , j = A exp[i ( jθ1 + kθ 2 )] (24)

θ1 and θ 2 are the discrete wave number in the two directions, and are integer multiple of 2πΔx (assuming Δx=Δy)
between -π and π.

Plug (24) into (23) Æ

1
Anew = (e − iθ1 Anew + eiθ1 A + e − iθ2 Anew + eiθ 2 A) (25)
4

5-12
Reduction in error after one iteration (remember the amplification factor?):

Anew eiθ1 + eiθ2

μ (θ ) = = (26)
A 4 − e − iθ1 − e − iθ2

Here, you want μ to be small – so that error is reduced faster!

5-13
When θ1 = θ2 = π (2Δ waves), μ = 1/3

but μ = 1 – O( h2 ) when θ1 and θ2 ~ O( h ) where h = Δx = Δy.

We can see that convergence is slow for the lowest wave numbers, i.e., for the longest waves (in terms of the
number of grid intervals).

The long waves can be represented rather accurately on a coarse resolution grid, on which the error can be reduced
much faster – this is the idea behind the multi-grid method.

Before looking at the multi-grid method, let's first look at the two-grid method.

Two-grid method

Let's write the Possion equation as

Lv = f. (27)

Let u be an approximate solution to Lv=f Æ

Lu – f = r (28)

Let v be the true solution to the finite difference form of equation, then error

e = v – u.

5-14
Since
Lv = f,

we have

Le = f – Lu = r (29)

where r is the residual.

On a grid with spacing h,

Lh eh = rh. (30)

Goal: To find e so that the true solution v = e + u can be obtained. We use relaxation plus iterations to reduce error
e, so that eventually u approaches v.

After a number of iterations on grid h, eh becomes smooth so that reduction in e becomes slow. The approximation
of (30) on a coarser grid with spacing H (H>h) is

LH eH = IhH rH (31)

where IhH is an operator that transfers r from grid h to grid H (when H = nh, we transfer the values at overlapping
points directly, and call the process injection).

Assume (31) is solved accurately on grid H, i.e., we found an eH that satisfies (31), we interpolate it to h grid and
adjust u so that

unewh = uh + IHh eH (32)

5-15
where IHh is a transfer operator from H to h grid. This operator usually performs interpolation.

It can shown that with υ iterations on the fine grid and a true solution of e obatined on the coarse grid somehow,
the error for ALL wave numbers are reduced by a factor of (1/2)υ when G-S and H/h=2 are used!

When υ=3, the error is reduced by a factor of 10 for ALL wave numbers!

Note that additional error is introduced during interpolation and injection. But usually, the error is relatively small.

Multi-grid method

Previous two-grid method did not say how to obtain the solution on the coarse grid. Since it has the same form as
that on the fine grid, it can be solved the same way as the fine grid equations, by using a still coarser grid, e.g., 2H.
Continuing this process recursively until the coarsest grid is trivial to solve, we obtain a simple multigrid method.

Total number of grid points (storage) involved:

(1+1/4 +1/16+1/2N-1) N2 < 4/3 N2 ~ O(N2).

We have only shown a simplest procedure for solving an elliptic equation using multigrid methods. There are many
variations and the method is very flexible and powerful. More details found in Fulton et al. (1986). Read the paper!

5-16
Chapter 6. Semi-Lagrangian Methods

References:
Durran Chapter 6.
Review article by Staniforth and Cote (1991) MWR, 119, 2206-2223.

6.1. Introduction

Semi-Lagrangian (S-L for short) methods, also called quasi-Lagrangian, have been around since the early 1960's.
T.N. Krishnamurti is probably the first to use S-L method. The S-L scheme becomes really popular following
Andre Robert's work in the early 80's (Robert 1982) when he combined the use of S-L with semi-implicit method
(SLSI) which allows for large time step size because both schemes are unconditionally stable. The SL scheme
deals with the advection part, and the SI scheme deals with faster wave propagation.

Two principal reasons for using SL methods in favor of more traditional Eulerian approaches:

Major: To overcome the CFL constraint which is based on stability, not accuracy considerations. That is, from on
accuracy point of view, we could live with a much larger Δt than which is required by the linear stability
condition because time truncation error is often smaller than spatial truncation error.

Minor: Because SL methodsG solve the equations in Lagrangian form, i.e., du/dt=0 rather than the advection
form, i.e., ∂u / ∂t + V ⋅ ∇u = 0 , there is no instability due to aliasing.

In the case of pure advection, stability depends upon the transport velocity. Since advection term does not appear in
SL methods, there is no time step constraint for SL schemes. (However, Δt must be small enough to keep the total

6-1
truncation error reasonably small. Just as with implicit schemes that are unconditionally stable, we don't want to
use too large a Δt due to temporal truncation error – remember that home work problem for heat transfer?).

Of course other modes can be present in the problem. For example, internal gravity waves, which can impose more
severe limitation on Δt unless they are treated implicitly. Hence the SL-SI methods.

Before looking at SL methods, recall the possible coordinate frameworks discussed earlier in the semester:

1. Eulerian – here, we use a fixed set of points. In the case of spatially staggered grids. Unless you use grid
refinement, you always work with a fixed number of points.

2. Lagrangian – the other extreme. In this case, the mesh actually follows the fluid and can become distorted,
necessitating a remaping operation from time to time.

3. Semi-Lagrangian – Here we seek to combine the two methods by moving data relative to a fixed (in space)
grid. It is really nothing more than the method of characteristics:

6-2
We "sit" at point i, and realizing that uin+1 = u*n , or the value of u traced back along the characteristic path. This is
because we have the compatibility equation

du = 0 along dx/dt = c for a pure advection problem ∂u / ∂t + c ∂u / ∂x = 0 .

Recall that by using quadratic interpolation to obtain value u* we obtained Lax-Wendroff scheme that is stable
when

c Δt ≤ Δx.

In other words, the departure point could be no further than Δx away from point i, i.e., the scheme is stable
provided than u* was determined by interpolation not extrapolation. This means that the departure point must be
surrounded by points used to determine its value.

6.2. Stability of S-L schemes

Why S-L schemes are unconditionally stable?

Consider a simple 1-D transport problem in our previous space-time diagram. If u is known at all previous times,
how do we find u(t+Δt)? From our discussions earlier in the semester, we know that

u(x, t+Δt) = u(x-ct, t).

Consider now that arrival point xI = I Δx at t +Δt = (n+1)Δt. Then, the departure point is

x* = i Δx – c Δt.

6-3
Thus,

u(iΔx, (n+1)Δt ) = u( x*, nΔt)

where, again, x* needs not be a grid point.

Consider a linear Lagrange interpolation polynomial for obtaining u* - this is simple yet non-trivial and is sufficient
for our analysis. For any 2 grid points, J and K, with J > K, we can write without loss of generality

δx
xK xJ

K J

x* − xK n xJ − x* n
u ( x* , nΔt ) = uJ + uK .
x J − xK x J − xK

Now if we apply a von Neumann stability analysis to this equation, using x*= i Δx – c Δt and u(x*, nΔt) = u(iΔx,
(n+1)Δt ), we obtain (verify for yourselves):

⎡ ⎧ 2( I − σ − J ) ⎫ ⎤ 2 ⎡ k Δx
2
⎤
| λ | = 1 + ⎢ −1 + ⎨−1 +
2
⎬ ⎥ sin ⎢ ( J − K )⎥
⎢⎣ ⎩ K − J ⎭ ⎥⎦ ⎣ 2 ⎦

where σ = cΔt / Δx is the Courant number. We want |λ|2 ≤ 1 for stability, from which it follows that

6-4
 2( I − σ − J )
0≤ ≤2
K−J

or equivalently, K ≤ (i - σ) ≤ J.

Multiplying through by Δx gives

xK ≤ x* ≤ x J

for stability. Thus, stability is guaranteed for any location of the departure point in the mesh provided that
interpolation is used to obtain u* instead of extrapolation!

Note, with, for example, upstream forward scheme, we have decided to use ui-1 and ui, therefore to meet the
'interpolation' requirement, cΔt has to be no greater than Δx – a consistent result.

Discussion:

• SL methods are no more accurate than their Eulerian counterparts.

• Since fewer time steps are taken, the time truncation error is actually smaller.
• The biggest drawback with SL methods is the difficulty in enforcing conservation.
• For constant flow, the time step size can be arbitrarily large so we can complete the integration of a pure
advection problem in one step. The only error comes from the spatial interpolation.

6.3. Implementation of S-L schemes

Let's now look at the actual ways of applying SL method.

6-5
Consider a 3 time-level scheme applied to the equation

dF
= 0.
dt

Let's write the scheme as

F[ xm(t+Δt), t+Δt ] – F[ x*(t -Δt), t - Δt ] = 0

where xm = mesh point,

x* = departure point.

Now, let α = distance a particle travels in a single time step. Clearly, it is a function of the local velocity and the Δt.
Let's look at an x-t diagram

xm – arrival point
n+1

n-1 x*

2α

6-6
Let's assume that the trajectory is a straight line.

The major problem in determining the trajectory is that

dx
= u ( x, t )
dt

is implicit - x depends on u and u depends on x.

From our diagram, we have

F[ xm, t+Δt] – F[xm - 2α, t-Δt ] = 0,

so, we need both α and F at the departure point (α determines the departure point).

Question: How do we find α?

α = f(u)
u = dx/dt
x = g(α) Æ implicit problem!

When u = constant everywhere (uniform flow), SL methods are essentially exact for obvious reasons.

To find α, we integrate the trajectory equation over time interval [t-Δt, t+Δt ] using the mid-point rule:

6-7
 x +Δx
∫x −Δx
f ( x)dx = 2Δxf ( x) + O(Δx3 )

x +Δx dx t +Δt
∫x −Δx dt
dt = ∫ u dt
t −Δt

x(t + Δt ) − x(t − Δt ) = 2Δtu + O(Δt 3 )

where u is some time-averaged value of u. But, time and space are related, so let

u ≡ u ( x − α , t ) - i.e., u at halfway along the trajectory.

∴ x(t + Δt ) − x(t − Δt ) = 2α = 2Δtu ( x − α , t )

Æ α = Δtu ( x − α , t ) .

To solve this equation, we make a first guess for α (say, the α calculated based on u at current grid point), and
iterate until convergence:

α v+1 = Δt u ( x − α v , t ) .

Now that we have α, i.e., we found that departure point, we use interpolation to find the value of F at the departure
point, which is the F at the mesh point at t+Δt.

Convergence: It turns out that iteration will converge if

6-8
 ∂u
Δt <1
∂x

(for 1-D problem) where || || is an L-2 norm. Therefore time step size is limited by the accurate calculation of
trajectory, not by stability.

Steps of Implementation:

1. Solve the α-equation iteratively to obtain the departure point

2. Use interpolation to obtain F at the departure point
3. Replace F at the arrival (mesh) point with that computed at the departure point.

SL methods can be applied to forced problems as well, e.g., dF/dt = G and to systems of equations (see review
article and Durran). But SL refers only to the advecton process.

Role of Interpolation

After we find the departure point, we must use interpolation to find the value of F at this point. Because we are
using interpolation, some damping or smoothing occurs and conservation is difficult to achieve. The accuracy of
the interpolation process determines the accuracy and order of the S-L scheme.

Interpolation is also expensive, especially in multiple dimensions. To overcome this, Ritchies (1986, MWR, 114,
135-146) devised a non-interpolating S-L method, which is a topic of next section.

In a model that advects many variables, such as hydrometeor types and chemical species, S-L method is attractive
since all variables (on non-staggered grid) share the same trajectories, and the interpolations can share common
interpolation weights.

6-9
Because the trajectory calculation is expensive, models based on S-L method usually use non-staggered grid, i.e.,
all variables are located at the same grid points.

6.4. Non-interpolating S-L Methods

The essence of this approach is that the trajectory vector is broken down into the sum of 2 vectors: one that extends
from the arrival point backwards to the grid point nearest the departure point, and the other that is approximated as
a residual. Schematically, we have:

n+1

Total
n
point to point
n-1
R = residual

The mesh-point to mesh-point vector is simple replacement since the values at grid points are always known. The
residual R is handled using an Eulerian approach, which requires that | uΔt/Δx | ≤ 1.

Functionally, the 2-part method works as follows.

6-10
Our governing equation is dF/dt = 0. Expanding (consider only 1-D case for now), we have

∂F ∂F
+u = 0.
∂t ∂x

pΔx ∂F
Now, add the quantity − to both sides, where p is an integer.
2Δt ∂x

∂F ⎛ pΔx ⎞ ∂F pΔx ∂F
+ ⎜u − ⎟ =− Æ
∂t ⎝ 2Δt ⎠ ∂x 2Δt ∂x

∂F pΔx ∂F ∂F pΔx
+ = u' where u' ≡ u − .
∂t 2Δt ∂x ∂x 2Δt

pΔx
The LHS now represents the change in F following the motion but at a speed , which means that, over two
2 Δt
time steps, the total displacement will be p grid intervals (can have p>1). The residual term on the RHS is simply
an Eularian advection term that can be treated using standard methods.

p is chosen to give a displacement to the grid point nearest the departure point x - 2α, i.e., p is the integer nearest
2uΔt/Δx

6-11
 Departure point xi − pΔx / 2 - where Eulerian term is evaluated

xi-p xi-3 xi-2 xi-1 xi arrival point

pΔx

2α = 2uΔx

xi – pΔx/2 is a grid point for p that is even

xi – pΔx/2 is a 1/2 grid point for p that is odd

So we have

∂F
F(x, t+Δt) – F(x - pΔx, t - Δt) = - u' (xi – pΔx/2, t).
∂x

Because x - pΔx is a grid point (p is an integer), no interpolation is needed to evaluate the LHS. The RHS will be
evaluated at a grid point (p-even) or at half-point (p-odd), where in the latter case we will use average of
neighboring points.

One can show that this method is stable if

⎧ u ' Δt ⎫
max ⎨2 ⎬ ≤1
⎩ Δx ⎭

6-12
where is the residual Courant number. In general, this is much cheaper than, and of equivalent accuracy to,
interpolating methods.

Note that due to the introduction of the residual term – we again have problem with nonlinear aliasing.

6-13
Chapter 7. Introduction to Spectral Methods

Reference: Durran Chapter 4. Haltiner and Williams Chapter 6.

Temperton (2000).

7.1. Introduction

• Grid points method – evaluates spatial derivatives using Taylor series expansion

• Galerkin method (a superset of spectral method) represents dependent variables as the sum of a set of
functions with pre-specified space structure (basis-function). We solve the equations (usually ODE's) for the
coefficients of these functions.

7.2. Galerkin procedure

For equation

L(u) = f(x), a≤x≤b (1)

where L is an operator involving partial derivatives of u.

Consider a set of linearly independent basis-functions φj(x), so that

N
u ( x) ≈ ∑U jφ j ( x) (2)
j =1

7-1
where Uj is the coefficient for basis function φj, and is usually independent of x (i.e., a function of t only for time
dependent problems). The spatial dependency of u is represented by the basis functions.

The error of (2) satisfying (1) is

⎛ N ⎞
e ≈ L ⎜ ∑U jφ j ( x) ⎟ − f ( x) . (3)
⎝ j =1 ⎠

Galerkin procedure requires that the error be orthogonal to each basis function, or in another word, the error is a
residual that cannot be expressed in terms of the given finite set of basis functions Æ
b

∫ e φ dx = 0
a
N i for i =1, …., N. (4)

(4) leads to

b
⎛ N ⎞ b

∫a φi L ⎜⎝ ∑
j =1
U jφ j ( x) ⎟ dx − ∫ φi f ( x)dx = 0 for i =1, …, N
⎠ a
(5)

(5) is a set of N algebraic equations for Uj. When the operator L includes time derivatives, we have a set of coupled
ODE's for Uj.

7-2
7.3. Choices of basis functions

• Spectral method uses orthogonal Fourier series (in term of sine-cosine or e-ikx ) as the basis function.

• Other local basis functions can be chosen, e.g., piecewise linear basis function

The latter gives rise to the finite element method.

Two main branches of Galerkin methods:

Galerkin

Spectral Finite Element

- pure spectral - fixed
- pseudo-spectral - moving
(using global basis functions) (wide choices of elements,
local basis functions)

7-3
7.4. Discrete or Finite Fourier Transform

Before we look at the actual use of spectral method to solve PDE's, we need to review / introduce the discrete
Fourier transform.

Assume we have a set of uniformly spaced grid points in 1-D:

xj = j Δx, j = 1, 2, …, N (6)

where NΔx = L.

We assume the functions we deal with are periodic with a period of L, which implies that x = 0 and x = L = N Δx
are equivalent points.

To write a Fourier series for a function f(x) whose values are given only at N grid points requires only N Fourier
coefficients. For generality, f(x) is allowed to be complex. The series is

N
f ( x j ) = f j = ∑ Fn e
ikn x j
(7)
n =1

where Fn are the coefficients of the Fourier components, or the spectral coefficients.

The values of kn should be properly chosen so that the Fourier components satisfy the periodic condition, and for
computational reasons, equally spaced. The following set of values meets these requirements:

2π n
kn = n=1,2,…, N (8)
N Δx

7-4
and the series (7) becomes

N 2π n N 2π nj
j Δx
f ( x j ) = f j = ∑ Fn e = ∑ Fn e
i i
N Δx N
for j =1, 2, …, N (9)
n =1 n =1

(9) is an expression of f in terms of a series of Fourier components.

The inverse of (9) is

2π nj
1 N −i
Fn = ∑ f j e N for n=1, 2, …, N (10)
N j =1

(10) gives us the spectral coefficients from the grid point values fj.

(10) defines the finite or discrete Fourier transform and (9) the corresponding inverse transform, which are the
discrete analogues of the standard Fourier transform and its inverse. The integral in the continuous transforms are
replaced by sums in the discrete expressions.

(10) can be proven by substituting it into (9) and recognizing the orthogonality among the basis functions.

7.5. Spectral Method as Applied to 1-D advection Equation

In this section, we use the discrete Fourier transforms to solve PDE's

We look at linear and nonlinear1-D advection equations:

7-5
 ∂u ∂u
+c = 0, (11)
∂t ∂x
∂u ∂u
and +u =0. (12)
∂t ∂x

Consider linear advection equation (11) first.

For a periodic domain L:

N
u ( x, t ) = ∑U n (t )einkx , (13)
n =1

where the basis functions are

φn ( x) = einkx , (14)

2π
for k = , L = N Δx , x = j Δx (kn in the previous section = n k here).
L

Substitute (13) into (11) Æ

N
⎡ dU n inkx ⎤
∑ ⎢⎣
n =1 dt
e + cinkU neinkx ⎥ = 0
⎦
(15)

Because of the linear independence or by requiring the error be orthogonal to each basis function [eq(5)], we have

7-6
 dU n inkx
e + cinkU n einkx = 0 for n=1, …, N (16)
dt

where is a set of N ODE's for the coefficients Un.

Finite difference is usually used for the time derivative. After Un is obtained, u(x,t) is obtained from (13). Note for
simple linear equation such as (11), (13) needs to be evaluated at the output times only when the physical u is
needed. The inverse transform for Un needs to be performed once at the initial time.

Notes:

Galerkin method leads naturally to energy conservation (see discussion in pages 186-187 of Haltiner and
Williams), and the time integration of ODE is also subject to stability condition.

Now let's consider nonlinear advection equation (12).

Equivalent of (15):

N
dU n inkx ⎛ N inkx ⎞⎛
N
⎞
∑
n =1 dt
e + ⎜ ∑U ne ⎟⎜ ∑ inkU n einkx ⎟ = 0
⎝ n=1 ⎠⎝ n=1 ⎠
(17)

The multiplication of two spectral series in the second term leads to N2 terms!! And it is expensive to evaluate
(note that for the linear equation, we do not need to evaluate this series every time step). We end up with equations
for N2 coefficients because wave interactions generate new waves.

• (17) represents creation of new waves via nonlinear interaction

7-7
 • Aliasing can be prevented by dropping waves numbers n>N (spectral filtering)

The need to evaluate spectral series due to the nonlinear advection makes the spectral method unattractive for
practical use, until the transform method was invented!

The other development that made spectral method practical is the Fast Fourier transform (FFT) algorithms.

Transform Method

In practice, the series multiplication in the nonlinear advection term is seldom directly evaluated. A transform
method was invented that makes the use of spectral method in numerical models practical.

∂u
For the advection term u , the spatial derivative is first evaluated at every grid point:
∂x

∂u N
v= = ∑ inkU n einkx (18)
∂x n=1

(uv) together is then expanded as a single variable in a spectral series:

N −1
(uv) = ∑Ve
n =− N
n
inkx
(Inverse Fourier transform) (19)

where
N −1
1
∑ (uv) e
− inkx j
Vn = j (Fourier transform) (20)
2N j =− N

7-8
We end up solving the following ODE's:

dU n
+ Vn = 0 . (21)
dt

With this method, there are 2N operations for the advection term instead of N2 for the true spectral method.

Note that it is (20) that is actually evaluated, not (19) because we need Vn in (21).

7.6. Advantages and Disadvantages of Spectral Method

Advantages

• Derivatives computed exactly

• Infinite convergence rate in space (in term of the order of accuracy)
• Can pick basis functions that are well-suited for the particular problem, e.g., spherical harmonics for flow on
a sphere (the negative side is that one does not have complete freedom to choose the basis function – often
subject to certain limitation such as the periodicity condition)
• Can obtain power spectra directly
• Can control aliasing, therefore NL instability easily
• Can apply spatial filters of very high order easily
• Often more accurate than FD method with the same number of degrees of freedom (grid points versus
spectral components)
• Conserves energy naturally (see Haltiner and Williams section 6.3)

Disadvantages

• More complicated to implement

7-9
 • Can't represent physical processes in spectral space
• Hard to parallelize on distributed memory computers
• Basis function global, not well suited for handling localized features and/or sharp gradients (remember the
Gibbs phenomenon). FEM and those based on local basis functions usually do better
• Most global models since 70s are spectral models.
• Spectral methods are expensive at high resolutions. The operation count for FFT is proportional to N ln(N)
instead of N as for grid point method – one of the reasons (the dominance of distributed-memory parallel
computers is another) why grid point method is coming back in fashion for global models.
• Few regional model uses spectral method because of the lack of natural periodic boundary condition (The
regional spectral model, Juang and Kanamitsu 1994, of NCEP solves for the perturbation fields from the
global model in which it is nested so that at the boundaries the perturbations are zero. Sine transform can
then be used).

Juang, H. M. H. and M. Kanamitsu, 1994: The NMC regional spectral model. Mon. Wea. Rev., 122, 3-26.

7-10
 Meteorology 5344, Fall 2005
Computational Fluid Dynamics
Dr. Ming Xue

Computer Problem #2: PDE and Method of Characteristics

Distributed Thursday September 15, 2005
Due Thursday September 19, 2005

1. Classify the following system of equations using matrix method as well as the auxiliary
equation method.
∂u ∂v
+8 = 0
∂t ∂x
∂v ∂u
+2 =0
∂t ∂x

2. Consider the situation in which a uniform inviscid supersonic flow with free-stream Mach
number 2 encounters a sine wave wrinkle in the floor of a wind tunnel, as shown in the sketch
below:

The this type of steady flow is governed by the linear equations:

∂u ∂v
− =0
∂x ∂y
∂v ∂u
− =0
∂x ∂y
where u and v are perturbation velocity components in the x- and y-directions, respectively. Let
us assume that, prior to encountering the sine-wave from the left, no perturbations are introduced
into the flow so that u=0 and v=0 along x=0, y>0. Further, let the velocity normal to the free
stream at the lower boundary be given by:
2πε 2π x
v( x, y = 0) = 2
cos( ) 0≤ x≤ L
L L
(Note that the above condition may be applied at y = 0 because the perturbation velocities are
assumed to be small.) Determine the solution for the perturbation velocities using the method of
characteristics, assuming that ε = 1, L = 10Δx, and Δx = Δy = 1.0. Write a computer code to
calculate the values of u and v at all grid points and plot the fields using contours.

1
Strategy: Derive the characteristic and compatibility equations and associated conservative
quantities (called Riemann invariants), and then make a sketch of the net of characteristic curves.
Knowing what is conserved along the characteristics, apply the boundary conditions to
determine the constants of integration and then determine u and v at each point in the mesh,
beginning at the left edge and proceeding in the x direction (be cautious at the point 0,0). Your
computer code should solve for the entire net of points, and you should make a 2-D contour plot
of the final solution. Pay attention to the coding style and code performance, and hand in your
computer code with your report.

2
 Meteorology 5344, Fall 2005
Computational Fluid Dynamics
Instructor: Dr. Ming Xue

Computer Problem #3 Accuracy of Finite Differences

Due Tuesday, October 11, 2005

1. Consider the analytic function f(x) = sin (πx/2). The goal of this exercise is to compare
the analytical values of df/dx at a point with those obtained using the following
numerical approximations:

f i +1 − f i −1
• Centered Difference:
2Δx

f i +1 − f i
• Forward Difference:
Δx

f i − fi −1
• Backward difference:
Δx

−1.5 fi + 2 f i +1 − 0.5 fi + 2
• 3-point Asymmetric:
Δx

f i − 2 − 8 f i −1 + 8 f i +1 − f i + 2
• 4-point Symmetric
12Δx

a) Evaluate df/dx at x = 0.5 using the above 5 formulae with a step size Δx = 0.1, and
compute the error relative to the analytical solution, i.e.,

⎛ df ⎞
⎜ ⎟
⎝ dx ⎠ discrete
Error = − 1.0 . (1)
⎛ df ⎞
⎜ ⎟
⎝ dx ⎠ analytical

Write a computer program for this.

b). For each of the above five schemes, derive an expression for the truncation error and
evaluate analytically the leading term at x = 0.5 then calculate the relative error
according to (1). Compare this number with the relative error found in part a) and
present your results in tabular form. Discuss the results.

c) Repeat part a) using Δx = 0.5, 0.25, 0.125, 0.1, 0.05, 0.025, 0.0125 and compute the
error for each Δx and plot the results on a graph using - log10| Error | as the vertical

-1-
 axis and - log10(Δx) as the horizontal axis. The slope of this line is the convergence
rate or actual order of accuracy of the scheme. How does this rate compare with that
predicted by the leading term of the truncation error? (Note: Double precision may be
required for the last two schemes. Consult the compiler man page to the compiler
option for double precision. On Sooner, it 'man xlf90'.).

2. Consider the function f(x) = tanh[ k (x-1) ] where x and k are positive numbers.

a) Plot this function for k = 1.0, 5.0 and 20.0 in a domain x ∈ [0.0, 2.0] using Δx = 0.01.
Note the structure of the function for different values of k.

b). Compute df/dx at x = 1.05 using all five schemes from problem 1 with Δx = 0.32,
0.16, 0.08, 0.04, 0.02, 0.01 for both k=5.0 and k=20.0. Compute the error of the
approximation as in part c) of problem 1 and plot it on a graph using - log10| Error | as
the vertical axis and - log10( Δx ) as the horizontal axis.

c) Discuss the nature of this error with changing Δx and k. What do the results suggest
about the influence of a particular function on the truncation error? Do higher order
schemes always give better results?

-2-
 Meteorology 5344, Fall 2005
Computational Fluid Dynamics
Instructor: Dr. Ming Xue

Homeword #4: Numerical Solution of Heat Transfer Equation

Distributed Tuesday Oct. 11, 2005

Due Thursday Nov. 3, 2005

The goal of this computer problem is to examine the stability and accuracy of explicit and
implicit schemes for solving a 1-D heat transfer equation.

1. The 1-D heat transfer equations is

∂T ∂ 2T
=σ (1)
∂t ∂ x2

where σ is a positive and constant difffisivity. The problem consists of a 0.30 m thick wall of
infinite lateral extent that is initially at a uniform temperature Tinitial = 100 K. The surface
temperatures at the two sides (Tside) are suddenly increased to and maintained at a temperature of
-6 2 -1
300 K. The wall is composed of a Nickel alloy that has a conductivity σ = 3.0 x 10 m s . The
analytic solution for the temperature distribution can be written

∞
⎡ − ( mπ / L )2 σ t 1 − (−1) m mπ x ⎤
T ( x, t ) = Tside + 2(Tinitial − Tside )∑ ⎢e
L ⎥⎦
sin( ) (2)
m =1 ⎣ mπ

where 0 < x < L. The size of m determines the accuracy of the solution.

a. Using the FTCS scheme, solve (1) out to time = 30 minutes and plot your solution for all x
points every 5 minutes using Δx = 0.015 m and Δt = 20 and 60 sec. Explain your results.

Consider the following two questions and make sure you know the answers. You are not
required to perform the actual computation, although you can do it as an exercise.

b. Using the exact solution given by (2), determine the convergence rate of the FTCS scheme
for this particular problem by running a series of experiments in which Δx is successively
refined over a wide range of values (the choices of which are up to you). Remember to vary
Δt such that , μ = σΔt /(Δx) 2 remains constant. Make plots of the RMS error versus
resolution, and the convergence rate versus resolution. Discuss your results in light of the
theoretical order of the FTCS scheme and other issues from lecture.

c. Choosing two "high resolution" solutions from part b, use Richardson extrapolation to
determine a fourth-order solution. Then, compare this solution with that obtained from your
code using μ = 1/6, i.e., a direct fourth-order solution. How do the two compare? To what

1
 degree is the Richardson extrapolation result a function of the grid size used for the two
"high resolution" solutions. You may wish to rerun a few coarsened-grid cases to answer
this last question.

2. For the same heat transfer equation given in problem 1:

a. Using the Crank-Nicholson scheme, solve (1) out to t = 30 minutes and plot your solution for
all x points every 5 minutes using Δx = 0.015 m and Δt = 20 sec. Employ the Thomas
algorithm (see also Appendix A of your textbook) to solve the tridiagonal matrix.

b. Compute additional solutions with successively larger timesteps (of your choice), and
compare the plotted results with the analytic solution given in problem 1. At what point does
the solution error become intolerably large, and is the scheme stable for all choices of Δt as
predicted by linear theory?

Additional question for you to consider. Actual computation not required but you need to be
sure you know the expected results.

c. Choose your favorite solution (i.e., a given Δt and Δx) from your FTCS experiments in
problem 1 and determine the CPU time and error relative to the analytic solution. Rerun
this same case with your C-N code and compare the CPU time and error. Given that a
larger timestep is possible in the C-N code, make two additional sets of runs:

1). For the Δx used in part c above, increase At in the C-N run until the solution error
deviates substantially (say 20%) from that of the FTCS experiment. How much
savings (in terms of CPU time) did you get by using an implicit scheme? In other
words, how much cheaper is the C-N scheme (at larger At) than the FTCS scheme (at
smaller Δt) for equivalent solution accuracy and Δx?

2) Assuming that you can use no more CPU time than from your first experiment in part
c, compute the best solution possible with the C-N scheme. That is, use the smallest
Δx possible, coupled with the largest At, to get the "best" solution within the CPU
time constraint given from your first experiment in part c.

2
 Meteorology 5344, Fall 2005
Computational Fluid Dynamics

Computer Problem #5: Linear Convective Transport

Distributed: Tuesday, October 11, 2005
Due: Thursday, November 3, 2005

Consider the 1-D linear convection equation

∂u ∂u
+c =0
∂t ∂x

where c is a positive and constant advection speed. This equation can be solved numerically using the two-
step MacCormack method:

n +1 * uin+1 − uin
Predictor: (u ) = u − cΔtn

Δx
i i

1⎡ (u n +1 )* − (uin−+11 )* ⎤
Corrector: uin +1 = ⎢uin + (uin +1 )* − cΔt i ⎥
2⎣ Δx ⎦

a. Derive the modified equation for this two-step scheme and determine the anticipated error type
(dispersive or dissipative). When trying to eliminate ( )t, make sure you use of FDE not PDE.

b. Use the von Neumann technique to assess the stability of this scheme, and plot the phase and amplitude
errors as a function of kΔx for several Courant numbers, including a few for which linear stability is
violated.

c. Write a computer code for this scheme, using as initial conditions the following function:

⎡ ⎛ 2π x ⎞ ⎤ ⎡ ⎛ 2π x ⎞ ⎤
u ( x, t = 0) = 2 + u0 ( x) ⎢1 + 0.3sin ⎜ ⎟ ⎥ ⎢1 + 0.4sin ⎜ ⎟⎥
⎣ ⎝ 9Δx ⎠ ⎦ ⎣ ⎝ 10Δx ⎠ ⎦
where
⎧−1 if 8 ≤ x ≤ 28
⎪
u0 ( x ) = ⎨ 1 if 28 < x ≤ 39
⎪0
⎩ otherwise

with Δx = 1.0 in a periodic domain of length 50. This somewhat unconventional initial condition
provides a stringent test of advective schemes because it contains sharp gradients and other spatial
irregularities.

The periodic boundary condition means that u ( x = 0) = u ( x = L) , u ( x = −l ) = u ( x = ( L − l )) and

u ( x = l ) = u ( x = ( L + l )) , where L is the length of the physical domain.

To facilitate the implementation of periodic conditions at the lateral boundaries, we usually define an
extra grid point outside each physical boundary, so for a physical domain of length L, we need nx = L/Δx
+ 3 number of grid points, as illustrated below:

1
 i=1 i=2 i=3 i=nx-2 i=nx-1 i=nx

` L
x=0 x=L

In the discrete form, the boundary conditions are: u(l) = u(nx-2) and u(nx) = u(3). When these conditions
are used, u(2) = u(nx-1) ) should be automatically satisfied (note a typo with the previous version of this
formula). For actual implementation, you integrate the finite difference equation forward in time for i = 2
to nx-1, and set boundary conditions at i = 1 and nx.

Assume c = 1.0, run your code with Courant numbers of 0. 10, 0.25, 0.50, and 1.00, show the plots for
the numerical solutions at time t = 50.0, 100.0, 200.0, and 400.0, and discuss your results in light of what
you know about this scheme based on the amplitude and phase error analyses. Is there anything special
about solution with Courant number = 1?

d. For Courant numbers of 0. 10, 0.25, 0.50, and 1.00 and for t = 50.0, 100.0, 200.0, and 400.0, use Takacs'
method to compute the amplitude and phase errors of numerical solutions relative to the exact solution
(which is simply the initial condition shifted to the right by the number of time steps multiplied by the
Courant number), and compare them with the theoretical predictions made in part b. Is the predominant
error type similar to that anticipated from the modified equation? Comment on your findings.

2
Think about the following questions and make sure you know the expected answers. Read Durran (1991
MWR). You do not need to hand in answers to these questions.

Consider once again, the 1-D advection equation in the previous problem:

∂u ∂u
+c =0
∂t ∂x

where c is a positive and constant advection speed.

Use a periodic domain containing 32 points with Δx 1/32 and an initial condition given by

⎧{64[( x − 1/ 2) 2 − 1/ 64}2 if 3 / 8 ≤ x ≤ 5 / 8
u0 ( x ) = ⎨
⎩ 0 otherwise

Also, let c = 0.25. With these conditions, the feature being advected completes one circuit through the
domain in a time of 1.0.

a. Using the leapfrog time-differencing scheme and a second-order centered-in-space discretization for the
advection term, run the solution to t = 3 using Courant numbers of 0.7, 0.5, and 0. 1. Compute the
dispersion, dissipation, and diffusion errors using Tackacs' method.

b. Repeat part a, this time adding the Asselin-Robert time filter with a filter coefficient of 0. 1. Discuss the
results.

c. Repeat part a, this time using a fourth-order centered-in-space difference for the advection term, given by

uin +1 − uin −1 ⎡ 4 un − un 1 un − un ⎤
= −c ⎢ i +1 i −1 − i + 2 i − 2 ⎥
2Δt ⎣ 3 2 Δx 3 4 Δx ⎦

Before running the program, perform a linear stability analysis and determine the phase and amplitude
errors of this scheme relative to that in part a.

d. Repeat-part c, this time using the third-order Adams-Bashforth time differencing scheme. Do not
perform a stability analysis. Discuss your results and comment on the possible advantages and
disadvantages of the third-order Adams-Bashforth scheme.

3
 Meteorology 5344, Fall 2003
Computational Fluid Dynamics
Hour Exam #1
September 29, 2003

1. (30%) Issues on supercomputing.

a) Give and briefly discuss the main reasons for the supercomputing industry to move during
past decade from conventional shared-memory parallel architectures to distributed-
memory parallel systems.

• Distributed-memory systems are cheaper to build and often use off-shelf processors
and components
• Distributed-memory systems are more scalable than shared-memory systems
• Of course, multi-processor parallel processing is essentially to solve large problems.
• Standardized tools and libraries (e.g., MPI) has matured and become commonly
available
• A large number of codes and algorithms have been developed in the past decade that
can run on distributed memory architectures.

b) For a program that contains a fraction s of non-parallelizable (serial) code, what is the
maximum possible speedup when run on p number of processors? Comment on your
result.

Write down or derive the Amhdal's Law. The speedup is optimal and assumes no
overhead. The overall speedup is seriously limited by the fraction of non-parallel code,
especially when p is large. E.g., for p=2, and p=1000,

2. (30%) Given the following three PDE's,

∂ 2u ∂ 2u
− =0 (1)
∂t 2 ∂x 2
∂ 2u ∂ 2u
+ =0 (2)
∂y 2 ∂x 2
∂u ∂ 2u
− =0 (3)
∂t ∂x 2

a) Classify these equations in terms of the canonical types;

Test B2 - 4AC and determine the types.

b) Why is the number of characteristics equations important in the classification of PDEs?

 The number of real characteristics that exist, if any, tells us if the equation support
propagation mode(s), therefore the fundamental properties of the equation.

c) Discuss their associated domain of dependence with the help of schematics.

See notes.
∂u
3. (40%) For first-order spatial derivative ,
∂x
a) Construct a consistent finite difference approximation by using three equally spaced grid
points, one at the current location, and two on the left (that's right, both on the left side).

∂u
Let = aui −1 + bui + cui +1 + O(∆x) m
∂x

Using our Taylor series for ui-1, ui and ui-2, we can write

aui − 2 + bui −1 + cui =

∂u (2∆x) 2 ∂ 2u (2∆x)3 ∂ 3u
a (ui − 2∆x + − + ...)
∂x 2! ∂x 2 3! ∂x 3
∂u ∂ 2u (∆x)3 ∂ 3u
+ b(ui − ∆x + − + ...)
∂x 2! ∂x 2 3! ∂x3
+ cui
∂u 4a + b ∂ 2u (8a + b)(∆x)3 ∂ 3u
= (a + b + c)ui − (2a + b)∆x + ( ∆x ) 2 2 − + ...
∂x 2 ∂x 3! ∂x3

Setting

a+b+c = 0

- (2a + b )∆x = 1

4a + b
(∆x) 2 =0
2

Solve for a, b and c, we obtain

1 4 3
a= ,b = − ,c = , therefore
2∆x 2∆x 2∆x

∂u ui − 2 − 4ui −1 + 3ui (∆x) 2 ∂ 3u

= + + ...
∂x 2∆x 3 ∂x3
b) Show that the scheme is consistent and determine the order of accuracy of the scheme.

And the scheme is second-order accurate.

The scheme is consistent because the truncation error goes to zero when ∆x goes to zero.

c) If you are to use this spatial difference scheme to solve simple advection equation
∂u ∂u
+c = 0 , in what situation this scheme is not suitable? When it is not, is there any
∂t ∂x
alternative to the difference formulation that would work?

When c > 0, the scheme is upstream biased, which is physically reasonable because the
signals described by the equation propagates from left to right, and the numerical domain
of dependence can cover the true domain of dependence. When c < 0, the scheme becomes
downstream biased, which is physically unreasonable and the scheme will be unstable. In
this situation, one can use 2 points to the right instead of those on the left.
 ∂u
Let = aui −1 + bui + cui +1 + O(∆x) m
∂x

Using our Taylor series for ui-1, ui and ui-2, we can write

aui − 2 + bui −1 + cui =

∂u (2∆x) 2 ∂ 2u (2∆x)3 ∂ 3u
a (ui − 2∆x + − + ...)
∂x 2! ∂x 2 3! ∂x 3
∂u ∂ 2u (∆x)3 ∂ 3u
+ b(ui − ∆x + − + ...)
∂x 2! ∂x 2 3! ∂x3
+ cui
∂u 4a + b 2 ∂ u
2
(8a + b)(∆x)3 ∂ 3u
= (a + b + c)ui − (2a + b)∆x + ( ∆x ) − + ...
∂x 2 ∂x 2 3! ∂x3

Setting

a+b+c = 0

- (2a + b )∆x = 1

4a + b
(∆x) 2 =0
2

Solve for a, b and c, we obtain

1 4 3
a= ,b = − ,c = , therefore
2∆x 2∆x 2∆x

∂u ui − 2 − 4ui −1 + 3ui (∆x) 2 ∂ 3u

= + + ...
∂x 2∆x 3 ∂x3

b)