Spectral methods for di↵erential equations

An example:
Find the 2⇡-periodic solutions of

u 00 + ru = f (x), x 2 (0, 2⇡)

with a constant r > 0.

1
Weak formulation, with V = Hper (0, 2⇡):
Find u 2 V such that

a(u, v ) = L(v ), 8v 2 V ,

where Z Z
2⇡ 2⇡
0 0
a(u, v ) = (u v̄ + ru v̄ )dx, L(v ) = f v̄ dx.
0 0
Introduce the Fourier expansion of u,
1
X 1
X
ikx
u(x) = ûk e = ûk k (x), with k (x) = e ikx .
k= 1 k= 1

Analytic solution
Remember the orthogonality condition:
Z 2⇡ Z 2⇡
¯
k (x) ` (x)dx = e ikx e i`x
dx = 2⇡ k` ,
0 0

where k` is the Kronecker delta.

Insert the Fourier expansion and test against all basis functions ` = e i`x :
Z 2⇡ X +1 +1
X
ikx
a(u, ` ) = ( ik ûk e i`e i`x + r ûk e ikx e i`x )dx
0 k= 1 k= 1
+1
X Z 2⇡
= ûk (ik · ( i`) + r )e i(k `)x
dx = 2⇡ û` (`2 + r ).
k= 1 0

Similarly,
Z 2⇡
L(v ) = f (x)e i`x
dx = 2⇡ fˆ` .
0
Hence,
1 ˆ
ûk = fk , k = 0, ±1, ±2, . . .
k2 +r
About the analytic solution
The example was: find the 2⇡-periodic solutions of

u 00 + ru = f (x), x 2 (0, 2⇡), with a constant r > 0,

and the Fourier coefficients for u can be determined in terms of the

Fourier coefficients for f ,

1 ˆ
ûk = fk , k = 0, ±1, ±2, . . . .
k2 + r

- If r = 0, would divide by 0 for k = 0. In this case, Eq. and BC only

determines u up to a constant: If u is a solution, so is u + C .
R 2⇡
Natural to require, fˆ0 = 0 (i.e. 0 f (x)dx = 0).
p p+2
- If f 2 Hper (0, 2⇡), then u 2 Hper (0, 2⇡):

+1
X +1
X +1
X
|fˆk |2
k 2p+4 2
|ûk | = k 2p+4
2 2
< k 2p |fˆk |2 < 1
(k + r )
k= 1 k= 1 k= 1

Galerkin’s method
N/2
X
Apply now Galerkin’s method with VN = {v |v = v̂k e ikx }:
k= N/2
Introduce an expansion of u 2 VN ,
N/2
X
u(x) = ûkN e ikx .
k= N/2

If we assume an truncated Fourier expansion of f (coe↵s still the

continuous Fourier coefficients),
N/2
X
f (x) = fˆk e ikx ,
k= N/2

then, similarly to the infinite expansion, we get:

1 ˆ
ûkN = fk , k = 0, ±1, ±2, . . . , i.e. ûkN = ûk
k2 + r
Error analysis: Galerkin’s method
Error estimation:
X
eN (x) = u(x) uN (x) = ûk e ikx
|k|>N/2

Theorem
- For all square integrable functions f ,

4
keN k  kf k
N2

(quadratic convergence).
p
- If f 2 Hper (0, 2⇡):
(p+1)
O N .

- If f 2 C 1 , we have exponential convergence.

Error analysis: Proofs

- For f 2 L2 (0, 2⇡),

X X |fˆk |2
2 2
keN k = 2⇡ |ûk | = 2⇡
(k 2 + r )2
|k|>N/2 |k|>N/2
1 2 16
 2 2
kf k  4
kf k2
(N /4 + r ) N
p
- If f 2 Hper (0, 2⇡):

X |fˆk |2 X k 2p |fˆk |2
keN k2 = 2⇡ = 2⇡
(k 2 + r )2 k 2p (k 2 + r )2
|k|>N/2 |k|>N/2

2⇡ X C (p)2
 2pˆ 2
k |fk |  2p+2
(N/2) (N 2 /4 + r )2
2p N
|k|>N/2
Galerkin’s method in practice
- Due to the structure of the FFT method, most efficient when N
even, and preferably even N = 2m .
- Therefore, one normally uses expansions.
N/2 1
X
u(x) = ûk e ikx .
k= N/2

- To make sure that the function stays real valued if all coefficients
and initial conditions are real valued, one would set û N/2 = 0.
Compare to interpolation.
- Error results on last few slides based on continuous Fourier coe↵s.
But in practice, will have DFT coe↵s. Similar results.

Collocation method
- Consider again the equation u 00 + ru = f .
PN/2 1
- Ansatz as before uN = k= N/2 ûkN e ikx .
- Collocation: Use test functions vj (x) = (x xj ) for xj = jh ⇡,
h = 2⇡/N. Equivalently,

uN00 (xj ) + ruN (xj ) = f (xj ), j= N/2, . . . , N/2 1.

- Insert expansion and enforce this pointwise equality:

N/2 1
X
[ûkN (k 2 + r ) fˆk ]e ikxj = 0, all j
k= N/2

=) ûkN (k 2 + r ) fˆk = 0, all k

- The solution becomes

fˆk
ûkN = .
k2 + r
This is the same solution as obtained by the Galerkin method.
Spectral methods for PDEs
- The Galerkin/Collocation approach can also be used for
time-dependent problems.
- Expand
N/2 1
X
N
u (x, t) = ûk (t)e ikx .
k= N/2

where the Fourier coefficients now depend on time.

Similarly, expand any given function in the right hand side and the
given initial conditions.
- Example: The heat equation with periodic boundary conditions,
ut = cuxx , u(0, t) = u(2⇡, t), u(x, 0) = f (x)
yields
d
ûk + ck 2 ûk = 0, ûk (0) = fˆk N/2  k < N/2
dt
That is, we get an ODE for each Fourier coefficient.
- Time step in Fourier space, transform back to real space when
solution is needed.

Non-linear PDEs - Galerkin’s method

Consider Burger’s equation

@u @u @2u
+u ⌫ =0
@t @x @x 2
Expanding u as u N , multiply by a test function and integrate, and using
orthogonality of the complex exponentials, we get
✓ \ N
◆
d @u
ûk + u N + ⌫k 2 ûk = 0, k= N/2, . . . , N/2 1
dt @x k

where
✓ \ ◆ Z 2⇡
@u N 1 @u N
u N = uN e ikx
dx
@x k 2⇡ 0 @x
is the kth Fourier coefficient for the nonlinear term.
For non-linear PDEs, the collocation approach and Galerkin approach
might not coincide.
Non-linear term
- Consider a term w (x) = u(x)v (x).
- We have that
Z 2⇡
d) = 1
ŵk = (uv uve ikx
dx
k
2⇡ 0

which yields
X
d) =
(uv ûm v̂n , N/2  k, m, n < N/2
k
m+n=k

- Convolution sum. Straightforward evaluation requires O(N 2 )

operations.
- Idea of pseudospectral treatment:
- Transform ûm and v̂n to physical (real) space by IFFT.
- Perform a multiplication in real space.
- Transform to Fourier space by FFT to obtain ŵk .

Aliasing errors for pseudospectral treatment

Introduce the discrete transforms
N/2 1 N/2 1
X X
ikxj
Uj = ûk (t)e , Vj = v̂k (t)e ikxj , j = 0, 1, . . . , N 1,
k= N/2 k= N/2

and define
Wj = U j V j , j = 0, 1, . . . , N 1,
and N 1
1 X ikxj
Ŵk = Wj e , k= N/2, . . . , N/2 1
N
j=0

where xj = 2⇡j/N.
Use of the discrete orthogonality condition leads to
X X X
Ŵk = ûm v̂n + ûm v̂n = ŵk + ûm v̂n
m+n=k m+n=k±N m+n=k±N
| {z }
Aliasing error
Removal of aliasing errors by padding
Introduce M where M > N. Introduce the discrete transforms
M/2 1 M/2 1
X X
ikxj
Uj = ũk (t)e , Vj = ṽk (t)e ikxj , j = 0, 1, . . . , M 1,
k= M/2 k= M/2

and define
W j = U j vj , j = 0, 1, . . . , M 1,
where yj = 2⇡j/M, and
( (
ûk if N/2  k < N/2, v̂k if N/2  k < N/2,
ũk = ṽk =
0, otherwise. 0, otherwise.

Let M 1
1 X ikyj
W̃k = Wj e , k= M/2, . . . , M/2 1.
M
j=0

If we define:
Ŵk = W̃k , k= N/2, . . . , N/2 1,
and M 3N/2, then the aliasing errors are removed.

Integrating factor technique

Consider the ODE
d
ûk + ⌫k 2 ûk = F̂k
dt
This yields
d h ⌫k 2 t i 2
e ûk = e ⌫k t F̂k
dt
Discretize 2 2
(tn + t) n+1 tn n
e ⌫k ûk e ⌫k ûk 2
tn
= e ⌫k Ĝkn
t
2
(tn + t)
or, dividing through by e ⌫k ,
⇣ ⌘
⌫k 2 t
ûkn+1 = e ûkn + t Ĝkn

where Ĝkn is a combination of di↵erent F̂k :s depending on the

time-stepping method.
Example: For Forward Euler, Ĝkn = F̂kn ,
for the third
⇣ order Adams Bashforth⌘ method (AB3),
Ĝkn = 1
12 23F̂kn 16F̂kn 1
+ 5F̂kn 2
.
Non-linear PDEs - Collocation method
Again, consider Burger’s equation
@u @u @2u
+u ⌫ 2 = 0,
@t @x @x
and expand u as u N .
For the collocation method, we require that u N staisfies the equation at
xj = 2⇡j/N, j = 0, . . . , N 1, i.e. that
@u N N @u
N
@ 2uN
+u ⌫ = 0.
@t @x @x 2 x=xj

This is discretized by
@u N
+ u N DN u N ⌫DN2 u N = 0,
@t x=xj

where DN is the Fourier collocation di↵erentiation operator.

N/2 1 N 1
X ik X
2⇡ik`/N 2⇡ikj/N
(DN u)` = ak e , ak = uj e .
N
k= N/2 j=0

Collocation method for Burger’s equation

Introducing vector notation,
U(t) = (u N (x0 , t), u N (x1 , t), . . . , u N (xN 1 , t)),
the discretized equation reads:
@U
+ U · DN U ⌫DN2 U = 0,
@t
where the · means pointwise multiplication and DN is the matrix that
represents the di↵erentiation.
Starting from a di↵erent but equivalent form of Burger’s equation:

@u 1 @ 2 @2u
+ (u ) ⌫ = 0,
@t 2 @x @x 2
the collocation discretization becomes,
@U 1
+ DN (U · U) ⌫DN2 U = 0.
@t 2
These two discretizations are not equivalent, even if the two forms of the
PDE are.
Comments on time-stepping
- For linear PDEs, taking the Galerkin view point, and solving ODEs
for the Fourier coefficients, one needs to transform back to physical
space only when the solution is needed, and not in each time step.
Very cheap!
- For non-linear PDEs, both with a pseudo-spectral treatment in the
Galerkin approach, or using the collocation formulation, FFTs will be
needed in each time step to go in between physical space and real
space.

(Pseudo) spectral methods - Summary

- Exponential convergence for smooth data. Very few degrees of
freedom needed for high accuracy.
- Global basis functions. Cf FEM with local basis functions. Sti↵ness
matrix full compared to sparse.
- FFTs used to accelerate computations.
- Computational cost O(N log N) per time step, where N is number of
Fourier modes.
- Use of integrating factor to remove sti↵ness from ODEs in Fourier
space, i.e. to relax CFL condition on explicit schemes.
- Aliasing errors arise from pseudo-spectral treatment of nonlinear
terms. Can be removed by e.g. padding to the cost of a factor of
3/2 larger FFTs.
- Inflexible in terms of geometry. BCs see next slide.
(Pseudo) spectral methods -
Non-periodic boundary conditions
- Fourier series are only well-suited for periodic boundary conditions.
- In case of Dirichlet boundary conditions, Chebyshev polynomials can
be used (Strang, p. 465),

Tk (x) = cos k✓, with ✓ = arccos x

- The Chebyshev polynomials are orthogonal over 1  x  p 1, if the

inner product is defined using a weight function w (x) = 1/ 1 x 2 .
- The Chebyshev expansion of a function u 2 L2w ( 1, 1) is
1
X Z 1
2
u(x) = ûk Tk (x), ûk = u(x)Tk (x)w (x)dx,
⇡ck 1
k=0

where c0 = 2 and ck = 1, k 1.
- The quadrature points are taken e.g. as the Gauss-Lobatto points,
xj = cos ⇡j
N , j = 0, . . . , N.
Then the Fast Fourier Transform can be applied in the computations.