Separation of Variables

Some slides adapted from Ake

Nordlund and Dennis Healy
 Separation of variables

Main principles
Why?

Techniques
How?
 Main principles
Why?
Because many systems in physics are
separable
In particular systems with symmetries
Such as spherically symmetric problems
Example:
Example: Spherical
Spherical harmonics
harmonics
u(r,,,t) = Y (r,,) eit
it
u(r,,,t) = Ylm
lm (r,,) e
 Partial Differential Equations:
Separation of Variables

Main principles
Why?
Common in physics!
Greatly simplifies things!

Techniques
How?
 Separation of variables:
the general method
Suppose we seek a PDE solution u(x,y,z,t)

Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
u/x
u/x == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
22u/x 22= X(x) Y(y) Z(z) T(t)
u/x = X(x) Y(y) Z(z) T(t)
 Separation of variables:
the general method
Suppose we seek a PDE solution
u(x,y,z,t):
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
u/y
u/y
u/y====X(x)
u/y X(x)Y(y)
X(x)
X(x) Y(y)Z(z)
Y(y)
Y(y) Z(z)T(t)
Z(z)
Z(z) T(t)
T(t)
T(t)
22u/y 22= X(x) Y(y) Z(z) T(t)
u/y = X(x) Y(y) Z(z) T(t)
 Separation of variables:
the general method
Suppose we seek a PDE solution
u(x,y,z,t):
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
u/y
u/z
u/z====X(x)
u/y X(x)Y(y)
X(x)
X(x) Y(y)Z(z)
Y(y)
Y(y) Z(z)T(t)
Z(z)
Z(z) T(t)
T(t)
T(t)
22u/z
u/y
u/y = X(x) Y(y)
u/z
22= X(x) Y(y) Z(z)
Y(y)Z(z)
Y(y) Z(z) T(t)
Z(z) T(t)
 Separation of variables:
the general method
Suppose we seek a PDE solution
u(x,y,z,t):
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
u/y
u/t
u/t====X(x)
u/y X(x)Y(y)
X(x)
X(x) Y(y)Z(z)
Y(y)
Y(y) Z(z)T(t)
Z(z) T(t)
T(t)
T(t)
22u/t
u/y
u/t
22=
u/y ==X(x)
22 X(x) Y(y)
X(x)Y(y) Z(z)
Y(y)Z(z)
Y(y) T(t)
Z(z)T(t)
Z(z) T(t)
T(t)
 Examples

u(x,y,z,t) = xyz22sin(bt)
u(x,y,z,t) = xyz sin(bt)

u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt

u(x,y,z,t) = (x22+y22) z cos(t)

u(x,y,z,t) = (x +y ) z cos(t)
 Examples
Separable?

9 u(x,y,z,t) = xyz22sin(bt)
u(x,y,z,t) = xyz sin(bt) Yes!

u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt No!

? u(x,y,z,t) = (x22+y22) z cos(t)

u(x,y,z,t) = (x +y ) z cos(t) Hm
 Examples
Separable?

9 u(x,y,z,t) = xyz22sin(bt)
u(x,y,z,t) = xyz sin(bt) Yes!

u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt No!

? u(x,y,z,t) = (x22+y22) z cos(t)

u(x,y,z,t) = (x +y ) z cos(t) Hm

(x 22+y22) p22
(x +y ) p
 Examples
Separable?

9 u(x,y,z,t) = xyz22sin(bt)
u(x,y,z,t) = xyz sin(bt) Yes!

u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt No!

9 u(p,z,t) = p
u(p,z,t) = p
22 zz cos(t)
cos(t) Yes!
 Example PDEs
The wave equation

cc2222uu== 22u/t
u/t
22

where
22 22/x22+ 22/y22+ 22/z22
/x + /y + /z

Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
 Separation of variables in
the wave equation
For the ansatz to work we must have (lets
set c=1 for now to get rid of a triviality!)

XYZT
XYZT ++ XYZT
XYZT ++ XYZT
XYZT == XYZT
XYZT

Divide though with XYZT, for:

X/X
X/X ++ Y/Y
Y/Y ++ Z/Z
Z/Z == T/T
T/T

This
Thiscan
canonly
onlywork
workififall
allof
ofthese
theseare
areconstants!
constants!
 Separation of variables in
the wave equation
Lets take all the constants to be negative
Negative bounded behavior at large x,y,z,t

X/X
X/X ++ Y/Y
Y/Y ++ Z/Z
Z/Z == T/T
T/T
These
Theseare
arecalled
called separation
separationconstants
constants

-- z 22 -- 22 -- 22 = - 22
z =-
This
Thisisistypical:
typical:IfIf//when
whenaaPDE PDEallows
allowsseparation
separationofof
variables,
variables,the
thepartial
partialderivatives
derivativesare
arereplaced
replacedwith
withordinary
ordinary
derivatives,
derivatives,and
andall allthat
thatremains
remainsofofthe
thePDE
PDEisisan
analgebraic
algebraic
equation
equationandandaaset setof ofODEs
ODEsmuch
mucheasier
easierto
tosolve!
solve!
 Separation of variables in
the wave equation
Solutions of the ODEs:

Example:
Example: x-direction
x-direction
X/X = - z 22
X/X = - z

General
General solution:
solution:
X(x)
X(x) == AA exp(i z x)
exp(i z x) ++ BB exp(-i
exp(-i z
z
x)
x)
or
or
X(x)
X(x) == A
A cos(z
cos(z x)x) ++ B
B sin(z
sin(z x)
x)
 Separation of variables in
the wave equation
Combining all directions (and time)

Example:
Example:
uu == exp(i
exp(i zz x)
x) exp(i y)
exp(i y) exp(i
exp(i z)z) exp(-i
exp(-i
ct)
ct)
or
or
uu == exp(i
exp(i (z
(z xx ++
yy ++
zz --
t))
t))
This
This is
is aa traveling
traveling wave,
wave, with
with wave
wave vector
vector {z,,}
{z,,}
and frequency.
and frequency . AA general
general solution
solution of
of the
the wave
wave
equation
equation is is aa super-position
super-position of
of such
such waves.
waves.
 Example PDEs
The Laplace equation

22uu== 00
where
22 22/x22+ 22/y22+ 22/z22
/x + /y + /z

Ansatz:
Ansatz:
u(x,y,z)
u(x,y,z) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z)
 Separation of variables in
the Laplace equation
For the ansatz to work we must have

XYZ
XYZ ++ XYZ
XYZ ++ XYZ
XYZ == 00

Divide though with XYZ, for:

X/X
X/X ++ Y/Y
Y/Y ++ Z/Z
Z/Z == 00

Again,
Again,this
thiscan
canonly
onlywork
workififall
allof
ofthese
theseare
areconstants!
constants!
 Separation of variables in
the Laplace equation
This time we cannot take all constants to be
negative!
X/X
X/X ++ Y/Y
Y/Y == 00

-- z 22 -- 22 = 0
z =0
At
At least
least one
one of
of the
the terms
terms must
must be positive
be positive
imaginary
imaginary wave
wave number;
number; i.e.,
i.e., exponential
exponential rather
rather
than
than sinusoidal
sinusoidal behavior!
behavior!
 Separation of variables in
the Laplace equation
Solutions of the ODEs:

Example:
Example: x-direction
x-direction
X/X = - z 22 << 00 assumed
X/X = - z assumed

General
General solution:
solution:
X(x)
X(x) == AA exp(i z x)
exp(i z x) ++ BB exp(-i
exp(-i z
z
x)
x)
or
or
X(x)
X(x) == A
A cos(z
cos(z x)x) ++ B
B sin(z
sin(z x)
x)
 Separation of variables in
the Laplace equation
Solutions of the ODEs:

Example:
Example: y-direction
y-direction
Y/Y = + z 22 >> 00 follows!
Y/Y = + z follows!

General
General solution:
solution:
Y(y)
Y(y) == C
C exp(z
exp(z y)
y) ++ DD exp(-z
exp(-z y)
y)
or
or
Y(y)
Y(y) == C
C cosh(z
cosh(z y)y) ++ B
B sinh(z
sinh(z y)
y)
 Separation of variables in
the Laplace equation
Combining x & y directions

Example:
Example:
uu ==[A
[Aexp(i
exp(izzx)
x)++BBexp(-i
exp(-izzx)]
x)][C
[Cexp(z
exp(zy)
y)++DDexp(-z
exp(-zy)]
y)]
or
or
uu ==[A
[Acos(z
cos(zx)
x)++B
Bsin(z
sin(zx)]
x)][C
[Ccosh(z
cosh(zy)
y)++D
Dsinh(-z
sinh(-zy)]
y)]
 Message
Complex exponentials are a natural basis
for representing the solutions of the wave
equation
A microphone samples the solution at a
given point
Variation in time is naturally expressed in
terms of Fourier series.
 Fourier Methods
Fourier analysis (harmonic analysis) a
key field of math
Has many applications and has enabled
many technologies.
Basic idea: Use Fourier representation to
represent functions
Has fast algorithms to manipulate them
(the fast Fourier Transform)
 Basic idea
Function spaces can have many different
types of bases
We have already met monomials and
other polynomial basis functions
Fourier introduced another set of basis
functions : the Fourier series
These basis functions are particularly
good for describing things that repeat with
time
 Fouriers Representation
F(t)= A0/2 + A1 Cos( t) + A2 Cos(2 t) + A3 Cos(3 t) +
+ B1 Sin( t) + B2 Sin(2 t) + B3 Sin(3 t) +

For coefficients that go to 0 fast enough these sums will converge

at each value of t .
This defines a new function, which must be a periodic function.
(Period 2 )

Fouriers claim: ANY periodic function f(t) can be written this way
 Music
http://www.phy.ntnu.edu.tw/ntnujava/viewtop
ic.php?t=33
 Fouriers Representation
Represent f(t) = t for | t | < and 2 periodic

Sin( t) +

-7.5 -5 -2.5 2.5 5 7.5

-1

-2
Fouriers Representation

1 Sin( t) +

-7.5 -5 -2.5 2.5 5 7.5

-1

-2
 Fouriers Representation

Sin( t) - 1/2 Sin( 2 t) +

-7.5 -5 -2.5 2.5 5 7.5

-1

-2
Fouriers Representation

1 Sin( t) - 1/2 Sin( 2 t) +

-7.5 -5 -2.5 2.5 5 7.5

-1

-2
Fouriers Representation

1 Sin( t) - 1/2 Sin( 2 t) +1/3 Sin( 3 t) +

-7.5 -5 -2.5 2.5 5 7.5

-1

-2
Fouriers Representation

1 Sin( t) - 1/2 Sin( 2 t) + 1/3 Sin( 3 t) +

-7.5 -5 -2.5 2.5 5 7.5

-1

-2
Fouriers Representation

20th degree Fourier expansion

-7.5 -5 -2.5 2.5 5 7.5

-1

-2
 How do you get the Coefficients
for a given f ?
A0/2 + A1 Cos( t) + A2 Cos(2 t) + A3 Cos(3 t) +
+ B1 Sin( t) + B2 Sin(2 t) + B3 Sin(3 t) +

Fouriers claim:
ANY periodic function f(t) can be written this way (SYNTHESIS)
The coefficients are uniquely determined by f: (ANALYSIS)

0
2
Ak = 1/ 2 f (t) cos( k t) d t

Bk = 1/ 2 2
f (t) sin ( k t) d t
0
 Fourier Analysis: match data with sinusoids
1

0.75

0.5

s(t) 0.25

-0.25

-0.5
2 4 6 8 10 12

Time t
-0.75

1 1 1

0.75 0.75 0.75

0.5 0.5 0.5

0.25 0.25 0.25

2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12
-0.25 -0.25 -0.25

-0.5 -0.5 -0.5

-0.75 -0.75 -0.75

-1 -1 -1
1 1 1

0.75 0.75 0.75

0.5 0.5 0.5

0.25 0.25 0.25

2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12
-0.25 -0.25 -0.25

-0.5 -0.5 -0.5

-0.75 -0.75 -0.75

-1 -1 -1

0.8

S[k] 0.6

0.4

=
s (t) cos( k t) d t
0.2

6 10 14 18 22 26 28

Frequency k
 Complex Notation
For f, periodic with period p

Fourier transform f(t) -> F[k]

0
p
F[k] = 1/p f (t) e- 2 i k t/p d t

0 f (t) cos(2 k t/p) dt

= p
1/p

0 f (t) sin(2 k t/p) dt

p
i/p

Inverse Fourier transform F[k] -> f(t)

f(t) = F[k] e 2 i k t/p

k in Z
 Sampling
Fourier representations work just fine with sampled data

Simple connection to Fourier of the continuous function it came form

Familiar example: Digital Audio

 Measuring and Discretizing Input field

Physical Field
(continuum)

PHYSICAL
LAYER
 Sample

Physical Field
(continuum)

PHYSICAL
LAYER
 Quantize

Physical Field
(continuum)

PHYSICAL discretized waveform

LAYER
 Code and output
Digital Representation
Physical Field
(continuum)

PHYSICAL
LAYER

3, 8, 10, 9, 3, 1, 2
 Sampling
Often must work with a discrete set of measurements of a continuous function

10 10
1

8
0.5
6

4 -2 -1 1

-0.5
2

-1
-3 -2
-3 -1 -1 1 12 3 3
 Sampling
Takes a function defined on R and creates a function defined on Z

10 10
1

8
0.5
6

4 -2 -1 1

-0.5
2

-1
-3 -2
-3 -1 -1 1 12 3 3

Sh f(t) [n] = f(n h) h

 Sampling
In this case, it is a periodic function on Z,

(Assuming p/h = N)
10 10
1

8
0.5
6

4 -2 -1 1

-0.5
2

-1
-3 -2
-3 -1 -1 1 12 3 3

0123 .. N

[n] = f(n h) [n+N] = [n] h

DFT N-1

0
p
f (t) e- 2 i k t/p dt [n] e -2 i k nh /p
n =0
10 10
1

8
0.5
6

4 -2 -1 1

-0.5
2

-1
-3 -2
-3 -1 -1 1 12 3 3

f(t) [n] = f(n h) h

DFT and its inverse for periodic
discrete data
N-1
[k] =
[n] e -2 i k n h /p p =Nh
n =0

N-1
= [n] e -2 i k n / N
n=0

This is automatically periodic in k with period N

Inverse is like Fourier series, but with only p terms
 DFT: Discrete time periodic
version of Fourier
time domain frequency domain
N

1/ [k] e -2 i k m/N = [m]

k =0
1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

1 2 3 4 5 6 7 8 9101112131415161718192021 1 2 3 4 5 6 7 8 9101112131415161718192021

[k]= [k] e 2 i m k/N

[k], on PN k =0
[m] on PN
i.e. on Z , Period N i.e. on Z , Period N
 o O C e e so so
Fourier
0
p
f (t) e- 2 i k t/p
1
0.5
0

d t /p
-0.5 1
-1

10
0.8

7.5
0.6
5

2.5 0.4
0
-1
-0.5
0.2

F[k] e 2 i k t/p
0
0.5

1 2 3 4 5 6 7 8 9 10 1112 1314 1516 1718 1920 21

f(t), period p
k in Z F[k] , on Z

S p/N N-1

1
1/ [k] e -2 i k m/N
1
PN

0.8 k =0 0.8

0.6 0.6

0.4 0.4

0.2 0.2

N-1

1 2 3 4 5 6 7 8 9101112131415161718192021 1 2 3 4 5 6 7 8 9101112131415161718192021

[k], on PN [m] on PN
[k] e 2 i m k/N
i.e. on Z , Period N i.e. on Z , Period N
k =0
time domain frequency domain
Discrete time Numerical Fourier Analysis

DFT is really just a matrix multiplication!

N-1
[m] = 1/ e -2 i k m/N [k]
k =0
time index
[0] 60 [0]

Freq. index
[1] 50 [1]
[2] 40
[2]
. = 30
.
. 20 .
. .
[N-1]
10

[N-1] 0
0 10 20 30 40 50 60

= FN
 Numerical Harmonic Analysis
FFT: Symmetry Properties permits Divide and Conquer
Sparse Factorization

Fmn = ( Fm I n ) T n
mn
( I m Fn ) Lmn
m

Naive
15000 60

50

12500 40

10000 30

20

7500 10

Fn FFT
5000
0 10 20 30 40 50 60

2500

20 40 60 80 100 120