You are on page 1of 39

# Lecture 15

Orthogonal Functions
Fourier Series

## LGA mean daily temperature time series

is there a global warming signal?

## Model that includes annual variability

T(t) = a + bt +
A1 cos(2f1t) + B1 sin(2f1t) +
A2 cos(2f2t) + B2 sin(2f2t) +
A3 cos(2f3t) + B3 sin(2f3t) +
with

## f1 = 1 cycle per year

f2 = 2 cycles per year
etc

cos{2f1(t-t0)}

cos{2f1(t-t0)}

## Cosine does not start at t=0

But remember cos(a+b)=cos(a)cos(b)-sin(a)sin(b)

## cos(a+b)=cos(a) cos(b) - sin(a) sin(b)

cos{2f1(t-t0)} =
cos(2f1t0) cos(2f1t)
sin(2f1t0) sin(2f1t) =
A cos(2f1t) + B sin(2f1t)

## cos(a+b)=cos(a) cos(b) - sin(a) sin(b)

cos{2f1(t-t0)} =
cos(2f1t0) cos(2f1t)
sin(2f1t0) sin(2f1t) =
A cos(2f1t) + B sin(2f1t)
So using both sines and cosines moves the delay, t0, out of
the cosine, and into the coefficients of the sines and
cosines. This trick linearizes the unknown, t0.

## Why more than one frequency?

f1 = 1 cycle per year
f2 = 2 cycles per year
etc
Allows us to represent non-sinusoidal
shape of annual cycle.

cos(ft)

0.3cos(2ft)

sum: cos(ft)+0.3cos(2ft)
exactly periodic,
but shape not
exactly sinusoidal

data
fit
constant term, a

error of fit, e
linear term, bt

## Statistics of linear term, bt

b = 0.31 degrees F per decade
d = [ eTe / N ]1/2 = 7 deg F
Cm = d2 [GTG]-1
b = [ d2 Cmb,b ]1/2 = 0.05 degrees F per decade
95% confidence
b = 0.310.1 degrees F per decade

So LGA is warming

## sines and cosines are

orthogonal functions
T(t) = A0 +
A1 cos(2f1t) + B1 sin(2f1t) +
A2 cos(2f2t) + B2 sin(2f2t) +
a Fourier
Seriest) +
A3 cos(2f3t)Called
+B
sin(2f
3
3
with f2=2f1, f3=3f1, etc

## Standard least-squares G matrix

1 cos(2f1t1) sin(2f1t1) cos(2f2t1) sin(2f2t1)
1 cos(2f1t2) sin(2f1t2) cos(2f2t2) sin(2f2t2)
G=

## 1 cos(2f1t3) sin(2f1t3) cos(2f2t3) sin(2f2t3)

1 cos(2f1t4) sin(2f1t4) cos(2f2t4) sin(2f2t4)
1 cos(2f1t5) sin(2f1t5) cos(2f2t5) sin(2f2t5)
1 cos(2f1t6) sin(2f1t6) cos(2f2t6) sin(2f2t6)

## With the proper choice of f1

the matrix GTG is diagonal
dot product of any pair of columns
of G is zero
columns of G are orthogonal

## The proper choice of f1

Suppose the time-series is N data points long, with
spacing t.
Then the lowest frequency must be f1 = 1 / (Nt)
one oscillation over the length of the time-series
And the highest frequency must be fN/2 = 1 / (2t)
one-half oscillation per sampling interval

f1 = 1 / (2Nt)

f1 = 1 / (2t)

## note sine is zero

Count of unknowns
The constant term, one unknown
plus
2 coefficients per frequency, N/2 frequencies so N unknowns
minus
One unknown since the fN/2 term, which has no sine term
equals
N unknowns, same as number of data

MatLab Code
N = 100;
dt = 0.5;
tmin = 0.0;
t = tmin + dt*[0:N-1]';
tmax = tmin + dt*(N-1);

% times vector

df = 1/(N*dt);
M = N;

% frequency spacing
% number of unknowns same as data

G = zeros(N,M);
G(:,1)=ones(N,1);
for p = 2*[1:M/2-1]
G(:,p) = cos(pi*p*df*t);
G(:,p+1) = sin(pi*p*df*t);
end
p=M/2;
G(:,M) = cos(2*pi*p*df*t);

## [GTG]11= [GTG]NN=N Other diagonal elements [GTG]ii=N/2

Off diagonal elements are zero

So least-squares solution is
m = [GTG]-1 GTd =
= diag( N-1, 2/N, 2/N, N-1 ) GT d
NO matrix inversion required!

## GTG for N=100

data, d
d=Gm with
m=[GTG]-1GTd
d=Gm with
m=DGTd
where D=diag( N-1, 2/N, 2/N, N-1 )

spectrum
amount of power at different
frequencies
si2 = Ai2 + Bi2
s i2

time-series has
a lot of
energy at
frequency fp

fi
fp

N=4380

2 mo

3 mo

4 mo

6 mo

12 mo

Big annual
cycle in Neuse
hydrograph

## Error Estimates for Fourier Series

Assume uncorrelated, normally-distributed data, d, with
variance d2
The problem Gm=d is linear, so the unknowns, m, (the
coefficients of the cosines and sines, Ai and Bi) are also
normally-distributed.
Since sines and cosines are orthogonal, GTG is diagonal
and Cm= d2 [GTG]-1 is diagonal, too
So that ms have uncorrelated errors. All but the first and last
have variance m2= 2d2/N.
The spectrum si2=Ai2+Bi2 is the sum of two uncorrelated,
normally distributed random variables and is thus 2distributed.
The 2-distribution has a variance of 4, so that s2= 8d2/N

## Switching to complex numbers

nothing different in principle
but calculations become easier

But first
Lets switch to angular frequency
measured in radians per second
i = 2 fi
Beats writing all those 2s !

Remember
Eulers formula
exp( it ) = cos( t ) + i sin( t )
?

## exp( it ) = cos( t ) + i sin( t )

exp( -it ) = cos( t ) - i sin( t )
cos( t ) = (1/2) [exp( it ) + exp( -it )]
sin( t ) = (1/2i) [exp( it ) - exp( -it )]

=1

Lets compare

=0
with p=p

p= -p
T(t) = A0 cos(0t) + B0 sin(0t) +
A1 cos(1t) + B1 sin(1t) +
A2 cos(2t) + B2 sin(First,
2t) +if T is real, then we must have C-p = Cp*
A3 cos(3t) + B3 sin(Then
exp(-pt) + Cp exp(pt) =
3t) +C-p

with
T(t) = ... +
C-2 exp(-i2t) +
C-1 exp(-1t) +
C0 exp(i0t) +

## (Cpr-iCpi) [cos(pt) - i sin(pt)] +

(Cpr+iCpi) [cos(pt) + i sin(pt)] =
2Cpr cos(pt) - 2Cpi sin(--pt)]

## So Ap= 2Cpr and Bp= -2Cpi

So these two representations are equivalent

T(t) = ... +
C-2 exp(-i2t) +
C-1 exp(-1t) +
C0 exp(i0t) +
C1 exp(i1t) +
C2 exp(i2t) +
C3 exp(i3t) +
T0
T1
T2
T3
T4

exp(-i2t0)
exp(-i2t1)
exp(-i2t2)
exp(-i2t3)

## Implies a simple form of

the equation d=Gm

exp(-i1t0)
exp(-i1t1)
exp(-i1t2)
exp(-i1t3)

exp(i0t0)
exp(i0t1)
exp(i0t2)
exp(i0t3)

exp( i1t0)
exp( i1t1)
exp( i1t2)
exp( i1t3)

exp(i2t0)
exp( i2t1)
exp( i2t2)
exp( i2t3)

C-2
C-1
C0
C1
C2

## Least-squares with complex numbers

real numbers:

complex nos:

given Gm =d
minimize E=eTe
implies m=[GTG]-1 GT d
The Hermitian
transpose, that is, the
transpose of the
complex conjugate.

given Gm =d
minimize E=eHe
where eH = e*T
implies m=[GHG]-1 GH d

The formula m=[GHG]-1GHd is not hard to work out using the standard
minimization procedure, but we dont have time to work it out in class.

T0
T1
T2
T3
T4

d=Gm
=

exp(-i2t0)
exp(-i2t1)
exp(-i2t2)
exp(-i2t3)

exp(-i1t0)
exp(-i1t1)
exp(-i1t2)
exp(-i1t3)

exp(i0t0) exp(i1t0)
exp(i0t1) exp(i1t1)
exp(i0t2) exp(i1t2)
exp(i0t3) exp(i1t3)

exp(i2t0)
exp(i2t1)
exp(i2t2)
exp(i2t3)

## exp(-i2t4) exp(-i1t4) exp(i0t4) exp(i1t4) exp(i2t4)

Note T2 i Ci exp(+it2)

C-2
C-1
C0
C1
C2

m=N-1GHm
exp(i2t0) exp(i2t1) exp(i2t2)
exp(i1t0) exp(i1t1) exp(i1t2)
=N-1 exp(i0t0) exp(i0t1) exp(i0t2)
exp(-i1t0) exp(-i1t1) exp(-i1t2)

exp(i2t3) exp(i2t4)
exp(i1t3) exp(i1t4)
exp(i0t3) exp(i0t4)
exp(-i1t3) exp(-i1t4)

## exp(-i2t0) exp(-i2t1) exp(-i2t2) exp(-i2t3) exp(-i2t4)

Note C2 i Ti exp(-iti)

C-2
C-1
C0
C1
C2

T0
T1
T2
T3
T4

T0
T1
T2
T3
T4

d=Gm
=

exp(-i2t0)
exp(-i2t1)
exp(-i2t2)
exp(-i2t3)

exp(-i1t0)
exp(-i1t1)
exp(-i1t2)
exp(-i1t3)

exp(i0t0) exp(i1t0)
exp(i0t1) exp(i1t1)
exp(i0t2) exp(i1t2)
exp(i0t3) exp(i1t3)

exp(i2t0)
exp(i2t1)
exp(i2t2)
exp(i2t3)

## exp(-i2t4) exp(-i1t4) exp(i0t4) exp(i1t4) exp(i2t4)

Note T2 i Ci exp(+it2)

C-2
C-1
C0
C1
C2

M=N-1GHm
exp(i2t0) exp(i2t1) exp(i2t2)
exp(i1t0) exp(i1t1) exp(i1t2)
=N-1 exp(i0t0) exp(i0t1) exp(i0t2)
exp(-i1t0) exp(-i1t1) exp(-i1t2)

Opposite
signs

exp(i2t3) exp(i2t4)
exp(i1t3) exp(i1t4)
exp(i0t3) exp(i0t4)
exp(-i1t3) exp(-i1t4)

## exp(-i2t0) exp(-i2t1) exp(-i2t2) exp(-i2t3) exp(-i2t4)

Note C2 i Ti exp(-iti)

C-2
C-1
C0
C1
C2

T0
T1
T2
T3
T4

## Discrete Fourier Transform

Find the coefficients C given the data, T
Note normalization factor of N
Equivalent to m = GHd

-1

## Ck = n=-N/2N/2 Tn exp(2ikn/N ) with k=-N, , N

Discrete Inverse Fourier Transform
Find the data T given the coefficients, C
Equivalent to d = N-1Gm Note normalization factor of N

-1

## Tn = N-1k=-N/2N/2 Ck exp( 2ikn/N ) with n=-N, , N

Warnings: 1) no one can agree on signs
2) no one can agree on normalizations

Counting unknowns
frequencies from (N/2) to (N/2)in steps of
So N+1 complex numbers, Cp
So 2N+2 real and imaginary parts, C pr and Cpi
But C-p = Cp*, so really only N/2+1 unknown complex
numbers
So N+2 real and imaginary parts , Cpr and Cpi (p0)
But C0i=0 and CN/2i=0 (always)
So N unknowns, matching N data

## % standard fft setup. The standard implementation of the digital fourier

% transform is VERY INFLEXIBLE. Learn these rules:
N=256;
% you can choose the length N of the time series
% in some implementations N can be any positive
% integer, but in others it MUST be a
% power-or-two. I set it here to 256, which
% is two-to-the-eigth-power.
dt=1.0;
% and you can choose the sampling interval dt
% but then the following variables are set
tmax=dt*(N-1); % we presume the time series starts at t=0, so
% the maximum time is tmax
t=dt*[0:N-1];
% time then goes from 0 to (N-1)*dt
fmax=1/(2.0*dt);
% the maximum frequency in the fft calculation is
% called the Nyquist frequency. It is
% determined by the two-points-per wavelength
% rule
df=fmax/(N/2); % the frequency spacing, df, assumes that a N-point
% time series is reperesented by an N-point fourier
% transform
f=df*[0:N/2,-N/2+1:-1]'; % The fourier transform has N values, from a negative
% frequency of -(fmax-df) through zero freqency, to
% positive frequency of fmax. But note the weird order. The
% zero and positive frequencies are put in the first
% half of the array and the negative frequencies are
% put in the second half.

## % p is the timeseries whose transform is being computed

w0 = 2*pi*fmax/10; % sample p, a simple sinusoid of frequency w0
p = sin(w0*t);
% fourier transform using MatLab's fft function. The help function
% says that it uses the NEGATIVE sign in the exponential.
pt=fft(p); % these are the coefficients, C, of the complex exponential
% presumably one would do something with the fourier transform
% at this point - apply a filter, for example. But I do nothing.
% Inverse fourier transform using the Matlab function ifft.
% Help says it uses the POSITIVE sign in the exponential, and that
% it has the right normalization that ifft(fft(x))=x. But
% BE WARNED, that doesn't mean that the normalization on fft
% is 1 and that the normalization on ifft is (1/N), like
% I had it in class. You can put any constant, b, in front
% of the fft integral, as long as you put 1/b in front of
% the IFFT integral. But judging by the Help, I think that
% Matlab used b=1.
pr=ifft(pt); % this reconstructs the function from the coefficients

## The fast Fourier transform algorithm

The Fourier Transform equation
m = [GTG]-1GTd = diag(N-1,2/N, 2/N,N-1) GT d
has N multiplications for each of N unknowns,
So N2 in total. For example, for N=1024, N2=1,048,576
But in the special case of N being a power of two,
there is an algorithm the fast fourier transform
algorithm - that can compute m in only Nlog2N
multiplications
For example, N=1024, Nlog2N=10,240