Environmental Data Analysis with MatLab

Lecture 18:

Cross-correlation
 SYLLABUS
Lecture 01 Using MatLab
Lecture 02 Looking At Data
Lecture 03 Probability and Measurement Error
Lecture 04 Multivariate Distributions
Lecture 05 Linear Models
Lecture 06 The Principle of Least Squares
Lecture 07 Prior Information
Lecture 08 Solving Generalized Least Squares Problems
Lecture 09 Fourier Series
Lecture 10 Complex Fourier Series
Lecture 11 Lessons Learned from the Fourier Transform
Lecture 12 Power Spectral Density
Lecture 13 Filter Theory
Lecture 14 Applications of Filters
Lecture 15 Factor Analysis
Lecture 16 Orthogonal functions
Lecture 17 Covariance and Autocorrelation
Lecture 18 Cross-correlation
Lecture 19 Smoothing, Correlation and Spectra
Lecture 20 Coherence; Tapering and Spectral Analysis
Lecture 21 Interpolation
Lecture 22 Hypothesis testing
Lecture 23 Hypothesis Testing continued; F-Tests
Lecture 24 Confidence Limits of Spectra, Bootstraps
 purpose of the lecture

generalize the idea of autocorrelation

to multiple time series

 Review of last lecture

autocorrelation
correlations between samples within a
time series
 high degree of short-term correlation

what ever the river was doing yesterday, its probably

doing today, too

because water takes time to drain away

 Neuse River Hydrograph
P S D , ( pc ef sr ) c y c l e / d da iys c h a r g e , c f s
4
x 10
2 A) time series, d(t)

0
0 500 1000 1500 2000 2500 3000 3500 4000
time, days
9
x 10
d(t), cfs

8
6
4
2

2
0
0 0.005 0.01 0.015
time0.02
t, days 0.025 0.03 0.035 0.04 0.045 0.05
frequency, cycles per day
low degree of intermediate-term correlation

what ever the river was doing last month, today it could
be doing something completely different

because storms are so unpredictable

 Neuse River Hydrograph
P S D , ( pc ef sr ) c y c l e / d da iys c h a r g e , c f s
4
x 10
2 A) time series, d(t)

0
0 500 1000 1500 2000 2500 3000 3500 4000
time, days
9
x 10
d(t), cfs

8
6
4
2

2
0
0 0.005 0.01 0.015
time0.02
t, days 0.025 0.03 0.035 0.04 0.045 0.05
frequency, cycles per day
moderate degree of long-term correlation

what ever the river was doing this time last year, its
probably doing today, too

because seasons repeat

 Neuse River Hydrograph
P S D , ( pc ef sr ) c y c l e / d da iys c h a r g e , c f s
4
x 10
2 A) time series, d(t)

0
0 500 1000 1500 2000 2500 3000 3500 4000
time, days
9
x 10
d(t), cfs

8
6
4
2

2
0
0 0.005 0.01 0.015
time0.02
t, days 0.025 0.03 0.035 0.04 0.045 0.05
frequency, cycles per day
 1 day 3 days 30 days
4 4 4
x 10 x 10 x 10
2.5 2.5 2.5

2 2 2

discharge lagged by 30 days

discharge lagged by 1 days

discharge lagged by 3 days

1.5 1.5 1.5

1 1 1

0.5 0.5 0.5

0 0 0
0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5 0 0.5 1 1.5 2 2.5
discharge 4 discharge 4 discharge 4
x 10 x 10 x 10
a u t o c o r r e l a t ioa nu t o c o r r e l a t i o n
Autocorrelation Function

6
x 10

0
-30 -20 -10 0 10 20 30
lag, days
6
x 10
5
0
-5

-3000 -2000 -1000 0 1000 2000 3000

lag, days

1 3 30
formula for covariance
formula for autocorrelation

autocorrelation
at lag (k-1)Δt
autocorrelation similar to convolution
autocorrelation similar to convolution

note difference in sign

autocorrelation in MatLab
 Important Relation #1
autocorrelation is the convolution of a
time series with its time-reversed self
 Important Relationship #2
Fourier Transform of an autocorrelation
is proportional to the
Power Spectral Density of time series
End of Review
 Part 1

correlations between time-series

 scenario

discharge correlated with rain

but discharge is delayed behind

rain

because rain takes time to drain

from the land
 dischagre, m3/s rain, mm/day

time, days
time, days
rain, mm/day

time, days
rain ahead of
discharge
dischagre, m3/s

time, days
rain, mm/day

time, days
shape not
exactly the
dischagre, m3/s

same, either

time, days
treat two time series u and v probabilistically

p.d.f.
p(ui, vi+k-1)

with elements lagged by time

(k-1)Δt

and compute its covariance

this defines the cross-correlation
just a generalization of the auto-correlation

different times in different times in

different time series the same time series
like autocorrelation, similar to convolution
 As with auto-correlation
two important properties

#1: relationship to convolution

#2: relationship to Fourier Transform

 As with auto-correlation
two important properties

#1: relationship to convolution

#2: relationship to Fourier Transform

cross-spectral density
cross-correlation in MatLab
 Part 2

aligning time-series
a simple application of cross-correlation
 central idea

two time series are best aligned

at the lag at which they are most correlated,
which is
the lag at which their cross-correlation is maximum
 two similar time-series, with a time shift
(this is simple “test” or “synthetic” dataset)

u(t)
-1
10 20 30 40 50 60 70 80 90 100

1
v(t)
0

-1
10 20 30 40 50 60 70 80 90 100
 cross-correlate

5
cross-correlation

-5

-20 -10 0 10 20
time
 find maximum
maximum

5
cross-correlation

-5

-20 -10 0 10 20
time

time lag
In MatLab
In MatLab

compute cross-
correlation
In MatLab

compute cross-
correlation

find maximum
In MatLab

compute cross-
correlation

find maximum

compute time lag

align time series with measured lag

u(t)
0

-1
10 20 30 40 50 60 70 80 90 100

v(t+tlag)
1

-1
10 20 30 40 50 60 70 80 90 100
 solar insolation and ground level ozone
(this is a real dataset from West Point NY)
A)
o z o n e , p p bs o la r, W / m 2o z o n e , p p bs o la r, W / m 2

500
0
2 4 6 8 10 12 14
time, days
100
50
0
2 4 6 8 10 12 14
B) time, days

500

0
2 4 6 8 10 12 14
time, days
100
50
0
2 4 6 8 10 12 14
time, days
 solar insolation and ground level ozone
o z o n e , p p bs o la r, W / m 2o z o n e , p p bs o la r, W / m 2

500
0
2 4 6 8 10 12 14
time, days
100
50
0
2 4 6 8 10 12 14
B) time, days

500

0
2 4 6 8 10 12 14
time, days
100
50
0
2 4 6 8 10 12 14
time, days

note time lag

 6 C)
maximum
x 10
4

3
cross-correlation

0
-10 -5 0 5 10
time, hours

time lag
3 hours
 A)
solar radiation, W/m2

500

0
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
time, days
B)
3.00 hour lag original
100 delagged
ozone, ppb

50

0
0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
time, days