Decision and estimation in information processing:

course nr. 8

April 6, 2021

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Part I

Stochastic signals

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Approaches in signal processing

The deterministic approach:

A signal x(t) is a function of time and that’s all.
In processing x(t) we do not take into account the class of
signals the signal belongs to.
The stochastic (random) approach:
A signal x(t) belongs to a class of signals.
The class of signals is formed by the totality of signals
belonging to it.
We are interested in introducing statistical functions that
characterise the whole class.
We use the statistical functions for a better processing of each
signal belonging to the class!

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stochastic signal: definition

def
Stochastic signal = the class of signals!
Mathematically:
ξ : Ω × R −→ R

not def
For ωk ∈ Ω fixed, we have ξ(ωk , t) = ξ (k) (t) = an instance
of the stochastic signal ξ(t).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stochastic signal: definition

def
Stochastic signal = the class of signals!
Mathematically:
ξ : Ω × R −→ R

not def
For ωk ∈ Ω fixed, we have ξ(ωk , t) = ξ (k) (t) = an instance
of the stochastic signal ξ(t).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stochastic signal: definition

def
Stochastic signal = the class of signals!
Mathematically:
ξ : Ω × R −→ R

not def
For ωk ∈ Ω fixed, we have ξ(ωk , t) = ξ (k) (t) = an instance
of the stochastic signal ξ(t).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stochastic signal: definition

def
Stochastic signal = the class of signals!
Mathematically:
ξ : Ω × R −→ R

not def
For ωk ∈ Ω fixed, we have ξ(ωk , t) = ξ (k) (t) = an instance
of the stochastic signal ξ(t).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

For t1 fixed, ξ(t1 ) is a random variable!
ξ(t1 ) : Ω −→ R
We know how to characterise RVs!
The signal’s CDF of order one:
def
Fξ (x1 , t1 ) = P(ξ(t1 ) ≤ x1 )

The PDF of order one:

def ∂Fξ (x1 , t1 )
wξ (x1 , t1 ) =
∂x1

Remark: both CDF and PDF depend on time, because we are

interested in characterising the value of the signal ∀t1 ∈ R
Decision and estimation in information processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

For t1 fixed, ξ(t1 ) is a random variable!
ξ(t1 ) : Ω −→ R
We know how to characterise RVs!
The signal’s CDF of order one:
def
Fξ (x1 , t1 ) = P(ξ(t1 ) ≤ x1 )

The PDF of order one:

def ∂Fξ (x1 , t1 )
wξ (x1 , t1 ) =
∂x1

Remark: both CDF and PDF depend on time, because we are

interested in characterising the value of the signal ∀t1 ∈ R
Decision and estimation in information processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

For t1 fixed, ξ(t1 ) is a random variable!
ξ(t1 ) : Ω −→ R
We know how to characterise RVs!
The signal’s CDF of order one:
def
Fξ (x1 , t1 ) = P(ξ(t1 ) ≤ x1 )

The PDF of order one:

def ∂Fξ (x1 , t1 )
wξ (x1 , t1 ) =
∂x1

Remark: both CDF and PDF depend on time, because we are

interested in characterising the value of the signal ∀t1 ∈ R
Decision and estimation in information processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

For t1 fixed, ξ(t1 ) is a random variable!
ξ(t1 ) : Ω −→ R
We know how to characterise RVs!
The signal’s CDF of order one:
def
Fξ (x1 , t1 ) = P(ξ(t1 ) ≤ x1 )

The PDF of order one:

def ∂Fξ (x1 , t1 )
wξ (x1 , t1 ) =
∂x1

Remark: both CDF and PDF depend on time, because we are

interested in characterising the value of the signal ∀t1 ∈ R
Decision and estimation in information processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

For t1 fixed, ξ(t1 ) is a random variable!
ξ(t1 ) : Ω −→ R
We know how to characterise RVs!
The signal’s CDF of order one:
def
Fξ (x1 , t1 ) = P(ξ(t1 ) ≤ x1 )

The PDF of order one:

def ∂Fξ (x1 , t1 )
wξ (x1 , t1 ) =
∂x1

Remark: both CDF and PDF depend on time, because we are

interested in characterising the value of the signal ∀t1 ∈ R
Decision and estimation in information processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

For t1 fixed, ξ(t1 ) is a random variable!
ξ(t1 ) : Ω −→ R
We know how to characterise RVs!
The signal’s CDF of order one:
def
Fξ (x1 , t1 ) = P(ξ(t1 ) ≤ x1 )

The PDF of order one:

def ∂Fξ (x1 , t1 )
wξ (x1 , t1 ) =
∂x1

Remark: both CDF and PDF depend on time, because we are

interested in characterising the value of the signal ∀t1 ∈ R
Decision and estimation in information processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

For t1 fixed, ξ(t1 ) is a random variable!
ξ(t1 ) : Ω −→ R
We know how to characterise RVs!
The signal’s CDF of order one:
def
Fξ (x1 , t1 ) = P(ξ(t1 ) ≤ x1 )

The PDF of order one:

def ∂Fξ (x1 , t1 )
wξ (x1 , t1 ) =
∂x1

Remark: both CDF and PDF depend on time, because we are

interested in characterising the value of the signal ∀t1 ∈ R
Decision and estimation in information processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

Based on the PDF of order one, one can compute statistical
moments:
The signal’s mean:
Z∞
ξ(t1 ) = x1 wξ (x1 , t1 )dx1
−∞

The squared mean:

Z∞
ξ 2 (t1 ) = x12 wξ (x1 , t1 )dx1
−∞

The signal’s variance:

2
σξ2 (t1 ) = ξ 2 (t1 ) − ξ(t1 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

Based on the PDF of order one, one can compute statistical
moments:
The signal’s mean:
Z∞
ξ(t1 ) = x1 wξ (x1 , t1 )dx1
−∞

The squared mean:

Z∞
ξ 2 (t1 ) = x12 wξ (x1 , t1 )dx1
−∞

The signal’s variance:

2
σξ2 (t1 ) = ξ 2 (t1 ) − ξ(t1 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

Based on the PDF of order one, one can compute statistical
moments:
The signal’s mean:
Z∞
ξ(t1 ) = x1 wξ (x1 , t1 )dx1
−∞

The squared mean:

Z∞
ξ 2 (t1 ) = x12 wξ (x1 , t1 )dx1
−∞

The signal’s variance:

2
σξ2 (t1 ) = ξ 2 (t1 ) − ξ(t1 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

Based on the PDF of order one, one can compute statistical
moments:
The signal’s mean:
Z∞
ξ(t1 ) = x1 wξ (x1 , t1 )dx1
−∞

The squared mean:

Z∞
ξ 2 (t1 ) = x12 wξ (x1 , t1 )dx1
−∞

The signal’s variance:

2
σξ2 (t1 ) = ξ 2 (t1 ) − ξ(t1 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

Based on the PDF of order one, one can compute statistical
moments:
The signal’s mean:
Z∞
ξ(t1 ) = x1 wξ (x1 , t1 )dx1
−∞

The squared mean:

Z∞
ξ 2 (t1 ) = x12 wξ (x1 , t1 )dx1
−∞

The signal’s variance:

2
σξ2 (t1 ) = ξ 2 (t1 ) − ξ(t1 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

Based on the PDF of order one, one can compute statistical
moments:
The signal’s mean:
Z∞
ξ(t1 ) = x1 wξ (x1 , t1 )dx1
−∞

The squared mean:

Z∞
ξ 2 (t1 ) = x12 wξ (x1 , t1 )dx1
−∞

The signal’s variance:

2
σξ2 (t1 ) = ξ 2 (t1 ) − ξ(t1 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order one

Based on the PDF of order one, one can compute statistical
moments:
The signal’s mean:
Z∞
ξ(t1 ) = x1 wξ (x1 , t1 )dx1
−∞

The squared mean:

Z∞
ξ 2 (t1 ) = x12 wξ (x1 , t1 )dx1
−∞

The signal’s variance:

2
σξ2 (t1 ) = ξ 2 (t1 ) − ξ(t1 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two

Used to characterise the joint statistical behaviour of the

values of the signal, at moments t1 and t2 .
The CDF of order two:
def
Fξξ (x1 , x2 , t1 , t2 ) = P ((ξ(t1 ) ≤ x1 ) ∩ (ξ(t2 ) ≤ x2 )) .

The PDF of order two:

def ∂ 2 Fξξ (x1 , x2 , t1 , t2 )

wξξ (x1 , x2 , t1 , t2 ) = ,
∂x1 ∂x2

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two

Used to characterise the joint statistical behaviour of the

values of the signal, at moments t1 and t2 .
The CDF of order two:
def
Fξξ (x1 , x2 , t1 , t2 ) = P ((ξ(t1 ) ≤ x1 ) ∩ (ξ(t2 ) ≤ x2 )) .

The PDF of order two:

def ∂ 2 Fξξ (x1 , x2 , t1 , t2 )

wξξ (x1 , x2 , t1 , t2 ) = ,
∂x1 ∂x2

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two

Used to characterise the joint statistical behaviour of the

values of the signal, at moments t1 and t2 .
The CDF of order two:
def
Fξξ (x1 , x2 , t1 , t2 ) = P ((ξ(t1 ) ≤ x1 ) ∩ (ξ(t2 ) ≤ x2 )) .

The PDF of order two:

def ∂ 2 Fξξ (x1 , x2 , t1 , t2 )

wξξ (x1 , x2 , t1 , t2 ) = ,
∂x1 ∂x2

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two

Used to characterise the joint statistical behaviour of the

values of the signal, at moments t1 and t2 .
The CDF of order two:
def
Fξξ (x1 , x2 , t1 , t2 ) = P ((ξ(t1 ) ≤ x1 ) ∩ (ξ(t2 ) ≤ x2 )) .

The PDF of order two:

def ∂ 2 Fξξ (x1 , x2 , t1 , t2 )

wξξ (x1 , x2 , t1 , t2 ) = ,
∂x1 ∂x2

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two

Used to characterise the joint statistical behaviour of the

values of the signal, at moments t1 and t2 .
The CDF of order two:
def
Fξξ (x1 , x2 , t1 , t2 ) = P ((ξ(t1 ) ≤ x1 ) ∩ (ξ(t2 ) ≤ x2 )) .

The PDF of order two:

def ∂ 2 Fξξ (x1 , x2 , t1 , t2 )

wξξ (x1 , x2 , t1 , t2 ) = ,
∂x1 ∂x2

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two: moments

The autocorrelation function (ACF):
ZZ
def
Rξ (t1 , t2 ) = ξ(t1 )ξ(t2 ) = x1 x2 wξξ (x1 , x2 , t1 , t2 )dx1 dx2
R2

The autocovariance function:

  
def
Kξ (t1 , t2 ) = ξ(t1 ) − ξ(t1 ) ξ(t2 ) − ξ(t2 )
= ξ(t1 )ξ(t2 ) − ξ(t1 ) ξ(t2 )

It can be shown (it has been shown in the context of a pair of

RVs):
Kξ (t1 , t2 ) = Rξ (t1 , t2 ) − ξ(t1 ) ξ(t2 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two: moments

The autocorrelation function (ACF):
ZZ
def
Rξ (t1 , t2 ) = ξ(t1 )ξ(t2 ) = x1 x2 wξξ (x1 , x2 , t1 , t2 )dx1 dx2
R2

The autocovariance function:

  
def
Kξ (t1 , t2 ) = ξ(t1 ) − ξ(t1 ) ξ(t2 ) − ξ(t2 )
= ξ(t1 )ξ(t2 ) − ξ(t1 ) ξ(t2 )

It can be shown (it has been shown in the context of a pair of

RVs):
Kξ (t1 , t2 ) = Rξ (t1 , t2 ) − ξ(t1 ) ξ(t2 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two: moments

The autocorrelation function (ACF):
ZZ
def
Rξ (t1 , t2 ) = ξ(t1 )ξ(t2 ) = x1 x2 wξξ (x1 , x2 , t1 , t2 )dx1 dx2
R2

The autocovariance function:

  
def
Kξ (t1 , t2 ) = ξ(t1 ) − ξ(t1 ) ξ(t2 ) − ξ(t2 )
= ξ(t1 )ξ(t2 ) − ξ(t1 ) ξ(t2 )

It can be shown (it has been shown in the context of a pair of

RVs):
Kξ (t1 , t2 ) = Rξ (t1 , t2 ) − ξ(t1 ) ξ(t2 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two: moments

The autocorrelation function (ACF):
ZZ
def
Rξ (t1 , t2 ) = ξ(t1 )ξ(t2 ) = x1 x2 wξξ (x1 , x2 , t1 , t2 )dx1 dx2
R2

The autocovariance function:

  
def
Kξ (t1 , t2 ) = ξ(t1 ) − ξ(t1 ) ξ(t2 ) − ξ(t2 )
= ξ(t1 )ξ(t2 ) − ξ(t1 ) ξ(t2 )

It can be shown (it has been shown in the context of a pair of

RVs):
Kξ (t1 , t2 ) = Rξ (t1 , t2 ) − ξ(t1 ) ξ(t2 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two: moments

The autocorrelation function (ACF):
ZZ
def
Rξ (t1 , t2 ) = ξ(t1 )ξ(t2 ) = x1 x2 wξξ (x1 , x2 , t1 , t2 )dx1 dx2
R2

The autocovariance function:

  
def
Kξ (t1 , t2 ) = ξ(t1 ) − ξ(t1 ) ξ(t2 ) − ξ(t2 )
= ξ(t1 )ξ(t2 ) − ξ(t1 ) ξ(t2 )

It can be shown (it has been shown in the context of a pair of

RVs):
Kξ (t1 , t2 ) = Rξ (t1 , t2 ) − ξ(t1 ) ξ(t2 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Characterisation of order two: moments

The autocorrelation function (ACF):
ZZ
def
Rξ (t1 , t2 ) = ξ(t1 )ξ(t2 ) = x1 x2 wξξ (x1 , x2 , t1 , t2 )dx1 dx2
R2

The autocovariance function:

  
def
Kξ (t1 , t2 ) = ξ(t1 ) − ξ(t1 ) ξ(t2 ) − ξ(t2 )
= ξ(t1 )ξ(t2 ) − ξ(t1 ) ξ(t2 )

It can be shown (it has been shown in the context of a pair of

RVs):
Kξ (t1 , t2 ) = Rξ (t1 , t2 ) − ξ(t1 ) ξ(t2 )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stationary stochastic signals

Stationarity: a simplifying hypothesis.

Most stochastic signals of interest can be approximated as
stationary (be it on short term).
There are two types of stationarity:
strict-sense stationarity (SSS);
wide-sense stationarity (WSS).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stationary stochastic signals

Stationarity: a simplifying hypothesis.

Most stochastic signals of interest can be approximated as
stationary (be it on short term).
There are two types of stationarity:
strict-sense stationarity (SSS);
wide-sense stationarity (WSS).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stationary stochastic signals

Stationarity: a simplifying hypothesis.

Most stochastic signals of interest can be approximated as
stationary (be it on short term).
There are two types of stationarity:
strict-sense stationarity (SSS);
wide-sense stationarity (WSS).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stationary stochastic signals

Stationarity: a simplifying hypothesis.

Most stochastic signals of interest can be approximated as
stationary (be it on short term).
There are two types of stationarity:
strict-sense stationarity (SSS);
wide-sense stationarity (WSS).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Stationary stochastic signals

Stationarity: a simplifying hypothesis.

Most stochastic signals of interest can be approximated as
stationary (be it on short term).
There are two types of stationarity:
strict-sense stationarity (SSS);
wide-sense stationarity (WSS).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

Definition: Definition: A signal is said to be strict-sense

stationary (of order n = 2) if its PDF of order two is invariant
to any translation in time:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 + τ, t2 + τ ), ∀τ ∈ R (1)

Property: SSS of order two implies, SSS of order one:

wξ (x1 , t1 ) = wξ (x1 , t1 + τ ), ∀τ ∈ R (2)

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

Definition: Definition: A signal is said to be strict-sense

stationary (of order n = 2) if its PDF of order two is invariant
to any translation in time:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 + τ, t2 + τ ), ∀τ ∈ R (1)

Property: SSS of order two implies, SSS of order one:

wξ (x1 , t1 ) = wξ (x1 , t1 + τ ), ∀τ ∈ R (2)

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

Definition: Definition: A signal is said to be strict-sense

stationary (of order n = 2) if its PDF of order two is invariant
to any translation in time:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 + τ, t2 + τ ), ∀τ ∈ R (1)

Property: SSS of order two implies, SSS of order one:

wξ (x1 , t1 ) = wξ (x1 , t1 + τ ), ∀τ ∈ R (2)

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

Definition: Definition: A signal is said to be strict-sense

stationary (of order n = 2) if its PDF of order two is invariant
to any translation in time:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 + τ, t2 + τ ), ∀τ ∈ R (1)

Property: SSS of order two implies, SSS of order one:

wξ (x1 , t1 ) = wξ (x1 , t1 + τ ), ∀τ ∈ R (2)

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

Definition: Definition: A signal is said to be strict-sense

stationary (of order n = 2) if its PDF of order two is invariant
to any translation in time:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 + τ, t2 + τ ), ∀τ ∈ R (1)

Property: SSS of order two implies, SSS of order one:

wξ (x1 , t1 ) = wξ (x1 , t1 + τ ), ∀τ ∈ R (2)

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t1 in equation (2), we get:

wξ (x1 , t1 ) = wξ (x1 , 0) = wξ (x1 )

The PDF of order one does not depend on time!

Consequences:

ξ(t1 ) = ξ,
ξ 2 (t1 ) = ξ 2 ,
σξ2 (t1 ) = σξ2 .

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t1 in equation (2), we get:

wξ (x1 , t1 ) = wξ (x1 , 0) = wξ (x1 )

The PDF of order one does not depend on time!

Consequences:

ξ(t1 ) = ξ,
ξ 2 (t1 ) = ξ 2 ,
σξ2 (t1 ) = σξ2 .

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t1 in equation (2), we get:

wξ (x1 , t1 ) = wξ (x1 , 0) = wξ (x1 )

The PDF of order one does not depend on time!

Consequences:

ξ(t1 ) = ξ,
ξ 2 (t1 ) = ξ 2 ,
σξ2 (t1 ) = σξ2 .

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t1 in equation (2), we get:

wξ (x1 , t1 ) = wξ (x1 , 0) = wξ (x1 )

The PDF of order one does not depend on time!

Consequences:

ξ(t1 ) = ξ,
ξ 2 (t1 ) = ξ 2 ,
σξ2 (t1 ) = σξ2 .

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t1 in equation (2), we get:

wξ (x1 , t1 ) = wξ (x1 , 0) = wξ (x1 )

The PDF of order one does not depend on time!

Consequences:

ξ(t1 ) = ξ,
ξ 2 (t1 ) = ξ 2 ,
σξ2 (t1 ) = σξ2 .

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t2 in ecuation (1), we get:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 −t2 , 0) = wξξ (x1 , x2 , t1 −t2 ), ∀τ ∈ R

The signal’s PDF of order two does not depend on the two
moments of time as such, but only on their difference!
Consequences:

Rξ (t1 , t2 ) = Rξ (t1 − t2 )
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

with
2
Kξ (τ ) = Rξ (τ ) − ξ

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t2 in ecuation (1), we get:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 −t2 , 0) = wξξ (x1 , x2 , t1 −t2 ), ∀τ ∈ R

The signal’s PDF of order two does not depend on the two
moments of time as such, but only on their difference!
Consequences:

Rξ (t1 , t2 ) = Rξ (t1 − t2 )
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

with
2
Kξ (τ ) = Rξ (τ ) − ξ

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t2 in ecuation (1), we get:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 −t2 , 0) = wξξ (x1 , x2 , t1 −t2 ), ∀τ ∈ R

The signal’s PDF of order two does not depend on the two
moments of time as such, but only on their difference!
Consequences:

Rξ (t1 , t2 ) = Rξ (t1 − t2 )
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

with
2
Kξ (τ ) = Rξ (τ ) − ξ

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t2 in ecuation (1), we get:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 −t2 , 0) = wξξ (x1 , x2 , t1 −t2 ), ∀τ ∈ R

The signal’s PDF of order two does not depend on the two
moments of time as such, but only on their difference!
Consequences:

Rξ (t1 , t2 ) = Rξ (t1 − t2 )
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

with
2
Kξ (τ ) = Rξ (τ ) − ξ

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t2 in ecuation (1), we get:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 −t2 , 0) = wξξ (x1 , x2 , t1 −t2 ), ∀τ ∈ R

The signal’s PDF of order two does not depend on the two
moments of time as such, but only on their difference!
Consequences:

Rξ (t1 , t2 ) = Rξ (t1 − t2 )
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

with
2
Kξ (τ ) = Rξ (τ ) − ξ

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Strict-sense stationarity

If we choose τ = −t2 in ecuation (1), we get:

wξξ (x1 , x2 , t1 , t2 ) = wξξ (x1 , x2 , t1 −t2 , 0) = wξξ (x1 , x2 , t1 −t2 ), ∀τ ∈ R

The signal’s PDF of order two does not depend on the two
moments of time as such, but only on their difference!
Consequences:

Rξ (t1 , t2 ) = Rξ (t1 − t2 )
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

with
2
Kξ (τ ) = Rξ (τ ) − ξ

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Wide-sense stationarity
A signal is said to be wide-sense stationary if:
its mean is time-independent
ξ(t1 ) = ξ

its autocorrelation function depends only on the difference

between its arguments:
Rξ (t1 , t2 ) = Rξ (t1 − t2 )

If ξ(t) is WSS:
ξ 2 (t1 ) = ξ 2
σξ (t1 ) = σξ
Kξ (t1 , t2 ) = Kξ (t1 − t2 )

Obviously, SSS =⇒ SSL, whereas

Decision WSS 6 in information
=⇒
and estimation SSS processing: course nr. 8
 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Let ξ(t) be a stationary signal. Its autocorrelation function is:

Rξ (τ ) = ξ(t)ξ(t + τ ), ∀t ∈ R

Properties of the ACF:

The ACF is even:

Rξ (τ ) = Rξ (−τ ) ∀τ ∈ R.

Proof. By taking t = t1 − τ , we have:

Rξ (τ ) = ξ(t1 − τ )ξ(t1 ) = Rξ (−τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Let ξ(t) be a stationary signal. Its autocorrelation function is:

Rξ (τ ) = ξ(t)ξ(t + τ ), ∀t ∈ R

Properties of the ACF:

The ACF is even:

Rξ (τ ) = Rξ (−τ ) ∀τ ∈ R.

Proof. By taking t = t1 − τ , we have:

Rξ (τ ) = ξ(t1 − τ )ξ(t1 ) = Rξ (−τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Let ξ(t) be a stationary signal. Its autocorrelation function is:

Rξ (τ ) = ξ(t)ξ(t + τ ), ∀t ∈ R

Properties of the ACF:

The ACF is even:

Rξ (τ ) = Rξ (−τ ) ∀τ ∈ R.

Proof. By taking t = t1 − τ , we have:

Rξ (τ ) = ξ(t1 − τ )ξ(t1 ) = Rξ (−τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Let ξ(t) be a stationary signal. Its autocorrelation function is:

Rξ (τ ) = ξ(t)ξ(t + τ ), ∀t ∈ R

Properties of the ACF:

The ACF is even:

Rξ (τ ) = Rξ (−τ ) ∀τ ∈ R.

Proof. By taking t = t1 − τ , we have:

Rξ (τ ) = ξ(t1 − τ )ξ(t1 ) = Rξ (−τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Let ξ(t) be a stationary signal. Its autocorrelation function is:

Rξ (τ ) = ξ(t)ξ(t + τ ), ∀t ∈ R

Properties of the ACF:

The ACF is even:

Rξ (τ ) = Rξ (−τ ) ∀τ ∈ R.

Proof. By taking t = t1 − τ , we have:

Rξ (τ ) = ξ(t1 − τ )ξ(t1 ) = Rξ (−τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Let ξ(t) be a stationary signal. Its autocorrelation function is:

Rξ (τ ) = ξ(t)ξ(t + τ ), ∀t ∈ R

Properties of the ACF:

The ACF is even:

Rξ (τ ) = Rξ (−τ ) ∀τ ∈ R.

Proof. By taking t = t1 − τ , we have:

Rξ (τ ) = ξ(t1 − τ )ξ(t1 ) = Rξ (−τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

The ACF reaches its peak in the origin:

Rξ (0) ≥ |Rξ (τ )| ∀τ ∈ R

Proof. We compute the squared mean of the random

variable ξ(t) − ξ(t + τ ):
(ξ(t) − ξ(t + τ ))2 = ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
= ξ 2 (t) − 2ξ(t)ξ(t + τ ) + ξ 2 (t + τ )
| {z } | {z } | {z }
Rξ (0) Rξ (τ ) Rξ (0)
= 2 (Rξ (0) − Rξ (τ )) ≥ 0 =⇒ Rξ (0) ≥ Rξ (τ ) We repeat for
RV ξ(t) + ξ(t + τ ) =⇒ Rξ (0) ≥ −Rξ (τ ).

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Assuming the signal has no deterministic or periodical

components, the valus of the ACF at infinity equals the square
of the mean:
2
Rξ (∞) = ξ
Proof. When τ → ∞, ξ(t) and ξ(t + τ ) become
uncorrelated, even though they beong to the same signal.
Thus:
2
Rξ (tau) = ξ(t)ξ(t + τ ) = ξ(t) ξ(t + τ ) = ξ .
τ →∞

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Assuming the signal has no deterministic or periodical

components, the valus of the ACF at infinity equals the square
of the mean:
2
Rξ (∞) = ξ
Proof. When τ → ∞, ξ(t) and ξ(t + τ ) become
uncorrelated, even though they beong to the same signal.
Thus:
2
Rξ (tau) = ξ(t)ξ(t + τ ) = ξ(t) ξ(t + τ ) = ξ .
τ →∞

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Assuming the signal has no deterministic or periodical

components, the valus of the ACF at infinity equals the square
of the mean:
2
Rξ (∞) = ξ
Proof. When τ → ∞, ξ(t) and ξ(t + τ ) become
uncorrelated, even though they beong to the same signal.
Thus:
2
Rξ (tau) = ξ(t)ξ(t + τ ) = ξ(t) ξ(t + τ ) = ξ .
τ →∞

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Assuming the signal has no deterministic or periodical

components, the valus of the ACF at infinity equals the square
of the mean:
2
Rξ (∞) = ξ
Proof. When τ → ∞, ξ(t) and ξ(t + τ ) become
uncorrelated, even though they beong to the same signal.
Thus:
2
Rξ (tau) = ξ(t)ξ(t + τ ) = ξ(t) ξ(t + τ ) = ξ .
τ →∞

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

Assuming the signal has no deterministic or periodical

components, the valus of the ACF at infinity equals the square
of the mean:
2
Rξ (∞) = ξ
Proof. When τ → ∞, ξ(t) and ξ(t + τ ) become
uncorrelated, even though they beong to the same signal.
Thus:
2
Rξ (tau) = ξ(t)ξ(t + τ ) = ξ(t) ξ(t + τ ) = ξ .
τ →∞

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

If the signal is periodical, its ACF is periodical too, with the

same period.

ξ(t) = ξ(t + T ) ∀t ∈ R =⇒ Rξ (τ ) = Rξ (τ + T ) ∀τ ∈ R.

Proof.

Rξ (τ +T ) = ξ(t) ξ(t + τ + T ) = ξ(t + T )ξ(t + τ + T ) = Rξ (τ ).

|{z}
ξ(t+T )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

If the signal is periodical, its ACF is periodical too, with the

same period.

ξ(t) = ξ(t + T ) ∀t ∈ R =⇒ Rξ (τ ) = Rξ (τ + T ) ∀τ ∈ R.

Proof.

Rξ (τ +T ) = ξ(t) ξ(t + τ + T ) = ξ(t + T )ξ(t + τ + T ) = Rξ (τ ).

|{z}
ξ(t+T )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

If the signal is periodical, its ACF is periodical too, with the

same period.

ξ(t) = ξ(t + T ) ∀t ∈ R =⇒ Rξ (τ ) = Rξ (τ + T ) ∀τ ∈ R.

Proof.

Rξ (τ +T ) = ξ(t) ξ(t + τ + T ) = ξ(t + T )ξ(t + τ + T ) = Rξ (τ ).

|{z}
ξ(t+T )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

Properties of the autocorrelation function

If the signal is periodical, its ACF is periodical too, with the

same period.

ξ(t) = ξ(t + T ) ∀t ∈ R =⇒ Rξ (τ ) = Rξ (τ + T ) ∀τ ∈ R.

Proof.

Rξ (τ +T ) = ξ(t) ξ(t + τ + T ) = ξ(t + T )ξ(t + τ + T ) = Rξ (τ ).

|{z}
ξ(t+T )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

The intercorrelation function

Let ξ(t) and η(t) be two stationary signals.
The intercorrelation function:
Rξη (τ ) = ξ(t)η(t + τ ) ∀t ∈ R

The intercovariance function:

  
Kξη (τ ) = ξ(t) − ξ(t) η(t + τ ) − η(t + τ ) = Rξη (τ )−ξ η, ∀t

We observe that:
Rξη (τ ) = Rηξ (−τ )
Kξη (τ ) = Kηξ (−τ )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

The intercorrelation function

Let ξ(t) and η(t) be two stationary signals.
The intercorrelation function:
Rξη (τ ) = ξ(t)η(t + τ ) ∀t ∈ R

The intercovariance function:

  
Kξη (τ ) = ξ(t) − ξ(t) η(t + τ ) − η(t + τ ) = Rξη (τ )−ξ η, ∀t

We observe that:
Rξη (τ ) = Rηξ (−τ )
Kξη (τ ) = Kηξ (−τ )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

The intercorrelation function

Let ξ(t) and η(t) be two stationary signals.
The intercorrelation function:
Rξη (τ ) = ξ(t)η(t + τ ) ∀t ∈ R

The intercovariance function:

  
Kξη (τ ) = ξ(t) − ξ(t) η(t + τ ) − η(t + τ ) = Rξη (τ )−ξ η, ∀t

We observe that:
Rξη (τ ) = Rηξ (−τ )
Kξη (τ ) = Kηξ (−τ )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

The intercorrelation function

Let ξ(t) and η(t) be two stationary signals.
The intercorrelation function:
Rξη (τ ) = ξ(t)η(t + τ ) ∀t ∈ R

The intercovariance function:

  
Kξη (τ ) = ξ(t) − ξ(t) η(t + τ ) − η(t + τ ) = Rξη (τ )−ξ η, ∀t

We observe that:
Rξη (τ ) = Rηξ (−τ )
Kξη (τ ) = Kηξ (−τ )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

The intercorrelation function

Let ξ(t) and η(t) be two stationary signals.
The intercorrelation function:
Rξη (τ ) = ξ(t)η(t + τ ) ∀t ∈ R

The intercovariance function:

  
Kξη (τ ) = ξ(t) − ξ(t) η(t + τ ) − η(t + τ ) = Rξη (τ )−ξ η, ∀t

We observe that:
Rξη (τ ) = Rηξ (−τ )
Kξη (τ ) = Kηξ (−τ )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

The intercorrelation function

Let ξ(t) and η(t) be two stationary signals.
The intercorrelation function:
Rξη (τ ) = ξ(t)η(t + τ ) ∀t ∈ R

The intercovariance function:

  
Kξη (τ ) = ξ(t) − ξ(t) η(t + τ ) − η(t + τ ) = Rξη (τ )−ξ η, ∀t

We observe that:
Rξη (τ ) = Rηξ (−τ )
Kξη (τ ) = Kηξ (−τ )

Decision and estimation in information processing: course nr. 8

 Introduction
Statistical characterisation of stochastic signals
Stationary stochastic signals
Properties of the autocorrelation function
The intercorrelation function

The intercorrelation function

Let ξ(t) and η(t) be two stationary signals.
The intercorrelation function:
Rξη (τ ) = ξ(t)η(t + τ ) ∀t ∈ R

The intercovariance function:

  
Kξη (τ ) = ξ(t) − ξ(t) η(t + τ ) − η(t + τ ) = Rξη (τ )−ξ η, ∀t

We observe that:
Rξη (τ ) = Rηξ (−τ )
Kξη (τ ) = Kηξ (−τ )

Decision and estimation in information processing: course nr. 8