ECTE301: Digital Signal Processing: Random Signals and Correlation Sequences

ECTE301: Digital Signal Processing
Lecture 5
Random Signals and Correlation Sequences
Part A’s contents
Textbook
Lecture Topic Practice
Chapter
Lab 1
L1 Introduction to DSP 1, 2.1, 2.2
Tutorial 1
Lab 2
L2 Convolution and LTI Systems 2.3, 2.4
Tutorial 2
The z-transform: Tutorial 3
L3 3.1, 3.2, 3.3
Forward Transform and Properties Lab 3
The z-transform:
L4 3.4, 3.5 Tutorial 4
Inverse Transform and Applications
2.4, 2.5,
Random signals and Tutorial 4
L5 2.6, 12.1
Cross-correlation Sequences Report
L6 Revision of Weeks 1 to 6 - -
Digital Signal Processing S. L. Phung & A. Bouzerdoum 2/64

Lecture 5’s sequence
5.1 Random signals
5.2 Correlation sequences
5.3 LTI system in correlation domain
5.4 Applications of correlation

Lecture 5's objectives
 Understand random signals, CDF, PDF.
 Define the cross-correlation and auto-correlation sequences.
 Study input-output relationships for LTI systems in ‘correlation domain’.
 Explore real-world signal processing applications of correlation (demo).

5.1 Random signals
 Most signals that occur in nature, e.g. speech, music, brain waves,
seismic signals, and noise, are random signals.
 Random signals cannot be predicted ahead of time, and do not have a

deterministic form.
 Consider a random variable X produced by a random process.
 Each observed value of X is called a realization of the random variable.

The observed value is denoted as x.

Random signals
 Example 5.1: At a given time n, a coin is tossed. The value of the
random variable x(n) is
● x(n) = 1 if head,
● x(n) = 0 if tail.
Two realizations of a random process

obtained by tossing a coin.
Random signals
Two different realizations of the phrase ‘DSP’.

5.1.1 CDF and PDF
 A random variable X is defined by its cumulative distribution function
(CDF):
 If X is a continuous variable, then its probability density function (PDF)

is defined as
 Therefore, the CDF can be expressed as

CDF and PDF
 Example 5.2: Consider a random variable X with mean 2 and standard
deviation 1. There are 2000 realizations of X.

CDF and PDF
Random variable X: mean 2, std 1, and 2000 realizations.

CDF and PDF
Random variable X: mean 2, std 1, and 2000 realizations.

CDF and PDF
P(x0)
p(x0)
x0
CDF is the area under the curve (blue shaded region) of PDF.
CDF and PDF
 Properties of the probability density function:
 Properties of the cumulative probability function:

5.1.2 Moments, Mean, and Variance
 The expectation operator is denotes as E[.].
 The expected value of a random variable X is defined as
 The expectation is a linear operator, i.e.

Moments
 A random variable is completely characterized by all its moments.
 The r-th moment is defined as
 For example,

Moments: Application in Image Processing
-100
-50
0
(a) Original image
50
100
150
-200 -100 0 100

(b) Distorted image (c) A normalized image of (a) and (b)
under affine transform using image moments (video)
Mean and Variance
 The first-order moment, denoted by μ, is the mean or expected value:
 The second-order moment is the mean-square value:
 The variance of random variable X is defined as
 σ is the standard deviation of X.

5.1.3 Common random signals
We study two common random signals with
 Uniform distribution
 Normal distribution

a) Uniform distribution
 A random variable X is uniformly distributed in interval [a, b] if its PDF
has the form
 For the uniform distribution, the mean and the variance are

a) Uniform distribution
-1 1 3 -1 1 3
PDF and CDF of a uniform distribution in [-1, 3]

b) Normal distribution
 A random variable X is normally distributed if its PDF has the form:
 The normal distribution is also called the Gaussian distribution.
 For the normal distribution, the mean and the variance are
 Normal distribution was illustrated in Example 5.2.

b) Normal distribution
Carl Friedrich Gauss (1777-1855)

French Mathematician
https://en.wikipedia.org/wiki/Carl_Friedrich_Gauss

Normal distribution
PDF and CDF of a normal distribution

5.1.4 Independent random variables
 Two random variables X and Y are statistically independent if their joint
PDF satisfies:
 Informally, the probability of observing {X = x AND Y = y} is equal to:

{probability of observing X = x } ×{probability of observing Y = y}.
 The variance of the sum of two independent random variables is the

sum of their variances:

Uncorrelated random variables
 Two random variable X and Y are said to be uncorrelated if
 That is,
{expected value of XY} = {expected value of X} × {expected value of Y}.
 If two variables are independent, then they are uncorrelated.
 If two variables are uncorrelated, they are not necessarily independent.

5.1.5 Random sequences
 For a random sequence {X(n)}, each sample X(n) is considered as a
random variable.
 Example 5.3: Consider the following random sequence {X(n)}
X(n) = 2 cos(2πfn) + w(n), where f = 0.05 and w(n) is Gaussian noise.
video

Random sequences
 A random sequence is characterized by its PDF and CDF:
 For a random sequence, the mean, mean-square, and variance are time-
varying.

5.1 Random signals

 The cross-correlation of two energy signals x(n) and y(n) is defined as
 The auto-correlation sequence of x(n) is defined as
 Index l is the time-shift (or lag) parameter.
 The difference between the first and second index is equal to the lag:

Correlation sequences
 Following are all valid expressions of the cross-correlation:

 Example 5.4: Cross-correlation of two sequences.
rxy(5) = 5
ryx(l) = rxy(-l)

 Example 5.5: Calculate the auto-correlation of x(n):
Answer:
After variable substitution &

overlap processing

Auto-correlation of signal
Properties of correlation sequences
 Important properties of the correlation sequences:
used many
times later
 The energy of a sequence:

Normalized correlation sequences
 The correlation sequences are often normalized to the range [-1, 1].
 The normalized correlation sequences are defined as

Correlation of random signals
 The cross-correlation of two real random signals X(n) and Y(n) is
 The auto-correlation of X(n) is
 For stationary random signals, the correlation depends only on the lag l ,
where l = (m – n):

Covariance and auto-covariance sequences
 The covariance and auto-covariance sequences are given by
 Therefore, the variance of a stationary random process is
 The mean-square value is given by
 The average power of a wide-sense stationary random signal is given by

 For WSS signals:
 Other useful properties:
 For a WSS signal with no periodic components:

 Example 5.6: Find the mean, auto-correlation, auto-covariance,
variance, and average power of the sinusoidal random signal
A and ω0 are constants. Φ is a uniformly distributed RV in [0, π].
Answer:
 We have:
 Mean:
 Auto-correlation:

 X(n) is a WSS signal because rxx(m, n) depends only on m-n.
 Let l = m-n, the auto-correlation sequence is
 The auto-covariance sequence is
 Thus, the variance and average power of the signal are

5.1 Random signals

 Consider a LTI system with impulse response h(n).
h(n) Notes
 The following properties can be obtained.
h(l)
h(l)
rhh(l)

Input-output correlation sequences
 Consider a system described by a difference equation:
 The following properties are obtained:

 Example 5.7: Find ryx(l) and ryy(l) for the following system
Answer:
 The impulse response is
 Applying the correlation properties:
 These correlation sequences satisfy the different equations:

Input-output correlations of a LTI system driven by white noise

5.1 Random signals

Correlation has applications in many areas, including
 pattern matching,
 filtering, prediction, estimation,
 signal modeling and synthesis,
 spectrum estimation,
 detection of hidden periodicities,
 detection of signals buried in noise,
 time-delay (arrival time) estimation, and
 system identification.

Applications of correlation
5.4.1 Detection of Hidden Periodicities
5.4.2 Detection of Signal in Noise
5.4.3 System Identification
5.4.4 Image matching

5.4.1 Detection of hidden periodicities
 Consider a periodic signal s(n) with a period of N:
 The signal is mixed with noise, represented by zero-mean Gaussian

random signal ω(n).
 We can observe only the noisy signal x(n) instead of s(n):
How to find period N from observed signal x(n)?

Detection of hidden periodicities
 We calculate the auto-correlation sequence of x(n):
 Because ω(n) is a zero-mean white noise signal:
 Therefore,
 Except for the sample at l = 0, rxx(l) has a period of N.

 Example 5.8:
It is difficult to estimate the period of noisy signal: x(n) = s(n) + noise

It is easier to estimate the period from auto-correlation rxx(l): N = 40

MATLAB code
f = 1/40; N = floor(1/f); L = 4* N;
n = 0:(L-1);
s = 1.25 * cos(2 * pi * f * n);
w = 1 * randn(1, L);
x = s + w;
[rxx, l] = xcorr(x);
[rss, l] = xcorr(s);
figure(1)
subplot(1,2,1); plot(n, s);
subplot(1,2,2); plot(n, x, 'r’);
figure(2)
subplot(1,2,1); plot(l, rss, ‘b’);
subplot(1,2,2); plot(l, rxx, ‘r’);

5.4.2 Detection of signal in noise
 An original signal s(n) is transmitted via a communication channel.
 The received signal x(n) is a delayed and noise-corrupted version of s(n):
 The task is to detect if s(n) is present in x(n).
 By the correlation property, we have:

Detection of signal in noise
Correlation-based signal detector

Matched filter
 The correlation detector can be implemented using the filter with the
following impulse response:
 The filter's output at time (L-1) :
Matched filter detector

Matched filter
 Example 5.9: Detecting a signal using a matched filter.

5.4.3 System identification
 How to find the impulse response of an unknown LTI system?
Approach:
 Give the system a white-noise input signal x(n).
 Measure the system's output y(n).
 Calculate the ryx(l). It is equal to h(l).

System identification
System identification by cross-correlation

System identification
 Example 5.10: Consider an unknown LTI system with

5.4.4 Image matching – Face detection
 Example 5.11: Find the location of an object (e.g. a face) in an image.

 A common approach is to scan every rectangular region of the image,
and determine if each region resembles a face.
Face detection
First, build a face

template, by averaging
14,000 faces.

Face detection
A rectangular region and the face template can be compared

using their correlation coefficient (demo video)
Lecture 5’s summary
To revise for this lecture:
 How to compute PDF, CDF, moments, mean, mean-square, variance,

power of a random variable.
 How to compute auto-correlation and cross-correlation sequences.
 How to compute covariance sequences of a random signal.
 How to use correlation sequences in applications.
 Practice: Tutorial 4, Lab 3.

ECTE301: Digital Signal Processing: Random Signals and Correlation Sequences

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ECTE301: Digital Signal Processing: Random Signals and Correlation Sequences

Uploaded by

Copyright:

Available Formats

ECTE301: Digital Signal Processing

Digital Signal Processing S. L. Phung & A. Bouzerdoum 2/64

5.1 Random signals

5.2 Correlation sequences

5.3 LTI system in correlation domain

5.4 Applications of correlation

Digital Signal Processing S. L. Phung & A. Bouzerdoum 3/64

 Understand random signals, CDF, PDF.

 Define the cross-correlation and auto-correlation sequences.

 Study input-output relationships for LTI systems in ‘correlation domain’.

 Explore real-world signal processing applications of correlation (demo).

Digital Signal Processing S. L. Phung & A. Bouzerdoum 4/64

 Random signals cannot be predicted ahead of time, and do not have a

 Consider a random variable X produced by a random process.

 Each observed value of X is called a realization of the random variable.

Digital Signal Processing S. L. Phung & A. Bouzerdoum 5/64

Two realizations of a random process

Two different realizations of the phrase ‘DSP’.

 If X is a continuous variable, then its probability density function (PDF)

 Therefore, the CDF can be expressed as

Digital Signal Processing S. L. Phung & A. Bouzerdoum 8/64

Digital Signal Processing S. L. Phung & A. Bouzerdoum 9/64

Random variable X: mean 2, std 1, and 2000 realizations.

Random variable X: mean 2, std 1, and 2000 realizations.

 Properties of the cumulative probability function:

Digital Signal Processing S. L. Phung & A. Bouzerdoum 13/64

 The expectation operator is denotes as E[.].

 The expected value of a random variable X is defined as

 The expectation is a linear operator, i.e.

Digital Signal Processing S. L. Phung & A. Bouzerdoum 14/64

 A random variable is completely characterized by all its moments.

 The r-th moment is defined as

Digital Signal Processing S. L. Phung & A. Bouzerdoum 15/64

-200 -100 0 100

 The second-order moment is the mean-square value:

 The variance of random variable X is defined as

 σ is the standard deviation of X.

Digital Signal Processing S. L. Phung & A. Bouzerdoum 17/64

We study two common random signals with

Digital Signal Processing S. L. Phung & A. Bouzerdoum 18/64

Digital Signal Processing S. L. Phung & A. Bouzerdoum 19/64

PDF and CDF of a uniform distribution in [-1, 3]

Digital Signal Processing S. L. Phung & A. Bouzerdoum 20/64

 A random variable X is normally distributed if its PDF has the form:

 The normal distribution is also called the Gaussian distribution.

 Normal distribution was illustrated in Example 5.2.

Digital Signal Processing S. L. Phung & A. Bouzerdoum 21/64

Carl Friedrich Gauss (1777-1855)

Digital Signal Processing S. L. Phung & A. Bouzerdoum 22/64

PDF and CDF of a normal distribution

Digital Signal Processing S. L. Phung & A. Bouzerdoum 23/64

 Informally, the probability of observing {X = x AND Y = y} is equal to:

 The variance of the sum of two independent random variables is the

Digital Signal Processing S. L. Phung & A. Bouzerdoum 24/64

 If two variables are independent, then they are uncorrelated.

 If two variables are uncorrelated, they are not necessarily independent.

Digital Signal Processing S. L. Phung & A. Bouzerdoum 25/64

Digital Signal Processing S. L. Phung & A. Bouzerdoum 26/64

Digital Signal Processing S. L. Phung & A. Bouzerdoum 27/64

5.1 Random signals

5.2 Correlation sequences

5.3 LTI system in correlation domain

5.4 Applications of correlation