Comm ch02 Random en PDF

Principles of Communications
Meixia Tao
Dept. of Electronic Engineering

Shanghai Jiao Tong University
Chapter 2: Signal, Random Process, and Spectra
Selected from Chapter 2.1-2.6, Chapter 5.1-5.3 of Fundamentals
of Communications Systems, Pearson Prentice Hall 2005, by
Proakis & Salehi
1
Signal and Noise in
Communication Systems
Information Output
Source Transmitter Channel Receiver Signal
uncertain Noise
Meixia Tao @ SJTU 2

Topics to be Covered
2.1.Signals
2.2.Review of probability and random variables
2.3.Random Processes: basic concepts
2.4.Guassian and White Processes
Meixia Tao @ SJTU 3

What is Signal?
 In communication systems, a signal is any function that
carries information. Also called information bearing signal
Meixia Tao @ SJTU 4

Classification of Signals (1/4)
 Continuous-time signal vs. discrete-time signal
 Continuous valued signal vs. discrete-valued signal
o Continuous-time and continuous valued: analog signal

o Discrete-time and discrete valued: digital signal
o Discrete-time and continuous valued: sampled signal
o Continuous-time and discrete valued: quantized signal
Meixia Tao @ SJTU 5

Meixia Tao @ SJTU 6

 Deterministic signal vs. random signal
Meixia Tao @ SJTU 7

 Energy signal vs. power signal
∞ T /2
 Energy
= ∫
= ∫
2
Ex x (t ) dt lim x(t ) 2 dt
T →∞
−∞ −T /2
T /2
1
T →∞ T ∫
 Power Px = lim x (t ) 2
dt
−T /2
 A signal is an energy signal iff Ex is finite

 A signal is a power signal iff Px is finite
Meixia Tao @ SJTU 8

Fourier Transform
Meixia Tao @ SJTU 9

2.1.Signals
Meixia Tao @ SJTU 10

Example 1
 Consider a binary communication system
P(0) = 0.3 P01 = P(receive 1 | sent 0) = 0.01

P00 = P(receive 0 | sent 0) = 1- P01 = 0.99
P10 = P(receive 0 | sent 1) = 0.1
P(1) = 0.7
P11 = P(receive 1 | sent 1) = 1- P10 = 0.9
 What is the probability that the output of this channel is 1?

 Assuming that we have observed a 1 at the output, what is
the probability that the input to the channel was a 1?

Conditional Probability
 Consider two events A and B

 Conditional probability P(A|B)
 Joint probability
P( AB ) = P( B ) P( A | B ) = P( A) P( B | A)
 A and B are said statistically independent iff

Law of Total Probability
 Let be mutually exclusive events
with
 For any event B, we have

Bayes’ Theorem
 Let be mutually exclusive events such
that and B is an arbitrary event with nonzero
probability. Then
 Example 1 on slide 11?

Random Variables (r.v.)
 A r.v. is a mapping from the sample space S to the set of
real numbers.
 A r.v. may be
 Discrete-valued: range is finite (e.g. {0,1}), or countable
infinite (e.g. {1,2,3 …})
 Continuous-valued: range is uncountable infinite (e.g. )

 The Cumulative distribution function (CDF of a r.v. X, is
∆
FX ( x ) = P ( X ≤ x )
 Key properties of CDF

1. 0 ≤ FX(x) ≤ 1 with
2. FX(x) is a non-decreasing function of x
3. F (x1 < X ≤ x2) = FX(x2) − FX(x1)
16
Probability Density Function (PDF)
 The PDF, of a r.v. X, is defined as
∆
f X ( x ) = FX ( x ) FX ( x ) = ∫ f X ( y )dy
d x
or
dx −∞
 Key properties of PDF

∞
1. p X ( x) ≥ 0 2. ∫
−∞
p X ( x)dx = 1
x2
3. P ( x1 < X ≤ x2 ) = PX ( x2 ) − PX ( x1 ) = ∫
x1
p X ( x)dx
Area = P( x1 < X ≤ x2 )
FX ( x ) f X (x )
1 1
Area = f X ( x )dx
P( x1 < X ≤ x2 )
Xmin 0 x1 x2 Xmax x Xmin 0 x1 x2 x x+dx Xmax x

Some useful distribution:
Bernoulli Distribution
 A discrete r.v taking two possible values, X = 1 or X = 0.
with probability mass function (pmf)
 Often used to model binary data

Binomial Distribution
 A discrete r.v. taking the sum of n-independent Bernoulli
r.v. , i.e.
where
 The PMF is given by
where
 That is, the probability that Y = k is the probability that k of

the Xi are equal to 1 and n-k are equal to 0
19
Example
 Suppose that we transmit a 31-bit long sequence with
error correction capability up to 3 bit errors
 If the probability of a bit error is p = 0.001, what is the
probability that this sequence is received in errors?
 If no error correction is used, the error probability is

Example
 Assume 10,000 bits are transmitted over a channel in
which the error probability is
 What is the probability that the total number of errors is
less than 3?
 Solution:
 n=10,000, p=0.001


Uniform Distribution
 A continuous r.v. taking values between a and b with equal
probabilities
 The probability density function (pdf) is given by
 The random phase of a sinusoid is often modeled as a

uniform r.v. between 0 and

Statistical Averages
 Consider a discrete r.v. which takes on the possible values
x1, x2, …, xM with respective probabilities P1, P2, …, PM.
 The mean (均值) or expected value (期望值) of X is
M
m X = E [X ] = ∑ xi Pi
i =1
 If X is continuous, then
mX = E [ X ] = ∫ xf X ( x )dx
∞
−∞
 This is the first moment (一阶矩) of X.

 The nth moment of X
[ ]= ∫
EX n
∞
−∞
x n p X ( x )dx
 Let n = 2, we have the mean-square value (均方值) of X
[ ]= ∫
EX 2
−∞
∞
x 2 p X ( x )dx

 n-th Central moment is
[
E ( X − mX ) = ∫
n
] ∞
−∞
( x − mx ) f X ( x )dx
n
 At n=2, we have the variance (方差)
σ X2 = E [( X − mX )2 ]
[
= E X 2 − 2mX X + m2X ]
= E [X ] − m
2 2
X
 σX is called the standard deviation (标准偏差)

 It is the average distance from the mean, a measure of the
concentration of X around the mean

Gaussian Distribution
 Gaussian or normal distribution (正态分布) is a continuous
r.v. with pdf
fX(x)
1  1 
f X ( x) = exp− 2 ( x − mX )2 
2πσ X2  2σ X 
x
0 mX
 A Gaussian r.v. is completely determined by its mean and

variance, and hence usually denoted as
(
X ~ N m X , σ X2 )
 By far the most important distribution in communications
The Q-Function
 The Q-function is a standard form to express error
probabilities without a closed form
∞ 1  u2 
Q( x ) = ∫ exp − du
x 2π  2
 The Q-function is the area under the tail of a Gaussian pdf
with mean 0 and variance 1
 Extremely important in error probability analysis!!!

27
More about Q-Function
 Q-function is monotonically decreasing
 Some features
 Craig’s alternative form of Q-function (IEEE MILCOM’91)
 Upper bound
 If we have a Gaussian variable X ~ N ( µ , σ 2 ) , then

 x−µ 
Pr( X > x) =
Q 
 σ 
Joint Distribution
 Consider 2 r.v.’s X and Y, joint distribution function is
defined as
FXY ( x, y ) = P( X ≤ x, Y ≤ y )
∂ 2 FXY ( x, y )
and joint PDF is f XY ( x, y ) =
∂x∂y
 Key properties of joint distribution

∞ ∞
∫ ∫
−∞ −∞
p XY ( x, y )dxdy = 1
y2 x2
P( x1 < X ≤ x2 , y1 < Y ≤ y2 ) = ∫ ∫ p XY ( x, y )dxdy
y1 x1

 Marginal distribution
∞ x
PX ( x) = P( X ≤ x, − ∞ < Y < ∞) = ∫ ∫ p XY (α , β )dαdβ
−∞ −∞
y ∞
PY ( y ) = ∫ ∫ p XY (α , β )dαdβ
−∞ −∞
 Marginal density
∞
p X ( x) = ∫ p XY ( x, β )dβ
−∞
 X and Y are said to be independent iff

PXY ( x, y ) = PX ( x) PY ( y )
p XY ( x, y ) = p X ( x) pY ( y )

Correlation (1/2)
 Correlation (相关) of the two r.v. X and Y is defined as
RXY = E [ XY ] = ∫
∞ ∞
−∞ −∞ ∫ xyf XY ( x, y )dxdy
 Correlation of the two centered r.v. X-E[X] and Y-E[Y], is

called the covariance (协方差) of X and Y
 If , i.e. , then X and Y are called

uncorrelated.
31
Correlation (2/2)
 The covariance of X and Y normalized w.r.t. σX σY is
referred to the correlation coefficient of X and Y:
 If X and Y are independent, then they are uncorrelated.

 The converse is not true (except the Gaussian case)
32
Joint Gaussian Random Variables
 X1, X2, …, Xn are jointly Gaussian iff
 x is a column vector
 m is the vector of the means
 C is the covariance matrix

Two-Variate Gaussian PDF
 Given two r.v.s: X1 and X2 that are joint Gaussian
 Then

 For uncorrelated X and Y, i.e.
X1 and X2 are also independent
If X1 and X2 are Gaussian and uncorrelated,

then they are independent.
Some Properties of Jointly Gaussian r.v.s
 If n random variables are jointly Gaussian,

any set of them is also jointly Gaussian. In particular, all
individual r.v.s are Gaussian
 Jointly Gaussian r.v.s are completely characterized by the
mean vector and the covariance matrix, i.e. the second-
order properties
 Any linear combination of is a Gaussian
r.v.

Law of Large Numbers
 Consider a sequence of r.v.
 Let
 If Xi’s are uncorrelated with the same mean and

variance
 Then
the average converges to the expected value

Central Limit Theorem
 If are i.i.d random variables with common
mean and common variance
 Then,
converges to a
the sum of many i.i.d random variables converges to

a Gaussian random variable
 Thermal noise results from the random movement of many

electrons – it is well modeled by a Gaussian distribution.

Example

Rayleigh Distribution
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 1 2 3 4 5 6
 Rayleigh distributions are frequently used to model fading

for non-line of sight (NLOS) signal transmission
 Very important for mobile and wireless communications

Exercise
 Let , where a and b are i.i.d Gaussian random
variables with mean 0 and variance
 Show that the magnitude of h follows Rayleigh distribution
and its phase follows a uniform distribution

2.1.Signals

Random Process
 A random process (stochastic process, or random signal)
is the evolution of random variables over time

Description of Random Process
 : random process
 : sample function of the random process
 : values of the random process at
x1(t)
Outcome of 1st
experiment
x2(t) Outcome of
Sample 2nd
space S experiment
xn(t) X(t1) X(t2)
Outcome of nth
t1 t2 experiment

Example
 Uniformly choose a phase between and generate a
sinusoid with a fixed amplitude and frequency but with a
random phase .
 In this case, the random process is

Statistics of Random Processes
 An infinite collection of random variables specified at time t
 Joint pdf

First Order Statistics
 = first order density of
 Mean
E [ X (t0 )] = E [ X (t = t0 )] = ∫ xf X ( x; t0 ) = X (t0 )
∞
−∞
 Variance

Second-Order Statistics
 = second-order density of
 Auto-correlation function (correlation within a process):

Example 1
 Consider , where is uniform in
 Mean:
 Auto-correlation:

Example 2
 Consider , where
 Find its mean and auto-correlation function

Stationary Processes (平稳过程)
 A stochastic process is said to be stationary if for any
and :
(1)
 First-order statistics is independent of

∞
= E { X (t )} ∫= xf X ( x)dx mX (2)
−∞
 Second-order statistics only depends on the gap

∞ ∞
= RX (t1 , t2 ) ∫ ∫ x1 x2 f X ( x1 , x2 , t2 − t1 )dx1dx2
−∞ −∞
=RX (t2 − t1 ) =RX (τ ) , where τ =t2 − t1 (3)

Wide-Sense Stationary (广义平稳）
 A random process is said to be WSS when:
∞
E { X (t )}
= ∫=xf ( x)dx m
X X
(2)
−∞
∞ ∞
RX (t1 , t2 ) ∫ ∫ x x f (x , x ,t
−∞ −∞ 1 2 X 1 2 2 − t1 )dx1dx2
=RX (t2 − t1 ) =RX (τ ) , where τ =t2 − t1 (3)
 A random process is strictly stationary (严格平稳) when:
(1)

Wide-Sense Stationary
 Example 1:
From the previous example:
X(t) is WSS
 Example 2: , where
Is Y(t) WSS?

Averages and Ergodic
 Ensemble averaging
∆
X (t ) = E [X (t )] = ∫ xp( x; t )dx
∞
−∞
∆
RX (t1 , t 2 ) = E [X (t1 ) X (t 2 )] = ∫
∞ ∞
−∞ −∞ ∫ x1 x2 p( x1 , x2 ; t1 , t 2 )dx1dx2
 Time averaging ∆ 1 T
T →∞ 2T ∫−T
< X (t ) > = lim x(t )dt
∆ 1 T
T →∞ 2T ∫−T
< X (t ) X (t − τ ) > = lim x(t ) x(t − τ )dt
 If ensemble average = time average, X(t) is said to be Ergodic

(各态历经)
So what?
Random process
Wide sense stationary
Strictly
Ergodic
stationary
 Applications
 Signal: WSS
 Noise: (strictly) stationary
 Time-varying channel: ergodic => ergodic capacity
Frequency Domain Characteristics of
Random Process
Power Spectral Density

(功率谱密度)
frequency
time

Power Spectral Density
 Given a deterministic signal
 Power:
 Truncate to get an energy signal
 x(t ) (t <T)
xT (t ) = 
. 0 (其余)
 According to Parseval theorem
 Thus,
Power spectral density of x(t)

PSD of Random Process
 Consider as a sample function of a random process
 Average power of
 Power spectral density of

Exercise
 Given a binary semi-random signal
 is a rectangular pulse shaping function with width

 is a random variable that takes +1 or -1 with equal probability,
and it is independent for different n
 Sample functions of X(t) are
x(t, ei ) Tb
0 t
 Find its power spectral density

x(t, ei +1 )
0 t

PSD of WSS Process
 Wiener-Khinchin theorem
∞
RX (τ ) = ∫ S X ( f ) exp( j 2πfτ )df
S X ( f ) ↔ RX (τ ) −∞
∞
S X ( f ) = ∫ RX (τ ) exp(− j 2πfτ )dτ
−∞
 Property:
∞
RX (0 ) = ∫ S X ( f )df = total power
−∞

Example
 For the random process
 We had
 Hence

Exercise
 Given a binary random signal
 is a rectangular pulse shaping function with width

 is a random variable that takes +1 or -1 with equal probability,
and it is independent for different n
 is a random time delay uniformly distributed within
 A typical sample function of X(t) is
x(t)
T τ
1
x(t, ei+1 )
0 t1 t2 t
Tb

 Find the autocorrelation function
 Find its power spectral density

Random Process Transmission Through
Linear Systems
 Consider a linear system (channel)
X(t) Impulse Y(t)

response
h(t)
∞
Y (t ) = X (t ) * h(t ) = ∫ h(τ ) X (t − τ )dτ
−∞
 Mean of the output Y(t)
If X(t) is WSS

Random Process Transmission Through
Linear Systems
 Autocorrelation of Y(t)
If X(t) is WSS
If input is a WSS process, the output is also a

WSS process

Relation Among the Input-Output PSD
 Autocorrelation of Y(t)
 PSD of Y(t):
SY ( f ) = H ( f ) S X ( f )
2

Deterministic vs. Random
Deterministic h(t)
signal
WSS Random
h(t)
signal
SY ( f ) = H ( f ) S X ( f )
2

Example
Differentiator
 Let the random pass through the above

system, then

2.1.Signals
2.4.Gaussian and White Processes

Gaussian Process
 Definition:
 is a Gaussian process if for all and all
the sample values have a joint
Gaussian density function
 Properties:
 If it is wide-sense stationary, it is also strictly stationary
 If the input to a linear system is a Gaussian process, the
output is also a Gaussian process

Noise
 Often modeled as Gaussian and stationary with zero mean
 White noise
N0
Sn ( f ) = N 0 2 Rn (τ ) = δ (τ )
2
White noise is completely
uncorrelated!
=
N 0 = 4.14 ×10−21
KT
= −174 dBm/Hz
Bandlimited Noise
White noise Filter Bandwidth Bandlimited white
Ｂ Hz noise n(t)
Q1. At what rate to sample the noise can

we get uncorrelated realizations?
Q2. What is the power of each sample?

Band-pass Noise
White noise Band-pass noise

Bandpass filter n(t)
 Canonical form of a band-pass noise process
In-phase component Quadrature component
Low-pass noise process

Properties
 Let n(t) be a zero-mean, stationary and Gaussian noise
 Then nc(t) and ns(t) satisfy the following properties
 Property 1：
 nc(t) and ns(t) are zero-mean, jointly stationary and jointly Gaussian
process

 Property 2：
 S n ( f − f 0 ) + S n ( f + f 0 ), f ≤ B / 2
S=
nc ( f ) S=ns ( f ) 
0 otherwise
 Proof
HL ( f )
× Z1 (t ) 1 nc (t )
-B/2 0 B/2 f
2cos ω0t
n(t )
−2sin ω0t
HL ( f )
× Z2 (t ) 1 ns (t )
-B/2 0 B/2 f

Envelop and Phase
 Angular representation of n(t)
=n(t ) R(t ) cos [ω0t + φ (t ) ]
=
R(t ) nc 2 (t ) + ns 2 (t ) envelop
where ns (t )
=φ (t ) tan −1 [0 ≤ φ (t ) ≤ 2π ] phase
nc (t )
1 R(t )
≈
n(t ) B
0 t
1
≈
f0

 Let n(t) be a zero-mean, stationary Gaussian process,
find the statistics of the envelop and phase
 Result：
 Envelop follows Rayleigh distribution while phase
follows uniform distribution
2π R  R2 
f ( R) =∫0 f ( R, φ )dφ =σ 2 exp − 2σ 2  R≥0
∞ 1
Proof？ f=(φ ) ∫ f ( R, φ= )dR 0 ≤ φ ≤ 2π
0 2π
 For the same t, the envelop variable R and phase
variable φ are independent (but not the two processes)

Summary
 For WSS: S X ( f ) ↔ RX (τ )
 WSS transmission through a linear system
h(t)
SY ( f ) = H ( f ) S X ( f )
2
 Gaussian, stationary, and zero-mean noise
 non-Gaussian noise?
 Snapping shrimp noise in shallow water acoustic communication

Suggested Reading & Homework
 Chapter 2.1-2.6, Chapter 5.1-5.3 of Fundamentals
of Communications Systems, Pearson Prentice
Hall 2005, by Proakis & Salehi
 Homework #1:
 Textbook Ch2: 2.5, 2.9, 2.13
 Textbook Ch5: 5.5, 5.10 (Hint: Q-fun), 5.22, 5.28,
5.40, 5.58

Comm ch02 Random en PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Comm ch02 Random en PDF

Uploaded by

Copyright:

Available Formats

Principles of Communications

Dept. of Electronic Engineering

Meixia Tao @ SJTU 2

2.2.Review of probability and random variables

2.3.Random Processes: basic concepts

2.4.Guassian and White Processes

Meixia Tao @ SJTU 3

Meixia Tao @ SJTU 4

 Continuous valued signal vs. discrete-valued signal

o Continuous-time and continuous valued: analog signal

Meixia Tao @ SJTU 5

Meixia Tao @ SJTU 6

Meixia Tao @ SJTU 7

 A signal is an energy signal iff Ex is finite

Meixia Tao @ SJTU 8

Meixia Tao @ SJTU 9

2.2.Review of probability and random variables

2.3.Random Processes: basic concepts

2.4.Guassian and White Processes

Meixia Tao @ SJTU 10

P(0) = 0.3 P01 = P(receive 1 | sent 0) = 0.01

 What is the probability that the output of this channel is 1?

Meixia Tao @ SJTU 11

 Consider two events A and B

 A and B are said statistically independent iff

Meixia Tao @ SJTU 12

 For any event B, we have

Meixia Tao @ SJTU 13

 Example 1 on slide 11?

Meixia Tao @ SJTU 14

Meixia Tao @ SJTU 15

 Key properties of CDF

 Key properties of PDF

Xmin 0 x1 x2 Xmax x Xmin 0 x1 x2 x x+dx Xmax x

 Often used to model binary data

Meixia Tao @ SJTU 18

 The PMF is given by

 That is, the probability that Y = k is the probability that k of

 If no error correction is used, the error probability is

Meixia Tao @ SJTU 20

Meixia Tao @ SJTU 21

 The random phase of a sinusoid is often modeled as a

Meixia Tao @ SJTU 22

 This is the first moment (一阶矩) of X.

Meixia Tao @ SJTU 23

 Let n = 2, we have the mean-square value (均方值) of X

Meixia Tao @ SJTU 24

 At n=2, we have the variance (方差)

 σX is called the standard deviation (标准偏差)

Meixia Tao @ SJTU 25

 A Gaussian r.v. is completely determined by its mean and

 Extremely important in error probability analysis!!!

 Craig’s alternative form of Q-function (IEEE MILCOM’91)

 If we have a Gaussian variable X ~ N ( µ , σ 2 ) , then

 Key properties of joint distribution

Meixia Tao @ SJTU 29

 X and Y are said to be independent iff

Meixia Tao @ SJTU 30

 Correlation of the two centered r.v. X-E[X] and Y-E[Y], is

 If , i.e. , then X and Y are called

 If X and Y are independent, then they are uncorrelated.

Meixia Tao @ SJTU 33