Digital Communication Unit-III Stochastic Process

Digital Communication Unit-
III Stochastic Process
Ashok N Shinde
ashok.shinde0349@gmail.co
m
International Institute of Information Technology
Hinjawadi Pune
July 26, 2017
Ashok N Shinde DC Unit-III Stochastic Process 1/28

Introduction to Stochastic Process
Signals
Deterministic: can be reproduced exactly with repeated
measurements.

Signals
measurements.
s(t) = A c o s ( 2 π f c t + θ)

Signals
measurements.
s(t) = A c o s ( 2 π f c t + θ)
where A , f c and θ are constant.

Signals
measurements.
s(t) = A c o s ( 2 π f c t + θ)
Random: signal that is not repeatable in a predictable

manner.

Signals
measurements.
s(t) = A c o s ( 2 π f c t + θ)

manner.
s(t) = A c o s ( 2 π f c t + θ)

Signals
measurements.
s(t) = A c o s ( 2 π f c t + θ)

manner.
s(t) = A c o s ( 2 π f c t + θ)
where A , f c and θ are variable.

Signals
measurements.
s(t) = A c o s ( 2 π f c t + θ)

manner.
s(t) = A c o s ( 2 π f c t + θ)
where A , f c and θ are variable.
Unwanted signals: Noise

Stochastic Process
Definition:
A stochastic process is a set of random variables indexed in time.
Mathematically:

Stochastic Process
Definition:
Mathematically:
Mathematically relationship between probability theory and
stochastic processes is as follows-

Stochastic Process
Definition:
Mathematically:
Sample point →Sample Function

Stochastic Process
Definition:
Mathematically:
Sample space →Ensemble

Stochastic Process
Definition:
Mathematically:
Random Variable→ Random
Process

Stochastic Process
Definition:
Mathematically:
Random Variable→ Random Process
Sample point s is function of time :

Stochastic Process
Definition:
Mathematically:
X ( s , t), − T ≤ t ≤ T

Stochastic Process
Definition:
Mathematically:
X ( s , t), − T ≤ t ≤ T
Sample function denoted as:

Stochastic Process
Definition:
Mathematically:
X ( s , t), − T ≤ t ≤ T
Sample function denoted as:
x j (t) = X ( t , s j ), − T ≤ t ≤ T

Stochastic Process
A random process is defined as the ensemble(collection) of time

functions together with a probability rule

Stochastic Process
A random process is defined as the ensemble(collection) of time

functions together with a probability rule
| x j (t)|, j = 1, 2, . . . , n

Stochastic Process
Stochastic Process
Each sample point in S is associated with a sample function x(t)

Stochastic Process
Stochastic Process
X ( t, s) is a random process

Stochastic Process
Stochastic Process
X (t , s) is a random process
is an ensemble of all time functions together with a probability rule

Stochastic Process
Stochastic Process
X ( t k , s j ) is a realization or sample function of the random process
{x 1 (t k ), x 2 (t k ), . . . , x n ( t k ) = X ( t k , s 1 ), X ( t k , s 2 ), . . . , X ( t k , s n )}

Stochastic Process
Stochastic Process
Probability rules assign probability to any meaningful event associated
with an observation An observation is a sample function of the random
process

Stochastic Process
Stochastic Process
process
A stochastic process X( t, s) is represented by time indexed ensemble

(family) of random variables {X(t, s)}

Stochastic Process
Stochastic Process
process
A stochastic process X(t, s) is represented by time indexed ensemble

Represented compactly by : X (t)

Stochastic Process
Stochastic Process
process
A stochastic process X(t, s) is represented by time indexed ensemble

Represented compactly by : X (t)
“A stochastic process X(t) is an ensemble of time functions, which,
together with a probability rule, assigns a probability to any meaningful
event associated with an observation of one of the sample functions of
the stochastic process”.

Stochastic Process:
Stationary Vs Non-Stationary Process
Stationary Process:

Stochastic Process:
Stationary Process:
If a process is divided into a number of time intervals exhibiting
same statistical properties, is called asStationary.

Stochastic Process:
Stationary Process:
It is arises from a stable phenomenon that has evolved into a
steady-state mode of behavior.

Stochastic Process:
Stationary Process:
Non-Stationary Process:

Stochastic Process:
Stationary Process:
different statistical properties, is called asNon-Stationary.

Stochastic Process:
Stationary Process:
different statistical properties, is called asNon-Stationary. It is
arises from an unstable phenomenon.

Classes of Stochastic Process:
Strictly Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t = −∞ is said to be

Stationaryin thestrict sense, orstrictly stationaryif,


F X ( t 1 +τ ) , X( t 2 + τ ) , . . . , X ( t k + τ ) ( x 1 , x 2 , . . . , x k ) =
F X ( t 1 ), X ( t 2 ) , . . . , X ( t k ) (x 1 , x 2 , . . . , x k ) Where,


F X ( t 1 + τ ) , X ( t 2 + τ ) , . . . , X ( t k + τ ) (x 1 , x 2 , . . . , x k ) =
F X ( t 1 ) , X ( t 2 ) , . . . , X ( t k ) (x 1 , x 2 , . . . , x k ) Where,
X(t 1 ), X(t 2 ), . . . , X ( t k ) denotes RVs obtained by sampling
process
X ( t ) at t 1 , t 2 , . . . , t k respectively.


F X ( t 1 +τ ) , X( t 2 + τ ) , . . . , X ( t k + τ ) ( x 1 , x 2 , . . . , x k ) =
F X ( t 1 ), X ( t 2 ) , . . . , X ( t k ) (x 1 , x 2 , . . . , x k ) Where,
X (t 1 ), X (t 2 ), . . . , X ( t k ) denotes RVs obtained by sampling process
F X ( t 1 ) , X ( t 2 ) , . . . , X ( t k ) (x 1 , x 2 , . . . , x k ) denotes Joint distribution
function of RVs.


X(t 1 ), X(t 2 ), . . . , X ( t k ) denotes RVs obtained by sampling process
function of RVs.
X ( t 1 + τ ), X ( t 2 + τ ), . . . , X ( t k + τ ) denotes new RVs obtained by
sampling process X ( t ) at t 1 + τ , t 2 + τ , . . . , t k + τ respectively.
Here
τ is fixed time shift.


X(t 1 ), X(t 2 ), . . . , X ( t k ) denotes RVs obtained by sampling process
function of RVs.
X ( t 1 + τ ), X ( t 2 + τ ), . . . , X ( t k + τ ) denotes new RVs obtained by
sampling process X ( t ) at t 1 + τ , t 2 + τ , . . . , t k + τ respectively.
Here
τ is fixed time shift.
F X ( t 1 +τ ) , X ( t 2 + τ ) , . . . , X ( t k +τ ) (x 1 , x 2 , . . . , x k ) denotes Joint
distribution function of new RVs.

Properties of Strictly Stationary Process:


For k = 1, we have F X ( t ) (x) = F X ( t + τ ) (x) = F X (x) for all t and τ . First-
order distribution function of a strictly stationary stochastic process is
independent of time t.


For k = 2, we have F X ( t 1 ) , X ( t 2 ) (x 1 , x 2 ) = F X ( 0 ) , X ( t 1 − t 2 ) (x 1 , x 2 ) for
all t 1 and t 2 . Second-order distribution function of a strictly stationary
stochastic process depends only on the time difference between the
sampling instants and not on time sampled.


Weakly Stationary Process:


A stochastic process X(t) is said to be weakly stationary(Wide-Sense
Stationary) if its second-order moments satisfy:


The mean of the process X ( t ) is constant for all time t.


The autocorrelation function of the process X (t ) depends solely on the
difference between any two times at which the process is sampled.


Summary of Random Processes


Wide-Sense Stationary processes


Strictly Stationary Processes


Ergodic Processes


Ergodic Processes
Non-Stationary processes
Mean, Correlation, and Covariance Functions of WSP
Mean:

Mean:
Mean of real-valued stochastic process X(t), is expectation of the
random variable obtained by sampling the process at some time t, as
shown by

Mean:
shown by
µ X (t) = E [X(t)]

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx
where f is the first-order probability density function of the
X(t)( x )
process X(t).

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx
X(t)( x )
process X(t).
µ X (t) = µ X for weakly stationary process

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx
X(t)( x )
process X(t).
Correlation:

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx
X(t)( x )
process X(t).
Correlation:
Autocorrelation function of the stochastic process X (t ) is product of two
random variables, X ( t 1 ) and X ( t 2 )

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx
X(t)( x )
process X(t).
Correlation:
MX X (t 1 , t 2 ) = E[X(t 1 )X(t 2 )]

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx
X(t)( x )
process X(t).
Correlation:
M X X (t 1 , t 2 ) = E,[X(t 1,)X(t 2 )]
∞ ∞
M X X (t1 2, t ) = x x f (x 1, x2 )dx dx
1 2
1 2 X ( t ) , X1 ( x ) 2
− ∞ − ∞
where f X ( t 1 ), X ( x 2 ) ( x 1 , x 2 ) is joint probability density function of the
process X ( t ) sampled at times t 1 and t 2 . M X X (t 1 , t 2 ) is a
second-order moment. It is depend only on time difference t 1 − t 2 so that
the process X ( t ) satisfies the second condition of weak stationarity and
reduces to.

Mean:
shown by
µ X (t) = E[X(t)]
,
µ X (t ) = −∞∞ x f X (t ) (x )dx
X(t)( x )
process X(t).
Correlation:
M X X (t 1 , t 2 ) = E,[X(t 1,)X(t 2 )]
∞ ∞
M X X (t1 2, t ) = x x f (x 1, x2 )dx dx
1 2
1 2 X ( t ) , X1 ( x ) 2
− ∞ − ∞
where f X ( t 1 ), X ( x 2 ) ( x 1 , x 2 ) is joint probability density function of the
process X ( t ) sampled at times t 1 and t 2 . M X X (t 1 , t 2 ) is a
second-order moment. It is depend only on time difference t 1 − t 2 so that
the process X ( t ) satisfies the second condition of weak stationarity and
reduces to.
M X X (t 1 , t 2 ) = E[X (t 1 )X (t 2 )] = R X X (t 2 − t 1 )

Properties of Autocorrelation Function

Autocorrelation function of a weakly stationary process X( t) can
also be represented as

Autocorrelation function of a weakly stationary process X( t ) can
R XX (τ ) = E [ X ( t + τ ) X ( t ) ]

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
where τ denotes a time shift; that is,t = t 2 and τ = t 1 − t 2

R X X (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:

R X X (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ Σ
RX X (0) = E X2(t) (Mean-Square Value)

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
R X X (0) = E XΣ (t) (Mean-Square Value)
R X X (τ ) = R X X ( − τ ) also R X X (τ ) = E [ X ( t − τ )X(t)]
(Symmetry)

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
R X X (0) = E X Σ (t) (Mean-Square Value)
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)
R X X (τ )
ρ X X (τ ) = R X X (0) (Normalization [−1, 1])

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)
R X X (τ )
ρ X X (τ )= R X X (0) (Normalization [−1, 1])
Covariance:

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)
R X X (τ )
Covariance:
Autocovariance function of a weakly stationary process X(t ) is
defined by

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)
R X X (τ )
Covariance:
Autocovariance function of a weakly stationary process X( t) is
defined by
C XX (t 1 , t 2 ) = E [ ( X ( t 1 ) − µ x ) ( X ( t 2 ) − µ x )]

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)
R X X (τ )
Covariance:
defined by
C XX (t 1 , t 2 ) = E [ ( X ( t 1 ) − µ x ) ( X ( t 2 )2 − µ x )]
C X X (t 1 , 2t ) = R X X (t 2− t )1 − µ x

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)
R X X (τ )
Covariance:
defined by
C XX (t 1 , t 2 ) = E [ ( X ( t 1 ) − µ x ) ( X ( t 2 )2 − µ x )]
C X X (t 1 , 2t ) = R X X (t 2− t )1 − µ x
The autocovariance function of a weakly stationary process X ( t )
depends only on the time difference (t 2 − t 1 )

R XX (τ ) = E [ X ( t + τ ) X ( t ) ]
Properties:
Σ 2
(Symmetry)
| R X X (τ )| ≤ R X X (0) (Bound)
R X X (τ )
Covariance:
defined by
C XX (t 1 , t 2 ) = E [ ( X ( t 1 ) − µ x ) ( X ( t 2 )2 − µ x )]
C X X (t 1 , 2t ) = R X X (t 2− t )1 − µ x
The autocovariance function of a weakly stationary process X (t )
depends only on the time difference (t 2 − t 1 )
The mean and autocorrelation function only provide a weak
description of the distribution of the stochastic process X(t).

Example Question:
InSem 2014, InSem 2015 (Sin), InSem 2016 ( − π2≤ Θ ≤ )π2
Examples
Show that the Random Process X ( t ) = Acos(ω c t + Θ) is wide sense stationary process,
where Θ is RV uniformly distributed in range (0, 2π)
Answer:
The ensemble consist of sinusoids of constant amplitude A and constant
frequency ωc, but phase Θ is random.

Example Question:
Examples
Answer:
The phase is equally likely to any value in the range (0, 2π).

Example Question:
Examples
Answer:
The phase is equally likely to any value in the range (0, 2π). Θ
is RV uniformly distributed over the range (0, 2π).

Example Question:
Examples
Answer:
1
f Θ (θ) = , 0 ≤ θ ≤ 2π
2π

Example Question:
Examples
Answer:
1
f Θ (θ) = , 0 ≤ θ ≤ 2π
2π
= 0, elsewhere

Example Question:
Answer:
Because cos(ω ct + Θ) is function of RV Θ, Mean of Random Process
X ( t) is

Example Question:
Answer:
X ( t) is
X(t) = Acos(ω c t + Θ)

Example Question:
Answer:
X ( t) is
X (t)
= Acos(ω c t + Θ)
= Acos(ω c t + Θ)

Example Question:
Answer:
X ( t ) is
¸ 2π c t + Θ)
= Acos(ω
cos(ω ct + Θ) = 0
cos(ω ct + θ)f (θ)dθ
Θ

Example Question:
Answer:
X ( t ) is
¸ 2π c t + Θ)
= Acos(ω
cos(ω ct + Θ) = 0
Θ
1 ¸ 2π
= cos(ω ct + θ)dθ
2π 0

Example Question:
Answer:
X ( t ) is
¸ 2π c t + Θ)
= Acos(ω
cos(ω ct + Θ) = 0
Θ
1 ¸ 2π
2π 0
=0

Example Question:
Answer:
X ( t ) is
¸ 2π c t + Θ)
= Acos(ω
cos(ω ct + Θ) = 0
Θ
1 ¸ 2π
2π 0
=0
X (t ) =0

Example Question:
Answer:
X ( t ) is
¸ 2π c t + Θ)
= Acos(ω
cos(ω ct + Θ) = 0
Θ
1 ¸ 2π
2π 0
=0
X (t ) =0
Thus the ensemble mean of sample function amplitude at any time
instant t is zero.

Example Question:
Answer:
X ( t ) is
¸ 2π c t + Θ)
= Acos(ω
cos(ω ct + Θ) = 0
Θ
1 ¸ 2π
2π 0
=0
X(t) =0
Thus the ensemble mean of sample function amplitude at any time
instant t is zero.
The Autocorrelation function R X X ( t 1 , t 2 ) for this process can also be determined
as
R XX (t 1 , t 2 ) = E [ X ( t 1 ) X ( t 2 ) ]
= A 2 cos(ω c t 1 + Θ)cos(ωct 2 + Θ)

Example Question:
Answer:
Continue. . .

Example Question:
Answer:
Continue. . .
= A 2 cos(ω c t 1 + Θ)cos(ω ct 2 + Θ)

Example Question:
Answer:
Continue. . .
= A
2 cos(ωc(t2 − t 1 )) + cos(ω c(t 2 + t 1 ) + 2Θ)

Example Question:
Answer:
Continue. . .
A
= cos(ω c(t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
The term cos(ωc(t 2 − t 1 )) does not contain RV Hence,

Example Question:
Answer:
Continue. . .
A Σ
Σ
= cos(ω (t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
The term cos(ωc(t 2 − t 1 )) does not containc RV Hence,
cos(ω c(t2 − t 1 )) = cos(ω c(t 2 − t 1 ))

Example Question:
Answer:
Continue. . .
A Σ
Σ
= cos(ω (t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
The term cos(ω c(t 2 − t 1 )) does not containc RV Hence,
cos(ω c(t2 − t 1 )) = cos(ω c(t2 − t 1 ))
The term cos(ωc(t 2 + t 1 ) + 2Θ) is a function of RV Θ, and it is

Example Question:
Answer:
Continue. . .
A Σ
Σ
= cos(ω (t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
cos(ω c(t2 − t 1 )) = cos(ω c(t2 − t 1 ))
The term cos(ω (t 2t +)t+

1 ) 2Θ)
+ 2Θ) 1 ¸ 2π RV Θ, and it is
cos(ω (t c+
c 2 1 = is a function of cos(ω
0
(t + t ) + 2θ)dθ
c 2 1
2π

Example Question:
Answer:
Continue. . .
A Σ
Σ
= cos(ω (t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
cos(ω c(t2 − t 1 )) = cos(ω c(t2 − t 1 ))

1 ) 2Θ)
cos(ω (t c+
0
(t + t ) + 2θ)dθ
c 2 1
2π
=0

Example Question:
Answer:
Continue. . .
A Σ
Σ
= cos(ω (t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
cos(ω c(t2 − t 1 )) = cos(ω c(t2 − t 1 ))

1 ) 2Θ)
cos(ω (t c+
0
(t + t ) + 2θ)dθ
c 2 1
2π
=0
= A 2 cos(ωc(t2 − t 1 )),
R XX (t 1 , t 2 ) 2

Example Question:
Answer:
Continue. . .
A Σ
Σ
= cos(ω (t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
cos(ω c(t2 − t 1 )) = cos(ω c(t2 − t 1 ))

1 ) 2Θ)
cos(ω (t c+
0
(t + t ) + 2θ)dθ
c 2 1
2π
=0
= A 2 cos(ωc(t2 − t 1 )),
R XX (t 1 , t 2 ) 2
= A 2 cos(ω c(τ )), · · · τ = t 2 − t 1
R XX (τ ) 2

Example Question:
Answer:
Continue. . .
A Σ
Σ
= cos(ω (t 2 − t 1 )) + cos(ω c(t2 + t 1 ) + 2Θ) 2
cos(ω c(t2 − t 1 )) = cos(ω c(t2 − t 1 ))

1 ) 2Θ)
cos(ω (t c+
c 2 1 = is a function of cos(ω (t + t ) + 2θ)dθ
0 c 2 1
2π
=0
= A 2 cos(ωc(t2 − t 1 )),
R XX (t 1 , t 2 ) 2
= A 2 cos(ω c(τ )), · · · τ = t 2 − t 1
R XX (τ ) 2
A2
From X ( t ) = 0 and R XX (τ ) = cos(ω c(τ )) it is clear that X ( t ) is
2
Wide Sense Stationary Process

Time Vs Ensemble Average and Ergodic Process
Ensemble Average

Ensemble Average
Difficult to generate a number of realizations of a random process

Ensemble Average
Difficult to generate a number of realizations of a random process Use
time averages

Ensemble Average
time averages
1 ¸ +T
Mean ⇒ µX(T ) = x (t)dt
2T − T

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
Autocorrelation ⇒ R X X (τ, T ) = x(t)x(t + τ )dt
2T − T

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
Ergodicity: A random process is called Ergodic if

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
Ergodicity: A random process is called Ergodic if it
is ergodic in mean:

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
is ergodic in mean:
l i m T → ∞ µ X (T ) = µ X

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
is ergodic in mean:
l i m T → ∞ µ X (T ) = µ X
l i m T → ∞ var [µ X (T )] = 0

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
is ergodic in mean:
l i m T → ∞ µ X (T ) = µ X
l i m T → ∞ var [µ X (T )] = 0
it is ergodic in autocorrelation:

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
is ergodic in mean:
l i m T → ∞ µ X (T ) = µ X
l i m T → ∞ var [µ X (T )] = 0
limT → ∞ R X X (τ, T ) = R X X (τ )

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
is ergodic in mean:
l i m T → ∞ µ X (T ) = µ X
l i m T → ∞ var [µ X (T )] = 0
l i m T → ∞ R X X (τ, T ) = R X X (τ )
l i m T → ∞ var [ R X X (τ, T )] = 0

Ensemble Average
time averages
1 ¸ +T
2T − T
1 ¸ +T
2T − T
Ergodic Process
is ergodic in mean:
l i m T → ∞ µ X (T ) = µ X
l i m T → ∞ var [µ X (T )] = 0
l i m T → ∞ R X X (τ, T ) = R X X (τ )
l i m T → ∞ var [ R X X (τ, T )] = 0
where µ X and R X X (τ ) are the ensemble averages of the same random
process.

Transmission of a Weakly Stationary Process through a
LTI Filter
Linear Time Invariant Filter

LTI Filter

Suppose that a stochastic process X (t ) is applied as input to a linear time-
invariant filter of impulse response h(t), producing a new stochastic
process Y (t) at the filter output.

LTI Filter


LTI Filter

It is difficult to describe the probability distribution of the output

stochastic process Y(t), even when the probability distribution of the
input stochastic process X(t) is completely specified

LTI Filter


For defining the mean and autocorrelation functions of the output
stochastic process Y (t) in terms of those of the input X(t), assuming
that X ( t) is a weakly stationary process.

LTI Filter


For defining the mean and autocorrelation functions of the output
stochastic process Y (t) in terms of those of the input X(t), assuming
that X ( t) is a weakly stationary process.
Transmission of a process through a linear time-invariant filter is
governed by the convolution integral

LTI Filter
Continue. . .

LTI Filter
Continue. . .
we may thus express the output stochastic process Y (t) in terms of the
input stochastic process X( t ) as

LTI Filter
Continue. . .
we may thus express the output stochastic process Y (t) in terms of
the input stochastic process X ( t) as
,+∞
Y (t) = −∞ h(τ 1)X ( t − τ 1)dτ 1

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
where τ 1is local time. Hence, the mean of Y (t) is

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [Y (t)]

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
Provided that the expectation E [X(t)] is finite for all t and the filter is
stable.

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
stable. ,
µ Y (t) = −+∞∞ h(τ 1)E [ X ( t − τ 1)] dτ 1

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
stable. ,
µ Y (t) = −+∞∞ h(τ 1)E [ X ( t − τ 1)] dτ 1
,
µ Y (t) = −+∞∞ h(τ 1)µX(t − τ )dτ1 1

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
stable. ,
µ Y (t) = −+∞∞ h(τ 1)E [ X ( t − τ 1)] dτ 1
,
µ Y (t) = −+∞∞ h(τ 1)µX(t − τ )dτ1 1
When the input stochastic process X ( t ) is weakly stationary, the mean
µ X (t) is a constant µ X

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
stable. ,
µ Y (t) = −+∞∞ h(τ 1)E [ X ( t − τ 1)] dτ 1
,
µ Y (t) = −+∞∞ h(τ 1)µX(t − τ )dτ1 1
When the input stochastic process X ( t ) is weakly stationary, the
mean µ X (t), is a constant µ X
µ Y= µ X −+∞∞ h(τ 1)dτ 1

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
stable. ,
µ Y (t) = −+∞∞ h(τ 1)E [ X ( t − τ 1)] dτ 1
,
µ Y (t) = −+∞∞ h(τ 1)µX(t − τ )dτ1 1
µ Y= µ X −+∞∞ h(τ 1)dτ 1
µ Y = µ H(0)
X

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
stable. ,
µ Y (t) = −+∞∞ h(τ 1)E [ X ( t − τ 1)] dτ 1
,
µ Y (t) = −+∞∞ h(τ 1)µX(t − τ )dτ1 1
µ Y= µ X −+∞∞ h(τ 1)dτ 1
µ Y = µ H(0)
X
where H(0) is the zero-frequency response of the system

LTI Filter
Continue. . .
,
Y (t) = −+∞∞ h(τ 1)X ( t − τ 1)dτ 1
µ Y (t) = E [YΣ (t)]
, +∞
µ Y(t) = E Σ − ∞ h(τ 1) X ( t − τ 1) dτ 1
stable. ,
µ Y (t) = −+∞∞ h(τ 1)E [ X ( t − τ 1)] dτ 1
,
µ Y (t) = −+∞∞ h(τ 1)µX(t − τ )dτ1 1
µ Y= µ X −+∞∞ h(τ 1)dτ 1
µ Y = µ H(0)
X
where H(0) is the zero-frequency response of the system
“The mean of the stochastic process Y(t) produced at the output of a
linear time-invariant filter in response to a weakly stationary process
X(t), acting as the input process, is equal to the mean of X(t) multiplied
by the zero-frequency response of the filter.”
LTI Filter
Autocorrelation

LTI Filter
Autocorrelation
Consider the autocorrelation function of the output stochastic
process Y (t)

LTI Filter
Autocorrelation
Consider the autocorrelation function of the output stochastic
process Y (t)
MY Y (t, u) = E [Y (t)Y (u)]

where t and u denote two values of the time at which the output
process Y (t) is sampled
Σ
Σ∫ +∞ ∫ +∞
. =E h(τ 1 )X(t − τ 1 )dτ 1 h (τ 2 )X (u − τ 2 )dτ 2
−∞ −∞
Σ
Here again, provided that the mean-square value E X (t)Σis 2finite
for all t and the filter is stable
∫ +∞ . ∫ +∞ Σ
= h(τ 1 ) dτ 2 h(τ 2 )E [ X (t − τ 1 )X (u − τ 2 )] dτ 1
−∞ −∞
∫ +∞ Σ ∫ +∞ Σ
= h(τ 1 ) d τ 2 h ( τ 2 ) M X X (t − τ 1 , u
− τ 2 ) dτ 1
−∞ −∞
When the input X ( t ) is a weakly stationary process, the autocorrelation

function of X ( t ) is only a function of the difference between the
sampling times tτ 1 and uτ 2 . Thus, putting τ = u t
LTI Filter
Autocorrelation

LTI Filter
Autocorrelation
function of X( t ) is only a function of the difference between the sampling
times tτ 1 and u τ 2 . Thus, putting τ = u t

LTI Filter
Autocorrelation
∫ +∞ ∫ +∞
R Y Y (τ ) = h (τ 1 )h(τ 2 )R X X (τ + τ 1 − τ 2 )dτ 1 dτ 2
−∞ −∞
which depends only on the time difference τ

LTI Filter
Autocorrelation
∫ +∞ ∫ +∞
R Y Y (τ ) = h(τ 1 )h (τ 2 )R X X (τ + τ 1 − τ 2 )dτ 1 dτ 2
−∞ −∞
.
“If the input to a stable linear time invariant filter is a weakly
stationary process, then the output of the filter is also a weakly
stationary process”.

LTI Filter
Autocorrelation
∫ +∞ ∫ +∞
−∞ −∞
.
mean square value of the outputΣ process
Σ Y (t) is obtained by putting
τ = 0. We haveR Y Y (0) = E Y 2(t)

LTI Filter
Autocorrelation
∫ +∞ ∫ +∞
−∞ −∞
.
τ = 0. We haveR Y Y (0) = E Y 2(t)
∫ +∞ ∫ +∞
Σ 2
E ΣY (t) = h(τ 1 )h (τ 2 )R X X (τ1 − τ 2 )dτ 1 dτ 2
−∞ −∞

LTI Filter
Autocorrelation
∫ +∞ ∫ +∞
−∞ −∞
.
τ = 0. We haveR Y Y (0) = E Y 2(t)
∫ +∞ ∫ +∞
Σ 2
E ΣY (t) = h(τ 1 )h (τ 2 )R X X (τ1 − τ 2 )dτ 1 dτ 2
−∞
−∞
which, of Ashok
course, is a constant.
N Shinde DC Unit-III Stochastic Process 18/28
Power Spectral Density of a Weakly Stationary Process
The impulse response of a linear time-invariant filter is equal to the
inverse Fourier transform of the frequency response of the filter

Using H ( f ) to denote the frequency response of the filter, we may
thus write

Using H ( f ) to denote the frequency response of the filter, we may
thus write
∫ +∞
h(τ 1 ) = H ( f ) exp(j2πfτ 1 )df
−∞
∫ +∞ ∫ +∞
Σ 2
E ΣY (t) = h(τ 1 )h (τ 2 )R X X (τ 1 − τ 2 )dτ 1 dτ 2
−∞ −∞
Σ
∫ +∞ ∫ +∞ Σ∫ +∞
= H ( f ) exp(j2πfτ 1 )df h ( τ 2 ) R X X (τ 1 − τ 2 )dτ 1 dτ 2
−∞ −∞ −∞
∫ +∞ Σ ∫ +∞ ∫ +∞ Σ
= H(f ) h(τ 2 )dτ 2 RX X (τ 1 − τ 2 ) exp(j2πfτ 1 )dτ 1 df
−∞ −∞ −∞
by τ = τ 1 − τ 2
∫ +∞ Σ∫ +∞ ∫ +∞
= H(f ) h(τ 2 ) exp(j2πfτ 2 )dτ 2 RX X (τ ) e x p (− j2 π fτ )dτ
−∞ −∞ −∞
∫ Σ
+∞ Σ∫ +∞
= | H ( f )|2 RX X (τ ) e x p( −j 2 πf τ )dτ df
−∞ −∞
as | H ( f )|2 = H ( f )H ∗ (f ) and square brackets represents fourier

transform of the autocorrelation function R X X of the input process
X (t ) Ashok N Shinde DC Unit-III Stochastic Process 19/28
We may now define a new function for fourier transform of the

autocorrelation function.


X X ( f ) is called the power spectral density, or power
The new function S
spectrum, of the weakly stationary process X ( t ) and denoted as


The new function S
∫ +∞
SX X (f ) = RX X (τ ) e x p( −j 2 πf τ )dτ
−∞
∫ +∞
Σ 2 2
E ΣY (t) = | H ( f )| S X ( f )df
X
−∞
which is the desired frequency-domain equivalent to the time-domain

relation


The new function S
∫ +∞
SX X (f ) = RX X (τ ) e x p( −j 2 πf τ )dτ
−∞
∫ +∞
Σ 2 2
E ΣY (t) = | H ( f )| S X ( f )df
X
−∞
which is the desired frequency-domain equivalent to the time-domain

relation
The mean-square value of the output of a stable linear time-invariant filter in

response to a weakly stationary process is equal to the integral over all
frequencies of the power spectral density of the input process multiplied by
the squared magnitude response of the filter.

Properties of the Power Spectral Density(PSD)
Properties of PSD

Properties of PSD
1 Zero Correlation among Frequency Components

Properties of PSD
The individual frequency components of the power spectral density
S X X ( f ) of a weakly stationary process X(t) are uncorrelated with each
other.

Properties of PSD
other.
No overlap, and therefore no correlation.

Properties of PSD
other.

Properties of PSD
other.
2 Zero-frequency Value of Power Spectral Density

Properties of PSD
other.

The zero-frequency value of the power spectral density of a weakly
stationary process equals the total area under the graph of the
autocorrelation function

Properties of PSD
other.

∫ +∞
S X X (0) = R X X (τ )dτ
−∞

Properties of PSD
other.

∫ +∞
S X X (0) =
RX X (τ )dτ
−∞
3 Mean-square Value of Stationary Process

Properties of PSD
other.

∫ +∞
S X X (0) = R X X (τ )dτ
−∞

The mean-square value of a weakly stationary process X(t) equals the total
area under the graph of the power spectral density of the process

Properties of PSD
other.

∫ +∞
S X X (0) = R X X (τ )dτ
−∞

The mean-square value of a weakly stationary process X(t) equals the total
area under the graph of the power spectral density of the process
∫ +∞
Σ
E ΣX 2(t) = S X X ( f )df
−∞

Properties of PSD

Properties of PSD
4 Nonnegativeness of Power Spectral Density

Properties of PSD
The power spectral density of a stationary process X(t) is always
nonnegative.

Properties of PSD
nonnegative.

Properties of PSD
nonnegative.
SX X (f ) ≥ 0

Properties of PSD
nonnegative.
SX X (f ) ≥ 0
5 Symmetry

Properties of PSD
nonnegative.
SX X (f ) ≥ 0
5 Symmetry
The power spectral density of a real-valued weakly stationary process is an
even function of frequency

Properties of PSD
nonnegative.
SX X (f ) ≥ 0
5 Symmetry
SX X (f ) = S X X (−f )

Properties of PSD
nonnegative.
SX X (f ) ≥ 0
5 Symmetry
SX X (f ) = S X X (−f )
6 Normalization

Properties of PSD
nonnegative.
SX X (f ) ≥ 0
5 Symmetry
SX X (f ) = S X X (−f )
6 Normalization
The power spectral density, appropriately normalized, has the properties
associated with a probability density function in probability theory

Properties of PSD
nonnegative.
SX X (f ) ≥ 0
5 Symmetry
SX X (f ) = S X X (−f )
6 Normalization
The power spectral density, appropriately normalized, has the properties
associated with a probability density function in probability theory
S X X (f )
PX X ( f ) = , ∞
S X X ( f )df
−∞

The Gaussian Process
Stochastic process of interest is the Gaussian process which builds
on the Gaussian distribution.

It is the most frequently encountered random process in the study of
communication systems.

Gaussian Process

Gaussian Process
Let us suppose that we observe a stochastic process X ( t ) for an
interval that starts at time t = 0 and lasts until t = T

Gaussian Process
we weight the process X (t ) by some function g(t) and then integrate
the product g(t)X(t) over the observation interval (0, T )

Gaussian Process
∫ T
Y = g(t)X(t)dt
0

Gaussian Process
∫ T
Y =
g(t )X (t )dt
0
We refer to Y as a linear functional
of X ( t )

Gaussian Process
∫ T
Y =
g(t)X(t)dt
0
We refer to Y as a linear functional of X (t )
1 If the weighting function g(t) is such that the mean-square value of the
random variable Y is finite

Gaussian Process
∫ T
Y =
g(t)X(t)dt
0
2 And if the random variable Y is a Gaussian-distributed random
variable for every g(t) in this class of functions

Gaussian Process
∫ T
Y =
g(t)X(t)dt
0
Then the process X (t ) is said to be a Gaussian process.

Gaussian Process
∫ T
Y =
g(t)X(t)dt
0
We refer to Y as a linear functional of X ( t )
Then the process X (t ) is said to be a Gaussian process.
A process X(t) is said to be a Gaussian process if every linear
functional of X (t)Ashok
is aN Shinde
Gaussian random variable
DC Unit-III Stochastic Process 23/28
Properties The Gaussian Process
Random variable Y has a Gaussian distribution if its probability
density function has the form
. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
where µ is the mean and σ 2 is the variance of the random variable Y

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
If a Gaussian process X(t) is applied to a stable linear filter, then the
stochastic process Y(t) developed at the output of the filter is also
Gaussian.

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
Gaussian.

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
Gaussian.

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
Gaussian.
2 Stationarity

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
Gaussian.
2 Stationarity
If a Gaussian process is weakly stationary, then the process is also
strictly stationary.

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
Gaussian.
2 Stationarity

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
Gaussian.
2 Stationarity
3 Independence

. Σ
1 (y − µ) 2
f Y (y) = √ exp − 2σ 2
2πσ2
1 Linear Filtering
Gaussian.
2 Stationarity
3 Independence
If the random variables X(t 1 ), X(t 2 ), . . . , X ( t n ) , obtained by
respectively sampling a Gaussian process X ( t ) at times t 1 , t 2 , . . . , t n , are
uncorrelated, that is
Σ
E Σ( X ( t ) − µ )(X(t ) − µ )=0 i ƒ=
k X(tk) i X ( t i)
k
then these random variables are statistically independent.
Noise
Unwanted signals that tend to disturb the transmission and

processing of signals in communication systems.

Noise

Sources of noise in a communication system

Noise

External

Noise

External
Atmospheric noise
Galactic noise
Man-made noise

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal
Noise that arises from the phenomenon of spontaneous fluctuations of
current flow that is experienced in all electrical circuits

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal
Shot noise - Discrete nature of current flow in electronic devices.
Thermal noise - Random motion of electrons in a conductor.

Noise

External
Atmospheric noise
Galactic noise
Man-made noise
Internal
Shot noise - Discrete nature of current flow in electronic devices.
Thermal noise - Random motion of electrons in a conductor.
The noise analysis of communication systems is based on a source of
noise called white-noise, which is to be discussed next.

White Noise
Adjective ’white’ is used in the sense that white light contains equal
amount of all frequencies within the visible band of electromagnetic
radiation.

White Noise
radiation.
White noise denoted by W (t), is a stationary process whose PSD
S W (f ) has constant value across all frequencies.

White Noise
radiation.
PSD of white noise is
S W W ( f ) = N 02for all f

White Noise
radiation.
S W W ( f ) = N 0 2for all f
Its auto-correlation function is
N0 2
R WW (τ ) = δ(τ )

White Noise
radiation.
S W W ( f ) = N 0 2for all f
Its auto-correlation function is
N0 2
R WW (τ ) = δ(τ )

ReferencesI
1 S. Haykin. Digital Communication, John Wiley & Sons, 2009.

2 A.B Carlson, P B Crully, J C Rutledge, Communication Systems,
Fourth Edition, McGraw Hill Publication.

Author Information
For further information please contact
Prof. Ashok N Shinde

Department of Electronics & Telecommunication Engineering
Hope Foundation’s
International Institute of Information Technology, (I 2 IT )
Hinjawadi, Pune 411 057
Phone - +91 20 22933441
www.isquareit.edu.in|ashoks@isquareit.edu.in

Digital Communication Unit-III Stochastic Process

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Digital Communication Unit-III Stochastic Process

Uploaded by

Copyright:

Available Formats

Digital Communication Unit-

III Stochastic Process

July 26, 2017

Ashok N Shinde DC Unit-III Stochastic Process 1/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 2/28

where A , f c and θ are constant.

Ashok N Shinde DC Unit-III Stochastic Process 2/28

where A , f c and θ are constant.

Random: signal that is not repeatable in a predictable

Ashok N Shinde DC Unit-III Stochastic Process 2/28

where A , f c and θ are constant.

Random: signal that is not repeatable in a predictable

Ashok N Shinde DC Unit-III Stochastic Process 2/28

where A , f c and θ are constant.

Random: signal that is not repeatable in a predictable

where A , f c and θ are variable.

Ashok N Shinde DC Unit-III Stochastic Process 2/28

where A , f c and θ are constant.

Random: signal that is not repeatable in a predictable

where A , f c and θ are variable.

Unwanted signals: Noise

Ashok N Shinde DC Unit-III Stochastic Process 2/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

Ashok N Shinde DC Unit-III Stochastic Process 3/28

A random process is defined as the ensemble(collection) of time

Ashok N Shinde DC Unit-III Stochastic Process 4/28

A random process is defined as the ensemble(collection) of time

Ashok N Shinde DC Unit-III Stochastic Process 4/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 5/28

A stochastic process X( t, s) is represented by time indexed ensemble

Ashok N Shinde DC Unit-III Stochastic Process 5/28

A stochastic process X(t, s) is represented by time indexed ensemble

Ashok N Shinde DC Unit-III Stochastic Process 5/28

A stochastic process X(t, s) is represented by time indexed ensemble

Ashok N Shinde DC Unit-III Stochastic Process 5/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

Ashok N Shinde DC Unit-III Stochastic Process 6/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28

The Stochastic process X(t) initiated at t = −∞ is said to be

Ashok N Shinde DC Unit-III Stochastic Process 7/28