04-Random Processes

Addis Ababa Science & Technology University
Department of Electrical & Electronics Engineering
Probability and Random Process (EEEg-2114)
Chapter 4: Random Processes

Random Processes
Outline
 Introduction
 Definition of a Random Process
 Characterization of Random Processes
 Mean, Correlation, and Covariance Functions
 Classification of Random Processes
 Power Spectral Densities of Random Processes
 Response of Linear Systems to Random Inputs
Semester-II, 2013/14 By Welelaw Y. 2

Introduction
 The theory of random processes was first developed in

connection with the study of fluctuations and noise in physical
systems.
 A random process is the mathematical model of an empirical

process whose development is governed by probability laws.
 Random processes provides useful models for the studies of

such diverse fields as statistical physics, communication and
control, time series analysis, population growth, and
management sciences.
Definition of a Random Process
 A random process is a family of random variables {X(t), tϵT}

defined on a given probability space, indexed by the parameter
t, where t varies over an index set T.
 In a random process {X(t), tϵT}, the index set T is called the

parameter set of the random process.
 The values assumed by X(t) are called states, and the set of all
possible values forms the state space E of the random process.
 If the index set T of a random process is discrete, then the

process is called a discrete-time random process.
Definition of a Random Process Cont’d…..
 A discrete-time random process is also called a random

sequence and is denoted by {Xn , n = 1, 2, 3, . . .).
 If T is continuous, then we have a continuous-time random

process.
 In fact, a random process {X(t), tϵT} is a function of two

arguments {X(t, ω), tϵT, ωϵΩ}.
 For a fixed time t=tk, X(tk, ω) = Xk(ω) is a random variable

denoted by X(tk), as ω varies over the sample space Ω.

Definition of a Random Process Cont’d…..
 On the other hand, for a fixed sample point ωi ϵΩ, X(t, ωi) =
Xi(t) is a single function of time t, called a sample function or a
realization of the process.
 The totality of all sample functions is called an ensemble.
 Of course if both ω and t are fixed, X(tk , ωi) is simply a real

number.
 In the following discussion, we use the notation X(t) to

represent X(t, ω).

Characterization of Random Processes
 If X(t) is a random process, then for fixed t=t1, X1=X(t1)

represents a random variable.
 Its distribution function is given by:
FX ( x1 , t1 )  P{ X (t )  x1 }
 Notice that FX(x, t) depends on t, since for a different t, we
obtain a different random variable.
 The first-order probability density function of the process X(t)
is defined as:
dFX ( x1 , t1 )
f X ( x1 , t1 ) 
dx1

Characterization of Random Processes Cont’d…..
 For t = t1 and t = t2, X(t) represents two different random

variables X1 = X(t1) and X2 = X(t2) respectively.
 Their joint distribution is given by:
FX ( x1 , x2 , t1 , t2 )  P{ X (t1 )  x1 , X (t2 )  x2 }
 The second-order probability density function of the random
process X(t) is:
 2 FX ( x1 , x2 , t1 , t2 )
f X ( x1 , x2 , t1 , t2 ) 
x1 x2
 Similarly f X ( x1 , x2 ,  xn , t1 , t 2  , t n ) represents the nth order

density function of the process X(t).
Mean, Correlation, and Covariance Functions
 As in the case of random variables, random processes are often

described by using statistical averages.
 The mean of X(t) is defined by:
 X (t )  E  X (t )
where X(t) is treated as a random variable for a fixed value of t.
 In general, μX(t) is a function of time, and it is often called the

ensemble average of X(t).

Mean, Correlation, and Covariance Functions …...
 A measure of dependence among the random variables of X(t)

is provided by its autocorrelation function, defined by:
R XX (t1 , t 2 )  EX (t1 ) X ( 2 )
 Note that:

R XX (t1 , t 2 )  R XX (t 2 , t1 ) and R XX (t , t )  E X 2 (t ) 
 The autocovariance function of X(t) is defined by:
C XX (t1 , t 2 )  CovX (t1 ) , X (t 2 )  EX (t1 )   X (t1 )X (t 2 )   X (t 2 )
 R XX (t1 , t 2 )   X (t1 )  X (t 2 )

Mean, Correlation, and Covariance Functions …...
 It is clear that if the mean of X(t) is zero, then:
C XX (t1 , t 2 )  R XX (t1 , t 2 )
 Note that the variance of X(t) is given by:


 X 2 (t )  Var X (t )  E X (t )   X (t )2 
 If X(t) is a complex random process, then its autocorrelation
function RXX(t1, t2) and autocovariance function CXX(t1, t2) are
defined, respectively, by:
 
R XX (t1 , t 2 )  E X (t1 ) X * (t 2 ) and

C XX (t1 , t 2 )  E X (t1 )   X (t1 )X (t 2 )   X (t 2 )
*

Classification of Random Processes
i. Stationary Processes
 A random process {X(t), tϵT} is said to be stationary or strict-
sense stationary (SSS) if, for all n and for every set of time
instants (ti ϵT, i = 1,2, . . . , n),
FX ( x1 ,........, x n , t1 , ....., t n )  FX ( x1 , ........, x n , t1   , ......, t n   )
 Hence, the distribution of a stationary process will be
unaffected by a shift in the time origin, and X(t) and X(t+τ)
will have the same distributions for any τ.
 Nonstationary processes are characterized by distributions
depending on the points t1, t2, . . . , tn.

Classification of Random Processes Cont’d……
ii. Wide-Sense Stationary Processes

 A random process X(t) is wide-sense stationary (WSS) if:
1 . E  X (t )    X (constant )
2. R XX (t1 , t 2 )  E X (t1 ) X (t 2 )   R XX  t 2  t1 
 Note that a strict-sense stationary process is also a WSS

process, but, in general, the converse is not true.

Classification of Random Processes Cont’d……
 Other types of random processes include:

 Independent Processes
 Markov Processes
 Normal Processes
 Ergodic Processes
 Poisson Processes

Power Spectral Densities of Random Processes
 The autocorrelation function of a continuous-time random

process X(t) is defined as:
R XX ( )  E X (t ) X (t   )
 Properties of RXX(τ):
1. R XX ( )  R XX ( )
2. R XX ( )  R XX (0)

3. R XX (0)  E X 2 (t )  0 
Power Spectral Densities of Random Processes……
 In case of a discrete-time random process X(n), the

autocorrelation function of X(n) is defined by:
R XX (k )  E X (n) X (n  k )
 Properties of RX(k):
1. R XX ( k )  R XX (k )
2. R XX (k )  R XX (0)

3. R XX (0)  E X 2 (n)  0 
 Two processes X(t) and Y(t) are called (mutually) orthogonal if:
R XY ( )  0 , for all 
 Similarly, the cross-correlation function of two discrete-time

jointly WSS random processes X(n) and Y(n) is defined by:
R XY (k )  E X (n)Y (n  k )
 The various properties of RXY(k) similar to those of RXY(τ) can

be obtained by replacing τ by k in the above equations.

 The power spectral density (or power spectrum) SXX(ω) of a

continuous-time random process X(t) is defined as the Fourier
transform of RXX(τ), i.e. ,

S XX     R XX ( )e  j d

 Thus, taking the inverse Fourier transform of SX(ω), we obtain:

1 
R XX     S XX ( )e j d
2 
 The above equations are known as the Wiener-Khinchin

relations.
 Properties of SXX(ω):
1. S XX ( ) is real and S XX ( )  0
2. S XX ( )  S XX ( )
 
3. E X (t )  R XX (0) 
2 1
2 


S XX ( ) d
 Similarly, the power spectral density SXX(Ω) of a discrete-time

random process X(n) is defined as the Fourier transform of
RXX(k): 
S XX     XX
R ( k ) e  jk
k  
 Thus, taking the inverse Fourier transform of SXX(Ω), we obtain:
1 
R XX (k ) 
2 

S XX ()e jk d
 Properties of SXX(Ω):
1. S XX (  2 )  S XX ()
2. S XX () is real and S XX ()  0
3. S XX ()  S XX ()
 
3. E X (n)  R XX (0) 
2 1
2

S

XX (  ) d

 The cross power spectral density (or cross power spectrum)

SXY(ω) of two continuous-time random processes X(t) and Y(t)
is defined as the Fourier transform of RXY(τ):

S XY     R XY ( )e  j d

 Thus, taking the inverse Fourier transform of SXY(ω), we get:

1 
R XY     S XY ( )e j d
2 

 Properties of SXY(ω):
 Unlike SXX(ω), which is a real-valued function of ω, SXY(ω), in
general, is a complex-valued function.
1. S XY ( )  S YX ( )
2. S XY ( )  S XY ( )
*
 Similarly, the cross power spectral density SXY(Ω) of two

discrete-time random processes X(n) and Y(n) is defined:

S XY ()   XY
R (
k  
k ) e  jk

 Taking the inverse Fourier transform of SXY(Ω), we get:

1 
R XY (k ) 
2 

S XY ()e jk d
 Properties of SXY(ω):
 Unlike SXX(Ω), which is a real-valued function of Ω, SXY(Ω), in
general, is a complex-valued function.
1. S XY (  2 )  S XY ()
2. S XY ()  S YX ()
*
3. S XY ()  S XY ()
Example on Random Processes
Example:
Consider a random process X(t) defined by
X (t )  A cos( 0 t   )
where  0 and A are constants and  is a uniform
random variable over the interval (0, 2 )

a. Find the mean  X (t ).
b. Find the autocorrelation function R XX (t1 , t 2 ).
c. Find the autocovariance function C XX (t1 , t 2 ).
d . Determine whether X (t ) is WSS random process or not.
e. Find the power spectral density of X (t ).

Example on Random Processes Cont’d……
Solution:
a.  X (t )  E  X (t )   E  A cos(  0 t   )   AE cos(  0 t   ) 
But , cos(  0 t   )  cos(  0 t)cos  - sin(  0 t)sin 
  X (t )  E  X (t )   AE cos(  0 t)cos  - sin(  0 t)sin  
 A cos(  0 t) E cos    A sin(  0 t) E sin  
1
E cos   
2
2 0
cos d   0
1
Similarly , E sin   
2
2 0
sin d   0
  X (t )  E X (t )   0

Solution:
b. R XX (t1 , t 2 )  E  X (t1 ) X (t 2 ) 
 E A cos(  0 t1   ) A cos(  0 t 2   ) 
 A 2 E cos(  0 t1   ) cos(  0 t 2   ) 
A2
 E cos  0 (t 2  t1 )  cos(  0 (t1  t 2 )  2 ) 
2
But , E cos  0 (t 2  t1 )   cos  0 (t 2  t1 ) and
E cos(  0 (t1  t 2 )  2 )   0
A2
 R XX (t1 , t 2 )  cos  0 (t 2  t1 )
2
Solution:
c. C XX (t1 , t 2 )  R XX (t1 , t 2 )   X (t1 )  X (t 2 )
A2
 cos  0 (t 2  t1 )  0
2
A2
 C XX (t1 , t 2 )  cos  0 (t 2  t1 )
2
d . Since the mean is constant and the autocorrelation function
depends on time difference only, X (t ) is a WSS random process.

Solution:
e. Since X (t ) is a WSS random process, the autocorrel ation
function can be simply wri tten as :
A2
R XX ( )  cos( 0 )
2
The power spectral density of X (t ) is given by :

S XX ( )  

R XX ( )e  j d
But from Fourier tr ansform pair table , we have :
FT cos( 0 t )   (   0 )   (   0 )
A 2 A 2
 S XX ( )   (   0 )   (   0 )
2 2
Response of Linear Systems to Random Inputs
 If a WSS random process X(t) with autocorrelation function

RXX(τ) is applied to a linear system with impulse response h(t),
then the cross correlation function RXY(τ) and the output
autocorrelation function RYY(τ) are given as follows.
X(t) h(t) Y(t)
R XY ( )  R XX ( ) * h * ( )
And ,
RYY ( )  R XY ( ) * h( )
 R XX ( ) * h * ( ) * h( )
Response of Linear Systems to Random Inputs…..
 Using properties of Fourier transform, we get:

f (t ) 
FT
F ( ) and g (t ) 
FT
G ( )
 f (t ) * g (t ) 
FT
F ( )G ( )
 Then using the above property, the cross and output power
spectral densities can be evaluated as:
 
S XY ( )  FT R XX ( ) * h * ( )  S XX ( ) H * ( )
And ,
S YY ( )  FT RYY ( )  FT R XY ( ) * h( )
2
 S XY ( ) H ( )  S XX ( ) H ( )

Example on Response of Linear Systems
Example:
Consider a WSS random process X(t) with autocorrelation
function given by:
a 
R XX ( )  e , where a is a real positive constant
Let the random process X(t) be applied to the input of an LTI

system with impulse response given by:
h(t )  e  bt u (t ) , where b is a real positive constant
Find the autocorrelation function of the output Y(t) of the

system.

Example on Response of Linear Systems Cont’d……
Solution:
The frequency response H ( ) of the system is :
1
H ( )  FT h (t ) 
j  b
The power spectral density of X (t ) is :
2a
S XX ( )  FT R XX ( )  2
  a2
Then, the power spectral density of Y (t ) is given by :
2  1  2 a 
S YY ( )  S XX ( ) H ( )  2 2  2 
   b    a 
2

Example on Response of Linear Systems Cont’d……
Solution:
a  2b  b  2a 
 S YY ( )  2  2 2 
 2  2 2 
 
a b b   b  a b b   b 
2 2
 
Taking the inverse Fourier tr ansform of both sides of the above
equation, we obtain :
R YY ( )  2
1

a  b2 b
ae


b 
 be
a 


Exercise on Random Processes
1. Consider a random process X(t) defined by
X (t )  A cos( 0 t   )
where  0 and  are constants and A is a uniform
random variable over the interval (0, 2)
b. Find the autocorrel ation function R XX (t1 , t 2 ).
c. Find the autocovari ance function C XX (t1 , t 2 ).

Exercise on Random Processes
2. Consider a random process X(t) defined by:
X (t )  A sin( 0 t   )
where A and  are independent random variables which are
  
uniformly distributed over the intervals[0, 1] and  , 
 2 2
respectively and  0 is a constant.
b. Find the autocorrelation function R XX (t1 , t 2 ).
c. Find the autocovariance function C XX (t1 , t 2 ).

Exercise on Random Processes Cont’d……
3. Two random processes X(t) and Y(t) are given by:
X (t )  A cos(0t   ) and Y (t )  A sin(0t   )

where A and 0 are constants and  is a uniform random
variable over the interval (0, 2 ).
a. Find the cross correlation function of X (t ) and Y (t ).
b. Verify that R XY (- )  R XY ( )

Exercise on Random Processes Cont’d……
4. Consider a discrete-time WSS random process X(n) with

autocorrelation function given by:
R XX (k )  2e |k |
Find the power spectral density of X(n).

04-Random Processes

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

04-Random Processes

Uploaded by

Copyright:

Available Formats

Addis Ababa Science & Technology University

Department of Electrical & Electronics Engineering

Probability and Random Process (EEEg-2114)

Chapter 4: Random Processes

Semester-II, 2013/14 By Welelaw Y. 2

 The theory of random processes was first developed in

 A random process is the mathematical model of an empirical

 Random processes provides useful models for the studies of

 A random process is a family of random variables {X(t), tϵT}

 In a random process {X(t), tϵT}, the index set T is called the

 If the index set T of a random process is discrete, then the

 A discrete-time random process is also called a random

 If T is continuous, then we have a continuous-time random

 In fact, a random process {X(t), tϵT} is a function of two

 For a fixed time t=tk, X(tk, ω) = Xk(ω) is a random variable

Semester-II, 2013/14 By Welelaw Y. 5

 The totality of all sample functions is called an ensemble.

 Of course if both ω and t are fixed, X(tk , ωi) is simply a real

 In the following discussion, we use the notation X(t) to

Semester-II, 2013/14 By Welelaw Y. 6

 If X(t) is a random process, then for fixed t=t1, X1=X(t1)

Semester-II, 2013/14 By Welelaw Y. 7

 For t = t1 and t = t2, X(t) represents two different random

 Similarly f X ( x1 , x2 ,  xn , t1 , t 2  , t n ) represents the nth order

 As in the case of random variables, random processes are often

 The mean of X(t) is defined by:

 In general, μX(t) is a function of time, and it is often called the

Semester-II, 2013/14 By Welelaw Y. 9

 A measure of dependence among the random variables of X(t)

R XX (t1 , t 2 )  EX (t1 ) X ( 2 )

C XX (t1 , t 2 )  CovX (t1 ) , X (t 2 )  EX (t1 )   X (t1 )X (t 2 )   X (t 2 )

Semester-II, 2013/14 By Welelaw Y. 10

 It is clear that if the mean of X(t) is zero, then:

 Note that the variance of X(t) is given by:

Semester-II, 2013/14 By Welelaw Y. 12

ii. Wide-Sense Stationary Processes

 Note that a strict-sense stationary process is also a WSS

Semester-II, 2013/14 By Welelaw Y. 13

 Other types of random processes include:

Semester-II, 2013/14 By Welelaw Y. 14

 The autocorrelation function of a continuous-time random

 In case of a discrete-time random process X(n), the

 Similarly, the cross-correlation function of two discrete-time

 The various properties of RXY(k) similar to those of RXY(τ) can

Semester-II, 2013/14 By Welelaw Y. 17

 The power spectral density (or power spectrum) SXX(ω) of a

 Thus, taking the inverse Fourier transform of SX(ω), we obtain:

 The above equations are known as the Wiener-Khinchin

 Similarly, the power spectral density SXX(Ω) of a discrete-time

 Thus, taking the inverse Fourier transform of SXX(Ω), we obtain:

2. S XX () is real and S XX ()  0

Semester-II, 2013/14 By Welelaw Y. 20

 The cross power spectral density (or cross power spectrum)

 Thus, taking the inverse Fourier transform of SXY(ω), we get:

Semester-II, 2013/14 By Welelaw Y. 21

 Similarly, the cross power spectral density SXY(Ω) of two

Semester-II, 2013/14 By Welelaw Y. 22

 Taking the inverse Fourier transform of SXY(Ω), we get:

where  0 and A are constants and  is a uniform

random variable over the interval (0, 2 )

b. Find the autocorrelation function R XX (t1 , t 2 ).

c. Find the autocovariance function C XX (t1 , t 2 ).

d . Determine whether X (t ) is WSS random process or not.

e. Find the power spectral density of X (t ).