Digital Communication & Information Theory

Course Code TC-311

LECTURE 5

Dr Sunila Akbar
 Random Signals
Random Variables
▪ All useful message signals appear random; that is, the receiver does
not know, a priori, which of the possible waveform has been sent.
▪ Let a random variable X(A) represent the functional relationship
between a random event A and a real number.
▪ The (cumulative) distribution function FX(x) of the random variable X is
given by
FX ( x) = P( X  x)
▪ Another useful function relating to the random variable X is the
probability density function (pdf)
dFX ( x)
PX ( x) =
dx
DCIT - Dr Sunila Akbar 2
 Ensemble Averages
 ▪ The first moment of a probability
m X = E{ X } =  xp
−
X ( x)dx distribution of a random variable X is
called mean value mX, or expected
value of a random variable X

▪ The second moment of a probability
E{ X 2 } = 
−
x 2 p X ( x)dx distribution is the mean-square value
of X

var( X ) = E{( X − m X ) 2 } ▪ Central moments are the moments of

the difference between X and mX and

the second central moment is the
= 
−
( x − m X ) 2 p X ( x)dx variance of X

var( X ) = E{ X 2 } − E{ X }2 ▪ Variance is equal to the difference

between the mean-square value and
the square of the mean

DCIT - Dr Sunila Akbar 3

 Random Processes
▪ A random process X(A, t) can be viewed as a function of two variables:
an event A and time.
▪ X(𝐴𝑘 ,t) is called a
sample function

▪ X(A, 𝑡𝑘 ) is simply
a random variable

▪ X(𝐴𝑘 , 𝑡𝑘 ) is just a
number

DCIT - Dr Sunila Akbar 4

 Statistical Averages of a Random Process
▪ A random process whose distribution functions are continuous can be
described statistically with a probability density function (pdf).
▪ A partial description consisting of the mean and autocorrelation
function are often adequate for the needs of communication systems.
▪ Mean of the random process X(t)

E{ X (tk )} =  xpX k ( x) dx = mX (tk )
−

▪ Autocorrelation function of the random process X(t)

RX (t1 , t2 ) = E{ X (t1 ) X (t2 )}
DCIT - Dr Sunila Akbar 5
 Stationarity
▪ A random process X(t) is said to be stationary in the strict sense if
none of its statistics are affected by a shift in the time origin.
▪ A random process is said to be wide-sense stationary (WSS) if two of
its statistics, its mean and autocorrelation function, do not vary with a
shift in the time origin.
E{ X (t )} = mX = a constant

RX (t1 , t2 ) = RX (t1 − t2 )

DCIT - Dr Sunila Akbar 6

Autocorrelation of a Wide-Sense Stationary
Random Process
▪ For a wide-sense stationary process, the autocorrelation
function is only a function of the time difference τ = t1 – t2;
RX ( ) = E{ X (t ) X (t +  )} for −    
▪ Properties of the autocorrelation function of a real-valued
wide-sense stationary process are
1. RX ( ) = RX ( − ) Symmetrical in τ about zero
2. RX ( )  RX (0) for all  Maximum value occurs at the origin
Autocorrelation and power spectral
3. RX ( )  GX ( f )
density form a Fourier transform pair
Value at the origin is equal to the
4. RX (0) = E{ X 2 (t )} average power of the signal

DCIT - Dr Sunila Akbar 7

 Time Averaging and Ergodicity
▪ When a random process belongs to a special class, known as an ergodic process,
its time averages equal its ensemble averages.

▪ The statistical properties of such processes can be determined by time averaging

over a single sample function of the process.

▪ A random process is ergodic in the mean if

T /2
1
mX = lim
x → T 
−T / 2
X (t )dt
▪ It is ergodic in the autocorrelation function if
T /2
1
RX ( ) = lim
x → T 
−T / 2
X (t ) X (t +  )dt

DCIT - Dr Sunila Akbar 8

 Ergodic Process

• For Ergodic process, fundamental electrical parameters can be

evaluated using signal statistics. Following is the summary of these
relations:

DCIT - Dr Sunila Akbar 9

Ergodic Process

DCIT - Dr Sunila Akbar 10

 Power Spectral Density and Autocorrelation
▪ A random process X(t) can generally be classified as a power signal
having a power spectral density (PSD) GX(f )
▪ Principal features of PSD function
1. GX ( f )  0 Nonnegative and is always real
valued
2. GX ( f ) = GX (− f ) for X(t) real-valued

3. GX ( f )  RX ( ) PSD and autocorrelation form a

Fourier transform pair
∞

4. 𝑃𝑋 = න 𝐺𝑋 (𝑓)𝑑𝑓 Relationship between average

normalized power and PSD
−∞
DCIT - Dr Sunila Akbar 11
 Example of calculation of
Autocorrelation Rx(τ)

DCIT - Dr Sunila Akbar 12

 Conclusion
▪ Rx(τ) provides information
regarding signal bandwidth

▪ If Rx(τ) ramps down slowly, we are

dealing with a low-bandwidth
signal

▪ If Rx(τ) ramps down fast, we are

dealing with a high-bandwidth
signal

DCIT - Dr Sunila Akbar 13

 Problem – Power Spectral Density
▪ Chapter 1: DCs – Sklar (Exercise 1.7)

DCIT - Dr Sunila Akbar 14

 Solution
▪ 1.7
1) X(f) is real ✓
2) 𝑋(𝑓) ≥ 0 ✓ YES
3) 𝑋 𝑓 = 𝑋(−𝑓) ✓

1) X(f) is real ✓
2) 𝑋(𝑓) ≥ 0
3) 𝑋 𝑓 = 𝑋(−𝑓) 
✓ NO
DCIT - Dr Sunila Akbar 15
 Solution
• 1.7.
1) X(f) is real
2) 𝑋 𝑓 ≥ 0
✓
✓ NO
3) 𝑋 𝑓 = 𝑋(−𝑓) 

YES
DCIT - Dr Sunila Akbar 16