Chapter One: Elements of Digital Communication

Systems with Review on Random and Stochastic

Process

Adama Science and Technology University

December 1, 2021

(ASTU) Digital Communication December 1, 2021 1 / 24

Outlines

1 Introduction

2 Model of digital communication systems

3 Review of Probability Theory

4 Mathematical Definition Stochastic Process

5 Mean Correlation, and Covariance Functions of Weakly Stationary

Process

(ASTU) Digital Communication December 1, 2021 2 / 24

Introduction

The purpose of a communication system is to transmit information

signals from source  destination.
Mostly, the message produced by the source is not electrical in nature.
Thus an input transducer is used to convert the message generated by
the source in to time varying electrical signal called the message
signal .
By using another transducer at the receiver, the original message will
be reproduced at the user destination.
The message signal can be analog form or digital form

(ASTU) Digital Communication December 1, 2021 3 / 24

Cont. . . .

In communication system there are three major parts

(ASTU) Digital Communication December 1, 2021 4 / 24

Cont. . . .

Three ways in which communication systems are classified

Nature of information signal
Analog or digital systems
Unidirectional/bidirectional communication
Simplex or half & full duplex systems
Technique of transmission
Base band or modulated systems

(ASTU) Digital Communication December 1, 2021 5 / 24

Model of digital communication systems

Source encoder converts source information in to digital

Channel encoder encodes digital information
Coded digital signal is then send to the digital
modulator.
Transmitter will amplify and sent the modulated signal
Receives a very weak signal and amplifies.
Demodulator separates coded digital from carrier.
Channel decoder decodes the coded digital signal in its
un-coded form and detects and corrects errors.
Source decoder finally converts the digital information
back to the original signal

(ASTU) Digital Communication December 1, 2021 6 / 24

Basic Digital Communication Transformations

(ASTU) Digital Communication December 1, 2021 7 / 24

Cont. . . .

(ASTU) Digital Communication December 1, 2021 8 / 24

Comparison of Digital and Analog Communication Systems

Digital communication has a number of advantages

Relatively inexpensive digital circuits can be used
Privacy is preserved by using data encryption
Greater dynamic range is possible
Data from voice, video, and data sources can be merged and
transmitted over a common digital transmission system.
In long distance systems, noise does not accumulate from repeater to
repeater.
Errors in detected data are small, even when there is a large amount of
noise on the received signal.
Errors can often be corrected by the use of coding
Digital communication also has disadvantages
Generally more bandwidth is required
Synchronization between the transmitter and receiver is required

(ASTU) Digital Communication December 1, 2021 9 / 24

Review of Probability Theory

Events can in general be classified as:

Random-the event may or may not occur
Certain- the occurrence of the event is inevitable (Incapable of being
avoided or prevented)
Impossible-the event will never occur
Probability is the study of randomness and uncertainty.
Probability theory provides powerful tools to explain, model and
design real world physical systems with some degree of uncertainty.
Some application areas of probability theory include:
Data communication systems
Wireless communication systems
Control systems, etc. . .

(ASTU) Digital Communication December 1, 2021 10 / 24

Cont. . . .
I. Random Experiment
A random experiment is an experiment in which the outcome varies in
an unpredictable manner when the experiment is repeated under the
same conditions.
Examples:
Tossing a coin
Rolling a die
II. Sample Space
The sample space is the set of all possible outcomes of a random
experiment.
The sample space is denoted by Ω and the possible outcomes are
represented by ωi
Ω = {ω1 , ω2 , ω3 , . . .}
III. Event
An event is any subset of the sample of the sample space, Ω
Events can be represented by A, B, C , . . .
(ASTU) Digital Communication December 1, 2021 11 / 24
Cont. . . .
Example-1:
Consider a random experiment of rolling a die once.
Sample Space
Ω = {1, 2, 3, 4, 5, 6}
Some possible events
An event of obtaining even numbers
A = {2, 4, 6}
An event of obtaining numbers less than 4
A = {2, 4, 6}
Example-2:
Consider a random experiment of flipping a fair coin twice.
Sample Space
Ω = {HH, HT , TH, TT }
Some possible events
An event of getting exactly one head
A = {HT , TH}
(ASTU) Digital Communication December 1, 2021 12 / 24
Cont. . . .
An event of getting at least one tail

A = {HT , TH, TT }

An event of getting at most one tail

A = {HT , TH}

Axioms and Properties of Probability

Probability is a rule that assigns a number to each event A in the
sample space, Ω.
In short , the probability of any event A is given by

n(A)
P(A) = (1)
n(Ω)

where
n(A) - Is the number of elements in the event A
n(Ω) - Is the number of elements in the sample space A

(ASTU) Digital Communication December 1, 2021 13 / 24

Cont. . . .

The probability of an event A is a real number which satisfies the

following axioms.
1 Probability is a non-negative number, i.e.,

P(A) ≥ 0

2 Probability of the whole set is unity, i.e.,

P(Ω) = 1

From axioms (1) and (2), we obtain

0 ≤ P(A) ≤ 1

3. Probability of the union of two mutually exclusive (disjoint) events is

the sum of the probability of the events, i.e.,
if A ∩ B = Φ then P(A ∪ B) = p(A) + P(B)
We can generalize axiom (3) for n pairwise mutually exclusive (disjoint)
events.

(ASTU) Digital Communication December 1, 2021 14 / 24

Cont. . . .
If A1 , A2 , A3 , . . . , An is a sequence of n pairwise mutually exclusive
(disjoint) events in the sample space Ω such that Ai ∩ Bj = Φ, for
i 6= j, then
n
X
P (∪ni=1 Ai ) = P(Ai )
i=1
Conditional Probability
The conditional probability of an event A given B, denoted by P(A/B),
is defined as:
P(A ∩ B)
P(A/B) = , P(B) > 0 (2)
P(B)
Similarly, the conditional probability of an event B given A, denoted by
P(B/A), is given by
P(A ∩ B)
P(A/B) = , P(A) > 0 (3)
P(A)
From equations (2) and (3), we will get
P(A ∩ B) = P(A/B)P(B) = P(B/A)P(A) (4)
(ASTU) Digital Communication December 1, 2021 15 / 24
Cont. . . .

Then using equation (4), we will get

P(B/A)P(A) P(A/B)P(A)
P(A/B) = OR P(B/A) = (5)
P(B) P(A)

We know that

P(B) = P(A ∩ B) + P(A ∩ B) = P(B/A)P(A) + P(B/A)P(A)

Thus
P(B/A)P(A)
P(A/B) = (6)
P(B/A)P(A) + P(B/A)P(A)
Similarly
P(A/B)P(B)
P(B/A) = (7)
P(A/B)P(B) + P(A/B)P(B)
Equations (6) and (7) are known as Baye’s Rule

(ASTU) Digital Communication December 1, 2021 16 / 24

Cont. . . .

Baye’s Rule can be extended for n events as follows.

Let events A1 , A2 , A3 , . . . , An be pairwise mutually exclusive (disjoint)
events and their union be the sample space Ω, i.e.
Ai ∩ Aj = Φ and ∪ni=1 Ai = Ω
n
X
P(∪ni=1 Ai ) = P(Ai )
i=1

The events B ∩ Ai and B ∩ Aj are mutually exclusive events.

P(B) = P(B ∩ A1 ) + P(B ∩ A1 ) + . . . + P(B ∩ An )

P(B) = P(B/A1 )P(A1 ) + P(B/A2 )P(A2 ) + . . . + P(B/An )P(An )

(ASTU) Digital Communication December 1, 2021 17 / 24

Cont. . . .

In short,
n
X X
P(B) = P(B ∩ Ai ) = = 1n P(B/Ai )P(Ai ) (8)
i i

Then using equation (4), we will obtain

P(B/Ai )P(Ai )
P(Ai /B) = P (9)
i = 1n P(B/Ai )P(Ai )

(ASTU) Digital Communication December 1, 2021 18 / 24

Cont. . . .
Independence of Events
Two events A and B are said to be statistically independent if and only
if
P(A ∩ B) = P(A)P(B) (10)
Similarly, three events A, B and C are said to be statistically
independent if and only if
P(A ∩ B ∪ C ) = P(A)P(B)P(C ) (11)
Generally, if A1 , A2 , . . ., An are a sequence of independent events, then
n
Y
P (∩ni=1 Ai ) = P(Ai ) (12)
i=1

If A and B are independent, then we have

P(A/B) = P(A)
P(B/A) = P(B)

(ASTU) Digital Communication December 1, 2021 19 / 24

Mathematical Definition Stochastic Process

Let X(t) denote state of throwing coin at time.

The collection of random variables is called Stochastic process.

Definition (1)
We define the mean of the Stochastic process X as:

(
+∞
mx (t) = E {X (t)} = −∞ xfxi dx, When X (t) is a continous
P+∞
i=0 Xi P{X (t)Xi }, When X (t) is a discrete R
(13)

(ASTU) Digital Communication December 1, 2021 20 / 24

Mean Correlation, and Covariance Functions of Weakly
Stationary Process

Variance
Definition (2)
It is a measure of the ”randomness” of the Stochastic process X at time t
(designated by X (t)).

The smaller is Dx (t), The more the samples of X (t) concentrate.

It is equal to the difference between the mean-square value and the
square of the mean.

Dx (t) = D{X (t)} = E {|X (t) − mx (t)|2 }

= E {|X (t)|2 } − |mx (t)|2

(ASTU) Digital Communication December 1, 2021 21 / 24

Cont. . . .

Autocorrelation
Definition (3)
We define the autocorrelation function of the Stochastic process X (t) as:

[4]

CX (t1 , t2 ) = E {[X (t1 ) − mX (t1 )].[X (t2 ) − mX (t2 )]}

= E {X (t1 ).X (t2 )} − mX (t1 ).mX (t2 )]

where
X (t1 ) and X (t2 ) are random variables obtained by observing X (t) at
times t1 and t2 ,respectively.
It is a function of two variables,t1 and t2 ;
It is the measure of the degree to which two time samples of the same
Stochastic process are related

(ASTU) Digital Communication December 1, 2021 22 / 24

Cont. . . .

Covariance
Definition (4)
We define the autocorrelation function of the Stochastic process X (t) as:

RX (t1 , t2 ) = E {X (t1 ).X (t2 )} (14)

where
X (t1 ) and X (t2 ) are random variables obtained by observing X (t) at
times t1 and t2 ,respectively.
It is a function of two variables,t1 and t2 ;
It is the measure of the degree to which two time samples of the same
Stochastic process are related

(ASTU) Digital Communication December 1, 2021 23 / 24

 Thank You

(ASTU) Digital Communication December 1, 2021 24 / 24