EEE301 Communication Theory

03-Multiple Random Variables

DR. THILINI RAJAKARUNA
Content
 Probability and Random Variables:
 Review of probability and random variables
 Cumulative distribution function (CDF)
 Probability density function (PDF), Probability mass function (PMF)
 Moments of a random variable, expectation, variance
 Joint distribution functions, Covariance and Correlation coefficient
 Jacobian and transformation of random variables

Ref:
Communication Systems, Simon Hykin, 4E
Communication Systems Engineering, John G. Proakis, M. Salehi, 2E

EEE301 COMMUNICATION THEORY 2

Multiple Random Variables
Joint CDF
Let 𝑋 and 𝑌 be two random variables defined in the
same sample space Ω.

Define joint CDF as,

𝑭𝐗, 𝐘 𝐱, 𝐲 = 𝐏 𝑿 ≤ 𝒙, 𝒀 ≤ 𝒚 𝒘𝒉𝒆𝒓𝒆 𝒙, 𝒚 ∈ ℝ𝟐

Valid for discrete, continuous or mixed random

variables.

EEE301 COMMUNICATION THEORY 3

Multiple Random Variables
Marginal CDF
Define Marginal CDFs,
𝑭𝑿 𝒙 = 𝑭𝑿, 𝒀 𝒙, ∞ = 𝒍𝒊𝒎 𝑭𝑿, 𝒀 𝒙, 𝒚
𝒚 →∞
𝑭𝒀 𝒚 = 𝑭𝑿, 𝒀 ∞, 𝒚 = 𝒍𝒊𝒎 𝑭𝑿, 𝒀 𝒙, 𝒚
𝒙 →∞
y y

x x

EEE301 COMMUNICATION THEORY 4

Properties of joint CDF

EEE301 COMMUNICATION THEORY 5

Question 1:

EEE301 COMMUNICATION THEORY 6

Multiple discrete random variables:
Joint PMF
Let 𝑋 and 𝑌 be two random variables defined in the same sample Ω. Define
joint PMF as,

𝑷𝐗, 𝐘 𝐱, 𝒚 = 𝐏 𝑿 = 𝒙, 𝒀 = 𝒚 𝒘𝒉𝒆𝒓𝒆 𝒙, 𝒚 ∈ ℝ𝟐

𝑷𝐗, 𝐘 𝐱, 𝒚 = 𝐏 𝑿 = 𝒙 𝒂𝒏𝒅 𝒀 = 𝒚

Define the range of X,Y 𝑹𝑿, 𝒀 = 𝒙, 𝒚 𝑷𝑿, 𝒀 𝒙, 𝒚 > 𝟎

𝑷𝑿, 𝒀 𝒙, 𝒚 = 𝟏
𝒙,𝒚∈ℝ𝟐

EEE301 COMMUNICATION THEORY 7

Multiple discrete random variables:
Marginal PMF
Let 𝑋 and 𝑌 be two random variables defined by,
𝑃𝑋 , 𝑌 𝑥, 𝑦 = 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦 𝑤ℎ𝑒𝑟𝑒 𝑥, 𝑦 ∈ ℝ2

Marginal PMFs 𝑃𝑋 𝑥 , 𝑃𝑌 𝑦 ,

𝑷𝑿 𝒙 = 𝑷𝑿, 𝒀 𝒙, 𝒚𝒋
𝒚𝒋 𝑹𝒚
∈

𝑷𝒀 𝒚 = 𝑷𝑿, 𝒀 𝒙𝒊, 𝒚
𝒙𝒊 ∈𝑹𝒙

EEE301 COMMUNICATION THEORY 8

Conditional PMF and Chain Rule
Let 𝑋 and 𝑌 be two random variables defined by,
𝑃𝑋 , 𝑌 𝑥, 𝑦 = 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦 𝑤ℎ𝑒𝑟𝑒 𝑥, 𝑦 ∈ ℝ2

The conditional PMF of 𝑋 given 𝑌 = 𝑦 is defined as,

Chain rule,

EEE301 COMMUNICATION THEORY 9

Independent discrete random variables
Let 𝑋 and 𝑌 be two random variables defined by,
𝑃𝑋 , 𝑌 𝑥, 𝑦 = 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦 𝑤ℎ𝑒𝑟𝑒 𝑥, 𝑦 ∈ ℝ2

𝑋 and 𝑌 are independent if,

𝑷𝑿, 𝒀 𝒙, 𝒚 = 𝑷X(x) 𝑷Y(y) for all 𝒙, 𝒚 ∈ ℝ𝟐
Then,
𝑷𝑿/𝒀 𝒙/𝒚 = 𝑷X(x) for all 𝒙, 𝒚 ∈ ℝ𝟐

EEE301 COMMUNICATION THEORY 10

Question 2:
Let 𝑋 ∼ 𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝑝) and Y∼Bernoulli(q) be independent,
where 0 < 𝑝 < 1, 0 < 𝑞 < 1.

Find the joint PMF and joint CDF for X and Y.

EEE301 COMMUNICATION THEORY 11

Multiple continuous random variables:
Joint PDF
Let 𝑋 and 𝑌 be two random variables defined in the same sample space Ω.
𝑋 and 𝑌 are jointly continuous random variables if their joint CDF is continuous in
both x and y.
Define joint PDF,
𝒙 𝒚
𝑭𝑿, 𝒀 𝒙, 𝒚 = 𝒇𝑿, 𝒀 𝒖, 𝒗 𝒅𝒖 𝒅𝒗, 𝒙, 𝒚 ∈ ℝ𝟐
−∞ −∞
Then
𝜕2𝐹𝑋, 𝑌 𝑥, 𝑦
𝑓𝑋, 𝑌 𝑥, 𝑦 =
𝜕𝑥𝜕𝑦
∞ ∞
𝑓𝑋 , 𝑌 𝑥, 𝑦 𝑑𝑥 𝑑𝑦 = 1
−∞ −∞
EEE301 COMMUNICATION THEORY 12
Question 3:

EEE301 COMMUNICATION THEORY 13

Multiple continuous random variables:
Marginal PDF’s

EEE301 COMMUNICATION THEORY 14

Independence of two random variables

EEE301 COMMUNICATION THEORY 15

Question 4:

EEE301 COMMUNICATION THEORY 16

Conditional PDF
Let 𝑋 and 𝑌 be two continuous random variables with joint PDF
𝑓𝑋 , 𝑌 𝑥, 𝑦 ,
Then the conditional PDF of Y given X is,

EEE301 COMMUNICATION THEORY 17

Question 5:

EEE301 COMMUNICATION THEORY 18

Question 6:
The random variable X is selected at random from the unit interval. The
random variable Y is then selected at random from the interval (0,X).

Find the CDF of Y.

Answer:
𝒚
𝒀≤𝒚 𝒊𝒇 𝟎 ≤ 𝒚 ≤ 𝒙
𝑷 𝑿=𝒙 = 𝒙
𝟏 𝒊𝒇 𝒚 ≥ 𝒙
𝑭𝒀 𝒚 = 𝒚 – 𝒚 𝒍𝒏(𝒚)

EEE301 COMMUNICATION THEORY 19

Expectation of two random variables:
Correlation and covariance
Let (𝑋, 𝑌)~𝑓𝑋, 𝑌 (𝑥, 𝑦) and let 𝑔(𝑥, 𝑦) be a function of 𝑥 and 𝑦. The expectation of
𝑔(𝑋, 𝑌) is defined as,

The correlation of X and Y is defined as E(XY)

The covariance of X and Y is defined as,

EEE301 COMMUNICATION THEORY 20

Correlation and covariance

EEE301 COMMUNICATION THEORY 21

Properties:
For two random variables X and Y,

 𝐶𝑂𝑉(𝑋, 𝑋) = 𝑉𝐴𝑅(𝑋)

 COV(X,Y) = COV(Y,X)

 COV(aX, Y) = a COV(X,Y), where a is a constant

 COV(X + c,Y) = COV(X,Y), where c is a constant

 If X and Y are independent random variables, then COV(X,Y) = 0

EEE301 COMMUNICATION THEORY 22

Correlation coefficient

EEE301 COMMUNICATION THEORY 23

Properties of the correlation coefficient:
1. −1 ≤ 𝜌(𝑋, 𝑌) ≤ 1;
2. if 𝜌(𝑋, 𝑌) = 1, then 𝑌 = 𝑎𝑋 + 𝑏, where 𝑎 > 0;
3. if 𝜌(𝑋, 𝑌) = −1, then 𝑌 = 𝑎𝑋 + 𝑏, where 𝑎 < 0;
4. 𝜌(𝑎𝑋 + 𝑏, 𝑐𝑌 + 𝑑) = 𝜌(𝑋, 𝑌) for 𝑎, 𝑐 > 0.

EEE301 COMMUNICATION THEORY 24

Question 7:

EEE301 COMMUNICATION THEORY 25

Transformation of random variables

EEE301 COMMUNICATION THEORY 26

Transformation of random variables

EEE301 COMMUNICATION THEORY 27

Question 8:

EEE301 COMMUNICATION THEORY 28