Chapter 3

Multiple Random Variables

By Fikadu M.
 Bivariate Random Variables:
2

 Let S be the sample space of a random experiment. Let X and Y be two

r.v.'s. Then the pair (X,Y) is called a bivariate r.v. (or two-dimensional
random vector) if each of X and Y associates a real number with every
element of S.
 The range space of the bivariate r.v. (X, Y) is denoted by R𝑥𝑦 and defined
by:
R𝑥𝑦 = {(x, y) : τ ϵ S and X(τ ) = x , Y(τ ) = y}
 Discrete bivariate r.v. – if both X and Y are discrete
Continuous bivariate r.v. – if both continuous themselves.
Mixed – if one continuous and the other discrete

By Fikadu Mulugeta 06-May-20

 Ex. Consider an experiment of tossing a fair coin twice. Let (X, Y) be a
3
bivariate r.v., where X is the number of heads and Y is the number of
tails that occurs in the two tosses.
(a) What is the range 𝑅𝑥 of X?
(b) What is the range 𝑅𝑦 of Y?
(c) Find and sketch the range 𝑅𝑥𝑦 of (X, Y).
(d) Find P(X = 2, Y = 0), P(X = 0, Y = 2), and P(X = 1, Y = 1).
S = {HH, HT, TH, TT)
a) 𝑅𝑥 = {0, 1,2}
b) 𝑅𝑦 = {0, 1,2}
c) 𝑅𝑥𝑦 = {(2, 0), (1 1), (0, 2)}

By Fikadu Mulugeta 06-May-20

4 d) Since the coin is fair, we have
P(X = 2, Y = 0) = P(HH} = 1/4
P(X = 0, Y = 2) = P{TT) = 1/4
P(X= 1, Y = 1)= P{HT, TH} = 1/2

By Fikadu Mulugeta 06-May-20

 Joint Distribution Functions
5
A. Joint cumulative distribution function
 The joint cdf of X and Y, denoted by 𝐹𝑥𝑦 (x, y), is the function defined by
Fxy (x, y) = P(X ≤ x, Y ≤y)
Let A ={τ ϵ S, X(τ ) ≤ x } and B ={τ ϵ S, Y(τ ) ≤ y }
A ⋂ B ={τ ϵ S, X(τ ) ≤ x and Y(τ ) ≤ y }
P(A) = 𝐹𝑥 (x) and P(B) = 𝐹𝑦 (y)
Fxy (x, y) = P(A ⋂ B)

B. Independent Random Variables:

If X and Y are independent r.v.
Fxy (x, y) = P(A ⋂ B) = P(A) P(B) = 𝐹𝑥 (x) 𝐹𝑦 (y)
 Fxy (x, y) = 𝐹𝑥 (x) 𝐹𝑦 (y)
By Fikadu Mulugeta 06-May-20
6 C. Properties of 𝐹𝑥𝑦 (x, y)
1. 0 ≤ 𝐹𝑥𝑦 (x, y) ≤ 1
2. If x1 ≤ x2 , and y1 ≤ 𝑦2 , then:
𝐹𝑥𝑦 (x1 , y1 ) ≤ 𝐹𝑥𝑦 (x2 , y1 ) ≤ 𝐹𝑥𝑦 (x2 , y2 ) and
𝐹𝑥𝑦 (x1 , y1 ) ≤ 𝐹𝑥𝑦 (x1 , y2 ) ≤ 𝐹𝑥𝑦 (x2 , y2 )

3. 𝑥→∞
lim 𝐹𝑥𝑦 (x, y) = 𝐹𝑥𝑦 (∞, ∞) = 1
𝑦→∞
4. lim 𝐹𝑥𝑦 (x, y) = 𝐹𝑥𝑦 ( − ∞, 𝑦) = 0
𝑥→−∞
𝑙𝑖𝑚 𝐹𝑥𝑦 (x, y) = 𝐹𝑥𝑦 (𝑥, −∞) = 0
𝑦→−∞

By Fikadu Mulugeta 06-May-20

7 5. lim+ 𝐹𝑥𝑦 (x, y) = 𝐹𝑥𝑦 (𝑎+ , 𝑦) = 𝐹𝑥𝑦 (a, y)
𝑥→𝑎
lim+ 𝐹𝑥𝑦 (x, y) = 𝐹𝑥𝑦 (x, 𝑏 + ) = 𝐹𝑥𝑦 (x, b)
𝑦→𝑏

6. P(𝑥1 < X ≤ 𝑥2 , Y ≤ y) = 𝐹𝑥𝑦 (𝑥2 , y) - 𝐹𝑥𝑦 (𝑥1 , y)

P(X ≤ x , 𝑦1 < Y ≤ 𝑦2 ) = 𝐹𝑥𝑦 (x, 𝑦2 ) - 𝐹𝑥𝑦 (x, 𝑦1 )

7. If x1 ≤ x2 , and y1 ≤ 𝑦2 , then:
𝐹𝑥𝑦 (x2 , y2 ) - 𝐹𝑥𝑦 (x2 , y1 ) - 𝐹𝑥𝑦 (x1 , y2 ) + 𝐹𝑥𝑦 (x1 , y1 ) ≥ 0
i.e. P(𝑥1 < X ≤ 𝑥2 , 𝑦1 < Y ≤ 𝑦2 ) ≥ 0

By Fikadu Mulugeta 06-May-20

8

Q1: Show that

P(𝑥1 < X ≤ 𝑥2 , 𝑦1 < Y ≤ 𝑦2 ) =
𝐹𝑥𝑦 (x2 , y2 ) - 𝐹𝑥𝑦 (x2 , y1 ) - 𝐹𝑥𝑦 (x1 , y2 ) + 𝐹𝑥𝑦 (x1 , y1 )

By Fikadu Mulugeta 06-May-20

9 (𝑥1 < X ≤ 𝑥2 , Y ≤ 𝑦2 ) = (𝑥1 < X ≤ 𝑥2 , Y ≤ 𝑦1 ) ⋃ (𝑥1 < X ≤ 𝑥2 , 𝑦1 < Y ≤ 𝑦2 )
Since disjoint sets:
P(𝑥1 < X ≤ 𝑥2 , Y ≤ 𝑦2 ) = P(𝑥1 < X ≤ 𝑥2 , Y ≤ 𝑦1 ) + P(𝑥1 < X ≤ 𝑥2 , 𝑦1 < Y ≤ 𝑦2 )
P(𝑥1 < X ≤ 𝑥2 , 𝑦1 < Y ≤ 𝑦2 ) = P(𝑥1 < X ≤ 𝑥2 , Y ≤ 𝑦2 ) - P(𝑥1 < X ≤ 𝑥2 , Y ≤ 𝑦1 )
= 𝐹𝑥𝑦 (x2 , y2 ) - 𝐹𝑥𝑦 (x1 , y2 ) – [𝐹𝑥𝑦 (x2 , y1 ) - 𝐹𝑥𝑦 (x1 , y1 ) ]
= 𝐹𝑥𝑦 (x2 , y2 ) - 𝐹𝑥𝑦 (x1 , y2 ) – 𝐹𝑥𝑦 (x2 , y1 ) + 𝐹𝑥𝑦 (x1 , y1 )
And since it’s probability:
P(𝑥1 < X ≤ 𝑥2 , 𝑦1 < Y ≤ 𝑦2 ) ≥ 0
𝐹𝑥𝑦 (x2 , y2 ) - 𝐹𝑥𝑦 (x1 , y2 ) – 𝐹𝑥𝑦 (x2 , y1 ) + 𝐹𝑥𝑦 (x1 , y1 ) ≥ 0

By Fikadu Mulugeta 06-May-20

10 D. Marginal Distribution Functions:
lim (X ≤ x, Y ≤y) = (X ≤ x, Y ≤∞) = (X ≤ x)
𝑦→∞

Hence:
lim 𝐹𝑥𝑦 (x, y) = 𝐹𝑥𝑦 (x, ∞) = 𝐹𝑥 (x)
𝑦→∞

Marginal cdf of X
lim 𝐹𝑥𝑦 (x, y) = 𝐹𝑥𝑦 (∞, 𝑦) = 𝐹𝑦 (y)
𝑥→∞
Marginal cdf of Y

By Fikadu Mulugeta 06-May-20

11
Ex. The joint cdf of a bivariate r.v. (X, Y) is given by:

Find the marginal cdf's of X and Y.

By Fikadu Mulugeta 06-May-20

 Discrete r.v.- Joint pmf
12

 Let (X, Y) be a discrete bivariate r.v. and take on the values (𝑥𝑖 , 𝑦𝑗 )
for certain integers i and j.
𝑝𝑥𝑦 (𝑥𝑖 , 𝑦𝑗 ) = P(X = 𝑥𝑖 ,Y = 𝑦𝑗 ) ------------joint pmf of X and Y

 Properties:
1. 𝑝𝑥𝑦 (𝑥𝑖 , 𝑦𝑗 ) ϵ [0,1]
2. σ𝑥𝑖 σ𝑦𝑗 𝑝𝑥𝑦 (𝑥𝑖 , 𝑦𝑗 ) = 1
3. P[(X, Y) ϵ A] = σ(𝑥𝑖 , 𝑦𝑗) 𝜖 𝑅𝐴 σ 𝑝𝑥𝑦 (𝑥𝑖 , 𝑦𝑗 )
The joint cdf becomes:
𝐹𝑥𝑦 (x, y) = σ𝑥𝑖 ≤𝑥 σ𝑦𝑗≤𝑦 𝑝𝑥𝑦 (𝑥𝑖 , 𝑦𝑗 )

By Fikadu Mulugeta 06-May-20

13 Marginal pmf:
 Suppose that for a fixed value X = 𝑥𝑖 , the r.v. Y can take on only the
possible values 𝑌𝑗 (j = 1, 2,. . . , n). Then:
Marginal pmf of X
P(X = 𝑥𝑖 ) = 𝑝𝑥 (𝑥𝑖 ) = σ𝑦𝑗 𝑝𝑥𝑦 (𝑥𝑖 ,𝑦𝑗 ) -------Over all (𝑥𝑖 ,𝑦𝑗 ) with
fixed 𝑥𝑖
Marginal pmf of Y
P(Y = 𝑦𝑗 ) = 𝑝𝑦 (𝑦) = σ𝑥𝑖 𝑝𝑥𝑦 (𝑥𝑖 ,𝑦𝑗 ) --------Over all (𝑥𝑖 ,𝑦𝑗 ) with
fixed 𝑦𝑗
 If independent r.v.
𝑝𝑥𝑦 (𝑥𝑖 ,𝑦𝑗 ) = 𝑝𝑥 (𝑥𝑖 ) 𝑝𝑦 (𝑦)

By Fikadu Mulugeta 06-May-20

14
Continuous r.v.- Joint pdf

 Let (X, Y) be a continuous bivariate r.v. with cdf 𝐹𝑥𝑦 (x , y)

𝜕2 𝐹𝑥𝑦 (x , y)
𝑓𝑥𝑦 (x , y) = 𝜕𝑥 𝜕𝑦
----- joint pdf of ( X, Y)

To get the cdf:

𝑥 𝑦
𝐹𝑥𝑦 (x , y) = ‫׬‬−∞ ‫׬‬−∞ 𝑓𝑥𝑦 (τ , η) dτ dη

By Fikadu Mulugeta 06-May-20

15  Properties:
1. 𝑓𝑥𝑦 (x , y) ≥ 0
∞ ∞
2. ‫׬‬−∞ ‫׬‬−∞ 𝑓𝑥𝑦 (𝑥 , 𝑦) dx dy = 1
3. 𝑓𝑥𝑦 (𝑥 , 𝑦) is continuous for all values of x or y except possibly a
finite set.
4. P[(X, Y) ∈ A] =‫ 𝑥( 𝑦𝑥𝑓 ׬ 𝑅׬‬, 𝑦) dx dy
𝐴
𝑏 𝑑
5. P(a < X ≤ b, c< Y ≤ d) = ‫𝑥( 𝑦𝑥𝑓 𝑐׬ 𝑎׬‬ , 𝑦) dx dy

 Since P(X = a) = 0 = P(Y = c), it follows that:

P(a < X ≤ b, c< Y ≤ d) = P(a ≤ X ≤ b, c ≤ Y ≤ d) = P(a ≤ X < b, c ≤ Y < d)
𝑏 𝑑
= P(a < X < b, c < Y < d) = ‫ 𝑥( 𝑦𝑥𝑓 𝑐׬ 𝑎׬‬, 𝑦) dx dy

By Fikadu Mulugeta 06-May-20

16 Marginal pdf
𝑥 ∞
 𝐹𝑥 (x) = 𝐹𝑥𝑦 (x, ∞) = ‫׬‬−∞ ‫׬‬−∞ 𝑓𝑥𝑦 (τ , η) d𝜏 d𝜂
𝑑𝐹𝑥 (x) ∞
𝑓𝑥 (x) = = ‫׬‬−∞ 𝑓𝑥𝑦 (𝑥 , η) d𝜂
𝑑𝑥

Hence:
∞
𝑓𝑥 (x) = ‫׬‬−∞ 𝑓𝑥𝑦 (𝑥 , y) dy ------marginal pdf of X
∞
𝑓𝑦 (y) = ‫׬‬−∞ 𝑓𝑥𝑦 (𝑥 , y) dx ------marginal pdf of Y

By Fikadu Mulugeta 06-May-20

17  If X and Y are independent r.v.
Fxy (x, y) = 𝐹𝑥 (x) 𝐹𝑦 (y)
𝜕2 Fxy (x, y) 𝜕 𝜕
= 𝐹 (x) 𝜕𝑦 𝐹𝑦 (y)
𝜕𝑥𝜕𝑦 𝜕𝑥 𝑥

fxy (x, y) = 𝑓𝑥 (x) 𝑓𝑦 (y)

By Fikadu Mulugeta 06-May-20

18 Ex. The joint pdf of a bivariate r.v. (X, Y) is given by:

where k is a constant.
(a) Find the value of k.
(b) Find the marginal pdf's of X and Y.
(c) Are X and Y independent?

By Fikadu Mulugeta 06-May-20

19  Since

By Fikadu Mulugeta 06-May-20

20

 Since fxy (x, y) ≠ 𝑓𝑥 (x) 𝑓𝑦 (y) , X and Y are not independent.

By Fikadu Mulugeta 06-May-20

 Conditional distributions
21
A. Conditional pmf
 If (X, Y) is a discrete bivariate r.v. with joint pmf 𝑝𝑋𝑌 (𝑥𝑖 , 𝑦𝑗 ) then the conditional
pmf of Y, given that X = 𝑥𝑖 , is defined by:
𝑝𝑋𝑌 (𝑥𝑖 ,𝑦𝑗 )
𝑝𝑌|𝑋 𝑦𝑗 𝑥𝑖 = 𝑝𝑋 (𝑥𝑖 )
, pX xi > 0

 Similarly:
𝑝𝑋𝑌 (𝑥𝑖 ,𝑦𝑗 )
𝑝𝑋|𝑌 (𝑥𝑖 |𝑦𝑗 ) = 𝑝𝑌 (𝑦𝑗 )
, 𝑝𝑌 (𝑦𝑗 ) > 0

Properties of 𝑝𝑌|𝑋 𝑦𝑗 𝑥𝑖 :
1. 0 ≤ 𝑝𝑌|𝑋 𝑦𝑗 𝑥𝑖 ≤ 1;
2. σ𝑦𝑗 𝑝𝑌|𝑋 𝑦𝑗 𝑥𝑖 = 1
3. If independent, 𝑝𝑌|𝑋 𝑦𝑗 𝑥𝑖 = 𝑝𝑌 (𝑦𝑗 ) and 𝑝𝑋|𝑌 (𝑥𝑖 |𝑦𝑗 ) = 𝑝𝑋 (𝑥𝑖 )

By Fikadu Mulugeta 06-May-20

22 B. Conditional pdf
 If (X, Y) is a continuous bivariate r.v. with joint pdf 𝑝𝑋𝑌 (𝑥 , 𝑦) then the
conditional pdf of Y, given that X =x, is defined by:
𝑓𝑋𝑌 (x,y)
𝑓𝑌|𝑋 (y|𝑥)= 𝑓𝑥 (𝑥)
, 𝑓𝑥 (𝑥) > 0.
 Similarly:
𝑓𝑋𝑌 (x,y)
𝑓𝑋|𝑌 (x|𝑦) = , 𝑓𝑦 (𝑦) > 0
𝑓𝑦 (𝑦)

Properties of 𝑓𝑌|𝑋 (y|𝑥):

1. 𝑓𝑌|𝑋 (y|𝑥) ≥ 0
∞
2. ‫׬‬−∞ 𝑓𝑌|𝑋 (y|𝑥) 𝑑𝑦 = 1
3. What if X and Y are independent ?
By Fikadu Mulugeta 06-May-20
 Covariance and correlation
23
Joint Moment:
The (k, n) th moment of a bivariate r.v. (X, Y) is defined by

 If n = 0, we get E(𝑋 𝑘 ) ………… 𝑘 𝑡ℎ moment of X

 𝑚1 0 = E(X) = 𝜇𝑋
 If k = 0, we get E(𝑌 𝑛 ) ………… 𝑛𝑡ℎ moment of Y
 𝑚0 1 = E(Y) = 𝜇𝑌

By Fikadu Mulugeta 06-May-20

24  If (X, Y) is a discrete bivariate r.v.,

 Similarly:

By Fikadu Mulugeta 06-May-20

25  If (X, Y) is a continuous bivariate r.v.,

 Similarly:

By Fikadu Mulugeta 06-May-20

 Correlation:
26
 Is the (1, 1)𝑡ℎ joint moment of (X, Y)
𝑚1 1 = E(X , Y)
 If E(X , Y) = 0, then we say that X and Y are orthogonal
Covariance:


 If Cov(X, Y) = 0, then we say that X and Y are uncorrelated

 Uncorrelated if independent : E(XY) = E(X)E(Y)
 But the converse is not necessarily true.
 Correlation coefficient:
Generally:

By Fikadu Mulugeta 06-May-20

 E.g. Suppose the joint pmf of a bivariate r.v. (X, Y) is given by:
27

(a) Are X and Y independent?

(b) Are X and Y uncorrelated?

By Fikadu Mulugeta 06-May-20

 a) Marginal pmf’s are:
28

Thus not Independent.

By Fikadu Mulugeta 06-May-20

 b) The expectations are
29

Thus uncorrelated.

By Fikadu Mulugeta 06-May-20

 Conditional means and variances
30
Discrete:
 If (X, Y) is a discrete bivariate r.v. with joint pmf 𝑝𝑥𝑦 (𝑋𝑖 , 𝑌𝑗 ), then the conditional
mean of Y, given that X = xi, is defined by:
…………………….a function of 𝑋𝑖 alone.

 The conditional variance of Y, given that X = xi, is defined by:

By Fikadu Mulugeta 06-May-20

 Continuous:
31
 If (X, Y) is a continuous bivariate r.v. with joint pdf 𝑓𝑥𝑦 (x, y), the
conditional mean of Y, given that X = x, is defined by:

 The conditional variance of Y, given that X = x, is defined by

By Fikadu Mulugeta 06-May-20

 N-variate random variables
32
 Given an experiment, the n-tuple of r.v.'s (𝑋1 , 𝑋2 , 𝑋3 ,…, 𝑋𝑁 ) is called an n-
variate r.v. if each 𝑋𝑖 , i = 1, 2, . . . , n, associates a real number with every
sample point τ ϵ S.
 Thus, an n-variate r.v. is simply a rule associating an n-tuple of real numbers
with every τ ϵ S.
 Let (𝑋1 , 𝑋2 , 𝑋3 ,…, 𝑋𝑁 ) be an n-variate r.v. on S. Then its joint cdf is defined as:

 Joint pmf:
 The probability of any n-dimensional event A:

06-May-20
By Fikadu Mulugeta
33
Properties of 𝑝𝑥1 . . . 𝑥𝑛 (𝑥1 , . . . , 𝑥𝑛 ):
1.
2.

 Marginal and conditional pmf, pdf and cdf

 Covariance and correlation
 N-variate mean and moments

06-May-20
By Fikadu Mulugeta
 Special distributions
34
A. Multinomial Distribution:
 extension of the binomial distribution.
 An experiment is termed a multinomial trial with parameters 𝑝1 ,𝑝2 ,…,𝑝𝑛 if it has
the following conditions:
1. The experiment has k possible mutually exclusive and exhaustive outcomes, say
A1, A2,..., Ak.
2.
 pmf for multinomial r.v. with parameters (n, 𝑝1 ,𝑝2 ,…,𝑝𝑘 ) is given by:

 When k = 2, it reduces to the binomial distribution.

By Fikadu Mulugeta 06-May-20

 Binomial
35  A r.v. X is called a binomial r.v. with parameters (n, p) if its pmf is:

…..binomial coefficient

The cdf becomes:

By Fikadu Mulugeta 06-May-20

 B. Bivariate Normal Distribution:
36
 A bivariate r.v. (X, Y) is said to be a bivariate normal (or Gaussian) r.v. if its joint pdf is
given by:

where

By Fikadu Mulugeta 06-May-20

37 Quiz
The joint pdf of a bivariate r.v. (X, Y) is given by:

Where k is constant.
a) Find the value of k.
b) Are X and Y independent?
c) Find P(X + Y < 1).

By Fikadu Mulugeta 06-May-20

 Solution:
38
a)

k=4
b)

c)

By Fikadu Mulugeta 06-May-20