Professional Documents
Culture Documents
Term 3, 2023
1 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Outline
3 Extra notes
Proofs
2 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Introduction
Many situations involve 2 (or more) random variables, and it is
useful to study both random variables together (or, jointly ). Doing
so will lead to important concepts such as independence of random
variables, and sum of random variables.
Example 1
As early as Weeks 1 and 3, we encountered the following example
involving 2 discrete random variables:
Data from the last 100 job applicants at a company resulted in the
following table. One of the applicants is selected at random.
Example 1 (continued)
Then, after converting all the numbers in the table on Slide 3 into
probabilities, we get:
4 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Joint pmf
For discrete RV’s, their joint distribution is specified using a joint
probability mass function.
Definition
Let X and Y be discrete random variables. The joint probability
mass function (joint pmf) f (x, y) is defined as
f (x, y) := P (X = x) ∩ (Y = y) .
XX
Due to the Axioms, we must have f (x, y) = 1, where
all x all y
the sum is taken over all x and y values that X and Y can take
with positive probabilities.
Marginal pmf
Definition (continued)
The marginal probability mass function (marginal pmf) of X,
denoted by fX , is just the pmf of X, and can be computed using
X
fX (x) := P(X = x) = f (x, y).
all y
Similarly, the marginal pmf of Y , denoted by fY , is just the pmf
of Y , and can be computed using
X
fY (y) := P(Y = y) = f (x, y).
all x
Activity 1 (5 minutes)
An unloaded dice is rolled twice. Let X be the maximum number
obtained from the two rolls, and Y be the minimum number from
the two rolls. (E. g. if the two rolls result in 3 and 5, then X = 5
and Y = 3.) Complete the joint pmf table below.
x⧹y 1 2 3 4 5 6 fX
1
1 36 0
1
2 18
3
4
5
6
fY 1
7 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Activity 1 (solution)
For instance, we have
f (3, 3) = P(max = 3 and min = 3) = P({(3, 3)}) = 1/36;
f (3, 2) = P(max = 3 and min = 2) = P({(3, 2), (2, 3)}) = 2/36.
x⧹y 1 2 3 4 5 6 fX
1 1
1 36 0 0 0 0 0 36
1 1 3
2 18 36 0 0 0 0 36
1 1 1 5
3 18 18 36 0 0 0 36
1 1 1 1 7
4 18 18 18 36 0 0 36
1 1 1 1 1 9
5 18 18 18 18 36 0 36
1 1 1 1 1 1 11
6 18 18 18 18 18 36 36
11 9 7 5 3 1
fY 36 36 36 36 36 36 1
Note that both of the marginal pmf’s sum up to 1, which is a
useful way to check your answers. 8 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Definition
Let X and Y be discrete random variables. X and Y are said to
be independent if and only if the events ‘X ∈ A’ and ‘Y ∈ B’ are
independent, for any sets A and B ⊆ R.
9 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Independence, continued
10 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Expected value
Theorem
XX
E g(X, Y ) = g(x, y) f (x, y).
all x all y
12 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
Activity 2 (solution)
x⧹y 1 2 3 4 fX
0 0.15 0.25 0.03 0 0.43
1 0.12 0.2 0.05 0 0.37
2 0.08 0.1 0 0.02 0.2
fY 0.35 0.55 0.08 0.02 1
13 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
where E(X) and E(Y ) are computed using the marginal pmf’s.
(Computing the same expectation in two ways allows you to check
your answer.)
14 / 41
Discrete joint Continuous joint Extra notes Joint and marginal pmf Independence Expected value
(c) The event ‘X < Y ’ means ‘fewer cars than TV’s’, and consists
of all (x, y) pairs satisfying x < y. The probabilities corresponding
to these pairs are shown in red on Slide 13. Summing up the
probabilities, we get
15 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Outline
3 Extra notes
Proofs
16 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Joint pdf
Definition
Let X and Y be continuous random variables. A two-variable
function f (x, y) ≥ 0 is called the joint probability density
function (joint pdf) of X and Y , if for any region C in the plane,
ZZ
P (X, Y ) ∈ C = f (x, y) dx dy.
(x, y)∈C
Z dZ b
P (a ≤ X ≤ b) ∩ (c ≤ Y ≤ d) = f (x, y) dx dy.
c a
Basic example
Example 2
Let k be a constant, and let T be the triangle in R2 bounded by
x ≥ 0, y ≥ 0, and x + y ≤ 1. Suppose that the joint pdf of X and
Y is given by (
k, on the triangle T ,
f (x, y) =
0, otherwise.
Activity 3 (5 minutes)
22 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Activity 3 (solution)
(a) The intersection between the region ‘x ≤ 1/2’ and the triangle
T is shaded in green above. This region is a trapezium with area
3/8, so the probability, as the volume of the corresponding prism,
is given by 3/8 × 2 = 3/4.
Alternatively, using double integration (and the picture above),
Z x=1/2 Z y=1−x
P(X ≤ 1/2) = 2 dy dx
x=0 y=0
Z 1/2 1/2
3
= 2 − 2x dx = 2x − x2 = .
0 0 4
23 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Marginal pdf
Definitions
Let X and Y be continuous random variables. The marginal
probability density function (marginal pdf) of X, denoted by
fX , is just the pdf of X, and can be computed using
Z ∞
fX (x) = f (x, y) dy.
−∞
Recall that we need to carefully specify the domain of any pdf (or
cdf), so the full answer is
(
2(1 − x), if 0 ≤ x ≤ 1,
fX (x) =
0, otherwise.
(b) Compute the marginal pdf’s of X and Y , and hence show that
X and Y are independent.
(c) What is the probability that the discharge of the first pipe
(with discharge X) is higher than that of the second pipe?
Activity 4 (solution)
Z ∞ Z ∞ Z 9 Z 10
(a) 1= f (x, y) dx dy = k dx dy = 52 k,
−∞ −∞ 4 5
therefore k = 1/25.
Z ∞ Z 9
1 1
(b) fX (x) = f (x, y) dy = dy = if 5 ≤ x ≤ 10,
−∞ 4 25 5
and 0 otherwise.
Z ∞ Z 10
1 1
fY (y) = f (x, y) dx = dx = if 4 ≤ y ≤ 9,
−∞ 5 25 5
and 0 otherwise.
On the square (5 ≤ x ≤ 10) ∩ (4 ≤ y ≤ 9), f (x, y) = fX (x) fY (y)
= 1/25, while outside the square, f (x, y) = fX (x) fY (y) = 0.
Therefore, f (x, y) = fX (x) fY (y) for all real x and y, hence X
and Y are independent (uniform) random variables.
29 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
30 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Harder example
Example 3
Let X and Y denote the lifetime (in years) of an iPhone and an
Android phone respectively. Suppose that X ∼ exponential(a),
Y ∼ exponential(b), and that X and Y are independent. Find the
probability that the iPhone fails before the Android phone.
Example 3, picture
33 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Example 3, continued
The integral on Slide 32 simplifies as
ZZ
P(X < Y ) = a e−ax b e−by dy dx
0<x<y<∞
Z x=∞ Z y=∞
= a e−ax b e−by dy dx
x=0 y=x
Z x=∞ h iy=∞
= a e−ax − e−by dx
y=x
Zx=0
∞
= a e−ax e−bx dx
Z0 ∞
= a e−(a+b)x dx
0 ∞
a −(a+b)x
= − e
a+b 0
a
= .
a+b
34 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
35 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Activity 5 (solution)
For P(Y ≥ X + 1), the required region of integration is
‘y ≥ x + 1’. Shaded below is part of this infinite region (intersect
with the 1st quadrant, where the joint pdf is non-zero).
36 / 41
Discrete joint Continuous joint Extra notes Joint pdf Example Marginal pdf Independence; E
Outline
3 Extra notes
Proofs
38 / 41
Discrete joint Continuous joint Extra notes Proofs
Joint distribution
The ‘full’ notation for f (x, y) is fX,Y (x, y), but the subscripts
are often dropped when the context is clear.
For any random variables X and Y , the joint cumulative
distribution function F (x, y) is defined as
F (x, y) := P (X ≤ x) ∩ (Y ≤ y) .
The joint cdf is very useful, as it can be used to compute all
kinds of probabilities regarding X and Y . For instance, we have
P(X ≤ x) = FX (x) = lim F (x, y),
y→∞
Independence
We prove that the two characterizations of independence on Slide 9 are
equivalent. Such a proof must proceed in both directions:
Expectation of a sum
Let X and Y be any (not necessarily independent) discrete RV’s,
and let g(X, Y ) = X + Y . Then the theorem on Slide 11 gives
XX
E(X + Y ) = (x + y) f (x, y)
all x all y
X X X X
= x f (x, y) + y f (x, y)
all x all y all y all x
X X
= x fX (x) + y fY (y)
all x all y
= E(X) + E(Y ).