Joint Probability Distributions,

Marginal Distributions,
Conditional Probability Distributions
By Ozlem Ulucan, PhD

Random Variables and Probability Distributions 1

 Joint Probability Distributions

• Suppose, we are interested in the

amount of precipitate P and
volume V of gas released from a

V = volume of gas released controlled chemical experiment.

• Here we have a 2-D sample space
consisting of the outcomes (p, v).

P = amount of precipitate
2
 Joint Probability Distributions (Discrete)

• Definition: If X and Y are two discrete random variables, the probability

distribution for their simultaneous occurrence can be represented by a
function with values f(x,y) for any pair of values (x, y) within the range of the
random variables X and Y. The function f(x,y) is the joint probability
distribution of X and Y.

• In the discrete case, f(x,y)=P(X=x,Y =y); that is, the values f(x,y) give the
probability that outcomes x and y occur at the same time.

3
 f (x, y) = P (X = x, Y = y);
that is, the values f (x, y) give the probability that outcomes x and y occur at
Joint Probability Distributions (Discrete)
the same time. For example, if an 18-wheeler is to have its tires serviced and X
represents the number of miles these tires have been driven and Y represents the
number of tires that need to be replaced, then f (30000, 5) is the probability that
the tires are used over 30,000 miles and the truck needs 5 new tires.

Definition 3.8: The function f (x, y) is a joint probability distribution or probability mass
function of the discrete random variables X and Y if
1. f (x, y) ≥ 0 for all (x, y),
!!
2. f (x, y) = 1,
x y

3. P (X = x, Y = y) = f (x, y).
!!
For any region A in the xy plane, P [(X, Y ) ∈ A] = f (x, y).
A

Example 3.14: Two ballpoint pens are selected at random from a box that contains 3 blue pens,
2 red pens, and 3 green pens. If X is the number of blue pens selected and Y is
the number of red pens selected, find
4
(a) the joint probability function f (x, y),
 Joint Probability Distributions (Discrete)

Example 1: Two ballpoint pens are selected at

random from a box that contains 3 blue pens, 2 red
pens, and 3 green pens. If X is the number of blue
pens selected and Y is the number of red pens
selected, find the joint probability function f(x,y).

5
 Chapter
be represented 3 formula
by the Random Variables and Probability Distributions
!3"!2"! 3 "
Joint Probability
f (x, y) =
Distributions
x y 2−x−y
! 8" ,
(Discrete)
5, it will become clear that the joint probability 2 distribution of Table 3.1 can
be represented • Solution:
for by
x = the The
formula
0, 1, 2; possible
y = 0, 1, 2; and pairs
0≤x+ ofy values
≤ 2. (x, y) are (0, 0), (0,
(b) The1), (1, 0), (1,that
probability 1),(X,
(0,Y2), and
) fall (2, region
in the 0) A is
!3"!2"! 3 "
P [(X, Y ) ∈ A] = P (X +
xY ≤y 1) 2−x−y
= f (0, 0) + f (0, 1) + f (1, 0)
f (x, y) = 3 3
!"
98 9 ,
= + + 2= .
28 14 28 14
for x = 0, 1, 2; yforTable
=x =0,
0, 1,
1, 2;2;
y =and
0, 1, 0
2; ≤
andx0+
≤ xy+≤
y ≤2.
2.
3.1: Joint Probability Distribution for Example 3.14
The probability that (X, Y ) fall in the region
x ARow
is
f (x, y) 0 1 2 Totals
3 9 3 15
P [(X, Y ) ∈ A] = P (X0+ Y ≤ 1) = f (0, 0) + f (0, 1) + f (1, 0)
28
3
28
3
28 28
3
y 1 0
14 14 7
3 2 3 9 0 09
1 1
= + + 5 15= 3 .
28 28

28 Totals
Column 14 1428 28 14
28 1

TableWhen
3.1:XJoint Probability
and Y are Distribution
continuous random variables, thefor
jointExample 3.14
density function 6
 x Row
f (x, y) 0 1 2 Totals
Joint Probability Distributions
3
28
9
28
3
0
28
(Continuous)
15
28
3 3 3
y 1 14 14 0 7
1 1
2 28 0 0 28
When X and Y are continuous
5 random
15 3 variables, the joint
Column Totals 14 28 28 1
probability distribution function f(x,y) is termed the joint
When X and Y are continuous random variables, the joint density function
f (x,density function.
y) is a surface lying above the xy plane, and P [(X, Y ) ∈ A], where A is any
region in the xy plane, is equal to the volume of the right cylinder bounded by the
baseDefinition:
A and the surface.

Definition 3.9: The function f (x, y) is a joint density function of the continuous random
variables X and Y if
1. f (x, y) ≥ 0, for all (x, y),
#∞ #∞
2. −∞ −∞ f (x, y) dx dy = 1,
##
3. P [(X, Y ) ∈ A] = A
f (x, y) dx dy, for any region A in the xy plane.

7
xample 3.15: A privately owned business operates both a drive-in facility and a walk-in facility.
 1. f (x, y) ≥ 0, for all (x, y),

2. −∞ −∞Probability
Joint f (x, y) dx dy = Distributions (Continuous)
#∞ #∞
1,
##
3.• PExample ∈ A]
[(X, Y ) 2: =
A privately f (x, y) dx dy, for any region A in the xy plane.
A owned business operates both a drive-in

facility and a walk-in facility. On a randomly selected day, let X and

mple 3.15: A privately owned business operates both a drive-in facility and a walk-in facility.
Y, respectively,
On a randomly be the
selected day,proportions
let X and ofY ,the time that the
respectively, bedrive-in and
the proportions of the
the the
time that walk-in facilities
drive-in andare
theinwalk-in
use, andfacilities
supposeare thatinthe joint
use, density
and suppose that the
joint density function of these random variables is
function of these random variables is
$
2
5 (2x + 3y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
f (x, y) =
0, elsewhere.

(a) Verify
(a) •Verify condition
condition 2 of2Definition
of Definition
3.9.
1 1 1
(b) •Find P [(X, Y ) ∈ A], where A = {(x, y) | 0 < x < , <
(b) Find P[(X,Y) ∈A], where A={(x,y)|0<x<1/2, 1/4 <y<1/2}.
2 4 y < 2 }.

8
 Joint Probability Distributions (Continuous) 97
3.4 Joint Probability Distributions

Solution : (a) The integration of f (x, y) over the whole region is

! ∞ ! ∞ ! 1 ! 1
2
f (x, y) dx dy = (2x + 3y) dx dy
−∞ −∞ 0 0 5
! 1" 2 #$x=1
2x 6xy $$
= + $ dy
0 5 5 x=0
! 1" # " #$1
2 6y 2y 3y 2 $$ 2 3
= + dy = + = + = 1.
0 5 5 5 5 $0 5 5

(b) To calculate the probability, we use

" #
1 1 1
P [(X, Y ) ∈ A] = P 0 < X < , < Y <
2 4 2
! 1/2 ! 1/2
2
= (2x + 3y) dx dy
1/4 0 5
! 1/2 " 2 #$x=1/2 ! 1/2 " #
2x 6xy $$ 1 3y
= + $ dy = + dy
1/4 5 5 x=0 1/4 10 5
" #$1/2
y 3y 2 $$
= +
10 10 $1/4
%" # " #&
1 1 3 1 3 13
= + − + = . 9
10 2 4 4 16 160
 continuous random variables, summations are replaced by integrals. We can now
Marginal Probability Distributions
make the following general definition.

Definition 3.10: The marginal distributions of X alone and of Y alone are

' '
g(x) = f (x, y) and h(y) = f (x, y)
y x

for the discrete case, and

! ∞ ! ∞
g(x) = f (x, y) dy and h(y) = f (x, y) dx
−∞ −∞

for the continuous case.

• Given
The termthemarginal
joint probability distribution
is used f (x, y) of
here because, inthe
thediscrete random
discrete case, the values of g(x)
and h(y) are
variables justYthe
X and , themarginal totals
probability of the g(x)
distribution respective columns
of X alone and rows when the
is obtained
values of f (x, y) are displayed in a rectangular table.
by summing f(x,y) over the values of Y. Similarly, the probability
distribution h(y) of Y alone is obtained by summing f(x,y) over the values
of X.
10
 Chapter 3 Random Variables and Probability Distributions
Marginal Probability Distributions
5, it will become clear that the joint probability distribution of Table 3.1 can
be represented • Example 3: For the given function, find the
by the formula
marginal probability distribution of X alone and Y
!3"!2"! 3 "
alone. x y 2−x−y
f (x, y) = ! 8" ,
2
for x = 0, 1, 2; y = 0, 1, 2; and 0 ≤ x + y ≤ 2.
x=0,1,2; y=0,1,2; 0≤x+y≤2
The probability that (X, Y ) fall in the region A is

P [(X, Y ) ∈ A] = P (X + Y ≤ 1) = f (0, 0) + f (0, 1) + f (1, 0)

3 3 9 9
= + + = .
28 14 28 14
11
For the random variable X, we
Chapter 3 see that Variables and Probability Distributions
Random
Marginal Probability Distributions 3 3 1 5
g(0) = f (0, 0) + f (0, 1) + f (0, 2) = + + = ,
5, it will become clear that the joint 28
probability 14 28
distribution 14 of Table 3.1 can
• Example 3: For the given function, 9 3 find the15
be representedg(1) by
= fthe
marginal formula
(1, 0) + f (1, 1) + f (1, 2) =
probability distribution
+
of X
+0=
alone
,
and Y
28 14 28
and alone. ! 3
"! 2
"! 3
"
x y 2−x−y
f (x, y) = ! 8" 3 , 3
g(2) = f (2, 0) + f (2, 1) + f (2, 2) 2
= +0+0= ,
28 28
for x are
which = 0,just
1, the
2; yx=0,1,2;
= 0, 1,
column 2; and
y=0,1,2;
totals 0 ≤ x3.1.
of0≤x+y≤2
Table + yIn≤a2.similar manner we could show
that
Thethe values •ofSolution:
probability h(y)
thatare
(X,given by the
Y ) fall in row
the totals.
regionInAtabular
is form, these marginal
distributions may be written as follows:
P [(X, Y ) ∈x A] =0P (X1 + Y2 ≤ 1) = fy (0, 0)0 + f1(0, 1) 2 + f (1, 0)
5 15 3 15 3 1
g(x) 3
14 289 3 28 h(y)
9 28 7 28
= + + = .
28 14 28 14
Find g(x) and h(y) for the joint density function of Example 3.15. 12
 Marginal Probability Distributions
ately owned business operates both a drive-in facility and a walk-in
randomly selected day,
• Example let Xtheand
4: Find g(x) Y , respectively,
and be the proportio
h(y) for the joint
hat the drive-in andfunction
density the walk-in
of facilities are in use, and suppose
ensity function of these random variables is
$
2
5 (2x + 3y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
f (x, y) =
0, elsewhere.

Verify condition 2 of Definition 3.9.

ind P [(X, Y ) ∈ A], where A = {(x, y) | 0 < x < 12 , 14 < y < 12 }.

13
 g(0)== f (0, 0)f +
3. P [(X, Y ) ∈ A] f (0,
(x, y) 1)dx
+ fdy,
(0, 2) = any+ region
for + A = in ,the xy plane.
A 28 14 28 14
9 3 15

mple 3.15:
Marginal Probability Distributions
g(1) = f (1, 0) + f (1, 1) + f (1, 2) =
28
+
14
+0=
28
,
A privately owned business operates both a drive-in facility and a walk-in facility.
and
On a randomly selected day, let X and Y , respectively, 3 be 3the proportions of the
Example
time •that the drive-in 4:
g(2)Find the
= f (2, 0)
and the g(x)
+ f (2,
walk-in f (2, 2) h(y)
1) +and
facilities=
28arefor
+ 0 + 0the
= joint
in use, ,
28and suppose that the
density
joint density
which are function
function
just theof theseof
column random
totals variables
of Table 3.1. In aissimilar manner we could show
that the values of h(y)$are given by the row totals. In tabular form, these marginal
2
distributions may be written as follows:
5 (2x + 3y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
f (x, y) = x 0 1 2 y 0 1 2
0, 5 15 3 elsewhere. 15 3 1
g(x) 14 28 28 h(y) 28 7 28

Solution:
(a) •Verify condition 2 of Definition 3.9.
Example 3.17: Find g(x) and h(y) for the joint density function of Example 3.15.
Solution : By P
(b) Find [(X, Y ) ∈ A], where A = {(x, y) | 0 < x < 12 , 14 < y < 12 }.
definition,
! ∞ ! 1 " #$y=1
2 4xy 6y 2 $$ 4x + 3
g(x) = f (x, y) dy = (2x + 3y) dy = + $ = ,
−∞ 0 5 5 10 y=0 5
for 0 ≤ x ≤ 1, and g(x) = 0 elsewhere. Similarly,
! ∞ ! 1
2 2(1 + 3y)
h(y) = f (x, y) dx = (2x + 3y) dx = ,
−∞ 0 5 5
for 0 ≤ y ≤ 1, and h(y) = 0 elsewhere.
The fact that the marginal distributions g(x) and h(y) are indeed the proba- 14
bility distributions of the individual variables X and Y alone can be verified by
 order
that we to be able
make to the
use of effectively compute
special type conditionalof probabilities.
of distribution the form f (x, This type
y)/g(x) in of dis-
tribution
Definition 3.11: order
Let to
X be
andis called
Y to a conditional
beeffectively
two random probability
variables, distribution;
discrete or continuous.theThe
formal definition
conditional
Conditional Probability Distributions
able compute conditional probabilities. This type of dis-
follows.
distribution
tribution is called of the random probability
a conditional variable Y given that X =the
distribution; x formal
is definition
follows.
Definition 3.11:• Definition:
Let X andconditional probability
Y be two random fvariables,
(x, y) discrete or continuous. The conditional
distribution f random
(y|x) = variable, Yprovided g(x)X>=0.x is
Definition 3.11: Let X and Y be of
twothe
random variables, given
g(x) discrete that
or continuous. The conditional
distribution of the random variable Y given that X = x is
Similarly, the conditional f (x, y) of X given that Y = y is
f (y|x)fdistribution
=(x, y)
, provided g(x) > 0.
f (y|x) = g(x)
, provided g(x) > 0.
f (x, y)
g(x)
f (x|y)distribution
Similarly, the conditional = , provided h(y)
of X given > Y0. = y is
that
h(y)
Similarly, the conditional distribution of X given that Y = y is
f (x,that
If we wish to find thef probability y) the discrete random variable X falls between
(x|y)f=
(x, y) , provided h(y) > 0.
a and b when it isf (x|y)
known h(y)
= that the discrete h(y)
, provided variable
> 0. Y = y, we evaluate
h(y)
! random variable X falls between
If we wish to find the probability that the discrete
• IfThe
we wish
a and
to find the probability
probability
b when itthat Pthe
is known < that
X< that
(a discretethe|the
brandom
Y =discrete random
y)variable
discrete
variable
X ffalls
=variable X falls
=between
Y(x|y),
between
a and
y, we evaluate b, it
a is
and b when it is known that the discrete
known that the discrete variable Y=y is variable Y =
a<x<b y, we evaluate
!
!
where the summation
P (a < X < b |
P (a < X < b | Y = y) = = fof(x|y),
extends overY all
= values
y) X fbetween
(x|y), a and b. When X and
Y are continuous, we evaluate a<x<b
a<x<b
• When
wheretheX and
the Y are continuous;
summation extends
where summation extends overover all values
all values of X"of X between
between
b a andab.and b. When
When X and X and
Y Yare
arecontinuous,
continuous,
we we evaluate
< X < b | Y = y) =
evaluate
P (a f (x|y) dx.
a
" b
" b
P (aP < | Yb |=Yy)==y) =f (x|y)
(aX< <Xb < f (x|y)
dx. dx. 15
Example 3.18: Referring to Example 3.14, find the conditionala
a
distribution of X, given that Y = 1,
 Conditional Probability Distributions
Chapter 3 Random Variables and Probability Distributions
• Example 5: For the function f(x, y), find the conditional
distribution
5, it will become clear thatof the
X, given
jointthat Y = 1, and
probability use it to of Table 3.1 can
distribution
be representeddetermine P(X=0|Y =1).
by the formula
!3"!2"! 3 "
x y 2−x−y
f (x, y) = ! 8" ,
2
for x = 0, 1, 2; y = 0, 1, 2; and 0 ≤ x + y ≤ 2.
x=0,1,2; y=0,1,2; 0≤x+y≤2
The probability that (X, Y ) fall in the region A is

P [(X, Y ) ∈ A] = P (X + Y ≤ 1) = f (0, 0) + f (0, 1) + f (1, 0)

3 3 9 9
= + + = . 16
 Chapter 3 Random Variables and Probability Distributions
Conditional Probability Distributions
5, it will become clear that the joint probability distribution of Table 3.1 can
• Example 5: ForChapter
the function f(x, y),
3 Random find the
Variables conditional
and Probability Distributions
be represented by the formula
distribution of X, given that Y = 1, and use it to determine
Therefore,
P(X=0|Y =1). !3"!2"! 3 "
! " ! "! " ! " ! "! "
7 7 3 x 1 y 2−x−y 7 7 3 1
f (0|1) =
3
f (x, 3y) =14
f (0, 1) = =
2
, !
f
8
"
(1|1) =
3
, f (1, 1) =
3 14
=
2
,
! " ! " 2
7 7
for x =
f (2|1) = f (2,
x=0,1,2;
3
1) =
y=0,1,2; 0≤x+y≤2
(0) = 0,
0, 1, 2; y = 0, 1, 2; and 0 ≤ x + y ≤ 2.
3
and theSolution: the
conditional conditional
distribution distribution
of X, given that of
Y =X,1,given
is that Y = 1
The probability that (X, Y ) fall in the region A is
x 0 1 2
f (x|1) 12 1
0
[(X, Y ) ∈ A] = P (X + Y ≤ 1) = f (0, 0) + f (0, 1) + f (1, 0)
PFinally, 2

P(X=0|Y =1) = f(0|1)=1/2

3P (X = 03 | Y = 1) 9 = 1.
9 = f (0|1)
= + + = . 2
Therefore, if it is known28that 141 of the28 2 pen 14
refills selected is red, we have a
probability equal to 1/2 that the other refill is not blue.
17
Table 3.1: Joint Probability Distribution for Example 3.14
 1
P (X = 0 | Y = 1) = f (0|1) = .
2
Therefore, ifConditional
it is known thatProbability
1 of the 2 pen Distributions
refills selected is red, we have a
probability equal to 1/2 that the other refill is not blue.
• Example 6: The joint density for the random variables (X, Y ), where X is the
unit temperature change and Y is the proportion of spectrum shift that a
le 3.19: The joint density for the random variables (X, Y ), where X is the unit temperature
certain atomic particle produces, is
change and Y is the proportion of spectrum shift that a certain atomic particle
produces, is #
10xy 2 , 0 < x < y < 1,
f (x, y) =
0, elsewhere.
(a) Find the marginal densities g(x), h(y), and the conditional density f (y|x).
• (a)the
(b) Find Find probability
the marginal densities
that theg(x), spectrum
h(y), and the conditional
shifts moredensity
thanf(y|x).
half of the total
observations, given that
• (b) Find the probability thatthe temperature
the spectrum is increased
shifts more than half ofby
the0.25
total unit.
observations, given that the temperature is increased by 0.25 unit.
olution : (a) By definition,
$ ∞ $ 1
g(x) = f (x, y) dy = 10xy 2 dy
−∞ x
%y=1
10 3 %% 10 18

= xy = x(1 − x3 ), 0 < x < 1,

 f (x, y) =
0, elsewhere.
(a) Find the marginal densities g(x), h(y), and the conditional density f (y|x).
Conditional Probability Distributions
(b) Find the probability that the spectrum shifts more than half of the total
• Solution:
observations, given that the temperature is increased by 0.25 unit.
Solution : (a) By definition,
$ ∞ $ 1
g(x) = f (x, y) dy = 10xy 2 dy
−∞ x
%y=1
10 3 %% 10
= xy % = x(1 − x3 ), 0 < x < 1,
3 y=x 3
$ ∞ $ y
%x=y
h(y) = f (x, y) dx = 10xy 2 dx = 5x2 y 2 %x=0 = 5y 4 , 0 < y < 1.
−∞ 0

Now
f (x, y) 10xy 2 3y 2
f (y|x) = = 10 = 3
, 0 < x < y < 1.
g(x) 3 x(1 − x 3) 1 − x
(b) Therefore,
! % " $ 1 $ 1
1 % 3y 2 8
P Y > % X = 0.25 = f (y | x = 0.25) dy = dy = .
2 % 1 − 0.253 9
1/2 1/2

Example 3.20: Given the joint density function

#
x(1+3 y 2 ) 19
4 , 0 < x < 2, 0 < y < 1,
f (x, y) =
 Statistical Independence
Definition:
Let X and Y be two random variables, discrete or
continuous, with joint probability distribution f(x,y) and
marginal distributions g(x) and h(y), respectively. The
random variables X and Y are said to be statistically
independent if and only if
f (x, y) = g(x)h(y) for all (x, y) within their range.

20
 find P (0Statistical
< X < 1 | Y Independence
= 2).
Example
3.437: Let
LetX denote the reaction
X denote time, in time,
the reaction seconds,
intoseconds,
a certain to
◦
a certain stimulus and Y denote the temperature
stimulus and Y denote the temperature (◦F) at which a certain ( F)
at which a certain reaction starts to take place. Sup-
reaction
posestarts
thattotwo
takerandom
place. Suppose that X
variables twoand
random variables
Y have X
the joint
and Ydensity
3 oranges, 2 ap- have the joint density
e of 4 pieces of !
f oranges and Y 4xy, 0 < x < 1, 0 < y < 1,
f (x, y) =
find 0, elsewhere.
f X and Y ;
Determine
Find whether the two random variables are dependent or
on that is given 1 1 1
(a) P
independent.(0 ≤ X ≤ 2
and 4
≤ Y ≤ 2
);
(b) P (X < Y ).
Solution: Independent
s both a drive-
On a randomly 3.44 Each rear tire on an experimental airplane is 21