CHAPTER 3: PROBABILITY THEORY

◼ 3.1 Concept
◼ 3.2 Discrete random variables
◼ 3.3 Continuous random variables
◼ 3.4 Conditional probability
◼ 3.5 Pair of random variables

02/11/2015 402064-Chapter 3: Probability Theory 1

 RANDOM POINT ON THE PLANE!
◼ Let X and Y denote two random variables defined on
the same sample space .
◼ Jointly, the random variables X and Y are said to map
the outcome  to the point (X(), Y()) in the
plane.

02/11/2015 402064-Chapter 3: Probability Theory 2

 RANDOM POINT ON THE PLANE!
◼ The random point (X, Y) in the plane is the model for
our observation.
◼ Before the trial is performed, we can only discuss the
probability that the point (X, Y) will be in a specified
region of the plane, or will have a specific value.
◼ The individual probabilistic descriptions of X and Y are
insufficient to determine the probabilistic behavior of the
random point (X, Y) in the plane.
◼ The random point (X, Y) is also called joint random
variable (X, Y) or bivariate random variable (X, Y) or
random vector (X, Y).
02/11/2015 402064-Chapter 3: Probability Theory 3
 JOINT CUMULATIVE DISTRIBUTION
FUNCTION
◼ Definition: The joint cumulative distribution function of
random variables X and Y is FX,Y(x,y) = P(X ≤ x and Y ≤ y).

02/11/2015 402064-Chapter 3: Probability Theory 4

 JOINT CUMULATIVE DISTRIBUTION
FUNCTION
◼ Properties: For any pair of random variables X,Y:
❖ 0 ≤ FX,Y(x,y) ≤ 1.
❖ FX(x) = FX,Y(x,) and FY(x) = FX,Y(,y).
❖ FX,Y(x,-) = FX,Y(-,y) = 0.
❖ If x ≤ x1 and y ≤ y1 then FX,Y(x,y) ≤ FX,Y(x1,y1).
❖ FX,Y(+,+) = 1.

02/11/2015 402064-Chapter 3: Probability Theory 5

 JOINT PROBABILITY MASS FUNCTION

◼ Definition: The joint probability mass function of

random variables X and Y is PX,Y(x,y) = P(X = x, Y = y).
◼ Theorem: For discrete random variables X,Y and any
set B in the (X,Y) plane,
P( B) = 
( x , y )B
PX ,Y ( x, y)

◼ Marginal PMF: For discrete random variables X,Y with

joint PMF PX,Y(x,y):
PX (x) = P
ySY
X ,Y ( x, y ) PY (y) = P
xS X
X ,Y ( x, y )

02/11/2015 402064-Chapter 3: Probability Theory 6

 A PICTURE IS WORTH THOUSAND
WORDS!

02/11/2015 402064-Chapter 3: Probability Theory 7

 EXAMPLE
pX(x)

P(XY>3) = ?
Y pY(y) Ans: 0.13

02/11/2015 402064-Chapter 3: Probability Theory 8

 JOINT PROBABILITY DENSITY
FUNCTION
◼ Definition: The joint probability density function (PDF)
of random variables X and Y is a function fX,Y(x,y) with
the property x y
FX ,Y ( x, y ) = 
− −
f X ,Y (u, v)dudv

◼ Properties:
 2 FX ,Y ( x, y )
❖ f X ,Y (x, y ) =
xy
❖ P( x1  X  x2 , y1  X  y2 ) = FX ,Y ( x2 , y2 ) − FX ,Y ( x2 , y1 )
− FX ,Y ( x1 , y2 ) + FX ,Y ( x1 , y1 )
❖ fX,Y(x,y) ≥ 0 for all (x,y).
+ +
❖

− −
f X ,Y (x, y)dxdy = 1

02/11/2015 402064-Chapter 3: Probability Theory 9

 JOINT PROBABILITY DENSITY
FUNCTION
◼ Theorem: The probability that the continuous y y=f(x)
random variables (X,Y) are in A is
A y=g(x)
P( A) =  f X ,Y (x, y)dxdy
◼ Example: A a b x
b  g ( x) 
 f XY ( x, y )dxdy =    f XY ( x, y )dy  dx

a  f ( x)

A 

y
y
If fXY(x,y) = c (constant) then  f X ,Y (x, y)dxdy = cS A
A
x x
02/11/2015 402064-Chapter 3: Probability Theory 10
 JOINT PROBABILITY DENSITY
FUNCTION
0 if x  0 or y  0
The graph of the joint CDF of (X,Y) in the above  2
example:  x if 0  x  1, y  x
2

FXY ( x, y ) =  y ( x − y ) if 0  y  x  1
 y (1 − y ) if x  1, 0  y  1

1 if x  1, y  1



02/11/2015 402064-Chapter 3: Probability Theory 11

 JOINT PROBABILITY DENSITY
FUNCTION
y y=f(x)
P(X2 + Y2  1)

A y=g(x)
1 y = 1 − x2
2
a b x
A b  g ( x) 
 f XY ( x, y )dxdy =    f XY ( x, y )dy  dx

a  f ( x)

1 A 
2
1  
1/ 2
x  1− x 2

◼ Method 1:  f XY ( x, y)dxdy =    2dy  dx +  
  
2dy dx =

A 0 0  1/ 2  0 
4
◼ Method 2: 2 
 f XY ( x, y )dxdy = 2S A =  1 =
2

A
8 4

02/11/2015 402064-Chapter 3: Probability Theory 12

 MARGINAL PDF
◼ Theorem: If X and Y are random variables with joint
PDF fX,Y(x,y) then
+ +
f X ( x) = 
−
f X ,Y ( x, y)dy fY ( y ) = 
−
f X ,Y ( x, y)dx

◼ fX(x) and fY(y) are called the marginal PDFs of random

variables (X,Y).
◼ Example:

02/11/2015 402064-Chapter 3: Probability Theory 13

 MARGINAL PDF
◼ Example:

◼ Solution:
 1 6( x + y 2 )  6x + 2
 
+
dy if x [0,1]  if x  [0,1]
f X ( x) =  f X ,Y ( x, y)dy =  0 5 = 5
− 0 if x  [0,1] 0 if x  [0,1]

 1 6( x + y 2 )  6 y2 + 3
 dx if y  [0,1] 
+
if y  [0,1]
fY ( x ) =  f X ,Y ( x, y)dy =  0 5 = 5
− 0 if y  [0,1] 0 if y  [0,1]
 

02/11/2015 402064-Chapter 3: Probability Theory 14

 FUNCTION OF TWO RANDOM
VARIABLES
◼ Theorem 1: For discrete random variables X and Y, the
derived random variable W = g(X,Y) has PMF:
PW ( w) = 
( x , y ):g ( x , y ) = w
PX ,Y ( x, y)

◼ Theorem 2: For continuous random variables X and Y,

the CDF of W = g(X,Y) is
FW ( w) = P(W  w) = 
g ( x , y ) w
f X ,Y ( x, y )dxdy

◼ In particular, if W = max(X,Y) then

w w
FW ( w) = P(W  w) = 
− −
f X ,Y ( x, y )dxdy

02/11/2015 402064-Chapter 3: Probability Theory 15

 EXPECTED VALUE OF PAIR OF RVS

◼ Definition: For random variables X and Y, the

expected value of W = g(X,Y) is
❖ Discrete case: E[W ] =   g ( x, y ) PX ,Y ( x, y )
xS X ySY

+ +
❖ Continuous case: E[W ] =   g ( x, y ) f
− −
X ,Y ( x, y)dxdy

◼ Properties:
❖ E[g1(X,Y) +…+ gn(X,Y)] = E[g1(X,Y)] + … + E[g1(X,Y)]
❖ Var[X+Y] = Var[X] + Var[Y] + 2E[(X - X)(Y - Y)]

02/11/2015 402064-Chapter 3: Probability Theory 16

 CONDITIONAL JOINT PMF
◼ Definition 1: For discrete random variables X and Y
and an event B with P(B) > 0, the conditional joint PMF
of X and Y given B is PX,Y|B(x,y) = P(X = x, Y = y|B).
◼ Theorem 1: For any event B, a region of the X,Y plane
with P(B) > 0,  PX ,Y ( x, y )
 if ( x, y )  B
PX ,Y |B ( x, y ) =  P( B)
0
 otherwise
◼ Definition 2: Given an event B with P(B) > 0, the
conditional joint PDF of X and Y is
 f X ,Y ( x, y )
 if ( x, y )  B
f X ,Y |B ( x, y ) =  P( B)
0
 otherwise
02/11/2015 402064-Chapter 3: Probability Theory 17
 CONDITIONAL EXPECTED VALUE & VARIANCE

◼ Definition 1: For random variables X and Y and an

event B of nonzero probability, the conditional expected
value of W = g(X,Y) given B is
❖ Discrete case: E[W | B] =   g ( x, y ) PX ,Y |B ( x, y )
xS X ySY
+ +

❖ Continuous case: E[W | B] =   g ( x, y ) f

− −
X ,Y | B ( x, y )dxdy

◼ Definition 2: The conditional variance of the random

variable W = g(X,Y) is Var[W|B] = E[(W - W)2|B].

02/11/2015 402064-Chapter 3: Probability Theory 18

 EXERCISE
y
1

1 x
  1 1 1 1
1= 
− −
f X ,Y ( x, y )dxdy =   cxydxdy = c  xdx  y 2 dy = c / 6  c = 6
0 0 0 0
1
x 
P( X  Y ) =  f X ,Y ( x, y )dxdy =    6 xy dy  dx
2

x y 00 
1
1
 2 x5  2
=  2 x dx = 
4
 =
0  5 0 5
x  1
2
 
1 1 8
x 1
P(Y  X 2 ) =  f X ,Y ( x, y )dxdy =    6 xy 2 dy  dx =  2 x 7 dx =   =

0 0
  4 0 4
y  x2  0

02/11/2015 402064-Chapter 3: Probability Theory 19

 EXERCISE
y
1
0.75
0.5

0.5 1 x
0.75
P(min( X , Y )  0.5) = 1 − P(min( X , Y )  0.5) = 1 − P( X  0.5, Y  0.5)
1
1 
= 1 −    6 xy 2 dy  dx = 1 − ( x 2 )  y 3 =
1 1 11
0.5  0.5  0.5 0.5 32

0.75
 0.75 
P(max( X , Y )  0.75) = P( X  0.75, Y  0.75) =    6 xy dy  dx
2

0  0 
= ( x2 )
0.75
3 0.75 729
y =
0 0 65536
02/11/2015 402064-Chapter 3: Probability Theory 20