STM2PM

Lecture 28

Conditional Expected Values

Independent Random Variables

28.1
 Conditional Expected Values

Definition: Conditional expected values are

calculated using the conditional probability den-
sity (or mass) function:
Z ∞
E(X|Y = y) = x fX|Y (x|y)dx
−∞
and for discrete distributions:
X
E(X|Y = yj ) = xi pX|Y (xi|yj )
xi

28.2
 Conditional Expected Values, Example,
Discrete case

Example (see Lecture 27) A fair coin is tossed

three times. Let X be the number of heads on
the first toss, and let Y be the total number of
heads (in the three tosses). From the lecture
27 we know the joint probability mass function
and the marginal probabilities are as follows:
y
0 1 2 3
x 0 1/8 2/8 1/8 0 1/2
1 0 1/8 2/8 1/8 1/2
1/8 3/8 3/8 1/8

We also know that pX|Y (1|2) = 2/3 and

pX|Y (0|2) = 1/3.

Let us calculate E(X|Y = 2):

28.3
 Conditional Expected Values, Example,
Continuous case

Example: For the continuous example form

the lecture 27 we have:
X and Y have a joint probability density func-
tion given by

4xy x ∈ [0, 1], y ∈ [0, 1]
f (x, y) =
0 elsewhere
And

2x x ∈ [0, 1], y ∈ [0, 1]
fX|Y (x|y) =
0 elsewhere

From this:
Z 1
E(X|Y = y) = xfX|Y (x|y)dx =
0

28.4
 Independence of two random variables,
Introduction

Recall: For two events (called A and B), we

defined independence (in Lecture 6) as fol-
lows:
P (A ∩ B) = P (A)·P (B)
so that
P (A ∩ B)
P (A|B) = = P (A)
P (B)
i.e. being given that B has happened does not
change the probability one assigns to A.

Recall: We know that for a random variable

an ‘event’ means that the variable X takes a
particular value (discrete)

P (X = a)
or its value comes from a set of values (dis-
crete or continuous)

P (a 6 X 6 b)

28.5
 Independence of two discrete random
variables

Definition: Two discrete random variables

defined on the same sample space are inde-
pendent if

P (X = x, Y = y) = P (X = x)P (Y = y) (∗)
and in terms of the joint and marginal prob-
ability mass functions

pXY (x, y) = pX (x)pY (y)

To show that two discrete random variables

are not independent, we only need to find one
value of X and one value of Y for which this
condition fails (one counterexample).

Example: In the previous discrete example

(Slide 2)
2
P (X = 1, Y = 2) = pXY (1, 2) = .
8
However, pX (1) = 1
2 and p Y (2) = 3,
8
so we see that
28.6
 Independent physical causes

Remark: When it is clear that random vari-

ables arise from independent physical causes,
they will be independent.

In the previous example, we could not expect

independence without checking, as the descrip-
tions of the events do not describe independent
processes.

Exercise: A fair coin is tossed twice. Let

X be the number of heads on the first toss and
Z be the number of tails on the second toss.
Are X and Z independent random variables?

Generally, however, independence depends on

the details of the probability function assigned,
so the condition (∗) must be checked.
28.7
 Independence of two continuous random
variables

Definition: Jointly continuous random vari-

ables X and Y are independent if and only if

P (a 6 X 6 b, c 6 Y 6 d) =
P (a 6 X 6 b)P (c 6 Y 6 d)
In terms of the joint and marginal cdfs:

F (x, y) = P (X ≤ x, Y ≤ y)
= P (X ≤ x)P (Y ≤ y)
= FX (x)FY (y)
Or In terms of the joint and marginal pdfs:

f (x, y) = fX (x)fY (y)

.

28.8
Independence of two continuous random
variables, Example

Example: For the previous continuous exam-

ple (Slide 4) in the Lecture 27 we found that
for

4xy x ∈ [0, 1], y ∈ [0, 1]
f (x, y) =
0 otherwise
the marginal probability density functions are

fX (x) = 2x x ∈ [0, 1]
and
fY (y) = 2y y ∈ [0, 1].

That means the random variables X and Y are

independent.

28.9
 More examples

Exercise: The function


 2 (x + 2y) 0 6 x 6 1, 0 6 y 6 1
f (x, y) = 3
0 otherwise
is a joint probability distribution function for X
and Y .
Are the variables X and Y independent?

28.10
 Properties of independent Random
Variables

Property 1: If X and Y are independent,

then for functions g and h, g(X) and h(Y ) are
independent.

Property 2: Independence can be extended to

more than two random variables X1, X2, . . . Xm
in an obvious way. Then functions of disjoint
subsets of the full set of independent variables
are also independent.

28.11
 Properties of independent Random
Variables

Property 3: If X and Y are independent,

then it is valid:

E(XY ) = E(X)E(Y ).
Proof in continuous case:

Remark: The property

E(X + Y ) = E(X) + E(Y )

is valid for any two random variables X and
Y (independence is not necessary)

28.12
 Properties of independent Random
Variables

Property 4: If X and Y are independent,

then it is valid:

E((X − µX )(Y − µY )) = 0
Proof:

28.13
 Properties of independent Random
Variables

Property 5: If X and Y are independent,

then it is valid:

Var(X + Y ) = Var(X) + Var(Y )

Proof:

28.14
 Mean and Variance for the Binomial
Distribution.

Let X1, X2, . . . , Xn be independent Bernoulli ran-

dom variables, each with the same success prob-
ability p.

What is the distribution of

Y = X1 + X2 + . . . + Xn?

From Lecture 23 we know that for each Xi,

E(Xi) = p.
Hence the mean of the binomial distribution
Bin(n, p) is:

We also found Var(Xi) = p(1 − p)

Hence the variance of the binomial distribution

is:

28.15