You are on page 1of 12

Chapter 6

Expectation and Conditional Expectation

Lectures 24 - 30

In this chapter, we introduce expected value or the mean of a random variable. First we define expectation for discrete random variables and then for general random variable. Finally we introduce the notion of conditional expectations using conditional probabilities.

Definition 6.1. Two random variables

defined on a probability space

equal almost surely (in short

a.s.) if

are said to be

Now we give a useful characterization of discrete random variables.

be a
discrete random variable defined on a probability space
of
and

Let

Theorem 6.0.23 Let

there exists a partition

. Then

such that

may be
.

may be

where

Proof. Let

. Here

be the distribution function of

. Since

be the set of all discontinuities of

is discrete, we have

Set

Then

is pairwise disjoint and

Now define

Then

is a partition of

and

Remark 6.0.6 If

is a discrete random variable on a probability space

, then the 'effective'

range of is at the most countable. Here 'effective' range means those values taken by which has positive probability. This leads to the name 'discrete' random variable.

Remark 6.0.7 If

that

is a discrete random variable, then one can assume without the loss of generality

Since if

, then set

and

for

.

Theorem 6.0.24 Let

Then if

be such that

then

Proof. For each

, set

Then clearly

Also if

then

. Therefore

This completes the proof.

is a countable partition of

.

Definition 6.2. Let

be a discrete random variable represented by

expectation of

denoted by

is defined as

. Then

provided the right hand side series converges absolutely.

Remark 6.0.8

In view of Remark 6.0.5., if

has range

, then

 Example 6.0.37 Let Hence Example 6.0.38 Let

Hence

be a Bernoulli(

) random variable. Then
be a Binomial
random variable. Then

Here we used the identity

 Example 6.0.39 Let Hence Example 6.0.40 Let

Hence

be a Poisson (

) random variable. Then

be a Geometric (
) random variable. Then

Theorem 6.0.25 (Properties of expectation) means. Then

(i) If

Let

and

be discrete random variables with finite

.

, then

(ii) For

be a representation of
. Then

Proof. (i) Let

. Hence

implies

for all

. Now by setting

(ii) Let

has a representation

one can use same partition for

and

. Therefore

Hence

Definition 6.3. (Simple random variable) A random variable is said to be simple if it is discrete and the distribution function has only finitely many discontinuities.

Theorem 6.0.26Let

be random variable in

satisfying
.

of simple random variables

(i) For each

,

such that

, then there exists a sequence

.

(ii) For each

Proof. For

as

, define simple random variable
as follows:

's satisfies the following:

be a non negative random variable and

Then

Lemma 6.0.3 Let

be a sequence of simple random

variables satisfying (i) and (ii) of Theorem 6.0.25. Then

Proof. Since

, we have

exists and is given by

(see exercise). Hence

exists. Also since
's are simple, clearly,
Therefore
simple and
,

Hence to complete the proof, it suffices to show that for

Let

where

Since

Also

. Fix

is a partition of

, set for

and

.

, we have for each

,

Hence

From the definition of

we have

Hence

 (6.0.1) Using continuity property of probability, we have Now let, in (6.0.1), we get Since, is arbitrary, we get This completes the proof. Definition 6.4. The expectation of a non negative random variable is defined as (6.0.2) where is a sequence of simple random variables as in Theorem 6.0.25. Remark 6.0.9 One can define expectation of , non negative random variable, as But we use Definition 6.4., since it is more handy. Theorem 6.0.27 Let be a continuous non negative random variable with pdf . Then Proof. By using the simple functions given in the proof of Theorem 6.0.25, we get (6.0.3)
where
is the point given by the mean value theorem.

Hence

Definition 6.5. Let

exists if either

where

be a random variable on

or

is finite. In this case

. The mean or expectation of

is defined as

is said to

Note that

is the positive part and

is the negetive part of

Theorem 6.0.28

Proof. Set

Let

be a continuous random variable with finite mean and pdf

. Then

Hence

Then

is a sequence of simple random variables such that

Similarly, set

Hence

Then

Now

and

(6.0.4)

(6.0.5)

The last equality follows by the arguments from the proof of Theorem 6.0.26. Combining (6.0.4) and (6.0.4), we get

Now as in the proof of Theorem 6.0.26, we complete the proof.

We state the following useful properties of expectation. The proof follows by approximation argument using the corresponding properties of simple random variables

Theorem 6.0.29 Let

be random variables with finite mean. Then

,

(ii) For

(iii) Let

. Then

be a random variable such that

has finite mean and

.

In the context of Riemann integration, one can recall the following convergence theorem.

`` If

is a sequence of continuous functions defined on the

, then

such that

uniformly in

i.e., to take limit inside the integral, one need uniform convergence of functions. In many situations in it is highly unlikely to get uniform convergence. In fact uniform convergence is not required to take limit inside an integral. This is illustrated in the following couple of theorem. The proof of them are beyond the scope of this course.

Theorem 6.0.30 (Monotone convergence theorem)

random variables such that

be an increasing sequence of nonnegative

Let

. Then
means

.]

[Here

Theorem 6.0.31 (Dominated Convergence Theorem)

(i)

has finite mean.

Let

(ii)

(iii)

Then

be random variables such that

Definition 6.6. (Higher Order Moments) Let

moment of

be a random variable. Then

th central moment of

th

. The second central moment is

is called the

and

is called the

called the variance. Now we state the following theorem whose proof is beyond the scope of this course.

Theorem 6.0.32 Let
be a continuous random variable with pdf
function such that the integral
is finite. Then

and

be a continuous

The above theorem is generally referred as the ``Law of unconscious statistician'' since often users treat the

above as a definition itself. Now we define conditional expectation denoted by E[Y|X] of the random variable Y given the information about the random variable X. If Y is a Bernoulli (p) random variable and X any discrete random variable, then we expect E[Y|X = x] to be P{Y = 1|X = x}, since we know that EY = p = P{Y = 1}. i.e.,

Where
is the

conditional pmf of Y given X. Now since we expect conditional expectation to be

liner and any discrete random variable can be written as a liner combination of Bernoulli random variable we get the following definition.

Definition 6.7. Let

are discrete random variable with conditional pmf

conditional expectation of

given

is defined as

.

Then

Example 6.0.41

. Calculate

Let

be independent random variables with geometric distribution of parameter

, where

Set

For

Now

Therefore

i.e.,

Now

When X and Y are discrete random variable. E[Y|X] is defined using conditional pmf of Y given X. Hence we define E[Y|X] when X and Y are continuous random variable with joint pdf f in a similar way as follows.

Definition 6.8. Let

be continuous random variable with conditional pdf

expectation of

given

is defined as

. Then conditional

Remark 6.0.10 One can extend the defination of E[Y|X] when X is any random variable (discrete, continuous or mixed) and Y is a any random variable with finite mean. But it is beyound the scope of this course.

, marginal pmfs

Theorem 6.0.33 (i)

respectively. Then if

Let

be discrete random variables with joint pmf

and

has finite mean, then

(ii) Let

Then if

be continuous random variables with joint pdf

, marginal pdfs

and

respectively.

has finite mean, then

Proof. We only prove (ii).

Example 6.0.42 Let

. Note that

be continuous random variables with joint pdf given by

Find

and hence calculate

and

elsewhere. Hence for

,

Also

elsewhere. Therefore