You are on page 1of 12

# Chapter 6 Expectation and Conditional Expectation Lectures 24 - 30

In this chapter, we introduce expected value or the mean of a random variable. First we define expectation for discrete random variables and then for general random variable. Finally we introduce the notion of conditional expectations using conditional probabilities.

## Definition 6.1. Two random variables

equal almost surely (in short a.s.) if

are said to be

## Theorem 6.0.23 Let

there exists a partition

where

may be

Proof. Let
. Here

## be the distribution function of may be . Since

is discrete, we have

Set

Then

## is pairwise disjoint and

Now define

Then

is a partition of

and

Remark 6.0.6 If

## , then the 'effective' which has

range of is at the most countable. Here 'effective' range means those values taken by positive probability. This leads to the name 'discrete' random variable.

Remark 6.0.7 If
that

is a discrete random variable, then one can assume without the loss of generality

Since if

, then set

and

for

## Theorem 6.0.24 Let

Then if

be such that

is a countable partition of

then

, set

Then clearly

Also if

then

. Therefore

expectation of

. Then

denoted by

has range

, then

be a Bernoulli(

Hence

be a Binomial

Hence

be a Poisson (

Hence

be a Geometric (

Hence

## Theorem 6.0.25 (Properties of expectation) Let

means. Then (i) If (ii) For , then .

and

## Proof. (i) Let

. Hence

be a representation of

. Then

implies

for all

(ii) Let

has a representation

. Now by setting

## one can use same partition for

and

. Therefore

Hence

Definition 6.3. (Simple random variable) A random variable is said to be simple if it is discrete and the
distribution function has only finitely many discontinuities. Theorem 6.0.26Let be random variable in satisfying . such that , then there exists a sequence

. as follows:

Proof. For

Then

## variables satisfying (i) and (ii) of Theorem 6.0.25. Then

Proof. Since
exists. Also since

Therefore

## Hence to complete the proof, it suffices to show that for

simple and

Let

where

is a partition of

. Fix

, set for

and

Since

Also

Hence

we have

Hence

(6.0.1)

## Using continuity property of probability, we have

Now let,

in (6.0.1), we get

Since,

is arbitrary, we get

## Definition 6.4. The expectation of a non negative random variable

is defined as (6.0.2)

where

Remark 6.0.9

## be a continuous non negative random variable with pdf

. Then

Proof. By using the simple functions given in the proof of Theorem 6.0.25, we get

(6.0.3)

where

Hence

## Definition 6.5. Let

exists if either or

is said to

where

Note that

Theorem 6.0.28

Let

. Then

Proof. Set

Hence

Then

## is a sequence of simple random variables such that

Similarly, set

Hence

Then

Now

(6.0.4)

and

(6.0.5)

The last equality follows by the arguments from the proof of Theorem 6.0.26. Combining (6.0.4) and (6.0.4), we get

Now as in the proof of Theorem 6.0.26, we complete the proof. We state the following useful properties of expectation. The proof follows by approximation argument using the corresponding properties of simple random variables

(i) If , then .

(ii) For

(iii) Let

. Then

## has finite mean and

In the context of Riemann integration, one can recall the following convergence theorem. `` If , then is a sequence of continuous functions defined on the such that uniformly in

i.e., to take limit inside the integral, one need uniform convergence of functions. In many situations in it is highly unlikely to get uniform convergence. In fact uniform convergence is not required to take limit inside an integral. This is illustrated in the following couple of theorem. The proof of them are beyond the scope of this course.

## Theorem 6.0.30 (Monotone convergence theorem)

random variables such that . Then

Let

[Here

means

.]

## Theorem 6.0.31 (Dominated Convergence Theorem) Let

(i) (ii) (iii) Then has finite mean.

## be random variables such that

Definition 6.6. (Higher Order Moments) Let moment of and is called the

is called the

th

## . The second central moment is

called the variance. Now we state the following theorem whose proof is beyond the scope of this course.

and

be a continuous

## function such that the integral

The above theorem is generally referred as the ``Law of unconscious statistician'' since often users treat the

above as a definition itself. Now we define conditional expectation denoted by E[Y|X] of the random variable Y given the information about the random variable X. If Y is a Bernoulli (p) random variable and X any discrete random variable, then we expect E[Y|X = x] to be P{Y = 1|X = x}, since we know that EY = p = P{Y = 1}. i.e.,

Where

## is the conditional pmf of Y given X. Now since we expect conditional expectation to be

liner and any discrete random variable can be written as a liner combination of Bernoulli random variable we get the following definition.

## Definition 6.7. Let

conditional expectation of

. Then

Example 6.0.41
. Calculate Set

Let

## be independent random variables with geometric distribution of parameter , where

For

Now

Therefore

i.e.,

Now

When X and Y are discrete random variable. E[Y|X] is defined using conditional pmf of Y given X. Hence we define E[Y|X] when X and Y are continuous random variable with joint pdf f in a similar way as follows.

## Definition 6.8. Let

expectation of given

## be continuous random variable with conditional pdf is defined as

. Then conditional

Remark 6.0.10 One can extend the defination of E[Y|X] when X is any random variable (discrete, continuous or mixed) and Y is a any random variable with finite mean. But it is beyound the scope of this course. Theorem 6.0.33 (i)
respectively. Then if Let be discrete random variables with joint pmf has finite mean, then , marginal pmfs and

## (ii) Let Then if

be continuous random variables with joint pdf has finite mean, then

, marginal pdfs

and

respectively.

Find

. Note that

and

## elsewhere. Hence for

Also

elsewhere. Therefore