You are on page 1of 12

Chapter 6 Expectation and Conditional Expectation Lectures 24 - 30

In this chapter, we introduce expected value or the mean of a random variable. First we define expectation for discrete random variables and then for general random variable. Finally we introduce the notion of conditional expectations using conditional probabilities.

Definition 6.1. Two random variables

equal almost surely (in short a.s.) if

defined on a probability space

are said to be

Now we give a useful characterization of discrete random variables.

Theorem 6.0.23 Let

there exists a partition

be a discrete random variable defined on a probability space of and

. Then such that


may be

. . Let be the set of all discontinuities of

Proof. Let
. Here

be the distribution function of may be . Since

is discrete, we have



is pairwise disjoint and

Now define


is a partition of


Remark 6.0.6 If

is a discrete random variable on a probability space

, then the 'effective' which has

range of is at the most countable. Here 'effective' range means those values taken by positive probability. This leads to the name 'discrete' random variable.

Remark 6.0.7 If

is a discrete random variable, then one can assume without the loss of generality

Since if

, then set



Theorem 6.0.24 Let

Then if

be such that

is a countable partition of


Proof. For each

, set

Then clearly

Also if


. Therefore

This completes the proof.

Definition 6.2. Let

expectation of

be a discrete random variable represented by is defined as

. Then

denoted by

provided the right hand side series converges absolutely.

Remark 6.0.8 In view of Remark 6.0.5., if

has range

, then

Example 6.0.37 Let

be a Bernoulli(

) random variable. Then


Example 6.0.38 Let

be a Binomial

random variable. Then


Here we used the identity

Example 6.0.39 Let

be a Poisson (

) random variable. Then


Example 6.0.40 Let

be a Geometric (

) random variable. Then


Theorem 6.0.25 (Properties of expectation) Let

means. Then (i) If (ii) For , then .


be discrete random variables with finite

Proof. (i) Let

. Hence

be a representation of

. Then


for all

(ii) Let

has a representation

. Now by setting

one can use same partition for


. Therefore


Definition 6.3. (Simple random variable) A random variable is said to be simple if it is discrete and the
distribution function has only finitely many discontinuities. Theorem 6.0.26Let be random variable in satisfying . such that , then there exists a sequence

of simple random variables (i) For each ,

(ii) For each

as , define simple random variable

. as follows:

Proof. For


's satisfies the following:

Lemma 6.0.3 Let

be a non negative random variable and

be a sequence of simple random exists and is given by

variables satisfying (i) and (ii) of Theorem 6.0.25. Then

Proof. Since
exists. Also since

, we have 's are simple, clearly,

(see exercise). Hence


Hence to complete the proof, it suffices to show that for

simple and



is a partition of

. Fix

, set for



, we have for each



From the definition of

we have



Using continuity property of probability, we have

Now let,

in (6.0.1), we get


is arbitrary, we get

This completes the proof.

Definition 6.4. The expectation of a non negative random variable

is defined as (6.0.2)


is a sequence of simple random variables as in Theorem 6.0.25.

Remark 6.0.9

One can define expectation of

, non negative random variable, as

But we use Definition 6.4., since it is more handy.

Theorem 6.0.27 Let

be a continuous non negative random variable with pdf

. Then

Proof. By using the simple functions given in the proof of Theorem 6.0.25, we get



is the point given by the mean value theorem.


Definition 6.5. Let

exists if either or

be a random variable on is finite. In this case

. The mean or expectation of is defined as

is said to


Note that

is the positive part and

is the negetive part of

Theorem 6.0.28


be a continuous random variable with finite mean and pdf

. Then

Proof. Set



is a sequence of simple random variables such that

Similarly, set







The last equality follows by the arguments from the proof of Theorem 6.0.26. Combining (6.0.4) and (6.0.4), we get

Now as in the proof of Theorem 6.0.26, we complete the proof. We state the following useful properties of expectation. The proof follows by approximation argument using the corresponding properties of simple random variables

Theorem 6.0.29 Let

(i) If , then .

be random variables with finite mean. Then

(ii) For

(iii) Let

be a random variable such that

. Then

has finite mean and

In the context of Riemann integration, one can recall the following convergence theorem. `` If , then is a sequence of continuous functions defined on the such that uniformly in

i.e., to take limit inside the integral, one need uniform convergence of functions. In many situations in it is highly unlikely to get uniform convergence. In fact uniform convergence is not required to take limit inside an integral. This is illustrated in the following couple of theorem. The proof of them are beyond the scope of this course.

Theorem 6.0.30 (Monotone convergence theorem)

random variables such that . Then


be an increasing sequence of nonnegative




Theorem 6.0.31 (Dominated Convergence Theorem) Let

(i) (ii) (iii) Then has finite mean.

be random variables such that

Definition 6.6. (Higher Order Moments) Let moment of and is called the

be a random variable. Then th central moment of

is called the


. The second central moment is

called the variance. Now we state the following theorem whose proof is beyond the scope of this course.

Theorem 6.0.32 Let

be a continuous random variable with pdf is finite. Then


be a continuous

function such that the integral

The above theorem is generally referred as the ``Law of unconscious statistician'' since often users treat the

above as a definition itself. Now we define conditional expectation denoted by E[Y|X] of the random variable Y given the information about the random variable X. If Y is a Bernoulli (p) random variable and X any discrete random variable, then we expect E[Y|X = x] to be P{Y = 1|X = x}, since we know that EY = p = P{Y = 1}. i.e.,


is the conditional pmf of Y given X. Now since we expect conditional expectation to be

liner and any discrete random variable can be written as a liner combination of Bernoulli random variable we get the following definition.

Definition 6.7. Let

conditional expectation of

are discrete random variable with conditional pmf given is defined as

. Then

Example 6.0.41
. Calculate Set


be independent random variables with geometric distribution of parameter , where






When X and Y are discrete random variable. E[Y|X] is defined using conditional pmf of Y given X. Hence we define E[Y|X] when X and Y are continuous random variable with joint pdf f in a similar way as follows.

Definition 6.8. Let

expectation of given

be continuous random variable with conditional pdf is defined as

. Then conditional

Remark 6.0.10 One can extend the defination of E[Y|X] when X is any random variable (discrete, continuous or mixed) and Y is a any random variable with finite mean. But it is beyound the scope of this course. Theorem 6.0.33 (i)
respectively. Then if Let be discrete random variables with joint pmf has finite mean, then , marginal pmfs and

(ii) Let Then if

be continuous random variables with joint pdf has finite mean, then

, marginal pdfs



Proof. We only prove (ii).

Example 6.0.42 Let

be continuous random variables with joint pdf given by


and hence calculate

. Note that


elsewhere. Hence for


elsewhere. Therefore