Chapter 4 Expectation, Moments and Moment Generating Functions

Chapter 4
Expectation, Moments and Moment

Generating Functions
AMS 7112
Mathematical Statistics
Department of Mathematics & Statistics

College of Arts & Sciences
University of Southeastern Philippines
Mean and Variance
Illustration.
Toss a fair, six-sided die many times. In the long run,

what would the average (or "mean") of the tosses
be? For example, we observe the following
outcomes:
Expectation, Moments
and Moment Generating
Functions
Page 2
Mean and Variance
Observations:
1. In reality, one-sixth of the tosses will

equal x only in the long run.
2. The mean is a weighted average, that is, an

average of the values weighted by their
respective individual probabilities.
3. The mean is called the expected value of X,

denoted E(X) or by μ.
Functions
Page 3
Mean and Variance
Functions
Page 4
Mean and Variance
Remarks:
1. In the definition of a mean of a random

variable, only density functions [in (i) and (ii)]
or distribution functions [in (iii)] were used;
hence we have really defined the mean for
these functions without reference to random
variables.
Functions
Page 5
Mean and Variance
Remarks:
1. In the definition of a mean of a random

variable, only density functions [in (i) and (ii)]
or distribution functions [in (iii)] were used;
hence we have really defined the mean for
these functions without reference to random
variables.
Functions
Page 6
Mean and Variance
2. E[X] is the center of gravity (or centroid) of the
unit mass that is determined by the density function
of X. So the mean of X is a measure of where the
values of the random variable X are" centered.“
3. (iii) of the definition is for all random variables;

whereas, (i) is for discrete random variables, and (ii)
is for continuous random variables. E[X] could have
been defined by just giving (iii) since (i) follows from
(iii) and (ii) follows from (iii).
Functions
Page 7
Mean and Variance
Example 1. The probability function for the
number of seizures, X, of a typical epileptic
person in any given year is given in the
following table.
Functions
Page 8
Mean and Variance
Sol’n.
𝐸 𝑋 = ෍ 𝑥𝑗 𝑓(𝑥𝑗 )
= 0(0.17) + 2(0.21) + 4(0.18) + 6(0.11) + 8(0.16) + 10(0.17) = 4.78
Functions
Page 9
Mean and Variance
Example 2. Fair six-sided die is labelled in one of
three ways: there are two sides labelled 1, three
sides labelled 2 and one side labelled 3. If it costs $1
to play and you win $1 × result from die, what is the
expected value of this game?
Die 1 2 3
x, payoff 0 1 2
f(x) 2/6 3/6 1/6
2 3 1 5
𝐸 𝑋 = ෍ 𝑥𝑗 𝑓 𝑥𝑗 =0 +1 +2 =
6 6 6 6
Functions
Page 10
Mean and Variance
Example 3. Let the random variable X denote the
time a person waits for an elevator to arrive.
Suppose the longest one would need to wait for
the elevator is 2 minutes, so that the possible
values of X (in minutes) are given by the
interval [0,2]. A possible pdf for X is given by
Find the E(X).

Functions
Page 11
Mean and Variance
Sol’n.
Functions
Page 12
Mean and Variance
Sol’n.
Functions
Page 13
Mean and Variance
Example 4. Let X be the random variable with
density function given by
−2𝑥
𝑓 𝑥 =ቊ2𝑒 if 𝑥 > 0
0 elsewhere
Find the E(X).
Functions
Page 14
Mean and Variance
Sol’n. 𝑓 𝑥 =ቊ 2𝑒 −2𝑥
if 𝑥 > 0
0 elsewhere
Functions
Page 15
Mean and Variance
Functions
Page 16
Mean and Variance
Note:
1. The variance is a measure of spread since if the

values of a random variable X tend to be far from
their mean, the variance of X will be larger than
the variance of a comparable random variable Y
whose values tend to be near their mean.
2. It is clear from (i) and (ii) and true for (Hi) that
variance is nonnegative.
3. Variance represents the moment of inertia of the
same density with respect to a perpendicular
axis through the center of gravity.
Functions
Page 17
Mean and Variance
Functions
Page 18
Mean and Variance
Example 1. Compute the variance of the random

variable X in Example 2.
5
𝐸 𝑋 =
6
Functions
Page 19
Mean and Variance
Sol’n.
𝑉𝑎𝑟 𝑋
2 2 2
2 5 2 5 3 5 1 17
= ෍ 𝑥𝑗 − 𝜇 𝑓 𝑥𝑗 = 0− + 1− + 2− =
6 6 6 6 6 6 36
Functions
Page 20
Mean and Variance
Example 2. Find the variance of the random variable
X in example 4.
2𝑒 −2𝑥 if 𝑥 > 0
Sol’n. 𝑓 𝑥 =ቊ
0 elsewhere
E(X) = 1/2
Functions
Page 21
Mean and Variance
Example 2. Find the variance of the random variable
X in example 4.
2𝑒 −2𝑥 if 𝑥 > 0
0 elsewhere
E(X) = 1/2
Functions
Page 22
Mean and Variance
Sol’n.
+∞ 2
1
𝑉𝑎𝑟 𝑋 = න 𝑥− . 2𝑒 −2𝑥 𝑑𝑥
0 2
+∞ 2 1
= 2‫׬‬0 𝑥 −𝑥+ 4
𝑒 −2𝑥 𝑑𝑥
+∞ 2 −2𝑥 +∞ +∞ 1 −2𝑥
=2 ‫׬‬0 𝑥 𝑒 𝑑𝑥 − ‫׬‬0 𝑥𝑒 −2𝑥 𝑑𝑥 + ‫׬‬0 𝑒 𝑑𝑥
4
1
𝑉𝑎𝑟 𝑋 = 4
Functions
Page 23
Mean and Variance
Functions
Page 24
Mean and Variance
Functions
Page 25
Mean and Variance
Proof of (iv)
Functions
Page 26
Mean and Variance
Proof.
Functions
Page 27
Mean and Variance
Theorem. Below are some properties of the
variance:
1. Var (c) = 0, for a constant c
2. Var[c g(x)] = c2 Var [g(x)], for a constant c
3. Var [c g(x) + d] = c2 Var [g(x)], for constants c

and d
Functions
Page 28
Mean and Variance
Proof. (3) Var [c g(x) + d] = c2 Var [g(x)]
Functions
Page 29
Mean and Variance
Functions
Page 30
Mean and Variance
Example 1. Suppose X is a random variable with
E(X2) = 16 and E(X) = 2. Find the following:
a) Var (X) Ans. 12
b) E(5X - 2) Ans. 8
c) Var (5X - 2) Ans. 300
Functions
Page 31
Mean and Variance
Example 2. Let X have the pdf
Find E(6X + 3X2).
Sol’n.
Functions
Page 34
Chebyshev and Jensen Inequality
Proof.
Functions
Page 35
Functions
Page 36
Remarks:
1
1. 𝑃( 𝑋 − 𝜇𝑥 < 𝑟𝜎𝑥 ) ≥ 1 − 𝑟 2 gives a lower bound
⟹ 𝑃(−𝑟𝜎𝑥 < 𝑋 − 𝜇𝑥 < 𝑟𝜎𝑥 )
1
= 𝑃(𝜇𝑥 − 𝑟𝜎𝑥 < 𝑋 < 𝜇𝑥 + 𝑟𝜎𝑥 ) ≥ 1 −
𝑟2
2. The Chebyshev inequality gives bound which does not

depend on the distribution of X, for the probability of
particular event described in terms of a random variable and
its mean and variance.
Functions
Page 37
Example. If X is a random variable such that E(X) = 3
and E(X2) = 13, use Chebyshev Inequality to
determine the lower bound for the P(-2 < X < 8).
Sol’n.
Functions
Page 38
Functions
Page 39
Functions
Page 40
Functions
Page 41
Functions
Page 42
To use Jensen Inequality, we need to determine if a
function g is convex.
A twice-differentiable function g: 𝐼 → ℛ is convex if

and only if 𝑔′′ 𝑥 ≥ 0 for all 𝑥 ∈ 𝐼.
Example. Let X be a positive random variable.

Compare 𝐸[𝑋 𝑎 ] with (𝐸[𝑋])𝑎 for all values of 𝑎 ∈
ℛ.
Functions
Page 43
Note if 𝑎 = 0, then
𝐸 𝑋 0 = 𝐸 1 = 1 and
(𝐸[𝑋])0 = 1
Note if 𝑎 = 1, then
𝐸 𝑋1 = 𝐸 𝑋 and
(𝐸[𝑋])1 = 𝐸[𝑋]
Functions
Page 44
For 𝑎 ≠ 0 and 𝑎 ≠ 1, let 𝑔 𝑥 = 𝑥 𝑎 . Then
𝑔′ 𝑥 = 𝑎𝑥 𝑎−1
𝑔′′ 𝑥 = 𝑎(𝑎 − 1)𝑥 𝑎−2
We see that 𝑔′′ 𝑥 > 0 (that is positive) if either

𝑎 < 0 or 𝑎 > 1. Thus, by Jensen inequality,
𝐸[𝑋 𝑎 ] ≥ (𝐸[𝑋])𝑎
and 0 < 𝑎 < 1, 𝐸[𝑋 𝑎 ] ≤ (𝐸[𝑋])𝑎

Functions
Page 45
Moments and
Moment Generating Function
Illustration:
𝜇1′ = 𝐸 𝑋1 = 𝐸 𝑋 = 𝜇𝑋
𝜇2′ = 𝐸 𝑋 2
𝜇3′ = 𝐸 𝑋 3
Functions
Page 46
Moments and
Illustration:
𝜇1 = 𝐸 (𝑋 − 𝜇𝑋 )1 = 0
𝜇2 = 𝐸 (𝑋 − 𝜇𝑋 )2 = 𝜎𝑋2
𝜇3 = 𝐸 𝑋 − 𝜇𝑋 3 (measure of skewness
or asymmetry)
𝜇4 = 𝐸 𝑋 − 𝜇𝑋 4 (measure of excess or
Expectation, Moments kurtosis)
Functions
Page 47
Moments and
Note:
𝜇3
1. The ratio 3 which is a unitless is called the
𝜎
coefficient of skewness. In general,
𝜇3
< 0, the distribution is skewed to the left
𝜎3
𝜇3
> 0, the distribution is skewed to the right
𝜎3
𝜇3
= 0, the distribution is symmetric
𝜎3
Functions
Page 48
Moments and
Functions
Page 49
Moments and
2. The quantity
𝓈 = (𝑚𝑒𝑎𝑛 − 𝑚𝑒𝑑𝑖𝑎𝑛)/(𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑎𝑖𝑡𝑖𝑜𝑛)
is an alternative measure of skewness where

− 1 ≤ 𝓈 ≤1.
𝜇4
3. The ratio − 3 is the coefficient of excess or
𝜎4
kurtosis. In general,
𝜇4
− 3 < 0, the distribution is platykurtic
𝜎4
𝜇4
𝜎4
− 3 > 0, the distribution is leptokurtic
𝜇4
− 3 = 0, the distribution is mesokurtic
𝜎4
Functions
Page 50
Moments and
Functions
Page 51
Moments and
Functions
Page 52
Moments and
Functions
Page 53
Moments and
Example. Given the following pmf,

1
𝑝𝑋 𝑥 = 5 𝐼 0,1,2,3,4 (𝑥),
Find Q1, Q2 and Q3.
Solution.
Q1
Functions
Page 54
Moments and
Q2
Q3
Functions
Page 55
Moments and
1
2. Let 𝑓𝑋 𝑥 = 𝑥 𝐼[0,2] . Find the median of X.
2
Sol’n.
Functions
Page 56
Moments and
Illustration:
At r = 1, E(X)
At r = 2, E[X(X – 1)]
At r = 3, E[X(X – 1)(X – 2)]
Functions
Page 57
Moments and
Functions
Page 58
Moments and
Applications:
1. If a moments generating function exists, then 𝑚(𝑡)is

continuously differentiable in some neighborhood of the
origin. If we differentiate the moments generating
function r times with respect to t, we have
and letting 𝑡 → 0, we find

where the symbol on the left is to be interpreted to
mean the rth derivative of 𝑚(𝑡) evaluated as 𝑡 → 0
Functions
Page 59
Moments and
2. By replacing 𝑒 𝑡𝑥 by its series expansion, the mgf is

already in terms of the moments, thus,
𝑚𝑋 𝑡 = 𝐸( 𝑒 𝑡𝑥 )
𝑥𝑡 2 𝑥𝑡 3 𝑥𝑡 4
= 𝐸( 1 + 𝑥𝑡 + 2! + 3! + 4! + ⋯)
𝑡2 𝐸 𝑥 2 𝑡3 𝐸 𝑥 3
=1+𝑡𝐸 𝑥 + +
2! 3!
𝑡4 𝐸 𝑥 4
+ +⋯)
4!
2 3 4
𝑡 𝑡 𝑡
= 1 + 𝑡𝜇1′ + 𝜇2′ + 𝜇3′ + 𝜇4′ + ⋯
2! 3! 4!
Functions
Page 60
Moments and
Example 1. Let X has a density function given by
−2𝑥
𝑓 𝑥 =ቊ 2𝑒 if 𝑥 > 0
0 elsewhere
Find the 𝑚𝑋 𝑡 and use it to derive the mean and

variance of X.
Functions
Page 61
Moments and
2𝑒 −2𝑥 if 𝑥 > 0
0 elsewhere
Functions
Page 62
Moments and
Solve for E(X)
Functions
Page 63
Moments and
Solve for Var(X)
Functions
Page 64
Moments and
Example 2. Let X has a pmf given by
𝑒 −3 3𝑥
𝑓𝑋 𝑥 = ቐ 𝑥! , 𝑥 = 0, 1, 2, …
0 elsewhere
Find the 𝑚𝑋 𝑡 and use it to derive the mean and

variance of X.
Functions
Page 65
Moments and
𝑒 −3 3𝑥
Sol’n. 𝑓𝑋 𝑥 = ቐ 𝑥! , 𝑥 = 0, 1, 2, …
0 elsewhere
Functions
Page 66
Moments and
Solve for E(X)
Functions
Page 67
Moments and
Solve for Var(X)
Functions
Page 68
Moments and
1 𝑡 2 2𝑡 3 3𝑡
Example 3. Let 𝑚𝑋 𝑡 = 𝑒 + 𝑒 + 𝑒 . Find
6 6 6
the density function of X.
Sol’n.
Functions
Page 69
Moments and
Theorem. If a and b are constants, then
1. 𝑚𝑋+𝑎 𝑡 = 𝑒 𝑎𝑡 ∙ 𝑚𝑋 (𝑡)
2. 𝑚𝑏𝑋 𝑡 = 𝑚𝑋 (𝑏𝑡)
𝑎
𝑡 𝑡
3. 𝑚𝑋+𝑎 𝑡 = 𝑒 𝑏 ∙ 𝑚𝑋
𝑏
𝑏
Functions
Page 70
Moments and
Example. Let X has a density function given by
−2𝑥
𝑓 𝑥 =ቊ 2𝑒 if 𝑥 > 0
0 elsewhere
Find the 𝑚𝑋+1 𝑡 using the definition and using the

result in the previous theorem.
Functions
Page 71
Moments and
Sol’n. 𝑓 𝑥 = ቊ2𝑒
−2𝑥
if 𝑥 > 0
0 elsewhere
Functions
Page 72
Moments and
Cont’.
Functions
Page 73
Moments and
Functions
Page 74
Moments and
Remark:
The factorial moment generating function is used to generate
factorial moments in the same way as the raw moments are
obtained from E[etX] except that t approaches 1 instead of 1.
It sometimes simplifies finding moments of discrete
distributions
𝑑 𝑥
Note: 𝑑𝑡
𝐸 𝑡 |𝑡=1 = 𝐸(𝑋)
𝑑 ′′ 𝑥 |
𝐸 𝑡 𝑡=1 = 𝐸[(𝑋 𝑋−1 ]
𝑑𝑡 2
⋮
𝑑𝑟
𝐸 𝑡 𝑥 |𝑡=1 = 𝐸[(𝑋 𝑋 − 1 𝑋 − 2 … (𝑋 − 𝑟 + 1)]
𝑑𝑡 𝑟
Functions
Page 75
Moments and
Example 1. Let X has a pmf given by
𝑒 −3 3𝑥
𝑓𝑋 𝑥 = ቐ 𝑥! , 𝑥 = 0, 1, 2, …
0 elsewhere
Find the factorial moment generating function

𝐸 𝑡 𝑥 and use it to derive the mean and variance of
X.
Functions
Page 76
Moments and
Sol’n. 𝑒 −3 3𝑥
𝑓𝑋 𝑥 = ቐ 𝑥! , 𝑥 = 0, 1, 2, …
0 elsewhere
Functions
Page 77
Moments and
E(X)
Functions
Page 78
Moments and
Var(X)
Functions
Page 79
Moments and
Example 2. Let X has a pdf given by
−2𝑥
𝑓 𝑥 =ቊ 2𝑒 if 𝑥 > 0
0 elsewhere
Find the factorial moment generating function

𝐸 𝑡 𝑥 and use it to derive the mean and variance of
X.
Functions
Page 80
Moments and
Soln.
Functions
Page 81
Moments and
E(X)
Functions
Page 82
Moments and
Var(X)
Functions
Page 83

Chapter 4 Expectation, Moments and Moment Generating Functions

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 4 Expectation, Moments and Moment Generating Functions

Uploaded by

Copyright:

Available Formats

Chapter 4

Expectation, Moments and Moment

Department of Mathematics & Statistics

Toss a fair, six-sided die many times. In the long run,

1. In reality, one-sixth of the tosses will

2. The mean is a weighted average, that is, an

3. The mean is called the expected value of X,

1. In the definition of a mean of a random

1. In the definition of a mean of a random

3. (iii) of the definition is for all random variables;

= 0(0.17) + 2(0.21) + 4(0.18) + 6(0.11) + 8(0.16) + 10(0.17) = 4.78

f(x) 2/6 3/6 1/6

Find the E(X).

Find the E(X).

1. The variance is a measure of spread since if the

Example 1. Compute the variance of the random

1. Var (c) = 0, for a constant c

2. Var[c g(x)] = c2 Var [g(x)], for a constant c

3. Var [c g(x) + d] = c2 Var [g(x)], for constants c

a) Var (X) Ans. 12

c) Var (5X - 2) Ans. 300

Find E(6X + 3X2).

⟹ 𝑃(−𝑟𝜎𝑥 < 𝑋 − 𝜇𝑥 < 𝑟𝜎𝑥 )

2. The Chebyshev inequality gives bound which does not

A twice-differentiable function g: 𝐼 → ℛ is convex if

Example. Let X be a positive random variable.

We see that 𝑔′′ 𝑥 > 0 (that is positive) if either

and 0 < 𝑎 < 1, 𝐸[𝑋 𝑎 ] ≤ (𝐸[𝑋])𝑎

is an alternative measure of skewness where

Example. Given the following pmf,

1. If a moments generating function exists, then 𝑚(𝑡)is

and letting 𝑡 → 0, we find

2. By replacing 𝑒 𝑡𝑥 by its series expansion, the mgf is

Example 1. Let X has a density function given by

Find the 𝑚𝑋 𝑡 and use it to derive the mean and

Solve for E(X)

Solve for Var(X)

Example 2. Let X has a pmf given by

Find the 𝑚𝑋 𝑡 and use it to derive the mean and

Solve for E(X)

Solve for Var(X)

Theorem. If a and b are constants, then

Example. Let X has a density function given by

Find the 𝑚𝑋+1 𝑡 using the definition and using the

Example 1. Let X has a pmf given by

Find the factorial moment generating function

Example 2. Let X has a pdf given by

Find the factorial moment generating function

You might also like