RS - Chapter 3 - Moments

Chapter 3
Moments of a Distribution

Expectation

We develop the expectation operator in terms of the Lebesgue integral.

• Recall that the Lebesgue measure λ(A) for some set A gives the
length/area/volume of the set A. If A = (3; 7), then λ(A) =|3 -7|= 4.

• The Lebesgue integral of f on [a,b] is defined in terms of Σi yi λ(Ai),

where 0 = y1 ≤ y2 ≤ ... ≤ yn, Ai = {x : yi ≤ f (x) < yi+1}, and λ(Ai) is the
Lebesgue measure of the set Ai.

• The value of the Lebesgue integral is the limit as the yi's are pushed
closer together. That is, we break the y-axis into a grid using {yn} and
break the x-axis into the corresponding grid {An} where
Ai = {x : f (x) є [yi; yi+1)}.

1
RS - Chapter 3 - Moments

Taking expectations: Riemann vs Lebesgue

• Riemann’s approach
Partition the base. Measure the height of the function at the center of
each interval. Calculate the area of each interval. Add all intervals.
• Lebesgue approach
Divide the range of the function. Measure the length of each
horizontal interval. Calculate the area of each interval. Add all
intervals.

Taking expectations: Riemann vs Lebesgue

• A Borel function (RV) f is integrable if and only if |f| is integrable.

• For convenience, we define the integral of a measurable function f

from (Ω, Σ , μ) to ( ¯R, ¯ B), where ¯R = R U {−∞, ∞}, ¯ B = σ(B
U{{∞}, {−∞}}).

Example: If Ω = R and μ is the Lebesgue measure, then the

Lebesgue integral of f over an interval [a, b] is written as ∫[a,b] f(x) dx =
∫ab f(x) dx, which agrees with the Riemann integral when the latter is
well defined.

However, there are functions for which the Lebesgue integrals are
defined but not the Riemann integrals.

• If μ=P, in statistics, ∫ X dP = EX = E[X] is called the expectation or

expected value of X.

2
RS - Chapter 3 - Moments

Expected Value
Consider our probability space (Ω, Σ, P) . Take an event (a set A of ω є
Ω) and X, a RV, that assigns real numbers to each ω є A.

• If we take an observation from A without knowing which ω є A will

be drawn, we may want to know what value of X(ω) we should expect to
see.

• Each of the ω є A has been assigned a probability measure P[ω],

which induces P[x]. Then, we use this to weight the values X(ω).

• P is a probability measure: The weights sum to 1. The weighted sum

provides us with a weighted average of X (ω). If P gives the "correct"
likelihood of ω being chosen, the weighted average of X(ω) –E[X]–
tells us what values of X(ω) are expected.

Expected Value
• Now with the concept of the Lebesgue integral, we take the possible
values {xi} and construct a grid on the y-axis, which gives a
corresponding grid on the x-axis in A, where
Ai = {ω є A: X(ω) ∈ [xi; xi+1)}.

Let the elements in the x-axis grid be Ai. The weighted average is
n n n

 x P[ A ]   x P
i 1
i i
i 1
i X [ X  xi ]  x
i 1
i f X ( xi )

• As we shrink the grid towards 0, A, becomes infinitesimal. Let dω be

the infinitesimal set A. The Lebesgue integral becomes:
n   
lim
n 

i 1
x i P [ Ai ]   x P[ d  ]   x PX [ X  x i ]  x f X ( x i ) dx
  

3
RS - Chapter 3 - Moments

The Expectation of X: E(X)

The expectation operator defines the mean (or population average) of
a random variable or expression.

Definition
Let X denote a discrete RV with probability function p(x) (probability
density function f(x) if X is continuous) then the expected value of X,
E(X) is defined to be:
E X   xp  x    xi p  xi 
x i

and if X is continuous with probability density function f(x)


E X   xf x dx


Sometimes we use E[.] as EX[.] to indicate that the expectation is being

taken over f X(x) dx.

Interpretation of E(X)
1. The expected value of X, E(X), is the center of gravity of the
probability distribution of X.
2. The expected value of X, E(X), is the long-run average value of X.
(To be discussed later: Law of Large Numbers)

0.4

0.3

0.2

0.1

0
0 1 2 3 4 5 6 7

E(X)

4
RS - Chapter 3 - Moments

Example: The Binomal distribution

Let X be a discrete random variable having the Binomial distribution --
i.e., X = the number of successes in n independent repetitions of a
Bernoulli trial. Find the expected value of X, E(X).

n
p  x     p x 1  p 
n x
x  0,1, 2, 3, K , n
 x
n n
n
E  X    xp  x    x   p x 1  p 
n x

x0 x 0  x
n
n
  x   p x 1  p 
n x

x 1  x
n
n!
x p x 1  p 
n x

x 1 x ! n  x  !

Example: Solution
n n
n
E  X    xp  x    x   p x 1  p 
n x

x 0 x 0  x 
n
n
  x   p x 1  p 
n x

x 1  x 
n
n!
x p x 1  p 
n x

x 1 x ! n  x  !
n
n!
 p x 1  p 
n x

x 1  x  1 ! n  x  !
n! n!
p1 1  p   p 2 1  p  
n 1 n2

0! n  1 ! 1! n  2  !
n! n!
K p n 1 1  p   pn
 n  2 !1!  n  1!0!

5
RS - Chapter 3 - Moments

Example: Solution
  n  1 !  n  1 ! p 1 1  p n  2 
p 0 1  p    
n 1
 np 
 0 !  n  1  ! 1!  n  2  !

 n  1 ! p n  2 1  p   n  1 ! p n 1 
K 
⋯   
 n  2  !1!  n  1  !0 ! 

 n  1  0  n  1 1
 p 1  p     p 1  p 
n 1 n2
 np  
 0   1 
 n 1  n  1 
⋯
K   p
n2
1  p    n 1
 p 
 n  2   n  1  

 n p 1 
n 1
 n p  p  1  p  
n 1
 np

Example: Exponential Distribution

Let X have an exponential distribution with parameter . The

probability density function of X is:

 e   x x0
f  x  
 0 x0

The expected value of X is:

 
EX    xf  x  dx   x e
 x
dx
 0

 x e
 x
We will determine dx

 udv  uv   vdu

6
RS - Chapter 3 - Moments

Example: Exponential Distribution

 xe
 x
We will determine dx using integration by parts.

In this case u  x and dv   e   x dx

Hence du  dx and v   e   x
1
 x e dx   xe   x   e   x dx   xe   x  e x
 x
Thus



 x  1
E X    x e
 x
dx   xe  e x
0
0  0

 1 1
 0  0    0   
  
Summary: If X has an exponential distribution with parameter  then:
1
EX  


Example: The Uniform distribution

Suppose X has a uniform distribution from a to b.

Then:
 b 1 a a xb
f  x  
0 x  a, x  b

The expected value of X is:

 b
EX    xf  x  dx   x 1
ba
dx
 a
b
 x2  b2  a 2 ab
  b 1 a   
 2 a 2 b  a  2

7
RS - Chapter 3 - Moments

Example: The Normal distribution

Suppose X has a Normal distribution with parameters  and .
Then:
2
 x 
1 
f  x  e 2 2
2
The expected value of X is:
  x  2
1 
E  X    xf  x  dx   x e 2 2 dx
  2

Make the substitution:

x 1
z dz  dx and x    z
 


1 z2
Hence EX       z  e  2 dz
 2
 
1 2
 2

 
z  z2
 e 2 dz  ze dz
 2 2 
 
1  z22 2

 
z
Now e dz  1 and ze 2 dz  0
 2 

T hus E  X  

8
RS - Chapter 3 - Moments

Example: The Gamma distribution

Suppose X has a Gamma distribution with parameters  and .

Then:
 
 x  1e   x x0
f  x      
 0 x0

Note:
 
   1   x
 f  x  dx   x e dx  1 if   0,   
 0
  
This is a very useful formula when working with the Gamma
distribution.

Example: The Gamma distribution

The expected value of X is:
 

E X    xf  x  dx   x x  1 e   x dx
 0
  
     x
   0
 x e dx This is now
equal to 1.

     1      1

     1 
0
   1 
x  e   x dx

   1      
  
        

9
RS - Chapter 3 - Moments

Example: The Gamma distribution

Thus, if X has a Gamma ( ,) distribution, the expected value of X

is:
E(X) =  /

Special Cases: ( ,) distribution then the expected value of X is:

1. Exponential () distribution:  = 1,  arbitrary

1
E X  

2. Chi-square () distribution:  = /2,  = ½.

E X   2 
1
2

Example: The Gamma distribution

0.3

0.25

0.2

0.15

0.1

0.05

0
0 2 4 6 8 10

EX  


10
RS - Chapter 3 - Moments

The Exponential distribution

0.25

0.2

0.15

0.1

0.05

0
0 5 10 15 20 25

1
EX  


The Chi-square (2) distribution

0.14

0.12

0.1

0.08

0.06
0.04

0.02

0
0 5 10 15 20 25
E  X  

11
RS - Chapter 3 - Moments

Expectation of a function of a RV
• Let X denote a discrete RV with probability function p(x) (or pdf f(x)
if X is continuous) then the expected value of g(X), E[g(X)], is defined
to be:

E  g  X     g x p x   g x  p x 
x i
i i

and if X is continuous with probability density function f(x)


E  g  X     g  x  f  x  dx


• Examples: g(x) = (x – μ)2  E[g(x)] = E[(x – μ)2]

g(x) = (x – μ)k  E[g(x)] = E[(x – μ)k]

Expectation of a function of a RV
Example: Suppose X has a uniform distribution from 0 to b. Then:

1 0  x  b
f  x   b
0 x  0, x  b
Find the expected value of A = X2 .
If X is the length of a side of a square (chosen at random form 0 to
b) then A is the area of the square
b
 b
 1 x3  b 3  03 b 2
E  X    x f  x  dx   x b a dx   b  
2 2 2 1

 a  3  0
3  b  3

= 1/3 the maximum area of the square

12
RS - Chapter 3 - Moments

Median: An alternative central measure

• A median is described as the numeric value separating the higher half
of a sample, a population, or a probability distribution, from the lower
half.

Definition: Median
The median of a random variable X is the unique number m that satisfies
the following inequalities:
P(X ≤ m) ≥ ½ and P(X≥ m) ≥ ½.

For a continuous distribution, we have that m solves:

m 



f X ( x ) dx  f
m
X ( x ) dx  1 / 2

Median: An alternative central measure

• Calculation of medians is a popular technique in summary statistics

and summarizing statistical data, since it is simple to understand and
easy to calculate, while also giving a measure that is more robust in the
presence of outlier values than is the mean.

An optimality property
A median is also a central point which minimizes the average of the
absolute deviations. That is, a value of c that minimizes
E(|X – c|)
is the median of the probability distribution of the random variable X.

13
RS - Chapter 3 - Moments

Example I: Median of the Exponential Distribution

Let X have an exponential distribution with parameter . The
probability density function of X is:

 e   x x0
f  x  
 0 x0
The median m solves the following integral of X:


f
m
X ( x )dx  1 / 2

 

  e dx    e dx   e |m  e  1 / 2
 x  x  x   m

m m

That is, m = ln(2)/λ.

Example II: Median of the Pareto Distribution

Let X follow a Pareto distribution with parameters α (scale) and xs
(shape, usually notated xm). The pdf of X is:

  x s
 if x  x s  0
f ( x )   x  1
 0 if x  0

The median m solves the following integral of X: 
m
f X ( x ) dx  1 / 2

 
 x s x  (   1 ) 1
 x x
  (  1 ) 
dx   x s dx   x s C
m
1
m
 (  1)  1
  x s x   C |m  x s m   1 / 2  m  x s 21 / 

Note: The Pareto distribution is used to describe the distribution of

wealth.

14
RS - Chapter 3 - Moments

Moments of a Random Variable

The moments of a random variable X are used to describe the behavior
of the RV (discrete or continuous).

Definition: Kth Moment

Let X be a RV (discrete or continuous), then the kth moment of X is:
  xk p x if X is discrete
 x
k  E  X k
  
  x k f  x  dx if X is continuous
 - 
• The first moment of X,  = 1 = E(X) is the center of gravity of
the distribution of X.
• The higher moments give different information regarding the shape
of the distribution of X.

Moments of a Random Variable

Definition: Central Moments
Let X be a RV (discrete or continuous). Then, the kth central moment of
X is defined to be:
   x   k p  x  if X is discrete
 x
k  E  X   
 
k
0
 
     x   k f  x  dx if X is continuous
 - 

where  = 1 = E(X) = the first moment of X .

• The central moments describe how the probability distribution is

distributed about the center of gravity, .

15
RS - Chapter 3 - Moments

Moments of a Random Variable – 1st and 2nd

The first central moments is given by:
 10  E  X   
The second central moment depends on the spread of the
probability distribution of X about It is called the variance of X
and is denoted by the symbol var(X).

 20  E   X    
2

  = 2nd central moment.

is called the standard deviation of X

E  X    
2
 20 
  and is denoted by the symbol .

var  X    20  E   X      
2 2
 

Moments of a Random Variable – Skewness

 30  E   X    
3
• The third central moment
 
contains information about the skewness of a distribution.

 30  30
• A popular measure of skewness: 1  
3   20 
3 2

• Distribution according to skewness:

1) Symmetric distribution
0 .0 9
0 .0 8
0 .0 7
0 .0 6
0 .0 5
0 .0 4
30  0,  1  0
0 .0 3
0 .0 2
0 .0 1
0
0 5 10 15 20 25 30 35

16
RS - Chapter 3 - Moments

Moments of a Random Variable – Skewness

2) Positively skewed distribution
0 .0 8

30  0,    0
0 .0 7
0 .0 6
0 .0 5
0 .0 4
0 .0 3
0 .0 2
0 .0 1
0
0 5 10 15 20 25 30 35

3) Negatively skewed distribution

0 .0 8
0 .0 7

30  0,  1  0
0 .0 6
0 .0 5
0 .0 4
0 .0 3
0 .0 2
0 .0 1
0
0 5 1 0 1 5 2 0 2 5 3 0 3 5

Moments of a Random Variable – Skewness

• Skewness and Economics

- Zero skew means symmetrical gains and losses.
- Positive skew suggests many small losses and few rich returns.
- Negative skew indicates lots of minor wins offset by rare major losses.

• In financial markets, stock returns at the firm level show positive

skewness, but at stock returns at the aggregate (index) level show
negative skewness.

• From horse race betting and from U.S. state lotteries there is evidence
supporting the contention that gamblers are not necessarily risk-lovers
but skewness-lovers: Long shots are overbet (positve skewness loved!).

17
RS - Chapter 3 - Moments

Moments of a Random Variable – Kurtosis

 40  E   X    
4
• The fourth central moment
 
It contains information about the shape of a distribution. The
property of shape that is measured by this moment is called kurtosis.

 40  40
• The measure of (excess) kurtosis:  2  3  3
 4   20 
2

• Distributions:
1) Mesokurtic distribution
0 .0 9
0 .0 8
   0,  40 moderate in size
0 .0 7
0 .0 6
0 .0 5
0 .0 4
0 .0 3
0 .0 2
0 .0 1
0
0 5 1 0 1 5 2 0 2 5 3 0 3 5

Moments of a Random Variable – Kurtosis

2) Platykurtic distribution
   0,  40 small in size

0
0 2 0 4 0 6 0 8 0

3) Leptokurtic distribution
   0,  40 large in size

0
0 2 0 4 0 6 0 8 0

18
RS - Chapter 3 - Moments

Moments of a Random Variable

Example: The uniform distribution from 0 to 1

1 0  x 1
f x  
0 x  0, x  1

Finding the moments

 1
 x k 1 
1
1
 k   x f  x  dx   x 1dx  
k k
 
 0  k  10 k  1
Finding the central moments:
 1

 x    f  x  dx    x  12 
k k
  0
k 1dx
 0

Moments of a Random Variable

Finding the central moments (continuation):
 1

 x    f  x  dx  x  
k k
  0
k
1
2 1dx
 0

m a k in g th e s u b s titu tio n w  x  1
2

 12    
1
1 k 1 k 1
2
 w k 1  2 1
 k   w dw  
0 k
 
2

 12  k  1   12 k 1

 1
1    1
k 1
 2k k  1 if k even
 k 1   
2  k  1 
 0 if k odd

19
RS - Chapter 3 - Moments

Moments of a Random Variable

1 1 1 1
H e n c e  20   ,  30  0 ,  40  4 
2 3  1 2
2
2 5  8 0
1
Thus, v a r  X    20 
12
1
The standard deviation   var  X   20 
12

 30
The measure of skewness 1  0
3
 40 1 80
The measure of kurtosis 1  4  3   3   1.2
 1 12 
2

Alternative measures of dispersion

When the median is used as a central measure for a distribution, there are
several choices for a measure of variability:
- The range –-the length of the smallest interval containing the data
- The interquartile range -the difference between the 3rd and 1st quartiles.
- The mean absolute deviation – (1/n) Σi |xi – central measure(X)|
- The median absolute deviation (MAD) – MAD= mi(|xi - m(X)|)

These measures are more robust (to outliers) estimators of scale than the
sample variance or standard deviation.

They especially behave better with distributions without a mean or

variance, such as the Cauchy distribution.

20
RS - Chapter 3 - Moments

Rules for Expectations

  g x p x if X is discrete

 x
E  g  X     
  g  x  f  x  dx if X is continuous
 
• Rules:
1. E  c   c where c is a constant
Proof: 
if g  X  c then E  g  X    E  c    cf  x  dx
 

 c  f  x  dx  c

The proof for discrete random variables is similar.

Rules for Expectations

2. E  aX  b   aE  X   b where a , b are constants

Proof

if g  X   aX  b then E  aX  b     ax  b  f  x  dx

 
 a  xf  x  dx  b  f  x  dx
 

 aE  X   b

The proof for discrete random variables is similar.

21
RS - Chapter 3 - Moments

Rules for Expectations

3. var  X   20  E  X      E  X 2    E  X    2  12

2 2

 
Proof:

var  X   E  X        x    f  x  dx
2 2

 
 

  x  2 x    f  x  dx
2 2


  
  x f  x  dx  2  xf  x  dx    f  x  dx
2 2

  

 EX 2
  2  E  X    
 2     2  12

The proof for discrete random variables is similar.

Rules for Expectations

4. var  aX  b   a 2 var  X 

Proof:
aX b  E  aX  b   aE  X   b  a  b

var  aX  b   E  aX  b  aX b  
2

 
 E  aX  b   a  b  
2

 
 E a 2  X    
2

 
 a 2 E  X      a 2 var  X 
2

 

22
RS - Chapter 3 - Moments

Moment generating functions

The expectation of a function g(X) is given by:
  g x p x if X is discrete
 x
E  g  X     
  g  x  f  x  dx if X is continuous
 
Definition: Moment Generating Function (MGF)
Let X denote a random variable. Then, the moment generating function of X,
mX(t), is defined by:
-   e tx p  x  if X is discrete
 x
m X  t   E  e tX    
  e tx f  x  dx if X is continuous
  

23
RS - Chapter 3 - Moments

MGF: Examples

1. The Binomial distribution (parameters p, n)

n
p  x     p x 1  p 
n x
x  0,1, 2,K , n
 x
The MGF of X , mX(t) is:

mX  t   E  etX    etx p  x 
x
n
n
  etx   p x 1  p 
n x

x 0  x
n
  t x
n n
n
 
    e p 1  p      a x b n  x
n x

x 0  x  x 0  x 

 
n
  a  b   et p  1  p
n

MGF: Examples

2. The Poisson distribution (parameter )

x
p  x  e x  0,1, 2,K
x!
The MGF of X , mX(t) is:

mX  t   E  etX    etx p  x    etx  e  

n x

x x 0 x!

 e 
x
t
 
ux
 e  using eu  
t
 e   ee
x 0 x! x 0 x !

e

 et 1 

24
RS - Chapter 3 - Moments

MGF: Examples
3. The Exponential distribution (parameter )
 e   x x0
f  x  
 0 x0
The MGF of X , mX(t) is:
 
mX  t   E  e  
tX
 e f  x  dx   e
tx tx
 e  x dx
 0


 e   t   x
   e t   x dx    
0  t   0
 
 t
   t
 undefined t

MGF: Examples
4. The Standard Normal distribution ( = 0,  = 1)
1  x22
f  x  e
2
The MGF of X , mX(t) is:

mX  t   E e  
tX
 e f  x  dx
tx



1  x22
 e
tx
e dx
 2

1  x2 22 tx
 
 2
e dx

25
RS - Chapter 3 - Moments

MGF: Examples

We will now use the fact that

  xb  2
1  2


 2 a
e 2 a dx  1 for all a  0, b We have
completed
the square
 
1  x222tx t2 1  x222txt2
mX  t    e dx  e 
2
e dx
 2  2

t2 1  x  t 2 t2



e 2
e 2 dx  e 2
 2

This is 1

MGF: Examples
4. The Gamma distribution (parameters , )
 
 x  1 e   x x0
f  x      
 0 x0


The MGF of X , mX(t) is:


mX  t   E  e  
tX
 e f  x  dx
tx



   1   x
 e tx
x e dx
0
  

   1   t  x
 x e dx
0
  

26
RS - Chapter 3 - Moments

MGF: Examples
We use the fact

b a a 1  bx
0   a  x e dx  1 for all a  0, b  0

   1   t  x
mX  t    x e dx
0
  
      t   1   t  x
 
  
   t  0   
 
x e dx   
  t 
Equal to 1
The Chi-square distribution with degrees of freedom =/2, =½):

mX  t   1  2t 
 2

MGF: Properties
1. mX(0) = 1

 
m X  t   E e tX , hence m X  0   E e 0  X  E 1   1  
Note: The MGFs of the following distributions satisfy the property
mX(0) = 1

 
n
i) B in o m ial D ist'n m X t   e t p  1  p

ii) P o isso n D ist'n m X  t   e


 e t 1 

  
iii) E x p o n en tial D ist'n m X t    
 t
t2
iv) S td N o rm al D ist'n m X t   e 2

  
v) G am m a D ist'n m X t    
 t

27
RS - Chapter 3 - Moments

MGF: Properties
2.

We use the expansion of the exponential function:

 
mX  t   E etX

MGF: Properties
dk
3. mXk 
0   k m X t   k
dt t0
Now

and m X  0   1

and m X  0    2
continuing we find m X
k
0    k

28
RS - Chapter 3 - Moments

MGF: Applying Property 3 – Binomial

Property 3 is very useful in determining the moments of a RV X.
Examples:
 
n
i) B in o m ial D ist'n m X t   e t p  1  p

   pe 
n 1
m X  t   n e t p  1  p t

0   n e p  1  p   pe   np  
n 1
m X 0 0
1 


m X  t   np   n  1  e t p  1  p  e p  e  e  et 
n2 n 1
t t t
p 1 p
 

     
n2
 npe t e t p  1  p   n  1 e t p  e t p  1  p 
 
e p 1 p
n2
 npe t t
 ne t p  1  p 

m 'X' ( 0 )  np [ np  1  p ]  np [ np  q ]  n 2 p 2  npq   2

MGF: Applying Property 3 – Poisson

ii) Poisson Dist'n m X  t   e


 e t 1 


 et 1  
 et 1  t 
mX  t   e   e    e
t

mX  t    e

   et  1   2 e   e 1 2 t   e   e 1 t
 et 1  t t t

 
  e 1  2 t
  e t  2    e     e t  1
t t
 e 1  t
m X  t    2 e

 
 e t 1  2 t
  e t  3    e  
 e t 1  t
  2e
 
 e t 1  3 t  
 e t 1  2 t  
 e t 1  t
  3e  3 2 e  e

29
RS - Chapter 3 - Moments

MGF: Applying Property 3 – Poisson

To find the moments we set t = 0.

1  mX  0    e
  
 e0 1  0

2  mX  0    2 e
    e  e 1 0   2  
 e0 1  0 0

3  mX  0    3e0  3 2 e0t   e0   3  3 2  

MGF: Applying Property 3 – Exponential

  
iii) E xponential D ist'n m X  t    
 t 
d   t 
1
d   
m X  t     
dt    t dt

    1    t    1      t 
2 2

m X  t      2    t    1  2     t 
3 3

X  t   2    3    t    1  2 3      t 
4 4
m 

m X4   t   2  3     4    t    1    4 !     t 
5 5

m Xk   t    k !     t 
 k 1

30
RS - Chapter 3 - Moments

MGF: Applying Property 3 – Exponential

Thus,
1
1    mX  0       
2


2
2  mX  0   2    
3

2

k!
k  mXk   0    k !    
 k 1

k
We can calculate the following popular descriptive statistics:
- σ2 = μ02 = μ2 – μ2 = (2/λ2) – (1/λ)2 = (1/λ)2
- γ1= μ03 /σ3 = (2/λ3) / [(1/λ)2]3/2 = 2
- γ2= μ04/σ4 – 3 = (9/λ4) / [(1/λ)4] – 3 = 6

MGF: Applying Property 3 – Exponential

Note: the moments for the exponential distribution can be calculated
in an alternative way. This is done by expanding mX(t) in powers of t
and equating the coefficients of tk to the coefficients in:

 1 1
mX t      1 u  u2  u3 L
 t 1 t 1 u

t t2 t3
 1   L
 2 3

Equating the coefficients of tk we get:

k 1 k!
 or  k 
k! k k

31
RS - Chapter 3 - Moments

MGF: Applying Property 3 – Normal

iv) Standard normal distribution mX(t)=exp(t2/2)

We use the expansion of eu.

We now equate the coefficients tk in:

MGF: Applying Property 3 – Normal

If k is odd: k = 0.

For even 2k: 2k 1


 2 k ! k
2 k!

or 2k 
 2 k !
2k k !

2! 4!
T hus  1  0,  2   1,  3  0,  4  2 3
2 2  2 !

32
RS - Chapter 3 - Moments

The log of Moment Generating Functions

Let lX (t) = ln mX(t) = the log of the MGF.

T h en l X  0   ln m X  0   ln 1  0

1 mX  t  mX  0 
l X  t   mX  t   l X  0    1  
mX  t  mX  t  mX  0 

mX  t  mX  t    mX  t  
2

l X  t  
 mX  t  
2

mX  0  mX  0    mX  0  
2

l X  0    2   1    2
2

 mX  0  
2

The Log of Moment Generating Functions

Thus lX (t) = ln mX(t) is very useful for calculating the mean and
variance of a random variable

1. l X  0   
2. l X  0    2

33
RS - Chapter 3 - Moments

Log of MGF: Examples – Binomial

1. The Binomial distribution (parameters p, n)

   e p  q
n n
m X  t   et p  1  p t

l X  t   ln mX  t   n ln  e p  q 
t

1 1
l X  t   n et p   l X  0   n p  np
e pq
t
pq

l X  t   n
 
e t p et p  q  e t p et p  
e p  q
2
t

p  p  q  p  p
 2  l X  0   n  npq
 p  q
2

Log of MGF: Examples – Poisson

2. The Poisson distribution (parameter )

mX  t   e

 et 1 

l X  t   ln mX  t    et  1  
l X  t    et   l X  0   

l X  t    et  2  l X  0   

34
RS - Chapter 3 - Moments

Log of MGF: Examples – Exponential

3. The Exponential distribution (parameter )
 
 t
mX  t      t
undefined t  
l X  t   ln mX  t   ln   ln    t  if t  
1
l X  t      t 
1

 t
1
l X  t   1   t   1 
2

  t 
2

1 1
Thus   l X  0   and  2  l X  0  
 2

Log of MGF: Examples – Normal

4. The Standard Normal distribution ( = 0,  = 1)
t2
mX  t   e 2

l X  t   ln mX  t   t2
2

l X  t   t , l X  t   1

Thus   l X  0   0 and  2  l X  0   1

35
RS - Chapter 3 - Moments

Log of MGF: Examples – Gamma

5. The Gamma distribution (parameters , )

  
mX  t    
  t 
l X  t   ln mX  t     ln   ln    t  
 1  
l X  t      
 t   t

l X  t     1   t   1 
2

  t 
2

 
Hence   l X  0   and  2  l X  0   2
 

Log of MGF: Examples – Chi-squared

6. The Chi-square distribution (degrees of freedom )

mX  t   1  2t 
2


l X  t   ln mX  t    ln 1  2t 
2
 1 
l X  t     2  
2 1  2t 1  2t
2
l X  t     11  2t   2  
2

1  2t 
2

Hence   l X  0    and  2  l X  0   2

36
RS - Chapter 3 - Moments

Characteristic functions
Definition: Characteristic Function
Let X denote a random variable. Then, the characteristic function of X,
φX(t) is defined by:
 X ( t )  E ( e itx )

Since eitx = cos(xt) + i sin(xt) and ║eitx║≤ 1, then φX(t) is defined for all
t. Thus, the characteristic function always exists, but the MGF need not
exist.

Relation to the MGF: φX(t) = miX(t) = mX(it)

 k  X (t )
Calculation of moments: |t  0  i k  k
t

37