Professional Documents
Culture Documents
Faculty of Engineering
Electrical and Electronics Engineering Department
Salma Elkawafi
Salma.elkawafi@uob.edu.ly
Goals
Understand the following:
Moments
Moments Generation function
Characteristic Function
Moments
Moments are a set of statistical parameters to measure a distribution. The shape of any distribution can be described
by its various ‘moments’. Four moments are commonly used:
1. Mean: the average
2. Variance: Standard deviation is the square root of the variance: an indication of how closely the values are spread
about the mean.
3. Skewness: measure the asymmetry of a distribution about its peak.
a. It is often approximated by Skew = (Mean - Median) / (Std dev).
b. If skewness is positive, the mean is bigger than the median and the distribution has a large tail of high values.
c. If skewness is negative, the mean is smaller than the median and the distribution has a large tail of small values.
4. Kurtosis: measures the peakedness or flatness of a distribution.
a. Positive kurtosis indicates a thin pointed distribution.
b. Negative kurtosis indicates a broad flat distribution.
Moments
The 𝑛th moment of a random variable X is defined to be
𝜇𝑛 = 𝐸[𝑋 𝑛 ]
The 𝑛th central moment of 𝑋 is defined to be
𝑛
𝑚𝑛 = 𝐸[ 𝑋 − 𝐸 𝑋 ]
If 𝑌 = 𝑎𝑋 + 𝑏, then 𝑀𝑌 𝑠 = 𝑒 𝑠𝑏 𝑀𝑋 𝑎𝑠
𝑒 𝑠 −1
If 𝑋 ∼ 𝑃𝑜𝑖𝑠𝑠𝑜𝑛 𝜆 ,then 𝑀𝑋 𝑠 = 𝑒 𝜆
𝜆
If 𝑋 ∼ 𝐸𝑥𝑝(𝜆), then 𝑀𝑋 (𝑠) =
𝜆−𝑠
𝜎2 𝑠2
If 𝑋 ∼ 𝑁 𝜇, 𝜎 , then 𝑀𝑋 (𝑠) = 𝑒 2 + 𝜇𝑠
Moment Generating Function 𝑀𝑋 𝑠 = 𝐸 𝑒 𝑠𝑋
Moment generating function and moments
The 𝑛-th derivative of the moment generating function evaluated at 0 equals the 𝑛-th moment of a
random variable:
𝑑𝑛 𝑀𝑋 𝑠
ቚ = 𝜇𝑛
𝑑𝑠 𝑛 𝑠=0
𝑀𝑋′ 𝑠 = 𝐸 𝑋𝑒 𝑠𝑋 → 𝑀𝑋′ 0 = 𝐸 𝑋
𝑀𝑋′′ 𝑠 = 𝐸 𝑋 2 𝑒 𝑠𝑋 → 𝑀𝑋′′ 0 = 𝐸 𝑋 2
(𝑛) (𝑛)
𝑀𝑋 𝑠 = 𝐸 𝑋 𝑛 𝑒 𝑠𝑋 → 𝑀𝑋 0 = 𝐸[𝑋 𝑛 ]
Moment Generating Function
Examples:
For each of the following random variables, find the MGF.
a) 𝑌 is a 𝑈𝑛𝑖𝑓𝑜𝑟𝑚 0,1 random variable.
b) 𝑋 is a discrete random variable with PMF
1
, 𝑘=1
𝑃𝑋 𝑘 = 3
2
, 𝑘=2
3
Solution
1 𝑒 𝑠 −1
a) 𝑀𝑌 𝑠 = 𝐸 𝑒 𝑠𝑌 = 0 𝑒 𝑠𝑋 𝑑𝑦 = 𝑠
1 2
b) 𝑀𝑋 𝑠 = 𝐸 𝑒 𝑠𝑋 = 𝑒 𝑠 + 𝑒 2𝑠
3 3
Moment Generating Function
Examples: Find the moment-generating function of the binomial random variable 𝑋 and then use it to
verify that 𝜇 = 𝑛𝑝 and 𝜎 2 = 𝑛𝑝𝑞.
Solution
We can solve this question directly using the definition of MGF, an easier way to solve it is to use the
fact that a binomial random variable can be considered as the sum of 𝑛 independent and identically
distributed (𝑖. 𝑖. 𝑑. ) Bernoulli random variables. Thus, we can write
𝑋 = 𝑋1 + 𝑋2 + ⋯ + 𝑋𝑛 ,
where 𝑋𝑖 ∼ 𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝑝). Thus,
𝑛
𝑀𝑋 𝑠 = 𝑀𝑋1 𝑠 𝑀𝑋2 𝑠 ⋯ 𝑀𝑋𝑛 𝑠 = 𝑀𝑋1 𝑠 (since the 𝑋𝑖′𝑠 are i.i.d.)
Also,
𝑀𝑋1 𝑠 = 𝐸 𝑒 𝑠𝑋1 = 𝑝𝑒 𝑠 + 𝑞.
Thus, we conclude
𝑀𝑋 𝑠 = 𝐸 𝑒 𝑠𝑋 = 𝑝𝑒 𝑠 + 𝑞 𝑛 .
Characteristic Function
The characteristic function defined as
𝜙𝑋 𝜔 = 𝑀𝑋 (𝑗𝜔) = 𝐸[𝑒 𝑗𝜔𝑋 ],
From the theory of Fourier transforms, we can easily determine the density function
from the characteristic function.
1 ∞
𝑓𝑋 𝑥 = න 𝜙𝑋 (𝜔) 𝑒 −𝑗𝜔𝑥 𝑑𝜔
2𝜋 −∞
Characteristic Function
Determine the moments from the characteristic function.
𝑛
1 𝑑𝑛
𝐸𝑋 = 𝑛 𝜙 𝜔 ቚ
𝑗 𝑑𝜔 𝑛 𝑋 𝜔=0
𝑓𝑍 𝑧 = 𝑓𝑋 𝑥 ∗ 𝑓𝑌 𝑦 → 𝜙𝑍 𝜔 = 𝜙𝑋 𝜔 𝜙𝑌 𝜔
1 ∞
𝑓𝑍 𝑧 = න 𝜙𝑍 (𝜔) 𝑒 −𝑗𝜔𝑍 𝑑𝜔
2𝜋 −∞
Characteristic Function
Example
If 𝑋 ∼ 𝐸𝑥𝑝(𝜆) find 𝜙𝑋 𝜔 , 𝑓𝑋 𝑥 = 𝜆𝑒 −𝜆𝑥 𝑢 𝑥 , where 𝑢 𝑥 is a step function. Also find the variance using the
characteristic function.
Solution
∞ ∞
𝑗𝜔𝑋 −𝜆𝑥 𝑗𝜔𝑥
𝜆 ∞ 𝜆
𝜙𝑋 𝜔 = 𝐸 𝑒 = න 𝜆𝑒 𝑒 𝑑𝑥 = න 𝜆 𝑒 𝑗𝜔−𝜆 𝑥 𝑑𝑥 = 𝑒 𝑗𝜔−𝜆 𝑥 ቚ =
0 0 −(𝜆 − 𝑗𝜔) 0 𝜆 − 𝑗𝜔
𝑛
1 𝑑
𝐸 𝑋𝑛 = 𝑛 𝜙 𝜔 ቚ
𝑗 𝑑𝜔 𝑛, 𝑋 𝜔=0
𝑑 𝑗𝜆 𝑑2 −(𝑗𝜆)(2 𝜆 − 𝑗𝜔 ) (−𝑗)
𝜙𝑋 𝜔 = , 𝜙𝑋 𝜔 =
𝑑𝜔 𝜆 − 𝑗𝜔 2 𝑑𝜔 2 𝜆 − 𝑗𝜔 4
1 𝑑 1 𝑗𝜆 1
𝐸𝑋 = 𝜙𝑋 𝜔 ቚ = =
𝑗 𝑑𝜔 𝜔=0 𝑗 𝜆 − 𝑗0 2 𝜆
2
1 𝑑2 1 −(𝑗𝜆)(2 𝜆 − 𝑗𝜔 ) (−𝑗) 2𝜆2 2
𝐸𝑋 = 2 𝜙 𝑋 𝜔 ቚ = ቚ = =
𝑗 𝑑𝜔 2 𝜔=0 𝑗2 𝜆 − 𝑗𝜔 4 𝜔=0 𝜆4 𝜆2
2
2 2 2 1 1
𝑉𝑎𝑟 𝑋 = 𝐸 𝑋 − 𝐸 𝑋 = 2 − =
𝜆 𝜆 𝜆2