Lectures 1 & 2

Review

Definition 1: Let 𝑋 be a random variable (r.v.) with probability density function

(𝑝𝑑𝑓) 𝑓(𝑥; 𝜃), 𝜃 ∈ 𝛺, where 𝜃 is unknown parameter (may be a vector) and 𝛺 is

the parameter space:

1. For any function of 𝑋 (say, 𝑔(𝑥)). The expected value of 𝑔(𝑥) is,

∑ 𝑔(𝑥)𝑓(𝑥; 𝜃), 𝑖𝑓 𝑋 𝑖𝑠 𝑎 𝑑𝑖𝑠𝑐𝑟𝑒𝑡𝑒 𝑟. 𝑣.

𝐸𝑔(𝑋) = ∀𝑥
∞
∫ 𝑔(𝑥)𝑓(𝑥; 𝜃)𝑑𝑥, 𝑖𝑓 𝑋 𝑖𝑠 𝑎 𝑐𝑜𝑛𝑡𝑖𝑛𝑢𝑜𝑢𝑠 𝑟. 𝑣.
{ −∞

2. The mean of 𝑋 is,

𝜇 = 𝐸𝑋

That is, take 𝑔(𝑋) = 𝑋 in Part (1).

3. The variance of 𝑋 is,

𝜎 2 = 𝐸(𝑋 − 𝐸𝑋)2

= 𝐸𝑋 2 − (𝐸𝑋)2

𝐸𝑋 2 can be obtained by taking 𝑔(𝑋) = 𝑋 2 in Part (1).

4. The moment generating function (MGF) of 𝑋 is,

𝑀𝑋 (𝑡) = 𝐸𝑒 𝑡𝑋

Provided that 𝑀𝑋 (𝑡) exist for −ℎ < 𝑡 < ℎ, ℎ > 0. Note that, 𝑀𝑋 (𝑡) is a

function of 𝑡. 𝐸𝑒 𝑡𝑋 can be obtained by taking 𝑔(𝑋) = 𝑒 𝑡𝑋 in Part (1).

1
 Deriving the Mean of some Distributions

We state here some prominent statistical distributions and some of their properties

like their mean, variance and MGF. After that, we try to show how can we obtain

these properties.

Some Prominent Statistical Distributions

Name Symbol 𝑝𝑑𝑓 E( X ) Var ( X ) M X (t )
---------------------------------------------------------------------------------------------------------------------

𝑛
Binomial 𝑋~𝑏𝑖𝑛(𝑛, 𝑝) 𝑓(𝑥) = ( ) 𝑝 𝑥 (1 − 𝑝)𝑛−𝑥 , 𝑥 = 0,1,2, … , 𝑛 𝑛𝑝 𝑛𝑝(1 − 𝑝) (1 − 𝑝 + 𝑝𝑒 𝑡 )𝑛
𝑥
𝑥
𝑒−𝜆 𝜆 𝑡 −1)
Poisson 𝑋~𝑃𝑜(𝜆) 𝑓(𝑥) = 𝑥! , 𝑥 = 0,1,2,3, … 𝜆 𝜆 𝑒 𝜆(𝑒

1 𝑏+𝑎 (𝑏−𝑎)2 e bt  e at
Uniform 𝑋~𝑈(𝑎, 𝑏) 𝑓(𝑥) = , a  x  b, a  b
𝑏−𝑎 2 12 (b  a )t

(𝑥−𝜇)2
1
e  t 
2 −
𝜎2
2 2
t /2
Normal 𝑋~𝑁(𝜇, 𝜎 ) 𝑓(𝑥) = 𝑒 2𝜎2 𝜇
𝜎 √2𝜋

1 1
Exponential 𝑋~exp(𝜃) 𝑓(𝑥) = 𝑒 −𝑥/𝜃 , x  0,   0 𝜃 𝜃2
𝜃 1−𝜃𝑡

1 x0 1 𝑛/2
Chi-Square 𝑋~𝜒2𝑛 𝑓(𝑥) = 𝑥 𝑛/2−1 𝑒 −𝑥/2 , 𝑛 2𝑛 (1−2𝑡)
𝛤(𝑛/2)2 𝑛/2
n  1,2,3,...

1 x0 1 𝛼
Gamma 𝑋~𝛤(𝛼, 𝛽) 𝑓(𝑥) = 𝑥 𝛼−1 𝑒 −𝑥/𝛽 , 𝛼𝛽 𝛼𝛽 2 (1−𝛽𝑡)
𝛤(𝛼)𝛽 𝛼
  0,   0

𝛤(𝛼+𝛽) 𝛼 𝛼𝛽
Beta 𝑋~ 𝑏𝑒𝑡𝑎(𝛼, 𝛽) 𝑓(𝑥) = 𝛤(𝛼)𝛤(𝛽) 𝑥 𝛼−1 (1 − 𝑥)𝛽−1 , 0 < 𝑥 < 1 not tractable
𝛼+𝛽 (𝛼+𝛽+1)(𝛼+𝛽)2

2
Note that:

a. * If X is a discrete r. v. with 𝑝𝑑𝑓 𝑓(𝑥; 𝜃) then ∑∀𝑥 𝑓(𝑥; 𝜃) = 1,

∞
* If X is a continuous r. v. with 𝑝𝑑𝑓 𝑓(𝑥; 𝜃) then ∫−∞ 𝑓(𝑥; 𝜃)𝑑𝑥 = 1

b. 𝜒𝑛2 and exp( ) are special cases of 𝛤(𝛼, 𝛽). For example,

𝑛
* If 𝑋~𝛤(𝛼, 𝛽) and if   n / 2 and   2 then 𝑋~𝛤 ( , 2) = 𝜒2𝑛 , n  1, 2, 3,....
2

and
* If 𝑋~𝛤(𝛼, 𝛽) and if   1 then 𝑋~𝛤(1, 𝛽) = exp(𝛽).

Example 1: Let 𝑋 be a random variable (r.v.) with 𝑝𝑑𝑓 given below. Derive the

mean of 𝑋 in each of the following cases:

1. 𝑋~𝑏𝑖𝑛(𝑛, 𝑝)
2. 𝑋~𝑃𝑜(𝜆)
3. 𝑋~𝑈(𝑎, 𝑏)
4. 𝑋~𝛤(𝛼, 𝛽)
5. 𝑋~exp(𝜃)
6. 𝑋~ 𝑏𝑒𝑡𝑎(𝛼, 𝛽)
7. 𝑋~𝑁(𝜇, 𝜎 2 )
Solution:

1. 𝑋~𝑏𝑖𝑛(𝑛, 𝑝), which is a discrete random variable. The mean of 𝑋 is,

𝜇 = 𝐸𝑋
𝑛

= ∑ 𝑥 𝑏𝑖𝑛(𝑛, 𝑝)
𝑥=0
𝑛
𝑛
= ∑ 𝑥 ( ) 𝑝 𝑥 (1 − 𝑝)𝑛−𝑥
𝑥
𝑥=0

3
 𝑛
𝑛!
= ∑𝑥 𝑝 𝑥 (1 − 𝑝)𝑛−𝑥
𝑥! (𝑛 − 𝑥)!
𝑥=1
𝑛
𝑛!
=∑ 𝑝 𝑥 (1 − 𝑝)𝑛−𝑥
(𝑥 − 1)! (𝑛 − 𝑥)!
𝑥=1
𝑛
(𝑛 − 1)!
= 𝑛𝑝 ∑ 𝑝 𝑥−1 (1 − 𝑝)𝑛−𝑥
(𝑥 − 1)! (𝑛 − 𝑥)!
𝑥=1
𝑛
𝑛 − 1 𝑥−1
= 𝑛𝑝 ∑ ( ) 𝑝 (1 − 𝑝)𝑛−𝑥 .
𝑥−1
𝑥=1

Let 𝑦 = 𝑥 − 1 and 𝑚 = 𝑛 − 1 then

𝑚
𝑚
𝜇 = 𝑛𝑝 ∑ ( 𝑦 ) 𝑝 𝑦 (1 − 𝑝)𝑚−𝑦
𝑦=0
= 𝑛𝑝

2. 𝑋~𝑃𝑜(𝜆). Again, 𝑋 is a discrete random variable. Therefore, the mean of 𝑋

is,

𝜇 = 𝐸𝑋
∞

= ∑ 𝑥 𝑃𝑜(𝜆)
𝑥=0
∞ −𝜆 𝑥
𝑒 𝜆
= ∑𝑥
𝑥!
𝑥=0
∞
𝑒 −𝜆 𝜆𝑥
=∑
(𝑥 − 1)!
𝑥=1
∞
𝑒 −𝜆 𝜆𝑥−1
= 𝜆∑
(𝑥 − 1)!
𝑥=1
Let 𝑦 = 𝑥 − 1 then
∞
𝑒 −𝜆 𝜆𝑦
𝜇 =𝜆∑
𝑦!
𝑦=0

=𝜆

4
3. 𝑋~𝑈(𝑎, 𝑏). 𝑋 is a continuous random variable. Therefore, the mean of 𝑋 is,

𝜇 = 𝐸𝑋
𝑏
= ∫ 𝑥 𝑈(𝑎, 𝑏)𝑑𝑥
𝑎
𝑏
1
=∫ 𝑥 𝑑𝑥
𝑎 𝑏−𝑎
𝑏
1
= ∫ 𝑥 𝑑𝑥
𝑏−𝑎 𝑎
1 1 2
= (𝑏 − 𝑎2 )
𝑏−𝑎2
1 1
= (𝑏 − 𝑎)(𝑏 + 𝑎)
𝑏−𝑎2
𝑏+𝑎
=
2
4. 𝑋~𝛤(𝛼, 𝛽). Again, 𝑋 is a continuous random variable. First, note that:
∞ 1 ∞
∫0 𝑥 𝛼−1 𝑒 −𝑥/𝛽 𝑑𝑥 = 1 ⇒ ∫0 𝑥 𝛼−1 𝑒 −𝑥/𝛽 𝑑𝑥 = 𝛤(𝛼)𝛽𝛼
𝛤(𝛼)𝛽 𝛼

The mean of 𝑋 is,

𝜇 = 𝐸𝑋
∞
= ∫ 𝑥 𝛤(𝛼, 𝛽)𝑑𝑥
0
∞
1
=∫ 𝑥 𝑥 𝛼−1 𝑒 −𝑥/𝛽 𝑑𝑥
𝑎 𝛤(𝛼)𝛽𝛼
∞
1
=∫ 𝑥 𝛼+1−1 𝑒 −𝑥/𝛽 𝑑𝑥
0 𝛤(𝛼)𝛽 𝛼

𝛤(𝛼 + 1)𝛽𝛼+1
=
𝛤(𝛼)𝛽 𝛼

= 𝛼𝛽

5
5. 𝑋~ exp(𝜃). We know that exp(𝜃) = 𝛤(1, 𝜃). Therefore and from Part (4), the
mean of 𝑋 is,
𝜇 = 𝐸𝑋 = 1 ∗ 𝜃 = 𝜃

6. 𝑋~ 𝑏𝑒𝑡𝑎(𝛼, 𝛽). Again, 𝑋 is a continuous random variable. First, note that:

1 𝛤(𝛼+𝛽) 1 𝛤(𝛼)𝛤(𝛽)
∫0 𝑥 𝛼−1 (1 − 𝑥)𝛽−1 𝑑𝑥 = 1 ⇒ ∫0 𝑥 𝛼−1 (1 − 𝑥)𝛽−1 𝑑𝑥 =
𝛤(𝛼)𝛤(𝛽) 𝛤(𝛼+𝛽)

The mean of 𝑋 is,

𝜇 = 𝐸𝑋
1
= ∫ 𝑥 𝑏𝑒𝑡𝑎(𝛼, 𝛽)𝑑𝑥
0
1
𝛤(𝛼 + 𝛽) 𝛼−1
=∫ 𝑥 𝑥 (1 − 𝑥)𝛽−1 𝑑𝑥
0 𝛤(𝛼)𝛤(𝛽)
𝛤(𝛼 + 𝛽) 1 𝛼+1−1
= ∫ 𝑥 (1 − 𝑥)𝛽−1 𝑑𝑥
𝛤(𝛼)𝛤(𝛽) 0
𝛤(𝛼 + 𝛽) 𝛤(𝛼 + 1)𝛤(𝛽)
=
𝛤(𝛼)𝛤(𝛽) 𝛤(𝛼 + 1 + 𝛽)
𝛤(𝛼 + 𝛽) 𝛼𝛤(𝛼)𝛤(𝛽)
=
𝛤(𝛼)𝛤(𝛽) (𝛼 + 𝛽)𝛤(𝛼 + 𝛽)
𝛼
=
𝛼+𝛽
7. 𝑋~𝑁(𝜇, 𝜎 2 ). Again, 𝑋 is a continuous random variable. First, note that:
∞
1 (𝑥−𝜇)2
−
∫ 𝑒 2𝜎2 𝑑𝑥 =1
−∞ 𝜎 √2𝜋

𝑎𝑛𝑑 if 𝜇 = 0 and 𝜎 = 1 (standard normal) then,

𝑥 2
∞ 1 −
∫−∞ √2𝜋 𝑒 2 = 1.

The mean of 𝑋 is,

6
 𝜇 (𝑚𝑒𝑎𝑛) = 𝐸𝑋
∞
= ∫ 𝑥 𝑁(𝜇, 𝜎 2 )𝑑𝑥
−∞
∞
1 (𝑥−𝜇)2
−
=∫ 𝑥 𝑒 2𝜎2 𝑑𝑥
−∞ 𝜎√2𝜋
𝑥−𝜇
Let 𝑦 = then 𝑥 = 𝜎𝑦 + 𝜇 and 𝑑𝑥 = 𝜎𝑑𝑦. Thus,
𝜎
∞ 𝑦2
1 −
𝜇(𝑚𝑒𝑎𝑛) = ∫ (𝜎𝑦 + 𝜇) 𝑒 2 𝑑𝑦
−∞ √2𝜋
∞ 𝑦2 ∞ 𝑦2
1 − 1 −
= 𝜎∫ 𝑦 𝑒 2 𝑑𝑦 +𝜇∫ 𝑒 2 𝑑𝑦
−∞ √2𝜋 −∞ √2𝜋

= 𝜎(0) + 𝜇(1)

=𝜇

𝑦2
1 −
Note that, 𝑦 is an odd function and 𝑒 2 is an even function. Their
√2𝜋

multiplication is odd function and its integral is even function. That is,

1 𝑦2
−
∫𝑦 𝑒 2 𝑑𝑦 𝑖𝑠 𝑒𝑣𝑒𝑛 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛
√2𝜋

and its integration over the symmetric interval (−∞, ∞) is zero.