Professional Documents
Culture Documents
- Dripto Bakshi
Inferential Statistics
Estimation Theory
Weak Law of Large Numbers & Central Limit
Theorem
Pre - Poll / Election Survey
Markov’ Inequality
+∞
E(X) = 0 𝑥 𝑓𝑋 (x) dx
Markov’ Inequality
+∞ +∞
E(X) = 0 𝑥 𝑓𝑋 (x) dx ≥ 𝑋𝑓 𝑥 𝑎 (x) dx , where a > 0
Markov’ Inequality
+∞ +∞
E(X) = 0 𝑥 𝑓𝑋 (x) dx ≥ 𝑋𝑓 𝑥 𝑎 (x) dx , where a > 0
+∞ +∞
𝑋𝑓 𝑥 𝑎 (x) dx ≥ 𝑎 𝑎 . 𝑓𝑋 (x) dx = a . P(X ≥ a)
Markov’ Inequality
+∞ +∞
E(X) = 0 𝑥 𝑓𝑋 (x) dx ≥ 𝑋𝑓 𝑥 𝑎 (x) dx , where a > 0
+∞ +∞
𝑋𝑓 𝑥 𝑎 (x) dx ≥ a 0 1. 𝑓𝑋 (x) dx = a . P(X ≥ a)
𝐸(𝑋)
P(X ≥ a) ≤
𝑎
Chebyshev’s Inequality
Consider any random variable X ~ 𝐹𝑋 (𝜇, 𝜎 2 )
Chebyshev’s Inequality
Consider any random variable X ~ 𝐹𝑋 (𝜇, 𝜎 2 )
𝐸((𝑋 −𝜇)2 ) 𝜎2
P ((𝑋 − 𝜇)2 > 𝜀 2 ) ≤ =
𝜀2 𝜀2
Chebyshev’s Inequality
Consider any random variable X ~ 𝐹𝑋 (𝜇, 𝜎 2 )
2 2 𝐸((𝑋 −𝜇)2 ) 𝜎 2
P ((𝑋 − 𝜇) > 𝜀 ) ≤ = 2
𝜀2 𝜀
𝜎2
P(|X − µ| ≥ 𝜀) ≤ 𝜀2
Weak Law of Large Numbers
𝑋1 + 𝑋2 + 𝑋3 ….𝑋𝑛
𝑀𝑛 =
𝑛
Weak Law of Large Numbers
𝑋1 + 𝑋2 + 𝑋3 ….𝑋𝑛
𝑀𝑛 =
𝑛
𝑋1 + 𝑋2 + 𝑋3 ….𝑋𝑛
𝑀𝑛 =
𝑛
𝑋1 + 𝑋2 + 𝑋3 ….𝑋𝑛
𝑀𝑛 = 𝑛
𝑋1 + 𝑋2 + 𝑋3 ….𝑋𝑛
𝑀𝑛 =
𝑛
𝑋𝑗 ~ Ber (f)
𝑋1 + 𝑋2 + 𝑋3 ….𝑋𝑛
𝑀𝑛 =
𝑛
𝑓(1−𝑓)
E(𝑀𝑛 ) = f & V(𝑀𝑛 ) =
𝑛
Using Chebyshev’s Inequality
V(𝑀𝑛 )
P(|𝑀𝑛 − f| ≥ .01) ≤ 2
(.01)
Using Chebyshev’s Inequality
𝑓(1−𝑓)
V(𝑀𝑛 ) 𝑛
P(|𝑀𝑛 − f| ≥ .01) ≤ =
(.01)2 (.01)2
Using Chebyshev’s Inequality
𝑓(1−𝑓)
V(𝑀𝑛 ) 𝑛
P(|𝑀𝑛 − f| ≥ .01) ≤ =
(.01)2 (.01)2
𝑓(1−𝑓)
We want 𝑛
< 0.05
(.01)2
𝑓(1−𝑓)
We want 𝑛
< 0.05
(.01)2
𝑓(1−𝑓)
Choose the sample size (n) such that 𝑛
< 0.05
(.01)2
𝑓(1−𝑓)
We want 𝑛
(.01)2
< 0.05
𝑓(1−𝑓)
Choose the sample size (n) such that 𝑛
(.01)2
< 0.05
𝑓(1−𝑓)
1
Therefore 𝑛
≤
(.01)2 4𝑛(.01)2
f ∈ 0,1 & 𝑓 1 − 𝑓 ≤ ¼
𝑓(1−𝑓)
1
Therefore 𝑛
≤
(.01)2 4𝑛(.01)2
1
Now < 0.05 if n ≥ 50000
4𝑛(.01)2
f ∈ 0,1 & 𝑓 1 − 𝑓 ≤ ¼
𝑓(1−𝑓)
1
Therefore 𝑛
≤
(.01)2 4𝑛(.01)2
1
Now < 0.05 if n ≥ 50000
4𝑛(.01)2
𝑆𝑛 = 𝑋1 + 𝑋2 + 𝑋3 … . 𝑋𝑛
Central Limit Theorem
𝑋1 , 𝑋2 , 𝑋3 ,…… 𝑋𝑛 are iid, where 𝑋𝑗 ~ F (𝜇, 𝜎 2 ) ∀ 𝑗 = 1,2, … . . 𝑛
𝑆𝑛 = 𝑋1 + 𝑋2 + 𝑋3 … . 𝑋𝑛
𝑆𝑛 = 𝑋1 + 𝑋2 + 𝑋3 … . 𝑋𝑛
𝑆𝑛 −E(𝑆𝑛 ) 𝑆𝑛 −n𝜇 𝑀𝑛 −𝜇
Standardize: 𝑍𝑛 = = = 𝜎
𝜎 n
V(𝑆𝑛) n
E(𝑍𝑛 ) = 0 & V(𝑍𝑛 ) = 1
E(𝑍𝑛 ) = 0 & V(𝑍𝑛 ) = 1
P(Z ≤ c) is the standard normal CDF, Φ(c), available from the normal tables
Election problem – (with CLT)
𝑓(1−𝑓)
E(𝑀𝑛 ) = f & V(𝑀𝑛 ) =
𝑛
Election problem – (with CLT)
𝑓(1−𝑓)
E(𝑀𝑛 ) = f & V(𝑀𝑛 ) =
𝑛
𝑓(1−𝑓)
E(𝑀𝑛 ) = f & V(𝑀𝑛 ) =
𝑛
𝑀𝑛 −f .01
P(| |≥ ) ≤ .05
𝑓(1−𝑓) 𝑓(1−𝑓)
𝑛 𝑛
Election problem – (with CLT)
𝑓(1−𝑓)
E(𝑀𝑛 ) = f & V(𝑀𝑛 ) =
𝑛
𝑀𝑛 −f .01
P(| |≥ ) ≤ .05
𝑓(1−𝑓) 𝑓(1−𝑓)
𝑛 𝑛
.01
P(|Z| ≥ ) ≤ .05
𝑓(1−𝑓)
𝑛
f ∈ 0,1 & 𝑓 1 − 𝑓 ≤ ¼
f ∈ 0,1 & 𝑓 1 − 𝑓 ≤ ¼ => 𝑓 1 − 𝑓 ≤ ½
f ∈ 0,1 & 𝑓 1 − 𝑓 ≤ ¼ => 𝑓 1 − 𝑓 ≤ ½
.01 .01 𝑛
≥ 1 = .02 𝑛
𝑓(1−𝑓)
2
𝑛
f ∈ 0,1 & 𝑓 1 − 𝑓 ≤ ¼ => 𝑓 1 − 𝑓 ≤ ½
.01 .01 𝑛
≥ 1 = .02 𝑛
𝑓(1−𝑓)
2
𝑛
.01
P(|Z| ≥ ) ≤ P(|Z| ≥ .02 𝑛)
𝑓(1−𝑓)
𝑛
If n = 10000, P(|Z| ≥ .02 𝑛) = P(|Z| ≥ 2) < .05
If n = 10000, P(|Z| ≥ .02 𝑛) = P(|Z| ≥ 2) < .05
Point Estimation
Interval estimation
Point Estimation
The Statistic
1
Sample mean 𝑋 = σ𝑛𝑖=1 𝑋𝑖
ത
𝑛
1
2
Sample variance 𝑠 = σ𝑛𝑖=1(𝑋𝑖 − 𝑋)
ത 2
𝑛−1
Sample Statistics
1
Sample mean 𝑋 = σ𝑛𝑖=1 𝑋𝑖
ത
𝑛
1
2
Sample variance 𝑠 = σ𝑛𝑖=1(𝑋𝑖 − 𝑋)
ത 2
𝑛−1
i. Unbiasedness
ii. Consistency
The “Good” Properties
The good / desirable properties of a point estimator are
i. Unbiasedness:
ത =𝜇
E(𝑋)
E(𝑠 2 ) = 𝜎 2
ii. Consistency
The “Good” Properties
The good / desirable properties of a point estimator are
i. Unbiasedness:
ത =𝜇
E(𝑋)
E(𝑠 2 ) = 𝜎 2
ii. Consistency:
ത =0
lim 𝑉(𝑋)
𝑛→ ∞
lim 𝑉(𝑠 2 ) = 0
𝑛→ ∞
Interval estimation of mean
Sample observations S = { 𝑋1 , 𝑋2 , 𝑋3 … . 𝑋𝑛 }
Interval estimation of mean
Sample observations S = { 𝑋1 , 𝑋2 , 𝑋3 … . 𝑋𝑛 }
𝑋𝑖 ~ N (𝜇, 𝜎 2 )
Interval estimation of mean
Sample observations S = { 𝑋1 , 𝑋2 , 𝑋3 … . 𝑋𝑛 }
𝑋𝑖 ~ N (𝜇, 𝜎 2 )
1
𝑋 = σ𝑛𝑖=1 𝑋𝑖
ത
𝑛
Interval estimation of mean
Sample observations S = { 𝑋1 , 𝑋2 , 𝑋3 … . 𝑋𝑛 }
𝑋𝑖 ~ N (𝜇, 𝜎 2 )
1
𝑋 = σ𝑛𝑖=1 𝑋𝑖
ത
𝑛
𝜎2
𝑋ത ~ N (𝜇, )
𝑛
Interval estimation of mean
Sample observations S = { 𝑋1 , 𝑋2 , 𝑋3 … . 𝑋𝑛 }
𝑋𝑖 ~ N (𝜇, 𝜎 2 )
1
𝑋ത = σ𝑛𝑖=1 𝑋𝑖
𝑛
𝜎2
𝑋ത ~ N (𝜇, )
𝑛
𝑋ത −𝜇
Z= 𝜎 ~ N (0,1)
𝑛
The Standard Normal Distribution
The Interval
P( 𝑍 < 𝑍𝛼/2 ) = 1 - 𝛼
The Interval
P( 𝑍 < 𝑍𝛼/2 ) = 1 - 𝛼
𝑋ത −𝜇
P(- 𝑍𝛼/2 < 𝜎 < 𝑍𝛼/2 ) = 1 - 𝛼
𝑛
The Interval
P( 𝑍 < 𝑍𝛼/2 ) = 1 - 𝛼
𝑋ത −𝜇
P(- 𝑍𝛼/2 < 𝜎 < 𝑍𝛼/2 ) = 1 - 𝛼
𝑛
𝜎 𝜎
P(𝜇 ∈ (𝑋ത − . 𝑍𝛼 , 𝑋ത + . 𝑍𝛼 ) = 1 - 𝛼
𝑛 2 𝑛 2
The Interval
P( 𝑍 < 𝑍𝛼/2 ) = 1 - 𝛼
𝑋ത −𝜇
P(- 𝑍𝛼/2 < 𝜎 < 𝑍𝛼/2 ) = 1 - 𝛼
𝑛
𝜎 𝜎
P(𝜇 ∈ (𝑋ത − . 𝑍𝛼 , 𝑋ത + . 𝑍𝛼 ) = 1 - 𝛼
𝑛 2 𝑛 2
𝑛−1 .𝑠 2
~ χ2(𝑛−1)
𝜎2
The t - statistic
𝑛−1 .𝑠 2
~ χ2(𝑛−1)
𝜎2
𝑋ത −𝜇
Define a statistic t = 𝑠
𝑛
The t - statistic
𝑛−1 .𝑠 2
~ χ2(𝑛−1)
𝜎2
ഥ −𝜇
𝑋
𝜎
𝑋ത −𝜇 𝑛
Define a statistic t = 𝑠 =
𝑠/𝜎
𝑛
The t - statistic
𝑛−1 .𝑠 2
~ χ2(𝑛−1)
𝜎2
ഥ −𝜇
𝑋 ഥ −𝜇
𝑋
𝜎 𝜎
𝑋ത −𝜇 𝑛 𝑛 𝑍
Define a statistic t = 𝑠 = = = ~ 𝑡𝑛−1
𝑠/𝜎 𝑛−1 .𝑠2 1
𝑛 .𝑛−1 χ2(𝑛−1)
𝜎2 .
𝑛−1
The t - statistic
𝑛−1 .𝑠 2
~ χ2(𝑛−1)
𝜎2
ഥ −𝜇
𝑋 ഥ −𝜇
𝑋
𝜎 𝜎
𝑋ത −𝜇 𝑛 𝑛 𝑍
Define a statistic t = 𝑠 = = = ~ 𝑡𝑛−1
𝑠/𝜎 𝑛−1 .𝑠2 1
𝑛 .𝑛−1 χ2(𝑛−1)
𝜎2 .
𝑛−1
(𝛼/2) (𝛼/2)
P(-𝑡𝑛−1 < t< 𝑡𝑛−1 ) =1-𝛼
(𝛼/2) 𝑋ത −𝜇 (𝛼/2)
P(-𝑡𝑛−1 < 𝑠 <𝑡𝑛−1 ) =1-𝛼
𝑛
The Interval
(𝛼/2)
P( 𝑡 < 𝑡𝑛−1 )=1-𝛼
(𝛼/2) (𝛼/2)
P(-𝑡𝑛−1 < t < 𝑡𝑛−1 ) = 1 - 𝛼
(𝛼/2) 𝑋ത −𝜇 (𝛼/2)
P(-𝑡𝑛−1 < 𝑠 <𝑡𝑛−1 ) =1-𝛼
𝑛
𝑠 (𝛼/2) ത 𝑠 (𝛼/2)
P(𝜇 ∈ (𝑋ത − . 𝑡𝑛−1 , 𝑋 + . 𝑡𝑛−1 ) =1-𝛼
𝑛 𝑛
The Interval
(𝛼/2)
P( 𝑡 < 𝑡𝑛−1 ) = 1 - 𝛼
(𝛼/2) (𝛼/2)
P(-𝑡𝑛−1 < t < 𝑡𝑛−1 ) = 1 - 𝛼
(𝛼/2) 𝑋ത −𝜇 (𝛼/2)
P(-𝑡𝑛−1 < 𝑠 <𝑡𝑛−1 ) = 1 - 𝛼
𝑛
𝑠 (𝛼/2) 𝑠 (𝛼/2)
P(𝜇 ∈ (𝑋ത − . 𝑡𝑛−1 , 𝑋ത + . 𝑡𝑛−1 ) = 1 - 𝛼
𝑛 𝑛
(𝛼/2)
We can compute 𝑋ത from the sample. n is the sample size.𝑡𝑛−1 can be computed from the t - distribution
table.
Interval estimation of Variance
𝑛−1 .𝑠 2
~ χ2(𝑛−1)
𝜎2
𝛼 𝛼
2
(1− ) 𝑛−1 .𝑠2 ( )
2 2
P( χ 𝑛−12 < < χ 𝑛−1 ) = 1-𝛼
𝜎2
Thus we have found the interval such that 𝜎 2 lies in that interval with probability
(1- 𝛼 )