Probability Theory and Mathematical Statistics

Homework 3, Vitaliy Pozdnyakov

Part 1

1. Let𝑋 ∼ 𝑃𝑜𝑖𝑠(𝑎), where a is unknown. Apply the maximal likelihood method to estimate 𝑎.
𝐿(𝑥1 ,…, 𝑥𝑛 ,𝑎) = ∏𝑛𝑖=1 𝑎𝑥𝑥𝑖 𝑖𝑒!−𝑎
ln𝐿 = ∑𝑛𝑖=1 ln 𝑎𝑥𝑥𝑖 𝑖𝑒!−𝑎 = ∑𝑛𝑖=1 (ln 𝑎𝑥𝑖 − ln 𝑥𝑖 ! + ln 𝑒−𝑎 ) = ∑𝑛𝑖=1 (𝑥𝑖 ln𝑎 − ln 𝑥𝑖 ! − 𝑎)
∂ln𝐿
∂𝑎 = ∑𝑛𝑖=1 ( 𝑥𝑎𝑖 − 1) = 1𝑎 ∑𝑛𝑖=1 𝑥𝑖 − ∑𝑛𝑖=1 1 = 1𝑎 ∑𝑛𝑖=1 𝑥𝑖 − 𝑛 = 0
𝑎 = ∑𝑛𝑥𝑖 = 𝑋⎯⎯⎯⎯⎯
2. Let 𝑋 ∼ 𝐵𝑖(1,𝑝) , where 𝑝 is unknown. Apply the maximal likelihood method to estimate 𝑝.

𝑋 ∼ 𝐵𝑖(1,𝑝) = Bernoulli(𝑝)
𝐿(𝑥1 ,…, 𝑥𝑛 ,𝑝) = 𝑝𝑚 (1 − 𝑝)𝑛−𝑚, where 𝑚 is the number of attempts with 𝑥𝑖 = 1
ln𝐿 = 𝑚ln𝑝 + (𝑛 − 𝑚)ln(1 − 𝑝)
∂ln𝐿 = 𝑚 − 𝑛−𝑚 = 0
∂𝑝 𝑝 1−𝑝
𝑝 = 𝑚𝑛
4. Let 𝑋 ∼ 𝑈(𝑎,𝑏) , where 𝑎 and 𝑏 are unknown. Apply the maximal likelihood method to estimate 𝑎 and
𝑏.
Let 𝐼 be an indicator:

𝐼(requirement) = { 1 if satisfiable

0 otherwise
𝐿(𝑥1 ,…, 𝑥𝑛 ,𝑎,𝑏) = (𝑏 − 𝑎)−𝑛 ∏𝑛𝑖=1 𝐼(𝑎 ≤ 𝑥𝑖 ≤ 𝑏) = (𝑏 − 𝑎)−𝑛𝐼(𝑎 ≤ 𝑥min )𝐼(𝑥max ≤ 𝑏)
where 𝑥min = min(𝑥1 ,…, 𝑥𝑛 ) and 𝑥max = max(𝑥1 ,…, 𝑥𝑛 )
Thereby if𝑎 or 𝑏 ∈ (𝑥min , 𝑥max ) then 𝐿 = 0
On the other hand, if (𝑏 − 𝑎) will be increasing, then L will be decreasing.

Consequently max𝐿 = (𝑥max − 𝑥min ) −𝑛 ⇒ 𝑏 = 𝑥max and 𝑎 = 𝑥min

Part 2

1. Let 𝑋 ∼ 𝑁(𝜇, 𝜎 2 ), where 𝜇 and 𝜎 2 are unknown. Apply the method of moments to estimate 𝜇 and 𝜎 2 .
(𝑥−𝜇)2
𝜈1 = ∫−∞+∞ 𝑥1 √2𝜋𝜎1 2 𝑒− 2𝜎2 𝑑𝑥 = 𝜇 — the first theoretical raw moment
− (𝑥−𝜇)2
𝜇2 = ∫−∞ (𝑥 − 𝜇) √2𝜋𝜎2 𝑒 2𝜎2 𝑑𝑥 = 𝜎 2 — the second theoretical central moment
+∞ 2 1

𝜈1̃  = ∑𝑛𝑥1𝑖 = 𝑋⎯⎯⎯⎯⎯ — the first empirical raw moment

𝜇̃ 2 = ∑(𝑥𝑛𝑖−⎯⎯𝑥)2 = 𝑠2 — the second empirical central moment
Thereby 𝜇 = 𝑋⎯⎯⎯⎯⎯ and 𝜎 2 = 𝑠2
2. Let 𝑋 ∼ 𝑈(𝑎,𝑏) , where 𝑎 and 𝑏 are unknown. Apply the method of moments to estimate 𝑎 and 𝑏 .

𝜈1 = ∫𝑎𝑏 𝑥1 𝑏−𝑎2 𝑑𝑥 = 12 (𝑏 + 𝑎) — the first theoretical raw moment

𝜈3 = ∫𝑎𝑏 𝑥3 𝑏−𝑎2 𝑑𝑥 = 13 (𝑎2 + 𝑎𝑏 + 𝑏2 ) — the third theoretical raw moment
𝜈1̃  = ∑𝑛𝑥1𝑖 = 𝑋⎯⎯⎯⎯⎯ — the first empirical raw moment
𝜈3̃  = ∑𝑛𝑥3𝑖 = ⎯𝑋⎯⎯⎯⎯⎯⎯2⎯ = 𝑠2 + 𝑋⎯⎯⎯⎯⎯2 — the third empirical raw moment
⎯⎯⎯⎯⎯
1 (𝑏 + 𝑎) = 𝑋
2
{ 13 (𝑎2 + 𝑎𝑏 + 𝑏2 ) = 𝑠2 + 𝑋⎯⎯⎯⎯⎯2
Let the solution be 𝑎 = 𝑋 − √3𝑠 and 𝑏 = 𝑋⎯⎯⎯⎯⎯ + √⎯⎯3𝑠 and check:
⎯⎯⎯⎯⎯ ⎯⎯
 ⎯⎯⎯⎯⎯
1 (𝑏 + 𝑎) = 𝑋 ⎯⎯⎯⎯⎯ − ⎯⎯𝑠 + ⎯⎯⎯⎯⎯ + ⎯⎯𝑠) = ⎯⎯⎯⎯⎯
1 (𝑋
2
{ 13 (𝑎2 + 𝑎𝑏 + 𝑏2 ) = 𝑠2 + 𝑋⎯⎯⎯⎯⎯2 ⇒ { 1 ⎯⎯⎯⎯⎯ √3⎯⎯ 2 𝑋 ⎯⎯⎯⎯√⎯ 3 ⎯⎯ 𝑋⎯⎯⎯⎯⎯ ⎯⎯ ⎯⎯⎯⎯⎯ ⎯⎯ 2 2 ⎯
2
3 ((𝑋 − √3𝑠) + (𝑋 − √3𝑠)(𝑋 + √3𝑠) + (𝑋 + √3𝑠) ) = 𝑠 + 𝑋
⎯⎯⎯⎯⎯ = ⎯⎯⎯⎯⎯
⇒ { 1 (𝑋⎯⎯⎯⎯⎯2𝑋− 2√⎯⎯3𝑠𝑋⎯⎯⎯⎯⎯ + 3𝑠2 + 𝑋⎯⎯⎯⎯⎯2 − 3𝑠2 + 𝑋⎯⎯⎯⎯⎯2 + 2√⎯⎯3𝑠𝑋⎯⎯⎯⎯⎯ + 3𝑠2 = 𝑠2 + 𝑋⎯⎯⎯⎯⎯2
𝑋
3
⎯⎯⎯⎯⎯ ⎯⎯⎯⎯⎯ ⎯⎯⎯⎯⎯ ⎯⎯⎯⎯⎯
𝑋 = 𝑋
⇒ { 1 (3𝑋⎯⎯⎯⎯⎯2 + 3𝑠2 ) = 𝑠2 + 𝑋⎯⎯⎯⎯⎯2 ⇒ { 2 = ⎯𝑋⎯⎯⎯⎯2 2 ⎯⎯⎯⎯⎯2
𝑋
3 𝑠 +𝑋 =𝑠 +𝑋
⎯⎯⎯⎯⎯ ⎯⎯ ⎯⎯⎯⎯⎯ ⎯⎯
Thereby 𝑎 = 𝑋 − √ 3𝑠 , 𝑏 = 𝑋 + √ 3𝑠

Part 3

1. In the city of Perm there are two hospitals. 1000

babies were born in the first maternity hospital and
500 were born in the second hospital during the summer. Let the probability of babies born male be
equal to 50% .

a. What are the probabilities:

i. The first hospital has > 45% of male babies;
𝑋 ∼ Bernoulli(𝑝), where 𝑝 = 0.5
𝑛1 = 1000
Via De Moivre–Laplace theorem:

𝑃(𝑎 ≤ 𝑚𝑛 ≤ 𝑏) ≈ 12 [Φ(𝑧2 ) − Φ(𝑧1 )] where 𝑎 is the left bound, 𝑏 is the right bound, 𝑧1 = √𝑎−𝑝
𝑝𝑞/𝑛1 ,
𝑧2 = √𝑏−𝑝𝑝𝑞/𝑛1 , and Φ is the Laplace function Φ(𝑥) = √22𝜋 ∫0𝑥 𝑒 −𝑡22 𝑑𝑡
𝑧1 = √0.5⋅0.5/1000
0.45−0.5 ≈ −3.16 ; 𝑧2 = 1−0.5 ≈ 31.62
√0.5⋅0.5/1000
𝑃1 (0.45 ≤ 𝑚𝑛 ≤ 1) ≈ 12 [Φ(31.62) − Φ(−3.16)] = 12 [Φ(31.62) + Φ(3.16)] = 12 [1 + 0.9984] = 0.9992
ii. The second hospital has > 45% of male babies.

𝑛2 = 500
𝑧1 = √0.45−0.5
0.5⋅0.5/500 ≈ −2.24 ; 𝑧2 =
1−0.5 ≈ 22.36
√0.5⋅0.5/500
𝑃2 (0.45 ≤ 𝑚𝑛 ≤ 1) ≈ 12 [Φ(22.36) − Φ(−2.24)] = 12 [Φ(22.36) + Φ(2.24)] ≈ 12 [1 + 0.9749] ≈ 0.9875
b. Compare these probabilities and explain.

𝑋 = 𝑋1+⋯+𝑛 𝑋𝑛 with the mean

We have the RV

𝑀(𝑋) = 𝑀( 𝑋1+⋯+𝑛 𝑋𝑛 ) = 1𝑛 𝑛𝑀(𝑋𝑖 ) = 𝑀(𝑋𝑖 ) = 𝜇 independs of 𝑛

But the variance

𝐷(𝑋) = 𝐷( 𝑋1+⋯+𝑛 𝑋𝑛 ) = 𝑛12 𝑛𝐷(𝑋𝑖 ) = 𝐷(𝑛𝑋𝑖 ) = 𝜎𝑛2 depends on 𝑛

Hence, an increase of 𝑛 leads to a decrease of 𝐷(𝑋) and values concentrate around of the mean and
𝑃1 > 𝑃2
c. Now imagine the situation that 515 male babies were born in the first maternity hospital. 240
male babies were born in the second maternity hospital. Is the difference statistically significant?

Let𝑝1 and 𝑝2 be a share of birth of boys in the first and second maternity hospitals respectively
𝐻0 : 𝑝1 = 𝑝2 = 𝑝 where 𝑝 = 0.5 and 𝑞 = 1 − 𝑝 = 0.5
𝐻1 : 𝑝1 ≠ 𝑝2
Significance level 𝛼 = 0.05

We use the z-score for compare of share:

𝑧= 𝑤1 −𝑤2 ∼ 𝑁(0,1) where 𝑤 is emperical share

√𝑝𝑞( 𝑛1 + 𝑛2 )
1 1

⎯⎯⎯⎯
Critical area 𝐺 = [−∞,−𝑧𝑐𝑟 ] ∪ [𝑧𝑐𝑟 ,+∞]
Critical values [Φ(𝑧𝑐𝑟 ) = Φ(𝑧1−𝛼 ) = 0.95] ⇒ 𝑧𝑐𝑟 = 𝑧0.95 = 1.96

1000 − 500
515 240
𝑧= ≈ 1.28
√0.5⋅0.5( 10001 + 5001 )
P-value = 𝑃(𝑝 ∈ [−∞,−1.28] ∪ [1.28,+∞]) = 1 − Φ(1.28) = 1 − 0.7984 = 0.2016 > 𝛼
𝑧 ∉ ⎯𝐺⎯⎯⎯ thereby accept 𝐻0
Answer: the difference is not statistically significant
2. A group of people want to prove that less than half of all voters support the President's policies about
𝑣
managing the financial crisis. Let be a share of voters who support the president’s policies. What is
𝐻1 and 𝐻0 with 𝛼 = 0.05
? A recent poll showed that 228
out of 500
voters support the president’s
policies. Test the hypothesis with 𝛼 = 0.05
. Find the P-value and explain the results.

Let𝑝=𝑣
𝐻0 : 𝑝 = 𝑝0 = 0.5 and 𝑞 = 1 − 𝑝 = 0.5
𝐻1 : 𝑝 < 0.5
Significance level 𝛼 = 0.05

We use the z-score for a share:

𝑧 = (𝑤−√𝑝𝑝𝑞0)√𝑛 ∼ 𝑁(0,1)
⎯⎯⎯⎯
Critical area for the left-tail case 𝐺 = [−∞, 𝑧𝑐𝑟 ]
Critical value for the left-tail case [Φ(𝑧𝑐𝑟 ) = Φ(𝑧1−2𝛼 ) = 0.9] ⇒ 𝑧𝑐𝑟 = 𝑧0.9 = −1.64

( 228
500 −0.5)√500
𝑧= √0.5⋅0.5 ≈ −1.97
P-value = 𝑃(𝑧 ∈ [−∞,−1.97]) = 1−Φ(−1.97)
2 = 2 = 0.0244 < 𝛼 = 0.05
1−0.9512

𝑧 ∈ ⎯𝐺⎯⎯⎯ thereby reject 𝐻0

Thereby the fact that less than half of all voters support the President's policies is statistically significant.