Aid Sheet

1. Distributions:
Distribution pdf or pmf mean variance MGF
Bernoulli(✓) ✓x (1 ✓)1 x ✓ ✓(1 ✓) (1 ✓) + ✓et
m x
Binomial (m, ✓) x ✓ (1 ✓)m x
m✓ m✓(1 ✓) [(1 ✓) + ✓et ]m
x
e
Poisson( ) x! ⇢ tb exp[ (et 1)]
ta
(e e )/t(b a) , t 6= 0
Uniform[a, b] 1/(b a) (a + b)/2 (b a)2 /12
1 ,t = 0
Normal (µ, 2 ) (2⇡ 2
) 1/2
exp[ 2 1 2 (x µ)2 ] µ 2
exp[µt + 2 t2 /2]
Exponential( ) e x 1/ 1/ 2 (1 t/ ) 1
1 x
Exponential(✓) ✓e
✓ ✓ ✓2 (1 t✓) 1
↵
↵ 1
Gamma(↵, ) (↵) x e x ↵/ ↵/ 2 (1 t/ ) ↵ , t <
(a+b) a 1 ab
P1 ⇣Qk 1 ↵+r ⌘ tk
Beta(a, b) (a) (b) x (1 x)b 1
a/(a + b) (a+b)2 (a+b+1) 1 + k=1 r=0 ↵+ +r k!

1
P P D 2
2
(m) /m (n 1)S 2
2. LLN: Xi ! E[Xi ]; CLT: X̄n ! N (µ, n ); q 2Z
n i ⇠ t(m) ; 2 /n ⇠ F(m,n) ; 2 ⇠ 2
(n 1)
(m)
/m (n)
P
3. Method of Moments: n1 i Xik is an estimator of E[X k ]

4. Maximum Likelihood Estimation:

Q P
• Log-likelihood, l(✓) = log i f✓ (xi ) = i logf✓ (xi )
@
• Score, l0 (✓) = @✓ l(✓)

ˆ l0 (✓) = 0
• At mle (✓),
2
@ @
• Fisher Info for single obs, I(✓0 ) = E[{ @✓ logf (X|✓) ✓=✓0
}2 ] = E[ @✓ 2 logf (X|✓)
✓=✓0
]
2
@ @
• Fisher info for n obs, nI(✓0 ) = E[{ @✓ logf (X1 , X2 , ..., Xn |✓) ✓=✓0
}2 ] = E[ @✓ 2 logf (X1 , X2 , ..., Xn |✓)
✓=✓0
]
D P D
• l0 (✓0 ) ! N (0, nI(✓0 )) ; 1 00
n l (✓0 ) ! I(✓0 ); ✓ˆ ! N (✓0 , nI(✓
1
0)
)
2
ˆ =
• Plug-in est. of Fisher Info, nI(✓) @
E[ @✓ 2 logf (X1 , X2 , ..., Xn |✓)]
✓=✓ˆ

V [T2 ] 1
5. Bias[T ] = E[T ] ✓; M SE[T ] = V ar[T ] + (Bias[T ])2 ; ef f [T1 , T2 ] = V [T1 ] ; C-R Inequality: V [T ] nI(✓0 )

P
6. Consistency in probability, Tn ! ✓ as n ! 1

7. Factorization theorem: T (X1 , X2 , ...Xn ) is said to be sufficient for ✓ if the joint probability function
factors in the form, f (x1 , x2 , ...xn |✓) = g[T (x1 , x2 , ...xn ), ✓] ⇤ h(x1 , x2 , ...xn )

7. Formula related to confidence intervals: !

q
(n 1)S 2 (n 1)S 2
X̄ ± z (1+ ) p
n
; X̄ ± t (1+ )
(df =n 1)
⇤ pS ;
n 2 , 2 ; ✓ˆ ± z (1+ )
1
ˆ ;
nI(✓)
2 2 1+ 1 2
(n 1) (n 1)
2 2

7. Some quantiles from z, t and 2 table: these following numbers are just for giving you an idea how the
table values might look like. The actual quantiles that will be provided to during your test will be di↵erent.
e.g instead of z0.95 you might see z0.9

z0.95 = 1.644854 z0.975 = 1.959964 z0.99 = 2.326348 z0.995 = 2.575829

2 2 2
0.95(1) = 3.841459 0.95(2) = 5.991465 0.95(3) = 7.814728 F0.95(1,28) = 4.195972
t0.95(28) = 1.701131 t0.975(28) = 2.048407 t0.95(29) = 1.699127 t0.975(29) = 2.04523