Exam Cheat Sheet

Useful Formulas for Probability Measures:

P (E c ) = 1 P (E)
P (E) P (F ) if E F
P (E) = P (EF ) if E F
P (E F ) = P (E) + P (F ) P (EF )
P (E) = P (EF ) + P (EF c )

Conditional Probability: The probability of event E, given that we observe F , is:

P (E | F ) =

P (EF )
P (F )

Also, P (E | F ) defines a new probability measure with F being the restriced sample space.
Independence: Two events are independent if P (EF ) = P (E)P (F ). If P (E | F ) = P (E), E
and F are independent; when P (F ) 6= 0, the two definitions are exactly equivalent. Note that
this is true as well with the distribution functions of random variables.
Useful Formulas for Conditional Probability: Note that all of these apply to the distribution
functions of random variables as well.

P (EF ) = P (E | F )P (F ) = P (F | E)P (E)

P (E) = P (E | F )P (F ) + P (E | F c )P (F c )
P (E1 E2 En ) = P (E1 )P (E2 | E1 )P (E3 | E2 E1 ) P (En | En1 En2 E1 )
= P (En )P (En1 | En )P (En2 | En1 En ) P (E1 | E2 E3 En )
P (F | E)P (E)
P (E | F ) =
P (F )
Binomial Coefficient: For r n,


n
m


=

n!
m!(n m)!

represents the number of possible combinations of n objects taken m at a time.

Discrete Distributions:
If X Bernoulli (x; p), where p is the probability of success, then

x=1
p
1p x=0
,
P(X = x) =

0
otherwise
If X Binomial (x; n, p), where p is the prob. of success and n is the number of tries, then
 n
x
nx x {0, 1, ..., n}
x p (1 p)
P(X = x) =
,
0
otherwise
If X Geometric(x; p), then

P(X = x) =

(1 p)x p x {0, 1, ...}

,
0
otherwise

If X Poisson(x; ), where is the rate parameter, then

 e x
x {0, 1, 2, ...}
x!
P(X = x) =
0
otherwise
Continuous distributions:
If X Uniform(x; a, b), with a < b, then

fX (x) =

1
ba

x [a, b]
otherwise

If X Exponential (x; ), with being the rate, then


ex x 0
fX (x) =
0
otherwise


If X N x; , 2 (Normal Distribution), where is the expectation and 2 is the variance,
then
1
2
2
fX (x) =
e(x) /2
2
Miscellaneous Equations Sigmoid Function: The sigmoid function is used in Logistic Regression. For k features, where i is the index of the observation, it is defined by
1
f (xi ; w) =
1 + exp[(w0 + w1 xi1 + w2 xi2 + + wk xik )]
Linear Regression: For a matrix of explanatory features X and a vector of response variables y, the MLE coefficients for linear regression are given by
= (MT M)1 MT y
where M is a matrix formed by augmenting X with a column of 1s.