Fundamentals and conditional probabilities

P[Ac ] = 1 P[A] P[A B] = P[A] + P[B] P[AB]

P[AB]
P[A | B] = P[AB] = P[B]P[A | B]
P[B]

A and B are mutually exclusive if P[AB] = 0.

A and B are independent if P[AB] = P[A]P[B], which also implies that P[A | B] = P[A].
Three events A, B and C are independent if every possible combination factors, e.g., P[ABC] = P[A]P[B]P[C],
P[Ac C] = P[Ac ]P[C], etc.
P
If A1 , A2 , . . . , An are mutually exclusive and P[Ai ] = 1 (i.e., the Ai are a way of breaking things down into all
possible cases), then
X
P[B] = P[B | Ai ]P[Ai ] Law of Total Probability
i
P[Aj B] P[B | Aj ]P[Aj ]
P[Aj | B] = =P Bayes Theorem
P[B] i P[B | Ai ]P[Ai ]

CDFs and densities:

F (x) = P[X x]
Z b
d
If F (x) is continuous, then f (x) = F (x) is the density and P[a X b] = f (x)dx
dx a
If F (x) has a jump at a, i.e. F (a) > lim F (x), then the size of the jump is the probability of X = a, i.e.,
xa
P[X = a] = F (a) F (a ).

Moments:
X Z
E[X] = x P[X = x] + xf (x)dx
x

(for discrete random variables, only the first part applies, for purely continuous, only the second, for mixed distri-
butions use both)
Z
If X 0, then E[X] = P[X > x]dx.
0
P
If X 0 and only takes on integer values, E[X] = k=0 P[X > k].
X Z
2 2
E[X ] = x P[X = x] + x2 f (x)dx
x
X Z
E[g(X)] = g(x)P[X = x] + g(x)f (x)dx
x
h i
2
Var[X] = E (X X ) = E X 2 (E[X])2
 

Var[aX] = a2 Var[X]
p
SD[X] = Var[X]
Cov(X, Y ) = E[XY ] E[X] E[Y ]
Cov(X, X) = Var[X]
 Cov(X, Y )
Corr(X, Y ) =
SD(X)SD(Y )
SD(X)
Coefficient of Variation of X =
E(X)

Covariance is bilinear, meaning that it distributes as you might expect. This means that Cov(aX + bY, cZ) =
acCov(X, Z) + bcCov(Y, Z),
Var(aX + bY ) = a2 VarX + 2abCov(X, Y ) + b2 VarY .

Moment generating functions:

MX (t) = E etX
 

MX (0) = E e0 = 1
 

d 
MX (t) = E[X]
dt 0
(n)

d
MX (t) = E[X n ]

dt(n) 0
MaX+b (t) = E[eatX+bt ] = ebt MX (at)

Normal, Gamma, and Binomial are the most common mgfs used on the exam.
If X and Y are independent, then MX+Y (t) = MX (t)MY (t).

Joint moment generating functions:

MX1 ,X2 (t1 , t2 ) = Eet1 X1 +t2 X2


(m) (n) 
E[X1m X2n ] = M (t , t )

(m) (n) 1 2 
x 1 x 1

(0,0)
t1 X1 +0X2
MX1 (t1 ) = Ee = MX1 ,X2 (t1 , 0)

Convolutions and transformations

Z Z
If Z = X + Y , then fZ (z) = fX,Y (x, z x)dx = fX (x)fY (z x) The last equality (convolution formula)

only holds if X and Y are indepedent.
You can also find fZ (z) by finding the CDF and differentiating.
 
d 
If Y = g(X) and g is one-to-one, then fY (y) = fX [g 1 (y)]  g 1 (y) If g is two-to-one (or more), then you can
dy
use this with each inverse and add them up.

If (Y1 , Y2 ) = g(X1 , X2 ), then fY1 ,Y2 (y1 , y2 ) = fX1 ,X2 [g 1 (y1 , y2 )] J(g 1 ),
 

 g11 g11 
 
   
1
 y1 y2   a b 
where J(g ) =   and    = ad bc
1
 g2 g21  c d 
y1 y2