MAA203 Final Exam Cheat Sheet

Discrete probability spaces Independence of random variables Lp spaces

p
- Ω is the sample space and P : P(Ω) → [0, 1] is a - Let X1 , . . . , Xn a family of random variables taking - Lp (Ω, P) = {X : Ω → R : E[|X| ] < ∞}
p
probability if values in the sets E1 , . . . , En . (X1 , . . . , Xn ) is a family - If X ∈ Lp , the p norm is given by (E[|X| ])1/p
(i) P(Ω) = 1 of r.v. iff for any choice of sets QA 1 ⊆ E1 , . . . , An ⊆ En , - The p norm is a semi-norm satisfying homogeneity
n
(ii)
SFor (An )n∈NP
a sequence of pairwise disjoint events, P[X1 ∈ A1 , . . . , Xn ∈ An ] = i=1 P[Xi ∈ Ai ]. and the triangle inequality
P n∈N An = n∈N P[An ] (σ-additivity) - X and Y are independent iff ∀x ∈ E, y ∈ - If 0 < p ≤ q < ∞, Lq (Ω, P) ⊆ Lp (ω, P)
- Uniform probability, ∀A ⊆ Ω, P[A] = Card(A) F, pX,Y (x, y) = pX (x)pY (y). - If X, Y ∈ L2 (Ω, P), then XY ∈ L1 (Ω, P)
Card(Ω)
- If X, Y independent then so is f (X), g(Y ) for some
functions f, g. Inequalities
Properties of DPS
- The convolution of pX and pY is the func- - Markov If X a non-negative r.v in L1 and  > 0,
- For A, B ⊆ Ω, if A ⊆ B,
(i) P[B \ A] = P[B] − P[A] P pX ∗ pY : R → R defined by pX ∗ pY (z) =
tion P[X ≥ ] ≤ E[X] 
x∈R pX (x)pY (z − x). - Chebyshev If X ∈ L2 , P[|X − E[X]| ≥ ] ≤ V ar(X)
(ii) P[A] ≤ P[B] -PThe density of X + Y is given by pX+Y (z) = 2
- P[A ∪ B] = P[A] + P[B] − P[A ∩ B] - Jensen If φ a convex function and E[X], E[φ(X)] <
x∈R pX,Y (x, z − x). ∞, then E[φ(X)] ≥ φ(E[X])
- Conditional probability: P[A | B] = P[A ∩ - If X, Y are independent, pX+Y = pX ∗ pY
B]/P[B]. -pCauchy-Schwartz For X, Y ∈ L2 , E[|XY |] ≤
- Bayes’s theorem: P[A | B] = P[B|A]P[A] E[X 2 ]E[Y 2 ] p
P[B] Expectation
- |Cov(X, Y )| ≤ V ar(X)V ar(Y )
- For a r.v. X, E[X] exists iff one of the two
Independence of events conditions is verified:
P CDF
- A, B are independent events if P[A ∩ B] = P[A]P[B]. (i) ω∈Ω,X(ω)≥0 X(ω)P[{ω}] < ∞ - Alternative to law of X, the CDF FX , for all x ∈ R
If P[B] > 0, P[A | B] = P[A].
P
(ii) ω∈Ω,X(ω)<0 X(ω)P[{ω}] > −∞ is given by FX (x) = P[X ≤ x]
- For a family (Ai )i∈I of events.
T They are Q indepen-
P
Then, E[X] = ω∈Ω X(ω)P[{ω}]. - Properties of the CDF:
dent if for a finite J ⊆ I, P[ j∈J Aj ] = j∈J P[Aj ] - For a function g : E → R, E[g(X)] exists iff one of (i) Non-decreasing
theP two conditions is verified: (ii) Right-continuous, limy→x− FX (y) = FX (x)
Discrete random variables (i) x:g(x)≥0 g(x)pX (x) < ∞ (iii) limx→∞ FX (x) = 1
- For a DPS (Ω, P) and E an arbitrary set, a discrete
P
(ii) x:g(x)<0 g(x)pX (x) > −∞ (iv) limx→−∞ FX (x) = 0
r.v. is a function X : Ω → E. - Continuity from below Saying a P is σ-additive
P
Then, E[g(x)] = x∈X(Ω) g(x)pX (x)
- The law/distribution of X, denoted by µX , - For two independent r.v. such that is equivalent to finite σ-additivity and Sfor any in-
is the application µX : P(E) → [0, 1] defined by E[X], E[Y ], E[XY ] < ∞, E[XY ] = E[X]E[Y ]. creasing sequence of events (Bn )n≥0 , P[ n≥0 Bn ] =
µX (A) = P[{ω ∈ Ω : X(ω) ∈ A}] = P[X −1 (A)]. The - Properties of expectation: lim supn→∞ P[Bn ]. Similar def. for continuity from
law satisfies: (i) Monotonicity, if X(ω) ≤ Y (ω)∀ ω ∈ Ω, then above.
(i) µX (E) = 1 E[X] ≤ E[Y ]. - pX (x) = FX (x) − FX (x− ) where FX (x− ) =
(ii) For (AS n )n∈N a sequence
P of pairwise disjoint (ii) |E[X]| ≤ E[|X|] limy→x FX (y) from below.
events, µX ( n∈N An ) = n∈N µX (An ). (iii) Linearity,
P∞ E[aX + Y ] = aE[X] + E[Y ]
- The discrete density is given by - E[X] = k=0 P[X > k]
pX : E → [0, 1], pX (x)
P = P[{ω ∈ Ω : X(ω) = x}] Moment Generating Functions
∀ A ⊆ E, µX (A) = x∈A pX (x) Variance and Convariance - The moment generating function of X is MX : R →
- For X a r.v. s.t. E[X 2 ] < ∞, V ar(X) = E[(X − R+ ∪ ∞ given by MX (t) = E[exp(tX)]
Joint, marginal density E[X])2 ] - If for some a > 0, MX (t) < ∞ ∀ t ∈ (−a, a), then X
- The covariance of X, Y is Cov(X, Y ) = E[(X − admits moments of all order and ∀ t ∈ P(−a, a) MX (t)
- The joint density pX,Y for two r.v., X : Ω → E, ∞ n
E[X])(Y − E[Y ]) is given by the Taylor series MX (t) = n=0 E[X n ] tn!
Y : Ω → F is the discrete density of the random (n)
vector (X, Y ) given by pX,Y (x, y) = P[X = x, Y = y] - For X ∈ L2 , V ar(X) = Cov(X, X) for all k ∈ N. MX (t) is of class C ∞ and MX (0) =
for (x, y) ∈ (X, Y ). - If E[X 2 ], E[Y 2 ] < ∞: E[X n ]
- ∀y ∈ F, pY (y) =
P (i) V ar(X) = E[X 2 ] − E[X]2
x∈E pX,Y (x, y). (Same for
PX (x)) (ii) Cov(X, Y ) = E[XY ] − E[X][Y ]
(iii) V ar(X + Y ) = V ar(X) + V ar(Y ) + 2Cov(X, Y )
 Distances Gaussians Convergence of r.v.
- Law of small numbers. Let (Xi )1≤i≤N a col- - An absolutely continuous r.v. X is a Gaussian of - Quadratic Convergence The sequence (Xn )n≥1
2
lection P of i.i.d Bernoulli’s of parameter p. Then mean µ ∈ R and variance
 σ >2 0 if its density is given converges to X quadratically if
N
X̄ N = i=1 Xi is a binomial of parameters N and by fX (x) = σ√12π exp −(x−µ)
. We have E[X] = µ (i) E[Xn2 ], E[X 2 ] < ∞ for all n ∈ N
n 2σ 2
λ 2
p and if p = N , limN →∞ pX̄ N (n) = exp(−λ) λn! i.e. and V ar(X) = σ 2 (ii) limn→∞ E[|Xn − X| ] = 0 ⇐⇒
the discrete density of Po(λ) 2
- If X ∼ N (µ, σ 2 ), then Z ∼ N (0, 1) where Z = X−µ limn→∞ kXn − XkL2 = 0
- The
P distance between µX and µY is d(µX , µY ) = ´∞ x2 √ σ
- Convergence in mean (Xn )n≥1 converges to X
1 - −∞ e− 2 dx = 2π
2 x∈E |pX (x) − pY (x)|. in mean if
- It is a true distance if d(µX , µY ) = 0 iff µX = (i) E[|Xn |], E[|X|] < ∞ for all n ∈ N
µY , d(µX , µY ) = d(µY , µX ) and for any triplet, Sum and independence of r.v. (ii) limn→∞ E[|Xn − X|] = 0
d(µX , µY ) ≤ d(µX , µZ ) + d(µZ , µY ). - Let (X1 , . . . , XN ) be real r.v. defined all on - Convergence in probability (Xn )n≥1 converges
- If X1 , . . . , XN ind. Bernoulli r.v. withPXi ∼ Be(pi ) (Ω, A, P). They are independent iff for all to X in probability if ∀  > 0, limn→∞ P[|Xn − X| ≥
n
and Y a Poisson r.v. of parameter i=1 pi , then possible choices of [a1 , b1 ], . . . , [aN , bN ], P[X1 ∈ ] = 0
Pn PN 2 QN
d( i=1 µXi , µY ) ≤ i=1 pi . If Xi ∼ Be(λ/N ), for [a1 , b1 ], . . . , XN ∈ [aN , bN ]] = i=1 P[Xi ∈ [ai , bi ]]. - Almost sure convergence (Xn )n≥1 converges
2
all i, d(µX1 +...+XN , µY ) ≤ λN - If X, Y two absolutely continuous and indepen- to X almost surely if P[{ω ∈ Ω : limn→∞ Xn (ω) =
- If X ∼ Be(p) and Y ∼ P oi(p), d(µX , µY ) ≤ p2 dent r.v. then X + Y is absolutely ´∞ continuous and X(ω)}] = 1
- If X ∼ P oi(λ), Y ∼ P oi(δ), d(µX , µY ) ≤ |λ − δ| fX+Y (z) = (fX ∗ fY )(z) = −∞ fX (x)fY (z − x)dx - Quad =>C.S.I Mean =>M.I Prob <=
2 Almost sure
- If X, Y independent r.v. s.t. X ∼ N (µX , σX ) and
General probability spaces Y ∼ N (µY , σY ), then X +Y ∼ N (µX +µY , σX +σY2 ).
2 2 - Weak law of large numbers (Xn )n≥1 a sequence
For Ω an arbitrary set, A ⊆ P(Ω) a sigma algebra of i.i.d r.v. s.t.PV ar(X1 ) < ∞. Define for each
n
and P : A → [0, 1]. The triplet (Ω, A, P) is a general Beta distributions n ≥ 1, X̄n = n1 i=1 Xi . Then, (X̄n )n≥1 converges
probability space if: - X is a beta r.v. of parameters a, b ∈ (0, ∞) if quadratically to E[X1 ]. Theorem also holds of we
(i) P(Ω) = 1
1
fX (x) = β(a,b) xa−1 (1−x)b−1 1(0,1) (x) where β(a, b) = assume (Xn )n ≥ 1 to be uncorrelated rather than
´ 1 a−1 independent.
(ii)SIf (An )n≥0 P
a sequence of disjoint events in A, then x (1 − x)b−1 dx
P[ n∈N An ] = n∈N P[An ]
0
a
- Strong law of large numbers (Xn )n≥1 i.i.d r.v.
- We have E[X] = a+b , V ar(X) = (a+b)2ab (a+b+1) s.t. E[X14 ] < ∞. Then, (X̄n )n≥1 converges almost
- The Borel sigma algebra is defined for I ⊆ R as
T - If we consider X1 , . . . , Xn independent r.v. fol- surely to E[X1 ]
B(I) = {[a,b]:a,b∈I}⊆A⊆P(I) A
lowing a uniform distribution on [0, 1] and define Yk
- There exists a G.P.S ([0, 1], P, B([0, 1])) s.t. the k-th smallest value in {X1 , . . . , Xn }, then Yk ∼
P[[a, b]] = b − a for all a, b ∈ R. Distributions
Beta(k, n − k + 1) - Bernoulli pX (k) = p if k = 1, 1 − p if k = 0,
- A function X : Ω → R is a real r.v. iff for all a, b ∈ R,
X −1 ([a, b]) − {ω : X(ω) ∈ [a, b]} ∈ A pX (k) = pk (1 − p)k−1 . Mean=p, Variance=p(1 − p)
General formulae - Binomial pX (k) = nk pk (1 − p)n−k . Mean=np,


- A real r.v. X is absolutely continuous if there P∞ n
- ex = limn→∞ (1 + nx )n = i=0 xn! Variance=npq
exists a function fX : R → R≥0 s.t. for all a, b ∈ R,
´b - Law of total P: If (Bi )i∈I a finite partition P of Ω s.t. - Geometric pX (k) = (1 − p)k−1 p. Mean= p1 ,
P[X ∈ [a, b]] = a fX (x)dx where fX is the density P[Bi ] > 0 ∀ i ∈ I, for any A ⊆ Ω, P[A] = i∈I P[A |
of X. Variance= 1−p
p2
Bi ]P[Bi ] k −λ
- The cumulative distribution function of an ab- Pn−1
- Geometric sum i=0 ri = 1−r
n - Poisson pX (k) = λ k!
e
. Mean=Variance=λ
solutely continuous r.v. is defined as ´for a discrete 1−r
- Exponential pX (k) = λe−λx . Mean= λ1 ,
t - Chernoff bound P[X ≥ e−ta MX (t)
Qa] ≤ tX 1
r.v., for all t ∈ R, Fx (t) = P[X ≤ t] = −∞ fX (x)dx. −ta Variance= λ2
- P[X ≥ a] ≤ mint>0 e i E[e ´ ]
i
1[a,b] (x)
It is continuous and of class C 1 if fx is continuous. - Uniform pX (x) = . Mean= a+b
2 ,
b
- A function g : (a, b) → (−∞, ∞) is piecewise C 1 if it - Integration by parts: u(x)v 0 (x)dx = b−a
b ´ b 0
a 1
Variance= 12 (b − a)2
is continuous on (a, b) and there exists a = a0 < a1 < u(x)v(x) a − a u (x)v(x)dx
. . . , < aN = b s.t. g is continuously differentiable on - Quotient derivative: For f (x) = g(x)/h(x),
0 0
each interval (ai , ai+1 ) for all i ∈ {0, . . . , N − 1}. f 0 (x) = g (x)h(x)−g(x)h
h2 (x)
(x)

- If FX is piecewise C 1 , then X is absolutely continu- - Exponentiation properties: am an = am+n , am

=
0 an
ous whose density is fX (x) = FX (x) am−n , (am )n = amn
- If X is absolutely continuous r.v. with density fX
and g : R → [0, ∞) a non-negative ´ ∞ piecewise continu-
ous function, then E[g(X)] = −∞ g(x)fX (x)dx.