Exchangeable Sequences,

Laws of Large Numbers,

and the Mortgage Crisis.

Myung Joo Song

Advisor: Prof. Jan Mandel

May 1 2009
Introduction
The law of large numbers for i.i.d. sequence gives convergence
of sample means to a constant, i.e., a deterministic quantity.

The yield from a mortgage can be understood as a random variable.

If a bank can make a large number of mortgages such their yields
are i.i.d., the average yield will converge to a deterministic quantity.

But events that appear to be independent may in fact be only exchangeable.

In a real financial market, there is always a fundamental factor which

can affect all the events at the same time, such as a war, or an economic
crisis which ruins the assumption of independence.
Independence of random variables
Definition:
Let ( Ω, Y , P ) be a probability space. Then, the function
X : Ω → ℜ is a (real - valued) random variable if
{ω : X (ω ) ≤ r} ∈ Y ∀r ∈ ℜ.

Two random variables X and Y are independent iff

P ( X ≤ a, Y ≤ b ) = P ( X ≤ a ) ⋅ P (Y ≤ b ) ∀a, b
then also, E {( XY )} = E { X } ⋅ E {Y }

note: independent ⇒ uncorrelated (cov ( X , Y ) = 0)

but the converse is not true.
Laws of Large Numbers

Weak Law of Large Numbers:

Given ( X 1 , X 2 , L ) an infinite sequence of i.i.d . r.v.s with

E ( X i ) = µ < ∞, ∀i ∈ Ν ,
1 P
X n = ( X1 + L + X n ) → µ as n → ∞
n
( )
That is, lim P X n − µ < ε = 1 for any ε .
n→ ∞
Laws of Large Numbers

Strong Law of Large Numbers:

Given ( X 1 , X 2 , L ) an infinite sequence of i.i.d . r.v.s with

E ( X i ) = µ < ∞ , ∀i ∈ Ν ,
1 a .s .
X n = ( X1 + L + X n ) → µ as n → ∞
n
(
That is, P lim X n = µ = 1.
n →∞
)
Exchangeability

D efinition:

An infinite sequence of X 1 , L , X n , L of random

variables is said to be exchangeable if ∀ n ≥ 2,
D
( X 1 ,L , X n ) = (X π (1) , L , X π ( n ) ) ∀ π ∈ S ( n ),
w here S ( n ) is the group of perm utations of {1, L , n }.
Exchangeability
Example: Polya’s urn

An urn has initially r red and b black balls. Draw a ball at random and note
its color, and replace the ball back and add another ball of the same color.
Let X i =1 if the ith draw yields a red ball and X i =0 otherwise.
Then,
r r +1 b r+2
P (1,1,0,1) = ⋅ ⋅ ⋅
b + r b + r +1 b + r + 2 b + r + 3
r b r +1 r+2
= ⋅ ⋅ ⋅ = P (1,0,1,1) .
b + r b + r +1 b + r + 2 b + r + 3
Similarly for other cases.
The sequence X1 ,L, X n ,L is exchangeable.
 Conditional Expectation
Let X be r.v. on probability space ( Ω, S , P ) . Given another
σ -algebra Y ⊂ S , a r.v. Y is called conditional expectation
of X given Y if Y is Y -measurable, that is,
{ω ∈ Ω : Y (ω ) ≤ a} ∈ Y ∀a ∈ℜ
and ∫ YdP = ∫ XdP
A A
∀A ∈ Y .

In this case, denote Y = E ( X | Y ).

Roughly speaking, E ( X | Y ) is averaging of X to the granularity

of Y (if Y is finite, averaging on the atoms of Y ).
 LLN for exchangeable sequences
Let ( Y n )n≥0 be a sequence of σ -algebras on ( Ω, Y , P ) . Y n ⊂ Y n +1
∀n ≥ 0. Then, a sequence of r.v.s ( X n )n≥0 is called a martingale if,
(i ) E { X n } < ∞, each n;
(ii ) X n is Y n -measurable, each n;
(iii )E { X n | Y m } = X m a.s., each m ≤ n.

Martingale Convergence Theorem:

Let ( X n )n≥1 be a martingale s.t. sup E { X n } < ∞.

n

Then lim X n = X exists a.s. (and is finite a.s.). Moreover, X is in L1.

n →∞
 LLN for exchangeable sequences
Recall

{Xi} i.i.d., Ε  X 1  < +∞

N
1
⇒
N
∑X
i =1
i → Ε [ X 1 ] a.s. , a deterministic number.

Theorem:
{ X i } exchangeable, Ε  X 1  < +∞
N
1
⇒
N
∑X
i =1
i → Ε [ X1 | Y ] a.s. , a random variable for some Y
 LLN for exchangeable sequences
proof : Let an infinite sequence X = ( X 1 , X 2 ,L) of random variables
be exchangeable and Let Y n be the σ -algebra generated by all the n-
symmetric functions of X . Y n ⊇ Y n +1
If f is a measurable function for which E  X 1  < +∞, and if Y = g ( X )
is bounded n-symmetric r.v., then for 1 ≤ j ≤ n,
{ } { }
E f ( X j ) g ( X ) = E f ( X 1 ) g ( X j , X 2 ,L , X j −1 , X 1 , X j +1 ,L)
= E { f ( X 1 ) g ( X )} ,
1 n 
so that E  ∑ f ( X j ) Y  = E { f ( X1 ) Y } .
 n j =1 
(continued )
LLN for exchangeable sequences
proof : (continued )
Then, take Y as the indicator of A, 1A , so that
1 n
∫A n ∑
j =1
f ( X j ) dP = ∫ f ( X 1 ) dP
A
( A ∈ Y n ).
Then, by the definition of conditional expectation,
1 n
∑
n j =1
f ( X j ) = E { f ( X 1 ) | Y n }.

Since partial sums form a martingale, by the Martingale convergence

theorem,
∞
1 n
lim ∑ f ( X j ) = E { f ( X 1 ) | Y ∞ } a.s. where Y ∞ = I Y n .
n →∞ n
j =1 n =1
 De Finetti’s Theorem
Definition :

Let { X i } be r.v.s and let Y be a σ -field.

Say { X i } is conditionally i.i.d . given Y if for Ai ⊂ ℜ

P ( X i ∈ Ai , 1 ≤ i ≤ n | Y ) = ∏i P ( X i ∈ Ai | Y ) , and

P(Xi ∈ A |Y ) = P(X j ∈ A|Y ) a.s., for each A, i ≠ j.

De Finetti’s Theorem
De Finetti's Theorem :
If { X i } is an exchangeable sequence then { X i } is conditionally i.i.d.
given Y ∞

Proof : By the LLN of exchangeable sequence,

∞
1 n
lim ∑ f ( X j ) = E { f ( X 1 ) | Y ∞ } a.s. where Y ∞ = I Y n .
n →∞ n
j =1 n =1

1 y ≤ x 1 n 1
Let f ( y ) =  . Then, ∑ f ( X j ) = # { j ≤ n; X j ≤ x}
0 y > x n j =1 n
1 n
and lim ∑ f ( X j ) = E { f ( X 1 ) | Y ∞ } = P ( X 1 ≤ x | Y ∞ ) = F ( x ) ,
n →∞ n
j =1

where F ( x ) =P { X 1 ≤ x | Y ∞ } is a random distribution function. (cont.)

 De Finetti’s Theorem
Proof : (cont.) X 1 ,L X n are i.i.d. ⇔ ∃ F distribution function s.t.
P ( X 1 ≤ x1 ∧ L ∧ X n ≤ xn ) = F ( x1 ) ⋅L ⋅ F ( xn ) ∀x1 ,L , xn .

{
Since P ( X 1 ≤ x1 ) = E I ( −∞ , x1 ] ( X 1 ) ,}
{
P ( X 1 ≤ x1 ∧ L ∧ X k ≤ xk | ℑ∞ ) = E I ( −∞ , x1 ] ( X 1 ) ⋅L ⋅ I ( −∞ , xk ] ( X k ) | Y ∞ }
= F ( x1 ) ⋅L ⋅ F ( xk )
∀x1 ,L , xk : ω a F (ω , x1 ) ⋅L ⋅ F (ω , xk ) is Y ∞ -measurable.

{{ } }
E E I ( −∞ , x1 ] ( X 1 ) ⋅L ⋅ I ( −∞ , xk ] ( X k ) | Y ∞ | F = E {F ( x1 ) ⋅L ⋅ F ( xk ) | F }
where F ⊂ Y ∞ , and F ( xi ) is F -measurable ∀i = 1, 2,L k .
Then, { }
E I ( −∞ , x1 ] ( X 1 ) ⋅L ⋅ I ( −∞ , xk ] ( X k ) | F = F ( x1 ) ⋅L ⋅ F ( xk ) ,
and thus, P ( X 1 ≤ x1 ∧ L ∧ X k ≤ xk | F ) = F ( x1 ) ⋅L ⋅ F ( xk ) .
Application to the mortgage mess
Let a r.v. X i be the payoff from mortgage i and bank wants to spread
the risk by making a large number of such mortgages and create a mortgage
pool with deterministic payoff.

However, if X i are not i.i.d but only exchangeable, there is some nontrivial
σ -algebra Y that underlies them all, i.e. X i are conditionally i.i.d. on Y .
Then the payoff seems to be (asymtotically) deterministic but is actually a
random variable, Y -measurable, so its value changes depending on which
set in Y , the event ω is in.

Thus, there are only exchangeable sequences in reality, there is no such a

thing as i.i.d. sequence since there are always some underlying assumptions
for which set S ∈ Y we have ω ∈ S that can change .
References
Aldous, D. (1985). Exchangeability and related topics. In: École d'Été
de Probabilités de Saint-Flour XII— Hennequin P. L., ed. (1985)
Berlin: Springer. 1–198. Lecture Notes in Mathematics 1117
Doob, J.L. The development of rigor in mathematical probability (1900-
1950). Amer. Math. Monthly, 103(7):586-595, 1996. Reprinted from
Development of mathematics 1900-1950, edited by J.P.Pier, pp.157-
170, Birkhauser, Basel, 1994.
Kingman, J.F.C. Uses of exchangeability. Ann. Probability, 6(2):183-
197, 1978
Jacod, J. and Protter, P. Probability essentials. Universitext. Springer-
Verlag, Berlin, second edition, 2003
Shiryaev, A.N. Probability, Vol 95 of Graduate Texts in Mathematics.
Springer-Verlag, New York, second edition, 1996.