BST 401 Probability Theory

Xing Qiu Ha Youn Lee
Department of Biostatistics and Computational Biology
University of Rochester
October 5, 2010
Qiu, Lee BST 401
Outline
1
Product Measures and Fubini’s Theorem
2
Kolmogorov’s Extension Theorem
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
The Classical Product Measure Theorem
Calculus analogy: double integral. Draw a diagram.
For E ∈ F
1
×F
2
, let E
x
(E
y
) be the cross-section at x (y),
E
x
= {y : (x, y) ∈ E} , E
y
= {x : (x, y) ∈ E} .
The measure of E can be calculated by iterated integration:
µ(E) =


1
µ
2
(E
x
)dµ
1
(x) =


2
µ
1
(E
y
)dµ
2
(y).
There is one and only one measure which satisfies
µ(A
1
×A
2
) = µ
1
(A
1

2
(A
2
). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ
1
(·) and µ(·), denoted by µ
1
⊗µ
2
).
Qiu, Lee BST 401
The Classical Product Measure Theorem
Calculus analogy: double integral. Draw a diagram.
For E ∈ F
1
×F
2
, let E
x
(E
y
) be the cross-section at x (y),
E
x
= {y : (x, y) ∈ E} , E
y
= {x : (x, y) ∈ E} .
The measure of E can be calculated by iterated integration:
µ(E) =


1
µ
2
(E
x
)dµ
1
(x) =


2
µ
1
(E
y
)dµ
2
(y).
There is one and only one measure which satisfies
µ(A
1
×A
2
) = µ
1
(A
1

2
(A
2
). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ
1
(·) and µ(·), denoted by µ
1
⊗µ
2
).
Qiu, Lee BST 401
The Classical Product Measure Theorem
Calculus analogy: double integral. Draw a diagram.
For E ∈ F
1
×F
2
, let E
x
(E
y
) be the cross-section at x (y),
E
x
= {y : (x, y) ∈ E} , E
y
= {x : (x, y) ∈ E} .
The measure of E can be calculated by iterated integration:
µ(E) =


1
µ
2
(E
x
)dµ
1
(x) =


2
µ
1
(E
y
)dµ
2
(y).
There is one and only one measure which satisfies
µ(A
1
×A
2
) = µ
1
(A
1

2
(A
2
). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ
1
(·) and µ(·), denoted by µ
1
⊗µ
2
).
Qiu, Lee BST 401
Fubini’s Theorem
Motivation: f (x, y) defined on D = A ×B. How do we know
that

D
f (x, y)dxdy can be computed by iterated
integration:

A

B
f (x, y)dy

dx?
The answer:

D
|f (x, y)|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measure is
replaced by a σ-finite measure. If f is integrable (


|f |dµ is
finite), we have


1
×Ω
2
f dµ =


1


2
f dµ
2


1
=


2


1
f dµ
1


2
.
Qiu, Lee BST 401
Fubini’s Theorem
Motivation: f (x, y) defined on D = A ×B. How do we know
that

D
f (x, y)dxdy can be computed by iterated
integration:

A

B
f (x, y)dy

dx?
The answer:

D
|f (x, y)|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measure is
replaced by a σ-finite measure. If f is integrable (


|f |dµ is
finite), we have


1
×Ω
2
f dµ =


1


2
f dµ
2


1
=


2


1
f dµ
1


2
.
Qiu, Lee BST 401
Fubini’s Theorem
Motivation: f (x, y) defined on D = A ×B. How do we know
that

D
f (x, y)dxdy can be computed by iterated
integration:

A

B
f (x, y)dy

dx?
The answer:

D
|f (x, y)|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measure is
replaced by a σ-finite measure. If f is integrable (


|f |dµ is
finite), we have


1
×Ω
2
f dµ =


1


2
f dµ
2


1
=


2


1
f dµ
1


2
.
Qiu, Lee BST 401
Fubini’s Theorem (II)
Without the integrability condition, sometimes you may end
up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because


1


2
f (x, y)dµ(x)dµ(y) =


1

lim
n→∞
f
n

dµ(y),
where f
n
are rectangle approximations of f from inside out
(picture it for the students) so that f
n
is dominated by |f |.
The theorem in the book allows another case, f 0 (even
if


f = +∞. The analogy: the monotone convergence
theorem, with +∞being a valid limit.
Qiu, Lee BST 401
Fubini’s Theorem (II)
Without the integrability condition, sometimes you may end
up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because


1


2
f (x, y)dµ(x)dµ(y) =


1

lim
n→∞
f
n

dµ(y),
where f
n
are rectangle approximations of f from inside out
(picture it for the students) so that f
n
is dominated by |f |.
The theorem in the book allows another case, f 0 (even
if


f = +∞. The analogy: the monotone convergence
theorem, with +∞being a valid limit.
Qiu, Lee BST 401
Fubini’s Theorem (II)
Without the integrability condition, sometimes you may end
up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because


1


2
f (x, y)dµ(x)dµ(y) =


1

lim
n→∞
f
n

dµ(y),
where f
n
are rectangle approximations of f from inside out
(picture it for the students) so that f
n
is dominated by |f |.
The theorem in the book allows another case, f 0 (even
if


f = +∞. The analogy: the monotone convergence
theorem, with +∞being a valid limit.
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒


|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
finite expectation).
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒


|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
finite expectation).
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒


|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
finite expectation).
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒


|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
finite expectation).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an infinite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an infinite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an infinite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an infinite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an infinite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
Measure Extension
This definition is good for rectangles because rectangles
are essentially finite dimensional.
F of course includes much more than just rectangles
because we allow countable infinite set operations. If we
fix finite dimensions, these set operations lead to finer and
finer measurable sets (a la Carathéodory extension
theorem). We also have another countable infinity: the
dimensions, which is not a problem for finite-dim space. A
typical example is an infinite-dim rectangle: A =
¸

n=1
A
i
,
A
i
∈ F
i
.
Heuristically speaking, P (A) =
¸

n=1
P(A
i
) or equivalently,
log(P (A)) =
¸

n=1
log(P(A
i
)). This series converges
because P is a probability measure so log(P(A
i
)) 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Measure Extension
This definition is good for rectangles because rectangles
are essentially finite dimensional.
F of course includes much more than just rectangles
because we allow countable infinite set operations. If we
fix finite dimensions, these set operations lead to finer and
finer measurable sets (a la Carathéodory extension
theorem). We also have another countable infinity: the
dimensions, which is not a problem for finite-dim space. A
typical example is an infinite-dim rectangle: A =
¸

n=1
A
i
,
A
i
∈ F
i
.
Heuristically speaking, P (A) =
¸

n=1
P(A
i
) or equivalently,
log(P (A)) =
¸

n=1
log(P(A
i
)). This series converges
because P is a probability measure so log(P(A
i
)) 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Measure Extension
This definition is good for rectangles because rectangles
are essentially finite dimensional.
F of course includes much more than just rectangles
because we allow countable infinite set operations. If we
fix finite dimensions, these set operations lead to finer and
finer measurable sets (a la Carathéodory extension
theorem). We also have another countable infinity: the
dimensions, which is not a problem for finite-dim space. A
typical example is an infinite-dim rectangle: A =
¸

n=1
A
i
,
A
i
∈ F
i
.
Heuristically speaking, P (A) =
¸

n=1
P(A
i
) or equivalently,
log(P (A)) =
¸

n=1
log(P(A
i
)). This series converges
because P is a probability measure so log(P(A
i
)) 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Implications
What we can describe (measure) in the infinite product
space that can not be described by finite product? Limits.
We can measure for example these sets:
A =

ω : lim
n→∞
1
n
n
¸
i =1
X
i
(ω) = EX
¸
,
B =

ω : lim
n→∞
1

n
n
¸
i =1
(Y
i
(ω) −EY) ∈ (−1.96σ, 1.96σ)
¸
.
Qiu, Lee BST 401
Implications
What we can describe (measure) in the infinite product
space that can not be described by finite product? Limits.
We can measure for example these sets:
A =

ω : lim
n→∞
1
n
n
¸
i =1
X
i
(ω) = EX
¸
,
B =

ω : lim
n→∞
1

n
n
¸
i =1
(Y
i
(ω) −EY) ∈ (−1.96σ, 1.96σ)
¸
.
Qiu, Lee BST 401
Homework
Page 470, Exercise 6.3.
Pg 121, Ash’s book, # 2.
Qiu, Lee BST 401
Homework
Page 470, Exercise 6.3.
Pg 121, Ash’s book, # 2.
Qiu, Lee BST 401

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer: Get 4 months of Scribd and The New York Times for just $1.87 per week!

Master Your Semester with a Special Offer from Scribd & The New York Times