# BST 401 Probability Theory

Xing Qiu Ha Youn Lee
Department of Biostatistics and Computational Biology
University of Rochester
October 5, 2010
Qiu, Lee BST 401
Outline
1
Product Measures and Fubini’s Theorem
2
Kolmogorov’s Extension Theorem
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Deﬁnition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well deﬁned on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Deﬁnition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well deﬁned on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Deﬁnition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well deﬁned on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Deﬁnition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well deﬁned on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
Product measure space
The usual Euclidean space equipped with the usual
Lebesgue measure, (R
n
, B(R
n
, µ
(n)
) is a product of
several (R, B, µ).
Deﬁnition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A
1
×A
2
×. . . A
n
, A
1
, . . . , A
n
are measurable sets in each
one-dim space.
the product measure: well deﬁned on measurable
rectangles, µ(A
1
×A
2
×. . . A
n
) = µ
1
(A
1

2
(A
2
) . . . µ
n
(A
n
).
Qiu, Lee BST 401
The Classical Product Measure Theorem
Calculus analogy: double integral. Draw a diagram.
For E ∈ F
1
×F
2
, let E
x
(E
y
) be the cross-section at x (y),
E
x
= {y : (x, y) ∈ E} , E
y
= {x : (x, y) ∈ E} .
The measure of E can be calculated by iterated integration:
µ(E) =

1
µ
2
(E
x
)dµ
1
(x) =

2
µ
1
(E
y
)dµ
2
(y).
There is one and only one measure which satisﬁes
µ(A
1
×A
2
) = µ
1
(A
1

2
(A
2
). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ
1
(·) and µ(·), denoted by µ
1
⊗µ
2
).
Qiu, Lee BST 401
The Classical Product Measure Theorem
Calculus analogy: double integral. Draw a diagram.
For E ∈ F
1
×F
2
, let E
x
(E
y
) be the cross-section at x (y),
E
x
= {y : (x, y) ∈ E} , E
y
= {x : (x, y) ∈ E} .
The measure of E can be calculated by iterated integration:
µ(E) =

1
µ
2
(E
x
)dµ
1
(x) =

2
µ
1
(E
y
)dµ
2
(y).
There is one and only one measure which satisﬁes
µ(A
1
×A
2
) = µ
1
(A
1

2
(A
2
). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ
1
(·) and µ(·), denoted by µ
1
⊗µ
2
).
Qiu, Lee BST 401
The Classical Product Measure Theorem
Calculus analogy: double integral. Draw a diagram.
For E ∈ F
1
×F
2
, let E
x
(E
y
) be the cross-section at x (y),
E
x
= {y : (x, y) ∈ E} , E
y
= {x : (x, y) ∈ E} .
The measure of E can be calculated by iterated integration:
µ(E) =

1
µ
2
(E
x
)dµ
1
(x) =

2
µ
1
(E
y
)dµ
2
(y).
There is one and only one measure which satisﬁes
µ(A
1
×A
2
) = µ
1
(A
1

2
(A
2
). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ
1
(·) and µ(·), denoted by µ
1
⊗µ
2
).
Qiu, Lee BST 401
Fubini’s Theorem
Motivation: f (x, y) deﬁned on D = A ×B. How do we know
that

D
f (x, y)dxdy can be computed by iterated
integration:

A

B
f (x, y)dy

dx?

D
|f (x, y)|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measure is
replaced by a σ-ﬁnite measure. If f is integrable (

|f |dµ is
ﬁnite), we have

1
×Ω
2
f dµ =

1

2
f dµ
2

1
=

2

1
f dµ
1

2
.
Qiu, Lee BST 401
Fubini’s Theorem
Motivation: f (x, y) deﬁned on D = A ×B. How do we know
that

D
f (x, y)dxdy can be computed by iterated
integration:

A

B
f (x, y)dy

dx?

D
|f (x, y)|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measure is
replaced by a σ-ﬁnite measure. If f is integrable (

|f |dµ is
ﬁnite), we have

1
×Ω
2
f dµ =

1

2
f dµ
2

1
=

2

1
f dµ
1

2
.
Qiu, Lee BST 401
Fubini’s Theorem
Motivation: f (x, y) deﬁned on D = A ×B. How do we know
that

D
f (x, y)dxdy can be computed by iterated
integration:

A

B
f (x, y)dy

dx?

D
|f (x, y)|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measure is
replaced by a σ-ﬁnite measure. If f is integrable (

|f |dµ is
ﬁnite), we have

1
×Ω
2
f dµ =

1

2
f dµ
2

1
=

2

1
f dµ
1

2
.
Qiu, Lee BST 401
Fubini’s Theorem (II)
Without the integrability condition, sometimes you may end
up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because

1

2
f (x, y)dµ(x)dµ(y) =

1

lim
n→∞
f
n

dµ(y),
where f
n
are rectangle approximations of f from inside out
(picture it for the students) so that f
n
is dominated by |f |.
The theorem in the book allows another case, f 0 (even
if

f = +∞. The analogy: the monotone convergence
theorem, with +∞being a valid limit.
Qiu, Lee BST 401
Fubini’s Theorem (II)
Without the integrability condition, sometimes you may end
up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because

1

2
f (x, y)dµ(x)dµ(y) =

1

lim
n→∞
f
n

dµ(y),
where f
n
are rectangle approximations of f from inside out
(picture it for the students) so that f
n
is dominated by |f |.
The theorem in the book allows another case, f 0 (even
if

f = +∞. The analogy: the monotone convergence
theorem, with +∞being a valid limit.
Qiu, Lee BST 401
Fubini’s Theorem (II)
Without the integrability condition, sometimes you may end
up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because

1

2
f (x, y)dµ(x)dµ(y) =

1

lim
n→∞
f
n

dµ(y),
where f
n
are rectangle approximations of f from inside out
(picture it for the students) so that f
n
is dominated by |f |.
The theorem in the book allows another case, f 0 (even
if

f = +∞. The analogy: the monotone convergence
theorem, with +∞being a valid limit.
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒

|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
ﬁnite expectation).
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒

|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
ﬁnite expectation).
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒

|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
ﬁnite expectation).
Qiu, Lee BST 401
The L
1
space
Denote all integrable functions on measure space
(Ω, F, µ) by L
1
(Ω) := L
1
(Ω, F, µ).
f ∈ L
1
(Ω) ⇐⇒

|f |dµ < ∞. This L
1
space is a linear
space equipped with a “length”, the L
1
norm.
We can change the order of iterative integrals of L
1
functions.
The dominated convergent theorem says that if f
1
, f
2
, . . .
are a sequence of L
1
(Ω) functions and they are dominated
by g ∈ L
1
(Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L
1
space. We have to pay special attention to functions
(random variables) that are not in L
1
space (do not have
ﬁnite expectation).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Differentiating under the integral
We have established conditions under which we can
interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) −u(y) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = Ee
θZ
< ∞for θ ∈ [−, ], then EZ = ϕ

(0).
Qiu, Lee BST 401
Inﬁnite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also deﬁne product measure/integral for
inﬁnitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the inﬁnite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Inﬁnite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with ﬁnite sample).
Qiu, Lee BST 401
Inﬁnite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also deﬁne product measure/integral for
inﬁnitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the inﬁnite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Inﬁnite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with ﬁnite sample).
Qiu, Lee BST 401
Inﬁnite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also deﬁne product measure/integral for
inﬁnitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the inﬁnite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Inﬁnite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with ﬁnite sample).
Qiu, Lee BST 401
Inﬁnite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also deﬁne product measure/integral for
inﬁnitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n →∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the inﬁnite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Inﬁnite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with ﬁnite sample).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an inﬁnite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only ﬁnitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an inﬁnite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only ﬁnitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an inﬁnite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only ﬁnitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an inﬁnite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only ﬁnitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
The construction
Let (Ω
i
, F
i
, µ
i
), i = 1, 2, . . . be an inﬁnite sequence of
probability spaces.
the whole space Ω =
¸

i =1

i
is again the Cartesian
product.
Measurable rectangles are:
A
1
×A
2
×· · · ×A
n
×Ω
n+1
×Ω
n+2
×. . .. I.e., only ﬁnitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
P {ω ∈ Ω : ω
1
∈ A
1
, . . . , ω
n
∈ A
n
} =
n
¸
i =1
P
i
(A
i
).
Qiu, Lee BST 401
Measure Extension
This deﬁnition is good for rectangles because rectangles
are essentially ﬁnite dimensional.
F of course includes much more than just rectangles
because we allow countable inﬁnite set operations. If we
ﬁx ﬁnite dimensions, these set operations lead to ﬁner and
ﬁner measurable sets (a la Carathéodory extension
theorem). We also have another countable inﬁnity: the
dimensions, which is not a problem for ﬁnite-dim space. A
typical example is an inﬁnite-dim rectangle: A =
¸

n=1
A
i
,
A
i
∈ F
i
.
Heuristically speaking, P (A) =
¸

n=1
P(A
i
) or equivalently,
log(P (A)) =
¸

n=1
log(P(A
i
)). This series converges
because P is a probability measure so log(P(A
i
)) 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Measure Extension
This deﬁnition is good for rectangles because rectangles
are essentially ﬁnite dimensional.
F of course includes much more than just rectangles
because we allow countable inﬁnite set operations. If we
ﬁx ﬁnite dimensions, these set operations lead to ﬁner and
ﬁner measurable sets (a la Carathéodory extension
theorem). We also have another countable inﬁnity: the
dimensions, which is not a problem for ﬁnite-dim space. A
typical example is an inﬁnite-dim rectangle: A =
¸

n=1
A
i
,
A
i
∈ F
i
.
Heuristically speaking, P (A) =
¸

n=1
P(A
i
) or equivalently,
log(P (A)) =
¸

n=1
log(P(A
i
)). This series converges
because P is a probability measure so log(P(A
i
)) 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Measure Extension
This deﬁnition is good for rectangles because rectangles
are essentially ﬁnite dimensional.
F of course includes much more than just rectangles
because we allow countable inﬁnite set operations. If we
ﬁx ﬁnite dimensions, these set operations lead to ﬁner and
ﬁner measurable sets (a la Carathéodory extension
theorem). We also have another countable inﬁnity: the
dimensions, which is not a problem for ﬁnite-dim space. A
typical example is an inﬁnite-dim rectangle: A =
¸

n=1
A
i
,
A
i
∈ F
i
.
Heuristically speaking, P (A) =
¸

n=1
P(A
i
) or equivalently,
log(P (A)) =
¸

n=1
log(P(A
i
)). This series converges
because P is a probability measure so log(P(A
i
)) 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable inﬁnite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we deﬁne the probability for intervals of each member
space, there is one and only one way to deﬁne a countable
inﬁnite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable inﬁnite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we deﬁne the probability for intervals of each member
space, there is one and only one way to deﬁne a countable
inﬁnite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable inﬁnite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we deﬁne the probability for intervals of each member
space, there is one and only one way to deﬁne a countable
inﬁnite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem
For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable inﬁnite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we deﬁne the probability for intervals of each member
space, there is one and only one way to deﬁne a countable
inﬁnite product measure.
By the way, X
1
, X
2
, . . . is called a stochastic process with
index set N. And yes, i .i .d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Implications
What we can describe (measure) in the inﬁnite product
space that can not be described by ﬁnite product? Limits.
We can measure for example these sets:
A =

ω : lim
n→∞
1
n
n
¸
i =1
X
i
(ω) = EX
¸
,
B =

ω : lim
n→∞
1

n
n
¸
i =1
(Y
i
(ω) −EY) ∈ (−1.96σ, 1.96σ)
¸
.
Qiu, Lee BST 401
Implications
What we can describe (measure) in the inﬁnite product
space that can not be described by ﬁnite product? Limits.
We can measure for example these sets:
A =

ω : lim
n→∞
1
n
n
¸
i =1
X
i
(ω) = EX
¸
,
B =

ω : lim
n→∞
1

n
n
¸
i =1
(Y
i
(ω) −EY) ∈ (−1.96σ, 1.96σ)
¸
.
Qiu, Lee BST 401
Homework
Page 470, Exercise 6.3.
Pg 121, Ash’s book, # 2.
Qiu, Lee BST 401
Homework
Page 470, Exercise 6.3.
Pg 121, Ash’s book, # 2.
Qiu, Lee BST 401