7

Independence

7.1 Independent events and random variables

Although a probability space is nothing more than a measure space with the
measure of the whole space equal to one, probability theory is not merely
a subset of measure theory. A distinguishing and fundamental feature of
probability theory is the notion of independence.
Definition 7.1.1: Let (Ω, F, P ) be a probability space and
{B1 , B2 , . . . , Bn } ⊂ F be a finite collection of events.
(i) B1 , B2 , . . . , Bn are called independent w.r.t. P , if

k  +
k
P Bij = P (Bij ) (1.1)
j=1 j=1

for all {i1 , i2 , . . . , ik } ⊂ {1, 2, . . . , n}, 1 ≤ k ≤ n.

(ii) B1 , B2 , . . . , Bn are called pairwise independent w.r.t. P if P (Bi ∩
Bj ) = P (Bi )P (Bj ) for all i, j, i = j.
Note that a collection B1 , B2 , . . . , Bn of events may be independent with
respect to one probability measure P but not with respect to another mea-
sure P  . Note also that pairwise independence does not imply independence
(Problem 7.1).
Definition 7.1.2: Let (Ω, F, P ) be a probability space. A collection of
events {Bα , α ∈ A} ⊂ F is called independent w.r.t. P if for every finite
220 7. Independence

subcollection {α1 , α2 , . . . , αk } ⊂ A, 1 ≤ k < ∞,

 k  + k
P Bαi = P (Bαi ). (1.2)
i=1 i=1

Definition 7.1.3: Let (Ω, F, P ) be a probability space. Let A be a

nonempty set. For each α in A, let Gα ⊂ F be a collection of events.
Then the family {Gα : α ∈ A} is called independent w.r.t P if for every
choice of Bα in Gα for α in A, the collection of events {Bα : α ∈ A} is
independent w.r.t. P as in Definition 7.1.2.
Definition 7.1.4: Let (Ω, F, P ) be a probability space and let {Xα : α ∈
A} be a collection of random variables on (Ω, F, P ). Then the collection
{Xα : α ∈ A} is called independent w.r.t. P if the family of σ-algebras
{σXα  : α ∈ A} is independent w.r.t. P , where σXα  is the σ-algebra
generated by Xα , i.e.,
σXα  ≡ {Xα−1 (B) : B ∈ B(R)}. (1.3)

Note that the collection {Xα : α ∈ A} is independent iff for any

{α1 , α2 , . . . , αk } ⊂ A, and Bi ∈ B(R), for i = 1, 2, . . . , k, 1 ≤ k < ∞,
+
k
P (Xαi ∈ Bi , i = 1, 2, . . . , k) = P (Xαi ∈ Bi ). (1.4)
i=1

It turns out that if (1.4) holds for all Bi of the form Bi = (−∞, xi ], xi ∈ R,
then it holds for all Bi ∈ B(R), i = 1, 2, . . . , k. This follows from the
proposition below.
Proposition 7.1.1: Let (Ω, F, P ) be a probability space. Let A be a
nonempty set. Let Gα ⊂ F be a π-system for each α in A. Let {Gα : α ∈ A}
be independent w.r.t. P . Then the family of σ-algebras {σGα  : α ∈ A} is
also independent w.r.t. P .
Proof: Fix 2 ≤ k < ∞, {α1 , α2 , . . . , αk } ⊂ A, Bi ∈ Gαi , i = 1, 2, . . . , k − 1.
Let
 k−1
+  
L ≡ B : B ∈ σGαk , P (B1 ∩ · · · ∩ Bk−1 ∩ B) = P (Bi ) P (B) .
i=1
(1.5)
 7.1 Independent events and random variables 221

It is easy to verify that L is a λ-system. By hypothesis, L contains the

π-system Gαk . Hence by the π-λ theorem (cf. Theorem 1.1.2), L = σGα .
Iterating the above argument k times completes the proof. 2
Corollary 7.1.2: A collection {Xα : α ∈ A} of random variables
on a probability space (Ω, F, P ) is independent w.r.t. P iff for any
{α1 , α2 , . . . , αk } ⊂ A and any x1 , x2 , . . . , xk in R, the joint cdf Fα1 ,α2 ,...,αk
of (Xα1 , Xα2 , . . . , Xαk ) is the product of the marginal cdfs Fαi , i.e.,

Fα1 ,α2 ,...,αk (x1 , x2 , . . . , xk ) ≡ P (Xαi ≤ xi , i = 1, 2, . . . , k)

+
k +
k
= P (Xαi ≤ xi ) = Fαi (xi ). (1.6)
i=1 i=1


Proof: For the ‘if’ part let Gα ≡ {Xα−1 (−∞, x] : x ∈ R}, α ∈ A. Now
apply Proposition 7.1.1. The only if part is easy. 2
Remark 7.1.1: If the probability distribution of (Xα1 , Xα2 , . . . , Xαk ) is
absolutely continuous w.r.t. the Lebesgue measure mk on Rk , then (1.6)
and hence the independence of {Xα1 , Xα2 , . . . , Xαk } is equivalent to the
condition that
+
k
fα1 ,α2 ,...,αk (x1 , x2 , . . . , xk ) = fαi (xi ), (1.7)
i=1

a.e. (mk ), where f(α1 ,α2 ,...,αk ) is the joint density of (Xα1 , Xα2 , . . . , Xαk ),
and fαi is the marginal density of Xαi , i = 1, 2, . . . , k. See Problem 7.18.
Proposition 7.1.3: Let (Ω, F, P ) be a probability space and let
{X1 , X2 , . . . , Xk }, 2 ≤ k < ∞ be a collection of random variables on
(Ω, F, P ).
(i) Then {X1 , X2 , . . . , Xk } is independent iff

+
k +
k
E hi (Xi ) = Ehi (Xi ) (1.8)
i=1 i=1

for all bounded Borel measurable functions hi : R → R, i = 1, 2, . . . , k.

(ii) If X1 , X2 are independent and E|X1 | < ∞, E|X2 | < ∞, then

E|X1 X2 | < ∞ and EX1 X2 = EX1 EX2 . (1.9)

Proof:
(i) If (1.8) holds, then taking hi = IBi with Bi ∈ B(R), i =
1, 2, . . . , k yields the independence of {X1 , X2 , . . . , Xk }. Conversely,
222 7. Independence

if {X1 , X2 , . . . , Xk } are independent, then (1.8) holds for hi = IBi

for Bi ∈ B(R), i = 1, 2, . . . , k, and hence for simple functions
{h1 , h2 , . . . , hk }. Now (1.8) follows from the BCT.
(ii) Note that by the change of variable formula (Proposition 6.2.1)

E|X1 X2 | = |x1 x2 |dPX1 ,X2 (x1 , x2 ),
R2

E|Xi | = |xi |dPXi (xi ), i = 1, 2,
R

where PX1 ,X2 is the joint distribution of (X1 , X2 ) and PXi is the
marginal distribution of Xi , i = 1, 2. Also, by the independence of
X1 and X2 , PX1 ,X2 is equal to the product measure PX1 ×PX2 . Hence,
by Tonelli’s theorem,

E|X1 X2 | = |x1 x2 |dPX1 ,X2 (x1 , x2 )
R2

= |x1 x2 |dPX1 (x1 )dPX2 (x2 )
R2
   
= |x1 |dPX1 (x1 ) |x2 |dPX2 (x2 )
R R

= E|X1 |E|X2 | < ∞.

Now using Fubini’s theorem, one gets (1.9). 2

Remark 7.1.2: Note that the converse to (ii) above need not hold. That is,
if X1 and X2 are two random variables such that E|X1 | < ∞, E|X2 | < ∞,
E|X1 X2 | < ∞, and EX1 X2 = EX1 EX2 , then X1 and X2 need not be
independent.

7.2 Borel-Cantelli lemmas, tail σ-algebras, and

Kolmogorov’s zero-one law
In this section some basic results on classes of independent events are estab-
lished. These will play an important role in proving laws of large numbers
in Chapter 8.
Definition 7.2.1: Let (Ω, F) be a measurable space and {An }n≥1 be a
sequence of sets in F. Then
∞

 
lim sup An ≡ lim An ≡ An (2.1)
n→∞
k=1 n≥k
7.2 Borel-Cantelli lemmas, tail σ-algebras, and Kolmogorov’s zero-one law 223

lim inf An ≡ lim An ≡ An . (2.2)

n→∞
k=1 n≥k

Proposition 7.2.1: Both lim An and lim An ∈ F and

lim An = {ω : ω ∈ An for infinitely many n}
lim An = {ω : ω ∈ An for all but a finite number of n}.


Proof: Since {An }n≥1 ⊂ F and F is a σ-algebra, Bk = An ∈ F for
∞ n≥k
each k ∈ N and hence lim An ≡ k=1 Bk ∈ F. Next,
ω ∈ lim An
⇐⇒ ω ∈ Bk for all k = 1, 2, ...
⇐⇒ for each k, there exists nk ≥ k such that ω ∈ A nk
⇐⇒ ω ∈ An for infinitely many n.
The proof for lim An is similar. 2
In probability theory, lim An is referred to as the event that “An happens
infinitely often (i.o.)” and lim An as the event that “all but a finitely many
An ’s happen.”
Example 7.2.1: Let Ω = R, F = B(R), and let
" 1#
0, for n odd
An = " n1 #
1 − n , 1 for n even.

Then lim An = {0, 1}, lim An = ∅.

The following result on the probabilities of lim An and lim An is very
useful in probability theory.
Theorem 7.2.2: Let (Ω, F, P ) be a probability space and {An }n≥1 be a
sequence of events in F. Then
∞
(a) (The first Borel-Cantelli lemma). If P (An ) < ∞, then
n=1
P (lim An ) = 0.

∞
(b) (The second Borel-Cantelli lemma). If P (An ) = ∞ and {An }n≥1
n=1
are pairwise independent, then P (lim An ) = 1.

Remark 7.2.1: This result is also called a zero-one law as it asserts that
∞pairwise independent events {An }n≥1 , P (lim An ) = 0 or 1 according to
for
n=1 P (An ) < ∞ or equal to ∞.
224 7. Independence

Proof:
n ∞
(a) Let Zn ≡ j=1 IAj . Then Zn ↑ Z ≡ j=1 IAj and by the MCT,
n ∞
EZn ≡ j=1 P (Aj ) ↑ EZ. Thus, j=1 P (Aj ) < ∞ ⇒ EZ < ∞ ⇒
Z < ∞ w.p. 1 ⇒ P (Z = ∞) = 0. But the event lim An = {Z = ∞}
and so (a) follows.
(b) Without loss of generality, assume P (Aj ) > 0 for some j. Let Jn =
EZn for n ≥ j where Zn is as above. Then, EJn = 1 and by the
Zn

pairwise independence of {An }n≥1 , the variance of Jn is

n
P (Aj )(1 − P (Aj ))
j=1 1
Var(Jn ) = ≤ .
(EZn )2 (EZn )
∞ n
If j=1 P (Aj ) = ∞, then EZn = j=1 P (Aj ) ↑ ∞, by the MCT.
Thus EJn ≡ 1, Var(Jn ) → 0 as n → ∞. By Chebychev’s inequality,
for all  > 0,
Var(Jn )
P (|Jn − 1| > ) ≤ → 0 as n → ∞.
2
Thus, Jn → 1 in probability and hence there exists a subsequence
{nk }k≥1 such that Jnk → 1 w.p. 1 (cf. Theorem 2.5.2). Since EZnk ↑
∞, this implies that Znk → ∞ w.p. 1. But {Zn }n≥1 is nondecreasing
in n and hence Zn ↑ ∞ w.p. 1. Now since lim An = {Z = ∞}, it
follows that P (lim An ) = P (Z = ∞) = 1. 2

Proposition 7.2.3: Let {Xn }n≥1 , be a sequence of random variables on

some probability space (Ω, F, P ).
∞
(a) If n=1 P (|Xn | > ) < ∞ for each  > 0, then
P ( lim Xn = 0) = 1.
n→∞

(b) If {Xn }n≥ are pairwise independent and P ( lim Xn = 0) = 1, then

∞ n→∞
n=1 P (|X n | > ) < ∞ for each  > 0.

Proof:
∞
(a) Fix  > 0. Let An = {|Xn | > }, n ≥ 1. Then n=1 P (An ) < ∞ ⇒
P (lim An ) = 0, by the first Borel-Cantelli lemma (Theorem 7.2.2 (a)).
But
(lim An )c = {ω : there exists n(ω) < ∞ such that for all
n ≥ n(ω), w ∈ An }
= {ω : there exists n(ω) < ∞ such that |Xn (ω)| ≤ 
for all n ≥ n(ω)}
= B , say.
7.2 Borel-Cantelli lemmas, tail σ-algebras, and Kolmogorov’s zero-one law 225
∞ ∞
Thus, n=1 P (An ) < ∞ ⇒ P (B ) = 1. Let B = r=1 B r1 . Now note
that
  ∞
ω : lim |Xn (ω)| = 0 = B r1 .
n→∞
r=1
∞
Since P (B ) ≤
c
r=1
c
P (B 1 ) = 0, P (B) = 1.
r

∞
(b) Let {Xn }n≥1 be pairwise independent and n=1 P (|Xn | > 0 ) = ∞
for some 0 > 0. Let An = {|Xn | > 0 }. Since {Xn }n≥1 are pairwise
independent, so are {An }n≥1 . By the second Borel-Cantelli lemma

P (lim An ) = 1.

But ω ∈ lim An ⇒ lim supn→∞ |Xn | ≥ 0 and hence

P (limn→∞ |Xn | = 0) = 0. This contradicts the hypothesis that
P (lim supn→∞ |Xn | = 0) = 1. 2

Definition 7.2.2: The tail σ-algebra of a sequence of random variables

{Xn }n≥1 on a probability space (Ω, F, P ) is
∞

T = σ{Xj : j ≥ n}
n=1

and any A ∈ T is called a tail event. Further, any T -measurable random

variable is called a tail random variable (w.r.t. {Xn }n≥1 ).
Tail events are determined by the behavior of the sequence {Xn }n≥1 for
large n and they remain unchanged if any finite subcollection of the Xn ’s
are dropped or replaced by another finite set of random variables. Events
such as {lim supn→∞ Xn < x} or {limn→∞ Xn = x}, x ∈ R, belong to T .
A remarkable result of Kolmogorov is that for any sequence of independent
random variables, any tail event has probability zero or one.
Theorem 7.2.4: (Kolmogorov’s 0-1 law ). Let {Xn }n≥1 be a sequence of
independent random variables on a probability space (Ω, F, P ) and let T be
the tail σ-algebra of {Xn }n≥1 . Then P (A) = 0 or 1 for all A ∈ T .
Remark 7.2.2: Note that in Proposition 7.2.3, the event A ≡
{limn→∞ Xn = 0} belongs to T , and hence, by the above theorem,
P (A) = 0 or 1. Thus, proving that P (A) = 1 is equivalent to proving
P (A) = 0. Kolmogorov’s 0-1 law only restricts the possible values of tail
events like A to 0 or 1, while the Borel-Cantelli lemmas (Theorem 7.2.2)
provide a tool for ascertaining whether the value is either 0 or 1. On the
other hand, note that Theorem 7.2.2 requires only pairwise independence
of {An }n≥1 but Kolmogorov’s 0-1 law requires the full independence of the
sequence {Xn }n≥1 .
226 7. Independence

Proof: For n ≥ 1, define the σ-algebras Fn and Tn by Fn =

σ{X1 , . . . , Xn } and Tn = σ{Xn+1 , Xn+2 , . . .}. Since Xn , n ≥ 1 are
independent,
∞ Fn is independent of Tn for all n ≥ 1. Since, for each n,
T = m=n Fm is a sub σ-algebra ∞of Tn , this implies Fn is independent of
T for all n ≥ 1 and hence A ≡ n=1 Fn is independent of T . It is easy to
check that A is an algebra (and hence, is a π-system). Hence, by Propo-
sition 7.1.1, σA is independent of T . Since T is also a sub-σ-algebra of
σA = σ{Xn : n ≥ 1}, this implies T is independent of itself. Hence for
any B ∈ T ,
P (B ∩ B) = P (B) · P (B),
which implies P (B) = 0 or 1. 2
Definition 7.2.3: Let (Ω, F, P ) be a probability space and let X : Ω → R̄
be a F, B(R̄)-measurable mapping. (Recall the definition of B(R̄) from
(2.1.4)). Then X is called an extended real-valued random variable or an
R̄-valued random variable.
Corollary 7.2.5: Let T be the tail σ-algebra of a sequence of indepen-
dent random variables {Xn }n≥1 on (Ω, F, P ) and let X be a T , B(R̄)-
measurable R̄-valued random variable from Ω to R̄. Then, there exists c ∈ R̄
such that
P (X = c) = 1.

Proof: If P (X ≤ x) = 0 for all x ∈ R, then P (X = +∞) = 1. Hence,

suppose that B ≡ {x ∈ R : P (X ≤ x) = 0} = ∅. Since {X ≤ x} ∈ T for all
x ∈ R, P (X ≤ x) = 1 for all x ∈ B. Define c = inf{x : x ∈ B}. Check that
P (X = c) = 1. 2
An immediate implication of Corollary 7.2.5 is that for any sequence
of independent random variables {Xn }n≥1 , the R̄-valued random variables
lim supn→∞ Xn and lim inf n→∞ Xn are degenerate, i.e., they are constants
w.p. 1.
Example 7.2.2: Let {Xn }n≥1 be a sequence of independent random
2
variables on (Ω, F, P ) with EXn = 0, √n =−11 for all2 n ≥ 1. Let
 x EX
Sn = X1 + . . . + Xn , n ≥ 1 and Φ(x) = −∞ ( 2π) exp(−y /2)dy, x ∈ R.
√
If P (Sn ≤ nx) → Φ(x) for all x ∈ R, then

Sn
lim sup √ = +∞ a.s. (2.3)
n→∞ n

√
To show this, let S = lim supn→∞ Sn / n. First it will be shown that
S is T , B(R̄)-measurable.
√ For any m ≥ 1, define √ the variables Tm,n =
(Xm+1 + . . . + Xn )/ n and Sm,n = (X1 + . . . + Xm )/ n, n > m. Note that
for any fixed m ≥ 1, Tm,n is σXm+1 , . . .-measurable and Sm,n (ω) → 0 as
 7.3 Problems 227

n → ∞ for all ω ∈ Ω. Hence, for any m ≥ 1,

S = lim sup (Sm,n + Tm,n )

n→∞

= lim sup Tm,n

n→∞

is σXm+1 , Xm+2 , . . .-measurable. Thus, S is measurable with respect to

∞
T = m=1 σXm+1 , Xm+2 , . . .. Hence, by Theorem 7.2.4, P (S = +∞) ∈
{0, 1}.
If possible, now suppose that P (S = +∞) = 0. Then, by Corollary√7.2.5,
there exists c ∈ [−∞, ∞) such that P (S = c) = 1. Let An = {Sn > nx},
n ≥ 1, with x = c + 1. Then,

0 < 1 − Φ(x) = lim P (An )

n→∞


≤ lim P Am
n→∞
m≥n

∞

∞ 
= P Am
n=1 m=n
S 
n
= P √ > x i.o.
n
≤ P (S ≥ c + 1) = 0.

This shows that P (S = +∞) must be 1. Also see Problem 7.16.

Remark 7.2.3: It will be shown in Chapter 11 that if {Xi }i≥1 are inde-
pendent and identically distributed (iid) random variables with EX1 = 0
and EX12 = 1, then
S 
n
P √ ≤ x → Φ(x) for all x in R.
n
(This is known as the central limit theorem.) Indeed, a stronger result
known as the law of the iterated logarithm holds, which says that for such
{Xi }i≥1 ,
Sn
lim sup √ = +1, w.p. 1.
n→∞ 2n log log n

7.3 Problems
7.1 Give an example of three events A1 , A2 , A3 on some probability space
such that they are pairwise independent but not independent.
(Hint: Consider iid random variables X1 , X2 , X3 with P (X1 = 1) =
228 7. Independence

1
2 = P (X1 = 0) and the events A1 = {X1 = X2 }, A2 = {X1 = X3 },
A3 = {X3 = X1 }.)
7.2 Let {Xα : α ∈ A} be a collection of independent random variables on
some probability space (Ω, F, P ). For any subset B ⊂ A, let XB ≡
{Xα : α ∈ B}.
(a) Let B be a nonempty proper subset of A. Show that the collec-
tions XB and XB c are independent, i.e., the σ-algebras σXB 
and σXB c  are independent w.r.t. P .
(b) Let {Bγ : γ ∈ Γ} be a partition of A by nonempty proper subsets
Bγ . Show that the family of σ-algebras {σXBγ  : γ ∈ Γ} are
independent w.r.t. P .
7.3 Let X1 , X2 be iid standard exponential random variables, i.e.,

P (X1 ∈ A) = e−x dx, A ∈ B(R).
A∩(0,∞)

Let Y1 = min(X1 , X2 ) and Y2 = max(X1 , X2 ) − Y1 . Show that Y1 and

Y2 are independent. Generalize this to the case of three iid standard
exponential random variables.
7.4 Let Ω = (0, 1), F = B((0, 1)), the Borel σ-algebra on (0,1) and P
be the Lebesgue measure on (0, 1). For each ω ∈ (0, 1), let ω =
∞ Xi (ω)
i=1 2i be the nonterminating binary expansion of ω.
(a) Show that {Xi }i≥1 are iid Bernouilli ( 12 ) random variables, i.e.,
is P (X1 = 0) = 12 = P (X1 = 1).
(Hint: Let si ∈ {0, 1}, i = 1, 2, . . . , k, k ∈ N. Show that the set
{ω : 0 < ω < 1, Xi (ω) = si , 1 ≤ i ≤ k} is an interval of length
2−k .)
(b) Show that
∞
X2i−1
Y1 = (3.1)
i=1
2i
∞
X2i
Y2 = (3.2)
i=1
2i

are independent Uniform (0,1) random variables.

(c) Using the fact that the set N × N of lattice points (m, n) is in
one to one correspondence with N itself, construct a sequence
{Yi }i≥1 of iid Uniform (0,1) random variables such that for each
j, Yj is a function of {Xi }i≥1 .
 7.3 Problems 229

(d) For any cdf F , show that the random variable X(ω) ≡ F −1 (ω)
has cdf F , where

F −1 (u) = inf{x : F (x) ≥ u} for 0 < u < 1. (3.3)

(e) Let {Fi }i≥1 be a sequence of cdfs on R. Using (c), construct a

sequence {Zi }i≥1 of independent random variables on (Ω, F, P )
such that Zi has cdf Fi , i ≥ 1.
∞
(f) Show that the cdf of the random variable W ≡ i=1 2X 3i is the
i

Cantor function (cf. Section 4.5).

(g) Let p > 1 be a positive integer. For each ω ∈ (0, 1) let ω ≡
∞ Vi (ω)
i=1 pi be the nonterminating p-nary expansion of ω. Show
that {Vi }i≥1 are iid and determine the distribution of V1 .
7.5 Let {Xi }i≥1 be a Markov chain with state space S = {0, 1} and
transition probability matrix

q 0 p0
Q= where pi = 1 − qi , 0 < qi < 1, i = 0, 1 .
p1 q 1

Let τ1 = min{j : Xj = 0} and τk+1 = min{j : j > τk , Xj = 0},

k = 1, 2, . . .. Note that τk is the time of kth visit to the state 0.
(a) Show that {τk+1 − τk : k ≥ 1} are iid random variables and
independent of τ1 .
(b) Show that

Pi (τ1 < ∞) = 1 and hence Pi (τk < ∞) = 1

for all k ≥ 2, i = 0, 1 where Pi denotes the probability distribu-

tion with X1 = i w.p. 1.
∞
(Hint: Show that k=1 P (τ1 > k | X1 = i) < ∞ for i = 0, 1
and use the Borel-Cantelli lemma.)
(c) Show also that Ei (eθ0 τ1 ) < ∞ for some θ0 > 0, i = 0, 1, where
Ei denotes the expectation under Pi .
7.6 Let X1 and X2 be independent random variables.
(a) Show that for any p > 0,
p
E|X1 + X2 |p < ∞ iff E|X1 | < ∞, E|X2 |p < ∞.

Show that this is false if X1 and X2 are not independent.

(Hint: Use Fubini’s theorem to conclude that E|X1 + X2 |p < ∞
implies that E|X1 + x2 |p < ∞ for some x2 and hence E|X1 |p <
∞.)
230 7. Independence

(b) Show that if E(X12 + X22 ) < ∞, then

Var(X1 + X2 ) = Var(X1 ) + Var(X2 ). (3.4)

Show by an example that (3.4) need not imply the independence

of X1 and X2 . Show also that if X1 and X2 take only two values
each and (3.4) holds, then X1 and X2 are independent.
7.7 Let X1 and X2 be two random variables on a probability space
(Ω, F, P ).
(a) Show that, if


P X1 ∈ (a1 , b1 ), X2 ∈ (a2 , b2 )



= P X1 ∈ (a1 , b1 ) P X2 ∈ (a2 , b2 ) (3.5)

for all a1 , b1 , a2 , b2 in a dense set D in R, then X1 and X2 are

independent.
(Hint: Show that (3.5) implies that the joint cdf of (X1 , X2 ) is
the product of the marginal cdfs of X1 and X2 and use Corollary
7.1.2.)
(b) Let fi : R → R, i = 1, 2 be two one-one functions such that
both fi and fi−1 are Borel measurable, i = 1, 2. Show that X1
and X2 are independent iff f1 (X1 ) and f2 (X2 ) are independent.
Conclude that X1 and X2 are independent iff eX1 and eX2 are
independent.
7.8 (a) Let X1 and X2 be two independent bounded random variables.
Show that

E(p1 (X1 )p2 (X2 )) = (Ep1 (X1 ))(Ep2 (X2 )) (3.6)

where p1 (·) and p2 (·) are polynomials.

(b) Show that if X1 and X2 are bounded random variables and (3.6)
holds for all polynomials p1 (·) and p2 (·), then X1 and X2 are in-
dependent.
(Hint: Use the facts that (i) continuous functions on a bounded
closed interval [a, b] can be approximated uniformly by polyno-
mials, and (ii) for any interval (c, d) ⊂ [a, b], any random variable
X and  > 0, there exists a continuous function f on [a, b] such
that E|f (X) − I(c,d) (X)| < , provided P (X = c or d) = 0.)
7.9 Let {Xn }n≥1 be a sequence of iid random variables on a probability
space (Ω, F, P
).∞Let R = R(ω) be the radius of convergence of the
power series n=1 Xn rn .
 7.3 Problems 231

(a) Show that R is a tail random variable.

(Hint: Note that

1
R= .)
lim sup |Xn |1/n
n→∞

(b) Show that if E(log |X1 |)+ = ∞, then R = 0 w.p. 1. and if

E(log |X1 |)+ < ∞, then R ≥ 1 w.p. 1.
(Hint: Apply the Borel-Cantelli lemmas to An = {|Xn | > λn }
for each λ > 1.)

7.10 Let {An }n≥1 be a sequence of events in (Ω, F, P ) such that

∞

P (An ∩ Acn+1 ) < ∞
n=1

and limn→∞ P (An ) = 0. Show that

P (lim sup An ) = 0.
n→∞

Show also that limn→∞ P (An ) = 0 can be replaced by

limn→∞ P ( j≥n Aj ) = 0.

(Hint: Let Bn = An ∩ Acn+1 , n ≥ 1, B = lim Bn , A = lim An . Show

that A ∩ B c ⊂ lim An .)

For any nonnegative random variable X, show that E|X| < ∞ iff

7.11 
∞
n=1 P (|X| > n) < ∞ for every  > 0.

7.12 Let {Xi }i≥1 be a sequence of pairwise independent and identically

distribution random variables.

(a) Show that limn→∞ Xn

n = 0 w.p. 1 iff E|X1 | < ∞.

∞
(Hint: E|X1 | < ∞ ⇐⇒ P (|Xn | > n) < ∞ for all  > 0.)
n=1

(b) Show that E(log |X1 |)+ < ∞ iff

 1/n
|Xn | →1 w.p. 1.

7.13 Let {Xi }i≥1 be a sequence of identically distributed random variables

and let Mn = max{|Xj | : 1 ≤ j ≤ n}.
232 7. Independence

(a) If E|X1 |α < ∞ for some α ∈ (0, ∞), then show that

Mn
→ 0 w.p. 1. (3.7)
n1/α

(Hint: Fix  > 0. Let An = {|Xn | > n1/α }. Apply the first
Borel-Cantelli lemma.)
(b) Show that if {Xi }i≥1 are iid satisfying (3.7) for some α > 0,
then E|X1 |α < ∞.
(Hint: Apply the second Borel-Cantelli lemma.)

7.14 Let X1 and X2 be independent random variables with distributions

µ1 and µ2 . Let Y = (X1 + X2 ).

(a) Show that the distribution µ of Y is the convolution µ1 ∗ µ2 as

defined by

(µ1 ∗ µ2 )(A) = µ1 (A − x)µ2 (dx)
R

(cf. Problem 5.12).

(b) Show that if X1 has a continuous distribution then so does Y .
(c) Show that if X1 has an absolutely continuous distribution then
so does Y and that the density function of Y is given by
 dµ  
(x) ≡ fY (x) = fX1 (x − u)µ2 (du)
dm

1
where fX1 (x) ≡ dµdm (x), the probability density of X1 .
(d) Y has a discrete distribution iff both X1 and X2 are discrete.

7.15 (AR(1) series). Let {Xn }n≥0 be a sequence of random variables such
that for some ρ ∈ R,

Xn+1 = ρXn + n+1 , n ≥ 0

where {n }n≥1 are independent and independent of X0 .

(a) Show that if |ρ| < 1 and E(log |1 |)+ < ∞, then

n
X̂n ≡ ρj j+1 converges w.p. 1
j=0

to a limit X̂∞ , say.

 7.3 Problems 233

(b) Show that under the hypothesis of (a), for any bounded contin-
uous function h : R → R and for any distribution of X0

Eh(Xn ) → Eh(X̂∞ ).

(Hint: Show that for each n ≥ 1, Xn − ρn X0 and X̂n have the

same distribution.)

7.16 Establish the following generalization of Example 7.2.2. Let {Xn }n≥1
be a sequence of independent random variables on some probability
space (Ω, F, P ). Suppose there exists sequences {an }n≥1 , {xn }n≥1 ,
such that an ↑ ∞, xn ↑ ∞ and for each k < ∞, limn P (Sn ≤ an xk ) ≡
F (xk ) exists and is < 1. Show that lim supn→∞ Sann = +∞ a.s.

7.17 (a) Let {Xi }ni=1 be random variables on a probability space

(Ω, F, P ) and let P (X1 , X2 , . . . , Xn )−1 (·) be dominated by
the product measure µ × µ × · · · × µ where µ is a σ-
finite measure on (R, B(R)) with Radon-Nikodym derivative
that {Xi }ni=1 are independent w.r.t. P
f (x1 , x2 , . . . , xn ). Show
n
iff f (x1 , x2 , . . . , xn ) ≡ i=1 hi (xi ) for all (x1 , x2 , . . . , xn ) ∈ R
where for each i, hi : R → R is Borel measurable.
(b) Use (a) to show that if (X1 , X2 ) has an absolutely continuous
distribution with density f (x1 , x2 ) then X1 and X2 are indepen-
dent iff
f (x1 , x2 ) = f1 (x1 )f2 (x2 )
where fi (·) is the density of Xi .
(c) Using (a) or otherwise conclude that if Xi , i = 1, 2 are both
discrete random variables then X1 and X2 are independent iff

P (X1 = a, X2 = b) = P (X1 = a)P (X2 = b)

for all a and b.

7.18 Let {Xn }n≥1 be a sequence of independent random variables such

that for n ≥ 1, P (Xn = 1) = n1 = 1 − P (Xn = 0). Show that
Xn −→p 0 but not w.p. 1.

7.19 Let (Ω, F, P ) be a probability space.

(a) Suppose there exists events A1 , A2 , . . . , Ak that are independent

with 0 < P (Ai ) < 1, i = 1, 2, . . . , k. Show that |Ω| ≥ 2k where
for any set A, |A| is the number of elements in A.
(b) Let {Xi }ki=1 be independent random variables such that Xi takes
n distinct values with positive probability. Show that |Ω| ≥
i k
j=1 ni .
234 7. Independence

(c) Show that there exists a probability space (Ω, F, P ) such that
|Ω| = 2k and k independent events A1 , A2 , . . . , Ak in F such
that 0 < P (Ai ) < 1, i = 1, 2, . . . , k.
7.20 (a) Let Ω ≡ {(x1 , x2 ) : x1 , x2 ∈ R, x21 + x22 ≤ 1} be the unit disc in
R2 . Let F ≡ B(Ω), the Borel σ-algebra in Ω and P = normalized
Lebesgue measure, i.e., P (A) ≡ m(A) π , A ∈ F. For ω = (x1 , x2 )
let
X1 (ω) = x1 , X2 (ω) = x2 ,


and R(ω), θ(ω) be the polar representation of ω. Show that
the random variables R and θ are independent but X1 and X2
are not.
(b) Formulate and establish an extension of the above to the unit
sphere in R3 .
7.21 Let X1 , X2 , X3 be iid random variables such that P (X1 = x) = 0 for
all x ∈ R.
(a) Show that for any permutation σ of (1,2,3)
  1
P Xσ(1) > Xσ(2) > Xσ(3) = .
3!
(b) Show that for any i = 1, 2, 3
  1
P Xi = max Xj = .
1≤j≤3 3

(c) State and prove a generalization of (a) and (b) to random vari-
ables {Xi : 1 ≤ i ≤ n} such that the joint distribution of
(X1 , X2 , . . . , Xn ) is the same as that of (Xσ(1) , Xσ(2) , . . . , Xσ(n) )
for any permutation σ of {1, 2, . . . , n} and P (X1 = x) = 0 for
all x ∈ R.
7.22 Let f , g : R → R be monotone nondecreasing. Show that for any
random variable X

Ef (X)g(X) ≥ Ef (X)Eg(X)

provided all the expectations exist.

(Hint:

Let Y be independent

of X with same distribution. Note that
Z = f (X) − f (Y ) g(X) − g(Y ) ≥ 0 w.p. 1.)
7.23 Let X1 , X2 , . . . , Xn be random variables on some probability space
(Ω, F, P ). Show that if P (X1 , X2 , . . . , Xn )−1 (·)  mn , the Lebesgue
measure on Rn then for each i, P Xi−1 (·)  m. Give an ex-
ample to show that the converse is not true. Show also that if
 7.3 Problems 235

P (X1 , X2 , . . . , Xn )−1 (·)  mn then {X1 , X2 , . . . , Xn } are indepen-

n
dent iff f(X1 ,X2 ,...,Xn ) (x1 , x2 , . . . , xn ) = i=1 fXi (xi ) where the f ’s
are the respective pdfs.