Probability Theory III

B. Stat. IInd Year Semester 1

Indian Statistical Institute

Problem Set 2
Almost sure, in probability and more weak convergence
d
1. Let X1 , X2 , X3 be three independent random variables, such that X1 + X3 = X2 + X3 . Will
d
then X1 = X2 hold?
2. Let an , a be positive and bn , b be real.
If Xn ⇒ X, an → a and bn → b, then (Xn − bn )/an ⇒ (X − b)/a.
If Xn ⇒ X and an → ∞, then Xn /an ⇒ 0, where the limiting random variable is the one
degenerate at 0.
If Xn ⇒ X and bn is unbounded, then Xn − bn cannot converge weakly.
If Xn ⇒ X and (Xn − bn )/an ⇒ Y , where X and Y are nondegenerate random variables,
then
0 < inf an < sup an < ∞ and sup |bn | < ∞.
If X and (X − b)/a have same distribution and they are nondegenerate, then a = 1 and
b = 0.
Convergence of types theorem: Suppose un and an are positive and vn and bn are
real. Further assume that (Xn − vn )/un ⇒ X and (Xn − bn )/an ⇒ Y , where X and Y are
nondegenerate. Then there exists positive a and real b, such that
an bn − vn X −b d
→ a, →b and = Y.
un un a
3. Suppose {Yn } are random variables such that there exists an > 0, bn ∈ R and nondegenerate
distribution functions G and Gt such that P[Yn ≤ an x + bn ] → G(x) and for all t > 0,
P[Y[nt] ≤ an x + bn ] → Gt (x). Show that there exists α(t) > 0 and β(t) ∈ R such that
G(x) = Gt (α(t)x + β(t)). Further show that α(t) = tθ , for some θ ∈ R and β(t) = c log t
when θ = 0 and β(t) = c(1 − tθ ) when θ 6= 0, for some c ∈ R.
4. A distribution function F is called extremal if it is nondegenerate and if, for some distribution
function G and constants an > 0 and bn , we have Gn (an x + bn ) ⇒ F (x). What does it mean
in terms of the corresponding random variables?
Let F be an extremal distribution function. Using convergence of types theorem, show that
there exists ck > 0 and dk such that
F k (x) = F (ck x + dk ), c1 = 1, d1 = 0, cjk = cj ck and djk = cj dk + dj = ck dj + dk .
The following three cases may occur:
(a) ck = 1 for all k. In that case, for any positive rational q = j/k, define dq = dj − dk
(why is this well-defined?) and for t > 0, define φt = inf 0<q<t,q∈Q dq . Show that φ is
decreasing and F t (x) = F (x + φ(t)) for all real x and positive t and φ(st) = φ(s) + φ(t).
Hence conclude that φ(t) = −β log t, for some β > 0. Further conclude that F (x) =
exp(e−x/β log F (0)) and thus up to a suitable centering and scaling F is of the type
F (x) = exp(−e−x ), x ∈ R.
1
2

This is called Gumbel distribution. Note that 0 < F (x) < 1 for all real x.
(b) ck0 6= 1 for some k0 . Then show that there exists x0 such that ck0 x0 + dk0 = x0 and thus
F (x0 ) is either 0 or 1. Assume F (x0 ) = 0 for this case and x0 = sup{x : F (x) = 0}. By
centering F suitably, we may take x0 = 0. Note that this will change the dk ’s. We shall,
without loss of generality assume, x0 = 0 and denote the changed dk ’s by dk again. If
dk 6= 0 for some k, show that there exists x, such that x and ck x + dk have opposite
signs and show that this leads to a contradiction. Thus all dk = 0.
For any positive rational q = j/k, define cq = cj /ck (why is this well-defined?) and for
t > 0, define γ(t) = inf 0<q<t,q∈Q cq . Show that γ is decreasing and F t (x) = F (xγ(t)) for
all positive x and t and γ(st) = γ(s)γ(t). Hence conclude that γ(t) = t−1/α , for some
α > 0. Further conclude that F (x) = exp(x−α log F (1)) for x > 0 and thus up to a
suitable scaling F is of the type
(
0, x ≤ 0,
F (x) = −α
.
exp(−x ), x ≥ 0
This is called Frechet distribution.
(c) ck0 6= 1 for some k0 and for x0 as in the case (b), F (x0 ) = 1. Argue as before, to conclude
that, for some α > 0, F is of the type
(
exp(−(−x)α ), x ≤ 0
F (x) = .
1, x≥0
Note that this is the distribution function of −X, where X has Weibull distribution and
is generally also called Weibull distribution or negative Weibull distribution.
For each case, considering G = F , show that there exist an > 0 and bn , such that F is
an extremal distribution.
Thus, we have proved the famous theorem due to Gnedenko and Kolmogorov (Fisher
and Tippett actually proved them more than 10 years earlier) that Gumbel, Frechet
and negative Weibull are extremal and, up to types, they are the only ones.
5. Let Xn be random variables taking integer values and Xn ⇒ X. Show that X also takes
P and P[Xn = j] → P[X = j] for all integers j. Show that the convergence
only integer values
is L1 , that is, j | P[Xn = j] − P[X = j]| → 0.
1/n
6. Let {Un } be i.i.d. Uniform(0, 1) random variables. Find centering and scaling for nj=1 Uj ,
Q
so that it converges to a nondegenerate weak limit. Identify the limit.
√
7. For a Poisson random variable Yλ with parameter λ, show that (Yλ − λ)/ λ ⇒ N (0, 1).
8. Consider the densities fn (x) = 1 − cos(2πnx) on [0, 1]. Check that the corresponding distri-
bution functions converge weakly to uniform distribution, but the densities do not converge.
9. Show that a collection of random variables {Xα } with their p-th moment bounded is tight.
10. Let {Xm } be i.i.d. uniformly chosen from {1, 2, . . . , m}. Let νm be the first instance of
repetition, that is,
νm = inf{n > 1 : Xn ∈ {X1 , . . . , Xn−1 }}.
Verify that
n  
Y i−1
P[νm > n] = 1− .
m
i=2
 3
√
Show that νm / m ⇒ ν where P[ν > x] = exp(−x2 /2).
11. Suppose {Xn } are i.i.d. nonnegative random variables with common density f satisfying
limt↓0 f (t) > 0. Show that n ∧ni=1 Xi has a weak limit.
12. Show that, for each n, φn (t) = sin(nt)/(nt) is a characteristic function. Find out the cor-
responding distribution function Fn . Where does φn converge pointwise? Does Fn converge
weakly? Where does Lévy-Cramer continuity theorem break down?
13. Let Xn be normal random variables with mean µn and variance σn2 and X be normal with
mean µ and variance σ 2 , such that Xn ⇒ X. Show that µn → µ and σn → σ.
Let {Xn } be i.i.d. Cauchy random
14. P P∞ variables and {an } be a real sequence. Show that
n
k=1 ak Xk converges weakly iff 1 |ak | < ∞.
15. RLet φn and φ be integrable characteristic functions with densities fn and f respectively. If
∞
−∞ |φn (t) − φ(t)|dt → 0, show that fn → f uniformly. All the densities are bounded and
continuous.
The L1 -convergence of the characteristic functions is necessary. Let f be an even density
with strictly positive characteristic function φ. Check that, for all a ∈ R,
1 − cos(ax)
ga (x) = g(x)
1 − φ(a)
is a density function with characteristic function
2φ(t) − φ(t + a) − φ(t − a)
φa (t) = .
2(1 − φ(t))
Show that, as a → ∞, φa (t) → φ(t) for all real t, but ga ’s do not converge to g pointwise.
Check that φa above is a characteristic function, whenever φ is.
16. If Xn and Yn are independent random variables for each n and Xn ⇒ X, Yn ⇒ Y , show
that Xn + Yn ⇒ X + Y , where X and Y are also independent copies. Does the result hold
without independence?
17. Let {φn } be a sequence of characteristic functions, converging pointwise to a function φ,
so that the convergence is uniform in an open interval around origin. Show that φ is a
characteristic function.

18. Let {φn } be a sequence of characteristic functions, such that φn (t) → 1 for all t ∈ (−δ, δ) for
some δ > 0. Show that φn (t) → 1 for all t.
19. Express both sides as characteristic functions and prove probabilistically that
∞
sin t Y t
= cos n .
t 2
n=1

20. Let X and Y √

be i.i.d. zero mean, unit variance random variables which have same distribution
as (X + Y )/ 2. Using CLT show that the common distribution is standard normal.
21. Lindeberg’s proof of CLT: Let {Xn } be a collection of i.i.d. random variables with
zero mean and unit variance, while {Zn } be an i.i.d. collection of standard normal random
variables, two collections being independent of each other. Lindeberg showed it is enough
√
that, for any real t, the difference between the characteristic functions of N
P
n=1 Xn / n and
4
PN √
n=1 Zn / n, both evaluated at t, converge to 0, as N → ∞. And he bounded the difference
as a telescopic sum, replacing Xn ’s by Zn ’s in the sum one at a time as follows:

N  k N k−1 N
" !# " !#
X it X X it X X 
E exp √ Gn + Xn − E exp √ Gn + Xn  .
 
 n n=1 n n=1 
k=1 n=k+1 n=k
P
k−1 PN  √
Define Y = √tn n=1 Gn + n=k+1 Xn and sn = t/ n. Show that
 " k−1 N
!#   
 it X X 1 2 
E exp √ Gn + Xn − 1 − sn E eiY  ≤ 2 E min(s2n X12 , |sn |3 |X1 |3 )
 
 n 2 
n=1 n=k

and
 
k N
" !#  
 it X X 1 2 
E exp √ Gn + Xn − 1 − sn E eiY  ≤ 2 E[|sn |3 |G1 |3 ].
 
 n 2 
n=1 n=k+1
Show the required convergence.
Can you adapt the above argument where exponential function is replaced by a bounded
C 3 function f , whose first three derivatives are bounded as well? Then one can prove and use
the following analogue of Portmanteau theorem: Xn converges weakly to X iff E[f (Xn )] →
E[f (X)] for any bounded C 3 function f , whose first three derivatives are bounded as well.