2.

1 Limit Theorems 41

which have all mean µ = 3.5 and variance (see Problem 2.1)
VarX [x] = EX [x2 ] − EX [x]2 = (1 + 4 + 9 + 16 + 25 + 36)/6 − 3.52 ≈ 2.92.
We now study the random variable Wm := m−1 m
P
i=1 [Xi − 3.5]. Since each
of the terms in the sum has zero mean, also Wm ’s mean vanishes. Moreover,
Wm is a multiple of Zm of (2.4). Hence we have that Wm converges to a
1
normal distribution with zero mean and standard deviation 2.92m− 2 .
Consequently the average of m tosses of the dice yields a random vari-
able with mean 3.5 and it will approach a normal distribution with variance
1
m− 2 2.92. In other words, the empirical mean converges to its average at
1
rate O(m− 2 ). Figure 2.3 gives an illustration of the quality of the bounds
implied by the CLT.

One remarkable property of functions of random variables is that in many

conditions convergence properties of the random variables are bestowed upon
the functions, too. This is manifest in the following two results: a variant
of Slutsky’s theorem and the so-called delta method. The former deals with
limit behavior whereas the latter deals with an extension of the central limit
theorem.

Theorem 2.4 (Slutsky’s Theorem) Denote by Xi , Yi sequences of ran-

dom variables with Xi → X and Yi → c for c ∈ R in probability. Moreover,
denote by g(x, y) a function which is continuous for all (x, c). In this case
the random variable g(Xi , Yi ) converges in probability to g(X, c).

For a proof see e.g. [Bil68]. Theorem 2.4 is often referred to as the continuous
mapping theorem (Slutsky only proved the result for affine functions). It
means that for functions of random variables it is possible to pull the limiting
procedure into the function. Such a device is useful when trying to prove
asymptotic normality and in order to obtain characterizations of the limiting
distribution.

Theorem 2.5 (Delta Method) Assume that Xn ∈ Rd is asymptotically

n (Xn − b) → N(0, Σ) for an → 0. Moreover, assume that
normal with a−2 2
d l
g : R → R is a mapping which is continuously differentiable at b. In this
case the random variable g(Xn ) converges
a−2 >
n (g(Xn ) − g(b)) → N(0, [∇x g(b)]Σ[∇x g(b)] ). (2.5)

Proof Via a Taylor expansion we see that

a−2 > −2
n [g(Xn ) − g(b)] = [∇x g(ξn )] an (Xn − b) (2.6)