2.

1 Limit Theorems 43

We next introduce the notion of a characteristic function of a distribution.1

Definition 2.7 (Characteristic Function) Denote by p(x) a distribution

of a random variable X ∈ Rd . Then the characteristic function φX (ω) with
ω ∈ Rd is given by
Z
d
φX (ω) := (2π) 2 F −1 [p(x)] = exp(i hω, xi)dp(x). (2.9)

In other words, φX (ω) is the inverse Fourier transform applied to the prob-
ability measure p(x). Consequently φX (ω) uniquely characterizes p(x) and
moreover, p(x) can be recovered from φX (ω) via the forward Fourier trans-
form. One of the key utilities of characteristic functions is that they allow
us to deal in easy ways with sums of random variables.

Theorem 2.8 (Sums of random variables and convolutions) Denote

by X, Y ∈ R two independent random variables. Moreover, denote by Z :=
X + Y the sum of both random variables. Then the distribution over Z sat-
isfies p(z) = p(x) ◦ p(y). Moreover, the characteristic function yields:
φZ (ω) = φX (ω)φY (ω). (2.10)
Proof Z is given by Z = X + Y . Hence, for a given Z = z we have
the freedom to choose X = x freely provided that Y = z − x. In terms of
distributions this means that the joint distribution p(z, x) is given by
p(z, x) = p(Y = z − x)p(x)
Z
and hence p(z) = p(Y = z − x)dp(x) = [p(x) ◦ p(y)](z).

The result for characteristic functions follows form the property of the
Fourier transform.
For sums of several random variables the characteristic function is the prod-
uct of the individual characteristic functions. This allows us to prove both
the weak law of large numbers and the central limit theorem (see Figure 2.4
for an illustration) by proving convergence in the Fourier domain.
Proof [Weak Law of Large Numbers] At the heart of our analysis lies
a Taylor expansion of the exponential into
exp(iwx) = 1 + i hw, xi + o(|w|)
and hence φX (ω) = 1 + iwEX [x] + o(|w|).
1 In Chapter ?? we will discuss more general descriptions of distributions of which φX is a special
case. In particular, we will replace the exponential exp(i hω, xi) by a kernel function k(x, x0 ).