UniversityofWaterloo,200UniversityAvenueWestWaterloo,Ontario,Canada,N2L3G1
5198884567,ext.31043Fax:5197465036www.watrisq.uwaterloo.ca
UNIVERSITYOFWATERLOO
WaterlooResearchInstituteinInsurance,
SecuritiesandQuantitativeFinance(WatRISQ)
WORKINGPAPERSERIES
201007
Simulatingrandomvariablesusing
momentgeneratingfunctionsandthe
saddlepointapproximation
DonMcLeish
DepartmentofStatistics&ActuarialScience
UniversityofWaterloo
Simulating random variables using moment
generating functions and the saddlepoint
approximation
Don McLeish
Department of Statistics and Actuarial Science, University of Waterloo
May 5, 2010
Abstract
When we are given only a transform such as the Laplace transform,
moment generating function or characteristic function of a distribution, it
is rare that we can simulation eciently simulate random variables. Poss
sible approaches such as inverse transform using numerical inversion of the
transform are computationally very expensive. However, the saddlepoint
approximation is known to be exact for many distributions including the
Normal, Gamma, and inverse Gaussian and remarkably accurate for a
large number of others. We explore the ecient use of the saddlepoint
approximation for simulating distributions and provide three examples of
the accuracy of these simulations.
Keywords: simulation, saddlepoint approximation, sum of gamma ran
dom variables, stochastic volatility, Heston model, Feller process, sum of uniform
random variables
1 Introduction
There are many techniques for simulating random variables based on either the
cumulative distribution function (inverse transform) or the probability density
function (for example acceptancerejection), but there are also cases where nei
ther of these methods can be applied becasue the information on the distribution
is provided in a less tractible form. We may, for example, be given only a trans
form of the density such as the Laplace transform, moment generating function
or characteristic function. In these cases, it is rare that we can provide an ex
act simulation from the distribution. One can attempt to invert the transform
numerically to obtain the c.d.f. for example, and then invert this, using inverse
transform methods. However this is compuationally very intensive and there
fore virtually impossible when one wishes to do a large number of simulations.
An easy alternative is to use a saddlepoint approximation to the density which
is wellknown to be exact for families of distributions including the Normal, the
1
Gamma and the inverse Gaussian distribution, but, in general, is quite accurate
whenever a distribution is obtainable as a sum or a mean of independent random
variables. See BarndorNeilson and Cox (1989) for a discussion of saddlepoint
approximations and their generalisations.
It is the purpose of this note to explore the use of the saddlepoint approxi
mation for simulating distributions that are otherwise intractable or computa
tionally demanding.
2 Simulating from the Saddlepoint Approxima
tion
Suppose we wish to generate a random variable having known cumulant gen
erating function /(t) = ln1(c

). The Lugannani and Rice(1980) saddlepoint
approximation is
1(A r) ' 1 (n) +c(n)
1
n
1
n
(1)
where
n = n(t) =
p
2(t/
0
(t) /(t):q:(t)
n = n(t) = t
p
/
00
(t)
and t solves /
0
(t) = r. Let q(r) be the inverse function of /
0
so that the solution
above is t = q(r). Consider the distribution of the random variable q(A) under
the approximation (1). This implies, with t = q(r),
1(q(A) t) ' (n(t)) +c(n(t))
1
n(t)
1
n(t)
(2)
n(t) =
p
2(t/
0
(t) /(t):q:(t)
n(t) = t
p
/(t)
Suppose we generate a random variable T having cdf (n(t))+c(n(t))
1
u()
1
u()
.
Then the random variable A = /
0
(T) has cdf given by the saddlepoint approx
imation (1). The density corresponding to the cdf (2) is (see BarndorNeilson
and Cox (1989) section 4.50, page 110) to rst order
p
/
00
(t)c(n(t)) =
r
/
00
(t)
2
exp(/(t) t/
0
(t)) (3)
This is the saddlepoint approximation to the density function of the distribu
tion of the maximum likelihood estimator in an exponential family constructed
by tilting the original measure. Note that its mode : satises /
000
(:) =
2t (/
00
(:))
2
and this is similar to the probability density function we would
obtain if we assumed that the random variable n(T) had a standard normal
2
distribution. In this latter case, assuming the function n(t) is monotone, the
pdf is
c(n(t))
dn
dt
= c(n(t))
t/
00
(t)
n(t)
1
u()
1
u()
1(t
n
) l
p
/
00
(t
n
)c(n(t
n
))
This is relatively slow since it requires obtaining a root for each variable gener
ated.
b. We can use acceptance rejection. As long as we can dominate the den
sity
p
/
00
(t)c(n(t)) with a multiple of a simple probability density function, this
is a reasonably ecient option. For example if the function ln
p
/
00
(t)c(n(t))
=
t/
0
(t) +/(t) +
1
2
ln(/
00
(t)) is a concave function for t :, then (see Devroye
(1986), p.289291) there is a simple rejection algorithm for generating from this
distribution. In other particular cases, such as those below, we can seek an
alternative deominating function.
c. When it is dicult to dominate (3), we can use a MetropolisHastings
MCMC algorithm, providing a Markov process with the correct limiting dis
tribution. A choice of proposal density may be motivated by the fact that the
saddlepoint density (3) is close to the density
/
00
(t) exp(/(t) t/
0
(t))
obtained by assuming that t/
0
(t)/(t) has a standard exponential distribution
or more simply use a local linear approximation to the log density t/
0
(t)+/(t)+
1
2
ln(/
00
(t)), equivalent to a proposal density which is an exponential distribution
conditioned to a neighbourhood of the current value.
In the examples below we use choice (b) since it is the simplest, often the least
computationally expensive, and provides for pseudoindependent observations.
Example 1 Sum of Gamma random variables
Many distributions are expressible as a sum of random variables each of
which has a gamma distribution. This is the case, for example, with the
AndersonDarling Statistic (see for example Murakami (2009)). Suppose we
3
wish to simulate values of the random variables A =
P
I
1
I
where the random
variables 1
I
are independent Ga::a(c
I
, ,
I
). This may be a nite or innite
sum, but in the latter case, obviously we require rates of convergence of the
parameters, for example we may require
P
I
c
I
,
I
< and
P
I
c
I
,
2
I
< . The
cumulant generating function and its derivatives are given below.
/(t) =
X
I
c
I
ln(1 ,
I
t) for t < min(
1
,
I
)
/
0
(t) =
X
I
c
I
,
I
1 ,
I
t
/
00
(t) =
X
I
c
I
,
2
I
(1 ,
I
t)
2
/
000
(t) =
X
I
2c
I
,
3
I
(1 ,
I
t)
3
t/
0
(t) /(t) =
X
I
c
I
,
I
t
1 ,
I
t
+ ln(1 ,
I
t)

1o

) exp(
P
I
c
I
) and
Y
I
(1,
I
t)
o
t
. Therefore
q

00
()
2t
exp(/(t)t/
0
(t)) = o(t
).
As t 1,,
I
, consider the term
s
1
(1 ,
I
t)
2
exp(c
I
,
I
t
1 ,
I
t
)(1 ,
I
t)
o
= exp(c
I
,
I
t
1 ,
I
t
)(1 ,
I
t)
1o
= c
o
exp(c
I
1
1 ,
I
t
)(1 ,
I
t)
1o
0
Therefore we can always bound the saddlepoint density with a function of the
form
c min(1, (1 /t)
o
) (4)
for 1 < a <
P
I
c
I
and suitable constants / = max(,
I
), c. It is easy to generate
from a density proportional to (4), namely
b(o1)
o
min(1, (1 /t)
o
) .by inverse
transform since the corresponding cdf and its inverse are:
1(t) =
1
o
(1 /t)
o+1
, t < 0
1
o
+ (1
1
o
)/t, 0 < t <
1
b
1
1
(l) =
1
b
1 (al)
1/(o1)
, 0 < l < a
1
oI1
b(o1)
, a
1
l < 1
(5)
4
Figure 1: Empirical cdf of saddlepoint simulations vs theoretical distribution
for sum of Gamma random variables
Having found the parameters a, c in (4), the acceptance rejection algorithm
is as follows:
1. Generate l l[0, 1] and dene T = 1
1
(l) using (5).
2. Generate \ l[0, 1].
3. If \
00
()
2
exp((T)T
0
(T))
c min(1,(1bT)
)
output the value /
0
(T).Otherwise return
to step 1.
We chose as an example a sum of 20 exponential random variables with
c
I
= 1, ,
I
= (0.8)
I1
, for i = 1, 2, ..., 20. The scale parameters dier by a factor
of up to almost 70. In a simulation of 1,000,000 runs of the AR algorithm
resulted in 565,677 draws from the saddlepoint density. The empirical c.d.f. is
plotted in Figure (1), together with the empirical cdf of values simulated from
the correct distribution. Clearly these two plots are essentially identical, with
very small dierences in the remote tails (r 13) where both cdfs are greater
than 0.999. The mean and variance of the simulated random variables agreed
well with the theoretical counterparts,
P
I
c
I
,
I
and
P
I
c
I
,
2
I
.
Example 2 Simulating a Volatility Process
A Feller or CIR process is a stochastic process \

driven by the stochastic
dierential equation
5
d\

= i(0 \

)dt +o
u
p
\

d\(t), \
0
=
0
0 (6)
where \ is a Brownian motion processes. In the Heston stochastic volatility
model, \

is the volatility process and it is of particular interest to simulate
jointly the spot volatility \
T
at the end of a period and the aggregate volatility
over this period
R
T
0
\
s
d:. The marginal distribution of \
T
is wellknown (see
Glasserman) and Glasserman and Kim (2009) obtain the Laplace transform
of the conditional distribution of
R
T
0
\
s
d: given \
0
and \
T
. This is described
as an integral over a Bessel distributed random variable j so that, translated to
our notation, this results in a moment generating function
1
h
c

0
\Js
\
0
, \
T
i
= 1
c
2
+ 2j
1sinhi
isinh1
where the outer expectation on the right side is over the distribution of a Bessel
random variable j having parameters i =
o
2
1 and . =
2r/c
2
sinh(r~)
\
0
\
T
and
where
c =
4i0
o
2
1 = 1(t) =
p
i
2
2o
2
t, and 1
0
=
o
2
1
=
T
2
,
. =
0
+
T
o
2
.
With some additional notation
c = c(t) = coth1(t), and c
0
= (1 c
2
)
o
2
1
=
c
2
+ 2j,
We can write the cumulant generating function of the distribution conditional
on \
0
, \
T
, j as:
/(t) = . [icothi 1c] + [ln(1sinhi) ln(isinh1)]
Our objective is to simulate eciently from this distribution. The derivatives
are:
6
/
0
(t) = . [1
0
c 1c
0
] +
1
0
1
sinh
0
1
sinh1
1
0
=
o
2
.
1
c + (1 c
2
)1
+
o
2
1
2
1 +1
cosh1
sinh1
=
o
2
.
1
c + (1 c
2
)1
o
2
1
2
[1 1c]
/
0
(0) =
o
2
.
i
cothi + (1 coth
2
i)i
o
2
i
2
[1 i cothi]
/
00
(t) =
o
2
.
1
2
1
0
c + (1 c
2
)1
+
o
2
.
1
c
0
+ (1 c
2
)1
0
2cc
0
1
+ 2
o
2
1
3
1
0
[1 1c] +
o
2
1
2
[1
0
c +1c
0
]
/
00
(t)1
o
4
= (
. +
1
)(
c
1
(1 c
2
)) + 2c.
2
(1 c
2
) 2
1
3
We will try dominating the saddlepoint density
q

00
(T)
2t
exp(/(T) T/
0
(T))
with a multiple of a density function similar to that used for the sum of gamma
distributed random variables, having cumulative distribution function 1(t) =
1
o
(1 /t)
o+1
, t < 0
1
o
+ (1
1
o
)/t, 0 < t <
1
b
where a = 2, / =
2c
2
r
2
. We used parameters as
follows
i = 6 o = 0.6 c = 1 j = 1 T = 1 \
0
= .5, \
T
= 1
In this case, . =
u0+u
c
2
=
1.5
0.6
2
= 4.1667, =
T
2
= 0.5, =
o
2
+ 2j = 2.5
/
0
(0) =
0.6
2
4.1667
6
coth3 + (1 coth
2
3)3
0.6
2
6
2
2.5 (1 3 coth3) = 0.29414
so that the mean of a simulated sample should be around this value. We give
the saddlepoint density for T and the dominating function in Figure (2). The
constant c = 0.023 was used to dominate the saddlepoint density. We simu
lated 217,845 values from the above conditional distribution using acceptance
rejection and obtained a sample mean of .2943, which compares well with the
theoretical value above, and a sample variance of 0.0024. This simulation re
quired less than one second of CPU time on a PC running Matlab with 2 Intel
CPU at 2 GZ. The probability histogram of the sample is displayed in Figure
(??).
Example 3 Sum of : Uniform[1,1]
Consider a random variable dened as
A =
n
X
I=1
1
I
7
Figure 2: The saddlepoint density for T together with the dominating function
c min(1, (1 /t)
o
), c = 0.023
Figure 3: Histogram of generated sample of aggregate volatility
8
where each random variable 1
I
is Uniform[1,1]. The cumulant generating
function of A and its derivatives are:
/
n
(t) = :ln
sinh(t)
t
= :ln(
sinh(t)
t
)
/
0
(t) = :
d
dt
(ln(sinh(t)) ln(t)) = :
1
tanht
1
t
/
00
(t) = :
d
dt
cosht
sinht
1
t
= :
1 +
1
t
2
1
tanh
2
t
t/
0
(t) +/(t) +
1
2
ln(/
00
(t)) = : +
1
2
ln(:) :t
cosht
sinht
+:ln(
sinh(t)
t
) +
1
2
ln
1 +
1
t
2
cosh
2
t
sinh
2
t
= constant :
t
tanht
ln(
sinh(t)
t
)
1
2:
ln
1 +
1
t
2
1
tanh
2
t
1
/
2
+t
2
We chose the scale constant of the Cauchy / =
p
n
4
to provide a reasonable t.
The saddlepoint density is quite simple:
r
/
00
n
(t)
2
exp(/
n
(t)t/
0
n
(t)) =
r
:
2
c
n
r
1 +
1
t
2
1
tanh
2
t
sinh(t)
t
n
exp(
:t
tanht
)
The actual density function of A = /
0
(T) is given by (see Devroye, p. 732)
using the inversion of the characteristic function
sin()

n
)(r) =
1
Z
0
sin(t)
t
n
cos(tr)dt
=
1
(i 1)!
1
2
I1
X
=0
(1)

i
/
(r (2/ :))
I1
for 2i 2 : < r < 2i :; i = 1, 2, ..., :
The acceptancerejection algorithm is as follows:
Generate l l[0, 1]. Generate T = /, tan(l) from a Cauchy(0, /)
distribution with / =
p
n
4
.
Generate \ l[0, 1]. If
\ <
:c
n
c/
2
r
1 +
1
T
2
1
tanh
2
T
sinh(T)
T
n
exp(
:T
tanhT
)(/
2
+T
2
)
9
Figure 4: Saddlepoint density and dominating multiple of Cauchy density
"accept" T, i.e. output A = /
0
(T) where
/0(T) = :
1
tanhT
1
T
Much more than documents.
Discover everything Scribd has to offer, including books and audiobooks from major publishers.
Cancel anytime.