The Acceptance-Rejection Method for Continues Random Variables

Suppose that we have a method for simulating a random variable having p.d.f. g(x). We can
use this as the basis for simulating from the continuous distribution having p.d.f. f(x) by
simulating Y from g and then accepting this simulated value with a probability proportional to
f(Y)/g(Y).
Specifically let c be a constant such that
f ( y)
c
g ( y)
for all y.
We then have the following technique for simulating a random variable having p.d.f. f.
Algorithm:
1) Simulate Y having p.d.f. g and simulate a random variable U.
f (Y )
if U
set X Y .
cg (Y )
2) Otherwise return to step 1).
Proposition:
The random variables X generated by the Acceptance-Rejection Method has p.d.f. f.
Proof:
Let X be value obtained, and let N denote the number of necessary iterations.
Then
P(X <= x) = P(YN <= x) = P(Y <= x | U <= f(Y)/cg(Y))

P (Y x,U f (Y ) / cg (Y ))
P (U f (Y ) / cg (Y ))
=
Denote K = P(U <= f(Y)/cg(Y)).
P ( X x)

P (Y x,U f (Y ) / cg (Y ))

P(Y x,U

f (Y ) / cg (Y ) | Y y ) g ( y )dy
K

( f ( y) / cg ( y)) g ( y)dy

f ( y)dy

Kc

Since

f ( y )dy 1 and

P ( X x) 1 if

K = 1/c and the proof is complete.

Remark: since each iteration of the method will, independently, result in an accepted value
with probability P(U<=f(Y)/cg(Y)) it follows that the number of iterations is geometric with
mean c.
Example 1: Generating a random variable from
f X ( x) 3 x 2 ,

Assume

g X ( x) 1,

0 x 1.

0 x 1.

Thus

max(

f X ( x)
) 3 c.
g X ( x)

f X ( x)
) x2.
cg X ( x)
Algorithm:
Generate two uniform random variables U1 and U2 from U(0, 1).
2) If U2 <= U12 accept U1 as the random variable from f X (x) ,
otherwise go to step 1).
1)

Example 2: Generating a random variable having p.d.f.

f(x) = 20x(1 - x)3,

0< x <1

Since this random variables (which is beta with parameters 2, 4) is concentrated in the
interval (0, 1), let us consider the acceptance-rejection method with
g(x) = 1,

0< x < 1

To determine the constant c such that f(x)/g(x) <= c, we use calculus to determine the
maximum value of
f(x)/g(x) = 20x(1 - x)3
Differentiation of this quantity yields
d f ( x)
20((1 x) 3 3 x(1 x) 2 )

dx g ( x)
Setting this equal to 0 shows that the maximal value is attained when x = 1/4,
and thus
3
f ( x)
135
1 3
20
c
g ( x)
64
4 4
Hence,
f ( x) 256

x(1 x) 3
cg ( x) 27
and thus the simulation procedure is as follows:
1) Generate two random numbers U1 and U 2 .
256
U2
U 1 (1 U 1 ) 3 set X U 1 and stop.
27
2) If
Otherwise return to step 1).
The average number of times that step 1) will be performed is c = 135/64.
Example 3: Generating a random variable having p.d.f.
f X ( x)

2
R 2

g X ( x)

1
,
2R

R 2 x 2 , R x R.

Assume
R x R.

Thus

max(

f X ( x)
2 * 2R
4
) max(
R 2 x 2 ) c.
2
g X ( x)

f X ( x)
4
1

R2 x2
R2 x2 .
cg X ( x) R 4
R
Algorithm:
1) Generate two uniform random variables U1 and U2 from U(0, 1).
2)

3)

Compute Y = -R + U2(R - (-R)) = R(2U2 - 1).

If

U1

1
R 2 R 2 (2U 2 1) 2 1 (2U 2 1) 2
R

which is equivalent to (2U2 - 1)2< = 1 - U12 ,

then accept
X = R(2U2 - 1)
as the random variable from f X (x) , otherwise go to step 1).
Example 4: Simulating a Unit Normal Random Variable
that the absolute value of Z has p.d.f.

f ( x)

2
2

ex

/2

0 x

Assume
g ( x) e x ,

0 x

Note that
f ( x)
2 x x2 / 2

e
g ( x)

2
and its maximum occurs at the value of x that maximizes x x / 2 .
Thus is happened at x = 1, thus

2e

2
f ( x)
e x x / 21 / 2 e
cg ( x )

( x 1) 2
2

The algorithm:
1) Generate random variable Y ~ Exp(1) and random variable U ~ U(0, 1).
(Y 1) 2 / 2

2) If U <= e
, or equivalently , if -log(U) >= (Y - 1)2/2 set X = Y.
Otherwise return to step 1).

Remarks:
1. We have simulated a random variable X distributed as the absolute value of a
unit normal we can obtain a unit normal Z by letting Z be equally likely to
be either X or -X.
2
2. If we want the normal random variable to have mean and variance ,
just take Z .
3. Since c 2e / ~ 1.32 , the foregoing requires a geometric distributed
number of iterations of Step 2 with mean 1.32.
4. If we want to generate a sequence of standard normal random variables, we
can use the exponential random variable Y obtained in Step 3 as the initial
exponential needed in Step 1 for the next normal to be generated.
Example 5: The Polar Method for generating Normal Random Variables.
Let X and Y independent unit normal random variables and let R and denote the polar
coordinates of the vector (X, Y). That is
R2 X 2 Y 2
Y
tan
X
Since X and Y are independent, their joint density is the product of their individual densities
and is thus given by

f ( x, y )

1
2

ex

/2

1
2

ey

/2

1 ( x 2 y 2 ) / 2
e
2

2
To determine the joint density of R and - call it f (d , ) - we make the change of
variables
y
y
d x 2 y 2 tan 1 ( ) arctg ( )
x
x

The Jacobian of this transformation is

dd
dx
J 1
d
dx
Thus
J

dd
2x
2y
2x
1
y
1
1
dy
y
( 2 )

d
y 2
y 2 x 2
x
1 ( )
1 ( )
x y2
dy
x
x
1
2

2
The joint density function of R and is given by

2y
2x 2
2y2
x
2

2
x y2 x2 y2
2
2
x y

1 1 d / 2
e
, 0 d , 0 2
2 2
However, as this is equal to the product of an exponential density having mean 2 (namely,
1 d / 2
e
2
2
) and the uniform density on (0, 2 ) (namely, 1/2 ), it follows that R and are
2
independent, with R being exponential with mean 2 and being uniformly distributed
over (0, 2 ).
f (d , )

We can now generate a pair of independent unit normal random variables X and Y by
generating their polar coordinates and then transforming back to rectangular coordinates. This
is accomplished as follow:
1) Generate random numbers U 1 and U 2 .
2
2
2) Set R 2 ln U 1 (and thus R is exponential with mean 2)
and 2U 2 (and thus is uniform between 0 and 2 ).
3) Let

X R cos 2 ln U 1 cos(2U 2 )
Y R sin 2 ln U 1 sin(2U 2 )
These transformations are known as the Box-Muller transformations.
Unfortunately, the use of the Box-Muller transformations is computationally not very
efficient because for this is the need to compute the sine and the cosine trigonometric
functions. There is a way to get around this time-consuming difficulty by an indirect
computation of the sine and cosine of a random angle (as opposed to a direct computation
which generates U and then computes the sine and cosine of 2U ) .
Note that if U is uniform on (0, 1) then 2U is uniform on (0, 2) and so 2U 1 is uniform
on (-1, 1).
Thu,s is we generate random numbers U 1 and U 2 and set

V1 2U 1 1
V2 2U 2 1
then ( V1 , V2 ) is uniformly distributed in the square of area 4 centered at (0,0).
Suppose now that we continually generate such pairs ( V1 , V2 ) until we obtain one that is
contained in the circle of radius 1 centered at (0,0) that is until ( V1 , V2 ) is such that
2

V1 V2 1 . It now follows that such a pair ( V1 , V2 ) is uniformly distributed in the circle.

If we let R and denote the polar coordinates of this pair, it is not difficult to verify that R
2
and are independent, with R being uniformly distributed on (0, 1) and with being
uniformly distributed on (0, 2 ) (prove it by yourself). Since is thus a random angle, it

follows that we can generate the sine and cosine of a random angle by generating a
random point ( V1 , V2 ) in the circle and then setting
sin

V2
V
2 2 2 1/ 2
R (V1 V2 )

cos

V1
V
2 1 2 1/ 2
R (V1 V2 )

It now follows from the Box-Muller transformations that we can generate independent unit
normal by generating a random number U and setting
X (2 ln U )1 / 2

Y (2 ln U )1 / 2

V1
2

(V1 V2 )1 / 2
V2
2

(V1 V2 )1 / 2

In fact, since R 2 V12 V22 is itself uniformly distributed over (0,1) and is independent of a

random angle , we can use it as a random number U in the last equations. Therefore, letting
S = R 2 , we obtain that
X (2 ln U )1 / 2

V1
2 ln S
V1

1/ 2
S
S

1/ 2

1/ 2

V2
2 ln S
Y (2 ln U )
V2

1/ 2
S
S

are independent unit normals when ( V1 , V2 ) is a randomly chosen point in the circle of
1/ 2

radius 1 centered at the origin, and S V12 V22 .

We thus have the following approach to generating a pair of independent unit normals:
1)
2)
3)
4)

Generate random numbers U 1 and U 2 .

Set V1 2U 1 1 , V2 2U 2 1 , S V12 V22 .
If S > 1 return to step 1).
Return the independent unit normals
2 ln S
X V1

1/ 2

2 ln S
, Y V2

1/ 2

The above is called the polar method. Since the probability that a random point in the square
will fall within the circle is equal to / 4 , it follows that, on average, the polar method will
require 4 / = 1.273 iterations in Step 1.

The Acceptance-Rejection Method for Discrete Random Variables

Suppose we have an efficient method for simulating a random variable having p.d.f. {qj, j> =
0}. We can use this as the basis for simulating from the distribution having p.d.f. {pj, j> = 0}.
Let c be a constant such that
pj

c for all j such that p j 0

qj
We now have the following technique for simulating a random variable X having p.d.f. pj =
P(X = j).

Algorithm:
q j,

1) Simulate the value of Y , having p.d.f.

2) Generate a random number U.

pY
, set X = Y and stop,
cqY
otherwise return to step 1).

3) If U

Theorem: The random variable X obtained from the rejection method

has p.d.f. P(X = i) = pi.
Proof:

P ( X i ) P (Y i | U

K P (U
Denote

pY
)
cqY

P (Y i, U

pY
)
cqY

p
P (U Y )
cqY

P (Y i, U

P(U

pi
)
cqi

pY
)
cqY

pY
)
cqY
P (Y i ) P (U

P( X i)

pi
)
cqi

qi pi
p
i
K cqi cK

Summing over all i yields that

1 P( X i)
i

1
cK

p
i

1
cK

and so we see that the generated random variable X indeed has the desired mass function.

Remark: since each iteration of the method will, independently, result in an accepted value
with probability P(U <= pY/cqY) it follows that the number of iterations needed will have a
geometric distribution with parameter 1/c and will thus have mean c.
Example: Simulating a random variable X having p.d.f. {pj, j = 1, . . . , 10} that takes one
of the values {1, 2, ...,10} with respective probabilities{0.11, 0.12, 0.09, 0.08, 0.12, 0.10,
0.09, 0.09, 0.10, 0.10}.
Assume

qj = 0.1, j = 1, . . . , 10.

For this choice of {qj} we can choose c by

c max

pj
qj

1 .2

and so the algorithm would be as follows:

1) Generate random variables U 1 , U 2 ~ U(0, 1) and set Y = [10U1] + 1.
pY
2) If U 2
set X = Y and stop,
1 .2 * 0 .1
otherwise return to step 1).