Simulation: Simulation is artificial imitation of reality.

Monte Carlo simulation: This is a type of simulation which combines probability theory with
sampling technique. Why this technique is called Monte Carlo is because Monte Carlo is a
small island between Italy and France and this island is famous for its casino.

Inverse Cumulative distribution function (cdf) technique is one method for generating
random variables from any distribution given its cumulative distribution function.
Computing the cumulative distribution function(CDF) = FX(x)= P(X≤x) of the distribution
maps a number in the domain to a probability between 0 and 1 and then inverting that
function. So if we take a cumulative frequency of some distribution that we want to
sample from, the function takes as input some value x and tells what the probability
of obtaining X≤x is. So FX(x) = P(X≤x)=r. Inverse of such a function, F−1 (r) will
take input as p and return x. Also r’s are uniformly distributed in [0, 1]. The idea is
basically this: it is easy to take values uniformly from U[0,1] so if one wants to
sample from some FX, one only needs to take values u∼U(0,1) and
pass through the inverse function F−1 to obtain the x’s.

Although in this method there is no loss of random numbers but it has its own
limitations, one being this method may not generate an x accurately. Secondly not
every function has its inverse, e.g. one cannot use this method for bivariate
distributions. In summary, it may not be the most efficient method in all situations;
in many cases better algorithms exist (e.g. Box Muller transformation is more
efficient for generating normal variates).
Next we discuss two algorithms, one inaccurate but which prevents loss of random
numbers and the other accurate but leads to loss of random numbers, for the
generation of binomial variates:
[1] Algorithm to generate a B(n,p) variate using inverse cdf technique
1. Input n, p
2. Generate u~U[0,1]
3. Find the maximum “r” such that
∑𝒓𝒙=𝟎 n
Cxpx(1-p)n-x ≤ u
4.Deliver r
where u is the U[0,1] variate generated by the computer. Its value is taken as a
value of the distribution function (because the distribution function is itself a U[0,1]
variate). We take maximum “r” because for no value of “r” the left hand side will be
exactly equal to right hand side. This aspect makes this algorithm an inaccurate
method of generating binomial variates. Its only advantage is that it requires only
one random number (one u) to generate one binomial variate and hence there is
no loss of random numbers.
[2] Another algorithm which is an accurate method of generating binomial variates
is:
1. Input n, p
2. c = 0
3. for i=1 to n

˿
Generate x U(0,1)
If x<p then c = c+1
end for
4. Deliver c. (c is the B(n,p) variate)

Although this method is accurate it leads to loss of random numbers as we need n

independent uniform variates to generate one binomial variate.

How to simulate a Poisson variate:

Using inverse cdf technique,
1. Input λ
2. Generate y~U(0,1)
3. Find the maximum r such that
Ʃe-λλx/x! ≤ y summed over x=0 to r
4. Deliver r

Problem: Write an algorithm to simulate a random variable X whose probability

distribution is given below:-
x: -3 -2 -1 0 1 2 3
P(X=x): 1/17 1/17 3/17 5/17 5/17 1/17 1/17
Solution:
1. Generate y~U(0,1)
2. If 0≤y≤1/17 then X=-3
3. If 1/17<y≤2/17 then X=-2
4. If 2/17<y≤5/17 then X=-1
5. If 5/17<y≤10/17 then X=0
 6. If 10/17<y≤15/17 then X=1
7. If 15/17<y≤16/17 then X=2
8. If 16/17<y≤1 then X=3
9. Deliver X

Simulating a Bernoulli variate:

X=0 1
1-p p
1. Input p
2. Generate y~U(0,1)
3. If 0≤y≤p then X=1
4. If p<y≤1 then X=0
5. Deliver X

f(y)=1, 0<y<1
Remark: P(0<Y<p)=∫f(y)dy between the limits 0 and p=∫0p 1.dy=p

Simulating a normal distribution:

Inverse cdf: 1. Generate y~U(0,1)
2. Find z such that ∫φ(z)dz from -∞ to z =y
3. Deliver z

This technique is not very efficient.

Suppose y=0.2356
From normal probability table the value of the integral is 0.23576
when z=-0.72 and the value of the integral is 0.23270 when
z=-0.73 which means the required z for y=0.2356 lies somewhere
between -0.72 and -0.73. The answer may be found using
Lagrange’s formula for inverse interpolation.
Box-Muller Transformation:
If U and V are two independent U(0,1) variates then Z1 and Z2 defined below will
be independent standard normal variates:
Z1= {√(-2lnU)}Cos(2ꙥV)
Z2={√(-2lnU)}Sin(2ꙥV)
If so, then X1=θ+Z1σ and X2=θ+Z2σ will be independent Normal variates with mean
θ and variance σ2.
Algorithm:
1. Input θ, σ
2. Generate U~U(0,1) and V~U(0,1) independently
3. Z1= {√(-2lnU)}Cos(2ꙥV); Z2={√(-2lnU)}Sin(2ꙥV)
4. X1=θ+Z1σ; X2=θ+Z2σ
5. Deliver X1 and X2

Exponential distribution:-
A continuous random variable X is said to follow exponential distribution if its
pdf is given by f(x)=θe-θx, x>0, θ>0
E(X)=∫xxf(x)dx=∫x θe-θxdx between the limits 0 to ∞
=θ∫xe-θxdx between the limits 0 to ∞
= θ∫e-θx x2-1dx between the limits 0 to ∞
=θ x Г(2)/θ2
=1!/θ
=1/θ

E(X2)= )=∫xx2f(x)dx=∫x2θe-θxdx between the limits 0 to ∞

= θ∫x2e-θxdx between the limits 0 to ∞
= θ∫e-θx x3-1dx between the limits 0 to ∞
= θ x Г(3)/θ3
= 2!/θ2
= 2/θ2
Var(X)=E(X2)-{E(X)}2
 =2/θ2 – (1/θ)2
= 2/θ2 – 1/θ2
=1/θ2

How to simulate an exponential variate:-

cdf (cumulative distribution function or simply distribution function)
FX(x)=P(X≤x) =∫f(x)dx between the limits 0 to x
=∫θe-θxdx between the limits 0 to x
=θ∫e-θxdx between the limits 0 to x
=θ[e-θx/(-θ)]0x
=(-1)[e-θx – 1]
= 1 – e-θx
Algorithm (inverse cdf) for simulating an exponential variate with parameter θ:-
1. Input θ
2. Generate y~U(0,1)
3. 1 – e-θx = y or, x= {1/(-θ)}ln(1-y)
4. Deliver x
-------------------------------------------
Lack of memory property:-
Exponential distribution has lack of memory property
P(X≥t+h/X≥t)=P(X≥h)

Gamma Function:
The integral ∫e-x xn-1dx between the limits 0 to ∞ is convergent for positive n.
The value of the integral is a function of n called the Gamma function. It is
denoted by Г(n). Numerically it is equal to (n-1)!.
I= Г(n)=∫e-x xn-1dx between the limits 0 to ∞
=[-xn-1e-x +∫e-x (n-1)x(n-1)-1dx] between the limits 0 to ∞=(n-1) Г(n-1)
Similarly Г(n-1)=(n-2) Г(n-2). Proceeding in this manner,
Г(n)=(n-1)(n-2)…..2.1=(n-1)! (we can also prove it by mathematical induction)
Theorem: ∫e-θx xn-1dx between the limits 0 to ∞= Г(n)/θn
Proof: Let y=θx or, x=y/θ and dx = dy/θ
The integral = ∫e-y (y/θ)n-1dy/θ between the limits 0 to ∞=1/θn x ∫ e-y yn-1dy
between the limits 0 to ∞= Г(n)/θn.
It can be shown that Г(1/2)=√ꙥ.

Geometric distribution:
Suppose we have a series of independent Bernoullian trials with fixed probability of success p in each
trial. We stop when the first success is reached. Then the number of failures preceding the first success is
my Geometric variate X. Clearly the probability mass function of X is given by
P(X=x)=(1-p)xp=qxp; x=0,1,2,3…..;0<p,q<1;p+q=1
MGF=MX(t)=E(etx)=ƩxetxP(X=x) = Ʃetxqxp where the summation extends for x =0 to ∞
= p Ʃ(qet)x where the summation extends for x =0 to ∞
=p [1+ qet + (qet)2+ (qet)3+…..]
=p[1/(1-qet)]
=p(1-qet)-1
E(X)=[dMX(t)/dt]t=0
=p[{-(1-qet)-2(-qet)}]t=0
=pq[(1-qet)-2et]t=0
=pq/(1-q)2
=pq/p2
=q/p
E(X2)=[d2MX(t)/dt2]t=0
= pq[(1-qet)-2et+ (-2)(1-qet)-3(-qet)et]t=0
=pq[(1-q)-2+2q(1-q)-3]
=pq/p2+2pq2/p3
=q/p + 2q2/p2
Var(X)=E(X2)-{E(X)}2
 = q/p + 2q2/p2 – (q/p)2
= q/p + 2q2/p2 – q2/p2
= q/p + q2/p2
= q/p(1+ q/p)
= q/p{(p+q)/p}
=q/p2 (as p+q=1)
Note: Geometric distribution also has lack of memory property.
How to simulate a Geometric variate:-
Algorithm 1(inverse cdf technique)
1. Input p
2. Generate y ~ U(0,1)
3. Find the maximum r such that Ʃ(1-p)xp≤y where summation extends from x =0 to r
4. Deliver r
Algorithm 2:
1. Input p
2. X=0
3. Generate Y ~ U(0,1)
4. If 0<Y<p then GO TO step 7
5. X=X+1
6. GO TO step 3
7. Deliver X

p=0.5
X=0
Y=0.432
X=0

p=0.5
X=0
Y=0.675
X=1
Y=0.964
X=2
Y=0.231
X=2

Negative Binomial distribution:-

Suppose we are making independent Bernoullian trials with fixed probability of success p and we shall
stop only when we get r successes. Suppose r+X trials are required to get r successes, then X is a negative
Binomial variate with parameters r and p. Clearly, we must get r-1 successes in r+X-1 trials followed by
the (r+X)-th trial which must be a success. The probability mass function of X is given by
P(X=x)= {x+r-1Cr-1(1-p)xpr-1}.p= x+r-1Cr-1(1-p)xpr, x=0, 1, 2…., 0<p<1, r is a constant, r=1, 2, 3….
When r=1, we get Geometric distribution as a special case.
x+r-1
Cr-1
= (x+r-1)(x+r-2)……(r+1)r{(r-1)!}/[{(r-1)!}{(x+r-1)-(r-1)}]
=(x+r-1)(x+r-2)……(r+1)r/x!
=(-1)x(-x-r+1)(-x-r+2)……(-r-1)(-r)/x!
= (-1)x {- rCx} because rCx=r!/{x!(r-x)!}=r(r-1)(r-2)……(r-x+1)/x! (put -r in place of r)
P(X=x)= {- rCx} (-q)xpr, x=0, 1, 2…., 0<p<1, r is a constant, r=1, 2, 3….
The pmf is the (x+1)-th term in the expansion of pr(1-q)-r, whose numerical value is unity, which is a
binomial expansion with a negative index. That’s why the distribution is called negative binomial
distribution.
Let p=1/Q and q=P/Q
Since p+q=1 hence 1/Q + P/Q =1 or (1+P)/Q=1 or Q-P=1
P(X=x)= {- rCx} Q-r(-P/Q)x, x=0, 1, 2….
This is the general term in the negative binomial expansion of (Q-P)-r.
For this distribution, Mean = rP and variance = rPQ.
How to simulate a negative binomial variate:-
Algorithm 1 (inverse cdf technique)
1. Input p, r
2. Generate y ~ U(0,1)
3. Find the maximum m such that
Ʃ x+r-1Cr-1(1-p)xpr≤y where the summation extends for x=0 to m
4. Deliver m

Algorithm 2:
1. Input p, r
2. X=0; S=0
3. Generate Y ~ U(0,1)
4. If 0<Y<p then S=S+1; if S=r then GO TO step 7
5. X=X+1
6. GO TO step 3
7. If r=1 then Deliver X else Deliver X-1
p=0.5, r=2
X=0, S=0
Y=0.6512
X=1
Y=0.2314
S=1
X=2
Y=0.3146
S=2
X=1
χ2 distribution and how to simulate:-
Chi-Square variate: The square of a standard normal variate is a Chi-Square (χ2) variate with one
degree of freedom. Mathematically, suppose X ~ N(θ, σ2)
Z = (X-θ)/σ ~ N(0, 1)
Z2 ~ χ21d.f.
The pdf of a χ2 with n degrees of freedom is given by
f(χ2)= [1/{2n/2Г(n/2)}]exp{- χ2/2}( χ2)n/2 – 1, χ2>0, n>0
MGF=E[exp(tχ2)]=∫ exp(tχ2) f(χ2)dχ2 between the limits 0 to ∞
=[1/{2n/2Г(n/2)}] ∫ exp(tχ2) exp{- χ2/2}( χ2)n/2 – 1dχ2 between the limits 0 to ∞
=[1/{2n/2Г(n/2)}] ∫ exp[-(1-2t)χ2/2]( χ2)n/2 – 1dχ2 between the limits 0 to ∞
=[1/{2n/2Г(n/2)}]Г(n/2)/[(1-2t)/2]n/2
=(1-2t)-n/2, │2t│<1
E(χ2)=n
Var(χ2)=2n
How to simulate a Chi-Square variate with 1 degree of freedom:-
1. Generate U1~U(0,1)
2. Generate U2~U(0,1) independently
3. Z1={√(-2lnU1)}Cos(2ꙥU2)
4. Z2= {√(-2lnU1)}Sin(2ꙥU2)
5. Deliver either Z12 or Z22
In order to generate Chi-square variates with n degrees of freedom, we shall be using additivity
property of Chi-square variates.
Additivity property of Chi-Square variates:-
Sum of independent Chi-square variates is again a Chi-square variate with degrees of freedom
equal to the sum of the degrees of freedom of the individual Chi-square variates.
Theorem: If X and Y are independent Chi Square variates with parameters m and n respectively,
then
(i) X/(X+Y) is a beta variate of first type with parameters (m/2, n/2) and
(ii) X/Y is a beta variate of second type with parameters (m/2, n/2)
This theorem can be used to generate beta variates of first and second type. We can also
simulate F variate (ratio of two independent Chi square variates divided by their respective
degrees of freedom) and Student’s t variate easily (which has the same form as beta variate of
second type.