Professional Documents
Culture Documents
Random Number Generation: Nuts and Bolts of Simulation
Random Number Generation: Nuts and Bolts of Simulation
Radu Trı̂mbiţaş
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 1 / 45
Introduction
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 2 / 45
Properties of Random Numbers
1 1
E (X ) = , V (X ) =
2 12
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 3 / 45
Properties
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 4 / 45
Generation of Pseudo-Random Numbers
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 5 / 45
Generation of Pseudo-Random Numbers II
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 6 / 45
Techniques for Generating Random Numbers
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 7 / 45
Linear Congruential Method
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 8 / 45
Example
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 9 / 45
Characteristics of a Good Generator
Maximum Density
Such that the values assumed by Ri , i = 1, 2, . . ., leave no large gaps
on [0, 1]
Problem: Instead of continuous, each Ri is discrete
Solution: a very large integer for modulus m
Approximation appears to be of little consequence
Maximum Period
To achieve maximum density and avoid cycling.
Achieved by: proper choice of a, c, m, and X0 .
Most digital computers use a binary representation of numbers
Speed and efficiency are aided by a modulus, m, to be (or close to) a
power of 2.
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 10 / 45
Maximum period I
Corollary
Xi +1 = (aXi + c ) mod 2n (c, n > 1) has the full period if c is odd and
a = 4k + 1 for some k.
Theorem
Xi +1 = aXi mod 2n had period at most 2n−2 . This can be achieved when
X0 is odd and a = 8k + 3 or a = 8k + 5 for some k.
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 11 / 45
Maximum period II
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 12 / 45
Combined Linear Congruential Generators
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 13 / 45
Combined Linear Congruential Generators II
with
Xi
m, Xi > 0
Ri = m1 −1
m , Xi = 0
the maximum period is
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 14 / 45
Example
For 32-bit computers, L’Ecuyer [1988] suggests combining k = 2
generators with m1 = 2, 147, 483, 563, a1 = 40, 014,
m2 = 2, 147, 483, 399 and a2 = 20, 692. The algorithm becomes:
Step 1 X1,0 in the range [1; 2, 147, 483, 562] for the 1st
generator, X2,0 in the range[1; 2, 147, 483, 398] for the
2nd generator. Set j = 0
Step 2 For each individual generator
X1,j +1 = 40, 014X1,j mod 2, 147, 483, 563
X2,j +1 = 40, 692X1,j mod 2, 147, 483, 399
Step 3 Xj +1 = (X1,j +1 − X2,j +1 ) mod 2, 147, 483, 562.
Step 4 Return
Xj +1
Xj +1 > 0
Rj + 1 = 2, 147, 483, 563
2, 147, 483, 562
Xj +1 = 0
2, 147, 483, 563
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 15 / 45
Feedback Shift Register Generators
Tausworthe, 1965
Define a sequence of binary digits by
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 16 / 45
Feedback Shift Register Generators II
W1 = b1 b2 · · · b`
Wi = b(i −1)`+1 b(i −1)`+2 · · · bi ` , i = 2, 3, . . .
The recurrence is
Wi = Wi − r ⊕ Wi − q
finally
Wi
Ui =
2`
if (`, 2q − 1) = 1 the period is 2q − 1
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 17 / 45
Feedback Shift Register Generators III
The name comes from a switching circuit linear feedback shift register
LFSR
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 18 / 45
Feedback Shift Register Generators IV
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 19 / 45
Feedback Shift Register Generators IV I
Yi = Yi −r ⊕ AYi −q
Ui = TYi
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 20 / 45
Random-Numbers Streams
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 21 / 45
Tests for Random Numbers
Two categories:
Testing for uniformity:
H0 : Ri ∼ U [0, 1]
H1 : Ri U [0, 1]
Failure to reject the null hypothesis, H0 , means that evidence of
non-uniformity has not been detected.
Testing for independence:
H0 : Ri independent
H1 : Ri ¬independent
Failure to reject the null hypothesis, H0 , means that evidence of
dependence has not been detected.
Level of significance α, the probability of rejecting H0 when it is true:
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 23 / 45
Frequency Tests
Test of uniformity
Two different methods:
Kolmogorov-Smirnov test
Chi-square test
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 24 / 45
Kolmogorov-Smirnov test I
Dn+ = sup {F n (x ) − F0 (x )}
x ∈R
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 25 / 45
Kolmogorov-Smirnov test II
Kolmogorov’s theorem ⇒
√ +∞
∑
2x 2
lim P ( nDn ≤ x |H0 ) = K (x ) = (−1)k e −2k , x > 0.
n→∞
k =−∞
Also,
√ √ 2
lim P ( nDn+ ≤ x ) = lim ( nDn− ≤ x ) = K ± (x ) = 1 − e −2x , x >0
n→∞ n→∞
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 27 / 45
Kolmogorov-Smirnov test IV
k −1
dn− = {F0 (xk ) − F n (xk − 0)} = max F0 (xk ) −
k =1,n n
For grouped data we can employ the test using class limits.
In our case F0 (x ) = x for x ∈ [0, 1].
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 28 / 45
Example
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 29 / 45
Chi-square test I
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 32 / 45
Tests for Autocorrelation
H0 : ρim = 0
H1 : ρim 6= 0
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 33 / 45
Tests for Autocorrelation
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 34 / 45
Example
Test whether the 3rd, 8th, 13th, and so on, for the following output
0.12 0.01 0.23 0.28 0.89 0.64 0.28 0.83 0.75 0.93
0.99 0.15 0.33 0.35 0.91 0.60 0.27 0.75 0.83 0.88
0.68 0.49 0.05 0.43 0.95 0.19 0.36 0.69 0.69 0.87
ρb35 = −0.1945
σ = 0.128
b
Z0 = −1.516 < z0.025 = 1.96
fail to reject
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 35 / 45
Runs Tests for Independence I
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 36 / 45
Runs Tests for Independence II
.41, .68, .89, .84, .74, .91, .55, .71, .36, .30, .09...
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 37 / 45
Runs Tests for Independence III
Let A denote the total number of runs “up and down” out of n
observations. (A = 6 in the above example.)
Amazing Fact: If n is large (at least 20) and the Rj ’s are actually
independent, then
2n − 1 16n − 29
A∼N ,
3 90
So if n = 100, we would expect around 67 runs!
We’ll reject the null hypothesis if A is too big or small. The
standardized test statistic is
A − E (A)
Z0 = p
V (A)
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 39 / 45
Theoretical tests I
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 40 / 45
Theoretical tests II
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 41 / 45
2D lattice structure for LCG 2D lattice structure for LCG
with m = 101 and a = 18 with m = 101 and a = 18
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 42 / 45
Figure: 3D lattice structure for LCG RANDU with m = 231 and a = 65539
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 43 / 45
Shortcomings
The test is not very sensitive for small values of M, particularly when
the numbers being tests are on the low side.
Problem when “fishing” for autocorrelation by performing numerous
tests:
If α = 0.05, there is a probability of 0.05 of rejecting a true hypothesis.
If 10 independent sequences are examined,
The probability of finding no significant autocorrelation, by chance
alone, is 0.9510 = 0.60.
Hence, the probability of detecting significant autocorrelation when it
does not exist = 40%
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 44 / 45
References
Radu Trı̂mbiţaş (Faculty of Math. and CS) Random Number Generation 1st Semester 2010-2011 45 / 45