You are on page 1of 42

Random Number Generation

Dr. Akram Ibrahim Aly
© Copyright Akram I. Aly 2012
Lecture (8)
Random Numbers
What is a random number?
– A single number is not random. Only an infinite
sequence can be described as random.
– Random means the absence of order.
– All subsequences are equally distributed.

Applications of Random Numbers
• Random numbers have a lot of real world
applications; e.g.:
– sampling
– Simulation: generate event times and random
variables in simulation
– statistical inference
– random search optimization
– lotteries, gambling

Properties of random numbers
• A sequence of random numbers R1, R2, …, Rn
must have 2 important statistical properties:
– Uniformity
– Independence
Uniformity
• Each random number is an independent sample
drawn from a continuous uniform distribution
between 0 and 1: R
i
[0,1]
¹
´
¦
s s
=
otherwise , 0
1 0 , 1
) (
x
x f
2
1
2
) (
1
0
1
0
2
= = =
}
x
xdx R E
12
1
4
1
3
1
2
1
3
) (
2
1
0
1
0
3
2
= ÷ =
|
.
|

\
|
÷ = =
}
x
dx x R V
x
f
(
x
)

0
1
PDF:
Independence
• Independence: the probability of observing a
value in a particular interval is independent of
the previous value drawn.
Generation of Pseudo-Random
Numbers
• Pseudo means “fake”
• “Pseudo”, because we utilize known methods to
generate numbers which removes the potential for
true randomness.
• Pseudo-random numbers: Deterministic sequence of
numbers with a repeat period but with the appearance
of randomness (if you don’t know the algorithm).

Generation of Random Numbers
• Goal: To produce a sequence of numbers in [0,1]
that simulates, or imitates, the ideal properties of
random numbers (RN).
• Important considerations in RN routines:
– Fast
– Portable to different computers
– Have sufficiently long cycle
– Replicable
– Closely approximate the ideal statistical properties of
uniformity and independence.

Techniques for Generating Random
Numbers
• Mid-square method
• Linear Congruential Method
• Combined Linear Congruential Method
• Tausworthe generators (self study)
• Extended Fibonacci generators (self study)
Mid-Square Method
• Proposed by Von Neumann and Metropolis in
the 1940’s
1. Start with an initial d-digit integer seed X
0
2. Square X
0
3. Take the middle d-digits as the next four-digit
number, X
1
4. Place a decimal point at the left of X
1
to get first
random number R
1
5. Square X
1
and continue process

SEED
• Random number seed-initial random number
used to generate the next random number
which is in turn transformed into the new seed
value.
Example
• Let X
0
= 5497
– X1= (5497)
2
= 30217009 →X
1
= 2170 → R
1
= 0.2170
– X2= (2170)
2
= 04708900 →X
2
= 7089 → R
2
= 0.7089
– X3= (7089)
2
= 50253921 →X
3
= 2539 → R
3
= 0.2539
– ...and so on...

Example 2: Poor Mid-Square
Generator
• Let X
0
= 5197
– X1= (5197)
2
= 27008809 →X1= 0088→R1= 0.0088
– X2= (0088)
2
= 00007744→X2= 0077→R2= 0.0077
– X3= (0077)
2
= 00005929→X3= 0059→R3= 0.0059
– X4= (0059)
2
= 00003481→X4= 0034→R4= 0.0034
• Once zeros appear, they are carried in subsequent
numbers.
Example 3: Poor Mid-Square
Generator
• Let X
0
= 6500
– X
1
= (6500)
2
= 42250000→X1= 2500→R1= 0.2500
– X2= (2500)
2
= 06250000→X2= 2500→R2= 0.25000
• Cannot choose a seed that guarantees that the
sequence will not degenerate and will have a long
period.
Advantages and disadvantages of
the MID-SQUARE method
• Advantages of the mid-square method
– Rather simple to implement
• Disadvantages of the mid-square method
– Difficult to choose initial seed that will give good
sequence
– Strong tendency to degenerate fairly rapidly to
zero

More Examples on the mid-square
Method
17
The Mid-product Method
– Improved Variation of Mid-square method
– Example: Start with two arbitrary 3-digit numbers
Z
0
= 123 Z
1
= 456
Z
0
Z
1
= (123)(456) = 56088 U
1
= 0.608 Z
2
= 608
Z
1
Z
2
= (456)(608) = 277248 U
2
= 0.772 Z
3
= 772
Z
2
Z
3
= (608)(772) = 469376 U
3
= 0.693 Z
4
= 693
… … …
Linear Congruential Method
by D. H. Lehmer, 1951
• To produce a sequence of integers, X
1
, X
2
, …between 0 and
(m-1) by following a recursive relationship:




• If c≠0, the form is called the mixed congruential
method.
• If c=0, the form is known as the multiplicative
congruential method

Linear Congruential Method
• The random integers are being generated in the
interval [0, m-1], and to convert the integers to
random numbers:
– For [0,1[


– For [0,1]

,... 2 , 1 = = i
m
X
R
i
i
,... 2 , 1
1
=
÷
= i
m
X
R
i
i
Example
• Use X0= 27, a = 17, c = 43, and m = 100.
• The Xi and Ri values are:
– X1= (17*27+43) mod 100 = 502 mod 100 = 2,R1= 0.02;
– X2= (17*2+32) mod 100 = 77, R2= 0.77;
– X3= (17*77+32) mod 100 = 52, R3= 0.52;
– …
21
Example

• For example,


• Starting with x
0
=5:


• The first 32 numbers obtained by the above procedure
10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5;
10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5.
• By dividing x's by 16:
0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,
0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125;
0.6250, 0.1875, 0.0000, 0.0625, 0.3750, 0.9375, 0.7500, 0.8125, 0.1250,
0.6875, 0.5000, 0.5625, 0.8750, 0.4375, 0.2500, 0.3125.
Selection of LCG Parameters
• The selection of the values for a, c, m, and X
0

affects the statistical properties and the cycle
length.
• Most natural choice for m is one that equals to
the capacity of a computer word.
– m = 2
b
(binary machine), where b is the number of
bits in the computer word.
– m = 10
d
(decimal machine), where d is the number
of digits in the computer word.
23
Selection of LCG Parameters
• The modulus m should be large
– All generated numbers, X
i
, are between 0 and m-1,
the period can never be more than m
• For mod m computation to be efficient, m
must be a power of 2 ¬ Mod m can be
obtained by dropping the leftmost binary
digits in X
i+1
and using only the b rightmost
binary digits
24
Example
• Use a 4-bit computer
– Let m = 2
4
= 16, a = 5, c = 3, Z
0
= 7
Use Z
i
= (5 Z
i-1
+ 3) (mod 16)
Z
1
= (5×7+3) (mod 16) = 38 (mod 16) = 6
Z
2
= (5×6+3) (mod 16) = 33 (mod 16) = 1
Z
3
= (5×1+3) (mod 16) = 8 (mod 16) = 8
– Z
1
= 38 (mod 16) ÷ binary (38) = 1 0 0 1 10 = 6
+
Integer overflow
Z
2
= 33 (mod 16) ÷ binary (33) = 1 0 0 0 0 1 = 1
Z
3
= 8 (mod 16) ÷ binary (8) = 0 0 1 0 0 0 = 8
– No need to perform division!

Example
• Using the multiplicative congruential method, find
the period of the generator for a = 13, m = 2
6
, and X
0

= 1, 2, 3, and 4. The solution is given in next slide.
• When the seed is 1 and 3, the sequence has period 16.
However, a period of length eight is achieved when
the seed is 2 and a period of length four occurs when
the seed is 4.
Random Numbers 26
Solution
Period Determination Using Various seeds

i X
i
X
i
X
i
X
i



0 1 2 3 4

1 13 26 39 52

2 41 18 59 36

3 21 42 63 20

4 17 34 51 4

5 29 58 23

6 57 50 43

7 37 10 47

8 33 2 35

9 45 7
10 9 27

11 53 31

12 49 19

13 61 55

14 25 11

15 5 15

16 1 3
27
Selection of LCG Parameters for
Maximum Period
• If c ≠0, the maximum possible period m
is obtained if and only if:
– Integers m and c are relatively prime, that is, have no
common factors other than 1
– Every prime number that is a factor of m is also a
factor of (a-1)
– If integer m is a multiple of 4, a-1 should be a
multiple of 4
• Notice that all of these conditions are met if m=2
b
,
a = 4k + 1, and c is odd.
Selection of LCG Parameters for
Maximum Period
• For m a power of 2, say m = 2
b
, and c = 0, the
longest possible period is P = m / 4 = 2
b-2
, which is
achieved provided that the seed X
0
is odd and the
multiplier, a, is given by a = 3 + 8k or a = 5 + 8k, for
some k = 0, 1,...
29
Multiplicative LCG with m=2
k
• m = 2
b

• Maximum period achieved if:
– multiplier a is of the form 8k±3
– the initial seed is an odd integer
• Maximum possible period 2
b-2
• the maximum possible period = One-fourth m,
may not be too small
30
Example

• Using a seed of x
0
=1
– 5, 25, 29, 17, 21, 9, 13, 1, 5,…
– Period = 8 = 32/4
• With x
0
=2, the sequence is
– 10, 18, 26, 2, 10, …
– Here, the period is only 4

31
Example
• Multiplier not of the form 8i ± 3:


• Using a seed of x
0
=1, we get the sequence
– 7, 17, 23, 1, 7,…
– The period is only 4
Selection of LCG Parameters for
Maximum Period
• For m a prime number and c = 0, the longest possible
period is P = m - 1, which is achieved provided that
the multiplier, a, has the property that the smallest
integer k such that a
k
- 1 is divisible by m is k = m - 1
33
Multiplicative LCG with m=2
k
• Modulus m = prime number
• With a proper multiplier a, period = m-1
• Maximum possible period = m
• If and only if the multiplier a is a primitive root of the modulus m
– a is a primitive root of m if and only if a
n
mod m =1
for n = 1, 2, …, m-2
34
Example

• Starting with a seed of x
0
=1
– 1, 3, 9, 27, 19, 26, 16, 17, 20, 29, 25, 13, 8, 24, 10, 30, 28, 22, 4, 12, 5,
15, 14, 11, 2, 6, 18, 23, 7, 21, 1, …
– The period is 30
– ¬3 is a primitive root of 31
• With a multiplier of a = 5
– 1, 5, 25, 1,…
– The period is only 3 ¬5 is not a primitive root of 31


– Primitive roots of 31= 3, 11, 12, 13, 17, 21, 22, and 24
35
Full period generators
• A generator that has the maximum possible period, m,
is called a full-period generator
• Examples:
Combined Linear Congruential
Generators,
L’Ecuyer[1988]
• Reason: Longer period generator is needed
because of the increasing complexity of
stimulated systems.
• Approach: Combine two or more
multiplicative congruential generators.
Combined Linear Congruential
Generators
• Suggested form:






Combined Linear Congruential
Generators
• The maximum possible period is:

Example
• Example: For 32-bit computers, L’Ecuyer [1988]
suggested combining k = 2 generators:
– m
1
= 2,147,483,563, a
1
= 40,014
– m
2
= 2,147,483,399, a
2
= 40,692.
• The algorithm
– Step 1: Select seeds
• X
1,0
in the range [1, 2147483562] for the 1
st
generator
• X
2,0
in the range [1, 2147483398] for the 2
nd
generator.
– Step 2: For each individual generator:
• X
1,j+1
= 40,014X
1,j
mod 2,147,483,563
• X
2,j+1
= 40,692X
1,j
mod 2,147,483,399
– Step 3: X
j+1
= (X
1,j+1
- X
2,j+1
) mod 2,147,483,562.
– Step 4:Return

Example
• The algorithm (cont’d)
• Step 4:Return





• Step 5: Set j = j+1, go back to step 2
– Combined generator has period: (m1–1)(m2–1)/2 ~
2x10
18


Random-Numbers Streams
• The seed for a linear congruential random-number
generator:
– Is the integer value X0 that initializes the random-
number sequence.
– Any value in the sequence can be used to “seed”the
generator.
• A random-number stream:
– Refers to a starting seed taken from the sequence X0,
X1, …, XP.
– If the streams are b values apart, then stream i could
defined by starting seed:
Random-Numbers Streams
• A single random-number generator with k
streams can act like k distinct virtual random-
number generators
• To compare two or more alternative systems it
is advantageous to dedicate portions of the
pseudo-random number sequence to the same
purpose in each of the simulated systems.