Generating Random Numbers

`

Consider two binary sequences
Sequence 1 -- 010101010101 Sequence 2 -- 010001010110

` `

`

Which one is more random? Note that although sequence 2 seems to be more random, you have the same probability for obtaining these 2 sequences when you throw a coin 12 times. This example shows that a definition based purely on probability is not enough.

`

`

`

No practical definition for defining what is a true random sequence In complexity theory, a binary sequence is considered random if its complexity is not smaller than its length. For example,
for sequence 1 of previous slide, the pseudocode: Print ³01´ 6 times for sequence 2, the pseudocode: Print ³010001010110´

`

How about if the length of sequence is 10000?

`

For practical purpose, we will instead consider that a sequence is random if it is uncorrelated between its subsequences of any length.

`

` `

`

Algorithm used in producing a sequence which is virtually indistinguishable from random sequence i.e. it cannot be identified by using statistical tests Note that the sequences generated by a pseudorandom number generator is not real random sequence since the algorithm itself defines one of the correlations (Refer back to complexity definition)

`

This algorithm follows the recurrence relation:

`

`

` `

For example, take a = 4, c = 3, s0 = 7. m = 7, the first 9 digit is 3, 1, 0, 3, 1, 0, 3, 1, 0 The conditions for achieving full period are listed in the note. Length of full period = m The n-tuples are confined into planes in hyper cube.

`

Generalizing Linear Congruential Generator we have

`

In practice, we use the following equation

`

By taking

, the period can be as long as

`

lagged fibonacci44497 in Boost Random Number Library Generators used p = 44497 and q = 21034 and ³+´ as operation and provided a period of the order We can also have the ³exclusive-or´ and ³multiplication´ operation. ³exclusive-or´ performs poorly unless for a large p ³multiplication´ requires more memory in storing the multiplied values

` ` `

`

Mersenne Twister follows the following linear recurrence:

` `

` `

A specific parameter set called MT19937 produced a period of MT19937 passed all diehard tests It is a high quality pseudo-random number generator.

The importance of random seed
‡ As shown previously, all pseudo-random number generator generates a number sequence using a recurrence relation. ‡ An initial value of the number sequence, called seed, must be provided. ‡ Most program with random number generator assume a static value for the seed, i.e. if you do not set the seed value, then you get the same ³random numbers´ everytime. ‡ So, remember to set seed.

The importance of random seed
‡ Usually people use computer clock as seed for its uniqueness ‡ But when the program is run on multiple machine, they would have the same seed if the starting time is the same. ‡ Provide some delays if you use computer clock as seed. ‡ If you use another random number sequence as seed, be sure that you use the random number sequence from other pseudo-random number generator

`

Definition of k-tuple: si is the element of a number sequence n If the sequence is ³random´ enough, we would expect a uniform distribution of the k-tuples over the k-dimensional hypercube. Linear Congruential Generator, it is known that the k-tuples fall only on a small number of hyperplane Nk, of order km1/k

` `

`

Random Numbers
‡ Uniform Random Numbers
± rand('seed',n), rand(n), rand(m,n) ± Interval 0 to 1 ± Interval a to b
‡ x = (b - a)*r + a;

‡ Gaussian Random numbers
± randn('seed',n), randn(n), randn(m,n) ± Normal Distribution
‡ Mean = 0, Standard Deviation = 1.0

± Modified Distribution
‡ Mean = b, Standard Deviation = a ‡ x = a*r + b; 13

New MATLAB Function

‡ rand() - Uniformly distributed random numbers and arrays ‡ Ex of syntax:
± A = rand(n) ± A = rand(m,n)

‡ Where m and n are dimensions of the matrix

Rand() con¶t: 2

‡ Scalars may be generated
± Ex. A = rand(1,1);

‡ Vectors may be generated
± Ex. A = rand(10,1);

Rand() con¶t: 3

‡ Generated random numbers will be between 0 and 1. ‡ Scaling can be done by multiplying the resulting matrix or vector by the number you wish to scale with

Mean, Variance, Standard Deviation
n

å
‡ Mean:
± mean(x)

xi n

m=

i= 1

n

‡ Variance:
± mean((x-mean(x).*(x-mean(x)))

s2=

å
i= 1

( xi - m) 2 n

n

å
‡ Standard Deviation
± std(x)

( xi - m) 2 n

s =

i= 1

Correlation Coefficient

‡ Correlation coefficient
function r = corco(x,y) mx = mean(x); my = mean(y); covxy = mean((x-mx).*(y-my)); r = covxy/(std(x)*std(y));

r=

s XY s Xs Y

‡ Or use Matlab function
± corrcoef

Random Numbers

‡ rand(M,N):

MxN matrix of uniformly distributed random numbers on (0,1) MxN matrix of normally distributed random numbers ( =0, 2=1)

‡ randn(M,N)

‡ normrnd(m,s,M,N) MxN matrix of normally distributed random numbers ( =m, =s)
x=randn(1,10000); mean((x<=1)) ans = 0.8449

F (1) = 0.8413

Correlation Coefficient

X : N (0,1) Y ! a b V !1

x=normrnd(0,1,1,100); y=2*x+1; r=corrcoef(x.',y.') plot(x,y,'+')

N : N (0,1) Y ! a b N V ! 0.9186

n=normrnd(0,1,1,100); y=2*x+1+n; r=corrcoef(x.',y.') plot(x,y,'+')

Joint Gaussian
f X ,Y ( x, y ) !
V !0

1 2TW 1W 2 (1  V )
2

e

« ( x  1 ) 2 ( y  2 ) 2 2 V ( x  1 )( y  1    2 ¬ 2 2 W 1W 2 W2 2(1 V ) ¬ W 1 ­

2)

» ¼ ¼ ½

Joint Gaussian

V ! 0.5

Joint Gaussian

V ! 0.9

Joint Gaussian

V ! 0.9

Generating Random Numbers
Uniform
ì 1 ï f ( x) = í ï b- a ï î

a£ x£ b

ì0 ï ï ï ï x- a F (x) = ï í ï b- a ï ï ï1 ï î

x< a a£ x£ b b< x

1. Generate 2. Return

U : U (0,1)
X = a + (b - a )U

Uniform
a = 10 b = 15
function genuni(N,a,b) u=rand(1,N); x=a+(b-a).*u; minx=min(x); maxx=max(x); NumBins=51; h=hist(x,NumBins); for k=1:NumBins, bincenters(k)=minx+((maxxminx)/NumBins)*(k-1/2); end h=h/sum(h); bar(bincenters,h)

Or use Matlab function unifrnd(a,b,M,N)

Generating Random Numbers
Exponential
ì 1 - e- l x ï F (x) = ï í ï0 ï î

f ( x) = { e l

- lx

x

0

x

0

x< 0

1. Generate 2. Return

(0,1) 1 X = - ln(1l

:

)

Exponential
l = 5
function genexp(N,lambda) u=rand(1,N); x=-1/lambda.*log(1-u);

Or use Matlab function exprnd(lambda,M,N)

Generating Random Numbers
Normal
1 e 2p s
( x- m)2 2s 2 x

f ( x) =

- ¥

x

¥

(x) =

ò
- ¥

1 e 2p s

-

( x- m) 2 2s 2

- ¥

x

¥

Rayleigh
x f ( x) = 2 e s
x2 2s
2

,

x

0

ì ï ï 1- eï (x) = í ï ï0 ï î : (0,1) Z= æ 1 2s ln ç ç ç1è
2
2

x2 2s 2

x x

0 0

1. 2. 3. 4.

Generate Set Generate Return

1

ö ÷ ÷ ÷ ÷ 1ø
2

2

:

(0,1) ) X 2 = m + Z sin(2p )

X 1 = m + Z cos(2p

Normal
m = 10 s 2 = 4
function gennormal(N,mu,sigma) for i=1:N u=rand; z=sigma*(sqrt(2*log(1/(1u)))); u=rand; x1(i)=mu+z*cos(2*pi*u); x2(i)=mu+z*sin(2*pi*u); end

Or use Matlab function normrnd(mu,sigma,M,N)

Generating Random Numbers
Binomial
æö k çn p (1- p )n- k p(k ) = ç çk ø è ì ï0 ï ï k ï æö i ï ÷ çn÷ p (1- p )n- i F (k ) = í å ç ÷ ï i= 0 çi ÷ è ø ï ï ï1 ï ï î k< 0 0£ k £ n k> n

k = 0,1,..., n

1. Generate 2. Return

Y1 , Y2 ,..., Yn

IID Bernoulli(p) random numbers

X = Y1 + Y2 + ... + Yn

Binomial
n = 20
function genbino(N,n,p) for i=1:N, u=rand(1,n); y=(u<=p); x(i)=sum(y); end

p = 0.5

Or use Matlab function binornd(n,p,M,N)

Generating Random Numbers
Geometric
ì 1- (1- p ) k ï F (k ) = ï í ï0 ï î

p ( k ) = (1- p )

k- 1

p

k = 1, 2,...

k k

1 1

1. Generate 2. Return

U : U (0,1) éln(1- U ) ù ú X= ê êln(1- p ) ú ê ú

Geometric
p = 0.5
function gengeo(N,p) u=rand(1,N); x=ceil(log(1-u)/log(1-p));

Or use Matlab function geornd(p,M,N)

Generating Random Numbers
Poisson
ì0 ï ï ï F (k ) = í k l i - l ïå e ï ï i= 0 i ! ï î k k 0 0

p(k ) =

l -l e , k!

k

k = 0,1,...

1. Set k = 0, P = 1 2. Generate U k + 1 : U (0,1) and replace P by P ×U k + 1 3. If P e- l accept X = k else increase k by one and return to step 2

Poisson
l = 5
function genpois(N,lambda) for i=1:N, k=0;p=1; u=rand; p=p*u; while p>=exp(-lambda) u=rand; p=p*u; k=k+1; end x(i)=k; end

Or use Matlab function poissrnd(lambda,M,N)

Example of a MATLAB script
% % % % % % script randscript A simple script to generate a vector of n random numbers. We calculate the numbers using (a) for loops, and (b) a direct function call. Professor Smyth, Oct 2007

Comment text: it is always important to put comments in your code. Comments at the top should clearly explain what the script or function does

Example of a MATLAB script
% % % % % % script randsum A simple script to generate a vector of n random numbers. We calculate the numbers using (a) for loops, and (b) a direct function call. Professor Smyth, Oct 2007

n = 100000; % the number of points for the "for loop´ y = zeros(n,1); % preallocate memory for y fprintf('Simulating %d random numbers.....\n\n',n);

Initialize various variables

Print out some information to the screen

Example of a MATLAB script
% % % % % % script randsum A simple script to generate a vector of n random numbers. We calculate the numbers using (a) for loops, and (b) a direct function call. Professor Smyth, Oct 2007

n = 100000; % the number of points for the "for loop´ y = zeros(n,1); % preallocate memory for y fprintf('Simulating %d random numbers.....\n\n',n); % first do the calculation using a "for loop" fprintf('For loop calculations.....\n'); (1) Calculate the n random numbers tic % set the timer and their sum using a for loop, for i=1:n (2) record the time taken, y(i) = rand(1); and (3) print to the screen end total = sum(y); fprintf('Sum of %d random numbers = %f\n',n,total); % read the time elapsed since "tic" (in seconds) t1 = toc; fprintf('Time taken, using for loop = %6.5f microseconds\n\n', (t1)*1000); «...

Example of a MATLAB script
««« « % now do the calculation using vectorization fprintf('Vectorization calculations.....\n'); tic % reset the timer z = rand(n,1); total = sum(z); fprintf('Sum of %d random numbers = %f\n',n,total); % read the time elapsed since "tic" (in seconds) t2 = toc; fprintf('Time taken, using vectorization = %6.5f microseconds\n', (t2)*1000);

(1) Now calculate n random numbers and their sum using a direct function call (2) record the time taken, and (3) print to the screen

MATLAB random number generator
‡ ‡ Function rand.m Generates a sequence (of length n) of pseudorandom numbers:

± sequence generation: x(i) = mod(a * x(i-1), m) ± initialized with a state (³seed´) value for x(0)

» help rand

RAND Uniformly distributed random numbers. RAND produces pseudo-random numbers. The sequence of numbers generated is determined by the state of the generator. Since MATLAB resets the state at start-up, the sequence of numbers generated will be the same unless the state is changed. S = RAND('state') is a 35-element vector containing the current state of the uniform generator. RAND('state',S) resets the state to S. RAND('state',0) resets the generator to its initial state. RAND('state',J), for integer J, resets the generator to its J-th state. RAND('state',sum(100*clock)) resets it to a different state each time. This generator can generate all the floating point numbers in the closed interval [2^(-53), 1-2^(-53)]. Theoretically, it can generate over 2^1492 values before repeating itself.

Example of a MATLAB function
function [sum, difference] = sumdiff(a, b);

Tells MATLAB this is a function

List of output values returned (can be any form of array data type)

Name of the function

List of input argument values, comma delimited (any form of array data type)

Example of a MATLAB function
function [sum, difference] = sumdiff(a, b); % function [sum, difference] = sumdiff(a, b); % % A simple function to compute the sum and difference % of two input arguments a and b % % Professor Smyth, Oct 2007 % % INPUTS: % a: array of size r x c % b: array of size r x c % % OUTPUTS: % sum: a + b % difference: a - b

Clear comments in function headers are very useful Note the explicit statement explaining what the inputs and outputs are (including their dimensionality)

Example of a MATLAB function
function [sum, difference] = sumdiff(a, b); % function [sum, difference] = sumdiff(a, b); % % A simple function to compute the sum and difference % of two input arguments a and b % % Professor Smyth, Oct 2007 % «««««.. % error checking [rowsa, colsa] = size(a); [rowsb, colsb] = size(b); if ( rowsa ~= rowsb ) | ( colsa ~= colsb) error(µsizes of a and b do not match¶); end

Error checking is always a good idea!

Example of a MATLAB function
function [sum, difference] = sumdiff(a, b); % function [sum, difference] = sumdiff(a, b); % % A simple function to compute the sum and difference % of two input arguments a and b % % Professor Smyth, Oct 2007 % «««««.. % error checking [rowsa, colsa] = size(a); [rowsb, colsb] = size(b); if ( rowsa ~= rowsb ) | ( colsa ~= colsb) error(µsizes of a and b do not match¶); end sum = a + b; difference = a ± b;

Finally, the actual computational part of the function

MATLAB functions in general
‡ Function line definition
± required of all functions

‡ List of inputs and outputs
± comma delimited: [y, z] = average(a, b, c) ± for more than one output, outputs are enclosed in square brackets

‡ Input variables
± variables within function are local to the function ± input variables are readable to the function, but not writable ± values of returned arguments are passed back

‡ Search path
± MATLAB searches in this order:
‡ variable name, subfunction, current directory, MATLAB search path

Example of a MATLAB function
function [meanr, stdr, z] = simulate(n);

Tells MATLAB this is a function

List of output values returned (can be any form of array data type)

Name of the function

List of input argument values, comma delimited (any form of array data type)

Example of a MATLAB function
function [meanr, stdr, z] = simulate(n); % function [meanr, stdr, z] = simulate(n); % % A simple function to simulate a vector of n % uniformly distributed random numbers and return the % mean and standard deviation of the numbers. % % Professor Smyth, Oct 2007 % % INPUTS: % n: number of random numbers generated (positive integer) % % OUTPUTS: % meanr: mean of the n random numbers % stdr: standard deviation of the random numbers % z: an n x 1 array of random numbers

Example of a MATLAB function
function [meanr, stdr, z] = simulate(n); % function [meanr, stdr, z] = simulate(n); % % A simple function to simulate a vector of n % uniformly distributed random numbers and return the % mean and standard deviation of the numbers. % % Professor Smyth, Oct 2007 % % INPUTS: % n: number of random numbers generated (positive integer) % % OUTPUTS: % meanr: mean of the n random numbers % stdr: standard deviation of the random numbers % z: an n x 1 array of random numbers

% simple error checking to check n is a positive integer if (rem(n,1)~=0) | n<=0 error('Input n must be a positive integer'); end

Example of a MATLAB function
«. « fprintf('Simulating %d random numbers.....\n\n',n); % generate the n random numbers z = rand(n,1); % calculate the mean and standard deviation meanr= mean(z); fprintf('Mean of the %d random numbers = %f\n',n,meanr); stdr= std(z); fprintf('Standard deviation of the %d random numbers = %f\n',n,stdr);

Simulate the random numbers and print out the results. No need for an explicit return statement Any values not returned are known only locally to the function

Creating a MATLAB function
In-class illustration of how to create and use a MATLAB function

Calling the MATLAB function
» [m, s] = simulate(1000000); Simulating 1000000 random numbers..... Mean of the 1000000 random numbers = 0.499702 Standard deviation of the 1000000 random numbers = 0.499702 » [m, s] = simulate(1000000); Simulating 1000000 random numbers..... Mean of the 1000000 random numbers = 0.499684 Standard deviation of the 1000000 random numbers = 0.288456 »m m = 0.4997 »s s = 0.2885

Another MATLAB function
function [meanr, stdr, z] = simplot(n,plotflag); % function [meanr, stdr, z] = simplot(n,plotflag); % % A simple function to simulate a vector of n % uniformly distributed random numbers and return the % mean and standard deviation of the numbers. If % plotflag is 1 a histogram of the numbers is plotted % % Professor Smyth, Oct 2007 % % INPUTS: % n: number of random numbers generated (positive integer) % plotflag: if plotflag=1, a histogram of z is plotted, % otherwise no plotting % % OUTPUTS: % meanr: mean of the n random numbers % stdr: standard deviation of the random numbers % z: an n x 1 array of random numbers

% simple error checking to check n is a positive integer if (rem(n,1)~=0) | n<=0 error('Input n must be a positive integer'); end

Simplot.m continued
fprintf('Simulating %d random numbers.....\n\n',n); % generate the n random numbers z = rand(n,1); % calculate the mean and standard deviation meanr= mean(z); fprintf('Mean of the %d random numbers = %f\n',n,meanr); stdr= std(z); fprintf('Standard deviation of the %d random numbers = %f\n',n,stdr); if nargin>1 & plotflag==1 figure hist(z, max(n/100,10)) end

Here is the new code. Nargin is the number of input variables If we have a 2nd input and it equals 1, then we plot a histogram of the random numbers syntax: hist(data vector, number of bins)

Demonstration: plotting of sample mean as a function of n

‡ Extend simplot.m (call it simplot2.m)
± for each value of i = 1 to n, calculate
‡ mean(i) = [sum (x(i)«« x(i)) ]/I ‡ use vectorization to do this ‡ mean(i) should converge to the true mean 0.5 as i gets large
± why? ³law of large numbers´ from statistics

± illustration of the use of the online debugger ± we can plot this to visualize it ± various additional plotting features
‡ grids, log axes, labels, titles

Code that was added to simplot.m
if nargin>1 & plotflag==1 figure % figure for a histogram to see how uniform the numbers are hist(z,max(n/100,10)) figure % figure to see visually how the sample mean converges to 0.5 cs = cumsum(z); % generate a vector of cumulative sums nvalues = 1:n; % generate a vector of sample sizes runningmean = cs./nvalues; % there¶s an error here (on purpose) plot(nvaluess,runningmean); % and another error here

grid; axis([1 n 0 1]); xlabel('Number of random numbers generated'); ylabel('Mean value'); title('Convergence of sample mean to true mean'); end

Code that was added to simplot.m
if nargin>1 & plotflag==1 figure % figure for a histogram to see how uniform the numbers are hist(z,max(n/100,10)) figure % figure to see visually how the sample mean converges to 0.5 cs = cumsum(z); % generate a vector of cumulative sums nvalues = 1:n; % generate a vector of sample sizes runningmean = cs./nvalues; % there¶s an error here (on purpose) plot(nvaluess,runningmean); % and another error here %correct code: %runningmean = cs./nvalues'; %plot(nvalues, runningmean); grid; axis([1 n 0 1]); xlabel('Number of random numbers generated'); ylabel('Mean value'); title('Convergence of sample mean to true mean'); end

Pseudo-random signal generation: "rand" generates uniform PDF random numbers between 0 & 1. Subtract 0.5 from each sample for zero mean & variance = 1/12. To produce array of 100 samples of uniform PDF process with zero mean & unit variance:. x = (rand(100) - 0.5) * SQRT(12); "randn" generates Gaussian PDF random numbers with zero mean & unit variance. To generate 100 samples of such a process x = randn(100); White PSD over frequency range 0 to qifs / 2 Hz. ³1-sided´ PSD for these samples is: 1 / ( qif s/ 2) Watts/Hz. To produce 100 samples of AWGN (white between 0 & qifs/2) of ³1-sided´ PSD N0 Watts/Hz : x = randn(100) * sqrt ( N0 * qifs/2 );

Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.