Modeling and Simulation

(MEMEE05/MPMEE02)

UNIT – 2
Probability Concepts in Simulation
Introduction

• Life is uncertain, we are not sure what the

future will bring
• Risk and probability is a part of our daily lives
• Probability is a numerical statement about the
likelihood that an event will occur
 Fundamental Concepts

1. The probability, P, of any event or state of

nature occurring is greater than or equal to 0
and less than or equal to 1. That is:

0  P (event)  1

2. The sum of the simple probabilities for all

possible outcomes of an activity must equal 1
 Diversey Paint Example
• Demand for white latex paint at Diversey Paint and
Supply has always been either 0, 1, 2, 3, or 4 gallons per
day
• Over the past 200 days, the owner has observed the
following frequencies of demand

QUANTITY
NUMBER OF DAYS PROBABILITY
DEMANDED
0 40 0.20 (= 40/200)
1 80 0.40 (= 80/200)
2 50 0.25 (= 50/200)
3 20 0.10 (= 20/200)
4 10 0.05 (= 10/200)
Total 200 Total 1.00 (= 200/200)
 Diversey Paint Example
• Demand for white latex paint at Diversey Paint and
Supply
Notice hasindividual
the always been either 0, 1, 2, 3, or 4 gallons per
probabilities
areday
all between 0 and 1
• Over the past 200 days, the owner has observed the
0 ≤frequencies
following P (event) of
≤1demand
And the total of all event
QUANTITY equals 1
probabilities NUMBER OF DAYS PROBABILITY
DEMANDED
0 ∑ P (event) = 1.00
40 0.20 (= 40/200)
1 80 0.40 (= 80/200)
2 50 0.25 (= 50/200)
3 20 0.10 (= 20/200)
4 10 0.05 (= 10/200)
Total 200 Total 1.00 (= 200/200)
 Types of Probability
Determining objective probability
• Relative frequency
• Typically based on historical data
Number of occurrences of the event
P (event) =
Total number of trials or outcomes

◼ Classical or logical method

◼ Logically determine probabilities without
trials
1 Number of ways of getting a head
P (head) =
2 Number of possible outcomes (head or tail)
 Types of Probability

Subjective probability is based on the

experience and judgment of the person making
the estimate
• Opinion polls
• Judgment of experts
• Delphi method
• Other methods
 Mutually Exclusive Events

Events are said to be mutually exclusive if only one

of the events can occur on any one trial

◼ Tossing a coin will result

in either a head or a tail
◼ Rolling a die will result in
only one of six possible
outcomes
Collectively Exhaustive Events

Events are said to be collectively exhaustive if the

list of outcomes includes every possible outcome
• Both heads and
tails as possible
outcomes of
OUTCOME
coin flips OF ROLL
PROBABILITY
• All six possible 1 1/
6
outcomes 2 1/
6
of the roll 3 1/
6
of a die 4 1/
6
5 1/
6
6 1/
6
Total 1
 Drawing a Card

Draw one card from a deck of 52 playing cards

P (drawing a 7) = 4/52 = 1/13

P (drawing a heart) = 13/52 = 1/4

◼ These two events are not mutually exclusive

since a 7 of hearts can be drawn
◼ These two events are not collectively
exhaustive since there are other cards in the
deck besides 7s and hearts
 Table of Differences

MUTUALLY COLLECTIVELY
DRAWS
EXCLUSIVE EXHAUSTIVE
1. Draws a spade and a club Yes No
2. Draw a face card and a Yes Yes
number card
3. Draw an ace and a 3 Yes No
4. Draw a club and a nonclub Yes Yes
5. Draw a 5 and a diamond No No
6. Draw a red card and a No No
diamond
Adding Mutually Exclusive Events

We often want to know whether one or a

second event will occur
◼ When two events are mutually
exclusive, the law of addition is –

P (event A or event B) = P (event A) + P (event B)

P (spade or club) = P (spade) + P (club)

= 13/52 + 13/52
= 26/52 = 1/2 = 0.50 = 50%
Adding Not Mutually Exclusive Events

The equation must be modified to account

for double counting
◼ The probability is reduced by
subtracting the chance of both events
occurring together
P (event A or event B) = P (event A) + P (event B)
– P (event A and event B both occurring)

P (A or B) = P (A) + P (B) – P (A and B)

P(five or diamond) = P(five) + P(diamond) – P(five and diamond)

= 4/52 + 13/52 – 1/52
= 16/52 = 4/13
 Venn Diagrams

P (A and B)

P (A) P (B) P (A) P (B)

Events that are mutually Events that are not

exclusive mutually exclusive

P (A or B) = P (A) + P (B) P (A or B) = P (A) + P (B)

– P (A and B)
Figure 2.1 Figure 2.2
Statistically Independent Events

Events may be either independent or dependent

• For independent events, the occurrence of one event
has no effect on the probability of occurrence of the
second event
Which Sets of Events Are Independent?

1. (a) Your education

Dependent events
(b) Your income level

2. (a) Draw a jack of hearts from a full 52-card deck Independent

(b) Draw a jack of clubs from a full 52-card deck events

3. (a) Chicago Cubs win the National League pennant Dependent

(b) Chicago Cubs win the World Series events

4. (a) Snow in Santiago, Chile

Independent events
(b) Rain in Tel Aviv, Israel
Three Types of Probabilities
• Marginal (or simple) probability is just the probability of an
event occurring
P (A)
◼ Joint probability is the probability of two or more events occurring and is
the product of their marginal probabilities for independent events
P (AB) = P (A) x P (B)

◼ Conditional probability is the probability of event B given that event A

has occurred
P (B | A) = P (B)
◼ Or the probability of event A given that event B has occurred
P (A | B) = P (A)
 Joint Probability Example

The probability of tossing a 6 on the first

roll of the die and a 2 on the second roll
P (6 on first and 2 on second)
= P (tossing a 6) x P (tossing a 2)
= 1/6 x 1/6 = 1/36 = 0.028
 Independent Events
A bucket contains 3 black balls and 7 green balls
◼ We draw a ball from the bucket, replace it, and
draw a second ball

1. A black ball drawn on first draw

P (B) = 0.30 (a marginal probability)
2. Two green balls drawn
P (GG) = P (G) x P (G) = 0.7 x 0.7 = 0.49
(a joint probability for two independent events)
 Independent Events
A bucket contains 3 black balls and 7 green balls
◼ We draw a ball from the bucket, replace it, and
draw a second ball
3. A black ball drawn on second draw if the first draw is
green
P (B | G) = P (B) = 0.30
(a conditional probability but equal to the marginal
because the two draws are independent events)
4. A green ball is drawn on the second if the first draw was
green
P (G | G) = P (G) = 0.70
(a conditional probability as in event 3)
 Random Variables

A random variable assigns a real number to

every possible outcome or event in an
experiment
X = number of refrigerators sold during the day

Discrete random variables can assume only a finite

or limited set of values
Continuous random variables can assume any one
of an infinite set of values
 Random Variables – Numbers
RANGE OF
RANDOM
EXPERIMENT OUTCOME RANDOM
VARIABLES
VARIABLES
Stock 50 Number of Christmas X 0, 1, 2,…, 50
Christmas trees trees sold
Inspect 600 Number of acceptable Y 0, 1, 2,…, 600
items items
Send out 5,000 Number of people Z 0, 1, 2,…, 5,000
sales letters responding to the
letters
Build an Percent of building R 0 ≤ R ≤ 100
apartment completed after 4
building months
Test the lifetime Length of time the S 0 ≤ S ≤ 80,000
of a lightbulb bulb lasts up to 80,000
(minutes) minutes
Random Variables – Not Numbers

RANGE OF
RANDOM
EXPERIMENT OUTCOME RANDOM
VARIABLES
VARIABLES
Students Strongly agree (SA) 5 if SA 1, 2, 3, 4, 5
respond to a Agree (A) 4 if A..
questionnaire Neutral (N) X= 3 if N..
Disagree (D) 2 if D..
Strongly disagree (SD) 1 if SD

One machine Defective Y= 0 if defective 0, 1

is inspected Not defective 1 if not defective

Consumers Good 3 if good…. 1, 2, 3

respond to Average Z= 2 if average
how they like Poor 1 if poor…..
a product
Example: In “measles Study”, we define a random variable 𝑋 as the number of parents in a married
couple who have had childhood measles.
This random variable can take values of 0, 1 𝑎𝑛𝑑 2.
Note:
• Random variable is not exactly the same as the variable defining a data.

• The probability that the random variable takes a given value can be computed using the rules governing
probability.
• For example, the probability that 𝑋 = 1 means either mother or father but not both has had measles is 0.32.
Symbolically, it is denoted as P(X=1) = 0.32
Probability Distribution

A probability distribution is a definition of probabilities of the values of random

variable.

Example: Given that 0.2 is the probability that a person (in the ages between 17 and 35) has had
childhood measles. Then the probability distribution is given by

X Probability

?
0 0.64
1 0.32
2 0.04
Probability Distribution
• In data analytics, the probability distribution is important with which many statistics making
inferences about population can be derived .

• In general, a probability distribution function takes the following form

𝒙 𝒙𝟏 𝒙𝟐 … … … … . . 𝒙𝒏
𝑓 𝑥 = 𝑃(𝑋 = 𝑥) 𝑓 𝑥1 𝑓 𝑥2 … … . . 𝑓(𝑥𝑛 )

Example: Measles Study

0.64

𝒙 0 1 2
𝑓 𝑥 0.64 0.32 0.04 0.32
f(x)
0.04

x
Taxonomy of Probability Distributions
Discrete probability distributions
•Binomial distribution
•Multinomial distribution
•Poisson distribution
•Hypergeometric distribution

Continuous probability distributions

•Normal distribution
•Standard normal distribution
•Gamma distribution
•Exponential distribution
•Chi square distribution
•Lognormal distribution
•Weibull distribution
Usage of Probability Distribution
1) A study involving testing the effectiveness of a new drug, the number of cured
patients among all the patients who use such a drug approximately follows a
binomial distribution.
2) Operation of ticketing system in a busy public establishment (e.g., airport), the
arrival of passengers can be simulated using Poisson distribution.

MANY MORE…………………..
Discrete Probability Distributions
Binomial Distribution
• In many situations, an outcome has only two outcomes: success and failure.
• Such outcome is called dichotomous outcome.
• An experiment when consists of repeated trials, each with dichotomous outcome is called Bernoulli process. Each
trial in it is called a Bernoulli trial.

Example 4.5: Firing bullets to hit a target.

• Suppose, in a Bernoulli process, we define a random variable X ≡ the number of successes in trials.
• Such a random variable obeys the binomial probability distribution, if the experiment satisfies the following
conditions:
1) The experiment consists of n trials.
2) Each trial results in one of two mutually exclusive outcomes, one labelled a “success” and the other a “failure”.
3) The probability of a success on a single trial is equal to 𝒑. The value of 𝑝 remains constant throughout the
experiment.
4) The trials are independent.
Defining Binomial Distribution

The function for computing the probability for the binomial probability distribution
is given by
𝑛!
𝑓 𝑥 = 𝑝 𝑥 (1 − 𝑝)𝑛−𝑥
𝑥! 𝑛 − 𝑥 !
for x = 0, 1, 2, …., n
Here, 𝑓 𝑥 = 𝑃 𝑋 = 𝑥 , where 𝑋 denotes “the number of success” and 𝑋 = 𝑥
denotes the number of success in 𝑥 trials.
Binomial Distribution
Example: Measles study
X = having had childhood measles a success
p = 0.2, the probability that a parent had childhood measles
n = 2, here a couple is an experiment and an individual a trial, and the number of trials is two.

Thus,
2!
𝑃 𝑥=0 = (0.2)0 (0.8)2−0 =?
0! 2−0 !

2!
𝑃 𝑥=1 = (0.2)1 (0.8)2−1 = ?
1! 2−1 !

2!
𝑃 𝑥=2 = (0.2)2 (0.8)2−2 = ?
2! 2 − 2 !
Binomial Distribution

Example: Verify with real-life experiment

Suppose, 10 pairs of random numbers are generated by a computer (Monte-Carlo method)

15 38 68 39 49 54 19 79 38 14

If the value of the digit is 0 or 1, the outcome is “had childhood measles”, otherwise, (digits 2 to 9), the outcome is “did
not”.
For example, in the first pair (i.e., 15), representing a couple and for this couple, x = 1. The frequency distribution, for
this sample is

x 0 1 2
f(x)=P(X=x) 0.7 0.3 0.0

Note: This has close similarity with binomial probability distribution!

The Multinomial Distribution
The binomial experiment becomes a multinomial experiment, if we let each trial has more than two possible
outcome.

If a given trial can result in the k outcomes 𝐸1 , 𝐸2 , … … , 𝐸𝑘 with probabilities 𝑝1 , 𝑝2 , … … , 𝑝𝑘 , then the
probability distribution of the random variables 𝑋1 , 𝑋2 , … … , 𝑋𝑘 representing the number of occurrences
for 𝐸1 , 𝐸2 , … … , 𝐸𝑘 in n independent trials is
𝑛
𝑓 𝑥1 , 𝑥2 , … … , 𝑥𝑘 = 𝑥1 ,𝑥2 ,……,𝑥𝑘
𝑝1 𝑥1 𝑝2 𝑥2 … … 𝑝𝑘 𝑥𝑘

𝑛 𝑛!
where 𝑥1 ,𝑥2 ,……,𝑥𝑘
=𝑥
1 !𝑥2 !……𝑥𝑘 !

σ𝑘𝑖=1 𝑥𝑖 = 𝑛 and σ𝑘𝑖=1 𝑝𝑖 = 1

The Hypergeometric Distribution
• Collection of samples with two strategies
• With replacement
• Without replacement
• A necessary condition of the binomial distribution is that all trials are independent to each other.
• When sample is collected “with replacement”, then each trial in sample collection is independent.

Example:

Probability of observing three red cards in 5 draws from an ordinary deck of 52 playing cards.
− You draw one card, note the result and then returned to the deck of cards
− Reshuffled the deck well before the next drawing is made
• The hypergeometric distribution does not require independence and is based on the sampling done without
replacement.
The Hypergeometric Distribution
• In general, the hypergeometric probability distribution enables us to find the probability of selecting 𝑥
successes in 𝑛 trials from 𝑁 items.

Properties of Hypergeometric Distribution

• A random sample of size 𝑛 is selected without replacement from 𝑁 items.
• 𝑘 of the 𝑁 items may be classified as success and 𝑁 − 𝑘 items are classified as failure.
Let 𝑋 denotes a hypergeometric random variable defining the number of successes.

The probability distribution of the hypergeometric random variable 𝑋, the number of successes in a
random sample of size 𝑛 selected from 𝑁 items of which 𝑘 are labelled success and 𝑁 − 𝑘 labelled as
failure is given by
𝑘 𝑁−𝑘
𝑥 𝑛−𝑥
𝑓 𝑥 =𝑃 𝑋=𝑥 = 𝑁
𝑛
max(0, 𝑛 − (𝑁 − 𝑘)) ≤ 𝑥 ≤ min(𝑛, 𝑘)
Multivariate Hypergeometric Distribution
The hypergeometric distribution can be extended to treat the case where the N items can be divided into 𝑘
classes 𝐴1 , 𝐴2 , … … , 𝐴𝑘 with 𝑎1 elements in the first class 𝐴1 , … and 𝑎𝑘 elements in the 𝑘𝑡ℎ class. We are
now interested in the probability that a random sample of size 𝑛 yields 𝑥1 elements from 𝐴1 , 𝑥2 elements
from 𝐴2 , … … , 𝑥𝑘 elements from 𝐴𝑘 .

If 𝑁 items are partitioned into 𝑘 classes 𝑎1 , 𝑎2 , … … , 𝑎𝑘 respectively, then the probability

distribution of the random variables 𝑋1 , 𝑋2 , … … , 𝑋𝑘 , representing the number of elements
selected from 𝐴1 , 𝐴2 , … … , 𝐴𝑘 in a random sample of size 𝑛, is
𝑎1 𝑎2 𝑎𝑘
𝑥1 𝑥2
…… 𝑥𝑘
𝑓 𝑥1 , 𝑥2 , … … , 𝑥𝑘 = 𝑃 𝑋1 = 𝑥1 , 𝑋2 = 𝑥2 , … … , 𝑋𝑘 = 𝑥𝑘 = 𝑁
𝑛
with σ𝑘𝑖=1 𝑥𝑖 = 𝑛 and σ𝑘𝑖=1 𝑎𝑖 = 𝑁
The Poisson Distribution
There are some experiments, which involve the occurring of the number of outcomes during a given time
interval (or in a region of space).
Such a process is called Poisson process.

Example:
Number of clients visiting a ticket selling counter in a metro station.
The Poisson Distribution
Properties of Poisson process

• The number of outcomes in one time interval is independent of the number that occurs in any other
disjoint interval [Poisson process has no memory]
• The probability that a single outcome will occur during a very short interval is proportional to the
length of the time interval and does not depend on the number of outcomes occurring outside this time
interval.
• The probability that more than one outcome will occur in such a short time interval is negligible.

The probability distribution of the Poisson random variable 𝑋, representing the number of
outcomes occurring in a given time interval 𝑡, is

𝑒 −𝜆𝑡 . (𝜆𝑡)𝑥
𝑓 𝑥, 𝜆𝑡 = 𝑃 𝑋 = 𝑥 = , 𝑥 = 0, 1, … …
𝑥!

where 𝜆 is the average number of outcomes per unit time and 𝑒 = 2.71828 …
 Descriptive measures
Given a random variable X in an experiment, we have denoted 𝑓 𝑥 = 𝑃 𝑋 = 𝑥 , the probability that 𝑋 = 𝑥.
For discrete events 𝑓 𝑥 = 0 for all values of 𝑥 except 𝑥 = 0, 1, 2, … . .

Properties of discrete probability distribution

1. 0 ≤ 𝑓(𝑥) ≤ 1
2. σ 𝑓 𝑥 = 1
3. 𝜇 = σ 𝑥. 𝑓(𝑥) [ is the mean ]
4. 𝜎 2 = σ 𝑥 − 𝜇 2 . 𝑓(𝑥) [ is the variance ]

In 2, 3 𝑎𝑛𝑑 4, summation is extended for all possible discrete values of 𝑥.

1
Note: For discrete uniform distribution, 𝑓 𝑥 = with 𝑥 = 1, 2, … … , 𝑛
𝑛 𝑛
1
𝜇 = ෍ 𝑥𝑖
𝑛
𝑖=1
1 𝑛
and 𝜎 2 = σ𝑖=1(𝑥𝑖 −𝜇)2
𝑛
Descriptive measures
1. Binomial distribution
The binomial probability distribution is characterized with 𝑝 (the probability of success) and 𝑛 (is the number of
trials). Then
𝜇 = 𝑛. 𝑝

𝜎 2 = 𝑛𝑝 1 − 𝑝

2. Hypergeometric distribution
The hypergeometric distribution function is characterized with the size of a sample (𝑛), the number of items
(𝑁) and 𝑘 labelled success. Then

𝑛𝑘
𝜇=
𝑁
𝑁−𝑛 𝑘 𝑘
𝜎2= . n. (1 − )
𝑁−1 𝑁 𝑁
Descriptive measures
3. Poisson Distribution
The Poisson distribution is characterized with 𝜆𝑡 where 𝜆 = 𝑡ℎ𝑒 𝑚𝑒𝑎𝑛 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 and 𝑡 = 𝑡𝑖𝑚𝑒 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙.

𝜇 = 𝜆𝑡
𝜎 2 = 𝜆𝑡
Continuous Probability Distributions
Continuous Probability Distributions

f(x)

x1 x2 x3 x4
X=x
Discrete Probability distribution

f(x)

X=x
Continuous Probability Distribution
Continuous Probability Distributions

• When the random variable of interest can take any value in an interval, it is called continuous random
variable.

• Every continuous random variable has an infinite, uncountable number of possible values (i.e., any value in
an interval)

• Consequently, continuous random variable differs from discrete random variable.

Properties of Probability Density Function
The function 𝑓(𝑥) is a probability density function for the continuous random variable 𝑋, defined over the
set of real numbers 𝑅, if

1. 𝑓 𝑥 ≥ 0, for all 𝑥 ∈ 𝑅
∝
2. ‫׬‬−∝ 𝑓 𝑥 𝑑𝑥 = 1
𝑏
3. 𝑃 𝑎 ≤ 𝑋 ≤ 𝑏 = ‫𝑥𝑑 )𝑥(𝑓 𝑎׬‬ f(x)
∝
4. 𝜇 = ‫׬‬−∝ 𝑥𝑓(𝑥) 𝑑𝑥
∝ a b
5. 𝜎 2 = ‫׬‬−∝ 𝑥 − 𝜇 2𝑓 𝑥 𝑑𝑥
X=x
Continuous Uniform Distribution
• One of the simplest continuous distribution in all of statistics is the continuous uniform distribution.

The density function of the continuous uniform random variable 𝑋 on the interval [𝐴, 𝐵] is:
1
𝐴≤𝑥≤𝐵
𝑓 𝑥: 𝐴, 𝐵 = ൞ 𝐵 − 𝐴
0 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Continuous Uniform Distribution

f(x)
c

A B
X=x
Note:
−∞ 1
a) ‫𝐵 = 𝑥𝑑 𝑥 𝑓 ∞׬‬−𝐴 × (𝐵 − 𝐴) = 1
𝑑−𝑐
b) 𝑃(𝑐 < 𝑥 < 𝑑)= 𝐵−𝐴 where both 𝑐 and 𝑑 are in the interval (A,B)
𝐴+𝐵
c) 𝜇 = 2
(𝐵−𝐴)2
d) 2
𝜎 =
12
Normal Distribution
• The most often used continuous probability distribution is the normal distribution; it is also known as
Gaussian distribution.

• Its graph called the normal curve is the bell-shaped curve.

• Such a curve approximately describes many phenomenon occur in nature, industry and research.

• Physical measurement in areas such as meteorological experiments, rainfall studies and measurement of
manufacturing parts are often more than adequately explained with normal distribution.

• A continuous random variable X having the bell-shaped distribution is called a normal random variable.
Normal Distribution
• The mathematical equation for the probability distribution of the normal variable depends upon the two parameters
𝜇 and 𝜎, its mean and standard deviation.

f(x)
𝜎

𝜇
x

The density of the normal variable 𝑥 with mean 𝜇 and variance 𝜎 2 is

2
(𝑥−𝜇)
1 ൗ 2
𝑓 𝑥 = 𝑒− 2𝜎 −∞ < 𝑥 < ∞
𝜎 2𝜋

where 𝜋 = 3.14159 … and 𝑒 = 2.71828 … . ., the Naperian constant

Normal Distribution
σ1 = σ2
σ1

σ2

µ1
µ1 µ2
µ2 µ1 = µ2
Normal curves with µ1< µ2 and σ1 = σ2 Normal curves with µ1 = µ2 and σ1< σ2

σ1

σ2

µ1 µ2
Normal curves with µ1<µ2 and σ1<σ2
Properties of Normal Distribution
• The curve is symmetric about a vertical axis through the mean 𝜇.
• The random variable 𝑥 can take any value from −∞ 𝑡𝑜 ∞.
• The most frequently used descriptive parameter s define the curve itself.
• The mode, which is the point on the horizontal axis where the curve is a maximum occurs at 𝑥 = 𝜇.
• The total area under the curve and above the horizontal axis is equal to 1.
∞ 1 ∞ − 1 2 (𝑥−𝜇)2
‫׬‬−∞ 𝑓 𝑥 𝑑𝑥 =
𝜎 2𝜋
‫׬‬−∞ 𝑒 2𝜎 𝑑𝑥 =1
1
∞ 1 ∞ − 2 (𝑥−𝜇)2
• 𝜇 = ‫׬‬−∞ 𝑥. 𝑓 𝑥 𝑑𝑥 = ‫׬‬−∞
𝑥. 𝑒 2𝜎 𝑑𝑥
𝜎 2𝜋
1 (𝑥−𝜇)
2 1 ∞ 2 −2[ ൗ𝜎2]
• 𝜎 = ‫׬‬ (𝑥 − 𝜇) . 𝑒 𝑑𝑥
𝜎 2𝜋 −∞
1 𝑥2 − 12 (𝑥−𝜇)2
• 𝑃 𝑥1 < 𝑥 < 𝑥2 = ‫ 𝑒 ׬‬2𝜎 𝑑𝑥
𝜎 2𝜋 𝑥1
denotes the probability of x in the interval (𝑥1 , 𝑥2 ).
𝜇 x1 x2
Standard Normal Distribution
• The normal distribution has computational complexity to calculate 𝑃 𝑥1 < 𝑥 < 𝑥2 for any two (𝑥1 , 𝑥2 ) and given 𝜇 and 𝜎

• To avoid this difficulty, the concept of 𝑧-transformation is followed.

𝑥−𝜇
z= [Z-transformation]
𝜎

• X: Normal distribution with mean 𝜇 and variance 𝜎 2 .

• Z: Standard normal distribution with mean 𝜇 = 0 and variance 𝜎 2 = 1.

• Therefore, if f(x) assumes a value, then the corresponding value of 𝑓(𝑧) is given by
1 𝑥2 − 1 2 (𝑥−𝜇)2
𝑓(𝑥: 𝜇, 𝜎) : 𝑃 𝑥1 < 𝑥 < 𝑥2 = ‫ 𝑒 𝑥׬‬2𝜎 𝑑𝑥
𝜎 2𝜋 1
1 𝑧2 −1𝑧 2
= ‫ 𝑒 ׬‬2 𝑑𝑧
𝜎 2𝜋 𝑧1

= 𝑓(𝑧: 0, 𝜎)
Standard Normal Distribution

The distribution of a normal random variable with mean 0 and variance 1 is called a standard normal
distribution.

0.09
0.4
0.08 σ σ=1
0.07
0.3
0.06

0.05
0.2
0.04

0.03

0.02 0.1

0.01

0.00 0.0
-5 0 5 10 15 20 25 -3 -2 -1 0 1 2 3

x=µ µ=0
f(x: µ, σ) f(z: 0, 1)
Exponential Distribution

The continuous random variable 𝑥 has an exponential distribution with parameter 𝛽 , where:
𝑥
1 −𝛽
𝑒
𝑓 𝑥: 𝛽 = ൞𝛽 where 𝛽 > 0
0

Note:
1) The mean and variance of gamma distribution are
𝜇 = 𝛼𝛽
𝜎 2 = 𝛼𝛽 2
2) The mean and variance of exponential distribution are
𝜇=𝛽
𝜎 2 = 𝛽2
RANDOM NUMBER
GENERATION
Criteria for Good Random Number Generators

• Long period
• Strong theoretical foundation
• Able to pass empirical statistical tests for independence and
distribution
• Speed/efficiency
• Portability: can be implemented easily using different languages
and computers
• Repeatability: should be able to generate same sequence from
same seed
• Be cryptographically strong to external observer: unable to predict
next value from past values
• Good distribution of points throughout domain (low discrepancy)
 cont..

• Ideal aim is that no statistical test can distinguish RNG output from
i.i.d. U(0, 1) sequence
• Not possible in practice due to limits of testing and limits of finite-period
generators
• More realistic goal is passing only key (relevant) tests
• Null hypothesis: sequence of random numbers is realization of i.i.d.
U(0, 1) stochastic process
• Almost limitless number of possible tests of this hypothesis
• Failing to reject null hypothesis improves confidence in generator but
does not guarantee random numbers will be appropriate for all
applications
• Bad RNGs fail simple tests; good RNGs fail only complicated and/or
obscure tests
Types of Random Number Generators

• Linear: commonly used

• Combined: can increase period and improve statistical
properties
• Nonlinear: structure is less regular than linear generators
but more difficult to implement and analyze
• Physical processes (e.g., timing in atomic decay, internal
system noise, etc.)
• Not as widely used as computer-based generators due to
costliness of implementation, lack of speed, and inability to
reproduce same sequence
Pseudo-Random Numbers
 cont..

• Approach: Arithmetically generation (calculation) of random numbers

• “Pseudo”, because generating numbers using a known method removes the potential for
true randomness.

• Goal: To produce a sequence of numbers in [0,1] that simulates, or imitates, the ideal
properties of random numbers (RN).
 cont..
Important properties of good random number routines:
• Fast
• Portable to different computers
• Have sufficiently long cycle
• Replicable
✓Verification and debugging
✓Use identical stream of random numbers for different systems
• Closely approximate the ideal statistical properties of uniformity and
independence
cont..
MIDSQUARE METHOD
• Start with a four-digit positive integer Z0
• Square it to obtain an integer with up to
eight digits (if necessary, append zeros to
the left to make it exactly eight digits)
• Take the middle four digits of this eight-
digit number as the next four-digit
number, Z1
• Place a decimal point at the left of Z1 to
obtain the first “U(0, 1) random number,”
U1
• Then let Z2 be the middle four digits of
Z21 and let U2 be Z2 with a decimal point
at the left, and so on
Challenges
• Not really random
✓Entire sequence determined by Z0
✓If Zi reappears, entire sequence will be recycled
• Objections applies to all arithmetic generators
• Arithmetic generators are called sometimes pseudorandom
LINEAR CONGRUENTIAL GENERATORS

Sequence of INTEGERS Z1, Z2, . . .

NONNEGATIVE INTEGERS:
m (modulus),
a (multiplier),
c (increment),
Z0 (seed) are
cont..
Problems
1. Calculate 20 random numbers using LCG. Provided that seed value =
19, a=22, c=4,and m=63.

2. Generate maximum possible random numbers using LCG when the

seed value is 7, a=7, c=7 and m=10.
Conditions for LCG
a) The only positive integer that (exactly) divides both m and c is 1.
b) If q is a prime number (divisible by only itself and 1) that divides m,
then q divides a - 1.
c) If 4 divides m, then 4 divides a - 1.

Because of condition (a), LCGs tend to behave differently for

➢ c > 0 (called mixed LCGs)
➢ c = 0 (called multiplicative LCGs)
VARIANCE REDUCTION
TECHNIQUES
Characteristic Features
• Increase the simulation efficiency.
• Reducing the variances of the random variables outputs from a
simulation without disturbing its expectation, we can obtain greater
precision with less simulation time.
• All VRTs require some effort of understanding the model and the
effects of VRT on it.
• Developed mainly for Monte Carlo simulation. But can also be used in
independent sampling and correlated sampling.
Some Common VRTs
• Common Random Numbers
• Antithetic Variables
• Control Variates
• Conditioning

The first method is typically used in a simulation study in which we

want to compare performance measures of two different systems. All
other methods are also useful in the case that we want to simulate a
performance measure of only a single system.
Common Random Numbers
• If X and Y are two random variables (for two systems), then

• X and Y are positively correlated,

• Covariance is a measure of how much two random variables vary
together. For example, height and weight of giraffes have positive
covariance because when one is big the other tends also to be big.
• Variance of (X − Y) will be smaller than in the case that X and Y are
independent
• Here, common refers to dependent
Characteristics
• Used when comparing two or more alternative system configurations
instead of investigating a single configuration.
• Sometimes called correlated sampling, matched streams, matched
pairs.
• Random numbers are used to generate random variates from other
distributions which are used to drive the simulation models.
• It is necessary to ensure that generated variates react monotonically
to U. Be aware of backfiring.
• It is important to dedicate a stream of random number to a specific
random variate generator.
Antithetic Variables
• Used in simulating single systems.
• Induced correlation between separate runs, but seeks negative
correlation.
• To make pairs of small observation and a large observation. These
observations should offset each other.
• These average of the two will be closer to the expected compared to
when the two outputs are independent.
• AV uses complimentary random numbers to drive the two replication
in pairs. It is important to point out to use U in one replication and (U
-1) in the succeeding.
 cont..
• If W1 and W2 are the outcomes of two successive simulation runs,
then,
Control Variates
• Attempts to take advantage of correlation between certain random
variables to obtain a variance reduction.
• These correlations may arise during simulation or induced through an
auxiliary simulation.
• Given X and Y random variables, if Y affects X, and if Y > v, then X >
and vice-versa.
Conditioning
• Indirect Estimation applied usually for queueing-type simulation
models where the measures to be estimated are steady-state
measures we can substitute actual expectations which can be arrived
at through an exact solution.
• Conditioning Similar to indirect estimation. Exploits some special
property of a model to replace an estimate of a quantity by its exact
analytical value.
Steps in determining the simulation run
length
• The simulation run length increases progressively until a subsequence
of n samples (taken from the original output sequence) appears to be
independent, as determined by the runs test.
• Accomplish this by systematic sampling, i.e., select a number l
between 1 and L, choose that observation and then every lth
observation thereafter. For example, n lag 5 systematic samples will
be observations x5k+1 for k = 0, 1, 2, . . . , n - 1.
• Here, the chosen ‘l’ will be sufficiently large so that the systematic
samples appear to be independent.
• This is possible because we assume the autocorrelations of the
output sequence die off as the lag increases.
 cont..
• To avoid incorrectly determining a correlated sequence to be
independent because it possesses some special structure, we use
both the runs-up and runs-down tests simultaneously.
• Thus, if the probability of an independent sequence passing the runs-
up test is p1 and passing the runs-down test is p2, the probability of an
independent sequence passing the combined runs test is p = p1p2.
• We compute the lag l at which the systematic samples appear to be
independent, as determined by the runs test.
• The minimum required sample size is then N = nl, i.e., the minimum
required simulation run length computed based on the data, which is
called the computation run length.