MATH 3800

MATHEMATICAL MODELS AND NUMERICAL METHODS

SIMULATION AND PROBABILISTIC MODELING

© 2020 Gary Bazdell (excluding images)

INTRODUCTION
Sometimes to obtain data it is not possible to conduct experiments or
observe the behavior of a system. In cases where the behavior cannot
be explained analytically or data collected directly, the modeler might
simulate the behavior indirectly in some manner and then test the
various alternatives to estimate how each affects the behavior. Then
the data can be collected.

Simulation modeling and analysis is the process of creating and

experimenting with a computerized mathematical model of a physical
system.

Simulation and Probabilistic Modeling - Introduction 2

MONTE CARLO SIMULATION
The Monte Carlo method is a numerical method for statistical
simulation which utilizes sequences of random numbers to perform the
simulation.

The phrase “A sequence of random numbers uniformly distributed in

an interval 𝑚 to 𝑛” means a set of numbers with no apparent pattern,
where each number between 𝑚 and 𝑛 can appear with equal
likelihood.

Simulation and Probabilistic Modeling - Introduction 3

EXAMPLE
If you toss a six-sided die 100 times and
wrote down the numbers showing on the
die each time, you will end up with a
sequence of 100 random integers
uniformly distributed between 1 and 6.

Simulation and Probabilistic Modeling - Introduction 4

PROBABILISTIC VS DETERMINISTIC
Probabilistic processes are processes with an element of chance
involved. The opposite of probabilistic is deterministic.

Example
1. The area under a curve is deterministic (even if we can’t find it
precisely).
2. The time between arrivals of customers at an elevator on a
particular day is probabilistic.

What about the arrivals of patients for a doctor's appointment?

Simulation and Probabilistic Modeling - Introduction 5

ADVANTAGES OF MONTE CARLO SIMULATIONS
1. Relative ease with which it can sometimes be used to approximate
very complex probabilistic systems.
2. Provides performance estimation over a wide range of conditions
rather than a very restricted range as often required by an analytic
model.
3. Particular sub-model can be changed easily (which has the potential
of conducting a sensitivity analysis).
4. Modeler has control over the level of detail in a simulation.

Simulation and Probabilistic Modeling - Introduction 6

DISADVANTAGES OF MONTE CARLO SIMULATIONS
1. Typically expensive to develop and operate; often require many
hours to construct; large amounts of computer time and memory to
run.
2. Probabilistic nature of simulation model limits the conclusion that
can be drawn from a particular run unless a sensitivity analysis is
conducted.

Simulation and Probabilistic Modeling - Introduction 7

 MATH 3800
MATHEMATICAL MODELS AND NUMERICAL METHODS

GENERATING RANDOM NUMBERS

Generating Random Numbers - Random Numbers

GENERATING RANDOM NUMBERS
A computer does not really generate random numbers because
computers employ deterministic processes. Computers generate a
pseudorandom numbers. We will look at two methods for generating
pseudorandom numbers ourselves:

1. Middle-Square Method
2. Linear Congruence Method

Generating Random Numbers - Random Numbers 9

MIDDLE-SQUARE METHOD
The middle square method works as follows:

1. Start with a 𝑛 digit number 𝑥0 , called the seed.

2. Square it to obtain an 2𝑛 digit number number (add a leading zero
if necessary)
3. Take the middle 𝑛 digits as the next random number.

Continuing, we obtain a sequence that appears random over the

integers from 0 to 10𝑛 − 1. These integers can be scaled to any interval
𝑎 to 𝑏.

Generating Random Numbers - Random Numbers 10

EXERCISE 1
Let’s start with the seed 𝑥𝑜 = 7317.

Generating Random Numbers - Random Numbers 11

EXERCISE 2
Let’s start with the seed 𝑥𝑜 = 2041.

NOTE
One disadvantage is this method has a tendency to degenerate to zero.

Generating Random Numbers - Random Numbers 12

LINEAR CONGRUENCE METHOD
This method is as follows:

1. Choose three integers 𝑎, 𝑏, and 𝑐.

2. Given an initial seed 𝑥0 , we generate a sequence using the rule

𝑥𝑛+1 = 𝑎𝑥𝑛 + 𝑏 mod 𝑐

NOTE
mod 𝑐 means to obtain the remainder after dividing the quantity
𝑎𝑥𝑛 + 𝑏 by 𝑐.
Generating Random Numbers - Random Numbers 13
EXERCISE 3
Let’s start with the seed 𝑥𝑜 = 7 where 𝑎 = 1, 𝑏 = 7, and 𝑐 = 10.

Generating Random Numbers - Random Numbers 14

CYCLING
One problem with any algorithm for generating pseudorandom
numbers is cycling. Cycling is when our sequence of random numbers
repeats itself.

The linear congruence method produces a sequence of integers

between 0 and 𝑐 − 1. Cycling is guaranteed with a most 𝑐 random
numbers in a sequence. To combat this, we typically choose 𝑐 to be
very large. Many computers use 𝑐 = 231 .

Generating Random Numbers - Random Numbers 15

 MATH 3800
MATHEMATICAL MODELS AND NUMERICAL METHODS

SIMULATING DETERMINISTIC BEHAVIOR

Simulating Deterministic Behavior - Simulating Deterministic

Behaviour
AREA UNDER A CURVE
GOAL
Use a Monte Carlo simulation to find an approximate value to the area
under a non-negative curve.

Suppose 𝑦 = 𝑓 𝑥 is some given continuous function satisfying

0≤𝑓 𝑥 ≤𝑀

over the closed interval 𝑎, 𝑏 . Here 𝑀 is an upper bound for 𝑓 𝑥

Simulating Deterministic Behavior - Simulating Deterministic
17
Behaviour
ALGORITHM
Input: Total number of 𝑛 of random points to be generated in the simulation
Output: Approximate area under the specified curve 𝑦 = 𝑓(𝑥) over the given
interval 𝑎 ≤ 𝑥 ≤ 𝑏 where 0 ≤ 𝑓(𝑥) ≤ 𝑀.
Step 1: Initialize COUNTER= 0
Step 2: for 𝑖 = 1 to 𝑛
Step 2a: calculate random coordinates 𝑥𝑖 and 𝑦𝑖
Step 2b: calculate 𝑓(𝑥𝑖 ) for the random 𝑥𝑖 coordinate
Step 2c: if 𝑦𝑖 ≤ 𝑓(𝑥𝑖 ), then increment COUNTER by 1
Step 3: Calculate the area
COUNTER
𝐴 = 𝑀(𝑏 − 𝑎) ×
𝑛
Step 4: Output A.

Simulating Deterministic Behavior - Simulating Deterministic

18
Behaviour
EXERCISE 4
Using a Monte Carlo Simulation, approximate the area under the curve
𝑦 = cos 𝑥 over the interval 0,1 .

Simulating Deterministic Behavior - Simulating Deterministic

19
Behaviour
 MATH 3800
MATHEMATICAL MODELS AND NUMERICAL METHODS

SIMULATING PROBABILISTIC BEHAVIOR

Simulating Probabilistic Behavior - Simulating Probabilistic

Behaviour
FLIPPING A FAIR COIN
Here, we will create simulation models for simple probabilistic
behavior. This type of simulation can be extended to more complex
probabilistic behavior.

1
The probability of obtaining heads or tails on a flip of a coin is . What
2
happens when we actually flip a coin? Will one out of every two slips
be tails?

Probability is a long run average. Thus, in the long run, the ratio of the
number of tails to the number of flips approaches 0.5.

Simulating Probabilistic Behavior - Simulating Probabilistic

21
Behaviour
FLIPPING A FAIR COIN
Define 𝑃 𝑥 as follows where 𝑥 is a random number between 0,1 :

1
heads 0 ≤ 𝑥 ≤
𝑃 𝑥 = 2
1
tails ≤𝑥≤1
2

So, 𝑃 𝑥 assigns either a “heads” or “tails” to a number between 0,1 .

Simulating Probabilistic Behavior - Simulating Probabilistic

22
Behaviour
ALGORITHM
Input: Total number 𝑛 of random flips of a fair coin to be generated in
the simulation
Output: Probability of getting “tails” when we flip a fair coin.
Step 1: Initialize: COUNTER= 0
Step 2: for 𝑖 = 1 to 𝑛
Step 2a: generate a random number 𝑥𝑖 between 0 and 1
Step 2b: if 0.5 < 𝑥 ≤ 1, then COUNTER=COUNTER+1
COUNTER
Step 3: Calculate 𝑃 𝑡𝑎𝑖𝑙𝑠 =
𝑛
Step 4: Output Probability of tails, 𝑃(𝑡𝑎𝑖𝑙𝑠)
Simulating Probabilistic Behavior - Simulating Probabilistic
23
Behaviour
EXERCISE 5
What is we flip 3 fair coins, what is the probability that we get 2 heads.

Simulating Probabilistic Behavior - Simulating Probabilistic

24
Behaviour
ROLLING A FAIR DIE
Input Total number 𝑛 of random rolls of a die in the simulation.
Output The percentage or probability for rolls 1,2,3,4,5,6 .
Step 1: Initialize COUNTER 1 through COUNTER 6 to zero.
Step 2: For 𝑖 = 1 to 𝑛
Step 2a: Obtain a random number satisfying 0 ≤ 𝑥𝑖 ≤ 1.
Step 2b: If 𝑥𝑖 belongs to these intervals, then increment
the appropriate COUNTER.

Simulating Probabilistic Behavior - Simulating Probabilistic

25
Behaviour
ROLLING A FAIR DIE
1
0 ≤ 𝑥𝑖 ≤ COUNTER 1 = COUNTER 1 + 1
6
1 2
< 𝑥𝑖 ≤ COUNTER 2 = COUNTER 2 + 1
6 6
2 3
< 𝑥𝑖 ≤ COUNTER 3 = COUNTER 3 + 1
6 6
3 4
< 𝑥𝑖 ≤ COUNTER 4 = COUNTER 4 + 1
6 6
4 5
< 𝑥𝑖 ≤ COUNTER 5 = COUNTER 5 + 1
6 6
5 6
< 𝑥𝑖 ≤ COUNTER 6 = COUNTER 6 + 1
6 6
Step 3: Calculate probability of each roll 𝑗 = {1, 2, 3, 4, 5, 6} by
𝐶𝑂𝑈𝑁𝑇𝐸𝑅 𝑗
𝑛
Step 4: Output probabilities
Simulating Probabilistic Behavior - Simulating Probabilistic
26
Behaviour
EXERCISE 6
Verify that we are simulating a fair die.

Simulating Probabilistic Behavior - Simulating Probabilistic

27
Behaviour
NUMBER OF TRIALS
The number of trials needed to ensure a predetermined level of
confidence requires more knowledge of statistics.

The general rule is that in order to double accuracy of the result, four
times as many experiments are necessary.

Simulating Probabilistic Behavior - Simulating Probabilistic

28
Behaviour