EMM 402: SIMULATION AND MODELING

Simulation with Arena

Lecture 6

Dr. James Wakiru

Aslı Sencer Erdem Slide 1

 Modeling detailed
operations

Terminating vs steady-state simulations

Strategy for data collection and analysis
Output analysis: Confidence interval and comparison of
alternatives
 Time Frame of Simulations
• Terminating simulation
▪ model dictates specific starting, stopping conditions
▪ run length will be well-defined (and finite)
• Non-terminating, steady-state simulation
▪ long-run (technically forever)
▪ theoretically, initial conditions don’t matter
(but practically they usually do)
▪ not clear how to terminate a simulation run
• This is really a question of intent of the study
▪ the simulation of a particular system might be either terminating
or non-terminating, depending on the objectives of the study
▪ sometimes it’s not clear which is appropriate
• Has major impact on how output analysis is done
▪ here: terminating
▪ non-terminating steady-state simulation in Chapter 7
▪ no steady-state: introduce terminating event and reduce to terminating
simulation
 Examples
• An aerospace manufacturer receives a contract to
produce 100 airplanes, which must be delivered within 18
months. The manufacturer would like to simulate various
manufacturing configurations to see which one can meet
the delivery deadline (at least cost).
• A company is going to build a new manufacturing system
that will operate 16 hours a day, 5 days a week. Assume
that production is continuous without breaks and that
loss of production between the end of one shift and the
beginning of the next shift is negligible.
▪ What if the company would like to determine the long-run
mean hourly throughput of the system?
▪ What if the company wants to know the time required for
the system to go from startup to “normal” operations?
 Strategy for Data Collection & Analysis
• For terminating case, make IID replications
▪ different replications use independent random numbers
▪ initialize System and Statistics between replications
• Model 6-1 (= Model 5-3, but 10 replications)

Note
cross-replication
variability

• How many replications?

▪ trial and error (now)
▪ approximate number for acceptable precision (below)
▪ sequential sampling (Chapter 12)
 Confidence Intervals for
Terminating Systems
• Cross-replication summary outputs

• Possibly most useful part – 95% confidence interval on

expected values
• Category Overview report
▪ if > 1 replication specified, Arena uses cross-replication data as above
▪ standard deviation not included
▪ other confidence levels, graphics, standard deviation – Output Analyzer
 Half Width and Number of Replications
•
n = number of replications
Confidence interval: X = sample mean
s = sample standard deviation
• Half-width = tn-1,1-α/2 = value from t tables

• Prefer smaller confidence intervals — precision

▪ can’t control t or s
▪ must increase n — how much?
• Set half-width = h, solve for
▪ not really solved for n (t, s depend on n)
• Approximation:
▪ replace t by z, corresponding standard normal critical value
▪ pretend that current s will hold for larger samples
s0 = sample standard deviation
▪ get 0
from “initial” number n of
0 n grows
replications
quadratically
• Easier but different approximation: as h
decreases
h0 = half width from “initial”
number n0 of replications
 Half Width and Number of Replications
(cont’d.)

• Application to Model 6-1

▪ from initial 10 replications, 95% half-width on Total Cost
was ±1605 ... let’s get this down to ±500 or less
▪ first formula:
– n  1.962(2243.382/5002) = 77.3, so 78
▪ second formula:
– n  10(16052/5002) = 103.0, so 103
▪ Model 6-2 (modified Model 6-1)
– number of Replications: 110 (conservative based on above)
▪ 95% confidence interval: 22241.71 ± 413.52
– satisfying criterion (overshot a bit?)
– got 11.65 ± 0.53 on Percent Rejected
– use max of sample sizes for precisions on multiple outputs
 Half Width and Number of Replications
(cont’d.)

• Central limit theorem

▪ The central limit theorem states that if you have a
population with mean µ and a standard deviation σ and
take sufficiently large random samples from the population
with replacement, then the distribution of the sample means
will be approximately normally distributed.
▪ Large sample size n≥ 30
▪ If population is normal, even if sample is smaller than 30
 Interpretation of Confidence Intervals
• A 95 % confidence interval is a “random” interval that’s
supposed to have “approximately” 95% chance of
containing the expected value
▪ “random”: next set of replications will give different interval
▪ “approximately”: coverage is not always close to 95%
▪ “target” expected value is a fixed, but unknown, number
▪ expected value = average of infinite number of replications
• Not: an interval that contains, say, 95% of the data
▪ that’s a prediction interval … useful too, but different
• Usual formulas assume normally-distributed data
▪ never true in simulation
▪ Central Limit Theorem: OK if n is large
▪ might be approximately true if output is a sum or an average, rather
than an extreme
– here: Total Cost is a sum
▪ Model 6-3 (1000 replications) to check normality (using Input Analyzer)
Interpretation of Confidence Intervals (cont)
 Comparing Two Alternatives
• Usually compare alternative system scenarios, configs, …
▪ for now, just two alternatives ... more later
• Model 6-4 (110 replications)
▪ add Output files to Statistic module
– Total Cost.dat
– Percent Rejected.dat
▪ saves output statistics for each replication to these files
▪ two versions
– (A) Base case – all inputs as originally defined, no extra resources
– (B) More-resources case – add 3 trunk lines (29), add one to New Sales,
New Tech 1, New Tech 2, New Tech 3, and New Tech All
– effect on Total Cost, Percent Rejected?
• Reasonable but not-quite-right idea
▪ make confidence intervals on expected outputs from each alternative,
see if they overlap
– Base case: 22241.71 ± 413.52, or [21828.12, 22655.16]
overlap!!
– More-resources: 22942.48 ± 449.42, or [22493.06, 23391.90]
▪ but this doesn’t allow for a precise, efficient statistical conclusion
 Output Analyzer

• Output Analyzer is a separate application

▪ launch separately from Windows, not from Arena
• Operates on .dat files produced by Arena
▪ to save output values: in Statistic data module
– Type = Output
– enter filename.dat in Output File column
▪ use different file names for alternative systems
– otherwise next run will overwrite this file
▪ .dat files are binary … can only be read by Output Analyzer
• Start Output Analyzer, open a new data group
▪ basically, a list of .dat files of current interest
▪ can save data group for later use – .dgr file extension
▪ add .dat files for the data group
 Output Analyzer: Confidence Intervals
• Analyze > Classical Conf Interval on Mean
▪ add data files … “A” and “B” for the two alternatives
▪ select “Lumped” for Replications field
▪ repeat for 10, 110 1000 replications
• Results:

10 replications

110 replications

1000
replications

• Solution: compare means via the Output Analyzer

 Output Analyzer: Compare Means
• Analyze > Compare Means menu option
▪ add data files … “A” and “B” for the two alternatives
▪ select “Lumped” for Replications field Paired-t test:
▪ accept Paired-t Test - sample sizes must be equal
- Zj = XAj – XBj IID random variables
▪ do not Scale Display
• Results:

• Confidence intervals on difference both miss 0

▪ so conclude that there is a (statistically) significant difference