performance of the existing system. Identify sources of randomness in the system, i.e., the stochastic inputvariables. Select an appropriate input probabilitydistribution for each stochastic input variable andestimate corresponding parameter(s).Software packages for distribution fitting andselection include ExpertFit, BestFit, and add-ons in somestandard statistical packages. These aids combinegoodness-of-fit tests, e.g.,
test, Kolmogorov-Smirnovtest, and Anderson-Darling test, and parameterestimation in a user friendly format.Standard distributions, e.g., exponential, Poisson,normal, hyperexponential, etc., are easy to model andsimulate. Although most simulation software packagesinclude many distributions as a standard feature, issuesrelating to random number generators and generatingrandom variates from various distributions are pertinentand should be looked into. Empirical distributions areused when standard distributions are not appropriate ordo not fit the available system data. Triangular, uniformor normal distribution is used as a first guess when nodata are available. For a detailed treatment of probabilitydistributions see Maria and Zhang (1997).
Step 4.Formulate and develop a model.
Developschematics and network diagrams of the system (How doentities flow through the system?). Translate theseconceptual models to simulation software acceptableform. Verify that the simulation model executes asintended. Verification techniques include traces, varyinginput parameters over their acceptable range andchecking the output, substituting constants for randomvariables and manually checking results, and animation.
Step 5.Validate the model.
Compare the model'sperformance under known conditions with theperformance of the real system. Perform statisticalinference tests and get the model examined by systemexperts. Assess the confidence that the end user placeson the model and address problems if any. For majorsimulation studies, experienced consultants advocate astructured presentation of the model by the simulationanalyst(s) before an audience of management and systemexperts. This not only ensures that the modelassumptions are correct, complete and consistent, butalso enhances confidence in the model.
Step 6.Document model for future use.
Documentobjectives, assumptions and input variables in detail.
4 HOW TO DESIGN A SIMULATIONEXPERIMENT?
A simulation experiment is a test or a series of tests inwhich meaningful changes are made to the inputvariables of a simulation model so that we may observeand identify the reasons for changes in the performancemeasures. The number of experiments in a simulationstudy is greater than or equal to the number of questionsbeing asked about the model (e.g., Is there a significantdifference between the mean delay in communicationnetworks A and B?, Which network has the least delay:A, B, or C? How will a new routing algorithm affect theperformance of network B?). Design of a simulationexperiment involves answering the question: what dataneed to be obtained, in what form, and how much? Thefollowing steps illustrate the process of designing asimulation experiment.
Step 7.Select appropriate experimental design.
Select a performance measure, a few input variables thatare likely to influence it, and the levels of each inputvariable. When the number of possible configurations(product of the number of input variables and the levelsof each input variable) is large and the simulation modelis complex, common second-order design classesincluding central composite, Box-Behnken, and full-factorial should be considered. Document theexperimental design.
Step 8.Establish experimental conditions for runs
.Address the question of obtaining accurate informationand the most information from each run. Determine if thesystem is stationary (performance measure does notchange over time) or non-stationary (performancemeasure changes over time). Generally, in stationarysystems, steady-state behavior of the response variable isof interest. Ascertain whether a terminating or a non-terminating simulation run is appropriate. Select the runlength. Select appropriate starting conditions (e.g., emptyand idle, five customers in queue at time 0). Select thelength of the warm-up period, if required. Decide thenumber of independent runs - each run uses a differentrandom number stream and the same starting conditions -by considering output data sample size. Sample size mustbe large enough (at least 3-5 runs for each configuration)to provide the required confidence in the performancemeasure estimates. Alternately, use common randomnumbers to compare alternative configurations by using aseparate random number stream for each samplingprocess in a configuration. Identify output data mostlikely to be correlated.
Step 9.Perform simulation runs
. Perform runsaccording to steps 7-8 above.
5 HOW TO PERFORM SIMULATIONANALYSIS?
Introduction to Modeling and Simulation