See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/306012498

Systems Analysis Using Simulation

Article  in  Journal of Business Logistics · June 2012

DOI: 10.1111/j.0000-0000.2012.01041.x

CITATIONS READS

38 362

2 authors, including:

Xiang Wan
The Ohio State University
24 PUBLICATIONS   451 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Marquette Research View project

Ohio State University View project

All content following this page was uploaded by Xiang Wan on 05 November 2017.

The user has requested enhancement of the downloaded file.

Systems Analysis Using Simulation

by

Philip T. Evers
(Corresponding Author)
Associate Professor
University of Maryland
Robert H. Smith School of Business
Logistics, Business and Public Policy
College Park, MD 20742-1815
Phone: 301-405-7164
Email: pevers@rhsmith.umd.edu

Xiang Wan
Assistant Professor
Marquette University
College of Business Administration
Milwaukee, WI 53201-1881
Phone: 414-288-3095
Email: xiang.wan@marquette.edu
 Systems Analysis Using Simulation

Abstract

This paper offers an expanded classification of methodologies used in systems analysis. Of

these methods, one in particular—simulation—is presented in some detail. Issues associated
with building and running the model, as well as analyzing the results, are examined within the
context of supply chain management. It is claimed that the ability to conduct simulation
studies, along with the ease of misinterpreting their results, has greatly increased over time.

Keywords: systems analysis; research methodologies, simulation; supply chain management

Page 1 of 30
INTRODUCTION

The purpose of research is to address significant and substantive questions. The wealth of

existing research methods represents the means available for researchers who attempt to

answer these questions. To successfully address a research question, the appropriate

methodology must be employed. Use of the wrong tool can lead either to an unsatisfactory

answer or, even worse, to an incorrect answer.

The focus of this paper is on one particular research methodology: simulation. This

method will be discussed within a broader framework of systems analysis and specifically as it

relates to supply chain management. The power, as well as the limitations, of simulation will

be presented along with a description of how a simulation study is performed. Issues arising

during a study will also be discussed as will the future of simulation as a methodology. All of

this is intended to encourage the use of simulation when appropriate.

SYSTEMS ANALYSIS

Whether the topic is medicine, sociology, mathematics, agriculture, history, or any other

discipline, the question being posed should always be the driver of the research and

consequently the methodology, not vice versa. In many of these same disciplines, a primary

topic of interest is the understanding of how entities (these could be organizations, individuals,

machines, organisms, etc.) relate to and interact with one another—in other words, the analysis

of systems. Systems analysis is especially integral to business research, in general, and supply

chain management research, in particular, as firms attempt to coordinate their activities such

that customer demands are met in both an efficient and an effective manner.

Supply chain systems

Page 2 of 30
 Systems analysis within supply chain management can be found in a multitude of areas,

including inventory, transportation, and warehousing systems, to name a few (Carter and

Jennings 2004; Evers 1996; Waller et al. 2008; Waller et al. 2010). In the case of inventory

management, firms in a supply chain attempt to size their inventories at levels that will satisfy

customer demand at the lowest cost. However, if each organization sets its inventory level

independently of one another, especially when there is a lack of information transfer, the result

is the well-known bullwhip effect—large fluctuations in inventory levels and order quantities

across the supply chain stemming from small variations in customer demands (Lee et al. 1997;

Wan and Evers 2011). Likewise, stock-keeping locations at the same echelon of a supply chain

will need to carry more inventory to maintain the same level of customer service if they operate

independently of one another than if they work together—a result of the portfolio effect (Zinn

et al. 1989; Evers 1996). These two examples highlight the importance of systems analysis as

the nature of inventory management within a supply chain is such that individual stock-keeping

locations seldom exist in isolation but rather within a larger structure.

With regard to transportation management, the movement of goods from one location to

another is quite obviously another instance of systems analysis (Stank and Roath 1998;

Daugherty et al. 2001). From the user’s perspective, systems-related questions can range from

spatial concerns, such as the allocation of product to various locations, to temporal concerns,

such as the determination of shipment size on a specific route. From the carrier’s perspective,

their operating networks are, by their very nature, systems, and any attempt to examine only a

portion of their activities could very well lead to sub-optimization of the larger structure.

Even within a facility, systems analysis is important. For example, the activities within a

Page 3 of 30
warehouse can be viewed as a system of handling and storage processes. A determination of

the degree of product variety most appropriate for a particular distribution center cannot be

made without considering the interactions of demand forecasting, unit price, service level,

sales quantity, and so on (Wan et al. 2011). Of course, similar arguments could be made for

other warehousing questions such as a comparison of fixed slotting (where stock-keeping units

are assigned predetermined storage locations) versus variable slotting (where stock-keeping

units are allowed to be stored wherever space is available).

In general then, viewing supply chains as a system of links (movement activities) and

nodes (storage and/or processing activities) highlights the underlying structure on which the

supply chain operates (Coyle et al. 2008). Consequently, research that investigates operating

relationships amongst supply chain members must recognize the importance of taking a

system-wide perspective. Having established this connection between supply chains and

systems, attention can now be turned to their analysis.

The elements of a system

A system can be fully described by its elements. These elements include the structure, the

variables or attributes, and the state. The structure of the system is the underlying relationship

among entities. Starting with the links and nodes, the system structure for a transportation

carrier network typically includes the origin and destination points, as well as the terminal

locations. Other resources could also be considered a part of the structure; in the carrier

network case, an important resource is the number of vehicles. Of course, changes typically

occur within a system even if its underlying structure remains constant—measures of these

system changes are variables. For example, the transportation carrier may see shipment

Page 4 of 30
volumes fluctuate over time within its network, and this volume could represent a variable of

interest. Moreover, each shipment may have particular characteristics such as size,

origin-destination points, handling requirements, etc. Associated with specific entities in the

system, these individual differences are termed attributes. The particular values that the

variables and attributes take on at any given time within the system represent its state. Thus,

the state of the carrier’s system on day 123 at time 14:00 is 400 shipments on 200 vehicles

between 30 origin-destination pairs…

How the state changes is very important in terms of analyzing the system. If the changes in

a system are distinct and discrete, this is referred to as a discrete-event system. On the other

hand, if changes within a system occur such that they cannot be precisely attributed to a distinct

point in time, this is referred to as a continuous-event system. In a transportation context,

pipeline operations typically fall into this category. Various products may constantly flow

through the pipeline in varying volumes, but the precise transition from one product to another

or from one volume level to another may not be clearly identified. Generally speaking, the

discrete-event system applies, or at least suffices, in most supply chains; the continuous-event

system is more suitable for flow processes (things like pipelines, refineries and chemical plants,

electricity distribution, and internet transmissions).

Methods of systems analysis

With the state and its manner of change defined, methods for analyzing the system itself

can now be enumerated. Law and Kelton (2000) provide a concise hierarchy of methods to

analyze systems. As shown in Figure 1, this paper expands their categorization to include an

even wider range of methods. At the highest level of systems analysis, the first distinction is on

Page 5 of 30
the basis of systems control. Many research methodologies are available that allow the analyst

to investigate a system without requiring control over it. Perceptual analyses, such as the use of

surveys and panel discussions, and factual analyses, such as the use of archival data and

objective observations, afford the opportunity to understand the system without actually

affecting it. In essence, the analyst can act as an outside observer and glean pertinent

information that can be examined using such tools as econometric models and structural

equations to understand the system. While there is an obvious advantage to these approaches

in that the actual system is being directly (in the case of factual analysis) or indirectly (in the

case of perceptual analysis) observed, there is a major disadvantage with these approaches, too.

Since the analyst has no control over the system, it can be difficult to separate out the effects of

confounding exogenous factors and to perform sensitivity analyses when relevant changes in

the system have not occurred.

----------------------------------------------------

Insert Figure 1 here

----------------------------------------------------

While methods not requiring control abound, there are also methods that allow the analyst

to be more directly involved with the system. These techniques afford the analyst the ability to

actually dictate system operations and observe their effects. They can be broken down into

three categories (though, as will be discussed in short order, only two are shown in Figure 1):

physical analysis, behavioral analysis, and logical analysis.

Physical analysis involves manipulating either the actual system or a scale replica of it.

Consider first the manipulation of an actual system: say, an existing warehouse which employs

Page 6 of 30
a set of docks used for both loading and unloading. Operations could be altered such that some

of the docks are used only for loading and others are used only for unloading. A comparison of

the two could then take place. While a big advantage of this approach is that effects on actual

operations can be witnessed firsthand, there are a couple of equally big disadvantages. First,

experimenting with the actual system could result in enormous costs and cause irreparable

damage—changing dock assignments might be extremely costly and create disruptions that

adversely affect customer service. Second, while the effects on actual operations can be seen,

they may still be confounded with other factors—the change in dock assignments might occur

just as an increase in demand occurs and so it may be hard to unambiguously attribute adverse

results to either the change in assignments or the increased demand.

Consider now the manipulation of a scale replica: a small version of the warehouse could

be built and different layouts tried in an attempt to analyze the impact of changing dock

assignments. The difficulty here lies in making the scale replica actually operate and reflect the

intricate nuances of a system. Consequently, it is not surprising that scale replicas are rarely

used to analyze supply chain systems, but they do, nevertheless, represent an option in the

appropriate situation and can be commonly found in other disciplines (for example,

engineering studies using scale models in wind tunnels and medical studies using lab mice to

examine human diseases).

As it is readily apparent that physical analysis of supply chain systems is seldom

performed, this category is not shown in Figure 1. On the other hand, the two

analyst-controlled categories actually displayed in Figure 1 are much more commonplace.

Behavioral analysis includes experiments and games. In these exercises, participants take

Page 7 of 30
on certain roles within a system. One of the best known business applications of an exercise is

the Beer Game, where individuals act as members of a beer supply chain (Forrester 1958;

Sterman 1989). For instance, Croson and Donohue (2006) employed a set of experiments with

human participants to study the behavioral causes of the bullwhip effect. Note that while the

Beer Game requires participants to follow steps specified by the analyst, they have the freedom

to make their own decisions, and so while experiments and games can be influenced by the

analyst they usually cannot be completely controlled. Moreover, participants often learn from

their involvement and respond to incentives, whether perceived or real, so they are apt to

modify their own actions over time. And in many cases, the behavioral exercise itself is

significantly removed from reality and, as a result, may lead to fundamental questions about

the research. Even with the Beer Game case, concerns may arise regarding the likelihood of

finding an organization where one person is aware of both customer demands and inventory

levels and is also responsible for placing orders with a supplier.

Offering complete control to the analyst, logical analysis involves mathematical models.

These can be either analytical or simulation models. An analytical model is one or more

equations incorporating factors solely based on the analyst’s discretion that are solved to

understand a system. For example, Lee et al (1997) analyzed a series of analytical models in

order to study the bullwhip effect in supply chains. If the model cannot be solved, as is often the

case, a heuristic result may still be available or limiting mathematical assumptions may need to

be made to allow for a solution. For instance, Evers and Beier (1993) extended the square root

law stemming from the portfolio effect by relaxing some of the restrictive assumptions. For the

warehouse example mentioned earlier, a series of equations reflecting vehicle arrivals and

Page 8 of 30
departures as well as forklift movements could be developed to reflect dock operations. If an

optimal solution can be determined, then the analytical method has great appeal. Of course,

this assumes that the underlying assumptions do not stray too far from reality; otherwise, the

optimal solution is to a different problem. If an optimal solution cannot be determined, a

heuristic solution may still be available (DeCroix and Zipkin 2005). However, a heuristic

solution relies on even more assumptions and is no longer guaranteed to find the optimal result

(Vashi et al. 1995).

When all else fails, there is simulation! Simulation is also based on a series of logical steps

that may include equations to represent the actual system as accurately as possible. As all

decisions and parameters are specified by the analyst, the analyst has complete control over the

system without having to make potentially restrictive assumptions and can isolate the results by

avoiding confounding factors. In the warehouse case, a simulation model could be built to

reflect the individual warehouse operations and then run to ultimately generate inferences

regarding the system, without having to worry about exogenous influences such as electrical

outages or supplier disruptions (unless of course these things are actually of interest). Even in

the case of simulation, however, there is a downside as optimality is not guaranteed and

simulation modeling skills are necessary (Mentzer 1989)…it is not easy to build a simulation

model of a functioning warehouse (though it is getting easier!).

SIMULATION

The previous discussion outlined various methods for performing systems analysis and

highlighted some of the advantages and disadvantages. Too be sure, there are many more pros

and cons than those listed above, so with regard to simulation, a more detailed discussion

Page 9 of 30
follows. A useful way of looking at simulation modeling is in terms of what it can, and cannot,

do.

First, what can simulation do? A lot! One of the biggest benefits of simulation is that it

allows the analyst to investigate complex systems without making limiting assumptions. A

simulation can be crafted to the appropriate precision to account for as many interactions as

deemed necessary. Indeed, anyone familiar with modern video games has no doubt seen this

stunning level of complexity and realism ranging from sports games to war and fantasy games

and everything in between (and beyond). Research simulations also span the gamut of

applications and the corresponding complexity continues to increase as computing power and

storage capabilities continue to grow.

A particularly attractive feature of simulation is that it can speed up time. Time-based

simulations can be run in a fraction of real time and changes to the system can be monitored

accordingly. And as alluded to before, perhaps the most powerful aspect of simulation is that it

provides the researcher with complete control of the system. No changes to the system can

occur unless and until the researcher says they can. In so doing, confounding influences can be

dramatically, if not entirely, eliminated leading to unambiguous results.

Other features of simulation include the ability to incorporate random occurrences into the

system to estimate their effects. Similarly, changes to the underlying simulation model or

inputs can be made to allow for sensitivity analysis. In other words, simulations can be used to

answer “what if” questions. For example, Closs et al. (2010) simulate the random daily demand

of computers and perform sensitivity analysis on the inputs to investigate the impacts of

product complexity and inventory level on fill rate performance.

Page 10 of 30
 Now, what can’t simulation do? Everything! Though numerous methods exist to help

researchers reduce output variation in simulation results, simulations do not provide specific

answers. Simulations are stochastic processes and, as such, rely upon statistical

inferences—they do not give exact results. And though numerous methods exist to help

researchers find near optimal results, they cannot guarantee optimality (Vashi et al. 1995). If

specific, optimal answers are desired, analytical methods should be used.

Because simulation models are a representation of an actual system, they also cannot

ensure generalizeability. Simulation results are only as generalizeable as the system under

consideration (Meredith 1998). For example, results from a simulation model of a continuous

review inventory management system may be very transferrable from one industry to another;

however, results from a simulation model of a humanitarian aid system may not. It all depends

on the research question, not the research method.

Another drawback of simulation is that it is hard to model human behavior. Simulation

models are built on logic. And while they typically incorporate randomness, there is still a

systematic process to it. Human behavior, on the other hand, can be erratic and based on

factors other than logic, such as love, fear, opportunism, etc. Traits such as these can be very

difficult to define and describe in logical terms. In these situations, participant exercises may

be more appropriate.

How it works

Simulation modeling has three general aspects: the model, its inputs, and its outputs.

Regarding the model itself, this is the “black box” at the heart of the study. As noted earlier, a

simulation model is the set of logical steps that reflects the system of interest. In other words, it

Page 11 of 30
reflects the relationships between the various entities (i.e., the underlying structure of the

system).

As shown in Figure 1, there are different types of simulation models. Law and Kelton

(2000) provide a succinct classification of them based on time, randomness (or lack thereof),

and change characteristics found in the system of interest. The first dichotomy is time.

Simulations models that reflect system operations over time are referred to as dynamic models,

whereas those that do not involve time are called static models. Most supply chain

management simulations are likely to be dynamic models as the investigator is typically

interested in how the system changes over time (Cattani et al. 2011; Klassen and Menor 2007).

For example, the analysis of continuous and periodic review systems is, by its very nature, the

analysis of inventory fluctuations over time. However, static models do exist. The analysis of

inventory management in a single period is an example of a static system. In this class of

problems, which includes the very well-known newsvendor problem, time is typically of little

importance and its effects, if any, can be ignored.

A second classification depends upon whether the simulation incorporates randomness.

Deterministic simulation models have no probabilistic elements (Dhouib et al. 2008; Vashi et al.

1995). These models are often used when the underlying system is just too complex to solve.

Within inventory management, an application of a deterministic model is in the analysis of

discrete lot sizes in a materials requirements planning environment. For dependent demand

items where the requirements are known (i.e., based on parent items), the important question of

production and/or order quantities to minimize costs remains. In this situation, even though

randomness is not an issue, a simulation may still be necessary to ascertain the impact of

Page 12 of 30
different lot sizing techniques on total costs, especially in products having many levels.

Simulations that incorporate randomness are called stochastic models. Again, the underlying

stochastic process may simply be too complex to solve. Independent demand items typically

fall into this category as they are difficult, if not impossible, to forecast with complete accuracy.

(To be sure, many other supply chain systems also fall in this category as supply chains tend to

be inherently full of randomness.)

The final distinction was mentioned earlier and reflects how system changes occur: either

on a discrete or continuous basis. In inventory management, a periodic review system, where

inventory levels are checked after specific intervals, is an example of a discrete system; a

continuous review system, where inventory levels are constantly monitored, is an example of a

discrete system.

While the model itself represents the foundation and is typically the first thing that comes

to mind when referring to simulation, it is by no means the only thing. Viewing simulation

analysis in terms of a process, it is clear that the inputs and outputs are equally important. The

inputs represent the parameters that the simulation model will assume. The inputs also

influence the relationships between entities as the model is run. For example, in a simulation of

materials handling systems within a cross-docking facility, while the interactions between the

conveyors, forklifts, and other resources are represented by the simulation model itself, the

influences on this underlying structure come from the model inputs: factors such as the precise

number of forklifts, the speed of the conveyors, the hours of operations, and the probability of

breakdowns. Inputs drive the simulation and can be considered the independent variables in

the process.

Page 13 of 30
 At the other end of the process are the outputs. Simulation outputs are the genesis of the

results that must first be appropriately extracted and analyzed. In the materials handling

example, outputs might include item throughput times, forklift movement distances, and

capacity utilization. Outputs can be considered the dependent variables in the process.

A SIMULATION STUDY

Conducting the research

Even though simulation may be a less costly way of conducting research when compared

with many other methodologies (Bowersox and Closs 1989), simulation still has a cost. Aside

from the simplest of systems, computer software is needed to construct the simulation model.

Simulations can be developed in a wide range of software applications ranging from common

spreadsheet packages (with add-ons if desired) and standard programming languages (such as

Visual Basic or C++) to simulation-specific packages (such as Arena and Simul8). While

spreadsheets and programming languages are widely available and easy to understand, the

simulation-specific packages are becoming very easy to work with as well. Of course, these

latter packages tend to be more expensive to acquire, but their simulation capabilities are

tremendous.

Beyond the expense of software, a related cost is the heretofore mentioned learning cost.

To some extent, the cost of learning how to build and run simulation models using any

particular package is a one-time upfront cost. However, like any computer software, continual

updates of simulation packages require continual learning. For example, many of those who

learned simulation using the SIMAN language have transitioned over time to the Arena system

and incurred learning costs needed to work in the transformed environment.

Page 14 of 30
 In addition to the expenses directly associated with the software, other simulation costs

involve data collection and output analysis. These costs are not unlike those associated with

other standard data collection and analysis techniques. The cost of time is also a factor. While

researchers are able to speed up time through simulation (as opposed to, say, survey research

where significant time elapses between the sending of questionnaires and the receiving of

responses), a substantial amount of time is required to develop the model. Thus, a discussion

of modeling issues is fitting.

Modeling issues

It is important to recognize upfront that modeling issues are more than just computer

coding. Indeed, the first modeling issue arises long before the task of coding is to commence.

The system of interest must first be framed and its boundaries established.

Any arbitrary system is a component of a larger system and, at the same time, encompasses

many smaller systems. Hence, accurately scoping the system is critical. This can be done by

referring to the research question—what exactly is being asked? If the research problem deals

with on-line retailers and the fulfillment of customer orders at a distribution center, then the

system should be confined to those activities (Yan et al. 2010). In this case, only supplier

shipment receipts are of interest—their manufacturing operations are not. Likewise, only

customer demands are of interest—their subsequent use of the product is not.

In a similar vein, the level of system detail should also be defined at the appropriate level

of complexity. In the distribution center fulfillment case, inventory management policies,

order arrival processes, and demand fulfillment procedures are all fundamental to analysis.

The routing of order pickers within the facility may not be very important. As a consequence,

Page 15 of 30
the system should be defined to the level necessary; anything more is overkill and results in

needless and time-consuming effort in terms of such tasks as understanding and coding the

process, data collection and analysis, and simulation run times. One of the most important tips

to remember about simulation is to not overdo it!

The next step is to map out the process. Drawing a flowchart of the system is extremely

useful in understanding how the different entities and activities relate and examples abound

(Closs et al. 2010; Evers 1994). This diagram can help to identify where gaps in the knowledge

of the process lie. In addition, it can help to determine whether the system boundaries have

been established correctly and whether its level of complexity has been set properly. Clearly,

these first two steps—framing and flowcharting—are typically done in an iterative fashion.

Once the system has been suitably framed and appropriately defined, and the process

mapping has been completed, attention can be turned to the inputs and outputs. What

parameters will be varied in the simulation and what data will be needed for input purposes?

On the output side, what variables will be collected for further analysis? For example, in order

to investigate the impacts of information sharing in supply chains, Angulo et al. (2004) used

stochastic demand and lead time values as input and measured average inventory levels, fill

rates, and inventory costs as output. A well-described flowchart helps tremendously with this

endeavor.

After all of this upfront work has taken place, simulation coding can begin. For years,

those unfamiliar with simulation often viewed this activity as the be-all and end-all of

simulation. The preceding discussion on inputs and outputs should put this notion to rest.

Moreover, coding was viewed as the primary task of simulation research, and it was considered

Page 16 of 30
the major hurdle because it required programming knowledge. The new wave of simulation

packages with their practically non-existent programming requirements puts to rest this latter

concern. To be sure, coding is extraordinarily important, but it can be done. A solid flowchart

can be readily translated into a simulation model.

Once the simulation model is in place, the next step is arguably the most important:

verification and validation. Verifying a simulation model means ensuring that the simulation

logic works as desired. Manually walking through the model is a brute force method of doing

this. Test running the simulation with no probabilistic elements is another way. In both cases,

the researcher should be able to identify any errors in coding or logic. In the distribution center

order fulfillment example, running the simulation without randomness is a straightforward way

of making sure that orders are properly made when reorder points are reached and that lead

times are working correctly, too. Validating a simulation model means ensuring that the

simulation logic reflects the system of interest. Beyond mere face validity which might be

established by perusing the flowchart, examination of sample outputs and, if possible,

comparison with actual system data can help to establish content validity. For example, Torres

and Maltz (2010) built a simulation model using the supply chain characteristics reported by an

actual firm and validated the simulation results by comparing them with actual financial data

provided by the company. If verification and validation cannot be corroborated, then the

simulation is nothing more than some arbitrary and meaningless set of procedures. On the

other hand, if they can be adequately established, the simulation process is finally about to take

place.

Input issues

Page 17 of 30
 The simulation model is set to go, but what will it run? The inputs still need to be

addressed. This is an extremely important step; if not carefully carried out, the research runs

the distinct possibility of “garbage in, garbage out.”

One issue is data collection. Since the simulation is intended to replicate a system, it is

imperative to collect data on that process, if possible. Standard data collection methods can be

used to do this, ranging from the use of historical data to time and motion studies and the like.

If the simulation deals with shipping dock utilization, for example, data on vehicle inter-arrival

times, loading times, etc. will need to be collected, some of which might come from company

archival sources while some may require direct observation.

Once the data has been pulled together, it needs to be incorporated into the simulation.

Some of the data might be used to set system constraints. This typically does not require much

effort as system resources are rather straightforward to identify. For example, in the dock

utilization case, the number of docks at the facility may represent a constraint. Indeed, it may

either be set at a fixed level if the goal is to examine utilization within the current system or be

allowed to vary if the goal is to determine the impact of changes to the facility itself. In either

case though, the determination of the number of docks is fairly simple as system constraints are

usually deterministic.

The more challenging task involves stochastic inputs. In most simulations, much of the

data used as input will be probabilistic in nature. In the dock utilization example, the vehicle

interarrival times will likely be random, the amount of product to be shipped will be as well. In

this case, the data will need to be appropriately converted. There are three common methods of

using data: theoretical distributions, empirical distributions, and the data themselves.

Page 18 of 30
 A theoretical distribution involves fitting a known distribution to the data and then

sampling from this fitted distribution as input to the simulation model. Take the vehicle

interarrival times mentioned above: when aggregated, this data might have the general shape of

an exponential distribution. The amount of product to be shipped may have a very different

shape, perhaps looking a bit like a normal distribution; however, since the normal distribution

can take on negative values, and the amount of product shipped must be at least zero (ignore the

possibility of returns or, better yet, consider the returns process separately), a different

distribution such as the lognormal might be better suited. The point is that the researcher may

need to consider multiple distributions.

Along with a theoretical distribution come its parameters. An exponential distribution is

fully defined by one parameter, its mean (the inverse of its rate), while a normal distribution is

defined by two, its mean and standard deviation. As part of the distribution fitting process,

both the theoretical distribution and its parameters must be set. Suffice it to say that various

statistical procedures exist to help ensure the goodness-of-fit between the aggregated data and

the theoretical distribution. Indeed, many software packages, including the major

simulation-specific packages, offer comprehensive data fitting abilities involving a wide range

of theoretical distributions and goodness-of-fit tests.

Try as one might, sometimes no theoretical distribution will work. The aggregate data is

so oddly shaped that nothing fits. In cases like these, an empirical distribution is necessary. An

empirical distribution is merely a distribution that is constructed directly from the aggregate

data. For example, if 45% of the interarrival times are 10-20 minutes, 10% are 20-30 minutes,

and 45% are 30-40 minutes, no theoretical distribution may adequately fit; an empirical

Page 19 of 30
distribution taking on these proportions can be used instead.

Comparing the use of theoretical and empirical distributions, there are pros and cons to

each. Regarding theoretical distributions, there is a vast array of distributions out there to

choose from (anywhere from well-known continuous ones like exponential, normal, and

uniform, to lesser known ones like Pearson Type V, Dagum, and Rayleigh, plus all of the

discrete distributions [Poisson, negative binomial, etc.]), though they do not cover all possible

shapes and situations. A good example of this latter possibility is found in transit times: most

transit times between two locations are typically within a close window. If there are a

significant number of delays, however, the distribution tail could be long and even have a

sizeable bump in it. Unfortunately, theoretical distributions do not usually look like this. This

big drawback for theoretical distributions is a big advantage for empirical distributions. On the

contrary, the big disadvantage of an empirical distribution is that it is entirely data driven and

dependent upon the sample collected. Thus, while it might reflect the data collected, it might

not accurately reflect the underlying process. A fitted theoretical distribution, on the other

hand, relies upon statistics to recognize that the distribution is an adequate fit, not an exact fit.

In general, a rule of thumb is to rely upon theoretical distributions wherever possible, but to use

empirical distributions when the data is extremely odd-shaped.

The third method is to directly use the data values themselves. Interarrival times

throughout the day can be directly input into the simulation in the same sequence that they were

collected. Of course, this removes the randomness from the input process, but this approach

can be very useful for verification and validation purposes. As the process has been observed

and the data collected, plugging the data directly into the simulation should lead to outputs

Page 20 of 30
identical to those observed in the actual process. If not, then either the model coding was not

performed correctly (verification is lacking) or the model mapping was not done correctly

(validation is lacking).

In some instances, little data is available. Even in these situations, input distributions can

be created. For example, if a range can be identified (i.e, minimum and maximum values), then

a uniform distribution can be used to provide at least some randomness. If the most frequently

occurring value can also be identified (i.e., the mode) along with the range, then a triangular

distribution can be used. A beta distribution is another useful option when data is lacking; only

the minimum and maximum values, along with the mode and the mean are needed. The point

is that even if only a modicum of data exists simulation can still be employed. In fact, in many

cases no data exists whatsoever and yet simulation remains the preferred method. In order to

provide a comprehensive data input, factorial experimental designs are generally employed to

generate input data sets (Cattani et al. 2011; Pagell and Melnyk 2004; Nachtmann et al. 2010;

Yang et al. 2010). Since simulation can be used to answer “what-if” questions, it is often the

case that “what-if” involves the consideration of different, unobserved inputs. The lack of data

does not mean that simulation cannot be used; it really boils down to whether simulation

validity can be established without actual data.

Output issues

Now that the simulation model is built and the input parameters have been set, the

simulation can be run and, as a consequence, the output generated. However, the work is not

complete; indeed, it is just beginning. All of the tasks performed up to this point are

meaningless if the output is not properly analyzed. The key to output analysis is the

Page 21 of 30
recognition that a single simulation run is merely one realization of the underlying process. In

as much as one could not claim that the transit time between two locations can be determined

by only making one actual shipment (there could be weather, congestion, etc.), one also cannot

claim that the transit time can be determined by only making one simulated shipment.

Simulation outputs, especially stochastic ones, must be statistically analyzed to account

for randomness. This is typically performed by running the same simulation multiple times

using different random realizations of the input parameters and then making statistical

inferences based on the results. Note that this is no different than collecting actual data based

on multiple observations and then making statistical inferences based on them. Simulation is

merely another method for generating data.

The output issues that arise with simulation generally revolve around how the simulation

model is run. Since a simulation model is usually stochastic and dynamic, the state of the

system randomly varies over time. Therefore, it is often the case that data collected from a

simulation run will not be independent of time, so standard statistical techniques that assume

independence are not appropriate. In an inventory system for example, the goal may be to

identify the average on-hand inventory level of a particular reorder policy, but the inventory

level at day 2 is related to the level at day 1, making the output analysis more difficult to

perform. There are three standard approaches to dealing with this problem.

The first approach is to make multiple runs of the simulation using the same input

parameters (Ross et al. 2009; Wan and Evers 2011). If each run is independent of one another

(i.e., the random realizations of the inputs within the simulation are unrelated across runs), then

the observations across the runs are independent as well and standard statistical methods that

Page 22 of 30
assume independence can be used. This is referred to as the replication approach. There are

two main factors that must be accounted for: 1) the number of replications, and 2) the length of

each replication.

With regard to the number of replications, this is based on standard statistics: the more

replications, the more precision in terms of statistical inferences. With regard to the length of

each replication, the primary concern revolves around the simulation starting point. Note that

the starting point for a simulation is often quite arbitrary. For example, in an inventory system,

the simulation could start with the on-hand inventory level at zero, at the reorder point, at the

reorder point plus order quantity, at two times the order quantity, or at any other random point.

As the process evolves over time, the impact of the starting point will fade away. Once this

opening period has passed, the process has reached a level of stability termed steady-state. The

data in this opening period is commonly referred to as initial transient data and is discarded.

All data generated after this can be collected as output data and analyzed accordingly. Thus,

the primary concern with regard to the length of each replication is the elimination of the

transient data and its effects.

The second approach is to make one long run in steady-state and then, after deleting the

initial transient data, average adjacent outputs until any autocorrelation has been eliminated

(Feinberg et al. 1992; Rabinovich and Evers 2003). For example, if on-hand inventory levels

are recorded at the end of each day, every two days of data are combined and their means

calculated. If correlation still exists, then every three days of data are combined and their

means calculated. If correlation still exists, then every four days of data are combined and so

on. At some point, correlation will be mitigated and the means can be analyzed using standard

Page 23 of 30
statistical methods. The combining of data is referred to as batching and this technique is

called batch means. When using this method, a couple of issues arise: 1) the number of

observations per batch, 2) the length of the run, and 3) the smoothing of variation. While the

first two issues have already been alluded to, the third one bears further mention. One

drawback of batch means is that taking an average of individual observations dampens the

variability witnessed within the system; if the concern is understanding the underlying

variability of the process, batch means is usually not a preferred method (on the other hand, the

replication approach can better account for the dispersion).

After either of the first two approaches is performed, standard statistical techniques that

assume independence can be applied and inferences made. The inferences made, or statistical

tests employed, should address the questions posed, or initial hypotheses generated, at the

beginning of the research project and thus complete the analysis. Of course, sensitivity

analysis may be undertaken as a result of the findings from the initial output analysis, but this

will proceed in a fashion identical to that outlined above.

The third approach is to simply analyze the simulation output of a long steady-state run

using appropriate methods after deleting the initial transient data. By doing so, the hypotheses

can be directly tested without performing batch means or making multiple replications.

Though the output might be highly correlated, advanced econometric techniques exist to

account for autocorrelation. Perhaps the biggest concern with this approach is that many

researchers may not be familiar with using statistical techniques that do not assume

independence.

THE FUTURE OF SIMULATION

Page 24 of 30
 Practically everything that has been presented up to this point regarding simulation has

been said before (Law and Kelton 2001). The fundamentals have been around for many years

and those who conduct research with the assistance of simulation know it. What has changed

in the past twenty years or so is the modeling capabilities. As mentioned earlier,

simulation-specific software packages now exist that make modeling very easy. Consequently,

the use of simulation is on the rise.

In the past, a researcher needed programming ability (and desire) to build a simulation

model—not anymore! Drop-and-drag packages like Arena and Simio have completely

changed the way models are developed, and a few hours spent learning the tool is all that is

necessary to build an extraordinarily representative model. Furthermore, input and output

processors are readily available with most of these packages to assist in finding input

distributions and analyzing output data. In some ways, simulation has become more of a black

box then every before as the user does not need to know all of the intricate details. And therein

lays the problem.

While the technical tasks of building and running the model have been significantly

simplified, the underlying cautions remain. Those using simulation must recognize, for

example, that the output is not the answer—it must be appropriately analyzed—or that

simulation is not the right tool for every situation—it must make sense to employ it. As

simulation becomes easier to do, it also becomes easier to do incorrectly. This admonition is no

different than for those employing high level statistical analyses software packages. It is easy

to run almost any arbitrary statistical model on a set of data, it is also easy to violate the

assumptions underlying the statistical model without even knowing it!

Page 25 of 30
 Assuming that the simulation is carried out correctly, the prospects are endless. For

example, advances are being made to help researchers identify near optimal solutions using

simulation (Alrefaei and Andradottir 1999). Likewise, the generation of correlated input data

is now easier then every before (Cezik and L’Ecuyer 2008). For practitioners in particular, the

graphical capabilities of simulation are amazing. The newest versions of simulation-specific

software packages allow the user to represent the system with realistic-looking pictures instead

of sleepy-looking diagrams. This capability makes presentations even more powerful when

selling the results of the simulation analysis. Moreover, as computer processing power

continues to improve, the size and complexity of models that the simulation-specific packages

can handle will continue to grow. And as previously indicated, the ease of use of these

packages continues to dramatically improve.

As explained earlier, in isolation simulation represents one of a number of different

methods for systems analysis. These various methods, however, can be quite complimentary

and they can be used in conjunction with one another to tackle a problem from several

perspectives. Indeed, it is often the case that analytical solutions are found and then

simulations run to cross-validate the results. There is no reason to expect that simulation

modeling could not be combined with other forms of analysis (say, survey or archival data

methodologies) to provide an even richer understanding of the system. This triangulation

approach is certainly something to be encouraged in the future as all of the various systems

analysis techniques (not just simulation) continue to become easier to use.

The purpose of research is to address significant and substantive questions. This paper

presented the details of one such methodology to assist researchers in their attempt to address

Page 26 of 30
such questions. Simulation was discussed as one method for addressing questions related to

systems analysis, specifically supply chain management in business research. It was suggested

that simulation is a very useful tool and is not that hard to use anymore. However, it was also

noted that with this newfound ease it can also be easy to take shortcuts that lead to a

fundamentally flawed analysis. Ultimately, like any research method, simulation requires a

great deal of diligence to ensure that it is done correctly. If this effort is taken seriously, the

extent of the research contribution is limited only by the extent of the research question.

Page 27 of 30
REFERENCES

Alrefaei, M. H., and Andradottir, S. 1999. “A Simulated Annealing Algorithm with Constant
Temperature for Discrete Stochastic Optimization.” Management Science 45(5):748–64.

Angulo, A., Nachtmann, H., and Waller, M. A. 2004. “Supply Chain Information Sharing in A
Vendor Managed Inventory Partnership.” Journal of Business Logistics 25(1):101–20.

Bowersox, D. J., and Closs, D. J. 1989. “Simulation in Logistics: A Review of Present Practice
and A Look to the Future.” Journal of Business Logistics 10(1):133–48.

Carter, C. R., and Jennings, M. M. 2004. “The Role of Purchasing in Corporate Social
Reponsibility: A Structural Equation Analysis.” Journal of Business Logistics 25(1):145–
86.

Cattani, K. D., Jacob, F. R., and Schoenfelder, J. 2011. “Common Inventory Modeling
Assumptions that Fall Short: Arborescent Networks, Poisson Demand, and Single-Echelon
Approximations.” Journal of Operations Management 29(5):488–99.

Cezik, M. T., and L’Ecuyer, P. 2008. “Staffing Multiskill Call Centers via Linear Programming
and Simulation.” Management Science 54(2):310–23.

Closs, D. J., Jacobs, M. A., Swink, M., and Webb, G. S. 2008. “Toward A Theory of
Competencies for The Management of Product Complexity: Six Case Studies.” Journal of
Operations Management 26(5):590–610.

Coyle, J. J., Langley, C. J., Gibson, B. J., Novack, R. A., and Bardi, E. J. 2008. Supply Chain
Management: A Logistics Perspective. 8th ed., South-Western.

Croson, R., and Donohue, K. 2006. “Behavioral Causes of the Bullwhip Effect and the
Observed Value of Inventory Information.” Management Science 52(3):323–36.

Daugherty, P. J., Autry, C. W., and Ellinger, A. E. 2001. “Reverse Logistics: The Relationship
Between Resource Commitment and Program Performance.” Journal of Business Logistics
22(1):107–23.

DeCroix, G. A., and Zipkin, P. H. 2005. “Inventory Management for An Assembly System
with Product or Component Returns.” Management Science 51(8):1250–65.

Dhouib, K., Gharbi, A., and Ayed, S. 2008. “Availability and Throughput of Unreliable,
Unbuffered Production Lines with Non-Homogeneous Deterministic Processing Times.”

Page 28 of 30
 International Journal of Production Research 46(20):5651–77.

Evers, P. T. 1994. “The Occurrence of Statistical Economies of Scale in Intermodal

Transportation.” Transportation Journal 33(4): 51 – 63.

Evers, P. T. 1996. “The Impact of Transshipments on Safety Stock Requirements.” Journal of

Business Logistics 17(1):109–33.

Evers, P. T., and Beier, F.J. 1993. “The Portfolio Effect and Multiple Consolidation Points: A
Critical Assessment of the Square Root Law.” Journal of Business Logistics 14(2):109–25.

Feinberg, F. M., Kahn, B. E., and McAlister, L. 1992. “Market Share Response When
Consumers Seek Variety.” Journal of Marketing Research 29(2):227–37.

Forrester, J.W. 1958. “Industrial Dynamics: A Major Breakthrough for Decision Makers.”
Harvard Business Review 36(4):37–66.

Klassen, R. D., and Menor, L. J. 2007. “The Process Management Triangle: An Empirical
Investigation of Process Trade-Offs.” Journal of Operations Management 25(5):1015–34.

Law, A.M., and Kelton, W. D. 2000. Simulation Modeling and Analysis. 3rd ed., McGraw-Hill.

Lee, H.L., Padmanabhan, V., and Whang, S. 1997. “Information Distortion in a Supply Chain:
The Bullwhip Effect.” Management Science 43(4):546–58.

Mentzer, J. T. 1989. “Symposium Feature: Approaches to Logistics Modeling.” Journal of

Business Logistics 10(1):104–05.

Meredith, J. 1998. “Building Operations Management Theory Through Case and Field
Research.” Journal of Operations Management 16(4):441–54.

Nachtmann, H., Waller, M. A., and Rieske, D. W. 2010. “The Impact of Point-Of-Sale Data
Inaccuracy and Inventory Record Data Errors.” Journal of Business Logistics 31(1):149–
58.

Pagell, M., and Melnyk, S. A. 2004. “Assessing the Impact of Alternative Manufacturing
Layouts in A Service Setting.” Journal of Operations Management 22(4):413–29.

Rabinovich, E., and Evers, P. 2003. “Product Fulfillment in Supply Chains Supporting
Internet-Retailing Operations.” Journal of Business Logistics 24(2):205–236.

Ross, A. D., Twede, D., Clarke, R. H., and Ryan, M. 2009. “A Framework for Developing
Implementation Strategies for A Radio Frequency Identification (RFID) System in A
Distribution Center Environment.” Journal of Business Logistics 30(1):157–83.

Page 29 of 30
Stank, T. P., and Roath, A. S. 1998. “Some Propositions on Intermodal Transportation and
Logistics Facility Development: Shipper’ Perspective.” Transportation Journal 37(3):13–
24.

Sterman, J. 1989. “Modeling Managerial Behavior: Misperceptions of Feedback in a Dynamic

Decision-Making Experiment.” Management Science 35(3):321–39.

Torres, O. C., and Maltz, A. B. 2010. “Understanding the Financial Consequences of the
Bullwhip Effect in A Multi-Echelon Supply Chain.” Journal of Business Logistics 31(1): 23
–41.

Vashi, V. H., Bienstock, C. C., and Mentzer, J. T. 1995. “The Use of Response Surface
Methodology to Optimize Logistics Simulation Models.” Journal of Business Logistics
16(2): 197–225.

Waller, M. A., William, B. D., Tangari, A., and Burton, S. 2010. “Marketing at the retail shelf:
An Examination of Moderating Effects of Logistics on SKU Market Share.” Journal of the
Academy of Marketing Science 38(1): 105–17.

Waller, M. A., William, B. D., and Eroglu, C. 2008. “Hidden Effects of Variable Order Review
Intervals in Inventory Control.” International Journal of Physical Distribution & Logistics
Management 38(3): 244–58.

Wan, X., and Evers, P. T. 2011. “Supply Chain Networks with Multiple Retailers: A Test of the
Emerging Theory on Inventories, Stockouts, and Bullships.” Journal of Business Logistics
32(1): 27–39.

Wan, X., Evers, P. T., and Dresner, M. E. forthcoming. “Too Much of A Good Thing: The
Impact of Product Variety on Operations and Sales Performance.” Journal of Operations
Management.

Yan, T., Rabinovich, E., Dooley, K., and Evers, P. T. 2010 “Managing Backlog Variation in
Order Fulfillment: The Case of Internet Retailers.” International Journal of Production
Economics 128 (1): 261 - 68.

Yang, K. K., Balakrishnan, J., and Cheng, C. H. 2010. “An Analysis of Factors Affecting Cross
Docking Operations.” Journal of Business Logistics 31(1):121–48.

Zinn, W., Levy, M., and Bowersox, D. J. 1989. “Measuring the Effect of Inventory
Centralization/Decentralization on Aggregate Safety Stock: The 'Square Root Law'
Revisited.” Journal of Business Logistics 10 (1): 1-14.

Page 30 of 30
FIGURES

Figure 1: Methods for Systems Analysis

System Analysis

Analyst Analyst
Uncontrolled Controlled

Perceptual Factual Behavioral Logical

Analysis Analysis Analysis Analysis

Panel Archival Objective Analytical

Survey Games Experiment Solution Simulation
Discussion Data Observation

Time Randomness Change

Static Dynamic Deterministic Stochastic Discrete Continuous

Note: The logical analysis branch is based on Law and Kelton (2000).
 SHORT BIOGRAPHIES

Philip T. Evers (Ph.D. University of Minnesota) is an Associate Professor of Logistics

Management at the Robert H. Smith School of Business, University of Maryland. His research
interests include inventory management, transportation operations management, and
intermodal transportation issues. He has published research in a variety of journals including
the Journal of Business Logistics, Transportation Journal, Journal of Operations Management,
European Journal of Operational Research, and Computers and Operations Research.

Xiang (Sean) Wan (Ph.D. University of Maryland) is an assistant professor of

management at Marquette University's College of Business Administration. His research
interests include inventory management, transportation management, supply chain
coordination, and product & service variety management in supply chains. He has published
research in Journal of Operations Management, Journal of Business Logistics, Transportation
Research (E): Logistics and Transportation Review, Business Research, and China Market
Modern Business.

View publication stats