You are on page 1of 41

# Computer Science, Informatik 4 Communication and Distributed Systems

Simulation
“Discrete-Event System Simulation”
Dr. Mesut Güneş

Computer Science, Informatik 4 Communication and Distributed Systems

Chapter 4
Statistical Models in Simulation

Computer Science, Informatik 4 Communication and Distributed Systems

Purpose & Overview
The world the model-builder sees is probabilistic rather than deterministic.
• Some statistical model might well describe the variations.

An appropriate model can be developed by sampling the phenomenon of interest:
• Select a known distribution through educated guesses • Make estimate of the parameters • Test for goodness of fit

In this chapter:
• Review several important probability distributions • Present some typical application of these models

Chapter 4. Statistical Models in Simulation

3

Dr. Mesut Güneş

Computer Science, Informatik 4 Communication and Distributed Systems

Review of Terminology and Concepts In this section, we will review the following concepts:
• • • • Discrete random variables Continuous random variables Cumulative distribution function Expectation

Chapter 4. Statistical Models in Simulation

4

Dr. Mesut Güneş

Computer Science.Let X be the number of jobs arriving each week at a job shop.2.2.…. Informatik 4 Communication and Distributed Systems Discrete Random Variables X is a discrete random variable if the number of possible values of X is finite. i = 1. ∑ ∞ i =1 p( xi ) = 1 • The collection of pairs [xi. p(xi)]. Statistical Models in Simulation 5 Dr.2.…} p(xi) = probability the random variable X is xi . … must satisfy: 1. p(xi) = P(X = xi) • p(xi). . is called the probability distribution of X.1. for all i 2. Chapter 4. Example: Consider jobs arriving at a job shop. and p(xi) is called the probability mass function (pmf) of X. Mesut Güneş . or countable infinite. Rx = possible values of X (range space of X) = {0. i = 1. p( xi ) ≥ 0.

RX ∫ f ( x)dx = 1 3. satisfies: 1. Mesut Güneş . f ( x) = 0. Statistical Models in Simulation 6 Dr. for all x in R X 2. b] is given by: P(a ≤ X ≤ b) = ∫ f ( x)dx a b f(x) is called the probability density function (pdf) of X. P ( a ≤ X ≤ b ) = P ( a < X ≤ b ) = P ( a ≤ X < b ) = P ( a < X < b ) Chapter 4. Informatik 4 Communication and Distributed Systems Continuous Random Variables X is a continuous random variable if its range space Rx is an interval or a collection of intervals. f ( x) ≥ 0 . The probability that X lies in the interval [a. P ( X = x0 ) = 0. if x is not in RX Properties 1. because ∫ f ( x)dx = 0 x0 x0 2.Computer Science.

a continuous random variable with pdf: ⎧1 −x / 2 ⎪ e . Informatik 4 Communication and Distributed Systems Continuous Random Variables Example: Life of an inspection device is given by X.14 2 2 Chapter 4. Statistical Models in Simulation 7 Dr.Computer Science. otherwise ⎩ Lifetime in Year • X has an exponential distribution with mean 2 years • Probability that the device’s life is between 2 and 3 years is: 1 3 −x / 2 P(2 ≤ x ≤ 3) = ∫ e dx = 0. x≥0 f ( x) = ⎨ 2 ⎪0. Mesut Güneş .

Informatik 4 Communication and Distributed Systems Cumulative Distribution Function Cumulative Distribution Function (cdf) is denoted by F(x). Mesut Güneş . then F ( x) = ∑ p ( xi ) xi ≤ x • If X is continuous. for all a ≤ b Chapter 4. then F (a ) ≤ F (b) 2. lim F ( x) = 1 x →∞ 3. lim F ( x) = 0 x → −∞ All probability question about X can be answered in terms of the cdf: P (a ≤ X ≤ b) = F (b) − F (a ). then F ( x) = ∫ f (t )dt −∞ x Properties 1. F is nondecreasing function.Computer Science. Statistical Models in Simulation 8 Dr. where F(x) = P(X ≤ x) • If X is discrete. If a ≤ b.

632 • The probability that it lasts between 2 and 3 years: P (2 ≤ X ≤ 3) = F (3) − F (2) = (1 − e − ( 3 / 2 ) ) − (1 − e −1 ) = 0. Mesut Güneş .Computer Science. Statistical Models in Simulation 9 Dr.145 Chapter 4. Informatik 4 Communication and Distributed Systems Cumulative Distribution Function Example: An inspection device has cdf: 1 x −t / 2 F ( x) = ∫ e dt = 1 − e − x / 2 2 0 • The probability that the device lasts for less than 2 years: P(0 ≤ X ≤ 2) = F (2) − F (0) = F (2) = 1 − e −1 = 0.

Informatik 4 Communication and Distributed Systems Expectation The expected value of X is denoted by E(X) • • • • If X is discrete If X is continuous E ( x) = ∑ xi p ( xi ) E ( x) = ∫ x ⋅ f ( x)dx −∞ all i ∞ a. m.k. Mesut Güneş Chapter 4. µ. or the 1st moment of X A measure of the central tendency The variance of X is denoted by V(X) or var(X) or σ2 • • • Definition: V(X) = E( (X – E[X])2 ) Also. V(X) = E(X2) – ( E(x) )2 A measure of the spread or variation of the possible values of X around the mean The standard deviation of X is denoted by σ • Definition: σ = V (x) • Expressed in the same units as the mean 10 Dr.Computer Science.a the mean. Statistical Models in Simulation .

Mesut Güneş .Computer Science. we first compute E(X2): 1 ∞ 2 −x / 2 2 e − x / 2 + ∞ e − x / 2 dx = 8 E ( X ) = ∫ x e dx = − x ∫0 2 0 0 2 ∞ Hence. Statistical Models in Simulation 11 Dr. Informatik 4 Communication and Distributed Systems Expectations Example: The mean of life of the previous inspection device is: ∞ 1 −x / 2 −x / 2 E ( X ) = ∫ xe dx = − xe + ∫ e − x / 2 dx = 2 0 2 0 0 ∞ ∞ To compute variance of X. the variance and standard deviation of the device’s life are: 2 V (X ) = 8 − 2 = 4 σ = V (X ) = 2 Chapter 4.

Computer Science. Informatik 4 Communication and Distributed Systems Expectations ∞ 1 ∞ −x / 2 −x / 2 E ( X ) = ∫ xe dx = − xe + ∫ e − x / 2 dx = 2 0 2 0 0 ∞ 1 ∞ −x / 2 + ∫ e − x / 2 dx = 2 E ( X ) = ∫ xe − x / 2 dx = − xe 0 2 0 0 Partial Integration ∞ ∞ ∫ u ( x)v' ( x)dx = u ( x)v( x) − ∫ u ' ( x)v( x)dx Set u ( x) = x v' ( x) = e − x / 2 ⇒ u ' ( x) = 1 v( x) = −2e − x / 2 1 ∞ −x/ 2 1 −x/ 2 ∞ E ( X ) = ∫ xe dx = ( x ⋅ (−2e ) − ∫ 1 ⋅(−2e − x / 2 )dx) 0 2 0 2 0 Chapter 4. Statistical Models in Simulation 12 Dr. Mesut Güneş ∞ .

The areas include: • • • • Queueing systems Inventory and supply-chain systems Reliability and maintainability Limited data Chapter 4. Statistical Models in Simulation 13 Dr. statistical models appropriate to some application areas are presented. Mesut Güneş .Computer Science. Informatik 4 Communication and Distributed Systems Useful Statistical Models In this section.

Sample statistical models for interarrival or service time distribution: • Exponential distribution: if service times are completely random • Normal distribution: fairly constant but with some random variability (either positive or negative) • Truncated normal distribution: similar to normal distribution but with restricted value.) Chapter 4. Informatik 4 Communication and Distributed Systems Useful models – Queueing Systems In a queueing system. • Gamma and Weibull distribution: more general than exponential (involving location of the modes of pdf’s and the shapes of tails. Mesut Güneş . interarrival and service-time patterns can be probabilistic. Statistical Models in Simulation 14 Dr.Computer Science.

there are at least three random variables: • The number of units demanded per order or per time period • The time between demands • The lead time = Time between placing an order and the receipt of that order Sample statistical models for lead time distribution: • Gamma Sample statistical models for demand distribution: • Poisson: simple and extensively tabulated. Mesut Güneş .Computer Science. • Negative binomial distribution: longer tail than Poisson (more large demands). Chapter 4. Informatik 4 Communication and Distributed Systems Useful models – Inventory and supply chain In realistic inventory and supply-chain systems. Statistical Models in Simulation 15 Dr. • Geometric: special case of negative binomial given at least one demand has occurred.

Mesut Güneş . Statistical Models in Simulation 16 Dr. Informatik 4 Communication and Distributed Systems Useful models – Reliability and maintainability Time to failure (TTF) • Exponential: failures are random • Gamma: for standby redundancy where each component has an exponential TTF • Weibull: failure is due to the most serious of a large number of defects in a system of components • Normal: failures are due to wear Chapter 4.Computer Science.

Mesut Güneş .Computer Science. Informatik 4 Communication and Distributed Systems Useful models – Other areas For cases with limited data. Statistical Models in Simulation 17 Dr. some useful distributions are: • Uniform • Triangular • Beta Other distribution: • Bernoulli • Binomial • Hyperexponential Chapter 4.

Mesut Güneş .Computer Science. we will learn about: • • • • Bernoulli trials and Bernoulli distribution Binomial distribution Geometric and negative binomial distribution Poisson distribution Chapter 4. Statistical Models in Simulation 18 Dr. Informatik 4 Communication and Distributed Systems Discrete Distributions Discrete random variables are used to describe random phenomena in which only integer values can occur. In this section.

x2. n Bernoulli process: • The n Bernoulli trials where trails are independent: p(x1. Statistical Models in Simulation 19 Dr.Xj = 0 if the j-th experiment is a failure • The Bernoulli distribution (one trial): xj =1 ⎧ p. . q := 1 − p.…. Mesut Güneş ..Xj = 1 if the j-th experiment is a success . each can be a success or a failure. x j = 0 ⎩ • where E(Xj) = p and V(Xj) = p(1-p) = pq j = 1..2. Informatik 4 Communication and Distributed Systems Bernoulli Trials and Bernoulli Distribution Bernoulli Trials: • Consider an experiment consisting of n trials. xn) = p1(x1)p2(x2) … pn(xn) Chapter 4..Computer Science. p j ( x j ) = p( x j ) = ⎨ ..

X.1. n p( x) = ⎨⎜ x ⎟ ⎝ ⎠ ⎪0. Statistical Models in Simulation 20 Dr... x = 0.. otherwise ⎩ The number of outcomes having the required number of successes and failures Probability that there are x successes and (n-x) failures • The mean. Mesut Güneş . E(x) = p + p + … + p = n*p • The variance. ⎧⎛ n ⎞ x n− x ⎪⎜ ⎟ p q .. Informatik 4 Communication and Distributed Systems Binomial Distribution The number of successes in n Bernoulli trials.2. has a binomial distribution.Computer Science. V(X) = pq + pq + … + pq = n*pq Chapter 4.

2. • E(x) = 1/p.1.. n p ( x) = ⎨ otherwise ⎩0... to achieve the 1st success: ⎧ q x −1 p. Informatik 4 Communication and Distributed Systems Geometric Distribution Geometric distribution • The number of Bernoulli trials. X. x = 0. Statistical Models in Simulation 21 Dr. and V(X) = q/p2 Chapter 4.Computer Science. Mesut Güneş ..

X. y = k . otherwise ⎩ ⎛ y − 1⎞ y − k k −1 p ( x) = ⎜ p ⎜ k − 1⎟ q p ⋅ { ⎟ ⎝ 44 44 k − th success 1 ⎠2 3 (k-1 ) successes • E(Y) = k/p. Informatik 4 Communication and Distributed Systems Negative Binomial Distribution Negative binomial distribution • The number of Bernoulli trials. ⎟ p ( x) = ⎨⎝ ⎠ ⎪0. then: ⎧⎛ y − 1⎞ y − k k ⎪⎜ ⎜ k − 1⎟ q p .. until the kth success • If Y is a negative binomial distribution with parameters p and k. and V(X) = kq/p2 Chapter 4. k + 2.Computer Science... Statistical Models in Simulation 22 Dr. Mesut Güneş . k + 1.

Computer Science..1. x = 0. Statistical Models in Simulation 23 Dr. • where α > 0. otherwise ⎩ • E(X) = α = V(X) F ( x) = ∑ i =0 x α i e −α i! Chapter 4.. ⎪0. Mesut Güneş .. Informatik 4 Communication and Distributed Systems Poisson Distribution Poisson distribution describes many random processes quite well and is mathematically quite simple. pdf and cdf are: ⎧α x −α ⎪ p( x) = ⎨ x! e .

18 • The probability of two or more beeps in a 1-hour period: p(2 or more) = 1 – ( p(0) + p(1) ) = 1 – F(1) = 0.677=0.Computer Science. Informatik 4 Communication and Distributed Systems Poisson Distribution Example: A computer repair person is “beeped” each time there is a call for service. p(3) = F(3) – F(2) = 0. Statistical Models in Simulation 24 Dr. • The probability of three beeps in the next hour: p(3) = 23/3! e-2 = 0.594 Chapter 4.857-0. Mesut Güneş .18 also. The number of beeps per hour ~ Poisson(α = 2 per hour).

Mesut Güneş .Computer Science. Informatik 4 Communication and Distributed Systems Continuous Distributions Continuous random variables can be used to describe random phenomena in which the variable can take on any value in some interval. the distributions studied are: • • • • • Uniform Exponential Weibull Normal Lognormal Chapter 4. Statistical Models in Simulation 25 Dr. In this section.

Statistical Models in Simulation 26 Dr. • P(x1 < X < x2) is proportional to the length of the interval [F(x2) – F(x1) = (x2-x1)/(b-a)] • E(X) = (a+b)/2 V(X) = (b-a)2/12 U(0.1) provides the means to generate random numbers. U(a. b). if its pdf and cdf are: ⎧ 1 ⎪ . ⎪x −a F ( x) = ⎨ .Computer Science. Chapter 4. a≤ x≤b f ( x) = ⎨ b − a ⎪0. from which random variates can be generated. b). Informatik 4 Communication and Distributed Systems Uniform Distribution A random variable X is uniformly distributed on the interval (a. otherwise ⎩ Properties x<a ⎧0. Mesut Güneş . a≤ x<b ⎪b − a x≥b ⎩1.

Computer Science. Mesut Güneş . x ≥ 0 ⎩ Chapter 4. ⎪ x F ( x) = ⎨ − λt − λx ⎪∫0 λe dt = 1 − e . • E(X) = 1/λ V(X) = 1/λ2 x<0 ⎧0. Statistical Models in Simulation 27 Dr. Informatik 4 Communication and Distributed Systems Exponential Distribution A random variable X is exponentially distributed with parameter λ > 0 if its pdf and cdf are: ⎧λe − λx . x ≥ 0 f ( x) = ⎨ elsewhere ⎩0.

the value of intercept on the vertical axis is λ. and to model service times that are highly variable • For several different exponential pdf’s (see figure). Chapter 4. Informatik 4 Communication and Distributed Systems Exponential Distribution • Used to model interarrival times when arrivals are completely random.Computer Science. and all pdf’s eventually intersect. Statistical Models in Simulation 28 Dr. Mesut Güneş .

Informatik 4 Communication and Distributed Systems Exponential Distribution Memoryless property • For all s and t greater or equal to 0: P(X > s+t | X > s) = P(X > t) • Example: A lamp ~ exp(λ = 1/3 per hour). . hence.The probability that the lamp lasts between 2 to 3 hours is: P(2 <= X <= 3) = F(3) – F(2) = 0.368 . Statistical Models in Simulation 29 Dr.5) = P(X > 1) = e-1/3 = 0.145 .5 | X > 2. on average.The probability that the lamp lasts longer than its mean life is: P(X > 3) = 1-(1-e-3/3) = e-1 = 0.The probability that it lasts for another hour given it is operating for 2. 1 failure per 3 hours.Computer Science.717 Chapter 4.5 hours: P(X > 3. Mesut Güneş .

Mesut Güneş . Informatik 4 Communication and Distributed Systems Exponential Distribution Memoryless property P( X > s + t ) P( X > s + t | X > s) = P( X > s) e −λ ( s +t ) = − λs e = e − λt = P( X > t ) Chapter 4. Statistical Models in Simulation 30 Dr.Computer Science.

otherwise ⎩ 3 parameters: • • • (−∞ < ν < ∞) Location parameter: υ.Computer Science. (β > 0) Shape parameter. Mesut Güneş . (> 0) Example: υ = 0 and α = 1: Chapter 4. Scale parameter: β . α. Informatik 4 Communication and Distributed Systems Weibull Distribution A random variable X has a Weibull distribution if its pdf has the form: ⎧ β ⎛ x −ν ⎞ β −1 ⎡ ⎛ x −ν ⎞ β ⎤ ⎪ exp⎢− ⎜ ⎟ ⎥. x ≥ ν f ( x) = ⎨α ⎜ α ⎟ α ⎠ ⎥ ⎝ ⎠ ⎢ ⎝ ⎣ ⎦ ⎪0. Statistical Models in Simulation 31 Dr.

Computer Science. Informatik 4 Communication and Distributed Systems Weibull Distribution Weibull Distribution ⎧ β ⎛ x −ν ⎞ β −1 ⎡ ⎛ x −ν ⎞ β ⎤ ⎪ exp ⎢− ⎜ ⎟ ⎥. X ~ exp(λ = 1/α) Chapter 4. Statistical Models in Simulation 32 Dr. Mesut Güneş . x ≥ ν ⎪0. υ=0 1 −x ⎧1 α ⎪ f ( x) = ⎨α exp . x ≥ ν f ( x) = ⎨α ⎜ α ⎟ ⎝ ⎠ ⎢ ⎝ α ⎠ ⎥ ⎣ ⎦ ⎪0. otherwise ⎩ For β = 1. otherwise ⎩ When β = 1.

the mean and mode are equal. Chapter 4. the pdf is symmetric about µ. Statistical Models in Simulation 33 Dr.Computer Science. Informatik 4 Communication and Distributed Systems Normal Distribution A random variable X is normally distributed if it has the pdf: ⎡ 1 ⎛ x − µ ⎞2 ⎤ 1 f ( x) = exp ⎢− ⎜ ⎟ ⎥.σ2) Special properties: • xlim f ( x) = 0. • The maximum value of the pdf occurs at x = µ. and lim f ( x) = 0 → −∞ x →∞ • f(µ-x)=f(µ+x). − ∞ < x < ∞ σ 2π ⎢ 2⎝ σ ⎠ ⎥ ⎣ ⎦ • Mean: − ∞ < µ < ∞ 2 • Variance: σ > 0 • Denoted as X ~ N(µ. Mesut Güneş .

Informatik 4 Communication and Distributed Systems Normal Distribution Evaluating the distribution: • Use numerical methods (no closed form) • Independent of µ and σ.Computer Science.1) • Transformation of variables: let Z = (X .µ) / σ. Statistical Models in Simulation 34 Dr. using the standard normal distribution: Z ~ N(0. where Φ( z ) = ∫ z −∞ 1 −t 2 / 2 e dt 2π Chapter 4. Mesut Güneş . x−µ ⎞ ⎛ F ( x ) = P ( X ≤ x ) = P⎜ Z ≤ ⎟ σ ⎠ ⎝ ( x−µ ) /σ 1 −z2 / 2 =∫ e dz −∞ 2π =∫ ( x−µ ) /σ −∞ − φ ( z )dz = Φ( xσ µ ) .

X. σ=2 • The probability that the vessel is loaded in less than 10 hours: ⎛ 10 − 12 ⎞ F (10) = Φ⎜ ⎟ = Φ(−1) = 0.1587 ⎝ 2 ⎠ . Informatik 4 Communication and Distributed Systems Normal Distribution Example: The time required to load an oceangoing vessel. Mesut Güneş . µ=12.Using the symmetry property.4). Statistical Models in Simulation 35 Dr. is distributed as N(12. Φ(1) is the complement of Φ (-1) Chapter 4.Computer Science.

σ2). Mesut Güneş .Computer Science. µ=1. Statistical Models in Simulation 36 Dr.5.1) 2 Relationship with normal distribution • When Y ~ N(µ. Informatik 4 Communication and Distributed Systems Lognormal Distribution A random variable X has a lognormal distribution if its pdf has the form: ⎧ 1 ⎡ (ln x − µ ) 2 ⎤ exp ⎢− ⎪ ⎥. then X = eY ~ lognormal(µ. otherwise ⎩ σ2=0.2. σ2) • Parameters µ and σ2 are not the mean and variance of the lognormal random variable X Chapter 4. • Mean E(X) = eµ+σ /2 2 2 • Variance V(X) = e2µ+σ /2 (eσ .1. x > 0 f ( x) = ⎨ 2π σx 2σ 2 ⎣ ⎦ ⎪0.

. t>=0} has independent increments Properties (λ t ) n − λ t P[ N (t ) = n] = e . n! for t ≥ 0 and n = 0.. t>=0} is a Poisson process with mean rate λ if: • Arrivals occur one at a time • {N(t). Informatik 4 Communication and Distributed Systems Poisson Distribution Definition: N(t) is a counting function that represents the number of events occurred in [0.. Statistical Models in Simulation 37 Dr. • Equal mean and variance: E[N(t)] = V[N(t)] = λt • Stationary increment: The number of arrivals in time s to t is also Poisson-distributed with mean λ(t-s) Chapter 4. Mesut Güneş . t>=0} has stationary increments • {N(t).1.Computer Science.t]. A counting process {N(t).2.

A2. A1. Mesut Güneş .t]. Statistical Models in Simulation 38 Dr. where Ai is the elapsed time between arrival i and arrival i+1 • The 1st arrival occurs after time t iff there are no arrivals in the interval [0. A2. …). are exponentially distributed and independent with mean 1/λ Arrival counts ~ Poisson(λ) Stationary & Independent Interarrival time ~ Exp(1/λ) Memoryless Chapter 4. hence: P(A1 > t) = P(N(t) = 0) = e-λt P(A1 <= t) = 1 – e-λt [cdf of exp(λ)] • Interarrival times. Informatik 4 Communication and Distributed Systems Poisson Distribution – Interarrival Times Consider the interarrival times of a Possion process (A1.Computer Science. ….

• N(t) = N1(t) + N2(t). Mesut Güneş . where N(t) is a Poisson processes with rates λ1 + λ2 N1(t) ~ Poisson[λ1] N2(t) ~ Poisson[λ2] Chapter 4.Computer Science. Informatik 4 Communication and Distributed Systems Poisson Distribution – Splitting and Pooling Splitting: • Suppose each event of a Poisson process can be classified as Type I. with probability p and Type II. Statistical Models in Simulation λ1 λ2 λ1 + λ2 N(t) ~ Poisson(λ1 + λ2) 39 Dr. with probability 1-p. where N1(t) and N2(t) are both Poisson processes with rates λ p and λ (1-p) N(t) ~ Poisson(λ) λ λp λ(1-p) N1(t) ~ Poisson[λp] N2(t) ~ Poisson[λ(1-p)] Pooling: • Suppose two Poisson processes are pooled together • N1(t) + N2(t) = N(t).

Mesut Güneş .Computer Science. Statistical Models in Simulation 40 Dr. • May be used when it is impossible or unnecessary to establish that a random variable has any particular parametric distribution. • Disadvantage: sample might not cover the entire range of possible values. Informatik 4 Communication and Distributed Systems Poisson Distribution – Empirical Distributions A distribution whose parameters are the observed values in a sample of data. Chapter 4. • Advantage: no assumption beyond the observed values in the sample.

and empirical distributions. Student should know: • Difference between discrete.Computer Science. In this chapter: • Reviewed several important probability distributions. e. Mesut Güneş . • Poisson process and its properties.. hypothesize a distributional form for the input data.g. Statistical Models in Simulation 41 Dr. Important task in simulation modeling is the collection and analysis of input data. Chapter 4. not deterministic. • Showed applications of the probability distributions in a simulation context. continuous. Informatik 4 Communication and Distributed Systems Summary The world that the simulation analyst sees is probabilistic.