© 2008 Winton

Distributions
© 2008 Winton
2

Recap
• An integrable function f : R → [0,1] such that ∫Rf(x)dx = 1 is
called a probability density function (pdf)
• The distribution function for the pdf is given by
x
F(x) = ∫ f(z)dz
-∞

and corresponds to the cumulative distribution function for the

discrete case
• Sampling from the distribution corresponds to solving the equation
rsample

x= ∫ f(z)dz
-∞
for rsample given random probability values 0 ≤ x ≤ 1
© 2008 Winton
3

Uniform Distribution
probability density function
(area under the curve = 1)
p(x)

1
(b-a)

rsample

• The pdf for values uniformly distributed across [a,b] is given by

1
f(x) =
(b-a)
© 2008 Winton
4

Sampling from the Uniform

Distribution
• (pseudo)random numbers x drawn from [0,1] distribute uniformly
across the unit interval, so it is evident that the corresponding values
rsample = a + x(b-a) rsample slope = (b-a)
will distribute uniformly across [a,b]
• Directly solving b

rsample

x= ∫ f(z)dz
-∞
a
x
for rsample as per 0 1
rsample rsample
1 z ⎤ rsample a
x= ∫
a
b-a
dz =
b - a ⎥⎦ a
=
b-a
-
b-a

also yields rsample = a + x(b-a) (exactly as it should!)

© 2008 Winton
5

Mean and Variance for the Uniform

Distribution
• The mean μ of the uniform distribution is given by
b
⎛ 1 ⎞ b 2
− a 2
1 b+a
μ = E(X) = ∫ z ⎜ ⎟ dz = = (midpoint of [a, b] )
a ⎝ b-a ⎠ 2 b-a 2
z f(z) dz

• The standard deviation σ of the uniform distribution is obtained from the

variance σ2 where
b 2
⎛ b+a⎞ ⎛ 1 ⎞ (b - a) 2
σ 2 = E((X - μ) 2 ) = ∫ ⎜z - ⎟ ⎜ ⎟ dz = (with some work)
a ⎝ 2 ⎠ ⎝ b-a ⎠ 12
(z-μ)2 f(z) dz
© 2008 Winton
6

Normal Distribution
• For a finite population the mean (m) and standard deviation (s)
provide a measure of average value and degree of variation from the
average value
• If random samples of size n are drawn from the population, then it
can be shown (the Central Limit Theorem) that the distribution of the
sample means approximates that of a distribution with
mean: μ = m and σ.
s
standard deviation: σ =
n
pdf:
(x −μ) 2
1 −
f(x) =
2
e 2σ
σ 2π
which is called the Normal Distribution
• The pdf is characterized by its "bell-shaped" curve, typical of
phenomena that distribute symmetrically around the mean value in
decreasing quantity as one moves away from the mean
© 2008 Winton
7

Empirical Rule for Normal

Distribution
• The "empirical rule" is that
– approximately 68% of sample values are in the interval [μ-σ,μ+σ]
– approximately 95% are in the interval [μ-2σ,μ+2σ]
– almost all are in the interval [μ-3σ,μ+3σ]
• This says that if n is large enough, then a sample mean for
the population is accurate with a high degree of
confidence, since σ decreases with n
– What constitutes "large enough" is largely a function of the
underlying population distribution
– The Central Limit Theorem assumes that the samples of size n
which are used to produce sample means are drawn in a random
fashion
© 2008 Winton
8

Measurements Found to Distribute

Normally
• Hogg and Craig (Introduction to Mathematical Statistics) cites as distributing normally
such disparate phenomena as
– The diameter of the hole made by a drill press
– The score on a test
– The yield of grain for a plot of ground
– The length of a newborn child
• The assumption that grades on a test distribute normally is the basis for so-called
"curving" of grades
– This assumes some underlying random phenomena controls the measure given by a test
• e.g., genetic selection
• The practice sometimes is used to assign grades of A,B,C,D,F based on how many
"standard deviations" separates a percentile score from the mean
– If the mean score is 77.5, and the standard deviation is 8, then the curve of the class scores
would be given by
• A: 94 and up (2.5%)
• B: 86-93 (13.5%)
• C: 70-85 (68%)
• D: 62-69 (13.5%)
• F: otherwise (2.5%)
– Most people "pass", but A's are hard to get
• This could be pretty distressing if the mean is 95 and the standard deviation is 2 (i.e., 90 is an F)
© 2008 Winton
9

Normal Distribution: Example Plot

• Plot with with μ = 30 and σ = 10
– The distribution is completely determined by knowing the value of μ and σ

f(x)

μ x
σ
© 2008 Winton
10

Standard Normal Distribution

• The standard normal distribution is given by μ = 0 and σ = 1, in which case the
pdf becomes 2
1 − x2
f(x) = e
2π

nsample

nsample z2
1 −
x= ∫
−∞ 2π
e 2
dz
© 2008 Winton
11

Sampling from the Normal

Distribution
• The linear relationship
nsample
rsample = μ + σ⋅nsample
between a normal distribution and
the standard normal distribution
means it is sufficient to sample
from the standard normal
distribution
• There is no "closed-form formula"
for nsample, so approximation 0 x
techniques for inverting the .5 1
cumulative distribution function or
a method such as accept-reject has
to be used
substitute z=µ+σ·t
dz = σdt

rsample (z -μ ) 2 μ + σ⋅nsample (μ + σt -μ ) 2 μ + σ⋅nsample ( t −μ ) 2

1 - 1 1
∫
−∞ σ 2π
⋅e 2
dz = ∫
-∞ σ 2π
⋅e 2σ 2
σdt = ∫
-∞
2π
⋅e 2
dt
© 2008 Winton
12

Exponential Distribution
• The exponential distribution arises in connection with Poisson
processes
– A Poisson process is one exhibiting a random arrival pattern in the
following sense:
• For a small time interval Δt, the probability of an arrival during Δt is λΔt,
where λ = the mean arrival rate
• The probability of more than one arrival during Δt is negligible
• Interarrival times are independent of each other.
– This is a kind of "stochastic" process, one for which events occur in a
random fashion
• Under these assumptions, it can be shown that the pdf for the
distribution of interarrival times is given by
f(x) = λe -λ x
which is the exponential distribution
© 2008 Winton
13

Properties of the Exponential

Distribution
• If it can be shown that the number of arrivals during an interval is
Poisson distributed (i.e., the arrival times are Poisson distributed), then
the interarrival times are exponentially distributed
– The mean arrival rate is given by λ and the mean interarrival time is given
by 1/λ
– The Poisson distribution is a discrete distribution closely related to the
binomial distribution and will be considered later
• It can be shown for the exponential distribution that the mean is equal
to the standard deviation; i.e.,
– μ = σ = 1/λ
• The exponential distribution is the only continuous distribution that is
"memoryless", in the sense that P(X > a+b | X > a) = P(X > b)
© 2008 Winton
14

Graphical Appearance of the

Exponential Distribution
f(x)

f(x)= λe-λ x

x
rsample
© 2008 Winton
15

Standard Exponential Distribution

• The case λ = 1 gives the standard exponential
distribution
– Inverting is straight forward since
nsample
- z nsample
∫ e dz = - e
−z
= 1 - e -nsample = x so nsample = -loge(1-x)
0
0

– closed form formula for obtaining a normalized sample

value (nsample) using a random probability x
• General sample values (rsample) are obtained
from the standard exponential distribution by
1 1
rsample = nsample = - log e (1 - x)
λ λ
© 2008 Winton
16

Sampling Function for the Standard

Exponential Distribution
rsample

1
- log e (1 - x)
λ

1/λ

x
0 (e-1)/e 1
© 2008 Winton
17

Considerations in Using the

Exponential Distribution
• The utility of the exponential distribution for discrete systems
simulation is its effectiveness for modeling a random arrival pattern
– Implicit characteristic of a Poisson process
– Sampling for interarrival times is a natural approach for introducing new
items into the model one at a time
– Care must be taken that when used for this purpose, the exponential
distribution is applied to relatively short time periods during which the
arrival rate is not dependent on time of day
• For example, the model could progress in 1 hour service intervals representing
slow, moderate, and peak demand, each governed by an exponential
distribution with an appropriate mean interarrival time
• A more sophisticated approach is to adjust the arrival rates
dynamically with time, a concept studied under the topic of joint
probability distributions
© 2008 Winton
18

Verfication that μ = σ for the

Exponential Distribution
∞ ∞
∞ - 1 − λx ∞ 1
μ = E(X) = ∫ z λe - λz
dz = - ze - λz
+ ∫ e dz = 0 + e
- λz
=
0 0 0
λ 0 λ
u dv
(integration by parts) uv - ∫ vdu
( )
∞ ∞ ∞
σ 2 = E (X - 1/λ ) = ∫ (z - 1/λ ) λe -λz dz = - (z - 1/λ ) e -λz - ∫ - e -λz 2(z - 1/λ )dz
2 2 2

0 0 0
u dv
∞ ∞
1 2 2 1
= 2 + ∫ z λ e dz - 2 ∫ λe dz = 2
− λz − λz

λ λ0 λ 0 λ
= 1/λ = 1 (by definition)
• For the pdf of the exponential distribution -λ x note that
f(x) = λe
f’(x) = - λ2 e-λ x so f(0) = λ and f’(0) = - λ 2
• Hence, if λ < 1 the curve starts lower and flatter than for the standard
exponential. The asymptotic limit is the x-axis.