Continuous Probability

Distributions
 Continuous Random Variables and
Probability Distributions
• Random Variable: Y
• Cumulative Distribution Function (CDF): F(y)=P(Y≤y)
• Probability Density Function (pdf): f(y)=dF(y)/dy
• Rules governing continuous distributions:

 f(y) ≥ 0  y

 

f ( y )dy  1
b
 P(a≤Y≤b) = F(b)-F(a) = 
a
f ( y )dy

 P(Y=a) = 0  a
 Expected Values of Continuous RVs

Expected Value :   E (Y )   yf ( y )dy (assuming absolute convergenc e)


E  g (Y )   g ( y ) f ( y )dy




  
Variance :   V (Y )  E (Y  E (Y ))   ( y   ) 2 f ( y )dy 
2 2


   y  2 y    f ( y )dy   y f ( y )dy  2 
   
2 2 2
yf ( y )dy   2  f ( y )dy 
   

 E Y   2 (  )   (1)  E Y   
2 2 2 2

E  aY  b   (ay  b) f ( y )dy  a  yf ( y )dy  b 

  
f ( y )dy 
  

 a(  )  b(1)  a  b

V  aY  b  E  (aY  b)  E (aY  b)   
2
 


 (ay  b)  (a  b)  f ( y )dy 
2

 
  (ay  a ) 2 f ( y )dy  a 2  ( y   ) 2 f ( y )dy  a 2V (Y )  a 2 2
 

 aY b  a 
 Example – Cost/Benefit Analysis of
Sprewell-Bluff Project (I)
• Subjective Analysis of Annual Benefits/Costs of
Project (U.S. Army Corps of Engineers assessments)
• Y = Actual Benefit is Random Variable taken from a
triangular distribution with 3 parameters:
 A=Lower Bound (Pessimistic Outcome)
 B=Peak (Most Likely Outcome)
 C=Upper Bound (Optimistic Outcome)
 6 Benefit Variables
 3 Cost Variables
Source: B.W. Taylor, R.M. North(1976). “The Measurement of Uncertainty in Public Water Resource
Development,” American Journal of Agricultural Economics, Vol. 58, #4, Pt.1, pp.636-643
 Example – Cost/Benefit Analysis of
Sprewell-Bluff Project (II) ($1000s, rounded)
Benefit/Cost Pessimistic (A) Most Likely (B) Optimistic (C)
Flood Control (+) 850 1200 1500
Hydroelec Pwr (+) 5000 6000 6000
Navigation (+) 25 28 30
Recreation (+) 4200 5400 7800
Fish/Wildlife (+) 57 127 173
Area Redvlp (+) 0 830 1192
Capital Cost (-) -193K -180K -162K
Annual Cost (-) -7000 -6600 -6000
Operation/Maint(-) -2192 -2049 -1742
 Example – Cost/Benefit Analysis of Sprewell-
Bluff Project (III) (Flood Control, in $100K)
Triangular Distribution with:
k ( y  8.5) 3.5 8.5  y  12.0

lower bound=8.5 f ( y )  k (15.0  y ) / 3.0 12.0  y  15.0
0 elsewhere ( y  8.5, y  15.0)

Peak=12.0
upper bound=15.0 Triangular Distribution (Not Scaled)

0.9

Choose k  area under density curve is 1: 0.8

0.7

Area below 12.0 is: 0.5((12.0-8.5)k) = 1.75k 0.6

Probability Density
Area above 12.0 is 0.5((15.0-12.0)k) = 1.50k 0.5

0.4

Total Area is 3.25k  k=1/3.25 0.3

0.2

0.1

0
8 8.5 9 9.5 10 10.5 11 11.5 12 12.5 13 13.5 14 14.5 15 15.5 16
Flood Control Benefits ($100K)
 Example – Cost/Benefit Analysis of
Sprewell-Bluff Project (IV) (Flood Control)
( y  8.5) 11.375 8.5  y  12.0

f ( y )  (15.0  y ) / 9.75 12.0  y  15.0
0 elsewhere


y  8.5  F ( y )  0


8.5  y  12  F ( y )   (t  8.50) 11.375 dt  (1/11.375)  t 2   8.5t 
y y
2

8.5 8.5

    y 2  8.5 y     8.5 2  8.5  

2 2 2
11.375   y 2  17 y  8.52  22.75


12  y  15  F ( y )  F (12)   (15  t ) 9.75 dt  .5385  (1/ 9.75) 15t   t 2  
y y
2

12 12

 
 .5385  15 y   y 2 2    15(12)  12 2 2   9.75 

 .5385   216  30 y  y 2  19.5 12  y  15

y  15  F ( y )  1
 Example – Cost/Benefit Analysis of
Sprewell-Bluff Project (V) (Flood Control)

( y  8.5) 11.375 8.5  y  12.0


f ( y )  (15.0  y ) / 9.75 12.0  y  15.0
0 elsewhere


0 y  8.5
3.175824  0.747253 y  0.043956 y 2 8.5  y  12

F ( y)  
  10.538462  1 . 538462 y  0 .051282 y 2
12  y  15

1 y  15
Example – Cost/Benefit Analysis of Sprewell-Bluff Project (VI) (Flood Control)

Cumulative Distribution Function

0.9

0.8

0.7

0.6
F(y)

0.5

0.4

0.3

0.2

0.1

0
8 8.5 9 9.5 10 10.5 11 11.5 12 12.5 13 13.5 14 14.5 15 15.5 16
y
 Example – Cost/Benefit Analysis of Sprewell-Bluff Project
(VII) (Flood Control)

12 15
 12  y  8.5  15  15  y   y3 8.5 y 2   15 y 2 y3 
E (Y )   yf ( y )dy   y  dy  12 y 11.375  dy   29.25  19.5    22.75  34.125  
 8 .5
 9. 75      8.5   12
 123 8.5(12) 2   8.53 8.53   153 153   15(12) 2 123 
                
 29.25 19 . 5   29.25 19. 5   22. 75 34. 125   22. 75 34. 125 
   59.08  62.77    21.00  31.49    148.35  98.90   94.95  50.64  
 3.69  10.49  49.45  44.41  11.84
12 15
2  y  8.5  2  15  y   y 4 8.5 y 3   15 y 3 y4 
E Y    y f ( y )dy   y 
 12 15
2 2
dy   y   dy        
 8.5
 9. 75  12
 11. 375   39 29 .25  8.5  34 .125 45.5  12
 12 4 8.5(12) 3   8.54 8.54   154 154   15(12) 3 12 4 
                
 39 29. 25   39 29.25   34 . 125 45 .5   34. 125 45. 5 
   531.69  502.15  133.85  178.46    1483.52  1112.64    759.56  455.74   
 29.54  44.61  370.88  303.82  141.21
 V (Y )  E Y 2    E (Y )  141.21  11.84 2  141.21  140.19  1.02
2

   1.02  1.01
 Uniform Distribution
• Used to model random variables that tend to occur
“evenly” over a range of values
• Probability of any interval of values proportional to its
width
• Used to generate (simulate) random variables from
virtually any distribution
• Used as “non-informative prior” in many Bayesian
analyses

 1 0 ya
 b  a a yb
ya
f ( y)   F ( y)   a yb
 b  a
0 elsewhere 1 yb
 Uniform Distribution - Expectations
2 b
b  1   1 y b 2  a 2 (b  a )(b  a ) b  a
E (Y )   y dy      
a
ba ba 2 a
2(b  a ) 2(b  a ) 2
3 b
b 3  a 3 (b  a )( a 2  b 2  ab)
E Y 2   
b  1   1 y
y2  dy      
a
ba ba 3 a
3(b  a ) 3(b  a )
(a 2  b 2  ab)

3
2
  ab)  b  a 
 V (Y )  E Y    E (Y ) 
2 2
2 ( a b
2
  
3  2 
4(a 2  b 2  ab)  3(b 2  a 2  2ab) a 2  b 2  2ab (b  a ) 2
  
12 12 12
(b  a ) 2 b  a
    0.2887(b  a )
12 12
 Exponential Distribution
• Right-Skewed distribution with maximum at y=0
• Random variable can only take on positive values
• Used to model inter-arrival times/distances for a
Poisson process

 1  y /
 e y0

f ( y)  
0 elsewhere



F ( y)  
y 1 t 

e dt 
1
1 
 1  t 
 e
y
 
 e  y    e  0  1  e  y  y0
0
  0
 Exponential Density Functions (pdf)
Exponential pdf's

1.2

0.8

f(y|th=1)
f(y|th=2)
f(y)

0.6
f(y|th=5)
f(y|th=10)

0.4

0.2

0
0 1 2 3 4 5 6 7 8 9 10
y
Exponential Cumulative Distribution Functions (CDF)
Exponential CDF

0.9

0.8

0.7

0.6

F(y|th=1)
F(y|th=2)
F(y)

0.5
F(y|th=5)
F(y|th=10)

0.4

0.3

0.2

0.1

0
0 3 6 9 12 15
y
 Gamma Function

( )   y  1e  y dy
0

(  1)   y  e  y dy Integratin g by Parts :
0

u  y   du  y 1dy
dv  e  y dy  v  e  y
 
 y 
 (  1)   y e dy  uv   vdu   y e
 y
  y  1e  y dy 
0 0 0

 0  (0)    y  1e  y dy  ( ) (Recursive Property)
0

Note that if  is an integer, ( )  (  1)!


Consider t he integral : 
0
y  1e  y  dy Letting x  y  :
 y  x  dy  dx
  
 y  1  y 
e dy   ( x )  1  x
e dx   
 x 1e  x dx    ( )
0 0 0

EXCEL Function: =EXP(GAMMALN(

 Exponential Distribution - Expectations
  1 y   1  y  
E (Y )   y e dy   ye dy   y 2 1  y 
e dy 
0
   0 0

1
 (2) 2   (2  1)!  

 
2

EY  y  e 2 1 y  
dy 
1  2 y 
 y e dy  

y 31  y 
e dy 
0
   0 0

1
 (3) 3   2 (3  1)!  2 2

 
 V (Y )  E Y 2   E (Y )  2 2  ( ) 2   2
2

 
 Exponential Distribution - MGF
1 

M (t )  E  e tY    ty  1  y  
 1   y   t 
e  e dy  0 e dy
0
  
 1t 
1   y  1  y * 
  e dy   e
 
dy where   *

 0  0 1  t
1  1  y *  * * 1
 M (t )    e   ( 0  1)    (1  t ) 1

1  *    0   1  t

M ' (t )  1(1  t )  2 ( )   (1  t )  2

M ' ' (t )  2 (1  t ) 3 ( )  2 2 (1  t ) 3

 E (Y )  M ' (0)  
 V (Y )  M ' ' (0)   M ' (0)  2 2   2   2
2
 Exponential/Poisson Connection
• Consider a Poisson process with random variable X being
the number of occurences of an event in a fixed time/space
X(t)~Poisson(t)
• Let Y be the distance in time/space between two such
events
• Then if Y > y, no events have occurred in the space of y

Exponentia l "Survival ": P(Y  y )  e  y 

e  y ( y ) 0
Poisson Probabilit y : P ( X ( y )  0)   e  y
0!
   1  Inter - arrivals distances in Poisson Process are Exponentia l with mean   1 
 Gamma Distribution
• Family of Right-Skewed Distributions
• Random Variable can take on positive values only
• Used to model many biological and economic characteristics
• Can take on many different shapes to match empirical data

 1  1  y 
 ( )   y e y  0,  ,   0

f ( y)  
0 otherwise


Obtaining Probabilities in EXCEL:
To obtain: F(y)=P(Y≤y) Use Function: =GAMMADIST(y,,1)
 Gamma/Exponential Densities (pdf)
Exponential and Gamma density functions

0.5

0.4

0.3
exp(2.0)
exp(5.0)
f(y)

gam(2,2)
gam(2,3)
gam(3,2)
0.2

0.1

0
0 2 4 6 8 10
y
Gamma Distribution - Expectations
  1  1  y   1 
E (Y )   y  
y e 
dy   0
y  y 
e dy 
0
 ( )   ( ) 
1  1 (  1) 
 0
( 1) 1  y   1
 y e dy  
(  1)   
( )  ( )  ( )
( ) 
  
( )
  1  y  
 
E Y   y 
2

2 1

y e 
dy 
1
 0

y  1  y 
e dy 
0
 ( )   ( ) 
1  1 (  2)  2
 0
(  2 ) 1  y   2
 y e dy  
(  2)   
( )  ( )  ( )
(  1)(  1)  2 (  1)( )  2
   (  1)
( ) ( )
 
 V (Y )  E Y 2   E (Y )  (  1)  ( ) 2   2  2   2   2  2   2
2

   
 Gamma Distribution - MGF
  1  y  
M (t )  E  e   1
tY
e ty  
y e dy 
0
 ( )  
1   1 t 
1   y   t 
1   y  

 0  0
 1    1   
 y e dy  y e dy
( )  ( ) 
1  
( )   0
 1  y  *
 y e dy where  *

1  t

 M (t ) 
1
( )  
 ( )   
* 
 (1   t ) 

M ' (t )   (1   t )  1 (  )   (1  t )  1
M ' ' (t )  (  1) (1  t )   2 (   )   (  1)  2 (1   t )   2

 E (Y )  M ' (0)  
 V (Y )  M ' ' (0)   M ' (0)   (  1)  2  ( ) 2   2
2
 Gamma Distribution – Special Cases
• Exponential Distribution – 

• Chi-Square Distribution – (≡ integer)

– E(Y)= V(Y)=2
– M(t)=(1-2t)-
– Distribution is widely used for statistical inference
– Notation: Chi-Square with  degrees of freedom:

Y~ 
2
 Normal (Gaussian) Distribution
• Bell-shaped distribution with tendency for individuals to
clump around the group median/mean
• Used to model many biological phenomena
• Many estimators have approximate normal sampling
distributions (see Central Limit Theorem)

1 ( y   )2
1 
f ( y)  e 2 2
   y  ,       ,   0
2 2

Obtaining Probabilities in EXCEL:

To obtain: F(y)=P(Y≤y) Use Function: =NORMDIST(y,,,1)
 Normal Distribution – Density Functions (pdf)

Normal Densities

0.045

0.04

0.035

0.03

N(100,400)
0.025 N(100,100)
f(y)

N(100,900)
0.02 N(75,400)
N(125,400)
0.015

0.01

0.005

0
0 20 40 60 80 100 120 140 160 180 200
y
 Normal Distribution – Normalizing Constant
( y  )2
 
Consider t he integral :  e 2 2
dy  k (we want to solve for k )


y dz 1
Changing variables : z     dy  dz
 dy 
( y  )2 z2 z2
    k  
k  e 2 2
dy   e 2
dz    e 2 dz
   

    z12  z 22 
2 z2 z2 1
k   1   2
     e dz1  e dz 2    e 2
2 2
dz1dz2

     

Changing to Polar Co - Ordinates :

z1  r cos  , z2  r sin  with domains : r  (0, ),   [0,2 ) and dz1dz 2  rdrd
    z12  z 22  2   r 2  cos 2   sin2  
2 1 1 1
k 2   r 2
    e 2
dz1dz 2    e 2
rdrd    e 2 rdrd (cos 2   sin 2   1)
  0 0 0 0

1 
2  r2 2 2 2
  e 2
d   (0  (1)) d   d   0
 2
0 0 0
r 0
2
k
    2  k 2  2 2  k  2 2
 
 Obtaining Value of 

From Previous slide, we get :  e z2 2
dz  2


 1
 e z2 2
dz    2
0
2
 1    1 2  1 u 
Now, Consider :     u e du  u 1 2 e u du
2 0 0

z2
Changing Variables : u   du  zdz
2
1 2
 1   z 
2
  1  z2 2
       e z2 2
zdz   2  e zdz 
2 0  2  0
z
 1
 2 e z2 2
dz  2   2  
0
2
 Normal Distribution - Expectations
1  12 z 2
Z ~ N (0,1)  f ( z )  e
2

    
1 1 1
1  z2 1   z2 1  z2
E ( Z )   z  e 2 dz 
  ze 2
dz   e

2   0  (0)  0

 2 2 2
 
   
 1  12 z 2  1
 1   2  2 z2
 2

E Z   z  2
e dz  2
   z e dz
 2  2  0


1
Changing Variables : u  z 2  du  2 zdz  du  zdz
2
1
 1   2  2 z2 2   12 z 2 2  u 2  1 
 2   z e dz   ze zdz   u e  du 
 2  0
2 0
2 0
2
1 
 3 2  1 1 3 1  1   1  3/ 2 2
32


u 2
 u e du    2 3 2  
    2  1
2 0
2  2  2 2 2 2 2
 
 V ( Z )  E Z 2   E ( Z )  1  0 2  1    1
2

Note : If Y ~ N   ,  then Y    Z
2

 E (Y )  E (   Z )    E ( Z )     (0)  
 V (Y )  V (   Z )   2V ( Z )   2 (1)   2
 Normal Distribution - MGF
 1  1  y    2   y2 y  2 
 
M (t )  E e   e 
tY

ty
 2 2
exp   dy 

1 
  2 2  2 2 2 dy 
exp      ty
 2 
2

  2 2
1   y2 y (   t 2 )  2 
2  
 exp  2   2 dy
2  2  2
2 
Completing the square : (   t 2 ) 2   2  2 t 2  t 2   2

1   y 2 y (   t 2 )  2 2 t 2  t 2   2
 
2t 2  t 2 
2

2  
 M (t )  exp  2   2  dy 
 2  2 2 2
2 2 2
2 

1   y 2  
y (   t 2 )  2  2 t 2  t 2
2
 
2 t 2  t 2 
2

2  
 exp  2    dy 
2  2   2
2  2
2 2


1 
exp  
    
 1  y  (   t 2 ) 2  2 t 2  t 2 2 
2  
 dy 
2 2
    2  2 2



 exp 
 
 2 t 2  t 2 2  

1
exp  
 
 1  y  (   t 2 ) 2 
dy
2  2   
2 2
  2 2 

The last integral being 1, since it is integratin g over the density of a normal R.V. : Y ~ N   t 2 ,  2 
 M (t )  exp 
 
 2 t 2  t 2 2
 
  exp t 
t 2 2 

 2 2   2 
 Normal(0,1) – Distribution of Z2
 
1 
 1 z
  1   z 2  t 
M Z 2 (t )  E etZ   e tz  
2 2 2
e 2
dz  e 2 
dz 

 2  2 

 1 2 t   2 
2  z2   2  z2  

2 
0
e  2 
dz  
 0
e  1 2 t 
dz

1
Changing Variables : u  z  z  u and dz  2
du
2 u
 2  1  2 
2  u   1 1  1  u  
 M Z 2 (t ) 
 0
e  1 2 t 

2 u
du 
2 
0
u 2
e  1 2 t 
du 
12
1  1  2   2
     (1  2t ) 1 2  (1  2t ) 1 2
2  2  1  2t  2
 Z 2 ~ 12
 Beta Distribution
• Used to model probabilities (can be generalized to
any finite, positive range)
• Parameters allow a wide range of shapes to model
empirical data

 (   )  1  1
 ( )(  ) y (1  y ) 0  y  1,  ,   0

f ( y)  
0 otherwise



Obtaining Probabilities in EXCEL:

To obtain: F(y)=P(Y≤y) Use Function: =BETADIST(y,a,b)
 Beta Density Functions (pdf)
Beta Density Functions

4.5

3.5

Beta(1,1)
2.5 Beta(2,2)
f(y)

Beta(4,1)
2 Beta(1,3)
Beta(5,5)
1.5

0.5

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
y
 Weibull Distribution

0 y0

F ( y)  
   y 
1  exp    y  0 ( ,   0)
   
dF ( y )     y    1    y   
   y   exp  
 1
f ( y)    y exp    for y  0
dy           
   1    y   
E (Y )   y  y exp    dy
0
       
y  y  1
Changing variables : u   du  dy and y  (u )1 
 
     y        1
 E (Y )   y  y exp    dy   (u )1  e u du   1   u1  e u du   1  1  
 1
0
        0 0
 
   1    y      2
EY  
2

y  y exp  
2
 
 dy   (u ) 2  e u du   2   u 2  e u du   2  1  
0
        0 0
 
  2  1 
 
 V (Y )  E Y 2   E (Y )   2  1     2 1  
2

     
Note: The EXCEL function WEIBULL(y, uses parameterization: *=
 Weibull Density Functions (pdf)
Weibull pdf's

1.2

0.8

W(1,1)
W(1,2)
f(y)

0.6
W(2,1)
W(2,2)

0.4

0.2

0
0 1 2 3 4 5 6 7 8 9 10
y
 Lognormal Distribution

2
1  log y   
1   
 e 2  
y  0,      ,   0
 2y 2 2

f ( y)  
0 otherwise



Note : Y *  ln( Y ) ~ N   ,  2 
  2 12      2 2 
E (Y )  E  e   M Y * (t  1)  exp   (1) 
Y*
  e
 2 
 2  2 2   2    2 
2
Y* 2
2Y * 
E Y   E  e   E  e   M Y * (t  2)  exp   (2) 
2 
  e

 V (Y )  E Y    E (Y )  e  2   2   e 2  
2 2
2

Obtaining Probabilities in EXCEL:

To obtain: F(y)=P(Y≤y) Use Function: =LOGNORMDIST(y,,)
 Lognormal pdf’s
Lognormal pdf's

1.2

0.8

LN(0,1)
LN(0,4)
f(y)

0.6
LN(1,1)
LN(1,4)

0.4

0.2

0
0 1 2 3 4 5 6 7 8
y