Chapter 3

Random Variables, Distributions,

and Density Functions
ECEN 5513 Stochastic Systems

School of Electrical and Computer Engineering

Oklahoma State University
Cumulative Distribution Function (CDF)
 Definition 3.1: The CDF of a random variable, X, is

FX ( x)  Pr( X  x) Don' t be confused with PMF :

PX ( x )  Pr( X  x )
 The properties of CDFs are summarized below
x1  X  x2
(1) FX ()  0, FX ()  1
(2) 0  FX ( x)  1
FX ( x1 ) x2
x1

(3) For x1  x2 , FX ( x1 )  FX ( x2 ) FX ( x2 )

(4) For x1  x2 , Pr( x1  X  x2 )  FX ( x2 )  FX ( x1 )

They are equivalent Pr( x1  X  x2 )  FX ( x2 )  FX ( x1 )  PX ( x1 )
for a continuous
random variable, why?
Pr( x1  X  x2 )  FX ( x2 )  FX ( x1 )  PX ( x2 )
2
 (1) FX ()  0, FX ()  1
(2) 0  FX ( x)  1
Example (3) For x1  x2 , FX ( x1 )  FX ( x2 )

 Which of the following mathematical functions could be the

CDF of some random variable?

1 1
(1) FX ( x)   tan 1 ( x)
2 

(2) FX ( x)  [1  e  x ]u ( x)

 x2
(3) FX ( x)  e

(4) FX ( x)  x 2u ( x)
3
CDF for Discrete Random Variables
 For discrete random variables, the CDF can be written in
terms of the probability mass function (PMF).
 Consider a discrete random variable X which can take on values from
the discrete set {x1, x2, x3, ….}
x1 x2 xk xk 1
k
FX ( x )  Pr( X  x )   PX ( xi ) for xk  x  xk 1
i 1
 For example, a random variable is equally likely to take on any integer
value in the set {0, 1, 2, 3}. Sketch the CDF of X.
1

3
4 CDF of discrete random variables
are always staircase-like.
1
2
1
4

0 1 2 3 4 X

4
 FX ( x)  Pr( X  x)

Example of CDF (1)

 Example 3.3
  
Suppose a random variable has a CDF by FX (t )  1  e  t u (t ) . Find the
following quantities
 (a) Pr(𝑋 > 5) Pr( X  5)  1  Pr( X  5)
 1  FX (5) Pr( X  5)  ?
 e 5

 (b) Pr(𝑋 < 5) Pr( X  5)  Pr( X  5)

For a continuous random variable,
 FX (5) you can ignore the probability at a
particular value. However, it is not
FX (5)  Pr( X  5) true for a discrete random variable.
 1  e 5

5
 FX ( x)  Pr( X  x)

Example of CDF (2)

 Example 3.3
  
Suppose a random variable has a CDF by FX (t )  1  e  t u (t ) . Find the
following quantities
 (c) Pr(3 < 𝑋 < 7)  Pr( X  7)  Pr( X  3) 3 X 7
 FX (7)  FX (3) FX (3)
x1 x2
 e 3  e 7 FX (7)

 (d) Pr(𝑋 > 5|𝑋 < 7) Pr( X  5 | X  7) 

Pr( X  5, X  7)
Pr( X  7)
Pr(5  X  7) FX (7)  FX (5)
 
Pr( A, B)
Pr( X  7) 1  e 7
Pr( B | A) 
Pr( A) 1  e 7  (1  e 5 ) e 5  e 7
 7

1 e 1  e 7
6
Cumulative Distribution Function (CDF)
 Definition 3.1: The CDF of a random variable, X, is
Piece-wise
continuous FX ( x)  Pr( X  x) Don' t be confused with PMF :
1

3
PX ( x )  Pr( X  x )
4
1
2
1
4

0 1 2 3 4 X
 The properties of CDFs are summarized below
(1) FX ()  0, FX ()  1 x1  X  x2

FX ( x1 )
(2) 0  FX ( x)  1 x1 x2
FX ( x2 )
(3) For x1  x2 , FX ( x1 )  FX ( x2 )
(4) For x1  x2 , Pr( x1  X  x2 )  FX ( x2 )  FX ( x1 )
7
Example of CDF
 Suppose we flip a balanced coin four times, and the let random
variable X represent the number of times tails occurs.
 What kind of random variable is X? What is the PMF/CDF of X, i.e., 𝑃𝑋 𝑥
and 𝐹𝑋 𝑥 ? List all probabilities associated with the PMF/CDF of X
n k  
4
Pr( X  k )  PX (k )    p (1  p ) n  k PX (k )     (k  0,1,2,3,4)
4 1
k   k  2 
1 1 6 1 1
𝑃𝑋 0 = ,𝑃 1 = , 𝑃𝑋 2 = ,𝑃 3 = , 𝑃𝑋 4 =
16 𝑋 4 16 𝑋 4 16

 0 x0
15  1 / 16 0  x 1
1 11 16 
 5 / 16 1 x  2
FX ( x)  Pr( X  x)
16
5 FX ( x)  
1 16
11 / 16 2 x3
16
0 1 2 3 4 𝑋
15 / 16 3 x  4

 1 x4
8
Example of CDF (3) (Cont’d)
 Suppose we flip a balanced coin four times, and the let random
variable X represent the number of times tails occurs.
 Sketch the cumulative distribution function (CDF) of X, i.e., 𝐹𝑋 (𝑥).
1 15
16
11
16

5
16
1
16

0 1 2 3 4 𝑋
 Write 𝐹𝑋 (𝑥) analytically in terms of unit step functions.
1 1 6 1 1
𝑃𝑋 0 = 16, 𝑃𝑋 1 = 4, 𝑃𝑋 2 = 16，𝑃𝑋 3 = 4 , 𝑃𝑋 4 = 16

1 1 6 1 1
𝐹𝑋 𝑥 = 𝑢 𝑥 + 𝑢 𝑥−1 + 𝑢 𝑥−2 + 𝑢 𝑥−3 + 𝑢 𝑥−4
16 4 16 4 16
9
Probability Density Function (PDF)
 Why PDF?
 CDFs may be hard to work with that may not have a closed form.
 It is difficult to infer various properties of a random variable from CDF.
 PDF is only suitable for continuous random variables.
 Definition 3.2
 The PDF of the random variable X f X (x)
evaluated at the point x is
Pr( x  X  x   )
f X ( x)  lim
 0 
F ( x   )  FX ( x)
 lim X FX (x)
 0 

dFX ( x)

dx
10
Properties of PDF
 From the properties of the CDFs, we can also infer several
important properties of PDFs as follows

 (1) f X ( x)  0
dFX ( x)
 (2) f X ( x) 
dx
x
 (3) FX ( x)   f X ( y )dy Is that possible for f X ( x )  1 ?


 (4)


f X ( x)dx  1
b
 (5)

a
f X ( x)dx  Pr(a  X  b)  FX (b)  FX (a )

11
Example of PDF (1)
 Example 3.4



f X ( x )dx   e  x dx  e  x  1
0
x
 (a) f X ( x)  e u ( x)  0 

x
 (b) f X ( x)  e  e  x u ( x)  e x u ( x)
3 2
 ( x  1), x 2
 (c) f X ( x)   4
 0, Otherwise

1, 0  x  1
 (d) X
f ( x )  
0, Otherwise
  
 f X ( x )dx  0 2 xe dx   e  y dy  e
y
 x2 0
 1
 x2 
 (e) f X ( x)  2 xe u ( x)
0

y  x 2 dy  2 xdx
12
Example of PDF (2)
 Example 3.5:
  
A random variable has a CDF FX ( x)  1  e  x u ( x) . Its PDF is given by

dFX ( x)
f X ( x)   e x u (x)
dx
 x2
 Likewise, if a random variable has a PDF given by f X ( x)  2 xe u ( x ),
then its CDF is given
x x
FX ( x )   f X ( y )dy   2 ye  y2
u( y )dy

x x2
  2 ye  y2
dy  0 e  z dz
0
z  y 2 dz  2 ydy

e z 0
x2

 1 e  x2
u( x)
13
Noise Distribution of Image Pixels

14
Gaussian Random Variable
 Why it is important?
 Many physical phenomenon can be modeled as Gaussian random
variables, including the thermal noise in electronic circuits.
 ( x   )2 
f X x  
1
exp  
2 2
 2 2


Carl Friedrich Gauss

(1777–1855)
German Mathematician

 According to the Central Limit Theorem, the sums of random variables

converge to a Gaussian random variable in distribution (under some
mild conditions).
15
Gaussian Random Variable
 Definition 3.3
 A Gaussian random variable is one whose PDF can be written in the
general form as  ( x  m) 2 
f X x  
1
exp    X ~ N ( m,  2 )
2 2
 2 2 
 m is the mean
 x 2   X ~ N (0,1)
f X x  
1
 𝜎 is the standard deviation exp  
 𝜎 2 is the variance 2  2 (standard Gaussian)
 The CDF of a Gaussian random variable is written as

 ( y  m) 2 
FX  x   
x 1
exp  dy

2 2
 2 2


Unfortunately, we don’t have

a closed form of the CDF of
a Gaussian random variable.

16
Gaussian Random Variable
 Definition 3.3
 A Gaussian random variable is one whose PDF can be written in the
general form as  ( x  m) 2 
f X x  
1
exp  
  X ~ N ( m,  2
)
 m is the mean
2 2
 2 2

 𝜎 is the standard deviation

 x2 
f X x  
1
Q( x)  1  Q( x) 2
exp  
 2
X ~ N (0,1)
Q( x)  Q( x)  1 (standard Gaussian)
x 0 x
  t2 
Q x  
1
1  Q( x) 2 
x
exp  dt
 2
17
Q-Function Table

18
 Gaussian Noise Models
 x2 
f X x  
1
exp  2 
2  2 
2

(Zero - mean additive Gaussian noise)

 0   20
( Noise free image)

  13
 8 http://homepages.inf.ed.ac.uk/rbf/HIPR2/noise.htm

19
Homework Problem Solving
 Pro. 3.26
 The IQ of a randomly chosen individual is modeled using a Gaussian
random variable. Given that 50% of the population have an IQ above 100
(and 50% have an IQ below 100) and that 50% of the population have an
IQ in the range of 90-110 (typo in the textbook), what is the percentage
of the population have an IQ of at least 140 (genius)?
 First, we find the mean is 100 and we need to find the standard deviation ()
 According to the condition, Pr(90  IQ  110)  0.5 Pr(IQ  90) - Pr(IQ  110)  0.5
 90  100   110  100    10   10   10   10 
Q   Q   0.5 Q   Q   0.5 1  Q   Q   0.5
              

 10   10  10
1  2Q   0.5 Q   0.25  0.675   14.8
    
 What is the probability of genius (IQ>140)?
 140  100   Q2.7  0.003467  0.3467%
Q 
 14 . 8 
20
 Uniform Random Variable
 The PDF and CDF of a uniform random variable are defined as
 1 x  0, x  a,
 , a xb FX ( x)   f X ( y )dy x a
f X x    b  a  FX  x    , a  x  b,
 0, elsewhere, b  a
 1, x  b.

 Applications
 Can be used to create many other random variables of interest.
 When there is no prior knowledge available rather than the range, we
can assume the random variable is uniformly distributed over an interval.
21
 Uniform Random Variable Example (1)
 Problem 3.27
 The phase of a sinusoid, , is uniformly distributed over [0, 2),
f     0, x  0,
 1
, 0    2 
f      2  
1
2 F    , 0  x  2 ,
 0, elsewhere, 2 
  1, x  2 .
2
3𝜋
 (a) Find Pr(Θ > )
4  1
3  3   3  , a xb
   
  1 3 5
 f X x    b  a
Pr      1  Pr     1 F 
 4   4   4  8 8  0, elsewhere,

FX  x    f X  y dy
x
3𝜋
 (b) Find Pr(Θ < 𝜋|Θ > ) 
4
 3   0, x  a,
Pr     |   
 4  x a
FX  x    , a  x  b,
 3
Pr       Pr      Pr     F ( )  F  3
 3   b  a
 x  b.
 
4   4   4  1/ 2  3 / 8 1/ 8  1,
3  3 
    1/ 5

Pr   

1  Pr      3  1 3/8 5/8 Uniform Random Variable
 1  F  
 4   4   4 
22
Uniform Random Variable Example (2)
 Problem 3.27
 The phase of a sinusoid, , is uniformly distributed over [0, 2),
 1 f     0, x  0,
 , 0    2
f      2 1 
F     , 0  x  2 ,
2
 0, elsewhere,
  2
 1, x  2 .
2

 5 
Pr       1
 (c) Find Pr(cos Θ < 1/2) 3 3  
f X x    b  a
, a xb
 5     0, elsewhere,
 Pr      Pr    
 3   3
FX  x    f X  y dy
x

 5   
 F    F   

 3   3  0, x  a,
5  x a
 FX  x   

 3
5  3
 3 , a  x  b,
2 2 b  a
2 2
3 3
 1, x  b.
5 1 4 2
    Uniform Random Variable
6 6 6 3
23
Exponential Random Variable
 The exponential random variable has a PDF and CDF given by
 x   x 
f X  x   exp  u( x ), (b  0)
1
FX  x   1  exp  u ( x)
b  b   b 

 The parameter b>0 is related to the width of the PDF and the PDF has a
peak value of 1/b which occurs at x=0.
 Applications
 Exponential random variables are commonly encountered in the study of
queuing systems, for example, the time between arrivals of customer at a
bank or duration of voice conversation in a telephone network.
24
Exponential Random In which time period,
the arrival is more like
Variable Example to be less than 1minute?

 Queuing problem b2

 For a telephone switch during three periods of b 1
time in a day (morning, afternoon, evening), we use b 2
3

2
three exponential random variables (𝑏 = 2, 1, 3) to
model the time between arrivals of next call. For
each time periods find Pr(2 < 𝑋 < 3).
Pr(2  X  3)  FX (3)  FX (2)
 [1  exp( b3 )]  [1  exp( b2 )]  x
f X x  
1
exp  u( x ), (b  0)
b  b
 exp( b2 )  exp( b3 )
FX  x    f X  y dy
x

b  2 (slow) : Pr(2  X  3)  0.1447



  x 
b  1 (med) : Pr(2  X  3)  0.0855 FX  x   1  exp  u ( x)
  b 
b  23 (fast) : Pr(2  X  3)  0.0387 Exponential random variable
25
 Laplace Random Variable
 A Laplace random variable has a PDF which takes the form of
a two-sided exponential, and its PDF can CDF are given by
 x  1  x
f X x  
1
exp  , (b  0)  2 exp b , x0
FX  x     
2b  b
1  1 exp  x , x  0
b2  2  b

f X  x  is NOT differentiable everywhere FX  x  is differentiable everywhere

 The width of the PDF is determined by the parameter b, while the peak
1
value of the PDF is .
2𝑏
 Application
 Can be used to model the amplitude of a speech signal (voice) signal.
26
Five Continuous Random Variables
 1
 , a xb
f X x    b  a
 0, elsewhere,

 x
f X x  
1
exp  u( x ), (b  0)
b  b

 x
f X x  
1
 (x  ) 2
 exp  , (b  0)
f X x  
1
exp   2b  b
2 2  2 2 

 x2 
f X  x   2 exp 
x
u ( x )
2 
  2 

27
  ( x  m) 2 
f X x  
1
exp  
2 2
 2 2

Example (1)
 Given a Gaussian PDF f X  x   c exp 2( x  1) 2  ,
 Find 𝑚, 𝜎, 𝑐
2     14 ,  
2
m  1
2 1 1
2 2
1 1
c 
2 2  /2
 Using Q function to find the following probability
Pr( X  2  1) x  b  x  b or x  b
 Pr ( X  2  1)  ( X  2  1) 
 Pr( X  3)  Pr( X  1)
 Pr( X  3)  1  Pr( X  1)
 Q( 13/21 )  1  Q( 11/ 21 )  Q(8)  1  Q(4)
28
 Example (2)
 Prob. 3.31
 Let R be a Rayleigh random variable whose PDF is

f R r   c  r  exp(  r 2 )u( r )
 Find the value of the constant c.
 c  c c
 c  r  exp(  r )dr   exp(u )du  exp(u )    1 c  2
2 0
0
2 0 2 2
u  r 2   du  2r dr  r dr  du2
It is too
complicated!

 x2 
f X  x   2 exp   2 u( x )
x 1 1 1
 1    c  2  2
2

  2  2 2 2 
29
  x2    x2  
f X  x   2 exp  FX  x   1  exp  2  u ( x)
x
u ( x )
2 
  2    2  

Example (3)
 Prob. 3.31
 Let R be a Rayleigh random variable whose PDF is

f R r   2r  exp(r )u (r ) 2

FR r   1  exp(r ) u (r ) 2

 Find Pr(R>r) for an arbitrary constant r.
Pr( R  r )  1  Pr( R  r )  1  FR (r )  exp(  r 2
)u (r )

 Find Pr(R>1|R<2)
Pr(1  R  2) Pr( R  1)  Pr( R  2) e 1  e 4
Pr( R  1 | R  2)   
Pr( R  2) 1  Pr( R  2) 1  e 4
Pr( R  2)  Pr( R  1) FR (2)  FR (1)
 
Pr( R  2) FR (2)
30
Motivation of Conditional PDF
Assuming the IQ distribution of general
population follows a Gaussian PDF (m=100 and
𝜎 = 15 , try to answer the following three
conditional PDF related questions as follows:

Questions 1: What is the IQ distribution for

people with IQ between 70 and 130?

Questions 2: What are the IQ distribution

for those whose IQs are above mean?

Questions 3: What are the IQ distribution

for male and female people? https://www.google.com/webhp?sourceid=chrome-
instant&ion=1&espv=2&ie=UTF-8#q=IQ+distribution+female+male

31
Conditional PDF
 Definition 3.5
 The conditional PDF of a random variable X conditioned on event A is
Pr( x  X  x   | A)
f X | A  x   lim
 0 
 Properties of conditional PDF (same as those of PDF)
 (1) f X | A ( x)  0
dFX | A ( x) There are two kinds of condition A
 (2) f X | A ( x)  (1) Given a related factor
dx (2) Given a smaller sample space
x
 (3) FX |A ( x )   f X |A ( y )dy

Conditional

(4)

 
f X |A ( x )dx  1 CDF

b
 (5) 
a
f X | A ( x)dx  Pr(a  X  b | A)
32
Conditional PDF with
Smaller Sample Space
 We can generalize the conditional PDF of a random variable,
when the conditioning event is 𝐴 = 𝑎 ≤ 𝑋 < 𝑏 :
 f X ( x)
 , a X b
f X | A  x    Pr(a  X  b)
 0, otherwise
 The conditional PDF takes on the same functional form, but is
scaled by the probability of the conditioning event, shown as:

33
Example of Conditional PDF
 Example 3.11
 Let X be a random variable representing the length of time we spend waiting in
the grocery store checkout line. Suppose X has an exponential PDF given by
1 𝑥
𝑓𝑋 𝑥 = 𝑐 exp − 𝑐 𝑢 𝑥 , where c=3 min. What is the PDF of the amount of
time we spend waiting in line given that we have already been waiting for 2 min?
  f X ( x)
f X ( x)  a  2   X 2

f X | A  x    Pr( a  X  b)
, a X b 
b     f X |{ X  2}  x    Pr( X  2)
,
 0, otherwise    0, otherwise
1  x
exp  
f X ( x) 2 x
u ( x  2)  f X ( x) u ( x  2)  3  3  u ( x  2)  exp
1
f X |{ X  2}  u ( x  2)
Pr( X  2) 1  FX (2)  2
exp  
 3  3 
 3 0.35
T=0

  x 
T=2

Recall CDF : FX  x   1  exp  u ( x)

0.3 T=3
T=4
0.25

  c  0.2

In general, we have 0.15

T  x 
0.1

f X ( x) 1
f X |{ X T }  u ( x  T )  exp u ( x  T ) 0.05

Pr( X  T ) 3  3  0
0 2 4 6 8 10

34
Take a Deep Look (1)
1 T  x 
f X |{ X T }  exp u ( x  T )
3  3 
Probability of a 0.35
person who has T=0
waited 2 minutes to T=2
be served between 0.3 T=3
the 2nd to 4th T=4
minutes.
0.25

0.2

Probability of a 0.15
newly arrived
person to be
0.1
served between the
2nd to 4th minutes.
0.05

0
0 2 4 6 8 10
35
Take a Deep Look (2)
1 T  x 
f X |{ X T }  exp u ( x  T )
Probability of a newly 3  3 
arrived person to wait
0.35
less than 2 minutes.
T=0
(from time 0)
T=2
0.3 T=3
T=4

Probability of a person 0.25

who has waited 2
minutes to wait less 0.2
than 2 minutes.
(from the 2nd minute)
0.15

0.1
Probability of a person
who has waited 4
minutes to wait less 0.05
than 2 minutes.
(from the 4th minute)
0
0 2 4 6 8 10

36
Problem Solving Exercise
 Prob. 3.39
 Suppose V is a uniform random variable
0.5 0  v  2, 0.5
fV ( v )  
0 OTW
0 1 2
 Find the condition PDF 𝑓𝑉|{𝑉>1} (𝑣).

fV (v )
fV |{V 1} (v)  for V  1
Pr(V  1) 1

1 1  v  2,
fV |{V 1} ( x)  
1 2
0 OTW

37
Problem Solving Exercise
 Prob. 3.39
 Suppose V is a uniform random variable
0.5 0  v  2, 0.5
fV ( v )  
0 OTW
0 1 2
 Find the condition PDF 𝑓𝑉|{1<𝑉<3} (𝑣).
2 2

fV (v )
fV |{ 1 V  3 } (v)  for 2 V 
1 3
2 2
Pr( 2  V  2 )
1 3 2
1

1 1
 v  32 ,
fV |{ 1 V  3 } ( x)   2 1 3
2 2
2 2
0 OTW
38
Problem Solving Exercise
 Prob. 3.39 1
 Suppose V is a uniform random variable

0.5 0  v  2, 1 3

fV (v )   2 2

0 OTW 1 1
 v  32 ,
fV |{ 1 V  3 } (v)   2
2 2
0 OTW
 Find the condition CDF 𝐹𝑉|{1<𝑉<3} (𝑣).
2 2 1
v
FV |{ 1 V  3 } (v)   fV |{ 1 V  3 } (u )du
2 2  2 2
1 3
v
 1 du  v  12
2 2

 0 v 1
2

FV |{ 1 V  3 } (v)  v  12 V 
2 1 3

( 12  v  32 )
2 2
 1
2 2

 v 3
2
39
Laplacian Random Variable Examples
 Image thermal noise modeling
 Gaussian PDF
 Laplacian PDF

 x2 
f X x  
1
exp  2  (Gaussian)
2  2 
2

 x
f X x  
1
exp  , (b  0) (Laplacian)
2b  b

40
Reyleigh Random Variable
 A Reyleigh random variable has a one-sided PDF, and its PDF
and CDF are given (for any 𝜎 > 0) by
 x2    x2  
f X  x   2 exp 
x
u ( x )
2 
FX  x   1  exp  2  u ( x)
  2    2  

 The Reyleigh distribution is described by a single parameter 𝜎 2 , which is

related to the width of PDF, but 𝜎 2 is not the variance.
 Application
 The Reyleigh distribution can be used to model the magnitude of a
complex number whose real and imaginary parts both follow a zero-
mean Gaussian distribution.
41
 Reyleigh Random Variable Example
 Reyleigh random variable for life expectance modeling
Celebrity drug users

http://bespokeblog.wordpress.com/

 x2 
f X  x   2 exp  2 u ( x)
x
  2 
1
(Reyleigh PDF) 2 
1000

42
Comparison

43
Q-Function
 Given a standard Gaussian random variable 𝑋~𝑁(0,1)
 x2 
f X x  
1
exp  
2  2
 The Q-function is defined as
1   t2 
Q x    exp  dt
2 x
 2 0 x

 We can compute Pr(𝑋 > 𝑥) of a Gaussian 𝑋~𝑁(𝑚, 𝜎 2 ) to be

 1  ( y  m) 2 
Pr( X  x)   exp  dy
x
2 2
 2 2

 1  t2   xm
t
ym  x  m exp  dt  Q 
  2  2   
 Table of the Q-function (p585),
44
Q-Function in Graph

 x2 
f X x  
1
exp   X ~ N (0,1)
2  2 (standard Gaussian)

x0 x

1  Q x  Qx 

1  Q x   Q x  Q x   Q x   1
45
  xm
Pr( X  x )  Q   Q x   1  Q x 
Example   

 Example 3.7  ( x  3) 2  X ~ N (m,  2 )

A random variable has a PDF f X  x  
1
 exp  
8  8  (m  3,   2)
 Find following probabilities in terms of Q-functions with a positive input.
 (a) Pr( X  0)  1  Pr( X  0)  1  Q( 02 3 )  1  Q(1.5)
 (b) Pr( X  4)  Q (( 4  ( 3)) / 2)  Q (3.5)
5 1
 (c) Pr( X  3  2)  Pr( 2  X  3  2)  Pr( 5  X  1)
x  a  a  x  a  Pr( X  5)  Pr( X  1)
 Q( 523 )  Q( 123 )  Q(1)  Q(1)  1  2Q(1)

 Pr( X  2  1)  Pr ( X  2  1)  ( X  2  1)   Pr( X  3)  Pr( X  1)

(d)
x  b  x  b or x  b  Q( 32 3 )  1  Pr( X  1)
 Q(3)  1  Q( 123 )  Q(3)  1  Q(2)
46
Motivation
Assuming the IQ distribution of general
population follows a Gaussian PDF, try to
answer the following two conditional PDF
related questions as follows:

Questions 1: What are the conditional PDF

for those whose IQs are above mean?

Questions 2: What are the conditional PDF

for the female or male population?

Questions 3: Given the conditional PDF for https://www.google.com/webhp?sourceid=chrome-

instant&ion=1&espv=2&ie=UTF-8#q=IQ+distribution+female+male
the female and male, how to obtain the PDF
of all population?
47
 Example of Conditional PDF
 Example 3.11
 Let X be a random variable representing the length of time we spend waiting in
the grocery store checkout line. Suppose X has an exponential PDF given by
1 𝑥
𝑓𝑋 𝑥 = 𝑐 exp − 𝑐 𝑢 𝑥 , where c=3 min. What is the PDF of the amount of
time we spend waiting in line given that we have already been waiting for T min?

f X ( x)
f X |{ X T }  u( x  T ) 0.35
T=0

Pr( X  T ) 0.3
T=2
T=3
T=4
0.25

T  x 
0.2

1
 exp u ( x  T ) 0.15

3  3  0.1

0.05

0
0 2 4 6 8 10

48
Problem Revisited (1)
1 T  x 
f X |{ X T }  exp u ( x  T )
3  3 
Probability of a 0.35
person who has T=0
waited 2 minutes to T=2
be served between 0.3 T=3
the 2nd to 4th T=4
minutes.
0.25

0.2

Probability of a 0.15
newly arrived
person to be
0.1
served between the
2nd to 4th minutes.
0.05

0
0 2 4 6 8 10
49
Problem Revisited (2)
1 T  x 
f X |{ X T }  exp u ( x  T )
Probability of a newly 3  3 
arrived person to wait
0.35
less than 2 minutes.
T=0
(from time 0)
T=2
0.3 T=3
T=4

Probability of a person 0.25

who has waited 2
minutes to wait less 0.2
than 2 minutes.
(from the 2nd minute)
0.15

0.1
Probability of a person
who has waited 4
minutes to wait less 0.05
than 2 minutes.
(from the 4th minute)
0
0 2 4 6 8 10

50
Problem Solving Exercise
 Prob. 3.39
 Suppose V is a uniform random variable
0.5 0  v  2, 0.5
fV ( v )  
0 OTW
0 1 2
 Find the condition PDF 𝑓𝑉|{𝑉>1} (𝑣).

fV (v )
fV |{V 1} (v)  for V  1
Pr(V  1) 1

1 1  v  2,
fV |{V 1} ( x)  
1 2
0 OTW

51
Problem Solving Exercise
 Prob. 3.39
 Suppose V is a uniform random variable
0.5 0  v  2, 0.5
fV ( v )  
0 OTW
0 1 2
 Find the condition PDF 𝑓𝑉|{1<𝑉<3} (𝑣).
2 2

fV (v )
fV |{ 1 V  3 } (v)  for 2 V 
1 3
2 2
Pr( 2  V  2 )
1 3 2
1

1 1
 v  32 ,
fV |{ 1 V  3 } ( x)   2 1 3
2 2
2 2
0 OTW
52
Problem Solving Exercise
 Prob. 3.39 1
 Suppose V is a uniform random variable

0.5 0  v  2, 1 3

fV (v )   2 2

0 OTW 1 1
 v  32 ,
fV |{ 1 V  3 } (v)   2
2 2
0 OTW
 Find the condition CDF 𝐹𝑉|{1<𝑉<3} (𝑣).
2 2 1
v
FV |{ 1 V  3 } (v)   fV |{ 1 V  3 } (u )du
2 2  2 2
1 3
v
 1 du  v  12
2 2

 0 v 1
2

FV |{ 1 V  3 } (v)  v  12 V 
2 1 3

( 12  v  32 )
2 2
 1
2 2

 v 3
2
53
Conditional PDF with Multiple Factors

Overall entry-level salary

distribution 𝑝𝑋 (𝑥)

High-school (HS) College BS MS/PhD graduate

graduate salary graduate salary salary distribution
distribution distribution 𝑝𝑋|𝐺𝐴 (𝑥)
𝑝𝑋|𝐻𝑆 (𝑥) 𝑝𝑋|𝐵𝑆 (𝑥)

p X |HS ( x) Pr(HS | X  x)  ?

 
p X |BS ( x)   p X ( x) Tom' s salary x   Pr(BS | X  x)  ?
Pr(GA | X  x)  ?
p X |GA ( x)  
54
Extension of Total Probability Theorem
 Recall the Theorem of Total Probability
 Let 𝐵1 , 𝐵2 , … . , 𝐵𝑛 be a set of mutually exclusive and exhaustive events
n n
Pr( A)   Pr( A | Bi ) Pr( Bi )   Pr( A, Bi )
i 1 i 1

 We extend this theorem to the cases of PDF

n
f X ( x)   f X |Bi ( x) Pr( Bi )
i 1

Unconditional CDF/PDF of X can be found by a weighted

sum of conditional ones with weights determined by the
probabilities that each of the conditioning events is true.

55
Extension of Bayes’ Theorem
 Recall Bayes’s Theorem
 Let 𝐵1 , 𝐵2 , … . , 𝐵𝑛 be a set of mutually exclusive and exhaustive events
Pr( A | Bi ) Pr( Bi )
Pr( Bi | A)  n Pr( A)
 Pr( A | B ) Pr( B )
j 1
j j

 Similarly, given the event that the random variable 𝑋 = 𝑥, we

can compute the probability of each possible event as follows:
Pr( X  x | Bi ) Pr( Bi )
 If X is discrete, we have Pr( Bi | X  x)  n

 Pr( X  x | B ) Pr( B )
j 1
j j

f X (x)
f X |Bi ( x) Pr( Bi )
 If X is continuous, we have Pr( Bi | X  x)  n

f j 1
X |B j ( x) Pr( B j )
56
 f X |Bi ( x) Pr( Bi )
Pr( Bi | X  x)  n

Example f j 1
X |B j ( x) Pr( B j )

 Example 3.12
 In a junior swimming competition, swimmers are placed into one of two
categories based on their performance. The fastest swimmers are placed
in the A category, while the slower swimmers are put in the B group.
 Let X be a random variable representing a child’s time (in seconds) in
the 50m freestyle race. Suppose we have two following conditional PDF
for a child’s time in each category:
f X | A ( x)  4 
1 / 2
 
exp  ( x  40) 2 / 4 (Group A) f X |B ( x )  4 
1 / 2
exp ( x  45) 2 / 4  (Group B)

 Furthermore, assume that 30% of the swimmers are in the A group and
70% are in the B group. If a child swims the race with a time of 42 s,
what is the probability that the child was in the B group?
 42  45 2
f X |B (42) Pr( B) exp( )  0. 7
Pr( B | X  42)   4
f X | A (42) Pr( A)  f X |B (42) Pr( B) exp( 42445  )  0.3  exp( 40442  )  0.7
2 2

0.7 exp( 94 ) What is the probability of

  0.4007
0.3 exp(1)  0.7 exp( 94 ) being in the A group?
57
Problem Solving Exercise
 Prob. 3.36
 A digital communication system sends two messages, 𝑀 = 0 and 𝑀 = 1,
each with equal probability. A receiver observe a voltage signal which can
be modeled as a Gaussian random variable, whose PDFs conditioned on
the transmitted message are given by the following functions
1 𝑥−𝑚 2
𝑓𝑋|𝑀=𝑚 = exp − , 𝑚 = 0, 1
2𝜋𝜎 2 2𝜎 2
 Find and plot Pr(𝑀 = 0|𝑋 = 𝑥) as a function of x for 𝜎 2 = 1 & 𝜎 2 = 5
f X |B ( x) Pr( Bi )
Pr( Bi | X  x)  n i
(Extended Bayesian Theorem)

 f X |B ( x) Pr( B j )
j 1
j

When it is 0.5?
 x  2
exp  2 
  2  
1
 x2   ( x  1) 2   2x 1 
exp  2   exp   1  exp 2 
 2   2 2   2 
0.5
58
Problem Solving Exercise
 Prob. 3.36
 A digital communication system sends two messages, M=0 and M=1,
each with equal probability. A receiver observe a voltage signal which can
be modeled as a Gaussian random variable, whose PDFs conditioned on
the transmitted message are given by the following functions
1 𝑥−𝑚 2
𝑓𝑋|𝑀=𝑚 = exp − , 𝑚 = 0, 1
2𝜋𝜎 2 2𝜎 2
 Repeat (a) assuming that Pr(𝑀 = 0) = 1/4，Pr(𝑀 = 1) = 3/4.
f X |Bi ( x) Pr( Bi )
Pr( Bi | X  x)  n
 f X |B j ( x) Pr( B j )
j 1 When it is 0.5?
1  x  2
exp  2 

4  2  1

 x  3  ( x  1)  2x 1 
 1  3 exp
2 2
1
exp  2   exp  2  2 
4  2  4  2   2 
 0.599
59
Motivation of Conditional PDF
Assuming the IQ distribution of general
population follows a Gaussian PDF (m=100
and 𝜎 = 15) , answer the following two
conditional PDF related questions as follows:
Questions 1: What are the IQ distribution Questions 2: How the overall IQ distribution
for those whose IQs are above mean? is related to those of male and female people?

60
 Problem Solving Exercise
 x2    x2  
f X  x   2 exp 
x
 Prob. 3.40 u ( x )
2 
FX  x   1  exp  2  u ( x)

  2    2  
 Suppose V is a Rayleigh random variable

fV (v)  v exp( )u (v)  FV (v)  1  exp( ) u (v)

v2
2
v2
2

 Find the conditional PDF 𝑓𝑉|{1<𝑉<3} (𝑣).

2 2

fV |{ 1 V  3 } (v) 
2 2
fV (v )

fV (v )
Pr( 2  V  2 ) Pr(V  32 )  Pr(V  12 )
1 3
 12  v  32 
v exp( v2 )
2
fV (v )
 
FV ( 2 )  FV ( 2 ) 1  exp( 89 )   1  exp( 18 ) 
3 1

 v exp( v2 )
2

v exp( v2 )
2
 1
V  3
 fV |{ 1 V  3 } (v)   exp( 18 )  exp( 89 ) 2 2
exp( 18 )  exp( 89 ) 2 2

 0 OTW
61
 Problem Solving Exercise
 Prob. 3.40
1 3
 Suppose V is a Rayleigh random variable 0 2 2 v

fV (v)  v exp( )u (v) FV (v)  1  exp( ) u (v)

v2
2  v2
2

 Find the conditional CDF 𝐹𝑉|{1<𝑉<3} (𝑣). fV |{ 1 V  3 } (v) 
fV (v )
( 1
 V  3
2)
2 2 2 2
FV ( 32 )  FV ( 12 ) 2
v
 fV (u )du
 12  v  32 
v
FV |{ 1 V  3 } (v)   fV |{ 1 V  3 } (u )du 
1
2
2 2  2 2
exp( )  exp( )
1
8
9
8
v 1

 fV (u )du   fV (u )du FV (v)  FV ( 12 )

2

 0 0 
exp( 18 )  exp( 89 ) exp( 18 )  exp( 89 )
 0 V 1
2

exp( 18 )  exp( v2 )
2
 exp( 1 )  exp( v 2 )
 FV |{ 1 V  3 }   V 
8 2 1 3

 exp( 8 )  exp( 8 )
1 9 2 2

exp( 18 )  exp( 89 )
2 2

 1 V 3
2

62
Extension of Bayes’ Theorem
 Extended total probability theorem.
 Let 𝐵1 , 𝐵2 , … . , 𝐵𝑛 be a set of mutually
exclusive and exhaustive events
n
f X ( x)   f X |Bi ( x) Pr( Bi )
i 1
 Given the event that the random variable 𝑋 = 𝑥, we can
compute the probability of each possible event as follows:
Pr( X  x | Bi ) Pr( Bi )
 If X is discrete, we have Pr( Bi | X  x)  n

 Pr( X  x | B ) Pr( B )
j 1
j j

f X (x)
f X |Bi ( x) Pr( Bi )
 If X is continuous, we have Pr( Bi | X  x)  n

fj 1
X |B j ( x) Pr( B j )
63
Problem Solving Exercise
 ( x  m) 2 
f X x    X ~ N (m,  2 )
1
 Example 2 2
exp 
 2 2


 Considering the entry-level salary distribution ($K) for three different
education groups, High-school diploma (30%), College BS (50%), and
Graduate degree MS/PhD (GA, 20%), each of which follows a Gaussian
distribution as follows, answer the following questions.
f X |HS ( x)  N (30,10 2 ), f X |BS ( x)  N (60,152 ), f X |GA ( x)  N (80,20 2 )
 What is the overall salary distribution for all groups?
n
f X ( x)   f X |Bi ( x) Pr( Bi )
i 1

 0.3  N (30,10 2 )  0.5  N (60,152 )  0.2  N (80,20 2 )

 If Tom has the first job of $70K, what is the probability, he is not a HS graduate?
Pr(not HS | X  70)  1  Pr( HS | X  70) f X |Bi ( x) Pr( Bi )
f X |HS (70) * 0.3 Pr( Bi | X  x) 
 1  0.9997 f X ( x)
f X (70)
64
Overview
 Lifetime PDF, reliability function, and failure rate
Lifetime Reliability Function Failure Rate
PDF

0 t t  dt
 What is the life-time distribution a certain computer ?
 What is the probability a new computer can work for 5 years?
 For a 5-year old computer, how likely it will break soon?
 Major Issues of System Reliability Analysis
PDF or CDF
X (lifetime)

Reliability Failure
Function Rate
65
Reliability Function
 Definitions 3.6
 Let X be a random variable which represents the lifetime of a device.
That is, if the device is turned on at time zero, X would represent the
time at which the device fails. The reliability function of the device, 𝑅𝑋 (𝑡),
is simply the probability that the device is still functioning at time t:

RX (t )  Pr( X  t )
 Relationship with CDF and PDF
 We can represent the reliability function as the complement of the CDF.
RX (t )  1  Pr( X  t )  1  FX (t )
 Often it is more convenient to work with PDF rather than CDF, then we
can take derivative on both sides,
RX' (t )   f X (t )
66
Failure Rate
 Definition 3.7
 Let X be a random variable which represents the lifetime of a device.
The failure rate function that indicates the conditional probability density
of X given that the device has survived until time t is defined as
r (t )  f X |{ X t } ( x) | x t Is r(t) a PDF?

 From which we can derive the probability that the device will fail in the
next time instant of length dt.
Pr(t  X  t  dt | X  t )  r (t )dt
 Relationship with CDF and PDF
f X ( x )u ( x  t ) f X (t )
r (t )  f X |{ X t } ( x ) |x t  
Pr( X  t ) x t 1  FX (t )
 Animals normally have an increasing failure rate function.
67
System Reliability Analysis: Example
 x2    x2  
f X  x   2 exp 
x
FX  x   1  exp   u ( x)
 Example   2
u ( x )
2 
   2 2 

 Suppose the lifetime of an animal specie follows a Raleigh distribution as
f X ( x )  2bx exp( bx )u(t )  FX x   1  exp  bx u( x )
2 2 b  1.4  104

X   75
4b
 What are the reliability function and the failure rate function?
RX (t )  Pr( X  t ) 1  FX (t )  exp(bt )u (t )
2
f X (t )
r (t ) 
 What is the failure rate? 1  FX (t )
2bt exp(bt 2 )
r (t )  f X |{ X t } ( x)  u (t )  2btu (t ) The failure rate is linearly
x t exp(bt )
2
increasing in time.
0.012 1 0.06
f X (x ) RX (t ) r (t )
0.01 0.8 0.05

0.008 0.04

0.006
0.6 So what?
0.03
Is r(t) a PDF?
0.4
0.004 0.02 So what?
0.002 0.2 0.01

0 0 0
0 50 100 150 200 0 50 100 150 200 0 50 100 150 200

68
System Reliability Analysis: Example
 Example
 Suppose the lifetime of human follows a Raleigh distribution given by
f X ( x )  2bx exp( bx )u(t )  FX  x   1  exp bx u( x )
2 2 b  1.4  104

X   75
4b
 What is the conditional PDF in terms of the function of t, 𝑓𝑋|{𝑋>𝑡} (𝑥)?
f X ( x) 2 bx exp(  bx 2
)
f X |{ X t }  x  
f X ( x) f X ( x)
   ( X  t)
Pr( X  t ) 1  Pr( X  t ) 1  FX (t ) e bt 2

0.03 0.03 0.03 0.03

f X |{ X t } (t  0) f X |{ X t } (t  50) f X |{ X t } (t  75) f X |{ X t }
0.025 0.025 0.025 0.025
(t  100)
0.02 0.02 0.02 0.02

0.015 0.015 0.015 0.015

0.01 0.01 0.01 0.01

0.005 0.005 0.005 0.005

0 0 0 0
0 100 200 0 100 200 0 100 200 0 100 200

Recall : r (t )  f X |{ X t } ( x ) |x t
69
Complex System Reliability Analysis:
Series Case
 If we have a system with N components, each of which has a
lifetime described by the random variable, Xn, n=1,2,…,N.
 For the system to function, all N components must be functioning.
 Given the reliability function 𝑅𝑋𝑛 𝑡 and the failure rate 𝑟𝑛 𝑡 of each
individual component, can we calculate the same functions for the whole
system, i.e., 𝑅𝑋 𝑡 and 𝑟(𝑡)?

Component Component Component Component

1 2 3 N

 The system lifetime is defined a random variable X.

RX (t )  RX1 (t ) RX 2 (t )...RX N (t )
X  min( X 1 , X 2 ..., X N ) N
r (t )   rn (t )
n 1
70
Complex System Reliability Analysis:
Parallel Case
 Consider a system which consists of a parallel interconnection
of components, and the system will be functional as long as any
of the components are functional.
Component 1

Component 2

Component N

 The system lifetime is defined a random variable X.

 
N
RX (t )  1   1  RX n (t )
X  max( X 1 , X 2 ..., X N ) n 1
 1 N rn (t )
r (t )    1
 R X (t )  n 1 1
1
RX n (t )
71
Problem Solving Exercise
 RX' (t )

 exp   r (u )du u (t )
t
 Prob. 3.41 r (t )   R X (t )
 0
RX (t ) 
 A serial connection system has 10 components and the failure rate
function is the same constant for all component and is 1 per 100 days.
 Determine the probability that the lifetime of the system exceeds 10 days.
 t
  t 1 
RX n (t )  exp   r (u )du u(t )  exp   du u(t )  e 0.01t u (t ) ( n  1,...,10)
 0   0 100 
RX (t )  RX 1 (t ) RX 2 (t )... RX N (t )  e 0.1t u(t ) RX (10)  e 1  0.3679

 What is the probability that the lifetime of one component exceeds 10 days?

RX n (t )  e 0.01t u(t ) RX n (10)  e 0.1  0.9048

 What is the failure rate of the whole system?
N 10
1 1
r (t )   rn (t )   
n 1 n 1 100 10
72