Discrete Random Variables

Randomness

• The word random effectively means

unpredictable
• In engineering practice we may treat some
signals as random to simplify the analysis
even though they may not actually be
random
 Random Variable Defined

A random variable X (
) is the assignment of numerical

values to the outcomes
of experiments
 Random Variables
Examples of assignments of numbers to the outcomes of
experiments.
Discrete-Value vs Continuous-
Value Random Variables
• A discrete-value (DV) random variable has a set
of distinct values separated by values that cannot
occur
• A random variable associated with the outcomes
of coin flips, card draws, dice tosses, etc... would
be DV random variable
• A continuous-value (CV) random variable may
take on any value in a continuum of values which
may be finite or infinite in size
 Probability Mass Functions

The probability mass function (pmf ) for a discrete random

variable X is
()
PX x = P
 X = x  .
 Probability Mass Functions
A DV random variable X is a Bernoulli random variable if it
takes on only two values 0 and 1 and its pmf is
1
p , x = 0

PX ()
x = p , x =1
0 , otherwise

and 0 < p < 1.
Probability Mass Functions

Example of a Bernoulli pmf

 Probability Mass Functions
If we perform n trials of an experiment whose outcome is
Bernoulli distributed and if X represents the total number of 1’s
that occur in those n trials, then X is said to be a Binomial random
variable and its pmf is

 n
x
( ) { }
n
x
  p 1
p , x  0,1,2,
, n
PX ()
x =  x 
0
 , otherwise
Probability Mass Functions

Binomial pmf
 Probability Mass Functions
If we perform Bernoulli trials until a 1 (success) occurs and the
probability of a 1 on any single trial is p, the probability that the
( )
k
1
first success will occur on the kth trial is p 1
p . A DV random
variable X is said to be a Geometric random variable if its pmf is

(
 p 1
p
) { }
x
1
, x  1,2,3,...
PX ()
x =
0 , otherwise
Probability Mass Functions
Geometric pmf
 Probability Mass Functions
If we perform Bernoulli trials until the rth 1 occurs and the
probability of a 1 on any single trial is p, the probability that the
rth success will occur on the kth trial is
 k
1 r
( ) ( )
k
r
P rth success on kth trial =   p 1
p .
 r
1
A DV random variable Y is said to be a negative - Binomial
or Pascal random variable with parameters r and p if its pmf is
 y
1 r
( ) { }
y
r

 , y  r,r + 1,
,
( )  p 1
p
PY y =  r
1

0
, otherwise
Probability Mass Functions
Negative Binomial
(Pascal) pmf
 Probability Mass Functions
Suppose we randomly place n points in the time interval 0  t < T
with each point being equally likely to fall anywhere in that range.
The probability that k of them fall inside an interval of length t < T
inside that range is
 n k
( ) n!
( )
n
k n
k
P
 k inside t  =  p 1
p = p k 1
p
k ( )
k! n
k !
where p = t / T is the probability that any single point falls within
t . Further, suppose that as n  , n / T =  , a constant. If 
is constant and n   that implies that T   and p  0. Then 
is the average number of points per unit time, over all time.
Probability Mass Functions
Events occurring at random times
 Probability Mass Functions
It can be shown that
n
k
    k

P  k inside t  = lim 1
 = e
k! n
n k!


=e


where  = t. A DV random variable is a Poisson random

variable with parameter  if its pmf is
 x

PX ()
x =  x! {
 e , x  0,1,2,, }
0
 , otherwise
 Cumulative Distribution
Functions
The cumulative distribution function (CDF) is defined by
()
FX x = P  X  x  .
For example, the CDF for tossing a single die is
( ) ( ) ( ) 
u x
1 + u x
2 + u x
3
() ( )
FX x = 1 / 6 
( ) ( ) ( )
 + u x
4 + u x
5 + u x
6
1 , x  0
()
where u x 

0 , x < 0
Functions of a Random Variable

Consider a transformation from a DV random variable X

( )
to another DV random variable Y through Y = g X . If the
( )
function g is invertible, then X = g
1 Y and the pmf for Y is

( ) ( ( )) where P ( x ) is the pmf for X.

PY y = PX g
1 y X
 Functions of a Random Variable
If the function g is not invertible the pmf and pdf of Y can be found
by finding the probability of each value of Y. Each value of X with
non-zero probability causes a non-zero probability for the
corresponding value of Y. So, for the ith value of Y ,
P Y = yi  = P  X = xi,1  + P  X = xi,2  +

n
+ P  X = xi,n  =
P  X = xi,k 
k =1

The function to the right is an

example of a non-invertible
function.
 Expectation and Moments
Imagine an experiment with M possible distinct outcomes
performed N times. The average of those N outcomes is
1 M
X =  ni xi where x i is the ith distinct value of X and ni
N i=1
is the number of times that value occurred. Then
1 M M
ni M
X =  ni xi =  xi =  ri xi
N i=1 i=1 N i=1

The expected value of X is

M M M
ni
E  X  = lim  xi = lim  ri xi =  P  X = xi  xi
N 
N 

i=1 N i=1 i=1
 Expectation and Moments
Three common measures are used in statistics to indicate
an "average" of a random variable are the mean, the
mode and the median. The mean is the sum of the values
1 M
divided by the number of values X =  ni xi .
N i=1
The mode is the value that occurs most often.
( ) ()
PX xmode
PX x for all x.
The median is the value for which an equal number
of values fall above and below.
( ) (
PX X > xmedian = PX X < xmedian )
 Expectation and Moments
The first moment of a random variable is its expected value
M
E  X  =
xi P  X = xi 
i=1

The second moment of a random variable is its mean-squared

value (which is the mean of its square, not the square of its
mean).
M
E  X 2  =
xi2 P  X = xi 
i=1

The name "moment" comes from the fact that it is mathematically

the same as a moment in classical mechanics.
 Expectation and Moments

The nth moment of a random variable is defined by

M
E  X n  =
xin P  X = xi 
i=1

The expected value of a function g of a random variable is

( ) ( )
M
E  g X  =
g X P  X = xi 
i=1
 Expectation and Moments
A central moment of a random variable is the moment of
that random variable after its expected value is subtracted.

( )
( )
M
E X
E  X  =  xi
E  X  P  X = xi 
n n

  i=1
The first central moment is always zero. The second central
moment (for real-valued random variables) is the variance,

( )

( )
M
 = E X
E  X  =  xi
E  X  P  X = xi 
2 2
2
X   i=1
The variance of X can also be written as Var  X  . The positive
square root of the variance is the standard deviation.
 Expectation and Moments
Properties of expectation
 
E  a = a , E  aX = a E  X , E   X n  =  E  X n 
     
 n n
where a is a constant. These properties can be use to prove
the handy relationship,
 2X = E  X 2 
E 2  X 
The variance of a random variable is the mean of its square
minus the square of its mean. Another handy relation is
Var  aX + b  = a 2 Var  X  .
 Conditional Probability Mass
Functions
The concept of conditional probability can be extended to a
conditional probability mass function defined by

()
 PX x
, x A
PX |A ()
x =  P  A


0 , otherwise
where A is the condition that affects the probability of X .
Similarly the conditional expected value of X is
()
E  X | A
 =  x PX |A x and the conditional cumulative
xB

()
distribution function for X is FX |A x = P  X
x | A
 .
 Conditional Probability
{ }
Let A be A = X
a where a is a constant.

( ) ( )
P  X
x  X
a 
Then FX |A ()
x = P  X
x | X
a  =
P  X
a 
.

( ) ( )
If a
x then P  X
x  X
a  = P  X
a  and
P  X
a 
()
FX |A x = P  X
x | X
a  =
P  X
a 
= 1.

( ) ( )
If a  x then P  X
x  X
a  = P  X
x  and

()
FX |A x = P  X
x | X
a  = =
()
P  X
x  FX x
()
P  X
a  FX a