Chapter 2.

Discrete Distributions

1. Discrete Random Variables and Probability Distributions

(1) A random variable  is a real-valued function defined on a sample space  . The range of
 is the set of real numbers          ∈  .

(2) Let  be a random variable defined on a sample space  . If  is a subset of the set of real
numbers, then the set ∈     ∈   is an event in  and    ∈      ∈     ∈   .

(3) Let  be the sum of upturned faces of two dice.


       ⋯     ,         ,      ⋯   ,     ⋯   



   ∈  ∞     ≤             



   ∈        ≤  ≤                   


(4) If the range of  is a countable set, then  is called a discrete random variable.

(5) The indicator function of  is defined by

  ∈        
 for ∈ 
.
otherwise

(a)             for   

 otherwise

(b)                     for     

otherwise


(c)         ∞      for   
otherwise

(6) Maclaurin series for   

∞
     
    

 
 ,      :  th derivative of   

∞
     
(a)  



converges if   ,

  lim  .
 
→ ∞ 

1
 ∞

(b)     
   

∞
(c)       


,   (geometric series)

∞
(d)            


,   ,  : positive integer (negative binomial series)

∞
    
     ⋯       
(e)      
 
 
,    ,  : real number (binomial series)

(7) For a discrete random variable  , the function

        

is called the probability mass function (pmf) of  .

(8) The pmf    of a discrete random variable  satisfies the following properties:

(a)    ≥  ;

(b)      ;


(c)   ∈       ,
∈ 
where  is a subset of the set of real numbers.

(9) Let  be the number of heads in four tosses of a coin. The sample space for this

experiment has  points, which we assume to be equally likely. Then

 
           TTTT   ,            HTTT THTT TTHT TTTH    ,
 

and similarly

  
           ,            , and            .
  

That is, the pmf of  can be written as

      
 

           or       
 

         .

(10) For a discrete random variable  with pmf    ,

         

is called the support of  .

2
(11) The function

      ≤  , ∞∞

is called the cumulative distribution function (cdf) of a random variable  .

(12) Let  be the number of heads in four tosses of a coin. Then the cdf of  is given by


  
  ≤
  ≤


    
   ≤
   ≤
  ≤

   
                              ∞    .
   

This is a step function with jumps of size    for        .

(13) Let  be a discrete random variable with pmf    and support        ⋯ . Then

the cdf    is a step function with jumps of size    for ∈   .

2. Expectations and Moments

(1) Let  be a discrete random variable with pmf   . The mathematical expectation or
expected value or mean of  , denoted by  or    , is given by

      


provided that     ∞ .


If     ∞ ,

we say that    does not exist.

(2) Let  be the number of heads in four tosses of a coin. Then

  

                           .
      
(3) Let  be a discrete random variable with pmf   and    be a real-valued function of  .
The mathematical expectation or expected value of    , denoted by       , is given by

           


provided that the sum exists.

3
(4) The mathematical expectation  satisfies the following properties:

(a) If  is a constant, then    .

(b) If  is a constant and  is a function, then              .

(c) If  and  are constants and   and   are functions, then

                               

Proof: (a)               

∈ ∈ 

(b)                          
∈   ∈ 

(c)               
∈
              
   


∈
              
 
∈ 
 

                 

(5) Property (c) can be extended to more than two terms by mathematical induction; that is,

   
              
   
    

(6) Let  be a positive integer. The expectation

       ′

is called the  th moment of (the distribution of)  . In addition, the expectation

       

is called the  th central moment of  and

          ⋯       

is called the  th factorial moment of  .

(7) If  is a random variable with mean      , the variance of  is defined to be

the expected value of    . That is,

            .

The standard deviation of  is the positive square root of     ,

  
   .

4
(8) Let  be a random variable with mean      , then

          

Proof:                    

                    

(9) Let  be the number of heads in four tosses of a coin.

   

                           ,
     
                     , 

(10) If  is a random variable with the mean  and the variance  , then the mean and

the variance of      are      and     .

Proof:                    

              

                                   

(11) The moment generating function (mgf)    of a random variable  is defined to be the

expected value of    if it exists for all  in the interval       for some    .

That is,          .

(12) If    exists, then    is continuously differentiable in some neighborhood of the origin.

If we differentiate  times   with respect to  , we have

    
 
                           ,
    

and letting    , we find

  
 ′               .
    

(13) If we replace   by its series expansion

∞ ∞
   
    
   
   ,
   

we obtain the series expansion of    in terms of the moments of  ;

5
  ∞ 
  ∞
    ∞
    
                
           
            

(14) If  is a discrete random variable with        ⋯  and pmf            , then

∞ ∞
    
 

       
 

 
.

(15) Let  be the number of heads in four tosses of a coin. Then


  
     



                      ,
 

     
 ′          ,  ′′                    ,
  

     ′     ,      ′′     ,           .

(16) If  has the mgf

  

              ,
 
 ∞    ∞,

then the support of  is       and the associated probabilities are

  
       ,        ,        .
  

(17) Suppose the mgf of  is

  
     ,   ln .
 
  

Recalling that

                ⋯ ,      ,

we have



 



 


    



 
     


 


 ⋯ 
 

  
         
     ⋯

    


when      and thus   ln . That is,

 
  
     ,        ⋯ .

6
(18) Let  and  be two random variables with cdf  and  , respectively. Suppose that

    and     both exist and are equal for all  in the interval       for some

   . Then the two cumulative distribution functions  and  are equal. That is,

 and  have the same distribution.

3. Discrete Uniform Distribution

(1)  has a discrete uniform distribution on  points   ⋯    if the pmf of  is given by


        ⋯       .


(2) Let  have a discrete uniform distribution on   ⋯   .

 
 
(a)     
  
     
 

 
   

 
   
 
 
   
(b)              
  
   
  
   
  


(3) If  has a discrete uniform distribution on   ⋯  , then


       
      

      ,
   


               
     





 
    ,
  

and

             
                      .
  

4. Binomial Distribution

(1) A random experiment with only two possible outcomes success and failure is called
a Bernoulli trial. Let  be the number of successes in  independent Bernoulli trials
with success probability  . Then the possible values of  are    ⋯   . If  successes

occur, where   ⋯ , then    failures occur. The number of ways selecting 
positions for the  successes in the  trials is



   



. 
Since the trials are independent and since the probabilities of success and failure on

7
 each trial are, respectively,  and      , the probability of each of these ways is

    . Thus,    , the pmf of  is the sum of the probabilities of the  mutually
exclusive events; that is,

       
,     ,      ⋯   .

(2) The probability distribution of  is called a binomial distribution with parameters  and  ,

 ∼     , if the pmf of  is given by

          
   ⋯      ,  ≤  ≤ ,     ,     ⋯

(3)  and  play a game in which their chances of winning are in the ratio    . Find  ’s
chance of winning at least three games out of five games played.

Solution: Let  be the probability that  wins the game and let  be the number of games

that  wins out of five games played. Then we are given    and  has the



binomial distribution with parameters    and    . Hence, the required


probability that  wins at least three games is given by

   

       

  ≥     
               .
  

(4) Recall that if  is a positive integer, then


    

    
.

Thus, if we use this binomial expansion with    and        , then the sum of the
binomial probabilities is

 



     

    
       .

(5) If  has a     distribution, then

     and           .

Proof: The mean of the     random variable  is

 
    

       

 
.

8
 Since the first term of this summation equals zero when    , and since

     ,

we can write

 

      

   

  
     

  
.

If we factor  out of each term in the sum and let      , then

   
           
    
 
 
  
  
  
 


          .

Notice that        is the binomial pmf with parameters    and  .

The second factorial moment of  is


             

 
.

The first and second terms of this sum equal zero when    and    . If we use

       
  
        

and let      , then


    
 
        
         
        



          

   


                     .



Thus

               

                 

          

     

(6) We now use the binomial expansion to find the mgf for a     random variable 
and then the mean and variance. The mgf is

9
  
         

   
  

  
 
   
        
      ,  ∞    ∞.

The first two derivatives of    are

 
 ′         

and
    
 ′′                        .

Thus,

     ′    

and

                 ′′      ′                      .

(7) If  ∼   and let      , then the pmf of  is

                    
              
 
,        ⋯   .

Thus      has   distribution.


(8) The mgf of a random variable  is      . Find     ≤   .
 
 

Solution: Since


  

                ,
 


 has   
    distribution. Thus
 
     ,      
 

and



    
     .
Hence,

    ≤      ≤ 
  
     ≤  


                         .


10
5. Hypergeometric Distribution

(1) Consider a collection of  objects which can be classified into two classes, say class 
and class  . Suppose that there are  objects in class  and    objects in class  .

A collection of  objects is selected from these  objects at random and without

replacement. Let  denote the number of class  objects in a sample of size  . We are
interested in finding out the probability that exactly  of these  objects are from class  .

If  of these  objects are from class  , then the remaining    objects must be from

class  . We can select  objects from class  in any one of  ways. Similarly, the 

 
remaining    objects can be selected in    ways. Thus, the number of ways one can
 

select a subset of  objects from a set of  objects, such that  number of objects will be

from class  and    number of objects will be from class  , is given by     .
     
Hence

 
      
       ,  ≤  ≤ ,  ≤ ,   ≤   .
 
(2) A random variable  is said to have a hypergeometric distribution with parameters      ,

 ∼        , if the pmf of  is of the form

 
      
          .
  ⋯    
 

Binomial coefficient  have been defined only if  ≥  , and so the range of  is

additionally restricted by the pair of inequalities

≥ and    ≥   ,

which can be combined as

      ≤  ≤  .

In many cases  is small compared to  and  (  ≤  and  ≤    ), so the range

 ≤  ≤  will be contained in the range       ≤  ≤  .

11
(3) If we consider           and     , we can prove that

       .



Thus,

 
       

  
    


             
 .
 



 

(4) A lot consisting of  bulbs is inspected by taking at random  bulbs and testing them.
If the number of defective bulb is at most  , the lot is accepted; otherwise, it is rejected.
If there are, in fact,  defective bulbs in the lot, then

   
 
    
 
 accept the lot       .
 
(5) If  ∼        , then

 

    
   and  

          .
    
Proof: We compute the mean and variance of  by computing the expected value and
the second factorial moment of  . Since

    
    
     
 

 
and       ,
the expected value of  is given by

 

   
  
    
 
 
    


  

 
 
  
 
               ,

 
where     

 
  .


12
 Similarly, since

        
            
         
     


and

       
   
         
,

the second factorial moment of  is given by

 
    

     
         
 
 
 

                 

      


   
 
        

     

             , where     

        
 .
    

Therefore, the variance of  is

                  

             
           .

      

        

(6) If we define    we can re-express the mean and variance of the hypergeometric


distribution as


     and 
       .
 
6. Geometric and Negative Binomial Distribution

(1) The probability distribution of  is called a geometric distribution with parameter  ,

 ∼    , if the pmf of  is given by

           ⋯    ,    ≤ ,     .

13
(2) Recall that for a geometric series, the sum is given by

∞




 
 

when      . Thus, for the geometric distribution,

∞ ∞ ∞



     

    


   .


(3) Let  be the number of failures prior to the first success in a sequence of independent
Bernoulli trials with success probability  . Then the pmf of  is given by

             failures and a success     ,        ⋯ .

Hence  has the    distribution.

(4) Memoryless property of the geometric distribution

If  has a geometric distribution with parameter  , then

   ≥     ≥       ≥   for       ⋯ .

The geometric distribution “forgets” what has occurred. The probability of getting an
additional  failures, having already observed  failures, is the same as the probability
of observing  failures at the start of the sequence. In other words, the probability of

getting a run of failures depends only on the length of the run, not on its position.

Proof: We first note that for any integer  ≥  ,

∞ ∞

  ≥    

   

     ,

       ⋯ ,

and hence

  ≥      ≥  
   ≥     ≥    
  ≥  

  ≥    

  ≥  

  



      ≥  

14
(5) The probability that a machine produces a defective item is  . Each item is checked as
it is produced. Assuming that these are independent trials, what is the probability that at

least  items must be checked to find one that is defective?

Solution: Let  denote the trial number on which the first defective item is observed.
Then the number of failures prior to the first success is    which has

   distribution. Hence

  ≥        ≥        ≥       

(6) A random variable  has a negative binomial distribution with parameters  and  ,

 ∼   , if its pmf is of the form

            
 
   ⋯    ,    ≤ ,     ,   ⋯ .

(7) Let  be the number of failures before the  th success in a sequence of independent
Bernoulli trials with success probability  . Then

           successes in the first      trials   th success

       
 

       ,  
  ⋯ .

Thus  ∼     .

(8) Using Maclaurin’s series expansion, we have the negative binomial series

∞
           


,     .

Hence the sum of the probabilities for the negative binomial distribution is  because

∞ ∞ ∞

             
 
 
       


         .

(9) If    , the     distribution is the    distribution.

(10) If  ∼    , then


    ,

     ln     ,

     ,


      .


15
 Proof: The mgf of  is

∞
          

  

∞
        

 


       ,   ln          

 
  ,
      ln    .

Thus

             

 ′    
 
and  ′′    
 
 
 
.
           

Hence, we have

  
     ′      ,
     

         
     ′′      
 
,
  

and

      
     
 
 
.
  

(11) If  ∼    , then

  
     ,   ln     ,      ,       .

   

(12) The negative binomial distribution is sometimes defined in terms of the random variable
  number of trials until  th success. This formulation is statistically equivalent to

the one given in terms of   number of failures before the  th success, since      .

Using the relationship between  and  , the pmf of the alternative form of the negative
binomial distribution is

                          
  
        ⋯    .

16
(13) The probability that the  th head is observed on the  th independent flip of a coin is
given by

    
          
       


   
 
 


     ,


where  is the number of trials needed to observe  th head.

7. Poisson Distribution

(1) A discrete random variable  is said to have a Poisson distribution with parameter  ,

 ∼    , if the pmf of  is given by

   
         ⋯    ,   .


(2) From the Maclaurin series expansion of   , we have

∞ ∞ ∞
    


      

 
  

   

      .

(3) If  ∼    , then


        
,      ,       .

Proof: First, we find the moment generating function of  .

∞ ∞ 
      
 
 
                                .
   

Thus,

 ′        
       and      ′         .

Similarly,

 ′′     ′         and      ′′       .

Hence

            .

(4) The number of traffic accidents per week in a small city has a Poisson distribution with
the mean equal to  . Find the probability that  accidents occur in  weeks.

Solution: Let  be the number of accidents in two weeks. The mean number of accidents
in two weeks is        . Hence,  ∼   and

17
    
              .


(5) Let  ∼    . Find the conditional probability   ≥    ≤   .

  ≥    ≤  
Solution:   ≥    ≤    
  ≤  

  ≤  ≤  

  ≤  

   
  

 ,     


  


   
        
   

    
            
    





18