Discrete Probability Distributions

Maram Salem & Noha Youssef

The American University in Cairo

Maram Salem & Noha Youssef (AUC) 1 / 32

Important Definitions

Random Variable
A random variable Y Y is said to be discrete if it can assume only a finite
or countably infinite number of distinct values. The probability that Y
takes on the value y, P (Y =yy), is defined as the sum of the probabilities
of all sample points in S that are assigned the value y. We will sometimes
denote P (Y = y) by p(y). The probability distribution for a discrete
variable Y can be represented by a formula, a table, or a graph that
provides p(y) = P (Y = y) for all y.

For any discrete probability distribution, the following must be true:

1. 0 ≤ p(y) ≤ 1 for all y.
P
y p(y) = 1, where the summation is over all values of y with
2.

non-zero probability.

Maram Salem & Noha Youssef (AUC) 2 / 32

Important Definitions

Expected Value
P
It is defined to be E(Y ) = y yp(y) = µ. The expected value has the
following properties

E(c) = c. P P
Proof:E(c) = y cp(y) = c y p(y) = c × 1 = c.
E(cg(y) = cE(g(y)).P P
Proof: E(cg(y)) = y cg(y)p(y) = c y g(y)p(y) = cE(g(y)).
E( ki=1 gi (y)) = ki=1 E(gi (y)).
P P
Proof: let k = 2, then
P P
E(g1 (y) + g2P
(y)) = y (g1 (y)P+ g2 (y))p(y) = y g1 (y)p(y) +
g2 (y)p(y) = y g1 (y)p(y) + y g2 (y)p(y) = E(g1 (y)) + E(g2 (y)).

Maram Salem & Noha Youssef (AUC) 3 / 32

Important Definition

Varince
It is defined to be V ar(Y ) = E(Y − µ)2 = E(Y 2 ) − µ2 .
Proof:V ar(Y ) = E(Y − µ)2 = E(Y 2 − 2Y µ + µ2 ) =
E(Y 2 ) − 2µE(Y ) + E(µ2 ) = E(Y 2 ) − 2µ2 + µ2 = E(Y 2 ) − µ2 .

Moment Generating Function

It is defined as MYy (t) = E(eY t ) = y eyt p(y). This function is used to
P
obtain different moments by differentiating the moment generating
function with respect to t and and then substitute with t = 0. E.g. To get
the first moment, obtain the first derivative and then replace t = 0. No
two distributions share the same moment generating function.
Note that: Uniqueness
1. The Standard deviation is the positive square root of the variance.
2. The rth moment is : E(Xr).
Maram Salem & Noha Youssef (AUC) 4 / 32
Famous Discrete Probability Distributions

For each distribution we are going to prove that the the

determine distribution is a prob-
form of the
ability distribution. In addition we need to know how to obtain the mean,
median,
the variance and moment generating function. The distributions we
are going to study are Binomial, Geometric, Negative Binomial, Hyper Ge-
ometric and Poisson.

Maram Salem & Noha Youssef (AUC) 5 / 32

Binomial Distribution

A binomial experiment possesses the following properties

1. The experiment consists of a fixed number, n, of identical trials.
2. Each trial results in one of two outcomes: success, S, or failure, F .
3. The probability of success on a single trial is equal to some value p
and remains the same from trial to trial. The probability of a failure is
equal to q = (1 − p).
4. The trials are independent.
5. The random variable of interest is Y , the number of successes
observed during the n trials.

Maram Salem & Noha Youssef (AUC) 6 / 32

Binomial Distribution

A Binomial Variable
A random variable Y is said to have a binomial distribution based on n
trials with success probability p if and only if

P (Y = y) = p(y) = cny py q n−y , y = 0, · · · , n, and 0 ≤ p ≤ 1.

The expected value of the binomial distribution is E(Y ) = np and the

variance is var(Y ) = npq. The moment generating function of the
binomial distribution is M (t)y = (et p + q)n .
Y

Maram Salem & Noha Youssef (AUC) 7 / 32

Binomial Distribution Proofs

To find the mean

M 0 (t) = n(et p + q)n−1 et p

E(X) = M 0 (0) = np

To find the variance, we find E(X 2 ) using the moment generating function

M 00 (t) = n(n − 1)(et p + q)n−2 et pet p + n(et p + q)n−1 et p

E(X 2 ) = M 00 (0) = n(n − 1)p2 + np
V ar(X) = E(X 2 ) − E 2 (X) = n2 p2 − np2 + np − n2 p2 = np(1 − p) = np

= npq

Maram Salem & Noha Youssef (AUC) 8 / 32

Example 1
Suppose that 40% of a large population of registered voters favour candidate
Jones. A random sample of 10 voters will be selected, and the number
favouring Jones is to be observed.
(a) What is the probability that three will favour Jones?
Sol:

p(3) = c10 3
3 0.4 0.6
7

(b) What is the probability that at most two will favour Jones?
Sol:

p(Y ≤ 2) = p(0)+p(1)+p(2) = c10 0 10 10 2 9 10 2

0 0.4 0.6 +c1 0.4 0.6 +c2 0.4 0.6
8

(c) What is the probability that at least three will favour Jones?
Sol:
P (Y ≥ 3) = 1 − P (Y ≤ 2)
Maram Salem & Noha Youssef (AUC) 9 / 32
Example 1

(d) What is the probability that all will not favour Jones?
Sol:
P (Y = 0).
(e) How many voters would you expect to favour Jones?
Sol:
E(Y ) = np = 10 ∗ 0.4 = 4voters.
(f) Find the standard deviation
Sol: √ √
p √
var(Y ) = npq = 10 ∗ 0.4 ∗ 0.6 = 2.4.

Maram Salem & Noha Youssef (AUC) 10 / 32

Example 2

If the moment generating function of Y is given as

1 2
M (t) = ( et + )5 ,
3 3
Find P (Y < 2).
Sol:
n = 5 and p = 13 By the uniqueness of the mgf .... The dist. is binomial

1 2 1 2
P (Y < 2) = p(0) + p(1) = c50 ( )0 ( )5 + c51 ( )1 ( )4 .
3 3 3 3

Maram Salem & Noha Youssef (AUC) 11 / 32

Geometric Distribution

The Geometric distribution has the following properties

1. The experiment consists of y identical independent trials.
2. Each trial results in one of two outcomes: success, S, or failure, F .
3. The probability of success on a single trial is equal to some value p
and remains the same from trial to trial. The probability of a failure is
equal to q = (1 − p).
4. The random variable of interest is Y , the number of trials till
we reach the first success.

Maram Salem & Noha Youssef (AUC) 12 / 32

Geometric Distribution

A Geometric Variable
A random variable Y is said to have a geometric probability distribution if
and only if

p(y) = q y−1 p, y = 1, 2, 3, · · · , 0 ≤ p ≤ 1.

The expected value E(Y ) = p1 and the variance is var(Y ) = 1−p

p2
.
The moment generating function for the geometric distribution
pet
M (t)y = 1−qe t.

Maram Salem & Noha Youssef (AUC) 13 / 32

Example 3
Suppose that 30% of the applicants for a certain industrial job possess
advanced training in computer programming. Applicants are interviewed
sequentially and are selected at random.
(a) Find the probability that the first applicant with advanced training in
programming is found on the fifth interview.
Sol:

p(5) = 0.3 ∗ 0.74 .

(b) Find the probability that at least five people have to be interviewed to
find the first applicant with advanced training in programming.
Sol:

P (Y ≥ 5) = 1 − P (Y ≤ 4)
= 1 − 0.3 + 0.3 ∗ 0.7 + 0.3 ∗ 0.72 + 0.3 ∗ 0.73
 

= 1 − 0.7599 = 0.2401.
Maram Salem & Noha Youssef (AUC) 14 / 32
Example 3

(c) How many interviews would you expect to do to find the first
applicant with advanced training in programming? What is the
standard deviation?
Sol:

r r
1 1 q 0.7
E(Y ) = = = 3.3 and sd = = = 2.867.
p 0.3 p2 0.32

Maram Salem & Noha Youssef (AUC) 15 / 32

Geometric Distribution Proofs

1 − qet pet − pet −qet

0
M (t) =
(1 − qet )2
E(X) = M 0 (0) = 1/p.
For the variance find E(X 2 ),

2
1 − qet pet − pet 2 1 − qe)t −qet



M 00 (t) =
(1 − qet )4
2
1 − qet pet + 2pet qet 1 − qet



=
(1 − qet )4
1+q
M 00 (0) =
p2
(1 + q) 1 q
Var(X) = E X 2 − [E(X)]2 =


2
− 2 ⇒ 2.
p p p
Maram Salem & Noha Youssef (AUC) 16 / 32
Negative Binomial Distribution

The Negative Binomial distribution has the following properties

1. The experiment consists of y identical independent trials.
2. Each trial results in one of two outcomes: success, S, or failure, F .
3. The probability of success on a single trial is equal to some value p
and remains the same from trial to trial. The probability of a failure is
equal to q = (1 − p).
4. The random variable of interest is Y , the number of trials till
we reach the rth success.

Maram Salem & Noha Youssef (AUC) 17 / 32

Negative Binomial Distribution

A Negative Binomial Variable

The negative binomial distribution takes the following form

p(y) = cy−1 r y−r

r−1 p q , y = r, r + 1, · · · , 0 ≤ p ≤ 1.

The negative binomial has a mean E(Y ) = pr , variance var(Y ) = r(1−p)

2
h t
ir p
pe
and a moment generating function My (t) = E(eY t ) = 1−(1−p)et .

Maram Salem & Noha Youssef (AUC) 18 / 32

Negative Binomial Distribution Proofs

r
−r−1
M 0 (t) = (1 − p)ret pet 1 − (1 − p)et

r−1
−r
+ pret pet 1 − (1 − p)et
(1 − p)r + pr
At t=0 =
p
r
M 0 (0) = E(X) = .
p

σ 2 = E X 2 − (E[X])2
 

r −r−2 
M 00 (t) = r pet (−r − 1) 1 − (1 − p)et −(1 − p)et
 

r−1 −r−1
+ r2 pet pet 1 − (1 − p)et



M 00 (0) = r(p)r (−r − 1)[1 − (1 − p)]−r−2 [−(1 − p)]

+ r2 (p)r−1 (p)[1 − (1 − p)]−r−1
Maram Salem & Noha Youssef (AUC) 19 / 32
Example 4

A geological study indicates that an exploratory oil well drilled in a particular

region should strike oil with probability 0.2. Find the probability that the
third oil strike comes on the fifth well drilled.
Sol:
p = 0.2 and r = 3
Y ~ Negative Binomial p(5) = c5−1 3
3−1 0.2 0.8 .
2

distribution

Maram Salem & Noha Youssef (AUC) 20 / 32

Hyper-geometric Distribution

A Hyper-geometric experiment possesses the following properties

1. The population contains a finite number N of elements that possess
one of two characteristics. N consists of 2 partitions, the partition of
interest is of size r and the other partition is of size N − r.
2. A sample of n elements is randomly selected from the population.
3. The random variable of interest is Y , the number of successes
in the sample.
4. The trials here are not independent therefore the distribution is not
binomial.

Maram Salem & Noha Youssef (AUC) 21 / 32

Hyper-geometric Distribution

A Hyper-Geoemtric Variable
A random variable Y is said to have a hyper-geometric probability
distribution if and only if
N −r
cry cn−y
p(y) =
cN
n
where y is an integer 0, 1, 2, · · · , n, subject to the restrictions y ≤ r and
n − y ≤ N − r. that is: y = 0,1,2,...., min.(r,n)
The hypergeometric distribution has a mean, E(Y ) = nr N , variance of
var(Y ) = n( Nr ( NN−r )( N −n
N −1 )), and the moment generating function does
not exists in closed form.

Maram Salem & Noha Youssef (AUC) 22 / 32

Example 5

A particular industrial product is shipped in lots of 20. A sample of 5 items

is selected from each lot and the lot is rejected if more than 1 defective is
observed. If a lot contains 4 defectives, what is the probability that it will
be rejected?
Sol: N = 20, r = 4 and ny = 5

c40 c16
5 c4 c16
P (Y > 1) = 1−P (Y ≤ 1) = 1−p(0)−p(1) = 1− 20 − 1 204 = 0.2487.
c5 c5

Maram Salem & Noha Youssef (AUC) 23 / 32

Poisson Distribution

The Poisson distribution is a discrete probability distribution for the counts

of events that occur randomly in a given interval of time (or space). So a
random variable Y is defined to be the number of events in a given interval.
The Poisson probability distribution often provides a good model for the
probability distribution of the number Y of rare events that occur in space,
time, volume, or any other dimension, where λ is the average value of Y .

Maram Salem & Noha Youssef (AUC) 24 / 32

Poisson Distribution

A Poisson Variable
A random variable Y is said to have a Poisson probability distribution if
and only if
λy −λ
p(y) = e , y = 0, 1, 2, · · · λ > 0.
y!
The expected value of the Poisson distribution is E(Y ) = λ and the
variance is var(Y ) = λ.
t
The moment generating function of the Poisson Distribution is eλ(e −1) .

To get the mean using the moment generating function we get the first
derivative first
t
M 0 (t) = eλ(e −1) .λ(et )
and then replacing t with zero

M 0 (t)|t=0 = 1.λ = λ.

Maram Salem & Noha Youssef (AUC) 25 / 32

Poisson Distribution

As a rule of Thumb:
n > 20 and p < 0.05

For those situations in which n is large and p is very small, the Poisson
distribution can be used to approximate the binomial distribution. The
larger the n and the smaller the p, the better is the approximation.

Maram Salem & Noha Youssef (AUC) 26 / 32

Poisson distribution as an approximation to the binomial
distribution

To prove that the Poisson distribution is an approximation of the binomial

distribution we define λ, the average number of occurrences in a certain
period of time consisting of n subintervals to be λ = np
n!
p(y) = py q n−y
(n − y)!y!
n! λ y λ
= ( )(1 − )n−y
(n − y)!y! n n

by taking the limits when n → ∞

n! λ λ
lim p(y) = lim ( )y (1 − )n−y .
n→∞ n→∞ (n − y)!y! n n

Maram Salem & Noha Youssef (AUC) 27 / 32

Proof:

We break down the right hand side into three different parts The first part

1 n! 1 n(n − 1) · · · , (n − y + 1)
lim =
n→∞ ny (n − y)!y! ny y!
we have y brackets so therefore
each having awe have
limit ny to 1
equal
1
= 1× .
y!
The second part
λ n
lim (1 − ) = e−λ .
n→∞ n

Maram Salem & Noha Youssef (AUC) 28 / 32

Proof:

The third part

λ −y
lim (1 − ) = 1,
n→∞ n
this will lead to
λy e−λ
p(y) = .
y!

Maram Salem & Noha Youssef (AUC) 29 / 32

Example 6

Customers arrive at a checkout counter in a department store according to

a Poisson distribution at an average of seven per hour. During a given hour,
what are the probabilities that
1 no more than three customers arrive?
Sol:
λ = 7 per hour
P (y ≤ 3) = p(0) + p(1) + p(2) + p(3)
2 at least two customers arrive?
Sol:

P (Y ≥ 2) = 1 − P (Y ≤ 1).
3 exactly five customers arrive?
Sol:
P (Y = 5).

Maram Salem & Noha Youssef (AUC) 30 / 32

Example 6

t
If the moment generating function of Y is given as e3(e −1) Find P (Y < 2).
Sol:
λ = 3 and P (Y < 2) = P (Y ≤ 1) = p(0) + p(1).
By the uniqueness of the mgf, the distribution is Poisson

Maram Salem & Noha Youssef (AUC) 31 / 32

Example 7

A salesperson has found that the probability of a sale on a single contact

is approximately 0.03. If the salesperson contacts 100 prospects, find the
exact and approximate probabilities of making at least one sale?
Sol:
Exact means using the binomial distribution, n = 100 and p = 0.03 and
then P (Y ≥ 1) = 1 − P (Y = 0) The approximate means using the Poisson
distribution, λ = np = 0.03 ∗ 100 = 3 and then carry on.

Maram Salem & Noha Youssef (AUC) 32 / 32