1
Particular Distributions
Discrete Probability Distributions
Bernoulli
A random variable X takes two values, 0 and 1, with probabilities q and p, respectively. That is:
P (X = 1) = p and P (X = 0) = q
where q = 1 − p. This is called a Bernoulli variate and is said to follow a Bernoulli distribution, where
p and q are the probabilities of success and failure, respectively.
The probability function for the Bernoulli distribution is given by:
P (X = x) = px (1 − p)1−x
The mean or expectation of the Bernoulli distribution is:
X
E[X] = x · p(x) = 0 × (1 − p) + 1 × (p) = p
x∈X
i.e., the mean of a Bernoulli distribution with parameter p is equal to p itself.
The variance of a distribution measures how much the outcomes “spread.” The variance is defined
as:
V [X] = Var[X] = E[(X − E[X])2 ]
For the Bernoulli random variable, the variance is:
X
V [X] = (x − E[X])2 p(x)
x∈X
Substitute the values for x:
V [X] = (0 − p)2 (1 − p) + (1 − p)2 p
= p2 (1 − p) + (1 − p)2 p
= [(1 − p) + p]p(1 − p)
= p(1 − p)
Thus, the variance of a Bernoulli random variable is:
Var[X] = p(1 − p)
The Binomial Probability Distribution
The probability function for the binomial distribution is given by:
n x
P (x) = P (X = x) = p (1 − p)n−x , x = 0, 1, 2, . . . , n
x
where:
• n is the number of trials,
• p is the probability of success on a single trial,
• nx is the binomial coefficient.
Muindi J.
2
The expected value, variance, and standard deviation of a binomial random variable X are:
µX = np
2
σX = np(1 − p)
p
σX = np(1 − p)
A probability distribution is binomial if:
• There are n trials, where n is fixed in advance.
• Each trial has two possible outcomes: success or failure.
• The trials are independent of one another.
• The probability of success is the same for each trial.
Example A lawyer estimates she wins 40% (p = 0.4) of her cases. Assume each trial is independent
of one another and, in general, this problem obeys conditions of a binomial experiment. The lawyer is
currently involved in 10 (n = 10) cases. For example, there is a 4.2% chance she wins 7 of her 10 cases.
The probability of winning exactly 7 out of 10 cases is given by the binomial probability formula:
n x
P (X = 7) = p (1 − p)n−x
x
where:
• n = 10 is the number of trials (cases),
• x = 7 is the number of successes (wins),
• p = 0.4 is the probability of success (winning a case).
Substitute the values into the formula:
10
P (X = 7) = (0.4)7 (1 − 0.4)10−7
7
10
P (X = 7) = (0.4)7 (0.6)3
7
Now, calculate the binomial coefficient:
10 10! 10! 10 × 9 × 8
= = = = 120
7 7!(10 − 7)! 7!3! 3×2×1
Thus, the probability becomes:
P (X = 7) = 120 × (0.4)7 × (0.6)3
Finally, compute the probability:
P (X = 7) = 0.04247.
Therefore, the probability that the lawyer wins 7 out of 10 cases is approximately:
P (X = 7) ≈ 4.2
Muindi J.
3
Poisson
If events happen randomly at independent times, with an average intensity (number of events per unit
time) λ, then the probability that a certain number of events x occur within a time interval is Poisson
distributed, denoted as Po(λ).
The probability mass function is given by:
λx exp(−λ)
P (X = x) = , λ > 0, x = 0, 1, 2, . . .
x!
Examples of Poisson-distributed events include radioactive decay, network traffic, customer arrivals,
etc.
The expectation and variance of the Poisson distribution are both:
E[X] = V [X] = λ,
which can be derived from the limiting Binomial.
Example: The average number of homes sold by a real estate company is two homes per day. Assuming
Poisson distribution, what is the probability that 3 homes will be sold tomorrow?
Solution: The average number of homes sold by the real estate company is λ = 2 homes per day. We
are asked to find the probability that 3 homes will be sold tomorrow, i.e., P (X = 3), where X is Poisson
distributed with λ = 2.
The probability mass function for the Poisson distribution is given by:
λr exp(−λ)
P (X = r) =
r!
Substitute r = 3 and λ = 2 into the formula:
23 exp(−2)
P (X = 3) =
3!
8 exp(−2)
P (X = 3) =
6
Now, approximate exp(−2):
exp(−2) ≈ 0.1353
Thus:
8 × 0.1353 1.0824
P (X = 3) = ≈ ≈ 0.1804
6 6
Therefore, the probability that 3 homes will be sold tomorrow is approximately:
P (X = 3) ≈ 0.1804
e ≈ 2.71828
Continuous Probability Distributions
Uniform Distribution
The uniform distribution is a continuous probability distribution and is concerned with events that are
equally likely to occur. A continuous random variable X is said to be uniformly distributed in the interval
(a, b), denoted by X ∼ U (a, b), if it has the following probability density function (PDF):
1
f (x) = , for a ≤ x ≤ b.
b−a
Muindi J.
4
The mean and variance of the uniform distribution are:
a+b
E(X) = (center of the interval)
2
(b − a)2
Var(X) = .
12
Example
The amount of time (in minutes) that a person must wait for a bus is uniformly distributed between
zero and 15 minutes inclusive.
• What is the probability that a person waits fewer than 12.5 minutes?
Let X be the number of minutes a person must wait for a bus, with a = 0 and b = 15, i.e.,
X ∼ U (0, 15).
The PDF is:
1
f (x) = , for 0 ≤ x ≤ 15.
15
We seek P (X < 12.5):
12.5 − 0
P (X < 12.5) = = 0.8333.
15
Thus, the probability that a person waits fewer than 12.5 minutes is 0.8333.
• On average, how long must a person wait? Find the mean µ and the standard deviation σ.
a+b 15 + 0
µ= = = 7.5 minutes.
2 2
Muindi J.
5
r r
(b − a)2 (15 − 0)2
σ= = ≈ 4.3 minutes.
12 12
• Ninety percent of the time, the time a person must wait falls below what value?
This asks for the 90th percentile, i.e., find k such that P (X < k) = 0.90.
k−0
P (X < k) = = 0.90 =⇒ k = 0.90 × 15 = 13.5 minutes.
15
Thus, 90% of the time, a person must wait at most 13.5 minutes.
Exponential Distribution
The exponential distribution is often used to represent lengths of time, such as the time until a radioactive
particle decays, the time between eruptions of a volcano, or the time you have to wait for a bus to arrive
at a bus stop.
A continuous random variable X is said to have the exponential distribution with rate λ > 0 if it has
the PDF:
f (x) = λe−λx , for x ≥ 0 and 0 otherwise.
We write X ∼ Exp(λ).
Example
The length of time (in years) that a lightbulb works before needing replacement is modeled as an
exponential distribution with rate λ = 2. What is the probability the lightbulb lasts more than a year
but less than three years?
If X ∼ Exp(2) is the lifetime of the lightbulb, we seek P (1 ≤ X ≤ 3).
Z 3 3
2e−2x dx = −e−2x 1 = e−2 − e−6 ≈ 0.132.
P (1 ≤ X ≤ 3) =
1
Theorem
Suppose X ∼ Exp(λ). Then:
R∞
• f (x) is indeed a PDF in that 0
f (x)dx = 1.
• The CDF of X is F (x) = 1 − e−λx .
• The expectation of X is E[X] = λ1 .
• The variance of X is Var(X) = 1
λ2 .
Muindi J.
6
Alternative Calculation Using the CDF
Returning to the lightbulb example where X ∼ Exp(2), we could alternatively calculate P (1 ≤ X ≤ 3)
using the CDF:
P (1 ≤ X ≤ 3) = F (3) − F (1) = 1 − e−2×3 − 1 − e−2×1 = e−2 − e−6 ≈ 0.132,
which is the same answer as before.
Normal Distribution
The normal distribution is the most widely known and used of all distributions. Because the normal
distribution approximates many natural phenomena so well, it has developed into a standard of reference
for many probability problems.
A Normal (aka Gaussian) random variable is parameterized by a mean (µ) and variance (σ 2 ). If X
is a normal variable we write X ∼ N (µ, σ 2 ).
Normal distributions (aka. Gaussian distributions) are a family of symmetric, bell-shaped density curves
defined by a mean (µ) and variance (σ 2 ).
The probability density function (PDF) for a normal X ∼ N (µ, σ 2 ) is:
1 1 (x−µ 2
f (x) = √ e− 2 σ )
σ 2π
A normal distribution with µ = 0, and σ = 1 is called the standard normal distribution, denoted as
N (0, 1).
If X has a normal distribution, then to find probabilities about X is to find areas under a normal
curve N (µ, σ).
Muindi J.
7
But,... there is no simple formula to find areas under a Normal curve. Need to use softwares or the
normal probability table.
Muindi J.
8
Muindi J.
9
Muindi J.
10
Example:
SAT scores are distributed nearly normally with mean 1500 and standard deviation 300. ACT scores are
distributed nearly normally with mean 21 and standard deviation 5. A college admissions officer wants
to determine which of the two applicants scored better on their standardized test with respect to the
other test takers: Pam, who earned an 1800 on her SAT, or Jim, who scored a 24 on his ACT?
Solution: Standardizing with Z scores
Since we cannot just compare these two raw scores, we instead compare how many standard deviations
beyond the mean each observation is.
• Pam’s score is 1800−1500
300 = 1 SD above the mean.
• Jim’s score is 24−21
5 = 0.6 SD above the mean.
Muindi J.
11
These are called standardized scores, or Z scores.
• Z score of an observation is the number of SDs it falls above or below the mean.
observation−mean x−µ
Z= SD = σ
• Observations that are more than 3 SD away from the mean (|Z| > 3) are usually considered un-
usual.
Properties of the normal distribution
Let X ∼ N (µ, σ 2 ) be a normally distributed random variable. Then:
• f(x) is indeed a PDF in that
R∞
−∞
f (x) dx = 1
• E[X] = µ
• Var (X) = σ 2
• In particular, if Z ∼ N (0, 1) is a standard normal distribution, then E[Z] = 0 and Var (Z) = 1.
Practice Questions
• The life of army shoes is normally distributed with mean 8 months and standard deviation 2
months. if 5000 pairs are issued, how many pairs would be expected to need replacement after 12
months?
• The top 5% of applicants (as measured by GRE scores) will receive scholarships. If GRE ∼
N (500, 1002 ), how high does your GRE score have to be to qualify for a scholarship? (665)
• Family income ∼ N ($25000, $100002 ). If the poverty level is $10,000, what percentage of the
population lives in poverty? (.0668)
• A new tax law is expected to benefit “middle income” families, those with incomes between $20,000
and $30,000. If Family income ∼ N (25000, 100002 ), what percentage of the population will benefit
from the law? {38%}
• Body temperatures of healthy humans are distributed nearly normally with mean 98.2◦F and
standard deviation 0.73◦F. What is the cutoff for the lowest 3% of human body temperatures?
(96.8◦F)
Muindi J.
