Ch03

Additional Topics in Probability

 CHAPTER CONTENTS
• 3.1 Introduction ............................................................................ 112
• 3.2 Special Distribution Functions ................................................ 112
• 3.3 Joint Probability Distributions ................................................. 139
• 3.4 Functions of Random Variables ............................................... 152
• 3.5 Limit Theorems ........................................................................ 159
• 3.6 Chapter Summary .................................................................... 168
• 3.7 Computer Examples (Optional) ................................................. 170
• Projects for Chapter 3 ..................................................................... 175
 3.1 Introduction
• A better knowledge of the population will result in better inferential results.
3.2 Special Distribution Functions
 3.2.1 THE BINOMIAL PROBABILITY DISTRIBUTION
• Bernoulli probability distribution ： A random variable X associated with an
dichotomous experiment is said to have a Bernoulli probability distribution if X
takes the values 1 and 0, respectively, corresponding to the two possible
experiment outcomes. P(x)

(1 - p)

P
x
0 1

• The probability function of a Bernoulli random variable X ：

 p, x  1
 p x (1  p )1 x , x  0,1 
p( x )  P( X  x )   , i.e. (1  p ), x  0
0, otherwise 0, otherwise


• E[X] = p, Var(X) = pq, the moment-generating function MX(t) = pet + (1  p)

 Binominal probability distribution
• Binomial probability distribution ： The probability of observing exactly k
successes in n independent Bernoulli trials

Definition 3.2.2 A random variable X is said to have binomial probability distribution with parameters
(n, p) if and only if
 n  x n x
P( X  x )  p( x )    p q
 x
 n! x n x
 p q , x  0,1,, n,0  p  1, andq  1  p
  x! (n  x )!
0, otherwise


Theorem 3.2.1 If X is a binomial random variable with parameters n and p, then

E(X) =  = np, Var(X) = 2 = np(1 – p), Mx(t) = [pet + (1 – p)]n
 3.2.2 POISSON PROBABILITY DISTRIBUTION
The Poisson probability distribution ：
introduced by the French mathematician Sime´on-Denis Poisson in
his book published in 1837, which dealt with the applications of
probability theory to lawsuits, criminal trials, and the like.

Applications:
• the number of misprints in a book
• radioactivity counts per unit time
• the number of plankton (microscopic plant or animal organisms that float in bodies of water) per aliquot of seawater
• count of bacterial colonies per petri plate in a microbiological study
• to analyze the redundancy of clusters in the stem cell databas

A random variable that counts the number of occurrences of an event of interest is called a counting random
variable. The Poisson random variable is an example of a counting random variable.

Here we assume that the numbers of occurrences in disjoint intervals are independent and the mean of the number
occurrences is constant.
 Poisson probability distribution
• Poisson probability distribution ： The probability of observing exactly k
occurences of an event of interest in an interval

Definition 3.2.3 A discrete random variable X is said to follow the Poisson probability
distribution with parameter >0, denoted by Poisson (), if

e   x
P ( X  x )  p ( x,  )  f ( x )  , x  0,1,2,
x!

Theorem 3.2.2 If X is a Poisson random variable with parameter , then

 ( e t 1)
E ( X )   , Var ( X )   , and M X (t )  e .
 Examples of Poisson Distribution
Example 1
A life insurance salesman sells on the average 3 life insurance policies per week. Use Poisson's law to calculate the
probability that in a given week he will sell
(a. Some policies
(b. 2 or more policies but less than 5 policies.
(c. Assuming that there are 5 working days per week, what is the probability that in a given day he will sell one policy?

Example 2
Twenty sheets of aluminum alloy were examined for surface flaws. The frequency of the number of sheets
with a given number of flaws per sheet was as follows:
What is the probability of finding a sheet chosen at random which contains 3 or more surface flaws?

Number of flaws Frequency

0 4
1 3
2 5
3 2
4 4
5 1
6 1
 (continued)

Example 3
If electricity power failures occur according to a Poisson distribution with an average of 3 failures every
twenty weeks, calculate the probability that there will not be more than one failure during a particular week.

Example 4
Vehicles pass through a junction on a busy road at an average rate of 300 per hour.
Find the probability that none passes in a given minute.
What is the expected number passing in two minutes?
Find the probability that this expected number actually pass through in a given two-minute period.

Example 5
A company makes electric motors. The probability an electric motor is defective is 0.01. What is the
probability that a sample of 300 electric motors will contain exactly 5 defective motors?
POISSON APPROXIMATION TO THE BINOMIAL PROBABILITY
DISTRIBUTION

Theorem 3.2.3 If X is a binomial r.v. with parameters n and p,

then for each value x  0, 1, 2,and as p  0, n   with np   constant,
n x n x e  x
lim   p (1  p ) 
x
n    x!
.

When to use Poisson approximation ？

Ans:
p  0.1 and n  40
or n>50, and np<5.
 Example of Poisson Approximation
EXAMPLE 3.2.5
If the probability that an individual suffers an adverse reaction from a particular drug is known to be 0.001,
determine the probability that out of 2000 individuals, (a) exactly three and (b) more than two individuals will
suffer an adverse reaction.

Solution
3.2.3 UNIFORM PROBABILITY DISTRIBUTION
 3.2.4 NORMAL PROBABILITY DISTRIBUTION

X  N(, 2) ： The random variable X has a normal distribution with mean  and variance 2
X  N(0,1) : the random variable X has a standard normal probability distribution,
 Test of Normality

Quantile-Quantile Plot
 Log-normal distribution

If a random variable Y is normally distributed, then exp(Y) has a log-normal distribution.

Thus, the natural logarithm of a log-normally distributed variable is normally distributed.
That is, if X is a random variable log-normal distribution, then ln(X) is normally distributed.

Most biological evidence suggests that the growth processes of living tissue proceed by multiplicative,
not additive, increments. Thus, the measures of body size should at most follow a log-normal rather than
normal distribution.

the sizes of plants and animals are approximately log-normal.

3.2.5 GAMMA PROBABILITY DISTRIBUTION
Gamma probability distribution
 Erlang distribution

The Erlang distribution ： the gamma probability distribution with  to be a positive integer.

The Erlang distribution is used extensively in queuing theory to model waiting times.
 Exponential probability distribution
In the pdf of the gamma, we let  = 1, we get the pdf of an exponential random variable.
 Exponential random variables are often used
for modeling the lifetimes of electronic components such as
fuses,
for survival analysis,
for reliability analysis,
for developing models of insurance risks.

The exponential distribution is related to Poisson distribution. When the events can
occur more than once within a given unit of time and the time elapsed between two
consecutive occurrences is exponentially distributed and independent of previous
occurrences of the events then the random variable defined by the number of
Chi-square distribution
The chi-square distribution is useful in statistical inference
problems
 Beta distribution

The beta distribution plays a crucial role in Bayesian statistics.

The beta distribution is used as a prior distribution for binomial or geometric
proportions.
 “How do we know which distribution to use in a given physical
problem?”
1. construct a histogram from the information at hand;
2. (from the shape of this histogram) decide whether the random variable
follows a particular distribution
3. statistically estimate the parameters of this distribution

Chapter 5 ： how to estimate these parameters.

Chapter 7 ： a goodness-of-fit test to see whether the distribution
model seems to be the right one
3.3 JOINT PROBABILITY DISTRIBUTIONS
Something
wrong
Expected value of bivariate distribution.
Conditional expectation
 3.3.1 COVARIANCE AND CORRELATION

Correlation is the measure of the linear relationship between the random variables X and
Y
 3.4 FUNCTIONS OF RANDOM VARIABLES
• We are given the distribution of X, and
we are required to find the distribution of g(X).
3.4.1 METHOD OF DISTRIBUTION FUNCTIONS
3.4.2 THE pdf OF Y=g(X), WHERE g IS DIFFERENTIABLE AND
MONOTONE INCREASING OR DECREASING
3.4.3 PROBABILITY INTEGRAL TRANSFORMATION
 3.4.4 FUNCTIONS OF SEVERAL RANDOM VARIABLES: METHOD
OF DISTRIBUTION FUNCTIONSGAMMA PROBABILITY DISTRIBUTION
3.4.5 TRANSFORMATION METHOD
3.5 Limit Theorems
Chebyshev’s Theorem
Law of Large Numbers

The law of large numbers is also called the “law of averages.”

Thus, without any knowledge of the probability distribution function of Sn,
the (weak) law of large numbers states that the sample mean, X=Sn/n, will
differ from the population mean by less than an arbitrary constant, >0,
with probability that tends to 1 as n tends to 1.

This result basically states that we can start with a random experiment
whose outcome cannot be predicted with certainty, and by taking averages,
we can obtain an experiment in which the outcome can be predicted with a
high degree of accuracy.
The law of large numbers states that if the sample size n is large, the sample
mean rarely deviates from the population mean.
 CENTRAL LIMIT THEOREM

X_bar ~ N(, 2/n)

No matter what the shape of the original distribution is,
the (sampling) distribution of the mean approaches a
normal probability distribution.
The Central Limit Theorem basically says that when we repeat an
experiment a large number of times, the average (almost always) follows a
Gaussian distribution.
3.6 Chapter Summary
3.7 Computer Examples (Optional)
Projects for Chapter 3