Bayes’ Theorem

Bayes’ Theorem

• Relate marginal and conditional

probabilities of two random variables
• Or it is the basic formula for conditional
probability under dependence is called
Bayes’ Theorem.
• Or it provides the means of calculating
posterior probabilities from prior probability
The basic formula for conditional probability can be written as

P(A)P(B/A)
P(A/B) 
P(B)
P(A) is the prior probability or marginal probability of A.
P(A|B) is the conditional probability of A, given B. It is
also called the posterior probability
P(B|A) is the conditional probability of B given A.
P(B) is the prior or marginal probability of B
• To derive the theorem, we start from the definition
of conditional probability. The probability of event
A given event B is
P(A  B)
P(A/B) 
P(B)
• Likewise, the probability of event B given event A
is
P(A  B)
P(B/A) 
P(A)
• Rearranging and combining these two
equations, we find

P(B/A).P(A)  P(A  B)  P(A/B).P(B)

• Dividing both sides by P(B), providing that it is

non-zero, we obtain Bayes' theorem:

P(B/A).P(A)
P(A/B) 
P(B)
 Calculating Posterior Probabilities
Example.
• We have equal number of two types of biased
(A&B) dice in a bowl. On A type die, one comes
up 40 percent of the time; therefore P (1) = 0.4.
On B type die, one comes up 70 percent of the
time; P (1) =0.7.
• One die is drawn, rolled once, and comes up
one. What is the probability that it is A type die?
Or What is the probability that it is B type die?
Finding the marginal probability of
getting an ace
Elementary Marginal Posterior Joint Probability
event Probability probability

A P(A) = 0.5 P(1/A) = 0.4 P (1∩A)

= 0.4×0.5 =0.20
B P(B) = 0.5 P(1/B) =0.7 P (1∩B)
= 0.7×0.5 =0.35
Total 1 P(1) = 0.55
The revised probabilities are

P(1/A).P(A)
P(A/1) 
P(1)

P(1/B).P(B)
P(B/1) 
P(1)
The revised probabilities are

• P(A | 1) = 0.2/0.55 = 0.364

• P(B | 1) = 0.35/0.55 = 0.636

• Before we rolled this dice, the best we could say
was that there is a 0.5 chance it is a type A die
and 0.5 chance it is a type B die.

• However after rolling the die, we have been able

to alter or revise, our prior probability estimate.

• Our new posterior estimate shows there is a

higher probability (0.636) that the die in our hand
is of type B than type A (only 0.364).
Bayes' theorem is often restated by noting that

where AC is the complementary event of A (often called "not A"). So the

theorem can be restated as

More generally, where {Ai} forms a partition of the event space,

for any Ai in the partition.

Example
• A physical therapist at state university knows
that the football team will play 40 percent of its
games on artificial turf this season. He also
knows that a football player’s chances of
incurring a knee injury are 50 percent higher if
he is playing on artificial turf instead of grass. If a
player’s probability of knee injury on artificial turf
is 0.42, what is the probability that
– A randomly selected football player incurs a knee
injury?
– A randomly selected football player with a knee injury,
incurred it while playing in grass?
• The physical therapist in the above problem is also
interested in studying the relationship between foot
injuries and position played. His data, gathered over a
3 year period, are summarized in the following table

Offensive Defensive Offensive Defensive

line line backfield backfield
Number of 45 56 24 20
players
Number of injured 32 38 11 9
• Given that randomly selected player incurred foot
injury, What is the probability that he plays in the
– Offensive line
– Defensive line
– Offensive backfield
– Defensive backfield