STA 247

Probability with Computer Applications

Professor K. H. Wong

3B - Law of Total Probability & Bayes Rule

1/8
Learning Outcomes

Explain the difference between conditional probability and

probabilities involving union and intersection of sets
Identify given information and compute conditional probabilities (or
any of P(A ∩ B), P(A), P(B))
Use conditional probability and disjoint events to find total probability
of an event (Law of Total Probability)
Apply conditional probability in conjunction with LTP to define Bayes
Rule

2/8
Law of Total Probability

Given two events, we’ve learned that their conditional probability defined as
P(A ∩ B)
P(A|B) = , where P(B) > 0
P(B)
Can be rearranged to:
P(A ∩ B) = P(A|B) · P(B)
Suppose now we have a sample space consisting only of events
A, B1 , B2 , ....Bk
where the Bi0 s partitions the sample space. That is, the Bi0 s are disjoint
and ki=1 Bi = Ω. From axiom 3, we can show that:
S

P(A) = P(A ∩ B1 ) + P(A ∩ B2 ) + P(A ∩ B3 ) + ... + P(A ∩ Bk )

Combining this with our knowledge of conditional probability, we get the
Law of Total Probability.
3/8
Law of Total Probability

Law of Total Probability

If B1 , B2 , ...Bk is a collection of mutually exclusive (disjoint) and
exhaustive events, then for any event A,

P(A) = P(A|B1 ) · P(B1 ) + P(A|B2 ) · P(B2 ) + ... + P(A|Bk ) · P(Bk )

k
X
P(A) = P(A|Bi ) · P(Bi )
i=1

4/8
Bayes’ Rule

Putting together the Law of Total Probability and definition of conditional

probability, we get Bayes’ Rule
Bayes’ Rule
Let B1 , B2 , ...Bk form a partition of the sample space and let A be an
event in Ω. Then
P(A ∩ Bi )
P(Bi |A) =
P(A)
P(A|Bi ) · P(Bi )
P(Bi |A) = Pk
i=1 P(A|Bi ) · P(Bi )

5/8
Bayes’ Rule Examples

Example 1 Returning to HIV testing. An HIV test will correctly test

positive 95% of the time (sensitivity), and incorrectly test positive 1%
(false positive rate) of the time. Suppose we know that 99.99% of the
population is HIV-free. Under these conditions, what is the probability
that a patient who tested positive is actually HIV positive?

6/8
Bayes’ Rule Examples

Example 1 Returning to HIV testing. An HIV test will correctly test

positive 95% of the time (sensitivity), and incorrectly test positive 1%
(false positive rate) of the time. Suppose we know that 99.99% of the
population is HIV-free. Under these conditions, what is the probability
that a patient who tested positive is actually HIV positive?
Let’s represent the situation with a tree diagram:

Healthy(0.9999) HIV(0.0001)

(+) (–) (+) (–)

0.01 0.99 0.95 0.05

6/8
Bayes’ Rule Examples

Note conventionally, branch probabilities are conditional probabilities. i.e.

the first probability of testing positive (0.01) is conditional on the patient
being from the healthy branch.

Let H denote the event that the patient is healthy. Let (T) denote the
event that the patient tests positive.

7/8
Bayes’ Rule Examples

Example 2 A ball is drawn at random from an urn containing one red and
one white ball. If the white ball is drawn, it is put back into the urn. If the
red ball is drawn, it is returned to the urn together with two more red
balls. Then a second draw is made. What is the probability a red ball was
drawn on both the first and the second draws?

8/8