EN.553.

310/311 Probability and Statistics Fall 2022

Lecture 4: Law of Total Probability

he content of this lecture roughly corresponds to Sec. 3.7 of Sheldon and Ross’ Introduction to Probability
and Statistics for Engineers and Scientists.
Sn
Theorem 4.1. (Law of Total Probability) If A1 , A2 , · · · , An are mutually exclusive and Ω = i=1 Ai (i.e.,
exhaustive), then for any event B
n
X
P (B) = P (B|Ai )P (Ai ).
i=1

Proof. (Proof when n = 4) Since {Ai }4i=1 are mutually exclusive (see Figure ??), B can be written as
B = (B ∩ A1 ) ∪ (B ∩ A2 ) ∪ (B ∩ A3 ) ∪ (B ∩ A4 ). Hence,

P (B) = P (B ∩ A1 ) + P (B ∩ A2 ) + P (B ∩ A3 ) + P (B ∩ A4 )
= P (B|A1 )P (A1 ) + P (B|A2 )P (A2 ) + P (B|A3 )P (A3 ) + P (B|A4 )P (A4 )

Remark. The general case can be shown by proof by induction. Hint: B = ∪n−1
i=1 (B ∩ Ai ) ∪ (B ∩ An ).

Figure 4.1: Illustration of proof when n = 4.

Example 1. (Continued from Lecture 3)

Compute P (R). B1 and B2 are mutually exclusive and exhaustive, so from Theorem 4.1.

P (R) = P (R|B1 )P (B1 ) + P (R|B2 )P (B2 )

1 1 2 1
= · + ·
2 2 3 2
1 1 7
= + = .
4 3 12
Example 2. (Continued from Lecture 3)
Toss a fair coin 3 times. The sample space is Ω = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}. Let A be
the event we obtain exactly 2 heads (We already know P (A) = 83 ). Here is another way to see this.

4-1
4-2 Lecture 4: Law of Total Probability

Let H1 be the event the 1st toss is a head and H1c be the event the 1st toss is a tail. Then, H1 , H1c are
mutually exclusive and exhaustive. Therefore,
P (A) = P (A|H1 )P (H1 ) + P (A|H1c )P (H1c )
1 1 1 1
= · + ·
2 2 4 2
1 1 3
= + = .
4 8 8
Comment: It is helpful to verbalize the conditional probabilities using the “if” statement. For example,
P (A|H1 ) is described as “If the 1st toss is heads, then the (conditional) probability of exactly 2 heads in 3
tosses is the same as exactly 1 head in 2 tosses”.
Example 3. (Roll two 4−sided dice)
Let F be the event of the sum of two dice is 4. Let Ai = red die is i, for i = 1, 2, 3, 4. Then, A1 , A2 , A3 , A4
are mutually exclusive and exhaustive
P (F ) = P (F |A1 )P (A1 ) + P (F |A2 )P (A2 ) + P (F |A3 )P (A3 ) + P (F |A4 )P (A4 )
1 1 1 1 1 1 1
= · + · + · +0·
4 4 4 4 4 4 4
3
= .
16
Comment: The Law of Total Probability can be used to compute an (unconditional) probability in terms
of conditional probabilities and probabilities of simpler events when cond. probabilities are easy to compute.
We saw this in the ”red and blue marbles in boxes” example. Here’s another situation:
Example 4. (An experiment that is performed in stages)
Consider the experiment where you roll a 6-sided die. If the die comes up 1 or 2, flip a coin ONCE. If the
die comes up 3 or 4, flip a coin TWICE, and if the die comes up 5 or 6, flip a coin THRICE. Compute the
probability that you toss exactly one head.
Solution: Let’s call H the event of tossing exactly one head. Also, let A, B and C be the events the die
comes up 1 or 2, 3 or 4 and 5 or 6, respectively. Then, P (A) = 26 = 13 = P (B) = P (C). Moreover,
P (H|A) = 21 , P (H|B) = 24 = 12 and P (H|C) = 38 . From Law of Total Probability,
P (H) = P (H|A)P (A) + P (H|B)P (B) + P (H|C)P (C)
1 1 1 1 3 1
= · + · + ·
2 3 2 3 8 3
11
= .
24

4.0.1 Bayes’ rule

Theorem 4.2. The Bayes’ rule

Let A1 , A2 , · · · , An be mutually exclusive and exhaustive and let B be any eveny. Then, for any Ai
P (B|Ai )P (Ai )
P (Ai |B) = (4.1)
P (B|A1 )P (A1 ) + P (B|A2 )P (A2 ) + · · · + P (B|An )P (An )
Remark. Note that the denominator in Eq. ?? is equal to P (B) from the Law of Total Probability . The
numerator is equal to P (B ∩ Ai ). Hence, Eq. ?? can be also stated as
P (Ai ∩ B)
P (Ai |B) = ,
P (B)
Lecture 4: Law of Total Probability 4-3

which is the conditional probability formula introduced in Lecture 3.

Example 5. (Golf course) Fred is working at a golf course. If it is sunny outside, there’s a 10% chance Fred
will call in sick. If it is cloudy outside, there’s a 20% chance Fred will call in sick. If it is raining outside,
there’s a 50% chance Fred will call in sick. Assume that on any given day of the golf season, there is a
60%, 30% and 10% chance it will be sunny, cloudy and rainy, respectively. If Fred calls in sick, what is the
probability it is sunny, cloudy, and rainy?
Solution: (Step 1: Introduce mathematical symbols/notations to describe the problem) Let B = Fred calls
in sick. Also, let A1 , A2 and A3 be the events it is sunny, cloudy and rainy, respectively.
(Step 2: Compute known probabilities) We know that P (A1 ) = .6, P (A2 ) = .3, P (A3 ) = .1 and P (B|A1 ) =
.1, P (B|A2 ) = .2, P (B|A3 ) = .5.
(Step 3: Apply Bayes’ rule) The probabilities we are looking to compute are P (B|A1 ), P (B|A2 ) and P (B|A3 ).
From Bayes’ rule,

P (B|A1 )P (A1 )
P (A1 |B) =
P (B|A1 )P (A1 ) + P (B|A2 )P (A2 ) + P (B|A3 )P (A3 )
0.1 · 0.6
=
0.1 · 0.6 + 0.2 · 0.3 + 0.5 · 0.1
.06
=
.06 + .06 + .05
6
= ≈ 35%.
17
6 5
Verify that P (A2 |B) = 17 and P (A3 |B) = 17 .

Example 6. (Speeding ticket) About 10% of drivers are speeding in a certain stretch of a road. If you are
speeding, you will get a ticket 90% of the time. If you are not speeding, you will get a ticket 5% of the time.
What proportion of drivers who receive tickets are actually speeding (true positive)?
Solution: (Step 1) Let S be the event that a driver is speeding and T be the event that a driver receives a
ticket. The probability in question is to compute P (S|T ). (Step 2) We know that P (S) = 0.1, P (T |S) = 0.9
and P (T |S c ) = 0.05. (Step 3) Finally, Bayes’ rule yields

P (T |S)P (S)
P (S|T ) =
P (T |S)P (S) + P (T |S c )P (S c )
0.9 · 0.1 2
= = .
0.9 · 0.1 + 0.05 · 0.9 3
Note: This means that 2 out of 3 drivers who received tickets for speeding were actually speeding and 1
out of 3 drivers who received tickets for speeding were not (false positive)!

Figure 4.2: Exercise

Example 7. (Marbles experiment) There are 3 boxes of marbles. Box 1 has 1 red and 1 blue marbles. Box
2 has 2 red marbles, and Box 3 has 2 red and 1 blue marbles. Pick a box uniformly at random and draw a
marble. If you drew a red marble, what is the probability that you chose Box 2?
4-4 Lecture 4: Law of Total Probability

Solution: Let R = the red marble is drawn. Let B1 , B2 , B3 be the events that Box 1,2, 3 are selected,
respectively. We have that P (B1 ) = P (B2 ) = P (B3 ) = 31 . Also, P (R|B1 ) = 12 , P (R|B2 ) = 1 and
P (R|B3 ) = 23 . Applying Bayes’ rule

P (R|B2 )P (B2 )
P (B2 |R) =
P (R|B1 )P (B1 ) + P (R|B2 )P (B2 ) + P (R|B3 )P (B3 )
1 · 31
= 1 1 1 2 1
2 · 3 +1· 3 + 3 · 3
6
= ≈ 46%.
13
Caution: Notice that the answer is not 25 = 0.4 (some would think that the answer should be 2/5 since 2
of the 5 red marbles that could be selected are in Box 2). However, as an exercise, you should check that if
we change the probabilities to P (B1 ) = 72 , P (B2 ) = 27 and P (B3 ) = 73 , then P (B2 |R) = 25 .