You are on page 1of 8

# Basic Probability Theory

Lecture 3
Lecturer: Ali Ghodsi
Notes: Ahmad Al-Shishtawy
October 5, 2007

1

Problems From the Previous Lecture

Problem 13. We are given three coins: one has heads in both faces, the second
has tails in both faces, and the third has a head in one face and a tail in
the other. We choose a coin at random, toss it, and it comes heads. What
is the probability that the opposite face is tails?
Solution: Let H and N be the events that we saw a head and it is a normal
coin, respectively.
H = {Saw a head},

N = {N ormal coin}.

Now we need to calculate the conditional probability that it is a normal
coin given that we saw a head:

P (N | H) =
=
=

P (N ∩ H)
P (H)
P (N )P (H | H)
P (N )
1 1
1
3 · 2
=
1
3
2

In this problem, there is a tendency to reason that since the opposite face
is either heads or tails, the desired probability is 1/2. This is, however,
wrong, because given that heads came, it is more likely that the twoheaded coin was chosen.
Lesson learned: Proceed axiomatically and never follow your intuition.
Remark: P (A)P (B | A) = P (A ∩ B) = P (B)P (A | BA)

2

Using Conditional Probability for Modeling

When constructing probabilistic models for experiments that have a sequential
character, it is often natural and convenient to first specify conditional probabilities and then use them to determine unconditional probabilities as shown in
the following example.
1

Let A and B be the events A = {an aircraf t is present}.05 · 0. and consider also their complements Ac B c = {an aircraf t is not present}. 1. Each possible outcome corresponds to a leaf of the tree. The desired probabilities of false alarm and missed detection are P (not present. If an aircraft is present in a certain area. a radar correctly registers its presence with probability 0.10.10 = 0. The given probabilities are recorded along the corresponding branches of the tree describing the sample space. as shown in Fig.95 · 0.0005.05. P (present. and the probability of missed detection (nothing registers. What is the probability of false alarm (a false indication of aircraft presence). B = {the radar generates an alarm}.01 = 0. 2 . If it is not present.9 Radar Detection. as shown in Fig. 1. = {the radar does not generate an alarm}. no detection) = P (A ∩ B c ) = P (A)P (B c | A) = 0. and its probability is equal to the product of the probabilities associated with the branches in a path from the root to the corresponding leaf.Figure 1: Sequential description of the experiment for the radar detection Example 1. We assume that an aircraft is present with probability 0. even though an aircraft is present)? Solution: A sequential representation of the experiment is appropriate here. the radar falsely registers an aircraft presence with probability 0.095. f alse alarm) = P (Ac ∩ B) = P (Ac )P (B | Ac ) = 0.99.

The multiplication rule can be verified by writing P (∩ni=1 Ai ) = P (∩ni=1 Ai ) P (∩ni=1 Ai ) n−1 = P (∩i=1 Ai ) · n−1 P (∩i=1 Ai ) . Three cards are drawn from an ordinary 52-card deck without replacement (drawn cards are not placed back in the deck). the probability that none of the three cards is a heart. A cumbersome approach. We assume that at each step. Example 1. using the multiplication rule P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 | A1 )P (A3 | A1 ∩ A2 ) We have P (A1 ) = 3 39 52 . We wish to find the probability that none of the three cards is a heart. is to count the number of all card triplets that do not include a heart. Instead. . . P (An | ∩i=1 Ai ). {third not heart}. n−2 n−1 P (A1 ∩ A2 ) P (∩i=1 Ai ) P (∩i=1 Ai ) P (A1 ∩ A2 ) P (A1 ∩ A2 ∩ A3 ) P (A1 ) · · P (A1 ) P (A1 ∩ A2 ) P (A1 ∩ A2 ) · . . this implies that every triplet of cards is equally likely to be drawn. P (∩ni=1 Ai ) = n−1 P (∩i=1 Ai ) P (∩ni=1 Ai ) · n−2 n−1 P (∩i=1 Ai ) P (∩i=1 Ai ) n−1 P (A1 )P (A2 | A1 )P (A3 | A1 ∩ A2 ) .. By symmetry. P (An | ∩i=1 Ai ). that we will not use. . . we have n−1 P (∩ni=1 Ai ) = P (A1 )P (A2 | A1 )P (A3 | A1 ∩ A2 ) .. we use a sequential description of the experiment in conjunction with the multiplication rule. n−1 P (∩i=1 Ai ) P (∩ni=1 Ai ) · P (∩ni=1 Ai ) = P (A1 ∩ A2 ∩ A3 ) . n−2 n−1 P (∩i=1 Ai ) P (∩i=1 Ai ) P (∩ni=1 Ai ) P (∩ni=1 Ai ) = P (∩ni=1 Ai ) = n−1 P (A1 ∩ A2 ∩ A3 ) P (∩i=1 Ai ) P (∩ni=1 Ai ) · .10. . each one of the remaining cards is equally likely to be picked..Remark: Always draw a tree when you encounter sequential events 3 Multiplication Rule Assuming that all of the conditioning events have positive probability. and divide it with the number of all possible card triplets... A2 A3 = = {second not heart}. Solution: We define the events A1 = {f irst not heart}. Now we are seeking P (A1 ∩ A2 ∩ A3 )..

38 of which are not hearts 38 P (A2 | A1 ) = 51 similarly 37 P (A3 | A1 ∩ A2 ) = 50 and using the multiplication rule we have 39 38 37 P (A1 ∩ A2 ∩ A3 ) = · · 52 51 50 Example 1. {student 1.12. you can stick to your initial choice. 2. ∩3i=1 Ai = A3 . Consider the following strategies: 1. and 3 are in dif f erent groups}. If we know that the first card is not heart then we are left with 51 cards. We are seeking P (A) and we calculate it using the multiplication rule: P (A) = P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 | A1 )P (A3 | A1 ∩ A2 ) 12 8 4 · · = 15 14 13 Example 1. 3. A friend opens for you one of the remaining two doors. after making sure that the prize is not behind it. Switch to the other unopened door. based on an old American game show. This is a much discussed puzzle. While if you switch after door 2 or 3 is opened then the probability to win is 2/3 (shaded area on the outer circle). {student 1. or switch to the other unopened door. What is the probability that each group includes a graduate student? Solution: We define the events A1 A2 = = A3 = A = {student 1 and 2 are in dif f erent groups}. At this point. You point to one of the doors. 2. Which is the best strategy? Solution: A simple solution is by using Figure 2. That is if the prize is behind door 1. 4 . This is because the probability of the prize being behind door 2 or 3 P (2 ∪ 3) = 2/3 but you know that P (Opened) = 0 so P (closed) = 2/3 and P (initial choice) = 1/3 which means that it is better to switch. You are told that a prize is equally likely to be found behind any one of three closed doors in front of you. You win the prize if it lies behind your final choice of a door.11 A class consisting of 4 graduate and 12 undergraduate students is randomly divided into 4 groups of 4.because there are 39 cards that are not red in a 52 card deck (3/4). Stick to your initial choice. if your initial choice is 1 (inner circle) then the probability to win if you don’t switch is 1/3 (shaded area on the middle circle). and 4 are in dif f erent groups}. 2. For example. The Monty Hall Problem.

. Solve the Monty Hall problem using the total probability theorem.Inner Circle: Your initial choice 1 3 2 2 1 3 1 2 2 2 3 3 3 1 1 Middle Circle: Probability to win if you don’t switch 3 1 2 1 3 Outer Circle: Probability to win if you switch 2 Figure 2: The Monty Hall Problem. Solution: Define the following events W = {W in if switch} IP P = {Initially pick prize} We are seeking P (W ) that can be represented as follows (see Figure 4) P (W ) = P (IP P )P (W | IP P ) + P (IP P c )P (W | IP P c ). . Then. . we have P (B) = P (A1 ∩ B) + P (A2 ∩ B) + . . 5 . The shaded area shows the probability to win. . we have P (IP P ) = 1/3. 4 Total Probability Theorem Let Ai be disjoint events that form a partition of the sample space and assume that P (Ai ) > 0. for any event B. Example. . for all i. ∪ An )) Remark: The total probability theorem is for both discrete and continuous probabilistic models. + P (An )P (B | An ). Hint: to prove it start with P (B ∩ Ω) = P (B ∩ (A1 ∪ A2 ∪ . + P (An ∩ B) = P (A1 )P (B | A1 ) + P (A2 )P (B | A2 ) + .

similarly P (W | IP P c ) = 1 thus P (W ) = 2 2 1 ·0+ ·1= 3 3 3 Not initially pick prize Win if switch Initially pick prize Figure 4: Using the total probability theorem to solve the Monty Hall problem 6 . if initially picked prize but switched then P (W | IP P ) = 0.A1 B A3 A2 Figure 3: Visualization of the total probability theorem. and P (IP P c ) = 2/3.

. is P (A|B) P (A)P (B|A) P (A)P (B|A) + P (Ac )P (B|Ac ) 0. You enter a chess tournament where your probability of winning a game is 0. Let also B be the event of winning. by the total probability theorem. A random person drawn from a certain population has probability 0.4. Given that the person just tested positive. and if the person does not have the disease. The False-Positive Puzzle.001 of having the disease.4 against a quarter of the players (call them type 2). What is the probability of winning? Solution: Let Ai be the event of playing with an opponent of type i. P (A3 ) = 0.001 · 0. for all i. the desired probability. = 7 .5.5. You play a game against a randomly chosen opponent. P (B | A2 ) = 0. .95. Thus.5 · 0. and 0.5 against the remaining quarter of the players (call them type 3). 0.3. Then.375.4 + 0. .Example 1. We have P (A1 ) = 0.95.3 against half the players (call them type 1). we have P (Ai | B) = = P (Ai )P (B | Ai ) P (B) P (Ai )P (B | Ai ) Σni=1 P (Ai )P (B | Ai ) Example 1.95 + 0.001 · 0.18.05 = 0. what is the probability of having the disease? Solution: If A is the event that the person has the disease. for any event B such that P (B) > 0. P (B | A3 ) = 0. P (A2 ) = 0.999 · 0. the probability of winning is P (B) = = = 5 P (A1 )P (B | A1 ) + P (A2 )P (B | A2 ) + P (A3 )P (B | A3 ) 0.25 · 0. A2 .5 0. We have P (B | A1 ) = 0.25.13. and assume that P (Ai) > 0. An be disjoint events that form a partition of the sample space. P (A|B). A test for a certain rare disease is assumed to be correct 95% of the time: if a person has the disease. the test results are negative with probability 0.25.25 · 0. the test results are positive with probability 0. and B is the event that the test results are positive.0187.3 + 0. Inference and Bayes Rule Let A1 .95 = 0. .

let Sm be the set of all m-tuples (i1 . 1. . assuming that P (B) > 0. . Modified Monty Hall Problem. . 2. < im ≤ n.16. Problem 14. Show the following generalizations of the formula P (A ∪ B) = P (A) + P (B) − P (A ∩ B).* The inclusion-exclusion formula. Let A and B be events. . .i3 )∈S3 − . Then. and C be events. S2 = {(i1 . .i2 . Let A. B. 8 . . . . An be events. and 1. Then. Read examples 1. X X P (∪nk=1 Ak ) = P (Ai1 ∩ Ai2 ) P (Ai ) − i∈S1 + X (i1 .14. and more generally. Solve the Monty Hall problem again assuming that your friend randomly opens one of the two doors (without making sure that there is no prize behind it). If one of the four is defective. . . i2 )|1 ≤ i1 < i2 ≤ n}. A batch of one hundred items is inspected by testing four randomly selected items. If the prize is behind the door your friend picked then you start over the game.15. Show that P (A ∩ B|B) = P (A|B).i2 )∈S2 P (Ai1 ∩ Ai2 ∩ Ai3 ) (i1 . Examples. . the batch is rejected. Let A1 . P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A ∩ B) −P (B ∩ C) − P (A ∩ C) +P (A ∩ B ∩ C). Let S1 = {i|1 ≤ i ≤ n}. + (1)n−1 P (∩nk=1 Ak ). im ) of indices that satisfy 1 ≤ i1 < i2 < . A2 .6 Problems for Next Lecture Problem 9. 1. What is the probability that the batch is accepted if it contains five defectives? Problem 15.