Lecture 4

Lecture 4: Probability Rules II
Chen Xu
Department of Mathematics and Statistics
University of Ottawa
Chen
c Xu (uOttawa) Prob rules II All rights reserved 1 / 21
Outline
1 Review conditional probability
2 Multiplication, Total, and Bayes’ Rules
3 Summary
Chen
Review conditional probability
3 Summary
Chen
Review: conditional probability
Consider two events A and B with P(B) > 0. The conditional

probability of A, given that B has occurred is defined by
P(A ∩ B)
P(A|B) = .
P(B)
• “Given event B” indicates that B has occurred and is now treated as

a condition, which may affect event A.
• In A|B, the randomness come from A, not B.
Chen
Example 1: Hockey teams

S = {Canadian citizens}
A = {A Senators fan}, B = {A Maple Leafs fan}
n(A) n(B)
P(A) = , P(B) =
n(S) n(S)
n(A∩B)
P(A ∩ B) n(S) n(A ∩ B)
P(A|B) = = n(B)
=
P(B) n(B)
n(S)
n(A∩B)
P(A ∩ B) n(S) n(A ∩ B)
P(B|A) = = n(A)
=
P(A) n(A)
n(S)
Chen
Example 1: Hockey teams

S = {Canadian citizens}
A = {A Senators fan}, B = {A Maple Leafs fan}
n(A) n(B)
P(A) = , P(B) =
n(S) n(S)
n(A∩B)
P(A ∩ B) n(S) n(A ∩ B)
P(A|B) = = n(B)
=
P(B) n(B)
n(S)
n(A∩B)
P(A ∩ B) n(S) n(A ∩ B)
P(B|A) = = n(A)
=
P(A) n(A)
n(S)
• Usually, P(A|B) 6= P(B|A). P(A|B) is the proportion of Maple Leafs

fans who also support Senators; P(B|A) is the proportion of Senators
fans who also support Maple Leafs.
Chen
Example 2: Bilingualism (A = English, B = French)
P(A) = 60% + 37.1% = 97.1%, P(B) = 1.6% + 37.1% = 38.7%
P(A ∩ B) 37.1% P(A ∩ B) 37.1%

P(A|B) = = ≈ 95.9%, P(B|A) = = ≈ 38.2%
P(B) 38.7% P(A) 97.1%
Chen
Remarks on conditional probability
• Condition “Given B” may affect P(A) or P(Ac ), but it does not

change the compliment relationship between A and Ac .
P(A) + P(Ac ) = 1
P(A|B) + P(Ac |B) = 1
• Condition “Given C” may affect P(A) or P(B), but it does not

change the addition rule.
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

P(A ∪ B|C ) = P(A|C ) + P(B|C ) − P(A ∩ B|C )
Chen
Multiplication, Total, and Bayes’ Rules
3 Summary
Chen
Multiplication rule
By definition,
P(A ∩ B)
P(A|B) =
P(B)
Chen
Multiplication rule
By definition,
P(A ∩ B)
P(A|B) =
P(B)
• If you know P(A|B) and P(B), then
Multiplication rule
P(A ∩ B) = P(A|B)P(B)
Chen
Multiplication rule
By definition,
P(A ∩ B)
P(A|B) =
P(B)
• If you know P(A|B) and P(B), then
Multiplication rule
P(A ∩ B) = P(A|B)P(B)
• Of course, if you know P(B|A) and P(A),
P(A ∩ B) = P(B|A)P(A)
Chen
Example 3: light bulbs
Consider a box of 20 light bulbs, where 2 bulbs are defective. Suppose

that we randomly choose two bulbs without replacement. What is the
probability that both bulbs are not defective?
Chen

• A={1st bulb is good}

• B={2nd bulb is good}
Chen

• A={1st bulb is good}

• B={2nd bulb is good}
• By multiplication rule,
18 17
P(A ∩ B) = P(A)P(B|A) = × = 0.805
20 19
Chen
Multiplication rule (extended)
The multiplication rule can be extended to the case with more than two
events.
P(E1 ∩ . . . ∩ En ) = P(E1 )P(E2 |E1 )P(E3 |E1 ∩ E2 ) . . . P(En |E1 ∩ . . . ∩ En−1 )
Chen
Example 4: Picking balls

There are 3 white balls and 7 black balls in a bag. Cindy randomly picks 3
balls one at time from the bag (without replacement). What is the
probability that she gets a black ball first, then two white balls?
Chen

• A={1st ball is black}, B={2nd ball is white}, C ={3rd ball is white}
• P(A ∩ B ∩ C ) =?
Chen

• P(A ∩ B ∩ C ) =?
P(A ∩ B ∩ C ) = P(A)P(B|A)P(C |A ∩ B)
7 3 2
= × ×
10 9 8
≈ 5.8%
Chen

• P(A ∩ B ∩ C ) =?
P(A ∩ B ∩ C ) = P(A)P(B|A)P(C |A ∩ B)
7 3 2
= × ×
10 9 8
≈ 5.8%
• By classical method,
n(A ∩ B ∩ C ) 7P1 × 3P2
P(A ∩ B ∩ C ) = = ≈ 5.8%
n(S) 10P3
Chen
Total probability rule
Let E1 , E2 , . . . Ek be a partition of S, we have

Total probability rule
P(A) = P(A ∩ E1 ) + P(A ∩ E2 ) + . . . + P(A ∩ Ek )

= P(E1 )P(A|E1 ) + P(E2 )P(A|E2 ) + . . . + P(Ek )P(A|Ek )
Chen
Example 5: Disease prevalence

In a city, 25% citizens are young, 45% citizens are mid-aged, and 30%
citizens are old. A certain disease has a prevalence to young people at 5%,
to mid-aged people at 8%, and to old people at 12%. What is the general
prevalence of this disease in the city?
• S={citizens in the city}
• D={a randomly selected citizen who has the disease}, P(D) =?
Chen
Example 5: Disease prevalence

In a city, 25% citizens are young, 45% citizens are mid-aged, and 30%
citizens are old. A certain disease has a prevalence to young people at 5%,
to mid-aged people at 8%, and to old people at 12%. What is the general
prevalence of this disease in the city?
• S={citizens in the city}
• D={a randomly selected citizen who has the disease}, P(D) =?
• Let Y , M, O denote the event that a citizen is young, mid-aged, or
old respectively.
Note that Y , M, O form a partition of S, By the total probability rule,
P(D) = P(Y )P(D|Y ) + P(M)P(D|M) + P(O)P(D|O)

= 25% × 5% + 45% × 8% + 30% × 12%
= 8.45%
Chen
Bayes’ rule
Suppose we know P(A), P(B), and P(A|B), then
P(A ∩ B)
P(B|A) = (by definition)
P(A)
P(B)P(A|B)
= (by multiplication rule)
P(A)
In Example 5, we know P(O) = 30%, P(D|O) = 12%, and find that

P(D) = 8.45%. Suppose we randomly select a citizen and find that he has
the disease. What is the probability he is old?
P(O)P(D|O) 30% × 12%

P(O|D) = = ≈ 42.6% > P(O) = 30%
P(D) 8.45%
Chen
Bayes’ rule
In general, let E1 , . . . , Ek be a partition of S. We have, for 1 ≤ j ≤ k,
P(A ∩ Ej )
P(Ej |A) =
P(A)
P(Ej )P(A|Ej )
=
P(A)
P(Ej )P(A|Ej )
= (by what rule?)
P(A|E1 )P(E1 ) + . . . + P(A|Ek )P(Ek )
Bayes’ rule
P(Ej )P(A|Ej )
P(Ej |A) =
P(A|E1 )P(E1 ) + . . . + P(A|Ek )P(Ek )
Chen
Example 6: Nissan car
Nissan sold 3 models of cars in 1999: Sentra, Maxima, and Pathfinder. Of the
vehicles sold, 50% were Sentras, 30% were Maximas, and 20% were Pathfinders.
In the same year, 12% of Sentras, 15% of Maximas, and 25% of Pathfinders had
a problem in ignition system. (a) What is the probability that a 1999 Nissan car
had a problem with ignition? (b) Suppose a 1999 Nissan car has a problem with
ignition. What is the probability that it is a Pathfinder?
Chen
Nissan sold 3 models of cars in 1999: Sentra, Maxima, and Pathfinder. Of the
vehicles sold, 50% were Sentras, 30% were Maximas, and 20% were Pathfinders.
In the same year, 12% of Sentras, 15% of Maximas, and 25% of Pathfinders had
a problem in ignition system. (a) What is the probability that a 1999 Nissan car
had a problem with ignition? (b) Suppose a 1999 Nissan car has a problem with
ignition. What is the probability that it is a Pathfinder?
• Let E1 = {Sentra}, E2 = {Maxima}, and E3 = {Pathfinder}. Let

A = {defective ignition}
• P(E1 ) = 50%, P(E2 ) = 30%, P(E3 ) = 20%
• P(A|E1 ) = 12%, P(A|E2 ) = 15%, P(A|E3 ) = 25%
Chen
• (a) By total probability rule,
P(A) = P(E1 )P(A|E1 ) + P(E2 )P(A|E2 ) + P(E3 )P(A|E3 )

= 50% × 12% + 30% × 15% + 20% × 25% = 0.155
• (b) By Baye’s rule,
P(A ∩ E3 ) P(E3 )P(A|E3 ) 0.2 × 0.25

P(E3 |A) = = = = 0.32
P(A) P(A) 0.155
Chen
• (a) By total probability rule,
P(A) = P(E1 )P(A|E1 ) + P(E2 )P(A|E2 ) + P(E3 )P(A|E3 )

= 50% × 12% + 30% × 15% + 20% × 25% = 0.155
• (b) By Baye’s rule,
P(A ∩ E3 ) P(E3 )P(A|E3 ) 0.2 × 0.25

P(E3 |A) = = = = 0.32
P(A) P(A) 0.155
• Note that P(E3 ) = 20%. Given the defective ignition, the chance for
getting a Pathfinder is increased by 12%.
Chen
Summary
3 Summary
Chen
Summary
Summary
• Multiplication rule:
P(A ∩ B) = P(B)P(A|B)
• Total probability rule:
P(A) = P(A|E1 )P(E1 ) + . . . + P(A|Ek )P(Ek )
• Bayes’ rule:
P(Ej )P(A|Ej )
P(Ej |A) =
P(A|E1 )P(E1 ) + . . . + P(A|Ek )P(Ek )
Chen
Summary
See you next time!
• Email: cx3@uottawa.ca
Chen

Lecture 4

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 4

Uploaded by

Copyright:

Available Formats

Lecture 4: Probability Rules II

1 Review conditional probability

2 Multiplication, Total, and Bayes’ Rules

1 Review conditional probability

2 Multiplication, Total, and Bayes’ Rules

Review: conditional probability

Consider two events A and B with P(B) > 0. The conditional

• “Given event B” indicates that B has occurred and is now treated as

Example 1: Hockey teams

Example 1: Hockey teams

• Usually, P(A|B) 6= P(B|A). P(A|B) is the proportion of Maple Leafs

Example 2: Bilingualism (A = English, B = French)

P(A) = 60% + 37.1% = 97.1%, P(B) = 1.6% + 37.1% = 38.7%

P(A ∩ B) 37.1% P(A ∩ B) 37.1%

Remarks on conditional probability

• Condition “Given B” may affect P(A) or P(Ac ), but it does not

P(A|B) + P(Ac |B) = 1

• Condition “Given C” may affect P(A) or P(B), but it does not

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

1 Review conditional probability

2 Multiplication, Total, and Bayes’ Rules

• If you know P(A|B) and P(B), then

• If you know P(A|B) and P(B), then

• Of course, if you know P(B|A) and P(A),

Example 3: light bulbs

Consider a box of 20 light bulbs, where 2 bulbs are defective. Suppose

Example 3: light bulbs

Consider a box of 20 light bulbs, where 2 bulbs are defective. Suppose

• A={1st bulb is good}

Example 3: light bulbs

Consider a box of 20 light bulbs, where 2 bulbs are defective. Suppose

• A={1st bulb is good}

Multiplication rule (extended)

P(E1 ∩ . . . ∩ En ) = P(E1 )P(E2 |E1 )P(E3 |E1 ∩ E2 ) . . . P(En |E1 ∩ . . . ∩ En−1 )

Example 4: Picking balls

Example 4: Picking balls

Example 4: Picking balls

Example 4: Picking balls

Total probability rule

Let E1 , E2 , . . . Ek be a partition of S, we have

P(A) = P(A ∩ E1 ) + P(A ∩ E2 ) + . . . + P(A ∩ Ek )

Example 5: Disease prevalence

Example 5: Disease prevalence

Note that Y , M, O form a partition of S, By the total probability rule,

P(D) = P(Y )P(D|Y ) + P(M)P(D|M) + P(O)P(D|O)

Suppose we know P(A), P(B), and P(A|B), then

In Example 5, we know P(O) = 30%, P(D|O) = 12%, and find that

P(O)P(D|O) 30% × 12%

In general, let E1 , . . . , Ek be a partition of S. We have, for 1 ≤ j ≤ k,

Example 6: Nissan car

Example 6: Nissan car

• Let E1 = {Sentra}, E2 = {Maxima}, and E3 = {Pathfinder}. Let

Example 6: Nissan car

• (a) By total probability rule,

P(A) = P(E1 )P(A|E1 ) + P(E2 )P(A|E2 ) + P(E3 )P(A|E3 )

• (b) By Baye’s rule,

P(A ∩ E3 ) P(E3 )P(A|E3 ) 0.2 × 0.25

Example 6: Nissan car

• (a) By total probability rule,

P(A) = P(E1 )P(A|E1 ) + P(E2 )P(A|E2 ) + P(E3 )P(A|E3 )

• (b) By Baye’s rule,

P(A ∩ E3 ) P(E3 )P(A|E3 ) 0.2 × 0.25

1 Review conditional probability