Principles of Statistics

Week 4
Conditional probability.
Applying conditional probability to higher- order intersections.
Unconditional and inverse probabilities. Bayes’ theorem (2.4).
Independence. (2.5)

Mihaela Angelova
mihaela.urb@gmail.com
 Main topics
Conditional Probability
Any probability that is revised to take into
account the (known) occurrence of other events
 Conditional Probability
•Experiment: toss a fair die
•What is the probability that 6 appears
•Event A={6 appears}={6}
•P(A)= 1/6

•What is the probability that 6 appears if we

know an even number appears
•B={2,4,6}
•P(A|B) = 1/3
 Conditional Probability
• The symbol P(A|B)— read “the probability of A given B” is
used to denote a conditional probability - the probability
that A will occur given that B has already occurred.
• Suppose that S is a finite sample space with n outcomes, all
equally likely. Assume that A and B are two events
containing a and b outcomes, respectively, and let c
denote the number of outcomes in the intersection of A and
B.
• The conditional probability of A given B is the ratio of c to b.
But c/b can be written as the quotient of two other ratios.

Bayes’ Theorem
Conditional Probability
 Conditional Probability
• Toss a fair die
• What is the probability to appear a number
smaller than 5, given the number is odd?
 Conditional Probability
• Toss a fair die
• What is the probability to appear a number
smaller than 5, given the number is odd?

• S={1,2,3,4,5,6}
• A={1,2,3,4} N<5
• B={1,3,5} N is odd
• A ∩ B = {1,3}
• P(A ∩ B) = 𝑃 {1,3} = 𝑃(1) + 𝑃(3)
∩
 Conditional Probability
•Find P(A ∩ B) if P(A) = 0.2, P(B) = 0.4, and
P(A|B)+ P(B|A)=0.75
 Conditional Probability
•Find P(A ∩ B) if P(A) = 0.2, P(B) = 0.4, and
P(A|B)+ P(B|A)=0.75
 Conditional Probability
•Suppose that two fair dice are tossed.
•What is the probability that the sum equals
10 given that it exceeds 8?
 Conditional Probability
• Suppose that two fair dice are tossed.
• What is the probability that the sum equals 10 given that it exceeds 8?
• Sample space
S={(1,1), (1,2)…(1,6)
(2,1), (2,2), ..(2,6)
…
(6,1), (6,2)…(6,6)} 36 outcomes
• A={(4,6), (5,5),(6,4)} - 3 outcomes for sum equals 10
• B={(3,6), (4,5), (4,6),(5,4), (5,5), (5,6), (6,3), (6,4), (6,5), (6,6)}
10 outcomes for sum exceeds 8 P(B)=10/36
• A ∩ B = {(4,6),(5,5),(6,4)} P(A ∩ B)=3/36
 Conditional Probability
•A card is drawn from a poker deck. What is
the probability that the card is a club, given
that the card is a king?
•let C be the event “Card is a club”
•let K be the event “Card is a king.”
 Conditional Probability
•A card is drawn from a poker deck. What is
the probability that the card is a club, given
that the card is a king?
•P(C ∩ K) =1/52 (at the same time club and king)
The king is equally likely to be a heart,
diamond, club, or spade.
•P(K)=4/52
 Applying Conditional Probability to
Higher-Order Intersections
• frequently useful expression for the probability of
an intersection
P(A ∩ B)= P(A|B). P(B)

By thinking of A ∩ B as a single event— let's say D

P(A ∩ B ∩ C)= P(C ∩ D)= P(C|D).P(D)
= P(C|A ∩ B).P(A ∩ B)= P(C|A ∩ B).P(B|A).P(A)
Repeating this same argument for n events, A1 , A2
,..., An , gives a formula for the general case:

P(A1 ∩ A2 ∩…∩ An )=
P(An|A1 ∩ A2 ∩…∩ An−1) x P(An−1|A1 ∩ A2 ∩…∩ An−2)…x P(A2|A1 ).P(A1)
Applying Conditional Probability to
Higher-Order Intersections
• An urn contains six white chips, four black
chips, and five red chips. Five chips are drawn
out, one at a time and without replacement.
• What is the probability of getting the sequence
(black, black, red, white, white)?
 Applying Conditional Probability to
Higher-Order Intersections
• An urn contains six white chips, four black chips, and five
red chips. Four chips are drawn out, one at a time and
without replacement.
• What is the probability of getting the sequence (black,
black, red, white, white)?

• Sample space:
• Event A: black chip 1 st draw
• Event B: black chip 2 nd draw
• Event C: red chip 3 rd draw
• Event D: white chip 4 th draw
• Event E: white chip 5 th draw

P(A ∩ B ∩ C ∩ 𝐷 ∩ E )=
𝑃(E|A ∩ 𝐵 ∩ 𝐶 ∩ 𝐷 )x 𝑃(D|A ∩ 𝐵 ∩ 𝐶) x 𝑃(𝐶|A ∩ B) x 𝑃(B|A) x 𝑃(A)
 Applying Conditional Probability to
Higher-Order Intersections
• An urn contains 6 white chips, 4 black chips, and 5 red
chips (overall 15 chips). Five chips are drawn out, one at
a time and without replacement.
• What is the probability of getting the sequence (black,
black, red, white, white)?

P(A ∩ B ∩ C ∩ 𝐷 ∩ E )=
P(E|A ∩ 𝐵 ∩ 𝐶 ∩ 𝐷 )x P(D|A ∩ 𝐵 ∩ 𝐶) x P(𝐶|A ∩ B) x P(B|A) x P(A) =
 Applying Conditional Probability to
Higher-Order Intersections
• An urn contains 12 balls with the numbers 1 to 12.
What is the probability of getting the sequence
(7, 4, 1, 8, 10)?
 Applying Conditional Probability to
Higher-Order Intersections
• An urn contains 12 balls with the numbers 1 to 12.
What is the probability of getting the sequence
(7, 4, 1, 8, 10)?
Main topics

Unconditional probability
 Unconditional probability
• A set of events A1, A2, . . . , An is a “partition” of S
if every outcome in the sample space belongs to
one and only one of the Ai ’s
• That is, the Ai ’s are mutually exclusive and their
union is S
• Ui A i = 𝑆
• Ai ∩ A j = ∅ 𝑓𝑜𝑟 i ≠ j
• P(Ai )>0 for i =1,2,...,n

Partitioned sample space

 Unconditional probability
• Then for any event B in the sample space S
• B=B ∩ S
• B=B ∩ (A1 ∪ A 2 ∪…A n )
• B =(B ∩ A 1) ∪ (B ∩ A 2) ∪ …∪ (B ∩ A n)
• From probability axiom 3 ⇒
• P(B)= P(B ∩ A1 )+ P(B ∩ A2 )+…+ P(B ∩ An )
⇒ P(B)= P(B|A1)P(A1 )+ P(B|A2)P(A2 )+…+
P(B|An)P(An)

Partitioned sample space

 Unconditional probability
• Theorem for Unconditional probability

• An unconditional probability is the chance that a single

outcome results among several possible outcomes.

• The term refers to the likelihood that an event will take

place irrespective of whether any other events have
taken place or any other conditions are present.
Unconditional probability

• Bayes’ theorem for conditional probability

• From unconditional probability theorem=>

P(B)= P(B|A1)P(A1)+P(B|A2)P(A2)+…+P(B|An)P(An )
Bayes’ theorem

In the denominator we
put the condition
Bayes’ theorem
 Unconditional probability
• A toy manufacturer buys ball bearings from three different
suppliers—50% of the total order comes from supplier 1,
30% from supplier 2, and the rest from supplier 3.
• Past experience has shown that the quality-control
standards of the three suppliers are not all the same. 2%
of the ball bearings produced by supplier 1 are defective,
while suppliers 2 and 3 produce defective bearings 3%
and 4% of the time, respectively.
• What proportion of the ball bearings in the toy
manufacturer’s inventory are defective?
 Unconditional probability
• Event Ai =bearing comes from supplier i
• Event B = a bearing in the manufacture in defective
• P(B)=?

• P(A1)=0.5, P(A2)=0.3, P(A3)=0.2

• P(B|A1)=0.02, P(B|A2)=0.03, P(B|A3)=0.04

• Unconditional theorem
• P(B)=P(B|A1).P(A1)+P(B|A2).P(A2)+P(B|A3).P(A3) =
0.02x0.5 + 0.03x0.3 + 0.04x0.2 = 0.027
 Unconditional probability

• Urn 1 contains 3 red chips and 5 white chips;

• urn 2 contains 4 reds and 4 whites;
• urn 3 contains 5 reds and 3 whites.

• One urn is chosen at random and one chip is drawn from

that urn.
• Given that the chip drawn was red, what is the probability
that urn 3 was the urn sampled?
 Unconditional probability
• Event Ai – urn i is chosen, i=1,2,3
• P(Ai)= 1/3 Urn 1 Urn 2 Urn 3
• Event B – a red chip is drawn
•3 r •4 r •5 r
• P(A3|B)=?
•5 w •4 w •3 w
From Bayes’ Theorem =>

2 2 3 3
 Unconditional probability
• Suppose 15% of olympic athletes take drugs
• if P(fails test|takes drugs)=0.9
• and P(fails test|doesn’t take drugs)=0.12
• what is the probability that an athlete who fails the test
has taken drugs?
 Unconditional probability
• Suppose 15% of olympic athletes take drugs
• if P(fails test|takes drugs)=0.9
• and P(fails test|doesn’t take drugs)=0.12
• what is the probability that an athlete who fails the test
has taken drugs?
 Unconditional probability
• Two sections of a senior probability course are being
taught. From what Anna has heard about the two
instructors listed, she estimates that her chances of
passing the course are 0.85 if she gets Professor X and
0.60 if she gets Professor Y .
• The section into which she is put is determined by the
registrar.
• Suppose that her chances of being assigned to Professor
X are four out of ten.
• Fifteen weeks later we learn that Anna did, indeed, pass
the course. What is the probability she was enrolled in
Professor X’s section?
 Unconditional probability
• P(pass|X)=0.85
• P(pass|Y)=0.6
• P(X)=0.4 => P(Y)=0.6
• P(X|pass)=?
Main topics
Independence
• Two events A and B are said to be
independent if P(A∩B)=P(A).P(B)

• Equivalently,
P(A|B) = P(A)= P(A|Bc)

• Intuition:
The probability of one event occurring is not
affected by whether or not the other occurs
Independence
• Suppose you toss a fair coin twice
• Consider the events:
• A: head on first throw
• B: head on second throw
• P(A ∩ B) =?
• P(A).P(B) =?
• Are they independent?
P(A∩B)=P(A).P(B) ?
 Independence
• A ={(H,H), (H,T)} => P(A)= ½ (head on first throw)
• B = {(H,H), (T,H)} => P(B)= ½ (head on second throw)
• A ∩ B = 𝐻,𝐻 => P(A ∩ B )= ¼ (head both on first & second throw)

• P(A).P(B)= ½. ½ = P(A∩B),

so the events are

independent
Independence
• Suppose you throw a fair die.
• Consider the events:
• C=the number is odd
• D=the number is less than 4
• 𝑃(C ∩ D) =?
• 𝑃(C). 𝑃(D) =?
• Are they independent?
Independence
• S={1,2,3,4,5,6}
• C={1,3,5} =>P(C)= ½ number is odd
• D={1,2,3} =>P(D)= ½ less than 4
• C ∩ D = {1,3} =>
P(C ∩ D) =2/6=1/3
• ≠ 𝑃(𝐷) × 𝑃(C)=1/4

• so they are not independent

 Deducing Independence
• Sometimes the physical circumstances
surrounding two events make it obvious that the
occurrence (or nonoccurrence) of one has
absolutely no influence or effect on the
occurrence (or nonoccurrence) of the other.

• If that should be the case, then the two events

will necessarily be independent
 Deducing Independence
• Urn I has 3 red, 2 black and 5 white chips;
• Urn II has 2 red, 4 black, and 3 white.
• One chip is drawn at random from each urn.
• What is the probability that both chips are the
same color?
Urn 1 Urn 2 • Event R1 – draw red chip from I urn
• Event B1 – draw black chip from I urn
•3r •2r • Event W1 – draw white chip from I urn
•2b •4b • Event R2 – draw red chip from II urn
•5w •3w • Event B2 – draw black chip from II urn
• Event W2 – draw white chip from II urn

• Event E – draw chips from the same

colour
Deducing Independence
• P(E)=P{(R1 ∩ 𝑅2) ∪ (B1 ∩ B2) ∪ (W1 ∩ 𝑊2)} =
P(R1).P(𝑅2) + P(B1).P(B2) + P(W1).P(W2)

• Because the individual draws are independent

Urn 1 Urn 2

•3 r •2 r
•2 b •4 b
•5 w •3 w
Independence of More Than Two Events

Events A, B and C are independent if:

• 𝑃 (A ∩ 𝐵 ∩ 𝐶) = 𝑃(A) × 𝑃(B) × 𝑃(C)

and
• 𝑃 (A ∩ 𝐵) = 𝑃(A) × 𝑃(B)
• 𝑃 (A ∩ C) = 𝑃(A) × 𝑃(C)
• 𝑃 (𝐵 ∩ C) = 𝑃(B) × 𝑃(C)

• Two events are mutually exclusive if

A∩B=∅
 Announcements
• Midterm - 16 November – online test
Thank you for your attention!

Mihaela Angelova
mihaela.urb@gmail.com