CH05 Conditioning and Independence

Elementary Probability Theory, Fourth Edition
Kai Lai Chung, Farid AitSahlia
Chapter 5 Conditioning and Independence
Probability Theory and Statistics

Zhengyao Bai, PhD, Professor
School of Information Science & Engineering, Yunnan University
Email: baizhy@ynu.edu.cn
Mobile: 13577060089
QQ & Wechat: 1302255793
Room: 1417#, SISE Building
0
• Examples of conditioning
• Basic formulas
• Sequential sampling
• Polya's urn scheme
• Independent and relevance
1
• The probability of a set A is its weighted proportion
relative to the sample space Ω: P(A) = |A|/|Ω|.
• When Ω is countable and each point ω has the weight

P(ω) = P({ω}) attached to it:
2
• In many questions we are interested in the proportional
weight of one set A relative to another set S. this means
the proportional weight of the part of A in S, namely the
intersection A ∩ S, or AS, relative to S.
It is called the conditional probability of A relative to S.
3
• Example 1. All students on a certain college campus are
polled as to their reaction to a certain presidential
candidate.
Let D denote those who favor him. Now the student population Ω
may be cross-classified in various ways, for instance according to sex,
age, race, etc. Let A = female, B = black, C = of voting age.
Then Ω is partitioned into 8 subdivisions ABC, ABCc,..., AcBcCc. The
set D will in general cut across the various divisions.
These two formulas denote the proportion of male black students of

voting age who favor the candidate, and the proportion of male students
of voting age who do not favor the candidate, respectively.
4
• Example 2. A perfect die is thrown twice. Given that the
total obtained is 7, what is the probability that the first
point obtained is k, 1 ≤ k ≤ 6?
Let us look at the outcome space: 62 points in the space
5
• Example 2. The outcomes with total equal to 7 are those
on the second diagonal, and their number is 6. Hence the
conditional probability is equal to 1/6.
6
• Example 2. Let X1 and X2 respectively denote the point
obtained in the first and second throw.
Other examples:
7
• Example 2. Given X1 = 4, what is the probability that X2
= k provided that the two trials are independent?
Since the second throw is not affected by the first, so the

conditional probability P{X2 = k | X1 = 4} must be the same
as the unconditional one P{X2 = k}.
8
• Example 3. Suppose you toss a biased coin repeatedly
until a head turns up. Let X be the waiting time.
Knowing that it has fallen tails three times, what is the
probability that it will fall heads within the next two
trials?
It is the conditional probability:
How to compute the probability?
9
• Example 3. We know that
From which we can calculate: How to get the sum?
10
• Example 3. The probability that the coin falls heads (at
least once in two trials):
11
• Example 4. An analogy between the geometrical
distribution and the exponential distribution:
waiting time
12
• Example 5. Consider all families with two children and
assume that boys and girls are equally likely. The sample
space may be denoted schematically as: {(bb), (bg), (gb),
(gg)}. The order in each pair is the order of birth. the 4
points have probability 1/4 each.
13
• Example 5. Q1: If a family is chosen at random from Ω
and found to have a boy in it, what is the probability that
it has another boy, namely that it is of the type (b, b)?
14
• Example 5. Q2: If a child is chosen at random from these
families and is found to be a boy, what is the probability
that the other child in his family is also a boy?
15
• Basic formulas
• Simple but fundamental propositions involving
conditional probabilities
• Proposition 1 (general mulplicative formula) For
arbitrary events A1, A2, ..., An, we have
provided P(A1A2...An-1) > 0.

• Application of multipllicative formula
16
• Basic formulas
• Proposition 2 (total probability formula) Sppose that
is a partition of the sample space into disjoint sets. Then

for any set B we have
17
• Basic formulas
• Proposition 3 (Bayes' Theorem) Under the assumption
and notation of Proposition 2, we have also
provided P(B) > 0. The denominator above is equal to

P(B) by Proposition 2.
18
• Basic formulas
• Example 6. What is the probability of throwing six
perfect die and getting six different faces? Put
etc. Assuming that the dice act independently, we have
(Proposition 1)
19
• Basic formulas
• Example 7. The family dog is missing after the picnic.
Three hypotheses are suggested:
(A) it has gone home;
(B) it is still worrying that big bone in the picnic area;
(C) it has wandered off into the woods.
The a priori probabilities, which are assessed from the
habits of the dog, are estimated respectively to be 1/4 , 1/2 ,
1/4 . A child each is sent back to the picnic ground and the
edge of the woods to look for the dog. If it is in the former
area, it is a cinch (90%) that it will be found; if it is in the
latter, the chance is only a toss-up (50%).
20
• Basic formulas
• Q1: What is the probability that the dogwill be found in
the park?
Let D = “dog will be found in the park.” Then we have
21
• Basic formulas
• Q2: What is the probability that the dog will be found at
home?
Let D' = “the dog will be found at home.” and assume
that P(D'| A) = 1. Clearly P(D'| B) = P(D'| C) = 0. Then we
have
22
• Basic formulas
• Q3: What is the probability that the dog is “lost”? It is
23
• Basic formulas
• Example 8. Urn one contains 2 black and 3 red balls; urn
two contains 3 black and 2 red balls. We toss an unbiased
coin to decide on the urn to draw from but we do not
know which is which. Suppose the first ball drawn is
black and it is put back.
U1 U2
24
• Basic formulas
• Example 8. Q1: What is the probability that the second
ball drawn from the same urn is also black?
For two urns U 1 and U 2 , the a priori probability that
either one is chosen by the coin-tossing is 1/2:
Denote the event that the first ball is black by B1, that the
second ball is black by B2.
25
• Basic formulas
Let A1 = B1 is from U1, A2 = B1 is from U2. The second
drawing is conditioned on B 1 , the probabilities of the
hypotheses are really conditional ones:
it is obvious that
26
• Basic formulas
We obtain the conditional probability:
comparing this with
We see that the knowledge of the first ball drawn being

black has strengthened the probability of drawing a second
black ball, because it has increased the likelihood that we
have picked the urn with more black balls.
27
• Basic formulas
• Example 8. Q2: Given that the first two balls drawn are
both black and put back, what is the probability of
drawing a third black ball from the same urn?
In similar notations, we have:
It is easy to see that we can extend the result to any

number of drawings:
28
• An urn contains b black balls and r red balls. One ball is
drawn at a time without replacement. Let X n = 1 or 0
depending on whether the nth ball drawn is black or red.
Each sample point ω is then just the sequence {X1(ω),
X2(ω),... , Xb+r(ω)}, briefly {Xn, 1 ≤ n ≤ b + r}.
• Problem. A ball is drawn from the urn and discarded.

Without knowing its color, what is the probability that a
second ball drawn is black?
29
• Let , then we have
30
• P(B 2 ) = P(B 1 ), namely if we take into account both
possibilities for the color of the first ball, then the
probabilities for the second ball are the same as if no ball
had been drawn (and left out) before.
• Theorem 1. For each n, we have
31
• Theorem 2 (Poisson’s Theorem). Suppose in an urn
containing b black and r red balls, n balls have been
drawn first and discarded without their colors being
noted. If m balls are drawn next, the probability that
there are k black balls among them is the same as if we
had drawn these m balls at the outset [without having
discarded the n balls previously drawn].
32

• As before the urn contains b black and r red balls to
begin with, but after a ball is drawn each time, it is
returned to the urn and c balls of the same color are
added to the urn, where c is an integer, and when c < 0
adding c balls means subtracting -c balls.
• Polya’s urn model when c >= -1.
33

• Problem. What is the probability that in Polya’s model
the first three balls drawn have colors {b, b, r} in this
order? or {b, r, b}? or {r, b, b}?
34

• Theorem 3. The probability of drawing (from the
beginning) any specified sequence of k black balls and n
− k red balls is equal to
for all n ≥ 1 if c ≥ 0; and for 0 ≤ n ≤ b + r if c = −1.
35

• An extreme and extremely important case of conditioning
occurs when the condition has no effect on the probability.
• Definition of Independent Random Variables. The

countably valued random variables X1, ... , Xn are said to
be independent iff for any real numbers x 1 , ... , x n , we
have
36

• Proposition 4. We have for arbitrary countable sets S1, ... ,
Sn:
• Proposition 5. The events {X1 ∈ S1},... , {Xn ∈ Sn}are

independent.
• Proposition 6. Let 1, ... , n be an arbitrary real-valued

function on (−∞, ∞); then the random variables
1(X1), ... , n(Xn)
are independent.
37

• Example 10. A letter from Pascal to Fermat (dated
Wednesday, 29th July, 1654), contains, among many other
mathematical problems, the following passage:
38

M. de Mere told me that he had found a fallacy in the
theory of numbers, for this reason: If one undertakes to get
a six with one die, the advantage in getting it in 4 throws is
as 671 is to 625. If one undertakes to throw 2 sixes with two
dice, there is a disadvantage in undertaking it in 24 throws.
And nevertheless 24 is to 36 (which is the number of
pairings of the faces of two dice) as 4 is to 6 (which is the
number of faces of one die). This is what made him so
indignant and made him say to one and all that the
propositions were not consistent and Arithmetic was self-
contradictory: but you will very easily see that what I say is
correct, understanding the principles as you do.
39

• Example 10. To throw a six with 1 die in 4 throws means
to obtain the point “six” at least once in 4 trials. Define Xn,
1 ≤ n ≤ 4, as follows:
• assume that X1, X2, X3, X4 are independent. Put An = {Xn =

6}; then the event in question is A1∪A2 ∪A3∪A4. It is
easier to calculate the probability of its complement. The
trials are assumed to be independent and the dice
unbiased.
40

• Example 10. Next consider two dice; let (X'n, X''n) denote
the outcome obtained in the nth throw of the pair, and let:
Then
41

• Example 11. If two points are picked at random from the
interval [0, 1],what is the probability that the distance
between them is less than 1/2?
The double integral can be easily

computed geometrically.
See the Figure. The probability

is equal to the area of S:
42

• Example 12. Suppose X 1 , X 2 , ... , X n are independent
random variables with distributions F1, F2, ... , Fn. Let
Find the distribution functions of M and m.
Using (5.5.7), we have for each x
43

• Example 12. As for the minimum, it is convenient to
introduce the “tail distribution” Gj corresponding to each
Fj as follows:
Then we have, using the analogue of (5.5.2) this time

with Sj = (xj ,∞),
44
Chapter
Chapter 6 5 Conditioning and Independence
To be updated…
THANKS
2020-10-7

CH05 Conditioning and Independence

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CH05 Conditioning and Independence

Uploaded by

Copyright:

Available Formats

Elementary Probability Theory, Fourth Edition

Kai Lai Chung, Farid AitSahlia

Chapter 5 Conditioning and Independence

Probability Theory and Statistics

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

• When Ω is countable and each point ω has the weight

Chapter 5 Conditioning and Independence

It is called the conditional probability of A relative to S.

Chapter 5 Conditioning and Independence

These two formulas denote the proportion of male black students of

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Since the second throw is not affected by the first, so the

Chapter 5 Conditioning and Independence

How to compute the probability?

Chapter 5 Conditioning and Independence

From which we can calculate: How to get the sum?

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

provided P(A1A2...An-1) > 0.

Chapter 5 Conditioning and Independence

is a partition of the sample space into disjoint sets. Then

Chapter 5 Conditioning and Independence

provided P(B) > 0. The denominator above is equal to

Chapter 5 Conditioning and Independence

etc. Assuming that the dice act independently, we have

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

comparing this with

We see that the knowledge of the first ball drawn being

Chapter 5 Conditioning and Independence

It is easy to see that we can extend the result to any

Chapter 5 Conditioning and Independence

• Problem. A ball is drawn from the urn and discarded.

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

• Theorem 1. For each n, we have

Chapter 5 Conditioning and Independence

Chapter 5 Conditioning and Independence

• Polya's urn scheme

Chapter 5 Conditioning and Independence

• Polya's urn scheme

Chapter 5 Conditioning and Independence

• Polya's urn scheme

for all n ≥ 1 if c ≥ 0; and for 0 ≤ n ≤ b + r if c = −1.

Chapter 5 Conditioning and Independence

• Independent and relevance

• Definition of Independent Random Variables. The

Chapter 5 Conditioning and Independence

• Independent and relevance