1.

2 - Sample Space and events

I

Denition
A sample space, denoted S , is the collection of all outcomes when
an experiment is performed.

1.2 - Sample Space and events

I

Denition
A sample space, denoted S , is the collection of all outcomes when
an experiment is performed.
I Example 1: Toss a coin two times

1.2 - Sample Space and events

I

Denition
A sample space, denoted S , is the collection of all outcomes when
an experiment is performed.
I Example 1: Toss a coin two times
S

= {HH , HT , TH , TT }

1.2 - Sample Space and events

I

Denition
A sample space, denoted S , is the collection of all outcomes when
an experiment is performed.
I Example 1: Toss a coin two times
S
I

= {HH , HT , TH , TT }

Example 2: Toss a coin until a head appears

1.2 - Sample Space and events

I

Denition
A sample space, denoted S , is the collection of all outcomes when
an experiment is performed.
I Example 1: Toss a coin two times
S
I

= {HH , HT , TH , TT }

Example 2: Toss a coin until a head appears

S

= {H , TH , TTH , TTTH , . . .}

Events
I

Denition
An event is a set of outcomes form the sample space. The events
are usually denoted A, B , C , . . .. If we regard S as a set, then an
event is a subset of S .

Events
I

Denition
An event is a set of outcomes form the sample space. The events
are usually denoted A, B , C , . . .. If we regard S as a set, then an
event is a subset of S .
I Union and intersection of events
I

The union of two events A and B , denoted A B , is the set of

all outcomes that are either in A or in B

Events
I

Denition
An event is a set of outcomes form the sample space. The events
are usually denoted A, B , C , . . .. If we regard S as a set, then an
event is a subset of S .
I Union and intersection of events
I

The union of two events A and B , denoted A B , is the set of

all outcomes that are either in A or in B
The intersection of two events A and B , denoted A B , is the
set of all outcomes that are both in A and in B

Events
I

Denition
An event is a set of outcomes form the sample space. The events
are usually denoted A, B , C , . . .. If we regard S as a set, then an
event is a subset of S .
I Union and intersection of events
I

The union of two events A and B , denoted A B , is the set of

all outcomes that are either in A or in B
The intersection of two events A and B , denoted A B , is the
set of all outcomes that are both in A and in B
The complement of an event A, denoted A, is the set of all
outcomes that are not in A

Events
I

Most of elementary operations on sets can be expressed in

terms of union, intersection and complement.

Events
I

Most of elementary operations on sets can be expressed in

terms of union, intersection and complement.
- Example: A \ B is the set of outcomes that are in A but not
in B ;

Events
I

Most of elementary operations on sets can be expressed in

terms of union, intersection and complement.
- Example: A \ B is the set of outcomes that are in A but not
in B ;
- show that A \ B = A B

Mutually exclusive and exhaustive events

I

Denition
A set of events A1 , A2 , A3 , . . . is mutually exclusive if and only if
Ai

Aj = ,

6= j

(1)

Mutually exclusive and exhaustive events

I

Denition
A set of events A1 , A2 , A3 , . . . is mutually exclusive if and only if
Ai

Aj = ,

6= j

(1)

Denition
A set of events A1 , A2 , A3 , . . . is exhaustive if
A1

A2 A3 . . . = S

(2)

Mutually exclusive and exhaustive events

I

Denition
A set of events A1 , A2 , A3 , . . . is mutually exclusive if and only if
Ai

6= j

(1)

Denition
A set of events A1 , A2 , A3 , . . . is exhaustive if
A1

Aj = ,

A2 A3 . . . = S

(2)

Denition
A set of events A1 , A2 , A3 , . . . is mutually exclusive and exhaustive
(called also partition) if (1) and (2) are both satised.

Probability Axioms
I

Axiom
A probability, denoted P is an application, P : S [0, 1],
satisfying:
I

P ()

= 0,

P (S )

= 1, and

Probability Axioms
I

Axiom
A probability, denoted P is an application, P : S [0, 1],
satisfying:
I
I

P () = 0, P (S ) = 1, and
for every set A1 , A2 , A3P
, . . . of mutually exclusive events,

P (A1 A2 A3 . . .)

P (Ai )

= P (A1 ) + P (A2 ) + P (A3 ) + . . .

Probability Axioms
I

Axiom
A probability, denoted P is an application, P : S [0, 1],
satisfying:
I
I

P () = 0, P (S ) = 1, and
for every set A1 , A2 , A3P
, . . . of mutually exclusive events,

P (A1 A2 A3 . . .)

P (Ai )

= P (A1 ) + P (A2 ) + P (A3 ) + . . .

Property
(basic rules of probability)
I

(union rule) P (A B )

= P (A) + P (B ) P (A B )

Probability Axioms
I

Axiom
A probability, denoted P is an application, P : S [0, 1],
satisfying:
I
I

P () = 0, P (S ) = 1, and
for every set A1 , A2 , A3P
, . . . of mutually exclusive events,

P (A1 A2 A3 . . .)

P (Ai )

= P (A1 ) + P (A2 ) + P (A3 ) + . . .

Property
(basic rules of probability)
I

(union rule) P (A B )

(complement rule) P (A)

= P (A) + P (B ) P (A B )
= 1 P (A)

Example
P (A \ B )

= P (A) P (A B )

Example
P (A \ B )

Proof.

= P (A) P (A B )

= (A \ B ) (A B ) (mutually exclusive events)

= P (A \ B ) + P (A B ), hence
P (A \ B ) = P (A) P (A B ).
A

P ( A)

Example
P (A \ B )

= P (A) P (A B )

Proof.

= (A \ B ) (A B ) (mutually exclusive events)

= P (A \ B ) + P (A B ), hence
P (A \ B ) = P (A) P (A B ).
A

P ( A)

Exercise
Let A, B and C be three events, show that
P (A B

C) =

P (A)+ P (B )+ P (C ) P (A B ) P (A C ) P (B C )+ P (A B C )

I Inclusion-Exclusion formula:

More generally, if A1 , A2 , . . . , An are n events, then

P (A1

n
X
i ,j ,k =1

A2 . . . An ) =
P ( Ai

n
X
i =1

P (Ai )

n
X

i ,j =1

P (Ai

Aj ) +

Aj Ak ) + . . . + (1)n1 P (A1 A2 . . . An )

1.3 - Counting
I

Denition
A selection of k objects out of n objects without replacement and
without order is called combination, denoted Cnk , and dened
by: Cnk = k !(nn! k )! (Binomial coecient)

1.3 - Counting
I

Denition
A selection of k objects out of n objects without replacement and
without order is called combination, denoted Cnk , and dened
by: Cnk = k !(nn! k )! (Binomial coecient)
Denition
A selection of k objects out of n objects without replacement and
with order is called permutation, denoted k Pn , and dened
by: k Pn = (nn!k )!

1.3 - Counting
I

Denition
A selection of k objects out of n objects without replacement and
without order is called combination, denoted Cnk , and dened
by: Cnk = k !(nn! k )! (Binomial coecient)
Denition
A selection of k objects out of n objects without replacement and
with order is called permutation, denoted k Pn , and dened
by: k Pn = (nn!k )!
a) In how many dierent ways can a group of 2 persons be
formed if 5 persons are available ?

1.3 - Counting
I

Denition
A selection of k objects out of n objects without replacement and
without order is called combination, denoted Cnk , and dened
by: Cnk = k !(nn! k )! (Binomial coecient)
Denition
A selection of k objects out of n objects without replacement and
with order is called permutation, denoted k Pn , and dened
by: k Pn = (nn!k )!
a) In how many dierent ways can a group of 2 persons be
formed if 5 persons are available ?
C52

1.3 - Counting
I

Denition
A selection of k objects out of n objects without replacement and
without order is called combination, denoted Cnk , and dened
by: Cnk = k !(nn! k )! (Binomial coecient)
Denition
A selection of k objects out of n objects without replacement and
with order is called permutation, denoted k Pn , and dened
by: k Pn = (nn!k )!
a) In how many dierent ways can a group of 2 persons be
formed if 5 persons are available ?
C52

b) In how many dierent ways can a this be done if the group

must contain a president and a vice-president ?

1.3 - Counting
I

Denition
A selection of k objects out of n objects without replacement and
without order is called combination, denoted Cnk , and dened
by: Cnk = k !(nn! k )! (Binomial coecient)
Denition
A selection of k objects out of n objects without replacement and
with order is called permutation, denoted k Pn , and dened
by: k Pn = (nn!k )!
a) In how many dierent ways can a group of 2 persons be
formed if 5 persons are available ?
C52

b) In how many dierent ways can a this be done if the group

must contain a president and a vice-president ?
2 P5

Property
The number of permutations of n distinct objects is n!

Property
The number of permutations of n distinct objects is n!
Property

If an experiment has n1 outcomes, and if

another experiment has n2 outcomes, then the total number of
outcomes is n1 n2 .
(Basic counting rule)

Property
The number of permutations of n distinct objects is n!
Property

If an experiment has n1 outcomes, and if

another experiment has n2 outcomes, then the total number of
outcomes is n1 n2 .
a) Example: In how many dierent ways can a team of 2 men
and 3 women be formed if 5 men and 7 women are available ?
(Basic counting rule)

Property
The number of permutations of n distinct objects is n!
Property

If an experiment has n1 outcomes, and if

another experiment has n2 outcomes, then the total number of
outcomes is n1 n2 .
a) Example: In how many dierent ways can a team of 2 men
and 3 women be formed if 5 men and 7 women are available ?
(Basic counting rule)

C52

C73

Property
The number of permutations of n distinct objects is n!
Property

If an experiment has n1 outcomes, and if

another experiment has n2 outcomes, then the total number of
outcomes is n1 n2 .
a) Example: In how many dierent ways can a team of 2 men
and 3 women be formed if 5 men and 7 women are available ?
(Basic counting rule)

C52

C73

b) In how many ways can this be done if one man and one
woman refuse to be together in the team ?

Property
The number of permutations of n distinct objects is n!
Property

If an experiment has n1 outcomes, and if

another experiment has n2 outcomes, then the total number of
outcomes is n1 n2 .
a) Example: In how many dierent ways can a team of 2 men
and 3 women be formed if 5 men and 7 women are available ?
(Basic counting rule)

C52

C73

b) In how many ways can this be done if one man and one
woman refuse to be together in the team ?
C52

C73 C41 C62

Permutations of indistinguishable objects

I

Property
The number of permutations of n objects, from which n1 are the
same, n2 are the same, ..., and nk are the same
(n1 + n2 + . . . + nk = n) is
n!
n1 ! n2 ! . . . nk !

(Multinomial coecient)

Permutations of indistinguishable objects

I

Property
The number of permutations of n objects, from which n1 are the
same, n2 are the same, ..., and nk are the same
(n1 + n2 + . . . + nk = n) is
n!
n1 ! n2 ! . . . nk !

(Multinomial coecient)

Proof.
There are Cnn1 choices for the rst n1 objects, and Cnn2 n1 choices
for the n2 objects, ..., and Cnnn1 n2 ...n 1 choices for the nk
objects. By the basic counting rule, the total number of choices is
k

Cnn1

Cnn2 n1 . . . Cnnn1 n2 ...n

k

n!
n1 ! n2 ! . . . nk !

Permutations of indistinguishable objects

I

Property
The number of permutations of n objects, from which n1 are the
same, n2 are the same, ..., and nk are the same
(n1 + n2 + . . . + nk = n) is
n!
n1 ! n2 ! . . . nk !

(Multinomial coecient)

Proof.
There are Cnn1 choices for the rst n1 objects, and Cnn2 n1 choices
for the n2 objects, ..., and Cnnn1 n2 ...n 1 choices for the nk
objects. By the basic counting rule, the total number of choices is
k

Cnn1

Cnn2 n1 . . . Cnnn1 n2 ...n

k

n!
n1 ! n2 ! . . . nk !

- Example: How many dierent words can be formed by using

the letters of the word MISSISSIPPI ?

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?
8! 3!

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?
8! 3!
b) Find the probability of a full-house in a poker hand.

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?
8! 3!
b) Find the probability of a full-house in a poker hand.
13 12 C43 C42

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?
8! 3!
b) Find the probability of a full-house in a poker hand.
13 12 C43 C42
c) In how many dierent ways can 15 students be seated in 3
classes of size 5 each ?

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?
8! 3!
b) Find the probability of a full-house in a poker hand.
13 12 C43 C42
c) In how many dierent ways can 15 students be seated in 3
classes of size 5 each ?
5
C15

5 C5 =
C10
5

15!
5!5!5!

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?
8! 3!
b) Find the probability of a full-house in a poker hand.
13 12 C43 C42
c) In how many dierent ways can 15 students be seated in 3
classes of size 5 each ?
5
C15

5 C5 =
C10
5

15!
5!5!5!

d) In how many ways can this be done if two of the students,

Asma and Maya, want to be in the same class room ?

Exercises

a) In how many dierent ways can a group of 10 persons be

seated in a row if 3 of them insist to be next to each other ?
8! 3!
b) Find the probability of a full-house in a poker hand.
13 12 C43 C42
c) In how many dierent ways can 15 students be seated in 3
classes of size 5 each ?
5
C15

5 C5 =
C10
5

15!
5!5!5!

d) In how many ways can this be done if two of the students,

Asma and Maya, want to be in the same class room ?
!
3 3!13
5!5!

1.4 - Conditional Probability

I

Denition
Let A and B be two events with P (B ) > 0. The conditional
probability of A given B , denoted by P (A|B ), is dened by
P (A|B )

B)
= PP(A(
B)

1.4 - Conditional Probability

I

Denition
Let A and B be two events with P (B ) > 0. The conditional
probability of A given B , denoted by P (A|B ), is dened by
P (A|B )

B)
= PP(A(
B)

Example
5 cards are drawn at random and without replacement from a deck
of 52 playing cards. Find the probability that all 5 cards are spades
if at least 3 cards are spades.

1.4 - Conditional Probability

I

Denition
Let A and B be two events with P (B ) > 0. The conditional
probability of A given B , denoted by P (A|B ), is dened by
P (A|B )

B)
= PP(A(
B)

Example
5 cards are drawn at random and without replacement from a deck
of 52 playing cards. Find the probability that all 5 cards are spades
if at least 3 cards are spades.
- Let A={event all 5 cards are spades}, and B ={event at least
3 cards are spades}

1.4 - Conditional Probability

I

Denition
Let A and B be two events with P (B ) > 0. The conditional
probability of A given B , denoted by P (A|B ), is dened by
P (A|B )

B)
= PP(A(
B)

Example
5 cards are drawn at random and without replacement from a deck
of 52 playing cards. Find the probability that all 5 cards are spades
if at least 3 cards are spades.
- Let A={event all 5 cards are spades}, and B ={event at least
3 cards are spades}
P (A|B )

P (A B )
P (B )

P ( A)
P (B )

5
C13
3 C2 + C4
C13
39
13

1 + C5
C39
13

Proposition
The conditional probability P (|B ) is itself a probability.

Proposition
The conditional probability P (|B ) is itself a probability.
Proof:
I

P (|B )

P ( B )
P (B )

P ()

P (B )

=0

Proposition
The conditional probability P (|B ) is itself a probability.
Proof:
I

P (|B )

P (S |B )

P ( B )

P (B )
P (S B )
P (B )

=
=

P ()

P (B )
P (B )
P (B )

=0
=1

Proposition
The conditional probability P (|B ) is itself a probability.
Proof:
P ( B )

P ()

=0

P (|B )

P (S |B )

Let A1 , A2 , A3 , . . . be mutually exclusive events,

P (B )
P (S B )
P (B )

=
=

P (B )
P (B )
P (B )

=1

A2 A3 . . .) B )
P (B )
P ((A1 B ) (A2 B ) (A3 B ) . . .)
=
P (B )
P (A1 B )
P (A2 B )
P (A3 B )
+
+
+ ...
=
P (B )
P (B )
P (B )
= P (A1 |B ) + P (A2 |B ) + P (A3 |B ) + . . .
P (A1

A2 A3 . . . |B ) =

P ((A1

The previous proposition says that all rules of probability still

apply to conditional probability;

The previous proposition says that all rules of probability still

apply to conditional probability;
for example,
- P (A|C ) = 1 P (A|C )

The previous proposition says that all rules of probability still

apply to conditional probability;
for example,
- P (A|C ) = 1 P (A|C )

- P (A B |C ) = P (A|C ) + P (B |C ) P (A B |C )

The previous proposition says that all rules of probability still

apply to conditional probability;
for example,
- P (A|C ) = 1 P (A|C )

- P (A B |C ) = P (A|C ) + P (B |C ) P (A B |C )
- P (D |A) = 1 P (D |A) and not 1 P (D |A) !!!

Property
(Multiplication rule)
I

P (A B )

= P (A|B ) P (B ) = P (B |A) P (A)

Property
(Multiplication rule)
I

P (A B )

P (A1
P (An

= P (A|B ) P (B ) = P (B |A) P (A)

. . . A ) = P (A1 ) P (A2 |A1 ) P (A3 |A1 A2 ) . . .
|A1 A2 . . . A 1 )
n

Property
(Multiplication rule)
I

P (A B )

P (A1
P (An

= P (A|B ) P (B ) = P (B |A) P (A)

. . . A ) = P (A1 ) P (A2 |A1 ) P (A3 |A1 A2 ) . . .
|A1 A2 . . . A 1 )
n

- Example: Cards are drawn at random and without

replacement from a deck of 52 cards. Find the probability that
the sixth spade is drawn at the tenth draw.

Property
(Multiplication rule)
I

P (A B )

P (A1
P (An

= P (A|B ) P (B ) = P (B |A) P (A)

. . . A ) = P (A1 ) P (A2 |A1 ) P (A3 |A1 A2 ) . . .
|A1 A2 . . . A 1 )
n

- Example: Cards are drawn at random and without

replacement from a deck of 52 cards. Find the probability that
the sixth spade is drawn at the tenth draw.
Let A={event ve spades in the rst nine cards}, and
B ={tenth card is spade}

Property
(Multiplication rule)
I

P (A B )

P (A1
P (An

= P (A|B ) P (B ) = P (B |A) P (A)

. . . A ) = P (A1 ) P (A2 |A1 ) P (A3 |A1 A2 ) . . .
|A1 A2 . . . A 1 )
n

- Example: Cards are drawn at random and without

replacement from a deck of 52 cards. Find the probability that
the sixth spade is drawn at the tenth draw.
Let A={event ve spades in the rst nine cards}, and
B ={tenth card is spade}
P (A B )

= P (A) P (B |A)

Property
(Multiplication rule)
I

P (A B )

P (A1
P (An

= P (A|B ) P (B ) = P (B |A) P (A)

. . . A ) = P (A1 ) P (A2 |A1 ) P (A3 |A1 A2 ) . . .
|A1 A2 . . . A 1 )
n

- Example: Cards are drawn at random and without

replacement from a deck of 52 cards. Find the probability that
the sixth spade is drawn at the tenth draw.
Let A={event ve spades in the rst nine cards}, and
B ={tenth card is spade}
P (A B )

= P (A) P (B |A)
C5 C4
8
= 13 9 39
C52

43

1.5 - Independence
I

Example
A fair coin is tossed twice. Let
A ={rst toss is head}, B ={second toss is head}

1.5 - Independence
I

Example
A fair coin is tossed twice. Let
A ={rst toss is head}, B ={second toss is head}

- S = {HH , HT , TH , TT }, A = {HH , HT }, and

B

= {TH , HH }

1.5 - Independence
I

Example
A fair coin is tossed twice. Let
A ={rst toss is head}, B ={second toss is head}

- S = {HH , HT , TH , TT }, A = {HH , HT }, and

B

= {TH , HH }

- P (B ) = 1/2

1.5 - Independence
I

Example
A fair coin is tossed twice. Let
A ={rst toss is head}, B ={second toss is head}

- S = {HH , HT , TH , TT }, A = {HH , HT }, and

B

= {TH , HH }

- P (B ) = 1/2
- P (B |A) = P P(B(A)A) =

1/4
2/4

= 1/2 = P (B )

1.5 - Independence
I

Example
A fair coin is tossed twice. Let
A ={rst toss is head}, B ={second toss is head}

- S = {HH , HT , TH , TT }, A = {HH , HT }, and

B

= {TH , HH }

- P (B ) = 1/2
- P (B |A) = P P(B(A)A) =
I

1/4
2/4

= 1/2 = P (B )

Denition
Two events A and B are independent if and only if P (A|B ) = P (A)
or P (B |A) = P (B ).

Proposition
Two events A and B are independent if and only if
P (A B )

= P ( A) P ( B )

Proposition
Two events A and B are independent if and only if
P (A B )

= P ( A) P ( B )

Proof: P (A B ) = P (B |A) P (A) = P (B ) P (A)

Proposition
Two events A and B are independent if and only if
P (A B )

= P ( A) P ( B )

Proof: P (A B ) = P (B |A) P (A) = P (B ) P (A)

I

Property
If A and B are independent, then so are
b) A and B
a) A and B

c) A

and B

Proposition
Two events A and B are independent if and only if
P (A B )

= P ( A) P ( B )

Proof: P (A B ) = P (B |A) P (A) = P (B ) P (A)

I

Property
If A and B are independent, then so are
b) A and B
c) A and B
a) A and B
Proof: a) P (B |A) = 1 P (B |A) = 1 P (B ) = P (B )

Proposition
If n events A1 , A2 , . . . , An are mutually independent, then
P (A1

A2 . . . An ) = P (A1 ) P (A2 ) . . . P (An )

Proposition
If n events A1 , A2 , . . . , An are mutually independent, then
P (A1

A2 . . . An ) = P (A1 ) P (A2 ) . . . P (An )

Example
Three men A, B and C re on target. The probability of hitting is
0.9, 0.8, and 0.7 respectively. Find the probability that all three
men hit the target.

Proposition
If n events A1 , A2 , . . . , An are mutually independent, then
P (A1

A2 . . . An ) = P (A1 ) P (A2 ) . . . P (An )

Example
Three men A, B and C re on target. The probability of hitting is
0.9, 0.8, and 0.7 respectively. Find the probability that all three
men hit the target.
- Let A ={person A hits the target}, B ={person B hits the
target} and C ={person C hits the target}

Proposition
If n events A1 , A2 , . . . , An are mutually independent, then
P (A1

A2 . . . An ) = P (A1 ) P (A2 ) . . . P (An )

Example
Three men A, B and C re on target. The probability of hitting is
0.9, 0.8, and 0.7 respectively. Find the probability that all three
men hit the target.
- Let A ={person A hits the target}, B ={person B hits the
target} and C ={person C hits the target}
- P (A B C ) = P (A).P (B ).P (C ) = 0.9 0.8 0.7 = 0.504

Proposition
If n events A1 , A2 , . . . , An are mutually independent, then
P (A1

A2 . . . An ) = P (A1 ) P (A2 ) . . . P (An )

Example
Three men A, B and C re on target. The probability of hitting is
0.9, 0.8, and 0.7 respectively. Find the probability that all three
men hit the target.
- Let A ={person A hits the target}, B ={person B hits the
target} and C ={person C hits the target}
- P (A B C ) = P (A).P (B ).P (C ) = 0.9 0.8 0.7 = 0.504
I Find the probability that exactly one of them hit the target.

Proposition
If n events A1 , A2 , . . . , An are mutually independent, then
P (A1

A2 . . . An ) = P (A1 ) P (A2 ) . . . P (An )

Example
Three men A, B and C re on target. The probability of hitting is
0.9, 0.8, and 0.7 respectively. Find the probability that all three
men hit the target.
- Let A ={person A hits the target}, B ={person B hits the
target} and C ={person C hits the target}
- P (A B C ) = P (A).P (B ).P (C ) = 0.9 0.8 0.7 = 0.504
I Find the probability that exactly one of them hit the target.
- P (A B C ) + P (A B C ) + P (A B C ) =
0.9 0.2 0.3 + 0.1 0.8 0.3 + 0.1 + 0.2 0.7 = 0.056

Note that the events A1 , A2 , . . . , An can be ind two by two but

not mutually independent.

Note that the events A1 , A2 , . . . , An can be ind two by two but

not mutually independent.

Example
A fair coin is tossed twice. Let
A1 ={rst toss is head}, A2 ={second toss is head}, A3 ={the two
tosses show up the same face}

Note that the events A1 , A2 , . . . , An can be ind two by two but

not mutually independent.

Example
A fair coin is tossed twice. Let
A1 ={rst toss is head}, A2 ={second toss is head}, A3 ={the two
tosses show up the same face}
= P (A2 ) = P (A3 ) = 21 , and
1
1
P (A1 A2 A3 ) = P (A1 A2 ) = 4 6= 8 =
P (A1 )P (A2 )P (A3 ), hence A1 , A2 , A3 are not mutually
P (A1 )

independent.
But P (A1 A2 ) =

1
4 = P (A1 )P (A2 ),
1
P (A2 A1 ) = 4 = P (A2 )P (A3 ).
1
P (A1 A2 ) = 4 = P (A1 )P (A3 )

A3 ) =
A3 ) =
hence A1 , A2 , A3 are independent two by two.
P ( A2
P (A1

Exercises

e) A pair of dice is rolled. Find the probability that the sum of

two dice is 11 if at least one of the dice show up 5.

Exercises

e) A pair of dice is rolled. Find the probability that the sum of

two dice is 11 if at least one of the dice show up 5.
f) Let A and B be two independent events with P (A) = 1/3
and P (B ) = 1/4. Find P (A|A B )

Exercises

e) A pair of dice is rolled. Find the probability that the sum of

two dice is 11 if at least one of the dice show up 5.
f) Let A and B be two independent events with P (A) = 1/3
and P (B ) = 1/4. Find P (A|A B )
g) Five Americans, four French and three Russians participate
in a chess tournament. In how many dierent ways can they
be classied if only the nationality of the players matters ?

Exercises

e) A pair of dice is rolled. Find the probability that the sum of

two dice is 11 if at least one of the dice show up 5.
f) Let A and B be two independent events with P (A) = 1/3
and P (B ) = 1/4. Find P (A|A B )
g) Five Americans, four French and three Russians participate
in a chess tournament. In how many dierent ways can they
be classied if only the nationality of the players matters ?
h) A, B , and C take turns in ipping a fair coin, A rst, then
B , then C , then A, and so on. The rst one to get a head
wins. Find the probability of winning for player A.

1.6 - Total Probability Formula and Bayes Rule

I

Proposition

Let A1 , A2 , . . . , An be a set of mutually

exclusive and exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n.
Then for any event B ,
(Total probability formula)

P (B )

n
X
i =1

P (Ai )P (B |Ai )

1.6 - Total Probability Formula and Bayes Rule

I

Proposition

Let A1 , A2 , . . . , An be a set of mutually

exclusive and exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n.
Then for any event B ,
(Total probability formula)

P (B )

n
X
i =1

P (Ai )P (B |Ai )

Proof:
- P (B ) = P (B S ) = P (B (A1 A2 . . . An ))

1.6 - Total Probability Formula and Bayes Rule

I

Proposition

Let A1 , A2 , . . . , An be a set of mutually

exclusive and exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n.
Then for any event B ,
(Total probability formula)

P (B )

n
X
i =1

P (Ai )P (B |Ai )

Proof:
- P (B ) = P (B S ) = P (B (A1 A2 . . . An ))
= P ((B A1 ) (B A2 ) . . . (B An ))

1.6 - Total Probability Formula and Bayes Rule

I

Proposition

Let A1 , A2 , . . . , An be a set of mutually

exclusive and exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n.
Then for any event B ,
(Total probability formula)

P (B )

n
X
i =1

P (Ai )P (B |Ai )

Proof:
- P (B ) = P (B S ) = P (B (A1 A2 . . . An ))
= P ((B A1 ) (B A2 ) . . . (B An ))
= P (B A1 ) + P (B A2 ) + . . . + P (B An )

1.6 - Total Probability Formula and Bayes Rule

I

Proposition

Let A1 , A2 , . . . , An be a set of mutually

exclusive and exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n.
Then for any event B ,
(Total probability formula)

P (B )

n
X
i =1

P (Ai )P (B |Ai )

Proof:
- P (B ) = P (B S ) = P (B (A1 A2 . . . An ))
= P ((B A1 ) (B A2 ) . . . (B An ))
= P (B A1 ) + P (B A2 ) + . . . + P (B An )
= P (B |A1 )P (A1 ) + P (B |A2 )P (A2 ) + . . . + P (B |An )P (An )

Proposition

(Bayes rule) Let A1 , A2 , . . . , An be a set of mutually exclusive and

exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n. Let B be any
event with P (B ) > 0, then
P (Ai |B )

P (B |Ai )P (Ai )

= n
X

i =1

P (Ai )P (B |Ai )

Proposition

(Bayes rule) Let A1 , A2 , . . . , An be a set of mutually exclusive and

exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n. Let B be any
event with P (B ) > 0, then
P (Ai |B )

P (B |Ai )P (Ai )

= n
X

i =1

Proof: P (Ai |B ) =

P (Ai

B)

P (B )

P (Ai )P (B |Ai )

Proposition

(Bayes rule) Let A1 , A2 , . . . , An be a set of mutually exclusive and

exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n. Let B be any
event with P (B ) > 0, then
P (Ai |B )

P (B |Ai )P (Ai )

= n
X

i =1

Proof: P (Ai |B ) =

P (Ai

P (Ai )P (B |Ai )

B)

P (B )

= P (B |PA(B)P) (A )
i

Proposition

(Bayes rule) Let A1 , A2 , . . . , An be a set of mutually exclusive and

exhaustive events, with P (Ai ) > 0 for i = 1, 2, . . . , n. Let B be any
event with P (B ) > 0, then
P (Ai |B )

P (B |Ai )P (Ai )

= n
X

i =1

Proof: P (Ai |B ) =

P (Ai

P (Ai )P (B |Ai )

B)

P (B )

= P (B |PA(B)P) (A )
= n P (B |A )P (A )
X
P (B |Ai )P (Ai )
i =1
i

Example
(Urn model)

Three balls are selected at random and without replacement from

an urn U1 containing 5 white balls and 4 blue balls, and put in an
empty urn U2 . A ball is then drawn from U2 , its blue. Find the
probability that all three balls transferred from U1 to U2 were blue.

Example
(Urn model)

Three balls are selected at random and without replacement from

an urn U1 containing 5 white balls and 4 blue balls, and put in an
empty urn U2 . A ball is then drawn from U2 , its blue. Find the
probability that all three balls transferred from U1 to U2 were blue.
Solution:
Let Bi ={event i blue balls selected from U1 } (i = 0, 1, 2, 3),
and B ={event blue ball selected from U2 }

Example
(Urn model)

Three balls are selected at random and without replacement from

an urn U1 containing 5 white balls and 4 blue balls, and put in an
empty urn U2 . A ball is then drawn from U2 , its blue. Find the
probability that all three balls transferred from U1 to U2 were blue.
Solution:
Let Bi ={event i blue balls selected from U1 } (i = 0, 1, 2, 3),
and B ={event blue ball selected from U2 }
P (B3 |B )

=
P (B |B3 )P (B3 )

P (B |B0 )P (B0 ) + P (B |B1 )P (B1 ) + P (B |B2 )P (B2 ) + P (B |B3 )P (B3 )

Example
(Urn model)

Three balls are selected at random and without replacement from

an urn U1 containing 5 white balls and 4 blue balls, and put in an
empty urn U2 . A ball is then drawn from U2 , its blue. Find the
probability that all three balls transferred from U1 to U2 were blue.
Solution:
Let Bi ={event i blue balls selected from U1 } (i = 0, 1, 2, 3),
and B ={event blue ball selected from U2 }
P (B3 |B )

=
P (B |B3 )P (B3 )

P (B |B0 )P (B0 ) + P (B |B1 )P (B1 ) + P (B |B2 )P (B2 ) + P (B |B3 )P (B3 )

1
3

1 C43

0 C53 + C41 C52 + 23 C42 C51 + 1 C43

3
28

Problems for solving sessions

I

section 1.2 : 8

section 1.3 : 3, 5, 11, 12, 17

section 1.4 : 1, 4, 10, 11, 13, 16, 19

section 1.5 : 5, 8, 15, 17, 19

section 1.6 : 9, 13

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:
a) The letters can be repeated?

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:
a) The letters can be repeated?
b) The letters cannot be repeated?

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:
a) The letters can be repeated?
b) The letters cannot be repeated?

Exercise 1.3-11: The world series in baseball continues until

either American league or the National league wins four games.
How many dierent orders are possible if the series goes

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:
a) The letters can be repeated?
b) The letters cannot be repeated?

Exercise 1.3-11: The world series in baseball continues until

either American league or the National league wins four games.
How many dierent orders are possible if the series goes
a) Four games?

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:
a) The letters can be repeated?
b) The letters cannot be repeated?

Exercise 1.3-11: The world series in baseball continues until

either American league or the National league wins four games.
How many dierent orders are possible if the series goes
a) Four games?
b) Five games?

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:
a) The letters can be repeated?
b) The letters cannot be repeated?

Exercise 1.3-11: The world series in baseball continues until

either American league or the National league wins four games.
How many dierent orders are possible if the series goes
a) Four games?
b) Five games?
c) Six games?

Exercises - section 1.3

I

Exercise 1.3-2: How many dierent license plates are possible

if a state uses
a) Two letters followed by a four digit integer?
b) Three letters followed by a three digit integer?

Exercise 1.3-5: How many four-letter code can be formed by

the letters of the word HOPE if:
a) The letters can be repeated?
b) The letters cannot be repeated?

Exercise 1.3-11: The world series in baseball continues until

either American league or the National league wins four games.
How many dierent orders are possible if the series goes
a) Four games?
b) Five games?
c) Six games?
d) Seven games?