Chapter 1: Introduction to Probability

Phan Thi Khanh Van

E-mail: khanhvanphan@hcmut.edu.vn

April 5, 2020

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 1 / 37

Contents

1 Set Theory

2 Counting techniques

3 Sample space, outcomes, events, Probability

4 The total probability formula and Bayes formula

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 2 / 37

Introduction to Set theory
Empty set: ∅
Subset: A ⊂ B
Proper subset: A B
Disjoint: A ∩ B = ∅
Venn diagram: A pictorial representation of the relationships
between sets.
Cardinality |A|: The number of elements of A.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 3 / 37

Set Operations

Union (A union B): A + B or A ∪ B

A + B = {x|x ∈ A or x ∈ B}

Intersection (A intersected B): AB or A ∩ B

AB = {x|x ∈ A and x ∈ B}

1 {1, 3, 4, 7} ∩ {2, 3, 5} = {3}.

2 {1, 3, 4, 7} ∪ {2, 3, 5} = {1, 2, 3, 4, 5, 7}.
3 [1, 4) ∩ [2, 7] = [2, 4).
4 [1, 4) ∪ [2, 7] = [1, 7].
(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 4 / 37
Complement A or Ac or A0
A = Ω\A
A = {x|x ∈ Ω and x ∈
/ A}
A=A

Example
1 Ω = {1, 2, 3, 4, 5, 6}, A = {1, 3, 5}
⇒ Ac = {2, 4, 6}.
2 Ω = R, A = [2, 4]
⇒ Ac = (−∞, 2) ∪ (4, ∞).

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 5 / 37

Difference (B without A ): B\A
B\A = {x|x ∈ B and x ∈
/ A}

Symmetric difference: A M B
A M B = {x|x ∈ A\B or x ∈ B\A}

Example
1 {1, 3, 4, 7}\{2, 3, 5} = {1, 4, 7}.
2 {1, 3, 4, 7} M {2, 3, 5} = {1, 2, 4, 5, 7}.
3 [1, 4)\[2, 7] = [1, 2).
4 [1, 4) M [2, 7] = [1, 2) ∪ [4, 7].

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 6 / 37

Distributive law
1 A(B + C ) = AB + AC
2 A + (BC ) = (A + B)(A + C )

De Morgan’s Rules
1 A + B = A B.
2 AB = A + B

Difference laws
1 A\(B + C ) = (A\B)(A\C )
2 A\(BC ) = (A\B) + (A\C )

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 7 / 37

Add exclusive options

Example
Choose among a group of 3 boys and 4 girls to be one member of your
team, you have 7 options.

Add exclusive options

1 A, B: disjoint sets, |A| = m, |B| = n. Choose 1 item from A and B:
m + n outcomes.
2 A1 , A2 , ..., An : disjoint sets, |Ai | = mi , ⇒ m1 + m2 + ... + mn
outcomes.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 8 / 37

Multiply successive choices

Example 1
Find the number of possible outfits to wear do 3 shirts, 2 pairs of pants:
3 × 2 = 6.

Example 2
Find the number of possible outcomes do 3 coin flips:
2 × 2 × 2 = 23 .

Multiply successive choices

1 First choose 1 item from A, then 1 from B: |A, B| = |A|.|B|
2 |A1 , A2 , ..., An | = |A1 |.|A2 |....|An |.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 9 / 37

Shepard’s Law

Example
1 Find the number of possible committees of three people that can be
formed from a group of ten people
10.9.8
3.2.1 = 120.
2 Find the number of ways of choosing two disjoint committees of three
people from a group of ten people. 10.9.8 7.6.5
3.2.1 3.2.1 = 4200.

Shepard’s Law
] of sheep = ] of legs /4: over-counting by a uniform multiplicative factor
can be corrected by dividing by this factor.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 10 / 37

Mississippi Rule

Examples
11!
1 distinct permutations of MISSISSIPPI. 4!4!2! = 34650.
2 distinct stacks of 6 red, green, and blue blocks with two of each color.
6!
2!2!2! = 90.

Mississippi Rule:
] of distinct permutations of a string of letters (repeat is allowed) = ] of
permutations as if the letters were distinct /] of equivalent permutations
that can be made by permuting identical letters.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 11 / 37

Choose k out of n
1 Order counts, with replacement
nk .
2 Order counts, without replacement
n!
Akn = (n−k)! .
n!
3 Order does not count, without replacement Cnk = k!(n−k)! .

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 12 / 37

Example
A printed circuit board has 8 different locations in which 1 component can
be placed. If 4 different components are to be placed on the board, how
many designs are possible?
8!
A48 = (8−4)! .

Example
In a hospital, a operating room needs to schedule 3 knee surgeries (k) and
2 hip surgeries (h) in a day. How many sequences are there?
5!
C52 = 2!3! = 10
Example
A bin of 50 parts contains 3 defectives and 47 non-defective parts. A
sample of 6 parts is selected without replacement. How many samples of
size 6 contain 2 defective parts?
3! 47!
C32 .C47
4 =
2!.1! . 4!(47−4)!
(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 13 / 37
Sample space, outcomes, events
Consider an experiment (or trial):
One possible result is called an outcome.
The sample space Ω of an experiment is the set of all possible
outcomes.
A subset of Ω is called an event.

Probability P ∈ [0, 1]
is the chance that a particular outcome (event) will occur.

Probability with equally likely outcomes

|A|
If |Ω| is finite, and all outcomes are equally likely, then P(A) = |Ω|

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 14 / 37

Sample space, events, Probability

Rolling a die: An experiment.

Ω = {1, 2, 3, 4, 5, 6} : Sample space, where
P(1) = P(2) = ... = P(6) = 16 : equally likely outcomes.
Some events: A = {1}, P(A) = 61 .
B = {1, 3, 5} : odd number, P(B) = 36 = 12 .
C = {3, 6} : number devisible by 3, P(C ) = 13 .
D = {2, 4}, P(D) = 13 .
C + B = {1, 3, 5, 6}.
BC = {3}.
A = {2, 3, 4, 5, 6}, P(A) = 1 − P(A) = 56 .
AC = ∅ ⇒ A and C : mutually exclusive.
B + C + D = Ω ⇒ B, C , D are collectively exhaustive.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 15 / 37

Example of recycle time of a flash
Randomly select a camera and record the recycle time of a flash
⇒ Ω = R+ = {x|x > 0} - continuous.
Suppose it is known that all recycle times are between 1.5 and 5
seconds. ⇒ Ω = {x|1.5 < x < 5} - continuous.
It is known that the recycle time has only 3 values(low, medium or
high). ⇒ Ω = {low , medium, high} - discrete.
Does the camera conform to minimum recycle time specifications?
⇒ Ω = {yes, no} - discrete.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 16 / 37

Event combinations
The Union of two events A, B: A + B.
The Intersection of two events A, B: AB.
The Complement of an event A: A, Ac , A0 .

2 events A, B The event AB The event The event AC

(A + B)C

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 17 / 37

Mutually Exclusive Events
Two events, denoted as A and B, that are disjoint: AB = ∅, are said to
be mutually exclusive.

Exhaustive Events
A set of events E1 , E2 ...Ek is called collectively exhaustive if there union is
Ω.

An example of mutually exclusive and exhaustive events:

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 18 / 37

Example
Suppose there are 9 socks loose in a drawer in a dark room which are
identical except 6 are orange and 3 are blue. Someone selects 2 at
random, all possibilities being equally likely. What is the probability the 2
socks are the same color? Also, if instead, 3 socks were selected, what is
the probability that at least 2 of them are the same color?

The number of possible outcomes: |Ω| = C92 .

A1 : 2 socks are orange
A2 : 2 socks are blue
A : 2 socks have the same color = A1 + A2
Because A1 and A2 are exclusive:
|A1 | + |A2 | C2 + C2
P(A) = P(A1 ) + P(A2 ) = = 6 2 3 = 0.5
|Ω| C9

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 19 / 37

Example
If 3 fair coins are tossed, what is the probability that 3 faces will not be
the same?

A: 3 faces are not the same, A = {HHT , HTH, THH...}

A: 3 faces are the same, A = {HHH, TTT }
|A| 2 1 3
P(A) = 1 − P(A) = 1 − =1− 3 =1− =
|Ω| 2 4 4

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 20 / 37

Modern Probability - Three Axioms
Denote by A the collection of all events. Probability is a function

P : A → [0, 1]

such that
1 0 ≤ P(A) ≤ 1, ∀A ∈ A.
2 P(Ω) = 1.
3 For 2 disjoint events A, B:
P(A + B) = P(A) + P(B).
For any event A ∈ A, P(A) is called the probability of A

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 21 / 37

Properties of Probability
1 0 ≤ P(A) ≤ 1
2 P(Ω) = 1 and P(∅) = 0.
3 P(A) = 1 − P(A)
4 For two disjoint events A, B
P(A + B) = P(A) + P(B)
For A1 , · · · , An : disjoint events,
P(A1 + · · · + An ) = P(A1 ) + · · · + P(An )

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 22 / 37

 P(A + B) = P(A \ B) + P(B).
P(A \ B) + P(AB) = P(A).

Addition Rule
For 2 events
P(A + B) = P(A) + P(B) − P(AB)
For 3 events

P(A + B + C )
= P(A) + P(B) + P(C ) − P(AB) − P(AC ) − P(BC ) + P(ABC )

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 23 / 37

Example
A patient arrives at a doctor’s office with a sore throat and low-grade
fever. After an exam, the doctor decides that the patient has either a
bacterial infection or a viral infection or both. The doctor decides that
there is a probability of 0.7 that the patient has a bacterial infection and a
probability of 0.4 that the person has a viral infection. What is the
probability that the patient has both infections?

A: the patient has a bacterial infection, P(A) = 0.7.

B: the patient has a viral infection, P(B) = 0.4.
Because the patient has at least one infection: P(A + B) = 1.
The probability that the patient has both infections:
P(AB) = P(A) + P(B) − P(A + B) = 0.7 + 0.4 − 1 = 0.1.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 24 / 37

Conditional Probability

The conditional probability of an event B given an event A, denoted by

P(B|A) is
P(BA)
P(B|A) = , for P(A) > 0.
P(A)

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 25 / 37

Example
A ordinary six-sided die has been rolled once, find the probability that a 2
is rolled, given that the number is even.

B = {2, 4, 6}:even number.

1
A = {2}, P(A) = = P(AB).
6
1
P(A|B) = .
3
Example
Rolling a die once, we consider A = {2, 3, 4, 6}, B = {2, 5, 6}. Find
P(A|B).
1 1
AB = {2, 6}, P(AB) = , P(B) = .
3 2
P(AB) 1/3 2
P(A|B) = = = .
P(B) 1/2 3

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 26 / 37

Example
Given that at least one from 2 children is a girl. Find the probability that
both are girls.

Ω = {BB, BG , GB, GG }
1
A: Both are girls, P(A) = .
4
3
B: At least one child is a girl, P(B) = .
4
1
AB = A ⇒ P(AB) = P(A) = .
4
P(AB) 1/4 1
P(A|B) = = = .
P(B) 3/4 3

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 27 / 37

Example
Given that the older from 2 children is a girl. Find the probability that
both are girls.

Ω = {BB, BG , GB, GG }
A: Both are girls, P(A) = 14 .
1
B = {GG , GB}: The older child is a girl, P(B) = .
2
1
AB = A ⇒ P(AB) = P(A) = .
4
P(AB) 1/4 1
P(A|B) = = = .
P(B) 1/2 2

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 28 / 37

Multiplication rule
P(AB) = P(B|A)P(A) = P(A|B)P(B)

Example
The probability that a part made in the 1st stage of a machining operation
meets specifications is 0.90. The probability that it meets specifications in
the 2nd stage, given that met specifications in the first stage is 0.95.What
is the probability that both stages meet specifications?

P(both) = P(2nd|1st).P(1st) = 0.95 × 0.9 = 0.855

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 29 / 37

Example
Take one card from an ordinary deck of 52 playing cards.
a Compute the probability that the card is an Ace.
b Given that the card is red. Find the probability that the card is an
Ace.
4 1
a A: the card is an Ace, P(A) = = .
52 13
b B: the card is red.
2 1
P(A|B) = = = P(A).
26 13
A and B are independent events.

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 30 / 37

Independent events
A is independent of B (knowing B occurred doesn’t impact whether A
occurred) if one of the following equivalent statements is true
1 P(A|B) = P(A)
2 P(B|A) = P(B)
3 P(AB) = P(A)P(B)
The events A1 , · · · , Ak are independent (or mutually independent) if, for
every subset Ai1 , · · · , Aij of j of these events (j = 2, 3, · · · , k),

P(Ai1 Ai2 · · · Aij ) = P(Ai1 ) · · · P(Aij )

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 31 / 37

We have: P(B) = P(BA) + P(BAC )

Total probability rule (Two events)

P(B) = P(B|A)P(A) + P(B|Ac )P(Ac )

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 32 / 37

Semiconductor Contamination
Information about product failure based on chip manufacturing process
contamination is given below. Let F and H denote the events that the
product fails and the chip is exposed to high contamination, resp. Find the
probability of failure.

Using the multiplication rule:

P(FH) = P(F |H).P(H) = 0.02
P(FH c ) = P(F |H c ).P(H c ) = 0.004
Then applying the total probability formula:
P(F ) = P(FH) + P(FH c ) = 0.02 + 0.004 = 0.024
(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 33 / 37
We have: P(B) = P(BE1 ) + P(BE2 ) + P(BE3 ) + P(BE4 )

Theorem
[Total probability rule - Multiple events]
k
P k
P
P(B) = P(BEi ) = P(B|Ei )P(Ei ),
i=1 i=1

k
S
where Ei ∩ Ej = ∅ (disjoint), for i 6= j and Ω = Ei (exhaustive).
i=1

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 34 / 37

Bayes formula (simple version)
From the Multiplication rule:

P(AB) = P(A|B)P(B) = P(B|A)P(A),

we have
P(B|A)P(A)
P(A|B) = P(B) , for P(B) > 0: Bayes formula

Bayes formula
If E1 , E2 , ..., Ek are k mutually exclusive and exhaustive events, then from
the total probability formula:

P(B) = P(B|E1 )P(E1 ) + P(B|E2 )P(E2 ) + · · · + P(B|Ek )P(Ek ),

we have
P(B|E1 )P(E1 ) P(B|E1 )P(E1 )
P(E1 |B) = P(B) = P(B|E1 )P(E1 )+P(B|E2 )P(E2 )+···+P(B|Ek )P(Ek ) ,

for P(B) > 0: Bayes formula

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 35 / 37
Example
Given 2 identical bottles. In the 1st one, there are 4 blue candies, 2 red
ones. In the 2nd bottle, there are 3 blue candies and 3 red ones. Suppose
that we randomly choose a bottle and draw one candy from it, and given
that a blue candy has been drawn. Find the probability that this candy is
from the 1st bottle.

B1 : The event that the candy is from the 1st bottle, B1 : The event that
1
the candy is from the 2nd bottle. We have: P(B1 ) = P(B1c ) = .
2
A: The event that the candy is blue.
The probability that the candy is blue, given that it is from 1st bottle:
2
P(A|B1 ) =
3
The probability that this candy is from the 1st bottle.
2 1 3 1 7
P(A) = P(A|B1 ).P(B1 ) + P(A|B1c ).P(B1c ) = . + . = .
3 2 6 2 12
P(A|B1 )P(B1 ) 2/3.1/2 4
⇒ P(B1 |A) = = = .
P(A) 7/12 7
(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 36 / 37
Thank you for your attention!

(Phan Thi Khanh Van) Chap 1: Introduction to Probability April 5, 2020 37 / 37