Review of Basic

Probability Theory

Probability and Counting

Rules
Chapter Objectives
1. Determine sample spaces and find the
probability of an event, using classical
probability or empirical probability.
2. Find the probability of compound events,
using the addition rules.
3. Find the probability of compound events,
using the multiplication rules.
4. Find the conditional probability of an event.

2
Chapter Objectives
5. Find total number of outcomes in a sequence of
events, using the fundamental counting rule.
6. Find the number of ways that r objects can be
selected from n objects, using the permutation
rule.
7. Find the number of ways for r objects selected
from n objects without regard to order, using the
combination rule.
8. Find the probability of an event, using the
counting rules.
3
Probability
Probability can be defined as the chance of an
event occurring. E.g. The chance of financial
crisis to happen next year.

Itcan be used to quantify what the “odds” are

that a specific event will occur.

Some examples of how probability is used

everyday would be weather forecasting, “75%
chance of snow” or for setting insurance rates.

4
4-1 Sample Spaces and Probability
 Typically, the starting point of an investigation is
an experiment.
 This may involve either a simple rolling of a pair
of dice or as complicated as conducting a large-
scale survey of households or firms.
 Experiment: An experiment is defined as a
process of observation or measurement.
 Examples:
1. Tossing a coin
2. Rolling a die
3. Counting defective products of a company etc. 5
 An experiment is said to be a Random
experiment if the following three conditions are
satisfied:
1. all possible distinct outcomes are known ahead
of time
2. the outcome of a particular trial is not known a
priori
3. the experiment can be duplicated, in principle,
under ideal conditions.
 A probability experiment is a chance process that
leads to well-defined results called outcomes.
 An outcome is the result of a single trial of a
probability experiment.
 A sample space is the set of all possible outcomes
of a probability experiment.
It is usually denoted by the letter S
 An event consists of outcomes.
Example: 1. in tossing two coins at the same time, event A
contains at least one head.
A  HH , HT , TH 
Sample Spaces

Experiment Sample Space

Toss a coin Head, Tail
Roll a die 1, 2, 3, 4, 5, 6
Answer a true/false True, False
question
Toss two coins HH, HT, TH, TT

8
Example 4-1: Rolling Dice
Find the sample space for rolling two dice.
Example 4-3: Gender of Children
Find the sample space for the gender of the
children if a family has three children. Use B for
boy and G for girl.

BBB BBG BGB BGG GBB GBG GGB GGG

Example 4-4: Gender of Children
Use a tree diagram to find the sample space for
the gender of three children in a family.
B BBB
B
G BBG
B
B BGB
G
G BGG
B GBB
B
G GBG
G
B GGB
G
G GGG
 Example: when a pair of dice is rolled, “total score of 7”
is an event represented by the sample points in A as
follows: A = {(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)},

 where the first number is the outcome of the first die and
the second number is the outcome of the second die.
 Null (impossible) event: the null or impossible event is
represented by . It refers to the event that never occurs
at all.
Sample Spaces and Probability
There are three basic interpretations of
probability:

Classical probability

Empirical probability

Subjective probability

13
Sample Spaces and Probability
Classical probability If a random experiment
can result in N collectively exhaustive, mutually exclusive
and equally likely outcomes and if “n” of these outcomes
result in the occurrence of event E (or favourable
outcomes or success), then the probability of event E
is defined by

nE  # of desired outcomes

P E   
n  S  Total # of possible outcomes
That is, the probability of an event is given by the number of favourable
outcomes divided by the total number of possible outcomes.

14
 Example1: Toss a fair coin twice, then the
sample space S = {HH, HT, TH, TT} ; N=4
Let the event A be “observing at least one
head”, then A = {HH, HT, TH}; n = 3
3
 P ( A) 
4
Example 4-6: Gender of Children
If a family has three children, find the probability
that two of the three children are girls.

Sample Space:
BBB BBG BGB BGG GBB GBG GGB GGG

Three outcomes (BGG, GBG, GGB) have two

girls.

The probability of having two of three children

being girls is 3/8.
Exercise 4-13c: Rolling Dice
If two dice are rolled one time, find the probability
of getting a sum of 7 or 11.

62 2
P sum of 7 or 11  
36 9
Limitations
 The main limitations of this definition of
probability are:
1. it is not applicable to situations where
there is infinite number of possible
outcomes.
2. All outcomes are not mutually exclusive
and equally likely
Sample Spaces and Probability
The complement of an event E ,
denoted by E , is the set of outcomes
in the sample space that are not
included in the outcomes of event E.
P E  = 1- P E 

19
Example 4-10: Finding Complements
Find the complement of each event.

Event Complement of the Event

Rolling a die and getting a 4 Getting a 1, 2, 3, 5, or 6

Selecting a letter of the alphabet Getting a consonant (assume y is a

and getting a vowel consonant)

Selecting a month and getting a Getting February, March, April, May,

month that begins with a J August, September, October,
November, or December

Selecting a day of the week and Getting Saturday or Sunday

getting a weekday
Example 4-11: Residence of People
If the probability that a person lives in an
industrialized country of the world is 1/5, find the
probability that a person does not live in an
industrialized country.
P Not living in industrialized country 
= 1  P living in industrialized country 
1 4
 1 
5 5
Sample Spaces and Probability
There are three basic interpretations of
probability:

Classical probability

Empirical probability

Subjective probability

22
 Sample Spaces and Probability
Empirical probability relies on actual experience to determine the likelihood of outcomes.

The basic argument of the frequency approach is that probability does not have to be restricted to the situations of apparent symmetry
(equally likely) since the notion of probability should be interpreted as stemming from the observable stability of empirical frequencies.

f frequency of desired class

P E   
n Sum of all frequencies

23
 Let NA be the number of times the same event (say
event A) occurs in large number of N trials of an
experiment.
 If there exists a real number p such that
NA
p  lim
N  N
, then p is called the probability of event A and is
denoted by P(A).
NB: The probability of an event, according to this definition,
is the proportion of that event of the same kind will occur
in the long run.
Example 4-13: Blood Types
In a sample of 50 people, 21 had type O blood, 22
had type A blood, 5 had type B blood, and 2 had
type AB blood. Set up a frequency distribution and
find the following probabilities.
a. A person has type O blood.
Type Frequency
f
A 22 P O  
B 5 n
AB 2 21

O 21 50
Total 50
Example 4-13: Blood Types
In a sample of 50 people, 21 had type O blood, 22
had type A blood, 5 had type B blood, and 2 had
type AB blood. Set up a frequency distribution and
find the following probabilities.
b. A person has type A or type B blood.
Type Frequency
22 5
A 22 P  A or B   
B 5 50 50
AB 2 27

O 21 50
Total 50
Example 4-13: Blood Types
In a sample of 50 people, 21 had type O blood, 22
had type A blood, 5 had type B blood, and 2 had
type AB blood. Set up a frequency distribution and
find the following probabilities.
c. A person has neither type A nor type O blood.
Type Frequency
P  neither A nor O 
A 22
B 5 5 2
 
AB 2 50 50
O 21 7

Total 50 50
Example 4-13: Blood Types
In a sample of 50 people, 21 had type O blood, 22
had type A blood, 5 had type B blood, and 2 had
type AB blood. Set up a frequency distribution and
find the following probabilities.
d. A person does not have type AB blood.
Type Frequency
A 22 P  not AB 
B 5  1  P  AB 
AB 2 2 48 24
O 21  1  
50 50 25
Total 50
Limitations:
 Despite improvement over the classical
approach, there are some objections:
 what is mean by limit as N goes to infinity? How
can we generate infinite sequence of trials?
 What happens to phenomena where repeated
trials are not possible?
Sample Spaces and Probability
There are three basic interpretations of
probability:

Classical probability

Empirical probability

Subjective probability

30
Sample Spaces and Probability
Subjective probability uses a probability value based on an
educated guess or estimate, employing opinions and inexact
information.

According to this approach, probability refers to the “degree of belief”

of individuals in assessing the uncertainty of a particular situation.
Since there is no past experience, it depends on educated guesses.

Examples: weather forecasting, predicting outcomes of sporting events

31
4.2 Addition Rules for Probability
 Two events are mutually exclusive
events if they cannot occur at the same
time (i.e., they have no outcomes in
common)
Addition Rules
P  A or B   P  A   P  B  Mutually Exclusive
P  A or B   P  A   P  B   P  A and B  Not M. E.

32
Example 4-15: Rolling a Die
Determine which events are mutually exclusive
and which are not, when a single die is rolled.
a. Getting an odd number and getting an even number

Getting an odd number: 1, 3, or 5

Getting an even number: 2, 4, or 6

Mutually Exclusive
Example 4-15: Rolling a Die
Determine which events are mutually exclusive
and which are not, when a single die is rolled.
c. Getting an odd number and getting a number less
than 4

Getting an odd number: 1, 3, or 5

Getting a number less than 4: 1, 2, or 3

Not Mutually Exclusive

 Example 4-21: Medical Staff
In a hospital unit there are 8 nurses and 5
physicians; 7 nurses and 3 physicians are females.
If a staff person is selected, find the probability that
the subject is a nurse or a male.
Staff Females Males Total
Nurses 7 1 8
Physicians 3 2 5
Total 10 3 13
P  Nurse or Male   P  Nurse   P  Male   P  Male Nurse 
8 3 1 10
   
13 13 13 13
The Axioms of Probability
 In mathematics an axiomatic system is required to be
complete, non-redundant and consistent.
 A system is said to be complete if the set of axioms
postulated should enable us to prove every other
theorem in theory in question using the axioms and
mathematical logic.

 Non-redundancy refers to the impossibility of deriving

any axiom of the system from the other axioms.

 Consistency refers to the non-contradictory nature of

the axioms.
 Axiom 1: For and event A in S, the probability of A is a
nonnegative real number. That is, P(A) ≥ 0.
 Axiom 2: The probability of the sample space S is unity,
i.e.,P(S) = 1.
 Axiom 3: If A1, A2, ... is a finite or infinite sequence of
mutually exclusive events in S then
 
P (  Ai )
i 1
  P( A )
i 1
i
 1.4 Some Theorems of Probability
Theorem 1.1: If A is an event in a discrete sample space S,
then P(A) equals the sum the probabilities of the individual
outcomes comprising A.

 Proof: Let O1, O2, ..., be the finite or infinite sequence of

outcomes that comprise the event A.
Thus, A = O1  O2 O3 ... . Since individual outcomes
are mutually exclusive,

P(A) = P(O1) + P(O2) + P(O3) + ... (axiom 3)

Examples:
1. In tossing a fair coin twice, find the probability of getting
at least one head.
The sample space is: S  {HH , HT , TH , TT }
An event A is the one that constitutes at least one head .
HH , HT , TH 
1 1 1 3
P ( A)    
4 4 4 4
 Then,

 Example 2:
A die is loaded in such a way that each odd number is likely
to occur twice as each even number. Find the probability
that an even number will occur on a single roll.

Solution: Let w be the probability that even number will occur.

2w + w + 2w + w + 2w + w = 1
9w = 1
1
w 
9
1 1 1 3
Then, P( E )    
9 9 9 9
Example 3:
For a given experiment: O1 , O2 , O3 ,..., the probability is
given by: i
1
P (Oi )   
2

for i= 1, 2, 3, ….
a. Verify that P(Oi ) satisfies axiom 2.
b. If Oi is the event that a person is tossing a fair coin,
find the probability of getting a tail on the first four
tosses.
c. If Oi is the event that a person is tossing a fair coin,
find the probability that the first head will occur on an
even numbered tosses.
 Theorem 1.2: If an experiment can result in any one of
the N different equally likely outcomes, and n of these
outcomes together constitute event A, then the probability
of event A is given by .
n
P( A) 
N
 Proof: Let O1, O2, ...,ON = individual outcomes in S each
with a probability of 1/N
A = union of n elements of S that are mutually exclusive.
Then, P(A) = P(O1  O2  ...  On)
= P(O1) + P(O2) + ... + P(On) (axiom 3)
= 1/N + 1/N + ... + 1/N (n terms in all)
= n/N
Theorem 1.3: P() = 0 for any sample space S.
Proof: Since S  = S and the events S and  are mutually
exclusive.
 P(S  ) = P(S)
 P(S) + P() = P(S) by axiom 3
 P() = 0
Note: P(A) = 0 does not imply A = 
Theorem 1.4: If A and B are events in a sample space S
and A  B, then P(A) ≤ P(B) (Monotonicity of the
probability function).
Proof: Since1. A  B, we can write B = A  (A/  B)
2. A and (A/  B) are mutually exclusive, it
follows that
P(B) = P(A) + P (A/  B) by axiom 3, therefore
P(B) ≥ P(A), as P(A/ B) 0 (by axiom 1)
Theorem 1.5: If A and A/ are complementary events
in a sample space S, then,
P(A/) = 1 - P(A)
Proof: P(S) = 1, by axiom 2
= P(A A/)
= P(A) + P(A/) by axiom 3
 1 - P(A) = p(A/)
Theorem 1.6: 0 ≤ P(A) ≤ 1
 Proof: Using theorem 2 and the fact that   A  S for any
event A and sample space S, we have
P() ≤ P(A) ≤ P(S). But P() = 0 and P(S) = 1.
Thus, 0 ≤ P(A) ≤ 1
Theorem 1.7: If A and B are two events in a
sample space S, then
P(A  B) = P(A) + P(B) - P(A  B)
Assignment: 1. If A, B, and C are any three events,
then show that P(A  B  C) = P(A) + P(B) + P(C) -
P(A  B) - P(A  C) - P(B  C) + P(A  B  C).

Assignment 2: An integer is chosen at random from the

first 400 positive integers. What is the probability that
the integer chosen is divisible by 6 or 8?
Assignment 3: Box 1 contains 3 red balls, 2 white
balls and 1 blue ball. Box 2 contains 1 red ball, 2
white balls and 3 blue balls. One ball is selected at
random from each urn.
I. Describe the sample space
II. Find the probability that both balls will be of the same
colour.
III. Is the probability that both balls are red greater than
the probability that both balls are white?
IV. Let A be the event that the first ball is white and B the
event that the second ball is blue. Find the probability
that A or B occur.
Exercise: Three people A, B, and C toss a coin
in turn until a head occurs. The person who
tosses a head wins.

a. Find the probability that A wins.

b. Find the probability that C wins.
c. If the coin is biased and head occurs with
probability P, what is the probability that B wins?
 1.5. Counting Rules

Fundamental Rule of Counting- the multiplication

Rule
The fundamental counting rule is also called the
multiplication of choices.
choices
In a sequence of n events in which the first one has k1
possibilities and the second event has k2 and the third
has k3, and so forth, the total number of possibilities of
the sequence will be
k1 · k2 · k3 · · · kn

53
Example : Paint Colors
A paint manufacturer wishes to manufacture several
different paints. The categories include
Color: red, blue, white, black, green, brown, yellow
Type: latex, oil
Texture: flat, semigloss, high gloss
Use: outdoor, indoor
How many different kinds of paint can be made if you can
select one color, one type, one texture, and one use?

 # of
 
# of # of # of
colors types textures uses  
7  2  3  2
84 different kinds of paint
Example: A college freshman must take an economics
course, a management course, and a statistics course. If
she may select any one of 3 economics courses, any of 4
management courses and any of 2 statistics courses, in
how many ways can she arrange her program?

n1 = 3, n2 = 4, n3 = 2 n1n2 n3 = 24
Counting Rules
Factorialis the product of all the positive
numbers from 1 to a number.

n !  n  n  1 n  2   3  2 1
0!  1

1.5.1.Permutation is an arrangement
of objects in a specific order. Order
matters.
Suppose that we have n distinct objects in a set and
we are interested in how many different ways these
objects may be arranged. 56
For example, if we have the letters a, b, and c, we can
consider the following arrangements: abc, acb, bac, bca,
cab, cba. Thus, the three letters can be arranged in six
different ways.

Theorem: The number of ordered arrangements or

permutations of n distinct objects taken all at a time (with
out repetition) is given by n! and denoted by n P n  n !

E.g. A, B and C: 3! = 3*2*1 = 6. That is, ABC, ACB, BAC,

BCA, CAB, CBA.

What about with repetition??? nn

 Example: If three of the letters A, B, C are arranged
all at a time, the number of arrangement is: 33=27.
AAA, AAB, AAC, ABA, ACA, ABB, ACC
BBB, BBA, BBC, BAB, BCB, BAA, BCC
CCC, CCA, CCB, CAC, CBC, CAA, CBB
ABC, ACB, BAC, BCA, CAB, CBA
 Theorem: The number of permutations of n distinct
objects taken r at a time (with out repetition) is given
by
n Pr  n(n  1)(n  2)...(n  r  1)
n!

 n  r !
 Example: Find the number of ways in which two
lottery tickets can be drawn from 20 for first and
second prizes.
 Solution:
20!
P
20 2   20 19  380
 20  2 !
 Exercise: Four married couples bought 8 seats
in a row for a concert. In how many different
ways can they be seated:
a. with no restriction?
b. If each couple is to sit together?
c. If all the men sit together to the right of all the
women?
 Solution:
 8! = 40,320
 (4x2)x(3x2)x(2x2)x(1x2) = 384
 (4!)x(4!) = 576

Theorem: The number of permutations of n distinct

objects taken r at a time (with repetition) is given
by: nr.

 Example: If three of the letters A, B, C are arranged

2 at a time, the number of arrangement is: 32=9 AA
,AB, AC,BB, BA, BC, CC, CA, CB,
Example : Television News Stories
A television news director wishes to use 3 news stories
on an evening show. One story will be the lead story, one
will be the second story, and the last will be a closing
story. If the director has a total of 8 stories to choose
from, how many possible ways can the program be set
up?

Since there is a lead, second, and closing story, we know

that order matters. We will use permutations.
8!
8 P3   336 or P  8
8 3  7  6  336
5! 3
Example : Business Locations
Suppose a business owner has a choice of 5 locations in
which to establish her business. She decides to rank
each location according to certain criteria, such as price
of the store and parking facilities. How many different
ways can she rank the 5 locations?

   
first second third fourth fifth

choice choice choice choice choice 
5  4  3  2  1
120 different ways to rank the locations

Using factorials, 5! = 120.

Using permutations, 5P5 = 120.
Example : Business Locations
Suppose the business owner in the above Example
wishes to rank only the top 3 of the 5 locations. How
many different ways can she rank them?

  
first second third
choice choice choice 
5  4  3
60 different ways to rank the locations

Using permutations, 5P3 = 60.

 Theorem: The number of permutations of n
distinct objects arranged in a circle is (n-1)!
(Circular permutation).

 Theorem: The number of permutations of n

objects of which n1 are of one kind (or alike), n2
of a second kind, …, nk of a k kind is
n!
n1! n 2 !...n k !
 Example: The permutations of letters of
the word “FOOD” is: 4!/2!1!1!=12

 FOOD, FODO, FDOO, DOFO, DOOF, DFOO,

OODF, OOFD, ODOF, OFOD, OFDO, ODFO
1.5.2. Combinations – refers to unordered
arrangements (i.e., no concern about the order
of the arrangements).
The number of ways of choosing r objects from a total of n
distinct objects, called the number of combinations of n
objects taken r at a time, is given by (order is not important
in this case).
n!
n Cr 
 n  r ! r !
Pr
 n

r!
67
 1)

 Note: when we select r objects from n distinct ones, we

leave n - r objects and hence, there are as many ways of
selecting r from n as there are not selecting n-r objects.
Thus
 n  n 
 1)   
r nr

 n   n  1   n  1
 2)  
r
   r
 
  r  1
Example : School Musicals
A newspaper editor has received 8 books to review. He
decides that he can use 3 reviews in his newspaper. How
many different ways can these 3 reviews be selected?

The placement in the newspaper is not mentioned, so

order does not matter. We will use combinations.
8!
8 C3   8!/ 5!3!  56
5!3!

87 6 P3
or 8C3   56 or 8C3   56
8

3 2 3!
Example : Committee Selection
In a club there are 7 women and 5 men. A committee of 3 women and
2 men is to be chosen. How many different possibilities are there?

There are not separate roles listed for each committee member, so
order does not matter. We will use combinations.

There are 35·10=350 different

7! possibilities.5!
Women: 7C3   35, Men: 5C2   10
4!3! 3!2!
 Example : From a group of 5 men and 3 women
how many committees of size 3 are possible:
a. with no restriction?
b. With 2 men and 1 woman?
c. With 1 man and 2 women if certain woman must
be on the committee?
4.5 Probability and Counting Rules

By using the fundamental counting rule, the permutation

rules, and the combination rule, you can compute the
probability of outcomes of many experiments, such as
getting a full house when 5 cards are dealt or selecting a
committee of 3 women and 2 men from a club consisting
of 10 women and 10 men.

72
Example 4-52: Committee Selection
A store has 6 TV Graphic magazines and 8 Newstime
magazines on the counter. If two customers purchased a
magazine, find the probability that one of each magazine
was purchased.

TV Graphic: One magazine of the 6 magazines

Newstime: One magazine of the 8 magazines
Total: Two magazines of the 14 magazines

C1 8 C1 6  8 48
6
 
14 C2 91 91
Example 3: A box of 20 spare parts for a certain
type of machine contains 15 good parts and 5
defective ones. If 4 parts are selected by chance
from this box, what is the probability that exactly
3 of them will be good?
 Solution: Let A be the event that 3 of the 4 parts
selected are good, then
15   5  15! 5!
 n( A)          2, 275
 3  1  3!15  3! 1!5  1!

 Let S be the sample space, then

 20  20!
 n( S )      4,845
4  4! 20  4 !

n A  2,275
 PA     0.47
n S 4,845
1.6. Conditional Probability and
Statistical Independence
 Conditional probability is the probability that the
first event B occurs given that the second event A
has occurred.
 Thus, this concept answers the question: what is
the probability that event A occurs, knowing that
event B has already occurred. Symbolically,

P( A  B)
P( A / B)  if P ( B )  0
P( B)
 Using the Venn diagram, conditional probability
is the area shown below:
Example : Women in the Military
A recent survey asked 100 people if they thought
women in the armed forces should be permitted to
participate in combat. The results of the survey are
shown.
Example : Women in the Military
a. Find the probability that the respondent answered yes
(Y), given that the respondent was a female (F).

8
P  F and Y  100 8 4

P YF   P  F

50
 
50 25
100
Example : Women in the Military
b. Find the probability that the respondent was a male
(M), given that the respondent answered no (N).

18
P  N and M  18 3

P MN   P N
 100
60
 
60 10
100
Example…
 Consider the experiment of tossing two fair coins. Let
S = {HH, HT, TH, TT}, and assume each point is equally
likely. Find:
a. the probability of two heads given a head on the
first coin.
b. The probability of two heads given at least one
head.
Solution: Let A1 = {head on the first coin} = {HH, HT}
Let A2 = {head on the second coin} = {HH, TH}
a. P  ( A1  A2) / A1 
P(( A1  A2)  A1)
P( A1)

P ( A1  A2) 14 1
  
P ( A1) 12 2

P  A1  A2   A1  A2 
b. P  A1  A2  /  A1  A2  
P  A1  A2 

P  A1  A2   A1  A2 

P  A1  P  A2   P  A1  A2 

P  A1  A2  14 1
  
P  A1  P  A2   P  A1  A2  1 2 1 2 1 4 3
Example
Let a pair of fair dice be rolled. If the sum is 6, find the
probability that one of the dice is a 2.
Solution
Let A - be an event that 2 appears on at least one die
Let B - be an event that the sum is 6

P(A/B) ?
 Theorem: If A is an event in S, then P(A//B) = 1 – P(A/B)
Proof:

P  A/  B 
P  A/ / B   but
P( B)

A
/
 SA
 A B 
/
 S  A  B
 S B    A B 

 P  A/  B   P  S  B    A  B 
 P  B   A  B 
 P  B   P  B  A  B 
 P  B   P  A B 
 Hence,
P  B   P  A B 
P  A/ / B  
P B
P  A B 
 1
P B
 1 P  A / B
Theorem: If A1 and A2 are elements of S, then
P  A1 / B   P  A1  A2  / B   P  A1  A2/  / B 
 Note: 1) 0 ≤P(A|B) ≤1
2) P(S/B) = 1
 Proof: PS / B   PS  B   PB   1
PB  PB 

 P(A/B) and P(B/A) are not necessarily equal. They are

equal iff P(A) = P(B)>0
 Theorem: For every two events A1 and A2 in S, then

P  A1  A2  / B   P  A1 / B   P  A 2 / B   P  A1  A 2  / B 

 Proof:
Theorem: P  / B   0
Proof:
P  / B   0

Theorem: If A1, A2, A3, …, An are mutually exclusive events

in S, then
 
n
P 
n

 
 i 1
Ai / B 

  PA
i 1
i / B

 n 
Ai  Aj  , i  j
n
P Ai    P( A ) i
 i 1  i 1

But  n  
 n  
P  Ai   B 
P  Ai  / B    i 1  
 i 1   P B
 n 
P  Ai  B 
  i 1 
P B
n

 PA i B 
 i 1
as  Ai  B   A j  B    for i  j
P B
n
P  Ai  B  n
 
i 1 P B
  PA
i 1
i / B
General Multiplication rule:
 If A and B are any two events in S and P(A) and P(B) are
not zero, then
 a) P(A  B) = P(A).P(B/A). This implies that the
probability that A and B will both occur is the product of
the probability of A and the conditional probability of B
given A. Alternatively,
 b) P(A  B) = P(B).P(A/B)
 Thus, P(A  B) = P(A).P(B/A) = P(B).P(A/B)

The general multiplication rule can be extended to the

case of more than two events. Let A, B, and C be any
three events in S such that P(A)> 0 and P(A  B) > 0,
then
 P(A  B  C) = P(A).P(B/A).P(C/(A  B))
Example: Suppose there is a box containing 20 fuses of
which 5 are defective. If 3 fuses are selected at random
and removed successively, what is the probability that all
are defective?
Let A = first fuse is defective
B/A = second fuse is defective given that A has occurred
C/A∩B = third fuse is defective given that A & B have
occurred
Therefore, P(A  B  C) = P(A).P(B/A ).P(C/(A  B)) =
(5/20).(4/19).(3/18) = 1/114  0.0088
 Statistical Independence of Two Events.
 Suppose that the conditional probability of A given B is
equal to the unconditional probability of A (i.e., P(A/B) =
P(A)).
 Thus, the occurrence or non occurrence of B does not
affect the probability occurrence of A.
 Definition: Let A and B be two events in S. Events A
and B are defined to be independent iff any of the
following conditions are satisfied:
i. P(A∩B) = P(A)P(B)
ii. P(A/B) = P(A) if P(B) > 0
iii. P(B/A) = P(B) if P(A) > 0
Note the difference between statistical independence and
mutually exclusive of two events.
 If A and B are mutually exclusive, then A  B =  and
hence corresponding probability is zero
 but independence requires that P(A  B) be the product of
the individual (non zero) probabilities.
 These two events are statistical independence and mutually
exclusive simultaneously iff either P(A) = 0 or P(B)= 0 or
both.
 But according to our conditions, this is ruled out. Thus,
mutually exclusive events can not statistical independence
at the same time.
 Example: Toss a fair coin twice.
S = {HH, HT, TH, TT}
Let A = {HH, HT}
B = {HT, TT}
C = {TT}
P(A) = 0.5, P(B) = 0.5, P(C) = 0.25, and P(A/C) = 0
 Now
1. P(A) = 0.5 and P(A/C) = 0, therefore A and C are not
independent. But they are mutually exclusive.
2. P(C) = 0.25 and P(C/B) = 0.5, therefore B and C are not
independent. They are also not mutually exclusive
 P(A  B) 0.25
P(A|B)=   0.5  P(A)
3. , P(B) 0.5
therefore A and B are independent. But they are not
mutually exclusive.
4. P(A  B) 0.25
P(B|A)=   0.5  P(B)
P(A) 0.5

therefore A and B are independent. But they are not

mutually exclusive.
 Note:
1. The sample space, S, and any arbitrary event A
are independent. That is, P(A/S) = P(A) or
P(S  A) = P(A).P(S) = P(A) because P(S  A) =
P(A) and P(S) = 1
2. The null event,, and any arbitrary event A are
independent. That is, P(/A) =P() or P(A  ) =
P(A). P() = 0 because P(A  ) = P()and P()
=0
3. If two disjoint events A and B are independent, then either
P(A) = 0 or P(B) = 0 or both.
i. If two sets are disjoint, (A  B) = 
ii. therefore, P(A  B) = P() = 0
iii. P(A  B) = P(A).P(B) = 0.

4. Independence of events is not transitive. i.e., if A and B

are independent, and B and C are independent, it does not
follow that A and C are also independent.
1.7. Bayes’ Theorem
 Thomas Bayes was the 18th century British minister and
clergyman. He worked on probability and his work was later
refined by Laplace and gave the name “Bayes’ theorem”.
The theorem is based on conditional probabilities.
 If A and B are two events with positive probabilities, then
P(A)P(B|A)
P(A|B)=
P(B)
 Proof: P(A  B)
P(A|B)= , and P(A  B)=P(B|A)P(A)
P(B)

 Combining the two equations, we derive the desired result.

 Proof: The nominator:
P  Ai  B 
P  Ai / B  
P B
But P  Ai  B   P  B / Ai  P  Ai  , multiplication rule

 The denominator can be written as:

B  S B
n
 (  A j  B )
j 1

 ( A1  B ) ( A2  B ) ( A3  B ) ... ( An  B )

also ,  Ai  B   A j  B    for i  j (orAi & A j are mutualy exclusive)

 Examples
1. Three machines A, B and C produce respectively 50%,
30% and 20% of the total number of items of a factory.
The percentages of defective output of these machines
are 3%, 4% and 5%, respectively. If an item is selected
at random and is found to be defective, find the
probability that the item was produced by machine A.
solution: Let X= the event that an item is defective
P( A)  0.5, P( B)  0.3, P(C )  0.2
P( X / A)  0.03, P( X / B)  0.04, P( X / C )  0.05
But P ( A  X )  P ( A).P ( X / A)
2. In a town 10% of all adults over 50 years have diabetes.
If a doctor correctly diagnoses 90% of all persons with
diabetes as having the disease and incorrectly
diagnoses 2% of all persons without diabetes as having
the disease, what is the probability of an adult over 50 is
diagnosed by this doctor as having diabetes and actually
has the disease.

Let A = An adult over fifty with diabetes

B = An adult over 50 diagnosed as having diabetes
 P(A) = 0.1, P(B/A) = 0.9,
P(B/A) = 0.9, and P(B/A/) = 0.02
Question: Find P(A/B)
 P  A P  B/ A 
P  A/ B  
P B 
but we do not know P(B)
Now 1. B = (B  A)  (B  A/) see Venn Diagram

Thus P(B) = P[(B  A)  (B  A/)]

= P(B  A) + P(B  A/)
= P(A).P(B/A) + P(A/).P(B/A/)
 It then follows that
P  A B 
P  A/ B  
PB 
P  A P  B/ A 

P  A  P  B / A  P  A /  P  B / A / 
0.1 x 0.9 0.09 0.09
    0.83
 0.1 x 0.9    0.9 x 0.02  0.09  0.018 0.108
3.There are five urns, and they are numbered 1 to 5. Each
urn contains 10 balls. Urn i has i defective balls and 10-i
non-defective balls. First an urn is selected at random
and then a ball is selected at random from the selected
urn without knowing which urn was selected, then

i. What is the probability that a defective ball is selected?

ii. if we have already selected the ball and noted that it is
defective, what is the probability that it came from urn 5?
 Solution: Let A denote the event that a defective ball is
selected, and Bi the event that urn i is selected, i =
1,2,..5, then P(Bi) = 1/5, i=1,2,…,5 and P(A/Bi) = i/10.

5
i 1 1 5 3
 
i 1 10

5
 
50 i 1
i 
10
Exercise
 1. We are interested with the completion of a highway construction
job, which may be delayed by a strike. Suppose that the probability
are 0.60 that there will be a strike, 0.85 that the job will be
completed on time if there is no strike, and 0.35 that the job will be
completed on time if there is a strike. What is the probability that the
job will be completed on time?
2. A number of a consulting firms rent cars from three rental agencies:
60% percent from agency I, 30% from agency II and 10% from
agency III. If 9% from I need a tune up, 20% from II need tune up
and 6% from III need tune up.
 a) what is the probability that a rental car delivered to the firm will
need tune up?
 b) if a delivered rental car needs a tune up, what is the probability
that it came from rental agency II.
3. In a given state 25% of all cars emit excessive pollutant. If
the probability is 0.99 that a car emitting excessive pollutant
will fail the vehicular emission test, and the probability is
0.17 that a car not emitting excessive pollutants will fail the
test, what is the probability that a car fails the test actually
emits excessive amount of pollutant.
 A certain town has two taxi companies: Blue Birds,
whose cabs are blue, and Night Owls, whose cabs are
black. Blue Birds has 15 taxis in its fleet, and Night Owls
has 75. Late one night, there was a hit-and-run accident
involving a taxi. The 90 town’s taxis were all on the
streets at the time of the accident. A witness saw the
accident and claims that a blue taxi was involved. At the
request of the police, the witness undergoes a vision test
under conditions similar to those on the night in question.
Presented repeatedly with a blue taxi and a black taxi, in
random order, he shows he can successfully identify the
color of the taxi 4 times out of 5.
 Which company is more likely to have been involved in
the accident?
 Let A1 = the event that a taxi is blue
 A2 = the event that a taxi is black
 B = the event that the witness claims a blue taxi was
involved
a) What is the probability that a taxi is blue?
b) What is the probability that a taxi is black?
c) What is the probability that the witness claims the taxi is
blue if indeed it is blue?
d) What is the probability that the witness claims the taxi is
blue if it is black?
e)What is the probability that the taxi is blue if the witness
says it is blue?
f) What is the probability that the taxi is black if the witness
says it is blue?
g) Which company has the highest probability to have been
involved in the accident?