Probability Theory and Optimization | Probability Theory | Mathematical Analysis

# 1

Probability Theory and Optimization

Module III
Probability –Classical approach, Conditional Probability and Independence

The objectives attained by this module are:
1. To study the classical approach to probability based on equally likely outcomes.
2. To familiarize calculation of the probability using the classical approach.
3. To understands the notion of conditional probability.
4. To understands the concept of independence of events.
5. To discuss problems based on conditional probability and independence.

Probability – Classical approach:

The classical approach to probability is based on a random experiment with the
sample space containing finite number of elements with the probability of occurrence
of all of the elements are same (equally likely).

Historically, the classical definition is the one of the oldest definition of
probability. The definition is as follows:

If a random experiment or trial results in ‘n’ exhaustive, mutually exclusive and equally
likely outcomes, out of which m are favorable to the occurrence of an event E, then the probability
of occurrence of E, is given by,
( )
Number of favourablecases to E m
P E
Total number of cases n
= =
Illustration:
Consider the random experiment of tossing of a die with numbers 1 to 6 written
on its six faces. The sample space, { } 1, 2,3, 4,5, 6 S = . Assume that the occurrences of all
the six faces are equally likely. Consider the event { } 2,3,5 A= . Here out of the 6
2

possible mutually exclusive, exhaustive and equally likely outcomes of the experiment,
three are (ie., 2,3 and 5) favorable to the event A. Hence by the definition,

3 1
( )
6 2
P A = = .

Consider the random experiment of picking a card from a well shuffled pack of
52 playing cards. Consider the event of ‘getting an ace’. There is nothing wrong to
assume that all the 52 cards having the same chance to be picked out in a trial. Out of
these 52 cards, four are ace cards. That is out of the 52 equally likely, mutually
exclusive and exhaustive possibilities of the experiment; four are favorable to the event
under consideration. Hence the probability of the event of getting an ace is
4
52
.

Classical definition makes the probability calculation of various events, when the
sample space under consideration is finite with equally likely possible elements.

A strong practical objection to this definition is its assumption that all possible
outcomes are equally likely. But in many situations, as illustrated above, this
assumption can be profitably used.

Xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Finding the probabilities of events using classical approach - problems:

A card is chosen at random from a well shuffled pack of 52 cards. Find the probabilities
of getting a card of (i) A king (ii) A red queen (iii) A black card or a queen.
Solution:
Let ‘A’ be the event of getting a king card. ‘B’ be the event of getting a red
queen. ‘C’ be the event of getting a black card and ‘D’ be the event of getting a
queen card.

The content of a pack of 52 cards is four sets of 13 cards each named as Heart,
Diamond, Spade and Club. Among these Spade and Club cards are BLACK in colour
and Heart and Diamond cards are RED. Each set of 13 cards consists of a king, a
queen, an ace card and cards with numbers 2, 3, 4…, 10 written on them. Hence in a
pack of cards there are 26 black cards, 26 red cards, 2 black kings, 2 red kings, 2 black
queens, 2 red queens, 4 ace cards, etc.
3

The problems are solving under the assumption that all possible outcomes
are equally likely. The random experiment under consideration is having 52
equally likely possible outcomes.

(i) The pack contains 4 kings. That is 4 out pf 52 cards are favorable to the
event A.
Hence,
4
( )
52
P A = .
(ii) The pack contains 2 red queens. That is 2 3vents are favorable to ‘B’
Hence,
2
( )
52
P B =
(iii) The event of ‘black card or a queen’ is represented as ( ) C D
( ) ( ) ( ) ( ) P C D P C P D P C D = + ÷ ·
Pack of card contains 26 black cards. Hence,
26
( )
52
P C = .
Pack of card contains 4 queens. Hence,
4
( )
52
P D = .
Out of 52 cards 2 cards are black queen cards,
Hence,
2
( )
52
P C D · = .
Now,
26 4 2
( )
52 52 52
P C D = + ÷ =
28
52

Two unbiased dice are thrown. What is the probability of getting ‘sum of the numbers
shown is less than 5’?

Solution:
When two unbiased dice are thrown, we can expect any of 36 equally
likely possible outcomes. The sample space is,
( ) ( ) ( ) ( ) ( ) { }
1,1 , 1, 2 , 1,3 ,..., 4,5 ,..., 6, 6 S =
Let ‘A’ be the event of the sum of the numbers shown is less than 5.
Out of the 36 outcomes, the following 6 outcomes are favorable to the event A.
( ) ( ) ( ) ( ) ( ) ( ) 1,1 , 1, 2 , 1,3 , 2,1 , 2, 2 , 3,1
4

Hence,
6 1
( )
36 6
P A = =

A box contains 5 red balls 6 white balls and 3 blue balls. Two balls are randomly chosen
from the box. Find the probabilities of (i) the balls are red (ii) the balls are 1 white and 1
red (iii) the balls are blue.

Solution:
The number of ways of choosing r objects from n distinguishable objects
is
( )
!
! !
n
r
n
C
r n r
= =
÷
.
Hence the number of ways of choosing 2 balls from a box of 14 balls is
14
2
C
. The balls are randomly chosen. So the
14
2
C possibilities are equally likely.

(i) To get red 2 red balls from the box, the selection should be from the 5 red
balls contained in the box. For this there are
5
2
C different ways. Therefore
out of
14
2
C equally likely ways for choosing 2 balls from the box the
5
2
C
ways are favorable for the event of getting red balls.
Hence P (the balls are red) =
5
2
14
2
10
91
C
C
= .
(ii) To get 1 white and 1 red ball, one ball should be selected from the given 6
white balls and the other ball from the 5 red balls. Total ways for selecting
one whit ball is
6
1
C and that of red ball is
5
1
C .
If first job can be performed in
1
n ways, second job in
2
n ways, …, the
th
r
job in
r
n ways, then the entire r jobs can be performed simultaneously in
( )
1 2
...
r
n n n × × × ways.
Now, the job of selecting one white and one red ball can be performed in
6 5
1 1
C C × ways out of
14
2
C ways of selecting two balls from the box.
So, P (the balls are red and white) =
6 5
1 1
14
2
30
91
C C
C
×
=
(iii) As explained in earlier cases, the number of ways for getting 2 blue balls
from the box with 3 blue balls is
3
2
C . Since the total number of ways for
getting 2 balls from the box is
14
2
C , the required probability is,
5

P (the balls are red) =
3
2
14
2
3
91
C
C
= .

What is the probability that two persons in a room having same date of birth?
Solution:
The birth date of each one of these persons may be any of the 365 days in a
year (we are ignoring the possibility of someone having date of birth on 29
th
February). Hence there are a total of
2
365 365 (365) × = equally likely possible
outcomes for the date of births.

P (the two persons having the same date of birth)
= 1- (the two persons having different date of births)

Let the first person is having his birth date on any one of the 365 days. For
the occurrence of date of births on different days for these two persons, the date
of birth of the second person should be any of the remaining 364 days. Hence,
for the event of ‘the two persons having different date of births’ there are
365 364 × possibilities out of
2
(365) ways of the total possible date of births.

This gives,
P (the two persons having different date of births)
( )
2
365 364 364
365
365
×
= =
Thus,
P (two persons in a room having same date of birth) =
364 1
1
365 365
÷ = .

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Conditional probability:

The probability of an event A, given that an event B of the same sample space
has happened is called the ‘conditional probability’ of A given B and is denoted by
P(A/B).

Consider the random experiment of tossing an unbiased die. Let the events
{ } 1,3 A= and { } 1,3,5 B = . Then P(A/B) is the probability of the outcomes 1 or 3, under
6

the information that the experiment has resulted in an odd number (ie.,
{ } 1,3,5 B = is an
event of odd numbers in a die tossing experiment) .

Since the sample space of the experiment
{ } 1, 2,3, 4,5, 6 S = , by the classical
definition, the unconditional probabilities of A and B are,
2
( )
6
P A = and
3
( )
6
P B = . But
when we are considering the conditional probability P (A/B), we are given the
experiment was resulted in B. Now it need only to find how many cases of B are
favorable to the event A. Here B contains 3 cases out of which the two cases (1 and 3)
are favorable to A. Hence the probability of the occurrence of A, through B is
2
3
. That is
2
( / )
3
P A B = .
In other words, the conditional probability ( / ) P A B is the probability of A, when
the sample space of the experiment is reduced to the event B.

Mathematically, ( / ) P A B is defined as,
( )
( / ) , , ( ) 0
( )
P A B
P A B provided P B
P B
·
= =
Or,
( )
( / ) , ( ) 0
( )
P A B
P B A provided P A
P A
·
= = .
From the above definition, it can reduce,

( ) ( ) ( / ) P A B P A P B A · = × , ( ) 0 P A = or ( ) ( ) ( / ) P A B P B P A B · = × , ( ) 0 P B = --- (*)
Equations (*) are known as the multiplication theorem for two events A and B.

Independence of events:

If the information that the event A has happened makes no change in the
probability of the happening of the event B, we say that A and B are two independent
events. That is if ( / ) ( ) P B A P B = itself, we say that A and B are independent events.

From (*), we have, ( ) ( ) ( / ) P A B P A P B A · = × provided, ( ) 0 P A =
Then, if A and B are independent events, we get,
7

( ) ( ) ( ) P A B P A P B · = × , provided, ( ) 0 P A =
Also we have,
( ) ( ) ( / ) P A B P B P A B · = × , provided ( ) 0 P B =

Since for two independent events A and B, ( / ) ( ) P A B P A =

We get,

( ) ( ) ( ) P A B P A P B · = × , provided, ( ) 0 P B =

Hence, in short, two events A and B are said to be independent, if

( ) ( ) ( ) P A B P A P B · = × , where ( ) 0 P A = and ( ) 0 P B = .

Note: Two disjoint (mutually exclusive) events cannot be independent.
Justification:
Assume two disjoint events A and B. The events A and B are
independent, if
( ) ( ) ( ) P A B P A P B · = × , where ( ) 0 P A = and ( ) 0 P B =
Since it is given A and B are disjoint,
( ) 0 P A B · =
For them to be independent, 0 ( ) ( ) P A P B = × , where ( ) 0 P A = and ( ) 0 P B = But
this cannot happen. Hence two disjoint (mutually exclusive) events cannot be
independent.

Problems:

Two unbiased dice are thrown. Let ‘A’ denote the event that the sum of the numbers
shown on the faces is odd and ‘B’ denote the event that ‘at least one die shows the face
with the number 1’. (i) Describe the complete sample space (ii) Find probabilities of the
events (a)
c c
A B · (b)
c
A B · (c) / A B (d) /
c c
A B .

Solution:
8

(i) The sample space contains 36 possibilities listed below. Since the given dice
are unbiased, the possibilities are equally likely and each is with probability
1
36
.

( ) { ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )}
1,1 , 1, 2 , 1, 3 , 1, 4 , 1, 5 , 1, 6 ,
2,1 , 2, 2 , 2, 3 , 2, 4 , 2, 5 , 2, 6 ,
3,1 , 3, 2 , 3, 3 , 3, 4 , 3, 5 , 3, 6 ,
4,1 , 4, 2 , 4, 3 , 4, 4 , 4, 5 , 4, 6 ,
5,1 , 5, 2 , 5, 3 , 5, 4 , 5, 5 , 5, 6 ,
6,1 , 6, 2 , 6, 3 , 6, 4 , 6, 5 , 6, 6
S =

(ii) The following 18 points are favorable to the event A, ie.,
( ) { ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )}
1, 2 , 1, 4 , 1, 6 , 2,1 , 2, 3 , 2, 5 ,
3, 2 , 3, 4 , 3, 6 , 4,1 , 4, 3 , 4, 5 ,
5, 2 , 5, 4 , 5, 6 , 6,1 , 6, 3 , 6, 5
A =

Hence, P(A)=
18
36
.
Out the 36 cases, the following 11 contained at least one 1.
This gives,
( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
(1,1), 1, 2 , 1, 3 , 1, 4 , 1, 5 , 1, 6 ,
2,1 , 3,1 , 4,1 , 5,1 , 6,1
B
¦ ¹
¦ ¦
=
´ `
¦ ¦
¹ )

Hence, P(B) =
11
36

(a) By De Morgan’s law,
( ) ( ) 1 ( )
c c c
P A B P A B P A B · = = ÷
9

( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( )}
(1,1), (1, 2), (1, 3), 1, 4 , (1, 5), 1, 6 , 2,1 , 2, 3 , 2, 5 ,
(3,1), 3, 2 , 3, 4 , 3, 6 , 4,1 , 4, 3 , 4, 5 ,
(5,1), 5, 2 , 5, 4 , 5, 6 , 6,1 , 6, 3 , 6, 5
A B
¦ ¹
¦ ¦
¦ ¦
=
´ `
¦ ¦
¦ ¦
¹ )

This contains 23 elements out of 36 equally likely possibilities. Then,
23
( )
36
P A B = .

23 13
( ) 1
36 36
P A B ¬ · = ÷ =
(b)
( ) { ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( )}
( ) 2, 3 , 2, 5 , 3, 2 , 3, 4 , 3, 6 , 4, 3 , 4, 5 ,
5, 2 , 5, 4 , 5, 6 , 6, 3 , 6, 5
A B · =

Since ( ) A B · contains 12 elements, ( ) P A B ·
12
36
= .
(c)
( )
( / )
( )
P A B
P A B
P B
·
=

( ) ( ) { ( ) ( ) ( ) ( ) ( )}
1, 2 , 1, 4 , 1, 6 , 2,1 , 4,1 , 6,1 A B · = .
( ) A B · contains 6 elements,
( )
6
36
P A B ¬ · = .
Hence,
( )
6
6
36
( / )
11
( ) 11
36
P A B
P A B
P B
·
= = = .
(d)
( )
( / )
( )
c c
c c
c
P A B
P A B
P B
·
=
We have,
13
( )
36
c c
P A B · =
10

and
11 25
( ) 1 ( ) 1
36 36
c
P B P B = ÷ = ÷ =

13
36
25
35
( ) 13
( / )
( ) 25
c c
c c
c
P A B
P A B
P B
·
¬ = = = .
A box of 10 items, 2 are defective. Items are randomly selected from the box one by and
examined. What is the probability that the sixth item selected is the last defective.

Solution
For the occurrence of the event ‘the sixth item is the last defective’, the
first five selections should contain one of the two defectives and the sixth
selection should be the second defective.
Let ‘A’ be the event of getting one defective from the first five selections
and ‘B’ be the event of getting the last (second) defective in the sixth selection.
The required probability is ( ) P A B · .
We have ( ) ( ). ( / ) P A B P A P B A · =
Here, the first five items can be selected in
10
5
C equally likely ways. Out
of these 5 items, to get 4 good and 1 defective, the 4 good items are selected from
the 8 good items and 1 defective is to be collected from the available two
defectives. This can be performed in
8 2
4 1
C C × ways.
Hence,
8 2
4 1
10
5
( )
C C
P A
C
×
= .
If ‘A’ has already happened, then the box contains five items, out of
which one is defective. Hence the probability of the occurrence of B under the
condition that the event A already happened is the probability of selecting the
last defective item from the box of remaining 5 items.
This gives,

1
( / )
5
P B A = .
Thus,
8 2
4 1
10
5
1
( )
5
C C
P A B
C
×
· = ×
1
9
= .

A and B are two independent events in the sample space S. Show that (i)
c
A and B (ii)
c c
A and B are also independent pairs.
11

Solution
Given A and B are independent events, this gives,
( ) ( ) ( ) P A B P A P B · = × , where ( ) 0 P A = and ( ) 0 P B = .
Since ( ) 0 P A = and ( ) 0 P B = , ( )
c
P A and ( )
c
P B are also non zeroes.
(i) Here to show ( ) ( ) ( )
c c
P A B P A P B · = ×
We have ( ) ( ) ( )
c
P A B P B P A B · = ÷ ·
( ) ( ) ( ) P A B P A P B · = × gives,
( ) ( ) ( ) ( ) P A B P B P A P B · = ÷

| | ( ) 1 ( ) P B P A = ÷
( ) ( ) ( )
c c
P A B P B P A ¬ · =
Hence by definition A and B are independent.
(ii) Consider ( ) P A B ·
By De Morgan’s law, ( ) ( )
c c c
P A P A B · =
| | ( ) 1 ( ) 1 ( ) ( ) ( ) P A B P A B P A P B P A B ¬ · = ÷ = ÷ + ÷ ·
1 ( ) ( ) ( ) ( ) P A P B P A P B = ÷ ÷ +
(Since A and B are independent)
| | ( ) 1 ( ) ( ) 1 ( )
c c
P A B P A P B P A · = ÷ ÷ ÷
| | | | ( ) 1 ( ) 1 ( ) ( ) ( )
c c c c
P A B P A P B P A P B ¬ · = ÷ × ÷ =
This implies A and B are independent.