Probability Theory

DEFINITION OF PROBABILITY

The probability of a given even is an expression of likelihood of occurrence of an event. A

probability is a number which ranges from 0 (zero) to 1 (one) – zero for an event which
cannot occur and 1 for an event which is certain to occur.

IMPORTANCE OF THE CONCEPT OF PROBABILITY

According to Ya-lun Chou, “Statistics, as a method of decision-making under uncertainty, is

founded on probability theory, since probability is at once the language and the measure of
uncertainty and the risks associated with it.”

1. Probability models can be used for making predictions.

2. Probability theory facilitates the construction of econometric model.
3. It facilitates the managerial decisions on planning and control.

CLASSICAL OR A PRIORI PROBABILITY APPROACH

The basic assumption of classical approach is that the outcomes of a random experiment are
“equally likely”. The terms ‘experiment’, ‘event’ and ‘equally likely’ are defined as follows:

Term Meaning Example

1. Experiment An experiment is the process of Throwing a dice
making an observation or taking a
measurement which results in
different possible outcomes.
2. Event An event is the outcome of the Occurrence of 1, 2, 3, 4, 5,
experiment. 6
3. Equally likely It implies that each outcome of an Occurrence of 1, 2, 3, 4, 5,
experiment has the same chance of 6 are equally likely events
appearing as any other:
According to Laplace, a French Mathematician: “Probability, is the ratio of the number of
‘favourable’ cases to the total number of equally likely cases.” If probability of occurrence of
A is denoted by p(A), then by this definition, we have:

Number of favourable cases(a)

p(A) =
Total number of equally likely cases n
Thus,

a
Probability of occurrence of an event (called its success) (p) =
n

b
Probability of non-occurrence of an event (called its failure) (q) =
n

1
a + b = n (since the sum of success and failure outcomes is equal to total number of events)
Dividing the above equation by n

a b n a b
+ = = 1 ∴ p + q = 1 (since = p, = q)
n n n n n

Classical probability is often called a priori probability because if one keeps using orderly
examples of unbiased dice, fair coin, etc. one can state the answer in advance (a priori)
without rolling a dice, tossing a coin, etc.

Classical definition of probability is not very satisfactory because of the following reasons:

(i) It fails when the number of possible outcomes of the experiment is infinite.
(ii) It is based on cases which are “equally likely” and as such cannot be applied to the
experiments where the outcomes are not equally likely.
(iii) It may not be possible practically to enumerate all the possible outcomes of
certain experiments and in such cases the method fails.
(iv) The definition is circular in nature, i.e., the definition of probability depends on
“equally likely” cases which means cases “with equal probability”.

MODERN DEFINITION OF PROBABILITY

Probability is the limit of the proportion of times that a certain event A will occur in repeated
trials of an experiment.

Let S be the sample space, let ∑ be the class of events and let P be a real valued function
defined on ∑ The P is called a probability measure, and P(A) is called the probability of the
event A if P satisfies the following axioms.

1. 0 < P(A) < 1 for every event A belonging to

2. P(S) = 1.
3. For every finite or infinite sequence of disjoint events A1, A2 …………
P(A1  A2  A3  …) = P(A1) + P(A2) + P(A3) + …

OBJECTIVE APPROACH AND SUBJECTIVE APPROACH TO PROBABILITY

The Objective approach to probability is arrived on opinion basis or an empirical basis. It is

given by the ratio of frequency of an outcomes to the total number. In contrast, the subjective
approach to probability is not concerned with the relative or expected frequency of an
outcome. It is concerned with the strength of a decision maker’s belief that an outcome will
not occur. As such, the subjective approach is particularly oriented toward decision-making
situations.

ILLUSTRATION 1

An unbiased coin is tossed. What is the probability that it is a head ? What is the probability
that it is not a head ?

Solution

2
Total number of equally likely events = 2 (Head or Tail), Number of heads = 1

1
Probability of getting a head (p) = = 0.5
2

1(Tail)
Probability of not getting a head (q) = 0.5 or 1 – p = 1 – 0.5 = 0.5
2

1 1
Thus, p + q = + =1
2 2

ILLUSTRATION 2

An unbiased cubis dice is thrown. What is the probability of getting –

(a) 6 (b) an even number (c) an odd number (d) a multiple of 2 (e) a multiple of 3

Solution

(a) Total numbers in a dice = 6, Number of 6 in a dice = 1

1
Probability of getting a six (p) =
6
(b) Total numbers in a dice = 6, Total even numbers in a dice (i.e., 2, 4, 6) = 3
3 1
Probability of getting an even number = = = 0.5
6 2
3 1
(c) Total numbers in a dice = 6, Total odd numbers in a dice (i.e. 1, 3, 5) = = = 0.5
6 2
(d) Total numbers in a dice = 6, Total even numbers in a dice (i.e., 2, 4, 6) = 3
3 1
Probability of getting an even number = = = 0.5
6 2
(e) Total numbers in a dice = 6, Total numbers which are a multiple of 3 (i.e., 3, 6) = 2
2 1
Probability of getting a multiple of 3 = = = 0.33
6 3

ILLUSTRATION 3

One card is drawn from a well shuffled standard pack of 52 playing cards. What is the
probability that it is a –

(a) king (b) king of red colour (c) king of heart (d) numeric card (e) numeric card
bearing a multiple of 2 (f) red numeric card bearing a multiple of 2 (g) Black odd numeric
card

Solution

(a) Total number of cards = 52. Total number of kings = 4

4 1
Probability of getting a king = =
52 13
(b) Total number of cards = 52, Total number of kings of red colour = 2

3
 2 1
Probability of getting a king of red colour = =
52 26
(c) Total number of cards = 52, Total number of kings of heart = 1
1
Probability of getting a king of heart =
52
(d) Total number of cards = 52, Total numeric cards = 36
36 9
Probability of getting a numeric card = =
52 13
(e) Total number of cards = 52,
Total number of numeric cards bearing a multiple of 2 (i.e., 2, 4, 6, 8, 10) = 5 x 4 = 20
20 5
Probability of getting a numeric card bearing a multiple of 2 = =
52 13
(f) Total number of cards = 52
Total number of red numeric cards bearing a multiple of 2 = 5 x 2 = 10
10 5
Probability of getting a red numeric card bearing a multiple of 2 = =
52 26
(g) Total number of cards = 52, Total number of Black odd numeric card = 4 x 2 = 8
8 2
Probability of getting a black odd numeric card = =
52 13

ILLUSTRATION 4

A bag contains 30 balls numbered from 1 to 30. One ball is drawn at random.
Find the probability that the number of the drawn ball will be a multiple of (a) 3 (b) 5 (c) 7
(d) 3 and 5 (e) 3 and 7 (f) 5 and 7.

Solution

Total number = 30

(a) Probability of the number being multiple of 3 is –

10 1
P (3, 6, 9, 12, 15, 18, 21, 24, 27, 30) = =
30 3
(b) Probability of the number being multiple of 5 is –
6 1
P (5, 10, 15, 20, 25, 30) = = = 0.2
30 5
4 2
(c) Probability of the number being multiple of 7 is – P (7, 14, 21, 28) = =
30 15
2 1
(d) Probability of the number being multiple of 3 and 5 is – P (15, 30) = =
30 15
1
(e) Probability of the number being multiple of 3 and 7 is – P (21) =
30
0
(f) Probability of the number being multiple of 5 and 7 is – P = =0
30

ILLUSTRATION 5

4
A bag contains 3 black, 4 white and 5 red balls. A ball is drawn at random. What is the
probability that it is a – (a) black ball (b) white ball (c) red ball (d) non-black ball

Solution

Total number of balls in the bag = 3 + 4 + 5 = 12

3 1
(a) Total number of black balls = 3, Probability of getting a black ball (p) = =
12 4
4 1
(b) Total number of white balls = 4, Probability of getting a white ball = =
12 3
5
(c) Total number of red balls = 5, Probability of getting a red ball =
12
(d) Total number of non-black balls = 4 + 5 = 9,
9 3
Probability of getting a non-black ball = =
12 4

EXPERIMENT

The term ‘experiment’ refers to processes which result in different possible outcomes or
observations. An experiment is called a random experiment if –

(a) all possible outcomes are known in advance (b) none of the outcomes can be
predicted with certainty.

EVENT

The term ‘events’ refer to the outcomes of an experiment. Events are generally denoted by
capital letters A, B, C etc. An event may be a certain or sure event, random event and
impossible event. Certain event (or sure event) – An event which is bound to occur in any
case when a certain random experiment is performed, is called a certain event or sure event.
For example, in a throw of an unbiased cubic die, the occurrence of any one of the numbers
1, 2, 3, 4, 5, 6 is a certain event or sure event.

Random event – An event which may or may not occur when a certain random experiment is
performed is called a random event. For example, in a throw of an unbiased cubic die the
occurrence of number 6 is a random event.

Impossible event – An event which can never occur when a certain random experiment is
performed is called an impossible event. For example, in a throw of an unbiased cubic die the
occurrence of number 7 is an impossible event.

TYPES OF EVENTS

1. Equally Likely Events

Meaning – Two or more events are said to be equally likely if any one of them can not be
expected to occur in preference to the others.

5
 Examples of Equally Likely Events

Experiment Equally likely events

1. In a tossing of an unbiased coin
2. In a throw of an unbiased cubic die
3. In drawing a card from a well shuffled
standard pack of playing cards
Head is likely to come up as tail
Any number out of 1, 2, 3, 4, 5, 6 is likely to
come up
Any card out of 52 is likely to come up

2. Collectively Exhaustive Events

Meaning – Events are said to be collectively exhaustive when their totality includes all the
possible outcomes of a random experiment.

Examples of Collectively Exhaustive Events

Experiment Collectively exhaustive events

1. In a tossing of an unbiased coin
2. In a throw of an unbiased cubic die
3. In drawing a card from a well shuffled
standard pack of playing cards
Possible outcomes – Head or tail
Exhaustive No. of cases – 2
Possible outcomes – 1, 2, 3, 4, 5, 6
Exhaustive no. of cases – 6
Possible outcomes – Ace to king
Exhaustive No. of cases – 52

3. Complementary Events

Meaning – A is called the complementary event of B (and vice versa) if A and B are
mutually exclusive and exhaustive.

Examples of Complementary Events

Experiment Complementary events

1. In a tossing of an unbiased coin
2. In a throw of an unbiased cubic die
3. In drawing a card from a well shuffled
standard pack of playing cards
Occurrence of head and tail
Occurrence of an even number (2, 4, 6) and
odd number (1, 3, 5)
Occurrence of a black card and a red card

4. Simple Events

Meaning – An event is said to be simple when the occurrence of a single event is considered.

Examples of Simple Events

Experiment Simple events

1. In a tossing of two unbiased coins Occurrence of 1 head
2. In a throw of two unbiased cubic dice Occurrence of 1 six

6
5. Compound Events

Meaning – An event is said to be compound when the joint occurrence of two or more events
is considered.

Examples of Compound Events

Experiment Compound events

1. In a tossing of two unbiased coins Occurrence of 1 or more heads
2. In a throw of two unbiased cubic dice Occurrence of 1 or more six

6. Mutually Exclusive Events

Meaning – Two or more events are said to be mutually exclusive if they can not occur
simultaneously in the same trial (i.e., the occurrence of one prevents the occurrence of the
other).

Examples of Mutually Exclusive Events

Experiment Mutually Exclusive Events

1. In a tossing of an unbiased coin
2. In a throw of two unbiased cubic dice
3. In drawing a card from a well shuffled
standard pack of playing cards
Head or tail
Occurrence of 1 or 2 or 3 or 4 or 5 or 6
1. Card is a spade or heart.
2. Card is a diamond or club.
3. Card is a king or queen.

Venn diagram – With the help of a set theory, these events can be show by the following
Venn diagram:

Definition – With the help of a set theory, mutually exclusive events are defined as follows :

Two events A and B are said to be mutually exclusive if A  B = f (i.e., A and B are disjoint
sets). In other words, when they have no elements in common.

Probability Function of Mutually Exclusive Events

P (A  B) = P (A) + P (B)

7. Overlapping Events (or Intersection Events)

7
Meaning – Two or more events are said to be overlapping or intersection events if part of
one event and part of another event can occur simultaneously in the same trial.

Examples of Overlapping or Intersection Events

Experiment Overlapping events

1. In a throw of an unbiased cubic die
2. In drawing a card from a well shuffled
standard packing cards events,
Occurrence of Multiple of 2 or 3
If 6 occurs, both the events (i.e., multiple of
2, multiple of 3) have occurred in the same
trial.
1. Card is a spade and card is a king
if king of spade is drawn both the cards of
playing events, (i.e., Spade, king) have
occurred in the same trial.

Venn Diagram – With the help of a set theory these overlapping events can be shown by the
following Venn diagram :

Definition – With the help of set theory, these events are defined as follows :

“Two or more events are said to be intersection events if A  B = f (i.e., A and B are not
disjoint sets). In other words, they have some element in common.

Probability Function – P (A  B) = P (A) + P (B) – P (A  B)

8. Independent Events

Meaning – Two or more events are said to be independent if the occurrence of one does not
affect the occurrence of the other.

13
Example – Probability of drawing a spade card from a pack of playing cards is and that
52
13
of another spade card after replacing the first one is also . Here the result of 2nd trial need
52
not be affected by the result of 1st trial since the card was replaced.

Probability Function – P (A  B) = P (A). P (B)

9. Dependent Events

8
Meaning – Two or more events are said to be dependent if the occurrence of one affects the
occurrence of the other.

13
Example – Probability of drawing a spade card from a pack of playing cards is while that
52
12
of another spade card when first card being not replaced is since the card was not
51
replaced before 2nd experiment.

Probability Function – P (A  B) = P (A) : P (B/A) or P (A  B) = P(B). P (A/B)

Thus, the experiment with replacement leads to independent events whereas experiment
without replacement leads to ‘dependent events’.

Distinction between Overlapping Events and Dependent Events

In overlapping, intersection of events is examined while in dependence, effect of 1st trial upon
the 2nd trial is examined.

Can the two events be mutually exclusive and independent simultaneously

Normally not but only in case when probability of not happening of other event is zero.

10. Mutually Exclusive and Exhaustive Events

A number of events are said to be mutually exclusive & exhaustive events if (i) every two of
them are mutually exclusive & (ii) one of them necessarily occurs in any trial. For example,
in experiment of drawing a card from a well shuffled pack of 52 cards, the following events
are mutually exclusive and exhaustive.

A: The card is a spade

B: The card is a club
C: The card is a diamond
D: The card is a heart

THEOREMS OF PROBABILITY

There are two important theorems of probability which are as follows:

1. The Addition Theorem; and

2. The Multiplication Theorem.

ADDITION THEOREM WHEN EVENTS ARE MUTUALLY EXCLUSIVE

Definition – The addition theorem states that if two events A and B are mutually exclusive
then the probability of the occurrence of either A or B is the sum of the individual probability
of A and B Symbolically.

P (A or B) or P (A  B) = P (A) + P (B)

9
Proof of the Theorem If an event A can happen in a 1 ways and B in a2 ways, then the
number of ways in which either event can happen is a1 + a2. If the total number of
possibilities is n, then by definition the probability of either the first or the second event
happening is

a1 +a2 a1 +a2
=
n n

a1
But = P (A)
n

a2
and = P (B)
n

Hence P (A or B) = P (A) + P (B)

The theorem can be extended to three or more mutually exclusive events. Thus,

P (A or B or C) = P (A) + P (B) + P (C)

Example – The probability of getting a multiple of 3 or 5 in a single throw of an unbiased

dice is an example where the events are mutually exclusive and addition rule of probability
applies.

ILLUSTRATION 6

An unbiased cubic dice is thrown. What is the probability of getting –

(a) a multiple of 3 or 4 (b) a multiple of 2 or 5

Solution

Total numbers in a dice = 6

2 1
(a) Probability of the number being multiple of 3 is P (3, 6) = =
6 3
1
Probability of the number being multiple of 4 is P (4) =
6
1 1 3 1
Probability of the number being multiple of 3 or 4 = + = =
3 6 6 2
3 1
(b) Probability of the number being multiple of 2 is P (2, 4, 6) = =
6 2
1
Probability of the number being multiple of 5 is P (5) =
6
1 1 4 2
Probability of the number being multiple of 2 or 5 = + = =
2 6 6 3

ILLUSTRATION 7

10
From a well shuffled standard pack of 52 playing cards, one card is drawn. What is the
probability that –

(a) it is either a red card or black card

(b) it is either a heart card or diamond card
(c) it is either a spade card or club card
(d) it is either a red card or spade card
(e) it is either a black card or heart card
(f) it is either a king or queen
(g) it is either a king of red colour or queen of black colour
(h) it is either a king of heart or queen of diamond

Solution

(a) No. of red cards = 26, No. of black cards = 26, No. of total cards = 52
26 26
Probability of red card = , Probability of black card =
52 52
26 26
Probability of red or black card = + =1
52 52
(b) No. of heart cards = 13, No. of diamond cards = 13, No. of total cards = 52
13 13
Probability of heart card = , Probability of diamond card =
52 52
13 13 26 1
Probability of heart or diamond card = + = =
52 52 52 2
(c) No. of spade cards = 13, No. of club cards = 13, No. of Total Cards = 52,
13 13
Probability of spade card = , Probability of club card =
52 52
13 13 26 1
Probability of spade or club card = + = =
52 52 52 2
(d) No. of red cards = 26, No. of spade cards = 13
26 13
Probability of red card = , Probability of spade card =
52 52
26 13 39 3
Probability of red or spade card = + = = = 0.75
52 52 52 4
(e) No. of black cards = 26, No. of heart cards = 13
26 13
Probability of black card = , Probability of heart card =
52 52
26 13 39 3
Probability of black or heart card = + = = = 0.75
52 52 52 4
(f) No. of kings = 4, No. of queens = 4
4 4
Probability of kings = , Probability of queens =
52 52
4 4 8 2
Probability of kings or queens = + = = = 0.154
52 52 52 13
(g) No. of kings of red colour = 2, Probability of queens of black colour = 2

11
 2
Probability of kings of red colour = ,
52
2
Probability of queen of black colour =
52
2 2 4 1
Probability of king of red colour or queen of black + = = = 0.077
52 52 52 13
(h) No. of king of heart = 1, No. of queen of diamond = 1
1 1 2 1
Probability of king of heart or queen of diamond = + = = = 0.0385
52 52 52 26

ILLUSTRATION 8

A bag contains 3 black, 4 white and 5 red balls. One ball is drawn at random. Find the
probability that –

(a) it is either a black ball or white ball

(b) it is either a black ball or red ball
(c) it is either a white ball or red ball

Solution

(a) No. of total balls = 12, No. of black balls = 3, No. of white balls = 4
3 4
Probability of black ball = , Probability of white ball =
12 12
3 4 7
Probability of black of white ball = + =
12 12 12
(b) No. of total balls = 12, No. of black balls = 3, No. of red balls = 5
3 5
Probability of black ball = , Probability of red ball =
12 12
3 5 8 2
Probability of black ball or red ball = + = =
12 12 12 3
(c) No. of total balls = 12, No. of white balls = 4, No. of red balls = 5
4 5
Probability of white ball = , Probability of red ball =
12 12
4 5 9 3
Probability of white or red ball = + = = = 0.75
12 12 12 4

ILLUSTRATION 9

Let A be the event that a business executive selected at random has stomach ulcer and B that
he has heart disease.

Interpret the following

(i) Ac  A (ii) Ac  A (iii) Ac  B (iv) A  Bc (v) (A  B)c

where, c stands for complement.

Solution

12
13