Theory of Probability

THEORY OF PROBABILITY

Introduction

In the theory of probability we deal with events which are the outcomes of
experiments, repeated under essentially homogeneous and similar conditions. An
experiment is defined as an action, which we conceive, do or intend to do. The experiments
are of two types:

(a) Deterministic experiments: Here chance does not play any part or role in the
outcome of an experiment. In other words, the given conditions completely determine the
outcome. That is to say, we can completely predict the outcome.
Thus for example, the pressure p and volume v of a gas are known to be related by
an equation of the form
pvk = c
from which we can determine p as soon as we know v; k and c being constants.

(b) Probabilistic experiments: Here chance plays an important role in the outcome
of an experiment. In other words, the given conditions do not determine completely the
outcomes and the phenomena is non-deterministic or probabilistic or stochastic.

Thus for example :

(i) One cannot guess with certainty whether it will be a rainy day or a sunny day
tomorrow.
(ii) In rolling an honest or unbiased die, one cannot predict with certainty which
face will turn up.

An event

An event is a set of specified outcomes that can result from an experiment.

If a die is tossed, we may be interested in the "event", 'A', that the number on the top
face is an even number. The event corresponds to the outcomes 2, 4 and 6. If an event is
denoted by A, the complementary event, i.e., non-occurence of A is denoted by A. In the
present case, the complementary event is the occurrence of 1, 3 and 5. In general, if there
are n cases out of which m are favourable to A, n-m outcomes are favourable to A.

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 1

Theory of Probability

Probability

Terminology related to Probability

Experiment: Experiment is an act that can be repeated under given conditions.

Unit experiment is known as trial. This means that trial is a special case of
experiment. Experiment may be a trial or two or more trials. For example, in the
tetrazolium test, each seed can be considered a trial.

Outcomes: The results of an experiment are known as out comes. For example, in
the tetrazolium test there are two possible outcomes: stained and non-stained.

Events: One or more outcomes of an experiment constitute an event. Events are

generally denoted by capital letters A. B., C, etc. For example, in the tetrazolium
test, stained seeds constitute an event.

Equally likely otcomes: Outcomes of a trial are said to be equally likely if we

have no reason to expect any one rather than the other. For example, in the
tetrazolium test of a seed, the outcomes stained and non-stained may be equally
likely.

Mutually exclusive outcomes: Outcomes or cases are said to be mutually

exclusive if the happening of any one of them precludes the happening of all
others. For example, in the tetrazolium test, the outcomes stained and non-stained
are mutually exclusive.

Favourable outcomes: The outcomes of experiment are said to be favourable to

an event if they entail the happening of the event. For example, in the tetrazolium
test of 5 seeds, the favourable outcomes of the event stained will be 5 because it
may be any one of the theses 5 seeds.

Exhaustive outcomes: The outcomes of experiment are said to be if they include

all possible outcomes. For example, in the tetrazolium test of a seed, exhaustive
number of outcomes are 2.

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 2

Theory of Probability

Classical or Apriori Definitions of Probability:

The porbability that an event A occurs is a measure of the level of certainty that the
outcome of a trial is A. If an experiment can result in n exhaustive, mutually exclusive and
equally likely outcomes, and m of these outcomes are favourable to an event A, then the
probability of the event A is defined as

If m = 0 then P(A) = 0 which means that A is impossible. If m = n then P(A) = 1 and the
event A is certain.

In general P(A) lies between these two extremes and a high value of P(A) indicates a high
level of certainty for A.

Example 1a

There are 10 chickens in a pen of which 5 are red, 3 are black and 2 are white. When the
door of the pen is opened the chickens are equally likely to escape in any order.

R R R R R

B B B W W

The probabilities that the first chicken out is red, black or white are 0.5, 0.3 and 0.2
respectively.

Other examples:

Coin tossing: Probability of a head =

Die rolling: Probability of a six =

Card cutting: Probability of a diamond =

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 3

Theory of Probability

Exclusiveness

Events are exclusive if they cannot occur together. In this case

P(A or B) = P(A) + P(B)

Example 1b

The chicken example gives a simple application of exclusiveness. The first chicken
out can either be red or black but cannot be both colours at the same time. The probability
that the first chicken out is either red or black is

P (R or B) = P(R) + P(B) = 0.5 + 0.3 = 0.8

Similarly the probability that the first chicken is either red or black or white is

P(R or B or W) = P(R) + P(B) + P(W) = 0.5 + 0.3 + 0.2 = 1

This means that it is certain that the first chicken out is either red or black or white,
which is the expected solution.

Non-exclusiveness

If there is an overlap of two events then the sum of their probabilities does not give
the probability that either one or the other occurs since the overlap would be double
counted. An adjustment overcomes the problem.

P(A or B) = P(A) + P(B) - P(A and B)

Example 1c

In the chicken example suppose that all the black birds are male, all the white birds
are female, but of the red birds there are three males and two females.

RM RM RM RF RF

BM BM BM WF WF

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 4

Theory of Probability

Suppose that the problems is to find the probability that the first bird out is either red
or male.
Now P(R) = 0.5 and P(M) = 0.6 so notice that the sum of these probabilities exceeds
1, which cannot be true. The explanation is that the two events can overlap since it is
possible for a chicken to be both red and male at the same time. The solution is

P(R or M) = P(R) + P(M) - P(R and M) = 0.5 + 0.6 - 0.3 = 0.8

Observe that the number of birds which are either red or male or both is 8, so the
problem could have been solved directly from the definition.

Independence

Two trials are independent if the outcome of the first trial has no effect on the
outcome of the second.

Suppose that two independent trials are conducted, then the probability that the
outcome of the first trial is A and the outcome of the second is B is given by the product

P(A and B) = P(A).P(B)

Example 1d

In the chicken example suppose that the door of the pen is opened so the chickens
come out, that they are rounded up then let out a second time. There are nine possible
arrangements of the coat colour for the first chicken out on the two occasions.

P(RR) = (.5)(.5) = 0.25

P(RB) = (.5)(.3) = 0.15
P(RW) = (.5)(.2) = 0.10
P(BR) = (.3)(.5) = 0.15
P(BB) = (.3)(.3) = 0.09
P(BW) = (.3)(.2) = 0.06
P(WR) = (.2)(.5) = 0.10
P(WB) = (.2)(.3) = 0.06
P(WW) = (.2)(.2) = 0.04

Total = 1.00

The nine possibilities are exclusive because none of them can occur simultaneously,

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 5

Theory of Probability

for example the sequence RB cannot occur if RW occurs, and RB is different from BR.
Moreover the list is exhaustive since all possibilities are covered. For any set of events that
is exclusive and exhaustive the total probability is 1.
As another example suppose it was required to calculate the probability that for the
same conditions
(a) a black chicken emerged first on either run.
(b) a black chicken emerged first both times.
(c) a black chicken did not emerge first on either occasion.
(d) a black chicken emerged first on exactly one run.

Solutions:
(a) The possibilities, which fulfil these conditions, are RB, BR, BB, BW, and WB. From
the table the sum of these probabilities is 0.51.
(b) The only possibility is BB, whose probability is 0.09
(c) The possibilities are RR, RW, WR, WW whose total probability is 0.49.
(d) Possibilities are BR, BW, RB, WB. Total probability 0.42.

Observe that the total probability for (a) and (c) is 1 since the two cases are
exclusive and exhaustive. Also note that the same point applies to (b), (c) and (d), as the
number of times that a black bird can emerge first in two runs is 0, 1 or 2.

Dependence

If there is dependence between two trials then the probability that the outcome are
respectively A and B is given by the product

where means the probability that B occurs given that A has already occurred.

Example 1e

In the chicken example there were 5 red, 3 black and 2 white birds and in example
1a they were released so that birds emerged in random order.

(a) Calculate the probability that the first out on release is red and that the second
out is black.

Solution:

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 6

Theory of Probability

Initially there are 5 red birds out of ten so the probability that the first out is red is
0.5. Now if the first out is taken to be red this leaves 4 red, 3 black and 2 white birds, so the
probability that the second bird is black given that the first is red is 3/9 or 1/3. The
probability P(RB) is given by

Example 1f
An urn contains 8 white and 3 red balls. If two balls are drawn at random, find the
probability that (a) both are white, (b) both are red and (c) one is of each colour.

Solution:
Total number of balls in the urn = 8 + 3 = 11

Two balls can be drawn out of 11 balls in 11C2 ways.

Exhaustive number of cases =

(a) Two white balls to be drawn

Out of 8 white , can be done in 8C2 = = 28 ways

The probability that both are white =

(b) Two red balls to be drawn

Out of 3 red balls can be done in 3C2 = 3 ways.

Hence, the probability that both are red =

(c) The number of favourable cases for drawing one white ball and one red ball is
8
C1 x 3C1 = 8 x 3 = 24.

Therefore, the probability (one red and one white)

Empirical Approach
Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 7
Theory of Probability

Classical approach is useful for solving problems involving games of chances-

throwing dice, coins, etc but if applied to other types of problems it does not provide
answers. For instance, if a man jumps from a height of 300 feet, the probability of his
survival will not be 50%, since survival and death are not equally alike. If we toss a coin 20
times, the classical probability suggests that we should have heads ten times. But in practice
it may not be so. The empirical approach suggests, that if a coin is tossed a large number of
times, say 1,000 times, we can expect 50% heads and 50% tails. Von Mises explained, "If
the experiment be repeated a large number of times under essentially identical conditions,
the limiting value of the ratio of the number of times the event A happens to the total
number of trials of the experiments as the number of trials increases identically, is called
the probability of the occurence of A".

Thus

Aposteriori Empirical or Statistical

If an event A occurs r times in a series of n independent trials, all made under the
same essential conditions, the ratio r/n is called the relative frequency of the event A in n
trials. The limit of , as n tends to infinity, is the probability P(A) of the event A. That is

This is called empirical, statistical or aposteriori probability because it is bassed on

empirical data or statistical record and not on prior consideration.

Subjective Probability

Subjective probability is the likelihood (probability) of a particular event

happening that is assigned by an individual based on whatever information is available.

Joint Probability:

A probability that measures the likelihood two or more events will happen
concurrently.
Problem # 1

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 8

Theory of Probability

A. A sample of 2,000 licensed drivers revealed the following number of speeding

violations.
No. of Violations No. of Drivers
0 1,910
1 46
2 18
3 12
4 9
5 or more 5
Total 2,000

I. What is the probability that a particular driver had exactly two speeding
violations?
II. What is the probability that a particular driver had more than two speeding
violations?
III. Which concept of probability did you use to make this estimate?

B. A sample of employees of Worldwide Enterprises is to be surveyed about a new

pension plan. The employees are classified as follows:

Classification Event No. of

Employees
Supervisors A 170
Maintenance B 150
Production C 1,250
Management D 350
Secretarial E 80

I. What is the experiment and explain what type of experiment is it?

II. What is the probability that the first person selected is:
(i) either in maintenance or a secretary?
(ii) not in management?
III. Which concept of probability did you use to make this estimate?

PROBLEM # 2

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 9

Theory of Probability

Three tables listed below show “random variables” and their “probabilities”. However,
only one of these is actually a probability distribution.
a. Which is it?
Tab. # 1 Tab. # 2 Tab. # 3
x P(x) x P(x) x P(x)
5 .3 5 .5 5 .2
10 .4 10 .3 10 .5
15 .2 15 -.2 15 .4
20 .1 20 .4 20 .3

b. Using the correct probability distribution, find the probability that x is:
(1) Exactly 20 (2) No more than 15 (3) More than 10 (4) At least 15

PROBLEM # 3

What is the probability that a card chosen at random from a standard deck of cards
will be either a king or a heart?Draw a A Venn diagram portrays these outcomes, which are not
mutually exclusice.

Solution
We may be inclined to add the probability of a king and the probability of a hart. But this
creates a problem. If we do that, the kign of hearts is couted with the kings and also with the
hearts. So, if we simply add the probadility of a king (there are 4 in a deck of 52 cards) to
the prbadility of a heart (there are 13 in a deck of 52 cared) and report that 17 out of 52
cards meet the requirement, we have counted the king of hearts twice. We need to subtract 1
card from the 17 so the king of hearts is counted only once. Thus, there are 16 cards that
are either hearts or kings. So the probability is 16/52=.3077.

card probability Explanation

King P(A) = 4/52 4 kings in a deck of 52 cards
Heart P(B) = 13/52 13 hearts in a deck of 52 cards
King of Hearts P(A and B) = 1/52 1king of hearts in a deck of 52 cards

From formula,
P(A or B) = P(A) + P(B) - P (A and B)
= 4/52 + 13/52 - 1/52
= 16/52, or .3077

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 10

Theory of Probability

A Venn diagram portrays these outcomes, which are not mutually exclusice.

Summary of Approaches to probability

Approaches to Probability

Objective Subjective

Based on available
information
Classical Empirical
Probability Probability

Based on equally Based on relative

likely outcomes frequencies

Professor Dr. Khandoker Saif Uddin Lecture # 5, Page 11