Introduction to Probability

Quantitative Business Methods - Oct 2018

Ritavan - Adjunct Faculty, EU Business School

1 Uncertainty as Probability
A Priori (Classical Probability): The way mathematicians study uncertainty
Empirical (Relative Frequency): The way statisticians study uncertainty
Subjective Probability: The way uncertainty is modeled in most real world situations

2 Sample Space
An impossible event, that is an event not contained in the sample space can never occur and has proba-
bility 0. But in an infinite sample space, an event with probability 0 can occur.

In a finite sample space an event with probability 1 is certain, that is it has to occur every time. In an
infinite sample space that is not the case.

Example: Consider the Lebesgue measure also called the uniform distribution on the unit interval [0, 1].
The event containing only the real number 21 has probability 0 and its complement has probability 1. If
you randomly pick a real number it can happen that you pick 12 and thus it can happen that an event with
probability 0 occurs while an event with probability 1 doesn’t occur.
Remember that phenomena like this are common to infinite sample spaces, but for the purpose of this
course we will usually have finite sample spaces and we will assign probability 0 to events that cannot
occur and are not in the sample space. Similarly events that have probability 1 will be certain and will
always occur.

Unless explicitly stated, we will assume that all sample spaces in this chapter are finite.

3 Conditional Probability
Consider event A in sample space S, then P (A) = P (A|S).

We define the conditional probability of A given B (where P (B) 6= 0), denoted by P (A|B) as

P (A ∩ B)
P (A|B) = (1)
P (B)

Note that you can see this on a Venn diagram by considering relative frequency.

1
4 Mutually Exclusive & Independence
Two events are mutually exclusive (also called disjoint) if they cannot both occur, that is the occur-
rence of one event excludes the occurrence of the other.
Trivial example is P(A) and P(Ac ).
Two events are independent if the occurrence of one does not influence the other and vice versa.
That is both of these equalities are true

P (A|B) = P (A)

P (B|A) = P (B)

Mutually exclusive events are not independent, because the occurrence of one of them
excludes the occurrence of the other.
Independent events cannot be mutually exclusive by definition.

5 Total Probability Theorem

Let Ai be a partition of the sample space S into n events, that is Ai are pairwise mutually exclusive
events whose union is the entire sample space, then for any event B we have

n
X
P (B) = P (B|S) = P (B|Ai ) (2)
i=1

6 Bayes’ Rule
By combining the formulae for conditional probability

P (A ∩ B)
P (A|B) =
P (B)

and the multiplication law

P (A ∩ B) = P (A|B)P (A)
we obtain Bayes’ Rule
P (B|A)P (A)
P (A|B) =
P (B)

The term P (A) is called the prior and after the occurrence of event B it is updated to P (A|B) which is
called the posterior. This is especially useful when the the term P (B|A) called the likelihood is known.
Bayes’ Rule is a way of updating probabilities based on new information. For example one could begin
with subjective probabilities and then conduct an experiment and update the probabilities based on the
outcome of the experiment, thereby enriching the model with new information obtained from data.
A variant of Bayes’ Rule is obtained by replacing P (B) by using the Total Probability Theorem to get

P (B|Ai )P (Ai )
P (Ai |B) = Pn
i=1 P (B|Ai )