Ignacio Cascos Fernández

Department of Statistics
Universidad Carlos III de Madrid

Introduction to Probability

Statistics — 2011-2012 (revised in 2022)

A random experiment is an experiment that can result in different out-
comes, even though it is repeated in the same manner every time and a
probability is used to quantify the likelihood, or chance, that an outcome of
a random experiment will occur.
The sample space is the set of all possible outcomes of a random experi-
ment. We will denoted the sample space by S. The sample space is said to
be discrete if it is finite or denumerable (countably infinite) and continuous
if it is uncountable.
An event is a subset of the sample space of a random experiment. We
will represent the events by capital letters, A, B, C, . . ..
Events are subsets (of a sample space), and thus the basic set operations
(union, intersection, and complement) are used to describe new events from
combinations of existing events.
Union. The union of two events A and B, denoted A ∪ B is the event
that consists of all outcomes that are contained in either of the two
events. It occurs whenever at least one from A and B occurs.
Intersection. The intersection of two events A and B, denoted A ∩ B
is the event that consists of all outcomes that are contained in both of
the two events. It occurs when both A and B occur simultaneously.
Complement. The complement of an event A in a sample space S,
denoted A or Ac is the set of outcomes in the sample space that are
not in the event. It occurs when A does not occur.
Set difference. The (Set) difference. The difference of two events A and
B expressed as A minus B, denoted A \ B is the event that consists of
all outcomes that are contained in A, but not contained in B. It occurs
when A occurs, but B does not.

1
 An elementary event is any element (singleton) from the sample space .
A compound event is any event that can be obtained as a union of several
elementary events.
The null or impossible event is the one that never occurs, therefore it is
represented as ∅. Its complement is the certain event, which consists of the
whole sample space, S.
Two events A and B are said to be mutually exclusive if they cannot
occur simultaneously, that is, their intersection is the empty set, A ∩ B = ∅.
A collection of events A1 , A2 , . . . , An is said to be mutually exclusive if for all
pairs, Ai ∩ Aj = ∅ for i ̸= j.
A collection of events A1 , A2 , . . . , An is exhaustive if their union is the
whole sample space ∩ni=1 Ai = S.
Definition 1. A probability P associated with a random experiment whose
sample space is S assigns to each member of a collection of events from
the random experiment, A, a real number P (A), that is interpreted as the
probability of A. A probability satisfies the following properties::
1. (nonnegativity) P (A) ≥ 0 for every event A;

2. (additivity) given a family of events {Ai }i∈I with I denumerable and

such that Ai ∩ Aj = ∅ if i ̸= j, then
n
[  n
X
P Ai = P (Ai );
i=1 i=1

3. (normalization) P (S) = 1.

Properties. Given two events A and B,

P (A) ≤ 1;

P (A) = 1 − P (A);

P (∅) = 0;

if A ⊂ B, then P (A) ≤ P (B);

P (A \ B) = P (A) − P (A ∩ B);

Addition rule. P (A ∪ B) = P (A) + P (B) − P (A ∩ B).

2
Definition 2. Two events A and B are independent if

P (A ∩ B) = P (A)P (B).

Property. If A and B are independent, then A and B are also independent,

and the same happens with A and B.
Definition 3. Given two events A and B with P (B) > 0, the conditional
probability of A given B is the probability that A occurs given that B has
occurred, it is denoted by P (A|B),
P (A ∩ B)
P (A|B) = .
P (B)
The conditional probability is a probability itself and, as a consequence,
it satisfies all the properties that are common to probabilities.

Property. Two sets A and B with P (A), P (B) > 0 are independent if and
only if P (A|B) = P (A) (or alternatively P (B|A) = P (B)).

Multiplication rule. Given n events A1 , A2 , . . . , An with P (Ai ) > 0 for

i = 1, . . . , n

P (A1 ∩ A2 ∩ A3 ∩ . . . ∩ An ) = P (A1 )P (A2 |A1 )P (A3 |A1 ∩ A2 ) · · ·

· · · P (An−1 |A1 ∩ A2 ∩ . . . ∩ An−2 )P (An |A1 ∩ A2 ∩ . . . ∩ An−1 ).

Total probability rule. Given a collection of n mutually exclusive and

exhaustive events, that is, given A1 , A2 , . . . , An such that Ai ∩ Aj = ∅ if i ̸= j
and ∪ni=1 Ai = S, the probability of any event B is given by
n
X
P (B) = P (Ai )P (B|Ai ).
i=1

Bayes’ rule. Given a collection of n mutually exclusive and exhaustive

events, that is, given A1 , A2 , . . . , An such that Ai ∩ Aj = ∅ if i ̸= j and
∪ni=1 Ai = S, for any event B such that P (B) > 0, it holds:
P (Ai ∩ B) P (Ai )P (B|Ai )
P (Ai |B) = = Pn .
P (B) j=1 P (Aj )P (B|Aj )

3
Assessment of probabilities in practice: Laplace rule. When a ran-
dom experiment has a finite number of equally likely possible outcomes, then
the probability of any event A is
number of outcomes favourable to A
P (A) = .
total number of possible outcomes

Basic principle of counting. If a random experiment can be split into

2 stages and the first stage has n1 possible outcomes, for each of them, the
second stage has n2 possible outcomes, then the total number of possible
outcomes of the experiment is: n1 · n2 .

Generalized principle of counting. If a random experiment can be split

into a fixed number of stages k and the first stage has n1 possible outcomes,
for each of them, the second stage has n2 possible outcomes and so forth
until the k-th stage, which has nk possible outcomes, then the total number
of possible outcomes of the experiment is:

n1 · n2 · . . . · nk .

Permutations. Given n distinct objects, the number of possible ordered

sequences that can be formed with them is called the permutations of n
elements, n! = n(n − 1)(n − 2) · · · 2 · 1.

k-permutations. Given n distinct objects, the number of possible sequen-

ces of 1 ≤ k ≤ n elements that can be formed with them is called k-
permutations of n elements, n(n−1) · · · (n−k+1) = n!/(n−k)! (by agreement
0! = 1).

Combinations. Given n distinct objects, the number of possible sets of

1 ≤ k ≤ n elements that be formed with them is the number of combinations
of k out of n objects  
n n!
= .
k k!(n − k)!