Elements of Probability Theory

1
Preliminaries
Most people know intuitively what the word probability means. Many of us have
encountered probabilities in our daily activities.
Probability is used to model uncertainty (or chance) when dealing with experiments or
natural phenomena whose outcome is not predictable in advance. From now on, we will
simply refer to "random experiments" to include both experiments and phenomena.
Even if the outcome of the random experiment isn't known in advance, let us suppose
that the set of all possible outcomes is known.

• Sample space or outcome space H: the set of all possible outcomes of an

experiment. It can be

- finite: the number of possible outcomes is limited

- countable: outcomes can be put into one-to-one correspondence with the positive
integers
- uncountable

A sample space is discrete if it has a finite or countable number of outcomes otherwise it

is continuous.

2
Example "  H finite
+Ñ If the random experiment consists in tossing a coin, the sample space contains two
outcomes, heads or tails, that is

H ={=2 , => }

,Ñ If the random experiment consists in throwing a die, the sample space contains six
outcomes, i.e.

H œ Ö=" ß =# ß =$ ß =% ß =& ß =' ×

where =3 is the outcome which occurs if 3 dots appear on the die.

Example # H countable
If the random experiment consists in flipping a coin until the first head is shown, the
sample space is given by
H œ Ö=3 ß 3 œ ",#,á ×
where =3 is the outcome which occurs if the first head occurs on exactly the 3-th flip
(3 œ ",#,á ).

3
Example $ H uncountable
Consider the random experiment which consists in recording the effective stock of
Apple quote tomorrow. The continuous sample space is given by
H œ Ö=B ß B  !×
where =B is the outcome which occurs if the quote (US dollars) is exactly B (B  !).

• Event E: any collection of possible outcomes of an experiment (E is any subset of H).

An event is said to occur if the actual outcome of the experiment belongs to that event.
Events with exactly one outcome are called single or elementary events.

Note that the sample space H is the sure event since it will always occur, while the
empty set g, that is a subset of H, is called the impossible event since it will never occur.
Some notes about set theory are essential since the events are sets.

4
Example ".b
Consider the random experiment consisting in throwing a die.

The elementary events are Ö=" ×ß Ö=# ×ß Ö=$ ×ß Ö=% ×ß Ö=& ×ß Ö=' ×Þ
The event ‘even face’ is given by E œ Ö=# ß =% ß =' ×Þ
The event ‘odd face’ is given by F œ {=" , =$ , =& }.

Example 2.b
Consider the random experiment which consists in flipping a coin until the first head is
shown.

The elementary event Ö=3 × occurs if getting the first head on exactly the 3-th flip
(3 œ ",#,á ). The event ‘the first head appears after ten flips’ is given by
E œ Ö=3 ß 3 œ "", "#, á ×.

Example 3.b
Consider the random experiment consisting in recording tomorrow Apple stock quote.

The elementary event Ö=B × occurs if the quote (US dollars) is exactly B (B  !).
The event ‘the quote is greater than 10 US dollars’ is given by E œ Ö=B ß B  "!×Þ

5
Some elements of Set Theory

Venn diagrams: a rectangle is used to represent the sample space while the inner parts of
closed curves (ovals or rectangles for example) represent subsets.

Venn diagram of I

Subset/Inclusion: E § F . If E occurs, then F also occurs.

Venn diagram of E § F

B
A

6
Union event: E ∪ F œ {=: = − E or = − F }
It is the event 'either E or F occurs'ß i.e. E ∪ F occurs if E or F or both occur.

Venn diagram of the union of E and F

A B

 E3
In general, if M is a set of indices, the union event

3−M

occurs if at least one of the E3 events occurs.

In every discrete sample space H, any event E is the union (at most countable) of its

E œ  Ö A×
elementary events

=−E
7
Intersection event: E ∩ F œ {=: = − E , = − F }
It is the event 'both E and F occurs'ß i.e. E ∩ F occurs if A and F occur.
Venn diagram of the intersection of E and F
B
A

Events E and F are said to be mutually exclusive if they cannot occur together. This
means that the sets are disjoint.
Venn diagram of E and F mutually exclusive (disjoint)

B
A

 E3
In general, the intersection event

3−M

occurs if all the events E3 occur.

8
Some properties of union and intersection

Let E, F , G subsets of H.

Commutative property E∪F œF∪E

E∩F œF∩E

Associative property E ∪ ÐF ∪ GÑ œ ÐE ∪ FÑ ∪ G

E ∩ ÐF ∩ GÑ œ ÐE ∩ FÑ ∩ G

Distributive property E ∩ ÐF ∪ GÑ œ ÐE ∩ FÑ ∪ ÐE ∩ GÑ

E ∪ ÐF ∩ GÑ œ ÐE ∪ FÑ ∩ ÐE ∪ GÑ

9
Complement (or negation) of E: E- œ Ö= − H : =  E×
It is the event 'E doesn’t occur'.

Venn diagram of E and E-

Ac
A

Note that H- œ g and g- œ HÞ

In general, the following relations are fulfilled

E ∪ E- œ Hß E ∩ E- œ gß

( E3 )- œ  E3- , ( E3 )- œ  E3- De Morgan's Laws

3−M 3−M 3−M 3−M

10
Difference event: F Ï E œ Ö= : = − Fß =  E×
It is the event 'F occurs and E does not occur'.

In particular, when E is a proper subset of F , that is E § Fß F Ï E is called proper

difference event.
Venn diagram of F Ï E when E § F

B
A

In generale, note that F is the union of the two disjoint sets E ∩ F and F Ï E.

11
Partition: a collection of 8 events E" ß E# ß á ß E8 such that

1. E3 œ H - the events are said to be exhaustive

8

3œ"

#Þ E3 ∩ E4 œ g , a3 Á 4=1ß 2ß á ß 8 - the events are disjoint or mutually exclusive

This means that each single outcome belongs to one and only one event of the partition
and thus one and only one event of the partition always occurs.
Note that the single events constitute a partition of the sample space H.

Venn diagram of six events forming a partition

E2
E1
E3
E4
E5 E6

12
Probability space
For each event E © H, we would like to be able to associate a number between 0 and 1
which will be called the probability of E, denoted by T ÐEÑ.

Events are subsets of H, but need all the subsets of H be events?

Unfortunately, only if H is a discrete space, it's possible to consider the set of all
possible subsets of H, that is the power set of H denoted by c ÐHÑÞ

When H is continuous, there are some technical difficulties to overcome and it is

necessary to take a subcollection T of the set of all subsets of H, which has to be a 5-
algebra.

Let T be a 5 -algebraß i.e. a collection of subsets of H, containing H and g such that T is

3Ñ closed under complementation: if E − T also E- − T
33Ñ closed under countable unions: if E3 − T is a countable sequence of sets,  E3 − T
∞

Since  E3- œ Ð  E3 Ñ- , T is also closed under countable intersection.

3œ"
∞ ∞

3œ" 3œ"

13
Associated with the sample space H there are many 5 -algebras.

Examples of 5 -algebra

The smallest 5 -algebra associated with H is the collection T œ Ögß H×Þ

If E is any subset of H, then T œ Ögß E, E- ß H× is a 5 -algebra.

The power set of H, c ÐHÑ, is a 5 -algebra.

Remark. When H is finite, i.e. H œ Ö=" ß =# ß á ß =R ×, the number of its possible subsets,
that is the cardinality of c ÐHÑ, is given by #R .

• There is  R0  œ 1 subset of H with ! outcome, that is g.

• There are  R"  œ R subsets of H with " outcome: Ö=" ×,Ö=# ×,á ,Ö=8 × .

• There are  R#  œ R ÐR "Ñ

# subsets of H with # outcomes: Ö=" ß =# ×,á ,Ö=8" ß =8 ×.

• So on, till considering the  R 

R œ " subset of H with R outcomes, that is H.

14
Thus, the number of all the possible subsets of H is given by

#c ÐHÑ œ  + + +á +  œ    œ (1+1)R œ 2R

R
R R R R R
0 " # R 3œ!
3

Example of power set

Random experiment:
- selecting a ball from an urn containing $ balls numbered with "ß #, $

- registering the number on the selected ball

The sample space is given by

H œ Ö=" ß =# ß =$ ×,
where =3 is the outcome which occurs if the ball with number 3 is selected, the
cardinality of the power set is #$ œ ) events and the power set is given by

T œ c ÐHÑ œ Ögß Ö=" ×,Ö=# ×,Ö=$ ×ß Ö=" ß =# ×, Ö=" ß =$ ×, Ö=# ß =$ ×ß H×

15
With any experiment we may associate a pair ÐHß TÑ, where H is the sample space and
T is a 5 -algebra of subsets of HÞ

A probability measure is a function T : T Ä Ò!ß "Ó such that

"Þ T ÐHÑ œ 1
#Þ if ÐE3 Ñ3− − T is a sequence of events pairwise disjoint, that is E3 ∩ E4 œ g a3 Á 4,

T ÐE3 Ñ œ T ÐE3 Ñ
∞ ∞
5-additivity property
3œ" 3œ"

The triple ÐHß T, T Ñ is called a probability space.

The previous properties are usually referred to as the Kolmogorov Axioms. When H is
finite, the axiom 2 can be weakened with

#w Þ if E and F are two mutually esclusive events, T ÐE ∪ FÑ œ T ÐEÑ  T ÐFÑ

16
Some properties of the probability measure T

"  T ÐE- Ñ œ 1  T ÐEÑ

In particular T ÐgÑ œ !

Proof. Since H œ E ∪ E- and E, E- are mutually esclusive

" œ T ÐHÑ œ T ÐEÑ  T ÐE- Ñ
#  Given two events Eß F , with F § E,
T ÐFÑ Ÿ T ÐEÑ
Proof. Note that
E œ F ∪ ÐE Ï FÑ
Since F and E Ï F are mutually esclusive
T ÐEÑ  T ÐFÑ œ T ÐE Ï FÑ   !

17
$  Given two events E, F
T ÐE ∪ FÑ œ T ÐEÑ  T ÐFÑ  T ÐE ∩ FÑ

Since T ÐE ∪ FÑ Ÿ 1, Bonferroni's inequality follows

T ÐE ∩ FÑ   T ÐEÑ  T ÐFÑ  "

Proof. Since E ∪ F œ F ∪ ÐE Ï FÑ, being F and E Ï F mutually escluvive eventsß it

holds
T ÐE ∪ FÑ œ T ÐFÑ  T ÐE Ï FÑ

Moreover since

E œ ÐE ∩ FÑ ∪ ÐE Ï FÑ
and E ∩ F and E Ï F are mutually esclusive
T ÐE Ï FÑ œ T ÐEÑ  T ÐE ∩ FÑ

18
%  Let ÐE3 Ñ3− § T be an increasing sequence of events (i.e. E3 § E3" ß 3 œ "ß á Ñß
then

T Ð  E3 Ñ œ lim T ÐE3 Ñ
∞

3œ" 3Ä∞

%  Let ÐF3 Ñ3− § T be a decreasing sequence of events (i.e. F3" § F3 ß 3 œ "ß á Ñ,

then

T Ð F3 Ñ œ lim T ÐF3 Ñ
∞

3œ" 3Ä∞

For any sample space many different probability functions, which satisfy the axioms,
can be defined.

19
Example
If the experiment consists in tossing a coin, we can take

H ={=2 , => }, T œ {g, {=2 }, {=> }ß H }

Fixed : − Ò!ß "Ó, a probability measure T À T Ä Ò!ß "Ó is given by

T ÐgÑ œ !, T Ð{=2 }Ñ œ :, T Ð{=> }Ñ œ "  :ß T ÐHÑ œ "

When the coin is fair : œ "# .

T is called Bernoulli probability measure with parameter :.

Note that the family of probability measures is uncountable even if H is a finite sample
space.

20
Finite sample space
When H œ Ö=" ß =# ß á ß =R × and T œ c ÐHÑ, any probability measure T is determined
by the values T Ð{=" }Ñ ß á ß T Ð{=R }Ñ since

T ÐEÑ œ T Ð  Ö=3 ×Ñ œ  T Ð{=3 }Ñ

3 À =3 −E 3 À =3 −E

that is, the probability of an event E is the sum of the probabilities of the single events
which are contained in it. Note that

T Ð{=3 }Ñ œ "
R

3œ"

Given a finite sequence of non-negative numbers :" ß á ß :R ,with

:3 œ "ß
R

3œ"

a probability measure T can be univocally determined by

T ÐEÑ œ  :3
3 À =3 −E

21
In particular if :" œ á œ :R œ R" ß T is said the uniform probability on H and the
classical definition of probability is obtained since
#E
T ÐEÑ œ
R
that is the probability of an event E is the proportion of the outcomes that are in E.

Example
Consider the experiment which consists in throwing a die
H œ Ö=" ß =# ß =$ ß =% ß =& ß =' ×ß T œ c ÐHÑ
+Ñ The die is fair: the probability of any single event is
"
T (Ö=3 ×) œ 3 œ "ß á ß 'Þ
'
For any E § T, T ÐEÑ œ #E ' Þ
For example, the event ‘odd face ’ E œ {=" , =$ , =& } has probability
$
T ÐEÑ œ œ !Þ&.
'

22
,Ñ The die is unfair: the probability of getting the face with three dots is twice the
probability of getting any other face.

Let
T (Ö=3 ×) œ :3 3 œ "ß #ß $ß %ß &ß '

Denoting by : the probability of the faces with a number of dots different from three,
the die is unfair such that
:$ œ #: :1 œ :# œ :% œ :& œ :' œ :Þ
Since the sum of the probabilities of the elementary events must be "ß that is
:"  :#  :$  :%  :&  :' œ (: œ ",
"
Ê :œ (

Therefore :$ œ #( while :3 œ "( for 3 œ "ß #ß %ß &ß 'Þ

The event ‘odd face’ E has probability
" # " %
T ÐEÑ œ :"  :$  :& œ (  (  ( œ (

23
Countable sample space
When H œ Ö=" ß =# ß á ×, T is still c ÐHÑ and any probability measure T is always
determined by the values T Ð{=" }Ñ ß T Ð{=# }Ñß á since it continues to hold

T ÐEÑ œ T Ð  Ö=3 ×Ñ œ  T Ð{=3 }Ñ

3 À =3 −E 3 À =3 −E

owing to axiom #. In particular

T Ð{=3 }Ñ œ "
∞

3œ"

Also in this case, given a sequence of non-negative numbers :" ß :# ß á such that

:3 œ "ß
∞

3œ"

a probability measure T can be univocally determined by

T ÐEÑ œ  :3
3 À =3 −E

24
Example
Consider the experiment which consists in throwing a die until an even face occurs
H œ Ö=3 ß 3 œ ",#,á ×ß T œ c ÐHÑ
where =3 is the outcome which occurs if an even face occurs for the first time at throw 3.
If the die is fair, the probability that an even face occurs is "# , in such a way that

T (Ö=3 ×) œ  
" 3
3 œ ",#,á
#
Note that the sum of the series of probabilities of the elementary events is ", that is

T Ð{=3 }Ñ œ   œ #
∞ ∞ "
" 3
"
œ"
3œ" 3œ"
# " #

For any event E − Tß T ÐEÑ œ  T Ð{=3 }Ñ œ   "# 

3

3 À =3 −E 3 À =3 −E
If E is the event 'an even face occurs in three throws' that is E œ Ö=" ß =# ß =$ ×
" " " (
T ÐEÑ œ #  %  ) œ )

T is said geometric probability measure with parameter "# .

25
Uncountable sample space
When H œ Ö=B ß B − Z × and Z is uncountable, it is not possible to assign a probability to
any events E − c ÐHÑÞ

Probabilities are assigned only to the events in a suitable 5-algebra T § c ÐHÑ and it is
not possible to characterize all probability measures on T.

An important example - Z œ ‘
Let T be the smallest 5-algebra which contains all the events of the form
Ö=B ß B − Ð+ß ,Ñ×
for all real numbers + and ,. It can be straightforwardly proven that T automatically
contains the events
Ö=B ß B − Ò+ß ,Ó×ß Ö=B ß B − Ð+ß ,Ó×ß Ö=B ß B − Ò+ß ,Ñ×

The 5 -algebra formed by all the intervals Ð+ß ,Ñ is called the Borel 5-algebra of ‘ and its
usually denoted by U БÑ.

26
ÖB À 0 ÐBÑ Ÿ >× is a borelian set of U Ð‘Ñ for any real >Ñ, with ∞ 0 ÐBÑ .B œ ",
Given a non-negative continuous function or more in general borelian function 0 (i.e.
∞

Lebesgue ("*!&) proved that there exists a unique probability measure T on the Borel 5-
algebra U Ð‘Ñ such that

T ÐÖ=B ß B − Ð+ß ,Ñ×Ñ œ  0 ÐBÑ .B

,

More precisely, given an event E which belongs to U БÑ, the function

T À E È  0 ÐBÑ .B
E

is a probability measure on TÞ

27
Conditional probability
Given a probability space (Hß Tß T Ñß sometimes the probability T has to be updated
when new information is acquired. Supposing that the event F − T had occurred.
The probability of E − T has to be modified taking into account this information.

From the Venn diagram, it is at once apparent that, as only outcomes in F can have
occurred, in order to evaluate the probability of the event E, only the outcomes in
E ∩ F have to be considered.
H
B
A

Example
Consider the experiment which consists in throwing a fair die. Supposing that the event
F œ Ö=# ß =% ß =' × occurred, even if the precise outcome is still unknown. The probability
of the event 'odd face' has been reduced from "# to !.

28
The conditional probability given the event F is a probability measure.

• Since T ÐE ∩ FÑ   ! and T ÐFÑ  !ß T ÐE ± FÑ   !.

• T ÐH ± FÑ œ T TÐHÐFÑ
∩FÑ
œ TT ÐFÑ
ÐFÑ
œ"
• Given two mutually exclusive events E and G , the events E ∩ F and G ∩ F are
mutually exclusive too in such a way that
T ÐÐE ∪ GÑ ∩ FÑ T ÐÐE ∩ FÑ ∪ ÐG ∩ FÑÑ
T ÐE ∪ G ± FÑ œ œ
T ÐFÑ T ÐFÑ
T ÐE ∩ FÑ  T ÐG ∩ FÑ T ÐE ∩ FÑ T ÐG ∩ FÑ
œ œ 
T ÐFÑ T ÐFÑ T ÐFÑ
œ T ÐE ± FÑ  T ÐG ± FÑ
The previous results can be straightforwardly extended to the case when a sequence E3
of disjoint events are considered so that 5 -additivity property is satisfied.

29
Formally, if T ÐFÑ  ! the conditional probability of event E given the event F is
defined by
T ÐE ∩ FÑ
TF ÐEÑ œ T ÐE ± FÑ œ
T ÐFÑ
Thus the probability space (Hß Tß TF Ñ is considered when F has occurred.

Moreover, if T ÐEÑ  !, also the conditional probability of event F given the event E
can be defined as

T ÐE ∩ FÑ
TE ÐFÑ œ T ÐF ± EÑ œ
T ÐEÑ

From the previous two definitions, the product rules for probabilities of intersections are
derived
T ÐE ∩ FÑ œ T ÐE ± FÑT ÐFÑ œ T ÐF ± EÑT ÐEÑ

30
The product rules can be generalized to more than two events. In particular, if three
events E, F and G are considered
T ÐE ∩ F ∩ GÑ œ T ÐG ± E ∩ FÑT ÐF ± EÑT ÐEÑ
Generalizations for more than three events are straightforward.
It is worth noting that product rules for probabilities of intersections are frequently used
in random experiments that can be considered as combinations of sub-experiments.

Example
Consider an urn containing ' white balls and % red balls. The random experiment
consists in consecutively selecting three balls without replacement. The goal is to
determine the probability of obtaining three red balls.

Denote by E=< the event 'a red ball is selected at =-th selection', for = œ "ß #ß $Þ
Therefore, using the product rules for probabilities of intersections, it holds
T ÐE"< ∩ E#< ∩ E$< Ñ œ T ÐE$< ± E#< ∩ E"< ÑT ÐE#< ± E"< ÑT ÐE"< Ñ

# $ % "
œ † † œ
) * "! $!

31
Example
Consider the experiment which consists in selecting a ball from an urn containing *!
balls numbered with "ß á , *!:

H œ Ö=3 ß 3 œ ",á , *!×ß T œ T ÐHÑ

Let E and F be the event 'a ball with number multiple of 10 is selected' and 'a ball with
even number is selected' respectively, that is

E œ Ö="! ß =#! ß á , =*! ×, F œ Ö=#3 ß 3 œ "ß á ß %&×

Their probabilitites are given by

* " %& "
T ÐEÑ œ œ ß T ÐFÑ œ œ
*! "! *! #

The probability of E given F is obtained by

T ÐE ∩ FÑ T ÐEÑ "
T ÐElFÑ œ œ œ
T ÐFÑ T ÐFÑ &
"
The probability of E has been modified from "! to &" Þ
Note that T ÐFlEÑ œ ": if a multiple of ten is obtained, it's surely even.
32
Now, if E is the event 'a ball numbered "ß á ß "! is selected'
"! "
T ÐEÑ œ œ
*! *
and the probability of E given F is
&
T ÐE ∩ FÑ *! "
T ÐElFÑ œ œ "
œ
T ÐFÑ #
*

Ê The knowledge that F has occurred does not modify the probability of E.

The converse is also true: conditioning on E, the probability of F does not change.
&
T ÐF ∩ EÑ *! "
T ÐFlEÑ œ œ "
œ œ T ÐFÑ
T ÐEÑ *
#

Then, the knowledge that one of the two events has occurred does not change the
probability of the other event.

33
Example
Consider the random experiment which consists in two consecutive throws of a fair die
H œ ÖÐ=3 ß =4 Ñ À 3 œ "ß á ß 'ß 4 œ "ß á ß '×

Let E be the event "six dots in the first throw" and F be the event "six dots in the second
throw"
E œ ÖÐ=' ß =4 Ñ À 4 œ "ß á ß '× ß F œ ÖÐ=3 ß =' Ñ À 3 œ "ß á ß '×

Since T ÖÐ=3 ß =4 Ñ× œ "Î$', as the number of outcomes is $' and all the outcomes are
equally likely, the probabilities of E and F are given by
' "
T ÐEÑ œ T ÐFÑ œ œ
$' '
Moreover

T ÐE ∩ FÑ " T ÐE ∩ FÑ "
T ÐF ± EÑ œ œ œ T ÐFÑ, T ÐE ± FÑ œ œ œ T ÐEÑ
T ÐEÑ ' T ÐFÑ '

Ê The knowledge that E has occurred does not change the probability that F occurs
and the knowledge that F has occurred does not change the probability that E occurs .
34
Independent events
An event F is said to be independent from an event E if the probability that F occurs is
not influenced by the knowledge that E has occurred
T ÐE ∩ FÑ
T ÐF ± EÑ œ œ T ÐFÑ
T ÐEÑ
Note that if F is independent from E, then T ÐE ∩ FÑ œ T ÐEÑT ÐFÑ and hence

T ÐE ∩ FÑ T ÐEÑT ÐFÑ
T ÐE ± FÑ œ œ œ T ÐEÑß
T ÐFÑ T ÐFÑ

that is E is independent from F , so E and F are called independent.

The independence can be characterized by means of the probability of intersection:

E and F are independent iff
T ÐE ∩ FÑ œ T ÐEÑT ÐFÑ

If E and F are independent, also E- and Fß F - and E, E- and F - are independent.

35
Let E and F be independent.

• E- and F are independentÞ Since E and E- form a partition of H, it holds

T ÐE- ∩ FÑ œ T ÐFÑ  T ÐE ∩ FÑ œ T ÐFÑ  T ÐEÑT ÐFÑ œ T ÐFÑT ÐE- Ñ

• F - and E are independentÞ Since F and F - form a partition of H, it holds

T ÐF - ∩ EÑ œ T ÐEÑ  T ÐE ∩ FÑ œ T ÐEÑ  T ÐEÑT ÐFÑ œ T ÐEÑT ÐF - Ñ

•E- and F - are independentÞ Since E- ∩ F - œ ÐE ∪ FÑ- it holds

T ÐE- ∩ F - Ñ œ T ÐÐE ∪ FÑ- Ñ œ "  T ÐE ∪ FÑ
œ "  T ÐEÑ  T ÐFÑ  T ÐE ∩ FÑ
œ "  T ÐEÑ  T ÐFÑ  T ÐEÑT ÐFÑ œ T ÐE- ÑT ÐF - Ñ

36
Note that the independence is a consequence of how the probability measure has been chosen.

Example
Consider the experiment which consists in throwing a die
H œ Ö=" ß =# ß =$ ß =% ß =& ß =' ×ß T œ c ÐHÑ
and consider the following two events
E œ Ö=# ß =% ß =' ×, F œ Ö=" ß =# ×
+Ñ The die is fair. E and F are independent since
" " "
T ÐE ∩ FÑ œ T ÐÖ=# ×Ñ œ œ † œ T ÐEÑT ÐFÑ
' # $
Remark. E- and Fß F - and E, E- and F - are independent.

,Ñ The die is unfair. The probability of getting the face with three dots is twice the
probability of getting any other face. E and F are not independent since
" $ #
T ÐE ∩ FÑ œ T ÐÖ=# ×Ñ œ ( Á ( † ( œ T ÐEÑT ÐFÑ

37
More generally, a family ÖE3 À 3 − M× is called independent if

T Ð E3 Ñ œ T ÐE3 Ñ for all finite subsets N of M .

3−N 3−N

If the family ÖE3 À 3 − M× has the properties

T ÐE3 ∩ E4 Ñ œ T ÐE3 ÑT ÐE4 Ñ for all 3 Á 4

it is called pairwise independent. Pairwise-independent families are not necessarily

independent.

Example
Consider the experiment which consists in selecting a ball from an urn containing % balls
numbered with "ß #ß $ß %ß i.e. H œ Ö=3 ß 3 œ ", #ß $ß %×ß T œ T ÐHÑß and consider the
following three events
E œ Ö=" ß =# ×, F œ Ö=" ß =$ ×, G œ Ö=" ß =% ×

Eß F and G are pairwise independent

T ÐE ∩ FÑ œ T ÐEÑT ÐFÑ œ "% , T ÐE ∩ GÑ œ T ÐEÑT ÐGÑ œ "% ,
T ÐF ∩ GÑ œ T ÐFÑT ÐGÑ œ "%
but not independent since
" "
œ T ÐE ∩ F ∩ GÑ Á T ÐEÑT ÐFÑT ÐGÑ œ
% )
38
Example
A company produces light bulbs which are the result of assemblying two different
components of type A and type B . The bulb is defective if at least one of the two
components is defective.
The probability of being defective is !Þ!& for component A, !Þ!1 for component B .

We want to compute the probability that the light bulb will not be defective.

To this aim, let us consider the following event:

 I is the event "the bulb is not defective"ß
 E is the event "component A is defective"
 F is the event "component B is defective" Þ

Note that
I œ E- ∩ F -
As the two events E and F can be supposed to be independent, also their complements
E- and F - are independent. Therefore
T ÐIÑ œ T ÐE- ∩ F - Ñ œ T ÐE- ÑT ÐF - Ñ œ !Þ*& ‚ !Þ** œ !Þ*%!&

39
Bayes rule
By the product rule of intersection: T ÐE ± FÑT ÐFÑ œ T ÐF ± EÑT ÐEÑ

From the previous expression, the Bayes rule arises

T ÐF ± EÑT ÐEÑ
T ÐE ± FÑ œ
T ÐFÑ

Given a partition E" ß E# ß á ß E8 of H where T ÐE3 Ñ  ! for any 3 œ "ß á ß 8, for any

F œ  ÐF ∩ E3 Ñ
8
event F − T, it holds
3œ"

T ÐFÑ œ  T ÐF ∩ E3 Ñ œ  T ÐFlE3 ÑT ÐE3 Ñ

8 8
By using the additivity property
3œ" 3œ"
and from the Bayes rule it holds
T ÐF ± E3 ÑT ÐE3 Ñ
 T ÐFlE3 ÑT ÐE3 Ñ
T ÐE3 ± FÑ œ 8

3œ"

40
Example
Consider two dice:
 the first is a fair die
 the second is such that the probability of getting a face with an odd number of dots is
twice the probability of getting a face with an even number of dots.

The random experiment consists in flipping a coin, if heads appears the fair die is
thrown, otherwise the manipulated die is thrown.
The aim is to compute the probability of getting the face with three dots and, given that
the face with three dots has appeared, the probability that the manipulated die has been
thrown.

As to the fair die, all the faces have the same probability of appearing.

As to the manipulated die, denoted by : the probability of getting a face with an even
number of dots, the probability of getting a face with an odd number is #:. Therefore
:"  :#  :$  :%  :&  :' œ #:  :  #:  :  #:  : œ "
and : œ "Î*ß in such a way that the probability of getting a face with three dots is #Î*Þ

41
Denoted by E the event "the face with three dots has appeared" and by F the event "tails
has appeared", where T ÐFÑ œ "Î#Þ Since
E œ ÐE ∩ FÑ ∪ ÐE ∩ F - Ñ
it follows that
T ÐEÑ œ T ÐE ∩ FÑ  T ÐE ∩ F - Ñ
# " " " (
œ T ÐE|FÑT ÐFÑ  T ÐE|F - ÑT ÐF - Ñ œ †  † œ
* # ' # $'
Moreover, given that the face with three dots has appeared, by using Bayes rule the
probability that the manipulated die has been thrown is given by

# "
T ÐE ∩ FÑ T ÐE|FÑT ÐFÑ * † # %
T ÐFlEÑ œ œ œ (
œ
T ÐEÑ T ÐEÑ $'
(

42
Example
An analyst predicts that there is a %!% chance that the U.S. economy performs well. If
the U.S. economy performs well, then there is an )!% chance that Asian countries will
also perform well. On the other hand, if the U.S. economy performs poorly, the
probability of Asian countries performing well goes down to !Þ$!Þ

 E is the event ‘Asian countries perform well’

 F is the event ‘U.S. economy performs well’

The analyst aims to compute

 T ÐE ∩ FÑ : probability that both the U.S. and the Asian countries will perform well
 T ÐEÑ: probability that the Asian countries perform well
 T ÐFlEÑ: probability that the U.S. economy will perform well, given that the Asian
countries perform well.

Since T ÐFÑ œ !Þ%!ß T ÐElFÑ œ !Þ)! and T ÐElF - Ñ œ !Þ$!Þ

T ÐE ∩ FÑ œ T ÐElFÑT ÐFÑ œ !Þ)! † !Þ%! œ !Þ$#Þ

43
To determine T ÐEÑ, note that E can be expressed as the union of disjoint events as

E œ ÐE ∩ FÑ ∪ ÐE ∩ F - Ñ

so that
T ÐEÑ œ T ÐE ∩ FÑ  T ÐE ∩ F - Ñ œ T ÐElFÑT ÐFÑ  T ÐElF - ÑT ÐF - Ñ
œ T ÐElFÑT ÐFÑ  T ÐElF - ÑT ÐF - Ñ
œ !Þ)! † !Þ%!  !Þ$! † Ð"  !Þ%!Ñ œ !Þ$#  !Þ") œ !Þ&
As to the probability that the U.S. economy will perform well, given that the Asian
countries perform well, using Bayes rule

T ÐElFÑT ÐFÑ !Þ$#

T ÐFlEÑ œ œ œ !Þ'%
T ÐEÑ !Þ&

44