Chap.

2 Axioms of Probability and Basic Theorems

 A probability model is just some kind of mathematical
representation of a real-world situation, wherein
random/chance experiments are encountered.
 The set of all possible outcomes (of a chance
experiment) is called the sample space. Let us denote
it as S.

1
  Individual outcomes are called (sample) points.
Briefly denoted, * + (in
terminology of sets, points elements)
 Quite often we are interested in some particular
event that contains certain sample points as its
members.
 Essentially, an event is a subset of S.
 Probabilities are assigned to events.
 Basics of set theory
2
 Recall that an event is a subset
 Set
 Element
 Equality of sets
 Subset
 Size of a set (finite, countably infinite, countable,
uncountably infinite, etc.)
# countable: finite or countably infinite
 Union of sets ( )
3
  Intersection of sets ( )
 Difference of sets
 Complement of a set (A), denoted as or ̅, in an
universal set (denoted as U)
 In probability, the universal set is the sample space.
 Empty set (denoted as )
 Def: Two sets A and B are said to be disjoint (aka.
mutually exclusive) iff .
 DeMorgan’s rules
4
  First rule: ( )
 Second rule: ( )
 Generalization to multiple-set cases is
straight-forward.
 Def: Consider a class: * +. F is called a
sigma algebra if it satisfies the conditions:
1.
2. If , then
3. If for , then
5
 Komogorov’s axiomization: Let S be the sample space.
For each (well-behaved) event A of S, a real number,
denoted by P(A), is associated with it. If P satisfies the
axioms below, then it is a probability and P(A) is said to
be the probability of A:
Axiom 1: P(A) 0
Axiom 2: P(S) = 1

6
 Axiom 3: ( ) ∑ ( ), for every
collection of pairwise mutually exclusive events , ,
, … of S.
 The requirement of events being “well-behaved” is
always met when S is countable.
 Precisely speaking, the “well-behaved” event means
that it is a set out of a sigma algebra.
 When S is countable, all events are subsets out of a
sigma field.
7
 Even when S is uncountable, for practical purposes,
all events considered in this course are well behaved.
 When S is some set of real numbers, then the
associated sigma field consists of:
1. a finite interval
2. complement of any finite interval
3. union of any countably number of finite intervals
4. intersection of any countably number of finite
intervals
8
 For a countable sample space S, it suffices to assign a
probability ( ) to each point such that
( ) and ∑ ( ) . Then, a probability
measure P on S is defined by assigning the probability of
each event A as ( ) ∑ ( ).
 If all sample points are equally likely, then ( )
( )
– this is the proportion interpretation of
( )

probability.
9
 Thm (#1): ( )
 Prf: Noting that (in words, and
are mutually exclusive) and applying A3 (i.e. Axiom 3),
we obtain ( ) ( ) ( )  ( )
 Thm (#2): Consider , , …, : (pairwise) mutually
exclusive events. Then, ( ) ∑ ( ).
 Prf: First augment the given sequence of events with
, , …. And then, apply A3 and
Thm (#1).
10
 Thm (#3): Given an event A, we have ( ) ( ).
 Prf: # By definition of complement, we have
.
# By A2, we have ( ) ( )
# By definition, and are mutually exclusive.
# By Thm (#2), we have ( ) ( ) ( )
# Therefore, ( ) ( ) (QED)
 Thm (#4): For any event A, ( )
 Prf: # By A1, we have ( ) and ( )
11
 # By Thm (#3), ( ) ( )
# Therefore, ( ) (QED)
 Thm (#5, aka. the addition rule for probabilities):
( ) ( ) ( ) ( )
 Prf: Check [Tijms, pp.26-27]
 Graphically, we can adopt this kind of
visualization 
 Can be generalized to the case of
( )
12
  Ex: ( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
(Similar visualization as the case of two events)
(Some kind of inclusion-exclusion)
 Ex (inclusion-exclusion): [Tijms, p.37, Example 1.19]
 Consider sequence of events: , , ,…
 Def: This sequence is increasing iff .
 Def: This sequence is decreasing iff .
 Limit of increasing/decreasing sequences of events:
13
  For increasing seq.,
 For decreasing seq.,
 Thm (Continuity of probability): If a sequence , ,
, … is either increasing or decreasing, then
( ) ( )
 Prf: Check [Tijms, p.24]
 Ex: [Ghahramani, p.28, Example 1.20]

14