EE313 Probability for EE

Probability Axioms
Murat Saraçlar
Probability of an Event
• Our current model of an experiment consists of a procedure and
observations. This leads to a set-theory representation with
• a sample space (universal set 𝑆),
• outcomes (𝑠 ∈ 𝑆), and
• events (𝐴 ⊂ 𝑆).
• To complete the model, we assign a probability 𝑃[𝐴] to every event,
𝐴, in the sample space.
• With respect to our physical idea of the experiment, the probability of
an event is the proportion of the time that event is observed in a
large number of runs of the experiment.
• This is the relative frequency notion of probability.
EE313 Probability for EE -- Murat Saraçlar 2
Axioms of Probability
Definition 1.5: A probability measure 𝑃[·] is a function that maps
events in the sample space to real numbers such that
Axiom 1 For any event 𝐴, 𝑃[𝐴] ≥ 0.
Axiom 2 𝑃[𝑆] = 1.
Axiom 3 For any countable collection 𝐴!, 𝐴", … of mutually exclusive
events
𝑃 𝐴! ∪ 𝐴" ∪ ⋯ = 𝑃 𝐴! + 𝑃 𝐴" + ⋯.
In other words,
𝑃 ⋃% 𝐴
#$! # = ∑%
#$! 𝑃[𝐴# ]

EE313 Probability for EE -- Murat Saraçlar 3

Some Theorems that follow from the Axioms
• Theorem 1.3 For mutually exclusive events 𝐴! and 𝐴",
𝑃[𝐴! ∪ 𝐴"] = 𝑃[𝐴!] + 𝑃[𝐴"].
• Theorem 1.4 If 𝐴 = 𝐴! ∪ 𝐴" ∪ ⋯ ∪ 𝐴& and 𝐴# ∩ 𝐴' = ∅ for 𝑖 ≠ 𝑗,
then 𝑃 𝐴 = ∑& #$! 𝑃[𝐴# ].
• Theorem 1.5 The probability of an event 𝐵 = {𝑠!, 𝑠", … , 𝑠& } is the
sum of the probabilities of the outcomes contained in the event:
𝑃 𝐵 = ∑& #$! 𝑃 𝑠# .
Proof: Let 𝐵# = 𝑠# .
Note that 𝐵 = 𝐵! ∪ 𝐵" ∪ ⋯ ∪ 𝐵& and 𝐵# ∩ 𝐵' = ∅ for 𝑖 ≠ 𝑗.
Applying Theorem 1.4 yields Theorem 1.5.
EE313 Probability for EE -- Murat Saraçlar 4
Some Theorems that follow from the Axioms
• Theorem 1.3 For mutually exclusive events 𝐴! and 𝐴" (𝐴! ∩ 𝐴" = ∅),
𝑃[𝐴! ∪ 𝐴"] = 𝑃[𝐴!] + 𝑃[𝐴"].
Proof: Let 𝐵! = 𝐴!, 𝐵" = 𝐴" and 𝐵# = ∅ for 𝑖 > 2.
Note that 𝐵# ∩ 𝐵' = ∅ for 𝑖 ≠ 𝑗 and let 𝐵 = 𝐵! ∪ 𝐵" ∪ 𝐵( ∪ ⋯
Axiom 3 says 𝑃 𝐵 = 𝑃 𝐵! + 𝑃 𝐵" + ⋯ = 𝑃 𝐵! + ∑% #$" 𝑃[𝐵# ]
Now note that 𝐵" ∪ 𝐵( ∪ ⋯ = 𝐵"
Axiom 3 says 𝑃 𝐵" = 𝑃 𝐵" + 𝑃 𝐵( + ⋯ = ∑% #$" 𝑃[𝐵# ]
which gives us 𝑃 𝐵 = 𝑃 𝐵! + 𝑃 𝐵"
Finally 𝐵 = 𝐵! ∪ 𝐵" = 𝐴! ∪ 𝐴" so that 𝑃[𝐴! ∪ 𝐴"] = 𝑃[𝐴!] + 𝑃[𝐴"].
EE313 Probability for EE -- Murat Saraçlar 5
Comments on Notation
• 𝑃 ⋅ : the probability of an event
• 𝑃[𝐴] can be viewed as a function that transforms event 𝐴 to a
number between 0 and 1.
• {𝑠# }: a set with a single element 𝑠# .
• For convenience, we will sometimes write 𝑃[𝑠# ] instead of 𝑃[{𝑠# }]
• We will also abbreviate the notation for the probability of the
intersection of two events, 𝑃[𝐴 ∩ 𝐵]. Sometimes we will write it as
𝑃[𝐴, 𝐵] and sometimes as 𝑃[𝐴𝐵].
Thus by definition, 𝑃[𝐴 ∩ 𝐵] = 𝑃[𝐴, 𝐵] = 𝑃[𝐴𝐵].

EE313 Probability for EE -- Murat Saraçlar 6

Equally Likely Outcomes
• Theorem 1.6: For an experiment with sample space S = {s1 , . . . , sn }
in which each outcome si is equally likely,
1
𝑃 𝑠# = , 1 ≤ 𝑖 ≤ 𝑛.
𝑛

EE313 Probability for EE -- Murat Saraçlar 7

Equally Likely Outcomes
• Theorem 1.6: For an experiment with sample space S = {s1 , . . . , sn }
in which each outcome si is equally likely,
1
𝑃 𝑠# = , 1 ≤ 𝑖 ≤ 𝑛.
𝑛
• Proof:
• Since all outcomes have equal probability,
there exists 𝑝 such that 𝑃[𝑠! ] = 𝑝 for 𝑖 = 1, … , 𝑛.
• Theorem 1.5 implies
𝑃 𝑆 = 𝑃 𝑠" + ⋯ + 𝑃 𝑠# = 𝑝 + ⋯ + 𝑝 = 𝑛𝑝
• Since Axiom 2 says 𝑃[𝑆] = 1, we must have 𝑝 = 1/𝑛.

EE313 Probability for EE -- Murat Saraçlar 8

Example 1.13
Roll a six-sided die in which all faces are equally likely.
• What is the probability of each outcome?
• Find the probabilities of the events:
• “Roll 4 or higher,”

• “Roll an even number,”

• “Roll the square of an integer.”

EE313 Probability for EE -- Murat Saraçlar 9

Example 1.13
Roll a six-sided die in which all faces are equally likely.
• What is the probability of each outcome? 𝑃[𝑖] = 1/6, 𝑖 = 1,2, … , 6.
• Find the probabilities of the events:
• “Roll 4 or higher,”
𝑃[Roll 4 or higher] = 𝑃[4] + 𝑃[5] + 𝑃[6] = 1/2
• “Roll an even number,”
𝑃[Roll an even number] = 𝑃[2] + 𝑃[4] + 𝑃[6] = 1/2
• “Roll the square of an integer.”
𝑃[Roll the square of an integer] = 𝑃[1] + 𝑃[4] = 1/3

EE313 Probability for EE -- Murat Saraçlar 10

Theorem 1.7
The probability measure 𝑃 · satisfies
(a) 𝑃[∅] = 0.

(b) 𝑃[𝐴) ] = 1 − 𝑃[𝐴].

(c) For any 𝐴 and 𝐵 (not necessarily disjoint),

𝑃[𝐴 ∪ 𝐵] = 𝑃[𝐴] + 𝑃[𝐵] − 𝑃[𝐴 ∩ 𝐵]

(d) If 𝐴 ⊂ 𝐵, then 𝑃[𝐴] ≤ 𝑃[𝐵].

EE313 Probability for EE -- Murat Saraçlar 11

Theorem 1.8
For any event 𝐴, and event space {𝐵!, 𝐵", … , 𝐵& },
𝑃 𝐴 = ∑& #$! 𝑃[𝐴 ∩ 𝐵# ]

EE313 Probability for EE -- Murat Saraçlar 12

Theorem 1.8
For any event 𝐴, and event space {𝐵!, 𝐵", … , 𝐵& },
𝑃 𝐴 = ∑& #$! 𝑃[𝐴 ∩ 𝐵# ]
• Proof: The proof follows directly from Theorem 1.2 and Theorem 1.4.
Note that 𝐶# = {𝐴 ∩ 𝐵# } are disjoint sets.

EE313 Probability for EE -- Murat Saraçlar 13

Example 1.14
• Consider the following table of probabilities:
Voice Data Fax
Long 0.30 0.12 0.15
Brief 0.20 0.08 0.15

•𝑃 Long = 𝑃 Long, Voice + 𝑃 Long, Data + 𝑃 Long, Fax = 0.57.

•𝑃 Brief = 𝑃 Brief, Voice + 𝑃 Brief, Data + 𝑃 Brief, Fax = 0.43.
•𝑃 Voice = 𝑃 Long, Voice + 𝑃 Brief, Voice = 0.50.
•𝑃 Data = 𝑃 Long, Data + 𝑃 Brief, Data = 0.20.
•𝑃 Fax = 𝑃 Long, Fax + 𝑃 Brief, Fax = 0.30.

EE313 Probability for EE -- Murat Saraçlar 14