Professional Documents
Culture Documents
Handout1 PDF
Handout1 PDF
Department of Mathematics
1
1 Probability Models
Probability Model: A probability model consists of sample space and probability law.
• Probability law denoted by P(·): describes our belief which outcomes are more likely
than the others. Probability law satisfies certain axioms.
Sample Space Ω
- mutually exclusive: that is, if head comes in a coin tossing experiment then
tail doesn’t come and vice-versa (or there is a unique outcome).
- collectively exhaustive: no matter what happens in the experiment, we always
obtain an outcome that has been included in the sample space.
Discrete Sample Space: If the collection of outcomes is finite or countably infinite then
the sample space is called discrete.
Example 1.2. Two rolls of a tetrahedral die . The sample space can be represented by
2
Remark 1.1. Note that (1, 2) and (2, 1) denote different outcomes of the experiment. (1, 2)
denotes that there is a one on the first roll and two on the second roll.
Example 1.3. Number of tosses required to get the first head in a coin tossing experiment.
In this case Ω = {1, 2, . . .}
Continuous Sample Space
Ω = {(x, y) : 0 ≤ x, y ≤ 1}
Consider throwing a dart on the square target and viewing the point of impact as outcome.
The outcome of a single through is going to be a point in the region denoted by Ω. Since
there are infinitely many points, the sample space is of size infinite and is uncountable.
Definition 1.1 (Event). A subset of the sample space to which a probability is assigned is
called an event. Note that all subsets of the sample space are not necessarily considered as
events.
Probability Axioms are set of rules that every probabilistic model or probability law has
to follow (obey).
1. Non- negativity : P(A) ≥ 0.
2. Normalization : P(Ω) = 1.
Proposition 1.1. If A1 , A2 , . . . , An are disjoint sets, using probability axioms show that
n
X
P(A1 ∪ A2 ∪ . . . ∪ An ) = P(Ai ).
i=1
3
Proposition 1.2. If A ⊆ B, show that P(A) ≤ P(B).
4
Probability law (finite sample space). Consider two
rolls of a tetrahedral die. Let each possible outcome has
probability 1/16.
Then
1
P(w1 ) + P(w2 ) + · · · + P(wn ) = 1 =⇒ np = 1 =⇒ p =
n
=⇒ P(A) = P(w1 ) + P(w2 ) + · · · + P(wr ) =
r
n
.
5
Continuous Uniform Law. Probabilistic models with con-
tinuous sample space differ from their discrete counterparts in
that the probabilities of the single element events may not be sufficient to characterize the
probability.
P({x}) = p > 0, 0 ≤ x ≤ 1.
In this example, it make sense to assign probability b − a to any sub-interval [a, b] of [0, 1],
and to calculate probability of a more complicated set by evaluating its “length”.
This assignment satisfies the three probability axioms and hence qualifies as a legitimate
probability law.
(a) P(X + Y ≤ 1) = 21 .
Example 1.5. Consider the number of tosses required to get the first head in a coin filliping
experiment.
The first head could come in first, second, 100th or millionth toss. So it is appropriate to
choose Ω = {1, 2, . . . , n, . . . }. Suppose it is given that
P(n) = 2−n , n = 1, 2, . . . .
6
Now
(not valid)
P(even number of outcomes are required) = P({2, 4, 6, . . . }) = P(2) + P(4) + · · ·
1 1 1 1 1
= + 4 + ··· = 2 1 + 2 + 4 + ···
22 2 2 2 2
1 1 1
= 2 · 1 = .
2 1− 4 3
Countable sum is not valid by finite additivity axiom. Hence we need additional axiom to
make countable additivity as a valid step.
or ∞
X
P (∪∞
i=1 ) = P(Ai )
i=1
Proposition 1.4. Uniform-discrete-law is not applicable for countably infinite sample spaces.
1. If P(ωi ) = p > 0, ∀i, then by countable additivity axiom np > 1 for sufficiently large n,
hence contradiction.
P
2. Further, if P(ωi ) = 0, ∀ i, then i∈N 0 = 0, hence contradiction.
7
More on countable additivity. The concept of countable additivity is more subtle then it
looks. Consider the sample space given by Ω = {(x, y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1}. Define P(A) =
area(A) (which is a legitimate probability law). We have
X
1 = P(Ω) = P (∪x,y {(x, y)}) = P({x, y}) (countable additivity??)
x,y
X
= 0 (Since area of a point is 0)
x,y
= 0. (contradiction)
Note that there is no contradiction in above case when we assign 0 probability to indi-
viduals outcomes since countable additivity is not applicable because we are not dealing
with countable sets here.
Remark 1.2. A zero-probability event does not imply that the event can not occur, rather it
occur very infrequently given that the set of possible outcomes is infinite.
References
Dimitri Bertsekas and John N. Tsitsiklis (2008). Introduction to Probability, Athena
Scientific, 2nd edition.
8
Sheldon M. Ross (2009). Introduction to Probability and Statistics for Engineers and
Scientists, Academic Press.