Professional Documents
Culture Documents
Stat20f Note1
Stat20f Note1
Probability Theory
1. Algebra of Sets
(1) Let denote the set of all elements under consideration; that is, is the universal set.
(2) For any two sets and , we will say that is a subset of , or is contained in
(denoted ⊂ ), if every point in is also in .
(4) If is a subset of , the complement of , denoted by , is the set of points that are
(5) The union of and , denoted by ∪ , is the set of all points in or or both;
∪ ∈ or ∈ .
(6) The intersection of and , denoted by ∩ , is the set of all points in both and ;
∩ ∈ and ∈
(7) The union of sets , , ⋯ , : ∪ ∪ ⋯ ∪
∈ for some ⋯
∞
The union of sets , , , ⋯ : ∪ ∪ ∪ ⋯
∈ for some ⋯
The intersection of sets , , ⋯ , : ∩ ∩ ⋯ ∩
∈ for all ⋯
∞
The intersection of sets , , , ⋯ : ∩ ∩ ∩ ⋯
∈ for all ⋯
∞
,
,
∞
1
2. Basics of Probability Theory
(2) The sample space is the set of all possible outcomes of a random experiment.
(4) If contains at most a countable number of elements, finite or countably infinite, we say
that is a discrete sample space. If contains an uncountable number of elements,
we say that is a continuous sample space.
(6) A function from a set to a set is a rule that assigns a unique (single) element
∈ to each element ∈ . The set of all possible input values is called the domain
of the function. The set of all output values of as varies throughout is called
the range of the function. When the range of a function is a set of real numbers,
the function is said to be real-valued. A set function is a real-valued function defined on
a collection of sets.
Probability is a set function that assigns, to each event in the sample space ,
a number , called the probability of the event , such that the following properties
are satisfied:
(a) ≥ ;
(b) ;
2
(c) if ⋯ are mutually exclusive events, then
∞ ∞
∪ ∪ ∪ ⋯
.
(countable additivity)
(9) A coin is tossed two times. The four elements of the sample space are
, , , .
If the coin is balanced, are equally likely outcomes. Hence we can assign equal
probabilities; that is, , . If the coin is unbalanced and ,
it does not seem appropriate to assign equal probabilities to elements of . Under certain
conditions it is reasonable to assign , , , .
(10) If is a continuous sample space, we cannot assign positive probability to each element
of without violating . In this case we can assign probabilities to compound
events consisting of intervals.
. Hence .
(12) ∅ .
Proof: Since ⊂ , ≤ ≤ .
Proof: The event ∪ can be expressed as a union of mutually exclusive events, namely,
3
∪ ∪ ∩ . Hence ∪ ∩ . However, ∩ ∪ ∩ ,
∪ ∩ , we obtain ∪ ∩ .
∪ ∪ ∩ ∩ ∩ ∩ ∩
∩ ∩ ∩
⋯ ∩ ∩ ⋯ ∩
3. Methods of Enumeration
(1) Let ⋯ be a finite sample space. If each has the same probability of occurring,
we say that the outcomes are equally likely. That is,
, ⋯ . If the number
,
(3) The number of possible outcomes of the rolling of a pair of dice and two tosses of a coin
is .
(4) Suppose that positions are to be filled with different objects. There are choices for
filling the first position, for the second, ⋯ , and choice for the last position. So,
by the multiplication principle, there are ⋯ possible arrangements.
Each of the arrangements of different objects is called a permutation of the objects.
(5) If only positions are to be filled with objects selected from different objects, ≤ , then
P ⋯ .
4
(6) If objects are selected from a set of objects, and if the order of selection is noted,
then the selected set of objects is called an ordered sample of size .
(7) Sampling with replacement occurs when an object is selected and then replaced
(8) By the multiplication principle, the number of possible ordered samples of size taken
(10) By the multiplication principle, the number of possible ordered samples of size taken
from a set of objects without replacement is P , the number of permutations of
Consider a class of students. The birthdays of these students form a sample of size
from the days in the year. We assume that every student has an equal probability of
being born on any day during the year, independent of everyone else, and ignore the
additional complication presented by leap years (that is, assume that nobody is born on
February 29). Then the probability that all birthdays are different is
P
⋯
.
(12) Let denote the number of unordered samples of size that can be selected from
different objects without replacement. We can obtain each of the P ordered subsets
by first selecting one of the unordered subsets of objects and then ordering these
objects. Since the latter ordering can be carried out in ways, the multiplication
principle yields ordered subsets; so must equal P . Thus, we have
P
C .
5
Each of the ( choose ) unordered samples is called a combination of objects taken
at a time. We could also say that the number of ways in which objects can be selected
(14) Binomial theorem
The terms are called binomial coefficients since they arise in the binomial expansion.
(15) An urn contains red, green blue, and white balls. A sample of size is selected at
random without replacement. The probability that the sample contains red, green,
.
⋮ ⋮ ⋮ ⋮ ⋮
The th row of this triangle gives the coefficients for . To find an entry in the
table other than a on the boundary, add the two nearest numbers in the row directly
above. The equation
(17) If we consider and , we can prove that
and .
6
(18) Suppose that a set contains objects of two types: of one type and of the other
type. The number of permutations of different objects is . However, in this case,
the objects are not all distinguishable. To count the number of distinguishable
arrangements, first select out of the positions for the objects of the first type. This
can be done in ways. Then fill in the remaining positions with the objects of the
(19) Suppose that in a set of objects, are of one kind, of a second kind, ⋯ , of
of the objects is
⋯
⋯
.
The terms ⋯
are often called multinomial coefficients because they occur
in the expansion of the multinomial term ⋯ raised to the th power.
⋯ ⋯ ⋯ ,
(20) To count the number of unordered samples of size selected from objects with
replacement, think of defining bins in which we can place stars. So if
and , a possible outcome is
* ** * ***
which says that zero ’s, one , two ’s, one and three ’s. That is, . In
the number of unordered samples of size that can be selected out of objects
when sampling with replacement, is
7
.
(21) Number of samples of size from objects
without with
replacement replacement
ordered
unordered
(22) Suppose that we are going to calculate all possible averages of numbers selected from
with replacement. The outcomes in the ordered and unordered sample spaces,
The probabilities come from considering the outcomes in the ordered sample space
to be equally likely. This corresponds to the common interpretation of sampling with
replacement; namely, one of the three items is chosen, each with probability ; the item
is noted and replaced; the items are mixed and again one of the three items is chosen,
each with probability . If we are only interested in the average of the sampled numbers,
the ordering is unimportant, and thus the total number of distinct samples is obtained by
counting according to unordered, with-replacement sampling. The total number of distinct
samples is and the outcomes in the unordered sample space are not
equally likely. To calculate the probability distribution of the sample averages, we must
count the different ways that a particular average can occur. It is possible that two
different unordered samples will result in the same average. For example, the unordered
8
samples and both result in an average value of . The first sample has
probability and the second has probability , giving the value probability .
average
probability .
4. Conditional Probability
(1) The conditional probability of an event , given that event has occurred, is defined by
∩
provided that .
(2) The conditional probability function ⋅ satisfies three axioms for a probability function
with .
(a) ≥
(b)
∞ ∞
(c)
,
where ⋯ are mutually exclusive events.
(3) An urn contains red, blue, and white balls. Four balls are to be selected at random
without replacement. The conditional probability of two red balls (event ) given that
at least one ball of each color (event ) is
∩
.
/
/
∩ ,
if and .
(5) The multiplication rule can be extended to three or more events. In the case of three
events, using the multiplication rule for two events, we have
9
∩ ∩ ∩ ∩ ∩ .
This type of argument can be used to extend the multiplication rule to more than three
events, and the following formula for events can be proved by mathematical induction.
∩ ∩ ⋯ ∩ ∩ ⋯ ∩ ∩ ⋯ ∩
(6) An urn contains balls of which are black and are white. At each trial a ball is
selected at random, its color is noted, and it is replaced along with two additional balls
of the same color. What is the probability that a black ball is selected in each of the first
three trials?
Solution: Let be the event that a black ball is selected on the th trial. Then
∩ ∩ ∩
5. Independent Events
(1) Two events and are independent if any of the following holds:
∩ ,
,
.
(2) A red die and a white die are rolled. Let on the red die , sum of dice is odd ,
and sum of dice is . Of the equally likely outcomes, , , ,
∩ and ∩ . Then
∩
and
≠ ∩ .
Hence and are independent events but and are dependent events.
(3) If and are independent events, then the following pairs of events are also independent:
(a) and ;
(b) and ;
10
(c) and .
∩
,
(4) Events ⋯ are independent if and only if the following conditions hold:
∩ ⋯ ∩ ⋯ .
(7) A rocket has a built-in redundant system. In this system, if component fails, it is
is used. Suppose that the probability of failure of any one component is , and assume
that the failures of these components are mutually independent events. Let denote the
event that component fails for . Because the system fails if fails and fails
and fails, the probability that the system does not fail is given by
∩ ∩ ∩ ∩
(8) A red die and a white die are rolled. Let {odd number on the red die},
{odd number on the white die}, and {sum of dice is odd}. Of the equally likely
outcomes, , ∩ for ≠ , and ∩ ∩ . Then
∩ for ≠ ,
and
∩ ∩ ≠
11
6. Bayes’ Theorem
∩ ∅ , for ≠ .
.
∩ ∩ ∪ ∪ ⋯ ∪ ∩ ∪ ∩ ∪ ⋯ ∪ ∩ .
Thus
∩ .
, ⋯ .
Proof: The proof follows directly from the definition of conditional probability and
the total probability theorem. Note that
∩
Remark : The probability is called the prior probability of and the conditional
(4) There are five urns, and they are numbered to . Each urn contains balls. Urn has
defective balls and nondefective balls, ⋯ . For instance, urn has three
defective balls and seven nondefective balls. Consider the following random experiment:
First an urn is selected at random, and then a ball is selected at random from the
12
selected urn. (The experimenter does not know which urn was selected.) Let us ask two
questions:
(b) If we have already selected the ball and noted that it is defective, what is the
probability that it came from urn ?
Solution: Let denote the event that a defective ball is selected and the event that
, ⋯ ,
and
, ⋯ .
.
.
Similarly,
13