Statistics Ch 2: Probability Page 2-1

SAMPLE SPACE

<def> The set of all possible outcomes of a statistical experiment is called the
sample space and is represented by the symbol S.

 Each outcome in a sample space is called a sample point.

 A process that generates a set of data is called an experiment.
 Sample spaces with a large or infinite number of sample points are best described
by a statement or rule such as:
 Example: S={x | x is a city with a population over 1 million}
S  {( x, y ) | x 2  y 2  4} .
 Example: Tossing a die.  S1  {1,2 ,3,4 ,5,6} or S 2  {even, odd }
Which one is better?  In general, it is desirable to use a sample space that gives
the most information concerning the outcomes of an experiment.

EVENTS

Definitions:

<def> If each element of a set A1 is also an element of set A2, then A1 is a subset of
A2, A1  A2 .
<def> If a set A has no elements, then A is called null set, A=  .
<def> An event, A, is a subset of a sample space, i.e. A  S .
<def> The complement of an event A with respect to S is the subset of S that are not
in A. The complement of A is denoted by A' , Ac or A .

Ch 2-1
Statistics Ch 2: Probability Page 2-2

 Example: Tossing two fair coins in succession

S={HH, HT, TH, TT}
E= at least one head ={HH, HT, TH}
F= at least one tail ={HT, TH, TT}
E’= no head ={TT}
F’= no tail ={HH}
 Example: Lifetime of a car
S=[0,  )
E= between 2 to 7 years =(2, 7)
F= between 3 to 10 years=(3, 10)

ALGEBRA OF SETS:
1. Union: The union of events A and B, A  B , is the event containing all elements
that belong to A or B or both. A  B  x: x  A or x  B
2. Intersection: The intersection of events A and B, A  B , is the event containing
all elements that are common to A and B. A  B  x: x  A and x  B

 Example: Tossing two fair coins in succession

E  F = at least one head or one tail={HH, HT, TH, TT}
E  F = at least one head and one tail={HT, TH,}

<def> Two events A and B are mutually exclusive or disjoint if A  B  , i.e., A

and B have no elements in common.
3. The difference or relative complement of two events A and B is the set of
elements which belong to A but not to B, i. e., A \ B  x: x  A and x  B .

4. For any set A,   A.

5. Identity Laws: A    , A    A, A  S  A, A  S  S .
6. Commutative Laws: A  B  B  A, A  B  B  A.
7. Associative Laws: ( A  B)  C  A  ( B  C),( A  B)  C  A  ( B  C)

Ch 2-2
Statistics Ch 2: Probability Page 2-3

8. Distributive Laws:
A  ( B  C)  ( A  B)  ( A  C), A  ( B  C)  ( A  B)  ( A  C)
9. De Morgan's Law: ( A  B)'  A'  B',( A  B)'  A'  B'
10. Complement Laws: A  A'  S , A  A'  ,( A')'  A, S '  , '  S .
11. If A  B and B  A , then A=B.
12. If A  B and B  C , then A  C .

Venn Diagram:

A B

COUNTING SAMPLE POINTS

Generalized multiplication rule:
If an experiment consists of a sequence of k steps in which there are n1 possible

results for the first step, n2 possible results for the second step, and so on, then

the total number of experimental outcomes is given by n1n2 n3  nk .

 Example: How many 3-digit even numbers can be constructed by 1, 2, 5, 6, 9

(each number can be used only one time)?
個位 十位 百位
1 5
2 5 6
6 9

6
9
1
2
...
5
9
2 4 3 = 24

Ch 2-3
Statistics Ch 2: Probability Page 2-4

 Example: (Bradley Investments)

Bradley has invested in two stocks, Markley Oil and Collins Mining. Bradley
has determined that the possible outcomes of these investments three months
from now are as follows.
Investment Gain or Loss in 3 Months (in $000)
Markley Oil Collins Mining
10 8
5 -2
0
-20

=> Total Number of Experimental Outcomes: n1n2  (4)(2)  8 .

 Tree Diagram:

Gain 8 (10, 8) Gain $18,000

(10, -2) Gain $8,000

Lose 2
Gain 10 Gain 8 (5, 8) Gain $13,000

(5, -2) Gain $3,000

Gain 5 Lose 2
Gain 8
(0, 8) Gain $8,000
Even
(0, -2) Lose $2,000
Lose 2
Lose 20 Gain 8
(-20, 8) Lose $12,000

Lose 2 (-20, -2) Lose $22,000

Ch 2-4
Statistics Ch 2: Probability Page 2-5

Permutation: (排列)
A permutation is an arrangement of all or part of a set of objects.

I. The number of permutations of n distinct objects is n! where 1!=1 and 0!=1.

Example: How many 5-digit numbers can be constructed by 1, 2, 5, 6, 9 ?

 5  4  3  2 1  5! 120

II. The number of permutations of n distinct objects taken r at a time is

n!
n Pr  n  (n  1)(n  r  1) 
(n  r )!
 Example: How many 2-digit numbers can be constructed by 1, 2, 5, 6, 9 ?
5  4  3  2 1 5!
   20
3  2 1 3!

III. The number of permutations of n distinct objects arranged in a circle is (n-1)!.

IV. The number of distinct permutations of n things of which n1 are of one kind, n2

n!
of a second kind, ..., n k of a k-th kind is .
n1!n2! nk !
 Example: How many 5-digit numbers can be constructed by 1, 1, 5, 5, 9?
5  4  3  2  1 5!
   30
(2  1)( 2  1) 2!2!

Partitioning: (分組)
The number of ways of partitioning a set of n objects into k cells with n1 elements

in the first cell, n2 elements in the second cell, and so forth, is

 n  n!
   .
 1 2
n , n ,  , n k  n1!n2 !nk !

Ch 2-5
Statistics Ch 2: Probability Page 2-6

Note: (i) The relationship between II and partitioning is

 n  n! n!
P   
 r, n  r  r!  r! 
r!(n  r )! (n  r )!
n r
 
(ii) The IV and partitioning are basically the same.

Combination: (組合)
The number of combinations of n distinct objects taken r at a time is
n n!
Crn    
 r  r!(n  r )!
Note: A combination is actually a partition with two cells, the one cell containing
the r objects selected and the other cell containing the (n-r) objects that are
left.

 Example: 1. How many outcomes when we draw 2 cards from a bridge deck?

 52  52! 52  51
C252       1326
 2  2!50! 1 2
2. How many outcomes that we have a black jack?

C116C14  64
3. What is the probability to have a black jack?
64
P{black jack }   0.048
1326

 Example: Lottery（小樂透）
The number of combinations of 42 numbers taken 6 at a time is

 42  42! 42  41  40  39  38  37
C642       5,245 ,786
 
6 6!36! 1  2  3  4  5  6

 36 
特：C136     36 Ch 2-6
1
Statistics Ch 2: Probability Page 2-7

頭獎 1
C56C136 6  36
二獎  6
C136 36
三獎
C56C136  6  6  36  6  210
152 ,467
  2 .9 %
四獎 C 6C 36  15  630  9450 5,245 ,786
4 2

普獎
C36C336  20  7140  142 ,800

Assigning Probabilities:
• Classical Method
Assigning probabilities based on the assumption of equally likely outcomes.
• Relative Frequency Method
Assigning probabilities based on experimentation or historical data.
• Subjective Method
Assigning probabilities based on the assignor’s judgment.

0  P( Ei )  1
Basic Requirement:  .
 P ( E1 )  P ( E 2 )    P ( E n )  1
1 Certain

Classical Method:
If an experiment has n possible outcomes, this method
would assign a probability of (1/n) to each outcome. 0.5

 Example: Rolling a fair die

Experiment => Rolling a die
Sample Space => S = {1, 2, 3, 4, 5, 6} 0 Impossible
Probabilities => Each sample point has a 1/6 chance of occurring.

Ch 2-7
Statistics Ch 2: Probability Page 2-8

Relative Frequency Method:

If an experiment is repeated a large number of times, the data can be used to
estimate the proportion of time the outcome will occur.
# of defectives/100 frequency Probability
0 4 0.10 = 4/40
1 6 0.15 = 6/40
2 18 0.45 = 18/40
3 10 0.25= 10/40
4 2 0.05= 2/40
40 1.00

Subjective Method:
• When economic conditions and a company’s circumstances change rapidly it
might be inappropriate to assign probabilities based solely on historical data.
• We can use any data available as well as our experience and intuition, but
ultimately a probability value should express our degree of belief that the
experimental outcome will occur.
• The best probability estimates often are obtained by combining the estimates
from the classical or relative frequency approach with the subjective estimates.

 Example: (Bradley Investments)

Applying the subjective method an analyst made the following probability
assignments.
Exper. Outcome Net Gain/Loss Probability
( 10, 8) $18,000 Gain .20
( 10, -2) $8,000 Gain .08
( 5, 8) $13,000 Gain .16
( 5, -2) $3,000 Gain .26
( 0, 8) $8,000 Gain .10
( 0, -2) $2,000 Loss .12
(-20, 8) $12,000 Loss .02
(-20, -2) $22,000 Loss .06

Ch 2-8
Statistics Ch 2: Probability Page 2-9

PROBABILITY OF AN EVENT
• An event is a collection of sample points. (A subset of the sample space.)
• The probability of any event is equal to the sum of the probabilities of the
sample points in the event.

<def> The probability of the event E is equal to the sum of the probabilities of the
sample points in the event E. Therefore,
0  P( E )  1, P( )  0, P( S )  1 .

<def> If an experiment can result in any one of N different equally likely outcomes,
and if exactly n of these outcomes correspond to event A, then the probability
n
of event A is P A  .
N

 Example: Tossing two fair coins in succession

P(S)=1,
P(E)=P{at least one head}=P{HH, HT, TH}=3/4
P(F)=P{at least one tail}=P{TT, HT, TH}=3/4

 Example: In a poker hand consisting of 5 cards, find the probability of

holding 2 aces and 3 jacks.
 52 52 !
Total # of outcomes: N      2598960 ,
 5  5! 47 !

 4  4
The # of ways to hold 2 aces and 3 jacks: n       24
 2  3

n 24
 P  A    9  10  6
N 2598960

Ch 2-9
Statistics Ch 2: Probability Page 2-10

SOME BASIC RELATIONSHIPS OF PROBABILITY:

Complement of an Event （互補事件）:
• The complement of event A is defined to be the event consisting of all
sample points that are not in A. The complement of A is denoted by Ac .
• P( Ac )  P( A)  1

Union of Two Events（聯集）:

• The union of events A and B is the event containing all sample points that
are in A or B or both.
• P( A  B)  P( A)  P( B)  P( A  B) .

Intersection of Two Events:（交集）

• The intersection of events A and B is the set of all sample points that are in
both A and B.
• P( A  B)  P( A)  P( B)  P( A  B) .

Ch 2-10
Statistics Ch 2: Probability Page 2-11

Mutually Exclusive Events:（互斥事件）

Two events are said to be mutually exclusive（互斥） if the events have no
sample points in common. That is, two events are mutually exclusive if, when
one event occurs, the other cannot occur.

Addition Law for Mutually Exclusive Events: P( A  B)  P( A)  P( B)

In general:
P( E1  E2  En )   P( Ei )   P( Ei  E j )   P( Ei  E j  Ek )
i i j i jk
.
n 1
 ( 1) P( E1  E2  En )

 n  n
 If 1 2
E , E ,  , E n are mutually exclusive, then P  Ei    P( Ei ) .
 i 1  i 1
 If A  B , then P( A)  P( B) .

A partition of a sample space:

If A1 , A2 ,  , An is a partition of a sample space S, then they are mutually
exclusive and P( A1  A2  An )   P( Ai )  P( S )  1 .
i

Sample Space S
A n A1
A2
A3

Ch 2-11
Statistics Ch 2: Probability Page 2-12

CONDITIONAL PROBABILITY: （條件機率）

• The probability of an event given that another event has occurred is called a
conditional probability.
• The conditional probability of A given B is denoted by P(A|B).
• A conditional probability is computed as follows:
P( A  B) if
P( A | B)  P( B)  0 .
P( B)

 Revise Original Sample Space to Account for New Information (Eliminates

Certain Outcomes)

 Example: Tossing two fair coins in succession

P{HT,TH} 2/4 2
P(E|F)=P{at least one head|at least one tail}=  
P{TT,HT,TH} 3 / 4 3

Independent Events（獨立事件）:
• Events A and B are independent if P( A | B)  P( A) .

• Multiplication Law for Independent Events: P( A  B)  P( A) P( B)

• The multiplication law also can be used as a test to see if two events are
independent.
• That is, the knowledge that A ( or B) has occurred does not affect the
probability that B (or A) occurs. Otherwise, A and B are dependent.

Ch 2-12
Statistics Ch 2: Probability Page 2-13

Multiplicative Rules:

1. P( A  B)  P( B) P( A| B)  P( A) P( B| A) .
2.
P ( E1  E 2  E n )  P( E1 ) P ( E 2 | E1 ) P ( E 3 | E1  E 2 )
.
 P( E n | E1  E 2  E n 1 )

3. If E1 , E 2 , , E n are independent, then

P( E1  E2    En )  P( E1 ) P( E2 ) P( E3 ) P( En )

Note: Events E1, E2 ,  , En are said to be independent if for every subset

E1' , E2' ,, Er ' , r  n of these events P( E1' E2' Er ' )  P( E1' ) P( E2' ) P( Er ' ) .
The knowledge of the occurrence of any of these events does not affect the
probability of any other event.

 Example: Tossing two fair dice in succession

E=sum equals to 6 ==> P(E)=5/36
F=sum equals to 7 ==> P(F)=6/36
G=the first die is 4 ==> P(G)=1/6
=> P(EG)=1/36  P(E)P(G)=5/216, =>E and G are dependent.
=> P(FG)=1/36=P(F)P(G)=1/36, =>F and G are independent.

Example: S={1,2,3,4},E={1,2},F={1,3},G={1,4}
P(EF)=1/4=P(E)P(F) =>E and F are independent.
P(EG)=1/4=P(E)P(G) =>E and G are independent.
P(FG)=1/4=P(F)P(G) =>F and G are independent.
But, P(EFG)=1/4  P(E)P(F)P(G)=1/8 =>E, F and G are dependent.
Events E, F and G are pairwise independent but not jointly independent.

Ch 2-13
Statistics Ch 2: Probability Page 2-14

 Example: Suppose n men throw their hats in a bag and each randomly selects
a hat. What is the probability that none has his own hat?

Let E i be the event that man i has his own hat. Then, P( Ei )  1 / n ,
1 (n  2)! (n  k )!
P( E i1  E i2 )   ,  , P( E i1  E i2  E ik )  .
n(n  1) n! n!
The probability that none has his own hat is
 n 
1  P  Ei   1   P( Ei )   P( Ei E j )   P( Ei E j E k ) ( 1) n P( E1E2 En )
 i 1  i i j i jk

1  n (n  2)!  n (n  k )!  n (n  n)!
 1 n    ( 1) k    ( 1) n  
n  2 n!  k  n!  n n!
1 1 1 1 n
    ( 1) n  e 1
2! 3! 4! n!

x x2 x3
NOTE: e  1  x   
2! 3!

BAYES’ RULE:
• Often we begin probability analysis with initial or prior probabilities（事前
機率）.
• Then, from a sample, special report, or a product test we obtain some
additional information.
• Given this information, we calculate revised or posterior probabilities （事
後機率）.
• Bayes’ theorem provides the means for revising the prior probabilities.

Application
Prior New Posterior
of Bayes’
Probabilities Information Probabilities
Theorem

Ch 2-14
Statistics Ch 2: Probability Page 2-15

Theorem of total probability:

 P( B)  P( B  A1 )  P( B  A2 )  P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )

Sample Space S
An A 1
A1 A2 B A2
B
A3
In general:
P( B)  P( B  A1 )  P( B  A2 )    P( B  An )
n n
  P( B  Ai )   P( Ai ) P( B | Ai )
i 1 i 1
 P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )    P( An ) P( B | An )

 Example:
New Information
Prior Probabilities Conditional Probabilities

P(B|A1) = .2 P(A1  B) = .14

P(A1) = .7 P(B) = . 41

P(Bc|A1) = .8 P(A1  Bc) = .56

P(B|A2) = .9 P(A2  B) = .27
P(A2) = .3

P(Bc|A2) = .1 P(A2  Bc) = .03

Ch 2-15
Statistics Ch 2: Probability Page 2-16

Bayes’ Theorem （貝氏定理）:

To find the posterior probability that event Ai will occur given that event B has

occurred we apply Bayes’ theorem.

P( Ai ) P( B | Ai )
P( Ai | B) 
P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )  ...  P( An ) P( B | An )

Bayes’ theorem is applicable when the events for which we want to compute
posterior probabilities are mutually exclusive and their union is the entire sample
space.

 Example:
P( A1 ) P( B | A1 ) (0.7)(0.2) 0.14
P( A1 | B)     0.34
P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 ) (0.7)(0.2)  (0.3)(0.9) 0.41
P( A2 ) P( B | A2 ) (0.3)(0.9) 0.27
P( A2 | B)     0.66
P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 ) (0.7)(0.2)  (0.3)(0.9) 0.41
 Example: In a multiple choice test,
p = the probability that a student knows the answer
1- p = the probability that a student guesses
m = the number of choices in each question
What is the probability that a student knew the answer given that he answered
it correctly?
Let E be the event that he answers the question correctly and F be the event
that he knows the answer.
1
Then, P( E )  P( F ) P( E | F )  P( F ' ) P( E | F ' )  p  1  (1  p) .
m
P ( FE ) P( F ) P( E | F ) p mp
P( F | E )    
P( E ) P( F ) P( E | F )  P( F ' ) P( E | F ' ) 1 1  (m  1) p
p  1  (1  p )
m

Ch 2-16
Statistics Ch 2: Probability Page 2-17

Decision Process:

Decision Process: a series of decisions Life can only be

understood going
backwards, but it must be
lived going forward. --
Kierkegaard

別靠錯梯子！

現在 未來
5年或10年後的你！

Ch 2-17