Math 231 Probability & Statistics 5/29/2012

Wei-Min Huang
Lehigh University
Chapter 2: Probability

Math 231

Wei-Min Huang
Summary
Probability is a way of quantifying uncertainty.
We are interested in the probability of an event - the likelihood
of the event occurring.
The probability of an event ranges from 0 to 1. The closer the
probability is to 0, the less likely the event is to occur. The
closer the probability is to 1, the more likely the event is to
occur.
There are two ways to determine probability: Theoretical
(Classical) and Empirical (Observational).
Theoretical methods use the nature of the situation to
determine probabilities.
Empirical methods use a series of trials that produce outcomes
that cannot be predicted in advance (hence the uncertainty).

2
Probability questions arise when we are faced with a situation
that involves uncertainty. Such a situation is called a random
experiment, an experiment that produces an outcome that cannot
be predicted in advance (hence the uncertainty).

Here are a few examples of random experiments:

Toss a coin once and record whether you get heads (H) or tails
(T). The possible outcomes that this random experiment can
produce are: {H, T}.
Toss a coin twice. The possible outcomes that this random
experiment can produce are: {HH, HT, TH, TT}.
Toss a coin 3 times. The possible outcomes in this case are:
{HHH, THH, HTH, HHT, HTT, THT, TTH, TTT}.
3 4
Basic Ideas
Definition: A random experiment is a process that
results in an outcome that cannot be predicted in
advance with certainty.
Examples:
rolling a die
tossing a coin
weighing the contents of a box of cereal

5
Sample Space
Definition: The set of all possible outcomes of an
experiment is called the sample space for the
experiment.

Examples:
For rolling a fair die, the sample space is {1, 2, 3, 4, 5, 6}.
For a coin toss, the sample space is {heads, tails}.
For weighing a cereal box, the sample space is (0, ), a more
reasonable sample space is (12, 20) for a 16 oz. box.

6
Probabilities
Each event in the sample space has a probability of
occurring. Intuitively, the probability is a quantitative
measure of how likely the event is to occur.

Given any experiment and any event A:
The expression P(A) denotes the probability that the
event A occurs.
P(A) is the proportion of times that the event A would
occur in the long run, if the experiment were to be
repeated over and over again.

Math 231 Probability & Statistics 5/29/2012
Wei-Min Huang
Lehigh University
Interpretations of Probability


Probability

Used to quantify likelihood or chance
Used to represent risk or uncertainty in
engineering applications
Can be interpreted as our degree of belief or
relative frequency
A Question

After doing this experiment, an important
question naturally comes to mind. How would
we know if the coin was not fair? Certainly,
classical probability methods would never be
able to answer this question. In addition,
classical methods could never tell us the actual
P(H). The only way to answer this question is
to perform another experiment.

8
So, these types of experiments can verify
classical probabilities and they can also
determine when games of chance are not
following fair practices. However, their real
importance is to answer probability questions
that arise when we are faced with a situation
that does not follow any pattern and cannot be
predetermined. In reality, most of the
probabilities of interest to us fit the latter
description.
9
Assigning Probabilities
Fair Coin expect that the probability of
getting a head on a flip is .5. We can say that
this is a common agreement for an assignment
of that probability.
Around 1900, Karl Pearson (a well-known
statistician) tossed a coin 24,000 times. 12,012
heads, a proportion of .5005.
Why didnt he get .5 for his answer?
Are coins always balanced?
0.4
0.5
0.6
10
1
10
2
10
3
10
4
Number of tosses
Figure 4.1.1 Proportion of heads versus number of tosses
for John Kerrich's coin tossing experiment.
From Chance Encounters by C.J. Wild and G.A.F. Seber, John Wiley & Sons, 2000.
Long run behavior of coin tossing
Try It! Let us conduct an experiment to see if we can find
the probability of getting a head when we spin a penny on
our table.
Each student spin their coin 10 times and record the
number of heads. When asked give that information to
the instructor.
What did you get?
Does these proportions converge to a limit?
The Law of Large Numbers



Math 231 Probability & Statistics 5/29/2012
Wei-Min Huang
Lehigh University
Law of Large Numbers (LLN)
Does the empirical relative frequencies
converge to a limit?
The LLN in probability shows that it DOES
under some conditions.
The limit is in fact the theoretical proportion
and it can be used as the probability of the
underlying event.
Central Limit Theorem (CLT)
Suppose that we want to answer the following:
What is the probability that the difference
(error) between the empirical relative
frequency and theoretical relative
frequency is at most 0.01?
Very often this requires another important
result in probability: Central Limit Theorem

The sequence of random experiments required is more complex and more time
consuming. This would be a good case to perform (or to simulate) the
experiments on a computer, Monte Carlo Simulation!

Toss a coin until you get the first tails (T). When we conduct this experiment,
one possible outcome is that we get T in the first toss and we are done.
Another possible outcome is that we get H in the first toss, toss a second time,
get T and be done. We might need three tosses until we get the first T, etc. The
possible outcomes of this random experiment are therefore: {T, HT, HHT,
HHHT, ...}. (Note that in this example the list of possible outcomes is not
finite.)

15
Craps Game
Win if Point Lose if 7 (Out 7) Continue otherwise
Continue otherwise Win if Point Lose if 7 (Out 7)
Win if 7, 11 Lose if 2, 3, 12 Set Point
Come Out Roll
Questions
What is the probability that the roller wins?
Note that this is not a simple problem.
The probability of win at later rolls depends on the point
value, e.g., if the point is 8, P(8)=5/36
({2,6},{3,5},{4,4},{5,3},{6,2}) and if the point is 10,
P(10)=1/36 ({5,5}).
How many rolls (on the average) will the game
last?
Exact
The exact value of probability of win can be
calculated:
( )
( ) 4929 . 0
1980
976
win
interval confidence 95% 0.4937 win 0.4921
= =
s s
P
P
Math 231 Probability & Statistics 5/29/2012
Wei-Min Huang
Lehigh University
Final Note
The game length is an
exponentially
distributed random
variable, however its
mean over N=1000
trials (as shown
before) is
approximately
normally distributed
(Central Limit
Theorem)
0 10 20 30 40
0
500
1000
1500
2000
2500
3000
3500
Sample Simulation
Trial win loss gamelength
1 0 1 2
2 1 0 7
3 1 0 3
4 0 1 2
5 1 0 5
6 0 1 10
7 1 0 8
8 0 1 2
9 0 1 4
10 1 0 1
Program Output
Total over 10 simulations
win loss gamelength
5 5 4.4
Summary Data
Generation of Useful Data
Remember we are using a random number generator!
We also need a larger sample size. We will run the problem
for many more games!
Let us now try 1000 games and divide #wins and #loss by M
to determine the probabilities (comment out the disp
commands to suppress output of data).
Also evaluate the mean of gamelengths.
Our point estimators are
P(win) = data(1000,1)/1000
P(loss) = data(1000,2)/1000
L = mean(data(:,3))
Sample Simulations (M=1000)

Run P(win) P(loss) L
1 0.479 0.521 3.646
2 0.496 0.504 3.774
3 0.497 0.503 3.351
4 0.467 0.533 3.450
5 0.512 0.488 3.386
6 0.485 0.515 3.435
7 0.480 0.520 3.476
8 0.494 0.506 3.591
9 0.492 0.508 3.466
10 0.495 0.505 3.312
Histograms
Histogram of P(win) for 1000 trials. Histogram of L for 1000 trials.
( ) { }
( ) { }
4 - 2
10 2.483 win var
4929 . 0 win
= o =
= =
p
p
P
P E
{ }
{ } 0.0083 var
3.3748
2
= o =
= =
L
L
L
L E
0.4 0.45 0.5 0.55 0.6
0
10
20
30
40
50
60
70
3 3.2 3.4 3.6 3.8
0
10
20
30
40
50
60
24
More Terminology
Definition: A subset of a sample space is called an
event.

For any sample space, the empty set C is an event, as
is the entire sample space.

A given event is said to have occurred if the outcome
of the experiment is one of the outcomes in the event.
For example, if a die comes up 2, the events {2, 4, 6}
and {1, 2, 3} have both occurred, along with every
other event that contains the outcome 2.

Math 231 Probability & Statistics 5/29/2012
Wei-Min Huang
Lehigh University
Once we have defined a random experiment, we can talk about an event of interest, which is a
statement about the nature of the outcome that we're actually going to get once the experiment is
conducted. Events are denoted by capital letters (other than S, which is reserved for the sample
space).
Example: Consider example 3, tossing a coin three times. Recall that the sample space in this case
is:
S = {HHH, THH, HTH, HHT, HTT, THT, TTH, TTT}

We can define the following events:
Event A: "Getting no H"
Event B: "Getting exactly one H"
Event C: "Getting at least one H"

Note that each event is indeed a statement about the outcome that the experiment is going to
produce. In practice, each event corresponds to some collection (subset) of the outcomes in the
sample space:

Event A: "Getting no H" TTT
Event B: "Getting exactly one H" HTT, THT, TTH
Event C: "Getting at least one H" HTT, THT, TTH, THH, HTH, HHT, HHH
25
Sample Spaces, Events, and Tree Diagrams

Tree Diagrams
Sample spaces can also be described graphically
with tree diagrams.
When a sample space can be constructed in several
steps or stages, we can represent each of the n
1
ways
of completing the first step as a branch of a tree.
Each of the ways of completing the second step can be
represented as n
2
branches starting from the ends of
the original branches, and so forth.
Sample Spaces and Events

Figure Tree diagram for three messages.
28
Experiment Toss a fair dice
Sample space
{1, 2, 3, 4, 5, 6}
Simple events
{1}, {2}, {3}, {4}, {5}, {6}
Some events
{1}, {5}, { 2, 4, 6}, {2, 3}, {1, 2, 3,5}
How many possible events are there?
Example
Experiment Toss a fair dice
Sample space = {1, 2, 3, 4, 5, 6}
List of all possible events
{ }
{1}, {2}, {3}, {4}, {5}, {6},
{1,2},{1,3},{1,4},{1,5},{1,6},{2,3},{2,4},{2,5},{2,6},{3,4},{3,5},{3,6},{4,5},{4,6},{5,6},

{1,2,3},{1,2,4},{1,2,5},{1,2,6},{1,3,4},{1,3,5},{1,3,6},{1,4,5},{1,4,6},{1,5,6},{2,3,4},{2,3,5},{2,3,6},{2,4,5},{2,4,6},{2,5,6},
{3,4,5},{3,4,6},{3,5,6},{4,5,6},
{1,2,3,4},{1,2,3,5},{1,2,3,6},{1,2,4,5},{1,2,4,6},{1,2,5,6},{1,3,4,5},{1,3,4,6},{1,3,5,6},{2,3,4,5},{2,3,4,6},{2,3,5,6},
{2,3,5,6},{3,4,5,6},

{1,2,3,4,5},{1,2,3,4,6},{1,2,3,5,6},{1,2,4,5,6},{1,3,4,5,6},{2,3,4,5,6},

{1,2,3,4,5,6}

There are 2^6 = 64 possible events altogether. (1+6+15+20+15+6+1=64)
Example
Math 231 Probability & Statistics 5/29/2012
Wei-Min Huang
Lehigh University
Pascal's Triangle 1+6+15+20+15+6+1 = 64 = 2^6

1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 10 10 5 1
1 6 15 20 15 6 1
1 7 21 35 35 21 7 1
How many possible subsets out of 6 distinct objects?
1+6+15+20+15+6+1 = 64 = 2^6
sample space = {1,2,3,4,5,6}
1 6 15 20 15 6 1
1 = only 1 empty subset (i.e., 1 subset of size 0)
6 = 6 different subsets of size 1
15 = 15 different subsets of size 2
20 = 20 different subsets of size 3
15 = 15 different subsets of size 4
6 = 6 different subsets of size 5
1 = only 1 with all 6 (i.e., 1 subset of size 6)



Experiment Toss a fair dice
Sample space = {1, 2, 3, 4, 5, 6}
There are 2^6 = 64 possible events altogether. (1+6+15+20+15+6+1=64)
interpretation of some events
Example
{2, 4, 6}
The event that an odd number is shown on the dice.
{1, 3, 5}
The event that an even number is shown on the dice.
{3, 4, 5, 6}
The event that at least a value of 3 is shown on the dice.
Experiment Toss a fair dice
Sample space = {1, 2, 3, 4, 5, 6}
There are 2^6 = 64 possible events altogether. (1+6+15+20+15+6+1=64)
interpretation of some events
Example
Which event is more likely to occur? {1, 3, 5} or {2, 4, 6}?
Is the event {1,2,3,4,5,6} always occurred?
Which event is more likely to occur? {1, 3, 5} or {1, 2, 4, 6}?
The sample space itself {1,2,3,4,5,6} is an event and it is a sure event.
The probability of this event is 1.
The sample space itself {1,2,3,4,5,6} is an event and it is
a sure event.
The probability of this event is 1.
The empty sunset { } is always an event and it is an
impossible or null event.
The probability of this event is 0.
So it is obviously then that the probability of any other event
Must be between 0 and 1.
What is probability?
36
Section 2.2: Counting Methods
The Fundamental Principle of Counting:
Assume that k operations are to be performed.
If there are n
1
ways to perform the first
operation, and if for each of these ways there
are n
2
ways to perform the second calculation,
and so on, then the total number of ways to
perform the sequence of k operations is

1 2 k
n n n
Math 231 Probability & Statistics 5/29/2012
Wei-Min Huang
Lehigh University
Counting Techniques

Multiplication Rule


38
Example 7
When ordering a certain type of computer, there
are 3 choices of hard drive, 4 choices for the
amount of memory, 2 choices of video card, and
3 choices of monitor. In how many ways can a
computer be ordered?
Permutations
Permutations

Sample Spaces and Events

Counting Techniques
Permutations : Example

Sample Spaces and Events

Counting Techniques
Permutations of Subsets
Sample Spaces and Events

Counting Techniques
Permutations of Subsets: Example
Math 231 Probability & Statistics 5/29/2012
Wei-Min Huang
Lehigh University
43
Permutations
A permutation is an ordering of a collection
of objects. The number of permutations of n
distinct objects is n!, where

Note: We define 0! = 1.
The number of permutations of k objects chosen
from a group of n objects is

When order matters, use permutations.

! ( 1)( 2) (3)(2)(1) n n n n =
!
( )!
n
n k
44
Example 8
1. Five people stand in line at a movie theater.
Into how many different orders can they be
arranged?

2. Five lifeguards are available for duty one
Saturday afternoon. There are three lifeguard
stations. In how many ways can three
lifeguards be chosen and ordered among the
stations?
Sample Spaces and Events

Counting Techniques
Permutations of Similar Objects: Example

46
Combinations
Combinations are an unordered collection of objects.

The number of combinations of k objects chosen from
a group of n objects is


The number of ways to divide a group of n objects
into groups of objects where
, is

!
!( )!
n
k n k
1
, ,
r
k k
1 r
k k n + + =
1
!
! !
r
n
k k
Sample Spaces and Events

Counting Techniques
Combinations: Example

48
Example 9
1. At a certain event, 30 people attend, and 5 will be
chosen at random to receive door prizes. The prizes
are all the same, so the order in which people are
chosen does not matter. How many different groups
of five people can be chosen?

2. A die is rolled 20 times. Given that three of the rolls
came up 1, five came up 2, four came up 3, two came
up 4, three came up 5, and three came up 6, how
many different arrangements of the outcomes are
there?