Basics of Probability Theory

Dr. Gita A. Kumta

Probability as a Numerical Measure
of the Likelihood of Occurrence
0 1 .5
Increasing Likelihood of Occurrence
Probability:
The event
is very
unlikely
to occur.
The occurrence
of the event is
just as likely as
it is unlikely.
The event
is almost
certain
to occur.
Probability
Experiment of chance: a phenomena whose outcome is
uncertain.
Probabilities Chances
Probability Model
Sample Space
Events
Probability of Events
Sample Space: Set of all possible outcomes
Event: A set of outcomes (a subset of the sample space). An
event E occurs if any of its outcomes occurs.
Probability: The likelihood that an event will produce a
certain outcome.
An Experiment and Its Sample Space
An experiment is any process that generates
well-defined outcomes.
The sample space for an experiment is the
set of all experimental outcomes.
An experimental outcome is also called a
sample point.
A Counting Rule for
Multiple-Step Experiments
If an experiment consists of a sequence of k steps
in which there are n
1
possible results for the first step,
n
2
possible results for the second step, and so on,
then the total number of experimental outcomes is
given by (n
1
)(n
2
) . . . (n
k
).
A helpful graphical representation of a multiple-step
experiment is a tree diagram.

Govind Investments can be viewed as a
two-step experiment. It involves two stocks,
each with a set of experimental outcomes.
Mukund Oil: n
1
= 4
Collins Mining: n
2
= 2
Total Number of
Experimental Outcomes: n
1
n
2
= (4)(2) = 8
A Counting Rule for
Multiple-Step Experiments
A second useful counting rule enables us to count
the number of experimental outcomes when n
objects are to be selected from a set of N objects.
Counting Rule for Combinations
C
N
n
N
n N n
n
N
=
|
\

|
.
|
=

!
!( )!
Number of Combinations of N Objects Taken n at a Time
where: N! = N(N - 1)(N - 2) . . . (2)(1)
n! = n(n - 1)(n - 2) . . . (2)(1)
0! = 1
Number of Permutations of N Objects Taken n at a Time
where: N! = N(N - 1)(N - 2) . . . (2)(1)
n! = n(n - 1)(n - 2) . . . (2)(1)
0! = 1
P n
N
n
N
N n
n
N
=
|
\

|
.
|
=

!
!
( )!
Counting Rule for Permutations
A third useful counting rule enables us to count the
number of experimental outcomes when n objects are to
be selected from a set of N objects, where the order of
selection is important.
Assigning Probabilities
Classical Method
Relative Frequency Method
Subjective Method
Assigning probabilities based on the assumption
of equally likely outcomes
Assigning probabilities based on experimentation
or historical data
Assigning probabilities based on judgment
Classical Method
If an experiment has n possible outcomes, this method
would assign a probability of 1/n to each outcome.
Experiment: Rolling a die
Sample Space: S = {1, 2, 3, 4, 5, 6}
Probabilities: Each sample point has a
1/6 chance of occurring
Example
Relative Frequency Method
Number of
Polishers Rented
Number
of Days
0
1
2
3
4
4
6
18
10
2
Lucas Tool Rental would like to assign probabilities to the number
of car polishers it rents each day. Office records show the
following frequencies of daily rentals for the last 40 days.
Example: Lucas Tool Rental
Each probability assignment is given by dividing the
frequency (number of days) by the total frequency
(total number of days).
Relative Frequency Method
4/40
Probability
Number of
Polishers Rented
Number
of Days
0
1
2
3
4
4
6
18
10
2
40
.10
.15
.45
.25
.05
1.00
Subjective Method
When economic conditions and a companys circumstances
change rapidly it might be inappropriate to assign probabilities
based solely on historical data.
We can use any data available as well as our experience and
intuition, but ultimately a probability value should express our
degree of belief that the experimental outcome will occur.
The best probability estimates often are obtained by combining
the estimates from the classical or relative frequency approach
with the subjective estimate.
Events and Their Probabilities
The process of making an observation or recording a
measurement under a given set of conditions is a trial or
experiment.
Outcomes of an experiment are called events. An event is
a collection of sample points.
The probability of any event is equal to the sum of the
probabilities of the sample points in the event.
We denote events by capital letters A, B, C,
The probability of an event A, denoted by P(A), in general,
is the chance A will happen.

Probability
Consider a deck of playing cards





Sample Space: Set of 52 cards
Event: R: The card is red. F:The card is a face card.
A:The card is a heart. B:The card is a 3.
Probability: P(R) = 26/52 P(F) = 12/52
P(A) = 13/52 P(B) = 3/52

Events and variables
Can be described as random or deterministic:
The outcome of a random event cannot be predicted:
The sum of two numbers on two rolled dice.
The time of emission of the i
th
particle from
radioactive material.
The measured length of a table to the nearest cm.
Motion of macroscopic objects (projectiles, planets, space
craft) as predicted by classical mechanics.
The outcome of a deterministic event can be predicted:
Random variables
Can be described as discrete or continuous:
A discrete variable has a countable number of values.
Number of customers who enter a store before one purchases
a product.
The values of a continuous variable can not be listed:
Distance between two oxygen molecules in a room.
Random Variable Possible Values
Gender Male, Female
Class Fresh, Soph, Jr, Sr
Height (inches) # in interval {30,90}
College Arts, Education, Engineering, etc.
Shoe Size 3, 3.5 18
Consider data collected for undergraduate students:
Is the height a discrete or continuous variable?
Tree Diagram
A tree diagram is a way of describing all the
possible outcomes from a series of events
A tree diagram is a way of calculating the
probability of all the possible outcomes from a
series of events

Example a fair coin is flipped twice
H
H
H
T
T
T
HH
HT
TH
TT
2
nd
1
st

Possible
Outcomes
Outcome Table
If you flip a coin twice, you can model also model
the results with an outcome table
Flip 1 Flip 2 Simple
Event
H H HH
H T HT
T H TH
T T TT
Tree Diagrams For flipping a coin
Probability of two or more events
1
st
Throw 2
nd
Throw
T H H H H H T T T T
1/2 1/2 1/2 1/2 1/2 1/2 1/2
OUTCOMES
H,H
H,T
T,H
T,T
P(Outcome)
P(H,H) =1/4=1/2x1/2
P(H,T) =1/4=1/2x1/2
P(T,H) =1/4=1/2x1/2
P(T,T) =1/4=1/2x1/2
Total P(all outcomes) = 1
Total=4 (2x2)

Rules for calculating probability

The addition rule
for pairwise mutually exclusive events
P(A
1
+ A
2
+ ...+A
n
)= P(A
1
)+P(A
2
)+ ...+P(A
n
)
for two non-mutually exclusive events A and B
P(A+B) = P(A) + P(B) P(AB).
Multiplicative rule
P(AB) = P(A) P(B|A) = P(B) P(A|B).
Formula of total probability
P(B)= P(A
1
)P(B|A
1
)+P(A
2
)P(B|A
2
)+ ...+P(A
n
)P(B|A
n
).

Some Basic Relationships of Probability
There are some basic probability relationships that can be used to
compute the probability of an event without knowledge of all the
sample point probabilities.
Complement of an Event
Intersection of Two Events
Mutually Exclusive Events
Union of Two Events
The complement of A is denoted by A
c
.
The complement of event A is defined to be the event
consisting of all sample points that are not in A.
Complement of an Event
Event A A
c
Sample
Space S
Venn
Diagram
The union of events A and B is denoted by A B.
The union of events A and B is the event containing
all sample points that are in A or B or both.
Union of Two Events
Sample
Space S
Event A Event B
The intersection of events A and B is denoted by A B.
The intersection of events A and B is the set of all
sample points that are in both A and B.
Sample
Space S
Event A Event B
Intersection of Two Events
Intersection of A and B
The addition law provides a way to compute the
probability of event A, or B, or both A and B occurring.
Addition Law
The law is written as:



P(A B) = P(A) + P(B) P(A B)
Mutually Exclusive Events
Two events are said to be mutually exclusive if the
events have no sample points in common.
Two events are mutually exclusive if, when one event
occurs, the other cannot occur.
Sample
Space S
Event A Event B
Mutually Exclusive Events
If events A and B are mutually exclusive, P(A B) = 0.
The addition law for mutually exclusive events is:



P(A B) = P(A) + P(B)
theres no need to
include P(A B)
The probability of an event given that another event
has occurred is called a conditional probability.
A conditional probability is computed as follows :





The conditional probability of A given B is denoted
by P(A|B).
Conditional Probability
( )
( | )
( )
P A B
P A B
P B

=
Multiplication Law
The multiplication law provides a way to compute the
probability of the intersection of two events.
The law is written as:



P(A B) = P(B)P(A|B)
Independent Events
If the probability of event A is not changed by the
existence of event B, we would say that events A
and B are independent.
Two events A and B are independent if:



P(A|B) = P(A) P(B|A) = P(B)
or
The multiplication law also can be used as a test to see
if two events are independent.
The law is written as:



P(A B) = P(A)P(B)
Multiplication Law
for Independent Events
Bayes Theorem
New
Information
Application
of Bayes
Theorem
Posterior
Probabilities
Prior
Probabilities
Often we begin probability analysis with initial or
prior probabilities.
Then, from a sample, special report, or a product
test we obtain some additional information.
Given this information, we calculate revised or
posterior probabilities.
Bayes theorem provides the means for revising the
prior probabilities.
A proposed shopping center will provide strong
competition for suburban businesses like
L. S. Clothiers. If the shopping center is built, the owner of L.
S. Clothiers feels it would be best to relocate to the center.
Bayes Theorem
Example: L. S. Clothiers
The shopping center cannot be built unless a zoning change
is approved by the town council. The planning board must
first make a recommendation, for or against the zoning
change, to the council.
Prior Probabilities
Let:
Bayes Theorem
A
1
= town council approves the zoning change
A
2
= town council disapproves the change
P(A
1
) = .7, P(A
2
) = .3
Using subjective judgment:
New Information
The planning board has recommended
against the zoning change. Let B denote the
event of a negative recommendation by the
planning board.
Given that B has occurred, should L. S.
Clothiers revise the probabilities that the
town council will approve or disapprove the
zoning change?
Bayes Theorem
Conditional Probabilities
Past history with the planning board and the town
council indicates the following:
Bayes Theorem
P(B|A
1
) = .2 P(B|A
2
) = .9
P(B
C
|A
1
) = .8 P(B
C
|A
2
) = .1
Hence:
P(B
c
|A
1
) = .8
P(A
1
) = .7
P(A
2
) = .3
P(B|A
2
) = .9
P(B
c
|A
2
) = .1
P(B|A
1
) = .2
P(A
1
B) = .14
P(A
2
B) = .27
P(A
2
B
c
) = .03
P(A
1
B
c
) = .56
Bayes Theorem
Tree Diagram
Town Council

Planning Board

Experimental
Outcomes
Bayes Theorem
1 1 2 2
( ) ( | )
( | )
( ) ( | ) ( ) ( | ) ... ( ) ( | )
i i
i
n n
P A P B A
P A B
P A P B A P A P B A P A P B A
=
+ + +
To find the posterior probability that event A
i
will occur given
that event B has occurred, we apply Bayes theorem.
Bayes theorem is applicable when the events for
which we want to compute posterior probabilities
are mutually exclusive and their union is the entire
sample space.
Posterior Probabilities
Given the planning boards recommendation not to
approve the zoning change, we revise the prior
probabilities as follows:
1 1
1
1 1 2 2
( ) ( | )
( | )
( ) ( | ) ( ) ( | )
P A P B A
P A B
P A P B A P A P B A
=
+
=
+
(. )(. )
(. )(. ) (. )(. )
7 2
7 2 3 9
Bayes Theorem
= .34
Conclusion

The planning boards recommendation is good news
for L. S. Clothiers. The posterior probability of the
town council approving the zoning change is .34
compared to a prior probability of .70.
Bayes Theorem