PROBABILITY AND STATISTICS

LECTURE 3
PROBABILITY THEORY

Adapted from http://www.prenhall.com/mcclave

 OUTLINE

1. Basic concepts: Random Experiment, Outcome,

Event, Sample Space
2. Probabilities
3. Probability Rules
4. Conditional probability by Contingency Table and
Venn Diagram
5. Independent events and Multiplication rule
6. Bayes’s Theorem
 1. BASIC CONCEPTS

 Random Experiment: a process leading to an uncertain

outcome
 Sample Point = Basic Outcome: a possible outcome of a
random experiment
 Sample Space (S): the collection of all possible outcomes of a
random experiment
 Event: any subset of basic outcomes from the sample space
 EXAMPLE

Identify the sample space for the following random

experiments:

Random Experiment Sample Space

- Select 1 Card 2, 2, ..., A (52)

- Inspect a Part for Defective, OK
checking Quality
 PROPERTIES OF OUTCOMES IN SAMPLE
SPACE

1. Mutually Exclusive
Outcomes Cannot Occur at the Same Time

2. Collectively Exhaustive
Outcome in Sample Space Must Occur

3. Example
 EXERCISE

Identify the sample space and formulate some events for

the following random experiments:
 Toss a coin
 Roll 2 dice
 Randomly select 10 students from a population of 200 students
 VISUALIZING
SAMPLE SPACE

1. Listing: S = {Head, Tail}

2. Venn Diagram
3. Tree Diagram
4. Contingency table
 VENN DIAGRAM

Experiment: Toss 2 Coins.

Event
Tail
TH
HH HT
Outcome
TT
S
S = {HH, HT, TH, TT} Sample Space
 TREE DIAGRAM

Experiment: Toss 2 Coins.

H HH
H
T HT
Outcome
H TH
T
T TT
S = {HH, HT, TH, TT} Sample Space
 CONTINGENCY TABLE

Experiment: Toss 2 Coins

nd
2 Coin
st
1 Coin Head Tail Total
What is Outcome
this Head HH HT HH, HT (Count,
event? Total %
Tail TH TT TH, TT Shown
Usually)
Total HH, TH HT, TT S

S = {HH, HT, TH, TT} Sample Space

 COUNTING RULES

1- Permutation: The number of different ways that N

distinct things may be arranged in order is N! – N factorial.
N! = 1.2.3…(N-1).N where 0! = 1

2- Ordered Combination: The number of ways of

selecting and arrange r objects from among N distinct
objects.

For example: { a, b, c, d, e} selecting 3: abc, bcd, bca,…

 COUNTING RULES

3- Combination: The number of ways of

selecting and arrange r objects from among N
distinct objects. We are not interested in the order
of objects, but only in the number of ways that r
objects can be selected from among N.

For example: { a, b, c, d, e} selecting 3: abc, bcd,…

but not (bca)
 COMBINATION OF EVENTS

1. Intersection
 Outcomes in Both Events A and B
 ‘AND’ Statement
  Symbol (i.e., A  B)
2. Union
 Outcomes in Either Events A or B or Both
 ‘OR’ Statement
  Symbol (i.e., A  B)
 EVENT INTERSECTION EXAMPLE

Experiment: Draw 1 Card.

Black Event
Sample Black:
Space: 2B, ...,
2R, 2R, AB
2B, ..., AB
Ace S
Event Ace: Joint Event (Ace  Black):
AR, AR, AB, AB AB, AB
 EVENT UNION EXAMPLE

Experiment: Draw 1 Card.

Black Event
Sample Black:
Space: 2B,
2R, 2R, 2B,...,
2B, ..., AB AB
Ace S
Event Ace: Event (Ace  Black):
AR, AR, AB, AB AR, ..., AB, 2B, ..., KB
 MORE ON EVENTS

1. Complement of Event A
 Event consisting of All Sample points Not In A:
Ac
 Q: A  Ac = ?

2. Mutually Exclusive Events

 No basic outcomes in common
 EXERCISE

Discuss in pairs, and work out:

1. an example of complement of an event

2. an example of mutually exclusive events

 2. PROBABILITY

1. Numerical Measure

1 Certain
of Likelihood that Event
Will Occur
 P(Event)
 P(A)
 Prob(A) .5
2. Lies Between 0 & 1

0 Impossible
 PROBABILITY REQUIREMENTS

1. A is any event in sample space S, then

2. Let A be an event in S, and let Oi denote the basic

outcomes. Then

3.
 ASSIGNING PROBABILITY

What’s the
probability?
3 approaches to assigning probabilities to
experimental outcomes
 Classical
 Relative frequency
 Subjective
 CLASSICAL APPROACH

1. All outcomes are equally likely

Q: what is the probability of each
outcome?

2. P(Event) = X / T
© 1984-1994 T/Maker Co.

 X = No. of Event Outcomes

 T = Total Outcomes in Sample Space

3. Example
 RELATIVE FREQUENCY APPROACH

1. Actual Data
Collected Of 100 Parts
Inspected, Only
2. P(Event)  X / T 2 Defects!
Repeat Experiment

T Times
 Event Observed X Times
 Condition: T is large
3. Example
 SUBJECTIVE APPROACH

1. Individual opinion or © 1984-1994 T/Maker Co.

belief
2. May involve empirical
evidence and past
experience
3. Example
 3. PROBABILITY RULES

Addition Rule:

What if A and B are mutually exclusive?

Complement rule:
 EVENT PROBABILITY USING
CONTINGENCY TABLE

Event
Event B1 B2 Total
A1 P(A1  B1) P(A1  B2) P(A1)
A2 P(A2  B1) P(A2  B2) P(A2)
Total P(B1) P(B2) 1

Joint Probability Marginal Probability

 CONTINGENCY TABLE EXAMPLE

Experiment: Draw 1 Card.

Color
Type Red Black Total
Ace 2/52 2/52 4/52
Non-Ace 24/52 24/52 48/52 P(Ace)

Total 26/52 26/52 52/52

P(Red) P(Ace AND Red)

 EXERCISE

One unit is randomly selected from 10 units.

Using the Addition Rule, What’s the Probability?

P(A  D) =
Event
P(B  C) = Event C D Total
A 4 2 6
B 1 3 4
Total 5 5 10

Can you figure out the above probabilities

without using the addition rule?
 4. CONDITIONAL PROBABILITY

1. Event Probability Given that Another Event

Occurred
2. Revise Original Sample Space to Account for
New Information Outcomes
3.
 CONDITIONAL PROBABILITY USING
VENN DIAGRAM

Black ‘Happens’:
Eliminates All
Black
Other Outcomes

Ace
Black
S (S)
Event (Ace AND Black)
 CONDITIONAL PROBABILITY USING
CONTINGENCY TABLE

Experiment: Draw 1 Card.

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
 5. INDEPENDENT EVENTS

1. Event Occurrence
Does Not Affect
Probability of Another
Event: P(A | B) = P(A)
2. Tests For Independence
MULTIPLICATION RULE

 Q: What if the Events are Independent?

 PROBLEM

Suppose a survey of a random sample of 100 people shows the

following results:

 ‘

 QUESTIONS

A person is randomly selected from the 100

respondents.
a. What is the probability that this person will use
seat belt?
b. Find all the joint and marginal probabilities
c. Are the events ‘Having children’ and ‘Wearing
seat belt’ independent? (Use 2 methods)
d. Are the events ‘Having children’ and ‘Wearing
seat belt’ mutually exclusive?
e. Distinguish mutually exclusive and independent
events.
6. BAYES’ THEOREM
 Let A1, A2, ... , An be a set of mutually exclusive
and collectively exhaustive events.
 Let B be any event from the same sample space,
such that P(B) > 0. Then,
EXAMPLE
 Marie is getting married tomorrow, at an
outdoor ceremony in the desert. In recent
years, it has rained only 5 days each year.
Unfortunately, the weatherman has
predicted rain for tomorrow. When it
actually rains, the weatherman correctly
forecasts rain 90% of the time. When it
doesn't rain, he incorrectly forecasts rain
10% of the time. What is the probability
that it will rain on the day of Marie's
wedding?
EXAMPLE
 The sample space is defined by two mutually-
exclusive events - it rains or it does not rain.
 Additionally, a third event occurs when the
weatherman predicts rain.
EXAMPLE
 Event A1. It rains
 Event A2. It does not rain

 Event B. The weatherman predicts rain.

 EXAMPLE

 P( A1 ) = 5/365 =0.0136985
 [It rains 5 days out of the year.]

 P( A2 ) = 360/365 = 0.9863014

 [It does not rain 360 days out of the year.]

 P( B | A1 ) = 0.9 [When it rains, the weatherman predicts

rain 90% of the time.]
 P( B | A2 ) = 0.1 [When it does not rain, the weatherman
predicts rain 10% of the time.]
 EXAMPLE

 P( A1 ) = 5/365 =0.0136985
 P( A2 ) = 360/365 = 0.9863014

 P( B | A1 ) = 0.9

 P( B | A2 ) = 0.1
P(A1)P( B | A1 )
 P(A1 | B ) =
P(A1)P( B | A1 ) P(A2)P( B | A2 )
 EXAMPLE

P(A1)P( B | A1 )
 P(A1 | B ) =
P(A1)P( B | A1 ) P(A2)P( B | A2 )
 CONCLUSION

1. Basic concepts: Random Experiment, Outcome,

Event, Sample Space
2. Probabilities
3. Probability Rules
4. Conditional probability by Contingency Table and
Venn Diagram
5. Independent events and Multiplication rule
6. Bayes’s Theorem