Introduction

CHAPTER 1
Introduction
Sample Space and Events
Basic definitions
• Random experiment:
Any activity whose outcome is not known in advance
with certainty each time it is repeated.
– Flipping a coin
– Tossing a die
– Product conformity
– Number of defectives in a sample
– Demand for a product
– Life of a product
Dr. Shokri Z. Selim 2
Basic definitions
• Sample Space, S: The set of all possible outcomes.
– Flipping a coin
– Tossing a die

Basic definitions
• Event, A: A subset of the sample space.
– Flipping a coin
– Tossing a die

Sets
• Set
• Subset E  F if and only if, if x  E , then x  F
• Equality: E  F if and only if E  F and F  E
• Intersection: E ∩ F is the set of elements that
belong to both E and F.

Sets
• Union: E ⋃ F is the set of elements that belong
to either E or F or both.

Sets
• A and B are Mutually Exclusive if: A ∩ B = Φ
• The Complement of E, Ec is the set of all
elements that are not in E.

Sets
(Ec⋂G)⋃F

Axioms of Probability
• Let S be the sample space of a random
experiment.
• Suppose that to each event A  S, a number
denoted by P(A), is associated with A.
• If P satisfies the following axioms, then it is
called a probability and the number P(A) is
said to be the probability of A.

Axioms of Probability
• P(A) ≥ 0 for any event A.
• P(S) = 1 ( S is the sample space. )
• If A ∩ B = Φ then P(A ⋃ B ) = P(A)+P(B)
• If A1, A2,…, A∞, is a set of mutually exclusive
events (Ai ∩ Aj = Φ for all i and j), then
P(A1 ⋃ A2 … A∞ ) = P(A1)+P(A2)+ … +P(A∞)

Elementary Properties of Probability
• P(Ac ) = 1 - P(A)
• P(Φ ) = 0
• P(A) ≤ 1
• P(A ⋃ B ) = P(A)+P(B) - P(A ∩ B )
• P(A1 ⋃ A2 … A n ) = P(A1)+P(A2)+ … +P(A n)

Probability of a Partitioned Event
• Suppose events A1, A2,… A n partition event A
Then
P(A ) = P(A1)+P(A2)+ … +P(A n)

Probability of union of events
• Consider events A, B, and C
Then
P(A ⋃ B ⋃ C ) = P(A)+P(B)+P(C)
– P(A⋂B) – P(A⋂C) –
P(B⋂C)
+ P(A⋂B ⋂C)
• Important result:
P(A ⋃ B ⋃ C ) ≤ P(A) + P(B) + P(C)
The set difference
• A – B = { x : x  A and x  B }
• P(A – B)?
A = (A – B) + (A∩B)
disjoint
P(A) =P(A – B) + P(A∩B)

P[A⋃B] = P[A]+ P[B]- P[A⋂B]
• A⋃B = (A – B) + B
disjoint
A⋃B = [ A – (A∩B) ] + B

Conditional Probability
P  A B
P  B | A  , P  A  0
P  A
P  A  B   P  B | A P  A 
 P  A | B  P B 
P  A  B C  D   P D | A  B C  P C | A  B  P B | A P  A

P[B|A] versus P[B]
Outcome Prob Outcome Prob Outcome Prob
O1 0.2 O1 0.2 O1 0.8
O2 0.3 O2 0.6 O2 0.2
O3 0.5 O3 0.2 O3 0.0
Event Prob Event Prob Event Prob
A={O1, O2} 0.5 A={O1, O2} 0.8 A={O1, O2} 1
B={O2, O3} 0.8 B={O2} 0.6 B={O2, O3} 0.2
P(A∩B) = 0.3 P(A∩B) = 0.6 P(A∩B) = 0.2
P(B|A) = 0.6 P(B|A) = 0.75 P(B|A) = 0.2
Independence
Events A and B are independent if any of the
following equivalent statements is true:
P(A|B) = P(A)
P(B|A) = P(B)
P(A⋂B) = P(A)P(B)

Exercise
Disks of polycarbonate plastic from a supplier are
analyzed for scratch and shock resistance. The results from 100 disks are
summarized as follows:
Let A denote the event that a disk has high shock resistance, and let B denote the event that a disk has
high scratch resistance. Are events A and B independent?

Independence
Independent events are events where knowing the outcome of one doesn't change
the probability of the other

Example
The circuit operates only if there is a path of
functional devices from left to right.
The probability that each device functions is
shown on the graph.
The devices fail independently.
Find the probability that the circuit operates.

Solution
• Let T denote the event that the top device

operates
• Let B denote the event that the bottom device
operates
Then
P( T or B) = P( T ) + P ( B ) - P( T and B)
= P( T ) + P ( B ) - P( T )P( B)
= 0.95*2 - 0.95*0.95 = 0.9975
Another Approach
P( T or B) = 1 – P ( T′ and B′)
= 1 – P(T′)P(B′)
= 1 – 0.05*0.05 = 0.9975

?

Independent Events
P[A∩B] = P[A] P[B] or P[A|B] = P[A]
If one of the following pairs of events is
independent then all the others are also
independent:
{A and B }
{A and Bc}
{Ac and B }
{Ac and Bc}.

jointly(mutually) Independent
Events
Events A1, A2, … , An are mutually (jointly)
independent if and only if the following relations hold for
all combinations of indices where 1 ≤ i < j < k < … ≤ n
P  Ai  A j   P  Ai  P  A j 
P  Ai  A j  Ak   P  Ai  P  A j  P  Ak 

P  A1  A2   An   P  A1  P  A2  P  An 
Jointly independent events
Source: Wikipedia
27
Pairwise independent but not jointly
(mutually) independent
Source: Wikipedia
28
Independent events
E1 and E2 are independent and

E2 and E3 are independent
True or false:
E1 and E3 are independent

Independent events
Outcome Prob Event Prob Event Prob
O1 0.4 A={O1, O2} 0.8 P(A∩B)=P(O2) 0.4
O2 0.4 B={O2, O3} 0.5 P(A∩C)=P(O1) 0.4
O3 0.1 C={O1, O3} 0.5 P(B∩C)=P(O3) 0.1
O4 0.1
A and B are independent
A and C are independent
However B and C are dependent

Independent events
for E1, E2 and E3
Pr[E1⋂ E2... ⋂ En] = Pr[E1] Pr[E2] … Pr[En]
True or false:
All pairs of events are independent

Independent events
Outcome Prob Event Prob
O1 0.7857 A={O1, O2} 0.7957
O2 0.01 B={O2, O3} 0.11
O3 0.1 C={O2, O4} 0.1143
O4 0.1043
P(A∩B∩C) = P(O2) = 0.01 = P(A)P(B)P(C)
P(A∩B) = P(O2) = 0.01 ≠ P(A)P(B)=0.0875
P(A∩C) = P(O2) = 0.01 ≠ P(A)P(C)=0.0909
P(B∩C) = P(O2) = 0.01 ≠ P(B)P(C)=0.0126

Posterior Probabilities
Law of Total Probability
S  {1, 2,3, 4,5, 6}
 S  A1    An and Ai : mutually exclusive
 B  ( A1  B )    ( An  B ) and ( Ai  B ) : mutually exclusive
 P ( B )  P ( A1  B )    P ( An  B )
 P ( A1 ) P ( B | A1 )    P ( An ) P ( B | An )

• Law of Total Probability

If A1 , A 2 ,..., An is a partition of a sample space, then the probability of an
event B can be obtained from the probabilities P ( Ai )
and P( B | Ai ) using the formula
P ( B )  P ( A1 ) P ( B | A1 )    P ( An ) P ( B | An )

Example
Machine A B C
% of total qA qB qC
production
Prob(defective) pA pB pC
P(defective part) = ?

Example

Example 2
In an experiment; E1, E2 and E3 are MEE.

The experiment is repeated until E1 or E2 occurs
Find P(E1 occurs before E2 )
Let A be the event E1 occurs

B be the event E2 occurs
C be the event E3 occurs

Example 3

Example 3
P ( A before B )  P ( E1 )  1  P ( E 2 )  0
 P( E3 )  P( A before B)
P ( E1 ) P ( E1 )
P ( A before B )  =
1 - P( E3 ) P( E1 )  P( E2 )
If P( E3 )  0; P( A before B )  P ( E1 )

• Example: Car Warranties
If A1 , A2 , A3 , and A4 are, respectively, the events that a car is assembled in
plants I, II, III, and IV, then they provide a partition of the sample space,
and the probabilities P ( Ai ) are the supply proportions of the four plants.
B = { a claim is made }
= the claim prob. for the four individual plants
P( B)  P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )  P( A3 ) P( B | A3 )  P( A4 ) P( B | A4 )
 (0.20  0.05)  (0.24  0.11)  (0.25  0.03)  (0.31 0.08)
 0.0687

Thomas Bayes (1701-1761)
Concerned with integrating

“prior knowledge” into
calculations of probability

Bayes’ Theorem
• Information is often presented in terms of
conditional probabilities
• Conditional probability = probability of an event
(failure) given a condition ( high or low
contamination
• An outcome is generated form a random
experiment >>>>>>>
What is the probability of that a condition
was present ( high contamination) given an
outcome ( failure)
Bayes’ Theorem
P( A  B )
P( A | B ) 
P( B )
P ( B | A) P ( A)
 for P ( B )  0
P( B )

Example 2-36 page 55
What is the conditional probability a high level of

contamination was present when a failure
occurred is to be determined.
Bayes’ Theorem
If E1 , E2 ,..., Ek are mutually exclusive and exhaustive
and B is any event,
P( B  E1 )
P( E 1| B )  for P( B )  0
P( B )
P( B | E1 ) P( E1 )

P( B | E1 ) P( E1 )  P( B | E2 ) P( E2 )  ...  P ( B | Ek ) P( Ek )

Example 32
In the flu season 3 viruses are known to cause flu.
70% of the population carry virus 1,
5% of them become sick,
20% carry virus 2,
20% of them become sick,
10% carry virus 3,
80% of them become sick.
If a person is sick which virus is likely to be the
cause?

Solution
70% carry virus 1 5% of them become sick
Vi = Event person carries virus i, i=1,2,3

S = Event person is sick
P(V1) = 0.7, P(V2) = 0.2, P(V3) = 0.1
P(S|V1) = 0.05, P(S|V2) = 0.2, P(S|V3) = 0.8

Solution
Vi = Event person carries virus i, i=1,2,3
S = Event person is sick
P(V1) = 0.7, P(V2) = 0.2, P(V3) = 0.1
P(S|V1) = 0.05, P(S|V2) = 0.2, P(S|V3) = 0.8
P ( S V1 )
P (V 1| S ) 
P( S )
P ( S | V1 ) P (V1 )

P ( S | V1 ) P (V1 )  P ( S | V2 ) P (V2 )  P ( S | V3 ) P (V3 )

Solution
Bayes’ formula gives:
P( virus 1 | person is sick) = 0.226
Is this answer intuitive?

Solution
Even though virus 1 is more common

and virus 3 is least common, the latter
is likely to be the cause of sickness.

Example 33
Part is Part is not

defective defective
0.0001 0.9999
Examinations
0.99 0.05
says defective
Examinations
says not defective 0.01 0.95
If a part is claimed defective, what is the probability

that it is really defective?
Solution
• Let D denote the event that a part is defective.
• Let S denote the event that inspection Part is Part is not
claims part is defective. defective defective
0.0001 0.9999
• Find P(D|S) ? Examinations 0.99 0.05
says defective
Examinations
P( D  S )
P( D | S )  says not
defective
0.01 0.95
P( S )
P( S | D) P( D)

P( S | D) P( D)  P( S | D c ) P( D c )
0.99 * 0.0001

0.99 * 0.0001  0.05 * 0.9999
Solution Using Trees
P( S | D ) P( D )
P( D | S ) 
P( S | D ) P( D )  P( S | D c ) P( D c )
0.99 * 0.0001
  0.001976
0.99 * 0.0001  0Z. Selim
Dr. Shokri .05 * 0.9999 53
Example 34
Printer failures are associated with hardware, software, or other problems
with probabilities 0.1, 0.6, and 0.3 respectively.
P(printer failure due to hardware problem) = 0.9

P(printer failure due to software problem) = 0.2
P(printer failure due to other problems) = 0.5
If a printer fails what is the probability it is caused by H/W?

Solution
Let H denote the event that the printer has a H/W problem.
Let S denote the event that the printer has a S/W problem.
Let O denote the event that the printer has other problems.
Let F denote the event that the printer fails.
P( HW  F )
P( HW | F ) 
P( F )
P( HW  F )

P( HW  F )  P( SW  F )  P(O  F )
P( F | HW ). P( HW )

( F | HW ). P ( HW )  P( F | SW ). P( SW )  P( F | O ). P(O )
P(HW) = 0.1, P(SW) = 0.6, P(O) = 0.3
P(F|HW) = 0.9, Dr.P(F|SW) = 0.7, P(F|O) = 0.5
Shokri Z. Selim 55
Solution
P(H|F)=1/4
P(S|F)=1/3
P(O|F)=5/12

Summary
 P( Ai ) and P( B | Ai )  P( Ai | B)  ?
 P( A1 ), , P( An ) : the prior probabilities
 P( A1 | B), , P( An | B) : the posterior probabilities
P( Ai  B ) P ( Ai ) P ( B | Ai ) P ( Ai ) P ( B | Ai )
 P ( Ai | B )    n
P( B) P( B)
 P(A ) P( B | A )
j 1
j j
• Bayes’ Theorem
If A1 , A 2 ,..., An is a partition of a sample space, then the posterior
probabilities of the event Ai conditional on an event B can be obtained
from the probabilities P ( Ai ) andP( B | Ai ) using the formula
P ( Ai ) P ( B | Ai )
P( Ai | B)  n
 P(A ) P( B | A )
j 1
j j

Additional Example
• Example: Car Warranties
- The prior probabilities
P( plant I )  0.20, P( plant II )  0.24
P( plant III )  0.25, P( plant IV )  0.31
- If a claim is made on the warranty of the car, how does this change these
probabilities?
P ( plant I ) P (claim | plant I ) 0.20  0.05
P ( plant I | claim)    0.146
P (claim) 0.0687
P ( plant II ) P (claim | plant II ) 0.24  0.11
P ( plant II | claim)    0.384
P (claim) 0.0687
P ( plant III ) P (claim | plant III ) 0.25  0.03
P ( plant III | claim)    0.109
P (claim) 0.0687
P ( plant IV ) P (claim | plant IV ) 0.31 0.08
P ( plant IV | claim)    0.361
P (claim) 0.0687

Additional Example cont.
- No claim is made on the warranty
P ( plant I ) P (no claim | plant I )

P ( plant I | no claim) 
P (no claim)
0.20  0.95
  0.204
0.9313
P ( plant II ) P (no claim | plant II )
P ( plant II | no claim) 
P (no claim)
0.24  0.89
  0.229
0.9313
P ( plant III ) P (no claim | plant III )
P ( plant III | no claim) 
P (no claim)
0.25  0.97
  0.261
0.9313
P ( plant IV ) P (no claim | plant IV )
P ( plant IV | no claim) 
P (no claim)
0.31 0.92
  0.306
0.9313
Compound Experiments
• Consider two random experiments Ea, and Eb.
• Sample space of Ea, Sa={ a1, a2, … am}
• Sample space of Eb, Sb={ b1, b2, … bn}
• A compound experiment is Ea×Eb.

Compound Experiments
• The sample space of Ea×Eb is the product set S.
• For discrete outcomes
 a1b1 a1b2  a1bn 
a b a b  a b 
 2 1 2 2 2 n
S  Sa  Sb   
     
 amb1 amb2  ambn 
• For continuous outcomes
S  Sa  Sb  ( a, b) : a  Sa and b  Sb 
Example
Two coins and three dice are tossed.

The compound experiment: E1×E2×E3×E4×E5
S = {H, T}×{H, T}×
{1,2,3,4,5,6}×{1,2,3,4,5,6}×{1,2,3,4,5,6}
There are 22× 63 outcomes

Introduction

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction

Uploaded by

Copyright:

Available Formats

CHAPTER 1

Dr. Shokri Z. Selim 3

Dr. Shokri Z. Selim 4

Dr. Shokri Z. Selim 5

Dr. Shokri Z. Selim 6

Dr. Shokri Z. Selim 7

Dr. Shokri Z. Selim 8

Dr. Shokri Z. Selim 9

Dr. Shokri Z. Selim 10

Dr. Shokri Z. Selim 11

Dr. Shokri Z. Selim 12

P(A) =P(A – B) + P(A∩B)

Dr. Shokri Z. Selim 14

Dr. Shokri Z. Selim 15

Dr. Shokri Z. Selim 16

Dr. Shokri Z. Selim 18

Dr. Shokri Z. Selim 19

Dr. Shokri Z. Selim 20

Dr. Shokri Z. Selim 21

• Let T denote the event that the top device

Dr. Shokri Z. Selim 23

Dr. Shokri Z. Selim 24

Dr. Shokri Z. Selim 25

E1 and E2 are independent and

Dr. Shokri Z. Selim 29

Dr. Shokri Z. Selim 30

All pairs of events are independent

Dr. Shokri Z. Selim 31

Dr. Shokri Z. Selim 32

Dr. Shokri Z. Selim 33

• Law of Total Probability

Dr. Shokri Z. Selim 34

Dr. Shokri Z. Selim 35

Dr. Shokri Z. Selim 36

In an experiment; E1, E2 and E3 are MEE.

Let A be the event E1 occurs

Dr. Shokri Z. Selim 37

Dr. Shokri Z. Selim 38

Dr. Shokri Z. Selim 39

Dr. Shokri Z. Selim 40

Concerned with integrating

Dr. Shokri Z. Selim 41

Dr. Shokri Z. Selim 43

What is the conditional probability a high level of

Dr. Shokri Z. Selim 45

Dr. Shokri Z. Selim 46

Vi = Event person carries virus i, i=1,2,3

Dr. Shokri Z. Selim 47

Dr. Shokri Z. Selim 48

Dr. Shokri Z. Selim 49

Even though virus 1 is more common

Dr. Shokri Z. Selim 50

Part is Part is not

If a part is claimed defective, what is the probability

P(printer failure due to hardware problem) = 0.9

If a printer fails what is the probability it is caused by H/W?

Dr. Shokri Z. Selim 54

Dr. Shokri Z. Selim 56

Dr. Shokri Z. Selim 57

Dr. Shokri Z. Selim 58

- No claim is made on the warranty

P ( plant I ) P (no claim | plant I )

Dr. Shokri Z. Selim 60