Basics of Probability

Basics of Probability
UCT102: Probability and Statistics 1

Probability
• Sample Spaces and Events
• Probability
• The Axioms of Probability
• Some Elementary Theorems
• Conditional probability
• Bayes’ Theorem

Sample Space and Events
• A set of all possible outcomes of an experiment is
called a sample space. It is denoted by S.
• A particular outcome, i.e. an element in S is called a
sample point.
• An experiment consists of the simple process of
noting outcomes.
• The outcome of an experiment may be a simple
choice between two alternatives; it may be result of a
direct measurement or count; or it may be answer
obtained after extensive measurements and
calculations.

(cont’d)
• A sample space is said to be discrete if it has
finitely many or a countable (denumerable)
infinity of elements.
• If the elements of a sample space constitute a
continuum, the sample space is said to be
continuous. For example, all points on a line, all
points on a line segment, or all points in a plane.
• Any subset of a sample space is called an
event.
• The subset {a} consisting of a single sample
point a  S is called an elementary event.
 and S itself are subsets of S and so are events.
(cont’d)
Example 1: A technician has to check the
suitability of 3 solid crystal lasers and 2 CO 2
lasers for a given task.
Let (a,b) represent the event that the technician
will find a of the solid crystal lasers and b of the
CO2 lasers suitable for the task, then the sample
space will be
S = {(0,0), (0,1), (0,2), (1,0), (1,1), (1,2), (2,0),
(2,1), (2,2), (3,0), (3,1), (3,2)}.

(cont’d)
R = {(0,0), (1,1), (2,2)}
is the event that equally many solid crystal lasers
and CO2 lasers are suitable.
T = {(0,0), (1,0), (2,0), (3,0)}
is the event that none of the CO2 lasers are suitable
for the task.
U = {(0,1), (0,2), (1,2)}
is the event that fewer solid crystal laser than the
CO2 lasers are suitable for the task.

(cont’d)
If two events have no element in common, they are called
mutually exclusive events, i.e. they cannot occur
simultaneously.
In above example R &T are not mutually exclusive events
while R & U and T & U are mutually exclusive events.
In order to discuss the basic concepts of the probabilistic
model which we wish to develop, it will be very
convenient to have available some ideas and concepts of
mathematical theory of sets.
Sample spaces and events, particularly relationship
among events are depicted by means of Venn diagrams

(cont’d)
If A and B are any two sets in a sample space S,
A B = {x  S | x  A or x  B (or both)};
AB = {x  S | x  A and x  B};
The complement of A i.e. A  {x  S | x  A}.
A \ B = A – B = {x  S | x  A and x  B}
The symmetric difference of sets A and B i.e.
A  B = A \ B  B \ A = A B - AB
A set is called empty set and denoted by  which has
no elements.

(cont’d)
Venn diagrams
S S
A A
A A
(cont’d)
Venn diagrams
S S
A B A B
AB AB

(cont’d)
Venn diagrams
S S
A B A B
A\B or A - B AB

(cont’d)
RU = {(0,0), (1,1), (2,2), (0,1), (0,2), (1,2)}
is the event that the number of suitable CO2 lasers is
grater than or equal to the number of suitable solid
crystal lasers.
RT = {(0,0)}
is the event that neither CO2 laser nor solid crystal laser
is suitable.
T = {(0,1), (0,2), (1,1), (1,2), (2,1), (2,2), (3,1), (3,2)}
is the event that at least one CO2 laser is suitable.

Probability
The classical probability concept
If there are n equally likely possibilities, of
which one must occur and s are regarded as
favorable, or as a “success”, then the
probability of a “success” is given by s / n.
Ex: If a card is drawn from a well shuffled deck
of 52 playing cards, then find probability of
drawing
(a) a red king, (b) a 3, 4, 5 or 6, (c) a black
card (d) a red ace or a black queen.
Ans: (a) 1/26, (b) 4/13, (c) 1/2, (d) 1/13.

Probability (cont’d)
A major shortcoming of the classical probability
concept is its limited applicability.
There are many situations in which the various
possibilities cannot all be regarded as equally
likely.
For example, if we are concerned with the
question of whether it will rain the next day,
whether a missile launching will be a success, or
whether a newly designed engine will function for
at least 1000 hours.
Probability (cont’d)
The frequency interpretation of
probability:
The probability of an event (or outcome)
is the proportion of times the event
would occur in a long run of repeated
experiments.
If the probability is 0.78 that a plane from
Mumbai to Goa will arrive on time, it
means that such flights arrive on time
78% of the time.
if weather service predicts that there is a
40% chance for rain this means that
under the same weather conditions it
will rain 40% of the time.
• In the frequency interpretation of
probability, we estimate the probability
of an event by observing what fraction
of the time similar event have occurred
in the past.

The Axioms of probability
We define probabilities mathematically as the
values of additive set functions.
f : A  B, A : domain of f
If the elements of the domain of the function are
sets, then the function is called Set function.
Ex: Consider a function n that assigns to each
subset A of a finite sample space S the number
of elements in A, i.e.
n : P( S )  N

The Axioms of probability (cont’d)
A set function is called additive if the number
which it assigns to the union of two subsets
which have no element in common is sum of
the numbers assigned to the individual
subsets.
In above example n is additive set function;
that is
n( A  B )  n( A)  n( B ) if A  B  

Let S be a sample space, let C be the class of
all events and let P be a real-valued function
defined on C. Then P is called a probability
function and P(A) is called the probability of
event A when the following axioms hold:
Axiom 1 0  P(A)  1 for each event A in S.
Axiom 2 P(S) = 1.
Axiom 3 If A and B are mutually exclusive
events in S, then
P(AB) = P(A) + P(B).

Ex: If an experiment has the three possible and
mutually exclusive outcomes A, B and C,
check in each case whether the assignment of
probabilities is permissible:
a) P(A) = 1/3, P(B) = 1/3 and P(C) = 1/3;
b) P(A) = 0.64, P(B) = 0.38 and P(C) = –0.02;
c) P(A) = 0.35, P(B) = 0.52 and P(C) = 0.26;
d) P(A) = 0.57, P(B) = 0.24 and P(C) = 0.19.
Ans: a) Y, b) N, c) N, d) Y
Some Elementary Theorems
Generalization of third axiom of probability
Theorem 1: If A1, A2,….,An are mutually
exclusive events in a sample space, then
P(A1 A2….An) = P(A1) + P(A2) + …+ P(An).
Rule for calculating probability of an event
Theorem 2: If A is an event in the finite sample
space S, then P(A) equals the sum of the
probabilities of the individual outcome
comprising A.

(cont’d)
Proof: If E1, E2,……En be the n outcomes
comprising event A, then A = E1E2 ……  En.
Since the E’s are individual outcomes they are
mutually exclusive, and by Theorem 1, we have
P(A) = P(E1E2 ……  En)
= P(E1) + P(E2) + …+ P(En).
General addition rule for probability
Theorem 3: If A and B are any events in S, then
P(AB) = P(A) + P(B) – P(AB).
(cont’d)
Note: When A and B are mutually exclusive so
that P(AB) = 0, Theorem 3 reduces to the third
axiom of probability therefore the third axiom of
probability also called the special addition rule
Probability rule of the complements
Theorem 4: If A is any event in S, then
P( A ) = 1 – P(A).

(cont’d)
Proof: Since A and A are mutually exclusive by

definition and A  A = S. Hence we have
P(A) + P( A ) = P (A  A ) = P(S) = 1.
P(  ) = 1 – P(  ) = 1 – P( S ) = 0.
If A  B then P(B\A) = P(B) - P(A)
P(A  B) = P(A) + P(B) - 2 P(AB)

Problem 1: The probability that a new airport
will get an award for its design is 0.16, the
probability that it will get award for the
efficient use of materials is 0.24 and the
probability that it will get both awards is
0.11.
(a) What is the probability that it will get at least
one of the two awards ?
(b) What is the probability it will get only one of
the two awards?

Solution: Let A is the events that the airport will
get an award for its design and B is the events
that the airport will get an award for efficient
use of material.
(a) P(AB) = P(A) + P(B) - P(AB)
= 0.16 + 0.24 – 0.11 = 0.29.
(b) P(AB) = P(A) + P(B) - 2P(AB)
= P(AB) - P(AB) = 0.29 - 0.11
= 0.18.
Conditional Probability
• If we ask for the probability of an event then it is
meaningful only if we mention about the sample
space.
• When we use the symbol P(A) for probability of
A, we really mean the probability of A with
respect to some sample space S.
• Since there are problems in which we are
interested in probabilities of A with respect to
more sample spaces than one, the notation P(A|
S) is used to make it clear that we are referring to
a particular sample space S.
Conditional Probability (cont’d)
P(A|S)  conditional probability of A relative
to S.
Conditional probability: If A and B are any
events in S and P(B)  0, the conditional
probability of A given B is
P( A  B)
P( A | B) 
P( B)
P (A|B) is the probability that event A occurs once

event B has occurred
Reduced Sample Space
AB
S
B A
P(A|B) measures the relative probability of A with

respect to the reduced sample space B

If A and B are any two events in the sample space S,
Then the event A is independent of the event B if and
only if
P(A|B) = P(A)
i.e. occurrence of B does not influence the occurrence
of A.
But B is independent of A whenever A is independent of
B.
 A and B are independent events if and only if
either P(A|B) = P(A) or P(B|A) = P(B)
General multiplication rule of probability
Theorem 5: If A and B are any events in S, then
P(AB) = P(A) · P(B|A) if P(A)  0
= P(B) · P(A|B) if P(B)  0
Special product rule of probability
Theorem 6: Two events A and B are independent
events if and only if
P(AB) = P(A) · P(B)

The mutually exclusive events are not independent
unless one of them has zero probability.
If an event A is independent of itself then P(A) = 0 or
P(A) = 1
If the events A and B are independent, then so are
events A and B, events A and B ,, and events A and
B
.

Problem 2: With reference to the problem 1,
find the probabilities that the airport will get
the design award given that
(a) it got the award for the efficient use of
materials;
(b) It did not get the award for the efficient use
of materials.
Solution: Given P(A) = 0.16, P(B) = 0.24,
P(AB) = 0.11

P( A  B ) 0.11
(a ) P( A | B)    0.458.
P( B) 0.24
P ( A  B ) 0.05
( b) P ( A | B )    0.066
P( B ) 0.76
Since
P ( B )  1  P( B )  1  0.24  0.76
and
P ( A  B )  P( A \ B)  P ( A)  P( A  B)
 0.16  0.11  0.05

Bayes’ Theorem
Let S be a sample space and B1, B2,….Bn be mutually
exclusive events such that
S = B1B2 …… Bn
and A be an event in the sample space S. Then
A = AS = A(B1B2 …… Bn)
= (A B1)  (A B2) ……. (A Bn).
Since all A  Bi ’s are mutually exclusive events
P(A)= P(AB1) + P(AB2) +……. + P(ABn).

Bayes’ Theorem (cont’d)
n
or P( A)   P( A  Bi )
i 1
But from multiplication rule for probability
P(ABi) = P(Bi)·P(A|Bi), for i = 1, 2, …, n
hence we have
n
P ( A)   P ( Bi )  P( A | Bi )
i 1

Rule of total probability
Theorem 7 : Let A be an event in a sample space S and
if B1, B2,……Bn are mutually exclusive events such that
S = B1B2 …… Bn and P (Bi)  0 for i = 1, 2, …, n,
then
n
P( A)   P( Bi )  P ( A | Bi )
i 1

Problem 3: A factory uses 3 machines X, Y and
Z to produce certain items. Suppose
machine X produces 50% of the items of
which 3% are defective, machine Y produces
30% of the items of which 4% are defective
and machine Z produces 20% of the items of
which 5% are defective. Find the probability
that a randomly selected item is defective.

Solution: Let D be the event that an item is
defective and X, Y and Z are events that
respective machine produce the item.
Given: P (X) = 0.50, P (D|X) = 0.03,
P (Y) = 0.30, P (D|Y) = 0.04, P (Z) = 0.20, P
(D|Z) = 0.05.
From rule of total probability
P (D) = P (X).P (D|X) + P (Y).P (D|Y)
+ P (Z).P (D|Z)
= 0.037

For k  1, 2,....., n
from the definition of
P ( Bk  A)
P ( Bk | A)  conditional probability
P ( A)
but according to multiplication rule of probability, we

have
P (Bk  A) = P(Bk)·P(A|Bk).
Hence, we have
P ( Bk )  P ( A | Bk )
P ( Bk | A) 
P ( A)
Using rule of total probability, we have following result.
Bayes’ Theorem
Theorem 8: Let A be an event in a sample space S and
if B1, B2,……Bn are mutually exclusive events such
that S = B1B2 …… Bn and P (Bi)  0 for i = 1, 2, …,
n, then
P ( Bk )  P ( A | Bk )
P ( Bk | A)  n
 P( Bi )  P( A | Bi )
i 1
for k = 1, 2,….., n.

This theorem provides a formula for finding the
probability that the “effect” A was “caused” by
the event Bk.
Note: Here, each P(Bk ) is called prior probability and
each P(Bk | A) is called posterior or inverse probability.

Problem 4: Consider the problem 3, suppose a
defective item is found among the output.
Find the probabilities that it came from
machine X, Y and Z respectively.
Solution: Recall P(D) = 0.037. Using Bayes’

theorem
P( X ).P( D | X ) (0.50).(0.03) 15
P( X | D)     0.405
P( D) .037 37

P(Y ).P ( D | Y ) (0.30).(0.04) 12
P (Y | D )     0.325
P( D) .037 37
P( Z ).P( D | Z ) (0.20).(0.05) 10
P( Z | D)     0.27
P( D) .037 37

Problem 5: Two firms V and W consider bidding
on a road-building job, which may or may not
be awarded depending on the amounts of the
bids. Firm V submits a bid and the probability
is 3/4 that it will get the job provided firm W
does not bid. The probability is 3/4 that W will
bid, and if it does, the probability that V will
get the job is only 1/3.
(a) What is the probability that V will get the job?
(b) If V gets the job, what is the probability that W
did not bid?

Solution: Let V is the event that the firm V will get
job and W is the event that the firm W will bid.
3 3
Given P (V | W )  , P (W ) 
4 4
1 1
and P (V | W )  , so P(W ) 
3 4
(a ) P(V )  P(W )  P(V | W )  P (W )  P(V | W )

3 1 13 7
     0.4375
4 3 4 4 16

P (W ) P (V | W )
( b) P (W | V ) 
P (V )
1 3

4 4 3
   0.4286
7 7
16

END

Basics of Probability

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Basics of Probability

Uploaded by

Copyright:

Available Formats

Basics of Probability

UCT102: Probability and Statistics 1

UCT102: Probability and Statistics 2

UCT102: Probability and Statistics 3

UCT102: Probability and Statistics 5

UCT102: Probability and Statistics 6

UCT102: Probability and Statistics 7

UCT102: Probability and Statistics 8

UCT102: Probability and Statistics 10

UCT102: Probability and Statistics 11

UCT102: Probability and Statistics 12

UCT102: Probability and Statistics 13

UCT102: Probability and Statistics 16

UCT102: Probability and Statistics 17

UCT102: Probability and Statistics 18

UCT102: Probability and Statistics 19

UCT102: Probability and Statistics 21

UCT102: Probability and Statistics 23

Proof: Since A and A are mutually exclusive by

UCT102: Probability and Statistics 24

UCT102: Probability and Statistics 25

P (A|B) is the probability that event A occurs once

P(A|B) measures the relative probability of A with

UCT102: Probability and Statistics 29

UCT102: Probability and Statistics 31

UCT102: Probability and Statistics 32

UCT102: Probability and Statistics 33

UCT102: Probability and Statistics 34

UCT102: Probability and Statistics 35

UCT102: Probability and Statistics 36

UCT102: Probability and Statistics 37

UCT102: Probability and Statistics 38

UCT102: Probability and Statistics 39

but according to multiplication rule of probability, we

UCT102: Probability and Statistics 41

UCT102: Probability and Statistics 42

Solution: Recall P(D) = 0.037. Using Bayes’

UCT102: Probability and Statistics 43

UCT102: Probability and Statistics 44

UCT102: Probability and Statistics 45

(a ) P(V )  P(W )  P(V | W )  P (W )  P(V | W )

UCT102: Probability and Statistics 46

UCT102: Probability and Statistics 47

UCT102: Probability and Statistics 48

You might also like