Probability & Statistics for Scientist and Engineers

Dr. M. M. Bhatti

College of Mathematics and Systems Science, Shandong University of Science and

Technology, Qingdao, China
mmbhatti@sdust.edu.cn, mubashirme@yahoo.com

March 9, 2023

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 1 / 25

Outline

1 Introduction
Conditional Probability
Theorems on Conditional Probability
Conditional probability and independent events
Theorems on Conditional Probability
Independent Events

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 2 / 25

Introduction

Books:
1 Bluman, A. G. (2009). Elementary statistics: A step by step approach.
New York;: McGraw-Hill Higher Education.
2 Walpole, R. E., Myers, R. H., Myers, S. L., & Keying, E. Y. (2013). Es-
sentials of Probability & Statistics for Engineers & Scientists: Pearson
New International Edition PDF eBook. Pearson Higher Ed.
Marks strategy:

Class participation 40%

Final paper 60%

Class participation: Assignments+Attendance+Mid-term exam.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 3 / 25

Conditional Probability
Let A and B be two events (Fig. 1-3) such that P (A) > 0. Denote by
P (B|A) the probability of B given that A has occurred. Since A is known
to have occurred, it becomes the new sample space replacing the original
S. From this we are led to the definition
P (A ∩ B)
P (B|A) ≡ (1)
P (A)
or
P (B|A)P (A) ≡ P (A ∩ B) (2)

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 4 / 25

Conditional Probability

In words, (2) says that the probability that both A and B occur is equal to
the probability that A occurs times the probability that B occurs given that
A has occurred. We call P (B|A) the conditional probability of B given A,
i.e., the probability that B will occur given that A has occurred. It is easy
to show that conditional probability satisfies the axioms.
Example 1.12
Find the probability that a single toss of a die will result in a number less
than 4 if
a. no other information is given and
b. it is given that the toss resulted in an odd number.

(a) Let B denote the event {less than 4}. Since B is the union of the events
1, 2, or 3 turning up, we see by Theorem 1.5 that

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 5 / 25

Conditional Probability

1 1 1 1
P (B) = P (1) + P (2) + P (3) = + + = (3)
6 6 6 2
assuming equal probabilities for the sample points.
3 1
(b) Letting A be the event {odd number}, we see that P (A) = = .
6 2
2 1
Also, P (A ∩ B) = = . Then
6 3
P (A ∩ B) 1/3 2
P (B|A) = = = (4)
P (A) 1/2 3

Hence, the added knowledge that the toss results in an odd number raises
the probability from 1/2 to 2/3.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 6 / 25

Theorems on Conditional Probability

Theorem 1.9
For any three events A1 , A2 , A3 , we have

P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 |A1 )P (A3 |A1 ∩ A2 ) (5)

The above statement describes: the probability that A1 and A2 and A3 all
occur is equal to the probability that A1 occurs times the probability that
A2 occurs given that A1 has occurred times the probability that A3 occurs
given that both A1 and A2 have occurred. The result is easily generalized
to n events.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 7 / 25

Conditional probability and independent events
Problem 1.8
A fair die is tossed twice. Find the probability of getting a 4, 5, or 6 on the
first toss and a 1, 2, 3, or 4 on the second toss.

Let A1 be the event “4, 5, or 6 on first toss,” and A2 be the event “1, 2, 3,
or 4 on second toss.” Then we are looking for P (A1 ∩ A2 ).
Method 1:
P (A1 ∩ A2 ) = P (A1 )P (A2 |A1 ) = P (A1 )P (A2 )
  
3 4 1
= = (6)
6 6 3
We have used here the fact that the result of the second toss is inde-
pendent of the first so that P (A2 |A1 ) = P (A2 ). Also we have used
P (A1 ) = 3/6 (since 4, 5, or 6 are 3 out of 6 equally likely possibilities)
and P (A2 ) = 4/6 (since 1, 2, 3, or 4 are 4 out of 6 equally likely
possibilities).
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 8 / 25
Conditional probability and independent events

Method 2: Each of the 6 ways in which a die can fall on the first
toss can be associated with each of the 6 ways in which it can fall on
the second toss, a total of 6 · 6 = 36 ways, all equally likely.
Each of the 3 ways in which A1 can occur can be associated with each
of the 4 ways in which A2 can occur to give 3 · 4 = 12 ways in which
both A1 and A2 can occur. Then

12 1
P (A1 ∩ A2 ) = = (7)
36 3
This shows directly that A1 and A2 are independent since
  
1 3 4
P (A1 ∩ A2 ) = = = P (A1 )P (A2 ) (8)
3 6 6

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 9 / 25

Conditional probability and independent events
Problem 1.9
Find the probability of not getting a 7 or 11 total on either of two tosses of
a pair of fair dice.

The sample space for each toss of the dice is shown in Fig. 1-9. For example,
(5, 2) means that 5 comes up on the first die and 2 on the second. Since
the dice are fair and there are 36 sample points, we assign probability 1/36
to each.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 10 / 25

Conditional probability and independent events

If we let A be the event “7 or 11,” then A is indicated by the circled portion

in Fig. 1-9. Since 8 points are included, we have P (A) = 8/36 = 2/9. It
follows that the probability of no 7 or 11 is given by
2 7
P (A0 ) = 1 − P (A) = 1 − = (9)
9 9
Using subscripts 1, 2 to denote 1st and 2nd tosses of the dice, we see that
the probability of no 7 or 11 on either the first or second tosses is given by
  
0 0 0 0 0 7 7 49
P (A1 )P (A2 |A1 ) = P (A1 )P (A2 ) = = (10)
9 9 81

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 11 / 25

Conditional probability and independent events

Problem 1.10
Two cards are drawn from a well-shuffled ordinary deck of 52 cards. Find
the probability that they are both aces if the first card is (a) replaced, (b)
not replaced.

Method 1: Let A1 = event “ace on first draw” and A2 = event “ace on

second draw.” Then we are looking for P (A1 ∩ A2 ) = P (A1 )P (A2 |A1 ).
(a) Since for the first drawing there are 4 aces in 52 cards, P (A1 ) = 4/52.
Also, if the card is replaced for the second drawing, then P (A2 |A1 ) =
4/52, since there are also 4 aces out of 52 cards for the second drawing.
Then
  
4 4 1
P (A1 ∩ A2 ) = P (A1 )P (A2 |A1 ) = = (11)
52 52 169

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 12 / 25

Conditional probability and independent events
(b) As in part (a), P (A1 ) = 4/52. However, if an ace occurs on the first
drawing, there will be only 3 aces left in the remaining 51 cards, so
that P (A2 |A1 ) = 3/52. Then
  
4 3 1
P (A1 ∩ A2 ) = P (A1 )P (A2 |A1 ) = = (12)
52 51 221
Method 2: The first card can be drawn in any one of 52 ways, and since
there is replacement, the second card can also be drawn in any one of 52
ways. Then both cards can be drawn in (52)(52) ways, all equally likely.
(a) In such a case there are 4 ways of choosing an ace on the first draw
and 4 ways of choosing an ace on the second draw so that the number
of ways of choosing aces on the first and second draws is (4)(4). Then
the required probability is
(4)(4) 1
= (13)
(52)(52) 169
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 13 / 25
Conditional probability and independent events

(b) The first card can be drawn in any one of 52 ways, and since there is
no replacement, the second card can be drawn in any one of 51 ways.
Then both cards can be drawn in (52)(51) ways, all equally likely.
In such a case there are 4 ways of choosing an ace on the first draw
and 3 ways of choosing an ace on the second draw so that the number
of ways of choosing aces on the first and second draws is (4)(3). Then
the required probability is

(4)(3) 1
= (14)
(52)(51) 221

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 14 / 25

Conditional probability and independent events

Problem 1.11
Three balls are drawn successively from the box of Problem 1.7a . Find the
probability that they are drawn in the order red, white, and blue if each ball
is (a) replaced, (b) not replaced.
a
Problem 1.7: A ball is drawn at random from a box containing 6 red balls,
4 white balls, and 5 blue balls. Determine the probability that it is (a) red, (b)
white, (c) blue, (d) not red, (e) red or white.

Let R1 = event “red on first draw,” W2 = event “white on second draw,”

B3 = event “blue on third draw.” We require P (R1 ∩ W2 ∩ B3 ).

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 15 / 25

Conditional probability and independent events
(a) If each ball is replaced, then the events are independent and
P (R1 ∩ W2 ∩ B3 ) = P (R1 )P (W2 |R1 )P (B3 |R2 ∩ W2 )
= P (R1 )P (W2 )P (B3 )
   
6 4 5
=
6+4+5 6+4+5 6+4+5
8
= (15)
225
(b) If each ball is not replaced, then the events are dependent and
P (R1 ∩ W2 ∩ B3 ) = P (R1 )P (W2 |R1 )P (B3 |R1 ∩ W2 )
= P (R1 )P (W2 )P (B3 )
   
6 4 5
=
6+4+5 5+4+5 5+3+5
4
= (16)
91
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 16 / 25
Conditional probability and independent events
Problem 1.12
Find the probability of a 4 turning up at least once in two tosses of a fair
die.
Let A1 = event “4 on first toss” and A2 = event “4 on second toss.” Then
A1 ∪ A2 = event “4 on first toss or 4 on second toss or both”
= event “at least one 4 turns up,” (17)
and we require P (A1 ∪ A2 ).
Method 1: Events A1 and A2 are not mutually exclusive, but they
are independent. Hence, by (theorem 1.6) and by definition of
independent events,
P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) − P (A1 ∩ A2 )
= P (A1 ) + P (A2 ) − P (A1 )P (A2 )
  
1 1 1 1 11
= + − = (18)
6 6 6 6 36
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 17 / 25
Conditional probability and independent events

Method 2:

P (at least one 4 comes up) + P (no 4 comes up) = 1 (19)

Then

P (at least one 4 comes up) = 1 − P (no 4 comes up)

= 1 − P (no 4 on 1st and 2nd toss)
= 1 − P (A01 ∩ A02 ) = 1 − P (A01 )P (A02 )
  
5 5 11
=1− = (20)
6 6 36

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 18 / 25

Conditional probability and independent events

Problem 1.13
One bag contains 4 white balls and 2 black balls; another contains 3 white
balls and 5 black balls. If one ball is drawn from each bag, find the probability
that (a) both are white, (b) both are black, (c) one is white and one is black.

Let W1 = event “white ball from first bag,” W2 = event “white ball from
second bag.”
(a)

P (W1 ∩ W2 ) = P (W1 )P (W2 |W1 )

= P (W1 )P (W2 )
  
4 3 1
= = (21)
4+2 3+5 4

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 19 / 25

Conditional probability and independent events

(b)

P (W10 ∩ W20 ) = P (W10 )P (W20 |W10 )

= P (W10 )P (W20 )
  
2 5 5
= = (22)
4+2 3+5 24

(c) The required probability is

1 5 13
1 − P (W1 ∩ W2 ) − P (W10 ∩ W20 ) = 1 − − = (23)
4 24 24

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 20 / 25

Theorems on Conditional Probability

Theorem 1.10
If an event A must result in one of the mutually exclusive events
A1 , A2 , . . . An , then

P (A) = P (A1 )P (A|A1 ) + P (A2 )P (A|A2 ) + · · · + P (An )P (A|An ) (24)

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 21 / 25

Conditional probability and independent events
Problem 1.14
Prove Theorem 1.10:

P (A) = P (A1 )P (A|A1 ) + P (A2 )P (A|A2 ) + · · · + P (An )P (A|An ) (25)

We prove the theorem for the case n = 2. Extensions to larger values of n

are easily made. If event A must result in one of the two mutually
exclusive events A1 , A2 , then
A = (A ∩ A1 ) ∪ (A ∩ A2 ) (26)
But (A ∩ A1 ) and (A ∩ A2 ) are mutually exclusive since A1 and A2 are.
Therefore, by Axiom 3,
P (A) = P (A ∩ A1 ) + P (A ∩ A2 )
= P (A1 )P (A|A1 ) + P (A2 )P (A|A2 ) (27)
Using conditional probability formula (see topic 1.8).
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 22 / 25
Conditional probability and independent events
Problem 1.15
Box I contains 3 red and 2 blue marbles while Box II contains 2 red and
8 blue marbles. A fair coin is tossed. If the coin turns up heads, a marble
is chosen from Box I; if it turns up tails, a marble is chosen from Box II.
Find the probability that a red marble is chosen.

Let R denote the event “a red marble is chosen” while I and II denote
the events that Box I and Box II are chosen, respectively. Since a red
marble can result by choosing either Box I or II, we can use the results of
Problem 1.14 with A = R, A1 = I, A2 = II. Therefore, the probability of
choosing a red marble is
P (R) = P (I)P (R|I) + P (II)P (R|II)
     
1 3 1 2
= +
2 3+2 2 2+8
2
= (28)
5
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 23 / 25
Independent Events
If P (B|A) = P (B), i.e., the probability of B occurring is not affected by
the occurrence or non-occurrence of A, then we say that A and B are
independent events. This is equivalent to

P (A ∩ B) = P (A)P (B) (29)

as seen from (2). Conversely, if (29) holds, then A and B are independent.
We say that three events A1 , A2 , A3 , are independent if they are pairwise
independent

P (Aj ∩ Ak ) = P (Aj )P (Ak ) j 6= k, j, k = 1, 2, 3 (30)

and

P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 )P (A3 ) (31)

N.B: Note that neither (30) nor (31) is by itself sufficient. Independence of
more than three events is easily defined.
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 24 / 25
Thank you for your kind attention!

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 3 March 9, 2023 25 / 25