Theory of Probability and

Mathematical Statistics

Lecture 2 - Probability

Akerke Zhailaubek
akerke.zhailaubek@astanait.edu.kz

Lecture overview:

1. Introduction to probability.
2. Independent and dependent events.
3. Classical probability. Conditional probability.
4. The law of total probability. Bayes' Theorem.
5. Tree and Venn diagrams.

2
We will review the vocabulary by considering the
action of repeatedly throwing a six-faced die and
recording the number displaying upwards.

3
The experiment is the action of repeatedly throwing
the die.
The outcome is the number displayed upwards.
Getting a ‘5’, say, is an event. Getting a ‘5 or 6’ is also
an event.

The set of all possible outcomes,

i.e. 1, 2, 3, 4, 5 and 6 is called the
sample space.
4
Putting this in general terms;
• An experiment is a repeatable process that gives rise
to a number of outcomes.
• An event is a collection (or set) of one or more
outcomes.
• A sample space is the set of all possible outcomes of
an experiment.

5
The probability of an event, e.g. the probability of
getting a ‘5’ when throwing a die, is the chance that the
event will occur as a result of an experiment.

In this example, there are 6 possible outcomes (1, 2, 3,

4, 5, 6) and each is equally likely. So, the probability of
getting a 5 is 1/6.

6
Where outcomes are equally likely then probability of
an event is the number of outcomes in the event
divided by the total number of possible outcomes in
the sample space.
So, if the event were getting a ‘5 or 6’, then the
probability would be 2/6, i.e. 1/3.

7
• An impossible event has a probability of 0.
• A certain event has a probability of 1.
• So, all events have probabilities between 0 and 1.
• Probabilities can be written as fractions, decimals or
percentages.

8
Two fair spinners each have four faces numbered 1
to 4. The two spinners are thrown together and the
sum of the numbers indicated on each spinner is
recorded.

Find the probability of the spinners indicating a sum

of
a exactly 5,
b more than 5.

+ =?
1 2 1 2

4 3 4 3
9
10
 See the notation used here. P for
probability, the event is in brackets.
11
 Remember: a sample space is all possible
This is a Venn diagram outcomes and an event is a collection of
one or more outcomes.

Closed curves :

12
You can write the number of outcomes for each event
or the probabilities of the events in the Venn diagram.

The following slides give the meaning of some

important terminology; Union, intersection and
complement.

13
Intersection: A and B both occur , A ⋂ B

14
Union: A or B or both occur, A ⋃ B

15
Complement: any outcome but not A occurs, A’

16
A survey of 79 families found that 22 owned cats , 27
owned dogs and 7 owned both cats and dogs.

a) Draw a Venn diagram to represent this

information

A family is chosen at random. Find the probability

that the family

b) owns dogs only,

c) does not own cats,
d) does not own dogs or cats.

17
Our Venn diagram needs to show the event ‘owns a cat’
and the event ‘owns a dog’.

Of course it is possible that a family owns both cats and

dogs and it is possible that they own neither.

The Venn diagram shows the number of families

belonging to each of four regions

families that own : only cats

only dogs
both cats and dog
neither cats or dogs
18
The 79 refers to the sum of these four numbers

The 22 refers to families that own cats but maybe some of

these families own dogs too.

The 27 refers to families that own dogs but maybe some

of these families own cats too.

The 7 refers to families that own both cats and dogs

19
 S

C D

Cats only Cats Dogs

and only
22-7=15 dogs
7 27-7 = 20
Neither
cats or
dogs ?

20
 S

C D

Cats only Cats Dogs

and only
22-7=15 dogs
7 27-7 = 20

79- (15+7+20)= 37
21
All that is really necessary for the Venn diagram is this:

C D
15 7 20 37

Remember the original question?

A family is chosen at random. Find the probability that the
family
b) owns dogs only,
c) does not own cats,
d) does not own dogs or cats
22
 C D
15 7 20 37

A family is chosen at random. Find the probability that

the family
b) owns dogs only, 20/79
c) does not own cats, 57/79
d) does not own dogs or cats 37/79

23
The addition rule

P(A ⋃ B)=P(A) + P(B) − P(A ⋂ B )

24
This is the addition rule:

P(A ⋃ B) = P(A) + P(B) – P(A ⋂ B)

We can rearrange it:

P(A ⋂ B) = P(A) + P(B) – P(A ⋃ B)

25
• Consider a two event Venn diagram to see why this is
true.
• Let P(A) = a, P(B) = b and P(A ⋂ B) = i
• Let the sample space be S

A B S
a–i i b–i

26
 A B S
a–i i b–i

Now, P(A ⋃ B) = a – i + i + b – i = a + b – i
Hence, P(A ⋃ B) = P(A) + P(B) – P(A ⋂ B)

27
 Example

A and B are two events and P(A)=0.6,

P(B)=0.7 and P(A ⋃ B)=0.9 .
Find
a P(A ⋂ B)
b P(A’)
c P(A’ ⋃ B)
d P (A’ ⋂ B )
28
We can use the addition formula and
the complement formula to answer
parts a and b.

29
To answer parts c and d, we will use a Venn diagram.

A B S
0.2 0.4 0.3
From part a
0.1

Given: P(A) = 0.6, P(B) = 0.7 and P(S) = 1

30
Part c: P(A’ ⋃ B)
This means we want to be not in A or in B.
Find this area on the Venn diagram:

A B S
0.2 0.4 0.3
0.1
So, P(A’ ⋃ B) = 0.4 + 0.3 + 0.1 = 0.8
31
Part d: P(A’ ⋂ B)
Method 1
Using the formula:
P(A’ ⋂ B) = P(A’) + P(B) – P(A’ ⋃ B)
and P(A’) = 1 – P(A)

So, P(A’ ⋂ B) = 1 – 0.6 + 0.7 – 0.8 = 0.3

32
Part d: P(A’ ⋂ B)
Method 2
This means we want to be not in A and in B.
Find this area on the Venn diagram:

A B S
0.2 0.4 0.3
0.1

So, P(A’ ⋂ B) = 0.3

33
Venn diagrams
to describe 3 events

34
If the 3 events are A, B and C then we need eight
regions

1. A and B and C 5. A only

2. A and B (but not C) 6. B only
3. A and C (but not B) 7. C only
4. B and C ( but not A) 8. Neither A or B or
C

35
A B
A only A and B
but not C B only

A and B
A and C and C
B and C
but not B
but not A

C only
Neither A or B or
C C
36
 Example
A vet surveys 100 of her clients. She finds that
25 own dogs, 15 own dogs and cats, 11 own dogs and
tropical fish, 53 own cats, 10 own cats and tropical fish, 7
own dogs, cats and tropical fish, 40 own tropical fish.

a) Draw a Venn diagram to represent this information

A client is chosen at random. Find the probability that

the client

b) owns dogs only,

c) does not own tropical fish,
d) does not own dogs, cats or tropical fish.
37
To draw a Venn diagram, we need to decide how many
events there are, i.e. how many circles we need to draw.

How many events are there?

38
• Since the question is asking about the probability of
owning the various animals, then each animal is one
event.
• There are three types of animals; dogs, cats and fish.
• So, we will have three circles (one for each event).
• Use the first letter for each animal as labels for the
events; D, C and F (we use capital letters for events).

39
D C S

40
• Now we must enter the numbers given in the
question into each section.
• Think about what each section represents.
• Firstly, fill in the numbers given in the question.
• Then work out any missing numbers.

Note: This is example 5 on page 82 in the S1 textbook.

They have entered the probabilities rather than the
numbers into the Venn diagram. Both methods are
correct and lead to the same answers.

41
Enter the easy numbers first.

D (25) C (53) S (100)

F (40)
42
From the question: D ⋂ C = 15 (dog and cat)

D (25) C (53) S (100)

8
7

F (40)
43
Continuing: C ꓵ F = 10 (cat and fish)
and D ꓵ F = 11 (dog and fish)

D (25) C (53) S (100)

8
7
4 3

F (40)
44
Using the totals for D, C, F and S we can find the
missing numbers.

D (25) C (53) S (100)

25 – 8 – 7 – 4 8 53 – 8 – 7 – 3
=6 = 35
7
4 3
40 – 4 – 7 – 3
100 – 6 – 8 – 35 = 26
– 4 – 7 – 3 – 26
= 11 F (40)
45
So, the final Venn diagram is:

D C S
6 8 35
7
4 3
26
11
F
46
• Now we must answer the rest of the question.
• To find the probability of each section we simply
divide the section number by the total number of
animals, i.e. 100.
• Identify the correct section of the Venn diagram for
each part of the question.

47
 P(owns dogs only)
= 6/100 = 3/50

D C S
6 8 35
7
4 3
26
11
F
48
 P(does not own tropical fish)
= (6 + 8 + 35 + 11)/100
= 60/100 = 3/5

D C S
6 8 35
7
4 3
26
11
F
49
 P(does not own dogs, cats or fish)
= 11/100

D C S
6 8 35
7
4 3
26
11
F
50
 Question
A group of 100 people produced the following
information relating to three attributes.
The attributes were wearing glasses, being left-handed
and having dark hair.
Glasses were worn by 36 people, 28 were left-handed
and 36 had dark hair.
There were 19 who wore glasses and had dark hair, 17
who wore glasses and were left-handed, and 15 who
were left-handed and had dark hair.
Only 10 people wore glasses, were left-handed and had
dark hair.
a) Represent these data on a Venn diagram.
 Solution
Glasses Left

10 7 6

10
9 5

41 12
Dark
Based on the Venn diagram you drew, solve the following
questions.

A person was selected at random from this group.

Find the probability that this person
b) Wore glasses but was not left-handed and did not have
dark hair
c) Did not wear glasses, was not left-handed and did not
have dark hair
d) Had only two attributes
Solution
!"
b) = 0.1
!""

#!
c) = 0.41
!""

$%&%' (!
d) = = 0.21
!"" !""
 Mutually exclusive events
• When events have no outcomes in common, then
they are mutually exclusive.

• This means that the events cannot occur together

• e.g.
1) The event it is raining and the event it is not raining
(at the same time, in the same place)
2) The event of getting an even number and the event
of getting an odd number
 Mutually exclusive events in Venn diagram

B S
A

No intersection, P(A ∩ B) = 0

Can you state the addition rule for mutually exclusive events?
Addition rule for mutually exclusive events

P(A ⋃ B) = P(A) + P(B) – P(A ∩ B)

P(A ⋃ B) = P(A) + P(B)

Example

Events A and B are mutually exclusive and

P(A) = 0.2 and P(B) = 0.4.
Find
a) P(A ⋃ B)
b) P(A ∩ B’)
c) P(A’ ∩ B’)
 Solution S
a) P(A⋃B) A B
= P(A) + P(B) 0.2 0.4
= 0.2 + 0.4 = 0.6
0.4
b) P(A ∩ B’) 1 – 0.2 – 0.4 = 0.4
we locate in A and not B: P(A ∩ B’) = 0.2

c) P(A’ ∩ B’)
we locate not A and not B: P(A’ ∩ B’) = 0.4
Conditional probability

Consider two events: Event A and event B

We may wish to calculate the probability of event A,

given that event B has occurred.

The probability of event A given that event B has

occurred is called the conditional probability of A
given B.
Conditional probability formula

Notation:
The probability of event A given B is written as P(
A|B ).

Formula:
𝐏(𝑨 ∩ 𝑩)
𝐏 𝑨𝑩 =
𝐏(𝑩)
Why?

We can see where this formula comes from by

considering a Venn diagram:

A B S

Since event B has already occurred, our total number

of outcomes is reduced from S to B.
We are interested in the probability of A given B. So,
locate the area for A on the Venn diagram within B.

A B S B

This shaded area is A ∩ B.

Remember that probability is found by
P(desired outcomes)/P(total).
For us, P(A|B) =P(A ∩ B)/P(B).
Multiplication rule

Rearranging this formula leads to the multiplication

rule:

P(A ∩B ) = P(A|B ) × P(B)

Example

C and D are two events such that P(C) =0.2 ,

P(D)=0.6 and P(C|D) =0.3 .

Find
Solution
a) P(D|C) a) We can use the multiplication rule:
b) P(C’∩D’) P(C ∩ D) = P(C|D) x P(D)
c) P(C’∩D) = 0.3 x 0.6 = 0.18

Now P(D|C) = P(C ∩ D)/P(C)

= 0.18/0.2 = 0.9
 Solution
For parts b) and c) we will use a Venn diagram.
S
C D

0.2 – 0.18 0.6 – 0.18

0.18
= 0.02 = 0.42

1 – 0.02 – 0.18 – 0.42 = 0.38

Solution

b) P(C’∩D’)
This means not in C and not in D.
P(C’ ∩ D’) = 0.38
S
C D

0.2 – 0.18 0.6 – 0.18

0.18
= 0.02 = 0.42

1 – 0.02 – 0.18 – 0.42 = 0.38

Solution

c) P(C’∩D)
This means not in C and in D.
P(C’ ∩ D) = 0.42
S
C D

0.2 – 0.18 0.6 – 0.18

0.18
= 0.02 = 0.42

1 – 0.02 – 0.18 – 0.42 = 0.38

 A tree diagram
A tree diagram is used to show all the possible outcomes of
a sequence of events.
Example 2
The probability that Adil spends more than four hours
between 11.00pm and 7.00am playing computer games is
0.7. If he does do this the probability that he fails to
attend his 9.00am seminar is 0.9. If he doesn’t do this the
probability that he misses his 9.00am seminar is 0.2.
Represent this information using a tree diagram.
 Solution
Probability that Adil misses his 9.00am seminar or not
is dependent upon whether he plays computer games
for more than four hours during the night. This is a
conditional probability situation.

Let A represent the event that Adil plays computer

games for more than four hours.

Let B represent the event that Adil misses his 9.00am

seminar.
Going back to the question,
• the probability of missing a seminar (Event B) if playing
games more than 4 hours (Event A) is 0.9. That is
𝑃 𝐵 𝐴 = 0.9.

• the probability of missing a seminar (Event B) if not

playing games more than 4 hours (Event A’) is 0.2. That
is 𝑃 𝐵 𝐴′ = 0.2.
• The first event is A, either it happens or it doesn’t.
Adil plays computer games for more than four hours
or he doesn’t.

0.7 A
After the event A or A’
either B occurs or B’.
A’
0.3 This will be shown on the
second branch of the tree
diagram
Branch 1

Sum of the numbers from the same root

should be equal to 1.
 These branches correspond to the
conditional probabilities because either A
happens or not is assumed.

0.9
? B ? = P(B|A)
0.7 A
B’ ? = P(B’|A)
0.1?
0.2? B ? = P(B|A’)
0.3 A’
B’ ? = P(B’|A’)
0.8
?
Can you complete the
tree diagram by
Branch 1 Branch 2 identifying the four
missing probabilities?
The conditional probability 𝑷 𝑩 𝑨 and 𝑷 𝑨 ∩ 𝑩 are
different!!!!!

𝑃(𝐵|𝐴) B The multiplication rule says,

A 𝑃 𝐴 ∩ 𝐵 = 𝑃(𝐴)×𝑃(𝐵|𝐴)
𝑃(𝐴) Hence, in order to obtain
B’
𝑃(𝐵′|𝐴) 𝑃(𝐴 ∩ 𝐵), we multiply the
𝑃(𝐵|𝐴′) B numbers following the
𝑃(𝐴′)
A’ appropriate routes.
B’
𝑃(𝐵′|𝐴′)
What about 𝑃 𝐴) ∩ 𝐵 ?
These are conditional probabilities!
Example continued.
From the tree diagram in the previous slide,
find 𝑃(𝐴 ∩ 𝐵), 𝑃 𝐴 ∩ 𝐵) , 𝑃(𝐴) ∩ 𝐵) and 𝑃(𝐴) ∩ 𝐵) ).

What is the sum of all these four probabilities?

The sum is 1!
 Example continued
What is the probability that Adil played a game for
more than 4 hours (event A) on a certain day given that
he missed the 9 am seminar next day (event B)?

0.9 B
A
𝑷 𝑨 𝑩 =?
0.7
0.1 B’
B
A’ 0.2
0.3
This is 𝑃 𝐵 𝐴 ,
0.8 B’ not 𝑃(𝐴|𝐵)
 Solution
By definition,
𝑃(𝐴 ∩ 𝐵)
𝑃 𝐴𝐵 =
𝑃(𝐵)

𝑃 𝐴 ∩ 𝐵 = 0.7×0.9 = 0.63
𝑃 𝐵 = 𝑃 𝐴 ∩ 𝐵 + 𝑃 𝐴′ ∩ 𝐵
= 0.7×0.9 + 0.3×0.2 = 0.69

".+, +,
Therefore, 𝑃 𝐴 𝐵 = ".+& = +&
 Example

A bag contains seven green beads and five blue

beads. A bead is taken from the bag at random, the
color is recorded and it is not replaced. A second
bead is then taken from the bag and its color
recorded.

Find the probability that one bead is green and

another bead is blue.
 Solution
6#
11 G2
7#
12 G1 5#
11 B2
5#
7#
11 G2
12 B1
4#
11 B2
It can be (G1 and B2) or (B1 and G2)
7 5 5 7 70 35
𝑃 𝐺! ∩ 𝐵" + 𝑃 𝐵! ∩ 𝐺" = × + × = =
12 11 12 11 132 66
 Question
A bag contains seven green beads and five blue beads.
A bead is taken from the bag at random, the color is
recorded and it is not replaced. A second bead is then
taken from the bag and its color is recorded and it is
not replaced. A third bead is taken from the bag and its
color is recorded.
Find the probability that the third bead is blue given
that the first two were green.
 Solution
5#
10 G3
6#
𝑃(𝐵, |𝐺! 𝐺( )
11 G2 𝟓# B3
𝟏𝟎

7# G1 6#
10 G3
12 5#
11 B2
7#
11 G2 4#
10 B3
5#
12 B1 7#
10 G3
4#
11 B2
3#
10
B3
 Conditional probability using a Venn diagram
Example 4. Do you remember the three event Venn diagram
from the first probability lecture? Use the Venn diagram to find
the following conditional probabilities for a randomly chosen
client.
a) own cats given that they own dogs
b) own cats given they own dogs and tropical fish
c) own cats given they own dogs or tropical fish or both.

D S
6 8 35 C
7
4 3
26
11 F
Solution
a) Since owning a dog is already assumed, you need to
restrict your interest to “D” only and the total number
of outcomes is 6+4+7+8=25.
In the region for “D”, 8+7=15 also own a cat.
!'
Therefore, 𝑃 𝐶 𝐷 = = 0.6
('

$
b) Similarly, 𝑃 𝐶 𝐷 ∩ 𝐹 =
!!

-%$%, !- !
c) 𝑃 𝐶 𝐷 ∪ 𝐹 = = =
+%#%-%$%,%(+ '# ,
 Independent events
Recall: What are mutually exclusive events?

• If one event has no effect on another event, then the

events are independent, e.g. the event that a student
passes an exam in Kazakhstan and the event that
there is a hurricane in the Atlantic Ocean.
• If A and B are independent, then the probability of A
is the same, whether B has happened or not.
• This means that if A and B are independent then,
P(A|B) = P(A)
 Multiplication rule for independent events
Remembering the formula for conditional probability,
𝐏(𝑨 ∩ 𝑩)
𝐏 𝑨𝑩 =
𝑷(𝑩)
And now assume that A and B are independent,
P(A|B ) = P(A)
We have
P(A ∩ B ) = P(A ) × P(B )
Example 5

Events A and B are independent and

P(A) = 1/3 and P(B) = 1/5.

Find
a) P(A ∩ B)
b) P(A ∩ B’)
c) P(A’ ∩ B’)
Solution

a) P(A ∩ B) = P(A) x P(B) = 1/3 x 1/5 = 1/15

b) If A and B are independent, then so are A and B’.

Therefore, P(A ∩ B’) = P(A) x P(B’)

Remember that P(B’) = 1 – P(B) = 1 – 1/5 = 4/5

So, P(A ∩ B’) = 1/3 x 4/5 = 4/15

Solution

c) A’ and B’ must also be independent.

Therefore P(A’ ∩ B’) = P(A’) x P(B’)

and P(A’) = 1 – P(A) = 1 – 1/3 = 2/3

so, P(A’ ∩ B’) = 2/3 x 4/5 = 8/15

 Example 6
A bag contains seven green beads and five blue beads.
A bead is taken from the bag at random, the color is
recorded and the bead is replaced. A second bead is
then taken from bag and its color recorded.

Draw a tree diagram and hence answer the following;

a) Find the probability that one bead is green and
another bead is blue.

b) Show that the event ‘the first bead is green’ and

the event ‘the second bead is green’ are
independent.
Solution
𝟕#
𝟏𝟐 G2
𝟕#
𝟏𝟐 G1 𝟓#
𝟏𝟐 B2
𝟓#
𝟕#
𝟏𝟐 G2
𝟏𝟐 B1
𝟓#
𝟏𝟐 B2
$ ' ' $
a) 𝑃 𝐺! ∩ 𝐵( + 𝑃 𝐵! ∩ 𝐺( = !( × !( + !( × !( =
,'
$(
 Solution continued
b) We want to show that
𝑃 𝐺! ∩ 𝐺( = 𝑃 𝐺! ×𝑃(𝐺( )

$
𝑃 𝐺! = from the tree diagram.
!(

How about 𝑃 𝐺( = ?
The first bead can be either green or blue.
That is, (G1 and G2) or (B1 and G2).
 Solution continued
𝑃 𝐺( = 𝑃 𝐺! ∩ 𝐺( + 𝑃 𝐵! ∩ 𝐺(

$ $ ' $ $
= × + × =
!( !( !( !( !(

$ $ $
𝑃 𝐺! ∩ 𝐺( = × and 𝑃 𝐺! = 𝑃 𝐺( = .
!( !( !(
Hence, 𝑃 𝐺! ∩ 𝐺( = 𝑃 𝐺! ×𝑃(𝐺( ) and we can
conclude that 𝐺! and 𝐺( are independent.
The Law of Total Probability
• Let 𝐴! , 𝐴( , … , 𝐴. be mutually exclusive and
exhaustive events. Then for any other event 𝐵,

𝑃 𝐵 = 𝑃 𝐵 𝐴! 𝑃 𝐴! + ⋯ + 𝑃 𝐵 𝐴. 𝑃 𝐴.
.

= H 𝑃 𝐵 𝐴/ 𝑃 𝐴/
/0!
Bayes’ Theorem
• Let 𝐴! , 𝐴( , … , 𝐴. be a collection of 𝑘 mutually
exclusive and exhaustive with prior probabilities
𝑃 𝐴/ 𝑖 = 1, … , 𝑘 . Then for any other event B for
which 𝑃(𝐵) > 0, the posterior probability of 𝐴1 given
that B has occurred is

2 3! ∩5 2 5|3! 2(3! )
𝑃 𝐴1 𝐵 = = , 𝑗 = 1, … , 𝑘
2 5 "#$ 2 𝐵 𝐴/ 2 3"
∑%
Example
Solution
Solution cont.
References:
1. Statistics 1 Edexcel AS and A Level Modular
Mathematics S1 published by Pearson
Education Limited
2. Lecture notes, NUFYP, Nazarbayev
University.

98