Mathematical Statistics I

Module for Math 150.1

an
lim
Di
UP
s,
ic
at
em
h
at
M
of
te
itu
st
In
Chapter 1: The Basics

1.1 The Probability Space

An experiment is any process that can be repeated and can generate a well-defined set of
possible outcomes or data. An example of an experiment is the tossing of a coin, where

an
there are only two possible outcomes, either a head or a tail. We say that an experiment

lim
is random if it has more than one possible outcome (e.g., tossing a fair coin) while it is

Di
called deterministic if only one outcome is guaranteed (e.g., tossing a Master ball to catch
a Pokémon).
UP
ics,

Whenever we have a random experiment, a probability space can be defined. The proba-
at

bility space is composed of three components: a non-empty set Ω, a “nice” set of subsets
em

of Ω, and a function which measures such subsets of Ω.

h
at
M

Definition 1.1 Let Ω be a non-empty set. We call Ω as the sample space, which is the
of

collection or totality of all possible outcomes of a random experiment.

te

Definition 1.2 Let A ⊆ Ω. The complement of A with respect to Ω is the set A0 , defined
itu

by A0 = {ω ∈ Ω | ω ∈
st

/ A} = Ω − A.
In

Definition 1.3 A collection F of subsets of Ω is called a σ-algebra (or a σ-field) of Ω if

1. Ω ∈ F;
2. whenever A ∈ F we have A0 ∈ F; and
∞
[
3. whenever A1 , A2 , . . . ∈ F, we have Ai ∈ F.
i=1

The most typical σ-algebra F consist of all subsets of Ω, that is, F = P(Ω), the power set
of Ω.
Different scenarios give rise to different σ-algebras, and in our case we shall call the desired
σ-algebra F as the event space and any element of F is called an event.

2
Chapter 1. The Basics 3

Remark: Let F be a σ-algebra. We then get the following results:

1. ∅ ∈ F.
∞
\
2. If A1 , A2 , . . . ∈ F, then Ai ∈ F.
i=1
3. If A, B ∈ F, then A − B ∈ F.

Proof:

1. Since Ω ∈ F then ∅ = Ω0 ∈ F.
2. Suppose A1 , A2 , . . . ∈ F then A01 , A02 , . . . ∈ F. Then
∞ ∞
!0
[ \
F3 A0i = Ai .

an
i=1 i=1

lim
 ∞  0 0 ∞
T T

Di
Hence Ai = Ai ∈ F.
i=1 i=1
3. Since A, B ∈ F then A ∈ F. Moreover, A − B = B ∩ A0 ∈ F.
0 UP 

Let A ⊆ Ω. The σ-algebra generated by A, which we denote by σ(A) is the smallest

s,
ic

σ-algebra that contains A.

at
em

Example: A coin tossed twice gives Ω = {HH, HT, T H, T T }.

h

• The trivial σ-field is F = P(Ω) = {∅, {HH}, {HT }, {T H},

at

{T T }, {HH, HT }, {HH, T H}, {HH, T T }, {HT, T T }, {T H, T T }, {HT, T H},

M

{HH, HT, T H}, {HH, HT, T T }, {HT, T H, T T }, {HH, T H, HT }, Ω}.

of

• Another σ-field is the set G = σ({HH, HT }) = {φ, {HH, HT }, {T T, T H}, Ω},

te
itu

which is the σ-field generated by the set {HH, HT }.

st
In

For the meantime, we shall consider P(Ω) as our event space, unless otherwise stated.
Given two events A and B, we say that the events are disjoint or mutually exclusive if
A ∩ B = φ.
We now define the third component of a probability space.

Definition 1.4 Let Ω and F be the outcome space and event space, respectively. A function
P : F → [0, 1] is called a probability function/measure if it satisfies

1. P(φ) = 0
2. P(Ω) = 1
∞
! ∞
[ X
3. for A1 , A2 , . . . ∈ F, pairwise disjoint, P Ai = P(Ai ).
i=1 i=1
Chapter 1. The Basics 4

Theorem 1.5 For A, B ∈ F,

1. If A ⊆ B, then P(A) ≤ P(B).

2. P(A0 ) = 1 − P(A).
3. P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Proof:

1. Since A ⊆ B then the sets A and B − A are disjoint. Hence,

P(B) = P(A ∪ [B − A]) = P(A) + P(B − A) ≥ P(A).

2. The sets A and A0 are disjoint. Thus,

an
lim
P(A) + P(A0 ) = P(A ∪ A0 ) = P(Ω) = 1.

Di
UP
3. Consider the sets A − B, B − A and A ∩ B. These sets are pair-wise disjoint, so
s,

P(A ∪ B) = P ([A − B] ∪ [B − A] ∪ [A ∩ B]) = P(A − B) + P(B − A) + P(A ∩ B)

ic
at

= P(A − B) + P(A ∩ B) + P(B − A) + P(A ∩ B) − P(A ∩ B)

em

= P([A − B] ∪ [A ∩ B]) + P([B − A] ∪ [A ∩ B]) − P(A ∩ B)

h
at

= P(A) + P(B) − P(A ∩ B)

M
of


te
itu

Exercise: Prove that for A, B, C ∈ F, P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) −

st

P(B ∩ C) − P(A ∩ C) + P(A ∩ B ∩ C).

In

Definition 1.6 The probability space, denoted by (Ω, F, P), is a triple consisting of a
sample space Ω, a σ-algebra F, and a probability measure P.

If A and B are events, then A ∩ B and A ∪ B are also events. Note that A ∪ B occurs
if either A or B occur. Similarly, A ∩ B occurs if both A and B occur. Furthermore,
if A1 , A2 , . . . , An are events, then the event A1 ∪ A2 ∪ . . . ∪ An occurs if at least one of
A1 , A2 , . . . , An occurs. The event A1 ∩ A2 ∩ . . . ∩ An if all the events A1 , A2 , . . . , An occur.
Examples:
Chapter 1. The Basics 5

1. A smoke detector system uses two devices. If smoke is present, the probability
that it will be detected by the first device is 0.95, while that by the second device
is 0.90, and by both devices is 0.88. If smoke is present, find the probability
that the smoke is

1. detected by at least one of A and B.

2. not detected by either A or B.

Solution: Let A be the event that smoke is detected by the first device and B
be the event that smoke will be detected by second device.

1. The event that smoke is detected by at least one of devices A and B

is A ∪ B. Hence P(A ∪ B) = 0.95 + 0.90 − 0.88 = 0.97.

an
2. Define C to be the event that smoke is not detected, so C = (A ∪ B)0 .

lim
Then P(C) = 1 − P(A ∪ B) = 1 − 0.97 = 0.03.

Di
3. Suppose we have the following information: UP
a. There is a 60% chance of rain today.
ic s,

b. There is a 50% chance of rain tomorrow.

at

c. There is a 20% chance that it will not rain on both days.

hem

Find the following probabilities:

at
M

1. The probability that it will rain today or tomorrow.

of

2. The probability that it will rain both today and tomorrow.

te

3. The probability that it will rain today but not tomorrow.

itu

4. The probability that it will rain either today or tomorrow, but not
st

on both days.
In

Solution: Let A be the event that it will rain today, and B the event that it
will rain tomorrow. In terms of probability symbols, the given are as follows:

a. P(A) = 0.6
b. P(B) = 0.5
c. P(A0 ∩ B 0 ) = 0.2

So

1.

P(A ∪ B) = 1 − P([A ∪ B]0 ) = 1 − P(A0 ∩ B 0 )

= 1 − 0.2 = 0.8
Chapter 1. The Basics 6

2. P(A ∩ B) = P(A) + P(B) − P(A ∪ B) = 0.6 + 0.5 − 0.8 = 0.3

3. P(A ∩ B 0 ) = P(A − B) = P(A) − P(A ∩ B) = 0.6 − 0.3 = 0.3
4. Similar to the previous problem:

P(B ∩ A0 ) = P(B − A) = P(B) − P(A ∩ B) = 0.5 − 0.3 = 0.2.

Note that the events, that it will rain today but not tomorrow and it
will rain tomorrow but not today, are disjoint. Hence

P([A ∩ B 0 ] ∪ [B ∩ A0 ]) = P(A − B) + P(B − A) = 0.3 + 0.2 = 0.5.

1.2 Counting Rules

an
lim
In many cases, problems in probability can be solved by simply counting the number of

Di
ways that a certain event can occur. The mathematical theory of counting is formally
UP
known as combinatorial analysis. The basic principle of counting is given as follows.
ic s,
at

The Basic Principle of Counting (Multiplication Principle): If an experiment has

em

n1 distinct outcomes, and if for each of these outcomes, a second experiment has n2 distinct
h

outcomes, and for each pair from the first two experiments, a third experiment has n3 dis-
at
M

tinct outcomes, so on and so forth, then the sequence of k experiments has n1 · n2 · n3 · · · nk

of

distinct outcomes.
te
itu

Examples:
st
In

1. How many 7-place license plates are possible if the first three places are to be
occupied by letters and the final four by numbers?
Solution: By the multiplication rule, we will have 26 · 26 · 26 · 10 · 10 · 10 · 10 =
175, 760, 000 ways.
2. Suppose you have 20 different t-shirts, 6 different pairs of pants and 3 different
pairs of shoes. How many days will your wardrobe last, if each day you need to
wear a distinct outfit?
Solution: By the multiplication rule, you will have 20 · 6 · 3 = 360 possible
outfits.
3. How many odd four-digit numbers can be formed from the digits 0, 1, 3, 4, 7,
and 8 if each digit can be used only once?
Chapter 1. The Basics 7

Solution: Since the number must be odd, there are only three choices for the
ones digit. On the other hand, there will be four choices for the thousands digit
since it cannot be 0 and each digit can be used only once. For the hundreds
and tens digits, there will be four and three choices, respectively. Hence, there
will be 4 · 4 · 3 · 3 = 144 odd four-digit numbers.

There are problems in counting where we are interested in all possible orders or arrange-
ments of a group of objects. For example, we may want to know how many different ways
a group of 5 people can line up, or how many arrangements of the letters a, b, and c are
possible. These different arrangements are called permutations.

Definition 1.7 A permutation is an arrangement of all or part of a set of objects.

an
lim
Theorem 1.8 The number of permutations of n distinct objects is n!, referred to as n f actorial,

Di
and is defined by n! = n · (n − 1) · (n − 2) · · · 2 · 1, with 0! = 1.
UP
Theorem 1.9 The number of arrangements of n distinct objects, taken r at a time is given
s,

by
ic

n!
at

n Pr = .
(n − r)!
em

Theorem 1.10 The number of permutations of n distinct objects arranged in a circle is

h
at

(n − 1)!.
M
of

Theorem 1.11 The number of distinct permutations of n things of which, n1 are of one
te

kind, n2 of a second kind, . . . , nk of a kth kind is

itu

n!
st

.
n1 ! · n2 ! · · · nk !
In

Examples:

1. How many different arrangements are there of the digits 1 2 3 4 5 6?

Solution: Since there are 6 distinct digits, there will be 6! = 720 possible
arrangements.
2. There are 15 candidates for 3 different executive positions. How many different
ways can you fill the positions?
Solution: The positions can be filled up in 15 P3 = 2730 ways.
3. Suppose there are 4 military missions, each requiring 1 soldier. How many ways
can 4 soldiers from a squadron of 80 be assigned to these 4 missions?
Solution: The missions can be assigned to 4 soldiers in 80 P4 = 37, 957, 920 ways.
Chapter 1. The Basics 8

4. You have 20 construction workers. You wish to assign 8 to job site 1, 5 to

job site 2, and 7 to job site 3. In how many different ways can you make this
arrangement?
Solution: The number of ways to assign the workers to the different job sites is
given by
20!
= 99, 768, 240.
8!5!7!
5. How many different letter arrangements can be made from the letters of the
word PROBABILITY?
Solution: Probability has 11 letters and since the letters B and I are both
repeated twice, the number of letter arrangements is given by

an
11!

lim
= 9, 979, 200.
2!2!

Di
There are times that we are interested in determining the number of different groups of
UP
r objects can be formed from a total of n objects, without regard to order. For example,
s,

we may want to identify how many different groups 4 can be selected from 7 objects. We
ic
at

define the following.

em

Definition 1.12 A combination, denoted n Cr , represents the number of different groups

h
at

of size r that can be selected from a set of n objects when order is not relevant, defined by
M

 
n n!
of

n Cr = = .
r (n − r)!r!
te
itu

Remark:  
n n Pr
st

= n Cr = .
In

r r!
Examples:

1. How many different committees of 4 members can be formed from a group of

20 people?
Solution: The number of possible committees is
 
20 20! 20 · 19 · 18 · 17
= = = 4845.
4 (16!)(4!) 4·3·2·1
2. From a group of 5 women and 7 men, how many different committees consisting
of 2 women and 3 men can be formed? What if 2 of the men are feuding and
refuse to serve on the committee together?
Chapter 1. The Basics 9

Solution: The number of different committees with 2 women and 3 men is

computed   
5 7 5! 7!
= · = (10)(35) = 350.
2 3 (3!)(2!) (4!)(3!)
If two of the men refuse to serve on a committee together, either these two men
are not in the group or only one is in the group. In that case, we have
       
2 5 5 2 5 5
+ = 300.
0 3 2 1 2 2
Exercises

1. Suppose an automobile license plate is designed to show an alphabet and this

an
is followed by an even 4-digit number. How many license plates can be issued?

lim
2. How many different 5-card hands can be chosen from a 52-card deck?

Di
3. Suppose you are to choose a basketball team of 5 players from 12 available
athletes.
UP
a) How many ways can you choose a team?
ic s,

b) How many ways can you choose a team composed of 2 guards, 2

at

forwards and a center?

em

4. 8 people are to line up in a bus.

h
at

a) In how many ways can they be lined up to get on a bus?

M

b) If 3 specific persons among 8, insist on following each other, how

of

many ways are possible?

te
itu

c) If 2 specific persons, among 8, refuse to follow each other, how many

st

ways are possible?

In

1.3 Discrete Probability Models

Let Ω be a countable set (either finite or countably infinite) and F = P (Ω). We then call
the probability space (Ω, F, P) a discrete probability model.
If A ⊆ Ω then !
[ X
P(A) = P {ωi } = P({ωi }).
ωi ∈A ωi ∈A

For brevity, we just write P(ωi ) instead of P({ωi }).

Chapter 1. The Basics 10

An important case of a discrete probability model is when the outcome space Ω is finite
and each outcome is equally likely, i.e.,
1
Ω = {ω1 , ω2 , . . . , ωn } and P(ωi ) = for all i ∈ {1, 2, . . . , n}.
n
|A|
Moreover, in such a model, if A ⊆ Ω then P(A) = n
, where |A| is the cardinality of A.
Examples

1. Consider the experiment of drawing a card from a normal deck of playing cards.
What is the probability of getting

1. a heart

an
2. an ace

lim
3. a face card

Di
4. the queen of spades
UP
Solution: Note that there are 52 cards (sans the joker).
13 1
s,

1. 52
= 4
ic

4 1
2. =
at

52 13
em

12 3
3. 52
= 13
h

1
4.
at

52
M

2. Consider the experiment of rolling two (regular) dice. What is the probability
of

of getting a sum of 5?
te
itu

Solution: Ω = {1, 2, 3, 4, 5, 6}2 , so |Ω| = 36. Let A be the event that the sum of
st

|A| 4
the dice is 5. So A = {(1, 4), (2, 3), (3, 2), (4, 1)}. Thus P(A) = = = 91 .
In

|Ω| 36

3. An urn contains 10 marbles, 6 of which are black while 4 are white. You are
to pick 2 marbles without replacement. What is the probability that you get 2
marbles of the same color?
Solution: Without explicitly stating what Ω is, we can just count |Ω| = 10 C2 =
10!
(8!)(2!)
= 45.
If A is the event of getting 2 marbles of the same color, then we can decompose
it as the union of two disjoint events, namely, B: the event of getting 2 white
marbles and C: the event of getting 2 black marbles. Hence

4 C2 6 C2 6 15 7
P(A) = P(B) + P(C) = + = + = .
10 C2 10 C2 45 45 15
Chapter 1. The Basics 11

Exercises

1. Consider 5-card hand dealt from a standard 52-card deck. Two important
events are A={you draw a flush} and B={you draw a straight}. Find P(A),
P(B) and P(A ∩ B).
2. Suppose you are at a party together with k other people, where k < 365.
Assuming that there are only 365 possible birthdays, what is the probability
that

a. somebody at the party has the same birthday as you?

b. at least two attendees (including yourself) have the same birthday?

an
3. Tickets numbered from 1 to 999 are to be raffled with only 1 winning ticket.

lim
What is the probability that the winning ticket is

Di
a. odd?
b. is greater than 500 and is even?
UP
c. has non-repeating digits?
ics,

d. is greater than 100, odd and has non-repeating digits?

at
h em

1.4 Conditional Probability

at
M

In most applications of probability, we are confronted with knowing the chances of an event
of

given that another event has happened. For example, in the experiment of tossing a coin
te
itu

twice, what are the chances of getting a second head, given that the first toss turned out
st

to be a tail?
In

Definition 1.13 Let (Ω, F, P) be a probability space. Let A and B be in F. Then the
conditional probability of A given B is defined by
P(A ∩ B)
P(A|B) = , (1.1)
P(B)

provided that P(B) > 0.

Theorem 1.14 Let (Ω, F, P) be a probability space.. If B ∈ F s.t. P(B) > 0 then the
function PB : F → [0, 1] s.t. PB (A) = P(A|B) for all A ∈ F is a probability measure.

The proof of this theorem is left as an exercise.

Chapter 1. The Basics 12

Examples:

1. Suppose that a coin is tossed twice. What is the probability that the second
toss is a tail given that the first toss is a head?
Solution: The sample space of this experiment is {HH, HT, T H, T T }. Let A
be the event of getting a tail on the second toss, and B the event of getting a
head on the first toss. Then the conditional probability of A given B is
P(A ∩ B) (1/4) 1
P(A|B) = = = .
P(B) (1/2) 2
2. What is the probability that a card drawn from an ordinary deck of playing
cards is a face card, given that the suit is spades?

an
Solution: Let A be the event of getting a face card, and B the event of getting

lim
spades. Then

Di
P(A ∩ B) (3/52) 3
P(A|B) = = UP = .
P(B) (13/52) 13
s,

An offshoot of the definition of the conditional probability is the next theorem, whose proof
ic
at

is very clear.
em

Theorem 1.15 (Multiplication Rule) Let (Ω, F, P) be a probability space. For A, B ∈

h
at

F, P(A ∩ B) = P(A|B)P(B).
M
of

The following theorem lists properties of the conditional probability.

te
itu

Theorem 1.16 Let (Ω, F, P) be a probability space and let A, B ∈ F and {Ai }∞
i=1 ⊆ F.
st

The following holds true.

In

1. P(∅|B) = 0 and P(Ω|B) = 1

∞
! ∞
[ X
2. If {Ai }∞
i=1 are mutually exclusive events, then P Ai B = P(Ai |B)
i=1 i=1
3. P(A0 |B) = 1 − P(A|B)
4. P(A1 |B) = P(A1 ∩ A2 |B) + P(A1 ∩ A2 0 |B)
5. P(A1 ∪ A2 |B) = P(A1 |B) + P(A2 |B) − P(A1 ∩ A2 |B)
6. If A1 ⊆ A2 then P(A1 |B) ≤ P(A2 |B)
∞
! ∞
[ X
7. For any collection {Ai }∞
i=1 we have P Ai B ≤ P(Ai |B)
i=1 i=1

The generalization of the Multiplication Rule (Theorem 1.15 ) is presented as follows.

Chapter 1. The Basics 13

Theorem 1.17 (Law of Total Probability) Let (Ω, F, P) be a probability space, I be a

[
countable index and {Bi }i∈I be a collection of mutually exclusive events such that Bi = Ω
i∈I
and P(Bi ) > 0 for every i ∈ I. Then for every A ∈ F,
X
P(A) = P(Bi )P(A|Bi ).
i∈I

Proof: Let A ∈ F. Define the events Ci = A ∩ Bi for all i ∈ I. Then C1 , C2 , . . . are all
mutually exclusive events. Furthermore,
!
[ [ [
Ci = (A ∩ Bi ) = A ∩ Bi = A.
i∈I i∈I i∈I

an
X X
Hence P(A) = (A ∩ Bi ) = P(Bi )P(A|Bi ). 

lim
i∈I i∈I

Di
Theorem 1.18 (Bayes’ Theorem) Let (Ω, F, P) be a probability space, I be a countable
UP [
index and {Bi }i∈I be a collection of mutually exclusive events such that Bi = Ω and
s,
i∈I
P(Bi ) > 0 for every i ∈ I. Then for every A ∈ F with P(A) > 0,
ic
at

P(Bk )P(A|Bk )
em

P(Bk |A) = X .
P(Bi )P(A|Bi )
h
at

i∈I
M

Proof:
of

P(Bk ∩ As) P(A|Bk )P(Bk )

te

P(Bk |A) = =
itu

P(A) P(A)
P(Bk )P(A|Bk )
st

= X .
In

P(Bi )P(A|Bi )
i∈I


Examples:

1. In a clothing factory, three machines, M1, M2, and M3, make 45%, 25%, and
30%, respectively, of the products. It is known from past experience that 3%,
2%, and 3% of the products made by each machine, respectively, are defective.
Now, suppose that a finished product is randomly selected. If a product was
chosen randomly and is found to be defective, what is the probability that it
was made by machine M1?
Chapter 1. The Basics 14

Solution: Define the following events:

A = product is defective
B1 = the product is made by machine M1
B2 = the product is made by machine M2
B3 = the product is made by machine M3

Using Theorem 1.17, we can compute for the probability that a randomly chosen
product turns out to be defective:

P(A) = P(B1 )P(A|B1 ) + P(B2 )P(A|B2 ) + P(B3 )P(A|B3 )

an
= (0.45)(0.03) + (0.25)(0.02) + (0.3)(0.03) = 0.0275.

lim
Di
Consequently, by Theorem 1.18,
UP
P(B1 )P(A|B1 )
P(B1 |A) =
P(B1 )P(A|B1 ) + P(B2 )P(A|B2 ) + P(B3 )P(A|B3 )
s,

(0.45)(0.03)
ic

= = 0.4909.
at

0.0275
h em
at

2. Suppose that 35% of Canadians play hockey. Suppose further that 20% of peo-
M

ple in the USA and 15% of people in Mexico play hockey and none for the rest
of

of the 38 North American countries and territories. Canada has 6% of the pop-
te
itu

ulation in North America, USA has 57%, and Mexico has 21%. If a randomly
st

chosen North American person plays hockey, what is the probability that he is
In

from Canada?

Solution: Let A denote the event that a person plays hockey, and B1 , B2 , and B3
denote the events that a person is from Canada, USA, and Mexico, respectively.
Then

P(A|B1 ) = 0.35
P(A|B2 ) = 0.20
P(A|B3 ) = 0.15
P(A|Bi ) = 0 for i = 4, 5, . . . , 41.
Chapter 1. The Basics 15

Applying Bayes’ Theorem, the probability of randomly choosing a Canadian

given that he is a hockey player is given by
P(A|B1 )P(B1 )
P(B1 |A) =
P(B1 )P(A|B1 ) + P(B2 )P(A|B2 ) + P(B3 )P(A|B3 )
(0.35)(0.06) 0.021
= = ≈ 0.1261.
(0.35)(0.06) + (0.20)(0.57) + (0.15)(0.21) 0.1665

Exercises

1. A biased die is tossed in an experiment. The die is biased in such a way that
odd numbers have the same probablity of occurence, even numbers have the

an
same probability of occurence, but an even number is twice as likely to come

lim
up as an odd number.

Di
a. Find the probability that the outcome is a perfect square.
UP
b. Suppose the die is tossed and is known that the result will be greater
than 3. What is the probability that the outcome is a perfect square?
s,
ic
at

2. A box contains 4 black balls and 6 white balls. A sample of of size 3 is drawn
em

from the box without replacement. What is the probability that 1 black ball
h

and 2 white balls are drawn, given that at least 2 of the balls in the sample are
at
M

white?
of

3. A small beverage company has 2 bottling machines. Machine A produces 60%

te

of the bottles and machine B produces 40%. Two out of every 20 bottles filled
itu

by machine A is rejected for some reason, while 3 out of 30 bottles from machine
st

B is rejected.
In

a. What portion of the bottle is rejected?

b. What is the probability that a bottle is made from machine A given
that it is accepted?

4. Suppose we have 2 boxes. In box 1, there are 5 sealed envelopes. 3 of them

contain P1 coins and 2 of them contain P5 coins. In box 2, there are 10 sealed
envelopes; 7 of them contain P1 coins and 3 contain P5 coins. If a box is selected
at random and from it an envelope is chosen, what is the probability that it
contains a P5 coin?
5. An insurance company estimates that 40% of policyholders who have only a car
policy will renew next year, 60% of policyholders who have only a homeowners
Chapter 1. The Basics 16

policy will renew next year, and 80% of policyholders who have both policies
will renew at least one of those policies next year. Company records show that
65% and 50% of policyholders have a car and a homeowners policy, respectively,
and 15% have both. Calculate the percentage of policyholders that will renew
at least one policy next year.
6. Suppose that 80% of the seniors, 70% of the juniors, 50% of the sophomores,
and 30% of the freshmen of a college use the library of their campus frequently.
If 30% of all students are freshmen, 25% are sophomores, 25% are juniors, and
20% are seniors,

a. what percent of all students use the library frequently?

an
b. what is the probability that a randomly selected student who uses

lim
the library frequently is a junior?

Di
7. A clinical study tracked a group of persons for 6 years. At the beginning, 35%
UP
were classified as light smokers, 20% as heavy smokers, and 45% as nonsmok-
s,

ers. Results of the study showed that light smokers were twice as likely as
ic

nonsmokers to die during the six-year study, but only one third as likely as
at
em

heavy smokers. Calculate the probability that a randomly selected participant

h

from the study died over the six-year period.

at
M

8. Team-Ang and Team-Ba were assigned to design a certain project. The proba-
bility that Team-Ang’s design will be successful is 32 , for Team-Ba it is 12 . If the
of
te

probability that at least one of the 2 designs is successful is 34 , find the prob-
itu

ability that Team-Ba designed the project, given that it was a success. [Hint:
st

Assume that there is a chance of having 2 successful designs]

In
Chapter 1. The Basics 17

1.5 Independence
One of the important assumptions when dealing with probabilities is that the events are
independent of each other. This assumption, though in a few cases oversimplifies the
problems, is widely used.

Definition 1.19 Two events A and B are said to be independent of each other if at least
one of the following is satisfied:

1. P(A ∩ B) = P(A) P(B);

2. P(A|B) = P(A)
3. P(B|A) = P(B)

an
lim
If A and B are not independent, the events are said to be dependent.

Di
Remark 1.20 For independent events A and B, with P(B) > 0, conditioning on B does
UP
not change the probability of occurrence of event A.
ics,

Theorem 1.21 Suppose A and B are two independent events defined on a probability space
at

(Ω, F, P). Then the following are also independent:

h em

i. A0 and B
at
M

ii. A and B 0
of

iii. A0 and B 0
te
itu

Proof: Note that B can be written as a union of two disjoint sets, i.e., B = (A∩B)∪(A0 ∩B).
st

Then
In

P(B) = P(A ∩ B) + P(A0 ∩ B).

This implies

P(A0 ∩ B) = P(B) − P(A ∩ B)

= P(B) − P(A)P(B)
= P(B)(1 − P(A))
= P(B)P(A0 ).


Proofs for items ii. and iii. are left as an exercise.
For more than two events, the definition of independence is slightly complex.
Chapter 1. The Basics 18

Definition 1.22 Given a probability space (Ω, F, P), the events A1 , A2 , . . . , An in F are
(mutually) independent if for any subset {Ak1 , Ak2 , . . . , Akm } of {A1 , . . . , An },
m
! m
\ Y
P Aki = P(Aki ).
i=1 i=1

Moreover, the events A1 , A2 , . . . , An are said to be pairwise independent if

P(Ai Aj ) = P(Ai )P(Aj ), i 6= j.

Theorem 1.23 Suppose A1 , A2 , . . . , An are independent events defined on a probabilty

space (Ω, F, P). Then their complements are independent and any combination of the events
and complements of events are independent.

an
Theorem 1.24 Suppose A1 , A2 , . . . , An are independent events defined on a probability

lim
space (Ω, F, P). Then !
n n

Di
[ Y
P Ai =1− UP[1 − P(Ai )] .
i=1 i=1
n n
s,
[ \
Proof: Let E = Ai . Then E C = AC
i . So,
ic

i=1 i=1
at

n
!
em

\
P(E) = P(Ω) − P AC
i
h

i=1
at

= 1− (P(A1 )P(AC
C C
2 ) · · · P(An ))
M

= 1 − [(1 − P(A1 ))(1 − P(A2 )) · · · (1 − P(An ))]

of

n
te

Y
= 1− [1 − P(Ai )] .
itu

i=1
st


In

Examples:

1. Consider 2 tosses of a die where

A : event of getting an odd on first toss

B : event of getting a number greater than or equal to 3 on second toss

Are A and B independent?

Solution: Take Ω = {1, 2, 3, 4, 5, 6}2 . Note that

A = {1, 3, 5} x {1, 2, 3, 4, 5, 6}
B = {1, 2, 3, 4, 5, 6} x {3, 4, 5, 6}
A ∩ B = {1, 3, 5} x {3, 4, 5, 6}
Chapter 1. The Basics 19

18 1 24 2
Then P(A) = = , P(B) = = . Moreover,
36 2 36 3
12 1
P(A ∩ B) = = = P(A)P(B).
36 3
Hence, A and B are independent.

2. Consider the experiment of tossing two dice. Let A denote the event of an odd
total, B the event of an ace on the first die, and C the event of a total of seven.
Determine if the following are independent of each other:

a. A and B
b. A and C

an
c. B and C

lim
1
Solution: Calculating probabilities, we have P(A|B) = 2
= P(A), P(A|C) =

Di
1 1
1 6= P(A) = 2
and P(C|B) = 6
= P(C). Then A and B are independent, A and
UP
C are dependent, and B and C are independent.
s,

3. Let A1 denote the event of an odd face on the first die, A2 the event of an
ic
at

odd face on the second die, and A3 the event of an odd total in the random
em

experiment that consists of tossing two dice. Are A, B, and C independent?

h

1
= P(A1 ∩A2 ), P(A1 )P(A3 ) =
at

Solution: Calculating probabilities, P(A1 )P(A2 ) = 4

M

1 1
4
= P(A1 ∩ A3 ), and P(A2 )P(A3 ) = 4
= P(A2 ∩ A3 ). These imply that
of

1
the events are pairwise independent. However, P(A1 )P(A2 )P(A3 ) = 8
6= 0 =
te

P(A1 ∩ A2 ∩ A3 ). Hence they are dependent.

itu
st

Exercises
In

1. Using a standard deck of cards, what is the probability of

a. drawing 2 cards of the same color?

b. drawing 5 cards of the same color?
c. drawing 5 cards composed of only royal cards (J-Q-K)?

2. Three fair dice are thrown once. Given that no two show the same face,

a. What is the probability that the sum of the faces is 7?

b. What is the probability that exactly one die lands in 1?
c. What is the probability that at least one of the dice lands in 1?
Chapter 1. The Basics 20

3. A couple plans to have three children. Suppose that there are equal chances for
having a boy or a girl, and that the sex of each child is independent from one
another. What is the probability of

a. having 3 boys?
b. 2 girls and 1 boy?
c. having the last two children to be both girls, given that the 1st child
is a boy?

4. Three fair dice are tossed independently. Let Ei be the event that the ith die
results in a 6. Find P(E1 ∪ E2 ∪ E3 ).
5. A ball is drawn at random from a box containing 10 balls numbered sequentially

an
from 1 to 10. Let X be the number on the ball selected, A be the event that X

lim
is an even number, B be the event that X ≥ 6, and C the event that X ≤ 4.

Di
Which of the pairs A and B, A and C, and B and C are independent?
UP
a. A and B only
s,

b. A and C only
ic
at

c. B and C only
em

d. A & B and A & C only

h
at

e. all pairs
M

6. Show that if two independent events A and B have positive probabilities, then
of

A and B can not be disjoint.

te

7. Consider the experiment of tossing 2 dice. Let

itu

A: event that the sum of the 2 dice is 7

st
In

B: event that the 1st die lands in 4

C: event that the 2nd die lands in 3
Show that A, B and C are pairwise independent but are not mutually indepen-
dent.
8. Give an example of events A, B and C s.t.

P(ABC) = P(A) · P(B) · (C),

but the three events are not mutually independent.