Probability Theory and Statistics

Lecture 1: Random events and their probabilities

2 September 2019

Lecturer: Mikhail Zhitlukhin (Михаил Валентинович Житлухин)

Email: icef.statistics@gmail.com
 About this course
Part 1: Probability theory
We will study operations with random events and related concepts.

Part 2: Statistics
We will study how describe data and make conclusions from data.

1/23
Structure of the course

• Weekly lectures, classes, home assignments

• 4 written exams
Exams include problems and theoretical questions

• Grade determination: see the syllabus on Icef-Info

• Textbooks
• Wonnacott. Introductory Statistics for Business and Economics
• Peresetsky, Katyshev. Problem sets with solutions
• Newbold, Carlson, Thorne. Statistics for Business and Economics.
• Other books: Icef-Info → Textbooks

2/23
 The mathematical model of a random experiment
Some examples

1. Flipping a coin
Outcomes: H, T Probabilities: 1/2

2. Rolling a die
Outcomes: 1, 2, 3, 4, 5, 6 Probabilities: 1/6

3. Passing an exam
Outcomes: pass, fail Probabilities: 0.7, 0.3 (for example)

3/23
Probability space
The mathematical model of an experiment with a random outcome is
a probability space, which consists of:

• A set of all possible outcomes Ω = {ω1 , ω2 , . . .} (“sample space”)

These are all the possible distinct results of the experiment;
only one of them will occur.

• Probability P (ω) of the outcomes ωi :

• 0 6 P (ω) 6 1 – the probability of an outcome is between 0 and 1
• P (ω1 ) + P (ω2 ) + . . . = 1 – the total probability is 1

Usual notation: pi = P (ωi )

In probability theory, P (ω) are assumed to be given.

4/23
 Random events
A sample space Ω = {ω1 , ω2 , . . .} contains all possible outcomes.
A random event is some subset of Ω.
The probability of a random event is the sum of the probabilities of the
possible outcomes that constitute it.
For example, if A = {ω1 , ω2 , ω5 }, then P (A) = p1 + p2 + p5 .

5/23
Examples

1. Odd number of points on a die

2. Two heads when flipping 3 coins

3. A borrower fails to repay a debt

4. The price of shares of stock of a company will rise by 5% in a month

6/23
Special random events
The following are also random events:

• A random event consisting of only one possible outcome, A = {ωi }

• The sample space Ω = {ω1 , . . . , ωn }

P (Ω) = 1

• The null event A = {} which contains no outcomes (an “impossible

event”). Notation: ∅.
P (∅) = 0

7/23
 Operations with random events
The intersection of A and B is the event “A and B” which happens
whenever both A and B happen. Notation: A ∩ B.
A ∩ B consists of all outcomes that are both in A and B.

A A∩B B

8/23
The union of A and B is the event “A or B” which happens whenever
at least one of the events A or B happens. Notation: A ∪ B.
A ∪ B consists of all outcomes that are at least in one of A or B.

A∪B

A B

9/23
The difference A \ B is the event “A but not B” which happens whenever
A happens but B does not happen.
A \ B consists of all outcomes that are in A but not in B.

A\B

A B

10/23
The complement of A is the event “not A” which happens whenever A
does not happen. Notation: A.
A consists of all outcomes that are not in A.

11/23
Operations with many events

A∪B∪C A∩B∩C

A A

B C B C

Ω Ω

12/23
Relations between random events
Events A and B are called disjoint (or mutually exclusive) if A ∩ B = ∅.
“A and B cannot happen at the same time.”

A B

13/23
A is a subevent (or a subset) of B if B contains all outcomes from A.
Notation: A ⊂ B.
“If A happens then B happens.”

14/23
 Formulas
1. Probability of union and intersection
For any two random events A, B

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
P (A ∩ B) = P (A) + P (B) − P (A ∪ B)

Proof:
A∩B
A B
x + y + z = (x + y) + (y + z) − y
| {z } | {z } | {z } |{z} x y z
A∪B A B A∩B

for x, y, z being the areas on the picture

15/23
2. Probability of disjoint events
If A and B are disjoint then

P (A ∪ B) = P (A) + P (B)

More generally, if A1 , . . . , An are disjoint (any two events Ai , Aj are

disjoint), then

P (A1 ∪ A2 ∪ . . . ∪ An ) = P (A1 ) + P (A2 ) + . . . + P (An )

3. Probability of a complementary event

For any random event A

P (A) = 1 − P (A)

16/23
4. Other useful formulas

• If A ⊂ B, then P (A) 6 P (B)

“The probability of a subevent is not greater than of the event”

• P (Ω) = 1
“Something will definitely happen”

• P (∅) = 0
“The probability of an impossible event is 0”

17/23
Example
A student will pass one exam with probability 0.5, and pass another exam
with probability 0.7. He will pass both the exams with probability 0.4.
What is the probability he will fail both the exams?
Answer: P (fail both) = 0.2

18/23
 Bonus
1. Some Greek letters
Aα alpha Ππ pi
Bβ beta Pρ rho
Eε epsilon Σσ sigma
θθ theta Φϕ phi
Λλ lambda Xχ chi
Mµ mu Ωω omega

19/23
2. The Monty Hall problem

3 doors, behind one of them is the prize, and nothing behind the other two.
You select a door, and Monty opens one of the other two doors without the
prize. He asks you if you want to change your selection. What will you do?

20/23
Strategy “change”: P (win) = 2/3
ω You Monty You Result
1 1 2 or 3 3 or 2 lose
2 1 3 2 win
3 1 2 3 win

Strategy “don’t change”: P (win) = 1/3

ω You Monty You Result
1 1 2 or 3 1 win
2 1 3 1 lose
3 1 2 1 lose

21/23
3. The inclusion-exclusion formula

P (A1 ∪ A2 ∪ A3 ) = P (A1 ) + P (A2 ) + P (A3 )

− P (A1 ∩ A2 ) − P (A1 ∩ A3 ) − P (A2 ∩ A3 )
+ P (A1 ∩ A2 ∩ A3 )

P (A1 ∩ A2 ∩ A3 ) = P (A1 ) + P (A2 ) + P (A3 )

− P (A1 ∪ A2 ) − P (A1 ∪ A3 ) − P (A2 ∪ A3 )
+ P (A1 ∪ A2 ∪ A3 )

and similarly for any number of events (see Durrett, §2.5.1).

22/23
 Reading
Basic reading
• Wonnacotts: § 3.1, 3.2, 3.3
• Newbold, Carlson, Thorne: § 3.1

Detailed reading
• Bertsekas, Tsitsiklis: § 1.1, 1.2 (skip “Continuous models”)
• Durret: § 1.1

23/23