Chapter Three

Elementary Probability Theory

Probability Experiment
 An action, or trial, through which specific results
(counts, measurements, or responses) are obtained.
Outcome
 The result of a single trial in a probability experiment.
Sample Space
 The set of all possible outcomes of a probability experiment.
1
 Elementary Probability Theory

Event
 Consists of one or more outcomes and is a subset of the
sample space.
 Probability experiment (Example 1): Roll a die
 Outcome: {3}
 Sample space: {1, 2, 3, 4, 5, 6}
 Event: {Die is even}={2, 4, 6} 2
 Elementary Probability Theory

Example 2: A probability experiment consists of tossing a

coin and then rolling a six-sided die. Describe the sample
space.
Solution:
 There are two possible outcomes when tossing a coin : a
head (H) or a tail (T) and six possible outcomes when
rolling a die: 1, 2, 3, 4, 5, or 6.
3
 Elementary Probability Theory

 Then, the sample space S has 12 outcomes.

S={H1, H2, H3, H4, H5, H6, T1, T2, T3, T4, T5, T6}
Probability
 Used to make decisions in the face of uncertainty.
 probability is used in many
different areas
 Like insurance, science, marketing and many other areas.
4
 Elementary Probability Theory

Probability of an event
 The theoretical probability of an event E is defined as the
number of ways the event can occur divided by the
number of events of the sample space S.
 Mathematically

𝑛(𝐸)
𝑃 𝐸 =
𝑛(𝑆)
5
 Elementary Probability Theory

Probability of an event
Example: Probability of a sum of 7 when two dice are rolled.
Solution:
 we know that there are 36 possible sums that can occur
when two dice are rolled.
 Then, the sample space {(1,6), (6,1), (2,5), (5,2), (4,3),
(3,4)} shows that there are 6 possible ways to get a sum of 7.
6
 Elementary Probability Theory

Probability of an event
𝑛(𝐸) 6 1
Solution: Therefore, 𝑃 𝐸 = = =
𝑛(𝑆) 36 6

Some properties of probability

 𝟎≤𝑷 𝑬 ≤𝟏
 𝑷 𝑬𝟏 + 𝑷 𝑬 𝟐 + 𝑷 𝑬𝟑 + ⋯ 𝑷 𝑬𝒏 = 𝟏

7
 Elementary Probability Theory

Exercise: A quiz contains a multiple choice question with five

possible answers, only one of which is correct. A student
plans to guess the answer.
a) What is sample space?
b) Assign probabilities to the simple events
c) Probability student guesses the wrong answer
d) Probability student guesses the correct answer.
8
 Elementary Probability Theory

 Generally, there are three approaches to assign

probabilities.
1) Classical approach
2) Relative Frequency
3) Subjective Approach
(Assignment: Read about each of them and report to the
class).
9
 Elementary Probability Theory

Conditional probability
 Definition: Let A and B be two events. The conditional
probability of A given B, denoted by 𝑃 𝐴 𝐵 , is defined as

𝑃(𝐴 ∩ 𝐵)
𝑃 𝐴𝐵 =
𝑃(𝐵)
Example:

10
 Elementary Probability Theory

Solution:
 Let A be the event that a family with two children has two boys, and
let B be the event that a family with two children has at least one
boy. Then, it follows that
 𝐴 = *𝐵𝐵+, and 𝐵 = 𝐵𝐵, 𝐺𝐵, 𝐵𝐺 , and 𝐴 ∩ 𝐵 = 𝐵𝐵
 Because the four possibilities are equally likely, it follows that
3 1 𝑃(𝐴∩𝐵) 1/4
𝑃 𝐵 = , and P A ∩ 𝐵 = , then 𝑃 𝐴 𝐵 = = = 1/3.
4 4 𝑃(𝐵) 3/4

11
 Elementary Probability Theory

Independent events
Definition: The events A and B are independent if and only
if P A ∩ 𝐵 = P A P B .
Example: consider an example in conditional probability
Family with two children, A and B are they independent?
Solution: No!
1 3
B/c 𝐏 A ∩ 𝐵 = ≠ =P A P B . 12
4 16
 Elementary Probability Theory

Random variables and expectation

Random variable
Definition: A random variable (RV) is a function from the
sample space of an experiment to the set of real numbers.
That is, a RV assigns a real number to each possible outcome.
Note: Note that a RV is a function. It is not a variable, and it
is not random!
13
 Elementary Probability Theory

Random variable
Example: Suppose that a coin is flipped three times. Let 𝑥(𝑡)
be the random variable that equals the number of heads that
appear when 𝑡 is the outcome. Then 𝑥(𝑡) takes on the
following values:
X 𝐻𝐻𝐻 = 3, X 𝐻𝐻𝑇 = 𝑋 𝐻𝑇𝐻 = 𝑋 𝑇𝐻𝐻 = 2
X 𝑇𝑇𝐻 = 𝑋 𝑇𝐻𝑇 = 𝑋 𝐻𝑇𝑇 = 1, 𝑋 𝑇𝑇𝑇 = 0
14
 Elementary Probability Theory

Expectation
 Let 𝑋 be a random variable on a probability space
S= 𝑠1 , 𝑠2 , 𝑠3 , … , 𝑠𝑛 .
 Then the mean or expectation of 𝑋 is denoted and defined
by:
 𝜇 = 𝐸 𝑋 = 𝑋(𝑠1 )𝑃 𝑠1 + 𝑋(𝑠2 )𝑃 𝑠2 + 𝑋(𝑠3 )𝑃 𝑠3 +
𝑛
⋯ + 𝑋(𝑠𝑛 )𝑃 𝑠𝑛 = 𝑘=1 𝑋(𝑠𝑘 )𝑃 𝑠𝑘 . (For discrete case)
15
 Elementary Probability Theory

Expectation
𝑛
 𝜇=𝐸 𝑋 = 𝑥𝑓
𝑖=1 𝑖
𝑥𝑖 . (For continuous case)
Example: Suppose a fair coin is tossed six times. The
number of heads which can occur with their respective
probabilities follows:
𝑥𝑖 0 1 2 3 4 5 6
P(𝑥𝑖 ) 1/64 6/64 15/64 20/64 15/64 6/64 1/64

𝜇 = 3. (How!? ) 16
 Elementary Probability Theory

Expectation
Exercise: Three horses a, b, and c are in a race; suppose their
1
respective probabilities of winning are ½, 1/3, and . Let X
6

denote the payoff function for the winning horse, and

suppose X pays $2, $6, or $9 according as a, b, or
c wins the race. Find the expected payoff for the race.
(Assignment: Try to have an example in Continuous case17)