PROBABILITY

PROBABILITY THEORY

How to deal with uncertainty in a rational and consistent manner?

• Probability: A numerical measure of the likelihood or

chance that a particular event or outcome will occur.

Eg. If one token is to be picked from a bag containing 2 red

tokens and 3 blue tokens, what is the chance of picking a
red token?

There are 5 tokens in the bag.

2 of them are red.
The chance of picking a red is
2/5 = 0.4

2
 HOW TO ASSIGN PROBABILITIES TO EVENTS?

There are three possible approaches.

1. Classical approach: uses logic or mathematics.

• A standard deck of 52 cards has four aces. What is the probability that
the first card drawn is an ace?

– 4 aces in a pack of 52 cards.

Probability the next card will be an ace is 4/ 52.

– If the first card drawn was an ace, what is the probability the
second card drawn will also be an ace?
There are 3 aces remaining within the 51 cards
currently in the deck. Therefore, probability is 3/ 51.

3
2. Relative frequency (statistical) approach:
It makes use of relatively frequencies
calculated from large sets of observations from
experiments or surveys.

Lifetime (hours) Number of Relative

batteries Frequency
13 – (<) 14 11 0.055
14 – (<) 15 90 0.450
15 – (<) 16 86 0.430
16 – (<) 17 13 0.065

4
 3. Subjective approach: It makes use of personal intuition, previous
experience or general feelings.
It is used when neither the classical nor the
relative frequency approach is applicable.

Since subjective probabilities reflect personal degrees of belief, they are

likely to be biased and inaccurate.

E.g.:
Consider the statement ‘this horse’s chance of winning in its race next Saturday
is 85 percent’.
It doesn’t mean that the horse has won 85 percent of the races that it has run,
or that if this race was run 100 times this horse would win it 85 times.
This statement is ‘meaningless’, except that it gives an indication of how a
given individual ranks the prospects of different outcomes.

5
 LAWS OF PROBABILITY
– Probabilities must satisfy two basic requirements:
i. Each probability value must be non-negative and not greater
than one.
ii. The sum of these probabilities must be one.

If event A is the collection of k of these n equally likely possible

outcomes, then the chance that this event occurs is

k
Probability of event A P(A) 
n

If A is an impossible event, i.e. it never happens, then k is zero and

P(A) = 0.

If A is a certain event, i.e. it always happens, then k = n and

P(A) = 1.

6
 – Since 0  k  n the 0  P(A)  1 inequalities are satisfied.

If A is an impossible event, If A is a certain event, i.e. it

i.e. it never happens, then k is always happens, then k = n
zero and P(A) = 0. and P(A) = 1.

Ex 1:
Suppose two questions from a sample survey are ‘Age group’ and ‘Response’
to the proposition that gambling facilities should not be permitted to be open 24
hours per day.

The answers are summarized in the following contingency table (cross-

tabulation):

7
 Two questions in a survey are ‘Age group’ and ‘Response’ to the proposition
that gambling facilities should not be permitted to be open 24 hours per day.

10 respondents are under 25 and 60 respondents are

agree with the statement. under 25.

Agree Neutral Disagree Total

Under 25 10 20 30 60

25 - 55 25 15 40 80

Over 55 45 10 5 60

Total 80 45 75 200

80 respondents agree with 200 respondents in the

the statement. sample, i.e. n =200.
The red numbers are joint frequencies, blue numbers are marginal frequencies.

8
From these frequencies (f ) we can calculate joint and marginal relative
frequencies (f/n), …

Agree Neutral Disagree Total

Under 25 10 20 30 60

25 - 55 25 15 40 80
60 out of 200
Over 55 45 10 5 60 are under 25.
ie. 60/200
Total 80 45 75 200

10 out of 200 are under 25 80 out of 200

and agree.ie. 10/200 agree ie. 80/200

.. these relative frequencies are joint and marginal probabilities.

The chance that two events The chance of the occurrence

occur simultaneously. of a single event.
9
Let U and A denote the events ‘under 25’ and ‘Agree’, respectively.

P(U) = 60/200 = 0.30 or 30%

There is a 30% chance that a person randomly
selected from the population is under 25.

P(A) = 80/200 = 0.40 or 40%

There is a 40% chance that a person randomly
selected from the population agrees.

P(U and A) = 10/200 = 0.05 or 5%

There is a 5% chance that a person randomly
selected from the population is under 25 and
agrees.

AND: both events happen, i.e.

U A

10
OR: at least one of the events happen, i.e. either U or A or both.

P(U or A)
= P (U) + P (A) – P (U and A)
U A
= 60/200 + 80/200 – 10/200
= 0.65 or 65%

There is a 65% chance that a person randomly

selected from the population is under 25 or agree.

Agree Neutral Disagree Total

130 out of 200 are
Under 25 10 20 30 60 under 25 or agree.

25 - 55 25 15 40 80
P(U or A)
Over 55 45 10 5 60 = 130/200
Total 80 45 75 200 = 0.65
or 65%

11
In general,
• The probability of the union of events, A and B:
P( A or B)  P( A)  P( B)  P( A and B) Addition rule

• A and B are said to be mutually exclusive events if they cannot occur

together.

No overlapping.

A B
A and B, do not share any
possible outcome, so they are
mutually exclusive.

Their joint probability is zero, P(A and B) = 0.

E.g.: P(under 25 and over 55)= 0.

It does not make sense,

it cannot happen.
12
 • A is the complement of “NOT A” and they are mutually exclusive.

A A is the complement of A
(meaning not A).
A

Since P(A) + P( A ) = 1, the probability of A complement is

P( A)  1  P( A) Complement rule

E.g.: Since in the sample 45 respondents have ‘neutral’ opinion,

P(N) = 45/200 = 0.225 (22.5%) and P( N )= 1 – 0.225 = 0.775 (77.5%).
There is 77.5% chance that a person randomly selected from
the population does not have ‘neutral’ opinion.

Sometimes it may be difficult to calculate a particular probability directly, but it

might be easy to calculate its complement.
13
Conditional Probability
• If we have partial knowledge we must use this information when finding
the probability of an event. Eg. If we know the person is over 55.

Conditional probability of A is the probability of A given B has occurred.

It is written as P(A given B) or P(A | B).

Agree Neutral Disagree Total

45, out of the 60
Under 25 10 20 30 60 respondents over
55, agree.
25 - 55 25 15 40 80

Over 55 45 10 5 60 P(A given O)

= 45/60 = 0.75
Total 80 45 75 200 or 75% of …

14
• The conditional probability of event A, given B:

P( A and B)
P( A | B)  (Refer example on previous slide)
P( B)

• From this definition we can derive the following formula for the joint
probability of events A and B :
P( A and B)  P( A | B)  P( B) Multiplication rule

• A and B are said to be independent events if knowing one of them

occurred has no effect on the outcome of the other, i.e.
P( A | B)  P( A) and P( B | A)  P( B)
Otherwise, A and B are dependent events.

If A and B are independent

P( A and B)  P( A)  P( B) Multiplication rule for

independent events
15
E.g.:
 We have found that P(A) = 0.40 and P(A | O) = 0.75.
Since they are not equal, A and O are statistically dependent events.
 A : person weighs more than 80 kilos; B : person is over 2 metres tall.
A and B are almost certainly not independent because very tall people
weigh more on average than others.
 A : person weighs more than 80 kilos; C : person has blue eyes.
A and C are almost certainly independent because blue eyed people
seem unlikely to weigh more on average than others (or less).

Ex 2:
An international aerospace company has submitted bids on two separate
federal government defence contracts, A and B. The company feels that it has
a 60% chance of winning contract A and a 30% chance of winning contract B.
Given that it wins contract B, the company believes it has an 80% chance of
winning contract A.
P(B) = 0.3 P(A | B) = 0.8
P(A) = 0.6

16
a) What is the probability that the company will win both contracts?
P(A and B) = ? Using the multiplication rule
P( A and B)  P( A | B)  P( B)  0.8  0.3  0.24

b) What is the probability that the company will win at least one of the two
contracts?
P(A or B) = ? Using the addition rule
P( A or B)  P( A)+P( B)  P( A and B)  0.6  0.3  0.24  0.66

c) If the company wins contract B, what is the probability that it will not win
contract A?
P(A | B) = ? Using the complement rule
P( A | B)  1  P( A | B)  1  0.8  0.2

17
Ex 3:
An investment analyst collects data on stocks and notes whether or not
dividends were paid and whether or not the stocks increased in price over the
last twelve months.
Let D : dividends were paid; S : stock price increased.

S S Total

D 35 80 115

D 85 50 135

Total 120 130 250

Assume that a stock is randomly selected from this list.

a) What is the probability that it increased in price?
120
P( S )   0.48
250
18
b) What is the probability that it paid dividends?
115
P( D)   0.46
250

c) What is the probability that it increased in price and paid dividends?

35
P( S and D)   0.14
250

d) What is the probability that it neither paid dividends nor increased in price?
50
P( D and S )   0.20
250

e) Given that the stock increased in price, what is the probability that it also
paid dividends?
35
P( D | S )   0.29
120

19
f) If a stock is known not to have paid dividends, what is the probability that it
increased in price?
85
P( S | D)   0.63
135

g) Are the events ‘paid dividends’ and ‘increased in price’ mutually exclusive?
In this sample D and S occurred together 35 times
P(D and S) =35/250 = 0.14 is not equal to zero, so D and S are not
mutually exclusive

h) Are the events ‘paid dividends’ and ‘increased in price’ independent?

P(D)=0.46 and P(D |S)= 0.29 are different, so D and S are dependent
events.

Alternatively, we can arrive at the same answer by recognising that

P(S)×P(D) = 0.48 × 0.46 = 0.22 is not equal to P(S and D) = 0.14.

20
 The Concept of Expected Values
(An application of probabilities)

If X is a discrete random variable, such as the revenue from a farm, then the
farmer’s expected revenue is:
N
E( X )   xi p ( xi )
i 1

where xi are the values of the random variable X, as in the revenues (say under
different weather conditions) and p(xi) are the associated probabilities.

If the probability of bad weather is 0.7 and that of good weather is 0.3 and the
forecast revenue in bad weather is $1m and the forecast revenue in good
weather is $10m, the farmer’s expected revenue is:

E( X )  1 0.7  10  0.3  $3.7m

The farmer should keep expenses to less than $3.7m in order to expect a profit.

21