Bayes’ Rule

A Short Lecture by Alex Robinson

Learning Objectives

1. Learn what Bayes’ Rule is.

2. Understand why Bayes’ Rule is important.

3. Be able to calculate an updated probability using
Bayes’ Rule.
 What is Bayes’ Rule?

• Suppose that we have a belief about the weather tomorrow: there is a 50%
chance that it will rain.
• The weather forecast says that it will rain tomorrow.
• How should we update our initial prediction about tomorrow’s weather?
What is Bayes’ Rule?

• Let’s
  call our initial prediction P(A) (called the prior), where A represents
rain tomorrow.
• We want to know P(A|B) (called the posterior), or the probability of rain
given what the forecast said.
• B represents the forecast, and P(B|A) measures how accurate the forecast
usually is.
Why does Bayes’ Rule matter?

• Any time we make a prediction or perform a test, Bayes’ Rule is important! Weather
forecasts, medical tests, DNA tests, etc.
• PROBLEM: Often, the accuracy of the test is confused with the updated probability.
• Example: If you test positive forP(A|B)
Covid,≠ does
P(B|A)that always mean you have Covid?
• Example: Brandon Mayfield and the 2004 Madrid Train Bombings.

  𝑃 ( 𝐵| 𝐴 ) 𝑃 ( 𝐴 )
𝑃 ( 𝐴|𝐵 ) =
𝑃 ( 𝐵)
 True Positives and True Negatives

• Sometimes, tests or predictions are not correct.

• 2 ways of being incorrect: False Positives (I predict it will rain and it doesn’t); and False Negatives (I predict
it won’t rain and it does)
It Rains It Doesn’t Rain
Forecast Rain True Positive False Positive
Forecast No Rain False Negative True Negative

• We can measure how good a test is with the True Positive Rate (Sensitivity) and the True Negative Rate
(Specificity)
• True Positive Rate = (Total # of True Positives)/(Total # Days it Rained)
• True Negative Rate = (Total # of True Negatives)/(Total # Days it Didn’t Rain)
• True Positive and True Negative Rates don’t tell the whole story… also need to know how often it rains, on
average
Updating Probabilities: Strep Throat

• Suppose you have a sore throat and take a rapid strep throat test.
• A rapid strep throat test will read positive for about 86% of strep throat
cases.
• It will read negative for 95% of non-strep throat cases.
• On average, about 15% of sore throats are due to strep.
• Assuming you test positive, how likely is it that you have strep?
Strep Throat Example

• Imagine 10,000 patients with sore throats take a strep throat test.

Strep Throat No Strep Throat Totals

Test Positive True Positives False Positives Total Positives
Test Negative False Negatives True Negatives Total Negatives
Totals Total Strep Total No Strep Total Patients
Strep Throat Example

• On average, 15% of the patients have strep throat: 0.15 x 10,000 = 1,500
Strep Throat No Strep Throat Totals
Test Positive True Positives False Positives Total Positives
Test Negative False Negatives True Negatives Total Negatives
Totals 1,500 8,500 10,000
Strep Throat Example

• The test has a True Positive Rate of 86% - so 86% of the 1,500 Strep Throat
cases will test positive.
0.86 x 1,500 = 1290. This means the remaining 1,500 – 1,290 = 210 Strep
cases are False Negatives

Strep Throat No Strep Throat Totals

Test Positive 1,290 False Positives Total Positives
Test Negative 210 True Negatives Total Negatives
Totals 1,500 8,500 10,000
Strep Throat Example

• The test has a True Negative Rate of 95% - so 95% of the 8,500 Non-Strep
Throat cases will test negative.
0.95 x 8,500 = 8075. This means the remaining 8,500 – 8,075 = 425 Non-
Strep cases are False Positives
Strep Throat No Strep Throat Totals
Test Positive 1,290 425 Total Positives
Test Negative 210 8,075 Total Negatives
Totals 1,500 8,500 10,000
 Strep Throat Example

• Finally, we can add up the rows:

1,290 + 425 = 1715 total positive tests
210 + 8,075 = 8,285 total negative tests
Strep Throat No Strep Throat Totals
Test Positive 1,290 425 1,715
Test Negative 210 8,075 8,285
Totals 1,500 8,500 10,000
 Strep Throat Example
Strep Throat No Strep Throat Totals
Test Positive 1,290 425 1,715
Test Negative 210 8,075 8,285
Totals 1,500 8,500 10,000

• Remember, we wanted to know: “If I test positive, how likely is it that I

have strep?”
• There were 1,290 true positive strep tests out of 1,715 total positive tests
• So: 1,290/1,715 positive tests were correct, or 75%
• Notice: 75% is NOT THE SAME as either the True Positive Rate or True
Negative Rate
 What did we just do?

S = Strep, H = Not Strep P(S | P) = P(P | S)P(S) /P(P)

P = Positive Test, N = Negative Test P(P | S) = 0.86
P(P) = P(P|S)P(S) +
P(P|H)P(H)

Strep Throat No Strep Throat Totals

Test Positive 1,290 425 1,715
Test Negative 210 8,075 8,285
Totals 1,500 8,500 10,000

P(N | H) = 0.95
P(S) = 0.15
   𝑃 ( 𝐵| 𝐴 ) 𝑃
Conclusions 𝑃 ( 𝐴 |𝐵 ) =
𝑃 ( 𝐵)

1. Bayes’ Rule is a mathematical formula for updating probabilities given

new information.
2. Whenever we perform a test or give a prediction, Bayes’ Rule is
crucial – it is important not to confuse the accuracy of the test with
the belief we want to update!
3. We can use Bayes’ Rule to calculate posterior probabilities by
constructing a data table, using a theoretical number of test subjects.