MATH11177: Bayesian Theory

Solutions to worksheet 1

1. We wish to calculate the probability of having chosen a route, r, given

that the journey, j, was completed in 1.5 hours P r(r|j = 1.5). By Bayes’
Theorem, we have,
P (j|r)P (r)
P (r|j) =
P (j)
Now, we have that P (r) = 0.25 for each r = 1, · · · , 4, and,
4
X
P (j) = P (j|r)P (r) = 0.6.
r=1

Then, substituting the values into Bayes’ Theorem, we obtain the proba-
bilities,

Route 1 2 3 4
P (r|j) 0.083 0.208 0.333 0.375

2. (a) By Bayes’ Theorem, we have,

P r(T + |I)P r(I)

P r(I|T +) = .
P r(T +)

By the Law of Total Probability, we have,

P r(T +) = P r(T + |I) ∗ P r(I) + (1 − P r(T − |S)) ∗ (1 − P r(I)).

Substituting the values for sensitivity (P r(T + |I) = 0.75), specificity

(P r(T − |S) = 0.99) and prior probability of infection for a person
in the medium risk category (P r(I) = 2.22 ∗ 10−2 ) we obtain,

0.75 ∗ 2.22 ∗ 10−2

P r(I|T +) = = 0.63.
(0.75 ∗ 2.22 ∗ 10−2 + (0.01 ∗ (1 − 2.22 ∗ 10−2 )

(b) Substituting the corresponding values for P r(I) in the expression

above we obtain:

1
 Moderate Medium High
0.89 0.97 0.99

(c) This exercise can be used to highlight both the benefit of the prior
to incorporate previously known information and the importance of
choosing an appropriate prior. It also highlights the importance of
stratification and repeat testing. Not all results are alike and we will
not have the same “confidence” in them. You might also want to
touch upon how difficult it is to know the real prevalence and hence
the need for hyper-parameters.
√
3. Poisson data with a prior, π(µ) ∝ 1/ µ.

(a) The posterior distribution for µ given y = y1 , · · · , yn :

n
Y
p(µ|y) ∝ f (yi |µ)π(µ)
i=1
Pn
−nµ i=1 yi −0.5
∝e µ µ
−nµ nȳ−0.5
=e µ

The right-hand term is the kernel of a Gamma(nȳ + 0.5, n) pdf.

(b) The posterior distribution for this sample is Gamma(52+0.5,10). The
posterior mean is the expected value for a Gamma random variable,
namely α/β, or in this case, 52.5/10 p = 5.25. The standard
√ deviation
is the square root of the variance, 52.5/102 = 0.525 = 0.725.
Note that as n increases, this posterior mean will converge to the
sample mean ȳ and the standard deviation will go to 0.

4. Prior predictive distribution for Y , which can take on the values of 0, 1,

or 2:
X
P r(Y = 0) = P r(Y = 0|θ)π(θ)
θ
= 0.2 ∗ 0.5 + 0.4 ∗ 0.25 + 0.6 ∗ 0.25 = 0.35
X
P r(Y = 1) = P r(Y = 1|θ)π(θ)
θ
= 0.22 ∗ 0.5 + 0.42 ∗ 0.25 + 0.62 ∗ 0.25 = 0.15
X
P r(Y = 2) = P r(Y = 2|θ)π(θ)
θ
= (1 − 0.2 − 0.22 ) ∗ 0.5 + (1 − 0.4 − 0.42 ) ∗ 0.25+
(1 − 0.6 − 0.62 ) ∗ 0.25 = 0.5

2
5. Extra-variation Poisson prior:
(a) The prior for θ is given by,
Z ∞ Z ∞ θ
λ exp(−λ) β α α−1
π(θ) = π(θ|λ)π(λ)dλ = λ exp(−βλ)dλ
0 0 θ! Γ(α)
Z ∞
βα
= λθ+α−1 exp(−(β + 1)λ)dλ
Γ(α)Γ(θ + 1) 0
Z ∞
βα Γ(θ + α) (β + 1)θ+α θ+α−1
= θ+α
λ exp(−(β + 1)λ)dλ
Γ(α)Γ(θ + 1) (β + 1) 0 Γ(θ + α)
Note that the integrand in the above integral is a Gamma(α + θ, β + 1) pdf.
 α  θ
Γ(θ + α) β 1
Γ(α)Γ(θ + 1) β + 1 β+1
Thus, comparing this expression with the pdf of a Negative Binomial,
we have that,
θ ∼ Negative Binomial(α, β)
(b) The sampling distribution for x is:
θ!
f (x|θ) = px (1 − p)θ−x .
x!(θ − x)!
Note that as a likelihood function for θ, θ must be greater than or
equal to x, i.e., θ ∈ {x, x + 1, x + 2, · · · }. Then the posterior for θ:

 θ
θ! Γ(θ + α) 1
p(θ|x) ∝ (1 − p)θ
(θ − x)! Γ(θ + 1) β + 1
 θ
Γ(θ + α)Γ(θ + 1) 1−p
=
Γ(θ − x + 1)Γ(θ + 1) β + 1
 θ
Γ(θ + α) 1−p
=
Γ(θ − x + 1) β + 1
for θ = x, x + 1, x + 2, · · · .
Because θ is discrete, the marginal distribution for x can be found
by summing over possible values of θ:
∞  θ
X Γ(θ + α) 1−p
m(x) = .
Γ(θ − x + 1) β + 1
θ=x

And the posterior pmf for θ is:

 θ
Γ(θ+α) 1−p
Γ(θ−x+1) β+1
p(θ|x) =
m(x)
for θ = x, x + 1, x + 2, · · · .