Exercises

Applied Bayesian Analysis and Numerical Methods

(STK4021)

Giovanni Romeo

22 September 2016
Exercise 8
A random sample of n students is drawn from a large
population, and their weights are measured. The
average weight of the n sampled students is ȳ = 150
pounds. Assume the weights in the population are
normally distributed with unknown mean θ and
known standard deviation 20 pounds. Suppose your
prior distribution for θ is normal with mean 180 and
standard deviation 40.
I Prior p(θ) = N(µ0 , τ 2 ) = N(180, 402 )
0
I Sampling distribution

p(y |θ) = N(θ, σ 2 ) = N(θ, 202 )

Exercise 8

(a) Give your posterior distribution for θ.

I p(θ|y ) ∝ p(y |θ)p(θ) = N(θ|µn , τn2 ) where:

I
1
µ + σn2 ȳ
τ02 0
1
180 + 20n2 150
402
µn = 1 = 1
τ02
+ σn2 402 + n
202
I
1 1 1 1 n
= + = + 2
τn2 τ02 σ2 40 2 20
Exercise 8

(b) A new student is sampled at random from the

same population and has a weight of ỹ pounds. Give
a posterior predictive distribution for ỹ .

p(ỹ |θ)p(θ|y )dθ ∝ N(ỹ |µn , σ 2 + τn2 )

R
I p(ỹ |y ) =
1
180+ 20n2 150
I p(ỹ |y ) ∝ N(ỹ | 402 1
+ n
, 202 + ( 4012 + n −1
202 ) )
402 202
Exercise 8

(c) For n = 10, give a 95% posterior interval for θ

and a 95% posterior predictive interval for ỹ .
1
180+ 20n2 150
I p(θ|y ) = N(θ| 402
1
+ n
, ( 4012 + n −1
202 ) )
402 202
I p(θ|y ) = N(θ|150.73, 6.252 )
1
180+ 20n2 150
I p(ỹ |y ) = N(ỹ | 402 1
+ n
, 202 + ( 4012 + n −1
202 ) )
402 202
I p(ỹ |y ) = N(ỹ |150.73, 20.952 )
Exercise 8

Posterior distribution 95% CI n=10 Predictive distribution 95% CI n=10

0.06

0.015
0.05
0.04

0.010
density

density
0.03
0.02

0.005
138.48 162.98 109.67 191.79
0.01

0.000
0.00

140 150 160 170 100 120 140 160 180 200

Figure: exercise 8 (c)

Exercise 8

(d) Do the same for n = 100.

I p(θ|y ) = N(θ|150.07, 1.9972 )

I p(ỹ |y ) = N(ỹ |150.07, 20.102 )
Exercise 8

Posterior distribution 95% CI n=100 Predictive distribution 95% CI n=100

0.020
0.20

0.015
0.15
density

density

0.010
0.10

0.005
0.05

146.16 153.98 110.67 189.47

0.000
0.00

144 146 148 150 152 154 156 100 120 140 160 180 200

Figure: exercise 8 (d)

Exercise 8

Prior and Posterior

0.15

Posterior n=100
0.10
density

0.05

Posterior n=10

Prior
0.00

100 120 140 160 180 200 220

Figure: comparison
Exercise 9

Suppose your prior distribution for θ, the proportion

of Californians who support the death penalty, is beta
with mean 0.6 and standard deviation 0.3.

I (a) Determine the parameters α and β of your

prior distribution. Sketch the prior density
function.
(a) Determine the parameters α and β of your prior
distribution. Sketch the prior density function.

α

E (θ) =

 = 0.6 (1)
α+β
αβ
Var (θ) =
 = (0.3)2(2)
(α + β) (α + β + 1)
2


I From (1) β = 23 α, then plug in in (2)

I p(θ) = Beta(θ|α, β) with α = 1 and β = 32 .

Exercise 9
3.0
2.5
2.0
1.5
1.0

0.0 0.2 0.4 0.6 0.8 1.0

Figure: Prior
(b) A random sample of 1000 Californians is taken,
and 65% support the death penalty. What are your
posterior mean and variance for θ? Draw the
posterior density function.

I Posterior distribution
p(θ) = Beta(θ|y + α, n − y + β)

I n = 1000 and y = 650

=⇒ p(θ) = Beta(651, 350.67)
Exercise 9
25

Posterior
20
15

Prior
10
5
0

0.0 0.2 0.4 0.6 0.8 1.0

Figure: Posterior
(c) Examine the sensitivity of the posterior
distribution to dierent prior means and widths
including a non-informative prior.
Exercise 9

Prior Beta(1,1)
25

Posterior
20
15
10
5

Prior
0

0.0 0.2 0.4 0.6 0.8 1.0

Figure: Exercise 9 (c)

Exercise 9

Prior Beta(2,8)
25

Posterior
20
15
10

Prior
5
0

0.0 0.2 0.4 0.6 0.8 1.0

Figure: Exercise 9 (c)

Exercise 9

Prior Beta(20,80)
25

Posterior
20
15
10

Prior
5
0

0.0 0.2 0.4 0.6 0.8 1.0

Figure: Exercise 9 (c)

Exercise 9

Prior Beta(50,200)

Posterior
25
20

Prior
15
10
5
0

0.0 0.2 0.4 0.6 0.8 1.0

Figure: Exercise 9 (c)

Exercise 18

18. Poisson model: derive the gamma posterior

distribution for the Poisson model parameterized in
terms of rate and exposure with conjugate prior
distribution.
Exercise 18
I Prior distribution α
p(θ) = Gamma(θ|α, β) = Γ(α) β
θα−1 e −βθ
I p(yi |θ) ∼ Poisson(xi θ) where θ = rate and xi =
exposure.

I
n
Y (θxi )yi e −θxi
p(y |θ) = =
yi !
i=1
I
n n
Y x yi yi −θxi
Y
= i
θ e ∝ θyi e −θxi
yi !
i=1 i=1
I
Pn Pn
p(y |θ) ∝ θ( i=1 yi ) e −(θ i=1 xi )
Exercise 18
I Posterior distribution p(θ|y ) ∝ p(y |θ)p(θ)
I

(
Pn
i=1 yi ) −(θ
Pn
i=1 xi )
β α α−1 −βθ
p(θ|y ) ∝ θ e θ e
Γ(α)
I
Pn Pn
( i=1 yi ) −(θ i=1 xi ) α−1 −βθ
p(θ|y ) ∝ θ e θ e
I
Pn Pn
p(θ|y ) ∝ θ(α+ i=1 yi −1) e −θ(β+ i=1 xi )

p(θ|y ) ∼ Gamma(αn , βn )
I
n
X n
X
Where αn = α+ yi and βn = β + xi
Exercise 12
Course Notes and Exercises by Nils Lid Hjort
Exercise 12
I Expected Loss
Z
E [L(a|y )] = L(θ, a)p(θ|y )dθ

R 0.15
I E [L(alarm|y )] = 1000 p(θ|y )dθ
0
R1
I E [L(no alarm|y )] = 5000 0.15 p(θ|y )dθ

I (a) p(θ) ∼Beta(1,1) =⇒ p(θ|y ) ∼

Beta(y+1,50-y+1)
I (b) p(θ) ∼Beta(2,8) =⇒ p(θ|y ) ∼
Beta(y+2,50-y+8)
Exercise 12
I (c) p(θ) ∼ 12 Beta(2,8)+ 12 Beta(8,2)
I

p(θ|y ) ∝ p(y |θ)p(θ) =

I

1
= θy (1 − θ)50−y · [θ1 (1 − θ)7 + θ7 (1 − θ)1 ] =
2
I

1 1
= [θy +2−1 (1−θ)50−y +8−1 ]+ [θy +8−1 (1−θ)50−y +2−1 ]
2 2
I

1 1
p(θ|y ) ∼ Beta(y +2, 50−y +8)+ Beta(y +8, 50−y +
2 2
Exercise 12

R 0.15
I E [L(alarm|y )] = 1000 0p(θ|y )dθ
R1
I E [L(no alarm|y )] = 5000 0.15 p(θ|y )dθ

I For which value of y:

E [L(alarm|y )] < E [L(no alarm|y )]
Exercise 12

Uniform prior
5000

● ● ● ●
●
●
No alarm
●
4000

●
3000
Loss Function

●
2000

●
1000

● ● ● ● ●
●
●
●
● ●
●
●
●
Alarm
● ● ●
● ● ● ● ● ● ●
0

0 5 10 15

Figure: Exercise 12 (a)

Exercise 12

Beta(2,8) prior
5000

● ● ●
●
●
● No alarm
●
4000

●
3000

●
Loss Function

●
2000

●
1000

● ● ● ● ●
●
●
●
● ●
●
●
● Alarm
● ● ●
● ● ● ● ● ● ●
0

0 5 10 15

Figure: Exercise 12 (b)

Exercise 12

Mixture of Beta prior

5000

● ● ● ●
●
●
● No alarm
●
4000

●
3000
Loss Function

●
2000

●
1000

● ● ● ● ● ● ● ● ●
Alarm
● ● ● ● ● ● ● ●
0

0 5 10 15

Figure: Exercise 12 (c)

Exercise 12
3.5

Uniform
Beta(2,8)
3.0

Mixture
2.5
2.0
1.5
1.0
0.5
0.0

0.0 0.2 0.4 0.6 0.8 1.0

Figure: Exercise 12 Priors