You are on page 1of 8

Bayesian Econometrics - Computer section

Leandro Magnusson
∗†
Department of Economics
Brown University
Leandro Magnusson@brown.edu
http://www.econ.brown.edu/students/Leandro Magnusson/
April 26, 2006
Preliminary Version
Abstract
This material is used as a reference for the computer exercises of the Bayesian econometrics course
taught by Professor Tony Lancaster. It covers a variety of models (linear regression models, limited
depended variable models (logit, probit and Tobit), etc). The Bayesian computation is done in BUGS
using R as its interface. Both softwares can be download freely from their respective web pages:
http://www.mrc-bsu.cam.ac.uk/bugs/ and
http://www.r-project.org/.

w

q
1
1 First Order Auto Regressive Model
Model: The AR(1) and ARMA(1,1) model are:
y
t
= ρy
t−1

t
, ε
t
∼ N(0, τ) (1.1)
y
t
= ρy
t−1

t
, ε
t
= u
t
−θu
t−1
, {u
t
}
T
t=0
is i.i.d., u
t
∼ N(0, σ) (1.2)
where −1 < ρ < 1 . The Durbin-Watson statistic of the AR(1) errors is:
dw =

T
t=2

t
−ε
t−1
)

T
t=2

2
t−1
)
ε
t
= y
t
−ρy
t−1
(1.3)
Exercise 1. (Based on Example 2.15, page 95, Lancaster (2004))
In R, generate a sequence of AR(1) and a sequence of ARMA(1,1) observations with the same initial
value. Let’s assume the both sample are AR(1), the prior distribution of ρ is improper and τ is known.
(a) What is the likelihood for the AR(1) model (p(y|ρ))?
(b) Generate ρ from its posterior distribution (ρ ∼ N(ˆ ρ,

T
t=1
y
2
t−1
)).
(c) Generate a sequence of y
rep
from its predictive distribution and compute
diff(ρ, y
rep
, y
obs
) = dw(ρ, y
rep
) −dw(ρ, y
obs
)
(d) Repeat items (b), (c) n times separately for AR(1) and ARMA(1,1).
(e) Plot the distributions of df(ρ, y
rep
, y
obs
) and draw your conclusions.
2 Linear Regression Models
2.1 Independent, normal, homoscedastic errors
Database: See Mankiw et al. (1992).
Model: Solow Model
without human capital: Y
t
= K
α
t
(A
t
L
t
)
1−α
with human capital: Y
t
= K
α
t
H
β
t
(A
t
L
t
)
1−α−β
where L
t
= L
0
exp(nt) and A
t
= A
0
exp(gt). Consider that A
0
= a + , where a is constant and is a
country-specific shock. The basic empirical specifications for the Solow model are:
ln
_
Y
L
_
= a +
α
1 −α
ln(s
k
) −
α
1 −α
ln(n +g +δ) + (2.1)
ln
_
Y
L
_
= a +
α
1 −α −β
ln(s
k
) +
β
1 −α −β
ln(s
h
) −
α +β
1 −α −β
ln(n +g +δ) + (2.2)
where s
k
is the capital savings rate, s
h
is the human capital savings rate, n is the population growth rate, g
is technical growth rate, δ depreciation. Mankiw et al. (1992) assume that g +δ = 0.05.
2
Exercise 2.
(a) Consider the following unrestricted specification
ln
_
Y
L
_
= a +b
1
ln(s
k
) +b
2
ln(n +g +δ) +
What is the likelihood for the above model? How would you choose the priors?
(b) Write and run a BUGS program for the linear regression model. Do separately for nonoil, intermediate
and oecd countries. Compare your answers with the maximum likelihood estimates.
(c) Propose and run a test for the coefficients b
1
and b
2
for each group of countries. How could you test the
hypothesis about the returns to scale?
(d) Include human capital and repeat parts (a), (b) and (c).
2.2 Heteroscedasticy
Database: See Yatchew & No (2001)
Model: Consider the log linear form of the demand of gasoline:
log(gas) = β
0

1
log(price) +β
2
log(income) +u (2.3)
In matrix notation, the OLS terms are: v = N − k (the degrees of freedom),
ˆ
β = (X

X)
−1
X

y and
s
2
=
(y−X
ˆ
β)

(y−X
ˆ
β)
v
. Assuming normality and homoscedasticity of the residuals, the likelihood function is:
1
p(y|β, τ) =
1
(2π)
N
2
_
τ
1
2
exp
_

τ
2
(β −
ˆ
β)X

X(β −
ˆ
β)
___
τ
v
2
exp
_

τv
2s
−2
__
If the residuals have the same variance we have:
E
_
u
2
−σ
2
¸
2
= 2σ
4
where σ
2
is the variance (σ
2
=
1
τ
). Therefore a simple statistic to verify if the residuals have equal variance
is:
T(β, σ
2
) =

N
i=1
((y
i
−x
i
β)
2
−σ
2
)
N
− 2σ
4
Student linear model Take a look at Lancaster (2004), chapter 3, pages 159-162. Assume that:
u
i
|X, β, τ, λ
i
∼ N(0, τλ
i
)
where {λ
i
}
N
i=1
is independently gamma distributed with mean 1 and scalar hyperparameter d. Multiplying
the density of u
i
|X, β, τ, λ
i
by the density of {λ
i
}
N
i=1
and integrating out λ’s one may show that u
i
|X, β, τ
follows a t- distribution with mean 0 and degrees of freedom equals to d.
1
For a formal derivation see Koop (2003)
3
Exercise 3.
(a) Assuming that the errors have the same variance, suggest a natural conjugate prior for this model
and derive the posterior distribution of β and τ. Write a program in R for sampling of the posterior
distribution of β and τ.
(b) Simulate β and σ
2
from the joint distribution and evaluate T(β, σ
2
). Repeat nrep times and the draw
the histogram of the test statistic. What are you conclusions?
(c) Write down a BUGS program for the student linear model. Explore how the posterior inferences about
β change by using different values of d. Remember that low values of d approximates the t-distribution
to a Cauchy (have tails) and large values of d approximates to a normal distribution.
3 Limited Dependent Variable Models
3.1 Logit
Database: See Slonim & Roth (1998).
Definition: (Ultimatum Game)
The ultimatum game consists of two players bargaining over the amount of money which it will be
called “pie”. One player, the proposer, proposes a division of the pie, and the second player, the
responder, accepts or rejects it. If the responder accepts , each player earns the amount specified
in the proposal and if the responder rejects, each player earns zero. At perfect equilibrium the
proposer receives all or almost all of the pie.
Model: Rejection behavior
Prob(R = 1) = Λ(a +b
0
∗ Of +b
M
∗ pM +b
H
∗ pH) (3.1)
where Λ is the logistic distribution function, R equals 1 if the offer is reject and 0 otherwise, Of is the
proportion of the pie offered (from 0 to 49,5%), pM = 1 if stakes are medium and pH = 1 is stakes are high.
Exercise 4.
(a) What is the likelihood for the above model? How would you choose the prior?
(b) Write and run a BUGS program for the logit model. Study the convergence of the sampler (try 3000 inter-
actions burning 1000 and 10000 interactions burning 8000). Compare your answers with the maximum
likelihood estimates.
(c) Evaluate the rejection probability at various offer values:
of =
_
0.1 0.2 0.3 0.4 0.5
_
;
do this separately for the low, medium, and high stakes conditions.
(d) Does the level of the stakes affect the rejection probability. Propose a test and compute it. What is your
conclusion?
4
3.2 Tobit
Database: See Mroz (1987)
Model: (Labor supply of Married Women) The reduced form of a woman’s labor supply is given by:
2
h
i
= a
0
+a
1
Y
i
+a

2
Z
i
+u
i
where h
i
is the ith woman’s hour of work during a given year, Y
i
is a measure of other income received by
the household, Z
i
is a set of control variables which includes her age, the number of children less than six
years old, the number of children between ages of five and nineteen her years of schooling, her experience in
years and the square of experience; and u
i
is a stochastic term which is assumed to be normally distributed.
Define the labor force participation as d
i
= 1 if h
1
> 0 and d
i
= 0 otherwise.
The Tobit likelihood is given by:
p(h|a

s, τ) =
N

i=1
_
Φ
_
τ
.5
(a
0
+a
1
Y
i
+a

2
Z
i
)
_
1−d
i
×
τ
.5
φ
_
τ
.5
(h
i
− (a
0
+a
1
Y
i
+a

2
Z
i
))
_
d
i
_
(3.2)
where Φ and φ are the normal distribution and density functions, respectively.
Sampling with data augmentation: Suppose we could observe h

the potential hours of work for
every married woman which may also be negative. Thence p(a

s, τ|h

) = p(a

s, τ|h

, h) is the posterior for
a

s and τ derived from the linear normal likelihood model. We do not observe h

but its conditional density
p(h

|h = 0, a

s, τ) is normally left truncated. Therefore we have the following Gibbs Sampler Algorithm
with the parameter set augmented by h

for censored observations:
1) Choose a initial value for h

, a’s and τ;
2) Sample a

s and τ from p(a

s, τ|h

);
3) Sample h

from the truncated normal p(h

|h = 0, a

s, τ);
4) Repeat 2 and 3 nrep times.
Exercise 5.
(a) Write a BUGS program for the Tobit model.
(b) Write a R program using data augmentation. (In the library msm provides the function tnorm(µ,σ
2
))
truncated normal
2
The woman’s wage rate, the endogenous explanatory variable, is assumed to be a linear function of Y
i
and Z
i
.
5
4 Instrumental Variable Model
4.1 Recursive Equation Model
Database: See Romer (1993)
Model: In a closed economy, the Lucas model for the relationship between output and inflation is:
y = y

+β(π −π
e
) (4.1)
where y is actual output, y

the natural rate, π inflation, and π
e
expected inflation, and where β > 0.
Unanticipated monetary shocks affect both prices and real output. The policy-maker’s objective (welfare)
function is:
W = −
1
2
π
2
+γy (4.2)
where γ > 0. The policy-maker chooses inflation π. Assuming rational expectations, i.e. private agents
known optimization problem of the policy maker, the equilibrium is π = π
e
= γβ and y = y

: inflation is
positive and output is at natural rate (suboptimal equilibrium).
Openness affects the output-inflation trade-off (increased openness raises de amount of inflation associated
with a given expansion of domestic output; that is, it reduces β) and the benefit of higher output relative to
the cost of higher inflation (γ is decreasing in the degree of openness).
Romer (1993) considers the following linear relation between inflation and output:
log π = b
0
+b
1

I
Y
+b
2
∗ log y +b3 ∗ dd +b4 ∗ rd (4.3)
where log π is the log of the average inflation 73-93,
I
Y
is the average share of imports in GDP or GNP, log y
is the log of real income per capita, dd are dummies for alternative measures of openness and inflation and
rd are regional dummies. A priori we expect that b
1
< 0. According to the author,
I
Y
is endogenous since
the adoption of protectionism policies that benefit some interest groups leads to larger budget deficits and
therefore inflation. As instrument he uses land area (in logarithms).
Exercise 6.
(a) How would you translate the instrumental variable model into a recursive system model? Define the
likelihood and the priors for the recursive system.
(b) Write a Bugs program for the recursive model. Could you propose an exogeneity test for openness?
4.2 Multinomial Approach (Bayesian Bootstrap)
Database: See Romer (1993)
Model: Another approach for the previous model is to consider only moment conditions. Under mean
independence we have:
E[Z

(y −Xβ)] = 0
6
The likelihood is derived from the multinomial distribution and its natural conjugate prior is the dirichlet
distribution. For more details, take a look at Lancaster (2004) chapter 3, section 3.4, pages 141-147.
Exercise 7.
(a) Sample β from its posterior and compare to the posterior distribution of the previous exercise. (Remember
that you can sample β from
β = [Z

GX]
−1
Z

Gy
where G is a diagonal matrix whose components are i.i.d exponentially distributed)
(b) How would you propose and exogeneity test in this context?
7
References
Koop, G. (2003), Bayesian Econometrics, first edn, Wiley.
Lancaster, T. (2004), An Introduction to Modern Bayesian Econometrics, first edn, Blackwell Publishing.
Mankiw, N. G., Romer, D. & Weil, D. N. (1992), ‘A contribution to the empirics of economic growth’,
Quartely Journal of Economics 107(2), 407–437.
Mroz, T. A. (1987), ‘The sensitivity of an empirical model of married women’s hours of work to economic
and statistical assumptions’, Econometrica 55(4), 765–799.
Romer, D. (1993), ‘Openness and inflation: Theory and evidence’, Quartely Journal of Economics
108(4), 869–903.
Slonim, R. & Roth, A. E. (1998), ‘Learning in high stakes ultimum games: an experiment in the slovak
republic’, Econometrica 66(3), 569–596.
Yatchew, A. & No, J. (2001), ‘Household gasoline demand in canada’, Econometrica 69(6), 1697–1709.
8