Lecture 3

Dynamic Equilibrium Models III : Infinite Periods

1. Introduction

In this lecture, we extend our analysis to infinite periods. The method of dynamic
programming can be easily applied to solve infinite horizon optimization problems. In fact,
in certain cases solving infinite horizon DGE models is easier than solving finite horizon
DGE models. For expositional simplicity, we will focus on stationary problems. The
assumption of stationarity implies that the period utility function and the laws of motion
are time-invariant functions i.e.

Ft (Xt , Ut ) = F (Xt , Ut ) & xt+1 = mx (Xt , Ut ) ∀ t. (1.1)

Time-invariant period utility function and laws of motion together with infinite horizon
generate time-invariant (or stationary) policy and value functions. Thus,

Ut = G(Xt ) & Wt (Xt ) = W (Xt ) ∀ t. (1.2)

We will also assume that period return function is bounded 0 ≤ F (Xt , Ut ) < ∞ and
limt→∞ β t F (Xt , Ut ) = 0. These assumptions ensure that we get interior solutions. All the
examples considered in this course will satisfy these assumptions.
Suppose that the optimization problem is as follows:
∞
X
W = max β t F (Xt , Ut ) (1.3)
Ut
t=0

subject to the laws of motion

xt+1 = mx (Xt , Ut ) ∀ xt ∈ Xt & ∀ t (1.4)

given X0 . We can recast this optimization problem as

W (Xt ) = max F (Xt , Ut ) + βW (Xt+1 ) (1.5)

Ut

subject to the laws of motion

xt+1 = mx (Xt , Ut ) ∀ xt ∈ Xt (1.6)

With stationary problems, it is customary to drop the time subscript. The optimiza-
tion problem is normally written as

W (X) = max F (X, U ) + βW (X 0 ) (1.5a)

U

subject to the laws of motion

1
 x0 = mx (X, U ) ∀ x ∈ X (1.6a)

where terms like X 0 denote next period values of state variables. Potentially we can solve
such Bellman equations in four ways:

(1.) guess and verify the policy function;

(2.) guess and verify the value function;
(3.) repeated substitution ; and
(4.) through iteration of value function.

Guess and verify methods are applicable to very limited type of cases. Essentially,
these methods work for only two classes of specifications of preferences and constraints,
namely, variants of specification with linear constraints and quadratic preferences or Cobb-
Douglas constraints and logarithmic preferences. More often than not one has to use either
method 3 or 4. Also only in the limited cases, dynamic programming problems can be
solved analytically. Generally, one uses approximation and/or numerical methods to solve
dynamic programming problems.

Example 1

Let us illustrate the four methods of solution using a concrete example. Consider in-
finite horizon version of optimal consumption problem with production. The optimization
problem is
∞
X
max β t ln ct (1.7)
ct ,kt+1
t=0

subject to
kt+1 = ktα − ct (1.8)

We can recast the above optimization problem as

W (kt ) = max ln ct + βW (kt+1 ) (1.9)

ct ,kt+1

subject to (1.8).

Method 1: Guess and Verify Policy Function

Let λt be the Langrangian multiplier associated with (1.8), then first order conditions
are

1
ct : = λt (1.10)
ct

2
 dW (kt+1 )
kt+1 : β = λt (1.11)
dkt+1
From envelope condition we have

dW (kt ) αktα−1
= (1.12)
dkt ct
Combining (1.10-1.12) we have
α−1
1 αβkt+1
= . (1.13)
ct ct+1
(1.13) is the Euler equation linking consumptions in adjacent periods. In order to find
the path of optimal consumption, we need to solve (1.13). We will use guess and verify
method to solve this equation. Suppose that the policy function has the following form:

ct = µktα (1.14)
Basically we are assuming that each period the decision maker consumes a constant fraction
of income. The trick is to find an expression for µ which satisfies (1.13). Putting (1.14) in
(1.13), we get
α−1
1 αβkt+1
= (1.15)
µktα α
µkt+1
From (1.15) we have
1 αβ
α = (1.16)
kt kt+1
This implies

kt+1 = αβktα (1.17)

Putting (1.14) and (1.17) in the budget constraint we have

ktα = µktα + αβktα (1.18)

(1.18) implies that µ = 1 − αβ. The solution for optimal consumption is then

ct = (1 − αβ)ktα (1.19)
Steps

(1) Set up the Bellman equation;

(2) Derive first order conditions and the Euler equations;
(3) Guess the functional form of the policy functions;

3
(4) Putting the guessed policy function in the Euler equation solve for the coefficients.

Method 2. Guess and Verify Value Function

Let us suppose that the value function has the following form:

W (kt ) = E + F ln kt (1.20)
We basically want to derive expressions for E and F . The guess depends on the form
of utility function and the number of state variables. With this guess the optimization
problem is

W (kt ) = max ln ct + βE + βF ln kt+1 + λt [ktα − ct − kt+1 ] (1.21)

ct ,kt+1

The first order conditions are:

1
ct : = λt (1.22)
ct

βF
kt+1 : = λt . (1.23)
kt+1
(1.22), (1.23), and the budget constraint imply
βF 1
= . (1.24)
ktα− ct ct
This implies
1
ct = ktα . (1.25)
1 + βF
Also
βF
kt+1 = kα . (1.26)
1 + βF t
Putting (1.25) and (1.26) in (1.21) we get
µ ¶ µ ¶
1 α βF α
W (kt ) = ln k + βE + βF ln k . (1.27)
1 + βF t 1 + βF t
(1.27) implies that

W (kt ) = some constant terms + α(1 + βF ) ln kt . (1.28)

Comparing (1.20) and (1.28) we have

E = some constant terms (1.29)

and

4
 α
F = (1.30)
1 − αβ
(1.25) and (1.30) imply that

ct = (1 − αβ)ktα (1.31)
which is identical to (1.19).

Steps

(1) Guess the form of the value function;

(2) Set up Bellman equation;
(3) Derive first order conditions and solve for the policy functions;
(4) Put the derived policy functions in the value function;
(5) Compare the new value function with the guessed one and solve for the coefficients.

Method 3. Use of Envelope Condition and Repeated Substitution

We go back to Euler equation (1.13)

α−1
1 αβkt+1
= . (1.13)
ct ct+1
Previously we solved this equation by guessing the form of policy function. Now we are
going to solve this equation another way. We know that (1 + r) = αk α−1 . Using this fact,
we can rewrite (1.13) as

ct+1 = β(1 + rt+1 )ct . (1.32)

Making use of (1.32) we have

ct+j = β j Πji=1 (1 + rt+i )ct . (1.33)

The budget constraint is

ktα = ct + kt+1 (1.34)

which can be rewritten as

αktα−1
kt = ct + kt+1 . (1.35)
α
From (1.35) we have another expression for kt
α
kt = [ct + kt+1 ]. (1.36)
1 + rt

5
Now we solve forward (1.36).

α α2 α2
kt = ct + ct+1 + kt+2 (1.37)
1 + rt (1 + rt )(1 + rt+1 ) (1 + rt )(1 + rt+1 )
T +1
α
Solving forward (1.37) repeatedly and assuming that limT →∞ QT −1−t kT = 0, we
(1+rt+j )
j=0
have

α α2 α3
kt = ct + ct+1 + ct+2 ..... (1.38)
1 + rt (1 + rt )(1 + rt+1 ) (1 + rt )(1 + rt+1 )(1 + rt+2 )

Combining (1.33) with (1.38) we have

α α2 β α3 β 2
kt = ct + ct + ct + ...... (1.39)
1 + rt 1 + rt 1 + rt
(1.39) can be rewritten as
1 + rt £ ¤ 1
kt = 1 + αβ + α2 β 2 + ...... ct = ct . (1.40)
α 1 − αβ
From (1.40) we again have

ct = (1 − αβ)ktα . (1.41)

Steps

(1) Steps 1-3 as in the method 1;

(2) Using the Euler equations solve for the future values of choice variables as functions
of the current values of choice variables;
(3) Solve forward the laws of motion;
(4) Combine expressions found in steps 3 and 4.

Method 4. Value Function Iteration

The optimization problem is

W (kt ) = max ln ct + βW (kt+1 ) (1.42)

ct ,kt+1

subject to
kt+1 = ktα − ct . (1.43)
Start at some time T and assume that kT +1 & WT +1 (kT +1 ) = 0. Actually you can assume
any finite value for kT +1 & WT +1 (kT +1 ). Then at time T the optimization problem reduces
to

6
 WT (kT ) = max ln cT + λT [kTα − cT ]. (1.44)
cT

The solution is
cT = kTα . (1.45)
The value function

WT (kT ) = α ln kT . (1.46)
Now go back to period T − 1. The optimization problem is

£ ¤
WT −1 (kT −1 ) = max ln cT −1 + αβ ln kT + λT −1 kTα−1 − cT −1 − kT (1.47)
cT −1 ,kT

The first order conditions are

1
cT −1 : = λT −1 (1.48)
cT −1

αβ
kT : = λT −1 . (1.49)
kT
From (1.48), (1.49), and the period budget constraint we have

αβ 1
= α . (1.50)
kT kT −1 − kT
From (1.50) we have

αβ
kT = kα . (1.51)
1 + αβ T −1
From the budget constraint we get
1
cT −1 = kα . (1.52)
1 + αβ T −1
Using (1.51) and (1.52) the value function for the period T − 1 can be written as

WT −1 (kT −1 ) = constant terms + α(1 + αβ) ln kT −1 . (1.53)

Now we go back to period T − 2. The optimization problem now is

WT −2 (kT −2 ) =
£ ¤
max ln cT −2 + β cons. + αβ(1 + αβ) ln kT −1 + λT −2 kTα−2 − cT −2 − kT −1 (1.54)
cT −2 ,kT −1

You can show that the solutions are

7
 αβ + α2 β 2
kT −1 = kα (1.55)
1 + αβ + α2 β 2 T −2

1
cT −2 = kα . (1.56)
1 + αβ + α2 β 2 T −2
Using (1.55) and (1.56) we can get expression for WT −2 (kT −2 ). Then we step back one
more period and solve for WT −3 (kT −3 ). We keep on repeating these steps for a large
number of periods. One can show that
1
cT −j = kTα−j . (1.57)
1 + αβ + α2 β 2 + ..... + αj β j
Then

lim cT −j ≡ ct = (1 − αβ)ktα . (1.58)

j→∞

The value function converges too.

Steps

(1) Start at some period T and assume that kT +1 & WT +1 (kT +1 ) = 0;

(2) Set up the Bellman equation for period T ;
(3) Derive the policy functions;
(4) Using the policy functions, derive the value function for T ;
(4) Step 1 period back and repeat steps 2-4;
(5) Keep on repeating step 4 till convergence is achieved in policy and value functions.

2. Uncertainty

The method of dynamic programming is readily applicable to infinite horizon opti-

mization problems with uncertainty. Potentially one can use all the four methods discussed
above to solve stochastic dynamic problems. We begin with a concrete example.

Example 2
Real Business Cycle (RBC) Model

We are going to analyze business cycle using the RBC model. This model has been very
influential in studying business cycles. More generally, it has changed the methodology of
macroeconomics and led to emergence of Dynamic Stochastic General Equilibrium (DSGE)
models. Most of the analysis in modern macroeconomics is done using this framework. As
discussed earlier, the DSGE models have the following properties: (i) They specify budget
constraints for households, technologies for firms, and resource constraints for the overall
economy; (ii) They specify household preferences and firm objectives; (iii) They assume

8
forward-looking behavior for firms and households; (iv) They include the shocks that firms
and households face; (v) They are models of the entire economy.
Before developing the RBC model, we summarize the most important business cycle
facts. Firstly, empirical evidence suggests that consumption, output, investment, and
employment are highly pro-cyclical. Real wage is acyclical or mildly pro-cyclical. Secondly,
the effect of temporary shocks on output is highly persistent. The effect of temporary
shocks last up to eight quarters. Thirdly, the response of output to a shock is hump-shaped.
The effect of shock on output peaks in fourth or fifth quarter. Fourthly, consumption is
less volatile relative to output, while investment is highly volatile relative to output. Any
good model of business cycle should be able to to match and explain these facts.
Broadly, there are two approaches to study business cycles: (i) RBC models and (ii)
Keynesian Models. There are two key differences between these two types of approaches.
RBC models assume Walrasian markets and they attribute business cycles primarily to
technology ( productivity/real/ supply side) shocks. Keynesian models on the other hand
assume imperfect markets with nominal rigidities and they attribute business cycles pri-
marily to aggregate demand or nominal shocks. We studied Keynesian models in ECON
501. In this lecture, we will develop RBC model and evaluate its performance in explaining
business cycle regularities.
The RBC model is a stochastic version of the optimal consumption problem analyzed
above. In the basic RBC model, it is assumed that there is uncertainty with regard to
technology or production function. This model brings out the effects of technology shock
on consumption, output, employment etc.
Consider a stochastic version of the optimal consumption problem analyzed above.
Suppose that the production function is given by At ktα where At = exp²t . ²t is an in-
dependently and identically distributed (i.i.d) random variable with mean 0 and
variance σ 2 . The random variable At is called technology or productivity shock. It is
assumed that the technology shock is realized at the beginning of period t before con-
sumption and investment decisions are made. Let Et denote the expectation operator
conditional on time t information set.
The optimization problem is
∞
X
max E0 β t ln ct (2.1)
ct ,kt+1
t=0

subject to

kt+1 = At ktα − ct . (2.2)

This optimization problem is known as the real business cycle model, which studies
the effects of technology shocks on investment, consumption, income etc. We can recast
the above optimization problem as

W (kt ) = max ln ct + βEt W (kt+1 ) + λt [At ktα − ct − kt+1 ] . (2.3)

ct ,kt+1

The first order conditions are

9
 1
ct : = λt (2.4)
ct

dW (kt+1 )
kt+1 : βEt = λt . (2.5)
dkt+1
From envelope condition we have

dW (kt ) αAt ktα−1

= . (2.6)
dkt ct
Combining (2.4)-(2.6) we get
α−1
1 At+1 kt+1
= αβEt (2.7)
ct ct+1
which is again the Euler equation. In order to solve (2.7), we will guess the policy function
(you can use other methods as well). Let

ct = µAt ktα . (2.8)

Putting (2.8) in (2.7) we have
1 1
α = αβEt . (2.9)
At kt kt+1
Combining (2.9) with the budget constraint we have
1 αβ
α = . (2.10)
At kt At ktα − ct
From (2.10) we get

ct = (1 − αβ)At ktα . (2.11)

From the budget constraint we get

kt+1 = αβAt ktα . (2.12)

Taking log of (2.12) we get fundamental stochastic difference equation which tells us how
capital stock evolves over time.

ln kt+1 = ln αβ + α ln kt + ²t (2.13)
Using (2.13) we can trace out how capital accumulation evolves over time in response to
a single shock, ²t (impulse response function). One can also derive moments of the
process of capital accumulation. Solving (2.13) backwards we have

ln kt+1 = ln αβ + α[ln αβ + αkt−1 + ²t−1 ] + ²t . (2.14)

10
If we keep on repeating this process, we will get

ln kt+1 = [1 + α + α2 + ...] ln αβ + [²t + α²t−1 + ......] (2.15)

Then

ln αβ
E(ln kt+1 ) = (2.16)
1−α

σ2
V (ln kt+1 ) = . (2.17)
1 − α2
Similarly we can derive moments of other variables like consumption, income etc., covari-
ances, and the associated impulse response functions.

Exercise: Guess the form of the value function as W (kt , At ) = E + F ln kt + G ln At

and solve the above example.

Denote output by yt . Then , we have

ln yt = ln At + α ln kt . (2.18)
By combining (2.13) and (2.18) we have,

ln kt = ln αβ + ln yt−1 . (2.19)
Then (2.18) and (2.19) imply that log of output follows a first order autoregressive process:

ln yt+1 = α ln αβ + α ln yt + ξt+1 . (2.20)

Let us now consider implications of technology shock on consumption, investment, and
output. Using (2.20) one can derive the implications of technology shock on output. For
this one needs to assume some value of α. α is estimated to be 0.33.
Now consider the effect of a one time positive technology shock. Let y0 be the output
at time t − 1. Suppose that at time T technology shock is realized and let ξT = 1. Suppose
that technology shock is entirely temporary, i.e. ξt = 0, ∀t > T . What would be the
response of output. In period T log of output will be one unit higher than log of y0 . In
T + 1 it will be higher by 1/3 compared to log of y0 . In T + 2 it will be higher by 1/9 and
so on.
There are two problems with this kind of response. First, response of output is not
hump shaped as in the data. Output rises in period T and then falls linearly. Secondly,
the effect of technology shock is not persistent. Its effect dies down quite quickly. There
are other problems as well. In the model, consumption and investment are a constant
proportion of output. Thus, both are as volatile as output. However, in the real data
consumption is much less volatile than output and investment is much more volatile. This
analysis suggests that this version of RBC model does not do a good job in accounting for
business cycle facts.

11
 Now let us modify the environment as follows. Suppose now that technology shock
follows an autoregressive process:

ξt+1 = ρξt + ut+1 (2.21)

where ut is i.i.d. with mean zero and variance σ 2 . Then (2.20) can be written as

ln yt+1 = α ln αβ + α ln yt + ρξt + ut+1 . (2.22)

(2.20) also implies that

ξt = ln yt − α ln αβ − α ln yt−1 . (2.23)
Then (2.23) and (2.24) imply that log of output follows a second order autoregressive
process:

ln yt+1 = Constnat + (α + ρ) ln yt − αρ ln yt−1 + ut+1 . (2.24)

(2.24) shows that the response of output to technology shock depends on two parameters,
α and ρ. If ρ is high, meaning that technology shock is highly persistent, the model can
generate a hump-shaped and persistent response of output to technology shock. However,
it does not fix other problems.
The models considered so far do not say anything about employment and real wage.
To consider the implications with regard to employment and real wage, let us introduce
labor-leisure choice in example 2. Let nt be labor supplied at time t. Now the problem is
"∞ #
X
t
max E0 β [ln ct + ln(1 − nt ) (2.25)
ct ,nt ,kt+1
t=0

subject to

At ktα n1−α
t = ct + kt+1 . (2.26)
The Bellman equation is

£ ¤
W (kt ) = max ln ct + ln(1 − nt ) + βEt W (kt+1 ) + λt At ktα n1−α
t − ct − kt+1 . (2.27)
ct ,kt+1

The first order conditions for consumption continues to be given by (2.4). The first
order conditions for labor supply and capital stock are
1
nt : = λt (1 − α)At ktα n−α
t ; (2.28)
1 − nt
and

α−1 1−α
kt+1 : λt = βαEt λt+1 At+1 kt+1 nt+1 . (2.29)

12
 (2.4) and (2.28) imply that
α−1 1−α
1 Et At+1 kt+1 nt+1
= αβ . (2.30)
ct ct+1
which is the Euler equation linking consumption in adjacent periods. Using guess and
verify method, one can show that

ct = (1 − αβ)At ktα n1−α

t ; (2.31)

kt+1 = αβAt ktα n1−α

t (2.32)
and
1−α
nt = . (2.33)
2 − α(1 + β)
Since nt is constant, the model implies that technology shock does not affect employment.
Also since real wage is equal to the marginal product of labor (= (1 − α)yt /nt ), real wage
is strongly pro-cyclical. Both these predictions are at odds with empirical evidence. As
discussed earlier, employment is highly pro-cyclical and real wage is essentially acyclical.
What we have seen that basic RBC models cannot account for most of the business
cycle facts. Researchers have modified basic RBC models in many ways to improve the
performance of these models. Some of the most prominent extensions are introduction of
government expenditure and fiscal shocks, taste shocks (shocks to utility function), more
complicated utility function (habit formation), shocks to inventory etc. Many researchers
have taken more fundamental departures abandoning the assumptions of Walrasian or
frictionless markets. Now, it is customary to assume friction in either labor market or goods
market or both. Frictions in the labor market generate unemployment and have allowed
researchers to also analyze unemployment dynamics. Some models also incorporate credit
markets, but in very rudimentary forms. These extensions have improved the performance
of these models in some dimensions, but not to a great extent. Of course, none of these
models were able to forecast the current great recession.

13
 Additional Exercises

(1) Brock-Mirman Optimal Growth Model: Suppose that the representative agent faces
the following optimization problem
∞
X
max ln ct
ct ,kt+1
t=0

subject to
£ ¤1−α
ktα λt nt = ct + kt+1
where the labor force nt = 1 and λ > 1 is the rate of labor augmenting technical
progress. Solve for the optimal path of consumption, income, and real wages.

(2) Endogenous Labor Supply: Suppose now that the representative agent faces the fol-
lowing optimization problem
∞
X
max ln ct + ln(1 − nt )
ct ,kt+1
t=0

subject to

yt = ct + kt+1 .
(a.) Let yt = ktα n1−α
t . Solve for the optimal path of consumption, income, employment,
and real wages.
1−α
(b.) Let yt = ktα [λt nt ] . Solve for the optimal path of consumption, income, employ-
ment, and real wages.

(3) Government Expenditure and the Business Cycle: Suppose that the representative
agent faces the following optimization problem
∞
X
max E0 ln ct + ln(1 − nt )
ct ,kt+1
t=0

subject to

yt = ct + kt+1 + gt .
Let yt = ktα n1−α
t . Also suppose that the government expenditure gt = θyt . Assume
that the representative agent ignores the effect of his investment decision on the gov-
ernment expenditure (an externality). Solve for the optimal path of consumption,
income, employment, and real wages. How do changes in θ affect the labor supply?

14
(4) Let the optimization problem be

X∞
c1−α
t
max
ct ,at+1
t=0
1 −α

subject to

(1 + r)at = ct + at+1 .
Solve for the optimal path of ct and at+1 .

(5) Let us introduce uncertainty in the previous problem. Let the optimization problem
be

X∞
c1−α
t
max E0
ct ,at+1
t=0
1 −α

subject to

at+1 = (1 + rt+1 )(at − ct )

where rt is an i.i.d random variable. Solve for the optimal path of ct and at+1 .

15