Dynamic Optimization

A very short Introduction

Jens Rubart

Darmstadt, December 17, 2003

Contents
1 Introduction 1
2 Some basics 1
3 The maximum principle: 3
4 Example 4
5 And know.... 6

Institute of Economics, Darmstadt University of Technology, Germany and Center for Empirical
Macroeconomics, University of Bielefeld, e-mail: rubart@vwl.tu-darmstadt.de.
1 Introduction
Decisions under restrictions are the general problems to be solved in economics.
For example, the well known (microeconomic) problem of an agent who seeks to
maximize his utility function under a given budget constraint. The method to solve
such a problem is well known from undergraduate studies: The Lagrange Multiplier
Method.
1
In principle we have the same problem in dynamic models. The main dierence
is, that a decision at time t, for example the decision to buy more consumption
goods, inuences the states at time t + 1, +2, ...., because in future time periods
less budget is left for subsequent consumption. The general structure of a dynamic
optimization problem is as follows:
1. An agent has to choose one or even more control variables in order to maximize
an objective function (e.g. an utility function) over a given nite or innite
time interval.
2. The chosen value of control variables determines the evolution of state variables
which also inuence the optimization problem in future time periods.
The application of dynamic optimization is often found in modern (dynamic)
macroeconomic theory, and in particular, in the theory of economic growth. Al-
though a numerous literature concerning dynamic optimization exists, for example
Feichtinger and Hartl (1986), Seierstad and Sydster (1987), Kamien and Schwartz
(1991), Leonard and van Long (1992) or Turnovsky (2000), the attempt of this
(very) short introduction is to help students to understand and to solve (continuous
time) dynamic economic models from a more intuitive point of view.
2
2 Some basics
The general form of a dynamic optimization problem in continuous time is as fol-
lows:
3
1
See, for example, Sydster and Hammond (1995), Ch. 18 or Chiang (1984) Ch. 12.
2
The solution of discrete time models is straightforward. The interested reader is referred to
Chow (1997), or to Uhlig (1999).
3
See Leonard and van Long (1992), Chapters 4 and 7 or Feichtinger and Hartl (1986), Chs.
1+2, for an introduction to the maximum principle in more detail. A toolkit - style introduction
1
For all t, nd the control variables c(t) that maximize
V =
_
T
0
v(s(t), c(t), t)dt (1)
subject to
s(t) = g(s(t), c(t), t) (2)
and
s(0) = s
0
, s(T) = s
T
, (3)
with
- V (0) : objective function at t = 0.
- c(t) : vector of control variables
- s(t) : vector of state variables (Note that the evolution of a state variable
is described by a dierential equation: s =
ds
dt
)
- T : nal date, also possible that T
Following the (heuristic) approach by Barro and Sala-I-Martin (1995) and setting
up the respective Lagrange function of the problem above:
L =
_
T
0
v(s(t), c(t), t)dt +
_
T
0
_
(t)
_
g(s(t), c(t), t) s(t)
_
_
dt +k(T) e
r(T)T
(4)
Comparable to the static problem, one has to maximize L with respect to c(t) and
s(t) for all t (0, T). The problem is, that we do not know the derivative of s(t)
with respect to s. Integrating the term (t)

k(t) by parts, we get:
4
L =
_
T
0
_
v(s(t), c(t), t) + (t)g(s(t), c(t), t)
_
. .
(H)
dt
+
_
T
0
(t)s(t)dt + (0)k(0) (T)k(T) + k(T) e
r(T)T
(5)
The expression inside the rst integral, accentuated by (H), is known as the Hamil-
tonian function. We dene:
H v(s(t), c(t), t) + (t)g(s(t), c(t), t) (6)
to dynamic optimization can also be found in Sydster et al. (2000).
4
A detailed solution of the Lagrange function can be found in Barro and Sala-I-Martin (1995),
p. 501.
2
Remark:
The Hamiltonian function given by equation (6) is referred to as a
current-value Hamiltonian.
In the literature you also nd the formulation of a present-value Hamil-
tonian:

H = e
rt
v(s(t), c(t), t) +

(t)g(s(t), c(t), t), (7)
with

H = H e
rt
,

(t) = (t) e
rt
. However, as pointed out by
Feichtinger and Hartl (1986), the necessary optimality conditions are
equivalent for both formulations !
Furthermore, it can be shown that it is sucient to concentrate only on the
Hamiltonian function to solve a dynamic optimization problem,
5
because calculating
the maximum of the Hamiltonian produces the same optimal solution as one solves
the integral of the Lagrange-function given by (4).
3 The maximum principle:
An optimal solution to the problem given by eqn. (6) is a triplet of (s(t), c(t), (t))
and must satisfy the following conditions:
(i) c(t) maximizes a current value Hamiltonian H(s(t), c(t),
i
(t), t), that
is
H
c
i
(t)
= 0 (8)
(ii) the state and co-state variables satisfy a pair of dierential equations:
s
i
(t) =
H

i
(t)
(9)

i
(t) = r(t)
H
s
i
(t)
(10)
(iii) the transversality condition has to hold:
6
lim
t
_
(t)s(t)

0 (11)
5
See, for example, Feichtinger and Hartl (1986), Ch. 2, for an elaboration.
6
Furthermore, the sucient condition for the maximum of the Hamiltonian function is, that it
is concave in the state variable, i.e.

2
H
s(t)
2
0.
3
If a current value Hamiltonian,

H, has to be solved, the necessary opti-
mality conditions are:

H
c(t)
= 0 (12)

=


H
s(t)
(13)
lim
t
(

(t)s(t)) = 0 (14)
Because of the equivalence of the optimality conditions, one almost nds the
current value representations in economic applications (there, you do not have to
care about the term e
rt
in your calculations).
Before turning to some examples, we give a brief economic interpretation of the
optimization problem. As in static optimization the co-state variables (or dynamic
Lagrange multipliers) denote the gain (or loss) in terms of utility (or prots) if there
is a change in the state variables. In this line, the idea of the maximum-principle is to
construct a system of shadow prices (co-state variables) which measure the inuences
of the decisions on state variables in that way which maximizes the overall value
H.
7
4 Example
Let us consider the Ramsey-Cass-Koopmans Model
8
. An innitely lived representa-
tive consumer chooses the time path of consumption in order to maximize lifetime
utility: W =
_

0
e
t
u(c(t))dt. We normalize the labor force to L = 1, furthermore,
> 0 denotes the subjective discount rate and gives u() an instantaneous utility
function. The constraint faced to the household is given by:
9

k = f(k) k c. (15)
7
See, for instance, Feichtinger and Hartl (1986).
8
See, for instance, Aghion and Howitt (1998), Ch. 1.2., or Barro and Sala-I-Martin (1995), Ch.
2.
9
This constraint says nothing else than: consumption plus investment equal net national prod-
uct.
4
Furthermore, we assume the following functional forms:
10
u(c(t)) =
_
c
1
1
1
, for > 1
ln(c) , for = 1
f(k) = k

Dening the respective Hamiltonian of the above problem, one obtains:

max
c
t

H = u(c
t
) e
t
+
t
_
f(k) k c
_
(16)
Note that (16) represents a present value Hamiltonian, by dening = e
t
one
obtains a current value specication:
max
c
t
H = u(c
t
) +
t
_
f(k) k c
_
(17)
The necessary rst order conditions follow as:
H
c
u

(c
t
) = (18)
= (f

(k) ) (19)
transversality condition:
lim
t
e
t
k = 0
Using the functional forms and rewriting equations (18) and (19), we obtain:
c

= (20)
=
_
k
1
) (21)
Together with the capital accumulation (equation (15)), the solution of the opti-
mization problem consists of 2 non-linear dierential equations and, furthermore, a
relation between the control and the co-state variable, equation (18). To solve the
problem we take the logarithm of equation (20) and dierentiate with respect to
time:
ln(c) = ln() | dt

c
c
=

, (22)
10
The functional form of the utility function is known as a Constant Relative Risk Aversion
(CRRA) utility function. See, for instance, Blanchard and Fischer (1989) Ch. 2.1, or Barro and
Sala-I-Martin (1995), Ch. 2.1.
5
dividing (21) by it follows:

= k
1
. (23)
With (22), we rewrite (23) to:

c
c
=
f

(k)

=
k
1

. (24)
If you combine equations (15) and (24), you obtain a system of dierential equations
dened in k and c, which can be examined for transitional dynamics or to calculate
the steady state stock of capital (not reported here, but have a look in Barro and
Sala-I-Martin (1995) Ch.2).
Now, let r = f

(k) denote the return of capital, you can rewrite (24) to:
11
c
c
=
r

. (25)
Equations (25) and (24) are often referred toas the Euler - equation. It is the basic
condition which denes optimal consumption over time. It says that households
choose consumption so as to equate the rate of return to capital, r, to the rate of
time preference , plus the decrease of marginal utility of consumption, u

(c), due
to growing per capita consumption.
12
5 And know....
you will nd a lot of applications of dynamic optimization in textbooks and research
articles.... see for example: Barro and Sala-I-Martin (1995) or Aghion and Howitt
(1998). Furthermore, in Asada et al. (1998) you will nd an extended solution of
the endogenous growth model of Romer (1990).
References
Aghion, P. and P. Howitt (1998). Endogenous Growth Theory (1st. ed.). Cambridge,
Mass: The MIT Press.
11
Note that f

(k) > has to hold along the optimal path. This condition ensures a positive
growth rate!
12
Following Barro and Sala-I-Martin (1995), p. 63, we can rewrite eqn. (25) as
r =
_
u

(c) c
u

(c)
_

c
c
.
6
Asada, T., W. Semmler, and A. Novak (1998). Endogenous growth and the balanced
growth equilibrium. Research in Economics 52, 189212.
Barro, R. J. and X. Sala-I-Martin (1995). Economic Growth (1st. ed.). New York:
McGraw-Hill.
Blanchard, O. J. and S. Fischer (1989). Lectures on Macroeconomics (1st. ed.).
Cambridge, Mass.: The MIT Press.
Chiang, A. C. (1984). Fundamental Methods of Mathematical Economics (3rd. ed.).
Singapore: McGraw-Hill.
Chow, G. C. (1997). Dynamic Economics. Oxford, New York: Oxford University
Press.
Feichtinger, G. and R. Hartl (1986). Optimale Kontrolle okonomischer Prozesse
(1st. ed.). Berlin, New York: Walter de Gruyter.
Kamien, M. I. and N. L. Schwartz (1991). Dynamic Optimization (2nd. ed.). Ams-
terdam, New York: Elsevier.
Leonard, D. and N. van Long (1992). Optimal Control Theory and Static Opti-
mization in Economics (1st. ed.). Cambridge, New York: Cambridge University
Press.
Romer, P. (1990). Endogenous Technological Change. Journal of Political Econ-
omy 98, 71102.
Seierstad, A. and K. Sydster (1987). Optimal Control Theory with Economic
Applications (1st. ed.). Amsterdam, New York: North-Holland.
Sydster, K. and P. J. Hammond (1995). Mathematics for Economic Analysis (1st.
ed.). Englewood Clis, New Jersey: Prentice-Hall.
Sydster, K., A. Strom, and P. Berck (2000). Economists Mathematical Manual
(3rd. ed.). Berlin, Heidelberg: Springer Verlag.
Turnovsky, S. J. (2000). Methods of Macroeconomic Dynamics (2nd. ed.). Cam-
bridge, Mass.: The MIT Press.
Uhlig, H. (1999). A toolkit for studying nonlinear dynamic stochastic models easily.
In R. Marimon and A. Scott (Eds.), Computational Methods for the Study of
Dynamic Economies, Oxford, pp. 3061. Oxford University Press.
7