Topic 1: Introduction To Intertemporal Optimization: Yulei Luo

Topic 1: Introduction to Intertemporal Optimization
Yulei Luo
SEF of HKU
September 6, 2013
Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 1 / 56

Warm-Up: Intertemporal/Dynamic Optimization
In static optimization, the task is to …nd a single value for each

control variable, such that the objective function will be maximized or
minimized. In contrast, in a dynamic setting, time enters explicitly
and we encounter a dynamic optimization problem. In such a
problem, we need to …nd the optimal time path of control and state
during an entire planning period.
Speci…cally, at any time t, we have to choose the value of some
control variable, c (t ), which will then a¤ect the value of the state
variable, x (t + 1), via a state transition equation given the current
state x (t ).
Two leading methods to solve DO problems are: optimal control (OC)
and dynamic programming (DP). They can be applied in deterministic
or stochastic and discrete-time or continuous-time settings.

The Setting of Optimal Control (OC)
In the dynamic discrete-time model, the objective should take the

form of summation from t = 0 to t = T :
T
max ∑ r (c (t ) , x (t ) , t ) ,
fc (t ),x (t +1 )g t =0
(1)
subject to
x (t + 1) x (t ) = f (c (t ) , x (t ) , t ) , (2)
x0 = x (0) , x (T + 1) 0 , (3)
for any t 0.

(conti.) As in the static optimization case, the Lagrangian for this
intertemporal (dynamic) problem is
T
L= ∑ fr (ct , xt , t ) + λt [f (ct , xt , t ) + xt xt +1 ]g + λT +1 xT +1 ,
t =0
(4)
where λt is the shadow price (i.e., the Lagrange multiplier).
The FOCs for ct are:
∂L
= rc0 (ct , xt , t ) + λt fc0 (ct , xt , t ) = 0 for t = 0, 1, ,T. (5)
∂ct
The FOCs for xt +1 are complicated because each xt appears in two
terms in the Lagrangian function.

(conti.) Consider some relevant terms in the above Lagrangian
function:
T T 1
∑ λt (xt xt +1 ) = (λ0 x0 λT xT +1 ) + ∑ xt +1 (λt +1 λt ) .
t =0 t =0
The FOCs for xt +1 :
∂L
= rx0 (ct +1 , xt +1 , t + 1) + λt +1 fx0 (ct +1 , xt +1 , t + 1)
∂xt +1
+λt +1 λt = 0,
for t = 0, ,T 1.

Theorem (The Maximum Principle)
The necessary FOCs for maximizing (1) subject to (2) are:
(i) ∂L
∂ct = 0 for all t = 0,
t
,T.
(ii) xt +1 xt = f (ct , xt , t ) for all t (The state transition equation)
(iii) rx0 (ct +1 , xt +1 , t + 1) + λt +1 fx0 (ct +1 , xt +1 , t + 1) + λt +1 λt = 0 for
all t (The costate (λt ) equation)
(iv) xT +1 0, λT +1 0, xT +1 λT +1 = 0 (The transversality condition, or
the complementary-slackness condition)

Example: Optimal Consumption-Saving Model
Suppose that a consumer has the utility function u (ct ) , where ct is

consumption at time t. The consumer’s utility is concave:
u 0 > 0, u 00 < 0, (6)
and u (ct ) satis…es the Inada conditions (They ensure that

consumption will always be interior):
lim u 0 (c ) = ∞ and lim u 0 (c ) = 0. (7)

c !0 c !∞
The consumer is also endowed with a0 = a (0) , and has income

stream derived from holding the asset: yt = rat , where r is the
interest rate. The consumer uses the income to purchase c. Any
income not consumed is added to the asset holdings as savings:
at + 1 = ( 1 + r ) a t ct . (8)

The Model Setup
The consumer’s lifetime utility maximization problem is to

T
max
fc ,a + g
t t 1
∑ β t u ( ct ) (9)
t =0
subject to
at +1 = Rat ct , t = 0, ,T, (10)

a0 = a ( 0 ) , aT + 1 0, (11)
where β is the consumer’s rate of time preference (β 1) and

R = 1 + r is the gross interest rate.

(conti.) Since limc !0 u 0 (c ) = ∞, c (t ) > 0 for all t T . Further, to
ensure that a solution exists, assets are constrained to be nonnegative
after the terminal period T (i.e., when the consumer dies), aT +1 0.
Note that this nonnegativity constraint must bind, i.e., aT +1 = 0. In
fact, this constraint serves two important purposes:
First, without this constraint aT +1 0, the consumer would want to
set aT +1 = ∞ and die with outstanding debts. This is clearly not
feasible.
Second, the fact that the constraint is binding in the optimal solution
guarantees that resources are not being thrown away after T .

Using Optimal Control to Solve the Model
Setting up the Lagrangian:

T
L= ∑ βt fu (ct ) + λt [Rat ct at +1 ]g + λT +1 aT +1 (12)
t =0
∂L
The FOC for ct is ∂ct = 0:
u 0 ( ct ) λt = 0 for t = 0, 1, ,T. (13)

∂L
The FOC for at +1 is ∂at +1 = 0:
λt + βRλt +1 = 0 for t = 0, ,T 1. (14)

(conti.) The terminal condition aT +1 0 implies that the
transversality condition (the complementary slackness condition) is
λ T +1 0, aT +1 0, aT +1 λT +1 = 0, (15)
which means that either the asset holdings (a) must be exhausted on
the terminal date, or the shadow price of capital (λt ) must be 0 on
the terminal date. Since u 0 > 0, the marginal value of capital (λ)
cannot be 0 and thus the capital stock should optimally be exhausted
by the terminal date T + 1, i.e., aT +1 = 0.

(conti.) Combining (13) with (14) gives
u 0 (ct ) = βRu 0 (ct +1 ), (16)
which is called the Euler equation characterizing optimal consumption

dynamics.
Suppose that the utility function has an isoelastic form
1 γ
ct 1
u ( ct ) = , (17)
1 γ
where γ 0 (Note that when γ = 1, the function reduces to ln ct ).

We have
ct + 1
ct
γ
= βRct +γ1 =) = ( βR )1/γ . (18)
ct
Therefore, if βR > 1, the optimal consumption will rise over time; if
βR < 1, the optimal consumption will decline over time. When
βR = 1, ct = ct +1 .

(conti.) (18) implies that the elasticity of intertemporal substitution
(EIS) is
d ctc+t 1 / ctc+t 1 d ln ctc+t 1 1
= = . (19)
dR/R d ln R γ
Hence, increasing γ i.e., reducing γ1 make the agent more unwilling

to postpone consumption (i.e., more unwilling to save). That’s why
we call this type of utility functions the isoelastic utility function.
The optimal problem is pinned down by a given initial condition (a0 )
and by a terminal condition (aT +1 = 0) . The sets of (T 1) Euler
equations (18) and constraints (10) then determine the time path of
c as well as a.

Obtaining the Intertemporal Budget Constraint
(conti.) (10) implies that
a1 = Ra0 c0 ,
a2 = Ra1 c1 ,
aT +1 = RaT cT .
aT + 1 c c1
T
+ TT + + + c0 = Ra0 =)
R R R
T
ct
∑ Rt = Ra0 , (20)
t =0
where we use the fact that aT +1 = 0.

(conti.) For simplicity, assume that γ = 1. Combining (18) with (20)
gives
1 1
c0 + ( βRc0 ) + + β T R T c0 = Ra0 =)
R RT
R
c0 = a0 . (21)
1+β+ + βT
Hence, the time path of optimal consumption is
R t +1 β t
ct = a0 (22)
1+β+ + βT
where t = 0, , T . Given ct , it is straightforward to determine the
time path of at :
R t +1 β t
at +1 = Rat ct = Rat a0 , (23)
1+β+ + βT
where t = 0, , T and aT +1 = 0.
(conti.) Consider an important special case, βR = 1. In this case,
(18) implies that c0 = c1 = = cT ,
1
c0 1 + + = Ra0 =)
RT
R 1
ct = a
(T +1 ) 0
. (24)
1 R
where t = 0, ,T.
Next, the optimal path of at can be derived by solving the following
di¤erence equation:
r
at +1 = Rat ct = Rat a
(T +1 ) 0
=)
1 R
1 1
at = 1 (T +1 )
a0 R t + a ,
(T +1 ) 0
(25)
1 R 1 R

Alternative Way to Solve Optimal Consumption
Instead of de…ning the Lagrange multiplier for each ‡ow budget

constraint, we may de…ne a Lagrange multiplier for the intertemporal
budget constraint (that is, the lifetime budget constraint):
T
ct
∑ Rt = Ra0 ,
t =0
where we have used the fact that aT +1 = 0. That is, we may write
the Lagrangian as follows
!
T T
ct
L = ∑ β u (ct ) + λ Ra0 ∑ t ,
t
t =0 t =0 R
where λ is the constant Lagrange multiplier for the lifetime budget

constraint.

(conti.) The FOCs for an optimum are then
1
β t u 0 ( ct ) = λ , where t = 0, ,T. (26)
Rt
Since λ is a constant, the above FOCs implies that the Euler
equations are
u 0 (ct ) = βRu 0 (ct +1 ), where t = 0, ,T 1,
which is identical to those Euler equations we derived above.

The In…nite Horizon Case
There are some reasons to consider the in…nite horizon (T = ∞) case:

People don’t live forever, but they may care about their o¤spring (a
bequest motive).
Assuming an in…nite horizon eliminates the terminal date complications
(stop saving, consume their entire value, etc. There is no real world
counterpart for these actions because the real economy continue
forever).
Many economic models with a long time horizon tend to show very
similar results to IH models if the horizon is long enough.
IH models are stationary in nature: the remaining time horizon doesn’t
change as we move forward in time.

The Model Setup
The consumer’s lifetime utility maximization problem

∞
max ∑ βt u (ct ) (27)
c (t ) t =0
subject to at +1 = Rat ct , 8 t 0, given the initial asset holdings

a0 = a ( 0 ) .
In addition, we need to impose the following institutional assumption:
no Ponzi game condition (nPg):
at + 1
lim 0, (28)
t !∞ Rt
means that in present value terms, the agent cannot engage in
borrowing and lending so that his terminal asset holdings are negative.

(conti.) (28) can help obtain an intertemporal budget constraint:
T
ct aT + 1
∑ Rt +
RT
= Ra0 .
t =0
Applying (28), we have

!
T T
ct aT + 1 ct aT + 1
lim
T !∞
∑ Rt + RT = lim
T !∞
∑ Rt + lim
T !∞ RT
t =0 t =0
∞
ct aT + 1
= ∑ Rt + lim
T !∞ RT
=)
t =0
∞
ct
∑ Rt Ra0 . (29)
t =0

(conti.) In an optimum, (29) must be binding:
∞
ct
∑ Rt = Ra0 (30a)
t =0
because (28) is binding, i.e., limT !∞ aRt +t 1 = 0 (Otherwise, the

agent can consume the left amount and reach a higher utility level,
which contradicts optimum).
Set up the Lagrangian:
!
∞ ∞
c
L = ∑ βt u (ct ) + λ Ra0 ∑ t
t
t =0 t =0 R
The FOCs w.r.t. ct for any t are

1
β t u 0 ( ct ) λ = 0, 8t 0, (31)
Rt
where λ is the Lagrangian multiplier associated with the
intertemporal budget constraint.
(conti.) Note that the FOCs imply the following Euler equation
linking consumption in two consecutive periods
u 0 (ct ) = βRu 0 (ct +1 ) , 8t 0. (32)
Suppose that the utility function is log, u (ct ) = log ct . (31) implies
that ct = ( βR )t c0 , 8t 1. Substituting them into the (30a) gives
!
∞ ∞
( βR )t
∑ R t c0 = Ra0 =) ∑ βt c0 = Ra0 =)
t =0 t =0
c0 = R ( 1 β ) a0 , (33)
and consumption in periods t 1 can be recovered:
ct = ( βR )t c0 = ( βR )t R (1 β ) a0 , 8 t 1. (34)
Similarly, we can determine the optimal time path of asset holdings:
at +1 = Rat ( βR )t R (1 β ) a0 .

Dynamic Programming (DP)
We use the same in…nite-horizon lifetime optimal
consumption-savings model to introduce DP and then provide more
general discussions on DP.
Consider the maximization problem
∞
Ut = ∑ βs t
u ( cs )
s =t
subject to as +1 = Ras cs , 8s t, given the initial asset holdings

at = a (t ). In addition, impose:
at + T + 1
lim 0.
T !∞ R t +T
Of course, DP is also applicable with a …nite horizon.
Assume that there is a function, called the value function, that gives
us the maximal constrained value of Ut as a function of the initial
wealth at . We write the value function as J (at ), and we assume it is
di¤erentiable.
(conti.) DP rests on a recursive equation involving the value function.
That equation, called the Bellman equation, characterizes
intertemporally optimizing plans:
A consumption plan optimal from the standpoint of t must maximize
Ut +1 subject to the future wealth at +1 produced by today’s
consumption plan ct . If not, the consumer could raise utility by
behaving di¤erently after t.
The foregoing property means that an optimizing consumer can behave
as if Ut = u (ct ) + βJ (at +1 ), where J (at +1 ) is the constrained
maximal value of Ut +1 .
If this is so, ct maximizing lifetime utility is the one maximizing
Ut = u (ct ) + βJ (at +1 ) subject to the same constraint.
Formally, the Bellman equation can be written as
J (at ) = max fu (ct ) + βJ (at +1 )g , (35)

ct
s.t. at +1 = Rat ct . (36)

(conti.) (35) can be rewritten as:
J (at ) = max fu (ct ) + βJ (Rat ct )g . (37)

ct
The …rst-order condition is thus
u 0 ( ct ) βJ 0 (Rat ct ) = 0. (38)
Note that (38) gives optimal consumption as an implicit function of

current wealth, c = c (a). Substituting it into (37):
J (a) = u (c (a)) + βJ (Ra c (a)) . (39)
Taking di¤erentiation w.r.t. a gives
J 0 (a) = u 0 (c (a)) c 0 (a) + βJ 0 (Ra c (a)) R c 0 (a) , or

J 0 (a) = βRJ 0 (Ra c (a)) , or J 0 (a) = Ru 0 (c ) , (40)
which is called the Envelop theorem.

(conti.) Implication: For an optimizing consumer, an increment to
wealth on any date has the same e¤ect on lifetime utility regardless of
how to put the wealth: consumption or saving.
Combining (38) with (38), we obtain the usual consumption Euler
equation:
u 0 (ct ) = βRu 0 (ct +1 ) . (41)
Following the same procedure used above, we can easily solve for
optimal consumption path.
Alternatively, we can solve for optimal consumption by using the
value function. Given the same utility function:
1 γ
c
u ( ct ) = t , (42)
1 γ
we …rst guess the form of the value function and then verify that the
guess was right. A natural guess is that the value function takes a
parallel form as u (ct )
a1 γ
J (a ) = A . (43)
1 γ
(conti.) Substituting this conjectured solution into
u 0 (ct ) βJ 0 (Rat ct ) = 0,
γ γ R
ct = βA (Rat ct ) , or ct = at . (44)
1 + ( βA)1/γ
Substituting it into the Bellman equation,

J (a) = u (c (a)) + βJ (Ra c (a)), gives
a1 γ c (a )1 γ (Ra c (a))1 γ
A = + βA , (45)
1 γ 1 γ 1 γ
1
A = γ, (46)
1/γ 1 1/γ
β β R 1
ct = ( βR )1/γ β 1/γ
R1 1/γ
1 at , (47)
which is consistent with that we obtained using optimal control.

Finally, given A, we can obtain J (a).
Dynamic Programming (DP): General Setting
Choose an in…nite sequence of controls fct gt∞=0 to maximize
∞
∑ βt r (xt , ct ) (48)
t =0
subject to xt +1 = g (xt , ct ), with x0 given.

Assume that r (xt , ct ) is a concave function and the set
f(xt +1 , xt ) : xt +1 g (xt , ct )g is convex and compact. DP seeks a
time-invariant policy function c = h (x ) mapping the state (x ) into
the control (c ), such that the sequence fct gt∞=0 generated by iterating
the h and g functions starting from x0 solves the original problem.
To …nd h, we need to know the value function that is the optimal
value of the original problem:
∞
V (x0 ) = max
∞ ∑ βt r (xt , ct )
f c t g t =0 t = 0
(49)
subject to the same constraint.

Note that if we knew V ( ), c can be obtained by solving for each x
the problem
max fr (x, c ) + βV (e
x )g (50)
c
subject to e
x = g (x, c ) with x given and e x denotes the state next
period.
Key point: Have exchanged the original problem of …nding fct gt∞=0
for the problem of …nding the optimal V (x ) and h (x ) that solves the
continuum of maximum problems (one maximum problem for each x).
The task is now to solve for (V (x ) , h (x )) linked by the Bellman
equation:
V (x ) = max fr (x, c ) + βV (g (x, c ))g . (51)
c
The maximizer of the right side of it is h (x ) that satis…es
V (x ) = r (x, h (x )) + βV (g (x, h (x ))) , (52)
which is a functional equation to be solved for unknown functions:
(V (x ) , h (x )). Note that the function equation must hold for every
value of x within the permitted domain and the solutions to it are
functions, V (x ) and h (x ).
Some Properties of the Bellman Equation
In the general setting, we encounter the same problems that we had

in the consumption-savings model:
We need to know the derivatives of the value function, V (x ).
The value function is unknown.
The envelop theorem of Benveniste and Scheinkman (1978) gives a
set of su¢ cient conditions under which one can obtain the derivative
of the value function:
State x 2 X , where X is a convex set with a nonempty interior.
Control c 2 C , where C is a convex set with a nonempty interior.
The period objective function, r , is concave and di¤erentiable.
The RHS of the state transition equation, g , is concave, di¤erentiable,
and invertible in c.

Under the above particular assumptions about r , g , x, and c, there are
four main …ndings:
1 The Bellman equation, (51), has a unique strictly concave solution.
2 This solution can be approached in the limit as j ! ∞ by iterations on
Vj +1 (x ) = max fr (x, c ) + βVj (e

x )g (53)
c
subject to e
x = g (x, c ) with x given, starting from any bounded and
continuous initial V0 .
3 There is a unique and time-invariant optimal policy of the form
c = h (x ), where h is chosen to maximize the right side of (51).
4 The limiting value function V is di¤erentiable.

Optimal Conditions
Since V is di¤erentiable, the FOC for (51) is
rc (x, c ) + βV 0 (g (x, c )) gc (x, c ) = 0. (54)
If we also assume that h is di¤erentiable, di¤erentiation of (52) gives
V 0 (x ) = rx (x, h (x )) + rc (x, h (x )) h0 (x ) (55)

0 0
+ βV (g [x, h (x )]) gx (x, h (x )) + gc (x, h (x )) h (x ) ,
which can be reduced to
V 0 (x ) = rx (x, h (x )) + βV 0 (g (x, h (x ))) gx (x, h (x )) . (56)

Solution Methods
There are three main types of solution methods for solving DP.
1 Guess and verify. It involves guessing and verifying a solution V to
the Bellman equation.
2 Value function iteration. It proceeds by constructing a sequence of
value functions and associated policy functions. The sequence is
created by iterating on the following equation, starting from V0 = 0,
and continuing until Vj has converged:
Vj +1 (x ) = max fr (x, c ) + βVj (e

x )g (57)
c
subject to e
x = g (x, c ) with x given.
3 Policy function iteration (or Howard’s improvement algorithm).

Value Function Iteration (optional)
First, consider an initial guess for the value function V0 (x ). (It does
not matter very much and we usually guess it is 0.)
Second, one can then calculate an updated value function V1 (x )
using:
V1 (x ) = max fr (x, c ) + βV0 (g (x, c ))g , (58)
c
and doing the maximization numerically over the domain of x. This

maximization de…nes approximately the function V1 (x ).
Using this new function V1 (x ), one can update again and get a new
V2 ( x ) :
V2 (x ) = max fr (x, c ) + βV1 (g (x, c ))g , (59)
c
over the domain of x. Repeated this step until the sequence of the
approximate value functions converges to V (x ). Note that in the
repeated calculations of the value function, one is also calculating
repeated approximations to the policy function, c = h (x ).
Finite-horizon Dynamic Programming
The basic idea of …nite-horizon DP is to solve the optimization
problem period by period – starting from the terminal period, taking
the previous periods’solutions as given, and then working back
sequentially to the …rst period.
Having optimized the terminal period (T ), this solution is substituted
into the period T 1 problem and then period T 1 is optimized.
Substituting these previous solutions, the solutions for T 2, T 3,
, 0 are obtained in sequence in the same way.
Suppose that the problem is to maximize:
U (xt ) = r (xt , ct ) + βV (xt +1 ) , (60)
for t = 0, , T , subject to
xt +1 = f (xt , ct ) , (61)
with x0 given and xT +1 = x. Note that here we assume that the
value function, V (xt +1 ), is time invariant and later we can see that it
is correct.
Consider the problem for period T . First, we maximize
U (xT ) = r (xT , cT ) + βV (xT +1 ) , (62)
with respect to cT , subject to xT +1 = f (xT , cT ) and taking xT as

given. Thus we maximize
U (xT ) = r (xT , cT ) + βV (f (xT , cT )) .
The FOC is
∂r (xT , cT ) ∂V (f (xT , cT ))
+β = 0,
∂cT ∂cT
which means that the solution for cT has the form:
cT = gT (xT ) . (63)

(conti.) Substituting this solution into U (xT ):
U (xT ) = r (xT , cT ) + βV (f (xT , cT ))

= r (f (xT 1 , cT 1 ) , gT (f (xT 1 , cT 1 )))
+ βV (f (f (xT 1 , cT 1 ) , gT (f (xT 1 , cT 1 )))) (64)
, VT (xT 1 , cT 1 ) .
Turning to period T 1, we maximize
U (xT 1) = r (xT 1 , cT 1) + βV (xT ) ,
w.r.t. cT 1 , subject to xT = f (xT 1 , cT 1) and taking xT 1 as

given. The FOC is
∂r (xT 1 , cT 1) ∂VT (xT 1 , cT 1)

+β = 0, (65)
∂cT 1 ∂cT 1
from which we can write the solution for cT 1 as
cT 1 = gT 1 (xT 1 ) .
Substituting this into U (xT 1) gives
U (xT 1) = r (xT 1 , gT 1 (xT 1 )) + βV (f (xT 1 , gT 1 (xT 1(66)

)))
, VT 1 (xT 2 , cT 2 ) .
We can proceed similarly in periods T 2, , 0.

The general solution for period t has the FOC
∂r (xt , ct ) ∂V (f (xt , ct ))
+β = 0, (67)
∂ct ∂ct
and optimal control:
ct = gt (xt ) . (68)

Application to the Same Consumption-Saving Model
Consider the same consumption-saving problem as above:

T
V ( a0 ) = max
fc ,a + g
t t 1
∑ β t u ( ct )
t =0
s.t.at +1 = Rat ct , f (at , ct ) , t = 0, ,T,
given a0 = a (0) and aT +1 = 0, where u (ct ) = ln (ct ).

We …rst consider the solution for period T . This requires to maximize
U (aT ) = u (cT ) + βV (aT +1 ) , (69)
with respect to cT , subject to aT +1 = f (aT , cT ), aT +1 = 0, and

taking aT as given. We thus have V (aT ) = u (cT ), i.e., no
maximization is required for the terminal period: cT = RaT , and
U (aT ) = u (RaT ) + βV (0) = ln (RaT ) + βV (0) . (70)

(conti.) The T 1 problem is to maximize:
U ( aT 1) = u ( cT 1) + βV (aT )
= u ( cT 1) + β [ln (RaT ) + βV (0)] (71)
with respect to cT 1 , subject to aT = RaT 1 cT 1 and taking

aT 1 as given. The FOC is:
1 1
β = 0,
cT 1 RaT 1 cT 1
which implies that

R
cT 1 = aT 1 (72)
1+β
and
R βR 2
V ( aT 1) = ln aT 1 + β ln aT 1 + βV (0) .
1+β 1+β
(73)
(conti.) Similarly, solving the T 2 problem yields the following
optimal solution:
R
cT 2 = aT 2. (74)
1 + β + β2
It is straightforward to show that the general solution for periods
0, 1, , T takes the following form:
R
ct = 2
a,
t t
(75)
1+β+β + + βT
which is identical to the corresponding solution we obtained using
optimal control (i.e., the Lagrange multiplier method). As T ! ∞,
we have
ct = (1 β) Rat . (76)

From Discrete-time to Continuous-time
We may treat the continuous-time case as a limiting case of the

discrete-time case as the time interval (∆t ) goes to 0. (Note that in
the discrete-time models, ∆t = 1.)
Speci…cally, in discrete-time, the ‡ow constraint is
x (t + 1) x (t ) = f (c (t ) , x (t ) , t ) , (77)
which can be rewritten as
x (t + ∆t ) x (t ) = f (c (t ) , x (t ) , t ) ∆t (78)
if the time interval is ∆t.

Dividing both sides by ∆t and letting it goes to 0 gives
x (t + ∆t ) x (t )
xt0 = lim = f (c (t ) , x (t ) , t ) (79)
∆t !0 ∆t

(conti.) Further, note that when the time interval is ∆t, the objective
function can be written as
T /∆t
∑ r (c (i ∆t ) , x (i ∆t ) , i ∆t ) ∆t; (80)
i =0
as ∆t ! 0,
T /∆t Z T
lim ∑
∆t !0 i =0
r (c (i ∆t ) , x (i ∆t ) , i ∆t ) ∆t =
0
r (c (t ) , x (t ) , t ) dt
(81)
Hence, the continuous-time version of the dynamic optimization
problem can be written as
Z T
max r (c (t ) , x (t ) , t ) dt,
c (t ) 0
dx (t )
s.t. xt0 = = f (c (t ) , x (t ) , t ) ,
dt
x0 = x (0) , and x (T ) 0.
Solving the Model
The Lagrange function for the above intertemporal problem is

Z T
L = r (ct , xt , t ) + λt f (ct , xt , t ) xt0 dt
0
Z T
= r (ct , xt , t ) + λt f (ct , xt , t ) + λt0 xt dt + λ0 x0 λT(82)
xT
0
where we use the fact that the rule of integration by parts implies
Z T Z T
λt xt0 dt = λt0 xt dt + λ0 x0 λ T xT (83)
0 0
Now we can regard the integral as the sum and take …rst derivatives
w.r.t. ct and xt :
rc (ct , xt , t ) + λt fc (ct , xt , t ) = 0 (84)

rx (ct , xt , t ) + λt fx (ct , xt , t ) + λt0 = 0 (85)

Hamiltonian Function
Similarly, we can de…ne a new function H, a Hamiltonian function,
H = r (ct , xt , t ) + λt [f (ct , xt , t )] . (86)
Hence, the FOC for ct , (84), can be rewritten as
∂Ht
= 0. (87)
∂ct
Theorem (The Maximum Principle)

The necessary FOCs for maximizing (81) subject to (79) are:
(i) Hc , ∂H∂ct = 0 for all t 2 [0, T ] .
t
0
(ii) xt = f (ct , xt , t ) for all t (The state transition equation)
(iii) Hx + λt0 = 0 rx (ct , xt , t ) + λt fx (ct , xt , t ) + λt0 = 0 for all t (The
costate (λt ) equation)
(iv) xT 0, λT 0, xT λT = 0 (The transversality condition, or the
complementary-slackness condition)
Continuous-time Version of the Consumption-Saving Model
Consider the following deterministic continuous-time optimal

consumption-saving problem:
Z T
max u (ct ) exp ( ρt ) dt, (88)
ct 0
subject to
at0 = rat ct , (89)
where ρ is called the discount rate, r is the instantaneous interest
rate, given a0 , and aT 0 (preventing the consumer from dying with
debts).
Here we assume that the utility function u (ct ) satis…es the usual
conditions (just like those discussed in the discrete-time case) and
ct > 0.

The Hamiltonian for this problem is
H = u (ct ) exp ( ρt ) + λt (rat ct ) . (90)
Using the maximum principle,
Hc = 0 =) λt = u 0 (ct ) exp ( ρt ) , (91)

Ha + λt0 = 0 =) r λt + λt0 = 0, (92)
at0 = rat ct . (93)
In addition, aT = 0, i.e., it is optimal for the consumer to leave no

bequest after T . Note that if at optimum aT > 0, λT > 0, which
contradicts (91).

(conti.) Using (92), we have
λt0
= r, (94)
λt
which can be easily solved for λt :
λt = λ0 exp ( rt ) . (95)
1 γ
ct 1
Suppose that u (ct ) = 1 γ . Then combining (91) with (95) gives
γ
ct = c0 γ exp ((ρ r ) t) . (96)

(conti.) Di¤erentiating (92) gives
at00 = rat0 ct0 .
Since
dct u 0 ( ct ) ct
ct0 , = 00 (ρ r) = (r ρ) ,
dt u ( ct ) γ
we have
ct
at00 = rat0 + (ρ r )
γ
1
= rat0 + (ρ r ) rat at0
γ
1 1
= r (ρ r ) at0 + r (ρ r ) at , (97)
γ γ
which is a second-order di¤erential equation (SODE) in terms of at .

(conti.) For r 6= γ1 (ρ r ), (i.e., no repeated roots for the
corresponding
h characteristic
i equation,
2 1 1
λ r γ (ρ r ) λ γ r (ρ r ) = 0,) this SODE has the
following general solution:
1
at = A exp (rt ) + B exp (ρ r) t , (98)
γ
where A and B can be pinned down by using the initial and terminal
conditions:
a0 = A + B, (99)
1
aT= A exp (rT ) + B exp (ρ r) T (= 0) . (100)
γ
h i
1
From (100), we have A = B exp γ (r ρ) r T .
Substituting it into (99):
1
B = 1 exp (r ρ) r T a0 , (101)
γ
1
A = exp (r ρ ) r T
Luo, Y. (SEF of HKU) ECON6012:γ
Macro Theory September 6, 2013 51 / 56
Optimal consumption:
1
ct , rat at0 = r A exp (rt ) + B exp (ρ r ) t
γ
1 1
Ar exp (rt ) (ρ r ) B exp (ρ r ) t (102)
.
γ γ
Consider a special case in which r = ρ:
u 0 ( ct )
ct0 = (ρ r ) = 0,
u 00 (ct )
i.e., ct = c, independent of time t. In this case, at0 = rat ct
becomes at0 = rat c, whose solution is:
c
at = exp (rt ) a0 (1 exp ( rt )) . (103)
r
c
At T , we have aT = exp (rT ) a0 r (1 exp ( rT )) = 0:
r
c= a0 . (104)
1 exp ( rT )
Continuous-time Bellman Equation (Optional)
Consider the same deterministic continuous-time consumption-saving

problem: Z ∞
J (a0 ) = max u (ct ) exp ( ρt ) dt (105)
ct 0
subject to at0 = rat ct , given a0 , and aT 0. Here J (a0 ) is de…ned

as the value function.
(105) can be rewritten as the following recursive form:
Z ∆t Z ∞
J (a0 ) = max u (ct ) exp ( ρt ) dt + u (ct ) exp ( ρt ) dt
c 0 ∆t
= max fu (c0 ) ∆t + exp ( ρ∆t ) J (a∆t )g (106)
c0
for a small interval, ∆t.

(conti.) Here we use the facts:
Z ∆t Z ∆t
u (ct ) exp ( ρt ) dt = u (c0 ) exp ( ρt ) dt = u (c0 ) ∆t,
Z0 ∞ 0
u (ct ) exp ( ρt ) dt
∆t
Z ∞
= exp ( ρ∆t ) u (ct ) exp ( ρ (t ∆t )) d (t ∆t )
0
, exp ( ρ∆t ) J (a∆t ) . (107)
Linearizing J (a∆t ) around a0 :
J (a∆t ) = J (a0 ) + J 0 (a0 ) (a∆t a0 ) .

(conti.) Substituting it back into (106):
J ( a0 ) = max u (c0 ) ∆t + (1 ρ∆t ) J (a0 ) + J 0 (a0 ) (a∆t a0 )

c0
u (c0 ) ∆t + J 0 (a0 ) (a∆t a0 )
() ρ∆tJ (a0 ) = max ,(108
c0 ρ∆tJ 0 (a0 ) (a∆t a0 )
where we use the fact that exp ( ρ∆t ) = 1 ρ∆t.

Dividing ∆t on both sides:
a ∆t a0
u (c0 ) + J 0 (a0 ) lim∆t !0 ∆t
ρJ (a0 ) = max
c0 ρJ 0 (a0 ) lim∆t !0 (a∆t a0 )
= max u (c0 ) + J 0 (a0 ) a00
c0
= max u (c0 ) + J 0 (a0 ) (ra0 c0 ) (109)
c0
and taking derivative w.r.t. the control c0 : u 0 (c0 ) = J 0 (a0 ), which

implies that c0 = h (a0 ).

(conti.) Substituting c0 = h (a0 ) into the Bellman equation (109):
ρJ (a0 ) = u (h (a0 )) + J 0 (a0 ) [ra0 h (a0 )] . (110)
The envelop theorem implies that
ρJ 0 (a0 ) = u 0 (h (a0 )) h0 (a0 ) + J 00 (a0 ) [ra0 h (a0 )] (111)

0 0
+ J ( a0 ) r h ( a0 )
= J 00 (a0 ) [ra0 h (a0 )] + J 0 (a0 ) r , (112)
which means that

dc0
(ρ r ) u 0 (c0 ) = u 00 (c0 ) [ra0 h (a0 )] , (113)
da0
c00 ( ρ r ) u 0 ( c0 ) c0
= 00
, (r ρ ) , (114)
c0 c0 u ( c0 ) γ
dc0
where we use the fact that u 00 (c0 ) da 0
= J 00 (a0 ).

Topic 1: Introduction To Intertemporal Optimization: Yulei Luo

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Topic 1: Introduction To Intertemporal Optimization: Yulei Luo

Uploaded by

Copyright:

Available Formats

Topic 1: Introduction to Intertemporal Optimization

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 1 / 56

In static optimization, the task is to …nd a single value for each

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 2 / 56

In the dynamic discrete-time model, the objective should take the

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 3 / 56

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 4 / 56

The FOCs for xt +1 :

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 5 / 56

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 6 / 56

Suppose that a consumer has the utility function u (ct ) , where ct is

u 0 > 0, u 00 < 0, (6)

and u (ct ) satis…es the Inada conditions (They ensure that

lim u 0 (c ) = ∞ and lim u 0 (c ) = 0. (7)

The consumer is also endowed with a0 = a (0) , and has income

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 7 / 56

The consumer’s lifetime utility maximization problem is to

at +1 = Rat ct , t = 0, ,T, (10)

where β is the consumer’s rate of time preference (β 1) and

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 8 / 56

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 9 / 56

Setting up the Lagrangian:

u 0 ( ct ) λt = 0 for t = 0, 1, ,T. (13)

λt + βRλt +1 = 0 for t = 0, ,T 1. (14)

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 10 / 56

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 11 / 56

u 0 (ct ) = βRu 0 (ct +1 ), (16)

which is called the Euler equation characterizing optimal consumption

where γ 0 (Note that when γ = 1, the function reduces to ln ct ).

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 12 / 56

Hence, increasing γ i.e., reducing γ1 make the agent more unwilling

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 13 / 56

(conti.) (10) implies that

where we use the fact that aT +1 = 0.

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 14 / 56

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 16 / 56

Instead of de…ning the Lagrange multiplier for each ‡ow budget

where λ is the constant Lagrange multiplier for the lifetime budget

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 17 / 56

u 0 (ct ) = βRu 0 (ct +1 ), where t = 0, ,T 1,

which is identical to those Euler equations we derived above.

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 18 / 56

There are some reasons to consider the in…nite horizon (T = ∞) case:

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 19 / 56

The consumer’s lifetime utility maximization problem

subject to at +1 = Rat ct , 8 t 0, given the initial asset holdings

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 20 / 56

Applying (28), we have

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 21 / 56

because (28) is binding, i.e., limT !∞ aRt +t 1 = 0 (Otherwise, the

The FOCs w.r.t. ct for any t are

u 0 (ct ) = βRu 0 (ct +1 ) , 8t 0. (32)

and consumption in periods t 1 can be recovered:

Similarly, we can determine the optimal time path of asset holdings:

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 23 / 56

subject to as +1 = Ras cs , 8s t, given the initial asset holdings

J (at ) = max fu (ct ) + βJ (at +1 )g , (35)

Luo, Y. (SEF of HKU) ECON6012: Macro Theory September 6, 2013 25 / 56

J (at ) = max fu (ct ) + βJ (Rat ct )g . (37)

The …rst-order condition is thus

Note that (38) gives optimal consumption as an implicit function of

J (a) = u (c (a)) + βJ (Ra c (a)) . (39)

Taking di¤erentiation w.r.t. a gives