48 views

Uploaded by fabianmauricio482

- Report Evaporator Sculptor Behr 120405
- HYDROTHERMAL COORDINATION FOR SHORT RANGE FIXED HEAD STATIONS USING FAST GENETIC ALGORITHM
- Ding 2008
- ppids.ps
- Lab9 Graded Solutions
- Bellman
- OR
- tv_23_2016_4_937_944
- 287
- Presentation Towards reliability-based structural fire safety: development and probabilistic applications of a direct stiffness method for concrete frames exposed to fire
- Economia Investigacion Mod Estoc
- water management strategies.pdf
- An Introduction to Reinforcement Learning
- 117FZ112017 - Copy
- Batch Size JIT
- Scheduling
- CAE FRAME WORK FOR AERODYNAMIC DESIGN DEVELOPMENT OF AUTOMOTIVE VEHICLES
- Recent Advances in Learning and Control
- Midterm Solution I (1)
- Master’s of Science in ElectricalElectronics Engineering

You are on page 1of 30

The University of Auckland Year

Programming in Macroeconomic Models

Ian King

University of Auckland, ip.king@auckland.ac.nz

http://researchspace.auckland.ac.nz/ecwp/230

A Simple Introduction to

Dynamic Programming in

Macroeconomic Models

Ian King*

Department of Economics

University of Auckland

Auckland

New Zealand

April 2002

(October 1987)

Abstract

This is intended as a very basic introduction to the mathematical methods used in Thomas

Sargent's book Dynamic Macroeconomic Theory. It assumes that readers have no further

mathematical background than an undergraduate "Mathematics for Economists" course. It

contains sections on deterministic finite horizon models, deterministic infinite horizon

models, and stochastic infinite horizon models. Fully worked out examples are also provided.

*

Financial support of the Social Sciences and Humanities Research Council of Canada is gratefully

acknowledged. I am indebted to David Backus, Masahiro Igarashi and for comments. I would like to thank Tom

McCurdy for his comments on earlier drafts, and for suggesting the last example presented here.

FOREWARD (2002)

I wrote this guide originally in 1987, while I was a graduate student at Queen’s University at

Kingston, in Canada, to help other students learn dynamic programming as painlessly as

possible. The guide was never published, but was passed on through different generations of

graduate students as time progressed. Over the years, I have been informed that several

instructors at several different universities all over the world have used the guide as an

informal supplement to the material in graduate macro courses. The quality of the photocopies

has been deteriorating, and I have received many requests for new originals. Unfortunately, I

only had hard copy of the original, and this has also been deteriorating.

Because the material in this guide is not original at all – it simply summarizes material

available elsewhere, the usual outlets for publication seem inappropriate. I decided, therefore,

to simply reproduce the handout as a pdf file that anyone can have access to. This required re-

typing the entire document. I am extremely grateful to Malliga Rassu, at the University of

Auckland for patiently doing most of this work. Rather than totally reorganize the notes in

light of what I’ve learned since they were originally written, I decided to leave them pretty

much as they were – with some very minor changes (mainly references). I hope people

continue to find them useful.

INTRODUCTION (1987)

This note grew out of a handout that I prepared while tutoring a graduate macroeconomics

course at Queen's University. The main text for the course was Thomas Sargent's Dynamic

Macroeconomic Theory. It had been my experience that some first year graduate students

without strong mathematical backgrounds found the text heavy going, even though the text

itself contains an introduction to dynamic programming. This could be seen as an introduction

to Sargent's introduction to these methods. It is not intended as a substitute for his chapter, but

rather, to make his book more accessible to students whose mathematical background does

not extend beyond, say, A.C. Chaing's Fundamental Methods of Mathematical Economics.

The paper is divided into 3 sections: (i) Deterministic Finite Horizon Models, (ii)

Deterministic Infinite Horizon Models, and (iii) Stochastic Infinite Horizon Models. It also

provides five fully worked out examples.

1

1. DETERMINISTIC, FINITE HORIZON MODELS

Let us first define the variables and set up the most general problem, (which is usually

unsolvable), then introduce some assumptions which make the problem tractable.

{v(t )}

subject to i) G ( x0 , x1 ,⋅ ⋅ ⋅, xT ; v0 ,v1 ,⋅ ⋅ ⋅,vT −1 ) ≥ 0

iii) x0 = x0 given

iv) xT ≥ 0

Where: xt is a vector of state variables that describe the state of the system at any point in

i

time. For example, xt could be the amount of capital good i at time t.

j

decision-maker. For example vt could be the consumption of good j at time t.

U (⋅) is the objective function which is, in general, a function of all the state and

control variables for each time period.

variables.

Ω is the feasible set for the control variables – assumed to be closed and

bounded.

In principle, we could simply treat this as a standard constrained optimisation problem. That

is, we could set up a Lagrangian function, and (under the usual smoothness and concavity

assumptions) grind out the Kuhn-Tucker conditions.

2

In general though, the first order conditions will be non-linear functions of all the state and

control variables. These would have to be solved simultaneously to get any results, and this

could be extremely difficult to do if T is large. We need to introduce some strong assumptions

to make the problem tractable.

Here it is assumed that both the U (⋅) and the G (⋅) functions are time-separable. That is:

where S ( xT ) is a "scrap" value function at the end of the program (where no further decisions

are made). Also, the G (⋅) functions follow the Markov structure:

x1 = G0 ( x0 , v0 )

x2 = G1 ( x1 , v1 ) “Transition equations”

xT = GT −1 ( xT −1, vT −1 )

i

Note: Recall that each xt is a vector of variables xt where i indexes different kinds of state

state & control variables, but only within periods.

Max

v t ; t = 0 ,1,..., T −1

T −1

t =0

U t ( xt , vt ) + S ( xT )

v t ∈Ω

i i

ii) x0 = x0 given ∀i = 1 ⋅ ⋅ ⋅ n

Once again, in principle, this problem could be solved using the standard constrained

optimisation techniques. The Lagrangian is:

3

L=

T −1

U t ( xt ,vt ) + S ( xT ) +

λ [G (x ,v ) − x ]

T −1 n

i

t

i

t t t

i

t +1

t =0 t = 0 i =0

This problem is often solvable using these methods, due to the temporal recursive structure of

the model. However, doing this can be quite messy. (For an example, see Sargent (1987)

section 1.2). Bellman's "Principle of Optimality" is often more convenient to use.

Problem A:

Max

v t ; t = 0 ,1,..., T −1

T −1

t =0

U t ( xt , vt ) + S ( xT )

v t ∈Ω

i i

ii) x0 = x0 given ∀i = 1 ⋅ ⋅ ⋅ n

Problem B:

Max

vt ; t = t 0 ,..., T −1

t =t 0

T −1

U t ( x t , vt ) + S ( xT )

vt ∈Ω

s ( )

"Principle of Optimality" asserts:

xt 0 ≡ xt 0 must also solve Problem B (i.e.: on the range t = t0 ⋅ ⋅ ⋅ T ).

4

(Note: This result depends on additive time separability, since otherwise we couldn'

t "break"

the solution at t0 . Additive separability is sufficient for Bellman'

s principle of optimality.)

Interpretation: If the rules for the control variables chose for the t0 problem are optimal for

*

any given xt 0 , then they must be optimal for the xt 0 of the larger problem.

s P. of 0. allows us to use the trick of solving large problem A by solving the smaller

Bellman'

problem B, sequentially. Also, since t0 is arbitrary, we can choose to solve the problem

Max U T −1 ( xT −1 ,vT −1 ) + S ( xT )

{vT −1}

subject to: i) xT = GT −1 ( xT −1 , v T −1 )

ii) xT −1 = xT −1 given

One can easily substitute the first constraint into the objective function,

and use straightforward calculus to derive:

This can then be substituted back into the objective fn to characterize the

solution as a value function:

5

Step 2: Set t0 = T − 2 so that Problem B is:

Max

!" v

T −1

{U T −2 (xT − 2 , vT − 2 ) + U T −1 (xT −1 , vT −1 ) + S (xT )}

v

T −2

ii) xT −1 = GT − 2 ( xT − 2 ,vT − 2 )

iii) xT − 2 = xT − 2 given

Bellman'

s P.O. implies that we can rewrite this as:

'(& U %

Max

vT - 2

T −2 (xT − 2 , vT − 2 ) + Max

v

{U T −1 (xT −1 , vT −1 ) = S (xT )}

T −1 #$

subject to (i), (ii) and (iii).

Recall that step 1 has already given us the solution to the inside maximization

problem, so that we can re-write step 2 as:

Max {U T − 2 ( xT − 2 , vT − 2 ) + V ( xT −1 ,1)}

{vT − 2 }

ii) xT − 2 = xT − 2 given

Once again, we can easily substitute the first constraint into the objective function,

and use straightforward calculus to derive:

This can be substituted back into the objective fn. to get a value function:

{vT − 2 }

6

Step 3: Using an argument analogous to that used in step 2 we know that, in general, the

problem in period T-k can be written as:

" Bellman's"

V ( xT − k , k ) = Max {U T − k ( x T − k , v T − k ) + V ( xT − k +1 , k − 1)}

{ vT − k } Equation

ii) xT − k = xT − k given

This maximization problem, given the form of the value function from the

previous round, will yield a control rule:

vT − k = hT − k ( xT − k )

Step 4: After going through the successive rounds of single period maximization

problems, eventually one reaches the problem in time zero:

Max {U 0 ( x 0 , v 0 ) + V ( x1 , T − 1)}

V (x 0 , T ) =

{v 0 }

ii) x0 = x0 given

v0 = h0 ( x0 )

Now, recall that x0 is given a value at the outset of the overall dynamic

7

Step 5: Using the known x 0 and v 0 and the transition equation:

x1 = G0 ( x0 ,v0 )

it is simple to work out x1 , and hence v1 from the control rule of that period.

This process can be repeated until all the xt and vt values are known. The

overall problem A will then be solved.

1.4 An Example:

This is a simple two period minimization problem, which can be solved using this algorithm.

Min

) 1

[x 2

t ]

+ v t2 + x 22 (1)

{ vt }

t =0

ii) x0 = 1 (3)

In this problem, T=2. To solve this, consider first the problem in period T-1 (i.e.: in period 1):

Step 1:

{

Min x12 + v12 + x 22 } (4)

{v1 }

subject to: i) x2 = 2 x1 + v1 (5)

ii) x1 = x1 given (6)

{

Min x12 + v12 + [2 x1 + v1 ]

2

}

{v1 }

FOC: 2v1 + 2[2 x1 + v1 ] = 0

* v = −x

1 1 Control rule in period 1 (7)

8

Now substituting 7 back into 4 yields (using 5):

V ( x1 ,1) = x1 + x1 + (2 x1 − x1 )

2 2 2

* V (x ,1) = 3x1

2

1 (8)

s equation tells us that the problem is:

{

Min x 02 + v 02 + V ( x1 ,1) } (9)

{v 0 }

ii) x0 = 1 (11)

{

Min 1 + v 02 + 3[2 + v 0 ]

2

}

{v 0 }

FOC: 2v 0 + 6[2 + v 0 ] = 0

+ v0 =

−3

2

Control value in period 0 (12)

x1 + 2 +

/01 − 3 . ,- + x1 =

1 (13)

2 2

−1

v1 = Control value in period 1 (14)

2

x2 = 1 −

1

2

+ x2 =

1

2

(15)

9

2. DETERMINISTIC, INFINITE HORIZON MODELS

2.1. Introduction:

One feature of the finite horizon models is that, in general the functional form of the

control rules vary over time:

vt = ht ( xt )

That is, the h function is different for each t. This is a consequence of two features of

the problem:

ii) The fact that U t ( xt ,vt ) and Gt ( xt ,vt ) have been permitted to depend on time

in arbitrary ways.

In infinite horizon problems, assumptions are usually made to ensure that the control

rules to have the same form in every period.

6 75 Max

vt ;t = 0,..., ∞

42 3 8 ∞

t =0

U t ( xt , v t )

v t ∈Ω

subject to: i) x t +1 = G t ( x t , v t )

ii) x0 = x0 given

For a unique solution to any optimization problem, the objective function should be

bounded away from infinity. One trick that facilitates this bounding is to introduce a

discount factor β t where 0 ≤ β t < 1 .

is stationarity:

Assume i) β t = β ∀t

10

A further assumption, which is sufficient for boundedness of the objective function, is

boundedness of the payoff function in each period:

This assumption, however, is not necessary, and there are many problems where this is

not used.

= >< Max

vt ; t = 0,..., ∞

;9 : ? ∞

t =0

β t U ( xt , v t )

v t ∈Ω

subject to: i) x t +1 = G ( x t , v t )

ii) x0 = x0 given

Vt ( xt ) =

{

Max β U ( xt ,vt ) + Vt +1 ( xt +1 )

t

}

{vt }

subject to (i) and (ii)

This equation is defined in present value terms. That is, the values are all discounted

back to time t = 0. Often, it is more convenient to represent these values in current

value terms:

vt ( xt )

Define: Wt ( xt ) = t

β

−t

Multiplying both sides of the Bellman equation by β yields:

Max{U ( x t , v t ) + βWt +1 ( x t +1 )}

Wt ( x t ) = (*)

{vt }

subject to (i) and (ii).

1

A subtle change in notation has been introduced here. From this point on, Vt ( xt ) represents the form of the

value function in period t.

11

It can be shown that the function defined in (*) is an example of a contraction mapping. In

Sargent’ s (1987) appendix, (see, also, Stokey et al, (1989)) he presents a powerful result

called the “contraction mapping theorem”, which states that, under certain regularity

conditions, 2 iterations on (*) starting from any bounded continuous W0 (say, W0 = 0 ) will

cause W to converge as the number of iterations becomes large. Moreover, the W(.) that comes

out of this procedure is the unique optimal value function for the infinite horizon

maximization problem. Also, associated with W(.) is a unique control rule vt = h( xt ) which

This means is that there is a unique time invariant value function W(.) which satisfies:

Max{U ( x t , v t ) + βW ( x t +1 )}

W (xt ) =

{v t }

subject to: i) x t +1 = G ( x t , v t )

ii) x0 = x0 given.

Associated with this value function is a unique time invariant control rule:

vt = h( xt )

There are several different methods of solving infinite horizon dynamic programming

problems, and three of them will be considered out here. In two, the key step is finding

the form of the value function, which is unknown at the outset of any problem even if

the functional forms for U(.) and G(.) are known. This is not required in the third.

2

Stokey et al., (1989) chapter 3 spell these conditions out. Roughly, these amount to assuming that U ( xt ,vt ) is

bounded, continuous, strictly increasing and strictly concave. Also, the production technology implicit in

G ( xt ,vt ) must be continuous, monotonic and concave. Finally, the set of feasible end of period x must be

compact

12

Consider the general problem:

D EC Max

vt ; t = 0,..., ∞

B@ A F ∞

t =0

β t U ( xt , v t )

v t ∈Ω

ii) x0 = x0 given

Max{U ( x t , v t ) + βWt +1 ( x t +1 )}

Wt ( x t ) =

{v t }

s.t. i) xt +1 = G ( xt , vt )

ii) x0 = x0 given.

Set Wt +1 ( xt +1 ) = 0 , and solve the maximization problem on the right side of the Bellman

equation. This will give you a control rule vt = ht ( xt ). Substitute this back into

U ( xt ,vt ), to get:

Wt ( xt ) = U ( xt ,ht ( xt ))

Max{U ( x t −1 , v t −1 ) + βU ( x t , h( x t ))}

Wt −1 ( x t −1 ) =

{v t −1 }

s.t. i) xt = G ( xt −1 ,vt −1 )

ii) x0 = x0 given.

Wt −1 ( x t −1 ) =

{v t −1 }

13

Solve the maximization problem on right, to get: vt −1 = ht −1 ( xt −1 )

.

Continue this procedure until the W(.)'

s converge. The control rule associated with the

W(.) at the point of convergence solves the problem.

This method is a variant of the first method, but where we make an informed guess

about the functional form of the value function. If available, this method can save a lot

computation. For certain classes of problems, when the period utility function U ( x t , v t )

lies in the HARA class (which includes CRRA, CARA, and quadratic functions), the

value function takes the same general functional form.3 Thus, for example, if the period

utility function is logarithmic, we can expect the value function will also be logarithmic.

Max {U ( x t , v t ) + βW ( x t +1 )}

W (xt ) =

{v t }

subject to: i) xt +1 = G ( xt , vt )

ii) x0 = x0 given.

G

(xt +1 ) , and substitute the guess into the

Bellman equation:

W (xt ) =

{

Max U ( x t , v t ) + βW G ( x t +1 ) } s.t. (i) and (ii)

{v t }

3

See Blanchard and Fischer (1989), chapter 6, for a discussion.

14

Third, perform the maximization problem on the right side. Then substitute the resulting

control rules back into the Bellman equation to (hopefully) verify the initial guess. (i.e.: The

guess is verified if W ( xt ) = W (xt +1 ) .)

G

If the initial guess was correct, then the problem is solved. If the initial guess was incorrect,

try the form of the value function that is suggested by the first guess as a second guess, and

repeat the process. In general, successive approximations will bring you to the unique

solution, as in method 1.

This is Sargent'

s preferred method of solving problems in his Dynamic Macro Theory

textbook. The advantage of using this method is that one is not required to know the form of

the value function to get some results. From Sargent [1987] p. 21, the B-S formula is:

∂W ( x t ) ∂U ( x t , h( x t )) β∂G (x, h( x )) ∂W (G ( x, h( x ))

= + B-S Formula 1

∂x t ∂x t ∂x t ∂x t +1

Sargent shows that if it is possible to re-define the state and control variables in such a

way as to exclude the state variables from the transition equations, the B-S formula

simplifies to:

∂W ( xt ) ∂U ( xt ,h( xt ))

= B-S Formula 2

∂xt ∂xt

(1) Redefine the state and control variables to exclude the state variables from the

transition equations.

(2) Set up the Bellman equation with the value function in general form.

(3) Perform the maximization problem on the r.h.s. of the Bellman equation. That is,

derive the F.O.C.'

s.

(4) Use the B-S Formula 2 to substitute out any terms in the value function.

As a result, you will get an Euler equation, which may be useful for certain purposes,

although it is essentially a second-order difference equation in the state variables.

15

2.3. Example 2: The Cass-Koopmans Optimal Growth Model.

G β H nC

∞

Max t

t s.t. i) k t +1 = Ak tα − C t

{ct , kt +1 } t = 0

ii) k0 = k0 given.

Where : C t ≡ consumptio n

kt ≡ capital stock

0 < β <1

0 <α <1

Since the period utility function is in the HARA class, it is worthwhile to use the guess-

verify method.

I

Max { nC t + βW (k t +1 )}

W (k t ) = s.t. (i) and (ii) (1)

{C t , k t +1 }

W (kt ) = E + F n k t J (2)

)

) into (1) to get:

W (kt ) =

Max {M n[Ak t

α

] M }

− kt +1 + β [E + F n kt +1 ]

(3)

{kt +1}

1 βF

FOC: α =

Akt − kt +1 kt +1

16

N kt +1 =

βF

1 + βF

α

Akt (4)

RST Q RST Q

U

W (k ) = n Ak

t

α

t −

βF

1 + βF

PO

Ak tα + βE + BF n

βF

U

1 + βF

Ak tα PO

Z XZ X

V W (k ) = [\] W n A − W n(1 + βF ) + βE + βF W n A + βF W n[\]

t

βF

1 + βF

Y Y + α (1 + βF )W n k t (5)

Now, comparing equations (2) and (5), we can determine the coefficient for n k t :

^

F = α (1 + βF )

_ F=

α

1 − αβ

(6)

Since both the coefficient E and F have been determined, the initial guess (equation 2)

has been verified.

α

kt +1 = αβAkt and using constraint (ii): (7)

C t = [1 − αβ ]Ak tα (8)

Given the initial k0 equations (7) - (8) completely characterize the solution paths of C and k.

17

2.4. Example 3: Minimizing Quadratic Costs

Min

` ∞

β t [x t2 + v t2 ]

{vt } t =0

subject to i) xt +1 = 2 xt + vt

ii) x0 = x0 given

W ( xt ) =

{

Min x t + vt + βW ( xt +1 )

2 2

} (1)

{vt }

s.t. i) xt +1 = 2 xt + vt

ii) x0 = x0 given

W ( xt ) = Pxt

2

(2)

a W ( xt +1 ) = Pxt +1

2

(2'

)

), then the result into (1) to get:

W ( xt ) =

{

Min x t + vt + βP[2 xt + vt ]

2 2 2

} (3)

{vt }

FOC: 2vt + 2 βP[2 xt + vt ] = 0

b v t [1 + βP] = −2 βPx t

b vt = −

2 βP

1 + βP

xt (4)

18

Substitute (4) back into (3) to get:

h g f e c fgh 2βP e c 2

x d + βP 2 x − xd

2

2 βP

W (x ) = x + −

2

1 + βP 1 + βP

t t t t t

n l m k n m l k k

t

m n l

b W (x ) = l 1 − 1 + βP ij j ii x + βP 2 − 1 + βP j i x

2 βP

2

2 β P 2

t

2

2

t

o W (x ) = pt 1 − 1 + βP x + βP 2 − 1 + βP x q p x

2 2

2

t t (5)

} 2 βP

}

P = } 1 −
+ βP 2 − 1 + βP
~ }

2 2

2 βP

(6)

1 + βP

The P that solves (6) is the coefficient that was to be determined. For example, set

β = 1, and equation (6) implies:

P = 4.24

For a given x0 , equations (7) and (8) completely characterize the solution. Note from

equation 8 that the system will be stable since 0.38 < 1.

19

2.5. Example 4: Life Cycle Consumption

Max

{ct }t = 0

∞

∞

t =0

β t U (C t ) (1)

Where: Ct ≡ consumptio n

yt ≡ known, exogenously given income stream

At ≡ non - labour wealth beginning in time t

Rt ≡ known, exogenous sequence of one-period rates of return on At

present value budget constraint is imposed:

∏R C ∏R y

∞ j −1 ∞ j −1

−1 −1

C +t t +k t+ j =y +

t t+k t+ j + At (4)

j =1 k =0 j =1 k =0

Here, we are not given an explicit functional form for the period utility function. We can

use method 3 in this problem to derive a well-known result.

To use the B-S formula 2, we need to define the state and control variables to exclude

state variables from the transition equation.

At +1

Control Variable: vt ≡ (5)

Rt

Now we can re-write the transition equation as:

At +1 = Rt vt (2'

)

Notice that (2' (6)

20

The problem becomes:

Max

∞

β t U ( At + y t − v t )

t =0 (1'

)

{v } ∞

t t =0

s.t. i) At +1 = Rt vt (2'

) Reformulated problem

{v t }

Substituting (2'

) into the Bellman equation yields:

Max {U ( At + y t − v t ) + βW ( Rt v t , y t +1 , Rt )}

W ( At , y t , Rt −1 ) = { vt }

W1 (Rt vt , yt +1 , R ) = U' A

t t +1 + yt +1 −

Rt +1

At + 2

≡ U ' (Ct +1 ) (8)

Now, to get further results, let’ s impose structure on the model by specifying a

particular utility function:

Let U (Ct ) ≡ n Ct

U'(C ) = C1

t

t

(10)

21

Substituting 10 into 9 yields:

C t+2

2

= β ∏ Rt + k Ct

1

k =0

In general: Ct + j = β

j ∏ R C

j −1

t +k t (12)

k =0

Ct +

∞

j =1

j −1

k =0

−1

t +k

j

j −1

k =0

t+k t t

∞

j =1

j −1

k =0

−1

t+k t+ j + At

¨ C +© β C = " t

∞

j =1

j

" t "

¨ ª βC= " ∞

j =0

j

" t "

¨ Ct

1− β

= " " "

° ¯ ® ¯ ° ®

¨ C = (1 − β ) y + ± ∏ R ¬ « y

t t

∞

j =1 k = 0

j −1

−1

t+k t+ j + At ¬« Life Cycle Consumption

Function

3.1. Introduction:

stationary, and boundedness of the payoff function. However, stochastic models are more

general than deterministic ones because they allow for some uncertainty.

22

The type of uncertainty that is usually introduced into these models is of a very specific kind.

To preserve the recursive structure of the models, it is typically assumed that the stochastic

shocks the system experiences follow a homogeneous first order Markov process.

In the simplest terms, this means that the value of this period'

s shock depends only on the

value of last period'

s shock, not upon any earlier values. (White noise shocks are a special

case of this: they do not even depend upon last period'

s shock.)

¶ ·µ Max

vt ;t = 0,..., ∞

´² ³ E ¸

0

∞

t =0

β tU ( x t , vt )

v t ∈Ω

ii) x0 given

where {ε t }t =0 is a sequence of random shocks that take on values in the interval [ε , ε ] and

∞

follow a homogeneous first order Markov process with the conditional cumulative density

function

(

I t = {x k }tk = 0 , {v k }tk = 0 , {ε k }tk =0 , G (.),U (.), F (.) )

The assumed sequence of events is:

(1) xt is observed

Max{U ( x t , v t ) + βE t W ( x t +1 , ε t +1 )}

W (xt , ε t ) =

{v t }

23

As in the deterministic infinite horizon problem, under regularity conditions4 iterations on

Max{U ( x t , v t ) + βE t Wt +1 ( x t +1 , ε t +1 )}

Wt ( x t , ε t ) =

{v t }

starting from any bounded continuous Wt +1 (say, Wt +1 = 0 ) will cause W (.) to converge

as the number of iterations becomes large. Once again, the W (⋅) that comes out of this

procedure is the unique optimal value function for the above problem.

which solves the maximization problem.

The solution techniques given in the deterministic infinite horizon problem still work in the

stochastic infinite horizon problem, and there is no need to repeat them. Perhaps the best way

to illustrate these results is by using an example.

This example is chosen not only to illustrate the solution techniques given above, but also to

introduce the reader to a particular type of model. This modelling approach is used in the

"Real Business Cycle" literature of authors such as Kydland and Prescott [1982] and Long

and Plosser [1983].

Under the appropriate assumptions made about preferences and technology in an economy,

the following optimal growth model can be interpreted as a competitive equilibrium model. 5

That is, this model mimics a simple dynamic stochastic general equilibrium economy.

Because of this, the solution paths for the endogenous variables generated by this model can

be compared with actual paths of these variables observed in real-world macroeconomies.

4

The same conditions as in Note 2, with the added assumption of homogeneous first order Markov shocks.

5

See Stokey et al., (1989), chapter 15 for further discussion on this topic.

24

Consider the following problem:

Max E0 ¹ ∞

t=0

β

t

º º

[ n Ct + δ n(1 − nt )] (1)

{Ct ,kt ,nt }

subject to: i) Ct + kt = yt (2)

α 1 −α

ii) yt +1 = At +1kt nt (3)

iii) » nA t +1 »

= ρ n At + ξt +1 (4)

parameter which lies in the interval (-1, 1), and ξt represents a white noise error process.6

The endogenous variable yt represents output in period t. The rest of the notation is the same

as in example 2.

These are only two substantive differences between this example and example 2. First, a

labour-leisure decision is added to the model, represented by the inclusion of the term (1 − nt )

in the payoff function (1) and the inclusion of nt in the production function (3). Second, the

to the Markov process (4).

Notice that in this example Ct ,kt , and nt are the control variables chosen every period by the

decision maker, whereas yt and At are the state variables. To solve this problem, we first set

up the Bellman equation:

W ( y t , At ) =

Max ¼ ¼

{[ n C t + δ n(1 − nt ) ] + βE t ( y t +1 , At +1 )}

(5)

{C t , k t , n t }

Since this is a logarithmic example, we can use the guess-verify method of solution. The

obvious first guess for the form of the value function is:

½

W ( yt , At ) = D + G n yt + H n At ½ (6)

2

6

For example, ξt is normally distributed with mean 0 and variance σξ .

25

where D, G and H are coefficients to be determined.

¾ ¿

W ( yt +1 , At +1 ) = D + G n yt +1 + H n At +1 ¿ (6'

)

) to get

À

W ( yt +1 , At +1 ) = D + G[ n At +1 + α n kt + (1 − α ) n nt ] + H n At +1À À À

Now substitute this into equation 5:

W [ y t , At ] = Á Á

Max {[ n C t + δ n(1 − n t )] + βE t [D + (G + H ) n At +1 + Gα n k t + G (1 − α ) n nt ]}

{C t , k t , n t }

Á Á Á

Using equation 2 to substitute out Ct in the above yields:

W [ y t , At ] = Â Â Â

Max {[ n ( y t − k t ) + δ n(1 − nt )] + βD + βGα n k t + βG (1 − α ) n n t β (G + H )E t n At +1 }

{k t , nt }

Â Â

(7)

nt :

−δ

1 − nt

+

βG (1 − α )

nt

=0 Ã nt =

βG (1 − α )

δ + βG (1 − α )

(8)

kt : −

1

yt − kt

+

βGα

kt

=0 Ä kt =

αβG

1 + αβG

yt (9)

Ë ÈÉÊ Ç Ë ÈÉÊ δ Ç Ë Ê É È Ç

y Æ +δ n Æ + βD + αβ n 1 + αβG y Æ Å

1 αβG

1 + αβG Å δ + βG (1 − α ) Å

W (y , A ) = n

t t t t

Ñ Ð Ï Î

βG (1 − α ) Ì

+ βG (1 − α )Ò n Í + β (G + H )E Ò n A (7'

)

δ + βG (1 − α ) t t +1

Ó Ó

Recall from equation 4 that n At +1 = ρ n At + ξ t +1 where ξ t +1 is a white noise error term

Ô

Et n At +1 = ρ n At . Ô

26

Collecting terms in equation 7'yields:

Õ Õ

W ( yt , At ) = (1 + αβG ) n yt + β (G + H )ρ n At + constants (10)

Ö

G = 1 + αβ G =

1

1 − αβ

(11)

H = ρβ (G + H ) Ö (1 −ρβαβ ) 2

(12)

Similarly, the constant D can be determined from equations 6 and 10, knowing the values of

G and H given in equations 11 and 12.

Since all the coefficients have been determined, the initial guess of the form of the value

function (equation 6) has been verified. We can now use equations 11 and 12 to solve for the

optimal paths of nt ,kt , and Ct as functions of the state yt .

β (1 − α )

From equation 8: nt = (14)

δ (1 − αβ ) + β (1 − α )

Notice that while the optimal kt and Ct are functions of the state yt , the optimal nt is not.

That is, nt is constant. This is a result that is peculiar to the functional forms chosen for the

preferences and technology, and it provides some justification for the absence of labour-

leisure decisions in growth models with logarithmic functional forms.

27

REFERENCES

Blanchard, O., and Fischer, S., Lectures on Macroeconomics, MIT Press, (1989).

Hill, New York (1984).

Kydland, F., and Prescott, E. "Time to Build and Aggregate Fluctuations", Econometrica 50,

pp. 1345-1370 (1982).

Long, J., and Plosser, C., "Real Business Cycles", Journal of Political Economy 90, pp. 39-69

(1983).

Manuelli, R., and Sargent, T., Exercises in Dynamic Macroeconomic Theory, Harvard

University Press, (1987).

Stokey, N., Lucas, R.E., with Prescott, E., Recursive Methods for Economic Dynamics,

Harvard University Press (1989).

28

- Report Evaporator Sculptor Behr 120405Uploaded bygosculptor
- HYDROTHERMAL COORDINATION FOR SHORT RANGE FIXED HEAD STATIONS USING FAST GENETIC ALGORITHMUploaded byAnonymous ox4aPr
- Ding 2008Uploaded byYoussef Majdoub
- ppids.psUploaded bycacacoco
- Lab9 Graded SolutionsUploaded bySandip Sureka
- BellmanUploaded bymsahrani
- ORUploaded byAbhishek Gupta
- tv_23_2016_4_937_944Uploaded byAli Veli
- 287Uploaded byjin11004
- Presentation Towards reliability-based structural fire safety: development and probabilistic applications of a direct stiffness method for concrete frames exposed to fireUploaded byRuben Van Coile
- Economia Investigacion Mod EstocUploaded byMeMo Gonzalez
- water management strategies.pdfUploaded byZafar Qadri
- An Introduction to Reinforcement LearningUploaded byangelvi
- 117FZ112017 - CopyUploaded byANKESH
- Batch Size JITUploaded byKao Akana
- SchedulingUploaded byAbdul Rahman
- CAE FRAME WORK FOR AERODYNAMIC DESIGN DEVELOPMENT OF AUTOMOTIVE VEHICLESUploaded byUmer Qureshi
- Recent Advances in Learning and ControlUploaded byRobson Cruz
- Midterm Solution I (1)Uploaded bySheenly Joy Abalajen
- Master’s of Science in ElectricalElectronics EngineeringUploaded byMin M. Gumkino
- OPTIMIZATION.pptUploaded bynmukherjee20
- Introduction to Operational ResearchUploaded byIordan Rata
- Direct search methods for unconstrained optimization.pptUploaded bymrudul patel
- [49] Multiobjective Switching State Selector for Finite States Model Predictive Control Based on Fuzzy Decision Making in a Matrix ConverterUploaded bynhatvp
- jbptunikompp-gdl-chandradar-33573-5-10.unik-a.pdfUploaded byAbi Guitara
- 05IPST160Uploaded byAishwaryNigam
- Stochastic ProgrammingUploaded byFang Tian
- 2011 (C. Dhavamani, T. Alwarsamy) review on optimization of machining operation.pdfUploaded byFikri Siplho
- Optimal Control Term ProjectUploaded byGene Perez
- 10.1.1.122Uploaded byKostas Petridis

- 300-cartillaUploaded byFabio Triana
- Creg089-2013Uploaded byO
- Circular108-2014Uploaded byfabianmauricio482
- Barro Final Exam Fall 2009Uploaded byfabianmauricio482
- Dividendo Económico de La PazUploaded byfabianmauricio482
- FX Cheat SheetUploaded byfabianmauricio482
- DOC-0001-2000Uploaded byfabianmauricio482
- 1 Ramsey RA ModelUploaded byfabianmauricio482
- 10.1.1.197.2500Uploaded byfabianmauricio482
- 600 Tutorial 7Uploaded byfabianmauricio482
- 627.10.Assign1Uploaded byfabianmauricio482
- Financial Literacy, Financial Education and Downstream Financial BehaviorsUploaded byfabianmauricio482

- FEM for Heat Transfer Problems (Finite Element Method) Part 2Uploaded bySibilla Unni
- systems in building security is door access control.pdfUploaded byvabs
- Computer Vision System Toolbox™ ReferenceUploaded byKhai Huynh
- Terminology for Petrel manual.docxUploaded byJoseGuevaraG
- Direct Stiffness MethodUploaded byMuhammad Shahid Hussain
- C Programming Examples on StringsUploaded bypostmanZZ
- Laplace Transform (Notes)Uploaded byAlex Oses
- simulation lab manual for signals and systemsUploaded byjagadeeshm.me
- Rsa ExampleUploaded byFadhillah Azmi
- Risk Appetite in Enterprise Risk Management FrameworkUploaded byjfrankparnell
- PROGRESS IN TOPOLOGY OPTIMIZATION WITH MANUFACTURING CONSTRAINTSUploaded byAltairEnlighten
- 100426_RobustDesign.pdfUploaded bydeeptish
- slide teori antrianUploaded byMuhammad Sandilaga
- 07cp18 Neural Networks and Applications 3 0 0 100Uploaded bytkec_csedept
- RSAUploaded byarchana_said
- Bessel Stirling Formula Numerical AnalysisUploaded byomed
- Chapter4Tutorial1(SOLUTION).docUploaded byPrice
- mukundan tchebichefUploaded byZaid Omar
- 88324 Control System - Solution Assignment 5Uploaded byresal3000
- Advanced Numerical Integration in MathematicaUploaded bymasondrool
- Threshold Logic SeminarUploaded byAkash Hulkod
- UPCcourse HandoutsUploaded bytasiafio
- BETTER PERFORMANCE OF SIMPLEX METHOD ON TRANSPORT NETWORK WITH NON ? INTEGER EDGE CAPACITIES.Uploaded byIJAR Journal
- Reed Solomon DocUploaded bykalkam
- Numerical Method Syllabus-SPFUUploaded byNicholas Thompson
- Ieee Tpe Ben MusicUploaded byVelibor Lazarević
- DSS_ Digital Signature Standard and DSA AlgorithmUploaded byShivam Verma
- 14666_1.AI Introduction(Lecture 1-5)Uploaded byprabjit_gill
- neural networksUploaded bySerag El-Deen
- homework_04.pdfUploaded byAnonymous 96NePMZ