Equality Constrained Endpoint

A maximization problem:

Z t1
0
max F t, x (t ) , x (t ) dt (1)
x t0

subject to x (t0 ) = x0 , R (t1 ) = x1 . (2)

where F is assumed to be continuous in its three arguments, and have
continuous partial derivatives with respect to the second and third
arguments.

() March 16, 2023 1 / 37

Equation (3) implies

R 0 (t1 ) = δx1 /δt1 . (3)

The maximization problem implies

Z t1
0
Fx dFx 0 /dt hdt + Fx 0 jt1 δx1 + F x Fx 0 jt1 δt1 = 0. (4)
t0

() March 16, 2023 2 / 37

 Equations (3) and (4) imply that the necessary conditions of the free
horizon problem are
a. Euler equation: Fx = dFx 0 /dt, t0 t t1 .
b. Legendre condition:

(max) Fx 0 x 0 0, t0 t t1

(min) Fx 0 x 0 0, t0 t t1 .

() March 16, 2023 3 / 37

c. Boundary conditions:
(i) x (t0 ) = x0 .
(ii) If x (t1 ) is …xed, then x (t1 ) = x1 is known.
(iii) If t1 is …xed, then t1 is known.
d. Transversality conditions:
(i) If x (t1 ) is free, then Fx 0 = 0 at t1 .
0
(ii) If t1 is free, then F x Fx 0 =0 at t1 .
(iii) If both x (t1 ) and t1 are free and satisfy R (t1 ) = x1 , then
0 0
F+ R x Fx 0 = 0 at t1 .

() March 16, 2023 4 / 37

Example 1:

Z t1 h 0 i2 1/2
min 1 + x (t ) dt (5)
x 0

subject to x (0) = 0, (6)

x (t1 ) = x1 , (7)
2
(t1 r) + x12 2
= r , (8)
Z t1 h i1/2
r2 (t r )2 x (t ) dt = A. (9)
0

() March 16, 2023 5 / 37

Salvage Value

A maximization problem:

Z t1
0
max F t, x (t ) , x (t ) dt + G (t1 , x1 ) (10)
x t0

subject to x (t0 ) = x0 . (11)

() March 16, 2023 6 / 37

 The problem implies

Z t1 +αδt1
0 0
g (a ) = F t, x (t ) + ah (t ) , x (t ) + ah (t ) dt
t0
+G (t1 + aδt1 , x1 + aδx1 ) . (12)

It suggests

Z t1
0 0
g (0) = Fx h + Fx 0 h dt + (F δt1 + Gt δt1 + Gx δx1 ) jt1 = 0. (13)
t0

() March 16, 2023 7 / 37

 0
Using h (t1 ) δx1 x (t1 ) δt1 , (14) yields

Fx = dFx 0 /dt (14)

0
F x Fx 0 + Gt δt1 + Fx 0 + Gx δx1 = 0 (15)

Thus, the necessary conditions are:

a. Euler equation: Fx = dFx 0 /dt, t0 t t1 .
b. Legendre condition:

(max) Fx 0 x 0 0, t0 t t1

(min) Fx 0 x 0 0, t0 t t1 .

() March 16, 2023 8 / 37

c. Boundary conditions:
(i) x (t0 ) = x0 .
(ii) If x (t1 ) is …xed, then x (t1 ) = x1 is known.
(iii) If t1 is …xed, then t1 is known.
(iv) If x (t1 ) and t1 must satisfy R (t1 ) = x1 , then this equation provides
a condition
d. Transversality conditions:
(i) If x (t1 ) is free, then Fx 0 + Gx = 0 at t1 .
0
(ii) If t1 is free, then F x Fx 0 + Gt = 0 at t1 .
(iii) If both x (t1 ) and t1 are free and satisfy R (t1 ) = x1 , then
0 0 0
F+ R x Fx 0 + Gx R + Gt = 0 at t1 .

() March 16, 2023 9 / 37

Example:

Z T
2
maxe rT
R e rt
z 0 (t ) dt (16)
x 0
subject to z (0) = 0, z (T ) = A. (17)

() March 16, 2023 10 / 37

Inequality Constraint Endpoints

Consider a maximization problem:

Z t1
0
max F t, x (t ) , x (t ) dt (18)
x t0

subject to x (t0 ) = x0 . (19)

Let J be the maximum value achieved in (19). Hence, we …nd

Z t1 +δt1 Z t1
0 0
J J = F t, x, x dt F t, x , x dt
t0 t0
Z t1 +δt1 Z t1
= F t, x, x 0 dt + F t, x, x 0 F t, x , x 0
dt
t1 t0
Z t1
= F (t1 ) δt1 + (x x ) Fx + x 0 x 0
Fx 0 dt, (20)
t0

() March 16, 2023 11 / 37

 Equation (21) implies:

Z t1
δJ = F (t1 ) δt1 + Fx 0 jt1 h (t1 ) + ( Fx dFx 0 /dt ) dt (21)
t0
0
Using h (t1 ) δx1 x (t1 ) δt1 , we obtain
Z t1
δJ = F x 0 Fx 0 jt1 δt1 + Fx 0 jt1 δx1 + ( Fx dFx 0 /dt ) dt (22)
t0

() March 16, 2023 12 / 37

Equation (23) yields Euler equation
0 0
Fx t, x (t ) , x (t ) = dFx 0 t, x (t ) , x (t ) /dt. (23)

We now consider inequality constant endpoints. When t1 T , we

get necessary conditions:

(T t1 ) F x 0 Fx 0 = 0 and F x 0 Fx 0 0. (24)

When x1 a, we have

(x1 a) Fx 0 (t1 ) = 0 and Fx 0 (t ) 0. (25)

() March 16, 2023 13 / 37

Example:

Z L
k
min (2g /A)k +1 y k +1 ( A y )k +1 W y0 dt (26)
0

subject to L L, y (0) = 0, y (L) = A. (27)

() March 16, 2023 14 / 37

Corners

A maximization problem:

Z t1
0
max F t, x (t ) , x (t ) dt (28)
x t0

subject to x (t0 ) = x0 , x (t1 ) = x1 (29)

Unlike before, here x (t ) is piecewise smooth. For instance, it is

continuously is continuously di¤erentiable on [t0 , t1 ] except at point
t2 .

() March 16, 2023 15 / 37

 Rewrite the maximization problem as

Z t2 Z t1
0 0
max F t, x (t ) , x (t ) dt + F t, x (t ) , x (t ) dt (30)
x t0 t2

subject to x (t0 ) = x0 , x (t2 ) = x2 , x (t1 ) = x1 , (31)

where both t2 and x (t2 ) are free.
It is obvious that Euler equation holds, i.e., Fx = dFx 0 /dt.

() March 16, 2023 16 / 37

 The Weierstrass-Eradmann Corner Conditions. In order to …nd the
conditions, we consider a problem

Z b
0
J= F t, x (t ) , x (t ) dt. (32)
a
Then, we have

δJ = F x 0 Fx 0 jb δb + Fx 0 jb δxb F x 0 Fx 0 ja δa Fx 0 ja δxa
Z b
+ ( Fx dFx 0 /dt ) hdt. (33)
a

Equation (34) suggests

h i
F x 0 Fx 0 j t F x 0 Fx 0 jt + δt2 + Fx 0 jt Fx 0 jt + δx2 = 0 (34)
2 2 2 2

() March 16, 2023 17 / 37

 Equation (36) implies the Weierstrass-Eradmann Corner Conditions:

Fx 0 j t = F x 0 j t + , (35)
2 2

and

F x 0 Fx 0 j t = F x 0 Fx 0 j t + (36)
2 2

() March 16, 2023 18 / 37

Example 1:

Z 3
2
min (x 2)2 x 0 1 dt (37)
0

subject to x (0) = 0, x (3) = 2. (38)

Example 2:

Z T
min c1 x 0 2 + c2 x dt (39)
0

subject to x (0) = 0, x (T ) = B. (40)

() March 16, 2023 19 / 37

Inequality Constraints in (t,x)

Consider optimizing

Z t1
F t, x, x 0 dt (41)
t0

subject to x (t0 ) = x0 , x (t1 ) = x1 ,

R (t ) x (t ) , t0 t t1 . (42)

() March 16, 2023 20 / 37

 Suppose that the solution of the Euler equation satis…es (43) over
[t0 , t2 ] and that (43) is tight over [t2 , t3 ].
Then, the optimal solution should satisfy

Z t2
F t, x, x 0 dt. (43)
t0

subject to x (t0 ) = x0 , x (t2 ) = x2 , R (t ) x (t ) .

and
Z t1
F t, x, x 0 dt. (44)
t2

subject to x (t2 ) = x2 , x (t1 ) = x1 , R (t ) x (t ) .

() March 16, 2023 21 / 37

 The change in (44) is

Fx 0 δx2 + F x 0 Fx 0 δt2 at t2 (45)

Using R 0 (t2 ) δt2 = δx2 , we get the change

F + Fx 0 R 0 x0 δt2 . (46)

The change in (45) is

Z t2 +δt2
F t, R (t ) , R 0 (t ) dt = F t, R (t ) , R 0 (t ) δt2 . (47)
t2

The total change is

F t, x, x 0 + Fx 0 R 0 x0 F t, R (t ) , R 0 (t ) jt2 δt2 . (48)

() March 16, 2023 22 / 37

 Equation (49) implies

F t, x2 , x 0 + Fx 0 t, x2 , x 0 R0 x0 F t, x2 , R 0 (t ) = 0 at t2 . (49)

Equation (50) says

R 0 (t2 ) = x 0 (t2 ) or Fx 0 x 0 (t, x (t2 ) , r ) . (50)

() March 16, 2023 23 / 37

 The necessary conditions are as follows
a. On every interval such that R (t ) > x (t ), the Euler equation must be
obeyed.
b. On every interval, the function x is determined by x (t ) = R (t ).
c. Let t 0 denotes a time of switching. Then we have R 0 t 0 = x 0 t 0
(provided Fx 0 x 0 6= 0) and R t 0 = x t 0 .

() March 16, 2023 24 / 37

Example:

Z T
min c1 x 0 2 + c2 ( x S ) dt (51)
0

subject to x (0) = x0 , x (t ) S (t ) , 0 t T. (52)

() March 16, 2023 25 / 37

In…nite Horizon Autonomous Problems

A problem of the form

Z ∞
max e rt
F x (t ) , x 0 (t ) dt (53)
0

subject to x (0) = x0 .

A problem is said to be autonomous if it does not depend on time

explicitly; t is not an argument.

() March 16, 2023 26 / 37

A steady state of stationary state is one in which x 0 = x" = 0.
The Euler equation is

Fx = rFx 0 + Fx 0 x 0 x 0 + Fx 0 x 0 x". (54)

The steady state is characterized by

Fx (xs , 0) + rFx 0 (xs , 0) = 0 (55)

subject to x (0) = x0 , lim x (t ) = xs . (56)

t !∞

() March 16, 2023 27 / 37

Example:
Z ∞
min e rt
x 2 + ax + bx 0 + cx 02 dt (57)
0

subject to x (0) = x0 .

() March 16, 2023 28 / 37

Most Rapid Approach Paths

Consider the in…nite horizon autonomous problem:

Z ∞
max e rt
M (x ) + N (x ) x 0 dt (58)
0

subject to x (0) = x0 , A (x ) x0 B (x ) . (59)

The Euler equation is

M 0 (x ) + N (x ) r = 0 (60)

The optimal solution to (59) to (60) is to move from x0 to xs as

quickly as possible and then remain at xs . This is called a most rapid
approach path (MRAP).

() March 16, 2023 29 / 37

 Rx
De…ne S (x ) = x0
N (y ) dy , then S 0 (x ) = N (x ). Therefore, (59) is
equivalent to
Z ∞
e rt
M (x ) + S 0 (x ) x 0 dt. (61)
0
It says
Z ∞
rt
e [M (x ) + rS (x )] dt. (62)
0
rt S
where we assume limt !∞ e (x (t )) = 0.
Equation (63) depends on x but not x 0 . Hence, one wants get to a
desirable state as soon as possible and stay there.

() March 16, 2023 30 / 37

Example 1:
Z ∞
min e rt
pf (K ) c K 0 + bK dt (63)
0

subject to K (0) = K0 .

Example 2:
Z ∞
min e rt
R (x ) x0 bx dt (64)
0

subject to x (0) = x0 > 0, bx x0 I bx.

() March 16, 2023 31 / 37

Diagrammatic Analysis

Example:
Z ∞
rt
max e [P (x ) C (u )] dt (65)
0
.
subject to x 0 = u bx, 0 < x0 < n. (66)

() March 16, 2023 32 / 37

Several Functions and Double Integrals

A maximization problem:

Z t1
0
max F t, x (t ) , y (t ) , x (t ) , y 0 (t ) dt (67)
x ,y t0
.
subject to x (t0 ) = x0 , y (t0 ) = y0 . (68)
where the terminal time and terminal values of the functions are not
necessarily …xed.

() March 16, 2023 33 / 37

 We have

Z t1
0
g (0) = Fx h + Fx 0 h0 + Fy k + Fy 0 k 0 dt + F jt1 δt1 = 0. (69)
t0

Rewrite this equation as

Z t1
[(Fx dFx 0 /dt ) h + (Fy dFy 0 /dt ) k ] dt + Fx 0 hjt1 + Fy 0 k jt1 + F jt1 δt1 =
t0
(70)
We know

h (t1 ) δx1 x 0 (t1 ) δt1 , (71)

k ( t1 ) δy1 y 0 (t1 ) δt1 (72)

() March 16, 2023 34 / 37

 By the previous analysis, we have

Z t1
[(Fx dFx 0 /dt ) h + (Fy dFy 0 /dt ) k ] dt + F x 0 Fx 0 y 0 Fy 0 jt1 δt1
t0
+Fx 0 jt1 δx1 + Fy 0 jt1 δy1 = 0 (73)

The necessary conditions are:

Fx = dFx 0 /dt, (74)

Fy = dFy 0 /dt, (75)

F x 0 Fx 0 y 0 Fy 0 jt1 = 0 if t1 is free, (76)

() March 16, 2023 35 / 37

 Fx 0 jt1 = 0 if x (t1 ) is free, (77)

Fy 0 jt1 = 0 if y (t1 ) is free. (78)

Example:
Find the extremal for
Z π/2
x 02 + y 02 + 2xy dt (79)
0

subject to x (0) = 0, x (π/2) = 1, y (0) = 0, y (π/2) = 1. (80)

() March 16, 2023 36 / 37

Assignments
Kamien and Schwartz

Exercise 10.3, 11.2, 11.3, 12.2, 13.1, 16.1, 17.3, 18.5

() March 16, 2023 37 / 37