Strong solutions of optimal control problems

J. Frédéric Bonnans
INRIA Saclay and CMAP, Ecole Polytechnique

SADCO Summer School

Imperial College, London, September 8, 2011

Weak solutions in the classical calculus of variations

Z T
Min `(t, yt, ẏt)dt; y0, yT given.
y 0

Weak solution: in a L∞ neighborhood for y, ẏ.

Tool 1: first order variation: Euler-Lagrange equation

Tool 2: analysis of second variation, conjugate points (Jacobi, 1837)

1
 Strong solutions in the classical calculus of variations
Strong solution: in a L∞ neighborhood for y only.
The Weierstrass necessary condition (1879) is that the excess function

E(y, u, v) := `(y, v) − `(y, u) − `u(y, u)(v − u), (1)

˙ ȳ) is a strong minimum, satisfies E(ȳ, ū, v) ≥ 0 for all v, or

if (ū = ȳ,
equivalently,
`(ȳ, ū) − `u(ȳ, ū)ū ≤ `(ȳ, v) − `u(ȳ, ū)v. (2)
In the language of optimal control `u(ȳ, ū) = −p̄ is the costate !
We recognize the Hamiltonian inequality (Pontryagin’s principle)

`(ȳ, ū) + pū ≤ `(ȳ, v) + pv.

2
 Orientation

• Expansion of solutions of controlled systems in the control space L1

• A decomposition principle of the criterium

• Back to the strong solutions

• Various extensions

3
 Sources

• Russian school : book by A.A. Milyutin an N.P. Osmolovskii (AMS,

1998).

• F.B. and N.P. Osmolovskii, J. Convex Analysis (2010).

• T. Bayen, F.B., F. Silva, (extensions to the elliptic framework, to appear).

4
Controlled system

ẏt = f (ut, yt), t ∈ (0, T ); y0 = y 0.

q ∈ [1, ∞]; Uq := Lq (0, T, Rm); Yq := W 1,q (0, T, Rn);

All function and their derivatives are C ∞ and bounded.

Unique solution y = y[u]; (y, u) ”trajectory” when y = y[u].

u 7→ y[u] Lipschitz Uq → Yq , C ∞ U∞ → Y∞.

5
 L1 expansion of trajectories

Nominal trajectory (ū, ȳ)

Pontryaguine linearization

żt = fy (ūt, ȳt)zt + f (ut, ȳt) − f (ūt, ȳt), t ∈ (0, T ).

Lemma 1. Let (u, y) be a trajectory. Then

ky − ȳ − zk∞ = O(ku − ūk21).

6
 Proof (dimension 1)
We recall that

żt = fy (ūt, ȳt)zt + f (ut, ȳt) − f (ūt, ȳt), t ∈ (0, T ).

Set w := ȳ + z − y. For a certain ŷt ∈ [ȳt, yt]

ẇt = fy (ūt, ȳt)zt + f (ut, ȳt) − f (ut, yt)

= fy (ūt, ȳt)zt + fy (ut, ŷt)(ȳt − yt) (3)
= fy (ūt, ȳt)wt + O ((|ut − ūt| + |ȳt − yt|)kȳ − yk∞) .

We conclude with Gronwall’s lemma, noting that

kȳ − yk∞ = O (ku − ūk1) .

7
 Second order expansion

Notation f¯(t) = f (ūt, ȳt). Second variation ζ solution of

ζ̇t = f¯y (t)ζt + 21 f¯yy (t)(zt)2 + [fy (ut, ȳt) − f¯y (t)]zt.

Lemma 2. Let (u, y) be a trajectory. Then

ky − ȳ − z − ζk∞ = O(ku − ūk31).

8
 Proof Set w := ȳ + z + ζ − y. Then

ẇt = f¯y (t)(zt + ζt) + f (ut, ȳt) − f (ut, yt)

+ 21 f¯yy (t)(zt)2 + [fy (ut, ȳt) − f¯y (t)]zt
= f¯y (t)wt + f¯y (t)(yt − ȳt)
+f (ut, ȳt) − f (ut, yt) + 21 f¯yy (t)(zt)2
+[fy (ut, ȳt) − f¯y (t)]zt
= f¯y (t)wt + fy (ut, ȳt))(yt − ȳt)
+f (ut, ȳt) − f (ut, yt) + 21 f¯yy (t)(zt)2
+(f¯y (t) − fy (ut, ȳt))(yt − ȳt − zt)
¯ 2


= fy (t)wt + O (|ut − ūt| + ku − ūk1)kȳ − yk∞
Conclusion with Gronwall’s lemma.

9
 Expansion of cost

Cost fonction, given φ : Rn → R, is

J(u) := φ(yT [u]).

Costate p̄ (p̄t is a row vector) solution of

−p̄˙t = p̄tfy (ūt, ȳt), t ∈ (0, T ); p̄T = φ0(ȳT ).

10
Expansion of cost

Let (u, y) be a trajectory, and set H[p](u, y) = pf (u, y):

RT
J(u) − J(ū) = 0
[H[p̄t](ut, ȳt) − H[p̄t](ūt, ȳt)]dt
1 T
+ 2 0 Hyy [p̄t](ūt, ȳt)(zt)2dt
R

RT
+ 0 [Hy [p̄t](ut, ȳt) − Hy [p̄t](ūt, ȳt)]ztdt
+ 12 φ00(ȳ0, ȳT )(z0, zT )2 + O(ku − ūk31)

11
 We rewrite the crossed term !

(dimension m = 1): For some ût ∈ [ūt, ut] :

[Hy [p̄t](ut, ȳt) − Hy [p̄t](ūt, ȳt)]zt

= Hu,y [p̄t](ût, ȳt)(ut − ūt, zt)
= Hu,y [p̄t](ūt, ȳt)(ut − ūt, zt) + ku − ūk1O(|ut − ūt|2)

and therefore this is equal to

Z T
Hu,y [p̄t](ūt, ȳt)(ut − ūt, zt)dt + O(ku − ūk32)
0

12
New expansion

RT
J(u) − J(ū) = 0
[H[p̄t](ut, ȳt) − H[p̄t](ūt, ȳt)]dt
1 T
+ 2 0 Hyy [p̄t](ūt, ȳt)(zt)2dt
R

RT
+ 0 [Hyu[p̄t](ūt, ȳt)(ut − ūt, zt)dt
+ 21 φ00(ȳ0, ȳT )(z0, zT )2 + O(ku − ūk32)

13
 Large deviation set
p
Let u ∈ U1. We set c(u) := ku − ūk1 et

B(u) := {t ∈ (0, T ); |ut − ūt| ≥ c(u)} .

Tchebycheff inequality:

T T
|ut − ūt|
Z Z
meas(B(u)) = 1{|ut−ūt|≥c(u)}dt ≤ dt = c(u).
0 0 c(u)

Set A(u) := [0, T ] \ B(u).

When u → ū dans L1, we have meas(B(u)) → 0 plus

“uniform convergence on A(u)”.

14
 Decomposition Principle:
Z
J(u) − J(ū0) = [H[p̄t](ut, ȳt) − H[p̄t](ūt, ȳt)]dt
B(u)

+ 12 Ω[p̄](uA − ū, z A) + o(ku − ūk22),

with uA = u on A(u), uA = ū on B(u), and

ż A = f 0(ū, ȳ)(uA − ū, z A), t ∈ (0, T ); z0A = 0,

Z T
Ω[p̄](v, z) = H 00[p̄t](ūt, ȳt)(vt, zt)dt + φ00(ȳ0, ȳT )(z0, zT )2.
0

15
 Necessary conditions

Expected classical consequences:

• Pontryaguin’s Principe

H[p̄t](ūt, ȳt) ≤ H[p̄t](ut, ȳt), for all u, p.p. t

• Nonnegativity of the tangent quadratic form

Ω[p̄](v, z) ≥ 0, for all v ∈ U2, z = z[v].

16
 Analysis of sufficient conditions I

Set y = y[u], δy = y − ȳ, δH = H[p̄](u, ȳ) − H[p̄](ū, ȳ), then

J(u) = J(ū) + φ0(ȳt)δy + O(|δyT |2)

But

0
RT
φ (ȳt)δy = [p̄δy]T0 = 0 p̄t(f (u, y) − f (ū, ȳ) − fy (ū, ȳ)δy)
RT RT
= 0 δH + 0 p̄t(f (u, y) − f (u, ȳ) − fy (u, ȳ)δy

and the second integral vanishes when y → ȳ uniformly, proving that:

17
 Analysis of sufficient conditions II
Lemma 3. Let ū be a strong solution and uk be bounded in L∞. If
lim inf k J(uk ) ≤ J(ū), and y k → ȳ uniformly, then
Z T
k
 
H[p̄t](ut , ȳt) − H[p̄t](ūt, ȳt) dt → 0
0

and hence for a subsequence

H[p̄t](ukt , ȳt) → H[p̄t](ūt, ȳt) p.p.

Under hypothesis of growth of Hamiltonian, uk → ū in L2.

18
 Analysis of sufficient conditions III

Hypothesis of growth of Hamiltonian: for some α > 0, a.a. t

H[p̄t](ūt, ȳt) + α|v − ūt|2 ≤ H[p̄t](v, ȳt), for all v

Second order sufficient condition: for some α > 0

Ω(v, z) ≥ αkvk21, for all v ∈ L2

Theorem 1. The above conditions imply the strong optimality.

”Bounded strong” variants: book by Milyutin and Osmolovskii.

19
 Constrained problems

T
 Z
 Min `(ut, yt)dt + φ(y0, yT );


0 (4)

 (i) ẏt = f (ut, yt); t ∈ (0, T );
(ii) Φ(y0, yT ) ∈ K,

with Φ = (Φ1, Φ2) and

K = {0}RnΦ1 × RnΦ2 .

End point Lagrangian and hamiltonian : α ≥ 0, Ψ ∈ RnΦ :

Φ[α, Ψ](y0, yT ) := αφ(y0, yT ) + ΨΦ(y0, yT ).

H[α, p](u, y) = α `(u, y) + pf (u, y).

20
 Optimality conditions

Costate equation

−p̄˙t = Hy [ᾱ, p̄t](ūt, ȳt)dt, on [0, T ],

p̄0− = −Φy0 [ᾱ, Ψ̄](ȳ0, ȳT )
p̄T + = ΦyT [ᾱ, Ψ̄](ȳ0, ȳT ),

Minimum principle

H[ᾱ, p̄t](ūt, ȳt) ≤ H[ᾱ, p̄t](ūt, ȳt), for all u.

Λ; set of (ᾱ, Ψ̄, p̄) (Pontryagin multipliers).

Problem assumed to be qualified (α = 1).

21
 Second order conditions I
Critical cone in Uq
 
 (v, z) ∈ Uq × Yq ; 
Cq := żt = f 0(ūt, ȳt)(vt, zt) sur (0, T ); z0 = 0. .
 0
Φ (ȳT )(zT ) ∈ TK (Φ(ȳT ))


Tangent quadratic form

Z T
Ω[ᾱ, Ψ̄, p̄](v, z) := H 00[ᾱ, p̄t](ūt, ȳt)(vt, zt)dt + Φ00[ᾱ, Ψ](ȳT )(zT )2.
0

Maximized quadratic form

Ω[Λ](v, z) := max Ω[ᾱ, Ψ̄, p̄](v, z)

(ᾱ,Ψ̄,p̄)∈Λ

22
Second order necessary conditions II

A necessary condition for a strong minimum is

Ω[Λ](v, z) ≥ 0, for all (v, z) ∈ C2.

Note that Λ includes only Pontryagin multipliers.

23
 Sufficient condition

Bounded strong minimum: if uk is bounded in L∞ and y[uk ] → ȳ

uniformly, then J(uk ) ≥ J(ū) for large enough k.
Theorem 2. A sufficient condition for a bounded strong minimum is
- quadratic growth of Hamiltonian (for at least one multiplier)
- uniform definiteness of Ω over the critical cone.

24
 Various extensions ?

• Mixed control and state constraints

• State constrained problems

• Control of elliptic equations

• Stochastic control

25
 Control of elliptic equations ?

Ω is here a smooth bounded open subset of Rn, the equation is

−∆y + ϕ(u, y) = 0 in Ω

with homogeneous Dirichlet condition y = 0 on ∂Ω; ϕy ≥ 0. Integral cost

Z
J(u) := `(u(x), y(x))dx.
Ω

Control constraint u(x) ∈ [a, b].

Strong solution ū, ȳ = y[ū].

26
 Control of elliptic equations I

Hamiltonian H[p](u, y) := `(u, y) − pϕ(u, y).

costate p̄ solution of
−∆p = Hy (ū, ȳ, p̄)
with homogeneous Dirichlet condition on ∂Ω.

Pontryagin’s principle for strong solutions

H(ū, ȳ, p̄) ≤ H(v, ȳ, p̄), for a.a. x ∈ Ω.

27
 Second order necessary optimality conditions

Z
Q(v) := H 00[p̄](ū, ȳ)(v, z)2 ≥ 0, for all v ∈ C(ū),
Ω
with z solution of linearized equation

−∆z + ϕ0(ū, ȳ)(v, z) = 0.

and C(ū) critical cone:

K = {v ∈ L2, a ≤ v(x) ≤ b a.e.}

C(ū) = {v ∈ TK(ū); J 0(ū)v ≤ 0}.

28
Challenge

Extend the decomposition principle to this setting

Extend the sufficient condition for strong solutions.

29
The End !

30