Alexander J. Zaslavski - Structure of Approximate Solutions of Optimal Control Problems-Springer (2013)

SPRINGER BRIEFS IN OPTIMIZATION
Alexander J. Zaslavski
Structure of
Approximate
Solutions of
Optimal Control
Problems
123
SpringerBriefs in Optimization
Series Editors
Panos M. Pardalos
János D. Pintér
Stephen M. Robinson
Tamás Terlaky
My T. Thai
SpringerBriefs in Optimization showcases algorithmic and theoretical tech-

niques, case studies, and applications within the broad-based field of optimization.
Manuscripts related to the ever-growing applications of optimization in applied
mathematics, engineering, medicine, economics, and other applied sciences are
encouraged.
For further volumes:

http://www.springer.com/series/8918
Structure of Approximate
Solutions of Optimal Control
Problems
123
Department of Mathematics
Technion – Israel Institute of Technology
Haifa, Israel
ISSN 2190-8354 ISSN 2191-575X (electronic)

ISBN 978-3-319-01239-1 ISBN 978-3-319-01240-7 (eBook)
DOI 10.1007/978-3-319-01240-7
Springer New York Heidelberg Dordrecht London
Library of Congress Control Number: 2013943267
Mathematics Subject Classification: 49J15, 90C31, 54E35, 54E50, 54E52
© Alexander J. Zaslavski 2013

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)

Preface
In this book we study the structure of approximate solutions of optimal control

problems considered on subintervals of a real line. We are interested in properties
of approximate solutions which are independent of the length of the interval, for all
sufficiently large intervals. The results in this book deal with the so-called turnpike
property of the optimal control problems. To have this property means, roughly
speaking, that the approximate solutions of the problems are determined mainly by
the integrand (objective function) and are essentially independent of the choice of
interval and endpoint conditions, except in regions close to the endpoints.
Turnpike properties are well known in mathematical economics. The term was
first coined by P. Samuelson in 1948 when he showed that an efficient expanding
economy would spend most of the time in the vicinity of a balanced equilibrium path
(also called von Neumann path). Now it is well known that the turnpike property is
a general phenomenon which holds for large classes of variational problems. For
these classes of problems, using the Baire category approach, it was shown that the
turnpike property holds for a generic (typical) problem.
In this book we generalize this result for a general class of optimal control
problems. More precisely, in Chap. 2 of this book we consider a class of opti-
mal control problems (with the same system of differential equations, the same
functional constraints, and the same boundary conditions) which is identified
with the corresponding complete metric space of objective functions (integrands).
The main results of Chap. 2 establish the turnpike property for any element of
a set which is a countable intersection of open everywhere dense sets in the
space of integrands. This means that the turnpike property holds for most optimal
control problems (integrands). In Chap. 3 we study infinite horizon optimal control
problems corresponding to the space of integrands introduced in Chap. 2. A class of
linear control problems is considered in Chap. 4.
Haifa, Israel Alexander J. Zaslavski
v
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.1 Infinite Horizon Variational Problems . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.2 The Turnpike Phenomenon . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5
1.3 Structure of Solutions of Variational Problems .. . .. . . . . . . . . . . . . . . . . . . . 7
2 Turnpike Properties of Optimal Control Problems . .. . . . . . . . . . . . . . . . . . . . 11
2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11
2.2 The Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13
2.3 Uniform Boundedness of Trajectory-Control Pairs . . . . . . . . . . . . . . . . . . . 17
2.4 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 22
2.5 Proofs of Theorems 2.1–2.3 and 2.5 . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 68
3 Infinite Horizon Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 77
3.1 Existence of Optimal Solutions . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 77
3.2 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 82
3.3 Proof of Proposition 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 85
3.5 Proof of Theorem 3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 91
3.6 Proof of Theorem 3.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 97
3.7 Proof of Theorem 3.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 100
4 Linear Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 105
4.1 The Class of Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 105
4.3 A Continuity Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 111
4.4 A Boundedness Property . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 116
4.5 The Existence and Structure of Solutions .. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 118
References .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 125
Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 127
vii
Chapter 1
Introduction
1.1 Infinite Horizon Variational Problems
The study of optimal control problems and variational problems defined on infinite
intervals and on sufficiently large intervals has been a rapidly growing area of
research [4, 5, 8–13, 16, 18, 22–24, 27, 34–38, 40, 45–47, 50]. These problems arise in
engineering [1, 25, 53], in models of economic growth [2, 3, 12, 15, 17, 24, 28, 33, 39,
41–43,50], in infinite discrete models of solid-state physics related to dislocations in
one-dimensional crystals [7, 44], in the calculus of variations on time scales [29, 32]
and in the theory of thermodynamical equilibrium for materials [14, 26, 30, 31].
Consider the infinite horizon problem of minimizing the expression
T
f (t, x(t), x (t))dt
0
as T grows to infinity where a function x : [0, ∞) → Rn is locally absolutely

continuous (a. c.) and satisfies the initial condition x(0) = x0 , and f belongs to a
complete metric space of functions to be described below.
We say that an a. c. function x : [0, ∞) → Rn is (f )-overtaking optimal if
T
lim sup [f (t, x(t), x (t)) − f (t, y(t), y (t))]dt ≤ 0
T →∞ 0
for any a. c. function y : [0, ∞) → Rn satisfying y(0) = x(0).

This notion, known as the overtaking optimality criterion, was introduced in
the economics literature [17, 43] and has been used in optimal control theory
[12, 24, 49, 50].
Another type of optimality criterion for infinite horizon problems was introduced
by Aubry and Le Daeron [7] in their study of the discrete Frenkel–Kontorova model
related to dislocations in one-dimensional crystals. This optimality criterion was
used in [14, 26, 30, 31, 44, 47, 50].
A.J. Zaslavski, Structure of Approximate Solutions of Optimal Control Problems, 1

SpringerBriefs in Optimization, DOI 10.1007/978-3-319-01240-7 1,
2 1 Introduction
Let I be either [0, ∞) or (−∞, ∞). We say that an a. c. function x : I → Rn is

an (f )-minimal solution if
T2 T2

f (t, x(t), x (t))dt ≤ f (t, y(t), y (t))dt ≤ 0
T1 T1
for each T1 ∈ I, each T2 > T1 , and each a. c. function y : [T1 , T2 ] → Rn which

satisfies y(Ti ) = x(Ti ), i = 1, 2.
It is easy to see that every (f )-overtaking optimal function is an (f )-minimal
solution.
In Chap. 1 of [50] and in [48] we considered a functional space of integrands M
described below. We showed that for each f ∈ M and each z ∈ Rn there exists a
bounded (f )-minimal solution Z : [0, ∞) → Rn satisfying Z(0) = z such that any
other a. c. function Y : [0, ∞) → Rn is not “better” than Z. We also established
that, given f ∈ M and a bounded set E ⊂ Rn , the C([0, T ]) norms of approximate
solutions x : [0, T ] → Rn for the minimization problem on an interval [0, T ] with
x(0), x(T ) ∈ E are bounded by some constant which depends only on f and E.
Let a > 0 be a constant and ψ : [0, ∞) → [0, ∞) be an increasing function such
that ψ(t) → ∞ as t → ∞.
Denote by | · | the Euclidean norm in the n-dimensional Euclidean space Rn and
denote by M the set of all continuous functions f : [0, ∞) × Rn × Rn → R1
which satisfy the following assumptions:
A(i) For each (t, x) ∈ [0, ∞)×Rn the function f (t, x, ·) : Rn → R1 is convex;
A(ii) The function f is bounded on [0, ∞)×E for any bounded set E ⊂ Rn ×Rn ;
A(iii) For each (t, x, u) ∈ [0, ∞) × Rn × Rn ,
f (t, x, u) ≥ max{ψ(|x|), ψ(|u|)|u|} − a;
A(iv) For each M, > 0 there exist Γ, δ > 0 such that
|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤ max{f (t, x1 , u1 ), f (t, x2 , u2 )}
for each t ∈ [0, ∞), each u1 , u2 ∈ Rn and each x1 , x2 ∈ Rn which satisfy
|xi | ≤ M, |ui | ≥ Γ, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ;
A(v) For each M, > 0 there exists δ > 0 such that
|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤
for each t ∈ [0, ∞), each u1 , u2 ∈ Rn and each x1 , x2 ∈ Rn which satisfy
|xi |, |ui | ≤ M, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ.

1.1 Infinite Horizon Variational Problems 3
Assumptions A(i)–A(v) were discussed in [50] with examples of integrands for

which these assumptions hold. Note that assumption A(iii) implies that the function
f grows to infinity as |x| → ∞ and grows superlinearly with respect to u while
assumption A(iv) means the uniform continuity of the function with respect to x
and u on bounded sets.
It is an elementary exercise to show that an integrand f = f (t, x, u) ∈
C 1 ([0, ∞) × Rn × Rn ) belongs to M if f satisfies assumptions A(i), A(iii),
sup{|f (t, 0, 0)| : t ∈ [0, ∞)} < ∞
and there exists an increasing function ψ0 : [0, ∞) → [0, ∞) such that
sup{|∂f /∂x(t, x, u)|, |∂f /∂u(t, x, u)|} ≤ ψ0 (|x|)(1 + ψ(|u|)|u|)
for each t ∈ [0, ∞), x, u ∈ Rn .

Therefore the space M contains many functions.
We equip the set M with the uniformity which is determined by the following
base:
E(N, , λ) = {(f, g) ∈ M × M : |f (t, x, u) − g(t, x, u)| ≤

for each t ∈ [0, ∞), each u ∈ Rn , and each x ∈ Rn satisfying |x|, |u| ≤ N }
∩{(f, g) ∈ M × M : (|f (t, x, u)| + 1)(|g(t, x, u)| + 1)−1 ∈ [λ−1 , λ]
for each t ∈ [0, ∞), each u ∈ Rn , and each x ∈ Rn satisfying |x| ≤ N }
where N > 0, > 0, λ > 1.

Clearly, the uniform space M is Hausdorff and has a countable base. Therefore
M is metrizable. We showed in Sect. 1.3 of [50] that the uniform space M is
complete.
Put
T2
f
I (T1 , T2 , x) = f (t, x(t), x (t))dt
T1
where f ∈ M, 0 ≤ T1 < T2 < ∞ and x : [T1 , T2 ] → Rn is an a. c. function.

For f ∈ M, a, b ∈ Rn and numbers T1 , T2 satisfying 0 ≤ T1 < T2 , put
U f (T1 , T2 , a, b) = inf{I f (T1 , T2 , x) : x : [T1 , T2 ] → Rn

is an a. c. function satisfying x(T1 ) = a, x(T2 ) = b},
σ f (T1 , T2 , a) = inf{U f (T1 , T2 , a, b) : b ∈ Rn }.
It is easy to see that −∞ < U f (T1 , T2 , a, b) < ∞ for each f ∈ M, each a, b ∈ Rn

and each pair of numbers T1 , T2 satisfying 0 ≤ T1 < T2 .
4 1 Introduction
Let f ∈ M. We say that an a. c. function x : [0, ∞) → Rn is an (f )-good

function if for any a. c. function y : [0, ∞) → Rn ,
inf{I f (0, T, y) − I f (0, T, x) : T ∈ (0, ∞)} > −∞.
In Chap. 1 of [50] and in [48] we studied the set of (f )-good functions and proved
the following results.
Theorem 1.1. For each h ∈ M and each z ∈ Rn there exists an (h)-good function
Z h : [0, ∞) → Rn satisfying Z h (0) = z such that:
1. For each f ∈ M, each z ∈ Rn and each a. c. function y : [0, ∞) → Rn one
of the following properties holds:
(i) I f (0, T, y) − I f (0, T, Z f ) → ∞ as T → ∞;
(ii) sup{|I f (0, T, y) − I f (0, T, Z f )| : T ∈ (0, ∞)} < ∞,
sup{|y(t)| : t ∈ [0, ∞)} < ∞.
2. For each f ∈ M and each number M > 0 there exist a neighborhood U of f

in M and a number Q > 0 such that
sup{|Z g (t)| : t ∈ [0, ∞)} ≤ Q
for each g ∈ U and each z ∈ Rn satisfying |z| ≤ M .

3. For each f ∈ M and each number M > 0 there exist a neighborhood U of f
in M and a number Q > 0 such that for each g ∈ U , each z ∈ Rn satisfying
|z| ≤ M , each T1 ≥ 0, T2 > T1 and each a. c. function y : [T1 , T2 ] → Rn
satisfying |y(T1 )| ≤ M the following relation holds:
I g (T1 , T2 , Z g ) ≤ I g (T1 , T2 , y) + Q.
4. For each f ∈ M and each z ∈ Rn the function Z f : [0, ∞) → Rn is an

(f )-minimal solution.
Corollary 1.2. Let f ∈ M, z ∈ Rn and let y : [0, ∞) → Rn be an a. c.
function. Then y is an (f )-good function if and only if condition (ii) of Assertion 1
of Theorem 1.1 holds.
Theorem 1.3. For each f ∈ M there exist a neighborhood U of f in M and a
number M > 0 such that for each g ∈ U and each (g)-good function x : [0, ∞) →
Rn ,
lim sup |x(t)| < M.

t→∞
Our next result, which was also proved in Chap. 1 of [50] and in [48], shows that
for every bounded set E ⊂ Rn the C([0, T ]) norms of approximate solutions x :
1.2 The Turnpike Phenomenon 5
[0, T ] → Rn for the minimization problem on an interval [0, T ] with x(0), x(T ) ∈
E, are bounded by some constant which does not depend on T .
Theorem 1.4. Let f ∈ M and M1 , M2 , c be positive numbers. Then there exist a
neighborhood U of f in M and a number S > 0 such that for each g ∈ U , each
T1 ∈ [0, ∞) and each T2 ∈ [T1 + c, ∞) the following properties hold:
(i) If x, y ∈ Rn satisfy |x|, |y| ≤ M1 and if an a. c. function v : [T1 , T2 ] → Rn
satisfies
v(T1 ) = x, v(T2 ) = y, I g (T1 , T2 , v) ≤ U g (T1 , T2 , x, y) + M2 ,
then
|v(t)| ≤ S, t ∈ [T1 , T2 ].
(ii) If x ∈ Rn satisfies |x| ≤ M1 and if an a. c. function v : [T1 , T2 ] → Rn satisfies
v(T1 ) = x, I g (T1 , T2 , v) ≤ σ g (T1 , T2 , x) + M2 ,
then
|v(t)| ≤ S, t ∈ [T1 , T2 ].
The results presented in this section are important ingredients in the proofs of
turnpike results for variational problems [47, 48, 50].
1.2 The Turnpike Phenomenon
In Chap. 2 of [50] and in [49] we studied the structure of approximate solutions of

the variational problems
T2
f (t, z(t), z (t))dt → min, z(T1 ) = x, z(T2 ) = y, (P)
T1
z : [T1 , T2 ] → Rn is an absolutely continuous function,
where T1 ≥ 0, T2 > T1 , x, y ∈ Rn , and f : [0, ∞) × Rn × Rn → R1 belongs to

the complete metric space of integrands M which was introduced in Sect. 1.1.
Let T1 ≥ 0, T2 > T1 , x, y ∈ Rn , f : [0, ∞) × Rn × Rn → R1 be an integrand
and let δ be a positive number. We say that an absolutely continuous (a. c.) function
u : [T1 , T2 ] → Rn satisfying u(T1 ) = x, u(T2 ) = y is a δ-approximate solution of
the problem (P) if
6 1 Introduction
T2 T2
f (t, u(t), u (t))dt ≤ f (t, z(t), z (t))dt + δ
T1 T1
for each a. c. function z : [T1 , T2 ] → Rn satisfying z(T1 ) = x, z(T2 ) = y.

In Chap. 2 of [50] and in [49] we deal with the so-called turnpike property of the
variational problems (P) associated with an integrand f . To have this property means
that there exists a bounded continuous function Xf : [0, ∞) → Rn depending
only on f such that for each pair of positive numbers K, > 0 there exist
positive constants L = L(K, ) and δ = δ(K, ) depending on , K such that if
u : [T1 , T2 ] → Rn is a δ-approximate solution of the problem (P) with
T2 − T1 ≥ L, |u(Ti )| ≤ K, i = 1, 2,
then
|u(t) − Xf (t)| ≤ for all t ∈ [T1 + τ1 , T2 − τ2 ],
where τ1 , τ2 ∈ [0, L].

If the integrand f possesses the turnpike property, then the solutions of varia-
tional problems with f are essentially independent of the choice of time interval
and values at the endpoints except in regions close to the endpoints of the time
interval. If a point t does not belong to these regions, then the value of a solution at t
is close to a trajectory Xf (“turnpike”) which is defined on the infinite time interval
and depends only on f . This phenomenon has the following interpretation. If one
wishes to reach a point A from a point B by a car in an optimal way, then one should
turn to a turnpike, spend most of the time on it and then leave the turnpike to reach
the required point.
Turnpike properties are well known in mathematical economics. The term was
first coined by Samuelson in 1948 (see [42]) who showed that an efficient expanding
economy would spend most of the time in the vicinity of a balanced equilibrium path
(also called von Neumann path). This property was further investigated for optimal
trajectories of models of economic dynamics (see, for example, [2,15,17,28,33,41,
43, 50]). Many turnpike results are collected in [50].
In the classical turnpike theory the function f does not depend on the variable t,
is strictly convex on the space Rn × Rn , and satisfies a growth condition common in
the literature. In this case, the turnpike property can be established, the turnpike Xf
is a constant function and its value is a unique solution of the minimization problem
f (x, 0) → min, x ∈ Rn .
It was shown in our research, which was summarized in [50], that the turnpike
property is a general phenomenon which holds for large classes of variational
problems without convexity assumptions. For these classes of problems a turnpike
is not necessarily a constant function (singleton) but may instead be an absolutely
continuous function on the interval [0, ∞) as it was described above [49, 50].
1.3 Structure of Solutions of Variational Problems 7
More precisely, in Chap. 2 of [50] we studied the turnpike properties for

variational problems with integrands which belong to the space M and showed that
the turnpike property holds for a generic integrand f ∈ M. Namely, we established
the existence of a set F ⊂ M which is a countable intersection of open everywhere
dense sets in M such that each f ∈ F has the turnpike property. This result is
presented and discussed in the next section.
1.3 Structure of Solutions of Variational Problems
Let a > 0 be a constant and let ψ : [0, ∞) → [0, ∞) be an increasing function

such that
ψ(t) → ∞ as t → ∞.
We use the notation and definitions introduced in the previous sections. We con-
sider the space of integrands M introduced in Sect. 1.1.
We equip the set M with two topologies where one is weaker than the other.
We refer to them as the weak and the strong topologies, respectively. For the set M
we consider the uniformity determined by the following base:
Es () = {(f, g) ∈ M × M : |f (t, x, u) − g(t, x, u)| ≤

for each t ∈ [0, ∞) and each x, u ∈ Rn },
where > 0. It is not difficult to see that the uniform space M with this uniformity
is metrizable and complete. This uniformity generates in M the strong topology.
We also equip the set M with the uniformity which is determined by the
following base:
for each t ∈ [0, ∞) and each x, u ∈ Rn satisfying |x|, |u| ≤ N,
(|f (t, x, u)| + 1)(|g(t, x, u)| + 1)−1 ∈ [λ−1 , λ]
for each t ∈ [0, ∞) and each x, u ∈ Rn satisfying |x| ≤ N },
where N > 0, > 0, λ > 1. This uniformity which was introduced in Sect. 1.1
generates in M the weak topology.
In Chap. 2 of [50] and in [49] we established the existence of a set F ⊂ M
which is a countable intersection of open (in the weak topology) everywhere
dense (in the strong topology) subsets of M such that the following theorems are
valid.
8 1 Introduction
Theorem 1.5.
1. For each g ∈ F and each pair of (g)-good functions vi : [0, ∞) → Rn , i = 1, 2,
|v2 (t) − v1 (t)| → 0 as t → ∞.
2. For each g ∈ F and each y ∈ Rn there exists a (g)-overtaking optimal function

Y : [0, ∞) → Rn satisfying Y (0) = y.
3. Let g ∈ F , > 0 and Y : [0, ∞) → Rn be a (g)-overtaking optimal function.
Then there exists a neighborhood U of g in M with the weak topology such that
the following property holds:
If h ∈ U and if v : [0, ∞) → Rn is an (h)-good function, then
|v(t) − Y (t)| ≤ for all large t.
Theorem 1.6. Let g ∈ F , M, > 0 and let Y : [0, ∞) → Rn be a (g)-overtaking

optimal function. Then there exists a neighborhood U of g in M with the weak
topology and a number τ > 0 such that for each h ∈ U and each (h)-overtaking
optimal function v : [0, ∞) → Rn satisfying |v(0)| ≤ M ,
|v(t) − Y (t)| ≤ for all t ∈ [τ, ∞).
Theorems 1.5 and 1.6 establish the existence of (g)-overtaking optimal functions
and describe the asymptotic behavior of (g)-good functions for g ∈ F .
Theorem 1.7. Let g ∈ F , S1 , S2 , > 0 and let Y : [0, ∞) → Rn be a (g)-
overtaking optimal function. Then there exists a neighborhood U of g in M with
the weak topology, a number L > 0, and an integer Q ≥ 1 such that if h ∈ U,
T1 ∈ [0, ∞), T2 ∈ [T1 + LQ, ∞) and if an a. c. function v : [T1 , T2 ] → Rn satisfies
one of the following relations:
(a) |v(Ti )| ≤ S1 , i = 1, 2, I h (T1 , T2 , v) ≤ U h (T1 , T2 , v(T1 ), v(T2 )) + S2 ;

(b) |v(T1 )| ≤ S1 , I h (T1 , T2 , v) ≤ σ h (T1 , T2 , v(T1 )) + S2 ,
then the following property holds:

There exist sequences of numbers {di }qi=1 , {bi }qi=1 ⊂ [T1 , T2 ] such that
q ≤ Q, bi < di ≤ bi + L, i = 1, . . . , q,
|v(t) − Y (t)| ≤ for each t ∈ [T1 , T2 ] \ ∪qi=1 [bi , di ].
Theorem 1.8. Let g ∈ F , S, > 0 and let Y : [0, ∞) → Rn be a (g)-overtaking

optimal function. Then there exist a neighborhood U of g in M with the weak
topology and numbers δ, L > 0 such that for each h ∈ U, each pair of numbers
T1 ∈ [0, ∞), T2 ∈ [T1 + 2L, ∞) and each a. c. function v : [T1 , T2 ] → Rn which
satisfies one of the following relations:
1.3 Structure of Solutions of Variational Problems 9
(a) |v(Ti )| ≤ S, i = 1, 2, I h (T1 , T2 , v) ≤ U h (T1 , T2 , v(T1 ), v(T2 )) + δ;

(b) |v(T1 )| ≤ S, I h (T1 , T2 , v) ≤ σ h (T1 , T2 , v(T1 )) + δ
the inequality |v(t) − Y (t)| ≤ is valid for all t ∈ [T1 + L, T2 − L].

Theorem 1.8 establishes the turnpike property for any g ∈ F .
According to the results presented in this section, the turnpike property is a
general phenomenon which holds for a large class of variational problems. For this
class of problems, using the Baire category approach, it was shown that the turnpike
property holds for a generic (typical) problem. Many results of this kind for other
classes of variational problems are collected in [50]. Note that the generic approach
of [50] is not limited to the turnpike property, but is also applicable to other problems
in the optimization theory and nonlinear analysis [6, 51].
In this book we use the Baire category approach and generalize Theorem 1.8 for
a general class of optimal control problems.
Our results are important for engineering and economic modeling. Optimal con-
trol problems studied in the book can be considered as a mathematical description of
the corresponding continuous time models of economic growth without convexity
assumptions which are usually present in the economic literature. Therefore the
results of this book essentially enlarge the class of models which posses the turnpike
property. In Chap. 4 of the book we apply our results to linear control systems which
are very important in engineering where these systems are usually considered with
quadratic cost functions. Here we again enlarge the class of linear control systems
with the turnpike property considering nonconvex and nonautonomous integrands.
Chapter 2
Turnpike Properties of Optimal
Control Problems
2.1 Preliminaries
Denote by | · | the Euclidean norm in the k-dimensional Euclidean space Rk . Let

m, n be natural numbers.
In this chapter we study a control system described by a differential equation
x (t) = G(t, x(t), u(t)) a. e. t ∈ I, (2.1)
where I is either R1 or [T1 , ∞) or [T1 , T2 ] (−∞ < T1 < T2 < ∞), and x : I → Rn
is an absolutely continuous (a. c.) function which satisfies
(t, x(t)) ∈ A for all t ∈ I, (2.2)
where A is a subset of Rn+1 . The control function u : I → Rm is Lebesgue

measurable and satisfies the feedback control constraints
u(t) ∈ U (t, x(t)) a. e. t ∈ I, (2.3)

m
where U : A → 2R is a point to set mapping with a graph
M = {(t, x, u) : (t, x) ∈ A, u ∈ U (t, x)}. (2.4)
We suppose that M is a Borel measurable subset of Rn+m+1 and that the function
G : M → Rn is borelian.
For any t ∈ R1 set
A(t) = {x ∈ Rn : (t, x) ∈ A}. (2.5)
We assume that the set A(t)

= ∅ for any t ∈ R1 .

12 2 Turnpike Properties of Optimal Control Problems
The performance of the above control system is described by an integral

functional
T2
I f (T1 , T2 , x, u) = f (t, x(t), u(t))dt, (2.6)
T1
where a borelian function f : M → R1 belongs to a complete metric space of

functions M defined below.
An a. c. function x : I → Rn , where I is either R1 or [T1 , ∞) or [T1 , T2 ]
(−∞ < T1 < T2 < ∞), will be called a trajectory if there exists a Lebesgue
measurable function (referred to as a control) u : I → Rm such that the pair
(x, u) satisfies (2.1)–(2.3) and the function t → f (t, x(t), u(t)) is locally Lebesgue
integrable on I.
For any s ∈ R1 set s+ = max{s, 0}.
Let a0 be a positive constant and let ψ : [0, ∞) → [0, ∞) be an increasing
function such that
ψ(t) → ∞ as t → ∞.
Denote by M the set of all borelian functions f : M → R1 which satisfy the

following growth assumption:
(A)
f (t, x, u) ≥ max{ψ(|x|), ψ(|u|),

ψ([|G(t, x, u)| − a0 |x|]+ )[|G(t, x, u)| − a0 |x|]+ } − a0
for each (t, x, u) ∈ M .

We equip the set M with the uniformity which is determined by the following base:

for each (t, x, u) ∈ M satisfying |x|, |u| ≤ N }
∩{(f, g) ∈ M × M : (|f (t, x, u)| + 1)(|g(t, x, u)| + 1)−1 ∈ [λ−1 , λ]
for each (t, x, u) ∈ M satisfying |x| ≤ N }, (2.7)
where N > 0, > 0 and λ > 1.

Clearly, the uniform space M is Hausdorff and has a countable base. Therefore
M is metrizable. It is not difficult to show that the uniform space M is complete.
We consider functionals of the form I f (T1 , T2 , x, u) [see (2.6)], where f ∈ M,
−∞ < T1 < T2 < ∞ and x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm is a
trajectory-control pair.
2.2 The Main Results 13
For f ∈ M, a pair of numbers T1 ∈ R1 , T2 > T1 and (T1 , y), (T2 , z) ∈ A set
U f (T1 , T2 , y, z) = inf{I f (T1 , T2 , x, u) : x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm

is a trajectory-control pair satisfying x(T1 ) = y, x(T2 ) = z}, (2.8)
f f
σ (T1 , T2 , y) = inf{U (T1 , T2 , y, h) : (T2 , h) ∈ A}. (2.9)
Here we assume that the infimum over empty set is ∞.
2.2 The Main Results
Denote by Mreg the set of all functions f ∈ M which satisfy the following
assumption:
(B) There exist a trajectory-control pair
xf : R1 → Rn , uf : R1 → Rm
and a number bf > 0 such that:

(i)
U f (T1 , T2 , xf (T1 ), xf (T2 )) = I f (T1 , T2 , xf , uf )
for each T1 ∈ R1 and each T2 > T1 ;

(ii)
sup{I f (j, j + 1, xf , uf ) : j = 0, ±1, ±2, . . . } < ∞;
(iii) For each S1 > 0 there exist S2 > 0 and an integer c > 0 such that
I f (T1 , T2 , xf , uf ) ≤ I f (T1 , T2 , x, u) + S2
for each T1 ∈ R1 , each T2 ≥ T1 + c, and each trajectory-control pair x :

[T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm which satisfies |x(T1 )|, |x(T2 )| ≤ S1 ;
(iv) For each > 0 there exists δ > 0 such that for each (T, z) ∈ A which satisfies
|z − xf (T )| ≤ δ
there are
τ1 ∈ (T, T + bf ] and τ2 ∈ [T − bf , T ),
and trajectory-control pairs
x1 : [T, τ1 ] → Rn , u1 : [T, τ1 ] → Rm ,
x2 : [τ2 , T ] → Rn , u2 : [τ2 , T ] → Rm
which satisfy
x1 (T ) = x2 (T ) = z,
xi (τi ) = xf (τi ), i = 1, 2,
|x1 (t) − xf (t)| ≤ for all t ∈ [T, τ1 ],
|x2 (t) − xf (t)| ≤ for all t ∈ [τ2 , T ],
I f (T, τ1 , x1 , u1 ) ≤ I f (T, τ1 , xf , uf ) + ,
I f (τ2 , T, x2 , u2 ) ≤ I f (τ2 , T, xf , uf ) + .
Note that assumption (B) means that the trajectory-control pair
xf : R1 → Rn , uf : R1 → Rm
is a solution of the corresponding infinite horizon optimal control problem associ-

ated with the integrand f , and that certain controllability properties hold near this
In this chapter we will establish the following result.
Theorem 2.1.
1. Let f ∈ Mreg and S0 > 0. Then there exists S > 0 such that for each pair of
real numbers T1 < T2 and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies |x(T1 )| ≤ S0 the following inequality holds:
I f (T1 , T2 , xf , uf ) ≤ I f (T1 , T2 , x, u) + S.
2. Let f ∈ Mreg . Then for each s ∈ R1 and each trajectory-control pair
x : [s, ∞) → Rn , u : [s, ∞) → Rm
one of the following relations holds:

(a)
I f (s, t, x, u) − I f (s, t, xf , uf ) → ∞ as t → ∞;
(b)
sup{|I f (s, t, xf , uf ) − I f (s, t, x, u)| : t ∈ (s, ∞)} < ∞.

2.2 The Main Results 15
Moreover, if the relation (b) holds, then
sup{|x(t)| : t ∈ [s, ∞)} < ∞.
For each f ∈ Mreg and each r > 0 we define a function fr ∈ M by
fr (t, x, u) = f (t, x, u) + r min{|x − xf (t)|, 1} for all (t, x, u) ∈ M.
It is easy to see that fr ∈ Mreg for each f ∈ Mreg and each r > 0.
Let A be a subset of Mreg such that fr ∈ A for each f ∈ A and each r ∈ (0, 1).
Denote by Ā the closure of A in the uniform space M and consider the topological
subspace Ā ⊂ M with the relative topology.
In this chapter we will establish the existence of a set F ⊂ Ā which is a countable
intersection of open everywhere dense sets in Ā and such the following theorems
hold.
Theorem 2.2. For each f ∈ F and each S > 0 there exist a neighborhood U of f
in M and positive numbers δ, Q such that the following assertions hold:
inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} < ∞
for each g ∈ U, each T1 ∈ R1 and each T2 > T1 ;

for each g ∈ U, each T1 ∈ R1 , each T2 ≥ T1 + 1 and each trajectory-control
pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S
and
I g (T1 , T2 , x, u) ≤ U g (T1 , T2 , x(T1 ), x(T2 )) + δ
the following inequality holds:
|x(t)| ≤ Q for all t ∈ [T1 , T2 ].
Theorem 2.2 establishes uniform boundedness of approximate solutions of

optimal control problems.
The next theorem is our first turnpike result.
Theorem 2.3. Let f ∈ F . Then there exists a bounded continuous function Xf :

R1 → Rn such that the following property holds.
For each S, > 0 there exist a neighborhood U of f in M and real numbers

Δ > 0, δ ∈ (0, ) such that for each g ∈ U, each T1 ∈ R1 , each T2 ≥ T1 + 2Δ and
each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S
and
I g (T1 , T2 , x, u) ≤ U g (T1 , T2 , x(T1 ), x(T2 )) + δ
|x(t) − Xf (t)| ≤ for all t ∈ [T1 + Δ, T2 − Δ].
Moreover, if |x(T1 ) − Xf (T1 )| ≤ δ, then
|x(t) − Xf (t)| ≤ for all t ∈ [T1 , T2 − Δ]
and if |x(T2 ) − Xf (T2 )| ≤ δ, then
|x(t) − Xf (t)| ≤ for all t ∈ [T1 + Δ, T2 ].
Corollary 2.4. Assume that f ∈ F , S, Δ are positive numbers and that
x : R1 → Rn , u : R1 → Rm
is a trajectory-control pair such that
I f (T1 , T2 , x, u) = U f (T1 , T2 , x(T1 ), x(T2 ))
for each T1 ∈ R1 and each T1 > T1 , and
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S
for each T1 ∈ R1 and each T2 > T1 + Δ. Then
x(t) = Xf (t) for all t ∈ R1 .
The next theorem is our second turnpike result.

2.3 Uniform Boundedness of Trajectory-Control Pairs 17
Theorem 2.5. Let f ∈ F , let a bounded continuous function Xf : R1 → Rn be

as guaranteed by Theorem 2.3. and let , M be a pair of positive numbers. Then
there exist a neighborhood U of f in M, real numbers l > 0, L > 0 and a natural
number p such that for each g ∈ U, each T1 ∈ R1 , each T2 ≥ T1 + L, and each
trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + M
there exist finite sequences
{ai }qi=1 , {bi }qi=1 ⊂ [T1 , T2 ],
where q ≤ p is a natural number, such that
ai ≤ bi ≤ ai + l for all integers i = 1, . . . , q
and
|x(t) − Xf (t)| ≤ for all t ∈ [T1 , T2 ] \ ∪qi=1 [ai , bi ].
This chapter is organized as follows. In Sect. 2.3 we study uniform boundedness

of trajectory-control pairs. Section 2.4 contains auxiliary results. Theorems 2.1–2.3
and 2.5 are proved in Sect. 2.5.
2.3 Uniform Boundedness of Trajectory-Control Pairs
Proposition 2.6. Let M0 , M1 , τ0 be positive numbers. Then there exists M2 > M1

such that for each f ∈ M, each T1 ∈ R1 , each T2 ∈ (T1 , T1 + τ0 ], and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
inf{|x(t)| : t ∈ [T1 , T2 ]} ≤ M1 ,
I f (T1 , T2 , x, u) ≤ M0 (2.10)
|x(t)| ≤ M2 for all t ∈ [T1 , T2 ]. (2.11)
Proof. Fix
δ ∈ (0, min{8−1 (1 + a−1

0 ), 16
−1
τ0 }) (2.12)
[recall a0 in assumption (A)]. By assumption (A) there exist h0 > M1 + 1 such that
f (t, x, u) ≥ 4(M0 + a0 τ0 )δ −1
for each (t, x, u) ∈ M satisfying |x| ≥ h0 (2.13)
and γ0 > 0 such that
f (t, x, u) ≥ 8[|G(t, x, u)| − a0 |x|]+ for each (t, x, u) ∈ M

satisfying |G(t, x, u)| − a0 |x| ≥ γ0 . (2.14)
Choose a number
M2 > 8(M0 + τ0 a0 + γ0 δ + h0 ) + 8M1 . (2.15)
Let f ∈ M, T1 ∈ R1 , T2 ∈ (T1 , T1 + τ0 ] and
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
be a trajectory-control pair satisfying (2.10). We show that (2.11) holds.

Assume the contrary. Then there exists t0 ∈ [T1 , T2 ] such that
|x(t0 )| > M2 . (2.16)
By the choice of h0 , (2.10), (2.12), (2.13), and assumption (A), there exists t1 ∈
[T1 , T2 ] such that
|x(t1 )| ≤ h0 and |t1 − t0 | ≤ δ. (2.17)
There exists
t2 ∈ [min{t0 , t1 }, max{t0 , t1 }]
such that
|x(t2 )| ≥ |x(t)| for all t ∈ [min{t0 , t1 }, max{t0 , t1 }]. (2.18)

We define
E1 = {t ∈ [min{t1 , t2 }, max{t1 , t2 }] :
|G(t, x(t), u(t))| ≥ a0 |x(t)| + γ0 },
E2 = [min{t1 , t2 }, max{t1 , t2 }] \ E1 . (2.19)
It follows from (2.1), (2.19), (2.17), (2.18), (2.14), (2.10), and assumption (A) that
t2
|x(t2 ) − x(t1 )| = | G(t, x(t), u(t))dt|
t1
t2
≤ a0 | |x(t)|dt| + (|G(t, x(t), u(t))| − a0 |x(t)|)+ dt
t1 E1

+ (|G(t, x(t), u(t))| − a0 |x(t)|)+ dt
E2

≤ a0 |x(t2 )|δ + γ0 δ + 8−1 f (t, x(t), u(t))dt
E1
≤ a0 |x(t2 )|δ + γ0 δ + 8−1 (M0 + τ0 a0 ).
Together with (2.12), (2.18), (2.16), and (2.17) this implies that
(7/8)M2 − h0 ≤ (7/8)|x(t2 )| − |x(t1 )| ≤ γ0 δ + 8−1 (M0 + τ0 a0 ).
This contradicts (2.15). The contradiction we have reached proves Proposition 2.6.

Proposition 2.6 and assumption (A) imply the following result.
Proposition 2.7. Let M1 > 0 and 0 < τ0 < τ1 . Then there exists M2 > 0 such
that for each f ∈ M, each T1 ∈ R1 , each T2 ∈ [T1 + τ0 , T1 + τ1 ] and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I f (T1 , T2 , x, u) ≤ M1 (2.20)
relation (2.11) holds.

Proposition 2.8. Let M1 , > 0 and 0 < τ0 < τ1 . Then there exists δ > 0 such
that for each f ∈ M, each T1 ∈ R1 , each T2 ∈ [T1 + τ0 , T1 + τ1 ], and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies (2.20) and each t1 , t2 ∈ [T1 , T2 ] satisfying |t1 −t2 | ≤ δ the inequality
|x(t1 ) − x(t2 )| ≤
holds.
Proof. Let a number M2 > 0 be as guaranteed in Proposition 2.7. By assump-
tion (A) there exists h0 > 0 such that
f (t, x, u) ≥ 4−1 (M1 + a0 τ1 + 8)(|G(t, x, u)| − a0 |x|)+

for each (t, x, u) ∈ M satisfying |G(t, x, u)| − a0 |x| ≥ h0 . (2.21)
Fix a number
δ ∈ (0, (4a0 M2 + 4h0 + 4)−1 ). (2.22)
Let f ∈ M, T1 ∈ R1 , T2 ∈ [T1 + τ0 , T1 + τ1 ],
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
be a trajectory-control pair satisfying (2.20) and let
t1 , t2 ∈ [T1 , T2 ], 0 < t2 − t1 ≤ δ. (2.23)
Set
E1 = {t ∈ [t1 , t2 ] : |G(t, x(t), u(t))| − a0 |x(t)| ≥ h0 },

E2 = [t1 , t2 ] \ E1 . (2.24)
It follows from the choice of M2 , (2.20), (2.11), (2.23), (2.24), (2.21), (2.22), and
(2.1) that
t2 t2
|x(t2 ) − x(t1 )| ≤ a0 |x(t)|dt + (|G(t, x(t), u(t))| − a0 |x(t)|)+ dt
t1 t1

≤ a0 δM2 + δh0 + (|G(t, x(t), u(t))| − a0 |x(t)|)dt
E1

≤ a0 δM2 + δh0 + (4(M1 + a0 τ1 + 8))−1 f (t, x(t), u(t))dt
E1
≤ a0 δM2 + δh0 + 4−1 ≤ .
Proposition 2.8 is proved.

Proposition 2.9. Let f ∈ M, 0 < c1 < c2 , and D, > 0. Then there exists
a neighborhood V of f in M such that for each g ∈ V , each T1 ∈ R1 , each
T2 ∈ [T1 + c1 , T1 + c2 ], and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
min{I f (T1 , T2 , x, u), I g (T1 , T2 , x, u)} ≤ D
the inequality
|I f (T1 , T2 , x, u) − I g (T1 , T2 , x, u)| ≤
holds.
Proof. By Proposition 2.7 there exists S > 0 such that for each g ∈ M, each
T1 ∈ R1 , each T2 ∈ [T1 + c1 , T1 + c2 ], and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ D + 1
|x(t)| ≤ S for all t ∈ [T1 , T2 ]. (2.25)
There exist δ ∈ (0, 1), N > S and Γ > 1 such that
ψ(N ) ≥ 4a0 + 4, δc2 ≤ 8−1 ,

(Γ − 1)(D + a0 c2 + c2 ) ≤ 8−1 . (2.26)
Define
V = {g ∈ M : (f, g) ∈ E(N, δ, Γ )}. (2.27)
Assume that g ∈ V , each T1 ∈ R1 , each T2 ∈ [T1 + c1 , T1 + c2 ] and
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
is a trajectory-control pair which satisfies
min{I f (T1 , T2 , x, u), I g (T1 , T2 , x, u)} ≤ D. (2.28)

Set
E1 = {t ∈ [T1 , T2 ] : |u(t)| ≤ N }, E2 = [T1 , T2 ] \ E1 . (2.29)
It follows from (2.28), (2.29), (2.27), and the definition of S that (2.25) holds
and that
|f (t, x(t), u(t)) − g(t, x(t), u(t))| ≤ δ for all t ∈ E1 . (2.30)
Define
h(t) = min{f (t, x(t), u(t)), g(t, x(t), u(t))}
for all t ∈ [T1 , T2 ]. By (2.25), (2.27), (2.29), (2.26), and assumption (A), for each
t ∈ E2 ,
(f (t, x(t), u(t)) + 1)(g(t, x(t), u(t)) + 1)−1 ∈ [Γ −1 , Γ ]
and
|f (t, x(t), u(t)) − g(t, x(t), u(t))| ≤ (Γ − 1)(h(t) + 1).
It follows from (2.29), (2.30), assumption (A), (2.28), (2.26) and the inequality
above that

|I f (T1 , T2 , x, u) − I g (T1 , T2 , x, u)| ≤ δc2 + (Γ − 1) h(t)dt + (Γ − 1)c2
E2
≤ δc2 + (Γ − 1)c2 + (Γ − 1)(D + a0 c2 ) ≤ .
Proposition 2.9 is proved.

2.4 Auxiliary Results
Consider the control system described by (2.1)–(2.5). Suppose that f ∈ M and that
there exists a trajectory-control pair
x∗ : R1 → Rn , u∗ : R1 → Rm
such that:
(i)
U f (T1 , T2 , x∗ (T1 ), x∗ (T2 )) = I f (T1 , T2 , x∗ , u∗ )

2.4 Auxiliary Results 23
for each T1 ∈ R1 and each T2 > T1 ;

(ii)
sup{I f (j, j + 1, x∗ , u∗ ) : j = 0, ±1, ±2, . . . } < ∞;
(iii) For each S1 > 0 there exist S2 > 0 and an integer c > 0 such that
I f (T1 , T2 , x∗ , u∗ ) ≤ I f (T1 , T2 , x, u) + S2
for each T1 ∈ R1 , each T2 ≥ T1 + c and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies |x(T1 )|, |x(T2 )| ≤ S1 ;

(iv) There exists b∗ > 0 such that for each > 0 there exists δ > 0 such that for
each (T, z) ∈ A which satisfies
|z − x∗ (T )| ≤ δ
there are
τ1 ∈ (T, T + b∗ ] and τ2 ∈ [T − b∗ , T ),
x1 : [T, τ1 ] → Rn , u1 : [T, τ1 ] → Rm ,
x2 : [τ2 , T ] → Rn , u2 : [τ2 , T ] → Rm
which satisfy
x1 (T ) = x2 (T ) = z,
xi (τi ) = x∗ (τi ), i = 1, 2,
|x1 (t) − x∗ (t)| ≤ for all t ∈ [T, τ1 ],
|x2 (t) − x∗ (t)| ≤ for all t ∈ [τ2 , T ],
I f (T, τ1 , x1 , u1 ) ≤ I f (T, τ1 , x∗ , u∗ ) + ,
I f (τ2 , T, x2 , u2 ) ≤ I f (τ2 , T, x∗ , u∗ ) + .
It follows from property (ii) and Proposition 2.7 that
sup{|x∗ (t)| : t ∈ R1 } < ∞. (2.31)

Note that properties (i)–(v) mean that the trajectory-control pair
x∗ : R1 → Rn , u∗ : R1 → Rm
is a solution of the corresponding infinite horizon optimal control problem associ-

ated with the integrand f and that certain controllability properties hold near this
Lemma 2.10. Let S0 > 0. Then there exists S > 0 and an integer c ≥ 1 such that
for each T1 ∈ R1 , each T2 ≥ T1 + c and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm ,
satisfying |x(T1 )| ≤ S0 the inequality
I f (T1 , T2 , x∗ , u∗ ) ≤ I f (T1 , T2 , x, u) + S (2.32)

holds.
Proof. Fix a number S2 which satisfies
S2 > S0 ,
ψ(S2 ) > a0 + sup{|I f (j, j + 1, x∗ , u∗ )| : j = 0, ±1, ±2, . . . }. (2.33)
By property (iii) there exists a number S1 > 0 and an integer c > 0 such that for
each T1 ∈ R1 , each T2 ≥ T1 + c, and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies |x(T1 )|, |x(T2 )| ≤ S2 the inequality
I f (T1 , T2 , x∗ , u∗ ) ≤ I f (T1 , T2 , x, u) + S1
holds.
Fix a number
S > S1 + 2 + 2a0 (2 + c)
+(c + 4) sup{|I f (j, j + 1, x∗ , u∗ )| : j = 0, ±1, ±2, . . . }. (2.34)
Let T1 ∈ R1 , T2 ≥ T1 + c and
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
be a trajectory-control pair satisfying |x(T1 )| ≤ S0 . We show that (2.32) holds. By

the choice of S, S1 , and c, we may assume that
|x(T2 )| > S2 . (2.35)

Set
T3 = sup{t ∈ [T1 , T2 ] : |x(t)| ≤ S2 }. (2.36)
It follows from the choice of S1 , (2.36), (2.33), and assumption (A) that
I f (T1 , T3 , x∗ , u∗ ) − I f (T1 , T3 , x, u)
≤ S1 + 2a0 (1 + c) + (c + 2) sup{|I f (j, j + 1, x∗ , u∗ )| : j = 0, ±1, ±2, . . . }.
(2.37)
By assumption (A), (2.33) and (2.36),
I f (T3 , T2 , x∗ , u∗ ) − I f (T3 , T2 , x, u)
≤ (T2 − T3 + 2) sup{|I f (j, j + 1, x∗ , u∗ )| : j = 0, ±1, ±2, . . . }
+2a0 − (ψ(S2 ) − a0 )(T2 − T3 )
≤ 2 sup{|I f (j, j + 1, x∗ , u∗ )| : j = 0, ±1, ±2, . . . }.
Together with (2.34) and (2.37) this implies (2.32). Lemma 2.10 is proved.

1
Fix a number d0 > 0 and define a continuous function φ : A → R by
φ(t, x) = min{|x − x∗ (t)|, d0 } for each (t, x) ∈ A. (2.38)
For any r > 0 we define fr ∈ M by
fr (t, x, u) = f (t, x, u) + rφ(t, x) for each (t, x, u) ∈ M. (2.39)
We have the following simple auxiliary result.

Lemma 2.11. Let V be a neighborhood of f in M. Then there exists r0 > 0 such
that fr ∈ V for every r ∈ (0, r0 ).
Fix r ∈ (0, 1] and set
Λ0 = sup{|I f (j, j + 1, x∗ , u∗ )| : j = 0, ±1, ±2, . . . },

Λ1 = sup{|x∗ (t)| : t ∈ R1 }. (2.40)
Lemma 2.12. Let b1 ≥ 1 be an integer and let b2 , Q, S > 0. Then there exists
S1 > S such that
|x(t)| ≤ S1 for all t ∈ [T1 , T2 ] (2.41)

for each T1 ∈ R1 , each T2 ≥ T1 + 2b1 , and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies the following conditions:

(a)
|x(T1 )|, |x(T2 )| ≤ S
and
I fr (T1 , T2 , x, u) ≤ U fr (T1 , T2 , x(T1 ), x(T2 )) + Q;
(b) There exist
τ1 ∈ (T1 , T1 + b1 ] and τ2 ∈ [T2 − b1 , T2 )
such that
U fr (T1 , τ1 , x(T1 ), x∗ (τ1 )) ≤ b2 ,

U fr (τ2 , T2 , x∗ (τ2 ), x(T2 )) ≤ b2 . (2.42)
Proof. Choose a number S0 such that
S0 > S + 1,
f
ψ(S0 ) > a0 + sup{|I (j, j + 1, x∗ , u∗ )| : j = 0, ±1, ±2, . . . } + 4. (2.43)
By property (iii), there exist Q1 > 0 and an integer c1 ≥ 1 such that for each
T1 ∈ R1 , each T2 ≥ T1 + c1 , and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti )| ≤ S0 + 1, i = 1, 2
I f (T1 , T2 , x∗ , u∗ ) ≤ I f (T1 , T2 , x, u) + Q1 . (2.44)
Fix a number
S̃ > 4 + Q + 2Q1 + 2a0 (b1 + c1 + 3) + 2b2 + 2
+ Λ0 [2(c1 + 3) + 2Λ0 (c1 + 3) + 2a0 (b1 + c1 + 3) + 2b2 + 2 + Q + 2Q1 ]. (2.45)

By Proposition 2.6 there exists S1 > S0 +2 such that for each g ∈ M, each T1 ∈ R1 ,
each
T2 ∈ (T1 , T1 + Q + 2b2 + 2 + 2a0 (b1 + c1 + 3) + 2Q1 + 2(c1 + 3)Λ0 ]
and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
inf{|x(t)| : t ∈ [T1 , T2 ]} ≤ S0 ,
I g (T1 , T2 , x, u) ≤ S̃ (2.46)
|x(t)| ≤ S1 for all t ∈ [T1 , T2 ]. (2.47)
Assume that T1 ∈ R1 , T2 ≥ T1 + 2b1 and that a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies conditions (a) and (b). Let τ1 and τ2 be as guaranteed in condition (b). We
show that (2.41) holds.
Assume the contrary. Then there exists T0 ∈ [T1 , T2 ] such that
|x(T0 )| > S1 . (2.48)
By condition (b) there exists a trajectory-control pair
x1 : [T1 , T2 ] → Rn , u1 : [T1 , T2 ] → Rm
such that
x1 (Tj ) = x(Tj ), x1 (τj ) = x∗ (τj ), j = 1, 2,

x1 (t) = x∗ (t) and u1 (t) = u∗ (t) for all t ∈ [τ1 , τ2 ],
I fr (T1 , τ1 , x1 , u1 ) ≤ b2 + 1,
I fr (τ2 , T2 , x1 , u1 ) ≤ b2 + 1. (2.49)
It follows from (2.49) conditions (a) and (b), and assumption (A) that
I fr (T1 , T2 , x, u) ≤ I fr (T1 , T2 , x1 , u1 ) + Q
≤ Q + I fr (T1 , T2 , x∗ , u∗ ) + 2a0 b1 + 2b2 + 2. (2.50)
Set
t1 = sup{t ∈ [T1 , T0 ] : |x(t)| ≤ S0 },

t2 = inf{t ∈ [T0 , T2 ] : |x(t)| ≤ S0 }. (2.51)
It follows from the choice of Q1 and c1 , (2.44), (2.51), (2.40), condition (a), and
assumption (A) that
I fr (T1 , t1 , x∗ , u∗ ) − I fr (T1 , t1 , x, u)
≤ Q1 + a0 (c1 + 2) + (c1 + 2)Λ0 ,
I fr (t2 , T2 , x∗ , u∗ ) − I fr (t2 , T2 , x, u)
≤ Q1 + a0 (c1 + 2) + (c1 + 2)Λ0 . (2.52)
By (2.39), (2.40), (2.51), (2.43), and assumption (A),
I fr (t1 , t2 , x∗ , u∗ ) − I fr (t1 , t2 , x, u)
≤ (t2 − t1 + 2)Λ0 + 2a0 − (t2 − t1 )(ψ(S0 ) − a0 ) ≤ 2Λ0 + 2a0 − 4(t2 − t1 ).
Together with (2.50) and (2.52) this implies that
t2 − t1
≤ 2Λ0 + 2a0 + Q + 2a0 b1 + 2b2 + 2 + 2Q1 + 2a0 (c1 + 2) + 2(c1 + 2)Λ0 .
(2.53)
It follows from the choice of S1 , (2.53), (2.46), (2.47), (2.48), and (2.51) that
I fr (t1 , t2 , x, u) > S̃.
Together with (2.39), (2.50), (2.52), (2.40), and assumption (A) this implies that
−Q − 2a0 b1 − 2b2 − 2
≤ I fr (T1 , T2 , x∗ , u∗ ) − I fr (T1 , T2 , x, u)
≤ 2Q1 + 2a0 (c1 + 2) + 2(c1 + 2)Λ0
+I fr (t1 , t2 , x∗ , u∗ ) − I fr (t1 , t2 , x, u)
≤ 2Q1 + 2a0 (c1 + 2) + 2(c1 + 2)Λ0 + Λ0 (t2 − t1 + 2) + 2a0 − S̃.
By this relation and (2.53),
S̃ ≤ Q + 2Q1 + 2a0 (b1 + c1 + 3) + 2(b2 + 1)

+Λ0 [2(c1 + 3) + 2Λ0 (c1 + 3) + 2a0 (b1 + c1 + 3) + 2b2 + 2 + Q + 2Q1 ].
This contradicts (2.45). The contradiction we have reached proves Lemma 2.12.

Lemma 2.13. Let b1 ≥ 1 be an integer and let b2 , Q, S > 0. Then there exists
S0 > 0 such that for each T1 ∈ R1 , each T2 ≥ T1 + 2b1 , and each trajectory-
control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies conditions (a) and (b) of Lemma 2.12 the following inequality holds:
I fr (T, T + 1, x, u) ≤ S0 for any T ∈ [T1 , T2 − 1]. (2.54)
Proof. Let S1 > S be as guaranteed in Lemma 2.12. By property (iii), there exist
Q1 > 0 and an integer c1 ≥ 1 such that for each T1 ∈ R1 , each T2 ≥ T1 + c1 , and
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti )| ≤ S1 + 1, i = 1, 2
I f (T1 , T2 , x∗ , u∗ ) ≤ I f (T1 , T2 , x, u) + Q1 . (2.55)
Fix a number
S0 > 4 + Q + 2Q1 + 2(b2 + 1) + 2a0 (b1 + c1 + 4) + 2(c1 + 4)Λ0 (2.56)
[recall a0 in assumption (A)].

Assume that T1 ∈ R1 , T2 ≥ T1 + 2b1 and that a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies conditions (a) and (b) of Lemma 2.12.

Let τ1 and τ2 be as guaranteed in conditions (a) and (b) of Lemma 2.12. We show
that (2.54) holds for any T ∈ [T1 , T2 − 1].
Assume the contrary. Then there exists T ∈ [T1 , T2 − 1] such that
I fr (T, T + 1, x, u) > S0 . (2.57)

By condition (b) of Lemma 2.12, there exists a trajectory-control pair
x1 : [T1 , T2 ] → Rn , u1 : [T1 , T2 ] → Rm
which satisfies (2.49). By (2.49), conditions (a) and (b) of Lemma 2.12 and
assumption (A), relation (2.50) is true. It follows from the choice of S1 and
Lemma 2.12 that
|x(t)| ≤ S1 for all t ∈ [T1 , T2 ] (2.58)
In view of the choice of Q1 , c1 , (2.58), (2.40), and (2.39),
I fr (T1 , T, x∗ , u∗ ) − I fr (T1 , T, x, u) ≤ Q1 + a0 (c1 + 2) + (c1 + 2)Λ0 ,

I fr (T + 1, T2 , x∗ , u∗ ) − I fr (T + 1, T2 , x, u) ≤ Q1 + a0 (c1 + 2) + (c1 + 2)Λ0 .
(2.59)
It follows from (2.50), (2.59), (2.57), (2.40), and assumption (A) that
−Q − 2a0 b1 − 2b2 − 2
≤ I fr (T1 , T2 , x∗ , u∗ ) − I fr (T1 , T2 , x, u)
≤ 2Q1 + 2a0 (c1 + 2) + 2(c1 + 2)Λ0
+I fr (T, T + 1, x∗ , u∗ ) − I fr (T, T + 1, x, u)
≤ 2Q1 + 2a0 (c1 + 4) + 2(c1 + 4)Λ0 − S0 .
This contradicts (2.56). The contradiction we have reached proves Lemma 2.13.

Lemma 2.14. Let Δ > 0. Then there exists δ ∈ (0, Δ) such that for each T1 ∈ R1 ,
each T2 > T1 , and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ, i = 1, 2 (2.60)
I f (T1 , T2 , x, u) ≥ I f (T1 , T2 , x∗ , u∗ ) − Δ. (2.61)
Proof. There exists δ ∈ (0, Δ) such that property (iv) holds with = 3−1 Δ (see
the definition of f ).
Assume that T1 ∈ R1 , T2 > T1 and that a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.60). By (2.60), the choice of δ and property (iv), there exist
τ1 ∈ [T1 − b∗ , T1 ) and τ2 ∈ (T2 , T2 + b∗ ]
x1 : [τ1 , T1 ] → Rn , u1 : [τ1 , T1 ] → Rm
and
x2 : [T2 , τ2 ] → Rn , u2 : [T2 , τ2 ] → Rm
such that
xi (Ti ) = x(Ti ), i = 1, 2,
xi (τi ) = x∗ (τi ), i = 1, 2,
I f (τ1 , T1 , x1 , u1 ) ≤ I f (τ1 , T1 , x∗ , u∗ ) + 3−1 Δ,
I f (T2 , τ2 , x2 , u2 ) ≤ I f (T2 , τ2 , x∗ , u∗ ) + 3−1 Δ. (2.62)
It follows from the choice of (x1 , u1 ), (x2 , u2 ) and property (i) that
I f (τ1 , T1 , x1 , u1 ) + I f (T1 , T2 , x, u) + I f (T2 , τ2 , , x2 , u2 )

≥ I f (τ1 , τ2 , x∗ , u∗ ).
Together with (2.62) this implies (2.61). Lemma 2.14 is proved.

Lemma 2.15. Let ∈ (0, d0 ). Then there exist δ ∈ (0, ) and Δ > 0 such that for
each T1 ∈ R1 , each T2 ≥ T1 + Δ, and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ, i = 1, 2,
I (T1 , T2 , x, u) ≤ U fr (T1 , T2 , x(T1 ), x(T2 )) + δ
fr
(2.63)
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 , T2 ]. (2.64)

Proof. There exists δ0 ∈ (0, 8−1 ) such that property (iv) holds with = 1 and
δ = δ0 . There exists an integer b1 such that
b∗ + 1 < b1 ≤ b∗ + 2 (2.65)
[recall b∗ in the definition of f , property (iv)].

Set
Q = 1,
S = sup{|x∗ (t)| : t ∈ R1 } + 4,
b2 = 2 + b∗ + 2a0 + (b∗ + 3)Λ0 . (2.66)
Let S0 be as guaranteed in Lemma 2.13. By Proposition 2.8 there exists δ1 ∈

(0, 8−1 ) such that for each g ∈ M, each T ∈ R1 , each trajectory-control pair
x : [T, T + 1] → Rn , u : [T, T + 1] → Rm
which satisfies
I g (T, T + 1, x, u) ≤ 1 + S0 + 2Λ0 + 2a0 (2.67)
and each t1 , t2 ∈ [T, T + 1] satisfying |t1 − t2 | ≤ δ1 the inequality
|x(t1 ) − x(t2 )| ≤ 16−1
holds.
Choose a number
1 ∈ (0, min{8−1 , 8−1 , 12−1rδ1 (4 + 4b∗ )−1 }). (2.68)
There exists
δ2 ∈ (0, 4−1 1 ) (2.69)
such that property (iv) (see the definition of f ) holds with = 1 , δ = δ2 . There
exists
δ ∈ (0, δ2 ) (2.70)
such that Lemma 2.14 holds with Δ = 16−1 δ1 r. Fix a number
Δ > 2b∗ + 2b1 + 4. (2.71)

Assume that T1 ∈ R1 , T2 ≥ T1 + Δ and that a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.63). We show that (2.64) holds.

Assume the contrary. Then there exists a number t1 such that
t1 ∈ [T1 , T2 ],
|x(t1 ) − x∗ (t1 )| > . (2.72)
By (2.63), property (iv), and the choice of δ2 there exist
τ1 ∈ (T1 , T1 + b∗ ] and τ2 ∈ [T2 − b∗ , T2 ) (2.73)
x1 : [T1 , τ1 ] → Rn , u1 : [T1 , τ1 ] → Rm ,
x2 : [τ2 , T2 ] → Rn , u2 : [τ2 , T2 ] → Rm
which satisfy
xi (Ti ) = x(Ti ), i = 1, 2,
xi (τi ) = x∗ (τi ), i = 1, 2,
|x1 (t) − x∗ (t)| ≤ 1 for all t ∈ [T1 , τ1 ],
|x2 (t) − x∗ (t)| ≤ 1 for all t ∈ [τ2 , T2 ],
I f (T1 , τ1 , x1 , u1 ) ≤ I f (T1 , τ1 , x∗ , u∗ ) + 1 ,
I f (τ2 , T2 , x2 , u2 ) ≤ I f (τ2 , T2 , x∗ , u∗ ) + 1 . (2.74)
Define a trajectory-control pair
x3 : [T1 , T2 ] → Rn , u3 : [T1 , T2 ] → Rm
by
x3 (t) = x1 (t), u3 (t) = u1 (t) for all t ∈ [T1 , τ1 ],

x3 (t) = x∗ (t), u3 (t) = u∗ (t) for all t ∈ (τ1 , τ2 ],
x3 (t) = x2 (t), u3 (t) = u2 (t) for all t ∈ (τ2 , T2 ].
The relations above, (2.63) and (2.74) imply that
I fr (T1 , T2 , x, u) ≤ I fr (T1 , T2 , x3 , u3 ) + δ. (2.75)
It follows from (2.75), (2.39), (2.74), (2.73), (2.65), and the definition of x3 that
I fr (T1 , T2 , x3 , u3 ) − I fr (T1 , T2 , x∗ , u∗ )
τ1
≤ φ(t, x1 (t))dt + I f (T1 , τ1 , x1 , u1 )
T1
T2
−I f (T1 , τ1 , x∗ , u∗ ) + φ(t, x2 (t))dt
τ2
+I f (τ2 , T2 , x2 , u2 ) − I f (τ2 , T2 , x∗ , u∗ )
≤ 21 + 21 b1 .
I f (T1 , T2 , x∗ , u∗ ) + 21 + 21 b1 + δ ≥ I fr (T1 , T2 , x, u)

T2
f
≥ I (T1 , T2 , x, u) + r φ(t, x(t))dt. (2.76)
T1
By (2.74), (2.73), (2.39), and (2.68),
U fr (T1 , τ1 , x(T1 ), x∗ (τ1 )) ≤ I f (T1 , τ1 , x∗ , u∗ ) + 1 + b∗ ,

U fr (τ2 , T2 , x∗ (τ2 ), x(T2 )) ≤ I f (τ2 , T2 , x∗ , u∗ ) + 1 + b∗ . (2.77)
There exists an interval
[d1 , d2 ] ⊂ [T1 , T2 ]
such that
d2 − d1 = 1,
t1 ∈ [d1 , d2 ]. (2.78)
It follows from (2.63), (2.68)–(2.70), (2.73), and (2.77) that the trajectory-control
pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies conditions (a) and (b) of Lemma 2.12 with b1 , b2 , Q, S defined by (2.65)
and (2.66). Therefore by the choice of S0 and Lemma 2.13,
I fr (d1 , d2 , x, u) ≤ S0 . (2.79)
It is easy to see that
I fr (d1 , d2 , x∗ , u∗ ) ≤ 2Λ0 + 2a0 . (2.80)
It follows from the choice of δ1 , (2.67), (2.78)–(2.80), and (2.72) that for every
t ∈ [d1 , d2 ] ∩ [t1 − δ1 , t1 + δ1 ]
we have
|x(t) − x(t1 )| ≤ 16−1 ,

|x∗ (t) − x∗ (t1 )| ≤ 16−1
and
|x(t) − x∗ (t)| ≥ (3/4).
Therefore
T2
r φ(t, x(t))dt ≥ (3/4)rδ1
T1
and in view of (2.76) and (2.68)–(2.70),
I f (T1 , T2 , x, u) ≤ I f (T1 , T2 , x∗ , u∗ ) − 4−1 rδ1 .
On the other hand in view of the choice of δ [see (2.70)], (2.63) and Lemma 2.14,
I f (T1 , T2 , x, u) ≥ I f (T1 , T2 , x∗ , u∗ ) − 16−1 rδ1 .
The contradiction we have reached proves Lemma 2.15.

Lemma 2.16. Let ∈ (0, min{1, d0 }). Then there exist δ ∈ (0, ) and Δ1 > 0
such that for each Δ2 > Δ1 there exists a neighborhood U of fr in M for which the
following property holds:
For each g ∈ U, each T1 ∈ R1 , each T2 ∈ [T1 + Δ1 , T1 + Δ2 ], and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ, i = 1, 2,
I (T1 , T2 , x, u) ≤ U g (T1 , T2 , x(T1 ), x(T2 )) + δ
g
(2.81)
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 , T2 ]. (2.82)
Proof. There exists δ0 ∈ (0, 8−1 ) such that property (iv) (see the definition of f )
holds with = 1 and δ = δ0 . By Lemma 2.15 there exist
δ1 ∈ (0, min{4−1 , 4−1 δ0 }),
Δ1 ≥ 2b∗ + 2 (2.83)
such that for each T1 ∈ R1 , each T2 ≥ T1 + Δ1 , and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ1 , i = 1, 2,
I fr (T1 , T2 , x, u) ≤ U fr (T1 , T2 , x(T1 ), x(T2 )) + δ1 (2.84)
relation (2.82) holds for all t ∈ [T1 , T2 ]. Fix
δ ∈ (0, 16−1 δ1 ). (2.85)
Let Δ2 > Δ1 . By Proposition 2.9 there exists a neighborhood U of fr in M such

that for each g ∈ U, each T1 ∈ R1 , each T2 ∈ [T1 + Δ1 , T1 + Δ2 ], and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
min{I fr (T1 , T2 , x, u), I g (T1 , T2 , x, u)} ≤ 4 + 2a0 + Δ2 + Λ0 (Δ2 + 2) (2.86)
the inequality
|I fr (T1 , T2 , x, u) − I g (T1 , T2 , x, u)| ≤ δ (2.87)
holds.
Assume that
g ∈ U, T1 ∈ R1 , T2 ∈ [T1 + Δ1 , T1 + Δ2 ]
and a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.81). We show that (2.82) holds.

By (2.81), (2.83), property (iv), and the choice of δ0 there exist
τ1 ∈ (T1 , T1 + b∗ ] and τ2 ∈ [T2 − b∗ , T2 ) (2.88)
x1 : [T1 , τ1 ] → Rn , u1 : [T1 , τ1 ] → Rm ,
x2 : [τ2 , T2 ] → Rn , u2 : [τ2 , T2 ] → Rm
such that
xi (Ti ) = x(Ti ), i = 1, 2,
xi (τi ) = x∗ (τi ), i = 1, 2,
|x1 (t) − x∗ (t)| ≤ 1 for all t ∈ [T1 , τ1 ],
|x2 (t) − x∗ (t)| ≤ 1 for all t ∈ [τ2 , T2 ],
I f (T1 , τ1 , x1 , u1 ) ≤ I f (T1 , τ1 , x∗ , u∗ ) + 1,
I f (τ2 , T2 , x2 , u2 ) ≤ I f (τ2 , T2 , x∗ , u∗ ) + 1. (2.89)
x3 : [T1 , T2 ] → Rn , u3 : [T1 , T2 ] → Rm
by

x3 (t) = x2 (t), u3 (t) = u2 (t) for all t ∈ (τ2 , T2 ]. (2.90)
Relations (2.88), (2.89), (2.90), (2.39), and (2.40) imply that
U fr (T1 , T2 , x(T1 ), x(T2 )) ≤ I fr (T1 , T2 , x3 , u3 )

T2
≤r φ(t, x3 (t))dt + I f (T1 , T2 , x3 , u3 )
T1
f
≤ 2b∗ + 2 + I (T1 , T2 , x∗ , u∗ ) ≤ 2b∗ + 2 + Λ0 (Δ2 + 2) + 2a0 . (2.91)
It follows from (2.91), (2.83), (2.81), and the choice of U [see (2.86), (2.87)] that
|U fr (T1 , T2 , x(T1 ), x(T2 )) − U g (T1 , T2 , x(T1 ), x(T2 ))| ≤ δ
and
|I fr (T1 , T2 , x, u) − I g (T1 , T2 , x, u)| ≤ δ.
I fr (T1 , T2 , x, u) ≤ U fr (T1 , T2 , x(T1 ), x(T2 )) + 3δ

≤ U fr (T1 , T2 , x(T1 ), x(T2 )) + δ1 .
Now (2.82) follows from the relation above, (2.81), (2.85) and the choice of δ1 , Δ1
[see (2.83), (2.84)]. This completes the proof of Lemma 2.16.

Lemma 2.17. Let S > 0 and δ ∈ (0, min{1, d0 }). Then there exist Δ ≥ 1 and
a neighborhood U of fr in M such that for each g ∈ U, each T1 ∈ R1 , each
T2 ≥ T1 + Δ and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(t) − x∗ (t)| ≥ δ for all t ∈ [T1 , T2 ] (2.92)
I g (T1 , T2 , x, u) > I g (T1 , T2 , x∗ , u∗ ) + S. (2.93)
Proof. Fix a number
D1 > 16(Λ0 + a0 + b∗ + S + 22). (2.94)
By Proposition 2.7 there exists D2 > D1 such that for each g ∈ M, each T1 ∈ R1 ,
each T2 ∈ [T1 + 8−1 , T1 + 8], and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ 2D1 + 4 (2.95)
we have
|x(t)| ≤ D2 for all t ∈ [T1 , T2 ].

By Lemma 2.10 there exist D3 > D2 and an integer c1 ≥ 1 such that for each
T1 ∈ R1 , each T2 ≥ T1 + c1 , and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfying |x(T1 )| ≤ 2D2 + 1 the inequality
I f (T1 , T2 , x∗ , u∗ ) ≤ I f (T1 , T2 , x, u) + D3 (2.96)
holds.
Choose numbers
S1 > 8(D3 + 20 + c1 + 4(b∗ + 4 + c1 )(Λ0 + a0 ) + S),

D4 > 8S1 (2 + 4a0 + 4Λ0 ). (2.97)
By Proposition 2.9 there exists a neighborhood U of fr in M such that for each

g ∈ U, each T1 ∈ R1 , each T2 ∈ [T1 + 4−1 , T1 + 8], and each trajectory-control
pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
min{I fr (T1 , T2 , x, u), I g (T1 , T2 , x, u)} ≤ 2D4 + 4 (2.98)
the inequality
|I g (T1 , T2 , x, u) − I fr (T1 , T2 , x, u)| ≤ 64−1 rδ (2.99)
holds.
Fix a number
Δ > 64(rδ)−1 [2S1 + (c1 + 2)(a0 + Λ0 )]. (2.100)
Assume that g ∈ U, T1 ∈ R1 , T2 ≥ T1 + Δ, and that a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.92). By assumption (A), (2.40), (2.97), and the choice of U [see (2.98),
(2.99)] for each t1 , t2 ∈ [T1 , T2 ] satisfying t2 − t1 ∈ [2−1 , 2]
I g (t1 , t2 , x∗ , u∗ ) ≤ I fr (t1 , t2 , x∗ , u∗ ) + 64−1 δ ≤ 3Λ0 + 2a0 + 1. (2.101)

We show that (2.93) holds. There exists an integer p such that
p − 1 < T2 − T1 ≤ p. (2.102)
Set
b0 = T1 , bj = T1 + j, j = 1, . . . , p − 2, bp−1 = T2 , τ0 = T1 . (2.103)
By induction we define a sequence {τj } ⊂ {bi }p−1

i=0 .
Assume that an integer q ≥ 0, the sequence {τi }qi=0 has been defined and that
τq = bj(q) , where 0 ≤ j(q) < p − 1. If
I g (bj(q) , bj(q)+1 , x, u) ≥ D1 , (2.104)
then we set τq+1 = bj(q)+1 . If (2.104) does not hold and there exists an integer
k ∈ {j(q) + 1, . . . , p − 2} such that
I g (bi , bi+1 , x, u) < D4 , i = j(q), . . . , k − 1,

I g (bk , bk+1 , x, u) ≥ D4 , (2.105)
then τq+1 = bk+1 . Otherwise τq+1 = bp−1 .

Evidently, the construction of the sequence {τi } is completed in a finite number
of steps. Let τQ be the last element of the sequence and let
q ∈ {0, . . . , Q − 1}, j(q) ∈ {0, . . . , p − 2}, τq = bj(q) . (2.106)
We estimate
I g (τq , τq+1 , x, u) − I g (τq , τq+1 , x∗ , u∗ ).
If (2.204) holds, then by (2.101) which holds with t1 = τq , t2 = τq+1 and (2.94),
I g (τq , τq+1 , x, u) − I g (τq , τq+1 , x∗ , u∗ ) ≥ (3/4)D1 . (2.107)
Assume that (2.104) does not hold and there exists k ∈ {j(q) + 1, . . . , p − 2} which
satisfies (2.105). We show that
I g (τq , τq+1 , x, u) − I g (τq , τq+1 , x∗ , u∗ ) ≥ (3/4)D4 . (2.108)
It follows from (2.105), the choice of τq+1 , (2.101), and (2.97) that
I g (τq , τq+1 , x, u) − I g (τq , τq+1 , x∗ , u∗ )

≥ I g (bj(q) , bk , x, u) − I g (bj(q) , bk , x∗ , u∗ ) + (7/8)D4 . (2.109)
By the choice of U [see (2.98), (2.99)], (2.105), (2.101), and (2.97) for i =
j(q), . . . , k − 1,
|I g (bi , bi+1 , y, v) − I fr (bi , bi+1 , y, v)| ≤ 64−1 δr,

(y, v) ∈ {(x, u), (x∗ , u∗ )}. (2.110)
It follows from (2.38), (2.39), and (2.92) that
I fr (bj(q) , bk , x, u) ≥ I f (bj(q) , bk , x, u) + rδ(k − j(q)). (2.111)
Since (2.104) does not hold it follows from the definition of D2 [see (2.95)] that
|x(t)| ≤ D2 for all t ∈ [bj(q) , bj(q)+1 ]. (2.112)
There are two cases: (i) k − j(q) ≥ S1 ; (ii) k − j(q) < S1 .

Consider the case (i). By (2.112), the choice of D3 , c1 [see (2.96)] and the
inequality k − j(q) ≥ S1 ,
I f (bj(q) , bk , x, u) ≥ I f (bj(q) , bk , x∗ , u∗ ) − D3 . (2.113)
Combining (2.109), (2.110), (2.111), (2.113), (2.97), and the inequality
k − j(q) ≥ S1
we obtain (2.108). If k − j(q) < S1 , then (2.108) follows from (2.109), (2.101),
(2.97), and assumption (A).
Assume that (2.104) does not hold and there is no k ∈ {j(q) + 1, . . . , p − 2}
satisfying (2.105). Then
τq+1 = bp−1 , q = Q − 1,
I g (bi , bi+1 , x, u) < D4 , i = j(q), . . . , p − 2. (2.114)
By the choice of U [see (2.98), (2.99)], (2.114), (2.101), and (2.97), relation (2.110)
holds for i = j(q), . . . , p − 2. Relation (2.110), which holds for i = j(q), . . . , p − 2,
(2.92) and (2.114) imply that
I g (τq , τq+1 , x, u) − I g (τq , τq+1 , x∗ , u∗ )

≥ I g (τq , τq+1 , x, u) − I g (τq , τq+1 , x∗ , u∗ ) + r(δ − 32−1 δ)(T2 − τq ). (2.115)
Since (2.104) does not hold it follows from the choice of D2 [see (2.95)] that
|x(t)| ≤ D2 for all t ∈ [τq , τq + 1].

By this relation, the choice of D3 , c1 [see (2.96)] and assumption (A),
I f (τq , T2 , x, u) − I f (τq , T2 , x∗ , u∗ )
≥ −D3 − a0 c1 − Λ0 (c1 + 2) − 2a0 .
Together with (2.115) this implies that
I g (τq , T2 , x, u) − I g (τq , T2 , x∗ , u∗ )
≥ 2−1 rδ(T2 − τq ) − D3 − (a0 + Λ0 )(c1 + 2) (2.116)
(here q = Q − 1 [see (2.114)].

We showed that if (2.104) holds, then (2.107) is valid, if (2.104) does not hold and
there is k ∈ {j(q) + 1, . . . , p − 2} satisfying (2.105), then (2.108) holds; otherwise
q = Q − 1 and (2.116) holds.
We show that (2.93) holds. For any integer q ∈ {0, . . . , Q} there is an integer
j(q) ∈ {0, . . . , p − 1} such that τq = bj(q) . We may assume that for q = Q − 1
(2.104) does not hold and there is not an integer
k ∈ {j(Q − 1) + 1, . . . , p − 2}
satisfying (2.105) with q = Q − 1. If T2 − τQ−1 ≥ 8−1 Δ, then (2.93) follows

from (2.100), (2.116), which holds with q = Q − 1, (2.107), (2.108), and (2.94).
Therefore we may assume that
T2 − τQ−1 < 8−1 Δ.
This implies that
(7/8)Δ ≤ τQ−1 − T1 . (2.117)
There are two cases:

(i) Relation (2.104) holds for each integer q ∈ {0, . . . , Q − 2};
(ii) There is an integer q ∈ {0, . . . , Q − 2} which does not satisfy (2.104).
Consider the case (i). By the choice of {τi }Q
i=0 , (2.116), which holds with q =
Q − 1, (2.117), (2.107), (2.100), and (2.94),
Q − 1 = τQ − T1 ,
I g (T1 , T2 , x, u) − I g (T1 , T2 , x∗ , u∗ )
≥ (3/4)(Q − 1)D1 − D3 − (c1 + 2)(Λ0 + a0 )
≥ 2−1 ΔD1 − D3 − (c1 + 2)(Λ0 + a0 ) ≥ 2S
and (2.93) holds.

Consider the case (ii). It is easy to see that (2.108) holds. Relation (2.93) follows
from (2.116), which holds with q = Q − 1, (2.108), (2.107), and (2.97). This
completes the proof of Lemma 2.17.

Lemma 2.18. There exists γ > 0 such that for each δ ∈ (0, γ) and each S > 0
there exist Δ ≥ 1 and a neighborhood U of fr in M such that for each g ∈ U, each
T1 ∈ R1 , each T2 ≥ T1 + Δ, and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ γ, i = 1, 2,
|x(t) − x∗ (t)| > δ for all t ∈ [T1 , T2 ] (2.118)
the inequality
I g (T1 , T2 , x, u) > U g (T1 , T2 , x(T1 ), x(T2 )) + S
holds.
Proof. There exists
γ ∈ (0, min{8−1 , d0 })
such that property (iv) (see the definition of f ) holds with = 1 and δ = γ.
Let δ ∈ (0, γ) and S > 0. By Proposition 2.9 there exists a neighborhood U1 of
fr in M such that for each g ∈ U1 , each T1 ∈ R1 , each T2 ∈ [T1 + 8−1 , T1 + 8(b∗ +
2)], and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
min{I fr (T1 , T2 , x, u), I g (T1 , T2 , x, u)} ≤ 16(Λ0 + a0 + 4)(2b∗ + 6) (2.119)
the inequality
|I g (T1 , T2 , x, u) − I fr (T1 , T2 , x, u)| ≤ 1 (2.120)
holds.
Fix a number
S1 > 2S + 1 + 2[1 + Λ0 (4 + b∗ ) + 4a0 + b∗ + 1] + 2a0 (1 + b∗ ). (2.121)

By Lemma 2.17 there exist Δ1 ≥ 1 and a neighborhood U of fr in M such that

U ⊂ U1 and for each g ∈ U, each T1 ∈ R1 , each T2 ≥ T1 + Δ1 , and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(t) − x∗ (t)| ≥ δ for all t ∈ [T1 , T2 ] (2.122)
I g (T1 , T2 , x, u) > I g (T1 , T2 , x∗ , u∗ ) + S1 . (2.123)
Fix a number
Δ > Δ1 + 4(4 + 4b∗ ) + 16. (2.124)
Assume that g ∈ U, T1 ∈ R1 , each T2 ≥ T1 + Δ and a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.118). We show that
I g (T1 , T2 , x, u) > U g (T1 , T2 , x(T1 ), x(T2 )) + S. (2.125)
By (2.118), the choice of γ and property (iv) there exist
τ1 ∈ (T1 , T1 + b∗ ] and τ2 ∈ [T2 − b∗ , T2 ) (2.126)
x1 : [T1 , τ1 ] → Rn , u1 : [T1 , τ1 ] → Rm ,
x2 : [τ2 , T2 ] → Rn , u2 : [τ2 , T2 ] → Rm
such that
xi (Ti ) = x(Ti ), i = 1, 2,
xi (τi ) = x∗ (τi ), i = 1, 2,
|x1 (t) − x∗ (t)| ≤ 1 for all t ∈ [T1 , τ1 ],
|x2 (t) − x∗ (t)| ≤ 1 for all t ∈ [τ2 , T2 ],
I f (T1 , τ1 , x1 , u1 ) ≤ I f (T1 , τ1 , x∗ , u∗ ) + 1,
I f (τ2 , T2 , x2 , u2 ) ≤ I f (τ2 , T2 , x∗ , u∗ ) + 1. (2.127)
x3 : [T1 , T2 ] → Rn , u3 : [T1 , T2 ] → Rm
by

x3 (t) = x2 (t), u3 (t) = u2 (t) for all t ∈ (τ2 , T2 ]. (2.128)
Relations (2.126)–(2.128) imply that
U g (T1 , T2 , x(T1 ), x(T2 )) ≤ I g (T1 , T2 , x3 , u3 ). (2.129)
By (2.124), (2.127), (2.128), (2.126), (2.38), and (2.39),
I fr (T1 , T1 + 1 + b∗ , x3 , u3 ) ≤ I f (T1 , T1 + 1 + b∗ , x3 , u3 ) + b∗
≤ I f (T1 , T1 + b∗ + 1, x∗ , u∗ ) + b∗ + 1
and
I fr (T2 − 1 − b∗ , T2 , x3 , u3 ) ≤ I f (T2 − 1 − b∗ , T2 , x3 , u3 ) + b∗
≤ I f (T2 − 1 − b∗ , T2 , x∗ , u∗ ) + b∗ + 1.
By these relations, (2.40) and the choice of U1 [see (2.119), (2.120)]
max{I g (T1 , T1 + 1 + b∗ , x3 , u3 ), I g (T2 − 1 − b∗ , T2 , x3 , u3 )}

≤ 1 + Λ0 (4 + b∗ ) + 4a0 + b∗ + 1. (2.130)
It follows from (2.118), (2.124), and the choice of U and Δ1 [see (2.122), (2.123)]
that
I g (T1 + 1 + b∗ , T2 − 1 − b∗ , x, u) > I g (T1 + 1 + b∗ , T2 − 1 − b∗ , x∗ , u∗ ) + S1 .
It follows from this relation, (2.128), (2.126), (2.130), (2.121), and assumption (A)
that
I g (T1 , T2 , x, u) − I g (T1 , T2 , x3 , u3 )
> S1 − 2[1 + Λ0 (4 + b∗ ) + 4a0 + b∗ + 1] − 2a0 (1 + b∗ ) > 2S + 1.
Together with (2.129) this implies (2.125). This completes the proof of Lemma 2.18.

Lemma 2.19. Let ∈ (0, min{1, d0 }). Then there exist δ ∈ (0, ), Δ ≥ 1 and
T2 ≥ T1 + Δ, and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ, i = 1, 2,
I g (T1 , T2 , x, u) ≤ U g (T1 , T2 , x(T1 ), x(T2 )) + δ (2.131)
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 , T2 ].
Proof. There exist Δ1 ≥ 1 and δ0 ∈ (0, ) such that Lemma 2.16 holds with δ = δ0 .
Let a number γ > 0 be as guaranteed in Lemma 2.18. Choose a number
δ ∈ (0, 8−1 min{δ0 , γ, 1}). (2.132)
By Lemma 2.18 there exist Δ2 ≥ Δ1 + 1 and a neighborhood U1 of fr in M such

that for each g ∈ U1 , each T1 ∈ R1 , each T2 ≥ T1 + Δ2 , and each trajectory-control
pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ γ, i = 1, 2,
|x(t) − x∗ (t)| > δ for all t ∈ [T1 , T2 ] (2.133)
the inequality
I g (T1 , T2 , x, u) > U g (T1 , T2 , x(T1 ), x(T2 )) + 1 (2.134)
holds.
Since Lemma 2.16 holds with δ = δ0 and Δ1 there exists a neighborhood U of
fr in M such that
U ⊂ U1
and for each g ∈ U, each T1 ∈ R1 , each
T2 ∈ [T1 + Δ1 , T1 + 2Δ1 + 2Δ2 + 8]
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ0 , i = 1, 2,
I (T1 , T2 , x, u) ≤ U g (T1 , T2 , x(T1 ), x(T2 )) + δ0
g
(2.135)
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 , T2 ]. (2.136)
Choose a number
Δ > 8Δ2 + 8Δ1 + 8. (2.137)
Assume that
g ∈ U, T1 ∈ R1 , T2 ≥ T1 + Δ
and that a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.131). Let
t1 ∈ [T1 , T2 − Δ1 − Δ2 − 1]. (2.138)
We show that there exists a number t2 such that
t2 ∈ [t1 + Δ1 , t1 + Δ1 + Δ2 ],
|x(t2 ) − x∗ (t2 )| ≤ δ. (2.139)
Assume the contrary and set
t̃1 = sup{t ∈ [T1 , t1 + Δ1 ] : |x(t) − x∗ (t)| ≤ δ},

t̃2 = inf{t ∈ [t1 + Δ1 , T2 ] : |x(t) − x∗ (t)| ≤ δ}. (2.140)
Then
t̃1 < t1 + Δ1 , t̃2 > t1 + Δ1 + Δ2 ,

|x(t̃i ) − x∗ (t̃i )| = δ, i = 1, 2.
There exist
b1 ∈ (t̃1 , t1 + Δ1 ) and b2 ∈ (t1 + Δ1 + Δ2 , t̃2 ) (2.141)
such that
|x(bi ) − x∗ (bi )| < γ, i = 1, 2. (2.142)
|x(t) − x∗ (t)| > δ for all t ∈ [b1 , b2 ]. (2.143)
By (2.141)–(2.143) and the choice of U1 , Δ2 [see (2.133), (2.134)],
I g (b1 , b2 , x, u) > U g (b1 , b2 , x(b1 ), x(b2 )) + 1.
This contradicts (2.131). The contradiction we have reached proves that for each
number t1 satisfying (2.138) there exists a number t2 which satisfies (2.139).
Therefore there exists a sequence of numbers {tj }Q
j=1 such that
t1 = T 1 , tQ = T 2 ,
tj+1 − tj ∈ [Δ1 , 2Δ1 + 2Δ2 + 4], j = 1, . . . , Q − 1,
|x(tj ) − x∗ (tj )| ≤ δ, j = 1, . . . , Q. (2.144)
In view of (2.131),
I g (tj , tj+1 , x, u) ≤ U g (tj , tj+1 , x(tj ), x(tj+1 )) + 2δ.
It follows from this inequality, (2.132), (2.144) and the choice of U [see (2.133),
(2.136)] that for all j = 1, . . . , Q − 1,
|x(t) − x∗ (t)| ≤ for all t ∈ [tj , tj+1 ].
This completes the proof of Lemma 2.19.

Lemma 2.20. Let S > 0 and ∈ (0, 1). Then there exist Δ ≥ 1, δ ∈ (0, ) and
T2 ≥ T1 + 2Δ, and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ inf{σ g (T1 , T2 , y) : (T1 , y) ∈ A} + S,

|x(t) − x∗ (t)| ≤ for all t ∈ [T1 + Δ, T2 − Δ]. (2.146)
Moreover, if
|x(T1 ) − x∗ (T1 )| ≤ δ,
then
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 , T2 − Δ]
and if
|x(T2 ) − x∗ (T2 )| ≤ δ,
then
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 + Δ, T2 ].
Proof. By property (ii) (see the definition of f ), assumption (A) and Proposition 2.9,
there exists a neighborhood U0 of fr in M such that for each g ∈ U0 , each T1 ∈ R1 ,
each T2 ≥ T1 + 8−1 ,
inf{σ g (T1 , T2 , y) : (T1 , y) ∈ A} < ∞. (2.147)
There exists δ1 > 0 such that property (iv) (see the definition of f ) holds with = 1,
δ = δ1 . By Proposition 2.9 there exists a neighborhood U1 of fr in M such that
U1 ⊂ U0
and for each g ∈ U1 , each T1 ∈ R1 , each T2 ∈ [T1 + 4−1 b∗ , T1 + 4b∗ ], and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
min{I fr (T1 , T2 , x, u), I g (T1 , T2 , x, u)}

≤ 4 + 4b∗ + 4Λ0 (b∗ + 1) + 4a0 (2.148)
the inequality
|I fr (T1 , T2 , x, u) − I g (T1 , T2 , x, u)| ≤ 1 (2.149)
holds.
By Lemma 2.19 there exist
δ ∈ (0, min{1, 2−1 δ1 , d0 , }), Δ0 ≥ 1 (2.150)
and a neighborhood U2 of fr in M such that
U2 ⊂ U1
and for each g ∈ U2 , each T1 ∈ R1 , each T2 ≥ T1 + Δ0 and each trajectory-control

pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ, i = 1, 2,
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 , T2 ]. (2.152)
By Lemma 2.17, there exist Δ1 ≥ 1 and a neighborhood U of fr in M such that
U ⊂ U2
and for each g ∈ U, each T1 ∈ R1 , each T2 ≥ T1 + Δ1 , and each trajectory-control

pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(t) − x∗ (t)| ≥ δ for all t ∈ [T1 , T2 ] (2.153)

I g (T1 , T2 , x, u) > I g (T1 , T2 , x∗ , u∗ )

+S + 10 + (b∗ + 4)(4 + 4Λ0 + 4a0 ) + 4a0 + 4.
Fix a number
Δ > 128(b∗ + 1 + Δ0 + Δ1 ). (2.154)
Assume that
g ∈ U, T1 ∈ R1 , T2 ≥ T1 + 2Δ
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.145).
By the definition of Δ1 , U [see (2.153)], (2.145) and (2.154) there exists t̃ ∈
[T1 , T2 ] which satisfies
|x(t̃) − x∗ (t̃)| ≤ δ.
Set
t1 = inf{t ∈ [T1 , T2 ] : |x(t) − x∗ (t)| ≤ δ},

t2 = sup{t ∈ [T1 , T2 ] : |x(t) − x∗ (t)| ≤ δ}. (2.155)
It follows from (2.155), property (iv), and the choice of δ1 that there exist
τ2 ∈ (t2 , t2 + b∗ ] and τ1 ∈ [t1 − b∗ , t1 ), (2.156)
x1 : [τ1 , t1 ] → Rn , u1 : [τ1 , t1 ] → Rm ,
x2 : [t2 , τ2 ] → Rn , u2 : [t2 , τ2 ] → Rm
such that
xi (ti ) = x(ti ), i = 1, 2,
xi (τi ) = x∗ (τi ), i = 1, 2,
|x1 (t) − x∗ (t)| ≤ 1 for all t ∈ [τ1 , t1 ],
|x2 (t) − x∗ (t)| ≤ 1 for all t ∈ [t2 , τ2 ],

I f (τ1 , t1 , x1 , u1 ) ≤ I f (τ1 , t1 , x∗ , u∗ ) + 1,
I f (t2 , τ2 , x2 , u2 ) ≤ I f (t2 , τ2 , x∗ , u∗ ) + 1. (2.157)
We may assume without loss of generality that
τ2 = t2 + b∗ , τ1 = t1 − b∗ . (2.158)
By (2.158), (2.157), (2.38), and (2.39),
I fr (τ1 , t1 , x1 , u1 ) ≤ I f (τ1 , t1 , x1 , u1 ) + b∗ ≤ I f (τ1 , t1 , x∗ , u∗ ) + b∗ + 1,

I fr (t2 , τ2 , x2 , u2 ) ≤ I f (t2 , τ2 , x2 , u2 ) + b∗ ≤ I f (t2 , τ2 , x∗ , u∗ ) + b∗ + 1.
(2.159)
By these relations, (2.158) and the choice of U1 [see (2.148), (2.149)],
|I g (τ1 , t1 , y, v) − I fr (τ1 , t1 , y, v)| ≤ 1, (y, v) ∈ {(x1 , u1 ), (x∗ , u∗ )},

|I g (t2 , τ2 , y, v) − I fr (t2 , τ2 , y, v)| ≤ 1, (y, v) ∈ {(x2 , u2 ), (x∗ , u∗ )}.
Together with (2.159) these relations imply that
I g (τ1 , t1 , x1 , u1 ) ≤ I g (τ1 , t1 , x∗ , u∗ ) + b∗ + 3,
I g (t2 , τ2 , x2 , u2 ) ≤ I g (t2 , τ2 , x∗ , u∗ ) + b∗ + 3. (2.160)
We show that
t1 − T1 ≤ 8−1 Δ.
Assume the contrary. Then
t1 − T1 > 8−1 Δ. (2.161)
x3 : [T1 , T2 ] → Rn , u3 : [T1 , T2 ] → Rm
by
x3 (t) = x∗ (t), u3 (t) = u∗ (t) for all t ∈ [T1 , τ1 ],

x3 (t) = x1 (t), u3 (t) = u1 (t) for all t ∈ (τ1 , t1 ],
x3 (t) = x(t), u3 (t) = u(t) for all t ∈ (t1 , T2 ]. (2.162)
Relations (2.161), (2.158), (2.154) and (2.155) imply that
τ1 − T1 ≥ 2Δ0 + 2Δ1 + 2,
|x(t) − x∗ (t)| ≥ δ for all t ∈ [T1 , τ1 ].
By these relations and the choice of U and Δ1 [see (2.153)]
I g (T1 , τ1 , x, u) > I g (T1 , τ1 , x∗ , u∗ )

+S + 10 + (b∗ + 4)(4 + 4Λ0 + 4a0 ) + 4a0 + 4.
It follows from this inequality, (2.162), (2.160), (2.158), and assumption (A) that
I g (T1 , T2 , x, u) − I g (T1 , T2 , x3 , u3 ) > S + 10.
This contradicts (2.145). The contradiction we have reached proves that
t1 − T1 ≤ 8−1 Δ. (2.163)
Analogously we can show that
T2 − t2 ≤ 8−1 Δ. (2.164)
It follows from the choice of U2 and δ [see (2.150)–(2.152)], (2.163), (2.164),

(2.154), (2.145), and (2.155) that
|x(t) − x∗ (t)| ≤ for all t ∈ [t1 , t2 ].
This completes the proof of Lemma 2.20.

Proposition 2.9, property (ii) (see the definition of f ), and assumption (A) imply
the following result.
Lemma 2.21. There exists a neighborhood U of fr in M such that for each g ∈ U,
each T1 ∈ R1 , each T2 > T1 ,
inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} ≤ I g (T1 , T2 , x∗ , u∗ ) < ∞.

(2.165)
Lemma 2.22. Let S > 0. Then there exist a neighborhood U of fr in M and
numbers δ, Q > 0 such that for each g ∈ U, each T1 ∈ R1 , each T2 ≥ T1 + 1
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S,

|x(t)| ≤ Q for all t ∈ [T1 , T2 ]. (2.167)
Proof. By Lemma 2.21 there exists a neighborhood U0 of fr in M such that (2.165)

holds for each g ∈ U0 , each T1 ∈ R1 , and each T2 > T1 . There exists a number
δ0 ∈ (0, 8−1 min{1, d0 }) (2.168)
such that property (iv) (see the definition of f ) holds with = 1, δ = δ0 . Fix
S1 > S + 1. (2.169)
There exist
Δ0 ≥ 16 + 8b∗ , δ1 ∈ (0, δ0 )
and a neighborhood U1 of fr in M such that Lemma 2.20 holds with
S = 4S1 + 8, = δ0 , Δ = Δ0 ,
δ = δ1 , U = U1 . (2.170)
By Proposition 2.9 there exists a neighborhood U of fr in M such that
U ⊂ U1 ∩ U2
and for each g ∈ U, each T1 ∈ R1 , each
T2 ∈ [T1 + 8−1 min{1, b∗}, T1 + 2Δ0 + 8]
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
min{I g (T1 , T2 , x, u), I fr (T1 , T2 , x, u)}

≤ 4 + 4b∗ + (4Λ0 + 4 + 4a0 )(3Δ0 + 4 + 4b∗ ) (2.171)
the inequality
|I g (T1 , T2 , x, u) − I fr (T1 , T2 , x, u)| ≤ 1 (2.172)
holds.
By Proposition 2.7 there exists
D1 > 100 + sup{|x∗ (t)| : t ∈ R1 } (2.173)
such that for each g ∈ M, each T1 ∈ R1 , each
T2 ∈ [T1 + 8−1 , T1 + 100(Δ0 + 1)]
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ S + 2b∗ + (Λ0 + 4)(8Δ0 + 8) + 4a0 (2.174)
the following relation holds:
|x(t)| ≤ D1 for all t ∈ [T1 , T2 ]. (2.175)
Choose numbers
δ ∈ (0, 4−1 δ1 ), Q > 4D1 + 4. (2.176)
Assume that
g ∈ U, T1 ∈ R1 , T2 ≥ T1 + 1
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.166). We show that (2.167) holds. There are two cases:
(1) T2 − T1 ≥ 4Δ0 + 4;
(2) T2 − T1 < 4Δ0 + 4.
Consider the case (1). It follows from Lemma 2.20 and the definition of Δ0 ≥ 1,
δ1 ∈ (0, δ0 ) [see (2.170)], (2.166), (2.176), and (2.169) that
|x(t) − x∗ (t)| ≤ δ0 for all t ∈ [T1 + Δ0 , T2 − Δ0 ]. (2.177)

By property (iv) and the choice of δ0 there exist
τ1 ∈ [T1 + Δ0 − b∗ , T1 + Δ0 ) and τ2 ∈ (T2 − Δ0 , T2 − Δ0 + b∗ ] (2.178)
x1 : [τ1 , T1 + Δ0 ] → Rn , u1 : [τ1 , T1 + Δ0 ] → Rm ,
x2 : [T2 − Δ0 , τ2 ] → Rn , u2 : [T2 − Δ0 , τ2 ] → Rm
such that
xi (τi ) = x∗ (τi ), i = 1, 2,
x1 (T1 + Δ0 ) = x(T1 + Δ0 ),
x2 (T2 − Δ0 ) = x(T2 − Δ0 ),
I f (τ1 , T1 + Δ0 , x1 , u1 ) ≤ I f (τ1 , T1 + Δ0 , x∗ , u∗ ) + 1,
I f (T2 − Δ0 , τ2 , x2 , u2 ) ≤ I f (T2 − Δ0 , τ2 , x∗ , u∗ ) + 1,
|x1 (t) − x∗ (t)| ≤ 1 for all t ∈ [τ1 , T1 + Δ0 ],
|x2 (t) − x∗ (t)| ≤ 1 for all t ∈ [T2 − Δ0 , τ2 ]. (2.179)
We may assume that
τ1 = T1 + Δ0 − b∗ , τ2 = T2 − Δ0 + b∗ . (2.180)
x3 : [T1 , T2 ] → Rn , u3 : [T1 , T2 ] → Rm
and
x4 : [T1 , T2 ] → Rn , u4 : [T1 , T2 ] → Rm
by
x3 (t) = x∗ (t), u3 (t) = u∗ (t) for all t ∈ [T1 , τ1 ],

x3 (t) = x1 (t), u3 (t) = u1 (t) for all t ∈ (τ1 , T1 + Δ0 ],
x3 (t) = x(t), u3 (t) = u(t) for all t ∈ (T1 + Δ0 , T2 ],
x4 (t) = x(t), u4 (t) = u(t) for all t ∈ [T1 , T2 − Δ0 ],
x4 (t) = x2 (t), u4 (t) = u2 (t) for all t ∈ (T2 − Δ0 , τ2 ],
x4 (t) = x∗ (t), u4 (t) = u∗ (t) for all t ∈ (τ2 , T2 ]. (2.181)
Relations (2.180), (2.179), and (2.181) imply that
I fr (T1 , T1 + Δ0 , x3 , u3 ) ≤ I f (T1 , T1 + Δ0 , x3 , u3 ) + b∗
≤ I f (T1 , T1 + Δ0 , x∗ , u∗ ) + b∗ + 1,
I fr (T2 − Δ0 , T2 , x4 , u4 ) ≤ I f (T2 − Δ0 , , T2 , x4 , u4 ) + b∗
≤ I f (T2 − Δ0 , T2 , x∗ , u∗ ) + b∗ + 1. (2.182)
By these relations, the choice of U [see (2.171), (2.172)], (2.180),
|I fr (T1 , T1 + Δ0 , y, v) − I g (T1 , T1 + Δ0 , y, v)| ≤ 1,

(y, v) ∈ {(x3 , u3 ), (x∗ , u∗ )},
|I fr (T2 − Δ0 , T2 , y, v) − I g (T2 − Δ0 , T2 , y, v)| ≤ 1,
(y, v) ∈ {(x4 , u4 ), (x∗ , u∗ )}.
I g (T1 , T1 + Δ0 , x3 , u3 ) ≤ I f (T1 , T1 + Δ0 , x∗ , u∗ ) + b∗ + 3,
I g (T2 − Δ0 , T2 , x4 , u4 ) ≤ I f (T2 − Δ0 , T2 , x∗ , u∗ ) + b∗ + 3. (2.183)
It follows from (2.166), (2.181) and (2.183) that
S ≥ I g (T1 , T2 , x, u) − I g (T1 , T2 , x3 , u3 )
= I g (T1 , T1 + Δ0 , x, u) − I g (T1 , T1 + Δ0 , x3 , u3 )
≥ I g (T1 , T1 + Δ0 , x, u) − b∗ − 3 − Λ0 (Δ0 + 2) − 2a0
and
I g (T1 , T1 + Δ0 , x, u) ≤ S + b∗ + 3 + Λ0 (Λ0 + 2) + 2a0 . (2.184)
Analogously,
I g (T2 − Δ0 , T2 , x, u) ≤ S + b∗ + 3 + Λ0 (Δ0 + 2) + 2a0 . (2.185)
By (2.184), (2.185) and the choice of D1 [see (2.173)–(2.175)]
|x(t)| ≤ D1 for all t ∈ [T1 , T1 + Δ0 ] ∪ [T2 − Δ0 , T2 ].
Together with (2.177), (2.173), (2.168), and (2.176) this implies (2.167).
Consider the case (2). By (2.166), the choice of U [see (2.171), (2.172)], and
assumption (A),
I g (i, i + 1, x∗ , u∗ ) ≤ Λ0 + 1, i = 0, 1, . . .
and
I g (T1 , T2 , x, u) ≤ I g (T1 , T2 , x∗ , u∗ ) + S
≤ S + 2a0 + (Λ0 + 1)(4Δ0 + 6).
Relation (2.167) follows from the relation above, the choice of D1 , (2.175) and
(2.176). This completes the proof of Lemma 2.22.

Lemma 2.23. Let 0 , S > 0. Then there exist Δ > 0 and a neighborhood U of
fr in M such that for each g ∈ U, each T1 ∈ R1 , each T2 ≥ T1 + Δ, and each
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ inf{σ g (T1 , T2 , y) : (T1 , y) ∈ A} + S, (2.186)
each τ ∈ [T1 , T2 − Δ],
min{|x(t) − x∗ (t)| : t ∈ (τ, τ + Δ)} < 0 . (2.187)
Proof. We may assume without loss of generality that
0 < 2−1 min{1, d0 , b∗ }. (2.188)
We may also assume without loss of generality that property (iv) holds with = 1,
δ = 0 .
By property (ii) and assumption (A) there is Λ̃ > 0 such that
I f (T, T + b, x∗ , u∗ ) ≤ Λ̃ for all T ∈ R1

and all b ∈ [2−1 min{b∗ , 1}, 2b∗ + 2]. (2.189)
By Proposition 2.9 there exists a neighborhood U1 of fr in M such that for each

g ∈ U1 , each pair of numbers
S1 ∈ R1 , S2 ∈ [S1 + 2−1 min{b∗ , 1}, S1 + 2b∗ + 2]

x : [S1 , S2 ] → Rn , u : [S1 , S2 ] → Rm
which satisfies
min{I g (S1 , S2 , x, u), I fr (S1 , S2 , x, u)}

≤ 2̃Λ + 2 + 2b∗ (2.190)
the inequality
|I g (S1 , S2 , x, u) − I fr (S1 , S2 , x, u)| ≤ 1 (2.191)
holds.
By (2.188) and Lemma 2.17 there exist Δ1 ≥ 1 and a neighborhood U of fr in
M such that
U ⊂ U1
and for each g ∈ U, each S1 ∈ R1 , each S2 ≥ S1 + Δ1 , and each trajectory-control

pair
y : [S1 , S2 ] → Rn , v : [S1 , S2 ] → Rm
which satisfies
|y(t) − x∗ (t)| ≥ 0 for all t ∈ [S1 , S2 ] (2.192)
I g (S1 , S2 , y, v) > I g (S1 , S2 , x∗ , u∗ )

+S + 2Λ̃ + 8 + 2b∗ (1 + a0 ). (2.193)
Choose a number
Δ ≥ Δ1 + 2b∗ + 2. (2.194)
Assume that
g ∈ U, T1 ∈ R1 , T2 ≥ T1 + Δ, τ ∈ [T1 , T2 − Δ], (2.195)
a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.186) and
τ ∈ [T1 , T2 − Δ].
By (2.195), the choice of U1 and (2.189),
I g (T1 , T2 , x, u) < ∞.
In order to complete the proof of the lemma it is sufficient to show that (2.187)
holds.
Assume the contrary. Then
|x(t) − x∗ (t)| ≥ 0 for all t ∈ (τ, τ + Δ). (2.196)
It is not difficult to see that there exist real numbers τ1 , τ2 such that
T1 ≤ τ1 < τ2 ≤ T2 ,
[τ, τ + Δ] ⊂ [τ1 , τ2 ], (2.197)
|x(t) − x∗ (t)| ≥ 0 for all t ∈ [τ1 , τ2 ] (2.198)
and one of the following cases holds:
|x(τi ) − x∗ (τi )| = 0 , i = 1, 2; (2.199)

|x(τ1 ) − x∗ (τ1 )| = 0 , τ2 = T2 ; (2.200)
|x(τ2 ) − x∗ (τ2 )| = 0 , τ1 = T1 ; (2.201)
τi = Ti , i = 1, 2. (2.202)
Define ξ1 , ξ2 ∈ Rn as follows:
if (2.199) holds, then ξi = x(τi ), 1, 2; (2.203)

if (2.200) holds, then ξ1 = x(τ1 ), ξ2 = x∗ (T2 ); (2.204)
if (2.201) holds, then ξ2 = x(τ2 ), ξ1 = x∗ (T1 ); (2.205)
if (2.202) holds, then ξi = x∗ (τi ), i = 1, 2. (2.206)
We will define a trajectory-control pair
y : [τ1 , τ2 ] → Rn , v : [τ1 , τ2 ] → Rm .
Since property (iv) holds with = 1 and δ = 0 it follows from (2.199)–(2.205) that
there exist trajectory-control pairs
y1 : [τ1 , τ1 + b∗ ] → Rn , v1 : [τ1 , τ1 + b∗ ] → Rm
and
y2 : [τ2 − b∗ , τ2 ] → Rn , v2 : [τ2 − b∗ , τ2 ] → Rm
such that
y1 (τ1 ) = ξ1 ,
y1 (τ1 + b∗ ) = x∗ (τ1 + b∗ ), (2.207)
|y1 (t) − x∗ (t)| ≤ 1 for all t ∈ [τ1 , τ1 + b∗ ], (2.208)
I f (τ1 , τ1 + b∗ , y1 , v1 ) ≤ I f (τ1 , τ1 + b∗ , x∗ , u∗ ) + 1, (2.209)
y2 (τ2 − b∗ ) = x∗ (τ2 − b∗ ),
y2 (τ2 ) = ξ2 , (2.210)
|y2 (t) − x∗ (t)| ≤ 1 for all t ∈ [τ2 − b∗ , τ2 ], (2.211)
I f (τ2 − b∗ , τ2 , y2 , v2 ) ≤ I f (τ2 − b∗ , τ2 , x∗ , u∗ ) + 1. (2.212)
By (2.208), (2.38), (2.39), (2.209), (2.211), and (2.212)
I fr (τ1 , τ1 + b∗ , y1 , v1 ) ≤ I f (τ1 , τ1 + b∗ , y1 , v1 ) + b∗
≤ I f (τ1 , τ1 + b∗ , x∗ , u∗ ) + 1 + b∗ (2.213)
and
I fr (τ2 − b∗ , τ2 , y2 , v2 ) ≤ I f (τ2 − b∗ , τ2 , y2 , v2 ) + b∗
≤ I f (τ2 − b∗ , τ2 , x∗ , u∗ ) + 1 + b∗ . (2.214)
By (2.197), (2.207), and (2.210) there exists a trajectory-control pair
y : [τ1 , τ2 ] → Rn , v : [τ1 , τ2 ] → Rm
such that
y(t) = y1 (t), v(t) = v1 (t) for all t ∈ [τ1 , τ1 + b∗ ],

y(t) = x∗ (t), v(t) = u∗ (t) for all t ∈ (τ1 + b∗ , τ2 − b∗ ),
y(t) = y2 (t), v(t) = v2 (t) for all t ∈ [τ2 − b∗ , τ2 ]. (2.215)
By (2.215), (2.189), (2.195), and the choice of U1 [see (2.190), (2.191)],
I g (τ1 + b∗ , τ2 − b∗ , y, v)
is finite.
By (2.215), (2.213), (2.214), and (2.189),
I fr (τ1 , τ1 + b∗ , y, v) ≤ I f (τ1 , τ1 + b∗ , x∗ , u∗ ) + 1 + b∗ ≤ Λ̃ + 1 + b∗ ,
I fr (τ2 − b∗ , τ2 , y, v) ≤ I f (τ2 − b∗ , τ2 , x∗ , u∗ ) + 1 + b∗ ≤ Λ̃ + 1 + b∗ .
Together with (2.215) and the choice of U1 [see (2.190), (2.191)] this implies that
I g (τ1 , τ1 + b∗ , y, v) ≤ Λ̃ + 2 + b∗ ,
I g (τ2 − b∗ , τ2 , y, v) ≤ Λ̃ + 2 + b∗ . (2.216)
By the construction of (y, v) [see (2.215)], (2.207) and (2.210),
y(τ1 ) = ξ1 , y(τ2 ) = ξ2 . (2.217)
By (2.186), (2.217) and the choice of ξ1 , ξ2 [see (2.199)–(2.206)],
S ≥ I g (τ1 , τ2 , x, u) − I g (τ1 , τ2 , y, v). (2.218)
In view of (2.218), (2.215), (2.216), and (A),
S ≥ I g (τ1 + b∗ , τ2 − b∗ , x, u) − I g (τ1 + b∗ , τ2 − b∗ , x∗ , u∗ )
+I g (τ1 , τ1 + b∗ , x, u) + I g (τ2 − b∗ , τ2 , x, u)
−I g (τ1 , τ1 + b∗ , y, v) − I g (τ2 − b∗ , τ2 , y, v)
≥ I g (τ1 + b∗ , τ2 − b∗ , x, u) − I g (τ1 + b∗ , τ2 − b∗ , x∗ , u∗ )
−2a0 b∗ − 2Λ̃ − 4 − 2b∗
and
I g (τ1 + b∗ , τ2 − b∗ , x, u) − I g (τ1 + b∗ , τ2 − b∗ , x∗ , u∗ )
≤ S + 2b∗ (1 + a0 ) + 4 + 2Λ̃. (2.219)
On the other hand it follows from (2.195), (2.194), (2.197), (2.198) and the choice
of U [see (2.192), (2.193)] that
I g (τ1 + b∗ , τ2 − b∗ , x, u) > I g (τ1 + b∗ , τ2 − b∗ , x∗ , u∗ )

+S + 2b∗ (1 + a0 ) + 8 + 2Λ̃.
This contradicts (2.219). The contradiction we have reached completes the proof of
Lemma 2.23.

We suppose that the sum over empty set is zero.
Lemma 2.24. Let S, > 0. Then there exist real numbers l > 0, Δ > 0, a natural
number Q, and a neighborhood U of fr in M such that for each g ∈ U, each
T1 ∈ R1 , each T2 ≥ T1 + Δ, and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I g (T1 , T2 , x, u) ≤ inf{σ g (T1 , T2 , y) : (T1 , y) ∈ A} + S (2.220)
{ai }qi=1 , {bi }qi=1 ⊂ [T1 , T2 ],
where q ≤ Q is a natural number, such that
and
{t ∈ [T1 , T2 ] : |x(t) − x∗ (t)| > }

⊂ ∪qi=1 [ai , bi ].
Proof. We may assume without loss of generality that
< min{1, d0 }. (2.221)
By Lemma 2.19, there exist
δ0 ∈ (0, ) and Δ1 ≥ 1,
and a neighborhood U1 of fr in M such that for each g ∈ U1 , each T1 ∈ R1 , each

T2 ≥ T1 + Δ1 , and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
|x(Ti ) − x∗ (Ti )| ≤ δ0 , i = 1, 2, (2.222)

g g
I (T1 , T2 , x, u) ≤ U (T1 , T2 , x(T1 ), x(T2 )) + δ0 (2.223)
|x(t) − x∗ (t)| ≤ for all t ∈ [T1 , T2 ]. (2.224)
By Lemma 2.23, there exist Δ2 > 0 and a neighborhood U2 of fr in M such that

for each g ∈ U2 , each pair of real numbers
T1 ∈ R1 , T2 ≥ T1 + Δ2 ,
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies (2.220) and each
τ ∈ [T1 , T2 − Δ2 ],
we have
min{|x(t) − x∗ (t)| : t ∈ (τ, τ + Δ2 )} < δ0 . (2.225)
Set
U = U1 ∩ U2 , (2.226)
l = 2Δ2 + 2Δ1 (2.227)
and choose
Δ ≥ 8Δ1 + 8Δ2 (2.228)
and a natural number
Q > 6 + 2δ0−1 S. (2.229)
Assume that
g ∈ U, T1 ∈ R1 , T2 ≥ T1 + Δ (2.230)
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies (2.220). By (2.230), (2.226), (2.228), (2.220), and the choice of U2 [see
(2.225)], there exists a strictly increasing sequence of real numbers {ti }pi=1 , where
p is a natural number, such that
{ti }pi=1 ⊂ [T1 , T2 ],

|x(ti ) − x∗ (ti )| < δ0 , i = 1, . . . , p, (2.231)
t1 ∈ [T1 , T1 + Δ2 ],
Δ2 ≤ ti+1 − ti ≤ 2Δ2
for each integer i satisfying 1 ≤ i < p,
tp ≥ T2 − 2Δ2 . (2.232)
Set
S 1 = t1 . (2.233)
By induction we construct finite strictly increasing sequences of numbers
{r(i)}ki=1 ⊂ {1, . . . , p},

{Si }ki=1 ⊂ {ti : i = 1, . . . , p}
such that
Si = tr(i) , i = 1, . . . , k, (2.234)
r(k) = p and Sk = tp (2.235)
such that the following two properties hold:

(C1) For each integer i satisfying 1 ≤ i < k − 1,
I g (Si , Si+1 , x, u) > U g (Si , Si+1 , x(Si ), x(Si+1 )) + δ0 ; (2.236)
(C2) If an integer i satisfies 1 ≤ i ≤ k − 1, (2.236) holds and
r(i + 1) > r(i) + 1,
then
I g (Si , tr(i+1)−1 , x, u) ≤ U g (Si , tr(i+1)−1 , x(Si ), x(tr(i+1)−1 ))+δ0 . (2.237)
Assume that an integer j ≥ 1 and we have already defined strictly increasing

sequences of numbers
{r(i)}ji=1 ⊂ {1, . . . , p},

{Si }ji=1 ⊂ {ti : i = 1, . . . , p}
such that
Si = tr(i) , i = 1, . . . , j,
S j < tp ,
for each integer i satisfying 1 ≤ i < j relation (2.236) holds, and if an integer i
satisfies 1 ≤ i < j and
r(i + 1) > r(i) + 1,
then (2.237) is true. (Clearly, for j = 1 our assumption holds.)

Let us define
r(j + 1) ∈ {1, . . . , p} and Sj+1 = tr(j+1) .
If
I g (Sj , tp , x, u) ≤ U g (Sj , tp , x(Sj ), x(tp )) + δ0 ,
then we set
r(j + 1) = p, Sj+1 = tp ,
complete our construction and it is easy to see that properties (C1) and (C2) hold.
Assume that
I g (Sj , tp , x, u) > U g (Sj , tp , x(Sj ), x(tp )) + δ0 . (2.238)
Set
r(j + 1) = min{i ∈ {r(j) + 1, . . . , p} :

I g (Sj , ti , x, u) > U g (Sj , ti , x(Sj ), x(ti )) + δ0 }. (2.239)
If r(j + 1) = p, then we set k = j + 1, complete our construction and it is easy to

see that properties (C1) and (C2) hold.
If r(j + 1) < p, then we set Sj+1 = tr(j+1) and it is easy to see that the
assumption made for j also holds for j+1. Clearly, our construction of the sequences
is completed after a finite number of steps. Let r(k) and Sk be their last elements
respectively. It follows from our construction that r(k) = p, Sk = tp and that (C1
and (C2) hold.
By (2.220) and (C1),
S ≥ I g (T1 , T2 , x, u) − inf{σ g (T1 , T2 , y) : (T1 , y) ∈ A}

≥ {I g (Si , Si+1 , x, u) − U f (Si , Si+1 , x(Si ), x(Si+1 )) :
i is an integer such that 1 ≤ i < k − 1} ≥ (k − 2)δ0
and
k ≤ 2 + δ0−1 S. (2.240)
Set
A = {i ∈ {1, . . . , k} : i < k and

Si+1 − Si > 2Δ2 + 2Δ1 }. (2.241)
Let
i ∈ A. (2.242)
By (2.234), (2.242), and (2.241),
tr(i+1) − tr(i) = Si+1 − Si > 2Δ2 + 2Δ1 . (2.243)
By (2.243) and (2.232),
r(i + 1) > r(i) + 1. (2.244)
By (2.242), (2.241), (2.244), (C1) and (C2),
I g (tr(i) , tr(i+1)−1 , x, u) ≤ U g (tr(i) , tr(i+1)−1 , x(tr(i) ), x(tr(i+1)−1 )) + δ0 .

(2.245)
By (2.244), (2.232), and (2.243),
tr(i+1)−1 − tr(i) ≥ tr(i+1) − 2Δ2 − tr(i) ≥ 2Δ1 . (2.246)
By (2.246), (2.245), (2.231), and (2.230), (2.226) and the choice of U1 , δ0 , Δ1 [see
(2.222)–(2.224)],
|x(t) − x∗ (t)| ≤ for all t ∈ [tr(i) , tr(i+1)−1 ]. (2.247)
In view of (2.247), (2.234), and (2.232),
|x(t) − x∗ (t)| ≤ for all t ∈ [Si , Si+1 − 2Δ2 ] (2.248)

for all i ∈ A. By (2.248),
{t ∈ [T1 , T2 ] : |x(t) − x∗ (t)| > }

⊂ (∪{[Si , Si+1 ] : i ∈ {1, . . . , k − 1} \ A}) ∪ {∪[Si+1 − 2Δ2 , Si+1 ] : i ∈ A}
∪[T1 , t1 ] ∪ [tp , T2 ]. (2.249)
The right-hand side of (2.249) is a finite union of closed intervals. By (2.240) and
(2.229) their number does not exceed
2k + 2 ≤ 6 + 2δ0−1 S ≤ Q
and in view of (2.241), ( 2.232), and (2.227) their maximal length does not exceed
2Δ2 + 2Δ1 = l.
Lemma 2.24 is proved.

2.5 Proofs of Theorems 2.1–2.3 and 2.5
Proof of Theorem 2.1. The validity of assertion 1 follows from Lemma 2.10 and
assumptions (A) and (B).
We will prove assertion 2. Let f ∈ Mreg , s ∈ R1 and
x : [s, ∞) → Rn , u : [s, ∞) → Rm
be a trajectory-control pair. Assume that there exists a sequence of numbers {tk }∞

k=1
such that
tk → ∞ as k → ∞, (2.250)
I f (s, tk , x, u) − I f (s, tk , xf , uf ) → ∞ as k → ∞. (2.251)
We will show that relation (a) holds. It follows from assumption (A) and (B) that
there exists a number S0 > 0 such that
S0 > 2|x(s)| + 8,
ψ(S0 − 4) − a0 − 8 ≥ 8 sup{|I f (j, j + 1, xf , uf )| : j = 0, ±1, ±2, . . . }.
(2.252)
Let a number S > 0 be as guaranteed in assertion 1. By assumption (A) and (2.252)

we may assume without loss of generality that
lim inf |x(t)| ≤ S0 − 1. (2.253)

t→∞
2.5 Proofs of Theorems 2.1–2.3 and 2.5 69
For each integer k ≥ 1 we set
τk = inf{t ∈ [tk , ∞) : |x(t)| ≤ S0 }. (2.254)
Let k ≥ 1 be an integer and t ≥ τk . It follows from (2.254), (2.251), (2.252), the

choice of S, assertion 1, and assumption (A) that
I f (s, t, x, u) − I f (s, t, xf , uf )
≥ I f (s, tk , x, u) − I f (s, tk , xf , uf )
+I f (tk , τk , x, u) − I f (tk , τk , xf , uf ) − S
≥ I f (s, tk , x, u) − I f (s, tk , xf , uf ) − S − 4a0
−2 sup{|I f (j, j + 1, xf , uf )| : j = 0, ±1, ±2, . . . } → ∞ as k → ∞.
Therefore relation (a) holds. Together with assertion 1 and Proposition 2.7 this
implies the validity of assertion 2.

Construction of the set F :
Let A be a subset of Mreg such that fr ∈ A for each f ∈ A each r ∈ (0, 1).
Denote by Ā the closure of A in the space M.
It is easy to see that for each f ∈ A and each r ∈ (0, 1) Lemmas 2.12–2.24 hold
with
x∗ = xf , u∗ = uf , b∗ = bf , d0 = 1.
Relation (2.31) implies that
sup{|xf (t)| : t ∈ R1 } < ∞ for each f ∈ A. (2.255)
Set
E = {fr : f ∈ A, r ∈ (0, 1)}.
It follows from Lemma 2.11 that E is an everywhere dense subset of Ā.

For each f ∈ A, each r ∈ (0, 1), and each integer k ≥ 1 there exist an open
neighborhood V (f, r, k) of fr and numbers
δ(f, r, k) ∈ (0, (2k)−1 ),

γ(f, r, k) ∈ (0, δ(f, r, k)),
Δ(f, r, k), Q(f, r, k) ≥ 1,
l(f, r, k) > 0, L(f, r, k) > 0
and a natural number p(f, r, k) such that:

(a) Lemma 2.20 holds for f and r with
S = k, = (2k)−1 , U = V (f, r, k),

Δ = Δ(f, r, k), δ = 2δ(f, r, k),
x∗ = xf , u∗ = uf ;
(b) Lemma 2.21 holds for f and r with U = V (f, r, k);

(c) Lemma 2.22 holds for f and r with
S = k, U = V (f, r, k),
δ = δ(f, r, k), Q = Q(f, r, k);
(d) Lemma 2.20 holds for f and r with
S = k, = δ(f, r, k), U = V (f, r, k),

Δ = Δ(f, r, k), δ = γ(f, r, k),
x∗ = xf , u∗ = uf ;
(e) Lemma 2.24 holds for f and r with
S = 4k, = (4k)−1 , U = V (f, r, k),

l = l(f, r, k), Δ = L(f, r, k), Q = p(f, r, k),
x∗ = xf , u∗ = uf .
We define
F = Ā ∩ [∩∞
k=1 ∪ {V (f, r, k) : f ∈ A, r ∈ (0, 1)}]. (2.256)
Clearly, F is a countable intersection of open everywhere dense sets in Ā.

Theorem 2.2 now follows from the definition of F , properties (b), (c) and
Lemmas 2.21 and 2.22.
Proof of Theorem 2.3. Let f ∈ F . For each integer k ≥ 1 there exist
fk ∈ A and rk ∈ (0, 1)
such that
f ∈ V (fk , rk , k). (2.257)

Let p, q be natural numbers. We show that
|xfp (t) − xfq (t)| ≤ δ(fp , rp , p) + δ(fq , rq , q) for all t ∈ R1 . (2.258)
Let N be a natural number. Set
T2 = 4N + 4 + 4Δ(fp , rp , p) + 4Δ(fq , rq , q),

T1 = −T2 . (2.259)
It follows from Theorem 2.2 that
inf{U f (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} < ∞. (2.260)
Therefore there exists a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
such that
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2}

+8−1 min{γ(fp , rp , p), γ(fq , rq , q)}. (2.261)
By property (d), for each integer k ≥ 1, Lemma 2.20 holds with
f = fk , r = rk , S = k,
= δ(fk , rk , k), U = V (fk , rk , k),
Δ = Δ(fk , rk , k), δ = γ(fk , rk , k),
x∗ = xfk , u∗ = ufk .
Together with (2.260), (2.261), (2.259), and (2.257) this implies that
|x(t) − xfp (t)| ≤ δ(fp , rp , p), t ∈ [−N, N ],

|x(t) − xfq (t)| ≤ δ(fq , rq , q), t ∈ [−N, N ]
and
|xfq (t) − xfp (t)| ≤ δ(fp , rp , p) + δ(fq , rq , q), t ∈ [−N, N ].
Since this relation holds for any integer N ≥ 1 we conclude that (2.258) holds.
It follows from (2.258) and (2.255) that there exists a bounded continuous function
Xf : R1 → Rn which satisfies
|Xf (t) − xfp (t)| ≤ δ(fp , rp , p) for all t ∈ R1 and all integers p ≥ 1. (2.262)
Let S, > 0. Fix an integer
p > 4 + 4S + 8−1 (2.263)
and set
U = V (fp , rp , p),
Δ = Δ(fp , rp , p), δ = δ(fp , rp , p). (2.264)
Assume that
g ∈ U, T1 ∈ R1 , T2 ≥ T1 + 2Δ (2.265)
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
such that
I g (T1 , T2 , x, u) ≤ inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S,

I g (T1 , T2 , x, u) ≤ U g (T1 , T2 , x(T1 ), x(T2 )) + δ. (2.266)
It follows from (2.264), (2.265), (2.266), (2.263), and Lemma 4.20 which holds with
f = fp , r = rp , S = p,
= (2p)−1 , U = V (fp , rp , p),
Δ = Δ(fp , rp , p), δ = 2δ(fp , rp , p),
x∗ = xfp , u∗ = ufp
that
|x(t) − xfp (t)| ≤ p−1 , t ∈ [T1 + Δ, T2 − Δ].
Moreover, if |x(T1 ) − xfp (T1 )| ≤ δ, then
|x(t) − xfp (t)| ≤ p−1 , t ∈ [T1 , T2 − Δ]
and if |x(T2 ) − xfp (T2 )| ≤ δ, then
|x(t) − xfp (t)| ≤ p−1 , t ∈ [T1 + Δ, T2 ].
Together with (2.262) and (2.263) this implies the validity of Theorem 2.3.

Proof of Theorem 2.5. Choose a natural number k such that
k > 4 + 4M + 8−1 . (2.267)
By (2.256) there exist
fk ∈ A, rk ∈ (0, 1)
such that
f ∈ V (fk , rk , k). (2.268)
Set
l = l(fk , rk , k), L = L(fk , rk , k)¯, p = p(fk , rk , k),

U = V (fk , rk , k). (2.269)
Assume that
g ∈ U, T1 ∈ R1 , T2 ≥ T1 + L (2.270)
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies
I g (T1 , T2 , x, u) ≤ inf{U g (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + M. (2.271)
By (2.270), (2.271), (2.269), (2.267), condition (e), and Lemma 2.24 which holds
with
= (4k)−1 , S = 4k,
f = fk , r = rk ,
U = V (fk , rk , k),
x∗ = xfk , u∗ = ufk ,
l = l(fk , rk , k), Δ = L(fk , rk , k), Q = p(fk , rk , k)
that there exist finite sequences of numbers
{ai }qi=1 , {bi }qi=1 ⊂ [T1 , T2 ],

where q is a natural number, such that
q ≤ p(fk , rk , k) = p, (2.272)
ai ≤ bi ≤ ai + l for all integers i = 1, . . . , q (2.273)
and
{t ∈ [T1 , T2 ] : |x(t) − xfk (t)| > (4k)−1 }

⊂ ∪qi=1 [ai , bi ]. (2.274)
By Theorem 2.3 there exist
δ0 ∈ (0, 1), τ0 > 0
such that for each τ ≥ τ0 and each trajectory-control pair
y : [−τ, τ ] → Rn , v : [−τ, τ ] → Rm
satisfying
I f (−τ, τ, y, v) ≤ inf{U f (−τ, τ, ξ1 , ξ2 ) : (−τ, ξ1 ), (τ, ξ2 ) ∈ A} + δ0 (2.275)
we have
|y(t) − Xf (t)| ≤ (8k)−1 , t ∈ [−τ + τ0 , τ − τ0 ]. (2.276)
Let
T > τ0 + Δ(fk , rk , k) (2.277)
y : [−T, T ] → Rn , v : [−T, T ] → Rm
satisfy
I f (−T, T, y, v) ≤ inf{U f (−T, T, ξ1 , ξ2 ) : (−T, ξ1 ), (T, ξ2 ) ∈ A}

+ min{δ0 , δ(fk , rk , k)}. (2.278)
By (2.277), (2.278) and the choice of δ0 , τ0 [see (2.275), (2.276)],
|y(t) − Xf (t)| ≤ (8k)−1 , t ∈ [−T + τ0 , T − τ0 ]. (2.279)

By (2.277), (2.278), (2.268), condition (a), Lemma 2.20 which holds for fk and rk
with
S = k, = (2k)−1 , U = V (fk , rk , k),

Δ = Δ(fk , rk , k), δ = 2δ(fk , rk , k),
x∗ = xfk , u∗ = ufk
we have
|y(t) − xfk (t)| ≤ (2k)−1 for all t ∈ [−T + Δ(fk , rk , k), T − Δ(fk , rk , k)].
|Xf (t) − xfk (t)| ≤ (8k)−1 + (2k)−1

for all t ∈ [−T + τ0 + Δ(fk , rk , k), T − Δ(fk , rk , k) − τ0 ].
Since T is any natural number satisfying (2.277) we conclude that
|Xf (t) − xfk (t)| ≤ (8k)−1 + (2k)−1 for all t ∈ R1 .
Together with (2.274) and (2.267) this implies that for all
t ∈ [T1 , T2 ] \ ∪qi=1 [ai , bi ],

|x(t) − Xf (t)| ≤ |x(t) − xfk (t)| + |xfk (t) − Xf (t)|
≤ (4k)−1 + (8k)−1 + (2k)−1 < k −1 < .
Theorem 2.5 is proved.

Chapter 3
Infinite Horizon Problems
In this chapter we continue to use the notation and the definitions of Chap. 2.
3.1 Existence of Optimal Solutions
Let f ∈ M and τ be a real number. A trajectory-control pair
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is (f )-overtaking optimal if for any trajectory-control pair
y : [τ, ∞) → Rn , v : [τ, ∞) → Rm
satisfying
y(τ ) = x(τ )
lim sup[I f (τ, T, x, u) − I f (τ, T, y, v)] ≤ 0.

T →∞
This notion, known as the overtaking optimality criterion, was introduced in the
economics literature [17,43] and has been used in optimal control theory [12,24,49,
50].
A trajectory-control pair
x : I → Rn , u : I → Rm ,

78 3 Infinite Horizon Problems
where I is either R1 or [T, ∞) (with T ∈ R1 ) is (f )-good [12, 17, 50] if there exists
a number S > 0 such that for each T1 , T2 ∈ I satisfying T2 > T1 ,
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S.
Remark 3.1. It follows from Theorem 2.1 and assumptions (A) and (B) that
xf : R1 → Rn , uf : R1 → Rm
is an (f )-good trajectory-control pair for each f ∈ Mreg .

We say that a function f ∈ M has the turnpike property if there exists a bounded
continuous function Xf : R1 → Rn such that:
for each S, > 0 there exist numbers Δ, δ > 0 such that for each pair of numbers
T1 ∈ R1 , T2 ≥ T1 + 2Δ
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S
and
I f (T1 , T2 , x, u) ≤ U f (T1 , T2 , x(T1 ), x(T2 )) + δ
|x(t) − Xf (t)| ≤ for all t ∈ [T1 + Δ, T2 − Δ].
We can easily deduce the following result.

Proposition 3.2. Assume that f ∈ M has the turnpike property, τ ∈ R1 and that
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is an (f )-good trajectory-control pair. Then
x(t) − Xf (t) → 0 as t → ∞.
The following optimality criterion for infinite horizon problems was introduced
by Aubry and Le Daeron [7] in their study of the discrete Frenkel–Kontorova model
related to dislocations in one-dimensional crystals.
Let f ∈ M. A trajectory-control pair
x : I → Rn , u : I → Rm ,
3.1 Existence of Optimal Solutions 79
where I is either R1 or [T1 , ∞) or [T1 , T2 ] (with −∞ < T2 < T2 < ∞) is (f )-

minimal if
I f (T1 , T2 , x, u) = U f (T1 , T2 , x(T1 ), x(T2 ))
for each pair of numbers T1 , T2 ∈ I satisfying T1 < T2 .

We say that a function f ∈ M has an (LSC) property if for each T1 ∈ R1 , each
T2 > T1 , and each sequence of trajectory-control pairs
xj : [T1 , T2 ] → Rn , uj : [T1 , T2 ] → Rm , j = 1, 2, . . .
which satisfies
sup{I f (T1 , T2 , xj , uj ) : j = 1, 2, . . . } < ∞
there exists a subsequence {(xjk , ujk )}∞

k=1 and a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
such that
xjk (t) → x(t) as k → ∞ for any t ∈ [T1 , T2 ]
and
I f (T1 , T2 , x, u) ≤ lim inf I f (T1 , T2 , xj , uj ).

j→∞
In Sect. 3.3 we will prove the following result.

Proposition 3.3. Assume that f ∈ Mreg has the (LSC) property and the turnpike
property, τ ∈ R1 and that
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is an (f )-good trajectory-control pair. Then there exists an (f )-good and (f )-

minimal trajectory-control pair
x∗ : [τ, ∞) → Rn , u∗ : [τ, ∞) → Rm
such that x(τ ) = x∗ (τ ).

The next result will be proved in Sect. 3.4.
Proposition 3.4. Assume that f ∈ Mreg has the (LSC) property and the turnpike
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is an (f )-good and (f )-minimal trajectory-control pair. Then (x, u) is an (f )-

overtaking optimal trajectory-control pair.
Propositions 3.3 and 3.4 imply the following result.
Theorem 3.5. Assume that f ∈ Mreg has the (LSC) property and the turnpike
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is an (f )-good trajectory-control pair. Then there exists an (f )-overtaking optimal

x∗ : [τ, ∞) → Rn , u∗ : [τ, ∞) → Rm
such that x(τ ) = x∗ (τ ).
Let f ∈ M and τ ∈ R1 . A trajectory-control pair
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is called (f )-agreeable [19–21, 52] if for each T0 > τ and each > 0 there exists
T > T0 such that for each T > T there exists a trajectory-control pair
y : [τ, T ] → Rn , v : [τ, T ] → Rm
such that
y(t) = x(t), v(t) = u(t) for all t ∈ [τ, T0 ]
and
I f (τ, T, y, v) ≤ σ f (τ, T, x(τ )) + .
The following theorem is proved in Sect. 3.5.

Theorem 3.6. Assume that f ∈ Mreg has the turnpike property, τ ∈ R1 and that
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is an (f )-good trajectory-control pair. Then the trajectory-control pair (x, u) is (f )-

minimal if and only if (x, u) is (f )-agreeable.
Let f ∈ M, M > 0 and τ ∈ R1 . A point ξ ∈ Rn is called (f, τ, M )-good if
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
3.1 Existence of Optimal Solutions 81
such that
x(τ ) = ξ
and for each T1 ≥ τ and each T2 > T1 ,
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , η1 , η2 ) : (Ti , ηi ) ∈ A, i = 1, 2} + M.
The following two turnpike results are proved in Sects. 3.6 and 3.7
respectively.
Theorem 3.7. Assume that f ∈ Mreg has the turnpike property, M > 0 and > 0.
Then there exists L0 > 0 such that for each τ ∈ R1 and each (f )-good and (f )-
minimal trajectory-control pair
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
for which x(τ ) is an (f, τ, M )-good point the following inequality holds:
|x(t) − xf (t)| ≤ for all t ≥ τ + L0 .
Theorem 3.8. Assume that f ∈ Mreg has the turnpike property, M > 0 and > 0.
Then there exists l > 0, L > 0 and a natural number Q such that for each T1 ∈ R1 ,
each T2 ≥ T1 + L and each trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , ξ1 , ξ2 ) : (Ti , ξi ) ∈ A, i = 1, 2} + M
{ai }qi=1 , {bi }qi=1 ⊂ [T1 , T2 ],
where q ≤ Q is a natural number, such that
and
|x(t) − xf (t)| ≤ for all t ∈ [T1 , T2 ] \ ∪qi=1 [ai , bi ].

3.2 Auxiliary Results
Let f ∈ Mreg and xf , uf , bf be as guaranteed by assumption (B).

Lemma 3.9. Let 0 > 0. Then there exists a number δ > 0 such that for each
(Ti , ξi ) ∈ A, i = 1, 2 which satisfy
T2 − T1 ≥ 2bf ,
|ξi − xf (Ti )| ≤ δ, i = 1, 2
U f (T1 , T2 , ξ1 , ξ2 ) ≤ I f (T1 , T2 , xf , uf ) + 0 < ∞.
Proof. Let
= 0 /2 (3.1)
and let δ > 0 be as guaranteed by B(iv). Let
T1 ∈ R1 , T2 ≥ T1 + 2bf (3.2)
and ξ1 , ξ2 ∈ Rn be such that
(ξi , Ti ) ∈ A, i = 1, 2 (3.3)
and
|ξi − xf (Ti )| ≤ δ, i = 1, 2. (3.4)
By (3.2)–(3.4) and B(iv) which holds with and δ there exist trajectory-control pairs
x1 : [T1 , T1 + bf ] → Rn , u1 : [T1 , T1 + bf ] → Rm ,
x2 : [T2 − bf , T2 ] → Rn , u2 : [T2 − bf , T2 ] → Rm
such that
x1 (T1 ) = ξ1 ,
x1 (T1 + bf ) = xf (T1 + bf ),
x2 (T2 ) = ξ2 ,
x2 (T2 − bf ) = xf (T2 − bf ),
|x1 (t) − xf (t)| ≤ for all t ∈ [T1 , T1 + bf ],
|x2 (t) − xf (t)| ≤ for all t ∈ [T2 − bf , T2 ],

I f (T1 , T1 + bf , x1 , u1 ) ≤ I f (T1 , T1 + bf , xf , uf ) + ,
I f (T2 − bf , T2 , x2 , u2 ) ≤ I f (T2 − bf , T2 , xf , uf ) + . (3.5)
By (3.2) and (3.5) there exists a trajectory-control pair
x̃ : [T1 , T2 ] → Rn , ũ : [T1 , T2 ] → Rm
such that
x̃(t) = x1 (t), ũ(t) = u1 (t), t ∈ [T1 , T1 + bf ],

x̃(t) = xf (t), ũ(t) = uf (t), t ∈ (T1 + bf , T2 − bf ),
x̃(t) = x2 (t), ũ(t) = u2 (t), t ∈ [T2 − bf , T2 ]. (3.6)
By (3.5) and (3.6),
x̃(Ti ) = ξi , i = 1, 2. (3.7)
By (3.1), (3.2), (3.5)–(3.7),
U f (T1 , T2 , ξ1 , ξ2 ) ≤ I f (T1 , T2 , x̃, ũ)

= I f (T1 , T1 + bf , x̃, ũ) + I f (T1 + bf , T2 − bf , x̃, ũ) + I f (T2 − bf , T2 , x̃, ũ)
= I f (T1 , T1 + bf , x1 , u1 ) + I f (T1 + bf , T2 − bf , xf , uf ) + I f (T2 − bf , T2 , x2 , u2 )
≤ I f (T1 , T2 , xf , uf ) + 2 = I f (T1 , T2 , xf , uf ) + 0 .

Lemma 3.10. Let 0 > 0. Then there exists a number δ > 0 such that for each
(Ti , ξi ) ∈ A, i = 1, 2 which satisfy T2 > T1 and
|ξi − xf (Ti )| ≤ δ, i = 1, 2
U f (T1 , T2 , ξ1 , ξ2 ) ≥ I f (T1 , T2 , xf , uf ) − 0 .
Proof. Let
= 0 /8 (3.8)
and let δ > 0 be as guaranteed by B(iv). Let T1 ∈ R1 , T2 > T1 ,
(Ti , ξi ) ∈ A, i = 1, 2 (3.9)
and
|ξi − xf (Ti )| ≤ δ, i = 1, 2. (3.10)
We show that
U f (T1 , T2 , ξ1 , ξ2 ) ≥ I f (T1 , T2 , xf , uf ) − 0 .
U f (T1 , T2 , ξ1 , ξ2 ) < ∞.
By (3.9), (3.10) and B(iv) which holds with and δ there exist trajectory-control
pairs
x1 : [T1 − bf , T1 ] → Rn , u1 : [T1 − bf , T1 ] → Rm ,
x2 : [T2 , T2 + bf ] → Rn , u2 : [T2 , T2 + bf ] → Rm
such that
x1 (T1 − bf ) = xf (T1 − bf ),
x1 (T1 ) = ξ1 ,
x2 (T2 ) = ξ2 ,
x2 (T2 + bf ) = xf (T2 + bf ),
|x1 (t) − xf (t)| ≤ for all t ∈ [T1 − bf , T1 ],
|x2 (t) − xf (t)| ≤ for all t ∈ [T2 , T2 + bf ],
I f (T1 − bf , T1 , x1 , u1 ) ≤ I f (T1 − bf , T1 , xf , uf ) + ,
I f (T2 , T2 + bf , x2 , u2 ) ≤ I f (T2 , T2 + bf , xf , uf ) + . (3.11)
By (3.11) there exists a trajectory-control pair
x : [T1 − bf , T2 + bf ] → Rn , u : [T1 − bf , T2 + bf ] → Rm
such that
x(t) = x1 (t), u(t) = u1 (t), t ∈ [T1 − bf , T1 ],

x(t) = x2 (t), u(t) = u2 (t), t ∈ [T2 , T2 + bf ], (3.12)
I f (T1 , T2 , x, u) ≤ U f (T1 , T2 , ξ1 , ξ2 ) + /4. (3.13)
3.3 Proof of Proposition 3.3 85
By (3.11) and (3.12),
x(T1 − bf ) = xf (T1 − bf ),
x(T2 + bf ) = xf (T2 + bf ). (3.14)
By (3.1), (3.12), (3.14), and B(i),
I f (T1 − bf , T2 + bf , xf , uf ) ≤ I f (T1 − bf , T2 + bf , x, u)
= I f (T1 − bf , T1 , x1 , u1 ) + I f (T1 , T2 , x, u) + I f (T2 , T2 + bf , x2 , u2 )
≤ I f (T1 − bf , T1 , xf , uf ) + + I f (T1 , T2 , x, u) + I f (T2 , T2 + bf , xf , uf ) +
and
I f (T1 , T2 , xf , uf ) ≤ I f (T1 , T2 , x, u) + 2. (3.15)
By (3.8), (3.13), and (3.15),
I f (T1 , T2 , xf , uf ) ≤ U f (T1 , T2 , ξ1 , ξ2 ) + 3
≤ U f (T1 , T2 , ξ1 , ξ2 ) + 0 .

Lemmas 3.9 and 3.10 imply the following result.
Lemma 3.11. Let > 0. Then there exists a number δ > 0 such that for each
(Ti , ξi ) ∈ A, i = 1, 2 which satisfy
T2 ≥ T1 + 2bf
and
|ξi − xf (Ti )| ≤ δ, i = 1, 2
|U f (T1 , T2 , ξ1 , ξ2 ) − I f (T1 , T2 , xf , uf )| ≤ .
3.3 Proof of Proposition 3.3
There exists a number S > 0 such that
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , y1 , y2 ) : (Ti , yi ) ∈ A, i = 1, 2} + S (3.16)
for each T1 ≥ τ and each T2 > T1 .

Let a bounded continuous function Xf : R1 → Rn be as guaranteed by

the turnpike property. It follows from Remark 3.1 and the turnpike property
that
xf (t) = Xf (t) for all t ∈ R1 . (3.17)
By (3.16) and (LSC) property, for each integer N ≥ 1 there exists a trajectory-
control pair
xN : [τ, τ + N ] → Rn , uN : [τ, τ + N ] → Rm
such that
xN (τ ) = x(τ ),
I f (τ, τ + N, xN , uN ) = σ f (τ, τ + N, x(τ )). (3.18)
Let k < N be integers. Relations (3.16) and (3.18) imply that
I f (τ, τ + k, xN , uN ) = I f (τ, τ + N, xN , uN ) − I f (τ + k, τ + N, xN , uN )
≤ I f (τ, τ + N, x, u) − I f (τ + k, τ + N, x, u) + S
= I f (τ, τ + k, x, u) + S. (3.19)
Therefore for each integer k ≥ 0 the sequence
{I f (τ + k, τ + k + 1, xN , uN )}∞
N =k+1
is bounded. By (LSC) property there exists a subsequence {(xNk , uNk )}∞

k=1 and a
x∗ : [τ, ∞) → Rn , u∗ : [τ, ∞) → Rm
such that
I f (τ + j, τ + j + 1, x∗ , u∗ ) ≤ lim inf I f (τ + j, τ + j + 1, xNk , uNk ) (3.20)

k→∞
for each integer j ≥ 1 and
xNk (t) → x∗ (t) as k → ∞ for any t ∈ [τ, ∞). (3.21)
Clearly,
x∗ (τ ) = x(τ ). (3.22)
We show that
x∗ : [τ, ∞) → Rn , u∗ : [τ, ∞) → Rm
is an (f )-good trajectory-control pair. It follows from (3.20), and (3.19) that for each
integer j ≥ 1
I f (τ, τ + j, x∗ , u∗ ) ≤ I f (τ, τ + j, x, u) + S. (3.23)
Let
T1 ≥ τ and T2 > T1 .
Fix an integer q > T2 − τ. By (3.23),
I f (T1 , T2 , x∗ , u∗ )
= I f (τ, τ + q, x∗ , u∗ ) − I f (τ, T1 , x∗ , u∗ ) − I f (T2 , τ + q, x∗ , u∗ )
≤ I f (τ, τ + q, x, u) + S − I f (τ, T1 , x, u) + S
−I f (T2 , τ + q, x, u) + S = I f (T1 , T2 , x, u) + 3S.
Together with (3.16) this implies that (x∗ , u∗ ) is an (f )-good trajectory-control pair.
We show that (x∗ , u∗ ) is an (f )-minimal trajectory-control pair. Assume the
contrary. Then there exists an integer Q ≥ 1 such that
I f (τ, τ + Q, x∗ , u∗ ) − U f (τ, τ + Q, x∗ (τ ), x∗ (τ + Q)) > 0. (3.24)
Set
Δ = 4−1 [I f (τ, τ + Q, x∗ , u∗ ) − U f (τ, τ + Q, x∗ (τ ), x∗ (τ + Q))]. (3.25)
There exists a trajectory-control pair
y : [τ, τ + Q] → Rn , v : [τ, τ + Q] → Rm
such that
y(τ ) = x∗ (τ ),
y(τ + Q) = x∗ (τ + Q),
I f (τ, τ + Q, x∗ , u∗ ) − I f (τ, τ + Q, y, v) ≥ 2Δ. (3.26)
There exists δ ∈ (0, Δ) such that Lemma 3.11 holds with = 32−1 Δ(bf + 1)−1 .
Since (x∗ , u∗ ) is an (f )-good trajectory-control pair, it follows from (3.17) and
Proposition 3.2 that
x∗ (t) − xf (t) → 0 as t → ∞.
Therefore there exists an integer T0 ≥ 1 such that
|x∗ (t) − xf (t)| ≤ 16−1 δ for all t ≥ τ + T0 . (3.27)
Fix an integer
j ≥ 4 + 4bf . (3.28)
By (3.20) and (3.21) there exists an integer k for which
k ≥ Q + j + T0 ,
I f (τ, τ + T0 + Q, x∗ , u∗ ) ≤ I f (τ, τ + T0 + Q, xNk , uNk ) + 8−1 δ,
|x∗ (τ + i) − xNk (τ + i)| < 8−1 δ, i = T0 + Q, T0 + Q + j. (3.29)
By (3.26)–(3.29), the choice of δ and Lemma 3.11, there exists a trajectory-control

pair
x̃ : [τ, τ + Nk ] → Rn , ũ : [τ, τ + Nk ] → Rm
such that
x̃(t) = y(t), ũ(t) = v(t) for all t ∈ [τ, τ + Q],

x̃(t) = x∗ (t), ũ(t) = u∗ (t) for all t ∈ (τ + Q, τ + T0 + Q],
x̃(t) = xNk (t), ũ(t) = uNk (t) for all t ∈ [τ + Q + T0 + j, τ + Nk ],
I f (τ + T0 + Q, τ + Q + T0 + j, x̃, ũ)
= U f (τ + T0 + Q, τ + Q + T0 + j, x̃(τ + T0 + Q), x̃(τ + Q + T0 + j)).
(3.30)
Relations (3.18), (3.30), (3.26), and (3.22) imply that
I f (τ, τ + Nk , x̃, ũ) ≥ I f (τ, τ + Nk , xNk , uNk ). (3.31)

It follows from (3.27)–(3.30), (3.18) B(i), the choice of δ and Lemma 3.11 that
|I f (τ + T0 + Q, τ + Q + T0 + j, x̃, ũ)
−I f (τ + T0 + Q, τ + Q + T0 + j, xf , uf )| ≤ 32−1 Δ,
|I f (τ + T0 + Q, τ + Q + T0 + j, xNk , uNk )
−I f (τ + T0 + Q, τ + Q + T0 + j, xf , uf )| ≤ 32−1 Δ. (3.32)
By (3.30), (3.26), (3.29), and (3.32),
I f (τ, τ + Nk , x̃, ũ) − I f (τ, τ + Nk , xNk , uNk )

= I f (τ, τ + Q, y, v) − I f (τ, τ + Q, xNk , uNk )
+I f (τ + Q, τ + Q + T0 , x∗ , u∗ ) − I f (τ + Q, τ + Q + T0 , xNk , uNk )
+I f (τ + T0 + Q, τ + Q + T0 + j, x̃, ũ)
−I f (τ + T0 + Q, τ + Q + T0 + j, xNk , uNk )
≤ −2Δ + I f (τ, τ + Q + T0 , x∗ , u∗ ) − I f (τ, τ + Q + T0 , xNk , uNk ) + 16−1 Δ + δ
≤ −2Δ + 16−1 Δ + 8−1 δ ≤ −Δ.
This contradicts (2.31). The obtained contradiction we have reached proves that
(x∗ , u∗ ) is an (f )-minimal trajectory-control pair. Proposition 3.3 is proved.
Let a bounded continuous function Xf : R1 → Rn be as guaranteed by the turnpike

property. It follows from Remark 3.1, assumption (B), and the turnpike property that
xf (t) = Xf (t) for all t ∈ R1 . (3.33)
By Proposition 3.2
x(t) − Xf (t) → 0 as t → ∞. (3.34)
Let
y : [τ, ∞) → Rn , v : [τ, ∞) → Rm
be a trajectory-control pair satisfying
y(τ ) = x(τ ).
We will show that
lim sup[I f (τ, T, x, u) − I f (τ, T, y, v)] ≤ 0. (3.35)

T →∞
Assume the contrary. Then there exists a number > 0 for which
lim sup[I f (τ, T, x, u) − I f (τ, T, y, v)] > 2. (3.36)

T →∞
By Theorem 2.1 and Remark 3.1, we may assume without loss of generality that
(y, v) is an (f )-good trajectory-control pair. Proposition 3.2 and (3.33) imply that
y(t) − xf (t) → 0 as t → ∞. (3.37)
Set
0 = 4−1 . (3.38)
There exists δ ∈ (0, 0 ) such that Lemma 3.11 holds with = 0 and δ. By (3.33),
(3.34), and (3.37) there exists an integer Q ≥ 1 such that
|x(t) − xf (t)|, |y(t) − xf (t)| ≤ 8−1 δ, t ∈ [Q, ∞). (3.39)
It follows from (3.36) that there exists a number T0 for which
T0 > Q + 4,
I f (τ, T0 , x, u) − I f (τ, T0 , y, v) > . (3.40)
Fix an integer j ≥ 2bf + 8. By (3.39) and (3.40), the choice of δ, (LSC) property
and Lemma 3.11, there exists a trajectory-control pair
x̃ : [τ, ∞) → Rn , ũ : [τ, ∞) → Rm
such that
x̃(t) = y(t), ũ(t) = v(t) for all t ∈ [τ, T0 ],

x̃(t) = x(t), ũ(t) = u(t) for all t ∈ [T0 + j, ∞),
I (T0 , T0 + j, x̃, ũ) = U f (T0 , T0 + j, x̃(T0 ), x̃(T0 + j)).
f
(3.41)
It follows from (3.39)–(3.41), the proposition assumptions, assumption (B) the

choice of δ and Lemma 3.11 that
|I f (T0 , T0 + j, x̃, ũ) − I f (T0 , T0 + j, x, u)| ≤ 20 . (3.42)

3.5 Proof of Theorem 3.6 91
Since (x, u) is an (f )-minimal trajectory-control pair (3.41) implies that
I f (τ, T0 + j, x, u) ≤ I f (τ, T0 + j, x̃, ũ). (3.43)
On the other hand it follows from (3.41), (3.42), (3.40), and (3.38) that
I f (τ, T0 + j, x̃, ũ) − I f (τ, T0 + j, x, u)

≤ I f (τ, T0 , y, v) − I f (τ, T0 , x, u) + 20
≤ − + 20 ≤ −0 .
This contradicts (3.43). The contradiction we have reached proves Proposition 3.4.
3.5 Proof of Theorem 3.6
Let a bounded continuous function Xf : R1 → Rn be as guaranteed by the turnpike

property. It follows from Remark 3.1, assumption (B), and the turnpike property
that
xf (t) = Xf (t) for all t ∈ R1 . (3.44)
Assume that (x, u) is an (f )-agreeable trajectory-control pair. We show that (x, u)

is (f )-minimal.
Assume the contrary. Then there exists S0 > τ such that
I f (τ, S0 , x, u) > U f (τ, S0 , x(τ ), x(S0 )).
This inequality implies that there exist a constant Δ0 > 0 and a trajectory-control
pair
x1 : [τ, S0 ] → Rn , u1 : [τ, S0 ] → Rm
such that
x1 (τ ) = x(τ ), x1 (S0 ) = x(S0 ), (3.45)

∞ > I f (τ, S0 , x, u) > I f (τ, S0 , x1 , u1 ) + Δ0 . (3.46)
Since the trajectory-control pair (x, u) is (f )-agreeable there is T > S0 and a

y : [τ, T ] → Rn , v : [τ, T ] → Rm
such that
y(t) = x(t), u(t) = v(t) for all t ∈ [τ, S0 ], (3.47)

I f (τ, T, y, v) = σ f (τ, T, x(τ )) + Δ0 /2 < ∞. (3.48)
By (3.45) and (3.47) there exists a trajectory-control pair
y1 : [τ, T ] → Rn , v1 : [τ, T ] → Rm
such that
y1 (t) = x1 (t), v1 (t) = u1 (t) for all t ∈ [τ, S0 ],

y1 (t) = y(t), v1 (t) = v(t) for all t ∈ (S0 , T ]. (3.49)
By (3.49), (3.46)–(3.48),
I f (τ, T, y1 , v1 ) = I f (τ, S0 , y1 , v1 ) + I f (S0 , T, y1 , v1 )

= I f (τ, S0 , x1 , u1 ) + I f (S0 , T, y, v)
< I f (τ, S0 , x, u) − Δ0 + I f (S0 , T, y, v)
= I f (τ, T, y, v) − Δ0 ≤ σ f (τ, T, x(τ )) − Δ0 /2.
This contradicts (3.49) and (3.45). The contradiction we have reached proves that
(x, u) is an (f )-minimal trajectory-control pair.
Now assume that the trajectory-control pair (x, u) is (f )-minimal. We claim that
(x, u) is an (f )-agreeable trajectory-control pair. By Remark 3.1 (xf , uf ) is an (f )-
good trajectory-control pair. Therefore there is Q1 > 0 such that for each T1 ∈ R1
and each T2 > T1 ,
I f (T1 , T2 , xf , uf ) ≤ inf{U f (T1 , T2 , ξ1 , ξ2 ) : (Ti , ξi ) ∈ A, i = 1, 2} + Q1 .

(3.50)
Since the trajectory-control pair (x, u) is (f )-good there is Q2 > 0 such that for
each T1 ≥ τ and each T2 > T1 ,
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , ξ1 , ξ2 ) : (Ti , ξi ) ∈ A, i = 1, 2} + Q2 . (3.51)
Let
T0 > τ and > 0.
By Lemma 3.11 there exists a number δ ∈ (0, 1) such that for each (Ti , ξi ) ∈
A, i = 1, 2 which satisfy
T2 ≥ T1 + 2bf
and
|ξi − xf (Ti )| ≤ δ, i = 1, 2 (3.52)
|U f (T1 , T2 , ξ1 , ξ2 ) − I f (T1 , T2 , xf , uf )| ≤ /8. (3.53)
By Proposition 3.2 and (3.44) there exists S0 > T0 such that
|x(t) − xf (t)| ≤ δ for all t ≥ S0 . (3.54)
It follows from the turnpike property and (3.44) that there exist
Δ0 > 0, δ0 ∈ (0, min{δ, /8}) (3.55)
such that for each pair of numbers
S1 ∈ R1 , S2 ≥ S1 + 2Δ0
y : [S1 , S2 ] → Rn , v : [S1 , S2 ] → Rm
which satisfies
I f (S1 , S2 , y, v)
≤ inf{U f (S1 , S2 , ξ1 , ξ2 ) : (Si , ξi ) ∈ A, i = 1, 2} + Q1 + Q2 + 4 (3.56)
and
I f (S1 , S2 , y, v) ≤ U f (S1 , S2 , y(S1 ), y(S2 )) + δ0 (3.57)
|y(t) − xf (t)| ≤ δ for all t ∈ [S1 + Δ0 , S2 − Δ0 ]. (3.58)
Choose a number
T > T0 + 4|S0 | + 2Δ0 + 2bf . (3.59)
Let T ≥ T . There exists a trajectory-control pair
y1 : [τ, T ] → Rn , v1 : [τ, T ] → Rm
such that
y1 (τ ) = x(τ ), (3.60)
I f (τ, T, y1 , v1 ) ≤ σ f (τ, T, x(τ )) + δ0 < ∞. (3.61)
By (3.60), (3.61), and (3.51),
I f (τ, T, y1 , v1 ) ≤ I f (τ, T, x, u) + 1
≤ inf{U f (τ, T, ξ1 , ξ2 ) : (τ, ξ1 ), (T, ξ2 ) ∈ A} + Q2 + 1. (3.62)
By (3.61), (3.62), (3.59), and the choice of Δ0 , δ0 [see (3.55), (3.57) and (3.56)],
|y1 (t) − xf (t)| ≤ δ for all t ∈ [τ + Δ0 , T − Δ0 ]. (3.63)
By (3.59),
T − Δ0 − 2bf ≥ S0 . (3.64)
By (3.64) and (3.54),
|x(T − Δ0 − 2bf ) − xf (T − Δ0 − 2bf )| ≤ δ, (3.65)

|x(T − Δ0 ) − xf (T − Δ0 )| ≤ δ. (3.66)
By (3.59),
τ + Δ0 < Δ0 + T0 < T − Δ0 − 2bf < T − Δ0 . (3.67)
By (3.67) and (3.63),
|y1 (T − Δ0 − 2bf ) − xf (T − Δ0 − 2bf )| ≤ δ, (3.68)

|y1 (T − Δ0 ) − xf (T − Δ0 )| ≤ δ. (3.69)
By (3.65), (3.66), (3.68), (3.69), and the choice of δ [see (3.52) and (3.53)],
|I f (T − Δ0 − 2bf , T − Δ0 , xf , uf )
−U f (T − Δ0 − 2bf , T − Δ0 , η1 , η2 )| ≤ /8 (3.70)
for all
η1 ∈ {x(T − Δ0 − 2bf ), y1 (T − Δ0 − 2bf )}
and all
η2 ∈ {x(T − Δ0 ), y1 (T − Δ0 )}.
There exists a trajectory-control pair
y2 : [T − Δ0 − 2bf , T − Δ0 ] → Rn , v2 : [T − Δ0 − 2bf , T − Δ0 ] → Rm
such that
y2 (T − Δ0 − 2bf ) = y1 (T − Δ0 − 2bf ),
y2 (T − Δ0 ) = x(T − Δ0 ), (3.71)
I f (T − Δ0 − 2bf , T − Δ0 , y2 , v2 )
≤ U f (T − Δ0 − 2bf , T − Δ0 , y2 (T − Δ0 − 2bf ), y2 (T − Δ0 )) + /16. (3.72)
By (3.71) and (3.60), the (f )-minimality of (x, u), (3.67), (3.72) and (3.70),
I f (τ, T − Δ0 , x, u)
≤ I f (τ, T − Δ0 − 2bf , y1 , v1 ) + I f (T − Δ0 − 2bf , T − Δ0 , y2 , v2 )
≤ I f (τ, T − Δ0 − 2bf , y1 , v1 )
+U f (T − Δ0 − 2bf , T − Δ0 , y2 (T − Δ0 − 2bf ), y2 (T − Δ0 )) + /16
≤ I f (τ, T − Δ0 − 2bf , y1 , v1 )
+I f (T − Δ0 − 2bf , T − Δ0 , xf , uf ) + /16 + /16. (3.73)
By (3.67) and (3.70),
I f (T − Δ0 − 2bf , T − Δ0 , y1 , v1 )
≥ U f (T − Δ0 − 2bf , T − Δ0 , y1 (T − Δ0 − 2bf ), y1 (T − Δ0 ))
≥ I f (T − Δ0 − 2bf , T − Δ0 , xf , uf ) − /8. (3.74)
By (3.73) and (3.74),
I f (τ, T − Δ0 , x, u) ≤ I f (τ, T − Δ0 , y1 , v1 ) + /4 + /16. (3.75)

y : [τ, T ] → Rn , v : [τ, T ] → Rm
such that
y(t) = x(t), v(t) = u(t), t ∈ [τ, T − Δ0 − 2bf ],
y(t) = y1 (t), v(t) = v1 (t), t ∈ [T − Δ0 , T ],
I f (T − Δ0 − 2bf , T − Δ0 , y, v)
≤ U f (T − Δ0 − 2bf , T − Δ0 , y(T − Δ0 − 2bf ), y(T − Δ0 )) + /16. (3.76)
By (3.76) and (3.67),
y(t) = x(t) for all t ∈ [0, T0 ]. (3.77)
By (3.76), (3.70), and (3.75) and the (f )-minimality of (x, u),
I f (τ, T, y, v) = I f (τ, T − Δ0 − 2bf , x, u)
+I f (T − Δ0 , T, y1 , v1 ) + I f (T − Δ0 − 2bf , T − Δ0 , xf , uf ) + /8 + /8
≤ I f (T − Δ0 , T, y1 , v1 ) + I f (τ, T − Δ0 − 2bf , x, u)
+U f (T − Δ0 − 2bf , T − Δ0 , x(T − Δ0 − 2bf ), x(T − Δ0 )) + 3(/8)
= I f (T − Δ0 , T, y1 , v1 ) + I f (τ, T − Δ0 − 2bf , x, u)
+I f (T − Δ0 − 2bf , T − Δ0 , x, u) + 3/8
= I f (T − Δ0 , T, y1 , v1 ) + I f (τ, T − Δ0 , x, u) + 3/8
≤ I f (T − Δ0 , T, y1 , v1 ) + I f (τ, T − Δ0 , y1 , v1 ) + 5/8 + /16. (3.78)
In view of (3.78), (3.61), and (3.55),
I f (τ, T, y, v) ≤ σ f (τ, T, x(τ )) + . (3.79)
Thus for any T ≥ T there is a trajectory-control pair
y : [τ, T ] → Rn , v : [τ, T ] → Rm
satisfying (3.77) and (3.79). Therefore (x, u) is an (f )-agreeable trajectory-control

pair. Theorem 3.6 is proved.
Let f ∈ Mreg possess the turnpike property with the turnpike Xf , M > 0 and
> 0. By Remark 3.1,
xf (t) = Xf (t) for all t ∈ R1 . (3.80)
Set
M0 = M + 4 + sup{|I f (t, t + 2bf , xf , uf )| : t ∈ R1 } + 2a0 bf . (3.81)
Lemma 3.12. Let τ ∈ R1 ,
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
be an (f )-good and (f )-minimal trajectory-control pair such that x(τ ) is an

(f, τ, M )-good point. Then there exists T̃ > τ such that for each T ≥ T̃ ,
I f (τ, T, x, u)
≤ inf{U f (τ, T, ξ1 , ξ2 ) : (τ, ξ1 ), (T, ξ2 ) ∈ A} + M0 .
Proof. There exists an (f )-good trajectory-control pair
y : [τ, ∞) → Rn , v : [τ, ∞) → Rm
such that
y(τ ) = x(τ ),
I f (S1 , S2 , y, v)
≤ inf{U f (S1 , S2 , ξ1 , ξ2 ) : (Si , ξi ) ∈ A, i = 1, 2} + M (3.82)
for each S1 ≥ τ and each S2 > S1 . By Proposition 3.2, (3.80), and (3.82),
lim |x(t) − xf (t)| = 0, (3.83)

t→∞
lim |y(t) − xf (t)| = 0. (3.84)

t→∞
There exists δ ∈ (0, 1) such that B(iv) holds with = 1. By (3.83) and (3.84) there
exists
T̃ > τ + 2bf + 4 (3.85)

such that
|x(t) − xf (t)| ≤ δ for all t ≥ T̃ − 2bf , (3.86)

|y(t) − xf (t)| ≤ δ for all t ≥ T̃ − 2bf , (3.87)
Let
T ≥ T̃ . (3.88)
By (3.88), (3.87), (3.86), the choice of δ and B(iv) (which holds with = 1) there
exist trajectory-control pairs
y1 : [T − 2bf , T − bf ] → Rn , v1 : [T − 2bf , T − bf ] → Rm ,
y2 : [T − bf , T ] → Rn , v2 : [T − bf , T ] → Rm
such that
y1 (T − 2bf ) = y(T − 2bf ),

y1 (T − bf ) = xf (T − bf ),
y2 (T − bf ) = xf (T − bf ),
y2 (T ) = x(T ),
I f (T − 2bf , T − bf , y1 , v1 ) ≤ I f (T − 2bf , T − bf , xf , uf ) + 1,
I f (T − bf , T, y2 , v2 ) ≤ I f (T − bf , T, xf , uf ) + 1. (3.89)
ỹ : [τ, T ] → Rn , ṽ : [τ, T ] → Rm
such that
ỹ(t) = y(t), ṽ(t) = v(t), t ∈ [τ, T − 2bf ],
ỹ(t) = y1 (t), ṽ(t) = v1 (t), t ∈ (T − 2bf , T − bf ],

ỹ(t) = y2 (t), ṽ(t) = v2 (t), t ∈ (T − bf , T ]. (3.90)
By (3.90), (3.89), (3.81), and the (f )-minimality of (x, u),
I f (τ, T, x, u) ≤ I f (τ, T, ỹ, ṽ). (3.91)

By (3.90) and (3.89),
I f (τ, T, ỹ, ṽ) = I f (τ, T − 2bf , y, v)

+I f (T − 2bf , T − bf , y1 , v1 ) + I f (T − bf , T, y2 , v2 )
= I f (τ, T, y, v) − I f (T − 2bf , T, y, v)
+I f (T − 2bf , T − bf , xf , uf ) + 1 + I f (T − bf , T, xf , uf ) + 1
≤ I f (τ, T, y, v) + 2a0 bf + I f (T − 2bf , T, xf , yf ) + 2. (3.92)
In view of (3.91), (3.92), and (3.82),
I f (τ, T, x, u) ≤ I f (τ, T, y, v) + 2 + 2a0 bf

+I f (T − 2bf , T, xf , uf )
≤ 2 + 2a0 bf + I f (T − 2bf , T, xf , uf )
+ inf{U f (τ, T, ξ1 , ξ2 ) : (τ, ξ1 ), (T, ξ2 ) ∈ A} + M
≤ inf{U f (τ, T, ξ1 , ξ2 ) : (τ, ξ1 ), (T, ξ2 ) ∈ A} + M0 .

Proof of Theorem 3.7. It follows from the turnpike property and (3.80) that there
exist numbers Δ > 0, δ > 0 such that for each pair of number
T1 ∈ R1 , T2 ≥ T1 + 2Δ
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
which satisfies
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , ξ1 , ξ2 ) : (Ti , ξi ) ∈ A, i = 1, 2} + M0 (3.93)
and
I f (T1 , T2 , x, u) ≤ U f (T1 , T2 , x(T1 ), x(T2 )) + δ (3.94)
|x(t) − xf (t)| ≤ for all t ∈ [T1 + Δ, T2 − Δ].

Let τ ∈ R1 ,
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
be an (f )-good and (f )-minimal trajectory-control pair such that x(τ ) is an

(f, τ, M )-good point. By Lemma 3.12 there exists T̃ > τ such that for each T ≥ T̃ ,
I f (τ, T, x, u)
≤ inf{U f (τ, T, ξ1 , ξ2 ) : (τ, ξ1 ), (T, ξ2 ) ∈ A} + M0 . (3.95)
Let
T ≥ T̃ + 2Δ.
Then (3.95) holds and since (x, u) is an (f )-minimal trajectory-control pair
I f (τ, T, x, u) = U f (τ, T, x(τ ), x(T )). (3.96)
By (3.95), (3.96), and the choice of Δ, δ [see (3.93) and (3.94)]
|x(t) − xf (t)| ≤ for all t ∈ [τ + Δ, T − Δ].
Since the inequality above holds for each T ≥ T̃ + 2Δ, we conclude that
|x(t) − xf (t)| ≤ for all t ≥ τ + Δ.

We suppose that sum over empty set is zero. Since f ∈ Mreg possesses the turnpike
property (with the turnpike Xf ) it follows from Remark 3.1 and assumption B(i)
that
xf (t) = Xf (t) for all t ∈ R1 . (3.97)
By Remark 3.1 (xf , uf ) is an (f)-good trajectory-control pair. Therefore there is

Q0 > 0 such that for each T1 ∈ R1 and each T2 > T1 ,
I f (T1 , T2 , xf , uf ) ≤ inf{U f (T1 , T2 , ξ1 , ξ2 ) : (Ti , ξi ) ∈ A, i = 1, 2} + Q0 .

(3.98)
Lemma 3.13. Let −∞ < T1 ≤ S1 < S2 ≤ T2 < ∞ and
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
be a trajectory-control pair such that
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , ξ1 , ξ2 ) : (Ti , ξi ) ∈ A, i = 1, 2} + M. (3.99)
Then
I f (S1 , S2 , x, u) ≤ inf{U f (S1 , S2 , ξ1 , ξ2 ) : (Si , ξi ) ∈ A, i = 1, 2} + M + 3Q0 .
Proof. By (3.99) and (3.98),
I f (S1 , S2 , x, u)
= I f (T1 , T2 , x, u) − I f (T1 , S1 , x, u) − I f (S2 , T2 , x, u)
≤ I f (T1 , T2 , xf , uf ) + M − (I f (T1 , S1 , xf , uf ) − Q0 ) − (I f (S2 , T2 , xf , uf ) − Q0 )
= I f (S1 , S2 , xf , uf ) + M + 3Q0
≤ inf{U f (S1 , S2 , ξ1 , ξ2 ) : (Si , ξi ) ∈ A, i = 1, 2} + M + 3Q0 .

Proof of Theorem 3.8. By the turnpike property and (3.97) there exist numbers Δ >
0, δ > 0 such that for each pair of number
S1 ∈ R1 , S2 ≥ S1 + 2Δ
x : [S1 , S2 ] → Rn , u : [S1 , S2 ] → Rm
which satisfies
I f (S1 , S2 , x, u) ≤ U f (S1 , S2 , x(S1 ), x(S2 )) + δ (3.100)
and
I f (S1 , S2 , x, u) ≤ inf{U f (S1 , S2 , ξ1 , ξ2 ) : (Si , ξi ) ∈ A, i = 1, 2} + M + 3Q0

(3.101)
|x(t) − xf (t)| ≤ for all t ∈ [S1 + Δ, S2 − Δ]. (3.102)

Fix
l > 1 + 2Δ, (3.103)
a natural number
Q > 3M δ −1 + 4 (3.104)
and
L > lQ. (3.105)
Assume that
T1 ∈ R 1 , T2 ≥ T1 + L
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies
I f (T1 , T2 , x, u) ≤ inf{U f (T1 , T2 , ξ1 , ξ2 ) : (Ti , ξi ) ∈ A, i = 1, 2} + M. (3.106)
By (3.106) and Lemma 3.13 for all numbers S1 , S2 satisfying
T 1 ≤ S1 < S2 ≤ T 2
we have
I f (S1 , S2 , x, u) ≤ inf{U f (S1 , S2 , ξ1 , ξ2 ) : (Si , ξi ) ∈ A, i = 1, 2} + M + 3Q0 .

(3.107)
Set
t0 = T 1 . (3.108)
Assume that an integer k ≥ 0 and we have defined a strictly increasing sequence
{ti }ki=0 ⊂ [T1 , T2 ]
such that tk < T2 and for each integer i satisfying 1 ≤ i ≤ k,
I f (ti−1 , ti , x, u) > U f (ti−1 , ti , x(ti−1 ), x(ti )) + δ (3.109)

and there a number si such that
ti−1 < si < ti , si > ti − 1,

I f (ti−1 , si , x, u) ≤ U f (ti−1 , si , x(ti−1 ), x(si )) + δ. (3.110)
(Clearly, for k = 0 our assumption holds.)

By (3.106) and (3.109),
M ≥ I f (T1 , T2 , x, u) − U f (T1 , T2 , x(T1 ), x(T2 ))

≥ {I f (ti−1 , ti , x, u) − U f (ti−1 , ti , x(ti−1 ), x(ti )) :
i is an integer and 1 ≤ i ≤ k} ≥ δk
and
k ≤ M δ −1 . (3.111)
Let us define tk+1 . If
I f (tk , T2 , x, u) ≤ U f (tk , T2 , x(tk ), x(T2 )) + δ,
then we set tk+1 = T2 and our construction is completed.

Assume that
I f (tk , T2 , x, u) > U f (tk , T2 , x(tk ), x(T2 )) + δ. (3.112)
Set
t̃k+1 = inf{τ ∈ (tk , T2 ] : I f (tk , τ, x, u) > U f (tk , τ, x(tk ), x(τ )) + δ}. (3.113)
Clearly, t̃k+1 is well defined and
T2 ≥ t̃k+1 > tk .
If t̃k+1 = T2 , then we set tk+1 = T2 and our construction is completed.

If t̃k+1 < T2 , then there exist
tk+1 ∈ (t̃k+1 , T2 ), sk+1 ∈ (tk , t̃k+1 )
such that
tk+1 − sk+1 < 1,
and in this case the assumption we made for k also holds for k + 1.
By (3.111) our sequence {ti } is finite. Let k ≥ 0 be an integer such that tk+1 is
its last element. By (3.111),
k + 1 ≤ M δ −1 + 1. (3.114)
It follows from the construction of the sequence that

tk+1 = T2
and that for each integer i ∈ {1, . . . , k + 1} there a number si ∈ (ti−1 , ti ] such that
s i ≥ ti − 1 (3.115)
and (3.110) holds. Set

A = {i ∈ {1, . . . , k + 1} : ti − ti−1 ≥ 2Δ + 1}. (3.116)
Let
i ∈ A. (3.117)
By (3.117), (3.116), and (3.115),
si − ti−1 ≥ 2Δ (3.118)
and (3.110) holds. By (3.107),
I f (ti−1 , si , x, u)≤ inf{U f (ti−1 , si , ξ1 , ξ2 ) : (ti−1 , ξ1 ), (si , ξ2 )∈A}+M +3Q0 .

(3.119)
By (3.110), (3.119), (2.118), and the choice of Δ, δ [see (3.100) and (3.101)],
|x(t) − xf (t)| ≤ for all t ∈ [ti−1 + Δ, si − Δ] (3.120)
for all i ∈ A.
By (3.120),
{t ∈ [T1 , T2 ] : |x(t) − xf (t)| > }

⊂ ∪{[ti−1 , ti ] : i ∈ {1, . . . , k + 1} \ A} ∪ {[ti−1 , ti−1 + Δ] : i ∈ A}
∪{[si − Δ, ti ] : i ∈ A}. (3.121)
Clearly, the right-hand side of (3.121) is a finite union of intervals, by (3.116),

(3.115), (3.103) their maximal length does not exceed 2Δ + 1 ≤ l and in view
of (3.104) and (3.114) their number does not exceed
3(k + 1) ≤ 3M δ −1 + 3 ≤ Q.

Chapter 4
Linear Control Systems
In this chapter we continue to use the notation and the definitions of Chaps. 2 and 3.
4.1 The Class of Problems
Consider the control system described by (2.1)–(2.5). Assume that
M = Rn+m+1 , U (t, x) = Rm , (t, x) ∈ Rn+1 , (4.1)

G(t, x, u) = Ax + Bu, t ∈ R1 , x ∈ Rn , u ∈ Rm , (4.2)
where A and B are given matrices of dimensions n × n and n × m. We also assume

that linear system
x (t) = Ax(t) + Bu(t) (4.3)
is controllable.
Denote by Mc the set of all continuous functions f ∈ M which satisfy the
following assumptions:
D(i) for each (t, x) ∈ Rn+1 the function f (t, x, ·) : Rn → R1 is convex;
D(ii) for each K > 0 there exists a constant af,K > 0 and an increasing function
ψf,K : [0, ∞) → [0, ∞)
such that
ψf,K (t) → ∞ as t → ∞
and
f (t, x, u) ≥ ψf,K (|u|)|u| − af,K

106 4 Linear Control Systems
for each t ∈ R1 , each u ∈ Rm , and each x ∈ Rn satisfying |x| ≤ K;

D(iii) for each M, > 0 there exist Γ, δ > 0 such that
|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤ max{f (t, x1 , u1 ), f (t, x2 , u2 )}
for each t ∈ R1 , each u1 , u2 ∈ Rm and each x1 , x2 ∈ Rn which satisfy
|xi | ≤ M, |ui | ≥ Γ, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ;
D(iv) for each M, > 0 there exists δ > 0 such that
|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤
for each t ∈ R1 , each u1 , u2 ∈ Rm , and each x1 , x2 ∈ Rn which satisfy
|xi |, |ui | ≤ M, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ.
D(v) the function f is bounded on R1 × E for any bounded set E ⊂ Rn × Rm .

Note that assumption D(ii) implies that the function f grows superlinearly with
respect to u while assumption D(iv) means the uniform continuity of the function
with respect to x and u on bounded sets.
It is an elementary exercise to show that an integrand f = f (t, x, u) ∈
C 1 (Rn+m+1 ) belongs to Mc if f satisfies Assumptions A and D(i), there exists
a constant af > 0 and an increasing function
ψf : [0, ∞) → [0, ∞)
such that
ψf (t) → ∞ as t → ∞
and
f (t, x, u) ≥ ψf (|u|)|u| − af
for each t ∈ R1 , each u ∈ Rm and each x ∈ Rn ,
sup{|f (t, 0, 0)| : t ∈ R1 } < ∞
and there exists an increasing function ψ2 : [0, ∞) → [0, ∞) such that
max{|∂f /∂x(t, x, u)|, |∂f /∂u(t, x, u)|} ≤ ψ2 (|x|)(1 + ψ(|u|)|u|)
for each t ∈ R1 , each x ∈ Rn and each u ∈ Rm .

We can easily deduce the following result (for the proof see Proposition 2.1
of [48]).
Proposition 4.1. Let f ∈ Mc and M, be positive numbers. Then there exist Γ > 0
and δ > 0 such that
|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤ min{f (t, x1 , u1 ), f (t, x2 , u2 )} (4.4)
|xi | ≤ M, |ui | ≥ Γ, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ. (4.5)
In the next section we prove the following result.

Proposition 4.2. Let f ∈ Mc . Then f has (LSC) property.
Let −∞ < T1 < T2 < ∞ and
xj : [T1 , T2 ] → Rn , uj : [T1 , T2 ] → Rm , j = 1, 2, . . .
be a sequence of trajectory-control pairs which satisfies
sup{I f (T1 , T2 , xj , uj ) : j = 1, 2, . . . } < ∞. (4.6)
By Proposition 2.7 there exists a number M0 > 0 such that
|xi (t)| ≤ M0 , t ∈ [T1 , T2 ], i = 1, 2, . . . . (4.7)
It follows from (4.6), assumption D(ii) and the properties of the functions ψf,M0
that the sequence of functions {|ui (·)|}∞
i=1 is equiabsolutely integrable on [T1 , T2 ].
Therefore there exists subsequence {(xik , uik )∞k=1 and
h1 ∈ L1 (Rn ; (T1 , T2 )), h2 ∈ L1 (Rm ; (T1 , T2 ))
such that
xik → h1 as k → ∞ weakly in L1 (Rn ; (T1 , T2 )),

uik → h2 as k → ∞ weakly in L1 (Rm ; (T1 , T2 )). (4.8)
This implies that
Axik + Buik → Ah1 + Bh2 as k → ∞ weakly in L1 (Rn ; (T1 , T2 )). (4.9)

We may assume without loss of generality that there exists
lim xik (T1 ).

k→∞
For each t ∈ [T1 , T2 ] set

t
x(t) = lim xik (T1 ) + [Ah1 (t) + Bh2 (t)]dt. (4.10)
k→∞ T1
Clearly,
xik (t) → x(t) as k → ∞
for any t ∈ [T1 , T2 ] and
h1 = x.
By Proposition 2.8,
xik (t) → x(t) as k → ∞ uniformly in [T1 , T2 ]. (4.11)
It remains to show that
I f (T1 , T2 , x, h2 ) ≤ lim inf I f (T1 , T2 , xik , uik ).

k→∞
We show that
T2
I f (T1 , T2 , xik , uik ) − f (t, x(t), uik (t))dt → 0 as k → ∞. (4.12)
T1
Set
M1 = sup{I f (T1 , T2 , xj , uj ) : j = 1, 2, . . . } < ∞,

M2 = lim inf {I f (T1 , T2 , xik , uik ). (4.13)
k→∞
Let > 0. By (4.7) and (4.11),
|x(t)| ≤ M0 , t ∈ [T1 , T2 ]. (4.14)
Fix a positive number
γ < 4−1 [1 + M1 + a0 (T2 − T1 )]−1 . (4.15)

By Proposition 4.1 there exist Γ > 0 and δ0 > 0 such that
|f (t, y1 , u1 ) − f (t, y2 , u2 )| ≤ γ min{f (t, y1 , u1 ), f (t, y2 , u2 )} (4.16)
for each t ∈ R1 , each u1 , u2 ∈ Rm , and each y1 , y2 ∈ Rn which satisfy
|yi | ≤ M0 , |ui | ≥ Γ, i = 1, 2, max{|y1 − y2 |, |u1 − u2 |} ≤ δ0 . (4.17)
Since the function f is continuous there exists a number
δ ∈ (0, δ0 ) (4.18)
such that
|f (t, y1 , u1 ) − f (t, y2 , u2 )| ≤ [8(T2 − T1 + 1)]−1 (4.19)
for each t ∈ [T1 , T2 ], each u1 , u2 ∈ Rm , and each y1 , y2 ∈ Rn which satisfy
|yi | + |ui | ≤ 2M0 + 2Γ + 2, i = 1, 2,

|y1 − y2 | + |u1 − u2 | ≤ δ. (4.20)
By (4.11) there exists an integer k0 ≥ 1 such that for every integer k ≥ k0 ,
|xik (t) − x(t)| ≤ 2−1 δ, t ∈ [T1 , T2 ]. (4.21)
Fix an integer k ≥ k0 . Set
E1 = [t ∈ [T1 , T2 ] : |uik (t)| ≥ Γ },

E2 = [T1 , T2 ] \ E1 . (4.22)
Clearly,
T2
I f (T1 , T2 , xik , uik ) − f (t, x(t), uik (t))dt = σ1 + σ2 , (4.23)
T1
where

σj = [f (t, xik (t), uik (t)) − f (t, x(t), uik (t))]dt, j = 1, 2. (4.24)
Ej
We estimate σ1 , σ2 separately.
Let t ∈ E2 . It follows from (4.22), (4.7), (4.14), (4.21), and the choice of δ (see
(4.19)) that
|f (t, xik (t), uik (t)) − f (t, x(t), uik (t))| ≤ [8(T2 − T1 + 1)]−1 .
This implies that
|σ2 | ≤ 8−1 . (4.25)
Let t ∈ E1 . It follows from (4.22), (4.7), (4.14), (4.18), (4.21), and the choice of
Γ, δ0 (see (4.17)) that
|f (t, xik (t), uik (t)) − f (t, x(t), uik (t))|

≤ γf (t, xik (t), uik (t)).
Together with assumption (A), (4.13) and (4.15) this implies that

|σ1 | ≤ γ f (t, xik (t), uik (t))dt
E1
f
≤ γ(I (T1 , T2 , xik , uik ) + a0 (T2 − T1 ))
≤ γ(M1 + a0 (T2 − T1 )) ≤ 4−1 . (4.26)
Combining (4.23)–(4.26) we obtain that

T2
|I f (T1 , T2 , xik , uik ) − f (t, x(t), uik (t))dt| ≤
T1
for each integer k ≥ k0 . Therefore (4.12) is valid.

We show that
I f (T1 , T2 , x, h2 ) ≤ M2 . (4.27)
Let δ > 0. We may assume without loss of generality that
I f (T1 , T2 , xik , uik ) ≤ M2 + 2−1 δ for each integer k ≥ 1. (4.28)
Denote by F the set of all functions u ∈ L1 (Rm ; (T1 , T2 )) satisfying

T2
f (t, x(t), u(t))dt ≤ M2 + δ.
T1
It follows from assumption D(i) that the set F is convex. By (4.12) and (4.28),
uik ∈ F for all large integers k. (4.29)
Since the set F is convex it follows (4.29) and (4.8) that there exists a sequence
{vi }∞
i=1 ⊂ F which satisfies
T2
|vi (t) − h2 (t)|dt → 0 as i → ∞.
T1
4.3 A Continuity Property 111
We can assume by extracting a subsequence and re-indexing that
vi (t) → h2 (t) as i → ∞ a. e. in [T1 , T2 ].
It follows from this relation, the continuity of f , assumption (A), the definition of
F , and Fatau’s lemma that
T2
f (t, x(t), h2 (t))dt
T1

T2
≤ sup f (t, x(t), vi (t))dt : i = 1, 2, . . . ≤ M2 + δ
T1
and
T2
f (t, x(t), h2 (t))dt ≤ M2 + δ.
T1
Since this relation holds for any positive δ we conclude that

T2
f (t, x(t), h2 (t))dt ≤ M2 .
T1
This completes the proof of Proposition 4.2.
4.3 A Continuity Property
We use the following result [25].

Proposition 4.3. For every ỹ, z̃ ∈ Rn and every T > 0 there exists a unique
solution x(·), y(·) of the following system
x = Ax + BB t y,
y = x − At y
with the boundary constraints
x(0) = ỹ, x(T ) = z̃
(where B t is a transpose of B).

Corollary 4.4. Let f ∈ Mc , −∞ < T1 < T2 < ∞ and x, y ∈ Rn . Then
U f (T1 , T2 , x, y) < ∞.
Let τ be a positive number. It follows from Proposition 4.3 that for each ỹ, z̃ ∈
Rn there exists a unique solution x(·), y(·) of the following system
(x , y )t = C((x, y)t ) (4.30)
with the boundary constraints x(0) = ỹ, x(τ ) = z̃ and
C((x, y)t ) = (Ax + BB t y, x − At y)t . (4.31)
For any initial value (x0 , y0 ) ∈ Rn × Rn there exists a unique solution of (4.30)
(x(s), y(s))t = esC (x0 , y0 )t , s ∈ R1 .
Clearly, for each ỹ, z̃ ∈ Rn there exists a unique pair of vectors
Dτ,1 (ỹ, z̃), Dτ,2 (ỹ, z̃) ∈ Rn
such that the function
(x(s), y(s)) = (esC ((Dτ,1 (ỹ, z̃), Dτ,2 (ỹ, z̃))t )t , s ∈ R1 (4.32)
satisfies (4.30) with the boundary constraints
x(0) = ỹ, x(τ ) = z̃.
Dτ,j : Rn × Rn → Rn , j = 1, 2
are linear operators.

Using assumption D(v) we can easily deduce the following result.
Proposition 4.5. Let f ∈ Mc , M, τ > 0. Then
sup{|U f (T, T + τ, y, z)| : T ∈ R1 , y, z ∈ Rn , |y|, |z| ≤ M } < ∞.
Proposition 4.6. Let f ∈ Mc , M, τ, > 0. Then there exists a number δ > 0 such
that:
1. for each T ∈ R1 , each y1 , y2 , z1 , z2 ∈ Rn satisfying
|yi |, |zi | ≤ M, i = 1, 2, |y1 − y2 |, |z1 − z2 | ≤ δ (4.33)
|U f (T, T + τ, y1 , z1 ) − U f (T, T + τ, y2 , z2 )| ≤ . (4.34)

2. for each T ∈ R1 , each y1 , y2 , z1 .z2 ∈ Rn satisfying (4.33) and each trajectory-

control pair
x1 : [T, T + τ ] → Rn , u1 : [T, T + τ ] → Rm
which satisfies
x1 (T1 ) = y1 , x1 (T2 ) = z1 ,
I f (T1 , T2 , x1 , u1 ) = U f (T1 , T2 , y1 , z1 )
x2 : [T, T + τ ] → Rn , u2 : [T, T + τ ] → Rm
such that
x2 (T1 ) = y2 , x2 (T2 ) = z2 ,
|I f (T1 , T2 , x2 , u2 ) − I f (T1 , T2 , x1 , u1 )| ≤ ,
|x1 (t) − x2 (t)| ≤ , t ∈ [T1 , T2 ].
Proof. Set
M0 = sup{|U f (T, T + τ, y, z)| : T ∈ R1 , y, z ∈ Rn , |y|, |z| ≤ M + 4} < ∞.

(4.35)
By Proposition 4.5, M0 < ∞. By Proposition 2.7 there exists a number M1 > 0
such that for each T ∈ R1 and each trajectory-control pair
x : [T, T + τ ] → Rn , u : [T, T + τ ] → Rm
satisfying
I f (T, T + τ, x, u) ≤ M0 + 4
|x(t)| ≤ M1 , t ∈ [T, T + τ ]. (4.36)
Choose a number δ1 > 0 such that
4δ1 (M0 + a0 τ + τ ) < . (4.37)

By Proposition 4.1 there exist Γ0 > 8 and δ2 ∈ (0, 8−1 ) such that
|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤ δ1 min{f (t, x1 , u1 ), f (t, x2 , u2 )} (4.38)
for each t ∈ R1 , each x1 , x2 ∈ Rn , and each u1 , u2 ∈ Rm which satisfy
|xi | ≤ M1 + 8, i = 1, 2, |ui | ≥ Γ0 − 4, i = 1, 2,
|x1 − x2 |, |u1 − u2 | ≤ δ2 . (4.39)
By assumption D(iv) there exists a number
δ3 ∈ (0, 4−1 min{δ1 , δ2 , }) (4.40)
such that
|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤ δ1 (4.41)
for each t ∈ R1 , each x1 , x2 ∈ Rn , and each u1 , u2 ∈ Rm which satisfy
|xi |, |ui | ≤ Γ0 + M1 + 4, i = 1, 2,
|x1 − x2 |, |u1 − u2 | ≤ δ3 . (4.42)
There exists a number
δ ∈ (0, 8−1 δ3 ) (4.43)
such that
(1 + ||B||)|etC ((Dτ,1 (y, z), Dτ,2(y, z))t )| ≤ 2−1 δ3 , t ∈ [0, τ ] (4.44)
for each y, z ∈ Rn satisfying |y|, |z| ≤ δ.

Assume that T ∈ R1 and y1 , y2 , z1 .z2 ∈ Rn satisfy (4.33). By Proposition 4.2
and Corollary 4.4, there exists a trajectory-control pair
x1 : [T, T + τ ] → Rn , u1 : [T, T + τ ] → Rm
such that
x1 (T ) = y1 , x1 (T + τ ) = z1 ,
I (T, T + τ, x1 , u1 ) = U f (T, T + τ, y1 , z1 ).
f
(4.45)
It follows from (4.45), (4.35), (4.33), and the definition of M1 that
|x1 (t)| ≤ M1 , t ∈ [T, T + τ ]. (4.46)
Define functions hj : R1 → Rn , j = 1, 2 as follows:
(h1 (s), h2 (s))t = esC ((Dτ,1 (y2 − y1 , z2 − z1 ), Dτ,2 (y2 − y1 , z2 − z1 ))t ), s ∈ R1 .

(4.47)
Set
x2 (t) = x1 (t) + h1 (t − T ), u2 (t) = u1 (t) + B t h2 (t − T ), t ∈ [T, T + τ ]. (4.48)
It follows from (4.47), (4.48), (4.45), (4.30)–(4.32), and the definition of Dτ,1 , Dτ,2
that
(x2 , u2 ) is a trajectory-control pair and

x2 (T ) = y2 , x2 (T + τ ) = z2 . (4.49)
By (4.48), (4.47), (4.44), (4.33), and the choice of δ,
|x2 (t) − x1 (t)|, |u2 (t) − u1 (t)| ≤ 2−1 δ3 , t ∈ [T, T + τ ]. (4.50)
Assumption (A), (4.45), (4.35), and (4.33) imply that for any measurable set E ⊂
[T, T + τ ],

f (t, x1 (t), u1 (t))dt ≤ M0 + a0 τ. (4.51)
E
Set
E1 = {t ∈ [T, T + τ ] : |u1 (t)| ≥ Γ0 },

E2 = [T, T + τ ] \ E1 .
It follows from (4.46), (4.50), (4.40), and the choice of Γ0 , δ2 and (4.51) that

|f (t, x1 (t), u1 (t)) − f (t, x2 (t), u2 (t))|dt
E1

≤ δ1 f (t, x1 (t), u1 (t))dt ≤ δ1 (M0 + a0 τ ). (4.52)
E1
By (4.46), (4.50) and the choice of δ3 ,

|f (t, x1 (t), u1 (t)) − f (t, x2 (t), u2 (t))|dt ≤ δ1 τ.
E2
This relation, (4.52), (4.49), (4.45), and (4.37) imply that
|I f (T1 , T2 , x2 , u2 ) − I f (T1 , T2 , x1 , u1 )| ≤ ,
U f (T, T + τ, y2 , z2 ) ≤ U f (T, T + τ, y1 , z1 ) + .
4.4 A Boundedness Property
We can easily deduce the following result.

Proposition 4.7. Assume that f ∈ Mc , τ, M1 , M2 > 0 and that
inf{U f (T, T + τ, x, y) : T ∈ R1 , x, y ∈ Rn , |x| + |y| ≥ M1 }
> sup{|U f (T, T + τ, 0, 0)| : T ∈ R1 } + 1. (4.53)
Then there exists an integer N > 2 such that:

1. For each T ∈ R1 , each integer q ≥ N and each sequence {xk }qk=0 ⊂ Rn
satisfying
{k ∈ {0, . . . , q} : |xk | ≤ M1 } = {0, q}


q−1
U f (T +kτ, T +(k+1)τ, xk , xk+1 )−U f (T +kτ, T +(k+1)τ, yk , yk+1 ) ≥M2 ,
k=0
(4.54)
where
yk = xk , k = 0, q, yk = 0, k = 1, . . . , q − 1.
2. For each T ∈ R1 , each integer q ≥ N , and each sequence {xk }qk=0 ⊂ Rn

satisfying
{k ∈ {0, . . . , q} : |xk | ≤ M1 } = {0}
relation (4.54) holds with y0 = x0 , yk = 0, k = 1, . . . , q.

3. For each T ∈ R1 , each integer q ≥ N , and each sequence {xk }qk=0 ⊂ Rn
satisfying
{k ∈ {0, . . . , q} : |xk | ≤ M1 } = {q}
relation (4.54) holds with yq = xq , yk = 0, k = 0, . . . , q − 1.

4.4 A Boundedness Property 117
Proposition 4.8. Assume that f ∈ Mc , τ, M1 , M2 > 0 and (4.53) holds. Then

there exists a number M3 > M1 + M2 such that:
1. For each T ∈ R1 , each integer q ≥ 1, and each sequence {xk }qk=0 ⊂ Rn
satisfying
max{|x0 |, |xq |} ≤ M1 , max{|xk | : k = 0, . . . , q} > M3 (4.55)
there is a sequence {yk }qk=0 ⊂ Rn such that y0 = x0 , yq = xq and (4.54)

holds.
satisfying
|x0 | ≤ M1 , max{|xk | : k = 0, . . . , q} > M3 (4.56)
there is a sequence {yk }qk=0 ⊂ Rn such that y0 = x0 and (4.54) is valid.

satisfying
|xq | ≤ M1 , max{|xk | : k = 0, . . . , q} > M3
there is a sequence {yk }qk=0 ⊂ Rn such that yq = xq and (4.54) is valid.

Proof. Let an integer N > 6 be as guaranteed in Proposition 4.7. Fix a large number
M3 > M1 + M2 .
We will prove assertion 1. Assume that T ∈ R1 , an integer q ≥ 1, and a sequence
{xk }qk=0 ⊂ Rn satisfies (4.55). Then there is j ∈ {0, . . . , q} such that |xj | > M3 .
Set
i1 = max{k ∈ {0, . . . , j} : |xk | ≤ M1 }, i2 = min{k ∈ {j, . . . , q} : |xk | ≤ M1 }.
If i2 − i1 ≥ N , then by the validity of assertion 1 follows from the definition of N

and Proposition 4.7.
If i2 − i1 < N , then we set
yk = xk , k ∈ {0, . . . , i1 } ∪ {i2 , . . . , q}, yk = 0, k = i1 + 1, . . . , i2 − 1 (4.57)
and it is easy to see that (4.54) holds when the constant M3 is large enough.
We will prove assertion 2. Assume that T ∈ R1 , and integer q ≥ 1 and a
sequence {xk }qk=0 ⊂ Rn satisfies (4.56). Then there is j ∈ {1, . . . , q} such that
|xj | > M3 . Set
i1 = max{k ∈ {0, . . . , j} : |xk | ≤ M1 }.

If |xk | > M1 for k = j, . . . , q, then we set
yi = xi , i = 0, . . . , i1 , yi = 0, i = i1 + 1, . . . , q.
Otherwise we set
i2 = min{k ∈ {j, . . . , q} : |xk | ≤ M1 }
and define {yk }qk=0 by (4.57). It is easy to verify that in both cases (4.54) holds.
We will prove assertion 3. Assume that T ∈ R1 , an integer q ≥ 1, and a sequence
{xk }qk=0 ⊂ Rn satisfies
|xq | ≤ M1 , max{|xk | : k = 0, . . . , q} > M3 .
Then there is j ∈ {0, . . . , q − 1} such that |xj | > M3 . Set
i2 = min{k ∈ {j, . . . , q} : |xk | ≤ M1 }.
If |xk | > M1 for k = 0, . . . , j, then we set
yi = xi , i = i2 , . . . , q, yi = 0, i = 0, . . . , i2 − 1.
Otherwise we set
i1 = max{k ∈ {0, . . . , j} : |xk | ≤ M1 }
and define {yk }qk=0 by (4.57). It is easy to verify that in both cases (4.54) holds.
4.5 The Existence and Structure of Solutions
Proposition 4.9. Let f ∈ Mc . Then f satisfies assumption (B).

Proof. By Propositions 4.2 and 4.5 and (LSC) property for each integer N ≥ 1
xN : [−N, N ] → Rn , uN : [−N, N ] → Rm
such that
I f (−N, N, xN , uN ) ≤ U f (−N, N, y, z) for each y, z ∈ Rn . (4.58)

4.5 The Existence and Structure of Solutions 119
Then there exists a number M1 > 0 which satisfies (4.53) with τ = 1. Let a number
M3 > M1 + 1 be as guaranteed in Proposition 4.8 with τ = 1, M2 = 1. We will
show that
sup{|xN (i)| : an integer i ∈ [−N, N ], N = 1, 2, . . . } ≤ M3 . (4.59)
Let N ≥ 1 be an integer. By (4.58) and (4.53) which holds with τ = 1 there exists
an integer q ∈ [−N, N ] such that
|xN (q)| ≤ M1 .
Relation (4.59) follows from this relation, (4.58), Proposition 4.8 which holds with
τ = 1, M2 = 1 and the choice of M3 .
By (4.58), (4.59) and Propositions 2.7 and 4.5,
sup{|xN (t)| : t ∈ [−N, N ], N = 1, 2, . . . } < ∞. (4.60)
By (4.58), (4.60), and Proposition 4.5 there exists a number M4 > 0 such that
I f (−i, i, xp , up ) ≤ I f (−i, i, xi , ui ) + M4 (4.61)
for each pair of integers i ≥ 1 and p ≥ i. Together with assumption (A) this implies
that for each integer j the sequence
{I f (j, j + 1, xp , up ) : p ≥ |j| + 1}
is bounded. Therefore by Proposition 4.2 and (LSC) property there exists a

subsequence {(xNk , uNk }∞
k=1 and a trajectory-control pair
xf : R1 → Rn , uf : R1 → Rm
such that
xNk (t) → xf (t) as k → ∞ for any t ∈ R1 (4.62)
and
I f (j, j + 1, xf , uf ) ≤ lim inf I f (j, j + 1, xNk , uNk ). (4.63)

k→∞
It follows from (4.60)–(4.63) that
sup{|xf (t)| : t ∈ R1 } < ∞

and
I f (−i, i, xf , uf ) ≤ I f (−i, i, xi , ui ) + M4 (4.64)
for each integer i ≥ 1.

By (4.63), (4.58), (4.62), and Proposition 4.6,
I f (i, j, xf , uf ) ≤ lim inf I f (i, j, xNk , uNk )

k→∞
≤ lim inf U f (i, j, xNk (i), xNk (j)) = U f (i, j, xf (i), xf (j)). (4.65)
k→∞
Therefore f satisfies assumption B(i). It follows from (4.65), (4.64), and Proposi-
tion 4.5 that f satisfies assumption B(ii).
We show that f satisfies assumption B(iii). Let S1 > 0. Set c = 1 and
c0 = sup{|xf (t)| : t ∈ R1 } + S1 + 2. (4.66)
By Proposition 4.5 there exists a number
c1 > sup{U f (T, T + 1, y, z) :
T ∈ R1 , y, z ∈ Rn , |y|, |z| ≤ c0 + 1}. (4.67)
Choose a number
S2 > M4 + 2a0 + 2c1 + 4 (4.68)
(recall a0 in assumption (A)).

Suppose that T1 ∈ R1 , T2 ≥ T1 + 1 and a trajectory-control pair
x : [T1 , T2 ] → Rn , u : [T1 , T2 ] → Rm
satisfies
|x(Ti )| ≤ S1 , i = 1, 2. (4.69)
Choose an integer
N > |T1 | + |T2 | + 4. (4.70)
By Proposition 4.5 there exists trajectory-control pair
y : [−N, N ] → Rn , u : [−N, N ] → Rm
such that
y(t) = xf (t), v(t) = uf (t), t ∈ [−N, T1 − 1] ∪ [T2 + 1, N ],

y(t) = x(t), v(t) = u(t), t ∈ [T1 , T2 ],
I f (τ, τ + 1, y, v) = U f (τ, τ + 1, y(τ ), y(τ + 1)) + 1, τ = T1 − 1, T2 . (4.71)
It follows from (4.71), (4.64), and (4.58) that
I f (−N, N, xf , uf ) − I f (−N, N, y, v)
≤ M4 + I f (−N, N, xN , uN ) − I f (−N, N, y, v) ≤ M4 . (4.72)
On the other hand by (4.71), assumption (A), (4.69), (4.66), (4.67),
I f (−N, N, xf , uf ) − I f (−N, N, y, v)
≥ I f (T1 , T2 , xf , uf ) − I f (T1 , T2 , x, u) − 2a0 − 2c1 − 2.
I f (T1 , T2 , xf , uf ) ≤ I f (T1 , T2 , x, u) + S2 .
Therefore f satisfies assumption B(iii). Set bf = 1. It follows from assumption

B(i), Proposition 4.6, and (4.64) that f satisfies assumption B(iv). Proposition 4.9 is
proved.
Proposition 4.10. Mc is a closed subset of M.
Proof. Assume that fi ∈ Mc , i = 1, 2, . . . and fi → f ∈ M as i → ∞. It is
sufficient to show that f ∈ Mc . Clearly, f is a continuous function. We show that
f satisfies assumption (D). It is easy to see that f satisfies assumptions D(i), D(iv),
and D(v). We show that f satisfies assumption D(ii).
Let K > 0. There exists an integer j ≥ 1 such that
(f, fj ) ∈ E(K, 2−1 , 2)
(see (2.7)). Therefore
|f (t, x, u)| + 1 ≥ 2−1 |fj (t, x, u)| + 2−1 , (t, x, u) ∈ Rn+m+1 , |x| ≤ K. (4.73)
Since fj ∈ Mc satisfies assumption D(ii) there exists a constant ak,j > 0 and an
increasing function
ψk,j : [0, ∞) → [0, ∞)

such that
ψk,j (t) → ∞ as t → ∞,
fj (t, x, u) ≥ ψk,j (|u|)|u| − ak,j , (t, x, u) ∈ Rn+m+1 , |x| ≤ K. (4.74)
It follows from (4.73) and (4.74) that
|f (t, x, u)| ≥ 2−1 ψk,j (|u|)|u| − 2−1 ak,j − 2−1 , (t, x, u) ∈ Rn+m+1 , |x| ≤ K.
(4.75)
There exists K0 ≥ 1 such that
ψk,j (K0 ) ≥ 8a0 + 8ak,j + 8. (4.76)
By (4.75), (4.76), and assumption (A) for each (t, x, u) ∈ Rn+m+1 satisfying |x| ≤
K, |u| ≥ K0 ,
f (t, x, u) ≥ 0 and f (t, x, u) ≥ 2−1 ψk,j (|u|)|u| − ak,j − 1.
It is easy to see that for each (t, x, u) ∈ Rn+m+1 satisfying |x| ≤ K,
f (t, x, u) ≥ 2−1 ψk,j (|u|)|u| − ak,j − 1 − ψk,j (|K0 |)|K0 | − a0 .
Therefore f satisfies assumption D(ii).

We show that f satisfies assumption D(iii).
Let M, > 0. Fix a number λ > 1 such that
λ2 − 1 < 8−1 . (4.77)
Clearly, there exists an integer j ≥ 1 such that
(f, fi ) ∈ E(M, , λ) for each integer i ≥ j. (4.78)
Since f, fj satisfy assumption D(ii) there exist a number aM > 0 and an increasing
function
ψM : [0, ∞) → [0, ∞)
such that
ψM (t) → ∞ as t → ∞,
fj (t, x, u), f (t, x, u) ≥ ψM (|u|)|u| − aM , (t, x, u) ∈ Rn+m+1 , |x| ≤ M.
(4.79)
By (4.77) and the properties of ψM there exists a number Γ0 such that
Γ0 > 1,
ψM (Γ0 ) ≥ 2a0 + 2aM ,
λ2 (1 + 2ψM (Γ0 )−1 )2 − 1 < 8−1 . (4.80)
Fix a positive number 1 which satisfies
81 [λ(1 + 2ψM (Γ0 )−1 )]2 < . (4.81)
By Proposition 4.1 there exist numbers Γ, δ > 0 such that
Γ > Γ0 ,
|fj (t, x1 , u1 ) − fj (t, x2 , u2 )| ≤ 1 min{fj (t, x1 , u1 ), fj (t, x2 , u2 )} (4.82)
|xi | ≤ M, |ui | ≥ Γ, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ. (4.83)
Assume that t ∈ R1 and u1 , u2 ∈ Rm , x1 , x2 ∈ Rn satisfy (4.83). It follows

from the choice of Γ, δ that (4.82) holds. By (4.78) and (4.83),
(|f (t, xi , ui )| + 1)(|fj (t, xi , ui )| + 1)−1 ∈ [λ−1 , λ], i = 1, 2. (4.84)
Relations (4.83), (4.79), and (4.80) imply that
min{fj (t, xi , ui ), f (t, xi , ui )} ≥ 2−1 ψM (γ0 ), i = 1, 2.
f (t, xi , ui )fj (t, xi , ui )−1

∈ [λ(1 + 2ψM (Γ0 )−1 )−1 , λ(1 + 2ψM (Γ0 )−1 )], i = 1, 2. (4.85)
f (t, x1 , u1 ) ≥ f (t, x2 , u2 ). (4.86)
It follows from (4.85), (4.82), (4.81) and (4.80) that
f (t, x1 , u1 ) − f (t, x2 , u2 )
−1
≤ λ(1 + 2ψM (Γ0 ) )fj (t, x1 , u1 ) − (λ(1 + 2ψM (Γ0 )−1 ))−1 fj (t, x2 , u2 )
= λ(1 + 2ψM (Γ0 )−1 )[fj (t, x1 , u1 ) − fj (t, x2 , u2 )]
+fj (t, x2 , u2 )[λ(1 + 2ψM (Γ0 )−1 ) − (λ(1 + 2ψM (Γ0 )−1 ))−1 ]
≤ 1 λ(1 + 2ψM (Γ0 )−1 )fj (t, x2 , u2 )
+fj (t, x2 , u2 )[λ(1 + 2ψM (Γ0 )−1 ) − (λ(1 + 2ψM (Γ0 )−1 ))−1 ]
≤ 1 [λ(1 + 2ψM (Γ0 )−1 )]2 fj (t, x2 , u2 )
+f (t, x2 , u2 )[λ2 (1 + 2ψM (Γ0 )−1 )2 − 1] ≤ f (t, x2 , u2 ).
Therefore the function f satisfies assumption D(iii). This completes the proof of
Proposition 4.10.
Assume that f ∈ Mc . By Proposition 4.9 f satisfies assumption (B). Let a
xf : R1 → Rn , uf : R1 → Rm
be as guaranteed by assumption (B).

For each r > 0 we define fr : Rn+m+1 → R1 as follows:
fr (t, x, u) = f (t, x, u) + r min{|x − xf (t)|, 1}, (t, x, u) ∈ Rn+m+1 .
It is easy to see that fr ∈ Mc for each r > 0.

By Proposition 4.10 Mc is a closed subset of M. It follows from Proposition 4.9
that Mc ⊂ Mreg . Moreover fr ∈ Mc for each f ∈ Mc and each r > 0.
It follows from the results of Chap. 2 that there exists a set F ⊂ Mc which
is a countable intersection of open everywhere dense sets in Mc and for which
Theorems 2.2, 2.3 and 2.5 hold.
Theorem 3.5 and Propositions 4.2 and 4.9 imply the following result.
Theorem 4.11. Assume that f ∈ Mc has the turnpike property, τ ∈ R1 , and
x : [τ, ∞) → Rn , u : [τ, ∞) → Rm
is an (f )-good trajectory-control pair. Then there exists an (f )-overtaking optimal-

trajectory pair
x∗ : [τ, ∞) → Rn , u∗ : [τ, ∞) → Rm
such that x∗ (τ ) = x(τ ).

References
1. Anderson, B.D.O., Moore, J.B.: Linear Optimal Control. Prentice-Hall, Englewood Cliffs
(1971)
2. Arkin, V.I., Evstigneev, I.V.: Stochastic Models of Control and Economic Dynamics.
Academic, London (1987)
3. Aseev, S.M., Besov, K.O., Kryazhimskii, A.V.: Russ. Math. Surv. 67, 195–253 (2012)
4. Aseev, S.M., Kryazhimskiy, A.V.: SIAM J. Control Optim. 43, 1094–1119 (2004)
5. Aseev, S.M., Veliov, V.M.: Dyn. Contin., Discret. Impulsive Syst. Ser. B: Appl. Algorithms 19,
43–63 (2012)
6. Aubin, J.P., Ekeland, I.: Applied Nonlinear Analysis. Wiley Interscience, New York (1984)
7. Aubry, S., Le Daeron, P.Y.: Physica D 8, 381–422 (1983)
8. Baumeister, J., Leitao, A., Silva, G.N.: Syst. Control Lett. 56, 188–196 (2007)
9. Blot, J., Cartigny, P.: J. Optim. Theory Appl. 106, 411–419 (2000)
10. Blot, J., Hayek, N.: ESAIM Control Optim. Calc. Var. 5, 279–292 (2000)
11. Blot, J., Michel, P.: Appl. Math. Lett. 16, 71–78 (2003)
12. Carlson, D.A., Haurie, A., Leizarowitz, A.: Infinite Horizon Optimal Control. Springer, Berlin
(1991)
13. Cartigny, P., Michel, P.: Automatica J. IFAC 39, 1007–1010 (2003)
14. Coleman, B.D., Marcus, M., Mizel, V.J.: Arch. Rational Mech. Anal. 117, 321–347 (1992)
15. Evstigneev, I.V., Flam, S.D.: Set-Valued Anal. 6, 61–81 (1998)
16. Gaitsgory, V., Rossomakhine, S., Thatcher, N.: Dyn. Contin., Discret. Impulsive Syst. Ser. B:
Appl. Algorithms 19, 43–63 (2012)
17. Gale, D.: Rev. Econ. Stud. 34, 1–18 (1967)
18. Guo, X., Hernandez-Lerma, O.: Bernoulli 11, 1009–1029 (2005)
19. Hammond, P.J.: Consistent Planning and Intertemporal Welfare Economics. University of
Cambridge, Cambridge (1974)
20. Hammond, P.J.: Rev. Econ. Stud. 42, 1–14 (1975)
21. Hammond, P.J., Mirrlees, J.A.: Models of Economic Growth, pp. 283–299. Wiley, New York
(1973)
22. Hayek, N.: Optimization 60, 509–529 (2011)
23. Jasso-Fuentes, H., Hernandez-Lerma, O.: Appl. Math. Optim. 57, 349–369 (2008)
24. Leizarowitz, A.: Appl. Math. Optim. 13, 19–43 (1985)
25. Leizarowitz, A.: Appl. Math. Optim. 14, 155–171 (1986)
26. Leizarowitz, A., Mizel, V.J.: Arch. Rational Mech. Anal. 106, 161–194 (1989)
27. Lykina, V., Pickenhain, S., Wagner, M.: J. Math. Anal. Appl. 340, 498–510 (2008)
28. Makarov, V.L., Rubinov, A.M.: Mathematical Theory of Economic Dynamics and Equilibria.
Springer, New York (1977)

SpringerBriefs in Optimization, DOI 10.1007/978-3-319-01240-7,
126 References
29. Malinowska, A.B., Martins, N., Torres, D.F.M.: Optim. Lett. 5, 41–53 (2011)
30. Marcus, M., Zaslavski, A.J.: Ann. Inst. H. Poincare, Anal. Non Lineare 16, 593–629 (1999)
31. Marcus, M., Zaslavski, A.J.: Ann. Inst. H. Poincare, Anal. Non Lineare 19, 343–370 (2002)
32. Martins, N., Torres, D.F.M.: J. Optim. Theory Appl. 155(2), 453–476 (2012)
33. McKenzie, L.W.: Econometrica 44, 841–866 (1976)
34. Mordukhovich, B.S.: Automat. Remote Control 50, 1333–1340 (1990)
35. Mordukhovich, B.S.: Appl. Anal. 90, 1075–1109 (2011)
36. Mordukhovich, B.S., Shvartsman, I.: Optimal Control, Stabilization and Nonsmooth Analysis.
Lecture Notes in Control and Information Sciences, pp. 121–132. Springer, New York (2004)
37. Moser, J.: Ann. Inst. H. Poincare, Anal. Nonlineare 3, 229–272 (1986)
38. Ocana, E., Cartigny, P.: SIAM J. Control Optim. 50, 2573–2587 (2012)
39. Ocana, E., Cartigny, P., Loisel, P.: J. Nonlinear Convex Anal. 10, 157–176 (2009)
40. Pickenhain, S., Lykina, V., Wagner, M.: Control Cybernet 37, 451–468 (2008)
41. Rubinov, A.M.: J. Soviet Math. 26, 1975–2012 (1984)
42. Samuelson, P.A.: Am. Econ. Rev. 55, 486–496 (1965)
43. von Weizsacker, C.C.: Rev. Econ. Stud. 32, 85–104 (1965)
44. Zaslavski, A.J.: Math. USSR Izvestiya 29, 323–354 (1987)
45. Zaslavski, A.J.: SIAM J. Control Optim. 33, 1643–1660 (1995)
46. Zaslavski, A.J.: SIAM J. Control Optim. 33, 1661–1686 (1995)
47. Zaslavski, A.J.: Nonlinear Anal. 27, 895–931 (1996)
48. Zaslavski, A.J.: Abstr. Appl. Anal. 3, 265–292 (1998)
49. Zaslavski, A.J.: Nonlinear Anal. 42, 1465–1498 (2000)
50. Zaslavski, A.J.: Turnpike Properties in the Calculus of Variations and Optimal Control.
Springer, New York (2006)
51. Zaslavski, A.J.: Optimization on Metric and Normed Spaces. Springer, New York (2010)
52. Zaslavski, A.J.: Portugal. Math. 68, 239–257 (2011)
53. Zaslavski, A.J., Leizarowitz, A.: Math. Oper. Res. 22, 726–746 (1997)
Index
A Infinite horizon problem, 1

Absolutely continuous function, 5 Integral functional, 12
Agreeable trajectory-control pair, 91 Integrand, 2
Approximate solution, 4
L
B
Lebesgue integrable function, 12
Borelian function, 12
Lebesgue measurable function, 12
Borel measurable set, 11
(LSC) property, 79
C
Complete metric space, 1 M
Complete uniform space, 3 Minimal solution, 2
Control constraint, 11
Control function, 11
Control system, 9 N
Neumann path, 6
D
Differential equation, 11
O
Overtaking optimality criterion, 1
E
Euclidean norm, 2
Euclidean space, 2 T
Topological subspace, 15
Trajectory-control pair, 12
G
Turnpike property, 6
Good function, 4
Good point, 81
U
I Uniformity, 3
Increasing function, 2 Uniform space, 3

SpringerBriefs in Optimization, DOI 10.1007/978-3-319-01240-7,

Alexander J. Zaslavski - Structure of Approximate Solutions of Optimal Control Problems-Springer (2013)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Alexander J. Zaslavski - Structure of Approximate Solutions of Optimal Control Problems-Springer (2013)

Uploaded by

Copyright:

Available Formats

SPRINGER BRIEFS IN OPTIMIZATION

SpringerBriefs in Optimization showcases algorithmic and theoretical tech-

For further volumes:

ISSN 2190-8354 ISSN 2191-575X (electronic)

Mathematics Subject Classification: 49J15, 90C31, 54E35, 54E50, 54E52

© Alexander J. Zaslavski 2013

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

In this book we study the structure of approximate solutions of optimal control

Haifa, Israel Alexander J. Zaslavski

1.1 Infinite Horizon Variational Problems

as T grows to infinity where a function x : [0, ∞) → Rn is locally absolutely

for any a. c. function y : [0, ∞) → Rn satisfying y(0) = x(0).

A.J. Zaslavski, Structure of Approximate Solutions of Optimal Control Problems, 1

Let I be either [0, ∞) or (−∞, ∞). We say that an a. c. function x : I → Rn is

for each T1 ∈ I, each T2 > T1 , and each a. c. function y : [T1 , T2 ] → Rn which

f (t, x, u) ≥ max{ψ(|x|), ψ(|u|)|u|} − a;

A(iv) For each M, > 0 there exist Γ, δ > 0 such that

|f (t, x1 , u1 ) − f (t, x2 , u2 )| ≤ max{f (t, x1 , u1 ), f (t, x2 , u2 )}

for each t ∈ [0, ∞), each u1 , u2 ∈ Rn and each x1 , x2 ∈ Rn which satisfy

|xi | ≤ M, |ui | ≥ Γ, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ;

A(v) For each M, > 0 there exists δ > 0 such that

for each t ∈ [0, ∞), each u1 , u2 ∈ Rn and each x1 , x2 ∈ Rn which satisfy

|xi |, |ui | ≤ M, i = 1, 2, max{|x1 − x2 |, |u1 − u2 |} ≤ δ.

Assumptions A(i)–A(v) were discussed in [50] with examples of integrands for

sup{|f (t, 0, 0)| : t ∈ [0, ∞)} < ∞

and there exists an increasing function ψ0 : [0, ∞) → [0, ∞) such that

sup{|∂f /∂x(t, x, u)|, |∂f /∂u(t, x, u)|} ≤ ψ0 (|x|)(1 + ψ(|u|)|u|)

for each t ∈ [0, ∞), x, u ∈ Rn .

E(N, , λ) = {(f, g) ∈ M × M : |f (t, x, u) − g(t, x, u)| ≤

where N > 0, > 0, λ > 1.

where f ∈ M, 0 ≤ T1 < T2 < ∞ and x : [T1 , T2 ] → Rn is an a. c. function.

U f (T1 , T2 , a, b) = inf{I f (T1 , T2 , x) : x : [T1 , T2 ] → Rn

It is easy to see that −∞ < U f (T1 , T2 , a, b) < ∞ for each f ∈ M, each a, b ∈ Rn

Let f ∈ M. We say that an a. c. function x : [0, ∞) → Rn is an (f )-good

inf{I f (0, T, y) − I f (0, T, x) : T ∈ (0, ∞)} > −∞.

sup{|y(t)| : t ∈ [0, ∞)} < ∞.

2. For each f ∈ M and each number M > 0 there exist a neighborhood U of f

sup{|Z g (t)| : t ∈ [0, ∞)} ≤ Q

for each g ∈ U and each z ∈ Rn satisfying |z| ≤ M .

4. For each f ∈ M and each z ∈ Rn the function Z f : [0, ∞) → Rn is an

lim sup |x(t)| < M.

v(T1 ) = x, v(T2 ) = y, I g (T1 , T2 , v) ≤ U g (T1 , T2 , x, y) + M2 ,

(ii) If x ∈ Rn satisfies |x| ≤ M1 and if an a. c. function v : [T1 , T2 ] → Rn satisfies

v(T1 ) = x, I g (T1 , T2 , v) ≤ σ g (T1 , T2 , x) + M2 ,

1.2 The Turnpike Phenomenon

In Chap. 2 of [50] and in [49] we studied the structure of approximate solutions of

z : [T1 , T2 ] → Rn is an absolutely continuous function,

where T1 ≥ 0, T2 > T1 , x, y ∈ Rn , and f : [0, ∞) × Rn × Rn → R1 belongs to

for each a. c. function z : [T1 , T2 ] → Rn satisfying z(T1 ) = x, z(T2 ) = y.

|u(t) − Xf (t)| ≤ for all t ∈ [T1 + τ1 , T2 − τ2 ],

where τ1 , τ2 ∈ [0, L].

More precisely, in Chap. 2 of [50] we studied the turnpike properties for

1.3 Structure of Solutions of Variational Problems

Let a > 0 be a constant and let ψ : [0, ∞) → [0, ∞) be an increasing function

Es () = {(f, g) ∈ M × M : |f (t, x, u) − g(t, x, u)| ≤

E(N, , λ) = {(f, g) ∈ M × M : |f (t, x, u) − g(t, x, u)| ≤

for each t ∈ [0, ∞) and each x, u ∈ Rn satisfying |x|, |u| ≤ N,

(|f (t, x, u)| + 1)(|g(t, x, u)| + 1)−1 ∈ [λ−1 , λ]

for each t ∈ [0, ∞) and each x, u ∈ Rn satisfying |x| ≤ N },

|v2 (t) − v1 (t)| → 0 as t → ∞.

2. For each g ∈ F and each y ∈ Rn there exists a (g)-overtaking optimal function

|v(t) − Y (t)| ≤ for all large t.