You are on page 1of 343

Selected Titles in This Series

153 S. Kh. Aranson, G. R. Belitsky, and E. V. Zhuzhoma, Introduction to the qualitative


theory of dynamical systems on surfaces, 1996
152 R. S. Ismagilov, Representations of infinite-dimensional groups, 1996
151 S. Yu. Slavyanov, Asymptotic solutions of the one-dimensional Schrodinger equation,
1996
150 B. Ya. Levin, Lectures on entire functions, 1996
149 Takashi Sakai, Riemannian geometry, 1996
148 Vladimir I. Piterbarg, Asymptotic methods in the theory of Gaussian processes and
fields, 1996
147 S. G. Gindikin and L. R. Volevich, Mixed problem for partial differential equations with
quasihomogeneous principal part, 1996
146 L. Ya. Adrianova, Introduction to linear systems of differential equations, 1995
145 A. N. Andrianov and V. G. Zhuravlev, Modular forms and Hecke operators, 1995
144 O. V. Troshkin, Nontraditional methods in mathematical hydrodynamics, 1995
143 V. A. Malyshev and R. A. Minlos, Linear infinite-particle operators, 1995
142 N. V. Krylov, Introduction to the theory of diffusion processes, 1995
141 A. A. Davydov, Qualitative theory of control systems, 1994
140 Aizik I. Volpert, Vitaly A. Volpert, and Vladimir A. Volpert, Traveling wave solutions of
parabolic systems, 1994
139 I. V. Skrypnik, Methods for analysis of nonlinear elliptic boundary value problems, 1994
138 Yu. P. Razmyslov, Identities of algebras and their representations, 1994
137 F. I. Karpelevich and A. Ya. Kreinin, Heavy traffic limits for multiphase queues, 1994
136 Masayoshi Miyanishi, Algebraic geometry, 1994
135 Masaru Takeuchi, Modern spherical functions, 1994
134 V. V. Prasolov, Problems and theorems in linear algebra, 1994
133 P. I. Naumkin and I. A. Shishmarev, Nonlinear nonlocal equations in the theory of waves,
1994
132 Hajime Urakawa, Calculus of variations and harmonic maps, 1993
131 V. V. Sharko, Functions on manifolds: Algebraic and topological aspects, 1993
130 V. V. Vershinin, Cobordisms and spectral sequences, 1993
129 Mitsuo Morimoto, An introduction to Sato's hyperfunctions, 1993
128 V. P. Orevkov, Complexity of proofs and their transformations in axiomatic theories,
1993
127 F. L. Zak, Tangents and secants of algebraic varieties, 1993
126 M. L. Agranovskil, Invariant function spaces on homogeneous manifolds of Lie groups
and applications, 1993
125 Masayoshi Nagata, Theory of commutative fields, 1993
124 Masahisa Adachi, Embeddings and immersions, 1993
123 M. A. Akivis and B. A. Rosenfeld, Elie Cartan (1869-1951), 1993
122 Zhang Guan-Hou, Theory of entire and meromorphic functions: Deficient and asymptotic
values and singular directions, 1993
121 I. B. Fesenko and S. V. Vostokov, Local fields and their extensions: A constructive
approach, 1993
120 Takeyuki Hida and Masuyuki Hitsuda, Gaussian processes, 1993
119 M. V. Karasev and V. P. Maslov, Nonlinear Poisson brackets. Geometry and quantization,
1993
118 Kenkichi Iwasawa, Algebraic functions, 1993
(Continued in the back of this publication)
Introduction to the
Qualitative Theory of
Dynamical Systems
on Surfaces
Translations of
MATHEMATICAL
MONOGRAPHS
Volume 153

Introduction to the
Qualitative Theory of
Dynamical Systems
on Surfaces
S. Kh. Aranson
G. R. Belitsky
E. V. Zhuzhoma

TPHTOI MH

a American Mathematical Society


p
y Providence, Rhode Is l and
C. X. ApaxcoH, T. P. Bernu cMH, E. B. IKy)KoMa
BBE1[EHJIE
B KALIECTBEHHYIO TEOPJIIO JJJ4HAMJ4LIECKFIX CJICTEM
HA HOBEPXHOCT,fJX
Translated by H. H. McFaden from an original Russian manuscript.

EDITORIAL COMMITTEE
AMS Subcommittee
Robert D. MacPherson
Grigorii A. Margulis
James D. Stasheff (Chair)
ASL Subcommittee Steffen Lempp (Chair)
IMS Subcommittee Mark I. Freidlin (Chair)
1991 Mathematics Subject Classification. Primary 58-02, 58F25; Secondary 58F10,
58F21, 34C28, 58F18, 58F36, 34C35, 34D30, 57R30, 54H20.
ABSTRACT. This book is an introduction to the qualitative theory of dynamical systems on man-
ifolds of low dimension (on the circle and on surfaces). Along with classical results, it reflects the
most significant achievements in this area obtained in recent times by Russian and Western math-
ematicians whose work has not yet appeared in the monographic literature. The main emphasis
is put on global problems in the qualitative theory of flows on surfaces.
The reader of this book need be familiar only with basic courses in differential equations and
smooth manifolds. All the main definitions and notions required for understanding the contents
are given in the text.
The book will be useful to mathematicians working in dynamical systems and differential equa-
tions, and geometry, and to specialists with a mathematical background who are studying dynam-
ical processes: mechanical engineers, physicists, biologists, and so on.

Library of Congress Cataloging-in-Publication Data


Aranson, S. Kh.
[Vvedenie v kachestvennuiu teoriiu dinamicheskikh sistem na poverkhnostiakh. English]
Introduction to the qualitative theory of dynamical systems on surfaces / S. Kh. Aranson,
G. R. Belitsky, E. V. Zhuzhoma; [translator H. H. McFaden].
p. cm.-(Translations of mathematical monographs, ISSN 0065-9282; v. 153)
Includes bibliographical references (p. - ).
ISBN 0-8218-0369-7 (alk. paper)
1. Flows (Differentiable dynamical systems) I. Belitsk, Genrikh Ruvimovich. II. Zhuzhoma,
E. V. III. Title. IV. Series.
QA614.82.A7313 1996
514'.74-dc20 96-19197
CIP

Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting
for them, are permitted to make fair use of the material, such as to copy a chapter for use
in teaching or research. Permission is granted to quote brief passages from this publication in
reviews, provided the customary acknowledgment of the source is given.
Republication, systematic copying, or multiple reproduction of any material in this publication
(including abstracts) is permitted only under license from the American Mathematical Society.
Requests for such permission should be addressed to the Assistant to the Publisher, American
Mathematical Society, P. O. Box 6248, Providence, Rhode Island 02940-6248. Requests can also
be made by e-mail to reprint-permisslon@ams.org.
© 1996 by the American Mathematical Society. All rights reserved.
The American Mathematical Society retains all rights
except those granted to the United States Government.
Printed in the United States of America.
Q The paper used in this book is acid-free and falls within the guidelines
established to ensure permanence and durability.
10987654321 010099989796
Contents

Foreword xiii
Chapter 1. Dynamical Systems on Surfaces 1
§1. Flows and vector fields 1
1.1. Definitions and examples 1
1.2. Connection between flows and vector fields 2
1.3. Vector fields and systems of differential equations 3
1.4. Diffeomorphisms of vector fields 3
§2. Main ways of specifying flows on surfaces 3
2.1. The projection method 4
2.2. Systems of differential equations in local charts 6
2.3. Specification of a flow with the help of a universal covering 7
2.3.1. Transformation groups 8
2.3.2. Flows on the torus 8
2.3.3. Flows on closed orientable surfaces of genus > 1 10
2.4. Specification of a flow with the help of a branched covering 12
2.4.1. Definition of a branched covering 12
2.4.2. Covering flows 12
2.4.3. Construction of transitive flows 13
2.5. The pasting method 15
2.6. Suspensions 16
2.6.1. The suspension over a homeomorphism of the circle 16
2.6.2. The suspension over an exchange of open intervals 17
2.7. Whitney's theorem 21
2.7.1. The theorem on continuous dependence on the initial
conditions 21
2.7.2. The rectification theorem 22
2.7.3. Orientability 22
§3. Examples of flows with limit set of Cantor type 23
3.1. The example of Denj oy 24
3.2. Cherry flows 27
3.3. An example of a flow on the sphere 31
§4. The Poincare index theory 34
4.1. Contact-free segments and cycles 34
4.2. The index of a nondegenerate cycle in a simply connected do-
main 35
4.3. The index of an isolated equilibrium state 36
4.4. The Euler characteristic and the Poincare index 38

vii
viii CONTENTS

4.5. Connection between the index and the orientability of foliations 38


4.6. An example of a foliation that is locally but not globally ori-
entable 40
Remark. About a result of El'sgol'ts 43

Chapter 2. Structure of Limit Sets 45


§1. Initial concepts and results 45
1.1. The long flow tube theorem, and construction of a contact-free
cycle 45
1.2. The Poincare mapping 47
1.3. The limit sets 48
1.4. Minimal sets 51
1.5. Nonwandering points 52
§2. The theorems of Maier and Cherry 53
2.1. Definitions of recurrence 53
2.2. The absence of nontrivial recurrent semitrajectories on certain
surfaces 55
2.3. The Cherry theorem on the closure of a recurrent semitrajectory 57
2.4. The Maier criterion for recurrence 61
2.5. The Maier estimate for the number of independent nontrivial
recurrent semitrajectories 65
§3. The Poincare-Bendixson theory 66
3.1. The Poincare-Bendixson theorem 67
3.2. Bendixson extensions 67
3.3. Separatrices of an equilibrium state 70
3.4. The Bendixson theorem on equilibrium states 74
3.5. One-sided contours 77
3.6. Lemmas on the Poincare mapping 78
3.7. Description of quasiminimal sets 83
3.8. Catalogue of limit sets 85
3.9. Catalogue of minimal sets 87
§4. Quasiminimal sets 87
4.1. An estimate of the number of quasiminimal sets 88
Remarks. The estimates of Aranson, Markley, and Levitt 89
4.2. A family of special contact-free cycles 89
4.3. Partition of a contact-free cycle 90
4.4. The Gardiner types of partition elements 91
4.5. The structure theorem 96

Chapter 3. Topological Structure of a Flow 101


§1. Basic concepts of the qualitative theory 101
1.1. Topological and smooth equivalence 101
1.2. Invariants 103
1.3. Classification 104
§2. Decomposition of a flow 104
2.1. Characteristic curves of a quasiminimal set 104
2.2. Periodic elements of a partition 106
2.3. Criterion for a flow to be irreducible 109
CONTENTS ix

2.4. Decomposition of a flow into irreducible flows and flows without


nontrivial recurrent semitrajectories 110
2.5. The Levitt decomposition 112
Remark 1. The canonical decomposition 116
Remark 2. The center of a flow 117
§3. The structure of an irreducible flow 117
3.1. Blowing-down and blowing-up operations 117
3.2. Irreducible flows on the torus 122
§4. Flows without nontrivial recurrent trajectories 125
4.1. Singular trajectories 125
4.2. Cells 127
4.3. Topology of cells 127
4.4. Structure of a flow in cells 131
4.5. Smooth models 132
4.6. Morse-Smale flows 136
4.7. Cells of Morse-Smale flows 139
§5. The space of flows 139
5.1. The metric in the space of flows 139
5.2. The concepts of structural stability and the degree of structural
instability 140
5.3. The space of structurally stable flows 141
5.4. Flows of the first degree of structural instability 143
5.5. On denseness of flows of the first degree of structural instability
in the space of structurally unstable flows 146

Chapter 4. Local Structure of Dynamical Systems 147


§1. Dynamical systems on the line 148
1.1. Linearization of a diffeomorphism 148
1.2. Lemmas on functional equations 149
1.3. Proof of Theorem 1.1 153
1.4. Flows on the line 154
§2. Topological linearization on the plane 156
2.1. Formulation of the theorem 156
2.2. Proof of the theorem 156
§3. Invariant curves of local diffeomorphisms 157
3.1. Invariant curves of a node 158
3.2. Invariant curves of a saddle point 159
§4. C1-linearization on the plane 160
§5. Formal transformations 163
5.1. Formal mappings 163
5.2. Conj ugacy of formal mappings 164
5.3. Formal vector fields and flows 166
§6. Smooth normal forms 168
6.1. Normal forms with flat residual 168
6.2. Smooth normal forms of a node 169
6.3. Smooth normalization in a neighborhood of a saddle point 172
6.4. The Sternberg-Chern theorem 176
6.5. The smoothness class as an obstacle to smooth normalization 178
§ 7. Local normal forms of two-dimensional flows 179
x CONTENTS

7.1. Topological and C1-linearization 179


7.2. Invariant curves of a flow 180
7.3. Smooth normal forms 181
7.4. The correspondence mapping at a saddle point 183
§8. Normal forms in a neighborhood of an equilibrium state (survey
and comments) 184

Chapter 5. Transformations of the Circle 189


§1. The Poincare rotation number 189
1.1. Definitions and notation 189
Remark 1. The rotation set of a continuous transformation of
degree 1 193
Remark 2. The rotation set of a topological Markov chain 195
Remark 3. The rotation set of a mapping of Lorenz type 195
1.2. Invariance of the rotation number 196
1.3. Continuous dependence of the rotation number on a parameter 197
1.4. The rotation number of a homeomorphism of the circle 198
§2. Transformations with irrational rotation number 199
2.1. Transformations semiconjugate to a rotation 199
2.2. A criterion for being conjugate to a rotation 202
2.3. Limit sets 203
2.4. Classification of transitive homeomorphisms 204
2.5. Classification of Denjoy homeomorphisms 205
2.6. Classification of Cherry transformations 209
§3. Structurally stable diffeomorphisms 213
3.1. The Cr-topology 213
3.2. Main definitions 213
3.3. Instability of an irrational rotation number 215
3.4. Openness and denseness of the set of weakly structurally stable
diffeomorphisms 217
3.5. Classification of weakly structurally stable diffeomorphisms 217
Remark. Diffeomorphisms of the first degree of structural insta-
bility 217
§4. The connection between smoothness properties and topological
properties of transformations of the circle 218
4.1. Continued fractions 218
4.2. The order of the points on the circle 220
4.3. The theorem of Denjoy 221
4.4. The theorem of Yoccoz 223
4.5. Corollary to the theorem of Yoccoz for Cherry transformations 227
4.6. The Herman index of smooth conjugacy to a rotation 227
§5. Smooth classification of structurally stable diffeomorphisms 230
5.1. Pasting cocycles 230
5.2. C°'a-conjugacy 232
5.3. Smooth classification 234
5.4. Corollaries 235
5.5. Conjugacy of flows 236
5.6. Inclusion of a diffeomorphism in a flow 238
5.7. Comments 238
CONTENTS xi

Chapter 6. Classification of Flows on Surfaces 239


§1. Topological classification of irreducible flows on the torus 239
1.1. Preliminary facts 239
1.2. Curvilinear rays 244
1.3. Asymptotic directions 245
1.4. The Poincare rotation number 249
1.5. The rotation orbit 251
1.6. Classification of minimal flows 253
1.7. Classification of Denjoy flows 254
Appendix. Polynomial Cherry flows 259
§2. The homotopy rotation class 262
2.1. Lobachevsky geometry and uniformization 262
2.2. The axes of hyperbolic isometries 263
2.3. Asymptotic directions 265
2.4. Arithmetic properties of the homotopy rotation class 268
2.5. The homotopy rotation class of a nontrivial recurrent semitra-
jectory 270
2.6. The connection between quasiminimal sets and geodesic lami-
nations 273
2.7. Accessible points of the absolute 279
2.8. Classification of accessible irrational points 280
2.9. The orbit of a homotopy rotation class 283
§3. Topological equivalence of transitive flows 285
3.1. Homotopic contact-free cycles 285
3.2. Auxiliary results 286
3.3. Construction of a fundamental domain 288
3.4. Necessary and sufficient conditions for topological equivalence
of transitive flows 290
Remark. Levitt's counterexample to a conjecture of Katok 293
§4. Classification of nontrivial minimal sets 294
4.1. Special and basic trajectories 294
4.2. The canonical set 295
4.3. Topological equivalence of minimal sets 297
4.4. Realization of nontrivial minimal sets by geodesic curves 299
§5. Topological equivalence of flows without nontrivial recurrent tra-
jectories 300
5.1. Schemes of semicells 300
5.2. Schemes of spiral cells 302
5.3. The orbit complex 303
5.4. Neighborhoods of limit singular trajectories 305
5.5. Main theorems 307

Chapter 7. Relation Between Smoothness Properties and Topological


Properties of Flows 309
§1. Connection between smoothness of a flow and the existence of a
nontrivial minimal set 309
1.1. The theorems of Denjoy and Schwartz 309
1.2. The theorem of Neumann 310
1.3. The theorem of Gutierrez 315
xii CONTENTS

§2. The problem of Cherry 315


2.1. Gray and black cells 315
2.2. The Poincare mapping in a neighborhood of a structurally stable
saddle 316
2.3. Sufficient conditions for the absence of gray cells 318
2.4. Cherry flows with gray cells 318

Bibliography 321
Foreword

This book is an introduction to the qualitative theory of dynamical systems on


manifolds of low dimension (on the circle and on surfaces). Along with classical
results, it reflects the most significant achievements in this area obtained in recent
times by Russian and foreign mathematicians whose work has not yet appeared in
the monographic literature. The main stress here is put on global problems in the
qualitative theory of flows on surfaces.
Despite the fact that flows on surfaces have the same local structure as flows on
the plane, they have many global properties intrinsic to multidimensional systems.
This is connected mainly with the existence of nontrivial recurrent trajectories for
such flows. The investigation of dynamical systems on surfaces is therefore a natural
stage in the transition to multidimensional dynamical systems.
The reader of this book need be familiar only with basic courses in differential
equations and smooth manifolds. All the main definitions and concepts required
for .understanding the contents are given in the text.
The results expounded can be used for investigating mathematical models of
mechanical, physical, and other systems (billiards in polygons, the dynamics of a
spinning top with nonholonomic constraints, the structure of liquid crystals, etc.).
In our opinion the book should be useful not only to mathematicians in all
areas, but also to specialists with a mathematical background who are studying
dynamical processes: mechanical engineers, physicists, biologists, and so on.

xiii
CHAPTER 1

Dynamical Systems on Surfaces


§1. Flows and vector fields

1.1. Definitions and examples. A flow f t or a dynamical system with


continuous time on a manifold M is defined to be a mapping f : M x JR -* M such
that

1) f (m, tl +t2) = f [f (m, tl), t2], m E M, tl, t2 ER,


2) f(m,O) = m, m EM.
A flow f t is called a C''-flow (r > 0) if the mapping f is of smoothness Cr. If in
def
addition the restriction f I {m} xR = ft (m): R -* M is of smoothness C''+1, then
the flow will be called a C'''T *flow.
It follows from the definition of a C''-flow that for each fixed t E Ilk the mapping
ft = f ( , t) : M -* M is a C''-diffeomorphism (a C°-diffeomorphism is understood
to be a homeomorphism). Therefore, a C''-flow on a manifold M can be defined
to be a one-parameter group of C''-diffeomorphisms (a C''-action of the additive
group Ilk on M).
Passing through each point m E M is the directed curve 1(m) = {f(m, t)
-00 < t < -boo}, called the trajectory (through m). A trajectory 1(m) = {m}
consisting of a single point is called an equilibrium state (or. rest point, or singular
point, or fixed point).
A trajectory homeomorphic to the circle S1 is said to be closed. A trajectory
that is not closed and is not an equilibrium state is said to be nonclosed.
The positive semitrajectory beginning at a point m E M is defined to be the
set l+(m) = {f(m, t) t > 0}. Similarly, l-(m) = {f(m, t) t < 0} is the negative
: :

semitrajectory.
EXAMPLES. 1) f t is the one-parameter group of rotations of the sphere 52
for which ft is the rotation of 52 about the SN axis (the north pole-south pole)
through an angle t E R (see Figure 1.1). The points N and S are equilibrium states
of the flow f t . The remaining trajectories are the parallels of the sphere.
2) We represent the two-dimensional torus T2 as the product S1 x S1 of two
unit circles. Let the numbers a and /3 be fixed, and let ft be the composition
of the rotation of the torus along the parallel {.} x S1 through the angle at and
the rotation along the meridian S1 x {} through the angle /3t (see Figure 1.2).
The trajectories of this flow are all closed or all nonclosed in dependence on the
numbers a and /3 (if /Q/a is rational, then the trajectories are closed, while if /Q/a is
1
2 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.1

FIGURE 1.2

irrational, then the trajectories are all nonclosed; in the latter case each trajectory
is dense on the torus).
1.2. Connection between flows and vector fields. Denote by TM the
tangent space of the manifold M. Recall that a vector field V of class Cr, r > 0,
on M is defined to be a section V : M -* TM of the bundle it : TM -* M, that is,
a mapping V of smoothness class Csuch that it o V = id.
If f t is a given C1-flow or C°'1-flow on M, then at each point m E M the
tangent vector
d
V(m) = tm )
dtf ( t=o
to the trajectory 1(m) passing through m is defined. The vector V (m) is called
the phase velocity of the point m. The correspondence m F-* V (m), where m runs
through the whole manifold M, is a vector field, which we denote by V f If f t is a
.

C''-flow, then Vf is a C''-smooth vector field: the field of phase velocities of the
flow f'.
It is known ([17], [67]) that any vector field V of smoothness class C, r > 1, on
a closed manifold M is the field of phase velocities of some C''-flow f t . Therefore,
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 3

one natural way of specifying a flow on a compact manifold is to specify a vector


field.
1.3. Vector fields and systems of differential equations. Each point mo
of the manifold M is covered by a local chart U in which a coordinate system is given
by a mapping (x1,... , x1) : U -* Rn, where n = dim M. To define a vector field V
in the chart U we must specify n functions v1, ... , vn : U -* Ilk, the components of
the field V in the chart U. If f t is a flow such that V = V f , then
d
v( m) _ -xj (f tm) m E U.
dt t=o
This implies that the trajectory f t(m) fl'l.L is a solution of the system of differential
equations
xZ = vZ xl,...,xn , i = 1,...,n,
with the initial condition m = (xi (0),. . , xn (0)) Indeed, it must be verified that
.

the rate of motion of the point f t (m) at each time to such that f to (m) E U is equal
to V (f t° (m)) (see Figure 1.3). Since f t is a one-parameter group,

[ft (m)] = _xj[fto+T(m)] = dTxi[fT (ft0(m))] = vz(ft0(m))


dt xz t=to T=O T=O

FIGURE 1.3
The field of phase velocities in a local chart is usually given as a system of
differential equations.
1.4. Diffeomorphisms of vector fields. Let cP : M -* N be a diffeomor-
phism, and let V be a vector field on the manifold M. It is known ([25], [47]) that
co induces a mapping co* : TM -* TN of the tangent spaces. The restriction of co*
to V is a vector field co* (V) on the manifold N, called the image of the vector field
V under the diffeomorphism co.
§2. Main ways of specifying flows on surfaces
The specification of a flow on a surface depends on and is closely connected
with the representation of the surface. For example, the surface can be represented
as a set of points in JR3 (the coordinates of which satisfy some equation), or it can be
represented as a two-dimensional manifold by means of local charts and compatible
coordinates. A surface can also be represented as the quotient space of the plane
with respect to some group of transformations. These and other representations
enable one to define a flow on a surface in various ways.
4 1. DYNAMICAL SYSTEMS ON SURFACES

2.1. The projection method. Suppose that the equation f( x, y) = 0 de-


termines p + 1 disjoint ovals (circles) on the Euclidean plane 1R2, where p of the
ovals have disjoint interiors and lie inside the last (see Figure 1.4 for p = 2). For
example, f2 (x, y) _ (x2+y2-16).[(x+2)2+y2-l].[(x-2)2+y2-l] (p= 2). Then
the equation y, z) = y) + z2 = 0 determines a closed orientable surface
M of genus p > 0 in R3 (see Figure 1.5, p = 2). If the ovals f( x, y) = 0 do not
have singularities, then the surface M also does not have singularities; therefore,
O = /)2 + (F')2 + (F')2 > 0. Denote by n = O-1 (F,F,F) the unit
p
vector normal to M.

FIGURE 1.4

Associated with an arbitrary vector field i7 on JR3 is the field Vtan (the projection
of i7 on the tangent plane of the surface according to the formula
Vtan = V - (n V )n.
Since n Vtan = n V - (n V)n2 = 0, Vtan is a vector field on M.
It is known that any continuous function on can be extended to a continuous
function on R3 ([67], [71]). Therefore, any vector field on M can be extended to
R3. The projection method can thus give us all conceivable vector fields on M.
Note that if the flow on JR3 determined by a field j7 is given by the system of
differential equations
x = P(x, y, z), y = Q(x, y, z), z = R(x, y, z),
then the projected flow on which is determined by the field Vtan, is given by
the system

{
z = R-O-1(PF
EXAMPLES. 1) If we project the vector field j7 = (0,0, -1) on the sphere
S2: x2 ,+ y2 .+ z2 = 1, then on S2 we get the field Tan of a flow with two equilibrium
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 5

FIGURE 1.5

states, at the north and south poles, and all the remaining trajectories of the flow
obtained "flow down" from the north pole to the south pole along meridians (see
Figure 1.6).

FIGURE 1.6

2) Let us consider the system


x=-y, y=x, z=0
in R3. Each point of the axis Oz is an equilibrium state of this system, and all the
remaining trajectories are closed. The sphere S2 and the torus T2 determined by
the equation (x2 + y2 - 1) (x2 + y2 -1/4) + z2 = 0 are integral surfaces of the field
V = (-y, x, 0) ; therefore, 1 an = i7. The flow on 82 is represented in Figure 1.1,
and the flow on T2 in Figure 1.7.

REMARK [68]. We consider a differential equation

(2.1) F(x, y, dam) = 0


6 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.7

not solved for the derivative, and we assume that (F)2 + (F)2 + (Ffl2IF(X,,Z)=o
0. Let M2 be the surface in IiS3 given by the equation F(x, y, z) = 0. Then M2 is
an integral surface of the system

dt - -F'z
(2.2) dt = -zFz
dt = Fx + zFy .

Indeed, F( -F) + Fy (-zFz) + F( F + zFy) 0; therefore, there is a flow f t on


M2 whose trajectories are solutions of the system (2.2). It is not hard to show that
if Fz 1M2 0, then the projections of the trajectories of f t on the (x, y)-plane f2
are solutions of the equation (2.1).
2.2. Systems of differential equations in local charts. Let (x1, x2) : u1
1R2 and (y1, Y2): U2 -p IR2 be the coordinates in overlapping local charts u1 and
U2 of the two-dimensional manifold M2, and let m E u1 rl U2. The coordinates
(vi, v2) and (w1, w2) of the same vector at m in the respective charts u1 and U2
are connected by the relations

w1 =axi 211 + axe 212

W2= a vl+a v2
or W = 3V, where

3=
ii
axi 01x2

axi 01x2

is the Jacobian of the transition from the coordinates (x1, x2) to the coordinates
(y1, y2). Thus, if the flow f t on M2 is determined in the charts u1 and U2 by the
corresponding systems of differential equations

xi = vi (xi, tea) J yi = wi (yi, y2)


1. X2 = V2(Xi,X2), l. Y2 = W2(yl,y2),
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 7

FIGURE 1.8

then (2.3) holds at all points of the intersection u1 f1 U2, and, conversely, if a
system of differential equations is given in each local chart, and if (2.3) holds in an
intersection of charts, then a vector field is defined on M2.
EXAMPLE. By means of the stereographic projections (see Figure 1.8) x : UN =
82 \ {N} fl 2 and y : us = 82 \ {S} -* we introduce the respective coordinates
(xi, x2) and (y1, y2) on the local charts UN and Us of the unit sphere S2, where
N(0, 0,1) is the north pole and S(0, 0, -1) the south pole of the sphere. The
transition from one set of coordinates to the other in the annulus f2 \ (0,0) is
realized by the formulas

yi+y2 y1 - xi
X2 = y
{x'= y2 = xi

The Jacobian of the transition from (x1, x2) to (y1, y2) has in the coordinates (Yl, Y2)
the form
yz - yi -Zyiy2
-2y1y2
Y?-Y22H
In the charts UN and us we now write the systems of differential equations
x yi (yi +y2 )
x 1 = 1 + i +x2 y1 = 1 +7/ +y2
x2 = 1+xi+x2 , y2 = y2(y2+y2)
1+yi +y2

It is not hard to verify that (2.3) holds, and hence a flow f t is given on S2. We
leave it for the reader to convince himself that f t is represented in Figure 1.6.
2.3. Specification of a flow with the help of a universal covering. An
advantage of this approach is that for surfaces of genus p > 1 the universal covering
is homeomorphic to IIg2 with a single coordinate system. After specification of a flow
on the universal covering its invariance with respect to the covering transformation
group must be verified.
8 1. DYNAMICAL SYSTEMS ON SURFACES

2.3.1. Transformation groups. A group F of transformations of a topological


space M acts freely on M if any element -y E F different from the identity does not
have fixed points. A freely acting group F is said to be discontinuous if each m E M
has a neighborhood U m such that -y(U) fl U = 0 for all -y E F, -y id.
It is known [78] that if a discontinuous freely acting group F of transformations
consists of homeomorphisms, then the quotient space NYC/F (each F-orbit is identified
with a point) is equipped with a topology in which the natural projection it : M -*
NYC/F is a covering.
2.3.2. Flows on the torus. On the Euclidean plane R2 (with Cartesian coordi-
nates x, y) we consider the group F of translations by an integer vector: (x, y) H
(x + k, y + r), (k, r) E Z2. It is not hard to verify that F acts freely and discontin-
uously on fl 2 . To represent the space 1R2 /F we take a suitably chosen fundamental
domain F (the closure of a set containing exactly one point from each F-orbit) and
identify its boundary by means of the action of F. We can take F to be the square
0 < x < 1, 0 < y < 1. Opposite sides of F are pasted together under the action
of F, and we obtain the torus T2 = 1R2 /F, a closed orientable surface of genus 1
(Figure 1.9).

FIGURE 1.9

A dynamical system
± = f1(x,y)
{ y = f2 (x, y)
on 1R2 projects into a dynamical system on T2 (that is, is a covering system) if and
only if Ii (x, y) and f 2 (x, y) are periodic functions of period 1 in both arguments.
Under the action of it (recall that a covering is a local homeomorphism) the tra-
jectories of a covering flow on are mapped into the trajectories of the flow on
T2.
t
EXAMPLE 1 (rational and irrational windings). The flow f given by the system

f±=1
y=µ
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 9

FIGURE 1.10
t
is a covering flow for a flow f t on T2. The integral curves of f are lines with slope
(Figure 1.10). The parametric equations of the trajectory to passing through a
point (xo,yo) at t=0 have the form x = xo +t, y = yo +,at.
The behavior of the trajectories on T2 depends on the number c. We consider
two cases.
a) = p/q is rational.
The trajectory to passes through the two points (x0, yo) and (xo + q, yo + p) corre-
sponding to t = 0 and t = q, and these points are mapped by it into a single point
on T2. Consequently, it (lo) is a closed trajectory. Conversely, if it (lo) is a closed
trajectory, then xo + t = xo + k and yo + pct = yo + r for some t = t 0, where
k, r E Z, and thus = r/k. Hence, all the trajectories of the flow f t on T2 are
closed if and only if is rational.
b) is irrational.
According to a), all the trajectories on T2 are nonclosed. Moreover, each trajectory
4f
is dense in T2 because each trajectory it (lo) intersects the circle it (x = xo) So
at the points {yo + np (mod 1), n E Z}, but the latter set is dense in So. Indeed,
we partition the circle So into k equal half-open intervals of length 1/k. Since is
irrational, the points yo + n E Z, are distinct. Therefore, among the
k +1 points yo + np (mod 1), n = 0,... , k, there are two, say yo +p C (mod 1) and
yo (mod 1), that lie in a single half-open interval of length 1/k. For definiteness
we assume that p> q, and we set r = p - q. Then 0 <r C (mod 1) < 1/k, and
in the sequence of points yo + (mod 1), n E Z, the distance between each two
adjacent points is (mod 1) < 1/k. For a given e > 0 we take a k such that
1/k <e. Then the e-neighborhood of any point of the circle contains points in the
sequence {yo + rnp (mod 1)} (see also [26], Chapter 3).
The flow f t on T2 is called a rational (irrational) winding of the torus if c
is rational (irrational); sometimes one refers to a rational (irrational) flow on the
torus. In the collection of rational windings we also include the flow on T2 whose
covering flow on 1R2 is given by the system x = 0, y = 1.
EXAMPLE 2impassable grain). The flow
d; =xa+y2
y= y2)
10 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.11

on R2 differs qualitatively from the flow in Example 1 in that an equilibrium state


with two saddle sectors is "planted" at the origin of coordinates (Figure 1.11). Such
an equilibrium state is called an impassable grain or fake saddle.
A flow on T2 with an impassable grain is covered by the flow
x = sine irx + sine 'iry
{ y = (sin 2 irx + sine iry).
A flow with any finite number of impassable grains at specified points can be
obtained similarly.
2.3.3. Flows on closed orientable surfaces of genus > 1. Denote by 0 the open
unit disk of the complex z-plane (z = x + iy). The linear fractional transformations
of the form
az+b
z'-4_
bz+a
2 -lbl =1,
al 2

carry 0 into itself and form the group I(0) .


The famous uniformization theorem [77] asserts that for any closed orientable
surface Mp of genus p > 1 there exists a finitely generated subgroup Fp of I (s)
such that:
1) Fp acts freely and discontinuously on O;
2) Mp = 0/I'p i
3) the natural projection it : O - /F7, = Mp is a universal covering.
A flow
J x = f1(x, y)
y = f2 (x, y)
on 0 is covering for some flow on Mp if and only if f2 ('y(m)) = f2 (m), i = 1, 2, for
all 'Y E I'p and all points m(x, y) E L.
An example of a flow on Mp with some fixed Riemannian metric that is in-
variant with respect to Fp can be obtained as follows. Take a function (z) that
is automorphic with respect to Fp, that is, (y(z)) _ (z) for all 'y E rp [78].
Then the vector field 17(z) = grad (z) is invariant with respect to Fp, and hence
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 11

a ;ai

Ci
d

FIGURE 1.12

FIGURE 1.13

is covering for some vector field Up on M7,. The field Up generates on YCp a flow
belonging to the class of so-called gradient flows.
EXAMPLE (a polar Morse-Smale flow). As a fundamental domain of the group
Fp we can take a curvilinear 4p-gon 'p bounded by arcs of Euclidean circles per-
pendicular to the circle S = & bounding the disk 0 (see Figure 1.12 for the case
p = 2), where each side of the polygon p is identified with one other side of p
by means of Ti,, and all the vertices of p are identified with a single point (Figure
1.12 shows an example of identification of the sides of 2). We represent the phase
portrait of a flow on p by locating a stable node at the center of the polygon,
a saddle at the midpoint of each arc of the boundary app, and an unstable node
at the vertices (Figure 1.13 for p = 2). On the sides of T?p this phase portrait "is
compatible with" the action of the group I',, and hence it determines the phase
portrait of some flow f t on the surface M7,. The flow f t has two nodes (one stable
and one unstable) and 2p saddles, and there are no other equilibrium states nor
12 1. DYNAMICAL SYSTEMS ON SURFACES

closed trajectories. The flow constructed (with certain restrictions on the eigenval-
ues of the equilibrium states) belongs to the class of so-called polar Morse-Smale
flows (see Chapter 3 for the exact definition of Morse-Smale flows). This flow is
a gradient flow; that is, the corresponding vector field can be represented as the
gradient of a function on M.
2.4. Specification of a flow with the help of a branched covering.
2.4.1. Definition of a branched covering. Denote by Sk the mapping z H z of
the z-plane C, k E N.
DEFINITION. A transformation it : M -* M of a two-dimensional manifold M
into a two-dimensional manifold M is called a branched covering if any point m E M
has a neighborhood U m such that the complete inverse image it -1(u) is a union
V1 U V2 U ... of disjoint neighborhoods, and the restriction 1r I vz is topologically
conjugate to some mapping Sk (that is, there exist homeomorphisms h : U -p C and
h2 : V -* C such that Sk o h2 Vi = h o it l v) . The number k is called the branching
index of the point z2 = it -1(m) fl Vi and is denoted by k (z2) .
We shall consider only (regular) coverings such that all the points in the inverse
image it -1(m) have the same branching index for any m E M. For such coverings
the branching order k(rn) of a point m E M is defined to be the branching index
of any point in it-1(m).
A point mo E M is called a branch point if k (mo) > 1. The collection Mo C M
of branch points is called the branch set; Mo is discrete, and it is finite for compact
M.
The number of points in the complete inverse image l(), x E M \ Mo, is
called the multiplicity of the covering. For an arcwise connected M this number is
independent of the point x E M \ Mo .

EXAMPLE. In i3 we consider a surface Mpq of genus p that is symmetric with


respect to the axis Oz and such that the axis intersects q "handles" (see Figure
1.14, where p = 6 and q = 3). Denote by G the group of transformations R3 -p
generated by the symmetry with respect to the axis Oz. Then Mpq is invariant
under G, and Mpq/G = Mo is a closed surface of genus [2j-] + 1. The surface Mo
can be obtained as follows. Take the part of Mpq lying in the half-space y < 0. This
part is a surface Fpq with [ + 1 handles and a boundary, and it is a fundamental
domain of the action of G on Mpq. The identification of the boundary points of
Fpq under the action of G amounts to pasting up the holes in the surface Fpq. As
a result we get a closed surface Mo with [2-j-] + 1 handles.
The natural projection Mpq -p Mo is a two-sheeted branched covering with 2q
branch points (the points A, B, C, D, E, and F in Figure 1.15).
2.4.2. Covering flows. Let f t be a smooth flow on a closed orientable surface
M, and let i7 be its phase
N
velocity field. Assume that there exists a two-sheeted
branched covering qr : M -* M, and each branch point coincides with an equilibrium
state of f. Then the restriction of the flow f t to the set M \ Mo, where Mo is the
branch set, is a flow fo. Let Vo be the phase velocity field of fo. Since the restriction
_ (moo) : M \ it -1(Mo) -p M \ Mo is an unbranched covering, and since an un-
branched covering is a local diffeomorphism in compatible differentiable structures,
there is a vector field Vo on M \ it -1(Mo) that covers Vo (that is, ir* (iZ) = Vo ,
where the mapping ir* of the tangent spaces is induced by the local diffeomorphism
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 13

FIGURE 1.14

FIGURE 1.15

1r l vt We define the field Vo to be zero at points of it-1(Mo) Since f t


_ 1 Mo ) . .

has equilibrium states at points of Mo, it follows from the definition of a branched
covering that the vector field i7 * obtained on M is continuous and covers the field
i7. Then the flow f t on M determined by 7* is a covering flow for f t .

2.4.3. Construction of transitive flows.


DEFINITION. A flow f t on M is said to be transitive if f t has a trajectory that
is dense in M.
An irrational winding on T2 is an example of a transitive flow.
We consider a two-sheeted covering it of the torus T2 by a pretzel: a closed
surface M2 of genus 2 (see Figure 1.16) with branch points A, B E T2. This covering
is completely analogous to the example described in 2.4.1.
Let f o be an irrational winding on T2. By moving the points A and B slightly
we can ensure that they lie on different tranjectories of the flow fo. Suppose that
14 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.16

the flowNf t differs from f o only in the existence of impassable grains at A and B,
and let f t be a covering flow for f t (by 2.4.2) on M2.
It is not hard to see that the dynamical system

on IR2 with a unique equilibrium state (a saddle) is a covering flow for the flow

{ y=0
(with an impassable grain at the origin) with respect to the two-sheeted branched
covering
N
z H z2 (Figure 1.17). Therefore, the only equilibrium states of the flow
f t on2 are the two saddles at the points it-1(A) and it-1(B).

FIGURE 1.17

Any trajectory l of f t different from an equilibrium state is dense in T2. Con-


sequently, the complete inverse image it-1(l), which consists of two trajectories l1
and l2 of the flow f, is dense in M2. It can be shown that in this case one of the
trajectories l1, l2 is dense in M2; that is, the flow f t is transitive.
Since any closed orientable surface of genus p > 2 is a two-sheeted covering of
the torus T2, the method described can be used to construct a transitive flow on
any p > 2.
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 15

2.5. The pasting method. Let f1 and f2 be two flows on the two-dimen-
sional manifolds M1 and M2, respectively, and let Bi C MZ be a disk whose bound-
ary intersects trajectories of f2 transversally except at the two points m12 and m,
i = 1, 2 (see Figure 1.18). Denote by d2 (d2) the arc of the boundary aB2 across
which trajectories of f2 enter (leave) BZ, i = 1, 2. We put impassable grains at the
points m12 and m2 and denote the resulting flow by f2 , i = 1, 2.

FIGURE 1.18

We now identify the boundaries of the manifolds M1 \ B1 and M2 \ B2 with the


help of an orientation-reversing homeomorphism h : 9B1 -* aB2 such that h(d1) =
d, h(d1) = d, and k = 1, 2. The manifold M = M1 #M2
obtained is called a connected sum of the manifolds M1 and M2.
m(1)h
The flows f i and f 2 determine a flow f t on M as follows. The points '
m(2) , i = 1, 2, are equilibrium states of the flow f t. Let m E M1 \ B1 and t E III,
t > 0. If (fl)T(m) E M1 \ B1 for 0 < T < t, then we set ft(m) = (fi)t(m). If
(fi)T (m) E B1 for some 0 <r < t, then there is a smallest T1 such that (f1 )Ti (m) E
d1 The trajectory ft(m) then "passes" to M2 \ B2i that is, for r1 < T < t
.

sufficiently close to r1, we set

f( m) = ()1(ho)1(m)).
If (f 2 )T (h o f (m)) E M2 \ B2 for all 0 < T < t - r1, then we set ft (m) =
(f2)t-ri (h o f 1T1 (m)). Otherwise, we again pass to M1 \ B1 via the pasting h. The
definition of ft (m) for t E III, t <0, and for points m E M2 \ B2 is analogous.
Note that the points i = 1, 2, are saddles.
EXAMPLE. We take M1 = M2 to be the two-dimensional torus T2, with irra-
tional windings f i = f 2 Let B1 = B2 = B be such that the points m1 and m2
.

lie on different trajectories. A pasting homeomorphism is constructed in a special


way. Let h1: d+ -p d- be a trajectorywise homeomorphism (see Figure 1.19); that
is, if m E d+, and with increasing time the trajectory 1(m) first intersects the arc
d- at time r, then hi(m) = (fi)(m). Let hld+ = h1 and hid- = h1 1. The past-
ing method gives a flow f t on the connected sum T 2 #T 2 (a surface of genus 2, a
pretzel).
16 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.19

We leave it as an exercise for the reader to prove that f t is a transitive flow.


2.6. Suspensions.
2.6.1. The suspension over a homeomorphism of the circle. Let f be a homeo-
morphism of the circle 51. We use f to paste together the boundary of the cylinder
51 x [0, 1]; that is, we identify the points (x, 1) and (f (x), 0) in 81 x [0, 1] for x e Si.
The manifold M f obtained is the torus if f preserves the orientation of the circle
(since an orientation-preserving homeomorphism is isotopic to the identity), and
the Klein bottle if f changes the orientation.
We define a flow on M1. To do this we represent an arbitrary point of M1 in
the form (x, r), where x e 51 and 0 < r < 1. For z = (x, r) E M1 and t e Ilk we set

ft (z) = (f[t+r](x),{t+r}),
where [a] ({a}) denotes the integer (fractional) part of a number a (Figure 1.20).
The proof of the group property fti +t2 = ftl ° ft2 is left to the reader.

FIGURE 1.20

The flow defined above from the homeomorphism f is called the suspension
over f and is denoted by sus(f).
The properties of f and its suspension are closely related, of course. For exam-
ple, we have
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 17

LEMMA 2.1. 1) 1f f has a dense orbit, then the flow sus(f) has a dense trajec-
tory (transitivity);
2) if f has a nowhere dense orbit, then sus(f) has a nowhere dense trajectory;
3) if f has a periodic orbit, then sus(f) has a periodic trajectory.
The converse assertions are also valid.
The proof of the lemma follows from the definition of the flow sus (f) and the
fact that any trajectory passing through a point (x, 0) E M1 passes through the
point (fTh(x), 0) at time t = n e 7L.
EXAMPLE. Let f = R, be the rotation of the circle x2 + y2 = (1/2ir)2 through
the angle 27r, E R. Then is a rational or irrational winding of the torus,
depending on whether is rational or irrational.
2.6.2. The suspension over an exchange of open intervals. Let a1,... , aT E S1
be points labeled successively as the circle S1 is traversed in the positive direction,
r > 2. Denote by L = (ad, a2+1), i = 1, ... , r, the open interval between the points
a2 and a2+1, traversed in the positive direction from a2 to a2+1, where ar+1 = al
Let b1, ... , bT be cyclically labeled points on S1, distinct from a1, ... , a,. in
general.
DEFINITION. A transformation T : S1 \ {a1,.. . , a} - 8 1 \ {bi,.. . , bT} that is
one-to-one and is an orientation-preserving homeomorphism on each interval OZ,
i = 1, ... , r, is called an exchange of open intervals.
Associated with each exchange of open intervals is a permutation r of the num-
bers {1,... , r} such that the interval z is mapped into the interval (bT(j), b()+1),
where br+1 = b1.
REMARK. In ergodic theory [47] an exchange is understood to be a one-to-one
mapping of the whole circle (or closed bounded interval) into itself that is a shift
on each half-open interval.
To construct the suspension over T we first construct the manifold MT on
which this suspension will be defined.

FIGURE 1.21

Denote by ?vt the manifold obtained by cutting the annulus Sl x [0, 1] along
the segments {a} x (1/2, 1], i = 1, ... , r (see Figure 1.21). In other words, M
is obtained by attaching disjoint rectangles cl(z ) x [1/2,1], i = 1, ... , r, to the
18 1. DYNAMICAL SYSTEMS ON SURFACES

annulus Sl x [0,1/2] (where cl(Di) denotes the closed interval [ad, ai+l]) The side
{ai} x [1/2,1] of the rectangle cl(Di) x [1/2,1] (c1(L_i) x [1/2,1]) is called the right
(left) side and denoted by r(l;,) (see Figure 1.22). We let ll = {al} x [1/2,1] C
cl(OT) x [1/2,1].

FIGURE 1.22

The side 13 of the manifold M is pasted together with the side rk by identifying
the points {a3} x t {ak} x t, 1/2 < t < 1, if and only if the segments T (z ) and
T(0k) are adjacent to each other. The manifold obtained as a result of this pasting
M
is denoted by M.
By the definition of an exchange of open intervals, each left side in M is pasted
M
together with one right side, and conversely. Therefore, M is a two-dimensional
manifold with two boundary components that are homeomorphic to S1 (Figure
M
1.23). Speaking loosely, we can say that on one component of the boundary aM
the intervals 01, ... , z are arranged in the original order, while the "exchanged"
intervals T (01), ... , T (z) lie on the other component of the boundary.

FIGURE 1.23

If in addition we identify the points (x, 1) and (T (x), 0) in k for x e S2 \


0
{a1, ... , ar }, then we get a manifold MT of genus p > 2, with punctures p1, ... , Ps
(the punctures correspond to the points a1, ... , ar, b1, ... , bT; if these points are
distinct, then there are s = 2r punctures, but in the general case r < s < 2r).
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 19

FIGURE 1.24
0
Let MT be the compactification of MT. It is clear that MT is an orientable closed
surface of genus p > 2 (Figure 1.24).
We now construct the flow sus (T) on MT.
On R2 consider the dynamical system xt given by

x = 0, = (x2+y2).[(x- 1)2 +y2],


with two impassable grains at the points (0,0) and (1,0). Denote by V the restric-
tion of the vector field of the flow xt to the square II: 0 < x < 1, y < 1/2 (see
Figure 1.25).

H
FIGURE 1.25

We represent the manifold M as the union of the rectangles ll = cl(z ) x [0, 1],
i = 1, ... , r. There is a natural distance-preserving diffeomorphism /: II -p lZ
carrying the segments {x} x [-1/2,1/2] into the segments {a} x [0, 1] (see Figure
1.26). Then the vector fields (b)(V), i = 1, ... , r, form a vector field V on M.
By the definition of the pasting together of the sides 13 and rk in M, the field V
induces a vector field VT on the manifold MT.
The flow determined by the vector field VT on MT is called the suspension over
the exchange T of open intervals and denoted by sus (T) .

We remark that the equilibrium states at the points p1, ... ,Ps with saddle
sectors are the only equilibrium states of the flow sus (T) .
20 1. DYNAMICAL SYSTEMS ON SURFACES

ri
1

FIGURE 1.26
As in the case of suspensions over homeomorphisms of the circle, the topolog-
ical properties of an exchange of open intervals are closely related to those of the
suspension over it. We have
LEMMA 2.2. Let T be an exchange of open intervals. Then:
1) T has a dense semi-orbit if and only if sus(T) has a dense semitrajectory;
2) T has a nowhere dense semi-orbit if and only if sus(T) has a nowhere dense
semitrajectory different from an equilibrium state;
3) T has a periodic orbit if and only if sus(T) has a closed trajectory.
The proof follows immediately from the construction of sus(T), though it is
somewhat more complicated than the proof of Lemma 2.1 due to the existence of
points at which T is undefined. We leave it to the reader.

FIGURE 1.27

EXAMPLE (from the paper [83]). We represent the circle S1 as the quotient
space of the line Ilk with respect to the action of the group of translations by in-
tegers, or as the interval
N
[0, 1] with identification of the endpoints. We consider
the exchange Ta = Ta (mod 1) of the open intervals (0,1/4), (1/4,1/2), (1/2, 3/4),
and (3/4,1), where Ta is given graphically in Figure 1.27. It is not hard to see that
if a is irrational, then Ta does not have periodic orbits, and, moreover, each orbit is
dense in S1. According to Lemma 2.2, the suspension sus(Ta) is a transitive flow.
Figure 1.28 represents sus(Ta), where the circles C1 and C2 are identified by means
of the transformation x H x + a (mod 1).
2. MAIN WAYS OF SPECIFYING FLOWS ON SURFACES 21

FIGURE 1.2$

2.7. Whitney's theorem. Let f t be a flow on the manifold M. Denote by


Fix(ft) the set of equilibrium states of f t The flow f t determines on M \ Fix(ft)
.

a family of curves: the trajectories of the restriction of f t to M \ Fix(f t). It is


clear that this family satisfies definite conditions. In [110] and [111] Whitney
obtained necessary and sufficient conditions for a family of curves to be imbedded
in a flow on a locally compact space. For simplicity we present Whitney's result
for two-dimensional manifolds (possibly open) without proof.
2.7.1. The theorem on continuous dependence on the initial conditions. Let
d be a fixed metric on the manifold M. The following result is an immediate
consequence of the definition of a flow f t .
THEOREM 2.1. For any point m e M and any numbers T> 0 and > 0 there
is a number S > 0 such that if d (m, m) <S and ti < T, then
d(ft(m),ft(m))

FIGURE 1.29

The condition of geometric continuity is an analogue of the theorem on contin-


uous dependence on the initial conditions.
22 1. DYNAMICAL SYSTEMS ON SURFACES

Let F be a family of disjoint curves on M. For a curve l in F and for points


m1i m2 E l denote by mim2 the arc of l between m1 and m2, oriented from m1 to
m2
We say that the family F satisfies the condition of geometric continuity if for
any > 0, any curve l E F, and any arc m1'm2 C l there is a number 8> 0 such
w N
that if d(m1, m1) < S, then there exists an arc rn1rn2 of a curve l E F passing
through m1 for which rn1rn2 is in the E-neighborhood of m m2 and d(m2, m2) <E.
The family of trajectories of the restriction of the flow f t to M \ Fix(ft) obvi-
ously satisfies the condition of geometric continuity.
2.7.2. The rectification theorem. A point of a flow different from an equilibrium
state is called a regular point.
THEOREM 2.2. Let m e M be a regular point of a C' -flow f t (r > 0) on a
two-dimensional manifold M. Then there exist a neighborhood 'IL of m and a CT -
diffeomorphism u - R2 carrying the arcs in U of trajectories into trajectories of
the dynamical system
x=1, y=0
on R2 with preservation of the direction in time.

FIGURE 1.30

This theorem is proved in many books on dynamical systems (for example, [3],
[26], [64], [74]).
A neighborhood 'IL satisfying Theorem 2.2 will be called a neighborhood with
the structure of a constant field, and the diffeomorphism U -* R2 will be called a
rectifying diffeomorphism.
DEFINITION. A family {la} of disjoint curves on a two-dimensional manifold M
is called a foliation (or a foliation without singularities) if M = Ua la, and for each
point m e M there exist a neighborhood U of m and a homeomorphism b: u - R2
such that the components of the intesection of the curves l a with U are carried by
into the lines y = const. The curves la are called leaves.
Theorem 2.2 shows that the collection of regular trajectories (that is, trajecto-
ries different from equilibrium states) of a flow f t forms a foliation on M \ Fix (f t) .
It can be shown that the leaves of a foliation satisfy the condition of geometric
continuity [73].
2.7.3. Orientability. We consider two orientable arcs ab and cd lying on the
manifold M. Denote by D (ab, cd) the set of homeomorphisms H : M -* M such
that H(ab) = cd, H(a) = c, and H(b) = d.
3. EXAMPLES OF FLOWS WITH LIMIT SET OF CANTOR TYPE 23

DEFINITION. The Frechet distance between the arcs ab and cd is defined to be


the number
v(ab, cd) = inf sup d(m, H(m)).
HED(ab,cd) mEab
A sequence of arcs p j, i = 1, 2, ... , converges to an arc pq in the Frechet sense if
(pq,p) -* 0 as i -* oo .

DEFINITION. A family F of curves on a manifold M is said to be orientable


if it is possible to specify an orientation a on all the curves of F (that is, to give
a positive direction on each curve) such that for any arc pq C L E F oriented in
the direction a and for any sequence of arcs p q2 C LZ E F converging to pq in the
Frechet sense there is an index io such that for i > io the orientation on each arc
p q2 from p2 to q2 coincides with the orientation a (where L and LZ are curves of
the family F).
According to Theorem 2.1, the family of regular trajectories of a flow is ori-
entable, and the direction of motion along a trajectory as time increases can be
taken as the positive direction on a regular trajectory. Thus, the condition that
a family of curves forms a foliation (regularity) and the condition of orientability
are necessary conditions for the family to be imbedded in a flow. Whitney proved
that these conditions are also sufficient. Moreover, he showed that the space on the
manifold not occupied by the curves can be filled by equilibrium states.
Let F = {La} be a family of disjoint curves not necessarily filling the whole
manifold M, with the union of the curves in F forming an open subset. The family
F is said to be imbeddable in a flow if there exists a flow f t on M such that each
curve in F is a regular trajectory of f, and each point in the set M \ Ua La is an
equilibrium state of f t .
THEOREM 2.3 (Whitney's theorem). Let F = {La} be a family of disjoint
curves on a manifold M (each curve La is the range of an imbedding of the line Ilk
in M). Then F can be imbedded in a flow if and only if F is orientable and regular
(that is, it forms a foliation on the set Ua La).

§3. Examples of flows with limit set of Cantor type


Let f t be a flow on a manifold M, and let 1(m) = {f(m, t) : -oo <t < oo} be
the trajectory of f t passing through a point m E M.
DEFINITION. The w-limit set of the trajectory 1(m) is defined to be the set
w[l (m)] of points m such that f to (m) = f (m, tn) - * m as n -* +0 for some
sequence {t}1 of numbers increasing to infinity.
In view of the group property f to+ti = fto o ftl the definition of the w-limit
set of 1(m) is independent of the choice of the initial point m and depends only on
def
the trajectory itself; that is, w[l (m1)] = w[l (m)] w(l) for any point m1 E 1(m).
If in the above definition we write limto = -oo instead of limto =
+oo, then we get the definition of the a-limit set a[l (m)] of the trajectory 1(m),
which also is independent of the choice of the initial point m: a(l) = a[l (m)], m e 1.
The w- (a-) limit set of a positive (negative) semitrajectory is defined in the
obvious way.
24 1. DYNAMICAL SYSTEMS ON SURFACES

The limit set of a flow is defined to be the union of the w-limit sets and the
a-limit sets of all its trajectories.
DEFINITION. Let f t be a flow on a two-dimensional manifold M. The w- or
a-limit set of a trajectory of f t is called a limit set of Cantor type if, with the
exception of a finite number (possibly zero) of points, it is locally homeomorphic
to the direct product of a Cantor set 1 and a closed bounded interval.
In this section we present the classical examples of Denj oy and Cherry flows,
along with a little known example of a flow on a sphere (or on a disk) with limit
sets of Cantor type.
3.1. The example of Denjoy.
DEFINITION. A Denjoy flow is defined to be a flow f t without equilibrium
states on the two-dimensional torus that has a limit set of Cantor type. We shall
obtain a Denjoy flow as the suspension over a Denjoy diffeomorphism of the circle
S1. Let us proceed to the construction of the diffeomorphism.
Denote by S(\) the circle obtained by identifying the endpoints of the interval
[0, \] C R. In particular, 51 = 8(1). Let Rw : x H x + w (mod 1) be the rotation
of 8 1 by some irrational number w. Take an arbitrary- point xo E S1 and let
xn=Rw(xo),nE7L.
Let an = 1 / (In I + 2) (In I + 3), Ti E 7L. Then the series > an converges; denote
its sum by a. We associate with each term an of the series an open interval Gn
of length a. We locate the intervals Gn, n e Z, on the circle 5(1 + a) in such a
way that they are disjoint and their mutual arrangement corresponds to the mutual
arrangement of the points xn, n e Z, on 81.
Since a is irrational, for any distinct points x2 and x3 there are points of
the orbit O(xo) = {R(xo), n e Z} on each of the arcs of 51 \ {x2, x3} into
which xi and x3 partition S1. Therefore, there are intervals Gn between any two
intervals GZ and G3, on both the arcs into which GZ and G3 break up the circle
8(1 + a). Consequently, the complement of UGn (Ti E Z) is a nowhere dense set 1
homeomorphic to the Cantor set.
0
Denote by 1 the subset of points in 1 that are not endpoints of the intervals
Gn, n E 7L.
Since the mutual arrangement of the intervals Gn corresponds to the mutual
arrangement of the points xn, n e Z, there exists an orientation-preserving contin-
uous mapping h : 5(1 + a) -* 5(1) = S 1 that carries cl (Gn) into xn , n e Z, and is
0
one-to-one on the set 1 (Figure 1.31).
We begin the construction of the desired diffeomorphism f : 5(1 +a) -* 8(1 +a)
by constructing its derivative F : 8(1 + a) -* R.
Let F I = 1. If Gn = (an, ,Qn) , then we set

F(x) = 1 + nk
a2n
-x) x EGn
where
kn =

1 Recall that a Cantor set on a closed bounded interval (or circle) is defined to be a nonempty
closed perfect nowhere dense subset of the interval (or circle).
3. EXAMPLES OF FLOWS WITH LIMIT SET OF CANTOR TYPE 25

0 r1.

FIGURE 1.31

It can be verified directly that

F dx = an+1.
IGn
Let us compute the ratio an+1 /an. We have that
an+i n+2
for n > 0,
an n+1
an+i InI+3 forn <-1 .
an lnI+1
Then
12
n n+4 forn >_ 0
12
kn= forn<-1.
Iri+1
The quantity (x - a)(/3 n - x)/ate takes a maximal value on the interval Gn =
(an, i3) at x = (ate From this and the form of the function F(x) we get
the inequalities

1/4<1-<F(x)<1,
+
n>0
(3.2)
1<Fx <1+ 3 <5/2, n<-1
1121+1

where x e G. Obviously, F(x) is continuous on the intervals Gn, n e Z. It follows


from (3.2) that F(x) is continuous at the points of 1. Therefore, F(x) is continuous
on S(1+a).
Denote by it : Ilk - S(1 + a) a covering of the circle 8(1 + a) by the line lit It
can be assumed without loss of generality that it (0) = ao is the left-hand endpoint
of the interval Go = (ao, ,@o). Let F : Ilk -* Ilk be a covering mapping for F, that is,
F(x) = F o ir(x), X E lit
Denote by meas (U) the Lebesgue measure of a subset U C S\).
Let a1 E [0,1 + a] be a point such that ir(al) = al, the left-hand endpoint of
the interval G1 = (a1, ,Q1) . We define the mapping f : Ilk -* Ilk by

7(x) = i + LX
26 1. DYNAMICAL SYSTEMS ON SURFACES

Then
rx rx+l+a
i[x+(1+a)]=i+Jo (x)dx+J Fdx

=7(x)+f Fdx= f(x)+J fdx+J Fdx


(i+Q) st uc
= 7(x) + me() + ate. = 1(x) + 1 + a.

Consequently, the mapping f is a covering for some mapping f : 8(1 + a)


8(1 + a) of degree 1. Since D f = F, it follows that D f = F. By (3.2), f is a
C1-diffeomorphism of the circle S(1+ a).
The diffeomorphism f is the desired Denjoy diffeomorphism. Let us consider
its properties.
It follows from the construction of the mapping h : S(1+ a) ---> 81 that for any
0
interval [a, /3] C 8(1 + a) with endpoints a, ,3 E SZ
(3.3) meas([a, Q] f1 SZ) = meas[h(a), h(Q)].
0
We show that f (1) = Q and that Rw o h = h o f . Let x E SZ and let I be an
index set such that Gk C [ao, x] if and only if k E I. Then xk E [h(ao), h(x)] for
k e I. By (3.1), (3.3), and the fact that Rw is a rotation,

meas[al, f (x)] =
f«o,xJ
F dx =
f Gk
F dx +
1 fl[«o,x]
F dx

_ ak+i +meas[R.,h(ao), Rh(x)].


kEl
O
Since x E SZ, it follows that h(x) O(xo) and Rh(x) O(xo), and hence
0
h'Rh(x) E Q.
It is clear that xk E [R,h(ao), R,,h(x)] _ [xi, Rh(x)] if and only if k - 1 E I.
By the construction of the mapping h, this means that Gk E [al, h-1RWh(x)] if
and only if k - 1 E I. Consequently (Figure 1.32), (3.3) gives us that
meas[al, h-1Rwh(x)] _ ak+l + meas(S2 fl [a,,h'R,h(x)])
kE7

_ a/+i + meas[R,,,h(ao), RWh(x)].


kEI

This implies that meas[al, h-1Ru,h(x)] = meas[al, f (x)], so f (x) = h-1Rh(x) E


0

0 0 0 0
The inclusion f -1(1) C SZ is proved similarly. Thus, f(1) = Q, and the relation
0 0
h o f = Rw o h holds on the set Q. It follows from the denseness of SZ in SZ and the
continuity of the mappings f and f -1 that f(1) = f '(Q) = SZ, and the relation
ho f = Rw o h holds on Q. This and the monotonicity off imply that f (Gn) = Gn+1
for all n E Z. Since h(Gn) = xn for n E Z, the relation h o f = Rw o h holds on the
whole circle 8(1 + a).
3. EXAMPLES OF FLOWS WITH LIMIT SET OF CANTOR TYPE 27

RWfiCx)

FIGURE 1.32

If x E SZ, then it follows from the monotonicity of h, the equality h(SZ) = 81,
and the denseness of each orbit of the rotation Rw : Si --> S1 (since w is irrational)
that the limit set of x coincides with SZ. If x § SZ, then the orbit O(x) intersects
each interval at precisely one point. Since diam Gn --> 0 as II ---> oo, the limit
set of the point x also coincides with SZ. Consequently, the limit set of the Denjoy
diffeomorphism f is the Cantor set Q.
The suspension sus (f) gives the desired Denj oy flow. It follows from the forego-
ing that the limit set 1(sus(f )) of sus(f ), which consists of the trajectories passing
through points of Q, is locally homeomorphic to the direct product of a closed
bounded interval and the Cantor set.
It follows from the denseness of each orbit of the rotation Rw, the relation
h o f = Rw o h, and the monotonicity of h that any orbit in SZ with respect to the
diffeomorphism f is dense in Q. Therefore, each trajectory of the flow sus(f) in the
limit set 1(sus (f)) is dense in Q (sus (f)) .

3.2. Cherry flows.


DEFINITION. A Cherry flow on the torus T 2 is defined to be a flow f t satisfying
the following conditions:
1) f t has a limit set Q of Cantor type containing a nonzero number of equilib-
rium states O1, ... , Ok;
2) All the equilibrium states 01,. . . , O are structurally stable saddles;
3) for three (of the four) separatrices of each saddle Oi, i = 1, ... , k, the w- or
a-limit set coincides with Q 2
We describe two Cherry flows with different topological structures. One of
them has smoothness C°°, and the other smoothness C1. The latter will be needed
in Chapter 7.
Both examples are based on a C°°-flow f t on R2 with the following properties
(Figure 1.33) :
1) f t has two equilibrium states: a stable node at the point (-1, 0) and a saddle
at the point (0, 0);
2) in some neighborhood of the boundary of the square [-2, 2] x [-2, 2] the
trajectories of f t are straight lines parallel to the axis Ox;

2It can be shown that the fourth separatrix of a saddle OZ does not belong to SZ and does
not have SZ as its a- or w-limit set.
28 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.33

3) the W-separatrices of the saddle 0(0, 0) (that is, the separatrices tending to
the saddle as t ---> +oo) intersect the line x = -2 at the points (-2, 1) and (-2, -1);
4) one a-separatrix of the saddle 0(0,0) (that is, a separatrix tending to the
saddle as t - -oo) tends to the node as t --> +oo, while the other a-separatrix
coincides with the ray y = 0, x > 0;
5) the vector field of phase velocities of f t is the image of the vector field of
the dynamical system (A) : x = x -I- x2, y = -y (Figure 1.34) under the action
of a C°°-diffeomorphism Sp : R2 --> J2 for which the arc C(x < 1) of the circle
x2 -I- y2 = 4 passes into the segment x = -2, -2 < y < 2, the arcs dl and d2 of
trajectories of the flow (A) (see Figure 1.34) pass into the arcs y = 2, -2 < x < 2
and y = -2, -2 < x < 2 of trajectories of f t, respectively, and the segment of the
line x = 2 between d1 and d2 passes into the segment x = 2, -2 < y < 2.

FIGURE 1.34

It is not hard to see that the trajectories of the dynamical system (A) intersect
the arc C transversally.
3. EXAMPLES OF FLOWS WITH LIMIT SET OF CANTOR TYPE 29

Denote by V the restriction of the vector field of phase velocities of ft to the


square [-2, 2] x [-2, 2]. If we identify opposite sides of this square, then we get a
torus T2 with a vector field Vo. Suppose that the field Vo determines a flow fo on
T2 that is given by the dynamical system (Ao):
x = P(x, y), y = Q(x, y),
where P (x, y) and Q (x, y) are C°°-functions, and are 1-periodic in each variable.
We consider the dynamical system (As) given by
x = P(x, y) - y), y = Q(x, y) + P(x, y).
It follows from results in Chapters 5 and 6 that there is a p = such that
the dynamical system (A,) does not have closed trajectories, and its limit set is
locally homeomorphic to the product of a closed bounded interval and the Cantor
set, with the exception of two points that are equilibrium states. The dynamical
system (A,) gives the desired C°°-Cherry flow on the torus (Figure 1.35). The
construction presented is an insignificant modification of the construction of the
C°°-Cherry flow in [39].

FIGURE 1.35

REMARK. In [81] Cherry constructed such a flow with analytic smoothness by


starting out from an analytic dynamical system with properties analogous to those
of the system (A0).
We can show the existence of an analytic Cherry flow by using the concept of the
rotation number, introduced in Chapters 5 and 6. To do this we take two parameters
i1 and such that the rotation numbers of the dynamical systems (A1) and (A2)
are distinct, and we approximate these systems by analytic dynamical systems (A1)
and (A2) in such a way that their rotation numbers are also distinct. Furthermore,
we make sure that (A1) and (A2) each have two equilibrium states: a structurally
stable node and a structurally stable saddle. In the space of analytic systems we join
the systems (A1) and (A2) by a family (Aa), where 1 < a < 2. Since the rotation
number depends continuously on the parameter, there is an a = a such that the
dynamical system (A) has an irrational rotation number and two equilibrium
states of the above type. Then (A) gives us the desired analytic Cherry flow.
We now construct a C1-Cherry flow by using the dynamical system (A0) and
a Denjoy flow. In addition to the properties 1)-5) satisfied by the system (A0) we
require that (A0) also have the following properties (Figure 1.36) :
30 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.36

6) in some neighborhood of the boundary of the rectangle [-4, 2] x [-2, 2] the


trajectories of the flow f t are lines parallel to the axis Ox;
7) any trajectory intersecting the line x = -4 at a point (-4, 0), 0 < 0 < 2,
intersects the line x = 2 at the point (2, 0) as time increases;
8) the ray y = 0, x < -4 is an W-separatrix of the saddle O that passes through
the point (-2, 1).
Let gt be a C1-Denjoy flow on the torus T2. Denote by 1(gt) the limit set of gt.
By the construction of a Denj oy flow, the set T 1 \ 1(gt) consists of a single simply
connected component w. We take a point m E 1(gt) on the boundary of w. Then
there is a neighborhood U of m that is partitioned by the component of 1(m) f1 U
containing m into two parts U+ and tL-, one of which (say U-) does not meet
(gt) (Figure 1.37).

FIGURE 1.37

It can be assumed without loss of generality that U is a neighborhood with the


structure of a constant field bounded by two arcs of trajectories and by two arcs 'yl
and 72 (Figure 1.37) transversal to the trajectories of gt, and there exists a rectifying
C1-diffeomorphism x : U --> R2 carrying U into the rectangle [-4, 2] x [-2, 2] such
that x('Yi) = {x = -4, -2 < y < 2}, x('y2) = {x = 2, -2 < y < 2}, and
x(tL-) = [-4, 2] x [-2,0].
3. EXAMPLES OF FLOWS WITH LIMIT SET OF CANTOR TYPE 31

We replace the vector field of phase velocities of the flow gt in the neighborhood
U by the vector field (x')(), where V is the restriction of the vector field of
the flow ft to the rectangle [-4, 2] x [-2, 2]. The vector field obtained on T2 is
denoted by T'. Let f t be the flow induced by the field W . Then f t is the desired
C1-Cherry flow (Figure 1.37). It follows from the property 7) that the limit set of
f t includes a saddle, a node, and a set coinciding with the limit set of the Denjoy
flow outside the neighborhood U.
3.3. An example of a flow on the sphere. We consider two disks D1 and
D2 bounded by circles S1 and S2 of unit length. Each circle will be represented as
def
the interval [0, 1] with endpoints identified (Si = [0, 1] i = 1, 2). Suppose that
on each disk Di, i = 1, 2, we are given the same foliation FZ, i = 1, 2, with two
singularities of "thorn" type and with leaves transversal to the boundary t9 D =
i = 1, 2 (Figure 1.38). Furthermore, the points x,1 - x E S2 lie on a single leaf for
anyxES2,i=1, 2.

FIGURE 1.38

We identify the boundaries of D1 and D2 with the help of the shift R: Si - S2


(x H x + (mod 1)), where is an irrational number. As a result of pasting
together the two disks we get the sphere 52. The foliations F1 and F2 form a
foliation F on 82 such that (since is irrational) each of its leaves is dense on the
sphere and none are compact. (Indeed, if from a point x e 3D1 we move along a
leaf first on D 1 and then on D2, then we hit the point x - ,t (mod 1) E 3D1; but
each orbit of the homeomorphism of the circle of the form x H x - (mod 1) is
dense on the circle).
The foliation F can also be obtained as follows. It is not hard to verify that
the quotient space of the Euclidean plane R2 by the action of the group G of
homeomorphisms of the form
5 XH (_1)kx+r
y (-1)ky + s
is homeomorphic to the sphere S2, that is, S2 R2/G. We can take as a fundamen-
tal polygon of G the rectangle 0 < x < 1/2, 0 < y < 1 with sides identified under the
32 1. DYNAMICAL SYSTEMS ON SURFACES

action of G as shown in Figure 1.39. The natural projection it : ll82 --> ll82/G = 82 is
a universal branched covering with four branch points ir(m/2, n/2) E 82, m, n E Z,
each with branching order two.

FIGURE 1.39

The family of lines y = -tux -I- A, A E R, forms a foliation on 1182 which projects
into the foliation F on S2. The lines passing through the points with integer and
half-integer coordinates project into leaves containing singularities of "thorn" type.
We remark that the foliation F is obtained as the image of an irrational winding
(without taking into account the direction in time) of the torus T2 under a two-
sheeted branched covering T2 --> 8 2 with four branch points of index two.
We take a leaf L of F and denote by xn, n E N, the points where L intersects
81. With the respective points xn we associate numbers an > 0 such that the series
aconverges. The subsequent construction is an operation of "blowing up"
the leaf L and is analogous to the similar operation in the construction of a Denjoy
flow.
With each number a, n E N, we associate an open interval Gn of length an,
and we arrange the intervals Gn, n e N, on the circle 8(1+ a), where a = 1 an,
corresponding to the way the points x, n E N, are arranged on S1, with the Gn
disjoint.
Since the set of xn, n E N, is dense in S1, the set SZ1 = S(1+ a) \ UGn (n E N)
0
is a Cantor set. Denote by SZ1 C Q the subset of points that are not endpoints of
the intervals Gn, n E N.
In view of the distribution of the Gn, n E N, there exists a continuous orienta-
tion-preserving mapping h : 8(1 -I- a) ---> 8 1 carrying cl(Gn) into xn, n E N, and
0
one-to-one on the set Q1.
Let D 1 C R2 be the disk bounded by the circle 8(1 -I- a). We construct on
D 1 a foliation F1 transversal to the boundary 3D1 = 8(1 + a) and having two
singularities of "thorn" type. For definiteness and for simplicity we assume that
the leaf L does not contain a singularity, although it will be clear from what follows
how to modify the construction in the opposite case.
Denote by b1, b2 E 8 1 points lying on leaves of F1 that contain singularities of
"thorn" type (Figure 1.38). In view of our assumption, b1, b2 § L. We require that
o
for the foliation F1 the points h-1(b1), h' (b2) E SZ1 lie on leaves with singularities
3. EXAMPLES OF FLOWS WITH LIMIT SET OF CANTOR TYPE 33

of "thorn" type. Suppose that the points cl, c2 E Sl do not lie on the leaf L and
0
are joined by a leaf of the foliation F1; then the points h-1(cl), h-1(c2) E Q1 are
o
also joined by a leaf of the foliation F1 (recall that h is one-to-one on SZ1) If the
.

points xi, x3 E L n Si are joined by a leaf of F1 (that is, these points belong to
a component of the intersection L n D 1), then we require that each point of the
interval cl(Gi) be joined by a leaf of F1 to some point of the interval cl(G3) (Figure
1.40). Since h* is an orientation-preserving continuous mapping, a foliation F1 with
the required properties exists.

FIGURE 1.40

We now realize similarly an operation of "blowing up" the leaf L for the foliation
F2 on the disk D2. We get a foliation on the disk D2 bounded by the circle 5(1 +a).
Because of the equality R( L n Si) = L n S2 and the fact that h is an orientation-
preserving continuous mapping, there is a homeomorphism f : 8(1+ a) - 8(1+ a)
such that

(3.4) Rµoh=ho f
(see §2.5 in Chapter 5 for a rigorous construction of the homeomorphism f).
We identify the boundaries of the disks D1 and D2 with the help of f. The
foliations F1 and F2 form a foliation F on the sphere 52 obtained. Denote by Q
the set of leaves of F passing through points in SZ1 C 5(1 -I- a) C S2. It is clear
that SZ is locally homeomorphic to the product of a closed bounded interval and
the Cantor set.
It follows from (3.4) that SZ is the union of all the self-limit leaves of F (a leaf L0
is called a self-limit leaf if for any point m0 E L0 and any segment > transversal to
the leaves and passing through m0 the intersection L0n> has m0 as an accumulation
point). All the remaining leaves of F are proper; that is, the intrinsic topology of
each of them coincides with the topology induced by the topology of the sphere S2
(consequently, these leaves are not self-limits).
34 1. DYNAMICAL SYSTEMS ON SURFACES

o ,.. o
Denote by F the restriction of F to the set 82 \ Q. The family F of curves is
0
orientable. Therefore, by Theorem 2.3 (Whitney's theorem), F can be imbedded
in a flow f t on 82, and each point of SZ is an equilibrium state of ft. The limit
set of any trajectory lying in 82 \ Q coincides with f Consequently, f t is the
desired C°-flow on the sphere with a limit set of Cantor type. The flow f t can be
constructed to be also of smoothness class C°° (see Chapter 7).
In conclusion we note that a foliation analogous to F can be obtained from
the diffeomorphism f0 : 82 _ SZ constructed by R. V. Plykin in the paper Sources
and sinks of A-diffeomorphisms of surfaces (Mat. Sb. 94 (136) (1974), 243-264;
English transl. in Math. USSR Sb. 23 (1974)). The unstable manifolds of the points
of the hyperbolic attractor of f0 form a family of curves that can be imbedded
in a foliation analogous to the one constructed. See also Plykin's survey, On the
geometry of hyperbolic attractors of smooth cascades (Uspekhi Mat. Nauk 39 (1984),
no. 6, 75-113; English transl. in Russian Math. Surveys 39 (1984)).

§4. The Poincare index theory


One of the basic concepts in the qualitative theory of dynamical systems is the
concept of the Poincare index. In this section we study the index theory, omitting
proofs and following the exposition [68] of Poincare. We regard this approach
as more acceptable in the treatment of continuous flows and foliations than the
currently prevalent approach based on the concepts of the rotation of a vector field
and the degree of a mapping.

4.1. Contact-free segments and cycles. Suppose that f t is a given flow


on a two-dimensional manifold M.

DEFINITION. A segment > C M (that is, the range of an imbedding of the


interval [0, 1] in M) is called a contact free segment or a transversal of the flow f t
if for any point m e > \ a there exist a neighborhood U m with the structure
of a constant field and a rectifying diffeomorphism Sp : U ---> R2 (with smoothness
the same as that of the flow) such that Sp(m) = (0,0) E R2 and Sp(> n U) =
{0} x [-1, 1] C R2 (we remark that Sp carries the trajectories of f t in U into the
lines y = const).

A simple closed curve C (the image of an imbedding of a circle in M) is called


a contact free cycle or a closed transversal of the flow if its arcs are contact-free
segments.
In the definition of a contact-free segment > it is possible that 9> = 0. In this
case > will be called an open contact free segment.
The next result follows directly from the rectification theorem.

LEMMA 4.1. Through each regular point of a flow there passes a contact free
segment.

It is not hard to give an example of a flow for which no regular point has a
contact-free cycle passing through it (for example, a flow on a sphere with two
saddles and four centers).
4. THE POINCARE INDEX THEORY 35

4.2. The index of a nondegenerate cycle in a simply connected do-


main. Let f t be a flow on a two-dimensional manifold M, suppose that the sim-
ple closed curve C C M bounds a simply connected domain D- in M, and let
D+ =NYC\(CUD-).
DEFINITION. We say that a trajectory l of f t is tangent to C at a point m E C
if there exists a neighborhood U m such that:
1) m is the only point common to C and the component d of l n u containing
m
2) the arc d either lies in D- U {m} (in this case m is a point of interior
tangency; see Figure 1.41, a) or d C D+ U {m} (in this case m is a point of exterior
tangency; see Figure 1.41, b).

a) b)

FIGURE 1.41

According to Poincare, the simple closed curve C is called a nondegenerate


cycle (of the flow f t) if:
1) C does not contain equilibrium states of ft;
2) C contains at most finitely many points m1,... , mk of tangency to trajec-
tories of f t (there may be none);
3) each arc of the complement C\ {mi,. . , mk } is an open contact-free segment
.

(Figure 1.42).

FIGURE 1.42
36 1. DYNAMICAL SYSTEMS ON SURFACES

Recall that an isotopy of a manifold 'f into M is defined to be a mapping


: 'f x [0, 1] -* M such that for any t E [0, 1] the mapping cot : 'f -* M, x H (x, t),
is an imbedding. If 'f = 51, then we say that the curve coo (S1) is isotopic to the
curve co, (S1) by means of the connecting family of curves cot (S1), 0 < t < 1.
LEMMA 4.2. Suppose that there are no equilibrium states of the flow f t on the
simple closed curve Co C M. Then there exists an isotopic nondegenerate cycle Cl
of f t, and f t does not have equilibrium states on any of the curves in the connecting
family Ct, 0 < t < 1.
This can be proved from the rectification theorem and theorems on reduction
to general position [40].
DEFINITION. The index j (C, f t) of a nondegenerate cycle C with respect to
the flow f t is defined to be the number
(c, f t) = 2 (2 kex -}- in)
where kex is the number of points of exterior tangency to C of trajectories of f t,
and lain the number of points of interior tangency.
It follows from topological considerations and from the theorem on continuous
dependence on the initial conditions that the number -kex is even, and hence
the index is an integer.
The definition of the index gives us the next result directly.
COROLLARY 4.1. If C is a contact free cycle for the flow f t, then j (C, f t) = 1
The proof follows from kex = 0. El]
4.3. The index of an isolated equilibrium state. Let m0 be an isolated
equilibrium state of the flow ft. This means that there exists a simply connected
neighborhood tL(mo) of m0 in which there are no equilibrium states of f t besides
m0. By Lemma 4.2, there is a nondegenerate cycle C C U(mo) that contains m0
in its interior.
DEFINITION. The index j (mo, f t) of an isolated equilibrium state m0 of a flow
f t is defined to be the index of a nondegenerate cycle lying in a simply connected
neighborhood of the equilibrium state and containing m0 in its interior.
The next lemma shows that the definition of the index of an equilibrium state
does not depend on the choice of nondegenerate cycle satisfying the conditions of
the definition.
LEMMA 4.3. Let CD and Cl be isotopic nondegenerate cycles of the flow f t,
and suppose that that each curve of a connecting family Ct, 0 < t < 1, does not
pass through equilibrium states of f t. Then j (Co, f t) = j (Cl, ft).
EXAMPLES. 1) the index of a topological node or focus is equal to +1 (Figure
1.43, a, b) ;
2) the index of a topological saddle with four separatrices is equal to -1 (Figure
1.43, c).

THEOREM 4.1 (Bendixson). Suppose that an isolated equilibrium state m0 has


h hyperbolic sectors, and e elliptic sectors. Then its index is equal to j (mo, ft) =
2(2-h+e)
4. THE POINCARE INDEX THEORY 37

J
a) b) c)

FIGURE 1.43

FIGURE 1.44

There is a proof of Theorem 4.1 in the book [3] (English pp. 511-515).
The following result can be proved by induction.
THEOREM 4.2 (the sum of the indices). Suppose that a nondegenerate cycle C
of a flow f t bounds a simply connected domain D on a two-dimensional manifold
M, and suppose that D contains finitely many equilibrium states of f t. Then the
index of C with respect to f t is equal to the sum of the indices of the equilibrium
states in D.
COROLLARY 4.2. If the index of a nondegenerate cycle bounding a simply con-
nected domain D is nonzero, then there is at least one equilibrium state in D. In
particular, a simply connected domain bounded by a contact free cycle (according to
Corollary 4.1) contains at least one equilibrium state.
COROLLARY 4.3. Suppose that a closed trajectory l of the flow f t bounds a
simply connected domain D on M, and that f t has only isolated equilibrium states.
Then D contains at least one equilibrium state.

PROOF. By Lemma 4.2, there exists a nondegenerate cycle C isotopic to l


such that each curve in the family connecting l and C does not pass through
equilibrium states of f t (Figure 1.45). Since l is closed, the number of points
of exterior tangency to C of trajectories is equal to the number of points of interior
38 1. DYNAMICAL SYSTEMS ON SURFACES

FIGURE 1.45

tangency, and therefore j (C, f t) = 1. The required assertion follows from this and
Corollary 4.2. LI

4.4. The Euler characteristic and the Poincare index. Recall that the
Euler characteristic x(M) of a closed orientable two-dimensional manifold M of
genus p > 0 is equal to x (M) = 2 - 2p. If M is nonorientable, then X (M) = 2 - p.
THEOREM 4.3 (Poincare). Let f t be a flow with finitely many equilibrium states
m1,.. . , mj on a closed two-dimensional manifold M. Then > 1 j (mif ft) =

COROLLARY 4.4. Assume the conditions of Theorem 4.3, and let Mq C M


be a connected orientable submanifold of genus q > 0 bounded by finitely many
contact-free cycles C1,. . . , CT for the flow ft. Then

j(mZn, ft) = 2 - 2q - r,
where the summation is over all equilibrium states in Mq.
PROOF. Mq can be imbedded in a closed orientable two-dimensional manifold
Mq of genus q (Mq is obtained by "attaching" disks to Mq along the contact-free
cycles C1,.. . , CT ), and a flow f t can be defined on Mq that has r nodes and the
equilibrium states of ft that lie in Mq. The required assertion follows from Theorem
4.3 and the fact that the index of a node is equal to 1. El

COROLLARY 4.5. Assume the conditions of Corollary 4.4, and suppose that f t
does not have equilibrium states on Mq. Then Mq is homeomorphic to an annulus.
PROOF. It follows from Corollary 4.4 that 2 - 2q = r > 1. Since q > 0 is an
integer, we have that q = 0 and r = 2.

4.5. Connection between the index and the orientability of folia-


tions. In the preceding subsections the presence of a motion along the trajectories
of the flow as time changes did not play an essential role. Therefore, the defini-
tions of a nondegenerate cycle and of the index carry over in their entirety to the
same concepts for foliations. All the preceding results also are preserved, except
4. THE POINCARE INDEX THEORY 39

that "singularities of the foliation" should replace equilibrium states. An essential


difference for foliations is the possibility of nondegenerate cycles and certain sin-
gularities being nonintegral. For example, singularities with indices 1/2 and -1/2
are represented in Figure 1.46 a and b, respectively. It is clear that the index being
nonintegral is connected with the foliations being nonorientable. In this subsection
we investigate this connection in greater detail. First we give a definition of an
orientable nondegenerate cycle, which is not homotopic to zero in general.

FIGURE 1.46

Let F be a foliation on a surface M with the set z (F) of singularities, and let
C C M \ z (F) be a simple closed curve with only finitely many points of tangency
to leaves of F. Such a curve, with a direction (taken to be positive) introduced on
it is called an orientable nondegenerate cycle.
If the surface M is orientable, then the nondegenerate cycle C has a neighbor-
hood DC homeomorphic to an annulus, and C divides it into two components DC
and DC, which are also homeomorphic to an annulus (Figure 1.47). Denote by DC
(respectively, DC) the component of DC \ C to the left (right) upon traversing C
in the positive direction.

FIGURE 1.47

In order for the following definitions to agree with the preceeding ones in the
case when C bounds a simply connected domain in M we denote by DC the com-
ponent belonging to the simply connected domain, and we orient the curve C
correspondingly.
40 1. DYNAMICAL SYSTEMS ON SURFACES

For the oriented nondegenerate cycle C we introduce the concepts of points of


interior and exterior tangency in a way completely analogous to the corresponding
concepts in §4.2, except that the domains D- and D+ are replaced by DC and DC.
DEFINITION. The index of an orientable nondegenerate cycle C with respect
to a foliation F is defined to be the number

(C, F) = 2 (2 - kex + kin) ,


where kex (kin) is the number of points of exterior (interior) tangency to C of the
leaves of F.
If an orientable nondegenerate cycle C coincides with C as a set, but its orien-
tation is opposite to that of C, then
j(C,F) =2-j(C,F).
This implies that whether the index is an integer does not depend on the orientation
of the nondegenerate cycle.
The index of any nondegenerate cycle with respect to a flow is an integer.
Therefore, if there exists an orientable nondegenerate cycle with nonintegral index
with respect to some foliation F, then the family of leaves lying in M \z(F) cannot
be imbedded in a flow. But if the index of any orientable nondegenerate cycle
with respect to F is an integer, then the family of leaves lying in M \ z(F) can be
imbedded in a flow.
Lemma 4.3, which is valid for foliations, enables us to give a definition of the
index of any simple orientable closed curve, not necessarily having only finitely many
tangencies to the leaves of the foliation (for example, a curve consisting of contact-
free segments and arcs of leaves). Namely, the index of a simple orientable closed
curve Co C M (with respect to a foliation F) not passing through singularities of F
is defined to be the index j (C1, F) of any orientable nondegenerate cycle C1 C M
isotopic to Co and such that every curve in a connecting family Ct, 0 < t < 1, does
not contain singularities of F. The isotopy from Co to C1 carries the orientation of
Co in the natural way into the orientation of C1. According to Lemma 4.2, such a
nondegenerate cycle exists, and by Lemma 4.3 the definition of the index j (Co, F)
does not depend on the concrete nondegenerate cycle C1. These arguments are
valid for the index of a curve with respect to a flow, of course.
4.6. An example of a foliation that is locally but not globally ori-
entable. In this section we present a foliation F on a closed orientable surface M3
of genus p = 3 that has only two singularities of saddle type with six separatrices
each. Thus, each singularity has index j = -2 and F is locally orientable in a
neighborhood of each singularity. However, the family of regular curves of F is not
orientable and cannot be imbedded in a flow. It will be clear from the construction
how the foliation F can be constructed to be transitive.
Let us begin with a foliation Fo with five singularities on a sphere: four
"thorns", and one "tripod". We consider two disks D 1 and D2 bounded by cir-
cles S1 and S2, and the foliations F1 and F2 on these disks are pictured in Figure
1.48. We paste D1 and D2 together along S1 and S2 with the help of a homeomor-
phism S1 -4 S2 in such a way that the foliation Fo obtained on the sphere 82 does
not have leaves joining two singularities nor closed (compact) leaves (we can take
4. THE POINCARE INDEX THEORY 41

FIGURE 1.48

the pasting homeomorphism to be such that we get a transitive foliation F0, and
therefore all the subsequent foliations will also be transitive).
On the disk D2 we take a leaf Lo of F2 dividing D2 into two domains D21 and
D22, one of which, say D21, contains one singularity of thorn type, while the other
(D22) contains a tripod and two thorns (Figure 1.48). The boundary &D22 of D22
consists of the leaf Lo and a segment transversal to the leaves of F2. Denote by
A0 the simple closed curve into which &D22 passes when D1 and D2 are pasted
together. Then A0 divides 82 into two disks, which we also denote by D1 and D2.
We introduce an orientation on A0 such that the disk D2 (which contains the tripod
and two thorns) is to the left upon moving along A0 in the positive direction. It is
not hard to see that the index of the oriented curve A0 with respect to the foliation
FD is equal to the index of a singularity of thorn type; that is, j (Ao, Fo) = 1/2
(Figure 1.49).

FIGURE 1.49

Denote by u1, u2 E D2 and u3, u4, u5 E D1 the thorns of the foliation FD (see
Figure 1.49). There exists a two-sheeted branched covering ir0 : T = M2 82 with
the four branch points u1, u2i u4, and u5 (each of index two). We describe the
covering 710 and its construction in greater detail. Let U12 C D2 and U45 C D1
be arcs without self-intersections joining the respective pairs u1, u2 and 'U4, u5 and
not passing through any other singularities of F0. Moreover, U12 and U45 can be
42 1. DYNAMICAL SYSTEMS ON SURFACES

assumed to be transversal to F0. Cutting 82 along U12 and U45, we get a surface
M0,2 of genus 0 with two holes. We imbed M0,2 in the three-dimensional Euclidean
space R3 with coordinates (x, y, z) in such a way that M0,2 intersects the axis Oz
only at the points u1, u2, u4, and u5, and each component of the boundary &M0,2
is invariant with respect to the rotation yz : -R3 about the axis Oz through a
180° angle. The transformation yz has the form x = -x, y = -y, z = z. Then the
surfaces M0,2 and yz (Mo,2) have a common boundary, and the union Mo,2 Uyz (Mo,2)
is a two-dimensional torus T. By construction, the quotient space T/G(yz) of T
by the group G('yz) generated by the element yz is homeomorphic to 82, and the
natural projection ir0 : T -* T/G('yz) 82 is a two-sheeted branched covering with
the four branch points u1, u2, u4, and u5.

FIGURE 1.50

On T there exists a foliation F covering the foliation F0. Two tripods at the
points 7f -1(Y) = {Y1, Y2 } and two thorns at the points i0 1(u3) = {Vi, v2} E T
make up the singularities of F.
Since j (Ao, F0) = 1/2, both the curves Al and )'2 in the complete inverse image
ir0 1(ao) also have index 1/2 with respect to F when equipped with the orientation
induced by the covering 710.
It follows from the construction of the covering 710 that the curves Al and A2 are
not homotopic and hence not homologous to zero; that is, each of them separates
the torus T (Figure 1.51). Therefore, there exist arcs d1, d2 C T \ {A1} not passing
through singularities of F and joining the respective pairs Y1, v1 and Y2, v2 of
points. The arcs d1 and d2 can be constructed to be transversal to the foliation
F. Next, we proceed as above, cutting T along d1 and d2 and constructing a two-
sheeted branched covering : M3 -* T with the four branch points Y1, Y2, Vi, and
v2 (Figure 1.52). The foliation F on M3 covering F has only two singularities:
each being a saddle with six separatrices. However, each of the closed curves Al
and ) 2 in the complete inverse image it (A1) has a nonintegral index with respect
4. THE POINCARE INDEX THEORY 43

FIGURE 1.51

FIGURE 1.52

to the foliation F. Therefore, the leaves of F are not globally orientable; that is,
the family of regular leaves of F cannot be imbedded in a flow.
Remark. About a result of El'sgol'ts. Suppose that a flow f t on a closed
two-dimensional manifold M has only isolated equilibrium states. According to
Theorem 4.3, the sum of the indices of all the equilibrium states of f t is equal to
the Euler characteristic x (M), and the number x gives a lower bound for the
sum of the moduli of the indices of all the equilibrium states. If the equilibrium
states are of the simplest type, that is, are stable and unstable nodes or saddles
44 1. DYNAMICAL SYSTEMS ON SURFACES

with four separatrices, then the lower bound of the number of equilibrium states
can be made concrete.
Let us consider a flow f t having equilibrium states only of the simplest type.
Define k0 (k2) to be the number of unstable (stable) nodes of f t, and k1 to be the
number of saddles with four separatrices. Since the index of a node is equal to 1,
while the index of a saddle is equal to -1, we have that
(k0 + k2) -k1 =x( )
We consider the case of a closed orientable two-dimensional manifold of genus
p > 1. Then
(k0+k2)-k1 =2-2p<0.
This implies that the smallest number k1 of saddles is equal to 2 - 2p, and then
k0 = k2 = 0. Thus,
ko > 0, k1 > 2-2p, k2 > O.
If M is the sphere S2, then (k0 + k2) -k1 = 2. Therefore, the smallest value of the
sum k0 + k2 is equal to 2, and then k1 = 0. It can be shown that
ko>1, k1>0, k2>1.
Analogous estimates can be obtained for nonorientable M. All the estimates are
sharp.
The result described was obtained with the use of Betti numbers by L. E.
El'sgol'ts (An estimate of the number of singular points of a dynamical system
given on a manifold, Mat. Sb. 26 (68) (1950), 215-223), and later reproved in
[100].
CHAPTER 2

Structure of Limit Sets


In this chapter we prove the classical theorems of Maier (a criterion for recur-
rence) and Cherry (on the closure of a recurrent trajectory), and we present Maier's
estimate of the number of independent nontrivial recurrent semitrajectories. The
chapter concludes with an exposition of the Poincare-Bendixson theory, a catalogue
of limit sets and minimal sets, and an investigation of the structure of quasiminimal
sets.

§1. Initial concepts and results


1.1. The long flow tube theorem, and construction of a contact-free
cycle.
The rectification theorem (sometimes called the flow tube theorem) generalizes
to the case when a finite arc of a regular trajectory is considered instead of a single
regular point.
THEOREM 1.1 (the long flow tube theorem). Let d be a compact arc of a regular
trajectory (that is, a trajectory different from an equilibrium state) of a C' f low f t
(r > 0), and suppose that d does not form a closed curve. Then there exists a
neighborhood U of d and a C' -diff eomorphism b : U - R2 carrying the arcs in U
of trajectories of f t into trajectories of the dynamical system x = 1, y = 0.
A proof follows from the rectification theorem and the theorem on continuous
dependence on the initial conditions. The books [3] and [26] contain the detailed
proof, and we omit it.

FIGURE 2.1

Just as in the rectification theorem, the neighborhood U in Theorem 1.1 will be


called a neighborhood with the structure of a constant field, and the diffeomorphism
b a rectifying diffeomorphism.
Let f t be a flow on a two-dimensional manifold M.
45
46 2. STRUCTURE OF LIMIT SETS

DEFINITION. An open domain U C M bounded by two contact-free segments


1 and 2 and two arcs d1 and d2 of trajectories of f t such that the union d1 U
d2 U U is a simple closed curve will be called an open flow box if U does not
contain equilibrium states of f t (Figure 2.2). The union of the domain U with the
curve d1 U d2 U 1 U will be called a flow box (or closed flow box).

FIGURE 2.2

LEMMA 1.1. Let y be either a regular point or a compact arc of a regular


trajectory of the flow f, and suppose that y does not form a closed curve in the
latter case. Then there exists a neighborhood U D y that is an open flow box.
PROOF. This follows immediately from the rectification theorem and the long
flow tube theorem. 0
LEMMA 1.2. Suppose that the trajectory l of the flow f t intersects a contact-
free segment at more than one point. Then there exists a contact free cycle that
intersects 1.

FIGURE 2.3

PROOF. Let m1 and m2 be successive points of intersection of l and with


respect to time (Figure 2.3). Then the arc y of l between m1 and m2 does not
intersect . According to Theorem 1.1, there exists a neighborhood U of y in
which the trajectories of f t are arranged as a family of parallel lines (Figure 2.3).
Therefore, U contains a contact-free segment with endpoints m1, m2 E , and it
can be assumed that E is tangent to at the endpoints (Figure 2.4).
The union of with the subsegment m1m2 C (the subsegment of between
m1 and m2) gives the desired contact-free cycle, which intersects l at least at the
point m1. 0
1. INITIAL CONCEPTS AND RESULTS 47

--y
mj

FIGURE 2.4

COROLLARY 1.1. If the trajectory l intersects some neighborhood with the


structure of a constant field in at least two disjoint arcs, then there exists a contact-
free cycle intersecting 1.
COROLLARY 1.2. If the w- (a-) limit set of a nonclosed trajectory l contains a
regular point, then there exists a contact free cycle that intersects 1.
PROOF. We draw a contact-free segment through a regular point m E w(l)
(a(l)). The last condition implies that l intersects more than once, and the
required assertion follows from Lemma 1.2. 0

1.2. The Poincare mapping. Let m m2 be the arc with endpoints m1 and
m2 on a trajectory l of a flow f, and let and 2 be disjoint contact-free segments
passing through m1 and m2, respectively, such that m m2 U ) = {ml, m2}
(Figure 2.5). For definiteness it will be assumed that m2 E l+ (ml ); that is, the
point m2 is hit upon moving along l from m1 with increasing time.
By Theorem 1.1, there exists a neighborhood C of m1 on such that for
any m E the positive semitrajectory l+ (m) intersects 2 without first intersecting
. Denote by rri the first point where l+ (m) intersects 2 (Figure 2.5).

FIGURE 2.5
def
DEFINITION. The mapping P = P(m, ) : -
assigning the point m E
2 to a point m E according to the rule above is called the Poincare mapping
(induced by the flow ft).
Theorem 1.1 gives us
LEMMA 1.3. Let P = P(m, ) : - 2 be the Poincare mapping of the
contact free segment into the contact free segment 2 induced by a C'' flow f t .
Then P is a C'' -diff eomorphism of onto its range.
48 2. STRUCTURE OF LIMIT SETS

Suppose now that m1 = m2 and 1 = 2; that is, l is a closed trajectory of


ft. It follows from the theorem on continuous dependence on the initial conditions
and the rectification theorem that for points m E = 1 = 2 sufficiently close to
mo = m1 = m2 the semitrajectories 1+ (m) \ {m} going out from them intersect
(Figure 2.6). Let m be the first point where l + (m) \ {m} intersects upon moving
away from m along l+ (m) with increasing time.

FIGURE 2.6

The mapping P = P(l, ) assigning the point m to a point m is called the


Poincare mapping for the closed trajectory 1.
LEMMA 1.4. For a closed trajectory l the Poincare mapping P(l, ) induced by
a C' flow is a C'-diffeomorphism in the domain where it is defined.
PROOF. The mapping P(l, ) can be represented as a composition of mappings
P1: -and P2 :
Lemma 1.3. 0
-
(see Figure 2.7), each of them a C''-diffeomorphism by

FIGURE 2.7

If m1 m2, but 1 = = , then the definition of the Poincare mapping


P: - 2 is analogous. Namely, if the semitrajectory 1+ (m), m E , intersects
again after the point m, then we set P(m) = m, where m is the first point where
l+(m) \ {m} intersects with increasing time.
1.3. The limit sets. Let f t be a flow on a closed surface M, and let 1(m) be
the trajectory of f t passing through a point m E M.
LEMMA 1.5. The w- (a-) limit set of the trajectory 1(m) is nonempty, con-
nected, closed, and invariant (that is, consists of whole trajectories).
1. INITIAL CONCEPTS AND RESULTS 49

PROOF. The closedness (compactness) of M implies that the limit set w(l (m))
(a(l(m))) is nonempty. The fact that it is closed follows immediately from the
definition of a limit set (see §3 in Chapter 1). If ml, m2 E w (l (m)) (a(l(rn)))
are two distinct points, then any neighborhoods of them are joined by an arc of
the trajectory 1(m), which is an arcwise connected set. This implies that w(m) is
connected. The invariance of w (l (m)) follows from the continuous dependence of
trajectories on the initial conditions. 0

If l is an equilibrium state or a closed trajectory, then w(l) = a(l) = 1.


LEMMMA 1.6. Suppose that the w- (a-) limit set of the trajectory l contains a
closed trajectory la . Then w (l) (a (l)) = la .
PROOF. Assume for definiteness that to c w(l) and M is an orientable surface
(the proof for a nonorientable surface is left to the reader as an exercise). If l = le,
then the assertion is obvious, so we consider the case when l l o .
Since M is orientable, to has a neighborhood U homeomorphic to an annulus
such that to separates U into two domains. It follows from the definition of f t and
the continuity of ft that the set Fix(f t) of equilibrium states is closed, and hence
separated from lo. We take the neighborhood U small enough that it does not
contain equilibrium states.
Through some point mo E to we draw a contact-free segment such that C U
and l o intersects only at m0. Since to C w (l) , l intersects at a point m1 mo
lying in the domain o C of the Poincare mapping P(lo, ) : o - (Figure
2.8).

FIGURE 2.8

Let m2 = P (ml) . By the theorem on continuous dependence on the initial


conditions, if m1 is sufficiently close to mo, then the arc m 'm2 of l between m1 and
m2 lies in U. Since to separates U, m m2 belongs to a single component of U \ lo,
and hence m1 and m2 are in one of the segments in \ mo. Therefore, the simple
closed curve yo = m m2 U ml m2 does not intersect 10 and lies in U, where ml m2
is the subsegment of with endpoints ml and m2.
The naturally oriented curve to has index +1 with respect to the flow f t (it
follows from the proof of Lemma 1.2 that yo is isotopic to a contact-free cycle,
which has index +1 in view of Corollary 4.1 in Chapter 1). If yo bounded a simply
connected domain Do in U, then Do C U would contain an equilibrium state by
Corollary 4.2 in Chapter 1, and this contradicts the choice of the neighborhood U.
50 2. STRUCTURE OF LIMIT SETS

Therefore, the curve yo in U is not homotopic to zero and together with to bounds
an annular domain K C U (Figure 2.8).
Denote by and Y the components of the set \ m0, and let m1, m2 E + .
There are two possibilities: 1) the positive semitrajectory l+(m2) does not intersect
- in some neighborhood of m0i 2) l+(m2) intersects - at points arbitrarily close
to ma .
In case 1) we show that m2 lies on between m1 and m0. Assume not; that
is, assume that m1 lies between m2 and m0. Then l + (m2) leaves the annulus K
and cannot hit it after that since on m1m2 C all the positive semitrajectories go
out of K (Figure 2.9). But in case 1) this means that l+ (m2) does not intersect
in some neighborhood of m0, which contradicts the inclusion m0 E w (l+ (m2)) .

FIGURE 2.9

Let us prove the lemma in case 1). Thus, m2 lies on between m0 and m1, and
hence m2 E o (the domain of the Poincare mapping F). We show that the point
m3 = P (m2) lies on between m2 and m0. Indeed, the semitra j ectory l + (m2) ,
upon entering K, cannot leave it because on the segment m1m2 all the positive
semitrajectories go into K (see Figure 2.8). For the same reason, m3 m1m2, and
hence m3 E m 1 ma C .

Continuing this iteration process, we get a sequence of points mk E , k E N,


with the following properties: a) mk+1 E l+(mk); b) the point mk+1 lies on
between mk and m0 (monotonicity); c) l+(m1) does not intersect at any points
other than mk, k E N (Figure 2.10).

FIGURE 2.10
1. INITIAL CONCEPTS AND RESULTS 51

In view of the property b), the sequence mk, k E N, has a limit mE . Since
mo E w (l + (mi)), we have that m= mo. Moreover, according to the properties b)
and c), mo is the unique accumulation point of the sequence mk, k E N.
Assume that a trajectory l* to is in the w-limit set w(l). Since l+(m1) C K,
l C K. It follows from the theorem on continuous dependence of trajectories on the
initial conditions and the long flow tube theorem (Theorem 1.1) that l intersects
the segment f1 K at some point m mo, but it follows from l C w (l) that
m must be an accumulation point of the set l+(m1) f1 = U{mk}, k E N. This
contradiction (to the fact that mo is the unique accumulation point of the sequence
mk, k E N) proves the lemma in case 1).
Let us consider case 2). It follows from the preceding arguments that in this
case the semitrajectory l+(m2) leaves K and does not hit K after that. In a way
completely analogous to the construction of K we construct an annulus K bounded
by an arc rn1 rn2 of the semitrajectory l + (m2) and by the segment m1 m2 C K
(Figure 2.11). Since to C w(l), the semitrajectory l+(n2) must enter the annulus
K, and hence the point m2 on - lies between m1 and mo. The rest of the proof
is entirely analogous to that in case 1), with K, m1i and m2 replaced by K, m1,
and m2, respectively. 0

The next result is an immediate consequence of the proof of Lemma 1.6 and
the theorem on continuous dependence of trajectories on the initial conditions.

FIGURE 2.11

COROLLARY 1.3. Suppose that the conditions of Lemma 1.6 hold and fix a point
m E 1. There exists a neighborhood 'U(m) of m such that the w- («-) limit set of
any trajectory passing through 1L(m) coincides with la.
1.4. Minimal sets.
DEFINITION. A minimal set of a flow is defined to be a nonempty closed in-
variant set not containing proper closed invariant subsets.
52 2. STRUCTURE OF LIMIT SETS

The simplest examples of minimal sets are equilibrium states and closed tra-
jectories. An irrational winding on the torus provides an example of a minimal set
that coincides with the torus.
In the limit set SZ (sus(f)) of a Denj oy flow (see §3.1 in Chapter 1) each trajectory
is dense in 1 (sus (f)) . Therefore, this limit set is a minimal set. Further, it is
nowhere dense on the torus and has the local structure of the product of a closed
bounded interval and the Cantor set.
LEMMA 1.7. Let N be a minimal set for a flow f t on a compact manifold M.
Then:
1) either N = M or N is nowhere dense in M;
2) each trajectory in N is dense in N, and, moreover, N coincides with the w-
(a-) limit set of any of its trajectories.
PROOF. It follows from the invariance of N and the theorem on continuous
dependence of trajectories on the initial conditions that the boundary aN (a point
xo is in aN if any neighborhood of it contains a point in N and a point not in
N) of N is also invariant. Since N is a minimal set, either aN = N or aN = 0.
Consequently, either N does not contain interior points (and then is nowhere dense
in M), or N coincides with the set int N of its interior points. In the last case
N = int N is open and closed, so that N = M.
According to Lemma 1.5, the w- (a-) limit set of any trajectory is invariant
and closed. This and the definition of a minimal set give us the assertion 2).

DEFINITION. A trajectory l of a flow f t is said to be recurrent in the Birkhoff


sense (or B-recurrent) if for any s > 0 there is a number T = T (s) > 0 such that
the whole of l is contained in the s-neighborhood of any arc of l of time length T.
An equilibrium state and a closed trajectory of any flow are B-recurrent tra-
jectories. Each trajectory of the minimal set of a Denj oy flow is B-recurrent, as
is each trajectory of an irrational winding of the torus. B-recurrent trajectories
belong to a minimal set of the flow in all the examples given. It can be shown
that any trajectory of a compact minimal set is B-recurrent. If a trajectory l is
B-recurrent, then its closure is a compact minimal set (these results are proved in
the book [61]).
We remark that the trajectories of a noncompact minimal set can fail to be B-
recurrent. For example, any trajectory of the flow x = 1, y = 0 on j2 is a minimal
set, but is not B-recurrent.
It is known [61] that every compact invariant set of a flow contains some mini-
mal set. From this and Lemma 1.5, the w- (a-) limit set of any trajectory of a flow
on a closed surface contains at least one minimal set.
1.5. Nonwandering points. Let f t be a flow on a manifold M. We recall
that ft denotes the mapping M - M that carries each point m E M along the
trajectory 1(m) of ft to the point corresponding to the time t E Ilk (the shift by the
time t along the trajectory).
DEFINITION. A point m E M is called a nonwandering point of a flow f t if for
any neighborhood U(m) of m and any number T> 0 there is a ,t > T such that
f t [u(m)] n 'LL(m) 0.

A point that is not nonwandering is called a wandering point of ft.


v
2. THE THEOREMS OF MATER AND CHERRY 53

The set of nonwandering points of a flow f t is denoted by NW (f t) .


An equilibrium state of any flow is a nonwandering point, as is each point of a
closed trajectory.
LEMMA 1.8. Suppose that f t is a flow on a compact manifold M. The set
of nonwandering points of f t is nonempty, closed, and invariant (that is, consists
of whole trajectories). Moreover, NW(ft) contains the w- and a-limit sets of any
trajectory of f t .
PROOF. By definition, a point m E M is wandering if there exist a neighbor-
hood 'LL(m) of m and a number To > 0 such that ft [tL(m)} f1 U(m) = 0 for all
t > T. This implies that the set of wandering points is open. Consequently, the
set of nonwandering points is closed.
The invariance of the set NW(ft) follows from the theorem on continuous
dependence of trajectories on the initial conditions.
Let us consider the limit set w (l) of a trajectory l of ft. We take an x E w(l)
and a point m E 1. By the definition of the w-limit set, there is a sequence of
numbers tk, k E N, with tk -p oo such that f tk (m) - x as k - oo. Therefore, the
points f tk (m) lie in any given neighborhood U(x) of x for sufficiently large indices
k > ko . Since tk increases to infinity, for any T> 0 there is a number tki with index
aef
k1 > Ico such that t tji - tko > T. Then ft [tL(x)] l tL(x) ftki (m) = ft [fto (m)];
that is, f t [tL(x)] f1 'LL (x) 0. This proves that w (l) C NW (f t) It can be shown
.

similarly that a(l) C NW(ft) for any trajectory 1. For a compact manifold w(l)
(a(l)) 0 (Lemma 1.5), and hence NW(ft) O. 0

Trajectories belonging to the nonwandering set are called nonwandering tra-


jectories. The remaining trajectories are said to be wandering.

§2. The theorems of Maier and Cherry


2.1. Definitions of recurrence. Let f t be a flow on a surface M, and let l
be a trajectory of ft.
DEFINITION. A trajectory l (a positive semitrajectory l+) is said to be P+
recurrent or recurrent in the positive direction if it belongs to its own w-limit set,
that is, l c w(l) (1+ C w(l+)).
A trajectory l (a negative semitrajectory 1) is said to be P- recurrent or
recurrent in the negative direction if it belongs to its own a-limit set: l C a(l)
(1 C a(l )).
A trajectory is said to be recurrent if it is P+ and P- recurrent (another
common term for this is Poisson-stable).
If at least one positive (negative) semitrajectory of l is P+ (P-) recurrent,
then l itself and any one of its positive (negative) semitrajectories are also P+
(P-) recurrent.
An equilibrium state and a closed trajectory are recurrent trajectories. Re-
current trajectories and semitrajectories different from these are called nontrivial
recurrent trajectories and semitrajectories. Each trajectory of an irrational winding
of the torus and each trajectory in the minimal set of a Denj oy flow are nontrivial
recurrent trajectories (even B-recurrent).
It follows from Lemma 1.7 that any B-recurrent trajectory is recurrent.
54 2. STRUCTURE OF LIMIT SETS

There are non-B-recurrent recurrent trajectories in a Cherry flow (see §3.2 in


Chapter 1). Namely, any regular trajectory in the limit set of a Cherry flow is
recurrent in one of the directions, but is not B-recurrent, because the limit set of
a Cherry flow contains an equilibrium state (a saddle), and hence is not a minimal
set.
If an impassable grain is put on one of the trajectories in an irrational winding
of the torus (see §2.3 in Chapter 1), then in the resulting flow each regular trajectory
becomes non-B-recurrent and recurrent in one direction (or both).
aef
Recall that cl(l) denotes the topological closure of a trajectory l (cl(l) 1UDl).
An immediate consequence of the definition of the w- (o-) limit set is that
w(l) U a(l) C cl(l).
If l is P+ recurrent, then l C w (l) Since the set w (l) is closed, cl(l) C w (l) .
.

Therefore, for a P+ recurrent trajectory


(2.1) w(l) = cl(l).
Similarly, if l is P- recurrent, then
(2.2) «(l) _ ii(i).
If l is a recurrent trajectory, then
(2.3) w(l) = a(l) = cl(l).
For any trajectory l we have l C cl(l), so the equalities (2.1) and (2.2) are equivalent
to P+ and P- recurrence, respectively.
DEFINITION. A P+ or P- nontrivial recurrent trajectory is said to be locally
dense (exceptional) if its closure contains interior points (does not contain interior
points).
Local denseness means that the trajectory is dense in some domain of the
manifold. Being exceptional means that the trajectory and its closure are nowhere
dense on the manifold.
Each trajectory of an irrational winding fo of the torus is locally dense. We
obtain a more nontrivial example from fo if on one of the trajectories we put an
impassable grain and then "blow up" this grain to form a disk D, deforming the
trajectories of fo in an obvious way (Figure 2.12). Then each trajectory in T2 \ D
is locally dense and dense in T2 \ D.

FIGURE 2.12
2. THE THEOREMS OF MAIER AND CHERRY 55

There are exceptional recurrent trajectories in Denjoy and Cherry flows.


LEMMA 2.1. Suppose that a P+ (P-) nontrivial recurrent trajectory l inter-
at an interior point. Then l intersects in
sects a contact free segment or cycle
a countable perfect set of points.'
PROOF. The set l f1 is perfect because l is nonclosed and is its own limit set.
Let ... , t,,, t,,,... be the successive times when l intersects , where the index
v belongs to some set of numbers. Since each point in is regular, and hence lies in
a neighborhood with the structure of a constant field, it follows that the numbers
t,, are isolated in R. What is more, the quantity t, - t,,' is bounded below by a
positive constant. Therefore, the set of values ... , t,,, t,,' , ... is countable. 0

COROLLARY 2.1. S2Lr1T1oS6 that the P+ (P-) nontrivial recurrent trajectory l


intersects a neighborhood 1[ with the structure of a constant field. Then 1f11L consists
of a countable set of disjoint arcs of 1.
2.2. The absence of nontrivial recurrent semitra jectories on certain
surfaces. All the results of this subsection are based on the following simple result.
LEMMA 2.2. Let f t be a flow on a surface M, and let C be a contact-free cycle
for ft. If C intersects some trajectory at more than one point, then C does not
separate M (that is, the set M \ C is connected).
PROOF. Since C is a contact-free cycle, it follows from the rectification theorem
that C has a neighborhood U homeomorphic to an annulus and such that U \ C
consists of two components Ul and U2. If it is assumed that C separates M into two
submanifolds M1 and M2, then one component, say U1, lies in M1, and the other
(U2) in M2. On the other hand, all the trajectories of f t intersect the contact-free
cycle C, passing from one component, say U2, into the other (U1). According to
the assumption of the lemma, there is a path (along an arc of a trajectory) from Ul
to U2 that does not intersect C (Figure 2.13), and this contradicts the assumption
that C separates M into the two components M1 and M2. 0

FIGURE 2.13

1 Recall that a set N is said to be perfect if any point x E N is an accumulation point of the
rest of the set, N \ {x}.
56 2. STRUCTURE OF LIMIT SETS

LEMMA 2.3. Let l be a P+ (P-) nontrivial recurrent trajectory. Then:


1) there exists a contact free cycle intersecting l;
2) any contact-free cycle intersecting l does not separate the surface.
PROOF. This follows from Lemmas 1.2, 2.1, and 2.2. 0

REMARK. It follows from the proof of Lemma 1.2 that a contact-free cycle can
be drawn through any given point of the trajectory 1.
LEMMA 2.4. Let f t be a low on the surface M, and let G C M be a subman-
ifold homeomorphic to the sphere S2 with finitely many holes (perhaps none). (In
particular, if G = S2, then M = S2). In this case G cannot entirely contain a
nontrivial recurrent semitrajectory.
PROOF. Assume the contrary. Then, by Lemma 2.3, there exists a contact-
free cycle C C G that does not separate G. But this contradicts the Jordan curve
theorem, which says that any simple closed curve separates a sphere with finitely
many holes (perhaps none) into two domains. 0

It follows from Lemma 2.4 that there are no flows on the sphere with nontrivial
recurrent semitrajectories (this was proved in [36]). According to § § 2.3 and 2.4 in
Chapter 1, all other closed orientable surfaces have transitive flows, and hence flows
with nontrivial recurrent semitrajectories. Among the closed nonorientable surfaces
there are two on which such flows are absent, namely, the projective plane (a closed
nonorientable surface of genus p = 1) and the Klein bottle (a closed nonorientable
surface of genus p = 2). For the projective plane this follows from the fact that the
sphere is a two-sheeted covering for it. As for the Klein bottle, the torus is a two-
sheeted covering for it on which there can be nontrivial recurrent semitrajectories,
and hence the absence of such semitrajectories on the Klein bottle necessitates a
special treatment. In the absence of equilibrium states on the Klein bottle this
proposition was proved by Kneser [90], while in the presence of equilibrium states
it was proved independently in 1969 by Markley [95] and Aranson [8]. In 1978
the result was reproved by Gutierrez [85]. At the same time, on any nonorientable
closed surface of genus p > 3 there exists a transitive flow (the construction of such
flows is left to the reader as an exercise). We sum up these facts as a lemma.
LEMMA 2.5. The following three surfaces do not have flows with nontrivial
recurrent semitrajectories:
1) the sphere (an orientable surface of genus 0);
2) the projective plane (a nonorientable surface of genus 1);
3) the Klein bottle (a nonorientable surface of genus 2).
All other closed surfaces (orientable or not) have flows with nontrivial recurrent
semitrajectories (even transitive flows).
PROOF. A flow having nontrivial recurrent trajectories can be constructed on
a closed nonorientable surface of genus p > 3 by starting from a Denjoy flow on the
torus and attaching the necessary number of Mobius bands in the domain T2\(ft),
where 1(f t) is the minimal set of the Denjoy flow f t.
We prove that on the projective plane RP2 there are no flows with nontrivial
recurrent semitrajectories. Assume the contrary: suppose that the flow f t on
has such semitrajectories. Denote by r : S2 a two-sheeted (unbranched)
v
2. THE THEOREMS OF MAIER AND CHERRY 57

covering, and by f' the flow on S2 covering f t . Our assumption implies that there is
a nonclosed semitrajectory l ( ) of f t on S2 whose limit set contains a regular point.
By Lemmas 1.2 and 2.2, there exists on S2 a simple closed curve not separating
S2, and this contradicts the Jordan curve theorem.
We show that the Klein bottle K2 does not have flows with nontrivial recurrent
semitrajectories. Of the three proofs in [8], [95], and [85] we present the proof of
Gutierrez [85].
Assume the contrary: let f t be a flow on K2 with a nontrivial recurrent semi-
trajectory l+. In view of Lemma 1.2, there exists a contact-free cycle C for f t that
intersects 1+. Since C is a contact-free cycle, it has a cylindrical neighborhood in
K2 (that is, C is a two-sided closed curve). Repeating the proof of Lemma 2.2,
we show that C does not separate K2. Therefore, K2 \ C is an annulus with two
boundary components C1 and C2, and to obtain a Klein bottle from K2 \ C we must
paste C1 and C2 together by means of an orientation-preserving homeomorphism
Sp : C1 - C2 (Figure 2.14).

FIGURE 2.14

Let m1, m2, m3 E l+ be successive points of intersection of l+ with C, and let


m1 m2 be the arc of C containing m3. By virtue of the identification ,o the points
m1, m2, and m3 are arranged on C1 and C2 as shown in Figure 2.14. This implies
that the semitrajectory l+(m3) intersects C only on the arc m3m2 C mlm2, which
does not contain m1. But this contradicts the recurrence of the semitrajectory l+,
whose w-limit set contains m1. 0

2.3. The Cherry theorem on the closure of a recurrent semitra jec-


tory. The purpose of this subsection is to prove the following theorem of Cherry
[80].

THEOREM 2.1. The topological closure cl(l) of a P+ (P-) nontrivial recurrent


semitrajectory l contains a continuum of nontrivial recurrent trajectories, each of
which is dense in cl(l) and contains l in its limit set.
We first prove some auxiliary statements.
DEFINITION. Let U be a neighborhood with the structure of a constant field.
Two sets U, U2 C U are said to be not coupled with respect to U if U does not
contain arcs of trajectories intersecting both U1 and U2 (Figure 2.15).
58 2. STRUCTURE OF LIMIT SETS

FIGURE 2.15

In Figure 2.15 the sets 11, U2 and the sets l1, u3 are not coupled with respect
to 1, but u2 and u3 are coupled with respect to 1.
By a closed disk we mean a set homeomorphic to the closed unit disk. Let int D
be the interior of a set D. In what follows we take l to be a positive semitrajectory
and denote it by l+.
LEMMA 2.6. Let 1 be a neighborhood with the structure of a constant field
intersecting the P+ nontrivial recurrent semitrajectory l+. Then for any closed
disk D C 1 with int D f1 l+ 0 there exist closed disks Do, D1 C D that are not
coupled with respect to 1 and such that int Di f1 l+ 0 for i = 0, 1.
PROOF. It follows from Lemma 2.1 and Corollary 2.1 that int D intersects l+
in a countable set of disjoint arcs of l+, and this yields the required assertion. U

Fix a metric on M, and denote by 11(m) the E-neighborhood (e > 0) of a point


m E M.
If a flow f t is given on M, then for a fixed number t E Ilk and a subset N we
denote by ft(N) the shift of the set N along trajectories of f t by the time t.
LEMMA 2.7. Fix a point mo on a P+ nontrivial recurrent semitrajectory l+,
and let D be a closed disk whose interior intersects l+. Then for any numbers rl > 0
andT > 0 there exists a closed disk (D, rl,T) C D such that: a) int (D,ij, T) f1
l+ 0; b) rl,T)] C U(mo) for some t = -t1 < -T and t = t2 >T, and
(D, rl, T) can have an arbitrarily small diameter.
PROOF. Since l+ is a P+ recurrent semitrajectory, there exists a t1 > T such
that f t 1(mo) E int D. By the theorem on continuous dependence of trajectories on
the initial conditions, there is a neighborhood 1(m1) of the point m1 = ft1(mo)
such that f _t1(1(m1)) C 1(mo) (Figure 2.16). Similarly, there exists a t2 >T
such that f t2 (m1) E 1(mo) and f t2 [112 (m1)] C U(mo) for some neighborhood
12 (m1) .

We take a closed disk (D,ij, T) encircling the point m1 and lying in 1(mi)fl
12 (m1). Then rl, T)] C 1(mo) for t = -t1 <-T and t = t2 >T. Since
m1 E int (D, rl,T)l1+, it follows that int (D, rl,T)l1+ 0. Obviously, (D, rl, T)
can have arbitrarily small diameter. 0

PROOF OF THEOREM 2.1. We take numerical sequences rjl > 72 > >
rln>... withrln-OandO<T1 <T2<<Tn<... within-oo,a
v
2. THE THEOREMS OF MATER AND CHERRY 59

2Cmo)

FIGURE 2.16

point mo E l+, and a neighborhood U with the structure of a constant field such
that U n l+ 0 and mo cl (c) def= D. By Lemma 2.7, there exists a closed disk
(D, ij1,T1) C D of diameter less than i1. According to Lemma 2.6, there are closed
disks Do, D 1 C (D, )1,T1) that are not coupled with respect to U and such that
int DZ n l+ 4 0 for i = 0,1 (Figure 2.17). Again using Lemma 2.7 for the numbers
72 and T2 and the closed disks Do and D 1, we get closed disks (DZ, )2, T2) C Di
(i = 0,1) with diameter less than 7)2, and so on. This process of obtaining closed
disks is pictured in Figure 2.18. As a result we have a family of closed disks
i E {0,1 }, with the following properties:

FIGURE 2.17

1) Dzl Zn-lZn C for both the values in = 0 and in = 1;


2) the diameter of D21..,Zn is less than rjn;
3) for Dz1there exist to < -Tn and to > Tn such that ft' (D1 .
,.Zn) C
U (mo) and ft'' (DjI...Zn) C urn (mo);
60 2. STRUCTURE OF LIMIT SETS

U
T1
C-

U U

L v v
U U U U
?,1) OL'2 '1,)
(1 v (1 v v v U )
ooo
v
oo i
U
4l0
U
o!!
U
cioo
v v
!!0
v
lii
v

FIGURE 2.18

4) the closed disks DZ1and are not coupled with respect to U if at


least one of the corresponding indices is not the same, ik jk .
With each sequence i1,. . , in,... of 0's and 1's we associate the sequence of
closed disks

(2.4) D21 , L2122 , . . , DZ1 ...2n . . . ,

which by 1) and 2) is a nested sequence of closed sets with diam(Dil,,,) - 0. By


Cantor's theorem, the intersection nD21of all the disks in the sequence (2.4)
consists of a single point Qi1...zn,,, .
It follows from the property 4) that different sequences i1i ... , in, ... (ik E
{0,1 }) determine different points Since the set of all possible sequences
of 0's and 1's has the cardinality of a continuum, so does the set of points QZ1 . , , .
We consider some point Q = QZ1 By construction, int DZ1 f1 1+ 0,
and hence Q E cl(l+). This and the continuous dependence of trajectories on the
initial conditions give us that the trajectory 1(Q) through Q lies in cl(l+). Since
the w- (a-) limit set is closed, we then get that
(2.5) w(l(Q)) U a(l(Q)) C cl(l(Q)) C cl(l+).
By the property 3), ft' (Q) E U(mo) and ft'' (Q) E U(mo). Since r1n - 0,
it follows that mo E w (l (Q)) f1 a (l (Q)) . The w- (a-) limit set is invariant, so
l + C w (l (Q)) f1 a (l (Q)) , and, consequently,

(2.6) cl(l+) c w(l(Q)) n «(l(Q)).


Comparing (2.5) and (2.6), we get that
cl(l+) = cl(l(Q)) = w(l(Q)) = a(l(Q));
v
2. THE THEOREMS OF MAIER AND CHERRY 61

that is, 1(Q) is a recurrent trajectory (see (2.3) in §2.1) which is dense in cl(l+).
The inclusion l+ C cl(l(Q)) implies that 1(Q) is a nonclosed trajectory.
It remains to show that the set of trajectories 1(Q) has the cardinality of a
continuum.
We take two different points Q2.. and Q3132... and let i7
, jn be the first
distinct indices. Then in view of the property 4) the respective points Q2122.., and
Q3132... lie in closed disks D21,.,Zn and that are not coupled with respect to 'IL,
and hence they belong to different arcs of trajectories in the neighborhood 'IL. Thus,
U contains a continuum of arcs of trajectories on which the points Q = Q2122... lie.
According to Corollary 2.1, each nontrivial recurrent trajectory intersects U in a
countable family of arcs; therefore, the set of distinct trajectories 1(Q), Q = Q1...,
has the cardinality of a continuum. 0

2.4. The Mater criterion for recurrence. Let f t be a flow on an orientable


compact surface M of genus p > 1. If l is a P+ (P-) nontrivial recurrent trajectory,
then it contains a regular point in its w- (a-) limit set and lies in the limit set of
some trajectory (indeed, l C w (l) (a(l)), and any point of l is regular). Maier [55]
proved that these necessary conditions for recurrence are also sufficient.
To prove Mater's theorem we need the following result.
LEMMA 2.8. Let M be a compact orientable surface of genus p > 1.
a) If the boundary DMA consists of a single component K, then any 2p simple
(that is, without self-intersections) disjoint arcs with endoints in K combine to
distinguish a flat simply connected domain on M.
b) If DMA consists of two components K1 and K2, then any 2p + 1 simple
disjoint arcs joining K1 and K2 combine to distinguish a flat simply connected
domain on M.
c) If aM consists of two components K1 and K2, and there is a countable
family of disjoint simple arcs joining K1 and K2 on then there exist two arcs
in this family that bound a simply connected domain on M.
PROOF. We prove a) for a surface M1 of genus p = 1 (a torus with a hole). If
a simple arc d1 with endpoints on DM1 does not bound a disk on M1, then cutting
M1 along d1 gives an annulus K2. Let d2 be a simple arc on M1 that is disjoint
from d1 and has endpoints on DM1. When M1 is cut, the arc d2 passes into an
arc d2 C K2 with endpoints on 9K2. It follows from geometric considerations that
either the endpoints of d2 lie on the same component of DK2, and d2 (together with
aK2) distinguishes a simply connected domain on K2, or the endpoints of d2 lie on
different components of DK2, and then cutting the annulus along d2 gives a disk.
Thus, a) is proved for a torus with a hole (p = 1). We suppose that the assertion
has been proved for surfaces of genus 1, ... , p -1 > 1, and we prove it for a surface
M of genus p > 2.
Let d1i ... , d2 be disjoint simple arcs on M with endpoints on DMA. If one
of these arcs separates M into two surfaces M and M 2 of respective genera P1
1

and p2, then in view of the equality P1 +p2 = p one of the surfaces has at least
2pi disjoint simple arcs that combine to distinguish a simply connected domain on
(and hence on Mr), because of the induction hypothesis for p2 <P .

Suppose that none of the arcs d1i ... , d2 separates M. Cutting M along d1,
we obtain a compact surface M,_1 of genus p - 1 with two boundary components
K1 and K2. Two cases are possible: 1) the endpoints of all the arcs d2,. . . , d2 ,
62 2. STRUCTURE OF LIMIT SETS

belong to a single component, say K1; 2) at least one of d2,. . , d22 (say d2) joins
.

K1 to K2. In case 1) we use a disk to paste closed one hole of the surface
along the component K2. On the resulting surface M_1 of genus p - 1 there are
2p - 1 disjoint simple arcs with endpoints on K1 = DM_ 1. By the induction
hypothesis, these 2p - 1 > 2p - 2 arcs distinguish a simply connected domain on
Mp_ 1. In case 2) we cut M_1 along d2. We get a surface M_1 of genus p -1 with
a single boundary component and 2p - 2 holes that, by the induction hypothesis,
distinguish a simply connected domain. This proves a).
The assertion b) follows from a), since cutting NYC, along one arc yields a surface
of genus p with a single boundary component and 2p disjoint simple arcs.
According to b), cutting M along 2p + 1 arcs leads to a flat simply connected
domain on which a countable family of disjoint simple arcs is left. This gives us c).
0
Following Gutierrez, we present the
DEFINITION. Let be a contact-free segment or a contact-free cycle for a flow
f, and let l be a trajectory of f t that intersects at points m1i m2 E l f1 . The
arc m 'm2 of l with endpoints m1 and m2 is called a - arc if m 7m2 n = {m1, m2 },
that is, m m2 intersects only at the endpoints m1 and m2 (Figure 2.19).

FIGURE 2.19

Suppose that a flow is given on a compact surface M.


THEOREM 2.2 (the Maier criterion for recurrence). If a nonclosed trajectory
l has a regular point mo in its w- (a-) limit set and lies in the limit set of some
trajectory l', then l is a P+ (P-) recurrent trajectory.
PROOF. We draw a contact-free segment through m0. For definiteness as-
sume that mo E w (l) . Then contains a sequence of points mi -p mo (as i - oo)
corresponding to an unbounded monotonically increasing sequence of times when l
intersects .
The part of between p, q E is denoted by pq.
We show that for any s-arc jq of the trajectory l the simple closed curve pgUpq
cannot bound a simply connected domain on M (it is not homotopic to zero), and
it cannot even separate M into two submanifolds (it is not homologous to zero).
Indeed, otherwise no trajectory (including l') can intersect pq in more than two
points (Figure 2.20). But this contradicts the condition p, q E w(l') (a(l')).
From what has been proved it follows that the genus of M is at least 1.
2. THE THEOREMS OF MAIER AND CHERRY 63

FIGURE 2.20

We now assume that the theorem is false, that is, l is not a P+ recurrent
trajectory. Then there is an interval J on that contains the point m1 E l f1
and does not contain any other points of the intersection l+ (m1) l (Figure 2.21).
Passing to a subsequence and changing the notation if necessary, we can assume
that all the points in {m}° belong to a single component of \ {mo} and form a
monotone sequence; that is, for each i = 2, 3, ... the point m2+1 lies on between
m2 and mo.

FIGURE 2.21

Assume for definiteness that l C w(l') (the proof is analogous in the case l C

We show that there exist a subsequence of points E {m2}, k = 1, 2, ... ,


and a sequence {lk } 1 of disjoint arcs of l' such that:
1) for k E N the endpoints of 13k lie on J and are the only points where
intersects J;
2) intersects between m3k and but does not intersect between
mo and m3k+l (Figure 2.22).
Indeed, since ml E w(l'), there exists an arc of l' satisfying 1). By the
theorem on continuous dependence of trajectories on the initial conditions, the arc
can be chosen to intersect between mo and m2. This implies the existence of
a point mil E {m}° satisfying 2). If we have already constructed arcs
k > 1, and points m1,. . , mjk satisfying the required conditions, then we again
l,
.

get from the inclusion m1 E w(l') and the theorem on continuous dependence of
trajectories on the initial conditions that there is an arc C l' that does not
intersect the arcs but intersects between mo and Since
is compact, there is a point E {m}° such that intersects between
64 2. STRUCTURE OF LIMIT SETS

FIGURE 2.22

m3k+l and but does not intersect between m0 and m3k+l+1 Continuing
in this way, we obtain the required family {lk } 1 of arcs.
Let us cut the surface M along J, and denote by M' the resulting surface with
boundary K. The arcs k E N, pass into arcs (which we also denote by 13k) on
M' with endpoints on K. According to Lemma 2.8, for some k0 the family of arcs
... , 0 distinguishes a simply connected flat domain G on M', and, moreover,
we can assume without loss of generality that is a part of the boundary of G.
Since intersects between and +1, while the remaining arcs of the
family distinguishing a simply connected domain do not intersect between m0
and it follows that one of and +1 lies interior to G (Figure 2.23).
By assumption, the semitrajectory l+ (m1) has at most one point of intersection
with the boundary of G (possibly at m1). Therefore, the whole of l+ (m1) must lie
in the flat simply connected domain G (as must m0). Then any s-arc pq of l+ (m1)
intersecting only at the endpoints p and q combines with the segment pq C to
bound a disk on NYC, and this is impossible. 0

FIGURE 2.23
v
2. THE THEOREMS OF MAIER AND CHERRY 65

2.5. The Mater estimate for the number of independent nontrivial


recurrent semitrajectories.
THEOREM 2.3. Let a flow be given on a compact surface M. If a P+ (P-)
nontrivial recurrent trajectory l' contains a P+ or P- nontrivial recurrent trajectory
l in its w- (a-) limit set, then l' is contained in the w- (a- ) limit set of 1.
PROOF. For definiteness we assume that l and l' are P+ recurrent trajectories
(the proofs are analogous in the other three cases P--P+, and P--P-).
According to Lemma 2.3, there exists a contact-free cycle C that is not homol-
ogous to zero and intersects 1. Since l C w(l'), it follows that C f1 l' 0.
We take a point m1 E l' f1 C and assume that the theorem is false. Then there
is an interval J C C such that m1 E J and J f1 l = 0. Since C f1 l 0 and l is
a P+ nontrivial recurrent trajectory, there is a point m0 E w(l) on C. The fact
that m1 E J implies that l' intersects J in a countable set of points. An argument
analogous to that in the proof of Theorem 2.2 then shows that either a positive
semitrajectory l+ C l intersects J, or l lies in a flat simply connected domain, which
is impossible. 0

Theorem 2.3 means that if one nontrivial recurrent semitrajectory is a limit for
another nontrivial recurrent semitrajectory, then the second is a limit for the first.
DEFINITION. Nontrivial recurrent semitrajectories l1 and 4) are said to be
independent if they are not limits for each other; that is, neither lies in the limit
set of the other.
According to Theorem 2.3, the relation of dependence of nontrivial recurrent
semitrajectories is an equivalence relation, and hence the set of nontrivial recurrent
semitrajectories of a flow is broken up into equivalence classes. The following the-
orem of Mater gives an upper estimate for the number of these equivalence classes.
THEOREM 2.4. On a closed orientable surface M of genus p > 0 there cannot
be more than p independent nontrivial recurrent semitrajectories.
PROOF. According to Lemma 2.4, there are no nontrivial recurrent semitra
jectories on a surface M = S2 of genus p = 0. Therefore, let p > 1.
Assume the contrary, that M has p -I-1 independent nontrivial recurrent semi-
trajectories lv,... Since these semitrajectories are independent, we can draw
through any point m2 E l a contact-free segment >2Z, i E {1,. . . , p + 1}, that is
disjoint from l for i j. We take a -arc AB of the semitrajectory l,
k = 1, ... , p -I- 1, and we form the simple closed curve KZ = AZBZ U AZBZ, where
AZBZ C >2Z is the part of Z between AZ and B.
We show that l intersects AZBZ C j, i = 1, ... , p, at infinitely many points.
For since AZ lies in the limit set of the semitrajectory l , the assumption int AZBZ n
1 $ ) = 0 implies that l intersects a segment of the complement >iZ \ AZBZ at points
arbitrarily close to AZ (Figure 2.24). It follows from the orientability of M and the
theorem on continuous dependence of trajectories on the initial conditions that l
intersects AZBZ at points arbitrarily close to BZ (infinitely many times).
66 2. STRUCTURE OF LIMIT SETS

FIGURE 2.24

Since l intersects AiB2 C KZ infinitely many times, each curve K1,... , Kp


does not separate M along the curves K1, ... , K
leads to a surface M0,2of genus 0 with 2p holes.
The surface M0,2is homeomorphic to a flat (2p - 1)-connected domain and
contains a nontrivial recurrent semitrajectory l4, which contradicts Lemma 2.4.
a
The estimate of the maximal number of independent nontrivial recurrent semi-
trajectories in Theorem 2.4 cannot be improved because on any orientable closed
surface M of genus p > 1 there exists a flow with p independent nontrivial recur-
rent trajectories. Such a flow can be constructed by starting from a Denjoy flow f t
on the torus and attaching p - 1 tori with a hole to the domain T 2 \ 1 (f t) , each
of which also has a Denj oy flow given on it; here 1(ft) is the minimal set of f t
(Figure 2.25).

FIGURE 2.25

§3. The Poincare-Bendixson theory


By the Poincare-Bendixson theory we understand the investigation of the pos-
sible behavior of individual semitrajectories (and trajectories) and a description
of their limit sets. At the end of the section we give a catalogue of limit sets of
trajectories and a catalogue of minimal sets.
3. THE POINCARE-BENDIXSON THEORY 67

3.1. The Poincare-Bendixson theorem.


THEOREM 3.1. If the w- (a-) limit set of a nonclosed positive (negative) semi-
trajectory l+ (l-) of a flow on the sphere S2 does not contain equilibrium states,
then w(l) (respectively, a(1)) is a closed trajectory.
PROOF. Assume the contrary. Then there is a nonclosed trajectory ll in w(l+).
Since w (l) 0, it follows from the inclusion w (l) C w (l) and the condition of the
theorem that w(l) contains a regular point. Then Theorem 2.2 gives us that ll is
a P+ nontrivial recurrent trajectory. This contradicts Lemma 2.4, by which there
are no such trajectories on the sphere. D

3.2. Bendixson extensions. Denote by (m) a contact-free segment drawn


through a regular point in E M of a flow ft. The point in divides (m) into two
segments >L (m) and >R (m) (left and right), as shown in Figure 2.26.

FIGURE 2.26

Let mo be an isolated equilibrium state of ft. Denote by U(mo) a neighborhood


of mo diffeomorphic to an open disk with smooth boundary.
We consider a semitrajectory l+ with w(l+) = mo.
DEFINITION. The semitrajectory l+ is said to be extendible to the right (left)
with respect to the neighborhood U(mo) if for any in E l+ f1 U(mo) there exists a
contact-free segment (in) C U(mo) such that any positive semitrajectory begin-
ning on R (m) (respectively, on L (in)) leaves U(mo) as the time increases (Figure
2.27).

FIGURE 2.27
68 2. STRUCTURE OF LIMIT SETS

There is an analogous definition for a negative semitrajectory l - with a(l-) =


m0.
Obviously, if a semitrajectory 10 is extendible with respect to a neighborhood
U(m0), then it is extendible with respect to any neighborhood U' (m0) C U(m0).
THEOREM 3.2. Let m0 be an isolated equilibrium state, and let U(m0) be a
neighborhood of m0 not containing other equilibrium states. Then there are only
finitely many semitrajectories tending to m0 and extendible with respect to tL(m0).
PROOF. Assume the contrary. Suppose that there are infinitely many semi-
trajectories l$}, n E N, that are extendible with respect to U(m0). We can as-
sume without loss of generality that all the l$} are positive semitrajectories, and
w(l) = m0. For each semitrajectory l,+z there is a last (with increasing time) point
An where l,+z intersects the boundary aU(m0) (Figure 2.28). Let A be an accumula-
tion point of the sequence An, n E N. It follows from the continuous dependence of
the trajectories on the initial conditions that l+(A) cannot leaveU(m0). Therefore,
there is a point in E l + (A) n u (m0) , and through it we draw a contact-free segment
(m) C U(m0). Since A is an accumulation point of the sequence {A}?°, (m)
intersects infinitely many of the semitrajectories l.

FIGURE 2.28

Let ml, m2, and m3 be the last (with increasing time) points where the re-
spective semitrajectories l,+zi = l+ (ml ), l2 = l+ (m2 ), and l3 = l+ (m3) intersect
J(m), and suppose that m2 lies on (in) between ml and m3. The semitrajectories
l+(ml) and l+(m3), the point m0, and the segment m1m3 C (m) bound a domain
D in tL(m0) (a "Bendixson bag"), and l+ (m2) enters D C U(m0) and cannot leave
it again (Figure 2.29). This contradicts the extendibility of ln2 U
.

Let m0 be an equilibrium state, and let l + and 1j be semitrajectories such that


m0 = w(l) = a(li ).
DEFINITION. The semitrajectory 11 is called a Bendixson extension of l+ to
the right (to the left) with respect to the neighborhood tL(m0) if for any points
in E l+ n U(m0) and ml E lj n U(m0) there exist contact-free segments gy(m),
>(m1) C tL(m0) such that any semitrajectory l+ (m) with m E R (in) (respectively,
m E >L (m)) intersects R(m1) (respectively, L (ml)) without first leaving tL(m0 ),
and the first point where l+ (m) intersects R(m1) (respectively, >L (ml)) tends to
ml as m -* in (Figure 2.30).
3. THE POINCARE-BENDIXSON THEORY 69

FIGURE 2.29

FIGURE 2.30

The semitrajectory lj is called a Bendixson extension of l+ to the right (to the


left) if there exists a neighborhood U(mo) such that lj is a Bendixson extension of
l+ to the right (to the left) with respect to any neighborhood U' (mo) C U(mo).
A Bendixson extension is defined similarly for the negative semitrajectory lj
with a(li) = mo. In particular, the semitrajectory l+ is a Bendixson extension of
lj (to the right) with respect to U(mo) in Figure 2.30.
LEMMA 3.1. Suppose that a semitrajectory l+ with w(l) = mo is extendible
to the right (to the left) with respect to a neighborhood U(mo) not containing equi-
librium states other than mo Then there exists a negative semitrajectory l i with
.

a(li) = mo that is a Bendixson extension of l+ to the right (to the left) with respect
to tL(mo).
PROOF. Suppose that l+ is extendible to the right with respect to tL(mo). We
take a point in E l+ n U(mo) and a contact-free segment (in) C U(mo) satisfying
the definition of extendibility of the semitrajectory l+. Then all the semitrajectories
l+(m) with m E j(in) leave tL(mo).
Assume that m2 -* m, m2 E R(m), and let AZ be the first point where
l + (m2) intersects au (mo) (Figure 2.31). Denote by A an accumulation point of
the sequence of points AZ, i E N. We show that the negative semitrajectory l - (A)
does not intersect gy(m). Since the neighborhood tL(mo) is simply connected, all
70 2. STRUCTURE OF LIMIT SETS

the semitrajectories l+(m2) beginning with some index i0 intersect >(m) only at
rn, i > io. If we assume that l-(A) intersects (m) at a point m*, then rn -* m*,
and hence m* = in, which is impossible.

FIGURE 2.31

It follows from the theorem on continuous dependence of trajectories on the ini-


tial conditions and from the inclusions rn A2 C U(mo), i E N, that l - (A) C U(mo),
and hence a (l - (A)) C u (mo) . By what was proved earlier, (m) f1 a (l - (A)) = 0.
This and the simple connectedness of U (mo) give us that a (l - (A)) does not contain
regular points; therefore, a(1-(A)) = m0. Since trajectories depend continuously
on the initial conditions, l - (A) = l j is a Bendixson extension of l + to the right. U

3.3. Separatrices of an equilibrium state.


DEFINITION. A positive semitrajectory l+ is called an w-separatrix of an equi-
librium state mo if w(l+) = m0 and if for any point in E l+ there is an e > 0 such
that for any 6-neighborhood tL5 (S > 0) of in there exists a semitrajectory going
out of U6 that leaves the e-neighborhood of l+ with increasing time (Figure 2.32).

oE(e+)

FIGURE 2.32

The definition of an a-separatrix l - of an equilibrium state m0 with a(1) = mo


is analogous.
For sufficiently small S > 0 the component l of l + f1 Ub containing in divides
U6 into two domains U5, L and U5, j as shown in Figure 2.33.
3. THE POINCARE-BENDIXSON THEORY 71

FIGURE 2.33
If in the preceding definition there exists for any sufficiently small S > 0 a
semitrajectory going out of Ub,R (U5,L) and leaving the e-neighborhood of l+ with
increasing time, then l+ is called a right-sided (left-sided) separatrix. A separatrix
that is right-sided and left-sided is said to be two-sided.
LEMMA 3.2. A semitrajectory l+ (1-) tending to an isolated equilibrium state
m0 is an w- (a-) separatrix (right-sided or left-sided) if and only if there exists a
neighborhood U(mo) of m0 with respect to which l+ (1-) is extendible (to the right
or to the left).
PROOF. This follows from the above definition and the theorem on continuous
dependence of trajectories on the initial conditions. U
The next result is a consequence of Theorem 3.2 and Lemmas 3.1 and 3.2.
LEMMA 3.3. An arbitrary w- (a-) separatrix of an isolated equilibrium state
has a Bendixson extension to the right or to the left that is an a- (w-) separatrix
of the same equilibrium state.
In the next lemma we give a sufficient condition for a semitrajectory tending
to an isolated equilibrium state to be a separatrix, along with a sufficient condition
for an equilibrium state to have separatrices.
LEMMA 3.4. Let m0 be an isolated equilibrium state.
a) If mo = w( l) (a( 1)), where l is a regular trajectory, and l lies in the limit set
of some semitrajectory 10, then l is an w- (a-) separatrix of m0, and a Bendixson
extension of it (to the right or to the left) together with m0 also lies in the limit set
of 10.
b) If mo lies in the w-limit set of some nonclosed trajectory 1, but mo w(l),
then there exist an w-separatrix l1 and an a-separatrix l2 of m0 such that l1 U {mo } U
12 C w(l), and l2 is a Bendixson extension (to the right or to the left) of l1.
PROOF. a) We take an e-neighborhood Oe (mo) of m0 containing no equilibrium
states other than m0 and such that there are points of l outside Oe (mo) . For
definiteness we assume that m0 = w( l) and l C w (l) .

We draw a contact-free segment >(m) C Oe (mo) through a point in E 1 f1


Oe (m0) . Since in E w (l) ,
at least one of the segments >L (m) and >R (m) is inter-
sected by a semitrajectory l+ C l at points arbitrarily close to in. Suppose that l+
intersects >R (m) at points arbitrarily close to m.
We show that any semitrajectory l(m), m E >R(m), either leaves Oe(mo) or
tends to m0, that is, w(l+(m)) = m0. Assume the opposite. Then w(l+(m)) C
72 2. STRUCTURE OF LIMIT SETS

Oe (mo ), and w (l + (m)) contains a regular point, through which we draw a contact-
free segment J C Oe (mo) . Two cases are possible: 1) l+(m) is a closed trajectory;
2) l+(m) is a nonclosed trajectory.
In case 1) l + (m) bounds a simply connected domain D in Oe (mo) . By the
choice of the neighborhood O( m), , l has points outside O( m),, and hence D
does not contain mo (mU "is joined by" an arc of l to the boundary a0 (mo), and
l f1 l+ (m) = 0). On the other hand, D contains an equilibrium state by virtue
of Corollary 4.3 in Chapter 1, and this contradicts the fact that Oe (mo) does not
contain equilibrium states other than mo .
In case 2) the semitrajectory l+(m) intersects J infinitely many times. An
arbitrary :1-arc pq of l+ (m) together with the segment pq C J forms a simple closed
curve y bounding a simply connected domain D in Oe (mo) (Figure 2.34). Since an
w-limit set is invariant, w (l + (m)) has a regular point not lying on l + (m) C l (in
particular, this point can lie on l - (m) f1 Oe (mo)) . Therefore, we take the contact-
free segment J to be disjoint from l + (m) . Then 1+ (m) does not intesect y, and
hence D does not contain mo. On the other hand, the index of y with respect
to the flow is equal to 1 (see §4.5 in Chapter 1), and in view of Corollary 4.2 in
Chapter 1 there is at least one equilibrium state in D C O( m),, which contradicts
the absence of equilibrium states in Oe (mo) \ {rno}.

FIGURE 2.34

We now show that any semitrajectory l + (m) with m E R (m) leaves the neigh-
borhood Oe (mo) . Assume the contrary. Then it follows from what has been proved
that w (l + (m)) = mo, and the closed curve l + (m) U l + (m) U {inO} U mm, where
mm C (m) is the subsegment of (m) with endpoints in and m, bounds a
"Bendixson bag" D. Since l+ intersects R(m) at points arbitrarily close to in, l+
enters D across the segment mm C (m) and cannot go out of D C O (mo) . This
contradicts the inclusion l C w (l) and the fact that there are points of l outside
O6 (mo) The contradiction shows that l is an w-separatrix. It follows from the
.

definition of Bendixson extendibility that the Bendixson extension of l+ (to the


right) belongs to w (l) .
We prove the assertion b). Since mo w (l) by assumption, there exists an
e-neighborhood O( m) with points of w (l) outside it and no equilibrium states
other than mo inside it.
3. THE POINCARE-BENDIXSON THEORY 73

Since m0 E w (l) , there is a sequence of points ink E l n O(m0) that tends to


m0 as k -* 60. The semitrajectories l+(mk), k E N, leave O(m) because there
are points of w (l) outside Oe (m0) . Denote by m the first point where l + (mk )
intersects aO (m0), and let m+ be an accumulation point of the sequence {m}
(Figure 2.35).

FIGURE 2.35

We show that the negative semitrajectory l - (m+) does not leave Oe (m0) . As-
sume the contrary. Then there is an arc m-m+ of l - (m+) such that m- O( in)
(Figure 2.35). By the long flow tube theorem, there exists a neighborhood U of
z
m-m+ with the structure of a constant flow. Therefore, all the trajectories inter-
secting u go out of Oe (m0) as time decreases unboundedly (Figure 2.36). We take
the neighborhood U small enough that m0 U. For sufficiently large k the point
m is in U. Since the arc mkm lies in Oe(m0), it follows that ink E U. On the
other hand, ink U for sufficiently large k because m0 U and ink -f m0. This
contradiction proves that l - (m+) C Oe (m0) , and hence a (l - (m+)) C cl Oe (m0) .

FIGURE 2.36

Let us show that m0 = a(1 - (in+)). Assume not. Then a(1 - (m+)) contains
a regular point. Since m+ is an accumulation point for the points in E l+,
1 - (m+) lies in the limit set of l + According to Theorem 2.2 (the Maier criterion
.
74 2. STRUCTURE OF LIMIT SETS

for recurrence), if l - (m+) is a nonclosed trajectory, then l - (m+) is a nontrivial


recurrent semitrajectory lying in O (mo), which contradicts Lemma 2.4. If l - (m+)
is a closed trajectory, then it bounds a simply connected domain D in Oe (m0) .
Since the semitrajectories l+(mk) leave Oe(m0), m0 cannot be in D. But by virtue
of Corollary 4.3 in Chapter 1 there is at least one equilibrium state interior to D,
and this contradicts the fact that Oe (m0) does not contain equilibrium states other
than m0. The contradiction proves that all (m+)) = m0, and b) now follows from
a). D

The conditions and assertions of Lemma 3.4 are illustrated in Figure 2.37.

FIGURE 2.37

3.4. The Bendixson theorem on equilibrium states.


THEOREM 3.3. If mo is an isolated equilibrium state, then either any neigh-
borhood of m0 contains a closed trajectory with m0 inside it, or there exists a
semitrajectory tending to mo.
PROOF. We take a neighborhoodU(mo) of m0 that is diffeomorphic to an open
disk and does not contain equilibrium states other than m0, and we assume that
U(mo) does not contain a closed trajectory with m0 inside it.
Let us first show that there exists a positive or negative semitrajectory lying
entirely inU(m0). Assume not. We take a sequence of points mk - m0i k E N, and
denote by m and m the first points where the trajectory l(mk) intersects aU(m0)
when we move along l (ink) in the negative and positive directions, respectively
(Figure 2.38). Such points exist in view of our assumption. Denote by m- an
accumulation point of the sequence {mk }°. By our assumption, there is an arc
pq of l (m) whose endpoints p and q lie outside U(mo) (Figure 2.38). According
to the long flow tube theorem, there exists a neighborhood W of pq with the
structure of a constant field. Therefore, all the semitrajectories l - (in) and l+ (in)
with in E W n U(mo) leave U (mo) . It is clear that W can be taken small enough
that m0 W. Since m- is an accumulation point of the sequence {m }°, for
sufficiently large k the arcs mm C l (mk) intersect W, and hence ink E W. This
contradicts the facts that ink -f ink and m0 W.
Thus, there exists a point in E U(mo) such that the semitrajectory l ) (in) lies
entirely inU(mo). For definiteness we take l ) (in) to be the positive semitrajectory,
def
that is, l )(in) = l + (m) l + . Then w (l +) C clU (mo)
.

If w (l +) = m0 , then the theorem is proved. Assume that w (l +) contains a


regular point m. We prove that w (l+ (m)) = m0, that is, l+ (m) is the desired
semitrajectory. Assume the contrary. Then w(l+(m)) contains a regular point. If
3. THE POINCARE-BENDIXSON THEORY 75

FIGURE 2.38
l+ (m) is a nonclosed semitrajectory, then we have a contradiction to Theorem 2.2
and Lemma 2.4. If l+(m) is a closed trajectory, then by the invariance of the w-
limit set and the inclusions w (l + (m)) C w (l +) C clU (mo) , the trajectory l + (m) =
w (l + (m)) lies in cl U (mo) and bounds a simply connected domain D C U (mo )
in the disk clU(mo). Since there are no closed trajectories in the neighborhood
U(mo) with m0 inside them, it follows that m0 D. But in view of Corollary
4.3 in Chapter 1 the domain D, and hence the neighborhood U(mo), contains an
equilibrium state other than m0, which contradicts the choice of U(mo). U
We consider an isolated equilibrium state m0 and a neighborhood U(mo) dif-
feomorphic to an open disk and not containing equilibrium states other than m0.
Suppose that the semitrajectories 4) and 4) tend to m0 (as t -* +oo or t -+ -oo),
and both have points outside U(mo). Denote by ml and m2 the last points 4) and
4) have in common with the boundary 9U(mo) (Figure 2.39).

FIGURE 2.39

The curve consisting of 4(mi), l2 (m2) C U (mo) and the point m0 separates
U(mo) into two domains U- and u+, called the curvilinear sectors bounded by the
semitrajectories l i and 4) in the neighborhood U(mo). In this notation we give the
following definition.
DEFINITION. The curvilinear sector U* in U(mo) bounded by the semitrajec-
tories lid and 4) is called a hyperbolic or saddle sector in the neighborhood U(mo)
76 2. STRUCTURE OF LIMIT SETS

if 4) and Q are separatrices of the equilibrium state mo that are Bendixson exten-
sions of each other (to the right or to the left), and any trajectory 1(m), m E u*,
leaves U* with increasing or decreasing time.
There are definitions of parabolic and elliptic sectors; however, we omit them
and confine ourselves to an illustration (Figure 2.40). See [3], §17 of Chapter 8, for
the precise definitions.

FIGURE 2.40

If the equilibrium state mo has only finitely many separatrices tending to it,
then there exists a neighborhood tL(mo) such that the number of hyperbolic, elliptic,
and parabolic sectors is the same in all neighborhoods tL' (mo) C 'la(ma). Therefore,
in this case we can refer to the number of hyperbolic, elliptic, and parabolic sectors
of the equilibrium state.
CONVENTION. Unless a statement to the contrary is made, each equilibrium
state below will be assumed to be isolated, and to have only finitely many separatrices
(possibly none) tending to it.
DEFINITION. An equilibrium state to which a finite nonzero number of sepa
ratrices tend is called a topological saddle if it has only hyperbolic sectors (Figure
2.41).

FIGURE 2.41
3. THE POINCARE-BENDIXSON THEORY 77

A topological saddle with four hyperbolic sectors will sometimes simply be


called a saddle (or structurally stable saddle).
Lemma 3.4 gives us
COROLLARY 3.1. Equilibrium states of a transitive flow on a closed surface are
topological saddles.
We note that such a flow can have impassable grains: topological saddles with
two hyperbolic sectors (see §2.3.2 in Chapter 1).
3.5. One-sided contours. Consider a flow on a two-dimensional manifold.
A trajectory of it that is both an a-separatrix of an equilibrium state and an w-
separatrix of an equilibrium state (possibly the same equilibrium state) will be
called a separatrix joining equilibrium states, or a trajectory connecting equilibrium
states, or a separatrix going from an equilibrium state to an equilibrium state (Figure
2.42).

FIGURE 2.42

DEFINITION. A family of separatrices ll,... , is joining equilibrium states, to-


gether with equilibrium states m1,. . . , ms such that a(l2) = mi (i = 1, ... , s) and
w(l) = m1 (j = 1, ... , s) (where ms+1 = ml) is called a right-sided (left-sided)
contour if for all i = 1, ... , s the separatrix l j+1 (where 1s+1 = ll) is a Bendixson
extension of li to the right (to the left).

FIGURE 2.43

A right-sided or left-sided contour is called a one-sided contour.


In particular, a one-sided contour can consist of a single equilibrium state and
a single separatrix (a separatrix loop).
LEMMA 3.5. Let l be a nonclosed trajectory of a flow on a compact orientable
surface M. In this case:
a) if the w- (a-) limit set of l contains a one-sided contour K, then w (l) (a (l)) =
K;
78 2. STRUCTURE OF LIMIT SETS

b) if the w- (a-) limit set of l consists of finitely many equilibrium states and
finitely many separatrices joining equilibrium states, then w(l) (a(l)) is a one-sided
contour.
The proof of a) is completely analogous to that of Lemma 1.6, in which it is
shown that if w (l) (a (l)) contains a closed trajectory, then w (l) (a (l)) is a closed
trajectory.
The assertion b) follows from Lemma 3.4 and a). 0
3.6. Lemmas on the Poincare mapping. Let f t be a flow on a compact
two-dimensional manifold PVC, and let E1 and E2 be disjoint contact-free segments
for f t such that the Poincare mapping P : E1 - E2 is defined at some point of E1
(see §1.2).
Assume that P is undefined at the endpoints of E1, and let Dom(P) be the
domain of P.
According to the theorem on continuous dependence of trajectories on the ini-
tial conditions, if P[Dom(P)] C int E2 (that is, the image of any point m E Dom(P)
is not an endpoint of E2), then the set Dom(P) is open in E1 (Figure 2.44). Con-
sequently, Dom(P) is a union of at most countably many disjoint open intervals,
which we call the components of Dom(P).

morn

FIGURE 2.44

The set Dom(P) and the concept of the components of it are defined similarly
for a contact-free cycle C of f t and for the Poincare mapping P : C - C (Figure
2.45).

FIGURE 2.45

We orient each contact-free segment E and each contact-free cycle C (a positive


direction is introduced) ; for E the orientation introduced is such that we hit the
component ER (m) when we move in the positive direction from any point m E E
(see §3.2 for the definition of ER (m)) . For points a, b E E (C) we denote by (a, b)
3. THE POINCARE-BENDIXSON THEORY 79

the interval of the contact-free segment (cycle) E (C) traversed from a to b in the
positive direction.
For a, b E E let ab be the interval of E with endpoints a and b.
LEMMA 3.6. Let E1 and E2 be disjoint contact free segments for a flow f t
on a compact two-dimensional manifold, and suppose that the Poincare mapping
P : E1 - E2 has nonempty domain Dom(P). Assume that P[Dom(P)] C int E2,
and let (m1, m2) be a component of the set Dom(P). Then:
a) the semitrajectories l+(m1) and l+(m2) are w-separatrices;
b) there exists a sequence l+(m1) = l1o,111, ... ,1111 of separatrices such that l1s
is a Bendixson extension of l1 S_ 1 to the right (s = 1, ... ,k1), and the separatrices
hi (i = 0, ... , k1 - 1) do not intersect E2, but 11 k1 does intersect E2;
c) there exists a sequence l+(m2) = 120,121,... ,12k2 of separatrices such that l2s
is a Bendixson extension of l2 S_1 to the left (s = 1, ... ,k2), and the separatrices
123 (j = 0, ... , k2 - 1) do not intersect E2, but 12k2 does intersect E2;
d) the image of the interval (m1, m2) under the action of P is an interval
m1m2 C E2, where mi is the first point where 1ikZ intersects E2 with increasing
time, i = 1,2;
e) for points a, b E (ml, m2) the interval (a, b) C El, the arcs aP(a) and bP(b)
of the respective semitrajectories l+(a) and l+(b), and the interval P(a)P(b) C E2
bound an open simply connected domain in 1vC (Figure 2.46).

FIGURE 2.46

PROOF. a) Since (m1, m2) is a component of the set Dom(P), l+ (m1) and
l+(m2) do not intersect the contact-free segment E2.
We show that w (l + (ml)) is an equilibrium state. Assume not. Suppose that
w (l + (m1)) contains a regular point A. We consider first the case when l+ (m1) is
a nonclosed semitrajectory. Obviously, AE2, and there is an arc mP(m) of a
semitrajectory l+(m) with m E (m1, m2) that does not contain A. Through A
we draw a contact-free segment E(A) disjoint from E2 U mP(m), and we take an
80 2. STRUCTURE OF LIMIT SETS

arbitrary E(A)-arc pq of l+ (ml) . Such an arc exists, because l+ (ml) intersects


(A) in a countable set of points. In view of Lemma 1.2 the simple closed curve
pq U pq can be approximated by a contact-free cycle C that intersects l+ (mi), and
it follows from the disjointness of E(A) and E2 UmP(m) that C can be constructed
to be disjoint from E2 U mP(m). We take such a cycle C (Figure 2.47).

FIGURE 2.47

Let us break up the closed interval cl(ml, m) aef [mi, m] into two sets: R1 =
{x E [mi, m] I the semitrajectory l+ (x) intersects C without first intersecting the
segment E2 }, and R2 = {x E [mi, m] I 1+ (x) intersects E2 without first intersecting
the cycle C}. Since m1 E R1 and m E R2, it follows that R1 0 and R2 0.
It then follows from the theorem on continuous dependence of trajectories on the
initial conditions that R1 and R2 are relatively open in the topology of the segment
[mi, m]. It is clear that R1 U R2 = 0. From this and the connectedness of [mi, m],
there exists a point m* E (mi, m) such that l+ (m*) is disjoint from C and E2,
which contradicts the fact that all the semitrajectories l+(m) with m E (ml, m2)
intersect E2.
If l+ (ml) is a closed trajectory disjoint from E2, then for points m E (ml, m2)
sufficiently close to ml the semitrajectories l+ (m) first intersect the contact-free seg-
ment E 1, and then the segment E2. This contradicts the definition of the Poincare
mapping P (see §1.2).
Thus, w(1 + (ml)) is an equilibrium state m0. Let tL(mo) be a neighborhood
of m0 disjoint from E2. It follows from the theorem on continuous dependence
of trajectories on the initial conditions that for points m E (ml, m2) sufficiently
close to m1 the semitrajectories l+(m) enter tL(mo) without first intersecting 2,
and then must leave tL(mo) in order to intersect 2. Therefore, l+ (ml) is an w-
separatrix of the equilibrium state mo (Lemma 3.2). The proof that 1+(m2) is also
an w-separatrix is completely analogous, and the assertion a) is proved.
We prove the assertion b). It follows from the foregoing that l+ (ml) is ex-
tendible to the right. By Lemma 3.3, there exists an a-separatrix 111 of m0 that is
a Bendixson extension of the w-separatrix 110 = l+ (ml) to the right. If ll 1 intersects
E2, then b) is proved. Suppose that 111 is disjoint from E2. Repeating the proof of
a) with no fundamental changes, and using the definition of a Bendixson extension,
we show that w(1i1) is an equilibrium state, and ll 1 is an w-separatrix (that is, 111 is
a separatrix joining equilibrium states). Continuing in this way, we get a sequence
110,111,.. , lu,... of separatrices, where lli is a Bendixson extension of 11 i_1 to the
right for each i > 1.
3. THE POINCARE-BENDIXSON THEORY 81

We show that 11i intersects E2 for some index i > 1 (which concludes the
proof of b)). Assume the contrary. By our convention about the finiteness of the
number of separatrices of any equilibrium state and the finiteness of the number
of equilibrium states on the compact manifold PVC, we then get that l1i contains
the semitrajectory 110 = l+ (m1) for some i > 1. Consequently, the separatrices
110,111,.. , l1i and their limit sets (the equilibrium states which they join) form a
right-sided contour. This contour is disjoint from E2 by assumption, so for points
m E (m1, m2) sufficiently close to m1 the semitrajectories 1+ (m) intersect E1 with-
out first intersecting E2 as time increases, and this contradicts the definition of the
Poincare mapping P : E 1 - E2. The contradiction proves b) .

The proof of c) is analogous.


We prove d). On its domain P is a homeomorphism (Lemma 1.3), so
P[(m1i m2)] is an open interval mi m2 C E. On the other hand, it follows from the
definition of the Bendixson extension that P(m) - mi as m - mi, m E (m1, m2)
(i = 1, 2), where mi is the first point where the separatrix liki intersects E2. This
implies that m1 = mi and m2 = m2.
The assertion d) follows from the fact that the domain bounded by the curve

(a, b) U aP(a) U bP(b) U P(a)P(b)

can be covered by finitely many long flow tubes. 0

We now consider a contact-free cycle C for a flow f t given on a compact ori-


entable two-dimensional manifold PVC, and we assume that the Poincare mapping
P : C - C induced by f t has nonempty domain Dom (P) . Recall that for any point
m E Dom(P) the positive semitrajectory l+(m) \ {m} intersects C, and the first
such point of intersection is denoted by P(m). The set Dom(P) is open and is the
union of an at most countable family of disjoint open intervals which we call the
components of Dom(P).
LEMMA 3.7. Let C be a contact free cycle for a flow f t on a compact orientable
two-dimensional manifold PVC, and let the Poincare mapping P : C - C induced by
f t have nonempty domain Dom(P). Let (m1, m2) be a component of Dom(P).
Then:
a) the semitrajectories l+ (m1) and l+ (m2) are w- separatrices;
b) there exists a sequence l+ (m1) = l o,1u, ,11I1 of separatrices such that l1s
is a Bendixson extension of l1 s_ 1 to the right (s = 1,... ,k1), and the separatrices
lli (i = 0, ... 1) do not intersect C, but 11k1 does intersect C;
c) there exists a sequence 1+ (m2) = 120,121, ... ,122 of separatrices such that l2s
is a Bendixson extension of l2 s_1 to the left (s = 1,... ,k2), and the separatrices
12j (j = 0, ... , k2 - 1) do not intersect C, but 12k2 does intersect C;
d) the image of (mi, m2) under the action of P is the interval (m1, m2), where
mi is the first point where liki intersects C, i = 1, 2;
e) for any endpoints a, b E (mi, m2) of an interval (a, b) C (m1, m2) the closed
curve (a, b) U aP(a) U bP(b) U (P(a), P(b)) bounds a simply connected open domain
in M, where aP(a) and bP(b) are arcs of the respective semitrajectories 1+ (a) and
l+(b).
82 2. STRUCTURE OF LIMIT SETS

PROOF. This can be proved according to the scheme used to prove Lemma 3.6.
We omit the proof and leave it to the reader as an exercise. 0

LEMMA 3.8. Suppose that a flow f t on a closed orientable two-dimensional


manifold PVC induces a Poincare mapping P: C - C with nonempty domain Dom(P)
on a contact free cycle C. Then Dom(P) has finitely many components.
PROOF. Assume the contrary. Let (ai, bi) (i = 1, 2,...) be the family of com-
ponents of the set Dom(P). In view of Lemma 3.7 the semitrajectories l+(ai) and
l + (bi) (i E N) are w-separatrices (Figure 2.48). Since each of ai and bi (i E N) is
a boundary point of at most two components of Dom(P), each w-separatrix in the
collection {l+(ai), l+(bi), i E N} is repeated at most twice; therefore, the collection
has countably many distinct w-separatrices. This contradicts our convention that
f t has finitely many equilibrium states on the compact manifold, and that each
equilibrium state has finitely many (perhaps none) separatrices. 0

C,
a
FIGURE 2.48

Lemma 3.8 generalizes to flows f t that can have infinitely many equilibrium
states. It is clear that in this situation the lemma is not true in general, because
there are examples of flows with countably many impassable grains for which the
semitrajectories tending to the impassable grains separate Dom(P) into countably
many components. We proceed to a generalization of Lemma 3.8 after introducing
the necessary definitions.
Suppose that the flow f, which can have infinitely many equilibrium states,
induces a Poincare mapping P : C -p C with nonempty domain Dom(P) on the
contact-free cycle C.
DEFINITION. Two points x1, x2 E C are said to be Gutierrez-equivalent (writ-
ten as xl N x2) if there exist points a, b E Dom(P) such that (a, b) contains xl
and x2, and the closed curve [a, b] U aP(a) U bP(b) U [P(a), P(b)] bounds a simply
connected domain on the manifold (Figure 2.49).

PC

P(a)
FIGURE 2.49
3. THE POINCARE-BENDIXSON THEORY 83

The relation of the Gutierrez equivalence is an ordinary equivalence relation,


and the set of points of C is broken up into disjoint equivalence classes. It follows
immediately from the definition and from the long flow tube theorem that an equiv-
alence class with more than one point is an open subset of C. Such a Gutierrez
equivalence class will be called an open class. The union of the open equivalence
classes will be denoted by D (C, P, r%1). Obviously, Dom(P) C D (C, P, ) In view
.

of Lemma 3.7 e) a component of Dom(P) belongs to a single open equivalence class.


The set D (C, P, r-i) is open and is the union of an at most countable family of open
intervals which we call its components.
LEMMA 3.9. Let f t be a flow (having infinitely many equilibrium states in
general) on a compact orientable two-dimensional manifold PVC, and suppose that f t
induces a Poincare mapping P : C -p C with nonempty domain on a contact free
cycle C. Then:
a) each open Gutierrez equivalence class is an open interval and is a component
of the set D(C, P, 'S-');
b) the number of open Gutierrez equivalence classes is finite.
PROOF. The assertion a) follows immediately from the definition of the Gutier-
rez equivalence.
We prove b). Assume the contrary: suppose that the family of open equivalence
classes is countable. By the definition of the Gutierrez equivalence, each open class
contains a point where P is defined. Let ai (i E N) be points of Dom(P) that are
in distinct open equivalence classes. The contact-free cycle C does not separate PVC
(Lemma 2.2), so cutting PVC along C leads to a manifold PVC' with two components
Kl and K2 of the boundary 9M' that are joined by the arcs aiP(ai). It follows
from Lemma 2.8 that there are two arcs a3P(ad) and akP(ak) that bound a simply
connected domain on M'. This contradicts the fact that a and ak belong to
different equivalence classes. 0
3.7. Description of quasiminimal sets. Let f t be a flow on a compact
two-dimensional manifold M, and assume that it has nontrivial recurrent semitra-
jectories.
DEFINITION. A set N C M is called a quasiminimal set of the flow f t if it is
the (topological) closure of a nontrivial recurrent semitrajectory.
According to Theorem 2.1, the topological closure of a nontrivial recurrent
semitrajectory 10 contains a continuum of nontrivial recurrent trajectories that
are dense in cl(l0). Therefore, a quasiminimal set is the topological closure of a
nontrivial recurrent trajectory (and can be so defined).
THEOREM 3.4. On a compact two-dimensional manifold PVC let f t be a flow
having nontrivial recurrent semitrajectories, and let N be a quasiminimal set of
f t . Then N is invariant (consists of whole trajectories) and can contain only the
following trajectories:
1) nontrivial recurrent trajectories;
2) w- (a-) separatrices that are P- (P+) nontrivial recurrent trajectories;
3) separatrices joining equilibrium states;
4) equilibrium states, to each of which at least one w-separatrix in N tends, and
at least one a-separatrix in N.
84 2. STRUCTURE OF LIMIT SETS

PROOF. Let N = cl(l), where l is a nontrivial recurrent trajectory. Since


cl(l) = w(l) = a(l) (see (2.3), §2.1), the invariance of N follows from that of an w-
(a-) limit set.
If mo E N is an equilibrium state,
N
then mo E w (l ), and NLemma 3.4 implies 4).
Consider
N
a regular trajectory l C N. The trajectory l cannot be closed, for
otherwise l = w (l) in view of Lemma 1.6, and this contradicts N
the fact that l is a
nontrivial recurrent trajectory. If the w- (a-) limit
N
set of l contains a regular point,
then by the Maier N
theorem
N
(Theorem 2.2), l is a P+ (P-) nontrivial recurrent
trajectory.
N
If w( l) (a( l)) is an equilibrium
N
state, then it follows from the inclusion
l C w(l) and Lemma 3.4 that l is an w- (a-) separatrix. 0

To prove Theorem 3.5 we need the following result.


LEMMA 3.10. Let 10 be an exceptional nontrivial recurrent semitrajectory (see
§2.1 for the definition), and let C be a contact free cycle intersecting l Then
cl(lO f1 C) is a Cantor set on C (that is, a perfect, nowhere dense, closed subset of
C).
PROOF. According to Lemma 2.1, 10 f1 C is a perfect set, and hence so is
cl (l O f1 C). The fact that cl (l O f1 C) is nowhere dense follows immediately from the
definition of an exceptional semitrajectory 10. 0

Since the Cantor set is perfect and nowhere dense, its complement consists of
countably many open intervals called the adjacent intervals of the Cantor set.
THEOREM 3.5. Let f t be a flow on a closed orientable two-dimensional man-
ifold M. IfN the w- (a-) limit set of a trajectory
N
l contains a nontrivial recurrent
trajectory l of f t, then w(l) (a(l)) = cl( I).
N
N
PROOF. For definiteness we assume that IC w(l), and we prove that w(l) =
cl (l ) N N N N N
Since cl( l) = w( 1), it follows
N
from l C w (l) that cl( l) = w( l) C w (l) . It
remains
N
to prove that w (l) C cl (l) . N
If l is a locally dense trajectory, then cl( l) contains interior points Nto whichN l
comes arbitrarily Nclose. Consequently,N
there
N
are points on l lying in cl( l) = w( l).
Therefore, l C w( l) and w (l) C w( l) = cl ( I). N N N
We now consider the case of an exceptional trajectory N
I. If I C cl( l) = w( I),
then the theorem is proved, so we assume N
that l cl( 1). By Lemma 2.3, there
exists a contact-free cycle C intersecting 1, and hence also I. According to Lemma
e N
3.10, 1 cl(l n C) is a Cantor set. Therefore, the complement C \ 1 consists
of countably many disjoint
N open intervals the adjacent intervals of ft By our
assumption that l cl( I), the trajectory l intersects C in the adjacent intervals.
We write the adjacent intervals C1, C2,... in the order of their intersection with 1,
beginning from some fixed time as we move in the positive direction.
N
Since l intersects C in a set that is dense in 1 and since l C w (l ), l intersects
a countable family of distinct adjacent intervals (in particular, l is a nonclosed
trajectory).
We denote by P : C - C the Poincare mapping induced by f t . Obviously,
l f1 C C Dom(P). Since the number of components of the domain Dom(P) is finite
3. THE POINCARE-BENDIXSON THEORY 85

(Lemma 3.8), the number of adjacent intervals containing endpoints of components


of Dom(P) is also finite.
We show that l cannot intersect an adjacent interval infinitely many times.
Assume the contrary. Since l intersects a countable family of distinct adjacent
intervals, our assumption implies that there is a countable family of adjacent inter-
vals Gi1, G2,... (not necessarily distinct) with the following property: each Gik
contains at least two points XZk, xi E l fl C such that P(xik) and P(xik) belong to
different adjacent intervals Gk and Gig (Figure 2.50), and the intervals Gi1, Gig, .. .
are distinct. The existence of the families {G1,2 G22) .. } and {G1, G22f... } means
Z

either the existence of an adjacent interval containing countably many endpoints of


components of Dom(P), or the existence of a countable family of distinct adjacent
intervals, each containing an endpoint of a component of Dom(P). This contra-
dicts the finiteness of the number of components of Dom(P), and the contradiction
proves that l cannot intersect an adjacent interval infinitely many times.

FIGURE 2.50

We can prove similarly that the adjacent intervals G0, G0+1, ... are distinct
beginning with some index no.
This and the fact that only finitely many adjacent intervals contain endpoints
of components of Dom(P) imply the existence of an index io > no such that the
adjacent intervals Gio , G0+1,... (which are successively intersected by l) are dis-
tinct and belong to Dom(P). It is clear that P(G3) = G3+1 for j > io. It can be
assumed without loss of generality that io = 1 and G1 Gi for i > 2 (that is, l
does not intersect G1 after a certain fixed time moment). The last point where l
intersects G1 as time increases is denoted by x0.
The equality P (G) = G1 (i > 1) implies that the set Q def Ul + (m) , m E G1,
is homeomorphic to the strip G1 x [0, oo). Since l+(xo) intersects each adjacent
interval G1, G2,... at only one point, the w-limit set of the semitrajectory l+(xo)
does not lie in Q.
Let x E w (l+ (xo)) = w (l) . Since Q is homeomorphic to G1 x [0, oo), and since
l+(xo) C Q and x Q, there is a sequence of points mk, k E N, belonging to the
boundary aQ and tending to x as k -p oo. All the trajectories in aQ belong to
w( l) because l fl C is dense in fZ. Therefore,
N
mk E w( I), and hence x E w( I). This
proves the required inclusion w (I) C w( 1). 0

3.8. Catalogue of limit sets. In this subsection we describe all the w-limit
sets of individual trajectories of a flow. The same description applies in the case of
a-limit sets, of course.
86 2. STRUCTURE OF LIMIT SETS

THEOREM 3.6. On a closed orientable surface let .f t be a flow with finitely


many equilibrium states and finitely many separatrices, and let w(l) be the w-limit
set of a trajectory l of f. Then one of the following possibilities is realized:
1) w (l) is a single equilibrium state (Figure 2.51 a)) ;
2) w(l) is a single closed trajectory (Figure 2.51 b));
3) w (l) is a single one-sided contour (Figure 2.51 c) );
4) w (l) is a quasiminimal set containing only trajectories of the type 1)-4) in
Theorem 3.4, any nontrivial recurrent semitrajectory in w (l) is dense in w (l ), and,
moreover, either a) each nontrivial recurrent semitrajectory in w(l) is locally dense,
or b) each nontrivial recurrent semitrajectory in w (l) is exceptional.

b)

FIGURE 2.51

PROOF. If w (l) consists solely of equilibrium states, then w (l) is a single equi-
librium state because an w-limit set is connected (Lemma 1.5).
If w (l) contains a closed trajectory, then w (l) is a single closed trajectory ac-
cording to Lemma 1.6.
The possibilities 1) and 2) hold if l is an equilibrium state or a closed trajectory,
respectively.
It remains to consider the case when l is nonclosed and w (l) contains a regular
point on a nonclosed trajectory, that is, w(l) contains a nonclosed trajectory.
If the w- and a-limit sets of any nonclosed trajectory in w (l) consist solely of
equilibrium states, then it follows from Lemmas 3.4 and 3.5 that w (l) is a single
one-sided contour.
For a nonclosed trajectory l C w(l) suppose that the set w( l) U a( l) contains
a regular point. By the Mater theorem (Theorem 2.2), l is a P+ or P- nontrivial
recurrent trajectory. The inclusion l C w (l) and the closedness of an w-limit set
(Lemma 1.5) imply that cl( l) C w (l) . According to Cherry's theorem (Theorem
2.1), cl ( l) contains a nontrivial recurrent trajectory 1. Therefore, l C w (l) By .

Theorem 3.5, w (l) = cl ( 1); that is, w (l) is a quasiminimal set.


Let li (lfl be a P+ (P-) nontrivial recurrent semitrajectory in the quasimini-
mal set w (l) = cl ( 1). Since the equalities cl (l) = w (l ) = a (l) hold for the recurrent
trajectory 1, li (lfl is contained in the limit set of 1. According to Theorem 2.3,
lis contained in the w- (a-) limit set of the semitrajectory li (lfl. Consequently,
li (lfl is dense in w(l).
Again using Theorem 2.3, we get the assertions 4, a) and 4, b). 0
The example in §3.3 of Chapter 1 of a flow on the sphere shows that Theorem
3.6 is false without the assumption that there are finitely many equilibrium states.
4. QUASIMINIMAL SETS 87

3.9. Catalogue of minimal sets. In this subsection we present all possible


minimal sets of flows on closed surfaces.
THEOREM 3.7. On a closed orientable surface M let f t be a flow with finitely
many equilibrium states and finitely many separatrices, and let N be a minimal set
of f t . Then one of the following possibilities is realized:
1) N is a single equilibrium state;
2) N is a single closed trajectory;
3) N consists of nontrivial recurrent trajectories, each dense in N, and either
a) N coincides with the whole of M, and M is the torus T2 in this case, or b) N is
nowhere dense in M, consists of exceptional nontrivial recurrent trajectories, and
is locally homeomorphic to the direct product of a closed bounded interval and the
Cantor set.
PROOF. By Lemma 1.7, a minimal set coincides with the w- (a-) limit set of
any of its trajectories. Therefore, the possible types of N are included in the list
1)-4) in Theorem 3.6.
An equilibrium state and a closed trajectory are minimal sets.
A one-sided contour contains an equilibrium state, which is a nonempty closed
invariant subset, and hence it is not a minimal set.
If N is a quasiminimal set, then by minimality it must not contain equilibrium
states, and hence separatrices. Therefore, in the last case N consists of nontrivial
recurrent (in both directions) trajectories, each dense in N.
According to Lemma 1.7, either N is nowhere dense in M, or N = M. If N is
nowhere dense, then by Theorem 3.6, the quasiminimal and minimal set N consists
solely of exceptional nontrivial recurrent trajectories. It follows from Lemma 3.10
that N is locally homeomorphic to the product of a closed bounded interval and
the Cantor set. If N = M, then by minimality the flow f t on M does not have
equilibrium states. According to Theorem 4.3 in Chapter 1, the sum of the indices
of the equilibrium states is equal to the Euler characteristic X (M) = 2 - 2p, where
p is the genus of the surface. In the case N = M we get that x(M) = 2 - 2p = 0,
so that p = 1. Consequently, the closed surface M is the two-dimensional torus. 0
A flow on a manifold M is said to be minimal if M itself is a minimal set of
the flow. According to Theorem 3.7, the torus is the unique compact orientable
surface admitting minimal flows (for example, irrational flows). The analogue of
minimal flows for surfaces of higher genus is provided by flows in which every one-
dimensional trajectory (that is, every trajectory different from a fixed point) is
dense. Following Gardiner, we call such flows highly transitive.
Examples of highly transitive flows on genus-2 surfaces were constructed in §2
of Chapter 1.

§4. Quasiminimal sets


Of all the limit sets of individual trajectories the most complicated are quasi-
minimal sets. These are the only limit sets containing a continuum of trajectories.
Quasiminimal sets in Denjoy and Cherry flows (see §3, Chapter 1) give an idea of
their possible structure. In such flows a quasiminimal set in a neighborhood of a
regular point has the structure of the product of a closed bounded interval and the
Cantor set, and is nowhere dense on the surface. A quasiminimal set of a transitive
flow coincides with the whole manifold. Theorem 3.6 lists all possible trajectories
88 2. STRUCTURE OF LIMIT SETS

that can lie (or necessarily lie) in a quasiminimal set. In this section we study its
structure in greater detail.
4.1. An estimate of the number of quasiminimal sets. Let f t be a flow
on a closed orientable two-dimensional manifold M, and suppose that a quasimin-
imal set NZ of f t is the topological closure of a nontrivial recurrent trajectory li
(i = 1, 2). If N1 C N2i then since l2 is dense in N2 (Theorem 3.6), the trajectory
l1 lies in the limit set of l2. According to Theorem 2.3, l2 lies in the limit set of l1,
and therefore N1 = N2. Thus, of any two distinct quasiminimal sets of a flow one
does not contain the other as a proper subset.

FIGURE 2.52

Quasiminimal sets can intersect. We give a schematic description of such a


flow. Starting from an irrational winding on the torus, we construct a transitive
flow f 1 on a torus T1 with a hole, where 0T1 consists of an equilibrium state and a
separatrix loop (Figure 2.52). We take the same flow f 2 on a torus T2 with a hole,
and identify 0T1 with 0T2 by means of a suitable homeomorphism (in particular,
this homeomorphism must paste together the equilibrium states). The result is a
pretzel (a closed orientable surface of genus 2) on which the flows If and f2 form
a flow f t with two quasiminimal sets, which intersect in an equilibrium state and
a separatrix loop (Figure 2.53).

FIGURE 2.53

THEOREM 4.1. Let f t be a flow on a closed orientable two-dimensional mani-


fold NYC. Then:
1) any nontrivial recurrent semitrajectory or trajectory belongs to exactly one
quasiminimal set of f t;
2) the number q(f t) of quasiminimal sets of f t is finite and does not exceed the
genus of M;
3) two quasiminimal sets can intersect only in equilibrium states and separatri-
ces going from an equilibrium state to an equilibrium state.
4. QUASIMINIMAL SETS 89

PROOF. According to Theorem 3.6, 4), each nontrivial recurrent semitrajectory


is dense in the quasiminimal set containing it. Therefore, such a semitrajectory
belongs to exactly one quasiminimal set, and two quasiminimal sets cannot intersect
in a nontrivial recurrent semitrajectory. It then follows from Theorem 3.4 that two
quasiminimal sets can intersect only in equilibrium states and separatrices joining
equilibrium states. The assertions 1) and 3) are proved.
We prove 2). In each quasiminimal set we take a nontrivial recurrent semitra-
jectory. It follows from 3) that these semitrajectories are independent. According
to Theorem 2.4, the number of independent nontrivial recurrent semitrajectories is
finite and does not exceed the genus p of the surface M. Therefore, q(f t) <p. 0

Remarks. The estimates of Aranson, Markley, and Levitt. Aranson's


estimate ([8], 1969; reproved by Markley in 1970 [96]) of the number of indepen-
dent nontrivial recurrent semitrajectories for flows on closed nonorientable surfaces
implies the estimate
t < p-1
q(f) - 2
for the number q(f t) of quasiminimal sets of flows on such surfaces, where p is the
genus of the surface, and [x] is the integer part of a number x. This estimate cannot
be improved; that is, on a closed nonorientable surface of genus p > 1 there is a
flow with exactly [j.] quasiminimal sets. In particular, there are no flows with
quasiminimal sets on the projective plane and the Klein bottle.
In his dissertation [93] Levitt considered foliations 3 on a compact orientable
surface M (with boundary in general) that have singularities only of saddle type,
including thorns, and he obtained the estimate

q(3)<-
[3P+b+e_2]
2

for the number q(3) of quasiminimal sets of such foliations, where p is the genus of
M, b is the number of boundary components, and e is the number of thorns. We
note that the existence of at least one singularity is assumed for a foliation on the
torus T2.
4.2. A family of special contact-free cycles.
LEMMA 4.1. On a closed orientable two-dimensional manifold M let f t be a
f low, and suppose that N1,.. . , N,k are (distinct) quasiminimal sets of f t . Then there
exists a family of disjoint contact-free cycles C1,. .. , C,k such that: 1) CZ fl Ni 0,
2 = 1, ... , k; 2) Ci fl N = 0 for 2 j, 2, j E {1,.. ., k}; 3) for any i = 1, ... , k
either Ci fl Ni = CZ or CZ fl Ni is a Cantor set.
PROOF. In the quasiminimal set Ni we take a nontrivial recurrent trajectory
li, i = 1, ... ,k. Since l1, ... , lk are independent, there exist contact-free segments
s1, ... , > with the following properties: a) >2i flli 0 for i = 1, ... ,k; b) >2i fll = 0
for i j, i, j E {1,.. . ,k}; c) if li is locally dense, then >2i = cl(fl li), that is,
>2i fl li is dense in i.
By a), there is a i-arc p j of the trajectory li for each index i = 1, ... ,k. Then
the family of simple closed curves Ci = pU (i = 1, ... ,k), where C >2i,
satisfies the following conditions: I) Ci fl Ni 0 for i = 1, ... ,k; II) Ci fl N = 0
for i j, i, j E {1,. . . ,k}. This and the proof of Lemma 1.2 give us that each
90 2. STRUCTURE OF LIMIT SETS

curve CZ can be approximated by a contact-free cycle CZ in such a way that the


family of contact-free cycles C1i... , C, satisfies the assertions 1) and 2). Moreover,
if a nontrivial recurrent trajectory is locally dense, then by c) the corresponding
contact-free cycle can be constructed so that CZ = cl(Ci f1 li), and hence so that
Ci = Ci f1 Ni. If a nontrivial recurrent trajectory li is exceptional, then by the
equality Ni = cl (li) and Lemma 3.10, the set Ni f1 Ci = cl (li f1 Ci) is a Cantor set.
a
DEFINITION. Let N1,... , N be quasiminimal sets of a flow f t . A family of
disjoint contact-free cycles C1,... , C, of f t satisfying Lemma 4.1 is called a special
family. If k = 1, then a contact-free cycle C1 constructed for the quasiminimal
set N1 and satisfying the assertion 3) of Lemma 4.1 is called a special contact-free
cycle.
According to Lemma 4.1, if f t has nontrivial recurrent trajectories, then there
is a special family of contact-free cycles for f t .
4.3. Partition of a contact-free cycle. Let f t be a flow on a compact
orientable surface M, and let N be a quasiminimal set of f. According to Lemma
2.3, there exists a contact-free cycle C intersecting at least one nontrivial recurrent
semitrajectory lying in N, and hence all such semitrajectories. We define a partition
eN of C into closed disjoint subsets (elements).
DEFINITION. The elements of the partition eN of the contact-free cycle C are
the closures of the intervals in C \ N and the points not belonging to these intervals.
We remark that in view of Lemma 2.1 the intersection N f1 C is a perfect set;
that is, each point in N f1 C is not isolated. From this it follows that the closures
of any two distinct intervals in C \ N are disjoint. Therefore, the elements of CN
are disjoint closed subsets of C.
EXAMPLES. 1) Let f t be an irrational winding on the torus T2, and let C be
an arbitrary contact-free cycle for ft. Then N = T2, and all the elements of CN
are one-point sets.
2) By using an irrational winding it is possible to construct a flow f t on T2
with a quasiminimal set N homeomorphic to T2 with finitely many disks removed.
There exists a contact-free cycle C for f t such that C \ N contains finitely many
intervals (Figure 2.54). Thus, CN consists of finitely many closed intervals and the
points of the contact-free cycle C not belonging to these closed intervals.

FIGURE 2.54
4. QUASIMINIMAL SETS 91

3) Let f t be a Denjoy flow on T2, and let C be a contact-free cycle for f t .

Then C f1 N is a Cantor set, where N is the unique quasiminimal and minimal set
of the Denjoy flow. The partition eN contains a countable family of closed intervals
(the closures of the adjacent intervals of the Cantor set C n N) and a continuum of
points not belonging to the closures of the adjacent intervals and forming a nowhere
dense subset of C.
By Lemma 4.1, for any quasiminimal set N there is a contact-free cycle C
(a special contact-free cycle) such that the partition CN either is the partition in
example 1) (that is, the partition into one-point subsets), or is the partition in
example 3).
4.4. The Gardiner types of partition elements. Suppose that a flow
f t on a compact orientable surface M has a contact-free cycle C and induces a
Poincare mapping P : C - C with domain Dom(P).
We first define the mappings Pl and PT on the endpoints of the components of
Dom(P). Let (a, b) C Dom(P) be a component of Dom(P). According to Lemma
3.7, the semitrajectory l+ (a) is an w-separatrix, and there exists a sequence of
separatrices l10 = l+(a),111, ... ,11i 1
such that 115 is a Bendixson extension of l1 s-1
to the right (s = 1, ... , k1), the separatrices l1 i (i = 0, ... ,k1 - 1) do not intersect
C, and 11k1 does intersect C (Figure 2.55; the arrow along C indicates the positive
direction on the curve). Denote by a the first point where 11k1 intersects C as
time increases. We define PT on all the left-hand endpoints of the components of
Dom(P) by setting Pr (a) = a E C.
Using Lemma 3.7, we define Pi on all the right-hand endpoints of the compo-
nents of Dom(P) similarly (see Figure 2.55).

FIGURE 2.55

We remark that some points x0 E C can be simultaneously left- and right-


hand endpoints of different components of Dom (P) , and both Pl (x0) = Pr (xo) and
Pi (xo) Pr (xo) are possible (Figure 2.56).
If m E C lies in Dom(P), then we set Pi (m) = Pr (m) = P(m).
Suppose now that f t has a quasiminimal set N, and C is a contact-free cycle
for f t that intersects N. We write an arbitrary element r of the partition CN as
an interval [ru, r = Tin if rj is a one-point set.
Since nontrivial recurrent trajectories in N are dense, such trajectories intersect C
at points arbitrarily close to E N on the left. Nontrivial recurrent trajectories
intersect C at points in Dom(P). According to Lemma 3.8, the number of compo-
nents of Dom(P) is finite, and hence two cases are possible: 1) ill is the right-hand
92 2. STRUCTURE OF LIMIT SETS

f(XQ)r P,(xD) (x)


Pt (x)

FIGURE 2.56

endpoint of some component of Dom(P) (Figure 2.57); or 2) rj E Dom(P). In


both cases P1(971) is defined, and in view of the orientability of M and the theorem
on continuous dependence of trajectories on the initial conditions, the nontrivial
recurrent trajectories in N intersect C at points arbitrarily close to P1(r1) on the
left. Therefore, P1 (i71) is the left-hand endpoint of an element of eN.

FIGURE 2.57

An analogous definition applies to the point Pr (rir) , which is the right-hand


endpoint of some element of CN.
Following [52], we give the
DEFINITION. An element r = [?7i, 7)r] E CN is called an element of type 1 if
the interval [P1(r1), Pr (iir)] (possibly a single point) is an element of CN. All the
remaining elements of the partition CN are called elements of type 2.
Figures 2.58 and 2.59 show elements of types 1 and 2, respectively.

FIGURE 2.58
4. QUASIMINIMAL SETS 93

cN

P(?

FIGURE 2.59

We examine more closely an element 97 = [97t, 97r] E eN of type 2. By definition,


P1(971) and Pr (9r) belong to different elements of CN, which we denote by rj -1 and
rj+1, respectively. Let rj-1 = [rjj', and rj+1 = [rt1, Since M is orientable,
1
Eli = PZ (971) and 9T 1
= Pr (9r) . It follows from the invariance of a quasi-minimal
set N and the denseness in N of nontrivial recurrent trajectories that there exist
elements 7-2 = [9712, 7T 2], 7+2 = [97t2, 9+2] E CN such that Pl (9i 2) = rfi 1 E 7+1
and Pr (9T 2) = 97_i Err 1 (Figure 2.60).

FIGURE 2.60

We show that 9 97+2 . Assume the contrary. Then Pl ('7a 2) = P1(971) =Eli 1 E
This and the fact that Pr (9r) E 97+1 give us that 97 is an element of type 1.
97+1.

The contradiction proves that 9 97+2.

9-2
The inequality 9 is proved similarly.
9 C 9-2, and 7-1 7+1, both the elements 7-2 and 7+2 have
97+2,
Since 9
type 2.
The equality 97-2 = 97+2 is possible, and in this case we get the saddle set
97-2,
97-1, 97,97+1,97+2 of elements of CN (Figure 2.61).
97-2
Suppose that 97+2. Then there exist elements 97-3, 97+3 E CN such that
973 97-3 and Pr(9T 2) = 9T 3 If 9-3 = 97+3, then we get the
p1(9712) = E E r1+3.

saddle set 97-3, 97-2, 97-1, 97, 97+2, 97+3 (Figure 2.62).
97+1,

If 97-3 97+3, then there exist elements 97-4, 97+4 E CN such that Pt(97j4) =
9i 3 E 97+3 and Pr(9T 4) = T 3 E 97-3,
94 2. STRUCTURE OF LIMIT SETS

FIGURE 2.61

+3 .r

FIGURE 2.62

Similarly, the equality rj+2 = ?7+4 implies that Pl (rii 2) = Pt(71t4) = E 97+3
and P(r)2) = r),+3 E r), which contradicts the fact that 97+2 has type 2. Therefore,
97+2 If we assume that = 97+4, then 77-2 = 97+2 (but we are considering the
97+4.

7)-2 rj-4 and 77-2-4


case 97+2), and therefore 7 The inequalities r)
97+4.

are proved in the same way. It can again be shown as before that 77-4 and 97+4 are
elements of type 2.
If 7-4 7+4, then we continue the process described above. Let us show that
this process is finite, that is, that there is a k E N such that 97-k = 97+k. Assume
not. Then there is an infinite sequence
(*) ...,r) -k ,...,r) -2 -1
,r!
+1 +2
,9,7) ,r),...,r) ,...
of elements of eN constructed by the process above. It can be proved by induction
that the elements with even superscripts are of type 2 (we leave it as an exercise
for the reader to supply the details). If 7)+2i = 77+2j (for definiteness let i < j),
then by the simple relations 2i - (i + j) = i - j and 2j - (i + j) = - (i - j) we
get that = r)-(i-j), which contradicts the assumption that the sequence (*) is
infinite. An analogous contradiction follows from the equalities 7-2i = 7-2j and
-2j = 77+2i, i, j E N. Thus, the elements 7)28, s E N, of type 2 in (*) are distinct.
We show that there are finitely many elements of type 2 in CN. Indeed, suppose
that r) = [r)1, r)T] E CN is an element of type 2. Since nontrivial recurrent trajectories
are dense in N, each of 7)a and 7)T either belongs to an open Gutierrez equivalence
4. QUASIMINIMAL SETS 95

class, or lies on the boundary of an open Gutierrez equivalence class (see §3.6).
Denote by O(r)t) (respectively, O (rir)) an open Gutierrez equivalence class that
either is adjacent to the point (ri) or contains in (lr).
We take points a E O (97t) and b E O (97r) through which there pass nontrivial
recurrent trajectories in the set N. Then [a, b] contains the interval [l7t, Assume
that O(r)t) = O(r),.); that is, a and b belong to a single open Gutierrez equivalence
class. By the definition of Gutierrez equivalence, the closed curve [a, b] U aP(a) U
bP(b)U[P(a), P(b)] bounds a disk D. Since 97 is an element of type 2, there are points
in N on [P(a), P(b)], and hence [P(a), P(b)] must intersect nontrivial recurrent
trajectories in N (Figure 2.63). Entering D with decreasing time, these trajectories
cannot leave D as t -p -oo, and this contradicts Lemma 2.4, according to which
nontrivial recurrent semitrajectories cannot lie entirely in D. The contradiction
proves that O (ii) O (r)T) . The fact that there are finitely many elements of type
2 now follows from the fact that there are finitely many open Gutierrez equivalence
classes (Lemma 3.9).

p(o,)

FIGURE 2.63

Thus, the sequence (*) constructed by the process above is finite. The collection
97 -, ... ,-1 , 71 +1 ... ,97 +k

of elements obtained is called a saddle set. The index of this saddle set is equal to
1-k. Figures 2.61 and 2.62 picture saddle sets of indices -1 and -2, respectively.
We remark that the elements of type 2 with even superscripts in a saddle set
are all distinct and different from r), with the possible exception of the extreme
ones rj-' and 97+k . This is not so in general for elements with odd superscripts (see
Figure 2.64).
We sum up the arguments of this subsection in the form of a lemma.
LEMMA 4.2. Let eN be the partition of a contact-free cycle C. Then:
1) there are finitely many elements of type 2 in CN;
2) any saddle set in CN is finite;
3) in any saddle set
r)-' ,...,r)-l,7) - 7)0,7)+1,...,7)-I-k

all the elements with even superscripts are elements of type 2, and are distinct with
the possible exception of the extreme elements 7- and 97+k;
96 2. STRUCTURE OF LIMIT SETS

FIGURE 2.64

4) each endpoint l or T of any element [m, 7r] E N is an accumulation point


for the points lying in the intersection with C of the nontrivial recurrent trajectories
in the quasiminimal set N;
5) each endpoint m or 1r of any element [77z, ran] E N either belongs to an open
Gutierrez equivalence class or is a boundary point of an open Gutierrez equivalence
class.
4.5. The structure theorem. In this subsection we study the global struc-
ture of a flow on a quasiminimal set. We first give some definitions.
Let f : M - M be a transformation of a manifold M into itself with domain
Dom(f) C M. Assume that all the iterates f n (z), n E Z, are defined for some point
z E M; that is, z E Dom(f n) for n E Z (we set f ° = id). The set 0(z) = {f(z),
n E Z} is called the nonsingular orbit of z with respect to the transformation f.
DEFINITION. A transformation f : M -* M is said to be minimal if it has at
least one nonsingular orbit, and each nonsingular orbit of f is dense in M.
We now assume that M is orientable.
DEFINITION. A transformation f 1: NYC -* NYC is topologically semiconjugate to
a transformation f2: M -* M if there exists a continuous orientation-preserving
mapping h : M -* M such that ho f1(z) = f2 o h(z) at all points z E Dom(f 1) .

It follows from the definition that h[Dom(f 1)] C Dom(f2).


If in this definition we require that h be a homeomorphism, then the transfor-
mations f1 and f2 are topologically conjugate.
We say that f1 is topologically semiconjugate (conjugate) to f2 by means of h.
THEOREM 4.2 (structure theorem). Let f t be a flow on a closed orientable
two-dimensional manifold M, and let N1,.. . , Nk be quasiminimal sets of ft. Then
there exist disjoint open connected sets V1,.. . , Vk and contactfree cycles C1,.. . , Ck
for f t such that the following conditions hold:
1) either C2 n N2 = C2 or C2 n N2 is a Cantor set in C2, i = 1, ... , k, and
C2 n N = 0 for i j (a special family of contact free cycles) ;
2) if P2 : C2 -* C2 is the Poincare mapping induced on C2 by f t (i = 1, ... , k),
then
a) P2 is topologically conjugate to a minimal exchange of open intervals T2:
Si S1 in the case C2 n N2 = C2, and
b) P2 is topologically semiconjugate to a minimal exchange of open intervals
Ti : S1 51 by means of a continuous mapping hi : C2 -* 51 in the case when
4. QUASIMINIMAL SETS 97

C2 fl N2 is a Cantor set, where the closure of each adjacent interval of C2 fl N2 is


carried by hi into a point, and the restriction of hi to the complement of the closures
of all the adjacent intervals is a homeomorphism onto its range;
3) for each i = 1,... , k the domain V2 contains C2 and all the nontrivial recur-
rent semitrajectories in the quasiminimal set NZ;
4) the boundary aV2 of V can be made up only of whole trajectories, equilibrium
states, and finitely many contact free segments, and, moreover, there do not exist
arcs of trajectories with endpoints on aV2 and interior points in V.
PROOF. According to Lemma 4.1, there exists a special family of contact-free
cycles C1,. . , C, satisfying 1).
We fix an index i E {1,. . , k} and prove 2). Let2 be the partition of the
.

contact-free cycle C. We recall that the elements of are closures of intervals in


the set C2 \ N2 and points not belonging to these intervals.
Denote by ' the family of elements i E i such that the interval [771, 77r] lies in
a single open Gutierrez equivalence class. It follows from Lemma 4.2, 5) and the
finiteness of the number of open Gutierrez equivalence classes (Lemma 3.9) that
the complement2 \' has finitely many elements.
We remark that in general the complement2 \' can contain elements of type
1 in addition to elements of type 2 (Figure 2.65).

FIGURE 2.65
N
We define the mapping P2 : ' -* as follows. For an element = r)T] E '
we set P2 (77) = [P1(r11), P,. (7ir) ] E j, where P : C2 - C2 is the Poincare mapping
induced on C2 by ft.
We introduce an equivalence relation r%1(i) by regarding points belonging to a
single element of as equivalent. Then the quotient manifold Ci/r%i(i) is homeo-
morphic to the circle S1, and the natural projection hi : C2 -* C2 / (i) is either a
Cantor function2 (if C2 fl N2 is a Cantor set), or a homeomorphism (if C2 f1 N2 = Ci
and all the elements of are one-point sets).
The mapping P2 : ' -* induces a transformation T2 : S1 -* 51 by means of
the natural projection hi : CZ - 51 = Ci/r%1(i). This and the definition of PZ imply
the commutative diagram

hij
51 Ti

2More precisely, a covering R1 R1 over hi is a Cantor function.


98 2. STRUCTURE OF LIMIT SETS

at points belonging to elements in Z. Since the number of elements in is finite,


T2 is undefined at only finitely many points. Since trajectories depend continuously
on the initial conditions, T2 is a homeomorphism from its domain onto its range.
Therefore, T2 is an exchange of open intervals.
The partition2 contains one-point elements through which nontrivial recurrent
trajectories pass. Consequently, nonsingular orbits of T2 pass through the points of
the circle CZ /r#(i) = S1 that correspond to these elements. The denseness in NZ of
any nontrivial recurrent semitrajectory lying in N; implies the denseness in Si of
any nonsingular orbit of T. Therefore, T2 is a minimal exchange of open intervals.
If C2 n N2 = C2, then all the elements ofi are one-point sets, and hi is a
homeomorphism. If C2 n N2 is a Cantor set, then the complement C2 \ (C2 n N2) is a
countable union of open intervals Gk, k E N. By definition, the topological closure
cl Gk of each adjacent interval Gk is an element of 2, and hence is mapped into a
point under the action of hi. Each point not in cl Gk, k E N, is an element ofi.
Thus, hi is one-to-one on the set C2 \ U1 cl Gk. It follows from the continuous
dependence of trajectories on the initial conditions that the restriction of hi to
C2 \ U1 cl Gk is a homeomorphism onto its range. The assertion 2) is proved.
It remains to construct open domains V1,.. . , Vj satisfying the conditions 3)
and 4).
We again fix an index i E {1,. . . , k} and construct the domain V. Denote by
01,.. . , OT the open Gutierrez equivalence classes on C2 with respect to the mapping
P. By the definition of an open equivalence class, for any points m1i m2 E 03
(j E {1,. . . , r}) there exist points a, b E 03 in Dom(P2) such that m 1 m2 C ab C 03,
and the simple closed curve ab U aPZ (a) U bPZ (b) U P2 (a)P2 (b) bounds an open disk
D(a, b) on the manifold.
Let D (O3) = UD (a, b) and 0 = UP2 (a) P2 (b) , where in both expressions the
union is over all points a, b E Dom(P2) n 03. According to Lemma 3.9, each open
class 03 is an open interval (as, b3). Therefore, 0 is also an open interval (a, b).
We remark that o1,. .. , 0;. are open Gutierrez equivalence classes with respect to
the mapping Pi 1.
Each endpoint of the open equivalence class 03 is an endpoint of a compo-
nent of Dom(P2), so in view of Lemma 3.6 the semitrajectories l(a) and l(b)
are w-separatrices that intersect CZ only at a3 and b, respectively. Similarly, the
semitrajectories l - and l - (b) are a-separatrices, and 1- (as) n C2 = {a} and
l
Let V(0) = 03 U 0 U D(03) U l(a) U l(b) U l(a) U l+(b3) (Figure 2.66),
and let Vi = U=1 V(03).
N
It follows from Theorem 3.6, 4) that the set V2 contains all nontrivial recurrent
semitrajectories in the quasiminimal set N.
Two cases are possible: 1) C2 = U=1 cl(Oj); 2) CZ U=1 cl(0j).
ae
In case 1) the set Vi Vi is an open set, and its boundary aVi satisfies the
assertion 4).
In the case 2) the set C2 \ U=1 cl (0 j) is made up of finitely many intervals
E1, ... , Ev C C. For each interval E we construct an open disk D() as shown
in Figure 2.67 50 that the parts of the boundary D() not in V2 are transversal
4. QUASIMINIMAL SETS 99

FIGURE 2.66

FIGURE 2.67

to the flow. Let V = Vi U=1 The set V2 is open, and its boundary 8Vi
satisfies the assertion 4).
We go through the construction described for all i = 1, ... , k. By choosing
D() small enough we can make the domains V1,.. , Vk disjoint. U
.
CHAPTER 3

Topological Structure of a Flow


In the first section of this chapter we give the basic concepts of the qualitative
theory of the theory of flows topological and smooth equivalence, classification,
and so on. In the second section we prove that in a certain sense a flow on a
surface breaks up into flows that do not have nontrivial recurrent trajectories and
irreducible flows. In this section we give a special decomposition of a flow which is
due to Levitt and is based on cutting a surface by closed transversals. In the next
section we show that an irreducible flow either is highly transitive or is obtained
from a highly transitive flow by a blowing-up operation. The fourth section is
devoted to flows without nontrivial recurrent trajectories. For such flows we single
out a family of (singular) trajectories that determine the qualitative structure of the
flow, and we study the components of the complement of the singular trajectories
on the surface. In the conclusion we introduce a metric on the space of flows,
turning that space into a metric and topological space and enabling us to investigate
important classes of flows from the point of view of such topological concepts as
openness, denseness, and so on.

§1. Basic concepts of the qualitative theory


Classification problems occupy a significant place in the theory of dynamical
systems. Corresponding areas of the theory of dynamical systems are determined
by the equivalence relation used in the classification. The basic equivalence relation
in the qualitative theory of flows is topological equivalence. In the framework of
this theory two flows are taken to be the same (equivalent) if the spaces of their
trajectories have the same topological structure. In this section we give the main
definitions in the qualitative theory of flows.
1.1. Topological and smooth equivalence.
DEFINITION. Two flows f t and gt on a manifold M are said to be topologically
equivalent if there exists a homeomorphism h : M -p M carrying each trajectory of
one flow into a trajectory of the other.
If h preserves the direction in time (the positive direction) on trajectories, then
f t and gt are said to be topologically orbitally equivalent.
Any two rational windings on the torus can serve as examples of topologically
orbitally equivalent flows (and, in particular, topologically equivalent flows) (Figure
3.1). On the other hand, the flows on an annulus pictured in Figure 3.2 are not
topologically equivalent (and, in particular, not topologically orbitally equivalent).
These two concepts are the basic concepts in the qualitative theory of flows
(dynamical systems with continuous time).
101
102 3. TOPOLOGICAL STRUCTURE OF A FLOW

FIGURE 3.1

FIGURE 3.2

Two flows are said to have the same topological (qualitative) structure if they
are topologically or topologically orbitally equivalent.
Topological equivalence has the properties of reflexivity, symmetry, and tran-
sitivity. Therefore, the collection of all dynamical systems breaks up into disjoint
classes of systems with the same topological structure.
In establishing the topological equivalence of flows the question naturally arises
of local topological equivalence.

DEFINITION. Two flows f t and gt on a manifold M are said to be locally topo-


logically equivalent at the respective points m1i m2 E M if there exist neighborhoods
U (m1) and U (m2) of m1 and m2 and a homeomorphism h : U (m1) -* U (m) such
that h(m1) = m2 and h carries arcs inU(m1) of trajectories of f t into arcs inU(m2)
of trajectories of gt.
If h preserves the direction in time on arcs, then the flows f t and gt are said
to be locally topologically orbitally equivalent at the respective points m1 and m2.

According to the rectification theorem (Theorem 2.2 in Chapter 1), any flows
are locally topologically orbitally equivalent at regular points.
If in the above definition m1 and m2 are understood to be invariant compact
sets for the respective flows f t and gt and it is required in addition that the home-
omorphism h carry each trajectory in m1 into a trajectory in 7122, then we get the
definition of local topological equivalence of flows f t and gt on the sets m1 and m2.
In particular, the sets m1 and m2 can be equilibrium states, closed trajecto-
ries, and contours made up of equilibrium states and separatrices joining them.
Accordingly, we refer to local topological equivalence of equilibrium states, closed
trajectories, and contours.
1. BASIC CONCEPTS OF THE QUALITATIVE THEORY 103

The famous Grobman-Hartman theorem ([27], [74]) asserts that a hyperbolic


equilibrium state of a C1-flow on a finite-dimensional manifold is locally topologi-
cally orbitally equivalent to an equilibrium state of a flow given by a linear vector
field.
The contours pictured in Figure 3.3 are not locally topologically equivalent.

FIGURE 3.3

The flows in Figure 3.4 are not topologically equivalent, although their equilib-
rium states are locally topologically orbitally equivalent.

FIGURE 3.4

The concept of smooth equivalence is a generalization of topological equivalence


(a C°-diffeomorphism is understood to be a homeomorphism).
DEFINITION. Two C'-flows f t and gt (r > 0) on a manifold M are said to be
C'-smoothly equivalent (0 < k < r) if there exists a CI-diffeomorphism h : NYC -* NYC
carrying each trajectory of one flow into a trajectory of the other.
If h preserves the direction in time on trajectories, then the flows f t and gt are
said to be CI-smoothly orbitally equivalent.
The definitions of local CI-smooth equivalence and local CI-smooth orbital
equivalence are analogous to the definition of local equivalence.
According to the rectification theorem, any regular points of a C'-flow are
locally C'-smoothly equivalent.
1.2. Invariants. One of the main problems in the qualitative theory is to sin-
gle out quantities, characteristics, or properties of a dynamical system that coincide
for topologically equivalent dynamical systems and indicate by their difference that
dynamical systems belong to different topological equivalence classes. Such quan-
tities (characteristics, properties) are called topological Invariants of a dynamical
system.
104 3. TOPOLOGICAL STRUCTURE OF A FLOW

For example, the number of quasiminimal sets of a flow is a topological invari-


ant.
Invariants of smooth equivalence are called smooth invari ants. For example,
the characteristic numbers of the linear parts for equilibrium states of a C1-flow are
smooth invariants that are not topological invariants (if the flow has equilibrium
states).
There is no substantive universal topological invariant for all flows. A class
of flows is usually singled out in a special way, and then a topological invariant is
introduced for dynamical systems of this class.
For example, the number of equilibrium states and the number of closed trajec-
tories are topological invariants in the class of flows with finitely many equilibrium
states and closed trajectories.
For a given class 92 of dynamical systems we say that a topological invariant
is complete if two arbitrary dynamical systems in 92 are topologically equivalent
precisely when this topological invariant is the same for the systems.
For instance, in the class of minimal flows on the torus the Poincare rotation
number is a complete topological invariant up to recomputation with the help of a
unimodular integer matrix (1 in Chapter 6).
1.3. Classification. Let us consider some class 92 of dynamical systems. A
topological classification of the dynamical systems in 92 is defined to be a solution
of the following two problems: a) find a complete topological invariant for the
dynamical systems in 91; 2) the realization problem.
A realization is defined to be a determination of the admissible values of a
topological invariant, and the construction, from a given admissible topological
invariant, of a dynamical system in 92 with that topological invariant.
A C''-smooth (r > 1) classification of dynamical systems in the given class is
defined similarly.
Let 02 be the set of minimal flows on the torus. As mentioned earlier (and
as will be proved in §1 of Chapter 6), up to recomputation with the help of a
unimodular integer matrix, the Poincare rotation number is a complete topological
invariant of a flow in 02. The rotation number of any flow in 92 is an irrational
number. Conversely, for any irrational number µ E Ilk there exists a minimal flow
on the torus with Poincare rotation number equal to µ. Thus, both topological
classification problems are solved for the set of minimal flows on the torus.

§2. Decomposition of a flow


In this section we prove the existence of a decomposition of a flow into flows with
simpler topological structure. Namely, each flow in the decomposition either does
not contain nontrivial recurrent semitrajectories or contains only one quasiminimal
set, and it does not admit further nontrivial decomposition. Our presentation
follows [83]. Such a decomposition can also be obtained from the structure theorem.
In the conclusion we give a decomposition of Levitt [91] that differs essentially
from the one mentioned above and is based on cutting the manifold along closed
transversals of the flow.
2.1. Characteristic curves of a quasiminimal set. Let f t be a flow with
a quasiminimal set N on an orientable closed surface M. According to Lemma 4.1
in Chapter 2, there exists a contact-free cycle C for f t that intersects all nontrivial
2. DECOMPOSITION OF A FLOW 105

recurrent semitrajectories in N, with either N fl C = C or N fl C a Cantor set on


C
Let N be the corresponding partition of the contact-free cycle C. By virtue of
Lemma 4.2, each of the endpoints m and )T of any element _ 97T] E N either
belongs to an open Gutierrez equivalence class or is a boundary point of such a
class. Denote by F (i11) (respectively, F (71r)) an open class which either contains the
point it (i)r) or is such that 711 E aF (7l) (respectively, 7T E arr (7r)) .
Let r) _ [711, 71T] E N be an element of type 1. Since nontrivial recurrent
trajectories are dense in N, the open classes I`(7l) and I`(rir) contain respective
points i and T that lie in the domain Dom(P) of the Poincare mapping P : C -* C
and are such that the interval i contains points of the element 7j (it is possible
that 7i = 711, 77. = 7k)
The closed curve S(71) _ 7777.. U P (7i) P U i P() U 71.P(71) .is called a
characteristic curve of the element 77 E N (Figure 3.5).

FIGURE 3.5

Let - , ... , 7-1, 970,71+1,. .. , 71+k be a saddle set, and take an element 712 j =
[i, r] of it with even superscript together with points F(i) and E
I`(7Ti) belonging to the domain Dom(P).
The closed curve S(71-Ic,... , 7-1, 770, 71+i,. . . , 71+k) consisting of the segments
[j23, i], [p(23), p(i)] C C and the arcs Eli and iP(i), where 712j
runs through all the elements of the saddle set with even superscripts, is called a
characteristic curve of the saddle set 7-k, ... , 770, .. , 71+k (Figure 3.6, for k = 2).

FIGURE 3.6
106 3. TOPOLOGICAL STRUCTURE OF A FLOW

From the definition of Gutierrez equivalence classes it follows that a charac-


teristic curve (of an element of type 1 or a saddle set) is determined up to a free
homotopy.
DEFINITION. The family S(N, C) of characteristic curves constructed for all
elements of type 1 and saddle sets of the partition N is called the characteristic
family of curves of the quasiminimal set N.
We impose additional restrictions on the characteristic curves. Let =
be an element of N of type 1. Since there are no one-sided contours in the quasi-
minimal set N (otherwise, the limit set of each nontrivial recurrent semitrajectory
in N would coincide with a one-sided contour by Theorem 3.6 in Chapter 2), the el-
ement [P1(r)1), Pr (rJr )] E N does not intersect = rh]. Therefore, taking points
7]l E r E I'() sufficiently close to the respective points ril and 7T, we
get a simple (that is, without selfintersections) characteristic S().
If the elements in a saddle set -, ... are distinct with the ex-
ception of the extreme elements -k = +k, then a simple characteristic curve
+k) is constructed similarly.
In what follows we assume that characteristic curves constructed for elements
of type 1 and characteristic curves of saddle sets are simple under the condition
indicated above.
2.2. Periodic elements of a partition. In the notation of the preceding
subsection we give the following
DEFINITION. An element E N is said to be periodic if there exists a collection
of elements = r)1, rJ2i ... , TM = such that:
a) if the element riz = is of type 1, then z+1 = [P1 (i Z) ), Pr for
i E {1, ... , n - 1};
b) if a subfamily i1z1+1, ... , '/i2 is in a saddle set G = {_k,.. , 0, , + },

G for i1 > 1, and 112+1 G for i2 < n, then the first element 'lit of the
subfamily is an element of G with even superscript, while the last element rf2 of
the subfamily is an element of G with odd superscript (Figure 3.7);
c) except for the extreme elements r71 = ran, the elements of the collection
r)1 i ... , ran are distinct.

FIGURE 3.7

The elements rJ1 = m ... , ran = i are called a periodic chain containing the
periodic element r.
2. DECOMPOSITION OF A FLOW 107

FIGURE 3.8

Figure 3.8 shows a periodic element of the partition eN on a closed transversal


C of a flow ft; f t is transitive, and its unique quasiminimal set N coincides with
the pretzel.

LEMMA 2.1. On an orientable compact surface M let f t be a flow with quasi-


minimal set N, and let C be a special contact free cycle intersecting N (that is,
either C fl N = C or C f1 N is a Cantor set on C). Assume one of the following
conditions:
1) there is a periodic element of the partition ;
2) the family S(N, C) contains a characteristic curve that is not homotopic to
zero.
Then there exists a simple closed curve that is not homotopic to zero and does
not intersect nontrivial recurrent semitrajectories of ft.

PROOF. Suppose that 1) holds, and let ch(rl) = rl, X72, ... , ran = rl} be a
periodic chain containing the periodic element E N
We take an element Ti2 E ch(rl) and assume that = [, rlr] is a one-point set;
that is, i = r = r12. We show that the two semitrajectories l - (rl2) and l+ (rl2)
are a- and w-separatrices, respectively. Assume not: for definiteness assume that
l+ (re) is not an w-separatrix. Since 72 is a one-point set, the curve C contains
points arbitrarily close to rfrom both sides of rthat belong to nontrivial re-
current trajectories in N. Therefore, all the points of the intersection l + (rl2) fl C
form one-point elements of N in view of the theorem on continuous dependence
of trajectories on the initial conditions. Since a quasiminimal set does not contain
closed trajectories nor one-point contours (Theorem 3.6), it follows that the one-
point elements containing points in l+ (re) fl C are distinct. This contradicts the
periodicity of the element rl. The contradiction shows that for a one-point element
= [,J E ch(rl) the trajectory l (rl2) is a separatrix joining equilibrium states.
We take an element T2 E ch(rl) of type 1. It follows from the preceding re-
sult that there is a simple arc y(rl2) that has endpoints in the respective intervals
[rli , J and [Pz(iit), Pr()] and does not intersect nontrivial recurrent semitrajec-
tories (Figure 3.9).
An analogous arc 'y(71i1 , ... , j2) exists for a family {77j1, x121+1, ... , rf2 } C ch(rl)
that satisfies the condition b) in the definition of periodicity of an element rl.
Since rl is periodic, the arcs y(rl2) constructed for the elements rl2 E ch(rl) of type
1 and the arcs 'y(7121, ... , '/22) constructed for all subfamilies {h1,.. . , r122 } C ch(rl)
satisfying b) can be supplemented by segments of C so as to form a simple closed
curve 'y(rll, ... , rln) that does not intersect nontrivial recurrent semitrajectories.
108 3. TOPOLOGICAL STRUCTURE OF A FLOW

c c
FIGURE 3.9

FIGURE 3.10

Let us orient the curve 'y(771,. . , r). It follows from the construction of the
.

arcs 'y (j) and ! (' li 1 , ... , r1j) that the intersection index of the curve 'y
with the contact-free cycle C is nonzero. Since C is not homotopic to zero (Lemma
2.3 in Chapter 2), it follows that , r1n) is also not homotopic to zero.
'y(ll1,...

Suppose that the condition 2) holds, and let S(r1) be a characteristic curve
that is constructed for an element E N of type 1 and is not homotopic to zero.
If the elements _ [r11, r J and [Pi(rii), Pr (r1r)] are one-point sets, then there is a
simple closed curve S(r1) made up of equilibrium states and separatrices joining
equilibrium states that is homotopic to the characteristic curve S(r1) (Figure 3.11).
Then S(r1) is not homotopic to zero and does not intersect nontrivial recurrent
semitrajectories.

FIGURE 3.11
2. DECOMPOSITION OF A FLOW 109

If at least one of the elements and [P(j(rij), Pr (rir)] is not a one-point set,
then the characteristic curve S(ri) can be deformed "inside" the set M \ N to
form a simple curve S(ri) satisfying the assertion of the lemma (Figure 3.12). The
required curve can be constructed similarly from a characteristic curve of a saddle
set that is not homotopic to zero. 0

FIGURE 3.12
2.3. Criterion for a flow to be irreducible.
DEFINITION. A flow f t on a two-dimensional manifold M is said to be ir-
reducible if f t has only one quasiminimal set, and any closed curve that is not
homotopic to zero on M intersects at least one nontrivial recurrent semitrajectory
of ft.
Any highly transitive flow is irreducible. In the next section it will be shown
that an irreducible flow on an orientable surface either is highly transitive or can be
obtained from a highly transitive flow by means of a so-called blowing-up operation.
Suppose that a flow f t on a closed orientable surface M has a quasiminimal set
N (not unique in general). According to Lemma 4.1 in Chapter 2, there is a special
contact-free cycle CN satisfying the following conditions: 1) CN intersects N and
does not intersect quasiminimal sets different from N; 2) either CN f1 N = CN or
CN f1 N is a Cantor set.
In the given notation we formulate a criterion for a flow to be irreducible.
THEOREM 2.1. Suppose that f t is a flow with a quasiminimal set N on a
closed orientable surface M2, and let CN be a corresponding special contact-free
cycle. Then f t is irreducible if and only if all curves in the characteristic family
S(N, CN) of curves of N are homotopic to zero, and the partition N on CN does
not have periodic elements.
PROOF. NECESSITY. It follows from Lemma 2.1.
SUFFICIENCY. Suppose that all the curves in the characteristic family S(N, CN)
are not homotopic to zero, and the partition N on CN does not have periodic el-
ements. Then each curve S E S(N, CN) is simple and bounds a disk D (S) on
M2
We show that M = CN U D (S) is open and closed, where the union is over all
curves S in the family S (N, CN).
It follows from the construction of characteristic curves (see §2.1) that N C M.
110 3. TOPOLOGICAL STRUCTURE OF A FLOW

We prove that M is open. Since the disks D (S) are open, it suffices to show
that any point m E CN is an interior point. Any point m E CN belongs to some
element i E N, so there are two characteristic curves Si, S2 E S(N, CN) containing
m (Figure 3.13). It follows from the construction of characteristic curves that there
exists a neighborhood tL(m) of m such that tL(m) C D (S1) U CN U D (S2) . Therefore,
M is open.

FIGURE 3.13

Let m be a limit point of a sequence {m}1, mi E M. It will be assumed


that m, m2 CN (otherwise there is nothing to prove), m2 E D(Si), and Si E
S(N, CN). Since each disk D (Si) is bounded by the curve Si, m is a limit point
for the points m2 E Si, i = 1, 2, .... It follows from m CN and the construction
of characteristic curves that for sufficiently large i the points fft lie on nontrivial
recurrent trajectories of N, and hence m E N C M. It is proved that M is closed.
Since the surface M2 is connected, M = M2.
We show that f t has only one quasiminimal set N. Assume the contrary: let
N1 be a quasiminimal set different from N. Since N1 does not intersect CN and
2
= CN U D(S), it follows that N1 lies in one of the disks D(S), S E S (N, CN),
which contradicts Lemma 2.4 in Chapter 2.
Let y be a closed curve that is not homotopic to zero, and assume that it does
not intersect nontrivial recurrent semitrajectories. Then 'y lies in a union of finitely
many closed disks cl D (S1 ),, cl D (Sik) and can intersect their boundaries only
in segments of CN that are elements ofN . Then from the elements of eN that
intersect 'y we can form a periodic chain; that is, we can get a periodic element of
N . The contradiction means that on M2 there are no closed curves that are not
homotopic to zero and do not intersect nontrivial recurrent semitrajectories. This
concludes the proof that f t is irreducible. 0

REMARK. In the preceding arguments the contact-free cycle can be replaced


by a contact-free segment > with the additional condition that the endpoints of >
do not lie on nontrivial recurrent semitrajectories. We introduce the partition
of >, and the concepts of characteristic curves and a periodic element. It is proved
as above that if does not have periodic elements and all the characteristic curves
are homotopic to zero, then the flow is irreducible.
2.4. Decomposition of a flow into irreducible flows and flows without
nontrivial recurrent semitrajectories. Let f t be a flow on a two-dimensional
2. DECOMPOSITION OF A FLOW 111

manifold M. We consider a compact C°-submanifold SC C M. At each point of


the boundary aM we locate an equilibrium state, and we denote the resulting flow
by f f. If f t is given by a system of differential equations, then by multiplying the
right-hand side of the system by a function equal to 0 on aM and strictly positive on
M \ aM we get a system of differential equations determining the flow f. Note that
f i is not uniquely determined by f, but two such flows f i and f2 are topologically
orbitally equivalent (ff and f 2 can differ by phase velocities, but their trajectories
coincide).
The submanifold M is invariant with respect to the flow ff. Denote by f t 1
the restriction f flN of f f to this invariant submanifold.
We regard M as an independent manifold with the flow f fl = f t given on
it. Let us identify each component of aM with a point and denote the resulting
closed manifold by M. The flow f t 1 passes into a flow f * on M* , and each
component of aM under the natural mapping M -* M* passes into an equilibrium
state of ftk.
DEFINITION. The flow f t l is said to be irreducible if the flow f on Mis
irreducible.
We say that the flow f t I does not have nontrivial recurrent semitrajectories
if ftk does not have nontrivial recurrent semitrajectories on M.
THEOREM 2.2. Let f t be a flow on a closed orientable surface M. Then on M
there is a finite family E of simple closed curves C1,. , CT that are not homotopic
. .

to zero, have union U=1 CZ disjoint from nontrivial recurrent semitrajectories of


f, and are such that for the closure M of a component of the set NYC\Uz 1 CZ either
the flow f t 1 is irreducible or it does not have nontrivial recurrent semitrajectories.
PROOF. If f t does not have nontrivial recurrent semitrajectories, then the as-
sertion of the theorem is obvious. Therefore, we assume that f t has a quasiminimal
def
set N1. Let CNi = C1 be a special contact-free cycle (N1 n C1 to), let Nl = 1
be the partition on C1, and let S(N1i C1) be the characteristic family of the quasi-
minimal set N1.
By Lemma 2.1, for each periodic chain ... , ran of we take a simple closed
curve 'y(1,.. . , r) that is not homotopic to zero, and for each characteristic curve
S E S(N1i C1) not homotopic to zero we take a simple curve S that is not homotopic
to zero. Denote by E1 the family of all such curves rh), S. Since the
number of saddle sets is finite, the number of periodic (distinct) chains is also
finite. Similarly, we get that the number of characteristic curves constructed for
saddle sets is finite. It follows from Lemma 2.8 in Chapter 2 that the number
of characteristic curves S(ri) not homotopic to zero and constructed for elements
E 1 of type 1 is finite. Therefore, E 1 is a finite family.
We remark that the curves in E 1 intersect in general. It follows from the
construction that by a slight perturbation outside the quasiminimal sets we can
make the curves in E 1 intersect at a finite (possibly zero) number of points or in a
finite number of arcs.
According to Lemma 2.1, each curve in E 1 does not intersect nontrivial recur-
rent semitrajectories.
Let us cut the surface M successively along the curves of the family E 1. We
take the closure M1 of the component of M \ UC (CZ E E 1) containing N1. By the
112 3. TOPOLOGICAL STRUCTURE OF A FLOW

criterion for a flow to be irreducible (Theorem 2.1, and the remark after it), the
flow f t k;; is irreducible.
According to Theorem 4.1 in Chapter 2, f t has finitely many quasiminimal sets.
Therefore, there are only finitely many components of the set M \ UCZ (CZ E E 1)
different from M1 and containing nontrivial recurrent semitrajectories. Continuing
the process with these components, we get the required family of curves C1,. . . , C,.
and the required decomposition of the original flow f t into irreducible flows and
flows without nontrivial recurrent semitrajectories. 0

A family of curves C1, ... , Cr satisfying Theorem 2.2 is said to be reductive,


and each curve in this family is said to be a reductive curve.
REMARKS. 1) Gardiner [83] proved that a reductive family can be constructed
from semitransversals, that is, curves consisting of finitely many contact-free seg-
ments and finitely many arcs of trajectories.
2) Theorem 2.2 is analogous to Theorem B in Gutierrez's paper, Smoothing
continuous flows and a converse of the Denjoy-Schwarz theorem, An. Acad. Brasil.
Cienc. 51 (1979), 581-589.

FIGURE 3.14

3) In his dissertation [93] Levitt obtained a result (Theorem 111.4.1) analogous


to Theorem 2.2 for arational foliations on a compact surface that have singularities
only of saddle and thorn type (arational means that the foliation does not have
leaves homeomorphic to a circle, nor leaves joining singularities). We formulate the
result of Levitt. The transformations of a foliation in a neighborhood of a saddle
pictured in Figure 3.14 are called Whitehead transformations. Foliations obtained
from each other by Whitehead transformations are said to be Whitehead equivalent.
THEOREM [93]. Let 3 be an arational foliation on a compact surface M. Then
there exist a foliation 31 Whitehead equivalent to 3 and a finite family {C}..1 of
disjoint transversals of such that no curve in the family {C}..1 intersects quasi-
minimal sets of 31, and there is at most one quasiminimal set in any component
of the set NYC \ Uz 1 CZ
2.5. The Levitt decomposition. We denote by the set of flows on a
closed orientable surface Mp of genus p > 2 that have only saddles as equilibrium
states and that do not have separatrices joining equilibrium states.
The setp contains all transitive flows on Mp with structurally stable saddles.
2. DECOMPOSITION OF A FLOW 113

We consider a flow f t Etp and a subset E C Mp homeomorphic to a closed


disk with two holes.
DEFINITION. We say that a flow f t Etp has standard structure on E if: 1)
f t has exactly one equilibrium state (a saddle) on E; 2) all the components of the
boundary DE are contact-free cycles of f t; 3) there are no closed trajectories of f t
on E (Figure 3.15).

FIGURE 3.15

A flow f t Etp has an almost standard structure on E if the standard structure


conditions 1) and 2) hold for f t and instead of 3) it is assumed that E has closed
trajectories of ft that are homotopic to components of DE (Figure 3.16).

FIGURE 3.16

In other words, an almost standard structure is obtained from a standard struc-


ture by attaching to the components of DE annuli on which f t has closed trajectories
but not equilibrium states, with the boundaries of the annuli transversal to ft.
THEOREM 2.3. For any flow f t E 3, p > 2, there exists a finite family
C1i... , Ck of contact free cycles such that: 1) the closure of each component of
the set Mp \ U=1 CZ is homeomorphic to a disk E with two holes; 2) the curves
114 3. TOPOLOGICAL STRUCTURE OF A FLOW

C1,.. . , Cj do not intersect closed trajectories of f t; 3) f t has standard or almost


standard structure on the closure of each component of Mp \ U=1 C.
PROOF. By the definition of the class gyp, the equilibrium states of f t are
isolated. Therefore, f t can have only finitely many equilibrium states on the closed
surface M. Since the index of each saddle is equal to -1, and the sum of all the
indices of f t is equal to 2 - 2p < -2 (Theorem 4.3 in Chapter 1), f t has a nonzero
finite number of saddles S1, ... , Sa.
For each saddle SZ we construct a closed neighborhood Ei homeomorphic to E
on which f t has standard structure.
We suppose that we have already constructed j - 1 neighborhoods, where 1 <
j < 1, and we take the saddle S, which belongs to the component M of Mp \
UI E. The saddle S3 has two a-separatrices 13 and l and two w-separatrices.
The boundary &M of M either is empty or consists of contact-free cycles that do
not intersect closed trajectories of f t.
Inside M we construct contact-free cycles C and C' with the following prop-
erties:
a) either C = C' or C n C' = 0;
b) C and C' do not intersect closed trajectories of ft;
c) with increasing time the a-separatrix 13 (respectively, intersects C (C')
at a point m (m'), and the semitrajectory l (m) (respectively, (m')) is disjoint
from C' (C).
Obviously, there are contact-free cycles C and C' satisfying the conditions a)-c)
if 13 and l intersect &M (recall that there are no closed trajectories of f t in some
neighborhood of 3M3).
Assume that 13 is disjoint from &M. It follows from the definition of the class
p of flows and Theorem 3.6 in Chapter 2 that the w-limit set of 13 is a closed
trajectory, a one-sided contour, or a quasiminimal set. In all cases there exists
a contact-free segment > intersecting 13 at more than one point. Therefore, by
Lemma 1.2 in Chapter 2, there is a contact-free cycle C intersecting 13. Since
w (13) C int M, we can construct a cycle C lying in int M according to the proof
of Lemma 1.2 in Chapter 2. Further, since 13 is a nonclosed trajectory, there is a
contact-free segment > disjoint from the closed trajectories of f Therefore,. there
exists a contact-free cycle C disjoint from the closed trajectories (Figure 3.17 shows
the cycle C in the case when w (l) consists of a closed trajectory).

FIGURE 3.17
2. DECOMPOSITION OF A FLOW 115

If the a-separatrix l intersects C, then we set C' = C. If l does not intersect


C but does intersect &M, then we take C' to be a closed transversal in a sufficiently
small neighborhood of the component of &M intersecting l. If l intersects neither
C nor &M, then the construction of the cycle C' is analogous to that of the cycle C
for 13. As a result we obtain cycles C and C' (C f1 C' = 0) satisfying the conditions
a)-c).
We proceed to the construction of the closed neighborhood E3. Let C = C'. By
the definition of the class gyp, there are four hyperbolic sectors in a neighborhood of
the curve l (m) U S3 U l( m'). The points m and m' lie on a single contact-free
cycle C. Therefore, since Mp is orientable, there exist in a neighborhood of the
union C U l (m) U S3 U l( m') two contact-free cycles C1 and C2 which together
0
with C bound a domain E3 on Mp containing S3 (Figure 3.18). It is not hard to
0
see that the closed neighborhood E = cl(E3) is homeomorphic to a disk with two
holes, and the flow f t has standard structure on E3.

FIGURE 3.18

Let C C'. As above, there exists in a neighborhood of C U l (m) U S3 U


l( m') U C' a contact-free cycle C3 which together with C and C' bounds a
0 0
domain E on Mp, and f t has standard structure on E = cl(E3).

FIGURE 3.19

Continuing this process, we construct for each saddle SZ (i = 1, ... , l) a closed


neighborhood EZ on which f t has standard structure.
We take the closure K of a component of the set MP \ UZ_ 1 E. By construction,
each component of the boundary aK of K is a closed transversal of f t . According
116 3. TOPOLOGICAL STRUCTURE OF A FLOW

to the definition of gyp, there are no equilibrium states interior to K. Then K is


homeomorphic to an annulus by Corollary 4.5 in Chapter 1. D

It follows from Theorem 2.3 that if a flow f t Ep does not have closed tra-
jectories, then there exists a family of contact-free cycles that partition Mp into
submanifolds with standard structure. The flow f t can be represented as pictured
in Figure 3.20.

FIGURE 3.20

Remark 1. The canonical decomposition. In [91] Levitt generalized The-


orem 2.3 to foliations on surfaces with boundary. We sketch this generalization.
Denote by Mp,b an orientable compact surface of genus p > 0 with b > 0
boundary components, and by 3 p,b the class of orientable foliations on Mp,b that
have singularities only of saddle type and are transversal to the boundary 0Mp,b .

Besides this we require that each foliation in p,b not have leaves joining singularities
and that any compact leaf intersect at least one closed transversal of the foliation.
Since we are considering orientable foliations, for each foliation' E'p,b the
components of 0Mp,b can be broken up into "incoming" and "outgoing" compo-
nents. Denote by p,r,s the class of foliations 3 E Yp,b such that 3 has r > 0 in-
coming components of 0Mp,b and s > 0 outgoing components (obviously, r + s = b).
For an arbitrary triple (p, r, s) with 2p + r + s > 0 Levitt proposed two models
Mp,r,s and which he called canonical, of foliations belonging to 3 p,r,s .
It is proved in [91] that any foliation 3 E p,r,s (where 2p + r + s > 0) is
topologically equivalent either to the foliation Mp,r,s or to the foliation Mrs,
according to the following conditions: 1) any semileaf of intersects 0Mp,b; 2)
there exists a semileaf of disjoint from 0Mp,b.
An analogous canonical representation of orientable foliations on nonorientable
compact surfaces was obtained in V. Nordon's paper,. Description canonique de
champs de vecteur sur une surface, Ann. Inst. Fourier 32 (1982), no. 4, 151-156.
3. THE STRUCTURE OF AN IRREDUCIBLE FLOW 117

Remark 2. The center of a flow. The center IZ (f t) of a flow f t is defined


to be the closure of the points lying on recurrent semitrajectories of f.
One method of determining the center of f t goes back to Birkhoff. Let N be
an invariant set of a flow f t on a manifold M. We denote by f t I N the restriction of
f t to N, that is, the one-parameter group of homeomorphisms of N induced by f t .

The definition of the nonwandering set NW (f t N) of the flow f t N is analogous to


§1.5 in Chapter 2. Let Ii = NW (f t), ... , IZZ+1 = NW (f t ),. . , ft = fli<w SZi
.

(where w is the ordinal type or ordinal number of the set of natural numbers),
IZw+1 =
There exists a smallest ordinal number c for which 1 = IZa+1 = .... It can
be shown that 1L is the center of the flow f t (there is a proof of this in [61]). The
ordinal number c is called the depth of the center.
The center of a flow f t is the largest closed invariant set N C M such that the
flow f tIN does not have wandering trajectories. The depth of the center of a flow is
one of the basic topological invariants characterizing the topological structure of a
flow, and it shows the number of steps needed to reach the center in the transfinite
process described above.
In a paper of A. J. Schwartz and E. S. Thomas (The depth of the center of
2-manifolds, Proc. Sympos. Pure Math., vol. 14, Amer. Math. Soc., Providence,
RI, 1970, pp. 253-264) it is proved that the depth of the center of any flow on
an orientable compact surface does not exceed 2. On a nonorientable compact
surface the depth of the center of any flow does not exceed 3 (see E. S. Thomas,
Flows on nonorientable 2-manifolds, J. Differential Equations 7 (1970), 448-453).
The estimates are sharp in the orientable and nonorientable cases. (We mention
that both the cited papers treat flows for a collection of surfaces including compact
surfaces.)

§3. The structure of an irreducible flow


A transitive flow is irreducible. We recall that a flow f t on M is said to be
transitive if there is a semitrajectory of f t that is dense in M. Obviously, this
semitrajectory is a locally dense nontrivial recurrent semitrajectory, and the two-
dimensional manifold M is itself a quasiminimal set.
The main result in this section is a proof that an irreducible flow on an ori-
entable closed surface either is highly transitive or is obtained from a highly tran-
sitive flow by means of a blowing-up operation.
3.1. Blowing-down and blowing-up operations. Let f t be a flow on a
manifold M.
DEFINITION. A Poisson pencil is defined to be either a single nontrivial recur-
rent trajectory, or, in the case of a transitive flow f, a collection of equilibrium
states (saddles) joined by separatrices, together with all the separatrices of these
equilibrium states (Figure 3.21).
THEOREM 3.1. Let f t be an irreducible flow on a closed orientable surface M
with finitely many equilibrium states, and suppose that each equilibrium state has
finitely many separatrices (possibly none) tending to it. Then there exist a highly
transitive flow gt on M and a continuous mapping h : M -* M homotopic to the
identity and with the following properties:
1) if S is a Poisson pencil of the flow gt, then h-1(S) is an invariant set of f t;
118 3. TOPOLOGICAL STRUCTURE OF A FLOW

FIGURE 3.21

2) if L is a nontrivial recurrent trajectory of gt, then h-1(L) contains at most


two nontrivial recurrent trajectories and at least one nontrivial recurrent semitra-
jectory of f, and, furthermore,
a) if h' (L) contains two nontrivial recurrent trajectories, then they both lie
on the accessible (from within) boundary of h-1(L),1
b) if h' (L) contains a single trajectory 1, then l is a nontrivial recurrent
trajectory, and the restriction hIl : l -- L is a homeomorphism onto its image;
3) any P+ (P-) nontrivial recurrent trajectory of f t is mapped by h home-
omorphically onto its image, which is a union of finitely many saddles (possibly
none), finitely many separatrices (possibly none) joining these saddles, and a single
P+ (P-) nontrivial recurrent trajectory of gt;
4) if m, h(m) E M are regular points of the respective flows f t and gt lying on
nontrivial recurrent semitrajectories of these flows, then h preserves the direction
in time in some neighborhood of m;
5) if N is the quasiminimal set of f t (by assumption, f t has exactly one quasi-
minimal set), then h(N) = NYC;
6) for any point m E M the set h' (m) is arcwise connected and contractible;
7) let h = 9 o hl, where the mappings 9 and h1 satisfy the conditions 1)-6) with
the corresponding transitive flows ge and g. Then 9 is a homeomorphism realizing
a topological equivalence between the flows gt and g.
PROOF. The flow gt is obtained from f t by "removing" (blowing down) the
domains lying in the complement of the quasiminimal set N.
We first show that if f t is a highly transitive flow, then we can set h = id (that
is, it is impossible to "blow down" any further). For this we prove the property
7). Assume that 9 is not a homeomorphism, and that h, 9, and h1 satisfy the
conditions 1)-6). This assumption and the transitivity of the flow gt imply the
existence of a point m E M lying on a nontrivial recurrent semitrajectory l of gt
such that 9-1(m) contains at least two points. According to 1) and 2), the set
9-1(l) is invariant and contains a nontrivial recurrent semitrajectory L of the flow
g. If 9-1(l) contains only one nontrivial recurrent semitrajectory, then L = 9-1(l),
and OIL is a homeomorphism onto its image (the property 3) ), which contradicts
our assumption. If 9-1(l) contains two nontrivial recurrent semitrajectories, then
by 2) both of them lie on the accessible (from within) boundary of 9-1(l), and
hence 9-1(l) contains interior points. According to 2), the invariant set int 9-1(l)
(the set of interior points) does not contain nontrivial recurrent semitrajectories,
and this contradicts the transitivity of the flow g. The assertion 7) is proved.

1 The accessible (from within) boundary 6K of the set K is defined to be the subset of the
boundary 8K such that for any point x E 6K C 8K there is an arc A with x as one endpoint and
all its remaining points interior to K.
3. THE STRUCTURE OF AN IRREDUCIBLE FLOW 119

It follows from 7) that if f t is transitive and h satisfies the conditions 1)-6),


then h realizes a topological equivalence between f t and gt. Obviously, h = id
satisfies 1)-6).
The flow f t will be assumed to be nontransitive in what follows. For such a
flow we construct h as the composition of two quotient mappings.
Denote by Ti the family of simply connected domains bounded by separatrices
going from an equilibrium state to an equilibrium state (Figure 3.22). By our
assumption that there are finitely many equilibrium states and separatrices, the
family D consists of finitely many domains.

FIGURE 3.22

We take a quotient of M, regarding all the points in the closure of each domain
D E D as equivalent. Then the resulting quotient manifold M1 is homeomorphic
to M, and the natural projection '- : M -- M1 is a continuous mapping homotopic
to the identity.
The regular trajectories of f t induce on M1 via the mapping r a family of
curves that satisfies the Whitney theorem (Theorem 2.3 in Chapter 1); therefore,
this family can be imbedded in a flow f i The points 'r(Ti) (Ti E Ti) and the
.

images of the equilibrium states of f t under T are equilibrium states of the flow
f i By construction, f i is irreducible and has exactly one quasiminimal set N1.
.

We remark that each nontrivial recurrent semitrajectory of f t is mapped by T


homeomorphically into a nontrivial recurrent semitrajectory of f i , and, conversely,
for any nontrivial recurrent semitrajectory 11 of f i the complete inverse image
T-1(ll) is a nontrivial recurrent semitrajectory of f. It can be assumed without
loss of generality that T preserves the direction in time on such semitrajectories.
According to Lemma 4.1 in Chapter 2, there exists a contact-free cycle C for
the flow f i such that either C fl N1 = C or C fl N1 is a Cantor set. Let P : C -p C
be the Poincare mapping, and the corresponding partition on the cycle C (see
§4.3 in Chapter 2).
We consider an element = E of type 1. Let P(ry) stand for the
element [Pj(r11), Pr E Since f i is irreducible,
. P(ry) by virtue of Theorem
2.1, and the characteristic curve E S(N1, C) is a simple closed curve.
If rll E Dom(P), then we denote by A(r)1, Pl (r)1)) the arc of the trajectory l (r)l )
between rll and Pl (r)l) . If 1Dom(P), then r)l is the right endpoint of a component
of the domain Dom(P). It follows from Lemma 3.7 in Chapter 2 that there is a
sequence of separatrices l+() = 11i ... ,1k such that the separatrix l2 (1 < i < k) is
a Bendixson extension of l2_ 1 to the left, and the separatrices 11, ... , lk_ 1 are disjoint
from C, but 1k intersects C first at the point Pl (r)1) as time increases. In this case we
denote by A(r)1, Pl (r)1)) the union of the separatrices l+ (r)l) ,12, ... , lk_ 1, l - (Pl
and the equilibrium states to which these separatrices tend.
120 3. TOPOLOGICAL STRUCTURE OF A FLOW

The curve PT(?],.)) is defined similarly (Figure 3.23).

A(2;P(r))

FIGURE 3.23

The closed curve S(ri) = [nj, J U Pi (ni)) U [P1 (i , Pr(?Ir)] U A(?]r, Pr (?]r))
is homotopic to the characteristic curve S(ri). Since S(ri) bounds a disk D(ry) on
M1 and S(ri) C T(i), S(ri) bounds a closed set C T(i) (Figure 3.23). We
remark that is not homeomorphic to a closed disk in general. In particular, for
a continuum of one-point elements the set is the arc rll The possible
types of the sets are pictured in Figure 3.24.

FIGURE 3.24

The subsets of homeomorphic to a closed disk are broken up into segments


Ia in such a way that they form a foliation, and each segment Ia joins points on
the curves P1(1)) and Pr (?]r)) (Figure 3.24).
We realize an analogous construction for the saddle set -, ... ,°, ... ,+.
Since f i is irreducible, the characteristic curve ,°, ... , +k) is homo-
topic to zero. As above, we define a closed curve +k) that is
homotopic to the characteristic curve +k) and that bounds a closed
set R(rrIc,... , ... , +k) (Figure 3.25). The subsets of +k)
homeomorphic to a closed disk are broken up into segments Ia forming a foliation
with one singularity; namely, if the saddle set -, ... , +k has index 1 - k, then
k ... ,
the foliation of segments Ia has a single saddle singularity o +k) (of index
1 - k) with 2k separatrices (Figure 3.25).
We take a quotient of the manifold M1, regarding the points lying on each Ia
as equivalent. Since the segments Ia form a foliation on simply connected subsets
homeomorphic to a disk, the resulting quotient manifold M2 is homeomorphic to
M1, and hence to the original manifold M. Denote by Tl : M1 --f M2 the natural
projection.
The subsets of and +k) (where runs through all the
elements of type 1, and the family -, ... ,°, ... ,+ runs through all saddle sets
3. THE STRUCTURE OF AN IRREDUCIBLE FLOW 121

FIGURE 3.25

of the partition ) homeomorphic to a closed disk are transformed under the action
of Tl into arcs and #q+k). Since the closed intervals [1,']r] C C
corresponding to elements of with more than one point belong to the family
{Ia}, the arcs and +k) can intersect each other only at the end-
points. Therefore, the curves formed by these arcs and the images of the nontrivial
recurrent semitrajectories of f i under 'Ti determine a foliation' on M2 with sin-
gularity set z(3). We remark that 3 has only saddle singularities at the points
Tl +k)), where {_1c,.. , +k} runs through all the saddle sets of
. .

By construction, the family of regular leaves of (that is, the leaves lying in
M2 \ z(3)) satisfy Whitney's theorem (Theorem 2.3 in Chapter 1) ; therefore, this
family can be imbedded in a flow f 2 .

Since f i is irreducible, the complement M1 \ N1 is covered by the sets


and R(rf, ... , +k), and thus through each point of M1 \ N1 there passes some
segment Ia in the foliation {Ia} of segments constructed above. Each segment Ia
is mapped by Tl into a point. Since the endpoints of any Ia belong to N1, it follows
that T1(Nl) = M2, and the flow f2 is transitive.
By construction, Tl maps the nontrivial recurrent semitrajectories of f i homeo-
morphically onto their images. Let Tl (ml) = m2, where m1 E M1 and m2 E M2 are
regular points of the respective flows fi and f2, and suppose that m1 lies on a non-
trivial recurrent semitrajectory 4) ) of f i . It can be assumed that in a neighborhood
of m1 on 4) the mapping 'Ti preserves the direction in time (otherwise, we replace
the motion in time for the flow f2 by its opposite motion). Since the foliation 3
is orientable, 'Ti then preserves the direction in time for any regular points m1 and
m2i 'Ti(ml) = m2, with the above properties.
By construction, 'Ti is a continuous mapping homotopic to the identity. We
show that the transformation h = Tl o 'T and the flow gt def f2 satisfy the theorem.
Indeed, 1), 3), 4), 5), and 6) follow immediately from the definitions of T and
'Ti. We prove 2). Let L be a nontrivial recurrent trajectory of gt, and assume
that a nontrivial recurrent semitrajectory l of f t passes through an interior point
m of the set h-1(L). Let be a contact-free segment containing m and lying in
int [h-1(L)] . In view of Lemma 1.2 in Chapter 2 and the invariance of int [h-1(L)] we
can construct a contact-free cycle C* intersecting l and lying in h-1(L). According
to Lemma 2.3 in Chapter 2, C* is not homotopic to zero, which contradicts the
fact that h is homotopic to the identity. This and the fact that any nontrivial
122 3. TOPOLOGICAL STRUCTURE OF A FLOW

recurrent semitrajectory of f i intersects the contact-free cycle C gives us that if


h-1(L) contains interior points, then any nontrivial recurrent semitrajectory of f t
in the set h-1(L) lies on the accessible (from within) boundary of h-1(L).
Suppose that h' (L) contains more than two nontrivial recurrent trajectories.
Then Ti 1(L) contains at least three nontrivial recurrent trajectories 11, 12, and l3
of f i . Since 11, 12, and l3 are nontrivial recurrent trajectories, each of them is a
component of the accessible (from within) boundary of Ti 1(L). By the construction
of r1, the set Ti 1(L) then contains saddle sets of the partition , and this contradicts
the fact that L is a trajectory that is not a separatrix. Thus, h' (L) contains at
most two nontrivial recurrent trajectories.
Suppose that h-1(L) contains one trajectory l of f. Since h-1(L) is invariant,
l = h-1(L). According to 5), h(N) = M2, and hence l C N. Since h-1(L) contains
only one trajectory, l is not a separatrix of an equilibrium state. It follows from
Theorem 3.4 in Chapter 2 that l is a nontrivial recurrent trajectory. 0

DEFINITION. A mapping h satisfying 1)-7) in Theorem 3.1 is called a blowing-


down operation (or a blowing down) of the flow f t to the flow gt . If gt is obtained
from f t by a blowing-down operation, then we say that f t is obtained from gt by
a blowing-up operation.
For an irreducible flow f t the construction of a blowing-down operation and
a highly transitive flow gt satisfying Theorem 3.1 is not unique. The connection
between different blowing-down operations and highly transitive flows for a fixed
flow f t is given in the following theorem of Gardiner [83].
THEOREM 3.2. Let f t be an irreducible flow on a closed orientable surface
M, and let h1i h2 : M - M be blowing-down operations of f t to highly transitive
flows gi and g2, respectively. Then gi and g2 are topologically equivalent by a
homeomorphism v: M --f M (carrying a trajectory of gi into a trajectory of g2)
such that for any point m E M the points v o h1 (m) and h2 (m) belong to a single
Poisson pencil of g2.
3.2. Irreducible flows on the torus.
LEMMA 3.1. Any flow f t on the torus T2 having a nontrivial recurrent semi-
trajectory is irreducible.
PROOF. By assumption, f t has a quasiminimal set N. Since the genus of
the torus is 1, it follows from Theorem 4.2 in Chapter 2 that N is the unique
quasiminimal set of f t .
Let -y be a closed curve on T2 that is not homotopic to zero. We show that y
intersects any nontrivial recurrent trajectory l of f t Assume the contrary. Then
.

there exists a simple closed curve y C T2 that is not homotopic to zero and disjoint
from 1. The set T2 \ y is homeomorphic to an annulus K, and l C K, which
contradicts Lemma 2.4 in Chapter 2. 0

Recall that a Denjoy flow is defined to be a flow f t without equilibrium states


on T2 that has a limit set of Cantor type.
LEMMA 3.2. A flow f t without equilibrium states on T2 is a Denjoy flow if
and only if f t is an irreducible nontransitive flow.
3. THE STRUCTURE OF AN IRREDUCIBLE FLOW 123

PROOF. Let f t be a Denjoy flow. According to the catalogue of limit sets


(Theorem 3.6 in Chapter 2), f t has a quasiminimal set N.
We show that f t does not have closed trajectories. For if it had a closed
trajectory homotopic to zero, then it would have an equilibrium state by virtue of
Corollary 4.3 in Chapter 1. Further, by Lemma 3.1, f t is irreducible, and hence
does not have closed trajectories which are not homotopic to zero.
Since f t does not have equilibrium states nor closed trajectories, the w- and
cr-limit sets of any trajectory of f t coincide with N. Consequently, N is a limit set
of Cantor type and N T2. This implies that f t is not transitive.
Let f t be an irreducible nontransitive flow, and let N be the unique quasimin-
imal set of f. Then f t does not have closed trajectories. This and the absence of
equilibrium states imply that N is the unique minimal set of f t . The nontransi-
tivity gives us that N T2. Therefore, N is locally homeomorphic to the direct
product of a closed bounded interval and the Cantor set in view of Theorem 3.7 in
Chapter 2; that is, N is a limit set of Cantor type. 0
COROLLARY 3.1. Let f t be a Denjoy flow. Then:
a) f t does not have closed trajectories;
b) f t has a unique quasiminimal set, which coincides with the unique minimal
set N of f t;
c) N is nowhere dense and locally homeomorphic to the direct product of a
closed bounded interval and the Cantor set.
DEFINITION. A flow f t on a manifold M is said to be minimal if M is a minimal
set of f.
It follows from the definition and Lemma 1.7 in Chapter 2 that any semitrajec-
tory or trajectory of a minimal flow is dense in the manifold. Obviously, a minimal
flow is transitive.
According to Theorem 3.7 in Chapter 2, the torus is the only orientable closed
two-dimensional manifold on which there is a minimal flow. It can be shown that
minimal flows do not exist on nonorientable closed surfaces.
LEMMA 3.3. Suppose that f t is a Denjoy flow. Then there exists a blowing-
down operation h : T2 -- T2 of f t to a minimal flow gt on T2 with the following
properties:
1) h carries each trajectory of f t into a trajectory of gt with preservation of the
direction in time;
2) h maps any nontrivial recurrent trajectory of f t homeomorphically onto its
image (which is a nontrivial recurrent trajectory of gt) ;
3) h(N) = T2, where N is the unique minimal (quasiminimal) set of f t;
4) if w is a component of the set T2 \ N, then
a) w is simply connected,
b) the accessible (from within) boundary of w consists of precisely two trajecto-
ries ll and l2 belonging to N,
c) h(w U ll U l2) is a trajectory of gt;
0
5) if N C N consists of the trajectories not lying on the accessible (from within)
0
boundary of any component of the set T2 \ N, then the restriction hIN of h to N is
0
a homeomorpism of N onto its image.
124 3. TOPOLOGICAL STRUCTURE OF A FLOW

PROOF. The simple connectedness of the component w follows from the irre-
ducibility of f t (Lemma 3.2). The remaining properties follow from Theorem 3.1.
0
REMARK. In [20] there is a proof of Lemma 3.3 independent of Theorem 3.1,
and the mapping h is constructed explicitly ([20], published in 1976, and Theorem
3.1 was proved in [83], published in 1985).
LEMMA 3.4. 1) Any minimal low on T 2 is topologically equivalent to the sus-
pension over a minimal homeomorphism of the circle S1.
2) Any transitive flow on T2 with finitely many equilibrium states is obtained
from a minimal flow by adjoining a certain number of impassable grains (in partic-
ular, the equilibrium states of a transitive flow on T 2 are impassable grains).
3) A transitive flow without equilibrium states on T2 is minimal.
PROOF. 1) By Lemma 2.3 in Chapter 2, there exists a contact-free cycle C for
f t that is not homotopic to zero. According to Lemma 2.4 in the same chapter, any
semitrajectory of a minimal flow f t intersects C; that is, C is a global section of
the flow. Therefore, the Poincare mapping P : C --f C is a homeomorphism. Since
the flow is minimal, the homeomorphism P is minimal (Lemma 2.1 in Chapter 1).
The fact that C is not homotopic to zero implies that T2 \ C is homeomorphic to
an open annulus. The trajectories of the flow pass from one boundary component
of the annulus to the other with increasing time because there are no equilibrium
states (Figure 3.26). Therefore, f t is topologically equivalent to the suspension over
the homeomorphism P : C --f C.

FIGURE 3.26

We prove 2). It follows from the transitivity of f t that its equilibrium states
are topological saddles (Corollary 3.1 in Chapter 2). Since the sum of the indices
of all the equilibrium states is equal to the Euler characteristic of the surface (the
Euler characteristic of the torus is equal to 0) according to Theorem 4.3 in Chap-
ter 1, and since any topological saddle has nonpositive index, the index of each
equilibrium state of f t is equal to zero. Consequently, each equilibrium state is a
topological saddle with two hyperbolic sectors (Theorem 4.1 in Chapter 1), that is,
is an impassable grain (Figure 3.27).
To conclude the proof it remains to show that a transitive flow gt on T2 without
equilibrium states is minimal. Indeed, since gt is irreducible and does not have
4. FLOWS WITHOUT NONTRIVIAL RECURRENT TRAJECTORIES 125

equilibrium states, it does not have closed trajectories. It follows from Theorem
3.7 in Chapter 2 (the catalogue of minimal sets) that T2 is a minimal set of gt . D

§4. Flows without nontrivial recurrent trajectories


In this section we consider flows not having nontrivial recurrent trajectories
(and hence semitrajectories; see Cherry's theorem in Chapter 2). While topological
invariants connected with the asymptotic behavior of recurrent semitrajectories are
effective (see Chapter 6) in the investigation of the topological structure of a flow
having nontrivial recurrent trajectories (in particular, of an irreducible flow), in the
investigation of flows without nontrivial recurrent trajectories a large role is played
by trajectories separating the manifold into domains in which the trajectories have
the same behavior (the meaning of the concept of "the same" behavior is revealed
in Lemma 4.1). These domains are called cells, and the trajectories separating the
manifold into cells are called singular trajectories.
The definition of singular trajectories is connected with the class of flows being
considered. For flows on the two-dimensional sphere the singular trajectories, which
determine the qualitative structure of the flow, were found by Leontovich and Mater
([50], [51]). In these papers the concept of a singular trajectory is based on the
concept of an orbitally unstable trajectory ([3], [4]). Another approach for deter-
mining the singular trajectories, based on the concept of topological equivalence,
was presented in [29].
4.1. Singular trajectories. We define singular trajectories of flows on sur-
faces by explicitly listing such trajectories.
DEFINITION. Let f t be a flow without nontrivial recurrent trajectories on a
closed surface M. The singular trajectories of f t are the following types of trajec-
tories:
1) equilibrium states;
2) separatrices of equilibrium states;
3) limit cycles;2
4) closed trajectories l such that for any neighborhood U of l there are both
closed trajectories and nonclosed trajectories in U \ {l} (a foliated pie);
5) closed trajectories l such that for any neighborhood U J l there exists a
neighborhood V C U of l homeomorphic to a Mobius band and filled by closed
trajectories.
The family of singular trajectories of a flow f t will be denoted by E(f t) .
REMARK. Mater [55] defined singular trajectories of flows having nontrivial
recurrent trajectories on orientable surfaces. For such flows a singular trajectory
2A limit cycle is defined to be an isolated closed trajectory of a flow, that is, a trajectory
with a neighborhood in which there are no other closed trajectories.
126 3. TOPOLOGICAL STRUCTURE OF A FLOW

is defined to be either one of the trajectories 1)-4) above, or a nontrivial recurrent


trajectory l satisfying the following condition: for any point m E l there exists a
neighborhood U of m such that the component of l f1 U (an arc of l) containing
m partitions U into two half-neighborhoods, one of which does not contain points
belonging to nontrivial recurrent trajectories. We remark that orientable surfaces
do not have singular trajectories of type 5).
The singular trajectories on nonorientable surfaces were determined by Aranson

The next lemma shows that the nonsingular trajectories (that is, trajectories
that are not singular) form an open set, and the limit behavior of nearby nonsingular
trajectories is the same in a certain sense.
LEMMA 4.1. Let f t be a flow with isolated equilibrium states on an oriented
closed surface M, and let 1(m), m E M, be a nonsingular trajectory of f. Then
there exists a neighborhood U of m with the following properties:
1) all the trajectories l (m), rim E U, are nonsingular;,
2) if 1(m) is closed, then all the l (m), rim e U, are closed, and together with
1(m) they bound an annular domain;
3) if 1(m) is nonclosed, then all the trajectories l (m), m E U, are nonclosed,
and
w(l+(m)) = w(l+(m)), (m)) = (m))
PROOF. If 1(m) is a closed trajectory, then in view of its nonsingularity it
cannot be a limit for nonclosed semitrajectories and limit cycles. Therefore, all the
trajectories in some neighborhood of 1(m) are closed (and hence nonsingular), and
together with 1(m) they bound an annular domain because M is orientable.
Suppose that 1(m) is nonclosed. Then w (l+ (m)) is a single equilibrium state
m0, a single closed trajectory lo, or a single one-sided contour (recall that in this
section we are considering flows without nontrivial recurrent trajectories). Let us
analyze all these cases.
Suppose that w (l+ (m)) = m0 is an equilibrium state, and that the neigh-
borhood U(mo) of mo does not contain equilibrium states other than m0. We
assume that for any neighborhood U(m) of m there is a point m E u(m) with
w(l+(m)) mo. Then the semitrajectory l+(m) leaves U(mo) as time increases,
and hence 1+ (m) can be extended with respect to U(mo). According to Lemma 3.2
in Chapter 2, 1+ (m) is an w-separatrix, which is a contradiction. Therefore, there
exists a neighborhood U of m such that w (l+ (m)) = w (l+ (m)) = m0 for all points
m E U. This implies that the trajectories l (m), m E U, are nonclosed. Since l+ (m)
is not extendible with respect to U(mo), the semitrajectories l+(m), rim e U, are
not extendible with respect to U(mo) for a sufficiently small neighborhood U.
Let w (l+ (m)) = 1 0 be a closed trajectory. By Corollary 1.3 in Chapter 2, there
is a neighborhood U of m such that w (l+ (m)) = to for all m E U. It can be assumed
without loss of generality that to f1U = 0. Then all the trajectories l (rn), m e U,
are nonclosed. The case when w(l+ (m)) is a one-sided contour is handled in a
completely analogous way.
The preceding arguments are repeated for the a-limit set of the semitrajectory
1(m).
If w (l (m)) (c (l (m))) is an equilibrium state, then the trajectories l (m), m E
U, are nonsingular because the semitrajectories l± (rn), m e U, are not extendable.
4. FLOWS WITHOUT NONTRIVIAL RECURRENT TRAJECTORIES 127

If w (l (m)) (c (l (m))) is a closed trajectory or a one-sided contour, then the


trajectories l (m), m E U are nonsingular because they are nonclosed. 0

COROLLARY 4.1. The union of the singular trajectories of a flow f t with iso-
lated equilibrium states on an orientable closed surface forms a closed invariant set.
The w- (a-) limit set of any semitrajectory of f t consists of singular trajectories.
The proof is left to the reader as an exercise.
4.2. Cells. Denote by E(f t) the union of the singular trajectories of f t.
DEFINITION. For a flow f t on a manifold M, a component of PVC \ E (f t) is called
a cell of ft .

LEMMA 4.2. Suppose that f t is a flow with a finite family E(f t) of singular
trajectories on an orientable closed surface M. Then f t has finitely many cells.
PROOF. Let l be a singular trajectory of ft that is not an equilibrium state.
We take a point m e l and show that only finitely many cells intersect some
neighborhood U(m) of m.
Assume first that l is not a limit for any singular trajectory of ft. Since the
boundary of any cell consists of singular trajectories, it follows from the theorem
on continuous dependence of trajectories on the initial conditions that l lies on the
boundary of at most two cells.
Suppose now that l can be a limit for singular trajectories. Since f t does
not have nontrivial recurrent trajectories, all the singular trajectories tending to
l cannot be limit trajectories for other trajectories (Theorem 2.2 in Chapter II),
and hence such singular trajectories lie on the boundary of at most two cells. The
finiteness of the family E(f t) gives us that only finitely many cells intersect some
neighborhood U(m). It follows from the theorem on continuous dependence of
trajectories on the initial conditions and the finiteness of E(f t) that l belongs to
the boundary of finitely many cells.
The boundary of each cell contains at least one regular singular trajectory. The
finiteness of the number of cells of f t follows from this, the compactness of M, and
the finiteness of the family E(f t) . D

4.3. Topology of cells.


LEMMA 4.3. On an orientable closed surface M let f t be a flow with isolated
equilibrium states and let R be a cell of ft. Then the trajectories in R are either
all closed or all nonclosed.
PROOF. This follows from Lemma 4.1 and the connectedness of R. 0

We now investigate the topological structure of cells filled by closed or by


nonclosed trajectories.
THEOREM 4.1. Suppose that f t is a flow on an orientable closed surface M,
and let R be a cell of the flow that is filled by closed trajectories. Then:
1) R is homeomorphic either to an open annulus or to the torus T 2 (in the
latter case f t is a rational winding on the torus);
128 3. TOPOLOGICAL STRUCTURE OF A FLOW

R. jx (o.i) Rc T
a) b)

FIGURE 3.2$

2) any two closed trajectories in R bound a domain homeomorphic to an open


annulus and lying in R (Figure 3.28).

PROOF. Let l1 and l2 be closed trajectories in R. Assume that l1 and l2 do


not bound an annular domain, and take an arc d12 With endpoints d1 E ll and
d2 E l2 that lies completely in R. We partition the interior points of d12 into two
classes D1 and D2 by setting Di = {the points in d12 through which there pass
trajectories bounding an annular domain together with l2 }, i = 1, 2. According to
Lemma 4.1, Di 0 (i = 1, 2), and D1 f1 D2 = 0 by our assumption. It follows from
the connectedness of the arc d12 that there is a point m E d12 that does not lie in
the union D1 U D2 but does have points of D1 U D2 in any of its neighborhoods.
This contradicts Lemma 4.1, 2). Thus, any two closed trajectories in R bound an
annular domain in R.
Two cases are possible: 1) DR = 0; 2) DR 0. Since M is connected, we get
M = R in case 1), and hence all the trajectories of f t are closed. It follows from
Theorem 4.3 in Chapter 1 that the Euler characteristic of M is equal to zero, and
thus M=R=T2.
Since M is orientable, in case 2) R contains two sequences {m}° and {m}0
of points tending to the boundary DR and such that the open annular domains Ki
bounded by the closed trajectories 1(m1) and l (m) form an expanding sequence of
sets K1 C ... C Ki C .... Since mi, mi -* aR (i -* oo), it follows that R = UKi.
This implies that R is homeomorphic to an open annulus. 0

THEOREM 4.2. Let f t be a flow with finitely many singular trajectories on an


orientable closed surface M, and let R be a cell of the flow that is filled by nonclosed
trajectories. Then:
1) R is homeomorphic either to an open disk or to an open annulus (Figure
3.29);
2) aR has at most two connected components;
3) all the trajectories in R have the same w- and a-limit sets;
4) the w- (a-) limit set of any trajectory in R belongs to DR, and, moreover,
each component of DR contains points in the w- or a-limit sets of trajectories in R.

PROOF. The connectedness of R and Lemma 4.1, 3) imply 3).


According to Corollary 3.1, the w- (a-) limit set of any nonclosed trajectory
1 C R consists of singular trajectories. Consequently, w(l) U a(l) C DR.
4. FLOWS WITHOUT NONTRIVIAL RECURRENT TRAJECTORIES 129

FIGURE 3.29

We show that each component R* of the boundary DR contains limit points


of trajectories in R. Note that since the set of singular trajectories is invariant
(Corollary 4.1), R* is a connected invariant set. We assume that R* does not
contain limit points of any trajectory in R.
Suppose that R* consists solely of equilibrium states. Since R* is connected,
and each equilibrium state is isolated, R* is a single equilibrium state m0. Any
neighborhood of m0 intersects nonclosed trajectories in R. This and the connect-
edness of R give us that there is no closed trajectory of f t lying arbitrarily close to
m0 and enclosing m0 inside itself. According to the Bendixson theorem on equilib-
rium states (Theorem 3.3 in Chapter 2), there are trajectories of ft tending to m0.
By the fact that m0 is an isolated equilibrium state and by our assumption that
m0 is not a limit point of any trajectory in R, there exist a neighborhood U(m0)
not containing equilibrium states other than m0 and a hyperbolic (saddle) sector
S in U(m0) that is bounded by separatrices it and l2 and intersects the cell R
(Figure 3.30). Since S contains points in R arbitrarily close to m0, the separatrices
l1 and l2 belong to DR. This contradicts the fact that R* consists only of the
single equilibrium state m0.

FIGURE 3.30

Suppose that the component R* contains a regular point m. Since f t does


not have nontrivial recurrent trajectories, the singular trajectory l (m*) is either
a closed trajectory or a separatrix of some equilibrium state. If l (m*) is a closed
trajectory, then some neighborhood of l (m*) does not contain closed trajectories
other than l (m*) because there are finitely many singular trajectories. It follows
that the trajectory l (m*) C R* belongs to the limit set of the trajectories in the
cell R.
We consider the case when l (m*) is a separatrix. If l (m*) belongs to a one-
sided contour K, then it can be shown as above that R* contains limit points for
the trajectories in R. If l (m*) does not belong to a one-sided contour, then finitely
130 3. TOPOLOGICAL STRUCTURE OF A FLOW

many Bendixson extensions of it from the side from which the trajectories in R
approach m* lead to a separatrix l * of an equilibrium state O* , with l * either not
Bendixson extendible (beyond O*) or tending to a closed trajectory 10. In this case
either the equilibrium state O* is in R*, or the closed trajectory to C R* belongs
to the limit set of the trajectories in R.
Thus, each component R* of DR contains limit points of (all) the trajectories in
R. From this, the connectedness of the w- (a-) limit set of any trajectory (Lemma
1.5 in Chapter 2) and the fact that all the trajectories in R have the same w- and
a-limit sets, it follows that DR consists of at most two components.
We proceed to the proof of 1). Two cases are possible:
a) any contact-free segment intersects each trajectory in R at most at one point;
b) There exist a contact-free segment and a trajectory in R that intersect at
more than one point.
Let us consider the case a). Since all the trajectories in R are nonclosed, the w-
(ce-) limit set of each trajectory in R consists of a single equilibrium state in this case.
Therefore, any two sufficiently close trajectories in R bound a domain in R that is
homeomorphic to the open strip (0, 1) x R1, and hence homeomorphic to an open
disk (note that the indicated transformation (0, 1) x II81 -* R is a homeomorphism
of the strip (0, 1) x R1 onto its image in the topology induced by this mapping; in
the topology of R induced by the topology of M the transformation (0, 1) x II81 -* R
is not a homeomorphism in general). Since R is connected, any two trajectories
in R bound a domain in R homeomorphic to an open disk. In this case it can be
shown as in the proof of Theorem 4.1 that R is homeomorphic to an open disk.
We consider the case b). By Lemma 1.2 in Chapter 2, there exists a contact-free
cycle C intersecting trajectories in the cell R. Two subcases are possible:
b 1) C lies entirely in R;
b2) C intersects R in an arc J.
In the subcases bl) we denote by R1 (respectively, R2) the set of trajectories of R
intersecting (respectively, not intersecting) the cycle C. By the compactness of C
and the theorem on continuous dependence of trajectories on the initial conditions,
both the sets and R2 are open. By construction, R1 0. If R2 = 0, then
R1.

we get a contradiction to the connectedness of R. Therefore, R2 = 0, that is, any


trajectory in R intersects C.
We show that in the subcase bl) each trajectory in R intersects C exactly once.
Suppose not. Then a Poincare mapping P : C -* C is defined on C with nonempty
domain Dom(P). If Dom(P) C, then by Lemma 3.7 in Chapter 2, a separatrix of
some equilibrium state passes through an endpoint of a component of Dom(P), and
this contradicts the fact that R does not contain singular trajectories. Therefore,
Dom(P) = C, and the restriction of the flow f t to R is topologically equivalent to a
suspension over the circle. Then R = T2, and f t is a flow on T2 without equilibrium
states and closed trajectories. According to Theorem 3.6 in Chapter 2, such a flow
has nontrivial recurrent trajectories, which is impossible. The contradiction shows
that each trajectory in R intersects C at exactly one point. This implies that the
cell R is homeomorphic to an open annulus.
In the subcase b2) it again follows from the connectedness of R that the arc
intersects any trajectory in R. If J intersects each trajectory at exactly one point,
then R is homeomorphic to the open strip (0, 1) x R1, and hence to an open disk.
If J intersects some trajectory in R at more than point, then by Lemma 1.2 in
Chapter 2, there exists a contact-free cycle in R; that is, the subcase b2) reduces
4. FLOWS WITHOUT NONTRIVIAL RECURRENT TRAJECTORIES 131

to the subcase bl) treated above, and the cell R is homeomorphic to an open disk.
D

4.4. Structure of a flow in cells. Let R be a cell of the flow f t Since .

the set R is invariant, the restriction f t R of f t to R can be regarded as a flow


on an open submanifold. There are only four types of flows f t R, that is, f t R is
topologically equivalent to one of four standard flows ([1], [100]). This subsection
is devoted to a description of these standard flows. The main result is one of the
key results in solving the problem of topological equivalence of flows not having
nontrivial recurrent trajectories (Chapter 6).
Let R2 be the Euclidean plane with Cartesian coordinates x and y. Denote by
H the open strip bounded by the lines y = 0 and y = 1.
DEFINITION. The flow fn on H given by the system x = 1, y = 0 is called
a parallel flow on an open strip (Figure 3.31). Any flow on an open manifold or
submanifold that is topologically equivalent to f n will also be called a parallel flow
on an open strip.

ify

FIGURE 3.31

We introduce a polar coordinate system (r, Sp) on R2, and denote by O the pole
(r = 0). Obviously, the set K = R2 \ O is homeomorphic to the open annulus
(0,1) x S'.
DEFINITION. The flow fK1 on K given by the system r = 0, b = 1 is called a
parallel flow on an open annulus (Figure 3.32, a)).

a) b)

FIGURE 3.32

The flow f K2 on K given by the system r = r, b = 0 is called a spiral flow on


an open annulus (Figure 3.32, b)).
132 3. TOPOLOGICAL STRUCTURE OF A FLOW

Any flow on an open manifold or submanifold that is topologically equivalent


to the flow fK1 (respectively, fK2) will also be called a parallel (respectively, spiral)
flow on an open annulus.
We recall that a rational winding on the torus T2 is defined to be a flow on
T2 for which a covering flow on R2 with respect to the universal covering R2 -*
R2/Z2 T2 has the form x = 1, y = p/q, where p/q is a rational number (see
§2.3.2 in Chapter 1).
The next result follows directly from the proofs of Theorems 4.1 and 4.2.
THEOREM 4.3. Let f t be a flow with finitely many singular trajectories on an
orientable closed surface M, and let R be a cell of ft. Then the restriction f t I R
of the flow to R is a parallel flow on an open strip, or a spiral flow on an open
annulus, or a parallel flow on an open annulus, or a rational winding on the torus
(in the last case R = M = T2).
4.5. Smooth models. We recall that the accessible (from within) boundary
SR of a set R is defined to be the subset of points x in DR such that there is an arc
with x as one endpoint and all other points interior to R.
For example, let R be the cell pictured in Figure 3.33. The trajectories 1 1 and
l2 (separatrices) belong to the boundary DR of R, as does the limit cycle to . But
to does not belong to the accessible (from within) boundary SR. The trajectories
1 1 and l2 do belong to OR.

FIGURE 3.33

We consider a flow f t not having nontrivial recurrent trajectories on a closed


orientable surface M. Let R be a cell of ft. It follows from the theorem on contin-
uous dependence of trajectories on the initial conditions that OR consists of whole
trajectories. We remove the equilibrium states from OR (if there are any) and de-
note the resulting set by OR. Then the set R U SR is invariant and consists solely
of regular trajectories of f t.
This subsection is devoted to the study of the topological structure of the flow
f t I RusR (that is, the restriction of ft to the invariant set RU OR). For cells R with
nonempty boundary it is shown that there are four standard constructions enabling
us to construct, from a given flow f t and a given cell R, a C°°-flow topologically
equivalent to f t I RubR In other words, any flow f t I RuaR is topologically equivalent
(and even topologically orbitally equivalent) to a certain C°°-flow, which is called
a smooth model for the flow f t I RusR This result is given in Chapter 7 in the proof
of the fact that any continuous flow with finitely many singular trajectories on a
4. FLOWS WITHOUT NONTRIVIAL RECURRENT TRAJECTORIES 133

compact orientable surface is topologically equivalent to some C°°-flow, that is,


is smoothable (Neumann's theorem). The collection of all smooth models breaks
up into four classes corresponding to the four constructions that reduce to these
models. We proceed to a description of the constructions and the corresponding
families of smooth models. Our presentation follows [100].
1) A flow f o on a strip. As in the preceding subsection, we use Cartesian and
polar coordinate systems in the Euclidean plane R2.
On the line y = i (i = 0, 1) we take a set PZ that is empty, or coincides with
the whole line y = i, or is a family of isolated points. There exists a C°°-function
II -* [0, 1] equal to 0 on Po U P1 and strictly positive on the set II def II \ P0 U P1
(where II is the closed strip on R2 bounded by the lines y = 0 and y = 1) [47].
Then the flow fo on II given by the system x = Sp(x, y), y = 0,(x, y) E II, is smooth
of class C°° (Figure 3.34). The restriction of f o to II = int II is a parallel flow on
an open strip.

FIGURE 3.34

The restriction of f o to ti is denoted by f o Then f o has only nonclosed regular


.

trajectories and is a C°°-flow.


2) A parallel flow f i on an annulus. Let K = {(r, Sp) 1 < r < 2} be the
:

closed annulus bounded by the circles J1 = {r = 1} and J2 = {r = 2} (where


(r, Sp) are polar coordinates on R2). On the circle J (i = 1, 2) we take a set P2+1
that coincides with J , or is empty, or is a family of isolated points. There exists
a C°°-function : K -* [0, 1] equal to 0 on P2 U P3 and taking positive values on
the set K def K \ P2 U P3 [47]. Then the flow f 1 on K given by the system r = 0,
SP = /'(r, gyp), (r, gyp) E K, is smooth of class C°° (Figure 3.35). The restriction of f 1
to int K is a parallel flow on an open annulus.

P3

FIGURE 3.35
134 3. TOPOLOGICAL STRUCTURE OF A FLOW

Let f i denote the restriction of f I to K. By construction, f 1 is a C°°-flow.


3) Spiral flows f 2 and f 3 on an annulus. We use the notation of case 2).
On the closed annulus K there are only two topological types of flows without
equilibrium states and without closed trajectories other than 2 C aK (by the
definition of a flow, the boundary aK of the annulus K is an invariant set of any
flow on K; therefore, if a flow on _K does not have equilibrium states, then both the
boundary components J1, 2 C aK are closed trajectories). Any such flow on K is
topologically equivalent to one of the flows in Figure 3.36. One topological class
contains flows for which the motions along the trajectories J1 and J2 with increasing
time are realized either both clockwise or both counterclockwise (Figure 3.36, a)).
The second topological class consists of flows for which the motions along J1 and
J2 with increasing time are in opposite directions (Figure 3.36, b)). A flow in the
first (second) topological class will be indicated by a "plus" ("minus") sign.

FIGURE 3.36

a) A spiral flow J2 of "plus" type. Let the sets P2 and P3 and the function
/(r, Sp) be the same as in case 2), and let the C°°-function v : K -* [0, 1] be given
by
e r-i r-2 , 1<r < 2
v(r, cp) _
0, r=1, r =2.
Then the flow f 2 on K given by the system r = v (r, p), cp = b (r, p), (r, Sp) E K,
is smooth of class C°°. The restriction of f 2 to int K is a spiral flow on an open
annulus. _
The restriction of f 2 to K = K \ P2 U P3 is denoted by f 2 . By construction, f 2
is a C°°-flow, and is called a spiral flow of "plus" type on an annulus.
b) A spiral flow f 3 of "minus" type. We use the preceding notation, and we
take a C°°-function 2b1 : K -* [0, 1] that is equal to 0 on P2 U P3 and on the closed
neighborhood 1.5 - < r < 1.5 + s (0 < s < 0.125) of the circle r = 1.5. At
the remaining points of K the function b1 takes strictly positive values. Then the
function
-i/.i1(r, Sp), 1 < r < 1.5
2(r, = 1(r, gyp) , 1.5 <r<_ 2
_
is smooth of class C°°, and the flow f 3 on K given by the system r = v (r, p),
_ /)2(r, p), (r, p) E K, is a C°°-flow. The restriction of f t to int K is also a spiral
flow on an open annulus.
4. FLOWS WITHOUT NONTRIVIAL RECURRENT TRAJECTORIES 135

Let J3 be the restriction of f3 to K. By construction, f3 is a C°°-flow, and is


called a spiral flow of "minus" type on an annulus.
The flows f o , f l , f, and f 3 constructed according to the above four construc-
tions are called model flows or smooth models.
LEMMA 4.4. Suppose that f t is a flow with finitely many singular trajectories
on an orientable closed surface M, and let R be a cell of f t with nonempty boundary
DR. Then the restriction f t I RUbR of ft to the invariant set R U SR (where SR is
the accessible (from within) boundary of R) is topologically equivalent (and even
topologically orbitally equivalent) to one of the model C°°- flows f, i = 0, 1, 2, 3.
PROOF. Since DR 0, Theorem 4.3 gives us that the restriction f t I R of ft
to R is a parallel flow on an open strip, or a spiral flow on an open annulus, or a
parallel flow on an open annulus. Let us consider the case when f t I R is a parallel
flow on an open annulus. We let Ri (i = 0, 1) be the line y = i, and we use the
notation in §4.5, 1).
Let fo be the flow given by the system x = 1, y = 0 on the open strip H bounded
by the lines Ro and R1 (Figure 3.31). We take a trajectory L of fo that coincides
as a set with the line y = yo, 0< yo < 1. Define H+ = {(x,y) E H : yo < y < 1}
andH ={(x,y) EH:0<y<yo}. _
Let L be a trajectory in the cell R. By Theorem 4.3, L separates R into two
disjoint invariant subsets Ro and R, where Ro (respectively, _Ro) denotes the
subset locally lying to the left (right) of L upon moving along L in the positive
direction (Figure 3.37). We set R+ = Ro U L and R_ - = Ro U L. Denote by OR+
the accessible (from within) boundary of R+, with L removed, and let 5R+ be the
family of regular trajectories in SR+. We introduce the analogous concepts and
notation 0R- and 0R- for the set R.

FIGURE 3.37

If SR = SR+USR- = 0, then the flow f t I RUSK = f t I R is topologically equivalent


to the parallel flow fo on the open strip H. The flow fo is the model C°°-flow fo
with Po = Ro and P1 = R1.
Suppose that O1 0; say SR+ 0. By the structure of the flow f t I R (Theorem
4.3), there exists a contact-free segment with endpoints on O1 and L that lies
in R+ U OR+ and is such that each trajectory of the invariant set Ro intersects
at exactly one point (Figure 3.38, a)).
The trajectories in the cell R induce a positive direction J on OR+ (Figure
3.38, a)) . Let ... ,l_1, lo, l1, ... be regular trajectories on OR+ (that is, trajectories
in SR+), written in the order in which they occur on OR+ upon moving in the
positive direction J. Here we require that one of the endpoints of belong to lo.
In the notation of §4.5, 1) we take a set P1 C R1 such that the number of regular
136 3. TOPOLOGICAL STRUCTURE OF A FLOW

b)

FIGURE 3.38

trajectories of the flow f o , constructed as in §4.5, 1), on the line R1 is equal to


the number of regular trajectories of the flow f t on SR+, that is, to the number of
trajectories of the family ... , 1_1, lo, l1, .... Let ... , L_1, Lo, L1, ... be the regular
trajectories of the flow f o on R1, written out in the order in which they occur upon
moving along R1 in the positive direction of the axis Ox. Then there exists a one-
to-one correspondence T : {. , 1_i, lo, l 1, ... } -* {. . , L_1, L0, L1,. . } preserving
. . . .

the order of the trajectories. It can be assumed without loss of generality that
T(lo) = Lo, and that Lo intersects the axis Oy.
The segment o = {(x, y) E x = 0, yo < y < 1} is a contact-free seg-
ment of the flow f o . Let h : - * o be an arbitrary homeomorphism satisfying
the conditions h(lo f1 ) = Lo f1 o and h(L f1 ) = L f1 o. Since each trajec-
tory of the invariant set R+ of fo intersects o at exactly one point (Figure 3.38,
b)), the homeomorphism h can be extended with the help of r to a homeomor-
phism H+: R+ U SR+ - II+ U SII+ realizing a topological equivalence (and even a
topological orbital equivalence) of the flows f t I R+ uoR+ and fIu Similarly,
we construct a homeomorphism H: R- U SR- -* II- U S1 L realizing a topo-
logical orbital equivalence of the flows f t I R_ u6R- and f o I n_ and coinciding
with H+ on L. The homeomorphisms H+ and H- thus form a homeomorphism
H : RU SR -* H U 6H realizing a topological orbital equivalence of the flows f t I Ru5R
and f.
The cases when f t I R is topologically equivalent to parallel or spiral flows on an
open annulus can be handled according to the scheme presented above. We leave
it as an exercise for the reader to fill in the details of the proof. 0

We remark that SR cannot be replaced by SR in the assertion of Lemma 4.4


because the number of equilibrium states has nothing to do with the number of
regular trajectories of an arbitrary flow f t on SR in general, in contrast to model
flows (see Figure 3.38, a), b)).
4.6. Morse-Smale flows. The Morse-Smale flows occupy a special position
among flows without nontrivial recurrent trajectories. For example, in the class of
flows on closed orientable surfaces the Morse-Smale flows form a dense open set.
Before giving the precise definition of a Morse-Smale flow we introduce the concept
of a hyperbolic equilibrium state and the concept of a hyperbolic closed trajectory.
Let f t be a C''-flow (r > 1) on a surface M, and let mo be an equilibrium state
of ft. Recall that ft denotes the shift of the points of M along trajectories by the
time t. Then ft is a C''-diffeomorphism of M onto itself, and mo is a fixed point of
ft for any tElit
4. FLOWS WITHOUT NONTRIVIAL RECURRENT TRAJECTORIES 137

Denote by TmoM the tangent space of M at the point mo, a real two-dimensional
vector space. The derivative D f t (mo) of the diffeomorphism ft at mo is a linear
transformation Tmo NYC - * Tmo NYC, which is given by the Jacobi function matrix (of
partial derivatives) in a chart containing mo [76].
DEFINITION. An equilibrium state mo of a C''-flow f t (r > 1) is said to be
hyperbolic if the eigenvalues of the linear transformation D f 1(mo) are not equal to
1 in modulus.
We give an equivalent definition of a hyperbolic equilibrium state. Suppose
that in a chart U containing mo and in a coordinate system (x, y) : U -* T2 the
flow f t is given by the system
x = P (x, y) , y = Q (x , y)
and let (XO, yo) be the coordinates of mo in U. Then the functions P (x, y) and
Q (x, y) are smooth of class C'' (r > 1), and P (xo, yo) = Q (xo , yo) = 0.
An equilibrium state is hyperbolic if and only if
Px (xo, yo) Py (xo , yo) 0
O=
Qx(xo, yo) Qy(xo, yo)
and o = Px (xo, yo) + Qy (xo, yo) 0 for 0 > 0; that is, the roots of the equation
A2 -tea+0 = 0
are not purely imaginary.
Both definitions of hyperbolicity are independent of the coordinate system in
a neighborhood of m0.
If an equilibrium state is hyperbolic, then it is isolated. What is more, a
hyperbolic equilibrium state is locally topologically equivalent either to a node
(stable or unstable), or to a saddle [3] (Figure 3.39).

FIGURE 3.39

Let to be a closed trajectory of a C1'-flow f t of period r > 0, and let m E to be


an arbitrary point.
DEFINITION. A closed trajectory to of period r > 0 for a C''-flow f t (r > 1) is
said to be hyperbolic if exactly one eigenvalue of the linear transformation D f,. (m)
is equal to 1, and the modulus of the second is different from 1 (consequently, the
second eigenvalue is a real number with modulus different from 1).
By the group property f t1 +t2 = ftl ° ft2, the definition of hyperbolicity of a
closed trajectory to does not depend on the choice of the point m E lo.
We give an equivalent definition of hyperbolicity of a closed trajectory. Let
be a contact-free segment containing a point m, and let P : - * be the Poincare
138 3. TOPOLOGICAL STRUCTURE OF A FLOW

mapping for the trajectory lo. According to Lemma 1.4 in Chapter 2, P is a Cr-
diffeomorphism (r > 1) in its domain. Clearly, m E Dom(P).
A closed trajectory to is hyperbolic if and only if P' (m) 11.
If a closed trajectory to is hyperbolic, then it is isolated in the set of closed
trajectories of ft; that is, to is a limit cycle. For IP'(m) < 1 the limit cycle to is
stable (Figure 3.40, a) ), while for IP'(m)I > 1 it is unstable (Figure 3.40, b)).

FIGURE 3.40

DEFINITION. A C'-flow f t (r > 1) on a surface M is called a Morse-Smale


low if the following conditions hold:
1) f t has finitely many equilibrium states and finitely many closed trajectories,
and they are all hyperbolic;
2) the limit set of any semitrajectory of f t consists either of a single equilibrium
state or of a single closed trajectory;
3) there are no separatrices going from a saddle to a saddle (in particular, no
separatrix loops).
It follows from the condition 2) that in Morse-Smale flows there are no non-
trivial recurrent trajectories.
The simplest example of a Morse-Smale flow on the sphere is the flow in §2.1
of Chapter 1, of "North-South Pole" type.
In Figure 3.41 we picture the simplest Morse-Smale flow with four equilibrium
states on the torus (one stable node, one unstable node, and two saddles).

FIGURE 3.41

As shown in §2.3 of Chapter 1, a pretzel (a closed orientable surface of genus


two) can be obtained from an octagon after an appropriate identification of its sides
(Figure 3.42, a)). Using this, we picture in Figure 3.42, b) a Morse-Smale flow on
5. THE SPACE OF FLOWS 139

W
i CD 7T7\i

a) b)

FIGURE 3.42

a pretzel. The flow has one stable node w, one unstable node (at the center of the
octagon), and four saddles , , 03, and 04.
DEFINITION. A Morse-Smale flow is said to be polar if it does not have closed
trajectories, and the family of equilibrium states contains exactly one stable node
and exactly one unstable node (the rest of the equilibrium states, if there are any,
are saddles).
All the above examples of Morse-Smale flows are polar. There are other exam-
ples of Morse-Smale flows in the books [3], [64], and [74].
4.7. Cells of Morse-Smale flows. The next result is an immediate conse-
quence of the definition of a Morse-Smale flow and of Theorems 4.1-4.3.
LEMMA 4.5. Let f t be a Morse-Smale flow on an orientable closed surface M,
and let R be a cell of ft. Then:
1) R consists of nonclosed trajectories;
2) as t -> oo all the trajectories in R tend either to a single equilibrium state
(a stable node or focus) or to a single limit cycle, and the same is also true as
t - -oo;
3) if R is homeomorphic to an open disk, then its accessible (from within)
boundary contains a saddle and at least two separatrices.

§5. The space of flows


In this section flows on a fixed surface M are regarded as points of a set. In
this set of flows we introduce a metric, which in general depends on the differen-
tiable structure (and hence the metric) on M, and which turns the set of flows
into a metric topological space. This enables us to investigate various classes of
flows (in particular, Morse-Smale flows) according to their situation in the space of
flows. Using the metric introduced, we define important concepts such as structural
stability, degree of structural instability, and others.
5.1. The metric in the space of flows. Suppose that a compact surface
M has the structure of a two-dimensional differentiable manifold. This means that
there is a covering E (M) of M by domains (charts) homeomorphic to an open disk
such that each domain U E E (M) has a coordinate system (x, y) : U -* T2, and the
transition from one set of coordinates to another in overlapping domains is realized
by means of analytic functions with nonzero Jacobian.
140 3. TOPOLOGICAL STRUCTURE OF A FLOW

Let f t and gt be C'0-flows on M (r > 0). Then in each chart U E E (M)


with coordinate system (x, y) : U -* T2 the flows f t and gt are given by respective
systems of differential equations

x = P(x, y), y = Q(x, y)


x = P(x, y), y = Q(x, y)
where P(x, y), Q(x, y), P(x, y), and Q(x, y) are smooth functions of class Cr.
For an integer 0 < k < r we define the distance pk (f t , gt) between the flows f t
and gt to be the maximum of the following quantities over all domains U E E (NYC)

mtaxlP-PI,
sp _ as P 3Q _ 38Q
mtLax max
axzayi axzay3 'U 8xiay3 8xiayi
where i +j = s <k.
The distance introduced determines a metric pk on the set of C'-flows, 0 < k <
r.

Let X (M) be the metric space of C''-flows on M, equipped with the metric pk .

def (m) .
For k = r we let 3C' (NYC) XT

5.2. The concepts of structural stability and the degree of struc-


tural instability. On a compact surface M we fix a metric p compatible with the
differentiable structure on M.
DEFINITION. A homeomorphism Sp : M -* M is called an e-homeomorphism
(e> 0) if p(m, (p(m)) <e for any point m E M.
Let f t and gt be C''-flows (r > 1) on M, and let 1 < k < r.
DEFINITION. The two flows f t and gt are said to be S-close (S > 0) in the
space X( M) if pk (f t, gt) <5
DEFINITION. A C'-flow f t is said to be structurally stable in the space X (M)
if for any e> 0 there is a 5> 0 such that each flow gt that is S-close to f t in X (M)
is topologically equivalent to f t by an e-homeomorphism.
Thus, structural stability of a flow means that any small perturbation of this
flow results in a flow topologically equivalent to the original flow, with the home-
omorphism implementing the topological equivalence close to the identity in the
C°-topology.
The concept of structural stability was first introduced in 1937 by Andronov and
Pontryagin [5]. They considered dynamical systems in a planar domain bounded
by a contact-free cycle, and they defined S-closeness in the space X, r > 1. Anosov
therefore proposed using the name Andronov-Pontryagin structural stability for
the above definition of structural stability [6].
If in the definition of structural stability it is not required that the homeomor-
phism implementing a topological equivalence of f t and gt be an e-homeomorphism,
then we get the definition of weak structural stability of a flow f t (or structural sta-
bility in the Peixoto sense [6]), which was introduced in [103].
We proceed to the concept of the degree of structural instability.
5. THE SPACE OF FLOWS 141

DEFINITION. A C'-flow f t (r > 1) on a compact surface M is called a flow of


the first degree of structural instability in the space X (M), 1 < k < r, if it is not
structurally stable and if for any e > 0 there is a 6> 0 such that each structurally
unstable flow gt that is S-close to f t in X( M) is topologically equivalent to f t by
an e-homeomorphism.
Thus, flows of the first degree of structural instability are flows that are struc-
turally unstable but are "structurally stable in the set of structurally unstable
flows" (that is, are relatively structurally stable). These flows play a large role
in the theory of bifurcation of flows on surfaces, since the simplest bifurcations of
structurally stable flows pass through them.
Similarly, flows of the jth degree of structural instability (j > 2) are defined
to be flows that are not structurally stable and are not flows of the 1st, ... , j - 1st
degrees of structural instability, but are relatively structurally stable in the set of
flows remaining after removal from Xr (M) of the structurally stable flows and the
flows of the 1st, ... , j - 1st degrees of structural instability.
The degrees of structural instability establish a hierarchy in the space of flows
according to the degree of sensitivity to perturbations.
5.3. The space of structurally stable flows. This subsection has the
nature of a survey.
THEOREM 5.1. A C'-flow f t (r > 1) on a closed orientable surface M is
structurally stable in the space X (M) for any 1 < k < r if and only if f t is a
Morse-Smale flow.
THEOREM 5.2. On a closed orientable surface M the set of Morse-Smale C7'-
flows (r > 1) is open and dense in the space X (M) for any 1 < k < r.
For the sphere S2 = M Theorems 5.1 and 5.2 follow from [5]. They are proved
in [103] for the general case. There are proofs of these theorems in the books [64]
and [74].
The structural stability of the flow in Theorems 5.1 and 5.2 can be understood
both in the Andronov-Pontryagin sense and in the Peixoto sense.
As follows from [8], [64], [65], [85], and [103], Theorems 5.1 and 5.2 are valid
for closed nonorientable surfaces of genus p = 1 (the projective plane), p = 2 (the
Klein bottle), and p = 3 (the torus with a Mobius cap attached). Further, by the
C1-closing lemma [69], both the theorems hold in the space X' (M), both for ori-
entable and for nonorientable closed surfaces M. As for the remaining possibilities
of generalizing Theorems 5.1 and 5.2 for two-dimensional surfaces, there are several
open questions here. It is not known (at present) whether a structurally stable flow
f t E X( M), 1 <k < r, necessarily belongs to the class of Morse-Smale flows for
nonorientable closed surfaces of genus p > 4 (this assertion is valid in the other
direction). The question of whether the set of structurally stable C''-flows (r > 1)
is dense in XT (M) (1 <k < r) also remains open for a nonorientable closed surface
of genus p > 4. The main difficulty in the investigation of these questions is the
existence of nonorientable nontrivial recurrent trajectories. We describe this result
in greater detail.
Let f t be a flow with a nontrivial recurrent trajectory l on a closed nonorientable
surface M. This trajectory is called a nonorientable nontrivial recurrent trajectory
if for any point m E l and any contact-free segment passing through m the
semitrajectories l+(m) and 1 - (m) both have the following property: there exists
142 3. TOPOLOGICAL STRUCTURE OF A FLOW

a E-arc ab of the semitrajectory l ) (m) (a, b E l O (m) l ) such that the simple
closed curve ab U ab (where ab C E) is a one-sided curve (that is, has a neighborhood
homeomorphic to an open Mobius band (Figure 3.43).

FIGURE 3.43

In the paper, Smooth nonorientable nontrivial recurrence on two-manifolds (J.


Differential Equations 29 (1978), 338-395), Gutierrez showed that on any nonori-
entable compact surface of genus p > 4 there exists a smooth flow with a nonori-
entable nontrivial recurrent trajectory. The existence of such trajectories prevents
extending Peixoto's proofs of Theorems 5.1 and 5.2 to the case of a nonorientable
closed surface M of genus p > 4 and the space X (NYC) , 1 < k < r. We remark
that the flow with a nonorientable nontrivial recurrent trajectory constructed by
Gutierrez in the above paper can be approximated by Morse-Smale flows in the
C°°-topology.
Recall that on nonorientable closed surfaces of genera p = 1 and p = 2 there
are no flows with nontrivial recurrent trajectories (§2.2 in Chapter 2). Such flows
exist on a nonorientable closed surface of genus p = 3, but as shown in [85], they do
not have nonorientable nontrivial recurrent trajectories. Therefore, the proofs of
Theorems 5.1 and 5.2 extend to the case of a nonorientable closed surface of genus
1, 2, or 3.
In [79] there is a treatment of flows on the torus T2 given by polynomial vector
fields. Let :
2 -* T2 be the universal covering of the form (x, y) _ (e, eiy) .

Recall that the expression

Mk (x, y) _ (amncosmxcosny + bmn sin mx cos ny


j =O m+n= j

+ cmn cos mx sin ny + dmn sin mx sin ny)

is called a trigonometric polynomial of degree k.


A vector field V on T 2 is a polynomial vector field of degree k if for a covering
vector field on R2 (with respect to the covering ) both components are trigono-
metric polynomials of degree k.
5. THE SPACE OF FLOWS 143

Denote by Ak the set of polynomial vector fields of degree k that determine a


flow on the torus with at least one equilibrium state.
It is shown in [79] that the set of Morse-Smale vector fields (that is, the fields
giving Morse-Smale flows) is open and dense in Ak for all k E N.
We dwell on a paper by Gutierrez and de Melo, The connected components
of Morse-Smale vector fields on two-manifolds (Lecture Notes in Math., vol. 597,
Springer-Verlag, Berlin, 1977, pp. 230-251). Here the set of Morse-Smale vector
fields on an orientable compact surface is regarded as a topological space, and the
arcwise connected components of this space are investigated.
Denote by Y '(M) the space of Morse-Smale vector fields of class Cr (r > 1)
on an orientable compact surface M.
DEFINITION. Two vector fields X, Y E r (M) are said to be isotopically equiv-
alent if there exists a continuous mapping F : [0, 1] - * r (M) such that F(0) = X
and F(1) = Y.
In the paper under consideration there are necessary and sufficient conditions
for Morse-Smale vector fields in '(M) to be isotopically equivalent. We illustrate
the results in the simplest case of polar Morse-Smale vector fields.
Let X E '(M) be a polar Morse-Smale vector field which determines a flow
f t E Xr(M), r > 1. By the definition of a polar vector field, f t has exactly one
unstable node u(X). For each saddle S of the flow ft two of its w-separatrices
combine with u(X) and S to form a simple closed curve called a stable cycle of the
vector field X. Two different stable cycles of X intersect only at the point u(X).
It is proved in the paper that two polar vector fields X, Y E Y '(M) are isotopi-
cally equivalent if and only if there exists a bijection (a one-to-one correspondence)
between the stable cycles of X and the stable cycles of Y such that stable cycles
corresponding under this bijection are homotopic.
Similar assertions are proved for gradient-like Morse-Smale vector fields also
in the general case.
5.4. Flows of the first degree of structural instability. We first describe
the structurally unstable equilibrium states and closed trajectories that are possible
for a flow f t of the first degree of structural instability on an orientable surface M.
It can be assumed without loss of generality that the equilibrium state mo or
the closed trajectory l of f t under consideration belongs to a single chart U E E (M)
with coordinate system (x, y) : U - R2 in which the flow f t of the first degree of
structural instability is given by the system

(5.1) x = P(x, y), y = Q(x, y).


It is shown in [3] that the concept of the first degree of structural instability is
meaningful in the space X( M) of flows, r > 3. Therefore, it will be assumed that
f t E Xr (NYC) , r > 3, and hence the functions P (x, y) and Q (x, y) are smooth of class

Let (0, 0) be the coordinates of the equilibrium state mo E U C M, that is,


P(0, 0) = Q(0, 0) = 0. We assume that

0 = (PQ y - I(0,0) = 0,

t7 = (Px + Qy) I (0,0) 0.


144 3. TOPOLOGICAL STRUCTURE OF A FLOW

It is known [3] that with the help of a nonsingular linear change of the coordinates
(x, y) we can pass to new coordinates in which the right-hand sides of the system
(5.1) have the form (we denote the new coordinates again by x, y)

P(x,y) = Pz(x, y) + o(x,y), Q(x, y) = y + Q2 (x, y) +'tb(x, y),

where P2 (x, y) and Q2 (x, y) are homogeneous second-degree polynomials in x and


y, and the functions (p and and their derivatives through second order are equal
to zero atx=y=0.
The equilibrium state (0, 0) of the system (5.1) is called a saddle-node of mul-
tiplicity two if P2 (1, 0) 0 (Figure 3.44).

FIGURE 3.44

Let us consider the equilibrium state (0, 0) of (5.1) when

0 (0, 0) >0, cr(0, o) = 0.

Then ([3], [4]) the system (5.1) can be reduced by a change of coordinates to the
form

(5.2) x = y + So(x, y), y = -x + b(x, y)


where (p and and their first-order derivatives are equal to zero at x = y = 0.
After passage to polar coordinates (p, 0) the system (5.2) reduces to the equation

= R(p, 0),
dB

where R(0,0) - 0.
We expand the function R(p, 0) at the point p = 0 according to the Taylor
formula: R(p, 0) = R1(0) p + R2(0)p2 + R3(0)p3 + .... It is known ([2]) that the
first nonzero quantity cj = J " R(0) d8, i = 1, 2, ... , has odd index. Since o = 0,
it follows that ctrl = 0, and hence a2 = 0.
The equilibrium state (0, 0) of the system (5.1) is called a compound focus of
multiplicity 1 if L(0, 0) > 0, cr(0, 0) = 0, and cti3 0.
We proceed to a description of structurally unstable closed trajectories.
Let to be a closed trajectory of the flow ft. Through a point m E to we draw
a contact-free segment and introduce on it a regular parameter S : [-1, 1] - *
with S(0) = m. The Poincare mapping P : - * is defined in a neighborhood of
m. We expand the function Sp(n) = S-1 o P o S(n), n E [-1, 1], in a neighborhood
of the point n = 0: Sp(n) = hln + h2n2 +... (obviously, Sp(O) = 0).
5. THE SPACE OF FLOWS 145

FIGURE 3.45

FIGURE 3.46

The closed trajectory to is called a double limit cycle if h1 = 1 and h2 0


(Figure 3.45). This definition does not depend on the choice of the point m E to
and the segment .

The trajectory l is said to be doubly asymptotic to the limit cycle to if w(l) =


c( l) = to (Figure 3.46).
Obviously, in this case the limit cycle to is semistable (from one side the tra-
jectories wind off of the cycle lo, and from the other side they wind onto lo) and
nonhyperbolic.
DEFINITION. A trajectory l of a flow f t on an orientable surface M is called
a structurally unstable singular trajectory if it is one of the following trajectories:
1) a nonhyperbolic equilibrium state; 2) a nonhyperbolic closed trajectory; 3) a
separatrix going from an equilibrium state to an equilibrium state.
It follows from Theorem 5.1 that a structurally unstable flow f t on an orientable
closed surface M has a structurally unstable singular trajectory or a nontrivial
recurrent trajectory.
THEOREM 5.3 [7]. A C'-fl low f t (r > 3) on a closed orientable surface M is
a flow of the first degree of structural instability in the space X( M), 3 < k < r, if
and only if the following conditions hold:
1) f t does not have nontrivial recurrent trajectories.
2) f t does not have trajectories doubly asymptotic to a double limit cycle.
3) f t has one and only one structurally unstable singular trajectory, which is
one of the following types:
a) a compound focus of multiplicity 1;
b) a saddle-node of multiplicity 2;
146 3. TOPOLOGICAL STRUCTURE OF A FLOW

c) a double limit cycle;


d) a separatrix going from a saddle to a saddle, and if it returns to the initial
saddle, then the saddle value is nonzero.
4) the separatrices of saddles and of a saddle-node satisfy the following condi-
tions:
a) a separatrix cannot twist onto a separatrix forming a loop (or twist off of
this separatrix);
b) a separatrix cannot twist onto (twist off of) a double limit cycle if there is
another separatrix twisting off of it (twisting onto it);
c) the separatrices of a saddle-node cannot go to a saddle (that is, they are
not two-sided separatrices) and cannot belong to the boundary of a node sector of a
saddle-node that forms a loop.
There is a proof in [7], but the reader of this paper should first see [2] and [3].
5.5. On denseness of flows of the first degree of structural instability
in the space of structurally unstable flows. According to Theorem 5.2, the
set of structurally stable C''-flows (r > 1) on a closed orientable surface M is dense
in the space r (M) of C''-flows, 1 < k < r. It is natural to suppose that the set
of flows of the first degree of structural instability (that is, relatively structurally
stable flows) is dense in the set of structurally unstable flows. However, as the next
theorem shows, the situation here is more complicated and depends on the genus
of the surface.
Denote by )V' (M) the set of structurally unstable C"'-flows (r > 1) on an
orientable closed surface M. We shall regard O (M) as a topological space with
the topology induced by the spacer (M).
Let X ,r (NYC) be the set of C'-flows (r > 3) of the first degree of structural
instability on an orientable closed surface M.
THEOREM 5.4. 1) The set 1,T (NYC) of flows of the first degree of structural
instability is open in the space OV' (M) of structurally unstable flows.
2) The space X ,T (S2) of flows of the first degree of structural instability on the
sphere S2 is dense in the space OV' (S2 ), r > 3.
3) The set X1,T (Mp) of flows of the first degree of structural instability on a
closed orientable surface M of genus p > 2 is not dense in the space
r>3.
4) If M1 = T2 is the torus (an orientable closed surface of genus p = 1), then:
a) in the space of Clows on T 2 without equilibrium states the set of lows of
the first degree of structural instability is dense in the space of structurally unstable
lows;
b) in the space of C' lows on T 2 with equilibrium states the set of lows of the
first degree of structural instability is not dense in the space of structurally unstable
lows, r > 3.
There is a proof of Theorem 5.4 in [12] and [13].
CHAPTER 4

Local Structure of Dynamical Systems


The so-called method of normal forms, which goes back to Poincare, is a power-
ful instrument for studying the local structure of dynamical systems. The essence
of this method is to choose a coordinate system in a neighborhood of the set of
most interest to us in which the system has as simple a form as possible (a "normal
form"). The dynamics of a normal form is usually easy to study by virtue of its
simplicity.
The rectification theorem (see §2.7.2 in Chapter 1) is the simplest example of
this approach. The system
=1, i = 0 (x=(,i)ER2)
is taken as the normal form of a dynamical system in a neighborhood of a regular
point. The trajectories of this system are straight lines. The rectification theorem
thereby completely solves the problem of the local structure of trajectories in a
neighborhood of a regular point.
The structure of a dynamical system in a neighborhood of a rest point is con-
siderably more complicated in general. The simplest system in a neighborhood of
the point x = 0 serving as a rest point is a linear system
x = Ax.
Therefore, it is natural first to study the possibility of reducing a system to linear
normal form in a neighborhood of a rest point.
In connection with local behavior the general concept of structural stability
leads to the definition of a hyperbolic singular point. For a system of differential
equations
x = Ax -I- o(x)
of class C1 the point x = 0 is said to be hyperbolic if the spectrum of the linear
part A does not intersect the imaginary axis.
A C 1-diffeomorphism
F(x) = Ax + o(x)
is said to be hyperbolic at the point x = 0 if the spectrum of A does not intersect
the unit circle.
We recall that diffeomorphisms F and G are said to be C- conjugate 1 in a
neighborhood of a common fixed point xo, or locally C-conjugate at xo if there
exist a neighborhood V xo and a diffeomorphism
: (V,xo) - (W,xo)
1In this chapter the notation Cr'y` assumes that r > 0 is an integer and a E [0, 1]. This is
the space of mappings in Cr with highest derivatives in Lip a. By definition, Cr,o = Cr.
147
148 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

(with a different neighborhood W in general) such that


(F')(x) = G(x)
for all x sufficiently close to x0. We also say that F and G are locally conjugate in
the class C.
If r = 0, then must be a homeomorphism. A homeomorphism of class C°,« is
understood to be a homeomorphism that is in Lip a along with its inverse -1.
The concept of local conjugacy carries over in the obvious way to flows: it is
required that
=Gt(x)
for small t and for x sufficiently close to x0. In the classification of flows, especially
the topological classification, it is natural to consider equivalence and orbital equiv-
alence (see the definitions in §1 of Chapter 3). However, it should be noted that,
from the point of view of the methods and results to be discussed in this chapter,
conjugacy and equivalence of flows do not differ significantly, at least in dimension
>2.

§1. Dynamical systems on the line


1.1. Linearization of a diffeomorphism. Let
F : (R1, 0) - (R1, 0)
be a diffeomorphism of class C''with the fixed point x° = 0, that is, F(0) = 0,
and let
A = F'(0).
The general definition of a hyperbolic fixed point reduces here to the inequality
lAl71.
THEOREM 1.1. If r + a> 1 and if a diffeomorphism F of the line of class C"'
is hyperbolic at the point x° = 0, then it is locally conjugate in the class Cto a
linear diffeomorphism. A diffeomorphism F of class C1 is locally conjugate in the
class C°"3 to a linear diffeomorphism for any / < 1.
We give straightway an example showing that the statement of the theorem is
sharp: the inequality 3 < 1 cannot be replaced by 3 = 1 in general.
Let us consider the local diffeomorphism

F(x) = Ax + x
In x I'
F(0) =0.
We show that it is not locally conjugate to a linear diffeomorphism in the class
C°". Indeed, suppose that E C°", (0) = 0, and
(Fx) = A(x)
for small x. Then

x =1 (n=0,1,...).
Since , -1 E Lip 1 (according to the definition of a homeomorphism of class
C°"), it follows that
c2IxI < $(x) < c1IxI
1. DYNAMICAL SYSTEMS ON THE LINE 149

for some constants ci > 0. Therefore,

1>
- ci
2 lAThFThxl _ c2
lxi l 11
A-iIF(Fk-lx)I _
lFk-ixl cl fj+
k
(
1 l
1niFk-1xI/

To get a contradiction it suffices to verify that


00
(1.1)
In IFS-ixl = -oo.

Indeed, if (5> 0 is sufficiently small, then


(x(5, k=0,1,...).
Consequently,
1 1

In Fkxi - k In A(1 - E) + In xi'


which yields (1.1).
1.2. Lemmas on functional equations. The proofs of the conjugacy theo-
rems reduce to proofs that the equation
(1.2) c(Fx) = G(1(x)), 1(xo) = xo,
is solvable in a neighborhood of xo with the additional condition that be a local
diffeomorphism of class For r > 1 the last condition is in turn equivalent to
the derivative ' (xo) being nonsingular. If r = 0, then an additional check that
be a local homeomorphism is required.
We look for a reducing diffeomorphism in the form
(x) = x + Sp(x), SP(0) = 0.
Then for SP the equation (1.2) takes the form
cp(Fx) = G(x + o(x)) - F(x).
If we set
F(x) _ Ax + f (x), G(x) = ax + g(x)
here, where
f(0) =g(0) =0, f'(O) =g'(O) =0,
then we get that
(1.3) cp(Fx) - Acp(x) = g(x + p(x)) - f (x).
If G is linear, then g = 0, and we arrive at the linear equation
(1.4) cp(Fx) - Acp(x) _ - f (x).
We prove two lemmas, needed in the proof of Theorem 1.1, about the solvability
of the equations (1.3) and (1.4).
LEMMA 1.1. Let Al < 1 and r + a > 1. The equation (1.4) has a solution
cp E Cwith cp(0) = cp'(0) = 0 in a neighborhood of the origin. Any two such
solutions coincide in some neighborhood of the point x = 0.
150 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

PROOF. Let us write (1.4) in the form


(1.5) o(x) = A'f(x).
We let
(Tp)(x) = )C'f(x)
and choose S > 0 such that
F(x)I<6
The operator T acts in the space Lo (S) of all functions cp of class C'' defined on
[-S, S] and such that
o(O) = o'(O) = 0.
By making 8 smaller if necessary, we can ensure that
q 1A'I IF'(x)l1° <1 (lxi 6),
where
B = 8(a)
ifa>0,
1 ifa=0.
Let c1 > 0. Then for SP E L(6) with
(1.6) ko'(x)I <_ ciIxla (lxi <_ b)
we have that
I(Tco)'(x)I (qci + bi)ixi° (xi 6),
where
bl = max 1A'f'(x) I
Ixle
Fix c1 such that
bl
cl > 1
q
Then it follows from the inequality (1.6) that
(1.7) I(Tco)'(x)I < cx (Ix < 6).
Next, let c = {c}1 be a sequence of positive numbers. Denote by Sa(c) the
subset of functions cp E Loa(b) for which (1.6) holds along with the inequalities
(x)1<c (j<r).
If cp E So(c), then for j > 2
j-1
I(Tco)(x)I

where
b = IxI«
max IA'f(x),
and the coefficients a23 are determined by the derivatives of F. We choose numbers
cj so that
j-1 \
Cj 1 1 1 llijC{ (j 2).
1. DYNAMICAL SYSTEMS ON THE LINE 151

Then by (1.7) the set So (c) is invariant under T. If r = 00, then So (c) is a compact
convex set. By the fixed point principle, equation (1.5) has a solution SP E So().
Suppose now that r < oo, and denote by S(c, N) the compact convex set of
functions SP E So (c) such that
- (r)(x") <- Nw(I x' - x"I) (Ix', Ix"I s),
where w (u) is the modulus of continuity of the function2 f (T) (x) (w(u) = Iu « if
a> 0). There is a number N = N(c) large enough that the compact set S(c, N) is
invariant under T. Consequently, (1.5) has a solution SP E S(c, N).
Suppose now that SP is another solution, and let

M = max 10'(x) -
IxI<6 IxIcx

Then by the equation,


M<-qM.
Since q < 1, it follows that M = 0. The proof of the lemma is complete.

Let us now consider (1.3). We regard it on the whole line and assume accord-
ingly that f (x) and g(x) are functions bounded on the line. Moreover,
(1.8) lf'(x)l Ig'(x)I .

LEMMA 1.2. Suppose that Al < 1, r = 1, and a = 0. For each Q < 1 there
is a number E = (/3, A) such that (1.8) implies the existence of a bounded solution
cp E Lip/3 of (1.3) on the whole line, unique in the class of all bounded functions on
the line.
PROOF. We rewrite (1.3) in the form
(1.9) o(x) _ g(F-lx + o(F-lx)) - .f (F-lx)

If (1.8) holds, then

F-lx = A-lx + f (x), l?(x)l <_ e . ICI-1

Let Q < 1 be fixed. Denote by Tcp the operator on the right-hand side of (1.9), and
by S(c, N) the compact convex set of functions cp : I[81 -> I[81 satisfying the estimates

ko(x)l <c, ko(x') - co(x")l N lx' - x"l.


Since
Al IlA'la <1,
for sufficiently small a there exist numbers c and N large enough that S(c, N) is
invariant under T. Consequently, (1.9) has a solution SP E S(c, N).
Suppose now that SP is another bounded solution, and let
M = sup ko(x) - (x)l.

2 Recall that the modulus of continuity wh (u) of a function is defined by wh (u) _


mare/. //p<u Ih() - h()
152 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

Then in view of the equation,


M (IAI+)M.
Consequently, M = 0, and the lemma is proved.

By analogous arguments we can prove the solvability of a somewhat more com-


plicated equation. Namely,
(1.10) o(x) _ s(x, o(x)),
where
F : R1 X - R1, g:
f1 X fl
1
fl
1

are given C-functions. Let

A = (O,0), f(x,y) = F(x,y) - Ax.


Under the assumptions
f(0) = g(0) = 0, f'(O) = g'(O) = 0
we have the following result.
LEMMA 1.3. Suppose that < 1 and the integer l < r is such that
IIAt<1, g(x, 0) = O(xt) (x -0).
Then (1.10) has in a neighborhood of the origin a solution cp E C'"a satisfying the
condition
(x) = O(x1) (x -40).
Any two solutions with this condition coincide in some neighborhood of the point
x=0.
By analogy with Lemma 1.2 we now consider the equation (1.10) on the whole
line. Suppose that r > 0, a > 0, and f and g are bounded everywhere in the plane
together with all derivatives. It is assumed that the rth-order derivatives satisfy
the condition Lip a uniformly in the plane. Also, let
of of Dg Dg
< E, < E, < E,
ax ay ax ay

LEMMA 1.4. Suppose that IAI > 1, < 1, and


IILI . IAIr+a < 1.

For each ,3 < a there is an _ (i3) such that the conditions (1.11) imply the exis-
tence of a solution cc' E Cof (1.10) that is bounded together with its derivatives,
and cc' is unique in the class of all bounded functions on the line.
Although the proof uses a topological fixed point principle, in all the compact
sets arising we can introduce a metric such that our operators are contractions, and
then we can employ the Banach contraction mapping theorem in a metric space.
1. DYNAMICAL SYSTEMS ON THE LINE 153

1.3. Proof of Theorem 1.1. In the case r + a > 1 it suffices for us to


establish the existence of a solution SP E Cof (1.4) such that cp(0) _ cp' (0) = 0.
It can be assumed that Al < 1, and the assertion we need is contained in Lemma
1.1.
We now consider the case r = 1. The necessity of verifying also that (x)
x + cp(x) is a homeomorphism leads to a certain complication.
Since we are interested in local conjugacy, we can assume that

If'(x)I < (x E

where E is sufficiently small. Let us consider the two equations


o(Fx) - -f(x),
f(ax) - A(x) = f (x + fi(x))
on the line. By Lemma 1.2, each equation has a bounded solution of class C°'Q on
the line.
Let
(x) 'I'(x) =x+(x).
Then
(Fx) = A(x), f(ax) =F(i(x)).
C onsequent ly,

and
o W)(Ax) = A( o

Let
H(x) _ (Wo)(x).
Then
H(x)=x+h(x),
where
h(x) = o(x) + b(x + o(x))
is a bounded function on the line such that
h(Fx) - ah(x) = f (x + h(x)) - f (x) (x E II81).

By the uniqueness of the solution, h = 0. Similarly,

(oW)(x) = x + h(x),
N
where h is a bounded solution of the equation

h(ex) _ Ah(x)

on the line; that is, h = 0. Thus,


W o = id, o W = id.

Consequently, and '1' are mutually inverse homeomorphisms of class C°'.


154 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

REMARK. In reality we have proved global conjugacy to the linear part of a


diffeomorphism F(x) = Ax + f (x) under the condition
If'(x)I <_
where is sufficiently small.
1.4. Flows on the line. The local orbital classification of one-dimensional
flows is empty of content: all curves coincide locally with f1.
However, conjugacy of flows has more meaning, and the results here are com-
pletely analogous to the corresponding results on diffeomorphisms.
We consider the following equation on the line:
(1.12) x = vx + f (x), v 0, f(0) = f' (0) = 0.
Let f Then corresponding to (1.12) the local flow
Ft (x) = ?tx + ft (x), ft (0) = f(0) = 0,
is also of class C''' «
THEOREM 1.2. If r + a > 1, then the flow Ft is conjugate in the class Cr«
to the linear flow eUt. If r = 1, then Ft is locally conjugate in the class C°' to a
linear flow for any ,3 < 1.
As in the case of diffeomorphisms, the assertion of the theorem is sharp.
EXAMPLE. Suppose that v < 0 in (1.12), and
1
f (x) =
i;;ilt dt, IxI < 2 .
LX

This equation is of class C1. We show that it cannot be linearized in this class.
Indeed, suppose that a transformation x H (x) of class C1 carries the equation
into the linear equation

From the equation


(Ftx) = (t E Ilk+),
where Ft is the flow of the equation under consideration, we get that

(1.13) e -Ut (1 t (x1' (x)


dx ' Ftx
( )
- '1'0() (x) t -4.
+oo

On the other hand, we get from the formula


t
Ft (x) = ?tx + f (Fsx) ds
0

after differentiation with respect to x that


t e-mss
dFs (x)
e
-mot dF't (x) ds .
dx 1+ o InIFsxI dx
By (1.13), the left-hand side of this equality has a limit as t +oo, while the
integral on the right-hand side diverges, since
In IF8xI N vs.
1. DYNAMICAL SYSTEMS ON THE LINE 155

PROOF OF THE THEOREM. We can assume that the function f is defined


everywhere on the line and is bounded, and that
If'(x)I <_ ,
where is sufficiently small. Then the field (1.12) has a global flow
Ft (x) = evtx + ft (x),
where ft (x) is bounded for each t E R1. It can be assumed that
I(ItI<_ 1, x E R1).
By Theorem 1.1, the diffeomorphism F - Ft It_1 is conjugate to a linear diffeomor-
phism, in the class C" for r + a> 1 and in the class C°"3 for r = 1. Further, the
conjugating diffeomorphism satisfies the conditions (0) = 0 and '(0) = 1 in
the case r + a> 1, while in the case r = 1 it can be assumed that (x) = x + cp(x),
where SP is bounded on the line. Let us consider the flow
Ft(x) =
This flow is either of class C'''a (for r + a> 1), or of class C°"3 with /3 < 1. Since
F' (x) = evx, the flow Ft commutes with the linear mapping:
Ft (evx) = ev Ft (x) .
Let
t(x) = e-vtFt(x)
(Iti <_ 1).
From the commutation condition we get that
t(ex) = e111t(x).
If we now set
t(x) = x + cot(x),
then
Sot (evx) = evSpt (x) .
Further, if r + a > 1, then cp(0) = cp' (0) = 0, that is, SP = 0 (the uniqueness in
Lemma 1.1). But if r = 1, then SP is bounded on the line, and hence SP = 0 (the
uniqueness in Lemma 1.2). Thus,
e-vtFt (x) = x,
from which
evtx.
The theorem is proved.

In passing from diffeomorphisms to flows we can use another device due to


Sternberg. Namely, as above let E C'''a be a linearizing diffeomorphism for
F - Ft t=1 Then the family of diffeomorphisms
I1t = e-vt o o Ft

is 1-periodic in t. Indeed, W° = , and


t+1 = e-vte-v
o o F o Ft = e-vt o o Ft = fit.
156 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

Let
H(x)=f Ws(x) ds.
Then by periodicity,
-t+1
evtH(x) _ W3 (Ftx) ds = H(Ftx).
-t
If r + a> 1, then H is a local diffeomorphism because H' (0) = 1. Consequently,
H conjugates the flows evt and Ft. In the case E C°'Q it must also be checked
that is a homeomorphism (see [74]).

§2. Topological linearization on the plane


The well-known Grobman-Hartman theorem asserts that a C 1-diffeomorphism
is locally topologically conjugate to a linear diffeomorphism at a hyperbolic fixed
point x° E Rn. We refine this theorem in the spirit of §1.1.
2.1. Formulation of the theorem. Let
F(x) = Ax + 1(x), 1(0) = 0, f'(0) = 0,
be a diffeomorphism of the plane that is hyperbolic at the point x = 0. Denote by Al
and A2 the eigenvalues of A. We define a number r = r(A). If the spectrum of A is
not separated by the unit circle (the case of a node), that is, either max(I A 1 I, I A2 I) <
1 or min(I a1 I, I A2 I) > 1, then we set

T(A) =min (lnIAiI In Ia2I


In A ' In1A121
11

If the numbers Al and A2 lie on different sides of the unit circle (the case of a
saddle), then r(A) = 1.
Obviously, r(A) < 1. In addition to the case of a saddle, equality also holds in
the case when the eigenvalues have the same modulus: IA1 I = I A2 I.
THEOREM 2.1. For any a < r(A) the diffeomorphism F is locally conjugate in
the class C°,a to a linear diffeomorphism.
The theorem carries over to the multidimensional situation. We need only
refine the definition of the number r (A) . Namely, let A1,. . . , Al be the eigenvalues
lying interior to the unit disk, and let X61, ... , has be those exterior to it. Then the
proper definition is
In A2 In I Z
T (A) = min mm I I mm I

ia In I
AC I
ia In I ,a I

With this definition the formulation (and proof) of the theorem carries over imme-
diately to the multidimensional case.
2.2. Proof of the theorem. We must solve the equation (1.2) in which
x E R2, and
F(x) = Ax + 1(x), G(x) = Ax + g(x)
are diffeomorphisms of the plane such that
f(0) = g(0) = 0, f'(0) = g'(0) = 0.
3. INVARIANT CURVES OF LOCAL DIFFEOMORPHISMS 157

In this notation the equation for the transformation


(x) = x + (x), (O) = 0,
reduces to the form (1.3):
(2.1) o(Fx) - A(x) = g(x + (x)) - f(x).
We can assume that f and g are bounded in the whole plane.
LEMMA 2.1. For each a E [0, T(A)) there is an e = e(a, A) such that the
conditions
IIf'(x)II <_ e, Ig'(x)Il <
imply the existence of a solution cp E C°'a that is bounded in the whole plane, unique
in the class of bounded mappings.
PROOF. In the case of a node we assume that IAjl <1. We rewrite (2.1) in the
form
(2.2) (x) = Acp(F-ix)
+ 9(F'-lx + (Fx)) - f (F-lx).
Under the assumption that
lAu <1<1A21
we rewrite (2.1) in the case of a saddle as the following system in coordinates:
P1(x) = A1(pl (F-lx) + g1(F-lx + cp(F-lx)) - ,f 1(F-lx)
(2.3) 2 -1 2 -1 2 -1 2
P (x) = A2 P (Fx) - a2 g (x + (x)) + a2 f (x) .
Here cp = (1, cp2) , g = (g', g2), and f = (f1, f2). The proof of existence of a
bounded solution of class CO, for the equations (2.2) and (2.3) is carried out by
the scheme of the proof of Lemma 1.2 in §1.2 with the definition of r (A) taken into
account. The proof of uniqueness is analogous.

Theorem 2.1 can now be derived from the lemma as in the one-dimensional
case.

§3. Invariant curves of local diffeomorphisms


We consider the following linear operator in the plane:
Ax = (Aie,A2ii) (x = (,i) E I1)
The coordinate lines
ll = {x = (e,0)}, l2 = {x = (0,)}
are invariant under A. Let
F(x) = (Aie+f1(x),A2i+f2(x)), F(0) =0,
be a diffeomorphism with linear approximation A. If F is locally conjugate to A by
the conjugating diffeomorphism ,
= A,
then the curves
-1(li)
Mi = (i= 1,2)
158 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

are invariant under F. The smoothness class of these curves is not less than that
of .In particular, the diffeomorphism F has invariant curves of class C°, « by
Theorem 2.1.
According to the well-known (multidimensional) Hadamard-Perron theorem, a
diffeomorphism of class Cr has at a saddle point invariant manifolds of class Cr
whose tangents coincide with 11 and 12, respectively. Here we prove this theorem
for the plane and consider also the case of a node.
These invariant curves are described by equations
M1 = {x 1 i = yl ( ) }, M2 = {x 1 = y2 (rf) },

where the y2 are smooth functions such that


'y(0) = 0, 'y(0) = 0.
The smoothness class of the curves Mi is the same as that of the functions 'y2.
3.1. Invariant curves of a node. Suppose that
1x11<1A21<1.
THEOREM 3.1. The diffeomorphism F E Cr'Qe has a unique local invariant
manifold M1 of class Cr'`x, and it also has an invariant manifold M2 of class CqQ
for
In Ia11
q + ,C3 < min r + cx,
In 1A21

As we see, M2 has lower smoothness class than the original diffeomorphism in


general. Moreover, M2 is not unique.
EXAMPLE 3.1. Let

F(x) = (A2e+ 0< A <1.


This diffeomorphism is analytic. Our theorem ensures the existence of a curve M2
of class C1"3 for any , 3 < 1. We show that F does not have an invariant curve M2
of class C1'1. Indeed, assume that
M2 = {e = 'y(i) } , y(o) = y' (o) = o,
is an invariant curve. Then the function y satisfies the equation
A2y(i) + 2a2i2 =
y E C1'1, then the function
S =
is bounded in a neighborhood of zero and satisfies the equation
s(ay) - s(i) =1.
However, iterating the last equality, we get that

which contradicts the boundedness.


3. INVARIANT CURVES OF LOCAL DIFFEOMORPHISMS 159

The uniqueness of the manifold M2 is violated even for a linear mapping. In-
deed, let Ai > 0. The curve
T (lnIifl' T
lnA1
e=IlIy In A 1 - In A 2 '
where y is any smooth 1-periodic function, is invariant under A. If y E Cr, r > [T],
then the smoothness class of such a curve is not less than [r] (T - 1, if r is an odd
integer).
In any case if IA1 I IA2 I , then in view of Theorem 3.1 the diffeomorphism F
has smooth invariant curves M1 and M2. Let A 1 I < I A2 and let
( ) = (C - 'y2(11),11 - 71(C)).
Then is a local diffeomorphism of class CqQ, where
InIA11
q ,Q < min r +c, In I AZ

+
Thus, we have
COROLLARY 3.1. Let IA1 I IA2 I. Then the diff eomorphism F is locally conju-
gate in the class CqQ to a diffeomorphism for which the lines li are invariant.
PROOF OF THEOREM 3.1. The function 71 in the definition of the curve M1
must satisfy the equation
(3.1) l f2
71(C) = A2 1y1(AlC + f 1(C 71(C))) - A2 (C, 71(C)).

Since IA1I lAd < 1, it follows from Lemma 1.3 in §1.2 that the equation has a
unique local solution 71(C) = o(C) of class C?',a
To prove the second part of the theorem it suffices to establish the existence of
the curve M2 for the inverse diffeomorphism

= (Aj1S + f 1(x), A2 1 + 12(x))


Then for the function 72 corresponding to M2 we get the equation
72(77) = ai'Ya(Aa lil +
Here
I
1, IAuI I
<1.
By Lemma 1.4, this equation has a local solution y2 E C". The theorem is proved.

3.2. Invariant curves of a saddle point. Here the situation is better. Let
IA1I <1 < 1A2I.

We have
THEOREM 3.2. At a saddle point the diffeomorphism F has unique local in-
variant curves M1 and M2 of class C''a.
COROLLARY 3.2. At a saddle point a diffeomorphism of class C''a is locally
conjugate in C?', a to a diff eomorphism for which the axes l1 and l2 are invariant.
160 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

PROOF OF THEOREM 3.2. The function yl satisfies the equation (3.1), which,
by Lemma 1.3, has a unique local solution yl () = O() of class C. Similarly,
passing to the inverse F-1, we get the existence of a unique curve M2 of class Cr', « .

§4. C1-linearization on the plane


First, suppose that ) are the eigenvalues of the linear approximation A = F' (0)
of a local diffeomorphism
F(x) = Ax + f (x) (x E 1R2)

of class Cl,. «Recall that r = T(A) = 1 if x = 0 is a saddle point, and


Cln A1l 1nIA211
T(A) =min
In A2l' In IAill
if it is anode.
THEOREM 4.1. Let c> 1 - r(A). Then F is locally conjugate in the class C1
to a linear diffeomorphism.
PROOF. We must prove that the equation
(Fx) = A(x) - f (x) f = (f 1 f 2)
,

has a local C1-solution cp with (O) = 0 and p' (O) = 0. For definiteness, assume
that
IA1I <_ 1A21.
In the case of a node (1A21 <1) and under the condition
In IA11
_1+a>
.. , In IA21

let us rewrite our equation in the form


(x) = A 1 P(Fx) + A- l f (x)
Then the operator
(T)(x) = A(Fx), p(0) = 0, P'(0) = 0,
is a contraction with respect to the norm
Ik' (x)II
lIpIl = sup IIxIla II=II <_ b,

for some small S, and our equation has the required solution cp. Now assume that
either 1A21 <1 and
1 +cx< In lA1 l
- In 1A21
or IA1 I < 1 < IA2 I. Then, as follows from Corollaries 3.1 and 3.2, we can assume
that the coordinate axes li are invariant under F. If we write F in coordinates,
F(x) _ (alb + f 1(x), 12rf + f 2(x)),
then the invariance means that
f'(,0) = f2(,9) = 0.
4. C1-LINEARIZATION ON THE PLANE 161

Denote by £, the space of C1-mappings


h:
JR2
- 2, h(o) = 0, h' (o) = 0.
We have the following direct sum decomposition into subspaces:
£, =J, 1 +1'2 +J'3
where
ah
J= hEJ
{ 1 I
a
=o },
= hEJ ah_0{
},
I
a
={hE£ Ih(0,9)=h(e,0)=0}.
In other words, the mappings in £ 1 do not depend on the second argument, and
the mappings in £2 do not depend on the first. Onto each of the subspaces there
is a natural projection
PZ : J -*J (i=1,2,3).
Namely,

(Pih)(x) _ h(e,0), (Pah)(x) _ h(0,r1)> (Pah)(x) = h(x) - h(e, o) - h(0,r1)


Let
A(x).
The subspaces
j, 2+ j, 3 C j,, £3CJ
are invariant under T. To prove that the equation
Tcp=-f, (pEJ,
is solvable it suffices to prove that each of the following equations is solvable:
P1TS = -P1f, S E C1,
(4.1) P2Tb = -P2 (f + TS), b E J2
P3Tcp = -P3(f + TS + (p E1'3.

Indeed, since the subspaces £2 + £,3 and £,3 are invariant, we have the equalities
P1Th = PiT (S + ib+ gyp),
P2T(S+ib) = P2T(S+ + SP)
Consequently,

T (S + ib+ p) = (P1 + P2 + P3 )T (S + + gyp)


= P1T (S) + P2T (S + ib) + P3T (S + + (p) = -1.
Thus, the proof has been reduced to the solvability of the three equations (4.1).
The first two are in a single variable. Here we can invoke results in § 1.2.
We consider the last of the equations (4.1). It has the form
(4.2) o(Fx) - A(x) = 'y(x), 7E
162 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

where 'y belongs to the class C. We must prove the existence of a solution cp E £3
of class C1. In coordinates, the equation (4.2) reduces to a pair of equations of the
form

(4.3) h(Fx) - /h(x) = 9(x), 9(0, i) = 0(, 0) = 0


with respect to a function h(e, i) of class C1 satisfying the condition

h(0, r) = h(e, 0) = 0.

Further, the coefficient /2 in (4.3) satisfies one of the conditions

(4.4)

(4.5)

LEMMA 4.1. Suppose that in the equation (4.3) the function 9 belongs to the
class Cl,a, and the number tc satisfies one of the conditions (4.4) or (4.5). Then
the equation has a local solution of class C1.
PROOF. Assume the condition (4.4). We write (4.3) in the form

(4.6) h(x) = 2-1h(Fx) - tc-19(x)

We can assume that the function 9 satisfies the condition


1 ao 1 ao
<00, <00,
p a P I-;l« a-j

and the derivative f' (x) is sufficiently small on the whole plane:

IIf'(x)II <e.
We consider the space £3 of all C1-functions h : IR2 - * R2 satisfying the conditions

h(0, h(e, 0) = 0,
ah
IIhII
I7i a
The operator on the right-hand of (4.6) is a contraction in this norm; therefore, the
equation has a solution h E £4.
Suppose now that the second condition on µ is satisfied, that is, (4.5). Then
we rewrite (4.3) in the form

h(x) = µh(F-1x) + e(F-1x),


and the rest of the argument is analogous.
The lemma is proved, and with it the theorem on C1-linearization.
5. FORMAL TRANSFORMATIONS 163

§5. Formal transformations


To investigate the local structure of a dynamical system it is sometimes required
to reduce it to "normal form" with the help of a transformation of a higher class
of smoothness than C1. However, in the linearization of a system obstacles arise
already in the class C2, for example, connected with the "formal" solvability of the
equation (1.3). For instance, as we saw in §3, the diffeomorphism
F(x) _ (A2e+ 2 A <1,
not only cannot be linearized in C2, but does not even have a local invariant C2-
manifold M2. This necessitates looking for other "normal forms", different from
linear forms, to which a diffeomorphism can be reduced by a C'-transformation
with r>2.
5.1. Formal mappings. We recall that a formal mapping
P : W -*Rm
at the point xo = 0 is defined to be a formal series

p= P(k),
=o
where p(k) : IRn -f Itr is a homogeneous polynomial mapping of degree k. In
particular,
P°) (x) = const E IR'n
is the free term of the mapping P. In the coordinates

the formal mapping can be written as a series


00

P(x)- {>>Pj
k-p III=k
x
1=1

Here I = (Ii,... , In) is an integer multi-index, and


III=Il+...+In, xI =( 1)"...(r)' .

The set of all formal mappings will be denoted by IR[n, m]. This set is a linear
space over Ilk with respect to the obvious operations.
The space Ilk [n, 1] has an obvious structure of an algebra over IIS, and the space
R[n, m] has the structure of an 1 [n,1]-module. Finally, if P E IR[n, m] is a mapping
without free term (P(°) = 0), and 1 E IR[m, l] is arbitrary, then the composition
o P E Ilk [n, l ]

is defined (substitution of a series in a series).


The space JRo [n, n] with zero free term is a semigroup with respect to compo-
sition. The subgroup of invertible mappings consists of the elements
P(x) = Tx + P(k) (x)
k>a
164 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

with nonsingular linear part:


det T 0.
We call such mappings formal diffeomorphisms.
In coordinates we can define the partial derivative
aP ap(k)
- P E IR [n, m],

and the Jacobi matrix (over the ring Ilk [n, 1] )


(aPi
P'(x)

With each Cr-mapping F : W- -f IRt we can associate the formal mapping


r
/3(F) =
=o
where
(k) 1
akF I
III=k 1 ! n

For r < oo the mapping /3(F) is called the Taylor series segment, and for r = 00
the Taylor series of the mapping F.
According to a well-known lemma of Borel (see, for example, [27]), for r = 00
the correspondence
(5.1) F H f3(F)
is surjective: for each formal mapping P E IR[n, m] there is a C°°-mapping F : IRn
IRt such that
/3(F) = P.
The correspondence (5.1) is clearly not injective.
5.2. Conjugacy of formal mappings. Two formal mappings F, G E Ro [n, n]
are said to be conjugate if there exists a formal diffeomorphism E JRo [n, n] such
that
(5.2) = C.
Let
F(x) = Ax + ...E IRo [n, n].
We determine the simplest possible form to which this can be reduced by conjuga-
tion. Let
(x)=x+So(x),
where cp E JRo [n, n] is a mapping with zero linear part:

(x) _ co(k) (x) .


k>2

Then we get from (5.2)


- A2( x) = C2( x) - F2( x),
5. FORMAL TRANSFORMATIONS 165

and for k > 3 the recursion relation


p(k) (Ax) - Ask(x) = G(k) (x) - Rk (k-1))
(x)
in which Rk depends only on "lower" terms. We consider the subspace
Ilk (k) [n, n] C Ilk [n, n]

of kth-degree homogeneous polynomial mappings h : IRn -p IRE. Acting in this space


is the operator
Akh=hoA-Ah.
Denote by A1,. . . , an the eigenvalues of the operator A. Then the spectrum of Ak
consists of all possible numbers of the form
{A ...inn-A2}, P1+...+pn=k, piEZ+.
Therefore, if
(pi+.+p=k,pEZ+),
then Ak is a nonsingular operator. The equality

in which P1 + + pn = k, is called a resonance relation (or resonance) of order


k. If for some k > 2 the operator Ak does not have any resonances of order k, then
F can be reduced by conjugation to a mapping G such that G(k) = 0. Indeed, let
(p(l) = 0 (2 < l < k - 1). Then the recursion formula for (p(k) takes the form

Akcp(k) = G(k) - F(k).


Using the surjectivity of Ak, we choose (p(k) so that
_F(k) = Ak(p(k),

Then G(k) = 0, and the formal mapping


(x) = x (p(k) (x)

reduces F to the necessary form.


Further, if for some r > 2 the operator A does not have any resonances of
orders 2 < l < r, then F can be reduced by conjugation to the form
G(x) = Ax + G(k) (x) .
k>r+1
In particular, the mapping F can be linearized in the absence of resonances of all
orders r > 2:
(F1)(x) = Ax.
In general, linearization is impossible in the presence of resonances, but it is
possible to determine a certain normal form.
We consider what resonances can be realized at a hyperbolic singular point in
the plane.
Let spec A = {A1, A2}. If the numbers A 1 and A2 are not separated by the unit
circle, then a unique resonance is possible. It has the form
"1 = "2
166 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

for some integer p > 2. Assume that


Au <1 < A21.
If
Al =
for some p, q E Z+, then p > 2 and q > 1. Therefore, if there is at least one
resonance, then
S
A1A2=1 a

for some integers 1, s > 1. It can be assumed that the numbers l and s are relatively
prime. Then all resonances have the form
.1
Al = (A1 A2) ' A2 = (A1A2) ' A2.

Corresponding to these possibilities we consider the following three types of formal


mappings on the plane:
I. G(x)=Ax;
II. G(x) = (Ae+i,Aui);
III. G(x) =
A1S+ >lCj(1)3,
A2ui +u>j=1 d( jr Is j
oo
ui )

THEOREM 5.1. If the eigenvalues of the linear operator A : j2 - J2 are dif-


ferent from 1 in modulus, then the formal diffeomorphism
F(x) = Ax + .. .
can be reduced to a diffeomorphism of one of the types I-III by conjugation.
This is the well-known theorem, going back to Dulac, on reduction to a so-called
resonance normal form (see, for example, [27]).
5.3. Formal vector fields and flows. A formal mapping P E 11 o [n, n] can
be interpreted as a formal vector field giving a formal system
(5.3) x = P(x).
According to this interpretation, the space o [n, n] can be regarded as a Lie algebra
£ [n] with the commutation operation
[P, Q](x) = P'(x)Q(x) - Q'(x)P(x)
It is natural to regard this algebra, in turn, as the Lie algebra of the (Lie) group
C3 [n] of formal diffeomorphisms. The existence of an exponential mapping
exp A [n] - C3 [n]
is ensured by a "formal Cauchy theorem", which asserts that to each field P (x)
there corresponds a one-parameter subgroup Ft E C3 [n] such that

Ft(x) = P(Ftx).
dt
The subgroup Ft will be called a formal flow.
The formal transformation
x --> (x)
5. FORMAL TRANSFORMATIONS 167

carries the system (5.3) into the system


N N
x = P(x), P(x) = ('(x))1P((x)).
Two systems connected by such a transformation will be said to be conjugate.
Supplementing coordinate transformations by multiplication by a "formal function"
h E 11{n, 1] with nonzero free term, we arrive at the concept of (formal) equivalence,
or orbital equivalence of formal vector fields.
Let
P(x)=Ax+ EC{nJ,
and let
spec A = {v}=. 1
The equalities
n
2,
i=1
serve as analogues of resonance relations in connection with the system (5.3). If
such equalities are absent for all k > 2, then the system is conjugate to a linear
system.
By analogy with formal diffeomorphisms, either the resonances
vl = P112, Re vl , Re 112 > 0,

or the resonances
vl = (kl + 1)vl + ksv2, 112 = klv1 + (ks + 1)v2 (k= 1, 2, ... )
are realized for hyperbolic vector fields on the plane, where 1, s E Z+ are relatively
prime, and
1111 + s v2 = 0.
On the plane we consider the following three types of formal vector fields:
I'. Q(x) = Ax;
II'. Q(x) = (pv+r,v77);

III'. Q(x) _
vl + C,j

112 97 + 71 >J= 1 d3 ( ii ), where 1111 + sv2 _ 0.

THEOREM 5.2. A formal hyperbolic vector field on the plane is conjugate to a


field of one of the types I'-III'.
COROLLARY. A formal hyperbolic vector field on the plane is orbitally equiva-
lent to a linear field, or to a field
Q(x) = (p'+jP,j), PEZ+,
or to a field of the form
00
(5.4) Q(x) = (14' >b(h18)i,ij), ii -m
j=1
=
168 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

§6. Smooth normal forms


Assume that the operator A is reduced to Jordan normal form. Then the
mappings I-III in §5.2 satisfy the relation
(6.1) G(x) = Ax +g(x), g(A*x) = A*g(x),
where A* is the adjoint operator, and the series g begins with quadratic terms. It
is obvious that the equality
(6.2) g(Ax) = Ag(x)
also holds, but this is connected with the specific nature of two-dimensional hy-
perbolic operators: any (not necessarily hyperbolic) formal diffeomorphism on 1l
can be reduced by conjugation to the form (6.1) (see, for example, [31]), but it is
impossible to reduce to the form (6.2) in general.
Conversely, if G satisfies (6.1), then it has the form I-III. Correspondingly, we
call a local C°°-diffeomorphism G with the form (6.1) a normal form. In the case
of a node, such a diffeomorphism is always a polynomial diffeomorphism. In the
case of a saddle point, this is no longer so, not even in the absence of resonances.
EXAMPLE. Let A1 <1 < IA I. There exist a, /3> 0 such that
IA1I
IA2I
= 1.
Let
exp (-iiii) ( . 0),

= {0
Then the C°°-diffeomorphism
G(x) = (A1 + h(, 1), A27)
is a normal form by our definition. It turns out, however, that the transition to C°°-
normal forms does not introduce new invariants: if two (hyperbolic) normal forms
have the same (or conjugate) formal Taylor series, then they are locally conjugate
in the class C°°. This assertion is contained in the well-known multidimensional
theorem of Sternberg and Chern, which will be presented in §6.4.
6.1. Normal forms with flat residual. Let
F(x) = Ax + f (x), f (O) = f'(0) = 0,
be a C''-diffeomorphism of the plane that is hyperbolic at zero.
THEOREM 6.1. There exists a local C°°-diffeomorphism : (2,0) (J2, 0)
reducing F to the form
(F)(x) = G(x) + h(x),
where G is a normal form, and the residual h E C'' has all derivatives equal to zero
at the point x = 0:
ar+qh
(6.3) a g (0) = 0
a r71
(p+q< r).
A mapping h with the condition (6.3) is said to be r-flat at x = 0, or simply
flat in thecase r=o0.
6. SMOOTH NORMAL FORMS 169

Subsequent normalization of the mapping F now consists in "killing" the flat


residuals h.
PROOF OF THEOREM 6.1. Let r < oo. We consider a formal mapping
reducing the Taylor series segment /3(F) to the normal form G. For the mappings
and G we throw out all terms of orders > r + 1, and denote by and G the
resulting polynomial mappings. Then the difference
h= G
is r-flat at zero, and G is a normal form.
Suppose now that r= oo. Following Borel's lemma, we "extend" the formal
diffeomorphisms G and to local diffeomorphisms G and . Then the difference
h= G
N
is flat at zero. However, G Ncan fail to be a normal form in general. To finish the
proof we need to "correct" G so that (6.1) holds. With this goal we set
N
G(x) = G(x) + T (X),
where T is an unknown flat residual. The relation (6.1) gives for T the equation
N N
T (AX) - AT (x) = AG(x) - G(Ax).
Here the right-hand side is flat at zero. The assertion that such equations are
solvable in flat mappings T is part of the proof of the Sternberg-Chern theorem
(see §6.4).

6.2. Smooth normal forms of a node. Let F E Cbe a diffeomorphism


of the form
F(x) = G(x) + h(x),
where G is a normal form, and h is r-flat. We assume here that
laul <_ Ia21 <1.
Let
In 1 A21 _- rT (A) .
T = r In A
111
THEOREM 6.2. If a> 1 - T, then F is locally conjugate in the class Cr to G.
In the case of a node there are only the normal forms I-II, so we get the
COROLLARY. If rT(A) > 1, then a C''-diffeomorphism F can be reduced by a
local C'-transformation either to a linear diff eomorphism or to the normal form
II with some p < r. If rT(A) = 1 and a > 0, then a C-diffeomorphism F can
be reduced by a local Cr-transformation either to a linear diffeomorphism or to the
normal form II with p = r. If rT(A) < 1 and a > 1 - rr(A), then F is locally
conjugate in the class C'' to a linear diffeomorphism.
We shall see later (§6.5) that the inequality for a in the case rT(A) < 1 cannot
be replaced by the equality a = 0. In this connection it is interesting to note that
the condition rT (A) > 1 is equivalent to the absence of "nonintegral" resonances
(fl+y=r, fl,'Y>-o).
170 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

At the same time, the inequality rT (A) < 1 means that

IAuI = 1+ (r - 1)T 1 > o.

PROOF OF THEOREM 6.2. Let


G(x) = Ax + g(x), g(O) = g'(O) = 0.
Actually, either g = 0 or
g( , i) _ (uiP,o).
Suppose first that rT(A) > 1. As before, we look for a reducing diffeomorphism in
the form

where o E C'' is r-flat. The equation for cp is


o(Fx) - [9(x +40(x)) - 9(x)] _ -h(x).
We rewrite it in the form
(6.4) o(x) = A-lcp(Fx) - A-1 [9(x + 40(x)) - 9(x)] + A-lh(x)
Let Lo(8) be the space of all mappings cp: I[82 , II82 defined in the 6-neighborhood
of the origin and r-flat at zero. If b > 0 is such that

then the operator Tip on the right-hand side of (6.4) acts in the space Lo(S).
We shall denote by cp(k)(x) the k-th order derivative of cp, that is, the collection
of all partial derivatives
(k)(x)_f40(x) p + q -k
SP
aaq
For a fixed x E j2 this derivative can be interpreted as an element of the space of
tensors of corresponding rank. Assume next that r < oo. Then
(T40)(7')(x) A(x)4(T)(Fx) - A-1g'(x +
where
A(x) = A-1 ® (FI(x))®r.
The set of eigenvalues of the operator A(0) coincides with the set of all numbers of
the form
{-lA7P}, 91
(p,q E Z+)
It follows from the inequalities
P2lT<P1I<la21<1
that all these numbers have modulus < 1. Therefore, we can introduce a norm in
II82 such that for sufficiently small 8 > 0
(6.5) IIA(x)II ' IIF'(x)II 4<1 (lixil 8).
In L(6) we consider the subspace of mappings So such that

IIcoIlsup <00.
= IIxII°
6. SMOOTH NORMAL FORMS 171

It follows from (6.5) that for sufficiently small S there is a ball


S(D) _ E Lo(b) I IIcolI D}
in which T is a contraction (recall that g(x) _ (crjP, 0)). The equation (6.4) has a
solution So E 8(D).
It also follows from (6.5) that any two solutions So and Sp satisfying the relations

So(x) = o(II xIl ' ), (x) = o(II xIl ' )

coincide in some neighborhood of zero.


Now let r = oo. We fix a number ro such that
(6.6) 11A1M ' I< q < 1 (k > TO, Mxli < S).
For a sequence c = {c }r0 we denote by S(c) C Lo (S) the compact convex set
consisting of all C°°-mappings satisfying
max <- ck (IIclI <- b, k = ro, ro + 1, ...) .
r<k I I co(x) I I
By (6.6) the sequence c can be chosen so that the compact set S(c) is invariant
with respect to T. The equation (6.4) has a solution So E S().
Suppose now that rT (A) < 1. Then G is linear, and the proof of the theorem
is almost a repetition of the arguments in §2 relating to the case r = 1.
Namely, if
In I A 1 I
r+a > In 1A21

then, as we have seen, the operator T is a contraction with respect to the norm
(x) II
llcolI =SUP oEL(6).
= llxIl° '

In the case
r+a< Inin 1A21
IA1I

it can be assumed that the axes 11 and l2 are invariant with respect to F. Under
this condition we should prove the existence of a Cr-solution of the equation
So(Fx) - ASo(x) = h(x),
with an r-flat part h E C. Repeating the arguments for the case r = 1, we
reduce the proof to a proof that the three equations in (4.1) are solvable, with the
difference that the solution must be in Cr. The first two equations are in a single
variable, and we again can use the results in §1.2. The third equation in (4.1) can
be written in the form
(x) = A-14o(Fx) + 0(x),
where 0 E £3 is a mapping of class C. By the condition on the number a, to get
a contraction we should give the norm by the formula
1
ar T
(6.7) llco = mac (sup air sup
j k>1

This concludes the proof of the theorem.


172 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

6.3. Smooth normalization in a neighborhood of a saddle point. Here


there is a more essential lowering of the smoothness class of the reducing diffeo-
morphism than in the case of a node.
As before, let

(6.8) F(x) = G(x) + h(x) E CT'a, r < oo,


where G is a normal form and h E Cis an r-flat residue. We assume that
IAu I < 1 < 1A21, Ai E spec F'(0).

THEOREM 6.3. For any a > 0 the diffeomorphism F is locally conjugate in


the class CP to the normal form G, where p = [r/2]. The condition « > 0 can be
replaced by the condition a = 0 provided that it is required in addition that

PROOF. This is based on an elaboration of the methods used in Theorem 6.2.


We consider the following collection of linear spaces:

= {h I h(x) (i = 1, ... ,p + 1)

where 'y: (1181, 0) - (j2, 0) is a mapping of class C7'-i+1,a that is (r - i + 1)-flat at


the point i = 0. Further,
= {h I h(x) = i-P-28(e)} (i = p +2, ... , 2p +2),
where 8: (1181, 0) (I182, 0) is a mapping of class C-+2, that is also (r-i+p+2)-
flat at the point = 0. Finally, we denote by £2p+3 the space of all Cr-mappings
h: (12,O)

that satisfy the conditions


ai+.7 h ai+ h
a) = a is =0

b is P_i <
aDPh(x) - cIe
CI
a ICI
i

Let

Zr=
i=1
As before, let Lo'a be the space of all C'-mappings that are k-flat at zero. We
have the inclusions
N
Lo'a C Zr C Lo
(recall that p = [r/2]).
The natural projections
Ni : L' a -i
6. SMOOTH NORMAL FORMS 173

act in the space Lo'a. Namely,

(Nh)(x)=
1 _1DhI
(z - 1)' Di-1 I =o

(Nih)(x) _ (i _P_ 2)! (p+ 2 <i < 2p+ 2)


zP+z
(NzP+sh)(x) = h(x) - (Nih)(x).

To prove the theorem we must prove the existence of a p-flat solution of class
Cfor the equation
(6.9) So(Fx) - ASo(x) - [g(x + o(x)) - g(x)] = h(x)
where h = g - f.
It can be assumed from the start that the coordinate axes 11 and l2 are invariant
under F and that
1A111 IA1 I' (0< j< p, v=1,2),
1A111 (0< j< p, v=1,2),
(v = 1, 2).
Under these conditions we prove the inclusion
N
T (L') Lo'a
where T is the operator on the left-hand side of (6.9). The theorem will thereby be
proved.
Observe first that it suffices to prove the existence of solutions Spi E £i of the
equations
N1T4p1 = h1
N2T(cp1+...+ci)=hi
(2<i<2p+3),
where hi = Ni h. Indeed, let
O _ O1 + ... + O2p+3.

It follows from the definitions of the spaces £i and the projections Ni that
N2T (Sp1 +... + So) = NT (So) (1 <_ i < 2p + 3).
Consequently,
h = N1T Sp1 + N2T (Spl + (p2) +... + N23T (Sp)
= N1 T (Sp) + N2T (Sp) + ... + N23T (Sp) = T
which is what was required.
Proceeding to the proof that the equations in (6.10) are solvable, we note first
of all that the equations with indices i = 1, i = p + 2, and i = 2p +3 are nonlinear
in ci, while the remaining equations are linear. More precisely, let
Spi(, i) = i_lT(jl) (2 <_ p + 1),
(pi(', I) _ 2- (p+ 2 < i <- 2p+ 2).
174 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

Then for i = 1 we get the equation


(6.11) 'Yi(Azu1 +.f2(O,i/)) - - [g('y(i),il +7(u1)) -9(O,i1)] =
Further, for 2 < i < p + 1
(6.12) (a1 +fl (o,jj))Z-l.yz(Azjj +f2(o,ur))-A'rz(i1)
-g1(),+y(?))yj(?)) =h(ij),

where
= - Nz[9(x + + ... + Pz-i) - 9(x)1
Similarly, for Sl we have the equation

(6.13) s (A1 + f' (, o)) - nsl [g( + 8), 8)) - o)] = o),

and for p +3 < i < 2p + 2 the linear equations


(6.14) (A2 + f2(, 0))iP 8__i(Ai + f1(e,o)) - A8__1()
- g1( + =
where
N
hi () = hi () - Ni [g(x + 401 + ... + 4i-1) - g(x)l.
Finally, for So = 4021+3 we again have a nonlinear equation

(6.15) o(Fx) - Acp(x) - g(x + Cpl + + (p2p+2 + cp)


+ 9(x +401 +" . + 402p-I-3) -
where
N
h2+3 (x) = h2+3 (x) - Ni [g(x + 4°1 + ... + 021+2) - g(x)].

The proof that (6.11)-(6.15) are solvable uses the same functional methods as
above. For instance, let us consider (6.15).
Since we are interested in local solutions, we can assume that the mappings f
and g have compact support. We consider the Banach space B of all Cr-mappings

40 E £21+3,

that are p-flat at zero, with the norm


1
Il(nll = ann ma.x <00.
z 1p-i+c1711z
I

N
It can be assumed that E B.
Suppose now that

IIf'(x)II C e, 9'(x)5; e (x E II82).

LEMMA 6.1. There is an e = e(A) such that the equation (6.15) has a unique
solution cp E B defined in the whole plane.
6. SMOOTH NORMAL FORMS 175

PROOF. For definiteness assume that


lAu < IAiIP+a IAI< IA2I

We write out the equation as the system


cpl (x) _ Alcpl (F-lx) + gl (F-lx + B(F-lx) + o(F-lx))
- g1 (F1x + 0(Fx)) + h3(x),

o2(x) _ A2 1cp2(Fx) - 21g2(x + 9(x) + o(x)) + A2 1g2(x + B(x)) - 1


where

i=1
We write this system, in turn, in the operator form
(6.16) cp(x) _ y(x),
where
'Y(x) = (x), 1 (x)).
If So E B, then the quantities
1
17,
i -
(rnl = ma.sr cim
i I
1p-i+c .

are finite for l = 0, ... , p. If e is sufficiently small, then


Di (T So) < qDi (So) + Ri (Do (So) , ... , Di

where q < 1, and Ri is a function determined by f, g, and 0. In particular,


Ro = Do (y)
Let
co=Do1_q,
('Y)

and let the numbers c1, ... , c be chosen by induction so that

Ci =
Ri (Co, ... , ci-1)
1-q ( --
1<l<
p)

Then the ball


S(c) = {cp E ci, 0 < l < p}
I

is invariant under T. We choose a metric in this ball so that T is a contraction.


With this purpose we observe that if So, b E S(c), then
(6.17) Dl (TSo - T'ib) < qDi (Sp + b) + bi max D( -
j<l-1
where bi = b1(). In particular, bo = 0.
Let b = maxi> 1 b1. We fix a number q E (q, 1) and define the numbers

a=1, a= b q>i+1
q
a
176 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

For p,'/ E S(c) let


P
d() = -
=o
Then by the choice of the numbers aj we get from (6.17) that
d(Tp,Tb) qd(p,b).
Consequently, (6.16) has a solution cp E S(c).
To prove uniqueness in the whole of B we set

where cp is another solution. Then by the equation,


M<qM, q<1,
that is, M = 0.
The existence (and uniqueness in a suitable class) of solutions of the equations
(6.11)-(6.14) is proved similarly.

This proves Theorem 6.3.

6.4. The Sternberg-Chern theorem. We recall that this is a theorem


about conjugacy of C°°-diffeomorphisms.
THEOREM 6.4. If the formal Taylor series of C°°-diffeomorphisms F and G
at a hyperbolic fixed point are conjugate, then F and G are locally conjugate in the
class C°°.
This theorem has already been proved for the case of a node in §6.2. It remains
only for us to treat the case of a saddle point. For this we must prove the existence
of a local C°°-solution Sp of (6.9) such that Sp(O) = gyp' (0) = 0.
We can assume from the start that the right-hand side h = f - g of (6.9) is
flat at zero. Beginning with this, we prove the existence of a local solution Sp E C°°
that is flat at zero.
As before, we assume that the coordinate axes are invariant with respect to F.
We note first that each mapping h E C°° that is flat at zero can be written as
a sum
h(x) = h+(x) + h_(x), h E C°°,
where
az+j h+ (x) az+ h_ (x)
8 z8 j - °' 8 z8 j - °To

prove that there is such a decomposition we construct a mapping h+ such that


82+ h+ (x)
_ °' a za 77=0 9i971'
for all i and j. The mapping can be constructed in this way by generalizing the
proof of Borel's lemma to the case when the derivatives of all orders are given on the
"coordinate cross" ll U l2. However, we can invoke the general Whitney extension
theorem (see [59]). After constructing the term h+ it remains to set
h_ (x) = h(x) - h+(x).
6. SMOOTH NORMAL FORMS 177

Next, to prove that (6.9) is solvable it now suffices to prove that the following
two equations are solvable:
o+(Fx) - AP(x) - [9(x + (P+(x)) - 9(x)] =
(6.1R'
Ao_(x) - g(x + o(x) + o_(x)) + g(x + o(x)) = h_(x).
Then the sum cp = p+ + cp- is the desired solution.
As before, assume that the mappings f, g, and hf are given in the whole plane
and are bounded, together with their derivatives of all orders. Also as before, it is
assumed that
IIf'(x)ll e, g'(x) e.

LEMMA 6.2. There is an e = e(A) such that each of the equations in (6.18)
has in the plane a unique solution satisfying for all r = 0, 1, ... , and v = 0, 1, .. .
the conditions

and
)
Iko_ (x)II <cr,l77ly,
respectively.
PROOF. Let us consider the first of our pair of equations. We write it in the
form
o(x) = A1o(Fx) - A1[g(x + o+(x)) - g(x)] - A1h(x).
For each r = 0, 1, ... we choose a number v(r) large enough that

Let
DTv((p) - sup
x
In this case if e is sufficiently small, then for v > v(r)
DT,v(T(p+) < gDT,v((p+) + RT,v(Dr_l,v((p+) ... Do,v(SP+))'
Here, as before, T denotes the operator on the right-hand side of the equation, and
RT,v is a function determined by the given mappings. In particular,

The factor q is less than 1.


Suppose now that
C = {Cr,v}v>v(r)
is the infinite collection of numbers determined by
c = Do,(-A'h+) 1

g
1
CT,v =
1-q --Rr,v(CT-1,v,... ,CO,)
(r 1).

Then the compact convex set


S(c) _ {gyp DT,v ((p) < Cr,v } (r=0,1,..., v> v(r) )
is invariant under T. Consequently, T has a fixed point Sp+ in S().
178 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

We prove uniqueness. Let SP+ be another solution satisfying our conditions.


We set
M = Do,U0(p - SP), vo = v(0).
Then by the equation,
M< M, q<1,
that is, M = 0.
The assertions of the lemma for the second of the equations in (6.18) are proved
similarly (just make the substitution x H F-1 x) .
The lemma is proved, and with it the Sternberg-Chern theorem.

We remark that a metric in which T is a contraction can be introduced in the


compact set S().
We remark also that Lemma 6.2 also finishes the proof of Theorem 6.1 for the
case C°°.
6.5. The smoothness class as an obstacle to smooth normaliza-
tion. By Theorem 1.1, a diffeomorphism of the line of class CT , r > 2, can be
linearized in the same smoothness class at a hyperbolic fixed point. The example
after the formulation of the theorem shows that this is no longer so for r = 1. Thus,
there exist one-dimensional C1-diffeomorphisms that are not smoothly conjugate
to a diffeomorphism of higher smoothness class (not even C1'« for an a > 0) in
a neighborhood of a hyperbolic fixed point. The global variant of this effect is
considerably stronger: in §5 of Chapter 5 we shall see that for any r < oo there
exist structurally stable CT -diffeomorphisms on the circle that are not smoothly
(not even in the class C1) conjugate to CT In some sense the
4-diffeomorphisms.

smoothness class is an invariant of smooth conjugacy.


Even in the local situation there is a greater diversity on the plane. In the
case of R2 we now show that for any r < oo there exist CT -diffeomorphisms that
are not conjugate in the class CT to CT+1-diffeomorphisms in a neighborhood of a
hyperbolic fixed point.
To construct an appropriate example we set
T
(r 0), h(0) =0.
Then h E CT is an r-flat function. Let us consider the diffeomorphism
H(x) _ (AT ( + h(i)), ark), 0 < A < 1.
We show that H not only cannot be linearized in the class CT but cannot even
have an invariant curve M2 of class C. Thus, it follows from Theorem 6.2 (the
case rr(A) = 1 of the corollary) that H is not locally conjugate in CT to a CT,«-
diffeomorphism for a > 0.
An invariant curve
M2={=y(i)}
must satisfy the equation
Ay(Ai) - y(i) =
Assume now that ry E C. Then y(i) =O(AT), and the function

al) _
7. LOCAL NORMAL FORMS OF TWO-DIMENSIONAL FLOWS 179

must have limit equal to -y(T) (0) as i -* 0. At the same time,


i
b(ay) - lnlil'
Integrating, we get that
n-i
a(a) - a(i) _ i s In A-I- In ICI

The left-hand side of the last equality is bounded, while the right-hand side, being
a sum of the terms of a harmonic series, is not. Consequently, -y C' .
More subtle examples are given in [108]. In particular, there are examples
showing that the estimate of the smoothness class for normalization of a saddle
cannot be improved.

§7. Local normal forms of two-dimensional flows


The results on normal forms for diffeomorphisms carry over to flows.
7.1. Topological and C1-linearization. Let us consider the equation
(7.1) x = Ax + h(x), h(0) = h'(0) = 0,
where x E R2, h e C1, and A : R2 -* R2 is a linear operator whose spectrum
does not intersect the imaginary axis. By analogy with §2.1, we define the number
0 = 0(A). Namely, if the spectrum of A is not separated by the imaginary axis (the
case of a node), that is, if either
max(Re vl, Re v2) <0
or
min (Re vl , Re v2) > 0
(here v2 E spec A), then
(Re vl Re v21
8(A) =min
Re v2' Re vl J
But if the numbers vl and 112 lie on different sides of the imaginary axis, then
8(A) = 1.
THEOREM 7.1. For any a < 8(A) the system (7.1) is conjugate in the class
C°' to a linear system.
The proof goes according to the same scheme as in the one-dimensional case
with the use of Lemma 2.1.
THEOREM 7.2. Let h E C1, a> 1-0(A). Then the system (7.1) is conjugate
in the class C1 to a linear system.
Since a normal form is linear, and the smoothness class is > 1, we can make
direct use of the Sternberg device mentioned in §1.4 for the proof. Namely, let
E C1 be a local diffeomorphism linearizing the mapping F = FtIt=1, where Ft
180 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

is the flow of the system (7.1). Such a diffeomorphism exists in view of Theorem
4.1. Then the local diffeomorphism

W(x) =
f 1

dt

l inearizes the field (7.1).


7.2. Invariant curves of a flow. As in the case of diffeomorphisms, invariant
curves of the system (7.1) whose tangents coincide with the coordinate axes are
described by equations
M1 ={xIi='y1()}, M2={xI ='y2(r)},
where the 'y2 are smooth functions with 'y2 (0) = 'y2 (0) = 0. The smoothness class of
the curves coincides with the smoothness class of the function We assume that
r + a> 1 and that v1 <v2 <0 in the case of a node.
THEOREM 7.3. In the case of a node, a system (7.1) of class C''« has a unique
invariant manifold M1 of class C''«, as well as an invariant manifold M2 of class
for
q+,Q < min r+a,vl-
112

In the case of a saddle, the system has unique local invariant curves M1 and M2 of
class CT, «

PROOF. As before, let Ft be the flow of the system, and let F = FtIt=1 Let
MZ be the invariant curves for F; their existence is ensured by Theorem 3.1. Let
Mt = Ft(M) (i = 1, 2).
For each t the curves Mi are invariant under F, and the coordinate axes l2 are
tangent at zero. Therefore, Mi = MZ in the cases when Theorem 3.1 ensures
uniqueness; that is, the MZ are invariant for the system (7.1). It thus remains
to prove the existence of M2 in the case of a node, when there is certainly not
uniqueness. We indicate one method with use of the uniqueness in Lemma 1.4.
Recall that in our notation ill < v2 < 0. The function 'y2 giving the curve M2
satisfies the differential equation
y2 (1 (v2 1 + h2 (y2 (i), i)) = 111'y2(11) + h1(y2 (i ), i) .
Here, as before, x = (, E R2, and h = (h1, h2). Passing to the flow, we get the
equation
'1 t ft
(7.2) 'y2(7) = e-11ty2 (e12ti + ft (y2 (17), 77)) - e_ (y2 (r/), r/),
where
(7.3) Ft(x) _ (evit + ft (x) ft (x))
is the flow of the system. The equation (7.2) must be satisfied for small t. We
extend the residual h to the whole plane in such a way that it is bounded together
with all its derivatives, and the first-order derivatives of the functions ft and ft
are sufficiently small (see Lemma 1.4) over the whole plane (with respect to x) for
Iti < 1. By Lemma 1.4, (7.2) then has a solution 'y2 E for t = -1 that is
7. LOCAL NORMAL FORMS OF TWO-DIMENSIONAL FLOWS 181

defined on the whole axis and unique in the class of bounded functions. For a fixed
t we consider the curve
Ft(y2(r),r) ={=
This curve is invariant with respect to the diffeomorphism F-1 = Ft It=-i. Conse-
quently, rye satisfies the equation

= +f?i(7z(7r),1r)) -
its definition, rye is bounded. By uniqueness,
')4(77) = 'y2(11) (tI 1).
Consequently, Mz = { = is the desired curve.

7.3. Smooth normal forms. The normal forms I'-III' in §5.3 satisfy the
relation
(7.4) G(x) = Ax + g(x), g'(x)A*x = A*g(x).
By analogy with the discrete case, each C°°-flow satisfying (7.4) will be called a
normal form. As in the case of a diffeomorphism, the equation (7.1) can be reduced
by a C°°-transformation to the form
(7.5) x =G(am) + h(am),

where G is a normal form, and h E CTS`" is an r-flat residual.


THEOREM 7.4. Suppose that the origin of coordinates is a node for the system
(7.5). If a> 1 - r9(A), then the system (7.5) is locally conjugate in the class C''
to the normal form

PROOF. Assume first that


r8(A) > 1.
Again passing to the flows Ft and Gt and invoking Theorem 6.2, we get a
diffeomorphism
(7.6) (x) = x + (x), p(x) C(II xII T+«),
which conjugates Ft and Gt for t = 1. Let
S = G-S o o FS.

For each s the diffeomorphism 4 also has the form (7.6) and conjugates the dif-
feomorphisms Ft and Gt for t = 1. However, as noted in the proof of Theorem 6.2,
a local diffeomorphism with the indicated properties is unique. Consequently,

If r8(A) < 1, then the normal form is linear, and we can use the Sternberg
device of passing from diffeomorphisms to flows.
Finally, there is also an analogue of Theorem 6.3 for a saddle point. We for-
mulate it also for r = oo, so that the formulation will include the Sternberg-Chern
theorem for flows.
182 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

THEOREM 7.5. Suppose that either r = o0 or that r < oo and a > 0, and
assume that the origin of coordinates is a saddle singular point for the system (7.5)
of class CT'". Then (7.5) is conjugate in the class CP to a normal form, p = [r/2].
In the case r < oo it is possible to take a = 0 if it is required in addition that
vj7p(vi -I- v2) (j1,2).
PROOF. It suffices to prove the existence of a local CP-solution cp of
p (x)(Ax + g(am) + h(am)) - [g(x + gyp) -g(am)] =h(am)

that is p-flat at the origin. We can assume that the coordinate axes are invariant
for the original system. Letting Tcp be the operator on the left-hand side of this
equation, we reduce solvability of the equation
TSp=h
to solvability of the equations
N1TSp1 = h1
NZT(Sp1 + ... + Sp2) = h2 (2< i< 2p+3)
with respect to the mappings SPz E £ (see the proof of Theorem 6.3).
The proof of solvability of (7.7) can be carried out independently, by "func-
tional" methods. However, it is also possible to employ the arguments already
used for diffeomorphisms, with uniqueness of the solutions taken into account. For
instance, let us consider (7.7) for i = 2p + 3.
We write it in the form
cp (x)(Ax -f- g(x) + h(x)) = Acp(x) + g(x -I- 9(x) + p(x)) - g(x + p(x)) + ry(x),
where ry E ,G2P+3, and

0(x) = p1(x) + + (P2p+2(X).

We consider the system


=Ax+g(x)+h(x),
y = Ay -f- g(x + 8(x) + y) - g(x+ 9(x)) + ry(e).
Its flow has the form
Vt(x,y) _ (Ft(x),Ht(x,y))
Further,
t
Ht (x, 0) = ett-SSA-y(Fsx)ds E £2p+3.
0
Moreover, we have the formula
(7.8) Ht+s (x, y) = Ht (Fsx, Hs (x, y))
It suffices to prove the existence of a p-flat Cr-mapping p such that
So(Ftx) =
(7.9) Ht(x, So(x))
7. LOCAL NORMAL FORMS OF TWO-DIMENSIONAL FLOWS 183

for small t. We can assume that the mappings g, h, and -y are extended to the
whole plane in such a way that for small t the residuals
ft (x) = Ftx - eAtx,
t
9t(x, y) = Ht(x, y) - etAy = f eat_[9(F3x -I- 9(FSx) + y) - 9(Fsx + 9(Fsx)] ds
0

satisfy the conditions of Lemma 6.1 for the equation (7.9) with t = 1,

(Fx) = +f ell-s)Ag(Fsx + 9(Fsx) + (x)) ds


1
e(1-s)Ag(Fsx + 9(Fsx)) ds + Hl (x, 0).

Unlike in Lemma 6.1, the nonlinearities in Sp are not essential to the equation here:
if
IIf'(x)II e g'(x) e
and e is sufficiently small, then the last equation has a unique solution p E B
defined for all x E IIBz. We now let

(pt (x) =
Ht (F-tx,
Sp(F-tx))
Then Pt E B and, moreover,
Spt(Fx) = Ht (F1-tx, Sp(F1-tx)) = Ht (F'1-tx, H1(F-tx, (p(F-tx)))
= Ht+1(F-tx, (p(F-tx))
= H1(x, Ht(F-tx, (p(F-tx)) = H1(x, Spt(x)))
This long chain of formulas uses the flow condition (7.8) and the equality
Sp(Fx) = H1(x, Sp(x)) (F = Ft It=1 , H1 = Ht It=1)
which means that p is a solution of (7.9) for t = 1. Finally, we get that
Spt (Fx) = R 1(x, (pt (x))
Since cot E B, it follows from uniqueness that cot = p for all small t. Consequently,
Sp satisfies (7.9) for small t, which is what was required.
The case r = oo (the Sternberg-Chern theorem) can be treated similarly, with
the uniqueness in Lemma 6.2 taken into account.

7.4. The correspondence mapping at a saddle point. Suppose that the


origin of coordinates is a saddle singular point for the system (7.1). We fix numbers
e > 0 and 6> 0 and consider the two "sections"
= {x = (e, i)}, 2 = {x = (, )}.
Let us take a point (e, i) E El with r > 0. The integral curve of our system passing
through the point (e, intersects the surface element E2 at some point
=
The smooth function f arising in this way is called the Poincare mapping or cor-
respondence mapping of the system (7.1) at the point x = 0.
184 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

EXAMPLE. Suppose that the system (7.1) is linear:


1=v11, 1i=v217, 111 <0<112.

Its flow has force


Ft(, rl) _
The curve Ft(, li) intersects the surface element 2 at the time

t0 =t0(i1) = 1 in-.
v2

Therefore,
f (i) - l7V ' E v-
Obviously, the function f depends on the choice of e and b. However, in
different dynamical problems (see, for example, [48], and also §2 in Chapter 3) it
is only the power behavior of f and its derivatives as li - 0 that is important.
This behavior is independent of the choice of e and b and the local coordinate
system. Sometimes the coordinate system is required to have sufficiently high (> 2)
smoothness. Nevertheless, to study the behavior of the correspondence mapping
we can use a normal form (which may be nonlinear).
If the original function is sufficiently smooth, then it can be reduced by a C2-
transformation to a linear form, or to the normal form

or to the normal form

or to the normal form

The behavior of the Poincare mapping for these nonlinear normal forms was
studied in [48], and will be used by us in §2 of Chapter 7.

§8. Normal forms in a neighborhood of an


equilibrium state (survey and comments)
We present a summary of the preceding sections.
1. In a neighborhood of a hyperbolic fixed point any (C1-) diffeomorphism
(flow) is conjugate in the class Co,« to a linear diffeomorphism (flow) for any a <
T(A) (see Theorem 2.1). This assertion refines the Grobman-Hartman theorem
ensuring topological linearization, and carries over to the multidimensional case for
an appropriate definition of T (A) .
2. In a neighborhood of a hyperbolic point any diffeomorphism (flow) in R2 of
class C1,« with a> 1- T(A) is conjugate in the class C1 to a linear diffeomorphism
(flow). This is a refinement of the theorem of Hartman [74]: in a neighborhood
of a hyperbolic point any C2-diffeomorphism of the plane is conjugate in the class
C1 to a linear diffeomorphism. The Hartman theorem carries over to the mul-
tidimensional case only for a node (see [74]). However, already in R3 there are
examples (see [74]) of analytic local diffeomorphisms that cannot be smoothly lin-
earized in a neighborhood of a saddle. A condition is given in [31] that ensures
8. NORMAL FORMS IN A NEIGHBORHOOD OF AN EQUILIBRIUM STATE 185

smooth linearization. Namely, suppose that the eigenvalues ) , ... , an of a linear


transformation A satisfy the condition
'ail IAI (IAZI < 1 < IThen

the C2-diffeomorphism F(x) = Ax +... is locally conjugate in the class C1


to a linear diffeomorphism.
3. A diffeomorphism of the plane R2 of class C''' « , a> 1 - rT (A) , in a neigh-
borhood of a node can be reduced by a C'-transformation either to linear form or
to the normal form
G(x) = (app + rIp, Ai), p < r.
Consequently, the system of equations can be reduced either to linear form or to
the form
=p +i , =m pEZ+, p>2.
For r < oo and a > 0 the diffeomorphism can be reduced in a neighborhood of a
saddle point by a Cr-transformation with p = [r/2] to the polynomial normal form
Al + cj (61r )i,
G `x) = m sj l
) 7+ j=1 d j (6 r) .
This normal form corresponds to the resonance
i a2 = 1,
where 1 and s are relatively prime, and
r-1
_ l+s.

In particular, if l + s > r -1, that is, there are only "distant" resonances, then the
normal form is linear.
Sharp estimates are given in [108] (with appropriate counterexamples) for the
smoothness class of the reducing transformation in dependence on the value of the
In a2
ratio r = However, the estimate [r/2] cannot be improved in the set
lna
I I .

of all linear approximations. For linear normal forms there is a multidimensional


analogue of this estimate: a C'-diffeomorphism
F(x) = Ax + o(IIxIIr), x E IfBn,

that is hyperbolic at the origin is locally conjugate in the class Cp to a linear


diffeomorphism, p = [!i] (see [31]).
4. In general a C°°-diffeomorphism cannot be reduced by a C°°-transformation
to a polynomial mapping in a neighborhood of a hyperbolic point. However, this
can be done by a transformation of any finite smoothness class. For example, by
Theorem 6.3, a C°°-diffeomorphism can be reduced in a neighborhood of a saddle
point by a C1-transformation to the form
A1S + S . c1 (SCrS)i,

1 A2i 1d7(Slis)i,
where

m<_ 2k
l+s
-1
albs=1
2
1
186 4. LOCAL STRUCTURE OF DYNAMICAL SYSTEMS

with 1, s E Z+ relatively prime. However, the degrees of the polynomials increase


in general as the smoothness of the reducing transformation increases.
The following question arises in this connection. Fix numbers d,k E Z. What
conditions must be satisfied by a local diffeomorphism
F: (RTh,O) -* (RTh,O)

in order to be reducible by a Ck-transformation to a polynomial of degree < d?


In view of the Sternberg-Chern theorem the answer must be formulated for
hyperbolic C°°-diffeomorphisms in terms of the properties of the coefficients of the
normal form of the Taylor series /3(F). For instance, in the presence of the resonance
(8.1) C-linearization is possible if in the normal form III we have the condition
1.
(8.2) '>2k
The case d = 1, that is, the possibility of Ck-linearization, seems most interesting.
It has been thoroughly studied by Samovol [72] (see also [113]) for arbitrary values
of k and arbitrary dimension n.
5. Nonhyperbolic equilibrium states are also of interest in a whole series of
problems, in particular, for bifurcation theory. Here the formal theory of normal
forms becomes richer in content (see [31]). In the one-dimensional case all C°°-
diffeomorphisms can be reduced in a neighborhood of a fixed point by a local
C°°-transformation to one of the normal forms
G(x) = :Ex + xs1+1 + ax2St+1, 1 E Z+,

up to a set of infinite codimension, where s = 1 or 2, depending on the sign of


the first term. This normal form was found by Takens in [109], where an orbital
normal form was determined for "almost all" systems on the plane. For C°°-
diffeomorphisms and vector fields in Rn satisfying the so-called single-resonance
condition (a multidimensional analogue of a resonance of the form (8.1)) a classifi-
cation and normal forms were obtained in [34] and [35].
6. Finally, analytic dynamical systems (real and complex) are a classical sub-
ject of investigation. The first results are due to Poincare and laid the foundation
of the theory of normal forms. He proved the theorem on analytic linearization of
a node in the absence of resonances. Then Dulac proved the possibility of analytic
reduction to normal form in a neighborhood of a node. At a singular point that is
not a node, analytic normalization is hindered by so-called small denominators: the
differences A3 - A' ... A, where the ai are the eigenvalues of the linear approxima-
tion, appear in the denominators of the formal transformations and can interfere
with convergence when they are "pathologically" small. In the absence of reso-
nances small denominators are the only obstacle to linearization. However, finding
precise conditions ensuring convergence is a difficult and substantive problem. The
first step was made by Siegel. He proved that a one-dimensional complex-analytic
diffeomorphism
F(z) = Az + f (z)
can be linearized by a locally analytic transformation under the arithmetic condition

(8.3) A
- ii cq', q E Z+.
8. NORMAL FORMS IN A NEIGHBORHOOD OF AN EQUILIBRIUM STATE 187

with some c, v > 0. The condition (8.3) holds for almost all numbers A with
respect to Lebesgue measure. This theorem was carried over by Siegel [45] to the
multidimensional case. However, all the fundamental difficulties already appear in
the one-dimensional problem. Siegel used the direct majorant method, and that
involved a considerable quantity of technical details. Essential progress became
possible with the emergence of the KAM method. By using this method Bryuno
[38] replaced the condition (8.3) by a weaker condition. Recently Yoccoz [112]
determined that the Bryuno condition is necessary: if this condition fails to hold,
then there is a residual f such that the diffeomorphism F cannot be analytically
linearized.
In the problem of analytic normalization in the presence of resonances other ob-
stacles arise in addition to small denominators. This problem has been thoroughly
studied by Bryuno [38]. He determined conditions on a formal normal form which
together with "nonsmallness" of the denominators ensure an analytic normaliza-
tion. These conditions are very stringent. For example, even the one-dimensional
complex-analytic mappings
F(z) = z + o(z), z E C,
cannot as a rule be reduced to their formal normal form (which here has the form
G(z) = z + zq + az2q+1, q > 2,
with some q E Z+ and a E C). Moreover, the analytic classification of such
mappings leads to functional invariants (the Ecalle-Voronin moduli; see [41]).
CHAPTER 5

Transformations of the Circle


The investigation of flows on two-dimensional manifolds leads in a natural way
to the study of transformations of the circle, because a flow with a closed transversal
induces on it a Poincare mapping (a transformation of the closed transversal into
itself). Many properties of a flow can be extracted from the properties of the
Poincare mapping on a global section of the flow (that is, a closed transversal that
intersects any regular trajectory of the flow).
The results in this chapter are used in the next chapter in studying topological
equivalence problems and the topological classification of transitive flows and Den-
joy flows on the torus, as well as in Chapter 7 in studying the interrelation between
smoothness properties and topological properties of flows.
The study of transformations of the circle is also of independent interest. Sev-
eral directions of investigation have a finished form and serve as prototypes of
corresponding results in the multidimensional theory of dynamical systems.
We present rigorous results mainly for transformations of the circle of degree
1 (in particular, for homeomorphisms and diffeomorphisms) with nondecreasing
covering mapping. However, for completeness we mention (in the form of remarks)
results of interest in our view on transformations of the circle with other properties.

§ 1. The Poincare rotation number


In 1885 Poincare [68] introduced for homeomorphisms of the circle the concept
of the rotation number, which has turned out to be very fruitful in the theory of
dynamical systems. The meaning of the rotation number of a transformation of
the circle is that for a particular point of the circle the rotation number gives an
asymptotic indication (that is, in the limit) of the average rotation of the point
along the circle under the action of the transformation. For a large class of trans-
formations the rotation number does not depend on the point (in particular, for
homeomorphisms) and characterizes the transformation itself.
In this section we introduce the rotation number for a transformation of the
circle of degree 1 with a monotonically nondecreasing covering mapping, and we
prove some properties of the rotation number.
_ 1.1. Definitions and notation. Denote by P(R) the set of transformations
f : R --R satisfying the conditions:
1) f (x + 1) = f (x) + 1, x E Ilk; _ _
2) f is monotonically nondecreasing, that is, f (xl) < f (x2) if xl <X2.
These conditions immediately yield the properties:
fk def
3) the transformation = 10.. .o f is monotonically nondecreasing for k E N;
4) f (x+r)=f (x)+r,XER,kEN,rEZ.
189
190 5. TRANSFORMATIONS OF THE CIRCLE

LEMMA 1.1. Let f E P(1[8), and suppose that r < f k(xo) - xo <r +1 for some
k E N, r E 7G, and xo E I[8. If f(x)-xZforallxER 1, the n r < f (x)-x<r+1
for all xEIIB.
k
PROOF. Let D def {x E Ilk : r < f (x) -x}. By the condition 1) in the definition
of the set P(R), x0 + n E D for all n E Z.
Let x1 E D. We show that [x1, oo) E D. Indeed, since r < t (xl) - xl, the
k k c
condition 3) gives us that x + r < f (x1) < f (x) for all xl < x < f (xl) -r. Since
f (x) - x 7L, we have the strict inequality r < f (x) - x, that is, [x1, 1c(xi) - r] C
k

D.
Let {x}r be a monotonically increasing sequence of points xn E D converging
to a point x E R. If we assume that x D, that is, r > 7k(x) then we -
get that
Jk(5) < f k <x n + r for the points xn with 7k(x) - r <x n <x,
and this contradicts the inclusion xn E IL Consequently, x E D, and hence
[xi,oo) C D.
The equality D = R follows from this and the inclusion x0 +n E D, n E Z; that
k k
is, r < f (x) - x for all x E R. It can be proved similarly that f (x) - x <r + 1
forallxER.
THEOREM 1.1. For any transformation f E P(R) the limit

lim
T(x) def
=a
n--) oo n
exists and does not depend on the choice of the point x E III. If J'(xo) = xo + r for
some xo E R, k E N, and r E Z, then the number a is rational and equal to r/k.
PROOF. We first assume that the limit (1.1) exists for some point xo, and we
take an arbitrary point x E R. Let s E Z be such that xo + s < x <x0 + s + 1.
By the properties 3) and 4), 7(xo + s) = T(xo) + s < T(x) < j7(xo + s + 1) =
T(xo) + s + 1. From this,
T(x) -T(xo) as n -* oo,
n n

and hence the limit (1.1) exists for all x E R and does not depend on x.
We now prove that the limit (1.1) exists for some point.
k
Let f (xo) = x0 + r for some x0 E R, k E N, and r E 7L. Any positive integer
n can be written in the form n = qk + s, where 0 < s <k. Then by the property
4) and the equality

f 4 (xO) = fk o... o lk(xo) = x0 + qr,


q
JS qk
we have that r(xo) = of (x0) = IS(xo + qr) = 7S(xo) + qr, which implies that
IS(xo)
+ qr _ IS(xo) + qr _ IS(xo) + r r
(q,n - oo)
n n qk +s n k+ s/q k
Thus, in this case the limit (1.1) exists and is equal to r / k E Q.
1. THE POINCARE ROTATION NUMBER 191

Assumenowthat fk(x)-xZforallxERandkEN. ThenforfixedxER


k
and k E N there exists a number qk (x) E Z such that qk (x) < f (x) - x < qk (x) + 1.
def
According to Lemma 1.1, qk (x) q does not depend on the point x E R, that is,
(1.2) q<lk(x)_x <q+1
for any x E R. We apply (1.2) to the points x = 0,1 (0),..
k
. , f k('i-1) (0):

q<f (0)-f (0)<q+1


q < f k(0)-fk(n-1)(0)<q+1.
Adding these inequalities, we get that
qn<f (0)<(q+1)n,
or

q f k(0) q 1
n < nk <k+k
In view of (1.2) for x = 0, we have that

q fk(0) q 1

This and (1.3) give us that


7flk()
fr(o) 1

nk k

Since k and n are arbitrary, the above computations can be repeated with k and n
interchanged. Then we get that
7(o) _ f nk (0) 1
n nl < n
Combining the last two inequalities, we get that
jfl() fr(o)
n k

that is, the sequence {fTh(0)/n}0 satisfies the Cauchy condition. Therefore, the
limit (1.1) exists for x = 0, and hence for all x E III. 0

DEFINITION. Let f E P(R). The number


n
lim I (x) def -
= rot (f) ,
x E R,
n--) oon
is called the rotation number of the transformation 7.
192 5. TRANSFORMATIONS OF THE CIRCLE

LEMMA 1.2. For f E P(R) and for fixed k E N and r E 7L the transformation
fk(x)
71(x) = -I- r belongs to the set P(R) and has rotation number rot(f1) _
rot(fk +r) = krot(f) +r.
PROOF.

-
rot (f 1) = lim
f (x) = lim f k(x) - nr
1

n-- oo n n-- oo n

= k lim
Tk(x) -
+ r = k rot (f) -}- r.

LEMtvta, 1.3. Let f E P(R).


1) Ifr< fi(x)-x<r+1 andxEIIB, then

k <rot(f)<rk1
2) If f is continuous and r/k < rot(f) < (r + 1)/k, then there exists a number
il > 0 such that the inequalities

r+r <7k(x)_x <r+1 -


hold for all x ER.
k
PROOF. 1) Since the transformation f is nondecreasing and since f (x + r) =
7k(x)+r for x E R and r E Z, it follows from the inequalities r+x < ik(x) < r+1+x
that nr -}- x < f k(x) < n(r + 1) + x. Consequently,

r
- < lim nrn+ x < lim
- < lim
r(x) = rot(f) n (r + 1) -}- x
=
r+1
n n

We prove 2). According to the (proven) assertion 1), r < 7k(x) - x for all
k
x E III. It follows from Theorem 1.1 that if r = f (x) - x for some x E Ilk, then
rot(f) = r/k. Therefore, r <7k(x) - x for all x E R.
The equality 7'(x -I- 1) = 7'(x) + 1 implies that the function t(x) - x is
periodic with period 1. Consequently, there exists a number i > 0 such that
r+r/1 < fk(x)-xforallxER.
It can be proved similarly that there is an 772 > 0 such that 7k(x)_x <r+1-772.
The required 77 is min{r/l, 7/2}.

REMARK. The assertion 2) in Lemma 1.3 is not true in general for discontinu-
ous transformations in the class P(R) (Figure 5.1). This assertion is used in what
follows for proving that the rotation number depends continuously on a parameter.
It can be shown that under the additive introduction of a parameter (see Theorem
1.2) the rotation number is a discontinuous function in general for a discontinuous
transformation in P(R).
1. THE POINCARE ROTATION NUMBER 193

x x
FIGURE 5.1

The group Z of integers can be identified in a natural way with the group of
mappings of Ilk of the form x '-p x + n, x E IIS, n E Z. The quotient space Ilk/7L is
homeomorphic to the circle Si, which can be represented as the closed interval [0, 1]
with endpoints identified. The natural projection it : Ilk - Ilk/7L S1 is a universal
covering.
By the condition 1) in the definition of the set P(R), each transformation
f E P(R) is a covering for some transformation f : 51 51, that is,
f f.
Denote by P(S1) the set of transformations of the circle that have covering
transformations in P(R).
LEMMA 1.4. Let f E P(S1).
1) If f e P(IR) is a covering transformation for f, then so is f + n, n E 7L.
2) If 11,12 _E P (]R) are covering transformations for f, then there exists a
k E Z such that f2(x) = 71(x) +k for all x E R.
PROOF. The assertion 1) follows from the definition of the covering it because
itof=iro(f+n).
We prove 2). By assumption, for each x E Ilk there exists an integer k (x)
such that f2(x) - f 1(x) = k (x) . Since fi (x + 1) = f i (x) + 1 and since f i is
monotonically nondecreasing (i = 1, 2), f - 7(x2)I < 1 for sufficiently close
points x1, x2 E I1t This implies that the function k(x) is locally constant. Then
k (x) = k = const because k (s) is in Z.

DEFINITION. Let f E P(S1). The number

lim
f (x) (mod 1)def
= rot(f ), x E Ilk,
n
where f E P(R) is a covering transformation for f, is called the rotation number of
the transformation f.
By Lemma 1.2 and 1.4, the definition of the rotation number of a transformation
f E P(S1) does not depend on the choice of the covering f E P(R).
Remark 1. The rotation set of a continuous transformation of de-
gree 1. We recall that a transformation 1:51 - S1 is a transformation of degree
k if there exists a covering transformation f : Ilk - Ilk for f such that f (x + 1) =
194 5. TRANSFORMATIONS OF THE CIRCLE

7(x) + k. Thus, all the transformations in the set P(S1) are transformations of
degree 1.
Let f : R - R be a covering for some transformation f : S1 -* Si of degree 1.
If the condition 2) in the definition of P(R) does not hold for f, that is, f is not
monotonically nondecreasing, then the limit (1.1) can fail to exist in general or can
depend on the point x. Newhouse, Palis, and Takens (Stable families of dynamical
systems. I: Diffeomorphisms, preprint, I.M.P.A., Pio, Brasil, 1978) proposed a
generalization of the concept of the rotation number to continuous mappings (not
necessarily one-to-one) of the circle of degree 1. For a given x E R the rotation
number is defined to be the limit superior of the sequence {7Th(x)/n}°:
f mi(x)
rot(f, x) = lim n
(the largest limit of the convergent subsequences). The rotation set of f is rot(f) =
{rot (f , x) : x E R}. Since a covering of a transformation f is determined up to an
integer addend, the rotation set of f, defined to be rot (f) = rot (f) (mod 1), does
not depend on the covering f.
The following two lemmas were proved by R. Ito in the paper, Rotation sets
are closed (Math. Proc. Cambridge Philos. Soc. 89 (1981), no. 1, 107-111).
LEMMA. The rotation set rot(f) is bounded.
PROOF. Since 7(x) is a covering for a continuous transformation of the circle of
degree 1, 7(x) - x is a periodic continuous function of period 1. Therefore, setting
L aef max f - x l , we have that
n
.f(.fi-'(x)) _ 7'(x) < nL,
i=1
and from this
-L < lim
f (x) - x -
= rot (f , x) < L.
n

LEMMA. If A, ,c E rot(f) and A < p/q < , where p E Z and q E N, then there
exists an x E R such that f q(x) = x -}- p (that is, the transformation f : S1 - S1
has a periodic point ir(x) of period q with rotation number p/q (mod 1)).
PROOF. Assume the contrary. Then since J'(x) - x is continuous, either
r(x) - x < p for all x E R or f ' (x) - x > p for all x E R. For definiteness
assume the former. Since J'(x) - x is periodic and continuous, there exists an
e > 0 such that 7'(x) - x <p - e. Then 77(x) - x < n(p - e), and hence
rot (f , x) < (p - e) /q < p/q for all x E R, which contradicts our assumption.

With the help of these lemmas, Ito proved the following theorem.
THEOREM (R. Ito). Let f : R - R be a covering for a continuous transforma-
tion of the circle of degree 1. Then the rotation set off is either a point or a closed
bounded interval.
In the paper, Rotation intervals of endomorphisms of the circle (Ergodic Theory
Dynamical Systems 4 (1984), 493-498), Bamon, Malta, Pacifico, and Takens defined
1. THE POINCARE ROTATION NUMBER 195

the rotation set rot (f, x) of a point x with respect to a continuous transformation
f : Ilk -* R covering a transformation of the circle of degree 1. The set rot (J, x) was
defined to be the union of all the limit points of the sequence {fTh(x)/n}r. In that
paper it was proved that rot (f , x) is a closed subinterval of the interval rot (f) for
any x E R, and for a given [a, ,3] C rot(f) (a < ,3) there is a point x E R such that
rot(f, x) = [a,/3].
Remark 2. The rotation set of a topological Markov chain. Following
a paper of V. M. Alekseev on symbolic dynamics (Eleventh Mathematical School,
Kiev, 1976), we consider a topological Markov chain (TMC) (L, H, o) with a Haus-
dorff state space L, a transition matrix H that is a closed subset of L x L, and
the left-shift mapping o on the space Stn of two-sided infinite sequences {x}0,
xn EL, such that (x,x+i) E H for all n E 7L.
We represent the state space L as the union L1 U LZ of two disjoint subsets L1
and L2. Denote by X2 the indicator function of the subset L2 C L: X2 (x) = 1 for
x E L2 and X2 (x) = 0 for x L2. It will be assumed that L2 0.
The concepts of the rotation number of a point x = {x},0 E SZn and the
rotation set of a TMC were introduced in the following way in [57].
The number rot (o, x) = limn n j o X2 (xk) is called the rotation number
of a point x = {xk} E SZn with respect to the partition L = L1 U L2. The set
rot (o-,1 -) = {rot (o-, x) : x E 1-} is called the rotation set of the TMC (L, H, o)
with respect to the partition L = L1 U L2.
THEOREM [57]. Let (L, H, o-) be an irreducible finite TMC, and let L = L1 U LZ
be a fixed partition of the state space. Then:
1) the rotation set rot(o,1 -) with respect to the partition L = L1 U LZ is either
a point or a closed bounded interval;
2) if the transformation °-Ic is topologically mixing, then rot(o,1 i-) is a closed
interval with rational endpoints that are the rotation numbers of two periodic points.

Remark 3. The rotation set of a mapping of Lorenz type. The results


in the preceding remark can be applied to Lorenz-type mappings of an interval
[58]. Denote by 3 L the set of mappings f : I - I of I = [0, 1] having a single point
c = c(f) of discontinuity and satisfying the following conditions:
1) f is continuous and monotonically increasing on [0, c) and (c, 1];
2) the set D = U>o f -n(c) is dense in I;
3) lime f (x) = 0 and 1(x) = 1 (Figure 5.2).

FIGURE 5.2
196 5. TRANSFORMATIONS OF THE CIRCLE

A symbolic model of a mapping f E 3 L can be constructed as follows. Let


us associate with an arbitrary point x E I \ D the sequence k(x) = {k(x)}0 of
symbols ±1 with
-1, f n (x) E [0, c),
kn (x) = 1
fTh(x) E (c, 1].
We get a mapping k from I \ D to a one-sided Bernoulli scheme 1 in two symbols
(with the product topology). Denote by f the closure of k(I \ D) in 1, and
by o the one-sided shift. The symbolic model of f is defined to be the TMC
(L, II, a), where L = {+1, -1} and 1 n = f . The rotation set rot (f) of a mapping
f EL is defined to be the rotation set of the corresponding TMC with respect to
the partition L = L1 U L2, where L1 = {-1} and L2 = {+1}.
According to [57], rot(f) is either a point or a closed bounded interval.
THEOREM [58]. If rot(f ), f E YL, is an interval, then the topological entropy
of f is strictly positive, and there exists a k E N such that for any n > k the
mapping f has a periodic point with smallest period n.
We remark that if we identify the endpoints of the interval I, then on the result-
ing circle a Lorenz-type mapping induces a transformation of the circle for which it
is also possible to introduce the rotation set in terms of a covering transformation
[57].
1.2. Invariance of the rotation number.
DEFINITION. A mapping f E P(IES) is semiconjugate to a mapping f o E P(IES)
if there exists a continuous mapping h E P(IES) such that
(1.4) f oh=ho fo.
We say that f is semiconj ugate to f o by means of h.
Semiconj ugacy of an f E P (S1) and an f o E P (S1) by means of a continuous
h E P(S1) is defined similarly:
(1.5) f oh=ho fo
LEMMA 1.5. a) If f is semiconjugate to f o by means of h, where f, f o, h E
P(IES), then
rot (f) = rot (f o) .
b) 1f f is semiconjugate to fo by means of h, where f, fo, h E P(S1), then
rot(f) = rot(fo).

PROOF. a) By virtue of (1.4), f o h= h o j, n E N. Therefore,


- = lim TOi(x)) = lim ho,fo(x)
rot (f) n n
The function h(x) - x def
= (x) is periodic and continuous, so (x)I < M <
+oo. Consequently,

rot (f) = lim


h o J(x) _
- lim 7(x) + o(fo (x)) = lim fo (x) -
= rot (f o) .
n- oo n fl-O0 n n-,00 n
1. THE POINCARE ROTATION NUMBER 197

Let us prove b). Take coverings f, J, and h for f, fo, and h, respectively. By
(1.5), f o h= h o f0 + k= h o (f0 + k) for some k E 7L . From this,
rot (f) = rot (f o + k) = rot(70) + k,
and hence rot (f) = rot (f o) . 0
DEFINITION. Mappings f, g E P(IES) are conjugate if there exists a homeomor-
phism h E P (Ilk) such that f o h= h o g.
DEFINITION. Mappings f, g E P(S1) are conjugate if there exists a homeomor-
phism h E P(S1) such that f o h= h o g.
The next result follows immediately from Lemma 1.5.
COROLLARY 1.1. a) If the transformations 7, g E P(IES) are conjugate, then
rot(f) = rot().
b) If the transformations f, g E P(S1) are conjugate, then rot(f) = rot(g).
1.3. Continuous dependence of the rotation number on a parame-
ter. If a transformation f belongs to the set P(R), then for any fixed number t E Ilk
the transformation ft(x) = f(x) + t also belongs to P(IES).
THEOREM 1.2. If f E P(IES) is continuous, then the function
r(t) = rot(f t) = rot[ f (x) + t]
is continuous and monotonically nondecreasing, and Ilk.

PROOF. We prove that r(t) is continuous at a point to. For a given E > 0 we
choose a natural number k > 2/E and an integer r such that
r
< rot(f)o < r+1
i
where fo=7 + to. By Lemma
some r > 0.
Then there exists a 6> 0 such that for I to - t <6
r< ft(xo)-xo <r+1.
According to Lemma 1.3, 1), we have r/ k < rot(ft) < (r + 1)/k. Therefore,
rot(f o) - rot(f t)l <2/k <e.
Since f is nondecreasing, it follows that f tl < f t2 for t1 <t2 and n E N. From
this,
-
rot(f ) = nlim
7(x) < lim 7(x) -
= rot (f t2 ) .
1
--> oo n fl-,00 n
Consequently, r (t) is a nondecreasing function.
It follows from Lemma 1.2 that Ilk. 0

In the set Homeo+ (S1) of orientation-preserving homeomorphisms of the circle


we introduce the metric
po(f,g) = maxlf(x) -g(x),

f,g E Homeo+(S').
198 5. TRANSFORMATIONS OF THE CIRCLE

The rotation number rot(f ), considered only for f E Homeo+ (S1), is a real-
valued function rot with domain Homeo+ (S1) . The next theorem shows that this
function is continuous.
THEOREM 1.3. The function rot : Homeo+ (S1) -* [0, 1) is continuous at each
point f E Homeo+ (S1) .
PROOF. This is analogous to the proof of continuity of the function r(t) in
Theorem 1.2. We omit it (and leave it to the reader as an exercise).
1.4. The rotation number of a homeomorphism of the circle. In this
subsection we establish a connection between an arithmetic property of the rotation
number (namely, its rationality or irrationality) and a dynamical property of a
homeomorphism of the circle with that rotation number (namely, the presence or
absence of a periodic orbit).
LEMMA 1.6. Suppose that the transformation f E P(S1) is a homeomorphism.
Then rot (f) is rational if and only if f has a periodic orbit, that is, there exist a
point x0 E S1 and a number k E N such that f ' (xo) = xo.
PROOF. Let f E P(IES) be a covering transformation for f. Then the presence
of a periodic point for f is equivalent to the existence of a point x0 E R and numbers
k EN andrE Z such that f'(xo)=x0+r.
According to Theorem 1.1, if f has a periodic orbit, then rot (f) is rational.
We prove that the rationality of rot(f) implies the existence of a periodic orbit.
Take a covering f such that f (O) E [0, 1) (by Lemma L4, there is such a covering).
We first consider the case rot(f) = 0 and show that f has a fixed point. Assume
that 7(x) x for all x E Ilk. Since f is a homeomorphism, it can be assumed for
definiteness that 1(x) > x for all x E R. Then 0 <7(0) <
{fTh
<f(0) < ..., that
is, the sequence (0)} is monotonically increasing.
We show that 7(O) < 1 for all n E N. Indeed, if 78(0) > 1 for some s E N,
then 72S() jS(7S()) 7S()
= > 1(1) = +1 > 2.
Similarly, 7flS(o) > n for n E N. Then
rot (f) = lim f nS (0) /ns > 1/s 0,

which contradicts the assumption that rot (f) = 0. Therefore, 7(o) < 1 for all
n E N, that is, the sequence {f(0)} is bounded.
Consequently, limn f n (0) = x0 exists, and f (xo) = limn f (fn (0)) =
limf n+1
(0) = x0.
'
Suppose now that rot(f) = r/k. Then rot[ f - r] = k rot(f) - r = 0 in
view of Lemma 1.2. By what has been proved, there exists a point x0 such that
f (x0) - r = x0 Then it(o) E S1 is a fixed point of the homeomorphism f k . 0
.

COROLLARY 1.2. A homeomorphism f E P(S1) does not have periodic points


if and only if the rotation number rot(f) is irrational.
We remark that a transformation f E P(S1) that is not a homeomorphism can
fail to have periodic points, even if the rotation number rot (f) is rational. However,
if f has a periodic point, then rot (f) is rational in view of Theorem 1.1.
2. TRANSFORMATIONS WITH IRRATIONAL ROTATION NUMBER 199

§2. Transformations with irrational rotation number


In this section we consider transformations of the circle of degree 1 with irra-
tional rotation number. According to Theorem 1.1, these transformations do not
have periodic orbits. Therefore, such transformations arise in a natural way in the
investigation of flows without closed trajectories.
2.1. Transformations semiconjugate to a rotation.
LEMMA 2.1. Let f E P(IES), and suppose that rot(f) E Ilk \ Q. Then T1(x) +
m1 < T2 (x) +m2 if and only if n1 rot(7)+m1 <n2 rot(f) +m2i where n1, n2 E N,
m1, m2 E 7L, and x E Ilk.
PROOF. Let f 1(xo) + m1 < f 2 (x0) + m2 for some xo E Ilk. For definiteness
assume that n1 < n2 . Then yo + m1 < f
n2 -fl1
(yo) + m2, where yo =
11 (xo). By
Lemma 1.1, y + m1 < fn2-n1(y) + 2
m2 for all y E R, so f n1(x) + m1 <f (x) + m2

It follows from the definition of the rotation number that if g1(x) < 2(x) for
all x E R, then rot (g 1) < rot (g2) . Using this property, we get that rot (f 1 + ml) =
n1 rot (f) + m1 < rot (f "2 + m2) = n2 rot (f) + m2. Since rot (f) is irrational, the
last inequality must be strict, that is, n1 rot (f) + m1 <n2 rot (f) + m2 LI
.

Denote by Ra : Ilk -* Ilk a mapping of the form x H x + a, x E Ilk, where a is


a fixed number. Obviously, Ra E P(IES). Since (x) = x + na, n E Z, it follows
that rot(Ra) = a.
THEOREM 2.1. Let f E P(IES) and rot(f) = a E Ilk/Q.
1) If there exists a point xo E Ilk such that the set A = { + m : n E N,
m E Z} is dense in Ilk, then f is a homeomorphism conjugate to the transformation
Ra.
2) If h1 and h2 are two homeomorphisms Ilk -* Ilk realizing a conjugacy between
f and Ra, then there exists a E R such that h1 = R, o h2.
PROOF. We define a mapping ho of A into the set B = {na + m : n E N,
m E Z} according to the rule ho (f (xo) + m) = ma _+ m. Since a is irrational, the
mapping ho is one-to-one. According to Lemma 2.1, ho is monotonically increasing.
This implies that ho is continuous.
Since a is irrational, B is dense in Ilk.
Let x E Ilk be arbitrary. Since cl(A) = Ilk, there exists a sequence {x}° of
points xn E_ A converging to x. It follows from the monotonicity of ho that the
sequence {ho(x)} has at least one accumulation point y E lit The equality
cl(B) = IlS implies that the accumulation point of {ho(x)}i° is unique. Indeed,
otherwise there would be two points y1, y2 E B between two accumulation points
of {ho(x)}° (Figure 5.3). Since ho is monotone, there exist two subsequences of
{x}° that do not intersect the interval on IlS between ho 1(y1) and ho 1(y2) (Figure
5.3). This contradicts the uniqueness of the limit of the sequence {x}°.
It can be shown similarly that y does not depend on the choice of the sequence
{x}0 converging to x. _
Let h(x) = y. The mapping h is an extension of ho. Since cl(A) = cl(B) =
IIS, h is a mapping of IlS onto R. Since ho is monotone, so is h. The equality
200 5. TRANSFORMATIONS OF THE CIRCLE

FIGURE 5.3

cl(B) = R implies that h is one-to-one. This and monotonicity give us that h is a


homeomorphism.
Let x E A. Then x = f (xo) + m for some n E N and m E Z. We have that

h0 o f (x) = ho[f +1(x0) + m] _ (n + 1)a + m


=na+m+a=ho(x)+a=Raoho(x).
By continuity we get that h o f (x) = Ra o h(x) for any x E Ilk, that is, f is conjugate
to Ra by the homeomorphism h (we leave it to the reader as an exercise to verify
the equality h(x + 1) = h(x) + 1). This implies that f is a homeomorphism.
We prove 2). Suppose that the homeomorphisms hl and h2 realize a conjugacy
between 7and Ra, and let ,3 = hl (xo) - h2 (xo) . Then h1(xo) = Ro h2 (xo) . For
anypoint x= fT(xo)+mEAwehavethat

ii(x) = hi [f (xo) + m] = Ra o hi(xo) + m = R o Rp o ha(xo) + m


= Rp o Ra o hz(xo) + m = Rp o h2 [f (xo) + m] = Rp o h2(x).

By continuity, hl (x) = o h2 (x) for any x E cl(A) = Ilk. 0

THEOREM 2.2. Suppose that f E P(R), rot(f) = a E I[8 \ Q, and there exists a
point xo E ][8 such that the set A = {fTh(xo) + m : n E ICY, m E 7G} is not dense in
In this case:
1) f is semiconjugate to Ra by means of a continuous mapping h E P(II) that
is not a homeomorphism;
2) if hl and h2 are two continuous transformations realizing a semiconjugacy be-
tween fand Rte, then there exists a E Il8 such that hl = Rpoh2 (consequently, any
continuous transformation semi conjugating fand Ra is not a homeomorphism);
def fx E
3) if h E P(II) realizes a semi conjugacy between f and Rte, and x(7, lZ) i
R: h-1(x) contains more than one point}, then
a) the inverse image h-1(x) is a closed interval for any x E x(7, h),
def
b) the set SZ(f) II8 \ U int h-1(x), x E X(f , h), is a nonempty closed perfect
set that does not depend on the semiconjugating transformation h,
c) h[1(J)] =IL
2. TRANSFORMATIONS WITH IRRATIONAL ROTATION NUMBER 201

PROOF. Following the proof of Theorem 2.1, we define a monotonically in-


creasing one-to-one continuous mapping ho : A -p B = {na + m : n E N, m E Z} by
n
ho [ f (xo) + m] = ma + m. As in the proof of Theorem 2.1, ho extends to a continu-
ous monotonically nondecreasing mapping (which we again denote by ho) of the set
cl(A) into cl(B) = lit Since the set A is not dense by assumption, the complement
IlS \ cl (A) contains at least one interval I = (a, b) with endpoints a, b E cl (A)
.

We show that ho (a) = ho(b). Assume not. Since B is dense in ll, the interval
(hio(a))io(b)) contains a point yo E B. We get that ho 1(yo) E (a,b) because
the mapping ho A : A -* B is one-to-one and monotone, and this contradicts the
equality A n (a, b) = 0. _ _ _
For any component (a, b) of the set IlS \ cl(A) we let h(x) = ho (a) = h(b),
where x E (a, b). By the foregoing, h is a monotonically nondecreasing continuous
mapping of R onto R that coincides with ho on cl(A). It follows from the definition
of h that h is not a homeomorphism. _ _ _ _
As in the proof of Theorem 2.1 we can show that h0 o f c1(A) = R« o ho 1(A)
The definition of h gives us that _h o f = R« o h on the whole line ll, that is, h
realizes a semiconj ugacy between f and R« .

Forx=f (xo)+mEAwehavethat
h(x+1)=ho[fn(xo)+m+1] =na+m+1=ho(x)+1=h(x)+1.
This and the definition of the extension of ho to the transformation h implies that
h (x + 1) = h (x) + 1 for all x E ll, which proves 1).
The proof of 2) is completely analogous to that of 2) in Theorem 2.1.
Let us prove 3). The monotonicity of h implies 3a). Since the union U int h
x E x (f , h), is an open set, the set 1(f) is closed. By 3a), any point x E SZ (f )
cannot be simultaneously an endpoint of two intervals h-1 (y1) and h-1(y2) with
y1, y2 E x (f , h) ; therefore, 1(f) is perfect and nonempty.
The assertion 3c) follows immediately from the construction of h and the prop-
erty 3a).
The fact that 1(f) is independent of the semiconj ugating transformation h
follows from 2). LI

COROLLARY 2.1. Assume the conditions of Theorem 2.2, and let f be a home-
omorphism. Then (in the notation of Theorem 2.2) :
1) the set x(f, h) is invariant under R«; _
2) 1(f) is a Cantor set and is invariant under f ;
3) [c(J)] = Ilt.
PROOF. _The_homeomorphism f carries an interval into an interval. From this,
the equality h o f = R« o h, and the assertion 3a) in Theorem 2.2 we get the
invariance of x(f, h) under Ra and the invariance of the set Uh-1(x), x E x(f, h),
under 7. Since f is a homeomorphism, it follows that i(7) = IlS \ U int
x E x (f , h), is also invariant under f .
Since a is irrational and x(f, h) is invariant under R«, x(f , h) is dense in R.
Therefore, between any two points in IlS there are points carried under the action
of h into x(f, h). This implies that 1(f) is nowhere dense, and hence a Cantor set.
0
202 5. TRANSFORMATIONS OF THE CIRCLE

The transformation Ra : x H x + a is a covering for the mapping Ra : S1 _* S1


given by x H x + a (mod 1), which is called a rotation or shift of the circle. It is
not hard to see that Ra+n = Ra for n E Z, and rot (Ra) = a (mod 1).
def
For a transformation f : S1 -* S1 the set {f(xo) : n a nonnegative integer}
O+(xo) is called the positive semi-orbit of the point xo E S1.
The next two lemmas follow directly from Theorems 2.1 and 2.2.
LEMMA 2.2. Suppose that f E P(S1) has an irrational rotation number a, and
that f has a dense positive semi-orbit. In this case:
1) f is a homeomorphism that is conjugate to the rotation Ra;
2) if h1 and h2 are homeomorphisms realizing a conjugacy between f and Ra,
then there exists a number ,3 such that h1 = Ro h2.
LEMMA 2.3. Suppose that f E P(S1) (rot(f) E Ilk \ Q) has a positive semi-orbit
that is not dense. In this case:
1) f is semiconjugate to Ra by means of a continuous mapping h E P(S1) that
is not a homeomorphism;
2) if h1 and h2 are continuous transformations realizing a semiconjugacy be-
tween f and Ra, then there exists a number ,3 such that h1 = Ro h2 (conse-
quently, any continuous transformation semiconjugating f and Ra is not a home-
omorphism);
def
3) if h E P(S1) realizes a semi conjugacy between f and Ra and X(f, h) {x E
S1 : h-1(x) contains more than one point}, then
a) for any x E X(f , h) the inverse image h-1(x) is a closed interval,
b) the set 1(f) def S1 \ U int h-1(x), x E X(f , h), is a nonempty closed perfect
set independent of the semiconjugating transformation h,
c) h[1(f)] = S1.
COROLLARY 2.2. Assume the conditions of Lemma 2.3, and let f be a home-
omorphism. Then (in the notation of Lemma 2.3):
1) x (f , h) is invariant under Ra ;
2) 1(f) is a Cantor set and is invariant under f ;
3) h[1(f)] = S1.
2.2. A criterion for being conjugate to a rotation.
DEFINITION. A transformation f E P(S1) is said to be transitive if it has a
dense positive semi-orbit on the circle.
EXAMPLE. A rotation Ra : 51 5 1 with irrational a is a transitive transfor-
mation. Moreover, the orbit of any point x E 51 with respect to Ra is dense in S1;
that is, Ra is a minimal transformation (see Example 1 in §2.3.2 of Chapter 1 for
a proof).
LEMMA 2.4. If a transformation f E P(S1) with irrational rotation number
rot (f) = a is transitive, then f is a minimal homeomorphism that is conjugate to
Ra.
PROOF. This is immediate from Lemma 2.2 and the minimality of the rotation
Ra. D
Denote by Homeo+ (S1) = Diff° (S1) the set of orientation-preserving homeo-
morphisms of the circle 51. It is obvious that P(S1) Homeo+ (S1) .
2. TRANSFORMATIONS WITH IRRATIONAL ROTATION NUMBER 203

For f E Homeo+ (S1) and for a point x0 E S1 the set O- (x0) = {fTh(xo),
n > 0} is called the negative semi-orbit of the point x0.
We recall that the family {f ni } is uniformly continuous if for any E > 0
there exists a 6> 0 independent of the n2 such that if x1, x2 E S1 are 6-close, then
f ni (x1) and f ni (x2) are E-close.
The following criterion for being conjugate to a rotation holds for homeomor-
phisms of the circle with irrational rotation number.
LEMMA 2.5. A homeomorphism f E Homeo+ (S1) with a = rot(f) E Ilk \ Q is
conjugate to the rotation Ra if and only if there exists a sequence {n}1, n2 E N,
increasing to infinity such that the family {fni }is uniformly continuous.
PROOF. NECESSITY. Let f = h-1 o Ra o h, where h is a homeomorphism
conjugating f and Ra. Then f fl = h-1 o Ra o h, n E N. Since h and h-1 are
uniformly continuous (being continuous mappings of a compact set), and since Ra
is an isometry, the family {fn}_1 is uniformly continuous.

SUFFICIENCY. Suppose that there is a uniformly continuous family {f ni }°O 1,


n2 -p +00. According to Lemma 2.4, it suffices to prove that the homeomorphism
f -1 is transitive. Assume not. Suppose that the semi-orbit O-(xo) = {fTh(xo),
n > 0} is not dense in S1. Let J be a component of the set S1 \ cl(O- (xo)) . Since
f is a homeomorphism, f -n(:J) is a component of the set S1 \ cl[O-(xo)] for any
n > 0. The rotation number rot(f) is irrational, so the components f -n (1), n > 0,
are disjoint, and hence the lengths of the intervals f -ni (J) tend to zero as n2 -* 00.
But this contradicts the uniform continuity of the family {f ni }0i. 0

2.3. Limit sets. Let f E P(S1).


DEFINITION. The w-limit set of a point x0 E S1 with respect to the transfor-
mation f is defined to be the set w (x0) = {x E 51 : there exists a sequence {flk}°_1,
n E N, increasing to infinity such that f nk (x0) -p x as nk -p oo } .
The union of the w-limit sets of all the points of the circle is denoted by SZ+ (f).
If f is a homeomorphism, then the a-limit set a (xo) of an arbitrary point
xo E S1 is defined similarly (with nk -* -00 instead of nk -p +oo). The union of
the a-limit sets of all the points of the circle with respect to the homeomorphism
f is denoted by 1 (f) .

Any semi-orbit of a rotation Ra with irrational a is dense in S1, so SZ+ (Ra) =


1 (Ra) = 51 .

This implies that if f E P(S1) is conjugate to a rotation Ra with a E Ilk \ Q,


then +(f) = ( f) = S1.

LEMMA 2.6. Suppose that f E P(S1) has an irrational rotation number a =


rot(f ), and let h : 51 S1 be a continuous mapping that is not a homeomorphism
and that realizes a semiconjugacy between f and Ra. Then +(f) = (f) J
51 \ U int h-1(x), where the union is over all x E X (f , h). Moreover, if f is a
homeomorphism, then +(f) = SZ- (f)
PROOF. Take an interval G carried by h into a point, and let x E G. If we
assume that f n (x) E G for some n E N, then Rah(x) = h f n(x) = h(G) = h(x);
that is, Ra has a periodic point, which contradicts the irrationality of a. Therefore,
204 5. TRANSFORMATIONS OF THE CIRCLE

f n (x) G for all n E N. This means that any point in G does not belong to the
w-limit set of a point of the circle. Thus, +(f) C SZ(f) .
Take an x E 1(f), and an arbitrary point xo E S1. We show that x E w(xo).
Let U be a neighborhood of x. Since x E 1(f) the set h(U) contains more than
one point. The monotonicity of h implies that there is a closed interval I C h(tL)
covering the point h(x). It follows from the irrationality of a that Ra is minimal,
and hence Ra (h(xo)) C I for some n E N. Since h-1 (I) C U because h is monotone,
we have that f n (xo) E U. From this, +(f) = SZ (f) .
We consider the case when f is a homeomorphism. It follows from the relation
ho f= Ra o h that ho f -1 = Ra l oh = R_ a o h; that is, h realizes a semiconj ugacy
between f -1 and R_ a . Therefore, 1- (f) = S (f_i) = SZ (f) = S (f) LI .

COROLLARY 2.3. If a homeomorphism f E Homeo+ (S1) has an irrational ro-


tation number, then either +(f) = 1(f) = Si, or SZ+ (f) = 1(f) = 1(f) is a
Cantor set that is invariant under f and f -1 .
2.4. Classification of transitive homeomorphisms. The following theo-
rem shows that the Poincare rotation number is a complete topological invariant
for a transitive homeomorphism of the circle.
THEOREM 2.3. 1) Two transitive homeomorphisms of the circle are conjugate
if and only if their rotation numbers are the same.
2) Any transitive homeomorphism of the circle has an irrational rotation num-
ber not exceeding 1, and for any irrational a E (0, 1) there exists a transitive
f E Homeo+ (S1) such that rot(f) = a.
PROOF. 1) The necessity follows from Corollary 1.1. We prove the sufficiency.
Let a = rot (f) = rot (g) , where f and g are transitive homeomorphisms of the
circle. By Lemma 2.4, both f and g are conjugate to the rotation Ra, and hence
to each other.
2) Suppose that the orbit of a point x0 E Si under a transitive homeomor-
phism f is dense in S. We show that the rotation number rot (f) is irrational.
Assume not. Then by Lemma 1.6, f has a periodic orbit, say of period k > 1.
This implies that the points x0, f (xo), ... , f' (xo) belong to respective intervals
1, ... , Jj that are invariant under the homeomorphism f'. Therefore, the se-
quence {f(f(xo))}_ has a limit both as n -* +00 and as n -* -oo for each
0 < i < k - 1. This contradicts the denseness of the orbit of x0, because any point
f m (xo) can be represented in the form f m (xo) = f n [ f i (xo )] , 0 < i < k - 1.
By the definition of the rotation number of a transformation f E P(S1), we have
that 0 < rot (f) < 1. Therefore, rot (f) E (0, 1) for a transitive f E Homeo+ (S1) .
Since rot (Ra) = a, we can take f = Ra. Since a is irrational, Ra is a transitive
(and even minimal) homeomorphism (see Example 1 in §2.3.2 of Chapter 1). 0

Two transformations f and g commute if f o g= g o f. The centralizer of a


transformation f is defined to be the set of transformations commuting with f. The
following lemma shows that for a transitive rotation of the circle the group of all
rotations of the circle is the centralizer in the set P(S1) (obviously, two rotations
commute).
LEMMA 2.7. Suppose that Ra : Si _ Si is a rotation with a E Ilk \ Q. If
h o Ra = Ra o h and h E P(S1), then h = R, for some 13 E Ilk.
2. TRANSFORMATIONS WITH IRRATIONAL ROTATION NUMBER 205

PROOF. Since a is irrational, Ra is a minimal transformation.


Take a point xo E S1 and a number ,3 E Ilk such that h (xo) = R(xo) Then .

h o R« (xo) = R« o h (xo) = R« o Ra (xo) = Ra o R« (xo) , n E 7L. The orbit O (xo )


of xo with respect to Ra is dense in S1. It follows from Ra o R= Ro Ra
that h [O (xo )] = R[O (xo )] is an orbit of Ra, which is also dense in S1. Since
h [R« (xo) ] = R[R« (xo) ] and since h is monotone, it follows that h = Ron the
whole circle. 0

2.5. Classification of Denjoy homeomorphisms.


DEFINITION. A homeomorphism f of the circle with irrational rotation number
is called a Denjoy homeomorphism if its limit set 1(f) is nowhere dense.
The set of Denjoy homeomorphisms is denoted by Den(S1).
According to Corollary 2.3, the limit set 1(f) is a Cantor set that is invariant
under f and f*
By Lemma 2.3, f is semiconj ugate to the rotation Ra, a E rot (f) .

DEFINITION. Let f E Den(S1) and let h : S1 -* S1 be a continuous mapping


realizing a semiconj ugacy between f and Ra, a = rot (f) . The set X (f , h) = {x E
S1 : h-1 (x) contains more than one point} is called the characteristic set of the
homeomorphism f with respect to the semiconj ugating mapping h.
LEMMA 2.8. Let f E Den(S1) and let h be a continuous mapping realizing a
semiconjugacy between f and Ra, a = rot(f). Then X(f,, h) is made up of an at
most countable family of orbits of the rotation Ra.
PROOF. According to Lemma 2.3, h1 (x) is a closed interval for any point
x E X(f, , h). Therefore, h[h-1(x)] = h[int h1 (x)] = x, and hence X(f, h) =
h[U int h-1(x)] = h[S1 \ (f)], x E X(f, , h). Since 1(f) is a Cantor set, S1 \ 1(f)
consists of a countable family of open intervals. Therefore, X(f,, h) consists of a
countable family of points.
In view of Corollary 2.2, X(f, h) is invariant under Ra. 0

For two sets X1, X2 C S1 we write X1 = X2 if one of them is the image of the
other under a rotation of the circle, that is, X2 = Ra (X1), ,3 E Ilk.
We remark that if continuous mappings h1 and h2 realize a semiconjugacy
between a homeomorphism f E Den(S1) and the rotation Ra, a = rot(f ), then
X (f , h 1) - X (f , h2) by virtue of Lemma 2.3, 2).
THEOREM 2.4. Suppose that the continuous mappings h1 and h2 realize a semi-
conjugacy between the Denjoy homeomorphisms f and g and the respective rotations
Rrot(f) and Rrot(g) Then f and g are conjugate if and only if rot(f) = rot(g) and
X(f, hl) = X(g, h2) .
PROOF. NECESSITY. Let h o f = g o h, where h E Homeo+ (S1) (Figure 5.4).
Then rot (f) = rot (g) by Corollary 1.1.
Since h is a homeomorphism realizing a conjugacy between f and g, it follows
that h[1(f )] = 1(g). We take a point x e X(f, hl ). By Lemma 2.6, the closed
interval h-1(x) intersects 1(f) only in its endpoints, and thus h[hj 1(x)] is a closed
interval intersecting 1(g) only at the endpoints. This implies that h2 takes the
interval h[hi 1(x)] into a point, that is, h[hj 1(x)] E X(g, h2). Since the complete
206 5. TRANSFORMATIONS OF THE CIRCLE

Si

FIGURE 5.4

inverse image hj 1(x) is a single point for each point x E Si \ x(f , h1), the mapping
ho = h2 o ho hi 1 is well defined, and ho [x (f , h1)] C x(g, h2). The transformation ho
is continuous because the mappings h1, h2, and h are monotone and continuous. In
a completely analogous way it can be shown that the mapping h1 oh-1 o h2 1 = ho 1
is well defined and continuous, and that ho 1 [x (g, h2 )] C x (f , h1). Consequently, ho
is a homeomorphism carrying x (f , h1) onto x (g, h2).
We show that ho commutes with R. Let x E S1 \ x (f, h1). Then
Ra o ho (x) = Ra o h2 o h o hi 1(x) = h2 o g o h o hi 1(x)
= h2 o h o f o hj1(x) = h2 o h o hj1 o Ra (x) = h0 o Ra (x) .

By continuity, Ra o ho (x) = h0 o Ra (x) for all x E S. According to Lemma 2.7,


h0 = Rfor some E ItB, and ho [x (f , h1)] = R[x (f , h1) ] = x (g, h2) , that is,
x(f, h1) = x(g, h2).
0
SUFFICIENCY. For a Cantor set Q on the circle 51 we set Q = Q \ r(Q), where
r(Q) is the set of endpoints of the adjacent intervals of S1 \ Q.
By assumption, there exists a E Tt such that x(g, h2) = R[x(f, h1)]. Since
0 0
h1 [Q(f )] = 51 \ x(f, h1) and h2 [Q(g)] = 51 \ x(g, h2), the mapping h21 o RQ o
0 0
hl I : Q(f) -* (g) is defined.
f
We show that this mapping is uniformly continuous. Let e> 0. There exists a
finite collection of adjacent intervals J1, ... , of the Cantor set Q(g) such that the
length of any component of the set 51 \ UZ 1 J is less than e. Since h2 [U,, J] C
x (g, h2) and R[x (f , h,)] = x (g, h2 ), there are adjacent intervals C,,. .. , Gk of
the Cantor set Q(f) such that R[h1(UZ 1 G2)] = h2 (U , J). Suppose that the
number 0 < S < 2 is less than the length of each G2, i = 1, ... ,k. In this case if
0
the distance between points x1, x2 E Q (f) is less than S, then the smaller circular
arc with endpoints x1 and x2 does not contain the interval GZ (i = 1... ,k) .
Consequently, one of the arcs with endpoints h2 1 o Ro h, (x2), i = 1, 2, does not
contain any of the intervals J, i = 1, ... , k. Therefore, Ih2 1 o Ro h1 (x1) - h2 1 o
Ro h1(x2) I <e, which proves that the mapping h2 1 o Ro h1 is uniformly
continuous.
2. TRANSFORMATIONS WITH IRRATIONAL ROTATION NUMBER 207

0 0
It is proved similarly that the mapping hi 1 o RQ 1 o h2I(9) : Q(g) -* (f) is
also uniformly continuous.
Since h21 o Ro h 1 I(f) is uniformly continuous, it can be extended to a contin-

uous mapping h: cl[Q(f )] = Q(f) -* Q(g) = cl[o(g)]. We show that h[r(Q(f ))] _
r(Q(g)) (that is, the endpoints of the adjacent intervals of Q(f) are carried by h into
the endpoints of adjacent intervals of Q (g)) . We take x E F (Q (f)) and assume that
0 0
h(x) E Q(g). Then x1 = (h21 o Ro h1)-1(h(x)) E Q(f ). Since h is an extension of
the monotone mapping h2 1 o Ro h 1, it is also monotone. Consequently, one of the
arcs I of Si with endpoints x1 and x has the property that h[I n Q(f )] = h(x). By
0
the construction of the Cantor set, I contains an infinite family of points in Q(f ).
Then the last equality contradicts the one-to-oneness of h2 1 o Ro h1 and the
(f )
equality h I = h21 o Ro h 1 . Thus, h [r (Q (f)) ] C r [Q (g) ] . The monotonicity
0
of h and the denseness of Q in the Cantor set Q imply that h[r(Q(f ))] = r[SZ(g)].
We show that h is one-to-one. To do this it suffices to prove that the restriction
h I r(e(f)) is one-to-one. Take two distinct points x1, x2 E r (Q (f)) . If these points
are not endpoints of a single adjacent interval of the Cantor set (f), then both the
0
arcs in 51 \ {x1, X2} contain points in Q (f) Since h I(f) is monotone and one-to-
.

one, h(x1) h(x2). Suppose now that x1 and x2 are endpoints of a single adjacent
interval, and assume that h (x 1) = h (x2) E r (Q (g)) Let J be the adjacent interval
.

of the Cantor set Q(g) with endpoints h(x1) and y1. Since h[r(Q(f ))] = r(Q(g)), it
follows that h-1(y1) 0. Let x E h-1(Yi). By the structure of a Cantor set, each of
the arcs in 51 \ {x1, X2} contains points in Q (f) It follows from the monotonicity
.

of h that there are points in Q(f) carried by h into J, which cannot be. This
contradiction proves that h is one-to-one, and h preserves the order of points on
the circle. Thus, h : Q (f) -* Q (g) is a homeomorphism.
Let us now extend h to a mapping of the circle. We break up the adjacent
intervals of the Cantor set Q (f) into equivalence classes: intervals G' and G" are
regarded as equivalent if there exists an n E Z such that f n (G') = C". From each
equivalence class we choose a representative, obtaining G1, G2, .... We extend h
from the endpoints of GZ to an arbitrary homeomorphism h2 : GZ -* 51 \ Q(g) on
the whole interval G2, i = 1, 2, .... For the points x E fn(GZ), n E Z, we set
h2,n (x) = gn o h2 o f -n (x) .
We extend h to the whole circle by means of the homeomorphisms h2,n, i =
1, 2, ... , n E Z. Denote the mapping obtained again by h. By construction,
h: 51 - 51 is a homeomorphism.
0
If x E Q(f ), then ho f (x) = h21 o Ro h1 o f (x) = h2' o Ro Ra o h1(x) _
go h2 1 o Ro h1(x) = go h(x). By continuity, ho f (x) = go h(x) for x E Q(f ).
If x E f (C), then h o f (x) = h2,n+1 [f (x)] = gn+1 ° h2 ° f -(n+1) (f (x))
g o [gn o h2 o f ](x) = g o h(x). Thus, h o f (x) = g o h(x) for all x E 51. O

Theorem 2.4 shows that the characteristic set is a topological invariant of a


Denjoy homeomorphism up to a rotation. We remark that the characteristic set
itself is determined by the Denjoy homeomorphism also up to a rotation. The next
theorem shows that the characteristic set is a complete topological invariant.
208 5. TRANSFORMATIONS OF THE CIRCLE

THEOREM 2.5. Let x be an at most countable family of orbits of the rotation


Ra, c E IR \ Q. Then there exist an f E Den(S1) and a continuous mapping h
semiconjugating f and Ra such that x = x(f, h).
PROOF. The proof is by the scheme for constructing a homeomorphism of the
circle with Cantor limit set in the description of a Denjoy flow (see X3.1 in Chapter
1). The construction consists in a "blowing up" of points of the orbits in the
family x.
On each orbit in x we choose one point, obtaining x 1, ... , xl , .... With a point
Ra (xl) (n E Z) we associate a number anl> > 0 so that

anl) = a < +oo.


n,l

(For example, anl = (ml + 1+ 1)-1(Inl + l + 2)-i.)


In place of each point Ra (xl) we put on the circle a segment Inl of length a.
This operation can be realized formally as follows. We remove from S1 a point
Ra (xl) (n and l fixed). The closure of the open arc obtained will be homeomorphic
to a closed segment whose endpoints we identify with the endpoints of the segment
Inl of length a. As a result we get a circle 5(1+ and ) of length 1+ a. Obviously,
ais homeomorphic to 8(1+
S1 \ {R(xj)} it can be assumed that the
remaining points in x \ {R(xj)} lie on 5(1+ Continuing this process, we get
a circle 8(1+ a) of length 1+ a, and by construction the mutual arrangement of the
intervals Inl , n E Z, l = 1, 2, ... , on 8(1 +a) is the same as the mutual arrangement
of the points xnl = Ra (xl), n E Z, l = 1, 2, ... , on S. (The construction given is
called a blowing up; the circle 8(1+ a) is obtained by blowing up each point in x.)
We introduce an equivalence relation on 8(1 + a) : the points x 1, x2 E 8(1 + a)
are equivalent if they lie on a single interval Inl . Taking the quotient of 8(1 + a)
by this equivalence relation (identifying each interval Inl with a point), we get the
circle S. Let h : 8(1+ a) - S1 be the natural projection (the mapping h is called
a blowing-down). By construction, h(I7 P) = xnl and h(Ul n i Th = x.
Since x is dense in S1 and h is monotonically nondecreasing, the set Q =
S1 \ U,1 int Inl is a Cantor set.
The mapping h is one-to-one on SZ = S1 \ U,1
n and by monotonicity is
a homeomorphism onto its range S1 \ x. Therefore, RaI sl\x : S1 \ x -' 5 1 \ x
induces a homeomorphism f io = h-1 o Ra o h I d: Q -* Q by means of h. As
in the proof of uniform continuity of the mapping hi 1 o Ra o hl for Theorem
2.4, it can be shown that fl is uniformly continuous and can be extended to a
homeomorphism f c : Q -* SZ, which can in turn be extended to a homeomorphism
f : S(1+a) -* S(1+a).
o
The set Q is invariant under f and f, and any orbit 0(x) = {fTh(x) : n E Z},
0 0
x E Q, is dense in fZ. Therefore, f E Den[S(1 + a)]. Since the lengths of the
intervals f n (I) tend to zero as n - +oo, it follows that Q(f) = Q. The mapping
h realizes a semiconjugacy between f and Ra, and x(f, h) = x.
2. TRANSFORMATIONS WITH IRRATIONAL ROTATION NUMBER 209

A linear diffeomorphism 8(1 + a) -* 81 carries f into the required f E


Den(S1). O

DEFINITION. Let x(f, h) be the characteristic set of a Denjoy homeomorphism


f. The number of orbits or the cardinality Ix(f , h) of the collection of orbits making
up x (f , h) is called the characteristic of f.
By Lemma 2.3, 2), the characteristic of a homeomorphism f does not depend
on the semiconjugating mapping h. It can be defined as the number or cardinality
of the set of equivalence classes of adjacent intervals of the Cantor set Q (f) , where
two adjacent intervals G' and G" are taken to be equivalent if G" = f fl (G') for
some n E 7L. The characteristic is obviously a topological invariant of a Denjoy
homeomorphism. But, as the next lemma shows, the characteristic is not a complete
topological invariant.
LEMMA 2.9. For any irrational number c E (0, 1) there is a continuum of
nonconjugate Denjoy homeomorphisms with rotation number c and characteristic
equal to 2.
PROOF. Two sets xl, x2 C S1 will be said to be equivalent if xl = x2 (that
is, one of them can be superimposed on the other by a rotation of the circle).
According to Theorems 2.4 and 2.5, it suffices to show that there is a continuum of
nonequivalent sets, each of which consists of two orbits of the rotation R.
We fix an orbit O1 of Ra and form the set x, = O1 U 0,, where 0, is an orbit
of Ra different from O1. Since the set of orbits has the cardinality of a continuum,
there is a continuum of distinct sets x .
Denote by {x,2} the family of all sets of the form equivalent to x, (that
is, consisting of two orbits and with a fixed orbit O1) . Let x' = O1 U 0, E {x} .
Then R1 (x') = x for some 131 E IR. Since Rat o Ra = Ra o R1, the rotation Rat
carries an orbit into an orbit of R. Therefore, there are only two possibilities: 1)
R1 (01) = O1; 2) R1 (01) = 0,. In case 1) R1 (0,) = 0, by the minimality of
Ra , and hence 0, = 0 1 , that is, x' = x . In case 2) R1 (0,) = 01.
Let x" = 0 1 U 0/22 E {x/2} be different from It follows from the foregoing
that Rae (O1) = 0 and Rae (0/22) = O1 for some /32 E IR. The minimality of Ra
and the equalities 0 = R1 (01) = Rae (O1) give us that Rae = Rat o Ra for some
n E Z. Therefore, 0/22 = RQ2 (O1) = RQ11(O1) = 0,, that is, x' = x".
Consequently, each class {x, } has only two representatives, and since there
is a continuum of distinct sets x/2, there is also a continuum of distinct classes
{x/2}. Sets and x/22 in different classes {x/2i } and {x/22 } are not equivalent;
therefore, representatives of all the classes {x/2} give the required continuum family
of nonequivalent sets, each consisting of two orbits of Ra. O

2.6. Classification of Cherry transformations. In this subsection we


consider Cherry transformations of the line and the circle. Cherry transformations
of the circle arise in a natural way in the study of the Poincare mapping on contact-
free cycles in Cherry flows. Recall that P(IR) denotes the set of monotonically
nondecreasing transformations of IR of degree 1.
DEFINITION. A transformation f E P(IR) is called a Cherry transformation of
the line IR if it satisfies the following conditions:
210 5. TRANSFORMATIONS OF THE CIRCLE

1) on any finite interval, f has finitely many intervals of constancy (that is,
intervals on which f takes a constant value) and finitely many points of disconti-
nuity; _
2) f is continuous at endpoints of intervals of constancy;
3) if xo is a point of discontinuity, then f (x) T f (xo) as x T xo (that is, f is
left-continuous at points of discontinuity; see Figure 5.5);
4) f has an irrational rotation number;
5) if [a, b] is an interval of constancy, then for any n E N the complete in-
verse image 7([a, b]) is a closed interval with a neighborhood in which f is a
homeomo_rphism; _ _
6) if f is discontinuous at xo (see Figure 5.5) and if [c, d] = [f(xo), limo f (x)],
then for any n E NU {0} the image f([c, d]) is a closed interval with a neighborhood
in which f is a homeomorphism;
7) all the images of the intervals of constancy and all the points of discontinuity
of f lie in Q (f) .
The set of Cherry transformations of the line is denoted by Ch (IR) .

xo

FIGURE 5.5

DEFINITION. A transformation f E P(S1) is called a Cherry transformation of


the circle Si if there exists a covering Cherry transformation of the line for f.
The set of Cherry transformations of the circle is denoted by Ch(51).
Unless otherwise stated, we assume below that a Cherry transformation is not
a homeomorphism (that is, has at least one point of discontinuity or at least one
interval of constancy). Since the rotation number of a Cherry transformation of the
circle is irrational, this implies that the positive semi-orbit of any point under this
transformation is not dense (for example, if a Cherry transformation has an interval
I of constancy, then any positive semi-orbit intersects I in at most one point).
Therefore, by Lemma 2.3, a Cherry transformation of the circle is semiconjugate
to a rotation by means of a continuous monotonically nondecreasing mapping that
is not a homeomorphism.
DEFINITION. Let f E Ch(51) be semiconjugate to Ra, c E rot(f ), by means of
a continuous mapping h E P(S1), and suppose that a closed interval I is mapped
by h into a point, with h-1 [h (I )] = I. If f n (I) is an interval for all n E Z, then I
is called a gray interval.
By Lemma 2.3, the definition of a gray interval does not depend on the semi-
conjugating mapping h.
2. TRANSFORMATIONS WITH IRRATIONAL ROTATION NUMBER 211

Let I be a gray interval. It follows from the equality Raoh = ho f n, n E N, that


the interval f2(I) belongs to some gray interval, denoted by f Z (I) , for any i E 7L.
We remark that the equality f2(I) = f Z (I) does not always hold (for example, f(I)
can be adjacent to an interval of constancy, and then f (I) is equal to the union of
the interval f (I) and the interval of constancy).
DEFINITION. Let I be a gray interval of a transformation f E Ch(S1). The
union J(I) = UnE f (I) is called a gray cell.
We now give the definitions of a black interval and a black cell (negative and
positive) of a Cherry transformation f of the circle Si.
Let [a, b] C 51 be an interval of constancy of a transformation f E Ch(S1).
According to condition 5) in the definition of a Cherry transformation, the complete
inverse image f -n ([a, b]) is an interval for any n E Z+ (Z+ is the set of nonnegative
integers), called a black negative interval. The union
f-n([a>b])defJ(a,b)
U
nE7L+

is called a black negative cell.


If xo E 51 is a point of discontinuity of a transformation f E Ch(S1), then
[f(xo), limo f (x)] de [c, d] is an interval, which we denote by J(xo). According
to condition 6) in the definition of a Cherry transformation, f n ([c, d]) is an interval
for any n E Z+, called a black positive interval. The union
defJ(x0)
U
nEZ+

is called a black positive cell.


Let x (f , h) = Uh (I) , where the union is over all gray intervals I. The set x (f , h)
is an at most countable family of orbits of the rotation Ra , where c = rot (f) .
Let x- (f , h) = Uh(I), where the union is over all black negative intervals I.
According to the condition 1) in the definition of a Cherry transformation, the set
x- (f, h) is made up of finitely many negative semi-orbits of R.
Similarly, x+ (f , h) = Uh(I ), where the union is over all black positive intervals
I, is made up of finitely many positive semi-orbits of R.
If a gray interval [a, ,3] contains intervals of constancy, then we assign a code
(e1, E2) - to the point h([ca, ,3]) as follows. According to 5) and 7), any interval
[a, b] C [a, /3] of constancy has a common endpoint with the interval [a, /3]. There-
fore, [ca, /3] contains at most two intervals of constancy. If [ca, 3] contains two in-
tervals of constancy, then we set (e1, E2) - _ (1, 1)-. If [ca, /] contains a single
interval of constancy [a, b], then either a = c or b = /3. In the first case we set
(E1, E2)- _ (1, 0)-, and in the second (E1, E2)- _ (0,1)-.
We assign a code (e1, E2)+ to the point h([ca, 3]) in an analogous way if the
gray interval [a, 3] contains intervals of the form f (xo), where xo is a point of
discontinuity of f.
Note that points equipped with a code are initial points of semi-orbits in
x- (f, h) or x+ (f, h) which belong to orbits in x(f , h). Note also that two codes
are assigned to some points in x- (f , h) n x+ (f, h).
212 5. TRANSFORMATIONS OF THE CIRCLE

Let f E Ch(S1). The scheme S(f, h) of a transformation f with respect to


a semiconjugating mapping h is defined to be the collections x(f, h) of orbits,
x- (f, h) and x+ (f , h) of semi-orbits, and x* (f , h) of coded points equipped with
the corresponding codes.
The schemes of transformations f, g E Ch(S1) are said to be isomorphic if there
exists a ,3 E IR such that

Ra[x(f> hi)] = X(9,ha)> Ra[x (f, hi)] = X (9,ha),


Ra[X+(f, hi)] = X+(9, h2), Ra[x*(f> h1)] = X*(9, hz),
and each point in x* (f , h1) is carried by Rinto a point with the same code (or
codes), where h1 and h2 realize semiconjugacies between f, g and the respective
rotations Rrot(f), Rrot(9)
By Lemma 2.3, the schemes of a transformation f E Ch(S1) with respect to
different mappings h semiconjugating f and Rrot(f) are isomorphic.
THEOREM 2.6. Suppose that the transformations f, g E Ch(S1) are semicon-
jugate to rotations by means of mappings h1 and h2i and let S(f, h1) and S(g, h2)
be the schemes of f and g with respect to h1 and h2i respectively. Then f and g
are conjugate if and only if rot(f) = rot(g) and the schemes S(f, h1) and S(g, h2)
are isomorphic.
With obvious changes the proof repeats that of Theorem 2.4, and we omit it.
Let us consider a rotation Ra with c E IR \ Q. An admissible scheme is defined
to be a collection made up of an at most countable family x of orbits of Ra and finite
families x+ and x- of positive and negative semi-orbits, respectively, satisfying the
following conditions:
a) a code (e1, e2) + (-) with ei E {O, 1} and e 1 + 2 > 1 is assigned to each initial
point of a positive (negative) semi-orbit in the intersection x+ f1 x (x n x);
b) all the semi-orbits in x+ U x- not belonging to x are disjoint;
c) for each orbit O E x there are at most four semi-orbits in x- U x+ lying on
O, and at most two in each of x- and x+, and any point of O belongs to at most
two semi-orbits in x- U x+;
d) if two semi-orbits in x- U x+ intersect (and thus lie on some orbit in x), then
the codes (e1, e2) and (ei , e2) of their initial points are opposites, that is, e2 +E 2 = 1
fori=1, 2.
A transformation f E Ch(S1) has an admissible scheme with respect to any
semiconjugation between f and Rlot(f). Indeed, let S(f, h) be the scheme of f
with respect to a semiconjugating mapping h. If two semi-orbits Oi) and 02) in
x- (f, h) U x+ (f, h) do not belong to x(f, h), then the black cells corresponding to
them do not intersect the gray intervals, and do not have intersecting black intervals
in view of the conditions 5) and 6) in the definition of Cherry transformations.
Consequently, the semi-orbits Oi) and 02) are disjoint.
The conditions c) and d) follow from the condition 7).
THEOREM 2.7. Let Ra : S1 -* 81 be a rotation with c E IR \ Q, and let S be
an admissible scheme. Then there exists an f E Ch(S1) that is semiconjugate to
Ra (so that c = rot(f )) by means of an h such that S(f, h) = S.
The proof is by the scheme used for Theorem 2.5, and we omit it.
3. STRUCTURALLY STABLE DIFFEOMORPHISMS 213

§3. Structurally stable diffeomorphisms


In this section we consider the space of diffeomorphisms of the circle, and in
that space we single out the dense open subspace of structurally stable (or weakly
structurally stable) diffeomorphisms. The results in this section go back to Poincare
[68], Maier [54], Pliss [65], and others, and are reflected in the books [62], [64],
and [89]. Our exposition thus bears a schematic character.
3.1. The C'-topology. For a numerical CT-smooth function f : IR -* IR we
denote by DT f the derivative of order r E N. _
Let f : Si - 51 be a transformation of degree 1, and let f : IR -* IR be a
covering transformation. Then f (x) = x + h(x), where h(x) is a periodic function
of period 1. If the derivative DT f exists, then we set DT f = Dr f for r E N and
call DT f the rth-order derivative of f. Since any covering transformation for f has
the form f (x) + n, n E Z, the definition of DT f is independent of the choice of the
covering transformation.
Forr=Owe takeDTf =D°f = f.
A homeomorphism f of the circle is said to be a C''-homeomorphism if Dz f
exists for i = 1, ... , r.
The set of C''-homeomorphisms of the circle is denoted by Homeor (51).
If f _E Homeor (S1) for all r E N, then f is a C°°-homeomorphism. If a
covering f for f is an analytic function, then f is said to be an analytic or Cw-
homeomorphism.
A homeomorphism f of the circle is said to be a C''- diff eomorphism if f, f1 E
Homeor (S1) . The set of C''-diffeomorphisms of the circle is denoted by Diffr (S1) .
Let Homeo(S1) = Diff° (S1) .
We introduce a metric pT on the set Homeor (S1), 0 < r < oo. First let po (f , g)
be defined for f, g E Homeo (S 1) by
po (f , g) = max l f (x) -g(x) = max f - g (x) (mod 1).
S 0<x<1
If f, g E Homeor (S 1) for 1 < r <00, then we set
pT (f , g) = po (f , g) + po (Df , Dg) +... + po (DT f, Drg),
where po (D2f, Dzg) = I- D29(x)
The topology induced by pk on the set Homeor (S1) is called the C''-topology,
and this topology turns Homeor (S1) into a topological space.
The C°°-topology on the set Homeo°° (S1) is defined to be the weakest topology
induced by the C''-topologies on Homeor (S1), r < 00, and by the imbeddings
Homeo°° (51) C Homeor (S 1) , r E N.
The subset Diffr (S1) C Homeor (S1) is open in Homeor (S1), r > 0, and is
equipped with the induced C''-topology as a subset of that topological space.
We introduce the C°°-topology on the set Homeow (Si) of analytic homeomor-
phisms and the set Diff w (S 1) of analytic diffeomorphisms.
3.2. Main definitions.
DEFINITION. A CT-homeomorphism f, r > 1, is said to be structurally stable
if for any e> 0 there is a neighborhood 'IL of this homeomorphism in Homeor (S1)
such that any g E IL is conjugate to f by means of a homeomorphism h that is
e-close to the identity in the metric p0.
214 5. TRANSFORMATIONS OF THE CIRCLE

If the requirement that the conjugating homeomorphism h be e-close to the


identity is dropped in this definition, then we get the definition of a weakly struc-
turally stable C''-homeomorphism f.
A structurally stable C'-homeomorphism is obviously weakly structurally sta-
ble.
DEFINITION. A fixed point x° E Si of a C''-homeomorphism f, r > 1, is said
to be hyperbolic if D f (x°) 1.
A point x° is called a sink or attracting point if D < 1.
If I> 1, then x° is called a source or repelling point.
DEFINITION. For a C''-homeomorphism f a periodic point of period m is said
to be hyperbolic (attracting or repelling) if it is a hyperbolic fixed point (attracting
or repelling) for f'n .
LEMMA 3.1. Suppose that f E Diff''(S1), r > 1, has fixed points, and each
fixed point of f is hyperbolic. Then f is weakly structurally stable.
PROOF. It follows from the definition of hyperbolicity that a hyperbolic fixed
point is isolated in the set of fixed points of the diffeomorphism. Therefore, f has
finitely many fixed points. Topological considerations give us that half the fixed
points are sinks and half are sources (in particular, the number of fixed points is
even), and they alternate on 51 (Figure 5.6).

FIGURE 5.6

Let xi, ... , xi be the sinks and xi, ... ,x' the sources off.
For a subset 'IL C 51 we denote by m(tL) its Lebesgue measure.
Let ; be a sink, and 'Ui x2 a neighborhood of it in which IDf I < A < 1.
Then m [ f ('u2) ] < Am (tL2) < m (tLi) , and hence cl [ f (tLi) ] C 'u2 . Moreover, if i is the
length of a minimal interval in U2 \ f (tL2) , then g (tL2) C u2 for any g e Homeo'' (S 1)
that is -close to f in the C°-topology. Therefore, g has a fixed point in U. If
pi (f, g) < (1 - A)/3, then Dg < A + (1 - A)/3 < (2A + 1)/3 < 1. Consequently, g
has exactly one attracting fixed point in ' L2.
An analogous neighborhood can be constructed for each sink and each source.
We take disjoint neighborhoods Vj of the fixed points x, x, i = 1, ... ,1,
j = 1, ... , 21, together with a number > 0 such that if pl (f , g) < i, then g
has in each Vj exactly one hyperbolic fixed point of the same character as f. The
3. STRUCTURALLY STABLE DIFFEOMORPHISMS 215

complement S 1 \ UVj consists of disjoint closed segments Ii,.. . , I21. None of these
segments contain fixed points of f, so
v = min p(x, f (x)) >0,
xeuIj
where p is the metric on 51 induced by the covering it : IR -* S1. Then any g E
Homeo'' (S1) that is (v/2)-close to f in the C°-topology does not have fixed points
on 51 \ UV3.
Let S = min{, v/2}. It follows from the foregoing that any g E Homeo''(S1)
with pr (f , g) <6 has on 51 only 21 fixed points, and they all are hyperbolic: half
sinks x2 , i = 1, ... ,1, and half sources x2 , i = 1,... ,1.
For both f and g the attracting and repelling points alternate on 51. On the
arc between a sink and the source next to it all the points move in the direction of
the sink under the action of the homeomorphism.
We consider two such (open) arcs: I( f) for f and 12(g) for g. Let x E 12(f) and
x' E 12(g), and let h : [x, f (x)] -* [x', g(x')] be an arbitrary orientation-preserving
homeomorphism. Since the segments f n [x, f (x)], n E Z, (respectively, gn [x', g(x')],
n E Z) have disjoint interiors, and their union is 12(f) (respectively, 12(g)), h can
be extended to a homeomorphism 12(f) -* 12(g) as follows:
hI fn [x,f(x)] = gn ° h ° f -' I fn [X,f(X)] : fn [x, f (x)] -* gn [x', g(x')]
It is easy to verify that on 12(f)
g o h = h o f.
Making this construction on all the arcs 12(f), i = 1, ... , 21, and setting h(x2) = x2
and h (xu) = xu , we get a homeomorphism h : 5i - S 1 conjugating f and g. This
proves the weak structural stability of f. O

REMARK 1. By suitably choosing the neighborhoods U2 and the points x E


12(f) and x' E 12(g) we can ensure that h moves points by a distance not exceeding
a given e > 0. Therefore, f is also a structurally stable diffeomorphism.
REMARK 2. If f E Diffr(S1), r > 1, has only a finite nonzero number of
periodic points, then the smallest period is the same for all these points. If all
the periodic points are hyperbolic, then their number is even: half sinks and half
sources. The proof that f is weakly structurally stable and structurally stable is
analogous to that for Lemma 3.1.
3.3. Instability of an irrational rotation number.
THEOREM 3.1. Suppose that a transformation f E P(J8) is continuous and
rot(f) E I[8 \ Q. Then for any t > 0
rot(R_t o f) G rot(f) < rot(Rt o f).

PROOF. Since 7(x + 1) = 7(x) + 1, f is a covering for some transformation


f E P(51). We show that there is a point x° E 81 such that for any connected
neighborhood U of it each of the open intervals in U \ {x°} contains points in
the positive semi-orbit O+ (x°) (Figure 5.7). Indeed, by Lemmas 2.2 and 2.3, f is
conjugate or semiconj ugate to the rotation Ra , c = rot (f) . If f is conjugate to
Ra, then the existence of the required point x° follows from the irrationality of c
216 5. TRANSFORMATIONS OF THE CIRCLE

(in this case any orbit of f is dense in S1) Suppose now that f is semiconjugate to
.

Ra by means of a continuous mapping h e P(S1). We take an arbitrary point xo


in the nonempty perfect set S1(f) that is not an endpoint of any interval h-1(x),
x e x (f , g). There is such a point in view of Lemma 2.3. Let U be an arbitrary
connected neighborhood of xo, and assume that O+(xo) is disjoint from one of the
intervals in U \ {XO} (denote this interval by tiC+). We show that h(U) is a point.
Indeed, if h(U) contains an interval, then the irrationality of a implies that h(U)
contains points of the form Ra [h(xo)] for a countable family of numbers k e N.
From this and the monotonicity of h it follows that u+ n O+(xo) ; 0. Therefore,
h (U) is a point, and xo E 1z(f) is an endpoint of the interval h -1 [h(U)], which
contradicts the choice of xo. The contradiction proves that O+(xo) intersects both
the intervals in U \ {x0}.

xo) u
Xo

FIGURE 5.7

_Assume now that the theorem is false. According to Theorem 1.2, the function
rot (Rt o f) = rot (f + t) is monotonic_ally nondecreasing and continuous. Therefore,
there is a to > 0 such that rot(Rt o f) = rot(f) for all 0 < t < to (or -to < t < 0,
in which case the proof is analogous).
We take a point xo E ir-1(xo). By the properties of xo, there exist k e N and
r e Z such that
xo + r - to/2 < fk(xo) < xo + r.
Then rot(f) < r/k in view of Lemma 1.3.
f fl
Since Rt0 o f > 7, it follows that (R0 o f) n > for all n E N. Therefore,
7)k k-i (xo)
(Rto o (o) = (Rto o f) o (Rto o 7)
7k1 (o) = 7k +
> CRto f) Cxo) to.
k
This and the inequality f (xo) > xo + r - to/2 imply that (Rt0 o f) k (xo) > xo + r +
to/2. Lemma 1.3 again gives us that rot(_Rto o f) > r/k. Then rot(f) = r/k E Q,
which contradicts the irrationality of rot (f) . El

It follows from Theorem 3.1 that a structurally stable or weakly structurally


stable orientation-preserving diffeomorphism of the circle has a rational rotation
number.
LEMMA 3.2. Suppose that a continuous transformation f e P(51) has a ratio-
nal rotation number r/k (k e N). Then f has a periodic point of period k.
PROOF. We first consider the case rot(f) = 0 and show that f has a fixed
point. Take a covering transformation f E P(IR) for f with rot(f) = 0, and assume
that f (x) ; x for all x e JR. For definiteness let f (x) > x. The monotonicity of
7 gives us that x < 7(x) < < T(x) < 7Th(x) < , that is, {7Th(x)}o
3. STRUCTURALLY STABLE DIFFEOMORPHISMS 217

is a monotonically increasing sequence. According to the equality rot(f) = 0 and


n+1(x) n def xo
Lemma 1.3, f <x + 1 for n e N. Therefore, the limit limn.,o f (x)
1n+1
exists. Since f is continuous, it follows that 7(xo) = limn (x) = xo, and
hence lr (xo) is a fixed point of f.
Suppose now that rot(f) = r/k ; 0. We take a covering transformation f for
f, with rot(f) = r/k. Then g(x) def jk(x) - r e P(IR) and rot(g) = k rot(f) - r =0
(Lemma 1.2). According to what was proved above, the transformation f has a
fixed point. El
COROLLARY 3.1. A structurally stable or weakly structurally stable diffeomor-
phism of the circle has a periodic point.
3.4. Openness and denseness of the set of weakly structurally stable
diffeomorphisms. In the space Diff+(S1), r > 1, of orientation-preserving C''-
diffeomorphisms of the circle we consider the subset W (S1) of diffeomorphisms
having periodic points, each hyperbolic.
LEMMA 3.3. 1) The set (S1), r > 1, is open and dense in Diff+(S1).
2) The set of weakly structurally stable orientation-preserving diffeomorphisms
of the circle coincides with (Si).
There is a proof of this result in the books [62], [64], and [65], so we omit it.
3.5. Classification of weakly structurally stable diffeomorphisms.
THEOREM 3.2. Two weakly structurally stable diff eomorphisms f, g e Diff +(S1)
are conjugate if and only if they have the same number (necessarily even) of periodic
orbits, with the same minimal period.
The proof is analogous to that of Lemma 3.1, and we omit it.
Thus, in the space of diffeomorphisms of the circle the weakly structurally stable
diffeomorphisms form a dense open set. The (even) number of periodic orbits and
the minimal period of any periodic orbit make up a conjugacy invariant of a weakly
structurally stable diffeomorphism (the number of periodic points can be taken
instead of the minimal period).
Remark. Diffeomorphisms of the first degree of structural instabil-
ity. Just as relatively structurally stable flows (that is, flows of the first degree of
structural instability) are singled out in the space of structurally unstable flows,
diffeomorphisms of the first degree of structural instability are singled out in the
space of diffeomorphisms of the circle.
A structurally unstable C''-diffeomorphism f e Diff + (S1) \ W (S1) is called a
diffeomorphism of the first degree of structural instability if for any > 0 there is
a neighborhood U of this diffeomorphism in Diff + (S1) such that any structurally
unstable diffeomorphism g E U is conjugate to f by means of a homeomorphism
that is E-close to the identity diffeomorphism in the space Homeo(S1).
Aranson (Generic bifurcations of diffeomorphisms of the circle, Differentsial'nye
Uravneniya 23 (1987), 388-394; English transl. in Differential Equations 23 (1987))
obtained necessary and sufficient conditions for a diffeomorphism to belong to the
set of diffeomorphisms of the first degree of structural instability and, by using a
well-known theorem of Herman [89], proved that the set of diffeomorphisms of the
first degree of structural instability is open and dense in the set of all structurally
unstable diffeomorphisms of the circle.
218 5. TRANSFORMATIONS OF THE CIRCLE

§4. The connection between smoothness properties and


topological properties of transformations of the circle
The first part of the section goes back to Poincare's conjecture about the exis-
tence of analytic nontransitive diffeomorphisms of the circle with irrational rotation
number [68]. In 1932 Denjoy proved that any C2-diffeomorphism with irrational
rotation number is transitive (thereby disproving Poincare's conjecture). In 1984
Yoccoz generalized Denjoy's theorem to analytic homeomorphisms.
The second part of the section is devoted to smooth conjugacy of diffeomor-
phisms of the circle (invariants of smooth conjugacy are singled out for weakly
structurally stable diffeomorphisms, and a result of Herman is presented for a tran-
sitive diffeomorphism).
4.1. Continued fractions. We present the needed facts about continued
fractions (see [75] for proofs).
A continued fraction is defined to be an expression of the form
1 def
(4.1) ao +
1
a 1 -}-
a2+...
where the numbers a2 are called the partial quotients or elements of the expansion.
Any real number a e (0, 1) can be uniquely represented as a continued fraction
(4.2) [ai, a2,..

with positive integer elements a2. If a is rational, then the continued fraction (4.2)
is finite, and if a is irrational, then (4.2) is infinite.
The elements a2 of the expansion (4.2) can be obtained with the help of the
Gauss transformation G : (0, 1) --* (0, 1),
G(x) = 1/x-[1/x], x E (0,1),
where [z] is the integer part of a number z. If for a real number a e (0, 1) the
iterates G(a), ... , G1(a) are not equal to zero, then

=
,a2
[]
,.. [Gn1a] . ,a =
In what follows we assume that a e (0, 1) \ Q. Then a can be written as an
infinite continued fraction.
For a = [ai, a2,. ..] the fractions pn/qn = [ai,. . . , an] are called the conver-
gents of the continued fraction. The numerators pn and denominators qn of the
convergents satisfy the recursion relations

po=0, q=1, P1=1, ql=al,


pn = anpn-1 + pn-2, n 2,
(qn=anqn_1+qn_2, n 2.

It follows from (4.3) that

(4.4) qn > 2 n21 ,


4. CONNECTION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES 219

1 pn < 1 < 1
qn (qn + qn+1) - a- q2
qn gngn-1
that is, the convergents of the continued fraction are rational approximations of a,
and
a- pn >
qn
a- qn-1
pn-I-1

(each convergent is a better approximation than the previous one).


The convergents with even indices form a monotonically increasing sequence,
those with odd indices form a monotonically decreasing sequence, and both se-
quences converge to a (Figure 5.8).

o(, p
per,- _1

Q
21C+1 2K'- S

FIGURE 5.8

The convergents of a continued fraction are best approximations of the second


kind for a; that is, if p/q ; pn/qn and 0 < q < qn, then
(4.6) qna-pn <Iqa-pI.
Conversely, every best approximation of the second kind for a is one of its con-
vergents. This means that the inequality (4.6) is valid for all fractions of the form
p/q with denominators 0 < q < qn-1, q ; qn, and the number qn+1 is the smallest
positive integer among the q e N such that I qna - pn I > Iqa - pi for some p e 7L:
(4.7) qn+1 = min{q E N : Iqa - pnI > I qa - pI }.
The fractions (pn + 2pn+1) / (qn + 2qn+ 1) , 0 < i < an+2 , are called intermediate
quotients. They are situated between the convergents pn/qn and Pn+1/qn+1 on the
number line, and they form an increasing sequence for even n (Figure 5.9) and a
decreasing sequence for odd n.

Q n+ 2 intervals
d
Pn
-- P+L
n pn+
c
qn+ qn+ 9n+ i, n+i q n+ 2

FIGURE 5.9

REMARK. If an+2 = 1, then there are no intermediate quotients between the


convergents pn/qn and pn+1/qn+1
220 5. TRANSFORMATIONS OF THE CIRCLE

4.2. The order of the points on the circle. Let Ra : Si --, Si be a


rotation of the circle by an irrational number a E (0, 1), and let pn/qn be the
convergents of a, n E N. The following arrangement of the iterates of an arbitrary
point x0 E S1 under the action of the rotation follows from the theory of continued
fractions. Denote by [x0, Ran (XO)] the arc between x0 and Ran (x0) containing the
point Ran+2 (XO). Then on this arc there are no points Ra (xo) with qn+1 < i <
qn+1 + qn, and, what is more, the first point Ra (xo) with positive minimal i > qn+1
falling on the arc [x0, Ran (xo)] is the point Ran+1+Qn (xo) (Figure 5.10 for even n),
and there are no points Ra (xo) with 0 < i < qn+1 + qn on the arc
[Rn+1+n (x0), Ran (x0)] C [x0, Ran (x0)]

between R' + Qn (xo) and Ran (XO). Indeed, if


Ra (xo) E fRan+l+Qn (xo)
R«n (x0)]

for some 0 < i < i +1 +qn, then it follows from (4.6) that i > qn. Thus 0 <i - qn <
qn+1, and the point Ra Qn (xo) lies on the arc [R' (x0), xo] between Ran+1 (x0) and
x0, closer to x0 than R' (x0), which contradicts the definition (4.7) of qn+1. If
Ran (x0)] Ran+1+Qn
R(x0) E [x0, Ran (x0)] \ [R+' +Qn (x0), [x0, (x0)]

for some qn+1 < i < qn+1 + qn, then 0 < i - qn+1 < qn and Ra2q
-,L+1
(x0) E
[x0, Ran (xo)], which contradicts the definition of qn.

Xo
r(x)
4

Rqn+s
(xe) R+t(x0) RF1
a
+ n Cxo)
a

FIGURE 5.10. If an+2 = 1, then qn+2 = qn+1 + qn.

Since Ra is an isometry, the points RaQn+1+Qn (x0), k = 1,.. . an+2, lie on the
arc [x0, Ran (xo)] in the order pictured in Figure 5.11 (n is even, and an+2 > 1).

Q n+2 intervals

xo RQn+ c1
R41(xo)
F cxo Ra cxo 1
a (x0)
a

FIGURE 5.11

Denote by Iqa the length of the circular arc between x0 and Rai (XO) not
containing R' (x0), i > 1. Then an analysis of Figure 5.11 leads to the equality

I= Iq+2aIf.
4. CONNECTION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES 221

LEMMA 4.1. Let Ra : Si --* Si be a rotation with irrational a, and let pn/qn be
the convergents of the continued fraction expansion of a. Denote by Jn the open arc
with endpoints RaQn (x0) and Ran (x0) that contains x0, where x0 is an arbitrary but
fixed point, and by In C Jn the open arc with endpoints x0 and R; n (x0) (Figure
5.12). Then:
1) the arcs Ra(In), 0 < i < qn+1, are disjoint;
2) each point of the circle belongs to at most two arcs in the family Ra (Jn),
-n+1 Qn (x0),
3) the points Ra Qn (x0), Ra Qn+1 (x0), and Ran (x0) are located
in cyclical order on the circle.

xo

11.

FIGURE 5.12. (n even)

PROOF. Assume that 1) is false, that is, Ra (In) fl Ra (In) ; 0 for some 0 <
k, j < q+1. For definiteness assume that j < k. Then Ra-3(In) fl In ; 0, and
hence Ra c(xo) is closer to x0 than Ran (x0). Since 0 < k - j < q7+1, we get a
contradiction to the definition (4.7) of q+1.

The assertion 2) follows from 1) and the inclusion Jn C In U Ran (In) U {x0}.
The assertion 3) follows from the cyclical arrangement of the points Ran (XO),
Ran+ 1 +Qn (x0) , xo,
Ran+1(XO), and Ra Qn (x0) on the circle (Figure 5.10) and the fact
that Ra is an isometry. U

Since a homeomorphism of the circle with irrational rotation number is conju-


gate or semiconjugate to a rotation by means of a monotone transformation, Lemma
4.1 gives us
COROLLARY 4.1. Let f be a homeomorphism of the circle with irrational rota-
tion number a, and let qn be the denominators of the convergents of the continued
fraction expansion of a, n E N. Denote by Jn the open arc with endpoints f -an (x0)
and f Qn (x0) that contains x0, where x0 is an arbitrary but fixed point, and denote
by In C Jn the open arc with endpoints x0 and f -an (x0) . Then:
1) the arcs f i(In), 0 2 < qn+l, are disjoint;
2) each point of the circle belongs to at most two arcs in the family f Z(en),

3) if f is a Denjoy homeomorphism and J is an adjacent interval of the Cantor


set 11(f), then the intervals f -Qn (J), f -Qn-Qn+1 (J), .J, f-Qn+1 (J), and f Qn (J) are
arranged in cyclical order on the circle.
4.3. The theorem of Denjoy. In [68] Poincare presupposed that there is an
analytic diffeomorphism of the circle without periodic points and with a nowhere
dense (hence Cantor) limit set. This conjecture gave birth to an entire direction
222 5. TRANSFORMATIONS OF THE CIRCLE

involving the interrelation of smoothness properties and topological properties in


the qualitative theory of dynamical systems (see the survey in [25]). A half-century
later Poincare's conjecture was refuted by Denjoy [82]; namely, he proved the fol-
lowing theorem.
THEOREM 4.1. Suppose that f e Diff + (S1), r > 1, has an irrational rotation
number and a derivative D f of bounded variation. Then (f) = Si. That is, f is
a transitive diff eomorphism.
PROOF. Assume the contrary. Then (f) is a Cantor set by Corollary 2.3. Let
Go C S1 \ (f) be an adjacent interval, and denote by xo its left endpoint (the
positive direction on S1 is induced by the positive direction on the line R and by
the covering it : IR --* 51). Let In, be an arc satisfying the condition in Corollary
4.1. Since Go fl SZ (f) = 0 and xo E SZ (f), it follows that Go C In for all odd n e N
(Figure 5.13). The right-hand endpoint of the arc In is equal to f -qn (xo) (by the
def
definition of the arc In). Therefore, the adjacent interval f -qn (Go) = G_qn adjoins
In.

Go G-4 n
xO
p 9n
cXo
Y
In

FIGURE 5.13. (n odd)

Let Gk, k E Z. It is a consequence of the mean value theorem that


m(Gk+1) = Df(zk)m(Gk), where zk E Gk (m(U) is the Lebesgue measure of a set
1.L C Si).
We form the Denjoy sum

[lnDf(z_+) - In D f

Then

-
_
qn-1

In Df () -_In
D f (zi)
qn-1
Df (z-qn+2)
D f (zi )
n z_o i=0
qn-1 [m(Go 2
m(G-gn+z+1) m(Gz) = In
- In
m(G-qn+z) m(GZ+1) m(G-gn)m(Ggn )
z=0

Since the lengths m (Gk) of the adjacent intervals Gk tend to zero as k --* oo, it
follows that

(4.8) lim = +oo.


n--> o0
n
4. CONNECTION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES 223

We estimate the Denjoy sum in another way. By Corollary 4.1, 1), the arcs
f Z (In) (0 < i < qn) are disjoint and have their endpoints in SZ (f), so the arcs
f Z
(In) U f z (G-qn) = f z (In) U G_Qn+z (0 < i < qn) are also disjoint. This implies
that n < varsl In D f for all n e Z. According to the hypotheses of the theorem,
D f > const > 0 and varsi D f < oo, and hence varsl In D f = M < oo. This gives
us that n < M < oo, which contradicts (4.8). El

4.4. The theorem of Yoccoz. In 1981 Hall [87] gave a construction of a C°°-
homeomorphism of the circle with a single critical point (that is, a point at which
the derivative is zero; this homeomorphism is thus not a C1-diffeomorphism), an
irrational rotation number, and a Cantor limit set. Therefore, the group Diff + (S1)
in Theorem 4.1 cannot be replaced by Homeo+ (S1), r > 1. In 1984 Yoccoz [112]
showed that, nevertheless, under certain restrictions on the behavior of a smooth
homeomorphism in a neighborhood of critical points it cannot have a Cantor limit
set. In this subsection we present the result of Yoccoz.
We say that an f e Homeo'' (S1), r > 1, satisfies the Yoccoz conditions if:
1) f has finitely many critical points x1,.. , xl e 81; .

2) log D f has bounded variation on any compact interval not containing critical
points;
3) for each critical point x2, i = 1, ... ,1, there exist strictly positive constants
AZ, BZ, and CZ and an E'-neighborhood tL of xi such that
a) BZ I t I ci < D f (x2 + t) < AZ I t I ci for ti < i ,

b) the function (D f)-1/2 is convex (downward) on each of the intervals in


tL2 \ {x2 } .

REMARKS. I) If f e Homeo'' (S1) , r > 2, then the convexity of (Df)/2 means


that the function
1 D2f l'
- 2(/j)3
is increasing on each of the intervals in tLi \ {x2 }.
II) If f e Homeor(S1), r > 3, then the convexity of (D f)-1/2 is equivalent to
the condition
1 " Sf

where
D3 f 3 (D2f\\2
S(f) D ff Df
is the Schwarzian derivative. Consequently, S f < 0 in the neighborhood U2, i =
1,.. .,l.
III) If f e Homeo°° (S1), then the condition 3) holds when at each critical point
f is not flat (that is, some finite-order derivative at the critical point is nonzero).
IV) Obviously, the condition 3) holds for an analytic homeomorphism f.
Denote by 1(51) the set of all possible compact -connected segments of 51.
For brevity we denote the Lebesgue measure meas (I) of a segment I E I (S1)
by m(I).
We define the Yoccoz function on Homeo'' (S1) x I (S1), r > 1, by

M(f,I) = m(f (I ))[Df(a)Df(b)]-1/2

m(I)
224 5. TRANSFORMATIONS OF THE CIRCLE

(f e Homeor (S1), I E 1(81)) if the endpoints a and b of I are not critical points
of the C''-homeomorphism f, and by M (f , I) _ +oo otherwise.
It is not hard to see that the Yoccoz function has the multiplicative property
M(f o g, I) = M(f, g(I)) M(g, I).
Indeed,

M(.f o 9, I) = m(m( j I)) [Df ° 9(a)Df o


= m[s(I)]
- m[g(I)] m(I) 1Df(a)Df(6)]
= M(f, g(I))M(g, I)
LEMMA 4.2. Suppose that f e Homeo'' (S1), r > 1, satisfies the Yoccoz con-
ditions, and let I C 81 be a compact connected interval with endpoints a, b e S1.
Then:
1) if I does not contain critical points, then
exp (- 2 varl log D f) < M (f , I) < exp (2 varl log D f) ;
2) if I lies in the EZ -neighborhood Ui of a critical point x2 but does not contain
x2f then
M(f,I) > 1;
3) if I lies in U2 and contains x2, then
BZ
M >
f'I-2A(C+1)'
2 2

4) if I contains one of the intervals (x2 - Elf xz - E2/2) or (x2 + E2/2, xi + E2),
then there exists a S > 0 independent of i such that
M(f,I) S/D,
where D = maxsl D f.
PROOF. 1) Let v = varsi log D f (x). Since m(f (I)) = D f (c)m(I), c e I, it
follows that
log M(f, I) = log D f (c) - Z log D f (a) - Z log D f (b)
= 2 [logDf(c) - log D f (a)] + 2 [logDf(c) - log D f (b)].
From this, I log M(f, 1)1 < 2 v, and hence

e M(f,I) e.
1

2) Let u(t) = at + Q be the linear function such that u(a) _ (Df(a))- 2 and
u(b) _ (Df(b))- 2 (Figure 5.14). Since (Df)- 2 is a convex (downward) function,
i
it follows that u(t) > (Df(t))-2 on I.
It is not hard to verify that
dt b dt m(I)
,1a t2(t) - ,la (at+f3)2 u(a)u(b)
4. CONNECTION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES 225

FIGURE 5.14

Then

From this,
f b D.f (t) dt > 6
Ja u2(t)
dt - u(a)u(b)
m(I
)

M(.f, I) _ 2 , 6 Df (t) dt > 1.


3) The point x2 divides I into two intervals. Suppose that the smallest I1 of
the two has endpoints x2 and b (the proof is analogous for x2 and a). Then
Df(a)Df(b)
where t1 is the distance between x2 and b. Further,
b tl
m[f(I1)] = xi
Df(x +Z)
t dt >- B Z
p
tC%i dt =
BZ tG'i+1
ci+1 1

and

M I> r1[Df(ayJJf(b)1 -( 1)
2

> BZ tCi+1 1 A2t2ci - 2 = B2


- C2+ 1 1 2t 1 ( Z
1 ) 2A2(C2+ 1)
4) Denote by E the family of intervals [x2 - Ez, xz - Ei/2], [xi + EZ/2, x2 + EZ],
i = 1, ... ,1. Since these intervals do not contain critical points,
S def min{m[f (J)] : E E} >0.
Obviously, m(J) < 1 for J E E. Therefore,

M(f,I) . D) = 6/D. U

THEOREM 4.2. Suppose that f e Homeo''(S1), r > 1, has rot(f) E IR \ Q and


satisfies the Yoccoz conditions. Then (f) = Si.
226 5. TRANSFORMATIONS OF THE CIRCLE

PROOF. Assume the contrary. Then 1(f) is a Cantor set. We take an adjacent
interval 1 C S1 \ 1(f) and a point xo E I
Let qn be the denominators of the convergents of the continued fraction expan-
sion of rot(f ).
Denote by J the arc between f- Qn (xo) and f Qn (x0) containing x0. According
to Corollary 4.1, 3), the intervals f -Qn (J), f -Qn+1 -Qn (J) J f -Qn+1 (J) and f Qn (J) are
located in cyclical order on S1. Therefore, J contains the intervals f-Qn+1 -Qn(J),
J, and f -Qn+1 (J) .
Since f Qn-1 [f_n+i -Qn (J)] = f-an (J) and f n+1 [f_n+1 (J)] = J, there exist
points a, b E S1 such that
rn[f_n (J)] = f(a)rn[f_'_ (a)],
rn(J) =

Denote by I the interval with endpoints a and b that contains xo. According
to the foregoing, I C
Let us estimate the Yoccoz function M(f 9n}1, I), We have that

I) _ 'n[.fm(j)(I )J [Jf(a)Jf(b)J 2

i
- (I)]
rn[f-n()]
qn (J)] (J)] \
- rn(I) m()
i
)
frn[f_'ln+1(J)]
_______
- [m()]
m[fn()}
The adjacent intervals f -Qn (J), n E N, do not intersect, and therefore the sequence
{m[ f -Qn (J)]}.1 converges to zero. By passing to a subsequence if necessary, we can
assume that m[ f -Qn+1 (J)]/m[ f 9n (J)] < 1. Thus, M(f Qn+1, I) -- 0 as n -- +oo.
We estimate the function M (f Qn+1 , I) in another way. By the multiplica-
tive property of the Yoccoz function, M (f Qn+ 1 , I) = M (f ,12) , where
12 = f 9 (I) . We partition the family of intervals Ij, j = 0, ... , qn+1 - 1, into
four subfamilies:
= {I2 disjoint from the (e2/2)-neighborhoods of the critical points x2i i =
1,...,1},
= {I2 lying in the e2-neighborhood u2 of x2 but not covering x, i = 1, ... , l },
= {12 lying in U2 and covering x2},
4 = {I2 intersecting the (e2/2)-neighborhoods of the points x2 and not con-
tained in U2 }
Let U be the complement of the (e2/2)-neighborhoods of the critical points
x2i i = 1,... , 1. In view of the condition I C J and Corollary 4.1, 2), each
point in U is covered by at most two intervals 12 in the family Fi. Therefore,
>j var13 log T f < 2 vary, log D f . Lemma 4.2, 1) gives us that

def k1.
H M (f , Ii) > ex(_ 2 var log D f > exp (- vary, log D f) =
I E91 I E91

By Lemma 4.2, 2), [TIE92 M(f, 1.


4. CONNECTION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES 227

According to Corollary 4.1, 2), there are at most 21 and 41 intervals in the
respective families 33 and 3 4 (l is the number of critical points). Therefore, by
Lemma 4.2, 3), and 4),
B Zl def
= k2,
H M(f,Ij) [2Ac+1
Ii E 3"3

where A = max{A2, 2 = 1, ... , l }, B = min{B2, 2 = 1, ... , l }, C = max{C2,


i = 1,...,1}, and

Ii E4
H M (f , Ij) >- -Db def
()41k3.
We note that the constants k1, k2, and k3 are strictly positive and independent of
n. Then
M(f qn+1, I) > k1k2k3 > 0
for all n E N, which contradicts the previously proved relation M(f qn+1, I) - 0 as
n -* 00. 0

4.5. Corollary to the theorem of Yoccoz for Cherry transformations.


We formulate the Yoccoz conditions for a Cherry transformation f E Ch(Sl):
1) f has finitely many points of discontinuity z1, ... , zs E S1, and is of smooth-
ness class Cr' on the set S1 \ U7=1 z2i r > 1;
2) f has finitely many isolated critical points x1, ... , xl E S1 and finitely many
intervals of constancy (a1, b1), ... , (aj, bk) C S1;
3) log D f has bounded variation on any compact interval not containing critical
points, intervals of constancy, nor points of discontinuity;
4) if z is a critical point or a point of discontinuity of f, then there exist
constants A = A(z) > 0, B = B(z) > 0, and C = C(z) and an e-neighborhood u
of z such that
a) Bitic < Df(z+t) < AItIC for 0 < ti <
1
b) the function (Df)- 2 is convex (downward) on each of the intervals in U\ {z}
(Figure 5.15);
5) for each interval of constancy (a, b2), i = 1, ... , k, of f there exist constants
A2 > 0, B2 > 0, and C2 > 0 and e2-neighborhoodsU2 (a2) andU2 (b2) of the respective
points a2 and b2 such that
a) BZ l tl Ci < D f (a + t) < AZ I tICi for -e2 <t < 0 and Bj l tl Ci < D f (b + t) <
AZltIci for 0 < t < e2,
1
b) the function (D f)-2 is convex (downward) on each of the intervals in'LLi(ai)U
'u2(bz) \ [a, bz]
COROLLARY 4.2. Suppose that f E Ch(S1) has rot(f) E I[8 \ Q, satisfies the
Yoccoz conditions, and does not have intervals of constancy. Then f does not have
gray cells.
The proof is completely analogous to that of Theorem 4.2 (Yoccoz), and we
omit it.
4.6. The Herman index of smooth conjugacy to a rotation. In this
subsection we consider the problem of C''-conjugacy of a smooth diffeomorphism
of the circle to a rotation. Significant results toward solving this problem were
228 5. TRANSFORMATIONS OF THE CIRCLE

Qi Po
i.
`
FIGURE 5.15

obtained by Herman [89], who proved Arnol'd's conjecture about the existence of a
set A C [0, 1] of measure 1 such that if an analytic diffeomorphism of the circle has
rotation number in A, then it is analytically conjugate to a rotation. Herman also
introduced the index of C''-conjugacy of a diffeomorphism to a rotation. We present
Herman's result only in connection with C1-conjugacy of an orientation-preserving
C1-diffeomorphism of the circle (with irrational rotation number) to a rotation.
DEFINITION. Two diffeomorphisms f, g E Diff' (S1), k > 1, are said to be
Cr-conjugate (r > 1) if there exists an orientation-preserving C''-diffeomorphism
h : S1 -* S1 such that h o f = go h.
For a Cr-diffeomorphism f E Diff' (S1) we define the Hermann index Hr (f) E
Ilk U {+oo} to be (r > 1)

Sup D.f'C*-1 = SuPD.fO + ... + Dr.fn0


nE7L nEZ

LEMMA 4.3. If f E Diff r (S1), r > 1, is Cr - conjugate to a rotation, then


Hr (f) <+00.
PROOF. By assumption, f = h-1 o Ra o h, where h : S1 51 is an orientation-
preserving Cr-diffeomorphism. Then f fl = h-1 o Ra o h, n E 7L. Since DRa - 1
and DzRQ - 0 (i = 2, ... , r) for n E Z, the required follows from the formula
for differentiating a composition function and the fact that max{ I h l (J r , Ih1Icr} <
+oo. 0

Herman proved that the inequality Hr (f) < +00 is also a sufficient condition
for a C7'-diffeomorphism f to be Cr-conjugate to a rotation. We give a proof for
the case r = 1. Let us first find an equivalent formulation for Cr-conjugacy of a
diffeomorphism f to a rotation. If h o f = Ra o f for some h E Diff 1(S1), then
(T)f o f )D f = (DRa o h)Dh = Dh. From this, log Dh o f + log D f = log Dh, or
log D f= log D h- log D h o f.
We now proceed to the proof of a lemma due to Gottschalk and Hedlund [84].
LEMMA 4.4. Let f be a minimal homeomorphism of a compact manifold M,
and let x : M --p Ilk be a continuous function. Then the following statements are
equivalent:
1) there exists a continuous function cp : M - Ilk such that x = cp - cp o f ;
4. CONNECTION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES 229

2) there exists a point xo E 1Vt such that

sup
nEN
f

PROOF. 1) == 2) because
n n n
X ° f f (x0)
2=0 2=0

_ ko(xo) - P o fi(xo)I < 2m <+00.


We show that 2) implies 1). Let F: M x Ilk - M x Ilk be defined by F(x, t) _
(f(x),-x(x)+t), and RA : Mx Ilk - Mx Ilk by RA (x, t) _ (x,t+A). Since f is a
homeomorphism, so is F. It can be checked directly that
FoRI =RA0F.
We get from the definition of F that Fn (xo,
N
0) _ (f n (xo) , - Z o x o fZ (xo)) .
According to the condition 2), the closure N of the set {FTh(xo, 0), n E N} is a
nonempty compact subset of the space M x R. Since F is a homeomorphism, the
set N is invariant. Therefore, N contains a minimal subset N with respect to F.
Since the homeomorphism f is minimal, the projection (x,.) H x of N on the
first factor coincides with the set M. We show that the subset N C M x Ilk is the
graph of a function; that is, for any x E M there is a unique y E Ilk such that (x, y) E
N. Assume that (x, y), (x, y + A) E N. Since RA (N) = RA o F(N) = F o RA (N), it
follows that RA (N) is an invariant compact set with respect to F. By assumption,
RA (N) f1 N 0, so RA (N) = N by the minimality of N. From this, Rna (N) = N
for all n E N. If A 0, then the equality Rna (N) = N contradicts the compactness
of N.
Thus, N is the graph of a function p: M -f R. Since N is compact, (p is
continuous. It can be verified directly that the equality x = cp - cp o f holds
on the set {fTh(xo), n E N}. Indeed, cp [fTh(xo)] _ - Z o x o f Z (xo) and (p o
f [f n (xo )] _ (P [f'(xo)] _ - Zo x o f Z (xo) , and therefore (P [f(xo)] - (P o
f [f n (xo)] _- Z o x o f Z (xo) + Z o x o f Z (xo) = [fTh(xo)]. The set {f(xo),
n E N} is dense in M because f is a minimal homeomorphism, so the equality
x = cp - cp o f holds on the whole manifold M. 0

THEOREM 4.3. An orientation-Preserving Cl-diffeomorphism of the circle with


irrational rotation number is Cl -conjugate to a rotation if and only if Hl (f) <+00.
PROOF. NECESSITY. If f is C'-conjugate to a rotation, then Hl (f) <+00 by
Lemma 4.3.

SUFFICIENCY. Let Hl (f) < +oo. Then f - f n (y) I < lDfThIoIx - y <
Hl (f)Ix - for any x, y E S1 and any positive integer n. Therefore, the family
I

{fn} 1of mappings f n is uniformly continuous. In view of Lemma 2.5 the homeo-
morphism f is conjugate to the rotation Ra with a = rot (f) E Ilk \ Q. Consequently,
f is minimal.
Since f E Diff 1(S1), x aef log D f is continuous. Moreover, 1/H, (f) < D f n <
Df-n[fn(x)]
H1(f) for any n E N because = 1/Dfn(x) < sup H,(f)
230 5. TRANSFORMATIONS OF THE CIRCLE

for n E N, and hence supn E N I log D f n o< +oo. But log D f n= Z o log V f of z=
Z o xof. Z According to Lemma 4.4, there exists a continuous function cp : S1 - Ilk
such that log VI = x = cp - cp o 1. Obviously, the last equality is satisfied by
any function cp, = cp + c for any constant c. Take c such that fo eH dx =
def
eC f0 e ° dx = 1. Then the transformation h(x) fo dx is covering for
def
some C1-diffeomorphism h : S1 -* S1, and log Dh = cp(x)+c cpl (x). Substituting
the function cp, = log Dh in place of cp in the equality log D f = cp - cp o f , we get that
log D f= log D h- log D h o f, which gives us that D h= (Dh o f) D f. Consequently,
the diffeomorphisms h and ho f have the same derivative. Therefore, Rp oh = ho f
for some E R. By Lemma 1.5, fi = rot (f) . U

We note that Theorem 4.3 is valid also for a diffeomorphism f with rational
rotation number.
We give without proof another result of Herman.
THEOREM 4.4. An orientation-preserving C1-diffeomorphism f of the circle is
C1-conjugate to the rotation Ra, a = rot(f ), if and only if KDfTh - loo -* 0 as na
(mod 1) -* 0.

§5. Smooth classification of structurally stable diffeomorphisms


Recall that structurally stable diffeomorphisms of the circle are characterized
by rational rotation numbers and hyperbolicity of all cycles. If the rotation number
of a diffeomorphism F is equal to T = p/q, then Fq has only fixed points. Therefore,
the problem of classification with respect to conjugacy reduces to classification of
just such diffeomorphisms.
The number of fixed points is a unique topological invariant of orientation-
preserving structurally stable diffeomorphisms. The multipliers (derivatives) at
fixed points are obvious supplementary invariants in the smooth classification. How-
ever, it turns out that in addition to these invariants there is a further "massive"
functional invariant ("modulus") of the smooth classification. Its construction uses
the existence of an invariant covering on whose elements the diffeomorphism is
smoothly conjugate to a standard model -a linear mapping of the line.
Here we consider classification with respect to the group of orientation-preserv-
ing diffeomorphisms.
5.1. Pasting cocycles. Let us fix a collection
P={p1,...,p2k}
of points on the circle. We assume that in correspondence to the orientation
pl < ... < P2k .
Further, let
0 < A2z_, < 1 < A2z (1< i< k)
be real numbers. Denote by Dr (p, A) the set of all C''-diffeomorphisms F : S1 -* S1
with fixed points p,, ... , P2k and multipliers

F' (pz )
5. SMOOTH CLASSIFICATION OF STABLE DIFFEOMORPHISMS 231

Thus, the diffeomorphisms F are orientation-preserving, the fixed points p2i-1 are
sinks for them, and the points p22 are sources. _
We remark first of all that a diffeomorphism F E Dr' (p, A) is "pasted together"
from linear diffeomorphisms of the line.
Namely, we set
U2 = (Pi-i,Pi+i), po = p2k, p2k+1 = pl
The arc U2 is clearly invariant under F. In the case k = 1 (two fixed points)
Ul=S1\{p2}, U2=S1\{pl}.
LEMMA 5.1. For any a < 1 there exist orientation-preserving homeomorphisms
. (U,p) -* (R1, 0), Z E COa
such that
(F)(x) = Ax (x E R1).
If r > 2, then the Z can be chosen to be Cr-diffeomorphisms.
PROOF. Let Z be local transformations of the corresponding class (see Theo-
rem 1.1 in Chapter 4). Then
Z (Fz) = (z)
for points z E S1 sufficiently close to p2. By this equality the transformation Z
can be extended successively to the whole arc.
The rest of our arguments are somewhat different for the case of two fixed
points and the case of more than two.
First let k = 1. We consider the diffeomorphism
MF: Ill \ {0} -* Ill \ {0}
of the punctured line defined by'
MF(x) =
We call this a pasting cocycle for F. This diffeomorphism belongs to the same class
as Z and satisfies the functional equation
MF(AiX) _ AzMF(x)
Consequently,
Y+ In ac
In ai x >0,
MF(x) - IxI°.
{ y-
In lxi
In ai x <0,
where the -yf (t) are 1-periodic functions of the same smoothness class as MF, and
In 2
A1.

e In
Moreover, the functions yf satisfy the conditions
(5.1) -y(t) <0 <'y_(t) (t E R')
'That is, (U,, 41) and (U2, 42) are interpreted as a C''-atlas on S' with transition function
MF.
232 5. TRANSFORMATIONS OF THE CIRCLE

Th
and

(5.2) 'y(t) +-yf(t) in A2 0 (t E II81).

Suppose now that k > 2. We define a pasting cocycle for F to be the collection
MF = (M2) of diffeomorphisms
MZ:R+-*R_ (1<i<2k)
defined by2
MZ = Z+1 2 1 (1< i< 2k), 2k+11
diffeomorphisms MZ are of the same class as Z and satisfy the equations
M2(A2x) = A,M2(x).
Consequently,
In x
M( x) = xei'y2 (j-,--)'
Z

where
9_ In a2+1
Z '
In AZ
and 'yz is a negative periodic function satisfying the condition
(5.3) 'y (t) + 'yz (t) In a2+1 O. 0

5.2. C°'a-conjugacy. As already mentioned, all the diffeomorphisms F E


D1 (p, A) are topologically conjugate to each other. This assertion can be refined:
THEOREM 5.1. All the diffeomorphisms F E D1 (p, A) are mutually conjugate
in the class C°'a for any a < 1.
N
PROOF. We first consider the case of two fixed points. Let F, F E Dr (p, X)
and let MF and MF be pasting cocycles belonging to the class CO,. a We set
G(x) = MF 1MF(x) (x E R1).
Then G(A1x) = A1G(x). Therefore,
In x
b+ In a1 x > 0 ,

G(x)=x { In lxi
b_ In a 1 x 0 ,

where the S are periodic functions of class CO,. aConsequently, extending the
definition of G by
G(0) = 0,
we get a C°'a-homeomorphism of the line. Let
'I' I Ui = 2 1 I UZ (i = 1, 2),
where Z and Z are linearizing diffeomorphisms for F and F, respectively. Since
1
2As in the case k = 1, the collection of (U2, ci) forms a C"'-atlas on S1, and the Mi are the
transition functions.
5. SMOOTH CLASSIFICATION OF STABLE DIFFEOMORPHISMS 233

in view of the definition of G, it follows that 'I': S1 - 51 is a well-defined homeo-


morphism of class CO,. aFurther,
'I'FIUi = 1
1 F'I'IUi.

N
Consequently, W conjugates F and F.
Suppose now that there are at least four fixed points, and let
MF = { MZ }, MF = { MZ }
N
be pasting cocycles for F and F, respectively.
Consider the homeomorphism
H2(x) = o M,1(x) (x E R_).
Then Hz maps I[8_ onto itself, and
(5.4) H2(A2x) _ A2H2(x).
Consequently,
lxI\
-
H2x
() =xr
1

In
A(xER- ),
2
where T is a 1-periodic function of class CO,. a Setting
In x
HZ(x) _ xT (xER), H(O)=O,
In a2

we get a C°'a-homeomorphism of the axis that commutes with a linear homeomor-


phism (see (5.4)). If we now set
H3 (x) = MZ o H2 o M2 1(x),
we get a C°'a-homeomorphism of the semi-axis IlL which, when extended in the
same way, gives a homeomorphism of the axis commuting with multiplication by
A3
Continuing this process, we get a sequence of C°'a-homeomorphisms
HZ : R1 - R1, H2(A2x) = aZH2(x),
such that

We now set
Y I Ui = Z (1< i< 2k)
where Z and Z are linearizing homeomorphisms for F and F, respectively. By the
construction of the homeomorphisms HZ, 'I' is a well-defined C°'a-homeomorphism
of the circle. Further, we have that
`YFI Ui = z 1 I Ui 2 1H2 I Ui

Z 1 I Ui = I Ui

F and F, and the theorem is proved. 0


234 5. TRANSFORMATIONS OF THE CIRCLE

5.3. Smooth classification. Here we construct an invariant smooth classifi-


cation for C''-diffeomorphisms. As before, we separate the case of two fixed points
from the case of more than two.
First assume that k = 1. We introduce the following equivalence relations in
the set {'y±} of pairs of periodic Cr- functions satisfying (5.1) and (5.2):
{'y±}'-'{±}
if
ye(t) = a'y±(t + b)
for some a> 0 and b E 1. The set of equivalence classes is denoted by L'' (A1 i )¼2).
Similarly, for k > 2 and for collections {'y} of periodic functions of class Cr
satisfying (5.3) we introduce the equivalence relation

{ /2} {'y2 }
'Y. (#I
- \i+ i 1 'Yi (t - 2 )
for some f 2 E Ilk with 132k+, - t1. As above, the set of equivalence classes is denoted
by L'' (A) . The equivalence of the collections {'y}, {±} and {'y2 }, {y2 } means that
the cocycles M and M constructed from them are connected by the relations
MZ (a2x) = a2+, M2 (x), a2 > 0
(let a2 = )¼Qi for k 2).
For a diffeomorphism F E D7'(, )) we denote by 1(F) the equivalence class of
the collection {'y} (of the pair {'y}) determined by the pasting cocycle MF.
THEOREM 5.2. Let r > 2. Then: _
a) for conjugacy of diffeomorphisms F, F E Dr (p, A) in the class C1 it is nec-
essary that
1(F) = 1(F),
N
and, conversely, if this equality holds, then F and F are conjugate in the class Cr;
b) for each element I E L7 '(A) there is a diffeomorphism F E Dr (p, A) such that
I(F) = I.
Thus, a) shows that 1(F) is a complete invariant of the smooth classification.
The asssertion b) is usually formulated as "a realization of the invariant".
Denote by Dr (p, A) the set of classes of C1-conjugacy of diffeomorphisms in
Dr(, A). Then Theorem 5.2 asserts that the mapping
I: Dr(, A) -* Lr(A)
is bijective. In this sense the set Lr ()) is a "moduli space" (of invariants) of the
smooth classification.
PROOF OF THEOREM 5.2. Suppose that the diffeomorphisms F and F are
conjugate, and
'I':S'-*
is a conjugating C1-diffeomorphism. It can be assumed that the arcs U2 are invari-
ant under 'I'. Then we get from the equality
N
'I'FIUi = F'I'U,
5. SMOOTH CLASSIFICATION OF STABLE DIFFEOMORPHISMS 235

N
by using the linearizations of F and F on the arcs U2 that
`Y 2 1 2 2 = 1 A2 `Y I u .

We set
'Tr
H2 =
Then H2 is a C1-diffeomorphism of the line commuting with multiplication by ).
In view of the smoothness of this diffeomorphism, it is linear:
H2 (x) = a2x (xER1),
where a2 E Ilk+, because it preserves orientation. We now get from the equalities
IuZ = 2
1 HZ 2 I Ui nu2+1 = 2+1 HZ+1 2+1 I Ui nU2+1 Consequently,

(5.5) M2(a2x) = a+,M(x).


This gives us the direct assertion in a).
Conversely, suppose that (5.5) holds. Let
IUz = 2 H2(x) = a2x (i1,...,k).
N
Then 'I': S1 -- S1 is a well-defined Cr-diffeomorphism conjugating F and F.
We prove the assertion b). Let {'y±} (or {'y2 }) be a representative of the given
class I E Lr (A) . From this representative we construct a cocycle M:
Inac\
Y+ ( lna
iVl lJl =ICI S
,y- J)
/i., iTi \

for k = 1, and
In x
M2 (x) = xe2 x>0
In 2
for k > 2. We "factor" the cocycle M2, setting
Mi (x) -
where j: U2 -* R1 is a Cr-diffeomorphism. Now let

FI u = 1(A) (1 i k)
Then F is a well-defined diffeomorphism of the circle in D, A). Obviously, MF =
I. The theorem is proved. U

5.4. Corollaries. Theorem 5.2 implies the existence for any r E [2, oo] of
diffeomorphisms in Dr (p, A) that are not conjugate in the class C1. What is more,
we have the following because Lr (A) has the cardinality of a continuum.
COROLLARY 5.1. The set D7'(, A) of C1-conjugacy classes has the cardinality
of a continuum for any r.
236 5. TRANSFORMATIONS OF THE CIRCLE
N _
COROLLARY 5.2. If diffeomorphisms F, F E C' (p, X) are conjugate in the class
C1, then they are also conjugate in the class C' .
In other words, there are no "intermediate" conjugacy invariants between C1
and C?'.
Since the equivalence of collections {'yj} preserves smoothness class, we get a
criterion for "smoothability" of a diffeomorphism F.
COROLLARY 5.3. A diffeomorphism F E D'(, A) is smoothly (C1-) conjugate
to a diffeomorphism of class Cq if and only if the pasting cocycle MF (that is, the
functions 'y± or 'yz) belongs to this class.
COROLLARY 5.4. The set DT (p, A) contains diffeomorphisms not conjugate in
the class C1 to any C1-diffeomorphism.
5.5. Conjugacy of flows. We recall that structurally stable flows on the
circle are flows either without any equilibrium states at all, or with hyperbolic rest
points.
As in the local situation the orbital classification of one-dimensional flows is
obvious. In the problem of conjugacy of flows, numerical invariants arise instead of
functional invariants.
Let Ft be a flow without equilibrium states. We denote by x(Ft) the "time
of a complete circuit" of the circle by the trajectories Ft (z); in other words, the
smallest number t > 0 such that
Ft(z)=z, z E S1.

This is well defined: x(Ft) does not depend on z and can be computed by the
formula

(5.6) (Ft)=f
1 (z)'
where
dFt (z)
t-o
The number x(Ft) is the only invariant of topological and smooth conjugacy of
flows without equilibrium states. Namely, if C''- flows Ft and Ft are topologically
conjugate, then
x(Ft) =x(Ft).
N
Conversely, if this equality holds, then the flows Ft and Ft are conjugate in the
class C''. A conjugating diffeomorphism can be given by the formula
(z) = Ft(z) F-t(z) (z) .
Here t (z) is the smallest t> 0 such that
Ft(zo) = z,
and the "initial" point zo of the construction is an arbitrary but fixed point.
Suppose now that Ft is a flow of class C?' with fixed points p1, ... , p2k and
multipliers 11i,. . , v2k. As in the case of diffeomorphisms, there are linearizing
.

transformations
i : (U2, pi) - (R',O)
5. SMOOTH CLASSIFICATION OF STABLE DIFFEOMORPHISMS 237

such that
1(x) = etvix(x e Ill).
These transformations belong to the class C0,! for r = 1, and to the class C'' for
r>2.
Setting
Nr2(x) -
(M(x) _ 1(x) in the case k = 1), we get the diffeomorphisms

Mi : R+ - 118 _

(diffeomorphisms of the punctured line in the case k = 1). Since

Mi (evitx) = eve+ltMi (x) (t e Il l ),

it follows that
Mi (x) = xei c (x e R+, ci <0),
where 92 = v2+1/v2. In the case k = 1

f C, C> 0
M(x) = xl°

where c+ <0 < c_.


Thus, the pasting cocycle of a flow has a very special form. For this reason
there is a simplification of the invariant. Namely, we set

in the case k = 1, and


2k
(5.7) x(Ft) =

fork>2.
THEOREM 5.3. Two structurally stable C1 flows with the same equilibrium
states and the same multipliers are conjugate in the class CO, for any a < 1.
N
Taking two C''-flows Ft and Ft with the same rest points pl, ... , P2k and the
same multipliers vl, ... , 112k, we get the next result.
THEOREM 5.4. Let r > 2. Then:
a) for the flows Ft and Ft to be conjugate in the class C1 it is necessary that

x(Ft) = x(Ft),
N
and if this equality holds, then Ft and Ft are conjugate in the class Cr;
b) for each x e R+ there is a C'' flow Ft (r > 2) such that x(Ft) = x.
Theorems 5.3 and 5.4 can be proved by the same scheme as for diffeomorphisms.
238 5. TRANSFORMATIONS OF THE CIRCLE

5.6. Inclusion of a difeomorphism in a flow. We recall that a diffeo-


morphism F is said to be includable in a flow Ft of class C'' if F = Fto for some
to.
The property of being includable in a flow of class C'' is an invariant with respect
to conjugacy. A diffeomorphism F with irrational rotation number is includable
in a C''-flow if and only if it is conjugate in C'' to a rotation. Indeed, if is a
conjugating diffeomorphism, then
Ft =
is a flow containing F. Here 'rt is a flow of linear diffeomorphisms (rotations).
Conversely, each C''-flow without equilibrium states is conjugate in C'' to a flow of
rotations (see §5.5). _
Further, since all diffeomorphisms F E DT (p, A) are conjugate in the class CO,,
they can be included in a flow of this class. It suffices to observe that DT (p, A)
contains at least one diffeomorphism includable in a C1-flow of diffeomorphisms.
However, if F is includable in a flow of class C1, then its pasting cocycle is
such that the functions 'yz ('± for k = 1) are constants. This is a necessary and
sufficient condition for includability in a C1-flow. If it holds, then F is includable
in a C''-flow.
Includability in a smooth flow thus turns out to be a fairly rare property (see
[37]).
5.7. _Comments. 1. The problem of a C1-classification of C1-diffeomorphisms
in D1 (p, A) has been left out of our considerations, as has classification of analytic
structurally stable diffeomorphisms.
The main obstacle to the construction of a C1-classification of diffeomorphisms
in D1 (p, A) is the absence of "local models". It is unclear how to describe the
"normal forms" of such diffeomorphisms with respect to local C1-conjugacy. For a
particular multiplier there is probably a continuum of C1-diffeomorphisms that are
pairwise not smoothly conjugate in a neighborhood of a fixed point.
As for analytic conjugacy, the scheme we have presented is perfectly suitable
also here when the local (one-dimensional) analytic theorem of Poincare is taken
into account (see §8 of Chapter 4).
2. It is no accident that the same notation x(Ft) appears in (5.6) (for a field
without singularities) and in (5.7): it is shown in [38] that the invariant x(Ft) is
equal to the integral in (5.6), understood in the principal value sense. This makes
it possible to produce sufficiently simple normal forms of flows in D'(, A), r > 2.
Some corresponding fields are given by trigonometric polynomials.
3. In contrast to the case of flows, a serious deficiency of the description given
here of the moduli space LT (A) is its nonconstructivity: the problem of "comput-
ing" the invariant 1(F) from a given diffeomorphism F E Dr(, A) and thereby
making it possible, in particular, to explicitly determine diffeomorphisms that are
not smoothly conjugate. One roundabout possibility is to specify a diffeomorphism
by its cocycle.
CHAPTER 6

Classification of Flows on Surfaces


This chapter is devoted to the questions of topological equivalence and classifi-
cation of flows on closed orientable surfaces. We confine ourselves mainly to flows
with finitely many singular trajectories. It was shown in the third chapter that a
flow (with finitely many equilibrium states and separatrices) on a closed orientable
surface can be decomposed into flows without nontrivial recurrent semitrajectories,
and irreducible flows (that is, flows with a single quasiminimal set: the closure of
a nontrivial recurrent semitrajectory). Therefore, it is natural to investigate flows
without nontrivial recurrent semitrajectories and irreducible flows separately.
Irreducible flows exist only on surfaces of genus at least 1 (among closed ori-
entable surfaces). However, the study of irreducible flows on the torus (genus 1)
is essentially different from that on surfaces of genus greater than 1, due to the
asymptotic behavior of trajectories of covering flows on the universal coverings.
In the first section we consider irreducible flows on the torus. The second, third,
and fourth sections are devoted to irreducible flows on closed orientable surfaces
of genus greater than 1. Flows without nontrivial recurrent semitrajectories are
treated in the fifth section.
§1. Topological classification of irreducible flows on the torus
Any flow with a nontrivial recurrent semitrajectory on the torus is irreducible
(Lemma 3.1 in Chapter 3). The asymptotic behavior of a nontrivial recurrent
semitrajectory is described by the Poincare rotation number: a topological invari-
ant that is the same for all nontrivial recurrent positive (respectively, negative)
semitrajectories of a fixed irreducible flow.
In the set of irreducible flows on the torus we single out two classes the class of
minimal flows and the class of Denjoy flows for which the problem of topological
classification has been solved.
1.1. Preliminary facts. Let it : TE 2 -> T2 be the universal covering of the
torus T 2 by the Euclidean plane R2 (with Cartesian coordinates x, y). The integer
translations of the Euclidean plane form the group F of covering transformations
of ir, which is isomorphic to Z2 and to the fundamental group irl (T2) of the torus.
DEFINITION. Two sets A, B C R2 (possibly single points) are said to be con-
gruent (with respect to the group F) if -y(A) = B for some -y e F.
Together with the universal covering R2 we use the unit disk D2 = {(x, y)
x2 + y2 <i} with coordinates (, that are connected with the coordinates (x, y)
by the relations
x y
77
1+x2+y2 1+x2+y2
239
240 6. CLASSIFICATION OF FLOWS ON SURFACES

(1.2) x= y=

The homeomorphism D2 -> 1[82 given by the formulas (1.2) is denoted by T.


The unit circle S = 8D2 = {(x, y) : x2 + y2 = 1} is called the absolute.
The following result is a consequence of the properties of a covering.
LEMMA 1.1. Let l be a simple closed curve nonhomotopic to zero on T2. Then
the inverse image ir-1(l) of this curve breaks up into a countable set {l}i° of
congruent (with respect to I') disjoint nonclosed curves such that:
1) for each curve E ir-1(l) there exists an infinite cyclic subgroup G(li) C I'
of elements carrying l;, into itself,
2) an element of I' carrying some point on li E ir-1(l) into a point on the same
curve leaves invariant the whole curve l;, and belongs to G(l);
3) G(li) = G(i) for any l,l E
4) there exists a line l with rational slope that is invariant under any trans-
formation in G(li) and such that the deviation of any curve li e ir-1(l) from l is
bounded;
5) each curve T-1(li), li e ir-1(l), has two boundary points, which lie on the
absolute S and are diametrically opposite;
6) all the curves {T' (l)}r have common boundary points coinciding with the
boundary points of the curve T-1(l) (Figure 6.1).

FIGURE 6.1

PROOF. We take a point mo E l and one of its inverse images mo E R2,


(rno) = mo. According to the covering homotopy theorem [43], there exists a
curve 101 with endpoints mo,ml e it-1(mo) such that ir(lol) = l and no proper
part of the curve lol \ {ml } is mapped by it onto 1.
Since l is a closed curve nonhomotopic to zero, it follows that mo ml and
there exists an element 'Y E F with 'Y id such that 'y(o) = ml. Since no proper
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 241

part of the arc lol \ {ml } is mapped by it onto the whole curve 1, and l does not
have self-intersections, -y is not a power of any other element of F.
Denote by l the line passing through m0 and ml. Since 101 is compact, the
deviation of it from the line us bounded. Then the deviation of the curve l0 =
U'yT (lol) (n e Z) from 1 is also bounded (Figure 6.2).
It follows from the definitions of the covering it and the group F that it (l o) = 1.
Since l is a simple curve, so is l0.
Denote by G(lo) aef G the cyclic subgroup generated by the element 'y. It
follows from 'y(o) = ml that l0 and l are invariant under the action of G. Since
y is not a power of some element of F, any element of F carrying some point on l0
into a point on the same curve l0 leaves the whole of l0 invariant and belongs to G.

FIGURE 6.2

The facts that the deviation of to from Hs bounded and l is simple imply
that for any y e F \ G the curves y(lo) and l0 are different and disjoint. This
and the countability of the factor group F/G give us that the complete inverse
image it-1(l) _ {y(lo) : y E F} breaks up into a countable set of congruent (with
respect to F) disjoint nonclosed curves satisfying the conditions 1)-3) because F is
commutative.
The rationality of the slope of l follows from the equality 'y(o) = ml (rno' ml E
1), and the fact that -y is a translation by an integer vector.
The properties 5) and 6) follow from (1.1), (1.2), and the property 4).

REMARK. By the covering homotopy theorem, if l C T2 is a simple closed


curve homotopic to zero, then the complete inverse image it-1(l) breaks up into a
countable set of congruent (with respect to F) disjoint closed (hence homotopic to
zero on 112) curves.
We recall that a transformation h : R2 - R2 is called a covering (or lifting) for
a transformation h: T2 -> T2 if 7r o h= h o r.
LEMMA 1.2. Suppose that h : T2 - T2 is a continuous mapping. Then:
1) there exists a continuous covering h : R2 - R2 for h, and if h is a homeo-
morphism, then so is h; _
2) if h is a covering for h, then for any element 'Y E F the transformation 'Y o h
is also a covering_ for h;
3) if h1 and h2 are two continuous liftings of h, then there exists a 'Y E F such
that h2 = 'y o h1.
242 6. CLASSIFICATION OF FLOWS ON SURFACES

PROOF. Fix a point mo e T 2 and an inverse image mo e R2 of it, 7r (O) = mo.


Also, take a point mo in ir-1[h(mo)]. We now define the mapping h at an arbitrary
point m e R2. To do this we join mo and m by an arc d (Figure 6.3). Since 71 is
a covering, there exists a unique lifting d of the arc h[er(d)] with endpoint mo [43].
Denote by m the second endpoint of the arc d and let h(m) = m.

FIGURE 6.3

A closed loop on 112 is mapped under the action of 71 into a curve on T2


homotopic to zero. This and the remark after Lemma 1.1 imply that h is well
defined, that is, that the point m is independent of the form of the arc d. _
It follows immediately from the construction and the continuity of h that h is
continuous.
Any arc on 112 with endpoints m and 'y(m), where -y e F, projects into a closed
curve on T2. Therefore, by Lemma 1.1 and the regularity of the covering 71, the
transformation h is a covering for h (regularity of the covering means that all the
liftings of any loop are either closed or open). From this and the fact that 71 is a
local homeomorphism it follows that if h is a homeomorphism, then so is h.
The assertion 2) is an immediate consequence of the definitions of the covering
71 and the group F.
Since F is a discontinuous group, any two liftings of h coinciding at some point
coincide on the whole plane. This implies 3). U

Any homomorphism of the two-dimensional integer lattice Z2 = Z x Z is given


by a 2 x 2 integer matrix d
, a, b, c, d e Z, and the homomorphism has the form
(m, n) H (am + bn, cm + dn), (m, n) E Z2. Conversely, any 2 x 2 integer matrix
determines a homomorphism of the group Z (of the indicated form).
The group F of integer translations of the plane 112 is naturally isomorphic
to the group Z2. Therefore, each homomorphism of F is given by a 2 x 2 integer
matrix.
We remark that the fundamental group irl (T 2) of the torus can be identified
with Z2 F for a fixed basis in irl (T 2) . As a basis in irl (T 2) we take elements
containing the zero parallel it (y = 0, 0 < x < 1) and the zero meridian it (0 < y < 1,
x = 0). In the identification V 711(T2) we identify the zero parallel and the zero
meridian with (1, 0), (0, 1) E Z2, respectively.
It is well known that a continuous mapping of any manifold into itself induces
a homomorphism of its fundamental group. Consequently, any continuous mapping
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 243

h : T 2 - T 2 induces a homomorphism h* : irl (T 2) -> 'irl (T2) , which is given by a


integer 2 x 2 matrix.
LEMMA 1.3. Let h : T2 -> T2 be a continuous transformation, and h : R2 -
a covering mapping for h. Then: _
1) there exists a homomorphism h* : F -> F such that h* ('y) o h = h o 'y for all
'yEF;
2) h* = h*, where h* is the homomorphism of the fundamental group 1(T2) "_'
F induced by h (consequently, (hl ) * = (h2) * for any covering transformations h1
and h2 of h);
3) if h* is written as a 2 x 2 integer matrix d
, then the covering mapping
ii= (hl, h2) has the form
hl(x, y) = ax + by +1(x, y)
h2 (x, y) = cx + dy + i/i2 (x, y)
where b1 (x, y) and b2 (x, y) are continuous periodic functions with period 1 in each
argument.
PROOF. Since a covering transformation carries congruent points into congru-
ent points, for any m e R2 and any element -y e F there is an element -y' E F such
that 'y' o h(m) = h o y(m) It follows from the continuity of h and the discontinuity
.

of the group F that the last equality is valid for all points m e R2, and hence -y'
does not depend on m e R2. We leave it to the reader as an exercise to prove that
the correspondence -y F--* y' = h* ('y) is a homomorphism.
The assertion 2) follows from the equality h* ('y) o h = h o y and the commuta-
tivity of F.
We prove 3). The square 0 < x < 1, 0 < y < 1 is a fundamental domain
F for the transformation group F. For any point (x, y) E F we set /1(x, y) =
hl (x, y) -ax -by, b2 (x, y) = h2 (x, y) - cx - dy.
Denote by 'yl,o E F an element carrying a point (x, y) into (x + 1, y). By
assumption, h* maps 'yl,o into the element -ya,b : (x, y) -* (x + a, y + b). Then
the equality 'ya,b o h(0, y) = h o 'yl,o (0, y) takes the form hl (1, y) = a + hl (0, y),
h2 (1, y) = b + h2 (0, y) Therefore, b1(1, y) = b1(0, y) and b2 (1, y) = b2 (0, y) .
.

The equalities b (x, 1) = b (x, 0), i = 1, 2, are proved similarly. We extend the
functions /1 and /2 from F to R2 by periodicity with period 1 in both arguments.
Let m(x, y) E I182 be arbitrary. Since F is a fundamental domain, there exists
a -y e F such that -y-1(m) E F (Figure 6.4). Let 'y: (x, y) H (x + n, y + k) .

Then h* ('y) : (x, y) H (x + an + bk, y + cn + dk). We have that h1('y-1(m)) =


h1(x-n, y-k) = a(x-n)+b(y-k) +i/i1(x, y). In view of the equality h*('y)oh = ho'y
(Figure 6.4), we get that hl (x, y) = hl ['y-1(m)] + an + bk = ax + by + iIi1(x, y).
Similarly, h2 (x, y) = cx + dy + /2(x, y). U

COROLLARY 1.1. Assume the conditions of Lemma 1.3, and let h be a homo-
topy equivalence (in particular, a homeomorphism). Then the matrix h* = cdb
is unimodular.
PROOF. Since h is a homotopy equivalence, the homomorphism h* is an auto-
morphism. Therefore, the matrix h* is invertible. U
244 6. CLASSIFICATION OF FLOWS ON SURFACES

FIGURE 6.4

LEMMA 1.4. Let h : T2 - T2 be a continuous mapping homotopic to the iden-


tity, and let h : R2 - R2 be a covering mapping. Then there exists a constant
a> 0 such that the distance between the points m and h(m) does not exceed a for
all m E R2.
PROOF. Since h is continuous, and the set F = {(x, y) E R2 0 < x < 1,
0 <y < 1} is compact, it follows that a = max m - h(m) I < +oo for m_ E F. The
fact that h is homotopic to the identity implies that h* = id, and hence h o y = y oh
for y E F. _
For any_m e R2 there is a y e F such that y(m) E F. Therefore, Im - h(m) I =
'Y(m) - yo h(m) I = 'Y(m) - h o 'y(m) I <_ a. U

1.2. Curvilinear rays. Denote by R+ the set of nonnegative real numbers,


and let Sp : R+ -> M be a homeomorphism onto its image, where M is a manifold.
The set cp(TEt) is called a half-closed (or half-open) infinite curve. The mapping (p
determines on the curve Sp(R+)
a! l+ a parameter t H Sp(t), t e R+.
Let it : M -> M be a covering with universal covering space M. Since (p is
one-to-one, the complete inverse image it-1(1+) is a family of disjoint half-closed
infinite curves li , ... , with a parametrization on each of them determined by Sp.
DEFINITION. A half-closed infinite curve l+ C PVC is called a curvilinear ray if
there exists a lifting l+ C M of it (which is also a half-closed infinite curve) onto
the universal covering M such that l+ leaves any compact set as the parameter
increases unboundedly, and does not return to it, beginning with some value of the
parameter.
We mention the possibility that M = M and l+ = l+ It follows from the
.

definition that a lifting l+ of a curvilinear ray is also a curvilinear ray.


It is sometimes more convenient to parametrize a half-closed curve by the set
of nonpositive numbers. The concept of a curvilinear ray is defined similarly for
such curves.
Each semitrajectory of a flow has a natural parametrization, which we have in
mind when we refer to a semitrajectory that is a curvilinear ray.
LEMMA 1.5. Let f t be a flow on the torus, and let f t be a flow on R2 covering it.
Then a semitrajectory l of f t is a curvilinear ray if and only if each semitrajectory
of f t in the inverse image it 4 [10] is a curvilinear ray.
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 245

PROOF. This follows from the fact that any two semitrajectories in it-1 [l0]
can be carried one into the other by an integer translation. U
For brevity we call a trajectory a curvilinear ray if both its semitrajectories are
curvilinear rays.
t
LEMMA 1.6. Suppose that f t is a flow on the torus T2, and f is a flow on R2
covering it. Then the following semitrajectories of f t and f t are curvilinear rays:
t
1) a semitrajectory of f that projects into a closed trajectory of f t nonhomo-
topic to zero;
2) a nonclosed semitrajectory of f t containing in its limit set a closed trajectory
nonhomotopic to zero, or a one-sided contour nonhomotopic to zero, or a nontrivial
recurrent semitrajectory;
3) a nontrivial recurrent semitrajectory of ft.
PROOF. 1) and 2) follow from 3) and Lemma 1.1. We prove 3). Let l+ be
a positive nontrivial recurrent semitrajectory of f t (the proof is analogous for a
negative semitrajectory). Assume that a semitrajectory l+ E it-1(l +) does not
leave the compact set K C I182 or that it returns to K infinitely many times as the
time increases. Then w(l+) 0. Since l+ is a nontrivial recurrent semitrajectory,
w(l+) contains at least one regular point m. Since it is nonclosed, the semitrajectory
l+ intersects more than once a contact-free segment passing through m. Therefore,
there exists a contact-free cycle C intersecting l+ (Corollary 2.1 in Chapter 2).
Denote by D the disk bounded by the curve C. We consider two cases: 1)
m belongs to a disk congruent to D (possibly D itself); 2) there is no 'y E F
such that m belongs to 'y(D). In case 1) there is a semitrajectory li congruent
to l + whose w-limit set belongs to D. Therefore, w (l i) is a single equilibrium
state, or a closed trajectory, or a one-sided contour. Then the w-limit set of the
semitrajectory l+ = ir(li) also is a single equilibrium state, or a closed trajectory,
or a one-sided contour, and this contradicts the fact that l+ is a nontrivial recurrent
semitrajectory. In case 2) we get a set ir(D) that is bounded by a contact-free cycle
ir(C) homotopic to zero, and l+ f1 ir(C) 0, contradicting Lemma 2.3 in Chapter
2.U
1.3. Asymptotic directions. Let f t be a flow on the torus. Along with a
covering flow f on I182 we consider a flow f t on the unit disk D2 that is a covering
flow for f t with respect to the covering it o T (see § 1.1). The flow f t is isomorphic
to f t by means of the diffeomorphism TN -1 defined by the formulas (1.1) in §1.1.
Corresponding objects of the flow f t on D2 will be equipped with a tilde above
them.
t
DEFINITION. Suppose that l+ is a positive semitrajectory of a flow f on I182
and is a curvilinear ray, and let l+ = T1(l) be the corresponding semitrajectory
of the flow f t on D2. The semitrajectories l+ and l+ have an asymptotic direction
if the w-limit set of l + consists of a single point a( l) belonging to the absolute
S = aD2.
An analogous definition for the existence of an asymptotic direction applies to
the negative semitrajectories 1 - and 1.
246 6. CLASSIFICATION OF FLOWS ON SURFACES

FIGURE 6.5
The point a( i)) is said to be accessible by the semitrajectory 1).
We remark that if a semitrajectory l0 is a curvilinear ray, then the w- (a-) limit
set of the semitrajectory l ) _ -r-1(l 0) is nonempty and belongs to the absolute.
The existence of an asymptotic direction indicates that the w- (a-) limit set consists
of a single point.
In Theorem 1.1 we prove that if a semitrajectory of a covering flow on f2 is a
curvilinear ray, then it has an asymptotic direction.
t
EXAMPLE 1. A covering flow f F for a rational or irrational winding of the
torus is given by the system x = 1, y = c. Each trajectory l (x(t) = xo + t,
y(t) = yo + pct) is a curvilinear ray. We leave it to the reader as an exercise to
prove that all the semitrajectories of f µ have an asymptotic direction, and the
semitrajectories of the corresponding flow fµ on D2 have accessible points on the
absolute as shown in Figure 6.6 (all the positive semitrajectories have the accessible
point (+ ,G2, a/-s/1 + ,2 ), and all the negative ones have the diametrically
opposite point).

FIGURE 6.6

EXAMPLE 2. We consider a covering flow f t whose phase portrait is pictured


in Figure 6.7, a). All the semitrajectories of f t are curvilinear rays and have an
asymptotic direction. The phase portrait of the corresponding flow f t is pictured
in Figure 6.7, b). The points (-1, 0) and_(1, 0) are accessible both for the positive
and for the negative semitrajectories of f t.
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 247

1y
i

=2 k
a)
b)

FIGURE 6.7

LEMMA 1.7. Let l+( = x(t), y = y(t)) be a positive semitrajectory of some


covering flow on ][82, and assume that it is a curvilinear ray and has an asymptotic
direction. Then a( 1+) _ (µi, µ2) E S is accessible by the semitrajectory l+ _
T-1(l+) if and only if one of the following conditions holds: a)
0, in which case v(l) _ (fl, 0); b) 0, in which case v(l+) _
(0, fl); c) both the limits limt y(t)/x(t) = µ2/µ1 den µ and limt_ x(t)/y(t) _
µi /,a2 exist and are finite and nonzero, and
1 µ
(1.3)
= + ' = +

PROOF. Since l+ is a curvilinear ray, x2 (t) + y2 (t) -* +00 as t -* +00.

NECESSITY. Let a = (ui, /2), where


x(t) y(t)
= lim 1+ +2t lug = lim 1 + x2(t) + 2 t
too x2(t)
'
too
and i + c2 = 1. If = 1, then 0; therefore, Ix(t) = 00. The
relation
1 =1
t-'oo 1 /x2(t) + 1 + y2(t) /x2(t)
implies that y(t)/x(t) = 0. Similarly, x(t)/y(t) = 0 if = 0 and
IP2I = 1.
In the case 0 < i < 1, i = 1, 2, we get from (1.1) that limt x I =
I= oo. It follows from the relation i + p = 1 that (1.3) holds, and
the case c) is realized.

SUFFICIENCY. Let = e(t), rs = r7(t) be the equations of the semitrajectory


l+. Since l+ is a curvilinear ray, it follows that e2 (t) + r)2 (t) -* 1 as t -* 00.
If a) holds, then 77(t)/e(t) = limty(t)/x(t) = 0, from which I,ai I =
limtI= 1 and r7(t) = 0. Similarly, it follows from the condition
b) that cl =0and IP2I = 1.

limit limt . y(t)/x(t) def


= ,
In the case c) we get from limt [x2 (t) + y2 (t)] = oo and the finiteness of the
-

0, that limt . y (t) = limt . I= 00. This


and the formulas (1.1) imply (1.3). 0
248 6. CLASSIFICATION OF FLOWS ON SURFACES

COROLLARY 1.2. Let l+ be a positive semitrajectory of some covering flow on


R2, and assume that it is a curvilinear ray and has an asymptotic direction. Then
any semitrajectory l i congruent to it is a curvilinear ray and has an asymptotic
direction, and a( l) = a( l i ) .
PROOF. Assume that the case a) of Lemma 1.7 is realized for the semitrajectory
l+(x = x(t), y = y(t)), that is, limt_ y(t)/x(t) = 0. Since li is a semitrajectory
congruent to l+, li is given by the equations x = x(t) +k, y = y(t) -I- n for some
k, n E 7L. It follows from
+1[x2(t) +y2(t)] = 00
that
lim x
t-+oo
= 00,
and therefore,
lim [y(t) + nJ/[x(t) + kJ = 0.

By Lemma 1.7, 11 has an asymptotic direction, and cr(l + ) = cr(l i) = (±1,0).


The remaining cases b) and c) are treated similarly. 0

To prove Theorem 1.1 we need the following result.


LEMMA 1.8. A minimal flow on the torus is topologically equivalent (and even
topologically orbitally equivalent) to an irrational winding by means of a homeomor-
phism homotopic to the identity.
PROOF. Let f t be a minimal flow on the torus. Since each trajectory of f t
is dense, there exists a contact-free cycle C that intersects any semitrajectory of
f t (that is, C is a global section). Therefore, f t induces a Poincare mapping
P : C -* C that is a homeomorphism. Since f t is minimal, each semi-orbit of P is
dense. Then by Corollary 1.2 and Theorem 2.3 in Chapter 5, P is conjugate to a
rotation Ra : 81 -* S1 with irrational a.
We consider an irrational winding f a on the torus with covering system x = 1,
y = a. The zero median Co = ir(0 < y < 1, x = 0) is a contact-free cycle of fa on
which f a induces the Poincare mapping Ra. Then a homeomorphism realizing a
conjugacy between P and Ra can be extended to a homeomorphism h : T2 * T2
realizing topological equivalence (and even a topological orbital equivalence) of
the flows f t and fa.
If the homeomorphism h is nonhomotopic to zero, then we take a linear dif-
feomorphism ho of the torus with covering of the form x = ax -I- by, y = cx + dy,
where h* = d
is the isomorphism of irl (T2) = r induced by h. It follows from
Lemma 1.3 that (ho) * = h.
Under the action of the linear diffeomorphism ho 1 the irrational winding f a
clearly passes into an irrational winding f o . Then f t is topologically equivalent to
fo by means of the homeomorphism ho 1 o h, which is homotopic to the identity
because (hi' o h) * = (h1) * o h* = id. 0

THEOREM 1.1. Let f t be a flow on the torus, and let f t and f t be flows covering
it on R2 and D2, respectively. Then:
1) each semitrajectory of f t that is a curvilinear ray has an asymptotic direction;
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 249

2) for any two semitrajectories of f t having an asymptotic direction the acces-


sible points on the absolute either coincide or are diametrically opposite;
3) if f t has a nontrivial recurrent semitrajectory, then all the positive (respec-
tively, negative) semitrajectories of ft that are curvilinear rays have a common
accessible point E S (o E S), and the points and o are diametrically
opposite.

PROOF. Suppose that a positive semitrajectory l+ of f t is a curvilinear ray.


We show that it has an asymptotic direction (the proof is analogous for a negative
semitrajectory).
Consider the semitrajectory l+ = lr(l+). It follows from the properties of a
covering and the fact that l + is a curvilinear ray that w (l +) does not consist of a
single equilibrium state, and is neither a closed trajectory homotopic to zero nor
a one-sided contour homotopic to zero. Therefore, by Theorem 3.6 in Chapter 2,
w(l+) is:
a) a closed trajectory to nonhomotopic to zero;
b) a one-sided contour nonhomotopic to zero;
c) the closure of a nontrivial recurrent trajectory.
t
In the case a) we take a trajectory to of f such that ir(lo) = lo. It follows from
Lemma 1.1 that both semitrajectories to and to of to have an asymptotic direction,
and the points cr(lo) and cr(lo) are diametrically opposite. This implies that l+
also has an asymptotic direction, and cr(l +) E cr(lo) U cr(l o ) .

We prove 2) in the case a). If l i is a semitrajectory of ft that is a curvilinear


ray, then the limit set of the semitrajectory it [l i ] is a closed trajectory nonhomo-
topic to zero or a one-sided contour ko nonhomotopic to zero, because in this case f t
does not have nontrivial recurrent trajectories. The curves to and ko are homotopic,
and hence one of the points a( l) and a( l) is accessible for the semitrajectory
li)
The case b) is handled similarly.
We proceed to the case c). According to Lemma 3.1 in Chapter 3, the flow f t
is irreducible. Therefore, by Theorem 3.1 in Chapter 3, there exists a continuous
mapping h : T2 -* T2 homotopic to the identity (a blowing-down operation) carry-
ing the trajectories of f t into trajectories or unions of trajectories of some transitive
flow f o . According to Lemma 3.4 in Chapter 3, a transitive flow is obtained from a
minimal flow by adjoining a certain number of impassable grains (possibly none).
Therefore, in view of Lemma 1.8 the flow fo can be assumed to be an irrational
winding with a certain number of impassable grains (possibly none). By Lemma
1.4, a covering transformation homotopic to the identity moves the points of the
plane a distance bounded by a fixed constant. This and Example 1 imply the
assertions 1)-3) in the case c). 0

1.4. The Poincare rotation number. Let f t be a flow on T2, and let f t and
f t be flows covering it on R2 and D2, respectively. Suppose that a semitrajectory
l of f t is a curvilinear ray and hence has an asymptotic direction. This means
that the corresponding semitrajectory l) = 'r-1(l a) tends to a single point ( 10)
on the absolute S.
250 6. CLASSIFICATION OF FLOWS ON SURFACES

DEFINITION. Let the point o(10) E S = 3D2 have coordinates (/c1, /-L2). If
0, then the number c = 2/1 is called the Poincare rotation number of the
flow ft. If 1 = 0, then the Poincare rotation number is set equal to oo.
By Theorem 1.1, this is well defined: it does not depend on the choice of the
semitrajectory l of f t having an asymptotic direction.
The point cr( 10) and the diametrically opposite point on the absolute are called
the rotation points of f t .
The Poincare rotation number of a flow f t is denoted by rot (f t) .

Not every flow on the torus has a Poincare rotation number. For example, a
flow with covering flow pictured in Figure 6.8 does not have a rotation number
because no semitrajectory of the covering flow is a curvilinear ray.

FIGURE 6.8

LEMMA 1.9. Let f t be a low on the torus, and let f t be a low on R2 covering
it. In this case:
1) if f t has a rotation number = rot(f t), then for any positive semitrajectory
l+ (x = x(t), y = y(t)) of ft that is a curvilinear ray
(1.4) µ = tl y(t)/x(t),
and the points (iii, µ2) and (-µi, -µ2) with µi = 1/ 1 + µ2 and µ2 = µ/ 1 + µz
are the rotation points of f t;
2) if there exists a positive semitr¢jectory l+(x = x(t),y = y(t)) of ft that is
a curvilinear ray, then the limit (1.4) exists and is equal to the rotation number of
f t, and the points (µl,µ2) and (-µi, -µ2) with µl and µ2 given by (1.3) are the
rotation points of ft.
PROOF. This follows from Lemma 1.7, Theorem 1.1, and the definition of the
rotation number of a flow. 0

Lemma 1.9 yields the following connection between the rotation number of a
flow and the rotation number of the Poincare mapping the flow induces on the zero
meridian.
COROLLARY 1.3. Suppose that a low f t on the torus has a global section Co
(that is, a contact free cycle intersecting each regular trajectory) coinciding with
the zero meridian of the torus, and assume that f t induces a Poincare mapping
P: Co -* Co. Then rot(P) = rot(f t) (mod 1).
PROOF. For a covering flow f t the lines x = n, n E Z, in the inverse image
it 4 (Co) are contact-free lines. By the hypotheses, there exists a semitrajectory
l+ of f t intersecting Co countably many times (perhaps at a single point if 1 + is
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 251

a closed trajectory). Then f t has a semitrajectory l+ (yo) intersecting all the lines
x = n, n E N, with yo on the line x = 0.
Denote by Pn the Poincare mapping induced by f t from the line x = 0 to the
line x = n. Since f t is a covering flow, the relations it o Pn = Pn o ir, n E N, hold
on the line x = 0. From Lemma 1.9 and the definition of the rotation number of a
transformation of the circle (§1 in Chapter 5) we get that
rot(f t) = tll
m y(t)/x(t) =nlim
+oo
y(n)/n = lim Pn(yo)/n = rot(P1),
n +oo
where x = x(t), y = y(t) are the equations of the semitrajectory l+(yo). 0

REMARK. A connection between the rotation number of a flow without com-


pact trajectories on the torus and the rotation number of the Poincare mapping on
an arbitrary global section was found by I. Strelcyn in the paper, Flots sur le tore
et nombres de rotation (Bull. Soc. Math. France 100 (1972), 195-208).
The following theorem is analogous to Lemma 1.6 in Chapter 5, and indicates a
connection between the dynamical properties of a flow and the arithmetic properties
of the rotation number.
THEOREM 1.2. Suppose that a low f t on the torus has a rotation number c.
Then:
1) is rational or = oo if and only if f t has a closed trajectory nonhomotopic
to zero or a one-sided contour nonhomotopic to zero;
2) is irrational if and only if f t has a nontrivial recurrent semitrajectory.
PROOF. Since f t has a rotation number, f t has a) a closed trajectory nonho-
motopic to zero, or b) a one-sided contour nonhomotopic to zero, or c) a nontrivial
recurrent semitrajectory, and the case c) cannot be realized simultaneously with
one of the cases a) or b).
It follows from Lemma 1.1 that in the cases a) and b) the rotation number of
f t is rational or c = oo.
Suppose that the case c) is realized. Then there exists a continuous mapping
homotopic to the identity (a blowing-down operation) that carries the trajectories
of f t into those of an irrational winding with some number of impassable grains
(possibly none). Therefore, rot(ft) is irrational. 0

1.5. The rotation orbit. Suppose that a flow f t on the torus has a rotation
number = rot (f t) , and let (,ai, /2), (-,a1, -/2) E S be the rotation points of
ft, with their coordinates related to by the formulas (1.3):

G1 = 11 + 2,
(we note that if = oo, then the rotation points of the flow are (0, 1) and (0, -1),
and vice versa).
DEFINITION. The rotation orbit of a flow f t is defined to be the set of points
(/1, lug) , (-/1, -/2) of the absolute S with coordinates determined from the
relations (1.3)
µi = 11 + µ2, µ2 = Wi + µ2,
252 6. CLASSIFICATION OF FLOWS ON SURFACES

where is computed according to the formula


_ c+drot(ft)
(1.5)
a + b rot (ft)

for all possible unimodular integer matrices d


If rot(ft) = oo or if rot(ft) is
.

rational, then runs through the set of rational numbers, and the points (0, 1) and
(0, -1) are adjoined to the rotation orbit.
In the next theorem it is shown that the rotation numbers of two topologically
equivalent flows can be calculated one from the other by means of a unimodular
integer matrix, that is, according to the formula (1.5), and hence the rotation orbit
is an invariant of topological equivalence.
THEOREM 1.3. Suppose that flows f t and gt on T2 have Poincare rotation
numbers rot(ft) = and rot (gt) = A, and that f t is topologically equivalent to gt
by means of a homeomorphism h. In this case if oo, then

k1
16
a+b'
where h* = (L d)
is the homomorphism induced by h on the fundamental group
ir(T2) 7L. If = oo, then a = d/b.

PROOF. Let f t and gt be covering flows for f t and gt, respectively, and let h be
a covering homeomorphism for h. Then h carries trajectories of f t into trajectories
of gt .
Since f t has a rotation number, there exists a semitrajectory l 0 of this flow
having an asymptotic direction. Let l0 be a positive semitrajectory. If x = x(t),
y = y(t) are the parametric equations of l+, then = y(t)/x(t) by Lemma
1.9.
Let l' = h(l+) be the semitrajectory of gt into which l+ passes under the action
of h. We show that l' is a curvilinear ray.
According to Lemma 1.3, the parametric equations x = x' (t), y = y' (t) of the
semitrajectory l' have the form
x'(t) = ax(t) + by(t) + ib1(x(t), y(t)),
(17)
.
y '(t) = cx(t) + dy(t) +'b2(x(t), y(t)),
where the 4/i (x, y) are continuous 1-periodic functions in each argument, and h* =
h* = c d is a unimodular integer matrix (Corollary 1.1).
We remark that the motion of the point (x' (t), y' (t)) E R2 along the semitra-
jectory l' as t -p oo by virtue of the formulas (1.7) does not necessarily correspond
to the motion along l' as a phase trajectory of gt as time increases. This motion can
correspond to decreasing time, which means that h reverses the direction_ of motion
along trajectories. For definiteness and for simplicity we assume that h preserves
the positive direction of motion along trajectories.
To prove that l' is a curvilinear ray it is necessary to show that [x'(t)]2 +
[y'(t)]2 -* 00 as t -* oo. Three cases are possible:
a) = 0;
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 253

b) c = 00;
c) c 0, 00.
In case a) we have from the equalities y(t)/x(t) = 0 and
limt_ [x2(t) + y2(t)] = 00 that Ix(t)I = 00. Then we get from (1.7) that

hm x'(t)
t -oo x(t)
= t-hm
oo
a + b y(t)
x(t) + x(t)
=a
+
1
=
a'
lim y' (t) = lim c d
y(t) -2 = c d =c .
t-oo x(t) t-oo + x(t) + x(t) +
Since the matrix h* is unimodular, a2 + c2 0. Therefore, Ix'(t)I = 00 or
limtIy'(t)I = 00
In case b) the argument is analogous to the preceding, with use of the equality
limt_ x(t)/y(t) = 0 (each of the equalities in (1.7) is divided by y (t)) .
In case c) it suffices to show that one of the limits

lim x' (t) = a+bj, lim y' (t) = c+d


t-oo x(t) t-oo x(t)
-bµ b = 0, which
is nonzero. But if a + 0 and c + 0, then acdb = I-d,dI
contradicts the fact that h* is a unimodular matrix.
Thus, l' is a curvilinear ray.
If / oo, then = oo. In this case
I

. y' (t) c + d Wi(t) + c+


t oo x '(t) t oo a + b y(t) + a + b1u
mo(t) mo(t )

If = oo, then I
= 00, and

lim
y'
t->oo x' (t)
(t) = lim cwt
y() +d+ y() _ db-. 0
t+oo a mo(t) + b -I- 1
y(t) y(t)

1.6. Classification of minimal flows. We recall that a flow is said to be


minimal if each trajectory of it is dense in the manifold. According to Theorem
1.2, a minimal flow on the torus has an irrational (and hence not infinite) Poincare
rotation number.
ows f t and gt on the torus with rotation num-
THEOREM 1.4. Two minimal flows
bers = rot(f t) and A = rot(gt) are topologically equivalent if and only if there
exists a unimodular integer matrix d
such that

A=c+
a -}-

PROOF. NECESSITY. It follows from Theorem 1.3 and Corollary 1.1.


254 6. CLASSIFICATION OF FLOWS ON SURFACES

SUFFICIENCY. Assume (1.8). We show that f t and gt are topologically equiv-


alent.
According to Lemma 1.8, a minimal flow on the torus is topologically equivalent
to an irrational winding by means of a homeomorphism homotopic to the identity.
Since a homeomorphism homotopic to the identity induces the identity mapping
of the fundamental group of the torus, it follows from Theorem 1.3 that f t and gt
can be assumed to be irrational windings with respective rotation numbers and
A and with covering flows

f (x=1
_t _t
g
(x=1
:

y= y= A
.

By (1.8), the linear transformation h : R2 R2 given by


x=ax+by, y=cx+dy
db
realizes a topological equivalence of the flows f t and gt . By assumption,
. cd is
a unimodular integer matrix. Therefore, h is a covering for some homeomorphism
h: T2 -p T2 that realizes a topological equivalence of the flows f t and gt. 0

COROLLARY 1.4. Two minimal flows on the torus are topologically equivalent
if and only if their rotation orbits coincide.
1.7. Classification of Denjoy flows. We recall that a Denjoy flow is defined
to be a flow on the torus without equilibrium states and closed trajectories and with
a limit set of Cantor type (see §3.1 in Chapter 1).
The set of Denjoy flows is denoted by Dent (T2) .
Since the limit set SZ (ft) of a flow f t E Dent(T2) is nowhere dense and since f t
does not have equilibrium states nor closed orbits by definition, SZ (ft) consists of
nontrivial recurrent trajectories which are nowhere dense on the torus. Obviously,
each nontrivial recurrent trajectory of f t lies in S (ft). Since the torus has genus 1,
any flow f t E Dent(T2) is irreducible, and hence any nontrivial recurrent trajectory
of f t is dense in c (f t) . Therefore, SZ (ft) is a minimal set of the flow ft.
According to Lemma 3.3 in Chapter 3, any Denjoy flow can be obtained from
some minimal flow on the torus by a blowing-up operation. This result enables us
to classify Denjoy flows.
The next result follows from Lemma 3.3 in Chapter 3 and Theorem 1.3 be-
cause a minimal flow on a torus is topologically orbitally equivalent to an irrational
winding by means of a homeomorphism homotopic to the identity (Lemma 1.8).
LEMMA 1.10. Let f t be a Denjoy flow, and let fo be an irrational winding
with rotation number rot (f t) . Then there exists a continuous mapping h : T 2 -* T 2
(a blowing-down operation) that is homotopic to the identity and has the following
properties:
1) h carries each trajectory of f t into a trajectory of fo with preservation of
direction in time;
2) any nontrivial recurrent trajectory of f t is mapped by h homeomorphically
onto its image;
3) h[cZ(ft)] = T2;
4) for any component w of the set T2 \ (ft)
a) w is simply connected,
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 255

b) the accessible (from within) boundary of w consists of exactly two trajectories


11i l2 E S (ft),
c) h(w U 11 U 12) is a trajectory of the flow fo;
0
5) if (ft) C (ft) stands for the trajectories not lying on the accessible (from
within) boundary of any component of T2 \ 1 (f t), then the restriction hI t is a
0
homeomorphism of (ft) onto its image.
NOTATION. Let f t be a Denjoy flow, and suppose that the mapping h satisfies
Lemma 1.10. Let
X(ft,h) = Uh(w),
where the union is over all the components w of the set T 2 \ SZ (f t) .
The set X(f t, h) is an at most countable family of trajectories of an irrational
winding.
DEFINITION. Two sets A, B C T2 are said to be equivalent if there exists a
homeomorphism F : T 2 -* T 2 with a covering of the form
(1.9) x = x, y = y -I- = const ER,
and such that F(A) = B.
It follows from the lemma that for a fixed flow f t E Dent(T2) the sets X (f t , h)
are equivalent for different blowing-down operations h.
LEMMA 1.11. Suppose that flows f i , f 2 E Dent (T 2) are topologically equivalent
by means of a homeomorphism ' : T2 -* T2 homotopic to the identity, and let hi
be a blowing-down operation constructed by virtue of Lemma 1.10 for the flow f,
i = 1, 2. Then the sets X(fi, h1) and X(f2, h2) are equivalent.
PROOF. Suppose for definiteness that P carries trajectories of f f into tra-
jectories of f2. Since P is homotopic to the identity, Theorem 1.3 gives us that
def
rot(f1) = rot(f2) = .
Denote by f o an irrational winding with covering flow f o given by the system
x = 1, y = 1c. Then hi is a blowing-down operation of the flow f Z to the flow f o ,
i = 1, 2.
t
Obviously, a mapping F : R2 - R2 of the form (1.9) carries trajectories of f o
into trajectories of the same flow f o . We show that there exists a E R such that
the F in (1.9) maps it-1(X(f, i, h1)) onto it-1 [X(f2, h2)] .

Denote by hi a mapping covering hi, i = 1, 2, and take some component


w C R2 \ it -1 [1(ff)J . Then iii) is a component of the set R2 \ r -1 [1(f)J, where
P is a homeomorphism covering gyp. According to Lemma 1.10, 4), the sets h1 (w)
and h2 o i(ii) are trajectories of the flow f. By the form of the trajectories of f o,
there exists a number such that the mapping F : R2 - R2 defined by (1.9) carries
hl (w) into h2 o(w).
Suppose that F projects to the mapping F : T 2 -* T2. It follows from (1.9)
that F is homotopic to the identity. Since h1, h2, and P are also homotopic to the
identity, Lemma 1.3 gives us that
F ° hl ° 'Y(w) _ 'Y ° F ° hl (w) _ 'Y ° h2 ° SP(w) = h2 ° SP ° 'Y(w)
256 6. CLASSIFICATION OF FLOWS ON SURFACES

for any element ry e t (recall that I' is the group of integer translations). From
this and the denseness of {'y(Ei) ry e P} in the family of components of the
set I[82 \ ir-1 [1(f)] it follows that F o hl (ii) = h2 0 (ii) for any component v C
Then F( ir-1[X(fi, hi)]) _ ir-1[X(f2> hz)], and hence F[X(fi, hi)] _
X(fi,h2). D

DEFINITION. Let f t be a Denjoy flow, and h the blowing-down operation con-


structed by virtue of Lemma 1.10. The characteristic class x(ft) of f t is defined
to be the family of sets equivalent to the set x (f t , h).
According to Lemma 1.11, the characteristic class of a flow f t e Dent (T2) is
the collection of sets x (f t , h) constructed for all possible blowing-down operations
h.

DEFINITION. Two characteristic classes x(f l) and x(f2) of respective Denjoy


flows f l and f 2 are said to be commensurable if for any representatives xl e x(f i )
and x2 E x (f) of these classes there exists a homeomorphism F : T 2 -* T 2,
F(X1) = x2, with a covering of the form
x = ax + by,
(1.10)
y=cx+dy+,
where (L d is a unimodular integer matrix, and e e Ilk.
)
The next lemma shows that up to commensurability the characteristic class is
a topological invariant of a Denjoy flow.
LEMMA 1.12. If flows f, f2 E Dent (T2) are topologically equivalent, then their
characteristic classes x(E) and x(f2) are commensurable.
PROOF. Suppose that the homeomorphism cp : T2 -* T2 carries trajectories of
fi into trajectories of f2. According to Lemma 1.3 and Corollary 1.1, cp induces an
isomorphism cp* : ir1(T2) -* ir1(T2) given by a unimodular integer matrix d
.

Denote by F the diffeomorphism with covering of the form


x = ax + by,
y = cx+dy.
The flow f 3 = F o f i o F-1 is topologically equivalent to the flow fl by means
of the homeomorphism F. If h1 is a blowing- down operation of f i to an irrational
winding f o , then h3 = Fo h1 o F -1 is a blowing-down operation of f 3 to the irrational
winding F o fo o F, and x(f3, h3) = h3 [T2 \ 1(f3)] = F o h1 o F-1 [T2 \ 1(f3)] =
F o h1 [T2 \ 1(fi)] = F[x(fi, h1)]
The flows f 3 and f 2 are topologically equivalent by means of the homeomor-
phism cp o F-1, which is homotopic to the identity, and therefore, by Lemma 1.11,
x (f3 , h3) is equivalent to x (f 2 , h2), where h2 is a blowing-down operation of f 2 to an
irrational winding. Consequently, there exists a homeomorphism carrying x (f l , h1)
into x (f 2 , h2), and it has a covering of the form (1.9). U

In what follows it will be proved that commensurability of characteristic classes


implies topological equivalence of Denjoy flows. For this we first prove two lemmas.
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 257

LEMMA 1.13. Any Denjoy flow is topologically equivalent to the suspension


over some Denjoy homeomorphism.
PROOF. Since a Denjoy flow f t has nontrivial recurrent trajectories, there ex-
ists a contact-free cycle C of ft that is nonhomotopic to zero (Lemma 2.3 in Chapter
2). The set T2 \ C is homeomorphic to an annulus, so in view of Lemma 2.4 in
Chapter 2 each nontrivial recurrent semitrajectory intersects C, and hence C is a
global section of f t .
Denote by P : C -* C the Poincare mapping induced by ft. Since f t does
not have equilibrium states, P is a homeomorphism. It follows from the nowhere
denseness of the limit set of ft that P is conjugate to a Denjoy homeomorphism.
Consequently, f t is topologically equivalent to the suspension over a Denjoy home-
omorphism. EJ

LEMMA 1.14. Suppose that C is a simple closed curve on the torus that is
nonhomotopic to zero, and let f t be a Denjoy flow. Then there exists a contact-free
cycle of ft that is homotopic to C.
PROOF. By Lemma 1.13, it can be assumed that f t is the suspension sus(f )
over a Denjoy homeomorphism f. We recall that sus (f) is defined on the torus
obtained from the cylinder Si x [0, 1] after identifying the points (x, 1) and (f (x), 0)
for x e 51, and the segments (x, t), 0 < t < 1, project into trajectories of the flow
sus(f).
Since f is a Denjoy homeomorphism, f (x) ; x for all x e 51, n E Z. This
implies that any closed geodesic on the torus that is homotopic to C is a transversal
of the flow sus(f). EJ

COROLLARY 1.5. Any Denjoy flow is topologically equivalent to the suspension


over some Denjoy homeomorphism by means of a homeomorphism homotopic to
the identity.
PROOF. It suffices to prove that there is a contact-free cycle C homotopic to
the zero meridian no of the torus. Let the homeomorphism cp : T2 - T2 carry
trajectories of the Denjoy flow into trajectories of a suspension sus(f) (Lemma
1.13). According to Lemma 1.14, the flow sus(f) has a contact-free cycle C that
is homotopic to the simple closed curve cp (no) . Then C = cp(C) is the desired
contact-free cycle of ft. U

THEOREM 1.5. Denjoy flows f i and f2 are topologically equivalent if and only
if their characteristic classes x(fD and x(f2) are commensurable.
PROOF. By Lemma 1.12, topological equivalence of flows implies commensu-
rability of the characteristic classes.
Suppose that the classes x(fD and x(f2) are commensurable; that is, there
exists a transformation F : T2 - T2, with a covering of the form (1.10), that
carries x (f i , h 1) into x (f 2 , h2 ), where h2 is a blowing-down operation of fi to an
irrational winding, i = 1, 2. Then for the flow f3 = F o f i o F-1 we have that
oF-1
x(fL h3) = x(f2, h2), where h3 = F o h1
We show that the flows f2 and f3 are topologically equivalent. By Corollary
1.5, f 2 and f 3 are topologically equivalent to the suspensions sus (f 2) and sus (f 3 )
258 6. CLASSIFICATION OF FLOWS ON SURFACES

over some Denj oy homeomorphisms f2 and f3, respectively, by means of homeomor-


phisms homotopic to the identity. It suffices to prove that f2 and f 3 are conjugate.
It follows from x (f 2 , h2) = x (f3 , h3) that rot(f) = rot (f 3) , so rot [sus (f 2) ] =
def
µ From this, rot(f2) = rot(f3) = µ (mod 1).
rot [sus (f 3) ]
Denote by Rµ the rotation of the circle with rotation number µ (mod 1). Ac-
cording to Lemma 2.3 in Chapter 5, the homeomorphism f2 is topologically semi-
conjugate to Rµ by means of a continuous transformation h2: 81 - 51, i = 2,
3. This enables us to construct a special blowing-down operation from sus (f Z )
to sus (R1 ) . For a point m e T2 we denote by S2 (m) (respectively, So (m)) the
shift of m along the corresponding trajectory of sus(f2) (respectively, sus(Rµ)) by
the time t. Let t(m) be the smallest nonnegative number such that S2 t(m) (m) E
51 = ir(the line x = 0) (respectively, So t(m) (m) E S1 = ir(the line x = 0)). Let
hi (m) = Sot(m) o h2 o Sz t(m) (m), m e T2. By Lemma 2.3 in Chapter 5, h2 is a
blowing-down operation from the flow sus (f Z) to the flow sus (Rµ) , i = 2, 3.
It follows from x(f2, h2) =x(f3, h3) that the sets x[5u5(f2),h2] and x[sus(f3),h3]
are equivalent, and therefore the characteristic sets x (f 2 , h2) and x (f 3 , h3) of the
Denj oy homeomorphisms f2 and f 3 are carried one into the other by a rotation of
the circle 51, that is, x (f 2, h2) - x (f 3 , h3) in the notation of §2.5 in Chapter 5.
According to Theorem 2.4 in Chapter 5, f2 and f 3 are conjugate. This implies that
the suspensions sus (f 2) and sus (f 3) are topologically equivalent. EJ

Thus, the characteristic class of a Denjoy flow f t is a complete topological


invariant. It includes information about the Poincare rotation number of f t, the
cardinality of the set of components of T 2 \ S (ft), and their mutual arrangement.
We recall that a characteristic class is a family of pairwise commensurable
sets, each of which is an at most countable collection of trajectories of an irra-
tional winding. The next theorem solves the problem of constructing from a given
characteristic class a Denjoy flow with that characteristic class.

THEOREM 1.6. Let xo be an at most countable collection of trajectories of an


irrational winding fo, and let x be the family of sets commensurable with xo. Then
there exists a Denjoy flow f t such that x(f t) = x.

PROOF. The irrational winding fo induces on the zero meridian no a Poincare


mapping that is a rotation Ra with irrational cx. Obviously, xo = xo n no is an
at most countable family of orbits of Ra. According to Theorem 2.5 in Chapter
5, there exist a Denjoy homeomorphism f : S1 - S1 and a continuous mapping
h: 81 -, $1 = no semiconjugating f and Ra such that xo = x(f, h). Therefore,
def
there is a blowing-down operation h from the Denjoy flow f t sus (f) to the
irrational winding f o = sus (Ra) such that xo = x (f t , h). By Theorem 1.5, x(ft) =
xo
DEFINITION. Let f t be a Denjoy flow. The number of components of the set
T 2 \ SZ (ft) is called the characteristic of the flow ft. Obviously, the characteristic is
a topological invariant of the flow, but not a complete invariant, as the next result
shows.
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 259

LEMMA 1.15. There exists a continuum of topologically nonequivalent Denjoy


flows with the same rotation number and with characteristic equal to 2.
PROOF. According to Lemma 2.9 in Chapter 5, there exists a continuum {f}
of nonconjugate Denjoy homeomorphisms with the same rotation number and
with characteristic equal to 2. We show that the suspensions sus (f ) are pairwise
not topologically equivalent if is a transcendental number, that is, is not a root
of an algebraic equation with integer coefficients. Indeed, if sus (f mil) and sus (f 2 )
are topologically equivalent, then by Theorem 1.4

' c+dp
a+bp'
where d
is a unimodular integer matrix. But the last equality contradicts the
fact that is transcendental. Thus, {sus(f)} is the desired family of flows. U
REMARK. For Denj oy flows f t with characteristic 1 (that is, the set T 2 \1Z(ft) is
connected), just as for transitive flows, the Poincare rotation number is a complete
invariant of topological equivalence up to recomputation by the formula (1.8).
Appendix. Polynomial Cherry flows. Following [79], we show that there
are Cherry flows given by polynomial vector fields of degree 1.
To simplify the notation we define a universal covering :
j2 - T2 by the
formula (x, y) _ eiy) . Then the dynamical system
(x = 1 + cos x + sin y,
(Aa)
y = c(1 + sin x) + cos y
determines on the torus a polynomial vector field X« of degree 1, which in turn
induces a flow f a on the torus. We show that there is a continuum of values cx in
[0, 1] for which the flows f a are Cherry flows.
It suffices to investigate the system (A«) on the square K : 0 < x, y < 2ir, since
maps this square onto the torus, and K is a fundamental domain of the group of
covering transformations of ?.
We shall find and investigate the equilibrium states of the system (Aa). To do
this we form the system
J P« (x, y) = 1 + cos x + sin y = 0,
Q« (x, y) = cx(1 + sin x) + cos y = 0.
Both equalities are possible when cos x < 0, sin y < 0, and cos y < 0; that is, the
equilibrium states of the system (A«) (in K) lie in the rectangle Q : it/2 < x < 37r/2,
it <y < 3ir/2. The curves P« (x, y) = 0, Q« (x, y) = 0 in Q are determined by the
respective relations y = 2ir - aresin (1 + cos x), y = it + arccos [cx (1 + sin x) ] and
intersect at the two points S« (x«, y« ), F(3ir/2, 3ir/2) (Figure 6.9).
We have that
0= app
8x
app
8y = - sin x cos y
= sin x sin y - cx cos x cos y,
aQ« aQa cx cos x - sin y
8x 8y
aP« aQ«
a= _ - sin x - sin y,
ax + ay
0 [F (3ir/2, 3ir/2)] = 1 > 0, a [F (3ir/2, 3ir/2)] = 2 > 0,
and hence F is a simple unstable node.
260 6. CLASSIFICATION OF FLOWS ON SURFACES

QcJT-

r-

l IQl
r

mil `r

FIGURE 6.9

Since 0(Sa) = [(cx2 - 1) cos x - 1] sin x + a2 cos x <0 and (Sa) = 1 + cos x -
sin x = 1 - / sin(x - it/4) < 0 for it/2 < x < 3ir/2, it follows that the roots of
the equation A2 - 0-a + 0 = 0 are real and have different signs. Therefore, Sa is a
saddle for all cx e [0, 1] [3].
Further, Pa(0, y) = Pa(27r, y) = 2 + sin y > 0 and Qa(x, 0) = Qa(x, 27r) =
1+ cx(1 + sin x) > 0, and hence the zero meridian n0 and the zero parallel nl of the
torus (that is, the curves into which the respective lines x = 0 and y = 0 project)
are contact-free cycles of the flows fa, cx e [0, 1]. This yields a qualitative picture
of the phase portrait of the dynamical system (Aa) (Figure 6.10).

FIGURE 6.10

The w-separatrix of Sa intersecting the line x = 0 intersects the family of


lines x = k, k e Z, at most at countably many points. Therefore, there exists
a positive semitrajectory of the dynamical system (Aa) that intersects the lines
x = Zit, k e N, that is, is a curvilinear ray (moreover, there is a continuum of
such semitrajectories). Consequently, the flow fa has rotation number a for each
value of cx E [0,1].
We show that a depends continuously on cx. Indeed, fix cx0 E [0,1] and let to
be a positive semitrajectory of (Aao) intersecting the lines x = 2irk, k e N. Denote
by xxk the arc of to with endpoints x0 and xk lying on the respective lines x = 0
and x = Zit (note that to intersects each line x = Zit, k e N exactly once). On
the line x = 2irk take the point xo (respectively, xo) that lies above (respectively,
below) xk, is congruent to x0, and is closest to xk (xo ; xk if xk is congruent to x0)
(Figure 6.11). We approximate the union of the arc x xk and the segment [xk, xo ]
1. CLASSIFICATION OF IRREDUCIBLE FLOWS ON THE TORUS 261

jX+


X
0

FIGURE 6.11

(respectively, [xi, xk]) by a curve C (respectively, that is transversal to the


trajectories of the system (Aa0).
Since the point x (xi) is congruent to x0, there exists a transformation y1
('y) in the covering transformation group that carries x0 into the point x (xi).
Let x = x+ (t) , y = y+ (t) be parametric equations of the curve

C+ = u()(Cj),
where the union is over all n e Z. Then C+ is transversal to the trajectories of
the system (Aao) and projects into a closed curve on the torus. It follows from
the congruence of x0 and xo that the limit limty+ (t) /x+ (t) -1 i4 is a rational
number.
The number e Q is introduced similarly for the curve C- = U(y)n(C ),
nEZ.
It follows from the construction that ,u <«o < ,u. For values of cx suffi-
ciently close to cx0 the curves C+ and C- are transversal to the trajectories of the
system (A«), so <«< On the other hand,
4. and tend to p 0 as
k - +oo. This implies that the rotation number depends continuously on the
parameter cx.
Moreover, we have the following theorem.
THEOREM. Suppose that in the space Xr (T2) of flows, r > 1, there is a family
of flows fa, cx e [0, 1], depending continuously on a parameter cx (that is, the
mapping cx H f a defines a continuous curve in the space X r (T 2) ), and suppose that
the flows fa, cx e [0, 1], have a common contact-free cycle C. Assume also that
for each cx e [0, 1] the flow fa has a semitrajectory intersecting C countably many
times. Then there is a rotation number rot(f) for all cx e [0, 1], and it depends
continuously on o.
The proof of the theorem is left to the reader as an exercise.
A direct qualitative investigation of the dynamical systems (A0) : x = 1 +
cos x + sin y, y = cos y and (A1) : x = 1 + cos x + sin y, y = 1 + sin x + cos y shows
that their phase portraits have the form pictured in Figure 6.12, a), b), respectively.
Consequently, quo = rot(f) = 0 and i1 = rot(fl) = 1. Since the rotation number
depends continuously on the parameter, there is a continuum of values cx for which
the rotation number = rot(f) is irrational. According to Theorem 1.2, the
corresponding flows f a have nontrivial recurrent semitrajectories. These flows are
Cherry flows because of the presence of a hyperbolic (or structurally stable) saddle.
262 6. CLASSIFICATION OF FLOWS ON SURFACES

b)

FIGURE 6.12

§2. The homotopy rotation class


The asymptotic behavior of the trajectories of a flow covering an irreducible
flow on a closed orientable surface of genus greater than 1 differs essentially from
the asymptotic behavior of the trajectories of a flow covering an irreducible flow on
the torus (genus 1). While the trajectories of a flow covering an irreducible flow on
the torus have two diametrically opposite asymptotic directions (the corresponding
flow has two accessible points on the absolute, and this enables us to introduce a
topological invariant like the Poincare rotation number), the trajectories of a flow
covering an irreducible flow on a closed orientable surface of genus greater than
1 have a continuum of asymptotic directions, and this leads to a new topological
invariant the homotopy rotation class. This section is devoted to the definition of
the homotopy rotation class and to an investigation of its properties.
We first describe the Lobachevsky plane, which is a universal covering of a
closed orientable surface of genus > 1.
2.1. Lobachevsky geometry and uniformization. The non-Euclidean ge-
ometry of Lobachevsky is realized as the following model, proposed by Poincare.
Take the open unit disk 0 = {z: Izi <1} in the complex z-plane and introduce on
0 the metric
4(dx2 + dy2)
d2 s= (1-x2- 2 2'
y)
where z = x + iy, (x, y) being the Cartesian coordinates in f2.
The disk 0 with this metric is called the Lobachevsky plane (the Poincare
model).
Arcs of Euclidean circles orthogonal to the boundary 90 of 0 (in particular,
the arcs of Euclidean lines passing through the origin of coordinates and orthogonal
to 90) are the geodesics of the Lobachevsky plane.
The boundary S def aQ = {z : Izi = 1} of 0 is called the absolute.
The set I(0) of orientation-preserving isometries of the Lobachevsky plane
coincides with the group of fractional linear transformations
pz+q
zH_
qz+p
where p, q e (C and p 2 - 1q12 = 1. Each such nonidentity transformation has one or
two fixed points in the closed disk 0 U S. This leads to the following classification
of the isometries of the Lobachevsky plane.
2. THE HOMOTOPY ROTATION CLASS 263

An isometry y e I (L) is said to be elliptic if it has a fixed point interior to L.


An isometry y e I (s) is said to be hyperbolic if it has two fixed points on the
absolute.
An isometry y e I (L) is said to be parabolic if it has exactly one fixed point,
which lies on the absolute.
Any orientation-preserving isometry of the Lobachevsky plane is one of these
transformations.
An important class of subgroups of I(0) is made up of the Schottky groups.
A Schottky group F of genus p > 1 is defined to be a finitely generated sub-
group of I(0) consisting solely of hyperbolic isometries and having 2p generators
{ai, b1, ... , aP, bP} C I(0) which satisfy the relation
aibia1 1bi 1 ... aPbPaP 1bP 1 = id.
According to the Klein-Poincare-Koebe uniformization theorem [43], for any
closed orientable surface M of genus p > 2 there exists a Schottky group I'P of
genus p acting freely on 0 and with 0 as a domain of discontinuity (see Chapter
1) such that M 0/FP, and the natural projection it: O - O/I'P M, is a
universal covering of the surface M. Moreover, the limit set of the group I'P (the
set of accumulation points of the orbits F (z0) _ {y(zo) : y e F}) coincides with
the absolute Sam, and the set of fixed points of the isometries in I'P is countable
and dense in S.
The Lobachevsky plane 0 is a Riemannian manifold of constant negative cur-
vature -1. The covering it : 0 -* 0/F induces on P a metric of constant negative
curvature -1.
Points a, b e O are said to be congruent with respect to the group I'P if there
exists an element y e I'P such that y(a) = b. Obviously, ir(a) _ ir(b) for congruent
points. Conversely, if points a, b e 0 belong to the complete inverse image i-1(mo)
of a point m0 E M, then a and b are congruent.
Unless stated otherwise, it is assumed below that M = 0/F is a closed
orientable surface of genus p > 2, and I'P is a Schottky group of genus p > 2 acting
freely on O and with domain of discontinuity &
2.2. The axes of hyperbolic isometries. Consider a hyperbolic isometry
y E I'P in a Schottky group I. One fixed point y+ of y is attracting, and one
is repelling. The iterates yn(zo) of any point z0 different from y+ and y- (zo can
also lie on Sam) tend to ( as n -4 boo and to y- as n - -oo.
We draw the geodesic O(y) through y+ and y-. Since there is a unique (up to
orientation) geodesic joining the points y- and y+ in 0, and since the isometry y
carries geodesics into geodesics, it follows that y [O (y)] = O (y) .
The geodesic O(y) is called the axis of the element y e I.
The axis O ('y) is the unique geodesic on 0 that is invariant under y.
The elements of F with respect to which the geodesic O(y) is invariant form
an infinite cyclic subgroup {'r}' n e Z, generated by some y1 E F.
The element y1 is called a minimal element of the axis O (y) . This element
(and also yj 1) moves the axis along itself by a minimal distance.
LEMMA 2.1. On a closed orientable surface M of genus p > 2 let L be a closed
geodesic that is nonhomotopic to zero. In this case: _
1) the inverse image it (L) is made up of countably many geodesics {LZ}1
of the Lobachevsky plane;
264 6. CLASSIFICATION OF FLOWS ON SURFACES

2) each geodesic LZ E it -1(L) is the axis of some element y2 E


3) any geodesics LZ, L3 E it-1(L), i ; j, do not have common endpoints;
4) if {LZn }°° 1 is a subsequence of distinct geodesics LZn in it -1(L) with end-
points a1(LZn) and 2 (L), and the sequence {ai (LZn ) } converges to a point a e
Sam, then the sequence {o2(L)} also converges to a, and, moreover, the topological
limit 1 of the geodesics {LZn } is equal to a.
PROOF. By the covering hom_otopy theorem and the fact that 7r is a local
isometry, there exists a geodesic L_ C 0 such that ir(L) = L. Since L is closed,
there are two points z1 and z2 on L such that it (zl) = it (z2) = z, and the distance
between z1 and z2 is equal to the length of the geodesic L. Obviously, L has only
finitely many self-intersections, and it can be assumed that there is not a self-
intersection at z. By the equality it (zl) = it (z2) , there exists a y e F such that
y(zl) = z2. Since there is not a self-intersection at z e L, it follows that y(L) = L;
that is, L is the axis of the element y, L = O (y) . The assertion 2) is proved.
By construction, the distance between z1 and z2 is equal to the length of L.
Therefore, y generates an infinite cyclic subgroup F(L) of elements with respect to
which L is invariant.
The group I'P is a group with 2p > 4 generators and a single defining relation.
Therefore, the factor group F /F(L) is countable [53], and the complete inverse
image it-1(L) = {'y(L) : y e F} consists of countably many geodesics. The
assertion 1) is proved.
We prove 3). Assume the contrary. Suppose that geodesics LZ, L3 E it
i ; j, have a common endpoint 1 E S. The endpoint 2i of LZ is different from
the endpoint 23 of L3 because otherwise LZ = L3 (Figure 6.13).

(o2j

FIGURE 6.13

According to 2), LZ is the axis of some element y2 E F. Suppose for definiteness


that a1 = y2 and 2i = 'y. Then 'y (a2 j) - 02i, and 'y (L3) - * LZ as n -* oo. The
geodesics 'y3) project onto the geodesic L on M. Consequently, a neighborhood
of any point m e L intersects L in a countable set of arcs, which contradicts the
closedness of L. This proves 3).

1 The topological limit of a sequence of sets { Ai } 1 is defined to be the union of all possible
limits of sequences { ai } °_ 1 with ai E A.
2. THE HOMOTOPY ROTATION CLASS 265

It remains to prove 4). If the sequence {a2 (LZn ) } has a subsequence converging
to a* ; a, then the subsequence of corresponding geodesics would converge to the
geodesic joining a to a. Then some neighborhood of a point on M would intersect
L in countably many arcs, which is impossible. Therefore, a2 (LZn) -* a.
From this and the fact that the geodesics in 0 are Euclidean circles perpen-
dicular to S it follows that the topological limit of the sequence {LZn } is equal
to a. 0

The next result is a direct consequence of the proof of 3).


COROLLARY 2.1. If two elements y1, 72 E P have a common fixed point, then
their axes coincide (that is, they have two common fixed points). Two axes either
coincide or do not have common endpoints.
LEMMA 2.2. Let C be a simple closed curve that is nonhomotopic to zero on a
closed orientable surface M of genus p > 2. In this case:
1) the inverse image it -1(C) consists of a countable set of disjoint curves, each
with two endpoints lying on the absolute;
2) for each curve C E it -1(C) there exists an element y e F, y id, with
respect to which C is invariant, and C and the axis O(y) have common endpoints;
3) an element y e F with y ; id that carries some point of a curve C E
into a point on C leaves the whole curve C invariant;
4) distinct curves C1, C2 E it -1(C) do not have common endpoints; _
5) if {C}i° is a sequence of distinct curves CZ in it -1(C) with endpoints a1(CZ)
and a2 (CZ) and if the sequence {ai (C)}r of points converges to a point a e Sam,
then the sequence {a2(C)}i° also converges to a, and, moreover, the topological
limit of the curves {C}° is a.
PROOF. As in the proof of Lemma 1.1, we construct a curve C C 0 with
endpoints lying on the absolute and such that ir(C) = C and there exists an element
y E FP with respect to which C is invariant. There is a unique geodesic L C 0 with
endpoints coinciding with those of C. Since y(C) = C, it follows that y(L) = L.
This implies that ir(L) = L is a closed geodesic on M that is homotopic to C. Since
C is simple, so is L. We get from the properties of a covering that for each curve
C E it -1(C) with endpoints on the absolute there exists a geodesic L E it-1(L)
such that C and LL have common endpoints. The required assertions follow from
this and Lemma 2.1. U

2.3. Asymptotic directions. The concept of a curvilinear ray is defined as


in §1.2, to be a half-closed infinite curve l (on the surface M or on the universal
covering 0) that has a lifting l C 0 (if l C 0, then l = l) such that l leaves any
compact set of the Lobachevsky plane 0) .
It follows from the properties of the group I'P (in particular, the fact that I'P
consists of isometries) that all the curves in 0 congruent to a curvilinear ray l C 0
are also curvilinear rays. Consequently, if l C M is a curvilinear ray, then all the
curves in the complete inverse image it -1(l) are curvilinear rays.
As in §1.2, when we call a semitrajectory of some flow onP or O a curvilinear
ray, we shall have in mind the natural parametrization of this semitrajectory with
respect to time.
266 6. CLASSIFICATION OF FLOWS ON SURFACES

If a semitrajectory l of the flow f t on 0 is a curvilinear ray, then its w- (a-)


limit set lies on the absolute.
DEFINITION. Let l0 be a positive (negative) semitrajectory of a flow f t on O,
and let it be a curvilinear ray. We say that l 0 has an asymptotic direction if its w-
(ce-) limit set consists of a single point o-(10) belonging to the absolute.
The point a(l0) is said to be accessible by the semitrajectory l0.
Let 10 be a semitrajectory of a flow f t on and let it be a curvilinear ray.
ft
If at least one semitrajectory l0 E 7r-1(10) of a covering flow on O has an
asymptotic direction, then any semitrajectory in 7r-1(10) also has an asymptotic
direction. In this case we say that l has an asymptotic direction.
Everywhere below we consider flows on = 0/I' with finitely many equi-
librium states (unless otherwise stated).
THEOREM 2.1. Suppose that a semitrajectory 10 of a flow f t on M (p > 2)
intersects a contact-free cycle C infinitely many times. Then 10 is a curvilinear
ray having an asymptotic direction.
PROOF. For definiteness we assume that l0 is a positive semitrajectory l+.
By Lemma 2.2 in Chapter 2, C is nonhomotopic to zero (even nonhomologous
to zero). Therefore, any curve C E 7r -1(C) separates 0 in view of Lemma 2.2.
Let f t be a covering flow for ft. Since C is a contact-free arc of f, any
covering semitrajectory l+ E it -1(l+) intersects any curve C E it -1(C) at no more
than one point. Consequently, l+ intersects a countable family {C} of disjoint
curves in 7r -1(C) (Figure 6.14). Denote by a1(Ci) and a2 (CZ) the endpoints of CZ
as pictured in Figure 6.14. Simple geometric considerations show that the points
in {o-i (C) form a bounded monotone sequence, and hence this sequence has a
limit a E S. We get from Lemma 2.2, 5) that CZ -k a, and therefore w(l+) = a.
U

FIGURE 6.14

DEFINITION. We say that a pair of points x1, x2 E S is separated on the


absolute by a pair x3, x4 E S if one arc in the set S\ {x3, x4} contains x1 and the
other contains x2.
The next result follows immediately from the proof of Theorem 2.1.
2. THE HOMOTOPY ROTATION CLASS 267

COROLLARY 2.2. Suppose that the conditions of Theorem 2.1 hold and a cov-
ering semitrajectory l 0 for 10 intersects the family {CZ } of curves in the inverse
image 7r-1(C) . Then a ) (l0) = w(i) (c(l0)) coincides with the topological limit
of the family {CZ}, and hence for any point m E S different from a0 (l0) there
is an index io such that the pair of points m, o ) (l0) is separated by the pair of
points a1(CZ), a2 (CZ) for all i > io, where a1(CZ) and a2 (CZ) are the endpoints of
CZ .

LEMMA 2.3. Let f t be a flow on a closed orientable surface M of genus p > 2,


t t
and let f be a flow on 0 covering it. Then the semitrajectories of f t and f listed
below are curvilinear rays and have an asymptotic direction:
1) a nontrivial recurrent semitrajectory of f t;
t
2) a semitrajectory of f projecting into a closed trajectory of ft that is non-
homotopic to zero;
3) a nonclosed semitrajectory of f t whose limit set contains a nontrivial re-
current semitrajectory, or a closed trajectory that is nonhomotopic to zero, or a
one-sided contour that is nonhomotopic to zero.
PROOF. Let l be a nontrivial recurrent semitrajectory of f t. According to
Lemma 2.3 in Chapter 2, there exists a contact-free cycle intersecting l This
and Theorem 2.1 yield 1). The assertion 2) follows from Lemma 2.2, and 3) follows
from 1) and 2). U

COROLLARY 2.3. Assume the conditions of Lemma 2.3. Then any semitrajec-
t
tory of f t or f that is a curvilinear ray has an asymptotic direction. Moreover,
only those semitrajectories in Lemma 2.3 are curvilinear rays.
PROOF. This follows from Lemma 2.3 and Theorem 3.6 in Chapter 2 (the
catalogue of limit sets). See also the proof of Theorem 1.1. U

REMARK 1. We remark that Corollary 2.3 fails in general for flows f t on


with an infinite set of equilibrium states.
REMARK 2. The question of the existence of an asymptotic direction of a curvi-
linear ray has been answered most thoroughly in the following papers.
1. D. V. Anosov, On the behavior of trajectories in the Euclidean plane that
cover trajectories of flows on closed surfaces, I, II, Izv. Akad. Nauk SSSR Ser.
Mat. 51 (1987), 16-43, 52 (1988), 451-478; English transl. in Math. USSR Izv. 30
(1988), 32 (1989).
2. D. V. Anosov, On infinite curves on the torus and closed surfaces of negative
Euler characteristic, Trudy Mat. Inst. Steklov 185 (1988), 30-53; English transl.
in Proc. Steklov Inst. Math. 1990, issue 2.
In particular, Anosov proved the following result.
THEOREM. If the set of equilibrium states of a flow f t on a closed surface M
with nonpositive Euler characteristic is contractible to a point on M, then each
t
semitrajectory of a covering flow f either is bounded or tends to some point of the
absolute.
268 6. CLASSIFICATION OF FLOWS ON SURFACES

2.4. Arithmetic properties of the homotopy rotation class. We fix a


group p > 2, of hyperbolic isometries of the Lobachevsky plane L. With respect
to this group the points of the absolute S break up into two classes.
DEFINITION. A point a E S is said to be rational if a is a fixed point of some
element -y E G with 'y ; id. Points of the absolute that are not rational are said
to be irrational.
The set of rational points (and hence the set of irrational points) is invariant
under the action of I'. Indeed, if a E S is a fixed point of an element 'y E I' and
6 E I'r, then 6(a) is a fixed point of the element 6 o o 6' E I'.
Suppose that a flow f t on = 0/I' has a semitrajectory 10 that is a
curvilinear ray and hence has an asymptotic direction (Corollary 2.3). We consider
t
an arbitrary semitrajectory l0 of a flow f on 0 such that ir(10) = i0, and let
(l ) be a limit point of l 0 belonging to the absolute.
DEFINITION. The homotopy rotation class of the semitrajectory l of the flow
f t on M is defined to be the set

µ(i0) = U [a(l)].
Erp

In other words, the homotopy rotation class (HRC) of the semitrajectory l is


obtained by taking the union of the limit points of all semitrajectories covering l
(Figure 6.15), because ry[a(l )] = a[ry(l )].

FIGURE 6.15

It follows from the invariance of the set of rational (and irrational) points with
respect to the group I'r and the definition of the HRC that the HRC of a particular
semitrajectory i ) lies either in the set of rational points, or in the set of irrational
points. Therefore, the following definition is unambiguous.
DEFINITION. The homotopy rotation class of a semitrajectory i ) of a flow on
M is said to be rational (irrational) if it consists of rational (irrational) points of
the absolute.
2. THE HOMOTOPY ROTATION CLASS 269

THEOREM 2.2. 1) The homotopy rotation class of a closed trajectory that is


nonhomotopic to zero is rational, as is the HRC of a nonclosed semitrajectory whose
limit set contains a closed trajectory nonhomotopic to zero or a closed one-sided
contour nonhomotopic to zero.
2) The HRC of a nontrivial recurrent semitrajectory is irrational.
PROOF. If l is a closed trajectory nonhomotopic to zero, then by Lemma 2.2,
2), each curve l E 7r -1(l) is invariant with respect to some element of the group I'.
Therefore, p (l) is rational.
If 10 is a nonclosed semitrajectory whose limit set contains a closed trajectory
l0 nonhomotopic to zero, then for any E-neighborhood U (lo) of l0 there is a moment
of time beginning with which l enters U (lo) and does not leave it again. This
implies that the HRC (l0) = µ(lo) is rational. Analogous arguments prove that
the HRC is rational in the case when the limit set of l contains a one-sided contour
nonhomotopic to zero.
We prove 2). Let l+ be a P+ nontrivial recurrent semitrajectory, and let l+ be a
semitrajectory on 0 covering it. Assume the contrary. Then o (l +) = w (l +) E S
is a fixed point of some element 'y E I'. It can be assumed that Q-+ (1+) = ,y+ is an
attracting point of 'y (otherwise ry-1 could be taken instead of ry), and 'y a minimal
element.

FIGURE 6.16

According to Lemma 2.3 in Chapter 2, there exists a contact-free cycle C


intersecting l+. Then from Theorem 2.1 and Corollary 2.2 it follows that there is
a curve C E 7r -1(C) with endpoints 1(C), a2 (C) E S such that l+ intersects
C, and the pair of points ,y+, ry- is separated by the pair of points 1(C), ci2 (C)
(Figure 6.16). Therefore, l+ intersects the curve y(C) .
The curves C and 'y(C) separate 0 and are contact-free arcs of the covering flow
+
f t Consequently, l intersects C and 'y() in only a single point. Let a = l + f1 C
.

and b = l+ n ry(C). We approximate the curve made up of the segments ab C l+


and (a) b C 'y() by an arc A joining a and 'y() and transversal to the flow f t (see
t
Figure 6.16). Then the curve S = UThE7Z 'y( A) is transversal to f and invariant
with respect to 'y, and it has endpoints ,y+ and #y- .
270 6. CLASSIFICATION OF FLOWS ON SURFACES

Since ,y(j-+) =it follows that a+[ryT(l+)] = a+(l+) for any n E 7L. There-
fore, the semitrajectories l+(ry(a)) = y(l+(a)) and l+(ry-1(a)) = 'y (1 (a)) and
the arc d of S between al = ry(a) and a_1 = #y-1 (a) form a curvilinear triangle
T (Figure 6.17). A positive semitrajectory of f t upon entering T across the arc d
cannot leave T with increasing time.

FIGURE 6.17

Since lr(l+) is a nontrivial recurrent semitrajectory, a semitrajectory li con-


gruent to l + (that is, there is an element 'Y1 E I' such that 'y1(l +) = l) intersects
d at a point c arbitrarily close to a. It follows from the properties of the triangle
T that w(1 1) = a+ = +; therefore, 'Y1 (a+ ) = a+, and hence either 'y1 = 'y+ or
ryj = ,y+. According to Corollary 2.1, the axes of 'y and 'Y1 coincide. Since 'y is a
minimal element, 'Y1 = 'Y for some k E Z \ {O}.
By construction, ry(S) = S. Therefore, 'y1(S) = S. But then 'y1 (a) =
'y1(S n l + ) = 'y1(S) n 'y1(1 + ) = S n l i = c; that is, the points a and c are con-
gruent. Thus, any neighborhood of a contains points congruent to a, and this
contradicts the fact that ir is a covering. U

COROLLARY 2.4. Let l be a nontrivial recurrent semitrajectory, and let lid


and l2 be distinct coverings of it. Then a ) (iv) ; a( ) (lv).

PROOF. Since l 1 and l2 are congruent, it follows that ry [l i ] = l2 for some


E I'r. If a0 (l i ) = a0 (l2 ) - i a, then 'y(a) = a, and hence a is a rational point,
which contradicts Theorem 2.2. U

2.5. The homotopy rotation class of a nontrivial recurrent semitra-


jectory. All nontrivial recurrent positive (negative) semitrajectories of a flow on
the torus have the same asymptotic direction; that is, on the disk D2 any covering
semitrajectories for P+ (P-) nontrivial recurrent semitrajectories tend to the same
point of the absolute. On closed orientable surfaces of genus p > 2 the situation is
essentially different from that on the torus.
2. THE HOMOTOPY ROTATION CLASS 271

DEFINITION. A boundary nontrivial recurrent semitrajectory or trajectory l of


a flow f t is one for which any point m E l has a neighborhood U(m) m such that
the arc d in U(m) n l containing m separates u(m) into two domains, one of which
does not contain points lying on nontrivial recurrent semitrajectories of ft.
REMARK. Boundary trajectories are called singular trajectories in the paper of
Aranson and Grines, Topological classification of flows on closed two-dimensional
manifolds (Uspekhi Mat. Nauk 41 (1986), no. 1, 149-169; English transl. in Russian
Math. Surveys 41 (1986), no. 1).
A nontrivial recurrent semitrajectory or trajectory is a boundary semitrajectory
or trajectory if it intersects any contact-free cycle or segment at the endpoints of
intervals that do not intersect any nontrivial recurrent semitrajectories.
In a Denjoy flow ft the accessible (from within) boundary of each component
of T 2 \ SZ (ft), where SZ (ft) is the minimal set of ft, consists of boundary nontrivial
recurrent trajectories. In the general case if N is a quasiminimal set of a flow f t
on a surface M, then the boundary nontrivial recurrent semitrajectories in N are
in the accessible (from within) boundary of the components of M \ N.
We remark that a boundary nontrivial recurrent trajectory is exceptional (that
is, its closure is nowhere dense in the manifold).
An interior nontrivial recurrent semitrajectory or trajectory is one that is not
a boundary semitrajectory or trajectory (in other words, such a trajectory 1`ap-
proaches" itself from both sides).
In a transitive flow all the nontrivial recurrent semitrajectories and trajectories
are interior.
THEOREM 2.3. Suppose that a flow f t on a closed orientable surface M with
t
p > 2 has nontrivial recurrent semitrajectories, and let f be a covering flow on the
Lobachevsky plane L. Then the following hold.
1) Any point of the absolute is a limit point of at most two semitrajectories of
f t that cover nontrivial recurrent semitrajectories of ft.
2) If a E S is a limit point for a lifting of an interior nontrivial recurrent
t
semitrajectory of f t, then a is the limit of exactly one semitrajectory of f In .
t
particular, if f t is a transitive flow, then at most one semitrajectory of f tends to
any point of the absolute.
t
3) If a E S is a limit point for two semitrajectories of f covering nontrivial
recurrent semitrajectories 4) and 4) of f t, then l i and 4) are boundary nontrivial
recurrent semitrajectories (both positive or both negative) that belong to a single
quasiminimal set N of f t and are in the accessible boundary of the same component
of the set M \ N. Conversely, if a E S is a limit point for a semitrajectory l i of
f t covering a boundary nontrivial recurrent semitrajectory l i in some quasiminimal
set N of f t, then there exists a unique boundary nontrivial recurrent semitrajectory
4) ; ll that lies in N and has a covering semitrajectory l2 such that a = a(l).
PROOF. 1) Assume the contrary; that is, let a = a (l ) E S, i = 1, 2, 3,
where it (l ?) is a nontrivial recurrent semitrajectory. By Theorem 2.1 and Corollary
2.2, there is a contact-free arc C of f t with endpoints on the absolute which is
intersected by all three semitrajectories l i , l2 , and l3 (Figure 6.18). This implies
272 6. CLASSIFICATION OF FLOWS ON SURFACES

that the semitrajectories l , i = 1, 2, 3, are either all positive or all negative. For
definiteness we assume they are positive. Let 7i2 = li n C, i = 1, 2, 3, and suppose
that the point a2 lies on the arc al a3 of C between the points al and a3 . Since
ir(l2 ) is a nontrivial recurrent semitrajectory, the arc al a3 intersects a positive
semitrajectory l + congruent to l2 . But then w (l + ) = o (l + ) = a, which contradicts
Corollary 2.4.

FIGURE 6.18

2) Assume the contrary, that is, let a be the limit for the semitrajectories l0
and l i , where it (l 0) is an interior nontrivial recurrent semitrajectory. As in the
proof of 1), we construct the Bendixson "bag" bounded by 10, l i , and the arc a al
of the contact-free segment C, where a = l n C and al = l i n. Since it (l 0) is an
interior nontrivial recurrent semitrajectory, the arc a al intersects a semitrajectory
l2 congruent to l . Then a(i) = a, and this contradicts Corollary 2.4.
We prove 3). Suppose that a(lit) = a(l), where l= ir(l), i = 1, 2, are
nontrivial recurrent semitrajectories. By 2), 4) and 4) are boundary semitrajecto-
ries.
Denote by N the quasiminimal set containing l i , that is, N = cl (l i ) Accord-
.

ing to Lemma 4.1 in Chapter 2, there exists a contact-free cycle C that intersects
N and is disjoint from quasiminimal sets different from N. By Theorem 2.1 and
t
Corollary 2.2, there is a contact-free arc C E it-1(C) of f intersecting l i and l2 .

Therefore, 4) n C ; 0, and hence 4 C N.


It follows from 1) that the arc d of C between the points l i n C and l2 n C is
disjoint from all inverse images of nontrivial recurrent semitrajectories. Therefore,
ir(d) is disjoint from quasiminimal sets, and thus 4) and 4) are in the accessible
boundary of the same component of the set M \ N.
Conversely, suppose that a = a (l i ) E S, where it (l i ) = l i is a boundary
nontrivial recurrent semitrajectory lying in a quasiminimal set N of f t. We take
a contact-free cycle C with C n N ; 0 that does not intersect quasiminimal sets
different from N (Lemma 4.1 in Chapter 2), and we denote by P : C -k C the
2. THE HOMOTOPY ROTATION CLASS 273

Poincare mapping induced by ft. Since f t has boundary nontrivial recurrent semi-
trajectories by assumption, the intersection C n N def 1 is a Cantor set, and li
intersects C at the endpoints of adjacent intervals of f
For definiteness we assume that li is a positive semitrajectory. Denote by
{ In } _ 1 the sequence of adjacent intervals of 1 whose endpoints intersect l j as
time increases. Let In = (an, bn) C C, where an E 11F n C and P (an) = an+ 1
n E N. Trajectories in the quasiminimal set N pass through the points bn. Ac-
cording to Theorem 3.4 in Chapter 2, the regular trajectories in a quasiminimal
set are separatrices joining equilibrium states, or separatrices that are nontrivial
recurrent semitrajectories, or nontrivial recurrent trajectories. Therefore, in view
of the assumption that there are finitely many equilibrium states, the points bn lie
on a nontrivial recurrent semitrajectory l2 for a sufficiently large index (n > no).
Obviously, l2 is a boundary semitrajectory.
We show that there is a semitrajectory l2 covering l2 such that a(l2) = a.
By Corollary 2.2, the semitrajectory li intersects a countable family {CZ}°_1 of
contact-free arcs in 7r 1(C) with a as their topological limit. Let a2 = l + n CZ Then .

r(a2) = ai, and a2 is an endpoint of an interval IZ C CZ such that ir(IZ) = IZ C C.


Denote by b2 the endpoint of I z different from a2 . Then it (b2) = b.
By the assumption that the number of separatrices is finite, there is an index
nl such that the Poincare mapping P is defined on In for n > n1. Therefore,
the domain bounded by In , In+ i C C and the arcs a an+ 1 and bn bn+ 1 of the
semitrajectories li and l2 is simply connected. It follows from the properties
of the covering it that the points bn with n > max{no, nl } belong to a single
semitra'ector l+ covering l+. Since a is the topological limit ofC{}, a = a l+

2.6. The connection between quasiminimal sets and geodesic lamina-


tions. We fix a Riemannian metric of constant curvature -1 on a closed orientable
surface M of genus p > 2.
DEFINITION. A geodesic lamination 3 on M is defined to be a closed set
consisting of disjoint simple2 geodesics.
A simple closed geodesic provides a trivial example of a geodesic lamination.
The topological closure of a simple nonclosed geodesic is a more complicated ex-
ample.
We recall the definition of the geodesic flow. Denote by Tl M the space of unit
vectors tangent to the surface M. The geodesic flows is defined as follows to be
a one-parameter group of diffeomorphisms of the space Tl M : in the time t each
vector e E Tl M is shifted along the geodesic tangent to it by a distance t while
remaining tangent to this geodesic (Figure 6.19).
To each orientable geodesic on M there corresponds a unique trajectory of the
geodesic flow, and conversely.
We recall that a trajectory of a flow on a compact manifold is B-recurrent if
its closure is a compact minimal set.
DEFINITION. A geodesic on M is called a nontrivial B-recurrent geodesic if a
nonclosed B-recurrent trajectory corresponds to it in the geodesic flow.
2A geodesic (closed or nonclosed) is said to be simple if it does not have self-intersections.
274 6. CLASSIFICATION OF FLOWS ON SURFACES

FIGURE 6.19

A geodesic lamination is said to be nontrivial if it coincides with the closure of


a simple nontrivial B-recurrent geodesic.

DEFINITION. A nontrivial B-recurrent geodesic L C M is called a boundary


geodesic if any point m E L has a neighborhood U(m) m such that the arc d in
U(m) n L containing m divides U(m) into two domains, one of which does not have
points lying on L.
A nontrivial B-recurrent geodesic L is said to be interior if, for any neighbor-
hood U(m) of any point m E L, there are points on L in both the domains into
which the arc d in U(m) n L containing m divides U(m).
In a nontrivial geodesic lamination 3 C M (which is locally homeomorphic to
the direct product of the line and the Cantor set) the boundary geodesics form the
accessible (from within) boundaries of the components of M \ 3.
Two boundary geodesics in the accessible (from within) boundary of a single
component of Mr \ 3 are said to be associated.
The definition of a boundary (interior) geodesic of a nontrivial geodesic lami-
nation is completely analogous to the concept of a boundary (interior) trajectory
in a quasiminimal set. It is clear that a simple boundary (interior) nontrivial B-
recurrent geodesic L is a boundary (interior) geodesic of the nontrivial geodesic
lamination 3 = cl(L).
In this subsection we indicate a way of constructing a nontrivial geodesic lam-
ination from a quasiminimal set of a flow.
t
We consider a flow f t on Mr, p > 2, with a quasiminimal set N, and let f be a
t
covering flow on 0 for ft. Take a trajectory l of f covering a nontrivial recurrent
trajectory l C N. According to Theorem 2.3, the points o (l) = a (l) , o (l) =
w(l) E S are distinct. We join them by an orientable geodesic L(l) from o to
-+ (Figure 6.20).

FIGURE 6.20
2. THE HOMOTOPY ROTATION CLASS 275

By the properties of the group -- def= L(l) on the surface


the geodesic 7r[L(l)]
M is independent of the choice of the covering trajectory l E 7r -1(l). In this
notation we have the following result.
LEMMA 2.4. Let a flow with quasiminimal set N be given on p > 2, and
let l C N be a nontrivial recurrent trajectory. In this case:
1) L(l) is a simple nontrivial B-recurrent geodesic;
2) if l is an interior trajectory in N, then L(l) is an interior geodesic;
3) if ll is a boundary trajectory in N belonging to the accessible (from within)
boundary of a component w of M \ N, and if a nontrivial recurrent trajectory l2 is
in the accessible (from within) boundary of w and has a covering trajectory l2 for
which w(l2) = w(l1), where l1 is a covering trajectory for ll, then
a) L(l1) and L(12) are associated boundary geodesics when a(ll) = o (ll)
o (l2) = a(l2),
b) L(l1) is an interior geodesic when a(ll) = a(l2) .

PROOF. 1) Suppose that = w(l) and o = a(l), where l is a covering trajec-


tory for 1. If L (l) is not a simple geodesic, then L(i) n ry [L (l) ] ; 0 for some noniden-
tity element 'y E I'. Then the pairs of points (o, o) and (o('y(L)), c+ (ry (L)) )
are separated on the absolute, and hence l n y(l) ; 0, which is impossible (Figure
6.21).

FIGURE 6.21

According to Theorem 2.2, c and are irrational; therefore, the geodesic


L(l) is nonclosed.
It follows from Corollary 2.2 that there exists a sequence {C},0 of contact-
t
free arcs of f such that CZ n l ; 0, i E 74 and the point (o) is the topological
limit of the arcs {CZ} as i - +oo (respectively, i - -oo). The continuous
dependence of trajectories of the initial conditions and the self-limit property of
the trajectory l imply the existence of a sequence of elements ryj E k E N, such
that o ['y(l)] - as k -k oo (Figure 6.22). Therefore, the sequence of geodesics
Lk = L (ryk (l)) converges to the geodesic L (l) as k -k oo. This means that in the
geodesic flow a nontrivial recurrent trajectory corresponds to the geodesic L(l).
Since there are no equilibrium states in the geodesic flow, the recurrent trajectories
are B-recurrent. Consequently, L(l) is a B-recurrent geodesic. The assertion 1) is
proved.
276 6. CLASSIFICATION OF FLOWS ON SURFACES

FIGURE 6.22

2) Since l is an interior trajectory, there exists a sequence of geodesics Lk


congruent to L that converge to L from both sides. Therefore, L(1) is an interior
geodesic.
We prove 3), a). The geodesics 7r[L(l1)] and 7r[L(l2)] are disjoint because a
nonempty intersection would imply (the argument is analogous to the proof of 1) )
that trajectories of f t can intersect. Therefore, in the _domain _R C 0 bounded
by L(l1) and L (l2) there are no geodesics congruent to L(l1) or L(i2) and having
endpoint o+. From this, ir[L(l1)] and ir[L(l2)] are associated boundary geodesics.
It remains to prove 3), b). Denote by w the domain bounded by trajectories
l1 and l2. Since ir(l1) and ir(l2) are nontrivial recurrent trajectories, there are
domains congruent to w arbitrarily close to w from both sides. Consequently, there
are geodesics congruent to L = L(11) = L(l2) arbitrarily close to L from both sides,
and hence 7r(L) is an interior geodesic. D

DEFINITION. A family consisting of separatrices l1, ... , is and equilibrium


states m1,. . . , m3 _ 1 (s > 2) is called a right-sided (left-sided) Poisson pencil if
the following conditions hold:
1) w(li) =m, i = ,1,...,s-
... , s - 1;
2) a(li) =m_1, i = 2, ... , s;
3) li is a Bendixson extension of the separatrix li_1 to the right (left), i =
2,...,s;
4) the separatrices l1 and is are P- and P+ nontrivial recurrent trajectories,
respectively (Figure 6.23).

FIGURE 6.23

A right-sided or left-sided Poisson pencil is said to be one-sided.


Denote by P(11,. .. , ls) the one-sided Poisson pencil containing the separatrices
11i...,ls.
2. THE HOMOTOPY ROTATION CLASS 277

The complete inverse image 7r-1 [P(11,. ,1 )] breaks up into a countable family
. .

of disjoint curves i (li,. . . , is), i e N. Each such curve i (li,. , , , 1S) def j5i consists
of equilibrium states ml , ... , ms _ 1 and separatrices 11 i ... ,1 s of the covering flow
such that:
1) w(li) = mi, i = 1 s-1
2) a(li) =i_1, = 2, ... , s;
3) li is a Bendixson extension of the separatrix 1j-1 to the right or to the left,
i=2,...,s;
4) 7r(mi) = mi and i = 1, ... , s - 1, j = 1, ... , s.
Since the trajectories 11 and 1S are coverings for the nontrivial recurrent semi-
trajectories l1 and l2, the points a(ll) def o._ and w(ls) def o(Pi) lie on the
absolute and are distinct. _
We join these points by a geodesic, which we denote by L (P) (Figure 6.24).

FIGURE 6.24

def
The geodesic L(P) does not depend on the choice of the curve
i e 7r-1 [P(11,.. . , is )] . In this notation we have the following result.
LEMMA 2.5. Let f t be a flow on Mr, p > 2, for which there is a one-sided
Poisson pencil P = P(11, ... , la). Then:
1) L(P) is a simple nontrivial B-recurrent geodesic;
2) if both trajectories l1 and l2 in P(11,. , ls) are interior, then L(P) is an
. .

interior geodesic.
The proof is analogous to that of 1) and 2) in Lemma 2.4, and we omit it.
We remark that determining the cases for which L(P) is an interior or boundary
geodesic is more difficult than in Lemma 2.4 for a nontrivial recurrent trajectory.
For example, it is possible that the trajectories l1 and l2 in the one-sided Poisson
pencil P(11,. . . , ls) are both boundary trajectories, while the geodesic L(P) is inte-
rior (Figure 6.25, a)). Or one of l1 or l2 can be interior, while L(P) is a boundary
geodesic (Figure 6.25, b)).
LEMMA 2.6. Suppose that a flow f t on Mr, p > 2, has a quasiminimal set
def
N. Then the collection Ul L(l) U L(P) (N) forms a nontrivial geodesic lam-
ination, where the union is over all nontrivial recurrent trajectories l C N and all
one-sided Poisson pencils P C N.
278 6. CLASSIFICATION OF FLOWS ON SURFACES

a)

FIGURE 6.25

PROOF. It follows from Lemmas 2.4 and 2.5 that each geodesic in 3(N) is
simple. The geodesics in 3(N) are disjoint, because otherwise either the trajectories
t
of the covering flow f would intersect, or some geodesic would correspond to a
Poisson pencil that is not one-sided.
The fact that 3(N) is closed follows from the fact that N is closed and the fact
that any nontrivial recurrent semitrajectory 10 C N is dense in N.
According to Lemmas 2.4 and 2.5, any geodesic in 3(N) is a nontrivial B-
recurrent geodesic. D

Suppose that a flow f t on Mr, p > 2, has nontrivial recurrent semitrajectories.


Geodesics L(1) or L(P) constructed from nontrivial recurrent trajectories l or one-
sided Poisson pencils P of f t are called trajectory geodesics. The geodesics in the
complete inverse images 7r-1 [L(1)] and it-1 [L(P)] in the Lobachevsky plane will also
be called trajectory geodesics.
It will be assumed that all the geodesics are orientable, that is, equipped with
a positive direction. Trajectory geodesics are oriented as follows. If l is a covering
for a nontrivial recurrent trajectory l of some flow on Mr, p > 2, then the positive
direction from the point a (l) E S to the point w(1) E S on the geodesic L(i) C
0 induces a positive direction on the geodesic L(1), and this introduction of an
orientation does not depend on the choice of covering trajectory l E 7r-1(l). The
geodesics L(P) are oriented similarly for one-sided Poisson pencils.
Suppose that the geodesics L1, L2 C Mr intersect transversally at a point
m e Mr, and denote by e(Li, m) the unit tangent vector to Li at m, i = 1, 2.
If the vectors e(L1, m) and e(L2, m) form a right-hand (left-hand) frame in the
tangent space TmMp to the orientable surface Mr at m, then the intersection index
#(L1, L2, m) of the geodesics L1 and L2 at m is equal to +1 (-1) [76]. We say
that L1 intersects L2 orientably if the intersection index is the same (+1 or -1)
at all points of L1 f1 L2. Orientability of an intersection of C°°-smooth curves is
defined analogously.
We remark that two distinct geodesics cannot be tangent (otherwise they co-
incide), and intersect transversally wherever they intersect.
LEMMA 2.7. Let E be a nontrivial recurrent trajectory or a one-sided Poisson
pencil of a flow f t on Mr, p > 2, and let C be a contact-free cycle intersecting
2. THE HOMOTOPY ROTATION CLASS 279

a nontrivial recurrent semitrajectory in E. Then the geodesic L(E) intersects any


closed geodesic Lo homotopic to C orientably.
PROOF. By Lemma 2.3 of Chapter 2, the contact-free cycle C is nonhomotopic
to zero. Therefore, the closed geodesic Lo is also nonhomotopic to zero.
According to Lemmas 2.4 and 2.5, L() is a nonclosed geodesic. Consequently,
L() Lo, and L() and Lo intersect transversally.
Assume that L(E) intersects Lo nonorientably. Then there exist geodesics L e
it -1 [L (E) ] and L1, L2 E 7r -1(Lo) such that the intersection indices of L with L 1
and L2 are opposite in sign (Figure 6.26). We introduce an orientation on C. By
assumption, there exist curves C1, C2 E it-1(C) and a trajectory E in that
intersect with opposite indices (see Figure 6.26), and this is impossible. D

FIGURE 6.26

2.7. Accessible points of the absolute. We fix a covering it : O -*


0/F of a closed orientable surface Mr of genus p > 2 by the Lobachevsky plane
0.
DEFINITION. A point O' E S is said to be accessible if there exists a semitra-
jectory l of some covering flow on 0 such that o= w (l ) or o=
The set of accessible points (with respect to the covering it: O -* Mr) is de-
noted by D(S). This set is invariant under Fr and is nonempty. In this subsection
we show that D (S) S (that is, there are points on the absolute that are not
accessible by semitrajectories of covering flows), and, moreover, the set S \ D (S))
contains a subset with Lebesgue measure 27r (that is, the Lebesgue measure on the
absolute).
The initial and terminal points of an orientable geodesic on O are defined in
the natural way.
DEFINITION. A geodesic L C 0 is said to_be transitive if for any open intervals
11, 12 C S there is a geodesic congruent to L with initial point in I1 and terminal
point in I2.
Denote by T (S) the set of terminal points of all transitive geodesics.
In 1936 Hedlund [88] proved that the set T (S) has the measure of the ab-
solute. In the next lemma we show that any point in T (S) is not accessible
by a trajectory of a covering flow. This will prove that the set S \ D (S) of
nonaccessible points contains a subset of Lebesgue measure 27r.
280 6. CLASSIFICATION OF FLOWS ON SURFACES

LEMMA 2.8. T(SB) C So0 \ (S00).


PROOF. Assume the contrary. Suppose that a point o' E T (So0) is a limit point
of a semitrajectory l0 of some covering flow on 0. The following cases are possible:
1) a is a rational point; 2) o' is an irrational point.
In case 1) the limit set of the semitra j ectory 7r (l 0) consists either of a closed
trajectory to nonhomotopic to zero, or a one-sided contour K nonhomotopic to
zero (for some flow on Mr). If the limit set of the semitrajectory r(l ) consists
of a one-sided contour K, then there exists a contact-free cycle C homotopic to K_
because M is orientable. Since to and C are simple curves, there is a geodesic L
on o with endpoint osuch that 7r(L) is a simple closed geodesic on M. It follows
from o' E T (S) that there exists a transitive geodesic Lo with endpoint o'. It can
be assumed without loss of generality that ois the terminal point of L0. Geometric
considerations give us that, beginning with some parameter t0 on L0, any geodesic
intersecting L0 at a point with parameter t > to intersects also the geodesic L.
But by transitivity, the geodesic ir(Lo) intersects the geodesic 7r(L) for arbitrarily
large values of the parameter, and hence there exists a geodesic in ir-1(7r(L)) that
intersects L. This contradicts the fact that 7r(L) is simple.
In the case 2) the semitrajectory 7r(l0) is a nontrivial recurrent semitrajectory
in view of Theorem 2.2. Lemma 3.4 in Chapter 2 gives us that 7r(l0) belongs
either to some one-sided Poisson pencil or to some nontrivial recurrent trajectory.
According to Lemmas 2.4 and 2.5, o' is an endpoint of a geodesic L projecting into
a simple geodesic on M in both situations. We arrive at a contradiction to the
condition o' E T (So0) in a way completely analogous to that in the case 1). D

REMARK. It can be shown that the inclusion D(So) C S \ T(So) is proper.


That is, there are both rational and irrational points on the absolute that do not
lie in T (So0) and are not accessible by semitrajectories of covering flows (that cover
flows with finitely many equilibrium states and separatrices).
2.8. Classification of accessible irrational points. As before, we fix a
covering 7r : O -* 0/Iof a closed orientable surface M of genus p > 2 by
the Lobachevsky plane 0.
For simplicity, a geodesic on 0 covering an interior geodesic will also be said
to be interior.
DEFINITION. An irrational point o' E D(So) is said to be a point of the first
kind if it is an endpoint of an interior trajectory geodesic. The remaining accessible
irrational points are called points of the second kind.
LEMMA 2.9. Suppose that aE D (So0) is an irrational accessible point of the
first kind, and L is an interior trajectory geodesic with endpoint o. Then any
geodesic different from L and with endpoint aprojects into a geodesic with a self-
intersection on M (and consequently is not a trajectory geodesic).
PROOF. Let L1 be a geodesic with terminal point o', L1 L.
By Lemma 2.7, there is a family of geodesics Ci E it-1(C), i e N, (C a closed
geodesic on with topological limit osuch that L intersects each Ci at some
point . Then there is an index i0 such that L1 also intersects Ci for i > io. Let
yi = L1 n Ci, i > i0. Since L is an interior geodesic, there exists a subsequence
2. THE HOMOTOPY ROTATION CLASS 281

{ij}1 of indices such that the points xik are congruent to points xik on the arc
(i0, yio) C Cio tending to xio as ik - oo (Figure 6.27). Since L and L1 have
a common endpoint, the distance between xi and yi in the non-Euclidean metric
tends to zero as i -* +oo [77]. Therefore, the distance between yik = f k (jk) and
xik also tends to zero as ik -p +oo, where yik E Fr is an element carrying xik into
xik . From Lemma 2.7 (orientability of the intersection of a trajectory geodesic and
a closed geodesic) it follows that yik e (i0 , yio) C Cio for a sufficiently large index

FIGURE 6.27
def
The geodesic Lik yZk (L) is a trajectory geodesic. Therefore, o'+ (Lik) o'
by virtue of Corollary 2.4. Consequently, Lik intersects L1 (see Figure 6.27). Since
yik e (xio , yio ), this gives us that the geodesic yik (L1) intersects L1.

For a trajectory geodesic L C 0 denote by o- (L) and o.+ (L) its initial and
terminal points, respectively.
COROLLARY 2.5. Let aE S be an accessible irrational point of the first kind,
and let L be an interior trajectory geodesic with endpoint a= o.+ (L). Then the
t
following conditions hold for any flow f on 0 covering a transitive flow f t onr
without impassable grains and having a semitrajectory l+ for which ais accessible:
1) the semitrajectory l+ = 7r(l+) belongs to a nontrivial recurrent trajectory l
of ft; _
2) if l E it -1(l) contains l+, then a(l) = a- (L).
PROOF. 1) Assume the contrary. Then l+ is an o-separatrix of some saddle.
Since f t does not have impassable grains, l+ belongs to two different one-sided
Poisson pencils P1 and P2. We take curves i e ir-1(Pi), i = 1, 2, containing l+.
It follows from Lemmas 2.5 and 2.9 and the fact that o' is a point of the first kind
that o- (P1) = o (P2) def o,_ . Then the curves P1 and P2 and the point o- bound
a simply connected domain U on 0. The transitivity of f t gives us that there is a
trajectory l1 in U covering a nontrivial recurrent trajectory of ft. Then o- = a(ll),
which contradicts Theorem 2.3.
The assertion 2) follows from Lemma 2.9.

We proceed to the consideration of accessible points of the second kind. It


follows from Lemmas 2.3, 2.4, and 2.5 and Theorem 2.2 that if o' E S is an
282 6. CLASSIFICATION OF FLOWS ON SURFACES

accessible irrational point of the second kind, then there is at least one pair of
associated boundary geodesics with common endpoint o.
LEMMA 2.10. Let aE S be an irrational accessible point of the second kind,
and let L1 and_L2 be associated boundary trajectory geodesics with endpoint a=
o.+ (L1) = o.+ (L2) . Then any trajectory geodesic having endpoint aand projecting
into a B-recurrent geodesic on Mr coincides with either L1 or L2.
aef
PROOF. Denote by R(L1i L2) R the domain in 0 bounded by L1 and L2
and the corresponding arc of the absolute. Denote by R+ (respectively, R-) the
component of 0 \ R adjacent to L1 (respectively, L2) (Figure 6.28).

FIGURE 6.28

Under the action of Fr the domain R is transformed into domains arbitrarily


close to R and lying in both domains R+ and R. Therefore, it can be shown as in
the proof of Lemma 2.9 that any geodesic with endpoint olying in R+ UR- projects
into a geodesic with a self-intersection on Mr, and hence cannot be a trajectory
geodesic. _
Assume that there is a trajectory geodesic L3 with endpoint o that is distinct
from L1 and from L2 and projects into a B-recurrent geodesic on M. Then L3 C R.
According to Lemma 2.9, L3 is not interior. Therefore, it is a boundary geodesic,
and there exists a boundary trajectory geodesic L4 that is associated with L3 and
has endpoint o.
Denote by R(L3, L4) the domain bounded by L3 and L4 and the corresponding
arc of the absolute. Since L3 C R, either L1 or L2 lies outside R(L3i L4). By the
foregoing, either L1 or L2 projects into a geodesic with a self-intersection on YCr,
and this is impossible. D
COROLLARY 2.6. Suppose that aE S is an accessible irrational point of the
second kind, and let L1 and_L2 be associated boundary trajectory geodesics with
endpoint a= o.+ (L1) = o.+ (L2) . Then the following conditions hold for any flow
f t covering a transitive flow f t on Mr, p > 2, and having a semitrajectory l+ for
which ais accessible:
1) l+ = 7r(l+) is an a-separatrix and belongs to two one-sided Poisson pencils
E1 and E2 of f t;
2) if Ei E 7r-1(E) contains l+, i = 1, 2, then either O( j) = a(t), i = 1, 2,
or 0-(E2) = a-(L1) and a-(E1) = a-(L2).
2. THE HOMOTOPY ROTATION CLASS 283

PROOF. 1) If l+ belongs to some trajectory that is nontrivial recurrent in


both directions, then, since f t is transitive, l is dense in Mr and is interior. This
contradicts the condition that obe a point of the second kind. Consequently, l+ is
an a-separatrix.
The semitrajectory l+ belongs to exactly one one-sided Poisson pencil E only
in the case when all the equilibrium states on E are impassable grains. Then it
follows from the transitivity of f t that l+ is an interior semitrajectory. This again
contradicts the condition that obe a point of the second kind. Thus, l+ belongs
to two one-sided Poisson pencils E 1 and 2
The assertion 2) follows from Lemma 2.10. D
COROLLARY 2.7. Let aE S be an accessible irrational point of the second
kind, and let L1 and L2 be associated boundary trajectory geodesics with terminal
t
point a= o.+ (L1) = o.+ (L2). Suppose that a flow f covering a flow f t on YCr,
p > 2, has trajectories 11 and l2 such that o.+ (11) = o.+ (l2), and lr(ll) and ir(12)
are associated boundary trajectories in a minimal set of ft. Then either o (Ii) =
a- (L1) and a- (l2) = a- (L2 ), or o( l) = a- (L2) and a- (l2) = a- (L1) (Figure
6.29).

FIGURE 6.29

2.9. The orbit of a homotopy rotation class. Let cP : Mr -* Mr be a


homeomorphism of a closed orientable surface Mr 0/Fr, p > 2, and let P : 0 -p
0 be a homeomorphism covering it. For any point m e 0 and any element y E Fr
the points (i) and ('y(i)) are congruent, and therefore there is an element
y' E Fr such that y' o (i) = P o 'y(m) (Figure 6.30). Since the group Fr is
discontinuous, the last equation is valid for all points m e 0.

X odlm1
m

FIGURE 6.30
284 6. CLASSIFICATION OF FLOWS ON SURFACES

The correspondence y ' y' = P o y o P-1 is an automorphism of the group


F (we leave the proof of this to the reader as an exercise) and is denoted by P* .
Thus, P* ('y) ° P = P ° y
The next theorem is due to Nielsen ([101], [102] ).
THEOREM. 1) An arbitrary automorphism -r: I'r - F has the form P*; that
is, there exists a homeomorphism P : 0 -* 0 covering some homeomorphism of the
surface Mr 0/I', p > 2, such that 'r =
2) Any homeomorphism P : 0 -* 0 covering some homeomorphism of Mr
0/I'r, p > 2, can be extended to a homeomorphism p* : 0 U S -* 0 U S (that
is, can be extended to the absolute).
3) Let P1 and c°2 be covering homeomorphisms for some homeomorphism of
YCr = 0/I'r, p > 2, and let and PZ be extensions of them to the absolute. If
1* _ 2*' then 1 Is
It follows from the theorem that an arbitrary automorphism r : I'r - F in-
duces a homeomorphism of the absolute, which we denote by r.
Let H* be the set of homeomorphisms of the absolute induced by all possible
automorphisms of the group F.
DEFINITION. Suppose that a semitrajectory l of a flow f t on Mr, p > 2, has
a homotopy rotation class µ(l O) C S. The orbit 0(10) of the homotopy rotation
class of 10 is defined to be the set
= Ur*[1u(lO)],
where the union is over all r* E H*.
The next theorem shows that the orbit of the homotopy rotation class (HRC)
is a topological invariant.
THEOREM 2.4. Suppose that a flow f i on Mr, p > 2, is topologically equivalent
to a flow f2 by means of a homeomorphism cP : Mr -* Mr, and let P be a covering
homeomorphism for Sp that extends to a homeomorphism r : 0 U S -* 0 U S.
In this case if a semitrajectory l of f f has homotopy rotation class (l0), then the
semitrajectory cp(l 0) of f 2 also has a homotopy rotation class, equal to ?*[(l0)],
and L(lO) = 0[So(10)]
PROOF. Since S extends to a homeomorphism of the absolute, the semitrajec-
tory rp(l 0 ), where l 0 E 7r-1(lO ), has a limit point on the absolute. Consequently,
the semitrajectory co (l ) = it [cp (l ) ] has an HRC, equal to cc [(l )]. This and
the inclusion cp* s E H* give us that 0(10) = O [cp(l 0 )] . D

COROLLARY 2.8. Suppose that the flows f i and f 2 on Mr, p > 2, are topologi-
cally equivalent by means of a homeomorphism homotopic to the identity. Then for
any semitrajectory l1 of f i having HRC µ(l)) there is a semitrajectory 4) of f2
with (l) =µ(l20.
PROOF. We use the notation of Theorem 2.4. Since Sp is homotopic to the
identity, the automorphism P* is interior, and * s = for some element
P E Gr [68]. This and Theorem 2.4 gives us that ,u[cp(l i )] = 'y[µ(l i )] = µ(l i ) D
3. TOPOLOGICAL EQUIVALENCE OF TRANSITIVE FLOWS 285

§3. Topological equivalence of transitive flows


The problem of topological equivalence of transitive flows without impassable
grains and without separatrices joining equilibrium states can be solved with the
help of the concept of homotopy rotation class and the concept of the orbit of a
homotopy rotation class, introduced for a closed orientable surface of genus p > 2
in the preceding section.
3.1. Homotopic contact-free cycles. Everywhere in this section Mr
0/I'r is a closed orientable surface of genus p > 2.
LEMMA 3.1. Suppose that flows f f and f 2 on Mr, p > 2, have nontrivial
recurrent semitrajectories l j and l2 with the same homotopy rotation class. Then
f1 and f2 have mutually homotopic contact-free cycles C1 and C2 such that CZflli
0,i=1, 2.
PROOF. According to Lemma 2.3 in Chapter 2, f 2 has a contact-free cycle C
that is nonhomotopic to zero and such that C fl l2 0. It follows from Corollary
2.2 and the equality µ(l j) =µ(l2) that there exist semitrajectories l i E 7r-1(lj)
and 12 E 7r-1(12) with w(l1 ) = w(l2) = o, and a family {C}1 of contact-free
t
arcs Ci E 7r -1(C) of a flow f that have oas their topological limit. There is an
index i0 such that l i and l2 intersect all the arcs Ci with i 20 . For simplicity we
assume that i0 = 1.
Denote by m1 E l1 l C1 a point after which l i does not intersect C1 with
increasing time (Figure 6.31). By a small perturbation of C in a neighborhood
of the point m1 = 7r(ml) we ensure that the trajectories of f i intersect transver-
sally, while keeping the curve C a contact-free cycle of f 2 . Since l1 is a nontrivial
recurrent semitrajectory, it intersects infinitely many times. Denote by Si the
first point where l1 (ml) intersects . Then the arc mi s1 C li and the segment
1j j C form a simple closed curve Ci from which a closed transversal C1 of f i
intersecting li can be obtained by the standard method (Lemma 1.2 in Chapter
2). By Lemma 2.3 in Chapter 2, C1 is homotopic to zero.

FIGURE 6.31

Denote by s 1 a point in l 1 fl Ck such that 7r (s 1) = s 1 (Figure 6.31), and let


e be a segment containing sl . The segment contains a point congruent
to i. Then y(m1) E for some element y e I'r, and therefore Ck = y(C1).
286 6. CLASSIFICATION OF FLOWS ON SURFACES

Let 82 = l2 fl Cit. We use an arc d2 transversal to the flow f 2 to approximate


the union of the arc m2 s2 C l2 and the segment s2 y(m2) C C, where 7(m2) E Ck.
Let C2 = (d2) . Then C2 = r(C2) is a closed curve transversal to the flow
f 2 and homotopic to C1.
The curve C2 is not simple in general, but since it is homotopic to the simple
curve C1, we can use an isotopy to carry it into a simple curve C2 transversal to
f 2 . Indeed, by a small perturbation we first ensure that the self-intersections of C2
are transversal, and we consider two curves C, CJ E 7r -1(C2) .
Let a, b E C2i f1 C2 be two adjacent points of intersection of the curves C22
and C'2 such that the arcs of them with endpoints a and b bound a disk on 0 that
does not contain points in it-1(C2) (Figure 6.32). Such points exist because C2
is homotopic to the simple curve C1, and hence the endpoints of C' and C23 are
not separated on the absolute. In Figure 6.32 it is shown how C' and C23 can be
transformed into curves C2i and C2j that have two fewer points of intersection and
t
are transversal to f.

FIGURE 6.32

We now consider the intersections of C2i with the remaining curves in ir-1(C22)
By successively carrying out the process described above, we obtain a curve C2
transversal to f t and such that 7r(C2) = C2 is simple. Eli

3.2. Auxiliary results.


LEMMA 3.2. Suppose that flows f 1 and f2 on Mr, p > 2, have nontrivial
recurrent semitrajectories l1 and l2 with the same homotopy rotation class. Then
for any nontrivial recurrent positive semitrajectory L1 of f f lying in w(l1) there is
a nontrivial recurrent positive semitrajectory L2 of f2 in w(l2) such that /c(L1) =
(L2)
PROOF. According to Lemma 3.1, there exist mutually homotopic contact-free
cycles C1 and C2 of f f and f 2 that are nonhomotopic to zero and intersect l t and
l2 , respectively. It follows from Lt C w(l1) and Theorem 2.3 in Chapter 2 that
l j C w (Li) Therefore, L i intersects C1.
.
3. TOPOLOGICAL EQUIVALENCE OF TRANSITIVE FLOWS 287

We consider a curve C 1 E ?f -1(C1) and a semit ra j ectory L 1 E it -1(L 1) inter-


secting C1. Since C1 and C2 are homotopic, there is a curve C2 E it -1(C2) having
common endpoints with C1.
We show that there exist sequences of semitrajectories In 1 E it -1(l1) and
def
1n,2 E ?f -1(l2) with the following properties: 1) w (l n,1) = w (l n,2) n E S ;2)
1n.7 intersects C3 for j = 1, 2; 3) the points xn = 1n,1 f1 C1 converge to the point
m1 = L1 n C1 as n -*00 (Figure 6.33).

FIGURE 6.33

Suppose that the semitrajectories l1 E it-1(l1) and l2 E 7r -1(l2) have a com-


mon w-limit point E S. Since C1 and C2 are homotopic, Corollary 2.2 implies
the existence of families {C1}?° E it -1(C1) and {C2}° E it -1(C2) such that C21
and Ci2 have common endpoints on the absolute, and is the topological limit of
these families. There is an index i0 such that l intersects CZ for i > io, j = 1,
2. Since L1 C w (l1 ), there is a sequence of points zn E l1 f1 CZn 1 (in > io) such
that ir(zn) -* ir(m1) as n -* oo. Therefore, there exist elements 'Yn E Fp such that
,yn (zn) def xn E C1 and xn -* m1 as n -* oo. Then the sequences of semitrajectories
ln1 = 'y( l) i) and 1n,2 = 'y(i) satisfy the conditions 1)-3), where on =
Let yn = 1n,2 f1 C2. We show that the sequence {yn} is bounded on C2. Since
xn -* m1, the sequence {xn } is bounded on C1. Take an element 'y leaving invariant
the curve C1 (and hence also C2). There exists a k E Z such that the sequence
{xn} lies on the arc of C1 bounded by the points -y/c(x1) and #y-1C (x1) Then by .

the property 2), the sequence {yn} lies on the arc of C2 bounded by 71C (y1) and
7_k (i1) (Figure 6.34).
It follows from the continuous dependence of trajectories on the initial condi-
tions that there is an index ni for which the semitrajectories In 1 all intersect the
contact-free arc CZ for n > n2 (Figure 6.33). Since o is the topological limit of the
curves CZ , i E N, we have o -* o as n -* oo .
The bounded sequence {yn } of points has a subsequence {ynk } 1 converging
to some point m2 E C2. It can be assumed without loss of generality that all the
points ynk lie in one of the intervals in C2 \ {w2}.
288 6. CLASSIFICATION OF FLOWS ON SURFACES

FIGURE 6.34

We consider a semitrajectory L2 (m2) of the flow f2. Since ink -* m2, the
semitrajectory 7r[L2 (m2)] lies in the quasiminimal set c1(l2 ). Therefore, by The-
orem 3.4 in Chapter 2, it [L2 (2)] is either a nontrivial recurrent semitrajectory
or an w-separatrix. In the first case w [L2 (m2)] consists of a single point o2 E S
by Lemma 2.3. From ink -p m2 it follows that w (l nk 2) = ink -p 02 as k -* oo
, .

Therefore, 2 = o, and L2 = r [L2 (m2 )] is the desired semitrajectory.


If r [t(m2)} is an w-separatrix, then L2 (m2) = L2 is also an w-separatrix.
This case can be reduced to the preceding one with the help of a Bendixson exten-
sion of the separatrix L2 to the same side (to the left or to the right) on which
the points ink are located with respect to m2. By the assumption that there are
finitely many equilibrium states, a finite number of Bendixson extensions lead to
a semitrajectory L2 projecting into a nontrivial recurrent semitrajectory on M.
Then (L) = o, and ir(L2) = L2 is the desired semitrajectory. U
LEMMA 3.3. Suppose that the transitive flows f f and f2 on Mp, p > 2, do
not have impassable grains nor separatrices joining equilibrium states, and let the
semitrajectories li and l2 of covering flows f 1 and f2 have a common w-limit
point o E S. In this case: 1) if l1 is not an a-separatrix, then neither is l, and
a(ll) = a(l2) = o E Sam, where l2 is the trajectory containing l, i = 1, 2; 2) if
l1 is an a-separatrix of a saddle 01, then l2 is also an a-separatrix of a saddle
02. Moreover, for each w- (a-) separatrix of 01 there is an w- (a-) separatrix of
02 having with it a common a- (w-) limit point on the absolute, and conversely.
PROOF. 1) If li is not an a-separatrix, then the semitrajectory ir(l1) belongs
to a nontrivial recurrent trajectory of f. This and the transitivity of the flow
f 1 give us that o is an accessible irrational point of the first kind. The required
assertion now follows from Corollary 2.5.
The assertion 2) is obtained by successively using Corollary 2.6. 0

3.3. Construction of a fundamental domain. Everywhere in this sub-


section f t is a transitive flow without impassable grains and without separatrices
joining equilibrium states, given on a closed orientable surface Mp 0/F of genus
p > 2. Denote by f t a flow on the Lobachevsky plane covering ft.
3. TOPOLOGICAL EQUIVALENCE OF TRANSITIVE FLOWS 289

For a fixed closed transversal of f t and one of its liftings to 0 we construct


a fundamental domain M of Fp whose boundary 9 consists of contact-free arcs,
saddles of the flow f, and separatrices of these saddles.
Let C be a contact-free cycle of f t, and let C1 E it-1(C) be one of its inverse
images. Since f t is transitive and does not have separatrices joining equilibrium
states, each w-separatrix of any saddle intersects C for unboundedly small times.
Therefore, there exists on C1 a family of successively located points ml,... , m+1
def
satisfying the following conditions: 1) l+(mi) li is an w-separatrix of some saddle
OZ and does not intersect any curves in it -1(C) after the point = 1, ... , k -I-1;
2) the points m1 and mk+1 are congruent, and there are no other pairs of congruent
points on the arc m1mk+1 C C1 (between m1 and mk+1); 3) there are no points
other than m1i... , m+ 1 on the arc mim+ 1 C C1 that satisfy the condition 1).
Let lit and fir be separatrices of OZ that are Bendixson extensions of the sepa-
ratrix l2 to the left and to the right, i = 1, ... , k -I- 1. It follows from the condition
3) and from Lemma 3.6 in Chapter 2 that with increasing time 12r and 12+ 1,1 first
intersect one and the same curve (denote it by Ci+1) in the complete inverse image
it-1(C), i = 1, ... , k (Figure 6.35). Let mir = lir f1 C2+1 and m2+1,1 = 1i+1,1 f1 Cz+1,
and let Bi be the domain bounded by the w-separatrices l2 (m) and l (i+1),
the a-separatrices 12,, (mir) and 12+1,1(mi+1and the arcs mimi+l C C1 and
di+1 = mirmi+l,l C C2+1 (Figure 6.35).

FIGURE 6.35

LEMMA 3.4. The set M = U=1 cl(Bi) is a fundamental polygon of the group
Fp (Figure 6.36).

PROOF. We first show that there are no pairs of congruent points in int M.
Assume the contrary; that is, let x 1, x2 E int M be congruent points. Then the
semitrajectories l (x1) and l (x2) are also congruent. By the construction of M,
l (x1) and F(2) intersect the arc m1mk+1 C C1 at some points y1 and y2, and the

arcs xiyi C l (xi) do not intersect the curves in it 4 (C) at interior points, i = 1,
2. Consequently, y1 and y2 are congruent, which contradicts the condition 2).
It remains to show that for any point m E Mp the complete inverse image
7r 4 (m) intersects M. Let m be a regular point. The following cases are possible:
290 G. CLASSIFICATION OF FLOWS ON SURFACES

FIGURE 6.36

a) 1(m) is not an a-separatrix; b) 1(m) is an a-separatrix. In case a) we denote


by z E C the first point where l- (m) intersects C with decreasing time. Since
ir[m1i mk+1] = C, the arc m1mk+1 C C contains a point z such that ir(z) = z.
The open arc zm C 1(m) is disjoint from C, so there is an open arc z m C i(m)
that is disjoint from it -1(C) and such that ir(z m) = zm. From this, m E PVC and
m E it-1(m).
In case b) if l - (m) intersects C, then the argument is completely analogous to
case a). Therefore, we assume that l- (m) is disjoint from C, and we denote by lr an
w-separatrix for which 1(m) is a Bendixson extension to the right. Since f t does not
have separatrices joining equilibrium states, lr intersects C with decreasing time.
Denote by z E C the first such intersection point. Then by the conditions 1) and
2), one of the points m1i ... , m+1, say mi, covers z. Consequently, lr = ir(li) and
71(l ir) = 1(m). Since l - (m) does not intersect C, it follows that 7r -1(m) n lir E
aM c M.
If m is a saddle O, then in view of the conditions 1)-3) any w-separatrix of O
is covered by one of the separatrices l 1, ... , l i+1. Therefore, it -1(m) fl PVC 0. LI

REMARK. In the treatment of case b) it is possible to take an w-separatrix ll


for which 1(m) is a Bendixson extension to the left instead of the w-separatrix lr .
Then 1(m) = it (l jl) for some j = 2, ... , k -I- 1. This implies that any a-separatrix
lir (i is one of the numbers 1, ... , k) is congruent to some (unique) a-separatrix l jl
(j is one of the numbers 2, ... , k -I- 1). Therefore, the arc di+i C aM is congruent
to some arc d j C M.
3.4. Necessary and sufficient conditions for topological equivalence
of transitive flows.
THEOREM 3.1. Let f 1 and f 2 be transitive flows without impassable grains
and without separatrices joining equilibrium states on a closed orientable surface
VCr O/I'p, p > 2. Then f f and f2 are topologically equivalent by means of a
homeomorphism homotopic to the identity if and only if there exist semitrajectories
4) and 4) of f 1 and f 2 with the same homotopy rotation class.
3. TOPOLOGICAL EQUIVALENCE OF TRANSITIVE FLOWS 291

PROOF. NECESSITY. If the flows are topologically equivalent by means of a


homeomorphism homotopic to the identity, then the existence of l1 and 4) with
p (i 1 ) = p(4)) follows from Lemma 2.3 and Corollary 2.8.

SUFFICIENCY. Let p(i) = p(4)), where l is a semitrajectory of the flow f2 ,


i = 1, 2. Since replacing t by -t gives a flow topologically equivalent to the original
one, it can be assumed that 4) and 4) are positive semitrajectories.
According to Lemma 3.1, f 1 and f2 have mutually homotopic simple contact-
free cycles C1 and C2 that are nonhomotopic to zero. Let C 1 E r -1(C1) and
C(2) E ?f -1(C2) be curves with common endpoints c- and c+ on the absolute. We
construct a homeomorphism 9(C(1), C(2) ) : C(1 -* C(2) . Take a point m E C(1 .

Two cases are possible: a) l + (m) is not an w-separatrix; b) l+ (m) is an w-separatrix.


In case a) the semitrajectory 7r (l+ (m)) is a nontrivial recurrent semitrajectory.
+ (m)) def
Therefore, w (T E S. It follows from the transitivity of f f and f 2 and
Lemma 3.2 that f 2 has a semitrajectory l'+ with w (l') = o.

We show that the trajectory l' containing l'+ intersects the arc If the
trajectory l (W) is not an a-separatrix, then in view of Lemma 3.3 neither is l',
def
and a (l (m)) = a (l') o - . Since l(m) intersects C 1 at exactly one point, the
pairs (o, o) and (c-, c+) of points are separated on the absolute. Therefore,
l' intersects If l (n) is an a-separatrix of some saddle O, then l' is also an
a-separatrix of some saddle O' in view of Lemma 3.3. Denote by D (respectively,
D') the domain in the Lobachevsky plane bounded by the arc (respectively,
and the arc S- = c-c+ of the absolute not containing the point (Figure
6.37). Since i(m) intersects C 1 , it follows that O E D. This and the fact that
f i does not have separatrices joining equilibrium states imply that the w-limit set
of an arbitrary a-separatrix of O different from l (n) lies on S. By Lemma 3.3,
the w-limit set of an arbitrary a-separatrix of O' different from l' also lies on S.
Therefore, O' E D', and hence l' intersects (Figure 6.37).

C°_ f

FIGURE 6.37
292 G. CLASSIFICATION OF FLOWS ON SURFACES

Thus, in case a) the trajectory l' intersects the curve C(2) at some (unique)
point fl', and we set 9(C(1), C(2)) (m) = rra'.
In case b) i(n) is not an a-separatrix in view of the absence of separatrices
joining equilibrium states. Therefore, a[l(m)] def - E S. According to Lemma
3.3, there exists a trajectory l' of f 2 such that a(l') = o. Then l' intersects C(2)
at some point m', and we set 8 (C(1) , C(2)) (m) = m' .
It follows from the continuous dependence of trajectories on the initial condi-
tions that 9(C(1), C(2)) : C(1) - C(2) is a homeomorphism.
The construction of a homeomorphism 9(C(1), C(2)) : C(1) -* C(2) for any
curves C 1) E it -1(C1) and C 2) E it -1(C2) with common endpoints on the ab-
solute is completely analogous.
If the trajectories l and l' of the respective flows f f and f2 have a common w-
(a-) limit point on the absolute, then for any element -y E IFp the trajectories y(l)
and -y(l') also have a common w- (a-) limit point on the absolute. Therefore,

(3.1) 7 o e(C(1), C(2)) = e[7(C(1) 1)7(C2)}


)] 07.
According to §3.3 we construct the fundamental polygon M1 bounded by arcs of
the separatrices l 1, 11r, 121, 12r, ... , l k+1,1, l k+ 1 off 1 and by the contact-free segments
mlmk+1 C C1 1)
1 2 ,. .
, d(1) C C(1) . , d(1)
+1 C C(1) 1ying
on the curves C(1) E 7r -1 (C1).
Since C1 and C2 are homotopic, there is an arc E it -1(C2) with the same
endpoints on the absolute for each i = 1, ... , k + 1. Therefore, the homeo-
1)
morphisms 9 (C , C(2)) are defined, i = 1, ... , k -I- 1. From Lemma 3.3 and the
definition of the homeomorphisms 9 ( , ) it follows that an w-separatrix l j of the flow
f2 passes through the point m' = 9(C1, C2) (m), ), j = 1,... , k -I- 1. By the con-
gruence of the points m1 and mk+1 and the relation (3.1), the points m1 and m+1
are congruent, and there are no other pairs of congruent points on the segment
m1m+1 C C12) . Therefore, according to §3.3, there exists a fundamental polygon
NYC' bounded by the separatrices l1 = l (m), llr, l21 l2r l +l,l z+1 = l' (m.1)
of f 2 and by the contact-free segments m1 m 1 C C12) , d22) , ... , d 2i lying on
the curves in it-1(C2) It follows from the equalities m' = 9 (C11) , Ci2)) (.), j =
.

1, ... , k -I- 1, and Lemma 3.3 that w(ljr) = w(ljr), j = 1,... , k, and w(lZ1) = w(lZ1),
C(2)) 2) , j _ 2,
i = 2, ... , k -I- 1. From this we get that , k -I- 1
By (3.1) and the definition of the homeomorphisms O(., ), there exists a home-
omorphism : aM -* DM' with the following properties: 1) ' coincides with
9(C(1),
1
C(2)) on mlmk+1 C C(1) and with
1 1 2 2 C , 2 2

i = 2, ... , k + 1; 2) :i;(OZ) = OZ, i = 1,...,k+ 1; 3) (ljr fl) = l jr n aM',


j= 1, ... , k; 4) (l jl n aM) = l jl n aM', j =2,...,k+1; 5) if the points x, y E DM
are congruent, then fi(x) and i(y) are also congruent.
It follows from 1)-4) that b extends to a homeomorphism M -* M' (we again
denote it by ) which carries arcs of trajectories of the flow f t into arcs of trajecto-
ries of the flow J. According to the property 5), projects into a homeomorphism
3. TOPOLOGICAL EQUIVALENCE OF TRANSITIVE FLOWS 293

b : Mp -* Mr carrying trajectories of f 1 into trajectories of f 2 ; that is, f 1 and f 2


are topologically_ equivalent by means of the homeomorphism b.
Denote by b a covering homeomorphism for b such that b J = By the
property 1), b extends to a homeomorphism of the absolute that is the identity
l1
on a dense set of points accessible by trajectories of the flows and f 2. That is,
* Is = id. Consequently, b is homotopic to the identity. Eli

THEOREM 3.2. Let f 1 and f2 be transitive flows without impassable grains and
without separatrices joining equilibrium states on a closed orientable surface Mr,
p > 2. Then f f and f 2 are topologically equivalent if and only if they have respective
semitrajectories l1 and 4) with the same rotation orbit.
PROOF. NECESSITY. It follows from Theorem 2.4.

SUFFICIENCY. It follows from Theorem 3.1 and the fact that any automorphism
of Fp is induced by a homeomorphism of the Lobachevsky plane that covers some
homeomorphism of Mr (see Nielsen's theorem in §2.9). LI

COROLLARY 3.1. Let f f and f 2 be transitive flows without impassable grains


and without separatrices joining equilibrium states on a closed orientable surface
Mp, p > 2. Then f f and f 2 are topologically equivalent if and only if they have
respective separatrices l1 and l2 with the same rotation orbit.
Remark. Levitt's counterexample to a conjecture of Katok. For a
transitive flow f t on an orientable surface M, Katok defined in [46] the cone
K(f t) C H1(M, F, R) generated by the nontrivial invariant measures of f t (where
F is the set of equilibrium states of ft), and announced the following theorem.
THEOREM. Let fi and f2 be transitive flows on a closed orientable surface Mp
of genus p > 1, and suppose that the sets of equilibrium states coincide for these
flows, with each equilibrium state a nondegenerate saddle (consequently, there are
2p - 2 equilibrium states for each flow). If f 1 and f2 are sufficiently close and their
cones K (f 1) and K(f) intersect, then the flows are topologically equivalent.
Katok conjectured that this theorem is true not only for sufficiently close flows.
In 1983 Levitt refuted Katok's conjecture, proving the following theorem.
THEOREM [94]. Let F be a set of 2p - 2 distinguished points on a closed ori-
entable surface Mp of genus p > 2. Then for almost every (in the sense of Lebesgue
measure) element E W (M7,, F, R) there exists a family of transitive flows f2 ,
i E N, on Mp such that:
1) each equilibrium state of f, i E N, lies at one of the distinguished points
and is a nondegenerate saddle;
2) the cone K(f2) of the flow f, i E N, is equal to the ray {t. t > 0};
3) the flows f2 and f are not topologically equivalent for i j.
An element E H1(Mp, F, R) can be represented as a class of closed differ-
entiable cohomologous 1-forms differing by a differential of a function, with the
differential zero on F. Denote by (w) the foliation determined by a closed 1-form
w E [J. Then the preceding theorem is equivalent to the following theorem, proved
in [94].
294 G. CLASSIFICATION OF FLOWS ON SURFACES

THEOREM. Let F be a set of 2p - 2 distinguished points on a closed orientable


surface Mr of genus p > 2. Then for almost every (in the sense of Lebesgue
measure in the space H1 (Mr, F, R)) element E H1(Mr, F, R) there exists an
infinite family of closed differentiable 1 -forms wi, i E N, such that:
1) each singularity of the foliation (w), i E N, is nondegenerate and lies in
F;
2) (w) is transitive, i e N;
3) (w) is strictly ergodic;
4) the foliations (w) and (w) are not topologically equivalent for i j.

§4. Classification of nontrivial minimal sets


The nontrivial minimal sets not containing special pairs of trajectories can be
classified on closed orientable surfaces of genus p > 2 with the help of the homotopy
rotation class and the orbit of the homotopy rotation class. In solving the realization
problem we construct flows for which the trajectories in a nontrivial minimal set
are geodesic curves.
4.1. Special and basic trajectories. We recall that a minimal set of a
flow is defined to be a nonempty closed invariant set not containing proper closed
invariant subsets.
DEFINITION. A nontrivial minimal set is defined to be a nowhere dense set
that is not a closed trajectory nor an equilibrium state.
According to the catalogue of minimal sets (§3.9 in Chapter 2) a nontrivial
minimal set consists of nontrivial recurrent trajectories, with each of its trajectories
dense in the minimal set.
The nontrivial minimal sets on the torus are imbedded in Denjoy flows. That
is, if a flow f t on the torus has a nontrivial minimal set f, then there exists a
Denjoy flow f t with nontrivial minimal set f. The Denjoy flows were classified in
§1.7.
In this section we consider nontrivial minimal sets of flows on closed orientable
surfaces Mp of genus p > 2.
DEFINITION. Let f i and f2 be flows (perhaps the same) with minimal sets f
and f 2 on a surface M. The minimal sets f 1 and f 2 are said to be topologically
equivalent if there exists a homeomorphism of M carrying the trajectories in f
into trajectories in f 2 and mapping f onto SZ2.
Any nontrivial minimal set of a flow on a closed orientable surface of genus > 2
is locally homeomorphic to the direct product of a closed bounded interval and the
Cantor set. Therefore, a nontrivial minimal set in Mp, p > 2, contains boundary
and interior nontrivial recurrent trajectories (see §1.5 for the definition of boundary
and interior trajectories).
DEFINITION. A pair of boundary trajectories l1, l2 in a nontrivial minimal set
f is called a special pair, and the trajectories themselves are said to be special, if
there exists a simply connected component of Mp \ f whose accessible (from within)
boundary consists of l1 and l2.
A boundary trajectory in a nontrivial minimal set is said to be basic if it is not
special.
4. CLASSIFICATION OF NONTRIVIAL MINIMAL SETS 295

LEMMA 4.1. Suppose that a flow f t on a closed orientable surface Mr, p > 2,
has a nontrivial minimal set f. Two trajectories l1 and l2 in f form a special
pair if and only if there exist covering trajectories l2 E ir-1(l2), i = 1, 2, such that
w(l1) = W(12) and a(ll) = a(l2).
PROOF. NECESSITY. It follows from the properties of a covering.

SUFFICIENCY. Suppose that there are trajectories l2 E it -1(l2) i = 1, 2, with


w(l1) = w(l2) and a(ll) = a(l2) Then l1 and l2 bound a domain w on 0 (Figure
.

6.38). By Corollary 2.4, w does not contain inverse images of nontrivial recur-
rent semitrajectories of ft. Therefore, l1 and l2 are boundary trajectories. Since
the points w(l) and a(i) are irrational for i = 1, 2, the domain it (w) is simply
connected. 0

FIGURE 6.38

LEMMA 4.2. Any nontrivial minimal set of a flow on a closed orientable surface
Mr, p > 2, contains finitely many basic trajectories.
PROOF. Let f be a nontrivial minimal set. According to Lemma 2.3, there
exists a contact-free cycle C that intersects f. Following §4.3 in Chapter 2, we
introduce a partitions of C into closed disjoint subsets (elements). The elements
of t are closed intervals in C \ f and points not belonging to these closed intervals.
It follows from the definition of basic trajectories that any basic trajectory
passes through an endpoint of at least one closed interval forming an element of
of type two. Then Lemma 4.2 in Chapter 2 gives us that there are finitely many
basic trajectories. LI

REMARK. It is shown in [17] that the number of basic trajectories of a non-


trivial minimal set of a flow on a closed orientable surface of genus p > 2 does not
exceed 8(p - 1) and is at least 2.
4.2. The canonical set. In this subsection we construct a closed set con-
taining a nontrivial minimal set f which is said to be canonical. The results here
overlap in part those in §4 of Chapter 2.
LEMMA 4.3. Suppose that a flow f t on Mr, p > 2, has a nontrivial minimal
set f, and let C be a contact free cycle of f t intersecting f Then there exists a
closed set D(f, C) satisfying the following conditions:
296 6. CLASSIFICATION OF FLOWS ON SURFACES

1) 1 C D(1, C);
2) each component of the set D(1, C) \ SZ is simply connected;
3) the boundary of D(1, C) consists of finitely many simple closed curves, each
a union of an even number of arcs of basic trajectories in SZ and the same number
of contact free segments lying on C (Figure 6.39).

FIGURE 6.39

PROOF. We take a curve C E it-1(C) and congruent points a, b E C n S2,


where S2 = r-1(1k), such that there are no other pairs of congruent points on the
arc ab C C. Denote by C1 (respectively, C2) the first curve in it-1(C) that is
intersected by the semitrajectory l+(a) (respectively, l+(b)) after the point a (b).
We show that C1 C2. Assume not. Then the points a1 = C1 n l+ (a) and
b1 = C1 n l+ (b) are congruent, and there are no other pairs of congruent points
on the arc a1 b1 C C1. The trajectory arcs a a1 C l+(a) and b b1 C l+(b) are also
congruent, and therefore the curvilinear quadrangle bounded by the arcs ab, a1 b1,
a a1, b b1 projects into a torus a surface of genus 1. This contradicts the fact that
M is a surface of genus p > 2.
The set SZ n C is a Cantor set. Let (a, /3) bean adjacent interval of 1 n C such
that the trajectory ir[l (,Q)] is not basic, and let /1 E C be a point congruent to ,Q
aef ,Q,Q1
such that there are no other pairs of congruent points on the segment B C C.
Let S2o = B n SZ, and denote by C(m) the first curve in 7r-1 (C) intersected
by the semitrajectory l+ (m), m E S2o, with increasing time after the point m. By
the compactness of SZo and the continuous dependence of trajectories on the initial
conditions, there are only finitely many open intervals U1,... ,Uj that cover SZo
and are such that for any mi, mi' E 1 o n u2 the curves C(m2) and C(mi') coincide
(i = 1. , k), while for any m2, E UZ n 1 o and m E U3 n 1 o with i j the curves
C (m2) and !(m) are different. It follows from the first paragraph of the proof
that k > 2.
Denote by CZ the curve C(m), m E U2 n 1 o, i = 1, ... , k.
By the Cantor property of the set SZo, there exists for each i E {1,. .. , k} a
closed interval [j, b2] C CZ with endpoints a2, b2 E U2 n 1 o that contains all the
points in UZ n o. Since CZ C3 for i j, the trajectories it [l (a2) ] and it [l (b2) ] are
basic for i= 1, ... , k.
4. CLASSIFICATION OF NONTRIVIAL MINIMAL SETS 297

FIGURE 6.40

Let A2 = CZ n l (a2) and B2 = CZ n l (b2) . Denote by d2 the closure of the domain


bounded by the segments [a2, b2] C C and A2B2 C C2 and by the arcs a2A2 C l (j)
and 12B2 C l (bz) for i = 1,... ,k. Let D (SZ, C) = U=1 d2 (Figure 6.40).
We show that D(1, C) = ir[D(1, C)] is the desired set.
Since it (B) = C, it follows that 1 C D(1, C).
Let w be a component of D(1, C) \ f It follows from the construction of the
set D(, C) that the inverse image it-1(w) breaks up into curvilinear quadrangles
bounded by segments of the curves C and C2 and arcs of trajectories. This implies
that w is simply connected.
We take a point b21 E B, i1 k, with a basic trajectory l (b21) passing through
it (for simplicity we say that a trajectory covering a basic trajectory is also basic).
aef
Then B21 = C21 n l (b21) is an endpoint of an adjacent interval (B1, A*1) X21 of
the Cantor set C2 n SZ, and the trajectory i(A1) is basic. It follows from ir(B) = C
that there exists a point A1 = C31 n l() 1) , aj, E B, that is congruent to A1.
Since no basic trajectory passes through the point ,3, it follows that j1 1, and
def
hence the adjacent interval (b31 -1, j) 12 lies in B Let j 1 - 1 = i2. We
.

repeat the process described for the point b2, and so on. As a result we obtain
a sequence of alternating arcs of trajectories and segments of the curves C and
Ci : bit Bit, ail Ai2 , X12, b22 B22 , X22 , a23 A23 ..... There are finitely many points
a2, b2 E B with basic trajectories passing through them, so the sequence leads in
finitely many steps to the interval n,n+1 = (b21, ajn+1) Consequently, the union
.

of the above arcs of trajectories and segments projects into a closed curve a21 on
M. It follows from the construction that a21 is a simple curve, and a21 is in the
boundary of the set D(1, C). Since the point b21, i1 k, is arbitrary, this proves
3).

4.3. Topological equivalence of minimal sets.


THEOREM 4.1. For flows f f and f2 on p > 2, let 11 and 12 be nontrivial
minimal sets that do not contain special pairs of trajectories. Then Ii and SZ2 are
topologically equivalent by means of a homeomorphism homotopic to the identity if
and only if there exist two semitrajectories l) C SZ 1 and 4) C SZ2 with the same
homotopy rotation class.
298 G. CLASSIFICATION OF FLOWS ON SURFACES

PROOF. NECESSITY. It follows from Lemma 2.3 and Corollary 2.8.

SUFFICIENCY. Let (i)) = µ(Q), where l C SZZ, i = 1, 2. It can be assumed


without loss of generality that li) and 4) are positive semitrajectories.
According to Lemma 3.1, f i and f2 have mutually homotopic contact-free cycles
Cl and C2 that are nonhomotopic to zero and intersect it and l2 (and hence Ii
and 12), respectively.
Let C(1) E it -1(C1) and C(2) E it -1(C2) be curves with common endpoints on
the absolute. As in the proof of Theorem 3.1, we construct the homeomorphism
O(C(1), C(2)) C(1) n SZ1 , C(2) n 522, where SZ1 = ir-1(1Z1) and S22 =
:

Then for the curves y(C(1)) and C(2) = y(C(2)) ('y E F arbitrary) and the
homeomorphism 9(C(1), C(2) ) : C21) -* C22) we have the relation
O(1),2)) o 'rl-(1)n52
C - = 'r
1
e(c(1), c(2)) C n521

Since C(Z)nSZi is a Cantor set, i = 1, 2, the homeomorphism O(C(1), C(2)) can be


extended to a homeomorphism C(1) (which we also denote by 9(C(1), C(2)).
c2)
On the curve y(C(1)) (-y E F) we extend ) to the homeomor-
e(C(1),C(2)) o'Y_1: C(1)
phism -y o -* C(2). Then for any element -y E F

(4.1) o[((1)),((25] o'y = -y o 9(C(1), C(2)).


According to Lemma 4.3, there exists a set D(SZ1, C(1)) such that D(11, C1) =
r[D(SZ1, C(1))] is a canonical set for Ii .
Suppose that the curves C(1), Ci1), ... , C 1) E it-1(Cl) are in the boundary
def
of the set D 1(SZ 1, C' 1)) D 1, and that the intersection C(1) n D 1 consists of
the intervals [ail), bi1)], ... , [a(1), b l)]. It follows from (4.1) that the points ai2) =
9(C(1) , C(2)) (ail)) and b 2) = O(C(1), C(2))
(1)) are congruent, and there are no
other pairs of congruent points on the segment ai2) b2) C C(2) . Then by virtue of
def
Lemma 4.3 we construct a set D2 (1, C(2)) D2 such that D(12, C2) = it (D2) is
a canonical set for SZ2, and the intersection C(2) n D2 consists of intervals lying on
b2) .
the segment ai2)
By repeating the proof of Theorem 3.1 without fundamental changes we get
the following:
1) the boundary of D2 (522, C(1)) contains segments of the curves
(2)2)2) E(1
7r (C2 ),

where and C(2) have common endpoints on the absolute for i = 1, ... , k, and
2);
D2 n it -1(C2) C ( (2) u=i
2 the intersection C(2) n D2 consists of the segments
1ai2>>
bi2)J

=
b(l)])>
... , O(1),2))/U(1)
a> b]);
3) the intersection n D2 is equal to C(2)) (C(1) n D1);
4. CLASSIFICATION OF NONTRIVIAL MINIMAL SETS 299

4) there are no pairs of congruent points in the domain int DZ, i = 1, 2.


This implies the existence of a homeomorphism Sp : D1 - D2 that carries arcs
of trajectories in SZ 1 n D 1 into arcs of trajectories in SZ2 n D2 and that projects
into a homeomorphism Sp : D (1Z1, C1) -* D (1Z2, C2) carrying trajectories in SZ1 into
trajectories in 12 (because SZZ C D (SZZ, CZ) for i = 1, 2 in view of Lemma 4.3).
It follows from 1)-3) that to each simple closed curve al of the boundary
DD (11, C1) there corresponds a simple closed curve A2 of DD (12, C2) that is ho-
motopic to it, and conversely. Let b denote the correspondence Al -p A2
We take a component R1 of the set M \ D(11, 1, C1). Since to each component of
DR1 there corresponds a unique component of DD (12, C2) via the mapping b, there
exists a unique component R2 of M \ D (1Z2, C2) such that realizes a one-to-one
correspondence between the components of the boundaries DR1 and DR2. Since the
curves A C DR1 and b(A) C DR2 are homotopic, it follows from the condition 2)
of Lemma 4.3 that the homeomorphism Sp can be extended to a homeomorphism
R1 UD (1 1, C1) -* R2 UD (1Z2, C2). Going through this procedure for each component
of M \ D (SZ 1, C1), we get a homeomorphism : M -* M, carrying trajectories in
1 into trajectories in SZ2. The fact that is homotopic to the identity is shown
as in the proof of Theorem 3.1. U

THEOREM 4.2. For flows f i and f 2 on p > 2, let SZ1 and 12 be nontrivial
minimal sets not containing special pairs of trajectories. Then Ii and SZ2 are
topologically equivalent if and only if there exist semitrajectories lid C SZ1 and
4) C SZ2 with the same rotation orbit.
PROOF. NECESSITY. It follows from Theorem 2.4.

SUFFICIENCY. It follows from Theorem 4.1 and the fact that any automorphism
of the group F is induced by some covering homeomorphism (Nielsen's theorem,
§2.9). U

4.4. Realization of nontrivial minimal sets by geodesic curves.


THEOREM 4.3. For a flow f t on p > 2, let 1 be a nontrivial minimal set
not containing special pairs of trajectories. Then there is a flow fo on M such
that:
1) fo has a nontrivial minimal set 1 o whose trajectories are geodesic curves in
a metric of constant negative curvature;
2) SZo is topologically equivalent to SZ by means of a homeomorphism homotopic
to the identity.
PROOF. The nontrivial minimal set 1 is quasiminimal. According to Lemma
2.6, there exists a geodesic lamination 3(1) on M that consists of nontrivial B-
recurrent geodesics and has the following property: for any trajectory l C SZ there
a1
is a unique geodesic L L(l)_C Y(1Z) such that for each inverse image l E r-1(l)
there exists an inverse image L E r-1(L) with the same endpoints o (l) = o (L)
and a+ (l) = o+ (L) on the absolute.
We define on Y(1) the vector field V of unit tangent vectors to the geodesics
in 3(1). Since the lamination 3(1) is constructed from the trajectories of a flow,
the vector field V on Y(1) is continuous. We extend it to a continuous field Vo on
the whole surface and let fo be the flow induced by j.
300 6. CLASSIFICATION OF FLOWS ON SURFACES

Denote by I3(1)I the set of points lying on the geodesics in


Since 1 is a nontrivial minimal set, it follows that Iis a nowhere dense
invariant set for fo. Since each trajectory l C 1 is dense in 1, each trajectory
to C Iof fo is dense in IConsequently, Iis a nontrivial minimal
set of the flow fo.
By the construction of the geodesic lamination 3(1), for each semitrajectory
l C 1 of the flow f t there is a semitrajectory lob C 3(1) I of fo with the same
homotopy rotation class. This and Theorem 4.1 give us that the minimal sets 1
and Iare topologically equivalent by means of a homeomorphism homotopic
to the identity. U

REMARK. It was proved in [18] that the vector field j7 on 3(1) is not only
continuous but also Lipschitzian. By a theorem of Schwartz in [107], it cannot have
smoothness C', r > 2. It remains an open question as to whether V is a vector
field of class C1.

§5. Topological equivalence of flows


without nontrivial recurrent trajectories
In 1955 Leontovich and Mater [51] introduced a complete topological invariant
for flows on a sphere with finitely many singular trajectories the scheme of a flow
which included a qualitative description of the singular trajectories and their mutual
arrangement. In 1976 Neumann and O'Brien [100] introduced an invariant the
orbit complex as a generalization of this invariant to the set sR1, of flows on a
surface p > 2, with finitely many singular trajectories and without nontrivial
recurrent trajectories.
The present section is devoted to a description of this invariant.
We remark that the Morse-Smale flows classified by Peixoto [104] with the
help of a distinguishing graph belong to the set R1,.
Everywhere in this section OI denotes the set of flows with finitely many singu-
lar trajectories and without nontrivial recurrent trajectories on a closed orientable
surface M of genus p > 0. Unless otherwise stated, flows are allowed to have
infinitely many singular trajectories in §§5.1-5.3.

5.1. Schemes of semicells. According to Theorems 4.1 and 4.2 in Chapter


3, any cell R of a flow f t E OI is homeomorphic either to an open disk (simply
connected) or to an open annulus (doubly connected), and by Theorem 4.3 in the
same chapter, the restriction f t I R of ft to R is topologically equivalent to one of
the following flows:
1) a parallel flow on an open strip;
2) a parallel flow on an open annulus;
3) a spiral flow on an open annulus;
4) a rational winding on the torus.
For flows of the first three types we introduce in this subsection the concept of
a semicell and of its scheme.
Suppose that the flow f t I R has type 1) or 2), and let l C R be a trajectory of
f t . By Theorem 4.3 in Chapter 3, l divides the cell R into two domains R+ and
R- called semicells of types 1) or 2), respectively.
5. TOPOLOGICAL EQUIVALENCE OF FLOWS 301

Let f t R have type 3). According to Theorem 4.3 in Chapter 3, there exists a
contact-free cycle C of f t I R dividing R into two domains R+ and R- and inter-
secting each trajectory of f t I R at exactly one point. The domains R+ and R- are
called semicells of type 3).
Denote by SR the accessible (from within) boundary of R.
The accessible (from within) boundary SRS of the semicell R C R is defined
to be the part of SR adjacent to R± (consequently, the trajectory or contact-free
cycle dividing R into R+ and R- is not in the accessible (from within) boundary
of R+ or of R-).
We introduce the concept of the scheme of a semicell. Consider a semicell R± of
type 1). On the set of regular trajectories in the accessible (from within) boundary
SR± we introduce an order relation. Let ll, l2 C SR± be regular trajectories. Take
points m2 E l2, i = 1, 2, and disjoint contact-free segments >JZ, i = 1, 2, such that >Z
has m2 as an endpoint, and >Z \ {m2 } C R (Figure 6.41). According to Theorem
4.3 in Chapter 3, there are trajectories in R that intersect both 1 and 2

rn m2

FIGURE 6.41

We write ll < l2 if for any points m2 E l2 and any contact-free segments j,


i = 1, 2, there exist trajectories l C R± that first intersect 1 and then 2 as time
increases.
DEFINITION. The ordered set of regular trajectories in the accessible (from
within) boundary of a semicell of type 1) is called the scheme of the semicell of
type 1). The schemes of two semicells are said to be isomorphic or identical if there
exists an order-preserving one-to-one mapping of one scheme into the other.
We consider a semicell R of type 2) or 3). If SRS consists of a single trajectory
(a limit cycle), then the scheme of R is defined to be the empty set {O}. If SRS
consists of a single equilibrium state O and a single trajectory to with w(lo) =
a(lo) = O, then the single-element set {lo} is the scheme of R±. Now suppose
that SR+ contains at least two regular trajectories. We fix some "initial" regular
trajectory l 1 C SR±. Then on the remaining regular trajectories in SR± \ {li} an
order relation is introduced as above. Since the choice of the initial trajectory in
SR± is arbitrary, we use the term the cyclic order for all the orderings obtained on
SRS for different choices of the initial regular trajectory.
DEFINITION. The cyclically ordered set of regular trajectories in the accessible
(from within) boundary of a semicell is called the scheme of the semicell of type 2)
or 3
The scheme of a semicell R± will be denoted by w (R±) . The scheme of a semi-
cell of type 2) or 3) can be represented as a list (11,. .. , l, In+1 = ll) of trajectories,
302 6. CLASSIFICATION OF FLOWS ON SURFACES

,e ,,
e4, e e}

FIGURE 6.42

with the order in which the trajectories are encountered upon moving along SR±
in the direction induced by the flow f t I R± (Figure 6.42).
Two schemes of semicells of type 2) or 3) are said to be isomorphic or identical if
there exists a one-to-one correspondence of one scheme into the other that preserves
the cyclical order.
We remark that if the number of singular trajectories is finite, then the number
of singular regular trajectories in the accessible (from within) boundary of a semicell
is a complete invariant of the property of being isomorphic for schemes of semicells
(with the symbol 0 regarded as a special "number").
For a flow f t on M we denote by ft the restriction of ft to the set M \ Fix (f t) ,
where Fix(f t) is the set of equilibrium states of ft.
The next result follows immediately from Theorem 4.3 in Chapter 3 and the
definition of schemes of semicells.
LEMMA 5.1. Suppose that Ri and R2 are semicells of the same type for the
respective flows f i and f. If the schemes of the semicells are isomorphic, then the
restrictions f 1t IRi ubRi and f 2 I R2 uoR2 are topologically equivalent.

We remark that the flows f i l Ri ubRi and f 2 I R2 u6R2 are not topologically equiv-
alent in general (Figure 6.43).

FIGURE 6.43

5.2. Schemes of spiral cells. Let R be a cell of a flow f, and suppose that
the restriction f t I R is topologically equivalent to a spiral flow on an open annulus.
We give two possible topological types of the flow f t I R. Let K be the open
annulus on IjS2 bounded by the two circles Cl : x2 + y2 = 1 and C2 : x2 + y2 = 4.
There exists a homeomorphism b : R -* K carrying f t I R into the flow f = o
f t1R o-1 on K.
Two cases are possible: a) the trajectories of f, with increasing time induce the
same motion on Cl and C2, that is, either clockwise or counterclockwise (Figure
5. TOPOLOGICAL EQUIVALENCE OF FLOWS 303

FIGURE 6.44

6.44, a) ); b) the trajectories of f, induce opposite motions on C1 and C2 with


increasing time (Figure 6.44, b)).
We assign the type "plus" or "minus" to a cell for case a) or b), respectively.
DEFINITION. Let R be a cell of a flow f t such that the restriction f t I R is
topologically equivalent to a spiral flow on an open annulus, and let R+, R- C R
be semicells with schemes w (R) and w (R-) . The scheme of the cell R is defined
to be the union of the schemes w (R) and w (R) with the assigned types "+" or

We remark that if R1 and R2 are cells of respective types "+" and "-", then
by virtue of the opposite motions induced on the circles Cl and C2 of the annulus
K, the flows f t I R1 uoRl and f t I R2uoR2 cannot be orbitally equivalent. Since the
type of a cell does not change when the motion in time is reversed, f t I R1 uoRl and
f t I R2 UoR2 also are not topologically equivalent.

5.3. The orbit complex. Let f t be a flow on a surface M. We introduce


an equivalence relation on points of the surface are equivalent if they lie on a
single trajectory of f t. The quotient space with respect to this equivalence relation
is denoted by Mr/ft. The set Mr/ft, equipped with the quotient topology induced
by the natural projection a : M -+ Mr/ft, is called the trajectory space (or orbit
space) of the flow ft.
The orbit complex K(f t) is the trajectory space Mr/ft, equipped with the
additional structures given below.
The cell structure. The image of each cell R of f t under the projection a : M -+
Mr/ft is called a 1-cell. If f t I R is equivalent to a parallel flow on an open strip or
annulus, then A (R) is an open segment. If f t IR is equivalent to a spiral flow on an
open annulus or to a rational winding on the torus, then )(R) is a closed curve.
We call the 1-cell A (R) open or closed according as to which of these cases holds.
The image of a singular trajectory under A is called a 0-cell.
The fiber structure. Let rU be an i-cell (i = 0, 1), and take a point m E rU.
The inverse image )c4 (fi'i) is a closed regular trajectory, or a closed trajectory, or
an equilibrium state. We define the fiber over the i-cell rU to be a line, circle, or
point according as to which of these cases holds. It is not hard to show that this
definition does not depend on the choice of the point m E rU.
304 6. CLASSIFICATION OF FLOWS ON SURFACES

We remark that the fiber over a 0-cell can be a fiber of any one of these forms.
However, the fiber over a 1-cell cannot be a point.
The order structure. For each semicell R± the order introduced for regular
singular trajectories in the accessible (from within) boundary SR± induces an order
on the 0-cells corresponding to these singular trajectories. Moreover, if to E SR±
is an equilibrium state and l E SRS a regular trajectory, then we set a(lo) < a(l)
when to E a(l) and a(lo) > a(l) when to E w(l). If to a(l) U w(l), then no order
relation is established between a (lo) and A(l).
Further, suppose that the flow f t on the cell R is topologically equivalent to
a spiral flow on an open annulus. According to Theorem 4.3 in Chapter 3, all the
trajectories in R have a common w-limit set w (R) and a common a-limit set a (R) .

For any trajectories l C w(R) f1 SR and l C a(R) f1 SR we let


a(l) <A(R) < a(l).
The type of a closed 1-cell. Suppose that the flow f t on the cell R is topologically
equivalent to a spiral flow on an open annulus. In §5.2 the type "+" or "-" was
assigned to the flow f t I R. Accordingly, we assign the type "+" or "-" to the closed
1-cell A(R).
DEFINITION. The orbit complex K(ft) of a flow f t on a surface M is defined to
be the trajectory space Mr/ft equipped with the cell structure, the fiber structure,
the order structure, and the assigned type for certain closed 1-cells.
Two orbit complexes K (f i) and K(f) are said to be isomorphic if there exists
a homeomorphism h : K (f 1) -* K(f) preserving all the structures introduced on
the complexes.
EXAMPLES. In Figure 6.45, a) and b), we represent two flows f i and f 2 with
nonisomorphic orbit complexes (nonisomorphic order structures). It is not hard to
see that f i and f2 are not topologically equivalent.

U1 I L' I
L,a
Un
+ L2L+
21 22 e3 2y

> L2> e2> L42 > Q1 .

a) b)

FIGURE 6.45

In [104] Peixoto presents the graphs of two flows f i and f2 on the sphere that
are not topologically equivalent (Figure 6.46, a) and b)). The edges of the graphs
are separatrices of saddles (besides the four edges joining the sources in the "petals"
to the sink), and the vertices are equilibrium states. The directions of the arrows
correspond to the motion along trajectories as time increases.
5. TOPOLOGICAL EQUIVALENCE OF FLOWS 305

a) b)

FIGURE 6.46

The flows f i and f 2 each have one stable node on the disk D C 82 (the disk is
outlined by a dashed line), six saddles, and seven unstable nodes (six of them lie
in the "petals" on D, and one in S2 \ D) .
We leave it the reader as an exercise to prove that the graphs in Figure 6.46
are isomorphic, but that the orbit complexes K (f i) and K(f) are not isomorphic.
5.4. Neighborhoods of limit singular trajectories.
LEMMA 5.2. Suppose that a flow f t on a closed orientable surface p > 0,
has finitely many singular trajectories, and let l be a singular nonclosed trajectory
whose w -limit set w (l) contains regular points. In this case:
1) w (l) is either a closed trajectory, or a one-sided contour;
2) l belongs to the accessible (from within) boundary of a cell R of f t on which
f t is topologically equivalent to a parallel flow on an open strip;
3) the set w (l) has a neighborhood U satisfying the conditions
a) there are no equilibrium states nor closed trajectories in U \ w (l ),
b) each component of U \ w (l) is homeomorphic to an open annulus,
c) the component -y of DU intersected by l is a contact free cycle of ft that is
intersected by l only once (Figure 6.47), and, moreover, -y f1 w (l) _ 0, and each
positive semitrajectory intersecting -y has w-limit set equal to w (l) .

FIGURE 6.47
306 6. CLASSIFICATION OF FLOWS ON SURFACES

PROOF. Since f t does not have nontrivial recurrent trajectories and since w(l)
is not an equilibrium state by assumption, w (l) is either a closed trajectory or a
one-sided contour by virtue of the catalogue of limit sets (Theorem 3.6 in Chapter
2).
This and the finiteness of the number of singular trajectories gives us the exis-
tence of a neighborhood Uo of w (l) satisfying the assertions 3a), 3b).
Since l is a singular trajectory, it belongs to the boundary of some cell R.
According to Theorem 2.2 in Chapter 2, l cannot lie in the limit set of a trajectory
of ft, for otherwise l would be a nontrivial recurrent semitraj ectory. Therefore, l
belongs to the accessible (from within) boundary of R.
Let m0 E w (l) be a regular point, and draw a contact-free segment E C uo
through m0. It follows from the relation m0 E w (l ), the fact that l is a nonclosed
trajectory, and the proven assertion 3a) that there is a E-arc m m2 of l such that
m m2 C U0, and the segment mlm2 C E does not intersect w (l) According to .

Lemma 1.2 in Chapter 2, there is a contact-free cycle y C U0 with y n w (l) = 0 that


is intersected by 1.
Denote by Uo the component of U0 \ w (l) containing y. Since Uo is an open
annulus, y separates tLo into two open annuli 'lei and 'u2. Suppose that with
increasing time l intersects y and goes into the annulus U2 at some point m E y.
Then m is the unique point where l intersects y, and the semitrajectory l+ (m) \ {m}
does not intersect y. Removing the set cl (Ui) from U0, we get a neighborhood U
satisfying the assertions 3a)-3c).
Since l is a nonclosed trajectory and w (l) contains regular points, f t I R cannot
be topologically equivalent to a parallel flow on an open annulus. From Theorem
4.3 and the fact that the trajectory l C OR is not in the limit set of any trajectory
of f t it follows that f t R is not topologically equivalent to a spiral flow on an open
annulus. Obviously, f t I R is not a rational winding on the torus. Consequently, f t I R
is a parallel flow on an open strip.

REMARK. The assertion of Lemma 5.2 remains valid when w (l) is replaced by
a(l).
The annular domain bounded by the contact-free cycle y and w (l) will be called
a one-sided annular neighborhood and denoted by U (w (l)) .
Any positive semitrajectory of f t enters U(w (l)) by crossing y and cannot leave
this neighborhood. Therefore, any positive semitrajectory intersects the contact-
free cycle y at most at one point, and the restriction f t Iu(w(l)) is topologically
equivalent to a spiral flow on an open annulus.
Denote by OU(w (l)) the union of the regular trajectories belonging to w (l) .
In §4.5 of Chapter 3 we constructed C°°-flows fl, i = 0,1, 2, 3, (smooth models)
such that the restriction of the flow ft to the set R U OR (where OR is the union
of the regular trajectories belonging to the accessible (from within) boundary OR
of the cell R) is topologically orbitally equivalent to one of the model flows f,
i = 0,1, 2, 3 (Lemma 4.4 in Chapter 3). The flow f2 is called a spiral flow of "plus"
type, and is defined in the annulus 1 < r < 2.
The next result is a consequence of Lemma 4.4 of Chapter 3 and Lemma 5.2.
COROLLARY 5.1. Assume the conditions of Lemma 5.2. Then the restriction
f tI u(w(t))u6cL(w(l)) of the flow ft to the set U(w(l))UO'(w(l)) is topologically orbitally
5. TOPOLOGICAL EQUIVALENCE OF FLOWS 307

equivalent to the spiral model flow J2 of "plus" type, restricted to the annulus 1.5 <
r<2.
5.5. Main theorems.
THEOREM 5.1. Suppose that the flows f i , f 2 E OtJ have finitely many singular
trajectories on the closed orientable surface M, p > 0. Then they are topologi-
cally orbitally equivalent if and only if the orbit complexes K(f) and K(f2) are
isomorphic.
PROOF. If f i and f 2 are topologically orbitally equivalent, then a homeomor-
phism M -* Mp carrying trajectories of one flow into trajectories of the other
with preservation of direction in time induces an isomorphism of the orbit com-
plexes K(f) and K (f 2) .

Let K : K (f i) -* K(f) be an isomorphism of the orbit complexes. Since


K preserves the cell structure (in particular, 0-cells are carried into 0-cells), K
establishes a one-to-one correspondence between the singular trajectories of the
flows f i and f 2 . The isomorphism K preserves the fiber structure, so it establishes a
one-to-one correspondence between the equilibrium states, between the limit cycles,
and between the separatrices of f j and f2.
We consider the set GZ of singular trajectories of f Z that are not equilibrium
states and lie in the limit sets of other singular trajectories of f, i = 1, 2. Since
f2 does not have nontrivial recurrent trajectories, Theorem 3.6 in Chapter 2 gives
us that any trajectory LZ E Gi either is a separatrix and belongs to a one-sided
contour K2 , or is a limit cycle (which we also denote by K), i = 1, 2. If l Z is a
singular trajectory of f2 with Li C w(li), then w(li) = KZ in view of Lemmas 1.6,
3.4, and 3.5 in Chapter 2. Therefore, by Lemma 5.2 the trajectory li belongs to
the accessible (from within) boundary of a cell R of f2 on which f2 is topologically
equivalent to a parallel flow on an open strip i = 1, 2.
The orbit complex K(f2 ), equipped with the topology induced by the mapping
XZ : Mp -> M/ f i , is not a Hausdori space in general; that is, K (f Z) has points
without disjoint neighborhoods (points that cannot be separated), i = 1, 2.
The set A (G) C K(f) is a set of 0-cells with fiber IR1 or S1 that, in the set
O(f) C K (f Z) of 0-cells with fiber II81 or S1, are not separated from certain 0-cells
in O(fZ) \ A(G) with fiber R', i = 1, 2. Since K is a homeomorphism, this implies
the equalities
K[al (G1)1 = A2 (G2), K-1 [A2 (G2)] = A, (G, ).

The isomorphism K : K(f) -* K(f) induces a one-to-one correspondence be-


tween the cells of f i and f 2 of the same type, and an isomorphism of the schemes
of these cells. This and Lemma 5.1 imply the existence of a homeomorphism car-
rying trajectories in cells and in their accessible (from within) boundaries for f i
into trajectories in cells and in their accessible (from within) boundaries for f2,
with preservation of the direction of motion along trajectories. From the structure
of neighborhoods of singular trajectories of f i not lying on the accessible (from
within) boundary of any cell (Lemma 5.2) it follows that this homeomorphism can
be extended to a homeomorphism on the whole of M which carries trajectories of
fi into trajectories of f2 with preservation of the direction of motion.
For a flow f t we denote by ft the flow obtained by reversing the direction of
the motion in time.
308 6. CLASSIFICATION OF FLOWS ON SURFACES

The next theorem follows from Theorem 5.1.


THEOREM 5.2. Suppose that flows f, f2 E on the closed orientable surface
Mp, p > 0, have finitely many singular trajectories. Then f i and f 2 are topologically
equivalent if and only if the orbit complex K(f i) is isomorphic to one of the orbit
complexes K(f2) and K(f2 t).
REMARK. Theorem 5.1 was proved in [100] for orientable and nonorientable
(not necessarily compact and closed) surfaces. In the same paper it was shown that
the condition of finiteness of the number of singular trajectories in that theorem
can be replaced by the condition that the equilibrium states be isolated and that
there not be any regular singular trajectories lying in the limit sets of other singular
trajectories (that is, that there not be limit separatrices).
CHAPTER 7

Relation Between Smoothness Properties


and Topological Properties of Flows
In the whole myriad of relations between topological properties and smooth-
ness of flows we dwell on a problem going back to a conjecture of Poincare. While
treating flows on the torus in his memoir [68] Poincare showed that there are con-
tinuous flows with nontrivial minimal sets, that is, nowhere dense minimal sets,
that are different from equilibrium states and periodic trajectories. Locally, non-
trivial minimal sets have the structure of the product of a closed bounded interval
and the Cantor set. Poincare conjectured that there exist smooth (even analytic)
flows on the torus with nontrivial minimal sets. In a series of papers (of which the
fundamental one is the 1932 paper [82]) Denjoy showed that there are C1-flows
on the torus with nontrivial minimal sets, but not C''-flows for r > 2 (that is,
Denjoy in fact refuted Poincare's conjecture). In 1963 Schwartz [107] extended
the last result of Denjoy to surfaces of larger genus. Nevertheless, as far back as
1937 Cherry had shown [81] that Poincare's conjecture is valid for quasiminimal
sets. That is, Cherry constructed on the torus a flow of analytic smoothness with
a nontrivial quasiminimal set. The topological structure of Cherry's flow was not
made completely clear, and this led to the problem of Cherry which is solved in the
second section of the present chapter.
Investigations of Neumann [99] and Gutierrez [86] were devoted to converses
of the theorems of Denjoy and Schwartz. They showed that if a flow does not have
nontrivial minimal sets, then it is topologically equivalent to a C°°-flow. In the
general case the flow is topologically equivalent to a C1-flow.
§1. Connection between smoothness of a flow
and the existence of a nontrivial minimal set
In this section we present theorems of Denjoy [82], Schwartz [107], Neumann
[99], and Gutierrez [86].
1.1. The theorems of Denjoy and Schwartz.
THEOREM 1.1 (Denjoy). Suppose that a C'' flow f t (r > 2) on the torus does
not have equilibrium states nor closed trajectories. Then the whole torus is a min-
imal set of f t, and f t is topologically equivalent to an irrational winding.
PROOF. Assume the contrary. Then by Lemmas 3.1 and 3.2 in Chapter 3, f t
is a Denjoy flow and has a nontrivial minimal set N. Consequently, f t has a global
section C on which it induces a homeomorphism P : C -* C with Cantor limit set
N fl C. In view of Lemma 1.3 in Chapter 2, P is a C''-diffeomorphism, r > 2, and
this contradicts Theorem 4.1 in Chapter 5 (the Denjoy theorem for diffeomorphisms
of the circle). 0
309
310 7. RELATION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES

COROLLARY 1.1. A C''-flow (r > 2) on the torus does not have nontrivial
minimal sets.
PROOF. Assume the contrary. Let f t be a C''-flow (r > 2) on the torus with a
nontrivial minimal set N. Since f t does not have closed trajectories nonhomotopic
to zero, and sinceNsome neighborhood of N does not contain equilibrium states, there
exists a C2-flow f t on the torus without equilibrium states and closed trajectories
and with nontrivial minimal set N, and this contradicts Theorem 1.1. 0

THEOREM 1.2 (A. J. Schwartz). Let f t be a C''-flow, r > 2, on a two-dimen-


sional surface M. Then f t does not have compact nontrivial minimal sets. In other
words, if N is a compact minimal set of f t, then it is 1) an equilibrium state, or 2)
a closed trajectory, or 3) the whole of M, and in the last case M is a torus.
There is a proof of this theorem in the book [74], and we omit it. We note only
that the proof of the Denjoy theorem is based on Theorem 4.1 in Chapter 5, which
in turn is essentially based on the arithmetic properties of the Poincare rotation
number of a diffeomorphism of the circle. The proof of Theorem 1.2 actually re-
duces to an exchange of segments on the circle, but so far there is no corresponding
arithmetic theory for a topological invariant of an exchange of segments (a knead-
ing). Therefore, the proof of the Schwartz theorem differs essentially from that of
the Denjoy theorem.
To distinguish the idea behind the proof of the Schwartz theorem, it is presented
in [25] for a diffeomorphism of the circle (a degenerate exchange).
1.2. The theorem of Neumann. Let f t be a C°-flow on a surface M.
DEFINITION. f t is said to be smoothable if there exists a Ck-flow, k > 1, that
is topologically equivalent to ft.
In 1978 Neumann proved that a C°-flow without nontrivial recurrent trajecto-
ries and with finitely many equilibrium states on a compact orientable surface is
topologically equivalent to a C°°-flow.
For simplicity we present the Neumann theorem for a flow with finitely many
singular trajectories on a closed orientable surface.
We need the following result.
LEMMA 1.1. Let 9: D2 \ {0} -, D2 \ {0} be a C°° -mapping on the punctured
dish D2 \ {0}, D2 = {(x, y) : x2 -I- y2 < 1} (0 the origin of coordinates). Then there
exists a C°° function A : D2 -> [0, 1] such that:
1) A - 1 in the annulus {(x,y) :3/4< x2 + y2 < 1};
2) the mapping A9: 2\{0} -> R2 extends to a C°°-mapping D2 - R2 carrying
0 into 0.
PROOF. Let {B}i° be a locally finite covering of the annulus 0 <x2 -+2 < 1/2
by disks Bn in the annulus 0 < x2 + y2 < 3/4 and not tangent to the origin
(O0B).
For a function f : D2 - R we set

If 1o = sup ImED2
1. CONNECTION BETWEEN SMOOTHNESS AND EXISTENCE OF A MINIMAL SET 311

Let 91,92: D2 \ {0} -* R be the coordinate functions of the mapping e, that is,
e = (9i,9). Fork = (k1,k2), ki E N U {0}, we set
ak1+k2ei .
ak ei =
ak 1 x ak2 y

According to [6?] there exists for each disk Bn a C°°-function an : D2 [0, 1]


with support in Bn such that

(1.1) ICA Gi I°< 2-n, i= 1, 2,


for all k = (11,12) with ki E N U {0} and Iki < n.
Let )° : D2 -* [0, 1] be a C°°-function equal to 1 on the annulus 1/2 < x2 +y2 <
1 and to 0 on the disk x2 + y2 <_ 1/4.
We consider the function \ = -* R. It is not hard to verify that
A is of class C°°, and A - 1 on the annulus 3/4 < x2 + y2 <1.
We show that

lim ac aei (m) = 0

(i E {1, 2}) for fixed k = (11,12), ki e Z. For E> 0 we take a number N E N such
that Iki < N and 2-N+1 Since 0 aBn, there exists a disk B5 : x2 + y2 <62
disjoint from B1,. .. , BN_ 1 and the annulus 1/2 < x2 + y2 < 1. Then for any point
m E B5 \ {0} and the disks Bn1, ... , Bnr containing m we have that nj > N for
j = 1, ... , r. From (1.1),

IakAeimI < I + ... + I < 2-n = 2-N+1 <


n=N

which yields (1.2). Consequently, A is the desired function. 0

Let X and Y be disjoint topological spaces, and f : A - Y a homeomorphism


of a subspace A C X onto its image. Denote by X Uf Y the result of pasting
together X and Y along A by means of the mapping f ; that is, X Uf Y is the
quotient space of the union X U Y with points x E A C X and f (x) E Y regarded
as equivalent.
THEOREM 1.3. Suppose that a C° flow f t on a closed orientable surface Mp,
p > 0, has finitely many singular trajectories and does not have nontrivial recurrent
trajectories. Then f t is topologically equivalent to a C°° -flow on M.
PROOF. The idea of the proof is to break up M into subsets on each of which
the original flow is topologically orbitally equivalent to some smooth model flow,
and then to paste together the subsets and their smooth models. The resulting flow
in M is a C°°-flow in a new differential manifold structure on Mp, which is not in
general connected in an obvious way with the original manifold structure of M
Denote by G the family of limit cycles and one-sided contours of ft that form
the w- (a-) limit sets of singular trajectories of ft. Let l be a singular trajectory,
and let w (l) (a (l)) be in G. Associated with each element w (l) (a (l)) E G (in view
312 7. RELATION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES

FIGURE 7.1

of Lemma 5.2 in Chapter 6) is a set U(W(l)) (U(a(l))) homeomorphic to an open


annulus and bounded by w (l) (a (l)) and a contact-free cycle C (Figure 7.1).
The union of U(W(l)) (U(a(l))) and the regular trajectories in w(l) (a(l)) is
denoted byU(W(l)) (U(a(l))).
According to Corollary 5.1 in Chapter 6, the restriction f t l u (w (l )) (ftIu((I)))
is topologically orbitally equivalent to the spiral model C°°-flow restricted to a
certain subset of a closed annulus (points lying on the boundary are removed from
the closed annulus).
By assumption, the number of disjoint elements 1, ... of the family G
is finite. It can be assumed without loss of generality that the open domains
U(1),. .., U(r) are disjoint. Let 'l,Li = i = 1,... ,r (we remark that the
i
'1,L1, ... ,'1,Lr can intersect in singular regular trajectories). The restriction ft l is
topologically orbitally equivalent to ft k' where ft is the model spiral C°°-flow,
and Ki is a subset of the closed annulus 1 < r < 1.5, i = 1, ... , r, one of whose
boundary components Ji (r = 1.5) is in Ki .
Let f t (Ui) stand for f t and let ei : U2 - Ki be a homeomorphism realizing
a topological orbital equivalence.
Let R1,... , Rk be the cells of f t intersecting UUi (i = 1, ... , r) and take the
ae
cell R1 R. Since each set 'l,Li, i = 1, ... , r, is bounded by a contact-free cycle
and either a limit cycle or a one-sided contour, the cell R intersects at most two
components in the union UUi (i = 1, ... , r). Assume that R intersects U1 and U2 (if
R intersects only one component, then the rest of the arguments become simpler).
The restriction f t R is topologically equivalent to a parallel flow on the open strip
II = {(x, y) : 0 < y < 1, x E R}, so the trajectories in R go into one of the
components (say U1) with increasing time, and into the other (U2) with decreasing
time (Figure 7.2).

FIGURE 7.2
1. CONNECTION BETWEEN SMOOTHNESS AND EXISTENCE OF A MINIMAL SET 313

The intersection of R with the contact-free cycle C2 (i = 1, 2), which is in the


boundary of 'i,Li , is an open contact-free segment >. Let O 1 = Ul + (m) , m E cl ( 1) ,
and let O2 = Ul - (ift), ff E cl According to Lemma 5.2 in Chapter 6, O1
and 02 consist of regular points of ft. Since the restriction f t IR is topologically
equivalent to a parallel flow, the Poincare mapping P : > - > is defined, and it
is a homeomorphism and extends to a homeomorphism P : cl ( 1) -* cl
The intervals 91(s 1) and 92 (2) lie on the boundaries of the respective closed
annuli cl(Ki) and cl(K2). Therefore, there exists a C°°-diffeomorphism of the
closed rectangle IIo = {(x, y) E R2 : -1 < x < 1, 0 < y < 1} onto its image with
the following properties:
1) (Io n {x = -1}) = and (Io n {x = +1}) =
2) (Io) n [cl(K1) U cl(K2)] = U ();
3) together with the semitrajectories of the flows f t 101 and f t 02, the family
of curves (ly) (0 < y < 1), where 1, = {(x, y), -1 < x < 1, y = const}, forms a
C°°-foliation 3 on the set 9(01) U (Io) U 9(02);
4) 92 o P o 9j'[(-1, y)] = (+1, y) for any fixed 0 < y < 1 (Figure 7.3).

FIGURE 7.3

ae
By the property 3), there exists on the set IIo 9 (O1) U (Io) U 9(02) a C°°-
flow f* that coincides with f tl0i on 9(Oi), i = 1, 2, and has trajectories coinciding
with the leaves of 3. It follows from the property 4) that f t I R is topologically
orbitally equivalent to the restriction f t int no . It follows from this and Lemma 4.4
in Chapter 3 that there exist finitely many points Po C (lo) and P1 C (l1) and a
C°°-flow ft(R) on the set IIo such that the restriction of ft(R) to IIo \ (Po U P1) is
topologically orbitally equivalent to the flow f t I RUoR , where SR is the union of the
regular trajectories in the accessible (from within) boundary OR of the cell R. Thus,
aef
for the flow f t I RuoR we have obtained a model C°°-flow f t (R) Inoff(P0 uP1) fl
"compatible" with the model flows f (11L1) and f (1,L2) .
Carrying out the procedure described above for all the cells R = R1,. . . , Rk
intersecting UlLi (i = 1, ... , r), we get model C°°-flows f1, ... , fk "compatible"
314 7. RELATION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES
V V

with the model flows f (u 1) , ... , f ('r) . We denote the sets lb \ (P0 U P1) , ... on
V V V V
which the flows f 1, ... , f are defined by N1,.. . , Nk, respectively. V

We now consider all the cells R1,.. , Rq (the first k of which intersect UUi ) .

of the flow ft. According to Lemma 4.4 in Chapter 3, the flow f t IR; uoR; is topo-
logically orbitally equivalent to a C°°-flow f defined on the set I, j = 1... ,q. V V V V

By pasting together the sets R1 U SR1, ... , Rq U SRq, Ul, ... , Ur along the regu-
lar singular trajectories according to the definite scheme S, we can get the set
V

_ M \ Fix(f t), where Fix(f t) is the family of equilibrium states of f t. There-


fore, by pasting together the sets N1, ... K1, ... , kr according to the scheme V

S we obtain a set N homeomorphic to M.


V V V
We introduce on N a manifold structure such that the flow f on N "pasted
V V V

together" from f',. , f (tLr) is C°°- smooth.


. .
V V V V
Consider the two sets N2 and Nk with the model C°°-flows f i and f k , identified
along the singular regular trajectories Li and Lk, respectively. Assume first that V

Li and Lk are not closed trajectories. By the structure of the model flows ft and
V V V
f k , there are contact-free segments >i C N2 and >k C Nk whose endpoints lie
V

on Li and Lk, respectively. Since ft and f k are C°°-flows, there exist mappings
Si : 0] -* >i and Sj : (-E, 0] -" Y (Si(0) E Li, Sk(0) E Lk) and functions
qi : 0] -* R and q 0] -* R (E> 0 some number) such that the mappings
:

gi : Bi = {(x, t) : x E (-E, 0], ti < qi(x)} -* Ni and gk : Bk _ {(x, t) : x E (-E, 0],


t < q (x)} -> IJ given by gi(x, t) = f(S(x)) and gk(x, t) = /(Sk(x)) are C°°-
imbeddings (Figure 7.4). (Note that q(0) = qk (0) _ +00 because Li and Lk are
nonclosed trajectories.)

FIGURE 7.4
V V
We paste the sets Ni and Nk together by means of the diffeomorphism g =
V

g o gi 1 ILL: Li -* Lk C Nk. Then the mapping gi U gk : Bi U Bk -p Ni Ug Nk is a


homeomorphism onto its image. We define the set gi U gk (Bi U Bk) as one of the V V V V

charts of the atlas of the manifold Ni Ug Nk. The flows f i and f k induce a flow
V V V V

f i Ug f k on Ni Ug Nk that is a C°°-flow in the chart gi U g (Bi U Bk).


If Li and Lk are closed trajectories,
y
then by slightly modifying the given con-
V V V

struction we can introduce on Ni Ug Nk a chart in which the flow f i Ug f k is also a


V V V
C°°-flow. We paste the sets N1,. .. , Nk together with k1, ... , k according to the
V V V V

scheme S. Since the flows f 1, ... , f k are compatible with the flows f ('til) , ... , f ('tar) ,
we have a C°°-flow on a certain manifold.
2. THE PROBLEM OF CHERRY 315

Making all the identifications described, we get a manifold N homeomorphic


to M and a C°°-flow f on N that is topologically orbitally equivalent to the flow
de
J P
J
Recall that a differentiable manifold structure is defined to be a maximal col-
lection (atlas) of compatible charts. To specify a differentiable structure of class CS
it suffices to indicate an arbitrary CS-atlas contained in the differentiable structure
[76]. Denote by D (M) the differentiable structure of the manifold M.
The C°°-atlases of the sets Nl , ... , Nq, K1,.. . , Kr are in the differentiable
structure D (N), and to these we adjoin the charts of the form gi U gj (B, U Bk).
Let h : M -* N be a homeomorphism realizing a topological orbital equiva-
lence between the flows f t 1 lvtp and f. This homeomorphism h and the differen-
tiable structure D(N) of the manifold N induce on M the differentiable structure
aef
h-1 [D(N)] D1which is not connected in an obvious way with the original
differentiable structure D(7YC7,) in general. On the manifold M with the differen-
tiable structure D1 (tY,) the flow f i = h-1 o fo h (which is ismorphic (or conjugate)
to the flow f by means of the homeomorphism h-1) is a C°°-flow topologically or-
bitally equivalent to f, and hence to f t 1 .
It is known [98] that homeomorphic two-dimensional manifolds are diffeomor-
phic. Let h°: M- NYC, be a diffeomorphism carrying the manifold M with the
differentiable structure D into iYC with the structure D(M). Then the flow
f o = ho 1 o f i o h° is a C°°-flow on the manifold t with the original differentiable
structure D (M), and f o is topologically orbitally equivalent to the flow f t I Mp .
Denote by V° the vector field of phase velocities of the flow f o . According to
Lemma 1.1, there exists a C°°-function A : M -* [0, 1] such that the vector field
A 1)° extends to a vector field V on the whole manifold M. Then the flow f o induced
by the field V is a C°°-flow and is topologically orbitally equivalent to ft. 0

1.3. The theorem of Gutierrez. The most complete result in the problem
of smoothing C°-flows on two-dimensional manifolds was obtained by Gutierrez in
1986. In [86] he proved the following theorem.
THEOREM 1.4. Let f t be a C°-flow on a compact two-dimensional manifold M.
Then there exists on M a C1-flow that is topologically equivalent to f t. Moreover,
if f t does not have nontrivial minimal sets, then it is topologically equivalent to a
C°° -flow.

Note that there are no restrictions in Theorem 1.4 on the cardinality of the set
of equilibrium states of f t.
According to Theorems 1.1 and 1.2, if a flow f t has a nontrivial minimal set,
then it is not topologically equivalent to a C''-flow (r > 2). Theorem 1.4 asserts
that in this case the flow is topologically equivalent to a C1-flow.
We shall not present the proof of Theorem 1.4.

§2. The problem of Cherry

2.1. Gray and black cells. We recall that a Cherry flow on the torus T2 is
defined to be a C''-flow f t (r > 1) satisfying the following conditions:
316 7. RELATION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES

1) f t has a single nowhere dense quasiminimal set 1 containing a nonzero finite


number of equilibrium states 01, ... , O;
2) all the equilibrium states 01,. .. , O are structurally stable saddles;
3) for three (of the four) separatrices of each saddle Oi the w- or a-limit set
coincides with 1, but one separatrix of the saddle Oi does not belong to 1, and its
limit set intersects 1 only in Oi, i = 1, ... , k.
We remark that according to Theorem 4.1 of Chapter 2, a Cherry flow on the
torus has exactly one quasiminimal set.
From the relation O E 1 and Lemma 3.4 in Chapter 2 it follows that at least two
separatrices 1 1 and l2 of each saddle O E 1 are nontrivial recurrent semitrajectories,
and one separatrix is a Bendixson extension of another separatrix. It is obvious
that both the nontrivial recurrent semitrajectories l1 and l2 belong to 1 and are
dense in 1.
DEFINITION. Let f t be a Cherry flow with quasiminimal set 1, and let f *
be the union of 1 and the separatrices of all the saddles in 1 having 1 as an w-
or a-limit set. A component of the complement T2 \ V containing a separatrix
not tending to 1 of some saddle O E 1 is called a black cell of the flow f t (such
components necessarily exist). The remaining components of T2 \ 1 are called
gray cells (such components may not exist) (Figure 7.5).

FIGURE 7.5

LEMMA 2.1. The black and gray cells of a Cherry flow on the torus are simply
connected.
PROOF. Since the torus has genus 1, the assumption that a black or gray cell
is not simply connected leads to the quasiminimal set of the Cherry flow lying in
an annular domain, which contradicts Lemma 2.4 in Chapter 2. 0

2.2. The Poincare mapping in a neighborhood of a structurally stable


saddle. We consider a C0-flow f t (r > 5) on a two-dimensional manifold PVC
with a structurally stable saddle O E M. Suppose that the a-separatrix la of O is
a Bendixson extension of the w-separatrix L,. For definiteness we assume that l a is
a Bendixson extension of lw to the right (Figure 7.6). Then in some neighborhood
U of O there are contact-free segments E1 and E2 intersecting l,, and la (in U)
only at the endpoints m1 E E1 and m2 E E2 and such that the Poincare mapping
P : E1 \ {mi} - E2 \ {m2} is defined, and P(m) - m2 as m -* m1 (Figure 7.6).
2. THE PROBLEM OF CHERRY 317

FIGURE 7.6

It can be assumed without loss of generality that E1 and E2 are disjoint and are
the images of C°°- imbeddings [0, 1] -* PVC.
Let > 0 and )2 < 0 be the eigenvalues of the saddle O. The quantity
v = -\2/\1 is called the characteristic value of O.
The following lemmas are consequences of results in Chapter 4.
LEMMA 2.2. If the characteristic value v is > 1, then there exist parametriza-
tions x : [0,1] -* E1 (x(0) = m1), y : [0,1] - E2 (y(0) = m2) such that in the
coordinates x, y the Poincare mapping P : E1 \ {mi} - E2 \ {m2} and its deriva-
tives DP, D2P, and D3P have the forms
1) y = xL + xL+1 co(x),
2) y' = vxL-1 +
x' (x),
3) y" = v(v - 1)x'2 +
4) y" = v(v _ 1) (v - 2)xv-3 + xv-2((x)
in a half-neighborhood of the point x = 0, where co(x) , e(x)l, frj(x), ((x) < const.
LEMMA 2.3. If the characteristic value v is = 1, then there exist parametriza-
tions x : [0, 1] - E1 (x(0) = m1), y : [0, 1] - E2 (y(0) = m2) such that in the
coordinates x, y the Poincare mapping P : E1 \ {mi} - E2 \ {m2} has the form
y ='b(x) [1 + c1i,b(x) In x]
in a half-neighborhood of the point x = 0, where c1 = const, and the function '(x)
satisfies the conditions
1) b(x) = x +
2) /'(x) = 1 + x1 Sp1(x),
3) 'V" (x) = xa co2 (x),
where 0 < c < 1 and (x)I < const, i = 0, 1, 2.
COROLLARY 2.1. Assume the conditions of Lemma 2.2. Then in some half-
neighborhood (0 <x <e) of zero the Schwarzian derivative of the function y = P(x)
is negative.
PROOF. By Lemma 2.2, we get that
D3P 3 D2P
SP(x) =1o DP 2 DP
_ v(v - 1)v -2) 3 [v(v_ 1)]2 3
- - 2 (v - 1) (v -I- 3) <0
1

- v -2 v
for v > 1. 0
318 7. RELATION BETWEEN SMOOTHNESS AND TOPOLOGICAL PROPERTIES

COROLLARY 2.2. Assume the conditions of Lemma 2.3. Then


var log T 'P < +00
in some half-neighborhood 0 <x <e of zero.
PROOF. According to Lemma 2.3, P(x) = b(x) [1 + c1 (x) In x] If c1 = 0,
.

then lirn DP(x) = 1 as x . 0, and the function ID2PI = D2 is bounded from


above. Therefore, the function VD log DP(x) I = Iis bounded in
some half-neighborhood of zero, and hence var log DP < +00.
If c1 0, then it follows from 1)-3) in Lemma 2.3 that lirn DP(x) = 1 as
x . 0, and lirn D2P(x) = +oo or -oo according as to whether c1 <0 or c1 > 0.
Therefore, the function D log DP(x) = D2P(x)/DP(x) has constant sign in some
half-neighborhood of zero (but is not defined at the point x = 0). Consequently,
the function log DP(x) is monotone. This gives us the required assertion. 0

2.3. Sufficient conditions for the absence of gray cells. An analytic


Cherry flow was first constructed in 1937 [81].
The flow constructed had a single black cell, and Cherry posed the problem of
the presence or absence of gray cells in this flow. He himself showed that gray cells
are absent when the saddle value of a saddle in the quasiminimal set is equal to
zero
We present a partial solution of the problem of Cherry. See [11] for generaliza-
tions.
THEOREM 2.1. Let f t be a C'+1-Cherry flow (r > 5) on the torus with quasi-
minimal set 1, and suppose that the characteristic values of all the saddles Oi E 1
(i = 1, ... , k) are > 1, and each saddle Oi has an w-separatrix lying in a black cell.
Then f t does not have gray cells.
PROOF. According to Lemma 2.3 in Chapter 2, there exists a contact-free
cycle C intersecting 1. Therefore, f t induces a Poincare mapping P : C -* C with
nonempty domain Dom(P). Again by Lemma 2.3 in Chapter 2, C is nonhomotopic
to zero on the torus. Since the torus has genus 1, this implies that any two points in
each component of the set C \ Dom(P) are Gutierrez-equivalent (see §3.6 in Chapter
2). Consequently, the mapping P : Dom(P) -* C extends in a natural way to a
Cherry transformation P: C -p C with domain Dom(P) = C (§2.6 in Chapter 5).
According to Corollaries 2.1 and 2.2, P satisfies the Yoccoz conditions (see §4.5 in
Chapter 5), and thus does not have gray cells by virtue of Corollary 4.2. This yields
the required assertion. 0

2.4. Cherry flows with gray cells. In connection with Theorem 2.1, the
question arises of whether there are C'-Cherry flows (1 < r < 4) with gray cells.
THEOREM 2.2. For any irrational number a there exists a C1-Cherry flow on
the torus with Poincare rotation number a and with any previously specified finite
number of black cells and any previously specified finite or countable set of gray
cells.
PROOF. In §3.2 of Chapter 1 we constructed a C1-Cherry flow with one black
and one gray cell by starting out from a C1-Denjoy flow with characteristic equal
to 1. Taking as a basis a C1-Denjoy flow with rotation number a and previously
2. THE PROBLEM OF CHERRY 319

specified finite or countable characteristic, we can use the arguments in §3.2 of


Chapter 1 to construct the required C1-Cherry flow with an arbitrary previously
specified finite number of black cells.
Bibliography
1. A. A. Andronov and E. A. Leontovich, On the theory of changes in the qualitative structure
of a partition of the plane into trajectories, Dokl. Akad. Nauk SSSR 21 (1938), 427-431.
(Russian)
2. , Dynamical systems of the first degree of structural instability on the plane, Mat. Sb.
68 (110) (1965), 328-372. (Russian)
3. A. A. Andronov, E. A. Leontovich, I. I. Gordon, and A. G. Maier, Qualitative theory of
second-order dynamical systems, "Nauka", Moscow, 1966; English transl., Halstead Press,
NY-Toronto, 1973.
4. , Theory of bifurcations of dynamical systems on the plane, "Nauka", Moscow, 1967;
English transl., Halstead Press, NY-Toronto, 1973.
5. A. A. Andronov and L. S. Pontryagin, Systemes grossiers, Dokl. Akad. Nauk SSSR 14 (1937),
247-250.
6. D. V. Anosov, Structurally stable systems, Trudy Mat. Inst. Steklov. 169 (1985), 59-93;
English transl. in Proc. Steklov Inst. Math. 1986, no. 4.
7. S. Kh. Aranson, On the nonexistence of nonclosed Poisson-stable semitrajectories and tra-
jectories doubly asymptotic to a double limit cycle for dynamical systems of the first degree
of structural instability on two-dimensional orientable manifolds, Mat. Sb. 76 (118) (1968),
214-230; English transl. in Math. USSR Sb. 5 (1968).
8. , Trajectories on two-dimensional nonorientable manifolds, Math. Sb. 80 (122)
(1969), 314-333; English transl. in Math. USSR Sb. 9 (1969).
9. , On topological equivalence of foliations with singularities and of homeomorphisms
with invariant foliations on two-dimensional manifolds, Uspekhi Mat. Nauk 41 (1986), no. 3
(249), 167-168; English transl. in Russian Math. Surveys 41 (1986).
10. , On the topological structure of Cherry flows on the torus, Funktsional. Anal. i
Prilozhen. 20 (1986), no. 1, 62-63; English transl. in Functional Anal. Appl. 20 (1986).
11. , On the topological structure of quasiminimal sets of Cherry flows on the torus,
Methods of the Qualitative Theory of Differential Equations (E. A. Leontovich-Andronova,
ed.), Mezhvuz. Temat. Sb. Nauchn. Tr., Gor'kov. Gos. Univ., Gorki, 1985, pp. 3-18; English
transl. in Selecta Math. Soviet. 9 (1990).
12. , Generic properties of structurally unstable vector fields on closed surfaces, Meth-

ods of the Qualitative Theory of Differential Equations (E. A. Leontovich-Andronova, ed. ),


Mezhvuz. Temat. Sb. Nauchn. Tr., Gor'kov. Gos. Univ., Gorki, 1986, pp. 4-18; English transl.
in Selecta Math. Soviet. 10 (1991).
13. , On the nondenseness of fields of finite degree of structural instability in the space of
structurally unstable vector fields on closed two-dimensional manifolds, Uspekhi Mat. Nauk
43 (1988), no. 1 (259), 191-192; English transl. in Russian Math. Surveys 43 (1988).
14. , On the problem of gray cells, Mat. Zametki 47 (1990), no. 1, 3-14; English transl.
in Math. Notes 47 (1990).
15. , Topological invariants of vector fields in the disk on the plane with limit sets of
Cantor type, Uspekhi Mat. Nauk 45 (1990), no. 4 (274), 139-140; English transl. in Russian
Math. Surveys 45 (1990).
16. S. Kh. Aranson and V. Z. Grines, Certain invariant dynamical systems on two-dimensional
manifolds (necessary and sufficient conditions for topological equivalence of transitive sys-
tems), Mat. Sb. 90 (132) (1973), 372-402; English transl. in Math. USSR Sb. 19 (1973).

321
322 BIBLIOGRAPHY

17. , On topological invariants of minimal sets of dynamical systems on two-dimensional


manifolds, Qualitative Methods of the Theory of Differential Equations and Their Applica-
tions (E. A. Leontovich-Andronova, ed.), Uchen. Zap. Gor'kov. Gos. Univ. vyp. 187, Gor'kov.
Gos. Univ., Gorki, 1973, pp. 3-28; English transl in Selecta Math. Soviet. 5 (1986).
18. , Representation of minimal sets of flows on two-dimensional manifolds by geodesic
curves, Izv. Akad. Nauk SSSR Ser. Mat. 42 (1978), 104-129; English transl. in Math. USSR
Izv. 12 (1978).
19. , Topological classification of flows on closed two-dimensional manifolds, Uspekhi Mat.
Nauk 41 (1986), no. 1 (247), 149-169; English transl. in Russian Math. Surveys 41 (1986).
20. S. Kh. Aranson and E. V. Zhuzhoma, On topological classification of singular dynamical
systems on the torus, Izv. Vyssh. Uchebn. Zaved. Mat. 1976, no. 5 (168), 104-107; English
transl. in Soviet Math. (Iz. VUZ) 1976.
21. , The classification of transitive foliations on the sphere with four singularities of
"thorn" type, Methods of the Qualitative Theory of Differential Equations (E. A. Leontovich-
Andronova, ed.), Mezhvuz. Temat. Sb. Nauchn. Tr., Gor'kov. Gos. Univ., Gorki, 1984, pp. 3-
10; English transl. in Selecta Math. Soviet. 9 (1990).
22. , On the interrelation between topological and smoothness properties of transforma-
tions of the circle without periodic points and with finitely many critical points, Izv. Vyssh.
Uchebn. Zaved. Mat. 1985, no. 8 (279), 64-67; English transl. in Soviet Math. (Iz. VUZ)
1985.
23. , A C1-Cherry flow with gray cells, Methods of the Qualitative Theory of Differential
Equations and Bifurcation Theory (E. A. Leontovich-Andronova, ed. ), Mezhvuz. Temat. Sb.
Nauchn. Tr., Gor'kov. Gos. Univ., Gorki, 1988, pp. 5-10. (Russian)
24. , The C'' closing lemma on surfaces, Uspekhi Mat. Nauk 43 (1988), no. 5 (263),
173-174; English transl. in Russian Math. Surveys 43 (1988).
25. S. Kh. Aranson, E. V. Zhuzhoma, and M. I. Malkin, On the interrelation between smooth-
ness and topological properties of transformations of the circle (theorems of Denjoy type),
Manuscript No. 3052-84, deposited at VINITI, Gor'kov. Gos. Univ., Gorki, 1984. (Russian)
26. V. I. Arnol'd, Ordinary differential equations, "Nauka", Moscow, 1971; English transl., MIT
Press, Cambridge, MA-London, 1973.
27. , Supplementary chapters in the theory of ordinary differential equations, "Nauka",
Moscow, 1978; English transl., Geometric methods in the theory of ordinary differential
equations, Springer-Verlag, Berlin-New York, 1982.
28. , Small denominators. I. Mappings of the circle onto itself, Izv. Akad. Nauk SSSR
Ser. Mat. 25 (1961), 21-86; English transl. in Amer. Math. Soc. Transl. (2) 46 (1965).
29. V. S. Afraimovich and L. P. Shil'nikov, On singular trajectories of dynamical systems, Us-
pekhi Mat. Nauk 27 (1972), no. 3 (165), 189-190. (Russian)
30. N. N. Bautin and E. A. Leontovich, Methods and rules in the qualitative investigation of
dynamical systems on the plane, "Nauka", Moscow, 1976. (Russian)
31. G. R. Belitskii, Normal forms, invariants, and local mappings, "Naukova Dumka", Kiev,
1979. (Russian)
32. , Functional moduli of diffeomorphisms of the circle, Ukrain. Mat. Zh. 38 (1986),

369-370; English transl. in Ukrainian Math. J. 38 (1986).


33. , Smooth classification of one-dimensional diffeomorphisms with hyperbolic fixed points,
Sibirsk. Mat. Zh. 27 (1986), no. 6, 21-24; English transl. in Siberian Math. J. 27 (1986).
34. , Smooth equivalence of germs of vector fields with a single zero eigenvalue or a pair of
purely imaginary eigenvalues, Funktsional. Anal. i Prilozhen. 20 (1986), no. 4, 1-8; English
transl. in Functional Anal. Appl. 20 (1986).
35. , Finite determinacy of germs of C-diffeomorphisms, Teor. Funktsii Funktsional.

Anal. i Prilozhen. 47 (1989), 31-39; English transl. in J. Soviet Math. 48 (1990), no. 6.
36. I. Bendixson, Sur les courbes definies par les equations differentielles, Acta Math. 24 (1901),
1-88.
37. lvI. 1VI. Brin, On inclusion of a diffeomorphism in a flow, Izv. Vyssh. Uchebn. Zaved. Mat.
1972, no. 8 (123), 19-25. (Russian)
38. A. D. Bryuno, Analytic form of differential equations, Trudy Moskov. Mat. Obshch. 25
(1971), 119-262; English transl. in Trans. Moscow Math. Soc. 25 (1973).
39. I. A. Bykov, Smooth classification of flows on the circle, Teor. Funktsii Funktsional. Anal. i
Prilozhen. 48 (1990), 24-28; English transl. in J. Soviet Math. 49 (1990), no. 2.
BIBLIOGRAPHY 323

40. E. B. Vinberg and O. V. Shvartsman, Riemann surfaces, Itogi Nauki i Tekhniki: Algebra,
Topologiya, Geometriya, vol. 16, VINITI, Moscow, 1978, pp. 199-245; English transl. in J.
Soviet Math. 14 (1980), no. 1.
41. S. M. Voronin, Analytic classification of conformal mappings (C, 0) --> (C, 0) with linear
part the identity, Funktsional. Anal. i Prilozhen. 15 (1981), no. 1, 1-17; English transl. in
Functional Anal. Appl. 15 (1981).
42. M. Golubitsky and V. Guillemin, Stable mappings and their singularities, Springer-Verlag,
NY-Heidelberg, 1973.
43. B. A. Dubrovin, S. P. Novikov, and A. T. Fomenko, Modern Geometry. Methods and Ap-
plications, "Nauka", Moscow, 1979; English transl., Parts I, II, Springer-Verlag, Berlin-NY,
1984, 1985.
44. H. Seifert and W. Threlfall, Lehrbuch der Topologie, Teubner, Leipzig, 1934.
45. C. L. Siegel, Vorlesungen fiber Himmelsmechanik, Springer-Verlag, Berlin, 1956.
46. A. B. Katok, Invariant measures of flows on orientable surfaces, Dokl. Akad. Nauk SSSR
211 (1973), 775-778; English transl. in Soviet Math. Doklady 14 (1973).
47. I. P. Kornfel'd [Cornfeld], Ya. G. Sinai, and S. V. Fomin, Ergodic theory, "Nauka", Moscow,
1980; English transl., Springer-Verlag, Berlin-Heidelberg-New York, 1982.
48. E. A. Leontovich, On the creation of limit cycles from separatrices, Candidate's dissertation,
Gor'kov. Gos. Univ., Gorki, 1946. (Russian)
49. , On the creation of limit cycles from separatrices, Dokl. Akad. Nauk SSSR 78 (1951),
641-644. (Russian)
50. E. A. Leontovich and A. G. Maier, On trajectories determining the qualitative structure of
the partition of the sphere into trajectories, Dokl. Akad. Nauk SSSR 14 (1937), 251-257.
(Russian)
51. , On a scheme determining the topological structure of the partition into trajectories,
Dokl. Akad. Nauk SSSR 103 (1955), 557-560. (Russian)
52. A. M. Lyapunov, The general problem of stability of motion, 2nd ed., Mi ntz, Leningrad,
1935; reprint of French transl., Probleme general de la stabilite du mouvement, Princeton
Univ. Press, Princeton, NJ, 1947.
53. W. Magnus, A. Karrass, and D. Solitar, Combinatorial group theory, Interscience, New York-
London-Sydney, 1966.
54. A. G. Maier, Structurally stable transformations of the circle into the circle, Uchen. Zap.
Gor'kov. Gos. Univ. 1939, no. 12, 215-229. (Russian)
55. , On trajectories on orientable surfaces, Mat. Sb. 12 (54) (1943), 71-84. (Russian)
56. M. I. Malkin, Periodic orbits, entropy, and rotation sets of continuous mappings of the circle,
Ukrain. Mat. Zh. 35 (1983), 327-332; English transl. in Ukrainian Math. J. 35 (1983).
57. , Methods of symbolic dynamics in the theory of one-dimensional discontinuous map-
pings, Candidate's dissertation, Gor'kov. Gos. Univ., Gorki, 1985. (Russian)
58. , Rotation intervals and the dynamics of mappings of Lorenz type, Methods of the

Qualitative Theory of Differential Equations (E. A. Leontovich-Andronova, ed.), Mezhvuz.


Temat. Sb. Nauchn. Tr., Gor'kov. Gos. Univ., Gorki, 1986, pp. 122-139. (Russian)
59. B. Malgrange, Ideals of differentiable functions, Tata Inst., Bombay, Oxford Univ. Press,
London, 1967.
60. V. V. Nemytskii, Topological questions in the theory of dynamical systems, Uspekhi Mat.
Nauk 4 (1949), no. 6 (34), 91-153. (Russian)
61. V. V. Nemytskii and V. V. Stepanov, Qualitative theory of differential equations, GITTL,
Moscow-Leningrad, 1947; English transl., Princeton Univ. Press, Princeton, N.J., 1960.
62. Z. Nitecki, Introduction to differential dynamics, MIT Press, Cambridge, MA, 1971.
63. I. M. Ovsyannikov and L. P. Shil'nikov, On systems with a saddle-focus homoclinic curve,
Mat. Sb. 130 (172) (1986), 552-570; English transl. in Math. USSR Sb. 58 (1987).
64. J. Palis and W. de Melo, Geometric theory of dynamical systems, Springer-Verlag, Berlin-
Heidelberg-NY, 1982.
65. V. A. Pliss, On structural stability of differential equations on the torus, Vestnik Leningrad.
Univ. 1960, no. 13 (Ser. Mat. Mekh. Astr. vyp. 3), 15-23. (Russian)
66. L. S. Pontryagin, Smooth manifolds and their use in homotopy theory, 2nd ed, "Nauka",
Moscow, 1976; English transl. of 1st ed. (Trudy Mat. Inst. Steklov. 45 (1955)) in Amer.
Math. Soc. Transl. (2) 11 (1959).
67. lvi. M. Postnikov, Introduction to Morse theory, "Nauka", 1Vloscow, 1971. (Russian)
324 BIBLIOGRAPHY

68. H. Poincare, Sur les courbes defines par les equations differentielles, Euvres, vol. I, Gauthier-
Villars, Paris, 1928, pp. 3-84, 90-161, 167-221.
69. C. Pugh, The closing lemma, Amer. J. Math. 89 (1967), 966-1009.
70. L. E. Reizin' L. E. Reiziis], v Topological classification of dynamical systems without rest
points on the torus, Latv. Mat. Ezhegodnik 1969, no. 5, 113-121. (Russian)
71. V. A. Rokhlin and D. B. Fuks, Beginner's course in topology. Geometry chapters, "Nauka",
Moscow, 1977; English transl., Springer-Verlag, Berlin-NY, 1984.
72. V. S. Samovol, Equivalence of systems of differential equations in a neighborhood of a singular
point, Trudy Moskov. Mat. Obshch. 44 (1982), 213-224; English transl. in Trans. Moscow
Math. Soc. 1983, no. 2.
73. I. Tamura, Topology of foliations: An introduction, Iwanami Shoten, Tokyo, 1976; English
transl., Amer. Math. Soc., Providence, RI, 1992.
74. Phillip Hartman, Ordinary differential equations, Wiley, NY, 1964.
75. A. Ya. Khinchin, Continued fractions, 4th ed., "Nauka", Moscow, 1978; English transl. of
3rd ed., Univ. Chicago Press, Chicago, IL, 1964.
76. M. Hirsch, Differential topology, Springer-Verlag, NY-Heidelberg, 1976.
77. H. Zieschang, E. Vogt, and H.-D. Coldewey, Surfaces and discontinuous groups, Lecture
Notes in Math., vol. 835, Springer-Verlag, NY-Heidelberg, 1980.
78. G. Shimura, Introduction to the arithmetic theory of automorphic functions, Princeton Univ.
Press, Princeton, NJ, 1971.
79. A. G. dos Anjos, Polynomial vector fields on the torus, Bol. Soc. Brasil. Mat. (N.S.) 17
(1986), no. 2, 1-22.
80. T. M. Cherry, Topological properties of the solutions of ordinary differential equations, Amer.
J. Math. 59 (1937), 957-982.
81. , Analytic quasi-periodic curves of discontinuous type on a torus, Proc. London Math.
Soc. (2) 44 (1937), 175-215.
82. A. Denjoy, Sur les courbes definies par les equation differentielles a la surface du tore, J.
Math. Pures Appl. (9) 11 (1932), 333-375.
83. C. J. Gardiner, The structure of flows exhibiting nontrivial recurrence on two-dimensional
manifolds, J. Differential Equations 57 (1985), 138-158.
84. W. Gottschalk and G. A. Hedlund, Topological dynamics, Amer. Math. Soc., Providence, RI,
1955.
85. C. Gutierrez, Structural stability for flows on the torus with a cross-cap, Trans. Amer. Math.
Soc. 241 (1978), 311-320.
86. , Smoothing continuous flows on two-manifolds and recurrences, Ergodic Theory Dy-
namical Systems 6 (1986), 17-44.
87. C. R. Hall, A C-Denjoy counterexample, Ergodic Theory Dynamical Systems 1 (1981),
261-272.
88. G. A. Hedlund, Two-dimensional manifolds and transitivity, Ann. Math. 37 (1936), 534-542.
89. M.-R. Herman, Sur la conjugaison differentiable des diffeomorphismes du cercle a des rota-
tions (1979), Inst. Hautes Etudes Sci. Publ. Math. No. 49, 5-233.
90. H. Kneser, Regulc re Kurvenscharen auf Ringflachen, Math. Ann. 91 (1923), 135-154.
91. G. Levitt, Pantalons et feuilletages des surfaces, Topology 21 (1982), 9-23.
92. , Foliations and laminations on hyperbolic surfaces, Topology 22 (1983), 119-135.
93. , Feuilletages des surfaces, Dissertation, Paris, 1983.
94. , Flots topologiquement transitifs sur les surfaces compactes sans bond: contreexemples
a une conjecture de Katok, Ergodic Theory Dynamical Systems 3 (1983), 241-249.
95. N. G. Markley, The Poincare-Bendixson theorem for the Klein bottle, Trans. Amer. Math.
Soc. 135 (1969), 159-165.
96. , On the number of recurrent orbit closures, Proc. Amer. Math. Soc. 25 (1970), 413-
416.
97. , Homeomorphisms of the circle without periodic points, J. London Math. Soc. 20
(1970), 688-698.
98. J. R. Munkres, Obstructions to the smoothing of piecewise differentiable homeomorphisms,
Ann. of Math. (2) 70 (1960), 521-554.
99. D. Neumann, Smoothing continuous flows on 2-manifolds, J. Differential Equations 28
(1978), 327-344.
BIBLIOGRAPHY 325

100. D. Neumann and T. O'Brien, Global structure of continuous flows on 2-manifolds, J. Differ-
ential Equations 22 (1976), 89-110.
101. J. Nielsen, Uber topologische Abbildungen geschlossener Flc chen, Abh. Math. Sem. Univ.
Hamburg 3 (1924), no. 1, 246-260.
102. , Untersuchungen zur Topologie der geschlossenen zweiseitigen Flc chen. I, Acta Math.
50 (1927), 189-358; II, Acta Math. 53 (1929), 1-76; III, Acta Math. 58 (1932), 87-167.
103. M. M. Peixoto, Structural stability on two-dimensional manifolds, Topology 1 (1962), 101-
120; Topology 2 (1963), 179-180.
104. , On the classification of flows on 2-manifolds, Proc. Sympos. Dynamical Systems
(Univ. Bahia, Salvador, Brasil, 1971), Academic Press, NY, 1973, pp. 389-419.
105. , On structural stability, Ann. of Math. (2) 69 (1959), 199-222.
106. H. Rosenberg, Labyrinths in the disc and surfaces, Ann. of Math. (2) 117 (1983), 1-33.
107. A. J. Schwartz, A generalization of the Poincare-Bendixson theorem to closed two-dimensional
manifolds, Amer. J. Math. 85 (1963), 453-458.
108. D. Stowe, Linearization in two dimensions, J. Differential Equations 63 (1968), 183-226.
109. F. Takens, Normal forms for certain singularities of vector fields, Ann. Inst. Fourier 37
(1973), 163-165.
110. H. Whitney, Regular families of curves, Ann. of Math. (2) 34 (1933), 244-270.
111. , On regular families of curves, Bull. Amer. Math. Soc. 47 (1941), 145-147.
112. J. Ch. Yoccoz, Il n'y a pas de counter exemple de Denjoy analytique, C.R. Acad. Sci. Paris
Ser. I Math. 298 (1984), no. 7, 141-144.
113. I. Bronstein and A. Kopanskii, Smooth invariant manifolds and normal forms, World Scien-
tific Series on Nonlinear Science Ser. A, Vol. 7, World Scientific, Singapore, 1994.
Selected Titles in This Series
(Continued from the front of this publication)

117 Boris Zilber, Uncountably categorical theories, 1993


116 G. M. Fel'dman, Arithmetic of probability distributions, and characterization problems
on abelian groups, 1993
115 Nikolai V. Ivanov, Subgroups of Teichmuller modular groups, 1992
114 Seizo Ito, Diffusion equations, 1992
113 Michail Zhitomirskii, Typical singularities of differential 1-forms and Pfaffian equations,
1992
112 S. A. Lomov, Introduction to the general theory of singular perturbations, 1992
111 Simon Gindikin, hbe domains and the Cauchy problem, 1992
110 B. V. Shabat, Introduction to complex analysis Part II. Functions of several variables,
1992
109 Isao Miyadera, Nonlinear semigroups, 1992
108 Takeo Yokonuma, Tensor spaces and exterior algebra, 1992
107 B. M. Makarov, M. G. Goluzina, A. A. Lodkin, and A. N. Podkorytov, Selected problems
in real analysis, 1992
106 G.-C. Wen, Conformal mappings and boundary value problems, 1992
105 D. R. Yafaev, Mathematical scattering theory: General theory, 1992
104 R. L. Dobrushin, R. and S. Shlosman, Wulff construction: A global shape from
local interaction, 1992
103 A. K. Tsikh, Multidimensional residues and their applications, 1992
102 A. M. Il'in, Matching of asymptotic expansions of solutions of boundary value problems,
1992
101 Zhang Zhi-fen, Ding Tong-ren, Huang Wen-zao, and Dong Zhen-xi, Qualitative theory of
differential equations, 1992
100 V. L. Popov, Groups, generators, syzygies, and orbits in invariant theory, 1992
99 Norio Shimakura, Partial differential operators of elliptic type, 1992
98 V. A. Vassiliev, Complements of discriminants of smooth maps: Topology and
applications, 1992 (revised edition, 1994)
97 Itiro Tamura, Topology of foliations: An introduction, 1992
96 A. I. Markushevich, Introduction to the classical theory of Abelian functions, 1992
95 Guangchang Dong, Nonlinear partial differential equations of second order, 1991
94 Yu. S. Il'yashenko, Finiteness theorems for limit cycles, 1991
93 A. T. Fomenko and A. A. Tuzhilin, Elements of the geometry and topology of minimal
surfaces in three-dimensional space, 1991
92 E. M. Nikishin and V. N. Sorokin, Rational approximations and orthogonality, 1991
91 Mamoru Mimura and Hirosi Toda, Topology of Lie groups, I and II, 1991
90 S. L. Sobolev, Some applications of functional analysis in mathematical physics, third
edition, 1991
89 Valeril V. Kozlov and Dmitril V. Treshchev, Billiards: A genetic introduction to the
dynamics of systems with impacts, 1991
88 A. G. Khovanskil, Fewnomials, 1991
87 Aleksandr Robertovich Kemer, Ideals of identities of associative algebras, 1991
86 V. M. Kadets and M. I. Kadets, Rearrangements of series in Banach spaces, 1991
85 Mikio Ise and Masaru Takeuchi, Lie groups I, II, 1991
84 Dao Trong Thi and A. T. Fomenko, Minimal surfaces, stratified multivarifolds, and the
Plateau problem, 1991

(See the AMS catalog for earlier titles)

You might also like