Iterative Methods For Fixed Point Problems in Hilbert Spaces (PDFDrive)

Lecture Notes in Mathematics 2057
Editors:
J.-M. Morel, Cachan
B. Teissier, Paris
For further volumes:

http://www.springer.com/series/304
•
Andrzej Cegielski
Iterative Methods
for Fixed Point Problems
in Hilbert Spaces
123
Andrzej Cegielski
University of Zielona Góra
Faculty of Mathematics,
Computer Science and Econometrics
Zielona Góra, Poland
ISBN 978-3-642-30900-7 ISBN 978-3-642-30901-4 (eBook)

DOI 10.1007/978-3-642-30901-4
Springer Heidelberg New York Dordrecht London
Lecture Notes in Mathematics ISSN print edition: 0075-8434

ISSN electronic edition: 1617-9692
Library of Congress Control Number: 2012945521
Mathematics Subject Classification (2010): 47-02, 49-02, 65-02, 90-02, 47H09, 47J25, 37C25,
65F10, 65K15, 90C25
c Springer-Verlag Berlin Heidelberg 2012

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)

To my family:
Elżbieta, Joanna, Gustaw,
Szymon and Katarzyna
•
Motto:
If you want to find the source, you have to go
up, against the current
[John Paul II]

•
Preface
In this monograph we deal with iteration methods for finding fixed points (if they
exist) of nonexpansive operators defined on a Hilbert space, i.e., operators T having
the property
d.T x; T y/ cd.x; y/ for all x; y 2 X (1)
and for some constant c 2 Œ0; 1. The origin of these methods dates back to 1920,
when Stefan Banach (1892–1945) formulated his famous contraction mapping
principle. Banach proved that if T W X ! X is a contraction (an operator satisfying
(1) with c < 1) defined on a complete metric space, then T has a unique fixed point
x , i.e., a point for which T x D x . Furthermore, for any x 2 X a sequence
fT k xg1
kD0 converges geometrically to x . Many practical problems can be reduced
to finding a fixed point of a nonexpansive operator or a common fixed point of a
family of nonexpansive operators. A simple example is a system of linear equations
Ax D b. Any solution of this system can be identified with a common fixed point of
(nonexpansive) operators of orthogonal projections onto hyperplanes corresponding
to particular equations of the system.
Under some additional conditions a nonexpansive operator has a fixed point (see,
e.g., the Browder–Göhde–Kirk fixed point theorem), but the Banach theorem does
not guarantee the convergence of the sequence fT k xg1 kD0 . Therefore, it is of great
interest to develop methods for finding fixed points of nonexpansive operators.
The first iterative method for solving a linear system was proposed in 1937 by
a Polish mathematician, Stefan Kaczmarz (1890–1945), in a very short paper (3
pages) “Angenäherte Auflösung von Systemen linearer Gleichungen” (“Approxi-
mate solution of systems of linear equations”) published in Bulletin International
de l’Académie Polonaise des Sciences et des Lettres. A year later, an Italian math-
ematician, Gianfranco Cimmino (1908–1989), proposed another iterative method
for a linear system. He published his result in a paper “Calcolo approssimato per
le soluzioni dei sistemi di equazioni lineari” (“Approximate computation of the
solutions of linear systems”) in a La Ricerca Scientifica. As opposed to earlier
methods for solving systems of linear equations, both methods are motivated rather
by geometrical operations than algebraic ones. The main operation in both methods
ix
x Preface
is a cyclic (in the Kaczmarz method) or simultaneous (in the Cimmino method)
orthogonal projection onto hyperplanes described by particular equations of the
linear system. The results of Kaczmarz and Cimmino were disregarded for several
decades except few references. A great interest in these results started in the 1970s,
when it suddenly turned out that the Kaczmarz and Cimmmino methods can be
efficiently applied in computed tomography (CT), because the mathematical model
of CT can be reduced to a solution of large systems of linear equations with sparse
and unstructured matrices, and the methods behave well for such systems. At the
end of the twentieth century the interest in the Kaczmarz and Cimmino results was
still increasing. Both results have become fundamental to modern iterative methods
for fixed point problems for nonexpansive operators.
The third result which influenced the development of this area concerns alterna-
tive projections onto two subspaces of a Hilbert space. The result was published
in 1950 by John von Neumann in a paper “Functional Operators—Vol. II. The
Geometry of Orthogonal Spaces” in Annals of Mathematical Studies. Von Neumann
proved that a sequence generated by his method converges to the projection of
the starting point onto the intersection of the subspaces. The results of Kaczmarz,
Cimmino, and von Neumann have been generalized several times in the last decades.
Today it is known that the convergence holds for an essentially wider class of
operators than orthogonal projections onto hyperplanes, e.g., for nonexpansive
operators or for quasi-nonexpansive operators, satisfying some additional condi-
tions. All three methods belong today to classical iterative methods for finding
fixed points of nonexpansive operators defined on a Hilbert space. These methods
served as the basis for several methods, e.g., for: the Landweber method, projected
Landweber method, Douglas–Rachford method, sequential projection methods,
methods of cyclic and simultaneous subgradient projections, Dos Santos method
of extrapolated simultaneous subgradient projections, reflection-projection method,
surrogate projection method, and many others. Irrespective of their theoretical value,
they have found application in many areas of mathematics, physics, and technology.
The most spectacular application of the methods is an intensity-modulated radiation
therapy (IMRT).
Iterative methods for finding fixed points of nonexpansive operators in Hilbert
spaces have been described in many publications. In this monograph we try to
present the methods in a consolidated way. We introduce some classes of operators,
give their properties, define iterative methods generated by operators from these
classes, and present general convergence theorems. On this basis we present the
conditions under which particular methods converge. A large part of the results
presented in this monograph can be found in various forms in the literature. We
tried, however, to show that the convergence of a big class of iteration methods
follows from general properties of some classes of operators and from some
general convergence theorems, in particular from Opial’s theorem or from its
modifications. This theorem was presented in 1967 by a Polish mathematician,
Zdzisław Opial (1930–1974), in a paper “Weak convergence of the sequence of
successive approximations for nonexpansive mappings” published in the Bulletin of
the American Mathematical Society.
Preface xi
In this monograph we work with operators defined on a real Hilbert space,

although a part of the results presented herein holds for wider classes of spaces.
The monograph is divided into five chapters. In Chap. 1, we recall basic definitions
and facts from linear algebra, functional analysis, and convex analysis which we
apply in the further part of the monograph. In Chap. 2, we introduce some classes of
algorithmic operators (i.e., operators which generate some algorithms or iterative
methods): nonexpansive operators, quasi-nonexpansive operators, relaxed quasi-
nonexpansive operators, cutter operators, firmly nonexpansive operators, metric
projection, relaxed firmly nonexpansive operators, averaged operators, strongly
nonexpansive operators, and generalized relaxations of algorithmic operators. Then
we present the properties of these classes, in particular the relationships among
these classes and their closedness with respect to some algebraic operations on the
operators from these classes. In Chap. 3, we analyze the convergence properties of
the sequences generated by the operators introduced in Chap. 2. Opial’s theorem,
its generalizations, and modifications for sequences generated by a subclass of
the class of quasi-nonexpansive operators play the key role here. In Chap. 4, we
apply the properties of classes of operators presented in Chap. 2 to constructions of
operators used in many iterative methods for fixed point problems. In this chapter
we also give the properties of the following: the alternating projection, simultaneous
projection, cyclic projection, Landweber operator, projected Landweber operator,
and some generalizations and extrapolations of these operators. In Chap. 5, we apply
the results presented in the previous chapters in order to show the convergence of
sequences generated by many iterative methods for fixed point problems, some
of which are known in the literature, but several are new. The notions and facts
presented in the book are illustrated with 61 figures. Each chapter is followed by
several exercises.
Many persons have looked through successive versions of the monograph.
I am deeply grateful for their helpful remarks. In particular, I would like to
express my thanks to Prof. Simeon Reich from the Technion (Israel), Prof. Yair
Censor from the University of Haifa (Israel), Prof. Diethard Pallaschke from the
University of Karlsruhe (Germany), and Prof. Heinz Bauschke from the University
of British Columbia in Okanagan (Canada). Their valuable remarks have contributed
to substantial improvements in successive versions of the monograph and have
consolidated me in my aim to give the monograph its final shape. I am also very
grateful to my colleagues from the University of Zielona Góra, who looked through
the final version of the monograph and also made some useful remarks: Prof. Michał
Kisielewicz, Prof. Krzysztof Przesławski, and Prof. Jerzy Motyl. I would like to
express my thanks to my Ph.D. student, Rafał Zalas, for his help in the preparation
of figures which illustrate the notions and facts presented herein and to Danuta
Michalak for the technical composition of the monograph. Last but not least, I would
like to express my deep gratitude to my wife, Elżbieta, for her understanding and
assistance during the preparation of the monograph.
Zielona Góra, Poland Andrzej Cegielski

April 2012
•
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.1.1 Hilbert Space .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.1.2 Notations and Basic Facts . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5
1.2 Metric Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 16
1.2.1 Existence and Uniqueness of the Metric Projection .. . . . . . . 17
1.2.2 Characterization of the Metric Projection.. . . . . . . . . . . . . . . . . . 19
1.2.3 First Applications of the Characterization Theorem . . . . . . . 20
1.3 Convex Optimization Problems .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 23
1.3.1 Convex Minimization Problems .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 24
1.3.2 Variational Inequality . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 27
1.3.3 Common Fixed Point Problem . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 27
1.3.4 Convex Feasibility Problem . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 27
1.3.5 Linear Feasibility Problem . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 30
1.3.6 General Convex Feasibility Problem .. . .. . . . . . . . . . . . . . . . . . . . 33
1.3.7 Split Feasibility Problem . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 34
1.3.8 Linear Split Feasibility Problem . . . . . . . .. . . . . . . . . . . . . . . . . . . . 35
1.3.9 Multiple-Sets Split Feasibility Problem.. . . . . . . . . . . . . . . . . . . . 35
1.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 36
2 Algorithmic Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 39
2.1 Basic Definitions and Properties . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 40
2.1.1 Nonexpansive Operators .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 41
2.1.2 Quasi-nonexpansive Operators . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 45
2.1.3 Cutters and Strongly Quasi-nonexpansive
Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 53
2.2 Firmly Nonexpansive Operators . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 65
2.2.1 Basic Properties of Firmly Nonexpansive Operators .. . . . . . 66
2.2.2 Relationships Between Firmly Nonexpansive
and Nonexpansive Operators . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 70
2.2.3 Further Properties of the Metric Projection .. . . . . . . . . . . . . . . . 76
xiii
xiv Contents
2.2.4 Metric Projection onto a Closed Subspace.. . . . . . . . . . . . . . . . . 80

2.2.5 Metric Projection onto a Closed Affine Subspace .. . . . . . . . . 82
2.2.6 Properties of Relaxed Firmly Nonexpansive Operators .. . . 84
2.2.7 Fixed Points of Firmly Nonexpansive Operators .. . . . . . . . . . 90
2.3 Strongly Nonexpansive Operators . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 91
2.4 Generalized Relaxations of Algorithmic Operators .. . . . . . . . . . . . . . . . . 96
2.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 102
3 Convergence of Iterative Methods . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 105
3.1 Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 105
3.2 Properties of the Weak Convergence . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 106
3.3 Properties of Fejér Monotone Sequences .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 108
3.4 Asymptotically Regular Operators .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 111
3.5 Opial’s Theorem and Its Consequences . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 114
3.6 Generalization of Opial’s Theorem . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 116
3.7 Opial-Type Theorems for Cutters . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 118
3.8 Strong Convergence of Fejér Monotone Sequences . . . . . . . . . . . . . . . . . 123
3.9 Relationships Among Algorithmic Operators . . .. . . . . . . . . . . . . . . . . . . . 126
3.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 127
4 Algorithmic Projection Operators . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 129
4.1 Examples of Metric Projections.. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 129
4.1.1 Metric Projection onto a Hyperplane . . .. . . . . . . . . . . . . . . . . . . . 129
4.1.2 Metric Projection onto a Finite Dimensional
Affine Subspace .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 132
4.1.3 Metric Projection onto a Half-Space .. . .. . . . . . . . . . . . . . . . . . . . 133
4.1.4 Metric Projection onto a Band. . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 133
4.1.5 Metric Projection onto the Orthant .. . . . .. . . . . . . . . . . . . . . . . . . . 134
4.1.6 Metric Projection onto Box Constraints . . . . . . . . . . . . . . . . . . . . 135
4.1.7 Metric Projection onto a Ball . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 137
4.1.8 Metric Projection onto an Ellipsoid .. . . .. . . . . . . . . . . . . . . . . . . . 137
4.1.9 Metric Projection onto an Ice Cream Cone . . . . . . . . . . . . . . . . . 140
4.2 Cutters .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 142
4.2.1 Characterization of Cutters . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 142
4.2.2 Cutters with Subsets of Fixed Points Being
Affine Subspaces .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 143
4.2.3 Subgradient Projection.. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 144
4.3 Alternating Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 147
4.3.1 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 147
4.3.2 Fixed Points of the Alternating Projection .. . . . . . . . . . . . . . . . . 148
4.3.3 Alternating Projection for a Closed Affine Subspace . . . . . . 151
4.3.4 Generalized Relaxation of the Alternating Projection.. . . . . 152
4.3.5 Averaged Alternating Reflection . . . . . . .. . . . . . . . . . . . . . . . . . . . 160
4.4 Simultaneous Projection.. . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 162
4.4.1 Simultaneous Projection as an Alternating
Projection in a Product Space . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 163
Contents xv
4.4.2
Properties of the Simultaneous Projection .. . . . . . . . . . . . . . . . . 165
4.4.3
Simultaneous Projection for a System of Linear
Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 168
4.4.4 Simultaneous Projection for the Linear
Feasibility Problem . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 169
4.5 Cyclic Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 171
4.5.1 Cyclic Relaxed Projection . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 172
4.5.2 Cyclic-Simultaneous Projection .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 173
4.5.3 Projections with Reflection onto an Obtuse Cone .. . . . . . . . . 174
4.5.4 Cyclic Cutter . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 176
4.6 Landweber Operator .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 176
4.6.1 Main Properties . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 177
4.6.2 Landweber Operator for Linear Systems . . . . . . . . . . . . . . . . . . . 178
4.6.3 Extrapolated Landweber Operator for a System
of Linear Equations . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 181
4.7 Projected Landweber Operator.. . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 184
4.8 Simultaneous Cutter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 185
4.9 Extrapolated Simultaneous Cutter . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 187
4.9.1 Properties of the Extrapolated Simultaneous
Cutter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 187
4.9.2 Extrapolated Simultaneous Projection . .. . . . . . . . . . . . . . . . . . . . 189
4.9.3 Extrapolated Simultaneous Projection for LFP .. . . . . . . . . . . . 190
4.9.4 Surrogate Projection . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 191
4.9.5 Surrogate Projection with Residual Selection . . . . . . . . . . . . . . 196
4.9.6 Extrapolated Simultaneous Subgradient Projection . . . . . . . . 198
4.10 Extrapolated Cyclic Cutter . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 199
4.10.1 Useful Inequalities . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 200
4.10.2 Properties of the Extrapolated Cyclic Cutter . . . . . . . . . . . . . . . 201
4.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 202
5 Projection Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 203
5.1 Alternating Projection Methods .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 204
5.1.1 General Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 204
5.1.2 Alternating Projection Method for Closed
Linear Subspaces.. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 206
5.2 Extrapolated Alternating Projection Methods .. . .. . . . . . . . . . . . . . . . . . . . 208
5.2.1 Acceleration Techniques for Consistent Problems . . . . . . . . . 209
5.2.2 Acceleration Techniques for Inconsistent Problems . . . . . . . 210
5.2.3 Douglas–Rachford Algorithm .. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 212
5.3 Projected Gradient Method.. . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 213
5.4 Simultaneous Projection Method . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 215
5.4.1 Convergence of the SPM . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 215
5.4.2 Projected Simultaneous Projection Methods . . . . . . . . . . . . . . . 217
xvi Contents
5.5 Cyclic Projection Methods . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 218

5.5.1 Convergence .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 219
5.5.2 Projection-Reflection Method . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 220
5.6 Successive Projection Methods . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 222
5.6.1 Convergence .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 222
5.6.2 Control Sequences . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 223
5.6.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 227
5.7 Landweber Method and Projected Landweber Method .. . . . . . . . . . . . . 228
5.8 Simultaneous Cutter Methods .. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 230
5.8.1 Assumptions on Weight Functions . . . . .. . . . . . . . . . . . . . . . . . . . 231
5.8.2 Convergence Theorem .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 242
5.8.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 245
5.8.4 Block Iterative Projection Methods . . . . .. . . . . . . . . . . . . . . . . . . . 249
5.9 Sequential Cutter Methods . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 250
5.9.2 Control Sequences for Sequential Cutter Methods .. . . . . . . . 251
5.9.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 252
5.10 Extrapolated Simultaneous Cutter Methods .. . . . .. . . . . . . . . . . . . . . . . . . . 253
5.10.1 Assumptions on Step Sizes . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 254
5.10.3 Extrapolated Simultaneous Subgradient
Projection Method . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 258
5.11 Extrapolated Cyclic Cutter Method .. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 259
5.11.1 Convergence .. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 260
5.11.2 Accelerated Kaczmarz Method for a System
of Linear Equations . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 262
5.12 Surrogate Constraints Methods . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 263
5.12.1 Proper Control.. . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 264
5.12.3 Examples of Proper Control .. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 266
5.13 SCM with Residual Selection . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 268
5.13.1 General Properties . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 268
5.13.2 Description of the Method .. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 271
5.13.3 Obtuse Cone Selection .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 272
5.13.4 Regular Obtuse Cone Selection . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 273
5.14 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 274
References .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 275
Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 295
Chapter 1
Introduction
1.1 Background
In this section we present the notation, definitions and basic facts of convex subsets
and convex functions defined on a Hilbert space, convergence and differentiation
properties, properties of matrices, etc., which will be used in further parts of the
book.
1.1.1 Hilbert Space
pH with an inner product h; i, which is complete with respect to the
A linear space
norm kk WD h; i induced by this inner product is called a Hilbert space.
In what follows, we consider a real Hilbert space H, and denote by X a nonempty
closed convex subset of H and by I a finite subset f1; 2; : : : mg N, where m 2 N.
1.1.1.1 Properties of the Inner Product
The following equalities hold for all x; y 2 H:
kx C yk2 D kxk2 C kyk2 C 2hx; yi, (1.1)
kx yk2 D kxk2 C kyk2 2hx; yi (1.2)

and
hx C y; x yi D kxk2 kyk2 . (1.3)
Cauchy–Schwarz inequality. The following inequality holds for all x; y 2 H:
kxk kyk hx; yi kxk kyk . (1.4)
A. Cegielski, Iterative Methods for Fixed Point Problems in Hilbert Spaces, 1

Lecture Notes in Mathematics 2057, DOI 10.1007/978-3-642-30901-4 1,
© Springer-Verlag Berlin Heidelberg 2012
2 1 Introduction
Furthermore,
(i) hx; yi D kxk kyk if and only if x and y are positive linearly dependent vectors,
i.e., ˛x D ˇy for some ˛; ˇ 0,
(ii) hx; yi D kxk kyk if and only if x and y are negative linearly dependent
vectors, i.e., ˛x D ˇy for some ˛; ˇ 0.
Moreover, one can take ˛ D kyk and ˇ D kxk in (i) and in (ii).
Triangle inequality. The following inequality holds for all x; y 2 H:
j kxk kyk j kx C yk kxk C kyk . (1.5)
Furthermore,
(i) kx C yk D kxk C kyk if and only if x and y are positive linearly dependent,
(ii) kx C yk D j kxk kyk j if and only if x and y are negative linearly dependent.
Property (i) is known as the strict convexity of the norm and can also be
formulated equivalently in the form:
(iii) For any 2 .0; 1/ there holds
kxk D kyk D k.1 /x C yk H) x D y. (1.6)
Convexity of the norm. Let w D .!1 ; : : : ; !m / 2 Rm C (all coordinates of w are

nonnegative). The following inequality holds for all xi 2 H, i D 1; 2; : : : ; m;
m
X X m

!i i
x !i kxi k . (1.7)

i D1 i D1
If, moreover, !i > 0, i D 1; 2; : : : ; m, then the equality holds in (1.7) if and only if
all xi , i D 1; 2; : : : ; m, are pairwise positive linearly dependent.
Let w D .!1 ; : : : ; !m / 2 Rm . The following equalities hold for all xi 2 H,
i D 1; 2; : : : ; m;
m 2
X X
m X
m X
m

!i xi D !i !j hxi ; xj i D !i2 kxi k2 C 2!i !j hxi ; xj i.

i D1 i;j D1 i D1 i;j D1;i ¤j
Parallelogram law. The following equality holds for all x; y 2 H:
kx C yk2 C kx yk2 D 2.kxk2 C kyk2 /. (1.8)
Theorem 1.1.1. A Banach space with a norm kk is a Hilbert space with an inner
product defined by the equality
1.1 Background 3
1
hx; yi D .kxk2 C kyk2 kx yk2 /,
2
if and only if the parallelogram law holds.
1.1.1.2 Examples of Hilbert Spaces
Example 1.1.2. The Euclidean space Rn WD fx D .1 ; 2 ; : : : ; n / W j 2 R; j D

1; 2; : : : ; ng. An element x D .1 ; 2 ; : : : ; n / of Rn can be identified with a matrix
2 3
1
6 2 7
6 7
of type n 1 (column vector), i.e., we write x D 6 : 7 or x D Œ1 ; 2 ; : : : ; n > .
4 :: 5
n
The Euclidean space is equipped with the standard inner product defined by
X
n
hx; yi WD x > y D j j ,
j D1
1
where x D .1 ; : : : ; n /, y D .1 ; : : : ; n /, and with the norm kxk WD .x > x/ 2 .
Example 1.1.3. The space Rm with the inner product induced by a positive definite
matrix G D Œgij mm , defined by
X
m X
m
hw; viG WD w> Gv D gij !i j , (1.9)
i D1 j D1
1
where w D .!1 ; : : : ; !m /, v D .1 ; : : : ; m /, and with the norm kukG WD .u> Gu/ 2 .
Example 1.1.4. The space l2 of real sequences x D .1 ; 2 ; : : :/ which are square-
summable, i.e.,
1
X
l2 WD fx W k 2 R, k D 1; 2; : : : and k2 < 1g,
kD1
P1 P1 2 12
with the inner product hx; yi WD kD1 k k , and with the norm kxk WD. kD1 k / .
Example 1.1.5. L2 .Œa; b/, where a; b 2 Œ1; C1, a < b—the space of
(equivalence classes of) Lebesgue measurable functions f W R ! R which are
square-integrable, i.e.,
Z b
L2 .a; b/ WD ff W f 2 .x/dx < 1g
a
4 1 Introduction
with the inner product

Z b
hf; gi WD f .x/g.x/dx,
a
and with the norm Z b
1
kf k WD . f 2 .x/dx/ 2 .
a
Example 1.1.6. Let Hi be a Hilbert space with an inner product h; iHi and with
the norm kkHi induced by this inner product, i 2 I WD f1; 2; : : : ; mg. Let H D
H1 : : : Hm . The function h; i W H H ! R defined by
X
m
hx; yiw WD !i hxi ; yi iHi , (1.10)
i D1
where x D .x1 ; : : : ; xm /, y D .y1 ; : : : ; ym / 2 H, w D .!1 ; : : : ; !m / 2 Rm

CC (all
coordinates of w are positive), is an inner product in H, and the function kk W H !
R defined by the equality
X
m
1
kxkw WD . !i kxi k2Hi / 2
i D1
is the norm in H induced by this inner product. The space H with the inner product
h; iw defined by (1.10) is a Hilbert space and is called a product Hilbert space.
1.1.1.3 Weak Convergence in Hilbert Spaces
We say that a sequence fx k g1

kD0 of elements of a Hilbert space H converges weakly
to x 2 H if for any y 2 H the sequence fhy; x k ig1 kD0 converges to hy; xi. We
call the point x a weak limit of the sequence fx k g1 kD0 and write x
k
* x. If a
nk 1 k 1
subsequence fx gkD0 fx gkD0 converges weakly to x, then x is called a weak
cluster point of the sequence
fx k g1 k 1
kD0 . We say that fx gkD0 converges (strongly) to

x if limk!1 x x D 0.
k
Weakly convergent sequences have the following properties:
1. A weakly convergent sequence fx k g1 kD0 H has exactly one weak limit.
2. A weakly convergent sequence fx k g1 kD0 H is bounded.
3. A bounded sequence fx k g1 kD0 H includes a weakly convergent subsequence.
4. If a sequence fx k g1
kD0 H is bounded and has exactly one weak cluster point
x 2 H, then x k * x.
5. If a sequence fx k g1
kD0 converges to x 2 H, then it converges weakly to x 2 H.
6. A weakly convergent sequence fx k g1 kD0 of a finite dimensional Hilbert space H
is convergent.
1.1 Background 5
Note that a bounded sequence fx k g1

kD0 of a Hilbert space H needs not to include
a convergent subsequence.
1.1.2 Notations and Basic Facts
1.1.2.1 Euclidean Space
Elements of the Euclidean space Rn can be represented as vectors with real

coordinates, e.g., x D .1 ; : : : ; n /, y D .1 ; : : : ; n /, z D .1 ; 2 ; : : : ; n / or
as n 1 matrices (column vectors), e.g., x D Œ1 ; : : : ; n > , y D Œ1 ; : : : ; n T ,
z D Œ1 ; 2 ; : : : ; n > , where j , j , i are coordinates of x, y and z respectively,
j D 1; 2; : : : ; n. We usually denote the coordinates of a vector in the Euclidean
space with Greek letters. Let x; y 2 Rn . Then, x 0 means that all coordinates
of the vector x are nonnegative and x > 0 means that all coordinates of the vector
x are positive (the symbols x 0 and x < 0 are defined similarly). Furthermore,
x y denotes that y x 0 and x y denotes that x y 0. We use the
following notation: maxfx; yg WD .maxf1 ; 1 g; : : : ; maxfn ; n g/; minfx; yg WD
.minf1 ; 1 g; : : : ; minfn ; n g/; xC WD maxfx; 0g; x WD maxfx; 0g; x D xC
x ; RmC WD fx 2 R W x 0g denotes the nonnegative orthant; R WD fx 2
n m
R W x 0g denotes the nonpositive orthant; RCC WD fx 2 R W x > 0g

n m n
denotes the positive orthant; ej WD .0; : : : ; 0; 1; 0; : : : ; 0/ 2 Rm denotes the j th

unit vector, i.e., a vector, for which the j th coordinate equals 1 and the others
are zeros; e denotes a vector with all coordinates equal to 1, i.e., e D .1; : : : ; 1/;
m WD fu 2 Rm W u 0; e > u D 1g denotes the standard simplex; N denotes the set
of positive integers; For a finite subset J N, the symbol jJ j denotes the number
of elements of J .
1.1.2.2 Subsets of a Hilbert Space
Let V H. The subset
V ? WD fy 2 H W hy; xi D 0 for all x 2 V g
denotes the subspace orthogonal to V . Let V H be a closed subspace. Then

H D V ˚ V ? , i.e., for any x 2 H, there are unique v 2 V and w 2 V ? such that
x D v C w. The subset
B.x; / WD fy 2 H W ky xk g
denotes a ball with a centre x 2 H and radius > 0. C 0 denotes the complement of
a subset C H, i.e., C 0 D HnC . The subset
6 1 Introduction
C C D WD fz 2 H W z D x C y; x 2 C; y 2 Dg
denotes the Minkowski sum of C; D H. In particular,
C C a WD C C fag D fx C a W x 2 C g,
where C H and a 2 H. The subsets int C , cl C and bd C denote the interior, the
closure and the boundary of a subset C H, respectively. The subset
S.x; / WD bd B.x; / D fy 2 H W ky xk D g
denotes a sphere with a centre x 2 H and radius > 0. Lin S and Lin a denote
a linear subspace generated by the subset S H and by the vector a 2 H,
respectively. The subset
Œa; b WD fz 2 H W z D .1 /a C b; 2 Œ0; 1g
denotes a segment with endpoints a; b 2 H. The subset
Argmin f .x/ WD fz 2 C W f .z/ f .x/ for all x 2 C g,

x2C
where C H and f W C ! R is called a subset of minimizers of a function f .

An element of Argminx2C f .x/ is called a minimizer of f and is denoted by
argminx2C f .x/. The subset
H.a; ˇ/ WD fx 2 H W ha; xi D ˇg,
where a 2 H, a ¤ 0 and ˇ 2 R is called a hyperplane in a Hilbert space H. The

hyperplane H.a; ˇ/ is the boundary of two half-spaces
HC .a; ˇ/ WD fx 2 H W ha; xi ˇg
and
H .a; ˇ/ WD fx 2 H W ha; xi ˇg.
A subset K H is called an affine subspace if for all x; y 2 K and for any 2 R
we have .1 /x C y 2 K. The number
ha; bi
^.a; b/ WD arccos
kak kbk
denotes an angle between nonzero vectors a; b 2 H.

1.1 Background 7
1.1.2.3 Functions
Let X H, fi W X ! R, i 2 I . The function f WD maxi 2I fi is defined by
f .x/ WD maxffi .x/ W i 2 I g
In particular, fC WD maxff; 0g, f WD maxff; 0g.

Let f W X ! R. The subset
S.f; ˛/ D fx 2 X W f .x/ ˛g
is called a sublevel set of f at a level ˛ 2 R. The subset
epi f WD f.x; / 2 X R W f .x/g X R
is called an epigraph of f . We say that f is lower semi-continuous at x 2 X if

lim infy!x f .y/ f .x/. We say that f is weakly lower semi-continuous at x 2 X
if lim infy*x f .y/ f .x/. We say that a function f W H ! R is coercive if
limkxk!1 f .x/ D C1. We say that a function f W X ! R attains the global
minimum at a point x 2 X if f .x / f .x/ for all x 2 X . We say that a function
f W X ! R attains a local minimum at a point x 2 X if there exists ı > 0 such
that f .x / f .x/ for all x 2 X \ B.x ; ı/.
Theorem 1.1.7. A function f W X ! R is (weakly) lower semi-continuous if and
only if epi f is a (weakly) closed subset of X R.
Theorem 1.1.8. If a function f W X ! R is weakly lower semi-continuous on a
weakly closed and bounded subset X H, then f is bounded from below and
attains its minimum.
Theorem 1.1.9. If a function f W H ! R is weakly lower semi-continuous and
coercive and X H is weakly closed, then f attains the global minimum on X .
f .x C ts/ f .x/
Definition 1.1.10. If a limit limt #0 exists, then we call it a
t
directional derivative of a function f W H ! R at a point x 2 H and in a direction
s 2 H and we denote it by f 0 .x; s/.
If a function f W H ! R attains minimum at x 2 H and f 0 .x; s/ exists for some
s 2 H, then f 0 .x; s/ 0.
Definition 1.1.11. We say that a function f W H ! R is Fréchet-differentiable or,
shortly, differentiable at x 2 H if there exists y 2 H such that
f .x C h/ D f .x/ C hy; hi C o.khk/,
o.t/
where limt !0 D 0. The element y is called a derivative (or differential) of f
t
at x and is denoted by f 0 .x/.
8 1 Introduction
The derivative of a differentiable function is uniquely determined.

Definition 1.1.12. We say that a function f W H ! R is Gâteaux-differentiable at
a point x 2 H if it has directional derivatives f 0 .x; s/ for all s 2 H and
f 0 .x; s/ D hg; si
holds for some g 2 H. The element g is called a Gâteaux-derivative or Gâteaux-

differential of f at x and is denoted by Df .x/.
The Gâteaux-differential of a Gâteaux-differentiable function is uniquely deter-
mined.
Theorem 1.1.13. If a function f W H ! R is Fréchet-differentiable at x 2 H, then
it is Gâteaux-differentiable at x and f 0 .x/ D Df .x/.
Theorem 1.1.14. If a function f WH ! R is Gâteaux-differentiable in a neigh-
borhood of x and the Gâteaux-differential Df is continuous at x, then f is
Fréchet-differentiable at x.
If H is a Euclidean space (H D Rn and h; i is the standard inner product), then the
derivative Df .x/ is called a gradient of f at the point x and is denoted by rf .x/.
There holds the equality rf .x/ D . @f@.x/
1
; : : : @f@.x/
n
/, where @f@.x/
j
denotes a partial
derivative of f at the point x D .1 ; : : : ; n / with respect to j . Let x 2 Rn and
2f
assume that f W Rn ! R has partial derivatives of the second order @@i @ j
.x/ for all
i; j D 1; 2; : : : ; n. Then the matrix
2 3
@2 f @2 f
.x/ : : : @1 @n .x/ 7
6 @12
r 2 f .x/ WD 6
4 ::: ::: ::: 75
@2 f @2 f
@n @1
.x/ ::: .x/
@n2
is called the Hessian of f at .

2
Theorem 1.1.15 (Schwarz). Let f W Rn ! R. If @@i @ f
j
are continuous at a point
x 2 Rn , i; j D 1; 2; : : : ; n, then the Hessian is a symmetric matrix.
1.1.2.4 Operators
Let H; H1 ; H2 ; H3 be Hilbert spaces. The operator Id W H ! H denotes the identity,

i.e., Id x D x for all x 2 H.
Definition 1.1.16. Let A W H1 ! H2 be a bounded linear operator. The number
kAxkH2
kAk WD sup ,
xW0<kxk1 kxkH1
1.1 Background 9
where kkH1 and kkH2 denote the norms in H1 and H2 , respectively, is called a
norm of A in L.H1 ; H2 /.
Definition 1.1.17. Let A W H1 ! H2 be a bounded linear operator. An operator
A W H2 ! H1 with the property
hAx; yiH2 D hx; A yiH1
for all x 2 H1 and y 2 H2 , where h; iH1 and h; iH2 are inner products in H1 and
H2 , respectively, is called an adjoint operator.
For every A an adjoint operator A exists and is uniquely determined. Futher-
more, A is a bounded linear operator.
Definition 1.1.18. We say that a bounded linear operator A W H ! H is unitary if
A A D Id holds.
Definition 1.1.19. We say that a bounded linear operator A W H ! H is self-
adjoint, if A D A.
Definition 1.1.20. We say that a self-adjoint operator A W H ! H is nonnegative
if hAx; xi m kxk2 for some constant m 0 and for all x 2 H. If m > 0, then we
say that A is positive (or elliptic).
Theorem 1.1.21. For any bounded linear operator A W H1 ! H2 it holds
kAk2 D kA k D kA Ak D kAA k .

2
Theorem 1.1.22. If a bounded linear operator A W H ! H is self-adjoint, then

kAk D supkxk1 hAx; xi.
Theorem 1.1.23. Let A W H1 ! H2 be a bounded linear operator. Then the
operators AA and A A are nonnegative.
A number 2 R is called an eigenvalue of a bounded linear operator A W H ! H
if Ax D x for some x ¤ 0.
Theorem 1.1.24. If is an eigenvalue of a bounded linear operator A W H ! H,
then jj kAk. In particular, if A is self-adjoint, then jj supkxk1 hAx; xi.
Definition 1.1.25. We say that a bounded linear operator A W H1 ! H2 is compact
if for any bounded sequence fx k g1 k 1
kD0 H1 , the sequence fAx gkD0 H2 contains
a convergent subsequence.
If dim H1 < 1, then every bounded linear operator A is compact.
Theorem 1.1.26. Let A W H ! H be a nonnegative, nonzero compact linear
operator. Then:
(i) There exists an eigenvalue D max .A/ of the operator A such that
10 1 Introduction
D kAk D sup hAx; xi; (1.11)

kxk1
consequently, at least one eigenvalue of the operator A is positive.

(ii) If A is positive, then all of its eigenvalues are positive.
(iii) There exists z 2 H with kzk D 1 such that
hAz; zi D sup hAx; xi.

kxk1
Theorem 1.1.27. Let A W H1 ! H2 be a linear and compact operator. Then

p p
kAk D max .A A/ D max .AA / D kA k :
Definition 1.1.28. We say that an operator T W H ! H is monotone if
hT x T y; x yi kx yk2
for a constant 0 and for all x; y 2 X . If > 0, then we say that T is strongly
monotone or -strongly monotone.
Definition 1.1.29. We say that an operator T W H ! H is Lipschitz continuous if
kT x T yk
kx yk .
for a positive constant

. The constant
is called a Lipschitz constant. We also say
that T is
-Lipschitz continuous.
Definition 1.1.30. We say that an operator T W H1 ! H2 is differentiable at a
point x 2 H1 if there exists a bounded linear operator G 2 L.H1 ; H2 / such that
T .x C h/ D T .x/ C Gh C o.khk/.
The operator G is uniquely determined and is called a derivative or a differential

of the operator T at the point x and is denoted by DT .x/ or T 0 x.
Theorem 1.1.31. Let T W H1 ! H2 be continuously differentiable in a neighbor-
hood of a point x 2 H1 and U W H2 ! H3 be continuously differentiable in a
neighborhood of T x 2 H2 . Then there exists a differential D.U ı T /.x/ and
D.U ı T /.x/ D D.U.T x//DT .x/.
Definition 1.1.32. Let T W X ! Y , where X H1 and Y H2 , and let C X

and D Y . The subset
T .C / WD fT x W x 2 C g Y
1.1 Background 11
is called an image of C by the operator T . The subset
T 1 .D/ WD fx 2 X W T x 2 Dg
is called an inverse image of D by the operator T .

The iteration T k of an operator T W X ! X , where X H, is defined by
the recurrence: T 1 D T , T kC1 D T T k , k 1. For any x 2 X the sequence
fT k xg1
kD0 is called an orbit of T . A point z 2 X is called a fixed point of an operator
T W X ! H, if T z D z. The subset of all fixed points of the operator T is denoted by
Fix T . A subset C X is called a retract of X if there exists a continuous operator
T W X ! C (called a retraction) with Fix T D C . We say that an operator T W X !
X is idempotent or a projection if T T x D T x for all x 2 X , i.e., T .X / Fix T .
Since the converse implication holds for all operators, we have: T is idempotent if
and only if T .X / D Fix T . A continuous projection is a retraction. Let V H be
a closed subspace. The operator P W H ! V with the property P x x 2 V ? for
all x 2 H is called the orthogonal projection onto V . The orthogonal projection P
onto a closed subspace is defined uniquely and is a linear and bounded operator with
the norm kP k D 1. Let a 2 H. The orthogonal projection onto Lin a is denoted by
Pa . We say that an operator A W H1 ! H2 is affine if for all x; y 2 H1 and for any
2 R, A..1 /x C y/ D .1 /Ax C Ay holds. Let A W H1 ! H2 be a linear
operator. The subset
ker A WD fx 2 H1 W Ax D 0g
denotes a kernel or null space of A and the subset
Im A WD fy 2 H2 W y D Ax for some x 2 H1 g
denotes an image or range of A. The subsets ker A H1 and Im A H2 are

subspaces of H1 and H2 , respectively. It holds
.ker A/? D Im A ,
consequently,
H1 D ker A ˚ Im A .
1.1.2.5 Matrices
Definition 1.1.33. We say that a symmetric matrix A of type n n is positive

definite, if x > Ax > 0 for all nonzero vectors x 2 Rn . We say that A is positive
semi-definite if x > Ax 0 for all x 2 Rn .
Let v D .1 ; : : : ; m / 2 Rm . Then diag v denotes a diagonal matrix of type mm
with 1 ; : : : ; m on its diagonal, i.e.,
12 1 Introduction
2 3
1 0 ::: 0
6 0 2 ::: 0 7
diag v D 6
4:::
7.
::: ::: :::5
0 0 ::: m
Theorem 1.1.34. For a symmetric matrix A the following conditions are equiva-
lent:
(i) A is positive definite (positive semi-definite),
(ii) All eigenvalues of A are positive (nonnegative),
(iii) There exists an orthogonal matrix U and a diagonal matrix D with positive
(nonnegative) elements on the diagonal such that A D U > DU ,
(iv) There exists a positive definite (positive semi-definite) matrix B such that
A D B 2.
1
We denote by A 2 the matrix B with property (iv) above.
Definition 1.1.35. Let A be an m n matrix. The matrix A> A is called the Gram
matrix of the columns of A.
The Gram matrix A> A is positive semi-definite. Furthermore, it is positive
definite if and only if A has full column rank, i.e., the columns of A are linearly
independent.
Definition 1.1.36. The Moore–Penrose pseudoinverse AC of an m n matrix A is
a uniquely determined n m matrix satisfying the following conditions:
AAC A D A, AC AAC D AC , .AAC /> D AAC , .AC A/> D AC A:
If A is a full column rank or a full row rank matrix, then
AC D .A> A/1 A>
or
AC D A> .AA> /1 ,
respectively.
1.1.2.6 Convex Subsets
Let H, Hi , i D 1; 2; : : : ; m, be Hilbert spaces.

Definition 1.1.37. A subset C H is said to be convex, if .1 /x C y 2 C for
all x; y 2 C and for all 2 Œ0; 1.
An intersection of arbitrary family of convex subsets is convex. In particular,
a polytope, i.e., the intersection of a finite number of half-spaces, is convex. The ball
1.1 Background 13
B.x; / is convex for any x 2 H and for any 0. Let A W H1 ! H2 be an affine

operator. If C H1 is a convex subset, then the image A.C / H2 is convex. If
D H2 is convex, then the inverse image A1 .D/ H1 is convex. The product
C1 : : : Cm of convex subsets Ci Hi , i D 1; 2; : : : ; m, is a convex subset of
the product space H WD H1 : : : Hm .
Theorem 1.1.38. If C H is convex, then its closure and its interior are convex
subsets.
Theorem 1.1.39. A subset C H is closed and convex if and only if C is an
intersection of half-spaces.
The following theorem follows from the Mazur theorem (see, e.g., [158, Chap. I,
Sect. 1.2], [180, Sect. 2.3, Corollary 2], [267, Theorem 21.4], [300, Theorem 3.12]
or [214, Theorem 8.48]).
Theorem 1.1.40. If C H is a closed convex subset, then C is weakly closed.
Definition 1.1.41. The smallest convex subset containing S H is called a convex
hull of S and is denoted by conv S . The smallest affine subspace containing S H
is called an affine hull of S and is denoted by aff S . The subset
ri C WD fx 2 C W B.x; "/ \ aff C C for some " > 0g
is called a relative interior of a convex subset C H .

It holds
ri m D fu 2 Rm W u > 0; e > u D 1g,
where m denotes a standard simplex in Rn .
Definition 1.1.42. A subset K H is called a convex cone if ˛x C ˇy 2 K for
all x; y 2 K and for all ˛; ˇ 0.
Definition 1.1.43. For a convex cone C H the subset
C WD fx 2 H W hy; xi 0 for all y 2 C g
is called a polar cone to C H.

Definition 1.1.44. We say that a convex cone C H is obtuse if C C .
Theorem 1.1.45 (Moreau’s decomposition). Let K H be a nonempty closed
convex cone and x; x C ; x 2 H. Then the following conditions are equivalent:
(i) x D x C C x , where x C 2 K, x 2 K and hx C ; x i D 0,
(ii) x C D PK x and x D PK x.
Definition 1.1.46. The subset NC .x/ WD fy 2 H W hy; z xi 0 for all z 2 C g is
called a normal cone to a convex subset C H at x 2 C . The subset
14 1 Introduction

xk x
TC .x/ WD s 2 H W 9fx k g1
kD0 C;x !x
k 9ftk g1
kD0 ;tk #0
with s D lim
k tk
is called the tangent cone to a convex subset C H at x 2 C .

Theorem 1.1.47. Let C H be a closed convex subset and x 2 C . The following
equalities hold
.NC .x// D TC .x/ and .TC .x// D NC .x/.
1.1.2.7 Convex Functions
Let X; Y H be convex subsets of a Hilbert space H and let f W X ! R.

Definition 1.1.48. We say that f is convex if
f ..1 /x C y/ .1 /f .x/ C f .y/
holds for all x; y 2 X and for all 2 Œ0; 1. If the inequality is strict for all x; y 2 X ,
x ¤ y and for all 2 .0; 1/, then we say that f is strictly convex. We say that f is
˛-strongly convex, where ˛ > 0 or, shortly, strongly convex if
1
f ..1 /x C y/ .1 /f .x/ C f .y/ ˛.1 / kx yk2
2
for all x; y 2 X and for all 2 Œ0; 1. A function f W X ! R is called concave if
f is convex.
Properties of convex functions
1. If fi W X ! R, i 2 I , are convex, then the function f WD maxi 2I fi is convex.

2. If f W X ! R is convex and g W R ! R is convex and nondecreasing, then the
composition g ı f is convex.
3. If fi W X ! R, i D 1; 2; : : : ; m, are convex and F W Rm ! R is a
convex function which is nondecreasing with respect to any coordinate, then
the function f WD F .f1 ; : : : ; fm / is convex.
4. If f W Y ! R is a convex function and A W X ! Y is an affine operator, then
the function f ı A is convex.
5. If f W X Y ! R is convex, then the function h W X ! R, h.x/ WD
infy2Y f .x; y/ is convex.
6. Any sublevel set S.f; ˛/, ˛ 2 R, of a convex function f W X ! R is convex.
7. A function f W X ! R is convex if and only if its epigraph epi f is a convex
subset of X R.
8. A convex function f W Rn ! R is continuous.
1.1 Background 15
9. A strongly convex function f W H ! R is coercive.

10. A convex function f W H ! R has directional derivatives f 0 .; s/ for all s 2 H,
and for any x; s 2 H it holds
f .x C ts/ f .x/
f 0 .x; s/ D inf .
t >0 t
11. Let x 2 H. The directional derivative f 0 .x; / W H ! R of a convex function
f W H ! R is convex and positively homogeneous of degree 1, i.e., for all
s 2 H and for all ˛ 0 the equality f 0 .x; ˛s/ D ˛f 0 .x; s/ holds.
12. If f W H ! R is a convex function and f 0 .x; s/ 0 for all s 2 H, then f
attains its minimum at x 2 H.
13. The norm kk W H ! R is convex and differentiable on Hnf0g and D kxk D
x
kxk
for any x ¤ 0.
14. The function f W H ! R, f .x/ WD 12 kxk2 is differentiable and 1-strongly
convex, and Df .x/ D x for all x 2 H.
15. Let C H be nonempty. A function d.; C / W H ! R defined by d.x; C / WD
infz2C kz xk is called a distance function to the subset C . If C is convex,
then d.; C / is a convex function.
16. Let a function f W Rn ! R have continuous partial derivatives of the second
order. Then f is convex if and only if its Hessian r 2 f .x/ is a positive semi-
definite matrix for all x 2 Rn . The function f is ˛-strongly convex if and only
if s > r 2 f .x/s ˛ ksk2 for all x; s 2 Rn .
Theorem 1.1.49. If a convex function f W H ! R is bounded in a neighborhood
of a point x 0 2 H, then f is locally Lipschitz continuous on H, i.e., for any x 2 H
there is > 0 and
0 such that j f .u/ f .v/ j
ku vk for all u; v 2
B.x; /. In particular, a continuous convex function f W H ! R is locally Lipschitz
continuous on H.
If H is finite dimensional, then Theorem 1.1.49 can be strengthened. In this case the
local Lipschitz continuity can be replaced by the global one on bounded subsets.
Theorem 1.1.50. If a convex function f W H ! R is
-Lipschitz continuous in a
neighborhood of a point x 2 H, then for any s 2 H it holds
f 0 .x; s/
ksk .
Theorem 1.1.51. A lower semi-continuous convex function f W H ! R is weakly

lower semi-continuous.
Theorem 1.1.52. A continuous convex function f W X ! R defined on a closed
bounded convex subset X H attains its global minimum.
Corollary 1.1.53. If a continuous convex function f W X ! R defined on a closed
convex subset X H is coercive, then f attains its global minimum on X .
Theorem 1.1.54. A convex Gâteaux-differentiable function f WRn ! R is differen-
tiable.
16 1 Introduction
Subdifferential of a Convex Function and Its Properties
Definition 1.1.55. Let f W H ! R be convex. The subset
@f .x/ WD fg 2 H W hg; y xi f .y/ f .x/ for all y 2 Hg
is called a subdifferential of f at x 2 H. The function f is said to be

subdifferentiable at x if @f .x/ ¤ ;. An element of the subdifferential @f .x/ is called
a subgradient of f at x and is denoted by gf .x/. The affine functionfNx W H ! R
defined by
fNx .y/ WD hg; y xi C f .x/,
where g 2 @f .x/ is called a linearization of f at x.
Theorem 1.1.56. For any x 2 H the subdifferential @f .x/ of a continuous convex
function f W H ! R is a nonempty, weakly closed and bounded convex set.
Theorem 1.1.57. A continuous convex function f W H ! R is Gâteaux-
differentiable at x 2 H if and only if its subdifferential @f .x/ consists of one
point. In this case @f .x/ D fDf .x/g.
Theorem 1.1.58. Let f W H ! R be continuous and convex and x; s 2 H. The
following equality holds
f 0 .x; s/ D supfhy; si W y 2 @f .x/g.
In particular, if f is differentiable, then f 0 .x; s/ D hDf .x/; si If, moreover, H is a

Euclidean space, then f 0 .x; s/ D s > rf .x/.
The subdifferential of the norm kk has the form
(
f kxk
x
g if x ¤ 0
@.kxk/ D
B.0; 1/ if x D 0.
All the facts presented in this section can be found in the following references
[33, 38, 40, 61, 133, 158, 180, 181, 209, 210, 214, 234, 262–264, 267, 298, 300, 317,
325].
1.2 Metric Projection
In this section we define the metric projection in a Hilbert space H and present its
basic properties. This operator plays an important role in further parts of the book.
Other properties of the metric projection will be presented in Chap. 2.
Definition 1.2.1. Let C H be a nonempty subset and x 2 H. If there exists a
point y 2 C such that
1.2 Metric Projection 17
Fig. 1.1 Metric projection x
C
z
y = PC x
ky xk kz xk
for any z 2 C , then y is called a metric projection of x onto C and is denoted
by PC x (Fig. 1.1). The vector s D PC x x is called a projection vector of x
onto C . If PC x exists and is uniquely determined for all x 2 H, then the operator
PC W H ! C is called the metric projection (onto C ).
Remark 1.2.2. Let C H. If x 2 C , then it follows from the definition of the
metric projection that PC x D x. If x … C and there exists a metric projection PC x,
then PC x 2 bd C . Indeed, if PC x 2 int C , then B.PC x; "/ C for a sufficiently
small " > 0. Then
"
z WD x C .1 /.PC x x/ 2 B.PC x; "/ C ,
kPC x xk
because kz PC xk D ", and

"
kz xk D .1 / kPC x xk < kPC x xk
kPC x xk
which contradicts the definition of PC x.
1.2.1 Existence and Uniqueness of the Metric Projection
Definition 1.2.1 does not yield the existence and the uniqueness of a metric projec-
tion PC x for a point x 2 H and for a subset C H. If C H is compact, then the
existence of a metric projection follows from the Weierstrass theorem but it needs
not to be defined uniquely. For example, both points a; b 2 H are metric projections
of the midpoint x WD aCb 2 of the segment Œa; b onto the subset C D fa; bg.
Therefore, the convexity of C seems to be a natural assumption for the uniqueness of
the metric projection PC x for any x 2 H. Furthermore, it follows from the definition
of a metric projection and from the continuity of the norm that PC x (if it exists)
belongs to the closure of C . Actually, PC x 2 bd C for x … C (see Remark 1.2.2).
Therefore, the closedness of C is a natural assumption for the existence of the
metric projection PC x for any x 2 H. It turns out that in a Hilbert space these two
assumptions (the closedness and the convexity of C ) are sufficient for the existence
18 1 Introduction
and the uniqueness of the metric projection PC x for all x 2 C . The following
theorem can be found, e.g., in [61, Sect. 1.2.2, Theorem 1.7], [140, Theorem 3.4(2)],
[173, Theorem 7.43], [209, Chap. III, Sect. 3.1] or [267, Theorem 8.25].
Theorem 1.2.3. Let C H be a nonempty closed convex subset. Then for any
x 2 H there exists a metric projection PC x and it is uniquely determined.
Proof. First we prove the assertion for x D 0. Let d WD inffkyk W y 2 C g and

a sequence fy k g1 y k ! d . We split the proof onto three
kD0 C be such that
parts.
(a) We show that fy k g1kD0 is a Cauchy sequence. Let " > 0 and k0 0 be such
k 2
d C "=4 for k k0 . Let k; l k0 . Since C is convex, we have
2
that y
1 k l
1 k
2 y C 1 l
2 y 2 C . Therefore, 2 y C y d . The parallelogram law yields
now k
y y l 2 D 2 y k 2 C 2 y l 2 y k C y l 2
2.d 2 C "=4/ C 2.d 2 C "=4/ 4d 2 D ",

i.e., fy k g1
kD0 is a Cauchy sequence.
(b) It follows from (a) that y k converges to an element y 2 H, because H is
a complete space. Furthermore, y 2 C , because C is closed. Hence, by the
continuity of the norm, it follows that kyk D d . Therefore, y D PC 0.
(c) Now we show that the metric projection PC x is uniquely determined. Let y 0 2
C with ky 0 k D d . It follows from the convexity of C that 12 y C 12 y 0 2 C .
Furthermore,

1 1 0 1 1

d yC y kyk C y 0 D d;
2 2 2 2
i.e., ky C y 0 k D 2d . Again, by the parallelogram law, we have

y y 0 2 D 2 kyk2 C 2 y 0 2 y C y 0 2 D 2d 2 C 2d 2 4d 2 D 0;
i.e., y D y 0 .
We have proved that PC 0 exists and is uniquely defined. Of course, C x is
closed convex, because C is closed convex. Therefore, PC x 0 also exists and is
uniquely defined. It follows easily from the definition of the metric projection that
PC x D x C PC x 0,
consequently, the assertion is true for every x 2 H. t

u
One can ask if the converse theorem to Theorem 1.2.3 is also true. The answer is
positive in the Euclidean space and is known as the Motzkin theorem (see [336,
Theorem 7.5.5]). In general Hilbert spaces the problem is still open.
1.2.2 Characterization of the Metric Projection
The theorem below gives a criterion for y 2 C to be the metric projection of a point
x 2 H onto a convex set C H and is often used in applications. The theorem
can be found, e.g., in [140, Theorem 4.1], [173, Theorem 7.45], [209, Chap. III,
Theorem 3.1.1] and in [185, Proposition 3.5] which contains several other equivalent
conditions.
Theorem 1.2.4 (Characterization Theorem). Let x 2 H, C H be a convex
subset and y 2 C . The following conditions are equivalent:
(i) y D PC x,
(ii) hx y; z yi 0 for all z 2 C .
Proof. (i) )(ii). Let y D PC .x/, z 2 C and
z D y C .z y/
for 2 .0; 1/. Obviously, z 2 C because C is convex. By (i) and by the

properties of the inner product, we have
kx yk2 kx z k2 D kx y .z y/k2

D kx yk2 2hx y; z yi C 2 kz yk .
Since > 0, the above inequalities yield

hx y; z yi kz yk2 ,
2
and for ! 0 we obtain (ii) in the limit.
(ii) )(i). By the properties of the inner product and by (ii) we obtain for any z 2 C
kz xk2 D kz y C y xk2
D kz yk2 C ky xk2 C 2hz y; y xi
ky xk2 ,
which, by the definition of the metric projection, gives (i). t

u
Condition (ii) of Theorem 1.2.4 says that ^.x y; z y/ =2 if x y and z y
are nonzero vectors (see Fig. 1.2).
A simple proof of the following lemma is left to the reader.
Lemma 1.2.5. Let x; y; z 2 H. The following conditions are equivalent:
(a) hx y; z yi 0,
(b) hz x; y xi ky xk2 ,
(c) kz yk2 kz xk2 ky xk2 ,
(d) hz x; z yi 0.
20 1 Introduction
Fig. 1.2 Characterization of x

the metric projection
C
z
Fig. 1.3 Metric projection

onto a translated subset C
x−u
PC (x − u)
−u
u C+u
PC+ u x
x
It follows from Lemma 1.2.5 that the inequality in condition (ii) of the Char-
acterization Theorem 1.2.4 can be replaced by one of the conditions (a)–(d) of
Lemma 1.2.5.
1.2.3 First Applications of the Characterization Theorem
In this section we give some useful facts which follow from the Characterization
Theorem 1.2.4.
Lemma 1.2.6. Let x 2 H, C H be a convex subset and u 2 H. If a metric
projection PC .x u/ exists, then a metric projection PC Cu x also exists and
PC Cu x D PC .x u/ C u (1.12)
holds (Fig. 1.3).
Proof. The proof of the lemma is a direct application of the Characterization

Theorem 1.2.4 and is left to the reader. t
u
Lemma 1.2.7. Let u 2 H, C H be convex and A W H ! H be a unitary
operator. If a metric projection PC .A u/ exists, then a metric projection PA.C / u
also exists and
PA.C / u D A.PC .A u//
holds (Fig. 1.4).

onto a unitary transformation A(C )
of a subset
PA(C) u
u
A∗ P C (A ∗ u) C
A A∗ (u)
Proof. Let y D PC .A u/ and v D Ay. We show that v D PA.C / u. Since y 2 C ,

we have v 2 A.C /. Since A is a linear operator and C is convex, the subset A.C /
is convex. By the characterization of the metric projection (see Theorem 1.2.4), for
all w 2 A.C / and for z 2 C with Az D w, we have
0 hA u y; z yi
D hA u A Ay; z yi
D hu v; Az Ayi
D hu v; w vi.
Again by the characterization of the metric projection (see Theorem 1.2.4), we

obtain v D PA.C / u. u
t
Let Hi be Hilbert spaces with the inner products h; iHi , i 2 I WD f1; 2; : : : ; mg,
and H WD H1 : : : Hm be the product Hilbert space.
Lemma 1.2.8. Let Ci Hi be convex, i D 1; 2; : : : ; m, C WD C1 : : : Cm 2 H
and x D .x1 ; : : : ; xm / 2 H. If metric projections PCi xi , i D 1; 2; : : : ; m, exist, then
a metric projection PC x also exists and
PC x D .PC1 x1 ; : : : ; PCm xm /:
Proof. The proof of the lemma is a direct application of the Characterization

Theorem 1.2.4 and is left to the reader. t
u
Lemma 1.2.8 is illustrated in Fig. 1.5.
Lemma 1.2.9. Let C H be a nonempty closed convex set and PC W H ! H
be a metric projection onto C . Then for any x 2 H the following conditions are
equivalent:
(i) y D PC x
(ii) y 2 C and x y 2 NC .y/.
22 1 Introduction

onto a product of subsets

and normal cone
x
z
NC (y) C
y = PC x
Proof. By the definition of the normal cone, we can write the second part of the
right hand side of (ii) in the form hx y; z yi 0 for any z 2 C (Fig. 1.6).
The lemma follows now from the characterization of the metric projection (see
Theorem 1.2.4). t
u
For T W H ! H and 2 R denote T WD Id C.T Id/.
Corollary 1.2.10. Let T W H ! H, C H be a nonempty closed convex subset.
Then Fix.PC T / D Fix.PC T / for any > 0.
Proof. Let > 0. It follows from Lemma 1.2.9 that
x 2 Fix.PC T / ” PC .x C .T x x// D x

” .T x x/ 2 NC .x/
” T x x 2 NC .x/
” PC T x D x
” x 2 Fix.PC T /
which completes the proof. t

u
In Sect. 2.2.3 we give further properties of the metric projection.
1.3 Convex Optimization Problems 23
1.3 Convex Optimization Problems
In this section we present several convex optimization problems: convex mini-

mization, variational inequalities, convex feasibility problems and split feasibility
problems. These problems as well as the methods for solving them have applications
in various areas of mathematics, e.g., in solving linear equations, in probability
and statistics, in conformal mappings, in frame design problems and in many
other problems (for details, see, e.g., [139, 156, 182, 207, 329, 337]). Furthermore,
the mentioned above abstract problems can be treated as mathematical models
for many practical problems which arise in physical, medical, technical and
information sciences. The problems and methods presented in this book found
applications:
• In signal processing [192,275,316], in particular in resolution enhancement [109]
and in signal synthesis [117, 123],
• In image processing [37,312,316], in particular in image recovery [118,309,315],
in image restoration [354, 359], in image reconstruction [220, 221, 280, 289], in
image reconstruction from projections [98, 100, 111, 224, 330], in demosaicking
[248] and in color imaging [311],
• In discrete tomography [82],
• In medical imaging, in particular in computerized tomography [81,189,204,205,
270, 313], in proton computed tomography: [282] and in magnetic resonance
imaging [301],
• In radiation therapy treatment planning [81, 83–85, 93, 110, 199, 206, 206, 212,
241, 261, 324, 334, 335, 341],
• In physics [149, 240], in astrophysics [360], in acoustic [39], in crystallography
[328], in mechanics [265], in materials science [225],
• In seismic tomography [60, 332], in meteorology [323],
• In optics [316], in particular in phase retrieval problem [26, 27, 161] and in
holography [219, 310],
• In wavelet-based denoising [115], in watermarking [242], in adaptive filtering
[314, 355],
• In control problems [192], in control design problems [193, 194],
• In antenna design [195], in antenna pattern synthesis problems [288],
• In data compression [245], in video coding [281],
• In graph matching [340],
• In learning process [319], in supervised learning process [222]
• And in finance [208].
In the next chapters we will show how to solve convex optimization problems by
iterative methods constructed by application of some algorithmic operators. We will
also show that the problem of finding a solution of a convex optimization problem
is equivalent to finding a fixed point of an algorithmic operator.
24 1 Introduction
1.3.1 Convex Minimization Problems
Let f W H ! R and C H. The constrained minimization problem
minimize f .x/
(1.13)
with respect to x 2 C
is to find at least one local minimizer of the function f on C , i.e., a point x 2 C

such that f .x / f .x/ for all x 2 C \ B.x ; / and for some > 0, if it exists.
If f .x / f .x/ for all x 2 C , then x is called a global minimizer of the function
f on C . The point x is also called a (local or global) optimal solution of problem
(1.13). The value f D f .x / is called the minimum of the function f on the
subset C .
Theorem 1.3.1. Let C H be a convex subset and f W H ! R be a convex
function. If x is a local minimizer of f on C , then it is also a global one. If,
furthermore, f is strictly convex, then the minimizer is uniquely determined. If f is
continuous and strongly convex and C is closed, then Argminx2C f .x/ ¤ ;.
Proof. Let x 2 C be a local minimizer of f on C . Suppose that there is a point
x 0 2 C with f .x 0 / < f .x /. Let x D .1 /x C x 0 , where 2 .0; 1/. It is
clear that x 2 C , because C is convex. By the convexity of f , we have
f .x / f .x / .1 /f .x / C f .x 0 / < f .x /
for a sufficiently small > 0. The contradiction shows that f .x/ f .x / for all
x 2 C , i.e., x is a global minimizer of f on C . Suppose now that f is strictly
convex and that f .x 0 / D f .x / holds for some x 0 ¤ x . Then for x D 12 x C 12 x 0 ,
we obtain
1 1
f .x/ < f .x / C f .x 0 / D f .x / f .x/,
2 2
a contradiction which shows that the minimizer is uniquely determined. Now
suppose that f is continuous and strongly convex and that C is closed. Recall
that a strongly convex function is coercive. Therefore, Corollary 1.1.53 yields
Argminx2C f .x/ ¤ ;. t
u
The subset C in (1.13) is often given in the form
C WD fx 2 H W ci .x/ 0; i 2 I g,
where ci W H ! R, i 2 I WD f1; 2; : : : ; mg. In this case (1.13) can be written as the

following constrained minimization problem:
minimize f .x/
subject to ci .x/ 0; i 2 I , (1.14)
x 2 H.
The functions ci are called the constraints of problem (1.14). If the functions
f; ci ; i 2 I , are convex, then (1.14) is called a convex minimization problem (CMP).
If f; ci ; i 2 I , are differentiable, then (1.14) is called a constrained differentiable
minimization problem. In order to solve problem (1.13) or (1.14) one can apply
suitable methods (algorithms) which generate sequences approximating a solution
x of the problem. These algorithms should be constructed in such a way that the
sequences converge (in some sense) to a solution of the problem. Since an algorithm
should terminate after a finite number of iterations, we cannot expect in general that
the algorithm gives an exact solution. Let " 0. We say that x" 2 C is an "-optimal
solution of (1.13) or (1.14) if f .x" / f .x/ C " for all x 2 C .
If f is a convex function and C is a closed convex subset (or ci ; i 2 I , are
convex functions), then problem (1.13) (or (1.14)) is called a convex (constrained)
minimization problem.
For a constrained differentiable minimization problem (1.14) with H D Rn
(equipped with the standard inner product) a point .x ; y / 2 Rn Rm satisfying
the following system of equalities and inequalities
rf .x/ C y > rc.x/ D 0

c.x/ 0
(1.15)
y0
>
y c.x/ D 0,
where c.x/ D .c1 .x/; c2 .x/; : : : ; cm .x//, is called a Karush–Kuhn–Tucker point

or, shortly, KKT-point. If f and ci ; i 2 I , in (1.14) are convex (not necessarily
differentiable), then a point .x ; y / 2 Rn Rm satisfying the conditions
P
0 2 @f .x/ C i 2I i @ci .x/
c.x/ 0
(1.16)
y0
y > c.x/ D 0,
where y D .1 ; : : : m / 2 Rm , is also called a KKT-point. For convex differentiable
minimization problem, (1.15) is, of course, a special case of (1.16).
Below we give necessary conditions and sufficient conditions for a point x to be
an optimal solution of (1.13) or (1.14).
Theorem 1.3.2. A continuous and convex function f W H ! R attains its minimum
at a point x 2 H if and only if 0 2 @f .x/.
Proof. The theorem follows directly from the definition of the subdifferential. t
u
26 1 Introduction
Corollary 1.3.3. A differentiable convex function f W H ! R attains its minimum

at a point x 2 H if and only if Df .x/ D 0.
Theorem 1.3.4. Let f W H ! R be a continuous and convex function and C H
be a closed convex subset. The function f jC attains its minimum at a point x 2 C
if and only if @f .x/ NC .x/. In particular, a convex differentiable function f
attains its minimum at a point x 2 C if and only if Df .x/ 2 NC .x/, i.e.,
f 0 .x; y x/ D hDf .x/; y xi 0
for all y 2 C .
Proof. See [209, Chap. VII, Theorem 1.1.1] for finite dimensional case or [325,
Sect. 7.1] for an infinite dimensional one. t
u
Corollary 1.3.5. Let C H be a closed convex subset and f W H ! R be a
differentiable convex function. Then for any > 0 it holds
Argmin f .x/ D Fix PC .Id Df /.

x2C
Proof. Let > 0. It follows from Theorem 1.3.4 and from Lemma 1.2.9 that
x 2 Argmin f .x/
x2C
” Df .x/ 2 NC .x/

” Df .x/ 2 NC .x/
” x D PC .x Df .x//
” x 2 Fix PC .Id Df /

u
In the theorem below we assume that H D Rn and that the constraints of the
minimization problem (1.14) are regular in some sense, i.e., they satisfy a certain
condition called a constraints qualification. If the constraints ci are convex,
i D 1; 2; : : : ; m, then one often supposes that they satisfy the Slater constraints
qualification, i.e., there exists xN 2 Rn such that ci .x/ N < 0 for all non-affine
functions ci . Other constraints qualifications are also considered in the literature. We
omit the details and refer the reader to [169, Sect. 9.2], [209, Chap. VII, Sect. 2.2],
[38, Sect. 3.3] or [179, Sect. 2.2].
Theorem 1.3.6 (Karush–Kuhn–Tucker). Suppose that the constraints qualifica-
tion for the convex minimization problem (1.14) is satisfied. If a point x 2 Rm is
an optimal solution of (1.14), then there exists y 2 Rm such that .x ; y / is a
KKT-point.
Proof. See [209, Chap. VII, Theorem 2.2.5]. t
u
Theorem 1.3.7. Let f; ci W Rn ! R, i 2 I be convex. If .x ; y / 2 Rn Rm is a

KKT-point of (1.14), then x is an optimal solution of this problem.
Proof. See [169, Theorem 9.1.1] for the differentiable minimization problem or
[209, Chap. VII, Theorem 2.1.4] for the convex minimization problem. u
t
1.3.2 Variational Inequality
Let C H be closed convex and F W H ! H. The variational inequality problem

(VIP.F ; C /) is to find x 2 C such that
hF x; y xi 0 (1.17)
holds for all y 2 C . Note that, by Theorem 1.3.4, the differentiable convex
minimization problem (1.13) is a special case of the variational inequality. The proof
of the following theorem is similar to the proof of Corollary 1.3.5 and is left to the
reader.
Theorem 1.3.8. Let > 0. A point x 2 C is a solution of variational inequality
(1.17) if and only if x 2 Fix PC .Id F /.
For a general discussion on the existence of solutions of variational inequality
problems we refer the reader to [165, 226, 358]. Below we only recall sufficient
conditions for the existence and uniqueness of a solution of VIP.F ; C /.
Theorem 1.3.9. If F W H ! H is a strongly monotone and Lipschitz continuous
operator, then the variational inequality (1.17) has a unique solution x 2 C .
Proof. See, e.g., [358, Theorem 46.C]. t
u
1.3.3 Common Fixed Point Problem
T family of operators Ui W H ! H with Fix Ui ¤ ;, i 2 I , be given. If

Let a finite
F WD i 2ITFix Ui ¤ ;, then the common fixed point problem (CFPP) is to find a
point x 2 i 2I Fix Ui . In order to ensure that Fix Ui are closed and convex, some
additional assumptions on the operators Ui are usually made. The details will be
explained in Chap. 2.
1.3.4 Convex Feasibility Problem
Let a finite family ofTclosed convex subsets Ci , i 2 I , of a Hilbert space H be

given. Denote C WD i 2I Ci . If C ¤ ;, then the convex feasibility problem (CFP)
28 1 Introduction
is to find a point x 2 C . We see that the CFP can be formulated as the CFPP with
Ui WD PCi , i 2 I . On the other hand, the CFPP is a CFP with Ci D Fix Ui . In
applications concerning the CFP the subsets Ci are often called the constraints sets
or, shortly, constraints and are given in the form
Ci WD fx 2 H W ci .x/ 0g; (1.18)
where ci W H ! R are convex functions, called the constraints functions or, shortly,
constraints, i 2 I .
The CFP can also be considered without the assumption C ¤ ;. In this case
we should define an appropriate convex proximity function f W H ! RC which
measures a “distance” to the constraints. The proximity function should have the
property
f .x/ D 0 if and only if x 2 C . (1.19)
The convex feasibility problem can be formulated as a minimization of the
proximity function f . The proximity function f W H ! RC for the CFP can be
defined by the following general form
f .x/ WD F .f1 .x/; : : : ; fm .x//, (1.20)
where fi W H ! RC are convex functions with the property

D 0 for x 2 Ci
fi .x/ (1.21)
> 0 for x … Ci ,
C ! R is convex and increasing (or

i 2 I WD f1; 2; : : : ; mg, and the function F W Rm
at least nondecreasing) with respect to any variable and such that F .v/ D 0 if and
only if v D 0. In particular, if F .v/ D w> v for a vector w D .!1 ; : : : ; !m / 2 Rm
CC ,
the proximity function f has the form of a weighted sum of functions fi , i 2 I ,
X
f .x/ WD !i fi .x/. (1.22)
i 2I
If F .v/ WD maxi 2I i , where v D .1 ; : : : ; m /, the proximity function f has the

form
f .x/ D max fi .x/. (1.23)
i 2I
Two important examples of functions fi with properties (1.21) are fi WD d.; Ci /

and fi WD d 2 .; Ci /. We can also write fi .x/ D kPCi x xk in the first
case or fi .x/ D 12 kPCi x xk2 in the other one. Note that both functions are
convex and the other one is differentiable, as well. These facts will be proved in
Sect. 2.2.3 (see Lemmas 2.2.27 and 2.2.28 and Corollary 2.2.29). The computation
of these two functions requires, however, a simple structure of Ci . If we take
fi .x/D 12 kPCi x xk2 , i 2 I , in (1.22), then we obtain the following important

proximity function:
1X
f .x/ WD !i kPCi x xk2 . (1.24)
2 i 2I
In Chap. 4 we show that determining a minimizerP of the above function is equivalent
to finding a fixed point of the operator T D i 2I !i PCi (see Theorem 4.4.6).
When the subset Ci is given by (1.18), the function ciC WD maxf0; ci g expresses
a “distance” to the i th constraint, i 2 I . In this case we can define a function fi
satisfying (1.21) in the form
fi WD hi ı ciC (1.25)
for a convex and increasing function hi W RC ! RC such that hi .0/ D 0, i 2 I .
The function hi has often the form hi .u/ WD u or hi .u/ WD 12 u2 , i 2 I . In the first
case the proximity function f defined by (1.22) is a weighted sum of “distances” to
the constraints, i.e., X
f .x/ D !i ciC .x/,
i 2I
and, in the other one, f has the form
1X
f .x/ D !i .ciC .x//2 ,
2 i 2I
P
where w D .!1 ; : : : ; !m / 2 RmCC . One can additionally suppose that i 2I !i D 1
which leads to the assumption that w 2 ri m . This assumption does not change
the minimization problem. Both proximity functions are convex and the other one
is even differentiable.
As mentioned above, the CFP is, in general, to minimize a convex proximity
function f . Denote f WD infx2H f .x/. It is clear that f 0. We emphasize that
the proximity function f needs not to attain its minimum. If C ¤ ;, we have C D
Argminx2H f .x/ for any proximity function with the above described properties.
In this case, the problem of determining x 2 C and the problem of minimization
of the proximity function f are equivalent. Furthermore, in this case, a significant
information about f is available, namely f WD minx2H f .x/ D 0. One can say
that in the general case, i.e., without the assumption C ¤ ;, the CFP is to find an
element x 2 H, for which “distances” to the constraints Ci , i 2 I , are minimal in
some sense, if such x exists. For x 2 H, the value f .x/ expresses this “distance”.
If C D ; and the proximity function attains its minimum, then, of course, f > 0.
Note, however, that the nonemptiness of C does not follow from the fact that f WD
infx2H f .x/ D 0.
In some applications, the CFP is to find an element x 2 C or to state that C D ;.
We can also say that the CFP is to find a zero of the proximity function or to state
that the proximity function is positive for all arguments.
30 1 Introduction
1.3.5 Linear Feasibility Problem
In the case where all Ci are half-spaces, i.e.,
Ci WD H .ai ; ˇi / D fx 2 H W hai ; xi ˇi g
for ai 2 H, ai ¤ 0 and ˇi 2 R, i 2 I WD f1; 2; : : : ; mg, the CFP is called the linear

feasibility problem (LFP). If H D Rn , the linear feasibility problem is to solve a
system of linear inequalities
ai> x ˇi , i D 1; 2; : : : ; m, (1.26)
where ai 2 Rn , ai ¤ 0, ˇi 2 R, i 2 I , which can also be written in matrix form as
Ax b,
where A is a matrix with rows ai , i.e., A D Œa1 ; : : : ; am > and b D .ˇ1 ; : : : ; ˇm /.

By duality theorems, the linear feasibility problem in Rn is equivalent to the linear
programming problem (see, e.g., [4, Chap. 6]).
Similarly to the CFP, the linear feasibility problem can also be considered without
the assumption C ¤ ;. In this case, the problem is to minimize a proximity function.
One can apply general forms of a proximity function given by equalities (1.20)–
(1.25). Since the constraints for the LFP have the form ai> xˇi 0 or, equivalently,
kai k1 .ai> x ˇi / 0, as fi one often takes functions fi defined by
fi .x/ WD hi ..ai> x ˇi /C / (1.27)
or by
.ai> x ˇi /C
fi .x/ WD hi (1.28)
kai k
for some convex increasing functions hi W RC ! RC with hi .0/ D 0, i 2 I .
In most cases the functions hi are given by hi .u/ WD !i u or hi .u/ WD 12 !i u2 for
!i > 0, i 2 I . If we apply these definitions of fi and hi to formulas (1.20)–(1.25)
we obtain the following collection of proximity functions:
(a)
f .x/ WD max.ai> x ˇi /C , (1.29)
i 2I
in order to determine a point for which the value of the most violated constraint
is minimal,
(b)
.ai> x ˇi /C
f .x/ WD max , (1.30)
i 2I kai k
in order to determine a point for which the distance to the furthest constraint is
minimal,
(c) X
f .x/ WD !i .ai> x ˇi /C , (1.31)
i 2I
in order to determine a point for which the weighted sum of the values of the
constraints is minimal,
(d)
X .ai> x ˇi /C
f .x/ WD !i , (1.32)
i 2I
kai k
in order to determine a point for which the weighted sum of distances to the
constraints is minimal,
(e)
1X
f .x/ WD !i Œ.ai> x ˇi /C 2 , (1.33)
2 i 2I
in order to determine a point for which the weighted sum of squared values of
the constraints is minimal, or
(f)
> 2
1X .ai x ˇi /C
f .x/ WD !i , (1.34)
2 i 2I kai k
in order to determine a point for which the weighted sum of squared distances
to the constraints is minimal.
All of the proximity functions presented herein are convex. The last two functions
are even differentiable. If we denote the residuum of the i th constraint at a point
x by i .x/, i.e., i .x/ WD ai> x ˇi , i 2 I , and by r.x/ the residual vector,
i.e., r.x/ WD . 1 .x/; : : : ; m .x//, then the proximity function (1.31) has the form
f .x/ D w> rC .x/, where w D .!1 ; : : : ; !m / 2 Rm CC , and the proximity function
2
(1.33) has the form f .x/ D 2 krC .x/kW for W WD diag w. A system of linear
1
equations Ax D b is a special case of the LFP, because the system can be

presented as
A b
x .
A b
In this case the proximity function f given by (1.34) obtains the form
2
1X ai> x ˇi
f .x/ WD !i . (1.35)
2 i 2I kai k
Remark 1.3.10. If we multiply the inequality ai> x ˇi by a constant ˛i > 0,

i 2 I , we obtain an equivalent linear feasibility problem, regardless of the proximity
32 1 Introduction
function under consideration, whenever C ¤ ;. In the case, where C D ;, the new

LFP defined by linear inequalities
˛i ai> x ˛i ˇi ; i 2 I; (1.36)
is, in general, not equivalent to the original problem, even if we consider the
same proximity function as for the original problem. However, there are some
relationships between these two problems. The problems are equivalent if we use
the proximity functions (1.30), (1.32) or (1.34). If we apply the proximity functions
(1.31) or (1.33) with weights !i D ˛i1 i or !i D ˛i2 i , respectively, i 2 I , to
problem (1.36), then we obtain a new LFP which is equivalent to the original one
with the proximity function (1.31) or (1.33), respectively, and with weights !i D i ,
i 2 I . In particular, if we take ˛i D kai k1 , i 2 I , in (1.36), then the half-spaces in
the new LFP are defined by normalized vectors. Therefore, if we apply the proximity
function (1.33) with weights !i D kai k2 i , i 2 I , to the new LFP, then the problem
is equivalent to the original one with the proximity function (1.33) and with weights
!i D i , i 2 I . Note that we can suppose, without loss of generality, that the
vectors ai , i 2 I , are normalized. Otherwise, an appropriate change of the weights
leads to an equivalent problem, because the weights !i , i 2 I , can be multiplied
by a constant ˛ > 0 without changing the minimizers of the proximity function.
Therefore, we can suppose that w 2 ri m instead of w 2 Rm CC .
Example 1.3.11. (Linear least squares problem) Given a linear system Ax D b,

where A is a matrix of type m n, x 2 Rn and b 2 Rm , the problem
minimize 12 kAx bk2

(1.37)
subject to x 2 Rn
is called a linear least squares problem (LLSP). This problem is equivalent to the
following compatible system of linear equations
A> Ax D A> b (1.38)
Actually, any solution of (1.38) has the form x D x N C x, O where x N D AC b,

C
xO 2 ker A and A denotes the Moore–Penrose pseudoinverse of A. System (1.38)
is called a normal equation. The equivalence of (1.37) and (1.38) follows from the
necessary and sufficient condition for convex minimization (see Corollary 1.3.3). In
particular, any solution of a compatible system Ax D b is a solution of the LLSP. In
general, one can solve system (1.38) by so called SVD-decomposition of the matrix
A> A (for details see, e.g., in [181, 234, 317]). We show that a solution of (1.37)
(or, equivalently, (1.38)) with a minimal norm has the form x WD x N D AC b. This
solution is called the normal solution of problem (1.37). Consider the following
differentiable convex constrained minimization problem
minimize 12 kxk2
subject to A> .Ax b/ D 0 (1.39)
x 2 Rn .
It follows from the definition of the metric projection that a solution x of (1.39) is
equal to PL 0, where L WD fx 2 Rn W A> Ax D A> bg. The uniqueness of the metric
projection yields the uniqueness of x . The Lagrange function L W Rn Rn ! R
has the form
1
L.x; y/ D kxk2 C y > A> .Ax b/
2
and the KKT-system has the form
rx L.x; y/ D x C A> Ay D 0
(1.40)
ry L.x; y/ D A> .Ax b/ D 0.
By the KKT-theorem, this system has a solution .x; y/ 2 Rn Rn . It follows from

the definition of the Moore–Penrose pseudoinverse that
A> AAC b D A> .AAC /> b D .AAC A/> b D A> b.
Therefore, x N WD AC b satisfies the second equation of the KKT-system (1.40). By

sufficient optimality conditions for the differentiable convex minimization problem
(see Theorem 1.3.7), x N is a solution of (1.39). Therefore, x D x N . Subsuming,
x N WD AC b is the unique solution of (1.37).
Note that the LLSP is a special case of the LFP with the proximity function
defined by (1.35).
1.3.6 General Convex Feasibility Problem
In many applications of the convex feasibility problems one seeks a point, for
which some constraints are satisfied and a proximity function defined for the rest
of constraints is minimal. The constraints which should be satisfied are called hard
constraints and the rest of constraints are called soft constraints. We call the problem
a general convex feasibility problem (GCFP). In other words, the GCFP is to find
a point satisfying the hard constraints for which “distances” to soft constraints are
minimal in some sense. Such model was described in details in [123].
Let fCi gi 2I be a family of closed convex subsets. Let Ci for i 2 Ih be the
hard constraints and Ci for i 2 Is beTthe soft constraints,Twhere I D Ih [ Is and
Ih \ Is D ;. Furthermore, let Ch WD i 2Ih Ci and Cs WD i 2Is Ci . If Ch \ Cs ¤ ;,
then GCFP is to find x 2 Ch \Cs . If we do not suppose Ch \Cs ¤ ;, then, similarly
as for the CFP, one defines a convex proximity function f W Ch ! RC , defined on
the subset Ch of hard constraints, which measures “distances” to the soft constraints.
Similarly as for the CFP, we require the proximity function f to have the property
34 1 Introduction
Fig. 1.7 Split feasibility

H1 A H2
problem
A−1 (Q)
Ax
x
C Q
f .x/ D 0 if and only if x 2 Ch \ Cs . As a proximity function one can use functions

described in Sect. 1.3.4, e.g., the function f W H ! RC defined by (1.24) with I
replaced by Is .
1.3.7 Split Feasibility Problem
In some applications the general convex feasibility problem has the form: Given
closed convex subsets C H1 , Q H2 of Hilbert spaces H1 ; H2 and a bounded
linear operator A W H1 ! H2 , find a point x 2 C such that Ax 2 Q (if such a point
exists) (see Fig. 1.7). If H1 ; H2 are Euclidean spaces, then the above problem has
the form: Given closed convex subsets C Rn , Q Rm and a matrix A of type
m n, find a point x 2 C such that Ax 2 Q (if such a point exists).
The subset C is a hard constraint and the subset D WD A1 .Q/ D fx 2 H1 W
Ax 2 Qg is a soft constraint. It is clear that D is closed convex, because for a
bounded linear operator A, the inverse image of a closed convex subset Q is closed
and convex. This problem was introduced by Censor and Elfving [88] and was called
a split feasibility problem (SFP). If we do not suppose that C \ D ¤ ;, then,
similarly as for the GCFP, we introduce a convex proximity function f W C ! RC
and we present the SFP as minimization of f . Byrne [55, Sect. 2] has proposed a
proximity function defined by
1
PQ .Ax/ Ax 2 .
f .x/ WD (1.41)
2
The function f is convex as a composition of a linear operator

A and a convex
function 12 d 2 .; Q/. Note that d.Ax; Q/ D PQ .Ax/ Ax . The function defined
by (1.41) measures the square distance of the image of a point x 2 C by the operator
A to the subset Q. The function f has the required property:
x 2 C \ D , f .x/ D 0.
Practical applications of the SFP require a simple structure of C and Q, which

allows an easy computation of PC .x/ and PQ .y/ for any x 2 H1 and y 2 H2 .
1.3.8 Linear Split Feasibility Problem
Now consider the SFP in the finite dimensional case. If the subset Q Rm has the
form
Q WD fy 2 Rm W y bg, (1.42)
where b 2 Rm , then the problem is called a linear split feasibility problem (LSFP).
In other words, the problem can be written as follows: find x 2 C such that Ax b
(if such a point exists). In general, we do not suppose, however, that C \ fx 2 Rn W
Ax bg ¤ ;. Now we present a few examples of a proximity function for the LSFP
which are often used in applications. In Sect. 4 we will show that for the subset Q
defined by (1.42) we have
PQ .Ax/ Ax D b Ax.
(see Example 4.1.4). Therefore, the proximity function f W C ! R defined by

(1.41), which is applied to the LSFP can be written in the form
1X >
m
1
f .x/ WD krC .x/k2 D Œ.a x ˇi /C 2 .
2 2 i D1 i
Note that f is differentiable. Alternatively, one can also apply a proximity function
given by
Xm
f .x/ WD w> rC .x/ D !i .ai> x ˇi /C
i D1
or by
1X
m
f .x/ WD krC .x/k2W D !i Œ.ai> x ˇi /C 2 , (1.43)
2 i D1
where W WD diag w for w D .!1 ; : : : ; !m / 2 Rm

CC . The latter proximity function is
used in many applications.
1.3.9 Multiple-Sets Split Feasibility Problem
The following problem, called a multiple-sets split feasibility problem, has been
studied in the literature (see, e.g., [93, 103, 257, 344]): Given closed convex subsets
Ci H1 , i D 1; 2; : : : ; p, Qj H2 , j D 1; 2; : : : ; r, of Hilbert spaces H1 ; H2 ,
respectively, and a bounded linear
T p Tr operator A W H1 ! H2 , find a point x 2 C WD
C
i D1 i such that Ax 2 Q WD j D1 Qj .
This problem can be considered as a split feasibility problem with the proximity
function given by (1.41). The application of this proximity function requires,
however, a simple structure of C and Q, which allows an easy computation of PC x
36 1 Introduction
and PQ y for any x 2 Rn and y 2 Rm . If the structure of Ci , i D 1; 2; : : : ; p, and

Qj , j D 1; 2; : : : ; r, is simple, one can also use the following proximity function
f W Rn ! R proposed in [93]:
1X 1X
p r
2
f .x/ WD kPCi x xk2 C ˇj PQj Ax Ax .
2 i D1 2 j D1
Note that in this case the problem is a special case of the general convex feasibility
problem, where all subsets Ci , i D 1; 2; : : : ; p, and Qj , j D 1; 2; : : : ; r, are treated
as soft constraints. If Ch has a simple structure allowing an easy computation of
PCh x for any x 2 Rn , one can also use the proximity function f W Ch ! R
given by
1X 2
r
f .x/ WD ˇj PQj Ax Ax .
2 j D1
In this case the problem is a special case of the general convex feasibility problem,
where Ci , i D 1; 2; : : : ; p, are hard constraints and Qj , j D 1; 2; : : : ; r, are soft
constraints.
1.4 Exercises
Exercise 1.4.1. Prove that in a Hilbert space the Cauchy–Schwarz inequality and
the triangle inequality are equivalent.
Exercise 1.4.2. Let w D .!1 ; !2 ; : : : ; !m / 2 m . Prove that
m 2
X X m
1 X
m
2

!i xi D !i kxi k2 !i !j xi xj . (1.44)
2 i;j D1
i D1 i D1
Exercise 1.4.3. Prove the parallelogram law (see equality (1.8)).

Exercise 1.4.4. Prove Theorem 1.1.1.
Exercise 1.4.5. Let G be a positive definite matrix of type m m. Prove that the
function h; iG defined by
hw; viG WD w> Gv
is an inner product in Rm .
Exercise 1.4.6. Let H D l2 and ek D .ek1 ; ek2 ; : : :/ with

1 if j D k
ekj D
0 if j ¤ k,
j; k 0. Prove that fek g1

kD1 converges weakly in l2 but does not converge strongly.
1.4 Exercises 37
Exercise 1.4.7. Let f W H ! R be a convex function. Prove that:

(a) The sublevel set S.f; ˛/ is convex for any ˛ 2 R.
(b) The subset Argminx2H f .x/ is convex.
Exercise 1.4.8. Prove that the epigraph of a function f W H ! R is a convex
subset if and only if f is convex.
Exercise 1.4.9. Let A W H1 ! H2 be an affine operator, C H1 and D H2 be
convex subsets. Show that A.C / and A1 .D/ H1 are convex.
Exercise 1.4.10. Evaluate derivatives of the following functions f W H ! R:
(a) f .x/ D ha; xi,
(b) f .x/ D 12 hAx; xi, where A W H ! H is a bounded linear operator.
Exercise 1.4.11. Evaluate derivatives of the following functions f W H1 ! R:
(a) f .x/ D 12 kAx bk2 , where A W H1 ! H2 is a bounded linear operator and
b 2 H2 , 2
(b) f .x/ D 12 PQ .Ax/ Ax , where A W H1 ! H2 is a bounded linear operator
and Q H2 is a closed convex subset.
Exercise 1.4.12. Let A be an m n matrix. Prove that the matrix AA> is positive
semi-definite and that AA> is positive definite if and only if A has full row rank,
i.e., the rows of A are linearly independent.
Exercise 1.4.13. Let A be a positive definite matrix of type n n and min .A/
denote the smallest eigenvalue of A. Prove that:
(a) 12 x > Ax min .A/ kxk2 for any x 2 Rn ,
(b) The function f W Rn ! R defined by f .x/ WD 12 x > Ax is convex.
Exercise 1.4.14. Let C H be a polytope, i.e., C D fx 2 H W hai ; xi ˇi ,
i 2 I g and let x 2 C . Show that
NC x D conefai W i 2 I.x/g,
where I.x/ D fi 2 I W hai ; xi D ˇi g.

Exercise 1.4.15. Let m D fu 2 Rm W u 0; e > u D 1g denote a standard simplex
in Rn . Show that
ri m D fu 2 Rm W u > 0; e > u D 1g.
Exercise 1.4.16. Let C H be a nonempty convex subset. Show that the distance
function d.; C / W H ! R defined by d.x; C / WD infy2C kx yk is convex.
Exercise 1.4.17. Prove Lemma 1.2.5.
Exercise 1.4.18. Let f W H ! R be a differentiable convex function. Show that
the function h W H ! RC defined by h.x/ D ..f .x//C /2 is differentiable and
convex.
38 1 Introduction
Exercise 1.4.19. Let fi W X ! R, i D 1; 2; : : : ; m, be convex functions and

F W Rm ! R be a convex function which is nondecreasing with respect to any
coordinate. Show that the function f WD F .f1 ; : : : ; fm / is convex.
Exercise 1.4.20. Let A be a positive definite matrix of type n n. Show that the
function f W Rn ! R, f .x/ D 12 x > Ax is strongly convex.
Exercise 1.4.21. Consider the following problem: Find a point x 2 RnC for which
the sum of squares of distances to the balls Ci WD B.ai ; i /, where ai 2 Rn ,
i 0, i D 1; 2; : : : ; m, is minimal. Present the problem as a convex feasibility
problem and as a generalized convex feasibility problem, where RnC is a subset of
hard constraints. Write corresponding proximity functions.
Exercise 1.4.24. Prove Theorem 1.3.8.
Chapter 2
Algorithmic Operators
In Chap. 5 we will present several methods for solving convex optimization

problems. We will focus our study on iterative methods (we also call them iterative
procedures or algorithms) which are given in the form of the following recurrence
x kC1 D Tk x k (2.1)
defined on a closed convex subset X H, where Tk W X ! X is a sequence of

operators. We suppose that the starting point x 0 is an element of a starting subset
X0 X . Usually, one supposes that X0 D X . A sequence fx k g1 kD0 generated by
the iterative method (2.1) is called an approximating sequence. If Tk D T for all
k 0, then this sequence is called an orbit of T . Any iterative method for solving
a convex optimization problem is constructed in such a way that the approximating
sequences fx k g1
kD0 generated by this method converge (at least weakly) to a solution
of the optimization problem. As we will see, the solution is a fixed point of an
operator S W X ! H, which is usually a nonexpansive one. The form of this
operator depends on the considered optimization problem. A sequence of operators
Tk whichTdefines the iterative method is usually constructed in such a way that
Fix S 1 kD0 Fix Tk .
In this chapter we deal with general properties of operators which define
algorithms for solving convex optimization problems. In one iteration of the
algorithm an appropriate operator T W X ! X defines an actualization, also
called an update x C of the current approximation x of a solution of the convex
optimization problem. Usually, this actualization has the form x C D T x. We call T
an algorithmic operator. One can also consider algorithms, where the actualization
has the form x C 2 T x for a mapping (multifunction) T W X X . In this case, T
is called an algorithmic mapping.
Operators defining iterations of an algorithm usually depend on some parameters
which are constant or vary during the iteration process. The properties of approx-
imating sequences depend on the properties of algorithmic operators defining the
iterative method as well as on the choice of parameters defining these operators.

40 2 Algorithmic Operators
2.1 Basic Definitions and Properties
Let H be a Hilbert space. In what follows, we consider operators which are defined
on a nonempty closed convex subset X H.
P
Remark 2.1.1. Let Ui W X ! X , i 2 I WD f1; 2; : : : ; mg. If (i) U D i 2I !i Ui ,
where w D .!1 ; !2 ; : : : ; !m / 2 m , or (ii) U D Um Um1 : : : U1 , then the following
obvious inclusion holds \
Fix Ui Fix U
i 2I
The converse inclusion needs not to be true even if all Ui , i 2 I , have a common
fixed point (see Example 2.1.27).
Definition 2.1.2. Let T W X ! H and 2 Œ0; 2. The operator T W X ! H
defined by
T WD .1 / Id CT
is called a -relaxation or, shortly, relaxation of the operator T . Obviously, T D
Id C.T Id/. We call a relaxation parameter. If 2 .0; 1/, then T is called an
under-relaxation of T . If 2 .1; 2/, then T is called an over-relaxation of T and
if D 2, then T is called the reflection of T . If 2 .0; 2/, then T is called a strict
relaxation of T .
A relaxation T of an operator T can be defined for any 2 R. However, if we
do not extend explicitly the range of , we assume that 2 Œ0; 2.
Remark 2.1.3. Note that the equality .T / D T holds for all ; 2 R, con-
sequently .T /1 D T for ¤ 0.
Remark 2.1.4. It is clear that Fix T D Fix T whenever ¤ 0.
Let Ui W X ! X , i 2 I WD f1; 2; : : : ; mg, U WD Um Um1 : : : U1 and Qi WD
Ui Ui 1 : : : U1 Um : : : Ui C1 , i D 1; 2; : : : ; m. Denote Q0 WD Qm D U and U0 WD
Um . There exists a relationship among the subsets of fixed points of operators Qi ,
which is expressed by the following theorem.
Theorem 2.1.5. For i D 1; 2; : : : ; m there holds
Fix Qi D Ui .Fix Qi 1 /. (2.2)
Proof. Let i 2 I . First we prove the inclusion
Fix Qi Ui .Fix Qi 1 /. (2.3)
Let zi 1 2 Fix Qi 1 and zi D Ui zi 1 . Then we have
zi D Ui zi 1 D Ui Qi 1 zi 1 D Ui Ui 1 : : : U1 Um : : : Ui C1 Ui zi 1
D Ui Ui 1 : : : U1 Um : : : Ui C1 zi D Qi zi ,
2.1 Basic Definitions and Properties 41
which proves that (2.3) holds for any i 2 I . Consequently,
Fix Qi Ui .Fix Qi 1 / Ui Ui 1 .Fix Qi 2 /

: : : Ui Ui 1 : : : U1 .Fix Q0 / D Ui Ui 1 : : : U1 .Fix Qm /
Ui Ui 1 : : : U1 Um .Fix Qm1 /
: : : Ui Ui 1 : : : U1 Um Um1 : : : Ui C1 .Fix Qi /
D Qi .Fix Qi / D .Fix Qi /,
i 2 I , and all inclusions are, actually, equations. In particular, Fix Qi D

Ui .Fix Qi 1 /, i.e., (2.2) is satisfied for all i 2 I . t
u
2.1.1 Nonexpansive Operators
Definition 2.1.6. We say that an operator T W X ! H is:

(i) Nonexpansive (NE), if
kT x T yk kx yk
for all x; y 2 X ,
(ii) Strictly nonexpansive if
kT x T yk < kx yk or x y D T x T y
for all x; y 2 X ,
(iii) An ˛-contraction, where ˛ 2 .0; 1/ or, shortly, a contraction if
kT x T yk ˛ kx yk
for all x; y 2 X .
The theorem below, called the Banach fixed point theorem or the Banach theorem
on contractions, is widely applied in various areas of mathematics. The theorem
holds for any complete metric space, and hence, in particular, for every closed subset
of a Hilbert space.
Theorem 2.1.7 (Banach, 1922). Let X be a complete metric space and T W X !
X be a contraction. Then T has exactly one fixed point x 2 X . Furthermore,
for any x 2 X , the orbit fT k xg1
kD0 converges to x with a rate of geometric
progression.
Proof. See, e.g., original paper of Banach [15], [185, Theorem 1.1], [267, Theorem
24.2], [184, Theorem 2.1], [183, Theorem 2.1] or [36, Theorem 2.1]. t
u
The Banach fixed point theorem is a widely applied tool for an iterative approxi-
mation of fixed points. Unfortunately, its application is restricted to contractions. We
will need, however, appropriate tools for an iterative approximation of fixed points
of nonexpansive operators T with Fix T ¤ ;.
Below, we present several classical fixed points theorems.
Theorem 2.1.8 (Brouwer, 1912). Let X Rn be nonempty compact and convex
and T W X ! X be continuous. Then T has a fixed point.
Proof. See, e.g., original paper of Brouwer [43] or [191, Chap. II, 5, Theorem 7.2]
or [183, Theorem 7.6]. t
u
The Brouwer fixed point theorem was generalized by Juliusz Schauder.
Theorem 2.1.9 (Schauder, 1930). Let X be a nonempty compact and convex
subset of a Banach space and T W X ! X be continuous. Then T has a fixed point.
Proof. See, e.g., original paper of Schauder [302] or [191, Chap. II, 6, Theorem
3.2] or [183, Theorem 8.1]. t
u
For nonexpansive operators in a Hilbert space H the compactness of X H in the
Schauder theorem can be replaced by the boundedness of X . The following theorem
was proved independently by Browder [45, Theorem 1], Göhde [188] and by Kirk
[227]. The proof can also be found, e.g., in [185, Theorem 5.1], [191, Chap. I, 4,
Theorem 1.3], [183, Theorem 4.1] or [36, Theorem 3.1].
Theorem 2.1.10 (Browder–Göhde–Kirk, 1965). Let X be a nonempty closed,
convex and bounded subset of a uniformly convex Banach space (e.g., of a Hilbert
space H) and U W X ! X be nonexpansive. Then U has a fixed point.
Contrary to the Banach fixed point theorem, the theorems of Brouwer, Schauder
and of Browder–Göhde–Kirk are only of existential nature. In Chap. 3 we present
theorems which can be applied to iterative methods for determining fixed points of
nonexpansive operators.
Below, we present some properties of the subset of fixed points of a nonexpansive
operator. The following result can be found in [185, Proposition 5.3].
Proposition 2.1.11. The subset of fixed points of a nonexpansive operator T W
X ! H is closed and convex.
Proof. (cf. [185, Proposition 5.3]) Let x k 2 Fix T and x k ! x. We have x 2 X
because X is closed. By the continuity of T ,
x D lim x k D lim T x k D T x,
k
i.e., Fix T is a closed subset. Now we show the convexity of Fix T . Let x; y 2 Fix T ,
x ¤ y and z D .1 /x C y for 2 .0; 1/. By the nonexpansivity of T and by
the positive homogeneity of the norm we have
kx T zk D kT x T zk kx zk D kx yk (2.4)
and
kT z yk D kT z T yk kz yk D .1 / kx yk . (2.5)
Now, the triangle inequality yields
kx yk kx T zk C kT z yk
kx yk C .1 / kx yk
D kx yk .
Consequently,
kx yk D kx T zk C kT z yk .
By the strict convexity of the norm, the vectors x T z and T z y are positive
linearly dependent. Therefore, ˛.x T z/ C ˇ.y T z/ D 0 for some ˛; ˇ 0.
ˇ
Since x ¤ y, it follows that ˛ C ˇ > 0, and hence, T z D ˛Cˇ
˛
x C ˛Cˇ y. Now, the
nonexpansivity of T and inequalities (2.4) and (2.5) yield
ˇ
kx yk D kx T zk D kT x T zk kx zk D kx yk (2.6)
˛Cˇ
and
˛
kx yk D kT z yk D kT z T yk kz yk D .1 / kx yk . (2.7)
˛Cˇ
If at least one inequality in (2.6) and (2.7) is strict, then by summing up (2.6)
ˇ
and (2.7) we would obtain a contradiction. Therefore, ˛Cˇ D and ˛Cˇ
˛
D .1 /,
consequently T z D .1 /x C y D z. t
u
The closedness and convexity of the subset of fixed points of a nonexpansive
operator follows also from a property presented in Sect. 2.2 (see Corollary 2.2.48).
Lemma 2.1.12. Let Si W X ! X , i 2 I WD f1; 2; : : : ; mg, be nonexpansive.
Then:
P
(i) A convex combination S WD i 2I !i Si , where w D .!1 ; : : : ; !m / 2 m , is
nonexpansive. If, furthermore, at least one operator Si is a contraction and the
corresponding weight !i > 0, then S is a contraction;
(ii) A composition S WD Sm Sm1 : : : S1 is nonexpansive. If, furthermore, at least
one operator Si is a contraction, then S is a contraction.
Proof. Let x; y 2 X and Si be nonexpansive, i.e., kSi x Si yk ˛i kx yk,
where ˛i 2 .0; 1, i 2 I .
P P
(i) Let w 2 m , S WD i 2I !i Si and ˛ D j 2I !j ˛j . It is clear that ˛ 2 .0; 1.
By the convexity of the norm and the nonexpansivity of Si , i 2 I , we have

X

kS x Syk D !i .Si x Si y/

i 2I
X
!i kSi x Si yk
i 2I
X
!i ˛i kx yk
i 2I
X !i ˛i
D P ˛ kx yk
i 2I j 2I !j ˛j
D ˛ kx yk ,
i.e., S is a nonexpansive operator. Now suppose that Si0 is a contraction, i.e.,

˛i0 < 1 and that !i0 > 0, for some i0 2 I . Then ˛ 2 .0; 1/, i.e., S is a
contraction.
(ii) We have
kS x Syk D kSm Sm1 : : : S1 x Sm Sm1 : : : S1 yk ˛ kx yk ,
where ˛ D ˛m ˛m1 : : : ˛1 2 .0; 1. If Si0 is a contraction for some i0 2 I , i.e.,

˛i0 2 .0; 1/, then, of course, ˛ 2 .0; 1/ and S is a contraction. t
u
Theorem 2.1.13. Let Ui W X ! X be nonexpansive for all i 2 I WD f1; 2; : : : ; mg,
and U WD Um Um1 : : : U1 . If Uj .X / is bounded for at least one j 2 I , then
Fix U ¤ ;.
Proof. Let Uj .X / be bounded for some j 2 I . Since Ui are nonexpansive, i 2 I;
the boundedness of Uj .X / yields the boundedness of U.X /. Therefore, Y WD
cl conv U.X / is closed, convex and bounded. Since U.X / X and X is closed
and convex, we have Y X . The operator U jY maps a closed, convex and
bounded subset Y into itself. By the Browder–Göhde–Kirk Fixed Point Theorem,
the operator U jY has a fixed point z 2 Y . Of course, U z D U jY .z/ D z. t
u
Theorem 2.1.14. Let Ui W X ! H, i 2 I WD P f1; 2; : : : ; mg, be nonexpansive
operators with a common fixed point and U WD i 2I !i Ui with w 2 ri m . Then
\
Fix U D Fix Ui .
i 2I
T
Proof. The inclusion i 2I Fix Ui Fix U is always true (see RemarkT 2.1.1). Now
we show that the converse inclusion
T also holds. Let z 2 Fix U and u 2 i 2I Fix Ui .
If z D u, then, of course, z 2 i 2I Fix Ui . Otherwise, for z ¤ u, by the convexity
of the norm and by the nonexpansivity of Ui , i 2 I , we have
kz uk D kU z uk

X X

D !i Ui z u D !i .Ui z u/

i 2I i 2I
X X
!i kUi z uk D !i kUi z Ui uk
i 2I i 2I
X
!i kz uk D kz uk .
i 2I
Consequently,

X X X

!i .Ui z u/ D !i kUi z uk D !i kz uk . (2.8)

i 2I i 2I i 2I
Since !i > 0 for all i 2 I , the first equality in (2.8) yields a positive linear
dependence of all pairs of vectors Ui z u and Uj z u, i; j 2 I , i ¤ j , i.e.,

kUi z uk .Uj z u/ D Uj z u .Ui z u/. (2.9)
The second equality in (2.8), together with the inequality kUi z uk kz uk,
i 2 I , and the assumption !i > 0, i 2 I , yield
kUi z uk D kz uk (2.10)
for all i 2 I . Since z ¤ u, we have Ui z ¤ u, i 2 I . Now, it follows from (2.9)

and (2.10) that Ui z D v for all i 2 I and for some v 2 H. Consequently,
X X
z D Uz D !j Uj z D !j v D v D Ui z,
j 2I j 2I
T
for all i 2 I , i.e., z 2 i 2I Fix Ui . t
u
2.1.2 Quasi-nonexpansive Operators
Definition 2.1.15. We say that an operator T W X ! H is:

(i) Fejér monotone (FM) with respect to a nonempty subset C X if
kT x zk kx zk
for all x 2 X and z 2 C ,

Fig. 2.1 Equivalence (2.11)
x z
x+ y
2
(ii) Strictly Fejér monotone with respect to a nonempty subset C X if
kT x zk < kx zk
for all x … C and z 2 C .
Remark 2.1.16. Because of the following obvious equivalence

yCx
kz yk kz xk ” z ;y x 0 (2.11)
2
for arbitrary x; y; z 2 H (see Fig. 2.1), an operator T W X ! H is Fejér monotone
with respect to C if and only if

Tx x
z ; T x x 0. (2.12)
2
Furthermore, T is strictly Fejér monotone if and only if inequality (2.12) is strict
for all x … C . We have not supposed that C is closed convex in Definition 2.1.15.
Inequality (2.12) yields, however, that if T is (strictly) Fejér monotone with respect
to C , then T is (strictly) Fejér monotone with respect to conv C . Furthermore, the
continuity of the norm yields that if T is Fejér monotone with respect to C , then T
is Fejér monotone with respect to cl C . Therefore, we can suppose, without loss of
generality, that C is closed convex in Definition 2.1.15 (i) and that C is convex in
Definition 2.1.15 (ii).
There exists the largest subset, with respect to which an operator T is Fejér
monotone. This subset is closed and convex, as follows from the following lemma.
Lemma 2.1.17. Let T W X ! H. If the subset
\
Tx Cx

Fej T WD z2X W z ;Tx x 0 (2.13)
x2X
2
is nonempty, then Fej T is the largest subset, with respect to which T is Fejér
monotone.
Proof. The assertion follows directly from the equivalence (2.11). t

u
Remark 2.1.18. Because of frequent use we state some obvious properties of Fejér
monotone operators:
(i) If T is (strictly) Fejér monotone with respect to a nonempty subset C H,
then for an arbitrary 2 .0; 1/ its relaxation T is also (strictly) Fejér
monotone with respect to C .
(ii) If T is (strictly) Fejér monotone with respect to a nonempty subset C H,
then T is (strictly) Fejér monotone with respect to any nonempty subset
D C.
(iii) Every composition and every convex combination of operators which are Fejér
monotone with respect to a nonempty subset C H is Fejér monotone with
respect C .
Definition 2.1.19. We say that an operator T W X ! H having a fixed point is:
(i) Quasi-nonexpansive (QNE) if T is Fejér monotone with respect to Fix T , i.e.,
kT x zk kx zk
for all x 2 X and z 2 Fix T ,

(ii) Strictly quasi-nonexpansive (sQNE) if T is strictly Fejér monotone with
respect to Fix T , i.e.,
kT x zk < kx zk
for all x … Fix T and z 2 Fix T ,
(iii) C -strictly quasi-nonexpansive (C -sQNE), where C ¤ ; and C Fix T , if T
is quasi-nonexpansive and
kT x zk < kx zk
for all x … Fix T and z 2 C .

For an operator T having a fixed point the following relation is clear:
T is sQNE H) T is C -sQNE
where C Fix T . Furthermore, by definition,
T is Fix T -sQNE H) T is sQNE.
The metric projection onto a closed convex subset is a typical example of a strictly
quasi-nonexpansive operator.
A nonexpansive and strictly Fejér monotone operator is also called attracting
(see [22, Definition 2.1]). Yamada and Ogura use the name an attracting quasi-
nonexpansive operator for a strictly quasi-nonexpansive one (see [346, page
623]). Vasin and Ageev call these operators strongly Q-quasi-nonexpansive
(see [333, Definition 2.2]). Reich and Zaslavski define a more general operator
than the strictly quasi-nonexpansive one and call it an F -attracting mapping,
where F D Fix T (see [297, Sect. 1]). A continuous strictly quasi-nonexpansive
operator is also called a paracontraction (see, [164, Definition 1]). The class of
quasi-nonexpansive operators is denoted in [126, page 161] by F 0 . Properties
of quasi-nonexpansive operators in metric spaces have been intensively studied
since 1969 (see, e.g., [145, 148, 283], [50, Sect. 1], [113]), but the name quasi-
nonexpansive was introduced by Dotson [147].
Lemma 2.1.20. A nonexpansive operator U W X ! H with a fixed point is quasi-
nonexpansive.
Proof. Let U be nonexpansive and z 2 Fix U . Then
kUx zk D kUx U zk kx zk ,
i.e., U is quasi-nonexpansive. t
u
It is clear that the class of nonexpansive operators having a fixed point is an essential
subclass of quasi-nonexpansive operators, because a quasi-nonexpansive operator
needs not to be continuous. Moreover, a quasi-nonexpansive operator needs not to
be nonexpansive even if it is continuous (see Exercise 2.5.2). In this section we
present properties of the family of quasi-nonexpansive operators. In further parts of
the book we show that these operators play an important role in iterative methods
for fixed point problems.
The following lemma gives a relation between the subset Fej T and the subset
Fix T for an operator T W X ! H (cf. [24, Proposition 2.6 (ii)]).
Lemma 2.1.21. For any operator T W X ! H the inclusion Fej T Fix T holds.
If Fix T ¤ ; and T is quasi-nonexpansive, then the converse inclusion also holds.
Consequently, the subset of fixed points of a quasi-nonexpansive operator is closed
and convex.
Proof. If Fej T D ;, then the first part of the assertion is obvious. Now let
Fej T ¤ ; and w 2 Fej T . Then, for z D x D w in (2.13), we obtain
TwCw
0 hw ; T w wi
2
1
D kT w wk2 0,
2
i.e., T w D w. Therefore, Fej T Fix T . Now suppose that Fix T ¤ ; and that
T is quasi-nonexpansive, i.e., T is Fejér monotone with respect to Fix T . Then,
Lemma 2.1.17 yields the inclusion Fix T Fej T , which together with the first
part of the lemma gives Fix T D Fej T . The convexity and the closedness of Fix T
follows now from Lemma 2.1.17 and from the fact that the intersection of closed
half-spaces is closed and convex. t
u
Fig. 2.2 Nonconvex Fix T

for a Fejér monotone y
operator T
h
x = Tx
»2
Ty
a »1 b
Fix T
−h
Remark 2.1.22. It follows from Remark 2.1.18 (ii), Lemmas 2.1.17 and 2.1.21 that
a quasi-nonexpansive operator T W X ! X is Fejér monotone with respect to any
nonempty subset of Fix T . Therefore, we will restrict our further consideration of
Fejér monotone operators to quasi-nonexpansive ones. Note, however, that without
the quasi nonexpansivity of T the equality Fix T D Fej T needs not to be true. In
this case, Fix T needs not to be convex, even if T is Fejér monotone.
Example 2.1.23. Let H D R2 , X WD Œa; b R for 1 a b C1
and h W X ! RC be a function with infx2Œa;b h.x/ D 0. Define the operator T W
X ! R2 by
x if j2 j h.1 /
T x WD
.1 ; 0/ if j2 j > h.1 /,
where x D .1 ; 2 / (see Fig. 2.2).
The reader may check that Fej T D Œa; b f0g and that Fix T D fx 2 X W j2 j
h.1 /g. If h is positive in at least one point, then Fej T ¤ Fix T . If, moreover, h is
not concave, then Fix T is not convex.
Let Ui W X ! X , i 2 I WD f1; 2; : : : ; mg, and:
P
(i) U WD i 2I !i Ui , where w D .!1 ; !2 ; : : : ; !m / 2 m or
(ii) U WD Um Um1 : : : U1 .
As we observed before, the following inclusion holds
\
Fix Ui Fix U (2.14)
i 2I
(see Remark 2.1.1) and the converse inclusion holds in case (i) when all Ui ,
i 2 I , are nonexpansive operators with a common fixed point and w 2 ri m (see
Theorem 2.1.14). It turns out that, in both cases (i) and (ii), the inclusion converse
to (2.14)) is true for strictly quasi-nonexpansive operators (see [22, Proposition
2.12], where the property was formulated for attracting operators). P In case (i) this
property is also true for a more general form of the operator U D i 2I !i U , where
the weights !i , i 2 I , may depend on x.
Definition 2.1.24. A function w W X ! m , with w.x/ D .!1 .x/; : : : ; !m .x// is

called a weight function.
Definition 2.1.25. Let Ui W X ! H, i 2 I . We say that the weight function w W
X ! m is appropriate
T with respect to the family fUi gi 2I or, shortly, appropriate,
if for any x … i 2I Fix Ui there exists an index j 2 I such that
!j .x/ k Uj x x k ¤ 0. (2.15)
Denote
I.x/ WD fi 2 I W x … Fix Ui g (2.16)
for a family of operators Ui W X ! H, i 2 I . The subset I.x/ is called a subset of
violated constraints. Note that w is appropriate if and only if
wj .x/ > 0 for some j 2 I.x/ (2.17)

T
and for any x … i 2I Fix Ui (or, equivalently, for any x 2 H such that I.x/ ¤ ;).
A weight function w W X ! ri m is appropriate with respect to any family of
operators fUi gi 2I if:
(i) w 2 ri m is a vector of constant weights (this case was considered in [22,
Proposition 2.12]), or if
(ii) wi .x/ > 0 for all x … Fix Ui and for all i 2 I .
It is clear that property (2.15) is weaker than conditions (i) and (ii) above.
The followingT theorem extends important results of [22, Proposition 2.12],
where C D i 2I Fix Ui and only constant weights are considered (see also [25,
Proposition 2.5] for a related result). These extended results will be applied in
further parts of the book.
T
Theorem 2.1.26. Let the operators Ui W X ! TX , i 2 I , with i 2I Fix Ui ¤ ;,
be C -strictly quasi-nonexpansive, where C i 2I Fix Ui , C ¤ ;. If U has one of
the following forms:
P
(i) U WD i 2I !i Ui and the weight function w W X ! m is appropriate,
(ii) U WD Um Um1 : : : U1 ,
then \
Fix U D Fix Ui (2.18)
i 2I
and U is C -strictly quasi-nonexpansive.

T
Proof. The inclusion i 2I Fix Ui FixTU is obvious. Now we show that Fix U
T
Ti 2I Fix Ui . This inclusion
T is clear if i 2I Fix Ui D X . Now suppose that x …
i 2I Fix U i . Let z 2 i 2I Fix Ui . If z 2 C , then the C -strict quasi nonexpansivity
of Ui , i 2 I , yields
kUi x zk < kx zk for any i 2 I.x/: (2.19)

P
(i) Let Ux D i 2I !i .x/Ui x, where the weight function w W X ! m is
appropriate. Then the convexity of the norm, (2.19) and (2.15) yield

X

kUx zk D !i .x/.Ui x z/

i 2I
X X
!i .x/ kUi x zk !i .x/ kx zk D kx zk ,
i 2I i 2I
where the second inequality is strict if z 2 C .

(ii) Let j WD minfi 2 I W x … Fix Ui g. Then we have Uj Uj 1 : : : U1 x D Uj x
and (2.19) yields
kUx zk D kUm : : : U1 x zk

D Um : : : Uj x z

Um1 : : : Uj x z

: : : Uj x z kx zk ,
where the latter inequality is strict if z 2 C .

Now it is clear that x … Fix U because, otherwise, for z 2 C , in both cases (i)
and (ii) we would obtain
kx zk D kUx zk < kx zk ,
T
a contradiction. We have proved that Fix U i 2I Fix Ui . Hence, (2.18) holds
and, in both cases (i) and (ii), U is C -strictly quasi-nonexpansive. t
u
Note that equality (2.18) needs not to be true for nonexpansive operators, even if
they have a common fixed point.
Example 2.1.27. (cf. [22, Remark 2.11]) Let X H be a subspace with dim X > 0.
Let Ui W X ! X , Ui WD Id, i D 1; 2. We have U2 U1 D Id, consequently,
Fix.U2 U1 / D X , but Fix U1 \ Fix U2 D f0g.
The assumption on the C -strict quasi nonexpansivity in Theorem 2.1.26 (i) can
be weakened. In this case it suffices to suppose that all Ui are quasi-nonexpansive,
i 2 I , and at least one of them is C -strictly quasi-nonexpansive. The assumption
that the weight function w is appropriate should be replaced in this case by a stronger
one, namely: wj .x/ > 0 for all x such that I.x/ ¤ ; and for all j 2 I.x/. We leave
the proof of this fact to the reader.
A stronger version of the first part of Theorem 2.1.26 (ii) for two operators is
stated below (cf. [346, Proposition 1(d) (i)]).
Theorem 2.1.28. Let S W X ! X be quasi-nonexpansive, T W X ! X be

strictly quasi-nonexpansive and Fix S \ Fix T ¤ ;. Then Fix S T D Fix T S D
Fix S \ Fix T . Furthermore, S T is quasi-nonexpansive and T S is strictly quasi-
nonexpansive.
Proof. The inclusions Fix S \ Fix T Fix S T and Fix S \ Fix T Fix T S are
clear.
(i) We prove that Fix S T Fix S \Fix T . The inclusion is obvious if Fix S T D ;.
Suppose that Fix S T ¤ ; and let x 2 Fix S T be such that x … Fix S \ Fix T .
We consider two cases:
(a) x 2 Fix T . Then x D S T x D S x, i.e., x 2 Fix S . Therefore, x 2 Fix S \
Fix T .
(b) x … Fix T . Let z 2 Fix S \ Fix T . By the quasi nonexpansivity of S and by
the strict quasi nonexpansivity of T , we have
kx zk D kS T x zk kT x zk < kx zk .
In both cases we obtain a contradiction, which proves that Fix S T

Fix S \ Fix T .
(ii) We prove that Fix T S Fix T \Fix S . The inclusion is obvious if Fix T S D ;.
Suppose that Fix T S ¤ ; and let x 2 Fix T S be such that x … Fix T \ Fix S .
Consider two cases:
(a) S x 2 Fix T . Then x D T S x D S x, consequently, x 2 Fix S . Now we
have x D S x 2 Fix T , i.e., x 2 Fix T \ Fix S .
(b) S x … Fix T . Let z 2 Fix T \ Fix S . By the strict quasi nonexpansivity of
T and by the quasi nonexpansivity of S , we have
kx zk D kT S x zk < kS x zk kx zk .
In both cases we obtain a contradiction, which proves that Fix T S

Fix T \ Fix S .
Let now z 2 Fix T S D Fix T \ Fix S and x 2 X . We have
kS T x zk kT x zk kx zk ,
i.e., S T is quasi-nonexpansive. Furthermore,
kT S x zk kS x zk kx zk ,
where the second inequality is strict if x … Fix S and the first one is strict if x 2
Fix S and x … Fix T . Hence, T S is strictly quasi-nonexpansive. u
t
Fig. 2.3 Pcl conv C is a

separator of C z
x y
cl conv C
y = Pcl conv C (x)
Corollary 2.1.29. Let U WD Um Um1 : : : U1 , where U1 ; U2 ; : : : ; Um1 W X ! X

are quasi-nonexpansive, Um W T
T X ! X is strictly quasi-nonexpansive and
i 2I Fix Ui ¤ ;. Then Fix U D i 2I Fix Ui and U is strictly quasi-nonexpansive.
Proof. The corollary follows from Theorem 2.1.28. We leave to the reader an easy
proof by induction with respect to m. t
u
The assumption of Theorem 2.1.28 that T is strictly quasi-nonexpansive is essen-
tial. Note that the composition of quasi-nonexpansive operators needs not to
be quasi-nonexpansive (see Example 2.1.52). Furthermore, the assumptions of
Theorem 2.1.28 do not yield the strict quasi nonexpansivity of the operator S T
(see Example 2.1.54).
2.1.3 Cutters and Strongly Quasi-nonexpansive Operators
Definition 2.1.30. Let x 2 H. We say that y 2 H separates a subset C H from

x if
hx y; z yi 0
for all z 2 C . We say that an operator T W X ! H is a separator of a subset C X
or T separates a subset C , if y WD T x separates C from x for all x 2 H. We say
that T is an ˛-relaxed separator of C , where ˛ 2 Œ0; 2, if T is an ˛-relaxation of a
separator of C . Let T have a fixed point. We say that T is a cutter if T is a separator
of Fix T , i.e.,
hx T x; z T xi 0 (2.20)
for all x 2 X and all z 2 Fix T . We say that T is an ˛-relaxed cutter, where
˛ 2 Œ0; 2, if T is an ˛-relaxed separator of Fix T .
For any nonempty C H the projection Pcl conv C is a separator of C (see
Fig. 2.3). In general, a separator of C is not uniquely determined.
The name cutter expresses the fact that, for any x … Fix T , the hyperplane H.x
T x; hT x; x T xi/ cuts the space into two half-spaces, one of which contains the
point x while the other one contains the subset Fix T (see Fig. 2.4). In the literature
one can find different names for cutters. Bauschke and Combettes call the class of
Fig. 2.4 A cutter and an

˛-relaxed cutter Fix T
Tα x z
Tx
x − T x; u − T x 0
cutters a T -class (see [24, Definition 2.2] and [121, Definition 2.1]). Yamada and
Ogura (see [346, Sect. B]) and Măruşter (see [254]) call the operators firmly quasi-
nonexpansive. Zaknoon, Segal and Censor denoted cutters as directed operators (see
[104–106, 307, 356]). In [69] these operators were called separating operators. The
name cutter was proposed by Cegielski and Censor in [70].
Note that a separator and, in particular, a cutter need not to be continuous
operators.
Remark 2.1.31. Let T W X ! H have a fixed point. Then, by Lemma 1.2.5, the
operator T is a cutter if and only if
hT x x; z xi kT x xk2 (2.21)
holds for all x 2 X and for all z 2 Fix T (cf. [121, Proposition 2.3 (ii)]), and T is
an ˛-relaxed cutter, where ˛ 2 Œ0; 2, if and only if
˛hT x x; z xi kT x xk2 (2.22)
holds for all x 2 X and for all z 2 Fix T . Furthermore, if T is a cutter, then
T jFix T D Id. Therefore, T is a cutter (respectively, an ˛-relaxed cutter) if and only
if inequality (2.21) (respectively, (2.22)) is satisfied for all x … Fix T and for all
z 2 Fix T . Relaxed cutters were also studied in [253, 255, 346] and in [249], where
they were called averaged quasi-nonexpansive mappings.
Remark 2.1.32. Let T W X ! H be a separator of a subset C X . Then the

following obvious properties of T hold:
(i) T is a separator of the closed convex hull of C .
(ii) T is a separator of any subset D C .
(iii) For an arbitrary 2 Œ0; 1, the relaxation T of T is a separator of C .
(iv) C Fix T .
Corollary 2.1.33. Let U W X ! H, T WD 12 .U C Id/ and C X . Then:
(i) U is Fejér monotone with respect to C if and only if T is a separator of C .
(ii) If Fix U ¤ ;, then U is quasi-nonexpansive if and only if T is a cutter.

Proof. The corollary follows easily from equivalence (2.11) (see [24, Proposition
2.3 (v),(vi)] for a different proof). t
u
Corollary 2.1.34. Let T W X ! H and C X . If T is Fejér monotone with
respect to C , then T is Fejér monotone with respect to the closed convex hull of C .
Proof. The corollary follows directly from Corollary 2.1.33 (i) and from Remark
2.1.32 (i). t
u
By Remark 2.1.32 (iii), the right hand side of the equivalence in Corollary 2.1.33 (i)
can be written in the form: U is a separator of C for all 2 Œ0; 12 . Similarly, the
right hand side of the equivalence in Corollary 2.1.33 (ii) can be written in the form:
U is a cutter for all 2 Œ0; 12 . Corollary 2.1.33 (ii) can also be written equivalently
as follows:
U is quasi-nonexpansive if and only if there is a cutter S W X ! H and 2 Œ0; 2
such that U D S .
A subset C X for which the operator T W X ! H is a separator needs not to
be convex. However there exists the largest subset for which T is a separator, which
is closed and convex. This fact follows from the following lemma.
Lemma 2.1.35. Let T W X ! H. If the subset
\
Sep T WD fz 2 X W hz T x; x T xi 0g
x2X
is nonempty, then Sep T is the largest subset for which T is a separator. Further-
more, Sep T is a closed convex subset.
Proof. The first part of the lemma follows directly from Definition 2.1.30. The
second part follows from the fact that an intersection of closed convex subsets is
closed and convex. t
u
If T is nonexpansive, then Fix T is a closed convex subset (see Proposition 2.1.11).
It turns out that cutters have the same property. The second part of the following
lemma was proved in [24, Proposition 2.6 (i)–(ii)].
Lemma 2.1.36. Let T W X ! H. The following inclusion holds
Sep T Fix T . (2.23)
If T is a cutter, then a converse inclusion is also true. Hence, the subset of fixed
points of a cutter is closed and convex.
Proof. Let y 2 Sep T , i.e., hx T x; y T xi 0 for all x 2 X . If we take x D y,
we get ky T yk 0, and hence, T y D y, i.e., y 2 Fix T . Now suppose that T is a
cutter and that y 2 Fix T . Then for any x 2 X we have hy T x; x T xi 0g, i.e.,
y 2 Sep T . Therefore, we have Sep T D Fix T . The subset Fix T is closed convex
as an intersection of closed convex subsets. t
u
It follows from Remark 2.1.32 (ii) and from Lemmas 2.1.35 and 2.1.36 that a cutter
T W X ! X is a separator of any nonempty subset of Fix T . Therefore, we will
restrict our further considerations of separators to cutters. Note, however, that the
converse inclusion of (2.23) is not true in general (see Example 2.2.7). Hence, there
is a separator with a fixed point which is not a cutter.
It is an immediate consequence of the characterization of the metric projection
(see Theorem 1.2.4) that an operator T W H ! H is a metric projection onto a closed
convex subset if and only if T 2 D T and T is a cutter (a more general fact will be
presented in Theorem 2.2.5). In this case, we have T D PFix T . Even if a cutter
T is not idempotent, T is closely related to the metric projection. The following
corollary was proved in [121, Proposition 2.3 (iii)].
Corollary 2.1.37. Let T W X ! H be a cutter. Then, for any x 2 X , it holds
kT x xk kPFix T x xk . (2.24)
Proof. If x 2 Fix T , then inequality (2.24) is obvious. Now let x … Fix T . Then it
follows from inequality (2.21) for z WD PFix T x together with the Cauchy–Schwarz
inequality that
hT x x; PFix T x xi
kT x xk kPFix T x xk
kT x xk

u
Definition 2.1.38. Let ˛ 0 and assume that T W X ! H has a fixed point. We
say that T is ˛-strongly quasi-nonexpansive (˛-SQNE), if
kT x zk2 kx zk2 ˛ kT x xk2 (2.25)
for all x 2 X and z 2 Fix T . If T satisfies (2.25) with ˛ > 0, then T is called
strongly quasi-nonexpansive (SQNE).
A property which is more general than the strong nonexpansivity was introduced
by Halperin [198, Sect. 2] and was called '-property, where ' W Œ0; 1/ ! Œ0; 1/ is
a nondecreasing function. If '.t/ D t 2 for all t 2 Œ0; 1/, then '-property is equiva-
lent to the strong quasi-nonexpansivity. The notion strong quasi nonexpansivity was
introduced by Bruck [50, Sect. 1] for operators defined on a metric space. Strongly
quasi-nonexpansive operators are widely studied in the literature. Bauschke and
Borwein use the name strongly attracting operators for operators which are NE
and SQNE (see [22, Definition 2.1]). Reich and Zaslavski define a more general
operator and call it a uniformly F -attracting mapping, where F D Fix T (see
[297, Sect. 1]). Vasin and Ageev call the ˛-SQNE operators, where ˛ 2 .0; 1/,
Q-pseudocontractive operators (see [333, Definition 2.3]). Yamada and Ogura
use the notation ˛-attracting quasi-nonexpansive for the ˛-SQNE operators [346].
Crombez denotes the class of ˛-SQNE operators by F ˛ (see [126, pages 160–161])
and gives several equivalent conditions for T 2 F ˛ (see [126, Theorem 2.1]).
It follows easily from the equivalence (a),(c) of Lemma 1.2.5 that an operator
T which has a fixed point is a cutter if and only if it is 1-strongly quasi-
nonexpansive. The following theorem extends this property to relaxations of T (cf.
[121, Proposition 2.3 (ii)]).
Theorem 2.1.39. Assume that T W X ! H has a fixed point and let 2 .0; 2.
Then T is a cutter if and only if its relaxation T is 2
-strongly quasi-nonexpansive,
i.e.,
2
kT x zk2 kx zk2 kT x xk2 (2.26)

for all x 2 X and for all z 2 Fix T .
Proof. Since
T x x D .T x x/,
the properties of the inner product yield
2
kT x zk2 kx zk2 C kT x xk2

D kx z C .T x x/k2 kx zk2 C .2 / kT x xk2
D 2.kT x xk2 hz x; T x xi/
D 2hz T x; x T xi
for all x 2 X and for all z 2 C . The assertion follows directly from the equalities
above. t
u
The following corollary is an equivalent formulation of Theorem 2.1.39.
Corollary 2.1.40. Assume that U W X ! H has a fixed point and let ˛ 2 .0; 2.
Then U is an ˛-relaxed cutter if and only if U is 2˛
˛ -strongly quasi-nonexpansive.
In general, a relaxation T of a cutter T with 2 needs not to be strongly

quasi-nonexpansive. Nevertheless, the following proposition holds.
Proposition 2.1.41. Let T W X ! H be a cutter with int Fix T ¤ ; and > 0.
Then for any z 2 int Fix T and x … Fix T it holds
2ı
kT x zk2 kx zk2 .2 C / kT x xk2 , (2.27)
kT x xk
where ı > 0 is such that B.z; ı/ Fix T . If X is bounded, then T is int Fix T -
strictly quasi-nonexpansive for any 2 .0; 2.
Proof. Let z 2 int Fix T and x … Fix T . Then w WD z ı kTT xxk

xx
2 Fix T X and
inequality (2.21) yields
kT x zk2 D kx C .T x x/ zk2

D kx zk2 C 2 kT x xk2 2hz x; T x xi
D kx zk2 C 2 kT x xk2
2hz w; T x xi 2hw x; T x xi
kx zk2 C 2 kT x xk2
2ı kT x xk 2 kT x xk2
2ı
D kx zk2 .2 C / kT x xk2 .
kT x xk
Let X be bounded and d > 0 be such that kT u uk d for any u 2 X . The

existence of such d follows from Corollary 2.1.37. Denote " WD 2ı d
. Then (2.27)
yields
kT x zk2 kx zk2 .2 C " / kT x xk2 .
Consequently, T is int Fix T -strictly quasi-nonexpansive for any 2 .0; 2. t
u
The corollary below follows immediately from Proposition 2.1.41 and from
Theorem 2.1.26.
Corollary
T 2.1.42. Let Ui W X ! H, i 2 I be quasi-nonexpansive with C WD
Ti 2I Fix U i ¤ ; and let U WD Um Um1 : : : U1 . If int C ¤ ;, then Fix U D
i 2I Fix U i and U is int C -strictly quasi-nonexpansive.
An equivalent formulation of the following result appeared in [127, Theorem 3.2

(iii)].
Corollary 2.1.43. Assume that U W X ! H has a fixed point and let ˇ 0. Then
2
U is ˇ-strongly quasi-nonexpansive if and only if U is a ˇC1 -relaxed cutter.
Proof. It suffices to take ˛ D 2

ˇC1 in Corollary 2.1.40. t
u
Remark 2.1.44. Assume that T W X ! H has a fixed point and is ˛-strongly
quasi-nonexpansive, where ˛ 0.
(i) If ˛ D 0, then T is quasi-nonexpansive.
(ii) T is -strongly quasi-nonexpansive for all 2 Œ0; ˛.
(iii) If ˛ > 0, then T is strictly quasi-nonexpansive. Therefore, all properties
of strictly quasi-nonexpansive operators are also valid for strongly quasi-
nonexpansive operators and for cutters.
Fig. 2.5 Solution of (2.28) as

a function of ; 2 .0; 2/
γ = γ(λ, μ)
1
0
2
2
1
1
μ 0 λ
Cutters and strongly quasi-nonexpansive operators play an important role in

methods presented in further parts of the book. Therefore, we focus our attention
on the properties of these operators which enable us to construct new cutters or
strongly quasi-nonexpansive operators. Below, we show that a family of relaxed
cutters is closed under composition and under convex combination of operators
having a common fixed point. The first property of relaxed cutters follows from
the lemma below whose proof is left to the reader.
Lemma 2.1.45. Let ; 2 .0; 2/. The unique solution of the equation
!2
1 2
1 1 1 1
D. /. / (2.28)
2
is
2 4. C /
D D . (2.29)

. 2 C 1
2
/ C1 4
Moreover,
4 minf; g 4 maxf; g
0 < minf; g < < 2.
minf; g C 2 maxf; g C 2
A solution of (2.28) is illustrated in Fig. 2.5.

Theorem 2.1.46. Let T W X ! X be a -relaxed cutter, U W X ! X be a -
relaxed cutter, where ; 2 .0; 2, and let Fix T \ Fix U ¤ ;. If ; 2 .0; 2/,
then U T is a -relaxed cutter, where is given by (2.29). If D 2 and < 2 or
D 2 and < 2, then U T is a quasi-nonexpansive operator or, equivalently, U T
is a 2-relaxed cutter.
Proof. Suppose that T is a -relaxed cutter and that U is a -relaxed cutter. Take
a WD T x x and b WD U T x T x. Then it follows from inequality (2.22) that
hz x; ai 1 kak2 and hz T x; bi 1 kbk2 for any z 2 Fix U \ Fix T .
Let ; 2 .0; 2/ and be defined by (2.29). Then Lemma 2.1.45 yields
1
hz x; UTx xi kU T x xk2

1
D hz x; a C bi ka C bk2

1
D hz x; ai C hz x; bi ka C bk2

1
D hz x; ai C hz T x; bi C ha; bi ka C bk2

1 1 1
kak2 C kbk2 C ha; bi ka C bk2

1 1 1 1 2
D. / kak2 C . / kbk2 C .1 /ha; bi

s s 2
1 1 1
1
D a b 0.

Applying inequality (2.22) we obtain that U T is a -relaxed cutter. If D 2 and

< 2 or D 2 and < 2, then U T is quasi-nonexpansive by Theorem 2.1.28. u t
The following result is due to Yamada and Ogura (see [346, Proposition 1(d)]).
Corollary 2.1.47. Let T; U W X ! X have a common fixed point and ;
> 0. If
T is -SQNE and U is
-SQNE, then U T is ı-SQNE, where
1
ıD . (2.30)
1
C 1

Proof. Suppose that T is -SQNE and U is

-SQNE. It follows from Corol-
lary 2.1.43 that T is a -relaxed cutter and that U is a -relaxed cutter, where
D 1C 2
and D 1C
2
. By Theorem 2.1.46 the operator U T is a -relaxed cutter,
where
2 2
D D 1 1
.

. 2 C 1
2
/ C1 . 1 C
/ C1
Corollary 2.1.40 yields now that U T is ı-SQNE, where ı is given by (2.30). t

u
Theorem 2.1.48. Let Ti W X ! X be an ˛i -relaxed cutter, where ˛i 2 .0; 2/, i 2

I WD f1; 2; : : : ; mg, or, equivalently,T
Ti be ˇi -strongly quasi-nonexpansive, where
ˇi D 2˛
˛i
i
2 .0; C1/, i 2 I . Let i 2I Fix Ti ¤ ; and Um WD Tm Tm1 : : : T1 .
Then:
(i) The operator Um is a m -relaxed cutter, with
2
m D ˛m 1 . (2.31)
˛1
. 2˛ 1
C ˛2
2˛2
C ::: C 2˛m
/ C1
(ii) The operator Um is ım -strongly quasi-nonexpansive, with
1
ım D . (2.32)
1
ˇ1
C 1
ˇ2
C ::: C 1
ˇm
Moreover,
2m mini 2I ˛i 2m maxi 2I ˛i
0 < min ˛i < m <2
i 2I .m 1/ mini 2I ˛i C 2 .m 1/ maxi 2I ˛i C 2
(2.33)
and
mini 2I ˇi maxi 2I ˇi
0< ım . (2.34)
m m
Proof. The assertion is obvious for m D 1. Note that m D ım2C1 and that
Corollary 2.1.43 yields the equivalence of conditions (i) and (ii). We prove by
induction with respect to m that these conditions hold for any m 2.
10 If m D 2, then conditions (i) and (ii) follow directly from Theorem 2.1.46 and
from Corollary 2.1.47.
20 Suppose that (ii) is true for some m D k. Consequently, Uk is ık -SQNE. It
follows now from Corollary 2.1.47 that the operator UkC1 D TkC1 Uk is ı-SQNE,
where
1 1
ıD D D ıkC1 .
1
ık
C ˇkC1
1 1
ˇ1
C 1
ˇ2
C ::: C 1
ˇk
C 1
ˇkC1
Now, for m D k C 1, equality (2.31) follows from the above mentioned equivalence
of (i) and (ii).
Hence, we have proved that conditions (i) and (ii) hold for all m 1. Both
inequalities in (2.34) follow immediately from equality (2.32). Now we have
2 2 2 2m maxi 2I ˛i
m D D D < 2.
ım C 1 mini 2I ˇi
C1 mini 2I
2˛i
.m 1/ maxi 2I ˛i C 2
m
m
˛i
C1
In a similar way one can prove that
2m mini 2I ˛i
m > min ˛i > 0
.m 1/ mini 2I ˛i C 2 i 2I

u
Bauschke and Borwein proved that a composition of ˇi -SQNE operators with a
i 2I ˇi
common fixed point is ˇ-SQNE for ˇ WD min2m1 (see [22, Theorem 2.10 (ii)]). It is
clear that this result is weaker than Theorem 2.1.48 (ii), because ˇ minim2I ˇi ım .
Note that the first inequality is strict for m > 2 and that the other one is strict if
ˇi ¤ ˇj for at least one pair i; j 2 I .
Corollary 2.1.49. Let Ui W X ! H be cutters with a common fixed point, i 2
I WD f1; 2; : : : ;P
mg, and w W X ! m be an appropriate weight function. Then the
operator U WD i 2I !i Ui is a cutter.
P
Proof. Let U WD i 2I !i Ui . It is clear that a cutter is strictly quasi-nonexpansive
(see Remark
T 2.1.44 (iii)). Therefore, it follows from Theorem 2.1.26 (i) that
Fix U D i 2I Fix Ui . By Remark 2.1.31 and by the convexity of the function kk2 ,
we have
X
hUx x; z xi D !i .x/hUi x x; z xi
i 2I
X
!i .x/ kUi x xk2
i 2I
2
X

!i .x/Ui x x

i 2I
D kUx xk2
for all x 2 X and all z 2 Fix U . Again, by Remark 2.1.31, U is a cutter. t

u
Theorem 2.1.50. Let Ti W X ! H be an ˛i -relaxed cutter, where ˛i 2 .0; 2/,
i 2 I WD f1; 2; : : : ; mg, or, equivalently,TTi be ˇi -strongly quasi-nonexpansive,
where ˇi D 2˛
˛iP 2 .0; C1/, i 2 I . Let
i
i 2I Fix Ti ¤ ; and w 2 m . Then the
operator T WD i 2I !i Ti is an ˛-relaxed cutter with
X
˛ WD !i ˛i . (2.35)
i 2I
Consequently, T is ˇ-SQNE, with

X !i 1
ˇ WD . / 1. (2.36)
i 2I
ˇi C 1
Moreover,
0 < min ˛i ˛ max ˛i < 2 (2.37)

i 2I i 2I
and
0 < min ˇi ˇ max ˇi . (2.38)

i 2I i 2I
Proof. Without loss of generality we suppose that w 2 ri m . Let Ui WD .Ti /˛i1 ,

i.e.,
1
Ui D Id C .Ti Id/.
˛i
It is clear that Ui are cutters, i 2 I . Let ˛ be defined by (2.35) and i WD !i˛˛i ,
i 2 I . Note P that v D .1 ; 2 ; : : : ; m / 2 ri m , consequently, v is appropriate.
Define U WD i 2I i Ui . By Corollary 2.1.49, the operator U is a cutter. We have
X X i 1X 1
U D i Ui D Id C .Ti Id/ D Id C !i .Ti Id/ D Id C .T Id/,
i 2I i 2I
˛i ˛ i 2I ˛
i.e., T D Id C˛.U Id/ and T is an ˛-relaxed cutter. The second part of the theorem
follows now immediately from Corollaries 2.1.40 and 2.1.43. Inequalities in (2.37)
are obvious and inequalities in (2.38) follow easily from (2.36). t
u
Bauschke and Borwein proved that a convex combination of ˇi -SQNE operators,
i 2 I , with a common fixed point is ˇ-SQNE, where ˇ WD mini 2I ˇi (see [22,
Proposition 2.12]). By inequality (2.38) this result is weaker than Theorem 2.1.50.
Note that this inequality is strict if ˇi ¤ ˇj for at least one pair i; j 2 I for which
!i and !j are nonzero. The second part of Theorem 2.1.50 for m D 2 was proved
by Yamada and Ogura (see [346, Proposition 1(c)]).
The following important result extends Theorem 2.1.39.
Theorem 2.1.51. Let S W H ! X be nonexpansive, T W X ! H be a cutter and
2 .0; 2/. If Fix S \ Fix T ¤ ;, then, for any x 2 Fix S and z 2 Fix S \ Fix T , the
following estimations hold
kS T x zk2 kx zk2 .2 / kT x xk2 (2.39)
and
2
kS T x zk2 kx zk2 kS T x xk2 . (2.40)

Consequently, the operator S T jFix S is 2

-strongly quasi-nonexpansive.
Proof. Let x 2 Fix S and z 2 Fix S \ Fix T . Then the assumptions that S is
nonexpansive and T is a cutter yield
Fig. 2.6 Composition of

cutters needs not to be a cutter
1
B A∩B
1
Ux = U2 U1x
2
z
U1x 1
1
2
A
kS T x zk2 D kS T x S zk2 kT x zk2

D kx zk2 C 2 kT x xk2 2hz x; T x xi
kx zk2 .2 / kT x xk2
2
D kx zk2 kT x xk2

2
kx zk2 kS T x S xk2

2
D kx zk2 kS T x xk2

u
Below we give several examples which show that a composition of quasi-
nonexpansive operators does not need to be quasi-nonexpansive, that a composition
of a strictly quasi-nonexpansive operator and a quasi-nonexpansive one does not
need to be strictly quasi-nonexpansive and that a composition of cutters does
not need to be a cutter, even if they have a common fixed point.
Example 2.1.52. Let X WD Œ1; 1 R, S; T W X ! X , S WD Id and

x if x D 1
T x WD 1
2 x otherwise.
One can easily check that S; T are quasi-nonexpansive, Fix S D Fix T D f0g and
Fix S T D f0; 1g. The operator S T is not quasi-nonexpansive, because a subset of
fixed points of a quasi-nonexpansive operator is convex (see Lemma 2.1.21).
Example 2.1.53. Let X D H WD R2 , A WD fx 2 R2 W he; xi 1g, B WD fx 2 R2 W
2 0g, U1 WD PB , U2 WD PA and U WD U2 U1 . Then U1 and U2 are cutters and it
follows from Theorem 2.1.26 that Fix U D Fix U1 \ Fix U2 D A \ B ¤ ;. For x D
.0; 1/ and z D .1; 0/ 2 A \ B, we have Ux D . 12 ; 12 / and hx Ux; z Uxi D 12
(see Fig. 2.6). Therefore, U is not a cutter.
2.2 Firmly Nonexpansive Operators 65
Example 2.1.54. Let A; B H be nonempty closed convex subsets and A B.

Define S WD 2PA Id and T WD PB . We have Fix S \ Fix T D A. It follows easily
from the characterization of the metric projection that PA and PB are cutters. By
Theorem 2.1.39, T is strictly quasi-nonexpansive and S is quasi-nonexpansive.
By Theorem 2.1.28, the operator S T is quasi-nonexpansive. Unfortunately, S T is
not strictly quasi-nonexpansive, because for any x 2 BnA and for z WD PA x it holds
kS T x zk D kS x zk D kx zk .
2.2 Firmly Nonexpansive Operators
Definition 2.2.1. We say that an operator T W X ! H is firmly nonexpansive

(FNE), if
hT x T y; x yi kT x T yk2 (2.41)
for all x; y 2 X . Let 2 Œ0; 2. We say that T W X ! H is -relaxed firmly
nonexpansive (-RFNE) or, shortly, relaxed firmly nonexpansive (RFNE) if T is a
-relaxation of a firmly nonexpansive operator U , i.e., T D U D .1 / Id CU .
If, furthermore, 2 .0; 2/, then we say that T is strictly relaxed firmly nonexpan-
sive.
The definition of a firmly nonexpansive operator in a Hilbert space is due to
Browder (see [46]), who called it a firmly contractive operator. Bruck introduced the
name firmly nonexpansive for operators in a Banach space (see [49, Definition 6]).
In Hilbert spaces both definitions coincide, as we will show in Theorem 2.2.10.
Condition (vi) of this theorem is, actually, the definition of a firmly nonexpansive
operator proposed by Bruck.
The following lemma is obvious.
Lemma 2.2.2. Let T W X ! H and x; y 2 X . The following inequalities are
equivalent:
(i) hT x T y; x yi kT x T yk2 ,
(ii) hT x T y; .x T x/ .y T y/i 0,
(iii) hT y T x; x T xi C hT x T y; y T yi 0,
(iv) hT y x; T x xi C hT x y; T y yi kT x xk2 C kT y yk2 .
It follows from Lemma 2.2.2 that inequality (2.41) defining a firmly nonexpan-
sive operator can be replaced by any inequality in (i)–(iv).
Corollary 2.2.3. Let > 0. An operator S W X ! H is -RFNE if and only if
1
hy x; S x xi C hx y; Sy yi k.S x x/ .Sy y/k2 . (2.42)

Fig. 2.7 NE and monotone

operator which is not FNE Tx
Ty a
PLina (x − y)
' x
Proof. Let S WD T D Id C.T Id/ for a firmly nonexpansive operator T W

X ! H. Let x; y 2 X . It follows from the equivalence (i),(iv) in Lemma 2.2.2
and from the equality T D S1 (see Remark 2.1.3) that S is -RFNE if and only if
1
hT y x; S x xi C hT x y; Sy yi .kS x xk2 C kSy yk2 /.

Since T y x D y x C 1 .Sy y/ and T x y D x y C 1 .S x x/, the last

inequality is equivalent to
2 1
hyx; S xxiChxy; SyyiC hS xx; Syyi .kS x xk2 CkSy yk2 /.

The latter inequality is equivalent to (2.42). t
u
2.2.1 Basic Properties of Firmly Nonexpansive Operators
Theorem 2.2.4. A firmly nonexpansive operator T W X ! H is monotone and

nonexpansive.
Proof. Let T be firmly nonexpansive. By the Cauchy–Schwarz inequality, we have
kT x T yk kx yk hT x T y; x yi kT x T yk2 0,
for all x; y 2 X , which yields the monotonicity and the nonexpansivity of T . t

u
The converse of Theorem 2.2.4 is not true, e.g., the operator T W R ! R ,
2 2
T x WD .1 cos ' 2 sin '; 1 sin ' C 2 cos '/
is nonexpansive and monotone for ' 2 .0; =2/, but T is not firmly nonexpansive
(see Fig. 2.7).
Now we prove a property of firmly nonexpansive operators, which also appears
in Theorem 1.2.4. In particular, the characterization of the metric projection is,
actually, a corollary of the following theorem which is due to Goebel and Reich
(see [185, pp. 43–44]).
Theorem 2.2.5. Let T W X ! H be an operator with a fixed point.
(i) If T is firmly nonexpansive, then T is a cutter, i.e.,
hz T x; x T xi 0 (2.43)
for all x 2 X and z 2 Fix T .

(ii) If T is a projection, i.e., T .X / D Fix T , then the implication converse to .i/ is
also true. In this case, T D PFix T .
Proof. (i) Let T be firmly nonexpansive, x 2 X and z 2 Fix T . By the equivalence
(i),(iii) in Lemma 2.2.2, we have
hT y T x; x T xi C hT x T y; y T yi 0,
and for y D z 2 Fix T we obtain (2.43).

(ii) Suppose that T is a projection and that inequality (2.43) holds for all x 2 X
and z 2 Fix T . Let u; v 2 X . Taking x D u and z D T v in (2.43) we get
hT v T u; u T ui 0, (2.44)
and, taking x D v and z D T u in (2.43), we get
hT u T v; v T vi 0. (2.45)
Note that, in both cases, z 2 Fix T because T .X / D Fix T . Therefore, the

characterization of the metric projection yields that T D PFix T . Summing up
inequalities (2.44) and (2.45) we get
hT u T v; .T u T v/ .u v/i 0;
i.e., T is firmly nonexpansive (see equivalence (i),(ii) in Lemma 2.2.2). t

u
Suppose that Fix T ¤ ;. It follows from the equivalence of (i) and (iii) in
Lemma 2.2.2 that inequality (2.41) for y D z 2 Fix T gives (2.43). Therefore,
for T being a cutter, inequality (2.41) is required for all x 2 X and all y 2 Fix T ,
while for T being firmly nonexpansive this inequality should hold for all x; y 2 X .
Remark 2.2.6. Neither a projection nor a separator of a nonempty subset C H
need to be nonexpansive (note that a separator can even be discontinuous).
Furthermore, a nonexpansive separator and even a nonexpansive cutter need not
to be firmly nonexpansive (see Examples 2.2.7 and 2.2.8 below).
Fig. 2.8 NE separator which x

is not FNE y
B1
a Tx Ty
A
B Tz w =Tw
B2
z
Example 2.2.7. (cf. [204] and [78, Sect. 4.10]) Let a 2 H, kak D 1 and ˛ > 0.
Furthermore, let A WD fx 2 H W ha; xi D 0g, B1 WD fx 2 H W ha; xi D ˛g,
B2 WD fx 2 H W ha; xi D ˛g and B WD fx 2 H W jha; xij ˛g. The subset
B is a band with a width of 2˛ and is bounded by two hyperplanes B1 and B2 .
The hyperplane A cuts the band B into two bands bounded by A and B1 and by A
and B2 . Define the operator T W H ! H as follows
8
ˆ
ˆ PA x if jha; xij 2˛
<
2PB1 x x if ˛ < ha; xi < 2˛
Tx D (2.46)
ˆ 2PB2 x x if 2˛ < ha; xi < ˛
:̂
x if jha; xij ˛
Note that T projects onto A all points with the distance to A equal at least 2˛, T
reflects (with respect to the closest hyperplane B1 or B2 ) the points which do not
belong to the band B with the distance to A less than 2˛ and T does not move
the elements of the band B (see Fig. 2.8). The reader can easily check that T is
nonexpansive and that T is a separator of A but T is not firmly nonexpansive. Note
that Fix T D B and that T is not a cutter, i.e., it does not separate Fix T , but T
separates A (see Fig. 2.8).
Example 2.2.8. Let A WD R f0g and B WD f0g R be two subspaces of R2 and

T W R2 ! R2 be defined by
T x WD Œ1 .x/PA x C .x/PB x,
12
where .x/ D 12 C22
for x D .1 ; 2 / 2 R2 (see Fig. 2.9). We have PA x D .1 ; 0/,
PB x D .0; 2 /. Consequently,
1 22 2 2
Tx D . ; 2 1 2/
12C 2 1 C 2
2
for x ¤ .0; 0/. Note that z WD .0; 0/ is the unique fixed point of T .
The operator T is a cutter, because
12 22
hz T x; x T xi D 0
12 C 22
Fig. 2.9 NE cutter which is

not FNE »2 = »1
x
PB x Tx
FixT PA x
for all x ¤ z. (Since the weight function w W R2 ! 2 , w.x/ WD .1 .x/; .x//

is appropriate, this fact follows also from Corollary 2.1.49). Let x; y ¤ .0; 0/.
A straightforward calculation shows that
kT x T yk2 22 22 .1 1 /2 C 12 21 .2 2 /2

D
kx yk2 .12 C 22 /. 21 C 22 /Œ.1 1 /2 C .2 2 /2
holds for all x D .1 ; 2 / 2 R2 and for all y D . 1 ; 2 / 2 R2 , x ¤ y. If 1 1 D

2 2 D 0, then, of course T x D T y D .0; 0/. Suppose that 0 < 12 21 22 22 . Then
we have
12 21
kT x T yk2 .1 1 /2 C .
22 22 2
2 /2
D 1.
kx yk2 .1 C
12
/.1 C
21
/Œ.1 1 /2 C .2 2 /2
22 22
If 0 < 22 22 12 21 , then we have
22 22
kT x T yk2 .
12 21 1
1 /2 C .2 2 /2
D 1.
kx yk2 22 22
.1 C 12
/.1 C 21
/Œ.1 1 /2 C .2 2 /2
Therefore, T is nonexpansive. If we take x D . 12 ; 1/ and y D .1; 12 /, then T x D

. 25 ; 15 /, T y D . 51 ; 25 / and
1 2
hT x T y; x yi D < D kT x T yk2 .
5 25
Therefore, T is not firmly nonexpansive.
The following property of firmly nonexpansive operators (cf. [22, Lemma 2.4
(iv)]) is often used in applications.
Corollary 2.2.9. Let T W X ! H be an operator with a fixed point and 2

.0; 2. If T is firmly nonexpansive, then its relaxation T is 2

-strongly quasi-
nonexpansive, i.e.,
2
kT x zk2 kx zk2 kT x xk2 (2.47)

Proof. It follows from the first part of Theorem 2.2.5 that a firmly nonexpansive
operator having a fixed point is a cutter. Therefore, T is 2
-strongly quasi-
nonexpansive (see Theorem 2.1.39). t
u
2.2.2 Relationships Between Firmly Nonexpansive

and Nonexpansive Operators
One can find in the literature several equivalent definitions of firmly nonexpansive
operators. The properties of these operators were studied by Zarantonello [357,
Sect. 1], Bruck [49, Sects. 2 and 3], Rockafellar [299], Bruck and Reich [51, Sect. 1],
Goebel and Reich [185, Chap. 1, Sect. 11], Reich and Shafrir [296], Goebel and Kirk
[184, Chap. 12], Bauschke and Borwein [22, Sects. 2 and 3], Byrne [56, Sect. 2], and
by Crombez [127, Sect. 2].
The class of firmly nonexpansive operators is included in the class of nonexpan-
sive ones (see Theorem 2.2.4). Further important relationships between these two
classes are also useful for the investigation of firmly nonexpansive operators. These
relationships are given in the following theorem.
Theorem 2.2.10. Let T W X ! H. Then the following conditions are equivalent:
(i) T is firmly nonexpansive.
(ii) T is nonexpansive for any 2 Œ0; 2.
(iii) T has the form T D 12 .S C Id/; where S W X ! H is a nonexpansive operator.
(iv) Id T is firmly nonexpansive.
(v) For all x; y 2 X it holds
kT x T yk2 kx yk2 k.x T x/ .y T y/k2 . (2.48)
(vi) For all x; y 2 X and for any ˛ 0 it holds
kT x T yk k˛.x y/ C .1 ˛/.T x T y/k .
Proof. The equivalence (i),(iv) in Theorem 2.2.10 is obvious, because both

conditions can be written in the form hT x T y; .x T x/ .y T y/i 0 for all
x; y 2 X . Nevertheless, we prove the following relations among (i)-(vi):
(i) ) (ii) ) (iii) ) (iv) ) (v) ) (i) , (vi).

(i))(ii) Let T be firmly nonexpansive and x; y 2 X . By the definition of a firmly

nonexpansive operator, the Cauchy–Schwarz inequality and the nonexpansivity of
T (see Theorem 2.2.4), we have
kT x T yk2 D kT x C .1 /x T y .1 /yk2

D k.T x T y/ C .1 /.x y/k2
D 2 .kT x T yk2 hT x T y; x yi/
C.2 2 /hT x T y; x yi C .1 /2 kx yk2
.2 2 /hT x T y; x yi C .1 /2 kx yk2
.2 2 / kT x T yk kx yk C .1 /2 kx yk2
.2 2 / kx yk2 C .1 /2 kx yk2
D kx yk2 ,
i.e., T is nonexpansive.
(ii))(iii) This implication is obvious. It suffices to take S D T for D 2.
(iii))(iv) Let S be nonexpansive, T WD 12 .S C Id/ and G WD Id T . Then we
have G D 12 .Id S / and
kGx Gyk2 D hGx Gy; x yi C hGx Gy; .Gx Gy/ .x y/i

D hGx Gy; x yi
1
C h.S x Sy/ .x y/; .S x Sy/ C .x y/i
4
1
D hGx Gy; x yi C .kS x Syk2 kx yk2 /
4
hGx Gy; x yi,
for all x; y 2 X .
(iv))(v) Let G WD Id T be firmly nonexpansive. Then, for all x; y 2 X we
have
kT x T yk2 C k.Id T /x .Id T /yk2

kT x T yk2 C h.Id T /x .Id T /y; x yi
D kT x T yk2 hT x T y; x yi C kx yk2
D hT x T y; .x T x/ .y T y/i C kx yk2
kx yk2 ,
i.e., (2.48) holds.

(v))(i) Let x; y 2 X . If (2.48) holds, then, by the properties of the inner product,
we have
kT x T yk2 kx yk2 k.x y/ .T x T y/k2

D kT x T yk2 C 2hT x T y; x yi,
i.e., T is firmly nonexpansive.

(i),(vi) Let x; y 2 X . The function h W RC ! RC defined by
1
h.˛/ D k˛.x y/ C .1 ˛/.T x T y/k2
2
is convex as a composition of the convex function f ./ D 12 kk2 and an affine

function A W R ! H, A.˛/ D ˛.x y/ C .1 ˛/.T x T y/. Note that h.0/ D
2 kT
1
x T yk2 , h is differentiable and
h0 .0/ D hT x T y; .x y/ .T x T y/i.
Since h is convex, we have
h.0/ h.˛/ ” h0 .0/ 0
for all ˛ 0, i.e.,
kT x T yk2 k˛.x y/ C .1 ˛/.T x T y/k2

” hT x T y; x yi kT x T yk2

u
The same kind of correspondences between firmly nonexpansive operators and
nonexpansive ones (the equivalence (i),(iii) in Theorem 2.2.10) and between
cutters and quasi-nonexpansive operators (Corollary 2.1.33 (ii)) explain the name
firmly quasi-nonexpansive operators for cutters (see [346, page 624]).
Condition (ii) in Theorem 2.2.10 can be formulated equivalently as follows: (ii’)
T2 WD 2T Id is nonexpansive.
The nonexpansivity of T2 and Lemma 2.1.12 (i) yield the nonexpansivity of T
for all 2 Œ0; 2, because T D .1 2 / Id C 2 T2 . Moreover, the assumption that T2
is nonexpansive is sufficient in the implication (ii))(iii) as follows from the proof.
Now we present a series of corollaries of Theorem 2.2.10.
Corollary 2.2.11. Let T W X ! H. The operator T is firmly nonexpansive if and
only if its relaxation T is firmly nonexpansive for all 2 Œ0; 1.
Proof. Let T be firmly nonexpansive and 2 Œ0; 1. By the implication (i))(iii) in
Theorem 2.2.10 we obtain
1
T D .1 / Id C .Id CS / D ŒId C.1 / Id CS
2 2
for a nonexpansive operator S . Note that .1/ Id CS is nonexpansive as a convex

combination of nonexpansive operators (see Lemma 2.1.12 (ii)). Therefore, T is
firmly nonexpansive by the implication (iii))(i) in Theorem 2.2.10. The sufficiency
of the condition is obvious. t
u
Corollary 2.2.12. Let U W X ! H and 2 Œ0; 2. Then U is -RFNE if and only
if U is -RFNE for all 2 Œ; 2.
Proof. Let U WD T D Id C.T Id/, where T W X ! H is a firmly nonexpansive
operator, and 2 Œ; 2. It is easy to see that
U D Id C.T= Id/:
The corollary follows now from the fact that T= is firmly nonexpansive (see
Corollary 2.2.11). t
u
Corollary 2.2.13. Let X H be a closed convex subset and S W X ! H. The
following conditions are equivalent:
(i) S is nonexpansive,
(ii) S D 2T Id, where T W X ! H is a firmly nonexpansive operator.
Proof. (ii))(i) Let S WD 2T Id for a firmly nonexpansive operator T . It follows
from the implication (i))(ii) in Theorem 2.2.10 that S is nonexpansive.
(i))(ii) Let S be nonexpansive and T WD 12 .S C Id/. By the implication
(iii))(i) in Theorem 2.2.10 the operator F is firmly nonexpansive. Furthermore,
S D 2T Id. t
u
Definition 2.2.14. (cf. [127, Definition 2.1]) We say that an operator U W X ! H
is -firmly nonexpansive (-FNE), where > 0, if
kUx Uyk2 kx yk2 k.x Ux/ .y Uy/k2 .
Vasin and Ageev call a -firmly nonexpansive operator for 2 .0; 1/, a pseudo-
contractive operator (see [333, Definition 2.5]). In [127, Theorem 2.3] several
equivalent conditions for U to be -FNE are presented.
By the equivalence (i),(v) of Theorem 2.2.10, an operator is firmly nonex-
pansive if and only if it is 1-firmly nonexpansive. Note, however, that there is a
difference between a -RFNE operator and a -FNE operator. Below, we present
the relationship between these two notions.
Corollary 2.2.15. Let 2 .0; 2/. An operator U W X ! H is -relaxed firmly
nonexpansive if and only if U is 2

-firmly nonexpansive, i.e.,
2
kUx Uyk2 kx yk2 k.x Ux/ .y Uy/k2

for all x; y 2 X . If, furthermore, Fix U ¤ ;, then
2
kUx zk2 kx zk2 kUx xk2

for all x 2 X and z 2 Fix U , i.e., U is 2

Proof. Let U WD T for a firmly nonexpansive operator T and x; y 2 X . Applying
the properties of the inner product we get for G WD Id T
kUx Uyk2 D k.1 /x C T x .1 /y T yk2

D kx y .Gx Gy/k2
D kx yk2 2hx y; Gx Gyi C 2 kGx Gyk2 .
Since x Ux D x T x D Gx, the equalities above yield
2
kUx Uyk2 kx yk2 C k.x Ux/ .y Uy/k2

D kUx Uyk2 kx yk2 C .2 / kGx Gyk2
D 2.hx y; Gx Gyi kGx Gyk2 /.
The first part of the corollary follows now from the equivalence (i),(iv) in
Theorem 2.2.10, and now the other part follows directly from the definition of an
˛-strongly quasi-nonexpansive operator. t
u
Definition 2.2.16. Let ˛ 2 .0; 1/. We say that an operator T W X ! H is
˛-averaged or, shortly, averaged (AV) if
T D .1 ˛/ Id C ˛S
holds for a nonexpansive operator S W X ! H.

Averaged operators were studied, e.g., by Mann [252], Krasnosel’skiı̆ [238],
Baillon et al. [14, Sect. 2]. In [56, Sect. 2], Byrne gives relationships between
averaged operators and inverse strongly monotone operators, i.e., operators G W
X ! H such that
hGx Gy; x yi kGx Gyk2
for all x; y 2 X and for some constant > 0.
Definition 2.2.16 states that an operator is averaged if and only if it is an under-
relaxation of a nonexpansive operator.
Corollary 2.2.17. Let 2 .0; 2/ and ˛ D =2. An operator U W X ! H is

-relaxed firmly nonexpansive if and only if U is ˛-averaged.
Proof. ()) Let T W X ! H be firmly nonexpansive and U WD T D .1 /
Id CT . By the implication (i))(iii) in Theorem 2.2.10 we have T D 12 .S C Id/
for a nonexpansive operator S W X ! H. Hence, U D .1 ˛/ Id C ˛S , i.e., U is
˛-averaged.
(() Let U be ˛-averaged, i.e., U D .1˛/ Id C ˛S for a nonexpansive operator
S and for ˛ D =2 2 .0; 1/. By Corollary 2.2.13 we have
U D .1 ˛/ Id C ˛.2T Id/
D .1 2˛/ Id C2˛T
for a firmly nonexpansive operator T . Hence, U is the -relaxation of T W X ! H

with D 2˛ 2 .0; 2/. u
t
Corollary 2.2.18. Let G W X ! H. Then G is firmly nonexpansive if and only if
Id G is averaged for any 2 .0; 2/.
Proof. Necessity. Let G be firmly nonexpansive. We have
Id G D .1 =2/ Id C.=2/Œ2.Id G/ Id.
By the implications (i))(iv) and (i))(ii) in Theorem 2.2.10 the operator

2.Id G/ Id is nonexpansive. Consequently, the operator Id G is averaged.
Sufficiency. Let Id G be averaged for any 2 .0; 2/. Then Id G is nonex-
pansive for any 2 .0; 2/ and Id 2G is nonexpansive as a limit of nonexpansive
operators. Now, it follows from the implication (ii))(i) in Theorem 2.2.10 that G
is firmly nonexpansive. t
u
Corollary 2.2.19. Let U W X ! H and 2 .0; 2. The operator U is -relaxed
firmly nonexpansive if and only if its relaxation U is firmly nonexpansive for 2
Œ0; 1 .
Proof. Take U WD T for a firmly nonexpansive operator T W X ! H. Then the
claim follows from the equality U1 D T (see Remark 2.1.3) and Corollary 2.2.11.
The converse implication is obvious. t
u
The following corollary shows that the family of firmly nonexpansive operators is
closed under convex combination.
Corollary 2.2.20. Let Ti W X ! H, i 2 I WD f1; 2; : : : ; mg, be Pfirmly nonexpan-
sive and w D .!1 ; !2 ; : : : ; !m / 2 m . Then the operator T WD i 2I !i Ti is firmly
nonexpansive.
P
Proof. Let T WD i 2I !i Ti . By the implication (i))(iii) in Theorem 2.2.10,
we have Ti D 12 .Si C Id/ for a nonexpansive operator Si , i 2 I . Observe
Fig. 2.10 Basic relationships

among algorithmic operators
2
Fix Fix
=(2- )/
=(2- )/
=2
P
that T D 12 .S C Id/ for S WD i 2I !i Si . By Lemma 2.1.12 (i), the operator
S is nonexpansive. The corollary follows now from the implication (iii))(i) in
Theorem 2.2.10. t
u
In Fig. 2.10 we shortly present important relationships among the FNE operators,
cutters, QNE operators SQNE operators and AV operators, which are proved in
Sects. 2.1.3, 2.2.1 and 2.2.2. In Fig. 2.10, T W X ! H and U WD I D Id
C.T Id/ is its -relaxation, where 2 .0; 2/. We will extend this figure in
Sect. 3.9.
2.2.3 Further Properties of the Metric Projection
The basic facts concerning firmly nonexpansive operators presented in the previous
section yield further properties of the metric projection.
Theorem 2.2.21. Let C H be a nonempty closed convex subset and PC W H ! H
be the metric projection onto C . Then the operator PC is:
(i) Idempotent, consequently Fix PC D C ,
(ii) A cutter,
(iii) Firmly nonexpansive,
(iv) Monotone and nonexpansive,
(v) Averaged.
Proof. (i) The property follows directly from the definition of the metric projec-
tion.
(ii) It follows from (i) and from the characterization of the metric projection (see
Theorem 1.2.4) that hz PC x; x PC xi 0 for all x 2 H and for all z 2
C D Fix PC , which means that PC is a cutter.
(iii) The property follows directly from (i), (ii) and from Theorem 2.2.5 (ii).
(iv) By Theorem 2.2.4, any firmly nonexpansive operator is monotone and nonex-
pansive. Therefore, the property follows from (iii).
(v) By the firm nonexpansivity of PC and by the implication (i))(iii) in The-
orem 2.2.10, we can write PC D 12 .S C Id/ for a nonexpansive operator
S W X ! H. Hence, PC is averaged.
t
u
Definition 2.2.22. Let C H be a nonempty closed convex subset. We call a
relaxation of the metric projection PC W H ! C a relaxed metric projection onto
the subset C and we denote it by PC; or, shortly, by P . If < 1, then P is called
an under-projection. If > 1, then P is called an over-projection. If D 2, then
P is called the reflection.
We have
PC; D P D Id C.PC Id/.
Corollary 2.2.23. Let C H be a nonempty closed convex subset, 0 and
P W H ! H be a relaxed metric projection. Then
(i) P is a nonexpansive operator for all 2 Œ0; 2,
(ii) Fix P D C for all > 0,
(iii) For all x 2 H, z 2 C and 2 .0; 2 the following inequality holds
2
kP x zk2 kx zk2 kP x xk2 . (2.49)

Consequently, P is 2

-strongly quasi-nonexpansive for all 2 .0; 2.
Proof. Part (i) follows from the equivalence (i),(ii) in Theorem 2.2.10, because
PC is firmly nonexpansive (see Theorem 2.2.21 (iii)). Part (ii) is obvious, because
Fix PC D C . Part (iii) follows now from Corollary 2.2.9. t
u
Corollary 2.2.24. Let C H be a nonempty closed convex subset and x; y 2 H.
Then
kPC x PC yk2 kx yk2 k.PC x x/ .PC y y/k2 (2.50)

kx yk .kPC x xk kPC y yk/ .
2 2
(2.51)
In particular,
kPC x zk2 kx zk2 kPC x xk2 (2.52)
for all x 2 H and z 2 C . Consequently, the metric projection PC W H ! C is
strongly quasi-nonexpansive.
Proof. By Theorem 2.2.21 (iii), the metric projection is firmly nonexpansive.
Therefore, inequalities (2.50) and (2.51) follow directly from the implication
(i))(v) in Theorem 2.2.10 and from the Cauchy–Schwarz inequality. The second
part follows directly from Theorem 2.2.21 (i). t
u
Fig. 2.11 Function x + α2u

f .x/ D kPX .x C ˛u/ xk
is nondecreasing x + α1u
PX (x + α2u)
PX (x + α1u) X
u
Corollary 2.2.25. Let T W X ! H and 2 .0; 2/. If T is a cutter, then for any
x 2 X and z 2 Fix T the following estimations hold
kPX T x zk2 kx zk2 .2 / kT x xk2
and
2
kPX T x zk2 kx zk2 kPX T x xk2 : (2.53)

Consequently, the operator PXT W X ! X is 2

Proof. Note that PX is a nonexpansive operator and that
Fix PX \ Fix T D X \ Fix T D Fix T ¤ ;.
Therefore, the corollary follows from Theorem 2.1.51. t

u
The following corollary will be useful in further parts of the book (see also [327,
Lemma 2] and [172, Lemma 1] for related results).
Corollary 2.2.26. Let x 2 X; u 2 H and 0 ˛1 < ˛2 . Then the following
inequality holds
kPX .x C ˛2 u/ xk2
kPX .x C ˛1 u/ xk2 C kPX .x C ˛2 u/ PX .x C ˛1 u/k2 . (2.54)
Consequently, the function f W RC ! RC , f .˛/ WD kPX .x C ˛u/ xk is non-

decreasing.
Corollary 2.2.26 is illustrated in Fig. 2.11.
Proof. Inequality (2.54) is obvious for ˛1 D 0. Let now ˛1 > 0. Take y WD x C ˛2 u,
z WD x C ˛1 u and D ˛˛12 . Then we have 2 .0; 1/ and .x z/ D 1
.y z/.
Now inequality (2.54) can be written in the form
kPX y xk2 kPX z xk2 C kPX y PX zk2 . (2.55)

The characterization of the metric projection (see Theorem 1.2.4) and its mono-
tonicity (see Theorem 2.2.21 (iv)) yield
hx PX z; PX y PX zi D hx z; PX y PX zi C hz PX z; PX y PX zi

hy z; PX y PX zi 0,
1
i.e., hx PX z; PX y PX zi 0, which is equivalent to (2.55), by Lemma 1.2.5. u

t
Let C H be convex. Define the distance function d.; C / W H ! R by d.x; C / D
infy2C kx yk. It follows from the continuity of the norm and from the definition
of the metric projection that
d.x; C / D d.x; cl C / D kx Pcl C xk .
Therefore, we suppose without loss of generality that C is closed. It turns out that
the functions d.; C / and d 2 .; C / are convex and differentiable.
Lemma 2.2.27. Let C H be a closed convex subset. Then the function
f W H ! R, f .x/ WD 12 d 2 .x; C / is differentiable and Df .x/ D x PC x for
all x 2 H.
Proof. (cf. [167, Proposition 2.2] and [209, Chap. IV, Example 4.1.6]) Let x; h 2 H.
It follows from the definition of the metric projection and from the properties of the
inner product that
f .x C h/ f .x/ hx PC x; hi
1 1
D kx C h PC .x C h/k2 kx PC xk2 hx PC x; hi
2 2
1 1
kx C h PC xk2 kx PC xk2 hx PC x; hi
2 2
1
D khk2 .
2
Similarly, by the definition of the metric projection, the Cauchy–Schwarz inequality
and the nonexpansivity of the metric projection, we obtain
f .x C h/ f .x/ hx PC x; hi
1 1
D kx C h PC .x C h/k2 kx PC xk2 hx PC x; hi
2 2
1 1
kx C h PC .x C h/k2 kx PC .x C h/k2 hx PC x; hi
2 2
1
D khk2 C hPC x PC .x C h/; hi
2
1
khk2 kPC x PC .x C h/k khk
2
1 1
khk2 khk2 D khk2 .
2 2
Now we see that
1 1
khk2 .f .x C h/ f .x/ hx PC x; hi/ khk2 .
2 2
Consequently,
f .x C h/ D f .x/ C hx PC x; hi C o.khk/.
Therefore, f is differentiable and Df .x/ D x PC x. t

u
Lemma 2.2.28. Let C H be a closed convex subset. The function h W H ! R,
h.x/ WD d.x; C / is convex and differentiable for all x … C and
x PC x
Dh.x/ D . (2.56)
kx PC xk
Proof. Since h.x/ D infy2C kx yk, the convexity of h follows from the fact that
the function p W H H ! H, p.x; y/ WD kx yk is convex (as a composition
of a linear function .x; y/ ! x y and a convex function z ! kzk) and from
pthat for a convex function p, the function infy2C p.; y/ is convex. Since
the fact
h D d 2 .; C /, the differentiability of h as well as equality (2.56) for x … C
follow from Lemma 2.2.27 and from the formula D.kzk/ D kzk z
for z ¤ 0. t
u
Corollary 2.2.29. Let C H be a closed convex subset. Then the function f W
H ! R, f .x/ WD 12 d 2 .x; C / is convex.
Proof. The function f is convex as a composition f D g ı h of a convex function
h WD d.; C / and of a convex and increasing function g W RC ! R, g.t/ WD 12 t 2 .
t
u
2.2.4 Metric Projection onto a Closed Subspace
Let V H be a closed linear subspace. Since V is convex, the metric projection

PV is well defined. The theorem below states some properties of PV . In particular,
the first part of the theorem states that the metric projection onto V is equal to the
orthogonal projection onto V .
Theorem 2.2.30. Let V H be a closed subspace and x 2 H, y 2 V . Then
(i) y D PV x if and only if hx y; zi D 0 for all z 2 V ,
(ii) PV is a bounded linear operator and kPV k D 1,
(iii) PV is self-adjoint,
(iv) Id D PV C PV ? .
Proof. (i) Necessity. Let y WD PV x. By the characterization of the metric
projection (see Theorem 1.2.4), hx y; z yi 0 for all z 2 V . Suppose
that hx y; w yi < 0 for some w 2 V . Let u WD 2y w. Then u 2 V because
V is a linear subspace and we have
hx y; u yi D hx y; y wi > 0.
This contradiction shows that hx y; z yi D 0 for all z 2 V . If we take z WD

0 2 V in the latter equality, we obtain hx y; yi D 0. Hence, hx y; zi D 0
for all z 2 V .
Sufficiency. Let hx y; zi D 0 for all z 2 V . Taking z WD y 2 V we obtain
in particular hx y; yi D 0. Hence, hx y; z yi D 0 for all z 2 V .
By the characterization of the metric projection (see Theorem 1.2.4), we have
y D PV x.
(ii) Let x1 ; x2 2 H, ˛1 ; ˛2 2 R, y1 WD PV x1 , y2 WD PV x2 and x WD ˛1 x1 C ˛2 x2 ,
y WD ˛1 y1 C ˛2 y2 . We show that y D PV x. By (i) we have
hx y; zi D h˛1 .x1 y1 / C ˛2 .x2 y2 /; zi

D ˛1 hx1 y1 ; zi C ˛2 hx2 y2 ; zi
D 0,
for all z 2 V , i.e., y D PV x. Since PV is nonexpansive, it is bounded.

Furthermore,
kPV xk D kPV x PV 0k kx 0k D kxk
for all x 2 H and kPV xk D kxk for x 2 V . Hence, kPV k D 1.

(iii) Let x; u 2 H. It follows from (i) that
hx; PV ui D hPV x; PV ui
and
hu; PV xi D hPV u; PV xi.
By the symmetry of the inner product
hPV x; ui D hx; PV ui;
i.e., PV is self-adjoint.
(iv) Let x 2 H. By (i), we have x PV x 2 V ? and x PV x D PV ? x. Since
x D PV x C .x PV x/, it holds x D PV x C PV ? x. t
u
Corollary 2.2.31. Let V H be a closed subspace and x 2 H. Then
hPV x; xi D kPV xk2 .
Proof. Since PV is self-adjoint (see Theorem 2.2.30 (iii)), we have hPV x; ui D

hx; PV ui for all u 2 H. If we take u WD PV x we obtain the desired property. t
u
Corollary 2.2.32. A bounded linear operator is an orthogonal projection if and
only if it is idempotent and self-adjoint.
Proof. The necessity follows from Theorems 2.2.21 (i) and 2.2.30 (iii). Let now
T W H ! H be idempotent and self-adjoint. Let V WD T .H/. It is clear that
V D Fix T and that V is a closed subspace. Now we show that T D PV . Let x 2 H
and z 2 V . Then
hT x; zi D hx; T zi D hx; zi,
i.e., hT x x; zi D 0. Theorem 2.2.30 (i) implies now that T D PV . t
u
2.2.5 Metric Projection onto a Closed Affine Subspace
Let A H be a closed affine subspace and a 2 A. Then A a is a closed linear

subspace. In order to show some properties of the metric projection PA we apply
Theorem 2.2.30 together with
PA x D PAa .x a/ C a (2.57)
(see Lemma 1.2.6).

Theorem 2.2.33. Let A H be a closed affine subspace and x; u; v; w 2 H,
a; y 2 A. Then
(i) y D PA x if and only if hx y; z yi D 0 for all z 2 A,
(ii) PA u PA v D PAa .u v/ D PA .u v/ PA 0,
(iii) hPA u PA v; wi D hu v; PAa wi D hu v; PA w PA 0i,
(iv) hPA u PA v; u vi D kPA u PA vk2 ,
(v) ku vk2 D kPA u PA vk2 C k.PA u u/ .PA v v/k2 ,
(vi) PA is an affine operator.
Proof. (i) Since A a is a linear subspace, v 2 A a if and only if v D z y,
for some z 2 A: By (2.57) and Theorem 2.2.30 (i), we have
y D PA x , y a D PAa .x a/ , hy a .x a/; z yi D 0
for any z 2 A.
(ii) By (2.57) and the linearity of PAa , we have
PA u PA v D PAa .u a/ C a .PAa .v a/ C a/
D PAa .u v/
D PAa .u v a/ PAa .a/
D PA .u v/ PA 0.
(iii) Since PAa is self-adjoint, (2.57) and (ii) yield
hPA u PA v; wi D hPAa .u a/ .PAa .v a/; wi

D hu v; PAa wi
D hu v; PAa .w a/ PAa .a/i
D hu v; PA w PA 0i.
(iv) Property (i) yields
hPA u u; PA u PA vi D 0 and hPA v v; PA u PA vi D 0
Therefore,
h.PA u u/ .PA v v/; PA u PA vi D 0,
i.e.,
hPA u PA v; u vi D kPA u PA vk2 .
(v) It follows from the properties of the inner product and from property (iv) that
k.PA u u/ .PA v v/k2

D kPA u PA vk2 C ku vk2 2hPA u PA v; u vi
D ku vk2 kPA u PA vk2 .
(vi) Let 2 R. Since A a is a closed subspace, (2.57) and Theorem 2.2.30 yield
PA ..1 /u C y/ D PAa ..1 /.u a/ C .y a// C a

D .1 /.PAa .u a/ C a/ C .PAa .y a/ C a/
D .1 /PA u C PA y
which completes the proof.

t
u
2.2.6 Properties of Relaxed Firmly Nonexpansive Operators
In this section we present relationships among families of relaxed firmly nonexpan-

sive operators, contractions, averaged operators and strongly quasi-nonexpansive
operators. Furthermore, we give properties of relaxed firmly nonexpansive operators
which are used in many constructions of algorithmic operators.
Theorem 2.2.34. An ˛-contraction is .1 C ˛/-relaxed firmly nonexpansive.
Proof. Let T W X ! H be an ˛-contraction, i.e., kT x T yk ˛ kx yk for all
x; y 2 X , where ˛ 2 .0; 1/. Let U WD 1C2 ˛ T 1C
1˛
˛ Id, or, equivalently,
1C ˛ 1˛
T D UC Id ,
2 2
1C ˛
i.e.. T is 2 -averaged. By the convexity of the norm and the nonexpansivity of T ,

2 1˛

kUx Uyk D .T x T y/ .x y/
1C ˛ 1C ˛
2 1˛
kT x T yk C kx yk
1C ˛ 1C ˛
2˛ 1˛
kx yk C kx yk
1C ˛ 1C ˛
D kx yk ,
i.e., U is nonexpansive. Therefore, T is .1 C ˛/-relaxed firmly nonexpansive as a

. 1C2 ˛ /-averaged operator (see Corollary 2.2.17). t
u
The next results show that a family of relaxed firmly nonexpansive operators is
closed under convex combination and under composition.
Theorem 2.2.35. Let i 2 Œ0; 2 and UP i W X ! H be i -relaxed firmly nonexpan-
m
sive, i 2 I WD f1; 2; : : : ; mg, U WD i D1 !i Ui for w D .!1 ; : : : ;P
!m / 2 m .
m
Then the operator U is -relaxed firmly nonexpansive, where D j D1 !j j .
Consequently, U is strictly relaxed firmly nonexpansive if i 2 .0; 2/ for some i 2 I
and the corresponding weight !i > 0.
Proof. Let Ui WD Id Ci .Ti Id/, where Ti W X ! H are firmly P nonexpansive,
i 2 Œ0; 2, i 2 I , and w D .!1 ; : : : ; !m / 2 m . It is clear that WD mj D1 !j j 2
Œ0; 2. For D 0 the claim is obvious, because U D Id in this case. Let now
2 .0; 2. Since
X
m
!i i
Pm D 1,
i D1 j D1 !j j
the operator
X
m
!i i
T WD Pm Ti
i D1 j D1 !j j
is firmly nonexpansive as a convex P

combination of firmly nonexpansive operators
Ti (see Corollary 2.2.20). Let U WD mi D1 !i Ui . Then we have
X
m
U D !i ŒId Ci .Ti Id/
i D1
X
m
D Id C !i i .Ti Id/
i D1
0 1 !
X
m X
m
!i i X m
!i i
D Id C @ !j j A Pm Ti Pm Id
j D1 i D1 j D1 !j j i D1 j D1 !j j
D Id C.T Id/
and, consequently, U is -relaxed firmly nonexpansive. The second part of the

theorem is obvious. u
t
Corollary 2.2.36. A convex combination of averaged operators is an averaged
operator.
Proof. It suffices to apply Corollary 2.2.17 to Theorem 2.2.35. t
u
Theorem 2.2.37. Let T; U W X ! X and ; 2 Œ0; 2. If T is -RFNE and U is
-RFNE, then the composition V WD U T is -RFNE, with
8
ˆ
ˆ 0 if D 0 and D 0
ˆ
ˆ
ˆ
<2 if .2 /.2 / D 0
D 4. C / 2
ˆ
ˆ D otherwise.
ˆ
ˆ 4 1
:̂ . C / C1
2 2
(2.58)
Proof. If D 0 or D 0, then T D Id or U D Id, respectively, and the claim is
obvious, because the operator Id is 0-RFNE. If D 2 or D 2, then T and U are
nonexpansive (see Theorem 2.2.10 (ii)) and U T is nonexpansive as a composition of
nonexpansive operators. Therefore, U T is 2-RFNE (see Corollary 2.2.13). Let now
; 2 .0; 2/ and x; y 2 H. Denote a1 WD T x x, a2 WD T y y, b1 WD U T x T x
and b2 WD U T y T y. It is clear that
y x D T y T x C a1 a2 . (2.59)
By Corollary 2.2.3, we have

1
hy x; a1 i C hx y; a2 i ka1 a2 k2

and
1
hT y T x; b1 i C hT x T y; b2 i kb1 b2 k2 .

Therefore, the properties of the inner product, equality (2.59) and Lemma 2.1.45
yield
1
hy x; U T x xi C hx y; U T y yi .k.U T x x/ .U T y y/k2 /

1
D hy x; a1 C b1 i C hx y; a2 C b2 i .k.a1 C b1 / .a2 C b2 /k2 /

D hy x; a1 i C hx y; a2 i C hT y T x; b1 i C hT x T y; b2 i
1
Cha1 a2 ; b1 b2 i .k.a1 C b1 / .a2 C b2 /k2 /

1 1
ka1 a2 k2 C kb1 b2 k2 C ha1 a2 ; b1 b2 i

1
.k.a1 C b1 / .a2 C b2 /k2

1 1 2 1 1 2 2
D ka1 a2 k C kb1 b2 k C 1 ha1 a2 ; b1 b2 i

s s 2
1 1
1 1
D .a1 a2 / .b1 b2 / 0.

Now it follows from Corollary 2.2.3 that U T is -RFNE. t

u
Remark 2.2.38. Because of Corollary 2.2.17, Theorem 2.2.37 can be stated
equivalently in terms of averaged operators:
if T is ˛-averaged and U is ˇ-averaged, where ˛; ˇ 2 .0; 1/, then U T is
ı-averaged, with
˛ C ˇ 2˛ˇ
ı WD . (2.60)
1 ˛ˇ
This result is due to Ogura and Yamada (see [273, Theorem 3 (b)]). The fact that a
composition of averaged operators T WD .1 ˛/ Id C ˛R and U WD .1 ˇ/ Id CˇS
is averaged follows also from the following identity (cf. [56, Lemma 2.2 and
Proposition 2.1])
.1 ˇ/˛ ˇ
U T D .1 ˛/.1 ˇ/ Id C.˛ C ˇ ˛ˇ/Œ RC S T ,
˛ C ˇ ˛ˇ ˛ C ˇ ˛ˇ
and from the fact that the family of nonexpansive operators is closed under
compositions and convex combinations (see Lemma 2.1.12). Note, however, that
[273, Theorem 3 (b)] is stronger than the result mentioned above, because
ı < ˛ C ˇ ˛ˇ
for ˛; ˇ 2 .0; 1/ and ı given by (2.60). It follows from Corollary 2.2.17 that the
result of Ogura and Yamada is equivalent to Theorem 2.2.37 with ; 2 .0; 2/.
Moreover, the proof of this theorem differs from the proof of [273, Theorem 3 (b)].
Note that the property of composition of relaxed cutters with a common fixed point,
expressed in Theorem 2.1.46 and the property of compositions of relaxed firmly
nonexpansive operators presented in Theorem 2.2.37 are similar. Therefore, it is
quite natural that the proofs of both theorems are similar. But Theorem 2.1.46 is no
special case of Theorem 2.2.37 because a cutter needs not to be firmly nonexpansive,
even if it is nonexpansive (see Example 2.2.8).
An equivalent formulation of the following result can be found in [349,
Lemma 1].
Corollary 2.2.39. Let T; U W H ! H be firmly nonexpansive. Then the com-
position V WD U T is 43 -relaxed firmly nonexpansive. Consequently, V is firmly
nonexpansive for all 2 Œ0; 34 and nonexpansive for all 2 Œ0; 32 . If, furthermore,
V has a fixed point, then V is strongly quasi-nonexpansive for all 2 .0; 32 /.
Proof. If we take D D 1 in Theorem 2.2.37, we obtain that V is 43 -relaxed
firmly nonexpansive. Recall that .V / D V (see Remark 2.1.3). Corollary 2.2.19
yields the firm nonexpansivity of V for all 2 Œ0; 34 . By the implication (i))(ii)
in Theorem 2.2.10, V is nonexpansive for all 2 Œ0; 32 . Now let Fix V ¤ ; and
2 .0; 32 /. Then V is strongly quasi-nonexpansive, by Corollary 2.2.9. t
u
Yamada et al. also proved that, for any > 32 , there exist firmly nonexpansive
operators T; U such that V is not nonexpansive, where V WD U T (see [349,
Remark 1 (b)]). This means that the constant 32 is optimal in Corollary 2.2.39.
Remark 2.2.40. Let T; U W H ! H be firmly nonexpansive having a common fixed
point. Then it follows from Corollaries 2.2.39 and 2.2.15 that U T is 12 -strongly quasi
nonexpansive. A special case of this property was proved in [152, Proposition 1] for
T; U being orthogonal projections onto subspaces of H.
Corollary 2.2.41. Let T W H ! H be firmly nonexpansive and 2 Œ0; 2. If V is a
closed affine subspace, then the operator U WD .1 /PV C PV T is 4
4
-relaxed
firmly nonexpansive.
Proof. Let V be a closed affine subspace. By Theorem 2.2.33 (vi), the operator PV
is affine, consequently,
.1 /PV C PV T D PV T .
4
Now it follows from Theorem 2.2.37 that U is 4 -relaxed firmly nonexpansive,
because the metric projection is firmly nonexpansive. t
u
A weaker formulation of Corollary 2.2.41 can be found in [349, Lemma 2], where
T WD PC for a closed convex subset C .
Theorem 2.2.42. Let Ti W X ! X be i -relaxed firmly nonexpansive, where
˛i 2 Œ0; 2, i 2 I . Then the composition Sm WD Tm Tm1 : : : T1 is m -relaxed firmly
nonexpansive, where m D 0 if i D 0 for all i 2 I , m D 2 if i D 2 for at least
one i 2 I and
2
m D
1 , (2.61)
1
21 C 2
22 C ::: C m
2m C1
otherwise. Moreover,
2m mini 2I i 2m maxi 2I i
m , (2.62)
.m 1/ mini 2I i C 2 .m 1/ maxi 2I i C 2
consequently, m < 2 if i < 2 for all i 2 I .

Proof. Let i D 0 for all i 2 I . In this case, Sm D Id, i.e., Sm is 0-relaxed
firmly nonexpansive. Let i D 2 for some i 2 I . Then Sm is nonexpansive as a
composition of nonexpansive operators, i.e., Sm is 2-RFNE (see Corollary 2.2.13).
Let now i 2 Œ0; 2/ for all i 2 I and j > 0 for at least one j 2 I . We prove by
induction with respect to m that Sm is m -RFNE, where m is given by (2.61). Note
that (2.61) is equivalent to
m 1 2 m
D C C ::: C . (2.63)
2 m 2 1 2 2 2 m
10 For m D 2 the above fact follows directly from Theorem 2.2.37.

20 Suppose that, for some m D k, the operator Sm is m -RFNE. We prove that
SkC1 is kC1 -RFNE. If kC1 D 0, then TkC1 D Id, SkC1 is a composition of k
operators which are relaxed firmly nonexpansive and the claim follows from the
induction assumption. Let now kC1 2 .0; 2/, then we have SkC1 D TkC1 Sk , where
TkC1 is kC1 -RFNE and Sk is k -RFNE. It follows from Theorem 2.2.37 that SkC1
is -RFNE, where
2
D
1 ,
k kC1
2k
C 2kC1
C1
and, together with (2.63), this gives for m D k
k kC1
D C
2 2 k 2 kC1
1 2 k kC1
D C C ::: C C ,
2 1 2 2 2 k 2 kC1
consequently, D kC1 . We have proved that, for any m 2 N, the operator Sm is

m -RFNE, where m is given by (2.61).
Now we prove (2.62). By (2.63), we have
mini 2I i m maxi 2I i
m m ,
2 mini 2I i 2 m 2 maxi 2I i
which is equivalent to (2.62). t

u
A part of the results presented in Theorem 2.2.42 can be found in [122, Lemma
2.2 (iii)], where it was proved that a composition of i -RFNE operators Ti , where
i 2 Œ0; 2, i 2 I , is .m1/
2m maxi 2I i
maxi 2I i C2 -SQNE.
Corollary 2.2.43. Let Ti W X ! X , i 2 I , be firmly nonexpansive. Then the oper-

ator Sm D Tm : : : T1 is m -relaxed firmly nonexpansive with m D mC1
2m
. Conse-
1
quently, Sm is m -strongly quasi-nonexpansive.
Proof. It suffices to take i D 1; i 2 I , in (2.61). The second part of the corollary
follows from Corollary 2.2.9. t
u
Dye and Reich obtained a result which is a special case of the second part of
Corollary 2.2.43 with Ti , i 2 I , being orthogonal projections onto one-dimensional
subspace of a Hilbert space (see [152, Theorem on page 109]).
Corollary 2.2.44. Let Ti W X ! X be firmly nonexpansive, PSmi WD Ti : : : T1 , i 2 I ,
and w D .!1 ; : : : ; !m / 2 m . Then the operator S WD i D1 !i Si is -relaxed
firmly nonexpansive, where
X
m
2i
D !i . (2.64)
i D1
i C1
Proof. By Corollary 2.2.43, the operators Si are i -relaxed firmly nonexpansive

with i D i C1
2i
. By Theorem 2.2.35, S is -relaxed firmly nonexpansive, where
is given by (2.64). t
u
The composition of firmly nonexpansive operators needs not to be firmly nonexpan-
sive (see Exercise 2.5.10).
Definition 2.2.45. Let T W X ! H, 2 Œ0; 2. The operator R W X ! H,
R WD PX T is called a projected relaxation of T .
The theorem below gives important properties of the projected relaxation of a
firmly nonexpansive operator.
Theorem 2.2.46. Let T W X ! H be firmly nonexpansive, R WD PX T , be the
projected relaxation of T , where 2 .0; 2/. Then:
4
(i) R is 4 -relaxed firmly nonexpansive.
(ii) Fix R D Fix.PX T /.
(iii) If Fix.PX T / ¤ ;, then the operator R is 2

2
-SQNE, i.e.,
2
kR x zk2 kx zk2 kR x xk2 (2.65)
2
for all x 2 X and for all z 2 Fix.PX T /.

(iv) If Fix T ¤ ;, then the operator R is 2
-SQNE.
Proof. (i) Since the metric projection PX is firmly nonexpansive, it is 1-relaxed

4
firmly nonexpansive. By Theorem 2.2.37, the operator R is 4 -RFNE.
(ii) This property follows from Corollary 1.2.10.
(iii) Since R is -RFNE, where D 4 4
(see (i)), Corollary 2.2.9 yields
2
kR x zk2 kx zk2 kR x xk2

2
D kx zk2 kR x xk2 .
2
(iv) The claim follows from Corollary 2.2.25.
t
u
If X D H, then R D T , nevertheless, estimation (2.65) is weaker than
estimation (2.47). Furthermore, estimation (2.65) is weaker than estimation (2.53).
Note, however, that we have supposed in Corollary 2.2.25 that the operator T W
X ! H is a cutter, consequently Fix T ¤ ;, while in Theorem 2.2.46 (iii) we have
supposed that Fix.PX T / ¤ ;, which is weaker than the assumption Fix T ¤ ;.
2.2.7 Fixed Points of Firmly Nonexpansive Operators
A firmly nonexpansive operator is nonexpansive (see Theorem 2.2.4), therefore, the

subset of its fixed points is closed and convex (see Proposition 2.1.11). In this section
we show that the subsets Fix T for FNE- and for NE-operators are intersections of
half-spaces, which also yields the closedness and convexity of Fix T . Equivalent
formulations to the results below can be found in [185, Equalities (11.3) and (11.4)].
Theorem 2.2.47. Let X H be closed convex and T W X ! H be firmly
nonexpansive. Then
\
Fix T D fz 2 X W hT x x; T x zi 0g.
x2X
Consequently, Fix T is a closed convex subset.

2.3 Strongly Nonexpansive Operators 91
Proof. Since a firmly nonexpansive operator with a fixed point is a cutter (see
Theorem 2.2.5), the theorem follows from Lemmas 2.1.36 and 2.1.35. t
u
Corollary 2.2.48. Let X H be closed and convex. The subset of fixed points of a
nonexpansive operator S W X ! H has the form
\
Fix S D fz 2 X W 2hz x; S x xi kS x xk2 g, (2.66)
x2X
consequently, Fix S is a closed convex subset.

Proof. Let S W X ! H be nonexpansive. By Corollary 2.2.13, we have S D 2T Id
for a firmly nonexpansive operator T . It is clear that Fix S D Fix T . Theorem 2.2.47
yields now
\ 1 1
Fix S D fz 2 X W h .S x C x/ x; .S x C x/ zi 0g
x2X
2 2
which is equivalent to (2.66). t

u
2.3 Strongly Nonexpansive Operators
Definition 2.3.1. An operator T W X ! H is called strongly nonexpansive (SNE),

if T is nonexpansive and for all sequences fx k g1 k 1
kD0 ; fy gkD0 X the following
implication is true

k .x
k k
y /is bounded and H) .x k y k / .T x k T y k / ! 0,
x y k T x k T y k ! 0
The notion of strongly nonexpansive operators in Banach spaces was proposed

by Bruck and Reich in [51, Sect. 1], where also properties of these operators are
proved (see also [23, Sect. 4.3]).
Remark 2.3.2. It is clear that a contraction is a strongly nonexpansive operator.
Indeed, let T be a contraction, i.e., kT x T yk ˛ kx yk for all x;
yk 2 Xk and
for a constant ˛ 2 .0; 1/, and .x k
y k
/ be bounded and such that x y
k
T x T y k ! 0. Then we have
k
x y k T x k T y k .1 ˛/ x k y k ! 0.
Consequently, x k y k ! 0 and T x k T y k ! 0, i.e., T is strongly nonexpansive.

Remark 2.3.3. (S. Reich, A private communication (2009)) Let X H be com-

pact. Then a strictly nonexpansive operator defined on X is strongly nonexpansive.
Indeed, let T W X ! H be strictly nonexpansive, i.e.,
kT x T yk < kx yk or x y D T x T y
for all x; y 2 X , and X be compact. We show that T is strongly nonexpansive.

k
thatsequences fx k g1 k 1 k
Suppose
k kD0 and fy gkD0 are given such that x y
T x T y k ! 0 and that there exist subsequences fx nk g1 fx k g1 and
kD0 kD0
fy nk g1 k 1
kD0 fy gkD0 and a constant " > 0 such that
k.x nk y nk / .T x nk T y nk /k ".
Since X is compact, we can suppose without loss of generality that x nk ! x and

y nk ! y. Since T is continuous as a nonexpansive operator, we have T x nk ! T x
and T y nk ! T y. Hence, we obtain in the limit kx yk D kT x T yk, which
yields, due to strict nonexpansivity of T , that x y D T x T y. On the other hand,
we have
k.x y/ .T x T y/k D lim k.x nk y nk / .T x nk T y nk /k ",

k
a contradiction, which shows that T is strongly nonexpansive.

Theorem 2.3.4. Let T W X ! H be firmly nonexpansive and 2 .0; 2/. Then the
relaxation T of T is strongly nonexpansive.
k
Proof. Let fx k g1 k 1 k
kD0 ; fy gkD0 X be such that x y is bounded and
k
x y k T x k T y k ! 0:
The firm nonexpansivity of T yields

˚ the nonexpansivity
of T (see
1Theorem 2.2.10
(ii)), consequently, the sequence x k y k C T x k T y k kD0 is bounded.
Therefore, by the obvious equality T x x D .T x x/ and by Corollary 2.2.15,
we have
k
.x y k / .T x k T y k /2
2
D .T x k x k / .T y k y k /

x k y k 2 T x k T y k 2

2

D x k y k T x k T y k x k y k C T x k T y k ! 0,
2
k
i.e., .x y k / .T x k T y k / ! 0 and T is strongly nonexpansive. t
u
In the previous sections we have proved that the following classes of operators are
closed under composition and under convex combination:
(a) The class of strictly relaxed cutters with a common fixed point (see Theo-
rems 2.1.46 and 2.1.50),
(b) The class of strongly quasi-nonexpansive operators with a common fixed point
(see Corollary 2.1.47 and Theorem 2.1.50),
(c) The class of strictly relaxed firmly nonexpansive operators (see Theo-
rems 2.2.37 and 2.2.35)
(d) The class of averaged operators (see Remark 2.2.38 and Corollary 2.2.36).
It turns out that the class of strongly nonexpansive operators has the same properties.
The first part of the theorem below was proved by Bruck and Reich in [51,
Proposition 1.1] and the other one by Reich in [295, Lemma 1.3].
Theorem 2.3.5. Let T1 ; T2 W X ! X be strongly nonexpansive and T have one of
the following forms:
(i) T WD T2 T1 ,
(ii) T WD .1 /T1 C T2 , where 2 Œ0; 1.
Then T is strongly nonexpansive.
Proof. By Lemma 2.1.12, the operator T is nonexpansive. Let the ksequences

fx k 1
g ; fy k 1
g X be such that .x k
y k
/ is bounded and x y k
kD0 kD0
T x k T y k ! 0.
(i) By the nonexpansivity of T1 and T2 , we have

k
T x T y k D T2 .T1 x k / T2 .T1 y k / T1 x k T1 y k x k y k ,
k 0, consequently,
k
x y k T1 x k T1 y k ! 0
and

T1 x k T1 y k T2 .T1 x k / T2 .T1 y k / ! 0.
Since T1 and T2 are strongly nonexpansive, we have

.x k y k / .T x k T y k / D
.x k y k / .T1 x k T1 y k / C .T1 x k T1 y k / .T2 .T1 x k / T2 .T1 y k // ! 0;
i.e., T is strongly nonexpansive.
(ii) The assertion is clear when D 0 or D 1. Let 2 .0; 1/. By the convexity
of the norm and the nonexpansivity of T1 and T2 , we have
Fig. 2.12 SNE operator Ux

which is not AV
Tx
y
x
B(0, 1)
Ty
Uy
k
T x T y k D .1 /T1 x k C T2 x k .1 /T1 y k T2 y k

.1 / T1 x k T1 y k C T2 x k T2 y k

.1 / x k y k C x k y k D x k y k ,
consequently,
k
x y k T x k T y k

.1 /.x k y k T1 x k T1 y k / C .x k y k T2 x k T2 y k /.
Therefore, k
x y k T1 x k T1 y k ! 0
and k
x y k T2 x k T2 y k ! 0.
By the strong nonexpansivity of T1 and T2 , we have now
.x k y k / .T x k Ty k /
D .1 /..x k y k / .T1 x k T1 y k // C ..x k y k / .T2 x k T2 y k // ! 0,
i.e., T is strongly nonexpansive.

t
u
The following example shows that the class of averaged operators or, equivalently,
the class of strictly relaxed firmly nonexpansive operators is a proper subclass of the
class of strongly nonexpansive operators.
Example 2.3.6. Let X WD B.0; 1/ H be a unit ball, U W H ! H be a unitary
operator such that hUx; xi D 0 for all x 2 H (e.g., U W R2 ! R2 is defined by
Ux WD .2 ; 1 / for x D .1 ; 2 / 2 R2 with the standard inner product) and the
operator T W X ! X be defined by
T x WD ˛.x/Ux.
with ˛.x/ WD 1 12 kxk (see Fig. 2.12).

It is clear that ˛.x/Ux D U.˛.x/x/, consequently,
kT x T yk D k˛.x/Ux ˛.y/Uyk
D kU.˛.x/x/ U.˛.y/y/k
D k˛.x/x ˛.y/yk .
A straightforward calculation shows that
kx yk2 kT x T yk2
D kx yk2 k˛.x/x ˛.y/yk2
1 1
D kxk4 kyk4 C kxk3 C kyk3
4 4
1
hx; yi.kxk C kyk kxk kyk/
2
1
D .kxk kyk/2 .kxk C kyk/.1 .kxk C kyk//
4
1
C.kxk kyk hx; yi/.kxk C kyk kxk kyk/.
2
We have kxkCkyk kxkkyk, since kxk ; kyk 2 Œ0; 1. This fact and the Cauchy–
Schwarz inequality yield
1 1 1 1
kxk C kyk kxk kyk kxk kyk kxk kyk hx; yi,
2 2 4 4
consequently,
kx yk2 kT x T yk2
1 1
.kxk kyk/2 .kxk C kyk/.1 .kxk C kyk// C .kxk kyk hx; yi/2
4 4
and T is nonexpansive. We apply the above inequalities

tox D x k and y D y k .
Suppose that x k ; y k 2 X and that x k y k T x k T y k ! 0. Then, of course,
k
x y k 2 T x k T y k 2 ! 0;

because x k y k C T x k T y k is bounded. Therefore,
k k
x y ! 0

(note that 1 14 .x k C y k / 12 ) and
k k
x y hx k ; y k i ! 0.
Now we have
k
x y k 2 D x k y k 2 C 2 x k y k hx k ; y k i ! 0,
i.e., .x k y k / ! 0. Furthermore, .T x k T y k / ! 0, by the nonexpansivity of T ,

consequently,
.x k y k / .T x k T y k / ! 0,
i.e., T is strongly nonexpansive. Note that z D 0 is the unique fixed point
of T . Suppose that T is ˛-averaged, for a constant ˛ 2 .0; 1/. By Corol-
lary 2.2.17, the operator T is .2˛/-relaxed firmly nonexpansive. Consequently,
the operator V D T , where D .2˛/1 2 . 12 ; C1/ is firmly nonexpansive (see
Corollary 2.2.19) and V is a cutter (see Theorem 2.2.5), i.e.,
hx; x T xi C 2 kx T xk2
D hx C .T x x/; x T xi
D hT x; x T xi
D h0 V x; x V xi 0.
Dividing the inequalities above by > 0, we obtain, for all x ¤ z,
1 hx; x T xi kxk2 1
< D D .
2 kx T xk 2 2
kxk C ˛ .x/ kxk
2 2
1 C .1 12 kxk/2
Applying the inequalities above to a sequence fx k g1

kD0 with limk x D 0, we obtain
k
1 1 1
< lim D ,
2 k 1 C .1 1 x k /2 2
2
a contradiction, which proves that T is not averaged.
2.4 Generalized Relaxations of Algorithmic Operators
In the definition of a relaxation T of an operator T W X ! H we have supposed

that the relaxation parameter 2 Œ0; 2 (see Definition 2.1.2). Furthermore, the
assumption 2 .0; 2/ is necessary for the strong quasi nonexpansivity of the
-relaxation of a firmly nonexpansive operator T with Fix ¤ ; (see proof of
2.4 Generalized Relaxations of Algorithmic Operators 97
Theorem 2.1.39). However, in some applications, relaxations of operators (e.g., of

firmly nonexpansive ones) with the relaxation parameter which are greater than 2
are successfully used. In general, the convergence of sequences generated by such
operators is not guaranteed. It turns out that, if we allow to vary the relaxation
parameter in dependence on the current point, in such a way that the relaxed operator
is a cutter, then we can apply the usual convergence analysis for sequences generated
by such an operator. Below we define a generalization of a relaxation of an operator,
which permits us to extend the convergence results to sequences generated by the
generalized relaxation.
Definition 2.4.1. Let T W X ! H, 2 Œ0; 2 and
W X ! .0; C1/. The operator
T
; W X ! H,
T
; x WD x C
.x/.T x x/ (2.67)
is called the generalized relaxation of T , the value is called the relaxation
parameter and
is called the step size function. If
.x/ 1 for all x 2 X , then the
operator T
; is called an extrapolation of T .
Some special cases of generalized relaxations of some classes of nonexpansive
operators, presented in various forms and applied in most cases to the convex
feasibility problems, were studied by Gurin et al. [196, Sect. 3], Pierra [284,
Sect. 1], Cegielski [62, Sect. 4.3], Kiwiel [229, Sect. 3], Bauschke [17, Sects. 7.3 and
8.3], Combettes [118, Sects. 5.4–5.8], [120, Sect. IV], Bauschke et al. [30, Sect. 3]
Bauschke et al. [25] and by Cegielski and Suchocka in [76].
In this section we present properties of generalized relaxations of cutters and
give conditions for a generalized relaxation to be strongly quasi-nonexpansive.
These properties will be applied in one of the next chapters in order to prove the
convergence of sequences generated by such operators.
Denote T
D T
;1 .
Remark 2.4.2. Let T W X ! H, 2 Œ0; 2 and
W X ! .0; C1/.
(a) If
.x/ D 1 for all x 2 X , then T
; D T , i.e., the generalized relaxation of T
is reduced to the classical relaxation of T .
(b) The values of the step size function
for x 2 Fix T have no influence on the
form of an operator T
; because T
; jFix T D Id for any step size function
and
for any 2 .0; 2. Therefore, we can suppose without loss of generality that

.x/ D 1 for all x 2 Fix T .
(c) For any x 2 X the following equalities hold
T
; x x D
.x/.T x x/ D .T
x x/, (2.68)
i.e., T
; is a -relaxation of an operator T
.
(d) For any ¤ 0 it holds Fix T
; D Fix T (cf. Remark 2.1.4).
The corollary below is a version of Theorem 2.1.39.
Corollary 2.4.3. Let T W X ! H have a fixed point,

W X ! .0; C1/ be a step
size function and 2 .0; 2/. Then T
is a cutter if and only if T
; is 2

-strongly
quasi-nonexpansive. In both cases
kT
; x zk2 kx zk2 .2 /
2 .x/ kT x xk2 (2.69)

Proof. By Remark 2.4.2 (c), T
; is the -relaxation of T
. The first part of the
theorem follows now from Theorem 2.1.39. The 2
-strong quasi nonexpansivity
of T
; means
2
kT
; x zk2 kx zk2 kT
; x xk2 .

Applying now (2.68) to the inequality above we obtain (2.69). t
u
Let T W X ! H be an operator with a fixed point. Our aim is to give sufficient
conditions for the step size function
W X ! .0; C1/, at which T
is a cutter. The
following definition was proposed in [70, Definition 9.17].
Definition 2.4.4. We say that an operator T W X ! H with a fixed point is oriented
if for all x … Fix T
hz x; T x xi
ı.x/ WD inf > 0. (2.70)
z2Fix T kT x xk2
If ı.x/ ı > 0 for all x … Fix T , then we say that T is strongly oriented.
It follows from Remark 2.1.31 that T W X ! H is strongly oriented if and only
if T is an ˛-relaxed cutter for some ˛ > 0.
Corollary 2.4.5. Let T W X ! H be an oriented operator with Fix T ¤ ;. If a
step size function
W X ! .0; C1/ satisfies the inequality
hz x; T x xi

.x/ (2.71)
kT x xk2
for all x … Fix T and z 2 Fix T , then T

is a cutter. Consequently, for any 2 .0; 2/,
the generalized relaxation T
; of T is 2
Proof. Let x … Fix T , z 2 Fix T and

W X ! .0; C1/ be a step size function
satisfying (2.71). The existence of
follows from the assumption that T is oriented.
Then (2.68) and inequality (2.71) yield
hz x; T
x xi D hz x;
.x/.T x x/i
k
.x/.T x x/k2
D kT
x xk2 .
By the equivalence (a),(b) in Lemma 1.2.5, we have
hz T
x; x T
xi 0,
2
i.e., T
is a cutter. The
-strong quasi nonexpansivity of T
; follows now from
Corollary 2.4.3. t
u
The convergence of sequences generated by generalized relaxations of an algorith-
mic operator U , which we present in the next chapter, requires a stronger condition
than (2.71). As we will see, the convergence holds if we additionally suppose that
U is strongly oriented, or, equivalently, that the step size
.x/ ˛ for all x 2 X
and for a constant ˛ > 0. This leads to ˛-relaxed cutters (see Remark 2.1.31).
It is clear that if an operator T W X ! H with a fixed point is an ˛-relaxed cutter
for some ˛ > 0, then there exists a step size function
W X ! .0; C1/ satisfying
inequality (2.71), e.g.,
.x/ D ˛ 1 for all x 2 X (cf. (2.22)). In practice, however, it
is important to determine a step size
.x/ for which the difference between the right-
and the left-hand side of inequality (2.71) is as small as possible for all z 2 Fix T .
Theoretically, the best possibility would be
.x/ D ı.x/ for x … Fix U , where
ı.x/ is defined by (2.70), but the computation of ı.x/ is, in most cases, impossible,
because we usually do not know Fix T explicitly.
Having an ˛-relaxed cutter T we can construct its generalized relaxation T
;
with the range of the step size function
contained in Œ˛; C1/ and satisfying
assumptions of Corollary 2.4.5. The corollary below gives a collection of operators
which are ˛-relaxed cutters.
Corollary 2.4.6. Let U W X ! H have a fixed point. Then U is an ˛-relaxed cutter
with:
(a) ˛ D 1 if U is firmly nonexpansive,
(b) ˛ D if U is -relaxed firmly nonexpansive, where 2 .0; 2,
(c) ˛ D 2 if U is nonexpansive,
(d) ˛ D 2 if U is -averaged, where 2 .0; 1/,
(e) ˛ D 1Cˇ
2
if U is ˇ-strongly quasi-nonexpansive, where ˇ > 0.
Proof. (a) Let U be firmly nonexpansive. Then it follows from the first part of
Theorem 2.2.5 that T is a cutter, i.e., T is a 1-relaxed cutter.
(b) Let 2 .0; 2 and U WD Id C.T Id/ for a firmly nonexpansive operator T .
Then, by (a), we have
hz x; Ux xi D hz x; T x xi
1
kT x xk2 D kUx xk2 .

(c) Let U be nonexpansive. Then U D 2T Id for a firmly nonexpansive operator
T (see Corollary 2.2.13) and this case is covered by (b) for D 2.
(d) Let 2 .0; 1/ and U WD .1 / Id CS for a nonexpansive operator S . Then
U is 2-relaxed firmly nonexpansive (see Corollary 2.2.17). The claim follows
now from (b) with D 2.
(e) Let ˇ > 0 and U be ˇ-strongly quasi-nonexpansive. It follows from Corol-

2
lary 2.1.43 that U is a 1Cˇ -relaxed cutter.
t
u
In the lemma below we state some obvious properties of the generalized relaxation.
Lemma 2.4.7. Let T W X ! H be an operator with a fixed point, and f
j gj 2J W
X ! .0; C1/ be a family of step size functions.
(i) If T
j , j 2 J , are cutters, then Tsupj 2J
j is a cutter.
(ii) If
i
j for some i; j 2 J and T
j is a cutter, then T
i is a cutter.
If T is a cutter, then there exists a step size function
with
.x/ 1 for
all x … Fix T , for which T
is a cutter, e.g., a step size function
defined by
.x/ D ı.x/, where ı.x/ is given by (2.70) for x … Fix T . Consequently, the
generalized relaxation T
; is strongly quasi-nonexpansive for any 2 .0; 2/ (see
Theorem 2.4.5). Note that
.x/ 1, by Remark 2.1.31. The following example
shows, however, that there is a cutter T such that the generalized relaxation T
;
is strongly quasi-nonexpansive for all 2 .0; 2/ if and only if
.x/ 1 for all
x … Fix T .
Example 2.4.8. Let T
; be a generalized relaxation of the metric projection PC W
H ! H, where C H is a nonempty closed convex subset, i.e., T
; .x/ D
x C
.x/.PC x x/ for a relaxation parameter 2 .0; 2/ and for some step
size function
W H ! .0; C1/. For any x 2 H we have
kT
; x PC xk2 D kx C
.x/.PC x x/ PC xk2
D kx PC xk2 C 2
2 .x/ kPC x xk2 2
.x/ kPC x xk2
D kx PC xk2
.x/.2
.x// kPC x xk2 ,
consequently,
2
.x/
kT
; x PC xk2 D kx PC xk2 kT
; x xk2 . (2.72)

.x/
Let 2 .0; 2/. Suppose that T

; is strongly quasi-nonexpansive, i.e.,
kT
; x zk2 kx zk2 ˛ kT
; .x/ xk2 (2.73)
for some ˛ > 0, for all x 2 H and z 2 C WD Fix PC . Note that ˛ can depend on .
Let x … C and z D PC x. Then (2.72) and (2.73) yield
2
.x/
0<˛ .

.x/
There exists a constant ˛ satisfying the above inequalities for all 2 .0; 2/ if and
only if
.x/ 1.
Fig. 2.13 Operators T and

x
T
from Example 2.4.9
T¾x
√
z 1 2 2
Tx Ty
C
y
D
If T W X ! H is firmly nonexpansive with a fixed point, then T is oriented

and for the function ı defined by (2.70) it holds ı.x/ 1 for all x … Fix T .
Therefore, Corollary 2.4.5 applied to a firmly nonexpansive operator T with the
step size
.x/ WD ı.x/ for x … Fix T is an extension of Theorem 2.2.5 (i) for
generalized relaxations. Unfortunately, Theorem 2.2.5 (ii) cannot be analogously
extended. The fact that T W X ! H is a projection and T
is a cutter for some
step size function
W X ! .0; C1/ does not yield the firm nonexpansivity of T .
Even if we additionally suppose that T is nonexpansive, T needs not to be firmly
nonexpansive (see Example 2.2.7). Moreover, a projection T for which T
is a cutter
needs not to be continuous.
p
Example 2.4.9. Let H D R2 , C WD B.0; 1/, D WD bd B.0; 2/; a D .1; 0/.
Define the operator T W R2 ! R2 by
8
< PC x for kxk 2
T x WD a for kxk > 2, 1 0
:
a for kxk > 2, 1 < 0.
It is clear that T is a projection with Fix T D C . For kxk > 2, let Ux be the
unique common point of the segment Œx; T x and the circle D. Define the function
W R2 ! R by (
1 if kxk 2

.x/ WD kUxxk
kT xxk if kxk > 2.
Observe that for kxk > 2 it holds T

x D Ux. It follows from geometrical
considerations (note that the p square circumscribed on the circle bd B.0; 1/ is
inscribed in the circle bd B.0; 2/) that for all x 2 R2 and z 2 C D Fix T it
holds
hx T
.x/; z T
.x/i 0
(see Fig. 2.13). Therefore, T
is a cutter. Note that T is not continuous, therefore,
T cannot be firmly nonexpansive.
FNE operators
cutters with
a common fixed point
is closed under
relaxed cutters with convex combination
The family of RFNE operators
strictly RFNE
operators is closed under
composition
SQNE operators with
SNE operators
Fig. 2.14 Closedness of families of algorithmic operators
SUMMARY
In Fig. 2.14 we recall in a short form the properties of algorithmic operators

which were presented in this chapter. These properties are useful in construction
of projection methods. We will describe these constructions in Chaps. 4 and 5.
2.5 Exercises
Exercise 2.5.1. Show that .T / D T for all ; 2 R.

Exercise 2.5.2. Let T W R ! R,

x2 if jxj 34
Tx D
jxj 3
16 if jxj > 34 .
Show that T is quasi-nonexpansive and continuous, but T is not a nonexpansive

operator.
Exercise 2.5.3. Let fUi gi 2I be a finite family of operators, Ui W X ! H, i 2 I .
Let w W X ! m be a weight function satisfying !i .x/ > 0 for some i.x/ D
argmaxi 2I kUi x xk for all x 2 X . Prove that w is appropriate with respect to the
family fUi gi 2I .
2.5 Exercises 103
Exercise 2.5.4. Prove that the assumption on the C -strict quasi nonexpansivity
in Theorem 2.1.26 (i) can be weakened. In this case it suffices to suppose that all
Ui are quasi-nonexpansive, i 2 I , and at least one of them is C -strictly quasi-
nonexpansive. The assumption that the weight function w is appropriate should be
replaced in this case by a stronger one, namely: wj .x/ > 0 for all x such that
I.x/ ¤ ; and for all j 2 I.x/.
Exercise 2.5.5. Prove Corollary 2.1.29.
Exercise 2.5.7. Show that the operator T W R2 ! R2 ,
T x WD .1 cos ' 2 sin '; 1 sin ' C 2 cos '/
is nonexpansive and monotone for ' 2 .0; =2/, but T is not firmly nonexpansive.
Exercise 2.5.9. Show that the operator T presented in Example 2.2.7 is nonexpan-
sive and that T is a separator of A, but T is not firmly nonexpansive.
Exercise 2.5.10. Let H D R2 , A WD fx 2 R2 W 2 D 0g and B WD fx 2 R2 W
1 D 2 g. By Theorem 2.2.21 (iii) PA and PB are firmly nonexpansive. Check that
T WD PB PA is not firmly nonexpansive.
•
Chapter 3
Convergence of Iterative Methods
3.1 Iterative Methods
Convex minimization problems are usually presented as minimization of a continu-

ous convex function on a Hilbert space H or on a closed convex subset X . The
subset of solutions of this problem is a closed convex subset M X . We recall as
an example the split feasibility problem and a connected problem of minimization
of a convex proximity function (see Sect. 1.3.7). The CMP is equivalent to finding a
fixed point of an operator U W X ! H (usually nonexpansive) corresponding to the
objective function. The operator U is constructed in such a way that M D Fix U .
Iteration methods for solving these problems (finding an element x 2 M ) have the
form of a recurrence
x kC1 D Ux k
or a more general recurrence

x kC1 D Uk x k ,
k 0, where x 0 2 X is arbitrary, fUk g1

kD0 is a sequence of algorithmic operators
Uk W X ! X such that
1
\
Fix Uk Fix U D M:
kD0
If Uk D U for all k 0, then we say that the iterative method is autonomous,

otherwise, we say that the method is nonautonomous. In the first case we can
also write x k D U k x 0 . The algorithmic operator or the sequence of algorithmic
operators describing the method should be constructed in such a way that any
sequence fx k g1
kD0 generated by the method converges (weakly or strongly) to a
solution x 2 M .
106 3 Convergence of Iterative Methods
3.2 Properties of the Weak Convergence
In this section we present properties of the weak convergence which will be applied
in the sequel.

Lemma 3.2.1. If x k * x 2 H, then lim infk x k kxk.
Proof. Let x k * x. The lemma is clear for x D 0. Let now x ¤ 0. By the Cauchy–
Schwarz inequality, we have

lim inf kxk x k lim infhx; x k i D kxk2 ,
k k

i.e., lim infk x k kxk. t
u
Note that a sequence which is weakly convergent needs not to converge strongly.
Example 3.2.2. (cf. [267, Example 2.13]). Let H WD l 2 and x k D ek D
.ek1 ; ek2 ; : : :/, where
1 for i D k
eki WD
0 for i ¤ k.
k
Then x D 1, although fx k g1 kD0 does not contain a convergent subsequence,
p
because x k x l D 2 for all k; l, k ¤ l, i.e., fx k g1
kD0 is not a Cauchy sequence.
k
Note, however, that x * 0.
Under some additional assumptions the weak convergence yields the strong one.

Lemma 3.2.3. If x k * x 2 H and x k ! kxk, then x k ! x.
Proof. Let x k * x. It follows from the properties of the inner product that

x x k 2 D kxk2 C x k 2 2hx; x k i ! 0
as k ! 1. t
u
A Hilbert space has an important property which is expressed in the following
lemma.
Lemma 3.2.4 ([279]). If x k * y 2 H, then, for any y 0 2 H, y 0 ¤ y, the following
inequality holds
lim inf x k y 0 > lim inf x k y . (3.1)
k k
Proof. Let x * y and y ¤ y. Denote ı WD ky y 0 k2 . Since a weakly convergent

k 0
sequence is bounded, both lower limits in (3.1) are finite. The properties of the inner
product yield
3.2 Properties of the Weak Convergence 107
k
x y 0 2 D x k y C y y 0 2
2 2
D x k y C y y 0 C 2hx k y; y y 0 i
2
D x k y C ı C 2hx k y; y y 0 i.
By assumption, hx k y; y y 0 i ! 0, consequently,
2 2 2
lim inf x k y 0 D lim inf x k y C ı > lim inf x k y ,
k k k

i.e., lim infk x k y 0 > lim infk x k y . t
u
Lemma 3.2.4 was proved by Zdzisław Opial in [279, Lemma 1]. The property
described in this lemma is known under the name Opial property. A Banach space
for which the property holds is called an Opial space. Therefore, Lemma 3.2.4 can
be formulated as follows: any Hilbert space is an Opial space.
The following lemma, also proved by Opial in [279, Lemma 2], is a consequence
of the Opial property.
Lemma 3.2.5. Let T W X ! Hk be nonexpansive
and y 2 X be a weak cluster point
of a sequence fx k g1 . If T x x k ! 0, then y 2 Fix T .
kD0
k
Proof. Let x *y for a subsequence fx nk g1 k 1 k
kD0 of fx gkD0 and T x x ! 0.
nk
Suppose that T y ¤ y. Then, by the triangle inequality and by Lemma 3.2.4, we have
lim inf kx nk yk lim inf kT x nk T yk

k!1 k!1
D lim inf kT x nk x nk C x nk T yk
k!1
lim inf.kx nk T yk kT x nk x nk k/
k!1
D lim inf kx nk T yk
k!1
> lim inf kx nk yk .

k!1
A contradiction proves that y 2 Fix T . t

u
Definition 3.2.6. We say that an operator S W X ! H is demi-closed at 0 if for any
sequence x k * y 2 X with S x k ! 0 we have Sy D 0.
If we replace the weak convergence x k * y by the strong one in Definition 3.2.6,
then we obtain the definition of the closedness of S at 0. If H is finite dimensional,
then a weakly convergent sequence is convergent. Therefore, the notions of a demi-
closed operator and a closed operator coincide in a finite dimensional Hilbert space.
The property expressed in Lemma 3.2.5 is known under the name demi-
closedness principle (see, e.g., [22, Fact 1.2]). Opial writes that the result is due
to Browder [44] (see [278, Proposition 2.5]). The property says, actually, that for a
nonexpansive operator T in a Hilbert space, the operator T Id is demi-closed at 0.
3.3 Properties of Fejér Monotone Sequences
The notions of Fejér monotone operators, quasi-nonexpansive operators and

strongly quasi-nonexpansive operators (see Definitions 2.1.15, 2.1.38) are closely
related to the Fejér monotonicity of sequences which are generated by sequences of
such operators.
Definition 3.3.1. Let C H be nonempty. We say that a sequence fx k g1
kD0
H is:
(a) Fejér monotone (FM) with respect to C if
kC1
x z x k z (3.2)
for all z 2 C and for all k 0,

(b) Strictly Fejér monotone with respect to C if
kC1
x z < x k z
for all z 2 C and for all k 0 with x k … C ,

(c) Strongly Fejér monotone (SFM) with respect to C if there exists a constant
˛ > 0 such that
kC1 2 2 2
x z x k z ˛ x kC1 x k ,
for all z 2 C and for all k 0.

Similarly as for the Fejér monotone operators, we can suppose without loss
of generality that C is a closed convex subset in Definition 3.3.1. Let a sequence
fx k g1
kD0 X be generated by the recurrence x
kC1
D Uk x k , where Uk W X ! X .
If the operators Uk , k 0, are (strictly) Fejér monotone with respect to C X
(see Definition 2.1.15), then fx k g1
kD0 is (strictly) Fejér monotone with respect to C .
Similarly, if the operators Uk , k 0, with a common fixed point, are ˛-strongly
k 1
T1 for some ˛ > 0, then fx gkD0 is strongly Fejér monotone
quasi-nonexpansive
with respect to kD0 Fix Uk . It is clear that a Fejér monotone sequence is bounded,
consequently it has a weak cluster point. Below, we give other important properties
of Fejér monotone sequences.
Theorem 3.3.2. Let fx k g1
kD0 be Fejér monotone with respect to a closed convex
subset C H. Then:
3.3 Properties of Fejér Monotone Sequences 109
(i) fd.x k ; C /g1

kD0 is a nonincreasing sequence, consequently, it converges.
(ii) PC x k converges in norm to a point z 2 C .
(iii) For any weak cluster point xN of fx k g1kD0 we have PC x N D z . In particular, if

xN 2 C , then xN D z .
(iv) It holds

lim x k z D inf lim x k z . (3.3)
k z2C k
Proof. (i) By the definition of the metric projection and the Fejér monotonicity of
fx k g1
kD0 , we have

d.x kC1 ; C / D x kC1 PC x kC1 x kC1 PC x k

x k PC x k D d.x k ; C /.
(ii) (cf. [22, Theorem 2.16 (iv)]) Let l k 0. By the parallelogram law with
x D PC x l x l and y D PC x k x l , the definition of the metric projection,
the convexity of C and the Fejér monotonicity of fx k g1
kD0 , we have

PC x l PC x k 2 D 2 PC x l x l 2 C 2 PC x k x l 2
2
1
4 .P
2 C x l
C PC x k
/ x l

2 2 2
2 PC x l x l C 2 PC x k x l 4 PC x l x l
2 2
D 2.PC x k x l PC x l x l /
2 2
2.PC x k x k PC x l x l /
D 2.d.x k ; C / d.x l ; C //.
By (i), the right hand side of the latter equality goes to zero as k; l ! 1.
Therefore, fPC x k g1
kD0 is a Cauchy sequence, consequently, it converges in
norm to a point z 2 C .
(iii) Because any subsequence of a Fejér monotone sequence is Fejér monotone,
we can suppose, without loss of generality, that the whole sequence converges
weakly to xN 2 H. Let x 2 C and " > 0. Denote zk D PC x k . Let k0 0 be
such that
1
maxfzk x k zk z ; kz xk zk z ; hx k x;
N z xig < "
3
for all k k0 . Then, by the characterization of the metric projection and the
Cauchy–Schwarz inequality, we have
0 hzk x k ; zk xi D hzk x k ; zk z i C hzk x k ; z xi

D hzk x k ; zk z i C hzk z ; z xi
Chz x;
N z xi C hxN x k ; z xi

zk x k zk z zk z kz xk
Chz x;
N z xi C hxN x k ; z xi
" C hz x;
N z xi
for all k k0 , i.e., hz x;

N z xi ". Since " > 0 is arbitrary, we obtain

hz x;N z xi 0, i.e., z D PC x.

N
(iv) The definition of the metric projection and the continuity of the norm yield

lim x k z lim x k PC x k D lim x k z
k k k
for any z 2 C . Consequently,

inf lim x k z lim x k z . (3.4)
z2C k k
But z 2 C , therefore, the equality in (3.4) holds. t

u
A sequence fx k g1
kD0 which is Fejér monotone (or even strictly Fejér monotone) with
respect to a subset C H needs not be weakly convergent (see Exercise 3.10.1).
Furthermore, weak cluster points of a sequence which is Fejér monotone with
respect to a subset C H need not to have equal distance to the set C (see
Exercise 3.10.2).
The following result which is due to Browder (see [47, Lemma 6]) follows imme-
diately from Theorem 3.3.2 (iii). Nevertheless, we present below an independent
simple proof of this result.
Corollary 3.3.3. If a sequence fx k g1kD0 H is Fejér monotone with respect to a
nonempty subset C H, then fx k g1 kD0 has at most one weak cluster point in C .
Consequently, x k converges weakly to a point z 2 C if and only if all weak cluster
points of fx k g1
kD0 belong to C .

Proof. (cf. [22, Theorem 2.16 (ii)]). By assumption, the sequence fx k zg1
kD0 is
monotone, consequently, it converges for any z 2 C . Denote
2
˛.z/ WD lim x k z kzk2 .
k
We have
2 2
lim x k 2hx k ; zi D lim x k z kzk2 D ˛.z/.
k k
3.4 Asymptotically Regular Operators 111
Let fx mk g1 nk 1 k 1
kD0 and fx gkD0 be two subsequences of fx gkD0 such that x
mk
* z0 2
00
C and x * z 2 C as k ! 1. Then
nk
h 2 2 i
2 limhx k ; z0 z00 i D lim x k 2hxk ; z00 i x k 2hxk ; z0 i
k k
D ˛.z00 / ˛.z0 /.
But
limhx mk ; z0 z00 i D hz0 ; z0 z00 i
k
and
limhx nk ; z0 z00 i D hz00 ; z0 z00 i.
k
Therefore, hz ; z z i D hz00 ; z0 z00 i, i.e., kz0 z00 k D 0 and z0 D z00 .

0 0 00
t
u
The following lemma gives a sufficient condition for the strong convergence of
Fejér monotone sequences and was proved in [22, Theorem 2.16 (v)] (see also [283,
Theorem 1.1] for a stronger result).
Lemma 3.3.4. Let fx k g1
kD0 H be Fejér monotone with respect to a nonempty
subset C H. If at least one cluster point x of fx k g1
kD0 belongs to C , then
xk ! x .
Proof. Since a Fejér monotone sequence is bounded, fx k g1 kD0 has a weak cluster
point x . Suppose that a subsequence fx nk g1 k 1
kD0 fx gkD0 converges to x 2 C .
k 1
We prove that the entire sequence fx gkD0 converges to x . Suppose that x 0 2 H,

x 0 ¤ x , is a cluster point of fx k g1 mk 1
kD0 and that a subsequence fx gkD0 converges to
x . Let " WD 2 kx x k > 0, k0 be such that kx x k < " and kx nk x k < ",
0 1 0 mk 0
for all k k0 and let mk > nk0 . By the triangle inequality and by the Fejér
monotonicity of fx k g1 kD0 with respect to C , we obtain

2" D x 0 x x 0 x mk C kx mk x k < 2",
a contradiction. Therefore, x k ! x . t
u
3.4 Asymptotically Regular Operators
Definition 3.4.1. An operator U W X ! X is called asymptotically regular (AR) if

lim U kC1 x U k x D 0
k!1
for all x 2 X .
Asymptotically regular operators were studied, e.g., by Browder and Petryshyn

[48], Opial [279], Baillon, Bruck and Reich [14], Bruck and Reich [51] and by
Bauschke [19].
Remark 3.4.2. It is clear that any projection is asymptotically regular. In particular,
the metric projection onto a nonempty closed convex subset C H is asymptoti-
cally regular.
Since the notion of asymptotically regular operators plays an important role
in iterative methods for finding fixed points of operators, below we give several
sufficient conditions for an operator to be asymptotically regular.
Theorem 3.4.3. Let U W X ! X be an operator with a fixed point. If U is strongly
quasi-nonexpansive, then U is asymptotically regular.
Proof. Let U be strongly quasi-nonexpansive, x 2 X and z 2 Fix U . For x k D U k x
and for some constant ˛ > 0, we have
kC1 2 2 2 2
x z D Ux k z x k z ˛ Ux k x k .

Consequently, the sequence fx k zg1
kD0 is monotone and, therefore, it converges.
By setting k ! 1 in the above inequality, we obtain in the limit
kC1 2 2
U x U k x D Ux k x k ! 0,
i.e., U is asymptotically regular. t

u
Corollary 3.4.4. Let U W X ! H be an operator with a fixed point. If U is a
cutter, then its projected relaxation PX U W X ! X is asymptotically regular for
any 2 .0; 2/.
Proof. Let U be a cutter and 2 .0; 2/. It follows from Theorem 2.1.39 that U is
2

-strongly quasi-nonexpansive. Let x 2 X and z 2 Fix U . Note that z D PX z.
For x k D .PX U /k x we have
kC1 2 2 2
x z D PX U x k z D PX U x k PX z
2 2 2
U x k z x k z U x k x k 2 .

Similarly as in the proof of Theorem 3.4.3, we obtain limk U x k x k D 0. Since
PX is nonexpansive and x k 2 X , we have

PX U x k x k D PX U x k PX x k U x k x k ,

i.e., limk PX U x k x k D 0 and PX U is asymptotically regular. t
u
3.4 Asymptotically Regular Operators 113
Corollary 3.4.5. Let T W X ! H be a firmly nonexpansive operator with

Fix.PX T / ¤ ;. Then, for any 2 .0; 2/, the projected relaxation R D PX T of
the operator T is asymptotically regular.
Proof. Let 2 .0; 2/ and x 2 X . By Theorem 2.2.46 (iii), the operator R is
strongly quasi-nonexpansive. The asymptotic regularity of R follows now from
Theorem 3.4.3. t
u
There is an essential difference between Corollaries 3.4.4 and 3.4.5. Since
Fix T X , it is clear that Fix T \ Fix PX D Fix T ¤ ; in Corollary 3.4.4.
Note that the assumption Fix.PX T / ¤ ; in Corollary 3.4.5 is weaker than the
assumption Fix T ¤ ; in Corollary 3.4.4. On the other hand, for an operator T with
a fixed point, the assumption that T is firmly nonexpansive in Corollary 3.4.5 is
stronger than the assumption that T is a cutter in Corollary 3.4.4.
Corollary 3.4.6. Let T W H ! H be a firmly nonexpansive operator with a fixed
point. Then, for any 2 .0; 2/, the relaxation T of the operator T is asymptotically
regular.
Proof. It suffices to take X D H in Corollary 3.4.5. t
u
Corollary
Pm 3.4.7. Let Ti W X ! H, i 2 I , be firmly nonexpansive and T WD
i D1 i i , where w D .!1 ; : : : ; !m / 2 m . If Fix.PX T / ¤ ;, then, for any
! T
2 .0; 2/, the operator R defined by
X
m
R WD PX T D PX .Id C !i .Ti Id//
i D1
is asymptotically regular.
Pm
Proof. The operator T WD i D1 !i Ti is firmly nonexpansive (see Corol-
lary 2.2.20). Therefore, the asymptotic regularity of R follows from Corol-
lary 3.4.5. t
u
Corollary 3.4.8. Let Ti W X ! X , i 2 I , be strictly relaxed firmly nonexpansive
and the composition T WD T1 T2 : : : Tm have a fixed point. Then T is asymptotically
regular.
Proof. By Theorem 2.2.42, the operator T is strictly relaxed firmly nonexpansive.
Therefore, the asymptotic regularity of T follows from Corollary 3.4.6. t
u
The following result is due to Bruck and Reich (see [51, Corollary 1.1]).
Theorem 3.4.9. Let T W X ! X be a strongly nonexpansive operator with a fixed
point. Then T is asymptotically regular.
Proof. The operator T is quasi-nonexpansive, because a strongly nonexpansive
operator is nonexpansive and a nonexpansive operator with a fixed point is quasi-
nonexpansive (see Lemma 2.1.20). Let x 0 D x 2 X , x k D T k x and z 2 Fix T be
such that

lim x k z D inf lim x k z .
k z2Fix T k

The
existence
and uniqueness of z follows from Theorem 3.3.2. Since the sequence
fx k z g1
kD0 is decreasing, it converges and the sequence fx k 1
g kD0 is bounded.
We have
k
x z T x k z D x k z x kC1 z ! 0.
The equality above, the strong nonexpansivity of T and the equality T z D z yield
now k
T x x k D .T x k T z / .x k z / ! 0.
Therefore, T is asymptotically regular. t

u
3.5 Opial’s Theorem and Its Consequences
Below, we give a theorem, which is useful in proving the convergence of sequences

generated by iterative methods for convex optimization problems. The first part of
the theorem below was proved by Opial in [279, Theorem 1].
Theorem 3.5.1 (Opial, 1967). Let X H be a nonempty closed convex subset
of a Hilbert space H and U W X ! X be a nonexpansive and asymptotically
regular operator with a fixed point. Then, for any x 2 X , the sequence fU k xg1
kD0
converges weakly to a point z 2 Fix U . Furthermore, z is the unique fixed point
of U satisfying equality (3.3).
Proof. (cf. [279, Theorem 1]) Let x 2 X , x k WD U k x and z 2 Fix U . By the
nonexpansivity of U , we have
kC1
x z D U kC1 x z D U kC1 x U z U k x z D x k z .
Therefore, the sequence fx k g1

kD0 is bounded, consequently, it contains a subse-
quence fx nk g1
kD0 which is weakly convergent to a point y 2 X . Since U is
asymptotically regular,

lim U kC1 x U k x D lim Ux k x k D 0.
k k
Now, it follows from the demi-closedness principle (Lemma 3.2.5) that y 2 Fix U .
Theorem 3.3.2 yields that y D y ; where y is the unique fixed point of U
satisfying equality (3.3). t
u
If X H is a subset of a finite dimensional Hilbert space, then theorem holds by
weaker assumptions. In this case it suffices to suppose the closedness of U Id at 0
and quasi nonexpansivity of U instead of its nonexpansivity. The following theorem
holds.
3.5 Opial’s Theorem and Its Consequences 115
Theorem 3.5.2. Let X H be a nonempty closed convex subset of a finite

dimensional Hilbert space H and U W X ! X be an operator with a fixed point
and such that U Id is closed at 0. If U is quasi-nonexpansive and asymptotically
regular, then, for an arbitrary x 2 X , the sequence fU k xg1
kD0 converges to a point
z 2 Fix U .
Proof. Let x 2 X , x k WD U k x and z 2 Fix U . By the quasi nonexpansivity of U ,
we have
kC1
x z D Ux k z x k z ,
consequently, fx k g1
kD0 is bounded. Let z

2 X be an arbitrary cluster point
of fx gkD0 and fx gkD0 be a subsequence which converges to z . Since U is
k 1 nk 1
asymptotically regular,

kUx nk x nk k D U nk C1 x U nk x ! 0.
By the closedness of U Id at 0, we have U z D z , i.e., z 2 Fix U . By

Lemma 3.3.3, the whole sequence fx k g1
kD0 converges to z . t
u
Corollary 3.5.3. Let 2 .0; 2/ and T W H ! H be a firmly nonexpansive operator
with a fixed point. Then, for any x 2 H, the sequence fTk xg1
kD0 converges weakly
to a point z 2 Fix T .
Proof. Let x 2 H and 2 .0; 2/. The operator T is nonexpansive (see Theo-
rem 2.2.10) and strongly quasi-nonexpansive (see Corollary 2.2.9). Consequently,
T is asymptotically regular (see Theorem 3.4.3). By Opial’s theorem, the sequence
fTk xg1
kD0 converges weakly to a point z 2 Fix T . t
u
By the equivalence of averaged operators and strictly relaxed firmly nonexpansive
operators (see Corollary 2.2.17), Corollary 3.5.3 is equivalent to the following result,
known in the literature as the Krasnosel’skiı̆–Mann theorem (see, e.g., [56, Theorem
2.1], [361] and [57, Theorem 5.16]). Actually, Krasnosel’skiı̆ proved the strong
convergence for compact operators (see [238, Theorem 1]).
Theorem 3.5.4. Let X H be a nonempty closed convex subset and U W X ! X
be an averaged operator with Fix U ¤ ;. Then, for an arbitrary x 2 X , the
sequence fU k xg1
kD0 converges weakly to a point z 2 Fix U .
The following result follows from Corollary 3.5.3.

Corollary 3.5.5. Let T W X ! H be firmly nonexpansive and such that
Fix.PX T / ¤ ; and R D PX T be a projected relaxation of T , where 2 .0; 2/.
Then, for an arbitrary x 2 X , the sequence fRk xg1
kD0 converges weakly to a point
z 2 Fix.PX T /.
Proof. The operator R is relaxed firmly nonexpansive (see Theorem 2.2.46 (i)).
Therefore, the corollary follows from Corollary 3.5.3. t
u
3.6 Generalization of Opial’s Theorem
Applying Opial’s theorem we can prove the convergence of sequences generated

by iterating a nonexpansive and asymptotically regular operator. Below we present
a generalization of Opial’s theorem which can be applied to sequences generated
by a sequence of operators. First, we propose a definition of an asymptotically
regular sequence of operators, which extends Definition 3.4.1. Let fx k g1
kD0 X
be a sequence generated by the recurrence
x kC1 D Uk x k , (3.5)
where x 0 2 X is arbitrary and Uk W X ! X , k 0.

Definition 3.6.1. Let X H be a nonempty closed convex subset. We say that a
sequence of operators Uk W X ! X is asymptotically regular, if for any x 2 X
lim kUk Uk1 : : : U0 x Uk1 : : : U0 xk D 0

k
or, equivalently,
lim Uk x k x k D 0,
k
where the sequence fx k g1

kD0 is generated by recurrence (3.5) with x 0 D x.
It is clear that an operator U W X ! X is asymptotically regular, if a constant
sequence of operators Uk D U is asymptotically regular. Therefore, Definition 3.6.1
extends Definition 3.4.1 to a sequence of operators.
A weaker version of the first part of the following theorem appeared in [66,
Theorem 1].
Theorem 3.6.2. Let X H be a nonempty closed convex subset, S W X ! H
be an operator with a fixed point and such that S Id is demi-closed at 0. Let
fUk g1
kD0 be an asymptotically
T regular sequence of quasi-nonexpansive operators
Uk W X ! X such that 1 k 1
kD0 FixUk Fix S . Let the sequence fx gkD0 be
generated by recurrence (3.5) with an arbitrary x 2 X .
0
(i) If the sequence of operators fUk g1

kD0 has the property

lim Uk x k x k D 0 H) lim S x k x k D 0, (3.6)
k k
then fx k g1
kD0 converges weakly to a point z 2 Fix S .
(ii) If H is finite dimensional and the sequence of operators fUk g1 kD0 has the
property

lim Uk x k x k D 0 H) lim inf S x k x k D 0, (3.7)
k k
then fx k g1
kD0 converges to a point z 2 FixS .
3.6 Generalization of Opial’s Theorem 117
Opial’s Theorem is, actually, a corollary of Theorem 3.6.2 (i). Indeed, suppose
that the assumptions of Theorem 3.5.1 are satisfied. Then U is quasi-nonexpansive
(see Lemma 2.1.20) and U Id is demi-closed at 0 (see Lemma 3.2.5). We see that
all assumptions of Theorem 3.6.2 are satisfied for a constant sequence of operators
Uk D U , k 0. Therefore, for an arbitrary x 2 C , the sequence fU k xg1 kD0
converges weakly to a point x 2 Fix U .
Proof of Theorem 3.6.2. (cf. [70, Theorem 9.9]) Let x 2 X , z 2 Fix S and the
sequence fx k g1
kD0 be generated by recurrence (3.5). Since Uk is quasi-nonexpansive
and Fix Uk Fix S , we have
kC1
x z D Uk x k z x k z ,
k 0, Therefore, x k is Fejér monotone with respect to Fix S , consequently, x k is

bounded.
(i) Suppose that condition (3.6) is satisfied. By the asymptotic regularity
of the sequence fU g 1
, we have Uk x k x k ! 0, consequently,
k k kD0
S x x k ! 0. Let x 2 X be a weak cluster point of fx k g1 and
kD0
fx nk g1 k 1
kD0 fx gkD0 be a subsequence which converges weakly to x . Then, of
course, kS x nk x nk k ! 0 and x 2 Fix S , by the demi-closedness of S Id
at 0. Since x is an arbitrary weak cluster point of fx k g1 k
kD0 and x is Fejér
monotone with respect to Fix S , the weak convergence of the whole sequence
fx k g1
kD0 to x follows from Lemma 3.3.3.
(ii) Let H be finite dimensional and suppose that condition (3.7) is satisfied. By the
asymptotic regularity of the sequence fUk g1 k
kD0 , we have Uk x x ! 0,
k
consequently, limk kS x nk x nk k D 0 for a subsequence fx nk g1

kD0 fx k 1
gkD0 .
Due to the boundedness of x nk , there is a subsequence fx mnk g1kD0 fx nk 1
gkD0
which converges to a point x 2 X . Since S Id is demi-closed at 0, we have
x 2 Fix S . The convergence of the whole sequence fx k g1
kD0 to x follows
now from Lemma 3.3.4. t
u
Remark 3.6.3. We do not suppose that the operators Uk are nonexpansive in
Theorem 3.6.2. It allows application of this theorem to a sequence of relaxed cutters
or, equivalently, to a sequence of strongly quasi-nonexpansive operators. We only
suppose that the sequence of operators fUk g1 kD0 is asymptotically regular and that
condition (3.6) (or (3.7) in the finite dimensional case) is satisfied for an operator
S W X ! H such that S Id is demi-closed at 0, in order to ensure the claims of
Theorem 3.6.2. Condition (3.6) is satisfied if, e.g., the inequality
kUk x xk ˛ kS x xk (3.8)
holds for all x 2 X , for all k and for some ˛ > 0. Condition (3.7) is satisfied if,
e.g., inequality (3.8) holds for all x 2 X , for infinitely many k and for some ˛ > 0.
Remark 3.6.4. It follows from the proof that Theorem 3.6.2 remains true if
we replace the assumption that fUk g1 kD0 is asymptotically regular and the
assumption
(3.6) in
case (i) or (3.7) in case (ii)
by a weaker
assumption
limk!1 S x k x k D 0 in case (i) or lim infk!1 S x k x k D 0 in case (ii),
respectively. The formulation presented in Theorem 3.6.2 is preferred, because in
applications, the operators Uk are often relaxed cutters with relaxation parameters
guaranteeing the asymptotic regularity of fUk g1kD0 . Furthermore, various practical
algorithms which apply relaxed cutters have properties which yield (3.6), (3.7) or
some related conditions.
3.7 Opial-Type Theorems for Cutters
In this section we present modifications and applications of Theorem 3.6.2 for

sequences of firmly nonexpansive operators, sequences of cutters and for gener-
alized relaxations of cutters. In all these three cases we adopt Opial’s theorem or its
generalization (Theorem 3.6.2). Consider the following recurrence
x kC1 D PX .x k C k .Tk x k x k //, (3.9)
where x 0 2 X , k 2 Œ0; 2 and Tk W X ! H. A stronger version of the following

result was presented in [70, Proposition 9.12].
Corollary 3.7.1. Let S W X ! H be an operator with a fixed point and such that
S Id is demi-closed at 0, x 0 2 X and the sequence fx k g1 kD0 X be generated by
recurrence (3.9),
T where lim inf
k k .2 k / > 0 and T k X ! H is a sequence of
W
cutters with 1 kD0 Fix T k Fix S . Then:
T1
(i) For all z 2 kD0 Fix Tk
kC1 2 2 2
x z x k z k .2 k / Tk x k x k . (3.10)
(ii) If the following implication holds

lim Tk x k x k D 0 H) lim S x k x k D 0, (3.11)
k k
then x k converges weakly to a fixed point of S .

(iii) If H is finite dimensional and the following implication holds

lim Tk x k x k D 0 H) lim inf S x k x k D 0, (3.12)
k k
then x k converges to a fixed point of S .

3.7 Opial-Type Theorems for Cutters 119
T1
Proof. Let C D kD0 Fix Tk and z 2 C . Denote Uk WD PX .Id C k .Tk
Id//. Inequality (3.10) and the strong
k quasi nonexpansivity of Uk follow from
Corollary 2.2.25. Consequently, x z converges as a bounded and monotone

sequence and lim .2 / Tk x k x k D 0. Since lim infk k .2 k / > 0, we
k k k
have limk Tk x k x k D 0. The rest part of the theorem can be proved by similar
arguments as in the proof of Theorem 3.6.2. t
u
A special case of Corollary 3.7.1 (ii) with X D H and Tk D S for all k 0
can be found in [253, Theorem 1]. Note that condition (3.11) holds in this case
automatically.
Bauschke and Combettes proved a theorem [24, Theorem 2.9 (i)] which is related
to Corollary 3.7.1 (ii). They supposed that X D H, k D 2 Œ1; 2/ and, instead
T1
of (3.11), supposed that all cluster points of fx k g1
kD0 belong to kD0 Fix Tk (see
also [124, Theorem 2.6 (iii)] and [352, Theorems 2.1–2.3] for related results).
Remark 3.7.2. If int Fix S ¤ ;, then the range of the relaxation parameters in
Corollary 3.7.1 (ii) and (iii) can be extended. By Proposition 2.1.41, for any
z 2 int Fix S there is " > 0 such that
kC1 2 2 2
x z x k z k .2 C " k / Tk x k x k . (3.13)
If lim infk k > 0 and lim sup k 2, then, similarly, as in in the proof of
Corollary 3.7.1, we obtain limk Tk x k x k D 0, because
lim inf k .2 C 2" k / lim inf k .2 k / C 2" lim inf k > 0.

k k k
Therefore, if int Fix T ¤ ;, then the results of Corollary 3.7.1 (ii) and (iii)
remain true if we suppose that lim infk k > 0 and lim sup k 2 instead of
lim infk k .2 k / > 0. This explains the convergence of sequences generated by
several projection methods which apply the extended range of relaxation parameters
(see, e.g., [54, 266]).
If we iterate inequality (3.10) k-times, we obtain for an arbitrary z 2 Fix S the
following inequality
kC1 2 2 Xk
2
x z x 0 z l .2 l / Tl x l x l . (3.14)
lD0
If we know an upper approximation R of d.x 0 ; Fix S /, then inequality (3.14) can be

applied to the following estimation of d.x kC1 ; Fix S /
X
k
2
d 2 .x kC1 ; Fix S / R2 l .2 l / Tl x l x l ,
lD0
where R d.x 0 ; Fix S /.

In what follows, we will apply the following property of real sequences

fk g1
kD0 Œ0; 2.
lim inf k .2 k / > 0 ” .lim inf k > 0 and lim sup k < 2/.
k k
Corollary 3.7.3. Let T W X ! H be a cutter and such that T Id is demi-closed

at 0 (e.g., a firmly nonexpansive operator with Fix T ¤ ;). Further, let fx k g1
kD0 be
generated by the recurrence
x kC1 D PX .x k C k .T x k x k //,
where x 0 2 X and k 2 Œ0; 2. If lim inf k .2 k / > 0, then x k converges weakly
to a fixed point of T .
Proof. Denote Tk D T , for all k 0, and S D T . Then implication (3.11) is
obvious. Therefore, the corollary follows from Corollary 3.7.1. u
t
A related result to Corollary 3.7.3 was obtained by Reich [294, Theorem 2]
for a uniformly convex Banach space with a Frechét differentiable norm, where
it
Pis supposed (in an equivalent form) that T is firmly nonexpansive and that
1
kD0 k .2 k / D C1 instead of lim inf k .2 k / > 0.
If we take k D 2 .0; 2/ and X D H in Corollary 3.7.3, then we obtain
Corollary 3.5.3.
Denote Uk WD PX .Id C k .Sk Id// for an operator Sk W X ! H and for
k 2 .0; 2, k 0.
Corollary 3.7.4. Let U W X ! H be an operator with a fixed point and such that
U Id is demi-closed at 0, x 0 2 X and a sequence fx k g1 kD0 be generated by the
recurrence
x kC1 D PX .x k C k .Sk x k x k //, (3.15)
where k 2 .0; 2, lim supk k < 2, and T fSk g1
kD0 is a sequence of firmly
nonexpansive operators Sk W X ! H such that 1
kD0 Fix .PX Sk / Fix U , k 0.
(i) If the following implication holds

lim Uk x k x k D 0 H) lim Ux k x k D 0, (3.16)
k k
then x k converges weakly to a fixed point of U .

(ii) If H is finite dimensional and the following implication holds

lim Uk x k x k D 0 H) lim inf Ux k x k D 0, (3.17)
k k
then x k converges to a fixed point of U .

3.7 Opial-Type Theorems for Cutters 121
Fig. 3.1 Operators Tk x +¹k (Sk x − x)

and Uk
Sk x
Uk x
Tk x
X
x
(iii) Let S W X ! H be firmly nonexpansive. If Sk D S and k 2 Œ"; 2 " for

all k 0 and some " > 0, then implication (3.16) holds for U WD PX S" .
Consequently, x k converges weakly to a point x 2 Fix U D Fix PX S .
Proof. (i) and (ii) The operator Uk W X ! X is 4
4
k
-relaxed firmly nonexpansive
and Fix Uk D Fix .PX Sk / ¤ ;, k 0 (see Theorem 2.2.46). Let Tk WD .Uk / 4k
4
(see Fig. 3.1). Then Tk is firmly nonexpansive (see Corollary 2.2.19), Tk is a
cutter (see Theorem 2.2.5 (i)), Fix Tk D Fix Uk ¤ ; (see Remark 2.1.4) and
Uk D .Tk / 4 (see Remark 2.1.3). It is clear that recurrence (3.15) is a special
4k
case of (3.9) with X D H and k D 4

4k
. Note that Tk;k W X ! X and that
4
Uk x k x k D Tk x k x k ,
4 k
i.e., condition (3.16) is equivalent to (3.11) with S WD U and condition (3.17) is

equivalent to (3.12) with S WD U .
Setting k D 4 4
k
we easily obtain lim infk k .2 k / > 0. Now the weak
convergence in (i) and the convergence in (ii) follows from Corollary 3.7.1 (i) and
(ii), respectively.
(iii) Suppose that Sk D S in (3.15) and k 2 Œ"; 2 " for all k 0 and for
some " 2 .0; 1/. Let U WD PX S" D PX .Id C ".S Id//. It is clear that Fix U D
Fix PX S (see Corollary 1.2.10). Furthermore, U is nonexpansive as a composition
of nonexpansive operators PX and S" . Consequently, U Id is demi-closed at 0 (see
Lemma 3.2.5). It follows from Corollary 2.2.26 that

Uk x k x k D PX .x k C k .S x k x k // x k

PX .x k C ".S x k x k // x k D Ux k x k ,
i.e., implication (3.16) is satisfied. Consequently, x k converges weakly to a fixed

point of PX S . t
u
Note the difference in assumptions of Corollaries 3.7.1 and 3.7.4. An iteration in
recurrence (3.9) is defined by a projected relaxation of a cutter Tk with Fix Tk
Fix S , while iteration (3.15) is defined by a projected relaxations of a firmly
nonexpansive operator Sk for which we do not suppose that Fix Sk ¤ ;. We only
suppose that Fix.PX Sk / Fix S ¤ ;. Furthermore, conditions (3.11) and (3.16)

have a different nature. In the first one, we apply the cutters Tk , while in the other
one—projected relaxations of firmly nonexpansive operators Sk .
Remark 3.7.5. Similarly as in Theorem 3.6.2, we can replace the assumption on the
demi-closedness of S Id at 0 and assumptions (3.11) and (3.16) in Corollaries 3.7.1
and 3.7.4 with a weaker one:

• If limk Tk x k x k D 0 or limk Uk x k x k D 0, respectively, then all weak
cluster points of x k lie in Fix S or in Fix U , respectively.
In these cases the claims remain true (cf. Remark 3.6.4). Conditions of a similar
form were proposed by Schott [303, Theorem 3] and by Bauschke and Borwein [22,
Definition 3.7] for slightly different recurrences.
Now we present an Opial-type theorem for generalized relaxations of algorithmic
operators (see Sect. 2.4 for definitions).
Theorem 3.7.6. Let U W X ! H be a strongly oriented operator with a fixed
point and such that U Id is demi-closed at 0, and the sequence fx k g1
kD0 X be
generated by the recurrence
x kC1 D PX U;k .x k /, (3.18)
where x 0 2 X , lim infk k .2 k / > 0 and the step size function W X ! .0; C1/
satisfies the following condition
hz x; Ux xi
˛ .x/ (3.19)
kUx xk2
for all x 2 X , for all z 2 Fix U and for some ˛ > 0. Then
kC1 2 2 2
x z x k z k .2 k / 2 .x k / Ux k x k (3.20)
and x k converges weakly to a fixed point of U .

Proof. Let z 2 Fix U , " > 0 and k0 0 be such that k 2 Œ"; 2 " for all k k0 .
It follows from the second inequality in (3.19) that U is a cutter and that U;k
is 2 k
k -strongly quasi-nonexpansive (see Corollary 2.4.5). This, together with the
nonexpansivity of the metric projection and with the first inequality in (3.19), gives
kC1 2 2
x z D PX U;k x k z
2
D PX U;k x k PX z
2
U; x k z
k
3.8 Strong Convergence of Fejér Monotone Sequences 123
2 2 k
x k z U; x k x k 2
k
k
k 2 2
D x z k .2 k / 2 .x k / Ux k x k
2 2
x k z "2 ˛ 2 Ux k x k
for all k k0 . Therefore, k

kx iskFejér
monotone with respect to Fix U , consequently
x is bounded and Ux x ! 0. Let x 2 X be a weak cluster point of
k
fx k g1 nk 1 k 1
kD0 and fx gkD0 fx gkD0 be a subsequence which converges weakly to x .

By the demi-closedness of U Id, we have x 2 Fix U . This, together with the
Fejér monotonicity of fx k g1
kD0 with respect to Fix U , gives the weak convergence
of x k to x (see Lemma 3.3.3). t
u
If we iterate inequality (3.20) k-times, we obtain
kC1 2 2 Xk
2
x z x 0 z l .2 l / 2 .x l / Ux l x l (3.21)
lD0
for any z 2 Fix U . If we know an upper approximation R of d.x 0 ; Fix U /, then

inequality (3.21) can be applied to the following estimation of d.x kC1 ; Fix U /
X
k
2
d 2 .x kC1 ; Fix U / R2 l .2 l / 2 .x l / Ux l x l , (3.22)
lD0
where R d.x 0 ; Fix U /.
3.8 Strong Convergence of Fejér Monotone Sequences
Theorem 3.6.2 as well as other results presented in the last two sections, yield
only the weak convergence of a sequence fx k g1 kD0 generated by a sequence
of asymptotically regular and quasi-nonexpansive operators to a fixed point of
a nonexpansive operator. If H is finite dimensional, the weak convergence is
equivalent to the strong convergence. Under some assumptions on the structure of
the convex minimization problem we can obtain, however, the strong convergence
in any Hilbert space.
The following theorem is due to Bauschke and Borwein (see [22, Theorem 2.16
(iii)]).
Theorem 3.8.1. Let fx k g1
kD0 H be Fejér monotone with respect to a subset
C H. If int C ¤ ;, then fx k g1
kD0 converges strongly.
Proof. Let z 2 int C and " > 0 be such that B.z; "/ C . Denote d k D
"
.x k x kC1 /. Then, obviously, we have zk D z C d k 2 C . By the Fejér
kx k x kC1 k
monotonicity of fx k g1 kD0 with respect to C ,
kC1 2 2
x z C d k 2hx kC1 z; d k i
2
D x kC1 z d k
2 2 2
D x kC1 zk x k zk D x k z d k
2 2
D x k z C d k 2hx k z; d k i,
i.e.,
kC1 2 2
x z x k z 2hx k x kC1 ; d k i
2
D x k z 2" x k x kC1
and
k
x x kC1 1 x k z2 x kC1 z2 .
2"
Consequently,
k m1 X
x x kCm x kCl x kClC1 1 x k z2 x kCm z2
2"
lD0
k(3.23)

for all m 1. By the Fejér monotonicity of fx k g1 with respect to C , x z
kD0
converges and inequalities (3.23) yield that fx k g1
kD0 is a Cauchy sequence, therefore
it converges strongly. t
u
Before we formulate our next result, we prove some auxiliary lemmas.
T
Lemma 3.8.2. Let C WD i 2J H .ai ; ˇi /, where ai 2 H and ˇi 2 R, i 2 J , and
fx k g1kD0 be Fejér monotone with respect to C . Then x 2 x C Linfai ; i 2 J g for
k 0
all k 0.
P
Proof. Denote V WD Linfai ; i 2 J g and fix k 0. Since x kC1 D x 0 C klD0 .x lC1
x l /, it suffices to prove that x lC1 x l 2 V for all l 2 f0; 1; : : : ; kg. We leave it to
the reader to check that the Fejér monotonicity of fx k g1 kD0 with respect to C yields
that
1 lC1
w x x l ; x lC1 x l 0 (3.24)
2
for any w 2 C (see equivalence (2.11)). Let l 2 f0; 1; : : : ; kg, z 2 C , and ul 2 V
and vl 2 V ? be such that x lC1 x l D ul C vl . Suppose that x lC1 x l … V . Then
vl ¤ 0. For any ˛ 2 R and i 2 J , we have
3.8 Strong Convergence of Fejér Monotone Sequences 125
hai ; z C ˛vl i D hai ; zi C ˛hai ; vl i D hai ; zi ˇi ,

i.e., z C ˛vl 2 C . Then, for
l 2
1 u hz; x lC1 x l i 1
˛l < 2
2 C
vl 2
we obtain
1
hz C ˛l vl .x lC1 x l /; x lC1 x l i
2
1
D hz; x lC1 x l i C h˛l vl .x lC1 x l /; x lC1 x l i
2
1
D hz; x lC1 x l i C h˛l vl .ul C vl /; ul C vl i
2
1 2 1 2
D hz; x lC1 x l i C .˛l / vl ul < 0,
2 2
a contradiction with (3.24). Therefore, x lC1 x l 2 V which completes the proof.

t
u
Lemma 3.8.3. Let X be a finite dimensional affine subspace of H and
fx k g1
kD0 X . If x converges weakly to a point z 2 X , then x converges strongly
k k
to z.
Proof. Let x k * z 2 X . Denote y k WD x k x 0 . It is clear that y k 2 V WD X x 0 ,
y k * z x 0 and V is a finite dimensional subspace of H. Therefore, y k ! z x 0
and limk x k D limk y k C x 0 D z. t
u
A special case of the following Theorem was proved by Gurin et al. (see [196,
Theorem 1 (d)]).
Theorem 3.8.4. Let fx k g1 kD0 H be Fejér monotone with respect to a subset C
being the intersection of finitely many half-spaces. If fx k g1 kD0 converges weakly,
then it converges strongly.
T
Proof. Let C WD i 2I H .ai ; ˇi /, where ai 2 H and ˇi 2 R, i 2 I WD
f1; 2; : : : ; mg. Denote X D x 0 C Linfai ; i 2 I g. Then X is a finite dimensional
affine subspace. By Lemma 3.8.2, fx k g1 k 1
kD0 X . Suppose that fx gkD0 converges
k 1
weakly. Then the strong convergence of fx gkD0 to a point of X follows from
Lemma 3.8.3. t
u
Corollary 3.8.5. Suppose that a sequence fx k g1
kD0 generated by recurrence (3.5)
satisfies the assumptions of Theorem 3.6.2. Furthermore, suppose that one of the
following conditions is satisfied:
(i) X is a finite dimensional
T affine subspace of H,
(ii) The subset F WD T1 kD0 Fix Uk has a nonempty interior,
(iii) The subset F WD 1 kD0 Fix Uk is an intersection of finitely many half-spaces.
Then for an arbitrary x 0 2 X the sequence fx k g1

kD0 generated by recurrence (3.5)
converges strongly to a fixed point of S .
Proof. The assumptions of Theorem 3.6.2 guarantee the weak convergence of
fx k g1
kD0 to a fixed point of S . The strong convergence in case (i) follows from
Lemma 3.8.3, in case (ii)—from Theorem 3.8.1 and in case (iii)—follows from
Theorem 3.8.4. t
u
Sufficient conditions for the strong convergence of Fejér monotone sequences, in
particular of sequences generated by several projection methods for the convex
feasibility problems, were presented in [196], [283, Theorem 1.1], [16, 308], [22,
Sect. 5], [255] and in [256]. The strong convergence also holds under additional
assumptions on the operators which generate the sequences fx k g1 kD0 . For details see
[36, Chaps. 3 and 4], [14], [114, Chap. 6], [272] and [295, Theorem 1.7]. The papers
[32, 213, 258] contain examples of projection methods for the convex feasibility
problems, which generate sequences converging weakly but not strongly. In the
paper [24], a method is described which transforms algorithms of type (3.9) gen-
erating weakly convergent sequences to sequences which are strongly convergent.
The method is based on the idea of Haugazeau [202].
3.9 Relationships Among Algorithmic Operators
In this section we recall relationships among nonexpansive operators, firmly

nonexpansive operators, relaxed firmly nonexpansive operators, cutters, quasi-
nonexpansive operators, strongly quasi-nonexpansive operators, strongly nonex-
pansive and asymptotically regular operators which were presented in Chap. 2.
These relationships are presented in the form of a diagram. In Fig. 3.2, T W
X ! H and U WD T D Id C .T Id/ is a -relaxation of T , where
2 .0; 2/. The figure shows an important role of firmly nonexpansive operators,
cutters and strongly nonexpansive operators. It follows from Corollary 2.2.9, from
Theorem 3.4.3 and from Opial’s Theorem that for 2 .0; 2/, sequences generated
by the relaxation T of a firmly nonexpansive operator T converge weakly to its
fixed point, if Fix T ¤ ;. We also see that the weak convergence is guaranteed if U
is nonexpansive, Fix T ¤ ; and at least one of the conditions below is satisfied for
U D T :
(i) T is a cutter (Theorems 2.1.39 and 3.4.3),
(ii) U is strongly quasi-nonexpansive (Theorem 3.4.3),
(iii) U is strongly nonexpansive (Theorem 3.4.9).
Note that a relaxed firmly nonexpansive operator is nonexpansive (see implica-
tion (i))(ii) in Theorem 2.2.10). A strongly nonexpansive operator is nonexpansive
(see Definition 2.3.1). In cases (i) and (ii) we should additionally suppose that
U WD T is nonexpansive if we want to apply Opial’s Theorem. Note, however,
3.10 Exercises 127
U NE U contraction
U /2-AV
T - FNE U SNE
U -RFNE
FixT FixU
FixU
T 2=T =(2- )/
T cutter =(2- )/ U - AR
U -SQNE
U NE
FixU
=2 Ukx
U QNE x* FixU
Fig. 3.2 Relationships among algorithmic operators
that the generalization of Opial’s Theorem (Theorem 3.6.2) applied to a sequence

of relaxations Uk of operators Tk , which satisfy one of the conditions (i)–(ii)
above, does not require the nonexpansivity of Uk . In this case it suffices to suppose
that implication (3.6) holds for a nonexpansive operator S W X ! H in order
to guarantee the weak convergence of sequences generated by the sequence of
operators Uk . In one of the next chapters we will present examples which show
how to construct sequences of such operators Uk for a given nonexpansive operator
S , which guarantee implication (3.6). The basic tools in the proofs of convergence
of sequences generated by such sequences of operators are the generalizations of
Opial’s Theorem presented in Sects. 3.6 and 3.7.
3.10 Exercises
Exercise 3.10.1. Let H WD R2 , C WD Œ0; 1 f0g, x k WD .1 C k1 ; .1/k /. Check

that fx k g1 k 1
kD0 is strictly Fejér monotone with respect to C , but fx gkD0 does not
converge.
Exercise 3.10.2. Let H WD l2 and x k WD ek (see Example 3.2.2). Show that the
sequence fy k g1
kD0 with y
2kC1
D x k and y 2k D x 1 , k 0, is Fejér monotone with
respect to f0g and has two weak cluster points: 0 and x 1 (the latter one is even a
strong cluster point of y k ).
•
Chapter 4
Algorithmic Projection Operators
As we have mentioned in Chap. 2, the algorithms (or methods) for solving convex
optimization problems are defined by algorithmic operators. Usually, particular
iterations of these algorithms have the form
x C D Ux,
where x is the current approximation of a solution and x C is a next approximation
(also called an actualization or an update). If an operator U describing this actualiza-
tion belongs to the class of strongly quasi-nonexpansive operators (or, equivalently,
to the class of relaxed cutters), then we call them algorithmic projection operators.
The name projection operator can be explained by the following property of a cutter
T W H ! H: For any x … Fix T , T x is the metric projection of x onto a hyperplane
H.x T x; hT x; x T xi/ D fy 2 H W hy T x; x T xi D 0g
which separates the point x from the subset Fix T .

In this chapter we give examples of algorithmic projection operators and we show
their properties. These properties are, in most cases, corollaries of general properties
of operators presented in Chap. 2. Since the metric projection plays an important
role in the construction of algorithmic projection operators, in Sect. 4.1 we give the
formulas for the metric projection onto simple closed convex subsets.
4.1 Examples of Metric Projections
4.1.1 Metric Projection onto a Hyperplane
A hyperplane in a Hilbert space H has the form
H.a; ˇ/ WD fz 2 H W ha; zi D ˇg,
130 4 Algorithmic Projection Operators

onto a hyperplane
a H(a,β)
PH(a,β ) x
where a 2 H, a ¤ 0 and ˇ 2 R. It is clear that H.a; ˇ/ is a closed convex subset.

We show that
ha; xi ˇ
PH.a;ˇ/ x D x a. (4.1)
kak2
Let y WD x kak2 .ha; xi ˇ/a. A straightforward computation shows that
ha; yi D ˇ, i.e., y 2 H.a; ˇ/. Furthermore, for any z 2 H.a; ˇ/, we have
ha; xi ˇ
hx y; z yi D .ha; zi ha; yi/ D 0.
kak2
Now, the characterization of the metric projection (see Theorem 1.2.4) yields y D
PH.a;ˇ/ x (Fig. 4.1). We can also write
.x/
PH.a;ˇ/ x D x a;
kak2
where .x/ D ha; xi ˇ is the residuum of the equality ha; xi D ˇ at the point
x 2 H.
Example 4.1.1. In a Euclidean space (Rn with the standard inner product) we have
H.a; ˇ/ D fx 2 Rn W a> x D ˇg,
where a 2 Rn and ˇ 2 R. In this case equality (4.1) can be written in the form
a> x ˇ
PH.a;ˇ/ x D x a
kak2
or
.x/
PH.a;ˇ/ x D x a,
kak2
where .x/ D a> x ˇ.
Example 4.1.2. Denote by PCG x the metric projection of x 2 Rn onto a closed
convex subset C of Rn equipped with the inner product h; iG induced by a positive
definite matrix G of type n n, defined by hx; yiG WD x > Gy. We call PCG x an
G
oblique projection. We calculate the oblique projection PH.a;ˇ/ x for x 2 Rn , where
4.1 Examples of Metric Projections 131
Fig. 4.2 Oblique projection J G, x, ρ)

onto a hyperplane
H a,β
x
G
PH a,β x
a 2 Rn , a ¤ 0 and ˇ 2 R. It follows from the definition of the metric projection

that P G .x/ is a solution of the following differentiable minimization problem
minimize f .z/ D 12 kz xk2G

subject to ha; zi D ˇ (4.2)
z 2 Rn
(see Fig. 4.2). By the properties of the metric projection, this problem has a unique
solution. Since the minimization problem (4.2) is convex, the solution z D PH.a;ˇ/
G
x
can be derived from the Karush–Kuhn–Tucker conditions (see Theorem 1.3.7). The
Lagrange function L W Rn R ! R has the form
1
L.z; / D kz xk2G C .ha; zi ˇ/
2
and the KKT-point .z; / is the unique solution of the KKT-system
G.z x/ C a D 0
ha; zi D ˇ.
Consequently,
hG 1 a; G.z x/ C ai D 0
and
ha; z xi ha; xi ˇ
D 1
D .
ha; G ai kak2G 1
If we put the computed Lagrange multiplier into the first equation of the KKT-
system, we obtain
ha; xi ˇ 1
P G .x/ D x G a. (4.3)
kak2G 1
Example 4.1.3. In H D L2 .Œ˛; ˇ/ with the inner product defined by
Z ˇ
hf; gi WD f .x/g.x/dx
˛

onto an affine subspace
H(a2; ¯2 )
PH x
H (a1; ¯1 )
H
we have Z ˇ
H.g; / D fh 2 L2 .Œ˛; ˇ/ W g.x/h.x/dx D g,
˛
where g 2 L2 .Œ˛; ˇ/, g ¤ 0, and 2 R. By applying equality (4.1), we obtain

Rˇ
g.x/f .x/dx
PH.g; / f D f ˛
Rˇ g.
2
˛ g .x/dx
4.1.2 Metric Projection onto a Finite Dimensional Affine

Subspace
Tm H be an intersection of an finite number of hyperplanes in R , i.e., H D

n
Let
i D1 H.ai ; ˇi /, where ai 2 R , ai ¤ 0 and ˇi 2 R, i D 1; 2; : : : ; m. Suppose
that H ¤ ;. Obviously, H is an affine subspace in Rn . It is clear that H is closed
and convex as an intersection of closed convex subsets. Let A D Œa1 ; : : : ; am > be
a matrix, with rows ai and b D .ˇ1 ; : : : ; ˇm /. Suppose that Rn is p
equipped with
a standard inner product hx; yi WD x > y and with the norm kk WD hx; xi. Then
H D fy 2 Rn W Ay D bg. Let x 2 Rn be arbitrary. Note that
H x D fy 2 Rn W A.y C x/ D bg.
We have PH x 0 D AC .b Ax/, where AC denotes the Moore–Penrose pseudoin-

verse of A (see Example 1.3.11). Lemma 1.2.6 yields
PH x D x AC .Ax b/. (4.4)
If A has full row rank, then AC D A> .AA> /1 and
PH x D x A> .AA> /1 .Ax b/ (4.5)
(Fig. 4.3). If m D 1, we have A D a> 2 Rn and b D ˇ 2 R. In this case we have

AA> D a> a D kak2 and (4.5) obtains the form (4.1).

onto a half-space
a
H−(a,β)
PH − (a,β) x
4.1.3 Metric Projection onto a Half-Space
A half-space in a Hilbert space H has the form
H .a; ˇ/ WD fz 2 H W ha; zi ˇg,
where a 2 H, a ¤ 0 and ˇ 2 R. It is clear that H .a; ˇ/ is closed and convex. We

show that (
x ha;xiˇ a if ha; xi > ˇ
PH .a;ˇ/ x D kak2 (4.6)
x if ha; xi ˇ.
Equality (4.6) is clear if x 2 H .a; ˇ/, i.e., ha; xi ˇ. Let now ha; xi > ˇ and
y WD x kak2 .ha; xi ˇ/a. We easily see that ha; yi D ˇ, i.e., y 2 H.a; ˇ/
H .a; ˇ/. Furthermore, for an arbitrary z 2 H .a; ˇ/, we have
ha; xi ˇ
hx y; z yi D .ha; zi ha; yi/ 0.
kak2
Now, it follows from the characterization of the metric projection (see Theorem
1.2.4) that y D PH .a;ˇ/ x. We can also write
.ha; xi ˇ/C
PH .a;ˇ/ x D x a (4.7)
kak2
(Fig. 4.4) or
C .x/
PH .a;ˇ/ x D x a,
kak2
where C .x/ D .ha; xi ˇ/C .
4.1.4 Metric Projection onto a Band
A band in a Hilbert space H has the form
C WD fx 2 H W ˇ1 ha; xi ˇ2 g,

onto a band
z = PC z
PC x PC y
C
y
where a 2 H, a ¤ 0, ˇ1 ; ˇ2 2 R and ˇ1 < ˇ2 . Obviously, C is closed and

convex as an intersection of closed convex subsets H .a; ˇ2 / and H .a; ˇ1 /.
By equality (4.6), we easily obtain
8 ha;xiˇ2
<x
ˆ kak2
a if ha; xi > ˇ2
PC x D x if ˇ1 ha; xi ˇ2
:̂ x ha;xiˇ1
a if ha; xi < ˇ1
kak2
(see Fig. 4.5).
4.1.5 Metric Projection onto the Orthant
Let RnC be the nonnegative orthant in the Euclidean space Rn . Obviously, RnC is
closed and convex. We show that PRnC .x/ D xC . Let y WD xC . It is clear that y 2
RnC . Let z 2 RnC be arbitrary. Since x D xC x , hx ; zi 0 and hxC ; x i D 0,
we have
hx y; z yi D hx xC ; z xC i
D hx ; z xC i
D hx ; zi C hx ; xC i 0.
The characterization of the metric projection (see Theorem 1.2.4) yields
PRnC x D xC
(see Fig. 4.6).

In a similar way we can show that
PRn x D x : (4.8)
We obtain similar results in H D L2 .Œ˛; ˇ/, where 1 ˛ < ˇ C1,

Rˇ
equipped with the inner product hf; gi WD ˛ f .x/g.x/dx, for the subsets LC
2 WD

onto RnC PR +n x R +n
x
w = P R +n w
PR+n y P R +n z
y z
ff 2 L2 .Œ˛; ˇ/ W f 0g and L2 WD ff 2 L2 W f 0g. Obviously, both subsets

are closed and convex. By applying the same arguments as above, we obtain
PLC f D fC
2
and
PL
2
f D f .
Example 4.1.4. Let Q WD fu 2 Rm W u bg, where b 2 Rm . We have Q D Rm Cb
or, equivalently, Rm
D Q b. Applying equalities (1.12) and (4.8), we obtain
PQ .y/ D PRm .y b/ C b
D .y b/ C b
D y .y b/C .
If we take y WD Ax and r.x/ WD Ax b, we obtain
.PQ Id/Ax D .Ax b/C D rC .x/. (4.9)
This equality will be used in the sequel.
4.1.6 Metric Projection onto Box Constraints
The box constraints in Rn have the form a z b, where a; b 2 Rn , a b. Let

C WD fz 2 Rn W a z bg. Obviously, the subset C is closed and convex as the
Cartesian product of closed intervals,
C D Œ˛1 ; ˇ1 : : : Œ˛n ; ˇn . (4.10)
In a one-dimensional case the metric projection of a point 2 R onto the interval

Œ˛; ˇ has the form
PŒ˛;ˇ D medianf; ˇ; ˛g WD maxfminf; ˇg; ˛g.


onto box constraints y
¯2
PC y
x PC x
z = PC z
C
®2
®1 ¯1
Fig. 4.8 Metric projection f

onto C D ff 2 L2 W g
f hg h
g
PC f
a b
It follows from the above equality and and from Lemma 1.2.8 that
PC x D maxfminfx; bg; ag
(see Fig. 4.7). This formula can also be used when ˛j D 1 or ˇj D C1 for
some j , j D 1; 2; : : : ; n.
We obtain a similar formula in H D L2 WD L2 .Œa; b/ for the metric projection
of f 2 L2 onto the subset C WD ff 2 L2 W g f hg, where g; h 2 L2 and
g h. Obviously, C is closed and convex. It follows from the characterization of
the metric projection (see Theorem 1.2.4) that
PC f D maxfminff; hg; gg,
i.e., 8
< g.x/ if f .x/ < g.x/
.PC f /.x/ D f .x/ if g.x/ f .x/ h.x/
:
h.x/ if f .x/ > h.x/
(see Fig. 4.8).
The formula above can be extended to the case C D ff 2 L2 W g f h
on S g, where S Œa; b is a measurable subset and g h on S . We leave it to the
reader to check that
8
< g.x/ if f .x/ < g.x/ and x 2 S
.PC f /.x/ D f .x/ if g.x/ f .x/ h.x/ or x … S
:
h.x/ if f .x/ > h.x/ and x 2 S .

onto a ball
PB(z,ρ) x
y = PB(z,ρ) y
z
B(z,ρ)
ρ
If we take g D h on S , then C D ff 2 L2 W f .x/ D g.x/ for x 2 S g and we

obtain, in particular
g.x/ if x 2 S
.PC f /.x/ D
f .x/ if x … S .
The subset C defined by (4.10) is a special case of a polytope. The metric
projection onto the latter subset can be evaluated by several methods, e.g., by a
finite procedure called an active set method (see [339] for details).
4.1.7 Metric Projection onto a Ball
Obviously, a ball B.z; / H, where z 2 H and > 0, is a nonempty closed

convex subset. It follows easily from the characterization of the metric projection
(see Theorem 1.2.4) and from the Cauchy–Schwarz inequality that
(
x if kx zk
PB.z;/ .x/ D
zC kxzk .x z/ if kx zk >
(see Fig. 4.9). The details are left to the reader.
4.1.8 Metric Projection onto an Ellipsoid
An ellipsoid in Rn has the form
C D J.D; z; / WD fy 2 Rn W .y z/> D.y z/ g,
where D is a positive definite matrix, z 2 Rn and > 0. An ellipsoid is a closed

convex subset as a sublevel set of a convex function
1
f .x/ D .x z/> D.x z/
2

onto an ellipsoid
PC x
y = PC y
z
J (D; z ; ½)
(note that the Hessian r 2 f D D is positive definite). We present a method for

calculating the metric projection of a point x 2 Rn onto the ellipsoid C (cf. [316,
Sect. 3.4] and [228]). A different method was also presented in [128].
It follows from the definition of the metric projection that y D PC x if and only
if y is a solution of the following convex minimization problem
minimize f .y/ D 12 ky xk2

subject to .y z/> D.y z/ (4.11)
y 2 Rn
(see Fig. 4.10). This problem has a unique solution, because the objective is
strongly convex. The same also follows from Theorem 1.2.3. All assumptions of
the Karush–Kuhn–Tucker theorem are satisfied. The Slater constraints qualification
for problem (4.11) is satisfied, because int J.D; z; / ¤ ;. Therefore, y 2 Rn is a
solution of this problem if and only if there exists 2 RC such that .y; / is a KKT-
point (see Theorems 1.3.6 and 1.3.7). Define the Lagrange function L W Rn R ! R
by the equality
1
L.y; / D ky xk2 C Œ.y z/> D.y z/ .
2
The Karush–Kuhn–Tucker system has the form
y x C 2D.y z/ D0
.y z/> D.y z/
(4.12)
0
Œ.y z/> D.y z/ D 0.
If .x z/> D.x z/ , then x 2 J.D; z; / and .y; / D .x; 0/ is the only

KKT-point, i.e., PC x D x. If .x z/> D.x z/ > , then x … J.D; z; /, i.e.,
y ¤ x. It follows from the first and from the third equality in (4.12) that > 0,
consequently,
.y z/> D.y z/ D 0; (4.13)
by the complementary condition (fourth equality in (4.12)). The first equality in the
KKT-system gives
y D .Id C 2D/1 .x C 2Dz/
(note that Id C 2D is positive definite, because D is positive definite). Setting it in

equality (4.13), we obtain
Œ.x C 2Dz/> .Id C 2D/1 z> DŒ.Id C 2D/1 .x C 2Dz/ z D . (4.14)
By the existence and uniqueness of the metric projection PC x, equation (4.14) has
a unique solution . Summarizing, we obtain

x if .x z/> D.x z/
PC x D
.Id C 2D/ .x C 2Dz/ if .x z/> D.x z/ > ;
1
where > 0 is the unique solution of equality (4.14).

If D D W D diag w for w 2 RnCC , z D 0 and D 1, we have C D J.W; 0; 1/ D
fy 2 Rn W y > Wy 1g and (4.14) obtains the form
x > .Id C 2W /1 W .Id C 2W /1 x D 1. (4.15)
Since
2 3
.1 C 2!1 /1 0 ::: 0
6 0 .1 C 2! 2/
1
::: 0 7
.Id C 2W /1 D6
4
7,
5
::: ::: ::: :::
1
0 0 : : : .1 C 2!n /
(4.15) can be written in the form
X
n
!j j2
D1 (4.16)
j D1
.1 C 2!j /2
(note that this equation is considered only in the case x … J.W; 0; 1/, i.e.,
P n 2
j D1 !j j > 1). As mentioned before, this equation has a unique solution > 0,
which can be computed, e.g., by the Newton method with the starting point 0 D 0.
P !j 2
Note that the function h W RC ! R, h./ D nj D1 .1C2!j /2 is convex (h00 ./ > 0
j
for all 0), h.0/ > 1 and lim!C1 h./ D 0. Therefore, the Newton method
generates an increasing sequence k which converges to a unique solution
of (4.16). The details can be found in [316, Theorem 3.4-2].
Fig. 4.11 Ice-cream cone

and its polar
4.1.9 Metric Projection onto an Ice Cream Cone
Let a 2 H a ¤ 0 and ˛ 2 Œ0; 2 . A subset
C.a; ˛/ WD fz 2 H W ^.z; a/ ˛g [ f0g
is called an ice cream cone with axis a and angle ˛. Without loss of generality we
suppose that kak D 1. Then the ice cream cone has the form
C.a; ˛/ D fz 2 H W ha; zi kzkg,
where D cos ˛ 2 Œ0; 1. This subset is closed and convex as a sublevel set S.f; 0/
of a continuous convex function f W H ! R, f .x/ WD kxk ha; xi. It is clear
that C.a; ˛/ is a cone. The polar cone to an ice cream cone is again an ice cream
cone and has the form

.C.a; ˛// D C.a; ˛/ (4.17)
2
(see, e.g., [186, Sect. 3.1] or [187]) (Fig. 4.11).

We will give a formula for PC.a;˛/ x, where x 2 H and ˛ 2 .0; 2 /. If x 2
C.a; ˛/, then, obviously, PC.a;˛/ x D x. Let x 2 C.a; 2 ˛/. Then it follows
from equality (4.17) and from the characterization of the metric projection (see
Theorem 1.2.4) that PC.a;˛/ x D 0. Now suppose that x 2 C.a; ˛/0 \ C.a; 2 ˛/0 .
Let u WD a with 2 R, be such that
ha; u xi
D sin ˛ (4.18)
ku xk

onto an ice cream cone C ( a; π ®)
2
π
a 2 ® u a
®
xy C (a; ®)
y
or, equivalently,
q
kxk2 ha; xi2
D cos ˛. (4.19)
ku xk
Obviously, x ¤ u, because ˛ > 0, and kxk2 ha; xi2 0, by the Cauchy–Schwarz

inequality. Let y WD x C .u x/ with 2 R, be such that hy; x ui D 0 (see
Fig. 4.12).
A simple calculation shows that
q
D ha; xi C kxk2 ha; xi2 tan ˛ (4.20)
and
hx; x ui kxk2 ha; xi
D 2
D . (4.21)
ku xk ka xk2
Consequently,
.kxk2 ha; xi2 /

hy; ai D hx C .u x/; ai D (4.22)
ka xk2
and
q
kxk2 ha; xi2
kyk D kx C .u x/k D . (4.23)
ka xk
By (4.19), (4.22) and (4.23), we obtain hy; ai D kyk cos ˛, i.e., y 2 C.a; ˛/.
Furthermore, (4.17) and (4.18) yield x y D .x u/ 2 C .a; ˛/, and we have
hx y; yi D hx u; yi D 0,
consequently,
hx y; z yi D hx y; zi 0
for all z 2 C.a; ˛/. Now, the characterization of the metric projection yields that
y D PC.a;˛/ x. Subsuming all cases, we obtain
8
<x if ha; xi kxk cos ˛
PC.a;˛/ x D 0 if ha; xi kxk sin ˛
:
x C .a x/ if kxk sin ˛ < ha; xi < kxk cos ˛,
where and are given by (4.20) and (4.21). Equivalent formulas can also be found
in [17, Theorem 3.3.6] or in [316, Sect. 3.5].
4.2 Cutters
Recall that an operator T W H ! H with a fixed point is a cutter if
hT x x; T x zi 0
for all x 2 H and for all z 2 Fix T (see Definition 2.1.30).
4.2.1 Characterization of Cutters
It follows from Lemma 2.1.36 that, for T being a cutter, Fix T is closed and
convex,
T consequently, Fix T is an intersection of half-spaces, i.e., Fix T D
i 2J H .ai ; ˇi /, where ai 2 H, ai ¤ 0, ˇi 2 R, i 2 J . This fact is applied
in the theorem below which gives a necessary and sufficient condition for T to be a
cutter. Denote V WD Linfai ; i 2 J g.
Theorem 4.2.1. Let T W H ! H. For any y 2 H the following statements are
equivalent:
(i) T is a cutter.
(ii) For all x 2 H and for all w 2 .V C y/ \ Fix T , it holds
hT x x; T x wi 0 (4.24)
and
Tx x 2 V. (4.25)
Proof. Let z 2 Fix T , v? 2 V ? and ˛ 2 R. We prove that z C ˛v? 2 Fix T . Note
that hai ; v? i D 0 and hai ; zi ˇi , i 2 J , Therefore, we have
hai ; z C ˛v? i D hai ; zi C ˛hai ; v? i ˇi

T
for all i 2 J , i.e., z C ˛v? 2 i 2J H .ai ; ˇi / D Fix T .
4.2 Cutters 143
Fig. 4.13 Cutter T with u

affine Fix T
v
Tx Fix T
x
(ii))(i) Let x; y 2 H and z 2 Fix T . Suppose that conditions (4.24) and (4.25)
are satisfied for all w 2 .V C y/ \ Fix T . Let y; N zN 2 V and y ? ; z? 2 V ? be such
? ?
that y D yN C y and z D zN C z . It follows from the first part of the proof that
zN 2 Fix T and that zN C y ? 2 Fix T . It is clear that V C y D V C y ? , consequently,
zN C y ? 2 V C y. By (4.24) and (4.25), we have
hT x x; T x zi D hT x x; T x .Nz C y ? /i hT x x; z? y ? i 0
and T is a cutter.
(i))(ii) Suppose that T is a cutter. Let x 2 H and w 2 .V C y/ \ Fix T .
Inequality (4.24) is obvious. Suppose that (4.25) does not hold. Then T x x D
vN C v? for some vN 2 V and v? 2 V ? , v? ¤ 0. It follows from the first part of the
2
proof that w C ˛v? 2 Fix T for any ˛ 2 R. For ˛ < hT x x; T x wi= v? we
obtain
0 hT x x; T x .w C ˛v? /i D hT x x; T x wi ˛hNv C v? ; v? i
2
D hT x x; T x wi ˛ v? > 0,
a contradiction which shows that T x x 2 V . t

u
Corollary 4.2.2. T Let T W H ! H be a cutter. If Fix T is a polyhedral subset,
i.e., Fix T D i 2I H .ai ; ˇi /, where ai 2 H, ai ¤ 0 and ˇi 2 R, i 2 I WD
f1; 2; : : : ; mg, then T x 2 x C Linfai W i 2 I g.
4.2.2 Cutters with Subsets of Fixed Points Being Affine

Subspaces
In this section we apply Theorem 4.2.1 to an operator T W H ! H with the subset

of fixed points being a nonempty and affine subspace (Fig. 4.13).
Corollary 4.2.3. Let T W H ! H be an operator with Fix T being a nonempty
closed and affine subspace. The operator T is a cutter if and only if
hT x x; T x ui 0 (4.26)
for all x 2 H and for some u 2 Fix T and
hT x x; v ui D 0 (4.27)
for all x 2 H and for all v; u 2 Fix T .

Proof. Since Fix T is a closed affine subspace, it has the form Fix T D W C u,
where W is a closed linear subspace and u 2 Fix T . Denote V WD W ? . It is clear
that .V C u/ \ .W C u/ D fug. The corollary follows now from Theorem 4.2.1. u t
Corollary 4.2.3 can also be proved directly.
2nd Proof of Corollary 4.2.3. The sufficiency of the conditions and the necessity
of (4.26) is obvious. Suppose that T is a cutter and that
hT x x; v ui ¤ 0 (4.28)
for some x 2 H and for some v; u 2 Fix T . By the symmetry of (4.28) with respect
to u and v, it suffices to consider only the case hT x x; v ui > 0. Let w D
u C t.v u/ for t 2 R. Of course, w 2 Fix T , because Fix T is an affine subspace.
For
hT x x; T x ui
t<
hT x x; v ui
it holds
hT x x; T x wi > 0.
We have obtained a contradiction with the assumption that T is a cutter, which yields
the necessity of (4.27). t
u
4.2.3 Subgradient Projection
Definition 4.2.4. Let f W H ! R be a continuous convex function. Let gf .x/ 2

@f .x/ be a subgradient of f at x, x 2 H (note that f is subdifferentiable, i e., for
any x 2 H, there exists gf .x//. Let ˛ 2 R. The operator Pf;˛ W H ! H defined by
( .f .x/˛/C
x gf .x/ if gf .x/ ¤ 0
kgf .x/k
2
Pf;˛ x WD (4.29)
x if gf .x/ D 0
is called a subgradient projection relative to f by a level ˛. The operator Pf WD Pf;0

is called a subgradient projection relative to f or, shortly, a subgradient projection.
If gf .x/ ¤ 0, then
Pf;˛ x D PS.fNx ;˛/ .x/, (4.30)
4.2 Cutters 145
Fig. 4.14 Subgradient f

projection
x
Pf x
Fig. 4.15 Pf;˛ x D PS.fNx ;˛/ x
S (f,α)
x
Pf ,αx
S(f x, α)
where fNx D hgf .x/; xi C f .x/ denotes a linearization of f at x. Indeed, for

gf .x/ ¤ 0 the sublevel set S.fNx ; ˛/ is a half-space
S.fNx ; ˛/ D fy 2 H W hgf .x/; y xi C f .x/ ˛g
and, by (4.7),
.f .x/ ˛/C
PS.fNx ;˛/ x D x gf .x/ D Pf;˛ x.
gf .x/2
Equality (4.30) is illustrated in Figs. 4.14 and 4.15.

If ˛ infx2H f .x/ it holds that Argminx2H f .x/ S.f; ˛/ and we can write
( f .x/˛
x gf .x/ if x … S.f; ˛/
kgf .x/k
2
Pf;˛ x D (4.31)
x if x 2 S.f; ˛/,
because for x … S.f; ˛/ we have .f .x/ ˛/C D f .x/ ˛ and gf .x/ ¤ 0 (see
Theorem 1.3.2), and for x 2 S.f; ˛/ we have .f .x/ ˛/C D 0. Obviously, Pf;˛ x
depends on the choice of gf .x/ 2 @f .x/, x 2 H. Nevertheless, the properties of
the subgradient projection presented below hold for any choice of gf .x/ 2 @f .x/,
x 2 H. Note that Pf;˛ is well defined even if S.f; ˛/ D ;, i.e., if ˛ < infx2H f .x/
or if ˛ D infx2H f .x/ and the function f does not attain its minimum.
Lemma 4.2.5. Let Pf;˛ W H ! H be a subgradient projection relative to a convex
subdifferentiable function f by a level ˛ 2 R. If S.f; ˛/ ¤ ;, then
Fix Pf;˛ D S.f; ˛/.
Proof. The inclusion S.f; ˛/ Fix Pf;˛ is obvious. Suppose that x … S.f; ˛/ ¤ ;.
Then f .x/ > ˛ infx2H f .x/, consequently, gf .x/ ¤ 0 (see Theorem 1.3.2) and
f .x/ ˛
gf .x/ ¤ 0,
gf .x/2
i.e., x … Fix Pf;˛ . Therefore, Fix Pf;˛ D S.f; ˛/. t

u
Let ˛ 2 R. It is clear that S.f; ˛/ D S.f ˛; 0/. Furthermore, f is a convex
and subdifferentiable function if and only if f ˛ is convex and subdifferentiable.
Moreover, for any x 2 H we have @.f ˛/.x/ D @f .x/. Therefore, we can restrict
our further analysis of Pf;˛ to the case ˛ D 0.
Corollary 4.2.6. Let f be convex and subdifferentiable and S.f; 0/ ¤ ;. A
subgradient projection Pf is a cutter.
Proof. By Lemma 4.2.5, we have Fix Pf D S.f; 0/. Let x … S.f; 0/ and z 2
S.f; 0/, i.e., f .z/ 0 < f .x/. By the definition of a subgradient of a convex
function, we have
2
hPf x x; Pf x zi D Pf x x C hPf x x; x zi
!2
f .x/ f .x/
D hgf .x/; x zi
gf .x/ gf .x/2
!2
f .x/ f .x/
.f .x/ f .z//
gf .x/ gf .x/2
0

u
Theorem 4.2.7. Let f W H ! R be a convex function which is globally Lipschitz
continuous on bounded subsets (this holds if, e.g., H D Rn ). Then the operator
Pf Id is demi-closed at 0.

Proof. Let x k * z 2 H and Pf x k x k ! 0. Then we have
.f .x k //C
D Pf x k x k ! 0. (4.32)
gf .x k /
Note that fx k g1
kD0 is bounded as a weakly convergent sequence. By the global
Lipschitz continuity of f on bounded subsets, there exists a constant > 0 such
that
f 0 .x k ; gf .x k // gf .x k /
4.3 Alternating Projection 147
(see Theorem 1.1.50). By Theorem 1.1.58,

gf .x k /2 sup hg; gf .x k /i D f 0 .x k ; gf .x k // gf .x k / ,
g2@f .x k /

i.e., gf .x k / and the sequence fgf .x k /g1
kD0 is bounded. Now (4.32) yields
.f .x //C ! 0. Recall that a continuous and convex function is weakly lower semi-
k
continuous (see Theorem 1.1.51). Therefore,
f .z/ lim inf f .x k / lim inf.f .x k //C D 0

k k
and Pf z D z. t
u
4.3 Alternating Projection
4.3.1 Basic Properties
Definition 4.3.1. Let A; B H be a nonempty closed convex subset. We call an

operator T WD PA PB W H ! A an alternating projection (Fig. 4.16).
The alternating projection was introduced by John von Neumann [271], who
studied the convergence of sequences generated by this operator for closed convex
subspaces A; B H. The alternating projection or its modifications were studied by
Aronszajn [11], Gurin et al. [196], Deutsch [138, 139, 141], Bauschke and Borwein
[20, 21], Combettes [117], Bauschke et al. [30], Hundal [213], Kopecká and Reich
[235, 236], Bauschke et al. [25], Scolnik et al. [306], Cegielski and Dylewski [74]
and by Cegielski and Suchocka [75, 76].
First we give properties of an alternating projection T D PA PB in the case A \
B ¤ ;. It follows from Theorem 2.1.26 (ii) that in this case we have Fix T D A\B,
because the metric projection is strictly quasi-nonexpansive (see Corollary 2.2.24).
Lemma 4.3.2. Let A; B H be nonempty closed convex subsets and A \ B ¤ ;.
Then for all x 2 A and z 2 A \ B the following inequalities hold
hz T x; x T xi hPB x T x; x PB xi kT x PB xk2 ,
where T WD PA PB is an alternating projection. Consequently, the operator T jA is a

cutter and its projected relaxation PA T is asymptotically regular for all 2 .0; 2/.
Proof. Let x 2 A and z 2 Fix T D A \ B. It follows from the characterization
of the metric projections PA .PB x/ and PB x and from the equivalence (a),(b) in
Lemma 1.2.5 that
hz T x; x T xi D hz T x; x PB xi C hz T x; PB x T xi
hz T x; x PB xi
Fig. 4.16 Alternating

projection PA PB x
x A
PB x B
D hz PB x; x PB xi C hPB x T x; x PB xi
hPB x T x; x PB xi
kT x PB xk2 0,
i.e., the operator T jA is a cutter. The asymptotic regularity of of the projected

relaxation PA T for 2 .0; 2/ follows now from Corollary 3.4.4. t
u
If A \ B ¤ ;, then Lemma 4.3.2 and Theorem 2.1.39 imply the strong quasi
nonexpansivity of the relaxation T of the alternating projection T WD PA PB for any
2 .0; 2/. Note that Lemma 4.3.2 is not true without the assumption A \ B ¤ ;.
In the case A \ B D ; the asymptotic regularity of the projected relaxation PA T
holds only for a narrower range of the relaxation parameter .
Corollary 4.3.3. Let A; B H be nonempty closed convex subsets. Then the
alternating projection T WD PA PB is 43 -relaxed firmly nonexpansive. Furthermore,
for any 2 Œ0; 34 , the relaxation T of T is firmly nonexpansive. Consequently,
for any 2 .0; 32 /, the operator T is relaxed firmly nonexpansive and T is
asymptotically regular whenever Fix T ¤ ;.
Proof. By Corollary 2.2.39, the operator T is 43 -relaxed firmly nonexpansive as
a composition of FNE operators PA and PB . Moreover, by Corollary 2.2.19, its
relaxation T is firmly nonexpansive for any 2 Œ0; 34 . Suppose that Fix T ¤ ;.
Then Corollary 3.4.6 yields the asymptotic regularity of the operator T for any
2 .0; 32 /. t
u
If we take D 1 in Corollary 4.3.3, then we obtain that the alternating projection
T WD PA PB is relaxed firmly nonexpansive and asymptotically regular, whenever
Fix T ¤ ;.
4.3.2 Fixed Points of the Alternating Projection
Recall that the distance function d.; C / W H ! R, where C H, is defined

by d.x; C / WD infz2C kx zk. Denote d.A; B/ WD infx2A;y2B kx yk, where A;
B H.
If A \ B ¤ ;, then Fix PA PB D A \ B (see Theorem 2.1.26 (ii)). In a general

case, the alternating projection does not need to have fixed points (e.g., for A WD
f.x; y/ 2 R2 W y D 0g and B WD f.x; y/ 2 R2 W y e x g we have Fix PA PB D ;).
The theorem below gives relationships between fixed points of operators PA PB and
PB PA and minimizers of the distance functions d.; B/ W A ! R and d.; A/ W
B ! R. The theorem presents a part of the results of [21, Lemma 2.2].
Theorem 4.3.4. Let A; B H be nonempty closed convex subsets, x 2 A and
y 2 B. The following conditions are equivalent:
(i) x 2 Fix PA PB and y D PB x ,
(ii) y 2 Fix PB PA and x D PA y ,
(iii) kx y k D d.x ; B/ d.x; B/ for all x 2 A,
(iv) kx y k D d.y ; A/ d.y; A/ for all y 2 B.
(v) kx y k D d.A; B/.
Proof. The inequality in condition (iii) denotes that the function f WD 12 d 2 .; B/ W
A ! R attains its minimum at x 2 A. The function f is convex (see
Corollary 2.2.29) and differentiable and Df D Id PB (see Lemma 2.2.27). Now,
it follows from the necessary and sufficient optimality condition for the convex
differentiable minimization (see Theorem 1.3.4) and from Lemma 1.2.9 that
x 2 Argmin f .x/
x2A
” PB x x D Df .x / 2 NA .x /
” x D PA PB x (4.33)

” x 2 Fix PA PB .
(i),(ii) Let PA PB x D x and y D PB x . Then PA y D PA PB x D x

and, consequently, PB PA y D PB x D y . The converse implication can be
shown in a similar way.
(i),(iii) The equivalence of the first part of (i) and of the inequality in (iii)
was already proved in (4.33). If y D PB x , then the equality in (iii) follows
from the definition of the metric projection PB . The converse implication follows
from the uniqueness of the metric projection. In a similar way, one can show the
equivalence (ii),(iv).
(i),(v) The function h W A B ! R defined by h.x; y/ WD 12 kx yk2 is
convex as a composition of a linear function .x; y/ 7! x y and of a convex
function u 7! 12 kuk2 . Moreover, h is differentiable and
Dh.x; y/ D .x y; y x/
for all x 2 A and y 2 B. We leave it to the reader to check that NAB .x; y/ D
NA .x/ NB .y/. Now, using similar arguments as in the first part of the proof, we
Fig. 4.17 Fix.PA PB /,

A
Fix.PB PA / and minimizers x∗
of d.; A/ and of d.; B/ Fix( PB PA )
Fix( PA PB )
y∗
B
obtain
.x ; y / 2 Argmin h.x; y/
x2A;y2B
” .y x ; x y / D Dh.x ; y / 2 NAB .x ; y /
” y x 2 NA .x / and x y 2 NB .y /
” x D PA y and y D PB x
” x D PA PB x and y D PB x ,
i.e., conditions (i) and (v) are equivalent. t

u
Theorem 4.3.4 is illustrated in Fig. 4.17.
Part (ii) of the following corollary was proved by Cheney and Goldstein (see [112,
Theorem 2]).
Corollary 4.3.5. Let A; B H be nonempty closed convex subsets. Then:
(i) Fix PA PB ¤ ; ” Fix PB PA ¤ ;,
(ii) Fix PA PB D Argminx2A d.x; B/,
(iii) Fix PB PA D Argminy2B d.y; B/,
(iv) Fix PA PB Fix PB PA D Argmin.x;y/2AB kx yk,
(v) If A \ B ¤ ;, then Fix PA PB D A \ B.
The corollary below gives a sufficient condition for the nonemptiness of the
subset Fix PA PB .
Corollary 4.3.6. Let A; B H be nonempty closed convex subsets. If at least one
of the subsets A and B is bounded, then Fix PA PB ¤ ;.
Proof. Suppose that A is bounded. It is clear that PA PB W A ! A is nonexpansive.
It follows from the Browder–Göhde–Kirk theorem that Fix PA PB ¤ ;. Suppose
now that B is bounded. Then, Fix PB PA ¤ ; and the corollary follows from
Corollary 4.3.5. t
u
If we suppose that H D Rn and A; B Rn are polytopes, then Corollary 4.3.6 holds
without any assumption on the boundedness of A or B (see [112, Theorem 5]).
4.3.3 Alternating Projection for a Closed Affine Subspace
The alternating projection PA PB does not need not to be firmly nonexpansive even
if A an B are closed subspaces of H (see Exercise 2.5.9). Nevertheless, if we
suppose that A is a closed affine subspace and if we restrict the operator PA PB
to the subset A, then PA PB is firmly nonexpansive. This fact is formulated in the
following theorem which is an extension of an important result of Combettes [117,
Proposition 3].
Theorem 4.3.7. Let A H be a closed affine subspace and B H be a closed
convex subset. Then, for all x; y 2 A,
hT x T y; x yi kT x T yk2 C k.T x PB x/ .T y PB y/k2 (4.34)
and
hT x T y; x yi kT x T yk2 C .kT x PB xk kT y PB yk/2 . (4.35)
Consequently, the alternating projection
T WD PA PB W A ! A,
is firmly nonexpansive.
Proof. Let a 2 A. Applying the property
hPA u PA v; wi D hu v; PAa wi
for u; v; w 2 H (see Theorem 2.2.33 (iii)), the fact x y 2 A a for x; y 2 A, the

firm nonexpansivity of PB and Corollary 2.2.24, we obtain
hT x T y; x yi
D hPA PB x PA PB y; x yi
D hPB x PB y; PAa .x y/i
D hPB x PB y; x yi
kPB x PB yk2
kPA PB x PA PB yk2 C k.PA PB x PB x/ .PA PB y PB y/k2
D kT x T yk2 C k.T x PB x/ .T y PB y/k2
kT x T yk2 C .kT x PB xk kT y PB yk/2
kT x T yk2
for all x; y 2 A. Consequently, T is firmly nonexpansive. t

u
Corollary 4.3.8. Let A H be a closed affine subspace and B H a closed

convex subset. Further, let T W A ! A be a relaxation of the alternating projection
T WD PA PB , where 2 .0; 2/. Then
2
kT x T yk2 kx yk2 k.T x x/ .T y y/k2 (4.36)

for arbitrary x; y 2 A. If, furthermore, Fix T ¤ ;, then, for any x 2 A and z 2

Fix T it holds
2
kT x zk2 kx zk2 kT x xk2 ,

2
i.e., the operator T is -strongly quasi-nonexpansive and asymptotically regular.
Proof. By Theorem 4.3.7, the operator T WD PA PB is firmly nonexpansive.
Therefore, the corollary follows from Corollary 2.2.15 and Theorem 3.4.3. t
u
4.3.4 Generalized Relaxation of the Alternating Projection
Let A; B H be nonempty closed and convex and T WD PA PB be the alternating

projection with a fixed point. Consider a generalized relaxation T
; W A ! H of T ,
given by the formula
T
; .x/ D x C
.x/.T x x/, (4.37)
where the relaxation parameter 2 Œ0; 2 and

W A ! .0; C1/ is a step size
function. It is clear that if
.x/ D 1 for all x 2 A, then T
; coincides with the
classical relaxation T of the alternating projection T WD PA PB . Corollaries 2.4.3
and 2.4.5 suggest that a step size function
with values larger than 1, for which the
operator T
is a cutter, leads to an acceleration (at least local) of the corresponding
iterative procedures of the form
x kC1 D PA T
;k .x k /.
Therefore, the ability to construct such step sizes is of big interest. In the remaining
part of this section we will present various step sizes for which T
is a cutter. Recall
that Fix T
; D Fix T for all > 0 (see Remark 2.4.2 (d)). Denote
ı WD d.A; B/ D inf kx yk
x2A;y2B
and
N
ı.x/ WD kT x PB xk ,
where x 2 A. If A \ B ¤ ;, then, of course, ı D 0. Suppose that we know an upper

Q
approximation ı.x/ N
2 Œı; ı.x/ of ı, for any x 2 A. Recall that we still suppose that
Fix T ¤ ;.
Lemma 4.3.9. Let x 2 A be such that T x … Fix T . Then the vectors x PB x and
T x PB x are linearly independent.
Proof. The assumption yields that x PB x and T x PB x are nonzero vectors.
Suppose that x PB x and T x PB x are linearly dependent, i.e.,
T x PB x D .x PB x/ (4.38)
for some . By the characterization of the metric projection PA .PB x/, we have
kT x PB xk2 D hx PB x; T x PB xi kT x PB xk2 > 0. (4.39)
Therefore, > 0. By Lemma 1.2.9, we have x PB x 2 NB .PB x/, consequently,

.x PB x/ 2 NB .PB x/ and, again by Lemma 1.2.9,
PB .PB x C .x PB x// D PB x: (4.40)
Now, by (4.38) and (4.40), we obtain
T 2 x D PA PB T x D PA PB .PB x C .x PB x//

D PA PB PB x D PA PB x D T x,
a contradiction with the assumption T x … Fix T . Therefore, the vectors x PB x

and T x PB x are linearly independent. t
u
Define the step size function
W A ! .0; C1/ by the formula
8
ˆ
< kT x PB xk ı.x/
2 Q kPB x xk C hPB x x; T x xi
if x … Fix T;
.x/ WD kT x xk2
:̂ 1 if x 2 Fix T;
(4.41)
Q
where ı.x/ is a fixed element of the segment Œı; kT x PB xk (see Fig. 4.18). This
step size was proposed in [76, equality (13)].
Lemma 4.3.10. Let x 2 A and the step size
.x/ be defined by (4.41). Then
1

.x/ (4.42)
2
and the inequality is strict if T x … Fix T .
Proof. If x 2 Fix T , then
.x/ D 1. Let now x … Fix T . Denote a WD PB x x,
b WD T x x, c WD PB x T x (see Fig. 4.18) and ıQ WD ı.x/.
Q
Fig. 4.18 Step size

.x/
given by (4.41) with x
Q
ı.x/ WD kT x PB xk
Tσ x A
Tx
δ( x )
B
PB x
Since b ¤ 0, b D a c and ıQ kck, the Cauchy–Schwarz inequality yields
kck2 ıQ kak C ha; bi

.x/ D
kbk2
kck2 kak kck C ha; bi

kbk2
kck2 kak kck C ha; a ci
D
ka ck2
kak2 C kck2 kak kck ha; ci
D
kak2 C kck2 2ha; ci
.kak kck/2 C kak kck ha; ci
D .
.kak kck/2 C 2.kak kck ha; ci/
˛Cˇ
Let ˇ 2 R. Note that the function f W .2ˇ; C1/ ! R, f .˛/ WD ˛C2ˇ is
increasing. Therefore, if we take ˛ WD .kak kck/ and ˇ WD kak kck ha; ci in
2
the above inequalities, we obtain ˛ 0 2ˇ and
1

.x/ f .˛/ f .0/ D ,
2
i.e.,
.x/ 12 . If T x … Fix T , then a and c are linearly independent by
Lemma 4.3.9. In this case, ˇ D kak kck ha; ci > 0 and
.x/ > 12 . t
u
Inequality (4.42) can be strengthened. One can prove that
1 1

.x/ ;
1 C cos ˛.x/ 2
where ˛.x/ denotes the angle between the nonzero vectors x PB x and T x PB x
(for details see [76, Lemmas 3 and 4]).
Fig. 4.19 Solution of

system (4.44)–(4.45) with x
Q
ı.x/ WD ı
A
x∗
Tx
y δ
Tσ x y∗
δ B
PB x
Let x 2 A be such that T x … Fix T . One can prove that the step size
.x/ given
by (4.41) is characterized by the equality
hx C
.x/.T x x/ y; T x xi D 0, (4.43)
where y 2 aff.x; PB x; T x/ is a unique solution of the system
Q
hy PB x; x PB xi D ı.x/ kPB x xk (4.44)
hy PB x; T x PB xi D kT x PB xk2 (4.45)
Q
and ı.x/ 2 Œı; kT x PB xk (see Fig. 4.19). By Lemma 4.3.9, such a solution is
defined uniquely. For details, see [76, Lemma 6].
Theorem 4.3.11. Let A; B H be nonempty closed convex subsets, T WD PA PB ,
Fix T ¤ ; and the step size function
W A ! .0; C1/ be defined by (4.41). Then,
for any x … Fix T and z 2 Fix T ,
hz x; T x xi

.x/ (4.46)
kT x xk2
holds and the operator T

W A ! H is a cutter. Consequently, for any 2 .0; 2/,
the operator T
; is 2

-strongly quasi-nonexpansive and asymptotically regular.
Proof. Let z 2 Fix T , x … Fix T and 2 .0; 2/. It is clear that
Fix T
; D Fix T
D Fix T .
Denote w WD PB z. Note that kz wk D ı (see the implication (i))(v) in

Theorem 4.3.4). By the characterization of the metric projection PB x and by the
Cauchy–Schwarz inequality, we have
hz PB x; PB x xi D hz w; PB x xi C hw PB x; PB x xi
hz w; PB x xi
kz wk kPB x xk D ı kPB x xk .
Therefore, if we apply again the characterization of the metric projection PA .PB x/,
we obtain
hz x; T x xi D hz PB x; T x xi C hPB x x; T x xi
D hz PB x; T x PB xi C hz PB x; PB x xi
ChPB x x; T x xi
kT x PB xk2 ı kPB x xk C hPB x x; T x xi.
Now, (4.46) follows from the inequality ı ı.x/Q and from the definition of the
step size given by (4.41). We see that the step size
.x/ satisfies the assumptions
of Corollary 2.4.5. Therefore, the operator T
is a cutter and the operator T
; is
2
-strongly quasi-nonexpansive. The asymptotic regularity of T
; follows from
Theorem 3.4.3. t
u
Q
If A \ B ¤ ;, then, of course, ı D 0. If we take ı.x/ WD ı D 0 in (4.41), then we
obtain
8 2
< kT x PB xk C hPB x x; T x xi
if x … A \ B

.x/ D kT x xk2 (4.47)
:
1 if x 2 A \ B.
We have the following result (cf. [76, Lemma 10]).

Lemma 4.3.12. Let A \ B ¤ ; and x 2 AnB. Then
kT x PB xk2 C hPB x x; T x xi kPB x xk2

, (4.48)
kT x xk 2 hPB x x; T x xi
Furthermore, the equality holds in (4.48) if A is a closed affine subspace.

Proof. Denote a WD PB x x, b WD T x x and c WD PB x T x. We have
a D b C c and b ¤ 0, because x … A \ B D Fix T . By the characterization of the
metric projection PA .PB x/, we have
ha; bi D hc; bi C kbk2 kbk2 > 0, (4.49)
i.e., both sides of inequality (4.48) are well defined. Inequality (4.48) can be written
in the form
kb ak2 C ha; bi kak2

kbk2 ha; bi
Fig. 4.20 Step sizes

.x/
x
given by (4.47) and by (4.52)
Tx
A
Tσ1 x
y
PB x Tσ 2 x
B
which is equivalent to
.kak2 ha; bi/.ha; bi kbk2 / 0. (4.50)
It follows from the characterization of the metric projection PA .PB x/ that
hb; a bi D hb; ci 0. (4.51)
Therefore, the inequalities in (4.49), the Cauchy–Schwarz inequality and the

nonexpansivity of the metric projection PA yield
0 < kbk2 ha; bi kak kbk kak2 ,
consequently, (4.50) is true. Suppose now that A is a closed affine subspace. Then
hc; bi D 0 (see Theorem 2.2.33 (i)), consequently, ha; bi D kbk2 and the equality
in (4.50) holds. t
u
Suppose that A \ B ¤ ;. Define the step size function
W A ! .0; C1/ by the
formula 8
< kPB x xk2
if x 2 AnB ;

.x/ D hPB x x; T x xi (4.52)
:
1 if x 2 A \ B,
where T WD PA PB is the alternating projection. The step size defined by (4.52)
was proposed by Gurin et al. in [196, Sect. 3] and was applied in an accelerated
alternating projection method of the form x kC1 D PA T
x k . Note that Lemma 4.3.12
says that in the case A \ B ¤ ;, the step size given by (4.47) is equal to at least the
step size given by (4.52). Therefore, the estimation (2.69) for T WD PA PB with the
step size
.x/ given by (4.47) is better than the one with the step size
.x/ given
by (4.52) (see Fig. 4.20). The following lemma shows that both step sizes lead to
generalized relaxations T
which are extrapolations of T .
Lemma 4.3.13. Let A \ B ¤ ; and the step size function
W A ! .0; C1/ be
defined by (4.52). Then
.x/ 1 for all x 2 A and the equality holds if and only if
PB x 2 A.
Proof. Let x 2 A. If PB x 2 A, then

.x/ D 1. Let now PB x … A. Then, of
course, x … B. Similarly as in the proof of Lemma 4.3.12, one can prove that
hPB x x; T x xi > 0, i.e., the step size
.x/ is well defined. Furthermore, the
characterization of the metric projection PA .PB x/ yields
hPB x x; T x xi D kPB x xk2 C hPB x x; T x PB xi

kPB x xk2 kT x PB xk2
< kPB x xk2 .
Therefore,
.x/ 1 and the equality holds if and only if PB x 2 A. t
u
Corollary 4.3.14. Let A; B H be nonempty closed convex subsets, A \ B ¤ ;
and the step size function
W A ! .0; C1/ be defined by (4.52). Then the operator
T
W A ! H is a cutter. Consequently, for any 2 .0; 2/, the operator T
; is
2

Proof. Let x 2 A and 2 .0; 2/. It follows from Lemma 4.3.12 that
.x/ is not
greater than the step size given by (4.47). Theorem 4.3.11 yields now that
.x/
satisfies conditions of Corollary 2.4.5. Therefore, T
is a cutter and T
; is 2

-
strongly quasi-nonexpansive. t
u
Remark 4.3.15. Let x 2 AnB. Note that the step size
.x/ defined by (4.52) is the
unique solution of the equality
hx C
.x/.T x x/ PB x; PB x xi D 0
(see Fig. 4.20).

Suppose that A is a closed affine subspace and that A \ B ¤ ;. Then
Theorem 2.2.33 (i) yields
hPB x x; T x xi D kT x xk2 C hPB x T x; T x xi

D kT x xk2 ,
where x 2 A and T D PA PB . The step size

.x/ defined by (4.52) can be now
written in the form
8
ˆ 2
< kPB x xk if x 2 AnB;

.x/ D kT x xk2 (4.53)
:̂ 1 if x 2 A \ B,
(see Fig. 4.21). This step size was proposed by Bauschke et al. in [25, Corollary
4.11] and was applied in an accelerated alternating projection method of the form
x kC1 D T
x k .
Now suppose that A and B are closed subspaces of H. In this case, the metric pro-
jections PA , PB are, actually, orthogonal projections (see Theorem 2.2.30 (i)) and
Fig. 4.21 Step size

.x/ x Tx Tσ x A
given by (4.53)
PB x B
Fig. 4.22 Step size

.x/
given by (4.54) x Tx Tσ x
A
PB x
B
kPB x xk2 D hPB x; PB x xi C hx; x PB xi

D hx; x PB xi
D hx; x T xi C hx; T x PB xi
D hx; x T xi,
where x 2 A and T WD PA PB . The step size

.x/ defined by (4.53) can now be
written in the form
8
< hx; x T xi if x 2 AnB

.x/ D kT x xk2 (4.54)
:
1 if x 2 A \ B,
(see Fig. 4.22). This step size was applied in [30, equality 3.1.2] in order to
accelerate globally the alternating projection method (see [30, Theorem 3.23]).
Now we consider the case in which A is a closed affine subspace and B is a closed
convex subset. We will propose a step size function
W A ! .0; C1/ which leads
to an extrapolation T
of T and we will show that the generalized relaxation T
;
which employs this step size function is strongly quasi-nonexpansive. Define the
step size function
W A ! .0; C1/ by the formula
8
ˆ Q
< 1 C .kT x PB xk ı.x//
2
if x … Fix T

.x/ WD kT x xk2 (4.55)
:̂ 1 if x 2 Fix T ,
Q
where ı.x/ 2 Œı; kT x PB xk is an upper approximation of ı WD d.A; B/ D
infx2A;y2B kx yk. A geometrical interpretation of the steps size
.x/ is presented
Q
in Fig. 4.23, where
.x/ ı.x/ < kT x PB xk, tan ˛ D kT xxk
Q . We have
kT xPB xkı.x/
kT xxk
.x/ D 1 C cot2 ˛ D 1
sin2 ˛
and kT
x xk D
.x/ kT x xk D sin2 ˛
.
Fig. 4.23 Step size

.x/ x Tx Tσ x x∗ A
given by (4.55) ®
PB x ∗
® δ
δ(x)
PB x
Theorem 4.3.16. Let A H be a closed affine subspace, B H be a nonempty

closed convex subset and the step size function
W A ! .0; C1/ be defined by
(4.55). Then the operator T
W A ! A is a cutter. Consequently, for any 2 .0; 2/,
the operator T
; is strongly quasi-nonexpansive and asymptotically regular.
Proof. Let z 2 Fix T
, x … Fix T
and 2 .0; 2/. Note that Fix T
; D Fix T
D
Fix T . It is clear that
.x/ 1. We prove that
hz x; T x xi
.x/ kT x xk2
The inequality is obvious for x 2 Fix T . Let now x … Fix T . Since ı D

kT z PB zk (see equivalence (i),(v) in Theorem 4.3.4), inequality (4.35) yields
hz x; T x xi D kT x xk2 C hz T x; T x xi
D kT x xk2 C hT z T x; z xi kT z T xk2
kT x xk2 C .kT x PB xk kT z PB zk/2
.kT x PB xk ı/2
D .1 C / kT x xk2
kT x xk2
Q
.kT x PB xk ı.x// 2
.1 C / kT x xk2
kT x xk2
D
.x/ kT x xk2 .
We see that the step size

.x/ satisfies the assumptions of Theorem 2.4.5.
Therefore, the operator T
is a cutter and the operator T
; is 2
-strongly quasi-
nonexpansive. The asymptotic regularity of T
; follows now from Theorem 3.4.3.
t
u
4.3.5 Averaged Alternating Reflection
Let A; B H be closed convex subsets. An operator T W H ! H defined by

1
T WD .RA RB C Id/, (4.56)
2
Fig. 4.24 Averaged

RB x
alternating reflection
B
x
Tx
RA R B x
A
where RA WD 2PA Id and RB WD 2PB Id are reflection operators onto A and B,

respectively, is called an averaged alternating reflection (AAR) (see Fig. 4.24). The
properties of AAR were studied in [149, 246], [157, Sect. 4], [26, Sect. 5.D], [27,
Sect. 3], [28, Sect. 3] and in [29]. AAR is a special case of Elser’s difference map
(for details, see [161]). It is clear that a -relaxation of T has the form

T D .1 /Id C RA RB
2 2
More general results than the following one can be found in [28, Sect. 3].
Corollary 4.3.17. Let A; B H be closed convex, U WD RA RB W H ! H and
T WD 12 .U C Id/ be the averaged alternating reflection. Then
(i) A \ B Fix T ,
(ii) If int.A \ B/ ¤ ;, then A \ B D Fix T ,
(iii) If A\B ¤ ; and x 2 Fix T , then PB x D PA RB x, consequently, PB .Fix T / D
A \ B,
(iv) T is firmly nonexpansive, consequently, T is nonexpansive and 2
-strongly
quasi-nonexpansive for any 2 .0; 2/.
Proof. (i) The inclusion is obvious (see Remark 2.1.1).
(ii) Let C WD int.A \ B/ ¤ ;. It is clear that
C D int A \ int B D int Fix RA \ int Fix RB Fix RA \ Fix RB .
By Proposition 2.1.41, RA and RB are int C -strictly quasi-nonexpansive. Now

Theorem 2.1.26 (ii) and the facts that Fix RA D A and Fix RB D B yield
Fix T D Fix U D Fix RA \ Fix RB D A \ B.
(iii) Let x 2 Fix T (D Fix U ), b WD PB x, y WD RB x D 2b x and a D PA y. It is
clear that y D b C 12 .y x/ and x D RA y D a C 12 .x y/. Consequently,
b D a, i.e., PB x D PA RB x. Therefore, PB x 2 A \ B, i.e., PB .Fix T / D
PB .Fix U / A \ B. The converse inclusion follows immediately from (i) and
from the fact that PB .A \ B/ D A \ B.
(iv) The metric projections PA and PB are firmly nonexpansive (see
Theorem 2.2.21 (iii)). By the implication (i))(ii) in Theorem 2.2.10,
the reflections RA and RB are nonexpansive. Therefore, U WD RA RB is
nonexpansive. Now the implication (iii))(i) in Theorem 2.2.10 yields the
firm nonexpansivity of T WD 12 .U C Id/. Let 2 .0; 2/. Applying again the

implication (i))(ii) in Theorem 2.2.10, we obtain that T is nonexpansive.
The 2
-strong quasi nonexpansivity of T follows from Corollary 2.2.9. t
u
4.4 Simultaneous Projection
Definition 4.4.1. Let Ci H be nonempty closed convex subsets, i 2 I WD

f1; 2; : : : ; mg, and w D .!1 ; : : : ; !m / 2 m be a vector of weights. The operator
X
T WD !i PCi
i 2I
is called a simultaneous projection. The operator

X
!i PCi ;i ,
i 2I
where PCi ;i WD Id C i .PCi Id/, i 2 Œ0; 2, i 2 I , is called a simultaneous

relaxed projection (see Fig. 4.25).
The simultaneous projection was introduced by Gianfranco Cimmino [116] and was
investigated by many authors, e.g., by Auslender [12], Censor and Elfving [87],
Reich [295], Pierra [284], De Pierro and Iusem [134–137,215], Butnariu and Censor
[53], Combettes [117,118,120], Bauschke [17], Bauschke and Borwein [22], Censor
and Zenios [108] and by Matoušková and Reich [258].
OnePcan also consider a generalization of the simultaneous projection of the form
T WD i 2I !i PCi , where w W H ! m is a weight function (see Sect. 4.8). In this
section, however, we restrict ourselves
T to the classical simultaneous projection with
constant weights. Denote C WD i 2I P i . The metric projection PCi is a special case
C
of a simultaneous projection T WD i 2I !i PCi , where w D ei , i 2 I .
Corollary 4.4.2. LetP Ci H be nonempty closed convex subsets, i 2 I , C ¤ ;,
w 2 ri m and T WD i 2I !i PCi . Then Fix T D C .
T
Proof. T By the nonexpansivity of the metric projection and by the fact i 2I Fix
PCi D i 2I Ci D C ¤ ;, the claim follows from Theorem 2.1.14. t
u
P P
Remark 4.4.3. Let WD i 2I !i i . If D 0, then it is clear that i 2I !i PCi ;i D
Id. Now let > 0 and i D !ii , where w D .!1 ; !2 ; : : : ; !m / 2 ri m and i 2
Œ0; 2, i 2 I . Then
X X
!i PCi ;i D !i .Id C i .PCi Id//
i 2I i 2I
X
D Id C !i i .PCi Id/
i 2I
4.4 Simultaneous Projection 163
Fig. 4.25 Simultaneous

projection and simultaneous
relaxed projection PC1 x
x
C1
Tx
Tλ x
PC 2 x
PC 3 x
C2 C3
X !i i
D Id C .PCi Id/
i 2I

!
X
D Id C i PCi Id
i 2I
X
D i .Id C .PCi Id//
i 2I
X
D i PCi ; .
i 2I
We see that in both cases ( D 0 and > 0), the simultaneous relaxed projection
P
i 2I !P
i PCi ;i can be presented as a relaxation U of a simultaneous
P projection
U WD i 2I i PCi or as a simultaneous relaxed projection i 2I i PCi ; with the
same relaxation parameter 2 Œ0; 2 for all projections PCi . Therefore,
P we restrict
our further analysis of simultaneous relaxed projection operators i 2I !i PCi ;i to
the case i D 2 Œ0; 2 for all P i 2 I , or, equivalently, to the relaxation T of a
simultaneous projection T WD i 2I !i PCi with a relaxation parameter 2 Œ0; 2.
4.4.1 Simultaneous Projection as an Alternating Projection

in a Product Space
Let Ci H be nonempty
P closed convex subsets, i 2 I , w D .!1 ; !2 ; : : : ; !m / 2
ri m and T WD i 2I !i PCi be a simultaneous projection. In this subsection we
present the operator T as an alternating projection in a product Hilbert space H D
Hm with the inner product h; iH W H ! R defined by
X
m
hu; viH WD !i hui ; vi i,
i D1
where u D .u1 ; u2 ; : : : ; um / 2 H, v D .v1 ; v2 ; : : : ; vm / 2 H, and with the norm kkH

induced by this inner product. Define the subsets C H and D H by
C WD C1 : : : Cm
and
D WD fx D .x1 ; x2 ; : : : ; xm / 2 H W x1 D x2 D : : : D xm g.
By Lemma 1.2.8, we have
PC x D .PC1 x1 ; PC2 x2 ; : : : ; PCm xm /,
where x D .x1 ; : : : ; xm / 2 H. One can easily show that
PC; x D .PC1 ; x1 ; PC2 ; x2 ; : : : ; PCm ; xm /,
where 2 Œ0; 2. Now, we calculate the metric projection PD y for y D

.y1 ; y2 ; : : : ; ym / 2 H. By the definition of the metric projection, we have
1
PD y D argmin kx yk2H
x2D 2
1X
m
D argmin !i kx yi k2 .
x2H 2 i D1
P
Since the function f W H ! R defined by f .x/ WD 12 m 2
i D1 !i kx yi k is
differentiable and convex, x is a minimizer of this function if and only if x is its
stationary point. After differentiation we obtain the following condition
X
m
!i .x yi / D 0.
i D1
Pm Pm
By the equality i D1 !i D 1, we obtain x D i D1 !i yi , i.e.,
X
m X
m
PD y D . !i yi ; : : : ; !i yi /.
i D1 i D1
Subsuming, for the operator T WD PD PC W D ! D, we obtain
X
m X
m
Tx D . !i PCi x; : : : ; !i PCi x/ D .T x; : : : ; T x/.
i D1 i D1
Therefore, x 2Fix PD PC if and only if x 2 Fix T . Furthermore, for the operator

T D Id C .PD PC Id/ it holds
T x D x C .PD PC x x/
X
m X
m
D. !i PCi ; x; : : : ; !i PCi ; x/
i D1 i D1
D .T x; T x; : : : ; T x/ D PD PC; x,
where x D .x; : : : ; x/ 2 D. We see that the simultaneous projection T WD

P
i 2I !i PCi W H ! H can be presented equivalently as an alternating projection
T D PD PC W D ! D in the product space H D Hm . Note that D H is a closed
affine subspace, consequently, T is firmly nonexpansive (see TheoremP 4.3.7). This
leads to the firm nonexpansivity of the simultaneous projection T WD i 2I !i PCi .
This property also follows from Corollary 2.2.20.
The idea of representing the simultaneous projection as an alternating projection
is due to Pierra [284, Sect. 1], where !i D m1 , i 2 I . The idea was continued by
Combettes in [117, Sect. III] and [120, Sect. IV].
4.4.2 Properties of the Simultaneous Projection
Let Ci H be nonempty closed convex subsets, i 2 I , and w D .!1 ; : : : ; !m / 2

m be a vector of weights.P In this section we give basic properties of the
simultaneous projection T WD i 2I !i PCi and of the proximity function f W H !
R defined by
1X
f .x/ WD !i kPCi x xk2 . (4.57)
2 i 2I
P
Corollary 4.4.4. The simultaneous projection T WD i 2I !i PCi , where w 2 m ,
is firmly nonexpansive. If Fix T ¤ ;, then T is a cutter and, for any 2 .0; 2/,
its relaxation T is 2

Furthermore,
2
X
2 2
kT x zk kx zk .2 / .!i PCi x x/ (4.58)

i 2I
for any z 2 Fix T .

Proof. Since T is a convex combination of firmly nonexpansive operators PCi ,
i 2 I , the firm nonexpansivity of T follows from Corollary 2.2.20. Let Fix T ¤ ;.
Then T is a cutter, by Theorem 2.2.5 (i). Corollary 2.2.9 yields the 2
-strong quasi
nonexpansivity of T , i.e.,
2
kT x zk2 kx zk2 kT x xk2 .

Now inequality (4.58) follows from the obvious equality T x x D .T x x/.
The asymptotic regularity of T follows from Theorem 3.4.3. t
u
T
If i 2I Ci ¤ ;, then inequality (4.58) can be strengthened. In the following
theorem we allow that the weights !i , i 2 I , can depend on x 2 H.
Theorem 4.4.5. Let Ci H be nonempty closed convex subsets, i 2 I , C WD
T
Pi 2I Ci ¤ ;, w W H ! m be an appropriate weight function and T WD
i 2I !i PCi . Then
X
kT x zk2 kx zk2 .2 / !i .x/ kPCi x xk2 , (4.59)
i 2I
for all x 2 H, z 2 C and 2 Œ0; 2.

T T
Proof. Let x 2 H, z 2 C and 2 Œ0; 2. We have i 2I Fix Ti D i 2I Ci . By
Corollary 2.2.24, PCi is strictly quasi-nonexpansive, i 2 I , consequently, Fix T D
T
i 2I Ci (see Theorem 2.1.26). Furthermore, PCi is a cutter (see Theorem 2.2.21
(ii)), i.e., hPCi x x; z xi kPCi x xk2 (see Remark 2.1.31). Now the convexity
of the function kk2 yields
2
X
2
kT x zk D x C !i .x/.PCi x x/ z

i 2I
2
X
2
D kx zk C 2
!i .x/.PCi x x/

i 2I
X
2 !i .x/hz x; PCi x xi
i 2I
X
kx zk2 C 2 !i .x/ kPCi x xk2
i 2I
X
2 !i .x/ kPCi x xk2
i 2I
X
D kx zk2 .2 / !i .x/ kPCi x xk2
i 2I

u
2
Let C ¤ ;. The strong convexity of the function kk yields that estimation (4.59)
with a constant weight function w is stronger than estimation (4.58).
P
Theorem 4.4.6. Let T WD i 2I !i PCi be a simultaneous projection, where w 2
m , and a proximity function f W H ! R be defined by (4.57) Then
Fix T D Argmin f .x/.

x2H
If at least one of the subsets Ci , i 2 I , is bounded and the corresponding weight

!i > 0, then Fix T ¤ ;.
Proof. The proximity function f is convex and differentiable, and
X
Df .x/ D !i .x PCi x/ D x T x
i 2I
(see Lemma 2.2.27) Therefore, the sufficient and necessary optimality condition
(see Corollary 1.3.3) yields that z 2 Argminx2H f .x/ if and only if Df .z/ D
z T z D 0, i.e., z 2 Fix T . Now suppose that a subset Ci is bounded for some
i 2 I and that !i > 0. Then the function f is coercive. By Corollary 1.1.53,
Argminx2H f .x/ ¤ ; and Fix T ¤ ;. t
u
The proximity function f W H ! R defined by (4.57) has the following nice
property (cf. [215, Lemma 2]).
P
Lemma 4.4.7. Let T WD i 2I !i PCi , where w 2 m . Then, for any x 2 H, it
holds
1
f .T x/ f .x/ kT x xk2 .
2
Proof. Let x 2 H. It follows from the definition of the metric projection that
kPCi T x T xk kPCi x T xk. Therefore, the properties of the inner product
yield
1X 1X
f .T x/ D !i kPCi T x T xk2 !i kPCi x T xk2
2 i 2I 2 i 2I
1X 1X X
D !i kPCi x xk2 C !i kT x xk2 !i hPCi x x; T x xi
2 i 2I 2 i 2I i 2I
1X 1X
D !i kPCi x xk2 C !i kT x xk2 kT x xk2
2 i 2I 2 i 2I
1
D f .x/ kT x xk2
2
u
Let x 2 H. By the equality Df .x/ D x T x, Lemma 4.4.7 can be written in the
form
f .x Df .x// f .x/ kDf .x/k2 .
The equality above is related to the efficiency of a minimization method defined by
the recurrence x kC1 D x k Df .x k / (see [178, Definition 4.5]). It is known that in
the case H D Rn the sequence fx k g1 kD0 generated by this recurrence converges to a
minimizer of f (see [178, Theorem 4.6]). P
It is clear that for aTsimultaneous projection T WD i 2I !i PCi the following
inclusion is true C WD i 2I Ci Fix T . If C ¤ ; and w 2 ri m , then C D Fix T
(see Theorem 2.1.14) and, for the proximity function f W H ! R defined by (4.57)
it holds that minx2H f .x/ D 0.
4.4.3 Simultaneous Projection for a System of Linear

Equations
Suppose that Ci H are hyperplanes, Ci WD fy 2 H W hai ; yi D ˇi g, where

ai 2 H, ai ¤ 0, ˇi 2 R, i 2 I . Then the convex feasibility problem reduces to
the solution of a system of linear equations hai ; yi D ˇi , i 2 I . We have PCi x D
x C ˇi hai2;xi ai (see (4.1)) and kPCi x xk D jhaika
;xiˇi j
ik
. Now the simultaneous
kai k
projection can be presented in the form
X ˇi hai ; xi
Tx D x C !i ai , (4.60)
i 2I kai k2
where w D .!1 ; : : : ; !m / 2 m is a vector of weights. The proximity function f

defined by (4.57) has the form
1 X .hai ; xi ˇi /2
f .x/ D !i . (4.61)
2 i 2I kai k2
If kai k D 1, i 2 I , then T has the form

X
Tx D x C !i .ˇi hai ; xi/ai . (4.62)
i 2I
If H D Rn , the operator T and the proximity function f can be written in the

following, more convenient, matrix forms
T x D x A> D.Ax b/ (4.63)
and
1
f .x/ D .Ax b/> D.Ax b/, (4.64)
2
where D WD diag. !1 2 ; : : : ; !m 2 / We can suppose without loss of generality that
ka1 k kam k
w 2 ri m . In this case, the proximity function f has the form
1
f .x/ D kAx bk2D , (4.65)
2
1
where kukD D .u> Du/ 2 denotes the norm of the vector u 2 Rm , induced by
the positive definite matrix D. The operator T is firmly nonexpansive and its
relaxation T is 2

-strongly quasi-nonexpansive and asymptotically regular (see
Corollary 4.4.4), and Fix T D Argminx2H f .x/ (see Theorem 4.4.6). Furthermore,
both subsets are nonempty.
Theorem 4.4.8. Let a function f W H ! R be defined by (4.61). Then
Argminx2H f .x/ ¤ ;:
Proof. The minimization of the proximity function f W H ! R given by (4.61) is
equivalent to the following minimization problem
minimize h.y/ WD 12 kyk2

p
subject to i D !i .haika
;xiˇi /
ik
, i D 1; 2; : : : ; m, (4.66)
.x; y/ 2 H Rm ,
where y D . 1 ; 2 ; : : : ; m / 2 Rm and the product Hilbert space H Rm is

equipped with a scalar product hh; ii defined by hh.x; y/; .v; z/ii D hx; vi C y > z,
where .x; y/; .v; z/ 2 H Rm . Note that h is continuous, convex and coercive.
Furthermore, the subset
p .hai ; xi ˇi /
X WD f.x; z/ 2 H Rm W i D !i ; i D 1; 2; : : : ; mg
kai k
is closed and convex as an intersection of hyperplanes. Consequently, the function h

has a minimizer .x ; y / on X (see Corollary 1.1.53), i.e., x 2 Argminx2H f .x/.
t
u
De Pierro and Iusem proved a property which yields Theorem 4.4.8 in the case
H D Rn (see [137, Proposition 10]).
4.4.4 Simultaneous Projection for the Linear Feasibility

Problem
Suppose that Ci H are closed half-spaces, i.e., Ci WD fy 2 H W hai ; yi ˇi g,

where ai 2 H, ai ¤ 0, ˇi 2 R, i 2 I . Then the convex feasibility problem reduces
to the solution of a system of linear inequalities hai ; yi ˇi , i 2 I , PCi x D
.ha ;xiˇi /C .hai ;xiˇi /C
x i ai (see (4.7)) and kPCi x xk D kai k
. Similarly as in the
kai k2
linear case (see Sect. 4.4.3), the simultaneous projection can be presented in the
form
X .hai ; xi ˇi /C
Tx D x !i ai , (4.67)
i 2I kai k2
where w D .!1 ; : : : ; !m / 2 m is a vector of weights. The proximity function f

defined by (4.57) has the form
1 X Œ.hai ; xi ˇi /C 2
f .x/ D !i (4.68)
2 i 2I kai k2
(cf. (1.34)). If kai k D 1, i 2 I , then we have

X
Tx D x !i .hai ; xi ˇi /C ai . (4.69)
i 2I
If H D Rn , the operator T and the corresponding proximity function f can be

written in the following more convenient matrix form
T x D x A> D.Ax b/C (4.70)
and
1
f .x/ D .Ax b/>
C D.Ax b/C , (4.71)
2
where D WD diag. !1 2 ; : : : ; !m 2 /. We can suppose without loss of generality that
ka1 k kam k
w 2 ri m . In this case, the proximity function f has the form
1
f .x/ D k.Ax b/C k2D . (4.72)
2
The operator T is firmly nonexpansive and its relaxation T is 2

-strongly
quasi-nonexpansive and asymptotically regular (see Corollary 4.4.4), and Fix T D
Argminx2H f .x/ (see Theorem 4.4.6). Furthermore, both subsets are nonempty.
Theorem 4.4.9. Let a function f W H ! R be defined by (4.68). Then
Argminx2Rn f .x/ ¤ ;.
Proof. The minimization of the proximity function f W H ! R given by (4.68) is
equivalent to the following minimization problem
minimize h.y/ WD 12 kyk2

p
subject to i !i .haika
;xiˇi /
ik
; i D 1; 2; : : : ; m;
i 0; i D 1; 2; : : : ; m;
.x; y/ 2 H Rm .
The remaining part of the proof is similar to the proof of Theorem 4.4.8 with X
replaced by
p .hai ; xi ˇi /
X WD f.x; z/ 2 H Rm W i !i ; i 0; i D 1; 2; : : : ; mg.
kai k
Note that X is closed and convex as an intersection of closed half-spaces. t

u
4.5 Cyclic Projection 171
De Pierro and Iusem proved a property which yields Theorem 4.4.9 in case H D Rn
(see [137, Proposition 13]).
4.5 Cyclic Projection
Let Ci H be nonempty closed convex subsets, i 2 I WD f1; 2; : : : ; mg.

Definition 4.5.1. The operator T W H ! H defined by
T WD PCm PCm1 : : : PC1 (4.73)
is called a cyclic projection (see Fig. 4.26).

The cyclic projection was introduced by Stefan Kaczmarz for hyperplanes in Rn
(see [223]). The properties of the cyclic projection were investigated by many
authors, e.g., by Halperin [198], Bregman [42], Gurin et al. [196], Gordon et al.
[189], Tanabe [321], Herman [204], McCormick [259], Censor [78], Censor et al.
[86], Babenko [13], De Pierro and Iusem [137], Dax [129–132], Dye et al. [151],
Bauschke and Borwein [22], Bauschke et al. [23], Popa [285], and by Deutsch
and Hundal [142, 143]. In this section we present the basic properties of the cyclic
projection defined by (4.73).
T T
Corollary 4.5.2. If i 2I Ci ¤ ;, then Fix T D i 2I Ci .
T
Proof. Let i 2I Ci ¤ ;. Since Fix PCi D Ci and the metric projection is strictly
quasi-nonexpansive, the claim follows from Theorem 2.1.26. t
u
T
Without the assumption i 2I Ci ¤ ; the existence of a fixed point of a cyclic
projection T is not guaranteed in general. One can prove, however, that Fix T ¤ ;
if all Ci Rn are half-spaces, i 2 I (see [137, Lemma 1 and Proposition 13]).
Corollary 4.5.3. The cyclic projection T WD PCm PCm1 : : : PC1 is mC1
2m
-relaxed
firmly nonexpansive. Consequently, if Fix T ¤ ;, then T is m -strongly quasi-
1
nonexpansive and asymptotically regular.

2m
Proof. The operator T is mC1 -relaxed firmly nonexpansive as a composition of m
firmly nonexpansive operators (see Corollary 2.2.43). Let now Fix T ¤ ;. Then
Corollary 2.2.9 yields that T is m1 -strongly quasi-nonexpansive. The asymptotic
regularity follows from Corollary 3.4.6. t
u
T
Denote Si WD PCi PCi 1 : : : PC1 , i 2 I , and S0 WD Id. If C WD i 2I Ci ¤ ;, then
Fix Si C , i 2 I , by Corollary 4.5.2. If we apply inequality (2.52) m-times, then
we obtain the following estimation
X
m
kT x zk2 kx zk2 kSi x Si 1 xk2 (4.74)
i D1
for all x 2 H and z 2 C .

Fig. 4.26 Cyclic projection

C1 PC 1 x
PC 2 PC 1 x
x C2
Tx
C3
Fig. 4.27 Fixed point of a

composition of projections
C1 PC1 x
PC 2 PC 1 x
C2
x = Tx
C3
Corollary 4.5.4. If at least one of the subsets Ci , i 2 I , is bounded, then the cyclic
projection T WD PCm PCm1 : : : PC1 W Cm ! Cm has a fixed point z 2 Cm .
Proof. Since the metric projection is nonexpansive and PCi .X / D Ci , i 2 I , the
corollary follows from Theorem 2.1.13. t
u
A fixed point of a cyclic projection is illustrated in Fig. 4.27.
Let Ci H be closed half-spaces, i.e., Ci WD H .ai ; ˇi / D fx 2 H W hai ; xi
ˇi g, where ai 2 H, ai ¤ 0 and ˇi 2 R, i 2 I . The following theorem was proved
in [137, Lemma 1 and Proposition 13].
Theorem 4.5.5. If Ci Rn are half-spaces, i 2 I , then the cyclic projection
T WD PCm : : : PCm has a fixed point.
4.5.1 Cyclic Relaxed Projection
Definition 4.5.6. Let i 2 Œ0; 2, i 2 I . The operator
T WD PCm ;m PCm1;m1 : : : PC1 ;1 ,
where PCi ;i WD Id C i .PCi Id/, i 2 I , is called an operator of cyclic relaxed

projection (see Fig. 4.28).
Fig. 4.28 Cyclic relaxed

projection PC1 ,λ1 x PC2 ,λ2 PC1 ,λ1 x
C1
C2
x
C3
Tx
Corollary 4.5.7. Let i 2 .0; 2/, i 2 I , and T WDT PCm ;m PCm1 ;m1 : : : PC1 ;1 be
the operator of cyclic relaxed projection. If C WD i 2I Ci ¤ ;, then Fix T D C .
Proof. It follows from Theorem 2.2.21 (i) and from Remark 2.1.4 that Fix PCi ;i D
Ci , i 2 I . The relaxed metric projection PCi ;i is strictly quasi-nonexpansive for
i 2 .0; 2/, i 2 I , (see Corollary 2.2.23 (iii)). Therefore, the claim follows from
Theorem 2.1.26. t
u
Corollary 4.5.8. Let i 2 Œ0; 2, i 2 I . The relaxed cyclic projection T WD
PCm ;m PCm1;m1 : : : PC1 ;1 is -relaxed firmly nonexpansive, where
2mmax
D
.m 1/max C 2
and max D maxi 2I i . If max 2 .0; 2/ and Fix T ¤ ;, then T is asymptotically

regular.
Proof. The operator T is -relaxed firmly nonexpansive as a composition of i -
relaxed firmly nonexpansive operators, i 2 I (see Theorem 2.2.42). Now, let max 2
.0; 2/ and Fix T ¤ ;. Then 2 .0; 2/ and the asymptotic regularity follows from
Corollary 3.4.6. t
u
One can prove that the cyclic projection is asymptotically regular without the
assumption Fix T ¤ ; (see [19, Theorem 3.1]).
4.5.2 Cyclic-Simultaneous Projection
Definition 4.5.9. Let Ci H be closed convex subsets, i 2 I , and w 2 m . The

operator S W X ! H defined by
X
m
S WD !i Si , (4.75)
i D1
where Si WD PCi : : : PC1 , is called the cyclic-simultaneous projection (see Fig. 4.29).
Fig. 4.29 Cyclic- x

simultaneous projection S1x
S2x
C1
Sx
C2
S3x
C3
Corollary 4.5.10. The cyclic-simultaneousPprojection S defined by (4.75) is ˇ-

m
relaxed firmly nonexpansive, where ˇ D i D1 !i i C1 2 .0; 2/. Furthermore, if
2i
Fix S ¤ ;, then S˛ is strongly quasi-nonexpansive and asymptotically regular for

all ˛ 2 .0; 2ˇ 1 /.
P
Proof. Let ˇ WD m i D1 !i i C1 . It is clear that ˇ 2 .0; 2/. Since the metric projection
2i
is firmly nonexpansive, Corollary 2.2.44 yields that S is ˇ-RFNE. Therefore, Sˇ1

is FNE and .Sˇ1 / D Sˇ1 is -RFNE for all 2 .0; 2/, or, in other words, S˛ is
˛ˇ-RFNE for all ˛ 2 .0; 2ˇ 1 /. Now, the second part of the corollary follows from
Corollaries 2.2.9 and 3.4.6. t
u
The cyclic-simultaneous projection is a special case of string-averaging projections
which were studied in [91, Sect. 1], [105–107].
4.5.3 Projections with Reflection onto an Obtuse Cone
Let Ci H be closed convex subsets, i 2 I , and K TH be a closed convex and

obtuse cone, i.e., K K. Suppose that C WD K \ i 2I Ci ¤ ;. Consider the
following CFP: find a point x 2 C . Denote
P WD PCm PCm1 : : : PC1 ,
P WD Id C .P Id/, where 2 .0; 2/, and RK WD .PK /2 D 2PK Id, i.e., P is
a relaxation of the cyclic projection P and RK is the reflection operator onto K.
Definition 4.5.11. The operator RK P is called a projection-reflection and the
operator RK P , where 2 .0; 2/ is called a relaxed projection-reflection (see
Fig. 4.30). The operator PRK is called a reflection-projection and the operator
P RK , where 2 .0; 2/ is called a reflection-relaxed projection.
The following result can be found in [31, Lemma 2.1 (v)].
Lemma 4.5.12. If K H is a closed convex and obtuse cone, then RK x 2 K for
any x 2 H.
Fig. 4.30 Projection- x

reflection K
RK PC 1 x
PC1 x
C
C1
Proof. Let x 2 H and denote xC WD PK x. By the Moreau decomposition, we

have x D xC C x , where x WD PK x. Since K is obtuse, x 2 K K,
consequently,
RK x D 2PK x x D 2xC .xC C x / D xC x 2 K C K D K

u
Bauschke and Kruk studied the properties of the reflection-projection PRK which
defines a reflection-projection method (see [31]). This operator is closely related
to the projection-reflection operator RK P . Similarly to the equivalence (i),(ii) in
Theorem 4.3.4, one can prove that
.x 2 Fix RK P and y D P x / ” .y 2 Fix P RK and x D RK y /.

(4.76)
Corollary 4.5.13. Let K H be a closed convex
T and obtuse cone, Ci H be
closed convex subsets, i 2 I , and C WD K \ i 2I Ci ¤ ;. If 2 .0; mC1
m
/,
then:
(i) RK P and P RK are nonexpansive,
(ii) Fix RK P D Fix P RK D C ,
.1/mC1
(iii) RK P jK is ˇ-strongly quasi-nonexpansive, where ˇ D m
.
Proof. Since the metric projection is firmly nonexpansive (see Theorem 2.2.21 (iii)),
the reflection RK is nonexpansive (see Theorem 2.2.10 (ii)) and the cyclic projection
P is ˛-relaxed firmly nonexpansive with ˛ D mC1 2m
(see Corollary 4.5.3). Therefore,
its relaxation P is firmly nonexpansive for D ˛ 1 D mC1
2m (see Corollary 2.2.19).
Let 2 .0; mC1 m /. Then P is -RFNE, where D D ˛ 2 .0; 2/
(see Remark 2.1.3). Consequently, P is nonexpansive (see Theorem 2.2.10 (ii))
and strongly quasi-nonexpansive (see Corollary 2.2.9). Therefore, both operators
RK P and P RK are nonexpansive as compositions of nonexpansive operators.
Furthermore, Fix PCi D Ci (see Theorem 2.2.21 (i)),
\ \
Fix P D Fix P D Fix PCi D Ci D C
i 2I i 2I
(see Theorem 2.1.26), Fix RK D K and
Fix RK P D Fix P RK D Fix RK \ Fix P D C
(see Theorem 2.1.28). Now we see that all assumptions of Theorem 2.1.51 are
satisfied for S D RK . Therefore, the operator RK P jK is ˇ-strongly quasi-
nonexpansive, where
2 2 ˛ .1 /m C 1
ˇD D D
˛ m
u
4.5.4 Cyclic Cutter
The following definition generalizes the notion of a cyclic projection.

Definition 4.5.14. Let Ui W H ! H be cutters, i 2 I . The operator
U WD Um Um1 : : : U1
is called a cyclic cutter.
Note that the cyclic cutter U WD Um Um1 : : : U1 doe not need to be a cutter even
if Ui , i 2 I , have a common fixed point (see Example 2.1.53).
Corollary 4.5.15. Let Ui W H ! H be cutters with a common T fixed point, i 2 I ,
and U WD Um Um1 : : : U1 be a cyclic cutter. Then Fix U D i 2I Fix Ui and U is
1
m -strongly quasi-nonexpansive. Consequently, U is asymptotically regular.
Proof. Note that a cutter is 1-strongly

T quasi-nonexpansive (see Theorem 2.1.39).
Therefore, the equality Fix U D i 2I Fix Ui follows from Theorem 2.1.26. The
operator U is m1 -strongly quasi-nonexpansive as a composition of 1-strongly quasi-
nonexpansive operators Ui , i 2 I (see Theorem 2.1.48 (ii)). Now, the asymptotic
regularity of U follows from Theorem 3.4.3. t
u
4.6 Landweber Operator
Let H1 and H2 be two Hilbert spaces, A W H1 ! H2 be a nonzero bounded linear

operator and Q H2 be a nonempty closed convex subset.
Definition 4.6.1. The operator T W H1 ! H1 defined by the equality
1
T x WD x C A .PQ .Ax/ Ax/ (4.77)
kAk2
is called the Landweber operator (see Fig. 4.31).
4.6 Landweber Operator 177
Fig. 4.31 The Landweber A

operator H1 H2
Ax
Tx
s= PQAx-Ax
x+A* s
Q
A*
The Landweber operator is closely related to a method for the problem: find x 2 H1
such that Ax 2 Q. The method was proposed by Landweber for approximating
least-squares solution of a first kind integral equation [240] (see also [37, Sect. 6.1]).
Note that kAk ¤ 0, because A is a nonzero operator, consequently, the operator
T is well defined. We can also take max .A A/ or max .AA / instead of kAk2 in
Definition 4.6.1 if A is a compact linear operator (see Theorem 1.1.27). Define a
proximity function f W H1 ! RC by
1
PQ .Ax/ Ax 2
f .x/ WD (4.78)
2
(cf. (1.41)). The function f is convex as a composition of a linear operator A and a

convex function 12 d 2 .; Q/. Note that f has the required property: f .x/ D 0 if and
only if x 2 C D A1 .Q/ (cf. (1.19)).
4.6.1 Main Properties
Lemma 4.6.2. Let T W H1 ! H1 be the Landweber operator defined by (4.77) and

the corresponding proximity function f W H1 ! RC be defined by (4.78). Then
Fix T D Argmin f .x/.

x2H1
Proof. The function f is differentiable and Df .x/ D A .Ax PQ .Ax//. It follows
from the necessary and sufficient optimality condition (see Corollary 1.3.3) that
z 2 Argminx2H1 f .x/ if and only if A .Az PQ .Az// D 0, i.e., T z D z. t
u
Theorem 4.6.3. The Landweber operator is firmly nonexpansive.
Proof. Since PQ is firmly nonexpansive (see Theorem 2.2.21 (iii)), the implication
(i))(iv) in Theorem 2.2.10 yields that the operator Id PQ is firmly nonexpansive,
i.e.,
2
h.u PQ u/ .v PQ v/; u vi .u PQ u/ .v PQ v/
for all u; v 2 H1 . Let G WD Id T , where T W H1 ! H1 is the Landweber

operator given by (4.77). If we take u WD Ax and v WD Ay for x; y 2 H1 in the
above inequality and apply the inequality kA zk kA k kzk and the equality
kA k D kAk, then we obtain
hG.x/ G.y/; x yi
1
D 2
hA .Id PQ /Ax A .Id PQ /Ay; x yi
kAk
1
D h.Id PQ /Ax .Id PQ /Ay; Ax Ayi
kAk2
1
.Id PQ /Ax .Id PQ /Ay 2
2
kAk
kA k2
.Id PQ /Ax .Id PQ /Ay 2
D 4
kAk
1
A .Id PQ /Ax A .Id PQ /Ay 2
4
kAk
2
1 1
D
A .Id PQ /Ax A .Id PQ /Ay

kAk 2
kAk 2
D kG.x/ G.y/k2 ,
i.e., G is firmly nonexpansive. By the implication (iv))(i) in Theorem 2.2.10, the

Landweber operator T D Id G is also firmly nonexpansive. t
u
Corollary 4.6.4. Let 2 .0; 2/ and T W H1 ! H1 be a relaxation of the
Landweber operator T with nonempty Fix T . Then T is 2

-strongly quasi-
Proof. By Theorem 4.6.3 the Landweber operator T is firmly nonexpansive.
Therefore, the 2

-strong quasi nonexpansivity of T follows from Corollary 2.2.9
and the asymptotic regularity of T follows from Corollary 3.4.6. t
u
4.6.2 Landweber Operator for Linear Systems
Let b 2 H2 . If we take Q WD fbg, then fx 2 H1 W Ax 2 Qg is the solution set of

the linear equation Ax D b. Of course, PQ .Ax/ D b, and the Landweber operator
has the form
1
Tx D x C A .b Ax/. (4.79)
kAk2
Define the proximity function f W H1 ! RC for the linear equation Ax D b by

f .x/ WD 12 kAx bk2 .
Suppose that H1 D Rn and H2 D Rm with the standard inner product and that
b 2 Rm . If we take Q WD fu 2 Rm W u bg, then fx 2 H1 W Ax 2 Qg is the
solution set of a system of linear inequalities Ax b, where A is a matrix of type
m n with rows ai , i 2 I , x 2 Rn and b 2 Rm (we suppose without loss of
generality that ai ¤ 0, i 2 I ). In this case we have PQ .Ax/ Ax D .Ax b/C
(see equality (4.9)), kAk2 D max .A> A/ (see Theorem 1.1.27) and
1
Tx D x A> .Ax b/C . (4.80)
max .A> A/
We can also write the system of equations Ax D b or inequalities Ax b in

1 1 1 1
an equivalent form D 2 Ax D D 2 b or D 2 Ax D 2 b, respectively, where D is a
positive definite matrix (with nonnegative elements in the second case), e.g.,
!1 !m
D WD diag. 2
;:::; /, (4.81)
ka1 k kam k2
where w D .!1 ; : : : ; !m / 2 ri m . If we apply the Landweber operator to these new

systems, then we obtain
1
Tx D x A> D.Ax b/ (4.82)
max .A> DA/
and
1
Tx D x A> D.Ax b/C , (4.83)
max .A> DA/
respectively. We call the operators T given by (4.82) and (4.83), the Landweber
operators related to the matrix D. The corresponding proximity functions obtain
the form
1 1
2
1
f .x/ D D 2 .Ax b/ D kAx bk2D
2 2
and
1
f .x/ D k.Ax b/C k2D ,
2
respectively. Note that in both cases the Landweber operators and the proximity
functions differ from the for the original systems.
Remark 4.6.5. The Landweber operators defined by (4.82) and (4.83) related to the
matrix D given by (4.81), have a nice property. One can easily show that
X
m
!i
>
A D.Ax b/ D ai .ai > x ˇi /
i D1 kai k2
and
X
m
!i
A> D.Ax b/C D ai .ai > x ˇi /C .
i D1 kai k2
Therefore, the terms A> D.Ax b/ and A> D.Ax b/C in operators T defined
by (4.82) and (4.83), respectively, do not change if we rescale equations or
inequalities of the system. Moreover, max .A> DA/ also does not depend on the
rescaling. Therefore, the Landweber operators defined by (4.82) and (4.83) do
not change after rescaling. Note that they depend only on the vector of weights
w 2 ri m . Furthermore, if A has normalized rows and w D . m1 ; m1 ; : : : ; m1 /, then
D D diag. m1 ; m1 ; : : : ; m1 / and the Landweber operators (4.82) and (4.83) reduce
to (4.79) and (4.80), respectively.
Now we state some relationships between the Landweber operators defined
by (4.82) or (4.83), where D is given by (4.81), and the simultaneous projection
applied to the system Ax D b or Ax b, respectively, where A is a matrix of type
m n with nonzero rows ai , i 2 I , x 2 Rn and b 2 Rm .
Lemma 4.6.6. Let A WD Œa1 ; : : : ; am > be an m n matrix with nonzero rows ai ,
i 2 I , and D WD diag. !1 2 ; : : : ; !m 2 /, where w D .!1 ; : : : ; !m / 2 m . Then
ka1 k kam k
0 < max .A> DA/ 1.
Proof. Note that max .˛1 A1 C ˛2 A2 / ˛1 max .A1 / C ˛2 max .A2 /, where A1 ; A2
are positive semi-definite matrices and ˛1 ; ˛2 0. Therefore,
X
m
!i
max .A> DA/ D max . ai ai> /
i D1 kai k2
X
m
!i
max .ai ai> /
i D1 kai k2
X
m
!i
D max .ai> ai / D 1.
i D1 kai k2
Furthermore, A> DA is a nonzero matrix. Therefore, max .A> DA/ > 0. t

u
We call the parameter
1

L WD
max .A> DA/
a Landweber extrapolation parameter or the Landweber step size.

Corollary 4.6.7. The Landweber operators T defined by (4.82) and (4.83), where
D is given by (4.81), are extrapolations of the simultaneous projection operators
U defined by Ux WD x A> D.Ax b/ and by Ux WD x A> D.Ax b/C ,
respectively, i.e., T x D x C
L .Ux x/, where the Landweber step size
L 1.
Remark 4.6.8. Suppose that A is a matrix with normalized rows. If we take w D
. m1 ; m1 ; : : : ; m1 / in Lemma 4.6.6, then we obtain
max .A> A/ D mmax .A> DA/ m.
One can show even more: max .A> A/ s, where s denotes the maximal number
of nonzero elements in any column of A (see [55, Proposition 4.1] or [58, Corollary
2.8]), which is of course a stronger result than Lemma 4.6.6.
4.6.3 Extrapolated Landweber Operator for a System

of Linear Equations
Consider a system of linear equations Ax D b, where A is an m n matrix with

nonzero rows ai , i 2 I , x 2 Rn and b 2 Rm . Let f W Rn ! RC be a proximity
function defined by
1X a> x ˇi 2
f .x/ WD !i . i /, (4.84)
2 i 2I kai k
where w D .!1 ; : : : ; !m / 2 m . In this section we present an extrapolation of the
Landweber operator and we show that this extrapolation is a cutter. The proximity
function can be written in the form
1
f .x/ D .Ax b/> D.Ax b/,
2
where D WD diag. !1
; : : : ; !m 2 / (see (4.64)), or, equivalently, in the form
ka1 k2 kam k
1 >
f .x/ D x Gx C g > x C c; (4.85)
2
where G D A> DA, g D A> Db and c D 12 b > Db.

Let T be a simultaneous projection, i.e., T x D x A> D.Ax b/. Let x 2 Rn
and
s.x/ WD T x x D A> D.b Ax/. (4.86)
It follows from differential rules that
s.x/ D rf .x/ D Gx C g. (4.87)

Denote M WD Argminx2Rn f .x/. It is clear that Fix T D M ¤ ; (see

Theorem 4.4.8). Let T
; WD Id C
.T Id/ be a generalized relaxation of T
with the step size function
D
EL W Rn ! .0; C1/ defined by
ks.x/k2

EL .x/ WD (4.88)
s.x/> A> DAs.x/
for x … Fix T . We will prove that

EL .x/
L for all x 2 Rn (see Lemma 4.6.9
below). Therefore, we call the step size
EL defined by (4.88) an extrapolated
Landweber step size and operator T
EL with this step size—an extrapo-
lated Landweber operator. We have s.x/> A> DAs.x/ > 0 for x … Fix T . Indeed, if
1
s.x/> A> DAs.x/ D 0, then, of course, D 2 As.x/ D 0 and, by (4.86), we have
ks.x/k2 D s.x/> s.x/ D .b Ax/> DAs.x/

1 1
D .b Ax/> D 2 D 2 As.x/ D 0,
i.e., x 2 Fix T , a contradiction. Therefore, s.x/> A> DAs.x/ > 0 and the step size
EL .x/ is well defined. Equality (4.87) yields
s.x/> A> DAs.x/ D s.x/> Gs.x/ D .Gx C g/> .G 2 x C Gg/
which leads to the following equivalent form of the step size

EL .x/:
kGx C gk2

EL .x/ D . (4.89)
.Gx C g/> .G 2 x C Gg/
Lemma 4.6.9. Let x … Fix T and z 2 Fix T . Then the step size
EL .x/ defined by
(4.88) satisfies the following inequalities
1 hz x; T x xi
1 >

EL .x/ (4.90)
max .A DA/ kT x xk2
for any z 2 Fix T . Consequently, the operator T

EL is a cutter and, for all 2 .0; 2/,
the operator T
EL ; is 2
Proof. Let
EL .x/ be defined by (4.88). The first inequality in (4.90) follows from
Lemma 4.6.6. We have
ks.x/k2 ks.x/k2 1

EL .x/ D D ,
s.x/> A> DAs.x/ max .A> DA/ ks.x/k2 max .A> DA/
i.e., the second inequality in (4.90) is true. Applying (4.89), we write the third
inequality in (4.90) in the form
kGx C gk2 .x z/> .Gx C g/

, (4.91)
.Gx C g/> .G 2 x C Gg/ kGx C gk2
where z 2 Fix T D M . Note that s.z/ D 0 and Gz C g D rf .z/ D 0, by (4.87)

and by the sufficient optimality condition. Since G is positive semi-definite, there
exists an orthogonal matrix U and a diagonal matrix WD diag d with d D
(ı1 ; ı2 : : : ; ım / 0 such that G D U > U (see Theorem 1.1.34 (iii)). Note that
maxi 2I ıi > 0, because A is a nonzero matrix. Let v D .1 ; 2 ; : : : ; m / 2 Rm be
such that v> v > 0 and let x WD U > v C z. Then
Gx C g D G.U > v C z/ C g D GU > v D U > v
and
kGx C gk2 D v> 2 v > 0,
i.e., Gx C g ¤ 0. Note that x … Fix T since x T x D s.x/ D Gx C g ¤ 0.
Similarly, we obtain
.Gx C g/.G 2 x C Gx/ D v> 3 v > 0
and
.x z/> .Gx C g/ D v> v.
Therefore, (4.91) can be written in equivalent forms
v> 2 v v> v
> 3
> 2
v v v v
or
.v> 2 v/2 .v> v/.v> 3 v/ 0:
We prove that the latter inequality is true. Denote I 0 WD fi 2 I W ıi > 0g. We have
.v> 2 v/2 .v> v/.v> 3 v/

X X X
D. ıi2 i2 /2 . ıi i2 /. ıi3 i2 /
i 2I i 2I i 2I
XX
D .ıi2 ıj2 i2 j2 ıi ıj3 i2 j2 /
i 2I j 2I
XX
D .ıi2 ıj2 i2 j2 ıi ıj3 i2 j2 /
i 2I 0 j 2I 0
XX ıj 2 2
D ıi2 ıj2 .1 /
ıi i j
i 2I 0 j 2I 0
X ıi ıj
D ıi2 ıj2 .2 /i2 j2
ıj ıi
i;j 2I 0 ;i >j
X ıj
D ıi ıj3 . 1/2 i2 j2 0
ıi
i;j 2I 0 ;i >j
i.e., the third inequality in (4.90) is true. Note that T is a cutter as a firmly nonex-
pansive operator having a fixed point (see Theorem 2.2.5 (i) and Corollary 4.4.4),
consequently, T is oriented (see Definition 2.4.4). By Corollary 2.4.5, the operator
T
EL is a cutter. Let 2 .0; 2/. The 2
-strong quasi nonexpansivity and asymptotic
regularity of T
EL ; follow from Theorems 2.1.39 and 3.4.3. t
u
4.7 Projected Landweber Operator
Let H1 and H2 be two Hilbert spaces. Consider the following split feasibility
problem
find x 2 C with Ax 2 Q,
if such an x exists, where C H1 and Q H2 are closed convex subsets and
A W H1 ! H2 is a bounded linear operator.
Definition 4.7.1. An operator U W H1 ! H1 defined by
1
U WD PC .Id C 2
A .PQ Id/A/ (4.92)
kAk
is called the projected Landweber operator or an oblique projection (see Fig. 4.32).
The following result is due to Byrne [55, Proposition 2.1].
Proposition 4.7.2. Let R WD PC T , where > 0, be a projected relaxation of the
Landweber operator T defined by (4.77), i.e.,

R x WD PC .x C A .PQ Id/Ax/, (4.93)
kAk2
and f W C ! RC be the proximity function defined by (4.78). Then
Fix R D Argmin f .x/:

x2C
Proof. The function f is convex as a composition of a linear operator A and a

convex function 12 d 2 .; Q/. Furthermore, f is differentiable and
Df .x/ D A .Ax PQ .Ax//.

4.8 Simultaneous Cutter 185
Fig. 4.32 Projected A

Landweber operator H1 H2
Ax
s =PQ Ax-Ax
1
x x+ A* 2 A*s
Ux
C Q
A*
Similarly as in the proof of Corollary 1.3.5, for any > 0, we have
x 2 Argmin f ” Df .x/ 2 NC .x/

C
” Df .x/ 2 NC .x/

” x D PC .x Df .x//
” x D PC .x C .A .PQ .Ax/ Ax///

” x D PC .x C A .PQ Id/Ax/ D R x
kAk2
” x 2 Fix R ,

u
If we take C D H1 and D 1 in Proposition 4.7.2, then we obtain Lemma 4.6.2.
Corollary 4.7.3. Let 2 .0; 2/. The projected relaxation of the Landweber
4
operator, defined by (4.93) is 4 -relaxed firmly nonexpansive. If, furthermore,
Fix .PC T / ¤ ;, then R is 2 -strongly quasi-nonexpansive and asymptotically
2
regular.
Proof. By Theorem 4.6.3 the Landweber operator T is firmly nonexpansive.
4
The 4 -relaxed firm nonexpansivity of R follows from Theorem 2.2.46 (i).
Let Fix.PC T / ¤ ;. The 2 2 -strong quasi nonexpansivity of R follows
from Theorem 2.2.46 (iii) and the asymptotic regularity of R follows from
Corollary 3.4.5. t
u
4.8 Simultaneous Cutter
If we replace the projections PCi by cutters in the definition of a simultaneous

projection (see Definition 4.4.1) and allow the weights to depend on a current point
x 2 H, then we obtain a more general operator. In this section we prove that this
operator is strongly quasi-nonexpansive and asymptotically regular.
Fig. 4.33 Simultaneous

cutter x
U1x FixU1
Ux
U2x
FixU2
Definition 4.8.1. Let Ui W H ! H be cutters, i 2 I , and w W H ! m be a weight

function. An operator U W H ! H defined by
X
Ux WD !i .x/Ui x (4.94)
i 2I
is called a simultaneous cutter (see Fig. 4.33).

By Corollary 2.1.49, a simultaneous cutter defined by (4.94) is a cutter if w W H !
m is an appropriate weight function.
Theorem
P 4.8.2. Let Ui be cutters with a common fixed point, i 2 I , and U WD
i 2I i i be a simultaneous cutter, where the weight function w W H ! m is
! U
appropriate. Then U is a cutter,
X
kU x zk2 kx zk2 .2 / !i .x/ kUi x xk2 (4.95)
i 2I
and
kU x zk2 kx zk2 .2 / kUx xk2 (4.96)
for all 2 Œ0; 2, x 2 H and z 2 Fix U . Consequently, for all 2 .0; 2/, the
operator U is 2
Proof. Since a cutter is strongly quasi-nonexpansive (see Theorem 2.1.39)

it is strictly quasi-nonexpansive
T (see Remark 2.1.44 (iii)). It follows from
Theorem 2.1.26 that Fix U D i 2I Fix Ui . Let 2 Œ0; 2, x 2 H and z 2 Fix U .
The convexity of the function kk2 and property (2.21) yield
2
X

kU x zk2 D x C !i .x/.Ui x x/ z

i 2I
2
X

D kx zk2 C 2 !i .x/.Ui x x/

i 2I
X
2 !i .x/hz x; Ui x xi
i 2I
4.9 Extrapolated Simultaneous Cutter 187
X
kx zk2 C 2 !i .x/ kUi x xk2
i 2I
X
2 !i .x/ kUi x xk2
i 2I
X
D kx zk2 .2 / !i .x/ kUi x xk2 ,
i 2I
i.e., inequality (4.95) holds. Inequality (4.96) follows now from the convexity of the
function kk2 . Let 2 .0; 2/. The 2
-strong quasi nonexpansivity of U follows
from (4.96) and from the obvious equality
2
.2 / kUx xk2 D kU x xk2 .

The asymptotic regularity of U follows now from Theorem 3.4.3. t
u
4.9 Extrapolated Simultaneous Cutter
Let Ui W H ! H be cutters with a common fixed point, i 2 I WD f1; 2; : : : ; mg. In

this section we will show how to extend the results of Theorem 4.8.2 to a P
generalized
relaxation of a simultaneous cutter. We denote, as in Sect. 4.8, U WD i 2I !i Ui ,
where w W H ! m is a weight function. Note that the estimation of the distance
kU x zk2 given by (4.95) is stronger than that given by (4.96). Therefore, we can
parameter which guarantees the strong quasi
expect that the range of the relaxation T
nonexpansivity of U with respect to i 2I Fix Ui can be extended over the interval
Œ0; 2. As we will see, our expectation is true if we allow the relaxation parameter
to depend on the point x 2 H. Therefore, it is more convenient to use a generalized
relaxation U
; W H ! H of the operator U , defined by
U
; .x/ WD x C
.x/.Ux x/ (4.97)
T
(cf. (2.67)). Denote Ci WD Fix Ui and C WD i 2I Ci . If we suppose that the weight
function w W H ! m is appropriate, then Theorem 2.1.26 yields Fix U D C .
4.9.1 Properties of the Extrapolated Simultaneous Cutter
In this section we consider a generalized relaxations of a simultaneous cutter with a

step size function
W H ! .0; C1/ satisfying the following inequality

.x/
w .x/, (4.98)
Fig. 4.34 Extrapolated

simultaneous cutter U1x
x
Ux FixU1
U ¾x
U2x
FixU2
where P 2
i 2I !i .x/ kUi x xk

w .x/ WD P 2 (4.99)

i 2I !i .x/Ui x x
T
for all x … i 2I Fix Ui , and the weight function w W H ! T m is appropriate.
We suppose without loss of generality that
.x/ D 1 for all T x 2 i 2I Fix Ui . The
fact that w is appropriate ensures that Fix U
; D Fix U D i 2I Fix Ui ¤ ; (see
Theorem 2.1.26 (i)), consequently, the step size
w .x/ is well defined. If
.x/ 1
for all Px 2 H, then the operator U
is an extrapolation of the simultaneous cutter
U WD i 2I !i Ui . We call an operator U
with a step size function
1 satisfying
inequality (4.98) for all x 2 H an extrapolated simultaneous cutter (ESC) (see
Fig. 4.34). The existence of an ESC follows from the convexity of the function
kk2 . It turns out that an extrapolated simultaneous cutter is a cutter.
P
Theorem 4.9.1. Let U WD i 2I !i Ui be a simultaneous cutter with an appropriate
weight function w W H ! m , U
; be a generalized relaxation of U , defined
by (4.97) with the T step size function
W H ! .0; C1/ satisfying inequality
(4.98) for x … i 2I Fix Ui , and with 2 Œ0; 2. Then the operator U
is a
cutter. Consequently, for all 2 .0; 2/, the operator U
; is 2
-strongly quasi-
nonexpansive and asymptotically regular and
kU
; x zk2 kx zk2 .2 /
2 .x/ kUx xk2 (4.100)
T
for all x 2 H and for all z 2 i 2I Fix Ui .
Proof. Since a cutter is strictly
T quasi-nonexpansive (see Theorem 2.1.39), it holds
that Fix U
; D Fix U D i 2I Fix Ui for an appropriate weight functionT wWH!
m (see Remark 2.4.2 (d) and Theorem 2.1.26). Let x 2 H and z 2 i 2I Fix Ui . If
T
x 2 i 2I Fix Ui , the inequality hz x; U
x xi kU
x xk2 is clear. Now let
T
x … i 2I Fix Ui . It follows from (2.21) that hz x; Ui x xi kUi x xk2 , i 2 I ,
consequently,
X
hz x; Ux xi D hz x; !i .x/.Ui x x/i
i 2I
X
D !i .x/hz x; Ui x xi
i 2I
X
!i .x/ kUi x xk2
i 2I
D
w .x/ kUx xk2 .
Therefore, inequality (4.98) yields
hz x; Ux xi
.x/ kUx xk2 .
Multiplying both sides by

.x/ and applying the equality U
x x D
.x/.Ux x/
we obtain
hz x; U
x xi kU
x xk2 .
Therefore, U
is a cutter (see Remark 2.1.31). Now Theorem 2.1.39 yields the 2 -
strong quasi nonexpansivity of U
; for all 2 .0; 2/. The asymptotic regularity of
U
; follows from Theorem 3.4.3, and inequality (4.100) follows from the definition
of U
; x. t
u
The properties of extrapolations of the simultaneous projection or of simultaneous
cutters were studied by Pierra [284], Dos Santos [146, Sect. 4], Cegielski [62,
Sect. 4.3], Kiwiel [229, Sect. 3], Bauschke [17, Sects. 7.3 and 8.3], Combettes [118,
Sects. 5.4–5.8], [120, Sect. IV], [121, Sect. 2], Crombez [127, Sect. 4], Aleyner and
Reich [5, Sect. 3] and by Cegielski and Censor [70, Sect. 9.5]. In [121, Proposition
2.4] a special case of Theorem 4.9.1 was proved, where it was supposed that
w 2 ri m and that
D
w . Extrapolations of simultaneous cutters with appropriate
weight functions were introduced in [70, Sect. 9.5].
4.9.2 Extrapolated Simultaneous Projection
In this section we apply the results

P of Sect. 4.9.1 to the generalized relaxation of the
simultaneous projection U WD i 2I !i PCi , where w W H ! m is an appropriate
weight function, Ci H are nonempty closed convex subsets, i 2 I , with C WD
T
i 2I Ci ¤ ;. We use here the name ‘simultaneous projection’ for a more general
case than in Sect. 4.4, because we allow w toP depend on x 2 H. Note that U is a
special case of the simultaneous cutter U WD i 2I !i Ui , where Ui WD PCi , i 2 I .
We consider a generalized relaxation U
; of U , where 2 Œ0; 2 and the step size
function
W H ! .0; C1/ satisfies the inequality

.x/
w .x/, (4.101)
where
P 2
i 2I !i .x/ kPCi x xk

w .x/ WD P 2 (4.102)

i 2I !i .x/PCi x x
Fig. 4.35 Extrapolated

simultaneous projection with P C1
the step size
WD
w x C1
Ux
U ¾x
P C2 C2
for x … C and
w .x/ D 1 for x 2 C . We have C D Fix U D Fix U
; , because
w is an appropriate weight function (see Theorem 2.1.26 and Remark 2.4.2 (d)).
Therefore,
w is well defined. We call an operator U
with a step size function
1 satisfying inequality (4.101) an extrapolated simultaneous projection (see

Fig. 4.35).
Corollary 4.9.2. Let C ¤ ;, w W H ! m be an appropriate weight function and
U W H ! H be a simultaneous projection defined by
X
Ux WD !i .x/PCi x. (4.103)
i 2I
Furthermore, let U
; be the generalized relaxation of U , defined by (4.97) with
2 Œ0; 2 and with the step size function
(4.101). Then U
is a cutter. Consequently, for all 2 .0; 2/, U
; is 2
-strongly
quasi-nonexpansive and asymptotically regular.
Proof. Since the metric projection PC is a cutter with Fix PC D C (see Theo-
rem 2.2.21 (i) and (ii)), the corollary follows directly from Theorem 4.9.1. t
u
4.9.3 Extrapolated Simultaneous Projection for LFP
Consider a consistent system of linear inequalities in Rn : ai> y ˇi , i 2 I , where

ai 2 Rn , ai ¤ 0 and ˇi 2 R, i 2 I . This system can be written in the matrix form
Ay b, where A is a matrix of type m n with rows ai , i 2 I , and b 2 Rm is a
vector with coordinates ˇi , i 2 I . If we take Ci WD H .ai ; ˇi / and apply (4.7) and
(4.70), then we obtain the following expression for the step size
w .x/ defined by
(4.102):
P h > i2
.ai xˇi /C
i 2I !i .x/ kai k

w .x/ D , (4.104)
A> D.x/.Ax b/C 2
for x … C WD fy 2 Rn W Ay bg, where D.x/ WD diag. !1 .x/2 ; : : : ; !m .x/2 / and the

ka1 k kam k
weight function w W Rn ! m is appropriate.
P The generalized relaxation U
; of
the simultaneous projection U WD i 2I !i PCi with the step size
.x/ WD
w .x/
can be presented as a -relaxation of the operator Tw D U
w defined by
P h i2
.ai> xˇi /C
i 2I !i .x/ kai k
>
Tw x WD x A D.x/.Ax b/C (4.105)
A> D.x/.Ax b/C 2
.a> xˇ /
for x … C and Tw x D x for x 2 C . Note that the terms i kai k i C , i 2 I , and
A> D.x/.Ax b/C do not change if we rescale inequalities of the system (see
Remark 4.6.5), consequently, the operator Tw does not depend on the rescaling of the
inequalities. Therefore, we can suppose, without loss of generality, that kai k D 1,
i 2 I . Then we have
P
!i .x/Œ.ai> x ˇi /C 2 >
Tw x D x i 2I A W .x/.Ax b/C , (4.106)
A> W .x/.Ax b/C 2
for x … C , where W .x/ WD diag w.x/. As before, C D Fix U D Fix Tw . Moreover,

T˛w D Tw for any function ˛ W Rn ! .0; C1/. Therefore, we can take w W Rn !
RmC nf0g in (4.106) instead of w W R ! m .
n
Corollary 4.9.3. Let a system of linear inequalities ai> x ˇi , i 2 I; be consistent

and the weight function w W Rm ! m be appropriate. Then the operator Tw given
by (4.106) is a cutter. Consequently, for all 2 .0; 2/, its relaxation Tw; is 2
-
strongly quasi-nonexpansive and asymptotically regular.
Proof. Since Tw D U
w , where U is defined by (4.103) and
w is defined by (4.104),
the corollary follows from Corollary 4.9.2. t
u
4.9.4 Surrogate Projection
Similarly as in Sect. 4.9.3, we consider a consistent system of linear inequalities

Ay b. Denote Ci WD H .ai ; ˇi / D fy 2 Rn W ai> y ˇi g, where ai 2 Rn is the
i th row of the matrix A and ˇi is the i th coordinate of b 2 Rm , i 2 I . We have
\
C WD fy 2 Rn W Ax bg D Ci :
i 2I
Let v W Rn ! m be a weight function. Let x 2 Rn . If we multiply the particular

inequalities ai> y ˇi of the system Ay b by nonnegative weights i .x/, i 2 I ,
and if we add the resulting inequalities, then we obtain the inequality
v.x/> Ay v.x/> b. (4.107)
Denote X
a.x/ WD i .x/ai D A> v.x/,
i 2I
X
ˇ.x/ WD i .x/ˇi D v.x/> b
i 2I
and
Cv.x/ WD fy 2 Rn W v.x/> Ay v.x/> bg.
It is clear that C Cv.x/ for any weight function v. If a.x/ ¤ 0, then Cv.x/ is a
half-space, Cv.x/ D H .a.x/; ˇ.x//, which is called a surrogate constraint for the
system of linear inequalities Ax b.
Definition 4.9.4. We say that a weight function v W Rn ! m is essential (for the
system Ax b) if for any x … C it holds that
v.x/> .Ax b/ > 0 (4.108)
or, equivalently, x … Cv.x/ D H .a.x/; ˇ.x//.

It turns out that for a consistent system Ay b, an essential weight function v
defines, for any x 2 Rn , a half-space (a surrogate constraint) containing the solution
set C .
Lemma 4.9.5. Let C ¤ ;. If v W Rn ! m is an essential weight function, then for
any x … C it holds that A> v.x/ ¤ 0 and C H .a.x/; ˇ.x//.
Proof. Let x … C and v.x/> .Ax b/ > 0. Suppose that A> v.x/ D 0. Then
v.x/> b 0, because Cv.x/ C ¤ ;. Consequently,
0 < v.x/> .Ax b/ D .A> v.x//> x v.x/> b 0.
The contradiction proves that a.x/ WD A> v.x/ ¤ 0. Now we have C Cv.x/ D
H .a.x/; ˇ.x//. t
u
Let v W Rn ! m be an essential weight function and Sv x be the metric projection
of x 2 Rn onto the surrogate constraint H .a.x/; ˇ.x//. If x 2 C , then, of course,
Sv x D x. If x … C , then
.a.x/> x ˇ.x//C
Sv x D x a.x/
ka.x/k2
Œ.A> v.x//> x v.x/> bC >
Dx A v.x/
A> v.x/2
Œv.x/> .Ax b/C >

Dx A v.x/.
A> v.x/2
Inequality (4.108) yields
v.x/> .Ax b/ >

Sv x D x A v.x/. (4.109)
A> v.x/2
Fig. 4.36 Surrogate

projection C1
C
x Sv x
C2
Note that Sv x is well defined, because A> v.x/ ¤ 0 (see Lemma 4.9.5). We call
Sv W Rn ! Rn a surrogate projection (see Fig. 4.36). Note that an essential weight
function v can be arbitrarily defined on C .
A surrogate projection Sv is not a metric projection, because the weights vi
defining the half-space H .a.x/; ˇ.x// depend on x … C .
Theorem 4.9.6. Let C ¤ ;. If v W Rn ! m is an essential weight function, then
Fix Sv D C and Sv is a cutter. Consequently, for any 2 .0; 2/, its relaxation Sv;
is 2
Proof. Let v be an essential weight function. It is clear that C Fix Sv . We prove

that Fix Sv C . Let x … C . Then Lemma 4.9.5 and equality (4.109) yield Sv x ¤ x,
i.e., x … Fix Sv . We have proved that Fix Sv D C . Let x 2 Rn and z 2 C . If x 2 C ,
then Sv x D x and the inequality
hz Sv x; x Sv xi 0 (4.110)
is obvious. Let now x … C . Since Sv x D PCv x and C Cv , inequality

(4.110) follows from the characterization of the metric projection (Theorem 1.2.4).
Therefore, Sv is a cutter. The second part of the theorem follows now from
Theorem 2.1.39 and from Corollary 3.4.4. t
u
An appropriate weight function v W Rn ! m does not need to satisfy condition
A> v.x/ ¤ 0 for x … C , consequently it does not need to be an essential weight
function.
Example 4.9.7. Let a 2 Rn , a ¤ 0. Let a1 D a2 D a; a3 D a. Consider a
consistent system of inequalities a1> x 0, a2> x 1, a3> x 1. The weight function
v W Rn ! 3 defined by v.x/ D . 14 ; 14 ; 12 / for all x 2 Rn is appropriate, because for
any x … C we have i .x/.PCi x x/ ¤ 0 for at least one i , but
1 1 1
1 .x/a1 C 2 .x/a2 C 3 .x/a3 D a C a a D 0,
4 4 2
i.e., A> v.x/ D 0.

Definition 4.9.8. We say that a weight function v W Rn ! m considers only
violated constraints if, for all x … C and all i 2 I , it holds
x 2 Ci H) i .x/ D 0. (4.111)
It is clear that a weight function v W Rn ! m which considers only violated

constraints is appropriate.
The idea of surrogate constraints for a system of linear inequalities was intro-
duces by Merzlyakov [260] and was continued by Oko in [277], Yang and Murty
in [351], where various weight functions which consider only violated constraints
were proposed, by Kiwiel [229], Kiwiel and Łopuch [233] and Cegielski [67].
The following lemma gives a sufficient condition for a weight function to be
essential.
Lemma 4.9.9. Let C ¤ ;. If a weight function v W Rn ! m considers only
violated constraints, then v is essential and, consequently, A> v.x/ ¤ 0 for all
x … C.
Proof. Let a weight function v consider only violated constraints and x … C . Denote
I.x/ WD fi 2 I W x … Ci g D fi 2 I W ai> x > ˇi g. Let j 2 I.x/ be such that
vj .x/ > 0. The existence of such j follows from the fact that v.x/ 2 m . We have
X X
v.x/> .Ax b/ D i .x/.ai> x ˇi / D i .x/.ai> x ˇi /
i 2I i 2I.x/
j .x/.aj> x ˇj / > 0,
i.e., v is essential. Now the second part of the Lemma follows from Lemma 4.9.5.
t
u
Note that S˛v D Sv for any weight function v W Rn ! m and for any function
˛ W Rn ! .0; C1/. Furthermore, Sv does not change if we rescale the inequalities.
Therefore, we can suppose without loss of generality that kai k D 1, i 2 I , and that
v W Rn ! Rm C nf0g instead of v W R ! m .
n
More general versions of the following two results were proved in [68, Theo-
rems 5 and 8]. The first result shows that any extrapolated simultaneous projection
is a surrogate projection.
Proposition 4.9.10. Let C ¤ ;, w W Rn ! m be an appropriate weight function
and v W Rn ! RmC nf0g be defined by
v.x/ WD W .x/.Ax b/C ,
where W .x/ WD diag w.x/, x 2 Rn . Then v considers only violated constraints.

Moreover, Tw D Sv , where Tw and Sv are defined by (4.106) and (4.109),
respectively.
Proof. If x 2 C , then, of course, Sv x D x D Tw x. Now let x … C . We have
i .x/ D !i .x/.ai> x ˇi /C , i 2 I . By the fact that w is appropriate, we have
j .x/ > 0 for at least one j 2 I , i.e., v.x/ ¤ 0. If x 2 Ci , then .ai> x ˇi /C D 0
and i .x/ D 0, i 2 I , i.e., v considers only violated constraints. By Lemma 4.9.9,
the weight function v is essential and the operator Sv is well defined. Note that
X
.Ax b/>
C W .x/.Ax b/ D .ai> x ˇi /C !i .x/.ai> x ˇi /
i 2I
X
D !i .x/Œ.ai> x ˇi /C 2
i 2I
D .Ax b/>
C W .x/.Ax b/C .
Therefore, if we take v.x/ WD W .x/.Ax b/C in (4.109), then we obtain

P >
i 2I !i .x/Œ.ai x ˇi /C
2
>
Tw x D x 2 A W .x/.Ax b/C D Sv x
A> W .x/.Ax b/C

u
The following result gives a relationship between the operators Sv and Tw , which is,
in some sense, converse to this presented in Proposition 4.9.10.
Proposition 4.9.11. Let C ¤ ;, a weight function v W Rn ! Rm C nf0g consider only
violated constraints and Sv be the surrogate projection defined by (4.109). Then
there is an appropriate weight function w W Rn ! m such that Sv D Tw , where Tw
is the extrapolated simultaneous projection defined by (4.106).
Proof. Denote I.x/ WD fi 2 I W ai> x ˇi > 0g and define
(
˛.x/1 a>ixˇ
.x/
for i 2 I.x/
!i .x/ WD i i (4.112)
0 otherwise,
x 2 Rn , where
X i .x/
˛.x/ WD >
.
a x ˇi
i 2I.x/ i
If x 2 C , then, of course, Sv x D x D Tw x. Now let x … C . By Lemma 4.9.9,

we have v.x/> .Ax b/ > 0 and A> v.x/ ¤ 0. It is clear that w.x/ 2 m and that
w is an appropriate weight function. Note that
!i .x/.ai> x ˇi /C D !i .x/.ai> x ˇi /
for all i 2 I . We have

X
A> W .x/.Ax b/C D !i .x/.ai> x ˇi /C ai
i 2I
X
D !i .x/.ai> x ˇi /ai
i 2I
X
D ˛.x/1 i .x/ai D ˛.x/1 A> v.x/
i 2I
and
X X
!i .x/Œ.ai> x ˇi /C 2 D !i .x/.ai> x ˇi /.ai> x ˇi /C
i 2I i 2I
X
D ˛.x/1 i .x/.ai> x ˇi /C
i 2I
D ˛.x/ v.x/> .Ax b/C .

1
Therefore,
P >
i 2I !i .x/Œ.ai x ˇi /C
2
>
Tw x D x 2 A W .x/.Ax b/C
A> W .x/.Ax b/C
Œv.x/> .Ax b/C >

Dx A v.x/ D Sv x
A> v.x/2

u
Let Tw W Rn ! Rn be defined by (4.106) for an appropriate weight function w and
Sv W Rn ! Rn be defined by (4.109) for an essential weight function v. Denote
T WD fTw W w is appropriateg, S WD fSv W v considers only violated constraintsg and
S 0 WD fSv W v is essentialg.
Corollary 4.9.12. Let C ¤ ;. It holds
T D S S0. (4.113)
Proof. The equality in (4.113) follows from Propositions 4.9.10 and 4.9.11. The
inclusion in (4.113) follows from Lemma 4.9.9. t
u
One can easily prove that for a system Ax b containing at least two inequalities
defining two different half-spaces the inclusion in (4.113) is strict. We leave the
details to the reader.
Corollary 4.9.13. Let C ¤ ;. If a weight function v W Rn ! Rm C nf0g considers
only violated constraints, then a surrogate projection Sv W Rn ! Rn is a
cutter. Consequently, for any 2 .0; 2/, its relaxation Sv; is 2

-strongly quasi-
Proof. The corollary follows from Lemma 4.9.9 and from Theorem 4.9.6. Since
T D S, the corollary also follows directly from Corollary 4.9.3. t
u
4.9.5 Surrogate Projection with Residual Selection
In this section we consider a special case of the surrogate projection Sv defined by

(4.109). For a matrix A with rows ai , i 2 I , and for a subset L I , denote by AL
a submatrix with rows ai , i 2 L. For simplicity, suppose that L D f1; 2; : : : ; rg,

where r m, i.e.,
AL
AD .
AI nL
We apply the same notation as above for a subvector bL of b, for a subvector vL .x/
of the vector v.x/ and for a subvector rL .x/ of the residual vector r.x/ WD Ax b,
x 2 Rn , i.e.,

bL vL .x/ rL .x/
bD , v.x/ D and r.x/ D .
bI nL vI nL .x/ rI nL .x/
Let x … C D fx 2 Rn W Ax bg. Let L WD L.x/ I be such that:

(i) AL has full row rank, i.e., ai , i 2 L, are linearly independent,
(ii) .AL A> 1
L / .AL x bL / 0.
Note that for any x … C the weight vector
vL .x/ WD .AL A> 1

L / .AL x bL /
is essential for the system AL x bL , consequently, the weight vector v.x/ with
vI nL .x/ D 0 2 Rmr is essential for the system Ax b. Therefore, the surrogate
projection Sv x given by (4.109) is well defined.
We say that a subset L.x/ satisfying the conditions (i) and (ii) above is a residual
selection (RS) of I . Note that such a subset exists, e.g., L.x/ WD fj.x/g, where
j.x/ 2 I.x/ WD fi 2 I W ai> x > ˇi g. Residual selections were studied in
[72, Sect. 3.2] and [73, Sect. 3.1], where several constructions of maximal subsets
L satisfying the conditions (i) and (ii) above were presented. If we replace the
condition (ii) above by the following stronger condition:
(iii) .AL A>
L/
1
0 and AL x bL 0,
then the residual selection L.x/ I is called an obtuse cone selection (OCS).
The notion OCS can be explained by the fact that for a full row matrix AL the
cone generated by the rows of AL is obtuse (in Linfai W i 2 Lg) if and only if
.AL A>L/
1
0 (see [232, Lemma 3.1] or [65, Lemma 1.6]). A special case of an
obtuse cone selection is a regular obtuse cone selection (ROCS), for which the
condition (iii) above is replaced by
(iv) ai> aj 0 for i ¤ j , i; j 2 L, and AL x bL 0
(see Fig. 4.37).
The first inequality in (iii) says that the inverse of the Gram matrix G WD
AL A>L of rows of AL is nonnegative. Nonsingular matrices with nonnegative
inverses, called inverse-positive matrices, play an important role in many areas
of mathematics and were studied in [35]. The properties of Gram matrices with
nonnegative inverse were also studied in [232, Sect. 3] and [65, Sect. 2], where their
Fig. 4.37 Regular obtuse C1 a5

cone selection a1
PC 1 x C5
PC 3 x SÀ x
x
C
a2 PC 4 x
PC 2 x a4
a3 C6
C2 C4
a6
relations to obtuse cones are presented. We see that L is an obtuse cone selection if
and only if G is inverse-positive and AL x bL 0.
The first inequality in (iv) says that G has nonpositive off-diagonal elements.
Nonsingular matrices with a nonnegative inverse having this property are called M-
matrices or Minkowski matrices. The properties of such matrices were studied in
[166], [326, Sect. 4], [62, Chap. 5], [232, Sect. 3] and [65, Sect. 2]. In particular, it
follows from [166, Theorem 4.3] that a nonsingular Gram matrix with nonpositive
off-diagonals has a nonnegative inverse. This result also follows from [62, Theorem
5.4.A], from [232, Lemma 3.2], and from [65, Corollary 2.9]. Several constructions
of maximal subsets L.x/ being obtuse cone selection or regular obtuse cone
selection are presented in [326, Sect. 4], [62, Chap. 5], [65, Sect. 2]. In [62, Chap. 6],
[64, 230, 231] one can find applications of such selections to algorithms for convex
minimization problems.
Suppose that for any x … C the subset L.x/ I is a residual selection. If we
take vL .x/ D .AL A> 1
L / .AL x bL / and vI nL .x/ D 0, then equality (4.109) obtains
the form
Sv x D x A> > 1
L .AL AL / .AL x bL /. (4.114)
The operator Sv W Rn ! Rn defined by (4.114) is called a surrogate projection with
residual selection.
Further properties of the residual selection as well as its applications to the
surrogate projection methods are presented in Sect. 5.13.
4.9.6 Extrapolated Simultaneous Subgradient Projection
Let ci W H ! R be convex continuous functions, Ci WD fx 2 H W ci .x/ 0g be

nonempty and Ui be the subgradient projections relative to ci , i.e.,
(
c C .x/
x i 2 gi .x/ if ci .x/ > 0
Ui .x/ WD kgi .x/k (4.115)
x if ci .x/ 0,
where gi .x/ 2 @ci .x/ is a subgradient of ci at x, i 2 I (cf. (4.31)). It follows from

Lemma 4.2.5 and from Corollary 4.2.6 that Fix Ui D Ci and that Ui is a cutter.
4.10 Extrapolated Cyclic Cutter 199
P
We call the operator U WD i 2I !i Ui , where w W H ! m is a weight function,
a simultaneous subgradient projection. In this section we apply Theorem 4.9.1 to
the extrapolated simultaneous subgradient projection U
with the step size function
W H ! .0; C1/ satisfying the following inequality
1
.x/
w .x/, (4.116)
where
2
Pm ciC .x/
i D1 wi .x/ kgi .x/k

w .x/ WD 2 (4.117)
Pm ciC .x/

i D1 wi .x/ kgi .x/k2 gi .x/
T
for all x … i 2I Ci ¤ ;, and the weight function w is appropriate (for simplicity,
.c .x//
we use the convention that kgi i .x/kC D 0 if .ci .x//C D 0). Note that the step
size
w .x/ defined by (4.117) is a special case of the step size (4.99), where Ui
is the subgradient projection defined by (4.115), i 2 I .
Corollary 4.9.14. Let U
; beP a generalized relaxation of the simultaneous
subgradient projection U WD i 2I !i Ui with an appropriate weight function
w W H ! m , with
T a step size function
(4.116) for x … i 2I Ci ¤ ; and with 2 Œ0; 2. Then the operator U
is a cutter.
Consequently,
kU
; x zk2 kx zk2 .2 /
2 .x/ kUx xk2
2
D kx zk2 kU
; x xk2

T
for all x 2 H, z 2 i 2I Fix Ui and for all 2 .0; 2/, i.e., U
; is 2
-strongly
quasi-nonexpansive and asymptotically regular.
Proof. Since a subgradient projection Ui is a cutter and Fix Ui D Ci , i 2 I , the
corollary follows from Theorem 4.9.1. t
u
The properties of a special case of the extrapolated simultaneous subgradient projec-
tion were studied by Dos Santos in [146], where
D
w and an appropriate weight
function w is supposed to be constant, i.e., w 2 ri m . The study was continued
for various convex optimization problems by Cegielski [62, Sect. 4.3], Kiwiel [229,
Sects. 3 and 8], [230, 231], Kiwiel and Łopuch [233, Sect. 2], Combettes [118,
Sect. 5.6], [120, Sect. V] and by Cegielski and Censor [70].
4.10 Extrapolated Cyclic Cutter
Let Ui W H ! H be cutters with a common fixed point, i 2 I WD f1; 2; : : : ; mg.

In this section we will extend the results of Corollary 4.5.15 to the generalized
relaxation of the cyclic cutter. Similarly as in Sect. 4.9, we allow the relaxation
parameter to depend on x 2 H. Therefore, we work with a generalized relaxation

U
; of a cyclic cutter U WD Um Um1 : : : U1 , with a step size function
W H !
.0; C1/ andT a relaxation parameter 2 Œ0; 2. It follows from Corollary 4.5.15 that
Fix U D i 2I Fix Ui and that U is strongly quasi-nonexpansive. We will extend
this result to the generalized relaxation of U , defined by
U
; x WD I C
.x/.Ux x/.
4.10.1 Useful Inequalities
Let S0 WD Id and Si WD Ui Ui 1 : : : U1 , i D 1; 2; : : : ; m. Of course, U D Sm .

Denote
u0 WD x; ui WD Ui ui 1 and y i WD ui ui 1 , (4.118)
i D 1; 2; : : : ; m. We have ui D Si x.
Lemma 4.10.1. Let Ui W H ! H be cutters with a common fixed point, i 2 I . The
following inequalities hold
m
X m
1 X
m
2 X 2
y 1
hUxx; zxi hy C: : :Cy ; y i
i m i i
i
y , (4.119)
2 2m
i D1 i D1 i D1
Tm
where x 2 H and z 2 i D1 Fix Ui ¤ ;.
Proof. (cf. [71, Lemma 7]) Let x 2 H be arbitrary. We prove the first inequality in
(4.119) by induction with respect to m. For m D 1 this inequality follows directly
from the definition of a cutter and from inequality (2.21). Suppose that the first
inequality in (4.119) is true for some m D k. Let w 2 H. Define V1 WD Id, Vi WD
Ui Ui 1 : : : U2 for i D 2; 3; : : : ; kC1, v1 WD w, vi WD Ui vi 1 and zi WD vi vi 1 , i D
TkC1
2; 3; : : : ; k C 1. Take w WD U1 x and z 2 i D1 Fix Ui . Then, of course, Si x D Vi w,
ui D vi and y i D zi , i D 2; 3; : : : ; k C 1. It follows from the induction assumption
that
X
kC1
hVkC1 w w; z wi hzi C : : : C zkC1 ; zi i (4.120)
i D2
PkC1
Note that VkC1 w w D i D2 zi and w x D y 1 . Since U1 is a cutter, it follows
from (4.120) that
hSkC1 x x; z xi
D hVkC1 w x; z xi
D hVkC1 w w; z xi C hU1 x x; z xi
2
hVkC1 w w; z wi C hVkC1 w w; w xi C y 1
4.10 Extrapolated Cyclic Cutter 201
X
kC1 X
kC1
2
hzi C : : : C zkC1 ; zi i C h zi ; y 1 i C y 1
i D2 i D2
X
kC1 X
kC1
2
D hy i C : : : C y kC1 ; y i i C hy i ; y 1 i C y 1
i D2 i D2
X
kC1
D hy i C : : : C y kC1 ; y i i.
i D1
Therefore, the first inequality in (4.119) is true for all m 2 N. The second inequality
follows from the following equality
m
Xm X
m
X 2
1 y i D
2 1
hy i C : : : C y m ; y i i yi .
i D1
2 i D1
2
i D1
The third inequality in (4.119) follows from the convexity of the function kk2 . t
u
4.10.2 Properties of the Extrapolated Cyclic Cutter
We use the notation from the previous subsection. Define the step size function
max W H ! .0; C1/ by

Pm
hUx Si 1 x; Si x Si 1 xi

max .x/ WD i D1 (4.121)
kUx xk2
for x … Fix U and

.x/ D 1 for x 2 Fix U . If we take y i WD Si x Si 1 x,
i D 1; 2; : : : ; m (cf. (4.118)), then Lemma 4.10.1 yields
1 Pm
kSi x Si 1 xk2 1

max .x/ 2 i D1 2
, (4.122)
kUx xk 2m
where x … Fix U .
Note that
X
m X
m
hy i C : : : C y m ; y i i D hy 1 C : : : C y i ; y i i.
i D1 i D1
Therefore, the step size

max .x/ given by (4.121) can be equivalently written in the
following form
Pm
hSi x x; Si x Si 1 xi

max .x/ D i D1 . (4.123)
kUx xk2
Now consider the generalized relaxation U

; of the cyclic cutter U with a step
size function
.x/
max .x/ for all
x … Fix U , where
max .x/ is given by (4.121) or (4.123).
Theorem 4.10.2. The operator U
is a cutter. Consequently, for all 2 .0; 2/, the
operator U
; is 2
T
Proof. (cf. [71, Lemma 8]) TLet x 2 H, 2 .0; 2/ and z 2 i 2I Fix Ui . It is clear
that Fix U
; D Fix U D i 2I Fix Ui (see Theorem 2.1.26 and Remark 2.4.2 (d)).
The first inequality in (4.119) can be written in the form
hUx x; z xi
max .x/ kUx xk2 . (4.124)
Consequently,
hUx x; z xi
.x/ kUx xk2 .
Now Corollary 2.4.5 yields that U
is a cutter and U
; is 2 -strongly quasi-
nonexpansive. The asymptotic regularity of U
; follows from Theorem 3.4.3. u t
4.11 Exercises
Exercise 4.11.1. Let B.z; / H, be a ball with the centre z 2 H and a radius
> 0. Prove that B.z; / is a nonempty closed convex subset and that
(
x if kx zk
PB.z;/ .x/ D
zC kxzk .x z/ if kx zk > .
Exercise 4.11.2. Let A H1 and B H2 be closed convex subsets and .x; y/ 2

A B. Prove that NAB .x; y/ D NA .x/ NB .y/.
Exercise 4.11.3. Prove that max .˛1 A1 C ˛2 A2 / ˛1 max .A1 / C ˛2 max .A2 /,
where A1 ; A2 are positive semi-definite matrices and ˛1 ; ˛2 0.
Exercise 4.11.4. Prove Corollary 4.6.7
Exercise 4.11.5. Let C H be a closed convex subset. Prove that PC D Pd.;C / ,
where Pd.;C / denotes a subgradient projection relative to the distance function
d.; C / WD infy2C k yk.
Chapter 5
Projection Methods
Let X H be a nonempty closed convex subset and M X be a subset of

solutions of a convex optimization problem defined on X . Such problems were
defined in Sect. 1.3. The subset M is closed and convex and, usually, T nonempty.
As an example, consider the convex feasibility problem: find x 2 i 2I Ci , where
Ci H are nonempty closed convex subsets, i 2 I WD Tf1; 2; : : : ; mg, if such a point
exists. If the problem is consistent, we have M D i 2I Ci ¤ ;. If the problem
is inconsistent, then M is defined as a subset of minimizers of a corresponding
proximity function f W H ! RC .
Let xN 2 X be an approximation of a solution of a convex optimization problem.
An iteration in a projection method for this problem has the form x C D U x, N where
U W X ! X is an appropriate algorithmic projection operator constructed for
this problem. Most of projection methods employ algorithmic operators U which
are relaxations of cutters with Fix U M or, equivalently, SQNE operators (see
Theorem 2.1.39). By Corollary 2.2.9, any relaxed firmly nonexpansive operator
U W X ! X with Fix U M has this property. This is the reason why firmly
nonexpansive operators are often used in projection methods. Averaged operators
are also often employed in these methods. Note, however, that the class of averaged
operators and the class of strictly relaxed firmly nonexpansive operators coincide
(see Corollary 2.2.17). Therefore, without loss of generality, we will restrict our
further analysis to relaxed cutters and to relaxed firmly nonexpansive operators.
Any projection method can be presented in the form of a recurrence
x0 2 X – arbitrary
x kC1 D Rk x k ,
where Rk WD PX .Id Ck .Tk Id//, k 2 .0; 2, and Tk W X ! H is a cutter. In

most cases Fix Tk M or Fix.PX Tk / M , k 0. Examples of such operators
were presented in Chaps. 2 and 4. If the operator Tk D T and k D for all k 0,
then we say that the method is autonomous. Otherwise, we say that the method is
nonautonomous. The iteration can also be presented in the form
204 5 Projection Methods
x kC1 D PX .x k C k .Tk x k x k //. (5.1)
If X is a closed affine subspace and Tk W X ! X , then we can omit the operator

PX in (5.1) and we obtain
x kC1 D x k C k .Tk x k x k /. (5.2)
In autonomous methods the sequence of operators fTk g1 kD0 as well as the sequence
of relaxation parameters .k / are constant and these methods have the form
x kC1 D PX .x k C .T x k x k //. (5.3)
Generally, the convergence (at least weak) of sequences generated by projection

methods follows from Opial’s theorem (see Sect. 3.5) or from its generalizations
(see Sects. 3.6 and 3.7). In this chapter we show how to apply these theorems in
order to prove the convergence.
5.1 Alternating Projection Methods
The alternating projection method (APM), also known as the von Neumann method
is an iterative method for finding a common point of two nonempty closed convex
subsets A; B H, if such a point exists. One iteration of the method has the form
x kC1 D PA PB x k (5.4)
or x kC1 D T x k , where T WD PA PB is an alternating projection. The properties of

this operator were described in Sect. 4.3. In this section we consider a more general
method in the form
x kC1 D PA .x k C k .PA PB x k x k //, (5.5)
where k 2 Œ0; 2 which we call a relaxed alternating projection method. We can

also write x kC1 D PA Tk x k , where Tk is the k -relaxation of the alternating
projection T WD PA PB . Note that if k D 1, then (5.5) reduces to (5.4).
5.1.1 General Case
Corollary 5.1.1. Let A; B H be nonempty closed convex subsets with a common

point and lim inf k .2 k / > 0. Then, for an arbitrary x 0 2 A, the sequence
fx k g1
kD0 generated by the relaxed alternating projection method (5.5) converges
weakly to a point x 2 A \ B.
5.1 Alternating Projection Methods 205
Proof. The operator T WD PA PB is nonexpansive as a composition of nonexpansive

operators PA and PB , and Fix T D A\B (see Corollary 4.3.5 (v)). Furthermore, T W
A ! A is a cutter (see Lemma 4.3.2) and a demi-closed operator (see Lemma 3.2.5).
Corollary 3.7.3 with X D A yields now the weak convergence of x k to a point
x 2 A \ B. t
u
If we suppose that Fix PA PB ¤ ; instead of A \ B ¤ ; in Corollary 5.1.1, then the
weak convergence holds for a tighter range of relaxation parameters.
Corollary 5.1.2. Let A; B H be nonempty closed convex subsets with
Fix PA PB ¤ ; and k 2 Œ"; 32 " for some " 2 .0; 34 /. Then, for an arbitrary
x 0 2 A, the sequence fx k g1
kD0 generated by the relaxed alternating projection
method (5.5) converges weakly to a fixed point of the operator T WD PA PB .
Proof. By Corollary 4.3.3, the alternating projection T WD PA PB is 43 -relaxed
firmly nonexpansive and the operator S WD T 3 D Id C 34 .T Id/ is firmly nonex-
4
pansive. Note that Fix S D Fix T , Sk D Tk , where k D 43 k (see Remarks 2.1.3
and 2.1.4) and that lim infk k .2 k / > 0. Iteration (5.5) can be written in the
form x kC1 D PA Sk x k . The weak convergence of x k to a point x 2 Fix PA PB
follows now from Corollary 3.7.3. t
u
Since iteration (5.4) is a special case of (5.5), we obtain the following result
immediately.
Corollary 5.1.3. Let A; B H be nonempty closed convex subsets. If FixPA PB ¤
;, then for an arbitrary x 0 2 H the sequence fx k g1
kD0 generated by the alternating
projection method (5.4) converges weakly to a fixed point of the operator T WD
PA PB .
Corollary 5.1.3 is a special case of [42, Theorem 1], where the weak convergence
was proved for the cyclic projection method. The strong convergence of sequences
generated by the (relaxed) alternating projection method does not hold in general.
An example of a closed convex cone A H, a closed half-space B H, with
A\B D f0g and a point x 2 H for which .PA PB /k x converges weakly but does not
converge strongly was presented by Hundal in [213, Theorem 1]. A simplification
of Hundal’s example can be found in [258]. In the literature one can find several
conditions for the strong convergence in Corollary 5.1.3, e.g., one of the subsets
A; B is compact (see [112, Theorem 4 (a)]) or A; B are closed affine subspaces and
d.A; B/ is attained (see [21, Theorem 4.1]). In [20, Sects. 3–6], [14, Corollary 2.4],
[258, Theorem 4.1] and [235, Theorem 4.5] one can find other sufficient conditions
for the strong convergence.
Recall that a composition of RFNE operators is RFNE (see Theorem 2.2.37).
Therefore, Corollary 5.1.3 remains true if we replace the projections PA and PB by
their relaxations PA; WD .1 / Id CPA and PB; WD .1 / Id CPB , where
; 2 .0; 2/.
If A is a closed affine subspace, the weak convergence in Corollary 5.1.2 is
guaranteed for a broader range of the relaxation parameter. Note that in this case
T .A/ A and the recurrence (5.5) can be written in the equivalent form
x kC1 D x k C k .PA PB x k x k /. (5.6)
The following result is due to Combettes [117, Theorem 1], where A H is

supposed to be a closed subspace of H.
Theorem 5.1.4. Let A H be a closed affine subspace, B H a nonempty closed
convex subset and lim inf k .2 k / > 0. If Fix PA PB ¤ ;, then for an arbitrary
x 0 2 A the sequence fx k g1 kD0 generated by the relaxed alternating projection
method (5.6) converges weakly to a fixed point of the operator T WD PA PB .
Proof. The operator T WD PA PB is firmly nonexpansive (see Theorem 4.3.7).
Therefore, the theorem follows directly from Corollary 3.7.3. t
u
5.1.2 Alternating Projection Method for Closed Linear

Subspaces
Suppose that A and B are closed subspaces of a Hilbert space H. Then the strong
convergence holds in Corollary 5.1.3. The following theorem is due to John von
Neumann [271, Theorem 13.7].
Theorem 5.1.5 (von Neumann, 1933). Let A; B H be closed subspaces. Then,
for any x 0 2 H, the sequence fx k g1kD0 generated by the alternating projection
method (5.4) converges in norm to y WD PA\B x 0 .
Proof. Let x 0 2 H and x k D T k x 0 , k 0, where T WD PA PB . We divide the proof
into three parts.
(i) Since the metric projection is strictly quasi-nonexpansive (see Theorem 2.2.21
(iii) and Corollary 2.2.9) and A \ B ¤ ;, we have Fix T D A \ B (see
Theorem 2.1.26). The weak convergence of x k to a point x 2 A \ B follows
from Corollary 5.1.3.
(ii) We show that x D PA\B x 0P . Denote y 2l WD x l and y 2lC1 WD PB x l , l 0.
2k1 l
It is clear that y y D
i 2k
lDi y y
lC1
, for all i; k 0, i 2k. Let
z 2 A \ B be arbitrary. By Theorem 2.2.30 (i), we have
hy 2l y 2lC1 ; zi D hx l PB x l ; zi D 0
and
hy 2lC1 y 2lC2 ; zi D hPB x l PA PB x l ; zi D 0,
l 0. Consequently,
X
2k1
hy i x ; zi D limhy i y 2k ; zi D limh y l y lC1 ; zi
k k
lDi
X
2k1
D lim hy l y lC1 ; zi D 0,
k
lDi
5.1 Alternating Projection Methods 207
i 0. In particular,
hPB x 0 x ; x i D hy 1 x ; x i D 0
and
hx 0 x ; zi D hy 0 x ; zi D 0,
i.e., x D PA\B x 0 .
(iii) We show that limk x k x D 0. Since the metric projection onto a
closed subspace is self-adjoint (see Theorem 2.2.30 (iii)) and idempotent (see
Theorem 2.2.21 (i)), we have
.T /k T k D PB PA : : : PB PA PA PB : : : PA PB D PB .PA PB / : : : .PA PB / D PB T 2k1 .

„ ƒ‚ …„ ƒ‚ … „ ƒ‚ …
k-times k-times .k1/-times
Since x 2 Fix T , we have
hx k x ; x k i D hT x k1 T x ; x k i D hx k1 x ; T x k i.
If we iterate the above equalities k-times and apply the facts that x k * x and
PB x D x , we obtain
hx k x ; x k i D hx 0 x ; .T /k x k i D hx 0 x ; .T /k T k x 0 i
D hx 0 x ; PB T 2k1 x 0 i D hPB x 0 PB x ; T 2k1 x 0 i
D hPB x 0 x ; x 2k1 i ! hPB x 0 x ; x i D 0.
Consequently, the weak convergence x k * x yields

2
lim x k x D lim.hx k x ; x k i hx k x ; x i/ D 0
k k
i.e., x k ! x D PA\B x 0 . t
u
Several proofs of the von Neumann theorem can be found, e.g., in [152], [140,
Theorem 9.3] [235, Theorem 1.1] and [236].
Theorem 5.1.5 remains true if A and B are closed affine subspaces with a
common point. The details are left to the reader.
One can also estimate the rate of convergence of the alternating projection
method x kC1 D PA PB x k . It turns out that when A and B are closed subspaces of
H the rate depends on the angle between the subspaces. The following definition is
due to Friedricks [170, Sect. 1.1] (see also [139, Sect. 6] and [140, Definition 9.4]).
Definition 5.1.6. Let A; B H be closed subspaces and
c.A; B/ WD supfjhx; yij W x 2 A\.A\B/? ; y 2 B \.A\B/? ; kxk 1; kyk 1g

The value ˛.A; B/ WD arccos c.A; B/ 2 Œ0; 2 is called an angle between the
subspaces A and B.
It follows from the Cauchy–Schwarz inequality that c.A; B/ 2 Œ0; 1, conse-
quently, the the angle between the subspaces A and B is well defined. The proof of
the following theorem can be found in [11, page 379] or in [140, Sect. 9.8].
Theorem 5.1.7. Let A; B H be two closed subspaces, c WD c.A; B/, x 0 2 H,
and fx k g1
kD0 be generated by the alternating projection method (5.4). Then
k
x PA\B x 0 c 2k1 x 0 PA\B x 0 ,
consequently fx k g1
kD0 converges geometrically, if c < 1.
In papers [290, 291, 313] the angles between subspaces are also used to obtain
convergence rates for iterative projection methods. Further results on the alternating
projection method can be found, e.g., in [20, 21], [140, Chap. 9] and [174].
Applications of the alternating projection method in various areas of mathematics
were presented in [139].
5.2 Extrapolated Alternating Projection Methods
The sequences generated by the alternating projection method often converge very
slowly. Such a slow convergence is observed, e.g., if A is a hyperplane and B is a
ball which is tangent to A (see, e.g., [21, Example 5.3]) or A and B are two closed
subspaces with c.A; B/ close to 1. In these cases the angle between x k PB x k and
PA PB x k PB x k is close to 0 which causes very short steps PA PB x k x k . Since
the alternating projection method has many practical applications, any acceleration
technique seems to be important. In this section we give several modifications of
the alternating projection method which lead in practice to an acceleration of the
convergence. All these methods have the form
x kC1 D PA Tk ;k x k
where x 0 2 A, Tk ;k W A ! H is a generalized relaxation of the alternating projection

T WD PA PB , defined by
Tk ;k .x/ WD x C k k .x/.PA PB x x/
for x 2 H, where fk g1 kD0 W H ! .0; C1/ is a sequence of step size functions
and k 2 Œ0; 2 is a relaxation parameter (cf. Sect. 2.4). Suppose, for simplicity, that
.x/ D 1 for x 2 A \ B. We can also write
x kC1 D PA .x k C k k .x k /.PA PB x k x k //. (5.7)

5.2 Extrapolated Alternating Projection Methods 209
If k .x k / 1, k 0, then the method described by recurrence (5.7) is called an

extrapolated alternating projection method.
5.2.1 Acceleration Techniques for Consistent Problems
Suppose that A \ B ¤ ;. Below we present several step size functions W

A ! Œ1; C1/ which guarantee the week convergence of sequences generated by
recurrence (5.7) with k 2 Œ"; 2 " for some " 2 .0; 1/. Let T WD PA PB .
5.2.1.1 Gurin–Polyak–Raik Approach
Let
kPB x xk2
GPR .x/ WD , (5.8)
hPB x x; T x xi
for x 2 AnB. The step size (5.8) was proposed by Gurin et al. in [196, Sect. 3].
Recall that GPR .x/ 1 for all x 2 A and that GPR .x/ D 1 if and only if
PB x 2 A\B (see Lemma 4.3.13). Therefore, recurrence (5.7) with k D GPR .x k /
describes an extrapolated alternating projection method. The following theorem
extends the results of [196, Theorem 4].
Corollary 5.2.1. Let x 0 2 A and the sequence fx k g1
kD0 be generated by (5.7),
where
1 k .x/ GPR .x/
and lim inf k .2 k / > 0. Then x k converges weakly to a point x 2 A \ B.
Proof. Denote Tk WD Tk . It follows from Corollary 4.3.14 that the operator TGPR is
a cutter. Therefore, Tk are also cutters, k 0, (see Lemma 2.4.7 (ii)).The operator
T is nonexpansive as a composition of nonexpansive operators PA and PB . It is
clear that Fix Tk D Fix T D A \ B. We have

Tk x k x k D k .x k / T x k x k T x k x k

consequently, T x k x k ! 0 whenever Tk x k x k ! 0. Corollary 3.7.1 with
S D T and X D A yields now the weak convergence of x k to a point x 2 A\B. u t
Gurin et al. proved a special case of Corollary 5.2.1 with k D GPR and k D 1
for all k 0 (see [196, Theorem 4]).
5.2.1.2 Bauschke–Combettes–Kruk Approach
Let A H be a closed affine subspace and x 2 A. Then it follows from

Theorem 2.2.33 (i) that
hPB x x; T x xi D hPB x T x; T x xi C kT x xk2 D kT x xk2 .
Therefore, for x 2 AnB, we have
kPB x xk2 kPB x xk2

GPR .x/ D D
hPB x x; T x xi kT x xk2
The weak convergence of sequences generated by the extrapolated alternating

2
projection method (5.7) with the step size k .x/ WD kPB xxk2 for x … Fix T D
kT xxk
A\B, k 0, was proved by Bauschke, Combettes and Kruk in [25, Corollary 4.11].
5.2.1.3 Bauschke–Deutsch–Hundal–Park Approach
Let A; B H be closed subspaces and x 2 A. Then it follows from Theorem 2.2.30

(i) that
hx; x T xi D hx; x PB xi C hx; PB T xi D hx; x PB xi

D kx PB xk2 C hPB x; x PB xi D kx PB xk2 .
Therefore, for x … Fix T D A \ B, we have
kPB x xk2 hx; x T xi

GPR .x/ D 2
D .
kT x xk kT x xk2
The strong convergence of sequences generated by the extrapolated alternating

projection method (5.7) with the step size k .x/ WD hx;xT xi2 for x … Fix T D
kT xxk
A \ B, k 0, was proved by Bauschke et al. in [30, Theorem 3.23]. Actually,
Bauschke et al. proved that the method is faster than the alternating projection
method. Furthermore, they presented the rate of convergence of the method (see
[30, Theorem 3.16]).
5.2.2 Acceleration Techniques for Inconsistent Problems
In this section we suppose that Fix T ¤ ;, where T WD PA PB , but we do not

suppose that A \ B ¤ ;. We present two step size functions W A ! .0; C1/
5.2 Extrapolated Alternating Projection Methods 211
which guarantee the weak convergence of sequences generated by recurrence (5.7)

with k 2 Œ"; 2 " for an arbitrary " 2 .0; 1/. Denote
ı WD ı.A; B/ D inf kx yk
x2A;y2B
and
N
ı.x/ WD kT x PB xk
for x 2 A.
First we consider the general case, where A; B H are nonempty closed convex
subsets. Let
Q
kT x PB xk2 ı.x/ kPB x xk C hPB x x; T x xi
CS1 .x/ WD
kT x xk2
for x … Fix T , where ı.x/ Q N

2 Œı; ı.x/. Recall that CS1 .x/ 1
and that the
2
inequality is strict if T x … Fix T (see Lemma 4.3.10).
kD0 be generated by (5.7),
where
1
k .x/ CS1 .x/
2
and lim inf k .2 k / > 0. Then x k converges weakly to a point x 2 Fix PA PB .
Proof. Denote Tk WD Tk D Id Ck .T Id/. It follows from Theorem 4.3.11 that
TCS1 is a cutter. Therefore, Tk are also cutters, k 0, (see Lemma 2.4.7 (ii)). It is
clear that T is nonexpansive and that Fix Tk D Fix T . We have

Tk x k x k D k .x k / T x k x k 1 T x k x k ,
2

consequently, T x k x k ! 0 whenever Tk x k x k ! 0. Corollary 3.7.1 with
S D T and X D A yields now the weak convergence of x k to a point
x 2 Fix PA PB . t
u
The convergence of sequences generated by the extrapolated alternating projection
method (5.7) with the step size function k WD CS1 , k 0, was proved in [76,
Theorem 15 (i)].
Now we consider the case, where A H is a closed affine subspace. Let
Q 2
.kT x PB xk ı/
CS 2 .x/ WD 1 C
kT x xk2
Q
for x … Fix T , where ı.x/ N
2 Œı; ı.x/. It is clear that CS 2 .x/ 1.

kD0 be generated by recurrence
(5.7), where
1 k .x/ CS 2 .x/
and lim inf k .2 k / > 0. Then x k converges weakly to a point x 2 Fix PA PB .
Proof. The corollary can be proved similarly to Corollary 5.2.2, where one should
apply Theorem 4.3.16 instead of Theorem 4.3.11. t
u
The convergence of sequences generated by the extrapolated alternating projection
method (5.7) with the step size function k WD CS 2 , k 0,was proved in [76,
Theorem 15 (ii)].
In [75, 306] the problem of finding a fixed point of the alternating projection
PA PB , derived from an inconsistent system of linear equations is considered and
incomplete projection methods for finding Fix PA PB are proposed, where PA is
replaced by an ‘incomplete projection’ which approximates PA in some sense.
5.2.3 Douglas–Rachford Algorithm
Let A; B H be closed convex subsets. The Douglas–Rachford algorithm (DR) is

an iterative method for finding a common point of A and B, if such a point exists.
One iteration of the method has the form
1
x kC1 D .RA RB x k C x k /, (5.9)
2
where RA WD 2PA Id and RB WD 2PB Id are reflection operators onto A and B,
respectively. The iteration can also be written in the form x kC1 D T x k , where T W
H ! H is the averaged alternating reflection (AAR), i.e., T WD 12 .RA RB C Id/ (see
Sect. 4.3.5). The method is also called an averaged alternating reflection algorithm.
The method was introduced by Douglas and Rachford in order to solve a partial
differential equation describing the heat conduction problem and was studied in
[26–29, 157, 161, 246], where various applications of the method were presented.
Consider the following iteration
k
x kC1 D x k C .RA RB x k x k /, (5.10)
2
where k 2 Œ0; 2. If k D 1, then we obtain (5.9). We can also write x kC1 D Tk x k ,
where T denotes a -relaxation of the AAR operator T . The following corollary
extends the result of [26, Fact 5.9].
Corollary 5.2.4. Let A; B H be closed convex. If Fix.RA RB / ¤ ; and
lim infk k .2 k / > 0, then for any x 0 2 H the sequence fx k g1
kD0 generated
by (5.10) converges weakly to a fixed point of RA RB and the sequence fPB x k g1
kD0
is bounded. Furthermore, if A \ B ¤ ;, then:
5.3 Projected Gradient Method 213
(i) For any weak cluster point y of fPB x k g1

kD0 it holds that y 2 A \ B.
(ii) If dim H < 1, then y WD limk PB x k exists and y 2 A \ B.
Proof. By Corollary 4.3.17 (iv), the operator T WD 12 .RA RB C Id/ is firmly
nonexpansive. Therefore, Corollary 3.7.3 with X D H yields x k * x 2 Fix T .
The boundedness of fPB x k g1 k 1
kD0 follows from the boundedness of fx gkD0 and from
the nonexpansivity of PB . Let A \ B ¤ ; and z 2 A \ B. Then Corollary 4.3.17 (i)
and (iv) yields z 2 Fix T and
kC1 2 2 2 2
x z D T x k z x k z k .2 k / T x k x k .

Consequently, x k z converges and limk T x k x k D 0, by lim infk k .2
k / > 0. We have
1
limŒPA .2PB x k x k / PB x k D lim .RA RB x k x k / D lim.T x k x k / D 0.
k k 2 k
(i) Let y be a weak cluster point of fPB x k g1 kD0 . Because B is weakly closed
(see Theorem 1.1.40) and fPB x k g1 kD0 B, we have y 2 B. We prove that
y 2 A. Suppose y … A and let a subsequence fPB x nk g1 k 1
kD0 fPB x gkD0
converge weakly to y. By the boundedness of fPB x gkD0 and of fx k g1
k 1
kD0
and by the nonexpansivity of PA , the sequence fPA .2PB x nk x nk /g1 kD0 is
bounded, consequently, it contains a subsequence fPA .2PB x mk x mk /g1 kD0
which converges weakly to a point z 2 H. Since A is weakly closed, z 2 A.
Because fPA .2PB x k x k / PB x k g1
kD0 converges to 0, it converges weakly to
0 and fPA .2PB x mk x mk / PB x mk g1kD0 also converges weakly to 0. On the
other hand, fPA .2PB x mk x mk / PB x mk g1
kD0 converges weakly to z y ¤ 0.
A contradiction shows that y 2 A.
(ii) Let dim H < 1. The first part of the corollary and the nonexpansivity of PB
yield
lim PB x k PB x lim x k x D 0,
k k
for some x 2 Fix RA RB , i.e., limk PB x D PB x D y. By Corollary 4.3.17

k
(iii), y 2 A \ B. t
u
5.3 Projected Gradient Method
Several optimization problems can be reduced to a variational inequality problem,

e.g., finding the metric projection onto a nonempty closed convex subset of a Hilbert
space, convex minimization problem, elliptic control problem, etc. In this section
we present the most popular method for solving VIP.F ; C / where F W H ! H is
a Lipschitz continuous and strongly monotone operator and C H is a nonempty
closed convex subset. The method is called a projected gradient method and has
the form
x kC1 D PC .x k F x k /. (5.11)
The convergence of sequences generated by the method follows from the theorem
below, which shows an application of the Banach fixed point theorem and the
properties of the metric projection.
Theorem 5.3.1. Let C H be closed convex, F W H ! H be -Lipschitz
continuous and -strongly monotone over C , where ; > 0, and 2 .0; 2 2
/. Then
the operator T WD PC .Id F / is a contraction. Consequently, for any x 0 2 H, the
sequence fx k g1kD0 defined by (5.11) converges to the unique solution of VIP.F ; C /
with a rate of the geometric progression.
Proof. (cf. [358, Theorem 46.C]) Define G D Id F . Let x; y 2 H. Then
kGx Gyk2 D k.x y/ .F x F y/k2

D kx yk2 C 2 kF x F yk 2hF x F y; x yi
.1 C 2 2 2/ kx yk2
D .1 /2 kx yk2 ,
p
where WD 1 C 2 2 2 (note that is well defined, because and
2 .0; 2
2
/, and that 2 .0; 1/). Consequently,
kGx Gyk .1 / kx yk . (5.12)
Since PC is nonexpansive, (5.12) yields
kPC .x F x/ PC .y F y/k D kPC Gx PC Gyk

.1 / kx yk ,
i.e., PC .Id F / is a .1 /-contraction. The Banach fixed point theorem and
Theorem 1.3.8 yield that PC .Id F / has a unique fixed point xN 2 C and xN is
the unique solution of VIP.F ; C /. Furthermore, the Banach fixed point theorem
yields that for any x 0 2 H, the sequence fx k g1
kD0 generated by (5.11) converges
N
geometrically to x. t
u
In the literature one can find several method for solving VIP.F ; C / in particular
for solving VIP.F ; Fix T /, where T W H ! H is a quasi-nonexpansive operator, for
solving convex constrained minimization problems, for finding the metric projection
onto the intersection of a finite family of convex subsets, for solving elliptic control
problems, etc. For details we send the reader to [18,52,77,94–97,101,144,171,200,
203,211,216–218,237,244,250,251,269,273,274,276,318,320,338,342,343,345–
349, 353].
5.4 Simultaneous Projection Method 215
5.4 Simultaneous Projection Method
Let fCi gi 2I H be a finite family of nonempty closed convex subsets, where

I WD f1; 2; T: : : ; mg. We consider the convex feasibility problem of finding a point
x 2 C WD i 2I Ci , ifPsuch a point exists. The proximity function f W H ! RC is
defined by f .x/ WD 12 i 2I d 2 .x; Ci /. The simultaneous projection method (SPM)
for solving this problem has the following form
x0 2 H – arbitrary
P (5.13)
x kC1
D x C k . i 2I !i PCi x x /,
k k k
where fk g1 kD0 Œ0; 2 is a sequence of relaxation parameters and w D

.!1 ; : : : ; !m / 2 P
ri
m is a vector of weights. We can also write x kC1 D Tk x k ,
where T WD i 2I !i PCi is the operator of a simultaneous projection and
T WD Id C.T Id/ denotes a relaxation of T . Recall that
Fix T D Fix T D Argmin f .x/,

x2H
where > 0 and f W H ! R is a proximity function defined by the equality
1X
f .x/ WD !i kPCi x xk2 (5.14)
2 i 2I
(see Theorem 4.4.6) and that Fix T D C if C ¤ ; (see Corollary 4.4.2).
5.4.1 Convergence of the SPM
The convergence of sequences fx k g1

kD0 generated by the simultaneous projection
method (5.13) follows from the following result which is due to Combettes [117,
Theorem 4].
Corollary 5.4.1. Let fx k g1
kD0 H be a sequence generated by the simultaneous
projection method (5.13), where lim infk k .2 k / > 0. If Fix T ¤ ;, then x k
converges weakly to a point x 2 Fix T and the following estimation holds
X
k
2
d 2 .x kC1 ; Fix T / d 2 .x 0 ; Fix T / l .2 l / T x l x l , (5.15)
lD0
T
k 0. If, furthermore, C WD i 2I Ci ¤ ;, then
X
k X 2
d 2 .x kC1 ; C / d 2 .x 0 ; C / l .2 l / !i PCi x l x l , (5.16)
lD0 i 2I
k 0.
Proof. Let Fix T ¤ ;. The operator T is firmly nonexpansive as a convex combi-
nation of firmly nonexpansive operators PCi (see Corollary 2.2.20). Furthermore,
T is nonexpansive (see Theorem 2.2.4) and a cutter (see Theorem 2.2.5 (i)).
Consequently, T Id is demi-closed at 0 (see Lemma 3.2.5). If we take X D H,
Tk WD T and S WD T in Corollary 3.7.1, then we obtain that x k converges weakly
to a point x 2 Fix T and that
lC1 2 2 2 2
x z D T x l z x l z l .2 l / T x l x l ,
l 0. Applying this inequality .k C 1/-times for l D 0; 1; : : : ; k and for z D

PFix T x 0 , we obtain
2
d 2 .x kC1 ; Fix T / x kC1 z
2 2 Xk
2
D T x k z x 0 z l .2 l / T x l x l
lD0
X
k
2
D d 2 .x 0 ; Fix T / l .2 l / T x l x l .
lD0
If C ¤ ;, then estimation (5.16) can be proved in the same way as above applying
Theorem 4.4.5. t
u
If Ci H are closed subspaces, then the convergence in Corollary 5.4.1 is strong
and x D PC x 0 . This result is due to Reich [295, Theorem 1.7], where the
convergence was proved for uniformly convex Banach spaces.
The strong convergence of sequences generated by the simultaneous projection
method does not hold in general. Bauschke, Matoušková and Reich, relying on a
result of Hundal [213, Theorem 1], gave an example of two closed convex subsets
C1 ; C2 H with a nonempty intersection, a simultaneous projection T WD 12 PC1 C
1 k
2 PC2 and a point x for which T x converges weakly but does not converge strongly
[32, Theorem 5.1].
If H D Rn and Ci are hyperplanes or half-spaces, i.e., Ci WD H.ai ; ˇi / D fx 2
R W ai> x D ˇi g or Ci WD H .ai ; ˇi / D fx 2 Rn W ai> x ˇi g, then it follows from
n
equalities (4.63) and (4.70) that the simultaneous projection method (5.13) can be
written in the matrix form
x kC1 D x k k A> D.Ax k b/ (5.17)

5.4 Simultaneous Projection Method 217
or
x kC1 D x k k A> D.Ax k b/C , (5.18)
respectively, where D WD diag. !1 2 ; !2 2 ; : : : ; !m 2 /.
ka1 k ka2 k kam k
The simultaneous projection method (5.13) was introduced by Cimmino in [116],
where H D Rn , Ci WD H.ai ; ˇi /, i 2 I , C ¤ ; and k D 2 for all k 0. It follows
from (5.17) that Cimmino’s method has the form
x kC1 D x k 2A> D.Ax k b/. (5.19)
Cimmino proved the convergence of sequences generated by (5.19) to a solution of

the system Ax D b for a nonsingular nn matrix A. Special cases of method (5.13)
were studied by Auslender [12], De Pierro and Iusem [134, 135, 215], Pierra [284],
Combettes [117]. In particular, the (weak) convergence of the sequences generated
by recurrence (5.13) to a solution of the CFP was proved by:
(a) Auslender in [12], where H D Rn , C ¤ ;, !i D m1 , i 2 I , and k D 1,
(b) Pierra in [284, Theorem 1.1 (i)], where C ¤ ;, !i D m1 , i 2 I , and k D 1 for
all k 0,
(c) De Pierro and Iusem in [134, Theorem 3], where H D Rn , Ci WD H .ai ; ˇi /,
i 2 I , Fix T ¤ ; and k D 2 .0; 2/ for all k 0,
(d) Iusem and De Pierro in [215, Theorem 2], where H D Rn , and k D 1 for all
k 0,
(e) Combettes in [117, Theorem 4], where k 2 Œ"; 2 " for some " 2 .0; 1/.
5.4.2 Projected Simultaneous Projection Methods
In this section we consider the general

T convex feasibility problem of finding a point
x 2 X which is closest to C WD i 2I Ci , where Ci H are nonempty closed
convex subsets, i 2 I WD f1; 2; : : : mg. If X \ C ¤ ;, then the problem is to
find x 2 X \ C . The proximity function f W X ! RC is defined by f .x/ WD
1 P 2
2 i 2I d .x; Ci /. The projected simultaneous projection method has the following
form
x0 2 X – arbitrary
P (5.20)
x kC1 D PX .x k C k . i 2I !i PCi x k x k //,
where fk g1 kD0 Œ0; 2 is a sequence of relaxation parameters and w D
.!1 ; : : : ; !m / 2
m is a vector of weights. If X D H, then method (5.20)
is equivalent to the simultaneous projection
P method (5.13). We can also write
x kC1 D PX Tk x k , where T WD i 2I !i PC i is the operator of simultaneous
projection.
kD0 X be a sequence generated by the projected
simultaneous projection method (5.20), where k 2 Œ"; 2 " for some " 2 .0; 1/.
P
If Fix PX T ¤ ;, where T WD i 2I !i PCi , then x k converges weakly to a point
x 2 Fix PX T .
Proof. T is firmly nonexpansive (see Corollary 4.4.4). If we set Sk WD T , k 0,
in Corollary 3.7.4, then we obtain the weak convergence of x k to a point x 2
Fix PX T . t
u
Consider now the following recurrence
x kC1 D PX .x k C k .PX Ux k x k //, (5.21)

P
where x 0 2 H, U WD i 2I !i PCi , w D .!1 ; : : : ; !m / 2
m and k > 0, k 0.
Note the difference between (5.21) and (5.20).
kD0 X be a sequence generated by method (5.21),
where k 2 Œ"; 32 " for some " 2 .0; 34 /. If Fix PX U ¤ ;, then x k converges
weakly to a point x 2 Fix PX U .
Proof. U is firmly nonexpansive (see Corollary 4.4.4). This fact together with the
firm nonexpansivity of PX yield that PX U is 43 -relaxed firmly nonexpansive and
that .PX U / 3 is firmly nonexpansive (see Corollary 2.2.39). Recurrence (5.21) can
4
be written in the form
x kC1 D PX .x k C k .T x k x k //,
where k D 43 k , T WD .PX U / 3 D Id C 34 .PX U Id/. Note that k 2 Œ"0 ; 2 "0

4
where "0 D 43 " 2 .0; 1/. Now, the weak convergence of x k to a point x 2 Fix T D
Fix PX U follows from Corollary 3.7.3. t
u
5.5 Cyclic Projection Methods
Let fCi gi 2I H be a finite family of nonempty closed convex subsets, where

I WD f1;
T 2; : : : ; mg. Consider the convex feasibility problem of finding a point x 2
C WD i 2I Ci , if such a point exists. The cyclic projection method for solving this
problem has the following form
(5.22)
x kC1 D PCm : : : PC1 x k .
If we replace the operators PCi in (5.22) by their relaxations PCi ;i , where i 2
.0; 2/, i 2 I , then we obtain the following, more general method
5.5 Cyclic Projection Methods 219
(5.23)
x kC1
D PCm ;m : : : PC1 ;1 x ,
k
which we call the cyclic relaxed projection method. We can also write x kC1 D T x k ,
where T WD PCm ;m : : : PC1 ;1 is the cyclic relaxed projection. Denote S0 WD Id and
Si WD PCi ;i : : : PC1 ;1 for i D 1; 2; : : : ; m. We have Sm D T .
5.5.1 Convergence

kD0 H be a sequence generated by the cyclic relaxed
projection method (5.23), where i 2 .0; 2/, i 2 I . If Fix T ¤ ;, then x k converges
weakly to a point x 2 Fix T and the following estimation holds
X
k
2
d 2 .x kC1 ; Fix T / d 2 .x 0 ; Fix T / ˛ T x l x l , (5.24)
lD0
T
where ˛ D 2 max
mmax
and max WD maxfi W i 2 I g. If C WD i 2I Ci ¤ ; and i D 1
for all i 2 I , then
X
k X

d 2 .x kC1 ; C / d 2 .x 0 ; C / Si x l Si 1 x l 2 . (5.25)
lD0 i 2I
Proof. Let Fix T ¤ ;. It follows from Corollary 4.5.8 that T is -RFNE, where D
2mmax
.m1/max C2
and that T is asymptotically regular. Since T is nonexpansive, it follows
from Theorem 3.5.3 that x k converges weakly to x 2 Fix T . Corollary 2.2.9 yields
the ˛-strong quasi nonexpansivity of T , where ˛ WD 2
D 2 max
mmax
. Therefore, T is
˛-SQNE, i.e.,
l
T x z2 x l z2 ˛ T x l x l 2
for all z 2 Fix T , l D 0; 1; : : : If we take z D PFix T x 0 and apply the above inequality
for l D 0; 1; : : : ; k, we obtain
2
d 2 .x kC1 ; Fix T / x kC1 z
2 2 Xk
2
D T x k z x 0 z ˛ T x i x i
i D0
X
k
2
D d 2 .x 0 ; Fix T / ˛ T x i x i .
i D0
If C ¤ ;, then we obtain estimation (5.25) in the same way as above if we apply

inequality (4.74). t
u
The weak convergence of sequence generated by the cyclic projection method was
proved by Bregman [42, Theorem 1]. Sufficient conditions for strong convergence as
well as for geometric convergence were given by Gurin et al. [196]. Other conditions
for the strong convergence can be found in [14]. De Pierro and Iusem considered the
method for a inconsistent convex feasibility problem in Rn (see [137]) and studied
the convergence of sequences generated by the method. The rate of convergence for
the method in a Hilbert space was studied in [142, 143, 290, 291]. The paper [23]
contains a review of the results on the cyclic projection method in a Hilbert space.
The cyclic projection method for a consistent system of n linear equations in Rn
was introduced by Stefan Kaczmarz [223]. Therefore, the cyclic projection method
is also known as the Kaczmarz method. Kaczmarz supposed that the equations are
independent or, equivalently, that the solution is uniquely defined and proved the
convergence to the solution of sequences generated by the method [223]. Altman
proved the equivalence of the method with the Gauss–Seidel method for the system
A> Ax D A> b [7, Corollary in Sect. 2]. Halperin considered the cyclic projection
method for a finite system of subspaces
Vi H, i 2 I , and proved that for
any x 2 H it holds that limk!1 .PVm PVm1 : : : PV1 /k x PV x D 0 (see [198,
Theorem 2]). Note that this generalizes the von Neumann Theorem 5.1.5.
Gordon, Bender and Herman proposed a method for solving a system of linear
equations [189] which is equivalent to the Kaczmarz method, and called it an
algebraic reconstruction technique (ART). Tanabe proved the convergence for an
arbitrary system of linear equations in Rn [321, Corollary 9]. The Kaczmarz method
in Hilbert space was studied by McCormick in [259]. Since 1970 the Kaczmarz
method has been applied to computerized tomography and in radiation therapy (see,
e.g., [81, 82, 100, 189]).
Further properties of the Kaczmarz method as well their modifications can be
found in [9, 13, 34, 41, 81, 86, 129–132, 162, 197, 201, 239, 247, 268, 285–287].
5.5.2 Projection-Reflection Method
Let Ci H be closed convex subsets, i 2 I , and K TH be a closed convex and

obtuse cone, i.e., K K. Suppose that C WD K \ i 2I Ci ¤ ;. Consider the
following CFP: find x 2 C . Denote
P D PCm PCm1 : : : PC1 ,
P WD Id C.P Id/, where 2 .0; 2/, and RK WD .PK /2 D 2PK Id, i.e., P
is a relaxation of the cyclic projection P and RK is the reflection operator onto K.
Consider the iterative method defined by the following recurrence
5.5 Cyclic Projection Methods 221
Fig. 5.1 Projection-

K
reflection method and
k k
reflection-projection method yk x =RK y
xk+1=RK Pµxk
C
yk+1=Pµ RK y k
C1
x kC1 D RK P x k , (5.26)
where x 0 2 H. We can also write x kC1 D R x k , where R WD RK P is a -relaxed

projection-reflection operator (see Definition 4.5.11). We call the method a
projection-reflection method (see Fig. 5.1). Since K is obtuse, x k 2 K for all k 1
(see Lemma 4.5.12). Therefore, without loss of generality, we can suppose that
x 0 2 K and that R WD RK P in (5.26) is restricted to K.
Corollary 5.5.2. Let K H be a closed T convex and obtuse cone, Ci 0 H be
closed convex subsets, i 2 I , C WD K \ i 2I Ci ¤ ;, 2 .0; mC1
m / and x 2 K.
Then the sequence generated by the projection-reflection method (5.26) converges
weakly to a point x 2 C .
Proof. The operator R jK is nonexpansive and strongly quasi-nonexpansive (see
Corollary 4.5.13 (i) and (iii)). Consequently R jK is asymptotically regular (see
Theorem 3.4.3). By Corollary 4.5.13 (ii), we have Fix R D C . Let fx k g1 kD0 be
generated by the projection-reflection method (5.26). We see that all assumptions of
Opial’s Theorem 3.5.1 are satisfied. Therefore, x k converges weakly to x 2 C . u
t
Consider now sequences fy k g1
kD0 generated by the recurrence
y kC1 D P RK y k (5.27)
with y 0 2 H. We call the method a reflection-projection method. The method is

closely related to the projection-reflection method (5.26). If we take x 0 WD RK y 0 ,
we obtain that the sequence fx k g1
kD0 defined by x WD RK y is generated by (5.26).
k k
Furthermore, y kC1
D P x (see Fig. 5.1).
k
Bauschke and Kruk considered the reflection-projection method (5.27) in

H D Rn with D 1 [31] and proved that the sequences generated by the method
converge to a point x 2 C (see [31, Theorem 3.1]). The following corollary
generalizes this result.
Corollary 5.5.3. Let K H be a closed T convex and obtuse cone, Ci 0 H be
closed convex subsets, i 2 I , C WD K \ i 2I Ci ¤ ;, 2 .0; mC1
m
/ and y 2 H.
Then the sequence generated by the reflection-projection method (5.27) converges
weakly to a point y 2 C .
Proof. Let fy k g1 k 1
kD0 be generated by (5.27), x WD RK y and fx gkD0 be generated
0 0
by the projection-reflection method (5.26). We have x D RK y and y kC1 D P x k .

k k
By Corollary 4.5.13 (i) and (ii), P RK is nonexpansive and Fix P RK D C .

The operator R is strongly quasi-nonexpansive (see Corollary 4.5.13 (iii)),

consequently R is asymptotically regular (see Theorem 3.4.3). This fact together
with the nonexpansivity of P yield
kC1
y y k D P x k P x k1 x k x k1

D Rk x 0 Rk1 x 0 ! 0.
Therefore,

.P RK /kC1 y 0 .P RK /k y 0 D y kC1 y k ! 0,
i.e., P RK is asymptotically regular. We see that all assumptions of Opial’s

Theorem 3.5.1 are satisfied. Therefore, y k converges weakly to a point y 2 C . u
t
5.6 Successive Projection Methods
Let fCi gi 2I H be a finite family of nonempty closed convex subsets Ci , i 2 I WD

f1; 2; : : : ; mg, with a common
T point. We consider the convex feasibility problem of
finding a point x 2 C WD i 2I Ci , if such a point exists. The successive projection
method or sequential projection method for solving this problem has the following
form
(5.28)
x kC1 D x k C k .PCik x k x k /,
where fk g1 1
kD0 Œ0; 2 is a sequence of relaxation parameters and fik gkD0 I
is a control sequence or, shortly, control. Note that ik can depend on the current
approximation x k 2 H. In this case we identify ik with ik .x k /. It is convenient
to define ik .x/ for any x 2 H, i.e., to define a sequence of functions fik g1 kD0 W
H ! I . We will, however, relate this sequence of functions to recurrence (5.28).
If the sequence of functions fik g1
kD0 is constant, i.e., ik D i for all k 0, then the
function i W H ! I is called a control function or, shortly, control. In this case we
write ik D i.x k /. A comprehensive overview of the sequential projection methods
can be found in [78] and in [108, Chap. 5].
5.6.1 Convergence
The convergence of sequences generated by (5.28) depends on the properties of the

control sequence fik g1
kD0 . The following definition is due to Gurin et al. (see [196,
Sect. 1]).
5.6 Successive Projection Methods 223
Definition 5.6.1. A control sequence fik g1kD0 W H ! I is called an approximately

remotest set control if for any sequence fx k g1
kD0 generated by (5.28) the following
implication holds

lim PCik x k x k D 0 H) lim max PCi x k x k D 0. (5.29)
k k i 2I
Definition 5.6.2. A control sequence fik g1

kD0 W H ! I is called an approximately
semi-remotest set control if for any sequence fx k g1
kD0 generated by (5.28) the
following implication holds

lim PCik x k x k D 0 H) lim inf PCi x k x k D 0 for all i 2 I . (5.30)
k k
Remark 5.6.3. Since I is a finite subset, the equality on the right hand side of
(5.29) can be written in the following equivalent form: limk PCi x k x k D 0 for
all i 2 I . Therefore, implication (5.29) is equivalent to the following one

lim PCik x k x k D 0 H) lim PCi x k x k D 0 for all i 2 I .
k k
Corollary 5.6.4. Let C ¤ ; and fx k g1 kD0 H be a sequence generated by the

successive projection method (5.28), where lim infk k .2 k / > 0.
(i) If fik g1 k
kD0 is an approximately remotest set control, then x converges weakly

to a point x 2 C .
(ii) If H is finite dimensional and fik g1kD0 is an approximately semi-remotest set
control, then x k converges to a point x 2 C .
Proof. Setting X D H, Tk WD PCik and S WD PCi in Corollary 3.7.1, we obtain
x k * x 2 Fix PCi D Ci , i 2 I , in (i) and x k ! x 2 Fix PCi D Ci , i 2 I ,
in (ii). t
u
5.6.2 Control Sequences
Below we present some control functions or control sequences which satisfy

implication (5.29) or at least (5.30). It follows from Corollary 5.6.4 that the
successive projection methods with such controls and with relaxation parameters
k 2 Œ"; 2 " for some " 2 .0; 1/ generate sequences converging, at least weakly,
to a point x 2 C if C ¤ ;.
Definition 5.6.5. A control function i W H ! I is called a remotest set control if

i.x/ D argmax PCj x x .
j 2I
Definition 5.6.6. A control sequence fik g1 kD0 W H ! I is called an almost remotest

set control if there exists a constant ˛ 2 .0; 1 such that

PCik x k x k ˛ max PCj x k x k .
j 2I
Note that a remotest set control is an almost remotest set control. Furthermore,
an almost remotest set control is approximately remotest set control.
Definition 5.6.7. Let Cj WD fx 2 H W haj ; xi ˇj g, j 2 I , be half-spaces, where
aj 2 H, aj ¤ 0, and ˇj 2 R , j 2 I . A control i W H ! I is called a maximal
residual control if
i.x/ D argmax.haj ; xi ˇj /.
j 2I

Note that if aj D 1, j 2 I , then a maximal residual control is a remotest set
control.
Lemma 5.6.8. Let Cj D fx 2 H W haj ; xi ˇj g, j 2 I , be half-spaces, where
aj 2 H, aj ¤ 0, and ˇj 2 R , j 2 I . The maximal residual control is an almost
remotest set control. Consequently, it is an approximately remotest set control.

Proof. Let ˛ WD minj 2I aj , ˇ WD maxj 2I aj and i W H ! I be a maximal
residual control. We have
haj ; xi ˇj
hai.x/ ; xi ˇi.x/ D max.haj ; xi ˇj / D max.aj /
j 2I j 2I aj
haj ; xi ˇj
˛ max D ˛ max PCj x x
j 2I aj j 2I
T
for any x … j 2I Cj . Therefore,
hai.x k / ; x k i ˇi.x k /
˛
PCi.xk / x k x k D max PCj x x
ai.x k / ˇ j 2I
and the control function i W H ! I is an approximately remotest set control. t

u
Definition 5.6.9. We say that a control sequence fik g1 kD0 I is cyclic if for all
k 0 it holds
fik ; ikC1 ; : : : ; ikCm1 g D I .
Without loss of generality we can suppose that the cyclic control sequence has
the form ik D k.mod m/ C 1, k 0.
Note that a successive projection method with the cyclic control is closely related
to the cyclic projection method in the following sense. Let fx k g1
kD0 be generated by
the cyclic projection method and fy k g1

kD0 be generated by the successive projection
method with the cyclic control. If x 0 D y 0 , then x k D y km for all k 2 N.
The following definition generalizes the notion of the cyclic control (cf. [243],
[244, Definition 2.3] and [78, Definition 3.4]).
Definition 5.6.10. We say that a control sequence fik g1
kD0 is almost cyclic if
there exists a constant s m (called an almost cyclicality constant) such that
I fik ; ikC1 ; : : : ; ikCs1 g for all k 0.
An almost cyclic control is also called quasi-periodic (see, e.g., [8,153]). A more
general type of control, sometimes called quasi-cyclic is studied in [6, 331]. The
following definition generalizes the notion of an almost cyclic control (cf. [2, 8,
153]).
Definition 5.6.11. We say that a control sequence fik g1 kD0 is repetitive if for any
j 2 I and for any k 0 there exists l k such that il D j .
Various variants of successive projection methods with repetitive control were
studied, e.g., in [2, 70, 111, 164]. In [164, Sect. 3] the repetitive control was called
admissible.
Note that a cyclic control sequence fik g1 kD0 is m-almost cyclic and an almost
cyclic control sequence with the almost cyclicality constant s D m is cyclic.
Furthermore, any almost cyclic control sequence is repetitive. In the sequel we
will apply a cyclic control, almost cyclic control and repetitive control also in more
general recurrences than (5.28), where the projections operators will be replaced by
cutters.
In the next sections we present results which are stronger than the following one
(see Theorems 5.8.11, 5.8.14, 5.9.3 and 5.9.4).
Lemma 5.6.12. Let fx k g1
kD0 H be generated by the successive projection
method (5.28).
(i) If fik g1
kD0 is almost cyclic, then it is an approximately remotest set control.
(ii) If H is finite dimensional, C ¤ ;, lim infk k .2 k / > 0 and fik g1 kD0 is
repetitive, then it is an approximately semi-remotest set control.
Proof. (i) Let the sequence fik g1
kD0 be almost cyclic with s m being an almost
cyclicality constant. Suppose that

lim PCik x k x k D 0. (5.31)
k
Let i 2 I and rk 2 f0; 1; : : : ; s 1g be such that i D ikCrk , k 0. By the

triangle inequality, we have
k 1
kCr rX kClC1
x k x k x x kCl
lD0
k 1
rX

D kCl PCikCl x kCl x kCl
lD0
X
s2

kCl PCikCl x kCl x kCl ,
lD0
consequently,
lim x kCrk x k D 0. (5.32)
k
The definition of the metric projection and the triangle inequality yield,

PC x k x k PC x kCrk x k
i i

PCi x kCrk x kCrk C x kCrk x k

D PCikCr x kCrk x kCrk C x kCrk x k .
k

Therefore, (5.31) and (5.32) yield limk PCi x k x k D 0, i.e., fik g1
kD0 is
approximately regular.
(ii) Let H be finite dimensional, C ¤ ;, lim infk k .2 k / > 0 and fik g1 kD0 be
repetitive. Since (5.28) is a special case of (3.9) with X D H and Tk WD PCik ,
by Corollary 3.7.1, we obtain
kC1 2 2 2
x
z x k z k .2 k / PCik x k x k
for any z 2 C . Therefore, lim infk k .2 k / > 0 yields

lim PCik x k x k D 0.
k
Let i 2 I . Since fik g1 1 1

kD0 is repetitive, there is a subsequence fnk gkD0 fkgkD0
such that ink D i , k 0. Now we have

lim inf PCi x k x k D lim inf PCin x k x k D lim PCik x k x k D 0,
k k k k
i.e., fik g1
kD0 is an approximately semi-remotest set control sequence. t
u
Since the cyclic control is almost cyclic, it is an approximately remotest set
control. Therefore, Corollary 5.6.4 yields that the sequences generated by the cyclic
projection method with lim infk k .2 k / > 0 converge (at least weakly) to a point
x 2 C if C ¤ ;.
5.6.3 Examples
Example 5.6.13. Consider the successive projection method (5.28) with the cyclic
control fik g1
kD0 and with k 2 Œ"; 2 " for some " 2 .0; 1/. The method (also called
a cyclic projection method) was studied by:
(i) Kaczmarz in [223], where H D Rn , Ci WD fx 2 Rn W hai ; xi D ˇi g, i D
1; 2; : : : ; n, ai 2 Rn are linearly independent and k D 1 for all k 0.
Kaczmarz proved that for any starting point x 0 2 Rn the sequence fx k g1 kD0
generated by the method converges to a unique solution of the system hai ; xi D
ˇi , i D 1; 2; : : : ; n;
(ii) Tanabe in [321], where H D Rn , Ci WD fx 2 Rn W hai ; xi D ˇi g, i 2 I , (with
possibly an empty intersection) and k D 1 for all k 0. Tanabe proved the
convergence of any sequence generated by the method (see [321, Corollary 9]);
(iii) Bregman in [42], where Ci H are closed convex subsets with C ¤ ; and
k D 1 for all k 0. Bregman proved that any sequence generated by the
method converges weakly to a point x 2 C (see [42, Theorem 1]);
(iv) CensorTet al. in [86], where H D Rn , Ci D fx 2 Rn W hai ; xi D ˇi g, i 2 I ,
C D i 2I Ci D ; and k D 2 .0; 2/. Let x 0 2 Rn be arbitrary, x k be
generated by the successive projection method with cyclic control and x ./ D
limk x km . Censor et al. proved that if # 0, then x ./ converges to a least
squares solution (see [86, Theorem 1]). Small relaxation parameters for the
Kaczmarz method were also used in [201];
(v) Gurin et al. in [196, Theorem 1], where Ci H, i 2 I , are closed convex
subsets with C ¤ ; and k 2 Œ"; 2 " for some " 2 .0; 1/ and for all
k 0. Gurin et al. proved the strong convergence of sequences generated by
the method if the subsets Ci , i 2 I , have a special structure (e.g., they are
half-spaces or are uniformly convex or C has a nonempty interior).
Example 5.6.14. Let C ¤ ;. Consider the successive projection method (5.28)
with the approximately remotest set control and with k 2 Œ"; 2 " for some " 2
.0; 1/. Corollary 5.6.4 yields x k * x 2 C .
(i) The convergence properties of the successive projection method with the
remotest set control were studied by Agmon in [1] and by Motzkin and
Schoenberg in [266], where H D Rn , Ci D fx 2 Rn W hai ; xi ˇi g,
k D 2 .0; 2/. Agmon proved that the sequences generated by the method
converge linearly [1, Theorem 3]. Motzkin and Schoenberg also considered the
case D 2 if C has a nonempty interior and proved the finite convergence in
this case.
(ii) Bregman in [42] considered the convex feasibility problem with infinitely
many subsets Ci H, i 2 I , having a nonempty intersection, and studied
successive projection methods for this problem. In [42, Theorem 2] the
convergence x k * x 2 C was proved for the remotest set control (the
existence of such control was supposed). Bregman also considered a special
case of the problem with infinitely many half-spaces Ci H, Ci WD fx 2 H W
hai ; xi ˇi g, i 2 I , where fai W i 2 I g is bounded. In [42, Theorem 3] the

convergence x k * x 2 C was proved for the maximal residual control (the
existence of such control was supposed).
(iii) Goffin in [186] considered the linear feasibility problem in Rn and studied
sufficient conditions for finite convergence of the sequential projection method
with the remotest set control [186, Theorems 7.1 and 7.5].
(iv) Gurin et al. in [196] considered the convex feasibility problem with infinitely
many subsets Ci H, i 2 I , having a nonempty intersection. In [196,
Theorems 1 and 2] the convergence x k ! x 2 C was proved for an
approximately remotest set control, where Ci satisfy some conditions, e.g.,
Ci , i 2 I , are half-spaces or uniformly convex subsets, or the intersection has
an open interior.
(v) Dye and Reich in [152] studied successive projection methods for Ci H,
i 2 I , being one-dimensional subspaces.
5.7 Landweber Method and Projected Landweber Method
Let A W H1 ! H2 be a nonzero linear bounded operator and Q H2 be a nonempty

closed convex subset. The Landweber method (LM) for finding x 2 H1 satisfying
Ax 2 Q (if such an x exists) is defined by the recurrence
k
x kC1 D x k C A .PQ .Ax k / Ax k /,
kAk2
where x 0 2 H1 is arbitrary and k 2 Œ0; 2. We can also write x kC1 D Tk x k .
where T is a -relaxation of the Landweber operator T WD 1 2 A .PQ Id/A.
kAk
Landweber proposed the method with k D 2 .0; 2/, k 0, for solving an
integral equation (see [240]). The projected Landweber method (PLM) is an iterative
method for solving the following split feasibility problem: find x 2 C such that
Ax 2 Q, where C H1 , Q H2 are nonempty closed convex subsets of Hilbert
spaces H1 ; H2 and A W H1 ! H2 is a nonzero linear bounded operator, if such a
point exists. The iterative step of the projected Landweber method has the form
k
x kC1 D PC .x k C A .PQ .Ax k / Ax k //, (5.33)
kAk2
where k 2 Œ0; 2. We can also write x kC1 D PC Tk x k , where T is the -relaxation
of the Landweber operator T or x kC1 D Rk x k , where R WD PC T is the projected
-relaxation of the Landweber operator. It is clear that if C D H1 , then the projected
Landweber method reduces to the Landweber method. The projected Landweber
method was studied by Byrne [55] for H1 and H2 being Euclidean spaces and
for k D 2 .0; 2/, k 0. Actually, Byrne called the method a CQ algorithm
5.7 Landweber Method and Projected Landweber Method 229
and proved the convergence of sequences generated by the method to a solution

of the split feasibility problem, if such a solution exists (see [55, Theorem 2.1]).
Generalized versions of the projected Landweber method were studied by Xu [344]
for a multiple set split feasibility problem and by Zhao and Yang [361].
Theorem 5.7.1. Let the projected Landweber operator defined by (4.92) have a
fixed point. If lim infk k .2 k / > 0, then, for an arbitrary x 0 2 H1 , the sequence
fx k g1
kD0 generated by the projected Landweber method (5.33) converges weakly to
a point z 2 Fix U .
Proof. It follows from Theorem 4.6.3 that the Landweber operator T WD Id C 1
kAk2
A .PQ Id/A is firmly nonexpansive. The weak convergence of x k to a fixed point
of the projected Landweber operator U follows now from Corollary 3.7.4 (iii) with
S D T , X D C and k D k , k 0. t
u
Variants of the projected Landweber method for the consistent split feasibility
problem, i.e., such that C \ A1 .Q/ ¤ ;, were studied in [292, 293, 350], where
projection onto C and Q were replaced by projections onto closed convex sets
Ck C and Qk Q.
Recall that x 2 C is a fixed point of the projected Landweber operator U if
and only if it is a minimizer of the proximity function f W C ! R defined by
2
f .x/ WD 12 PQ .Ax/ Ax (see Proposition 4.7.2). Consequently, Theorem 5.7.1
gives sufficient conditions for the weak convergence of sequences generated by the
projected Landweber method to a solution of the split feasibility problem.
Suppose now that H1 D Rn , H2 D Rm and that A is a nonzero matrix of type
m n. If Q D fbg, then the projected Landweber method for finding a solution
x 2 C of the linear equation Ax D b has the form
k
x kC1 D PC .x k C A> .b Ax k //
max .A> A/
(cf. equality (4.79)). If A has nonzero rows ai 2 Rn , i 2 I , then we can

1 1
write the system Ax D b in an equivalent form D 2 Ax D D 2 b, where D D
diag. !1 2 ; !2 2 ; : : : ; !m 2 / and w D .!1 ; !2 ; : : : ; !m / 2 ri
m . The projected
ka1 k ka2 k kam k
Landweber method for the modified system has the form
k
x kC1 D PC .x k C A> D.b Ax k // (5.34)
max .A> DA/
(cf. (4.82). If Q D fy 2 Rm W y bg, then, similarly as above, the projected

Landweber method for finding a solution x 2 C of the linear inequality Ax b
has the form
k
x kC1 D PC .x k A> .Ax k b/C /
max .A> A/
1
(cf. (4.80)) and the projected Landweber method for the modified system D 2 Ax
1
D 2 b has the form
k
x kC1 D PC .x k A> D.Ax k b/C / (5.35)
max .A> DA/
(cf. (4.83). If we take a positive definite matrix D with nonnegative elements and
C D Rn in (5.35), then we obtain a method which was studied in [87, equality (12)
and Theorem 2] (see also [322], [132, Sect. 4], [98], [220, Theorem 2], [160, 222]
for related results for a system of linear equations). Note that method (5.34) for
1 1
solving the system D 2 Ax D D 2 b is an extrapolation of the simultaneous projection
method (5.17) for solving the system Ax D b (see Corollary 4.6.7). Similarly,
1 1
method (5.35) for solving the system D 2 Ax D 2 b is an extrapolation of the
simultaneous projection method (5.18) for solving the system Ax b, where the
extrapolation parameter (step size) L D .A1> DA/ 1.
max
5.8 Simultaneous Cutter Methods
In this section we consider the common fixed point problem for a finite family of
cutters U D fUi gi 2I , where the operators Ui W H ! H, i 2 I WD f1; 2; : : : ; mg,
have a common fixed point. We study the convergence ofPsequences generated by
relaxations of simultaneous cutters of the form V WD i 2J !i Vi , where V WD
fV
T i g i 2J is a finite
T family of cutters V i W H ! H, i 2 J , with the property
i 2J Fix V i i 2I Fix U i and w W H !
jJ j is an appropriate weight function.
One iteration of the recurrence is defined by
X
x C D V x D x C . !i .x/Vi x x/,
i 2J
where x 2 H is the current approximation of a solution, 2 Œ0; 2 is a relaxation

parameter and w.x/ D .!1 .x/; : : : ; !jJ j .x//. A simple example of such iteration
is the simultaneous projection method applied to the convex feasibility problem. In
this case U D V D fPCi gi 2I , w 2
m is a vector of constant weights and one
iteration of the method has the form
X
x C D U x D x C . !i PCi x x/.
i 2I
In this section we consider, however, more general methods, where in different

iterations different families V WD fVi gi 2J and different weight functions w W H !
jJ j can be applied. We will study the convergence of sequences generated by the

following recurrence
5.8 Simultaneous Cutter Methods 231
P (5.36)
x kC1
D x C k . i 2Jk !i .x /Vi x x /,
k k k k k k
where V k WD fVik gT T finite families of cutters Vi W H ! H, 1

i 2Jk is a sequence of
k
i 2 Jk ,
with the property i 2Jk Fix Vi i 2I Fix Ui ¤ ; for all k 0, fk gkD0
k
Œ0; 2 is a sequence of relaxation parameters, fwk g1 kD0 W H !

mk is a sequence
of appropriate weight functions, mk WD jJk j and wk .x/ D .!1k .x/; :::; !m k
k
.x//.
We call the method defined by (5.36) a simultaneous cutter method (SiCM). An
iteration in (5.36) can also be written in the form
x kC1 D x k C k .V k x k x k /,
P
where V k x WD i 2Jk !i .x/Vi x for x 2 H. It is clear that V is a cutter (see
k k k
Corollary 2.1.49). Iterations of type (5.36) were studied by several authors (see [70]
and the references therein).
5.8.1 Assumptions on Weight Functions
In this section we define some classes of weight functions w W H !

jJ j related to
a family of cutters V WD fVi gi 2J and some classes of sequences of weight functions
fwk g1
kD0 W H !
jJk j related to a sequence of families of cutters V WD fVi gi 2Jk ,
k k
k 1
k 0. It turns out that if we employ sequences fw gkD0 which belong to these
classes, then we can apply convergence results presented in Sects. 3.6 and 3.7
in order to prove the convergence of sequences generated by the simultaneous
cutter method. The assumption that the weight functions wk are appropriate is not
sufficient for the convergence.
Remark 5.8.1. A sequence of weight functions fwk g1
kD0 applied in recurrence
(5.36) can be:
(a) Such that the weights !ik .x k / essentially depend on the iteration’s counter k
and on the current point x k , i 2 I , k 0,
(b) A sequence of constant weight functions; in this case !ik .x k / D !ik , i 2 I ,
k 0,
(c) A constant sequence; in this case !ik .x k / D !i .x k /, i 2 I , k 0,
(d) A constant sequence of constant weight functions; in this case !ik .x k / D !i ,
i 2 I , k 0.
In the practical realization of a simultaneous algorithm there is no difference
between cases (a), (b) and (c) and we can use the notation !ik instead of !ik .x k / in
(a) and instead of !i .x k / in (c). Nevertheless, it is useful to distinguish the weights
which depend on the current point x and the weights which depend on iteration’s
counter. Furthermore, some properties of the simultaneous cutter methods depend
on the properties of corresponding simultaneous cutter operators, where appropriate

weight functions play an important role.
Definition 5.8.2. Let V WD fVi gi 2J be a finite family of cutters Vi W H ! H,
i 2 J , and ˇ > 0 be a constant. We say that a weight function w W H !
jJ j is
ˇ-regular with respect to the family fUi gi 2I , or, shortly, regular if for any x 2 H
there exists j 2 J such that
2
!j .x/ Vj x x ˇ max kUi x xk2 . (5.37)
i 2I
T T
Suppose that i 2J Fix Vi i 2I Fix Ui . Note that a weight function which is
regular with respect to the family U WD fUi gi 2I is appropriate with respect to the
family V WD fVi gi 2J (cf. Definition 2.1.25). It is also clear that if V U, then a
constant weight function w with all positive weights !i , i 2 J , is regular.
Example 5.8.3. Let V WD U, I.x/ WD fi 2 I W x … Fix Ui g and m.x/ WD jI.x/j
be the number of elements of I.x/, where x 2 H. The following weight functions
w W H !
m , where w.x/ D .!1 .x/; !2 .x/; : : : ; !m .x//, x 2 H, are regular:
(i) Positive constant weights, i.e.,
w.x/ WD w 2 ri
m (5.38)
for all x 2 H. To verify that w is regular it suffices to choose j 2 Argmaxi 2I

kUi x xk and take ˇ D mini 2I !i in Definition 5.8.2.
In particular, !i .x/ D m1 , i 2 I , x 2 H is regular.
(ii) Constant weights for violated constraints, i.e.,
(
P i for i 2 I.x/
!i .x/ WD j 2I.x/ j (5.39)
0 for i … I.x/,
i 2 I , x 2 H, where v D .1 ; : : : ; m / 2 ri
m . To verify that w is regular
it suffices to choose j 2 Argmaxi 2I kUi x xk and take ˇ WD mini 2I !i in
Definition 5.8.2. In particular, a weight function ! with
(
1
for i 2 I.x/
!i .x/ D m.x/ (5.40)
0 for i … I.x/,
i 2 I , x 2 H, is regular. The weights of the form (5.39) were applied in

[87, Sect. 2] and [215] for some variants of the simultaneous projection
method.
(iii) Weight functions w W H !
m , where the weights !i .x/ are proportional to
kUi x xk, i.e.,
( kUi xxk T
P for x … Fix Uj
!i .x/ WD kUj xx k
j 2I
T
j 2I
(5.41)
0 for x 2 j 2I Fix Uj ,
i 2 I , x 2 H. To verify that w is regular it suffices to choose j 2

Argmaxi 2I kUi x xk and take ˇ WD m1 in Definition 5.8.2.
(iv) Weight functions w W H !
m satisfying the condition
wi .x/ ı for all i 2 I.x/ (5.42)
x 2 H, for some constant ı 2 .0; m1 . To verify that w is regular it suffices to

choose j.x/ 2 Argmaxi 2I kUi x xk and take ˇ WD ı in Definition 5.8.2. The
weights satisfying (5.42) were applied in [120, Sect. III] and in [119, Sect. 1].
Note that the weight functions defined by (5.38) and by (5.39) satisfy (5.42).
(v) Weight functions w W H !
m which consider only almost violated
constraints, i.e., !i .x/ D 0 for all i … J .x/, where

J .x/ WD fj 2 I W Uj x x max kUi x xkg (5.43)
i 2I
for some 2 .0; 1, i 2 I , x 2 H. To verify that w is regular it suffices

2
to choose j WD j.x/ 2 J .x/ with !j .x/ m1 and take ˇ WD m in
Definition 5.8.2. The Pexistence of such j follows from the fact that !i .x/ 0
for all i 2 J .x/ and i 2J .x/ !i .x/ D 1.
In particular, the following weight functions satisfy (5.43):
(a) (
1 if i D argmaxj 2I Uj x x
!i .x/ WD (5.44)
0 in other cases,
i 2 I , x 2 H, where Ui D PCi for a closed convex subset Ci H, i 2 I .
In this case w defines a remotest set control (cf. Definition 5.6.5).
(b)
1 if i D j.x/
!i .x/ WD (5.45)
0 in other cases,
where j.x/ 2 J .x/ for some 2 .0; 1, i 2 I , x 2 H. If Ui D PCi
for a closed convex subset Ci H, i 2 I , then any sequence of weight
functions fwk g1
kD0 of this form (with independent of k) is an almost
remotest set control (cf. Definition 5.6.6).
The next definitions extend the notion of a regular weight function to a sequence of
weight functions.
Definition 5.8.4. Let V k WD fVik gi 2Jk , be a sequence of families of cutters Vik W
H ! H, i 2 Jk , k 0. We say that a sequence of weight functions wk W H !
jJk j
is regular (with respect to the family U WD fUi gi 2I ), if there exists a sequence
fˇk g1
kD0 .0; C1/ with lim infk ˇk > 0 such that w applied to the family V is
k k
ˇk -regular with respect to the family U, k 0.

It is clear that for a regular weight function w applied to the family U the constant
sequence wk WD w is regular.
Definition 5.8.5. Let V k WD fVik gi 2Jk be a sequence of cutters Vik W H ! H,
i 2 Jk , k 0, and the sequence fx k g1
kD0 be generated by recurrence (5.36). We
say that a sequence of appropriate weight functions wk W H !
jJk j applied to the
sequence of families fV k g1
kD0 is:
(i) Approximately regular (with respect to the family U WD fUi gi 2I ) if there

exists a sequence fik g1
kD0 with ik 2 Jk for all k 0, such that the following
implication holds
2
lim !ikk .x k / Vikk x k x k D 0 H) lim Ui x k x k D 0, for all i 2 I ,
k k
(5.46)
(ii) Approximately semi-regular (with respect to the family U WD fUi gi 2I ) if there
exists a sequence fik g1
kD0 with ik 2 Jk for all k 0, such that the following
implication holds
2
lim !ikk .x k / Vikk x k x k D 0 H) lim inf Ui x k x k D 0, for all i 2 I .
k k
(5.47)
A sequence of weight functions fwk g1 kD0 applied to recurrence (5.36) is also
called a control sequence or a control. If fwk g1
kD0 satisfies (i) or (ii), then we also
say that the control in recurrence (5.36) is approximately regular or approximately
semi-regular.
Remark 5.8.6. Let x k be generated by recurrence (5.36).
(i) T
By Corollary 3.7.1 (i), the sequence
P fx k g1
kD0 is Fejér monotone with respect to
i 2I Fix U i , because V k
WD !
i 2Jk T
k k
V
i i is a cutterT as a convex combination
of cutters (see Corollary 2.1.49) and i 2I Fix Ui i 2Jk Fix Vik . Therefore,
x k is bounded. Corollary 2.1.37 yields
k k
V x x k z x k
i
T
for any z 2 i 2I Fix Ui , consequently, Vik x k x k is bounded. Since !ik .x k /
is also bounded, we have the following equivalence
2
lim !ikk .x k / Vikk x k x k D 0 ” lim !ikk .x k / Vikk x k x k D 0 (5.48)
k k
for any ik 2 Jk , k 0. Therefore, implication (5.46) is equivalent to the

following one

lim !ikk .x k / Vikk x k x k D 0 H) lim Ui x k x k D 0 for all i 2 I ,
k k
(5.49)
and implication (5.47) is equivalent to the following one

lim !ikk .x k / Vikk x k x k D 0 H) lim inf Ui x k x k D 0 for all i 2 I .
k k
(5.50)
(ii) The following condition yields the approximate regularity of fwk g1 kD0 :
There is a sequence fik g1
kD0 with i k 2 Jk for all k 0, such that
lim inf !ikk .x k / > 0

k
and

lim Vikk x k x k D 0 H) lim Ui x k x k D 0 for all i 2 I . (5.51)
k k
In particular, implication (5.51) holds if

k k
V x x k ˇ max Ui x k x k (5.52)
ik
i 2I
for all k 0 and for a constant ˇ > 0. If Jk D I , k 0, then the following con-
dition also implies the approximate regularity of fwk g1 k k
kD0 : lim infk !i .x / > 0
and for all i 2 I with !i .x / > 0 it holds
k k
k k
V x x k ˇ Ui x k x k for all k 0. (5.53)
i
If Ui D PCi for closed convex subsets Ci H, i 2 I , then we say that an

algorithm satisfying the latter condition is linearly focusing (cf. [22, Defini-
tion 4.8]).
Lemma 5.8.7. Let V k WD fVik gi 2Jk be a sequence of families of cutters Vik W H ! H,
i 2 Jk , k 0, and a sequence fx k g1kD0 be generated by recurrence (5.36). If the
sequence of weight functions wk W H !
jJk j :
(i) Is regular, then wk is approximately regular;
(ii) Contains a regular subsequence fwnk g1 k
kD0 , then w is approximately semi-
regular.
Proof. (i) Let a sequence fwk g1 k 1
kD0 applied to the sequence of families fV gkD0 be
ˇk -regular, where lim infk ˇk > 0. Then, for an arbitrary k, there exists ik 2 Jk
such that
2 2
!ikk .x k / Vikk x k x k ˇk max Ui x k x k .
i 2I
2
If !ikk .x k / Vikk x k x k ! 0, then, of course,
2
lim ˇk max Ui x k x k D 0
k i 2I
and, due to the positivity of lim infk ˇk , we have

2
lim max Ui x k x k D 0.
k i 2I

Consequently, limk maxi 2I Ui x k x k D 0, i.e., limk Ui x k x k D 0 for
all i 2 I .
(ii) Let a subsequence fwnk g1 k 1
kD0 fw gkD0 applied to the sequence of families
V , k 0, be regular, i.e., for an arbitrary k, there exists ink 2 Jnk such that
nk
2

!innk .x nk / Vinn k x nk x nk ˇnk max kUi x nk x nk k2
k k i 2I
and lim infk ˇnk > 0. If

2
lim !ikk .x k / Vikk x k x k D 0
k
then, of course, 2

lim !innk .x nk / Vinn k x nk x nk D 0,
k k k
consequently,
lim ˇnk max kUi x nk x nk k2 D 0.
k i 2I
By the positivity of lim infk ˇnk , we have
lim max kUi x nk x nk k2 D 0,

k i 2I
i.e.,
2
lim inf max Ui x k x k D 0
k i 2I

or, in other words, lim infk Ui x k x k D 0 for all i 2 I . t
u
Now we present theorems which give sufficient conditions for important control
sequences to be approximately (semi-)regular. Consider a special case of recurrence
(5.36), where Jk D I , Fix Vik Fix Ui for all k 0 and for all i 2 I . We can write
recurrence (5.36) in the form
X
x kC1 D x k C k !ik .Vik x k x k /, (5.54)
i 2I
where k 2 Œ0; 2, k 0. Denote, for simplicity, !ik D !ik .x k /. Let Ik I be

nonempty and !ik D 0 for i … Ik , k 0. Then (5.54) is equivalent to
X
x kC1 D x k C k !ik .Vik x k x k /, (5.55)
i 2Ik
where we can suppose without loss of generality that wk W H !

jIk j . Recurrence
(5.55) describes two stage algorithms, where fIk g1 kD0 is a control sequence for the
outer stage and fwk g1kD0 is a control sequence for inner stages. The application of
such algorithms is convenient if m D j I j is a big number. In this case we can
split I onto smaller subsets. The control Ik is, actually, a subset of I on which the
algorithm works in kth iteration. Consider the following condition for sequences
generated by recurrence (5.55): For any k 0, there is lk 2 Ik such that
2
lim !lkk Vlkk x k x k D 0 H) lim max PFix Ui x k x k D 0. (5.56)
k k i 2Ik
Note that the latter condition is weaker than the approximate regularity of fwk g1
kD0
with
respect to the family fPFix Ui gi 2I , because
Ik I , consequently maxi 2Ik
PFix U x k x k maxi 2I PFix U x k x k .
i i
Definition 5.8.8. (cf. [22, Definition 3.18]) We say that the sequence fIk g1kD0 I
is p-intermittent, where p 2 N, or intermittent if I D Ik [ IkC1 [ : : : [ IkCp1 for
any k 0.
A simple example of an intermittent sequence is an almost cyclic control
(cf. Definition 5.6.10).
Theorem 5.8.9. Let fx k g1 1
kD0 be generated by (5.54), fIk gkD0 I be intermittent.
Suppose that for any k 0 there is lk 2 Ik , k 0, satisfying condition (5.56). If
wki D 0 for all i … Ik , k 0, then fwk g1
kD0 is approximately regular with respect to
the family of cutters U WD fUi gi 2I .

Proof. Let wki D 0 for all i … Ik , k 0. Define ik WD argmaxi 2Ik Vik x k x k ,
2

k 0. Suppose that limk !lkk Vlkk x k x k D 0. Denote Ci WD Fix Ui , i 2 I . By
property (5.56), we have

lim max PCi x k x k D 0. (5.57)
k i 2Ik
Corollary 2.1.37, the assumption Fix Vik Fix Ui D Ci for all i 2 I and k 0,
and the definition of the metric projection yield
k
V y y
P k y y

kPCi y yk ,
i Fix V i
y 2 H, consequently,
k k
V x x k D max V k x k x k max PC x k x k . (5.58)
ik i i
i 2Ik i 2Ik
Inequality (5.58) and equality (5.57) imply

lim Vikk x k x k D 0. (5.59)
k
Let i 2 I . Since Ik is intermittent, there is p 1 such that i 2 IkCrk for some

rk 2 f0; 1; : : : ; ; p 1g. By the triangle inequality,
P the convexity of the norm, the
definition of ikCl , l D 0; 1; : : : ; p1, the equality j 2IkCl !jkCl D 1 and by (5.59),
we have
k 1
kCr rX kClC1
x k x k x x kCl
lD0

k 1
rX X

D
kCl kCl kCl kCl
!j .Vj x kCl
x /
lD0 j 2IkCl
k 1
rX X

kCl !jkCl VjkCl x kCl x kCl
lD0 j 2IkCl
X
p1
X
kCl
kCl !jkCl VikCl
kCl
x kCl
x
lD0 j 2IkCl
X
p1
kCl
D kCl VikCl
kCl
x kCl
x !0
lD0
as k ! 1, i.e.,
k
Further, by the definition of the metric projection and by the triangle inequality,

PC x k x k PC x kCrk x k PC x kCrk x kCrk C x kCrk x k .
i i i
(5.61)
Since i 2 IkCrk , equality (5.57) yields limk PCi x kCrk x kCr
k D 0. Now
inequalities (5.61) and equality (5.60) imply lim PC x k x k D 0. Since Ui

k i
are cutters, Corollary 2.1.37 yields limk Ui x k x k D 0, i 2 I , i.e., condition
(5.46) is satisfied, which means that fwk g1
kD0 is approximately regular. t
u
Let now
Ik WD fi 2 I W !ik > 0g. (5.62)
Definition 5.8.10. We say that a control sequence fwk g1kD0 is p-intermittent, where
p 1, or intermittent if the sequence fIk g1
kD0 , where Ik is defined by (5.62), is
p-intermittent, i.e., I D Ik [ IkC1 [ : : : [ IkCp1 .
If jIk j D 1 or, equivalently, wk D eik , k 0, for a sequence fik g1kD0 I , then

the fact that fwk g1
kD0 is intermittent means that fi g 1
k kD0 is an almost cyclic control
sequence.
Suppose now that fx k g1 kD0 is generated by recurrence (5.54) having the following
property: For any subsequence fx nk g1 k 1
kD0 fx gkD0 it holds

lim Vink x nk x nk D 0 H) lim kPFix Ui x nk x nk k D 0, (5.63)
k k
for all i 2 I (actually, it suffices to consider only i 2 Ik in (5.63)).

Theorem 5.8.11. Let fx k g1 kD0 be generated by recurrence (5.54) having property
(5.63). If fwk g1
kD0 is intermittent and !i ı for all i 2 Ik , k 0 and for some
k
k 1
constant ı > 0, then fw gkD0 is approximately regular with respect to the family of
cutters U WD fUi gi 2I .
Proof. Let fwk g1 kD0 be p-intermittent, where p 1, and ı > 0 be such that !i ı
k
for all i 2 Ik , k 0. Define

ik D argmax !ik Vik x k x k ;
i 2Ik
2
k 0. Suppose that limk !ikk Vikk x k x k D 0. By equivalence (5.48), we have

lim !ikk Vikk x k x k D 0. (5.64)
k
Let i 2 I and rk 2 f0; 1; : : : ; ; p 1g be such that i 2 IkCrk . By the triangle

inequality, the convexity of the norm, the definition of ikCl , l D 0; 1; : : : ; p 1, and
by (5.64), we have
k 1
kCr rX kClC1
x k x k x x kCl
lD0

k 1
rX X

D
kCl kCl kCl kCl
!j Vj x kCl
x
lD0 j 2IkCl
k 1
rX X

kCl !jkCl VjkCl x kCl x kCl
lD0 j 2IkCl
X
p1
kCl kCl
m kCl !ikCl
kCl
VikCl x x kCl ! 0
lD0
as k ! 1, consequently,

k
Further, by the definition of the metric projection and by the triangle inequality, we
have

PC x k x k PC x kCrk x k (5.66)
i i

PCi x kCrk x kCrk C x kCrk x k ,
where Ci D Fix Ui , i 2 I . Since i 2 IkCrk , we have !ikCrk ı. By the definition

of ikCrk , we have

lim VikCrk x kCrk x kCrk ı 1 lim !ikCrk VikCrk x kCrk x kCrk
k k

k kCrk kCrk kCrk
ı 1 lim !ikCr
kCr
V i kCr
x x D 0,
k k k

i.e., limk VikCrk x kCrk x kCrk D 0. By condition (5.63),

lim PCi x kCrk x kCrk D 0:
k

Inequalities (5.66) and equality
(5.65) imply
now limk PCi x k x k D 0. Since
Ui are cutters, we have limk Ui x k x k D 0 (see Corollary 2.1.37), i 2 I , i.e.,
condition (5.46) is satisfied, which means that fwk g1
kD0 is approximately regular. ut
Remark 5.8.12. Since a sequence generated by (5.54) is bounded as a Fejér
monotone sequence, it has a weak cluster point y . Therefore, property (5.63) and
the demi-closedness of PFix Ui Id at 0 yields that y 2 Fix Ui . This leads to
the assumption that algorithm (5.54) is focusing (see [22, Definition 3.7], where
this property was applied to firmly nonexpansive operators). Furthermore, property
(5.63) applied to operators Vik being metric projections leads to the assumption that
algorithm (5.54) is strongly focusing (see [22, Definition 4.8]).
Consider now the following recurrence
X
x kC1 D x k C k !ik .Ui x k x k /, (5.67)
i 2Ik
where Ik WD fi 2 I W !ik > 0g, k 0, which is a special case of (5.54) with

V k D U, k 0. Define Ki WD fk 0 W i 2 Ik g, i 2 I .
Definition 5.8.13. We say that fIk g1 kD0 is repetitive if for any i 2 I the subset
Ki is infinite. We say that a control sequence fwk g1kD0 is repetitive if the sequence
fIk g1
kD0 is repetitive.
An extended classification of control sequences for infinite I can be found in

[119, Sect. 3].
If j Ik jD 1 or, equivalently, wk D eik , k 0, for a sequence fik g1 kD0 I ,
then fwk g1kD0 is repetitive if and only if fi k g 1
kD0 is a repetitive control in the sense
of Definition 5.6.11.
Note that Theorems 5.8.9 and 5.8.11 hold for anyTsequence of relaxation
parameters k 2 Œ0; 2 and without the assumption that i 2ITFix Ui ¤ ;. It turns
out that a repetitive control is approximately semi-regular if i 2I Fix Ui ¤ ; and
if we restrict the choice of k to an interval Œ"; 2 " for some " 2 .0; 1/.
Theorem 5.8.14. Let Ui W H ! H, i 2 I , be cutters with a common fixed point
and fx k g1
kD0 be generated by recurrence (5.67), where lim infk k .2 k / > 0 and
!ik ı for all i 2 Ik , k 0 and for some ı > 0. If the control sequence fwk g1
kD0 is
repetitive, then fwk g1
kD0 is approximately semi-regular.
Proof. By Theorem 4.8.2, we have
kC1 2 2 X 2
x z x k z k .2 k / !ik Ui x k x k .
i 2Ik
If we iterate the above inequality k-times, we obtain
kC1 2 2 Xk X 2
x z x 0 z l .2 l / !il Ui x l x l .
lD0 i 2Il
Consequently,
1
X X 2
k .2 k / !ik Ui x k x k < 1.
kD0 i 2Ik
Since the sum of an absolutely convergent series does not depend on the order of
summands, we have
X
m X
2
k .2 k /!ik Ui x k x k
i D1 k2Ki
1
X X 2
D k .2 k / !ik Ui x k x k < 1.
kD0 i 2Ik
Therefore, X 2
k .2 k /!ik Ui x k x k < 1,
k2Ki
i 2 I , and
2
lim k .2 k /!ik Ui x k x k D 0,
k!1;k2Ki
i 2 I . Since lim infk k .2 k / > 0, it holds lim infk!1;k2Ki k .2 k / > 0,

consequently,
2
lim !ik Ui x k x k D 0,
k!1;k2Ki
for all i 2 I . The assumption

!ik ı > 0 for all i 2 Ik , k 0, yields now
limk!1;k2Ki Ui x x D 0, i 2 I , i.e., lim infk Ui x k x k D 0 for all i 2 I
k k
which means that fwk g1 kD0 is approximately semi-regular. t

u
5.8.2 Convergence Theorem
Theorem 5.8.15. Suppose that:

(a) Ui W H ! H, i 2 I , are cutters with a common fixed point,
(b) Ui Id are demi-closed at 0, i 2 I ,
(c) V
T D fVi gi 2J
k k
T families of cutters Vi W H ! H, i 2 Jk , with the property
k are
k
i 2Jk Fix Vi i 2I Fix Ui , k 0,

k
k 1
(d) fw gkD0 W H !
jJk j is a sequence of appropriate weight functions,
(e) lim infk k .2 k / > 0,
(f) fx k g1
kD0 is generated by recurrence (5.36).
If the sequence of weight functions fwk g1

kD0 applied to the sequence of families
V :
k
(i) Is approximately regular with respect to the family U WD fUi gi 2I , then x k

converges weakly to a common fixed point of Ui , i 2 I ;
(ii) Is approximately semi-regular with respect to the family U WD fUi gi 2I and H
is finite dimensional, then x k converges to a common fixed point of Ui , i 2 I .
Proof. (cf. [70, Theorem 9.27]) Let V k W H ! H be defined by
X
V k x WD !ik .x/Vik x
i 2Jk
and Tk be the k -relaxation of the operator V k , i.e.,
Tk x WD Vkk x D x C k .V k x x/. (5.68)
The operators V k are cutters (see Corollary 2.1.49), consequently, Tk are strongly
quasi-nonexpansive (see Theorem 2.1.39). By Theorem 2.1.26, we have
\ \
Fix Tk D Fix V k D Fix Vik Fix Ui ,
i 2Jk i 2I
consequently,
1
\ \
Fix Tk Fix Ui .
kD0 i 2I
Let " > 0 be such that lim infk k > " and lim infk .2 k / > " and let z 2
T
i 2I Fix Ui . For a sufficiently large k we have 2 k 2 . Now, it follows from
"
Theorem 4.8.2 that, for sufficiently large k,

kC1 2 2
x z D Tk x k z
2 X 2
x k z k .2 k / !ik .x k / Vik x k x k
i 2Jk
2 2
x k z k .2 k / V k x k x k
2 2 k
D x k z Tk x k x k 2
k
k 2 " 2
x z Tk x k x k .
4

Therefore, x k z decreases, consequently, V k x k x k ! 0, and, for an
arbitrary ik 2 Jk ,
2 X k k k k 2
!ikk .x k / Vikk x k x k !i .x / Vi x x k ! 0 (5.69)
i 2Jk
as k ! 1.
(i) Suppose that fwk g1 kD0 is approximately regular with respect to the family U WD
fUi gi 2I . Letik 2 Jk , k
0, be such that implication (5.46) holds. Then (5.69)
yields limk Ui x k x k D 0 for all i 2 I . Therefore, condition (3.11) is
satisfied for Tk WD V k and for S WD Ui , i 2 I . We have proved that all
assumptions of Corollary 3.7.1 (ii) are satisfied for X D H, Tk WD V k and for
S WD Ui , i 2 I . Therefore, x k converges weakly to a common fixed point of
Ui , i 2 I .
(ii) Suppose that H is finite dimensional and fwk g1 kD0 is approximately semi-
regular with respect to the family U WD fUi gi 2I . Let ik 2 Jk , k 0,
be such
that implication (5.47) holds. Then (5.69) yields lim infk Ui x k x k D 0 for
all i 2 I . Therefore, condition (3.12) is satisfied for Tk WD V k and for S WD Ui ,
i 2 I . We have proved that all assumptions of Theorem 3.7.1 (iii) are satisfied
for X D H, Tk WD V k and for S WD Ui , i 2 I . Therefore, x k converges to a
common fixed point of Ui , i 2 I . t
u
Remark 5.8.16. (i) We can also consider recurrence (5.36), where k depends on
x 2 H, i.e.,
P (5.70)
x kC1 D x k C k .x k /. i 2Jk !ik .x k /Vik x k x k /,
where k W H ! .0; 2 is a sequence of relaxation functions. It follows from

the proof of Theorem 5.8.15 that the theorem remains true if we suppose that
lim infk k .x/ " and lim supk k .x/ 2 " for all x 2 H and for some
" 2 .0; 1/.
(ii) We can apply different relaxation parameters for different operators Vik , i 2 I ,
in recurrence (5.36) (we suppose, for simplicity, Jk D I for all k 0) which
leads to the following recurrence
P (5.71)
x kC1 D x k C i 2I ki .ik .x k /Vik x k x k /,
where fk g1 kD0 .0; 2 is a sequence of relaxation vectors, i.e.,

k
D
k 1
.1 ; : : : ; m /, and fv gkD0 is a sequence of weight functions v W H !
m .
k k k
Note, however, that recurrence (5.71) can be reduced to (5.70) (or to (5.36)
if we apply constant weight functions vk ) if we substitute the sequence of
weight functions fvk g1 k 1
kD0 and the sequence of relaxation vectors f gkD0 to
k 1 1
the following sequences fw gkD0 W H !
m and fk gkD0 W H ! .0; 2,
where X
k .x/ D ki ik .x/ (5.72)
i 2I
and
!ik .x/ D ki ik .x/=k .x/; (5.73)
i 2 I and x 2 H (cf. Remark 4.4.3). Note that this transformation maintains
the assumptions on weight functions and on relaxation parameters presented
before. In particular, if lim infk ki " and lim supk ki 2 " for all i 2 I
and for some " 2 .0; 1/, then lim infk k .x/ " and lim supk k .x/ 2" for
all x 2 H. Furthermore, if vk is an approximately regular sequence of weight
functions and lim infk k .2 k / > 0, then wk is also approximately regular.
Therefore, we apply recurrence (5.36) instead of (5.71) in Theorem 5.8.15 in
order to simplify the notation.
(iii) If we replace the assumption on the demi-closedness of Ui Id at 0, i 2 I and
the assumption on the approximate regularity of fwk g1 kD0 in Theorem 5.8.15
by the following one:

k k k

k 2
T k !ik .x / Vik x x
lim k
D 0 H) all weak cluster points of fx k g1
kD0 lie in
i 2I Fix U i ,
then the theorem remains true (cf. Remark 3.7.5).
5.8.3 Examples
Example 5.8.17. (Iusem and De Pierro [215]) Consider recurrence (5.36), where
Jk D I , k D 1, Vik D PCi , for all k 0, Ci H are closed convex subsets,
i 2 I , the sequence fwk g1
kD0 of weight functions w W H !
m is constant, i.e.,
k
w D w, and w has the form

k
(
P i for i 2 I.x/
!i .x/ WD j 2I.x/ j
0 for i … I.x/,
where v D T .1 ; 2 ; : : : ; m / 2 ri
m and I.x/ WD fi 2 I W x … Ci g. Suppose
that C WD i 2I Ci ¤ ;. Since wk is regular (see Example 5.8.3 (ii)), it follows
from Theorem 5.8.15 that x k * x 2 C . The convergence was proved by Iusem
and de Pierro [215, Corollary 4] for H D Rn . Iusem and de Pierro have also
considered the case C D ; and have k
P proved the local convergence of x to a
the operator T WD
fixed point of [ i 2I !i PCi with Fix T ¤ ; for a starting
point x 0 2 B.z; 12 mini 2I kPCi z zk/, see [215, Theorem 3]). Note that
z2Fix T
this convergence can be proved simply as follows. If x 0 2 B.z; r/ for some
z 2 Fix T , where r WD 12 mini 2I kPCi z zk, then !i .x 0 / D i , i 2 I . The quasi
nonexpansivity of T (see Corollary 4.4.4) yields that x 1 2 B.z; r/. One can prove
by induction with respect to k that !i .x k / D i for all i 2 I and for all k. The
convergence follows now from Corollary 5.4.1.
Example 5.8.18. (Aharoni and Censor [3]) Consider recurrence (5.36), where
, Jk D I , Vik D PCi for closed convex subsets Ci Rn , i 2 I , with
H D RnT
C WDP i 2I Ci ¤ ;, k 2 Œ"; 2 " for some " 2 .0; 1/, k 0, wk 2
m
with 1kD0 !i D C1, i 2 I . By Theorem 4.4.5, we have
k
kC1 2 2 X
m
2
x z x k z k .2 k / !ik PCi x k x k
i D1
If we iterate this inequality k-times, we obtain
kC1 2 2 Xk X
m
2
x z x 0 z l .2 l / !il PCi x l x l .
lD0 i D1
Consequently,
X 1
m X
2
k .2 k /!ik PCi x k x k
i D1 kD0
1
X X
m
2
D k .2 k / !ik PCi x k x k < C1
kD0 i D1
and
1
X 2
k .2 k /!ik PCi x k x k < C1
kD0
for any i 2 I . The assumption lim infk k .2 k / > 0 yields now

1
X 2
!ik PCi x k x k < C1,
kD0
P1
i 2 I . Since kD0 !ik D C1, we have lim infk PCi x k x k D 0, i 2 I , i.e.,
wk is approximately semi-regular. Theorem 5.8.15 (ii) yields now the convergence
x k ! x 2 C . (See [3, Theorem 1] for a different proof of convergence).
Example 5.8.19. (Butnariu and Censor [53]). Consider recurrence (5.36), where
H D Rn , Jk D I , Vik D PCi , Ci H are closed convex subsets, k 0, i 2 I ,
lim infk k > 0, lim supk k < 2, wk 2 T
m has a subsequence converging to a
point w 2 ri
m . Suppose that C WD i 2I Ci ¤ ;. Let " > 0 be such that
!i > " for all i 2 I . Then there exists a subsequence fwnk g1 k 1
kD0 fw gkD0
nk nk 1
such that !i > 2 for all i 2 I and k 0, consequently, fw gkD0 is 2 -regular
" "
(see Example 5.8.3 (i)). Therefore, fwk g1 kD0 is approximately semi-regular (see
Lemma 5.8.7 (ii)). Theorem 5.8.15 (ii) yields now x k ! x 2 C . If we suppose
that all cluster points w of fwk g1
kD0 have the property !i ı for all i 2 I and for
some ı 2 .0; m1 , then fwk g1kD0 is regular (see Example 5.8.3 (iv)) and the weak
convergence holds in general Hilbert spaces H.
Example 5.8.20. (Flåm and Zowe [168]) Consider recurrence (5.36), where Jk D
Ti are cutters with Fix Vi
k k
I for all k, lim infk k > 0 and lim supk k < 2 and V
Ci for closed convex subsets Ci H with C WD i 2I Ci ¤ ;, satisfying the
inequality k k
V x x k ˛ PC x k x k , (5.74)
i i
i 2 I , k 0, for some constant ˛ > 0. Furthermore, suppose that the sequence of

weights fwk g1
kD0 satisfies the following conditions:
(i) lim supk !ik > 0, i 2 I ,

(ii) !ik .PCi x k x k / ¤ 0 ) !ik > ı > 0.
If we take Ui WD PCi , i 2 I , then it follows from (5.74) and from (i) to (ii)
that the sequence of weights fwk g1 kD0 is regular. We see that all assumptions of
Theorem 5.8.15 (i) are satisfied. Therefore, x k * x 2 C . The convergence was
proved by Flåm and Zowe [168, Theorem 1] for H D Rn . Actually, Flåm and Zowe
considered recurrence (5.71), but it can be reduced to (5.36), as it was explained in
Remark 5.8.16 (ii).
Example 5.8.21. Consider recurrence (5.55) with Vik WD PCi , i 2 I , k 0, for
closed convex subsets Ci H having a common fixed point, where k 2 Œ"; 2 "
for some " 2 .0; 1/, Ik WD fi 2 I W !ik > 0g and !ik ı for all i 2 Ik such that x k …
Ci , k 0 (recurrences of this form with countable I were considered by Combettes
and Puh [125, Sect. 3]). Then

condition (5.56) is automatically satisfied (it ksuffices
to take lk WD argmaxi 2Ik PCi x k x k ). If fIk g1 1
kD0 is intermittent, then fw gkD0 is
approximately regularT (see Theorem 5.8.9) and it follows from Theorem 5.8.15 (i)
that x k * x 2 C D i 2I Ci .
Example 5.8.22. (Simultaneous subgradient projection method) Consider recur-
rence (5.36), where Jk D I for all k 0, lim infk k > 0 and lim supk k < 2
and Vik D Vi WD Pci is a subgradient projection relative to a continuous convex
function ci W H ! R which is globally Lipschitz continuous on bounded subsets,
i 2 I , k 0. Recall that Vi are cutters (see Corollary 4.2.6) and Vi Id are demi-
closedT at 0 (see Theorem 4.2.7). Denote Ci WD S.ci ; 0/, i 2 I , and suppose that
C WD i 2I Ci ¤ ;. Let fwk g1 kD0 be a sequence of appropriate weight functions.
(a) If fwk g1kD0 is approximately regular, then it follows from Theorem 5.8.15 (i)
that x k converges weakly to a point x 2 C .
(b) If fwk g1kD0 is approximately semi-regular and H is finite dimensional, then it
follows from Theorem 5.8.15 (ii) that x k converges to a point x 2 C .
Example 5.8.23. (Censor and Elfving [87]) Consider a consistent system of linear
inequalities Ax b and a method defined by the recurrence
x kC1 D x k C k .Tk x k x k /,
where Tk W Rn ! Rn is given by
1
Tk x WD x A> Mk .Ax b/C ,
max .A> Mk A/
with Mk WD Dk MDk for a positive definite matrix M with nonnegative elements

and a diagonal matrix Dk WD diag.d1k ; d2k ; : : : ; dmk / given by

1 if i 2 I k
dik WD
0 otherwise,
and Ik WD fi 2 I W ai> x k ˇi > 0g. Denote by Ak the submatrix of A with

rows ai , i 2 Ik , by bk the subvector of b with coordinates ˇi , i 2 Ik , and by
Rk the principal submatrix of M with rows and columns i 2 Ik . Further, denote
C WD fx 2 Rn W Ax bg, Ck WD fx 2 Rn W Ak x bk g. Of course, C Ck .
Then Tk is the Landweber operator for the system Ak x bk relative to the
positive definite matrix Rk . Therefore, Fix Tk D Ck (see Lemma 4.6.2) and Tk
is firmly nonexpansive (see Theorem 4.6.3), consequently, Tk is a cutter. Let U be
an orthogonal matrix, G D diag.g1 ; g2 ; : : : ; gm / be such that gi > 0, i 2 I , and
M D U > GU (the existence of U and G follows from Theorem 1.1.34 (iii)). Denote
Gk WD Dk GDk . It is clear that Gk D diag.g1k ; g2k ; : : : ; gm
k
/ where

gi if i 2 I k
gik WD
0 otherwise.
We leave it to the reader to check that
Mk D Dk U > GUDk D U > Gk U (5.75)
and 1 1
Mk2 D U > Gk2 U: (5.76)
Denote y.x/ D .1 .x/; 2 .x/; : : : ; m .x//> D UAx. By Theorem 1.1.27, we have
2
12 2 12 2 > 12
max .A Mk A/ D Mk A
>
D sup
M Ax D sup
k
U G UAx
k
kxkD1 kxkD1
2
12 X
D sup G
k y.x/ D sup
gik .i .x//2
kxkD1 kxkD1 i 2I
X 1 2

sup gi .i .x//2 D sup G 2 y.x/
kxkD1 i 2I kxkD1
1 2 2
1
D sup G 2 UAx D sup U > G 2 UAx
kxkD1 kxkD1
1 2 1 2

D sup M 2 Ax D M 2 A D max .A> MA/.
kxkD1
Let S W Rn ! Rn be the Landweber operator for the system Ax b related to

the matrix M , i.e., S has the form
1
S x WD x A> M.Ax k b/C .
max .A> MA/
Note that Mk .Ax k b/C D M.Ax k b/C . Because the system Ax b is equivalent
1 1 1
to M 2 Ax M 2 b, Lemma 4.6.2 yields that Fix S D fx 2 Rn W M 2 Ax
1
M 2 bg D C . Therefore,
1 >
Tk x k x k D A Mk .Ax k b/C
max .A> Mk A/
1 >
A Mk .Ax k b/C
>
max .A MA/
1 >
D A M.Ax k b/C
max .A> MA/

D S x k x k ,
i.e., condition (3.11) is satisfied and C D Fix S Fix Tk .

We see that all assumptions of Corollary 3.7.1 (iii) are satisfied. Therefore, x k !

x 2 Fix S D fx 2 Rn W Ax bg.
5.8.4 Block Iterative Projection Methods
Consider the convex feasibility problem: find a common point of a finite family
of nonempty closed convex subsets Ci H, i 2 I . If m is a large number, it is
convenient to split the set I into smaller blocks and to perform the operations of the
projection method in two stages: first on blocks and then on the family of blocks.
Let I D L1 [ L2 [ : : : [ Lr , where Lj are nonempty subsets which are pairwise
disjoint, j 2 J D f1; 2; : : : ; rg. Then each i 2 I can be transformed in a unique
way to a pair .j; l/ such that j 2 J and l 2 Lj . The subset Ci canTnow beTpresented
as Cj l and the convex feasibility problem obtains the form: x 2 j 2J l2Lj Cj l .
Consider the following two stage recurrence
x kC1 D Ujkk x k , (5.77)
where jk 2 J is a control for the outer loop and Ujkk is a relaxed simultaneous
projection which “employs” only projections PCjk ;l for l 2 Ljk , i.e., Ujkk has the
form X
Ujkk WD Id Ck . !ik PCjk ;l Id/, (5.78)
l2Ljk
where k 2 Œ0; 2 and wk W H !

jLj j is a sequence of weight functions applied in
k
the inner loop. The sequence f.jk ; wk /g1kD0 defines a two stage control. In particular,
the following operator Ujkk can be used in the inner loop of a two stage recurrence
(5.77):
Ujkk WD PCjk ;lk (5.79)
where lk 2 Ljk is a control sequence for the inner loop. Note that PCjk ;lk is a special
case of the operator Ujkk defined by (5.78), where k D 1 and wk D elk .
We can identify the weight function wk W H !
jLj j with a weight function
k
vk W H !
m defined by

!ik if i 2 Ljk
ik WD (5.80)
0 otherwise.
5.8.4.1 Double Layer Control Sequence
Consider the two stage recurrence (5.77), where the control sequence:
fjk g1
kD0 J applied in the outer stage is an almost cyclic control, i.e.,
J fjk ; jkC1 ; : : : ; jkCp1 g (5.81)

for some p r and, for all k, Ujkk is defined by (5.78), where the weight function
wk W H !
jLj j is ˛-regular with respect to the family fPCjk ;l gl2Ljk , where ˛ 2
k
.0; 1, i.e., there exists lk 2 Ljk such that
2 2

!lkk PCjk ;lk x k x k ˛ max PCjk ;l x k x k , (5.82)
l2Ljk
k 0. Examples of such weight functions were presented in Sect. 5.8.1. We call the
sequence f.jk ; wk /g1
kD0 defined above a double layer control. A special case of this
control is the following: ˛ D 1, k D 1, wk D elk 2 ext
jLj j and lk 2 Ljk is a
k
remotest set control, i.e., lk D lk .x k / 2 Ljk is such that

PCjk ;lk x k x k D max PCjk ;l x k x k . (5.83)
l2Ljk
The double layer control f.jk ; wk /g1kD0 can be identified with a sequence of
weight functions fvk g1
kD0 defined by (5.80).
Corollary 5.8.24. The double layer control is approximately regular with respect
to the family U WD fPCi gi 2I .
Proof. Recurrence (5.77), where Ujkk is defined by (5.78), is a special case of (5.55),
where Ik D Ljk , V k D U WD fPCi gi 2I . Furthermore, (5.82) yields condition (5.56).
Since fjk g1 1
kD0 is almost cyclic, fIk gkD0 is intermittent. The claim follows now from
Theorem 5.8.9. t
u
Theorem 5.8.25. Let C ¤ ; and x k be generated by recurrence (5.77) with a
double layer control and with a sequence of relaxation parameters fk g1 kD0 such
that lim inf k .2 k / > 0. Then for an arbitrary x 0 2 H the sequence fx k g1
kD0
converges weakly to a point x 2 C .
Proof. It follows from Corollary 5.8.24 that the double layer control is approxi-
mately regular. The claim follows now from Theorem 5.8.15 (i). t
u
Several block iterative projection methods are presented in [3, 53, 81, 229], [118,
Chap. 5], [108, Sect. 5.6], [5, 59, 159].
5.9 Sequential Cutter Methods
In this section we consider the common fixed point problem for a finite family of
cutters Ui W H ! H, i 2 I WD f1; 2; : : : ; mg, having a common fixed point, and
sequences generated by the following recurrence
x0 D x 2 H – arbitrary
(5.84)
x kC1 D x k C k .Uik x k x k /
5.9 Sequential Cutter Methods 251
where fk g1 1
kD0 .0; 2 is a sequence of relaxation parameters and fik gkD0 I is a
control sequence.
Definition 5.9.1. Let fx k g1
kD0 be generated by recurrence (5.84). A control
sequence fik g1
kD0 W H ! I is called:
(i) Approximately regular if the following implication holds

lim Uik x k x k D 0 H) lim Ui x k x k D 0 for all i 2 I , (5.85)
k k
(ii) Approximately semi-regular if the following implication holds

lim Uik x k x k D 0 H) lim inf Ui x k x k D 0 for all i 2 I .
k k
Note that recurrence (5.84) is a special case of (5.36), where wk D eik , k 0.

Therefore, a control sequence fik g1 kD0 is approximately (semi-)regular means that
feik g1
kD0 is an approximately (semi-)regular sequence of weight functions with
respect to the family U WD fUi gi 2I .
If Ui are metric projections, then an approximately (semi-)regular control
coincides with an approximately (semi-)remotest set control (cf. Definitions 5.6.1
and 5.6.2). If, furthermore, fik g1
kD0 is an almost remotest set control (see Defini-
tion 5.6.6), then the sequence feik g1 kD0 is a regular sequence of weight functions
(see Example 5.8.3 (v)).
If we take V k D U WD fUi gi 2I and wk WD eik , k 0, in Theorem 5.8.15, then we

obtain the following result.
Corollary 5.9.2. Let Ui W H ! H, be cutters with a common fixed point and such
that Ui Id are demi-closed at 0, i 2 I , and the sequence fx k g1
kD0 be generated by
recurrence (5.84), where lim inf k .2 k / > 0. If the control sequence fik g1
kD0 is:
(i) Approximately regular, then x k converges weakly to a common fixed point of

Ui , i 2 I .
(ii) Approximately semi-regular and H is finite dimensional, then x k converges to
a common fixed point of Ui , i 2 I .
5.9.2 Control Sequences for Sequential Cutter Methods
Corollary 5.9.2 shows the importance of approximate (semi-)regularity of the

control sequence fik g1
kD0 for the convergence of the sequences generated by the
sequential cutter method. In this section we present some control sequences which
are approximately (semi-)regular. The results presented in this section are special
cases of Theorems 5.8.11 and 5.8.14, where V k D U D fUi gi 2I and wk D eik .
Corollary 5.9.3. Let Ui W H ! H be cutters having the following property: for
any bounded sequence fy k g1
kD0 H

lim Ui y k y k D 0 H) lim PFix Ui y k y k D 0, (5.86)
k k
i 2 I . Let fx k g1kD0 be generated by recurrence (5.84). If the control sequence

fik g1
kD0 is almost cyclic, then fik g1
kD0 is approximately regular.
A family of operators fUi gi 2I has property (5.86) if, e.g.,
kUi y yk ı kPFix Ui y yk (5.87)
for a constant ı > 0, and for arbitrary i 2 I and y 2 H. Of course, the latter
property is satisfied for operators Ui being strict relaxations of metric projections
PCi onto nonempty closed convex subsets Ci , i 2 I . Therefore, Corollary 5.9.3
yields in particular that an almost cyclic control sequence applied to a successive
projection method (5.28), where lim infk k .2 k / > 0, is an approximately
remotest set control sequence (see Lemma 5.6.12 (i)).
Corollary 5.9.4. Let Ui W H ! H, i 2 I be cutters with a common fixed point and
fx k g1
kD0 be generated by recurrence (5.84), where lim infk k .2 k / > 0. If the
control sequence fik g1 1
kD0 is repetitive, then fik gkD0 is approximately semi-regular.
5.9.3 Examples
Example 5.9.5. (Aharoni, Berman and Censor [2]) Consider method (5.84), where
H is finite dimensional and the cutters Ui satisfy the demi-closedness principle,
i 2 I , (in [2] it was supposed that Ui have property (5.87), i 2 I ), k 2 Œ; 2 for
some 2 .0; 1/ and the control fik g1 kD0 is repetitive. By Corollary 5.9.4 the control
fik g1
kD0 is approximately semi-regular. Note that for a cutter Ui with property (5.87),
the operator Ui Id is closed at 0, because, by the nonexpansivity of the metric
projection, the operator PFix Ui Id is closed at 0, i 2 I (seeTLemma 3.2.5).
Therefore, Corollary 5.9.2 (ii) yields the convergence x k ! x 2 i 2I Fix Ui . The
convergence was proved by Aharoni et al. [2, Theorem 1] (see also [108, Sect. 5.5]),
where the method was called the .ı; /-algorithm. An interesting convergence result
for method (5.84) with k D 1 and Ui W Rn ! Rn being paracontractions
(continuous strictly quasi-nonexpansive operators) and with repetitive control was
obtained by Elsner et al. [164, Theorem 1] (see also [163, Theorem 1] for linear
case). This result is not covered by Corollaries 5.9.2 (ii) and 5.9.4. Conversely, the
5.10 Extrapolated Simultaneous Cutter Methods 253
convergence of sequences in Rn generated by (5.84) with repetitive control does

not follow from the [164, Theorem 1], because the cutters are not supposed to be
continuous.
Example 5.9.6. Consider method (5.84), where lim infk k .2 k / > 0 and Ui ,
i 2 I , are firmly nonexpansive operators with a common fixed point. Then Ui Id
are demi-closed at 0, i 2 I .
1
(i) If Ui have property (5.86), i 2 I , and the control sequence fi Tk gkD0 is almost

cyclic, then Corollaries 5.9.2 (i) and 5.9.3 yield x * x 2 i 2I Fix Ui (see
k
also [331, Theorem 2] and [6], where a quasi-cyclic order is considered, which
is slightly more general than the almost cyclic control).
(ii) If H is finite dimensional and the control sequenceT fik g1
kD0 is repetitive, then

Corollaries 5.9.2 (ii) and 5.9.4 yield x ! x 2 i 2I Fix Ui (see also [22,
k
Example 3.14], [331, Theorem 1] and [6, Theorem 3.1] for related results).
Example 5.9.7. (Sequential subgradient projection method) Consider method
(5.84), where lim infk k .2 k / > 0 and Ui , i 2 I , are subgradient projections,
i.e., Ui D Pci , where ci W H ! R are continuous convex functions, i 2 I , which
T holds if, e.g., H D R ).
n
are globally Lipschitz continuous on bounded subsets (this
Denote Ci WD S.ci ; 0/, i 2 I , and suppose that C WD i 2I Ci ¤ ;. Since Ui are
cutters (see Corollary 4.2.6) and Ui Id are demi-closed at 0 (see Theorem 4.2.7),
it follows from Corollary 5.9.2 (i) that x k converges weakly to a point x 2 C if the
control sequence fik g1 1
kD0 is approximately regular (e.g., if fik gkD0 is almost cyclic).
Similarly, it follows from Corollary 5.9.2 (ii) that x converges to a point x 2 C ,
k
if H is finite dimensional and the control sequence fik g1 kD0 is approximately semi-
regular (e.g., if fik g1
kD0 is repetitive). A special case of the sequential subgradient
projection method was presented in [102], where H D Rn and the control is
almost cyclic. In [136] a finitely convergent modification of the method of [102]
was presented. In [80] the method of [102] was applied for solving interval linear
inequalities. For a survey on subgradient projection methods, see [79].
5.10 Extrapolated Simultaneous Cutter Methods
We consider the common fixed point problemT for a finite family of cutters Ui W
H ! H, i 2 I WD f1; 2; : : : ; mg, with i 2I Fix Ui ¤ ;. In this section we
study convergence of sequences which are generated by generalized relaxations of
simultaneous cutters of the form
T x WD V; x D x C .x/.V x x/, (5.88)
where 2 Œ0; 2, W H ! .0; P1/ is a step size function, V W H ! H is a

simultaneous
T cutter, i.e.,
T V x WD i 2J i .x/Vi x, V WD fVi gi 2J is a finite family of
cutters, i 2J Fix Vi i 2I Fix Ui , and v W H !
jJ j is a weight function with
v.x/ D .1 .x/; : : : ; jJ j .x// 2

jJ j . In particular, we can take V D U WD fUi gi 2I .
In this case
T x D U; x D x C .x/.Ux x/,
P
where Ux WD i 2I !i .x/Ui x and w W H !
m . A sequence of operators Tk of the
form (5.88) describes, for any x 0 2 H, the recurrence x kC1 D Tk x k , i.e.,
X
x kC1 D x k C k k .x k /. ik .x k /Vik x k x k /, (5.89)
i 2Jk
where k 2 Œ0; 2, fk g1 kD0 W H ! .0; 1/ is a sequence of step size func-
tions, fvk g1 kD0 W H !
jJk j is a sequence of weight functions, i.e., vk .x/ D
.1 .x/; : : : ; jJk j .x// 2
jJk j and V k WD fVik gi 2Jk , k 0, is a sequence of families
k k
T T
of cutters with i 2Jk Fix Vik i 2I Fix Ui for all k 0. The iterative method
described by (5.89) is called an extrapolated simultaneous cutter method (ESCM).
If we take, in particular, V k WD U for all k 0, we obtain the following recurrence
X
k
x kC1 D x k C k k .x k /. ik .x k /Ui x k x k /, (5.90)
i 2I
where fvk g1 kD0 W H !

m is a sequence of weight functions and v .x/ D
k
.1 .x/; : : : ; m .x//, x 2 H. We can appropriately apply Definition 5.8.5 and

k k
Remark 5.8.6 also for sequences generated by recurrence (5.89).

If Ui are projections, then the methods defined by recurrence (5.90) are known in
the literature as extrapolated simultaneous projection methods or extrapolated meth-
ods of parallel projections. Various variants of extrapolated simultaneous cutter (in
particular projection) methods were studied in [146, 284, 351], [62, Chaps. 4 and 5],
[175, 229, 247], [17, Chap. 8], [118, Chap. 5], [89, 90, 98, 99, 119, 120, 176, 230–
233, 304, 305], [10, Theorem 1 and Corollary 4.4], [67–70, 92, 190].
5.10.1 Assumptions on Step Sizes
Let V WD fVi gi 2J be a finite family of cutters with a common fixed point and v W
H !
jJ j be an appropriate weight function. Define a step size function v W H !
R by the equality
8 P
< i .x/kVi xxk2 T
P
i 2J
if x … i 2J Fix Vi ,
k i .x/Vi xx k
2
v .x/ WD i 2J
T
:1 if x 2 Fix Vi .
i 2J
Since cutters are strictly

P quasi-nonexpansive (see Theorem
T 2.1.39), it follows from
Theorem 2.1.26 that i 2J i .x/Vi x ¤ x for x … i 2J Fix Vi , consequently, v is
well defined. Furthermore, it follows from the convexity of the function kk2 that
v .x/ 1 for all x 2 H.
Definition 5.10.1. Let Vi W H ! H, i 2 J , be cutters with a common fixed
point and v W H !
jJ j be a weight function which is appropriate with respect
to the family V WD fVi gi 2J . We say that a step size function W H ! .0; C1/
is ˛-admissible with respect to the family fVi gi 2J , where ˛ 2 .0; 1, or, shortly,
admissible, if
˛v .x/ .x/ v .x/ (5.91)
T
for all x … i 2J Fix Vi .
Theorem 5.10.2. Suppose that:

(a) Ui W H ! H, i 2 I , are cutters with a common fixed point,
(b) Ui Id, i 2 I , are demi-closed at 0,
(c) V k WD fVik gi 2Jk are families of cutters Vik W H ! H, i 2 Jk , with the
properties \ \
Fix Vik Fix Ui (5.92)
i 2Jk i 2I
and
max Vik x x max kUi x xk (5.93)
i 2Jk i 2I
for all x 2 H, k 0, and for some constant > 0.

(d) fvk g1kD0 W H !
jJk j is a sequence of appropriate weight functions,
(e) The step size k W H ! .0; C1/ is ˛-admissible with respect to V k , k 0, for
some ˛ 2 .0; 1,
(f) lim infk k .2 k / > 0,
(g) fx k g1
kD0 is generated by recurrence (5.89).
Then:
(i) If the sequence of weight functions fvk g1 kD0 applied to the sequence of families
V k is regular with respect to the family U WD fUi gi 2I , then x k converges weakly
to a common fixed point of Ui , i 2 I .
(ii) If the sequence of weight functions fvk g1 kD0 applied to the sequence of families
V k contains a subsequence of weight functions which is regular with respect to
the family U WD fUi gi 2I and H is finite dimensional, then x k converges to a
common fixed point of Ui , i 2 I .
Proof. (cf. [70, Theorem 9.35]) Let V k W H ! H be defined by
X
V k x WD ik .x/Vik x
i 2Jk
and Tk be a generalized relaxation of V k , i.e.,
Tk x WD Vkk ;k x D x C k k .x/.V k x x/. (5.94)
The operators V k areTcutters (see Corollary 2.1.49). By Theorem 2.1.26, we have

Fix Tk D Fix V k D i 2Jk Fix Vik (note that a cutter is strictly quasi-nonexpansive
T T
by Theorem 2.1.39). By (5.92), 1 kD0 Fix Tk i 2I Fix Ui . Let " 2 .0; 1/ and
k0 0 be such that k 2 Œ"; 2 " for all k k0 . By Theorem 4.9.1, the operator
Vkk is a cutter, consequently Tk is a k -relaxed cutter. Theorem 2.1.39 implies
kC1 2 2 2 k
x z x k z Tk x k x k 2
k
T
for all z 2 m Fix kUi and
i D1 all k k0 . Consequently, fx k g is bounded, x k z is
monotone and Tk x x k ! 0.
(i) Let ˇ > 0, k1 k0 and jk 2 f1; 2; : : : ; Jk g be such that
2

jkk .x/ Vjkk x x ˇ max kUi x xk2
i 2I
for any x 2 H and k k1 . Since k is ˛-admissible, the norm kk is a convex

function and Vjk x x maxi 2I kUi x xk for all j 2 Jk , we have

Tk x k x k D k k .x k / V k x k x k
P
k k k k

k 2
i 2Jk i .x / Vi x x
k ˛ P V k x k x k
k 2
i 2Jk i .x /Vi x x
k k k k
P k k k k

k 2
i 2Jk i .x / Vi x x
D k ˛ P
k .x k /V k x k x k
i 2Jk i i
2

jkk .x k / Vjkk x k x k
k ˛ P
k k k k

k
i 2Jk i .x / Vi x x
2
ˇ maxi 2I Ui x k x k
k ˛ P .
. i 2Jk ik .x k // maxi 2I Ui x k x k
"˛ˇ
D max Ui x k x k
i 2I
for all k k0 . Consequently,


Tk x k x k "˛ˇ max Ui x k x k (5.95)
i 2I

and Ui x k x k ! 0 for all i 2 I . Therefore, condition (3.6) is satisfied
for Uk D Tk and S D Ui , i 2 I . We have proved that all assumptions of
Theorem 3.6.2 (i) are satisfied for S D Ui , i 2 I . Therefore, x k converges
weakly to a common fixed point of Ui , i 2 I .
(ii) Suppose that H is finite dimensional and fvk g1 kD0 contains a ˇ-regular subse-
quence fvnk g1
kD0 . Let ˇ > 0, k1 k0 and jnk 2 I be such that
2

jnnk .x/ Vjnnk x x ˇ max kUi x xk2 .
k k i 2I
Similarly as in (i) one proves that
"˛ˇ
kTnk x nk x nk k max kUi x nk x nk k .
i 2I

Consequently, lim infk Ui x k x k D 0 for all i 2 I . If we take Uk D Tk and
S D Ui , i 2 I , in Theorem 3.6.2 (ii), we obtain the convergence of x k to a
fixed point of Ui for all i 2 I . t
u
The operators Tk defined by (5.94) do not need to be continuous, because we have
not assumed that the operators Vik as well as the weight functions ik are continuous.
This allows application of operators from an essentially broader family than the
nonexpansive ones. We have supposed in Theorem 5.10.2 that fvk g1 kD0 is regular
(in (i)) or that fvk g1
kD0 contains a regular subsequence (in (ii)). These assumptions
seem not to be too restrictive. The assumption on the step sizes guarantees that Tk
is a relaxed cutter. All these assumptions allow using algorithms which can treat the
violated and the nonviolated constraints in a more flexible way and provide long
steps.
Remark 5.10.3. In [119, Sect. 1] and [120, Sects. IV and V] Combettes presented
three methods which are quite similar to ESCM: two extrapolated methods of
parallel projections (EMOPP and EPPM2) and an extrapolated method of parallel
subgradient projection (EMOPSP):
(i) In EMOPP (see [119, Definition 1.2]) it is supposed that Jk D I , Vjk D PC k ,
i
where PC k is the metric projection onto a closed convex superset Cik of Ci
i
with
d.x k ; Cik / d.x k ; Ci /, (5.96)
for all i 2 I , k 0, and for some constant > 0, and the weights vk 2
m
satisfy condition (5.42), k 0.
(ii) In EPPM2 (see [120, Sect. IV.C]) it is supposed that Jk D I , Vjk D PC k , where
i
PC k is the metric projection onto a closed and convex superset Cik of Ci , i 2 I ,
i
k 0, and v 2
m . The supersets Cik of Ci are chosen in such a way that the
following condition is satisfied: for any subsequence fx nk g1 k 1
kD0 fx gkD0 and
for all i 2 I it holds

.x nk * x and lim PC nk x nk x nk D 0/ H) x 2 Ci . (5.97)
k i
(iii) In EMOPSP (see [120, Sect. V]) it is supposed that Jk I , Jk ¤ ;, Vik D Pfi
are subgradient projections relative to a (lower semi-)continuous convex func-
tion fi W H ! R,
ik ı > 0 for xk … Ci ; i 2 Jk : (5.98)
The above methods EMOPP and EMOPSP are covered by ESCM, where Vjk are
cutters, j 2 Jk , k 0, satisfying (5.92) and (5.93) and fvk g1 kD0 is a sequence of
appropriate weight functions. Note that (5.96) with ik satisfying ik ı > 0 for
xk … Ci , i 2 Jk , implies that fvk g1 kD0 is a regular sequence of weight functions
(see Example 5.8.3 (iv) and Definition 5.8.4). In EPPM2 condition (5.97) is slightly
weaker (only in an infinite dimensional case) than the approximate regularity of vk
satisfying (5.98), but Theorem 5.10.2 enables application of more general sequences
of weight functions than the ones satisfying (5.98). Furthermore, Theorem 5.10.2
T not krequire Ci T Ci , i 2 Jk , k 0. The theorem requires only the inclusion
k
does
i 2Jk Ci C WD i 2I Ci to be satisfied, k 0.
5.10.3 Extrapolated Simultaneous Subgradient

Projection Method
Let ci W H ! R be convex functions which are globally Lipschitz continuous

T on
bounded subsets, Ci WD fx 2 H W ci .x/ 0g, i 2 I and C WD i 2I Ci ¤ ;.
The extrapolated simultaneous subgradient projection method (ESSPM) is a
special case of an ESCM defined by recurrence (5.90), where Ui are subgradient
projections, i.e., Ui are defined by (4.115), i 2 I . By Corollary 4.2.6, Lemma 4.2.5
and by Theorem 4.2.7, Ui are cutters, Fix Ui D Ci and Ui Id are demi-closed at 0,
i 2 I . Let vk W H !
m be an appropriate weight function and a step size function
k W H ! .0; C1/ be defined by
P .c .x//
i 2I ik .x/. kgi i .x/kC /2
k .x/ WD 2 ,
P .c .x//
i 2I ik .x/ kgi .x/kC2 gi .x/
i
.ci .x//C
k 0 (for simplicity, we use the convention that kgi .x/k
D 0 if .ci .x//C D 0). The
ESSPM is defined by
5.11 Extrapolated Cyclic Cutter Method 259
X .ci .x k //C
x kC1 D x k k k .x k / ik .x k / k
gi .x /, (5.99)
i 2I
gi .x k /2
where x 0 2 H is arbitrary and k 2 .0; 2/, k 0. We can also write
x kC1 D Ukk ;k x k ,
where
Ukk ;k WD Id Ck k .U k Id/
is a generalized relaxation of the simultaneous subgradient projection
X
U k WD ik Pci ,
i 2I
k 0.
Theorem 5.10.4. Let x 0 2 H be arbitrary and x k be generated by recurrence
(5.99), where lim infk k .2 k / > 0.
(i) If fvk g1 k
kD0 is regular, then x converges weakly to a common fixed point of Ui ,
i 2 I.
(ii) If fvk g1
kD0 contains a subsequence of weight functions which is regular and H
is finite dimensional, then x k converges to a common fixed point of Ui , i 2 I .
Proof. The step sizes k are 1-admissible. Therefore, the theorem follows from
Theorem 5.10.2. t
u
The ESSPM was studied by Dos Santos in [146], where positive constant weights
w 2 ri
m were considered and the convergence in the finite dimensional case was
proved [146, Sect. 5]. Furthermore, the method was studied by Combettes in [120,
Sect. V] (cf. Remark 5.10.3 (iii)).
5.11 Extrapolated Cyclic Cutter Method
Let Ui W H ! H be cutters with a common fixed point, U D Um Um1 : : : U1 be a

cyclic cutter and x 0 2 H. Consider sequences fx k g1
kD0 generated by the recurrence
x kC1 D x k C k k .x k /.Ux k x k /, (5.100)
where k 2 .0; 2, the step size functions k W H ! .0; C1/ satisfy the inequality
˛ k .x/ max .x/ (5.101)

for all x 2 H with ˛ 2 .0; 2m

1
,
P
i 2I hUx Si 1 x; Si x Si 1 xi
max .x/ WD ,
kUx xk2
where Si WD Ui Ui 1 : : : U1 for i D 1; 2; : : : ; m and S0 WD Id (cf. (4.121)). The

existence of the step size k .x/ follows from inequality (4.122). Since
P
i 2I kSi x Si 1 xk2
m 1
kUx xk2
(see again (4.122)) we can use in particular step size functions k satisfying the
inequality P
kSi x Si 1 xk2
m˛ i 2I k .x/ max .x/ (5.102)
kUx xk2
for all x 2 H. We can also write recurrence (5.100) in the form x kC1 D Uk ;k x k ,
where Uk ;k WD Id Ck k .U Id/ is a generalized relaxation of the cyclic cutter
U with the step size function k satisfying inequalities (5.101).
5.11.1 Convergence
In [71] one can find a slightly weaker version of the following theorem.
Theorem 5.11.1. Let fx k g1 kD0 be generated by (5.100), where k .x/ satisfies
(5.101) and let z 2 Fix U . Then the following inequality holds
kC1 2 2 2
x z x k z k .2 k /˛ 2 Ux k x k . (5.103)
Moreover:
(i) If U Id is demi-closed at 0 and lim infk k .2 k / > 0, then x k * x 2
Fix U .
(ii) If the step size function k satisfies inequalities (5.102), then
Pm
kC1 2 2 k 2 2
i D1 Si x Si 1 x
k
. /
x z x k z k .2 k /m2 ˛ 2 2 .
Ux k x k
(5.104)
If, furthermore, Ui Id are demi-closed at 0, i D 1; 2; : : : ; m, and lim infk k
.2 k / > 0, then x k * x 2 Fix U .
Proof. Since a T
cutter is strictly quasi-nonexpansive (see Theorem 2.1.39), we
have Fix U D i 2I Fix Ui . Note that Uk ;k x k x k D k k .x k /.Ux k x k /.
5.11 Extrapolated Cyclic Cutter Method 261
By Theorem 4.10.2, the operator Uk is a cutter and
kC1 2 2 2 2 k
x z D U; x k z x k z U ; x k x k 2
k k
k
k 2 2
D x z k .2 k /k2 .x k / Ux k x k
2 2
x k z k .2 k /˛ 2 Ux k x k .

Therefore, x k z is decreasing, x k is bounded and
k
Ux x k ! 0. (5.105)
Let x 2 H be a weak cluster point of the sequence fx k g1 nk 1

kD0 and fx gkD0
k 1
fx gkD0 be a subsequence which converges weakly to x .
(i) Suppose that U Id is demi-closed at 0. Condition (5.105) yields x 2 Fix U .
The weak convergence of the whole sequence fx k g1
kD0 to x follows now from
Lemma 3.3.3.
(ii) Suppose that the step size function k satisfies inequality (5.102). Then
inequality (5.104) can be proved similarly to (5.103). Consequently,

Si x k Si 1 x k ! 0 (5.106)
for i D 1; 2; : : : ; m, because lim infk k .2 k / > 0 and Ux k x k is bounded.

Suppose that Ui Id are demi-closed at 0, i D 1; 2; : : : ; m. Condition (5.106)
for i D 1 yields
kU1 x nk x nk k D kS1 x nk S0 x nk k ! 0.
Due to the demi-closedness of U1 Id at 0, we have U1 x D x . Since
k.U1 x nk U1 x / .x nk x /k D kU1 x nk x nk k ! 0
and x nk * x , we have U1 x nk * U1 x D x . By (5.106) for i D 2,
kU2 .U1 x nk / U1 x nk k D kS2 x nk S1 x nk k ! 0.
Since U2 Id is demi-closed at 0 and U1 x nk * x , U2 x D x . In a similar

way we obtain that Ui x D x for i D 3; : : : ; m. Therefore,
Ux D Um : : : U1 x D x .
We conclude that the subsequence fx nk g1

kD0 converges weakly to a fixed point
of the operator U . The weak convergence of the whole sequence fx k g1 kD0 to
x 2 Fix U follows now from Lemma 3.3.3. t
u
Even if we take WD max and D 2, the generalized relaxation U; needs not
to be an extrapolation of U because max needs not to be greater or equal to 12 .
We only know that max 2m1
(see (4.122)). Note, however, that U is m1 -SQNE (see
2m
Corollary 4.5.15), consequently, it is a mC1 -relaxed cutter (see Corollary 2.1.43) and
U 1Cm is a cutter (see Remark 2.1.3). Since Umax is a cutter (see Theorem 4.10.2),
2m
Lemma 2.4.7 (i) yields that U is also a cutter, where D maxf 1Cm 2m
; max g.
Therefore, U; with 2 . 1Cm ; 2 is an extrapolation of U .
2m
Interesting schemes for accelerated cyclic projections for subspaces of H can be

found in [177] and in [141].
5.11.2 Accelerated Kaczmarz Method for a System

of Linear Equations
Consider a consistent system of linear equations
hai ; xi D ˇi ; i 2 I , (5.107)
where ai 2 H, ai ¤ 0 and ˇi 2 R, i 2 I . Suppose for simplicity that

kai k D 1. Let Ci T WD H.ai ; ˇi / D fx 2 H W hai ; xi D ˇi g, i 2 I . By
assumption, C WD i 2I Ci ¤ ;. Let Ui WD PCi , i 2 I . We have Ui x D
x .hai ; xi ˇi /ai , i 2 I (see (4.1)). Of course, Fix Ui D Ci and Ui a
cutter as the metric projection (see Theorem 2.2.21 (ii)), i 2 I . Since the metric
projection is strictly
T quasi-nonexpansive
T (see Corollary 2.2.24), Theorem 2.1.26
yields Fix U D i 2I Fix Ui D i 2I Ci D C . We apply method (5.100) with
k WD max and k D 1 for all k 0 to system (5.107). By inequality (5.103), any
sequence generated by (5.100) is Fejér monotone with respect to C . By (4.123),
Pm
i D1 hSi x x; Si x Si 1 xi
max .x/ D .
kUx xk2
Denote u0 WD x and ui WD Ui ui 1 D Si x, i D 1; 2; : : : ; m. Since hUi y; ai i D ˇi

for any y 2 H and for all i 2 I , we have
hSi x; ai i D hUi ui 1 ; ai i D ˇi ,
i 2 I , consequently,
X
m X
m
hSi x x; Si x Si 1 xi D hSi x x; Ui ui 1 ui 1 i
i D1 i D1
X
m
D hSi x x; .ˇi hai ; ui 1 i/ai i
i D1
5.12 Surrogate Constraints Methods 263
X
m
D .ˇi hai ; ui 1 i/.hSi x; ai i hx; ai i/
i D1
X
m
D .ˇi hai ; xi/.ˇi hai ; ui 1 i/
i D1
and we obtain the following form for the step size

Pm
i D1 .ˇi hai ; xi/.ˇi hai ; ui 1 i/
max .x/ D , (5.108)
kUx xk2
where x … Fix U . It follows from Theorem 5.11.1 that for any x 0 2 H, the
sequence fx k g1
kD0 generated by the recurrence x
kC1
D Uk x k with k D max is
Fejér monotone with respect to C and converges weakly to a solution of the system
(5.107). Moreover, by Theorem 3.8.4, the sequence converges strongly.
Remark 5.11.2. For any u 2 H and z 2 Ci , we have hUi u u; zui D kUi u uk2 .
Therefore, it follows from the proof of Lemma 4.10.1 that in this case the first
inequality in (4.119) is, actually, an equality. Consequently, we have an equality
in (4.124) and the operator U with the step size function given by (4.123) has the
property hU x x; z U xi D 0 for all z 2 C , or, equivalently, hU x x; z xi D
kU x xk2 . Consequently, the step size function WD max is optimal in the
following sense:
kU x zk D min kx C ˛.Ux x/ zk .

˛
One can expect that this property leads in practice to an acceleration of the
convergence to a solution of the system (5.107), of sequences generated by the
recurrence x kC1 D Umax x k in comparison to the classical Kaczmarz method.
A different acceleration scheme for the Kaczmarz method was presented in [247,
Sect. 3.1].
5.12 Surrogate Constraints Methods
In this section we consider a consistent system of linear inequalities Ay b, where

A is a matrix of type m n and b 2 Rm . Let ai 2 Rn denote the i th row of
the matrix A and ˇi 2 R denote the i th coordinate of b, i.e., A D Œa1 ; : : : ; am >
and b D Œˇ1 ; : : : ; ˇm > . Suppose, for simplicity, that kai k D 1, i T
2 I . Denote
Ci WD H .ai ; ˇi / D fy 2 Rn W ai> y ˇi g, i 2 I , and C WD i 2I Ci . The
surrogate constraints method (SCM) is described by the recurrence
vk .x k /> .Ax k b/ > k k

x kC1 D x k k A v .x /, (5.109)
A> vk .x k /2
where x 0 2 Rn , k 2 Œ0; 2 and fvk g1

kD0 is a sequence of essential weight functions,
i.e., weight functions vk W Rn ! Rm nf0g satisfying (4.108) for any x … C . It
follows from Lemma 4.9.5 that recurrence (5.109) is well defined. We can also write
x kC1 D Svk ;k x k ,
where fSvk g1 kD0 is a sequence of surrogate projection operators Svk W R ! R

n n
defined by (4.109) and Svk ;k WD Id Ck .Svk Id/ is the k -relaxation of the oper-
ator Svk , k 0. Recall that Svk x k is the projection of x k onto a surrogate constraint
Hk WD H .ak .x k /; ˇk .x k //, where ak .x k / WD A> vk .x k / and ˇk .x k / WD vk .x k /> b
(see Sect. 4.9.4) and that Svk is a cutter with Fix Svk D C (see Theorem 4.9.6).
Various versions of the surrogate projection method were studied by Merzlyakov
[260], Oko [277], Yang and Murty [351], Kiwiel [229–231], Kiwiel and Łopuch
[233], Cegielski [67, 68] and by Dudek [150].
5.12.1 Proper Control
Definition 5.12.1. Let a sequence fx k g1 kD0 be generated by the surrogate con-

straints method (5.109). We say that a sequence of weight functions (or the control)
fvk g1
kD0 is proper if

lim vk .x k /> .Ax k b/ D 0 H) lim inf .Ax k b/C D 0 (5.110)
k k
Now we apply Definition 5.8.5 (ii) to V k WD V D U D fPCi gi 2I with Ci D

H .ai ; ˇi /, i 2 I . We see that fvk g1
kD0 is approximately semi-regular if and only if
there exists a sequence fik g1 kD0 I such that the following implication holds
lim vkik .x k /> Œ.ai>k x k ˇik /C 2 D 0 H) lim inf max.ai> x k ˇi /C D 0 (5.111)

k k i 2I
It follows from Remark 5.8.6 (i) and from the equivalence of all norms in Rn that
condition (5.111) is equivalent to

lim vkik .x k /> .ai>k x k ˇik /C D 0 H) lim inf .Ax k b/C D 0. (5.112)
k k
If vk considers only violated constraints (see Definition 4.9.8), then

X
vk .x k /> .Ax k b/ D vk .x k /> .Ax k b/C D vki .x k /> .ai> x k ˇi /C .
i 2I
Therefore, the assumption that a sequence of weight functions fvk g1 kD0 which
consider only violated constraints is approximately semi-regular is stronger than
(5.110). This fact follows from a simple observation that if a sum of positive
quantities goes to zero, then all summands go to zero.
If the weight functions vk consider only violated constraints, k 0, then it
follows from Proposition 4.9.11 that recurrence (5.109) defines an extrapolated
simultaneous projection method. In this case one can apply Theorem 5.10.2 in
order to prove the convergence of a sequence generated by (5.109). In the next
subsection we prove that the convergence follows without assumptions that vk
considers only violated constraints and that the corresponding sequence of weight
functions fwk g1
kD0 (see (4.112)) is approximately semi-regular. It suffices to suppose
instead that fvk g1
kD0 is proper.
Theorem 5.12.2. Let x 0 2 Rn and a sequence fx k g1 kD0 be generated by the

surrogate constraints method (5.109), where lim infk k .2 k / > 0 and the
sequence of weight functions fvk g1 k
kD0 is proper. Then x converges to a solution

x of the system Ax b.
Proof. The operator Svk is a cutter and Fix Svk D C , k 0 (see Theorem 4.9.6).
Therefore, equality (4.109) and Theorem 2.1.39 yield
" #2
kC1 2 k 2 vk .x k /> .Ax k b/
x
z x z k .2 k / ,
A> vk .x k /

for any z 2 C . Consequently, x k z converges as a decreasing sequence, x k is
bounded and
vk .x k /> .Ax k b/
lim D 0.
k A> vk .x k /
We can suppose without loss of generality that vk .x k / 2

m . Since x k is bounded,
A> vk .x k / is also bounded. Therefore, limk vk .x k /> .Ax k b/ D 0. By (5.110), we
have
X
m
2
lim inf Œ.ai> x k ˇi /C 2 D lim inf .Ax k b/C D 0:
k k
i D1
This implies

lim inf PCi x k x k D lim inf.ai> x k ˇi /C D 0
k k
for all i 2 I . Corollary 3.7.1 (iii) for X D Rn , Tk WD Svk and S WD PCi yields now
the convergence of x k to x 2 Fix PCi D Ci for any i 2 I , i.e., x 2 C . t
u
5.12.3 Examples of Proper Control
Now we give examples of proper sequences of weight functions vk W Rn ! Rm C nf0g.

We suppose for simplicity that A has normalized rows, i.e., kai k D 1, i 2 I .
Example 5.12.3. Let fx k g1 kD0 be generated by the surrogate constraints method
(5.109), where fvk g1 kD0 is a sequence of weight functions v W R ! RC nf0g
k n m
>
defined by v .x/ WD W .x/ .Ax b/C , x 2 R , k 0, with W WD diag wk
k k n k
and fwk g1 kD0 being a sequence of weight functions w W R !

m which are
k n
approximately semi-regular with respect to fPCi gi 2I with Ci WD H .ai ; ˇi /, i 2 I .

It is clear that vk considers only violated constraints, consequently, vk is essential
(see Lemma 4.9.9). We show that the sequence fvk g1 kD0 is proper. We have
vk .x k /> .Ax k b/ D .Ax k b/> C W .x /.Ax b/

k k k
X
D !ik .x k /Œ.ai> x k ˇi /C 2
i 2I
If limk vk .x k /> .Ax k b/ D 0, then, for any sequence fik g1

kD0 I , we have
lim !ikk .x k /Œ.ai>k x k ˇik /C 2 D 0.

k

Since fvk g1 k
kD0 is approximately semi-regular, lim infk PCi x x D 0, i.e.,
k
lim infk .ai> x k ˇi /C D 0 for all i 2 I , or, equivalently, lim infk .Ax k b/C D 0.
In particular, a constant sequence vk WD v is proper, where v W Rn ! Rm C nf0g is
defined by
v.x/ WD W .x/.Ax b/C ,
for x … C , W .x/ WD diag w.x/ and w has one of the following forms:
(i) w is constant with positive weights, i.e., w.x/ DW w 2 ri
m for all x 2 Rn .
Note that w is regular (see Example 5.8.3 (i)), consequently, it is approximately
regular (see Lemma 5.8.7 (i)). If we take w D . m1 ; m1 ; : : : ; m1 /, then we obtain
i .x/ D m1 .ai> x ˇi /C , i.e., the weights vi .x/ are proportional to the residua of
violated constraints. Since the rescaling of weights does not change the operator
Sv defined by (4.109), we can apply, equivalently, i .x/ WD .ai> x ˇi /C or
.ai> xˇi /C
i .x/ WD P > . The latter weights were proposed by Yang and Murty
j 2I .aj xˇj /C
[351, page 167].
(ii) w considers only almost violated constraints, i.e., !j .x/ D 0 for all j 2 I such
that
.aj> x ˇj /C < max.ai> x ˇi /C ,
i 2I
where 2 .0; 1. Note that w is approximately regular (see Example 5.8.3 (v)
and Lemma 5.8.7 (i)). If we take D 1, then the surrogate constraints method
reduces to a successive projection method with the remotest set control.

(5.109), where fvk g1
kD0 is a sequence containing a subsequence fvnk g1 kD0 of weight
functions which consider only almost remotest constraints, i.e., ink .x/ D 0 for all
i … J .x/; where
J .x/ WD fj 2 I W .aj> x ˇj /C max.ai> x ˇi /C g

i 2I
x 2 Rn , with 2 .0; 1. We show that fvk g1

kD0 is proper. Suppose for simplicity that
vk .x/ 2
m , x 2 Rn . Let jnk D jnk .x nk / 2 J .x nk / be such that jnnk .x nk / m1 ,
k
k 0. The existence of such j follows from the fact that vnk .x nk / 2
m . We have
X
vnk .x nk /> .Ax nk b/ D ink .x nk /.ai> x nk ˇi /
i 2I
X
D ink .x nk /.ai> x nk ˇi /C
i 2J .x nk /
jnnk .x nk /.aj>n x nk ˇjnk /C

k k
1
max.ai> x nk ˇi /C .
m i 2I
Therefore, limk vk .x k /> .Ax k b/ D 0 yields > k

lim infk maxi 2I .ai x ˇi /C D 0

which is equivalent to lim infk .Ax b/C D 0. Note that the successive pro-
k
jection method with the remotest set control is a special case of the method with
D 1.
(5.109), where fvk g1
kD0 is a sequence of essential weight functions containing a
subsequence fvnk g1
kD0 satisfying the following condition

ı for some i 2 Argmaxi 2I .ai> x nk ˇi /
ink .x nk / (5.113)
D 0 for i … I.x nk /,
for all x nk … C , where I.x/ WD fi 2 I W ai> x > ˇi g, k 0, and ı > 0 is a small

predefined constant. We show that fvk g1
kD0 is proper. Let x
nk
… C and jnk 2 I.x nk /
nk
be such that i .x / ı and
nk
.aj>n x nk ˇjnk /C D max.ai> x nk ˇi /C .

k i 2I
Then we have
X
vnk .x nk /> .Ax nk b/ D ink .x nk /.ai> x nk ˇi /
i 2I
X
D ink .x nk /.ai> x nk ˇi /C
i 2I.x nk /
jnnk .x nk /.aj>n x nk ˇjnk /C

k k
ı max.ai> x nk ˇi /C .
i 2I
Suppose that limk vk .x k /> .Axk b/ D 0. Then

lim infk max
> k
i 2I .ai x ˇi /C D 0
k 1
which is equivalent to lim infk .Ax b/C D 0, i.e., fv gkD0 is proper.
k
Remark 5.12.6. Yang and Murty proved the finite convergence of sequences
fx k g1
kD0 generated by the surrogate constraints method (5.109) with weight func-
tions vk satisfying the condition

ı for i 2 I" .x k /
ik .x k / (5.114)
D 0 for i … I" .x k /,
> k
T constantsn "; ı>> 0, where I" .x/ WD fi 2 I W ai x ˇi > "g
for predefined small
and x … C" WD i 2I fx 2 R W ai x ˇi > "g (see [351, Theorem 3.3]) Note that
k
if a sequence fvk g1 nk 1
kD0 satisfies condition (5.114), then any subsequence fv gkD0
satisfies condition (5.113) for all x … C" . We leave it to the reader to check that
nk
the result of [351, Theorem 3.3] can be derived from Example 5.12.5.
5.13 SCM with Residual Selection
In this section we present surrogate constraints methods for solving a consistent

system of linear inequalities Au b, which employ a special kind of surrogate
projection operators. For any x … C the surrogate projection Sv x given by (4.109)
is determined by an essential weight v D v.x/ (see Definition 4.9.4) related to a
constraints selection L WD L.x/ I WD f1; 2; : : : ; mg. We use the same notation
as in Sect. 5.12.
5.13.1 General Properties
We start with some relationships between surrogate projections and metric projec-
tions onto solution sets of equality and inequality systems. Let x … C be a current
approximation of the solution, L WD L.x/ I be a nonempty subset of indices (a
constraints selection) and let and r WDj L j. Let AL denote the submatrix of A with
rows ai , i 2 L, and bL —the subvector of b with coordinates ˇi , i 2 L. Further,
denote \
CL WD Ci D fu 2 Rn W AL u bL g
i 2L
5.13 SCM with Residual Selection 269
and
HL WD fu 2 Rn W AL u D bL g.
Suppose, for simplicity, that L D f1; 2; : : : ; rg, i.e.,

AL bL vL .x/
AD ,bD and v.x/ D .
AI nL bI nL vI nL .x/
Denote
vL .x/> .AL x bL / >
SvL x WD x > AL vL .x/
A vL .x/2
L
(cf. (4.109)).
Lemma 5.13.1. Let x … C and L WD L.x/ I be such that AL has full row rank.
Then PCL x D PHL x if and only if
.AL A> 1
L / .AL x bL / 0.
Proof. Consider the following convex minimization problem
minimize 12 ku xk2
subject to AL u D bL (5.115)
u 2 Rn
whose solution is uN D PHL x. The corresponding KKT-point .Nu; y/

N 2 Rn Rr has
the form
uN D x A> > 1
L .AL AL / .AL x bL /
yN D .AL A> 1
L / .AL x bL /
(i) Suppose that .AL A> 1

L / .AL x bL / 0. Then .N N is also a KKT-point
u; y/
related to the convex minimization problem
minimize 12 ku xk2
subject to AL u bL (5.116)
u 2 Rn
whose solution is u D PCL . Therefore, uN D u .

(ii) Suppose that uN D PHL x D PCL x, i.e.,
uN D x A> > 1
L .AL AL / .AL x bL /. (5.117)
Let y be a vector of Lagrange multipliers for problem (5.116). Then .Nu; y /

satisfies the related KKT-system, consequently,
uN D x A>
L y D 0.
This equality together with (5.117) give
A> > > 1

L y D AL .AL AL / .AL x bL /.
Since AL has full row rank, we have
y D .AL A> 1
L / .AL x bL /
which is nonnegative, because y is a vector of Lagrange multipliers for

problem (5.116). u
t
Lemma 5.13.2. Let x … C , L WD L.x/ I be such that AL has full row rank and
vL WD vL .x/ 2 RrC be essential for the system AL u bL i.e., v>
L .AL x bL / > 0.
Then SvL x D PHL x if and only if
vL D ˛.AL A> 1
L / .AL x bL / (5.118)
for some ˛ > 0. Consequently, if any of both conditions is satisfied, then SvL x D
PCL x.
Proof. ((H) Let vL be defined by (5.118). Applying (4.109) and (4.5) with A WD
AL and v.x/ WD vL .x/, we obtain
v>
L .AL x bL / >
S vL x D x > 2 AL vL
A vL
L
v>
L .AL x bL / >
Dx AL vL
v> >
L AL AL vL
D x A> > 1
L .AL AL / .AL x bL / D PHL x.
(H)) Suppose that SvL x D PHL x. By (4.5) and (4.109) with A WD AL , we have
v>
L .AL x bL / > > > 1
> 2 AL vL D AL .AL AL / .AL x bL /.
A vL
L
Since AL has full row rank,
vL D ˛.AL A> 1
L / .AL x bL /,
where > 2
A vL
L
˛D
v>
L .AL x bL /
which is positive, because vL is essential for the system AL u bL and A>

L vL ¤ 0
(see Lemma 4.9.5). u
t
5.13.2 Description of the Method
Now we describe one iteration of the surrogate constraints method with residual
selection. Suppose that x k is obtained in the kth iteration. The next iteration consists
of two steps:
Step 1 (Constraints selection). Determine a subset Lk WD L.x k / such that
(a) AL has full row rank and
(b) The vector
vLk WD vLk .x k / D .ALk A> 1

Lk / .ALk x bLk /
k
(5.119)
is nonnegative and essential for the system ALk u bLk .

Step 2 (Actualization). Evaluate
x kC1 D x k C k .SvLk x k x k /, (5.120)
where k 2 .0; 2/.

By Lemmas 5.13.1 and 5.13.2, we have SvLk x k D PCLk x k D PHLk x k .
A simple way to obtain a constraints selection satisfying conditions (a) and (b)
above is to take Lk WD L.x k / D fik g D fi.x k /g, where
ai> x k ˇi
ik WD argmaxf W i 2 I g, (5.121)
kai k
consequently, vk D v.x k / D ıik . It is clear that vk is essential for the system Au b.

The surrogate constraints method with Lk and vk defined by (5.121) reduces to the
successive projection method with the remotest set control.
It turns out that the surrogate constraints method with residual selection Lk
satisfying conditions (a) and (b) above provides steps which are no shorter than
in the latter method if ik 2 Lk . This property leads to the convergence of sequences
generated by (5.120).
Theorem 5.13.3. Let x 0 2 Rn and a sequence fx k g1 kD0 be generated by the
surrogate constraints method (5.120), where lim infk k .2 k / > 0, vLk is given
by (5.119), Lk is obtained by a residual selection and Lk contains ik defined by

(5.121). Then x k converges to a solution x of the system Ax b.
Proof. By Lemma 5.13.2 and by the inclusion CLk Cik , we have

SvLk x k x k D PCLk x k x k PCik x k x k PCi x k x k
for all i 2 I . Consequently, condition (3.11) with Tk WD SvLk and S WD PCi ,

i 2 I , is satisfied. Because ALk has full row rank, ALk A> Lk is positive definite
and vLk is essential. Since for any v the surrogate projection Sv is a cutter with
Fix Sv D C (see Theorem 4.9.6), Corollary 3.7.1 yields the convergence of x k to a
point x 2 C WD fx 2 Rn W Ax bg. t
u
In the next subsection we present several constructions of constraints selection
L.x/ for x … C leading to full row rank matrix AL for which the vector vL WD
.AL A> 1
L / .AL x bL / is nonnegative and essential for the system AL u bL . More
general constructions of this type and their applications to the convex minimization
problems can be found in [62–65, 72, 73, 154, 155, 230–232].
5.13.3 Obtuse Cone Selection
Let x … C . In an obtuse cone selection a subset L WD L.x/ is constructed recursively

in such a way that AL x > bL , AL has full row rank and .AL A> L/
1
0. Therefore,
the vector vL 2 R given by
r
vL WD .AL A> 1
L / .AL x b/
is nonnegative and essential for the system AL u bL . Consequently, the vector

v WD .vL ; vI nL / with vI nL D 0 2 Rmr is essential for the system Au b. The
name “obtuse cone selection” can be explained in the following way. Denote CL WD
conefai W i 2 Lg and suppose that AL has full row rank. Then CL is obtuse (in
Linfai W i 2 Lg) if and only if .AL A> L/
1
0 (see [232, Lemma 3.1] or [65,
Lemma 1.6]).
Denote K D K.x/ WD fi 2 I W ai> x > ˇi g. For any z 2 C and for w WD x z we
have AK w > 0. Let L K be a current selection such that the matrix AL has full
row rank and .AL A> L/
1
0. The following lemmas give conditions under which
0
the update L WD L [ flg K maintains the described above properties of AL , i.e.,
AL0 has full row rank and .AL0 A> L0 /
1
0. Denote

AL
AL0 WD .
ai>
Recall that AC
L denotes the Moore–Penrose pseudoinverse of AL which for a full
row matrix AL has the form AC > > 1
L D AL .AL AL / . The proof of the following
Lemma can be found in [65, Corollary 2.7].
Lemma 5.13.4. Let A> L0 w > 0 for some w 2 R . If AL has full row rank,
n
C
.AL A>
L/
1
0 and ai> AL 0, then AL0 has full row rank and .AL0 A>
L0 /
1
0.
Let x D x k … C . If L D flg, where l 2 K, then, of course, AL has full row rank
(as a nonzero vector al ) and .AL A>
L/
1
D kal k2 0. If there is i 2 K such that
>
ai al 0, then
2
ai> AC >
L D ai al kal k 0.
Applying Lemma 5.13.4 we see that L0 has the same properties as L: the matrix AL0
has full row rank and .AL0 A> L0 /
1
0. By repeated application of Lemma 5.13.4
with L WD L we obtain a new subset L0 K such that AL0 has full row rank and
0
.AL0 A>L0 /
1
0.
Let L WD L.x/ be constructed by an obtuse cone selection. Since AL x > bL ,
the vector vL WD .AL A> 1
L / .AL x b/ has nonnegative coordinates. Therefore, all
assumptions of Lemma 5.13.2 are satisfied, consequently, SvL x D PHL x D PCL x.
If in each iteration of (5.120) we start the above defined construction of the subset
a> x k ˇ
Lk WD L.x k / with lk WD argmaxf i kai k i W i 2 I g, then Theorem 5.13.3 yields the
convergence of sequences generated by (5.120) to a solution of the system Ax b.
5.13.4 Regular Obtuse Cone Selection
Now we present a special case of the obtuse cone selection. The corollary below
follows from Lemma 5.13.4. We use the same notation as in Sect. 5.13.3.
Corollary 5.13.5. Let A>
L0 w > 0 for some w 2 R . If AL has full row rank,
n
.AL AL / 0 and AL ai 0, then AL0 has full row rank and .AL0 A>
> 1
L0 /
1
0.
Proof. Suppose that AL has full row rank, .AL A>
L/
1
0 and AL ai 0. Then, by
Lemma 5.13.4, we have
ai> AC > > > 1

L D ai AL .AL AL / 0.
Consequently, AL0 has full row rank and .AL0 A>

L0 /
1
0. t
u
Let x … C . Applying Corollary 5.13.5 and starting with L D flg, where l 2 K,
one can construct recursively a maximal subset L K such that ai> aj 0 for all
i; j 2 L, i ¤ j . Then, of course, AL has full row rank and .AL A> L/
1
0. The
subset L constructed in this way is called a regular obtuse cone selection (ROCS).
The regular obtuse cone selection is easier to perform than the general obtuse
cone selection presented in Sect. 5.13.3 and the residual selection presented in
Sect. 5.13.2. Application of all these methods leads in practice to an essential
acceleration of the convergence, in particular, if the solution set is “flat” (see the
numerical results presented in [72, 73, 155]).
5.14 Exercises
Exercise 5.14.1. Show that Theorem 5.1.5 remains true if A and B are closed
affine subspaces with a common point.
Exercise 5.14.2. Show that the result of [351, Theorem 3.3] can be derived from
Example 5.12.5.
Exercise 5.14.3. Prove equalities (5.75) and (5.76).
References
1. S. Agmon, The relaxation method for linear inequalities. Can. J. Math. 6, 382–392 (1954)
2. R. Aharoni, A. Berman, Y. Censor, An interior point algorithm for the convex feasibility
problem. Adv. Appl. Math. 4, 479–489 (1983)
3. R. Aharoni, Y. Censor, Block-iterative projection methods for parallel computation of
solutions to convex feasibility problems. Lin. Algebra Appl. 120, 165–175 (1989)
4. D. Alevras, M.W. Padberg, Linear Optimization and Extensions. Problems and Solutions
(Springer, Berlin, 2001)
5. A. Aleyner, S. Reich, Block-iterative algorithms for solving convex feasibility problems in
Hilbert and in Banach spaces. J. Math. Anal. Appl. 343, 427–435 (2008)
6. A. Aleyner, S. Reich, Random products of quasi-nonexpansive mappings in Hilbert space.
J. Convex Anal. 16, 633–640 (2009)
7. M. Altman, On the approximate solution of linear algebraic equations. Bulletin de l’Académie
Polonaise des Sciences Cl. III 3 , 365–370 (1957)
8. I. Amemiya, T. Ando, Convergence of random products of contractions in Hilbert space. Acta
Sci. Math. (Szeged) 26, 239–244 (1965)
9. R. Ansorge, Connections between the Cimmino-method and the Kaczmarz-method for
solution of singular and regular systems of equations. Computing 33, 367–375 (1984)
10. G. Appleby, D.C. Smolarski, A linear acceleration row action method for projecting onto
subspaces. Electron. Trans. Numer. Anal. 20, 253–275 (2005)
11. N. Aronszajn, Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 337–404 (1950)
12. A. Auslender, Optimisation, méthodes numériques (Masson, Paris, 1976)
13. V.N. Babenko, Convergence of the Kaczmarz projection algorithm. Zh. Vychisl. Mat. Mat.
Fiz. 24, 1571–1573 (1984) (in Russian)
14. J.B. Baillon, R.E. Bruck, S. Reich, On the asymptotic behavior of nonexpansive mappings
and semigroups in Banach spaces. Houston J. Math. 4, 1–9 (1978)
15. S. Banach, Sur les opérations dans les ensembles abstraits et leur application aux équations
intégrals. Fundamenta Mathematicae 3, 133–181 (1922)
16. H.H. Bauschke, A norm convergence result on random products of relaxed projections in
Hilbert space. Trans. Am. Math. Soc. 347, 1365–1373 (1995)
17. H.H. Bauschke, Projection Algorithms and Monotone Operators, Ph.D. Thesis, Department
of Mathematics, Simon Fraser, University, Burnaby, BC, Canada, 1996
18. H.H. Bauschke, The approximation of fixed points of compositions of nonexpansive mapping
in Hilbert space. J. Math. Anal. Appl. 202, 150–159 (1996)
19. H.H. Bauschke, The composition of the projections onto closed convex sets in Hilbert space
is asymptotically regular. Proc. Am. Math. Soc. 131, 141–146 (2002)
Lecture Notes in Mathematics 2057, DOI 10.1007/978-3-642-30901-4,
276 References
20. H.H. Bauschke, J. Borwein, On the convergence of von Neumann’s alternating projection
algorithm for two sets. Set-Valued Anal. 1, 185–212 (1993)
21. H.H. Bauschke, J. Borwein, Dykstra’s alternating projection algorithm for two sets. J. Approx.
Theor. 79, 418–443 (1994)
22. H.H. Bauschke, J. Borwein, On projection algorithms for solving convex feasibility problems.
SIAM Rev. 38, 367–426 (1996)
23. H.H. Bauschke, J.M. Borwein, A.S. Lewis, The method of cyclic projections for closed
convex sets in Hilbert space. Contemp. Math. 204, 1–38 (1997)
24. H.H. Bauschke, P.L. Combettes, A weak-to-strong convergence principle for Fejér-monotone
methods in Hilbert spaces. Math. Oper. Res. 26, 248–264 (2001)
25. H.H. Bauschke, P.L. Combettes, S.G. Kruk, Extrapolation algorithm for affine-convex
feasibility problems. Numer. Algorithms 41, 239–274 (2006)
26. H.H. Bauschke, P.L. Combettes, D.R. Luke, Phase retrieval, error reduction algorithm, and
Fienup variants: A view from convex optimization. J. Opt. Soc. Am. A 19, 1334–1345 (2002)
27. H.H. Bauschke, P.L. Combettes, D.R. Luke, Hybrid projection-reflection method for phase
retrieval. J. Opt. Soc. Am. A 20, 1025–1034 (2003)
28. H.H. Bauschke, P.L. Combettes, D.R. Luke, Finding best approximation pairs relative to two
closed convex sets in Hilbert spaces. J. Approx. Theor. 127, 178–192 (2004)
29. H.H. Bauschke, P.L. Combettes, D.R. Luke, A strongly convergent reflection method for
finding the projection onto the intersection of two closed convex sets in a Hilbert space.
J. Approx. Theor. 141, 63–69 (2006)
30. H.H. Bauschke, F. Deutsch, H. Hundal, S-H. Park, Accelerating the convergence of the
method of alternating projections. Trans. Am. Math. Soc. 355, 3433–3461 (2003)
31. H.H. Bauschke, S.G. Kruk, Reflection-projection method for convex feasibility problems with
an obtuse cone. J. Optim. Theor. Appl. 120, 503–531 (2004)
32. H.H. Bauschke, E. Matoušková, S. Reich, Projection and proximal point methods: conver-
gence results and counterexamples. Nonlinear Anal. 56, 715–738 (2004)
33. M.H. Bazaraa, H.D. Sherali, C.M. Shetty, Nonlinear Programming, Theory and Algorithms,
3rd edn. (Wiley, Hoboken, 2006)
34. M. Benzi, C.D. Meyer, A direct projection method for sparse linear systems. SIAM J. Sci.
Comput. 16, 1159–1176 (1995)
35. A. Berman, R.J. Plemmons, Nonnegative Matrices in the Mathematical Sciences (Academic,
New York, 1979)
36. V. Berinde, Iterative Approximation of Fixed Points (Springer, Berlin, 2007)
37. M. Bertero, P. Boccacci, Introduction to Inverse Problems in Imaging (Institute of Physics
Publishing, Bristol, 1998)
38. D.P. Bertsekas, Nonlinear Programming (Athena Scientific, Belmont, 1995)
39. D. Blatt, A.O. Hero, Energy-based sensor network source localization via projection onto
convex sets (POCS). IEEE Trans. Signal Process. 54, 3614–3619 (2006)
40. J.M. Borwein, A. Lewis, Convex Analysis and Nonlinear Optimization, Theory and Examples
(Springer, New York, 2000)
41. R. Bramley, A. Sameh, Row projection methods for large nonsymmetric linear systems.
SIAM J. Sci. Stat. Comput. 13, 168–193 (1992)
42. L.M. Bregman, Finding the common point of convex sets by the method of successive
projection (in Russian). Dokl. Akad. Nauk SSSR 162, 487–490 (1965); English translation
in: Soviet Math. Dokl. 6, 688–692 (1965)
43. L.E.J. Brouwer, Über Abbildung von Mannigfaltigkeiten. Math. Ann. 71, 97–115 (1912)
44. F.E. Browder, Fixed-point theorems for noncompact mappings in Hilbert space. Proc. Nat.
Acad. Sci. USA 53, 1272–1276 (1965)
45. F.E. Browder, Nonexpansive nonlinear operators in a Banach space. Proc. Nat. Acad. Sci.
USA 54, 1041–1044 (1965)
46. F.E. Browder, Convergence of approximants to fixed points of nonexpansive nonlinear
mappings in Banach spaces. Arch. Rational Mech. Anal. 24, 82–90 (1967)
References 277
47. F.E. Browder, Convergence theorems for sequences of nonlinear operators in Banach spaces.
Math. Zeitschr. 100, 201–225 (1967)
48. F.E. Browder, W.V. Petryshyn, The solution by iteration of nonlinear functional equations in
Banach spaces. Bull. Am. Math. Soc. 72, 571–575 (1966)
49. R.E. Bruck, Nonexpansive projections on subsets of Banach spaces. Pac. J. Math. 47, 341–355
(1973)
50. R.E. Bruck, Random products of contractions in metric and Banach spaces. J. Math. Anal.
Appl. 88, 319–332 (1982)
51. R.E. Bruck, S. Reich, Nonexpansive projections and resolvents of accretive operators in
Banach spaces. Houston J. Math. 3, 459–470 (1977)
52. R.S. Burachik, J.O. Lopes, B.F. Svaiter, An outer approximation method for the variational
inequality problem. SIAM J. Contr. Optim. 43, 2071–2088 (2005)
53. D. Butnariu, Y. Censor, On the behavior of a block-iterative projection method for solving
convex feasibility problems. Int. J. Comp. Math. 34, 79–94 (1990)
54. D. Butnariu, Y. Censor, P. Gurfil, E. Hadar, On the behavior of subgradient projections
methods for convex feasibility problems in Euclidean spaces. SIAM J. Opt. 19, 786–807
(2008)
55. C. Byrne, Iterative oblique projection onto convex sets and the split feasibility problem.
Inverse Probl. 18, 441–453 (2002)
56. C. Byrne, A unified treatment of some iterative algorithms in signal processing and image
reconstruction. Inverse Probl. 20, 103–120 (2004)
57. C.L. Byrne, Applied Iterative Methods (AK Peters, Wellesley, 2008)
58. C.L. Byrne, Bounds on the largest singular value of a matrix and the convergence of
simultaneous and block-iterative algorithms for sparse linear systems. Int. Trans. Oper. Res.
16, 465–479 (2009)
59. T.D. Capricelli, P.L. Combettes, Parallel block-iterative reconstruction algorithms for binary
tomography. Electron. Notes Discr. Math. 20, 263–280 (2005)
60. G. Casssiani, G. Böhm, A. Vesnaver, R. Nicolich, A geostatistical framework for incorporat-
ing seismic tomography auxiliary data into hydraulic conductivity estimation. J. Hydrol. 206,
58–74 (1998)
61. J. Cea, Optimisation: théorie et algorithmes (Dunod, Paris, 1971); Polish translation:
Optymalizacja: Teoria i algorytmy (PWN, Warszawa, 1976)
62. A. Cegielski, Relaxation Methods in Convex Optimization Problems (in Polish). Monographs,
vol. 67, Institute of Mathematics, Higher College of Engineering, Zielona Góra, 1993
63. A. Cegielski, in Projection Onto an Acute Cone and Convex Feasibility Problems, ed. by
J. Henry i J.-P. Yvon. Lecture Notes in Control and Inform. Sci., vol. 197 (Springer, London,
1994), pp. 187–194
64. A. Cegielski, A method of projection onto an acute cone with level control in convex
minimization. Math. Program. 85, 469–490 (1999)
65. A. Cegielski, Obtuse cones and Gram matrices with nonnegative inverse. Lin. Algebra Appl.
335, 167–181 (2001)
66. A. Cegielski, A generalization of the Opial’s theorem. Contr. Cybern. 36, 601–610 (2007)
67. A. Cegielski, Convergence of the projected surrogate constraints method for the linear split
feasibility problems. J. Convex Anal. 14, 169–183 (2007)
68. A. Cegielski, Projection methods for the linear split feasibility problems. Optimization 57,
491–504 (2008)
69. A. Cegielski, Generalized relaxations of nonexpansive operators and convex feasibility
problems. Contemp. Math. 513, 111–123 (2010)
70. A. Cegielski, Y. Censor, in Opial-Type Theorems and the Common Fixed Point Problem, ed.
by H.H. Bauschke, R.S. Burachik, P.L. Combettes, V. Elser, D.R. Luke, H. Wolkowicz. Fixed-
Point Algorithms for Inverse Problems in Science and Engineering, Springer Optimization
and Its Applications, vol. 49 (Springer, New York, 2011), pp. 155–183
71. A. Cegielski, Y. Censor, Extrapolation and local acceleration of an iterative process for
common fixed point problems. J. Math. Anal. Appl. 394, 809–818 (2012)
278 References
72. A. Cegielski, R. Dylewski, Selection strategies in projection methods for convex minimization
problems. Discuss. Math. Differ. Incl. Contr. Optim. 22, 97–123 (2002)
73. A. Cegielski, R. Dylewski, Residual selection in a projection method for convex minimization
problems. Optimization 52, 211–220 (2003)
74. A. Cegielski, R. Dylewski, Variable target value relaxed alternating projection method.
Comput. Optim. Appl. 47, 455–476 (2010)
75. A. Cegielski, A. Suchocka, Incomplete alternating projection method for large inconsistent
linear systems. Lin. Algebra Appl. 428, 1313–1324 (2008)
76. A. Cegielski, A. Suchocka, Relaxed alternating projection methods. SIAM J. Optim. 19,
1093–1106 (2008)
77. A. Cegielski, R. Zalas, Methods for variational inequality problem over the intersection of
fixed point sets of quasi-nonexpansive operators (2012) Numer. Funct. Anal. Optim. (in print)
78. Y. Censor, Row-action methods for huge and sparse systems and their applications. SIAM
Rev. 23, 444–466 (1981)
79. Y. Censor, Iterative methods for convex feasibility problems. Ann. Discrete Math. 20, 83–91
(1984)
80. Y. Censor, An automatic relaxation method for solving interval linear inequalities. J. Math.
Anal. Appl. 106, 19–25 (1985)
81. Y. Censor, Parallel application of block-iterative methods in medical imaging and radiation
therapy. Math. Program. 42, 307–325 (1988)
82. Y. Censor, Binary steering in discrete tomography reconstruction with sequential and
simultaneous iterative algorithms. Lin. Algebra Appl. 339, 111–124 (2001)
83. Y. Censor, M.D. Altschuler, W.D. Powlis, On the use of Cimmino’s simultaneous projections
method for computing a solution of the inverse problem in radiation therapy treatment
planning. Inverse Probl. 4, 607–623 (1988)
84. Y. Censor, A. Ben-Israel, Y. Xiao, J.M. Galvin, On linear infeasibility arising in intensity-
modulated radiation therapy inverse planning. Lin. Algebra Appl. 428, 1406–1420 (2008)
85. Y. Censor, T. Bortfeld, B. Martin, A. Trofimov, A unified approach for inversion problems in
intensity-modulated radiation therapy. Phys. Med. Biol. 51, 2353–2365 (2006)
86. Y. Censor, P.P.B. Eggermont, D. Gordon, Strong underrelaxation in Kaczmarz’s method for
inconsistent systems. Numer. Math. 41, 83–92 (1983)
87. Y. Censor, T. Elfving, New methods for linear inequalities. Lin. Algebra Appl. 42, 199–211
(1982)
88. Y. Censor, T. Elfving, A multiprojection algorithm using Bregman projections in a product
space. Numer. Algorithm 8, 221–239 (1994)
89. Y. Censor, T. Elfving, Block-iterative algorithms with diagonal scaled oblique projections for
the linear feasibility problems. SIAM J. Matrix Anal. Appl. 24, 40–58 (2002)
90. Y. Censor, T. Elfving, Iterative algorithms with seminorm-induced oblique projections. Abstr.
Appl. Anal. 8, 387–406 (2003)
91. Y. Censor, T. Elfving, G.T. Herman, in Averaging Strings of Sequential Iterations for Convex
Feasibility Problems, ed. by D. Butnariu, Y. Censor, S. Reich. Inherently Parallel Algo-
rithms in Feasibility and Optimization and their Applications (Elsevier, Amsterdam, 2001),
pp. 101–113
92. Y. Censor, T. Elfving, G.T. Herman, T. Nikazad, On diagonally relaxed orthogonal projection
methods. SIAM J. Sci. Comput. 30, 473–504 (2008)
93. Y. Censor, T. Elfving, N. Kopf, T. Bortfeld, The multiple-sets split feasibility problem and its
applications for inverse problems. Inverse Probl. 21, 2071–2084 (2005)
94. Y. Censor, A. Gibali, Projections onto super-half-spaces for monotone variational inequality
problems in finite-dimensional spaces. J. Nonlinear Convex Anal. 9, 461–475 (2008)
95. Y. Censor, A. Gibali, S. Reich, The subgradient extragradient method for solving variational
inequalities in Hilbert space. J. Optim. Theory Appl. 148, 318–335 (2011)
96. Y. Censor, A. Gibali, S. Reich, Extensions of Korpelevich’s extragradient method for the
variational inequality problem in Euclidean space. Optimization 61, 1119–1132 (2012)
References 279
97. Y. Censor, A. Gibali, S. Reich, Strong convergence of subgradient extragradient methods

for the variational inequality problem in Hilbert space. Optim. Meth. Software 26, 827–845
(2011)
98. Y. Censor, D. Gordon, R. Gordon, Component averaging: An efficient iterative parallel
algorithm for large and sparse unstructured problems. Parallel Comput. 27, 777–808 (2001)
99. Y. Censor, D. Gordon, R. Gordon, BICAV: A block-iterative, parallel algorithm for sparse
systems with pixel-related weighting. IEEE Trans. Med. Imag. 20, 1050–1060 (2001)
100. Y. Censor, G.T. Herman, On some optimization techniques in image reconstruction from
projections. Appl. Numer. Math. 3, 365–391 (1987)
101. Y. Censor, A.N. Iusem, S.A. Zenios, An interior point method with Bregman functions for the
variational inequality problem with paramonotone operators. Math. Program. 81, 373–400
(1998)
102. Y. Censor, A. Lent, Cyclic subgradient projections. Math. Program. 24, 233–235 (1982)
103. Y. Censor, A. Motova, A. Segal, Perturbed projections and subgradient projections for the
multiple-sets split feasibility problem. J. Math. Anal. Appl. 327, 1244–1256 (2007)
104. Y. Censor, A. Segal, The split common fixed point problem for directed operators. J. Convex
Anal. 16, 587–600 (2009)
105. Y. Censor, A. Segal, On the string averaging method for sparse common fixed point problems.
Int. Trans. Oper. Res. 16, 481–494 (2009)
106. Y. Censor, A. Segal, Sparse string-averaging and split common fixed points. Contemp. Math.
513, 125–142 (2010)
107. Y. Censor, E. Tom, Convergence of string-averaging projection schemes for inconsistent
convex feasibility problems. Optim. Meth. Software 18, 543–554 (2003)
108. Y. Censor, S.A. Zenios, Parallel Optimization, Theory, Algorithms and Applications (Oxford
University Press, New York, 1997)
109. A.E. Çetin, H. Özaktaş, H.M. Ozaktas, Resolution enhancement of low resolution wavefields
with POCS algorithm. Electron. Lett. 9, 1808–1810 (2003)
110. W. Chen, D. Craft, T.M. Madden, K. Zhang, H.M. Kooy, G.T. Herman, A fast optimization
algorithm for multicriteria intensity modulated proton therapy planning. Med. Phys. 7, 4938–
4945 (2010)
111. W. Chen, G.T. Herman, Efficient controls for finitely convergent sequential algorithms. ACM
Trans. Math. Software 37, 1–23 (2010)
112. W. Cheney, A.A. Goldstein, Proximity maps for convex sets. Proc. Am. Math Soc. 10, 448–
450 (1959)
113. C.E. Chidume, Quasi-nonexpansive mappings and uniform asymptotic regularity. Kobe J.
Math. 3, 29–35 (1986)
114. Ch. Chidume, Geometric Properties of Banach Spaces and Nonlinear Iterations (Springer,
London, 2009)
115. H. Choi, R.G. Baraniuk, Multiple wavelet basis image denoising using Besov ball projections.
IEEE Signal Process. Lett. 11, 717–720 (2004)
116. G. Cimmino, Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari.
La Ricerca Scientifica, II 9, 326–333 (1938)
117. P.L. Combettes, Inconsistent signal feasibility problems: Least-square solutions in a product
space. IEEE Trans. Signal Process. 42, 2955–2966 (1994)
118. P.L. Combettes, in The Convex Feasibility Problem in Image Recovery, ed. by P. Hawkes.
Advances in Imaging and Electron Physics, vol. 95 (Academic, New York, 1996),
pp. 155–270
119. P.L. Combettes, Hilbertian convex feasibility problem: Convergence of projection methods.
Appl. Math. Optim. 35, 311–330 (1997)
120. P.L. Combettes, Convex set theoretic image recovery by extrapolated iterations of parallel
subgradient projections. IEEE Trans. Image Process. 6, 493–506 (1997)
121. P.L. Combettes, in Quasi-Fejérian Analysis of Some Optimization Algorithm, ed. by
D. Butnariu, Y. Censor, S. Reich. Inherently Parallel Algorithms in Feasibility and Optimiza-
tion and their Applications (Elsevier, Amsterdam, 2001), pp. 115–152
280 References
122. P.L. Combettes, Solving monotone inclusions via compositions of nonexpansive averaged
operators. Optimization 53, 475–504 (2004)
123. P.L. Combettes, P. Bondon, Hard-constrained inconsistent signal feasibility problems. IEEE
Trans. Signal Process. 47, 2460–2468 (1999)
124. P.L. Combettes, S.A. Hirstoaga, Equilibrium programming in Hilbert spaces. J. Nonlinear
Convex Anal. 6, 117–136 (2005)
125. P.L. Combettes, H. Puh, Iterations of parallel convex projections in Hilbert spaces. Numer.
Funct. Anal. Optim. 15, 225–243 (1994)
126. G. Crombez, A geometrical look at iterative methods for operators with fixed points. Numer.
Funct. Anal. Optim. 26, 157–175 (2005)
127. G. Crombez, A hierarchical presentation of operators with fixed points on Hilbert spaces.
Numer. Funct. Anal. Optim. 27, 259–277 (2006)
128. Y.-H. Dai, Fast algorithms for projection on an ellipsoid. SIAM J. Optim. 16, 986–1006
(2006)
129. A. Dax, A note of the convergence of linear stationary iterative process. Lin. Algebra Appl.
129, 131–142 (1990)
130. A. Dax, Linear search acceleration of iterative methods. Lin. Algebra Appl. 130, 43–63 (1990)
131. A. Dax, On hybrid acceleration of a linear stationary iterative process. Lin. Algebra Appl.
130, 99–110 (1990)
132. A. Dax, The convergence of linear stationary iterative processes for solving singular
unstructured systems of linear equations. SIAM Rev. 32, 611–635 (1990)
133. L. Debnath, P. Mikusiński, Hilbert Spaces with Applications, 2nd edn. (Academic, San Diego,
1999)
134. A.R. De Pierro, A.N. Iusem, A simultaneous projections method for linear inequalities. Lin.
Algebra Appl. 64, 243–253 (1985)
135. A.R. De Pierro, A.N. Iusem, A parallel projection method of finding a common point of a
family of convex sets. Pesquisa Operacional 5, 1–20 (1985)
136. A.R. De Pierro, A.N. Iusem, A finitely convergent “row-action” method for the convex
feasibility problem. Appl. Math. Optim. 17, 225–235 (1988)
137. A.R. De Pierro, A.N. Iusem, On the asymptotic behavior of some alternate smoothing series
expansion iterative methods. Lin. Algebra Appl. 130, 3–24 (1990)
138. F. Deutsch, in Applications of von Neumann’s Alternating Projections Algorithm, ed. by
P. Kenderov. Mathematical Methods in Operations Research (Sophia, Bulgaria, 1983), pp.
44–51
139. F. Deutsch, in The Method of Alternating Orthogonal Projections, ed. by S.P. Singh. Approx-
imation Theory, Spline Functions and Applications (Kluwer Academic, The Netherlands,
1992), pp. 105–121
140. F. Deutsch, Best Approximation in Inner Product Spaces (Springer, New York, 2001)
141. F. Deutsch, in Accelerating the Convergence of the Method of Alternating Projections via
a Line Search: A Brief Survey, ed. by D. Butnariu, Y. Censor, S. Reich. Inherently Parallel
Algorithms in Feasibility and Optimization and their Application, Studies in Computational
Mathematics, vol. 8 (Elsevier Science, Amsterdam, 2001), pp. 203–217
142. F. Deutsch, H. Hundal, The rate of convergence for the cyclic projections algorithm, I. Angles
between convex sets. J. Approx. Theor. 142, 36–55 (2006)
143. F. Deutsch, H. Hundal, The rate of convergence for the cyclic projections algorithm, II. Norms
of nonlinear operators. J. Approx. Theor. 142, 56–82 (2006)
144. F. Deutsch, I. Yamada, Minimizing certain convex functions over the intersection of the fixed
point sets of nonexpansive mappings. Numer. Funct. Anal. Optim. 19, 33–56 (1998)
145. J.B. Diaz, F.T. Metcalf, On the set of subsequential limit points of successive approximations.
Trans. Am. Math. Soc. 135, 459–485 (1969)
146. L.T. Dos Santos, A parallel subgradient projections method for the convex feasibility problem.
J. Comput. Appl. Math. 18, 307–320 (1987)
147. W.G. Dotson Jr., On the Mann iterative process. Trans. Am. Math. Soc. 149, 65–73 (1970)
References 281
148. W.G. Dotson, Fixed points of quasi-nonexpansive mappings. J. Austral. Math. Soc. 13, 167–
170 (1972)
149. J. Douglas, H.H. Rachford, On the numerical solution of heat conduction problems in two or
three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)
150. R. Dudek, Iterative method for solving the linear feasibility problem. J. Optim Theor. Appl.
132, 401–410 (2007)
151. J. Dye, M.A. Khamsi, S. Reich, Random products of contractions in Banach spaces. Trans.
Am. Math. Soc. 325, 87–99 (1991)
152. J.M. Dye, S. Reich, On the unrestricted iteration of projections in Hilbert space. J. Math.
Anal. Appl. 156, 101–119 (1991)
153. J. Dye, S. Reich, Unrestricted iterations of nonexpansive mappings in Hilbert space.
Nonlinear Anal. 18, 199–207 (1992)
154. R. Dylewski, Selection of Linearizations in Projection Methods for Convex Optimization
Problems (in Polish), Ph.D. thesis, University of Zielona Góra, Institute of Mathematics, 2003
155. R. Dylewski, Projection method with residual selection for linear feasibility problems.
Discuss. Math. Differ. Incl. Contr. Optim. 27, 43–50 (2007)
156. M.G. Eberle, M.C. Maciel, Finding the closest Toeplitz matrix. Computat. Appl. Math. 22,
1–18 (2003)
157. J. Eckstein, D.P. Bertsekas, On the Douglas–Rachford splitting method and the proximal point
algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)
158. I. Ekeland, R. Témam, Convex Analysis and Variational Problems (North-Holland, Amster-
dam, 1976)
159. T. Elfving, A projection method for semidefinite linear systems and its applications. Lin.
Algebra Appl. 391, 57–73 (2004)
160. T. Elfving, T. Nikazad, Stopping rules for Landweber-type iteration. Inverse Probl. 23, 1417–
1432 (2007)
161. V. Elser, I. Rankenburg, P. Thibault, Searching with iterated maps. Proc. Natl. Acad. Sci. USA
104, 418–423 (2007)
162. L. Elsner, I. Koltracht, P. Lancaster, Convergence properties of ART and SOR algorithms.
Numer. Math. 59, 91–106 (1991)
163. L. Elsner, I. Koltracht, M. Neumann, On the convergence of asynchronous paracontractions
with application to tomographic reconstruction from incomplete data. Lin. Algebra Appl. 130,
65–82 (1990)
164. L. Elsner, I. Koltracht, M. Neumann, Convergence of sequential and asynchronous nonlinear
paracontractions. Numer. Math. 62, 305–319 (1992)
165. F. Facchinei, J.-S. Pang, Finite-Dimensional Variational Inequalities and Complementarity
Problems, Volume I, Volume II (Springer, New York, 2003)
166. M. Fiedler, V. Pták, On matrices with non-positive off-diagonal elements and positive
principal minors. Czech. Math. J. 12, 382–400 (1962)
167. S. Fitzpatrick, R.R. Phelps, Differentiability of the metric projection in Hilbert space. Trans.
Am. Math. Soc. 270, 483–501 (1982)
168. S.D. Flåm, J. Zowe, Relaxed outer projections, weighted averages and convex feasibility. BIT
30, 289–300 (1990)
169. R. Fletcher, Practical Methods of Optimization (Wiley, Chichester, 1987)
170. K. Friedricks, On certain inequalities and characteristic value problems for analytic functions
and for functions of two variables. Trans. Am. Math. Soc. 41, 321–364 (1937)
171. M. Fukushima, A relaxed projection method for variational inequalities. Math. Program. 35,
58–70 (1986)
172. E.M. Gafni, D.P. Bertsekas, Two metric projection methods for constrained optimization.
SIAM J. Contr. Optim. 22, 936–964 (1984)
173. A. Galántai, Projectors and Projection Methods (Kluwer Academic, Boston, 2004)
174. A. Galántai, On the rate of convergence of the alternating projection method in finite
dimensional spaces. J. Math. Anal. Appl. 310, 30–44 (2005)
282 References
175. U. Garcı́a-Palomares, Parallel projected aggregation methods for solving the convex feasibil-
ity problem. SIAM J. Optim. 3, 882–900 (1993)
176. U. Garcı́a-Palomares, A superlinearly convergent projection algorithm for solving the convex
inequality problem. Oper. Res. Lett. 22, 97–103 (1998)
177. W.B. Gearhart, M. Koshy, Acceleration schemes for the method of alternating projections.
178. C. Geiger, Ch. Kanzow, Numerische Verfahren zur Lösung unrestingierter Optimierungsauf-
gaben (Springer, Berlin, 1999)
179. C. Geiger, Ch. Kanzow, Theorie und Numerik restringierter Optimierungsaufgaben (Springer,
Berlin, 2002)
180. J.R. Giles, Convex Analysis with Application in Differentiation of Convex Functions (Pitman
Advanced Publishing Program, Boston, 1982)
181. P.E. Gill, W. Murray, M.H. Wright, Numerical Linear Algebra and Optimization (Addison-
Wesley, Redwood City, 1991)
182. W. Glunt, T.L. Hayden, R. Reams, The nearest ‘doubly stochastic’ matrix to a real matrix
with the same first moment. Numer. Lin. Algebra Appl. 5, 475–482 (1998)
183. K. Goebel, Concise Course on Fixed Points Theorems (Yokohama Publishing, Yokohama,
2002); Polish translation: Twierdzenia o punktach stałych (Wydawnictwo UMCS, Lublin,
2005)
184. K. Goebel, W.A. Kirk, Topics in Metric Fixed Point Theory (Cambridge University
Press, Cambridge, 1990); Polish translation: Zagadnienia metrycznej teorii punktów stałych
(Wydawnictwo UMCS, Lublin, 1999)
185. K. Goebel, S. Reich, Uniform Convexity, Hyperbolic Geometry, and Nonexpansive Mappings
(Marcel Dekker, New York, 1984)
186. J.L. Goffin, The relaxation method for solving systems of linear inequalities. Math. Oper. Res.
5, 388–414 (1980)
187. J.L. Goffin, On the finite convergence of the relaxation method for solving systems of
inequalities. Operations Research Center, Report ORC 71–36, University of California,
Berkeley, 1971
188. D. Göhde, Zum Prinzip der kontraktiven Abbildung. Math. Nachr. 30, 251–258 (1965)
189. R. Gordon, R. Bender, G.T. Herman, Algebraic reconstruction techniques (ART) for three-
dimensional electron microscopy and X-ray photography. J. Theoret. Biol. 29, 471–481
(1970)
190. D. Gordon, R. Gordon, Component-averaged row projections: a robust, block-parallel scheme
for sparse linear systems. SIAM J. Sci. Comput. 27, 1092–1117 (2005)
191. A. Granas, J. Dugundji, Fixed Point Theory (Springer, New York, 2003)
192. K.M. Grigoriadis, A.E. Frazho, R.E. Skelton, Application of alternating convex projection
methods for computation of positive Toeplitz matrices. IEEE Trans. Signal Process. 42, 1873–
1875 (1994)
193. K.M. Grigoriadis, R.E. Skelton, Low-order control design for LMI problems using alternating
projection methods. Automatica 32, 1117–1125 (1996)
194. K.M. Grigoriadis, R.E. Skelton, Alternating convex projection methods for discrete-time
covariance control design. J. Optim. Theor. Appl. 88, 399–432 (1996)
195. J. Gu, H. Stark, Y. Yang, Wide-band smart antenna design using vector space projection
methods. IEEE Trans. Antenn. Propag. 52, 3228–3236 (2004)
196. L.G. Gurin, B.T. Polyak, E.V. Raik, The method of projection for finding the common point in
convex sets. Zh. Vychisl. Mat. Mat. Fiz. 7, 1211–1228 (1967) (in Russian); English translation
in: USSR Comput. Math. Phys. 7, 1–24 (1967)
197. R. Haller, R. Szwarc, Kaczmarz algorithm in Hilbert space. Studia Math. 169, 123–132 (2005)
198. I. Halperin, The product of projection operators. Acta Sci. Math. (Szeged) 23, 96–99 (1962)
199. H.W. Hamacher, K.-H. Küfer, Inverse radiation therapy planning – a multiple objective
optimization approach. Discrete Appl. Math. 118, 145–161 (2002)
200. S.-P. Han, A successive projection method. Math. Program. (Ser. A) 40, 1–14 (1988)
201. M. Hanke, W. Niethammer, On the acceleration of Kaczmarz’s method for inconsistent linear
systems. Lin. Algebra Appl. 130, 83–98 (1990)
References 283
202. Y. Haugazeau, Sur les inéquations variationnelles et la minimisation de fonctionnelles

convexes (Thèse, Université de Paris, Paris, 1968)
203. H. He, S. Liu, H. Zhou, An explicit method for finding common solutions of variational
inequalities and systems of equilibrium problems and fixed points of an infinite family of
nonexpansive mappings. Nonlinear Anal. 72, 3124–3135 (2010)
204. G.T. Herman, A relaxation method for reconstructing objects from noisy X-rays. Math.
Program. 8, 1–19 (1975)
205. G.T. Herman, Fundamentals of Computerized Tomography: Image Reconstruction from
Projections, 2nd edn. (Springer, London, 2009)
206. G.T. Herman, W. Chen, A fast algorithm for solving a linear feasibility problem with
application to intensity-modulated radiation therapy. Lin. Algebra Appl. 428, 1207–1217
(2008)
207. N.J. Higham, Computing a nearest symmetric positive semidefinite matrix. Lin. Algebra
Appl. 103, 103–118 (1988)
208. N.J. Higham, Computing the nearest correlation matrix – a problem from finance. IMA
J. Numer. Anal. 22, 329–343 (2002)
209. J.-B. Hiriart-Urruty, C. Lemaréchal, Convex Analysis and Minimization Algorithms, Vol I, Vol
II (Springer, Berlin, 1993)
210. J.-B. Hiriart-Urruty, C. Lemaréchal, Fundamentals of Convex Analysis (Springer, Berlin,
2001)
211. S.A. Hirstoaga, Iterative selection methods for common fixed point problems. J. Math. Anal.
Appl. 324, 1020–1035 (2006)
212. J. Höffner, P. Decker, E.L. Schmidt, W. Herbig, J. Rittler, P. Weiß, Development of a fast
optimization preview in radiation treatment planning. Strahlentherapie und Onkologie 172,
384–394 (1996)
213. H.S. Hundal, An alternating projection that does not converge in norm. Nonlinear Anal. 57,
35–61 (2004)
214. J.K. Hunter, B. Nachtergaele, Applied Analysis (World Scientific, Singapore, 2000)
215. A.N. Iusem, A.R. De Pierro, Convergence results for an accelerated nonlinear Cimmino
algorithm. Numer. Math. 49, 367–378 (1986)
216. A.N. Iusem, A.R. De Pierro, A simultaneous iterative method for computing projections on
polyhedra. SIAM J. Contr. Optim. 25, 231–243 (1987)
217. A.N. Iusem, A.R. De Pierro, On the convergence properties of Hildreth’s quadratic program-
ming algorithm. Math. Program. (Ser. A) 47, 37–51 (1990)
218. A.N. Iusem, B.F. Svaiter, A row-action method for convex programming. Math. Program. 64,
149–171 (1994)
219. B.K. Jennison, J.P. Allebach, D.W. Sweeney, Iterative approaches to computer-generated
holography. Opt. Eng. 28, 629–637 (1989)
220. M. Jiang, G. Wang, Development of iterative algorithms for image reconstruction. J. X-Ray
Sci. Tech. 10, 77–86 (2002)
221. M. Jiang, G. Wang, Convergence studies on iterative algorithms for image reconstruction.
IEEE Trans. Med. Imag. 22, 569–579 (2003)
222. B. Johansson, T. Elfving, V. Kozlov, Y. Censor, P.-E. Forssén, G. Granlund, The application
of an oblique-projected Landweber method to a model of supervised learning. Math. Comput.
Model. 43, 892–909 (2006)
223. S. Kaczmarz, Angenäherte Auflösung von Systemen linearer Gleichungen. Bulletin Interna-
tional de l’Académie Polonaise des Sciences et des Lettres A35, 355–357 (1937); English
translation: S. Kaczmarz, Approximate solution of systems of linear equations. Int. J. Contr.
57, 1269–1271 (1993)
224. A.C. Kak, M. Slaney, Principles of Computerized Tomographic Imaging (IEEE, New York,
1988)
225. I.G. Kazantsev, S. Schmidt, H.F. Poulsen, A discrete spherical x-ray transform of orientation
distribution functions using bounding cubes. Inverse Probl. 25, 105009 (2009)
284 References
226. D. Kinderlehrer, G. Stampacchia, An Introduction to Variational Inequalities and Their

Applications (Academic, New York, 1980)
227. W.A. Kirk, A fixed point theorem for mappings which do not increase distances. Am. Math.
Mon. 72, 1004–1006 (1965)
228. Yu.N. Kiseliov, Algorithms of projection of a point onto an ellipsoid. Lithuanian Math. J. 34,
141–159 (1994)
229. K.C. Kiwiel, Block-iterative surrogate projection methods for convex feasibility problems.
Lin. Algebra Appl. 215, 225–259 (1995)
230. K.C. Kiwiel, The efficiency of subgradient projection methods for convex optimization.
I. General level methods. SIAM J. Contr. Optim. 34, 660–676 (1996)
231. K.C. Kiwiel, The efficiency of subgradient projection methods for convex optimization,
II. Implementations and extensions. SIAM J. Contr. Optim. 34, 677–697 (1996)
232. K.C. Kiwiel, Monotone Gram matrices and deepest surrogate inequalities in accelerated
relaxation methods for convex feasibility problems. Lin. Algebra Appl. 252, 27–33 (1997)
233. K.C. Kiwiel, B. Łopuch, Surrogate projection methods for finding fixed points of firmly
nonexpansive mappings. SIAM J. Opt. 7, 1084–1102 (1997)
234. A. Kiełbasiński, H. Schwetlick, Numerical Linear Algebra (in German) (Verlag Harri
Deutsch, Thun, 1988); Polish translation: Numeryczna algebra liniowa (WNT, Warszawa,
1992)
235. E. Kopecká, S. Reich, A note on the von Neumann alternating projections algorithm.
J. Nonlinear Convex Anal. 5, 379–386 (2004)
236. E. Kopecká, S. Reich, Another note on the von Neumann alternating projections algorithm.
J. Nonlinear Convex Anal. 11, 455–460 (2010)
237. G.M. Korpelevich, The extragradient method for finding saddle points and other problems.
Ekonomika i Matematicheskie Metody 12, 747–756 (1976)
238. M.A. Krasnosel’skiı̆, Two remarks on the method of successive approximations (in Russian).
Uspehi Mat. Nauk 10, 123–127 (1955)
239. S. Kwapień, J. Mycielski, On the Kaczmarz algorithm of approximation in infinite-
dimensional spaces. Studia Math. 148, 5–86 (2001)
240. L. Landweber, An iteration formula for Fredholm integral equations of the first kind.
Am. J. Math. 73, 615–624 (1951)
241. S. Lee, P.S. Cho, R.J. Marks, S.Oh, Conformal radiotherapy computation by the method of
alternating projections onto convex sets. Phys. Med. Biol. 42, 1065–1086 (1997)
242. S.-H. Lee, K.-R. Kwon, Mesh watermarking based projection onto two convex sets. Multi-
media Syst. 13, 323–330 (2008)
243. A. Lent, in A Convergent Algorithm for Maximum Entropy Image Restoration with a Medical
X-ray Application, ed. by R. Shaw. Image Analysis and Evaluation (SPSE, Washington DC),
pp. 249–257
244. A. Lent, Y. Censor, Extensions of Hildreth’s row-action method for quadratic programming.
SIAM J. Contr. Optim. 18, 444–454 (1980)
245. A.W.-C. Liew, H. Yan, N.-F. Law, POCS-based blocking artifacts suppression using a
smoothness constraint set with explicit region modeling. IEEE Trans. Circ. Syst. Video Tech.
15, 795–800 (2005)
246. P.L. Lions, B. Mercier, Splitting algorithms for the sum of two nonlinear operators. SIAM
J. Numer. Anal. 16, 964–979 (1979)
247. C. Liu, An acceleration scheme for row projection methods. J. Comput. Appl. Math. 57, 363–
391 (1995)
248. Y.M. Lu, M. Karzand, M. Vetterli, Demosaicking by alternating projections: theory and fast
one-step implementation. IEEE Trans. Image Process. 19, 2085–2098 (2010)
249. P.-E. Maingé, Inertial iterative process for fixed points of certain quasi-nonexpansive map-
pings. Set-Valued Anal. 15, 67–79 (2007)
250. P.-E. Maingé, Extension of the hybrid steepest descent method to a class of variational
inequalities and fixed point problems with nonself-mappings. Numer. Funct. Anal. Optim.
29, 820–834 (2008)
References 285
251. P.-E. Maingé, New approach to solving a system of variational inequalities and hierarchical
problems. J. Optim. Theor. Appl. 138, 459–477 (2008)
252. W.R. Mann, Mean value methods in iteration. Proc. Am. Math. Soc. 4, 506–510 (1953)
253. Şt. Măruşter, The solution by iteration of nonlinear equations in Hilbert spaces. Proc. Am.
Math. Soc. 63, 69–73 (1977)
254. Şt. Măruşter, Quasi-nonexpansivity and the convex feasibility problem. An. Ştiinţ. Univ. Al.
I. Cuza Iaşi Inform. (N.S.) 15, 47–56 (2005)
255. Şt. Măruşter, C. Popı̂rlan, On the Mann-type iteration and the convex feasibility problem.
256. Şt. Măruşter, C. Popı̂rlan, On the regularity condition in a convex feasibility problem.
Nonlinear Anal. 70, 1923–1928 (2009)
257. E. Masad, S. Reich, A note on the multiple-set split convex feasibility problem in Hilbert
space. J. Nonlinear Convex Anal. 8, 367–371 (2007)
258. E. Matoušková, S. Reich, The Hundal example revisited. J. Nonlinear Convex Anal. 4, 411–
427 (2003)
259. S.F. McCormick, The methods of Kaczmarz and row orthogonalization for solving linear
equations and least squares problems in Hilbert space. Indiana Univ. Math. J. 26, 1137–1150
(1977)
260. Yu.I. Merzlyakov, On a relaxation method of solving systems of linear inequalities (in
Russian). Zh. Vychisl. Mat. Mat. Fiz. 2, 482–487 (1962)
261. D. Michalski, Y. Xiao, Y. Censor, J.M. Galvin, The dose-volume constraint satisfaction
problem for inverse treatment planning with field segments. Phys. Med. Biol. 49, 601–616
(2004)
262. W. Mlak, Introduction to Hilbert Spaces (in Polish) (PWN, Warsaw, 1982)
263. W. Mlak, Hilbert Spaces and Operator Theory (Kluwer Academic, Boston, 1991)
264. J.-J. Moreau, Décomposition orthogonale d’un espace hilbertien selon deux cônes mutuelle-
ment polaires. C. R. Acad. Sci. Paris 255, 238–240 (1962)
265. J. Moreno, B. Datta, M. Raydan, A symmetry preserving alternating projection method for
matrix model updating. Mech. Syst. Signal Process. 23, 1784–1791 (2009)
266. T.S. Motzkin, I.J. Schoenberg, The relaxation method for linear inequalities. Can. J. Math. 6,
393–404 (1954)
267. J. Musielak, Introduction to Functional Analysis (in Polish) (PWN, Warszawa, 1989)
268. J. Mycielski, S. Świerczkowski, Uniform approximation with linear combinations of repro-
ducing kernels. Studia Math. 121, 105–114 (1996)
269. N. Nadezhkina, W. Takahashi, Strong convergence theorem by a hybrid method for nonexpan-
sive mappings and Lipschitz-continuous monotone mappings. SIAM J. Optim. 16, 1230–1241
(2006)
270. F. Natterer, The Mathematics of Computerized Tomography (Wiley, Chichester, 1986)
271. J. von Neumann, in Functional Operators – Vol. II. The Geometry of Orthogonal Spaces.
Annals of Mathematics Studies, vol. 22 (Princeton University Press, Princeton, 1950) (Reprint
of mimeographed lecture notes first distributed in 1933)
272. O. Nevanlinna, S. Reich, Strong convergence of contraction semigroups and of iterative
methods for accretive operators in Banach spaces. Israel J. Math. 32, 44–58 (1979)
273. N. Ogura, I. Yamada, Non-strictly convex minimization over the fixed point set of an
asymptotically shrinking nonexpansive mapping. Numer. Funct. Anal. Optim. 23, 113–137
(2002)
274. N. Ogura, I. Yamada, Nonstrictly convex minimization over the bounded fixed point set of a
nonexpansive mapping. Numer. Funct. Anal. Optim. 24, 129–135 (2003)
275. S. Oh, R.J. Marks, L.E. Atlas, Kernel synthesis for generalized time-frequency distributions
using the method of alternating projections onto convex sets. IEEE Trans. Signal Process. 42,
1653–1661 (1994)
276. J.G. O’Hara, P. Pillay, H.-K. Xu, Iterative approaches to finding nearest common fixed points
of nonexpansive mappings in Hilbert spaces. Nonlinear Anal. 54, 1417–1426 (2003)
277. S.O. Oko, Surrogate methods for linear inequalities. J. Optim. Theor. Appl. 72, 247–268
(1992)
286 References
278. Z. Opial, Nonexpansive and Monotone Mappings in Banach Spaces. Lecture Notes 67-1,
Center for Dynamical Systems, Brown University, Providence, RI, 1967
279. Z. Opial, Weak convergence of the sequence of successive approximations for nonexpansive
mappings. Bull. Am. Math. Soc. 73, 591–597 (1967)
280. S.C. Park, M.K. Park, M.G. Kang, Super-resolution image reconstruction: A technical
overview. IEEE Signal Process. Mag. 20, 21–36 (2003)
281. J. Park, D.C. Park, R.J. Marks, M. El-Sharkawi, Recovery of image blocks using the method
of alternating projections. IEEE Trans. Image Process. 14, 461–474 (2005)
282. S.N. Penfold, R.W. Schulte, Y. Censor, A.B. Rosenfeld, Total variation superiorization
schemes in proton computed tomography image reconstruction. Med. Phys. 37, 5887–5895
(2010)
283. W.V. Petryshyn, T.E. Williamson Jr., Strong and weak convergence of the sequence of
successive approximations for quasi-nonexpansive mappings. J. Math. Anal. Appl. 43, 459–
497 (1973)
284. G. Pierra, Decomposition through formalization in a product space. Math. Program. 28, 96–
115 (1984)
285. C. Popa, Least-squared solution of overdetermined inconsistent linear systems using
Kaczmarz’s relaxation. Int. J. Comp. Math. 55, 79–89 (1995)
286. C. Popa, Extensions of block-projections methods with relaxation parameters to inconsistent
and rank-deficient least-squares problems. BIT 38, 151–176 (1998)
287. C. Popa, R. Zdunek, Kaczmarz extended algorithm for tomographic image reconstruction
from limited data. Math. Comput. Simulat. 65, 579–598 (2004)
288. S. Prasad, Generalized array pattern synthesis by the method of alternating orthogonal
projections. IEEE Trans. Antenn. Propag. 28, 328–332 (1980)
289. J.L. Prince, A.S. Willsky, A geometric projection-space reconstruction algorithm. Lin.
Algebra Appl. 130, 151–191 (1990)
290. E. Pustylnik, S. Reich, A.J. Zaslavski, Convergence of infinite products of nonexpansive
operators in Hilbert space. J. Nonlinear Convex Anal. 11, 461–474 (2010)
291. E. Pustylnik, S. Reich, A.J. Zaslavski, Convergence of non-cyclic infinite products of
operators. J. Math. Anal. Appl. 380, 759–767 (2011)
292. B. Qu, N. Xiu, A note on the CQ algorithm for the split feasibility problem. Inverse Probl. 21,
1655–1665 (2005)
293. B. Qu, N. Xiu, A new halfspace-relaxation projection method for the split feasibility problem.
Lin. Algebra Appl. 428, 1218–1229 (2008)
294. S. Reich, Weak convergence theorems for nonexpansive mappings in Banach spaces. J. Math.
Anal. Appl. 67, 274–276 (1979)
295. S. Reich, A limit theorem for projections. Lin. Multilinear Algebra 13, 281–290 (1983)
296. S. Reich and I. Shafrir, The asymptotic behavior of firmly nonexpansive mappings. Proc. Am.
Math. Soc. 101, 246–250 (1987)
297. S. Reich, A.J. Zaslavski, Attracting mappings in Banach and hyperbolic spaces. J. Math. Anal.
Appl. 253, 250–268 (2001)
298. R.T. Rockafellar, Convex Analysis (Princeton University Press, Princeton, 1970)
299. R.T. Rockafellar, Monotone operators and the proximal point algorithm. SIAM J. Contr.
Optim. 14, 877–898 (1976)
300. W. Rudin, Functional Analysis, 2nd edn. (McGraw-Hill, New York, 1991); Polish translation:
Analiza funkcjonalna (PWN, Warszawa, 2002)
301. A.A. Samsonov, E.G. Kholmovski, D.L. Parker, C.R. Johnson, POCSENSE: POCS-based
reconstruction for sensitivity encoded magnetic resonance imaging. Magn. Reson. Med. 52,
1397–1406 (2004)
302. J. Schauder, Der Fixpunktsatz in Funktionalräumen. Studia Math. 2, 171–180 (1930)
303. D. Schott, A general iterative scheme with applications to convex optimization and related
fields. Optimization 22, 885–902 (1991)
304. H.D. Scolnik, N. Echebest, M.T. Guardarucci, M.C. Vacchino, A class of optimized row
projection methods for solving large nonsymmetric linear systems. Appl. Numer. Math. 41,
499–513 (2002)
References 287
305. H.D. Scolnik, N. Echebest, M.T. Guardarucci, M.C. Vacchino, Acceleration scheme for
parallel projected aggregation methods for solving large linear systems. Ann. Oper. Res. 117,
95–115 (2002)
306. H.D. Scolnik, N. Echebest, M.T. Guardarucci, M.C. Vacchino, Incomplete oblique projections
for solving large inconsistent linear systems. Math. Program. 111, 273–300 (2008)
307. A. Segal, Directed Operators for Common Fixed Point Problems and Convex Programming
Problems, Ph.D. Thesis, University of Haifa, Haifa, Israel, 2008
308. H.F. Senter, W.G. Dotson Jr., Approximating fixed points of nonexpansive mappings. Proc.
Am. Math. Soc. 44, 375–380 (1974)
309. A. Serbes, L. Durak, Optimum signal and image recovery by the method of alternating
projections in fractional Fourier domains. Comm. Nonlinear Sci. Numer. Simulat. 15, 675–
689 (2010)
310. N.T. Shaked, J. Rosen, Multiple-viewpoint projection holograms synthesized by spatially
incoherent correlation with broadband functions. J. Opt. Soc. Am. A 25, 2129–2138 (2008)
311. G. Sharma, Set theoretic estimation for problems in subtractive color. Color Res. Appl. 25,
333–348 (2000)
312. K.K. Sharma, S.D. Joshi, Extrapolation of signals using the method of alternating projections
in fractional Fourier domains. Signal Image Video Process. 2, 177–182 (2008)
313. K.T. Smith, D.C. Solman, S.L. Wagner, Practical and mathematical aspects of the problem of
reconstructing objects from radiographs. Bull. Am. Math. Soc. 83, 1227–1270 (1977)
314. R.A. Soni, K.A. Gallivan, W.K. Jenkins, Low-complexity data reusing methods in adaptive
filtering. IEEE Trans. Signal Process. 52, 394–405 (2004)
315. H. Stark, P. Oskoui, High resolution image recovery from image-plane arrays, using convex
projections. J. Opt. Soc. Am. A 6, 1715–1726 (1989)
316. H. Stark, Y. Yang, Vector Space Projections. A Numerical Approach to Signal and Image
Processing, Neural Nets and Optics (Wiley, New York, 1998)
317. J. Stoer, R. Bulirsch, Introduction to Numerical Analysis, 3rd edn. (Springer, New York, 2002)
318. C. Sudsukh, Strong convergence theorems for fixed point problems, equilibrium problems
and applications. Int. J. Math. Anal. (Ruse) 3, 1867–1880 (2009)
319. S. Świerczkowski, A model of following. J. Math. Anal. Appl. 222, 547–561 (1998)
320. W. Takahashi, Y. Takeuchi, R. Kubota, Strong convergence theorems by hybrid methods for
families of nonexpansive mappings in Hilbert spaces. J. Math. Anal. Appl. 341, 276–286
(2008)
321. K. Tanabe, Projection method for solving a singular system of linear equations and its
applications. Numer. Math. 17, 203–214 (1971)
322. K. Tanabe, Characterization of linear stationary iterative processes for solving a singular
system of linear equations. Numer. Math. 22, 349–359 (1974)
323. G. Tetzlaff, K. Arnold, A. Raabe, A. Ziemann, Observations of area averaged near-surface
wind- and temperature-fields in real terrain using acoustic travel time tomography. Meteorol-
ogische Zeitschrift 11, 273–283 (2002)
324. Ch. Thieke, T. Bortfeld, A. Niemierko, S. Nill, From physical dose constraints to equivalent
uniform dose constraints in inverse radiotherapy planning. Med. Phys. 30, 2332–2339 (2003)
325. J. van Tiel, Convex Analysis, An Introductory Text (Wiley, Chichester, 1984)
326. M.J. Todd, Some Remarks on the Relaxation Method for Linear Inequalities, Technical
Report, vol. 419, Cornell University, Cornell, Ithaca, 1979
327. Ph.L. Toint, Global convergence of a class of trust region methods for nonconvex minimiza-
tion in Hilbert space. IMA J. Numer. Anal. 8, 231–252 (1988)
328. W. Treimer, U. Feye-Treimer, Two dimensional reconstruction of small angle scattering
patterns from rocking curves. Physica B 241–243, 1228–1230 (1998)
329. J.A. Tropp, I.S. Dhillon, R.W. Heath, T. Strohmer, Designing structured tight frames via an
alternating projection method. IEEE Trans. Inform. Theor. 51, 188–209 (2005)
330. M.R. Trummer, SMART – an algorithm for reconstructing pictures from projections. J. Appl.
Math. Phys. 34, 746–753 (1983)
288 References
331. P. Tseng, On the convergence of the products of firmly nonexpansive mappings. SIAM
J. Optim. 2, 425–434 (1992)
332. A. Van der Sluis, H.A. Van der Vorst, in Numerical Solution of Large Sparse Linear Algebraic
Systems Arising from Tomographic Problems, ed. by G. Nolet. Seismic Tomography (Reidel,
Dordrecht, 1987)
333. V.V. Vasin, A.L. Ageev, Ill-Posed Problems with A Priori Information (VSP, Utrecht, 1995)
334. S. Webb, Intensity Modulated Radiation Therapy (Institute of Physics Publishing, Bristol,
2001)
335. S. Webb, The Physics of Conformal Radiotherapy (Institute of Physics Publishing, Bristol,
2001)
336. R. Webster, Convexity (Oxford University Press, Oxford, 1994)
337. R. Wegmann, Conformal mapping by the method of alternating projections. Numer. Math.
56, 291–307 (1989)
338. R. Wittmann, Approximation of fixed points of nonexpansive mappings. Arch. Math. 58,
486–491 (1992)
339. P. Wolfe, Finding the nearest point in a polytope. Math. Program. 11, 128–149 (1976)
340. B.J. van Wyk, M.A. van Wyk, A POCS-based graph matching algorithm. IEEE Trans. Pattern
Anal. Mach. Intell. 26, 1526–1530 (2004)
341. Y. Xiao, Y. Censor, D. Michalski, J.M. Galvin, The least-intensity feasible solution for
aperture-based inverse planning in radiation therapy. Ann. Oper. Res. 119, 183–203 (2003)
342. H.-K. Xu, Iterative algorithms for nonlinear operators. J. Lond. Math. Soc. 66, 240–256
(2002)
343. H.-K. Xu, An iterative approach to quadratic optimization. J. Optim. Theor. Appl. 116, 659–
678 (2003)
344. H.-K. Xu, A variable Krasnosel’skiı̆-Mann algorithm and the multiple-set split feasibility
problem. Inverse Probl. 22, 2021–2034 (2006)
345. I. Yamada, in The Hybrid Steepest Descent Method for the Variational Inequality Problem
Over the Intersection of Fixed Point Sets of Nonexpansive Mappings, ed. by D. Butnariu,
Y. Censor, S. Reich. Inherently Parallel Algorithms in Feasibility and Optimization and their
Applications, Studies in Computational Mathematics, vol. 8 (Elsevier Science, Amsterdam,
2001), pp. 473–504
346. I. Yamada, N. Ogura, Hybrid steepest descent method for variational inequality problem over
the fixed point set of certain quasi-nonexpansive mappings. Numer. Funct. Anal. Optim. 25,
619–655 (2004)
347. I. Yamada, N. Ogura, Adaptive projected subgradient method for asymptotic minimization of
sequence of nonnegative convex functions. Numer. Funct. Anal. Optim. 25, 593–617 (2004)
348. I. Yamada, N. Ogura, N. Shirakawa, in A Numerically Robust Hybrid Steepest Descent
Method for the Convexly Constrained Generalized Inverse Problems, ed. by Z. Nashed,
O. Scherzer. Inverse Problems, Image Analysis and Medical Imaging, American Mathemati-
cal Society, Contemp. Math., vol. 313 (2002), pp. 269–305
349. I. Yamada, N. Ogura, Y. Yamashita, K. Sakaniwa, Quadratic optimization of fixed points of
nonexpansive mappings in Hilbert space. Numer. Funct. Anal. Optim. 19, 165–190 (1998)
350. Q.Z. Yang, The relaxed CQ algorithm solving the split feasibility problem. Inverse Probl. 20,
1261–1266 (2004)
351. K. Yang, K.G. Murty, New iterative methods for linear inequalities. JOTA 72, 163–185 (1992)
352. Q. Yang, J. Zhao, Generalized KM theorems and their applications. Inverse Probl. 22, 833–
844 (2006)
353. Y. Yao, Y.-C. Liou, Weak and strong convergence of Krasnoselski–Mann iteration for
hierarchical fixed point problems. Inverse Probl. 24, 015015, 8 (2008)
354. D. Youla, Generalized image restoration by the method of alternating orthogonal projections.
IEEE Trans. Circ. Syst. 25, 694–702 (1978)
355. M. Yukawa, I. Yamada, Pairwise optimal weight realization – Acceleration technique for set-
theoretic adaptive parallel subgradient projection algorithm. IEEE Trans. Signal Process. 54,
4557–4571 (2006)
References 289
356. M. Zaknoon, Algorithmic Developments for the Convex Feasibility Problem, Ph.D. Thesis,
University of Haifa, Haifa, Israel, 2003
357. E.H. Zarantonello, in Projections on Convex Sets in Hilbert Space and Spectral Theory, ed.
by E.H. Zarantonello. Contributions to Nonlinear Functional Analysis (Academic, New York,
1971), pp. 237–424
358. E. Zeidler, Nonlinear Functional Analysis and Its Applications, III – Variational Methods and
Optimization (Springer, New York, 1985)
359. J. Zhang, A.K. Katsaggelos, in Image Recovery Using the EM Algorithm, ed. by V.K.
Madisetti, D.B. Williams. Digital Signal Processing Handbook (CRC Press LLC, Boca Raton,
1999)
360. D.F. Zhao, The principles and practice of iterative alternating projection algorithm: Solution
for non-LTE stellar atmospheric model with the method of linearized separation. Chin.
Astron. Astrophys. 25, 305–316 (2001)
361. J. Zhao, Q. Yang, Several solution methods for the split feasibility problem. Inverse Probl. 21,
1791–1799 (2005)
•
Glossary of Symbols
H Hilbert space
hx; yi Inner product of x; y 2 H
kxk Norm of x 2 H induced by h; i
hx; yiG Inner product of x; y 2 Rn induced by a positive definite
p matrix G
kxkG WD hx; xiG The norm of x 2 Rn induced by h; iG
^.x; y/ Angle between nonzero vectors x; y 2 H
X A nonempty closed convex subset of H
I WD f1; 2; :::; mg Finite subset of indices
xC ; x Positive and the negative part of x 2 Rn
RnC ; Rn Nonnegative and the nonpositive orthant
m Standard simplex in Rm
jJ j The number of elements of a finite subset J
V? Subspace orthogonal to a subspace V H
B.x; / Ball with a centre x and radius > 0
C0 Complement of a subset C H
bd C Boundary of a subset C H
int C Interior of a subset C H
cl C Closure of a subset C H
Lin S Linear subspace generated by S H
aff S Affine subspace generated by S H
H.a; ˇ/ Hyperplane fx 2 H W ha; xi D ˇg
H .a; ˇ/ Half-space fx 2 H W ha; xi ˇg
HC .a; ˇ/ Half-space fx 2 H W ha; xi ˇg
fC ; f Positive and the negative part of a function f
Argminx2X f .x/ Subset of minimizers of f W X ! R
argminx2X f .x/ Minimizer of f W X ! R
292 Glossary of Symbols
S.f; ˛/ Sublevel set of a function f at a level ˛ 2 R

epi f Epigraph of a function f
f 0 .x; s/ Directional derivative of a function f at x in a direction s
Df , f 0 , DT Derivative of a function f or of an operator T
rf .x/ Gradient of a function f at x
r 2 f .x/ Hessian of a function f at x
diag v Diagonal matrix with a vector v at the main diagonal
A> Matrix transposed to a matrix A
AC Moore–Penrose pseudoinverse of a matrix A
cone S Conical hull of a subset S H
conv S Convex hull of a subset S H
ri C Relative interior of a subset C H
C Polar cone to C H
NC .x/ Normal cone to a convex subset C H at x 2 H
TC .x/ Tangent cone to a convex subset C H at x 2 H
@f .x/ Subdifferential of a function f at x 2 H
gf .x/ Subgradient of a function f at x 2 H
PC Metric projection onto a subset C H
Pa Metric projection onto Linfag, where a 2 H
T Relaxation of an operator T
Fix T Subset of fixed points of an operator T
d.; C / Distance function to a subset C H
d.A; B/ Distance between subsets A; B H
L.H1 ; H2 / Space of all bounded linear operators A W H1 ! H2
max .A/ Largest eigenvalue of a nonnegative operator A W H ! H
Fej T See page 46
Sep T See page 55
Glossary of Acronyms
AAR Averaged alternating reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

APM Alternating projection method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
AR Asymptotically regular (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
ART Algebraic reconstruction technique . . . . . . . . . . . . . . . . . . . . . . . . . . 220
AV Averaged (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
CFPP Common fixed point problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
CFP Convex feasibility problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
CMP Convex minimization problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
DR Douglas–Rachford (algorithm) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
EMOPP Extrapolated method of parallel projections . . . . . . . . . . . . . . . . . . 257
EMOPSP Extrapolated method of parallel subgradient projections . . . . . . . . 257
EPPM Extrapolated parallel projection method . . . . . . . . . . . . . . . . . . . . . . 257
ESC Extrapolated simultaneous cutter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
ESCM Extrapolated simultaneous cutter method . . . . . . . . . . . . . . . . . . . . 254
ESSPM Extrapolated simultaneous subgradient projection method . . . . . . 258
FM Fejér monotone (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
FNE Firmly nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
-FNE -firmly nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
GCFP Generalized convex feasibility problem . . . . . . . . . . . . . . . . . . . . . . . 33
KKT-point Karush–Kuhn–Tucker point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
LFP Linear feasibility problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
LLSP Linear least square problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
LM Landweber method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
LSFP Linear split feasibility problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
NE Nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
OCS Obtuse cone selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
294 Glossary of Acronyms
PLM Projected Landweber method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

QNE Quasi-nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
RFNE Relaxed firmly nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . 65
-RFNE -relaxed firmly nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . 65
ROCS Regular obtuse cone selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
RS Residual selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
SCM Surrogate constraints method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
SiCM Simultaneous cutter method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
SFM Strongly Fejér monotone (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
SFP Split feasibility problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
SNE Strongly nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
SPM Simultaneous projection method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
sQNE Strictly quasi-nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . . . 47
C -sQNE C -strictly quasi-nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . 47
SQNE Strongly quasi-nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . . 56
˛-SQNE ˛-strongly quasi nonexpansive (operator) . . . . . . . . . . . . . . . . . . . . . . . 56
VIP Variational inequality problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Index
.ı; /-algorithm, 252 attracting operator, 47

autonomous method, 105
autonomous projection method, 203
actualization, 39 averaged alternating reflection, 161
adjoint operator, 9 averaged alternating reflection algorithm, 212
admissible control sequence, 225 averaged operator, 74
admissible step size function, 255 averaged quasi-nonexpansive mapping, 54
affine hull, 13
affine operator, 11
affine subspace, 6 ball, 5
algebraic reconstruction technique, 220 Banach fixed point theorem, 41
algorithm, 39 Banach theorem on contractions, 41
algorithmic mapping, 39 band, 133
algorithmic operator, 39 Browder–Göhde–Kirk theorem, 42
algorithmic projection operator, 129
almost cyclic control, 225
almost remotest set control, 224 closed operator, 107
alternating projection, 147 coercive function, 7
alternating projection method, 204 common fixed point problem, 27
angle, 6 compact operator, 9
angle between subspaces, 208 concave function, 14
angle between vectors, 6 constrained minimization problem, 24
appropriate weight function, 50 constraints functions, 28
approximately regular control, 234 constraints qualification, 26
approximately regular control sequence, 251 constraints sets, 28
approximately regular weight function, 234 contraction, 41
approximately remotest set control, 223 control function, 222
approximately semi-regular control, 234 control sequence, 222, 234
approximately semi-regular control sequence, convex cone, 13
251 convex feasibility problem, 27
approximately semi-regular weight function, convex function, 14
234 convex hull, 13
approximately semi-remotest set control, 223 convex minimization problem, 25
approximating sequence, 39 convex subset, 12
asymptotically regular convexity of the norm, 2
sequence of operators, 116 CQ algorithm, 229
operator, 111 cutter, 53
296 Index
cyclic control, 224 firmly contractive operator, 65

cyclic cutter, 176 firmly nonexpansive operator, 65
cyclic projection, 171 firmly quasi-nonexpansive operator, 54
cyclic projection method, 218 fixed point, 11
cyclic relaxed projection, 172 focusing algorithm, 240
cyclic relaxed projection method, 219 Fréchet-differentiable
cyclic-simultaneous projection operator, 173 function, 7
demi-closed operator, 107 Gâteaux-derivative, 8

demi-closedness principle, 107 Gâteaux-differentiable
derivative function, 8
of a function, 7 Gâteaux-differential, 8
of an operator, 10 general convex feasibility problem, 33
diagonal matrix, 11 generalized relaxation, 97
differentiable global minimizer, 24
function, 7 global minimum, 7
operator, 10 gradient, 8
differential, 7 Gram matrix, 12
of an operator, 10
directional derivative, 7
distance function, 15 half-space, 6
distance to a subset, 15 hard constraints, 33
Douglas–Rachford algorithm, 212 Hessian, 8
Hilbert space, 1
hyperplane, 6
eigenvalue, 9
ellipsoid, 137
elliptic operator, 9 ice-cream cone, 140
epigraph, 7 idempotent operator, 11
"-optimal solution, 25 identity operator, 8
essential weight function, 192 image, 11
Euclidean space, 3 intermittent control, 238
extrapolated alternating projection method, intermittent sequence of subsets, 237
209 inverse image, 11
extrapolated Landweber operator, 182 inverse strongly monotone operator, 74
extrapolated Landweber step size, 182 inverse-positive matrices, 197
extrapolated methods of parallel projections, iteration of an operator, 11
254 iterative method, 39
extrapolated simultaneous cutter, 188 iterative procedure, 39
extrapolated simultaneous cutter method, 254
extrapolated simultaneous projection method,
254 Kaczmarz method, 220
extrapolated simultaneous projection operator, Karush–Kuhn–Tucker point, 25
190 kernel, 11
extrapolated simultaneous subgradient Krasnosel’skiı̆–Mann theorem, 115
projection method, 258
extrapolation, 97
Landweber extrapolation parameter, 180
Landweber method, 228
F -attracting mapping, 48 Landweber operator, 176
Fejér monotone Landweber operator related to a matrix, 179
operator, 45 Landweber step size, 180
sequence, 108 linear feasibility problem, 30
Index 297
linear least squares problem, 32 paracontraction, 48

linear split feasibility problem, 35 parallelogram law, 2
linearization, 16 partial derivative, 8
linearly dependent vectors polar cone, 13
negative, 2 polytope, 12
positive, 2 positive definite matrix, 11
linearly focusing algorithm, 235 positive operator, 9
Lipschitz constant, 10 positive semi-definite matrix, 11
Lipschitz continuous operator, 10 positively homogeneous function, 15
local minimizer, 24 product Hilbert space, 4
local minimum, 7 projected gradient method, 214
lower semi-continuous function, 7 projected Landweber method, 228
projected Landweber operator, 184
projected relaxation, 89
maximal residual control, 224 projected simultaneous projection method, 217
metric projection, 17 projection, 11
minimizer, 6 projection method, 203
monotone operator, 10 projection vector, 17
Moore-Penrose pseudoinverse, 12 projection-reflection
Moreau decomposition, 13 method, 221
multiple-sets split feasibility problem, 35 operator, 174
proper control, 264
proper sequence of weight functions, 264
proximity function, 28
ni-firmly nonexpansive operator, 73 pseudo-contractive operator, 73
nonautonomous method, 105
nonautonomous projection method, 203
nonexpansive operator, 41 quasi-nonexpansive operator, 47
nonnegative operator, 9 quasi-periodic control, 225
norm of an operator, 9
normal cone, 13
normal equations, 32 range, 11
normal solution, 32 reflection, 40, 77
null space, 11 reflection-projection
method, 221
perator, 174
oblique projection, 130, 184 reflection-relaxed projection
obtuse cone, 13 operator, 174
obtuse cone selection, 197, 272 regular obtuse cone selection, 197, 273
Opial space, 107 regular weight function, 232
Opial’s property, 107 relative interior, 13
Opial’s theorem, 114 relaxation, 40
optimal solution, 24 relaxation parameter, 40
orbit, 11 relaxed alternating projection method, 204
orbit of an operator, 39 relaxed cutter, 53
oriented operator, 98 relaxed firmly nonexpansive operator, 65
orthant relaxed metric projection, 77
nonnegative , 5 relaxed projection-reflection
nonpositive, 5 operator, 174
positive, 5 relaxed separator, 53
orthogonal projection, 11 repetitive control sequence, 225, 240
orthogonal subspace, 5 repetitive sequence of subsets, 240
over-projection, 77 residual selection, 197
over-relaxation, 40 residual vector, 31
298 Index
residuum of a constraint, 31 strongly monotone operator, 10

retract, 11 strongly nonexpansive operator, 91
retraction, 11 strongly oriented operator, 98
strongly quasi-nonexpansive operator, 56
subdifferentiable function, 16
Schwarz inequality, 1 subdifferential, 16
Schwarz theorem, 8 subgradient, 16
segment, 6 subgradient projection, 144
self-adjoint operator, 9 sublevel set, 7
separating operators, 54 subset of minimizers, 6
separator, 53 successive projection method, 222
sequential projection method, 222 surrogate constraint, 192
simultaneous cutter, 186 surrogate constraints method, 263
simultaneous cutter method, 231 surrogate constraints method
simultaneous projection, 162 with residual selection, 271
simultaneous projection method, 215 surrogate projection, 193
simultaneous relaxed projection, 162 with residual selection, 198
simultaneous subgradient projection, 199
Slater constraints qualification, 26
soft constraints, 33 tangent cone, 14
sphere, 6 triangle inequality, 2
split feasibility problem, 34
standard scalar product, 3
standard simpleks, 5 under-projection, 77
step size function, 97 under-relaxation, 40
strict convexity of the norm, 2 uniformly F -attracting mapping, 56
strict relaxation, 40 unit vector, 5
strictly attracting operator, 47 unitary operator, 9
strictly convex function, 14 update, 39
strictly Fejér monotone
operator, 46
sequence, 108 variational inequality problem, 27
strictly nonexpansive operator, 41 violated constraints, 50
strictly quasi-nonexpansive operator, 47 von Neumann method, 204
strictly relaxed firmly nonexpansive operator,
65
strongly attracting operator, 56 weak cluster point, 4
strongly convex function, 14 weak convergence, 4
strongly Fejér monotone weak limit, 4
sequence, 108 weakly lower semi-continuous function, 7
strongly focusing algorithm, 240 weight function, 50

Iterative Methods For Fixed Point Problems in Hilbert Spaces (PDFDrive)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Iterative Methods For Fixed Point Problems in Hilbert Spaces (PDFDrive)

Uploaded by

Copyright:

Available Formats

Lecture Notes in Mathematics 2057

For further volumes:

ISBN 978-3-642-30900-7 ISBN 978-3-642-30901-4 (eBook)

Lecture Notes in Mathematics ISSN print edition: 0075-8434

Library of Congress Control Number: 2012945521

c Springer-Verlag Berlin Heidelberg 2012

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

[John Paul II]

In this monograph we work with operators defined on a real Hilbert space,

Zielona Góra, Poland Andrzej Cegielski

2.2.4 Metric Projection onto a Closed Subspace.. . . . . . . . . . . . . . . . . 80

5.5 Cyclic Projection Methods . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 218

1.1.1 Hilbert Space

1.1.1.1 Properties of the Inner Product

The following equalities hold for all x; y 2 H:

kx C yk2 D kxk2 C kyk2 C 2hx; yi, (1.1)

kx yk2 D kxk2 C kyk2 2hx; yi (1.2)

kxk kyk  hx; yi  kxk kyk . (1.4)

A. Cegielski, Iterative Methods for Fixed Point Problems in Hilbert Spaces, 1

j kxk kyk j  kx C yk  kxk C kyk . (1.5)

kxk D kyk D k.1 /x C yk H) x D y. (1.6)

Convexity of the norm. Let w D .!1 ; : : : ; !m / 2 Rm C (all coordinates of w are

Parallelogram law. The following equality holds for all x; y 2 H:

kx C yk2 C kx yk2 D 2.kxk2 C kyk2 /. (1.8)

1.1.1.2 Examples of Hilbert Spaces

Example 1.1.2. The Euclidean space Rn WD fx D .1 ; 2 ; : : : ; n / W j 2 R; j D

with the inner product

where x D .x1 ; : : : ; xm /, y D .y1 ; : : : ; ym / 2 H, w D .!1 ; : : : ; !m / 2 Rm

1.1.1.3 Weak Convergence in Hilbert Spaces

We say that a sequence fx k g1

Note that a bounded sequence fx k g1

1.1.2 Notations and Basic Facts

1.1.2.1 Euclidean Space

Elements of the Euclidean space Rn can be represented as vectors with real

R W x  0g denotes the nonpositive orthant; RCC WD fx 2 R W x > 0g

denotes the positive orthant; ej WD .0; : : : ; 0; 1; 0; : : : ; 0/ 2 Rm denotes the j th

1.1.2.2 Subsets of a Hilbert Space

Let V H. The subset

V ? WD fy 2 H W hy; xi D 0 for all x 2 V g

denotes the subspace orthogonal to V . Let V H be a closed subspace. Then

denotes the Minkowski sum of C; D H. In particular,

Œa; b WD fz 2 H W z D .1 /a C b; 2 Œ0; 1g

denotes a segment with endpoints a; b 2 H. The subset

Argmin f .x/ WD fz 2 C W f .z/  f .x/ for all x 2 C g,

where C H and f W C ! R is called a subset of minimizers of a function f .

H.a; ˇ/ WD fx 2 H W ha; xi D ˇg,

where a 2 H, a ¤ 0 and ˇ 2 R is called a hyperplane in a Hilbert space H. The

denotes an angle between nonzero vectors a; b 2 H.

Let X H, fi W X ! R, i 2 I . The function f WD maxi 2I fi is defined by

f .x/ WD maxffi .x/ W i 2 I g

In particular, fC WD maxff; 0g, f WD maxff; 0g.

is called a sublevel set of f at a level ˛ 2 R. The subset

epi f WD f.x; / 2 X  R W  f .x/g X  R

is called an epigraph of f . We say that f is lower semi-continuous at x 2 X if

f .x C h/ D f .x/ C hy; hi C o.khk/,

The derivative of a differentiable function is uniquely determined.

holds for some g 2 H. The element g is called a Gâteaux-derivative or Gâteaux-

is called the Hessian of f at .

Let H; H1 ; H2 ; H3 be Hilbert spaces. The operator Id W H ! H denotes the identity,

hAx; yiH2 D hx; A yiH1

kAk2 D kA k D kA Ak D kAA k .

Theorem 1.1.22. If a bounded linear operator A W H ! H is self-adjoint, then

kxk kyk hx; yi kxk kyk . (1.4)

j kxk kyk j kx C yk kxk C kyk . (1.5)

R W x 0g denotes the nonpositive orthant; RCC WD fx 2 R W x > 0g

Argmin f .x/ WD fz 2 C W f .z/ f .x/ for all x 2 C g,

epi f WD f.x; / 2 X R W f .x/g X R

Definition 1.1.33. We say that a symmetric matrix A of type n n is positive

B.x; / is convex for any x 2 H and for any 0. Let A W H1 ! H2 be an affine

C WD fx 2 H W hy; xi 0 for all y 2 C g

f ..1 /x C y/ .1 /f .x/ C f .y/

@f .x/ WD fg 2 H W hg; y xi f .y/ f .x/ for all y 2 Hg

2.d 2 C "=4/ C 2.d 2 C "=4/ 4d 2 D ",

kx yk2 kx z k2 D kx y .z y/k2

f .x / f .x / .1 /f .x / C f .x 0 / < f .x /