Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
5Activity
0 of .
Results for:
No results containing your search query
P. 1
Gateaux Differentials and Frechet Derivatives

Gateaux Differentials and Frechet Derivatives

Ratings: (0)|Views: 956|Likes:
Published by Niflheim
Quite helpful
Quite helpful

More info:

Published by: Niflheim on Oct 18, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

12/02/2012

pdf

text

original

 
Math 5311 – Gateaux differentials and Frechet derivatives
Kevin Long January 26, 2009
1 Differentiation in vector spaces
Thusfar,we’vedevelopedthetheoryofminimizationwithoutreferencetoderivatives. We’veevenbeenabletocompute minimizers of quadratic forms without using derivatives, by proving that the minimizer of a positivedefinite quadratic form must be the solution to the algebraic equation
Kx
=
. However, in computations it’sconvenient and efficient to use derivatives.We’re developing the theory of minimization of functions set in arbitrary vector spaces, so we need to developdifferential calculus in that setting. In multivariable calculus, you learned three related concepts: directionalderivatives, partial derivatives, and gradients. In arbitrary vector spaces, we will be able to develop a gener-alization of the directional derivative (called the Gateaux differential) and of the gradient (called the Frechetderivative). We won’t go deeply into the theory of these derivatives in this course, but we’ll establish the basicdifferentiation rules.A reference for differentiation in infinite-dimensional vector spaces, done at the level of this course, is D. R.Smith,
Variational Methods in Optimization
, Dover, 1998. A careful development of the finite-dimensional casecan be found in M. Spivak,
Calculus on Manifolds
, J. Munkres,
Analysis on Manifolds
, or Rudin, P
rinciples of  Mathematical Analysis
. If you’re interested, E. W. Cheney’s
Analysis for Applied Mathematics
gives a treatment of Gateaux and Frechet derivatives at a level one notch above the level of this course.
1.1 The Gateaux differential
The Gateaux differential generalizes the idea of a directional derivative.
Definition 1.
Let
:
U
be a function and let
h
=
0 and
x
be vectors in
. The Gateaux differential
d
h
isdefined
d
h
=
lim
0
 f 
(
x
+
h
)
(
x
)
.Some things to notice about the Gateaux differential:
There is not a single Gateaux differential at each point. Rather, at each point
x
there is a Gateaux differ-ential for each direction
h
. In one dimension, there are two Gateaux differentials for every
x
: one directed“forward,” one “backward.” In two of more dimensions, there are infinitely many Gateaux differentialsat each point!
The Gateaux differential is a one-dimensional calculation along a specified direction
h
. Because it’s one-dimensional, you can use ordinary one-dimensional calculus to compute it. Your old friends such as thechain rule work for Gateaux differentials. Thus, it’s usually easy to compute a Gateaux differential evenwhen the space
is infinite dimensional.1
 
1.2 Examples of Gateaux differentials
1.2.1 Linear and quadratic functions
Let
and
x
be vectors in an inner product space, and define
p
(
x
) =
x
 f 
. Then
d
h
 p
=
lim
0
x
 f 
+
h
 f 
x
 f 
d
h
(
x
 f 
) =
h
 f 
.Next, let
K
be a symmetric matrix, and define
p
(
x
) =
2
x
 f 
+
x
Kx
. Compute the Gateaux differential
d
h
 p
=
lim
0
2
x
 f 
+
h
 f 
+
x
Kx
+
2
h
Kx
+
2
h
Kh
2
x
 f 
x
Kx
=
h
 f 
+
2
h
Kx
.
1.2.2 The exponential function
There are a number of ways to define the exponential function; most convenient for present purposes is todefine it as the infinite series
e
x
=
1
+
x
+
x
2
2
+
···
+
x
k
k
!
+
···
.This makes sense even for vector-valued arguments provided multiplication is understood as “elementwise,”is in the Matlab “dot-star” operation. In the case of an infinite-dimensional vector space
(for example
1
)whose elements are real-valued functions, the exponential function
e
u
:
simply maps
u
(
x
)
pointwiseto its exponential
e
u
(
x
)
.With
e
x
defined as above, it can be shown (provided multiplication of members of 
is commutative!) bystraightforward calculation and rearrangement that
e
a
+
b
=
e
a
e
b
(see, e.g., blue or green Rudin. If you’reinterested, see a reference on Lie algebras for information on the exponentiation of non-commutative objects;that turns out to be useful in fields such as aircraft control and quantum physics.)With the identity
e
a
+
b
=
e
a
e
b
and the series defining
e
x
, we can compute the Gateaux derivative
d
h
(
e
u
) =
lim
0
e
u
e
h
e
u
=
e
u
lim
0
e
h
1
=
he
u
.
1.2.3 The absolute value function in
R
Let
(
x
) =
|
x
|
. Calculation of the limit gives
d
h
=
h
x
|
x
|
x
=
0
|
h
|
x
=
0.Notice that the Gateaux differentials of 
|
x
|
do exist at zero; however, at zero, the Gateaux differentials dependon
h
in a nonlinear way.2
 
1.2.4 A spatial derivative
d
h
dudx
=
lim
0
u
x
+
h
x
u
x
=
dhdx
1.2.5 A functional
Let
:
1
(
)
R
 be
 J 
[
u
] =
¢  
12
u
2
x
+
12
u
2
dx
.Then
d
h
 J 
=
lim
0
¡  
12
u
2
x
+
12
u
2
+
uh
+
u
x
h
x
+
12
2
h
2
x
+
12
h
2
12
u
2
x
12
u
2
dx
d
h
 J 
=
¢  
[
uh
+
u
x
h
x
]
dx
Note: it’s routine in infinite-dimensional optimization problems to exchange integration and Gateaux differ-entiation without comment. The integration variable
x
and the differentiation variable
u
are different. If thelimit
0 can be taken by means of a sequence
{
n
}
for which the difference quotient of the integrand isdominated by a Lebesgue-integrable function, the conditions of the Lebesgue dominated convergence theoremhold and the limit and integral can be interchanged safely. For the section on optimization, we’ll assume this to be true throughout. However, later in the semester we’ll see some examples with Fourier series and transformswhere differentiation of a convergent series or integral results in divergence at one or more points.There is, however, an important real-world case where this process requires a little more care: the case wherethe
limits
ofintegrationdependonthevariable
u
.Thisarisesinthefieldof 
shapeoptimization
, wherethe“designvariable”
u
determines the size and shape of a domain
.
1.3 Rules for Gateaux differentials
Computing from the definition quickly gets dull, so as with ordinary calculus, let’s work out rules for Gateauxdifferentials.
1.3.1 Differential of a constant
The Gateaux differential of a constant is zero:
d
h
c
=
0. The proof follows immediately from the definition.
1.3.2 Sum rule
Gateaux differentiation distributes over sums:
d
h
(
 f 
±
g
)=
d
h
±
d
h
 g
. The proof follows immediately fromthe definition.
1.3.3 Product rule and quotient rule
The Gateaux differential of an elementwise product
fg
is
d
h
(
 fg
) = (
d
h
)
 g
+
(
d
h
 g
)
.The Gateaux differential of an inner product
 f 
,
 g
(or
 g
) is
d
h
 f 
,
 g
=
 f 
,
d
h
 g
+
d
h
,
 g
. With transposenotation, this is
d
h
(
 f 
 g
) =
d
h
 g
+ (
d
h
)
 g
.The proofs are similar to what you would do in
R
, and are left as exercises; the little-
o
notation below will beuseful.3

Activity (5)

You've already reviewed this. Edit your review.
1 hundred reads
1 thousand reads
a_705074461 liked this
hadikami liked this
Varov liked this

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->