Professional Documents
Culture Documents
CHRISTOPHER HEIL
Definition 1.2. Let X be a vector space over the field F. Then a semi-norm on X is a
function k · k : X → R such that
(a) kxk ≥ 0 for all x ∈ X,
(b) kαxk = |α| kxk for all x ∈ X and α ∈ F,
(c) Triangle Inequality: kx + yk ≤ kxk + kyk for all x, y ∈ X.
A norm on X is a semi-norm which also satisfies:
(d) kxk = 0 =⇒ x = 0.
A vector space X together with a norm k · k is called a normed linear space, a normed vector
space, or simply a normed space.
Definition 1.3. Let I be a finite or countable index set (for example, I = {1, . . . , N } if
finite, or I = N or Z if infinite). Let w : I → [0, ∞). Given a sequence of scalars x = (xi )i∈I ,
set 1/p
X
p p
|xi | w(i) , 0 < p < ∞,
kxkp,w = i∈I
sup |xi | w(i), p = ∞,
i∈I
where these quantities could be infinite. Then we set
n o
`pw (I) = x = (xi )i∈I : kxkp < ∞ .
We call `pw (I) a weighted `p space, and often denote it just by `pw (especially if I = N). If
w(i) = 1 for all i, then we simply call this space `p (I) or `p and write k · kp instead of k · kp,w .
Exercise 1.4. Show that if 1 ≤ p ≤ ∞ then k · kp,w defines a semi-norm on `pw , and it is a
norm if w(i) > 0 for all i.
In particular, if I = {1, . . . , n} then `pw = Fn , and each choice of p and w gives a semi-norm
or norm on Fn .
Hints: The Triangle Inequality on `p is often called Minkowski’s Inequality. It is easy to
prove if p = 1 or p = ∞. There are several ways to prove it for other p. One ways is to begin
with
X X
kx + ykpp = |xi + yi |p = |xi + yi |p−1 |xi + yi |
i∈I i∈I
X X
≤ |xi + yi |p−1 |xi | + |xi + yi |p−1 |yi |.
i∈I i∈I
Then apply Hölder’s Inequality to each sum using the exponent p0 on the first factor and p
for the second (recall that p0 = p/(p − 1)). Then divide both sides by kx + ykp−1
p .
Exercise 1.7. Show that every subspace of a normed space is itself a normed space (using
the same norm).
Exercise 1.9. Show that d(·, ·) defines a metric on X (see Appendix A).
Since X has a metric and hence has an associated topology, all the standard topological
notions (open/closed sets, convergence, etc.) apply to X. For convenience, we give some
explicit definitions and facts relating to these topics for the setting of normed spaces.
Definition 1.10 (Convergence). Let X be a normed linear space (such as an inner product
space), and let {fn }n∈N be a sequence of elements of X.
(a) We say that {fn }n∈N converges to f ∈ X, and write fn → f , if
lim kf − fn k = 0,
n→∞
i.e.,
∀ ε > 0, ∃ N > 0 such that n > N =⇒ kf − fn k < ε.
Exercise 1.12. Let {fn }n∈N be a sequence of vectors in a normed space X. Show that if
kfn − fn+1 k < 2−n for every n, then {fn }n∈N is Cauchy.
Exercise 1.13. Let `pw (I) be the weighted `p space defined in Exercise 1.3, where we assume
w(i) > 0 for all i ∈ I. Let {xn }n∈N be a sequence of vectors in `pw (I), and let x be a vector in
`pw (I). Write the components of xn and x as xn = (xn (1), xn (2), . . . ) and x = (x(1), x(2), . . . ).
(a) Prove that if xn → x (i.e., kx − xn kp,w → 0), then xn converges componentwise to x,
i.e., for each fixed k we have limn→∞ xn (k) = x(k).
(b) Prove that if I is finite then the converse is also true, i.e., componentwise convergence
implies convergence with respect to the norm k · kp,w .
(c) Prove that if I is infinite then componentwise convergence does not imply convergence
in the norm of `wp (I).
4 CHRISTOPHER HEIL
It is not true in an arbitrary normed space that every Cauchy sequence must converge.
Normed spaces which do have the property that all Cauchy sequences converge are given a
special name.
Exercise 1.15. Show that the weighted `p space `pw (I) defined in Exercise 1.3 is a Banach
space if w(i) > 0 for all i ∈ I.
Hints: Consider the case I = N. Suppose that {xn }n∈N is a Cauchy sequence in `pw . Each
xn is a sequence of scalars. Write out the components of xn as
xn = (xn (1), xn (2), . . . ).
Prove that for a fixed component k we have
|xm (k) − xn (k)| ≤ C kxm − xn kp,w ,
where C is a fixed constant (determined by the weight and by k but independent of m and
n). Conclude that with k fixed, {xn (k)}n∈N is a Cauchy sequence of scalars, and hence con-
verges. Define x(k) = limn→∞ xn (k) and set x = (x(1), x(2), . . . ). Then we have constructed
a candidate limit for the sequence {xn }n∈N . However, so far we only have that each indi-
vidual component of xn converges to the corresponding component of x, i.e., xn converges
componentwise to x. This is not enough: to complete the proof you must show that xn → x
in the norm of `pw . Use the fact that {xn }n∈N is Cauchy together with the componentwise
convergence to show that kx − xn k`pw → 0 as n → ∞.
Compare this proof to the proof of Theorem 2.18 given below.
Exercise 1.16. Show that the space Lp (X) defined in Example 1.5 is a Banach space if
we identify functions that are equal almost everywhere. This is called the Riesz–Fisher
Theorem.
Hint: The argument is similar in spirit but more subtle than the one used to prove that
`pw (I) is a Banach space. First find a candidate limit and then show that the sequence
converges in norm to this limit.
Exercise 1.17. Let X be a normed linear space. If {fn }n∈N is a Cauchy sequence in X and
there exists a subsequence {fnk }k∈N that converges to f ∈ X, then fn → f .
Solution
Choose any ε > 0. Since {fn }n∈N is Cauchy, there is an N such that kfm − fn k < 2ε for m,
n > N . Also, there is a k such that nk > N and kf − fnk k < 2ε . Hence for n > N we have
ε ε
kf − fn k ≤ kf − fnk k + kfnk − fn k < + = ε.
2 2
CHAPTER 3. BANACH SPACES 5
Thus fn → f .
Exercise 1.18. Let {fn }n∈N be a Cauchy sequence in a normed space X. Show that there
exists a subsequence {fnk }k∈N such that
∀ k ∈ N, kfnk+1 − fnk k < 2−k .
Solution
Since {fn }n∈N is Cauchy, we can find an n1 such that
m, n ≥ n1 =⇒ kfm − fn k < 2−1 .
Then we can find an n2 > n1 such that
m, n ≥ n2 =⇒ kfm − fn k < 2−2 .
Continuing in this way, we inductively construct n1 < n2 < · · · such that for each k,
m, n ≥ nk =⇒ kfm − fn k < 2−k .
In particular, since nk+1 > nk , we have kfnk+1 − fnk k < 2−k .
Definition 1.20 (Absolutely Convergent Series). Let X be a normed space and let {fn }n∈N
be a sequence of elements of X. If
X∞
kfn k < ∞,
n=1
P
then we say that the series n fn is absolutely convergent in X.
PNote that the definition of absolute convergence does not by itself tell us that the series
n fn actually converges. That is always true if X is a Banach space, but need not be true
if X is not complete.
The converse of this exercise is also true, and is often a useful method for proving that a
given normed space is a Banach space.
6 CHRISTOPHER HEIL
Proposition 1.22. Let X be a normed space. Prove that X is a Banach space if and only
if every absolutely convergent series in X converges in X.
Proof. ⇒. This is Exercise 1.21.
⇐. Suppose that every absolutely convergent series is convergent. Let {fn }n∈N be a
Cauchy sequence in X. By Exercise 1.18, therePexists a subsequence {fnk }k∈N such that
kfnk+1 − fnk k < 2−k for every k. The series k (fnk+1 − fnk ) is absolutely convergent,
because
∞
X ∞
X
kfnk+1 − fnk k ≤ 2−k = 1 < ∞.
k=1 k=1
P
By hypothesis, we conclude that k (fnk+1 − fnk ) converges in X, say to f . In terms of the
partial sums, this says that
k
X
fnk+1 − fn1 = (fnj+1 − fnj ) → f as k → ∞,
j=1
or fnk → g = f +fn1 as k → ∞. Thus {fn }n∈N is a Cauchy sequence which has a subsequence
that converges to g. It therefore follows from Exercise 1.17 that fn → g. Therefore X is
complete.
Example 1.23 (The Harmonic Series). To illustrate the difference between convergence and
absolute convergence, consider
P 1 the one-dimensional case, i.e., X = F, the field of scalars.
PN 1Let
1
P
xn = n . Then n xn = n n is the harmonic series, and the partial sums sN = n=1 n of
this series are unbounded.
P 1 P 1not converge. Since sN → ∞, we
Thus the harmonic series does
usually say that n n diverges to infinity, and write n n = ∞.
On the other hand, consider the alternating series n (−1)n n1 . Since the terms alternate
P
signs and since n1 → 0, it follows that this series does converge to a finite scalar (in fact, it
converges to − ln 2). However, it does not converge absolutely.
Definition 1.24 (Unconditionally Convergent Series). P Let X be a Banach space and let
{fn }n∈N be a sequence of elements of X. The series f = ∞ n=1 fn is said to
P converge uncon-
ditionally if every rearrangement of the series converges. That is, f = ∞ n=1 fn converges
unconditionally if for each bijection σ : N → N the series
∞
X
fσ(n)
n=1
converges.
Example 1.28 (The Harmonic Series Revisited). To illustrate the importance of uncondi-
n 1
P
tional convergence, again consider X = F and the alternating series n (−1) n . We know
that this series converges, but does not converge absolutely.
1
Now consider what happens if we change the order of summation. Let pn = 2n and
1
qn = 2n+1 , i.e., the pn are the positive terms from the alternative series and the qn are the
P P
absolute values of the negative terms. Each series n pn and n qn diverges. Hence there
must exist an m1 > 0 such that
p1 + · · · + pm1 > 1.
Then, there must exist an m2 > m1 such that
p1 + · · · + pm1 − q1 + pm1 +1 + · · · + pm2 > 2.
Continuing in this way, we see that
p1 + · · · + pm1 − q1 + pm1 +1 + · · · + pm2 − q2 + · · ·
is a rearrangement of n (−1)n n1 which diverges to +∞.
P
In the same way, we can construct a rearrangement which diverges to −∞, which converges
to any given real number r, or which simply oscillates without ever converging. Moreover,
the same can be done for any series of real scalars which converges conditionally.
Proposition 1.29. Let XPbe a Banach space and let {fn }n∈N be a sequence of elements
of X. Then the series f = ∞ n=1 fn converges unconditionally if and only if it converges with
respect to the net of finite subsets of N, i.e., if
X
∀ ε > 0, ∃ finite F0 ⊆ N such that ∀ finite F ⊇ F0 ,
f − fn
< ε.
n∈F
Definition 1.31 (Limit Points, Closure, Density). Let X be a normed linear space and let
A ⊆ X.
(a) A point f ∈ A is called a limit point of A if there exist fn ∈ A with fn 6= f such that
fn → f .
(b) The closure of A is the smallest closed set Ā such that A ⊆ Ā. Specifically,
T
Ā = {F ⊆ X : F is closed and F ⊇ A}.
Exercise 1.32. (a) The closure of A equals the union of A and all limit points of A:
Ā = A ∪ {x ∈ X : x is a limit point of A} = {z ∈ X : ∃ yn ∈ A such that yn → z}.
(b) If X is a normed linear space, then the closure of an open ball Br (x) is the closed ball
Br (x) = {y ∈ X : kx − yk ≤ r}.
(c) Prove that A is dense if and only if
∀ x ∈ X, ∀ ε > 0, ∃ y ∈ A such that kx − yk < ε.
Solution
⇒. Suppose that F is closed but that there exists a limit point f that does not belong to
F . By definition, there must exist fn ∈ F such that fn → f . However, f ∈ X \ F , which is
open, so there exists some r > 0 such that B(f, r) ⊆ X \ F . Yet there must exist some fn
with kf − fn k < r, so this fn will belong to X \ F , which is a contradiction.
⇐. Exercise.
Remark 1.35. Some authors make the restriction that, when dealing with normed spaces,
the terminology “subspace” is used only for closed subspaces. Other authors use the termi-
nology “linear manifold” to denote a subspace that need not be closed. To avoid ambiguity,
we will use the following terminology.
Exercise 1.37. In finite dimensions, all subspaces are closed sets (this will be proved in
Proposition 3.5). This exercise demonstrates that, in infinite dimensions, subspaces need
not be closed sets.
(a) Fix 1 ≤ p ≤ ∞. Prove that
c00 = {x = (x1 , . . . , xN , 0, 0, . . . ) : N > 0, x1 , . . . , xN ∈ F}
is a subspace of `p (N) that is not closed (with respect to the `p -norm). Prove that c00 is
dense in `p (N) if p < ∞, but that it is not dense in `∞ (N).
(b) Define
c0 = {x = (xk )∞
k=1 : lim xk = 0}.
k→∞
∞ ∞
Prove c0 is a closed subspace of ` (N) (in ` -norm). Prove that c0 is the closure of c00
(under the `∞ -norm).
(c) Fix 1 ≤ p ≤ ∞. Prove that Cc (Rn ), the space of continuous, compactly supported
functions on Rn , is a subspace of Lp (Rn ) that is not closed. Prove that Cc (Rn ) is dense in
Lp (Rn ) if p < ∞, but not if p = ∞.
Hints: For a continuous function we have kf k∞ = sup |f (x)|. Hence, if {fn }n∈N is a
sequence of continuous functions in L∞ (Rn ) that converges in L∞ -norm, then it converges
uniformly. From undergraduate real analysis, we know that the limit of a uniformly conver-
gent sequence of continuous functions is continuous.
(d) Let C0 (Rn ) be the set of all continuous functions f : Rn → F such that
lim f (x) = 0. (1.1)
|x|→∞
More precisely, (1.1) means that for every ε > 0 there exists a compact set K such that
/ K. Prove that C0 (Rn ) is a closed subspace of L∞ (Rn ) (closed in
|f (x)| < ε for all x ∈
L∞ -norm). Prove that C0 (Rn ) is the closure of Cc (Rn ) (under the L∞ -norm).
(e) Let Cb (Rn ) be the set of all bounded, continuous functions f : Rn → F. Prove that
Cb (Rn ) is a closed subspace of L∞ (Rn ).
(f) Fix 1 ≤ p ≤ ∞, and let E be a (Lebesgue) measurable subset of Rn . Let M = {f ∈
Lp (Rn ) : supp(f ) ⊆ E}. Prove that M is a closed subspace of Lp (Rn ).
Remark 1.38. (a) We took the domain in the preceding exercise to be Rn just for conve-
nience; the definitions of the spaces Cc , C0 , Cb can be extended to domains that are more
general topological spaces.
(b) Beware that some authors use the notation C0 for the space that we are calling Cc !
Exercise 1.39. Let X be a Banach space and let M be a subspace of X. Then M is itself
a Banach space (using the norm from X) if and only if M is closed.
10 CHRISTOPHER HEIL
Solution
⇒. Suppose that M is a Banach space. Let f be any limit point of M , i.e., suppose
fn ∈ M and fn → f . Then {fn } is a convergent sequence in X, and hence is Cauchy in
X. Since each fn belongs to M , it is also Cauchy in M . Since M is a Banach space, {fn }
must therefore converge in M , i.e., fn → g for some g ∈ M . However, limits are unique, so
f = g ∈ M . Therefore M contains all its limit points and hence is closed.
⇐. Exercise.
Notation 1.40 (Notation for Closed Subspaces). Since we will often deal with closed sub-
spaces of a Banach space, we declare that the notation
M ≤ X
means that M is a closed subspace of the Banach space X.
Exercise 1.41. Find examples of normed spaces that are not Banach spaces.
Hint: Look for examples of normed spaces Y which are subspaces of a larger Banach
space X.
Remark 1.42. Is every normed space Y a subspace of a larger Banach space? The answer
is yes, given a normed space Y it is always possible to construct a Banach space X ⊇ Y such
that the norm on X, when restricted to Y , is the same as the norm on Y , and Y is dense in
X with respect to that norm. This space is called the completion of Y .
Definition 1.43. Suppose that X is a normed linear space with respect to a norm k · ka
and also with respect to another norm k · kb . Then we say that these norms are equivalent
if there exist constants C1 , C2 > 0 such that
∀ f ∈ X, C1 kf ka ≤ kf kb ≤ C2 kf ka . (1.2)
1 1
Observe that equation (1.2) can be rearranged to read C2
kf kb ≤ kf ka ≤ C1
kf kb .
Exercise 1.44. Let X be a vector space. If k · ka and k · kb are two norms on X, define
k · ka ∼ k · kb if k · ka and k · kb are equivalent. Prove that ∼ is an equivalence relation on
the class of norms on X.
The next result will show that equivalent norms define the same topology and the same
convergence criterion.
Proposition 1.45. Let k·ka and k·kb be two norms on a vector space X. Then the following
statements are equivalent.
(a) k · ka and k · kb are equivalent norms.
(b) k · ka and k · kb define the same topologies on X. That is, if U ⊆ X, then U is open
with respect to k · ka if and only if it is open with respect to k · kb .
CHAPTER 3. BANACH SPACES 11
(c) k · ka and k · kb define the same convergence criterion. That is, if {fn }n∈N is a sequence
in X and f ∈ X, then
lim kf − fn ka = 0 ⇐⇒ lim kf − fn kb = 0.
n→∞ n→∞
Proof. (b) ⇒ (a). Assume that statement (b) holds. Let Bra (f ) and Brb (f ) denote the open
balls of radius r centered at f ∈ X with respect to k · ka and k · kb . Since B1a (0) is open
with respect to k · ka , the hypothesis that statement (b) holds implies that B1a (0) is open
with respect to k · kb . Therefore, since 0 ∈ B1a (0), there must exist some r > 0 such that
Brb (0) ⊂ B1a (0).
Now choose any f ∈ X and any ε > 0. Then
(r − ε) f
∈ Brb (0) ⊆ B1a (0),
kf kb
so
(r − ε) f
< 1.
kf kb
a
Rearranging, this implies (r − ε) kf ka < kf kb . Since this is true for every ε, we conclude
that r kf ka ≤ kf kb .
A symmetric argument, interchanging the roles of the two norms, shows that there exists
an s > such that kf kb ≤ s kf ka for every f ∈ X. Hence the two norms are equivalent
The remaining implications are exercises.
Hints on (c) ⇒ (a): Suppose that statement (c) holds. Show that this implies that the
function ν : X → R given by ν(f ) = kf kb is continuous with respect to the norm k · ka . Since
(−1, 1) is an open subset of R, it follows that ν −1 (−1, 1) is an open subset of X (with respect
to k · ka ). Since 0 ∈ ν −1 (−1, 1), there must exist an r > 0 such that Brb (0) ⊂ ν −1 (−1, 1).
But ν −1 (−1, 1) = B1a (0), so the remainder of the proof proceeds exactly like the proof of (b)
⇒ (a).
Definition 2.2 (Continuous and Bounded Operators). Let X, Y be normed linear spaces,
and let L : X → Y be a linear operator.
(a) L is continuous at a point f ∈ X if fn → f in X implies Lfn → Lf in Y .
(b) L is continuous if it is continuous at every point, i.e., if fn → f in X implies Lfn → Lf
in Y for every f .
(c) L is bounded if there exists a finite K ≥ 0 such that
∀ f ∈ X, kLf k ≤ K kf k.
Note that kLf k is the norm of Lf in Y , while kf k is the norm of f in X.
(d) The operator norm of L is
kLk = sup kLf k.
kf k=1
(e) We let B(X, Y ) denote the set of all bounded linear operators mapping X into Y ,
i.e.,
B(X, Y ) = {L : X → Y : L is bounded and linear}.
If X = Y then we write B(X) = B(X, X).
(f) If Y = F then we say that L is a functional. The set of all bounded linear functionals
on X is the dual space of X, and is denoted
X 0 = B(X, F) = {L : X → F : L is bounded and linear}.
Another common notation for the dual space is X ∗ .
Note that since the norm on F is just absolutely value, the operator norm of a linear
functional L ∈ X 0 = B(X, F) is kLk = sup |Lf |.
kf k=1
Exercise 2.3. Show that if T : X → Y is linear and continuous, then ker(T ) is a closed
subspace of X and that range(T ) is a subspace of Y . Must range(T ) be a closed subspace?
(e) If L is bounded then kLk is the smallest value of K such that kLf k ≤ Kkf k holds
for all f ∈ X.
kLf k
(f) kLk = sup kLf k = sup .
kf k≤1 f 6=0 kf k
Exercise 2.5. Show that B(X, Y ) is a subspace of the vector space V consisting of ALL
functions A : X → Y . Moreover, show that the operator norm is a norm on the space
B(X, Y ), i.e.,
(a) 0 ≤ kLk < ∞ for all L ∈ B(X, Y ),
(b) kLk = 0 if and only if L = 0 (the zero operator that sends every element of X to the
zero vector in Y ),
(c) kαLk = |α| kLk for every L ∈ B(X, Y ) and every α ∈ F,
(d) kL + Kk ≤ kLk + kKk for every L, K ∈ B(X, Y ).
The preceding exercise shows that B(X, Y ) is a normed linear space, and we will show
in Theorem 2.18 that it is a Banach space if Y is a Banach space. However, in addition to
vector addition and scalar multiplication operations, there is a third operation that we can
perform with functions: composition.
Exercise 2.6. Prove that the operator norm is submultiplicative, i.e., prove if A ∈ B(X, Y )
and B ∈ B(Y, Z), then BA ∈ B(X, Z) and
kBAk ≤ kBk kAk. (2.1)
Exercise 2.7. Let Fn be n-dimensional Euclidean space over F, under the Euclidean norm,
and let Y be any normed linear space. Prove that if L : Fn → Y is linear, then L is bounded.
Hint: If x = (x1 , . . . , xn ) ∈ Fn then x = x1 e1 +· · ·+xn en where {e1 , . . . , en } is the standard
basis for Fn . Use the Triangle and the Cauchy-Schwarz Inequalities.
Solution
Given x ∈ Fn , we have
n
n n
X n
1/2 X 1/2
X
X
2 2
kLxk =
≤
L(xk ek )
|xk | kLek k ≤ |xk | kLek k = C kxk,
k=1 k=1 k=1 k=1
2 1/2
Pn
where C = k=1 kLek k . Hence L is bounded.
Remark 2.8. We will prove later that if X is any finite-dimensional vector space and k · k is
any norm on X, then any linear function L : X → Y is bounded. To do this we will use the
fact (that we will prove later) that all norms on a finite-dimensional space are equivalent.
14 CHRISTOPHER HEIL
The next lemma is a standard fact about continuous functions. L−1 (U ) denotes the inverse
image of U ⊆ Y , i.e., L−1 (U ) = {f ∈ X : Lf ∈ U }.
Solution
⇒. Suppose that L is continuous and that U is an open subset of Y . We will show that
X \ L−1 (U ) is closed by showing that it contains all its limit points.
Suppose that f is a limit point of X \ L−1 (U ). Then there exist fn ∈ X \ L−1 (U ) such that
fn → f . Since L is continuous, this implies Lfn → Lf . However, fn ∈ / L−1 (U ), so Lfn ∈
/ U,
i.e., Lfn ∈ Y \ U , which is a closed set. Therefore Lf ∈ Y \ U , and hence f ∈ X \ L−1 (U ).
Thus X \ L−1 (U ) is closed, so L−1 (U ) is open.
⇐. Exercise.
Remark 2.15. We will see later that if X and Y are Banach spaces and L : X → Y
is a bounded bijection, then L−1 is automatically bounded and hence L is a topological
isomorphism. Thus, when X and Y are Banach spaces, every continuous invertible map is a
topological isomorphism. Sometimes the abbreviation isomorphism or invertible map is used
to mean a topological isomorphism, but it should be noted that these terms are ambiguous.
Exercise 2.17. Let X be a Banach space and Y a normed linear space. Suppose that
L : X → Y is bounded and linear. Prove that if there exists c > 0 such that kLf k ≥ ckf k
for all f ∈ X, then L is injective and range(L) is closed.
16 CHRISTOPHER HEIL
The next theorem shows that B(X, Y ) is a Banach space whenever Y is a Banach space.
Theorem 2.18. If X is a normed space and Y is a Banach space, then B(X, Y ) is a Banach
space.
Proof. Assume that {An }n∈N is a sequence of operators An ∈ B(X, Y ) that is Cauchy in
operator norm. For any given f ∈ X, we have
kAm f − An f k ≤ kAm − An k kf k,
so we conclude that {An f }n∈N is a Cauchy sequence of vectors in Y . Since Y is complete,
this sequence must converge, say Afn → g ∈ Y . Define Af = g. This gives us a candidate
limit operator A.
Exercise: Show that A defined in this way is a linear operator.
To show that A is bounded, first recall that all Cauchy sequences are bounded. Hence we
must have C = sup kAn k < ∞. If f ∈ X, then since An f → Af we have
kAf k = lim kAn f k ≤ sup kAn f k ≤ sup kAn k kf k = C kf k.
n→∞ n∈N n∈N
Corollary 2.19. If X is a normed space, then its dual space X 0 = B(X, F) is a Banach
space.
The next exercise deals with the problem of extending an operator defined only a dense
subspace to the entire space.
Solution
(a) Fix any f ∈ X. Since Y is dense in X, there exist gn ∈ Y such that gn → f . Since
L is bounded, we have kLgm − Lgn k ≤ kLk kgm − gn k. But {gn }n∈N is Cauchy in X, so this
implies that {Lgn }n∈N is Cauchy in Z. Since Z is a Banach space, we conclude that there
exists an h ∈ Z such that Lgn → h. Define L̃f = h.
To see that L̃ is well-defined, suppose that we also had gn0 → f for some gn0 ∈ Y . Then
kLgn0 −Lgn k ≤ kLk kgn0 −gn k → 0. Since Lgn → h, it follows that Lgn0 = Lgn +(Lgn0 −Lgn ) →
h + 0 = h. Thus L̃ is well-defined.
To see that L̃ is linear, suppose that f , g ∈ X are given and c ∈ F. Then there exist fn ,
gn ∈ Y such that fn → f and gn → g. Since cfn + gn → cf + g and
L(cfn + gn ) = cLfn + Lgn → cL̃f + L̃g,
by definition we have that L̃(cf + g) = cL̃f + L̃g.
To see that L̃ is an extension of L, suppose that g ∈ Y is fixed. If we set gn = g, then
gn → g and Lgn → Lg, so by definition we have L̃g = Lg. Hence the restriction of L̃ to Y
is L. Consequently,
kL̃k = sup kL̃f k ≥ sup kL̃f k = sup kLf k = kLk.
f ∈X, kf k=1 f ∈Y, kf k=1 f ∈Y, kf k=1
Finally, suppose that f ∈ X. Then there exist gn ∈ Y such that gn → f and Lgn → L̃f ,
so
kL̃f k = lim kLgn k ≤ lim kLk kgn k = kLk kf k.
n→∞ n→∞
Hence kL̃k ≤ kLk. Combining this with the opposite inequality derived above, we conclude
that kL̃k = kLk.
(b) Suppose that L : Y → range(Y ) is a topological isomorphism. We already know that
L̃ : X → range(L̃) is bounded. We need to show that L̃ is injective, that L̃−1 : range(L̃) → X
is bounded, and that range(L̃) = range(L).
Fix any f ∈ X. Then there exist gn ∈ Y such that gn → f and Lgn → L̃f . Since L is a
topological isomorphism, we have by Exercise 2.16 that kgn k ≤ kL−1 k kLgn k. Hence
kgn k kf k
kL̃f k = lim kLgn k ≥ lim −1
= .
n→∞ n→∞ kL k kL−1 k
Consequently, L̃ is injective and for any h ∈ range(L̃) we have
kL̃−1 hk ≤ kL−1 k kL̃(L̃−1 h)k = kL−1 k khk.
Therefore L̃−1 : range(L̃) → X is bounded.
It remains only to show that the range of L̃ is the closure of the range of L. If f ∈ X, then
by definition there exist gn ∈ Y such that gn → f and Lgn → L̃f . Hence L̃f ∈ range(L), so
range(L̃) ⊂ range(L).
18 CHRISTOPHER HEIL
On the other hand, suppose that h ∈ range(L) Then there exist gn ∈ Y such that Lgn → h.
Since L̃−1 is bounded and L̃ extends L, we conclude that gn = L̃−1 (Lgn ) → L̃−1 (h). Hence
f = L̃−1 (h), so f ∈ range(L̃).
[x]B = ... .
cn (x)
Then the mapping T : X → Fn given by x 7→ [x]B is, by definition of basis, a linear bijection
of X onto Fn .
Exercise 3.2 (`pw Norms on X). Let X be a finite-dimensional vector space over F and let
B = {e1 , . . . , en } be any basis. Fix any 1 ≤ p ≤ ∞ and any weight w : {1, . . . , n} → (0, ∞).
Using the notation of Example 3.1, given x ∈ X define
n 1/p
X
p p
|ck (x)| w(k) , 1 ≤ p < ∞,
kxkp,w =
[x]B
p,w = k=1
max |ck (x)| w(k), p = ∞.
k
Note that while we use the same symbol k · kp,w to denote a function on X and on Fn , by
context it has different meanings depending on whether it is being applied to an element of
X or to an element of Fn .
Prove the following.
(a) k · kp,w is a norm on X.
(b) x 7→ [x]B is a isometric isomorphism of X onto Fn (using the norm k · kp,w on X and
the norm k · kp,w on Fn ).
CHAPTER 3. BANACH SPACES 19
(c) Let {xn }n∈N be a sequence of vectors in X and let x ∈ X. Prove that xn → x with
respect to the norm k · kp,w on X if and only if the coordinate vectors [xn ]B converge
componentwise to the coordinate vector [x]B .
(d) X is complete in the norm k · kp,w .
Now we can show that all norms on a finite-dimensional space are equivalent.
Theorem 3.3. If X is a finite-dimensional vector space over F, then any two norms on X
are equivalent.
Proof. Let B = {e1 , . . . , en } be any basis for X, and let k · k∞ be the norm on X defined in
Exercise 3.2. Since equivalence of norms is an equivalence relation, it suffices to show that
an arbitrary norm k · k on X is equivalent to k · k∞ .
Using the notation of Exercise 3.1, given x ∈ X we can write x uniquely as x = c1 (x) e1 +
· · · + cn (x) en . Therefore,
Xn Xn
kxk ≤ |ck (x)| kek k ≤ kek k max |ck (x)| = C2 kxk∞ ,
k
k=1 k=1
Pn
where C2 = k=1 kek k is a nonzero constant independent of x.
It remains to show that there is a constant C1 > 0 such that C1 kxk∞ ≤ kxk for every x.
First, let
D = {x ∈ X : kxk∞ = 1}
∞
be the ` -unit circle in X.
Exercise: Show that D is compact (with respect to the norm k · k∞ ). Hints: Suppose
that {xn }n∈N is a sequence of vectors in D. Then for each n, we have |ck (xn )| = 1 for some
k ∈ {1, . . . , n}. Hence there must be some k such that |ck (xnj )| = 1 for infinitely many j.
Since {c1 (xnj )}j∈N is an infinite sequence of scalars in the compact set {c ∈ F : |c| ≤ 1}, we
can select a subsequence whose first coordinates converge. Repeat for each coordinate, and
remember that the kth coordinate is always 1. Hence we can construct a subsequence that
converges to x ∈ D with respect to the `∞ -norm.
Our next goal is to show that D is also compact with respect to the norm k · k. Let
{xn }n∈N be any sequence of vectors in D. Since D is compact with respect to k · k∞ ,
there exists a subsequence {xnk }k∈N and an x ∈ D such that kx − xnk k∞ → 0. But then
kx − xnk k ≤ C2 kx − xnk k∞ → 0, so {xnk }k∈N is a subsequence that converges to x ∈ X with
respect to k · k. Hence D is compact with respect to k · k.
Now, k · k is a continuous function with respect to the convergence criteria defined by k · k
(this is part (b) of Exercise 1.11). The set D is compact with respect to the topology defined
by k · k. A real-valued continuous function on a compact set must achieve a maximum and
minimum on that set. Hence, there must exist constants m and M such that m ≤ kxk ≤ M
for all x ∈ D. Since x ∈ D if and only if kxk∞ = 1, this implies that
∀ x ∈ X, m kxk∞ ≤ kxk ≤ M kxk∞ .
If we had m = 0, then this would imply that there is an x ∈ D such that kxk = 0. But then
x = 0, which implies kxk∞ = 0, contradicting the fact that x ∈ D. Hence m > 0, so we can
take C1 = m.
20 CHRISTOPHER HEIL
Consequently, from now on we need not specify the norm on a finite-dimensional vector
space X—we can take any norm that we like whenever we need it.
Exercise 3.4. Let Fm×n be the space of all m × n matrices with entries in F. Fm×n is
naturally isomorphic to Fmn .
Prove that if k · ka is any norm on Fm×n , k · kb is any norm on Fn×k , and k · kc is any norm
on Fm×k , then there exists a constant C > 0 such that
∀ A ∈ Fm×n , ∀ B ∈ Fn×k , kABkc ≤ C kAka kBkb .
Exercise 3.7. Let X be a finite-dimensional normed space, and let Y be a normed linear
space. Prove that if L : X → Y is linear, then L is bounded.
Lemma 3.8 (F. Riesz’s Lemma). Let M be a proper, closed subspace of a normed space X.
Then for each ε > 0, there exists g ∈ X with kgk = 1 such that
a
Fix δ > 0 small enough that a+δ > 1 − ε. By definition of infimum, there exists v ∈ M such
that a ≤ ku − vk < a + δ. Set
u−v
g = ,
ku − vk
and note that kgk = 1. Given f ∈ M we have h = v + ku − vk f ∈ M , so
u − v − ku − vk f
kg − f k =
= ku − hk > a > 1 − ε.
ku − vk
ku − vk a+δ
Exercise 3.9. Let X be a normed linear space. Let B = {x ∈ X : kxk ≤ 1} be the closed
unit ball in X. Prove that if B is compact, then X is finite-dimensional.
Hints: Suppose that X is infinite-dimensional. Given any nonzero e1 with ke1 k ≤ 1, by
Lemma 3.8 there exists e2 ∈ X \span{e1 } with ke2 k ≤ 1 such that ke2 −e1 k > 21 . Continue in
this way to construct vectors ek such that {e1 , . . . , en } are independent for any n. Conclude
that X is infinite-dimensional.
Definition 3.10. We say that a normed linear space X is locally compact if for each f ∈ X
there exists a compact K ⊂ X with nonempty interior K ◦ such that f ∈ K ◦ .
Example 4.1 (Cosets in R2 ). Consider the vector space X = R2 . Let M be any one-
dimensional subspace of R2 , i.e., M is a line in R2 through the origin. A coset of M is
simply a rigid translate of M by a vector in R2 . For concreteness, let us specifically consider
the case where M is the x1 -axis in R2 , i.e., M = {(x1 , 0) : x1 ∈ R}. Then given a vector
y = (y1 , y2 ) ∈ R2 , the coset y + M is the set
y + M = {y + m : m ∈ M } = {(y1 + x1 , y2 + 0) : x1 ∈ R} = {(x1 , y2 ) : x1 ∈ R},
which is the horizontal line at height y2 . This is not a subspace of R2 , but it is a rigid
translate of the x1 -axis. Note that there are infinitely many different choices of y that give
the same coset. Furthermore, we have the following facts for this particular setting.
(a) Two cosets are either identical or entirely disjoint.
(b) The union of all the cosets is all of R2 .
(c) The set of distinct cosets is a partition of R2 .
Definition 4.2 (Cosets). Let M be a subspace of a vector space X. Then the cosets of M
are the sets
f + M = {f + m : m ∈ M }, f ∈ M.
Definition 4.4 (Quotient Space). If M is a subspace of a vector space X, then the quotient
space X/M is
X/M = {f + M : f ∈ X}.
CHAPTER 3. BANACH SPACES 23
Since two cosets of M are either identical or disjoint, the quotient space X/M is simply
the set of all the distinct cosets of M .
Example 4.5. Again let M = {(x1 , 0) : x1 ∈ R} be the x1 -axis in R2 . Then, by Example 4.1,
we have that
R2 /M = {y + M : y ∈ R2 } = {(x1 , 0) + M : x1 ∈ R},
i.e., R2 /M is the set of all horizontal lines in R2 . Note that R2 /M is in 1-1 correspondence
with the set of distinct heights, i.e., there is a natural bijection of R2 /M onto R. This is a
special case of a more general fact that we will explore.
Next we define two natural operations on the set of cosets: addition of cosets and multi-
plication of a coset by a scalar. These are defined formally as follows.
Remark 4.7. Before proceeding, we must show that these operations are actually well-
defined. After all, there need not be just one f that determines the coset f + M —how do
we know that if we choose different vectors that determine the same cosets, we will get the
same result when we compute (f + g) + M ? We must show that f1 + M = f2 + M and
g1 + M = g2 + M then (f1 + g1 ) + M = (f2 + g2 ) + M in order to know that Definition 4.6
makes sense.
Now we can show that the quotient space is actually a vector space under the operations
just defined.
Example 4.12. Let C(R) be space of continuous functions on R, and let P be the sub-
space containing the polynomials. Given f ∈ C(R), the coset f + P is f + P = {f + p :
p is a polynomial}. Further, f + P = g + P if and only if f − g is a polynomial. Thus, f + P
can be thought of as “f modulo the polynomials,” i.e., it is the equivalence class obtained
by identifying functions which differ by a polynomial.
In the same way, a coset f + M can be thought of as the equivalence class obtained by
identifying vectors which differ by an element of M . We can imagine the mapping that takes
f to f + M as “collapsing information modulo M .”1
Solution
(c) Suppose that u ∈ E and m ∈ M are given. Then
π(u + m) = u + m + M = u + M = π(u) ∈ π(E).
Hence u + m ∈ π −1 π(E) .
We will mostly be interested in the case where X is a normed space. The following result
shows that X/M is a semi-normed space in general, and is a normed space if M is closed.
Now we derive some basic properties of the canonical projection π of X onto X/M .
Proposition 4.16. Let M be a closed subspace of a normed linear space X. Then the
following statements hold.
(a) kπ(f )k = kf + M k ≤ kf k for each f ∈ X.
X/M
(b) Let BrX (f ) denote the open ball of radius r in X centered at f , and let Br (f + M )
denote the open ball of radius r in X/M centered at f + M . Then for any f ∈ X
and r > 0 we have
π BrX (f ) = BrX/M (f + M ).
(b) First consider the case f = 0 and r > 0. Suppose that g + M ∈ π BrX (0) . Then
g + M = h + M for some h ∈ BrX (0), i.e., khk < r. Hence kg + M k = kh + M k ≤ khk < r,
X/M
so g + M ∈ Br (0 + M ).
X/M
Now suppose that g + M ∈ Br (0 + M ). Then inf m∈M kg − mk = kg + M k < r. Hence
there exists m ∈ M such that kg − mk < r. Thus g − m ∈ BrX (0), so
g + M = g − m + M = π(g − m) ∈ π BrX (0) .
26 CHRISTOPHER HEIL
Therefore W is open.
(d) Suppose that U is an open subset of X. Then by Exercise 4.14(c), we have
π −1 π(U ) = U + M = {u + m : u ∈ U, m ∈ M } =
S
(U + m).
m∈M
But each set U + m, being the translate of the open set U , is itself open. Hence π −1 π(U )
is open, since it is a union of open sets. Part (c) therefore implies that π(U ) is open in
X/M .
Exercise 4.17. Let M be a closed subspace of a normed space X, and let π be the canonical
projection π of X onto X/M . Prove that kπk = 1.
Hint: Lemma 3.8.
Now we can prove that if X is a Banach space, then X/M inherits a Banach space structure
from X.
The following exercise shows that the converse of the preceding theorem is true as well.
Exercise 4.19. Let M be a closed subspace of a normed space X. Prove that if M and
X/M are both complete, then X must be complete.
Hints: Suppose that {fn }n∈N is a Cauchy sequence in X. Show that {fn + M }n∈N is a
Cauchy sequence in X/M , hence converges to some coset f + M . Thus kf − fn + M k → 0.
Does this imply that there exists a g ∈ M such that kf − fn + gk → 0? Or perhaps vectors
gn ∈ M such that kf − fn + gn k → 0?
The quotient space and canonical map will be useful tools for proving many later results.
The following proof illustrates their utility.
Solution
(b) Let {fn + M }n∈N be a countable dense subset of X/M , and let {gn }n∈N be a countable
dense subset of M . Then S = {fm + gn }m,n∈N is a countable subset of X, and we claim that
it is dense.
To see this, fix any f ∈ X. Then there exists an m such that
ε
inf kf − fm + hk = kf − fm + M k = k(f + M ) − (fm + M )k < .
h∈M 2
ε
Hence, there exists some h ∈ M such that kf − fm + hk < 2 . Since h ∈ M , there exists an
n such that kh − gn k < 2ε . Therefore
ε ε
kf − (fm + gn )k ≤ kf − fm − hk + kh − gn k < + = ε.
2 2
Exercise 4.22. Recall from Exercise 1.37 that c0 is a closed subspace of the Banach space `∞ .
(a) Prove that `∞ is not separable.
Hint: Consider
S = {(x1 , x2 , . . . ) : xk = 0 or 1 for every k}.
What is the distance between two distinct elements of S?
(b) Let x, y be vectors in S Prove that if xk 6= yk for at most finitely many k, then
6 yk for infinitely many k, then kx − y + c0 k = 1.
x + c0 = y + c0 . Prove that if xk =
(c) Use part (b) to prove directly that `∞ /c0 is not separable.
(e) Prove that the standard basis {en }n∈N is a Schauder basis for c0 (with respect to the
`∞ -norm).
(f) Use part (e) to show that c0 is separable.
Hint: Use one of the exercises about Schauder bases from the Chapter 1 lecture notes.
(g) Use part (e) and Exercise 5.4 to show that `∞ /c0 is not separable.
In the Hilbert space case, there is a very close relationship between π and the orthogonal
projection of H onto M ⊥ .
CHAPTER 3. BANACH SPACES 29
Exercise 4.23. Let M be a closed subspace of a Hilbert space H, and let π be the canonical
projection of H onto H/M . Prove that the restriction of π to M ⊥ is an isometric isomorphism
of M ⊥ onto H/M .
Remark 4.24. There is no analog of the preceding result for arbitrary Banach spaces. If X
is a Banach space and M is a closed subspace then we say that M is complemented in X if
there exists another closed subspace N such that M ∩ N = {0} and M + N = X.
It is not true that every closed subspace of every Banach space is complemented. In
particular, c0 is not complemented in `∞ . Also, if 1 < p ≤ ∞ and p 6= 2, then `p has
uncomplemented subspaces.
Next we will define the product or direct sum of normed spaces. An analogous definition
holds for the case of a finite collection of spaces.
Definition 4.25. Let {Xi }i∈N be a countable family of Banach spaces, and let k · ki denote
the norm on Xi . Define
∞
Y
Xi = f = (f1 , f2 , . . . ) : fi ∈ Xi .
i=1
For p = ∞, define
∞
Y
L
∞ Xi = f∈ Xk : kf k∞ = sup kfi ki < ∞ .
i
i=1
4.26. Let {Xi }i∈N be a countable family of Banach spaces and fix 1 ≤ p ≤ ∞.
ExerciseL
Let X = p Xi . Prove the following.
(a) X is a normed space.
(b) For each i, the projection Pi : X → Xi given by Pi (f1 , f2 , . . . ) = fi is continuous, and
kPi k = 1.
(c) X is a Banach space if and only if each Xi is a Banach space.
Exercise 4.27. Let X1 , . . . , Xn be finitely many normed spaces. Prove that the spaces ⊕p Xi
are equal for 1 ≤ p ≤ ∞, and that all the norms k · kp are equivalent. For this reason, we
often denote this space by X1 × · · · × Xn .
30 CHRISTOPHER HEIL
5. Linear Functionals
Definition 5.1 (Hyperplane). Let M be a subspace of a vector space X. Then we say that
M is a hyperplane if it has codimension 1, i.e., if codim(M ) = dim(X/M ) = 1.
Example 5.2. Let H be a Hilbert space, and let L be a bounded linear functional on H.
That is, L : H → F is a bounded linear operator, so L∗ : F → H is also a bounded linear
operator. Now, F is one-dimensional, and a linear operator is entirely determined by what
it does to the basis elements. In particular, {1} is a basis for F, so L∗ (c) = L∗ (c1) = c L∗ (1).
Hence range(L∗ ) = span{L∗ (1)}. If L∗ (1) = 0 then L∗ is the zero operator, and hence L is
the zero operator as well (why?). Otherwise, range(L∗ ) must be one-dimensional, and hence
is a closed subspace of H.
Therefore,
H = range(L∗ ) ⊕ range(L∗ )⊥ = range(L∗ ) ⊕ ker(L).
Hence, every vector f ∈ H can be written uniquely as
f = c L∗ (1) + k
for some scalar c ∈ range(L∗ ) and k ∈ ker(L). Hence, if π is the canonical projection of H
onto H/ ker(L), then since k ∈ ker(L) we have
f + ker(L) = π(f ) = cL∗ (1) + k + ker(L) = cL∗ (1) + ker(L).
Therefore
H/ ker(L) = {f + ker(L) : f ∈ H} = {cL∗ (1) + k + ker(L) : c ∈ F, k ∈ ker(L)}
= {cL∗ (1) + ker(L) : c ∈ F}
= span{L∗ (1) + ker(L)},
which is one-dimensional.
Thus, if L is a bounded linear functional on a Hilbert space H, then we conclude that
ker(L) is a hyperplane in H.
We will show that a similar result holds for arbitrary normed spaces. We will need the
following tool. In abstract algebra, the group version of the next result is called the First
Isomorphism Theorem or the Homomorphism Theorem. In the group setting, an isomor-
phism is a bijective homomorphism. In the vector space setting, an isomorphism is a linear
bijection.
CHAPTER 3. BANACH SPACES 31
Exercise 5.3 (Isomorphism Theorem). Let X and Y be vector spaces, and let ϕ : X → Y
be a linear surjection. Let M = ker(ϕ). Prove that ψ : X/M → Y given by ψ(f +M ) = ϕ(f )
is a well-defined linear bijection, and that ϕ = ψ ◦ π.
Now we can show that every hyperplane is the kernel of some (not necessarily continuous)
linear functional.
Proposition 5.5. Let M be a subspace of a normed space X. Then the following statements
are equivalent.
(a) M is a hyperplane.
(b) M = ker(µ) for some nonzero linear functional µ : X → F.
Proof. (b) ⇒ (a). Suppose that M = ker(µ) where µ is a nonzero linear functional. Then
by the Isomorphism Theorem, there is a linear bijection ψ : X/ ker(µ) → F. Since F is
one-dimensional, we conclude that X/ ker(µ) is as well.
(a) ⇒ (b). Assume that M is a hyperplane. Then X/M is one-dimensional, so there
exists a linear bijection ψ : X/M → F. Set ϕ = ψ ◦ π, then ψ : X → F. If f ∈ M , then
π(f ) = f + M = 0 + M , so ψ(f ) = ψ(π(f )) = ψ(0 + M ) = 0, so f ∈ ker(ϕ). Conversely,
if f ∈ ker(ϕ) then we have ψ(π(f )) = ϕ(0 + M ) = 0. But ψ is a bijection, so this implies
f + M = π(f ) = 0 + M . Hence f ∈ M , so ker(ϕ) ⊂ M .
Proof. We are given that X/M is one-dimensional. Let π be the canonical projection of X
onto X/M . The closure M of M is a subspace of X, so since π is linear we know that π(M )
is a subspace of X/M . But X/M is one-dimensional, so there only two possibilities.
First, we could have π(M ) = {0 + M }. In this case, M ⊂ ker(π) = M , so we have M = M
and M is closed.
Second, we could have π(M ) = X/M . In this case, we have by Exercise 4.14(c) that
X = π −1 (X/M ) = π −1 (π(M )) = M + M = M ,
Proof. ⇒. Exercise.
⇐. Suppose that ker(µ) is closed. We know by Proposition 5.5 that ker(µ) is a hyperplane,
so X/ ker(µ) is one-dimensional. Let π be the canonical projection of X onto X/ ker(µ).
Because M is closed, we know that π is continuous. By the Isomorphism Theorem, there
exists a linear bijection ψ : X/ ker(µ) → F. Since X/ ker(µ) is a one-dimensional normed
linear space, and linear map from X/ ker(µ) into another normed space is continuous by
Exercise 3.7. Therefore ψ is continuous, and hence µ = ψ ◦ π is continuous.
Since the norm on F is just absolute value, the operator norm of a linear functional L ∈
X 0 = B(X, F) is
kLk = sup |Lf |.
kf k=1
Since F is a Banach space, the dual space X 0 is a Banach space even if X is not.
In order to give some examples of dual spaces, we recall Hölder’s Inequality.
Exercise A.4. Prove that if X is a metric space, then (X, T ) is a topological space using
the preceding definition.