Professional Documents
Culture Documents
Lfreichnotesfuncan
Lfreichnotesfuncan
Term 1, 20102011
Vassili Gelfreich
Contents
1
Vector spaces
1.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Examples of vector spaces . . . . . . . . . . . . . . . . . . . . . . .
1.3 Hamel bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Normed spaces
2.1 Norms . . . . . . . . . . . . . . . . . . . .
2.2 Four famous inequalities . . . . . . . . . .
2.3 Examples of norms on a space of functions
2.4 Equivalence of norms . . . . . . . . . . . .
2.5 Linear Isometries . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
8
8
9
10
11
12
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
14
14
16
17
18
Banach spaces
4.1 Completeness: Definition and examples . . . . . . . . . . . . . . . .
4.2 The completion of a normed space . . . . . . . . . . . . . . . . . . .
4.3 Weierstrass Approximation Theorem . . . . . . . . . . . . . . . . . .
20
20
22
25
Lebesgue spaces
5.1 Lebesgue measure . . .
5.2 Lebesgue integral . . .
5.3 Lebesgue space L1 (R)
5.4 L p spaces . . . . . . .
.
.
.
.
28
28
30
32
34
.
.
.
.
36
36
37
38
40
.
.
.
.
.
.
41
41
42
44
44
46
47
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Hilbert spaces
6.1 Inner product spaces . . . . . . . . . . . .
6.2 Natural norms . . . . . . . . . . . . . . . .
6.3 Parallelogram law and polarisation identity
6.4 Hilbert spaces: Definition and examples . .
Orthonormal bases in Hilbert spaces
7.1 Orthonormal sets . . . . . . . . . .
7.2 Gram-Schmidt orthonormalisation .
7.3 Bessels inequality . . . . . . . . .
7.4 Convergence . . . . . . . . . . . .
7.5 Orthonormal basis in a Hilbert space
7.6 Separable Hilbert spaces . . . . . .
iii
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
2
4
50
50
51
53
56
56
58
59
10 Linear functionals
10.1 Definition and examples . . . . . . . . . . . . . . . . . . . . . . . .
10.2 Riesz representation theorem . . . . . . . . . . . . . . . . . . . . . .
61
61
61
63
63
64
66
69
69
70
71
13 Compact operators
13.1 Definition, properties and examples . . . . . . . . . . . . . . . . . .
13.2 Spectral theory for compact self-adjoint operators . . . . . . . . . . .
74
74
75
14 Sturm-Liouville problems
79
iv
Preface
These notes follow the lectures on Functional Analysis given in the Autumn of 2010.
If you find a mistake or misprint please inform the author by sending an e-mail to
v.gelfreich@warwick.ac.uk. The author thanks James Robinson for his set of
notes and selection of exercises which significantly facilitated the preparation of the
lectures.
Vector spaces
1.1
Definition.
A vector space V over a field K is a set equipped with two binary operations called
vector addition and multiplication by scalars. Elements of V are called vectors and
elements of K are called scalars. The sum of two vectors x, y V is denoted x + y, the
product of a scalar K and vector x V is denoted x.
It is possible to consider vector spaces over an arbitrary field K, but we will consider the fields R and C only. So we will always assume that K denotes either R or C
and refer to V as a real or complex vector space respectively.
In a vector space, addition and multiplication have to satisfy the following set of
axioms: Let x, y, z be arbitrary vectors in V , and , be arbitrary scalars in K, then
Associativity of addition: x + (y + z) = (x + y) + z.
Commutativity of addition: x + y = y + z.
There exists an element 0 V , called the zero vector, such that x + 0 = x for all
x V.
For all x V , there exists an element y V , called the additive inverse of x, such
that x + y = 0. The additive inverse is denoted x.
Associativity of multiplication:1 ( x) = ( )x.
Distributivity:
(x + y) = x + y
and
( + )x = x + x.
1.2
P=
k x k :
k K, n N .
k=0
(x1 , x2 , . . .) : xk K,
|xk | p <
k=1
The definition of the multiplication by scalars and vector addition is the same
as in the previous example. Let us check that the sum x + y ` p (K) for any
x, y ` p (K). Indeed,
|xk + yk | p
(|xk | + |yk |) p
k=1
k=1
p
k=1
p
p
p
p
p
p
2
|x
|
+
|y
|
=
2
|x
|
+
2
k
k
k
|yk | p < .
k=1
k=1
k=1
7. The space C[0, 1] of all real-valued continuous functions on the closed interval
[0, 1] is a vector space. The addition and multiplication by scalars are defined
naturally: for f , g C[0, 1] and R we denote by f + g the function whose
values are given by
t [0, 1] ,
t [0, 1] ,
| f (t)| dt <
is a vector space.
If f C[0, 1] then f L 1 (0, 1). Indeed, since [0, 1] is compact f is bounded (and
attains its lower and upper bounds). Then
Z 1
so f L 1 (0, 1).
We conclude that C[0, 1] is a strict subset of L 1 (0, 1).
3
1.3
Hamel bases
Definition 1.1 The linear span of a subset E of a vector space V is the collection of
all finite linear combinations of elements of E:
(
)
n
Span(E) =
x V :x =
j e j , n N, j K, e j E
j=1
je j = 0
1 = 2 = = n = 0
j=1
x=
je j
j=1
where n N, j K, and e j E.
Exercise: Prove the lemma.
Definition 1.5 We say that a set is finite if it consists of a finite number of elements.
Theorem 1.6 If V has a finite Hamel basis then every Hamel basis for V has the same
number of elements.
k e0k .
e1 =
k=1
11 k e0k ,
1km
k6=k1
Let S1 = {e1 } and S10 = {e0k1 }. The set E10 = (E 0 \ S10 ) S1 is linearly independent and
Span(E10 ) = Span(E 0 ) = V (check these two claims).
We can repeat the procedure inductively. Let S j = {e1 , . . . , e j }. Suppose for some
j, 1 j n 1, there is a set S0j = {e0k1 , . . . , e0k j } such that the set
E 0j = (E 0 \ S0j ) S j
and Span(E 0j ) = V . Then there are k , k K such that
e j+1 =
k e0k +
e0k E 0 \S0j
k ek .
ek S j
Since S j+1 is linearly independent, there is k j+1 such that k j+1 6= 0. Let S0j+1 =
S0j {ek j }. Then E 0j+1 is linearly independent and spans V (by the same arguments as
in the case j = 1).
After n inductive steps we get that En0 is linearly independent, which is impossible
because Sn = E and consequently En0 = (E 0 \ Sn0 ) E. This contradiction implies that
m = n.
Definition 1.7 If V has a finite basis E then the dimension of V (denoted dimV ) is
the number of elements in E. If V has no finite basis then we say that V is infinitedimensional.
Example: In Rn any basis consists of n vectors. Therefore dim Rn = n.
Let V and W be two vector spaces over K.
Definition 1.8 A map L : V W is called linear if for any x, y V and any K
L(x + y) = L(x) + L(y) .
Definition 1.9 If a linear map L : V W is a bijection, then L is called a linear
isomorphism. We say that V and W are linearly isomorphic if there is a bijective
linear map L : V W .
5
x=
je j .
j=1
Remark: This linearly independent set E is not a Hamel basis. Indeed, the sequence x = (x1 , x2 , x3 , . . .) with xk = ek belongs to ` p (K) for any p 1 but
cannot be represented as a sum of finitely many elements of the set E.
2. C[0, 1] is infinite-dimensional.
Proof: The set E = { xk : k N } is an infinite linearly independent subset of
C0 [0, 1]. Indeed, suppose
n
p(x) =
k x k = 0
k=1
Differentiating the equality n times we get p(n) (x) = n!n = 0. Which implies
n = 0. Therefore p(x) 0 implies k = 0 for all k.
Note that
f (x) =
x( x), for 0 x
0,
for x 1
Normed spaces
2.1
Norms
Definition 2.1 A norm on a vector space V is a map k k : V R such that for any
x, y V and any K:
1. kxk 0, and kxk = 0 x = 0
2. kxk = || kxk
(positive definiteness);
(positive homogeneity);
3. kx + yk kxk + kyk
(triangle inequality).
(a) kxk =
|xk |2
k=1
n
(b) kxk p =
!1/p
|xk | p
,1 p<
k=1
!1/p
|xk | p
kxk` p =
k=1
2.2
1
p
+ 1q = 1, then
a p bq
+ .
p
q
a p bq
1
abq/p + 0 .
p
q
1
p
+ 1q = 1, x ` p (K), y `q (K),
|x j y j | kxk` p kyk`q .
j=1
Proof. If 1 < p, q < , we use Youngs inequality to get that for any n N
n |x | |y |
n
1 1
1 |x j | p 1 |y j |q
j
j
kxk` p kyk`q p kxk pp + q kykqq p + q = 1
j=1
j=1
`
`
Therefore for any n N
|x j y j | kxk` p kyk`q .
j=1
Since the partial sums are monotonically increasing and bounded above, the series
converge and Holders inequality follows by taking the limit as n .
If p = 1 and q = :
n
|x j y j | max |y j |
j=1
1 jn
|x j | kxk`1 kyk` .
j=1
Therefore the series converges and Holders inequality follows by taking the limit as
n .
Lemma 2.4 (Cauchy-Schwartz inequality) If x, y `2 (K) then
j=1
j=1
!1/2
|x j y j | |x j |2
9
|y j |2
j=1
!1/2
.
Now we state and prove the triangle inequality for the ` p norm.
Lemma 2.5 (Minkowskis inequality) If x, y ` p (K) for 1 p then x + y
` p (K) and
kx + yk` p kxk` p + kyk` p .
Proof: If 1 < p < , define q from the equation 1p + 1q = 1. Then using Holders
inequality (finite sequences belong to ` p with any p) we get3
n
|x j + y j | p =
j=1
|x j + y j | p1|x j + y j |
j=1
n
|x j + y j | p1|x j | + |x j + y j | p1|y j |
j=1
j=1
!1/q
|x j + y j |(p1)q
|x j | p
j=1
(Holders inequality)
j=1
!1/q
!1/p
|x j + y j |(p1)q
j=1
!1/p
|y j | p
j=1
1/q
Dividing the inequality by nj=1 |x j + y j | p
and using that (p 1)q = p and 1
1
q
|x j + y j | p
j=1
!1/p
!1/p
|x j | p
j=1
|y j | p
!1/p
.
j=1
The series on the right hand side converge to kxk` p + kyk` p . Consequently the series on
the left hand side also converge. Therefore x + y ` p (K), and Minkowskis inequality
follows by taking the limit as n .
Exercise: Prove Minkowskis inequality for p = 1 and p = .
2.3
Each of the following formulae defines a norm on C[0, 1], the space of all continuous
functions on [0, 1]:
3 We
do not start directly with n = because a priori we do not know convergence for some of the
series involved in the proof.
10
2. the L1 norm
k f kL1 =
Z 1
| f (t)| dt ;
3. the L2 norm
k f kL 2 =
Z
1/2
| f (t)| dt
Exercise: Check that each of these formulae defines a norm. For the case of the L2
norm, you will need a Cauchy-Schwartz inequality for integrals.
Example: Let k N. The space Ck [0, 1] consists of all continuous real-valued functions which have continuous derivatives up to order k. The norm on Ck [0, 1] is defined
by
k
k f kCk =
supt[0,1] | f ( j)(t)| ,
j=0
2.4
Equivalence of norms
We have seen that various different norms can be introduced on a vector space. In
order to compare two norms it is convenient to introduce the following equivalence
relation.
Definition 2.6 Two norms k k1 and k k2 on a vector space V are equivalent if there
are constants c1 , c2 > 0 such that
c1 kxk1 kxk2 c2 kxk1
for all x V .
already saw this statement in Analysis III and/or Differentiation in Year 2. The proof is based
on the observation that the unit sphere S Rn is sequentially compact. Then we checked that f (x) =
kxk2 /kxk1 is continuous on S and consequently it is bounded and attains its lower and upper bounds on
S. We set c1 = minS f and c2 = maxS f .
11
max |t|n = 1 ,
t[0,1]
Z 1
n
t dt =
1
.
n+1
Suppose the norms are equivalent. Then there is a constant c2 > 0 such that for all n:
k f n k
= n + 1 c2 .
k f n kL 1
But it is not possible for all n. This contradiction implies the norms are not equivalent.
2.5
Linear Isometries
Finally, the triangle inequality follows from the triangle inequality for k kV :
kx + ykW = kL(x) + L(y)kV kL(x)kV + kL(y)kV = kxkW + kykW .
Therefore, k kW is a norm.
Note that in the proposition the new norm is introduced in such a way that L :
(W, k kW ) (V, k kV ) is a linear isometry.
Let V be a finite dimensional vector space and n = dimV . We have seen that V is
linearly isomorphic to Kn . Then the proposition implies the following statements.
Corollary 2.11 Any finite dimensional vector space V can be equipped with a norm.
Corollary 2.12 Any n-dimensional normed space V is isometric to Kn equipped with
a suitable norm.
Since any two norms on Rn (and therefore on Cn ) are equivalent we also get the
following statement.
Theorem 2.13 If V is a finite-dimensional vector space, then all norms on V are equivalent.
13
3.1
The norm on a vector space V can be used to measure distances between points x, y V .
So we can define the limit of a sequence.
Definition 3.1 A sequence (xn )
n=1 , xn V , n N, converges to a limit x V if for any
> 0 there is N N such that
kxn xk <
1
0.
n+1
14
1
1
1
m/n .
2 2
4
Consequently ( fn ) is not Cauchy in the sup norm and hence not convergent.
This example shows that the convergence in the L1 norm does not imply the pointwise convergence and, as a results, does not imply the convergence in the sup norm
(often called the uniform convergence). Note that in contrast to the uniform and L1
convergences the notion of pointwise convergence is not based on a norm on the space
of continuous function.
Exercise: The pointwise convergence does not imply the L1 convergence.
Hint: Construct fn with support in (0, 1/n) but make the maximum of fn very large
to ensure that k fn kL1 > n. Therefore fn (t) 0 for every t [0, 1] but fn is not bounded
in the L1 norm, hence not convergent.
Proposition 3.4 If fn C[0, 1] for all n N and fn f in the sup norm, then fn f
in the L1 norm, i.e.,
k fn f k 0
k fn f kL1 0 .
Proof:
0 k fn f kL1 =
Z 1
0
Therefore k fn f kL1 0.
We have seen that different norms may lead to different conclusions about convergence of a given sequence but sometime convergence in one norm implies convergence
in another one. The following lemma shows that equivalent norms give rise to the same
notion of convergence.
Lemma 3.5 Suppose k k1 and k k2 are equivalent norms on a vector space V . Then
for any sequence (xn ):
kxn xk1 0
kxn xk2 0 .
Proof: Since the norms are equivalent, there are constant c1 , c2 > 0 such that
0 c1 kxn xk1 kxn xk2 c2 kxn xk1
for all n. Then kxn xk2 0 implies kxn xk1 0, and vice versa.
15
3.2
3.3
Closed sets
Span(E) = W ,
Example: The subspace of polynomial functions is linear but not closed in C[0, 1]
equipped with the sup norm.
17
3.4
Compactness
quence xn j x with x K.
Proposition 3.14 A compact set is closed and bounded.
Theorem 3.15 A subset of Rn is compact iff it is closed and bounded.
Corollary 3.16 A subset of a finite-dimensional vector space is compact iff it is closed
and bounded.
Example: The unit sphere in ` p (K) is closed, bounded but not compact.
Proof: Take the sequence e j such that
e j = (0, . . . , 0, |{z}
1 , 0, . . .) .
jth place
We note that ke j ek k` p = 21/p for all j 6= k. Consequently, (e j )j=1 does not have any
convergent subsequence, hence S is not compact.
Lemma 3.17 (Riesz Lemma) Let X be a normed vector space and Y be a closed
linear subspace of X such that Y 6= X and R, 0 < < 1. Then there is x X
such that kx k = 1 and kx yk > for all y Y .
Proof: Since Y X and Y 6= X there is x X \Y . Since Y is closed, X \Y is open and
therefore
d := inf{ kx yk : y Y } > 0 .
xz
Since 1 > 1 there is a point z Y such that kx zk < d 1 . Let x = kxzk
. Then
kx k = 1 and for any y Y ,
xz
x (z + kx zky)
d
kx yk =
>
=,
kx zk y
=
kx zk
d 1
Theorem 3.18 A normed space is finite dimensional iff the unit sphere is compact.
Proof: If the vector space is finite-dimensional, Corollary 3.16 imply the unit sphere
is compact as the unit sphere is bounded and closed.
So we only need to show that if the unit sphere S is sequentially compact, then the
normed space V is finite dimensional. Suppose that dimV = . Then Riesz Lemma
can be used to construct an infinite sequence of xn S such that kxn xm k > 21 > 0
18
for all m 6= n. This sequence does not have any convergent subsequence (none of its
subsequences is Cauchy) and therefore S is not compact.
We construct xn inductively. First take any x1 S. Then suppose that for some n 1
we have found x1 , . . . , xn S such that kxl xk k > 21 for all 1 k, l n, k 6= l (note
that this property is satisfied for n = 1). The linear subspace Yn = Span(x1 , . . . , xn ) is
n-dimensional and hence closed (see Proposition 3.12). Since X is infinite dimensional
Yn 6= X. Then Riesz Lemma with Y = Yn and = 21 implies that there is xn+1 S such
that kxn+1 xk k > 12 for all 1 k n.
Repeating this argument inductively we generate the sequence xn for all n N.
19
Banach spaces
4.1
Definition 4.1 (Banach space) A normed space V is called complete if any Cauchy
sequence in V converges to a limit in V . A complete normed space is called a Banach
space.
Theorem 3.3 implies that R is complete, i.e., every Cauchy sequence of numbers
has a limit. We also know that C is complete (Do you know how to deduce it from the
completeness of R?).
Theorem 4.2 Every finite-dimensional normed space is complete.
Proof: Now let V be a vector space over K {R, C}, dimV = n < . Take any
basis in V . Then a sequence of vectors in V converges iff each component of the
vectors converges, and a sequence of vectors is Cauchy iff each component is Cauchy.
Therefore each component has a limit, and those limits constitute the limit vector for
the original sequence. Hence V is complete.
In particular, Rn and Cn are complete.
Theorem 4.3 (` p is a Banach space) The space ` p (K) equipped with the standard ` p
norm is complete.
Proof: Suppose that xk = (x1k , x2k , . . .) ` p (K) is Cauchy. Then for every > 0 there is
N such that
kxm xn k`pp =
j=1
for all m, n > N. Consequently, for each j N the sequence xkj is Cauchy, and the
completeness of K implies that there is a j K such that
xkj a j
as k . Let a = (a1 , a2 , . . .). First we note that for any M 1 and m, n > N:
M
|xmj xnj | p
j=1
j=1
|xmj a j | p .
j=1
20
|xmj a j | p .
j=1
We conclude that xm a ` p (K). Since ` p (K) is a vector space and xm ` p (K), then
a ` p (K). Moreover, kxm ak` p < for all m > N. Consequently xm a in ` p (K)
with the standard norm, and so ` p (K) is complete.
Theorem 4.4 (C is a Banach space) The space C[0, 1] equipped with the sup norm is
complete.
Proof: Let fk be a Cauchy sequence. Then for any > 0 there is N such that
sup | fn (t) fm (t)| <
t[0,1]
for all m, n > N. In particular, fn (t) is Cauchy for any fixed t and consequently has a
limit. Set
f (t) = lim fn (t) .
n
Lets prove that fn (t) f (t) uniformly in t. Indeed, we already know that
| fn (t) fm (t)| <
for all n, m > N and all t [0, 1]. Taking the limit as m we get
| fn (t) f (t)|
for all n > N and all t [0, 1]. Therefore fn converges uniformly:
k fn f k = sup | fn (t) f (t)| < .
t[0,1]
for all n > N. The uniform limit of a sequence of continuous functions is continuous.
Consequently, f C[0, 1] which completes the proof of completeness.
Example: The space C[0, 2] equipped with the L1 norm is not complete.
Proof: Consider the following sequence of functions:
( n
t for 0 t 1,
fn (t) =
1 for 1 t 2 .
This is a Cauchy sequence in the L1 norm. Indeed for any n < m:
k f n f m kL 1 =
Z 1
(t n t m ) dt =
21
1
1
1
<
,
n+1 m+1 n+1
1
.
N
Now let us show that fn do not converge to a continuous function in the L1 norm.
Indeed, suppose such a limit exists and call it f . Then
k fn f kL1 =
Z 1
|t f (t)| dt +
Z 2
|1 f (t)| dt 0 .
Since
| f (t)| |t n | |t n f (t)| | f (t)| + |t n |
implies that
Z 1
0
| f (t)| dt
Z 1
t dt
Z 1
|t f (t)| dt
Z 1
| f (t)| dt +
Z 1
t n dt ,
R
R
we have 01 |t n f (t)| dt 01 | f (t)| dt as n and consequently
Z 1
Z 2
| f (t)| dt +
|1 f (t)| dt = 0 .
4.2
A normed space may be incomplete. However, every normed space X can be con The minimal among these spaces5 is
sidered as a subset of a larger Banach space X.
called the completion of X.
Informally we can say that X consists of limit points of all Cauchy sequences in X.
Of course, every point x X is a limit point of the constant sequence (xn = x for all
If X is not complete, some of the limit points are not in
n N) and therefore X X.
dense in X.
22
Note that X is dense in V iff for every point v V there is a sequence xn X such
that xn v.
Theorem 4.6 Let (X, k kX ) be a normed space. Then there is a complete normed
space (X , k kX ) and a linear map i : X X such that i is an isometrical isomorphism between (X, k kX ) and (i(X), k kX ), and i(X) is dense in X .
Moreover, X is unique up to isometry, i.e., if there is another complete normed
space (X , kkX ) with these properties, then X and X are isometrically isomorphic.
Proof: The proof is relatively long so we break it into a sequence of steps.
Construction of X . Let Y be the set of all Cauchy sequences in X. We say that two
Let X be the space of all equivalence classes in Y , i.e., it is the factor space: X =
Y / . The elements of X are collections of equivalent Cauchy sequences from X.
We will use [x] to denote the equivalence class of x.
Exercises: Show that X is a vector space.
Norm on X . For an X take any representative x = (xn )
n=1 , xn X, of the
equivalence class . Then the equation
kkX = lim kxn kX .
n
(4.1)
k N take a representative x(k) (k) . Note that x(k) Y is a Cauchy sequence in the
space (X, k kX ). Then there is a strictly monotone sequence of integers nk such that
1
(k)
(k)
for all j, l nk .
(4.2)
x j xl
k
X
23
k=1
Next we will check that x is Cauchy, and consider its equivalence class = [x ]
X . Then we will prove that (k) in (X , k kX ).
The sequence x is Cauchy. Since the sequence of (k) is Cauchy, for any > 0 there
is M such that
(k)
(l)
(l)
kxn xn kX <
Then fix any > 0. If j, l >
(4.3)
k,l
and m > max{n j , nl , N/3
} we have
( j)
(l)
( j)
( j)
(l)
(l)
(l)
(k)
(k)
( j)
= lim kx j xj kX = lim kx j xn j kX
j
j
(k)
(k)
(k)
( j)
lim kx j xnk kX + kxnk xn j kX
j
1
+ < 2
k
Therefore (k) .
We have proved that any Cauchy sequence in X has a limit in X , so X is complete.
Density of i(X) in X . Take an X and let x . Take any > 0. Since x is
Cauchy, there is N such that kxm xk kX < for all k, m > N . Then
k i(xk )kX = lim kxm xk kX .
m
4.3
In this section we will prove an approximation theorem which is independent from the
discussions of the previous lectures. This theorem implies that polynomials are dense
in the space of continuous functions on an interval.
Theorem 4.7 (Weierstrass Approximation Theorem) If f : [0, 1] R is continuous
on [0, 1] then the sequence of polynomials
n
n
Pn (x) =
f (p/n)x p (1 x)np
p
p=0
uniformly converges to f on [0, 1].
25
Proof: First we derive several useful identities. The binomial theorem states that
n
n p np
n
x y
.
(x + y) =
p=0 p
Differentiating with respect to x and multiplying by x we get
n
n p np
n1
nx(x + y)
= p
x y
.
p
p=0
Differentiating the original identity twice with respect to x and multiplying by x2 we
get
n
n p np
2
n2
n(n 1)x (x + y)
= p(p 1)
x y
.
p
p=0
Now substitute y = 1 x and denote
n p
r p (x) =
x (1 x)np .
p
We get
n
r p(x)
= 1,
pr p(x)
= nx .
p=0
n
p=0
n
= n(n 1)x2 .
p=0
Consequently,
n
n
2
(p nx) r p(x)
p=0
2 2
p=0
p=0
p=0
2 2
26
k f k
.
2 2
2k f k
(p nx)2
2 2 r p(x)
p=0 n
= 2k f k
x(1 x) k f k
n 2
2n 2
as n .
x[0,1]
Consider the space P[0, 1] of all polynomial functions restricted to the interval [0, 1]
and equip this space with the sup norm. This space is not complete (think about Taylor
series). On the other hand polynomials are continuous, and therefore P[0, 1] can be
considered as a subspace of C[0, 1] which is complete. The Weierstrass approximation
theorem states that any continuous function on [0, 1] can be uniformly approximated by
polynomials. In other words, the polynomials are dense in C[0, 1]. Then Theorem 4.6
implies that the completion of the polynomials P[0, 1] is isometric to C[0, 1] equipped
with the sup norm.
Corollary 4.8 The set of polynomials is dense in C[0, 1] equipped with the supremum
norm.
27
Lebesgue spaces
Lebesgue spaces play an important role in Functional Analysis and some of its applications. These spaces consist of integrable functions. In this section we discuss the main
definitions and the properties required later in this module. Most of the statements
are given without proofs. A more detailed study of these topics is a part of MA359
Measure Theory module.
5.1
Lebesgue measure
n=1 Sn
Sn
(Sn) .
n=1
n=1
B = [ai , bi ] ,
i=1
vol(B) = (bi ai ) .
i=1
28
For any subset A of Rn , we can define its outer measure (A) by:
(
(A) = inf
[a j , b j ]
and
|b j a j | < .
j=1
j=1
Corollary 5.4 The Lebesgue measure has the following property: any subset of a measure zero set is measurable and itself has measure zero.
Exercise: Show that a countable union of measure zero sets has measure zero. Hint:
for An choose a cover with n = /2n .
Examples. The set Q of all rational numbers has measure zero. The Cantor set has
measure zero.
Definition 5.5 A property is said to hold almost everywhere or for almost every
x (and abbreviated to a.e.) if the set of points at which the property does not hold
has measure zero.
29
5.2
Lebesgue integral
(x) =
c j S j (x) ,
j=1
:=
c j (S j ).
j=1
f=
f+
30
f1 +
f2 .
f 0.
lim
n 1
0, n
lim
= 1 6=
= 1 6=
n (0,n)
lim n
1
0, n
= 0.
1
lim (0,n) = 0 .
n n
The following two theorems establish conditions which allow swapping the limit
and integration. They play the fundamental role in the theory of Lebesgue integrals.
Theorem 5.9 (Monotone Convergence
Theorem) Suppose that fn are integrable funcR
tions, fn (x) fn+1 (x) a.e., and fn < K for some constant independent of n. Then
there is an integrable function g such that fn (x) g(x) a.e. and
Z
g = lim
fn .
31
Proof:
Let fn (x) = n| f (x)|. This sequence satisfies MCT (integrable, increasing and
R
fn = 0 < 1), consequently there is an integrable g(x) such that fn (x) g(x) for a.e.
x. Since the sequence is increasing, fn (x) g(x) a.e. which implies | f (x)| g(x)/n
for all n and a.e. x. Consequently f (x) = 0 a.e.
Theorem 5.11 (Dominated Convergence Theorem) Suppose that fn are integrable
functions and fn (x) f (x) for a.e. x.. If there is an integrable function g such that
| fn (x)| g(x) for every n and a.e. x, then f is integrable and
Z
f = lim
fn .
f :=
Re f + i
Im f .
The MCT has no meaning for complex valued functions. The DCT is valid without
modifications (and indeed follows easily from the real version).
5.3
Definition 5.12 The Lebesgue space L1 (R) is the space of Lebesgue integrable functions modulo the following equivalence relation: f g iff f (x) = g(x) a.e. The
Lebesgue space is equipped with the L1 norm:
k f kL1 =
| f |.
Note that the value of the integral in the right-hand-side is independent from the choice
of a representative of the equivalence class.
It is convenient to think about elements of L1 (R) as functions R R interpreting
the equality f = g as f (x) = g(x) a.e.
From the viewpoint of Functional Analysis, the equivalence relation is introduced
to ensure non-degeneracy of theRL1 norm. Indeed, suppose f is an integrable function.
Then k f k1 = 0 is equivalent to | f | = 0, which is equivalent to f (x) = 0 a.e.
Theorem 5.13 L1 (R) is a Banach space.
The properties of the Lebesgue integral imply that L1 (R) is a normed space. The
completeness of L1 (R) follows from the combination of the following two statements:
The first lemma gives a criterion for completeness of a normed space, and the second
one implies that the assumptions of the first lemma are satisfied for X = L1 (R).
32
ky j k <
j=1
for all k, l nk .
Let y1 = xn1 and yk = xnk xnk1 for k 1. Since kyk k 21k for k 2,
k=1
k=1
x =
yj .
j=1
j=1
j=2
Lemma 5.15 If ( fk )
k=1 is a sequence of integrable functions such that k=1 k f k kL1 <
, then
1.
k=1 | f k (x)| converges a.e. to an integrable function,
2.
k=1 f k (x) converges a.e. to an integrable function.
Proof: The first statement follows from MCT applied to the sequence gn = nk=1 | fk |
and K =
k=1 k f k kL1 . So there is an integrable function g(x) such that
g(x) =
| fk (x)|
k=1
for almost all x. For these values of x the partial sums hn (x) = nk=1 fk (x) obviously
converge, so let
h(x) =
fk (x).
k=1
33
Moreover
n
n
|hn (x)| = fk (x) | fk (x)| | fk (x)| = g(x) .
k=1
k=1
k=1
Therefore the partial sums hn satisfy DCT and the second statement follows.
Exercise: Check that Lemma 5.15 implies that L1 (R) satisfies the assumptions of
Lemma 5.14.
In addition to L1 (R) we will sometimes consider the Lebesgue spaces L1 (a, b)
where (a, b) is an interval.
Proposition 5.16 The space C[0, 1] is dense in L1 (0, 1).
About proof: The proof uses that simple functions (=piecewise constant functions)
are dense in L1 (0, 1). Then check that every step function can be approximated by a
piecewise linear continuous function.
Consequently L1 (a, b) is isometric to the completion of C[a, b] in the L1 norm.
5.4
L p spaces
Z
|f|
1/p
<
1
1 + |x|
f = so it is not integrable
Theorem 5.17 L p (R) and L p (I) are Banach spaces for any p 1 and any interval I.
We will not give a complete proof but sketch the main ideas instead.
Let 1p + q1 = 1 and f L p (R), g Lq (R). Then the Holder inequality states7 that
Z
| f g| k f k p kgkq .
Note that the characteristic function I Lq (R) for any interval I and any q 1, more1
over kI kLq = |I| q where |I| = b a is the length of I. The Holder inequality with
g = I implies that
Z
Z
I | f | = | f | |I|1/q k f k p .
I
k f k kL 2 < .
k=1
gn =
!2
| fk |
k=1
proof of this inequality is similar to the proof of Lemma 2.3 provided we take for granted that a
product of two measurable functions is measurable. We will not discuss this proof further.
35
Hilbert spaces
6.1
(x, y) =
xk yk .
k=1
(x, y) =
xk yk .
k=1
(x, y) =
xk yk .
k=1
Note that the sum converges because k |xk yk | 12 k |xk |2 + |yk |2 .
4. Example: L2 (a, b) is an inner product space
Z b
f (x)g(x) dx .
( f , g) =
a
36
6.2
Natural norms
p
(x, x) is the natural norm induced by the inner
for all x, y V .
We have already proved the Cauchy-Schwartz inequality for `2 (K) using a different
strategy (see Lemma 2.4).
The Cauchy-Schwartz inequality in L2 (a, b) takes the form
Z b
Z b
1/2 Z b
1/2
2
2
|g(x)| dx
.
f (x)g(x) dx
| f (x)| dx
a
37
Lemma 6.5 If V is an inner product space equipped with the natural norm, then xn
x and yn y imply that
(xn , yn ) (x, y) .
Proof: Since any convergent sequence is bounded, the inequality
|(xn , yn ) (x, y)| = |(xn x, yn ) + (x, yn y)|
|(xn x, yn )| + |(x, yn y)|
kxn xk kyn k + kxk kyn yk
implies that (xn , yn ) (x, y).
The lemma implies that we can swap inner products and limits.
6.3
Example (some norms are not induced by an inner product): There is no inner
product which could induce the following norms on C[0, 1]:
k f k = sup | f (t)|
or
t[0,1]
k f kL1 =
Z 1
| f (t)| dt .
Indeed, these norms do not satisfy the parallelogram law, e.g., take f (x) = x and g(x) =
1 x, obviously f , g C[0, 1] and
k f k = kgk = k f gk = k f + gk = 1 ,
substituting these numbers into the parallelogram law we see 2 6= 4.
Exercise: Is the parallelogram law for the L1 norm satisfied for these f , g?
38
Lemma 6.7 (Polarisation identity) Let V be an inner product space with the natural
norm k k. Then
1. If V is real
4(x, y) = kx + yk2 kx yk2 ;
2. If V is complex
4(x, y) = kx + yk2 kx yk2 + ikx + iyk2 ikx iyk2 .
Proof: Plug in the definition of the natural norm into the right hand side and use
linearity of the inner product.
Lemma 6.7 shows that the inner product can be restored from its natural norm. Although the right hand sides of the polarisation identities are meaningful for any norm,
we should not rush to the conclusion that any normed space is automatically an inner
product space. Indeed, the example above implies that for some norms these formulae
cannot define an inner product. Nevertheless, if the norm satisfy the parallelogram law,
we indeed get an inner product:
Proposition 6.8 Let V be a real normed space with the norm k k satisfying the parallelogram law, then
(x, y) =
2
2
1
1
2 kx + 2y + zk 2 kx 2y + zk
2
2
2
1
2 (2kx + y + zk + 2kyk kx + zk )
21 (2kx y + zk2 + 2kyk2 kx + zk2 )
2
2
= kx + y + zk kx y + zk = 4(x + z, y) .
We have proved that
(x, y) + (z, y) = (x + z, y) .
Applying this identity several times and setting z = x/m we obtain
n(x/m, y) = (nx/m, y)
8 Can
and
39
m(x/m, y) = (x, y)
n
m
( x, y) = (x, y) .
We note that the right hand side of the definition involves the norms only, which commute with the limits. Any real number is a limit of rational numbers and therefore the
linearity holds for all R.
6.4
Definition 6.9 A Hilbert space is a complete inner product space (equipped with the
natural norm).
Of course, any Hilbert space is a Banach space.
1. Example: Rn is a Hilbert space
n
(x, y) =
!1/2
kxk =
xk yk ,
k=1
xk2
k=1
!1/2
kxk =
xk yk ,
|xk |2
k=1
k=1
(x, y) =
!1/2
kxk =
xk yk ,
k=1
|xk |2
k=1
( f , g) =
f (x)g(x) dx,
a
kxk =
Z
a
40
| f (x)| dx
1/2
.
The goal of this section is to discuss properties of orthonormal bases in a Hilbert space
H. Unlike Hamel bases, the orthonormal ones involve a countable number of elements:
i.e. a vector x is represented in the form of an infinite sum
x=
k ek
k=1
for some k K.
We will mainly consider complex spaces with K = C. The real case K = R is not
very different. We will use (, ) to denote an inner product on H, and k k will stand
for the natural norm induced by the inner product.
7.1
Orthonormal sets
Definition 7.1 Two vectors x, y H are called orthogonal if (x, y) = 0. Then we write
x y.
Theorem 7.2 (Pythagoras theorem) If x y then kx + yk2 = kxk2 + kyk2 .
Proof: Since (x, y) = 0
kx + yk2 = (x + y, x + y) = (x, x) + (x, y) + (y, x) + (y, y) = kxk2 + kyk2 .
Definition 7.3 A set E is orthonormal if kek = 1 for all e E and (e1 , e2 ) = 0 for all
e1 , e2 E such that e1 6= e2 .
Note that this definition does not require the set E to be countable.
Exercise: Any orthonormal set is linearly independent.
Indeed, suppose nk=1 k ek = 0 with ek E and k K. Multiplying this equality
by e j we get
!
n
0=
k ek , e j
k=1
k (ek , e j ) = j .
k=1
41
eikx
fk = : k Z
2
1
2
for all x:
| fk (x)|2 dx = 1 ,
and if j 6= k
Z
( fk , f j ) =
x=
Z
1 i(k j)x
ei(k j)x
fk (x) f j (x) dx =
e
dx =
i(k j)
= 0.
x=
Lemma 7.5 If {e1 , . . . , en } is an orthonormal set in an inner product space V , then for
any j K
2
n
n
j e j
= | j |2 .
j=1
j=1
Proof: The following computation is straightforward:
2
n
je j
=
j=1
=
j e j , l el
j=1
n n
j l (e j , el )
j=1 l=1
l=1
n
j l jl =
j=1 l=1
7.2
j j .
j=1
Gram-Schmidt orthonormalisation
for all k.
9 Remember that for any x R and any k Z: eikx = cos kx + i sin kx. Then |eikx | = 1 and eik =
cos k i sin k = (1)k .
42
Proof: Let e1 =
v1
kv1 k .
Then
Span{ v1 } = Span{ e1 }
ek = vk (vk , e j )e j .
j=1
k1
j=1 (vk , e j )e j
which implies that ek e j . Finally let ek = ek /kek k. Then { e1 , . . . , ek } is an orthonormal set such that
Span{ e1 , . . . , ek } = Span{ v1 , . . . , vk } .
If the original sequence is finite, the orthonormalisation procedure will stop after a
finite number of steps. Otherwise, we get an infinite sequence of ek .
Corollary 7.7 Any infinite-dimensional inner product space contains a countable orthonormal sequence.
Corollary 7.8 Any finite-dimensional inner product space has an orthonormal basis.
Proposition 7.9 Any finite dimensional inner product space is isometric to Cn (or Rn
if the space is real) equipped with the standard inner product.
Proof: Let n = dimV and e j , j = 1, . . . , n be an orthonormal basis in V . Note that
(ek , e j ) = k j . Any two vectors x, y V can be written as
n
x=
y=
and
xk ek
Then
n
(x, y) =
xk ek , y j e j
k=1
j=1
y je j .
j=1
k=1
xk y j (ek , e j ) =
k=1 j=1
xk yk .
k=1
43
7.3
Bessels inequality
k=1
n
k=1
Corollary 7.11 If E is an orthonormal set in an inner product space V , then for any
x V the set
Ex = { e E : (x, e) 6= 0 }
is at most countable.
Proof: For any m N the set Em = { e : |(x, e)| > m1 } has a finite number of elements.
Otherwise there would be an infinite sequence (ek )
k=1 with ek Em , then the series
2
k=1 |(x, ek )| = + which contradicts to Bessels inequality. Therefore Ex =
m=1 Em
is a countable union of finite sets and hence at most countable.
7.4
Convergence
In this section we will discuss convergence of series which involve elements from an
orthonormal set.
Lemma 7.12 Let H be a Hilbert space and E = (ek )
k=1 an orthonormal sequence.
2 < +. Then
The series
e
converges
iff
|
|
k=1 k
k=1 k k
2
(7.1)
k ek
= |k |2 .
k=1
k=1
44
Proof: Let xn = nk=1 k ek and n = nk=1 |k |2 . Lemma 7.5 implies that kxn k2 = n
and that for any n > m
2
n
n
kxn xm k2 =
k ek
= |k |2 = n m .
k=m+1
k=m+1
Consequently, xn is a Cauchy sequence in H iff n is Cauchy in R. Since both spaces
are complete, the sequences converge or diverge simultaneously.
If they converge, we take the limit as n in the equality kxn k2 = n to get (7.1)
(the limit commutes with k k2 ).
Definition 7.13 A series
n=1 xn in a Banach space X is unconditionally convergent
if for every permutation : N N the series
n=1 x (n) converges.
In Rn a series is unconditionally convergent if and only if it is absolutely convergent. Every absolutely convergent series is unconditionally convergent, but the
converse implication does not hold in general.
Example: Let (ek ) be an orthonormal sequence. Then
k ek
k=1
(x, ek )ek
k=1
converges unconditionally.
Lemma 7.15 Let H be a Hilbert space, E = (ek )
k=1 an orthonormal sequence and
x H. If x = k=1 k ek , then
for all k N.
k = (x, ek )
Proof: Exercise.
11 Prove
it
45
7.5
Definition 7.16 A set E is a basis for H if every x H can be written uniquely in the
form
x=
k ek
k=1
k=1 k ek
k=1 k ek . Then
j = (x, e j ) = ( k ek , e j ) =
k=1
k (ek , e j ) = j ,
k=1
(c) = (d): Let (x, ek ) = 0 for all k, then (c) implies that kxk = 0 hence x = 0.
(d) = (b): let y = x
k=1 (x, ek )ek . Corollary 7.14 implies that the series converges. Then Lemma 6.5 implies we can swap the limit and the inner product to get
for every n
!
(y, en ) =
x (x, ek )ek , en
k=1
Since (y, en ) = 0 for all n, then (d) implies that y = 0 which is equivalent to x =
Therefore x = 0.
(a) = (e): Since E is a basis any x = limn xn with xn = nk=1 k ek Span(E).
Example: The orthonormal sets from examples of Section 7.1 are also examples of
orthonormal bases.
7.6
xk ek .
k=1
48
12 How do we define the sum over an uncountable set? For any n N the set J = { j J : | f ( j)| >
n
} is finite (otherwise the sum is obviously infinite). Consequently, the set J ( f ) := { j J : | f ( j)| >
0 } is countable because it is a countable union of finite sets: J ( f ) =
n=1 Jn . Therefore, the number
of non-zero terms in the sum is countable and the usual definition of an infinite sum can be used.
1
n
49
8
8.1
(8.1)
Since d is the infinum, for any n there is an A such that kx an k2 < d 2 + n1 . Then
equation (8.1) implies that
2
2
2 2
kan am k 2d 2 + + 2d 2 + 4d 2 = + .
n
m
n m
Consequently (an ) is Cauchy and, since H is complete, it converges to some a . Since
A is closed, a A. Then
kx a k2 = lim kx an k2 = d 2 .
n
Therefore a is the point closest to x. Now suppose that there is another point a A
such that kx ak
= d, then (8.1) implies
ka ak
2kx a k2 + 2kx ak
2 4d 2 = 2d 2 + 2d 2 4d 2 = 0 .
So a = a and a is unique.
50
8.2
Orthogonal complements
Exercises:
1. If E is a basis in H, then E = { 0 }.
2. If Y X, then X Y .
3. X (X )
4. If X is a closed linear subspace in H, then X = (X )
Definition 8.5 The closed linear span of E H is a minimal closed set which contains
Span(E):
Span(E) = { u H : > 0 x Span(E) such that kx uk < } .
Proposition 8.6 If E H then E = (Span(E)) = (Span(E)) .
Proof: Since E Span(E) Span(E) we have (Span(E)) (Span(E)) E . So
we need to prove the inverse inclusion. Take u E and x Span(E). Then there is
xn Span(E) such that xn x. Then
(x, u) = ( lim xn , u) = lim (xn , u) = 0 .
n
and
for all x H .
Let v = x u. Let us show that v U . Indeed, take any y U and consider the
function : C R defined by
(t) = kv + tyk2 = kx (u ty)k2 .
Since the definition of u together with u ty U imply that (t) (0) = kx uk2 ,
the function has a minimum at t = 0. On the other hand
(t) = kv + tyk2 = (v + ty, v + ty)
= (v, v) + t(y, v) + t(v, y) + |t|2 (y, y) .
First suppose that t is real. Then t = t and
d
dt (0) = 0
implies
(y, v) + (v, y) = 0 .
Then suppose that t is purely imaginary, Then t = t and
d
dt (0) = 0
implies
(y, v) (v, y) = 0 .
Taking the sum of these two equalities we conclude
(y, v) = 0
for every y U.
Therefore v U .
In order to prove the uniqueness of the representation suppose x = u1 + v1 = u + v
with u1 , u U and v1 , v U . Then u1 u = v v1 . Since u u1 U and v v1 U ,
kv v1 k2 = (v v1 , v v1 ) = (v v1 , u1 u) = 0 .
Therefore u and v are unique.
Finally x = u + v with u v implies kxk2 = kuk2 + kvk2 . Consequently kPU (x)k =
kuk kxk. We also note that PU (u) = u for any u U. So PU2 (x) = PU (x) as PU (x) U
Corollary 8.9 If U is a closed linear subspace in a Hilbert space H and x H, then
PU (x) is the closest point to x in U.
52
8.3
Best approximations
(x, e j )e j .
jJ
(x, e j )e j .
k=1
Now suppose that the set E is not orthonormal. If the set E is finite or countable we
can use the Gram-Schmidt orthonormalisation procedure to construct an orthonormal
basis in Span(E). After that the theorem above gives us an explicit expression for the
best approximation. Lets consider some examples.
Example: Find the best approximation of a function f L2 (1, 1) with polynomials
of degree up to n. In other words, let E = { 1, x, x2 , . . . , xn }. We need to find u
Span(E) such that
k f ukL2 = inf k f pkL2 .
pSpan(E)
The set E is not orthonormal. Lets apply the Gram-Schmidt orthonormalisation procedure to construct an orthonormal basis in Span(E). For the sake of shortness, lets
write k k = k kL2 (1,1) .
53
2 and let
1
e1 = .
2
Then (1, x) =
R1
1 x dx
= 0 and kxk2 =
R1
2
1 |x| dx
r
e2 =
2
3
so let
3
x.
2
Then
e3 = x2 (x2 , e2 )e2 (x2 , e1 )e1
r Z
r
Z 1
1
3
3
1
1
2
2
= x
x
t
t dt
t 2 dt
2 1
2
2 1
2
Z 1
1
1
= x2
t 2 dt = x2 .
2 1
3
Taking into account that
2
ke3 k =
Z 1
1
1
x
3
2
we obtain
e3
=
e3 =
ke3 k
Exercise: Show that e4 =
7
3
8 (5x 3x)
2
dt =
8
45
5
(3x2 1) .
8
is orthogonal to e1 , e2 and e3 .
Z 1
f (t)(3t 2 1)dt
For example, if f (x) = |x| its best approximation by a third degree polynomial is
p3 =
15x2 + 3
.
16
3
.
16
Note that the best approximation in the L2 norm is not necessarily the best approximation in the sup norm. Indeed, for example,
2 + 3
15x
> 3
sup |x|
16 16
x[1,1]
(the supremum is larger than the values at x = 0). At the same time
1 1
2
sup |x| x +
= .
8 8
x[1,1]
55
A linear map on a vector space is traditionally called a linear operator. All linear
functions defined on a finite-dimensional space are continuous. This statement is no
longer true in the case of an infinite dimensional space.
We will begin our study with continuous operators: this class has a rich theory and
numerous applications. We will only slightly touch some of them (the most remarkable
examples will be the shift operators on `2 , and integral operators and multiplication
operators on L2 ).
Of course many interesting linear maps are not continuous. For example, consider the differential operator A : f 7 f 0 on the space of continuously differentiable
functions. More accurately, let D(A) = C1 [0, 1] L2 (0, 1) be the domain of A. Obviously, A : D(A) L2 (0, 1) is linear but not continuous. Indeed, consider the sequence
xn (t) = n1 sin(nt). Obviously kxn kL2 n1 so xn 0, but A(xn ) = cos(nt) does not
converge to A(0) = 0 in the L2 norm so A is not continuous.
Some definitions and properties from the theory of continuous linear operators can
be literally extended onto unbounded ones, but sometimes subtle differences appear:
e.g., we will see that a bounded operator is self-adjoint iff it is symmetric, which is
no longer true for unbounded operators. In a study of unbounded operators a special
attention should be paid to their domains.
9.1
A(u) =
A
u .
2kuk
Since
2kuk u
= 2 < we get kA(u)k 2kuk
and consequently A is bounded.
The space of all bounded linear operators from U to V is denoted by B(U,V ).
Definition 9.4 The operator norm of A : U V is
kA(x)kV
.
x6=0 kxkU
kAkB(U,V ) = sup
We note that kAkB(U,V ) is the smallest M such that (9.1) holds: indeed, it is easy to see
that the definition of operator norm implies
kA(x)kV kAkB(U,V ) kxkU
and (9.1) holds with M = kAkB(U,V ) . On the other hand, (9.1) implies M
any x 6= 0 and consequently M kAkB(U,V ) .
kAxkV
kxkU
for
Theorem 9.5 Let U be a normed space and V be a Banach space. Then B(U,V ) is a
Banach space.
Proof: Let (An )
n=1 be a Cauchy sequence in B(U,V ). Take a vector u U. The
sequence vn = An (u) is a Cauchy sequence in V :
kvn vm k = kAn (u) Am (u)k = k(An Am )(u)k kAn Am kop kuk.
Since V is complete there is v V such that vn v. Let A(u) = v.
57
The operator A is bounded. Indeed, An is Cauchy and hence bounded: there is constant
M R such that kAn kop < M for all n. Taking the limit in the inequality kAn uk Mkuk
implies kAuk Mkuk. Therefore A B(U,V ).
Finally, An A in the operator norm. Indeed, Since An is Cauchy, for any > 0
there is N such that kAn Am kop < or
kAn (u) Am (u)k kuk
9.2
Examples
Tl (x) = (x2 , x3 , x4 , . . .) .
and
|xk |2 = kxk`2 .
k=1
|xk |2 kxk`2 .
k=2
Z b
=
a
| f (t)x(t)| dt
k f k2
58
Z b
a
|x(t)| dt = k f k2 kxkL2 .
Z t0 +
| f (t)|2 dt | f (t)|2
as 0.
t0
Therefore kAkop = k f k .
3. Example: Integral operator on L2 (a, b):
Z b
(Ax)(t) =
K(t, s)x(s) ds
where
Z bZ b
a
|K(s,t)| ds dt < + .
|K(t, s)| ds
|x(s)| ds dt
a
a
Z bZ b
(Cauchy-Schwartz)
=
a
Consequently
kAk2op
Z bZ b
a
Note that this example requires a bit more from the theory of Lebesgue integrals
than we discussed in Section 5. If you are not taking Measure Theory and feel
uncomfortable with these integrals, you may assume that x, y and K are continuous functions.
9.3
59
We note that 0 Ker A for any linear operator A. We say that Ker A is trivial if
Ker A = { 0 }.
Proposition 9.7 If A B(U,V ) then Ker A is a closed linear subspace of U.
Proof: If x, y Ker A and , K, then
A(x + y) = A(x) + A(y) = 0 .
Consequently x + y Ker A and it is a linear subspace. Furthermore if xn x and
A(xn ) = 0 for all n, then A(x) = 0 due to continuity of A.
Note that the range is a linear subspace but not necessarily closed. Exercise: construct an example (see Examples sheet 3).
60
10
10.1
Linear functionals
Definition and examples
Rb
a
(t)x(t) dt is a
10.2
f (x) = (x, y)
Moreover, k f kH = kykH .
Since K is closed, Theorem 8.7 implies that every vector x H can be written
uniquely in the form
x = u+v
where u K and v K .
x H .
If there is another y0 H such that f (x) = (x, y0 ) for all x H, then (x, y) = (x, y0 ) for
all x, i.e., (x, y y0 ) = 0. Setting x = y y0 we conclude ky y0 k2 = 0, i.e. y = y0 is
unique.
Finally, the Cauchy-Schwartz inequality implies
| f (x)| = |(x, y)| kxk kyk,
i.e., k f kH = k f kop kyk. On the other hand,
k f kop
Consequently, k f kH = kykH .
62
11
11.1
In the next lectures we will discuss the spectral theory of linear operators. The spectral
theory looks more natural in complex spaces. In particular, the theory studies eigenvalues and eigenvectors of linear maps (i.e. non-zero solutions of the equation Ax = x).
In the finite-dimensional space a linear operator can be describe by a matrix. You already know that a matrix (even a real one) can have complex eigenvalues. Fortunately
a real Hilbert space can always be considered as a part of a complex one due to the
complexification procedure.
Definition 11.1 Let H be a real Hilbert space. The complexification of H is the complex vector space
HC = { x + iy : x, y H }
where the addition and multiplication are respectively defined by
(x + iy) + (u + iw) = (x + u) + i(y + w)
( + i )(x + iy) = (x y) + i(y + x) .
The inner product is defined by
(x + iy, u + iw) = (x, u) i(x, w) + i(y, u) + (y, w) .
Exercise: Show that HC is a Hilbert space.
Example: The complexification of `2 (R) is `2 (C).
Exercise: Show that kx + iyk2HC = kxk2 + kyk2 for all x, y H.
The following lemma states that any bounded operator on H can be extended to a
bounded operator on HC .
Lemma 11.2 Let H be a real Hilbert space and A : H H be a bounded operator.
Then
AC (x + iy) = A(x) + iA(y)
is a bounded operator HC HC .
Exercise: Prove the lemma.
63
11.2
Adjoint operators
for all x, y H .
for all x, y H.
(Ax)(t) =
K(t, s)x(s)ds
0
(A y)(s) =
K(t, s)y(t)dt
0
(Ax, y) =
K(t, s)x(s) ds y(t)
dt
Z 1Z 1
=
0
Z 1
K(t, s)x(s)y(t)
ds dt
!
Z
K(t, s)y(t) dt
x(s)
ds
= (x, A y) .
Note that we used Fubinis Theorem to change the order of integration.
3. Example: Shift operators: Tl = Tr and Tr = Tl . Indeed,
(Tr x, y) =
xk yk+1 = (x, Tl y) .
k=1
11.3
Self-adjoint operators
A f (t) =
K(t, s) f (s) ds
0
Exercise: Let A be a self-adjoint operator on a real Hilbert space H. Show that its
complexification AC : HC HC is also self-adjoint. Show that if is an eigenvalue of
AC , then there is x H, x 6= 0, such that Ax = x.
Theorem 11.9 If A is a bounded self-adjoint operator then
1. (Ax, x) is real for all x H
2. kAkop = supkxk=1 |(Ax, x)|
66
kuk
Au
kAuk
67
x, y D(A).
If A is symmetric then D(A) D(A ) and A coincides with the restriction of A onto
D(A). An operator is self adjoint if A = A , i.e., it is symmetric and D(A) = D(A ).
In general, the condition for a linear operator on a Hilbert space to be self-adjoint is
stronger than to be symmetric. If an operator is bounded then it is normally assumed
that D(A) = D(A ) = H and therefore a symmetric operator is self-adjoint.
The Hellinger-Toeplitz theorem states that an everywhere defined symmetric operator on a Hilbert space is bounded.
13 Optional
topic
68
12
12.1
= | | .
kxk
y6=0 kyk
kAkop = sup
Examples:
1. A linear map on an n-dimensional vector space has at least one and at most n
different eigenvalues.
2. The right shift Tr : `2 `2 has no eigenvalues, i.e., the point spectrum is empty.
Indeed, suppose Tr x = x, then
(0, x1 , x2 , x3 , x4 , . . .) = (x1 , x2 , x3 , x4 , . . . )
implies 0 = x1 , x1 = x2 , x2 = x3 , . . . If 6= 0, we divide by and conclude
x1 = x2 = = 0. If = 0 we also get x = 0. Consequently
p (Tr ) = 0.
/
3. The point spectrum of the left shift Tl : `2 `2 is the open unit disk. Indeed,
suppose Tl x = x with C. Then
(x2 , x3 , x4 , . . .) = (x1 , x2 , x3 , x4 , . . . )
is equivalent to x2 = x1 , x3 = x2 , x4 = x3 , . . . Consequently, x = (xk )
k=1 with
2
xk = k1 x1 for all k 2. This sequence belongs to `2 if and only if
k=1 |xk | =
2k
12.2
Invertible operators
and
Range(A) = V.
and
A1 A = IU .
and
BA = IU
12.3
Technical lemmas
The following two lemmas will help us in the study of the resolvent set: they establish
useful conditions which guarantee that an operator has a bounded inverse.
Lemma 12.8 If T B(H, H) and kT k < 1, then (I T )1 B(H, H). Moreover
(I T )1 = I + T + T 2 + T 3 + . . .
and
k(I T )1 k (1 kT k)1 .
Proof: Consider the sequence Vn = I + T + T 2 + + T n . Since
kT n xk kT k kT n1 xk
71
.
1 kT k
1 kT k
Since kT k < 1, Vn is a Cauchy sequence in the operator norm. The space B(H, H) is
complete and there is V B(H, H) such that Vn V . Moreover,
kV k 1 + kT k + kT k2 + = (1 kT k)1 .
Finally, taking the limit as n in the equalities
Vn (I T ) = Vn Vn T = I T n+1 ,
(I T )Vn = Vn TVn = I T n+1
and using that T n+1 0 in the operator norm we get V (I T ) = (I T )V = I.
Lemma 12.6 implies (I T )1 = V .
Lemma 12.9 Let H be a Hilbert space and T, T 1 B(H, H). If U B(H, H) and
kUk kT 1 k < 1, then the operator T +U is invertible and
(T +U)1
kT 1 k
.
1 kUk kT 1 k
72
(A I) = I = (A I) (A I)1
Consequently, (A I) has a bounded inverse (A I)1 (an adjoint of a bounded
operator). Therefore R(A) iff R(A ). Since the spectrum is the complement
of the resolvent set we also get (A) iff (A ).
Proposition 12.11 If A is bounded and (A) then | | kAkop .
Proof: Take C such that | | > kAkop . Since k 1 Akop < 1 Lemma 12.8 implies
that I 1 A is invertible and the inverse operator is bounded. Consequently, A I =
(I 1 A) also has a bounded inverse and so R(A). The proposition follows
immediately since (A) is the complement of R(A).
Proposition 12.12 If A is bounded then R(A) is open and (A) is closed.
Proof: Let R(A). Then T = (A I) has a bounded inverse. Set U = I.
Obviously, kUk = | |. Let
| | < kT 1 k1 ,
then Lemma 12.9 implies that T + U = A ( + )I also has a bounded inverse. So
+ R(A). Consequently R(A) is open and (A) = C \ R(A) is closed.
Example: The spectrum of Tl and of Tr are both equal to the closed unit disk on the
complex plane.
Indeed, p (Tl ) = { C : | | < 1 }. Since p (Tl ) (Tl ) and (Tl ) is closed, we
conclude that (Tl ) includes the closed unit disk. On the other hand, Proposition 12.11
implies that (Tl ) is a subset of the closed disk | | kTl kop = 1. Therefore
(Tl ) = { C : | | 1 } .
Since Tr = Tl and (Tl ) is symmetric with respect to the real axis, Lemma 12.10
implies (Tr ) = (Tl ).
73
13
13.1
Compact operators
Definition, properties and examples
Definition 13.1 Let X be a normed space and Y be a Banach space. Then a linear
operator A : X Y is compact if the image of any bounded sequence has a convergent
subsequence.
Obviously a compact operator is bounded. Indeed, otherwise there is a sequence
xn with kxn k = 1 such that kAxn k > n for each n. The sequence Axn does not contain a
convergent subsequence (it does not even contain a bounded subsequence).
Example: Any bounded operator with finite-dimensional range is compact. Indeed, in
a finite dimensional space any bounded sequence has a convergent subsequence.
Proposition 13.2 Let X be a normed space and Y be Banach. A linear operator A :
X Y is compact iff the image of the unit sphere is sequentially compact.
Theorem 13.3 If X is a normed space and Y is a Banach space, then compact linear
operators form a closed linear subspace in B(X,Y ).
Proof: If K1 , K2 are compact operators and 1 , 2 K, then 1 K1 + 2 K2 is also compact. Indeed, take any bounded sequence (xn ) in H. There is a subsequence xn1 j
such that K1 xn1 j converges. This subsequence is also bounded, so it contains a subsequence xn2 j such that K2 xn2 j converges. Obviously K1 xn2 j also converges and therefore
1 K1 xn2 j + 2 K2 xn2 j is convergent and consequently 1 K1 + 2 K2 is compact. Therefore the compact operators form a linear subspace.
Let us prove that this subspace is closed. Let Kn be a convergent sequence of
compact operators: Kn K in B(H, H). Take any bounded sequence (xn ) in X. Since
K1 is compact, there is a subsequence xn1 j such that K1 xn1 j converges. Since xn1 j is
bounded and K2 is compact, there is a subsequence xn2 j such that K2 xn2 j converges.
Repeat this inductively: for each k there is a subsequence xnk j of the original sequence
such that Kl xnk j converges as j for all l k.
Consider the diagonal sequence y j = xn j j . Obviously (y j )j=k is a subsequence of
(xnk j )j=1 . Consequently Kl y j converges as j for every l.
In order to show that K is compact it is sufficient to prove that Ky j is Cauchy:
kKy j Kyl k kKy j Kn y j k + kKn y j Kn yl k + kKyl Kn yl k
kK Kn k ky j k + kyl k + kKn y j Kn yl k .
Given > 0 choose n sufficiently large to ensure that the first term is less than 2 ,
then choose N sufficiently large to guarantee that the second term is less than 2 for
all j, l > N. So Ky j is Cauchy and consequently converges. Therefore K is a compact
operator, and the subspace formed by compact operators is closed.
74
(A f )(t) =
Z bZ b
K(t, s) f (s) ds
with
is compact.
Proposition 13.5 If X is a normed space and Y is a Banach space, then the operators
of finite range a dense among compact operators in B(X,Y ).
13.2
Consequently, (T xn ) does not have a convergent subsequence (none of the subsequences is Cauchy). This contradicts to the compactness of T . Consequently, p (T )
is either finite or a converging to zero sequence.
Now let 6= 0 be an eigenvalue and E the corresponding eigenspace. Let A :
= x for any x E , the operator
E E be the restriction of A onto E . Since Ax
A maps the unit sphere into the sphere of radius . Since A is compact, the image of
the unit sphere is sequentially compact. Therefore the sphere of radius is compact.
Since E is a Hilbert (and consequently Banach) space itself, Theorem 3.18 implies
that E is finite dimensional.
Theorem 13.8 (Hilbert-Schmidt theorem) Let H be a Hilbert space and T : H H
be a compact self-adjoint operator. Then there is a finite or countable orthonormal
sequence (en ) of eigenvectors of T with corresponding real eigenvalues (n ) such that
T x = j (x, e j )e j
for all x H.
y = x (x, e j )e j .
j=1
n1
j=1
Tx =
(x, e j ) j e j
j=1
yn = x (x, e j )e j .
j=1
Tx =
(x, e j ) j e j .
j=1
Tx =
j (x, e j )e j
for all x H.
j=1
Exercise: Deduce that operators with finite range are dense among compact selfadjoint operators.
Theorem 13.10 If H is an infinite dimensional Hilbert space and T : H H is a
compact self-adjoint operator, then (T ) = p (T ).
Proposition 13.7 states that p (T ) is either a finite or countably infinite set. Moreover, if p (T ) is not finite, zero is the unique limit point of p (T ). If p (T ) is finite, then the Hilbert-Schmidt theorem implies that the kernel of T is not trivial (since
Ker T = { en } ) and consequently 0 p (T ). Therefore Theorem 13.10 means that
(T ) = p (T ) { 0 }. In particular, (T ) = p (T ) if zero is an eigenvalue.
Proof: We will prove the theorem assuming that H is separable.14
14 The proof uses a countable basis in H. The theorem remains valid for a non-separable H but the
proof requires a modification, which is based on the following observation: Proposition 13.7 implies
that (Ker T ) has at most countable orthonormal basis of eigenvectors { e j }. Then for any vector x H
write x = PKer T (x) + j=1 (x, e j )e j where PKer T is the orthogonal projection on the kernel of T . Then
follow the arguments of the proof adding this term when necessary).
77
Tx =
j (x, e j )e j
j=1
(T I)x =
( j )(x, e j )e j .
j=1
Sy =
(y, e )
k k ek .
k=1
Lemma 7.12 implies that the series converges since |k | > and
(y, ek ) 2
2
|(y, ek )|2 = 2 kyk2 .
kSyk =
j=k
k=1 k
2
(T I)Sy =
and
( j )(Sy, e j )e j =
j=1
j=1
((T I)x, e j )
ej =
j
j=1
j (x, e j )e j = x .
S(T I)x =
j (y, e j )e j = y .
j=1
78
14
Sturm-Liouville problems
p(x)
+ q(x)u = u
with u(a) = u(b) = 0
dx
dx
where p and q are given functions on the interval [a, b]. The values of for which
the problem has a non-trivial solution are called eigenvalues of the Sturm-Liouville
problem and the corresponding solutions u are called eigenfunctions.
An eigenvalue is called simple, if the corresponding eigenspace is one-dimensional.
The main conclusion of this chapter is the following theorem:
Theorem 14.1 If p C1 [a, b], q C0 [a, b], p(x) > 0 and q(x) 0 for all x [a, b],
then
(i) eigenvalues of the Sturm-Liouville problem are all simple,
(ii) they form an unbounded monotone sequence,
(iii) eigenfunctions of the Sturm-Liouville problem form an orthonormal basis in L2 (a, b).
For a function u C2 [a, b] we define
du
d
p(x)
+ q(x)u .
L(u) =
dx
dx
Let L0 : D0 C0 [a, b] be the restriction of L onto the space
D0 := u C2 [a, b] : u(a) = u(b) = 0 .
We can equip both D0 and C0 [a, b] with the L2 norm. Integrating by parts, we can
check that L0 is symmetric, i.e. (u, L0 v) = (L0 u, v) for all u, v D0 . On the other hand,
considering L0 on the sequence un = n1 sin(n(x a)/(b a)), we can check that L0
is not bounded. We have not studied unbounded operators.
Eigenfunctions of the Sturm-Liouville problem are eigenvectors of L0 . In order
to prove Theorem 14.1, we will show that L0 is invertible and L01 coincides with the
restriction on C0 [a, b] of a compact self-adjoint operator A : L2 (a, b) L2 (a, b). The
Sturm-Liouville theorem (see Corollary 13.9) implies that eigenfunctions of A form
an orthonormal basis in L2 (a, b). Moreover, we will see that all eigenfunctions of A
belong to D0 and, consequently, L0 have the same eigenfunctions as A.
79
Differential equation Lu = f
Lemma 14.2 If both u1 and u2 satisfy the equation Lu = 0, i.e.
(pu0 )0 + qu = 0,
(14.1)
then
Wp (u1 , u2 ) = p(u01 u2 u1 u02 )
is constant. Moreover, if Wp (u1 , u2 ) 6= 0 then u1 and u2 are linearly independent.
Proof: Differentiating Wp with respect to x and using pu00 = p0 u0 + qu we obtain
Wp0 = p0 (u01 u2 u1 u02 ) + p(u001 u2 u1 u002 )
= p0 (u01 u2 u1 u02 ) + ((p0 u01 + qu1 )u2 (p0 u02 + qu2 )u1 ) = 0 .
Therefore Wp is constant.
Suppose u1 and u2 are linearly dependant, then there are constants 1 , 2 such that
1 u1 + 2 u2 = 0 and at least one of the constants does not vanish. Suppose 2 6= 0
(otherwise swap u1 and u2 ). Then u2 = 1 u1 /2 and u02 = 1 u01 /2 . Substituting
these equalities into Wp (u1 , u2 ) we see that Wp (u1 , u2 ) = 0. Therefore Wp (u1 , u2 ) 6= 0
implies that u1 , u2 are linearly independent.
Lemma 14.3 The equation (14.1) has two linearly independent solutions, u1 , u2
C2 [a, b], such that u1 (a) = u2 (b) = 0.
Proof: Let u1 , u2 be solutions of the Cauchy problems
(pu01 )0 + qu1 = 0
(pu02 )0 + qu2 = 0
According to the theory of linear ordinary differential equations u1 and u2 exist, belong
to C2 [a, b] and are unique.
Moreover, u1 and u2 are linearly independent. Indeed, suppose Lu = 0 for some
u C2 [a, b] and u(a) = u(b) = 0. Then
Z b
(pu0 )0 u + qu2 dx
(using definition of L)
a
b Z b
0
= p(x)u (x)u(x) +
p(u0 )2 + qu2 dx (using integration by parts)
0 = (Lu, u) =
Z b
p(u0 )2 + qu2 dx
Since p > 0 on [a, b], we conclude that u0 0. Then u(a) = u(b) = 0 implies u(x) = 0
for all x [a, b].
Consequently, as u2 (b) = 0 and u2 is not identically zero, u2 (a) 6= 0 and so
Wp (u1 , u2 ) = p(a)(u01 (a)u2 (a) u1 (a)u02 (a)) = p(a)u01 (a)u2 (a) 6= 0.
Therefore u1 , u2 are linearly independent by Lemma 14.2.
80
u(x) =
G(x, y) f (y) dy
a
belongs to C2 [a, b], satisfies the equation Lu = f and the boundary conditions u(a) =
u(b) = 0.
Proof: The statement is proved by a direct substitution of
u2 (x)
u(x) =
Wp (u1 , u2 )
Z x
a
u1 (x)
u1 (y) f (y) dy +
Wp (u1 , u2 )
Z b
x
u2 (y) f (y) dy
into the differential equation. Moreover, u1 (a) = u2 (b) = 0 implies u(a) = u(b) = 0.
Integral operator
Lemma 14.5 The operator A : L2 (a, b) L2 (a, b) defined by
Z b
(A f )(x) =
G(x, y) f (y) dy .
a
Z b
G(x, y)u(y) dy .
a
81
Obviously, |G(x, y)u(y)| kGk |u(y)| for all x, y [a, b]. Since G is continuous, the
Dominated Convergence Theorem implies that we can swap a limit x x0 and the
integration, and thus the integral in the right-hand-side is a continuous function of
x. Consequently, u is continuous. For a continuous u the integral is in C2 [a, b] and
satisfies the boundary conditions u(a) = u(b) = 0 due to Lemma 14.4. Thus u D0 .
Therefore, the eigenfunctions of A belong to D0 .
Proof of Theorem 14.1: Since A : L2 (a, b) L2 (a, b) is compact and self-adjoint,
Theorem 13.9 implies that its eigenvectors form an orthonormal basis in L2 (a, b). If u
is an eigenfunction of A, then Lemma 14.5 implies that u C2 [a, b] and u(a) = u(b) =
0. Moreover, Lemma 14.4 and Au = u with 6= 0 imply that Lu = u with = 1 .
Consequently, u is also an eigenfunction of the Sturm-Liouville problem.
Finally, suppose that u is an eigenfunction of the Sturn-Liuoville problem and u is
another eigenfunction which corresponds to the same eigenvalue. Both eigenfunctions
satisfy the linear odinary differential equation L(u) = u and u(a) = u(a)
= 0. Then
u(x)
= u(x)u0 (a)/u0 (a) due to uniqueness of the solution of the Cauchy problem. Thus
the eigenspace is one-dimensional.
d2u
= u,
dx2
u(0) = u(1) = 0 .
f (x) =
k sin kx
k=1
where
1
k =
2
Z 1
f (x) sin kx dx .
0
82