Hilbert Spaces

Joseph Muscat
2014-5-27
(A revised and expanded version of these notes are now published by
Springer.)

1

Inner Product Spaces

1.1

Introduction

Definition

An inner-product on a vector space X is a map
h , i:X ×X →C

such that
hx, y + zi = hx, yi + hx, zi,
hx, λyi = λhx, yi,
hy, xi = hx, yi,
hx, xi > 0;

hx, xi = 0 ⇔ x = 0.

In words, it is said to be a positive-definite sesquilinear form.
PN
The simplest examples are RN and CN with hx, yi =
¯n yn ; the
n=1 x
square
PN matrices of size N × N also have an inner-product given by hA, Bi =
¯
i,j=1 Aij Bij .
Proposition 1.1

1. hx, xi is real (and positive) and is denoted by kxk2
2. hx, yi = 0 ∀x ⇒ y = 0 (put x = y)
¯ yi (anti-linear); hx + y, zi = hx, zi + hy, zi;
3. hλx, yi = λhx,
1

1.1

Introduction

J Muscat

2

4. kλxk = |λ|kxk.
Definition Two vectors x, y are orthogonal when hx, yi = 0, written
as x ⊥ y. The angle between two vectors is given by cos θ = hx, yi/kxk kyk.
Proposition 1.2
1. kx + yk2 = kxk2 + 2Rehx, yi + kyk2 .
2. (Pythagoras) If z = x + y and hx, yi = 0 then kzk2 = kxk2 + kyk2.
3. For any orthogonal vectors x, y, kxk 6 kx + yk.
4. For any non-zero vectors x, y, there is a unique vector z and a
unique scalar λ such that x = z + λy and z ⊥ y.
Proof. The first three statements follow immediately from the axioms
and the first proposition. For the last statement, let z = x − λy as required;
then z ⊥ y holds ⇔ λ = hy, xi/hy, yi (check). 

Proposition 1.3

Cauchy-Schwarz
|hx, yi| 6 kxkkyk

Proof. Decompose x into orthogonal parts x = (x − λy) + λy where λ =

hy,xi
hy,yi

(assuming y 6= 0); now use Pythagoras’ theorem, and deduce that |λy| 6 kxk. 

Note that Pythagoras’ theorem and Cauchy-Schwarz’s inequality are still
valid even if the ‘inner-product’ is not positive definite but just semi-definite,
as long as kyk =
6 0.
Corollary
kx + yk 6 kxk + kyk
The proof is simply an application of the Cauchy-Schwarz inequality to
the expansion of kx + yk2. 

1.1

Introduction

J Muscat

3

Hence kxk is a norm, and all the facts about normed vector spaces apply
to inner-product spaces. In particular they are metric spaces with distance
d(x, y) = kx − yk, convergence of sequences makes sense as xn → x ⇔
kxn − xk → 0, continuity and dual spaces also make sense. Inner-product
spaces are special normed spaces which not only have a concept of length
but also of angle.
Definition A Hilbert space is an inner-product space which is complete as a metric space.
Proposition 1.4
subspace.

The closure of a linear subspace remains a linear

¯
Proof. If xn , yn ∈ A, then limn→∞ xn +limn→∞ yn = limn→∞ (xn +yn ) ∈ A. 

Recall that finite dimensional subspaces are always closed. (exercise)
From the theory of Banach spaces we know that addition and scalar
multiplication are continuous operations.
Proposition 1.5

+, λ·, h, i are continuous in each variable.

Proof. If kx − x′ k < δ and ky − y ′k < δ then

k(x + y) − (x′ + y ′)k 6 kx − x′ k + ky − y ′ k < 2δ.
Similarly,
kλ(x − x′ )k = |λ|kx − x′ k < |λ|δ.

We can consider φy := hy, i to be a functional mapping x → hy, xi.
This functional is continuous because |φy (x)| = |hy, xi| 6 kykkxk. In fact
kφy k = kyk. 

It follows that taking limits commutes with the inner-product:
hx, lim yn i = lim hx, yn i.
n→∞

n→∞

Do all norms on vector spaces come from inner-products, and if not, which
normed vector spaces are in fact inner-product spaces? The answer is given
by

7 R ¯ f g. gi = Proof.1. Verifying the parallelogram law. Expanding and adding kx + yk2 and kx − yk2 gives the parallelogram law.1 Introduction J Muscat 4 Proposition 1. 3. .1. Hence. Z Z Z 2 2 |f + g| + |f − g| = (|f |2 + f¯g + f g¯ + |g|2 + |f |2 − f¯g − f g¯ + |g|2. Conversely.1 Exercises 1. centred at y.  This law can be generalized further. yi = 2πi S 1 ky + zxk dz. and only if. = R then hx. we get the polarization identity hx.6 (Parallelogram law) A norm comes from an inner-product if. 1 N Proposition 1. yi = ∞ ¯n y n . yi = − Re ihx. Show that ℓ2 is a Hilbert space with inner-product hx. We assume L2 (R) is a complete normed space. yi =  1 ky + xk2 − ky − xk2 + iky + ixk2 − iky − ixk2 . Similarly show that L∞ and L1 are not inner-product spaces. yi P N 1 n n 2 2 n=1 ω ky + ω xk . yi.  1. 4 The converse is true but harder to prove. subtracting the two gives 4 Re hx. If ω N = 1 (N > 2). Hence find the associated inner-product from the previous proposition. π]. by noticing that Im hx. Find the angle between sin and cos in the space L2 [−π. yi. the inner product hx. L2 (R) is a Hilbert space with inner-product hf. yi is a ‘complex’ average of lengths on a ball of radius kxk. it satisfies kx + yk2 + kx − yk2 = 2(kxk2 + kyk2) Proof. yi = Re hix. hx. P 2. Even more generally. n=1 x 1 ∞ Show further that l and l are not inner-product spaces by finding two sequences which do not satisfy the parallelogram law. That is.

Let d = inf λ kx + λyk. from which we can define a p norm k. 8. x) = A(x. defines an inner-product. find a basis for this space. give the same angles between vectors. Thus a complex inner-product has more structure than a real (symmetric) inner-product e.. y) that satisfies A(y. x). 11. Show that the set of continuous functions C[a. 10. y) = 14 (q(x + y) + q(x − y) + iq(x + iy) − iq(x − iy)). on the same vector space. yi1 = λhx. and only if. as A(x. More generally. x = αy ∃α. y1iH1 + hx2 . gi := f (x)g(x)w(x) dx.1. yi2 for some positive real λ. R 13. hx. Show that the formula hf. Show that the real and imaginary parts of an inner-product give symmetric/antisymmetric real-valued inner-products. Show that the two norms are identical. 9.g. b] with dimension m + 1. where w(x) is a positive real function. Let H1 × H2 be the product of two inner-product spaces. show that any bilinear form A(x. The resulting space of functions is called a weighted L2 space. and only if. We have shown that an inner-product is determined by a norm that satisfies the parallelogram law. we can create an inner-product using Prop.k1 := h. 6. 6. Show that   h x1 . C2 has more structure than R4 . b]. y1 . Show that any finite-dimensional subspace of a Hilbert space is closed. . if. Show that the Cauchy-Schwarz inequality becomes an equality if. Starting from a norm k. . 7. Show that two inner-products. is determined by its real quadratic form q(x) = A(x. yi|2 6 (kxk2 − d2 )kyk2 . b] is a non-closed linear subspace of L2 [a. 5.1 Introduction J Muscat 5 4.k that satisfies the parallelogram law. 12. y2 iH2 defines an inner-product on H1 × H2 . x2 . are conformal i. Deduce that the set of polynomials of degree at most m forms a closed linear subspace of L2 [a.e.i. y). show that |hx. y2 i := hx1 .

xi = 0 ∀b ∈ B. hb. Let x ∈ A⊥ . For (iii). xi = 0 ∀y ∈ A }. X ⊥ = 0.2 Orthogonality Definition The orthogonal space of a set A is the set A⊥ = { x : hy.8 A⊥ is a closed linear subspace. let x ∈ B ⊥ i. In particular this is true for b ∈ A so that x ∈ A⊥ . 3. Now. Proposition 1. yi = h limn xn . Hence x ∈ A⊥⊥ . (i) is left as an easy exercise. Hence hx. * Consider the vectorR space of holomorphic functions f : R × R+ → C which satisfy supy>0 |f |2 dx < ∞.1 Exercises 1. That is. yi = limn hxn . Hence x ∈ A⊥ .  Proposition 1. Show that the formula hf. hx. for any y ∈ A. A ⊆ A⊥⊥ . Show that if { x }⊥ = X then x = 0.e. 3. ∀y ∈ A⊥ .2.  Note that A⊥⊥ is always a closed linear subspace even if A isn’t. yi = 0 ∀y ∈ A⊥ .1. gi := R limǫ→0+ f (x + iǫ)g(x + iǫ) dx gives an inner-product. A ∩ A⊥ ⊆ 0. For (ii). Proof. 1. there is a sequence of vectors xn ∈ A⊥ such that xn → x.2 Orthogonality J Muscat 6 14. 2. 1. Question: if M is a closed linear subspace is it necessarily true that M = M ⊥⊥ ? . 2. yi = 0. then hy. Show that 0⊥ = X. A ⊆ B ⇒ B ⊥ is a closed subspace of A⊥ . Proof. let x ∈ A. Show that if { x }⊥ = 0 then X is one-dimensional. That A⊥ is a linear subspace follows from the linearity of the innerproduct. xi = 0.9 1.

Theorem 1. kyn − ym k2 = 2kyn − xk2 + 2kym − xk2 − k(yn + ym ) − 2xk2 6 2kyn − xk2 + 2kym − xk2 − 4d2 →0 as n → ∞. d = inf y∈M d(x. 1]. For example l∞ has a closed subspace c0 . . y) ∀y ∈ M. Suppose y ∈ M is another closest point to x. by continuity of the norm.10 If M is a closed convex set and x is any point of a Hilbert space H. ∀x ∈ H ∃!x0 ∈ M : d(x. tx + (1 − t)y ∈ A. 1. the smallest distance is d = 1 and this is achieved by any bounded sequence (an ) with 0 6 an 6 2. Now. by the parallelogram law. Proof.3 Least Distance in Hilbert spaces 1. is not true in Banach spaces in general. i. y) (how do we know that d exists?) Then there is a sequence of vectors yn ∈ M such that kx − yn k = d(x. which does not refer to inner-products. Then kx0 − yk2 = 2kx0 − xk2 + 2ky − xk2 − k(x0 + y) − 2xk2 6 2kx0 − xk2 + 2ky − xk2 − 4d2 =0  Note that this theorem. then there is a unique point x0 ∈ M which is closest to x.1. . . In fact. yn ) → d.e. But M is closed (hence complete) and so yn → x0 ∈ M. but there is no closest sequence in c0 to the sequence (1. x0 ) 6 d(x. y) = d. . Exercise: Show that linear subspaces and balls are convex.3 J Muscat 7 Least Distance in Hilbert spaces Definition its points: A convex set is a set A that includes all line segments between ∀x. y ∈ A. d(x.). Exercise: prove these assertions. It follows. ∀t ∈ [0.e. Hence (yn ) is a Cauchy sequence. that kx − x0 k = d. Let d be the smallest distance from M to x i. 1.

x − x0 i = 0. Then A⊥ = A⊥⊥⊥ = H ⊥ = 0. xi = hb. ⊥ ⊥⊥ Therefore. Then 0 = hb. By Pythagoras’ theorem. ¯ = H.  [[A]] is dense in H if. x0 ∈ M is the closest point to x ∈ H if. note that [[A]] is the smallest closed linear subspace containing A. Since a is arbitrary we get x − x0 ⊥ M. since A⊥ is a Proof. Proof. Moreover M ∩ M ⊥ = 0. x − x0 i/kak2 and z = x − (x0 + λa). A⊥ = 0. we get kzk 6 kx − x0 k making x0 + λa even closer to x than the closest point x0 . x − x0 ∈ M ⊥ . then M ⊥⊥ = M. Corollary If M is a closed linear subspace of a Hilbert space H. Conversely. These two components are in M and M as proved previously.11 For a closed linear subspace M of a Hilbert space H. Then x = a + b where a ∈ M and b ∈ M ⊥ . Proof. if x − x0 ⊥ a for any a ∈ M. [[A]]  Corollary . since A ⊆ [[A]].g. where λ = ha. Let x ∈ M ⊥⊥ . ha. Write x = x0 + (x − x0 ). This is only possible if λ = 0 i. bi = kbk2 . and only if. For the second part. so that kx − x0 k 6 kx − ak.  Corollary If M is a closed linear subspace of a Hilbert space H. then H = M ⊕ M ⊥ . the space l∞ 6= c0 ⊕ M for any linear subspace M. ai + hb.1.  ⊥ Note that this is false for Banach spaces e. for any set A. then by Pythagoras’ theorem. Suppose [[A]] ¯ = A⊥⊥ = 0⊥ = H. kx − ak2 = kx − x0 k2 + kx0 − ak2 . making x0 the closest point to x. Let a be any point of M. More generally. A⊥⊥ = [[A]]. we get [[A]] ⊆ A⊥ and hence A⊥⊥ ⊆ [[A]] = [[A]]. Conversely.3 Least Distance in Hilbert spaces J Muscat 8 Let us concentrate on the case when M is a closed subspace of H. closed linear subspace. and only if. Proof.e. Then x − x0 = λa + z with a ⊥ z. Theorem 1. making b = 0 and x ∈ M.

Show that the map (x. Proof. Let P. Show that P Q is also a projection. 0) is an orthogonal projection in R3 . Q be two commuting projections. show that a and b are unique. .3 1.3. Find a projection which is not orthogonal. 0. In the decomposition x = a + b with a ∈ M and b ∈ M ⊥ . Find the closest points x0 to a closed linear subspace M from (i) a point x ∈ M. if M is a closed linear subspace of N. P is continuous since kxk2 = kx − P xk2 + kP xk2 by Pythagoras’ theorem so that kP xk 6 kxk. P x = x.1 Least Distance in Hilbert spaces J Muscat 9 Exercises 1. By the definition of P . 3.3. P is linear since (x + y) − (P x + P y) = (x − P x) + (y − P y) ∈ M ⊥ and P x + P y ∈ M.12 If M is a closed linear subspace of a Hilbert space H then the map P : H → H defined by P x = x0 is a continuous orthogonal projection with imP = M and ker P = M ⊥ . 2.3. z) 7→ (x. Definition A projection is a linear map P : H → H such that P 2 = P . P is a projection since P x ∈ M implies that P 2 x = P x. Moreover x ∈ ker P ⇔ P x = 0 ⇔ x = x − P x ∈ M ⊥. hence P (x + y) = P x + P y. 4. (ii) a point x ∈ M ⊥ .3 Exercises 1. for any a ∈ H. Theorem 1. y. 1.2 Projections Let us take a closer look at the map from x to x0 ∈ M. An orthogonal projection is one such that ker P ⊥ imP . 2. Deduce that if H = M ⊕ N where M is a closed linear subspace and M ⊥ N then N = M ⊥ .  1. Similarly P (λx) = λP x. P is onto M since for any x ∈ M. then M ⊕(M ⊥ ∩N) = N. (a − P a) ∈ M ⊥ and this property defines P a ∈ M. Show that.1.

0). . 2π]. Hence note that there is a 1-1 correspondence between closed linear subspaces and orthogonal projections. which can be extracted by solving. with basis 1. xi/kyi k . .4 Examples: Least Squares Approximation 1. b. Consider two closed linear subspaces M1 and M2 (with associated projections P1 and P2 ). is parallel to n. x3 . Find the orthogonal projection P which maps any point x to its closest point in M. . Substituting gives λ. p is characterized by the condition p − sin ⊥ q. giving four linear equations in the four unknowns a. Suppose that H = M ⊕N. Show that the iteration xn+1 = P2 P1 xn starting from x0 = y converges to the closest point y0 ∈ M1 ∩ M2 . 5. 8. Pn with Pi Pj = Pi δij . The space of cubic polynomials.3. Letting n = (2. x0 = x + λn such that it satisfies n · x0 = 0. d. Show that if P is a projection then H = ker P ⊕ imP . then I = P + Q and P Q = 0. x. 1. (Ans: P x = hy.e. . It must therefore have an element p(x) = a + bx + cx2 + dx3 which is closest to sin x. . Extend this by induction to the case when H = M1 ⊕ . The righthand sides can be worked out. . is a four-dimensional closed linear subspace of the Hilbert space L2 [0. hp(x). In particular we get four equations hp(x) − sin(x). .3 Least Distance in Hilbert spaces J Muscat 10 3. . the equation of the plane is given by n · x = 0. c. 2. xi i = 0 for i = 0. yN i. 6. Show that any orthogonal projection is continuous. 3. xiy) P 9. 1. (Ans: P x = i αi yi where 2 αi = hyi . equivalently. xi i = h sin(x). 4. . . and imP ⊥ ker P . Find P again. Let M =< y > (a closed linear subspace). ⊕ Mn to get I = P1 + .1. In fact. ∀q ∈ M. 7. 2π]. 2. . Show that if P is an orthogonal projection onto M then I − P is an orthogonal projection onto M ⊥ . Let M = hy1 . . . Find the best-fitting (closest in the space L2 ) cubic polynomial to the function sin in the region [0. if P and Q denote the orthogonal projections onto M and N respectively. Hence. Find the closest point in the plane 2x + y − 3z = 0 to the point x = (5. Show that. xi i. x2 . Therefore we are looking for a point x0 in the plane such that x − x0 is perpendicular to the plane i. or. a+bx+cx2 +dx3 . −3).

which translated becomes. Implement the above examples as Mathematica programs. bN ) in RN which is a Hilbert space. . Mi = 0 becomes two equations hy − b. . n Solving for m and c gives the usual regression line as used in statistics. ei i = 0. 1]. X X X m an + c 1= bn . . 4.1. Find that combination of sin and cos which is closest to 1 − x3 in L2 [0. We wish to find a line y = mx + c for which y = (yn ) where yn = man + c) are collectively as close to bn as possible. Finding the best-fitting (least-squares) line to a number N of points (an . . 3. .5 Exercises 1. . aN ) and e2 = (1.3. . . Hence the condition hy − b. 2. 2π]. Find that quadratic polynomial that is closest to sin x in L2 [0. These lines form a closed subspace M spanned by the two vectors e1 = (a1 . bn ) ∈ R2 . Find.3 Least Distance in Hilbert spaces J Muscat 11 3. 1) (check). and implement. One can consider such a sequence of N pairs to be a single point b = (b1 . n m X n a2n + c n X n an = n X bn an . . . 1. . . the best-fitting circle to a number N of points in R2 . .

). the Hilbert space is called separable. From now on we will assume that the Hilbert basis is countable. let x = (x1 . 0. . xi = 0 ∀n ⇒ x = 0. n=1 an en . Then for any i. hen . kxN − xM k2 = hxN − xM . . then they can be made so using the usual Gram-Schmidt process: 2. . em i P 2 = N n=M +1 |an | . . em i = δnm . . an i. . . 2 (an ) ∈ ℓ ⇔ Proof. N n=M +1 m=M +1 am emi PN = n. e0 := b0 /kb0 k en := bn /kbn k. Fourier Expansion Proposition 2. xi = 0. .1 Let M be a closed linear subspace M. Note that the first condition is equivalent to hen . 1. is a set of vectors {en } such that hen i is dense in X. 0.0. Let xN := PN n=1 ∞ X an en converges in M.0. 2. assuming N > M wolog. also called a total set of vectors. if the span of a countable number of vectors {an } is dense in X but not orthonormal. P bn := an − n−1 i=0 hei .2 Orthonormal Bases 2 J Muscat 12 Orthonormal Bases Definition A Hilbert basis. it is the first one that is crucial. x2 .1 b0 := a0 .m=M +1 an am hen . xi = hei . xN − xM i P P =h N an en . To prove the first condition. . Note also that the set of basis vectors need not be countable. with a countable Hilbert basis en .7 Gram-Schmidt orthogonalization Of the two properties. . hence x = 0. when it is. 2. Then. .) be a sequence in ℓ2 which is orthogonal to all the ei .6 Example The space of sequences ℓ2 has a Hilbert basis consisting of en = (0.

xien . Conversely. P ∀m hem . letting an = hen . and hence must converge as N → ∞. xN i P P an an + N = kxk2 − 2 N n. xi − hx. then kxN − xM k → 0. Let xN := we get hence PN n=1 hen . x − yi = hem .1 Fourier Expansion J Muscat 13 Suppose that (an ) ∈ ℓ2 . say to y ∈ M. then X (i) |hen . has a countable Hilbert basis en . xien . em i n=1 PN 2 2 = kxk − n=1 |an | . implying that (xn ) is a Cauchy sequence in M. 0 6 kx − xN k2 = hx − xN . N X n=1 |hen . n (ii) P x = x0 = X n Proof. is an increasing series. hen .2 (Bessel’s inequality) If a closed linear subspace M of a Hilbert space. proving the Bessel inequality. en i = 0. P N By the previous proposition. xien . x − xN i = kxk2 − hxN . xi|2 6 kxk2 . PN implying that n=1 |an |2 is a Cauchy sequence in C. xi − ∞ n=1 hen . n=1 . then ∞ X ∀x ∈ X x= hen . then this last sum converges to 0. xien converges. xihem . bounded above by kxk2 .2. xN i + hxN . the series of vectors n=1 hen . which must therefore converge to a point x ∈ M. Moreover. so that x − y ∈ { em }⊥ = M ⊥ . xi|2 6 kxk2 . Then xN ∈ M and. xi.  Theorem 2. suppose that the series xN converges. Hence the sum on the lefthand side must converge.  Corollary (Parseval’s identity) If en is a countable Hilbert basis for X. Hence y must be the closest point in M to x.m=1 an am hen .

The first part is immediate from the theorem since P x = x when M = X. for x.1 Fourier Expansion J Muscat and hx. bi are known coefficients. Substituting into Ax = b we get X X xn Aen = bn en . hx. and for A a continuous linear operator. yiX = hJx. i. xi are unknown coefficients and bn = hen . b in a function space.2. n. Note that isometries P must be 1-1 (their kernel must be trivial). was to write the unknown element x and the known P P element b in terms of a Hilbert basis en to get x = n xn en and b = n bn en where xn = hen . yi = where x = P n αn en and y = P X 14 αn βn . then X ≃ ℓ2 . we get X Anm xn = bn . n n Moreover we can expand the vectors Aen = coefficients.  Proposition 2.m P m Anm em and. Jyiℓ2 . comparing . Consider the map J : X → ℓ2 x 7→ (αn ) = (hen . Linearity is easily checked.e. n n Proof.3 If X has a countable Hilbert basis. In particular X kxk = ( |αn |2 )1/2 . J is onto ℓ2 since it maps n αn en to (αn ) and the former is a well-defined vector for (αn ) ∈ ℓ2 . The second part is a simple expansion of the two series in the inner-product making essential use of the linearity and continuity of h.1. Let en be a countable Hilbert basis.  2. xi) Then Bessel’s inequality shows that the resulting sequence is truly in ℓ2 . The Parseval identity is precisely the fact that the inner-products are preserved i.1 Solving Linear Equations One of the earliest methods for solving a linear equation Ax = b. n βn en . Proof.

2. J Muscat 15 This can be thought of as a matrix equation in ℓ2 with the matrix [Anm ] having a countable number of rows and columns.2. via the map J. First expand the function in an appropriate basis. then there are no solutions. bi. and only if. we have transferred the problem from one on H to one on ℓ2 . can be rewritten as ! Z X X X 1 (bn /λn )en = hen . The first N “large” coefficients can be used to “store” the function in a useful compressed way. then remove those coefficients which are smaller than a given threshold.1.2 Hilbert Bases for L2. then we get 0 = bn = hen . we must choose xn = bn /λn .e. this gives λn xn = bn . Each basis has particular properties that can be utilized in specific contexts. Effectively. Such bases are also used to filter out noise or pick out particular features in a function. One should treat these the same way that we treat bases in finite dimensional linear algebra. m n where the am are arbitrary constants. This has been used in JPEG and MPEG compression. Note that the latter part is the ”particular solution” and. then the equation simplifies because of Aen = λn en . as well as in compressing images for Microsoft Encarta. if this is false. 2. the homogeneous equation Ax = 0 has solutions). or manipulating functions is easily done using the Parseval identity. and by the FBI to store millions of fingerprints for rapid retrieval. When λn 6= 0.2 Hilbert Bases for L2 . In this case the eigenvectors of the zero eigenvalue { em } will span the homogeneous solutions. when λn = 0 (i. for the case of L2 . Thus there will be a solution if. b ⊥ ker A. Regenerate the function from the remaining coefficients. bien /λn = en (s)en (x) f (s) ds λn n n n equivalent to the Green’s function formulation of the particular solution. There are various Hilbert bases suitable for the space of L2 functions on different domains. otherwise we are free to choose xn arbitrarily. They are indispensable for actual . If the Hilbert basis elements en are chosen to be eigenvectors of A. and the complete solution will be X X x= am em + (bn /λn )en . Regenerating the function.2 Applications Hilbert bases are widely used to approximate functions.

is a countable Hilbert basis. (cos(y/2))2n dy = 2/π(n + 1) (use a reduction formula). Rπ |f (x)| 6 π(n+1) (cos(y/2))2n|f (x) − f (x + y)| dy 2 −π k(cos(y/2))2n kL2 [−π. . so that π(n + 1) f (x) = 2 Z π −π (cos(y/2))2n (f (x) − f (x + y)) dy. . Any other interval [a. kf (x) − f (x + y)kL2 [a.π] kf (x) − f (x + y)kL2 [−π. b] will do.4 The functions Hilbert basis for L2 [−π. . so 2 that we have Z π (cos(y/2))2n f (x + y) dy = 0. for a problem that has spherical symmetry. e2πinx is a Hilbert basis for L2 [0. Orthonormality of the functions follows from Z π inx imx he . b]: Fourier series Theorem 2.π] 6 π(n+1) 2 6 ckf (x) − f (x + y)kL2[−π. −π  Note that there is nothing special about the interval [−π. Multiplying by e gives e f (y) dy = 0 and.e. J Muscat 16 calculations. Rπ Proof (NOT FOR EXAM) Suppose that f ∈ { einx }⊥ .2 Hilbert Bases for L2 . ∀n ∈ Z. changing R −iny variables. √1π sin(x). that L2 functions are L2 -continuous i. π]. becomes e f (x + y) dy = 0. √1π cos(x). e i = eix(m−n) dx = 2πδnm . For example. Now (cos(y/2))2n = ( 21 einy/2 + 1 −iny/2 2n e ) is a linear combination of exponentials of various frequencies. For example.2. except that the basis functions have to be modified accordingly. .π] It is a theorem from Lebesgue integration (but probably not-covered in the introductory course in the third year). But we get Rπ −π −π Therefore.2. π]. but one has to be careful which basis to choose that makes the problem amenable. 1]. 2. Hence the function f must be zero. i.1 L2 [a.e. it would make sense to use a Hilbert basis adapted to spherical symmetry. −π e−iny f (y) dy = R in(x−y) inx 0. √1 einx 2π for n ∈ Z form a countable It follows that √12π .b] → 0 as y → 0.

This Fourier basis has the special property that they are eigenvectors of the differential operator: Deinx = ineinx . + a0 )cn einx = n Comparing coefficients of the nth term gives X bn einx . which can be solved to give the coefficients cn in terms of the known coefficients bn .  The Parseval identity for this Hilbert basis becomes ∞ ∞ X 1 X inx inx f (x) = he . Notice that this equation holds if. . and only if. b]. Corollary Proof. + a0 )cn = bn . The classical Parseval identity is none other than Z π X |f (x)|2 dx = kf k2L2 [−π. It can therefore be used to turn a constant-coefficient differential equation into a polynomial equation as follows: expanding unknown function P P the inx inx y(x) = n cn e and the known function f (x) = n bn e turns the differential equation (am D m + . The exponential functions can be approximated in L2 [a.2 Hilbert Bases for L2 . This can also be written in terms of the basis consisting of cosines and sines. b] as closely as needed by Taylor polynomials. . . the coefficients αn are in ℓ2 . Any other solution will not be found this way. which need not be the case. This means that the set of polynomials is dense in the set of exponential functions written above which in turn are dense in L2 . + a0 )y = f into the series X (am (in)m + . .π] = k(αn )k2ℓ2 = |αn |2 . f (x)ie = αn einx 2π n=−∞ n=−∞ R π 1 e−inx f (x) dx. J Muscat 17 The polynomials are dense in L2 [a.2. Of course this method assumes that both the solution y and the given function f are in L2 [−π. π]. . thus giving the usual Fourier series expanwhere αn = 2π −π sion. . (am (in)m + . −π n Moreover the Fourier operator F which sends functions in L2 to sequences of Fourier coefficients (in ℓ2 ) is a continuous isometry between Hilbert spaces. .

x2 i. etc. . etc.2. The first few terms are e−x/2 . However we can make them orthonormal using the Gram-Schmidt process. (1 − x)e−x/2 . 1]. ∞): Laguerre functions This Hilbert space does not contain any polynomials xn . 2 π √ int(n/2) X (−1)k (2k)!  n  (−1)n x2 /2 n −x2 2n n−2k −x2 /2 p hn (x) = p √ x e = √ e D e . ln (x) = n! k=0 k! k n! The Laguerre function are eigenvectors of S = xD 2 + D − x/4 1 Sln = −(n + )ln . 32 x. The general formula is   n 1 1 X n k −x/2 k 1 (−1) x e = ex/2 D n (xn e−x ). 2.4 L2 (R): Hermite functions This time we perform orthonormalization starting with the functions xn e−x 1 −x2 /2 √ 2 −x2 /2 We get the Hermite functions. as can be easily verified by calculating h1. . 1]: Legendre polynomials We have just found that the set of polynomials is dense in L2 [a. but we can modify them to xn e−x/2 which do belong.2 J Muscat 18 L2 [−1.3 L2 [0. The general formula is (!) q q   1 int(n/2) n + 2 X (−1)k (2n − 2k)! n n + 21 d n−2k x = ( )n (x2 −1)n . q q The first few polynomials are √12 . 2. the resulting polynomials are called the Legendre polynomials. 2 2. etc. √ .2.2.Hilbert Bases for L2 . A Gram-Schmidt orthonormalization gives the Laguerre functions. Pn (x) = n n k 2 n! (n − 2k)! 2 n! dx k=0 The Legendre functions are eigenvectors of T = D(x2 − 1)D = (x2 − 1)D 2 + 2xD: T pn = n(n + 1)pn . On the interval [−1.2 2. √ xe 4 πe 2 /2 . (1 − 2x + 12 x2 )e−x/2 . 32 52 (x2 − 13 ). b] but they turn out to be non-orthogonal. 2k 4k k! n! π k=0 2n n! π The Hermite functions are eigenvectors of R = D 2 − x2 : Rhn = −(2n + 1)hn .

deduce that hen . D m um i = −hD n−1 un . the modified Bessel functions on L2 (0. D m+1 um i and hence show that this inner-product must be zero unless n = m. for functions in L2 (R). This exercise is to prove that the Legendre polynomials are orthonormal. √1 π sin(nx).2. 5. 7. 10. Note that D k un (±1) = 0. 4. show that u(t. Show that the vectors must be dense in X and orthogonal.6 Exercises 1. Show that hD n un . . 1] with u(0) = u0 . e˜m ) form a Hilbert basis for X × Y . Laguerre and Hermite functions are indeed eigenvectors of the corresponding differential operators. Find all L2 [−1. Hermite. 1] solutions of the equation (x2 − 1)y ′′ + 2xy ′ + y = sin(πx). Suppose that en and e˜m form a Hilbert basis for X and Y respectively. (Hint: expand R2 ) 8. Suppose that en are a set of vectors such that khen . 6.5 Hilbert Bases for L2 . Solve y (4) − 2x2 y ′′ − 4xy ′ + (x4 − 2)y = g in series form. xikℓ2 = kxk for all x ∈ X.2. The Jacobi functions. 2. 2. π] to ℓ∞ continuously. Laguerre. Show that (en . the spherical harmonics on the sphere (L2 (S 2 )) etc. Let u be aPsolution of ∂t u = ∂x2 u on x ∈ [0. Write down the Parseval identity for the basis √1 .2. 9. xi = 0 except for a countable number of en . Show that the Bessel inequality is still valid even if the orthonormal set of vectors en is not countable. J Muscat 19 L2 (A) There are many other orthonormal Hilbert bases adapted to specific sets A or weights. x) = n αn (t)en (x) (justifies the separation of variables method). It is a theorem of Rodriguez that the functions fn (x) = 1 w(x)− 2 D n (w(x)p(x)n ) for any polynomial p and weight function w ∈ L2 (A) are orthogonal: the Legendre. Jacobi. √1 π 2π cos(nx). the Chebychev (on the circle). ∞). Chebychev functions are all of this type. 3. Define un (x) = (x2 − 1)n .2 2. (extra) Prove that the Legendre. Show that the Fourier operator F sends functions in L1 [−π.

The most common choices are the window-function h = χ[0. Such functions are necessarily nowhere differentiable. h(x − m)f (x)i|2 = kh(x − m)f (x)kL2 = |hh(x − m). such that hmn are orthonormal. |f (x)|i|2 Similarly summing the coefficients in m gives a windowed fourier transform of the function.1] . together with φ(x) = χ[0.1] . f i| = Pn |he = n |he2πinx . 1 ] − χ[ 1 .n (x) = e2πinx h(x − m). one of which is a frequency index.Hilbert Bases for L2 . Wavelet Bases The basis in this case consists of the following functions in L2 [0.2 2. Note that summing the coefficients in n gives the windowed function: P P 2 2πinx h(x − m). In contrast the coefficients that result from the Fourier operator. where h is a carefully chosen function.e. 2. for example. More recently.7 J Muscat 20 Frequency-Time Hilbert Bases A recent development in Hilbert bases are those bases for functions f (t) that give information in both frequency and time. A large nth coefficient means that there is a substantial amount of the term einx i.2. The m gives position (time) information. The classical Haar basis is generated by the case ψ(x) = χ[0. wavelets generated by a continuous 2 2 function have been used. 1] ψmn (x) := 2m/2 ψ(2m x − n).2. somewhere in the function f (x). with khkL2 = 1. The aim of frequency-time bases is to have coefficients amn depending on two parameters m and n. the other a ”time” index. only give information about the frequency content of the function. while the n gives frequency information. Windowed Fourier Bases The simplest way to achieve this is to define the basis functions by hm.8 Other Bases There are many other bases used specifically for compression etc.1] 2 and the gaussian h(x) = e−x /2 . A common one is the Walsh basis which consists of step functions that are the normalized . 2. f (x)i|2 n |hhmn . of frequency n.

Prove that the windowed Fourier basis with the window function h(x) = χ[0. 1 −1 Mn −Mn 2. Show that the Haar basis is orthonormal. Look up information on applications of wavelet and other Hilbert bases.2 Hilbert Bases for L2 . 3. 2.1] (x) are orthonormal. Mn+1 = . .9 Exercises 1.2.2. J Muscat 21 column vectors of the matrix MN where     1 1 Mn Mn M1 = .

Are there any other functionals besides these? In normed spaces this is generally the case e. First notice that for any z and y. for the Banach space of continuous bounded functions C[a. yi. every vector x gives rise to a continuous functional X → C x∗ = hx. non-zero.1 X∗ ≡ X In any inner-product space. yi − h˜ x. yi. from X to X ∗ more closely: .e. yi = hx.  Let us study this map. to get 0 = hz.1 (Riesz’ theorem) Every continuous functional of X is of the form hx. i. Assuming φ 6= 0. called the Riesz map. In Hilbert spaces however this is not the case: Theorem 3. suppose x˜ is another such x then.3 Dual Spaces 3 J Muscat 22 Dual Spaces 3. To show that it is unique. kzk2 where x = φ(z)z/kzk2 . while continuity follows from the Cauchy-Schwarz inequality |x∗ (y)| = |hx. i : y 7→ hx. ∀y so that x = x˜. φ(y)z − φ(z)yi = φ(y)kzk2 − φ(z)hz. b] which contains many other functions besides the continuous ones. pick z ⊥ ker φ. Hence φ(y) = φ(z) hz. ∀φ ∈ X ∗ ∃!x ∈ X φ = hx. Proof. i for some unique vector x i. b]. yi This is indeed linear. hx − x˜. yi| 6 kxkkyk. yi = φ(y) − φ(y) = 0. the dual space is L1 [a.g. yi = hx. φ(y)z − φ(z)y ∈ ker φ.

2 23 The map J : X → X∗ x 7→ x∗ is a bijective conjugate-linear isometry between X and X ∗ . Note further that Hilbert spaces are therefore reflexive Banach spaces i. 3. every linear isometry is 1-1 (ker J = 0 since the lengths of vectors is preserved). T xi|/kxkkyk). zi = λhx. In Banach spaces in general the most that can be said is that X is isomorphic to a closed subspace of X ∗∗ . the annihilator A◦ of a set corresponds to A⊥ .3. Prove the Hahn-Banach theorem for Hilbert spaces as follows. under the Riesz bijection. kyk kyk y using the Cauchy-Schwarz inequality. X ∗∗ ≡ X. zi = x∗ (z)+y ∗ (z) ¯ ¯ zi = for any z ∈ X. yi| |x∗ (y)| = sup = kxk.1 Exercises 1. zi+hy. λx To show that J is isometric. Prove the rest of the theorem. Hence kJxkX ∗ = kxkX .1 X∗ ≡ X J Muscat Proposition 3. zi = hx. ∗ ¯ (z).1. Let x and y be two vectors in X. Show that the norm of an operator is given by kT k = supx. J is onto by Riesz’ theorem. in particular with x = y. Use the Riesz map to show that the two equations kφk = supx (φ(x)/kxk) and kxk = supφ (φ(x)/kφk) become the same for Hilbert spaces. ∗ kx∗ kX ∗ = sup y |hx. Show that it must correspond to a vector in M.y (|hy. Proof. and hence find an extension of φ on X. 4. 2. Then J(x + y) = (x + y)∗ = x +y ∗ = Jx+Jy since (x+y)∗ (z) = hx + y. For bijectivity. Start with any functional φ on a closed subspace M. Show that.  This shows that X and X ∗ are essentially isomorphic as Hilbert spaces. . Similarly J(λx) = λJx since (λx)∗ (z) = hλx. for any y ∈ X.e. 3.

xi = hy. T xi = hy1 . ¯ ∗. xi = hx. kT ∗ k = kT k Proof. we get am = ym−1 so that L∗ (yn ) = (yn−1 ) i. so we should get an operator T ∗ : Y → X. ¯ ∗ y.e. xi = hλT . Then Ay = By for all y so that A = B.e. T ∗ is well-defined by the Riesz correspondence. (S + T )xi = hy. T xi ∀x. T. xi for any x. Moreover.2 Adjoint Map 3. T xi = hT ∗ y1 + T ∗ y2 . L∗ = R the right-shift operator.. xi = hλT ∗ y. xi.i where J is the Riesz map i. for every continuous operator T : X → Y there is a continuous adjoint or dual map T ⊤ : Y ∗ → X ∗ defined by T ⊤ φ(x) = φ ◦ T (x). (λT )∗ = λT (ST )∗ = T ∗ S ∗ . xi = hy. hT ∗∗ x. xi = hλy. yi = hx.1 Example Consider the left-shift map L : ℓ2 → ℓ2 defined by L(xn ) = (xn+1 ). it is defined by T ∗ (y) = Jhy. For Hilbert spaces. xi. T ∗ yi = hT ∗ y. and satisfies (S+T )∗ = S ∗ +T ∗ . Linearity: hT ∗ (y1 + y2 ). T xi = λhy. an xn = n n n In particular. Uniqueness: Suppose A and B are adjoints of T i. linear and continuous. T xi = hT x. the duals X ∗ and Y ∗ are essentially isomorphic to X and Y respectively.2 J Muscat 24 Adjoint Map Recall that for Banach spaces. taking x to be a basis vector em = (δnm ). h(λT )∗ y. T yi = hBy. y. T xi + hy2 . ¯ T xi = λhT ¯ ∗ y. yi. Lxi means X X X yn xn+1 = yn−1 xn . 3. hT ∗ (λy). T ∗ is unique. In fact. hT ∗ y. T xi = h(S ∗ + T ∗ )y. λT xi = λhT ∗ y. hAy.e. y ∈ X. We are therefore justified in using the notation T ∗ .2. xi. xi = hy. Let L∗ (yn ) = (an ). h(S + T )∗ y. xi = hy. xi. Proposition 3.3 T ∗∗ = T.3. Sxi + hy. xi = hy. xi = hy1 + y2 . then hL∗ y.

so that T ∗y ∈ M ⊥.5 A vector x minimizes kT x − bk if. T xi| = sup = kT k.2 Application When an operator T does not have an inverse. xi = hy. (ii) M is T -invariant ⇔ M ⊥ is T ∗ -invariant. M ◦ is T ⊤ -invariant. Conversely. T ∗ T x = T ∗ b. ker T = ker T ∗∗ = (imT ∗ )⊥ . M is T -invariant if. y ∈ ker T ∗ ⇔ hT ∗ y. Moreover. Then (i) ker T ∗ = (imT )⊥ . (ST )xi = hS ∗ y. for any y ∈ M ⊥ and x ∈ M. The problem is equivalent to finding a vector y = T x ∈ M which is closest to b. Proof. ⇔ hy. hT ∗ y. T xi = 0 ⇔ y ⊥ imT . kxkkyk kxkkyk x  Proposition 3. T xi = hT ∗ S ∗ y. . xi = 0 ∀x ∈ X.2 Adjoint Map J Muscat 25 h(ST )∗ y. Proof. the equation T x = b need not have a solution. In detail. Continuity: kT ∗ k = sup x.  3. xi = hy.4 Let T be a continuous operator and M a closed linear subspace.3. Proposition 3. Hence T ∗ (b − y) = 0 i. but T ∗∗ = T and M ⊥⊥ = M for a closed subspace M.e. Consider the closed linear subspace M = imT .2. and only if T ∗ T x = T ∗ b. xi. The next best thing is to ask for a vector x which minimizes kT x − bk. if M ⊥ is T ∗ -invariant then M ⊥⊥ is T ∗∗ -invariant. The second part is basically a translation of the fact that for Banach spaces. (imT ∗ )⊥ = ker T . xi| |hy. and only if.y |hT ∗ y. A necessary and sufficient condition is b − y ∈ M ⊥ = ker T ∗ . T xi = 0 since T x ∈ M.

there will be a unique such x with smallest norm. For T : H → H. T xi + ha. prove that kT ∗ T k = kT k2 . In the simple case when T is 1-1. 5. it is given by T † = (T ∗ T )−1 T (even if T is not onto). 2. ∗ −1 (T ) . then T has a continuous inverse (you need to use the open mapping theorem).2. and only if. 3. 6. 3.3 Exercises ¯ (ii) if T : H → H is invertible then (T −1 )∗ = 1. π] → ℓ2 .3. (Hint: consider q(x + h) > q(x) for all h). when ker T 6= 0. and deduce from this that kT ∗ k = kT k. Find the adjoint map of the Fourier map F : L2 [−π. 7. (Hint: this is equivalent to finding the closest vector in M to −a. xi + b occurs when (T + T ∗ )x + a = 0. Show that there is a unique vector x ∈ a+M with smallest norm. T ∗ is 1-1. 4. Show (i) (λI)∗ = λI.) 8. Let a + M be a coset of a closed linear subspace M. The mapping from b to this x is then well-defined and is denoted by T † . Show that imT is dense in X if. Moreover such an x need not be unique. 369 0 . Find the best approximation for x that solves     4 147 2 5 8 x = −1 .2 Adjoint Map J Muscat 26  Note however that kT x − bk may have no minimum when imT is not closed. Let X be a real Hilbert space and let T be a continuous operator. called the Moore-Penrose generalized inverse. a a fixed vector and b a fixed real number. However. Show that the minimum of the quadratic function q(x) = hx. Show that if T and T ∗ are onto.

(ii) follows from the fact that T ∗ x = 0 ⇔ kT ∗ xk = 0 ⇔ kT xk = 0 ⇔ T x = 0. Deduce that if hx. using (i). we can even say that they are identified by the numbers hx. Proof. . imT is dense in X ⇔ T is 1-1.4 Normal Operators 4 J Muscat 27 Normal Operators In this chapter and the next we will concentrate on operators from X to itself. T xi for all x.0. Also. T is 1-1. For complex Hilbert spaces. ∀x ⇔ T = 0. A normal operator T is a continuous operator such that ∗ A special case is that of self-adjoint operators which satisfy T ∗ = T . This is so because hx. let us note that an operator is identified by the numbers hy. T xi = hx. T T ∗ xi = hx. (i) kT ∗ xk = kT xk.1 For T a normal operator. Show that a continuous operator which satisfies kT ∗ xk = kT xk is normal. T xi = 0. T xi = kT xk2 . (imT )⊥ = ker T ∗ = ker T which is 0 if. (ii) ker T ∗ = ker T . Sxi. (i) follows from kT ∗ xk2 = hT ∗ x.  4. ∀x then T = S. Proposition 4. from which follows kT k2 6 kT 2 k.4 Exercises 1. Hence kT 2 xk = kT ∗ T xk. and kT xk2 = hx. T xi = 0. by substituting the vectors x + y and x + iy. and only if.e there are the usual binary operations of addition and multiplication on B(X) and a unary operation of taking the adjoint. Before we delve into the properties of these operators. T xi for all x ∈ X. 2. T ∗ T xi 6 kT ∗ T kkxk2 . These form a so-called C ∗ -algebra B(X) i. T ∗ T xi = hT x. y ∈ X. kT 2 k = kT k2. T ∗ xi = hx. Prove that for complex Hilbert spaces hx. ∀x ⇔ T = 0 (Prove!) Definition T T = T T ∗.

5. Axi > 0. Ayi|2 6 hx. Ayi. Claim: If A is self-adjoint such that hx. Proposition 4. Axi = hAx. But. and p(T ) for a polynomial p. then ∀x. Axi + 2t Re eiθ hx. Axi. Ayi) 6 ∴ |hx. there can be at most a countable number of distinct eigenvalues. First notice that for a self-adjoint operator A. Prove that for a separable Hilbert space. Axi is a real number since hx. |hx. (i) T ∗ . Deduce that the only normal nilpotent operator (i. T k = 0 for some k) is 0. by the open mapping theorem.) From the theory of Banach spaces. Prove that for a normal operator T .1 Spectrum Definition The spectrum of an operator T ∈ B(H) is the set σ(T ) := { λ ∈ C : T − λ does not have an inverse }. we can only show that the spectrum of a self-adjoint operator is non-empty. But the proof requires complex analysis. The proof is as follows: for any z = teiθ ∈ C. Ayi hx. The spectrum of any (self-adjoint) operator is non- Proof.) 4. Suppose T is diagonalisable i.2 empty. 06 = iθ 2 ∴ (Re e hx. For our purposes. Ayi|2 6 hx + zy. Axihy. every bijective continuous linear operator has a continuous inverse. Axihy. (Note: The standard definition requires that T −λ does not have a continuous inverse. xi = hx. Ayi hx. y. hx. Axihy. A(x + zy)i hx.4. T − λ. Ayi. (Hint: first show that T ∗ en = λ¯n en . Use (ii) of the proposition to show that ker T = ker T 2 = ker T k . so the two definitions are equivalent. . 6. it follows that the spectrum of any operator is a closed bounded non-empty subset of C. 4. Show that T is normal.e. (ii) ker T is T ∗ -invariant and (ker T )⊥ is T -invariant.1 Spectrum J Muscat 28 3. are also normal. Ayi + t2 hy.e there is a Hilbert basis of eigenvectors.

(iii) 0∈ / σ(T ) ⇔ ∃c > 0 kxk 6 ckT xk. and continuous: kT −1 yk 6 ckT T −1yk = ckyk. Proposition 4. λ belongs to σ(T ). Ayi|. for all x. Then T has a continuous inverse T −1 and kT −1 yk 6 ckyk. This implies. implying that either λ = µ or hy. xi = h¯ µy.4. Then T x = 0 implies x = 0 so that T is 1-1. xi = hy. Thus T is invertible. Conversely. Hence xn → x implying that yn = T xn → T x since T is continuous. (i) eigenvalues of T ∗ are conjugates of eigenvalues of T . A2 xi|2 6 hx.  That is. hx. Axihy. . Thus.1 Spectrum J Muscat 29 since θ can be chosen to rotate hx. let λ := inf kxk=1 hx. (ii) eigenvectors of distinct eigenvalues of T are orthogonal. implying kxk 6 ckT xk. x and y. Now.3  For a normal operator T . The image of T is closed since suppose yn = T xn is a Cauchy sequence in the image of T . (i) is a direct application of the previous proposition on ker(T − ¯ For eigenvalues λ and µ with corresponding eigenvectors λ) = ker(T ∗ − λ). xi. T xi = hT ∗ y. kAxk4 = |hx. xi = µhy. T xi → λ. we have λhy. a value λ is in the spectrum iff there are unit vectors xn such that (T − λ)xn → 0. These two facts imply that the image of T must be X itself since imT ⊥ = ker T ∗ = ker T = 0. Proof. T xi − λkxk2 > 0. Then kxn − xm k 6 ckT xn − T xm k = ckyn − ym k → 0 as n. T xi. (ii) Suppose the spectrum σ(T ) does not contain 0. Ayi into |hx. Ayi → 0 as hx. (T − λ)xi = hx. m → ∞. Using y = (T − λ)x and A = T − λ. xi = 0. suppose the right inequality is true for all x.

(T − λ)xi | > α > 0. (T − λ)xi| 6 kxkk(T − λ)xk.5 (Spectral Theorem in Finite Dimensions) A normal operator on a finite-dimensional vector space is diagonalisable.4 The spectrum of T has spectral radius kT k. . in which case there must be an eigenvector and λ is an eigenvalue (point spectrum). T xi/hx. This is equivalent to the diagonalizability of T . since the spectral radius of T is kT k.. show that the spectrum of a 0 −1 1 normal operator need not equal the closure of the set { hx. or it is not 1-1. Moreover. the “radius” of A itself is kT k. .1. and is a subset of the closure of the set {hx. T xi/hx. Proof.. Proof. ⇒ αkxk 6 k(T − λ)xk. Let A := {hx. xi}.1 Spectrum J Muscat 30 Proposition 4. Either T − λ is 1-1. Then λ ∈ / A implies that ∃α > 0 ∀x |λ − ⇒ ⇒ | hx. T xi| 6 kT kkxk2 we have that the set A is contained in the ball of radius kT k.4. T xi | > α. By taking T = and x = .1 Exercises     1 0 1 1. T xi/kxk2 }. contains no residual spectrum. Notice that since |hx. For normal operators ker(T − λ)k = ker(T − λ). 4. The spectral radius of σ(T ) is given by rT = limn→∞ kT n k1/n = k −k limk→∞ kT 2 k2 = kT k. so that the minimal polynomial consists of simple factors. xi}. . kxk2 αkxk2 6 |hx. 2. . B(X) has dimension (dim X)2 and so every operator T must satisfy a polynomial mT (x) = (x − λ)k . Show that kp(T )k = maxλ∈σ(T ) kp(λ)k when T is normal and p is a polynomial. This induces a decomposition of the vector space as X = ker(T − λ)k + . .  Theorem 4. In finite dimensions. kxk2 hx. in which case its image is dense in X and λ forms part of the continuous spectrum. ⇒ λ∈ / σ(T ).

 This proposition is basically saying that preserving the inner-product (lengths and angles) is equivalent to preserving lengths. For continuity. kT k = supx (kT xk/kxk) = 1. T yi = hx. T ∗ is itself unitary since (T ∗ )−1 = (T −1 )∗ = T ∗∗ . T are. y ∈ X. Proof.7 Let T be a unitary operator. By the comment at the beginning of this chapter. the adjoint T ∗ = T −1 is also unitary.2 4. and only if. Proposition 4. yi ∀x. which for the case of unitary operators. T ∗ T xi ∀x ⇔ T ∗ T = I. Finally. the set of unitary operators form a group i. yi ∀x. xi = hT x. it is onto and T ∗ T = I. 3. Proposition 4. hx.4.  . T is continuous with norm kT k = 1.e. we can also say that hx. Hence we conclude that a linear operator is unitary if. then 1. 2.e. Recall also that norm-preserving linear operators must be 1-1. is its adjoint. T is normal since T always commutes with its inverse. 4. Proof. T yi = hx. T is normal. yi = hT x. y ⇔ T ∗ T = I. (ST )∗ (ST ) = T ∗ S ∗ ST = I and ST is invertible when S. y ⇔ T ∗ T = I ⇔ kT xk = kxk ∀x. if S. T yi = hx. T ∗ T yi ∀x. T xi = hx. T are unitary then so is ST . hence T ∗ = T −1 .2 Unitary Operators J Muscat 31 Unitary Operators Definition A unitary operator is a Hilbert-space automorphism i. hT x. If T is unitary then it is invertible and T ∗ T = I. a linear bijection which preserves the inner-product.6 The following are equivalent hT x.

Show that T ∗ T −1 is unitary. T ∗ T = I ⇔ kT xk = kxk is also valid. Show that for real Hilbert spaces. (Hint: show hf (x + y).4. then T − λ = T (1 − λT ∗ ) and kλT ∗ k = |λ|kT ∗ k = |λ| < 1. In the proof of the equivalence of the first proposition of this section. Show that M ⊥ is also invariant. f (z)i = hf (x) + f (y). 4. 2. Show directly that eigenvalues of unitary operators must satisfy |λ| = 1. with continuous inverse.1 Exercises 1. preserve distances) on a Hilbert space must be of the type P (x) = Ux + a where U ∗ U = I.8 circle. T xi/kxk2 } and deduce that the spectrum of T is precisely the set of eigenvalues. 5. Let Tg be a group of unitary operators for g ∈ G a group. 6. Suppose M is Tg -invariant for all g ∈ G. Take |λ| < 1. ±i }. Deduce that isometries (i. Proof. are invertible. Show that T is unitary with eigenvalues { ±1. Let T : ℓ2 → ℓ2 be defined by T (xn ) = (in xn ). 7. Find the set { hx. f (z)i. so f (x+y)−f (x)− f (y) ∈ h im f i ∩ h im f i⊥ . Let T be an invertible normal operator. 3. Show that the right shift operator on ℓ2 satisfies R∗ R = I but is not unitary. and only if. Therefore (1 − λT ∗ ). Show that an operator is unitary if. The spectrum must lie in the unit ball since kT k = 1.e. use was made that the Hilbert space is over the complex field. J Muscat 32 The spectrum of a unitary operator lies in the unit σT ⊆ eiR . 8.  4. and so T − λ.) . it maps Hilbert bases to Hilbert bases.2 Unitary Operators Proposition 4.2. (*) Show that a function on a Hilbert space which preserves the innerproduct must be linear.

T yi−hy. Self-adjoint operators are obviously normal. T xi = hT ∗ x. Proposition 4.3. 4. and only if.e.10 Proof. Corollary The spectrum of a self-adjoint operator is real. B self-adjoint if. Proof. Hence hx. T xi is purely imaginary. then hx + y. yi. show that k(T − λ)xk > |β|kxk and hence that λ 6∈ σ(T ). By taking λ = α + iβ with β 6= 0 and expanding k(T − λ)xk.3 J Muscat 33 Self-Adjoint Operators Definition A self-adjoint operator is a continuous linear operator such that T ∗ = T .3 Self-Adjoint Operators 4. xi = hx. T is self-adjoint iff the set { hx. A = B = 0 (since then A = A∗ = (−iB)∗ = iB = −A. and T is normal if. R self-adjoint.9 Every operator in B(X) can be written uniquely as T = S + iR for S. Simply check that S = (T + T ∗ )/2 and R = (T − T ∗ )/2i are selfadjoint and add to T . and only if. xi = hT x. 2. Conversely. T xi is a real number. T yi + hy. starting with x + iy gives that hx. Prove that a continuous projection P is normal iff it is orthogonal iff it is self-adjoint. T is normal and S 2 + R@ = 1. and only if. if the given set is real. T xi/kxk2 } is real. Similarly. Thus T ∗ = T and T is self-adjoint. a normal operator whose spectrum is real must be self-adjoint. T xi/kxk2 : x ∈ X } is real. T yi implies that hx. Since T is self-adjoint. T is unitary if. T xi + hy. R = 0. and only if. T xi + hx.)  Note that in this case.1 Exercises 1. T ∗ = S − iR. T yi = hy. Proposition 4. T yi + hy. hx. T xi is real. T xi = hT x. T is self-adjoint if. Note that the converse is also true i. SR = RS. Hence the set { hx. T x + T yi = hx. Uniqueness follows from the fact that A + iB = 0 for A.4. .

4.n k + kTm (xn. In particular.n + Tm (xn. there is a convergent subsequence . . the non-zero part of the spectrum consists of a countable number of eigenvalues whose only possible limit point is 0. λ 6= 0. Show that in this case the map hhx.n ) + (Tm − T )xm. Each non-zero eigenvalue has a finitedimensional eigenspace.4 Compact Normal Operators Definition T is called a compact operator when for every bounded set B. there is a subsequence xn. if xn are unit vectors then T xn has a convergent subsequence. Now let xn be a bounded sequence. TN (an ) := (a1 . Compact operators are obviously continuous. then T is self-adjoint. .n − T xm. Show that hT u. Let λ ∈ σ(T ).4 Compact Normal Operators J Muscat 34 3. matrices T : CN → CM are compact since T B has to be finite-dimensional. such as ℓ2 . 4. Let T y = − p1 (py ′)′ + qy on the Hilbert space L2 [a.). TN is compact since its image is a subspace of CN . T yi defines an inner-product on X. Deduce that when we restrict to the subspace of those functions which also satisfy (i) the periodic condition y(a) = y(b). . . 2 Hence kT xn. 4. T vi + [−p(u′ v − uv ′)]ba . The map T : ℓ2 → ℓ2 defined by T (an ) := (an /n) is compact. A self-adjoint operator is called positive definite when hx.n k 6 k(T − Tn )xn. Example. T xi > 0 fora all x 6= 0. For example. Consider the linear maps TN : ℓ2 → ℓ2 .n − xm. 0.n k = k(T − Tn )xn.N k 6 N . For each N. Then there are unit vectors xn such that (T − λ)xn → 0. yii = hx. or (ii) the unmixed conditions α1 u(a) + α2 u′ (a) = 0. . say kxn kℓ2 6 M. a2 /2. aN /N.11 For T compact normal. the identity operator I is not compact (take any orthonormal sequence).n k →0 Thus every bounded sequence xn gives a convergent subsequence of T xn . But when T is compact.N − TN xm.n k + k(Tn − Tm )xn. But in infinite dimensions. vi = hu.n − xm. . Proof.N such that 1 kTN xn. hence compact. Theorem 4.n + (Tn − Tm )xn.n )k + k(Tm − T )xm. b]. T B is compact. so its closure must be closed and bounded. β1 u(b) + β2 u′ (b) = 0. .

From T en = λn en and the fact that T en → y for some subsequence. xien + v where v ∈ ker T . if λ has an infinite number of linearly independent eigenvectors. The only limit point must then be 0. and λ is an eigenvalue. choose a countable number of them en . that is the T restrictedPto M ⊥ is the zero operator. n n Thus compact normal operators are diagonalizable. . then their orthonormal eigenvectors would converge to en → y/λ. T y = λy. as T is normal. T restricted to the orthogonal space of M := [[en ]] is still compact normal. xiT en = λn hen . if λ 6= 0. make them orthonormal by the Gram-Schmidt algorithm. Since limits are unique. its eigenvectors en of distinct eigenvalues are orthogonal. Similarly. xien . and this furnishes another contradiction. we find X X Tx = hen . and there are an infinite number of distinct eigenvalues λn . and there can only be a countable number of non-zero eigenvalues. and these cannot have a convergent √ subsequence since ken − em k = 2. Yet it cannot have any non-zero eigenvalues because these are already accounted for in en . This implies λxni = T xni − (T − λ)xni → y so xni → y/λ and T xni → T y/λ. The bounded sequence λn must have a subsequence converging to some λ. n Proof. Its spectrum must therefore be { 0 }. a contradiction. (Note that y is not zero since xni are unit vectors.) Suppose there are an infinite number of unit eigenvectors en with nonzero eigenvalues λn . In terms of their representation in ℓ2 . Yet.4. Writing any vector x = n hen . Hence M ⊥ = ker T .4 Compact Normal Operators J Muscat 35 T xni → y.  Corollary Spectral Theorem for Compact Normal operators Every compact normal operator has a countable orthonormal set of eigenvectors en with non-zero eigenvalues λn . we get λn en → y. and X Tx = λn hen . every vector x ∼ (an ) is mapped to T x ∼ (λn an ). Keeping this in mind. xien .