18.

103 Fall 2013

Orthonormal Bases
Consider an inner product space V with inner product hf, gi and norm

kf k2 = hf, f i

Proposition 1 (Continuity) If kun − uk → 0 and kvn − vk → 0 as n → ∞, then

kun k → kuk; hun , vn i → hu, vi.

Proof. Note first that since kvn − vk → 0,

kvn k ≤ kvn − vk + kvk ≤ M < ∞

for a constant M independent of n. Therefore, as n → ∞,

|hun , vn i − hu, vi| = |hun − u, vn i + hu, vn − vi| ≤ M kun − uk + kukkvn − vk → 0

In particular, if un = vn , then kun k2 = hun , un i → hu, ui = kuk2 . 

For u and v in V we say that u is perpendicular to v and write u ⊥ v if hu, vi = 0. The
Pythogorean theorem says that if u ⊥ v, then

ku + vk2 = kuk2 + kvk2 (1)

Definition 1 ϕn is called an orthonormal sequence, n = 1, 2, . . . , if hϕn , ϕm i = 0 for n 6= m

and hϕn , ϕn i = kϕn k2 = 1.

Suppose that ϕn is an orthonormal sequence in an inner product space V . The following

four consequences of the Pythagorean theorem (1) were proved in class (and are also in the
text):
N
X
If h = an ϕn , then
n=1
N
X
2
khk = |an |2 . (2)
1

1
 N
X
If f ∈ V and sN = hf, ϕn iϕn , then
n=1

kf k2 = kf − sN k2 + ksN k2 (3)
If VN = span {ϕ1 , ϕ2 , . . . , ϕN }, then
kf − sN k = min kf − gk (best approximation property) (4)
g∈VN

If cn = hf, ϕn i, then
∞
X
2
kf k ≥ |cn |2 (Bessel’s inequality). (5)
n=1

Definition 2 A Hilbert space is defined as a complete inner product space (under the dis-
tance d(u, v) = ku − vk).

Theorem 1 Suppose that ϕn is an orthonormal sequence in a Hilbert space H. Let

∞
[
VN = span {ϕ1 , ϕ2 , . . . , ϕN }, V = VN
N =1

(V is the vector space of finite linear combinations of ϕn .) The following are equivalent.
a) V is dense in H (with respect to the distance d(f, g) = kf − gk),
b) If f ∈ H and hf, ϕn i = 0 for all n, then f = 0.
N
X
c) If f ∈ H and sN = hf, ϕn iϕn , then ksN − f k → 0 as N → ∞.
n=1

d) If f ∈ H, then
∞
X
2
kf k = |hf, ϕn i|2
n=1

If the properties of the theorem hold, then {ϕn }∞

n=1 is called an orthonormal basis or complete
orthonormal system for H. (Note that the word “complete” used here does not mean the
same thing as completeness of a metric space.)
Proof. (a) =⇒ (b). Let f satisfy hf, ϕn i = 0, then by taking finite linear combinations,
hf, vi = 0 for all v ∈ V . Choose a sequence vj ∈ V so that kvj − f k → 0 as j → ∞. Then
by Proposition 1 above
0 = hf, vj i → hf, f i =⇒ kf k2 = 0 =⇒ f = 0

2
 N
X
(b) =⇒ (c). Let f ∈ H and denote cn = hf, ϕn i, sN = cn ϕn . By Bessel’s inequality
1
(5),
∞
X
|cn |2 ≤ kf k2 < ∞.
1

Hence, for M < N (using (2))

N 2 N
X X
2
ksN − sM k = cn ϕ n = |cn |2 → 0 as M, N → ∞.


M +1 M +1

In other words, sN is a Cauchy sequence in H. By completeness of H, there is u ∈ H such

that ksN − uk → 0 as N → ∞. Moreover,

hf − sN , ϕn i = 0 for all N ≥ n.

Taking the limit as N → ∞ with n fixed yields

hf − u, ϕn i = 0 for all n.

Therefore by (b), f − u = 0.
(c) =⇒ (d). Using (3) and (2),
N
X
2 2 2 2
kf k = kf − sN k + ksN k = kf − sN k + |cn |2 , (cn = hf, ϕn i)
1

Take the limit as N → ∞. By (c), kf − sN k2 → 0. Therefore,

∞
X
2
kf k = |cn |2
1

Finally, for (d) =⇒ (a),

N
X
2 2
kf k = kf − sN k + |cn |2
1

Take the limit as N → ∞, then by (d) the rightmost term tends to kf k2 so that kf −sN k2 →
0. Since sN ∈ VN ⊂ V , V is dense in H. 

3
Proposition 2 Let ϕn be an orthonormal sequence in a Hilbert space H, and
X X
|an |2 < ∞, |bn |2 < ∞

then ∞ ∞
X X
u= an ϕn , v= bn ϕn
n=1 n=1

are convergent series in H norm and

∞
X
hu, vi = an b n (6)
n=1

Proof. Let
N
X N
X
uN = an ϕn ; vN = bn ϕ n .
1 1

Then for M < N ,

N
X
2
kuN − uM k = |an |2 → 0 as M → ∞
M

so that uN is a Cauchy sequence converging to some u ∈ H. Similarly, vN → v in H norm.

Finally,
N
X N
X N
X
huN , vN i = haj ϕj , bk ϕk i = aj bk hϕj , ϕk i = aj b j
j,k=1 j,k=1 j=1

since hϕj , fk i = 0 for j 6= k and hfj , fj i = 1. Taking the limit as N → ∞ and using the
continuity property (1), huN , vN i → hu, vi, gives (6). 
If H is a Hilbert space and {ϕn }∞
n=1 is an orthonormal basis, then every element can be
written ∞
X
f= an ϕn (series converges in norm)
n=1

The mapping X
{an } 7→ an ϕn
n

is a linear isometry from `2 (N) to H that preserves the inner product. The inverse mapping
is
f 7→ {an } = {hf, ϕn i}

4
 It is also useful to know that as soon as a linear mapping between Hilbert spaces is an
isometry (preserves norms of vectors) it must also preserve the inner product. Indeed, the
inner product function (of two variables u and v) can be written as a function of the norm
function (of linear combinations of u and v). This is known as polarization:
Polarization Formula.

hu, vi = a1 ku + ivk2 + a2 ku + vk2 + a3 kuk2 + a4 kvk2 (7)

with
a1 = i/2, a2 = 1/2, a3 = −(1 + i)/2, a4 = −(i + 1)/2
Proof.

ku + ivk2 = hu + iv, u + ivi

= kuk2 + hiv, ui + hu, ivi + kvk2
= kuk2 + i(hv, ui − hu, vi) + kvk2

Similarly,
ku + vk2 = kuk2 + (hv, ui + hu, vi) + kvk2
Multiplying the first equation by i and adding to the second, we find that

iku + ivk2 + ku + vk2 = (i + 1)kuk2 + 2hu, vi + (i + 1)kvk2

Solving for hu, vi yields (7).