CHAPTER 9

Inner Product Spaces and Hilbert Spaces

BY

Prof. M. Saha

Professor of Mathematics
The University of Burdwan
West Bengal, India
E-mail : mantusaha.bu@gmail.com
Chapter 9 Inner Product Spaces and Hilbert Spaces

Introduction and Objectives

In the preceding chapters, we discussed normed linear spaces and Banach spaces. These
spaces has linear properties as well as metric properties. Although the norm on a
linear space generalizes the elementary concept of the length of a vector, but the main
geometric concept other than the length of a vector is the angle between two vectors, In
this chapter, we take the opportunity to study linear spaces having an inner product, a
generalization of the usual dot product on finite dimensional linear spaces. The concept
of an inner product in a linear space leads to an inner product space and a complete
inner product space which is called a Hilbert space. The theory of Hilbert Spaces does
not deal with angles in general. Most interestingly, it helps us to introduce an idea of
perpendicularity for two vectors and the geometry deals in various fundamental aspects
with Euclidean geometry.
The basics of the theory of Hilbert spaces was given by in 1912 by the work of
German mathematician D. Hilbert (1862 -1943) on integral equations. However, an
axiomatic basis of the theory was given by famous mathematician J. Von Neumann
(1903 -1957). However, Hilbert spaces are the simplest type of infinite dimensional
Banach spaces to tackle a remarkable role in functional analysis.
This chapter is divided into four modules.
Module 1 ideals with the introduction of inner product spaces and Hilbert spaces
with some examples. Some basic properties are given in it.
Module 2 is concerned with the conception of orthogonal and orthonormal vectors in
a Hilbert space. Some basic result are shown..
Module 3 is devoted mainly with very fundamental results on inner product spaces.
Module 4 is discussed on some fundamental results on the Hilbert spaces particularly
on approximation theory of convex subsets of a Hilbert spaces.
Module 5 is emphasized on infinite series are those convergence in Hilbert spaces.
Also with an introduction of isometric isomorphism property between two infinite di-
mensional Hilbert spaces an important theorem namely Reisz Fischer’s Theorem has
been stated.

2
 Module-1: Inner Product Spaces

Definition 9.1.1: Let, X be a linear space over a field of complex numbers. If for
every pair (x, y) ∈ X × X there corresponds a scalar denoted by ⟨x, y⟩ called inner
product of x and y of X such that the following properties hold.

(IP.1) ⟨x, y⟩ = ⟨y, x⟩ where (x, y) ∈ X × X and ‘ ’ denotes the conjugate of the
complex number.

(IP.2) for all α ∈ C,

⟨αx, y⟩ = α⟨x, y⟩, ∀ (x, y) ∈ X × X.

(IP.3) for all x, y, z ∈ X

⟨x + y, z⟩ = ⟨x, z⟩ + ⟨y, z⟩.

(IP.4) ⟨x, x⟩ ≥ 0 and ⟨x, x⟩ = 0 iff x = θ.

Then (X, ⟨ . ⟩) is called an inner product space or pre-Hilbert space.

Remark 9.1.1: The following properties hold in an inner product space.

Let X be an inner product space then,

(i) for all α, β ∈ C, ⟨αx + βy, z) = α⟨x, z⟩ + β⟨y, z⟩ ∀ x, y, z ∈ X.

(ii) for all α ∈ C ⟨x, αy⟩ = ᾱ⟨x, y⟩ ∀ x, y ∈ X

(iii) for all α, β ∈ C ∀ x, y, z ∈ X.

⟨x, αy + βz⟩ = ᾱ⟨x, y⟩ + β̄⟨x, z⟩.

3
Chapter 9 Inner Product Spaces and Hilbert Spaces

Remark 9.1.2: Inner product induces a norm.

For this we proceed as follows:

Proof: Let X be an inner product space. Take x ∈ X

√
Define ||x|| = + ⟨x, x⟩ (9.1.1)

Now, ||x|| ≥ 0 as ⟨x, x⟩ ≥ 0 by (IP.4).

Also, ||x|| = 0 iff x = θ by (IP.4)
Take α ∈ C so,

||αx||2 = ⟨αx, αx⟩, α ∈ X

= αᾱ⟨x, x⟩, x ∈ X
= |α|2 ||x||2 , x ∈ X
so ||αx|| = = |α| ||x||, x ∈ X

To prove the triangle inequality we first state and prove Cauchy Schwarz Inequality.

Cauchy Schwarz Inequality:

|⟨x, y⟩| ≤ ||x|| ||y|| ∀ x, y ∈ X (9.1.2)

Proof: If y = θX then the result follows trivially.

Let y ̸= θX . Then for every scalars λ ∈ C,

⟨x + λy, x + λy⟩ ≥ 0
=⇒ ⟨x, x⟩ + ⟨x, λy⟩ + ⟨λy, x⟩ + ⟨λy, λy⟩ ≥ 0 (By (IP.3))
=⇒ ⟨x, x⟩ + λ̄⟨x, y⟩ + λ⟨y, x⟩ + ||λy||2 ≥ 0
=⇒ ||x||2 + λ̄⟨x, y⟩ + λ⟨x, y⟩ + |λ|2 ||y||2 ≥ 0
⟨x, y⟩
Take λ=−
⟨y, y⟩
⟨x, y⟩⟨x, y⟩ ⟨x, y⟩⟨x, y⟩ |⟨x, y⟩|2
So, ||x||2 − − + ||y||2 ≥ 0.
||y|| 2 ||y||2 ||y||4

|⟨x, y⟩| 2
|⟨x, y⟩| 2
|⟨x, y⟩| 2
So, ||x||2 − − + ≥0
||y|| 2 ||y||2 ||y||2

4
Chapter 9 Inner Product Spaces and Hilbert Spaces

|⟨x, y⟩|2
So ||x∥2 − ≥0
||y||2
=⇒ ||x||2 ||y||2 ≥ |⟨x, y⟩|2
=⇒ ||x|| ||y|| ≥ |⟨x, y⟩|
So |⟨x, y⟩| ≤ ||x|| ||y||

Sometimes this inequality is abbreviated as C-S inequality. We see that equality sign
will hold if and only if in above derivation ⟨x + λy, x + λy⟩ = 0 =⇒ ||x + λy||2 =
0 =⇒ x + λy = θX , i.e x and y are linearly dependent.
We shall now prove triangle inequality for norm. Now ∀ x, y ∈ X.

Now, ||x + y||2 = ⟨x + y, x + y⟩

= ⟨x, x⟩ + ⟨x, y⟩ + ⟨y, x⟩ + ⟨y, y⟩
= ||x||2 + ||y||2 + ⟨x, y⟩ + ⟨y, x⟩
 
 
So, ||x + y||2 = ||x||2 + ||y||2 + ⟨x, y⟩ + ⟨y, x⟩
 
 
≤ ||x||2 + ||y||2 + ⟨x, y⟩ + ⟨y, x⟩
≤ ||x||2 + ||y||2 + 2||x|| ||y||
( )2
= ||x|| + ||y||
So, ||x + y|| ≤ ||x|| + ||y||.

Hence, inner product induces a norm and consequently every inner product space is a
normed linear space.

Remark 9.1.3: So every inner product space is a metric space and the metric induced
by inner product is defined as follows:
for all x, y ∈ X define d : X × X → R by
√
d(x, y) = ||x − y|| = + ⟨x − y, x − y⟩ (9.1.3)

Theorem 9.1.1: Every inner product function is a continuous function.

(Equivalently, if f : X × X → C defined by f (x, y) = ⟨x, y⟩, ∀ x, y ∈ X then f is
continuous).

Proof: Let X be an inner product space. Define f : X × X → C by f (x, y) =

⟨x, y⟩, ∀ x, y ∈ X. Now take {xn } and {yn } be a sequence in X such that xn → x as

5
Chapter 9 Inner Product Spaces and Hilbert Spaces

n → ∞ and yn → y as n → ∞.
So, ||xn − x|| → 0 as n → ∞ and ||yn − y|| → 0 as n → ∞.
As, xn → x as n → ∞ then, ||xn || → ||x|| as n → ∞.
So, {||xn ||} are bounded. So, there exists a constant M > 0 such that
||xn || ≤ M, ∀ n.

Now, |⟨xn , yn ⟩ − ⟨x, y⟩|

= |⟨xn , yn ⟩ − ⟨xn , y⟩ + ⟨xn , y⟩ − ⟨x, y⟩|
= |⟨xn , yn − y⟩ + ⟨xn − x, y⟩|
≤ |⟨xn , yn − y⟩| + |⟨xn − x, y⟩
≤ ||xn || ||yn − y|| + ||xn − x|| ||y|| [By C-S inequality (9.1.2)]
≤ M ||yn − y|| + ||xn − x|| ||y||
→ 0 as n → ∞

i.e ⟨xn , yn ⟩ → ⟨x, y⟩ as n → ∞, implying that f (xn , yn ) → f (x, y) as n → ∞. So, f is

continuous.

Theorem 9.1.2 (Parallelogram Law): Let X be an inner product space and let
x, y ∈ X. Then,
( )
||x + y||2 + ||x − y||2 = 2 ||x||2 + ||y||2

Proof:

||x + y||2 = ⟨x + y, x + y⟩ = ⟨x, x⟩ + ⟨x, y⟩ + ⟨y, x⟩ + ⟨y, y⟩

= ||x||2 + ||y||2 + ⟨x, y⟩ + ⟨y, x⟩ (9.1.4)
and ||x − y||2 = ⟨x − y, x − y⟩ = ⟨x, x⟩ + ⟨x, −y⟩ + ⟨−y, x⟩ + ⟨−y, −y⟩
= ||x||2 + ||y||2 − ⟨x, y⟩ − ⟨y, x⟩ (9.1.5)

Adding (9.1.4) and (9.1.5) we get

( )
||x + y||2 + ||x − y||2 = 2 ||x||2 + ||y||2

Theorem 9.1.3 (Polarization Identity): Let X be an inner product space, let x, y ∈

X. Then
1[ ]
⟨x, y⟩ = ||x + y|| − ||x − y|| + i||x − iy|| − i||x − iy||
2 2 2 2
(9.1.6)
4

6
Chapter 9 Inner Product Spaces and Hilbert Spaces

Proof:

Now, ||x + y||2 = ||x||2 + ||y||2 + ⟨x, y⟩ + ⟨y, x⟩ (9.1.7)

||x − y||2 = ||x||2 + ||y||2 − ⟨x, y⟩ − ⟨y, x⟩ (9.1.8)

Replacing y by iy in (9.1.7) and (9.1.8)

||x + iy||2 = ||x||2 + ||iy||2 + ⟨x, iy⟩ + ⟨iy, x⟩

= ||x||2 + ||y||2 − i⟨x, y⟩ + i⟨y, x⟩ (9.1.9)
||x − iy||2 = ||x||2 + ||iy||2 − ⟨x, iy⟩ − ⟨iy, x⟩
= ||x||2 + ||y||2 + i⟨x, y⟩ − i⟨y, x⟩ (9.1.10)

(9.1.7) − (9.1.8) + i(9.1.9) −i(9.1.10), we get (9.1.6). Hence the result.

Theorem 9.1.4: Let X be an inner product space. Then

(i) Every Cauchy sequence is bounded

(ii) If {xn } and {yn } are two Cauchy sequences in X then {⟨xn , yn ⟩} is also a Cauchy
sequence in C and hence convergence in C.

Proof: (i) Let, {xn } be a Cauchy sequences in X. Then for ε = 1 there exists a
positive integer N such that, ||xn − xm || < 1, whenever, n, m ≥ N . In particular,
||xn − xN || < 1, whenever n ≥ N .
Now, ||xn || ≤ ||xn − xN || + ||xN || < 1 + ||xN || ∀ n ≥ N .
{ }
Let, M = max ||x1 ||, ||x2 ||, · · · , ||xN −1 ||, ||xN || + 1 so, ||xn || ≤ M ∀ n so, {xn } is
bounded.
(ii) Let {xn }, {yn } be two Cauchy sequences in X.
So, ||xn − xm || → 0 as n, m → ∞ and ||yn − ym || → 0 as n, m → ∞.
Also, by (i) ||xn || ≤ M for all n and for some M > 0. Similarly ||yn || ≤ K, for some
K > 0 and ∀ n.

Now, |⟨xn , yn ⟩ − ⟨xm , ym ⟩|

= |⟨xn , yn ⟩ − ⟨xn , ym ⟩ + ⟨xn , ym ⟩ − ⟨xm , ym ⟩|
= |⟨xn , yn − ym ⟩ + ⟨xn − xm , ym ⟩|
≤ |⟨xn , yn − ym ⟩| + |⟨xn − xm , yn ⟩|

7
Chapter 9 Inner Product Spaces and Hilbert Spaces

≤ ||xn || ||yn − ym || + ||xn − xm || ||yn || (By C-S inequality)

≤ M ||yn − ym || + K||xn + xm ||
→0 as n, m → ∞.

So, {⟨xn , yn ⟩} is a Cauchy sequences of scalars in C. As C is complete, {⟨xn , yn ⟩} is

convergent in C.

Definition 9.1.2: A complete inner product space is called a Hilbert space i.e. an
inner product space X which is complete with respect to a metric d : X × X → R
induced by the inner product ⟨ , ⟩ on X × X i.e. d(x, y) = ⟨x − y, x − y⟩1/2 ∀ x, y ∈ X.

Theorem 9.1.5: A Banach space X is a Hilbert space if and only if parallelogram law
holds in it.

Proof: We know that every Hilbert space X is a Banach space where parallelogram
law holds in it.
Conversely suppose that X is a Banach space where parallelogram law holds. With-
out loss of generality we can assume a function ⟨, ⟩ whose range is R. For all x, y ∈ X.
Define ⟨, ⟩ : X × X → R by
1[ ]
⟨x, y⟩ = ||x + y||2 − ||x − y||2 (9.1.11)
4
i.e. for real inner product space we start with (9.1.11) and sometimes we write R⟨x, y⟩ =
[ ]
1
4
||x + y||2
− ||x − y||2
, ∀ x, y ∈ X. Clearly ⟨x, y⟩ = ⟨y, x⟩ as ⟨x, y⟩ is real. Also
⟨x, x⟩ ≥ 0 ∀ x ∈ X and ⟨x, x⟩ = 0 iff x = 0. So, (IP.1) and (IP.4) holds.
¯
Now, for u, v, w ∈ X
( )
||u + v + w||2 + ||u + v − w||2 = 2 ||u + v||2 + ||w||2 (9.1.12)
( )
||u − v + w||2 + ||u − v − w||2 = 2 ||u − v||2 + ||w||2 (9.1.13)

By (9.1.12) − (9.1.13) we get

||u + v + w||2 + ||u + v − w||2 − ||u − v + w||2 + ||u − v − w||2

( )
= 2 ||u + v||2 − ||u − v||2
[ ]
=⇒ 4 ⟨u + w, v⟩ + ⟨u − w, v⟩ = 2.4⟨u, v⟩
=⇒ ⟨u + w, v⟩ + ⟨u − w, v⟩ = 2⟨u, v⟩ (9.1.14)
Put, u=w
⟨2u, v⟩ = 2⟨u, v⟩ (9.1.15)

8
Chapter 9 Inner Product Spaces and Hilbert Spaces

Again put, x1 = u + w, x2 = u − w, x3 = v then from (9.1.14) we get

⟨x1 , x3 ⟩ + ⟨x2 , x3 ⟩ = 2⟨u, v⟩ = ⟨2u, v⟩ (by (9.1.15))

= ⟨x1 + x2 , x3 ⟩ (9.1.16)

So, (IP.3) holds.

From (9.1.16) for x1 , x2 , x3 , x4 ∈ X,

⟨x1 + x2 + x3 , x4 ⟩ = ⟨x1 + x2 , x4 ⟩ + ⟨x3 , x4 ⟩ = ⟨x1 , x4 ⟩ + ⟨x2 , x4 ⟩ + ⟨x3 , x4 ⟩

Put x1 = x2 = x3 = x, x4 = y i.e. ⟨3x, y⟩ = 3⟨x, y⟩.

So, by Principle of Mathematical Induction for any positive integer n

⟨nx, y⟩ = n⟨x, y⟩ ∀ x, y ∈ X (9.1.17)

1[ ]
Now, ⟨−x, y⟩ = || − x + y||2 − || − x − y||2
4
1[ ]
= − ||x + y||2 + ||y − x||2 = −⟨x, y⟩ (9.1.18)
4

Take, n = −m (m > 0)

⟨nx, y⟩ = ⟨−mx, y⟩ = ⟨m(−x), y⟩

= m⟨−x, y⟩ (by (9.1.17))
= −m⟨x, y⟩ (by (9.1.18))
= n⟨x, y⟩

So (9.1.17) is also true for any negative integer. Thus ⟨λx, y⟩ = λ⟨x, y⟩, when λ is
either positive integer or a negative integer.
Take λ = p/q = a rational number where,, gcd(p, q) = 1 and p and q are integer.
⟨ ⟩
p
Now ⟨λx, y⟩ = x, y
q
⟨ ⟩
p
So q⟨λx, y⟩ = q x, y = ⟨px, y⟩ = p⟨x, y⟩
q
p
=⇒ ⟨λx, y⟩ = ⟨x, y⟩
q
⟨ ⟩
p p
=⇒ x, y = ⟨x, y⟩ (9.1.19)
q q

9
Chapter 9 Inner Product Spaces and Hilbert Spaces

Let, λ be any real. So there exists a sequence of rationals {rn } such that rn → λ as
n → ∞. So, ⟨rn x, y⟩ = rn ⟨x, y⟩ → λ⟨x.y⟩ as n → ∞

Now, |⟨rn x, y⟩ − ⟨λx, y⟩|

= |⟨(rn − λ)x, y⟩| ≤ ||(rn − λ)x|| ||y|| (by C-S inequality (9.1.2))
= |rn − λ| ||x|| ||y|| → 0 as n → ∞

⟨rn x, y⟩ → ⟨λx, y⟩ as n → ∞. So, ⟨λx, y⟩ = λ⟨x, y⟩ ∀ x, y ∈ X. So, (IP.2) holds.

So, X is an inner product space with respect to (9.1.11) consequently, X is a Hilbert
space.

Note: For a complete inner product space we start with Rl⟨x, y⟩ = 41 [||x + y||2 − ||x − y||2 ],
Im⟨x, y⟩ = 1
4
[||x + iy||2 − ||x − iy||2 ] , ∀x, y ∈ X and the proof is similar to the proof
of real inner product space. (Readers can verify it)

Example 9.1.1: The Euclidean space Rn is a Hilbert space.

Solution: Let x = (ζ1 , ζ2 , · · · , ζn ) and y = (η1 , η2 , · · · , ηn ) be two element of Rn .

We define the inner product of x and y by (x, y) = ζ1 η1 + ζ2 η2 + · · · + ζn ηn . Then
√ 1
||x|| = ⟨x, x⟩ = (ζ12 + ζ22 + · · · + ζn2 ) 2 . It may be easily verified that all the inner
product axioms are satisfied √ in Rn and the Euclidean metric d is obtained by d(x, y) =
1 ∑n
||x − y|| = ⟨x − y, x − y⟩ 2 = (ζi − ηi )2 . With respect to this metric we can at once
i=1
see that Rn is complete so as to make Rn , a Hilbert space.

Example 9.1.2: The Euclidean space Cn is a Hilbert space.

Solution: Let x = (ζ1 , ζ2 , · · · , ζn ) and y = (η1 , η2 , · · · , ηn ) be two elements of Cn . We

define the inner product of x and y by

⟨x, y⟩ = ζ1 η1 + ζ2 η2 + · · · + ζn ηn .
√ 1
Then ||x|| = ⟨x, x⟩ = (|ζ1 |2 + |ζ2 |2 + · · · + |ζn |2 ) 2 . It may be easily verified that all
the inner product axioms are satisfied in√Cn and the Euclidean metric d is obtained by
1 ∑n
d(x, y) = ||x − y|| = ⟨x − y, x − y⟩ 2 = (ζi − ηi )2 . With respect to this metric we
i=1
can at once see that Cn is complete so as to make Cn , a Hilbert space.

Example 9.1.3: The space l2 is a Hilbert space.

10
Chapter 9 Inner Product Spaces and Hilbert Spaces

Solution: Let x = {ζi } and y = {ηi } be elements of l2 . We define the inner product
∑∞
of x and y by ⟨x, y⟩ = ζi ηi convergence of the series on the right hand side follows
i=1
from the fact that x ∈ l2 and |ζi ηi | ≤ |ζ2i | + |η2i | .
2 2

( ∞ ) 12
√ ∑
Then ||x|| = ⟨x, x⟩ = ζi . It can be easily shown that all the inner product
i=1
axioms (IP.1) -(IP.4) are satisfied in l2 . The metric d of l2 is defined by d(x, y) =
(n ) 21
1 ∑
||x − y|| = ⟨x − y, x − y⟩ 2 = |ζi − ηi |2
. With respect to this metric we can at
i=1
once see that l2 is complete so as to make l2 a Hilbert space.
But for 1 ≤ p < ∞, lp (p ̸= 2 is not a Hilbert space. It can be shown by the
following example.

Example 9.1.4: For 1 ≤ p < ∞, lp (p ̸= 2) is not an inner product space and hence
not a Hilbert space.

Solution: Let x = (1, 1, 0, 0, · · · ) ∈ lp and y = (1, −1, 0, 0, · · · ) ∈ lp . Then ||x|| =

1
||y|| = 2 p and ||x + y|| = ||x − y|| = 2. Now we see that if p ̸= 2, the parallelogram law
does not hold.
Hence 1 ≤ p < ∞ lp (p ̸= 2) is not an inner product space and consequently it is not
a Hilbert space.

Example 9.1.5: The space c[a, b] of all real valued continuous in the closed interval
[a, b] is not an inner product space with respect to sup norm and hence not a Hilbert
space.

Solution: Here the norm defined by ||x|| = sup |x(t)|. Take x(t) = 1, ∀ t ∈ [a, b] and
a≤t≤b
y(t) = t−a
b−a
, ∀ t ∈ [a, b]. Then ||x|| = 1, ||y|| = 1, ||x + y|| = 2, ||x − y|| = 1. By simple
calculations we see that parallelogram law does not hold in it. Hence c[a, b] is not a
Hilbert space.

Example 9.1.6: The space L2 [a, b], the space of all square integrable functions over
[a, b] is a Hilbert space.
∫b
Solution: Define the inner product on L2 [a, b] by ⟨x, y⟩
√∫ = a
|x(t)y(t)| dt, ∀ x, y ∈
b
L2 [a, b] and the norm on L2 [a, b] is given by ||x|| = z
|x(t)|2 dt. Also with respect

11
Chapter 9 Inner Product Spaces and Hilbert Spaces

to this norm it can be shown that L2 [a, b] is complete with respect to a metric defined
by
[∫ b ] 12
d(x, y) = |x(t) − y(t)| dt
2
.
a

So L2 [a, b] is a Hilbert space.

12