by

Assistant Professor

Department of Mathematics
SHRI MATA VAISHNO DEVI UNIVERSITY
Email:

February 27, 2018 1 / 23

Orthogonal complements and Projection theorem:

Definitions:
1. Two vectors x and y in an inner product space are called orthogonal if
hx, y i = 0 and denoted by x ⊥ y .
2. A vector x of an inner product space X is called orthogonal to a non-empty
subset A of X , denoted by x ⊥ A, if hx, y i = 0, ∀y ∈ A.
3. Let A be a non-empty subset of an inner product space X . The set of all
vectors orthogonal to A, denoted by A⊥ is called the orthogonal complement
of A. It is clear that A⊥ = {x ∈ X : hx, y i = 0 for each y ∈ A}.
A⊥⊥ = (A⊥ )⊥ will denote orthogonal complement of A⊥ .
4. Two subsets A and B of an inner product space X are called orthogonal
denoted by A ⊥ B if hx, y i = 0, ∀x ∈ A and y ∈ B.

February 27, 2018 2 / 23

 Remarks:
1. Since hx, y i = hy , xi, hx, y i = 0 implies that hy , xi = 0 or hy , xi = 0 and
vice-versa. Hence x ⊥ y if and only if y ⊥ x.
2. x ⊥ 0 for every x belonging to an inner product space, then 0 is the only
vector orthogonal to itself.
3. It is clear that {0}⊥ = X and X ⊥ = {0}.
4. It is clear that if A ⊥ B then A ∩ B = {0}.
5. Non-zero orthogonal vectors x1 , x2 , . . . , xn of an inner product space are
linearly independent.

Theorem: Let X be an inner product space and A be its arbitrary subset.

Then the following results holds good:
(i) A⊥ is a closed subspace of X .
(ii) A ∩ A⊥ ⊂ {0}. A ∩ A⊥ = {0} iff A is a subspace.
(iii) A ⊂ A⊥⊥ .
(iv) If B ⊂ A, then B ⊥ ⊃ A⊥ .
February 27, 2018 3 / 23
 Proof:
(i) Let x, y ∈ A⊥ . Then hx, zi = 0, ∀z ∈ A and hz, y i = 0, ∀z ∈ A. Since
for arbitrary scalars α, β, hαx + βy , zi = αhx, zi + βhy , zi, by definition.
We get hαx + βy , zi = 0 i.e., αx + βy ∈ A⊥ . So A⊥ is a subspace of X .
For showing that A⊥ is closed, let {xn } ⊂ A⊥ and xn → y . We are required
to show that y must belongs to A⊥ . By definition of A⊥ , for every x ∈ X ,
hx, xn i = 0, ∀n. This implies that lim hx, xn i = lim hxn , xi = 0. Since
n→∞ n→∞
h·, ·i is a continuous function, hlim xn , xi = 0. Therefore hy , xi = 0 hence
n
y ∈ A⊥ .
(ii) If y ∈ A ∩ A⊥ , i.e., y ∈ A and y ∈ A⊥ by previous remark, y = 0
i.e., y ∈ {0}. If A is a subspace, then 0 ∈ A and 0 ∈ A ∩ A⊥ . Hence
A ∩ A⊥ = {0}.
(iii) Let y ∈ A, but y ∈/ A⊥⊥ . Then there exists an element z ∈ A⊥ such
6 0. Since z ∈ A⊥ , hz, y i = 0 which is a contradiction. Hence
that hy , zi =
y ∈A . ⊥⊥

February 27, 2018 4 / 23

(iv) Let y ∈ A⊥ then hy , zi = 0, ∀z ∈ A. Since every z ∈ B is an element
of A, we have hy , zi = 0, ∀z ∈ B. Hence y ∈ B ⊥ and so B ⊥ ⊃ A⊥ .

Definition: The angle θ between two vectors x and y of an inner product

space X is defined by the relation

hx, y i
cos θ = , . . . . . . . . . (1)
kxkky k

Remarks:
1. By the C-S-B inequality the right hand side of (1) less than or equal to
1, and so the angle θ is well defined, 0 ≤ θ ≤ π for every x and y different
from zero.
2. If X = R3 , x = (x1 , x2 , x3 ), y = (y1 , y2 , y3 ), then
x1 y1 + x2 y2 + x3 y3
cos θ = .
(x12 + x22+ x32 )1/2 (y12 + y22 + y32 )1/2

This is a well known relation in three dimension Euclidean space.

February 27, 2018 5 / 23
3. If x ⊥ y , then cos θ = 0 i.e., θ = π2 . In view of this, orthogonal vectors
are also called perpendicular vectors.

Note: A well-known result of plane geometry is that the sum of the squares
of the base and perpendicular in a right angled triangle is equal to the
square of the hypotenuse. This is known as Pythagorean theorem. Its
infinite dimensional analysis is as follows:

Theorem: Let X be an inner product space and x, y ∈ X . Then for x ⊥ y ,

we have

kx + y k2 = kxk2 + ky k2 .

Proof: kx + y k2 = hx + y , x + y i = hx, xi + hy , xi + hx, y i + hy , y i. Since

x ⊥ y , hx, y i = 0 and hy , xi = 0. Hence

kx + y k2 = kxk2 + ky k2 .

February 27, 2018 6 / 23

 Note: We recall that an algebraic projection on a vector space X is a linear
operator P on X into itself such that P 2 = P. The geometrical significance
of this concept is as follows:

1. An algebraic projection determines a pair of subspaces M and N such that

X = M ⊕N, where M = {P(x) ∈ X : x ∈ X } and N = {x ∈ X : P(x) = 0}
are the range and null space of P.
2. A pair of subspaces M and N such that such that X = M ⊕ N determine
an algebraic projection P whose range and null space are M and N.
Thus, the study of algebraic projection is equivalent to the study of pairs of
subspaces which are disjoint and span X .

Projection Theorem:
If M is a closed subspace of a Hilbert space X , then

X = M ⊕ M ⊥. . . . . . . . . . (1)

Proof: Later on.

February 27, 2018 7 / 23

 Remarks:

1. The above theorem implies that a Hilbert space is always rich in projec-
tions. In fact, for every closed subspace M of a Hilbert space X , there exists
a top projection on X whose range is M and whose null space is M ⊥ .
2. Equation (1) means that every z ∈ X is expressible uniquely in the form
z = x + y , where x ∈ M and y ∈ M ⊥ . Since M ∩ M ⊥ = {0}, equation (1)
is called the orthogonal decomposition of Hilbert space X .
3. Let x = R2 , x ∈ R2 , x = y + z, y ∈ M and z ∈ M ⊥ then
(i) Geometrical imaging of orthogonal decomposition is shown in fig1.
(ii) Projection theorem is not valid for inner product spaces.

February 27, 2018 8 / 23

February 27, 2018 9 / 23
We require the following results in the proof of the theorem.
Lemma 1. Let M be a closed convex subset of a Hilbert space X and
` = inf kxk. Then there exists x ∈ M such that kxk = `.
x∈M
Lemma 2. Let M be a closed subspace of a Hilbert space X , let x ∈
/ M
and let the distance between x and M be ` i.e., ` = inf kx − uk. Then
u∈M
there exists a unique vector w ∈ M such that kx − w k = `.
Lemma 3. If M is a proper closed subspace of a Hilbert space X , then
there exists a non-zero vector u ∈ X such that u ⊥ M.
Lemma 4. If M and N are closed subspaces of a Hilbert space X such that
M ⊥ N, the subspace M + N = {x + y : x ∈ M and y ∈ N} is also closed.
Remark: Lemma 1. is true in the following general form. Let M be closed
convex subset of Hilbert space X and for x ∈ X , let ` = inf kx − uk. Then
u∈M
there exists a unique element w ∈ M such that ` = kx − w k. w is called
the projection of x on M and we write Px = w .

February 27, 2018 10 / 23

Proof of Theorem:
Since M is a closed subspace of X , and by a previous theorem M ⊥ is also
a closed subspace of X . By choosing N = M ⊥ in Lemma 4. we find that
M +M ⊥ is a closed subspace of X . First we want to show that X = M +M ⊥ .
Let X 6= M + M ⊥ i.e., M + M ⊥ is a proper closed subspace of X . Then
by Lemma3., there exist a non-zero vector u such that u ⊥ M + M ⊥ . This
implies that hu, x + y i = 0, ∀x ∈ M and y ∈ M ⊥ . If we choose y = 0, then
hu, xi = 0, ∀x ∈ M i.e., u ∈ M ⊥ . On the other hand, if we choose x = 0,
then hu, y i = 0, ∀y ∈ M ⊥ , i.e., u ∈ M ⊥⊥ (since M and M ⊥ are subspaces
this choice is possible). Thus, u ∈ M ⊥ ∩ M ⊥⊥ . By previous theorem we
obtain u = 0. This is a contradiction as u 6= 0. Hence our supposition is
false and X = M + M ⊥ . Hence the theorem is proved.
Remark: Let M be a closed subspace of a Hilbert space X , then M = M ⊥⊥ .
(Proof later on).

February 27, 2018 11 / 23

 Applications of above theorem:

1. Let x = L2 (−1, 1). Then X = M ⊕ M ⊥ , where

M = {f ∈ L2 (−1, 1) : f (−t) = f (t), ∀t ∈ (−1, 1), space of even functions}
and M ⊥ = {f ∈ L2 (−1, 1) : f (−t) = −f (t), ∀t ∈ (−1, 1), space of odd
functions}.
2. Let X = L2 [a, b]; for c ∈ [a, b], let
M = {f ∈ L2 (a, b) : f (t) = 0 almost everywhere in (a, c)}.
and M ⊥ = {f ∈ L2 (a, b) : f (t) = 0 almost everywhere in (c, b)}.
Then, X = M ⊕ M ⊥ .

February 27, 2018 12 / 23

 Orthononarmal Systems and Fourier Expansion:
Definitions: Let X be a Hilbert space and {ϕn } be a sequence of elements
of X . Then
(i) {ϕn } is called an orthonormal system in the Hilbert space X if

0, if m 6= n;
hϕn , ϕm i =
1, if m = n.

(ii) For any f ∈ X , αp = hf , ϕp i, p = 0, ±1, ±2, . . . is called p th fourier

∞
X
coefficients with respect to the orthonormal system {ϕn } and hf , ϕp iϕp
−∞
is called Fourier series or Fourier expansion of f with respect to orhonormal
system {ϕn }.
(iii) The orthonormal family {ϕn } is said to be complete if there exists no
other orthonormal family containing it.

February 27, 2018 13 / 23

(iv) The orthonormal family {ϕn } is said to be an orthonormal basis or closed
orthonormal system if the sum of Fourier series of f with respect to {ϕn }
is equal to f , ∀f ∈ X , i.e.,
∞
X
f = hf , ϕp iϕp , f ∈ X .
p=1

Remarks:
1. If x = {0}, then X contains no orthonormal system.
2. An orthonormal system {ϕn } is complete if and only if for any x such that
x ⊥ {ϕn }, x must be zero.
3. Every non-zero Hilbert space contains a closed orthonormal set.
4. Every orthonormal system is closed if and only if it is complete.

February 27, 2018 14 / 23

Examples 1: Consider the Hilbert space L2 (0, 2π) of complex valued func-
inx
tions √e 2π , n = 0, ±1, ±2, . . . is orthonormal system as

2π
e imx e inx
Z
imx inx
he ,e i= √ . √ dx
0 2π 2π
Z 2π 
1 0, if m 6= n;
= e imx .e −inx dx =
2π 0 1 if m = n.

2π
e inx 
Z
1
f (x)e −inx dx, n = 0, ±1, ±2, . . .

αn = Cn = f , √ =√
2π 2π 0

are fourier
 coefficients
 of f ∈ L2 (0, 2π) with respect to the orthonormal
e inx 1
Cn e inx is the fourier series of f .
P
system √2π and √2π
This orthogonal system is known as complex trigonometric system.

February 27, 2018 15 / 23

  
Example 2: If we consider L2 (0, 2π), √1 , √1 cos nt, √1 sin nt , n =
2π π π
1, 2, 3,
. . . is an orthonormal
  system in L2 (0, 2π), i.e.,
 √1π cos nt 2 , when n is an even positive integer;


ϕn =  
 √1π sin n+1 2 t , when n is an odd positive integer.



Example 3: Let us define a sequence of functions ϕ0 (t), ϕ1 (t), . . . , ϕn (t), . . .

which satisfy the following conditions:
1
ϕ0 (t) = 1, if 0 ≤ t <
2
1
ϕ0 (t) = −1 if ≤t<1
2
ϕ0 (t + 1) = ϕ0 (t)
ϕn (t) = ϕ0 (2n t), n = 1, 2, 3, . . .

The functions ϕn (t) are called Rademacher’s functions. This system of

functions is orthogonal but not complete.
February 27, 2018 16 / 23
Bessel’s Inequality:

Theorem: Let {ϕi }, i = 1, 2, 3, . . . , n be orthonormal system of vectors in

a Hilbert space X . Then for any f ∈ X ,
Xn
|hf , ϕi i|2 ≤ kf k2 .
i=1
n
X
Proof: Let g = f − hf , ϕi iϕi . Then for any j, 1 ≤ j ≤ n, we have
i=1
 n
X 
hg , ϕj i = f − hf , ϕi iϕi , ϕj
i=1
n
X
= hf , ϕj i − hf , ϕi ihϕi , ϕj i
i=1
by using the property of inner product. Since

6 j;
0, if i =
hϕi , ϕj i =
1, if i = j.
February 27, 2018 17 / 23
The above relation gives us hg , ϕj i = hf , ϕj i − hf , ϕj i = 0. Hence g ⊥ ϕj
for 1 ≤ j ≤ n, and the vectors g , hf , ϕ1 iϕ1 , hf , ϕ2 iϕ2 ,. . ., hf , ϕn iϕn form
an orthonormal system.
By the pythagorean theorem, we have
n
n
X 2 X 2
2 2

kf k = g +
hf , ϕi iϕi = kg k +
hf , ϕi iϕi

i=1 i=1

Applying Pythagorean theorem for n−elements, we have

n 2 n
X X
khf , ϕi iϕi k2


hf , ϕi iϕi =

i=1 i=1
n
X
= |hf , ϕi i|2 kϕi k2
i=1
n
X
|hf , ϕi i|2 ∵ kϕi k2 = hϕi , ϕi i = 1


=
i=1

February 27, 2018 18 / 23

 n 2 n
X X
(Let x1 , x2 , . . . , xn be n-orthogonal elements. Then
xi
= kxi k2
i=1 i=1
follows by induction from the pythagorean theorem).
Thus,
n
X
kf k2 = kg k2 + |hf , ϕi i|2 |.
i=1

n
X
Since kg k2 ≥ 0, |hf , ϕi i|2 ≤ kf k2 . This proves the desired result.
i=1
∞
X
Corollary 1: For X and {ϕn } in example 3, we have |αn |2 < ∞.
n=1
Corollary 2: If an ’s and bn ’s are ordinary trigonometric fourier coefficients,
then lim an = 0 and lim bn = 0. This result is known as the Riemann-
n→∞ n→∞
Lebesgue theorem.

February 27, 2018 19 / 23

Proof: Since f ∈ L2 (0, 2π) and hf , ϕi i = αi , the
∞
X
1. Bessel’s inequality gives us |αn |2 ≤ kf k2 < ∞.
n=1
2. By the relationship between an ’s, bn ’s and αn ’s given in example 3, we
have
X ∞  Z 2π
X π
|αn |2 = α02 + π (an2 + bn2 ) ≤ kf k = |f (t)|2 dt < ∞.
2 0
n=1

∞
X
i.e., (an2 + bn2 ) ≤ ∞, which implies that
n=1
lim (a2 + bn2 ) = 0 ⇒ lim an2 = 0 and lim bn2 = 0. Therefore, lim an = 0
n→∞ n n→∞ n→∞ n
and lim bn = 0.
n

February 27, 2018 20 / 23

Theorem: For an orthonormal system {ϕi }, i = 1, 2, . . . , n in a Hilbert
space X , for each f ∈ X , the following relation holds:
n
X n
X
kf k2 = |hf , ϕi i|2 if and only if f = hf , ϕi iϕi .
i=1 i=1

Proof: Suppose that {ϕi } is a closed orthonormal system in X , i.e.,

n
X
f = hf , ϕi iϕi .
i=1

Then,

n
X 2 X
2
|hf , ϕi i|2 kϕi k2 .

kf k =
hf , ϕi iϕi
=
i=1

We get the above relation by applying the Pythagorean theorem for n or-
thogonal elements, namely

February 27, 2018 21 / 23

 hf , ϕ1 iϕ1 , hf , ϕ2 iϕ2 , . . . , hf , ϕn iϕn .
n
X
As kϕi k = 1, we have kf k2 = |hf , ϕi i|2 .
i=1
n
X
Conversely, if kf k2 = |hf , ϕi i|2 , as shown in the Bessel’s inequality, we
i=1
have
n
X n
X
2
kf k2 = kg k2 + |hf , ϕi i| , where g = f − hf , ϕi iϕi .
i=1 i=1
n
X
This implies that kg k2 = 0, i.e., g = f − hf , ϕi iϕi = 0
i=1
n
X
or f = hf , ϕi iϕi .
i=1

February 27, 2018 22 / 23

Remarks:
n
X
1. The relation kf k2 = |hf , ϕi i|2 is known as Parseval’s identity.
i=1
2. Parseval’s identity is valid if and only if the orthonormal system {ϕi } is
closed.

February 27, 2018 23 / 23