Mathematics I

Part 1: Linear Algebra

Week 8

1. Inner Product and Inner Product Space

We have seen that the concept of a vector space is an abstraction of the set of
vectors, the Euclidean space, the set of all polynomials, and the set of all m × n
matrices. In vectors, we have an idea of a dot product which is mainly used to check
the orthogonality of two vectors as well as to find the length of a vector. This concept
of R2 and R3 are used in parallel lines, perpendicular lines, perpendicular planes, and
parallel planes in respective spaces. Thus it is interesting to introduce a generalization
of the dot product in any vector space. This leads to a function, called inner product.
Definition 1 (Inner Product and Inner Product Space). Let (V, +, ·) be a real vector
space. A function ⟨, ⟩ : V × V → R is said to an inner product on V , if
(1) ⟨u, u⟩ ≥ 0 for all u ∈ V and ⟨u, u⟩ = 0 if and only if u = 0.
(2) ⟨u, v⟩ = ⟨v, u⟩ for all u, v ∈ V .
(3) ⟨u + v, w⟩ = ⟨u, w⟩ + ⟨v, w⟩ for all u, v, w ∈ V .
(4) ⟨αu, v⟩ = α⟨u, v⟩ for all u, v ∈ V and α ∈ R.
A vector space (V, +, ·) with an inner product ⟨, ⟩ is called an inner product space,
and we write that (V, +, ·, ⟨, ⟩) is an inner product space.
We now list a few examples of inner products on certain vector spaces.
(1) Consider the function ⟨, ⟩ : R2 × R2 → R given by
⟨u, v⟩ = x1 y1 + x2 y2 for u = (x1 , x2 ), v = (y1 , y2 ) ∈ R2 .
Then this function is an inner product on R2 , called dot product. Thus
⟨(−2, 3), (5, −1)⟩ = −10 − 3 = −13.
In general, the function ⟨, ⟩ : Rn × Rn → R defined as
⟨u, v⟩ = x1 y1 + x2 y2 + · · · + xn yn ,
where u = (x1 , x2 , . . . , xn ), v = (y1 , y2 , . . . , yn ∈ Rn , is an inner product on Rn .
This is usually called the standard inner product on Rn . In this inner product,
we have
⟨(−2, 3, 4), (5, −1, 2)⟩ = −10 − 3 + 8 = −5.
(2) The function ⟨, ⟩ : R2 × R2 → R given by
⟨u, v⟩ = y1 (2x1 + x2 ) + y2 (x1 + x2 ) for u = (x1 , x2 ), v = (y1 , y2 ) ∈ R2
is an inner product on R2 . In this inner product,
⟨(−2, 3), (5, −1)⟩ = 5(−4 + 3) + (−1)(−2 + 3) = −6.
(3) Let C[a, b] be the real vector space of all real-valued continuous functions on
the interval [a, b]. The function ⟨, ⟩ : C[a, b] × C[a, b] → R given by
Z b
⟨f, g⟩ = f (t)g(t)dt
a
1
2

is an inner product. In this case

Z b Z b
2 2 3
⟨3x, x ⟩ = 3t · t dt = 3t3 dt = (b4 − a4 ).
a a 4
(4) The function ⟨, ⟩ : Mn×n (R) × Mn×n (R) → R given by
n X
X n
⟨A, B⟩ = aij bij for A = (aij )n×n , B = (bij )n×n
i=1 j=1

in an inner product. Note that

   
−1 3 3 2
, = (−1).3 + 3.2 + 5.(−3) + 0.1 = −12.
5 0 −3 1
We also list a few examples that are not inner products.
(1) The function ⟨, ⟩ : R2 × R2 → R given by
⟨u, v⟩ = x1 x2 + y1 y2 for u = (x1 , x2 ), v = (y1 , y2 ) ∈ R2
is not an inner product due to the fact that ⟨u, u⟩ = 2x1 x2 is not necessarily
non-negative.
(2) The function ⟨, ⟩ : Pn [x] × Pn [x] → R given by
⟨p(x), q(x)⟩ = p(0)q(0)
is not an inner product due to the fact that ⟨p(x), p(x)⟩ = 0 does not necessarily
imply that p(x) = 0.
Properties: Let V be an inner product space. For any u, v, w ∈ V and a scalar c, we
have
(1) ⟨u, v + w⟩ = ⟨u, v⟩ + ⟨u, w⟩.
(2) ⟨u, cv⟩ = c⟨u, v⟩.
(3) ⟨u, 0⟩ = ⟨0, v⟩ = 0.

Proof. Using the definition of an inner product space, one can easily verify (1) and (2).
For (3), we have
⟨u, 0⟩ = ⟨u, 0 · v⟩ = 0 · ⟨u, v⟩ = 0.
Similarly, one can show that
⟨0, v⟩ = 0.
□
Definition 2 (Length of a vector). Let V be an inner product space with the inner
product ⟨, ⟩. Then the function || · || : V → R given by
p
||x|| = ⟨x, x⟩
is called a norm on V . For any vector x ∈ V , ||x|| is a nonnegative real number that
measures the length of x as well as the distance of the vector x from the zero vector 0.
 3

(1) For the inner product ⟨u, v⟩ = x1 y1 +x2 y2 +· · ·+xn yn for u = (x1 , x2 , . . . , xn ), v =
(y1 , y2 , . . . , yn ∈ Rn , we have
p q
||u|| = ⟨u, u⟩ = x21 + x22 + · · · + x2n ,
which denote the distance of u from the origin. Note that
√ √
||(−2, 3, 4)|| = 4 + 9 + 16 = 29.
(2) For the inner product ⟨u, v⟩ = y1 (2x1 + x2 ) + y2 (x1 + x2 ) for u = (x1 , x2 ), v =
(y1 , y2 ) ∈ R2 , we have
p p q
||u|| = ⟨u, u⟩ = x1 (2x1 + x2 ) + x2 (x1 + x2 ) = (x1 + x2 )2 + x21 .
As a particular case,
p √
||(−2, 3)|| = (−2 + 3)2 + (−2)2 = 5
under this norm.
(3) For the inner product
Z b
⟨f, g⟩ = f (t)g(t)dt
a

on C[a.b], we have
s
p Z b
||f (x)|| = ⟨f (x), f (x)⟩ = f (t)2 dt.
a

Thus s
2 √
Z b
r
2 4 5
||2x || = 4t4 dt = (b − a5 ) = √ b5 − a5 .
a 5 5
n X
X n
(4) For the inner product ⟨A, B⟩ = aij bij for A = (aij )n×n , B = (bij )n×n ∈
i=1 j=1
Mn×n (R), we have
v
u n X
n
uX
p
||A|| = ⟨A, A⟩ = t a2ij .
i=1 j=1

Thus
√
 
−1 3 p
= (−1)2 + 32 + 52 + 02 = 35.
5 0
Theorem 1.1 (Cauchy-Schwarz Inequality). Let V be an inner product space. For
any u, v ∈ V , we have
|⟨u, v⟩| ≤ ||u|| · ||v||.
The equality holds if and only if u and v are linearly dependent.
4

Proof. If u = 0, then the equality holds trivially. Suppose u ̸= 0, then we consider

⟨v, u⟩
z=v− u.
||u||2
Thus
⟨v, u⟩2
 
2 ⟨v, u⟩ ⟨v, u⟩ ⟨v, u⟩ ⟨v, u⟩
||z|| = ⟨z, z⟩ = v− u, v − u = ⟨v, v⟩ − ⟨u, v⟩ − ⟨v, u⟩ + ⟨u, u⟩
||u||2 ||u||2 ||u||2 ||u||2 ||u||4
⟨v, u⟩2
= ||v||2 − .
||u||2
Since ||z|| ≥ 0, we must have
⟨v, u⟩2
||v||2 − ≥ 0.
||u||2
As a result,
⟨v, u⟩2 ≥ ||u||2 ||v||2 ,
and hence the result follows.
The equality holds if and only if
||z|| = 0,
⟨v,u⟩ ⟨v,u⟩
which is true if and only if z = 0 or v − ||u|| 2 u = 0. Taking c = ||u||2
, we see that v = cu,
which is valid if and only if u and v are linearly dependent. □
Remark 1.2. If u and v are two nonzero vectors in an inner product space V , then
the Cauchy-Schwarz inequality gives
⟨u, v⟩
−1 ≤ ≤ 1.
||u|| · ||v||
Thus there exists a θ ∈ [0, π] such that
⟨u, v⟩
cos θ = .
||u|| · ||v||
This angle θ is called the angle between the two vectors u and v.
Properties: Let V be an inner product space. For u, v ∈ V and c ∈ R, we have
(1) ||u|| ≥ 0 and ||u|| = 0 if and only if u = 0.
(2) ||cu|| = |c| · ||u||.
(3) ||u + v|| ≤ ||u|| + ||v|| (Triangle Inequality).
(4) | ||u|| − ||v|| |≤ ||u − v||.
u
(5) For any nonzero vector u ∈ V , there exist v = ||u|| such that ||v|| = 1 and
u
u = ||u||v. Here v = ||u|| is called the unit vector along u.
Proof. The proof of (1) and (2) follows from the definition of norm and inner product
of a vector space.

(3) We see that

||u + v||2 = ⟨u + v, u + v⟩ = ||u||2 + 2⟨u, v⟩ + ||v||2 = (||u|| + ||v||)2 + 2(⟨u, v⟩ − ||u|| · ||v||).
 5

Using the Cauchy-Schwatz inequality, the result follows.

(4) Using the result of (3), we note that

||u|| = ||(u − v) + v|| ≤ ||u − v|| + ||v||,
and hence
||u|| − ||v|| ≤ ||u − v||. (1)
In the similar way,
||v|| = ||(v − u) + u|| ≤ ||v − u|| + ||u|| = ||u − v|| + ||u||,
and hence
−||u − v|| ≤ ||u|| − ||v||. (2)
Combining (1) and (2), we obtain the desired result.

(5) The proof is obvious. □

Definition 3 (Orthogonal vectors). Two vectors u and v in an inner product space V
are said to be orthogonal (or perpendicular) if ⟨u, v⟩ = 0.
Note that ⟨u, 0⟩ = 0 for any u in an inner product space, therefore 0 is orthogonal
to any vector of an inner product space. Moreover, 0 is the only vector in an inner
product space that is orthogonal to itself.
With respect to the standard inner product on R3 , the two vectors (2, −3, 1) and
(1, 1, 1) are orthogonal because of the fact that ⟨(2, −3, 1), (1, 1, 1)⟩ = 0. However,
(2, 3, 1) and (1, 1, 1) are not orthogonal due to the fact that ⟨(2, 3, 1), (1, 1, 1)⟩ = 6 ̸= 0.
Definition 4 (Orthonormal vectors). Two vectors u and v in an inner product space
V are said to be orthonormal if ⟨u, v⟩ = 0 and ||u|| = ||v|| = 1.
With respect to the standard inner product on R3 , the two vectors (2, −3, √1) and
 that ||(2,
(1, 1, 1) are orthogonal but not orthonormal due to the fact
√  −3, 1)||
 = 14 ̸= 1
and ||(1, 1, 1)|| = 3 ̸= 1. However, the two vectors √2 , 0, √2 and − √13 , √13 , √13
1 1

are orthonormal.
Definition 5 (Orthogonal set and Orthonormal set). A set of vectors A = {v1 , v2 , . . . , vk }
in an inner product space V is said to be an orthogonal set if ⟨vi , vj ⟩ = 0 for all vi , vj ∈ A
with vi ̸= vj . An orthogonal set A is said to be orthonormal set if ||vi || = 1 for all
vi ∈ A.
The set {(3, 5, 0), (−5, 3, 0), (0, 0, 4)} is an orthogonaln
set (but not orthonormal)
 with o
3 1 1 1 1
respect to the standard inner product on R . Moreover, √
2
, 2 , 0 , − 2 , 2 , 0 , (0, 0, 1)
√ √ √

is an orthonormal set with respect to the standard inner product on R3 .

It is an easy exercise to show that
Z π 
0, if m ̸= n;
sin mt sin ntdt =
−π
π, if m = n.
6

Therefore, the set

{sin x, sin 2x, . . . , sin kx}
is
Z an orthogonal set (but not orthonormal) with respect to the inner product ⟨f, g⟩ =
π
f (t)g(t)dt on the space C[−π, π]. Under the same inner product on the same space,
−π
the set  
1 1 1
√ sin x, √ sin 2x, . . . , √ sin kx
π π π
is an orthonormal set.
Theorem 1.3. Any orthogonal set of non-zero vectors in an inner product space is
always linearly independent.
Proof. Let V be an inner product space. Suppose A = {v1 , v2 , . . . , vk } is an orthogonal
set of non-zero vectors in V , and hence ⟨vi , vj ⟩ = 0 for all vi , vj ∈ A with vi ̸= vj . Let
a1 v1 + a2 v2 + · · · + ak vk = 0.
For each i with 1 ≤ i ≤ k, we have
⟨vi , a1 v1 + a2 v2 + · · · + ak vk ⟩ = ⟨vi , 0⟩ = 0.
This implies
ai ⟨vi , vi ⟩ = 0,
and hence ai = 0. Thus A is a linearly independent set. □
Remark 1.4. The converse of the above theorem is not true. All linearly independent
vectors are not necessarily orthogonal. For example {(1, 1), (1, 0)} is a linearly inde-
pendent set, but (1, 1) and (1, 0) are not orthogonal with respect to the standard inner
product on R2 .

2. Gram-Schmidt orthogonalization Process

An orthogonal basis for an inner space V is a basis of V that is an orthogonal set.
If a basis of an inner product space V is orthonormal, then the basis is called an
orthonormal basis of V . The following theorem, due to Gram-Schmidt, provides an
algorithm to find an orthogonal basis for any finite-dimensional inner product space
from a given basis. The process of finding an orthogonal basis for any finite-dimensional
inner product space from a given basis is called the Gram-Schmidt orthogonalization
process.
Theorem 2.1. Every finite-dimensional inner product space has an orthogonal basis.
Proof. Let V be an inner product space and B = {u1 , u2 , . . . , um } a basis for V . We
now construct a basis set of orthogonal vectors for V from the given basis B. Let
w1 = u1 ,
and then
⟨u2 , w1 ⟩
w 2 = u2 − w1 .
||w1 ||2
 7

It is easy to see that

 
⟨u2 , w1 ⟩ ⟨u2 , w1 ⟩
⟨w1 , w2 ⟩ = w1 , u2 − 2
w1 = ⟨w1 , u2 ⟩ − ⟨w1 , w1 ⟩ = 0 and w2 ̸= 0.
||w1 || ||w1 ||2
Using induction, suppose
{w1 , w2 , . . . , wk }
is an orthogonal set of nonzero vectors in V with 1 ≤ k < m. We now consider
k
X ⟨uk+1 , wj ⟩
wk+1 = uk+1 − wj .
j=1
||wj ||2

Note that wk+1 ̸= 0, otherwise, wk+1 ∈ span{w1 , w2 , . . . , wk }. Thus, for 1 ≤ i ≤ k, we

have
* k
+
X ⟨uk+1 , wj ⟩
⟨wk+1 , wi ⟩ = uk+1 − 2
wj , wi
j=1
||w j ||
k
X ⟨uk+1 , wj ⟩
= ⟨uk+1 , wi ⟩ − ⟨wj , wi ⟩
j=1
||wj ||2
= ⟨uk+1 , wi ⟩ − ⟨uk+1 , wi ⟩ = 0.
As a result, the set {w1 , w2 , . . . , wm } form an orthogonal set of vectors of V , and hence
it forms an orthogonal basis for the inner product space V because of Theorem 2.1. □
Remark 2.2. Every finite-dimensional inner product space has an orthonormal ba-
sis. If the orthogonal basis {w1 , w2 , . . . , wm } of V is obtained using
n the Gram-Schmidt o
w1 w2 wm
orthogonalization, then the orthonormal basis of V is given by ||w1 || , ||w 2 ||
, . . . , ||wm ||
.

Example: Find an orthogonal basis for the inner product space of all polynomials of
degree at most 2 with real coefficients P2 [x] starting from {1, x, x2 } with respect to the
usual inner product defined on C[0, 1].

Solution: Let u1 = 1, u2 = x, u3 = x2 . We consider

w1 = 1.
Then
⟨u2 , w1 ⟩ 1 1
w 2 = u2 − w 1 = x − · 1 = x − ,
||w1 ||2 2 2
because of the fact that
Z 1
1
⟨u2 , w1 ⟩ = ⟨x, 1⟩ = tdt =
0 2
and s
p Z 1
||w1 || = ⟨1, 1⟩ = dt = 1.
0
8

Finally,
 
⟨u3 , w1 ⟩ ⟨u3 , w2 ⟩ 2 1 1 1
w3 = u3 − 2
w1 − 2
w2 = x − − x − = x2 − x + ,
||w1 || ||w2 || 3 2 6
because of the facts that
Z 1
2 1
t2 dt = ,
⟨u3 , w1 ⟩ = x , 1 =
0 3
  Z 1  
2 1 2 1 1 1 1
⟨u3 , w2 ⟩ = x , x − = t t− dt = − = ,
2 0 2 4 6 12
and
s s 2 r
 Z 1 
1 1 1 1 1 1 1
||w2 || = x − ,x − = t− = − + =√ .
2 2 0 2 3 2 4 12
Therefore 1, x − 21 , x2 − x + 1

6
is an orthogonal basis for P2 [x].

Example: Find an orthonormal basis for R3 starting from the basis {(1, 1, 1), (1, 1, 0), (1, 0, 0)
with respect to the standard inner product on R3 .

Solution: Let u1 = (1, 1, 1), u2 = (1, 1, 0), u3 = (1, 0, 0). We consider

w1 = u1 = (1, 1, 1).
Then  
⟨u2 , w1 ⟩ 2 1 1 2
w2 = u2 − w 1 = (1, 1, 0) − (1, 1, 1) = , ,− .
||w1 ||2 3 3 3 3
Finally,
   
⟨u3 , w1 ⟩ ⟨u3 , w2 ⟩ 1 13 1 1 2 1 1
w3 = u3 − w1 − w2 = (1, 0, 0)− (1, 1, 1)− , ,− = ,− ,0 .
||w1 ||2 ||w2 ||2 3 32 3 3 3 2 2
Therefore (1, 1, 1), 13 , 13 , − 32 , 21 , − 21 , 0 is an orthogonal basis for R3 . Further, di-


viding each vector of the basis by its norm, we obtain the orthonormal basis for R3
as ( r ! 
 )
1 1 1 1 1 2 1 1
√ ,√ ,√ , √ ,√ , , √ , −√ , 0 .
3 3 3 6 6 3 2 2
Remark 2.3. Changing the order of u1 , u2 , u3 in the examples above, one can get a
different orthogonal and orthonormal basis for the inner product spaces.
Theorem 2.4. Let {u1 , u2 , . . . , un } be an orthonormal basis for an inner product space
V . For any x ∈ V , we have
Xn
x= ⟨x, ui ⟩ui .
i=1

Proof. Since {u1 , u2 , . . . , un } is a basis for V and x ∈ V , we must have

x = a1 u1 + a2 u2 + · · · + an un
 9

with ai ∈ R for 1 ≤ i ≤ n. For each i with 1 ≤ i ≤ n, we have

⟨x, ui ⟩ = ⟨a1 u1 + a2 u2 + · · · + an un , ui ⟩ = ai ⟨ui , ui ⟩ = ai

because of the fact that {u1 , u2 , . . . , un } is an orthonormal set. As a result, we obtain

n
X
x = ⟨x, u1 ⟩u1 + ⟨x, u2 ⟩u2 + · · · + ⟨x, un ⟩un = ⟨x, ui ⟩ui ,
i=1

completing the proof of the theorem. □

n    o
Consider the orthonormal basis B = √1 , √1 , 0 1 √1
, − 2 , 2 , 0 , (0, 0, 1) of R3 .
√
2 2
For any (a, b, c) ∈ R3 , we use the above theorem to deduce that
   
a+b 1 1 −a + b 1 1
(a, b, c) = √ √ , √ ,0 + √ − √ , √ , 0 + c(0, 0, 1)
2 2 2 2 2 2
   
√ , ⟨(a, b, c), − √1 , √1 , 0 ⟩ = −a+b
because of the fact that ⟨(a, b, c), √12 , √12 , 0 ⟩ = a+b √ ,
2 2 2 2
and ⟨(a, b, c), (0, 0, 1)⟩ = c.
Since the standard basis {(1, 0, 0), (0, 1, 0), (0, 0, 1)} of R3 is orthonormal, we have

(a, b, c) = a(1, 0, 0) + b(0, 1, 0) + c(0, 0, 1)

for any a, b, c) ∈ R3 because of the fact that ⟨(a, b, c), (1, 0, 0) = a, ⟨(a, b, c), (0, 1, 0) = b,
and ⟨(a, b, c), (0, 0, 1) = c.

3. Orthogonal Projection and Orthogonal Subspaces

For the vector u = (1, 2, 3) ∈ R3 , we know that (1, 2, 0) is the orthogonal projection
of u on the XY -plane. Similarly, (0, 2, 0) is the orthogonal projection of u on the Y -
axis. We now give a general definition of the orthogonal projection of any vector on a
subspace of an inner product space.

Definition 6. Let U be a subspace of an inner product space V . If {u1 , u2 , . . . , um }

is an orthonormal basis for U and x ∈ V . Then the orthogonal projection PU (x) of x
onto U is defined as
m
X
PU (x) = ⟨x, u1 ⟩u1 + ⟨x, u2 ⟩u2 + · · · + ⟨x, um ⟩um = ⟨x, uj ⟩uj .
j=1

It is easy to see that the orthogonal projection of x onto U is an element of U , i.e.,

PU (x) ∈ U . Moreover, for any v ∈ U , we have
m
X
v = a1 u1 + a2 u2 + · · · + am um = ai u i
i=1
10

with ai ∈ R for 1 ≤ i ≤ m. As a result,

⟨x − PU (x), v⟩ = ⟨x, v⟩ − ⟨PU (x), v⟩
* m + * m m
+
X X X
= x, ai u i − ⟨x, uj ⟩uj , ai u i
i=1 j=1 i=1
m
X m
XX m
= ai ⟨x, ui ⟩ − ai ⟨x, uj ⟩ ⟨uj , ui ⟩
i=1 j=1 i=1
Xm Xm
= ai ⟨x, ui ⟩ − ai ⟨x, ui ⟩
i=1 j=1
= 0.
Thus x − PU (x) is orthogonal to each vectors of U , and hence x − PU (x) is orthogonal
to U , and we write x − PU (x) ⊥ U . Therefore, x − PU (x) is called the complement of
x orthogonal to the subspace U .

Example: An orthonormal basis for the XY -plane is {u1 = (1, 0, 0), u2 = (0, 1, 0)}.
Therefore, the projection of u(1, 2, 3) on XY -plane is given by
PXY (u) = ⟨(1, 2, 3), (1, 0, 0)⟩(1, 0, 0) + ⟨(1, 2, 3), (0, 1, 0)⟩(0, 1, 0) = (1, 2, 0).
Further, the complement of u orthogonal to XY -plane is
u − PXY (u) = (1, 2, 3) − (1, 2, 0) = (0, 0, 3).
Similarly, an orthonormal basis for the Y -axis is {v1 = (0, 1, 0)}, and hence the
orthogonal projection of u on Y -axis is given by
PY (u) = ⟨(1, 2, 3), (0, 1, 0)⟩(0, 1, 0) = (0, 2, 0).
Moreover, the complement of u orthogonal to Y -axis is
u − PY (u) = (1, 2, 3) − (0, 2, 0) = (1, 0, 3).
Example: Consider the subspace U = {(x, y, z) ∈ R3 : x−y +2z = 0} of R3 . Consider
the orthogonal basis {(1, 1, 0), (−1, 1, 1)} of U , then B = {u1 = ( √12 , √12 , 0), u2 =
(− √13 , √13 , √13 )} is an orthonormal basis for U . For the vector v = (3, −1, 2), the
projection of v on U is given by
     
2 1 1 2 1 1 1 5 1 2
PU (v) = ⟨v, u1 ⟩u1 +⟨v, u2 ⟩u2 = √ √ , √ , 0 −√ −√ , √ , √ = , ,− .
2 2 2 3 3 3 3 3 3 3
Further the complement of v orthogonal to U is given by
   
5 1 2 4 4 8
v − PU (v) = (3, −1, 2) − , ,− = ,− , .
3 3 3 3 3 3
Definition 7 (Orthogonal Subspaces). Let U and W be two subspaces of an inner
product space V . Then U and W are said to be orthogonal, denoted by U ⊥ W , if
⟨u, w⟩ = 0 for all u ∈ U, w ∈ W.
 11

For example, the two vector subspaces U = {(x, y, z) ∈ R3 : x = y} and W =

{(x, y, z) ∈ R3 : x = −y, z = 0} are orthogonal due to the fact that ⟨(a, a, b), (p, −p, 0)⟩ =
0 for all (a, a, b) ∈ U and (p, p, 0) ∈ W .
Let U = span{(1, 0, 0), (0, 1, 0)}, the XY -plane and V = span{(0, 0, 1)}, the Z-axis.
For any (a, b, 0) ∈ U and (0, 0, c) ∈ V , we have ⟨(a, b, 0), (0, 0, c) = 0, and hence U ⊥ V .
Definition 8 (Orthogonal Complement). Let U be a subspace of an inner product
space V , then the orthogonal complement of U , denoted by U ⊥ , is defined as
U ⊥ = {v ∈ V : ⟨v, u⟩ = 0 for all u ∈ U }.
In other words, U ⊥ is the collection of all vectors of V which are orthogonal to each
element of U .
For example, if U = {(x, y, z) ∈ R3 : x = y} and W = {(x, y, z) ∈ R3 : x = −y, z =
0}, then U ⊥ = W and W ⊥ = U .
Let U = span{(1, 1, 0), (0, 1, 1)}, then
U ⊥ = {(x, y, z) : ⟨(x, y, z), (a, b, c)⟩ = 0 for all (a, b, c) ∈ U }
= {(x, y, z) : ⟨(x, y, z), (1, 1, 0)⟩ = 0 and ⟨(x, y, z), (0, 1, 1)⟩ = 0}
= {(x, y, z) : x + y = 0 and y + z = 0}
= span{(1, −1, 1)}.
Properties: Let U be a subspace of an inner product space V .
(1) For any x ∈ V , we have x − PU (x) ∈ U ⊥ .
(2) U ⊥ is a vector subspace of V .
(3) U and U ⊥ are orthogonal subspaces of V .
(4) U ∩ U ⊥ = {0}.
(5) U + U⊥ = V .
(6) For any x ∈ V , PU (x) is always unique.
(7) For any x ∈ V , PU (x) is the nearest vector from x on U .

Proof. (1) We have already shown above that

x − PU (x) ⊥ U for all x ∈ V.
Thus x − PU (x) ∈ U ⊥ .

(2) It is clear that 0 ∈ U ⊥ . Let u, v ∈ U ⊥ , then ⟨u, x⟩ = ⟨v, x⟩ = 0 for all x ∈ U . For
any c ∈ R, we have
⟨u + cv, x⟩ = ⟨u, x⟩ + c⟨v, x⟩ = 0.
Therefore, u + cv ∈ U ⊥ concluding that U ⊥ is a subspace of V .

(3) This follows obviously from the definition.

(4) Let x ∈ U ∩ U ⊥ , then x ∈ U and x ∈ U ⊥ . Thus we must have

⟨x, x⟩ = 0,
12

which implies that x = 0. Hence the result follows.

(5) Since both U and U ⊥ are subspaces of V , we must have

U + U ⊥ ⊆ V.
Let v ∈ V be arbitrary, then PU (v) ∈ U and v − PU (x) ∈ U ⊥ . Thus
v = PU (x) + (x − PU (x)) ∈ U + U ⊥ .
This implies that
V ⊆ U + U ⊥,
and hence
V = U + U ⊥.
(6) If possible, let PU (x) and QU (x) be two orthogonal projections of x on U . Since
PU (x), QU (x) ∈ U
and U is a vector subspace of V , we must have
PU (x) − QU (x) ∈ U.
Again,
x − PU (x), x − QU (x) ∈ U ⊥
and U ⊥ is a vector subspace of V , we must have
(x − QU (x)) − (x − PU (x)) = PU (x) − QU (x) ∈ U ⊥ .
As a result
PU (x) − QU (x) ∈ U ∩ U ⊥ ,
which implies that PU (x) − QU (x) = 0, and hence PU (x) = QU (x).

(7) We show that

||x − PU (x)|| ≤ ||x − u|| for all u ∈ U.
We have
||x − u||2 = ||(x − PU (x)) + (PU (x) − u)||2 .
If two vectors x, y are orthogonal, then Pythagoras’ Theorem states that ||x + y||2 =
||x||2 + ||y||2 (prove this !). We note that (x − PU (x)) ⊥ (PU (x) − u) (prove this !), and
hence
||x − u||2 = ||x − PU (x)||2 + ||PU (x) − u||2 ≥ ||x − PU (x)||2 .
Taking square root on both the sides, we complete the proof. □