Inner Product

The orthogonality
The Orthonormal Basis

Inner Product Spaces and Orthogonality

Mongi BLEL

King Saud University

January 23, 2021

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Table of contents

1 Inner Product

2 The orthogonality

3 The Orthonormal Basis

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Inner Product

Definition
Let V be a vector space on R.
We say that a function h , i : V × V −→ R is an inner product on
V if it satisfies the following:
For all u, v , w ∈ V , α ∈ R.
1 hu, v i = hv , ui
2 hu + v , w i = hu, w i + hv , w i
3 hαu, v i = αhu, v i
4 hu, ui ≥ 0
5 hu, ui = 0 ⇐⇒ u = 0

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Examples

1 The Euclidean inner product on Rn defined by:

n
X
hu, v i = xj yj = x1 y1 + . . . + xn yn ,
j=1

where u, v ∈ Rn , u = (x1 , . . . , xn ) and v = (y1 , . . . , yn ).

2 If E = C([0, 1]) the vector space of continuous functions on
[0, 1]. For all f , g ∈ E , we define the inner product of f and g
by:
Z 1
hf , g i = f (t)g (t)t..
0

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Remarks

If (E , h , i) is an inner product space and u, v , w , x ∈ E ,

a, b, c, d ∈ R, we have:

hu + v , w + xi = hu, w i + hu, xi + hv , w i + hv , xi.

hau + bv , cw + dxi = achu, w i + adhu, xi

+bchv , w i + bdhv , xi.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Example

Let u = (x, y ) and v = (a, b), we define

hu, v i = 2ax + by − bx − ay

h , i is an inner product on R2 .
It is enough to prove that hu, ui ≥ 0 and hu, ui = 0 ⇐⇒ u = 0.

hu, ui = 2x 2 + y 2 − 2xy = (x − y )2 + x 2 ≥ 0
and hu, ui = 0 ⇐⇒ u = 0.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Example

Let u = (x, y , z) and v = (a, b, c), we define

hu, v i = 2ax + by + 3cz − bx − ay + cy + bz

h , i is an inner product on R3 .
It is enough to prove that hu, ui ≥ 0 and hu, ui = 0 ⇐⇒ u = 0.

hu, ui = (y + z − x)2 − (z − x)2 + 2x 2 + 3z 2

= (y + z − x)2 + (x + z)2 + z 2 ≥ 0

hu, ui = 0 ⇐⇒ z = x = y = 0 ⇐⇒ u = 0.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Example

Let u = (x, y , z) and v = (a, b, c), we define

hu, v i = 2ax + by + cz − bx − ay + cy + bz

h , i is not an inner product on R3 .

hu, ui = (y + z − x)2 − (z − x)2 + 2x 2 + z 2

= (y + z − x)2 + x 2 + 2xz
= (y + z − x)2 + (x + z)2 − z 2 .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Example

If A = (aj,k ) ∈ Mn (R), we define the trace of the matrix A by:

n
X
tr(A) = aj,j
j=1

and
hA, Bi = tr(AB T )
for all A, B ∈ Mn (R).
hA, Bi is an inner product on the vector space Mn (R).

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Exercise
If u = (x1 , x2 , x3 ), v = (y1 , y2 , y3 ), we define the following
functions: f , g , h, k : R2 × R3 −→ R.
1 f (u, v ) = x1 y1 + x2 y2 + 2x3 y3 + x2 y1 + 2x1 y2 + x2 y3 + y2 x3 .
2 g (u, v ) = x1 y2 + x2 y1 + x2 y3 + x3 y2 + 3x1 y3 + 3x3 y1 .
3 h(u, v ) =
x1 y1 + x2 y2 + x3 y3 + x2 y1 + x1 y2 + x2 y3 + y2 x3 + x3 y1 + x1 y3 .
4 k(u, v ) = x1 y1 + x2 y2 + x3 y3 − x2 y3 − x3 y2 + x1 y3 + y1 x3 .
Select from which the functions f , g , h, k is an inner product
on R3 .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Solution
1 f (u, v ) − f (v , u) = x1 y2 − x2 y1 . Then f is not an inner
product on R3 .
2 g (u, u) = 2x1 x2 + 2x2 x3 + 6x1 x3 = 2(x1 + x3 )(x2 + 3x3 ) − 6x32 =
(x1 + x2 + 4x3 )2 − (x1 − x2 − 2x3 )2 − 6x32 . .
Then g is not an inner product on R3 .
3

h(u, u) = x12 + x22 + x32 + 2x1 x2 + 2x2 x3 + 2x1 x3

= (x1 + x2 + x3 )2

Then h is not an inner product on R3 because

h(u, u) = 0 6⇒ u = 0.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

k(u, u) = x12 + x22 + x32 − 2x2 x3 + 2x1 x3

= (x1 + x3 )2 + x22 − 2x2 x3
= (x1 + x3 )2 + (x2 − x3 )2 − x32

Then k is not an inner product on R3 because

k(u, u) = 0 6⇒ u = 0.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Example
Find the values of a, b such that

h(x1 , x2 ), (y1 , y2 )i = x1 y1 + x2 y2 + ax1 y2 + bx2 y1

is an inner product on R2 .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Solution

h(x1 , x2 ), (y1 , y2 )i = h(y1 , y2 ), (x1 , x2 )i if a = b.

h(x1 , x2 ), (x1 , x2 )i = x12 + x22 + 2ax1 x2

= (x1 + ax2 )2 + x22 (1 − a2 ).

Then h , i is an inner product on R2 if and only if |a| < 1.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Definition
Let (E , h , i) be an inner product space.
1 If u ∈ E , we define the norm of the vector u by:
p
kuk = hu, ui.

2 If u, v ∈ E , we define distance between u and v by:

d(u, v ) = ku − v k.

3 We define the angle 0 ≤ θ ≤ π between the vectors u, v ∈ E

by:

hu, v i
cos θ =
kuk.kv k

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Let the inner product space M2 (R), h , i) defined by:

hA, Bi = tr(AB T ).

Find 
cos θ If θ is the angle
 between
 the matrices
1 −1 2 1
A= and B = .
2 3 1 1
 
1 0
AB T = , kAk2 = 15, kBk2 = 7.
7 5
Then √
2 3
cos θ = √ .
35

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Theorem (Cauchy-Schwarz Inequality)

If (E , h , i) is an inner product space and u, v ∈ E ,
then

|hu, v i| ≤ kukkv k. (1)

We have the equality in (1) if the vectors u, v are linearly
dependent.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Proof

Let Q(t) be the polynomial

Q(t) = ku + tv k2 = kuk2 + 2thu, v i + t 2 kv k2 .

Since Q(t) ≥ 0 for all t ∈ R, then the discriminant of Q(t) is non

positive. Then
hu, v i2 ≤ kuk2 kv k2 .
If |hu, v i| = kukkv k, this mean that the discriminant of Q(t) is
zero. Then the equation Q(t) = 0 has a solution. This means that
the vectors u, v are linearly dependent.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Theorem
If (E , h , i) is an inner product space and u, v ∈ E , then

ku + v k ≤ kuk + kv k.

Proof

ku + v k2 = kuk2 + kv k2 + 2hu, v i
≤ kuk2 + kv k2 + 2kuk kv k = (kuk + kv k)2 .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Definition
If (E , h , i) is an inner product space. We say that the vectors
u, v ∈ E are orthogonal and we denote u ⊥ v if hu, v i = 0.

Theorem (Pythagor’s Theorem)

If u ⊥ v if and only if

ku + v k2 = kuk2 + kv k2 .

Proof
ku + v k2 = kuk2 + kv k2 + 2hu, v i = kuk2 + kv k2 .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Definition
If (E , h , i) is an inner product space. We say that set
S = {e1 , . . . , en } of non zeros vectors is orthogonal if

hej , ek i = 0, ∀1 ≤ j 6= k ≤ n.

and we say that S is normal if

kej k = 1, ∀1 ≤ j ≤ n.
and we say that it is orthonormal if

hej , ek i = δj,k , ∀1 ≤ j, k ≤ n.

(δj,k = 0 If j 6= k and δj,j = 1.)

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Theorem
Any set of non zero orthogonal vectors is linearly independent .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Theorem
If (E , h , i) is an inner product space and if S = {e1 , . . . , en } is an
orthonormal basis of E , then for all u ∈ E

u = hu, e1 ie1 + . . . + hu, en ien .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Proof

n
X Pn
If u = aj ej , then hu, ek i = j=1 aj hej , ek i = ak .
j=1

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Theorem
(Gramm-Schmidt Algorithm) If (E , h , i) is an inner product space
and (v1 , . . . , vn ) a set of linearly independent vectors in E , there is
a unique orthonormal set (e1 , . . . , en ) such that
1 for all k ∈ {1, . . . , n},

Vect(e1 , . . . , ek ) = Vect(v1 , . . . , vk ),

2 for all k ∈ {1, . . . , n},

hek , vk i > 0.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Proof
We construct in the first time an orthogonal set (u1 , . . . , un ) such
that: 

 u1 = v1

 hu 1 , v2 i
u2 = v2 − u1


ku1 k2



..

 .
 n−1

 X hui , vn i
u = v − ui .

n n

kui k2


i=1

We construct the set (e1 , . . . , en ) from (u1 , . . . , un ) as follows:

uk
ek = , k ∈ {1, . . . , n}.
kuk k

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Example

Let F be the vector sub-space of R4 spanned by the vectors

S = {u = (1, 1, 0, 0), v = (1, 0, −1, 0), w = (0, 0, 1, 1)}.
1 Prove that S is a basis of the sub-space F .
2 In use of Gramm-Schmidt Algorithm, find an orthonormal
basis of F . (with respect to the Euclidean inner product).

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Solution
 
1 1 0
1 0 0 
1
0 −1 1 with columns the vectors u, v , w .
Let A =  

0 0 1
 
1 1 0
0 1 0
The matrix A = 0 0 1 is a row reduced form of the


0 0 0
matrix A. This proves that S is a basis of the sub-space F .
2 u1 = √1 (1, 1, 0, 0), u2 = √1 (1, −1, −2, 0),
2 6
u3 = √112 (1, −1, 1, 3).
{u1 , u2 , u3 } is an orthonormal basis of the sub-space F.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Exercise

1 Prove that h(a, b), (x, y )i = ax + ay + bx + 2by is an inner

product in R2 .
2 Use Gramm-Schmidt algorithm to construct an orthonormal
basis of R2 from the basis {u1 = (1, −1), u2 = (1, 2)}.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Solution

1 • h(a, b) + (c, d), (x, y )i = (a + c)x + (a + c)y + (b + d)x +

2(b + d)y = h(a, b), (x, y )i + h(c, d), (x, y )i
• h(a, b), (x, y )i = ax + ay + bx + 2by = h(x, y ), (a, b)i
• hλ(a, b), (x, y )i = λax + λay + λbx + 2λby =
λh(a, b), (x, y )i
• h(a, b), (a, b)i = a2 + 2ab + 2b 2 = (a + b)2 + b 2 ≥ 0
• h(a, b), (a, b)i = 0 ⇐⇒ a + b = 0 = b ⇐⇒ a = b = 0
2 The vector u1 is unitary and the second vector is v2 = (1, 0).
Then {v1 = (1, −1), v2 = (1, 0)} is an orthonormal basis.

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Example

Let S = {u1 , u2 , u3 , u4 } is a basis

 of the
 space
 M2 (R)  suchthat
1 −1 1 0 1 0 0 1
u1 = , u2 = , u3 = , u4 =
0 1 1 1 0 2 1 1
We use the Gramm-Schmidt algorithm to construct an
orthonormal  basisfrom the basis S.
1 1 −1
v1 = √ .
3 0 1
2
hu2 , v1 i = √ ,
3  
1 1 2
u2 − hu2 , v1 iv1 = .
3 1
 3
1 1 2
v2 = √ .
15 3 1

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

√ 3
hu3 , v1 i = 3, hu3 , v2 i = √
15  
1 −1 3
u3 − hu3 , v1 iv1 − hu3 , v2 iv2 = .
  5 −3 4
1 −1 3
v3 = √ .
35 −3 4
6 4
hu4 , v1 i = 0, hu4 , v2 i = √ , hu4 , v3 i = √
15 35  
1 −10 −39
u4 − hu4 , v1 iv1 − hu4 , v2 iv2 − hu4 , v3 iv3 = .
  35 −29 −29
1 2 1
v4 = √ .
7 −1 −1

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Exercise
Let F be the vector sub-space of the Euclidean space R4 spanned
by the following vectors
u1 = (1, 2, 0, 2), u2 = (−1, 1, 1, 1).
1 Use Gramm-Schmidt algorithm to construct an orthonormal
basis of the vector sub-space F
2 Prove that the set F ⊥ = {u ∈ R4 : hu, v i = 0, ∀v ∈ F } is a
vector sub-space of R4 .
3 Find an orthonormal basis of the vector sub-space F ⊥ .

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

Solution
1
1 v1 = u1 , hu2 , v1 i = 1,
3
u2 − hu2 , v1 iv1 = (0, 3, 1, −1) − 31 (−1, 1, 1, 1) = 31 (−4, 1, 3, 1).
1
Then v2 = 3√ 3
(−4, 1, 3, 1).
(v1 , v2 ) is an orthonormal basis of the vector sub-space F .
2 If v1 , v2 ∈ F ⊥ , α, β ∈ R and u ∈ F , then

hαv1 + βv2 , ui = αhv1 , ui + βhv2 , ui = 0.

Then F ⊥ is a vector sub-space of R4 .

3 Let u = (x, y , z, t) ∈ R4 .
 
⊥ hu, u1 i = 0 x + 2y + 2t = 0
u ∈ F ⇐⇒ ⇐⇒
hu, u2 i = 0 −x + y + z + t = 0

Mongi BLEL Inner Product Spaces and Orthogonality

 Inner Product
The orthogonality
The Orthonormal Basis

x = 32 z
 
x + 2y + 2t = 0
⇐⇒
−x + y + z + t = 0 y = − z3 − t
Then u ∈ F ⊥ ⇐⇒ u = − z3 (−2, 1, −3, 0) + t(0, −1, 0, 1).
The vectors e1 = (−2, 1, −3, 0), e2 = (0, −1, 0, 1) is an orthogonal
basis of the vector sub-space F ⊥ .
w1 = √114 e1 , hw1 , e2 i = − √114 ,
1
e2 − he2 , w1 iw1 = 14 (2, 13, 3, 14).
Then ( √114 (−2, 1, −3, 0), 3√142 (2, 13, 3, 14)) is an orthonormal basis
of the vector sub-space F ⊥ .

Mongi BLEL Inner Product Spaces and Orthogonality