21

MTL101:::Lecture 17 and 18
(Inner product, Cauchy-Schwarz Ineq, Gram-Schmidt orthogonalization,best approximation)
(54) Let V be a vector space over F (where F = R or C). A map V ×V → F denoted by (u, v) 7→ hu|vi
is called an inner product on V if the following properties hold:
(a) hu|ui ∈ R and ≥ 0 for each u ∈ V ;
(b) hu|ui = 0 if and only if u = 0;
(c) hau + bv|wi = ahu|wi + bhv|wi;
(d) hu|vi = hv|ui (the complex conjugate).
A vector space together with an inner product is called an inner product space. Observe
that hu|av + bwi = hav + bw|ui = ahv|ui + bhw|ui = āhv|ui + b̄hw|ui = āhu|vi + b̄hu|wi.
Example: 1. Standard inner product on Rn and Cn : The dot product on R3 is an
inner products. Recall
(a1 , a2 , a3 ) · (b1 , b2 , b3 ) = a1 b1 + a2 b2 + a3 b3 .
n
P
In fact for every n ∈ N, the map h(x1 , x2 , . . . , xn )|(y1 , y2 , . . . , yn )i = xi yi is an inner product
i=1
n
on Rn . On Cn , h(x1 , x2 , . . . , xn )|(y1 , y2 , . . . , yn )i =
P
xi yi is an inner product. We leave the
i=1
verification to the students.
Example: 2. Suppose CC [0, 1] is the collection of complex valued continuous mapsR 1 on [0, 1]
which is a complex vector space (i.e. a vector space over C). The map hf |gi := 0 f (s)g(s)ds
is an inner product on C[0, 1]. Guess, how to get an inner product on the vector space CR [0, 1]
of real valued continuous functions on [0, 1].
Lemma: Suppose a > 0. Then at2 + bt + c > 0 for every t ∈ R if and only if b2 − 4ac < 0.
Hint: The quadratic expression aX 2 + bX + c has a real zero if and only if the discriminant
b2 − 4ac ≥ 0. Therefore this expression has no real zero if and only b2 − 4ac < 0. Since
s 7→ as2 + bs + c is a continuous map on R, it does not change sign on R if aX 2 + bX + c has
no real zero. Since a > 0, as2 + bs + c > 0 if aX 2 + bX + c has no real zero. 
Excercise: Suppose a > 0. Show that at2 +bt+c ≥ 0 for every t ∈ R if and only if b2 −4ac ≤ 0.
2
3. What are all inner products on  R :  

a b x2
Define h(x1 , y1 )|(x2 , y2 )i := x1 y1 = ax1 x2 + bx1 y2 + cx2 y1 + dy1 y2 (observe
c d y2
that term containing x1 y1 , x2 y2 , x2i or yi2 is not acceptable!).
Assertion:h(x1 , y1 )|(x2 , y2 )i = ax1 x2 + bx1 y2 + cx2 y1 + dy1 y2 defines an inner product on R2 if
and only if a > 0, b = c and ad − bc > 0.
To prove this if and only if statement, first assume that the expression defines an inner product.
Then h(1, 0)|(1, 0)i > 0 ⇒ a > 0 (by the first property). By symmetry (the last property) we
get b = c (to get this explicitly write down the property.). For any (x, y) ∈ R2 \ {(0, 0)}, we
have ax2 +2bxy +dy 2 = h(x, y)|(x, y)i > 0. Hence we must have (2by)2 −4a·(dy 2 ) < 0 (∀ y 6= 0)
i.e., ad − b2 > 0. We leave the “if” part for the students as an exercise. 
Remark: All the inner products on Rn can be described. But we skip this part since it is
beyond the scope of this course.
Example: 3. Suppose Pn be the real Rvector space of polynomials in X with real coefficients
b
of degree < n. Then the map hf |gi := a f (s)g(s)ds (for f, g ∈ Pn ) is an inner product on Pn
for any a < b ∈ R.
A map V → F denoted by v 7→ ||v|| is called a norm on V if the following conditions are
satisfied.
(a) ||v|| ≥ 0;
(b) ||v|| = 0 if and only if v = 0;
(c) ||av|| = |a|||v|| for any a ∈ F;
(d) ||u + v|| ≤ ||u|| + |v||.
A vector space with a norm is called a normed space.
22

Theorem: (Cauchy-Schwarz inequality) Suppose V is an inner product space. Then for any
u, v ∈ V we have
|hu|vi| ≤ ||u||||v||.

Proof: Suppose F = R. Then 0 ≤ hu + tv|u + tvi = ||u||2 + 2thu|vi + t2 ||v||2 . Using the Lemma
above we have (2hu|vi)2 − 4||u||2 ||v||2 ≤ 0 or |hu|vi| ≤ ||u||||v||.
If F = C, the let α = hu|vi. 0 ≤ hu + tαv|u + tαvi = ||u||2 + t(ᾱhu|vi + αhu|vi) + t2 |α|2 ||v||2 =
||u||2 +2t(|hu|vi|2 )+t2 |hu|vi|2 |2 ||v||2 . Hence, by Lemma above (2|hu|vi|2 )2 −4||u||2 ||v||2 |hu|vi|2 ≤
0 which implies |hu|vi| ≤ ||u||||v||.
Every inner p product space is a normed space. Suppose V is an inner product space, then the
map ||v|| = hv|vi is a norm. Verification of the first three properties is straight forward. We
verify the last property ||u + v|| ≤ ||u|| + ||v||.
||u+v||2 = ||u||2 +(hu|vi+hv|ui)+||v||2 = ||u||2 +2<(hu|vi)+||v||2 ≤ ||u||2 +2|hu|vi|+||v||2 (since
the real part of z= <(z) ≤ |z|). Use Cauchy-Schwarz inequality and get ||u+v||2 ≤ (||u||+||v||)2 .
Remark: If we know the concept of “metric space” we may observe the following: A norm on
a vector spaces defines the following metric: d(u, v) = ||u − v||. Observe that the properties of
a metric follows from the properties of a norm.
(55) A set {u1 , u2 , . . . , un } of an inner product space is called an orthogonal set of vectors if
hui |uj i = 0 if i 6= j and hui |ui i =6 0 for each i and each j. It is not difficult to show that a
orthogonal set of vectors is linearly independent (exercise for the students). Although not every
linearly independet set of vectors is orthogonal, we have a very interesting and usual theorem:
Theorem: (Gram-Schmidt Orthogonalization) Suppose {u1 , u2 , . . . , un } is a linearly inde-
pendent set of vectors in an inner product space. Then there is an orthogonal set {v1 , v2 , . . . , vn }
such that for each i,
span{u1 , u2 , . . . , ui } = span{v1 , v2 , . . . , vi }.
Indeed, v1 = u1 and for 2 ≤ i ≤ n,
i−1
X hui , vj i
vi = ui − vj .
j=1
||vj ||2

Remark: We expect students to verify for each i, span{u1 , u2 , . . . , ui } = span{v1 , v2 , . . . , vi }

and observe that hvi , vj i = 0 for i 6= j. Note that the description of vi uses each vj for j < i. vi is
constucted so that vi is orthogonal to the space span{v1 , v2 , . . . , vi−1 } and span{u1 , u2 , . . . , ui } =
span{v1 , v2 , . . . , vi }. Since by induction,
span{u1 , u2 , . . . , ui−1 } = span{v1 , v2 , . . . , vi−1 }
i−1
P
so we have span{u1 , u2 , . . . , ui } = span{v1 , v2 , . . . , vi−1 , ui }. Define vi = ui + xj vj for xj ∈ F.
k=1
Determine xj so that vi is orthogonal to each of vj (j < i). Take inner product of vi with vj to
hu ,v i
get 0 = hui |vj i + xj ||vj ||2 . Hence xj = − ||vij ||j2 for j < i.

Application: Every finite dimensional inner product space has an orthogonal basis.
Proof: Suppose {u1 , u2 , . . . , un } be a basis of the inner product space V (so that the dimension
is n). Apply Gram-Schmidt process on this set to get the orthogonal set {v1 , v2 , . . . , vn } which
also spans V .
Example: Find an orthogonal basis of R2 with the inner product given by h(x1 , y1 )|(x2 , y2 )i =
x1 x2 + 2x1 y2 + 2x2 y1 + 5y1 y2 . We know that {e1 , e2 } is a basis of R2 . Since he1 , e2 i = 2 6= 0, the
standard basis is not an orthogonal basis. To get an orthogonal basis we use Gram-Schmidt
process. v1 = e1 and v2 = e2 − he||e21|e||12i e1 = e2 − 2e1 . Thus {e1 , e2 − 2e1 } is an orthogonal basis.
 23

(56) Suppose V is an inner product space and W is a proper subspace of V . Given v ∈ V , a vector
w0 ∈ W is said to be a best approximation of v by a vector belonging to W if for each
w ∈ W we have
||v − w0 || ≤ ||v − w||.
Theorem: A vector w0 ∈ W is a best approximation of v ∈ V by a vector belonging to W if
and only if v − w0 is orthogonal to every vector belong to W .
Proof: Let w ∈ W . Then ||v − w||2 = ||(v − w0 ) + (w − w0 )||2 = ||v − w0 ||2 + ||w0 − w||2 +
2<(hv − w0 |w0 − wi). Suppose v − w0 is orthogonal to every vector belong to W . Then
||v − w||2 = ||v − w0 ||2 + ||w0 − w||2 Hence ||v − w|| ≥ ||v − w0 ||.
Conversely suppose w0 is a best approximation of v by a vector belonging to W . Since
||v − w|| ≥ ||v − w0 ||, we have ||w0 − w||2 + 2<(hv − w0 |w0 − wi) ≥ 0. Since any vector u in W
can be expressed in the form w − w0 for some w ∈ W , we have ||u||2 + 2<(hv − w0 |ui) ≥ 0 for
every u ∈ W . Choose u = − hv−w 0 |w0 −wi
||w0 −w||2
(w0 − w). Then you would get

|hv − w0 |w0 − wi|2 hv − w0 |w0 − wihv − w0 |w0 − wi

− 2< ≥0
||w0 − w||2 ||w0 − w||2
2
or − |hv−w 0 |w0 −wi|
||w0 −w||2
≥ 0 which implies hv − w0 |w0 − wi = 0 so that v − w0 is orthogonal to w − w0
for every w ∈ W . 
Proposition: If w1 , w2 are both best approximations of v ∈ V by a vector in W then w1 = w2 .
In particular, best approximation of a vector by a vector in a subspace of an inner product
space is unique.
Proof: By the preceding theorem v − w1 and v − w2 are both orthogonal to every vector in
W . Hence hv − w1 |w1 − w2 i = 0 and hv − w2 |w1 − w2 i = 0. Hence hw1 − w2 |w1 − w2 i =
h(v − w2 ) − (v − w1 )|(w1 − w2 )i = 0 
Theorem: Let W be a subspace of V . Suppose {w1 , w2 , . . . , wn } be an orthogonal basis of W .
Then the best approximation of v ∈ V is given by
hv|wi i
w0 = wi .
||wi ||2
n
P
Proof: Let w0 = xi wi be the best approximation of v by a vector belonging to W . Then
i=1
by the preceding theorem, v −w0 is orthogonal to every vector in W . Therefore, hv −w0 |wj i = 0
n
P
for each 1 ≤ j ≤ n. So hv|wj i − xi hwi |wj i = 0. Since {w1 , w2 , . . . , wn } is an orthogonal basis
i=1
hv|wj i
we have hv|wj i − xj hwj |wj i = 0 so that xj = ||wj ||2
. 
Exercise: Suppose R3 is the inner product space with inner product defined by
{(x1 , y1 , z1 )|(x2 , y2 , z2 )} = x1 x2 + 2x1 y2 + 2x2 y1 + 5y1 y2 + z1 z2 .
Find the shorest distance of v = e1 + e2 + e3 from the subspace W = {(x, y, z) ∈ R3 : z = 0}.
Solution: The set {e1 , e2 } is a basis of W . Use Gram-Schmidt process to get an orthog-
onal basis of W . For instance, {e1 , e2 − 2e1 } is an orthogonal basis of W . Thus the best
approximation of (e1 + e2 + e3 ) is given by w0 = he1 +e||e21+e ||2
3 |e1 i
e1 + he1 +e||e22+e 3 |e2 −2e1 i
−2e1 ||2
(e2 − 2e1 ) =
3 1
e + 1 (e2 − 2e1 ) = e1 + e2 .
1 1
Alternatively, Observe that v − (e1 + e2 ) = e3 is orthogonal to W with respect to the given
inner product space, then by the other theorem for shortest distance we must have w0 = e1 +e2 .
Exercise: Which of the following are inner products on R2 ?
(a) h(x1 , y1 )|(x2 , y2 )i = x1 y1 − x1 y2 − x2 y1 + 2y1 y2 ,
(b) h(x1 , y1 )|(x2 , y2 )i = x1 y1 + 2x1 y2 + 2x2 y1 + y1 y2 ,
(c) h(x1 , y1 )|(x2 , y2 )i = 54x1 y1 + 74x1 y2 + 74x2 y1 + 101y1 y2 ,
(d) h(x1 , y1 )|(x2 , y2 )i = x1 y1 − x1 y2 + x2 y1 + 2y1 y2 .
:::END:::