. Iff and g are in V such that f(v) = 0 implies g(v) = 0, prove that g = 2f for #18, Let $= (fisfrsnfa) be a finite set of linear functionals on V, and let Uj de of fj.Ifg € V is such that the kernel of g contains LJ", Uj, then prove that , combination of elements of S. ( 4.4 INNER PRODUCT SPACES In our discussion of vector spaces the specific nature of F as a field, other than the field, has played virtually no role. In this section we no longer consider vector spaces V 9 fields F rather, we restrict F to be the field of real or complex numbers. In the first case a real vector space, in the second, a complex vector space. We all have had some experience with real vector spaces—in fact both analytic the subject matter of vector analysis deal with these. What concepts used there can we more abstract setting? To begin with, we had in these concrete examples the idea of lengths we had the idea of perpendicularity, or, more generally, that of angle. These became speci the notion of a dot product (often called a scalar or inner product.) i Let us recall some properties of dot product as it pertained to the special case of net dimensional real vectors. Given the vectors v = (X1..%3.3) and w = (V1.2. 5) Where the 2a are real numbers, the dot product of v and w, denoted by v - w, was defined as v- w = 3) 43 x3y3. Note that the length of v is given by vv and the angle 6 between v and wis d r What formal properties does this dot product enjoy? We list a few: cos 6 = 1. v-v>Oand y+ v= 0 if and only if v 2 vew=weys 3. w+ (av-+ Bw) = a(uv) + Blu w); for any vectors u, v, w and real numbers a, p. Everything that has been said can be carried over to complex vector spaces. H get geometrically reasonable definitions we must make some modifications. If we ve w= yyy + XD? +.43N3 for v = (yy,X9,43) and w= (1, Yas )3)> Where the x's and.y's afe numbers, then it is quite possible that v - v = 0 with v # 0; this is illustrated by the 0). In fact, v - v need not even be real. If, as in the real case, we should want v + ¥ 10} how the length of v, we should like that this length be real and that a nonzero vector: zero length. We can achieve this much by altering the definition of dot product slightly. complex conjugate of the complex number ar, returning to the v and w of the par define v - w = x,y, + x2Fp + x,y. For real vectors this new definition coincides the other hand, for arbitrary complex vectors v # 0, not only is vv real, itis in we have the possibility of introducing, in a natural way, a nonnegative182 Chapter 4 Vector Spaces and Modules = impli = 0, prove that g = 1f for some ) in P such that f(v) = 0 implies g(v) = 0, prove < En css t pag fa) be a finite set of linear functionals on V, and let U, denote the of f.lfg ev? is such that the kernel of g contains (J/_, Uj, then prove that g is combination of elements of S. 4.4 INNER PRODUCT SPACES In our discussion of vector spaces the specific nature of F as a field, other than the fact that je field, has played virtually no role. In this section we no longer consider vector spaces V over: : fields F; rather, we restrict F to be the field of real or complex numbers. In the first case V is alley areal vector space, in the second, a complex vector space. We all have had some experience with real vector spaces—in fact both analytic geometry ang the subject matter of vector analysis deal with these. What concepts used there can we carry overtog more abstract setting? To begin with, we had in these concrete examples the idea of length; We had the idea of perpendicularity, or, more generally, that of angle. ‘These became special cases of the notion of a dot product (often called a scalar or inner product.) Let us recall some properties of dot product as it pertained to the special case of the three. dimensional real vectors. Given the vectors v = (x).x3,.%3) and w = (91,2, ¥3), where the x's andy's are real numbers, the dot product of v and w,, denoted by v - w, was defined as v - w it tay 23. Note that the length of vis given by /v-v and the angle @ between v and w is determined by cos6 = ae Nv wow « ) ‘What formal properties does this dot product enjoy? We list a few: 1. v-v>Oand v- v= 0 if and only if v = 0; "| 2 vewew-y; 3. u-(av + pw) = a(u- v) + p(w w); for any vectors u, v, w and real numbers a, f. Everything that has been said can fet geometrically reasonable definitions Sat = Ai) +2292 +393 for v = (x,x2,.x5) and w = O1,¥2,)3), Where the x’s and y’s are €0 “hen itis quite possible that v- v = 0 with v #0; this is illustrated by the vector ¥- Bye oem 3 (uti a(u-w) + yw); ign (av Bw) = BV) + BH w); complex numbers a, f and all complex vectors u,v, w. forall reiterate that in what follows F is either the field of real or complex numbers. ‘We IN. The vector space V over F is said to be an inner product space if there is defined for any two vectors MY € Van clement (u, v) in F such that 1 uv) = 0s 2. (uu) 2 Oand (u, w) = O if and only if u = 0; 3, (au t By, w) = a(u,w) + BOW, W); forany usw € Vand a, p € F. ‘A few observations about properties 1, 2, and 3 are in order. A function satisfying them is called an inner product. If F is the field of complex numbers, property 1 implies that (u u) is rea, and so property 2 makes sense. Using | and 3, we see that (u,av + Bw) = (av + Bw.) = alan +A, 1) = BIT) + BUGETD = Buy) + Blu w- | We pause to look at some examples of inner product spaces. By)s (uy ¥) = By + O82 + EXAMPLE 4.4.1. In F”) define, for u = (a,...,4,) and v = (Bi, -4-a,f,. This defines an inner product on F”). EXAMPLE 4.4.2 In ‘F define for u = (ay, a3) and v = (Bj, B2)s (uv) = 2a, By + aB> + a2) + Aap, tis easy to verify that this defines an inner product on F. EXAMPLE 4.4.3. Let V be the set of all continuous complex-valued functions on the closed unit interval (0, 1). If f(0), g(t) € V, define ik saitail : (00.80) = / foode nes an inner product on V. note an inner product space. written Iv. is defined by Ilvil it to the reader to verify that this defi the remainder of this section V will de Ifv € Vthen the length of v (or norm of + pv,au+ pv) = au, u) + ¢ ef M441 If u,v € Vand a, p € Fthen (au + }e *° Proof. property 3 defining an inner_product space, (au + By, au + Bv) = a(u,g BWaw oe but (u, au + fv) = Gu, u) + Blu, v) and (v, au + pv) = Fv, u) + Bv,v), § these in the expression for (au + Pv, au + fv) we get the desired result. Corollary |lau|l = lal llull- Proof. |jau\|? = (au, au) = aa(u,u) by Lemma 4.4.1 (with v = lull2, taking square roots yields |]au|] = |a_ |lu\|. ). Since ai We digress for a moment, and prove a very elementary and familiar result about real gu equations. LEMMA 4.4.2 Ifa, b, care real numbers such that a > Q and ab2+ 2b) + c 2 0 for all realm 2, then B? < ac. Proof. Completing the squares, a? + 2bd +e Since itis greater than or equal to 0 for all 2, in particular this must be true for & = ~bla. Thus ¢—(b*/a) > 0, and since a > 0 we get b? < ac. We now proceed to an extremely important inequality, usually known as the Schwarz, inequality: THEOREM 4.4.1 if u,v © V then {(u,)| < llull vl. Proof. If u = 0 then both (u, v) = 0 and |lul| lvl] = 0, so that the result is true there. Suppose, for the moment, that (u, v) is real and w #0. By Lemma 4.4.1, for any real 0 < Qu tv, Aut v) = (u,u) + 2(u, A+ (v,v). Let a = (uu), b t the hypothesis of Lemma 4.4.2 is satisfied, so that b? < ac. That is, (u, v)? < (uw, u)(v, v); from th is immediate that |(u, v)| < [lull |lvll. If a =(u, v) is not real, then it certainly is not 0, so that wa is meaningful. Now,1 < Hullilotl lal whence || < ||u| |Iv||. Putting in that a = (u, v) we obtain |(u,v)| < [lull llvll, the desired result. ; cee fs Specific cases of the Schwarz inequality are themselves of great interest. We point out two of lem. . oa 1. If V= F with (u,v) = 1B, + +4,8, 2 = 7 1 a,,, where u = (a,... ,4,) and v = (B;,-.-f,), then Theorem 4.4.1 implies that : pee” d asd ++ aA < (loy]? ++ lanl?) (Al? + Ia?) 2. If V is the set of all continuous, complex-valued functions on [0, 1] with inner product defined by 1 a, soa [ 10a, then Theorem 4.4.1 implies that 1 2 1 1 f FO g@ dt < th [foP at yp lecof ae. fo lo fh The concept of perpendicularity is an extremely useful and important one in geometry. We intro- duce its analog in general inner product spaces. DEFINITION If u,v € V then w is said to be orthogonal to v if (u, v) = 0. , Note that if w is orthogonal to v then v is orthogonal to w, for (v, u) = (iv) = 0 = 0. A set Sin V is said to be orthogonal if any two distinct elements in S are orthogonal to each other. MY EXAMPLE 4.4.4. If V =F” equipped with the inner product in Example 4.4.1, then the standard basis {¢), €2,....@,) is an orthogonal set. seal EXAMPLE 4.4.5. Let V be the space of continuous, complex-valued functions on [0, 1], equip, with the inner product from Example 4.4.3. For each natural number n, let ¢, € V be the a én (x) =e. If n # m, then e, and €,, are orthogonal to each other. ‘< {x € VI(x, w) = 0 for all w € W). LEMMA 4.4.3. W! is a subspace of V.Proof. Ifa,b€ w+ then forall a, p € Fandall w € W, (aa + fb, w) = a(a,w) + abew. Note that Wn W+ = (0), for ‘ eae 7 properties of an i 1 space rule out this possibility unless w = 0. See Preuey nave W-+ W+. Once this is done, the remark made sis y+, Once Rie eink for it will imply that V is the direct sum of W and W+. if w € Wo W* it must be self-orthogonal, that is (w, DEFINITION The set of vectors {v;} in Vis an orthonormal set if 1, Each v; is of length 1 (ie. (vj.¥) = (vis ¥p) = 0. LEMMA 4.4.4 If {v;) is an orthonormal set, then the vectors in {v;} are linearly i w= ayy +o +4, then a; = (Ww, V;) for i= 1,2...) Proof. Suppose that av, +4)v) +-~+4,¥,=0. Therefore 0 = (ay, +--+ 4%, a(0),¥,) + + a4(¥,.¥). Since (vj, vj) = 0 for j # i while (vj, vj) = 1, this equation res 4a; = 0. Thus the v;’s are linearly independent. If w = ayy; + + a,v, then computing as above yields (w, v) Similar in spirit and in proof to Lemma 4.4.4 is LEMMA 4.4.5 if {v,, (wv, v2)v2 = (w,¥)y; — 1p} is an orthonormal set in V and if w € V, then u = w — (Ww, (W, V_)¥, is orthogonal to each of V4, V95---4¥q- Proof. Computing (u,v) for any i = (u/llupll); then {w}, wo) is an orthonormal set. We continue. Let us = —(v3, w)W1 — (v3, W2)W2 + V3; a simple check veri- fies that (uz, 1) = (u3,w2) = 0. Since w;,w3, and v3 are linearly independent (for w), w2 are in the linear span of v, and v3), us # 0. Let w3 = (u3/Ilusll); then {w;,W2,W3} is an orthonormal set. ‘The road ahead is now clear. Suppose that we have constructed w),W,...,Wj, in the linear span of vj... .¥j, Which form an orthonormal set. How do we construct the next one, W;,,? Merely put Cig Wi) 1 — Cigrs Wade ~ Wigs WW, + Vig That yy # Oand that it is orthogo- nal to each of wy, ... ,w; we leave to the reader. Put Wi, = (uis1/ leis II)! In this way, given r linearly independent elements in V, we can construct an orthonormal set having r elements. If particular, when dim V = n, from any basis of V we can construct an orthonormal set having n elements, This provides us with the required basis for V. We illustrate the construction used in the last proof in a concrete case. Let F be the real field and let V be the set of polynomials, in a variable x, over F of degree 2.0r less. In V we define an inner product by: if p(x), g(x) € V, then 1 (PQ), ge) = fs F P(x)q(x) dx. Let us start with the basis v, = 1,v) =x,¥3 =a? of V. Following the construction used, My = —(¥2, 1) + V2, which after the computations reduces to u = x, and so os Tal ina 2b gunnny = (v5.1 = ara Fs and so ry Vv eyes ee ae em 4 llasll 2 i: ») a We mentioned the next theorem earlier as one of our goals. We are now able to prove it THEOREM 4.4.3. If Vis a finite-dimensional inner product space and if W is a subspace then V = W+W*. More particularly, V is the direct sum of Wand wi. Proof. Because of the highly geometric nature of the result, and because it is so basic, we give several proofs. The first will make use of Theorem 4.4.2 and some of the earlier lemmas. The second will be motivated geometrically. First Proof As a subspace of the inner product space V, W is itself an inner product space inner product being that of V restricted to W). Thus we can find an orthonormal set W,-.-5W, W which is a basis of W. If v € V, by Lemma 4.4.5, vy = v — (v1) — (Ys 2)W2 ae is orthogonal to each of wy, ...,w, and so is orthogonal to W. Thus vo & W!, and since (1), + + (0 w,w,), v € W+ W. Therefore V= W+ we. Since Wn W+ = (0), this sum is direct. ‘ Second Proof In this proof we shall assume that F is the field of real numbers. The proof in almost the same way, for the complex numbers; however, it entails a few extra details which tend to obscure the essential ideas used. Let v € V; suppose that we could find a vector wo € W such that ||v — woll < lv— wll w € W. We claim that then (v — Wo, w) = 0 for all w € W, that is, v— wo € W+. If w € W, then wo + w € W, in consequence of which (Y= Wo, ¥— Wo) S (v= (Wo +), v= (Wg +). However, the right-hand side is (w, w) + (v — Wo, v — Wo) — 2(0 = Wo, WW), leading (0 w) < (w, w) for all w € W. If m is any positive integer, since w/m € W we have that 2 Zr myn) =2 (vy 22) < (2.4) = Zon) and so 2(v ~ wo, w) < (1/m)(w, w) for any positive integer m. However, (I/m)(W é Bree neo siatary ee Wr enac 0 c=anrean eae __ Yielding (v ~ wo,w) = 0 for all w € W. Thus v— wy € W4; hence v € wo + WC s To finish the second proof we must prove the existence of a wy € W such th for all w W. We indicate sketchily two ways of proving the4.4 Inner Product Spaces Let u),..., ug be a basis of W; thus any w € Wis of the form w = Ayu +t ely, Let By = (uj,uj) and let y; = (v,u)) for v € V. Thus (v= w, v= w) = Ayuy Dalles V = Dy Wy more = Awe = YZ RdyBy — 2D duy,. This quadratic function in the 4's is nonnegative and 80, by results from the calculus, has a minimum. The 2’s for this minimum, 4, .,...444 give us the desired vector wy = A, uy + + yu, in W, A second way of exhibiting such a minimizing w is as follows. In V define a metric ¢ by ¢(y) = |lx— yl; one shows that ¢ is a proper metric on V, and V is now a metric space. Let $ = {w € WI lly wl < Iv}; in this metric 5 is a compact set (prove!) and so the continuous function 0») = [lv ~ wil defined for w € S takes on a minimum at some point wo S. We leave it to the reader to verify that wo is the desired vector satisfying ||v — woll < [|v — wl for all w € W. ‘The vector w constructed in the second proof of Theorem 4.4.3 is called the best approxi- mation to v in W. This terminology is justified because such a vector wy not only exists, but is also unique, as the following argument shows: If wy and w/, were two vectors in W such that llv — woll < Iv — w]], and lly — wall < lly — wll for all w © W. Then, by the proof, it would follow that v — wo € W* and v — w/, © WL, Therefore, w, — wp = (v ~ Wo) — 4) would belong to W+. But this vector also belongs to W, which naturally implies that Wo = Wh. This best approximation to v in W also has a simple enough expression if we are given an orthonormal basis of W (see Problem 8). Corollary if V is a finite-dimensional inner product space and W is a subspace of V then (W+y* = W. Proof. If w © W then for any w © W+,(w,u) = 0, whence W c (W!)*. Now V= W+ W! and V = w+ + (W+)'; from these we get, since the sums are direct, dim (W) = dim ((W2)*), Since WC (Wy! and is of the same dimension as (W+)*, it follows that W = (W: We end the section with a useful result concerning the dual of an inner product space. To put it in context, observe that if w € V, then the map f : V + F given by f(v) = (», u) is a linear functional. The next result merely states that these are the only linear functionals on V, provided V is finite dimensional. THEOREM 4.4.4. Let V be a finite dimensional inner product space, and f be a linear functional on V(i.e., f € V). Then, there is a unique vector uy € V such that f(v) = (v, ug) for allv € V. Proof. Let us first address the question of uniqueness. Suppose up and 1 are both vectors in V such that f(v) = (v, up) = (v, uy). Then, (ug — tt) Mo — M1) = (Uo = M1, Uo) — (Mg — tty, Wy) = f(g = uy) = S(up — uy) = 0. By definition, this implies that up = uy. Now forthe existence of such a vector. If is the zero functional, then we may clearly take upto be the zero vector. Iff is a nonzero linear functional, then the kernel off, denoted by W, is nontrivial subspace of V. By Theorem 4.4.3, there is a nonzero vector u © W4. Then, if v € V, consider the vector v’ = v— au, where a = f(v)/f(u). Then, fo”) = 0. Therefore, (»’, 4) = 0. Unwrapping this expression, we see that f(v) = (v, uo), where‘holds proving the theorem. een theorem are laid out in the below Problems. PROBLEMS In all the problems V is an inner product space over F. 4. If Fis the real field and V is F®, show that the Schwarz inequality implies that the ‘an angle is of absolute value at most 1. 12. If Fis the real field, find all 4-tuples of real numbers (a, 6, c, d) such that for u = (a;, as) v= (By.by) € F, (u,v) = aay py + bapf + caf, + dah} defines an inner product on, 3. In V define the distance ¢(u, v) from u to v by ¢(u, ¥) = [|u ~ vl]. Prove that (a) (u,v) > 0 and (u,v) = 0 if and only if « = (b) (u,v) = 60.0). ©) (uv) < S(u,w) + COw, v) (triangle inequality). 4, If (W1,..-4%} is an orthonormal set in V, prove that Ms Jo»)? < vt? for any v € v. (Bessel inequality) 5. If V is finite-dimensional and if {w,, ... , w,,,} is an orthonormal set in V such that DY low? = i? i for every v € V, prove that (¥),... 6. If dim V = n and if {w,,...,W,) is an orthonormal set in V, prove that there exist Wmsts sey Wy Such that (Wy, ..., Wms Wmgts +--+ Wy} iS an orthonormal set (and basis of 7. Use the result of Problem 6 to give another proof of Theorem 4.4.3. 8. Let W be a subspace of V, and {w;,w3,....W} be an orthonormal basis for W. If v € rove that the best approximation to v in W is given by the formula wy = Sk (vs 9. In V prove the parallelogram law: rn} must be a basis of V. Ha + vil? + lu — vil? = 2Cu? + [lvll?). ___ Explain what this means geometrically in the special case V = F®), ___ and where the inner product is the usual dot product, be the real functions y = f(x) sat that Vis a y= f(x) isting Ye a