LYNN H.LOOMIS and SHLOMO STERNBERG Department of Mathematics, Harvard University ADVANCED CALCULUS REVISED EDITION ® JONES AND BARTLETT PUBLISHERS. nEditorial, Sales, and Customer Service Offices: Jones and Bartlett Publishers, Inc. (One Exeter Plaza Boston, MA 02116 Jones and Bartlett Publishers International PO Box 1498 London W6 7RS England Copyright © 1990 by Jones and Bartlett Publishers, Ine. Copyright © 1968 by Addison-Wesley Publishing Company, Inc. All rights reserved. No part of the material protected ky this copyright notice may he reproduced or utilized in any form, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without written permission from the copyright owner. Printed in the Us W987 6 rary of Congress Cataloginy in-Publication Data Loomis, Lynn H. Advanced calculus / Lynn H. Loomis and Shlomo Sternberg. —Rev. ed. poem. Originally published: Reading, Ma: ISBN 0-86720-122-3, 1. Caleulus. I. Sternberg, Shlomo, II. Title QA303.L87 1990 515--de20 89-15620 Cie Addison-Wesley Pub, Co., 1968.PREFACE ‘This books based on an honors course in advanced ealeulus that we gave in the 1960's. The foundational material, presented in the unstarred sections of Chap- ters I through 11, was normally covered, but different applications of this basic material were stressed from year to year, and the book therefore contains more material than was covered in any one year. It ean accordingly be used (with omissions) as a text for a year's course in advanced caleulus, or as a text for & three-semester introduction to analysis. ‘These prerequisites are a good grounding in the ealeulus of one variable from a méthematically rigorous point of view, together with some acquaintance with linear algebra. The reader should be familiar with limit and continuity type arguments and have a certain amount of mathematical sophistication. As possi ble introductory texts, we mention Differential and Integral Calculus by R. Cou rant, Caleulus by T. Apostol, Caleulus by M. Spivak, and Pure Mathematics by G. Hardy. The reader should also have some experience with partial derivatives, In overall plan the book divides roughly into a first half which develops the calculus (principally the differential calculus) in the setting of normed vector spaces, and a second half which deals with the calculus of differentiable manifolds. Vector space calculus is treated in two chapters, the differential caleulus in Chapter 3, and the basic theory of ordinary differential equations in Chapter 6 ‘The other early chapters are auxiliary. The first two chapters develop the neces- sary purely algebraic theory of vector spaces, Chapter 4 presents the material (on compactness and completeness needed for the more substantive results 0° the calculis, and Chapter 5 contains a brief account of the extra structure en: countered in sealar product spaces. Chapter 7 is devoted to multilinear (tensori algebra and is, in the main, a reference chapter for later use. Chapter 8 deals with the theory of (Riemann) integration on Euclidean spaces and includes (in exercise form) the fundamental faets about the Fourier transform. Chapters § and 10 develop the differential and integral caleulus on manifolds, while Chapter 11 treats the exterior caleulus of E. Cartan, ‘The fist eleven chapters form a logical unit, each chapter depending on the results of the preceding chapters. (Of course, many chapters contain material that ean be omitted on first reading; this is generally found in starred sections.‘On the other hand, Chapters 12, 13, and the latter parts of Chapters 6 and 11 are independent of each other, and are to be regarded as illustrative apolieations of the methods developed in the earlier chapters. Presented here are elementary Sturm-Liouville theory and Fourier series, elementary differential geometry, potential theory, and classical mechanics. We usually covered only one or two of these topies in our one-year course. We have not hesitated to present the same material more than once from different points of view. For example, although we have selected the eontraction, ‘mapping fixed-point theorem as our basic approach to the implicit-funetion theorem, we have also outlined a “Newton's method” proof in the text and have sketched still a third proof in the exercises. Similarly, the ealculus of variations is encountered twwiee—onee in the context of the differential ealeulus of an infinite-dimensional vector space and later in the context of classical mechanics. ‘The notion of a submanifold of a vector space is introduced in the early chapters, while the invariant definition of a manifold is given later on. In the introductory treatment of vector space theory, we are more eareful ‘and precise than is customary. In fact, this level of precision of language is not ‘maintained in the later chapters. Our feeling is that in linear algebra, where the concepts are so clear and the axioms s0 familiar, it is pedagogieally sound to illustrate various subtle points, such as distinguishing between spaces that are normally identified, discussing the naturality of various maps, and so on, Later oon, when overly precise language would be more cumbersome, the reader should be able to produce for himself a more preeise version of any assertiors that he finds to be formulated too loosely. Similarly, the proofs in the first few chapters ‘are presented in more formal detail. Again, the philosophy is that once the student has mastered the notion of what constitutes a formal mathematical proof, itis safe and more convenient to present arguments in the usual mathe- matical colloquialisms. While the level of formality decreases, the level of mathematical sophist cation does not. ‘Thus inereasingly abstract and sophisticated matiematical ‘objects are introduced. It has been our experience that Chapter 9 contains the concepts most difficult for students to absorb, especially the notions of the tangent space to a manifold and the Lie derivative of various objects with respect to a veetor field.‘There are exercises of many different. kinds spread throughout the book. Some are in the nature of routine applications. Others ask the reader to fill in or extend various proofs of results presented in the text. Sometimes whole topics, such as the Fourier transform or the residue calculus, are presented in exercise form. Due to the rather abstract nature of the textual material, the stu- dent is strongly advised to work out as many of the exercises as he possibly can, Any enterprise of this nature owes much to many people besides the authors, but we particularly wish to acknowledge the help of L. Ahifors, A. Gleason, R. Kulkarni, R. Rasala, and G. Mackey and the general influence of the book by Dieudonné. We also wish to thank the staff of Jones and Bartlett for their invaluable help in preparing this revised edition. Cambridge, Massachusetts LHL. 1968, 1989 SS.hapter 1 Chapter 2 CONTENTS Loxie: quantifiers ‘The logical connectives Negations of quantifiers Sets Restricted variables Ordered pairs and relations Funetions and mappings Product sets; index notation Composition Duality ‘The Boolean operations Partitions and equivalence relations ‘Vector Spaces Fundamental notions Veetor spaces and geometry Product spaces and Hom(V, 1") Affine subspaces and quotient spaces Direot sums Bilinearity Finite-Dimensional Vector Spaces Bases Dimension ‘The dual space Matrices ‘Trace and determinant Matis computations ‘The diagonalization of a quadratic form “The Ditreremtial Cal Review in R Norms Continuity a 36 43 56 or 102 a 17 121 126Chapter 5 ‘Equivalent norms Tnfinitesimals ‘The differential Directional derivatives; the mean-value theorem, ‘The differential and product spaces ‘The differential and R* . . Elementary applications ‘The implicit-funetion theorem. Submanifolds and Lagrange multipliers Functional dependence ce Uniform continuity and function-valued mappings ‘Tho caleulus of variation ‘The second differer The Taylor formula wctness and Completeness spaces; open and closed sets ‘Topology bio Sequential convergence Sequential compactness ‘Compaetness and unifon Equicontinuity Completeness: ‘A first look at Banach algebras ‘The contraction mapping fixed-point theorem ‘The integral of a parametrized are ‘The complex number system ‘Weak methods Scalar Product Spaces Scalar products Orthogonal projection Self-adjoint transforms Orthogonal transforms Compact transformations nd the elassifcation of critical points 132 136 140 M6 152 156 161 164 12 15 179 182 186 191 195 201 202 210 215 216 223 228 236 248 252 257 262 264Differential Equations ‘The fundamental theorem Differentiable dependence on parameters ‘The linear equation : ‘The nth-order linear equation Solving the inhomogeneous equation ‘The boundary-value problem. Fourier series ‘Multilinear Funetionals Bilinear functionals Multilinear functionals Permutations ‘The sign of a permutation ‘The subspace @ of alternating tensors ‘The determinant ‘The exterior algebra. - Exterior powers of sealar produet space ‘The star operator Integration Introduction Axioms Rectangles and paved sets ‘The minimal theory ‘The minimal theory (continued) Contented sets When is a set contented? Behavior under linear distortions ‘Axioms for integration Integration of contented functions ‘The change of variables formula Successive integration Absolutely integrable functions Problem set: The Fourier transformChapter 9 Chapter 10 Chapter 1. Chapter 12 1 2 3 4 Differentiable Ma lds Atlases Functions, convergence Differentisble manifolds ‘The tangent space Flows and veetor fields ie derivatives inear differenti Computations with coordinates Riemann meteies ‘The Integral Caleulus on Manifolds Compactness Partitions of Densities, Volume density of « Riemann metric Pullback and Lie derivatives of densities ‘The divergence theorem More complicated domains ity Exterior Calculus Exterior differential forms Oriented marifolds and the integration of exterior differential forms ‘The operator d Stokes’ theorem Some illustrations of Stokes’ theorem ‘The Lie derivative of a differential form Appendix I. *Veetor analysis” Appendix IT, Elementary differential geometry of surfaces in E® Potential Theory in E* Solid angle Green's formalas ‘The maximum principle Green's funetions 364 367 313 376 383 390 393 307 403 405 408 au 416 419 424 429 433 438 442 449 452 487 450 am 476. 477 4795 6 8 9 10 u 2 13 Chapter 13 1 2 3 4 5 6 7 8 9 10 u 2 13 4 6 ‘The Poisson integral formula Consequences of the Poisson integral formula Harnaek’s theorem Subharmonie funetions Dirichlet’s problem Behavior near the boundary Dirichlet’s principle Physical applications Problem set: The caleulus of residues Classical Mechanies ‘The tangent and cotangent bundles Equations of variation ‘The fundamental linear differential form on T*(M) . ‘The fundamental exterior two-form on T*(M) Hamiltonian mechanies ‘The eentral-foree problem The two-body problem Lagrange's equations Variational principles Goodesie cootdinates Euler’s equations poeta Small oscillations Small oscillations (continued) Canonical transformations Sclected References Notation Index Index 482 485 487 480 491 495 499 501 503 bu 513 515 517 521 528 530 582 837 Bal 54 553 558 500 372 315CHAPTER 0 This preiminary chapter contains a short exposition of the set theory that forms the substratum of mathematical thinking today. It begins with a brief discussion of logic, so that set theory can be discussed with some precision, and continues with a review of the way in which mathematical objects can be defined as sets. The chapter ends with four sections which treat specific set-theoretic topies. Tt is intended that this material be used mainly for reference. Some of it ‘will be familiar to the reader and some of it will probably be new. We suggest that he read the chapter through “lightly” at first, and then refer back to it for details as needed. 1. LOGIC: QUANTIFIERS ‘A statement is a sentence which is true or false as it stands. Thus ‘1 < 2 and “4-43 = 5 are, respectively, true and false mathematical statements. Many sentences oecurring in mathematies eontain variables and are therefore not trae or false as they stand, but become statements when the variables are given values. Simple examples are ‘x <4’, ‘x < y’, ‘x isan integer’, ‘82? + y® = 10. Such sen:ences will be ealled slafement frames. If P(z) isa frame containing the one variable ‘’, then P() is the statement obtained by replacing ‘2’ in P(z) by the numeral ‘5". Vor example, if P(e) is ‘x <4’, then P(5) is ‘5 < 4’, PCV?) is ‘V2 <4, and so on "Another way to obtain a statement from the frame P(2) isto assert that P(e) is always true. We do this by prefixing the phrase ‘Tor every 2’. Thus, for every z,x <4’ isa false statement, and ‘for every 2,2” —1 = (2 — 1)@+1)'isa true statement. This prefixing phrase is called a wniversal quantifier. Sya- onymous phrases are ‘for each 2’ and ‘Tor all 2’, and the symbol customarily ‘used is ‘(Wz)’, which ean be read in any of these ways. One frequently presents sentences containing variables as being always true without explicitly writing the universal quantifiers. For instance, the associative law for the addition of ‘numbers is often written atta)=@tyts where it is understood that the equation is true for all x, y and z, ‘Thus the 12 wmopuerio ou actual statement being made is (ver) (¥y)(wa)[e + (y + 2) = (ety) + ale Finally, we can convert the frame P(z) into a statement by asserting that it is sometimes true, which we do by writing ‘there exists an « such that P(z). This process is called existential quantification. Synonymous prefixing phrases here are ‘there is an z such that’, ‘for some 2’, and, symbolically, ‘(31)’ The statement ‘(¥2)(« < 4)’ still contains the variable ‘2’, of course, but ‘2’ is no longer Jree to be given values, and is now called a bound variable, Roughly speaking, quantified variables are bound and unquantified variables are free. The notation ‘P(2)’ is used only when ‘2’ is free in the sentence being discussed. ‘ow suppose that we have a sentenee P(e, y) containing two free variables. Clearly, we need two quantifiers to obtain a statement from this sentence. This brings us to a very important observation. If quantifiers of both types are used, then the order in which they are written affects the meaning of the statement; (ay)(¥2)P(z, ») and (Wz) @y)PCe, v) say different things. ‘The first says that one y can be found that works for all 2: “there exists a y such that for all z...”. ‘The second says that for each 2 a y can be found that works: “for each x there exists a y such that...” But in the second ease, it may very well happen that when 2 is changed, the y that ean be found will also have to be changed. ‘The existence of a single y that serves for all x is thus the stronger statement. For example, itis true that (¥z)(3y)(e < y) and false that (3y)(¥2)(x < y). ‘The reader must be absolutely clear on this point; his whole mathematical future is at stake. ‘The seeond statement says that there exists a y, eall it yo, such that (vr)(e < yo), that is, such that every number is less than yo. This is false; vo + 1, in particular, is not less than yo. ‘The first statement says that for each x wwe ean find a corresponding y. And we can: take y 1 On the other hand, among a group of quantifiers of the same type the order does not affect the meaning, Thus (¥2)(¥y)’ and ‘(¥y)(¥2)' have the same mean- ing. We often abbreviate such clumps of similar quantifiers by using the quan- tification symbol only onee, asin “(¥r, )’, which ean be read ‘ior every «and y ‘Tihus the strietly correct ‘(¥z)(¥u)(V2)l + (y + 2) = (e+ y) + 2} receives the slightly more idiomatic rendition ‘(¥z, y, ae + (y+ 2) = (@-+u) +2). The situation is clearly the same for a group of existential quantifiers ‘The beginning student generally feels that the prefixing phrases ‘for every 2 there exists a y such that’ and ‘there exists a y such that for every 2” sound artificial and are unidiomatic. ‘Thisis indeed the ease, but this awkwardness the price that has to be paid for the order of the quantifiers to be fixed, so that the ‘meaning of the quantified statement is clear and unambiguous. Quantifiers do occur in ordinary idiomatic discourse, but their idiomatic oecurrences often house ambiguity. The following two sentences are good examples of such ambiguous idiomatic usage: “Every 2 is less than some y” and “Some y is greater than every 2”. Ifa poll were taken, it would be found that most men on thea2 ‘THE LOGICAL cONNEeTIVES 3 street feel that: these two sentences say the same thing, but half will fel that the ‘common assertion is false and half will think it true! ‘The trouble here is that the matrix is preceded by one quantifier and followed by another, and the poor reader doesn't know which to take as the inside, or first applied, quantifier. ‘The ‘two possible symbolic renditions of our first sentence, ‘{(¥z)(e < y)](3y)’ and “(vx){(x < y)(2y))’, are respectively false and true. Mathematicians do use hanging quantifiers in the interests of more idiomatie writing, but only if they are sure the reader will understand their order of application, either from the context or by comparison with standard usage. Tn general, « hanging quantihier ‘would probably be read as the inside, or first applied, quantifier, and with this ‘understanding our twvo ambiguous sentences become true and false in that order: Alter this apology: the reader should be able to tolerate the definition of sequential convergence. It involves three quantifiers and runs as follows: The sequence {tu} converges to x if (¥e)(3N)(¥n)(if n > N then lem — x] < ©. In exactly the same format, we define a funetion f to be continuous at a if (¥6)(28)(¥a)(if |e — al < 8 then [f(2) — J(@)| < &). We often omit an inside universal quantifier by displaying the final frame, so that the universal quanti- fication is understood. ‘Thus we define f to be continuous at a if for every € there is @ @ sueh that if e—al <4, then [fta) —f@)| < We shall study these definitions later. We remark only that it is perfeetly possible to build up an intuitive understanding of what these and similar quantified statements actually say. 2, TIE LOGICAL CONNECTIVES ‘When the word ‘and’ is inserted between two sentences, the resulting sentence is true if both constituent sentences are true and is false otherwise. ‘That is, the “truth value”, T or F, of the compound sentence depends only on the truth values of the constituent sentences. We ean thus deseribe the way ‘and! acts in compounding sentences in the simple “truth table” PQ Pant TT T TOF ¥ ror F FOP F where ‘P? and ‘Q' stand for arbitrary statement frames. Words like ‘and’ are called logical connectives. It is often convenient to use symbols for conncetives, and a standard symbol for ‘and? is the ampersand ‘&’. Thus ‘P & Q’ is read "PandQ.4 awtnopverion 02 Another logical connective is the word ‘or’. Unfortunately, this word is used ambiguously in ordinary discourse. Sometimes it is used in the exclusive sense, where ‘P or Q’ means that one of P and Q is true, but not both, and sometimes it is used in the inclusive sense that at least one is true, and possibly both are true, Mathematics cannot tolerate any fundamental ambiguity, and in mathe matics ‘or’ is alicaye used in the latter way. We thus have the truth table PQ Pog TT T TF T ror FOF oF ‘The above two connectives are binary, in the sense that they combine two sentences to form one new sentence. ‘The word ‘not’ applies to one sentence and really shouldn't be considered a connective at all; nevertheless, it is ealled a unary connective. A standard symbol for ‘not! is '~'. Its truth table is obviously P TOF T In idiomatic usage the word ‘not’ is generally buried in the interior of a sentence. We write ‘x is not equal to y’ rather than ‘not (x is equal to y)’. However, for the purpose of logical manipulation, the negation sign (the word ‘not? or & symbol like ‘~') precedes the sentence being negated. We shall, of course, continue to write ‘x + y’, but keep in mind that this is idiomatic for ‘not (# = y)! or ‘~(2 =v)". We come now to the troublesome ‘if..., then...” connective, which we write as either ‘if P, then Q’ or ‘P => Q’. ‘This is almost always applied in the universally quantified context (¥2)(P() = Q(z), and its meaning is best ‘unraveled by a study of this usage. We consider ‘if < 3, then x < 5! to bea true sentence. More exaetly, itis true for all z, s0 that the universal quantifi- cation (¥z)(¢ <3 2 <5) is a true statement. ‘This conclusion forces us to agree that, in particular, ‘2 < 32 <5’, 4<3—44<5', and '6<3— 6 < 5’ are all true statements. The truth table for ‘=’ thus contains the values entered below. PQ P=Q , T TOF - roT T FOF oT02 On the other hand, we consider ‘z <7=+.2 <5! to be a false sentence, and therefore have to agree that ‘6 < 7G < 5 is false. Thus the remaining row in the table above gives the value ‘F” for P = Q. Combinations of frame variables and logical connectives such as we have been considering are called truth-functional forms. We can further combine the clementary forms such as ‘P = Q' and ‘~P" by connectives to construct com- posite forms such as ‘~(P => Q)' and (P= @) & (Q=> PY. A senteneo has a iziven (truth-funetional) form if it ean be obtained from that form by substitution ‘Thus ‘z < yor ~(e < y)' has the form ‘P or ~P’, since itis obtained from this form by substituting the sentenee ‘x < y for the sentence variable ‘P’. Com- posite truth-funetional forms have truth tables that ean be worked out by combining the elementary tables. For example, ‘~(P = Q)’ has the table below, the truth value for the whole form being in the column under the conneetive which is applied last ('~? in this example. PQ ~P=®@ T in) T nF Fr ir) F rT ‘Thus ~(P = Q) is true only when P is true and @ is false. ‘A truth-funetional form such as ‘P or (~P)' which is always true (.e., has only ‘T’ in the final column of its truth table) is ealled a tautology or a tautologous form. The reader ean check that (P&(P3Q)3Q and (P3Q) &Q=R)) >= (PR) are also tautologous. Indeed, any valid principle of reasoning, that does not, involve quantifiers must be expressed by a tautologous form, ‘The ‘if and only if? form ‘P <> Q, or ‘P if and only if Q, or ‘P iff Q abbreviation for ‘(P = Q) & (Q=> PY’. Its truth table works out to be P T T t F ‘That is, P B is tautologous, and conversely.6 — wrnopverioN oO Replacing a sentence obtained by substitution in a form A by the equivalent sentence obtained by the same substitutions in an equivalent form B is a device much used in logical reasoning. ‘Thus to prove a statement P true, it suffices to prove the statement ~P false, since ‘P’ and ‘~(~P)' are equivalent. forms. Other important equivalences are ~(P orQ) © (~P) & (~0), (P3Q © Qor(~P), (PQ) © P&(~Q), A bit of conventional sloppiness which we shall indulge in for smoother idiom is the use of ‘if? instead of the correet, in definitions. We define f to be continuous at if so-and-so, meaning, of course, that fis continuous at x if and only if so-and-so. This eauses no difficulty, since it is clear that ‘if and only if’ is meant when a definition is being given. 3. NEGATIONS OF QUANTIFIERS ‘The combinations ‘~(¥2)' and ‘(3r)~’ have the same meanings: something is not always true if and only if it is sometimes false. Similarly, ‘~(3y)’ and ‘(¥y)~" have the same meanings. These equivalences can be applied to move a negation sign past each quantifier in a string of quantifiers, giving the following important practical rule: In taking the negation of a statement beginning with a string of quantifiers, ‘we simply change eack quantifier to the opposite kind and move the negation sign to the end of the string. ‘Thus ~(¥z)(3y)(¥2)P(z, ys 2) #9 (3z)(¥y)(32)~PC, v2). ‘There are other principles of quantifieational reasoning that can be isolated and which we shall oceasionally mention, but none seem worth formalizing here. 4. SETS It is present-day practice to define every mathematical object as a set of some Kind or other, and we must examine this fundamental notion, however briefly. ‘A set isa collection of objects that is itself considered an entity. ‘The objects the collection are called the elements or members of the set. ‘The symbol for is a member of is‘€’ (a sort of capital epsilon), so that ‘z € A’ is read “r isa member of A”, “r is an element of A”, “r belongs to A*, or “2 is in A". We use the equals sign ‘=’ in mathematics to mean logical identity; A = B means that A is B. Now a set A is considered to be the same object as a set B if and only if A and B have exactly the same members. That is, ‘A = B” means that (Vz)(e € A #2 € B),4 sem 7 ‘We say that a set A is a subset of a set B, or that A is included in B (or that, Bis a superset of A) if every clement of A is an element of B. The symbol for inclusion is ‘C’. Thus ‘AC B" means that (Vz)(c¢ A => 2B). Clearly, (A B) = (ACB) and (BCA). ‘This isa frequently used way of establishing set identity: we prove that A = B by proving that A.C B and that BC A. If the reader thinks about the above ‘equivalence, he will see that it depends first on the equivalence of the truth-fune- tional forms ‘P ®)(3n&). Some restriction was implieit on page 1. If the reader agreed that (v2)(c? — 1 = @ = 1)(@ + 1)) was true, he probably took z to be a real number.06 ORDERED PAIRS AND RELATIONS — 9 6. ORDERED PAIRS AND RELATIONS Ordered pairs are basic tools, as the reader knows from analytic geometry. According to our general principle, the ordered pair is taken to be a certain set, but here again we don’t eare which particular set it is so long as it guarantees the crucial characterizing property = @ x= aand ‘Thus <1, 3> % <3,1>. ‘The notion of a correspondence, or relation, and the special ease of a map- ping, or function, is fundamental to mathematics. A eorrespondenee is a pairing of objects such that given any two objects x and y, the pair <2, y> either docs or does not correspond. A particular correspondence (relation) is generally presented by a statement frame P(z, y) having two free variables, with z and y corresponding if any only if P(e, y) is true. Given any relation (correspondence), the set of all ordered pairs <2, y> of corresponding elements is called its graph ‘Now a relation isa mathematical object, and, as we have said several times, it is current practice to regard every mathematical object as a set of some sort or other. Since the graph of a relation is. set (of ordered pairs), itis efficient and mmary to take the graph fo be the relation. ‘Thus @ relation (correspondence) is simply a set of ordered pairs. If R is a relation, then we say that x has the relation 2 to y, and we write ‘rRy’, if and only if €R. We also say that x corresponds to y under R. ‘The set of all frst elements occurring in the ‘ordered pairs of a relation Wis called the domain of I and is designated dom & or 9(R). Thus dom R= (x: (y) €R} ‘The set of second elements is called the range of R: range R = {y: (3x) <2, y> ER} ‘The inverse, R~, of a relation R is the set of ordered pairs obtained by reversing those of R: R A statement frame P(z, y) having two free variables actually determines a pair of mutually inverse relations R&S, called the graphs of P, as follows: R= {xauri PGW), SH {:2 : 2A &y EB} of all ordered pairs with first element in A and second element in B is ealled the Cartesian product of the { i <> ER}10 awrnopvcrion o7 sets A and B. A relation R is always a subset of dom R X range R. If the two “factor spaces” are the same, we can use exponential notation: A? = Ax A. ‘The Cartesian product R? = RX R is the “analytic plane”. Analytic geometry rests upon the one-to-one coordinate correspondence between R? and the Euclidean plane E? (determined by an axis system in the latter), which ‘enables us to treat geometric questions algebraically and algebraic questions geometrically. In particular, since a relation between sets of real numbers is a subset of R?, we can “picture” it by the corresponding subset of the Euclidean plane, or of any model of the Euclidean plane, such as this page. A simple Cartesian product is shown in Fig, 0.1 (A U Bis the union of the sets A and B). TT T 1B when A=(1,2}42h, 3} and B=[1,19}v(2} Fig. 0. Fig. 02 If R isa rlation and A is any set, then the restriction of R to A, RTA, is the subset of R consisting of those pairs with first element in A RIA {: eRandze A}. ‘Thus R[ A = Rn (A x range R), where CA D is the intersection of the sets Cand D. If Risa relation and A is any set, then the image of A under R, R(A), is the set of second clements of ordered pairs in 2 whose first elements are in A: RIA} = (y: Gree A & ER}. ‘Thus RA] = range (R [ A), as shown in Fig, 0.2. 7. FUNCTIONS AND MAPPINGS A function is relation f such that each domain element 2 is paired with exactly fone range element y. This property can be expressed as follows: efand <227 Ef > yon07 FUNCTIONS AND MAPPINGS IL ‘The y which is thus uniquely determined by f and z is designated f(z): y=J@) @ ef One tends to think of a function as being active and a relation which is not ‘function as being passive. A function f acts on an element z in its domain to give f(x). We take x and apply f to it; indeed we often call a function an operator. On the other hand, if 2 is a relation but not a function, then there is in general ‘no particular y related to an element x in its domain, and the pairing of x and y ‘is viewed more passively. We often define a function f by specifying its value f(z) for each 2 in its domain, and in this eonneetion a stopped arrow notation is used to indicate the pairing. Thus x ++ x* is the function assigning to each number x its square z*. J <201 <2, 4 431 <> If we want it to be understood that f is this funetion, we ean write “Consider the funetion f:2++ 2". The domain of f must be understood for this notation to be meaningful If isa function, then f-" is of course a relation, but in general it is not a function, For example, if f is the function 2+ z*, then f—* contains the pairs <4,2> and <4, —2> and so is not a funetion (see Fig. 0.8). If" és a fune- tion, we say that fis one-to-one and that fis @ one-to-one correspondence between its domain and its range. Each x € dom f eorresponds to only one y € range (Fis function), and each y € range f corresponds to only one x € dom f (J! is # function). The notation PAB is read “a (the) funetion fon A into B” or “the function f from A to B’. The notation implies that f is a funetion, that dom f= A, and that range fc B. ‘Many people feel that the very notion of function should include all these ingredients; that is, a funetion should be considered an ordered triple < f, A, B>, where f is a funetion according to our more limited definition, A is the domain12 iwrmopucrion 08 of f, and B isa superset of the range of f, which we shall eall the codomain of fin this context. We shall use the terms ‘map’, ‘mapping’, and ‘transformation’ for such a triple, so that the notation f: A — B in its totality presents © mapping. ‘Moreover, when there is no question about which set is the codomain, we shall often call the function f itself a mapping, since the triple is then determined by f. The two arrow notations can be combined, as in: “Define FR Roy rez, A mapping f: A — B is said to be injective if f is one-to-one, surjective if range f= B, and bijective if it is both injective and surjective. A bijective mapping f: A — B is thus a one-to-one correspondence between its domain A and its eodomain B. Of course, a function is always surjective onto its range R, and the statement that Jis surjective means that R = B, where B is the under- stood codomain, 8, PRODUCT SETS; INDEX NOTATION One of the characteristic habits of the modern mathematician is that as soon as, a new kind of object has been defined and discussed a little, he immediately looks at the set of all such objects. With the notion of a function from A to S well in hand, we naturally consider the set of all funetions from A to S, which we designate S4. Thus Ris the set of all real-valued functions of one reel variable, and S™ is the set of all infinite sequences in S. (It is understood that an infinite sequence is nothing but a function whose domain is the set Z* of all positive integers.) Similarly, if we set % = {1,...,n}, then S* is the set of all finite sequences of length m in S. IB isa subset ofS, then its characteristic function (relative to 8) is the fune- tion on S, usually designated x», which has the constant value 1 on B and the constant value 0 off B. ‘The set of all characteristic functions of subsets of S is, thus 2° (since 2= {0,1}). But because this collection of functions is in a natural one-to-one correspondence with the collection of all subsets of S$, x2 corresponding to B, we tend to identify the two collections. ‘Thus 2° is also interpreted as the set ofall subsets of S. We shall spend most of the remainder of this section discussing further similar definitional ambiguities which mathe- maticians tolerate. ‘The ordered triple is usually defined to be the ordered pair <,2>. The reason for this definition is probably that a funetion of ‘two variables z and y is ordinarily considered a funetion of the single ordered pair variable , so that, for example, a real-valued funetion of two real variables is a subset of (RX R) X R. But we also consider such a function a subset of Cartesian 3-space R°. Therefore, we define R® as (RX R) x Rs that is, we define the ordered triple ,2>. On the other hand, the ordered triple <2, y, 2 could also be regarded as the finite sequence {<1, 2>, <2, y>, <3,2>}, which, of course, is different object. These two models for an ordered triple serve equally well, and, again,08 PRODUCT SETS; INDEX NoTATION 13 tathematicians tend to slur over the distinction. We shall have more to say con this point later when we discuss natural isomorphisms (Section 1.6). For the moment we shall simply regard R® and R® as being the same; an ordered e is something which can be “viewed” as being cither an ordered pair of ‘which the first clement is an ordered pair or asa sequence of length 3 (or, for that matter, as an ordered pair of which the second element is an ordered pair). Similarly, we pretend that Cartesian 4-space R* is K¥, R* X R*, or REX RE = Rx ((RXR) XR), ete. Clearly, wo are in effect assuming an associative law for ordered pair formation that we don’t really have. ‘This kind of ambiguity, where we tend to identify two objects that really are distinet, is a necesary corollary of deciding exactly what things are. It is one of the prices we pay for the precision of set theory; in days when mathematics was vaguer, there would have been a single fuzzy notion. ‘The device of indices, which is used frequently in mathematics, also has am- Diguous implications which we should examine. An indexed collection, as a set, is nothing but the range set ofa funetion, the indexing function, and a particular indexed object, say x,, is simply the value of that function at the domain element i. If the set of indices is I, the indexed set is designated {2;:i€ Z} or {x} ier (or {zj:721 in case I = Z*). However, this notation suggests that we view the indexed set as being obtained by letting the index run through the index set J and collecting the indexed objects. ‘That is, an indexed set is viewed as being the set together with the indexing function. ‘This ambivalence is reflected in the fact that the same notation frequently designates the mapping. ‘Thus we refer to the sequence {r,}7-1, where, of course, the sequence is the mapping n+ 2.. We believe that if the reader examines his idea of a sequence he will find this ambiguity present. He means neither just the set nor just the mapping, but the ‘mapping with emphasis on its range, or the range “together with” the mapping. But since set theory cannot reflect these nuanees in any simple and graceful way, we shall take an indexed set fo be the indexing function. Of course, the same range object may be repeated with different indices; there is no implication that an indexing is one-to-one. Note also that indexing imposes no restriction on the set being indexed; any set can at least be self-indexed (by the identity function). Except for the ambiguous ‘{;: i € J}’, there is no universally used notation for the indexing function. Since x; is the value of the function at i, we might think of ‘2/' as another way of writing ‘2(()', in which ease we designate the funetion ‘z’ or ‘x’. We certainly do this in the case of ordered n-tuplets when we say, “Consider the n-tuplet x = <21,...,z,>". On the other hand, there is no compelling reason to use this notation. We can call the indexing funeti anything we want; if it is f, then of course f(i) = 2; for all i. We come now to the general definition of Cartesian product. Earlier we argued (in a special case) that the Cartesian product A x B x C is the set of all ordered triples x = <2, 12, 23> such that 2, € A, x2 €B, and zg EC. More generally, A; X Ag X-+-+ X An, or [Tar 4s is the set of ordered n- tuples x = such that 2; € A; fori = 1,...,n. If we interpret,14 isrropucrion 09 an ordered n-tuplet as a function on = {1,..., n}, we have Tihs Avis the set of all functions x with domain % such that 2€ A; for all iE 7. This rephrasal generalizes almost verbatim to give us the notion of the Cartesian product of an arbitrary indexed collection of sets. Definition. ‘The Cartesian product: [TscrS; of the indexed colletion of sets {See 1) is the set ofall functions J with domain the index set T buch that fQ) € S; for all FZ. ‘We ean also use the notation [I{S;:7€ 7} for the product and f; for the value (i). 9. COMPOSITION If we are given maps f: A + B and g: BC, then the composition of 9 with f, gf, is the map of A into C defined by (oe N@) = (fe) forall zea. ‘This is the function of a function operation of elementary ealeulus. Iffand g are the maps from R.to R defined by f(z) = 23 +1 and g(e) = 2, then f(z) ©9941 = 2? +1, and go f(z) = (29 +1)? = 27 + 2x! +1, Note that the codomain of f must be the domain of g in order for g fto be defined. This operation is perhaps the basic binary operation of mathematies. Lemma. Composition satisfies the associative law: foeh=(Seaek Proof. (f2(g2h))(z) = s(ge bye) = S(o(h@))) = (Fe g)(hie)) = (Jeg) °h)(z) for all x dom h. O If A isa set, the identity map I4: A — A is the mapping taking every 2A toitsell. Thus 24 = { : 2 € A}. If f maps A into B, ther clearly fela=f= nef. Ifg:B — A issuch that gf = I4, then wo say that g is a left inverse of f and that f is a right inverse of 9. Lemma, If the mapping f: 4 — B has both a right inverse h and a left inverse g, they must necessarily be equal. just algebraic juggling and works for any associative operation. Tnoh= Gof)oh=ge(fol) =gela=o.00.10 peaury 15 In this ease we eall the uniquely determined map g:B— A such that Seg= In and go f= 14 tho inverse of J. We then have Theorem. A mapping f: A — B has an inverse if and only it itis bijective, in which ease its inverse is its relational inverse J-". Proof. If fis bijective, then the relational inverse j~* isa funetion from B to A, and the equations fof"? = Ip and J" » f = I are obvious. On the other hand, if fog = Zn, then J is surjective, since then every y in B can be written u=Jlo(y)). And if g°f= I, then f is injective, for then the equation Sle) = f(y) implies that 2 = g(jl2)) = 9(fo)) = y. Thus fis bijective if it has an inverse. 1 Now let S(A) be the set of all bijections f: A+ A. ‘Then &(A) is closed ‘under the binary operation of composition and 1) fo geh) = (fog) eh forall f,9,heS; 2) there exists a unique I © S(A) such that fo I = Io f= ffor all feS; 3) for each fe there exists a unique y € © such that fog = gef= I. Any set @ closed under a binary operation having these properties is called ‘4 group with respect to that operation. Thus (4) is a group with respect to composition. Composition can also be defined for relations as follows. If R CA x Band SCBXG, thon Se RCA X Cis defined by <2,2> ESoR © (3y°)(<2,y> ERE ES). If Rand S axe mappings, this definition agrees with our earlier on 10, DUALITY There is another elementary but important phenomenon ealled duality which occurs in practically all branches of mathematies. Let F: A x BC be any funetion of two variables. It is obvious that if x is held fixed, then F(z, y) is a function of the one variable y. That is, for each fixed = there is a function 1: BC defined by h*(y) = F(z, y). Then ++ k* is a mapping ¢ of A into C8. Similarly, each y € B yields a fanetion gy €C4, where g,(2) = Fv), and y+ gy is mapping 0 from B to C4. ‘Now suppose conversely that we are given a mapping y: A + C®. For each 2A we designate the corresponding value of ¢ in index notation as /*, so that J is a funetion from B to C, and we define F:A x BC by F(x, 1) = AeQ). We are now back where we started. ‘Thus the mappings A —> C#, F:A x BC, and 0: B— C4 are equivalent, and ean be thought of as three different ways of viewing the same phenomenon. The extreme mappings ¢ and 4 will be said to be dual to each other.16 istopverion 0.10 ‘The mapping ¢ is the indexed family of functions {i*:z€ A} CC®. Now suppose that $C C” is an unindexed collection of functions on B into C, and define F:5 x BC by Ff, y) = sly). Then 6: B —> C¥ is defined by 9y(f) = Sly). What is happening here is simply that in the expression f(y) we regard both symbols as variables, so that f(y) is a function on § X B. Then when we hold y fixed, we have a function on ¥ mapping into C. We shall see some important applications of this duality prineiple as our subject develops. For example, an m X n matrix is a function t = {() in R™**, We picture the matrix as a rectangular array of numbers, where ‘i is the row index and {is the column index, so that fi is the number at the inter- section of the ith row and the jth column. If we hold i fixed, we get the n-tuple forming the ith row, and the matrix ean therefore be interpreted as an m-tuple of row n-iuples. Similarly (dually), it ean be viewed as an n-tuple of column ‘m-tuples. In the same vein, an n-tuple of funetions from A to B can be regarded as a single n-tuple-valued function from A to B, ars - In a somewhat different application, duality will allow us to regard a finite- dimensional veetor space V as being its own second conjugate space (V*)*. It is instructive to look at elementary Euclidean geometry from this point of view. Today we regard a straight line as being a set of geometric points. An older and more neutral view is to take points and lines as being two different kinds of primitive objects. Accordingly, let A be the set of all points (so that A is the Euclidean plane as we now view it), and let B be the set of all straight ines. Let F be the incidence function: F(p, !) = Lif p and Uare incident (p is “on” I, Lis “on” p) and F(p, 1) = O otherwise. ‘Thus F maps A x B into {0, 1}. Then for each ! © B the function (p) = F(p, 2 is the characteristic funetion of the set of points that we think of as being the line ! (g1(p) has the value 1 if p is on 1 and 0 if p is not onl) Thus each line determines the set of points that are on it. But, dually, each point p determines the set of lines J “on” it, through its ehar- acteristic function h?(0). Thus, in complete duality we can regard a line as being set of points and a point as being a set of lines. ‘This duality aspect of geometry is basie in projective geometry. It is sometimes awkward to invent new notation for the “partial” funetion obtained by holding a variable fixed in a function of several variables, as we did above when we set g,(x) = P(r, ), and there is another device that is frequently ‘woeful in this situation. ‘This is to put a dot in the position of tho “varying variable”. ‘Thus F(a,-) is the funetion of one variable obtained from F(, ») by holding z fixed at the value a, so that in our beginning discussion of duality we have w= Fay), y= FCs w) If is a function of one variable, we ean then write f = f(-), and so express theout ‘THE BOOLEAN OPERATIONS — 17 above equations also as h“() = F(2,), oy) = F(-, y)- The flaw in this notation is that we can't indicate substitution without losing meaning. Thus the value of the function F(z, ) at b is F(x, 2), but from this evaluation we eannot read backward and tell what funetion was evaluated. We are therefore forced to some such cumbersome notation as F(,-)y which ean get out of hand. Never- theless, the dot deviee is often helpful when it can be used without evaluation dificulties, In addition to eliminating the need for temporary notation, as mentioned above, it ean also be used, in situations where it is strietly speaking superfluous, to direct the eye at once to the position of the variable. For example, later on DF will designate the directional derivative of the function F in the (xed) direction & This is a funetion whose value at « is DgF(a), and the notation DzF(-) makes this implicitly understood fact explicit. Ll, THE BOOLEAN OPERATIONS Let S be a fixed domain, and let ¥ be a family of subsets of S. The union of 5, ‘or the union of ail the sets in 5, is the set of all elements belonging to at least one set in F. We designate the union US or Uses A, and thus we have Us = f:GA)@eE A}, YEUT& GAM )y EA), We often consider the family § to be indexed. ‘That is, we assume given a set I (the set of indices) and a surjective mapping i» A; from I to 5, so that 5 = {4i:i€ I}. Then the union of the indexed collection is designated User As or U{Ai:2e 1}. The device of indices has both technical and psychological advantages, and we shall generally use it If is finite, and either it or the index set is listed, then a different notation is used for its union. If ¢ = {A, B}, we designate the union A U B, a notation that displays the listed names. Note that here we have z € A UB @2€ A or 2eB, If 5 = {Ac:i = 1,...,n), we generally write ‘Ay U Ag U-++U As’ or ‘Ulan AV for US. ‘The intersection of the indexed family (A,}.cr, designated ser Ay, is the set of all points that lie in every Ay. Thus rela: © WEN(we Ad, For an unindexed family 5 we use the notation M5 or Macs A, and if 5 = {A,B}, then NF = ANB. ‘The complement, A’, of a subset of S is the set of elements x = $ not in A: A’ = (25:2 4}. The law of De Morgan states that the complement of «an intersection is the union of the complements: (Q4y = Yao ‘This an immediate consequence of the rule for negating quantifiers. It is the18 1rropuerion oun equivalence between ‘not always in’ and ‘sometimes not in’: [~(¥i)(e € A.) (3) € A] says exactly that ze (Nay © rEU(A). Af and take complements again, we obtain the dual form: (UserB,)’ = Micr(B). Other principles of quantification yield the laws an(U a)=Uenad from P & (22)Q%2) + (32)(P & Q@)), Bu (0,4) = Neu as, Bo (0,4) = Dn aa, fet fer Bu(U 4) = U Bu ad. ‘In the ease of two sets, these laws imply the following familiar laws of set algebra (AuBy=A4'nB, (AN BY = AUB! De Morgan), An (BUC) = (ANB) U(ANO, AU(BNC) = (AUB) (AUC). Even here, thinking in terms of indices makes the laws more intuitive. ‘Thus (Ain 4a) = Ay U AS is obvious when thought of as the equivalence between ‘not always in? and ‘sometimes not in’. ‘The family ¢ is diajoine if distinet sets in ¥ have no elements in common, ie, if (WX, YS)(X x Y= XY = 9). For am indexed family (A,}ier the condition becomes i # j=» A: As = B. IF = {A,B}, we simply say that Aand B are disjoint. Given f: U + V and an indexed family {B,} of subsets of V, we have the following important identities: riya Ure and, for a single set BC V, r[Na)= ews, SB = mB. For example, res [N Bese eB AUG) B is any sur- jective mapping, and if for each value y in B we set A, =I"W) = @EA: Ie) =H, then § = {Ay:y €B} is a fibering of B to. Also yf is the projection x: A > 5, set Z of all z in A'such that (@) = f(a). ‘The above process of generating a fibering of A from a function on A is relatively trivial. A more important way of obtaining a fibering of A is from an equality-like relation on A called an equivalence relation. An equivalence relation ~ on A is a binary relation which is relerive (x ~ x for every 2 € A), symmetric (e ~ y =2 y ~ 2), and transitive (e ~ y and y~ z= 2 ~ 2). Every fibering 5 of A generates a relation ~ by the stipulation that x ~ y if and only if and y are in the same fiber, and obviously ~ is an equivalence relation. The most important fact to be established in this seetion is the converse A and g:y + Ay is a bijection from inee @ ° H(z) = ese) is the ‘Theorem. Every equivalence relation ~ on A is the equivalence relation of a fibering. Proof. We obviously have to det {vy ~ 2}, and our problem is to show that the fa obtained this way isa fibering The reflexive, symmetric, and transitive laws beeome as the set of elements y equivalent to x, F ofall subsets of A rey, cepsyer, and regandyer = vez. Refiexivity thus implies that 5 covers A. Transitivity says that if y € 2, then rep= rez; that is, ify ez, then 7 Cz But also, if y ez, then 2 ey by20 awtnopvertox 12 . Therefore, if two of our no B. In other words, if is we have a fibering. 0 symmetry, and so 2Cy. ‘Thus y © it sets @ and 5 have a point z in common, th not the set J, then @ and B are disjoint, an ‘The fundamental rote this argument plays in mathematics is due to the fact that in many important situations equivalence relations occur as the primary object, and then are used to define partitions and functions. We give two examples. Let Z be the integers (positive, negative, and zero). A fraction ‘m/n’ ean be considered an ordered pair of integers with n #0. ‘The set of all fractions is thus ZX (Z— {0}). Two fractions and are “equal” if and only if mg = np, and equality is checked to be an equivalence relation. ‘The equivalence class is the object taken to be the rational number m/n. Thus the rational number system Q is the set of fibers in a par- tition of Z x @ — {0}). Next, we choose a fixed integer p ¢Z and define a relation # on Z by Bn € p divides m —n, ‘Then His an equivalence relation, and the set 2, of its equivalence classes is ealled the integers modulo p. It is easy to see that mEn if and only if mand n have the same remainder when divided by p, so that in this ease there is an easily calculated function J, where j(m) is the remainder after dividing m by p, which defines the ibering. ‘The set of possible remainders is {0, 1,...,p — 1}, 0 that Z, contains p elements. ‘A function on a’set A can be “factored” through a fibering of A by the following theorem. ‘Theorem. Let g be a function on A, and let 5 be a fibering of 4. Then g is constant on each fiber of 5 if and only if there exists a function 9 on 5 such that g = J © 7. Proof. If g is constant on each fiber of 5, then the association of this unique value with the fiber defines the function J, and clearly g = J x. ‘The converse is obvious, 0CHAPTER 1 VECTOR SPACES ‘The calculus of functions of more than one variable unites the ealeulus of one variable, which the reader presumably knows, with the theory of veetor spaces, atid the adequacy of its treatment depends direetly on the extent to which vector space theory really is used. The theories of differential equations and differential geometry are similarly based on @ mixture of calculus and vector space theory. Such “vector calculus" and its applications constitute the subject matter of this book, and in order for our treatment to be completely satisfactory, we shall have to spend considerable time at the beginning studying veetor spaces them- selves. This we do principally in the first two chapters. ‘The present chapter is devoted to general vector spaces and the next chapter to finite-dimensional spaces. We begin this chapter by introducing the basie concepts of the subject— vector spaces, veetor subspaces, linear combinations, and linear transforma- tions—and then relate these notions to the lines and planes of geometry. Next wwe establish the most elementary formal properties of linear transformations and, Cartesian produet veetor spaces, and take a brief look at quotient vector spaces. ‘This brings us to our first major objective, the study of direet sum decomposi- tions, which we undertake in the fifth seetion. ‘The chapter concludes with a preliminary examination of bili 1. FUNDAMENTAL NOTIONS Vector spaces and subspaces. The reader probably has already had some contact with the notion of a veetor space. Most beginning ealeulus texts discuss eometrie veetors, which are represented by “arrows” drawn from a chosen origin 0. These veetors are added geometrically by the parallelogram rule: The sum of the vector OA (represented by the arrow from 0 to A) and the vector OB is the veetor OP, where P is the vertex opposite O in the parallelogeam having OA and OB as two sides (Fig. 1.1). Veetors ean also be multiplied by numbers: z(@A) is that vector OB such that B is on the line through O and 4, the distance from O to B is |x| times the distance from 0 to A, and B and A sre on the same side of 0 if x is positive, and on opposite sides if x is negative 2122 vector spaces Al 1B-+00-OX Fig. 13 (Fig. 1.2). These two veetor operations satisfy certain laws of algebra, which we shall soon state in the definition. The geometrie proofs of these laws are generally sketchy, consisting more of plausibility arguments than of airtight logic. For example, the geometric figure in Fig. 1.3 is the essence of the usual itive. In each case the final vector OX is from 0 in the parallelepiped constructed from the three edges 0, OB, and OC. The set of all geometric vectors, together with these two operations and the laws of algebra that they satisfy, constitutes fone example of a veetor space. We shall return to this situation in Section 2. ‘The reader may also have seen coordinate triples treated as veetors. In system a three-dimensional veetor is an ordered triple of numbers <2, 22, 23> which we think of geometrically as the coordinates of a point in space. Addition is now algebraically defined, X24, 20) 29> + ) as is multiplication by numbers, ¢<21, 22 23> = . The vector laws are much easier to prove for these objects, since they are almost algebraic formalities. The set R° of all ordered triples of numbers, together with these two operations, is a seeond example of a vector space.1 FUNDAMENTAL NOTIONS 23 If we think of an ordered triple <2, x2, 72> a8 a funetion x with domain the set of integers from 1 to 8, where 2; is the value of the funetion x at é (see Section 0.8), then this vector space suggests a general type called a function space, which we shall examine after the definition. For the moment we remark only that we defined the sum of the triple x and the triple y as that triple 2 such that 2¢ = 2;-+ ys for every i. ‘A veetor space, then, is a collection of objects that ean be added to eacl other and multiplied by numbers, subject to certain laws of algebra. In this context a number is often called a sealar. Definition. Let V be a set, and let there be given a mapping +» a+8 from Vx V to V, ealled addition, and a mapping <2,a> 2a from R X V to V, called multiplication by scalars, Then V is a veetor space with respect to these two operations if: AL a+ @+2)=(a+)+7 forall a6,7eV. AL a+ Basta forall a, BEV. AB. ‘There exists an element 0 V such that a+0— a for all ae V. Ad. For every a V there exists a @ € V such that a+ = 0. SL. (ey)a = 2(ya) forall z,yER, aeV. 82 @+ a= sa4 yo forall z,yeR, ae. 88. x(a +8) = rat a8 forall reR, a, 86. St laa forall av. In contexts where it is clear (as it generally is) which operations are intended, wwe refer simply to the vector space V. Certain further properties of a vector space follow direetly from the axioms. ‘Thus the zero clement postulated in A3 is unique, and for each « the 6 of Ad is unique, and is called a. Also 0a = 0, 20 = 0, and (—I)a= —a. These elementary consequences are considered in the exercises Our standard example of a vector space will be the set V = R¢ of all real- valued functions on a set A under the natural operations of addition of two funetions and multiplication of a function by a number. This generalizes the example R'"-23) = R® that we looked at above. Remember that a function f in R¢ is simply a mathematical object of a certain kind. We are saying that two of these objects can be added together in a natural way to form a third such object, and that the set of all such objects then satisfies the above laws for audition. Of course, {+ is defined as the funetion whose value at a is f(a) (a), $0 that (f+ 9)(a) = f(a) + a(a) for all a in A. For example, in R? we defined the sum x ++ yas that triple whose valueat fisr-+ ys forall. Similarly, of is the function defined by (ef)(a) = c(f(a)) for all a, Laws Al through S+ follow at once from these definitions and the corresponding laws of algebra for the real number system. For example, the equation (s+ Of = af + f means34 vEcToR spaces 1. that ((8 + 04)(@) = (af + Y)(@) for alla A. But (6+ DN@ = (+ O(a) = s(F@)) + 1H) GN(@) + WA) = OF + N10), where we have used the definition of scalar multiplication in 4, the distributive law in R, the definition of scalar multiplication in R“, and the definition of addition in R4, in that order. Thus we have S2, and the other laws follow similarly. "The set A can be anything at all. If A = R, then V = R® is the vector space of all real-valued functions of one real variable. If A = R x R, then. ¥ = R*** is the space of all real-valued functions of two real varables. If A= (1,2) =3, then V = R= R? is the Cartesian plane, and if A {1,...)n} =, then V = R* is Cartesian n-space. If A contains @ single point, then R4 is a natural bijective image of R itself, and of course B is trivially a vector space with respect to its own operations. Now let V be any veetor space, and suppose that IV is a nonempty subset of F that is closed under the operations of V. ‘That is, if «and B are in W, then so is a +B, and if a isin W, then so is za for every sealar 2. For example, let V be the veetor space 2"! of all real-valued functions on the closed interval [a, b] CR, and let W be the set €((a, b)) of all continuous real-valued funetions on [a,b ‘Then W is a subset of V that is closed under the operations of V, since f-+ g and ef are continuous whenever f and gare. Or let V be Cartesian 2-space B®, and let W be the sot of ordered pairs x = <2, 22> such that 21 + 22 = 0. Clearly, WY is elosed under the operations of V. Such a subset Wis always a veetor space in its own right. ‘The universally ‘quantified laws Al, A2, and SI through S4 hold in W7 because they hold in the larger set V. And since there is some 8 in W, it follows that 0 = 08 is in W ecause W is closed under multiplication by scalars. For the same reason, if @ is in W, then s0 is —a = (—1)a. ‘Therefore, A3 and Ad also hold, and we see that W'is a vector space. We have proved the following lemma. Lemma. If W is a nonempty subset of a veetor space V whieh is closed under the operations of V, then IV is itself a veetor space. We call W a subspace of V. Thus €((a, 6) is a subspace of R'*", and the pairs <2y,.22> such that zr +22 = 0 form a subspace of R?, Subspaces will he with us from now to the end. ‘A subspace of a veotor space Ri is called a function space. In other words, & ction space is collection of real-valued funetions on a eommon domain sh is closed under addition and multiplication by sealars. What we have defined so far ought to be called the notion of a ral vector space or a veetor space over R. ‘There is an analogous notion of a compler vector space, for which the sealars are the complex numbers. ‘Then laws SI through $4 refer to multiplication by complex numbers, and the space C4 of all complex fuAl FUNDAMENTAL NOTIONS — 25 valued functions on A is the standard example. In fact, if the reader knew what is meant by a field FP, we could give a single general definition of a vector space over F, where scalar multiplication is by the elements of F, and the standard example is the space V = F4 of all functions from A to F. ‘Throughout this book it will be understood that a vector space is a real vector space unless explic itly stated otherwise. However, much of the analysis holds as well for complex veetor spaces, and most of the pure algebra is valid for any scalar field F. EXERCISES LAL_ Sketch the geometre figure representing law 83, x04 + 0B) = 20a) + (0B), for geometric vectors. Assume that 2 > 1. 12. Prove S3 for RP using the explicit displayed form (ey, 22,25} for ordered triples. 1.3. The vector O postulated in A3 is unique, as elementary algebra fiddling will show. For suppose that 0” also satisfies A. ‘Then = 0+0 (A3foro =0+0 (2) -0 (AS for 0) Show by similar algebraic juggling that, given a, the 8 postulated in A4 is unique This unique Bis designated —a LA Prove similarly that Oa = 0, 0 = 0, and (—1)a 1.5. Prove that if ra = 0, then either 2 = 0 or « 1.6 Prove SI for function space BR. Prove S3. 1.1 Given that is any veetor in a veetor space V, show that the set {2a:2€ R} of al scalar multiples of ais a subspace of V. 18. Given that « and 8 are any’ two vectors in V, show that the set of all vetors za+ yB, where z and y are any real numbers, isa subspace of V. 1.9. Show thatthe set of triples xin R# such that 21 — 22+ 2ra = Osa subspace M. If is the similar subspace {x:21 + ze-f 23 = 0}, find a nonzero vector a in MOAN. Show that AF isthe set (ca:2 € BY} of all scalar multiples of a 1.10 Let A be the open interval (0,1), and let V be RA. Given a point z in (0,1), Jet Ve be the set of functions in V that have a derivative at z. Show that Visa sub: space of V. LIL For any subsets 4 and B of a vector space V we define the set sum 4 + B by 14+B= (@+B:aG | and BEB}. Show that (4+ B)+C = A+ (B+0) 112 MACY and XCR, we similarly define XA = f2a:2eX and a€ al} Show that a nonvoid set 4 is a subspace if and only if A+ A = A and RA = 4 LIS Let V be R2, and let M be the line through the origin with slope k. Let x be any nonaero veetorin Mf, Show that Af is the subspace Rx = féx:te R).26 vector spaces cmt 1.14 Show that any other line L with the same slope f is of the form M-+ a for some a. 115. Let M be a subspace of a veetor space V, and let a and 8 be any two veetors in V Given A = a+ AM and B= 6+ M, show that cither A= B or ANB = B. Show also that t+ B = (a-+8)-+ M0. 1.16 State more carefully and prove what is meant by a subspace”, LI7_ Prove that the interseetion of two subspaces of a veetor space is always itself a subspace. 1.18 Prove more generally that the interseetion W'= Aier We of any family (Ws: € 1} of subspaces of V is a subspace of V. 1.19 Let V again be Rand let H be the set of all Sunetions fin V sueh that f"(2) exists for every zin (0, 1). Show that I'js the interseetion ofthe collection of subspaces of the form V, that were considered in Exercise 1.10, 1.20: Let V bea function space Re, and fora point ain {let Wa be the set of funetions such that f(a) = 0. Wa is clearly a subspace, For a subset BC .L let Wa be the set of functions f in V such that f = O on B. Show that Wy is the intersection (wen We. 1.21 Supposing again that X and Yate subspaces of V, show that if X+ Y = Vand XY = {0}, then for every vector ¢ in V there is a unique pair of vectors EEX fand 9 such that ¢ = £-+ 9. 1.22 Show that if N’and ¥' are subspaces of a vector space V’, then the union XU Y ean only be a subspace if either XC Y or YON, subspace of a subspace is Linear combinations and linear span, Because of the commutative and assor ative laws for vector addition, the sum of a finite set of vectors is the same for all possible ways of adding them. For example, the sum of the three vectors a) ay ae can be calculated in 12 ways, all of which give the same result: (ea an) + ae = a + (ea + a8) = (e+ 4g) + a4 = 04 + (ae + a), ele. ‘Therefore, if I= {a,b,c} is the set of indices used, the notation Dicr ai, Which indicates the sum without telfing us how we got it, is unambiguous. Tn general, for any finite indexed set of veetors {ai :i = I) there is a uniquely determined sum veetor 3j Consider now the set of all linear combinations of the two veetors <1,1,1> and <0, 1, —1> in B®. It is the set of all veetors <1, 1, 1> + 180, 1, -L> = <8, 9-4 f,8 — ©, where s and Care any real numbers. Thus L= (Xs,8-+68— 0 : ER}. It will be clear on inspection that Lis closed under addition and sealar multiplication, and therefore is a subspace of B, Also, Z contains each of the two given veetors, with coefficient. pairs <1,0> and <0,1>, respectively. Finally, any subspace M of &* which contains each of the two given veetors will also contain all of their linear com! nations, and so will include L. That is, L isthe smallest subspace of B® containing <1, 1, 1> and <0, 1, —1>. Itisealled the linear span of the two vectors, or the subspace generated by the two veetors. In general, we have the following theorem.28 vecror spaces a ‘Theorem 1.1. If A is a nonempty subset of a veetor space V, then the set L(A) of all linear combinations of the veetors in A is a subspace, and it is the smallest subspace of V which includes the set A. Proof. Suppose first that A is finite. We ean assume that we have indexed A in some way, so that A = {a;:i€ J} for some finite index set I, and every element of TA) is of the form Sacr tam ‘Then we have (Lr) + (Lye) = Zee + vidas because the left-hand side becomes X; (cya; + yia) when it is regrouped by pairs, and then $2 gives the right-hand side. We also have AE nai) = Llexdas by $3 and mathematical induction, Thus L(A) is closed under addition and rultiplication by sealars and hence is a subspace. Moreover, L() contains each a (why?) and so includes Finally, if a subspace W includes A, then it contains each linear combination 5 zrya;, so it includes L(A). Therefore, L(A) can be directly characterized as the uniquely determined smallest subspace which includes the set A If A is infinite, we obviously can’t use a single finite listing. However, the sum (Ef zen) ++ (Cf 938)) of two linear combinations of elements of A is clearly a finite sum of scalars times elements of A. If we wish, we ean rewrite it as Dit xies, where we have set 8; = an4j and yj = tn4j for j= 1,...,m. In any ease, L(A) is again closed under addition and multiplication by scalars and so isa subspace. ( We eall L(A) the linear span of A. If L(A) V is finitesdimensionat if it has a finite spanning set. If V = R°, and if 3, 5%, and 4° aro the “unit points on the axes”, 8! = <1,0,0>, 6? = <0,1,0>, and 5*= <0,0,1>, then {4°}? spans V, since x= X2y 22, 25> = <21,0,0> + <0,22,0% + <0,0,29> = 28) + 236? + x56® = Df x6" for every x in R®, More generally, if V = R" and 3! is the n-tuple having I in the jth piace and 0 elsewhere, then we have similarly that X= Xry...,24> = Dh 2185, so that {8} spans BY. Thus R" is finite- dimensional. In general, a function space on an infinite set A will not be finite- dimensional. For example, it is true but not obvious that €((a, b]) has no finite spanning set. V, we say that A spans V; EXERCISES: 1.28 Given @ = <1,1,1>, 8 = <0, 1, —1>,7 = <2,0,1>, compute the Tinewr combinations a+ 8-17, 8a — 28+, za-+ yB-+ 27. Find ,y, and 2 such that ra-+yB-+27 = <0,0,1> = 88. Do the same for 8! and 3%. 1.24 Given a = <1,1,1>, 8 = <0,1,—1>, 7 = <1,0,2>, show that each of a, 8,7 is a linear combination of the other two. Show that it is impossible to find cooficients z,y, and z such that a+ yB+ 27 = 6Ll FUNDAMENTAL NoTIONS 29 1.23. a) Find the linear combination of the set A = <0, ~ 1, 2+ 1> with eoefti- cient triple <2, —1, 1>. Do the same for <0, 1, 1>. b) Find the coefficient triple for which the linear combination of the triple is (+1). Do the same for 1 ©) Show in fact that any polynomial of degree < 2 is linear combination of 1 1.26 Find the linear combination fof {e', -*} C R® such that f(0) = 1and"(0) = 2 1.27 Find a linear combination fof sin 2, cos z, and e* such that f(0) = 0, (0) = 1 and "(0) = 1. 1.28 Suppose that a sin z-+ beos 2 + cet is the zero funtion. Prove that a eno 1.29 Prove that <1, 1> and <1, 2> span R?, 1.30 Show that the subspace Mf = (x: +22 = 0) CR? is spanned by one veetor. LBL Let M be the subspace {x:21 —z2+2zs = 0} in RY. Find two vectors a and b in Af neither of which is a sealar multiple of the other. ‘Then show that 1 is, the linear span of a and b. 1.82 Find the intersection of the linear span of <1,1,1> and <0,1,—1> in BY with the coordinate subspace r2 = 0. Exhibit this intersection as a linear span. 1.83 Do the above exercise with the coordinate space replaced by M = (erm ta2 = 0). 1.34 By Theorem 1.1 the linear span L(4) of an arbitrary subset 4 of a veetor space V has the following two properties: i) L(A) is a subspace of V which includes 4; fi) If If is any subspace which includes A, then L(.1) CM. Using only (i) and (i), show that a) AC B= 14) C LB); b) L(L(A)) = L(A). 1.85. Show that a) if Mand N are subspaces of V, then so is M+ N; b) for any subsets A, BC V, L(A UB) = L(A) + LB). 1.36 Remembering (Exercise 1.18) that the interseetion of any family of subspaces ‘a subspace, show that the linear span L(A) of a subset A of a veetor space Vis the intersection ofall the subspaces of V that include A. This alternative characterization is sometimes taken as the definition of linear span. 1,37 By convention, the sum of an empty set of vectors is taken to be the zero vector. This is necessary if Theorem 1.1 is to be strictly correct. Why? What about the preceding problem? Linear transformations. The general function space R4 and the subspace €((a, b)) of R'" both have the property that in addition to being closed under the vector operations, they are also closed under the operation of multiplication of two functions. That is, the pointwise product of two functions is agai function [(fg)(a) = f(a)g(a)], and the produet of two continuous functions is ‘continuous. With respeet to these three operations, addition, multiplication,30. _vEcToR sPaces a and scalar multiplication, Rand €((a, b)) are examples of algebras. Tf the reader noticed this extra operation, he may have wondered why, at least in the eontext of funetion spaces, we bother with the notion of veetor space. Why not study all three operations? The answer is that the veetor operations are exactly the operations that are “preserved” by many of the most important: mappings of sets of functions. For example, define T:€((a, bl) > B by TU) = [e f(0 at. Then the laws of the integral ealeulus say that T(f+ 9) = T(f) + T(@) and T() = eT). Thus P “preserves” the vector operations. Or we can say that T “commutes” with the veetor operations, since plus followed by T equals 7’ followed by plus. However, 1 does not preserve multiplication: it is not true in general that Tu) = TUT). Another example is the mapping T:x + y from R® to R® defined by th = 2x1 — tat ta Yo = 21+ Bea — ea, for which we can again verify that T(x-+ y) = T(x) + T(y) and T(ex) = eT(a). The theory of the solvability of systems of linear equations is essentially the theory of such mappings 7’; thus we have another important type of mapping that preserves the vector operations (but not produets). ‘These remarks suggest that we study veetor spaces in part so that we ean study mappings which preserve the vector operations. Such mappings are called linear transformations. Definition. If V and W are veetor spaces, then a mapping T:V — Wis a Hinear transformation or a linear map it T(a+ 8) = Tla) + T(B) for all a, 8€V,and Tra) = xP(@) for allac V,2ER. ‘These two conditions on ean be combined into the single equation P(e} y8) = 2T(a) + yT(@) forall a@EV andall 2,yER, Moreover, this equation can be extended to any finite sum by induetion, s0 that if 7 is linear, then 1 (Zara) = Dated for any linear combination 5 rai. For example, f (Cf ef) = Dt eft Je EXERCISES 1.38 Show that the most general linear map from R to B is multiplication by a co stant. 1.39 For a fixed a in V the mapping z+» za from B to Vis linear. Why? 1.40 Why 1.41 Show that every linear mapping from R. to V is of the form 2+ xa for a fixed veetor a in V. is this true for a zer when 2 is fixed?rat FUNDAMENTAL NOTIONS 31 142 Show that every linear mapping from B® to V is of the form >> a+ z2a2 fora fixed pair of veetorsa; and azin V. What is the range of this mapping? 1.43 Show that the map f+ J? (0) dt from @((a, 6) to B does not preserve products LAt Let g be any fixed function in R4, Prove that the mapping T:R — Ré defined by T(f) = of is linear. 145. Let ¢ be any mapping from a set A ta n-set R. Show that composition by ¢ is a linear mapping from B® to R4, That is, show that 7:R —> Re defined by Tif) = fees linear. In order to acquire a supply of examples, we shall find all linear transforma tions having R" as domain space. It may be well to start by looking at one such transformation. Suppose we choose some fixed triple of functions {f,} in the space R* of all real-valued funetions on R, say fi(t) = sin t, fa(l) = cos f, and fall) = ef = exp(). Then for each triple of numbers x = {x} in R® we have ‘the linear combination 2-1 z:f;with (23) as coefficients. This is the element of R" whose valueat fis E3 2ifi(0) = x1 sint + x2 cost + x3¢', Different coefficient triples give different functions, and the mapping x» Dia zifi = 21 sin + 2.008 + 23 exp is thus a mapping from &° to R". It is clearly linear. If we call this mapping 7, then we can recover the determining triple of functions from 7 fas the images of the “unit points” & in RY; 706") = ¥ afi = fi, and so (83) = sin, T(6*) = cos, and (6%) = exp. We are going to see that every linear mapping from R? to R® is of this form. Tn the following theorem {3'}{ is the spanning set for R" that we defined earlier, s0 that x= Ef 2:8" for every ntuple x= <2y..-, 74> in B® ‘Theorem 1.2. If {8)}7 is any fixed n-tuple of vectors in a veetor space W, then the “linear combination mapping” x-+ Ef 28s is a linear trans- formation T from R" to W, and 7(8/) = 8; forj = 1,...,n Conversely, if 7 is any linear mapping from R* to W, and if we set 8) = 78) for 1,...)2, then 7'is the linear combination mapping x» Ef 28 Proof. The linearity of the linear combination map T' follows by exactly the ‘same argument that we used in Theorem 1.1 to show that L(A) is a subspace. ‘Thus Et wei = E (eae + v9 x tiBe + z uss = T(x) + Ty), and Tox) =F (ord = Sale) = +E 28s = aT. Also 1(8!) = Ef-1 88: = 85, since 6 = 1 and &f = 0 for i x j.82 VECTOR spaces 1 Conversely, if 7: RY — Wis linear, and if we set 8 = (6?) forall j, then for any x= in RY we have T(x) = T(LT x: 83) = Li xT (6) Efage Thus 7 is the mapping x» Ef 2G. O ‘This is a tremendously important theorem, simple though it may seem, and the reader is urged to fix it in his mind. To this end we shall invent some termi- nology that we shall stay with for the first three chapters. Ife = {a isan n-tuple of veetors in a veetor space W, let Le be the corresponding linear combination mapping x + Cf a: from R* to W. Note that the n-tuple a itself isan clement of W*. If 7 is any linear mapping from R" to W, we shall eall, the n-tuple {7'(8')}¥ the skeleton of 7. In these terms the theorem ean be restated as follows. ‘Theorem 1.2". For cach n-tuple a in W", the map La: R" — W is linear and its skeleton is @. Conversely, if Tis any linear map from ®" to W, then T = Lg where B is the skeleton of T. Or again: ‘Theorem 1.2”. ‘The map a+ Le is a bijection from 17" to the set of all linear maps T from R" to W, and 1’ skeleton (7) is its inverse. A linear transformation from a veetor space ¥” to the scalar field Ris ealled a linear functional on V. Thus f+ ft f(d) dt is a linear functional on V = €((a,b). The above theorem is particularly simple for a linear functional F: since W = R, each veetor 8; = F(8) in the skeleton of F is simply a number bi, and the skeleton {b,}} is thus an element of R". In this ease we would waite F(x) = Cf bay, putting the numerical coefficient ‘4 before the variable ‘2. Thus F(x) = 3x1 — 22+ 4s is the linear functional on R® with skeleton <8, —1,4>. The set of all linear functionals on R" is in a natural one-to-one correspondence with R” itself; we get b from F by by = F(s') for all i, and we get F from b by F(x) = ¥ bar; for all x in R”. We next consider the ease where the codomain space of Tis a Cartesian space R™, and in order to keep the two spaces clear in our minds, we shall, for the moment, take the domain space to be R®, Each veetor 6; = 7(3') in the skeleton of 1 is now an metuple of numbers. If we picture this m-tuple as a column of numbers, then the three m-tuples 8, can be pictured as a rectangular array of numbers, consisting of three columns each of m numbers. Let é; be the ‘th number in the jth column. Then the doubly indexed set of numbers {@} is called the matriz of the transformation 7. We eall it an m-by-3 (an m X 3) matrix because the pictured reetangular array has m rows and three columns ‘The matrix determines 7 uniquely, since its columns form the skeleton of ‘The identity T(x) = Li zjT(s’) = Li xy6; allows the m-tuple T(x) to be calculated explicitly from x and the matrix {4j}. Picture multiplying the column m-tuple 8; by the sealar x; and then adding across the three columns at1 FUNDAMENTAL NOTIONS 33 the ith row, as below (--Q-Q-@) Sinee (i; is the ith number Dh 28; is Dhan aotez- That the m-tuple fj, the ith number in the m-tuple we let y be the m-tuple T(x), then a we= DO bist; for i= 1,...,m, and this set of m sealar equations is equivalent to the one-vector equation y= Tw. We ean now replace three by n in the above discussion without changing anything except the diagram, and thus obtain the following specialization of Theorem 1.2. ‘Theorem 1.3. Every linear mapping from B* to B™ determines the mx n matrix t = {ly} having the skeleton of 7’ as its columns, and the ‘expression of the equation y = T(x) in linear combination form is equivalent to the m scalar equations n= Litany for Conversely, each m Xn matrix t determines the linear combination mapping having the eolumns of tas its skeleton, and the mapping t -+ 7 is therefore ction from the set ofall m x n matrices to the set ofall linear maps from B* to B". A linear funetional F on &* is a linear mapping from R” to BR, so it must beexpressed by al x nmatrix. That is, the n-tuple b in R* which is the skeleton of F is viewed as a matrix of one row and n columns. Asa final example of linear maps, we look at an important elass of special linear functionals defined on any function space, the so-called coordinate func- tionals. If V = B! and i € J, then the ith coordinate funetional 7; is simply evaluation at i, so that (J) = J(). These functionals are obviously linear. In fact, the vector operations on functions were defined to make them tinear; since af + Wy is defined to be that function whose value at i is sf(é) + tg(¢) for all é, we seo that sf-+t is by definition that funetion such that mu(sf +t) swf) + tw(o) for all i! Tf V is RY, then m; is the mapping x= <21,...,,> H+.23. In this ease ‘we know from the theorem that xj must be of the form 1,(x) = Ei} buts for some n-tuple b. What is b?34 -vreToR spaces 1 ‘The general form of the linearity property, T(E tai) = ExT (a:), shows that T and 7! both earry subspaces into subspaces. ‘Theorem 14. If T:V — I is linear, then the T-image of the linear span ‘of any subset A CV is the linear span of the T-image of A: TIL(A)] = L(T[A). In particular, i A isa subspace, then so is T[A]. Furthermore, if Y isa subspace of W, then 7—![Y'] isa subspace of V. Proof. According to the formula T(E xa;) = XxiT(a,), « veetor in W the 7-image ofa linear combination on A if and only if itis linear combination on T[A]. That is, TIL(A)] = L(T[A)). If A isa subspace, then A = L(A) and TA] = L(T[A), a subspace of W. Finally, if Y isa subspace of W and {a,) C TY), then TE xa) = LxM@) €L0) = V. Thus Dea € T) and T™{Y] is its own linear span. ‘The subspace T-1(0) = fae V: T(a) = 0} is called the null space, or Kernel, of , and is designated N(7) or (7). The range of 7 is the subspace TV] of W. It is designated R(T) or (7). Lemma 1.1. A linear mapping T is injective if and only if its mull space is 0) Proof. WET is injective, and if a 0, then T(a) # 7(0) = O and the null space accordingly contains only 0. On the other hand, if N(T) = {0}, then whenever a # 8, we havea — 8 # 0, T(a) — T(6) = T(a — 8) #0, and T(a) # TA); this shows that 1'is injective, A linear map 7: —+ W which is bijective is called an isomorphism. ‘Two vector spaces V and W aro isomorphic if and only if there exists an iso- morphism between them, For example, the map <¢,,-..,¢n% > D7! eax! I" with the veetor space of all polynomials of degree < ». Isomorphie spaces “have tho same form”, and are identical as abstract vector spaces. That is, they cannot be distinguished from each other solely on the basis of veetor properties which they do or do not have. ‘When a linear transformation is from V to itself, special things ean happen. One possibility is that 7’ can map a vector a essentially to itself, T(a) = xa for some z in B. In this ease a is called an eigenvector (proper veetor, character istie veetor), and « is the corresponding eigenvalue. an isomorphism of EXERCISES 1.46 In the situation of Exercise 1 by shoving that show that 1's an isomorphism if ¢ is bijective a) ¢ injective surjective, b) surjective T injectiveAl FUNDAMENTAL NOTIONS 1.47 Find the linear functional / on R? sueh that (<1, 1) = O and (<1, 2>) ‘That is, find b = <0j,b2> in R® such that /is the linear combination map x bien + bare 1.48 Do the same for 1(<2,1>) = —Band I(<1,2>) = 4. 1.49 Find the linear 7:R? — RB such that 7(<1,1>) = @ and T(<1,2>) = 8. That is, find the funetions f,(0) and fa(t) such that! T'is the linear combination map xo nifit refs 1.50, Lot Tbe the linear map from R? to RS such that T(3!) = <2,—1, 1>, 708) = <1,0,3>. Write down the matrix of T in standard rectangular form. Determine whether or not 8! isin the range of 7. L.5L_ Let T'be the linear map from R¥ to RS whose matris is 10209 2 0-1]. 3-101 Find T(x) when x = <1, 1,02; do the same for x = <3, —2, 1> 1.52 Lot M be the linear span of <1, —1,0> and <0, 1,12. Find the subspace TIM] by finding two vectors spanning it, where 7 is as in the above exercise 1.53 Let T’e the map —+ from R? to itself. Show that Tisa linear combination mapping, and write down its matrix in standard form, 54 Do the same for T: + <2 —2,2-+2,y> from BP to itself 1.55. Find a linear transformation 7 from B® to itself whose range space is the span of <1, -1,0> and <1, 0,2>. 1.56 Find two linear functionals on R¢ the intersection of whose mull spaces is the linear span of <1, 1,1, 1> and <1,0,—1,0>. You now have in hand a linear transformation whose null space is the above span. What is it? 1.57 Let V = €({a,b)) be the space of continuous real-valued functions on fa, 8), designated (a, b), and let 1 = €%((a,b)) be those having continuous first Mlerivatives, Let D: iV —» V be differentiation (Df =f’), and define T on V by T(f) = F, where F(x) = JZ f(t) dt. By stating appropriate theorems of the ealeulus, show that D and 7 are linear, T maps into W, and D is a left inverse of T (De T is the identity on V). 1.58 In the above exercise, identify the range of T’and the null space of D. We know that D is surjective and that T is injective. Why? 1.59 Let V’be the linear span of the functions sin x and cos x. Then the operation ‘of differentiation D isa linear transformation from V to V. Prove that D isan isomor- phism from V to V. Show that D? = —J on V. {60 a) Ae the reader would guess, @9(R) ix the sot of real-valued functions on RL having continuous derivatives up to and including the third. Show that f+” is a surjective linear map T' from €8(R) to €(R). b) For any fixed a in B show that f+ is an isomorphism from the null space N(T) to B®. [Hints Apply Taylor's formula with remainder.)36 vEcToR sraces 12 1.61 An integral analogue of the matrix equations ys = Tj tysn t= Ayes ym, is the equation . a) =f KGOK0 a, 610,11 Assuming that K(s;t) is defined on the square [0,1] X 0, 1] and is continuous as a function of ¢ for each s, check that f — g is a linear mapping from @((0, 1]) to Rl", 1.62 For a finite set = fai}, Theorem 1.1 Isa corollary of Theorem 1.4. Why? 1.63 Show that the inverse of an isomorphism i linear (and hence i an isomorphism). 1.64 Find the eigenvectors and eigenvalues of 7:R? — R? if the matrix of Tis tat 2 0. Since every sealar multiple za of an eigenvector a is elearly also an eigenvector, it will suffice to find one vector in each “eigendirection”. ‘This is a problem in elementary algebra, 1.65 Find the cigenveetors and eigenvalues of the transformations 7 whose matrices aor 1-1 1-1 27 Gob Go Gea 1.66 ‘The five transformations in the above two exercises exhibit four different kinds of behavior according to the number of distinet eigendirections they have. What are the possibilities? 1.67 Let V be the vector space of polynomials of degree < 3 and define 7:1 > V by f+ 4"). Find the eigenvectors and eigenvalues of 2. VECTOR SPACES AND GEOMETRY ‘The familiar coordinate systems of analytic geometry allow us to consider geometric entities such as lines and planes in veetor settings, and these geometric notions give us valuable intuitions about veetor spaces. Before looking at the vector forms of these geometric ideas, we shall briefly review the construction of, the coordinate correspondence for three-dimensional Euclidean space. As usual, the confident reader ean skip it. We start with the line. A coordinate correspondenee between a line Land the real number system & is determined by choosing arbitrarily on La zero point O and a unit point @ distinet from 0. ‘Then to each point X on Lis assigned the number x such that |2| is the distance from O to X, measured in terms of the segment 0Q as unit, and x is positive or negative according as X and Q are (on the same side of 0 or on opposite sides. ‘The mapping X ++ x is the coordinate correspondence. Now consider three-dimensional Euclidean space E*. We want, to sot up a coordinate correspondence between E* and the Cartesian vector space BR’. We first choose arbitrarily a zero point 0 and three unit points Q1,Q2, and Qs in such a way that the four points do not lic in a plane, Each of12 VECTOR SPACES AND GEOMETRY 37 the unit points Q, determines a Iine Ly through 0 and a coordinate correspon- dence on this line, as defined above. The three lines L1, L, and Ls are called the coordinate axes. Consider now any point X in ES. “The plane through X parallel to Lz and Ls intersects Ly at a point Xy, and therefore determines a number 21, the coordinate of X, on Ly. Ina similar way, X determines points Xz on Lz and X3 on Ls which have eo- ordinates 22 and a, respectively. Alto- gether X determines a triple XS Xen teta> in B°, and we have thus defined a mapping, a: Xx from E* to B® (see Fig, 1.4). We call @ the coordinate correspondence defined by the axis system. The conver tion implicit in our notation above is that A(Y) is y, (A) is a, ete. Note that the ‘unit point Q, on L1 has the coordinate triple a! = <1, 0,0>, and similarly, that (2) = 8 = <0, 1,0> and (Qa) = 8 = <0,0,1> Figs Lt ‘There are certain basie facts about the coordinate correspondence that have to be proved as theorems of geometry before the correspondence ean be used to treat geometric questions algebraically. ‘These geometric theorems are quite tricky, and are almost impossible to discuss adequately on the basis of the ustal secondary school treatment of geometry. We shall therefore simply assume them, They are: 1) Gis bijection from E* to R. }) ‘Two line segments AB and XY are equal in length and parallel, and the direction from A to B is the same as that from X to ¥ if and only if b — a= y —x (in the veetor space R'), This relationship between line segments is important enough to formalize. A directed line segment is a geometric line sex rent, together with a choice of one of the two directions along it. If we interpret AB as the directed line segment from A to B, and if we define the directed line segments AB and XY to be equivalent (and write AB ~ XY) if they are equal in length, parallel, and similarly directed, then (2) ean be restated AB~XY o b—a- 8) If X ¥ 0, then Y is on the line through 0 and X in E* if and only if y = tx for some fin R. Moreover, this tis the coordinate of ¥ with respeet to X ‘as unit point on the line through 0 and X.38 vector spaces 12 (ua. y-2) =1XVP (oa) ~\0r? Fig. 15 4) If the axis system in E* is Cartesian, that is, if the axes are mutually perpendicular and a common unit of distance is used, then the length |OX| of the segment OX is given by the so-called Euchidean norm on R°, |OX| = (Li 2*)"*. This follows directly from the Pythagorean theorem. ‘Then this formula and a second application of the Pythagorean theorem to the triangle OXY imply that the segments OX and OY are perpendicular if and only if the sealar product (x, y) = Lifes 20; has the value 0 (see Fig. 1.5). In applying this result, it is useful to note that the scalar product (x, y) is linear as @ funetion of either veetor variable when the other is held fixed. ‘Thus 2 a (ext dy, 2) =D (erst dudes tart dD yee = le, 2) + aly, 2) actly the same theorems hold forthe coordinate eorespondenee between the Euclidean plane E* and the Cartesian 2-space R*, except that now, of course, (wy) = Eh ean = zaps says. ‘We can easily obtain the equations for ines and planes in E* from these basic theorems. First, we see 4 ~\ from (2) and () that if fined points A and B are given, x th A #0, then the line through B parallel to the egment OA contains the point X ifand only i there 9 cnten acalar (such that x — b— ta (see Fig. 1.8). ‘Therefore, the equation of this line is ts x=la+b. Fig. 1.6 ‘This vector equation is equivalent to the three numerical equations x; = ‘ait ++ b;, i = 1,2, 3. These are customarily ealled the parametrie equations of the ine, since they present the coordinate triple x of the varying point X on the line as functions of the “parameter” t.12 VECTOR SPACES AND GEOMETRY 39 Next, we know that the plane through B perpendicular to the direetion of the segment OA contains the point X if and only if BX OA, and it therefore follows from (2) and (4) that the plane contains X if and only if (x — b, a) = 0 (Gee Fig. 1.7). But (x ~ b, a) = (x, a) — (b, a) by the linearity of the sealar product in its first variable, and if we set 1 = (b, a), we see that the equation of the plane is (x, a) or Do aees ‘That is, a point X is on the plane through B perpendicular to the direction of OA if and only if this equation holds for its coordinate triple x. Conversely, ifla + 0, then we can retrace the steps taken above to show that the set of points X in E* whose coordinate triples x satisfy (x, a) = lis a plane. ‘The fact that R® has the natural sealar product (x,y) is of course extremely important, both algebraically and geometrically. However, most vector spaces do not have natural sealar products, and we shall deliberately neglect scalar produets in our early veetor theory (but shall return to them in Chapter 5). ‘This leads us to seek a different interpretation of the equation L} We saw in Seetion I that x 5} azzs is the most general linear functional f on 8°. Therefore, given any plane IM in E®, there is a nonzero linear functional f on R¥ and a number / such that the equation of M is f(x) =. And conversely, given any nonzero linear functional f:R® —> Rand any 1 R, the locus of Lis a plane A in E®. ‘The reader will remember that we obtain the coefficient triple a from f by a= J(H), sinee then f(x) = f(EP x8) = ix: Finally, we seek the vector form of the notion of parallel translation, In plane geometry when we are considering two congruent figures that are pau and similarly oriented, we often think of obtaining one from the other by “sli40 vector spaces 12 ane along itself” in such a way that all lines remain parallel to their original positions, This description of a parallel translation of the plane can be more clegantly stated as the condition that every directed line segment slides to an equivalent one. If X slides to Y and 0 slides to B, then OX slides to BY, so that OX ~ BY and x = y — b by (2). Therefore, the coordinate form of such ‘8 parallel sliding is the mapping x + y = x +b. Conversely, for any b in R? the plane mapping defined by x-> y = x +b is easily seen to be a parallel translation. ‘These considerations hold equally well for parallel translations of the Eucldean space E°, It is geometrically clear that under a parallel translation planes map to parallel planes and tines map to parallel lines, and now we ean expect an easy algebraic proof. Consider, for example, the plane Mf with equation f(x) = 0; let us ask what happens to Mf under the translation x+y = x-+b. Since x = y — b, we see that point x is on A if and only if its translate y satisfies ‘the equation f(y — b) = Lor, since f is linear, the equation fy) = U, where 1 =1+J(b). But this is the equa‘ion of a plane N. Thus the translate of is the plane N. It is natural to transfer all this geometric terminology from sets in E to the corresponding sets in R® ard therefore to speak of the set of ordered triples x satisfying f(x) = Las a set of points in R® forming a plane in B°, and to call the mapping x++ x +b the (parallel) translation of R* through b, ete. Moreover, since R'is a vector space, we would expect these geometric ideas to interplay with veetor notions. For instance, translation through b is simply the operation of adding the constant veetor b: x x +b. Thusif Af isa plane, then the plane NV obtained by translating M through b is just the vector set, sum M+. If the equation of M is f(x) = J, then the plane AT goes through 0 if and only if 1 = 0, in which ease A isa veetor subspace of 8 (the null space off). It is easy to sce that any plane M isa translate of a plane through 0. Similarly, the line {la-+ b:1R) is the translate through b of the line {ta : £ R}, and this second line is a subspace, the linear span of the one veetor a. ‘Thus planes and lines in B ave translates of subspaces. ‘These notions all carry over to an arbitrary real veetor space in a perfectly satisfactory way and with additicnal dimensional variety. A plane in R® through 0 is a veetor space which is vo-dimensional in a strictly algebraic sense which we shall discuss in the next chapter, and a line is similarly one-dimensional. In R® there are no proper subspaces other than planes and lines through 0, but in a veetor space V with dimersion n > 3 proper subspaces occur with all dimensions from 1 ton — 1. We shall therefore use the term “plane” loosely to refer to any translate of a subspacs, whatever its dimension. More properly, translates of vector subspaces are called afine subspaces. We shall see that if V is a finite-dimensional space with dimension n, then the null space of a nonzero linear furetional fis always (n — 1)-dimensional, and therefore it cannot be a Euclidean-like two-dimensional plane exeept when12 VECTOR SPACES AND GEOMETRY 41 n= 8. Weuse the term hyperplane for such a null space or one of its translates ‘Thus, in general, a hyperplane is a set with the equation f(x) = 1, where fis ‘nonzero linear functional. Tt is a proper afine subspace (plane) which is maxi: mal in the sense that the only affine subspace properly including it is the whole of V. In R® hyperplanes are ordinary geometric planes, and in R? hyperplanes are lines! EXERCISES 2.1 Assuming the theorem AB ~ XY © b—a = y — x, show that OC is the sum of Od and OB, as defined in the preliminary discussion of Section 1, if and only if © =b-+a. Considering also our assumed geometric theorem (8), show that the mapping x++ OX from R® to the vector spare of geometric vectors is linear and hence an isomorphism, 2.2 Let L be the line in the Cartesian plane R? with equation zp = 3x1. Express L in parametric form as x = ta for a suitable ordered pair a, 2.3 Let V be any vector space, and let a and g be distinct vectors. Show that the line through a and has the parametric equation E=B+ da, ER. ‘Show also that the segment from «to 8 is the image of (0, 1] in the above mapping 24 According to the Pythagorean theorem, a triangle with side lengths a, , and ¢ hhas a right angle at the vertex “opposite c” if and only if c? = a? + b?. NS Prove from this that in « Cartesian coordinate system in E* the length [OX of @ segment OX is given by jox}? = ya, where x = <21, 22, 25> is the coordinate tripleof the point X. Next use our geometric theorem (2) to conclude that OXLOY ifandonly if (x,y) =0, where (x,y) = Dra (Use the bilinearity of (x, y) to expand |X — ¥]*) 2.5 More generally, the law of cosine says that in any triangle labeled as indicated, B= a? + 88 = Bab cos 0.42° vreron spaces 12 AN, (19) = Allyl e088, where (x, y) is the sealar produet D7 zis, |x] = (x, x)? = [OX], ete. 2.6. Given a nonzero linear unctioml f:R® — B, and given FE B, show that the set of points X in E® such that f(a) = Kisa plane. [/fint: Find a b in B® such that 1, and throw the equation fs) = # into the form (x — bh, a) = 0, ete] Show that for any b in R® the mapping X-+ ¥ from E¥ to itself defined by “+ b is parallel translation. “That is, show that if X-+ ¥ and ZW, then Apply this law to the diagram to prove that 2.8 Let Mf be the set in R with equation 314 — x2 +a = 2. Find tripletsa and b such that M is the plane through b perpendicular to the direction of a, What is the equation of the plane P= M+} <1,2,1>? 2.9 Continuing the above exercise, what is the condition on the triplet b in order for N= Mt hte pass theongh the origin? What is the equation of N? 2.10 Show that if the plane AM in R has the equation f(x) = J, then M isa translate of the null space NV of the linear functional f. Shovr that any two translates Mf and P entical or disjoint, What is the condition on the ordered triple b beat Generalize the above exercise to hyperplanes in BA, Of Nave either k in order that AP 2 BAZ _Let N be the subspace (plane through the origin) in R with equation f(x) = 0. Let and P be any two planes obtained from N by parallel translation. Show that Q = MP isa thied such plane, If Af and P have the equations f(x) = fy and {fls) = la find the equation for 2.13 If is the plane in R® with e uation f(x) = [, and if ris any nonzero number, show that the set produet rif isa plane parallel to 37 2.14 In view ofthe above two exercises, discuss how we might consider the set of all Parallel translates of the plane .V with equation f(x) = 0 as forming a new vector space. ‘Let L be the subspace (line through the origin) in R* with parametric equation Diseuss the set ofall parallel translates of 1 in the sprit of the above three 2.16. Tho best object to take as “bein,” the geometric vector TH is the equivalence flats of all directed ine segments XY such that XY ~ .1B. Assuming whatever you heed from properties (1) through (1), show that this fan equivalence relation on the Set of all directed line segments (Seven 0.12). 2.17 Assuming that the geometric vector AB is defined as in the above exercise, show that, strictly speaking, i is actually the mapping of the plane (or space) into itself that we have called the parallel translation through 1B, Show also that AB + CD is the ‘composition of the two translations.13 PRODUCT SPACES AND HOM(Y,m) 43. 3. PRODUCT SPACES AND HOM(Y, 1) If W is a vector space and A is an arbitrary set, then the set W-valued functions on A is a veetor space in exactly the same way that Ris. Addition is the natural addition of funetions, (f+ 9)(a) Ja) + g(a), and, similarly, (z/)(a) = 2((a)) for every funetion f and sealar x. Tavs AI through 84 follow just as before and for exactly the same reasons. For variety, let us check the associative law foraddition. ‘The equation f-+ (g++ h) = (F+49) +h means that (f+ (+ H))(a) = (f+ 9) +h)(@) for all ae A. But + OF D)@ = fa) + G+ NO) Sta) + (ala) + Ma)) = (fa) + o(@)) + Ma) = F+9@) +N) = (F+ 0) +2), where the middle equality in this chain of five holds by the associative law for W and the other four are applications of the definition of addition, ‘Thus the associative law for addition holds in W4 beeause it holds in W, and the other Jaws follow in exactly the same . As before, we let m; be evaluation at i, so that w,(f) = (i). Now, however, 7; is vector valued rather than scalar valued, because itis a mapping from V to W, and we eal it the ith coordinate projection rather than the ith coordinate functional. Again these maps are all linear. In fact, as before, the natural vector operations on W4 are uniquely defined by the requirement that the projections 1; all be linear. We call the value f(j) = ‘x,(f) the jth coordinate of the vector f. Here the analogue of Cartesian n-space is the set W* of all n-tuples a = of veetors in W; it is also designated W*. Clearly, ay is the jth coordinate of the n-tuple a. ‘There is no reason why we must use the same space W at each index, as we did above. In fact, if W:,..., Wy are any a vector spaces, then the set of all netuples @ = such that aj € W; for j= 1,...,n is a vector space under the same definitions of the operations and for the same reasons. ‘That is, the Cartesian product W = W, x Wz X+++X Wy is also a vector space of vector-valued functions. Such finite products will be very important tous. Of course, R is the product TT} Ws with each Wy = R; but R* can also be considered R" x R"—", or more generally, []f Ws, where Wy = R™ and Xfm; = n. However, the most important use of finite product spaces arises from the fact that the study of certain phenomena on a vector space V may Jead in a natural way to a collection {Vi}} of subspaces of V such that V is isomorphic to the product IT Vi. ‘Then the extra structure that V acquires when we regard it as the product space IT} V's is used to study the phenomena in question. This is the theory of direct sums, and we shall investigate it in Section 5. Later in the course we shall need to consider a general Cartesian product of vector spaces. We remind the reader that if {W;:i € I} is any indexed collection of vector spaces, then the Cartesian produet [Tier Wi of these vector spaces is44° vector spaces 13 defined as the set of all functions f with domain I such that j(é) € Ws for all TET (see Section 0.8). ‘The following is a simple conerete example to keep in mind. Let S be the ordinary unit sphere in R°, = {x: Ez? = 1}, and for each point x on S let W, be the subspace of R? tangent to 8 at x. By this we mean the subspace (plane through 0) parallel to the tangent plane to S at x, so that the translate W, +x is the tangent plane (soe Fig. 1.8). A funetion f in the product space W'= [lees W, is a function which assigns to each point x on S a veetor in W, that is, a veetor parallel to the tangent plane to S at x. Such a function is ealled ‘a vector field on S. ‘Thus the produet set: WV is the set of all vector fields on S, and W itself is a veetor space, as the next theorem states Fig. 18 Of course, the jth coordinate projection on W = TTies Ws is evaluation at j, m)(f) = JG), and the natural vector operations on W are uniquely defined by the requirement that the coordinate projections all be linear. ‘Thus f+ must be that element of W whose value at J, x)(F-+ 9), is (1) + mila) = JG) + 9G) forall j € I, and similarly for multiplication by sealars. Theorem 81. The Cartesian product of a collection of vector spaces can be made ito a vector space in exactly one way so that the coordinate pro- jeotions are all linear. Proof. With the vector operations determined uniquely as above, the proofs of ‘Al through S4 that we sampled earlier hold verbatim. ‘They did not require that the functions being added have all their values in the same space, but only that, the values at a given domain element ¢ all lie in the same space. Hom(¥, 7). Linear transformations have the simple but important properties that the sum of two linear transformations is linear and the composition of two Tinear transformations is Tinear. ‘These imprecise statements are in essence the theme of this section, although they need bolstering by conditions on domains and codomains. Their proofs are simple formal algebraic arguments, but the objects being discussed will inerease in conceptual complexity.13 PRODUCT SPACES AND HoM(y,™) 45 If Wis a veetor space and A is any set, we know that the space 14 of all mappings f: A — W is a veetor space of functions (now vector valued) in the same way that R4 is. If A is itself a vector space V, we naturally single out for special study the subset of 7” consisting of all lincar mappings. We designate this subset Hom(V, 17). ‘The following clementary theorems summarize its basie algebraic properties. ‘Theorem 3.2, Hom(V, W) is a vector subspace of 1". Proof. The theorem is an easy formality. If $ and T are in Hom(¥’, W), then S+ 1) (ea + yB) = Slea+ vB) + Tea + vB) #8(a) + yS(6) + 27 (a) + PG) = a(S + TH(a) + iS + THE), so S + Tis linear and Hom(V, W) is closed under addition. The reader should be sure he knows the justification for each step in the above eontinued equality ‘The closure of Hom(V, W) under multiplication by scalars follows similarly, and sinee Hom(V, W) contains the zero transformation, and so is nonempty, it is a subspace. 0 ‘Theorem 3.3. ‘The composition of linear maps is linear: if 7 € Hom(V, 1) and 3 © Hom(W, X), then Se 7 ¢Hom(V, X). Moreover, composition is distributive over addition, under the obvious hypotheses on domains and codomains: Gi tS) eT =SyoT+S:07 and So(Ty +12) Finally, composition commutes with salar multiplication: S01) = (cS) oT = $07) oT 4+8oT, Proof. We have Seo Tea+ yB) = S(T(@a+ y8)) = S(zT(a) + yT(B)) S(T(a)) + yS(T(B)) = x(8 © THa) + y(S 2 THB, so Se js linear. The two distributive laws will be left to the reader. 0 Corollary. If T’ < Hom(V, 1) is fixed, then composition on the right by T” is a Inear transformation from the veetor space Hom({V, X) to the vector space Hom(V,X). It is an isomorphism if 7’ is an isomorphism. Proof. ‘The algebraic properties of composition stated in the theorem can be ined as follows: (eyS1 + cpp) © T= e4(S1 © 7) + cn(S2 © 7), Se (eT s + e272) = ex(S © Ts) + a(S ° Ta). ‘The first equation says exactly that composition on the right by a fixed T isa linear transformation. (Write S © T' as 5(S) if the equations still don’t look right.) If 7 is an isomorphism, then composition by T—! “undoes” composition by 7, and so is its inverse. 046 vecron spaces 13 ‘The second equation implies a similar corollary about composition on the left by a fixed S. ‘Theorem 3.4. If W isa product veetor space, W = []y Ws, then a mapping 1 from a veetor space V to W is linear if and only if x; 7 is linear for each coordinate projection m: Proof. It T is linear, then w; 0 T is linear by the above theorem. N conversely, that all the maps a; 7’ are linear. Then ai(T(ea + yB)) = ws0 Tea + y8) = aC © TH(a) + ulmi e TYG) am(T(a)) + ymi(P(B)) = mi(2T(@) + yT(@)). But if (J) = wig) for all 4, then f= g. Therefore, T(ta-+ y8) = xT(a) ++ 0B), and T is linear. O If 7 is linear mapping from R* to W whose skeleton is {8;)*, then myo 7 has skeleton {7,(3,)}3u1. If W is BY, then x; is the sth coordinate functional y+ yo and 6; is the th column in the matrix t = {05} of T. Thus 7(B;) = tis and ;© Tis thelinear funetional whose skeleton isthe ith row of the matrix of 7. In the discussion centering around Theorem 1.3, we replaced the vector equation y = T(x) by the equivalent set of m scalar equations yj = Ear fs which we obtained by reading off the ith coordinate in the vector equation, But in “reading off” the ith coordinate we were applying the coordinate mapping ‘ror in more algebraie terms, we were replacing the linear map T by the set of linear maps {;° 7}, which is equivalent to it by the above theorem. Now consider in particular the space Hom(V, V), which we may as well designate ‘Hom(V)’- In addition to being a vector space, itis also closed under composition, which we consider a multiplication operation. Sinee composition of functions is aluays associative (see Section 0.9), we thus have for multipliea- tion the laws Ae (BoC) = (Ae B)oC, Ae (B+C)= (Ao B)+ (AoC), (A+B) eC = (AeC) + Boo), K(A © B) = (A) o B= Ao (KB) Any veetor space which has in addition to the vector operations an operation ‘of multiplication related to the veetor operations in the above ways is called aan algebra, ‘Thus, ‘Theorem 3.5. Hom(V) is an algebra, Wo noticed earlier that certain real-valued function spaces are also algebras Examples were R4 and €((0, 1). In these cases multiplication is eommutative, Dut in the ease of Hom(V) multiplication is not commutative unless V is a trivial space (V = {0}) oF V is isomorphie to R. We shall check this later when ‘we examine the finite-dimensional theory in greater detail,13 PuoDUCT SPacES AND Hom. 47 Product projections and injections. In addition to the coordinate projections, there is a second class of simple linear mappings that is of basie importanee in the handling of a Cartesian product space W = [[yex Ws. These are, for each 4, the mapping @; taking a veetor a W; to the function in the product space having the value « at the index j and 0 elsewhere. For example, 03 for Ws x W2 x W, is the mapping a++ <0, a, 0> from Ws to W. Or if we view RB? as RX RY, then dp is the mapping <22,25> + <0, <2, 23> > = <0,22, 43> We eal 9; the injection of W; into Ths Ws, The linearity of 8 is probably obvious. The mappings x; and 8; are clearly connected, and the following projection- injection identities state their exact relationship. If I; is the identity trans- formation on W), then mjeG=T; and wye8j=0 it ie) If K is finile and I is the identity on the produet space W, then In the case [[f1 Wi, we have 62° ro() = <0,a2,0>, and the identity simply says that ++ <0, a2,0> + <0,0,a3> = Xar, a, 43> for all a, az, a3. These identities will probably be elear to the reader, anid we leave the formal proofs as an exercise. The coonlinate projections x, are useful in the study of any product space, but beeause of the limitation in the above identity, the injections 4; are of interest principally in the ease of finite products. ‘Together they enable us to decompose and reassemble linear maps whose domains or codomains are finite product spaces. For a simple example, consider the 7 in Hom(R%, R®) whose matrix is 2-1 1 11 4, ‘Then #1 © Tis the linear functional whose skeleton <2, —1, 1> is the first row in the matrix of 7, and we know that we ean visualize its expression in equation form, ys = 221 — 2+ 2s, as being obtained from the veetor equation y T(x) by “reading off the first row”. Thus we “decompose” into the two linear functionals {j= x;° 7. Then, speaking loosely, we have the reassembly T= ; more exactly, T(x) = <2ry— 1p ty, t+ 42+ 4ty> = for all x. However, we want to present this reassembly as the action of the linear maps @, and @. We have = 61(L1G9)) + Oa(lals)) = (01 © m1 + 82° a)(TE)) = TOW), which shows that the decomposition and reassembly of 7’ is an expression of the identity 5 a0 7 = I. In general, it T © Hom(V, W) and W = TI, W, then Ty= mo isi V, W0) for cach #, and 1% ean be considered “the part of T going into W,”, since T(a) is the ith coordinate of T(a) for each a. ‘Then we48 vector spaces can reassemble the 17's to form T again by T= E06: Ts, for £02 Ts (Eoem)eT=1eT=T. Moreover, any finite collection of 7y's on a common domain ean be put together in this way to make a 7. For example, wwe can assemble an metuple {7,}1' of linear maps on a common domain V to form a single m-tuple-valued linear map 7. Given a in V, we simply define T(q) as that. m-tuple whose ith coordinate is Ti(a) for i= 1,...,.m, and then check that is linear. ‘Thus without having to ealeulate, we see’ from this assembly principle that T:x ++ <2ry — 22+ 2421+ 2-+ 44> is a linear ‘mapping from R® to R2, sinee we have formed 7 by assembling the two linear functionals ty(x) = 221 — 22+ 25 and le(x) = 21+ a+ 41g to form a single ordered-pair-valued map. ‘This very intuitive process has an equally simple formal justification. We rigorize our diseussion in the following theorem ‘Theorem 3.6. If 7; is in Hom(V, W:) for cach i in a finite index set I, and if W is the product space [Tier Wj, then there isa uniquely determined ‘Tin Hom(V, W) such that 7; = ay T for all é in 1 Proof. If T exists such that T; = ;° T for each i, then T= Iw © (Lem) T = LHe (xe T) = LG; T;. Thus 7 is uniquely determined as £6 T;. Moreover, this T’ does have the required property, since then ry 9 (3, Oe Ti) = Zi (ajo odo Vs xyoT =Tet= 7,0 Tn the same way, we ean decompose a linear T whose domain is a produet space V = [Ti Vj into the maps 7; = 7» 8; with domains Vj, and then reassemble these maps to form T by the identity T= Ejay 7) 0 x; (cheek it mentally}. Moreover, a finite collection of maps into a common codomain, space ean be put together to form a single map on the produet of the domain, spaces. ‘Thus an n-tuple of maps {7;}7 into W defines a single map 7 into W, where the domain of is the produet of the domains of the 7,’s, by the equation P(X ay. y4n>) = EI Tila) or T = LI Tye w;. For example, it 7:8 — R? is the map {> 12, 1> = <2,0>, and Tz and 's are similarly the maps to t<—1,1> and (+5 ¢<1,4>, then T= E} Tse 7; is the mapping from RS to B? whose matrix is there is a simple formal argument, and we shall ask the reader to write out the proof of the following theorem. ‘Theorem 3.7. If 7; is in Hom(V, W) for each j in a finite index set J, and if V = [jer Vis then there exists a unique T in Hom(V, 17) such that T° 0; = for each j in J. Finally we should mention that Theorem 3.6 holds for all produet spaces, finite or not, and states a property that characterizes product spaces. We shall13 PRODUCT SPACES AND HOM. 49 nvestigate this situation in the exercises. ‘The proof of the general case of ‘Theorem 8.6 has to get along without the injections 6; instead, it isan application of Theorem 3.4. ‘The reader may feel that we are being overly formal in using the projeetions ; and the injections 0; to give algebraic formulations of processes that ate visualized directly, such as reading off the sealar “components” of a veetor equation. However, the mappings xe and ye <0;.- 0,50 0,22 OF are clearly fundamental doviees, and making their relationships explicit now be helpful to us later on when we have to handle their oeeurrenees in more complicated situations. Show that RB" X Ris isomorphic to R™™ Show more generally that if I ne = n, then TTP, RY is isomorphie to R*. Show that if (2, C} isa partitioning of 1, then Be" and RX RE are isomorphic. Generalize the above to the ease where (1) partitions 4. Show that a mapping 7 from a veetor space 1 to a vector space I is linear if tnd only if (the graph off 7 is a subspace of VX. W, 8.6 Let S and be nonzero linear maps from V to 1. The definition of the map S++ Tis not the same as the set sum of (the graphs of) Sand T'as subspaces of VX 1 Show that the set sum of (the graphs of) S and 7’ cannot be a graph uinless S 3 32, 3.8 Prove the distributive laws given in ‘Theorem 3.3, 3.9. Let D:el((a, d)) — elfa,H) be differentiation, and let 8:€%(a, Hl) > B he the dofinite integral map f+ J2J.” Comte the composition Se D. Give the justifieation for each step of the ealeulation in Theore 3.10 We know that the general linear funetional F on Ris the map x arzi -| ase determined by the pair a in R2, and that the general linear map 7 in Hom determined by a mates, ‘Then Fe T is another linear functional, and hence is of the form «> bury + bare for some bin R2, Compute b from t and a, Your computation should stow yon that ams bis linear. What is its matrix? BAIL Given S and Tin Hom(R2) whose matrices are fa] [il respectively, find the matrix of Se 7 in Hom(R2)50 VECTOR spaces. 13, 3.12 Given S and Tin Hom(R*) whose matrices are ef ee] oe fe, if Sand T are in Hom(R*)? Verify your guess. isan isomorphism Se T surjective S surjective, $2 P injective 7 injective, and, therefore, that if 7’ © Hom(V, W), $¢ Hom(W, V), and SoT=y, TeSm1, then 7 is an isomorphism, 3.15 Show tha: if S-! and 7-t exist, then (So 7) exists and equals T-! 0 S: Give a more caroful statement of this result. 3.16 Show that if Sand 7’in Hom ¥ commute with each other, then the null space of T,N = N(P), anditsrange R = R(T) are invariant under $ (SIN]CN and SURIC R). BAIT. Show that if a is an eigenvector of T and $ commutes with 7, then S(a) is fan eigenvector of T and has the same cigenvalue, 3.18 Show tha: if $ commutes with 7 and 7-1 exists, then S commutes with 1 3.19 Given that ais an eigenvector of T' with eigenvalue 2, show that « is also an eigenvector of T2 = To 7, of T*, and of T-* (if T is invertible) and that the corre sponding eigenvalues are 22, 2%, and 1/2. Given that p() isa polynomial in , define the operator p(7), and under the above hypotheses, show that a is an eigenvector of p(T) with eigenvalue p(2). 3.20 If S and T are in Hom V, we say that $ doubly commutes with T (and write ‘See 7) if § commutes with every 4 in Hom V which commutes with 7. Fix 7, and set {7}"" = {S:SeoT}. Show that (7) is a commutative subalgebra of Hom V. 3.21 Given Tin Hom V and a in V’, let NV be the linear span of the “trajectory of a ‘under 2" (the set {Pain €Z*}). Show that N is invariant under 7. 8.22 Atransformation Tin Hom V such that 7 = 0 for some n issaid to be nilpotent. Show that if 7s nilpotent, then J — Tis invertible. [Jin The power series is a finite sum ifz is replaced by T.] 3.23 Suppose that 1'is nilpotent, that $ commutes with 7, and that S~! exists, where 8, T © Hom V. Show that (S — T)~! exists 3.24 Lety be an isomorphism from a veetor space V to a veetor space W. Show that T+ go Teg is an algebra isomorphism from the algebra Hom V to the algebra Hom IF,13 PRODUCT SPACES AND HOM) 51 8.25 Show the x's and 8,'s explicitly for BS = BX BX R using the stopped arrow notation. Also write out the identity 5 @;¢ xj = J in explicit form. 8.26 Do the same for B = BR? x RS, 3.27 Show that the first two projection-injection identities (ry® @ = Ieand xj2 6; = 0 if j x 1) are simply a restatement of the definition of 6. Show that the linearity of 6, follows formally from these identities and Theorem 3.4 3.28 Prove the identity 5 6/2 x; = Iby applying x; to the equation and remembering that f = 9 if x5(f) = ms{9) for all j (this being just the equation (4) = (3) for all j)- 3.29 Prove the general ease of Theorem 86. We are given an indexed collection of linear maps {Ts:4€ 1} with common domain V and codomains {W,:i eT}. The first question is how to define T:¥ — W = TJ; Ws. Do this by defining 7(® suitably for each £€ V and then applying ‘Theorem 3.4 to conclude that Tis linear. 3.30 Prove T 3.31 We know without ealeulation that the may corem 37 Tex > <38y — rat 33,22 25,21 — B45, 21> from B® to Rt is linear. Why’ (Cite relevant theorems from the text.) 8.82 Write down the matrix for the transformation T'in the above example, and then ‘write down the mappings 1'° @, from R to RY (lor i = 1, 2,8) in explicit ordered quadruplet form. 3.33 Let W = TIF Ws be a finite product veetor space and set pi = 852 x5, 60 that isin Hom W for all i. Prove from the projection-injection identities that 3 ps = I (the identity map on W), pie pj = Oi i # J, and pie ps = pi. Identify the range Re = Rip. In the context of the above exer dofine Tin Hom W as Xm Show that ais an eigenvector of T'if and only if a isin one of the subspaces Ri and that then the eigenvalue of e isi 3. In the same situation show that the polynomial (P= Ne--o (P=) is the zero transformation, 3.86 Theorems 3.6 and 3.7 can be combined if T€ Hom(V, W), where both V and 1 fre product spaces: veTIv, ad) w=T[w State and prove a theorem which says that such « 7 can be decomposed into a doubly indexed family (7,} when Ts € Hom(Vs, W;) and conversely that any such doubly indexed family can be assembled to form a single T form ¥ to W.52 vector spaces, 14 8.87 Apply your theorem to the special ease where V = RY and IV = R™ (that is, Vi = Wy = B for all Gand j). Now Ty is from B to Band hence is simply multipli- cation by a number fy. Show that the indexed collection {t,} of these numbers is the matris of 7. 3.38 Given an metuple of vector spaces {1}, suppose that there are a veetor space Vand maps psin Hom(X, 1), = 1,...,m, with the following property: P. For any m-tuple of linear maps (71 from a common domain space V to the above spaces I; (so that T/€ Hom(V, Wi), i= 1,...,m), there is a unique 7 in Hom(V, X) sueh that ;'= pio T,i = 1... ,m. Prove that there is a “eanonical” isomorphism from we TLw, to x under whieh the given maps p: become the projetions mz. [Remarks ‘The produet space W itsolf has property P by Theorem 3.6, and this exercise therefore shows that P is an abstract characterization of the product space.) "INE SUBSPACES AND QUOTIENT SPACES section we shall look at the “planes” in a veetor space V and see what happens to thom when we translate them, intersect them with each other, take their images under linear maps, and s0 on. ‘Then we shall confine ourselves to the sot of al. planes that aro translates of a fixed subspace and diseover that, this set itself is a veetor spaco in the most obvious way. Some of this material hhas been anticipated in Section 2. Affine subspaces. If N is a subspace of a vector space V and a is any veetor of V, then the set V-}a= {€4-a:£EN)} is called cither the coset of N containing « or the affine subspace of V through a and parallel toN. ‘The set N +a is also called the translate of N through a. We saw in Section 2 that affine sub- spaces are the general objects that we want to eall planes. If’ is given and fixed in a discussion, we shall use the notation a = N ++ a (see Seetion 0.12). We begin with a list of some simple properties of affine subspaces. Some of these will gene-alize observations already made in Section 2, and the proofs of some will be left as exercises 1) With a fixed subspace ¥ assumed, if Yea, then ¥= a For if 7 = aor then Y= at (n9-+ 2) €2, 807 CH. Alsoabn = 7+ (9 — m0) EF, s0%C7. Thus a= 7. 2) With N fixed, for any @ and 8, either 2 = 8 or @ and J are disjoint. For if @ and Bare not disjoint, then there exists a ¥ in each, and a= 7 = 8 by (1). ‘The reuder may find it illuminating to compare these calculations with the more general ones of Section 0.12, Here @ ~ 8 if and only if a ~ 8 e) 3) Now lo: @ be the collection of all affine subspaces of V;@ is thus the set of all cosets of all vector subspaces of V. ‘Then the interscetion of any sub-14 AFFINE SUBSPACES AND QUOTIENT SP\CES 53 family of @ is cither empty or itself an affine subspace. In fact, if (Ai}cer is ‘an indexed collection of affine subspaces and A, is a coset of the vector subspace W, for each i € 1, then Mer Acis either empty or a coset of the veetor subspace Aser We. For if 8 € Myer Ay, then (1) implies that Ay = 8 + Wy for all i, and then NA; = 6+ NW. 4) If A, Bea, then A+ BEG. That is, the set sum of any two affine subspaces is itself an affine subspace. 5) If A @@and Tc Hom(V, W), then T[A] is an affine subspace of W. In particular, if ¢ € R, then td € @. 6) If Bis an affine subspace of W and T ¢Hom(V, 1), then TB] is either empty or an affine subspace of V. 7) Por a fixed a € V the translation of V through a is the mapping SeiV > V defined by So() = €+ 4 for all ¢€ V. Translation is not linear; for example, Sa(0) = a. It is clear, however, that translation carries affine subspaces into affine subspaces, Thus S,(4) = A+ and S,(8-+ 1) = (e+) + W. 8) An affine transformation from a vector space V to a veetor space W is a linear mapping from V to W followed by a translation in W. ‘Thus an affin transformation is of the form £ +» T(8) + 8, where 1’ € Hom(V, W) and 8 € W. Note that + T(¢+ a) isaffine, since TE+ a) Ta). T() +6, — where Tt follows from (5) and (7) that an affine transformation carries affine subspaces of V into affine subspaces of W. Quotient space. Now fix a subspace N of V, and consider the set W of all translates (cosets) of N. We are going to see that W itself is a vector space in the most natural way possible. Addition will be set addition, and scalar multipli- cation will be set multiplication (except in one special ease). For example, if N isa line through the origin in R®, then W consists of all lines in R® parallel to N. We are saying that this set of parallel lines will automatically turn out to be a vector space: the set sums of any two of the lines in WV turn out to be a line in W! And if L € W and t # 0, then the set product tL is a line in W. ‘The translates, of L fiber B°, and the set of fibers is a natural vector spac During this discussion it will be helpful temporarily to indicate set sums by “he and set products by ‘,’. With NV fixed, it follows from (2) above that two cosets are disjoint or identical, so that the set W of all cosets is a fibering of V in the general case, just as it was in our example of the parallel lines. From (4) or by a direct calculation we know that #-+,8= a8. Thus W is closed under set addition, and, naturally, we take this to be our operation of addition on W. ‘That is, we define + on W by a+ 8 = a-+, 8, Then the natural map m:ar+a from V to W preserves addition, x(a-+ 8) = (a) + (8), since54 vecrou sraces 1 this is just our equation a-F B= a +B above, Similarly, if (© R, then the set product f+, & is either fa or {0}. Hence if we define as the set product when 1 Oand as 6 = N when ¢= 0, then x also preserves sealar multiplication, (la) t(a). We thus have two vectorlike operations on the set W of all cosets of N, ‘and we naturally expect 17 to turn out to bea veetor space. We could prove this, by verifying all the laws, but it is more elegant to notice the general setting for such a verifieation proof. ‘Theorem 4.1. Let V be a veetor space, and let W be a set having two veetorike operations, whieh we designate in the usual way. Suppose that there exists a surjective mapping T': V —+ I whieh preserves the operations: Toa + (8) = sP(a) + tT(@). ‘Then W isa vector space. Proof. We have to check laws Al through $4. However, one example should make it clear to the reader how to proceed. We show that 7'(0) satisfies A3 and hence is the zero vector of W. Since every 8 € W is of the form Ta), we have 70) +8 = TO) + Ta) = TO + @) = Te) = 6, which is A3, We shall ask the reader to cheek more of the laws in the exereises. ‘Theorem 4.2. ‘The set of cosets of a fixed subspace NV of a vector space themselves form a vector space, called the quotient space V/N, under the above natural operations, and the projection x is a surjective linear map from V to V/N. ‘Theorem 4.3, If T is in Hom({’, W), and if the null space of 7'ineludes the subspace M CV, then 7 has a unique factorization through V/M. ‘That there exists a unique transformation S in Hom(V/M, W) such that T= Som Proof. Since T is 2070 on M, it follows that T is constant on each coset A of A, so that T[A] contains only one vector. If we define S(A) to be the unique vector in T[A], then S(a) = T(a), so 8 om = T by definition. Conversely, if T= Rea, then R@) = Ko x(a) = Ta), and Ris ourabove S, The linearity of S is practically obvious, ‘Thus S@+A) = S@FA) = Ta +8) = Te) +70) = S@) +58, ‘and homogeneity follows similarly. ‘This completes he proof, 0 ‘One more remark is of interest here. If N is invariant, under a Tinear map Tin Hom V (that is, TIN] CN), then for each a in V, Ta] is a subset of the coset Ta), for Ta] = Ta + N] = Pla) +s TIN] C Ta) +. N = Te.1 AFFINE SUBSPACES AND QUOTIENT SPACES There is therefore a map S:V/N— V/N defined by the requirement that, S(@) = Tia) (or Ser = wo 1), and itis easy to check that S is linear. ‘There- fore, ‘Theorem 4.4. Ifa subspace N of a vector space V is carried into itself by a transformation 7 in Hom V, then there is a unique transformation S in Hom(V/N) such that Sem = we T. EXERCISES 4.1 Prove properties (4), (5), and (6) of affine subspaces 4.2. Choose an origin 0 in the Euclidean plane E? (your sheet of paper), and let I and Lz be two parallel lines not containing 0. Let X and ¥ be distinet points on and Zany point on La. Draw the figure giving the geometric sums OX+0% and OF +02 (parallelogram rule), and state the theorem from plane geometry that says that these ‘ovo sum points are on a third line Lg parallel to Ly and La, 4.3. 2) Prove the associative law for additi )_ Prove also laws Ad and S2 44 Return now to Exercise 2.1 and reexamine the situation in the light of Theorem 4.1. Show, finally, how we really know that the geometrie vectors form a vector space. 4.3. Prove that the mapping S of Theorem 4.3 is injective if and only if N is the null space of 7. 4.6 We know from Exercise 4.5 that if Tis a surjective element of Hom(V, W) and {Vis the null space of 7, then the S of Theorem 4.3 is an isomorphism from V/N to W'. Its inverse S~! assigns a coset of N to each in I. Show that the process of “indefinite integration” is an example of such a map S~!, ‘This is the process of calculating an integral and adding an arbitrary constant, as in [rinzds = o2-b0 yn for Theorem 4.1 4.7 Suppose that NV and M are subspaces of a veetor space Vand that NC M. Show that then 3{/W isa subspace of V/N and that V/.M is naturally isomorphic to the ‘quotient space (V/N)/(M/N). [2int: Every eoset of Nis a subset of some coset of MJ 48 Suppose that N and M are any subspaces of a vector space V. Prove that (M-+ N)/N is naturally isomorphic to M/(M AN). (Start with the fact that each coset of A 7. is included in a unique coset of N.) 4.9 Prove that the map S of Theorem 4.4 is linear. 4.10, Given 7 € Hom V, show that 72 = 0 (7? = To 7) ifand only if R(T) CN(T). 4.11 Suppose that 7’ Hom V and the subspace N are such that TT is the identity ‘on N and also on V/N. ‘The latter assumption is that the S of Theorem 4.4 is the identity on V/N. Set R = T — 1, and use the above exercise to show that R® = 0. Show that if 7 = I+ R and R? = 0, then there is a subspace N’ such that 7'is the identity on N and also on V/N.56 vEcTor sraces 18 4.12 We now view the above situation a little differently, Supposing that 7 is the identity on Nand on V/X, and setting R= I— 7, show that tere exists a K€ Hom(¥/N, ¥) such that R = Ke x. Show that for any coset 4 of X’ the action ‘of Ton A can be viewed as translation through K(A). Thatis if 8 A and y = K(.1), then T() = E+ 413 Consider the map T: <2 + 2r2, 22> in Hom B?, and let Whe the null space of R = T— I. Identify N and show that 1 is the identity on Vand ‘on R2/N. Find the map K of the above exercise, Such a mapping 7 iscalled a shear transformation of V’ parallel to V. Draw the unit square and its image under 7. 4.16 If we remember that the linear span 1.(.1) of a subset .1 of a vector space V ean bee defined as the intersection of all the subspaces of V that include .t, then the fact, that the intersection of any collection of affine subspaces of a vector space V is either an affine subspace or empty suggests that we define the affine span M(.1) of a nonempty subset 1 Vas the intersection of all afine subspaces including «1. ‘Then we know from (8) in our list of affine properties that M(.1) is an affine subspace, and by its definition above that itis the smallest affine subspace including A. We now naturally wonder whether AM(.1) ean be directly deseribed in terms of Hinear combinations, Show first that if a al, then M(A) = L(A —a) ++ a; then prove tha: M1) is the set of all linear combinations 5 ray on -{ such that Sox; = 1 4.15. Show that the linear span of a set B is the affine span of BU {0} 4.16 Show that M(L-++7) = MCU) -+7 for any Yin V and that M(zd) = 23f(A) for any in R. 5. DIRECT SUMS We come now to the heart of the chapter. It frequently happens that the study ‘of some phenomenon on a vector space V leads to a finite collection of subspaces {V2} such that V is naturally isomorphic to the product space []; Ve Under this isomorphism the maps 6;¢ m; on the product space become certain maps Pin Hom V, and the projection-injection identities are reflected in she identities, LP. = 1, Pye Pj = P; forall j, and Pro P; = Oif i x j. Also, Ve = range P,, ‘The product structure that V thus acquires is then used to study the phe nomenon that gave rise to it, For example, this is the way that we unravel the structure of a linear transformation in Hom V, the study of which is one of the central problems in lincar algebra. Direct sums. If V,.-.,Vq are subspaces of the veotor space V, then the mapping: + Sj ay is a linear transformation from [Tj Vs to V, since it is the sum 7 = Sf, of the coordinate projections Definition. We shall say that the V,'s are independent if mis injective and that V is the direet sum of the V,’s if is an isomorphism. We express the latter relationship by writing V = V1 ® +++ @ Va = @i Vie ‘Thus V = @fuy Vs if and only if is injective and surjective, ic., if and only if the subspaces {V7} are both independent and span ’. A useful restate-15 pinecr sums 57 ment of the direct sum condition is that each @ & V is uniquely expressible as a sum Sf ai, with a € Vi for all 7; « has some such expression because the V's span V, and the expression is unique by their independeneo. For example, let V = @(R) be the space of real-valued continuous functions on R, let V be the subset of even functions (funetions f sueh that f(—z) = fle) {or all), and let V, be the subset of odd funetions (functions sueh that f(—2) =f(2) for all 2). Tt'is clear that V_ and V, are subspaces of V, and we elaim that V= Ve @ Vy. To see this, note that for any fin V, g(e) = (f(a) +.f(—2))/2 is even, h(x) = (f(2) — f(—2))/2 is odd, and j= gh. Thus V = Ve-+ Vou Moreover, this decomposition of f is unique, for if f= 1 ++ hy also, where gy is even and 2, is odd, then g — gi = ly — h, and therefore g — hy — h, since the only funetion that is both even and odd is zero. The even-odd components of e* are the hyperbolie cosine and sine funetions: Since * is injective if and only if its null spaee is {0} (Lemma 1.1), we have: Lemma 5.1, ‘The independence of the subspaces {V/))7 is equivalent to the property that if ay = V; for all i and Df a; = 0, then a= 0 for all i Corollary. If the subspaces {V}7 are independent, a; € V; for all é, and Ef ay isan clement of V5, then a; = 0 for i x j. We leave the proof to the reader. The ease of two subspaces is particularly simple. Lemma 5.2. The subspaces M and N of V are independent if and only if MAN = 0}. Proof. Wae M,8eN,anda+=0,thena=—seM ON. MAN = {0}, this will further imply that a= = 0, so M and N are independent, On ‘the other hand, if 0 # 8 € MAN, and if we set a= —8, then ae BEN, and a +8 = 0, 0 M and N are not independent, 0 Note that the first argument above is simply the general form of the unique~ ness argument. we gave earlier for the even-odd decomposition of @ function on, Corollary. V = M @ N if and only if V = M +N and MN = 00} Definition. If V = M @ N, then Af and N are called complementary sub- spaces, and each is a complement of the other. Warning: A subspace M of V does not have a unique complementary subspace unless M is trivial (that is, Mf = {0} or M = V). If we view Rt? as coordinatized welidean 3-space, then I is a proper subspace if and only if M is a plane con- {aining the origin or M isa line through the origin (see Fig. 1.9). If Mand N are58 vecTor spaces 15 EY AG. manor a gobs Fig. 19 proper subspaces one of which is a plane and the other a line not lying in that plane, then Mf and N are complementary subspaces. Moreover, these are the only nontrivial complementary pairs in R®. The reader will be asked to prove some of these facts in the exereises and they all will be clear by the middle of the next chapter. ‘The following lemma is technically useful. Lemma 5.3. If V; and Vo are independent subspaces of V and {V3} are independent subspaces of Vo, then {Vj} are independent subspaces of V. Proof. It aj € Vi for all é and S73 ay = 0, then, setting ay = 53 ai, we have a1 + a9 = 0, with ag € Vo. Therefore, a1 = ay = 0 by the independence of Vi and Vo. But then ay = ay=-:+— ay =0 by the independence of {V,}3, and we are done (Lemma 5.1). 0 Corollary. V= V1 @ Vo and Vo = @t Vi together imply that v= Oe Vi Projections. If V = @lai Vs, if m is the isomorphism @ = Li ay and if 1; is the jth projection map ++ aj from Vi to V3, then (x; 0F)(a) = ay Definition, We call aj the jth component of a, and we eall the linear map Pj = 150-4" the projection of V onto V; (with respeet to the given direct, sum decomposition of V). Since each a in V is uniquely expressible asa sum «= Cia, with a in Vi for all i, we ean view Py(a) = a; as “the part of ain Vj". ‘This use of the word “projection” is different from its use in the Cartesian product situation, and each is different from its use in the quotient space con- text (ection 0.12) It is apparent that these three uses are related, and the ambiguity causes little confusion since the proper meaning is always clear from the context. ‘Theorem 5.1. If the maps Pare the above projections, then range P, Pro P; = Ofori x j,and Pi = I. Proof. Since x is an isomorphism and Pj = x; x-!, we have range Pj = range 7; = Vj. Next, it follows directly from the corollary to Lemma 5.1 that Va15 pinecr sems 59 if we V;, then Pi(a) = 0 for i # j, and so Pye Py = 0 for ij. Finally, LPP Eimer! = (Lj x) ow! = rex! = Land weare done, 0 jon properties are clearly the reflection in V of the pro jection identities for the isomorphie space IT} Vi. A converse theorem is also true. ‘Theorem 5.2. If {P)}}C Hom V satisfy D}Pi= I and Pio Pj i #j, and if we set V;= range Pi, then V = @lai Vs, and P; corresponding projection on Vz Proof. The equation a = I(a) = E} Pi(a) shows that the subspaces {Vi} span V. Next, if 6 V,, then P,(8) = 0 for i # j, since @ € range P; and for the ProPy=0 if £4 J. Then also Pi(@) = (I — Dogs PIB) = 106) = 8. Now consider a = 57 a; for any choice of a; € Vs. Using the above two facts, we have Pj(a) = Pj(Xfay a4) = ley Py(aq) = a3. Therefore, a = 0 implies that a;= PQ) = 0 for all j, and the subspaces V; are independent. Consequently, V = @j Vs. Finally, the fact that @ = ¥ Py(a) and Pi(a) € Vi for ll shows that P,(a) is the jth component of a for every a and therefore that, P; is the projection of V onto V;. 0 ‘There is an intrinsic characterization of the kind of map that is a projection. Lemma 5.4. The projections P; are idempotent (P? cach is the identity on its range. ‘The null space of P: Vj for j x & Proof. P? = P;0 (I — Eigs Pi) = Pj I = Pj. Since this ean be rewritten, as P,(P,(a)) = Pi(a) for every a in V, it says exactly that P, is the identity P), or, equivalently, the sum of the spaces on its range. Now set Ws= Digi Vj and note that if 6 € Ws, then Py(a) = 0 since PAV3) = 0 forj # i. ‘Thus W;c N(P). Conversely, if Pi(a) = 0, then Ma) = D4 P3(a) = Cie: Pile) € We. Thus N(P)) C We, and the two spaces are equal. 0) Conversely: Lemma 5.5. If P € Hom(V) is idempotent, then V is the direct sum of its range and null space, and P is the corresponding projection on its range. Proof. Setting Q= I — P, we have PQ = P — P? = 0. Therefore, V is the direct sum of the ranges of P and Q, and P is the corresponding projection on its range, by the above theorem, Moreover, the range of @ is the null space of P, by the corollary. 0 If V = M @ N and P is the corresponding projection on M, we call P the projection on M along N. The projection P is not determined by M alone, since ‘M does not determine N. A pair P and Q in Hom V such that P+ @Q = I and PQ = QP = 0 is called a pair of complementary projections.60 vecton spaces 15 In the above discussion we have noglected another fine point, Strictly speaking, when we form the sum = 5} xi, we are treating each m; as though it were from [Jj V; to V, whereas actually the codomain of w; is V;, And we want P, to be from V to V, wherea has codomain V,, so the equation Pj = x; 0m can't quite be true either. To repair these flaws we have to introduce the injection ¢;: Vj + V, which is the identity map on Vs, but which views V'j as a subspace of V and so takes V as its codomain. If our concept of a mapping includes a eodomain possibly larger than the range, then we have to admit such identity injections, ‘Then, setting #; = 1 © mj, we have the correct, equations w= S] Rand Pj = #0 x EXERCISES 5.1 Prove the corollary to Lemma 8. 5.2 Let a be the vector <1,1, 1 in RS, and let M = Ret be its one-timensional span. Show that each of the three coordinate planes is a complement of IM. 5.3. Show that a finite product space V’ = [Tt Vs has subspaces {W;}? such that We is isomorphic to V; and Y= @f W; Show how the corresponding projections {P2 are related to the x's and 0s 34 If Te Hom(V, W), show that (the graph of) T is a complement of W! = (0) x Win x W. 5.3. If is a linear functional on V (= Hom(V,R) = ¥*), and if ais a veetor in V such that (a) 0, show that I’ = NV @ 3, where Nis the null space of Cand Af = Ra is the linear span of a. What does this result say about complements in B®? 5.6 Show that any complement A of a subspace N’ of a vector space V is isomorphic to the quotient space V/¥, 5.7 We suppose again that every subspace has a complement, Show that if 1 Hom V is nol injective, then there is a nonzero S in Hom V such that Te 8 = 0, Show that if 7" Hom V is not surjective, then there is a nonzero $ in Hom V such that So T = 0, 5.8 Using the above exercise for half the arguments, show that TE Hom Vis injective if and only if T'° S = O=> 8 = Oand that T'is surjective if and only if So T 0= 8 = 0. We thus have characterizations of injectivity and surjectivity that are formal, in the sense that they do not refer to the fact that S and T'are transformations, ‘but refer only to the algebraic properties of S and 7'as elements of an algebra. 9 Let Mf and N be complementary subspaces of a vector space V, and let X be a subspace such that XN = {0}. Show that there is a linear injection from X to M. int: Consider the projection P of V onto M along N.] Show that any two comple- ments of a subspace V are isomorphic by showing that the above injection is surjective if and only if is a complement of N. 5.10 Going back to the first point of the preceding exercise let Y be a complement of P{X]in M. Show that XA ¥ = {0} and that X @ Y is a complement of \V. SAIL Let M be a proper subspace of ¥, and let fai:i I} be a finite set in V. Set L = L({as}), and suppose that If-+ L = V. Show that there is a subset J CF such1s pinecr sts OL that {ai:iE J} spans a complement of I [ints Consider a largest possible subset J such that MAL (au}s) = {0)-] BAZ Given PE Hom(T, W) and SE Hom(IF, X), show that a) So Tis surjective © 8 is surjective and R(T) + N(S) = Wi b) So Tis injective + Tis injective and R(T) AN(S) = {0}; ©) So Tisan isomorphisn © Sissurjective, Tis injective, and IF = R(T) ® N(S). 5.13 Assuming that every subspace of 1" has a complement, show that 7’ Hom V satisfies 72 = Oif and only if V has a direet sum decomposition V = AF @ N such that T= OonN and TIMCN. 5.14 Suppose nest that 7% = 0 but 7? x 0. Show that V ean be written as V ¥1@ VO Ve, where PUG] V2, Talc Vay and P= O on Mx. (Assume again that any subspace of a vector space has a complement.) 5.15 We now suppose that T* = but Tet x 0. Sct Ny — null space (7) for jn — 1, and let Vy be a complement of Net in V. Show first that TIVINN a2 = (0) and that 701 C Naa. Estend TIVs1 to a complement V9 of ¥ show that in this way we ean construct subspaces V,..-, Vx teh that 1, and ve@r, MWC for i V such that ToS = Iw, the identity on W. Thus a solution process picks one solution veetor £ € V for each » & W in such a way that the solving £ varies linearly with a. Taking this as our meaning of solving, we have the following fundamental reformulatio ‘Theorem 5.3. Tot T he a surjective linear map from the vector space V to the vector space W, and let N be its null space. ‘Then a subspace M is a ‘complement of NV if and only if the restriction of 7 to M is an isomorphism from M to W. ‘The mapping Mr (7 } M)~" isa bijection from the set of all such complementary subspaces M to the set of all linear right inverses of f.62 vector spaces 15 Proof. Tt should be clear that @ subspace A is the range ofa linear right inverse of T (a map Ssuch that T'© $= Tw) ifand only if 7 [ isan isomorphism to W, inwhich case S = (7 | M)~!. Strictly speaking, the right inverse must be from W to V and therefore must be R = ur S, where uy is the identity injection from M to V. Then (Re 7)? = Ro(ToR)oT =RelyeT = Re T, and R © T is a projection whose range is M and whose null space is N (since R is, injective). Thus V = M@ N. Conversely, if V = M @ N, then 7 [ Af is injective because MN = {0} and surjective because M-+'N that W = 21] = T[M +N] = TIM)-+ TIN] = TUM) + (0) Polynomials in T. ‘The material in this subsection will be used in our study of differential equations with constant coefficients and in the proof of the diagonal- izability of a symmetric matrix. In linear algebra it is basie in almost any approach to the eanonical forms of matrices. Tips) = Chait and p2(d) = E4 0,0 are any two polynomials, then their product is the polynomial 10 = ropa =F et where cy = Eiyinn ads = Cho adj. Now let 7 be any fixed element of Hom(V), and for any polynomial q(t) let q(T) be the transformation obtained by replacing t by T. That is, if g() = Soe, then (7) = Eb cal, where, of course, 7" is the composition product T'e T'° +++ T with | factors. Then the bilinearity of composition (Theorem 3.3) shows that if p(t) = pi(é)pa(t), then p(T) = p,(T) p2(7).. In particular, any two polynomials in 7 commute with each other under composition. More simply, the commutative law for addition implies that if pO = pil + pal, — then p(T) = pil) + pal. ‘The mapping p(t) ++ p(T) from the algebra of polynomials to the algebra ‘Hom(V) thus preserves addition, multiplication, and (obviously) scalar multipli- cation. That is, it preserves all the operations of an algebra and is therefore what is called an (algebra) homomorphism. ‘The word “homomorphism” is a general term describing a mapping @ between two algebraic systems of the same kind such that @ preserves the operations of the system. Thus a homomorphism between vector spaces is simply @ linear transformation, and a homomorphism between groups is a mapping preserving the one group operation. An accessible, but not really typical, example of the latter is the logarithm funetion, which isa homomorphism from the multiplicative group of positive real numbers to the additive group of R. ‘The logarithm function is actually a bijective homomorphism and is therefore 4 group isomorphism. If this were a course in algebra, we would show that the division algorithm ‘and the properties of the degree of a polynomial imply the following theorem, (However, seo Exercises 5.16 through 5.20.)15 pingcr sums 63 ‘Theorem 5.4. If p,(t) and pa(t) are relatively prime polynomials, then there cexist polynomials a,() and aa() such that ax()pr) + ar)p2) By rolatively prime we mean having no common factors except constants. ‘We shall assume this theorem and the results of the discussion preceding it in proving our next theorem. ‘We say that a subspace M C V is invariant under 1’ Hom(V) if 7(M].CM. [that is, 7 fat € Hom(aN). ‘Theorem 5.5, Let 7 be any transformation in Hom V, and let q be any polynomial. ‘Then the null space N of q(7) is invariant under 7, and if 4 = quia is any factorization of q into relatively prime factors and Ny and Ne are the null space of q(T) and ga(7), respectively, then N= Ny @ Nz Proof. Since T + q(T) = 9(T) » T, we see that if (T)(a) = 0, then g(T)(Ta) = T(q(T)(a)) = 0, 80 TINJCN. Note also that since q(T) = 91(1) ° ga(T), it follows that any a in Nis also in N, so Nz CN. Similarly, Ny CN. We ean therefore replace V by N and T by T {'N;hence we ean assume that 7’€ Hom N and q(T) = qu(T) © g2(T) = 0. ‘Now choose polynomials a; and a, s0 that aig: ++ dag2 = 1. Since p+ p(T) is an algebraic homomorphism, we then have a,(P) 2 qi(7) + a3(T) © 93(T) = T. Set Ay = a,(7), ete., so that Ay °Q1 + Az °Qz = 1,Q1 °Q2 = 0, and alll the operators Ay, Q; commute with each other. Finally, set Ps = Ase Qi = Qie As for i= 1,2. Then Py + Pp = Tand P;P; = PsP, = 0. Thus P; and P, are projections, and NV is the direct sum of their ranges: V = V; ® Vo. Since each range is the mull space of the other projeetion, we can rewrite this as N Ny @ Nz, where N; = N(P). It remains for us to show that N(P,) = N(@). Note first that since Qi °P2 = Qi 2Qz° Az = 0, we have Qi = Quel Q12 (P+ Ps) =Q1eP;. Then the two identities Py= Ase Q; and Q Q;¢ P;show that the null space of each of P; and Q; is included in the other, and so they are equal. ‘This completes the proof of the theorem. 0 Corollary. Let p(t) = ITM; pi(t) be a factorization of the polynomial () into relatively prime factors, let T be an element of Hom(V), and set Ny = N(p(1)) for i= 1,..., mand N = N(p(1)). Then N and all the ‘Weare invariant under 7, and NV ™ Proof. ‘The proof is by induetion on m. ‘The theorem is the ease m = 2, and if wo ect g—= TIT p(t) and M-—=N(q(7)), then the theorem implice that N=, @ Mand that N; and Mf are invariant under 7. Restricting T to M, see that the inductive hypothesis implies that Mf = Qf. N, and that Nj is invariant under T for i= 2,...,m. The corollary to Lemma 5.3 then yields our result.GL vector spaces 15 EXERCISES 5.16 Presumably the reader knows (or ean see) that the degree d(P) of a polynomial P satisfies the laws AUP + Q) < mas (AP), dQ}, AP -Q) = a(P) + dQ) if both P and Q are nonzero. ‘The degree of the zero polynomial is undefined. (Lt would have to be —= ) By indue- tion on the degree of P, prove that for any two polynomials P and D, with D # 0, there are polynomials @ and & such that P = DQ+ R and d(R) < d(D) or R = 0. [Win Te d(P) < d(D), we ean takeQ and Ras what? Ifd(P) > d(D), and if the lead ing terms of P and Dare ar* and br, respectively, with n > m, then the polynomial rrQe hhas degree less than d(P), so P” = DQ’ + R by the inductive hypothesis, Now finish the proof.) 5.17 Assuming the above result, prove that R and Q are uniquely determined by Pand D. (Assume also that P = DQ’+ R’, and prove from the properties of degree that 2” = Rand Q = Q.) These two results together constitute the division algoritho. for polynomials, 5.18 If P is any polynomial PQ) = Yaa", and if € is any number, then of course P(e) i the number Dat. Prove from the division algorithm that for any polynomial P and any number ¢ there isa polynomial Q such that PQ) = © — 09) + PO, and therefore that P(z) is divisible by (r — 0) if and only if P(@ = 0. 5.19 Let P and Q be nonsero polynomials, and choose polynomials .19 and Bo such that among all the polynomials of the form 1P ++ BQ the polynomial D = AoP + BoQ is nonzero and has minimum degree. Prove that D is a factor of both P and Q. (Sup- pose that D does not divide P and apply the division algorithm to get a contradietion with the choice of -lo and Bo.) 5.20, Let P au @ be nonzero relatively prime polywomials, This meas Unat if # ise common factor of P and Q(P = EP',Q = EQ’), then E is a constant. Prove that ‘there are polynomials A and B such’ that A(z)P(@)-+ B@)Q@) = 1. (Apply the above exercise.)13 pineer sums 65 5.21 In the context of Theorem 5.5, show that the restrietion of q2(7) = Q2 to isan isomorphism (from Ns to X1) 3.22 An involution on Vis a mapping 7. Hom V such that 72 = I. Show that if Tis an involution, then ¥ is a digeet sum V = V1 @ V2, where T(Q) = & for every EE V1 (T= Lon V1) and Tg) = —£ for every £E V2 (T = —Fon Va). (Apply. ‘Theorem 9.5.) 5.28 We noticed earlier (in an exercise) that if y is any mapping from a set -L Lo a set B, then f+ fo g isa linear map T, from B® to R4. Show now that if ¥: BC, then The = Te° Ty. (his should turn out to be a direct consequence of the associativity of composition.) 52h Let A be any set, and let g:1— 4 be such that go gla) = a for every a ‘Then Ty: f+ fog is an involution on V = R& (since Troy = Tye T.). Show that the decomposition of R* as the direct sum of the subspace of even functions and the subspace of odd funetions arises from an involution on R® defined by such a map RoR. 5.25 Let V be a subspace of BM consisting of differentiable functions, and suppose that V is invariant under differentiation (FE V=> Df V). Suppose also that on V the linear operator DE Hom V satisfies D? — 2D — 31 = 0. Prove that V’ is the direct sum of two subspaces Mf and A such that D = 32 on M and D = —T on YX. Actually, it follows that 1 is the linear span of a single vector, and similarly for Find these two functions, if you ean. (f/ = 3f—> f = 2) “Mock decompositions of linear maps. Given 7’ in Hom V and a direct sum decomposition V = @} Vs, with corresponding projections {P,}4, we can consider the maps Ty = Pie To P;, Although Tis from V to V, we may also ‘want to consider it as being from V; to V; (in whieh ease, strietly speaking, what is it?). We picture the 17's arranged schematically in a rectangular array ‘Similar to a matrix, as indieated below for Furthermore, sinee 7 = Ej Tis, we eall the doubly indexed family the block decomposition of 7 associated with the given direct sum decomposition of V More generally, if 7 = Hom(V, W) and W also has a direct sum deeomposi tion W = @P, Wi, with corresponding projections {Q;}7, then the family {73} defined by yj = Qce T° P; and pictured as an m X n rectangular array is the block decomposition of 7 with respect to the two direet sum decompositions. ‘Whenever 7'in Hom V has a special relationship to a particular direet sum decomposition of V, the corresponding block diagram may have features that, display these special properties in a vivid way; this then helps us to understand the nature of 7’ better and to calculate with it more easily.66 _yeeror spaces 15 Forexample, if V = V's ® Vs, then Vs isinvatiant under 7 (ie. 71Vi1C V1) if and only if the block diagram is upper triangular, as shown in the following diagram. Ti | Te 0 | Tox Suppose, next, that T? = 0. Letting V1 be the range of 7, and supposing that V has a complement V's, the reader should clearly see that the corresponding block diagram is | Tia fae ‘This form is ealled strictly upper triangular; it is upper triangular and also zero on the main diagonal. Conversely, if 7 has some strietly upper-triangular 2X 2 block diagram, then 7? = 0. If Bisa composition product, R = ST, then its block eomponents ean be computed in terms of those of Sand 7. ‘Thus Ram PARP, = PST, = PS (EP) Tr = E Syn We have used the identities = Tj, Pj and Py =P}. The 2x 2 ease is pictured below. SuTir + Sioa | 8) Sra oa Sais + SaoTar | SoiTr2 + SaaT oa From this we can read off a fact that will be useful to us later: If 7’ is 2x 2 upper triangular (21 = 0), and if Ti is invertible as a map from V; to Vs (= 1,2), then T is invertible and its inverse is Tat | Pata of te We find this solution by simply setting the produet diagram equal to nlo oln and solving; but of course with the diagram in hand it ean bbe eorreet. imply be checked to EXERCISES 5.26 Show that if Te Hom V, if V = @E Vs, and if (2) are the corresponding projections, then the sum of the transformation Ty; = Po Pe Py is 7.16 puuveariry 67 5.27 If 8 and 7 are in Hom V and {Sy}, {7} are their block components with respect to some direct sum decomposition of V, show that Sye Tw = Oi # l 5.28 Verify that if T has an upper-triangular block diagram with respect to the direct sum decomposition V = V1 @ Va, then Vs is invariant under 7. 45.29 Verify that ifthe diagram is strictly upper triangular, then 7? = 0. 5.30 Show that if V = Vy @ V2 @ Vy and 7'= Hom Y, then the subspaces Vy are all invariant under 7 if and only if the block diagram for Tis Mm 0 0 0 Tm 0 0 0 Ms, Show that 7 is invertible if and only if Ti is invertible (as an element of Hom V9) for each i. 5.81 Supposing that has an upper-triangular 2X 2 block diagram and that Tis is invertible as an element of Hom V; for ¢ = 1, 2, verify that T is invertible by form= ing the 2X 2 block diagram that is the product of the diagram for T and the diagram sgiven in the text as the inverse of 7, = 71 must have Tand SoT = 1 5.82 Supposing that Tis as in the preceding exercise, show that the given block diagram by considering the two equations Te in theie block form. 5.83 What would strictly upper triangular mean for a 3X 3 block diagram’? What is the corresponding property of 7? Show that 7' has this property if and only if it has strictly upper-triangular block diagram. (See Exercise 5.14.) 5.84 Suppose that 7 in Hom V satisfies 7" = 0 (but T*! +4 0). Show that 7 has fa strictly upper-triangular n Xn block decomposition. (Apply Exercise 5.15.) 6, BILINEARITY Bilinear mappings. ‘The notion of a bilinear mapping is important to the un derstanding of linear algebra because it is the veetor setting for the duality prineiple (Section 0.10). Definition. If U,V, and W are veetor spaces, then a mapping. : <> H+ WE, 9) from UX V to W is bilinear if it is linear in each variable when the other variable is held fixed. That is, if we hold € fixed, then ++ «(E, ») is linear [and so belongs to Hom(V, W)]; if we hold fixed, then similarly w(&, 9) is in Hom(U, W) as a function of £. ‘This is not the same notion as linearity on the product veetor space Ux V. For example, <2, y> — 2+y is a linear mapping from R to R, but it is not bilinear. If y is held fixed, then the mapping 2+ 2+ y is affine (translation through y), but it is not linear unless y is 0. On the other hand, <2, y> + 2y is bilinear mapping from R? to R, but it isnot linear. If y68 vector sracrs 16 is held fixed, then the mapping x ye is linear. But the sum of two ordered couples does not map to the sum of their images: + = , is thus from V to Hom(U, W), and ite linearity is due to the linearity of ein’ when & is held fixed (Een + at) = cw, n) + de £) = cay() + deg), weypar() so that so that Gendt = Cody + dary Similarly, if we define at by a(n) = 0, 9), then & at isa linear mapping from U to Hom(V, W). Conversely, if ¢: U + Hom(V, 1) is linear, then the funetion w defined by w(&, 9) = ¢(8(a) is bilinear. Moreover, at = o(2), 80 that ¢ is the mapping fw. 0 We shall see that bilinearity oceurs frequently. Sometimes the reinterpreta- tion provided by the above theorem provides new insights; at other times it seems less helpful. For example, the composition map <8, 17> Se 7 is bilinear, and the corollary of Theorem 3.3, which in effect states that composition on the right by a fixed T isa linear map, is simply part of an explicit statement of the bilinearity.. But the linear map 1’ > composition by is a complicated object that we have no need for except in the ease W = &. On the other hand, the linear combination formula 2} z,a;and Theorem 1.2 do receive new illumination. ‘Theorem 6.2. ‘The mapping w(x, #) = Df xia, is bilinear from R" x V* to V. The mapping a+ a is therefore @ lincar mapping from V™ to Hom(R", V), and, in fact, is an isomorphism, Proof. The linearity of w in x for a fixed x was proved in Theorem 1.2, and its Tincarity in « for a fixed s is seen in the same way. ‘Then « + wa is linear by ‘Theorem 6.1. Its bijeetivity was implicit in Theorem 1.2. 0 It should be remarked that we can use any finite index set J just as well as the special set 7 and conclude that w(x, a) = Lier «ass bilinear from @! x V!16 pouNeariry 69 to V and that a++ wa is an isomorphism from V! to Hom(R!, V). Also note that aa = La in the terminology of Seetion 1. Corollary. ‘The sealar product (x, a) = Einav is to R; therefore, a+ @ = Ly is an isomorphism from R" to Hom(R, R) Natural isomorphisms. We often find two veetor spaces related to each other in such a way that a particular isomorphism between then is singled out, ‘This phenomenon is hard to pin down in general terms but easy to deseribe by examples. Duality is one souree of such “natural” isomorphisms. For example, an mx n matrix {{;3} is a real-valued funetion of the two variables , and. as such it is an element of the Cartesian space R™**, We ean also view {l,)} as 1a sequence of n column vectors in R". This is the dual point of view where we hold j fixed and obtain a funetion of ¢ for each j.. From this point of view {ty} isan element of (R)*. This correspondence between R™** and (RF) is clearly an isomorphism, and is an example of a natural isomorphism. ‘We review next the various ways of looking at Cartesian n-space itself One standard way of defining an ordered n-tuplet is by induction, The ordered, triplet <2, y, 2> is defined as the ordered pair < , 2, and the ordered, netuplet <2)... 24> is defined as <<2y,..-, 2.1% 2_>- Thus we define R" inductively by setting R' = Rand R" = R™' x R, ‘The ordered n-tuplet can also be defined as the function on % = {1,..-,n} which assigns 2; to i. ‘Then Kai eptn® = (Kary for all EV. (This is duality again! (2) isa function of the two variables Zand &2) And it is not hard to see that this identification of 7 with {771% is an isomorphism from Tl; Hom(V, W,) to Hom(V, TI: 7). Similarly, Theorem 3.7 identifies an n-tuple of linear maps {71} into a com- mon eodomain V with a single linear map T of an n-tuple variable, and this id tification is a natural isomorphism from IT} Hom(V;, ¥) to Hom(IT} Wi, V).70 yeeton spaces 16 An arbitrary isomorphism between two veetor spaces identifies them in a transient way. For the moment we think of the veetor spaces as representing the same abstraet space, but only so long as the isomorphism is before us. If we shift to a different isomorphism between them, we obtain a new temporary identification. Natural isomorphisms, on the other hand, effect permanent identifications, and we think of paired objects as being two aspeets of the same ‘object in a deeper sense, ‘Thus we think of a matrix as “being” either a sequence of row veetors, a sequence of column veetors, or a single funetion of two integer indices. We shall take a final look at this question at the end of Seetion 3 in the next chapter. *We can now make the ultimate dissection of the theorems centering around the linear combination formula, Laws S1 through $3 state exactly that the scalar product za is bilinear. More precisely, they state that the mapping S: <2, a> +> ra from 8X W to W isbilinear. In the language of Theorem 6.1, xa = aq(z), and from that theorem we conclude that the mapping a wy is an isomorphism from W to Hom(R, W). ‘This isomorphism between W and Hom(R, W) extends to an isomor- phism from 1" to (Hom(R, W))*, which in turn is naturally isomorphic to Hom(R",WV) by the second Cartesian product isomorphism. ‘Thus W" is natu- rally isomorphic to Hom(R", WW); the mapping is a+ La, where La(x) = Cina, In particular, R" is naturally isomorphic to the space Hom(R®, R) of all linear functionals on R*, the x-tuple a corresponding to the functional ey defined by &4(x) = Ei asx ‘Also, (R")" is naturally isomorphic to Hom(R", R"). And sinee BP** is naturally isomorphie to (R")*, it follows that the spaces R™™™ and Hom(R”, R") are naturally isomorphic. This is simply our natural association of a transfor~ mation Tin Hom(R", 8") to an m Xn matrix {tj}CHAPTER 2 FINITE-DIMENSIONAL VECTOR SPACES We have defined a veetor space to be finite-dimensional if it has a finite spanning set. In this chapter we shall focus our attention on such spaces, although this restriction is unnecessary for some of our discussion. We shall see that. we ean assign to each finite-dimensional space V a unique integer, called the dimension of V, which satisfies our intuitive requirements about dimensionality and whieh becomes a principal tool in the deeper explorations into the nature of such spaces. A number of “dimensional identities” are crucial in these further investigations. We shall find that the dual space of all linear funetionals on V, V* = Hom(V, R), plays a more satisfactory role in finite-dimensional theory than in the context of general vector spaces. (However, we shall see later in the book that when we add limit theory to our algebra, there are certain special infinite-dimensional vector spaces for which the dual space plays an equally important role.) A finite-dimensional space ean be characterized as a vector space isomorphie to some Cartesian space R", and such an isomorphism allows a transformation 7 in Hom V to be “transferred” to R*, whereupon it acquires a matrix. The theory of linear transformations on such spaces is therefore mirrored completely by the theory of matrices. In this chapter we shall push much deeper into the nature of this relationship than we did in Chapter 1. We also include a seetion on matrix computations, a brief section describing the trace and determinant functions, and a short discussion of the diagonalization of a ‘quadratie form. 1. BASES. Consider again a fixed finite indexed set of veetors @ = {a:: i € 7} in V and the corresponding linear combination map La:x ++ Ea from R! to V having a 1s skeleton, Definition. ‘The finite indexed set {ai : i € Z} is independent if the above mapping Le is injective, and {a;} is a basis for V if Le is an isomorphism (onto V). Tn this situation wo call {ay: #€ 7} an ovtored basis or feame if T= 1= {1,...,n} for some positive intoger n. ‘Thus {as :7€ 1} isa basis if and only if for each § € V there exists a unique indexed “coefficient” set x= {r;:7ET} ER! such that €= Taq; The 772 FINITE-DIMENSIONAL VECTOR SPACES 2 numbers 2; always exist because {as: # € T} spans V, and x is unique because La is injective. For example, we ean check direetly that b! = <2,1> and b? = <1, —3> form a basis for R?, ‘The problem is to show that for each y € R? there is a unique x such that, Since this vector equation is equivalent to the two scalar equations yy = 221 +22 and yz = 21 — Bra, we ean find the unique solution 21 = (8y1-+ y2)/7, 2 = (yi ~ 2y2)/7 by the usual climination method of secondary school algebra, ‘The form of these definitions is dietated by our interpretation of the linear combination formula as linear mapping. ‘The more usual definition of indepen- dence is a corollary, 1<2,1> bap <1, —8> = <2ey $42, t1 — Bram. Lemma 11. The independence of the finite indexed set fai: i¢T} is ‘equivalent to the property that Sy xia; = 0 only if all the coefficients x; are 0. Proof. This is the property that the mull space of Le consist only of 0, and is thus equivalent to the injectivity of Le, that is, to the independence of {a}, by Lemma 1.1 of Chapter 1. 0 If {ac} is an ordered basis (frame) for V, the unique n-tuple x such that § = Dj za: is called the coordinate n-tuple of & (with respect to the basis {ai}), and 2:38 the ith coordinate of g We call za; (and sometimes x,) the ith component of & ‘The mapping Le will be called a basis isomorphism, and its inverse La’, which assigns to each vector & € V its unique coordinate n-tuple x, is a evordinale ‘isomorphism. ‘The linear functional ¢+> x; is the jth coordinate functional; itis the composition of the coordinate isomorphism +> x with the jth eoordi- nate projection x++ 2; on R®. We shall see in Section 3 that the n eoordinate functionals form a basis for V* = Hom(V, R). In the above paragraph we took the index set J to be 7 = {1,..., 1) and used the language of n-tuples. ‘The only difference for an arbitrary finite index set is that we speak of a coordinate function x = {zi: i € J) instead of a coondi- nate n-tuple. Our first concer will be to show that every finite-dimensional (finitely spanned) vector space has a basis. We start with some remarks about indices. We note first that a finite indexed set. {ai # € I} ean be independent only if the indexing is injective as a mapping into V, for ifm = a, then Eom where 2 = 1, 21 = —1, and x; = O for the remaining indices. Also, if fas: 7 is independent and J C I, then {aii € J} is independent, sinee if Dy za: and if we set x; = 0 for ie 1 —J, then Dr2.ai = 0, and so each 2; is 0. AA finite unindezed set is said to be independent if it is independent24 mases 73. (necessarily bijective) indexing. It will of course then be independent with respect to any bijective indexing. Aw arbitrary set is independent if every finite subset is independent. It follows that a set A is dependent (not independent) if ‘and only if there exist distinet elements ay, ---,a in A and sealars 21)... 5 0 not all zero such that Ef xia; = 0. An unindexed basis would be defined in the obvious way, However, a set can always be regurided as being indexed, by itself if necessary! Lemma 12. If B isan independent subset of a veetor space Vand B is any ‘veetor not in the linear span £(B), then BU (3) is independent. Proof. Otherwise there is a zero linear combination, «8 C7 28; — 0, where Bis. -- Beate distinet cloments of B and the couffiients ate not all 0. But then x eannot be zero: if it were, the equation would eontradiet the independence of B. We can therefore divide by and solve for 8, s0 that # € L(B), a contra ion, 0 ‘The reader will remember that we call a yeetor space TV’ finitetimensional if it has a finite spanning set {a}, We can use the above lemma to construct a basis for such a V by choosing some of the a's, We simply run through the sequence {ai} and choose those members that inerease the linear span of the preceding choices. We end up with a spanning set sinee fai}? spans, and our subsequence is independent at each step, by the lemma, In the same way we cean extend an independent set {2,} to a basis by choosing some members of spanning set {a,}{. This procedure is intuitive, but it is messy to set up rigor- ously. We shall therefore proceed differently. ‘Theorem LL. Any minimal finite spanning set is a basis, and therefore any finite-dimensional veetor space V has a basis. More generally, if (3: €J} isa finite independent set and {ay : 7 € I} is finite spanning set, and if isa smallest subset of F such that {8)} y U {a;} x spans, then this collection is independent and 2 basis. ‘Therefore, any finite independent subset of a finite-dimensional space ean be extencled to a basis. Proof. It is sufficient to prove the second assertion, since it includes the frst asa special ease. If {8;} U fai} x is not independent, then there is a nontrivial ‘ero linear combination Sy 938; -+ Lx tas = 0. I every a; were zero, this ‘equation would contradict the independence of {3;} s. Therefore, some ay is sot zero, and we ean solve the equation for aj. That is, if we set L = K — (8), then the linear span of {8;} U fa:)1,contains a. Tt therefore includes the whole ‘original spanning set and hence is V. But this contradicts the minimal nature of K, since L isa propersubset of K. Consequently, {8;} U {a;} x is independent. 0 We next note that B" itself has a very special basis. Tn the indexing map fa; the vector ay corresponds to the index Jj, but under the linear combi- nation map x E ziai the veetor a, corresponds to the funetion 3? which has the value 1 at j and the value 0 elsewhere, so that Ey Bay = ay. ‘This function7A FINITE-DIMENSIONAL VECTOR SPACES 21 81 is called a Kronecker delta function. Tt is clearly the characteristic function Xw of the one-point set B = {j}, and the symbol ‘6” is ambiguons, just as ‘Xn is ambiguous; in each ease the meaning depends on what domain is implicit from the context, We have already used the delta funetions on &" in proving ‘Theorem. 1.2 of Chapter 1 ‘Theorem 1.2, ‘Ihe Kronecker functions {8°}5 Proof. Since If 2:8'(j) = x; by the definition of 44, we see that Di 2:6! is the n-tuple x itself, so the linear combination mapping Lg: x + Ej 26" is the identity mapping x + x, a trivial isomorphism. form a basis for R, Among all possible indexed bases for R", the Kronecker basis is thus singled out by the fact that its basis isomorphism is the identity; for this reason it is called the standard basis or the natural basis for 6". ‘The same holds for R! for any finite set I. Finally, we shall draw some elementary conclusions from the existence of a basis, ‘Theorem 1.3. If T € Hom(V, W) is an isomorphism and @ = {a;: i J} isa basis for V, then {T(ai) : i € I} is a basis for W. Proof. By hypothesis La is an isomorphism in Hom(R®, V), and so 1° La is an isomorphism in Hom(R*, W). Its skeleton {7'(a)} is thereforea basis for WV. 0 We can view any basis {a} as the image of the standard basis {6'} under the basis isomorphism. Conversely, any isomorphism 6: R! > B becomes a basis isomorphism for the basis aj = 6(6'). ‘Theorem 1.4. IfX and Y are complementary subspaces of a vector space V, then the union of a basis for X and a basis for Y isa basis for V. Conversely, if a basis for V is partitioned into two sets, with linear spans X and Y, respectively, then X and Y are complementary subspaces of V. Proof. We prove only the first statement, If ai: ¢ €J) is a basis for X and {ai: i © K} isa basis for Y, then it is clear that {ai :i€ J UK) spans V, since its span includes both X and Y, and so X + Y = V. Suppose then that Crue tia; = 0. Setting ¢ = Ly re and y= Cx ries, we see that eX, 7 Y,and &-+ = 0. Butthen & 0, sinee X and Y are complementary. And then 2; = 0 for i €J because {aj} is independent, and 2; = 0 for ie K Deeause {a;} x is independent. Therefore, {ai} sux isa basis for V. We leave the converse argument as an exercise. 0 Corollary. If V = @i Vs and Bs is a basis for Vi, then B= Uj By is a basis for V. Proof. Wersee from the theorem that B; U By isa basis for V1 ® V3. Proceed- ing inductively we seo that Uj, By is a basis for @jey Vi for j and the corollary is the ease j = ». 0