INTRODUCTION TO LINEAR ALGEBRA Fourth Edition MANUAL FOR INSTRUCTORS Gilbert Strang Massachusetts Institute of Technology math.mit.edufinearalgebra web. mit.edu/18.06 video lectures: ocw.mit.edu math. mit.edw/~gs www.wellesleycambridge.com email: gs@math.mit.edu ‘Wellesley - Cambridge Press Box 812060 Wellesley, Massachusetts 024822 Solos to Exec Problem Set {.1, page 8 1 The combinations give (a) aline in R? (b) aplaneinR? (c) all of R. 2 vt w= (2,3) and v—w = (6,~1) willbe the diagonals ofthe parallelogram with and w as two sides going out from (0,0). ‘8 This problem gives the diagonals v + and v ~ w ofthe parallelogram and asks for the sides: The opposite of Problem 2, In this example v = (3,3) and w = (2,2) 4304 w= (7,5) mdcv+dw = Qe +d.c+2d), 5 ut = (-2,3,1)andu-+o+w = (0,0,0) and 2u-+204+w = (add fist answers) = (2,3,1), ‘The vectors u,v, w are inthe same plane because a combination gives (0.0.0), Stated another way: = —o— wis in the plane of v and w, 6 The components of every co + daw add to zero, ¢ = 3 and d = 9 give (3.3.6). 7 The nine combinations ¢(2,1) + d(0, 1) with ¢ = 0, 1,2 and d = (0, 1,2) wil lic on . If we took all whole numbers c and d, the lutice would li over the whole plane, © The other diagonal is v — w (orelse w—v). Adding diagonals gives 2v (or 2). 9 The fourth comer can be (4,4) of (4,0) or (2,2). Three possible parallelograms! (1,1,0) isin he base Ce-y plane). f+ J + = (1.1) isthe opposite comer 0), Poin inthe cube hare 0. af of this line is removed, leaving a7 that stars at (0,0), 48 The combinations cv + dw with 0 se = 1 andO = d 1 fil he parallelogram with sides and w. For example, if© = (1,0) and w = (1) then eu + dw fils the unit square 19 Withe > Oand d > 0 we get the ifaite “cone” or “wedge” between v and w. For example, © = (1-0) and'w = (0,1), then the cone isthe whole guadant x > 0, Y= 0. Question: Wim iw = ~w? The cone opens to ahale-space +. Teva he cone feSoluions to Exercises 3 20 (0) Ju-+ Ju + $w in the comer ofthe uiangle between u,v and wi fu + Jw lies betwocnu and w (Toil ihe wianglekeepc0,d2>0,e20,endé+d+e 21 The sum is (v—u) + (w—v) + (uw) = zero veetor. Those thre sides of wiangle are inthe same plene! 22 The vecor }(u +0 +) is ouside the pyramid becausec+d+e= + }+$> 1 28 Allvectorsare combinations of u,v, w as drawn oot inthe same plane) Startby Seeing thatcu + do fils. plane then ading ew fils al of R? ‘24 The combinations of w and v fil one plane. The combinations of w and w fll another Plane, Those planes meet ina line: only the vectors ev are in bth panes, 25 (6) For ine, choose w = v = w = eny nonzero vector (6) For plane, choose wand vin different directions. A combination like © = u + vis inthe same plane. £26 Two equations come from the two components: ¢ + 3d solution isc = 2and d = 4. Then 2(1,2)+ 40,1) 28 There are 6 unknown numbers vy, v2, Uy, Wa, The six equations come from the ‘components of v + w = (4,5,6) and © — w = (2, 8,8). Add find 20 = (6,10, 14) sov = (3,5,7) and w = (1,0,—1). 29 Two combinations out of infinitely many that produce b 1w — Jv. No, thre vectors (0.1) are ~2u + 0 and ation produces b then ‘wo (and infinitely many) combinations will produce b. This is uue even if u = 0; the ‘combinations can have different cu. 30 The combinations of w and w fill the plane unless w and w lie on the same line through (0,0). Four vectors whose combinations fill 4-dimensional space: one example isthe “standard basis” (1,0,0, 0), (0, 1,0,0),(0,0, 1,0),and (0,0,0, 1). “48448 =0,u-w = 264 26= 10. Then 1.4 < (1)(5) and 48 < (5)(10), confirming =e. 8 Unit vectors v/ful] = (3, $) = (.6,.8) and w/ilw|| = ($, 3) = (8,.6). The cosine of 1s p= pr = H. The vectors wa, —w make O° $0", 180" angles with w. 4@ vu ©) (+ u)-(0-w) = v-vtw-v-v-w-w-w THC J=( )=1 = 0806 = 90° (natce vow ve—40ew +0) (©) (9 2)-(0+2)4 Solutions Exercises 5 a = v/v = @,1)/ V0 and up = w/t = (2, 1,2)/3. Ur = 1—3/ VID is ‘perpendicular to u (and so is (—1,3)/V/10). U2 could be (1,~2,0)//5: There is @ ‘whole plane of vectors perpendicular to ua, and e whole cizce of unit vectrs in that plane. 6 Allvoctorsw = (c,2¢) are perpeaiculartov. All vectors (x.y,2) with + y+2 =0 Tie en a plane. All vectors perpendicular to (1 1,1) and (1,2,3) li ona line. 7 (@) cos0 = w-/Jlfwll = 1/(2)(1) 0.0 = 60° or x/3 radians (b) cos = 0 508 = 0" or x/2 radians (6) cosd = 2/(2)(2) = 1/2500 = 60° or x/3 @) cos = ~1/-/2 900 = 135" or 30/8. 6 (@) False: vand w are any vectors inthe plane perpendiculartouw (b) Tw: u-(v-+ 2w) = u-0-+2u-w = 0 (6) The, [a — of? = (u— 0) (u—v) splits ino meus yea 2uhenuey=0-u =O, 9 Ar vpp/o,u = = then vatig = —vy wy oF 0436, -+02102 = ve = O: perpendicular! 40 Slopes 2/1 and -1/2 multiply to give—I: then ww = 0 andthe vectors (the diretions) ae perpendicular. 41 vw < Omeans angle > 90°; these w's fill half of 3-dimensional space. 42 (1,1) perpendicular to (1,5) ~ e(1.1) i€6 ~ 2c = Oorc = 3; 9+ (w—cv) = Oif c= w+ w/u+, Subiracting cw is the key to perpendicular veetos. 49 The plane perpendicular 1 (1.0, 1) contains all vectors (c,d, —c). In that plane, w = (1,0,=1) and w = (0.1.0) are perpendicular. 44 One possibilty among many: = (1,~1,0.0),9 = (0,0,1,—1). 17 and (I, 1,11) are perpendicular to each other. “We can rotate those w hyperplane” 15 fox + y) = @ 48/2 = 5; con8 = 2/T6/ VTOVTO = 8/10. 46 ol? = 141-4---+1 = 950 fol = Shu = 0/3 = ()..... J) isa unit vectorin 9D; (1,=1.0,...,0)/ Tis anit vector in the 8D hyperplane perpendicular tov. 17 cosa, = 1/3, cos = 0,cosy = ~1/ 2. Forany vector v,cos*a-+e0s? f+-cos? y (oh +0 + })/wP? = 1. 418 ul? = 4? +2 = 20nd 2045. 49 Stare from the rules (1) (2), (3) fora = 10-0 anda-(v4-w) and (cv)-w. Use rule (2) for (o+w)(o-+w) = (v-+w)-0-+(v-+w)-w. Byrile (1) thisis w-(u-w)-+8-(0-+m), Role Q) again gives v-U-+ 0-w + w-u + wow = 0-0-4 20-w + ww, Notice 1v-w = wv The main point ist be fre to open up parentheses. 20 We now thet (0— w) + (v— w) = v-v~20-w-+ ww. The Law of Cosines writes [fulllol cos 8 forv-w, When 9 < 90° this vw is positive, sin his case v-v-+ ww ‘sarge than Jv-~ wf, 21 2o < 2Iullie leadsto v-+wl? = v-v-+2v.9+0-w < [ol?-+2lol ol +e ‘Thisis (lUl + Hl)#. Taking square roots gives [+ wh = [ful + He. 22 vu} + 20, wivaten + udu < vf} + ful + wt + ufud is true (cancel 4 terms) isvtud wyvaiva whichis (wa ~ vu) 2 0. Qh) 1.w in their 3D ce + 2 5. Pythagoras is 43.4)? =Solutions to Beercses 5 23 cos 6 = w;/|jwl| andsin B = w2/Jw|.. Then cos(6—a) = cos f cosa+sin A sine vywy/let eat + taws/fvl fll = v-w/ [ole Thsis 9 because Aa 24 Example 6 gives us [IUs| < $(u + UP) and [uallUs| < (ud + 1/2), The whole line ‘becomes .96 < (.6)(8) + (.8)(6) < 46? +8) + 30.8? + 62) = 1, True: 96 <1. 25 The cosine of 8 isx/./x? + y2, near side over hypotenuse. Then | cos 6)? is not greater than 1222/0492) = 1 28 The vectors w = (1.9) with (1,2)-w = x-+2y = 5 icons nein the xy plane. The shortest w on that ie is (1,2). (The Schwarz inequality te} > w+ w/v = V3 is anequality when cos = 0 and w = (1,2) and [us] = V5.) 27 The engh [ov — wl is between 2 and 8 (wiangle inequality when jul = 5 and [al = 3). The do product vw is between ~15 and 15 by the Schwere inequality. 28 Three vectors in the plane could make angles greater than 90° with each other: for ‘example (1, 0), (1,4), (=1,—4), Four vectors could not do this (360° total angle), How many can do this in Bor RO? 29 Fora specitic example, pick w = (1,2,-3) and then w = (-3,1,2), In this example coo = v-w/loflial = ~1/VidVie = 1/2 and 6 = 120° . This alvays happens when x-+ y 42 =0: 1 aay tyes lotr tot- patsy +z) 1 This is the sameas y+ w 1 loaf. Maen 00s = 3 0 Wikipedia gives this roof of geometric mean G = 3/872 < arithmetic mean A= (x+y -+2)/3, Firs thee is equality in case x 2. Otherwise A is somewhere beween the tree positive numbers, say for example 7 < A

g rays that B® > BPA = x(y + 2— AYA, But +z ~ AA = (y~ AYA 2) 492 > 92 ‘Substitute to ind A” > xy2 = G? as we wanted to prove. Not easy! ‘There are many proofs of G = (xa esq)" A= (xy bxp beset) /m In calealus you are maximizing G on the plane x; 34 ~--+%q =m. The maximum ‘occurs when allx's are equa, 31 The columns of the 4 by 4 “Hadamard matrix” (mes 4) are perpendicular unit vectors: pari y afi -aot ar z]i 1-1 ary: Parad 82 The commands V = randn (3,30); D = sqrt (diag (V' » ¥)); U = V\D; will give 30 random unit vectors inthe columns of U. Then u' « U is a row matrix of 30 dot products whose average absolute value may be close to 2/r.6 ‘Solutions to Exercises. Problem Set 1.3. page 29 1 2s, + 34a + 48s = (2.5.9). The same vector b comes from § times x = (2, 3,4): (STE (es)-6 2 Thesolutions are y1 = 1, y2 = 0, y3 = 0 (right side = column 1) and y ‘ys = 5, That second example illustrates that the first n odd numbers add to By ne Bh i o ‘(8 —By +B, =|-1 1 ol] Bs = +B Lot 1jLm oo 10] sna at ot 1 ya n” ant Wty to nan x Teimeneats=[t 4 The combination Ows + Ow + Ows always gives the zero vector, but this problem {ooks for other zere combinations (ben the vectors are dependent, they ie in & plane): tw: = (wr: + w)/2 so one combination tha gives ero is $wy — wz + 2s. 5 The rows ofthe 3 by 3 matrix in Problem 4 must also be dependent: > dri +73) ‘The column and row combinations that produce O are the same: hiss éuval. as [i 2 ‘| ns cola’ = 2 (column 1) + ealume 2 115. ot 10] tas cotunn3 = ~ eon 1+ colurn2 i 000 a 36 7 All tree rows ae perpendicular to the solution x (the three equations ry =x = O and ras = Oand ry-x = Otell us this). Then the whole plane ofthe rows is perpendicular 10. (the plane is also perpendicular to all multiples cx). Brosh nah 190 oyTh, Bon sh m= bt aftr oolfel ya Sons a bthehs | |i i 1 olf fa ae Boneh bth the LLL Td @ The eclie difference matrix C has line of solution Gi 4 dimensions) to Cx 1 HEM=-t}-——~Sete Bec 1 a-nsb 7 = bi —b—bs 1-1-1) py Wannabe hob oo 3] [e]-2 carne = oo TJ LS. 11 The forward differences ofthe squares ae (¢ + 1)? —@? = 12 420-4 1-12 = 2641 Differences ofthe nth power ae (¢ + 1)" — eat) oo, The lending term is the derivative no". The binomial theorem: gives all the terms of (¢+ 1). 12 Centered difference matrices of even size seem tobe invertible. Look at eqns. and 4 (41 HEHE EDEL 19 Odd size: The five centered difference equations lead to by + by + bs = 0. ‘Add equations 13,5 ‘The left side ofthe sum is zero ‘The right side is by +b3 + 5 ‘There cannot be a solution untess by +B + Bs = 0 146 An example is (a,b) = (3,6) and (c,d) = (1,2). The ratios a/c and b/d are equal. ‘Then ad = be. Then (when you divide by bd) the ratios a/b and c/d are equal! Problem Set 2.1, page 40 1 The columns are # = (1,0,0) and j = (0,1,0) and & = (0,0,1) and b = (2.3.4) = 2 +3) +4b. 2 The planesare the same: 2x = 4 sx = 2,3) = 9is y = 3,and 4z = 16is7 = 4, The solution is the same point X = x. The columns are changed; but same combinstion, 13 The solutions not change! The second pane and row 2 of te matrix andl eolumns ofthe marx (vector inthe com picture) are change 4M: = 2 then xt y = Omnd x—y = 2 give the poin (1.~1,2). Fz Esty = Gandy = 4 proce (31.0). Halfway between those 6 (3.0.1). 5 Ifx.y.2 sty the fist two equations they also sty the thd equation, Te line Lot aoluions consins v= (Icl,0) and w = (fl. andu = 40+ fw and combinations cv + dw withe-+d = I 6 Equation 1+ equation 2~ equation 3is now 0 = —4, Line mises pane no slaion. 7 Column 3 = Column 1 makes the matrix singular. Solutions (x, ,2) = (I, 1:0) oF (0, 1, 1) and you can add any multiple of (—1,0,1); 6 = (4.6,c) needs ¢ = 10 for solvailiy (he is in the plae of the colar. 8 Four planes in 4-dimensional space normally meet at a point. The solution to Ax = G,3,3,2) is. x = (0.0.1.2) if A bas columns (1,0,0,0),(1, 1.0.0), (1, 1, 1,0), GUD, Theequtonsamsr+y-2tia hy te sia RE Sie Rm 9) AE=(85,0 and) Ax = 6.4.5.5). thena Solutions wo Erereses 10 Multiplying s linear combinations of the columns gives the same Ax. By rows or by columns: 9 separate multiplications for 3 by 3. 114 Ax equals (14,22) and (0,0) and (3,7). 12 Ax equals (2,7, ) and (0,0,0) and @,3, 6). 13 (@) x has m components and Ax has m components _(b) Planes from each equation in Ax = b are in n-dimensional space, but the columns are in m-dimensional space 14 2x + 3y +2451 = Bis Ax = b with the I by 4mavix A = [2 31 5]. The solutions fill a 3D “plane” in 4 dimensions, Ie could be called a Ayperplane. s@r-[ t] oe-[ a seo rain ton A= [2 {foto a= [$9] O10 081 ” - ° jis 2d m0 = 10 8] momen) inte oe af8 eae nd E 10 0 =1 1 0] subtract the first component from the second. oot Loo 10 0 woe=|0 1 Of ands =| 0 1 0], 2x = 6.4.8) and E-'Eu recovers ToL 10 1 G43), so m=[) Sfronse ce seis anh = [9°] pst come mre [$] mtno= [2] [2 ~sf]ers coy hese te be reat wl 22 The dot product Ax = [1 4 al] = (1 by 3)@ by 1) is zero for points (x,¥.2) ‘ona pian in thee dimensions. The columns af A are one-dimensional vectors. 23 A=[1 2; 3 dlandx = [5 -2}" andb =[1 7)/.r =b—Avx prinis.as zero, 24 Any =(3 4 5]! and v! # v= 50, But wa A givesan error message from 3 by 1 times 3by 3, 25 ones(4.4) + ones(4,1) =[4 4 4 4]/:B+w=[10 10 10 10)". 126 The row picture has two lines meeting a the solution (4,2). The colurna picture will have 41, 1) + 2(-2, 1) = 4¢eolumn 1) + 2column 2) = right side (0,6). 27 The row picture shows 2 planes in 3-dimensional space. ‘The column picture is in ‘dimensional space. The solutions normally ie on aie.Solutions to Exercises 9 28 The row picture shows fous Hes inthe 2D plane, The column picwre i in four a und b > d would be impossible with integers. This leaves. ° i ‘aman i Pl mana 1 row 1 < row 2 OR row 2 < row 1, An example is M = [i teen [} {foams wt w= [NE TE Problem Set 2.4, page 78 1 allenes of A, B.C, D are I, then BA = 3 omes(5) is 5 by 5; AB = 5 ones(3) is 3 by 3; ABD = 15 ones(3, 1) is3 by 1. DBA and A(B + C) are not defined, 2 G@) A(column30of By b) Row 1 of A) B (c) (Row3 of AY(columa 4 of B) (@) Row 1 of CyD (column | of E). 2 4a Acibesnes aia +0)=[2 $] warrereen 4 48) = 489¢ 9 oe aso ts eget ove feucials cin ena tre ny lat doe soaa[} PJose=[} P] o #=[§ four=[% 2] scasara [9 sae cansaasa owe sraeat=["§ 3] 7 (a) Tue (b) False (c) True (4) False: usually (AB)? #¢ A?B?. i Therewr of Of e3 (uw Lot a)ansS ow 2of), Bahfor of EA ro¥ 201A Teas of Doe cole fans ln Fe te aan TE tea etcoadls Sum | ef 4) coun 20t A wie [22 Ero em ir ae 10 Fr ° é ]=st ECFA) a¢re ara E(FA) is not the same as F(EA) because multiplication fs not commutative.Solution o Exercises 18 oor oo 4. The only matrices thai commite with B and C (and all other matrices) are muliplesof I: A= al. 13 (A—B)? = (B— A)? = A(A~ B)— B(A~ B) = 4? AB~ BA + BP. Ina ypical case (when AB # BA) the matrix 4? — 2B + B2 is iffeent from (4 — B)? ‘44 (a) True (A? is only defined when A is square) _(b) False (i Ais m by mand B is n bym, then AB ism by m and BA isn by). (c) True (4) False (ake B = 0). 15 (@) mn (use every entry of A) (b) mnp = pxpart(a) (c) n? (n® dot products). 16 (2) Use only column 2 of B (b) Usecoly row 2of A (e}(d) Use row 2of fist A. bad was] i 22 |nasaz= fs raat ing A=] mt 1 1 | nasagy = HH! = 123 Lar yy is “altemating sign matrix”. A= | 2/1 2/2 2/3 | has ayy = i/j (this will be an aA 3/2 3/3 ‘example ofa rank one matrix), 4 Diagonal matrix lower triangular, symmetic, all rows equal. Zero matrix fits all four Way Olr=anfar Can-an @) on -Baz 0040 ooo8 2_| 0004] , |ooo00! , MP) Oy o0 | =| ooo [tem zeremmitis fr strictly iiangular A 0000 0000 x » a 8 y 2 ar ° ‘Then Av =A = Bus or Yo 7 nA 4 4 1 ° o ot | bas a? = 1; BC = ‘ : wteeadecce| St uuan=[ 2 2! Jest momt oe [ 5s 1 1] Lfo 07, 1a} |o of ED. You can find more examples. -1 0 i)fo 1p o ve=[t al a]-[a !16 Solutions to Exercises oan [2 } Joa 20. ste Anyi = mon = i oie o1 =|901}rsa=] 000 000 er tut A? = 6; sry triangular asin Problem 20 war=[9 77" ae Th th (4)" ie Gal ‘@ 6 cf 0 0] fa] [i o oj fa]fo 1 0] fejfo o 4} Bld e fol ol=ld He HS ghi joo 1 g A i 1 0 330) po 0 0 Columns of A % a|fs a offal 2 ele 6 ola[s 2 a] mee ie BH [5 ¢ §] [$4] 330 [3 f]-a TRI 27) (om 30 A-(olna 1 9f Ban ow 304) (eam 2B) we D2. “fie Ble Ei HIMM= 10 0° 10 0 29 Ey =]1 1 Ofand£ =| 0 1 0| produce zerosin the 2, 1 and3,1 entries O01 40 1 have A? = uvTuet = 0 if ou = 0, 20 ines ais 10 0 210 Mubiply £'stoget £ = En.£x =| 1 1 0). then £4 =|0 1 1] isthe 4 0 1 013 result of both E's since (Ey Ezi)A = E3i(EaiA)- winwcn[ 3} = [2 J] o-ctie=[} {]ieseimscanrstea 4 Bf eTal teste ee. Sete tin cone 8 ANG] -[A5 23]. sae 32 Atimes X = [x1 x2 x] will be the identity mattix ! =[Axy Ary Ax3}. mia [PJeme an tseett [i]: an [1 1 &]vitane tee f(s = 1.45 = 8 neh‘Solutions so Exercises Ww a+b a+b a+e b+6] whenbme w avome= [EF5 Efe] ewevwinonoe [25 53] city trenaa[e 2 o1o1 20 aJiotol plo 2 Dab,beb abydeh 162-ep Ba=lo ro 1 #=)2 0 abevade che,cde paths in 1010. 02 bbadibed dadyded the graph {96 Mutiplying AB =(m by \(n by p) needs mnp mukiplcaions. Then (AB)C needs ‘pq more, Maliply BC = (n by pXp by q) needs npg and then A(BC) needs ming. (@) fmm, pg are 2, 4,7,10 we compare (2)(4)(7) + (2)(2)(10) = 196 with the larger number (2)(4)(10) + (4)(7)(00) = 360. So AB firs is beter, so that we multiply that 7 by 10 mauix by 2s few rows as posible. (©) ifw.v,w are W by 1, then (aTo)we" needs 2N multiplications but w"(ww") needs 'N? to find vw and N? more to muliply by the ow vectoru*. Apologies o use the transpose symbol to earl. (© We are comparing mnp + mpq with mng + npq. Divide all eems by mnpa: Now we are comparing g~? +n! with p~! + m-". This yields a simple im- Portant ule. [fmatrices A and B are muliplying for ABw, don't multiply the ‘matrices fist ‘97 The proof of (AB)e = A(Be) used the column rile For matsix multiplication—his rule is clearly linea, column by column. Even for nonlinear transformations, A((c)) would be the “composition” of A with B {applying B then A). This composition A o Bis just AB for matrices. ‘One of many uses forthe associative law: The lefvinverse B = ri B= BAC) = (BAJC = C. ] faba,adacha,cda These show inverse C from Problem Set 2.5, page 89 fae ey ator fee co eee taal tata[f plana 2 =[4 g]ecet=[2 4] oor 2 Asm teens P= 1007-1 Pate por [1 9 ol aman 010. P+ = “transpose” of P, coming in Section 2.7 2 fe]=[s]me[t]=[ a] = bf} meetin ve AA =T column by column, the main idea of Gauss-Jordan elimination. 4 The equations are x +2y = 1 and 3x + 6y = 0, No solution because 3 times equation 1 pives 3x + 6y = 3. 5 Anupper triangular U with U? = 1isU=[9 _' 7 {J any anaaio18 Solutions o Erersses 6 (a) Multiply AB = AC by A~! to find B = C (since A is invertible) (b) As long asB—Charefom[ SJ] wehwea = Ac fora=[ | 7 G@) In Ax = (1,0,0, equation 1+ equation 2 — equation 3 is O = 1 () Ria Sides must sntyby-tby = by, (©) Row Sbecomesarow ofzeroi—no ie pol 8 (a) The vector = (L.1-1) solves Ax = (9) After elimination, cours 1 tnt Zend in zeros Ten so does clu 3 = column 1+ 2: no hi pv 9 Iryonrchapge ros a2 F Ao wach B youexge came an80 4! toreach B~!, In matrix notation, B = PA has B-* Vs AMP for this P. o 0 0 15 3-2 0 0 cia]? oY 8 ria |-4 3 0 0 wat=| 3 is 'f and Bt = |G G5 | Gvert each ho 8 0 0-7 6, block of B). 1109 8 =-Atencenany Ar aseommicitsainene. 0) 4=[5 {] snta=[2 S]ac toring 448 = xine 12 C= AB gives C= BA! 50. At = BC, “1B-1A~ so multiply on the left by C and the right by A: B-* aft ov @ yafa 07, ot 1 OP ast Patten scfm 15 If 4 hasacolumm of eos, 0 does BA. Then BA = I is impossible. There is no A~! ie [bY a fads be 0], Theinvere ofeach mai i ea ad —be | the other divided by ad — be i Poe Reverse the orderand change —I to +1 to getinverses Bi E5} " Pp L = E-+ Notice the I's unchanged by multiplying in this oder. 18 4?B = J canalsobe written as A(AB) = 1. Therefore A“? is AB. 49 The (1,1) entry requires 4a ~ 3b = 1; the (1,2) entry requires 25 — a = 0. Thon 20 A » ones(4, 1) isthe ze0 vector so A cannot be invertible, 21 Sixof the sixteen 0 ~ I matrices are invertible, including all four with thee 1'sSolutions to Beercsas 19 21 Uo tty ga 14 Wb 93 4]-1 4 10 10-3 4/3 oe ebseet eet cea Pose Peete ett oo 219) 100 1 g][5 34 faa $ oo ob a ad 1 oo oo oo 1 o|- Ee 3/2 ° us 3/2 v4] > 1 -43 ee eae =1 1/2" oo} 3/6 12 1/4 3p 34 A | a3 if 4 23/4, Lab 100) pa 010-0) 1001-2 ab HEDGES ~ soieotboree i lbors > 4 2aayt yp 3-1-1 p2-1 -19717_ fo affei] -4 3 3)(4 3 =) []= [2] 22 ae 1. 4t-1 -a aL -1 ad, 0, note. a oae[ STE Jel Teeoeoeefs “LE he J 1 2 Matinly by D = [5 4/2]t0reh DE:sBn.A = 1. Tn A! = DEiaEas aps 2 2[-2 1} a a" [ ae wf0 2! [22017 [20 apo a2 2201)*|0210J7|02 10]7lo1 12 0} ‘This is [1 A™"]: row exchanges are certainly allowed in Gauss-Jordan, 2 (a) True (If A has a row of zeros, then every AB has 100, and AB = I is impossible) () False (the matrix of all ones is singular even with diagonal I's: ones (3)has 3 equal rows) (c) Thue (the inverse of A~" is A and the inverse of A? is (4~")*). 90 This Ais not invertible for ¢ = 7 (equal columns), c= 2 (equal rows), ¢ = 0 (zero column). Loo $5 Sone ee ao a o-b 31 Bliminaton produces the pivots a anda—banda—b. A“! = | a a 0 we-D | O-2 a20 Solutions to Exercises ri oo saut=|9 $1 9), when eangsard stems 1 and ont diagonal, 0001 {tigen ion he gpa ed os petgoa Sra (uin sim Pr = QrwtP yea 8 10 at o -D I # [2 tel oteer sijelo 6 25 A cant mere wi goa 20. Bsn bese ach ow as 0 72 26 The eqnion LDLD = Iya LD = pas (4) ss on ven '37 ib(6) i not the exact Hilbert matrix because fractions are rounded off. So inv(hib(6)) isnot the exact either '38 The three Pascal matrices have P = LU = LL? and then inv(P) = inv(LT)iev(L). 99 Ax = b has many solutions when = ones (4,4) = singular matrix and b = ones (4,1). A\B in MATLAB will pick the shortest solution x = (1, 1,1,1)/4, This is the cnly solution that is combination of the rows of A (later it ‘comes from the “pseudoimverse" A = pinv(A) which replaces A~ when A is singular). Any vec- tor that solves Ax = 0 could be edded to tis particular solution x. smmmasa[? bo face [2 tt =| om oo 0 1 ‘would bea good example fr the cofactor formula A? = C'T/ det Ain Section 5.3) ce ¢ 00 1jl0e 01 fi. ee ft ‘that in this order the multipliers shows a,b,c.d,e, f am unchanged in the product (Gmportant for A= LU in Section 2.6). a-UV) Un+UUm=¥UY-*V) (iss testing forma 3) Te-UV +UUm—VE)'V-UVOUm=VU)-*Y (ep simplitying) = —UV$U Un VOM I VUY*V my Cormulas 1,2, 4 ar sire) 49 by 4a with Ty = 1 has pve 1,1, 1 reversing to 7" = UL makes TZ, 44 Add the equations Cx = b tofindO = by +b: +s +ba Same for Fx =B. Problem Set 2.6, page 102 vtec tf omll I] =Eel-eee- 0 IG]-ESolutions to Exercises 2 1 SLE] = [ffonere = [5] tino ovr vrsea[d tf] [J}emetrs [3] tsiaten 9 fy = 1nd tn = Zan = 1p mee ag get A= bon Ut Fatty so = Saas gis eae ome ETE Cs JEL -[YE}-E}--E) 1 210) pie sean[ oy aif £2]=[2 4 f= cwnetscanu= 301 005 ne Es DB be id]-e orf ls Sena £5} EU = LU, ‘The multipliers €31. 22 = 2 fall into place in L. 1 1 var oo Tanta A= [' fe: Jl: 3 2h This is tills). AL GS 101 106 0 2 0) =U. Purthose munipliers2,3.2intoL,Then A= |2 1 0/U = LU. 00 2, 321 secsoitve[' JE J LS, 2 ‘The multipliers are just a,b,c and the upper triangular U is J. In this ease A= L and fis inverse is that matric B = L~' Vio pt yee gy dane= tener orryzamonuannes|t 1 3]-[1 1 ][" 9 4] 7th 12d beat dopa insow 2 leads to zero in the second pivot postion: exchange rows and not singular, ends to zero inthe thir pivot position. In this case the matrix is singular. 4Le toe 24 8 nae [23 3] mt = 1 nao oergste tf > |: 1 0 0 7, 124 A=LU basU =A; A= LDU hasU+D~14=|0 1 3 | with 'sonthe diagonal 001 onlt ToL ]- (1G IB een fea G4 ba I+ Jip {ape8B Solutions to Exercises cee a poanette ots es BE ee Blade Need ¢ 2 barely =1 abe d xc forthis A wr ge web fp) Nee 2B of a ae sf} 9] x-[Joos-[S pnb bus ronte[s t]e=[2]=¢ Lo oy ye Ree hate > 6 [: 10 o=[s]oe [i}s=[3 i ils [Jew A Lider ees ok ; Toe dt we fart ttiioe st a stn ses Tr ; eh lt bl 17 (aL goestol_@)T goestoL™ (@) LU goes oU. Elimination maltiply by L*! 18 (a) Multiply LDU = LyD,Ui by imerses o get Ly'LD = DU\U-!. The left Sides lower triangular, dhe ph side is upper triangular > both sides are diagonal (8) L,U.Ly, Uy have diagonal 1's s0 D = Dy. Then L7*L and U\U~ arcboth 1. 1 11 0 aoa 0 @ elie tt Livia ake 8 ] = cone" » ] o1t 1 0b be, e Game U). tdiagonl mavix A has bidingonal factors L and U. 20 A widiagonal T has 2 nonzeros inthe piot row and otly one nonzero below te pivot (one operation to find € and then one forthe new pivot). = bidiagonal times bidigonal U7 24 For tho frst marx A, keeps the 3 lower zeros atthe stat of rows. But U may not have the upper zo where dag = 0. Fr the second matrix B, keeps the bottom left zero athe sar of row 4. U keeps the upper tight zero atthe stat of column 4, One 22t0in A and two 2er05 in B are filled in prs so 12 0 0 22 mining ses [3 3 i] [F 2 e]- [Fe o]=2 wena rial bias rtd. a lower triangular Land the multipliers are in an upper triangular UA = UE with met u- [3 1 i} 301 28 The 2by 2 per sabre fiw pve 2,7. Reason: Etnaon on A Saran be uperiet comer wy imation o 8 24 The uper et lock al facor at the wae ine 4: Ae it LU, 25 The i, j entry of L~"is j/é fori = J, And Lj;~1 is (1 —i)/i below the diagonal 26 (Ky = fn i + 1)/(e+ 1) ford > j (and symmetric): (n + 1)K™* looks good.Solutions to Exercises 23 Problem Set 2.7, page 115 we fl 9] yar vac[E SJmeare[h Ju has AT = A and 4-1 = 1 1 9 1 2 (ABT is not A°B™ except when AB = BA. Transpose that to find: BTAT = ATT. 3a) (ABT = (BAYT mw (AYT(B-T. This is also (ATBT). {b) IFU is upper triangular, so is U~': then (U~")" is lower triangular 4 A= [8 {Jom atte dapae 44s potene neni themselves. If ATA = 0, zero dot products => zero columns => A = zero matrix. a, 12 37[% 1 z 5 @ xTay=[0 aft 3 g]fif-s oxa-t 5 6) @ [3] stm [Se Sef MP M meds d= Aand BP Cand DY = D =[2 Shar = = Aand BT = = 1) Fuse[3 {] symm oad tA = AT) Fe: Tews of 48 wa a van sane 8 we ame [2 {J amgoeo [2 4] $o(AB)! = AB was BA = AB, aimee mm es ee Sr eae mdi w Sats ee mae mee C™BTAT(= CBA for symmetric matrices 4, B, and C). 8 The tino 1 hoist te 1 now 2a ~ Leos! ove O1 ooo) poor 210 san= [oot] o i}. [eto man = [io] 19 olo 9 o) ~ Li oo, oon 16 Ps nd Ps exchange deren pa of rvs, PyPa = PP des boexanages 10 (3.1.2.4) and (2,3, 1,4) keep 4 in place; 6 more even P's keep | oF 2 or 3 in place: (2.1.4.3) and (34, 1.2) exchange 2 pairs (1.2.3.4). (4.3.2. 1) make I2even P's. 01 07700 6) pl 2g " rele 0 ft 2 H| -[3 4] sere Maing cn 16 ole + 3) Lo0 ‘he rightby «orion matt P, exchanges he coluns. To make is A lowe i \ si ne an net Pw ccegs oe Band AaB = | ‘| lh HHH24 Solutions wo Exercises 12 (Px(Py) = Ingeneral Pay =x-PTy A x-Py: seman ES EEE BE) 010 taaseicr=[2 0 T]orinsmgne vite P= 156123) 2.3.0 100. G12) +0,2.9.7 =[ p]tortmesime Pas P= Po 1 14 Tho “reverse identity” P takes (I, are reversed, (PAP) i8 (A)a-i4, +n) into (n,.--, 1). When rows and also columns “j+t- In particular (PAP)11 i Ana. 18 HP mabe owes Pas ticon 0) #=[ vane =[2 {Jeera ted? ee cn enema. 16 A? B? (but not (A + B)(A ~ B), this is different and also ABA are symmetsc if A and B are symmetsic. 7) A= [} f]e a ie nrinvotse a = [|] sts excrne ora! Spee [ 3 48 (@) 544434241 = 1Sindependent entries if 4 = AT (b) Z has 10and D has 5; tote 15 in LDL" (¢) Zero diagonal if AT = —A, leaving 443-421 = 0 choices. 49 (a) The transpose of RTAR is RTATRTT = RTAR = n by n when AT = A (any m bymmatix R) (0) (RFR), = (column j of R) (column j of R) = length squared of column /) > 0. afb a}-( ifs sll ak FB te Se]fs 4] Baa Ils i 3 et = LDL", ween ey welt Slee smell [e 3] af bw Etde tI | : ; rows 2-3 then rows 34‘Solutions to Exercises 25 [ pony op 211 26 Pam Lisl 1 Ife 3 ]-[2 nl | ite wat 1 211 O 1/3 2. 2/3, 1 Pe Je les ad 25 The splu code will not end when abs(4(k )) < tol line 4 ofthe sla code on page 100. Instead splu looks for a nonzero entry below the diagonal inthe eurent column k, and ‘executes arow exchange. The 4 lines to exchenge row k with row r are atthe end of| ‘Seetion 2.7 (page 113) To find that nonzero entry A(r,k), follow abs(A(K, )) < tol by toexchange and aya isthe pivot, A = Li PiU 2040 heave sion incomplete 26 Ove va o deideeven soo oun al pita Phas inte wrong oder, Then sever or old when ta cout seven or OH, Hard stp: Show than exchange ava vite mt count Then 3 or exchange wl lems a oun 0 1 100 aw en=[3 1 Jeena emote ns=[3 2 ‘| 045 1 isttgmmaic winamatoints 2am. 6) Xowwetn =|! 1 | 41 to make the 3, 2entry zew and E3221 AT, EJ, = D also has zero in its 2,3 entry. Key point Elimination fom both sides gives the symmetric L DL" directly, oes wean]! 228) 2 er tes0.12.3inevry rw. dont hoe ny mers at} symm cnt ti) 29 Reordering the rows andlor the columns of [ aot be the ranspore (which doesn't move a 100 VW Pye yac + Yas 30 (a) Total currents are ATy = |-1 1 O}] yes | = |-yac+yes 0 =1 =I) Lyas. Yes = Yas. T(ATy) = xeyac + Zayas ~ xoYac + xc¥es ~ ‘will move the entry @. So the result (@) Either way (A2)"y AsYes ~ XsYas. 1 50 70 - 1 4 2 6820] uuek a1 | 40 100] [32] = 4s: ary = [ ] x I-L ] [é 2 | x 50 1000 50}] so | = [188000] 1 plane 32 Ax-y isthe cost of inputs while «Ay isthe value of ouput. 38 P= sothree rotations for 360"; P rotates around (1, 1,1) by 120°. o+[! 3]=[2 &][E 2] 21 = oer at ines rei 35 LU7)~ i lower wiengular times lower triangular, s0 lower triangular. The transpose ‘of UTDU is UTDTU™™ = U™DU again, so U" DU is symmetnic. The faciorization _mulplies lower triangular by symmetric to get LDU which is A26 Solutions to Exercises {36 These are group: Lover triangular with diagonal 1s, agonal inveible D,permus- tions P, orthogonal matrices with Q™ 37 Certainly BT is northwest. B* isa full matrix! B~! is southeast “The rows of Barwin everse order from a lower tiangular L, so B= PL. ‘Then 'P-" has the columns in reverse order from L~',”So B~! is southeast. PL times southeast PU is (PLP)U = upper rangula. ‘96 There are i! permutation matices of order n. Eventually nwo powers of P must be the same: If PY = P¥ then P?~# = I. Censinly r—5 m). 46 These bases are not unique! (a) (1,1, 1) forthe space ofall constant vectors (e.eye.e)_(B) (1,1, 0,0)4(1,0, 1,0), (10, 0,1) forthe space of vezior with sum of components =" (¢) (1,1, 1.0) (1,~i,0,—1) forthe space perpendic- lar to (1, 1,0,0) and (1.0, 1,1)" (@) The columos of Fare a basis for its column sph nyt ab comenio) fr NU) = fe esr 47 Thecolumn space of U = [5 9) 9 2] is R® so take any bases for R?; (row 1 and:0W 2) oro" 1 andro | +r 2) Gow 1 and =r 2) ar bas fr ow spaces of U. 18 (a) The 6 vectors might nor span R* (6) The 6 vectors are not independent (©) Any four might be a basis 49 n-independent columns = rank 7. Columns span R= rank m. Columns are bass for R™ => rank = m = n. The renk counts the numberof independent colimns. 20 One basis is (21,0), (~3.0,1). A bass fr the intersection withthe xy plane is (2.1.0). The normal vector (1.2.3) isa basis forthe line perpendicular othe plane 2 (@) The only solution to Ax = is x = O because the colunns are independent (@) Ax = b is solvable because the columns span RS.” Key point: A basis gives exactly oe solution for every 8. 22 (a) Tue (b) Falve because the basis vectors for RS might not be in, 24 Columns 1 end 2 ee bass for the (different) column spaces of A and U; rows 1 and 2 are bases forthe (equal) row spaces ofA and U; (I,~1, 1) is basi forthe (equal) aulspace, irs span the38 Soluioas o Bxersses 2A Fae A= [1 ths deen idgetet ow) Fle lama srw froespentord = [6 $] (Tre Bon dimen = 2504 were dimensions = O14 Other dincasns = |) Fb etna may be dependent, in that case not a basis for C(A). 25 Abas rank 2ifc = Oand d [eae snk 2 exon wen cn 19 0] fo 0 07 po oe w o[2 9 o}.[o i a].[ee 8] 00 lls o of loot 01 0) 0.0 1770 0 0 aut od], ]o 0 oj foot ee 010) 70 917 70 0 oF o[-t¢9).[§ 8 e).[s 9 i ooo 14 8 ole So, ‘These are simple bases (among many others)for (a) diagonal matrices (b) symmetric ‘matrices (¢) skew-symmetric matrices, The dimensions are 3,6, Lo oyri Oy Tl i oy rio Wri oo aie toh [e 2 8} [2 2 o} [8 FoR [a ffs oo alo o tbo o Loo th Lo ot es do not form & subspace; they spam the upper triangular matrices (not every U is echelon). Te o}fo 1 o}yo 0 sp pi-t opr oa wLiseh[e a chp oakhit sl Lt 64 29 (a) The invertible matrices span the space of all 3 by 3 matrices (b) The rank one Imaticeslso pan the space of ll3by 3 matices (1 byiself spans th pce of rmukiples cf. wo [-l 2 9][-1 0 2] 700 of 000 0 0 of 0 0 of |-1 2 of [-1 o 2p 31 (@) y(x) = constant (b) (x) = 3x this is one basis for the 2 by 3 matrices with 2) inher lspci subspace). (@) ya) = 38-+C = Jy + yn soles 32 y(0) = Orequires A + B +C = 0. One basis is cos x —cos2x end coxx —cos3z. 89 (a) (+) = eis a basis for, all solutions to y’ = 2y (b) y = xis a basis for ll solutions to dy/dx = y/x First-order linear equation => 1 basis function in solution space). 34 ys), yale), 9364) cam be x, 2x, 3x (Gin 1) oF 2,2? Gim2) or, 2? (aim3). 35 Basis , x, x2, x3, for cubic polynomials; basis x — 1, x2— 1, x3 — forthe subspace with p)’= 6. 86 Basis for; (1.0,—1,0), (0, 1,0,0) (1,04 SAT = muliples of ¢ : B dor o 0 0 1); basis for T: (1, ~1,0,0) and (0,0,2, 1 ullspace for 3 equation in Ré has dimensioa I,Solutions o Exercises 39 37 The subspace of matrices that have AS = SA hes dimension thre, 88 (a) No, 2 vectors don't span R®(b) No, 4 vectors in R? are dependent (c) Yes, a basis (@) No, these three vectors are dependent ‘89 If the 5 by 5 matrix [A.B] is invertible, is not a combination ofthe columns of A. IF[A_ 6] is singular, and the 4 columns of A are independent, b iss combination of ‘those columns. In this case Ax = B has a solution. 40 (a) The functions y = sinx, y = cosx,y = ey tod*y/dx* = y(n), () A particular solution to d*y/dx* = y(x) + Lis y(x) = 1. The complete solution is y(x) = ~1 +c,sin x +c9 08x + cye* +c4e™* (or use another basis forthe nulispace of the 4th derivative). 1 1 yf 1 Thesix P's arefi {-foalefoa def ad-fa '] fede TD d-b '] se ‘Those five are indegendeni= The dth ha Pty ~ 1 and cannot be & combination ofthe others. Then the 2nd cannot be (Eom Py2 = 1) and also Sth (Pyp = 1. Continuing, 4 nonzero combination of all fve could not be zero. Further challenge: How maay independent 4 by 4 permutation matrices? 42 The dimension of S spanned by all warrangements of x is (a) zero when x ©) one when = (1.11) (©) thee when x = (I,1,~1,—1) because all wear Fangements ofthis x are perpendicular 0 (1.1.11) (8) four when the x's are not ‘equal and don't add to zero. No x gives dim § = 2. 1 owe this nice problem to Mike ‘Atin—ihe answers are the same in higher dimension: 0, i= ls. 49 The problem isto show thatthe u's, o's, ws together are independent, We know the a's and o's together area basis for V, and the w's and w's together are ab Suppose a combination of u's, v's, w's gives 0. Tobe proved: Al coef ‘Key ida: tn that combination giving 0, the part from the u's and v's is in V. Sothe par from the w's is —x. This paris now in V andalso in W. Batif—x isin VW it ia combination of ws only. Now the combination uses only u's and vs (independent in V1) so all coefficients of u's and v's must be zero. Then x = O andthe coeficients ‘ofthe w's are also zero. ‘44 The inputs to an m by n matsx Hl R". The outputs (column spacel) have dimension +. "The nllspace has #~ special solutions. The formula becomes r+ (1 =r) =. Fae a bass for solutions Problem Set 3.6, page 190 1 (@) Row and column space dimensions = 5, nullspace dimension = 4, dim(N(A")) =2 sum=16= m+n (b) Column space is R left mullspace contains only 0 2 A: Row space basis = row 1 = (12,4) nullspace (~2, 1,0) and (4,0, 1); columns space basis = columnl = (1,2); let nulspace (-2, 1). B: Row space basis = both rows = (1,2,4) and (2,5, 8); column space basis = two cofumns = (1,2) and 2, 5); nullspace (4,0, 1); Je nullspace basis is empty beceuse the space contains only y = 0.40 Sofuons wo Exercises 3 Row space basis = rows of U = (0, 1.2.3.4) and (0.0.0. 1,2); column space ba pivot columns (of A not U) = (1,1,0) and (3,4, 1); nulispace basis (1,0,0,0,0), (,2,—1,0,0), (0,2,0,~2, 1) left ullspace (1,1, 1) = last row of E Lo 9 3 40) [1 0] @) impowiie r+in-rymasibe3 (6) (1 1] of ‘] (©) Impossible Row space = column spce requires m = n. Then m— 7 = m= 1; nullapaces have the same dimension. Section 4.1 will prove N(A) and NCAT) ethegonal othe cow and clara spaces respectivly-—here those a the same space 54 =[} {| ha those rove suming row space B = [12.1] a he same rows spain its nullspce and BAT 6 A: dim 2.2.2.1: Rows (0.3.3,3) and (0, 1,0.1); columns (8.0.1) and @.0.0) fupce (10.0.0) and (01,01): NCA?) (01,0). B dm 1.1.0.2 Row space (i, column space (1,5), ntlspace:emy bass, MCA) (4,16) and (80,1). 7 taverible3 by 3 marx A: row space bass = column space basis = (1.0.0) (1.0), (0.0.1); outspace ass and eft lispae bass ae empy. Marx B= [4” A} row Space tis (10.0, 1,0,0}, (01.0.0, 1,0) and (0,0, 1,0,0,1):columh space bass (€,0,0), (01.0), (00, i; allapace basis (~1,0,0,1.0,0) snd (O,~1-0,0,1,0) and (0.0.=i, 00,1) tf rallspce bai is emp. 8 [1 O]snd [1 "1: 0 0)and [0] = 3by Dhave row space dimensions = 33,0 Colurwsace dimensions: ulspee dimensions, 2: eh mulipace dimensions 2,3. 8 (a) Same row space and mulspace, So rank (iension of ro space) isthe same (@) Same cour space and let nlspane Same nk (mension of oluma space) 10 Forrand (3), almost surly rank 3, rlspace and lef rlspace contain onl (00.0). Forrand (35) he ranks almost surely 3 andthe dimension of the mllspace is 2. 11 (@) No solution means that r < m. Alvays + < n. Can't compare m and n bere. (6) Sincem =r > Othe lef rlspace must coniain «nonzero vector, [: i] ron 2 12 Anenctoieis]0 2][} 9 g]= [2 4 9]: r+@-0 vojlt 2 "li ot rot match +2 = 4, Only © = Oi in both NA) and C(47), 18 (Fale: Usa row pace clu space (ame dimension) (0) Trae: A and —A fave the same four subspaces () Fase (choose A and same ize aed iverble: then they hve the sume fo ubspes. 46 Row space basis canbe the nonzero rows of U: (1,2.3:4), (1.2.3) (0.0.1.2): rllspace bass (0, 1.~2. 1) a5 for U: column space bass (10,0), (8.10), 6.0.1) {happen ohave CCA) = CU) = RD} left aulspace has em basi 18 After arow exchange, he ow spce and ulspaze say the same; 2, 1,3,4) is inthe ‘et et ulspace afer the ro exchange. 16 IfAv = Oandvisarow of A env =0. 47 Row space = yz ple; colama space = xy plane; nullspace = x axis et rulspace = mis Ford +A: Row spece = colutn space ~ R° bah mllspaces contin only ‘hevero verter 3 doesSolutions to Exercises a 18 Row 3~2 row 24+ row 1 = zerorow so the vectors (1, ~2. 1) arein the lef nullspace. ‘The same vectors happen tobe in the nullspace (an accident for this matin. 19 (2) Elimination on Ax = 0 leads 0.0 = by ~ bo — by so (—1,—1, 1) is im the left nvllspace. (b) 4 by 3: Elimination leads to by ~ 2b, = 0 and by + by ~ 4b = 0, 50 (2.0, 1,0) and (=4, 10,1) are inthe left aullspace. Why? Thote vectors multiply the matrix to ive zero rows. Section 4.1 will show another approach: Ax = is solvable {8 isin C(A)) when bis onhogona tothe left nullspace. 20 (a) Special solutions (~1,2.0,0) and (—3,0,~3, 1) are perpendicular to the rows of R (and then ER). (@) Ay = 0 has I independent solution = last row of E° (EVA = R has a.zero row, which is just the transpose of Ay ~ 0). 24 (@) wandw — (b) vandz (6) rank < 2 if wand w are dependent or if v and = sre dependent (@) The rank of uo? + we" is 2 1277) 97 [3 2) hes column space spanned Alu wlfet at]=|2 2 [i i]- 42) bywandw, row space 41 51) spanned by'v and 2. 23 As in Problem 22: Row space basis (30,3), (I, 1,2); column space bass (1,4,2), (2.5.7) the rank of 3 by 2) times (2 by 3) cannot be larger than the rank of either factor, s0 rank = 2 end the 3 by 3 product isnot ivertbie. 24 ATy = d putsd in the row space of A: unique solution ifthe tet mulspace (nulispace of AY) contains only y = 0. 25 (a) Thue (A and AT have the same rank) (b) False A = [1 O] and AP have very ilflerect lft nlispaces—(¢) False (A can be invertble and unsymmeticcven if (A) = CCAD) () True (The subspaces for A and —A are always the same, If A= Aor AT = ~A they ate also the same for A") 26 The rows of C = AB are combinations of the rows of B. Sorank C < rank B. Also rank rank A, because the columas of C are combinations of the colurans of A 77 Chose d= e/a tomaie [tt] saa max Then teow sac bases (2) andthe nullspace has basis (=b, a), Those two vectors are perpendicular! 28 BandC (checkers and chess) both have rank 2ifp #0. Row 1 and2 are a bats forthe row space of C, B¥y = Ohas 6 special solutions with 1 sed 1 separated by a zo; N(C4) bas (~1,0,0,0,0,0,0, 1) and (0,~1,0,0,0,0, 1,0) and columas 3,4,5.6 of 1:M(C) isachllenge an ‘30 The subspaces for A = av™ are pairs of orthogonal lines (v and e+, w and w), FB hs those same four subspaces then B = ed withe # 0, 34 (a) AX = 0 if each columa of X is a mukiple of (1.1, 1): dim(nullspace) B then all columns of B add to zero; dimeasion of the Bs im(M?") = 9 entries in a3 by 3 matrix, ‘32 The key is equal row spaces. First row of A = combination ofthe rows of B: only possible combination (notice ) is | (row 1 of B). Sazme for each row so F = G. Lara = 0.03 = Laas = O.asa = Least = 0,425 = Lass = 0.021 = 1 3 6a2 ‘Solutions to Exercises Problem Set 4.1, page 202 1 Both nullspace vectors are orthogonal o the row space vector in R®. The column space is perpendicular tothe nullspace of A (two lies in RF because rank = 1). 2 The mullspace of a3 by 2 matsix wit rank 2 is Z (only zero vector) s0 Xq = 0, and ow space = R?, Colum space = plane perpendicular to left nullspace = lie in R. Pe 2 nyt s0[34 £] orm [-F] rotor?) [1] [8] 33 5 ek" Ls, €(A) and N (4") isimpossibe: nt perpendicular (@) Need A? = Ostake A = © (1.1) inthe mullspace (columns edd to 0) and also row space; no such mati. 4 1 AB = 0, the colurms of B are inthe mullspace of A. The rows of Aare inte left of B. If rank = 2, those four subspaces would have dimension 2 which is impossible for 3 by 3. 5 (a) If Az = B has a solution and ATy = 0, then y is perpendicular tb. By = (axyy = a7(ATy) = 0.) {CATy = (1.1, i) has asotution (1.1.1) isin dhe row space and is orthogonal to every = in the nllspace. {6 Muliply the equations by y1.32.)s = 1, 1.~1. Equations addt00 = 1 so0soation: 3 = (.1,—1) isin the ie nulispace. Ax = b would need 0 = (J7A)x = 9" 7 Multiply the 3 equations by y = (1,1,—1). Then x1 ~ x2 = 1 plus.x2~ 3 xr sa = Lis = 1, Key point Tis y in NCA") is not onhogonal to 50. is notin the eoluran space and Ax = b has no solution. Xr +n, where x isin the row space and xm sin the mullspace. Then Ax = 0 dnd AX = Ax, + Ate = Ax. AI AX are in C(A), 9 Ax ig always i the column space ofA. If ATAx = Othen Ax is also inthe mulspace (f°. So Ax is perpendicular to itself. Conclusion: Ax = @if ATAx 40 (e) With AT = A, the column and row spaces are the same __(&) x isinthe mullspace and zi in the column space = row space: so these "eigenvectors" have x72 = 0, 11 For A: The nullspace is spanned by (2, 1), the row space is spanned by (1.2). The column space isthe line through (1,3) and (4%) isthe perpendicular line through G.=0), For B: The nulispace of B is spanned by (0,1), te tow space is spanned by (1.0), The column space and lef nollspace are the same as for A. 12x splits into.x, + x4 = (1.—1) + (1.1) = (2,0). Notice NAT) is a plane (1,0) = CD24 C/T 2, +a 13 VTW = 2er0 makes exch basis vector for ¥ onhogonal to each bass vector for W. “Then every vin V is orthogonal to every w in W” (combinations ofthe basis Veco). 1.1) 14 Ax = BF means tat (A 8}[_E] = 0. Tae homogereos epaon in four vnkoowns aways fave «onze sition, Here x = G1) and 2 = (1.0) and Ax = BZ = (5,6,5) is in both column spaces. Two planes in R* must share a line, 15 A p-dimensional and aq-dimensional subspace of R" share atleastalineif p ++q > n. (he p +4 basis vectors of V and W cannot be independent.) 16 ATy = Oleads 0 (Az)"y = x"ATy =0, Theo y 1 Ax and (A). C(A).Solution o Exercises 43 17 IES is the subspace of R® containing only the zero vector, then Sis R. If $ is spanned by (1, 1,1, then $+ isthe plane spanned by (1, 1,0) and (1, 0,1). 1S is spanned by (2, 0,0) and (0, 0,3), then $+ is the line spanned by (0, 1,0). t8 stisdeoutmeect4=[} 3 {] metic Sienna int 19 L+ is the 2-dimensional subspace (a plane) in R? perpendicular to L. Then (L+)+ is ‘8 I-dimensional subspace (a line) perpendicular to L+. In fact (L+)* is L. 20 If V is the whole space R‘, then V+ contains only the zero vector. Then (V+)! = 21 Fernie (5.0.1. 0.1.-1.O)gmnStamsimeeot d=} 2 2 3 22 (11,11) ina basis for P4. A= [1111] hae P as its llspace and P+ os row space. 22 x in V+ is perpendicular to any vector in V. Since V contains all he vectors in $, + isalo perpendicular to any vectorin S. So every x in Vis also in $+, 24 AA“ = 1: Column 1 of A~! is onhogona the space spanned by the 28d, 3. inh ows of A. 25 ifthe columns of A are unit vectors, all mutually perpendicular, then ATA = [ [ 22 yl ‘This exemple shows a matrix with perpendicular columas. 2% 4=[-1 2 2], ATA =97 is diagonal: (A*A)y = (column of A) (column j of A). 2-1 2)" When te columns are unit vectors, then ATA = 1 ‘27 The lines 3x + y = by and 6x +2y = by are parallel. They are the same line if ‘bz = 2by. In that case (by, b2) is pexpeadicular to (~2, 1). The nullspace of the 2 by 2 matrix is the line 3x + y = 0. One particular vector in the ullspace is (~1,3). 28 (a) (1.~1,0) is in both planes. Normal vectors are perpendicular, but planes sill in- tersect! (b) Need three orthogonal vectors to span the whole orthogonal complement, (©) Lines can meet atthe zero vector without being orhogonal. 123] FL 1 17 Abuse = (2,3 nro apace nd column space Fs a-[ o-[} 13] 210 2-1 0]; Bhas v ini column space and aullspace, 301 30 =1)' veannot be inthe nullspace and row space, or in the left nullspace and coluran space. These spaces ae orthogonal and vy of 0, 90 When AB = 0, the column space of B is contained in the nullspace of A. Therefore the dimension of C(B) < dimension of N(A). This means rank(B) = 4— renk(A). ‘31 null(N’) produces a basis forthe row space of A (perpendicular to N{A)). 82 Weneed r™n = Oand eT = 0, All possible examples have the form aer™ witha # 0. ‘99 Both r's orthogonal to both a's, both c's orthogonal to both £'s, each pair independent All 4’s with these subspaces have the form [ey ¢2]€ [eral fora 2by 2 invertible M.4 Solution to Exercises Problem Set 4.2, page 214 1 (@) a°b/ata=5/3; p=S0/3e= 1h; p=ae 2 () The projection of b = (cos6.sind) onto a = (1.0) is p = (cos6.0) ©) The projection of = (1, Jontoa = (1,—1)is 7) =0. vod resflJonal ene[} jaetiit i. presean.0, Prpcaoon *=[o of} J-1 1] Pi P2 # O and Py + P2 is nota projection matrix. WEL -a ot 442 5h, [2 ‘ ‘} 44 3], mand rte projection 34 4 2371 matrices onto the lines through a matrix because ay 1 ap. XXX Above solution doesnot it in 3 ins. 6 P= (3-5, 5: 5)-Sopi + P+ Ps ba Cay pats rosnenaifa t a]eg[t 4 a]egfe tl 2 22 179424 ees EE SOR EE el & The projections of (1, 1) onto (1,0) and (1,2) are py = (1,0) and py = (0.6, 1.2). ‘Then py + 2 #5. 9 Since A is invertible, P = A(ATA)~! AT = AA~!(AT)~! AT = J: project on all of R?. w nel Bi}e=[) -[e] Ranga 11 (@) p= A(ATAY1ATH= (2, 3,0), € = (0,0,4), ATe =0 (6) p=(4, 10 0 [33] 008 projection matrix onto the column space of A (the xy plane) ne [3 os 3} ‘Projection matrix onto the second column space. hp bd-a oo 1. its when b isin that space. -1,2,2) and ay = (2,2,—1) PrP, = 0 6),e=0. RA walt : i} pose [2 3 14 The prjasion oft tthe ou pce one ° iP nsoncay P= i] $s] ater -9- [3] at ‘i 18 2A has the same column space as A. & for 2A is half of for A. ooo 100 O10 000, of Ais b =Solutions © Exercnes i 46 $01,2,-1) + 40,0,1) = 0.1.1). Sod is inthe plane Projection shows PB = b 47 If P? = P then (I — P)? = (1 - P)(- P) = 1 - P1-1P + P? = 1 — P. When P projects onto the column space, J — P projects onto the left nullspace. 18 (a) I — P isthe projection matrix onto (1,~1) in the perpendicular direction to (1,1) (©) 1 —P projects onto the plane x + y +2 =O perpendicular to (1, 1,1) i 56 6/3 For any bis et inthe plan x= ~2¢ = 0, 19 ay (1.0) and 0.1) he mats Fis [ig Hie “3 1 [6-1/6 -1/3 5/6 1/613" a.-[-i] ose -[46 V6 iA} 1-0 [8 16 “13 > mis 8 2p iB -18 1B 21 (ACATATATY! = A(ATA)(ATAATAY1AT = ACATA)"HAT. So P2 = P, Pb isin the column space (where P projects). Then its projection P(PB) is PB. 2 PP=(ACATAY ATT = A(ATAY"!YTAT = A(ATA)“1A? = P. (ATA is aymmettic}) 23-If Ais iveribie then its column space is all of R". So P = J ande 126 The nullspace of AT is orthogonal o the column space C(A). Sif ATS Jection of & onto C(A) should be p = 0. Check PB = A(A¥A)'A%D = AU 25 The column space of P will be $. Then r = dimension of S 26 A™! exists since the nk isr = m. Muliply A? = A by A~! 10 get A= 1 27 IEATAx = Othen Ax isin the nullspace of A°. But Ax is always in the column space ‘of A. To be in both of those perpendicular spaces, Axx must be zero. So A and ATA have the same nullspace. 28 P2 = P = PT give PTP = P. Then the (2,2) entry of P equals the (2,2) entry of ‘PT P which is the length squared of column 2 29 A = BT has independent columns, so ATA (which is BB") must be inverible 20 Tessa sent tne towsha = [2]s0 Pe = SHE = ALS (b) The row space is the line through » = (12:2) and Px = wot/ap. Always PA = A (columns of A project to themselves) and AP = A. Then PcAPR = A! 34 Theetrore = b~ p must be perpendicular wo all the a's. 32 Since Pyb is in CA), Po(P1b) equals 1b. So P2P: = Py = aaT/a’ ). the pro- ")0. where PaP; then § is contained in T or Tis contained in S. ‘34 BB™ is invenible asin Problem 29. Then (4° A)(BB") = product ofr by r invertible ‘matrices, so rank. AB can'thave rank < r, since A" and B° cannot increase the rank. Conclusion: A (m by r of rank r) times B (+ by n of rank r) produces AB of rank r.46 Solutions wo Exercises vel] Tose [F]oweeefb Seen [28] i seat= stops «[] ase a= | mrcar [] e\ecuroas ti ]pe7 | 8]. mis ar=nisunorabie] 5] pr 211 aT[S]=| a} cktnatonepecten 2] 3 [= [4] encty somes 1 afters 3 3 In Problem 2, p = A(ATA)“1ATH recta user A hinsbreat : 4 E = (C+ 0D)? + (C + 1D ~8)? + (C 43D — 8)? + (C + 4D — 20). Then @E/IC = 2C + 2(C + D—8) + 2(C + 3D — 8) + AC + 4D — 20) = O and OE /al 1.216 + D-8) bao + 2(C + 3D — 8) +4-2(C + 4D — 20) = 0. These, alsin wean [5] =[] 8 Ee CoC MHC Ber (Cm20%- A aL | | Nanda = 4] A [36] and (ATA) A’ ‘best height C. Errors € 6a = (1,11, 1) and b = (0,8,8, 20) give F “b/ata Sate pojeton is TNC, 11,1) = 0 and |lel] = Ba_= p ="(9.9,9,9). Then e'a = (9, VBI. (0.1.3. 4]", ATA = [26] and A% = [112], Best D = 112/26 = 56/13, 6/13, p= (56/130, 1,3, 4). (C, D) = (9, 56/13) don't match (C, D) = (1,4). Columns of A were not perpendicular so we can't project separately to find C and D. patos [19 roy fo os omen (ts) [feeds Se Pe oearey re) rey 78) Reon Wiis 9 mfpelt| sl ™| 2]: be SEP La Les 47 | ‘This Vandermonde matrix gives exact interpolation 41 (a) The bos line x = 1 + 4¢ gives the center point 5 = 9 when? ©) Tei geen Co + er 7 a by aeubie at, 1,3, 2.Solutions o Exercises a7 18 (ATAY1AT(6~ Ax) = F— x. Whene = b ~ Ax averages 10 0, s0 does 2 — x, 14 Tho matrix (2 — x)(@ ~ )¥is (ATA)! AT(b — Ax)(O ~ Ax)" A(ATA)~!. When the average of (& ~ A2)(@ ~ Ax)" is 07, the average of (@ — 2)(E ~ x)? willbe the ‘uiput covariance marx (A A)-* Aa ACA A) which simplifies o o?(AT A) 15 When A has 1 column of ones, Problem 14 gives the expected err (F — x)? as GAIA = f/m, By king m measures he vanaee drops tom 2? 0 ly Ie ed a . 16 {pb + Gio = jplbs + + bod. Knowing Py avoids ang al b's 1-1 1 ¢ ‘ 3 2)[¢] _ [3s wfid (s]-(7] Tresorion? = [4] coms tom [} Z][$]=[3] (i MGl-L 2 Ls 18 p = AF = (6,13,17) gives the heights ofthe closest line. The enor is 6 — p 0.76.4) Thiserore has Pe= P= Pp = p~p 19 1fb = erore then b is perpeniculro th columa space of A, Prjesion p = 0 20 11 = AF = (6,13,17) then = (9,4) and ¢ = O since is in the column space oA, 21 e isin N(AN); p isin C(A); & ein C(A"); N(A) = {0} = zero vector only. ae coonf® [C].[ § Z east squares equation is [4°] =[_,5], sctsion: © = 1, D Line 1~ 1, Symmetric 1's = diagonal AY 4 29 ¢isonhogonal io then le = €7(b— p) = Th = BT —Bp. 26 The derivatives of Ax — bf? = =™AAx — 26" Ax +5" (his term is constant) are ‘zero when 24 Ax = 24%, or x = (A°A-*A"D, 25 3 points on aline: Equal slopes (ba~bi)/ (tots) = (by~ba)/ (tata). Linear algebra ‘Orthogonal to (1, 1,1) and (t.fa, fa) is ¥ = (—fa.f5—f1.44—f) in the left nllspace. ‘is in the column space, Then y" = 0 isthe same equal slopes condition writen as (r~ bi ~ 8) = a= badite = : iA) 8 Hl-- BIE = 0,0the best plane 2—x~y has eight C = 2 = average of 127 The shonest link connecting two lines in space is perpendicular to those lines. 28 Only | plane contains 0, a,,03 unless a, a are dependent. Same tet for @1.-.-+4n. ‘29 There is exacly one byperplane containing them points 0,4... When then —1 vectors ay,... dq are linearly independent. (For n = 3, the Vectors a and aa must be independent. Then the three points 0, 4,47 determine a plane.) The equation ofthe plane in R® will be ax = 0. Here ay is any nonzero vector on the line (cs only @ line!) perpendicular 10 a3,....@q-1-48 ‘Solutions to Exercises Problem Set 4.4, page 239 11 (@) Independent (b) Independent and orthogonal (©) Independent and orthonormal. For orthonormal vectors, (a) becomes (1,0), (0,1) and (0) i (6. 8), (8,~ 6) WP 2 4H 2 Ege eteasn we Ls moo [38 8 3 (a) ATA willbe 167 (b) ATA will be diagonal with entries 1, 4, 9. 1 100 4@o- [3 i} eo [2 1 8] #1 am ovine ] : [3 at | ic 1 1 0 0 3 praay a 11 tt o-2 1 1d 0 0-3 11 0 0 0-44 2 [00] ar ptt Aimy liste =| 01 2 sn [fi] cons 2 ern bats Ga Seo be coluria Space of A. The m ~n columns of Qz are an orthonormal basis forthe left rullspace of A, Together the columns of Q = (Q1 Qa] are an onthonormal basis for 837 This quesion describes the nest gq in Gram-Schmidt using the mati Q withthe columns qy..-04 (instead of using those q's separately). Start ftom a, subract is projection » = O*a onto the carlier q's, divide by the length of e = a ~ Fa to get Gnas = e/lelt Problem Set 5.1. page 251 1 des(2A) = 8: det 2 dada) = (Leta det(A“!) = =1. 3 (a) False: dei{I +1) is n001 +1 () True: The pret rule extends to ABC (use o the=[ oh (1) decd and deu(—A) = Dard = 1; dea?) = 1; ioe) Fes didnt Fae 4 [2 9] snoae=[? ~f] iene 4 Exchange rows | and 3 to show [J3| = —1. Exchange rows J and 4, then 2 and 3 to show [fal = 1. 5 [Jsl=1, [Jel=—1, Vr]=—1. Determinants 1, 1,—1,—1 repeat s0 Uioi|= 1 6 To prove Rule 6, multiply the zero row by # = 2. The determinant is multiplied by 2 ‘Rule 3) but the matrix isthe same. So 2det(A) = dei(A) and det(A) = 0.| for rotation and det(Q) 1 for reflection (1—2sin? @—2.cos? @ = [Qi = 1 = [Q] = £1; Q" stays omhogonal o det cant blow up. 9 det A = | fromtwo row exchanges. det B = 2(subuact rows 1 nd 2 fom row 3 then columns 1 and 2 from column 3): det = O (equal rows) eventhough € = A + BY 40 IF the entries in every row at o 20, then (1,15... 1) is in the nllspace: singular A hs det = 0. (The columns add tothe zero colin so they are ineasy dependent) It every row adds to one, then rows of A= 1 add to zero (not nocessatly det A = 1). 11 CD = DC => detCD = (~1)" det DC and nor ~ det DC. If ni even we can have. an invertible CD. 12 det(A~') divides twice by ad ~ be (once for each row), aah 13 Pivots 1,1, give determinant = 1; pivots 1,~2,~3/2 give determinant = 3, 14 det(A) = 36 and the 4 by 4 second difference matrix has det = 5, 18 The fist determinant sO, the second is 1 — 212 + ¢4 = (1 — 12)2, 416 A singular rank one matrix has determinant = 0. The skew-symmetric K also det K = 0 (ee #17. 7 Any 3 by 3 skew-symmettic K has det(K®) = det(—K) = (=1)Pdeu(K). ‘This is det(K). But always det(X) = det(K). So we must have det(K) = O for 3 by 3 pag! |i a a pea W)1 b ) =] 0 ba Baa | = [PO HO | 0 sect 2 by 2 tec} [0 cma cna eve climinate a and a? in row 1 by column operations). Factor out b — a and ¢ — a from berm b-ae-o | 28 |= O-ale-ave~B 10 Forwiangulrmatices, jst mp the diagonal enix: det(U) = 6, (U") = $, And di() = 36. Zby 2 matrix: de(U) = ad deuU?) = a2d2, Ifa 4 On eu") = Ifa a~te b= Ld 20 det[2~ fo 5 ees toad ~be)(1~ Lt). The determina changesityou oo operations ance. 21 Rules and3 give Rule 2 (Sine Roles 4 and 3 ge 5, thy io give Rule 2) 22 dea) = 3.de4~Y) = § de ~A1) = 27-40-43. The umbers A = Vand give dt(A ~ 1) = 0. Note insrucor: i yo dics tis exec, you can plain that his she reason determinants come belo egemalues. saul 2 = 1 fod = Saath eigeralves of A 3-41 3 1h] a ea those at iene. 29 aw(a) = 10, = [18 iT} asta) = 100, 4-4 = Get(A 41) = 1272+ 10 Owhen d= Zord 26 Here A = LU with det(L) = 1 and det(U) of pivots, so also det(A) = 6. det(U-1 “1 / de(A) and det(U~!1L-1 A) is det = 1 25 When the, j entry is ij, row 2 = 2times row 1 s0 det A = 0.52 Soluions o Exercises 26 When the fj entry is f+ j, ow 3—row 2 = row2—row 1 so is singular: det A 27 det A = abc, det B = -abed, detC = a(b ~ a)(c —b) by doing elimination 28 (a) True: det(AB) = det(A)det(B) = 0. (b) False: A row exchange gives — det product of pivots. (¢) False: A = 21 and B = I have A— B = J but the determi- nants have 2" = 14 1 (@) True: det(AB) = det(A) det(B) = det(BA). 29 Ais rectangular so det(A™A) 9 (det A)(det A): these determinants are not defined, ie [yr yas) [ee ws] * Laf/ab af/ad | ~ = ad—be ad —be. 90 Desivaives of f = In(ad — be) fd Bog wa[4 t]-« ‘31 The Hilbert determinants are 1,8%10~, 4.6% 10"*, 1,6%1077, 3.7 10-", 5.410 eee ec iS Ba su ofS ‘nants so the 10th pivot is near 10~'°. The Hilbert matrix is numerically difficult (ill- win 32 ‘Typical determinants of rand(n) are 105, 1075, 1079, 102° for n = 50, 100, 200, 400. randn(r) with normal distribution gives 104, 10", 10'86, Inf which means > 27°, MATLAB allows 1.999999999999999 x 21023 nx 1.8 x 10°%* but one more 9 gives Inf! £8 Leow ae ht xing nin oma ame rt Se i iy varse oo D SEN tal Sec tia Oe ine eveyone Santas serch teen ‘njas) includes the solution for small n (and more references) when the problem is ee Ee ee Metres coun PES ‘5,9. Then the 1,—1 maximum for size n is 2"~* times the 0, | maximum for size n ~1 Se en ae ‘To reduce the 1,—1 problem from 6 by 6 to the 0,1 problem for 5 by 5, multiply the Dee Sane meee esetmnae tienen Taeeeret Se ee esis Here isan advanced MATLAB code and a 1, —1 matrix with largest det = 48 for n= 5:p = (a —1)%2; AD =ones(n); maxdet= 0; O:2%p—1 remdtoor(k. « 2.(—p + 1:0)),2); A = AD: AQ :n,2: 1) reshapo(Asub, n= i, ond OvtpurmaxA= 1 1 1 1 1 maxdet 48. Poroia-t 1ore-tot-t paris 1‘Solutions o Exercises 53 ‘34 Reduce B by row operations to [row 3; row 2; row 1]. Then det # = ~6 (odd per- riutation). Problem Sei 1 det = 14.18412—9—4~6 = 12,rowsare independent; det B = 0, row I-+r0w 2 = row 3; det C = —I, independent rows (det C has one term, odd permutation) 2 det = ~2, independent; det B = 0, dependent; det C = —1, independent. 3 Alleofactors of row 1 are zero. A has rank < 2. Each of the 6 terms in det Ais zero, Columa 2 has no 4 ayraayayadus gives ~1, because 2 ++ 3, ayeazyasoaa gives +1, det = detB 2-44.21 044-1 = 64-16 = 48, '5 Four zeros in the same row guarantee det det #0), 8 (@) Ihayy = aaa = 239 = Othen 4 terms ae sure 270s (b) 15 terms must be ze, 7 51/2 = 60 permutation matrices have det = +1. Move row 5 of Ito the top; staring from (5.1.3.3:4) eltioation wil do fourtow exchanges, 8 Some term a;q¢2p ---dqy in the big formula isnot zero! Move rows 1. 2, ows a, B,.-»y@ Thea these nonzeto a's wil he on the main diagonal. ‘9 To get +1 for the even permutations, the matrix needs an even numberof =’ To get +1 for the odd P's, the matrix needs an odd cumber of ~1's. So all six terms = +1 in the big formula and det = 6 are impossible: max(det) = 4. 10 The 41/2 = 12 even permutations are (1,2, 3,4).(2,1,4,3),3s 1.4.2) (4,3,2, 1), ‘and 8 P's with one number in place and even permutation ofthe oer three numbers, det(! + Peven) = 16 oF 4 oF 0 (16 comes from 7 + 1). nee[t t}o-[3-2 3 ba! eae page 263 0. A =F has 12 zer0s (maximum with 1m ito 1(0) +242) + 3(~35) = ~21 Puzzle: det D = 481 = (—21)2. Why? 321 400 ne=[} i 2festacr=[2 ‘ 3| Treo A! 123 00 4, 19 @ C=O, G==1, C= 0, G=1 1 then cofactors of column 1. Therefore Cio = "Ba =-1 14 We must choose 1's from column 2 then columa 1, column 4 then column 3,and s0 on. ‘Therefore m must be even to have det Ay #0. The numberof row exchanges i n/2 0 C= 1", 16 The I, cofsctor of then by n matix is Ey. The 1,2 cofactor has a single 1 in its first column, with cofactor Ey2 sign gives Ena. $0 Ey = E, a. Then Ey 10 Bg is 1,0, 1, ~1,0, 1 and this eyele of six wil repeat” Eyop = Ea =~. 16 The 1, 1 cofactor of the m by m matrix is Fy. The 1,2 cofactor has a 1 in column with cofactor Fa. Multiply by (-1)!*¥ and also (1) from the 1.2 entry to iid Fy (60 these determinants are Fibonacci numbers)a Solutions to Bxeeses iat s 17 Bu ol 12 =] ves[t 2 2] = auas~ {| HI 2}Bs| — |Bal. TBsl aa | Ba| are cofactors of row 4 of By. 18 Rule 3 (linearity in row 1) gives |B, [dal = [4a] = (a+ In = 1 19 Since x, x?, x? are all in the same row, they are never multiplied in det Vs. The deter- inant sero ats = a or bore, so detV hs factor (x ~a)(x~b)(x ~c). Moliply byte cot ¥, The Vandenonde mati Vy = (21) for ting polynomial POs) = bat the fins. has et V = pod of al xy ~ qr k > 20 Gz = —1, G3 = 2, Gy = —3, and Gy = (—1)"""(a — 1) = (product of the 2's ). 15, 3.b 21, The ule looks lke every second number in ibonscel’s : so the gues is Sy = 33, Polling the solton {oFrobiem 30 with3'sjasteed of 'sconfirms S4 = 81+1~-9—9—9 = $5, Problem 33 tirecty proves Sy = Fane. 22 Changing 3 to 2 in the comer reduces the determinant F,42 by 1 times the cofactor ‘ofthat comer entry. This cofactor i the determinant of Sx (one sz sale) which Us Fone Two changing 3 to 2 changes te detenmioat Wo Fara ~ Fay whch i Fans, 28 (a) If we chaos an entry from B we must choos an enty from the zero block; re sult zero. This leaves entries from A times entries from D leading to (det A)(det D) owe tier [h fuaa[s Se=[s shoals 9/ sens 24 (a) All L’shavedet= ;etU = det Ay =2,6,-6 fork=1,2,3 (6) Pivots2, 3,38. ae tatensagnada{ cy! 9] =tmaae[ 4B] = asionio-cr'e which is |AD — ACA™1B|. If AC = CA this is |AD ~ CAA™'B| = det(AD — CB). Ge AR esor dfs dacs teas esas ear aoa 5 Fee een at ad ani anh Sates TE ee ee meno: 27 (a) det, 111; + +++ + 49 Cin. Derivative with respect to a4; alow! iver sow} Osh mars nenene 29 There are five nonzero products, all 1's with a plus or minus sign. Here are the (row, column) numbers and the signs: + (1, 1)(2, 2)@. 3)(4.4) + (1, 2)(2, 1)3.4)(4,3) = CL. 2)(2, 13,304, 4) — (1. 12,2903, 44, 3) — (1, 12, 3). 2964, 4). Total 1 a0 Te Soa sn 2 cet 164 14-44 As 23 end 2)2)2)2) + D-DD ~ HDC DQ)R)— AAV) Q2)(-1N-D@). = cofitctor Ci. 4 det P = —1 because the cofactor of Pye = 1 in row one has sign (—1)!**, The big formula for det P has only one term (1+I-I-1) with minus sign because thee exchanges te 412,350 1.2.3.4, 6078) = Be PHGELP) = +10 ae G] =Solutions to Exercises 55 aol? {Jima ‘32 The problem is to show that Fan+2 = 3Fix — Fon. Keep using Fibonecci’s rule: Fanga= Fanti + Fan = Fan + Fan + Fn Fan + (Fan = Fan: Fan — Fan. 48 Thecitonnes om 20% 19 mp 9 y3cler =: ti ps by 3G) Telurtes ovement tnaepnant (leno bie: Cotes Gin tsiee tiem saben ties eon 25 Situacng | om Dena cy mbes tceatr Cnn sen it care alec iasSaeet Gy f edte Problem Set 5.3, page 279 to |} $faa|[d $]=6 |} [=a =-aa=-amen = 3/3 (b) jA| = 4,1B;| 3,[Bo| = 2.[Bs| = 1. Therefore x, 3/4 and ‘2X2 = ~1/2and2a = 1/4. 2 y=lehl/[e5]=cMaa—be) ey y= der By/derd = (fe id)/D. 3 (@) x = 3/0 and x2 = ~2/0; no solution —_(b) xy = x2 = 0/0: undetermined. 4 (0) x, = det([5 az a})/ et, ifdetA 0 () The detemminant slimes in 4 fintcolumn of; 62 314-1303 03/+13Ia3 a3 as), The last owo determinants ane zero because of peated columns, leaving [ay a2 | whichis det 5 Ifthe first column in Ais also the right side D then detA = det By, Both Br and By are singular since a column is repeated. Therefore xy = |Bx{/lA| = and x3 = a= 0. 1-30 321 i | An imventibiesymmeirc matrix © GZ]? $ 2) asa symmetric inverse Slias 7 ‘would be the zero matrix iit existed; cannot exist. (And 7 taller te coictr toma gies dot A'=0) A= [ |} 0 280 cof bats not iovenibe, 6-3 0 300} Thsis (dea)! anda = 2, sce[ 34 “T]emsact= 0 3 0], thet seonorat 4x0 621 9 031° Maipyingby #7100: no ang, 9 If we know the cofactors and det = 1, then C™ = A! and also det A~! = 1, [Now Ais the inverse of C¥, so A can be found from the cofactor matrix for C. 30 Take the determinant of AC™ = (det A). The eft side gives det ACT = (det A)(detC) “while the right side gives (det A)*. Divide by det Ato reach det = (det.A)*—, 11. The cofactors of A ae integers. Division by det A = 1 gives integer enties in A~! 112 Both det 4 and det A~ are intoges since the matrices contain ony integers. Bu det A 1/€et so det A must be 1 or =I56 Solutions w Execives O13 12 1 [Ie 1] mestcrmaixe =[ 3 alma 210 13-1 14) Lower whangular has cotton Con = Cox = Cyn = 0) Ci = Ca. Ga = GioiGh = Gop pate St marie") Ontopoa @ hs coer matric C = (det Q)(Q" 1)" = £0 also orthogonal. Notedet Q = 1 or =1 15 For = 5, C contains 25 cofactor and each 4 by 4 cofactor has 24 terms. Each term needs 3 multiplication: total 1800 multiplications v.125 for Gauss-lodan. 16 (a) Area |? 2| = 10 (>) and (©) Area 10/2 = 5, these triangles ae half ofthe parallelogram in). 3381 99, Ara of faces = ijk 17 Volume |}33l+20. Are scec grower =|} 4 18 ane Lor, 194 3 ti ~2j + 8k Jengih=6V2 (©) 5+ new triangle area S47 40 |} $| = 4 = [f 3] because the uanspose has the same determinant. See #22, 20 The edges of the hypercube have length YTFIFTT = 2. The volume det H is 24 = 16, (H/2 has onhonormal columns, Then det(H/2) = 1 leads again 10 det H = 16.) 21 The maximum volume LyLzL3L is weached when the edges are orthogonal in Re ‘With entries 1 and —1 al lengths are M4 = 2. The maximum determinant is 2* = 16, achieved in Problem 20, For a3 by 3 matrix, det A = (/3)? can't be achieved by tI. 22 XXX TO SENDIN EMAIL, 2 oa [Et be] [< ws 2 fom geata = altel neat = 101 100 24 The box has beight4 and volume = det] 0 1 0| = 4.0% J =kand (k-) = 4 234 25 The n-dimensional cube has 2" comers, 12°! edges and 2n (n—1) dimensional faces. Coefficients from (2-+ 2)” in Worked Example 244. Cube from 2/ has volume 2" 26 The pyramid has volume J. The 4-dimensional pyramid has volume 2 (and in RY) 27 x =1cos0,y =r sind give J = r. The columns are orthogonal and ther lengths are Vand Singcos@ peospsind —psingsind Singsind peospsin® psingcosd corp sing for ple integra inside spheres. arjax ar/y) | xin, lr, ajax B6/8y|=|—y/r xf? sing. This Jacobian is needed ind eee =| iar tbe eet a 7 > Jacobian in 27"Solutions to Exercnes 87 ‘90 The triangle with comers (0,0), (6.0). (1.4) has area 24. Rotated by @ = 60° the area i nchnged. Te tenia’ fe rton maxis J = | 8. ~ 88 2 -V3/2 N32 1/2 21 Be a 10,heigh2, ome 2. 240 22 Thevoleneatitetorisde| 1 3 0] =20 122 1 vy | uy ]82 9 pau] BS | guy YE D2 is w(x w) sa famously wo| |i, | fan af Tieton ‘36 (wxu)-¥ = (0x w)-u = (xv) -w: Even permutation of (a, 1) keeps the same ‘determinant. Odd permutations reverse the sgn, 35 § = (2.1,~1), ea IPQ x PSI = [(-2.~2,—1)| = 3. The other four comers ‘ean be (0,0,0),(0,0,2), (1, 2,2), (1, 1,0). The volume ofthe ted box is [det = I. aye 2 10.1002 3 aainanetentmenca| 19] -ysn0 ‘The “box” with those edges is attned to zero height. 121 ape aanli i] =te-54evimiezmmena.s snd(1,2.2). Te plane conning hw wo ets has quton 7 = 59 42 = 0 ‘combination of (2,3, 1) 88 Dovblingeach row multiplies the volume by 28. Then 2det A =:det(2A) only if (det A)I gives (det A)(detC) = (det A)*. Then det A = (det C)"!? with / det A, construct A! sing the cofactors. Invert tind A. ‘40 The cofactor formula adds | by 1 determinants (which are just entries) times thei co- factors of sizen ~ 1. Jacobi discovered that this formula can be generalized, For = Jacobi multiplied each 2 by 2 determinant from rows 1-2 (with columns a < b) times ‘23 by 3 determinant from rows 3-5 (using the remaining columns c < d or. (ua")u = u(u"v) = 4502 =1 0) Py = (uatjv = (uv) 10,0), £2 = (3.0, 1,0), £3 = (-$.0,0, 1) allhave Px = Or = vectors Of this rotation matsix are xy = (If) and x2 = (1,i) (more generally cx, and dr, with ed # 0), ¥5 The other two eigenvalues are A= 3(-1 1/3}; the three eigenvalues are 1, 1—1. ¥6 Set = Dim det(A ~ 21) = (ay ~ 4)... (dy ~A) t0 find det A = (Ay)(22)-~- (hn). May = Mot d + J@=dP FAD) and dy = Ya td-V—) add oa +d Abas Ay = 3and Aa = 4 then det(d — 21) = (13) —4) = 22 —72-+ 12, eope ara se redness cane [§ $].[ 2 2].[ 2 Zh 19 (@) rank =2 —() det(B™B) =O (4) eigenvalues of (B +1)! are 1... 20:4 = [of] sce 1 and determin 28,02 = and 7 Moving a3 0 16 Seompmionsan,c = [2.9 Him dete-21)= 18-46" 11h26= 6 1 6. (1 2)@— 29G — 2), Notice the wade 6 = 1+ 2+ 3, deteminant 6 = (1)2)(), and alo 11 = ()2) + (DG) +@)G). a= 0, smukiply 1,2,22 in Get(A ~ 1) = 92~ 29: A = companion mauris 21 (A~ AI) has the same determinant as (4 ~ A1)™ [} 3] eal} s] sate and 0 ‘because every square matrix has det M = det M7. [1 0. eigenvectors. 22. 2= (for Markov, 0 (for singular), ~} (0 sum of eigenvalues = trace = 3). af? 4 p 4 B If asazed teammate = onto, 1 0} {0 of [1 1} by the Cayley. Hamilton Theorem in Problem 62.32. 21.50.06 (otic rank | and race 6 with 25 = (02,1, = 0-20, 29 = 25 With the same n 2's and 2s, Ax = cydizy to---+ GpAnxn equals Bx = crAaey + Gadd forall vectors x, $0 A= B. ‘26 The block matrix has A = 1,2 from B and 5,7 from D. Allentres of Care multiplied bby zeros in det( ~ 21), 90 C has no effect on the eigenvalues. 29 Triangular matrix: 2(A) = 1.4.6; A(B) = 2, V3, —J3; Rank-t matric: 4(C) = af SL] = [ete] -e+9 [p= + temmaeete cae60 Soluioas to Exercises 34 Eigenvector(1,3,4) for A with A = 11 and eigenvector (3, 1,4) for PAP". Bigenvoc: Bee eed cele eeecteecel ays inthe column space, ‘82 (a) w ise basis forthe nullspace, w and w give a basis forthe column space (0.4.2) isa particular solution. Add any cu from the nullspace fad a solution, u would be in the column space: wrong dimension 3, 0 then A® (v'uu* is the zero matrix and 2? = 0,0 and A = 0,0 . This zero trace also comes from adding the diagonal entries of and trace (4) = Azul: 1 ts 1102] pas trace wivy + ua ae [Bee t= [ES ioe] mime wi tee ‘This reflects te fact that P? = 1. The so- (eal) and = e273, 2 = e°21/3 (complex conjugates). ‘The real eigenvector; = (I, 1,1) is not changed by the permutation P. The complex eigenvectors are x2 = (Ioe-2*4,e-4/9) and x3 = (1,02, e4) = 35 3 by 3 permutation mates: Since P™P = I gives (det P)? = 1, the determinant is 1 ‘or =I. The pivots ae always 1 but there may be row exchanges). The ace of P can be 3 for P'= 1) 0r1 (for row exchange) or 0 (for double exchange). The posible eigenvalues are and —1 and"? and 2/3, e179 give detsaz = 1 and trace 2 + Aa 2m with 6 = has this trace and det. So does every M~1AM 5 Swath cours of Aa 1, oa iene sd = Land te oerine~ 6 intbesemctomec (b) If c = 1.6 then both eigenvalues are 1, and all solutions to (A — J) x = @ are fae Gy (© te = 8.hecipewectos or = ae miles of 13) Steal pen 42 st ve cat ume = 14° wi apm 2(! 1) ae mau A wih ‘eigenvalues 1,0 and correct eigenvectors, (1,3) and (1,—1). Problem Set 6.2, page 307 GIG IG I ARG sI-L: 98 IE 4] + came’ anon [3 O08 SB 2 If A = SAS~ then the eigenvalue matrix for A + 20 is A+ 21 ‘atrin is still S. A-+2/ = S(A + 21)S~! = SAS“ + S(QI)S“! = A+21. 4 (@) False: don't know 2's (b) True (€) True (4) False: need eigenvectors of $ ASolutions to Exercises 61 5 With § = 1,4 = SAS™! = A isa diagonal matrix. If 5 is triangular, then Sis triangular, so SAS" is also wiangular. {6 The columns of S are nonzero multiples of (2,1) and (0,1): either order, Same for A~! (JP: Jl} dJe< Ete tskd- BORE [g S]eeane mes 7A = SAS" eazsast=[! lex i i O][ 4 -B] sats = 1 ofa i “ae 2nd components Fi rerra ° a a Of = 48/0 ~ Aa), s@a-[F ey da =~} withay = (1,1), x2 = (.-2) orl IE Sf} d-e-[h 40 The rule F-s2 = Fis + Fe produces the patter: even odd, 044, even, odd, od, 11 (a) True (a0 zero eigenvalues) (b) False (repented A = 2 may have only one line of eigenvectors) (c) False (repeated 1 may have a ful se of eigenvectors) 12 (@) False: don'tknow A (b) Thue: aneigenvectoris missing (¢) True. S]owstee,4=[2 f} «=[2 Sh aa Te weer de Hu ocd at ty cas ei Bes Sie Saline a ge w(t She-ft A] eft) Lal onl we(f]-or[ foe [2] [f]eoesmer[3] +3] 18 {3 3] - 5[! “IE ie iJaen = 2f} “le 2 1 TJ any tose tie ares to gee at = 231 RAs -11 -# 143 * " 1 r]fs offi x] _pst staat wae[t ifs To i]-[8 *3"} 20 det = (det $)(Get A}(Get $1) = det = Ar---Ag. This proot works when A is ‘diggonalieabe.62 Solutions o Exercises (ag + bs) + (cr + de) is equal to (ga + re) + (sb + td) = trace. Diagonalizable ease: the race of SAS" = wave of (AS“!)S = Az sum ofthe i's, singe AB gue BA = sero. ABBA Cpmutewtert(C) dns [} ]mwzs" ee ={-4 8] on aeronctan-[t S]-[8 SAL Seem the ational eigenvalues 243, ...,2%n 24 The 4’s form a subspace since cA and Ay + Ap all have the same S. When S = the A's with those eigenvectors give the subspace of diagonal mattices. Dimension 4 25 IA has columns...» then column by column, A? = A means every Ax) = Al vectors in the column space (Combinations of those columns x) are eigenvectors with 4 = 1. Always the nullspace has 1 = 0 (A might have dependent columns, #0 there could be fess than n eigenvectors with A = 1). Dimensions of those spaces add tom by the Fundamental Theorem, so A is diagonalizable (n independent eigenvectors altogether). 26 ‘lwo problems: ‘The nullspace and column space can overlap, s0 x could be in both. ‘There may not be independent eigenvectors in the column space. 27 Rees VRS"*m[} jas RA, VB neds 3 m VF and VT, te is ot a Natetine[-]_f] ean tne = etn. eatepae wa $f] 28 AT = A gives eTABe = (Az)"(Bx) < [Ax||BxI by the Schwarz inequality BY = a] of ‘] © [i {]} tee stow he unable css (@) hy 0") Ay> Dandi > 0” ©) A= atid witha >0 18 d/dtlet) = APATEAE LEAD oom AUF AEA LAI bo) ‘This is exactly Ae‘*, the derivative we expect. Wem 1481 hon seses vit # = 0)= [5 ~Af] Deve [9 “$] = 2 20 The solution attime t+ 7 rar Thus e™ times e47 equals e+, #6 )-b IG Ib a6 Alle Ib a}-[ i 22 AP = Agivese!! = 14 Art Att LAP 4 ae i sa eto [5 MF] at net eee ols HT aean[ S]-[5 218 $][o Ym afl 3p 13 " meme =! 3]'=[} J] =a. mata = [2 ~f] som. by ee mniain hel pol [o Mr} . So edt = PACH /M 4 AR TEE ¢ CD) were 26 (a) The inverse of ise“) I Ax = Ax then eM’ = eM ande® £0 Tose eMtx, write (4 A+ LAY? bande = (LEAL + S22 + ode elle,66 Soluons to Exercises 27 (9) = (ee) isa owing solution. The comet mati fo the exchanged = ounis[_2 “Zedoestov te san gear a ga mas. 28 Comesing rons Uass=[_ hy 1 Cane] Ym avar= 1 [ e!*!3 and e'*!9, Both eigenvalues have 4° = 1 so A® = J. Therefore U ames exec back 1 Vee ag Fist A has 2 = +f and At Second A has 1 = —1,—1 and fe cin [52 3 Jum ed ‘That matrix has orthonormal columns => orthogonal matrix => |Wnell = IW 31 (@) (cose = (Cosd)x () ACA) = 2x and 050 cos = 1.1 and eos = / ©) w(e) = Heo821)1, 1+ 1(C0801)(L,—1) fu! = Aa es exp, w= Au has cos} 30 Witha = Ar/2the trapezoidal step is Une = 5 ae a] Problem Set 6.4. page 337 Note Ifyou se a nice way to complet the proof atthe end of page 334, 1 would like 10 ditto mathsnitedwlinearalgebra asaya yaar 43.6) po -1 -2 14=[3 3 3]+]1 0 -3 : : 633) 7l2 3 0) symmetric + skew-symmetric. 2 (ATCA = ATCT(ATY? = ATCA. When A is 6 by 3, C willbe 6 by 6 and the tiple product ATCA is 3 by 3. 0,4, —2; unit vectors 4(0, 1, -1)/V/2 and £(2, 1, 1)/¥6 and £(1, -1,-1)/V3. =1oni-sion=['9_9].x=[!] mel. 7} mtenmatoatiot wetmino= Sef! 3]. er of “he comms of @ a unieigescts of 4 so-3[? 3 -l] Rowrmsenciiemanty 6 A= [2 12) has A = 0 and 25 s0 the columns of @ ae the two eigenvectors: ‘or we can exchange colurans or reverse the signs of any coluran. 19 [} J]imi--Imi2 ( Mephouinetesmegwantels (ime = Ay+Az = 2,50 A can'thave two ngative eigenvalues.Solutions to Brercses or 8 1 4? = Othen all 23 = Osa = onsina = [9 bf Ais symmetric then A? = QN9QT = Orequires A = 0, The only symmetic A is Q.0 QT = zero mati. 9 IFA is complex then Z is also an eigenvalue (A¥ = Tz), Always 4 +7 is real. The trace is real so the third eigenvalue of a3 by 3 ral matix most be rea 40 IF isnotreal hen A = x" Ax/x*x isnot always real, Can't assume real eigenvector! oft dd d(} Le B)oLS Ss a] 42 [x1 x2] isan orthogonal matrix so Py +P: = xx] -+x2x] = [21 22] G¢faa)x] = 0. Sond prool: P: Pa = Py(I ~ Py) = Py ~ Pi = Osince o 8 40} 4[o A 4=[.5 thmer-nmioresrmee [tS]? fo -symimenie wih 2 = i (ie) and 2 = ~ib (wie 14 skew-symmetric and onhagonals 23 must bef, —Ty = w have tree 2, 15.4 = [{_}] as = 0,0 and only oe independent eigen x = (1). The good property for complex matrices is not AT = A (symmetric) bat A" = A (Hermitian Sih eal cigemalies nd oroond eigervecor See Pot 20 and Scevon 10.2) 16 (a) If Az = Ay and ATy As then BI 8) =[-Az; ATy] = AL. —A is also an eigenvalue of B. (b) ATAr = AT(Ay) = 27x. (c) A 1 = (1,0,-1,0), x2 = ,1,0,=1), x3 = (1,0,1,0), x6 ), 1,0, 1). 0 ' ty papiiierg [: ° i}ns Vi.—Viby Problem 16 withey = [-'} ma a 1 me] Lies vi 18 1, y isin the nullspace of And x {sin he column space = row space because A= AT Those spaces ae perpendicular so 97 = 2 if Ax = Ax and Ay = By then shifty 8: (ABI) = (A~B)x and (A-Pl)y = and eguin xy. 11 oF 1017. Perpendicular for A 19 Amss=|1 -1 0): enss=|0 1 ol, Notperpendicola for 3 oot 00 2a) since BT B m0 A= [924,244] atemiion marc = A). sigs and ~4 we ‘real, Adjust equations (1)~(2) in the text to prove that 2 is always real when Ax = Ax leads to A¥ = Tz, Transpose toX7A = 37% using A ‘Then F" Ax = ¥7Ax and alsox" Ax = F"Ax.S0A = Lis real,68 Soluons to Exercises 2A (a) False. 12] @) Tre from AT = QA ; [oF] ORE Bier te 22 Awe A av ese bathe oreo he scan chang. = [21] Ay = fad 2a = i with = (11) ft for A but) = (1,1) fst foe AT, 23 is iverble, onhogoosl, permutation, dagonalizable, Markov, B is projection, agonalizable, Markov. A allows QR, SAS~', QAQ"; B allows SAS~" and QA" 24 Symmetry gives QAQT if b = |; repeated A and no S if b = I; singulerif 25 Orthogonal and symmetie requires [Al = 1 and A rel, oA = 1. Then ovo® [es “eels a]L-see e)-[e8 £1 or sin28 siné cos9|{0 —1]|—sind coo || sin20 —cos20 | 26 igenvectors (1,0) and (1,1) give a45° angle even with AT very close to A. 27 The roots of A? + bh + ¢ = O are 3(—b + VB®—4ac), Then Ay — 4a is VB? = de. Fordet(A + 1B — AI) we have b= ~3~ 81 andc = 2++ 161 — 12, The miniwum of BP Ae is 1/17 att = 2/17. Then dg — Ar = 1/7. 20 4=[p4; 754] =a pnt geal = 5 and —1 with ace = $d ‘det = —5. The solution to 20 proves that A is real when A = A is Hermitian; I did not intend wo repeat this pan. 29 (8) A = QAQ™ times A = QRTD™ equals A times A because ART = ATA Giagonal!) (b) step 2: The 1. entries of T? T and TT? are jal? and jal? + |b)? ‘Thikmakes b= Oand T 30 [avn =. gum) [2nd ay «+ Anan] Aman (lana? +22 + lean?) = Ane 34 (a) 27(Az) = (Ax)"x = xTATs = —27Ax. (b) #7Az is pure imaginary, its eel pan isx™Ax + yTAy =O40 (Cc) detA =Ay...A_ = 0: pairs of A's = 1b,—id. Problem Set 6.5. page 350 1 Suppose a > 0 and ac > b*s0 tha also c > b2/a > 0, _G) The eigenvalues have the same sign because Aida = dt = ac — 6? > 0. (i) That sign is postive because Ait Aa > O Gtequals teincea +¢> 0, Posiivedefinie [1 O][1 Bb 1oyfs 0 7p ob a mescpes [8 [> ou] =[2 Mle o2:][o t]=29 ivedofiito [1 o]f2 4 To7f2 0 Jf 2 ‘i were” [2 ][b cte)=[2 ifs <25][0 i] =12 4 Sry) =x? + Any +99? = (+29)? + 5y% WP + ay +99? = (CBW 5x24 Any $392 = (x + 29)? — 9? = adiference of squares is negative atx where the ist square is 20, 6A [ha] pet 0.2 = ai ali]-. ‘Abas A = Land i A's an indefinite matrix and f(x, y) = 2xy has a saddle point.Solution to Exercises 69 ree=[} SJauae=[$ 5 deta wa = [3 3 4 =|) g]andate=[§ §] are postive define; 35 4/is singular (and positive semidefinite). The frst two R's have independent columns. The 2 by 3 R cannot have fll column rank 3, with only 2 rows. adap? Jaf} O]f3 O]fL 2], Pivots3.4 outside squares, é =[6 w)=[2 alo 4] lo i} stares tay ea? ide, Meee Seve oan [it ) moos sants 2-1 0 2-1-1 1 fo wa-[i a “| asf 2 “tf snen [= [2] ota atte 7 Lo 19 Come deerninat 1 Ast = 30. The pos ar 2/1, 6/2 3076 ‘12 A is positive definite for ¢ > 1; determinants c,c? — 1, and (¢ ~ 1)(¢ + 2) > 0. B isnever positive definite determinants d ~ 4 und ~4d + 12 re never both postive). s04~[} f]smesmmlwine +> 2bbaee «02 oma pte sete 14 The eigenvahies of A~! ar positive because they are 1/2(A). And te ens of A“t pass the determinant tests, And x™A~!x = (A~*x)"A(A-!x) > O for all x #8, 18 Since x7Ax > Oand 2" Bx > Owe have 27(A + B)x = x7Ax +x"Bx > 0 for all x £ 0. Then A +2 is a postive definite muurix. The second proof uses the test A= RFR Gndependeot columns in R): ITA = R*R and B = S™S pass this test, then asB=[R sf [J also passes, and must be postive definite, 16 x7 Ax is 220 when (x1, X23) = (0, 1,0) because of the zer0 on the: ally x” Ax goes negative for x = (1, ~10,0) because the second pivot is negative, 7 If aj; were smaller than all 2's, A ~ ayj! would have all eigenvalues > 0 (positive Aefiite, But A ~ayp1 has a zero in the (jj) position; impossible by Problem 16. 18 If Ax = Ax then Ax = Lex. IFAs positive definitethis leads tod = xTAx/xTx > 0 (ratio of postive numbers). So positive energy = positive eigenvalues. 19 All cross terms are x7 = 0 because symmetric matrices have orthogonal eigeavec- tors, So positive eigenvalues => postive energy. 20 (a) The determinant is positive; all A > 0. (&) Al projection matrices except J are singular, (€) The diagonal entries of D are its eigenvalues (2) A= —I has det = +1 when n is even. 21 Als postive definite when s > 8; B is positive definite when «> S by determinants. LP? wb) p 2 ta Jha 1) [2 Theol? lore % ee [t pe=e[s lo-fi 3] 23 x?/a? + y2/b? is xTAx when A = diag(1/a?, 1/b?). Then Ay = 1/a? and Ay = 1/b* sod = Vf/X; and 1/J/Iz. The ellipse 9x? + 16y? = 1 has axes with hatftengths a = } and (0) and (0, $) are a the ends ofthe axes 270 Soluons wo Exercises 26 The elipse x? 4 xy + y? = ede ae te vi and 273. sarore-[E HEE S]-E ile s]ls t]mee- G5 3] 7 p30 0 vi 26 Tre Colesy aso © = (L/D) A 1 g]msc= O11 | ne OT 03 30s ‘square roots ofthe pivots from D. Note again C™C = LDL" = A. 2 Writing out x" Ax = x™LDLTx gives ax? +2bxy +ey? = alx + by)? So the LDL" fom elimington is exactly te same as completing the square, The ‘example 2x? + Bry + 10)? = 2G¢ +2y)? +27" with pivos 2,2 ouside the squares and multiplier inside. 2B det A = (1)(00)(1) = 105A = 2.and 5; = (6086, sin @), 2 = (—sin6,c056); the Hs are postive. So A is positive definite. vm =[% % Jis semis: / (Gx? +9)? = oon the cue $x? + y = 0; 6 17 2[0 1 inde ' ' m= [% S]=[P b] sindetnie a0, 1 where ts devine = 0. Tis sa saddle point ofthe function f(x,y). 90 ax? + 2bxy + cy? hasa saddle point if ae < b?. The matrix is indefinite (2 < O and 2 > 0) because ihe determinant ac —b? is negative. 34 Ife > 9the graph of zis bowl, fc < 9 the graph hat a saddle point. Whea the graph of 2 = (2x + 3y)*is a “rough” staying at zero along the line 2x + 3y 82 Onhogonal matrices, exponentials ¢4, matrices with det = 1 are groups. Examples Cf subgroups are orhogonal matrices with det = 1, exponentials 4 for integer m ‘Another subgroup: lower triangular elimination matrices E with diagonal I's, 33 A product AB of symmettic positive definite matrices comes into many applications. ‘The “generalized” eigenvalue problem Kx = 2M-x has AB = M™-1K. (often we use eig(X, M) without actually inverting M.) All eigenvalues 2 are positive: 9 ABx = Ax gives (Bx)'ABx = (Bx)"Ax. Then 2 = x"B™ABE/x"Bx > 0. ‘34 The five eigeovlues of K are 2—~2 os 8 Prosuct of those eigenvalues is 6 = det 95 Put parentheses in xTATCAx = (Ax)"C(Ax), Since C is asumed positive definite, this energy can dropto zero only when Ax = 0. Sine Ais assumed to have independent ‘columns, Ax = 0 only happens when x = O. Thus A°CA has positive energy and it positive deft My textbooks Computational Science and Enginering and Iniroduction 10 Ap- plied Mathematics sat with many example of ATCA ina wide range of applications, Teeliove this is unifying concept from linear algebra ~ V5,2-1,2,24 1.24 V3. The‘Solutions to Exercises am Problem Set 6.6, page 360 ee vo-[3 Jal TE i fhaweme LIB a(IE 3h EET BAR 5 3 ‘| [2 9]: [EL 8h [3 i] ae sini ya nve eign 1 and 0 D i 0 { [isbyitsetfandatso |? 2 isby itself with eigenvalues { and —1. {8 Ligh fanties of similar matrices: six matrices have 2. = 0, 1 (one family); three rmatices have A = 1, I and three have A = 0, 0 (two families each); one has & 1, ts one has A = 2 0; two matioes have 2 = $(1 v3) (they are in one fail) 7 (a) (M~'AM)(M~'x) = M~'(Az) = M~'0= 0 (b) The nullspuces of A and ‘of =! AM have the same dimension. Different vectors and differen bases. ‘Same A afer = [9 2] have the same line of eigenvectors oSeg oua=[ ojo [0 §] Soames veel} Jeb Teor pee ore[2 Joma [8 Hey" s+ wor = [5] = [20)} smo fe = [Jona = ae te “Fp = tw. Then w(t) = 2e¥ and v(t) must inclode 2te% (this comes from the ‘repeated A). To match v(0) = 5, the solution is v(t) = 21e* + SeX ‘may may maa ma 0 ma m3 0 42MM =KiensM=|_ 9 9 9 0 | Lyx]? mae mas 0 may maz may an 0 mz m3y 0 o 0 0 0 0 maz _mas_ 0. "That means may = maa = as = max = 0. M isnot invertible, J oot similar to K.2 Solutions to Exercises 19 The five 4 by 4 Jondan forms with A = 0,0,0,0 are J; = zero matrix and jE jE Problem 12 showed that Js and Js senor similar even with the same rank. Every rmguix wih all X= O 1 niloiet™ its mh power is A” = zero matrix) You tee I= Cor these matices, How many possible Jordan fomns form = 3 and all =O? 16 () Choose My = revere diagonal mati to get MJ: = MT in each bock (2) Ma has those diagonal blocks M; to get My "JMy = J*. (3) AT = TMT ‘equals (M~')EM5"JMoM? = (MMoM™)-"A(M BoM"), and AT is similarto A. 15 det(M~'AM — AJ) = det(M~'AM — M~'AIM). This is det(M~"(A — AI)M). By the product rule, the determinants of M and M™' cancel to leave det{A —A/).. 1 0 0 0 1 0 ° 0 6 [2 Gfisinierio[§ S} [2 f]issimiario [$f]. soewopais of sims roncrte[) S]mmcointr[?. J} toate 17 (a) False: Diagonalize a nonsymmetric A = SAS~!. Then A is symmetric and similar (©) Tre Asinglarmatic tas = 0.) Fae: [ 9 4 ]and| “5 ae sitar (they have A= £1) (d) True: Adding J increases all eigenvalues by 1 18 AB = B-'(BA)B so AB is similarto BA. if ABx = Ax then BA(Bx) = A(Bx). 49 Diagonal blocs 6 by 64 by 4; AB has the same eigenvalues as BA ps6 —4 2203 20 (a) A= M-'BM = A? = (M~'BM)(M~'BM) -*B2M. So A? is similar to B. (b) A? equals (—A)? but A may not be similar to B = —A (it could bel). [5 {insta S]hennsh 2 nmbeemac Co [}. {fons oeciseresrsoneigoizsle () PAs simiariod 2 hathae sowie srcon spertgos and incendie fr cae: rno.testysintntemie[? Juin [3 > t]eese[2 J} ee 0 0 0. Noten rote Anineresing questi Wk marie Ane compl qr toss AF Avil protien, Ba any Joa tes ark = Oat hhave sizes my > ng >... > mk > Mex = 0 that come in pits like 3 and 2 inthis example: ny = (nz or mz 1) and ns = (na or a1) and Soon,Solutions to Exercises 73 20 0) fa 1 0 Ate oa by 304 Jin mete [#8] [8 3} oo ello oo, a 12). oe ay mies 2.80) ‘ og 8] maT Tepencie aug.) a | “5b eee [te peer Problem Set 6.7. page 371 raouerefes olf fo of bal IE 2] ania [} Z]ee2} 2muitoran | tseospectsbas tac fal testis Merete 1 : roves eft] nati bf 2] coumspse [3]. may ef 3] 311A has rank 1 then so doss ATA. The only nonzero eigenvalue of ATA ists tac, which isthe sum of all af. (Each diagonal entry of AA isthe sum of a}; down one column, so the trace isthe sum down all columns.) Then &) = square rot ofthis urn, and? = this sum ofall a, 2 i {meena ot 34¥5 92 3-v5 Bwais saaeae F788 =F indente 0 = (1+ V5)/2 = ANCA), 02 = (VS ~ 1)/2= ~Aa(A)i wy = vy but Hy 5 A proof that elgshow finds the SVD. When V1 = (1.0), V2 = (0,1) the demo finds AV; and AV at some angle 6. A 90° tum by the mouse to V.2,~V finds AV and AV a the angle x ~ 8. Somewhere between, the constantly orthogonal wy and v2 ‘must produce Av, and Av at angle-x/2. Those orthogonal directions give uy and w2.14 Solutions w Exercises waste [oat =vines =] ened = twee [1%] 110 V//8 Wh wae[h 2 Toso witho, = | 2/¥6 |, oF =1withys =] 0 p14 WW. -Wi We rio Woo r wins = [122] me[p T]e te wtf 2B) a a 7 The mats A in Problem 6 had oy = V3 and oz = 1 in E. The smallest change o rank 1is to make @ = 0. In the factorization A= UEV" = ao,0] + 2020} this change o2 + O will leave the closest rank-1 matrix as ayoy0. See Problem 14 for the general case of this problem 8 The number Gaa(A™!)Ouae(A) isthe s0m¢ 38 Gau(A)/ Gein A). This is cetinly > 1 Woguals 1ifallo°s awe equal and A = UEV" is a multiple of an orthogonal matrix ‘The ratio uc One isthe important condition number of A studied in Section 9.2 9 A = UV" since allo; = 1, which means that 40 A rank-1 matrix with Av = 12u would have w in is column space, oA = wut for some vector w. 1 intended (but didn’t say) that w is a multiple of the unit vector GL 1,11) in the problem. Then A = [2uv¥ to get Av = 124 when vy = 1 11 If has onhogonal columns w,..., th of lengths o1,...,0, then ATA wit be die agonal with entries of.....03. So the o's are deftly the singular values of (as expected). The eigenvalues Of that diagonal mauix A*A are the columns of 7, s0 V= 1 inthe SVD. Then the w: are Av; /o; whch isthe unit ecto w:/o;. ‘The SVD of this A with orthogonal columns is A = UEV? = (AZ~)(5)(1) 12 Since A? = A we have of = Xf and o? = 33. But dais negative, 0 01 = 3 and 2 = 2. The unit eigenvectors of A are the same w, = v, as for ATA = AAT and ft = —v; (aotice the sgn change Uecause op =~, atin Problem 4) 13 Suppose the SVD of R is R= UEVT. Then multiply by Q to get A = OR. Sothe SWDof his Ais (QU)EV*. (Onthogoaal Q times onhogonal U'= ontogonal QU.) 44 Te sulle change in A isto so its smallest singular value c2 to zero See #7 18 The singular valves of A + J are not aj + 1. “They come fom eigenvalues of (At D441). 16 This simulates the random walk sed by Google on billions of sie to salve Ap = p. ts like he power method of Section 93 excep ta follows the Hinks in one walk” ‘where the vector py, = A* po averages over all walks. UEV" = [cosines including ws) dag(sart(2 ~ V3,2.2 + V3) sine mai" ‘AV = UBsays tat difeences of snes nV are cosines in U times o's “The SVD ofthe derivative on [0.2] with f(0) = Ohas w = sinnx, 0 = no = connx! 7