Definition and Propert; s of Householder Mats se DEFINITION aN af D PROPERTIE "HOUSEHOLDER MATRICES “ saat, we define unit vectors in IR pefinition 10-1 A vector x € IR” is called a unit vector if Irlle =1 « sfx lies on the unit sphere Sin IR defined by So={2ER": flap =1) Nest, we define Householder matrices, pefinition 10.2 A Householder matriz is a matrix of the form H=I-2ww™ shee w is a unit vector in IR", The next theorem describes the important algebraic and geometric properties of House- iolder matrices. Theorem 10.1 Let H be a Householder matrix defined by H=I-2ww™ vhere w is a unit vector in IR". Let S be the subspace of IR defined by S =span{w} Since R" = S@S+ (by Theorem 1.36), we know that each vector x € IR” can be written uniquely as Z=2g +n, where zgE€S and zy € St Then: () fz €S, then Hx =—2 (b) fre S+, then Hx =2 (¢) f2=25 +2y where rs € S and ty € S+, then Hr=~-2s3+zn (4) H is an involution, ie. H? =I DaHouseholder Matrices and Their Ab (©) H vas only egenvntves 2146 yy 2 (td) i vi =-lof His (8) The cigenspace corresponding to the eigenvalue At ol Ey, =N(H- MN)=S and the eigenspace corresponding to the eigenvalue Az = 1 of H is Ey, = M(H 2) = st (g) det(H) = -1 (h) H preserves the length (2-norm) of vectors in R" allo = llelle Vz € R” Le (i) H is symmetric and orthogonal wi roof: (a) fz €S, then 2 = aw for some a € R. Then Her = (I —2ww™)x = 2 — 2ww? (aw) = 2 — 2eew||wl|3 Since w is a unit vector, we have Hr=2-2aw=2-25= (b) fz €S, then z-w=0, and Ha = (I -2ww")2 = 2 — 2w(w?z) 2w(0) =2-0=2 (c) fz=z5 +2y where zs € S and zy € S+, then using (a) and (b), we have Hz =Hzs+Hty =-ts+0N (a) Let 2 be any vector in IR". Then x can be expressed uniquely as T=25+zy where z5€S and ry €S*+ Using (c), we have Hr= —Is tin and Be Hc = H(H2) = H(-rg+2y)=25+2y =2 This shows that H=IDefinition and Properti, on ies of Householder Matrices ae pi js any eigenvalue of H, then He =e for some nonzero vector x. Since H is an invelutior ny we have © = Tx = He = H(He) = H(\2) =e o (?-1)2=0 since t #0, we must have N= This shows that the only eigenvalues of H are £1 et be any vector in IR". Then « can be expressed uniquely as ph T=2s+a2n where ts€S and zy € S+ ‘Then using (a) and (b), we have (H- Mle = (H+ Dz = He +2 = (—25 + ay) + (25 + ty) = 2N Therefore, ceEN(H-dil) => ty=0 = cES This shows that Ey, =N(H-Ml)=S Similarly, using (a) and (b), we have (H = dol)x = (H = Ix = Hx — 2 = (-25 + ay) — (ts +2N) = 225 Therefore, ce@N(H—dsl) => tg =0 <> 2ESt This shows that Ey, = N(H - dal) = S* (2) From (f), it follows that the eigenvalue \; = —1 has geometric (and algebraic) multi- plicity 1 and the eigenvalue \2 = 1 has geometric (and algebraic) multiplicity n — 1. Therefore, det(H) =-1-1-1--1=-1 (t) Let x be any vector in IR". Then x can be expressed uniquely as g=2g+xn where ts€S and rye St ~ Then lal = esll3 + lew Also, Hal} = | -2s + 23 = lies 1+ lew Thus, it follows that ||H2\l2 = lltll2-396 Hous () From (i), it follows thot ry eats, eR Hence, we must have wet a5 nal, From (d), we have H? = Since the inverse which shows that H is orthogo 80 Of Hig ‘unique, we must have z HT = which shows that H is symmetric. ‘Alternatively, note that r HT =(1—2uw? = 1? ~ uu = 1-200" =F This completes the proof. Remark 10.1 Using Theorem 10.1, we can give a geometric interpretation for the Hoy, holder transformation Hx =(1—2ww")x, where |ftll2=1 We know that each vector x € IR” can be expressed uniquely as im r=as+z2n, where ts€S and an eS By Theorem 10.1 (c), we have Ha=-ts+2n which shows that Hz is just the reflection of the vector with respect to the (n—1) dimensional subspace S+ of IR". Because of this geometric property, the Householder matrix H is alsy called as an elementary reflector. . Given two different unit vectors u,v € IR", the next theorem describes how to determine a Householder matrix H such that Hu=v This theorem is very useful in applications. Theorem 10.2 Let u and v be two different unit vectors in IR". Let H be the Householder matrix defined by H=1-2%w™ where we u=ll2 Then Hu =v Proof: First, note that 7 _ (u-v)(uT —v?) = (ua v)uT ~ (w= v)o™ ual} (hulle + Tells = 2(u- v) Since u and v are unit vectors, ull =1 and |jullz = 1. Therefore, Eq, (10.1) reduces to ww’ (10.1) ww? = Ras {(u=o)u? — (uw v7]Definition and Property 0. = — “S of Houscholder Matrices 397 therefore, we have ~ Hu=u-2u 1 wud Bs Tory (= vn — (w= vy} (10.2) gince wis a unit vector and yy = yoy, wv, Eq. (10.2) reduces to Hum goin WSU TT luv) = (w= oy(u-v)] “ Hu=u-—1_ “S Trang td uv) su- (uv) =0 This completes the proof. Theorem 10.2 has the following important corollary: corollary 10-1 Let z and y be two nonzero vectors in IR" such that 2 # y. Define u and v by ° x us=—_ ang ypa_¥ Te °° Typ Let H be the Householder matrix defined by H=I-2ww™ where eo llu—vff2 Then . He=py aad ll |r \}2 pa Hele Tulle Proof: By Theorem 10.2, Hu =v. Therefore, f Ha = H( lat} = [sotalol = Hy = ay This completes the proof. Using Corollary 10.1, we can easily create zeros in a vector. In applications, we are often given a vector x € IR" and we need to find an orthogonal matrix H such that This can be easily achieved using Corollary 10.1. We define use and v= —sign(z1)er (elleIder Matri Househo! and Their APD cata, ai y 398 oer wif m1 20 where signer) = { -1 if <0 and c; = [1,0,0,...,0]7. We also define + sign(x1) [la llae lela ic yr in the first cor Note that b struction, we have avoided the cancellation erro! mponent oe Note that by const : Setting it follows immediately from Corollary 10.1 that the Householder matrix H defined vig BS J —2ww7 is such that —sign(cy) [ell Hx = yey, where of introducing zeros in a given vector x € IR" (Wilkinson 19g This is a numerically stable way ee We summarize the above calculations in t] Algorithm 10.1 (Introducing Zeros in a Vector using Householder Matrices) Given Vector = € R®, the following algorithm finds a Householder matrix H=I-2ww7, where |wll2 such that sign(1)||xl2 Hx=ey, where p= Step: Define u,v € R" by Step 2: Define u-v Tel Step 3: The required Householder matrix H is defined by H=I-2ww™ Next, we illustrate Algorithm 10.1 by some examples. EXAMPLE 10.1 Let \of 0.1 to find go Agito 1010 Find 9 Housholder mati wwe so that Hr= t 0 0, wedfne 0.5443 0.6804 oa and v= =sign(x))ey = ~0.2722 eae by : 08787 wats |: 0382 lu=vfe 0.2323 0.1549 speed Houser matrix His dened by 0.5443 -0.6804 -0.4082 0.2722 0.6804 0.7002 -0.1799 0.1199 H=I-2ww" = 0.4082 -0.1799 0.8921 0.0719 0.2722 oxt99 orig 0.950] >) 7 Note that STUTE OF Not 13485 vag wet! ON Ha =| 0000 : \ pat JF Ae we. 128388 ai ign(11) fa = ~7.3485 Ths, Ho = pe FXAMPLE 10.2 Let 5 2 3-5 3 ope 2g Amy ig. dy tg 1 13-2 4 Supise that we need to find a Householder matrix Hf such that ee dete HAR sa Orit400 This can be easily achieved by taking 5 3 r=| 3 1 atrix H so that Hz is and applying Algorithm 10.1 to find a Householder m: © multiple of Define wand v by 4. 0.5025 A xz _ |] 0.3015 = 4 “= | 0.8040 0 0.1005 We define w by —0.8668 u-v _| 0.1739 =n | 0.4638 0.0580 The required Householder matrix H is defined by 0.5025, 0.3015 0.8040 0.1005 . 0.3015 0.9395 —0.1613 0.0202 Bd 2am 0.8040 -0.1613 0.5697 —0.0538 0.1005 0.0202 0.0538 0.9933 Note that = 9.9499 —4.2212 0.5025 6.4322 0.0000 0.2484 2.5012 6.7059. 0.0000 —0.6709 3.3365 —5.1176 0.0000 = 3.4161 —1.8329 3.2353 HA= which has the desired form! Next, we show that there is no need to calculate all the entries of a Householder matrix H defined by H=I-2ww", where |jwll2=1 to perform a matrix-vector product Hz or matrix products of the form HA or AH. We can efficiently calculate these matrix products just by using the unit vector w defining the Householder matrix H, as described in the following results: Theorem 10.3 Let H be a Householder matrix defined by H=I-2ww™ where w is a unit vector in IR". Let x be any vector in IR. Then Hz=2-aw, where a= Aw-2)finition and Propert}, yo) a prites of Hetinchbldor Matstone 401 g ——Stseholder Matrices AOE Note that Hz = (I~ 2QwwT), — T ~ QwyT. + Qow? s = 2 — dw(w.2) = 2—ow aig completes the proof. Using Theorem 10.3, we can easily calcu! ‘ate matrix products of the form HA or AH. corollary 10.2 Let Hbea Householder matrix defined by H=1~2QuwyT see # Re unit vector in R". Let be any n x p matrix defined by A= 41, 42,...,a5] HA = |Hay, Haz,..., Hay) Hay = ax -—oxw with ay = 2(w - ax) Corollary 10.3 Let H be a Householder matrix defined by H=I-2wu™ shee w isa unit vector in IR". Let A be any m x n matrix defined by rT A=|" = nH rH AH=|"”. mH 5 oat TRH = rp —04w™ with ag = 2(w-rp) Prof: By Corollary 10.2, we have (eH)? = Hf = rf — oxw where a = 2(w - rf). Therefore, . o eH = (rf ~ ant) = re — ax i sles. Next, we illustrate the above results with some examples.bc) Householder Matrices and T EXAMPLE 10.3 Let H be a Houscholder matrix defined by 08 ~0.2 H=1-2uw™, where w=] 94 04 (Note that {I1v[]2 = 1) We find Hx and Hy, where ET) 4 -5 z=| 2] and y=| 73 -9 8 By Theorem 10.3, we have Hx=xr-aw and Hy=y- fw where a@=2Aw-z) and B=2%Aw-y) We find that -16.4 and 6 =124 Therefore, 12.1200, 5.9200 1.7200 _y- — | -2.5200 He=24+16aw= | g4aqg | and Hy=y—1dw= | “Toe 2.4400 3.0400 EXAMPLE 10.4 Let H be the Householder matrix defined by | Tis | 3 H=1-2wu", where w= |“ | VE 2 Ve (Note that {lull = 1) a | Consider the matrices A and B defined by -8 1 65 9. 3 -19 4 17 = 11-4 14 17 23 19 —20 17 A=| 9 -5 uu -16| md B=] "3 9 8-4 1 15 M28 18-19 We illustrate how to find the matrix products HA and BH without explicitly forming | Householder matrix H.lary 10.2, we have py corel HA here Note that (10.3), we get using —9.4348 18.1739 a x=} 142609 | * 702 = 5.1304 Therefore, HA= (Haj, Haz, Haz, Haa] = Next, we calculate BH. Set where nm 2 ay.09 162, 43,04) where i 5 ei : 4 5 [eas 8) gee ppd <4 1 = (Ha, Hag, Has, Had] and a4 = ~16 15 Hax = ax —axw with Oy = 2(w + ay) (10.3) ay = 9.7312, a2 = —2.0642, a3 = —5.6028 and a4 = —32.4372 1.3043 5.8261 13.7826 5.5217 | 1, _ | 9.8696 _ | 6.9130 —3.7826 | 777° | 14.3043 |? 3.1304 —3.3913 2.6522 24.5652 —9.4348 1.3043 5.8261 13.7826 18.1739 _—5.5217 9.8696 —6.9130 14.2609 -3.7826 14.3043 3.1304 5.1304 -3.3013 2.6522 24.5652 1 B=|" v3 4 = [3 -19 4 17] = [23 19 -20 17] = [8 -9 11 -15] r3 ra By Corollary 10.3, we have = [14 28 18 -19] nH ro r3H r4H BH=and Their 4, PPlica i where r T wi ou rH = re —anw™ with ax = 2(w- re) A simple calculation shows that mH (34348 13.1739 -21.7391 4.1304) rH { 28.1304 6.6522 0.5217 27.2609] rH = | 5.0870 5.5652 -0.6522 - 20.8261} ra} [18.0000 8.0000 34.0000 ~11.0000] Therefore, we have nH =3.4348 13.1739 -21.7391 4.1304 rH 28.1304 -6.6522 0.5217 27.2609 BH=)\7| =| 5.0870 5.5652 —0.6522 —20.8261 rH 18.0000 8.0000 34.0000 ~11.0000 10.2 HOUSEHOLDER QR FACTORIZATION As an application of Householder matrices, we discuss the QR factorization of real matrices, this section. The Householder QF factorization of matrices has many applications in Numeq Linear Algebra like the linear least-squares theory (both overdetermined and underdetemning, | cases), computation of eigenvalues of matrices, and computation of singular values of Matrices etc. 7 First, we prove the Householder QR factorization theorem. Theorem 10.4 (Householder QR Factorization Theorem) If A is any real n x n matrix | there exist an orthogonal matrix Q and an upper triangular matrix R such that :| A=QR where the matrix Q can be expressed as a product of Householder matrices. Proof: We give a constructive proof consisting of (n — 1) steps. Step 1: Find a Householder matrix Hy such that the matrix A=MA has zeros below the first diagonal entry in its first column, i.e. Ay has the form * a Ape HA =|" Oke & ‘This can be casily achieved as follows: Using Algorithm 10.1, we can find a Householder mat H}=1—-2uywy, where |jwill2 =Householder QR. Factor; 10. oh 405 ah that git ity iP H, || _ Jo H, Any 0. sin the a en ne 3t08 below the first diagonal entry in its first column. gore Av = 4 eye 1- Because Ay is stored over ‘A, we can write Ay as 11 ay O1n A=] 2% om 0 ang ann sep 2: Find a Householder matrix H such that the matrix As = HA = (Hp H,)A jas zeros below the first and second diagonal entries in its first two columns, ie. Ag has the bss Pars tok Aa = HeAy = * * (10.4) eee * a 00 « * This can be easily achieved as follows: First, using Algorithm 10.1, we can find a Householder Hy = In-1 — 2intiT, where ||tall2 such that on ¥ ~ | @32 0 fn). |= An2 9. Next, we define Hz by ho Ha= Ie A Setting 0 e = eR’ m-[5] We see that |jwella = |li2ll2 = and oO) Lu, 0 & 9 2 qh ‘ it = [ Hi | T—2wewt = [‘ Tyat — 208 Ons406 Householder Matrices and Thej Applic ooh Peay Thus. Hy is a Honscholder matrix. Note that by comtretton Bh wrvere the st Ay — Hy Ay has zeros bele 6 Recond created in Step 1, and also the product Az = HaAy hs zaren ofl fle ton in its second cohinm, i.e. Ag has the form (10.4). Store Ane rot ’, ix H, such that the matrix PEO ay, ly ont ity, Step ¢ Pind a Householder matr Hhdey =o = Ulla Hath A has zeros below the first # diagonal entries in its first # column, mein be CaBIlY eh, as follows. First, using Algorithm 10.1, we can find a Household 1 sh -gety — 20007, where fts[}a = 1 A, such that an * | Gite 0 Al. |=]: ay Jobo Next, we define Hj; by Big he [ 0 a] Setting n-[2]er we see that [}willa = [leslla = and t= [5 45° amar] =['s BJ =m 0 In-Gn1y — 2007" 0 i : Thus, H; is a Householder matrix. Note that by construction, H; preserves the zeros already created in the first i — 1 steps, and also the product A; = H;Ai—1 has zeros below the i diagonal entry in its i column. Store A; over A, ie. A — Aj. It is clear that the matrix A,_1 = Hn—1An—2 obtained at the end of Step (n — 1) is an upper triangular matrix R. Thus, we have R= Anat = Hy-tAn-2 = HA (105) = Hy-1Hp-2° Define Q by Q= HH Hy Since each Householder matrix H; is symmetric, we have QP = Wes Hypg HEH? = Hy y Hy HoH (10.6) From (10.5) and (10.6), we have R=QTAouseolder QR Fa ee ea we Household i ij sce ent Honscholder mattis 1, is orthogonal, (is orthogonal. ‘This completes the oA go illustrate Theore oh gels " sate m 10.4 by an example _gMPiE 108 Let Bs 2410-1 As| 7015 8 17 9-7 5 “6 8 19 ~15 sus example, ve use Theorem 104 to fi ; 6 walt matrix R so that, ind an orthogonal matrix Q and an uP is A=QR he process consists of n — 1 = 3 steps. sees Hee, we find Hi that baa A=mAs|o*** Ones Orne ting Algorithm 10.1, iti easy to see that Hy is defined by Hy =1,-2wywp shee 0.9032 0.2914 =) o2622 0.1748 Thelore, 19.0000 -6.0000 7.2105 8.7895 A= maa 0.0000 21.4516 8.8998 10.6163 0,000 10.1935 -7.8098 10.7453 0.0000 11.8710 19.5399 —18.8302 Step2; Here, we find Hz so that ‘Ag = HyA has the form 19,0000 6,000 7.2105 8.7805] [* * * * 0.0000 214516 8.8008 10.6103] _] 0 + * * Ar=HoAr=H2| — gyoq99 10.1935 7.8098 10.7453 | | 0 0 * * 0.9000 118710 19.5399 -188302 | [0 0% * First, we find Hy so that 21.4516 * Fy | 10.1935 | = | 0 nsno} [0405. fined bY is de a Using Algorithm 10.1, it Raut f 0. 9508 where eal ae kag 0.2351 ‘Thus, Ho Is defined by "8 Ih=| 0 Wa sie, Hy = Ta — 2uote where “tale wa | tie Therefore, we have soo San rai0s aos 1000 ~26.5518 12.9280 —4.2836 A= HaAr=| 0.0000 9 12.4450 7.5813 0.0000 914.1420 -22.5149 so that Az = HaAa has the form Step 3: Here, we find Hs 8.7895 ee ~19,0000 —6.0000 7.2105 : Popo 26.8618 —12.9280 ~4.2856 | _ O+ae As=HsAz= Hs} 9,000 9 —12.4450 7.5813 oon 0 14.1420 22.5149 000- 0.0000 First, we find Hy so that a) 24] =] * 3] 14.1420} | 0 Using Algorithm 10.1, it is easy to see that Fy is defined by Hy = In — 2307 _ [ -0.9112 ¢ 0.4119 [oz 0 Hy uy = [3] eR! where = Thus, Hg is defined by Hs u ie. Hy = Iq —2uyw, whereerates we Have ~19.0000 6.0000 7.2105 8.7895 = Hyd = 0.0000 26.5518 —12.9280 —4.2836 a 0000 0 18.8381 ~21,9106 0000 0 0.0000 +9.1826 ane 0.6316 —0.3846 0.5087 —0.4410 Q=MyHoHy = | 05263 -0.6839 -0.2461 -0.4413 70.4737 0.4956 -0.5304 0.4988 0.3158 —0.3727 0.6320 (0.6017 ethat the MATLAB command ‘[Q, R = qr(A)’ gives us the same matrices Q and R that Not ened in this ext : “ obtained in this example using Householder QR factorization method. ‘ee have Nest, we discuss the uniqueness of QR factorization for square matrices. For this, we first tablish the following lemma: tabs 210.1 Let P be areal n x n matrix. If P is both upper triangular and orthogonal, has the form then P #10 0 Oo +1 40 0 0 P=|}0 0 41 0 oo oe HS je. P isa diagonal matrix such that P? = J. Prof: We prove this lemma by induction. If n = 1, the assertion holds trivially. If m = 2, let Eley elo 4 . Therefore, Since P is orthogonal, we have P?P = [0.3] ,cy=0 and +2 =1 (10.7) zy zy |e prey Thus, we must have Since «? = 1, ¢ is nonzero. Hence, ny=0 = y=0 Thus, the equations in (10.7) simplify to a? =1,y=0, and y?and Their Appticag, a Thus, P is a diagonal matrix such that ph es las 0 v it ‘| Br Q(Q%a)=0 + a=0 Thus, the equations in (10.8) reduce to QQ=h, a=0 and 2? Therefore, we see that P has the form =/[2 0 fa le +1 where @ is a k x k orthogonal matrix. Since P is upper triangular, we know that Q is alo an upper triangular matrix. Hence, by the induction hypothesis, we know that Q is a kx diagonal matrix with Q? = I,. Hence, it follows that P is a diagonal matrix with a/[@ 0) [fh 0] _ r=[9 1}= [Lo a) > This shows that the induction hypothesis also holds for the case when n = k +1. Hence, by induction, the assertion holds for any n x n real matrix that is both upper triangular and orthogonal. This completes the proof. Using Lemma 10.1, we can easily establish the following theorem that describes the unique ness of QR factorizations of nonsingular matrices. Theorem 10.5 Let A be a real nonsingular matrix, and let A=OQR (109) where @ is orthogonal and 2 is upper triangular. If we restaiet to have positive dego™! entries, then Q and 2 are uniquely determined by (10,9). aau Factorization suppose that a _ ae A = Qh = and Qz are orthogonal, and 7 a Ok: Gon a Ry and Ry are Upper triangular with positive diagonal a whe oti 4 jg nonsingular, and Rim QTA and mm ata mediate that Ry and Re are both nons 1 be a matrix defined by” MMSINEUlar matrices, P= Qa, 10), we have (40.11) es P= QFQ1 = Roky? From (10.11), it is immediate that D is both orth i i iti a }ogonal and upper triangular with positive diagonal entries, = Lemma 10.1, we know that Dis a diagonal matrix such that ince D has positive diagonal entries, it follows that D = I. Hence, (10.11) becomes pel si om (10. = Q3Q1 = RoRz! hich shows that Q2=Q: and Re = R, This completes the proof. It is easy to see that Theorem 10.4 (Householder QR Factorization Theorem) for real square matrices can be easily generalized to obtain QR factorizations of rectangular matrices. Epplictly, we have the following theorem: Theorem 10.6 (Householder QR Factorization of a Rectangular Matriz) If A is any real mxnmatrix with m > n, then A can be expressed as A=QR=@ [3] o Moreover, where Q is an m x m orthogonal matrix and f is an n x n upper triangular matrix the matrix @ can be expressed a product of Householder matrices. (If A is an m x n matrix with m n and § € IR", Suppose that A bas full ak Suppose also that the QF factorization of A obtained by Householder OR factorization mettod (Theorem 10.6) is A=an=a[] (1035) Then « is the unique least-squares solution for the linear system (10.14) if and only if z isthe unique solution of the linear system Re (10.16) where (10.7) Jpouscholder QR Factorization 102 sirst, we establish that the matrix 2 is nonsingul i ivalent to showing that pot Tei hs full rank. Note that. iil Kea a m Rr =0= Q"(Az) =0 > Ar=02= nave used the fact that the matrices Q and A have full rank. Since Rx oar=9, ewe whet amediate that W(R) = {0} so that nullity(R) = 0. By the Rank-Nullity Theorem ore 1.42), it then follows that ( rank(R) =n — nullity(R) =n —0=n ences jis ann x m nonsingular matrix, eat, we know that the 2-norm is orthogonally invariant. Hence, it follows that lir(=)1E3 = b— Az|l3 = 1Q7 (b= Az)]}3 = ]Q7 - (QT A)=IB Using (10-15) and (10.7), we have tren = 1 [J] i= te— Rote + td Thus, it is immediate that z is a least-squares solution of the system (10.14) if and only if ation of the linear system (10.16). Since Fis nonsingular, the least-squares solution risasol ea alin ) is unique and is given by the unique solution of the linear system (10.16). ofthe system (10.14) This completes the proof. 10.2 (Least-Squares Solution of a Full-Rank Overdetermined Linear System using Ider QR Factorization) Given a linear system Az = b, where A is a real m xn b € IR”, the following algorithm computes seholder QR Algorithm. the Househol matrix with m 2 7n and having full rank, and the unique least-squares solution 2* to the linear system Ax = b using the Hou foctorization of the matrix A. sup 1: Using the Householder QR factorization method, find the QR factorization of A: ee oath A=QR=Q [ a where ft is an n x n upper triangular matrix having full rank. Step 2: Set Qo where c € R" and d€ R™". Step 3: Solve the linear system = Re ='c (10.18) ing (Note that the MATLAB command ‘x = Rie the linear system (10.18) has a unique solution solution to the given linear system Az = b. using Gaussian elimination with partial pivoti Vields this solution). Since fis nonsingular, @. This is the required unique least-squares Next, we illustrate Algorithm 10.2 with some examples.sand Their App holder M: 416 House = b, where EXAMPLE 10.7 Consider the linear system Ax = b, ¥ ae e 3 5 3 -2 b= Meet ae and b 1 101-5 In Example 10.6, we showed that R a-ar-o[] where 0.4804 0.6189 0.2743-—(0.5576 6.2450 —1.7614 0.489, =| 708006 -0.5330 0.1937 0.1935 | 0.0000 -2.9829 47494 0.3203 0.5244 —0.4948 -0.6145 0.0000 0.0000 4.7174 0.1601 -0.2407 -0.8015 0.5235 Since R is nonsingular, A has full rank. We set -2.4019 —3.2751 e = = [i] = 7 | -s0oes —0.2162 By Theorem 10.7, we know that the unique least-squares solution of the linear system Az = bis obtained by solving the linear system Fx = ¢, ie. the system 6.2450 1.7614 0.4804] [ =2.4019 0 -2.9829 4.7451 | | zo | = | 3.2751 (1049) 0 o 4774 | | 23 —3.6685 Solving (10.19), we see that the unique least-squares solution of the linear system Az =b is given by 0.3640, z= | -0.1391 -0.7776 EXAMPLE 10.8 Consider the linear system Ax = b, where ogee i -8 if : d b= 34 -2.),end b= 6.2, .3 6 Using Householder QV factorization method, it is easy to obtain matrices Q and R80 tht A=an=a[8] a12 HO seholder QR Factorization M7 here i —0.5601 -0.5273 —0.6342 a 0.1400 0.7900 0.4917 0.4201 0.3016 —0.0153 0.8558 0.7001 —0.0829 -0.5965 0.3836 nd s all OM. yeas —o.se02 0 -11,1188 5.6202 _ 0 0 -6.9608 since ® is nonsingular, A has full rank. We set =1.1202 4.5621 r ce a *=[4] =| -6.9737 5.8566 By Theorem 10.7, we know that the unique least-squares solution of the linear system “Ar= bis obtained by solving the linear system Az = c, ie. the system 7.1414 1.5403 —0.9802 ] [ 2 —1.1202 0 -11.1188 5.6202 xq | =| 4.5621 (10.20) oO. 0 6.9608 a3 -6.9737 Solving (10.20), we see that the unique least-squares solution of the linear system Az = b is given by =0.2171 0.9167 1.0019 ‘As another application of Householder QR factorization of rectangular matrices, we solve the least-squares problem of full-rank underdetermined linear systems. We begin by noting that unlike the case of full-rank overdetermined linear systems, the full-rank underdetermined near systems always admit infinitely many least-squares solutions. This is an inherent feature of the least-squares theory for the underdetermined linear systems irrespective of the rank of the underlying coefficient matrices. As in many physical situations large solutions are not acceptable, we compute the minimum-norm least-squares solution for the underlying linear systems. b, where A is a real m x nm matrix with Theorem 10.8 Consider the linear system Ar m AT2 =042=0 ir where we have used the fact that the matrices Q” and A” have fleas Hence, 1 ay so that nullity(R) = 0. By the Rank-Nullity Theorem (Theorem 1. 2 ), We have, i rank(R) = m — nullity(R) = m Hence R is an m x m nonsingular matrix. The QR factorization of A? is wane Thus, . A=[RT 0] QT Hence, the linear system Az = is the same as [RP 0] QT2=b (1023) Thus, if we define =|" i [2] where y1 €IR™ and yp = R"-™, the system (10.23) reduces to Ry, =b (10.24) agp since the system (10.24) does not involve yy RT is anmxm nonsingular matrix, it is immi system Az = b has a family of least-squares 2 a8 an unknown, and since the coefficient matrix iediate that the full-rank, underdetermined line” solutions given by c=Qy=o/"] 9] (AT) 10.25) pee ofp] =0[ 2] ¢ 1 is immediate that the minimum-norm least-squares solution 2* to the linear Ax = b is obtained by setting yo = 0 in (10.25), i, #=9[%] =9[% 4] This completes the proof,