You are on page 1of 19

4.3.

The QR Reduction
Reading Trefethen and Bau (1997), Lecture 7 The QR factorization of a matrix A 2 <m n is

{ Q 2 <m m is an orthogonal matrix { R 2 <m n is upper triangular { Assume (for the moment) m n
Motivation: { An alternate to Gaussian elimination when solving Ax = b More expensive than Gaussian elimination { Determining bases for orthogonal subspaces { Determining numerical rank through the SVD { Solving least squares problems min jjAx ; bjj2 x

A = QR

(1)

{ Solving algebraic eigenvalue problems Ax = x

1

n = 4 2 3 a11 a12 a13 a14 6 a21 a22 a23 a24 7 6 6 a31 a32 a33 a34 7 7 6 A=6a a a a 7 6 41 42 43 44 7 7 6 7 4 5 a51 a52 a53 a54 a61 a62 a63 a64 r12 a(1) 22 a(1) 32 a(1) 42 a(1) 52 a(1) 62 r13 a(1) 23 a(1) 33 a(1) 43 a(1) 53 a(1) 63 { Construct a re ection H1 such that 2 r11 6 0 6 6 6 0 H1 A = 6 0 6 6 6 0 4 0 r14 a(1) 7 24 7 (1) 7 a34 7 7 (1) 7 a44 7 a(1) 7 54 5 a(1) 64 3 2 .Householder QR Factorization Use Householder re ections to reduce successive columns of A to zero below their main diagonals { Suppose m = 6.

i = 3 : 6 i2 2 (1) 3 2 3 a 6 22 7 6 r22 7 6 a(1) 7 6 0 7 6 32 7 6 7 P2 6 a(1) 7 = 6 0 7 6 42 7 6 7 6 a(1) 7 4 0 5 4 52 5 0 a(1) 62 { Apply the transformation to get 1 0 H2 = 0 P 2 2 r11 6 0 6 6 6 H2H1A = 6 0 6 0 6 6 0 4 0 r12 r13 r22 r23 0 a(2) 33 0 a(2) 43 0 a(2) 53 0 a(2) a(2) 63 64 3 r14 r24 7 7 (2) 7 a34 7 7 (2) 7 a44 7 a(2) 7 54 5 3 .Householder QR Factorization Use a re ection to annihilate a(1).

Householder QR Factorization Repeat the process on the third column 2 r11 r12 r13 6 0 r22 r23 6 6 0 0 r33 H3H2H1A = 6 0 0 0 6 6 6 4 0 0 0 0 0 0 One more should do it r14 3 r24 7 7 r34 7 7 (2) 7 a44 7 7 a(2) 5 54 a(2) 64 3 r14 r24 7 7 r34 7 7 r44 7 7 7 0 5 0 2 r11 6 0 6 6 0 H4H3H2H1A = 6 0 6 6 6 0 4 0 r12 r13 r22 r23 0 r33 0 0 0 0 0 0 4 .

j)) A(j:m.house(A(j:m. for j = 1 : n v(j:m) = house(A(j:m.j:n). v(j:m)) if j < m A(j+1:m. On return.j:n) = row.j) = v(j+1:m) end end 5 . % the essential part of Q is stored in the lower triangular % part of A and R is stored in the upper triangular part.Householder QR Factorization Algorithm for a Householder QR factorization function A = houseqr(A) % houseqr: Overwrite A 2 <m n with the product % QR where Q 2 <m m is orthogonal and % R 2 <m n is upper triangular.

All components of v are divided by v(1) { The renormalization saves a storage location and allows v(j) to be stored in the lower part of A Upon completion of the algorithm 3 2 r11 r12 r13 r14 6 v(1) r22 r23 r24 7 7 6 2 6 (1) (2) 6 v3 v3 r33 r34 7 7 6 (1) (2) (3) (2c) A=6v v v r 7 7 44 6 4 4 4 (4) 7 6 v(1) v(2) v(3) v 7 4 5 5 5 5 5 (1) (2) (3) (4) v6 v6 v6 v6 6 .Householder QR Factorization As described in Example 2.2. 2 (v(j))T v(j) { v(j) is the Householder vector v used at step j { v(j) has the form ) ( v(j) = 0 0 ::: 0 1 vj(j+1 ::: vmj)]T (2a) (2b) Normalize the Householder vector so that vj(j ) = 1 { This requires modi cation of the function house of Section 4. v(j)(v(j))T Hj = I .4.

we compute ^ ^ QT (A + A) = R ^ { Q is exactly orthogonal ^ { R is upper triangular { The perturbation jj Ajj2 ujjAjj2 { This is acceptable 7 .Householder QR Factorization QT = HnHn.1 H1 so Q = H1H2 Hn { Q rarely needs explicit construction (3) The factorization requires about 2n2(m . n=3) FLOPs ( multiplications plus additions) { With m = n we require about 4n3=3 FLOPs { Gaussian elimination requires about 2n3=3 FLOPs With roundo .

Givens QR Factorization We can also use Givens rotations to construct the QR decomposition { The order of the reduction is { stands for a nonzero entry { The numbers indicate the rows that are rotated 2 3 2 3 2 3 6 7 7 26 7 6 7= 6 6 7= 6 7 A=6 7 36 7 360 7 4 5 4 5 4 5 4 0 0 2 3 2 3 2 3 1 7 7 260 7 260 6 7= 6 0 6 7= 6 A= 6 0 7 360 7 360 0 7 7 4 5 4 5 4 5 0 4 0 0 0 0 2 3 60 7 6 A= 3 6 0 0 7 7 4 5 4 0 0 0 8 .

Givens QR Factorization Algorithm for the Givens QR factorization function A = givensqr(A) % givensqr: Reduce A 2 <m n to upper triangular % form by the Givens QR procedure where Q 2 <m m % is orthogonal and R 2 <m n is upper triangular. A(i. n=3) FLOPs Re ections are generally preferred { Let Gi = G(i i .rot(A(i-1:i. % On return. 1 ).j). for j = 1:n for i = m: -1: j + 1 c.j)) A(i-1:i.j:n) = row. R is stored in the upper triangular part of A. n=3) multiplications and additions Re ections require 2n2(m . s) end end { The procedure does not save Q { The factorization requires about 3n2(m . | 1 st col: 2 nd col: n th col: (Gm} | {z ) 9 . c.s] = givens(A(i-1. then Q = (GmGm{z1 G2} (Gm {z G3} )| ) .j:n).

3 4 7 A = 6 5 1 .1:9762 { The Givens matrix corresponding to this rotation is G(3 4 )T = GT 4 2 3 2 1 0 0 0 1 0 0 60 1 0 07 60 1 = 6 0 0 c .1:0462 5 . Obtain the QR factorization of 2 3 3 2 1 6 2 .8:6023 6 4 0 s = 0:8137 2:0000 .0:0 6 7 6 5812 4 5 4 0 0 s c 0 0 0:8137 { We may check that GT A = A(1) 4 3 0 0 7 7 7 .3:8362 .0:5812 2 3:0000 6 2:0000 A(1) = 6 .0:8137 5 .s 7 = 6 0 0 .7) c = .3:0000 .1:5112 { Applying row.0:5812 10 .Givens QR Factorization Example 1.1 7 6 7 4 5 7 4 2 { Rotating the third and fourth rows using givens(5.rot to A(3 : 4 1 : 3) yields 3 1:0000 4:0000 7 7 7 .

Givens QR Factorization Rotate the second and third rows using givens(2.0:9740 0:2265 0 { We may check that GT GT A = A(2) 3 4 11 .1:9762 3 0 07 7 07 5 1 { The rotation matrix is 2 1 0 6 0 0:2265 GT = 6 0 0:9740 6 3 4 0 0 0 .6023) { Applying row.rot to A(1)(2 : 3 1 : 3) yields 2 3:0000 6 8:8318 (2) A =6 6 0 4 0 2:0000 3:0571 .1:5112 c = 0:2265 s = 0:9740 3 1:0000 1:9249 7 7 3:6592 7 5 . -8.0000.3:7908 .

rot to A(2)(1 : 2 1 : 3) yields 2 .3:5380 c = .9:3274 6 0 (3) A =6 6 0 4 0 .0000.1:5112 3 .3:7908 .0:3216 0 0 0 1 0 0 { The rst column has been reduced 12 .0:3216 s = 0:9469 0:9104 .0:9469 .0:3216 6 0:9469 GT = 6 6 2 0 4 0 .Givens QR Factorization Rotate the rst and second rows using givens(3. 8.1:9762 3 0 07 7 07 5 1 { The Givens matrix 2 .8318) { Applying row.2:1442 0:3278 7 7 3:6592 7 5 .

9104.0:3703 { Applying row.0:9760 0 7 GT = 6 0 0:9760 0:2177 0 7 6 7 3 4 5 0 0 0 1 { The second column is done 13 Givens QR Factorization .rot to A(4)(2 : 3 2 : 3) yields 3 2 .3:5380 .0:3703 7 7 4 5 0 0 0:3703 0:9289 Rotate the second and third rows using givens(0.Rotate the third and fourth rows using givens(-3.2:1442 6 0 4:1812 .9:3274 .rot to A(3)(3 : 4 2 : 3) yields 3 2 .9:3274 .3:1908 { The Givens matrix2 3 1 0 0 0 60 1 0 07 T =6 G4 6 0 0 0:9289 .5112) c = 0:9289 s = . -1.7908.3:1908 { The Givens matrix2 3 1 0 0 0 6 0 0:2177 .4:0809 2:6672 7 4 5 0 0 .2:5318 7 7 6 (5) A =6 0 0 0:9007 7 5 4 0 0 . -4.2:1442 6 0 0:9104 0:3278 7 6 7 (4) A =6 0 .3:5380 .0809) c = 0:2177 s = 0:9760 { Applying row.

2:5318 7 3:3154 7 5 0 { The Givens matrix 0 0 1 0 0 0:2717 0 0:9624 3 0 07 7 7 . -3.1908) { Applying row.rot to A(5)(3 : 4 3) yields 2 .9007.0:9624 5 0:2717 14 .The Last Rotation Rotate the third and fourth rows using givens(0.2:1442 7 .3:5380 c = 0:2717 s = 0:9624 4:1812 0 0 3 .9:3274 6 0 (6) A =6 6 0 4 0 2 1 60 GT = 6 0 6 4 4 0 .

0:2144 6 4 .0:3216 0:2062 6 .3:5380 4:1812 0 0 3 .0:7505 0:3216 { Check that QR = A { Check that QT Q = I nd col: 1 st col: 0:2511 0:3813 .9:3274 6 0 R=6 6 0 4 0 The orthogonal matrix { We have 3 .2:5318 7 3:3154 7 5 0 QT = (G4) (GT GT } (GT GT GT } 3 |{z} | 3{z 4 ) | 2 {z 4 ) rd col: 2 2 .0:8989 Q = 6 .0:8121 0:3635 3 0:8894 0:0232 7 7 0:0851 7 5 .2:1442 7 .0:2144 .0:4486 15 .The Factored Form The factored matrix 2 .0:5361 .

n = qn+1 qn+2 { Then range(A) = range(Qn) range(A)? = range(Qm.n 0 { Thus.n Proof: From (1) Rn = QnRn 0 qk g qk g (5) ak = Thus.Properties of the QR Factorization { A = a1 a2 an] etc. then spanfa1 a2 ak g = spanfq1 q2 qk g k = 1 : n (4a) qm ] (4b) (4c) (4d) The ? denotes the orthogonal complement { In addition A = QnRn Rn = R(1 : n 1 : n) Remark: R = Rn Q = Qn Qm.n) Theorem 1: Let A 2 <m n have rank n with m n. { Let Qn = q1 q2 qn]. Qm. A = QR = Qn Qm. spanfa1 k X j =1 rjk qj 2 spanfq1 q2 a2 ak g spanfq1 q2 16 . Let A = QR.

then the thin QR factorization (4c) is unique.1 hence. Qn is unique 2 n 17 AT A = RT QT QnRn = RT Rn n n n . dim(spanfa1 a2 ak g) = k So spanfa1 a2 ak g = spanfq1 q2 qk g Thus. (4a) is satis ed The rest of the Theorem follows directly 2 We call (4c) the thin or reduced QR factorization of A Theorem 2: Let A 2 <m n have rank n. { Rn is upper triangular with positive diagonal entries { Consider the Cholesky factorization of AT A = LLT (6a) L is lower triangular { Then Rn = LT (6b) Proof: Consider { Comparing this with (6a) yields (6b) { The Cholesky factorization is unique { From (4d) Qn = AR.Fat and Skinny QRs But rank(A) = n thus.

satisfy i = i 2 i=1:n (7a) { The eigenvectors xi of AT A and the singular vectors vi of A satisfy xi = vi i = 1 : n (7b) { The Euclidean norm q jjAjj2 = (AT A) = 1 (7c) The spectral radius (AT A) is the largest eigenvalue T 1 of A A 1 is the largest singular value of A { The condition number of A in the Euclidean norm satis es (A) (7d) (A) = 1 2 (A) n 18 . of AT A and the singular values i. i = 1 : n. i = 1 : n.The Condition Number of a Rectangular Matrix What is the condition number of an m n matrix? Theorem 3: Let A 2 <n n. then { The eigenvalues i.

SIAM J. 14. Stewart (1993). Matrix.The Condition Number of a Rectangular Matrix Proof: We can use this as a de nition of the condition number for a rectangular matrix { An O( ) perturbation in A introduces O( 2(A)) perturbations in Qn and Rn cf.. On the perturbation of LU Cholesky and QR factorizations. Anal. 1141-1145 19 .