Linear Algebra

Lecture Notes -
QR
Factorization

Linear Algebra

Lecture Notes - QR Factorization

Trakya University, 2021

Linear Algebra

Lecture Notes -
Recall the Gram-Schmidt orthogonalization process:
QR
Factorization Let {x1 , x2 , . . . , xk } be a basis for a subspace S of Rn .
Let
1
u1 = x1
kx1 k
and define
1
ui = (xi − pi−1 ) for i = 2, . . . , k
kxi − pi−1 k

where

pi−1 = (uT1 xi )u1 + (uT2 xi )u2 + · · · + (uTi−1 xi )ui−1

is the projection of xi onto the subspace Span(u1 , u2 , . . . , ui−1 )

for i = 2, . . . , k. Then {u1 , u2 , . . . , uk } is an orthonormal basis
for the subspace

S = Span(x1 , x2 , . . . , xk ) = Span(u1 , u2 , . . . , uk )
Linear Algebra

Lecture Notes -
QR
Factorization
Let A = (aij ) be a k × n matrix
 
↑ ↑ ↑
A = (a1 , a2 , . . . , an ) =  a1 a2 ··· an  ∈ Rk×n
↓ ↓ ↓

where  
a1j
 a2j 
aj =   ∈ Rk for j = 1, 2, . . . n
 
..
 . 
akj
If the column vectors of A are linearly independent, then the
set {a1 , . . . , an } forms a basis for the column space of A. By
means of the Gram-Schmidt process we can find an
orthonormal basis {q1 , . . . , qn } for C(A) by defining
Linear Algebra
1
q1 = a1
Lecture Notes -
QR
ka1 k
Factorization
and
1
qi = (ai − pi−1 ) for i = 2, . . . , n
kai − pi−1 k
where

pi−1 = (qT1 ai )q1 + (qT2 ai )q2 + · · · + (qTi−1 ai )qi−1

Therefore we write

a1 = ka1 kq1
ai = kai − pi−1 kqi + pi−1 for i = 2, . . . , n

Substituting pi−1 s into here we get

a1 = ka1 kq1
ai = (qT1 ai ) q1 + (qT2 ai ) q2 + · · · + (qTi−1 ai ) qi−1 + kai − pi−1 k qi
| {z } | {z } | {z } | {z }
r1i r2i r(i−1)i rii

for i = 2, . . . , n
Linear Algebra

Lecture Notes -
QR
Factorization
Thus, we may write

a1 = r11 q1
a2 = r12 q1 + r22 q2
.. ..
. .
an = r1n q1 + r2n q2 + · · · + r(n−1)n qn−1 + rnn qn

where

rji = qTj ai for j = 1, . . . , i − 1 and i = 2, . . . , n (j < i)

and

r11 = ka1 k and rii = kai − pi−1 k for i = 2, . . . , n

Linear Algebra If we set
 
Lecture Notes - ↑ ↑ ↑
QR
Factorization Q = (q1 , q2 , . . . , qn ) =  q1 q2 ··· qn  ∈ Rk×n
↓ ↓ ↓
and define the n × n upper triangular matrix
 
r11 r12 · · · r1n
 0 r22 · · · r2n 
R= .
 
. . . .. 
 . . . 
0 0 ··· rnn
then we get
 
r11 r12 ··· r1n
 0 r22 ··· r2n 
QR = (q1 , q2 , . . . , qn ) 
 
.. .. .. 
 . . . 
0 0 · · · rnn
n
X
= (q1 r11 , q1 r12 + q2 r22 , . . . , qi rin )
i=1
= (a1 , a2 , . . . , an ) = A
Linear Algebra

Lecture Notes -
QR
Factorization Therefore, we have established the following:
Theorem. If A is an k × n matrix of rank n, then A can be
factored into a product QR:

A = QR

Here
Q is an k × n matrix with orthonormal column vectors and
R is an upper triangular n × n matrix whose diagonal entries
are all positive.
Note that since rii > 0 for all i = 1, 2, . . . , n

det R = r11 r22 · · · rnn > 0

and hence R is nonsingular.

Linear Algebra

Lecture Notes -
QR
Factorization
Example.
Compute the Gram-Schmidt QR factorization of the matrix
 
1 −2 −1
 2 0 1 
A= 2 −4

2 
4 0 0

Solution. First of all verify that the column vectors of A are

linearly independent, that is, rank(A) = 3 and apply the
Gram-Schmidt orthogonalization process to the column vectors
     
1 −2 −1
 2   0   1 
a1 = 
 2  , a2 =  −4  , a3 =  2 
    

4 0 0
Linear Algebra
Accordingly
Lecture Notes -
QR p √
Factorization
r11 = ka1 k = 12 + 22 + 22 + 42 = 25 = 5

and  
1
5
2
 
1  5

q1 = a1 =  2

5 
 5


4
5

q2 is defined to be
1
q2 = (a2 − p1 )
r22
where

r22 = ka2 − p1 k, p1 = r12 q1 , r12 = qT1 a2

So,
r12 = qT1 a2 = −2, p1 = −2q1
Linear Algebra
 T
Lecture Notes - 8 4 16 8
QR Since a2 − p1 = − , , − ,
Factorization 5 5 5 5
r
64 16 256 64
r22 = ka2 − p1 k = + + + =4
25 25 25 25
Thus  
− 25
 1 
1  5 
q2 = (a2 − p1 ) = 
 −4

4  5


2
5

Finally
1
q3 = (a3 − p2 )
r33
where

r33 = ka3 − p2 k, p2 = r13 q1 + r23 q2 , r13 = qT1 a3 , r23 = qT2 a3

Linear Algebra Direct computation shows that
Lecture Notes -
QR r13 = qT1 a3 = 1, r23 = qT2 a3 = −1
Factorization

and  
3
5
1
 
 5

p2 = q1 − q2 = 
 6


 5 
2
5
 T
8 4 4 2
Since a3 − p2 = − , , ,−
5 5 5 5
r
64 16 16 4
r33 = ka3 − p2 k = + + + =2
25 25 25 25
Thus  
− 45
2
 
1  5

q3 = (a3 − p2 ) =  2

2 
 5


− 15
Linear Algebra

Lecture Notes - Therefore we have

QR
Factorization  
1
5 − 25 − 45
2 1 2
 
 5 5 5

Q = (q1 , q2 , q3 ) =  2


 5 − 45 2
5


4 2
5 5 − 15

and    
r11 r12 r13 5 −2 1
R= 0 r22 r23  =  0 4 −1 
0 0 r33 0 0 2
and
   
1
1 −2 −1 5 − 25 − 45  
   2 1 2
 5 −2 1
 2 0 1   5 5 5

A= =  0 4 −1  = QR

2
 2 −4 2  

5 − 45 2
5

0 0 2
  
4 2
4 0 0 5 5 − 15