Matrix Inverse Computation of the Matrix Inverse

A square matrix S ∈ Rn×n is invertible if there exists a matrix We want to find the inverse of S ∈ Rn×n , that is we want to find a
S −1 ∈ Rn×n such that matrix X ∈ Rn×n such that SX = I.
S −1 S = I and SS −1 = I. I Let X:,j denote the jth column of X, i.e., X = (X:,1 , . . . , X:,n ).
Consider the matrix-matrix product SX. The jth column of SX is
The matrix S −1 is called the inverse of S.
the matrix-vector product SX:,j , i.e., SX = (SX:,1 , . . . , SX:,n ).
I An invertible matrix is also called non-singular. The jth column of the identity I is the jth unit vector
A matrix is called non-invertible or singular if it is not invertible. ej = (0, . . . , 0, 1, 0, . . . , 0)T .
I A matrix S ∈ Rn×n cannot have two different inverses. Hence SX = (SX:,1 , . . . , SX:,n ) = (e1 , . . . , en ) = I implies that we
In fact, if X, Y ∈ Rn×n are two matrices with XS = I and SY = I,
can compute the columns X:,1 , . . . , X:,n of the inverse of S by
then
X = XI = X(SY ) = (XS)Y = IY = Y. solving n systems of linear equations
I If S ∈ Rn×n is invertible, then Sx = f implies x = S −1 Sx = S −1 f , SX:,1 = e1 ,
i.e., for every f the linear system Sx = f has a solution x = S −1 f .
The linear system Sx = f cannot have more than one solution
..
.
because Sx = f and Sy = f imply
S(x − y) = Sx − Sy = f − f = 0 and x − y = S −1 0 = 0. SX:,n = en .
Hence if S is invertible, then for every f the linear system Sx = f
has the unique solution x = S −1 f . Note that if for every f the linear system Sx = f has a unique
I We will see later that if for every f the linear system Sx = f has a solution x, then there exists a unique X = (X:,1 , . . . , X:,n ) with
unique solution x, then S is invertible. SX = I.
M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 1 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 2

Want inverse of
LU-Decomposition
 
1 0 2
S = 2 −1 3 . Consider  
4 1 8 1 0 2
S = 2 −1 3 .
Use Gaussian Elimination to solve the systems 4 1 8
SX:,1 = e1 , SX:,2 = e2 , SX:,3 = e3 for the three columns of X = S −1 Express Gaussian Elimination using Matrix-Matrix-multiplications
   
1 0 2 1 0 0 1 0 2 1 0 0
    
1 0 0 1 0 2 1 0 2
 2 −1 3 0 1 0  →  0 −1 −1 −2 1 0   −2 1 0   2 −1 3  =  0 −1 −1 
4 1 8 0 0 1 0 1 0 −4 0 1 −4 0 1 4 1 8 0 1 0
  | {z }| {z } | {z }
1 0 2 1 0 0 =E1 =S =E1 S

→  0 −1 −1 −2 1 0 
    
1 0 0 1 0 2 1 0 2
0 0 −1 −6 1 1  0 1 0  0 −1 −1  =  0 −1 −1 
    0 1 1 0 1 0 0 0 −1
1 0 0 −11 2 2 1 0 0 −11 2 2 | {z }| {z } | {z }
→  0 −1 0 4 1 0  →  0 1 0 −4 −1 0 . =E2 =E1 S =E2 E1 S=U

0 0 −1 −6 1 1 0 0 1 6 −1 −1 Inverses of E1 and E2 can be easily computed:

     
−11 2 2 1 0 0 1 0 0
S −1 =  −4 −1 0 . E1−1 =  2 1 0  , E2−1 =  0 1 0 .
6 −1 −1 4 0 1 0 −1 1

M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 3 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 4
 If we have computed the LU decomposition
We have
E2 E1 S = U S = LU,

then we can use it to solve

Hence Sx = f.
E1 S = E2−1 U, and S= E1−1 E2−1 U. We replace S by LU ,
LU x = f,
      and introduce y = U x. This leads to the two linear systems
1 0 2 1 0 0 1 0 0 1 0 2
Ly = f and U x = y.
2 −1 3 =  2 1 0   0 1 0  0 −1 −1 
4 1 8 4 0 1 0 −1 1 0 0 −1 Since L is lower triangular and
can be easily solved.
U is upper triangular, these two systems
| {z } | {z }| {z }| {z } Example:
=S =E1−1 =E2−1 =U
      
   1 0 2 1 0 0 1 0 2 −4
1 0 0 1 0 2 2 −1 3 =  2 1 0  0 −1 −1 , f =  −6 
4 1 8 4 −1 1 0 0 −1 −15
= 2 1 0  0 −1 −1  | {z } | {z }| {z }
=S =L =U
4 −1 1 0 0 −1
| {z }| {z }         
1 0 0 y1 −4 y1 −4
=E1−1 E2−1 =L =U  2 1 0   y2  =  −6  ⇒  y2  =  2 
4 −1 1 y3 −15 y3 3
Hence we have the LU-Decomposition of S,
| {z } | {z }
=L =f

S = LU,
        
1 0 2 x1 −4 x1 2
 0 −1 −1   x2  =  2  ⇒  x2  =  1 
0 0 −1 x3 3 x3 −3
where L is a lower triangular matrix and U is an upper triangular matrix. | {z
=U
} | {z }
=y
In Matlab compute using [L,U]=lu(S).
M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 5 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 6

In Matlab the matrix inverse is computed using the LU decomposition. Execute the following:
Given S, we want to compute S −1 . Recall that the columns
n = input(’Problem size = ’);
X:,1 , . . . , X:,n of the inverse S −1 = X are the solutions of
S = rand(n,n);
f = rand(n,1);
SX:,1 = e1 ,
.. ntry = 50;
.
SX:,n = en . tic
for i = 1:ntry
If we have computed the LU decomposition x = S\f;
end
S = LU, toc

then we can use it to solve the n linear systems SX:,j = ej , j = 1, . . . n. tic

Use the LU decomposition to compute the inverse of for i = 1:ntry
Sinv = inv(S);

1 0 2
 
1 0 0

1 0 2
 x = Sinv*f;
2 −1 3 =  2 end
1 0   0 −1 −1 
toc
4 1 8 4 −1 1 0 0 −1
| {z } | {z }| {z }
=S =L =U tic
[L,U] =lu(S);
What do you think of the following approach to solve Sx = f : for i = 1:ntry
x = U\(L\f);
>> Sinv = inv(S); end
>> x = Sinv*f; toc

M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 7 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 8