MATRIX ALGEBRA

MATRIX ALGEBRA 1 / 19
1 Matrix

2 Inversion Algorithm

MATRIX ALGEBRA 2 / 19
 Matrix

Outlines

1 Matrix

2 Inversion Algorithm

MATRIX ALGEBRA 3 / 19
 Matrix

Matrix
Consider the set Rm×n of all matrices of size m × n (m rows, n
columns). Let
 
a11 a12 ··· a1n
 a21 a22 ··· a2n 
A =  .. .. ..  ≡ [aij ]
 
 . . . 
am1 am2 ··· amn
 
b11 b12 ··· b1n
 b21 b22 ··· b2n 
B =  .. .. ..  ≡ [bij ] ∈ Rm×n
 
 . . . 
bm1 bm2 · · · bmn

in which aij ( bij , resp.) is called the element in the ith row, jth
column of A (B, resp.), i = 1, 2, · · · , m , j = 1, 2, · · · , n .
MATRIX ALGEBRA 4 / 19
 Matrix

The Operations on Matrices in Rm×n

Let A = [aij ] , B = [bij ] ∈ Rm×n . We define
The relation “equality” : A = B if aij = bij , for all , .
The algebraic operations : A + B = [aij + bij ] , and αA = [αaij ] ,
for all α ∈ R .
Multiplying Matrices
Let A = [aik ] ∈ Rm×n be a matrix of size m × n and let
B = [bkj ] ∈ Rn×p of size n × p . The product
C = [cij ] = AB ∈ Rm×p , of size m × p , is defined by
n
X
cij = ai1 b1j + ai2 b2j + · · · + ain bnj = aik bkj
k=1

for i = 1, 2, · · · , m , j = 1, 2, · · · , p .

MATRIX ALGEBRA 5 / 19
 Matrix
 
  b1j
 b2j 
  ..
 
   .

row i → 
 ai1 ai2 · · · ain   =



 bnj
↑
  column j

 ai1 bij + ai2 b2j + · · · + ain bnj  ← row i

↑
column j
n
 a vector u = (u1 , u2 , · · · , un ) ∈ R as a
Moreover, by identifying
u1
 u2 
column matrix, u =  ..  ∈ Rn×1
 
 . 
un
MATRIX ALGEBRA 6 / 19
 Matrix

we can define the product of a matrix with a vector, Au ∈ Rm×1 , as

  
a11 a12 · · · a1n u1
 a21 a22 · · · a2n   u2 
Au =  .. .. ..   .. 
  
 . . .   . 
am1 am2 · · · amn un
 
a11 u1 + a12 u2 + · · · + a1n un
 a21 u1 + a22 u2 + · · · + a2n un 
= ..  ∈ Rm×1
 
 . 
am1 u1 + am2 u2 + · · · + amn un
and then, the system of linear equations


 a11 x1 + a12 x2 + ··· + a1n xn = b1
 a21 x1 + a22 x2 + ··· + a2n xn = b2

.. .. .. ..


 . . . .
 a x + a x
m1 1 m2 2 + ··· + amn xn = bm
MATRIX ALGEBRA 7 / 19
 Matrix

can be written as a matrix equation Ax = b , with the matrix of

coefficients
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A =  .. .. ..  ∈ Rm×n
 
 . . . 
am1 am2 · · · amn

the vector of unknowns  

x1
 x2 
x= ..
 

 . 
xn

MATRIX ALGEBRA 8 / 19
 Matrix

and the vector of the right-hand side coefficients,

 
b1
 b2 
b =  .. 
 
 . 
bn

Furthermore, using the column vectors of A,

 
a1j
 a2j 
cj =  ..  , j = 1, 2, · · · , n
 
 . 
amj

the matrix by vector product (1) can be rewritten as

Au = u1 c1 + u2 c2 + · · · + un cn
MATRIX ALGEBRA 9 / 19
 Matrix

Remark
It is noted that the matrix multiplication is (i) associative,
A (BC ) = (AB) C , α (AB) = (αA) B = A (αB) , for all α ∈ R , (ii)
distributive with respect to the addition, A (B + C ) = AB + AC ,
(B + C ) A = BA + CA , (α + β) A = αA + βA , and
α (A + B) = αA + αB , for all α, β ∈ R , but it is NOT
commutative, i.e., AB 6= BA in general. For example, with
   
0 1 0 0
A= ,B =
0 0 1 0
   
1 0 0 0
we have AB = 6= BA = . Whenever AB = BA ,
0 0 0 1
we say that A and B are commutative.
For example, every square matrices A ∈ Rn×n are commutative with
zero matrix, , and with the identity matrix of size n, In , because
A0 = 0A = 0 and AIn = In A = A .
MATRIX ALGEBRA 10 / 19
 Matrix

Definition 1.
Let A, B ∈ Rn×n . We say that B is the inverse matrix of A, noted
B = A−1 , when AB = BA = In . The two matrices are called
invertible.
By definition, A is also the inverse matrix of B, A = B −1 , and then,
−1
(A−1 ) = A.
Example 1.  
1 −1 1
Consider the two matrices, A =  −1 2 1  and
  −2 3 1  
−1 4 −3 1 0 0
B =  −1 3 −2  . We have AB = BA =  0 1 0  = I3
1 −1 1 0 0 1
−1 −1
Hence, A = B and B = A .

MATRIX ALGEBRA 11 / 19
 Matrix

Moreover, let A, B be two invertible matrix, AA−1 = A−1 A = I and

BB −1 = B −1 B = I . Since
(AB) (B −1 A−1 ) = A (BB −1 ) A−1 = AA−1 = I .
It follows that the product matrix AB is also invertible and
(AB)−1 = B −1 A−1
Definition 2. A matrix E is called an elementary matrix if it can be
obtained from an identity matrix by performing a single elementary
row operation.
The following theorem gives a connection between the matrix
multiplication and the elementary row operations.
Theorem 3. Let A ∈ Rn×n . If we perform a single elementary row
operation as [ A| In ] → [B| E ] then B = EA.

MATRIX ALGEBRA 12 / 19
 Matrix

Example 2
By changing the first and third row,
     
1 2 3  1 0 0 7 8 9  0 0 1

[A| I3 ] = 4
 5 6  0 1 0 → 4 5 6
   0 1 0 

7 8 9  0 0 1 1 2 3  1 0 0

= [B| E ], we get
    
0 0 1 1 2 3 7 8 9
EA =  0 1 0   4 5 6  =  4 5 6  = B
1 0 0 7 8 9 1 2 3

By multiplying the first row by 2,

     
1 2 3  1 0 0 2 4 6  2 0 0

[ A| I3 ] = 4
 5 6  0 1 0 → 4 5 6
   0 1 0  = [B| E

7 8 9  0 0 1 7 8 9
MATRIX ALGEBRA
 0 0 1
13 / 19
 Matrix

By substracting the second row with 4 times the first row,

     
1 2 3  1 0 0 1 2 3  1 0 0
[ A| I3 ] = 4 5 6  0 1 0 → 0 −3
    −6  −4 1 0 
7 8 9  0 0 1 7 8 9  0 0 1

= [B| E ], we get
    
1 0 0 1 2 3 1 2 3
EA =  −4 1 0   4 5 6  =  0 −3 −6  = B
0 0 1 7 8 9 7 8 9

MATRIX ALGEBRA 14 / 19
 Matrix

As an application of Theorem 1, we define the inverse operations for

the elementary row operations as follows,
1. The inverse operation for the “interchange rows i and j” is itself,
“interchange rows i and j”.
2. The inverse operation for the “multiply the row i by ” is the
operation “multiply the row i by ”.
3. The inverse operation for the “add time row j to row i” is the
operation “add time row j to row i”.
Corollary 4
Every elementary matrix is invertible, and the inverse is also an
elementary matrix.

Let E be an elementary matrix which is the result of some single

elementary operation on the identity matrix, and let E0 be the matrix
that results by performing the corresponding inverse elementary
operation. It is clear that E0 E = I and E E0 = I
MATRIX ALGEBRA 15 / 19
 Inversion Algorithm

Outlines

1 Matrix

2 Inversion Algorithm

MATRIX ALGEBRA 16 / 19
 Inversion Algorithm

Inversion Algorithm

By combining the above result, we have

If we can perfom a sequence of elementary row operations that
reduces a matrix A ∈ Rn×n to the identity matrix In , then A is
invertible and its inverse A−1 is the result of performing the same
sequence of operations on In . That is

[ A| In ] → [ In | B] (1)

gives B = A−1 . Precisely, the sequence of elementary row operations

of (1) is similar to that of the Gauss-Jordan algorithm for converting
A → In with the only difference is that all leading 1 must be on the
diagonal.

MATRIX ALGEBRA 17 / 19
 Inversion Algorithm

Example 3. Let  
1 −1 1
A =  −1 2 1 
−2 3 1
By performing
  
1 −1 1  1
 0 0
[ A| I3 ] =  −1 2 1  0
 1 0 →
 −2 3 1  0
 0 1
  
1 −1 1  1 0 0 1 0 3  2 1 0
 0 1 2 
 1 1 0  →  0 1 2  1 1 0 
0 1 3   2 0 1  1 −1 1
0 0 1
1 0 0  −1 4 −3
→  0 1 0  −1 3 −2  = [I3 | A−1 ]
0 0 1  1 −1 1

MATRIX ALGEBRA 18 / 19
 Inversion Algorithm
 
−1 4 −3
we get A−1 =  −1 3 −2 
1 −1 1
If in the ith step of the Gauss-Jordan algorithm, the ith element of
row i and all elements below are zeros, this matrix can not convert
into the identity matrix and then is not invertible.
Example 4. Let  
1 3 −4
A =  1 5 −1 
3 13 −6
  
1 3 −4  1 0 0
Perform [A| I3 ] =  1 5 −1  0 1 0 
 3 13 −6
 0 0 1
 5
1 0 − 17 3
  
1 3 −4  1 0 0 2 

2
− 2
0
→  0 2 3  −1 1 0  →  0 1 32  − 21 12 0 
0 4 6  −3 0 1 0 0 0  −1 −2 1
Matrix A can not convert into the identity matrix and not invertible.
MATRIX ALGEBRA 19 / 19