MA102 Linear Algebra

Lectures-4 & 5: Matrices in Action

RA, RKS, MGPP, BD

Department of Mathematics
IIT Guwahati

March-June 2023

1 / 37
Outline

Topics:
Matrix-vector multiplication
Matrix-matrix multiplication
Block matrices
Outer product of vectors
Elementary matrices
Gauss-Jordan method

2 / 37
Matrices
Definition: A matrix is an array of numbers. An m × n matrix A has m rows and n
columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn

3 / 37
Matrices
Definition: A matrix is an array of numbers. An m × n matrix A has m rows and n
columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn
 
a1j
 .. 
The j-th column of A : aj :=  .  for j = 1 : n.
amj
 
The i-th row of A: âi := ai1 ai2 · · · ain for i = 1 : m.

3 / 37
Matrices
Definition: A matrix is an array of numbers. An m × n matrix A has m rows and n
columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn
 
a1j
 .. 
The j-th column of A : aj :=  .  for j = 1 : n.
amj
 
The i-th row of A: âi := ai1 ai2 · · · ain for i = 1 : m. Then
   
a11 · · · a1n − â1 −
A =  ... .. ..  =  a a · · · a  =  ..
.
 
. .  1 2 n  .
am1 · · · amn − âm −
3 / 37
Matrices
An m × n matrix said to be a square matrix if m = n.

4 / 37
Matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j.

4 / 37
Matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j. An n × n diagonal matrix D with
diagonal entries d1 , . . . , dn is given by
 
d1
D = diag(d1 , . . . , dn ) = 
 .. .

.
dn

Identity matrix: An n × n diagonal matrix with all diagonal entries equal to 1 is called
the identity matrix and is denoted by In or I .

4 / 37
Matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j. An n × n diagonal matrix D with
diagonal entries d1 , . . . , dn is given by
 
d1
D = diag(d1 , . . . , dn ) = 
 .. .

.
dn

Identity matrix: An n × n diagonal matrix with all diagonal entries equal to 1 is called
the identity matrix and is denoted by In or I .

Zero matrix: An m × n matrix with all entries 0 is called the zero matrix and is
denoted by Om×n or simply by O.

4 / 37
Matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j. An n × n diagonal matrix D with
diagonal entries d1 , . . . , dn is given by
 
d1
D = diag(d1 , . . . , dn ) = 
 .. .

.
dn

Identity matrix: An n × n diagonal matrix with all diagonal entries equal to 1 is called
the identity matrix and is denoted by In or I .

Zero matrix: An m × n matrix with all entries 0 is called the zero matrix and is
denoted by Om×n or simply by O.
   
1 0 0 0 0 0 0
Example: I := 0 1 0 and O := 0 0 0 0 .
0 0 1 0 0 0 0
4 / 37
Matrix addition and scalar multiplication
Let Fm×n denote the set of all m × n matrices with entries in F by where F = R or
F = C.

5 / 37
Matrix addition and scalar multiplication
Let Fm×n denote the set of all m × n matrices with entries in F by where F = R or
F = C. Let A := [aij ] and B := [bij ] be matrices ∈ Fm×n and α ∈ F.

5 / 37
Matrix addition and scalar multiplication
Let Fm×n denote the set of all m × n matrices with entries in F by where F = R or
F = C. Let A := [aij ] and B := [bij ] be matrices ∈ Fm×n and α ∈ F.

1 Matrix addition: A + B := [aij + bij ] ∈ Fm×n .

5 / 37
Matrix addition and scalar multiplication
Let Fm×n denote the set of all m × n matrices with entries in F by where F = R or
F = C. Let A := [aij ] and B := [bij ] be matrices ∈ Fm×n and α ∈ F.

1 Matrix addition: A + B := [aij + bij ] ∈ Fm×n .

2 Multiplication by a scalar: α A := [α aij ] ∈ Fm×n .

5 / 37
Matrix addition and scalar multiplication
Let Fm×n denote the set of all m × n matrices with entries in F by where F = R or
F = C. Let A := [aij ] and B := [bij ] be matrices ∈ Fm×n and α ∈ F.

1 Matrix addition: A + B := [aij + bij ] ∈ Fm×n .

2 Multiplication by a scalar: α A := [α aij ] ∈ Fm×n .

   
1 4 0 −3 1 −1
Let A := and B := . Then
−2 6 5 0 0 2

5 / 37
Matrix addition and scalar multiplication
Let Fm×n denote the set of all m × n matrices with entries in F by where F = R or
F = C. Let A := [aij ] and B := [bij ] be matrices ∈ Fm×n and α ∈ F.

1 Matrix addition: A + B := [aij + bij ] ∈ Fm×n .

2 Multiplication by a scalar: α A := [α aij ] ∈ Fm×n .

   
1 4 0 −3 1 −1
Let A := and B := . Then
−2 6 5 0 0 2
     
1 4 0 −3 1 −1
−2 5 −1
A+B = + =
−2 6 5 −2 6
0 0 2 7
   
2 8 0 −1 −4 0
2A = and (−1)A = .
−4 12 10 2 −6 −5

5 / 37
Properties of matrix addition and scalar multiplication

Theorem: Let A, B, C ∈ Fm×n and α, β ∈ F. Then

6 / 37
Properties of matrix addition and scalar multiplication

Theorem: Let A, B, C ∈ Fm×n and α, β ∈ F. Then

1 Commutative Law: A + B = B + A.
2 Associative Law: (A + B) + C = A + (B + C ).

6 / 37
Properties of matrix addition and scalar multiplication

Theorem: Let A, B, C ∈ Fm×n and α, β ∈ F. Then

1 Commutative Law: A + B = B + A.
2 Associative Law: (A + B) + C = A + (B + C ).
3 A + O = A, where O ∈ Fm×n is the zero matrix
4 A + (−A) = O, where −A := (−1)A.

6 / 37
Properties of matrix addition and scalar multiplication

Theorem: Let A, B, C ∈ Fm×n and α, β ∈ F. Then

1 Commutative Law: A + B = B + A.
2 Associative Law: (A + B) + C = A + (B + C ).
3 A + O = A, where O ∈ Fm×n is the zero matrix
4 A + (−A) = O, where −A := (−1)A.
5 α(A + B) = α A + α B.
6 (α + β)A = α A + β A.

6 / 37
Properties of matrix addition and scalar multiplication

Theorem: Let A, B, C ∈ Fm×n and α, β ∈ F. Then

1 Commutative Law: A + B = B + A.
2 Associative Law: (A + B) + C = A + (B + C ).
3 A + O = A, where O ∈ Fm×n is the zero matrix
4 A + (−A) = O, where −A := (−1)A.
5 α(A + B) = α A + α B.
6 (α + β)A = α A + β A.
7 α (β A) = (αβ)A.
8 1A = A.

Proof: Exercise. 

6 / 37
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .

7 / 37
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .
 >
1 2    >  
1 4 7 1 + i 2 1 + i 3
Example: 4 5 = and =
2 5 8 3 4 + 5i 2 4 + 5i
7 8

7 / 37
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .
 >
1 2    >  
1 4 7 1 + i 2 1 + i 3
Example: 4 5 = and =
2 5 8 3 4 + 5i 2 4 + 5i
7 8

Conjugate transpose: The conjugate transpose of an m × n complex matrix

A = [aij ]m×n is the n × m matrix denoted by A∗ and is given by

A∗ = [āji ]n×m = ([āij ]m×n )> = (Ā)> ,

where āij is the complex conjugate of aij .

7 / 37
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .
 >
1 2    >  
1 4 7 1 + i 2 1 + i 3
Example: 4 5 = and =
2 5 8 3 4 + 5i 2 4 + 5i
7 8

Conjugate transpose: The conjugate transpose of an m × n complex matrix

A = [aij ]m×n is the n × m matrix denoted by A∗ and is given by

A∗ = [āji ]n×m = ([āij ]m×n )> = (Ā)> ,

where āij is the complex conjugate of aij .

 
 ∗ −i 3
i 4 1+i
Example: = 4 4 − 5i 
3 4 + 5i 0
1−i 0
7 / 37
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

8 / 37
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A

8 / 37
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A
3 Hermitian if A∗ = A
4 skew-Hermitian if A∗ = −A.

8 / 37
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A
3 Hermitian if A∗ = A
4 skew-Hermitian if A∗ = −A.

Remark: Let A := [aij ]n×n . If A> = −A then ajj = 0 for j = 1 : n.

8 / 37
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A
3 Hermitian if A∗ = A
4 skew-Hermitian if A∗ = −A.

Remark: Let A := [aij ]n×n . If A> = −A then ajj = 0 for j = 1 : n. On the other hand,
if A∗ = −A then Re(ajj ) = 0 for j = 1 : n.

8 / 37
Linear combination

Let u and v be vectors in Fn . Let α and β be in F. Adding αu and βv gives the linear
combination αu + βv.
Example: Let u := [1, 1, −1]> , v := [2, 3, 4]> and w := [4, 5, 2]> . Then w = 2u + v.
Thus w is a linear combination of u and v.

9 / 37
Linear combination

Let u and v be vectors in Fn . Let α and β be in F. Adding αu and βv gives the linear
combination αu + βv.
Example: Let u := [1, 1, −1]> , v := [2, 3, 4]> and w := [4, 5, 2]> . Then w = 2u + v.
Thus w is a linear combination of u and v.

Definition: Let v1 , . . . , vm be vectors in Fn and let α1 , . . . , αm be in F. Then the

vector u := α1 v1 + · · · + αm vm is called a linear combination of v1 , . . . , vm .

9 / 37
Linear combination

Let u and v be vectors in Fn . Let α and β be in F. Adding αu and βv gives the linear
combination αu + βv.
Example: Let u := [1, 1, −1]> , v := [2, 3, 4]> and w := [4, 5, 2]> . Then w = 2u + v.
Thus w is a linear combination of u and v.

Definition: Let v1 , . . . , vm be vectors in Fn and let α1 , . . . , αm be in F. Then the

vector u := α1 v1 + · · · + αm vm is called a linear combination of v1 , . . . , vm .
Vector equation: Find real numbers x1 , x2 , x3 such that
       
1 0 0 1
x1  −1  + x2  1  + x3  0  =  3  .
0 −1 1 5

9 / 37
Matrix-vector multiplication
Let A := a1 · · · an ∈ Fm×n and x := [x1 , . . . , xn ]> ∈ Fn . We define the
 

matrix-vector multiplication Ax as the linear combination of columns of A.

10 / 37
Matrix-vector multiplication
Let A := a1 · · · an ∈ Fm×n and x := [x1 , . . . , xn ]> ∈ Fn . We define the
 

matrix-vector multiplication Ax as the linear combination of columns of A.

Definition: Matrix-vector multiplication
 
x1
an  ...  = x1 a1 + · · · + xn an .
 
Ax = a1 · · ·


xn

10 / 37
Matrix-vector multiplication
Let A := a1 · · · an ∈ Fm×n and x := [x1 , . . . , xn ]> ∈ Fn . We define the
 

matrix-vector multiplication Ax as the linear combination of columns of A.

Definition: Matrix-vector multiplication
 
x1
an  ...  = x1 a1 + · · · + xn an .
 
Ax = a1 · · ·


xn

Thus A acts on x and satisfies

A(x + y) = Ax + Ay for all x, y in Fn .
A(αx) = αAx for all x ∈ Fn and α ∈ F.

10 / 37
Matrix-vector multiplication
Let A := a1 · · · an ∈ Fm×n and x := [x1 , . . . , xn ]> ∈ Fn . We define the
 

matrix-vector multiplication Ax as the linear combination of columns of A.

Definition: Matrix-vector multiplication
 
x1
an  ...  = x1 a1 + · · · + xn an .
 
Ax = a1 · · ·


xn

Thus A acts on x and satisfies

A(x + y) = Ax + Ay for all x, y in Fn .
A(αx) = αAx for all x ∈ Fn and α ∈ F.
Remark: Let ej be the j-th standard unit vector in Fn . Then it follows that Aej = aj
is j-th column of A.

10 / 37
Matrix-vector multiplication
 
A row vector ai1 · · · ain is a 1 × n matrix. Therefore
 
x1
  . 
ai1 · · · ain  ..  = ai1 x1 + · · · + ain xn .
xn

11 / 37
Matrix-vector multiplication
 
A row vector ai1 · · · ain is a 1 × n matrix. Therefore
 
x1
  . 
ai1 · · · ain  ..  = ai1 x1 + · · · + ain xn .
xn

Example: Matrix-vector multiplication in two ways

        
1 0 0 x1 1 0 0
 1 1 0   x2  = x1  1  + x2  1  + x3  0 
0 1 1 x3 0 1 1
  
1 0 0 x
  
x1
   
=  x1 + x2  =  1 1 0 x 
x2 + x3  
0 1 1 x

11 / 37
Matrix-vector multiplication
      
a11 · · · a1n x1 a11 a1n
 .. ..   ..   .   . 
More generally  . ··· .   .  = x1  ..  + · · · + xn  .. 
am1 · · · amn xn a amn
     m1 
a11 x1 + · · · + a1n xn a11 · · · a1n x
= .. ..
= .
   
.  . 
am1 x1 + · · · + amn xn am1 · · · amn x

12 / 37
Matrix-vector multiplication
      
a11 · · · a1n x1 a11 a1n
 .. ..   ..   .   . 
More generally  . ··· .   .  = x1  ..  + · · · + xn  .. 
am1 · · · amn xn a amn
     m1 
a11 x1 + · · · + a1n xn a11 · · · a1n x
= .. ..
= .
   
.  . 
am1 x1 + · · · + amn xn am1 · · · amn x
 
− â1 −
Writing A :=

a1 · · ·

an and A =  ..
 , we have
 
.
− âm −
   
a11 x1 + · · · + a1n xn â1 x
Ax = x1 a1 + · · · + xn an =  ..   .. 
 =  . .

.
am1 x1 + · · · + amn xn âm x

12 / 37
Matrix-matrix multiplication
Let A ∈ Fm×n and B := ∈ Fn×p .
 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

13 / 37
Matrix-matrix multiplication
Let A ∈ Fm×n and B := ∈ Fn×p .
 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

Reason: Define AB to be the m × p matrix such that (AB)x = A(Bx) for all x ∈ Fp .

13 / 37
Matrix-matrix multiplication
Let A ∈ Fm×n and B := ∈ Fn×p .
 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

Reason: Define AB to be the m × p matrix such that (AB)x = A(Bx) for all x ∈ Fp .
Let C := AB be given by C = c1 · · · cp . Let ej ∈ Fp be the standard unit vector.
Then for j = 1 : p, we have Bej = bj and

13 / 37
Matrix-matrix multiplication
Let A ∈ Fm×n and B := ∈ Fn×p .
 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

Reason: Define AB to be the m × p matrix such that (AB)x = A(Bx) for all x ∈ Fp .
Let C := AB be given by C = c1 · · · cp . Let ej ∈ Fp be the standard unit vector.
Then for j = 1 : p, we have Bej = bj and
 
cj = C ej = (AB)ej = A(Bej ) = Abj =⇒ C = Ab1 · · · Abp .

13 / 37
Matrix-matrix multiplication
Let A ∈ Fm×n and B := ∈ Fn×p .
 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

Reason: Define AB to be the m × p matrix such that (AB)x = A(Bx) for all x ∈ Fp .
Let C := AB be given by C = c1 · · · cp . Let ej ∈ Fp be the standard unit vector.
Then for j = 1 : p, we have Bej = bj and
 
cj = C ej = (AB)ej = A(Bej ) = Abj =⇒ C = Ab1 · · · Abp .

Fact: Let A ∈ Fm×n . Let ei ∈ Fm and ej ∈ Fn be standard unit vectors. Then

Aej is the j-th column of A.
e>
i A is the i-th row of A.

13 / 37
Matrix-matrix

multiplication

−â1 −
..
 ∈ Fm×n , B := b1 · · · ∈ Fn×p . Then
 
Let A =  bp
 
.
−âm −
   
â1 b1 · · · â1 bp â1 B
   . ..  =  ..  .
AB = Ab1 · · · Abp =  .. ··· .   . 
âm b1 · · · â1 bp âm B

14 / 37
Matrix-matrix

multiplication

−â1 −
..
 ∈ Fm×n , B := b1 · · · ∈ Fn×p . Then
 
Let A =  bp
 
.
−âm −
   
â1 b1 · · · â1 bp â1 B
   . ..  =  ..  .
AB = Ab1 · · · Abp =  .. ··· .   . 
âm b1 · · · â1 bp âm B

Thus if A := [aij ]m×n , B := [bij ]n×p and C := AB = [cij ]m×p then

 
b1j n
  .  X
.
cij = âi bj = ai1 · · · ain  .  = aik bkj .
bnj k=1

14 / 37
Matrix-matrix

multiplication

−â1 −
..
 ∈ Fm×n , B := b1 · · · ∈ Fn×p . Then
 
Let A =  bp
 
.
−âm −
   
â1 b1 · · · â1 bp â1 B
   . ..  =  ..  .
AB = Ab1 · · · Abp =  .. ··· .   . 
âm b1 · · · â1 bp âm B

Thus if A := [aij ]m×n , B := [bij ]n×p and C := AB = [cij ]m×p then

 
b1j n
  .  X
.
cij = âi bj = ai1 · · · ain  .  = aik bkj .
bnj k=1

Remark: If A and B are n × n matrices then in general AB 6= BA.

14 / 37
Example
 
  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0

15 / 37
Example
 
  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0

15 / 37
Example
 
  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0
 
  13 5
Therefore AB = Ab1 Ab2 = .
2 −2

15 / 37
Example


  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0
 
  13 5
Therefore AB = Ab1 Ab2 = . On the other hand
2 −2
   
  4 −1     4 −1  
a1 B = 1
b 3 2  1 2  = 13 5 a2 B = 0
and b −1 1  1 2 = 2 −2 .
3 0 3 0

15 / 37
Example


  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0
 
  13 5
Therefore AB = Ab1 Ab2 = . On the other hand
2 −2
   
  4 −1     4 −1  
a1 B = 1
b 3 2  1 2  = 13 5 a2 B = 0
and b −1 1  1 2 = 2 −2 .
3 0 3 0
   
a1 B 13 5  
Therefore AB = = = Ab1 Ab2 .
b
a2 B
b 2 −2

15 / 37
Block matrices
Definition: An m × n block matrix (or a partitioned matrix) is a matrix of the form
 
A11 · · · A1n
A :=  ... .. 

··· . 
Am1 · · · Amn

where each Aij is a pi × qj matrix for i = 1 : m and j = 1 : n.

16 / 37
Block matrices
Definition: An m × n block matrix (or a partitioned matrix) is a matrix of the form
 
A11 · · · A1n
A :=  ... .. 

··· . 
Am1 · · · Amn

where each Aij is a pi × qj matrix for i = 1 : m and j = 1 : n.

 
A1j
Then Ai1 · · · Ain is the i-th block row of A and  ...  is the j-th block
   

Amj
column of A.

16 / 37
Block matrices
Definition: An m × n block matrix (or a partitioned matrix) is a matrix of the form
 
A11 · · · A1n
A :=  ... .. 

··· . 
Am1 · · · Amn

where each Aij is a pi × qj matrix for i = 1 : m and j = 1 : n.

 
A1j
   ..
Then Ai1 · · · Ain is the i-th block row of A and  .  is the j-th block


Amj
column of A.
 
1 2 2 0 1 4
 3 4 1 2 3 5 
Example: 
 5 7
 has 2 block rows and 3 block columns.
2 7 8 8 
3 4 1 9 2 2
16 / 37
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

17 / 37
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

Block matrix multiplication: LetP

A := [Aij ]m×n and B := [Bij ]n×p be block matrices. If
the matrix multiplication Cij := nk=1 Aik Bkj is well defined for i = 1 : m and j = 1 : p
then AB is an m × p block matrix given by AB = [Cij ]m×p .

17 / 37
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

Block matrix multiplication: LetP

A := [Aij ]m×n and B := [Bij ]n×p be block matrices. If
the matrix multiplication Cij := nk=1 Aik Bkj is well defined for i = 1 : m and j = 1 : p
then AB is an m × p block matrix given by AB = [Cij ]m×p .
Conformal partition: If an operation on block matrices A and B are well defined then
A and B are said to be partitioned conformably.

17 / 37
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

Block matrix multiplication: LetP

A := [Aij ]m×n and B := [Bij ]n×p be block matrices. If
the matrix multiplication Cij := nk=1 Aik Bkj is well defined for i = 1 : m and j = 1 : p
then AB is an m × p block matrix given by AB = [Cij ]m×p .
Conformal partition: If an operation on block matrices A and B are well defined then
A and B are said to be partitioned conformably.
  
A11 A12 B11 B12 B13
Example: =
A21 A22 B21 B22 B23
 
A11 B11 + A12 B21 A11 B12 + A12 B22 A11 B13 + A12 B23
.
A21 B11 + A22 B21 A21 B12 + A22 B22 A21 B13 + A22 B23

17 / 37
Block matrix multiplication

Example:
 
  1 1 1 1  
1 1 1 1  1 8 6 4 5
 2 2 2 1 1 
 =  10 9
1 1 
 3 6 7 
1 1 1 
3 3 2 2 18 15 10 12
3 2 1 2
 
  1 1 1 1  
1 1 1 1  1 8 6 4 5
 2 2 2 1 1 
1 1  3
 =  10 9 6 7 
1 1 1 
3 3 2 2 18 15 10 12
3 2 1 2

18 / 37
Outer product

Given two vectors x and y in Rn , we have seen that y> x is the inner product given by

hx, yi = x1 y1 + · · · + xn yn = y> x.

19 / 37
Outer product

Given two vectors x and y in Rn , we have seen that y> x is the inner product given by

hx, yi = x1 y1 + · · · + xn yn = y> x.

Outer product: The matrix product xy> is an n × n matrix and is given by

   
x1 x1 y1 x1 y2 · · · x1 yn
 x2 y1 x2 y2 · · · x2 yn 
 x2     
>
xy =  .  y1 y2 · · · yn =  . ..  .
 
.. ..
 ..   .. . . . 
xn xn y1 xn y2 · · · xn yn

The product xy> is called the outer product of x ∈ Fn and y ∈ Fn .

19 / 37
Outer product
 >  >
Example: If x := 4 1 3 and y := 3 5 2 then
   
4  12 20 8
xy> =  1  3 5 2 =  3 5 2  .


3 9 15 6

20 / 37
Outer product
 >  >
Example: If x := 4 1 3 and y := 3 5 2 then
   
4  12 20 8
xy> =  1  3 5 2 =  3 5 2  .


3 9 15 6

Outer product of matrices: 

Let X := x1 x2 · · · xn ∈ Fm×n and Y := y1 y2 · · · yn ∈ Fp×n . Then
  

XY > ∈ Fm×p can be written as sum of outer products of vectors

20 / 37
Outer product
 >  >
Example: If x := 4 1 3 and y := 3 5 2 then
   
4  12 20 8
xy> =  1  3 5 2 =  3 5 2  .


3 9 15 6

Outer product of matrices: 

Let X := x1 x2 · · · xn ∈ Fm×n and Y := y1 y2 · · · yn ∈ Fp×n . Then
  

XY > ∈ Fm×p can be written as sum of outer products of vectors

 > 
y1
 y> 
 2 
XY > = x1 x2 · · · xn  .  = x1 y1> + x2 y2> + · · · + xn yn> .
 
 .. 
yn>

20 / 37
Properties of matrix multiplication
Thoerm: Let A, B and C be matrices (whose sizes are such that the indicated
operations can be performed) and let α be a scalar. Then

21 / 37
Properties of matrix multiplication
Thoerm: Let A, B and C be matrices (whose sizes are such that the indicated
operations can be performed) and let α be a scalar. Then
1 Associative Law: (AB)C = A(BC )
2 Left Distributive Law: A(B + C ) = AB + AC
3 Right Distributive Law: (A + B)C = AC + BC

21 / 37
Properties of matrix multiplication
Thoerm: Let A, B and C be matrices (whose sizes are such that the indicated
operations can be performed) and let α be a scalar. Then
1 Associative Law: (AB)C = A(BC )
2 Left Distributive Law: A(B + C ) = AB + AC
3 Right Distributive Law: (A + B)C = AC + BC
4 Scalar multiplication: α(AB) = (αA)B = A(αB)
5 Multiplicative identity: If A is an m × n matrix then Im A = A = AIn .

21 / 37
Properties of matrix multiplication
Thoerm: Let A, B and C be matrices (whose sizes are such that the indicated
operations can be performed) and let α be a scalar. Then
1 Associative Law: (AB)C = A(BC )
2 Left Distributive Law: A(B + C ) = AB + AC
3 Right Distributive Law: (A + B)C = AC + BC
4 Scalar multiplication: α(AB) = (αA)B = A(αB)
5 Multiplicative identity: If A is an m × n matrix then Im A = A = AIn .
ai := e>
Proof: Let b i A and cj := C ej . Then for all i and j, we have

[(AB)C ]ij = (b
ai B)cj = b
ai Bcj and [A(BC )]ij = b
ai (Bcj ) = b
ai Bcj .

Hence (AB)C = A(BC ).

21 / 37
Properties of matrix multiplication
Thoerm: Let A, B and C be matrices (whose sizes are such that the indicated
operations can be performed) and let α be a scalar. Then
1 Associative Law: (AB)C = A(BC )
2 Left Distributive Law: A(B + C ) = AB + AC
3 Right Distributive Law: (A + B)C = AC + BC
4 Scalar multiplication: α(AB) = (αA)B = A(αB)
5 Multiplicative identity: If A is an m × n matrix then Im A = A = AIn .
ai := e>
Proof: Let b i A and cj := C ej . Then for all i and j, we have

[(AB)C ]ij = (b
ai B)cj = b
ai Bcj and [A(BC )]ij = b
ai (Bcj ) = b
ai Bcj .

Hence (AB)C = A(BC ). Next, for all j, we have

[A(B+C )]ej = A(Bej +C ej ) = ABej +AC ej = (AB+AC )ej =⇒ A(B+C ) = AB+AC .

21 / 37
The inverse of a matrix
Definition: An n × n matrix A is said to be invertible if there exists a matrix B such
that AB = In = BA. The matrix B is called an inverse of A.

22 / 37
The inverse of a matrix
Definition: An n × n matrix A is said to be invertible if there exists a matrix B such
that AB = In = BA. The matrix B is called an inverse of A.
Exercise: Show that inverse of an invertible matrix A is unique.

22 / 37
The inverse of a matrix
Definition: An n × n matrix A is said to be invertible if there exists a matrix B such
that AB = In = BA. The matrix B is called an inverse of A.
Exercise: Show that inverse of an invertible matrix A is unique.

We denote the inverse of A by A−1 .

22 / 37
The inverse of a matrix
Definition: An n × n matrix A is said to be invertible if there exists a matrix B such
that AB = In = BA. The matrix B is called an inverse of A.
Exercise: Show that inverse of an invertible matrix A is unique.

We denote the inverse of A by A−1 . Note that if A is invertible then the system
Ax = b has a unique solution x = A−1 b.

22 / 37
The inverse of a matrix
Definition: An n × n matrix A is said to be invertible if there exists a matrix B such
that AB = In = BA. The matrix B is called an inverse of A.
Exercise: Show that inverse of an invertible matrix A is unique.

We denote the inverse of A by A−1 . Note that if A is invertible then the system
Ax = b has a unique solution x = A−1 b.
 
2 5
The matrix A = is invertible since
1 3
       
2 5 3 −5 1 0 3 −5 2 5
= = .
1 3 −1 2 0 1 −1 2 1 3

22 / 37
The inverse of a matrix
Definition: An n × n matrix A is said to be invertible if there exists a matrix B such
that AB = In = BA. The matrix B is called an inverse of A.
Exercise: Show that inverse of an invertible matrix A is unique.

We denote the inverse of A by A−1 . Note that if A is invertible then the system
Ax = b has a unique solution x = A−1 b.
 
2 5
The matrix A = is invertible since
1 3
       
2 5 3 −5 1 0 3 −5 2 5
= = .
1 3 −1 2 0 1 −1 2 1 3

The n × n zero matrix O is not invertible.

22 / 37
The inverse of a matrix
Definition: An n × n matrix A is said to be invertible if there exists a matrix B such
that AB = In = BA. The matrix B is called an inverse of A.
Exercise: Show that inverse of an invertible matrix A is unique.

We denote the inverse of A by A−1 . Note that if A is invertible then the system
Ax = b has a unique solution x = A−1 b.
 
2 5
The matrix A = is invertible since
1 3
       
2 5 3 −5 1 0 3 −5 2 5
= = .
1 3 −1 2 0 1 −1 2 1 3

The n × n zero matrix O is not invertible.

If an n × n matrix A has a zero row or a zero column, then A is NOT invertible.
22 / 37
Properties of invertible matrices
Theorem: Let A and B be n × n invertible matrices.
1 The matrix A−1 is also invertible, and (A−1 )−1 = A.
2 If c 6= 0 then cA is also invertible, and (cA)−1 = c1 A−1 .

23 / 37
Properties of invertible matrices
Theorem: Let A and B be n × n invertible matrices.
1 The matrix A−1 is also invertible, and (A−1 )−1 = A.
2 If c 6= 0 then cA is also invertible, and (cA)−1 = c1 A−1 .
3 The matrix AB is invertible, and (AB)−1 = B −1 A−1 .
4 The matrix A> is invertible, and (A> )−1 = (A−1 )> .

23 / 37
Properties of invertible matrices
Theorem: Let A and B be n × n invertible matrices.
1 The matrix A−1 is also invertible, and (A−1 )−1 = A.
2 If c 6= 0 then cA is also invertible, and (cA)−1 = c1 A−1 .
3 The matrix AB is invertible, and (AB)−1 = B −1 A−1 .
4 The matrix A> is invertible, and (A> )−1 = (A−1 )> .
5 For any non-negative integer k, the matrix Ak is invertible, and (Ak )−1 = (A−1 )k .

23 / 37
Properties of invertible matrices
Theorem: Let A and B be n × n invertible matrices.
1 The matrix A−1 is also invertible, and (A−1 )−1 = A.
2 If c 6= 0 then cA is also invertible, and (cA)−1 = c1 A−1 .
3 The matrix AB is invertible, and (AB)−1 = B −1 A−1 .
4 The matrix A> is invertible, and (A> )−1 = (A−1 )> .
5 For any non-negative integer k, the matrix Ak is invertible, and (Ak )−1 = (A−1 )k .
 
a b
Let A := . If ad − bc 6= 0 then A is invertible, and
c d
 
−1 1 d −b
A = .
ad − bc −c a

If ad − bc = 0 then A is not invertible.

23 / 37
Elementary matrices
Type Operation Inverse operation
I Ri ←− α Ri Ri ←− α1 Ri
II Rj ←− cRi + Rj Rj ←− −cRi + Rj
III Ri ↔ Rj Ri ↔ Rj

24 / 37
Elementary matrices
Type Operation Inverse operation
I Ri ←− α Ri Ri ←− α1 Ri
II Rj ←− cRi + Rj Rj ←− −cRi + Rj
III Ri ↔ Rj Ri ↔ Rj

An elementary matrix is a matrix that is obtained by performing an elementary row

operation on the identity matrix.

24 / 37
Elementary matrices
Type Operation Inverse operation
I Ri ←− α Ri Ri ←− α1 Ri
II Rj ←− cRi + Rj Rj ←− −cRi + Rj
III Ri ↔ Rj Ri ↔ Rj

An elementary matrix is a matrix that is obtained by performing an elementary row

operation on the identity matrix.
     
1 0 0 x1 x1
Type I : E2 (α) = 0 α 0 ⇒ E2 (α) x2  = αx2 
0 0 1 x3 x3

24 / 37
Elementary matrices
Type Operation Inverse operation
I Ri ←− α Ri Ri ←− α1 Ri
II Rj ←− cRi + Rj Rj ←− −cRi + Rj
III Ri ↔ Rj Ri ↔ Rj

An elementary matrix is a matrix that is obtained by performing an elementary row

operation on the identity matrix.
       
1 0 0 x1 x1 Row1 (A)
Type I : E2 (α) = 0 α 0 ⇒ E2 (α) x2  = αx2  and E2 (α)A = α Row2 (A) .
0 0 1 x3 x3 Row3 (A)

24 / 37
Elementary matrices
Type Operation Inverse operation
I Ri ←− α Ri Ri ←− α1 Ri
II Rj ←− cRi + Rj Rj ←− −cRi + Rj
III Ri ↔ Rj Ri ↔ Rj

An elementary matrix is a matrix that is obtained by performing an elementary row

operation on the identity matrix.
       
1 0 0 x1 x1 Row1 (A)
Type I : E2 (α) = 0 α 0 ⇒ E2 (α) x2  = αx2  and E2 (α)A = α Row2 (A) .
0 0 1 x3 x3 Row3 (A)

The inverse of a Type I elementary matrix is a Type I elementary matrix

1
(E2 (α))−1 = E2 ( ).
α
24 / 37
Type II elementary matrices
Let E13 (2) be the matrix obtained by performing R3 ← 2R1 + R3 on I3 . Then
 
1 0 0
E13 (2) := 0 1 0
2 0 1

25 / 37
Type II elementary matrices
Let E13 (2) be the matrix obtained by performing R3 ← 2R1 + R3 on I3 . Then
 
1 0 0
E13 (2) := 0 1 0
2 0 1

Applying E13 (2) on a matrix results in row operation

     
x1 x1 Row1 (A)
E13 (2) x2  =  x2  and E13 (2)A =  Row2 (A) .
x3 x3 + 2x1 Row3 (A) + 2 Row1 (A)

25 / 37
Type II elementary matrices
Let E13 (2) be the matrix obtained by performing R3 ← 2R1 + R3 on I3 . Then
 
1 0 0
E13 (2) := 0 1 0
2 0 1

Applying E13 (2) on a matrix results in row operation

     
x1 x1 Row1 (A)
E13 (2) x2  =  x2  and E13 (2)A =  Row2 (A) .
x3 x3 + 2x1 Row3 (A) + 2 Row1 (A)

The inverse of a Type II elementary matrix is again a Type II elementary matrix

(E13 (2))−1 = E13 (−2) corresponds to R3 ←− −2R1 + R3 on I3 .

25 / 37
Type III elementary matrices
Let Eij be the matrix obtained by performing Ri ↔ Rj on the identity matrix I . Then
       
1 0 0 x1 x1 Row1 (A)
E23 =  0 0 1  =⇒ E23 x2  = x3  and E23 A = Row3 (A) .
0 1 0 x3 x2 Row2 (A)

26 / 37
Type III elementary matrices
Let Eij be the matrix obtained by performing Ri ↔ Rj on the identity matrix I . Then
       
1 0 0 x1 x1 Row1 (A)
E23 =  0 0 1  =⇒ E23 x2  = x3  and E23 A = Row3 (A) .
0 1 0 x3 x2 Row2 (A)
The inverse of a Type III elementary matrix is a Type III elementary matrix

(Eij )−1 = Eij corresponds to row operation Ri ↔ Rj on I .

26 / 37
Type III elementary matrices
Let Eij be the matrix obtained by performing Ri ↔ Rj on the identity matrix I . Then
       
1 0 0 x1 x1 Row1 (A)
E23 =  0 0 1  =⇒ E23 x2  = x3  and E23 A = Row3 (A) .
0 1 0 x3 x2 Row2 (A)
The inverse of a Type III elementary matrix is a Type III elementary matrix

(Eij )−1 = Eij corresponds to row operation Ri ↔ Rj on I .

Theorem:
Type I: Multiplying Ei (c) to A giving Ei (c)A amounts to performing the row
operation Ri ←− cRi on A.

26 / 37
Type III elementary matrices
Let Eij be the matrix obtained by performing Ri ↔ Rj on the identity matrix I . Then
       
1 0 0 x1 x1 Row1 (A)
E23 =  0 0 1  =⇒ E23 x2  = x3  and E23 A = Row3 (A) .
0 1 0 x3 x2 Row2 (A)
The inverse of a Type III elementary matrix is a Type III elementary matrix

(Eij )−1 = Eij corresponds to row operation Ri ↔ Rj on I .

Theorem:
Type I: Multiplying Ei (c) to A giving Ei (c)A amounts to performing the row
operation Ri ←− cRi on A.
Type II: Multiplying Eij (c) to A giving Eij (c)A amounts to performing the row
operation Rj ←− cRi + Rj on A.

26 / 37
Type III elementary matrices
Let Eij be the matrix obtained by performing Ri ↔ Rj on the identity matrix I . Then
       
1 0 0 x1 x1 Row1 (A)
E23 =  0 0 1  =⇒ E23 x2  = x3  and E23 A = Row3 (A) .
0 1 0 x3 x2 Row2 (A)
The inverse of a Type III elementary matrix is a Type III elementary matrix

(Eij )−1 = Eij corresponds to row operation Ri ↔ Rj on I .

Theorem:
Type I: Multiplying Ei (c) to A giving Ei (c)A amounts to performing the row
operation Ri ←− cRi on A.
Type II: Multiplying Eij (c) to A giving Eij (c)A amounts to performing the row
operation Rj ←− cRi + Rj on A.
Type III: Multiplying Eij to A giving Eij A amounts to performing the row
operation Ri ←→ Rj on A.
26 / 37
Row echelon form and elementary matrices
Recall that two m × n matrices A and B are said to be row equivalent if A can be
transformed to B by elementary row operations.

Fact: The m × n matrices A and B are row equivalent ⇐⇒ B = Ek · · · E2 E1 A for

some elementary matrices E1 , E2 , · · · , Ek .

27 / 37
Row echelon form and elementary matrices
Recall that two m × n matrices A and B are said to be row equivalent if A can be
transformed to B by elementary row operations.

Fact: The m × n matrices A and B are row equivalent ⇐⇒ B = Ek · · · E2 E1 A for

some elementary matrices E1 , E2 , · · · , Ek .

Row echelon form: Let ref(A) be a row echelon form of A. Then there exist elementary
matrices E1 , . . . , E` such that ref(A) = E` · · · E2 E1 A.

27 / 37
Row echelon form and elementary matrices
Recall that two m × n matrices A and B are said to be row equivalent if A can be
transformed to B by elementary row operations.

Fact: The m × n matrices A and B are row equivalent ⇐⇒ B = Ek · · · E2 E1 A for

some elementary matrices E1 , E2 , · · · , Ek .

Row echelon form: Let ref(A) be a row echelon form of A. Then there exist elementary
matrices E1 , . . . , E` such that ref(A) = E` · · · E2 E1 A.
Reduced row echelon form: Let rref(A) be the reduced row echelon form of A.

27 / 37
Row echelon form and elementary matrices
Recall that two m × n matrices A and B are said to be row equivalent if A can be
transformed to B by elementary row operations.

Fact: The m × n matrices A and B are row equivalent ⇐⇒ B = Ek · · · E2 E1 A for

some elementary matrices E1 , E2 , · · · , Ek .

Row echelon form: Let ref(A) be a row echelon form of A. Then there exist elementary
matrices E1 , . . . , E` such that ref(A) = E` · · · E2 E1 A.
Reduced row echelon form: Let rref(A) be the reduced row echelon form of A. Then
there exist elementary matrices Ê1 , . . . , Êp such that

rref(A) = Êp · · · Ê2 Ê1 E` · · · E2 E1 A.

27 / 37
Row echelon form and elementary matrices
Recall that two m × n matrices A and B are said to be row equivalent if A can be
transformed to B by elementary row operations.

Fact: The m × n matrices A and B are row equivalent ⇐⇒ B = Ek · · · E2 E1 A for

some elementary matrices E1 , E2 , · · · , Ek .

Row echelon form: Let ref(A) be a row echelon form of A. Then there exist elementary
matrices E1 , . . . , E` such that ref(A) = E` · · · E2 E1 A.
Reduced row echelon form: Let rref(A) be the reduced row echelon form of A. Then
there exist elementary matrices Ê1 , . . . , Êp such that

rref(A) = Êp · · · Ê2 Ê1 E` · · · E2 E1 A.

Fact: Let : [U | d] := ref([A | b]). Then there is an invertible matrix E such that
E [A | b] = [U | d]. Hence the system Ax = b and Ux = d are equivalent.

27 / 37
Gauss-Jordan method for computing inverse
Fact: Let A be an n × n matrix. If there exist elementary matrices E1 , . . . , Ep such that
Ep · · · E2 E1 A = In then A−1 = Ep · · · E2 E1 . Hence rref(A) = In ⇐⇒ A is invertible.

28 / 37
Gauss-Jordan method for computing inverse
Fact: Let A be an n × n matrix. If there exist elementary matrices E1 , . . . , Ep such that
Ep · · · E2 E1 A = In then A−1 = Ep · · · E2 E1 . Hence rref(A) = In ⇐⇒ A is invertible.

Observation: Elementary row operations that transform A to In transform In to A−1 .

28 / 37
Gauss-Jordan method for computing inverse
Fact: Let A be an n × n matrix. If there exist elementary matrices E1 , . . . , Ep such that
Ep · · · E2 E1 A = In then A−1 = Ep · · · E2 E1 . Hence rref(A) = In ⇐⇒ A is invertible.

Observation: Elementary row operations that transform A to In transform In to A−1 .

 
Note that if A is invertible, then the rref of A | In is given by
    
rref( A | In ) = Ep · · · E2 E1 A | In ] = Ep · · · E2 E1 A | Ep · · · E2 E1
= In | A−1 .
 

28 / 37
Gauss-Jordan method for computing inverse
Fact: Let A be an n × n matrix. If there exist elementary matrices E1 , . . . , Ep such that
Ep · · · E2 E1 A = In then A−1 = Ep · · · E2 E1 . Hence rref(A) = In ⇐⇒ A is invertible.

Observation: Elementary row operations that transform A to In transform In to A−1 .

 
Note that if A is invertible, then the rref of A | In is given by
    
rref( A | In ) = Ep · · · E2 E1 A | In ] = Ep · · · E2 E1 A | Ep · · · E2 E1
= In | A−1 .
 

So to find A−1 , use GJE to reduce A | In to reduced row echelon form In | X .

   

Gauss-Jordan method:

[A | In ] −→ [In | X ] =⇒ A is invertible and A−1 = X .

28 / 37
Gauss-Jordan

method

1 2 3
Let A := 2 5 7 . Then [A | I ] → [I | A−1 ] gives
3 7 9

29 / 37
Gauss-Jordan

method

1 2 3
Let A := 2 5 7 . Then [A | I ] → [I | A−1 ] gives
3 7 9
   
1 2 3 1 0 0 1 2 3 1 0 0
 2 5 7 0 1 0  →  0 1 1 −2 1 0  →
3 7 9 0 0 1 0 1 0 −3 0 1

29 / 37
Gauss-Jordan

method

1 2 3
Let A := 2 5 7 . Then [A | I ] → [I | A−1 ] gives
3 7 9
   
1 2 3 1 0 0 1 2 3 1 0 0
 2 5 7 0 1 0 → 0 1 1 −2 1 0  →
3 7 9 0 0 1 0 1 0 −3 0 1
   
1 0 1 5 −2 0 1 0 0 4 −3 1
 0 1 1 −2 1 0  →  0 1 0 −3 0 1 
0 0 −1 −1 −1 1 0 0 1 1 1 −1

29 / 37
Gauss-Jordan

method

1 2 3
Let A := 2 5 7 . Then [A | I ] → [I | A−1 ] gives
3 7 9
   
1 2 3 1 0 0 1 2 3 1 0 0
 2 5 7 0 1 0 → 0 1 1 −2 1 0  →
3 7 9 0 0 1 0 1 0 −3 0 1
   
1 0 1 5 −2 0 1 0 0 4 −3 1
 0 1 1 −2 1 0  →  0 1 0 −3 0 1 
0 0 −1 −1 −1 1 0 0 1 1 1 −1

 
4 −3 1
=⇒ A−1 =  −3 0 1 .
1 1 −1

29 / 37
Characterization of invertibility

Theorem: Let A ∈ Fn×n . Then the following statements are equivalent.

1 A is invertible.
2 Ax = b has a unique solution for every b in Fn .

30 / 37
Characterization of invertibility

Theorem: Let A ∈ Fn×n . Then the following statements are equivalent.

1 A is invertible.
2 Ax = b has a unique solution for every b in Fn .
3 Ax = 0 has only the trivial solution.
4 The reduced row echelon form of A is In .

30 / 37
Characterization of invertibility

Theorem: Let A ∈ Fn×n . Then the following statements are equivalent.

1 A is invertible.
2 Ax = b has a unique solution for every b in Fn .
3 Ax = 0 has only the trivial solution.
4 The reduced row echelon form of A is In .
5 A is a product of elementary matrices.
6 rank(A) = n
Proof: The implications (1) ⇒ (2) ⇒ (3) ⇒ (4) ⇒ (5) ⇒ (6) are immediate.

30 / 37
Characterization of invertibility

Theorem: Let A ∈ Fn×n . Then the following statements are equivalent.

1 A is invertible.
2 Ax = b has a unique solution for every b in Fn .
3 Ax = 0 has only the trivial solution.
4 The reduced row echelon form of A is In .
5 A is a product of elementary matrices.
6 rank(A) = n
Proof: The implications (1) ⇒ (2) ⇒ (3) ⇒ (4) ⇒ (5) ⇒ (6) are immediate.

Now if rank(A) = n then the reduced row echelon form of A has n pivot column
=⇒ rref(A) = In =⇒ A is invertible. Hence (6) ⇒ (1). This completes the proof. 

30 / 37
LU Factorization
Definition: Let A be an n × n matrix. A decomposition of A of the form A = LU is
called an LU factorization of A, where U is an upper triangular matrix and L is a unit
lower triangular matrix (the diagonal entries of L are 1).

31 / 37
LU Factorization
Definition: Let A be an n × n matrix. A decomposition of A of the form A = LU is
called an LU factorization of A, where U is an upper triangular matrix and L is a unit
lower triangular matrix (the diagonal entries of L are 1).

Theorem: If A −→ ref(A) without row interchange then A has an LU factorization.

31 / 37
LU Factorization
Definition: Let A be an n × n matrix. A decomposition of A of the form A = LU is
called an LU factorization of A, where U is an upper triangular matrix and L is a unit
lower triangular matrix (the diagonal entries of L are 1).

Theorem: If A −→ ref(A) without row interchange then A has an LU factorization.

Proof: By the assumption, there exist Type-II elementary matrices E1 , . . . , Ep such that
Ep . . . E2 E1 A = ref(A). Hence A = LU, where U := ref(A) and L := E1−1 E2−1 . . . Ep−1 .

31 / 37
LU Factorization
Definition: Let A be an n × n matrix. A decomposition of A of the form A = LU is
called an LU factorization of A, where U is an upper triangular matrix and L is a unit
lower triangular matrix (the diagonal entries of L are 1).

Theorem: If A −→ ref(A) without row interchange then A has an LU factorization.

Proof: By the assumption, there exist Type-II elementary matrices E1 , . . . , Ep such that
Ep . . . E2 E1 A = ref(A). Hence A = LU, where U := ref(A) and L := E1−1 E2−1 . . . Ep−1 .
Now Ej is Type-II =⇒ Ej is unit lower triangular =⇒ Ej−1 is unit lower triangular
=⇒ L := E1−1 E2−1 . . . Ep−1 is unit lower triangular. 

31 / 37
LU Factorization
Definition: Let A be an n × n matrix. A decomposition of A of the form A = LU is
called an LU factorization of A, where U is an upper triangular matrix and L is a unit
lower triangular matrix (the diagonal entries of L are 1).

Theorem: If A −→ ref(A) without row interchange then A has an LU factorization.

Proof: By the assumption, there exist Type-II elementary matrices E1 , . . . , Ep such that
Ep . . . E2 E1 A = ref(A). Hence A = LU, where U := ref(A) and L := E1−1 E2−1 . . . Ep−1 .
Now Ej is Type-II =⇒ Ej is unit lower triangular =⇒ Ej−1 is unit lower triangular
=⇒ L := E1−1 E2−1 . . . Ep−1 is unit lower triangular. 

Solution of Ax = b via LU factorization (when exists):

Compute A = LU.
Solve Ly = b for y - forward substitution.
Solve Ux = y for x - back substitution.

31 / 37
LU factorization
 
a11 a12 a13
Let A = a21 a22 a23  . Set m21 := a21 /a11 and m31 := a31 /a11
a31 a32 a33

when a11 6= 0 (pivot) and define

   
1 0 0 1 0 0
E1 := −m21 1 0 and E2 :=  0 1 0 .
0 0 1 −m31 0 1

32 / 37
LU factorization
 
a11 a12 a13
Let A = a21 a22 a23  . Set m21 := a21 /a11 and m31 := a31 /a11
a31 a32 a33

when a11 6= 0 (pivot) and define

   
1 0 0 1 0 0
E1 := −m21 1 0 and E2 :=  0 1 0 .
0 0 1 −m31 0 1
Then   
a11 a12 a13

a11 a12 a13
(1) (1) (1) (1) 
E2 E1 A = E2  0 a22 a23  =  0 a22 a23  .

(1) (1)
a31 a32 a33 0 a32 a33
(1) (1) (1)
Set m32 := a32 /a22 if a22 6= 0 (pivot) and define

32 / 37
LU factorization

 
1 0 0
E3 := 0 1 0 . Then we have
0 −m32 1
   
a11 a12 a13 a11 a12 a13
(1) (1)  (1) (1) 
E3 E2 E1 A = E3  0 a22 a23  =  0 a22 a23  = U.
 
(1) (1) (2)
0 a32 a33 0 0 a33

33 / 37
LU factorization

 
1 0 0
E3 := 0 1 0 . Then we have
0 −m32 1
   
a11 a12 a13 a11 a12 a13
(1) (1)  (1) (1) 
E3 E2 E1 A = E3  0 a22 a23  =  0 a22 a23  = U.
 
(1) (1) (2)
0 a32 a33 0 0 a33
 
1 0 0
Hence A = LU, where L = E1−1 E2−1 E3−1 = m21 1 0 .
m31 m32 1

33 / 37
Examples: LU factorization
 
1 1 1
Let A := 1 2 2 . Then A = LU, where
1 2 3
   
1 0 0 1 1 1
L = 1 1 0 and U = 0 1 1 .
1 1 1 0 0 1

34 / 37
Examples: LU factorization
 
1 1 1
Let A := 1 2 2 . Then A = LU, where
1 2 3
   
1 0 0 1 1 1
L = 1 1 0 and U = 0 1 1 .
1 1 1 0 0 1
 
2 4 −1
Let A := −4 −5 3  . Then A = LU, where
2 −5 −4
   
1 0 0 2 4 −1
L = −2 1 0 and U = 0 3 1  .
1 −3 1 0 0 0

34 / 37
Three theorems with similar proof structure

Theorem: Reduced row echelon form of a matrix is unique. Equivalently, if R1 and R2

are in rref and are row equivalent, then R1 = R2 .

35 / 37
Three theorems with similar proof structure

Theorem: Reduced row echelon form of a matrix is unique. Equivalently, if R1 and R2

are in rref and are row equivalent, then R1 = R2 .

Theorem: Let A and B be m × n matrices. Then the systems Ax = 0 and Bx = 0 are

equivalent (i.e. have same solutions) if and only if A and B are row equivalent.

35 / 37
Three theorems with similar proof structure

Theorem: Reduced row echelon form of a matrix is unique. Equivalently, if R1 and R2

are in rref and are row equivalent, then R1 = R2 .

Theorem: Let A and B be m × n matrices. Then the systems Ax = 0 and Bx = 0 are

equivalent (i.e. have same solutions) if and only if A and B are row equivalent.

Theorem: Let A and B be m × n matrices. Suppose that the systems Ax = b and

Bx = c are consistent. Then the two systems are equivalent if and only if the matrices
[ A | b ] and [ B | c ] are row equivalent.

35 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .

36 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .
Proof: Note that R1 and R2 are row equivalent. Hence

R1 x = 0 ⇐⇒ R2 x = 0. (1)

36 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .
Proof: Note that R1 and R2 are row equivalent. Hence

R1 x = 0 ⇐⇒ R2 x = 0. (1)

Step-1: Claim R1 e1 = R2 e1 .

36 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .
Proof: Note that R1 and R2 are row equivalent. Hence

R1 x = 0 ⇐⇒ R2 x = 0. (1)

Step-1: Claim R1 e1 = R2 e1 .
By (1), we have R1 e1 = 0 ⇐⇒ R2 e1 = 0.

36 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .
Proof: Note that R1 and R2 are row equivalent. Hence

R1 x = 0 ⇐⇒ R2 x = 0. (1)

Step-1: Claim R1 e1 = R2 e1 .
By (1), we have R1 e1 = 0 ⇐⇒ R2 e1 = 0.
If R1 e1 6= 0 then R1 e1 and R2 e1 are pivot columns. Hence

R1 e1 = e1 = R2 e1 .

36 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .
Proof: Note that R1 and R2 are row equivalent. Hence

R1 x = 0 ⇐⇒ R2 x = 0. (1)

Step-1: Claim R1 e1 = R2 e1 .
By (1), we have R1 e1 = 0 ⇐⇒ R2 e1 = 0.
If R1 e1 6= 0 then R1 e1 and R2 e1 are pivot columns. Hence

R1 e1 = e1 = R2 e1 .

Step-2: Induction: Suppose R1 ej = R2 ej for j = 1 : k. Claim: R1 ek+1 = R2 ek+1 .

36 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .
Proof: Note that R1 and R2 are row equivalent. Hence

R1 x = 0 ⇐⇒ R2 x = 0. (1)

Step-1: Claim R1 e1 = R2 e1 .
By (1), we have R1 e1 = 0 ⇐⇒ R2 e1 = 0.
If R1 e1 6= 0 then R1 e1 and R2 e1 are pivot columns. Hence

R1 e1 = e1 = R2 e1 .

Step-2: Induction: Suppose R1 ej = R2 ej for j = 1 : k. Claim: R1 ek+1 = R2 ek+1 . If

R1 ek+1 is a non-pivot column then there exist α1 , . . . , αk such that

36 / 37
Uniqueness of rref
Theorem: Let R1 and R2 be rref of A. Then R1 = R2 .
Proof: Note that R1 and R2 are row equivalent. Hence

R1 x = 0 ⇐⇒ R2 x = 0. (1)

Step-1: Claim R1 e1 = R2 e1 .
By (1), we have R1 e1 = 0 ⇐⇒ R2 e1 = 0.
If R1 e1 6= 0 then R1 e1 and R2 e1 are pivot columns. Hence

R1 e1 = e1 = R2 e1 .

Step-2: Induction: Suppose R1 ej = R2 ej for j = 1 : k. Claim: R1 ek+1 = R2 ek+1 . If

R1 ek+1 is a non-pivot column then there exist α1 , . . . , αk such that

R1 ek+1 = α1 R1 e1 + · · · + αk R1 ek =⇒ R1 x = 0, (2)

where x := [α1 , . . . , αk , −1, 0, . . . , 0]> .

36 / 37
Uniqueness of rref

Since R1 ej = R2 ej , j = 1, 2, . . . , k, by (1) and (2), we have

37 / 37
Uniqueness of rref

Since R1 ej = R2 ej , j = 1, 2, . . . , k, by (1) and (2), we have

R2 x = 0 ⇒ R2 ek+1 = α1 R2 e1 + · · · + αk R2 ek = R1 ek+1 .

37 / 37
Uniqueness of rref

Since R1 ej = R2 ej , j = 1, 2, . . . , k, by (1) and (2), we have

R2 x = 0 ⇒ R2 ek+1 = α1 R2 e1 + · · · + αk R2 ek = R1 ek+1 .

On the other hand, if R1 ek+1 is a pivot column then R2 ek+1 is also a pivot column.

37 / 37
Uniqueness of rref

Since R1 ej = R2 ej , j = 1, 2, . . . , k, by (1) and (2), we have

R2 x = 0 ⇒ R2 ek+1 = α1 R2 e1 + · · · + αk R2 ek = R1 ek+1 .

On the other hand, if R1 ek+1 is a pivot column then R2 ek+1 is also a pivot column.

If there are p pivot columns in the first k columns of R1 then R1 ek+1 = ep+1 and
R2 ek+1 = ep+1 .

37 / 37
Uniqueness of rref

Since R1 ej = R2 ej , j = 1, 2, . . . , k, by (1) and (2), we have

R2 x = 0 ⇒ R2 ek+1 = α1 R2 e1 + · · · + αk R2 ek = R1 ek+1 .

On the other hand, if R1 ek+1 is a pivot column then R2 ek+1 is also a pivot column.

If there are p pivot columns in the first k columns of R1 then R1 ek+1 = ep+1 and
R2 ek+1 = ep+1 . Hence R1 ek+1 = R2 ek+1 .

37 / 37
Uniqueness of rref

Since R1 ej = R2 ej , j = 1, 2, . . . , k, by (1) and (2), we have

R2 x = 0 ⇒ R2 ek+1 = α1 R2 e1 + · · · + αk R2 ek = R1 ek+1 .

On the other hand, if R1 ek+1 is a pivot column then R2 ek+1 is also a pivot column.

If there are p pivot columns in the first k columns of R1 then R1 ek+1 = ep+1 and
R2 ek+1 = ep+1 . Hence R1 ek+1 = R2 ek+1 .

Hence we have R1 = R2 . 

*** End ***

37 / 37