Lecture 21: 5.

6
Rank and Nullity

Wei-Ta Chu

2008/12/5
Rank and Nullity

 Definition
 The common dimension of the row and column space of a matrix
A is called the rank (秩) of A and is denoted by rank(A); the
dimension of the nullspace of A is called the nullity (零核維數)
of A and is denoted by nullity(A).

2008/12/5 Elementary Linear Algebra 2

Example (Rank and Nullity)

 Find the rank and nullity of the matrix

1 2 0 4 3 5
3 7 2 0 1 4
A  
2 5 2 4 6 1
 
4 9 2 4 4 7 
 Solution:
 The reduced row-echelon form of A is

1 0 4 28 37 13

0 1 2 12 16 5 
 

0 0 0 0 0 0
 
0
 0 0 0 0 0
 Sincet
he rearet wononz e
ror ows(twol e ading1’ s)
, ther ows
pac
eand
column space are both two-dimensional, so rank(A) = 2.

2008/12/5 Elementary Linear Algebra 3

Example (Rank and Nullity)
 To find the nullity of A, we must find the dimension of the
solution space of the linear system Ax=0.
 The corresponding system of equations will be
x1 –4x3 –28x4 –37x5 + 13x6 = 0
x2 –2x3 –12x4 –16 x5+ 5 x6 = 0
 It follows that the general solution of the system is
x1 = 4r + 28s + 37t –13u, x2 = 2r + 12s + 16t –5u,
x3 = r, x4 = s, x5 = t, x6 = u
or x1   4 28  37   13

x2   12  16   
  2     5 

x3  1 0  0  0  Thus, nullity(A) = 4.
    t  u  
r s
x4  
 0 1  0  0 

x5  0 0  1  0 
        
  
x6 
 
0 0 
  
0  
1 

2008/12/5 Elementary Linear Algebra 4

Theorems

 Theorem 5.6.2
 If A is any matrix, then rank(A) = rank(AT).
 Proof: rank(A) = dim(row space of A) = dim(column space of AT)
= rank(AT)

 Theorem 5.6.3 (Dimension Theorem for Matrices)

 If A is a matrix with n columns, then rank(A) + nullity(A) = n.

2008/12/5 Elementary Linear Algebra 5

Proof of Theorem 5.6.3

 Since A has n columns, Ax = 0 has n unknowns. These

fall into two categories: the leading variables and the free
variables.

 Thenumbe rofl eading1’ si

nther
educ
edr
ow-echelon
form of A is the rank of A

2008/12/5 Elementary Linear Algebra 6

Proof of Theorem 5.6.3

 The number of free variables is equal to the nullity of A.

This is so because the nullity of A is the dimension of the
solution space of Ax=0, which is the same as the number
of parameters in the general solution, which is the same
as the number of free variables. Thus
rank(A) + nullity(A) = n

2008/12/5 Elementary Linear Algebra 7

Theorem 5.6.4

 Theorem 5.6.4
 If A is an mn matrix, then:
 rank(A) = Number of leading variables in the solution of Ax = 0.
 nullity(A) = Number of parameters in the general solution of Ax = 0.

2008/12/5 Elementary Linear Algebra 8

Example (Sum of Rank and Nullity)

 The matrix
1 2 0 3
4 5
3 7 2 4
0 1
A  
2 5 2 1
4 6
 
4 9 2 4 4 7 
has 6 columns, so
rank(A) + nullity(A) = 6
 This is consistent with the previous example, where we
showed that
rank(A) = 2 and nullity(A) = 4

2008/12/5 Elementary Linear Algebra 9

Example

 Find the number of parameters in the general solution of

Ax = 0 if A is a 57 matrix of rank 3.
 Solution:
 nullity(A) = n –rank(A) = 7 –3 = 4
 Thus, there are four parameters.

2008/12/5 Elementary Linear Algebra 10

Dimensions of Fundamental Spaces

 Suppose that A is an mn matrix of rank r, then

 AT is an nm matrix of rank r by Theorem 5.6.2
 nullity(A) = n –r, nullity(AT) = m –r by Theorem 5.6.3

Fundamental Space Dimension

Row space of A r
Column space of A r
Nullspace of A n –r
Nullspace of AT m –r

2008/12/5 Elementary Linear Algebra 11

Maximum Value for Rank

 If A is an mn matrix
 The row vectors lie in Rn and the column vectors lie in Rm.
 The row space of A is at most n-dimensional and the column
space is at most m-dimensional.

 Since the row and column space have the same dimension (the rank
A), we must conclude that if m n, then the rank of A is at most the
smaller of the values of m or n.
 That is,
rank(A) min(m, n)

2008/12/5 Elementary Linear Algebra 12

Example

 If A is a 74 matrix, then the rank of A is at most 4 and,

consequently, the seven row vectors must be linearly
dependent. If A is a 47 matrix, then again the rank of A
is at most 4 and, consequently, the seven column vectors
must be linearly dependent.

2008/12/5 Elementary Linear Algebra 13

Theorem 5.6.5

 Theorem 5.6.5 (The Consistency Theorem)

 If Ax = b is a linear system of m equations in n
unknowns, then the following are equivalent.
 Ax = b is consistent.
 b is in the column space of A.
 The coefficient matrix A and the augmented matrix
[A | b] have the same rank.

2008/12/5 Elementary Linear Algebra 14

Proof of Theorem 5.6.5
(b) → (c)
 If b is in the column space of A, then the column spaces
of A and [A | b] are actually the same, from which it will
follow that these two matrices have the same rank.
 The column spaces of A and [A | b] can be expressed as
span{c1, c2, …, cn} and span{c1, c2,…, cn, b}
 If b is in the column space of A, then each vector in the
set {c1,c2, …, cn,b} is a linear combination of the vectors in
{c1,c2, …, cn}. Thus, from Theorem 5.2.4, the column
spaces of A and [A | b] are the same.

2008/12/5 Elementary Linear Algebra 15

Proof of Theorem 5.6.5
(c) → (b)
 Assume that A and [A | b] have the same rank r. By
Theorem 5.4.6a, there is some subset of the column
vectors of A that forms a basis for the column space of A.
 Suppose that those column vectors are
c’1, c’
2,
…, c’r
 These r basis vectors also belong to the r-dimensional
column space of [A | b]; hence they also form a basis for
the column space of [A | b] by Theorem 5.4.6a. This
means that b is expressible as a linear combination of c’1,
c’2,…, c’r, and consequently b lies in the column space of
A.
2008/12/5 Elementary Linear Algebra 16
Example

 We see from the third row that the system is inconsistent.

However, it is also because of this row that the reduced row-
echelon form of the augmented matrix has fewer zero rows
than the reduced row-echelon form of the coefficient matrix.
 This forces the coefficient matrix and the augmented matrix
for the system to have different ranks.

2008/12/5 Elementary Linear Algebra 17

Theorem 5.6.6

 Theorem 5.6.6
 If Ax = b is a linear system of m equations in n
unknowns, then the following are equivalent.
 Ax = b is consistent for every m1 matrix b.
 The column vectors of A span Rm.
 rank(A) = m.

2008/12/5 Elementary Linear Algebra 18

Proof of Theorem 5.6.6

 : The system Ax = b can be expressed as

from which we can conclude that Ax=b is consistent for

every matrix b if and only if every such b is
expressible as a linear combination of the column vectors
c1, c2,
…, cn, or, equivalently, if and only if these column
vectors span Rm.

2008/12/5 Elementary Linear Algebra 19

Proof of Theorem 5.6.6

 : From the assumption that Ax=b is consistent

for every matrix b, and from (a) and (b) of the
Consistency Theorem (5.6.5), it follows that every vector
b in Rm lies in the column space of A; that is, the column
space of A is all of Rm. Thus rank(A)=dim(Rm)=m.

2008/12/5 Elementary Linear Algebra 20

Proof of Theorem 5.6.6

 : From the assumption that rank(A)=m, it

follows that the column space of A is a subspace of Rm of
dimension m and hence must be all of Rm by Theorem
5.4.7.
 It now follows from parts (a) and (b) of the Consistency
Theorem (5.6.5) that Ax=b is consistent from ever vector
b in Rm, since every such b is in the column space of A.

2008/12/5 Elementary Linear Algebra 21

Overdetermined System

 A linear system with more equations than unknowns is

called an overdetermined linear system (超定線性方程
組).
 If Ax = b is an overdetermined linear system of m
equations in n unknowns (so that m > n), then the column
vectors of A cannot span Rm.

 Thus, the overdetermined linear system Ax = b cannot be

consistent for every possible b.

2008/12/5 Elementary Linear Algebra 22

Example
x1 2 x2 b1
x1  x2 b2
 The linear system x1  x2 b3
x1 2 x2 b4
x1 3 x2 b5
is overdetermined, so it cannot be consistent for all
possible values of b1, b2, b3, b4, and b5. Exact conditions
under which the system is consistent can be obtained by
solving the linear system by Gauss-Jordan elimination.

1 0 2b2 b1

0 1 b2 b1 
 

0 0 b3 3b2 2b1 
 
0
 0 b4 4b2 3b1 

0
 0 b5 5b2 4b1 


2008/12/5 Elementary Linear Algebra 23

Example

 Thus, the system is consistent if and only if b1, b2, b3, b4,
and b5 satisfy the conditions
2b1 3b2 b3 =0
2b1 4b2 b4 =0
4b1 5b2 b5 =0
or, on solving this homogeneous linear system, b1=5r-4s,
b2=4r-3s, b3=2r-s, b4=r, b5=s where r and s are arbitrary.

2008/12/5 Elementary Linear Algebra 24

Theorem 5.6.7

 Theorem 5.6.7
 If Ax = b is a consistent linear system of m equations in n
unknowns, and if A has rank r, then the general solution of
the system contains n –r parameters.

 If A is a matrix with rank 4, and if Ax=b is a

consistent linear system, then the general solution of
the system contains 7-4=3 parameters.

2008/12/5 Elementary Linear Algebra 25

Theorem 5.6.8

 Theorem 5.6.8
 If A is an mn matrix, then the following are equivalent.
 Ax = 0 has only the trivial solution.

 The column vectors of A are linearly independent.

 Ax = b has at most one solution (0 or 1) for every m 1

matrix b.

2008/12/5 Elementary Linear Algebra 26

Proof of Theorem 5.6.8

 : If c1, c2,
…,cn are the column vectors of A,
then the linear system Ax=0 can be written as

 If c1, c2,…, cn are linearly independent vectors, then this

equation is satisfied only by x1=x2=…=xn=0, which means
that Ax=0 has only the trivial solution.
 Conversely, if Ax=0 has only the trivial solution, then it is
satisfied only by x1=x2=…=xn=0, which means that c1, c2,
…, cn are linearly independent.

2008/12/5 Elementary Linear Algebra 27

Proof of Theorem 5.6.8

 : Assume that Ax=0 has only the trivial

solution. Either Ax=b is consistent or it is not. If it is not
consistent, then there are no solutions of Ax=b, and we
are done.
 If Ax=b is consistent, let x0 by any solution. From
Theorem 5.5.2, we conclude that the general solution of
Ax=b is x0+0=x0. Thus the only solution of Ax=b is x0.

2008/12/5 Elementary Linear Algebra 28

Proof of Theorem 5.6.8

 : Assume that Ax=b has at most one solution

for every matrix b. Then, in particular, Ax=0 has at
most one solution. Thus Ax=0 has only trivial solution.

2008/12/5 Elementary Linear Algebra 29

Example

 A linear system with more unknowns than equations is

called an underdetermined linear system.
 A consistent underdetermined linear system must have
infinitely many solutions.
 An Undetermined System
 If A is a 57 matrix, then for every 71 matrix b, the linear
system Ax = b is undetermined. Thus, Ax = b must be consistent
for some b, and for each such b the general solution must have 7
–r parameters, where r is the rank of A.

2008/12/5 Elementary Linear Algebra 30

Theorem 5.6.9 (Equivalent
Statements)
 If A is an nn matrix, and if TA : Rn  Rn is multiplication by A, then the following are
equivalent:
 A is invertible.
 Ax = 0 has only the trivial solution.
 The reduced row-echelon form of A is In.
 A is expressible as a product of elementary matrices.
 Ax = b is consistent for every n1 matrix b.
 Ax = b has exactly one solution for every nn1 matrix b..
A
 det(A) ≠0.
 The range of TA is Rn.
 TA is one-to-one.
 The column vectors of A are linearly independent.
 The row vectors of A are linearly independent.
 The column vectors of A span Rn.
 The row vectors of A span Rn.
 The column vectors of A form a basis for Rn.
 The row vectors of A form a basis for Rn.
 A has rank n.
 A has nullity 0.

2008/12/5 Elementary Linear Algebra 31