Chapter 4

Vector Spaces

4.1 Vectors in Rn
4.2 Vector Spaces
4.3 Subspaces of Vector Spaces
4.4 Spanning Sets and Linear Independence
4.5 Basis and Dimension
4.6 Rank of a Matrix and Systems of Linear Equations
4.7 Coordinates and Change of Basis
4.8 Applications of Vector Spaces 4.1
 n
4.1 Vectors in R
 An ordered n-tuple (有序的n項) :
a sequence of n real numbers ( x1 , x2 , , xn )

n
 R -space (n維空間) :
the set of all ordered n-tuples
n=1 R1-space = set of all real numbers
(R1-space can be represented geometrically by the x-axis)

n=2 R2-space = set of all ordered pair of real numbers ( x1 , x2 )

(R2-space can be represented geometrically by the xy-plane)
n=3 R3-space = set of all ordered triple of real numbers ( x1 , x2 , x3 )
(R3-space can be represented geometrically by the xyz-space)
n=4 R4-space = set of all ordered quadruple of real numbers ( x1 , x2 , x3 , x4 )

4.2
 Notes:
(1) An n-tuple ( x1 , x2 ,, xn ) can be viewed as a point in Rn
with the xi’s as its coordinates (座標)
(2) An n-tuple ( x1 , x2 ,, xn ) also can be viewed as a vector
x  ( x1 , x2 , , xn ) in Rn with the xi’s as its components (分量)
 Ex:

x1 , x2  x1 , x2 
or

0,0
a point a vector
※ A vector on the plane is expressed geometrically by a directed line segment
whose initial point is the origin and whose terminal point is the point (x1, x2)
4.3
 u   u1 , u2 , , un  , v   v1, v2 , , vn  (two vectors in Rn)
 Equality:
u  v if and only if u1  v1 , u2  v2 , , un  vn

 Vector addition (the sum of u and v):

u  v  u1  v1 , u2  v2 , , un  vn 

 Scalar multiplication (the scalar multiple of u by c):

cu  cu1 , cu2 ,, cun 

 Notes:
The sum of two vectors and the scalar multiple of a vector
in Rn are called the standard operations in Rn
4.4
 Difference between u and v:
u  v  u  (1) v  (u1  v1 , u2  v2 , u3  v3 ,..., un  vn )

 Zero vector (零向量):

0  (0, 0, ..., 0)

4.5
 Notes:
A vector u  (u1 , u2 ,, un ) in R n can be viewed as:
Use comma to separate components

a 1×n row matrix (row vector): u  [u1 u2 un ]

Use blank space to separate entries

or u1 
u 
a n×1 column matrix (column vector): u   2 
 
 
u n 

※ Therefore, the operations of matrix addition and scalar multiplication

generate the same results as the corresponding vector operations (see the
next slide)
4.6
 Vector addition Scalar multiplication
u  v  (u1 , u2 , , un )  (v1 , v2 , , vn ) cu  c(u1 , u2 ,, un )
 (u1  v1 , u2  v2 ,  , un  vn )  (cu1 , cu2 , , cun )

Regarded as 1×n row matrix

u  v  [u1 u2 un ]  [v1 v2 vn ] cu  c[u1 u2 un ]
 [u1  v1 u2  v2 un  vn ]  [cu1 cu2 cun ]

Regarded as n×1 column matrix

u1  v1  u1  v1  u1  cu1 
u  v  u  v  u  cu 
u v   2   2   2 2 cu  c  2    2 
          
         
un  vn  un  vn  un  cun 
4.7
 Theorem 4.2: Properties of vector addition and scalar multiplication
Let u, v, and w be vectors in Rn, and let c and d be scalars
n
(1) u+v is a vector in R (closure (封閉性) under vector addition)
(2) u+v = v+u (commutative (交換律) property of vector addition)
(3) (u+v)+w = u+(v+w) (associative (結合律) property of vector addition)
(4) u+0 = u (additive identity property)
that –u is just the notation of the additive inverse
(5) u+(–u) = 0 (additive inverse property) (Note
of u, and –u = (–1)u will be proved in Thm. 4.4 )
(6) cu is a vector in Rn (closure (封閉性) under scalar multiplication)
(7) c(u+v) = cu+cv (distributive property (分配律) of scalar multiplication
over vector addition)
(8) (c+d)u = cu+du (distributive property (分配律) of scalar multiplication
over real-number addition)
(9) c(du) = (cd)u (associative (結合律) property of multiplication)
(10) 1(u) = u (multiplicative identity property)
※ Except Properties (1) and (6), these properties of vector addition and scalar
multiplication actually inherit the properties of matrix addition and scalar
multiplication in Ch 2 because we can regard vectors in Rn as special cases of matrices 4.8
 Ex 5: Practice standard vector operations in R4
Let u = (2, –1, 5, 0), v = (4, 3, 1, –1), and w = (–6, 2, 0, 3) be
vectors in R4. Solve x in each of the following cases.
(a) x = 2u – (v + 3w)
(b) 3(x+w) = 2u – v+x

Sol: (a) x  2u  ( v  3w )
 2u  (1)( v  3w )
 2u  v  3w (distributive property of scalar multiplication over vector addition)
 (4,  2, 10, 0)  (4, 3, 1,  1)  ( 18, 6, 0, 9)
 (4  4  18,  2  3  6, 10  1  0, 0  1  9)
 (18,  11, 9,  8)

4.9
(b) 3(x  w )  2u  v  x
3x  3w  2u  v  x (distributive property of scalar multiplication over vector addition)
3x  x  2u  v  3w (subtract (3w+x) from both sides)
2x  2u  v  3w
x  u  12 v  32 w (scalar multiplication for the both sides with a scalar to be 1/2)
  2, 1,5, 0    2, 23 , 21 , 12    9, 3, 0, 29 
  9, 211 , 92 , 4 

4.10
 Notes:
(1) The zero vector 0 in Rn is called the additive identity (加法單
位元素) in Rn (see Property 4)
(2) The vector –u is called the additive inverse (加法反元素) of
u (see Property 5)

Theorem 4.3: (Properties of additive identity and additive inverse)

Let v be a vector in Rn and c be a scalar. Then the following
properties are true
(1) The additive identity is unique, i.e., if v+u = v, u must be 0
(2) The additive inverse of v is unique, i.e., if v+u = 0, u must be –v
(3) 0v = 0 
 These three properties are valid for any vector
(4) c0 = 0 
 space and will be proved on Slides 4.22-4.23
(5) If cv = 0, either c = 0 or v = 0 
(Since –v + v = 0, the additive inverse of –v is v, i.e., v can be expressed as –(–v).
(6) –(–v) = v Note that v and –v are the additive inverses for each other) 4.11
 Linear combination (線性組合) in Rn:
The vector x is called a linear combination of v1 ,v 2 ,...,v n ,
if it can be expressed in the form
x  c1v1  c2 v 2   cn v n , where c1 , c2 , , cn are real numbers
 Ex 6:
Given x = (–1, –2, –2), u = (0,1,4), v = (– 1,1,2), and
w = (3,1,2) in R3, find a, b, and c such that x = au + bv + cw.
Sol:
 b  3c   1
a  b  c  2
4a  2b  2c   2
 a  1, b  2, c  1
Thus x  u  2 v  w 4.12
 Keywords in Section 4.1:
 ordered n-tuple: 有序的n項
 Rn-space: n維空間
 equal: 相等
 vector addition: 向量加法
 scalar multiplication: 純量乘法
 zero vector: 零向量
 additive identity: 加法單位元素
 additive inverse: 加法反元素
 linear combination: 線性組合

4.13
 4.2 Vector Spaces
 Vector spaces (向量空間):
Let V be a set on which two operations (addition and scalar
multiplication) are defined. If the following ten axioms are
satisfied for every element u, v, and w in V and every scalar (real
number) c and d, then V is called a vector space, and the
elements in V are called vectors

Addition:
(1) u+v is in V
(2) u+v = v+u
(3) u+(v+w) = (u+v)+w
(4) V has a zero vector 0 such that for every u in V, u+0 = u
(5) For every u in V, there is a vector in V denoted by –u
such that u+(–u) = 0
4.14
 Scalar multiplication:
(6) cu is in V
(7) c(u  v)  cu  cv
(8) (c  d )u  cu  du
(9) c(du)  (cd )u
(10) 1(u )  u

※ This type of definition is called an abstraction definition (抽象法定義)

because you abstract (抽取) a collection of properties from Rn to form the
axioms for defining a more general space V
※ Thus, we can conclude that Rn is of course a vector space

4.15
 Notes:

A vector space consists of four entities:

a set of vectors, a set of real-number scalars, and two operations

V: nonempty set of vectors

c: any scalar
(u, v)  u  v : vector addition
(c, u)  cu : scalar multiplication
V , ,  is called a vector space
※ The set V together with the definitions of vector addition and scalar
multiplication satisfying the above ten axioms is called a vector space

4.16
 Four examples of vector spaces are shown as follows. (It is
straightforward to show that these vector spaces satisfy the above ten axioms)
n
(1) n-tuple space: R
(u1 , u2 ,un )  (v1 , v2 , v2 )  (u1  v1 , u2  v2 ,un  vn ) (standard vector addition)
k (u1 , u2 ,un )  (ku1 , ku2 , kun ) (standard scalar multiplication for vectors)

(2) Matrix space (矩陣空間): V  M mn

(the set of all m×n matrices with real-number entries)
Ex: (m = n = 2)
u11 u12  v11 v12   u11  v11 u12  v12 
u u   v v   u  v u  v  (standard matrix addition)
 21 22   21 22   21 21 22 22 
u11 u12   ku11 ku12 
k    (standard scalar multiplication for matrices)
 21 22   21
u u ku ku22  4.17
(3) n-th degree or less polynomial space (多項式空間): V  Pn
(the set of all real-valued polynomials of degree n or less)
p( x)  q( x)  (a0  b0 )  (a1  b1 ) x    (an  bn ) x n (standard
addition)
polynomial

kp( x)  ka0  ka1 x    kan x n (standard scalar multiplication for

polynomials)
※ By the fact that the set of real numbers is closed under addition and
multiplication, it is straightforward to show that Pn satisfies the ten axioms
and thus is a vector space
(4) Continuous function space (函數空間): V  C (, )
(the set of all real-valued continuous functions defined on the
entire real line)
( f  g )( x)  f ( x)  g ( x) (standard addition for functions)
(kf )( x)  kf ( x) (standard scalar multiplication for functions)

※ By the fact that the sum of two continuous function is continuous and the
product of a scalar and a continuous function is still a continuous
function, C (, ) is a vector space
4.18
  Summary of important vector spaces
R  set of all real numbers
R 2  set of all ordered pairs
R 3  set of all ordered triples
R n  set of all n-tuples
C (, )  set of all continuous functions defined on the real number line
C[a, b]  set of all continuous functions defined on a closed interval [a, b]
P  set of all polynomials
Pn  set of all polynomials of degree  n
M m,n  set of m  n matrices
M n ,n  set of n  n square matrices
※ The standard addition and scalar multiplication operations are considered if there is
no other specification
※ Each element in a vector space is called a vector, so a vector can be a real number,
an n-tuple, a matrix, a polynomial, a continuous function, etc. 4.19
 Notes: To show that a set is not a vector space, you need
only find one axiom that is not satisfied
 Ex 6: The set of all integers (整數) is not a vector space
Pf: 1V , and 12 is a real-number scalar
( 12 )(1)  12 V (it is not closed under scalar multiplication)
  noninteger
scalar

integer

 Ex 7: The set of all (exact) second-degree polynomial functions is

not a vector space
Pf: Let p( x)  x 2 and q( x)   x 2  x  1
 p ( x)  q ( x)  x  1  V
(it is not closed under vector addition)
4.20
 Ex 8:
V=R2=the set of all ordered pairs of real numbers
vector addition: (u1 , u2 )  (v1 , v2 )  (u1  v1 , u2  v2 )
scalar multiplication: c(u1, u2 )  (cu1,0) (nonstandard definition)
Verify V is not a vector space
Sol:
This kind of setting can satisfy the first nine axioms of the
definition of a vector space (you can try to show that), but it
violates the tenth axiom
1(1, 1)  (1, 0)  (1, 1)
the set (together with the two given operations) is
not a vector space
4.21
 Theorem 4.4: Properties of additive identity and additive inverse
Let v be any element of a vector space V, and let c be any
scalar. Then the following properties are true
(1) 0 v  0
(2) c0  0
(3) If cv  0, either c  0 or v  0
(4) (1) v   v (the additive inverse of v equals ((–1)v)
※ The first three properties are extension of Theorem 4.3, which simply
considers the space of Rn. In fact, these four properties are not only
valid for Rn but also for any vector space, e.g., for all vector spaces
mentioned on Slide 4.19.
Pf:
(8) (9) (5)
(1) 0v  (c  (c)) v  cv  (c) v  cv  ((cv))  0
4.22
 (4) (7)
(2) c0  c(0  0)  c0  c0
 c0  (c0)  (c0  c0) + (c0) (add (–c0) to both sides)
(3)
 c0  (c0)  c0  [c0 + (c0)]
(5) (4)
 0 = c0  0  0 = c0
(3) Prove by contradiction: Suppose that cv  0, but c  0 and v  0
(10)
 1  (9) 1 1
v  1v   c  v   cv    0   0 (By the second property, c0 = 0)
c  c c
   if cv = 0, either c  0 or v  0
(8)
(4) 0 v  (1  (1)) v  1v  (1) v
 0  v  (1) v (By the first property, 0v = 0)
(5)
 (1) v   v (By comparing with Axiom (5), (–1)v is the additive inverse of v)

※ The proofs are valid as long as they are logical. It is not necessary to follow the
same proofs in the text book or the solution manual 4.23
 Keywords in Section 4.2:
 vector space: 向量空間
 n-space: n維空間
 matrix space: 矩陣空間
 polynomial space: 多項式空間
 function space: 函數空間

4.24
 4.3 Subspaces of Vector Spaces
 Subspace (子空間):
(V , , ) : a vector space
W  
 : a nonempty subset of V
W V
(W , , ) : The nonempty subset W is called a subspace if W is
a vector space under the operations of vector
addition and scalar multiplication defined on V
 Trivial subspace (顯然子空間):
Every vector space V has at least two subspaces
(1) Zero vector space {0} is a subspace of V (It satisfies the ten axioms)

(2) V is a subspace of V

※ Any subspaces other than these two are called proper (or nontrivial) subspaces
4.25
 Examination of whether W being a subspace
– Since the vector operations defined on W are the same as

those defined on V, and most of the ten axioms inherit the

properties for the vector operations, it is not needed to verify
those axioms
– To identify that a nonempty subset of a vector space is a

subspace, it is sufficient to test only the closure conditions

under vector addition and scalar multiplication (只需驗證
向量加法與純量乘法的封閉性)
 Theorem 4.5: Test whether a nonempty subset being a subspace
If W is a nonempty subset of a vector space V, then W is
a subspace of V if and only if the following conditions hold
(1) If 𝐮 and 𝐯 are in W, then 𝐮 + 𝐯 is in W
(2) If 𝐮 is in W and c is any scalar, then 𝑐𝐮 is in W 4.26
Pf:
1. Note that if u, v, and w are in W, then they are also in V.
Furthermore, W and V share the same operations.
Consequently, vector space axioms 2, 3, 7, 8, 9, and 10 are
satisfied automatically
2. Suppose that the closure conditions hold in Theorem 4.5, i.e.,
the axioms 1 and 6 for vector spaces are satisfied
3. Since the axiom 6 is satisfied (i.e., cu is in W if u is in W), we
can obtain
3.1. for a scalar c  0, cu  0  W   zero vector in W
 axiom 4 is satisfied
3.2. for a scalar c  1, (  1)u  W    u  (  1)u
st. u +(  u)  u +(  1)u  0
 axiom 5 is satisfied
4.27
 Ex 2: A subspace of M2×2
Let W be the set of all 2×2 symmetric matrices. Show that
W is a subspace of the vector space M2×2, with the
standard
Sol: operations of matrix addition and scalar multiplication
First, we knon that W , the set of all 2  2 symmetric matrices,
is an nonempty subset of the vector space M 22
Second,
A1 W, A2 W  ( A1  A2 )T  A1T  A2T  A1  A2 (A1  A2 W )
c  R, A W  (cA)T  cAT  cA (cA W )
The definition of a symmetric matrix A is that AT = A

Thus, Thm. 4.5 is applied to obtain that W is a subspace of M 22 4.28

 Ex 3: The set of singular matrices is not a subspace of M2×2
Let W be the set of singular (noninvertible) matrices of
order 2. Show that W is not a subspace of M2×2 with the
standard matrix operations

Sol:
1 0  0 0 
A  W , B    W
0 0  0 1 

1 0 
A B     I W (W is not closed under vector addition)
0 1 
W is not a subspace of M 22

4.29
 Ex 4: The set of first-quadrant vectors is not a subspace of R2
Show that W  {( x1 , x2 ) : x1  0 and x2  0} , with the standard
operations, is not a subspace of R2

Sol:
Let u  (1, 1) W
1u  11, 1  1, 1W
(W is not closed under scalar multiplication)

W is not a subspace of R 2

4.30
 2
 Ex 6: Identify subspaces of R
Which of the following two subsets is a subspace of R2?
(a) The set of points on the line given by 𝑥 + 2𝑦 = 0
(b) The set of points on the line given by 𝑥 + 2𝑦 = 1
Sol:
(a) W  ( x, y) x  2 y  0  (2t, t ) t  R
(Note: the zero vector
(0,0) is on this line)

Let v1   2t1 , t1  W and v2   2t2 , t2  W

v1  v2   2  t1  t2  , t1  t2  W (closed under vector addition)

cv1   2  ct1  , ct1  W (closed under scalar multiplication)

W is a subspace of R 2
4.31
 (b) W   x, y  x  2 y  1 (Note: the zero vector (0, 0) is not on this line)

Consider v  (1, 0) W
   
1 v  1,0 W W is not a subspace of R 2

 Notes: Subspaces of R2
(1) W consists of the single point 0   0, 0 (trivial subspace)
(2) W consists of all points on a line passing through the origin
(3) R 2 (trivial subspace)

4.32
 Ex 8: Identify subspaces of R3
Which of the following subsets is a subspace of R 3?
(a) W  ( x1 , x2 ,1) x1 , x2  R (Note: the zero vector is not in W)
(b) W  ( x1 , x1  x3 , x3 ) x1 , x3  R (Note: the zero vector is in W)
Sol:
(a)

Consider v  (0, 0,1)  W

(1) v  (0, 0, 1)  W
W is not a subspace of R3

4.33
(b)

Consider v  (v1 , v1  v3 , v3 ) W and u  (u1 , u1  u3 , u3 ) W

v  u   v1  u1 ,  v1  u1    v3  u 3  , v3  u3  W
cv   cv1 ,  cv1    cv3  , cv3   W
W is closed under vector addition and scalar multiplication,
so W is a subspace of R 3
4.34
 Notes: Subspaces of R3
(1) W consists of the single point 0   0,0,0 (trivial subspace)

(2) W consists of all points on a line passing through the origin

(3) W consists of all points on a plane passing through the origin
(The W in problem (b) is a plane passing through the origin)
(4) R 3 (trivial subspace)

※ According to Ex. 6 and Ex. 8, we can infer that if W is a subspace of a

vector space V, then both W and V must contain the same zero vector 0

4.35
 Theorem 4.6: The intersection (交集) of two subspaces is a subspace
If V and W are both subspaces of a vector space U ,
then the intersection of V and W (denoted by V  W )
is also a subspace of U
Pf:
(1) For v1 and v 2 in V  W , since v1 and v 2 are in V (and W ),
v1  v 2 is in V (and W ). Therefore, v1  v 2 is in V  W
(2) For v1 in V  W , since v1 is in V (and W ),
cv1 is in V (and W ). Therefore, cv1 is in V  W

Consequently, we can conclude the intersection of V and W

(V  W ) is also a subspace of U
※ However, the union (聯集) of two subspaces is not a subspace (see an
exercise problem in Section 4.3) 4.36
Keywords in Section 4.3:
 subspace: 子空間
 trivial subspace: 顯然子空間

4.37
 4.4 Spanning Sets and Linear Independence
 This section introduces the spanning set, linear independence, and
linear dependence
 The above three notions are associated with the representation of
any vector in a vector space as a linear combination of a selected
set of vectors in that vector space

 Linear combination (線性組合):

A vector u in a vector space V is called a linear combination of
the vectors v1 , v 2 , , v k in V if u can be written in the form

u  c1v1  c2 v 2   ck v k ,

where c1 , c2 , , ck are real-number scalars

4.38
 Ex 2 and 3: Finding a linear combination
v1  (1,2,3) v 2  (0,1,2) v 3  (  1,0,1)
Prove (a) w  (1,1,1) is a linear combination of v1 , v 2 , v 3
(b) w  (1,  2,2) is not a linear combination of v1 , v 2 , v3
Sol:
(a) w  c1v1  c2 v 2  c3 v3
1,1,1  c1 1,2,3  c2 0,1,2  c3 1,0,1
 (c1  c3 , 2c1  c2 , 3c1  2c2  c3 )

c1  c3 1
 2c1  c2 1
3c1  2c2  c3 1

4.39
 1 0  1 1 1 0  1 1 

 2 1 0 1  
G.-J. E.
 0 1 2  1
 
3 2 1 1 0 0 0 0 

 c1  1  t , c2  1  2t , c3  t

(this system has infinitely many solutions)

t 1
 w  2 v1  3 v 2  v 3
t 2
 w  3v1  5v 2  2 v 3

4.40
(b)
w  c1 v1  c2 v 2  c3 v 3

1 0  1 1  1 0  1 1 
 2 1 0  2 
G.-J. E.
 0 1 2  4
 
3 2 1 2  0 0 0 7 

 This system has no solution since the third row means

0  c1  0  c2  0  c3  7
 w can not be expressed as c1v1  c2 v 2  c3 v3

4.41
 The span of a set: span(S)
If S = {v1, v2,…, vk} is a set of vectors in a vector space V,
then the span of S is the set of all linear combinations of
the vectors in S,
span(S )  c1v1  c2 v 2   ck v k ci  R
(the set of all linear combinations of vectors in S )

 Definition of a spanning set of a vector space:

If every vector in a given vector space V can be written as a
linear combination of vectors in a set S, then S is called a
spanning set (生成集合) of the vector space V

4.42
 Notes: The above statement can be expressed as follows
span( S )  V
 S spans (generates) V
 V is spanned (generated) by S
 S is a spanning set of V
 Ex 4:
(a) The set S  {(1, 0, 0), (0,1, 0), (0, 0,1)} spans R3 because any vector
u  (u1 , u2 , u3 ) in R 3 can be written as
u  u1 (1, 0, 0)  u2 (0,1, 0)  u3 (0, 0,1)

(b) The set S  {1, x, x 2 } spans P2 because any polynomial function

p( x)  a  bx  cx 2 in P2 can be written as
p ( x)  a (1)  b( x)  c( x 2 ) 4.43
 Ex 5: A spanning set for R3
Show that the set S  (1,2,3),(0,1,2),(2,0,1) spans R3
Sol:
We must examine whether any vector u  (u1 , u2 ,u3 ) in R 3
can be expressed as a linear combination of v1  (1, 2,3),
v 2  (0,1, 2), and v 3  (2, 0,1)
If u  c1v1  c2 v 2  c3 v3  c1  2c3  u1
2c1  c2  u2
3c1  2c2  c3  u3

The above problem thus reduces to determine whether this

system is consistent for all values of u1 , u2 , and u3
4.44
 1 0 2 ※ From Thm. 2.11, if A is an invertible matrix, then the
system of linear equations Ax = b has a unique solution
 A  2 1 0 0 (x = A-1b) given any b
3 2 1 ※ From Thm. 3.7, a square matrix A is invertible
(nonsingular) if and only if det (A)  0
 Ax  u has exactly one solution for every u

 span( S )  R3

 Notes:
※ For any set S1 containing the set S, if S can span R3, S1 can span R3 as well
(e.g., S1  v1 , v2 , v3 , v4   (1, 2,3),(0,1, 2),(2,0,1),(1,0,0)).
※ Actually, in this case, what S1 can span is only R3. Since v1, v2, and v3
span R3, v4 must be a linear combination of v1, v2, and v3. So, adding v4
will not generate more combinations. As a consequence, v1, v2, v3, and v4
can only span R3

4.45
 Theorem 4.7: span(S) is a subspace of V
If S = {v1, v2,…, vk} is a set of vectors in a vector space V, then
(a) span(S) is a subspace of V
(b) span(S) is the smallest subspace of V that contains S, i.e.,
every other subspace of V containing S must contain span(S)
any subspace
containing S

 v1  

    vk



 v3   v2

span(S)
V

※ For example, V = R5, S = {(1, 0, 0, 0, 0), (0, 1, 0, 0, 0), (0, 0, 1, 0, 0)} and
thus span(S) = R3, and U = R4, U contains S and contains span(S) as well
4.46
Pf:
(a)
Consider any two vectors u and v in span(S ), that is,
u =c1 v1  c2 v 2   ck v k and v =d1 v1  d 2 v 2   d k v k

Then u +v =(c1 +d1 ) v1  (c2 +d 2 ) v 2   (ck +d k ) v k  span(S )

and cu =(cc1 ) v1  (cc2 ) v 2   (cck ) v k  span(S ), because they
can be written as linear combinations of vectors in S

So, according to Theorem 4.5, we can conclude that span(S ) is

a subspace of V

4.47
(b)
Let U be another subspace of V that contains S , and we want to
show span(S )  U
k
Consider any u  span(S ), i.e., u =  ci v i , where v i  S
i =1
(because U is closed under
U is a subspace k vector addition and scalar
U contains S  v i  U  u =  ci v i  U multiplication, and any linear
combination can be evaluated
i =1 with finite vector additions
and scalar multiplications)

Since for any vector u  span(S ), u also belongs to U ,

we can conclude that span(S )  U , and therefore span(S )
is the smallest subspace of V that contains S  {v1 , v 2 , , v k }

4.48
 Definitions
of Linear Independence (L.I.) (線性獨立) and Linear
Dependence (L.D.) (線性相依):

S  v1 , v2 , , vk : a set of vectors in a vector space V

For c1v1  c2 v 2   ck v k  0
(1) If the equation has only the trivial solution (c1  c2   ck  0)
then S (or v1 , v 2 , , v k ) is called linearly independent
(2) If the equation has a nontrivial solution (i.e., not all ci 's are zero),
then S (or v1 , v 2 , , v k ) is called linearly dependent (The name of
linear dependence is from the fact that in this case, there exist at least
one v i which can be represented by the linear combination of {v1 , v 2 ,
, v i 1 , v i 1 , v k } in which the coefficients are not all zeros. This
statement will be proved in Theorem 4.8 on Slide 4.54) 4.49
 Ex 8: Testing for linear independence
Determine whether the following set of vectors in R3 is L.I. or L.D.
S  v1 , v 2 , v 3   1, 2, 3 ,  0, 1, 2  ,  2, 0, 1
Sol: c1  2c3  0
c1v1  c2 v 2  c3 v 3  0  2c1  c2  0
3c1  2c2  c3  0
1 0  2 0 1 0 0 0 
 2 1 0 0  G.-J. E. 
 0 1 0 0 

3 2 1 0 0 0 1 0 
 c1  c2  c3  0 only the trivial solution 
(or det(A)  1  0, so there is only the trivial solution)
 S is (or v1, v 2 , v3 are) linearly independent 4.50
 Ex 9: Testing for linear independence
Determine whether the following set of vectors in P2 is L.I. or L.D.

S  v1 , v 2 , v 3   1  x  2 x 2 ,2  5 x  x 2 ,x  x 2 
Sol:
c1v1+c2v2+c3v3 = 0
i.e., c1(1+x – 2x2) + c2(2+5x – x2) + c3(x+x2) = 0+0x+0x2

c1+2c2 =0  1 2 0 0 1 2 0 0 
 1 5 1 0    
 c1+5c2+c3 = 0   
G. E.
 0 1 3 0 
1
–2c1– c2+c3 = 0  2 1 1 0   
 0 0 0 0 
 This system has infinitely many solutions
(i.e., this system has nontrivial solutions, e.g., c1=2, c2= – 1, c3=3)
 S is (or v1, v2, v3 are) linearly dependent
4.51
 Ex 10: Testing for linear independence
Determine whether the following set of vectors in the 2×2
matrix space is L.I. or L.D.
2 1  3 0 1 0 
S  v1 , v 2 , v3     ,  ,  
0 1  2 1  2 0 

Sol:
c1v1+c2v2+c3v3 = 0

2 1  3 0 1 0 0 0
c1    c2    c3    
 0 1  2 1   2 0   0 0 

4.52
 2c1+3c2+ c3 = 0
c1 =0
2c2+2c3 = 0
c1 + c2 =0

2 3 1 0 1 0 0 0
1 0 0 0 G.-J. E. 0 1 0 0
   
0 2 2 0  0 0 1 0
   
1 1 0 0 0 0 0 0

 c1 = c2 = c3= 0 (This system has only the trivial solution)

 S is linearly independent

4.53
 Theorem 4.8: A property of linearly dependent sets
A set S = {v1,v2,…,vk}, for k  2, is linearly dependent if
and only if at least one of the vectors vi in S can be written
as a linear combination of the other vectors in S

Pf:
()
c1v1+c2v2+…+civi+…+ckvk = 0

S is linearly dependent (there exist nontrivial solutions)

 ci  0 for some i
 c1   ci 1   ci 1   ck 
 vi     v1    vi 1     vi 1    vk
 ci   ci   ci   ci  4.54
 ()
Let vi = d1v1+…+di-1vi-1+di+1vi+1+…+dkvk

 d1v1+…+di-1vi-1+(–1)vi +di+1vi+1+…+dkvk = 0
 c1 = d1 , c2 = d2 ,…, ci = –1 ,…, ck = dk (there exits at least this
nontrivial solution)
 S is linearly dependent

(A corollary is a must-be-true result based on

 Corollary to Theorem 4.8: the already proved theorem)
Two vectors u and v in a vector space V are linearly dependent
(for k = 2 in Theorem 4.8) if and only if one is a scalar multiple
of the other

4.55
Keywords in Section 4.4:
 linear combination: 線性組合
 spanning set: 生成集合
 trivial solution: 顯然解
 linear independence: 線性獨立
 linear dependence: 線性相依

4.56
 4.5 Basis and Dimension
 Basis (基底): Linearly
Spanning Bases
V: a vector space Sets
Independent
Sets
S = {v1, v2, …, vn}V
V
(1) S spans V (i.e., span(S) = V)
 (For any u  V ,  ci v i  Ax  u has at least one solution (If there is

 exact one solution (det( A)  0) or if there are infinitely many
 solutions (det( A)  0))
(2) S is linearly independent
(For  ci v i = Ax  0, there is only the trivial solution (det(A)  0).
See the definition on Slide 4.49 and Ex 8 on Slide 4.50)

 S is called a basis for V ( For  ci vi = Ax, det(A)  0)

 Notes:
A basis S must have enough vectors to span V, but not so
many vectors that one of them could be written as a linear
combination of the other vectors in S 4.57
 Notes:
(1) the standard basis for R3:
{i, j, k}, for i = (1, 0, 0), j = (0, 1, 0), k = (0, 0, 1)

(2) the standard basis for Rn :

{e1, e2, …, en}, for e1 = (1,0,…,0), e2 = (0,1,…,0),…, en = (0,0,…,1)
Ex: For R4, {(1,0,0,0), (0,1,0,0), (0,0,1,0), (0,0,0,1)}

※Express any vector in Rn as the linear combination of the vectors in the

standard basis: the coefficient for each vector in the standard basis is the
value of the corresponding component of the examined vector, e.g., (1, 3,
2) can be expressed as 1·(1, 0, 0) + 3·(0, 1, 0) + 2·(0, 0, 1)

4.58
(3) the standard basis for mn matrix space:
 aij  1
{ Eij | 1im , 1jn }, and in Eij 
other entries are zero
Ex: 2 2 matrix space:
1 0 0 1 0 0 0 0 
  ,  ,  ,  
0 0 0 0 1 0 0 1 

(4) the standard basis for Pn(x):

{1, x, x2, …, xn}
Ex: P3(x): {1, x, x2, x3}

4.59
 Ex 2: The nonstandard basis for R2
Show that S  {v1 ,v 2 }  {(1,1), (1, 1)} is a basis for R 2

 c1  c2  u1
(1) For any u  (u1 , u2 )  R , c1 v1  c2 v 2  u  
2

c1  c2  u2
Because the coefficient matrix of this system has a nonzero determinant, the
system has a unique solution for each u. Thus you can conclude that S spans R2

c1  c2  0
(2) For c1 v1  c2 v 2  0  
c1  c2  0
Because the coefficient matrix of this system has a nonzero determinant, you
know that the system has only the trivial solution. Thus you can conclude that S is
linearly independent

According to the above two arguments, we can conclude that S

2
is a (nonstandard) basis for R
4.60
 Theorem 4.9: Uniqueness of basis representation for any vectors
If S  v1 , v 2 ,, v n  is a basis for a vector space V, then every
vector in V can be written in one and only one way as a linear
combination of vectors in S
Pf:
 (1) span(S) = V
 S is a basis  
 (2) S is linearly independent
 span(S) = V Let v = c1v1+c2v2+…+cnvn
v = b1v1+b2v2+…+bnvn
 v + (–1)v = 0 = (c1– b1)v1 + (c2 – b2)v2 + … + (cn – bn)vn
S is linearly independent  with only the trivial solution
 coefficients for vi are all zero
 c1 = b1 , c2 = b2 ,…, cn = bn (i.e., unique basis representation)
4.61
Theorem 4.10: Bases and linear dependence


If S  v1 , v 2 ,, v n  is a basis for a vector space V, then

every set containing more than n vectors in V is linearly
dependent (In other words, every linearly independent set
contains at most n vectors)
Pf:
Let S1 = {u1, u2, …, um} , m > n
span( S )  V
u1  c11v1  c21v 2    cn1 v n
uiV  u 2  c12 v1  c22 v 2    cn 2 v n

u m  c1m v1  c2 m v 2    cnm v n
4.62
Consider k1u1+k2u2+…+kmum= 0
(if ki’s are not all zero, S1 is linearly dependent)
 d1v1+d2v2+…+dnvn= 0 (di = ci1k1+ci2k2+…+cimkm)

 S is L.I.  di = 0 i i.e., c11k1  c12k 2    c1m k m  0

c21k1  c22k 2    c2 m k m  0

cn1k1  cn 2 k 2    cnm k m  0

 Theorem 1.1: If the homogeneous system has fewer

equations (n equations) than variables (k1, k2, …, km), then
it must have infinitely many solutions
 m > n  k1u1+k2u2+…+kmum = 0 has nontrivial (nonzero) solution
 S1 is linearly dependent 4.63
Theorem 4.11: Number of vectors in a basis
If a vector space V has one basis with n vectors, then every
basis for V has n vectors
Pf: ※ According to Thm. 4.10, every linearly independent set contains at most n vectors
in a vector space if there is a basis of n vectors spanning that vector space
S = {v1, v2, …, vn}
are two bases with different sizes for V
S' = {u1, u2, …, um}
S is a basis spanning V  
  m  n
S ' is a set of L.I. vectors  
n m
S ' is a basis spanning V  
  n  m
S is a set of L.I. vectors  
※ For R3, since the standard basis {(1, 0, 0), (0, 1, 0), (0, 0, 1)} can span this vector
space, you can infer any basis that can span R3 must have exactly 3 vectors
※ For example, S = {(1, 2, 3), (0, 1, 2), (–2, 0, 1)} in Ex 5 on Slide 4.44 is another basis
of R3 (because S can span R3 and S is linearly independent), and S has 3 vectors 4.64
 Dimension:
The dimension of a vector space V is defined to be the
number of vectors in a basis for V
V: a vector space S: a basis for V

 dim(V) = #(S) (the number of vectors in a basis S)

 Finite dimensional:
A vector space V is finite dimensional if it has a basis
consisting of a finite number of elements

 Infinite dimensional:
If a vector space V is not finite dimensional, then it is called
infinite dimensional
4.65
 Notes: dim(V) = n
Linearly
(1) dim({0}) = 0 Spanning
Bases Independent
Sets Sets
(If V consists of the zero vector alone,
the dimension of V is defined as zero)

(2) Given dim(V) = n, for SV #(S)  n #(S) = n #(S)  n

S: a spanning set  #(S)  n (from Ex 5 on Slides 4.44 and 4.45)

S: a L.I. set  #(S)  n (from Theorem 4.10)
(Since a basis is defined to be a set of L.I.
S: a basis  #(S) = n vectors that can spans V, #(S) = dim(V) = n)
(see the above figure)

(3) Given dim(V) = n, if W is a subspace of V  dim(W)  n

※ For example, if V = R3, you can infer the dim(V) is 3, which is the number of
vectors in the standard basis
※ Considering W = R2, which is a subspace of R3, due to the number of vectors in
the standard basis, we know that the dim(W) is 2, that is smaller than dim(V) = 3 4.66
 Ex: Find the dimension of a vector space according to the
standard basis
※ The simplest way to find the dimension of a vector space is to count the
number of vectors in the “standard” basis for that vector space

(1) Vector space Rn  standard basis {e1 , e2 ,  , en}

 dim(Rn) = n
(2) Vector space Mmn  standard basis {Eij | 1im , 1jn}
 aij  1
and in Eij 
other entries are zero
 dim(Mmn) = mn
(3) Vector space Pn(x)  standard basis {1, x, x2,  , xn}
 dim(Pn(x)) = n+1
(4) Vector space P(x)  standard basis {1, x, x2, }
 dim(P(x)) =  4.67
 Ex 9: Determining the dimension of a subspace of R3
(a) W = {(d, c – d, c): c and d are real numbers}
(b) W = {(2b, b, 0): b is a real number}
(Hint: find a set of L.I. vectors that spans the subspace, i.e., find a basis for the
Sol: subspace.)
(a) (d, c – d, c) = c(0, 1, 1) + d(1, – 1, 0)
 S = {(0, 1, 1) , (1, – 1, 0)} (S is L.I. and S spans W)
 S is a basis for W
 dim(W) = #(S) = 2
(b) (2b, b, 0)  b(2,1, 0)
 S = {(2, 1, 0)} spans W and S is L.I.
 S is a basis for W
 dim(W) = #(S) = 1
4.68
 Ex 11: Finding the dimension of a subspace of M22
Let W be the subspace of all symmetric matrices in M22.
What is the dimension of W?
Sol:
a b  
W    a , b, c  R 
b c  
a b  1 0 0 1 0 0
   a   b   c 
 b c   0 0   1 0   0 1 
1 0 0 1 0 0 
 S    ,  ,   spans W and S is L.I.
0 0 1 0 0 1 
 S is a basis for W  dim(W) = #(S) = 3
4.69
 Theorem 4.12: Methods to identify a basis in an n-dimensional
space
Let V be a vector space of dimension n
(1) If S  v1, v2 , , vn  is a linearly independent set of
vectors in V, then S is a basis for V
(2) If S  v1, v2 , , vn  spans V, then S is a basis for V
(Both results are due to the fact that #(S) = n)

dim(V) = n

Linearly
Spanning
Bases Independent
Sets
Sets

#(S)  n #(S)  n
#(S) = n 4.70
Keywords in Section 4.5:
 basis: 基底
 dimension: 維度
 finite dimension: 有限維度
 infinite dimension: 無限維度

4.71
4.6 Rank of a Matrix and Systems of Linear Equations

 In this section, three vector spaces are investigated

 Row space: the vector space spanned by the row vectors of a matrix A
 Column space: the vector space spanned by column vectors of a matrix A
 Nullspace: the vector space consisting of all solutions of the
homogeneous system of linear equations Ax = 0
 Next, discuss the basis and the dimension of each vector space
 Finally, the relationship between the solutions of Ax = b and Ax
= 0 will be discussed
 To begin the introduction, I present the notation for the row and
column vectors of a matrix A with the size m×n on the next slide

4.72
 row vectors: (with size 1×n) row vectors of A
 a11 a12  a1n   A1  (a11 a12 a1n )  A(1)
a a  a  A  (a21 a22 a2n )  A(2)

2  
A   21 22 2n 

       
   
am1 am 2  amn   m  
A (am1 am 2 amn )  A(m )
※ So, the row vectors are vectors in Rn
 column vectors: (with size m×1) column vectors of A
 a11 a12 a1n   a11   a12   a1n 
a a22 a2 n  a   a  a 
A  21
  A1 A 2 A n    21   22    2 n 
         
      
 am1 am 2 amn  am1  am 2  amn 
|| || ||
(1) (2) (n)
※ So, the column vectors are vectors in Rm A A A
4.73
Let A be an m×n matrix:
 Row space (列空間) of A is a subspace of Rn spanned by the
row vectors of A:
RS ( A)  { 1 A(1)   2 A( 2 )  ...   m A( m ) |  1 ,  2 ,...,  m  R}
(If A(1), A(2), …, A(m) are linearly independent, A(1), A(2), …, A(m) can form a basis for RS(A))

 Column space (行空間) of A is a subspace of Rm spanned by the

column vectors of A:
CS ( A)  {1 A(1)   2 A(2)     n A( n ) 1 ,  2 ,   n  R}
(If A(1), A(2), …, A(n) are linearly independent, A(1), A(2), …, A(n) can form a basis for CS(A))

 Notes:
(1) The definitions of RS(A) and CS(A) satisfy automatically the
closure conditions of vector addition and scalar multiplication
(2) dim(RS(A)) (or dim(CS(A)) equals the number of linearly
independent row (or column) vectors of A 4.74
 In Ex 5 of Section 4.4, S = {(1, 2, 3), (0, 1, 2), (–2, 0, 1)} spans R3.
Use these vectors as row vectors to construct A
 1 2 3
A   0 1 2   RS ( A)  R3
 2 0 1 
(Since (1, 2, 3), (0, 1, 2), (–2, 0, 1) are linearly independent, they can form a basis for RS(A))

 Since S1 = {(1, 2, 3), (0, 1, 2), (–2, 0, 1), (1, 0, 0)} also spans R3,
1 2 3
0 1 2 
A1    RS ( A1 )  R 3
 2 0 1
 
1 0 0
(Since (1, 2, 3), (0, 1, 2), (–2, 0, 1) (1, 0, 0) are not linearly independent, they cannot be a
basis for RS(A1))
 Notes: dim(RS(A)) = 3 and dim(RS(A1)) = 3 4.75
 Theorem 4.13: Row-equivalent matrices have the same row space
If an mn matrix A is row equivalent to an mn matrix B,
then the row space of A is equal to the row space of B
Pf:
(1) Since B can be obtained from A by elementary row operations,
the row vectors of B can be expressed as linear combinations of
the row vectors of A  The linear combinations of row vectors
in B must be linear combinations of row vectors in A  any
vector in RS(B) lies in RS(A)  RS(B)  RS(A)
(2) Since A can be obtained from B by elementary row operations,
the row vectors of A can be written as linear combinations of
the row vectors of B  The linear combinations of row vectors
in A must be linear combinations of row vectors in B  any
vector in RS(A) lies in RS(B)  RS(A)  RS(B)
 RS ( A) = RS ( B) 4.76
  Notes:
(1) The row space of a matrix is not changed by elementary
row operations
RS(r(A)) = RS(A) r: any elementary row operation
(2) But elementary row operations will change the column space
 Theorem 4.14: Basis for the row space of a matrix
If a matrix A is row equivalent to a matrix B in the (reduced)
row-echelon form, then the nonzero row vectors of B form a
basis for the row space of A
1. The row space of A is the same of the row space of B (Thm. 4.13), spanned by all row vectors in B
2. For the row space of B, it can be constructed by the linear combinations of only nonzero row
vectors since it is impossible to generate more combinations when taking zero row vectors into
consideration (i.e., nonzero row vectors span the row space of B)
3. Since it is impossible to express a nonzero row vector as the linear combination of other nonzero
row vectors in a row-echelon form matrix (see Ex. 2 on the next slide), according to Thm. 4.8, we
can conclude that the nonzero row vectors in B are linearly independent
4. As a consequence, since the nonzero row vectors in B are linearly independent and span the row
space of B, according to the definition on Slide 4.57, they form a basis for the row space of B and
for the row space of A as well 4.77
 Ex 2: Finding a basis for a row space
 1 3 1 3
 0 1 1 0

Find a basis of the row space of A =  3 0 6  1
 
 3 4 2 1
 2 0  4 2

Sol:  1 3 1 3 1 3 1 3 w 1
 0 1 1 0 0 1 1 0 w 2
 
 3 0 6  1  1 w 3
A=   
G. E.
 B = 0 0 0

 3 4 2 1 0 0 0 0
 2 0  4 2 0 0 0 0
a1 a2 a3 a4 b1 b2 b3 b4

4.78
 a basis for RS(A) = {the nonzero row vectors of B} (Thm 4.14)
= {w1, w2, w3} = {(1, 3, 1, 3), (0, 1, 1, 0), (0, 0, 0, 1)}
(Check: w1, w2, w3 are linearly independent, i.e., aw1 + bw2 + cw3 = 0 has only the
trivial solution or it is impossible to express any one of them to be the linear
combination of the others (Theorem 4.8))
 Notes:
Although row operations can change the column space of a
matrix (mentioned in Slide 4.77), they do not change the
dependency relationships among columns
(1) {b1 , b 2 , b 4 } is L.I. (because these columns have the leading 1's)
 {a1 , a2 , a4 } is L.I.
(2) b3  2b1  b2  a3  2a1  a2
(The linear combination relationships among column vectors in B still hold for
column vectors in A)
4.79
 Ex 3: Finding a basis for a subspace using Thm. 4.14
Find a basis for the subspace of R3 spanned by
v1 v2 v3
S  {(1, 2, 5), (3, 0, 3), (5, 1, 8)}
Sol:
 1 2 5 v 1  1  2 5 w1
A =  3 0 3 v 2 G. E.
B  0 1 3 w 2
   
 5 1 8 v 3 0 0 0
(Construct A such that RS(A) = span(S))

a basis for span({v1, v2, v3})

= a basis for RS(A)
= {the nonzero row vectors of B} (Thm 4.14)
= {w1, w2}
= {(1, –2, – 5) , (0, 1, 3)}
4.80
 Ex 4: Finding a basis for the column space of a matrix
Find a basis for the column space of the matrix A given in Ex 2
 1 3 1 3
 0 1 1 0

A   3 0 6  1
 
 3 4 2 1
 2 0  4  2
Sol. 1:
 Since CS(A)=RS(AT), to find a basis for the column space of
the matrix A is equivalent to find a basis for the row space of
the matrix AT
1 0 3 3 2  1 0 3 3 2  w1
3 1 0 4 0  0 1 9 5 6  w 2
AT   
G. E.
 B
1 1 6 2 4  0 0 1 1 1 w 3
   
3 0 1 1 2  0 0 0 0 0  4.81
 a basis for CS(A)
= a basis for RS(AT)
= {the nonzero row vectors of B}
= {w1, w2, w3}
 1   0   0 
   1   0 
 0      

  3,  9 ,  1   (a basis for the column space of A)
 3     
    5  1 
 2   6  1 


4.82
 Sol. 2: 1 3 1 3 1 3 13
0 1 1 0  0 1 10 
 
A   3 0 6 1 G. E.
 B  0 0 01
   
3 4 2 1  0 0 00
 2 0 4 2  0 0 00 
a1 a2 a3 a4 b1 b2 b3 b4
Leading 1’s  {b1, b2, b4} is a basis for CS(B) (not for CS(A))
{a1, a2, a4} is a basis for CS(A)
※ This method utilizes that B is with the same dependency relationships among columns as
A (mentioned on Slides 4.77 and 4.79), which does NOT mean CS(B) = CS(A)

 Notes:
The bases for the column space derived from Sol. 1 and Sol. 2
are different. However, both these bases span the same CS(A),
which is a subspace of R5 4.83
 Theorem 4.16: The definition of the nullspace (零空間)
If A is an mn matrix, then the set of all solutions of the
homogeneous system of linear equations Ax = 0 is a subspace of Rn
called the nullspace of A, which is denoted as NS ( A)  {x  Rn | Ax  0}
Pf:
NS ( A)  R n
NS ( A) is not empty ( A0  0)
Let x1 , x 2  NS ( A) (i.e., Ax1  0 and Ax 2  0)
Then (1) A(x1  x 2 )  Ax1  Ax 2  0  0  0 (closure under addition)

(2) A(cx1 )  c( Ax1 )  c(0)  0 (closure under scalar multiplication)

Thus NS ( A) is a subspace of R n
Notes: The nullspace of A is also called the solution space (解
空間) of the homogeneous system Ax = 0 4.84
  Ex 6: Finding the solution space (or the nullspace) of a
homogeneous system with the coefficient matrix A as follows.
1 2 2 1 
A  3 6 5 4
1 2 0 3
Sol: The nullspace of A is the solution space of Ax = 0
1 2 2 1 0  1 2 0 3 0 
augmented matrix  3 6 5 4 0   G.-J. E.
 0 0 1 1 0 
1 2 0 3 0  0 0 0 0 0 
 x1 = –2s – 3t, x2 = s, x3 = –t, x4 = t
 x1   2 s  3t   2  3
x   s   1  0
 x   2     s    t    sv 1  t v 2
 x3    t   0   1
       
 4 
x t   0  1
 NS ( A)  {sv1  tv 2 | s, t  R} 4.85
 Theorem 4.15: Row and column space have equal dimensions
If A is an mn matrix, then the row space and the column
space of A have the same dimension
dim(RS(A)) = dim(CS(A))
※You can verify this result numerically through comparing Ex 2 (bases for
the row space) with Ex 4 (bases for the column space) in this section. In
these two examples, A is a 54 matrix, dim(RS(A)) = #(basis for RS(A)) = 3,
and dim(CS(A)) = #(basis for CS(A)) = 3

 Rank (秩):
The dimension of the row (or column) space of a matrix A
is called the rank of A
rank(A) = dim(RS(A)) = dim(CS(A))
 Nullity (核次數):
The dimension of the nullspace of A is called the nullity of A
nullity(A) = dim(NS(A)) 4.86
  Theorem 4.17: Dimension of the solution space
If A is an mn matrix of rank r, then the dimension of
the solution space of Ax = 0 (the nullsapce of A) is n – r, i.e.,
n = rank(A) + nullity(A) = r + (n – r)
(n is the number of columns or the number of unknown variables)
Pf:
Since rank(A)  r , the reduced row echelon form of [A|0] after G.-J. E. should be
1 0 0 c11 c12 c1,n-r 0  1. From Thm 4.14, the nonzero row
0 1 0 c c c2,n-r 0  
vectors of B, which is the (reduced)
 21 22
r row-echelon form of A, forms a
  basis for RS(A)
 
0 0 1 cr1 cr 2 cr ,n-r 0   2. From the definition that rank(A) =
0 0 0 0 0 0 0  dim(RS(A)) = r
   m  r ※ According to the above two facts,
  the reduced row echelon form
0 0 0 0 0 0  
 0 should have r nonzero rows like
r nr the left matrix 4.87
 Therefore, the corresponding system of linear equations is
x1  c11 xr 1 + c12 xr  2 + + c1, n  r xn = 0
x2 + c21 xr 1 + c22 xr  2 + + c2, n  r xn = 0

xr + cr1 xr 1 + cr 2 xr  2 + + cr , n  r xn = 0

Solving for the first r variables in terms of the last n – r parametric

variables produces n – r vectors in the basis of the solution space.
Consequently, the dimension of the solution space of Ax = 0 is n – r
because there are n – r free parametric variables. (For instance, in Ex 6,
there are two parametric variables, so dim(NS(A)) = 2)
 Notes:
(1) rank(A) can be viewed as the number of leading ones (or nonzero rows)
in the reduced row-echelon form for solving Ax = 0
(2) nullity(A) can be viewed as the number of free variables in the reduced
row-echelon form for solving Ax = 0 4.88
 Ex 7: Rank and nullity of a matrix
Let the column vectors of the matrix A be denoted by a1, a2,
a3, a4, and a5
 1 0 2 1 0
 0 1 3 1 3
A
 2 1 1 1 3
 
 0 3 9 0 12  45
a1 a2 a3 a4 a5
(a) Find the rank and nullity of A
(b) Find a subset of the column vectors of A that forms a basis for
the column space of A
(c) If possible, write the third column of A as a linear combination
of the first two columns
4.89
Sol: Derive B to be the reduced row-echelon form of A.

 1 0 2 1 0 1 0 2 0 1
 0 1  3 1 3 0 1 3 0  4
A  B
 2  1 1  1 3 0 0 0 1  1
   
 0 3 9 0  12 0 0 0 0 0
a1 a2 a3 a4 a5 b1 b2 b3 b4 b5

(by Theorems 4.13 and 4.14, rank(A) = dim(RS(A)) =

(a) rank(A) = 3
dim(RS(B)) = the number of nonzero rows in B)

nuillity( A)  n  rank( A)  5  3  2 (by Thm. 4.17)

4.90
(b) Leading 1’s
 {b1 , b 2 , b 4} is a basis for CS ( B)
{a1 , a 2 , a 4} is a basis for CS ( A)

 1  0  1
 0  1  1
a1   , a 2   , and a 4   ,
  2  1  1
 0  3  0
     

(c) b3  2b1  3b2  a3  2a1  3a 2

(due to the fact that elementary row operations do not change the dependency
relationships among columns)

4.91
 Theorem 4.18: Solutions of a nonhomogeneous linear system
If xp is a particular solution of the nonhomogeneous system
Ax = b, then every solution of this system can be written in
the form x = xp + xh , wher xh is a solution of the
corresponding homogeneous system Ax = 0

Pf:
Let x be another solution of Ax = b other than xp
 A(x  x p )  Ax  Ax p  b  b  0
 (x  x p ) is a solution of Ax = 0
Let x h be the solution of Ax  0 and x h  x  x p
 x  x p  xh
4.92
 Ex 8: Finding the solution set of a nonhomogeneous system
Find the set of all solution vectors of the system of linear equations

x1  2 x3  x4  5
3x1  x2  5 x3  8
x1  2 x2  5 x4  9
Sol:

 1 0 2 1 5  1 0 2 1 5
3 1 5 0 8  G.-J. E.
 0 1 1 3 7 
   
 1 2 0 5 9 0 0 0 0 0 
s t

4.93
  x1   2s  t  5  2  1  5
 x   s  3t  7  1  3  7
 x   2   s t 
 x3   s  0t  0  0  0  0
         
 x4   0s  t  0  1  1  0

 su1  tu 2  x p
5
 7 
i.e., x p    is a particular solution vector of Ax = b,
0
 
0

and xh = su1 + tu2 is a solution of Ax = 0 (you can replace

the constant vector with a zero vector to check this result)
4.94
 For any n×n matrix A with det(A) ≠ 0
※ If det(A) ≠ 0, Ax = b is with the unique solution (xp) and Ax
= 0 has only the trivial solution (xh = 0). According to
Theorem 4.18, the solution of Ax = b can be written as x =
xp + xh. The result of xh = 0 implies that there is only one
solution for Ax = b and the solution is x = xp + 0 = xp
※ In this scenario, nullity(A) = dim(NS(A)) = dim({0}) = 0.
Furthermore, according to Theorem 4.17 that n = rank(A) +
nullity(A) , we can conclude that rank(A) = n
※ Finally, according to the definition of rank(A) = dim(RS(A))
= dim(CS(A)), we can further obtain that dim(RS(A)) =
dim(CS(A)) = n, which implies that there are n rows (and n
columns) of A which are linearly independent (see the
definitions on Slide 4.74) 4.95
※ The relationship between the solutions of Ax = b and Ax = 0
for an n×n matrix A

det(A) ≠ 0 det(A) = 0

For Ax = 0 Only the trivial Infinitely many xh

solution xh=0
For Ax = b, the xp must exist xp exists xp does not exist
solution is x =
xp + xh shown
in Thm. 4.18 Only one solution Infinitely many Solution x = xp +
x = xp + 0 = xp solutions x = xp xh does not exist
+ xh

4.96
 Theorem 4.19: Solution of a system of linear equations
The system of linear equations Ax = b is consistent if and only
if b is in the column space of A (i.e., b can be expressed as a
linear combination of the column vectors of A)
Pf:
Let
 a11 a12 a1n   x1   b1 
a a22 a2 n  x  b 
A  21
, x   2  , and b   2 
     
     
 am1 am 2 amn   xn  bn 
be the coefficient matrix, the unknown vector, and the
constant-term vector, respectively, of the system Ax = b
4.97
Then
 a11 a12 a1n   x1   a11 x1  a12 x2   a1n xn 
a a a  x  a x  a x   a2 n xn 
Ax   21 22 2n   2    21 1 22 2

     
     
 m1
a a m2 a mn   xn   am1 x1  am 2 x2   amn xn 
 a11   a12   a1n 
a  a  a 
 x1  21   x2  22    xn  2 n   x1 A(1)  x2 A(2)   xn A( n )  b
     
     
 m1 
a  am 2   amn 

Hence, Ax = b is consistent (x has solutions) if and only if b is a

linear combination of the column vectors of A. In other words, the
system is consistent if and only if b is in the subspace of Rm
spanned by the column vectors of A
4.98
 Ex 9: Consistency of a system of linear equations depending
on whether b is in the column space of A
x1  x2  x3  1
x1  x3  3
3x1  2 x2  x3  1
Sol:
1 1 1 1 1 0 1 3
[ A b]   1 0 1 3 
G.-J. E.
 0 1 2 4 
3 2 1 1 0 0 0 0 
a1 a2 a 3 b w1 w2 w3 v
v  3w1  4w 2
 b  3a1  4a 2  0a3 (due to the fact that elementary row operations do not
change the dependency relationships among columns)

(In other words, b is in the column space of A)

 The system of linear equations is consistent
4.99
 Check for Ex. 9:
rank( A)  rank([ A | b])  2
 A property that can be inferred:
If rank(A) = rank([A|b]), then the system Ax = b is consistent

The above property can be analyzed as follows:

(1) By Theorem 4.19 in which Ax = b is consistent if and only if b is a
linear combination of the column vectors of A, we can infer that
appending b to the right of A does NOT increase the number of
linearly independent columns, so dim(CS(A)) = dim(CS([A|b]))
(2) By definition of the rank on Slide 4.86, rank(A) = dim(CS(A)) and
rank([A|b]) = dim(CS([A|b]))
(3) By combining (1) and (2), we can obtain rank(A) = rank([A|b]) if
and only if Ax = b is consistent 4.100
 Summary of equivalent conditions for square matrices:
If A is an n×n matrix, then the following conditions are equivalent
(1) A is invertible
(2) Ax = b has a unique solution for any n×1 matrix b
(3) Ax = 0 has only the trivial solution
(4) A is row-equivalent to In
(5) det (A)  0
(The above five statements are from Slide 3.39)
(6) rank(A) = n
(7) There are n row vectors of A which are linearly independent
(8) There are n column vectors of A which are linearly independent
(The last three statements are from the arguments on Slide 4.95)
4.101
Keywords in Section 4.6:
 row space: 列空間
 column space: 行空間
 nullspace: 零空間
 solution space: 解空間
 rank: 秩
 nullity: 核次數

4.102
 4.7 Coordinates and Change of Basis
 Coordinate representation relative to a basis
Let B = {v1, v2, …, vn} be an ordered basis for a vector space V
and let x be a vector in V such that
x  c1v1  c2 v 2    cn v n .
※ The “ordered” basis means the sequence of the vectors in the basis is
specified (被指定的)
The scalars c1, c2, …, cn are called the coordinates of x relative
to the basis B (x於基底B的座標). The coordinate matrix of x
relative to B is a real-number column matrix whose components
are the coordinates of x
 c1 
c 
xB   2 

 
 cn  4.103
 Ex 1: Coordinates and components in Rn
Find the coordinate matrix of x = (–2, 1, 3) in R3
relative to the standard basis
S = {(1, 0, 0), ( 0, 1, 0), (0, 0, 1)}
Sol:
x  (2, 1, 3)  2(1, 0, 0)  1(0, 1, 0)  3(0, 0, 1)

 2
[x]S   1
 3

※ For the standard basis in Rn, the coordinates of a vector are the
same as the components of that vector
4.104
 Ex 3: Finding a coordinate matrix relative to a nonstandard basis
Find the coordinate matrix of x = (1, 2, –1) in R3 relative to the
following (nonstandard) basis
B' = {u1, u2, u3}={(1, 0, 1), (0, – 1, 2), (2, 3, – 5)}
Sol:
x  c1u1  c2u 2  c3u3
(1, 2,  1)  c1 (1, 0, 1)  c2 (0,  1, 2)  c3 (2, 3,  5)
c1  2c3  1 1 0 2  c1   1
 c2  3c3  2 0  1 3 c    2
  2   
c1  2c2  5c3  1  1 2  5 c3   1

 c1   5
[x]B  c2    8
 c3   2  4.105
 Change of basis problem (基底變換問題):
You were given the coordinates of a vector relative to one
basis B' and were asked to find the coordinates relative to
another basis B
 Ex: Change of basis
Consider two bases for a vector space V
B  {u1 , u2}, B  {u1 , u2 } (B’ is the original basis and B is the
target basis)
a   c  (To represent the basis vectors in B’ by
If [u1 ]B   , [u2 ]B    the coordinate matrices relative to B)
b  d 
i.e., u1  au1  bu 2 , u2  cu1  du 2

4.106
  k1  (v is expressed as a coordinate matrix
Consider any v V , [ v]B    relative to the basis B’)
k 2 
 v  k1u1  k 2u2
 k1 (au1  bu 2 )  k 2 (cu1  du 2 )
 (k1a  k 2 c)u1  (k1b  k 2 d )u 2 (v can be expressed as a coordinate
matrix relative to the basis B)

 k1a  k2c  a c   k1 
 [ v ]B      k 
 1
k b  k 2 
d  b d  2 
 u1 B u2 B  v B

4.107
 Transition matrix from B' to B:
Let B  {u1 , u 2 ,..., u n } and B  {u1 , u2 ..., un } be two bases
for a vector space V
If [v]B is the coordinate matrix of v relative to B
[v]B’ is the coordinate matrix of v relative to B'
(the coordinate matrix relative to the target basis B is
then [ v]B  P [ v]B derived by multiplying the transition matrix P to the left
of the coordinate matrix relative to the original basis B')

 u1 B u2 B ... un B   v B 

where
P  u1 B u2 B ... un B 
is called the transition matrix from B' to B (從B'到B的轉移矩陣),
which is constructed by the coordinate matrices of ordered vectors
in B' relative to B 4.108
Theorem 4.20: The inverse of a transition matrix
If P is the transition matrix from a basis B' to a basis B in Rn,
then
(1) P is invertible
–1
(2) The transition matrix from B to B' is P
Pf:
B  {u1 , u 2 , ..., u n } and B'  {u1 , u2 , ..., un }
 v B  [u1 ]B [u2 ]B ... [un ]B   v B  P  v B
 v B  [u1 ]B [u 2 ]B ... [u n ]B   v B  Q  v B (derived by interchanging the roles of B and B' )
Replacing  v B in the first equation with the second equation
  v B  PQ  v B  PQ  I
 P is invertible and P 1  Q  [u1 ]B [u 2 ]B ... [u n ]B 
 P 1 is the transition matrix from B to B '
4.109
  Theorem 4.21: Deriving the transition matrix by G.-J. E.
Let B  {u1 , u2 , ..., un } and B'  {u1 , u2 , ..., un } be two bases
for Rn. Then the transition matrix P from B' to B can be found
by using Gauss-Jordan elimination on the n×2n matrix  B B
as follows Construct the matrices B and B’ by using ordered basis
vectors as column vectors

Note that the target basis is

always on the left  B B 
G.-J. E.
 I n P 
Similarly, the transition matrix P–1 from B to B' can be found
via
 B B  
G.-J. E.
  I n P 1 
The resulting matrix is
the transition matrix
from the original basis
to the target basis

(The next slide uses the case of n = 2 to show why  B B 
G.-J. E.
 I n P 
works) 4.110
 Consider two bases for a vector space V, B  {u1 , u2}, B  {u1 , u2 }
The transition matrix P from B ' to B is
a  c 
If [u1 ]B   , [u2 ]B    a c 
b d  (see the slide 4.107), so finding P
b  d   
i.e., u1  au1  bu 2 , u2  cu1  du 2 a  c
is equivalent to solving   and  
b  d 
   au11  bu21   u11 u21   a 
 u11 a 
u    au  bu   u      B  
 12   12 22   12 u22   b  b   u11  u21
 
 B u  u     B B 
 u11 1 0 aˆ   12 22 
 B  G.-J. E.  
 u 
12   0 1 ˆ
b  1 0 aˆ cˆ 
G.-J. E. 
ˆ ˆ   [ I P]
   cu11  du21   u11 u21   c 
 u21 c 0 1 b d 
u   cu  du   u      B  
 22   12 22   12 u22   d  d  ※ This method is very similar to
the method for solving A-1
 u21  1 0 cˆ  through G.-J. E., i.e.
 B  G.-J. E.   A | I     I | A1 
 ˆ
G.-J. E.

 u 22   0 1 d 
4.111
 Ex 5: (Finding a transition matrix)

B = {(–3, 2), (4,–2)} and B' = {(–1, 2), (2,–2)} are two bases for R2

(a) Find the transition matrix from B' to B

1 
(b) Let [ v]B '    , find [ v]B
2
(c) Find the transition matrix from B to B'

 1 2
For the original basis: B   
 2  2 
 3 4
For the target basis: B   
 2 2 
4.112
Sol:
(a)  3 4  1 2  1 0  3  2
 2  2  2  2
G.-J. E.
0 1  2  1
   
B B' I P

 3  2
P    (the transition matrix from B' to B)
2  1 

(b)
 3 2 1   1
[ v]B  P [ v]B       
 2 1  2  0 

4.113
 (c)
 1 2  3 4  1 0   1 2
 2  2  2  2 G.-J. E.
0 1   2 3
   
–1
B' B I P

  1 2
1
P    (the transition matrix from B to B')
  2 3 

 Check:
1 3 2  1 2  1 0
PP        I
 2 1  2 3 0 1

4.114
 Ex 6: Coordinate representation in P3(x)
Find the coordinate matrix of p = 3x3 – 2x2 + 4 relative to the
nonstandard basis in P3(x), S = {1, 1 + x, 1 + x2, 1 + x3}
Sol:
Solve p = a(1) + b(1+x) + c(1+x2 ) + d(1+x3 )
 p = 3(1) + 0(1+x) + (–2)(1+x2 ) + 3(1+x3 )

 3
 0
[p]S =  
 2
 
 3
4.115
 Ex: Coordinate representation in M2x2
Find the coordinate matrix of x = 
5 6
 relative to
7 8 
the standard basis in M2x2
1 0 0 1 0 0 0 0 
B =   ,  ,  ,  
0 0 0 0 1 0 0 1 
Sol:

5 6 1 0  0 1  0 0  0 0 
x   5  6  7  8 
 7 8   0 0   0 0   1 0   0 1 
5
6
  x B   
7 
 
8 
4.116
Keywords in Section 4.7:
 coordinates of x relative to B: x於基底B的座標
 coordinate matrix: 座標矩陣
 change of basis problem: 基底變換問題
 transition matrix from B' to B: 從 B' 到 B的轉移矩陣

4.117
 4.8 Applications of Vector Spaces
 Conic Sections (二次曲線) and Rotation (see Application in
Ch4.pdf)

4.118