Vector Spaces and Subspaces

We have seen that vectors in Rn can be added together and multiplied by

scalars. However, vectors in Rn are not the only kind of objects that can be
added together and multiplied by scalars. For example, polynomials

p (x) = a0 + a1 x + · · · + an xn

can be added together and multiplied by scalars. Thus, vectors in Rn and

polynomials share a common set of algebraic properties. We call any set of
objects sharing this common set of properties a vector space.

Definition 1. A vector space is a nonempty set V with two operations called

addition and multiplication which satisfy the following properties for all x, y, z ∈
V and all α, β ∈ R.
1. x + y = y + x.



2. x + y + z = x + y + z .

3. There is an element 0 ∈ V such that x + 0 = x for all x ∈ V .

4. For each x ∈ V there is −x ∈ V such that x + (−x) = 0.


5. α x + y = αx + αy.

6. (α + β) x = αx + βx.
7. α (βx) = (αβ) x.
8. 1 · x = x.
These are all familiar properties of Rn . The point is that many other impo-
tent sets of objects also have these properties. Thus, it is useful to study vectors
spaces as objects in their own right. (Abstraction)
Example 2. The Euclidean vector space Rn together with the usual addition
and multiplication by scalars.
Example 3. The set Rm×n of m × n matrices together with addition and
multiplication by a scalar.
Example 4. The trivial vector space {0}.

1
Example 5. The space of functions f : R → R from the real numbers into
the real numbers. Here, the addition of functions is pointwise, and so is the
multiplication by a scalar.
Example 6. The space of polynomial functions p : R → R of any degree is a
vector space. We write P for that vector space.

Example 7. Let the set of polynomials of degree mbe written Pm . So every

p ∈ Pm is a polynomial of the form
p(x) = cm xm + cm−1 xm−1 + . . . + c1 x + c0 .
We easily check that Pm together with addition and scaling is a vector space.

Example 8. Let A ∈ Rm×n be a matrix. Then the set of solutions S to the

homogeneous problem A · v = 0 is a vector space by itself. We check that

S = {v ∈ Rn : Av = 0}
is preserved under addition and multiplication by scalars: For any x, y ∈ S we
have x + y ∈ S and for any x ∈ S and α ∈ R we have αx ∈ S.
We will revisit the previous example from a different perspective later.

Example 9. The set {a · F + b · : a, b ∈ R} with

(a · F + b · ) + (c · F + d · ) = (a + c) · F + (b + d) ·
c · (a · F + b · ) = (ca) · F + (cb) ·

From the axioms of vector space, we can derive that the following statements
are true as well in any vector space.

1. 0x = 0 for every vector x.

2. α0 = 0 for every scalar α
3. (−1) x = −x for every vector x.
(See textbook for proofs).

Subspaces
Question 10. When is a subset H of a vector space V is also a vector space?

Definition 11. H is a subspace of a vector space V if H is a subset of V such

that
(i) 0 ∈ H.
(ii) If x, y ∈ H, then x + y ∈ H.

(iii) If x ∈ H, then αx ∈ H for every scalar α.

2
With these properties, H ⊂ V satisfies all 8 properties required for H to be a
vector space in its own right.
Example 12.
  
 x 
1. H = y  : x, y ∈ R is a subspace of R3 .
0
 

 
0
(i) 0 ∈ H;
0
     
x1 x2 x1 + x2
(ii)  y1  +  y2  =  y1 + y2  ∈ H;
0 0 0
   
x αx
(iii)α y  = αy  ∈ H.
0 0
    
 x  0
2. K = y  : x, y ∈ R is not a subspace of R3 , for example, 0 ∈
/ K.
1 0
 

The three properties that define subspaces can be checked at once with the
following criterion.

Theorem 13. A nonempty subset H of a vector space V is a subspace of V if

and only if
αx + βy ∈ H
for all x, y ∈ H and all scalar α, β.
Proof. To prove that two statements are equivalent, we show each implies the
other:
(⇒) Suppose H is a subspace of V . Then 0 ∈ H so that H is nonempty. If
x, y ∈ H and α, β ∈ R, then by (iii), αx, βy ∈ H and then by (ii) αx + βy ∈ H.
(⇐)Suppose H ⊂ V is nonempty and that αx + βy ∈ H for all x, y ∈ H and
α, β ∈ R. Since H is nonempty, there is an x ∈ H. Then
(i) 0x + 0x = 0 ∈ H.
(ii) If x, y ∈ H, then x + y = 1 · x + 1 · y ∈ H.

(iii) If x ∈ H, then αx = αx + 0x ∈ H for every scalar α.

Example 14.

3
 1. Let P2 be the set of polynomials a0 + a1 x + a2 x2 of degree 2 or less. Then
P2 is a subspace of P, the vector space of all polynomials.
2. The set of all symmetric 2 × 2 matrices A = AT is a subspace of all 2 × 2
matrices.
3. Given v 1 , . . . , v k ∈ V a vector span

span {v 1 , . . . , v k }

is a subspace of V . This is proven below.

Problem 15. Is the set of invertible 2 × 2 matrices a subspace of all 2 × 2
matrices?
 
0 0
No. is not invertible.
0 0
Theorem 16. Given v 1 , . . . , v k ∈ V, the vector span

span {v 1 , . . . , v k }

is a subspace of V .
Proof. Given x = a1 v 1 +· · ·+ak v k and y = b1 v 1 +· · ·+bk v k in span {v 1 , . . . , v k }.
Then

αx + βy = α (a1 v 1 + · · · + ak v k ) + β (b1 v 1 + · · · + bk v k )
= (αa1 + βb1 ) v 1 + · · · + (αak + βbk ) v k ∈ span {v 1 , . . . , v k } .

The set span {v 1 , . . . , v k } is often called the subspace generated by v 1 , . . . , v k .

Note that it should not be surprising that the set of all possible linear combi-
nations of v 1 , . . . , v k is closed under linear combinations. But this observation
is of fundamental importance.
Definition 17. The null space Null (A) of a m × n matrix A is the set of all
solutions to the homogeneous equation Ax = 0. That is

Null (A) = {x ∈ Rn : Ax = 0} .

Notice: all we have done is give a name to the solution set of Ax = 0.

 
1 3 5 0
Problem 18. Determine Null (A) for A = .
1 2 1 2
We can preform row operations to get
       
1 3 5 0 0 1 4 −2 0 1 4 −2 1 0 −7 6
→ → →
1 2 1 2 1 2 1 2 1 0 −7 6 0 1 4 −2

4
So x3 , x4 are free, and we get
       
x1 7s − 6t 7 −6
x2  −4s + 2t −4 2
 =  = s  + t .
 x3   s  1 0
x4 t 0 1

Thus     

 7 −6 
    
−4 , 2 

Null (A) = span  1   0 

 
0 1
 

a subspace of R4 .
Theorem 19. The null space of a m × n matrix A is a subspace of Rn .
Proof. To verify this, we need to check that
1. Null (A) is nonempty.
Indeed, A0n = 0m . Thus 0n ∈ Null (A).
2. αx + βy ∈ Null (A) for all x, y ∈ Null (A) and α, β ∈ R.
x, y ∈ Null (A) ⇒ Ax = 0, Ay = 0. Then, by linearity,


A αx + βy = αAx + βAy = α0 + β0 = 0.

Hence, αx + βy ∈ Null (A).

Definition 20. The column space Col (A) of a m × n matrix A is the span of
the columns of A. That is, given A = (a1 · · · an ),

Col (A) = span {a1 , . . . , an } .

Notice, all we have done is give a name to the span of the columns of A.
Problem 21. Let
  

 2s + t 

r − s + 2t
  
H=   : r, s, t ∈ R .

 3r + s  

2r − s − t
 

1. Find a matrix A for which Col (A) = H.

       
2s + t 0 2 1
r − s + 2t 1 −1 2
 3r + s  = r 3 + s  1  + t  0  .
       

2r − s − t 2 −1 −1

5
 Thus      

 0 2 1 
     
1 −1  2 

H = span   ,   ,  


 3 1 0  
2 −1 −1
 
 
0 2 1
1 −1 2 
and we can choose A =  3 1
 . (Note that there are infinitely
0
2 −1 −1
many choices).
 
3
2
4 ∈ H.
2. Determine if  

0
 
3
2
4 ∈ H ⇐⇒ there exist x1 , x2 , x3 such that
We have  

0
       
0 2 1 3
1 −1  2  2
3 + x2  1  + x3  0  = 4
x1        

2 −1 −1 0

is consistent. That is, we need to solve

   
0 2 1 3 1 0 0 1
 1 −1 2 2  exercise  0 1 0 1 
  −→  .
 3 1 0 4   0 0 1 1 
2 −1 −1 0 0 0 0 0

Therefore        
3 0 2 1
2 1 −1  2 
  =   +   +   ∈ H.
4 3  1   0 
0 2 −1 −1

Theorem 22. Given a m × n matrix A.


1. Col (A) = y ∈ Rm : y = Ax for some x ∈ Rn .
2. Col (A) is a subspace of Rm .
Proof. Given A = (a1 · · · an ) with a1 , . . . , an ∈ Rm .

6
 1. We have

y ∈ Rm : y = Ax for some x ∈ Rn


= y = x1 a1 + · · · + xn an : x1 , . . . , xn ∈ R
= span {a1 , . . . , an }
= Col (A) .

2. span {a1 , . . . , an } is a subspace of Rm by our theorem.

Linear Independence
Definition 23. Let V be a vector space. Given {v 1 , . . . , v k } an indexed set of
vectors in V, we say that

1. {v 1 , . . . , v k } is linearly dependent if there are scalars c1 , . . . , ck not all

zero such that c1 v 1 + · · · + ck v k = 0.
2. {v 1 , . . . , v k } is linearly independent if the vector equation such that
c1 v 1 + · · · + ck v k = 0 has only the trivial solution c1 = · · · = ck = 0.

Note that this definition generalizes the notions of linear dependence and inde-
pendence to the general vector spaces.
Example 24. Let
        
 1 0 0 1 
S = 0 , 1 , 1 , 1 .
0 0 1 1
 

The set S spans R3 since

   
1 0 0 −1 1 0 0 1
A = 0 1 1 1  −→ 0 1 0 0
0 0 1 1 0 0 1 1

has a pivot in every row. (So Ax = b is consistent for every b ∈ R3 .) Note that
the fourth column of A is the sum of the first and third column. Since
     
1 1 0
1 = 0 + 1
1 0 1
        
 1   1 0 0 
we find that S ∗ = S \ 1 = 0 , 1 , 1 still spans R3 . For
1 0 0 1
   
example, if

7
        
1 0 0 1
v = a 0 + b 1 + c 1 + d 1
0 0 1 1
is any linear combination of vectors from S, then

             
1 0 0 1 1 0 0
v = a 0 + b 1 + c 1 + d 1 = (a + d) 0 + b 1 + (c + d) 1
0 0 1 1 0 0 1

While S is not linearly independent, the reduced set S ∗ is linearly independent

since    
1 0 0 1 0 0
A∗ = 0 1 1 −→ 0 1 0
0 0 1 0 0 1
has a pivot in every column (so A∗ x = 0 has only the trivial solution). If
we remove any vector from S ∗ , the remaining vectors would no loner span R3 .
(Removing a column from S ∗ remove a pivot position from one of the row.)
Being linearly independent made S ∗ as small as a spanning set for R3 can be.
Note also that      
1 0 0
x1 0 + x2 1 + x3 1 = b
0 0 1
has a unique solution for every b ∈ R3 .
In the above example, the set S ∗ is a basis for R3 .
Definition 25. Given H a subspace of a vector space V . A set of vectors
B = {v 1 , . . . , v k } in H is a basis for H if
1. B is linearly independent, and
2. B spans H; that is H = span {b1 , . . . , bk } = spanB.
The example above suggests that if S is a linearly dependent set that spans
H, then we can remove vectors that are linear combinations of the remaining
vectors until we’re left with a basis. This observation is, in fact, the content of
the following theorem.
Theorem 26 (Spanning Set Theorem). Let S = {v 1 , . . . , v k } in a vector space
V , and let H = span {v 1 , . . . , v k }. If v i ∈ S is a linear combination of the
remaining vectors in H, then S ∗ = S \ {vi } still spans H.
Proof. Suppose that v i is a linear combination of the remaining vectors, so we
can write

v i = a1 v 1 + · · · + ai−1 v i−1 + ai+1 v i+1 + · · · + ak v k

8
We need to show that any linear combination of vectors in S can be written
without the vector vi , that is, as a linear combination using vectors from S ∗ .
Indeed, we observe

c1 v 1 + · · · + ci−1 v i−1 + ci v i + ci+1 v i+1 + · · · + ck v k

=(ci a1 + c1 )v 1 + · · · + (ci ai−1 + ci−1 )v i−1 + (ci ai+1 + ci+1 )v i+1 + · · · + (ci ak + ck )v k

This finishes the proof.

Theorem 27 (Spanning Set Theorem). Let S = {v 1 , . . . , v k } in a vector space
V , and let H = span {v 1 , . . . , v k }. If H 6= {0}, then some subset of S is a basis
for H.

Proof. If S is linearly independent, then it is a basis for its span H. If instead

S is linearly dependent, then one its vectors vi can be written as sum of the
remaining vectors. By the previous theorem, S ∗ = S \ {vi } still spans H, so we
just consider the set S ∗ of vectors. We keep repeating this until we have got a
basis.

Recall that a basis for a subspace H of a vector space V is a linearly inde-

pendent set of vectors that span H. It turns that the bases essentially describe
the entire vector space completely.
Question 28. Given bases B = {b1 , . . . , bn } and D = {d1 , . . . , dm } for a vector
space V . Could m 6= n?

Theorem 29. Let B = {b1 , . . . , bn } be a basis of the vector space V and let
D = {d1 , . . . , dm } be a set of linearly independent vectors in V. Then m ≤ n.

Corollary 30. If S = v 1 , . . . , v p is a linearly dependent set in V , then there is
an index k > 1 for which v k is a linear combination of the previous v 1 , . . . , v k−1 .

Theorem 31. If a vector space V has a basis of n vectors, then every basis of
V has n vectors.
Proof. Let B = {b1 , . . . , bn } and D = {d1 , . . . , dm } be bases for V . Since B is
linearly independent and D spans V , n ≤ m. Since D is linearly independent
and B spans V , m ≤ n. It follows that n = m.

Definition 32. The number of vectors in a basis for V is called the dimension
of V , written dim V .
Note, dim {0} = 0. (The empty set ∅ = {} is a basis for {0}.)
Theorem 33 (Basis Theorem). Given V a p-dimensional vector space.

1. Any linearly independent set of exactly p vectors in V is a basis for V .

2. Any set of exactly p vectors in V that spans V is a basis for V .

9
Problem 34. Find a basis for the row space of A.
Theorem 35. If A and B are row equivalent, then Row (A) = Row (B).
Proof. The rows of B are linear combinations of the rows of A and vice versa.
Be careful! The rows of a matrix A corresponding to the nonzero rows of
its Echelon form may not form a basis for Row (A). (Row operations do not
preserve dependence relations among the row.)
Example 36. Consider the following matrix Aand its reduced row echelon form
Ã.
     
1 2 0 0 2 1 2 0 0 2 1 2 0 0 2
A = −1 −2 1 1 1 −→ 0 0 1 1 3 −→ 0 0 1 1 3 = Ã.
2 4 −1 −1 3 2 4 −1 −1 3 0 0 0 0 0

Then the row span computes as

Row (A) = span {(1, 2, 0, 0, 2) , (−1, −2, 1, 1, 1) , (2, 4, −1, −1, 3)}
= span {(1, 2, 0, 0, 2) , (0, 0, 1, 1, 3)}

and the column span computes as

          
 1 2 0 0 2 
Col (A) = span −1 , −2 ,  1  ,  1  , 1
2 4 −1 −1 3
 
    
 1 0 
= span −1 ,  1  .
2 −1
 

Here, from the reduced row echelon form we can read off that the column space
of A is spanned by the first and third column. Lastly,
      

 −2 0 −2 

 0  −1 −3

 
     
Null (A) = span  1 , 0 , 0  .
     


  0   1   0  
 
0 0 1
 

Note:

dim (Row (A)) = dim (Col (A)) = 2 (= number of pivot positions)

dim (Null (A)) = 3 (= number of free variables = number of not pivot)
dim (Null (A)) + dim (Col (A)) = 5 (= number of columns of A)

Theorem 37 (Rank Theorem). Given an m × n matrix A.

1. dim (Row (A)) = dim (Col (A)) =: rank (A).

10
 2. rank (A) + dim (Null (A)) = n.
Proof. Let à = the reduced Echelon form of A.
1. Since the nonzero rows in à are a basis for Row (A)
dim (Row (A)) = #nonzero rows in Ã
= #pivot positions in Ã
= #pivot columns in A
= dim (Col (A)) .

2. We have
n = #pivot columns + #non-pivot columns
= dim (Col (A)) + dim (Null (A)) .

Problem 38. Given a 3 × 8 matrix A with rank (A) = 3. Determine:

1. dim (Row (A)).
dim (Row (A)) = rank (A) = 3.
2. dim (Null (A)).
dim (Null (A)) = n − rank (A) = 8 − 3 = 5.


3. rank AT
.


rank AT = dim Col AT = dim (Row (A)) = rank (A) = 3.



4. dim Null AT

.

dim Null AT = m − rank AT = 3 − 3 = 0.
The following result captures one of the most important properties of bases.
It is a simple consequence of bases being linearly independent and spanning
sets.
Theorem 39. Let B = {b1 , . . . , bn } be a basis for V . Then for each x ∈ V ,
there are unique scalars c1 , . . . , cn such that
x = c1 b1 + · · · + cn bn .
Proof. Suppose
x = c1 b1 + · · · + cn bn = d1 b1 + · · · + dn bn .
Then
(c1 − d1 ) b1 + · · · + (cn − dn ) bn = 0.
Since b1 , . . . , bn are linearly independent, we may deduce
c1 − d1 = · · · = cn − dn = 0
⇒ c1 = d1 , . . . , cn = dn .

11
 So for every vector v ∈ V there exist unique coefficients for which vis linear
combination of the basis vectors. It makes sense to give those coefficients a
special name. We call  
c1
 .. 
 .  = [v]B
cn
the B-coordinates vector of x.
Problem 40. Given    
1 5
B= , ,
−2 −6
 
4
which is a basis for R2 , and some vector x = , we want to find [x]B . So we
0
search for c1 , c2 such that
        
4 1 5 1 5 c1
= c1 + c2 =
0 −2 −6 −2 −6 c2
   
1 5 4 1 0 −6
⇒ →
−2 −6 0 0 1 2
 
−6
⇒ [x]B = .
2

Indeed, we check      
1 5 4
−6 +2 = .
−2 −6 0
Observe: Given B = {b1 , . . . , bn } a basis for Rn and x ∈ Rn
 
c1

 . 
x = c1 b1 + · · · + cn bn = b1 · · · bn  ..  = PB [x]B .
| {z }
=:PB cn
| {z }
=[x]B

This can be done for any (finite-dimensional) vector space.

Theorem 41. Given B = {b1 , . . . , bn } a basis for a vector space V . The map-
ping x 7→ [x]B is a one-to-one linear transformation from V onto Rn .

Problem 42. B = x2 − 1, x2 − x, x2 − 2x − 1 is a basis for P2 , the set
of
 all polynomials
 of degree ≤ 2). We want to find the coordinate vector
9x2 − 11x − 4 B with respect to that basis. In other words, we are looking
for coefficients c1, c2 , c3 ∈ R for which

9x2 − 11x − 4 = c1 x2 − 1 + c2 x2 − x + c3 x2 − 2x − 1

= (c1 + c2 + c3 ) x2 + (−c2 − 2c3 ) x + (−c1 − c2 ) .

12
We translate that into a linear system of equations and solve it:

 c1 + c2 + c3 = 9

−c2 − 2c3 = −11

−c1 − c2 = −4

     
1 1 1 9 1 1 1 9 1 1 1 9
⇒  0 −1 −2 −11  →  0 −1 −2 −11  →  0 −1 −2 −11 
−1 0 −1 4 0 1 0 13 0 0 −2 −2
 
1
⇒ 9x2 − 11x − 4 B = 5 .
 

Indeed, we check that

1(x2 − 1) + 5(x2 − x) + 3(x2 − 2x − 1) = 9x2 − 11x − 4

as intended.

13