You are on page 1of 104

Chapter 3

Vector Spaces
3.1 Vectors in Rn
3.2 Vector Spaces
3.3 Subspaces of Vector Spaces
3.4 Spanning Sets & Linear Independence
3.5 Basis and Dimension
3.6 Rank of a matrix
3.7 Inner Product Spaces
3.8 Linear Transformation
n
3.1 Vectors in R
◼ An ordered n-tuple :
a sequence of n real numbers ( x1 , x2 , , xn )

n
▪ R -space :
the set of all ordered n-tuples
n=1 R1-space = set of all real numbers
(R1-space can be represented geometrically by the x-axis)

n=2 R2-space = set of all ordered pair of real numbers ( x1 , x2 )


(R2-space can be represented geometrically by the xy-plane)
n=3 R3-space = set of all ordered triple of real numbers ( x1 , x2 , x3 )
(R3-space can be represented geometrically by the xyz-space)
n=4 R4-space = set of all ordered quadruple of real numbers ( x1 , x2 , x3 , x4 )

3.2
◼ Notes:
(1) An n-tuple ( x1 , x2 ,  , xn ) can be viewed as a point in Rn
with the xi’s as its coordinates
(2) An n-tuple ( x1 , x2 ,  , xn ) also can be viewed as a vector
x = ( x1 , x2 , , xn ) in Rn with the xi’s as its components
▪ Ex:

(x1 , x2 ) (x1 , x2 )
or

(0,0)
a point a vector
※ A vector on the plane is expressed geometrically by a directed line segment
whose initial point is the origin and whose terminal point is the point (x1, x2)
3.3
u = ( u1 , u2 , , un ) , v = ( v1 , v2 , , vn ) (two vectors in Rn)
▪ Equality:
u = v if and only if u1 = v1 , u2 = v2 , , u n = vn

▪ Vector addition (the sum of u and v):


u + v = (u1 + v1 , u2 + v2 ,  , un + vn )

▪ Scalar multiplication (the scalar multiple of u by c):


cu = (cu1 , cu2 ,  , cun )

▪ Notes:
The sum of two vectors and the scalar multiple of a vector
in Rn are called the standard operations in Rn
3.4
▪ Difference between u and v:
u − v  u + (−1) v = (u1 − v1 , u2 − v2 , u3 − v3 ,..., un − vn )

▪ Zero vector:
0 = (0, 0, ..., 0)

3.5
▪ Notes:
A vector u = (u1 , u2 , , un ) in R n can be viewed as:
Use comma to separate components

a 1×n row matrix (row vector): u = [u1 u2 un ]

Use blank space to separate entries


or u1 
u 
a n×1 column matrix (column vector): u =  2 
 
 
un 

※ Therefore, the operations of matrix addition and scalar multiplication


generate the same results as the corresponding vector operations (see the
next slide)
3.6
Vector addition Scalar multiplication
u + v = (u1 , u2 , , un ) + (v1 , v2 , , vn ) cu = c(u1 , u2 ,, un )
= (u1 + v1 , u2 + v2 ,  , un + vn ) = (cu1 , cu2 , , cun )

Regarded as 1×n row matrix


u + v = [u1 u2 un ] + [v1 v2 vn ] cu = c[u1 u2 un ]
= [u1 + v1 u2 + v2 un + vn ] = [cu1 cu2 cun ]

Regarded as n×1 column matrix


u1  v1  u1 + v1  u1  cu1 
u  v  u + v  u  cu 
u+ v =  2 +  2 =  2 2 cu = c  2  =  2 
          
         
un  vn  un + vn  un  cun 
3.7
◼ Theorem 3.1: Properties of vector addition and scalar multiplication
Let u, v, and w be vectors in Rn, and let c and d be scalars
n
(1) u+v is a vector in R (closure under vector addition)
(2) u+v = v+u (commutative property of vector addition)
(3) (u+v)+w = u+(v+w) (associative property of vector addition)
(4) u+0 = u (additive identity property)
(5) u+(–u) = 0 (additive inverse property)
(6) cu is a vector in Rn (closure under scalar multiplication)
(7) c(u+v) = cu+cv (distributive property of scalar multiplication over vector
addition)
(8) (c+d)u = cu+du (distributive property of scalar multiplication over real-
number addition)
(9) c(du) = (cd)u (associative property of multiplication)
(10) 1(u) = u (multiplicative identity property)

3.8
◼ Ex 5: Practice standard vector operations in R4
Let u = (2, –1, 5, 0), v = (4, 3, 1, –1), and w = (–6, 2, 0, 3) be
vectors in R4. Solve x in each of the following cases.
(a) x = 2u – (v + 3w)
(b) 3(x+w) = 2u – v+x

Sol: (a) x = 2u − ( v + 3w )
= 2u + (−1)( v + 3w )
= 2u − v − 3w (distributive property of scalar multiplication over vector addition)
= (4, − 2, 10, 0) − (4, 3, 1, − 1) − ( −18, 6, 0, 9)
= (4 − 4 + 18, − 2 − 3 − 6, 10 − 1 − 0, 0 + 1 − 9)
= (18, − 11, 9, − 8)

3.9
(b) 3(x + w ) = 2u − v + x
3x + 3w = 2u − v + x (distributive property of scalar multiplication over vector addition)

3x − x = 2u − v − 3w (subtract (3w+x) from both sides)


2x = 2u − v − 3w
x = u − 12 v − 32 w (scalar multiplication for the both sides with a scalar to be 1/2)
= ( 2, −1,5, 0 ) + ( −2, −23 , −21 , 12 ) + ( 9, −3, 0, −29 )
= ( 9, −211 , 92 , −4 )

3.10
▪ Notes:
(1) The zero vector 0 in Rn is called the additive identity in Rn (see
Property 4)
(2) The vector –u is called the additive inverse of u (see
Property 5)

◼Theorem 3.2: (Properties of additive identity and additive inverse)


Let v be a vector in Rn and c be a scalar. Then the following
properties are true
(1) The additive identity is unique, i.e., if v+u = v, u must be 0
(2) The additive inverse of v is unique, i.e., if v+u = 0, u must be –v
(3) 0v = 0
(4) c0 = 0
(5) If cv = 0, either c = 0 or v = 0
(6) –(–v) = v 3.11
◼ Linear combination in Rn:
The vector x is called a linear combination of v1 ,v 2 ,...,v n ,
if it can be expressed in the form
x = c1 v1 + c2 v 2 + + cn v n , where c1 , c2 , , cn are real numbers

▪ Ex 6:
Given x = (–1, –2, –2), u = (0,1,4), v = (– 1,1,2), and
w = (3,1,2) in R3, find a, b, and c such that x = au+bv+cw.
Sol:
− b + 3c = − 1
a + b + c = −2
4a + 2b + 2c = − 2
 a = 1, b = −2, c = −1
Thus x = u − 2 v − w 3.12
3.2 Vector Spaces
V ≠ ∅: Addition 𝐮 + 𝐯 & Scalar multiplication 𝑐𝐮
If these 10 axioms are satisfied, then V is called a vector space, and
the elements in V are called vectors
(1) 𝐮 + 𝐯 ∈ 𝑉, ∀𝐮, 𝐯 ∈ 𝑉 (closure under vector addition)

(2) u+v = v+u, ∀𝐮, 𝐯 ∈ 𝑉 (commutative property of vector addition)


(3) (u+v)+w = u+(v+w), ∀𝐮, 𝐯, 𝐰 ∈ V (associative property of vector addition)
(4) ∃𝟎 ∈V such that ∀u ∈ 𝑉: u+0 = u (additive identity property)
(5) ∀u ∈ 𝑉, ∃ –u ∈ 𝑉: u+(–u) = 0 (additive inverse property)
(6) 𝑐𝐮 ∈ 𝑉, ∀𝐮 ∈ 𝑉, ∀𝑐 ∈ 𝑹 (closure under scalar multiplication)
(7) c(u+v) = cu+cv, ∀𝐮, 𝐯 ∈ 𝑉, ∀𝑐 ∈ 𝑹 (distributive property of scalar
multiplication over vector addition)
(8) (c+d)u = cu+du, ∀𝐮 ∈ V, ∀𝑐, 𝑑 ∈ 𝑹 (distributive property of scalar
multiplication over real-number addition)
(9) c(du) = (cd)u, ∀𝐮 ∈ V, ∀𝑐, 𝑑 ∈ 𝑹 (associative property of multiplication)
(10) 1(u) = u, ∀u ∈ 𝑉 (multiplicative identity property) 3.13
◼ Notes:
A vector space consists of four entities:
a set of vectors, a set of real-number scalars, and two operations

V: nonempty set of vectors


c: any scalar
+ (u, v ) = u + v : vector addition
(c, u) = cu : scalar multiplication
(V , +, ) is called a vector space
※ The set V together with the definitions of vector addition and scalar
multiplication satisfying the above ten axioms is called a vector space

3.14
◼ Examples. (It is straightforward to show that these vector spaces satisfy the
above ten axioms)
n
(1) n-tuple space: R
(u1 , u2 ,un ) + (v1 , v2 , v2 ) = (u1 + v1 , u2 + v2 ,un + vn ) (standard vector addition)
k (u1 , u2 ,un ) = (ku1 , ku2 , kun ) (standard scalar multiplication for vectors)

(2) Matrix space : V = M mn


(the set of all m×n matrices with real-number entries)
Ex: (m = n = 2)
u11 u12   v11 v12   u11 + v11 u12 + v12 
u u  + v v  = u + v u + v  (standard matrix addition)
 21 22   21 22   21 21 22 22 
u11 u12   ku11 ku12 
k  =  (standard scalar multiplication for matrices)
 21 22   21
u u ku ku 22  3.15
(3) n-th degree or less polynomial space: V = Pn
(the set of all real-valued polynomials of degree n or less)
p( x) + q( x) = (a0 + b0 ) + (a1 + b1 ) x +  + (an + bn ) x n (standard
addition)
polynomial

kp( x) = ka0 + ka1 x +  + kan x n (standard scalar multiplication for


polynomials)
※ By the fact that the set of real numbers is closed under addition and
multiplication, it is straightforward to show that Pn satisfies the ten axioms
and thus is a vector space
(4) Continuous function space: V = C (−, )
(the set of all real-valued continuous functions defined on the
entire real line)
( f + g )( x) = f ( x) + g ( x) (standard addition for functions)
(kf )( x) = kf ( x) (standard scalar multiplication for functions)

※ By the fact that the sum of two continuous function is continuous and the
product of a scalar and a continuous function is still a continuous
function, C (−, ) is a vector space
3.16
▪ Summary of important vector spaces
R = set of all real numbers
R 2 = set of all ordered pairs
R 3 = set of all ordered triples
R n = set of all n-tuples
C (−, ) = set of all continuous functions defined on the real number line
C[a, b] = set of all continuous functions defined on a closed interval [a, b]
P = set of all polynomials
Pn = set of all polynomials of degree  n
M m ,n = set of m  n matrices
M n ,n = set of n  n square matrices

3.17
◼ Notes: To show that a set is not a vector space, you need
only find one axiom that is not satisfied
◼ Ex 6: The set of all integers is not a vector space
Pf: 1  V , and 12 is a real-number scalar
( 12 )(1) = 12  V (it is not closed under scalar multiplication)
  noninteger
scalar

integer

◼ Ex 7: The set of all (exact) second-degree polynomial functions is


not a vector space
Pf: Let p ( x) = x 2 and q( x) = − x 2 + x + 1
 p ( x) + q ( x) = x + 1  V
(it is not closed under vector addition)
3.18
◼ Ex 8:
V=R2=the set of all ordered pairs of real numbers
vector addition: (u1 , u2 ) + (v1 , v2 ) = (u1 + v1 , u2 + v2 )
scalar multiplication: c(u1 , u2 ) = (cu1 ,0) (nonstandard definition)
Verify V is not a vector space
Sol:
This kind of setting can satisfy the first nine axioms of the
definition of a vector space (you can try to show that), but it
violates the tenth axiom
1(1, 1) = (1, 0)  (1, 1)
the set (together with the two given operations) is
not a vector space
3.19
▪ Theorem 3.3: Properties of additive identity and additive inverse
Let v be any element of a vector space V, and let c be any
scalar. Then the following properties are true
(1) 0 v = 0
(2) c0 = 0
(3) If cv = 0, either c = 0 or v = 0
(4) (−1) v = − v (the additive inverse of v equals ((–1)v)

Pf:
(8) (9) (5)
(1) 0v = (c + (−c)) v = cv + (−c) v = cv + (−(cv)) = 0
3.20
(4) (7)
(2) c0 = c(0 + 0) = c0 + c0
 c0 + (−c0) = (c0 + c0) + (−c0) (add (–c0) to both sides)
(3)
 c0 + ( −c0) = c0 + [c0 + ( −c0)]
(5) (4)
 0 = c0 + 0  0 = c0
(3) Prove by contradiction: Suppose that cv = 0, but c  0 and v  0
 1  (9) 1
(10)
1
v = 1v =  c  v = ( cv ) = ( 0 ) = 0 (By the second property, c0 = 0)
c  c c
 →  if cv =0, either c = 0 or v = 0
(8)
(4) 0 v = (1 + (−1)) v = 1v + (−1) v
 0 = v + (−1) v (By the first property, 0v = 0)
(5)
 (−1) v = − v (By comparing with Axiom (5), (–1)v is the additive inverse of v)

3.21
Exercise 1
Let (d) be a straight line in xy-plane. Find the condition such
that (d) together with the usual + and scalar multiplication
form a vector space
y (d’)

x
O

(d)

3.22
Exercise 2
Decide whether W, equipped with the usual addition “+” and
usual scalar multiplication “.” on R3, is a vector space

(a) W = ( x1 , x2 , 2 x1 + 3 x2 ) x1 , x2  
(b) W = ( x1 , x2 ,3 x1 − 4 x2 + 1) x1 , x2  

3.23
3.3 Subspaces of Vector Spaces
◼ Subspace:
(V , +, ) : a vector space
W  
 : a nonempty subset of V
W V 
(W , +, ) : The nonempty subset W is called a subspace if W is
a vector space under the operations of vector
addition and scalar multiplication defined on V
▪ Trivial subspace :
Every vector space V has at least two subspaces
(1) Zero vector space {0} is a subspace of V (It satisfies the ten axioms)

(2) V is a subspace of V

※ Any subspaces other than these two are called proper (or nontrivial) subspaces
3.24
Test a subset to be a subspace
Theorem 3.4:
A nonempty subset W of a vector space V is a subspace of V if and
only if W is closed under addition and scalar multiplication, that is:

(1) If u and v are in W, then u+v is in W

(2) If u is in W and c is any scalar, then cu is in W

3.25
Pf:
1. Note that if u, v, and w are in W, then they are also in V.
Furthermore, W and V share the same operations.
Consequently, vector space axioms 2, 3, 7, 8, 9, and 10 are
satisfied automatically
2. Suppose that the closure conditions hold in Theorem 3.4, i.e.,
the axioms 1 and 6 for vector spaces are satisfied
3. Since the axiom 6 is satisfied (i.e., cu is in W if u is in W), we
can obtain
3.1. for a scalar c = 0, cu = 0 W   zero vector in W
 axiom 4 is satisfied
3.2. for a scalar c = −1, ( − 1)u  W   − u  ( − 1)u
st. u+( − u) = u+( − 1)u = 0
 axiom 5 is satisfied
3.26
▪ Ex 2: A subspace of M2×2
Let W be the set of all 2×2 symmetric matrices. Show that
W is a subspace of the vector space M2×2, with the
standard
Sol: operations of matrix addition and scalar multiplication
First, we knon that W , the set of all 2  2 symmetric matrices,
is an nonempty subset of the vector space M 22
Second,
A1  W, A2  W  ( A1 + A2 )T = A1T + A2T = A1 + A2 (A1 + A2 W )
c  R, A  W  (cA)T = cAT = cA (cA W )
The definition of a symmetric matrix A is that AT = A

Thus, Thm. 3.4 is applied to obtain that W is a subspace of M 22 3.27


◼ Ex 3: The set of singular matrices is not a subspace of M2×2
Let W be the set of singular (noninvertible) matrices of
order 2. Show that W is not a subspace of M2×2 with the
standard matrix operations

Sol:
1 0  0 0 
A=  W , B =   W
0 0  0 1 

1 0 
A+ B =   = I W (W is not closed under vector addition)
0 1 
W is not a subspace of M 22

3.28
▪ Ex 4: The set of first-quadrant vectors is not a subspace of R2
Show that W = {( x1 , x2 ) : x1  0 and x2  0} , with the standard
operations, is not a subspace of R2

Sol:
Let u = (1, 1) W
 (− 1)u = (− 1)(1, 1) = (− 1, − 1) W
(W is not closed under scalar multiplication)

W is not a subspace of R 2

3.29
▪ Ex 5: Identify subspaces of R3
Which of the following subsets is a subspace of R 3?
(a) W = ( x1 , x2 ,1) x1 , x2  R (Note: the zero vector is not in W)
(b) W = ( x1 , x1 + x3 , x3 ) x1 , x3  R (Note: the zero vector is in W)
Sol:
(a)

Consider v = (0, 0,1) W


(−1) v = (0, 0, −1) W
W is not a subspace of R 3

3.30
(b)

Consider v = (v1 , v1 + v3 , v3 )  W and u = (u1 , u1 + u3 , u3 )  W


v + u = ( v1 + u1 , ( v1 + u1 ) + ( v3 + u 3 ) , v3 + u3 ) W
cv = ( cv1 , ( cv1 ) + ( cv3 ) , cv3 )  W
W is closed under vector addition and scalar multiplication,
so W is a subspace of R3
3.31
◼ Notes: Subspaces of R3
(1) W consists of the single point 0 = ( 0, 0, 0 ) (trivial subspace)

(2) W consists of all points on a line passing through the origin


(3) W consists of all points on a plane passing through the origin
(The W in problem (b) is a plane passing through the origin)
(4) R 3 (trivial subspace)

※ According to Ex. 6 and Ex. 8, we can infer that if W is a subspace of a


vector space V, then both W and V must contain the same zero vector 0

3.32
▪ Theorem 3.5: The intersection of two subspaces is a subspace
If V and W are both subspaces of a vector space U ,
then the intersection of V and W (denoted by V  W )
is also a subspace of U
Pf:
(1) For v1 and v 2 in V  W , since v1 and v 2 are in V (and W ),
v1 + v 2 is in V (and W ). Therefore, v1 + v 2 is in V  W
(2) For v1 in V  W , since v1 is in V (and W ),
cv1 is in V (and W ). Therefore, cv1 is in V  W

Consequently, we can conclude the intersection of V and W


(V  W ) is also a subspace of U
※ However, the union of two subspaces is not a subspace
3.33
Quiz
1. Decide which are vector spaces. For those that are not, find
at least one axiom that fails to hold:
a) The set of all ordered triples of numbers {u=(x1, x2, x3)},
usual addition, and
r•(x1, x2, x3)= (3rx1, 3rx2, 3rx3)
b) The same as a) except
r•(x1, x2, x3)= (rx1, x2, x3)
2. Let a vector space V and a subset S of V are given.
Is S closed under addition? Is S closed under scalar
multiplication? Is S a subspace of V?
a) V=M3x3 , S={A|A is invertible}. Note that Mmxn is the set of
mxn matrices
b) V=M3x3 , S is the set of all 3x3 lower triangular matrices
3.34
3.4 Spanning Sets and Linear Independence

▪ Linear combination :
A vector u in a vector space V is called a linear combination of
the vectors v1 , v 2 , , v k in V if u can be written in the form

u = c1 v1 + c2 v 2 + + ck v k ,

where c1 , c2 , , ck are real numbers

3.35
▪ Ex 1: Finding a linear combination
v1 = (1,2,3) v 2 = (0,1,2) v 3 = ( − 1,0,1)
Prove (a) w = (1,1,1) is a linear combination of v1 , v 2 , v 3
(b) w = (1, − 2,2) is not a linear combination of v1 , v 2 , v 3
Sol:
(a) w = c1 v1 + c2 v 2 + c3 v 3

(1,1,1) = c1 (1,2,3) + c2 (0,1,2) + c3 (− 1,0,1)


= (c1 − c3 , 2c1 + c2 , 3c1 + 2c2 + c3 )

c1 − c3 =1
 2c1 + c2 =1
3c1 + 2c2 + c3 =1

3.36
1 0 − 1 1 1 0 − 1 1 

 2 1 0 1 
⎯⎯⎯⎯⎯⎯
Gauss Elimination
→0 1 2 − 1
 
3 2 1 1 0 0 0 0 

 c1 = 1 + t , c2 = −1 − 2t , c3 = t

(this system has infinitely many solutions)


t =1
 w = 2 v1 − 3v 2 + v 3
t =2
 w = 3v1 − 5v 2 + 2 v 3

3.37
(b)
w = c1 v1 + c2 v 2 + c3 v 3

1 0 − 1 1  1 0 − 1 1 
 2 1 0 − 2 ⎯⎯⎯
G.-J. E.
→ 0 1 2 − 4 
 
3 2 1 2  0 0 0 7 

 This system has no solution since the third row means


0  c1 + 0  c2 + 0  c3 = 7
 w can not be expressed as c1 v1 + c2 v 2 + c3 v 3

3.38
▪ The span of a set: span(S)
If S = {v1, v2,…, vk} is a set of vectors in a vector space V,
then the span of S is the set of all linear combinations of
the vectors in S,
span(S ) = c1v1 + c2 v 2 + + ck v k ci  R
(the set of all linear combinations of vectors in S )

▪ Definition of a spanning set of a vector space:


If every vector in a given vector space V can be written as a
linear combination of vectors in a set S, then S is called a
spanning set of the vector space V

3.39
▪ Notes: The above statement can be expressed as follows
span( S ) = V
 S spans (generates) V
 V is spanned (generated) by S
 S is a spanning set of V
▪ Ex 2:
(a) The set S = {(1, 0, 0), (0,1, 0), (0, 0,1)} spans R3 because any vector
u = (u1 , u2 , u3 ) in R 3 can be written as
u = u1 (1, 0, 0) + u2 (0,1, 0) + u3 (0, 0,1)

(b) The set S = {1, x, x 2 } spans P2 because any polynomial function


p( x) = a + bx + cx 2 in P2 can be written as
p( x) = a(1) + b( x) + c( x 2 ) 3.40
▪ Ex 3: A spanning set for R3
Show that the set S = (1,2,3),(0,1,2),(−2,0,1) spans R3
Sol:
We must examine whether any vector u = (u1 , u2 ,u3 ) in R3
can be expressed as a linear combination of v1 = (1, 2,3),
v 2 = (0,1, 2), and v3 = (−2, 0,1)
If u = c1 v1 + c2 v 2 + c3 v 3  c1 − 2c3 = u1
2c1 + c2 = u2
3c1 + 2c2 + c3 = u3

The above problem thus reduces to determine whether this


system is consistent for all values of u1 , u2 , and u3
3.41
1 0 −2
A=2 1 0 0
3 2 1

 Ax = u has exactly one solution for every u

 span( S ) = R 3

3.42
▪ Theorem 3.6: span(S) is a subspace of V
If S = {v1, v2,…, vk} is a set of vectors in a vector space V, then
(a) span(S) is a subspace of V
(b) span(S) is the smallest subspace of V that contains S, i.e.,
every other subspace of V containing S must contain span(S)
any subspace
containing S

 v1  

    vk



 v3   v2

span(S)
V

3.43
Pf:
(a)
Consider any two vectors u and v in span(S ), that is,
u =c1 v1 + c2 v 2 + + ck v k and v =d1 v1 + d 2 v 2 + + d k v k

Then u +v =(c1 +d1 ) v1 + (c2 +d 2 ) v 2 + + (ck +d k ) v k  span(S )


and cu =(cc1 ) v1 + (cc2 ) v 2 + + (cck ) v k  span(S ), because they
can be written as linear combinations of vectors in S

So, according to Theorem 3.4, we can conclude that span(S ) is


a subspace of V

3.44
(b)
Let U be another subspace of V that contains S , and we want to
show span(S )  U
k
Consider any u  span(S ), i.e., u =  ci v i , where v i  S
i =1
(because U is closed under
U is a subspace k vector addition and scalar
U contains S  v i  U  u =  ci v i  U multiplication, and any linear
combination can be evaluated
i =1 with finite vector additions
and scalar multiplications)

Since for any vector u  span(S ), u also belongs to U ,


we can conclude that span(S )  U , and therefore span(S )
is the smallest subspace of V that contains S = {v1 , v 2 , , v k }

3.45
Exercises

1. Determine whether the given set of vectors S spans vector


space V

a ) u1 = (1,1,1, 2), u2 = (2,1, 0, −3),


u 3 = (1, 0, −1, 4), u4 =(3,-2,2,4) in R 4
b) p ( x) = x − 2 x, q ( x) = −3x + x + 1,
2 2

and r ( x) = 2 x + 3 in P2

3.46
Linear Independence and Linear Dependence

Definition
S = v1 , v 2 , , v k : a set of vectors in a vector space V
For c1 v1 + c2 v 2 + + ck v k = 0

(1) If the equation has only the trivial solution (c1 = c2 = = ck = 0)


then S (or v1 , v 2 , , v k ) is called linearly independent
(2) If the equation has a nontrivial solution (i.e., not all ci 's are zero),
then S (or v1 , v 2 , , v k ) is called linearly dependent

3.47
▪ Ex 4: Testing for linear independence
Determine whether the following set of vectors in R3 is L.I. or L.D.
S = v1 , v 2 , v 3  = (1, 2, 3) , ( 0, 1, 2 ) , ( −2, 0, 1)
Sol: c1 − 2c3 = 0
c1 v1 + c2 v 2 + c3 v 3 = 0  2c1 + c2 + =0
3c1 + 2c2 + c3 = 0
1 0 − 2 0  1 0 0 0 
 2 1 0 0 ⎯⎯⎯ G.-J. E. 
→ 0 1 0 0 

3 2 1 0 0 0 1 0 
 c1 = c2 = c3 = 0 (only the trivial solution)
(or det(A) = −1  0, so there is only the trivial solution)
 S is (or v1 , v 2 , v 3 are) linearly independent 3.48
◼ Ex 5: Testing for linear independence
Determine whether the following set of vectors in P2 is L.I. or L.D.

S = v1 , v 2 , v 3  = 1 + x − 2 x 2 ,2 + 5 x − x 2 ,x + x 2 
Sol:
c1v1+c2v2+c3v3 = 0
i.e., c1(1+x – 2x2) + c2(2+5x – x2) + c3(x+x2) = 0+0x+0x2

c1+2c2 =0  1 2 0 0 1 2 0 0 
 1 5 1 0  ⎯⎯⎯  
 c1+5c2+c3 = 0   
G. E.
→ 0 1 3 0 
1
–2c1– c2+c3 = 0  −2 −1 1 0   
 0 0 0 0 
 This system has infinitely many solutions
(i.e., this system has nontrivial solutions, e.g., c1=2, c2= – 1, c3=3)
 S is (or v1, v2, v3 are) linearly dependent
3.49
◼ Ex 6: Testing for linear independence
Determine whether the following set of vectors in the 2×2
matrix space is L.I. or L.D.
  2 1  3 0  1 0  
S = v1 , v 2 , v 3  =    ,  , 
 0 1  2 1   2 0 

Sol:
c1v1+c2v2+c3v3 = 0

2 1  3 0 1 0   0 0 
c1   + c2   + c3   = 
 0 1  2 1   2 0   0 0 

3.50
 2c1+3c2+ c3 = 0
c1 =0
2c2+2c3 = 0
c1 + c 2 =0

2 3 1 0 1 0 0 0
1 0 0 0 G.-J. E. 0 1 0 0
  ⎯⎯⎯→ 
0 2 2 0  0 0 1 0
   
1 1 0 0 0 0 0 0

 c1 = c2 = c3= 0 (This system has only the trivial solution)

 S is linearly independent

3.51
◼ Theorem 3.7: A property of linearly dependent sets
A set S = {v1,v2,…,vk}, for k  2, is linearly dependent if
and only if at least one of the vectors vi in S can be written
as a linear combination of the other vectors in S

Pf:
()
c1v1+c2v2+…+civi+…+ckvk = 0

S is linearly dependent (there exist nontrivial solutions)

 ci  0 for some i
 c1   ci −1   ci +1   ck 
 v i =  −  v1 + +−  v i −1 +  −  v i +1 + +−  vk
 ci   ci   ci   ci  3.52
()
Let vi = d1v1+…+di-1vi-1+di+1vi+1+…+dkvk

 d1v1+…+di-1vi-1+(–1)vi +di+1vi+1+…+dkvk = 0
 c1 = d1 , c2 = d2 ,…, ci = –1 ,…, ck = dk (there exits at least this
nontrivial solution)
 S is linearly dependent

(A corollary is a must-be-true result based on


◼ Corollary to Theorem 3.7: the already proved theorem)
Two vectors u and v in a vector space V are linearly dependent
(for k = 2 in Theorem 3.7) if and only if one is a scalar multiple
of the other

3.53
Exercises
Let S={A, B, C, D} in V=M2x2

 0 −3  0 2
A=  , B=  ,
 2 −1  −5 1 
1 0  0 1 
C=  ,D =   in M 22
0 −3 3 6

a) Determine whether S spans V


b) Determine whether S is linearly independent or linearly
dependent

3.54
Exercises

Determine whether the following set is linearly independent or


linearly dependent

u = (1,1,1), v = (2,1, 0), and w = (1, 0, −1) in R 3

3.55
3.4 Basis and Dimension
◼ Basis: Linearly
Spanning Bases
V: a vector space Sets
Independent
Sets
S = {v1, v2, …, vn}V
V
(1) S spans V (i.e., span(S) = V)




(2) S is linearly independent

 S is called a basis for V

3.56
▪ Notes:
(1) the standard basis for R3:
{i, j, k}, for i = (1, 0, 0), j = (0, 1, 0), k = (0, 0, 1)

(2) the standard basis for Rn :


{e1, e2, …, en}, for e1=(1,0,…,0), e2=(0,1,…,0),…, en=(0,0,…,1)
Ex: For R4, {(1,0,0,0), (0,1,0,0), (0,0,1,0), (0,0,0,1)}

3.57
(3) the standard basis for mn matrix space:
 aij = 1
{ Eij | 1im , 1jn }, and in Eij 
other entries are zero
Ex: 2  2 matrix space:
1 0 0 1 0 0 0 0 
  ,  ,  , 
 0 0   0 0   1 0   0 1 

(4) the standard basis for Pn(x):


{1, x, x2, …, xn}
Ex: P3(x) {1, x, x2, x3}

3.58
▪ Ex 2: The nonstandard basis for R2
Show that S = {v1 ,v 2 } = {(1,1), (1, −1)} is a basis for R2

 c1 + c2 = u1
(1) For any u = (u1 , u2 )  R , c1 v1 + c2 v 2 = u  
2

c1 − c2 = u2

c1 + c2 = 0
(2) For c1 v1 + c2 v 2 = 0  
c1 − c2 = 0

According to the above two arguments, we can conclude that S


2
is a (nonstandard) basis for R
3.59
◼ Theorem 3.8: Uniqueness of basis representation for any vectors
If S = v1 , v 2 , , v n  is a basis for a vector space V, then every
vector in V can be written in one and only one way as a linear
combination of vectors in S
Pf:
 (1) span(S) = V
 S is a basis  
 (2) S is linearly independent
 span(S) = V Let v = c1v1+c2v2+…+cnvn
v = b1v1+b2v2+…+bnvn
 v + (–1)v = 0 = (c1–b1)v1 + (c2 – b2)v2 + … + (cn – bn)vn
S is linearly independent  with only the trivial solution
 coefficients for v i are all zero
 c1 = b1 , c2 = b2 ,…, cn = bn (i.e., unique basis representation)
3.60
Theorem 3.9: Bases and linear dependence

If S = v1 , v 2 , , v n  is a basis for a vector space V, then


every set containing more than n vectors in V is linearly
dependent (In other words, every linearly independent set
contains at most n vectors)
Pf:
Let S1 = {u1, u2, …, um} , m > n
span( S ) = V
u1 = c11 v1 + c21 v 2 +  + cn1 v n
uiV  u 2 = c12 v1 + c22 v 2 +  + cn 2 v n

u m = c1m v1 + c2 m v 2 +  + cnm v n
3.61
Consider k1u1+k2u2+…+kmum= 0
(if ki’s are not all zero, S1 is linearly dependent)
 d1v1+d2v2+…+dnvn= 0 (di = ci1k1+ci2k2+…+cimkm)

 S is L.I.  di = 0 i i.e., c11 k1 + c12 k 2 +  + c1m k m = 0


c21k1 + c22 k 2 +  + c2 m k m = 0

cn1k1 + cn 2 k 2 +  + cnm k m = 0

 Theorem: If the homogeneous system has fewer equations


(n equations) than variables (k1, k2, …, km), then it must
have infinitely many solutions
 m > n  k1u1+k2u2+…+kmum = 0 has nontrivial (nonzero) solution
 S1 is linearly dependent 3.62
◼Theorem 3.10: Number of vectors in a basis
If a vector space V has one basis with n vectors, then every
basis for V has n vectors
Pf:

S = {v1, v2, …, vn}


are two bases with different sizes for V
S' = {u1, u2, …, um}
S is a basis spanning V  
  m  n
S ' is a set of L.I. vectors  
 n = m
S ' is a basis spanning V  
  n  m
S is a set of L.I. vectors  

3.63
◼ Dimension:
The dimension of a vector space V is defined to be the
number of vectors in a basis for V
V: a vector space S: a basis for V

 dim(V) = #(S) (the number of vectors in a basis S)

◼ Finite dimensional:
A vector space V is finite dimensional if it has a basis
consisting of a finite number of elements
◼ Infinite dimensional:
If a vector space V is not finite dimensional, then it is called
infinite dimensional

3.64
▪ Notes: dim(V) = n
Linearly
(1) dim({0}) = 0 Spanning
Bases Independent
Sets Sets

(2) Given dim(V) = n, for SV #(S)  n #(S) = n #(S)  n

S: a spanning set  #(S)  n


S: a L.I. set  #(S)  n

S: a basis  #(S) = n

(3) Given dim(V) = n, if W is a subspace of V  dim(W)  n

3.65
◼ Ex 3: Find the dimension of a vector space according to the
standard basis

(1) Vector space Rn  standard basis {e1 , e2 ,  , en}


 dim(Rn) = n
(2) Vector space Mmn  standard basis {Eij | 1im , 1jn}
 aij = 1
and in Eij 
other entries are zero
 dim(Mmn) = mn
(3) Vector space Pn(x)  standard basis {1, x, x2,  , xn}
 dim(Pn(x)) = n+1
(4) Vector space P(x)  standard basis {1, x, x2, }
 dim(P(x)) =  3.66
◼ Ex 4: Determining the dimension of a subspace of R3
(a) W = {(d, c – d, c): c and d are real numbers}
(b) W = {(2b, b, 0): b is a real number}
(Hint: find a set of L.I. vectors that spans the subspace, i.e., find a basis for the
Sol: subspace.)
(a) (d, c – d, c) = c(0, 1, 1) + d(1, – 1, 0)
 S = {(0, 1, 1) , (1, – 1, 0)} (S is L.I. and S spans W)
 S is a basis for W
 dim(W) = #(S) = 2
(b) (2b, b, 0) = b(2,1, 0)
 S = {(2, 1, 0)} spans W and S is L.I.
 S is a basis for W
 dim(W) = #(S) = 1
3.67
▪ Ex 5: Finding the dimension of a subspace of M22
Let W be the subspace of all symmetric matrices in M22.
What is the dimension of W?
Sol:

a b  

W =   a , b, c  R 

 b c  

a b  1 0 0 1 0 0
  = a  + b  + c 
 b c   0 0   1 0   0 1 
1 0 0 1 0 0 
 S =   ,  ,   spans W and S is L.I.
0 0 1 0 0 1 
 S is a basis for W  dim(W) = #(S) = 3
3.68
◼ Theorem 3.11: Methods to identify a basis in an n-dimensional
space
Let V be a vector space of dimension n
(1) If S = v1 , v 2 , , v n  is a linearly independent set of
vectors in V, then S is a basis for V
(2) If S = v1 , v 2 , , v n  spans V, then S is a basis for V
(Both results are due to the fact that #(S) = n)

dim(V) = n

Linearly
Spanning
Bases Independent
Sets
Sets

#(S)  n #(S)  n
#(S) = n 3.69
Exercises

Determine whether the vectors form a basis for the given vector
space

a) u = (2,1, −2), v = (2,1, 4), and w = (1,3, −1) in R 3


 3 −2   −4 2 
b) A =   , B=  ,
 2 −1  −1 1 
1 1  3 −3
C=  ,D =   in M 22
 4 −2  1 4 

3.70
3.6 Rank of a Matrix
◼ Definition. The rank of a matrix A, denoted by rk(A), is the
maximum number of linearly independent row vectors of A.
◼ We say that a matrix A is row-equivalent to a matrix B if A
can be obtained from B by (finitely many) elementary row
operations

3.71
Rank of a Matrix
◼ The maximum number of linearly independent row vectors of
a matrix does not change if we change the order of rows or
multiply a row by a nonzero constant c, or take a linear
combination by adding a multiple of a row to another row.
This proves that rank is invariant under elementary row
operations

◼ Theorem: Row-equivalent matrices have the same rank

◼ Remark: we can determine rk(A) by reducing A to a matrix


in row-echelon form U, and rk(U)=rk(A)

3.72
Rank of a Matrix

◼ Example: Determine the rank of the matrix

 3 0 2 2
 
A =  −6 42 24 54 
 21 −21 0 15 
 

3.73
Example
◼ Determine the rank of the matrix

 3 0 2 2 3 0 2 2
  R 2+ 2 R1  
A =  −6 42 24 54  ⎯⎯⎯ ⎯
R 3− 7 R1
→  0 42 28 58 
 21 −21 0 15   0 −21 −14 1 
  
3 0 2 2 Row vectors
R 3+ R 2/2   containing
⎯⎯⎯⎯R 2/2
→ 0 21 14 29  = U pivots are
0 0 0 30  linearly

independent
The 3 row vectors of U are linearly independent.
So, rk(U)=3=rk(A)
3.74
Example 2
◼ Determine rk(A), where
 1 1 0 2
 
−2 −2 1 −5
A= 
 1 1 −1 3
 
 4 4 −1 9 
Solution:
 1 1 0 2 1 1 0 2

 −2 −2 1 −5 0 0 1 −1
A=
 1 1 −1 3 →⋯→ U =
  0 0 0 0
 4 4 −1 9  
0 0 0 0

rk(A)=2

3.75
A Criterion for Linearly Independent Vectors

Theorem. p vectors in Rn are linearly independent if the matrix


with these vectors as row vectors has rank p.
They are linearly dependent if that rank is less than p

Thus, to determine whether a set of p vectors in Rn is


independent or not, we form the matrix A whose rows are these
vectors Then, we check the condition rk(A)=p

3.76
Rank in terms of Column Vectors

Theorem. The rank of a matrix A equals the maximum number


of linearly independent column vectors of A
Hence A and its transpose AT have the same rank

Proof. For simplicity we write “rows” and “columns” for row


and column vectors. Let A be mxn matrix of rank r

3.77
Rank in terms of Column Vectors
◼ Proof…

A has r linear independent rows v(1) , v(2) ,..., v( r )


(regardless their position in A)
All the rows a(1) , a(2) ,..., a( m ) of A are linear
combinations of those, say
a(1) = c11v(1) + c12v(2) + ... + c1r v( r )
a(2) = c21v(1) + c22 v(2) + ... + c2 r v( r )

a( m ) = cm1v(1) + cm 2v(2) + ... + cmr v( r )


3.78
◼ These are vector equations for rows. To switch to column, we
re-write it in terms of components as n such systems, with
k=1,…,n

a1k = c11v1k + c12v2 k + ... + c1r vrk


a2 k = c21v1k + c22 v2 k + ... + c2 r vrk

amk = cm1v1k + cm 2v2 k + ... + cmr vrk

3.79
Proof…
◼ We collect components in columns:
 a1k   c11   c12   c1r 
a  c  c  c 
 2k  = v  21  + v  22  + + vrk  2 r 
  1k   2k    
       
 amk  cm1  cm 2   cmr 
◼ where k=1,…,n. The vector on left is the kth column of A.
◼ Each of these n columns is a linear combination of the same r
columns on the right
◼ Hence A cannot have more linearly independent columns
than rows, whose number is rank(A)=r

3.80
Proof…
◼ Rows of A are columns of AT
◼ For AT our conclusion is that AT cannot have more linearly
independent columns than rows
◼ So that A cannot have more linearly independent rows than
columns
◼ Together, the number of linearly independent columns of A
=rank(A)=r

3.81
Exercises

1. Determine whether the vectors form a basis for the given


vector space
u = (3, 4, −2), v = (−2,5, 4), and w = (2,3, −4) in R 3

2. Determine rk(A)
 2 2 −1 3 
 0 6 5 11 
A= 
1 −2 −3 −4 
 
3 3 2 2 

3.82
Null space
Let A be an m x n matrix and consider the
linear system Ax = 0 where x  Rn. Define
W = {x  Rn | Ax = 0 }.
W is a subspace of Rn called the null space of A, denoted by
NS(A).
Let x and y be solutions of the linear system,
i.e. Ax = 0 and Ay = 0.
Then A( x + y ) = Ax + Ay = 0 + 0 = 0.
So, x + y  W. Let c be a scalar, then
A(c x) = c ( Ax ) = c 0 = 0. So, c x  W and W is a subspace
of Rn. The null space is also called the kernel of A

3.83
Example
◼ Find a basis and dimension of NS(A), where

 1 1 0 2
 
 −2 −2 1 −5
A=
 1 1 −1 3
 
Solution:  4 4 −1 9

1 1 0 2 0 1 1 0 2 0
   
 −2 −2 1 −5 0  0 0 1 −1 0
A= → ... →
1 1 −1 3 0 0 0 0 0 0
   
 4 4 −1 −9 0 0 0 0 0 0

3.84
1 1 0 2 0 x1 + x2 + 2 x4 = 0

0 0 1 −1 0 x3 − x4 = 0
0 0 0 0 0
  Set x4 = s
0 0 0 0 0
→ x3 = s
Set x2 = r
Solution of Ax=0: → x1 = − r − 2s

 −r − 2s   −r   −2s   −1   −2
         
r r 0  = r   + s  0
1
x= = +
 s   0  s   0  1
         
 s   0   s   
0  1 
 −1   −2 
 1  0
u =   , v =   span NS ( A)
 0  1
 0  1
   
3.85
u , v are linearly independent, since
au + bv = 0
 (− a − 2b, a, b, b)T = (0,0,0,0)T
a=b=0

So, {u, v} forms a basis for NS ( A)


dim( NS ( A)) = 2

3.86
Row Space and Column Space
◼ Given an mxn matrix A. We call the span of the row vectors
the row space of A, denoted by RS(A)
◼ and the span of the column vectors the column space of A,
denoted by CS(A)

◼ Theorem. dim(RS(A))=dim(CS(A)= rk(A)=n-dim(NS(A)),


where n=number of columns of A

3.87
Important Remarks
1. RS(U) = span{nonzero vectors of U}, where U is a matrix
in row echelon form
2. If B is obtained from A by an elementary row operation,
then RS(B)=RS(A)
3. If A is reduced to a row echelon form matrix U by using
elementary row operations, then RS(A)=RS(U)
4. Thus, to find RS(A), we reduce it to U, and
RS(A)=span{nonzero vectors of U}, where U is a matrix
in row echelon form

3.88
Example
◼ Determine rk(A), find a basis for RS(A), where
 1 1 0 2
 
−2 −2 1 −5
A= 
 1 1 −1 3
 
 4 4 −1 9 
Solution:
 1 1 0 2 1 1 0 2

−2 −2 1 −5 0 0 1 −1

A=
 1 1 −1 3 U =
 
0 0 0 0
 4 4 −1 9  
0 0 0 0

Basis of RS(A): u = (1,1, 0, 2), v = (0, 0,1, −1)

rk(A)=2
3.89
Quiz
1. Determine rk(A), find a basis for RS(A), a basis for
CS(A), a basis for NS(A), and verify
dim(RS(A))+dim(NS(A))=n

1 2 0 −2 3 1
 −3 −2 −4 5 3 −3
A= ,
 −4 −2 3 6 5 2
 
 −3 0 3 4 8 3

CS(A)=RS(AT)

3.90
Exercises
1. Show that Ax=b has a solution if and only if b is in CS(A)
2. Determine if b lies in the column space of the given
matrix. If it does, express b as a linear combination of the
columns

 1 −3  2
A= , b =  
2 5  15 

3.91
3.7. Inner Product Spaces
◼ Inner product of two vectors in n-space

Let u = (a1 , a2 ,..., an ) , v = (b1 , b2 ,..., bn )  n


: vectors
The inner product (or dot product, scalar product) of u and v,
denoted by (u,v), or u v, is defined by
n
(u, v) = u v =  ai bi = a1b1 + + anbn
i =1

I. Linearity:  ,   , u, v, w  n

Properties: (u +  v).w = u.w +  v.w,


II. Symmetry: u.v = v.u , u , v  n

III. Positive-definiteness: u  n
u.u  0, and u.u = 0 if and only if u = 0
3.92
Inner Product
◼ A vector space V is called an inner product space
if we can define an inner product on V. That is, any
pair u, v in V, is associated with a real number,
denoted by (u, v), called the inner product of u and v
such that
I. Linearity:  ,   , u, v, w  V
(u +  v, w) =  (u, w) +  (v, w),
II. Symmetry: (u, v) = (v, u ), u, v V
III. Positive-definiteness: u  V
(u, u )  0, and (u, u ) = 0 if and only if u = 0

3.93
Inner Product Spaces
◼ Vectors whose inner product is zero are called orthogonal.
That is (u,v)=0, then u is orthogonal to v
◼ The length or norm of a vector u in V is defined by

|| u ||= (u, u )
◼ A vector of norm 1 is called a unit vector

3.94
Properties of norms

| (u, v) ||| u || || v || (Cauchy-Schwarz inequality)


| | u + v |||| u || + || v || (Triangle inequality)
| | u + v ||2 + || u - v ||2 = 2(|| u ||2 + || v ||2 )
(Parallelogram equality)
(u + tv, u + tv) =|| u + tv ||  0, t
2

(u + tv, u + tv) = (u, u ) + t (u, v) + t (v, u ) + t 2 (v, v)


= t 2 (v, v) + 2(u, v)t + (u, u )  0, t
  ' = (u, v) 2 − || u ||2 || v ||2  0
| (u , v) ||| u |||| v ||
3.95
Example.
◼ Consider the vector space of all continuous functions on the
interval [a,b]

The inner product of f , g  C[a, b] is defined by


b
( f , g ) =  f ( x) g ( x)dx,
a

The norm of f  C[a, b] is given by


b
|| f ||= 
2
( f ( x )) dx
a

3.96
3.8 Linear Transformations: Concept of functions
◼ A function f consists of sets D and R, and a rule f that assigns
to each element x of D a unique element y=f(x) of R
◼ The set D is called the domain of f
◼ We write “f: D→ R”

3.97
Matrix induces a function
◼ If A is an mxn matrix, its induced function

fA : n
→ m
, defined by f A ( x) = Ax  m

◼ Example 1: Let A be the nxn identity matrix I. Then

f I ( x) = Ix = x, so f I : n
→ n
is just the identity function

3.98
Example 2
◼ Let
 −1 0 
A=  . Then
 0 2
 x   −1 0  x   − x 
fA   =    =  
 y   0 2  y   2 y 

 −x   x  0 0
f A (u ) =   , u =   f A (v ) =   , v =  
 0  0 2y  y
f A reflects the x-axis f A expands the y-axis

3.99
Example 2
◼ Let
 −1 0 
A=  . Then
 0 2
 x   −1 0  x   − x 
fA   =    =  
 y   0 2  y   2 y 

 −x   x
f A ( w) =   , w =  
2y  y
f A combines the two cases
for a general vector

3.100
Linear Transformations
◼ Definition

Let V and W be vector spaces. A function


T : V → W is called a linear transformation
if for all u, v V , and for all scalars r ,
T (u + v) = T (u ) + T (v)
T (ru ) = rT (u )

Theorem: If A is an m  n matrix, then its induced


function f A : n
→ m
is a linear transformation

3.101
Matrix induced by linear transformation

Let T : n
→ m
be a linear transformation.
Then there exists an m  n matrix A such that
T = fA
We call this A the matrix induced by T and
denote it by A = [T ]
◼ A is the matrix whose columns are just the image of elements
of the standard basis

3.102
Quiz
◼ Determine whether the given formula defines an inner
product

1) u = (a1 , a2 , a3 ), v = (b1 , b2 , b3 )  3

a ) (u, v) = a1b1 + a2b3 + a3b2


b) (u, v) =| a1b1 | + | a2b2 | + | a3b3 |
b
2) ( f , g ) =  f ( x) g ( x)dx, f , g  C[ a , b ]
a

C[a,b] is the vector space of all continuous functions on the


interval [a,b]
3.103
Homework Chapter 3

◼ Page 162: 2, 4, 8, 18, 22, 24


◼ Page 176: 6, 7, 8, 10, 12, 22
◼ Page 178: 37, 40, 42, 45, 48
◼ Page 186: 8, 10, 16, 18, 20, 24
◼ Page 197: 4, 6, 16, 20, 26, 36, 40
◼ Page 258: 2, 8, 10, 12, 18, 20, 24, 26
◼ Deadline: 4 weeks

3.104

You might also like