Ch5 Inner Product Spaces

CHAPTER 5
INNER PRODUCT
SPACES
5.1 Length and Dot Product in Rn
5.2 Inner Product Spaces
5.3 Orthonormal Bases: Gram-Schmidt Process
5.4 Mathematical Models and Least Square Analysis
5.5 Applications of Inner Product Space
Elementary Linear Algebra 投影片設計製作者

R. Larson (8 Edition) 淡江大學電機系翁慶昌教授
CH 5 Linear Algebra Applied
Electric/Magnetic Flux (p.240) Heart Rhythm Analysis (p.255)
Work (p.248)
Revenue (p.266) Torque (p.277)

2/101
5.1 Length and Dot Product in Rn
 Length:
The length of a vector v  (v1 , v2 ,, vn ) in Rn is given by
|| v ||  v1  v2    vn
2 2 2
 Notes: The length of a vector is also called its norm.
 Notes: Properties of length

1 v 0
2 v  1  v is called a unit vector.
3 v  0 iff v  0
4 cv  c v
Elementary Linear Algebra: Section 5.1, p.232 3/101
 Ex 1:
(a) In R5, the length of v  (0 ,  2 , 1 , 4 ,  2) is given by
|| v ||  0 2  (2) 2  12  4 2  (2) 2  25  5
(b) In R3 the length of v  ( 2

, 2
, 3
) is given by
17 17 17
2 2 2
 2   2   3  17
|| v ||         1
 17   17   17  17
(v is a unit vector)

 A standard unit vector in Rn:
e1 , e2 ,, en   1,0,,0, 0,1,,0, 0,0,,1
 Ex:
the standard unit vector in R2: i, j  1,0, 0,1
the standard unit vector in R3: i, j, k  1,0,0, 0,1,0, 0,0,1
 Notes: (Two nonzero vectors are parallel)

u  cv
1 c  0  u and v have the same direction
2 c  0  u and v have the opposite direction
 Thm 5.1: (Length of a scalar multiple)
Let v be a vector in Rn and c be a scalar. Then
|| cv ||  | c | || v ||
Pf:
v  (v1 , v2 ,  , vn )
 cv  (cv1 , cv2 ,  , cvn )
|| cv ||  || ( cv1 , cv2 ,  , cvn ) ||
 (cv1 ) 2  (cv2 ) 2    (cvn ) 2
 c 2 (v1  v2    vn )
2 2 2
 | c | v1  v2    vn
2 2 2
 | c | || v ||
 Thm 5.2: (Unit vector in the direction of v)
v
If v is a nonzero vector in Rn, then the vector u 
|| v ||
has length 1 and has the same direction as v. This vector u
is called the unit vector in the direction of v.
Pf:
1
v is nonzero  v  0  0
v
1
 u v (u has the same direction as v)
v
v 1
|| u ||   || v ||  1 (u has length 1 )
|| v || || v ||
 Notes:
v
(1) The vector is called the unit vector in the direction of v.
|| v ||
(2) The process of finding the unit vector in the direction of v
is called normalizing the vector v.

 Ex 2: (Finding a unit vector)
Find the unit vector in the direction of v  (3 ,  1 , 2) ,
and verify that this vector has length 1.
Sol:
v  (3 ,  1 , 2)  v  3   1  22  14
2 2
v (3 ,  1 , 2) 1  3 1 2 
   (3 ,  1 , 2)   , , 
|| v || 3  (1)  2
2 2 2
14  14 14 14 
2 2 2
 3   1   2  14
        1
 14   14   14  14
v
 is a unit vector.
v
 Distance between two vectors:
The distance between two vectors u and v in Rn is
d (u , v)  || u  v ||
 Notes: (Properties of distance)

(1) d (u , v)  0
(2) )  only
d (uif, vand 0 if uv
(3) d (u , v)  d ( v , u)

 Ex 3: (Finding the distance between two vectors)
The distance between u = (0, 2, 2) and v = (2, 0, 1) is
d (u , v) || u  v ||  || (0  2 , 2  0 , 2  1) ||
 (2) 2  2 2  12  3

 Dot product in Rn:
The dot product of u  (u1 , u 2 ,  , u n ) and v  (v1 , v 2 ,  , v n )
is the scalar quantity
u  v  u1v1  u 2 v 2    u n v n
 Ex 4: (Finding the dot product of two vectors)

The dot product of u=(1, 2, 0, -3) and v=(3, -2, 4, 2) is
u  v  (1)(3)  (2)(2)  (0)(4)  (3)(2)  7

 Thm 5.3: (Properties of the dot product)
If u, v, and w are vectors in Rn and c is a scalar,
then the following properties are true.
(1) u  v  v  u
(2) u  ( v  w)  u  v  u  w
(3) c(u  v)  (cu)  v  u  (cv)
(4) v  v  || v || 2
(5) v ,vand
0 v  vifand
0 only if v 0

 Euclidean n-space:
Rn was defined to be the set of all order n-tuples of real
numbers. When Rn is combined with the standard
operations of vector addition, scalar multiplication, vector
length, and the dot product, the resulting vector space is
called Euclidean n-space.

 Ex 5: (Finding dot products)
u  (2 ,  2) , v  (5 , 8), w  (4 , 3)
(a) u  v (b) (u  v )w (c) u  (2 v ) (d) || w || 2 (e) u  ( v  2w )
Sol:
(a) u  v  (2)(5)  (2)(8)  6
(b) (u  v)w  6w  6(4 , 3)  (24 , 18)
(c) u  (2v)  2(u  v)  2(6)  12
(d) || w ||2  w  w  (4)(4)  (3)(3)  25
(e) v  2w  (5  (8) , 8  6)  (13 , 2)
u  ( v  2w )  (2)(13)  (2)( 2)  26  4  22

 Ex 6: (Using the properties of the dot product)
Given u  u  39 u  v  3 v  v  79
Find (u  2 v)  (3u  v)
Sol:
(u  2 v)  (3u  v)  u  (3u  v)  2 v  (3u  v)
 u  (3u)  u  v  (2 v)  (3u)  (2 v)  v
 3(u  u)  u  v  6( v  u)  2( v  v)
 3(u  u)  7(u  v)  2( v  v)
 3(39)  7(3)  2(79)  254

 Thm 5.4: (The Cauchy - Schwarz inequality)
If u and v are vectors in Rn, then
| u  v |  || u || || v || ( | u  v | denotes the absolute value of u  v )
 Ex 7: (An example of the Cauchy - Schwarz inequality)

Verify the Cauchy - Schwarz inequality for u=(1, -1, 3)
and v=(2, 0, -1)
Sol: u  v  1, u  u  11, v  v  5
 u  v  1  1
u v  u  u  v  v  11  5  55
 uv  u v

 The angle between two vectors in Rn:
uv
cos   , 0  
|| u || || v ||
Opposite Same
u v  0 u v  0 u v  0 direction
direction
   
  
     0    0
2 2 2
cos  1 cos  0 cos  0 cos  0 cos  1
 Note:
The angle between the zero vector and another vector is
not defined.

 Ex 8: (Finding the angle between two vectors)
u  (4 , 0 , 2 ,  2) v  (2 , 0 ,  1 , 1)
Sol:
u  u u   42  02  22   22  24
v  vv  2 2
   
 0   1  12
2 2
 6
u  v  (4)( 2)  (0)(0)  (2)( 1)  (2)(1)  12
uv  12 12
 cos     1
|| u || || v || 24 6 144
    u and v have opposite directions. (u  2 v )

 Orthogonal vectors:
Two vectors u and v in Rn are orthogonal if
uv 0
 Note:
The vector 0 is said to be orthogonal to every vector.

 Ex 10: (Finding orthogonal vectors)
Determine all vectors in Rn that are orthogonal to u=(4, 2).
Sol:
u  ( 4 , 2) Let v  (v1 , v2 )
 u  v  (4 , 2)  (v1 , v2 )
 4v1  2v2  1 
4 2 0  1 0
0  2 
t
 v1  , v2  t
2
 t 
 v   ,t , tR
 2 

 Thm 5.5: (The triangle inequality)
If u and v are vectors in Rn, then || u  v ||  || u ||  || v ||
Pf:
|| u  v || 2  (u  v)  (u  v)
 u  (u  v)  v  (u  v)  u  u  2(u  v)  v  v
 || u ||2 2(u  v) || v ||2  || u ||2 2 | u  v |  || v ||2
 || u || 2 2 || u || || v ||  || v || 2
 (|| u ||  || v ||) 2
|| u  v ||  || u ||  || v ||
 Note:
Equality occurs in the triangle inequality if and only if
the vectors u and v have the same direction.

 Thm 5.6: (The Pythagorean theorem)
If u and v are vectors in Rn, then u and v are orthogonal
if and only if
|| u  v || 2 || u || 2  || v || 2

 Dot product and matrix multiplication:
 u1   v1 
u  v  (A vector u  (u1 , u2 ,  , un ) in Rn
u   2 v   2
  is represented as an n×1 column matrix)
   
u n  v n 
v1 
v 
u  v  u T v  [u1 u 2  u n ]  2   [u1v1  u 2 v 2    u n v n ]
 
 
v n 

Key Learning in Section 5.1
▪ Find the length of a vector and find a unit vector.
▪ Find the distance between two vectors.
▪ Find a dot product and the angle between two vectors, determine
orthogonality, and verify the Cauchy-Schwarz Inequality, the
triangle inequality, and the Pythagorean Theorem.
▪ Use a matrix product to represent a dot product.
25/101
Keywords in Section 5.1
 length: 長度
 norm: 範數
 unit vector: 單位向量
 standard unit vector : 標準單位向量
 normalizing: 單範化
 distance: 距離
 dot product: 點積
 Euclidean n-space: 歐基里德n維空間
 Cauchy-Schwarz inequality: 科西-舒瓦茲不等式
 angle: 夾角
 triangle inequality: 三角不等式
 Pythagorean theorem: 畢氏定理
26/101
5.2 Inner Product Spaces
 Inner product:
Let u, v, and w be vectors in a vector space V, and let c be
any scalar. An inner product on V is a function that associates
a real number <u, v> with each pair of vectors u and v and
satisfies the following axioms.
(1) 〈u , v〉〈v , u〉
(2) 〈u , v  w〉〈u , v〉〈u , w〉
(3) c〈u , v〉〈cu , v〉
v〉 0
(4) 〈v ,and 〈v , ifv〉 0only if
and v0

 Note:
u  v  dot product (Euclidean inner product for R n )
 u , v  general inner product for vector space V
 Note:
A vector space V with an inner product is called an inner
product space.
Vector space: V , , 
Inner product space: V , , , , 
Elementary Linear Algebra: Section 5.2, Addition 28/101

 Ex 1: (The Euclidean inner product for Rn)
Show that the dot product in Rn satisfies the four axioms
of an inner product.
Sol:
u  (u1 , u2 ,, un ) , v  (v1 , v2 ,, vn )
〈u , v〉 u  v  u1v1  u2v2    un vn
By Theorem 5.3, this dot product satisfies the required four axioms.
Thus it is an inner product on Rn.

 Ex 2: (A different inner product for Rn)
Show that the function defines an inner product on R2,
where u  (u1 , u2 ) and v  (v1 , v2 ) .
〈u , v〉 u1v1  2u 2 v 2
Sol:
(a) 〈u , v〉 u1v1  2u2v2  v1u1  2v2u2 〈v , u〉
(b) w  (w1 , w2 )
 〈u , v  w〉 u1 (v1  w1 )  2u2 (v2  w2 )
 u1v1  u1w1  2u2 v2  2u2 w2
 (u1v1  2u2 v2 )  (u1w1  2u2 w2 )
〈u , v〉〈u , w〉

(c) c〈u , v〉 c(u1v1  2u2v2 )  (cu1 )v1  2(cu2 )v2 〈cu , v〉
(d ) 〈v , v〉 v1  2v2  0
2 2
〈v , v〉 0  v1  2v2  0  v1  v2  0 ( v  0)
2 2
 Note: (An inner product on Rn)
〈u , v〉 c1u1v1  c2u2v2    cnun vn , ci  0

 Ex 3: (A function that is not an inner product)
Show that the following function is not an inner product on R3.
〈u  v〉 u1v1  2u 2 v 2  u 3 v3
Sol:
Let v  (1 , 2 , 1)
Then  v , v  (1)(1)  2(2)(2)  (1)(1)  6  0
Axiom 4 is not satisfied.

Thus this function is not an inner product on R3.

 Thm 5.7: (Properties of inner products)
Let u, v, and w be vectors in an inner product space V, and
let c be any real number.
(1) 〈0 , v〉〈v , 0〉 0
(2)〈u  v , w〉〈u , w〉〈v , w〉
(3) 〈u , cv〉 c〈u , v〉
 Norm (length) of u:
|| u ||  〈u , u〉
 Note:
|| u ||2 〈u , u〉

 Distance between u and v:
d (u , v) || u  v || u  v, u  v
 Angle between two nonzero vectors u and v:

〈u , v〉
cos   , 0  
|| u || || v ||
 Orthogonal: (u  v )
u and v are orthogonal if〈u , v〉 0 .

 Notes:
(1) If || v ||  1 , then v is called a unit vector.
(2) v 1 v (the unit vector in the

 
Normalizin g
v 0 v direction of v)
not a unit vector

 Ex 6: (Finding inner product)
p , q  a0b0  a1b1    anbn is an inner product
Let p( x)  1  2 x 2 , q( x)  4  2 x  x 2 be polynomials in P2 ( x)
(a) 〈p , q〉 ? (b) || q || ? (c) d ( p , q)  ?
Sol:
(a) 〈p , q〉 (1)(4)  (0)(2)  (2)(1)  2
(b) || q ||  〈q , q〉 4 2  (2) 2  12  21
(c)  p  q  3  2 x  3x 2
 d ( p , q)  || p  q ||   p  q, p  q
 (3) 2  2 2  (3) 2  22

 Properties of norm:
(1) || u ||  0
(2) || u ||  0 if and only if u  0
(3) || cu ||  | c | || u ||
 Properties of distance:
(1) d (u , v)  0
(2) d (u , v)  0 if and only if u  v
(3) d (u , v)  d ( v , u)

 Thm 5.8：
Let u and v be vectors in an inner product space V.
(1) Cauchy-Schwarz inequality:
| u , v〉|  || u || || v ||
〈 Theorem 5.4
(2) Triangle inequality:
|| u  v ||  || u ||  || v || Theorem 5.5
(3) Pythagorean theorem :
u and v are orthogonal if and only if
|| u  v || 2  || u || 2  || v || 2 Theorem 5.6

 Orthogonal projections in inner product spaces:
Let u and v be two vectors in an inner product space V,
such that v  0. Then the orthogonal projection of u
onto v is given by
u , v
projv u  v
 v , v
 Note:
If v is a init vector, then 〈v , v〉 || v ||2  1.
The formula for the orthogonal projection of u onto v
takes the following simpler form.
projvu  u , v v

 Ex 10: (Finding an orthogonal projection in R3)
Use the Euclidean inner product in R3 to find the
orthogonal projection of u=(6, 2, 4) onto v=(1, 2, 0).
Sol:
 u , v  (6)(1)  (2)( 2)  (4)(0)  10
 v , v  12  22  02  5
uv
 projv u  v  105 (1 , 2 , 0)  (2 , 4 , 0)
vv
 Note:
u  projv u  (6, 2, 4)  (2, 4, 0)  (4,  2, 4) is orthogonal to v  (1,2,0).

 Thm 5.9: (Orthogonal projection and distance)
Let u and v be two vectors in an inner product space V,
such that v  0 . Then
u , v
d (u , projv u)  d (u , cv) , c
 v , v

▪ Determine whether a function defines an inner product, and find
the inner product of two vectors in Rn, Mm,n, Pn and C[a, b].
▪ Find an orthogonal projection of a vector onto another vector in
an inner product space.
42/101
 inner product: 內積
 inner product space: 內積空間
 norm: 範數
 distance: 距離
 angle: 夾角
 orthogonal: 正交
 unit vector: 單位向量
 normalizing: 單範化
 Cauchy – Schwarz inequality: 科西 - 舒瓦茲不等式
 triangle inequality: 三角不等式
 Pythagorean theorem: 畢氏定理
 orthogonal projection: 正交投影
43/101
5.3 Orthonormal Bases: Gram-Schmidt Process
 Orthogonal:
A set S of vectors in an inner product space V is called an
orthogonal set if every pair of vectors in the set is orthogonal.
S  v1 , v 2 ,, v n   V
 Orthonormal:  vi , v j   0 i j
An orthogonal set in which each vector is a unit vector is
called orthonormal.
S  v1 , v 2 ,  , v n   V
1 i j
 vi , v j   
 Note: 0 i j
If S is a basis, then it is called an orthogonal basis or an
orthonormal basis.
 Ex 1: (A nonstandard orthonormal basis for R3)
Show that the following set is an orthonormal basis.
v1 v2 v3
 1 1   2 2 2 2 2 2 1 
S   , , 0 ,     , 
 6 , 6 , 3 , 3
,
3 3 
 2 2   
Sol:
Show that the three vectors are mutually orthogonal.
v1  v 2   16  16  0  0
2 2
v1  v 3   0 0
3 2 3 2
2 2 2 2
v 2  v3     0
9 9 9

Show that each vector is of length 1.
|| v 1 ||  v 1  v1  1
2
 12  0  1
|| v 2 ||  v 2  v 2  2
36
 2
36
 8
9
1
|| v 3 ||  v 3  v 3  4
9
 94  1
9
1
Thus S is an orthonormal set.

 Ex 2: (An orthonormal basis for P3 ( x) )
In P3 ( x) , with the inner product
 p, q  a0b0  a1b1  a2b2
The standard basis B  {1, x, x 2 } is orthonormal.
Sol:
v1  1  0 x  0 x 2 , v2  0  x  0x2 , v3  0  0x  x2 ,
Then
 v1 , v 2   (1)(0)  (0)(1)  (0)(0)  0,
 v1 , v 3   (1)(0)  (0)(0)  (0)(1)  0,
 v 2 , v 3   (0)(0)  (1)(0)  (0)(1)  0

v1   v1 , v1   11  00  00  1,
v2  v2 , v2   00  11  00  1,
v3   v3 , v3   00  00  11  1

 Thm 5.10: (Orthogonal sets are linearly independent)
If S  v1 , v 2 ,, v n  is an orthogonal set of nonzero vectors
in an inner product space V, then S is linearly independent.
Pf:
S is an orthogonal set of nonzero vectors
i.e.  v i , v j   0 i  j and  v i , v i   0
Let c1 v1  c2 v 2    cn v n  0
 c1 v1  c2 v 2    cn v n , v i   0, v i   0 i
 c1  v1 , v i   c2  v 2 , v i     ci  v i , v i     cn  v n , v i 
 ci  v i , v i 
  v i , v i   0  ci  0 i  S is linearly independen t.
 Corollary to Thm 5.10:
If V is an inner product space of dimension n, then any
orthogonal set of n nonzero vectors is a basis for V.

 Ex 4: (Using orthogonality to test for a basis)
Show that the following set is a basis for R 4 .
v1 v2 v3 v4
S {(2 , 3 , 2 ,  2) , (1 , 0 , 0 , 1) , (1 , 0 , 2 , 1) , (1 , 2 ,  1 , 1)}
Sol:
v1 , v 2 , :vnonzero
3, v4
vectors
v1  v 2  2  0  0  2  0 v 2  v 3  1  0  0  1  0
v1  v 3  2  0  4  2  0 v 2  v 4  1  0  0  1  0
v1  v 4  2  6  2  2  0 v3  v 4  1  0  2  1  0
 S is orthogonal .
 S is a basis for R 4 (by Corollary to Theorem 5.10)

 Thm 5.11: (Coordinates relative to an orthonormal basis)
If B  {v 1 , v 2 ,  , v n } is an orthonormal basis for an inner
product space V, then the coordinate representation of a vector
w with respect to B is
w   w , v1  v1   w , v 2  v 2     w , v n  v n
Pf:
B  {v 1 , vis a basis for V
2 ,, vn}
w V
w  k1v1  k2 v 2    kn v n (unique representation)
 B  {v1 , v 2 ,  , v n } is orthonormal
1 i j
  vi , v j   
0 i j
〈w , v i〉〈(k1 v1  k 2 v 2    k n v n ) , v i〉
 k〈
1 v1 , v i〉   k〈
i v i , v i〉   k〈
n v n , v i〉
 ki i
 w   w, v1  v1   w, v 2  v 2     w, v n  v n
 Note:
If B  {v 1 , v 2 ,  , v n } is an orthonormal basis for V and w V ,
Then the corresponding coordinate matrix of w relative to B is
  w, v1  
 w , v  
w B  2 
  
 
  w , v 
n 

 Ex 5: (Representing vectors relative to an orthonormal basis)
Find the coordinates of w = (5, -5, 2) relative to the following
orthonormal basis for R3 .
B  {( 53 , 54 , 0) , ( 54 , 53 , 0) , (0 , 0 , 1)}
Sol:
 w, v1   w  v1  (5 ,  5 , 2)  ( 53 , 54 , 0)  1
 w, v 2   w  v 2  (5,  5 , 2)  ( 54 , 53 , 0)  7
 w, v 3   w  v 3  (5 ,  5 , 2)  (0 , 0 , 1)  2
  1
 [w ]B   7 
 2 

 Gram-Schmidt orthonormalization process:
B  {u1 , uis
2 , , u n }
a basis for an inner product space V
Let v1  u1 w 1  sp a n ({ v 1 } )
〈u 2 , v1〉
v 2  u 2  projW1 u 2  u 2  v1
〈v1 , v1〉 w 2  sp a n ({ v 1 , v 2 } )
〈u 3 , v1〉〈u 3 , v 2〉
v 3  u 3  projW2 u 3  u 3  v1  v2
〈v1 , v1〉〈v 2 , v 2〉

n 1
〈v n , v i〉
v n  u n  projWn1 u n  u n   vi
i 1〈v i , v i〉
 B'  {v1 , v 2 , , v n } is an orthogonal basis.

v1 v 2 vn
 B' '  { , , , } is an orthonormal basis.
v1 v 2 vn
 Ex 7: (Applying the Gram-Schmidt orthonormalization process)
Apply the Gram-Schmidt process to the following basis.
u1 u2 u3
B  {(1 , 1 , 0) , (1 , 2 , 0) , (0 , 1 , 2)}
Sol: v1  u1  (1 , 1 , 0)
u 2  v1 3 1 1
v2  u2  v1  (1 , 2 , 0)  (1 , 1 , 0)  ( , , 0)
v1  v1 2 2 2
u 3  v1 u3  v 2
v3  u3  v1  v2
v1  v1 v2  v2
1 1/ 2 1 1
 (0 , 1 , 2)  (1 , 1 , 0)  ( , , 0)  (0 , 0 , 2)
2 1/ 2 2 2
Orthogonal basis
1 1
 B'  {v1 , v 2 , v 3}  {(1, 1, 0), ( , , 0), (0, 0, 2)}
2 2
Orthonormal basis
v1 v 2 v 3 1 1 1 1
 B' '  { , , }  {( , , 0), ( , , 0), (0, 0, 1)}
v1 v 2 v 3 2 2 2 2

 Ex 10: (Alternative form of Gram-Schmidt orthonormalization process)
Find an orthonormal basis for the solution space of the
homogeneous system of linear equations.
x1  x 2  7 x4  0
2 x1  x 2  2 x3  6 x 4  0
Sol:
1 1 0 7 0 1 0 2  1 0
 2 1 2 6 0 G. J .E
0 1  2 8 0
   
 x1   2s  t   2  1 
 x   2s  8t   2   8
  2     s   t  
 x3   s   1   0 
       
 4  t   0   1 
x
Thus one basis for the solution space is
B  {v1 , v 2}  {( 2 , 2 , 1 , 0) , (1 ,  8 , 0 , 1)}
v1  u1   2, 2, 1, 0 
 u 2 , v1   18
v2  u2  v1  1,  8, 0, 1   2, 2, 1, 0
 v1 , v1  9
  3,  4, 2, 1
 B'   2,2,1,0 3,4,2,1 (orthogonal basis)
  2 2 1    3  4 2 1 
 B' '   , , ,0  ,  , , , 
 3 3 3   30 30 30 30 
(orthonormal basis)

▪ Show that a set of vectors is orthogonal and forms an
orthonormal basis, and represent a vector relative to an
orthonormal basis.
▪ Apply the Gram-Schmidt orthonormalization process.
60/101
 orthogonal set: 正交集合
 orthonormal set: 單範正交集合
 orthogonal basis: 正交基底
 orthonormal basis: 單範正交基底
 linear independent: 線性獨立
 Gram-Schmidt Process: Gram-Schmidt過程
61/101
5.4 Mathematical Models and Least Squares Analysis
 Orthogonal subspaces:
The subspaces W1 and W2 of an inner product space V are orthogonal
if v1 , v 2   0 for all v1 in W1 and all v 2 in W2 .
 Ex 2: (Orthogonal subspaces)
The subspaces
1 1 - 1
W1  span( 0, 1 ) and W2  span(  1  )
1 0  1 
are orthogonal because v1 , v 2   0 for any vector in W1 and any vector in W2 is zero .

 Orthogonal complement of W:
Let W be a subspace of an inner product space V.
(a) A vector u in V is said to orthogonal to W,
if u is orthogonal to every vector in W.
(b) The set of all vectors in V that are orthogonal to every
vector in W is called the orthogonal complement of W.
W   {v V |  v , w  0 ,  w W }
 Notes: W  (read “ W perp”)
(1) 0  V (2) V   0

 Notes:
W is a subspace of V
(1) W  is a subspace of V
(2) W  W   0
(3) (W  )   W
 Ex:
If V  R 2 , W  x  axis
Then (1) W   y - axis is a subspace of R 2
(2) W  W   (0,0)
(3) (W  )   W
Elementary Linear Algebra: Section 5.4, Addition 64/101

 Direct sum:
Let W1 and W2 be two subspaces of R n . If each vector x  R n
can be uniquely written as a sum of a vector w1 from W1
and a vector w 2 from W2 , x  w1  w2 , then R n is the
direct sum of W1 and W2 , and you can write .
R n  W1  W2
 Thm 5.13: (Properties of orthogonal subspaces)
Let W be a subspace of Rn. Then the following properties
are true.
(1) dim( W )  dim( W  )  n
(2) R n  W  W 
(3) (W  )   W
Elementary Linear Algebra: Section 5.4, pp.267-268 65/101
 Thm 5.14: (Projection onto a subspace)
If {u1 , u2 ,  , ut } is an orthonormal basis for the
subspace S of V, and v V , then
projW v   v , u1 u1   v , u 2 u 2     v , ut ut
Pf:
 projW v  W and {u1 , u2 ,  , ut } is an orthonorma l basis for W
 projW v   projW v, u1  u1     projW v, ut  ut

 v  projW  v , u1  u1    v  projW  v , ut  ut
( projW v  v  projW  v )
 v, u1  u1   v, ut  ut ( projW  v, ui   0,  i)

 Ex 5: (Projection onto a subspace)
w1  0, 3, 1, w2  2, 0, 0, v  1, 1, 3
Find the projection of the vector v onto the subspace W.
W  span({w1 , w 2})
Sol:
w1, w2  : an orthogonal basis for W
 w1 w 2   
 u1 , u 2    ), 1,0,0 :
3 1
,   (0, ,
 w1 w 2   10 10 
an orthonormal basis for W
projW v   v , u1  u1   v , u2  u2
)  1,0,0   (1, , )
6 3 1 9 3
 (0, ,
10 10 10 5 5
 Find by the other method:
A  w1 , w 2 , b  v
Ax  b
 x  ( AT A) 1 AT b
 projcs ( A)b  Ax  A( AT A) 1 AT b

 Thm 5.15: (Orthogonal projection and distance)
Let W be a subspace of an inner product space V, and v V.
Then for all w W , w  projW v
|| v  projW v ||  || v  w ||
or || v  projW v ||  min
wW
|| v  w ||
( projW v is the best approximation to v from W)

Pf:
v  w  ( v  projW v)  (projW v  w)
( v  projW v)  (projW v  w)
By the Pythagorean theorem
|| v  w ||2  || v  projW v ||2  || projW v  w ||2
w  projW v  projW v  w  0
|| v  w ||2  || v  projW v ||2
|| v  projW v |||| v  w ||

 Notes:
(1) Among all the scalar multiples of a vector u, the
orthogonal projection of v onto u is the one that is
closest to v. (p.250 Thm 5.9)
(2) Among all the vectors in the subspace W, the vector
projW v is the closest vector to v.

 Thm 5.16: (Fundamental subspaces of a matrix)
If A is an m × n matrix, then
(1) (CS ( A))   NS ( A )
( NS ( A ))   CS ( A)
(2) (CS ( A ))   NS ( A)
( NS ( A))   CS ( A )
(3) CS ( A)  NS ( AT )  R m CS ( A)  ( NS ( A))   R m
(4) CS ( AT )  NS ( A)  R n CS ( AT )  (CS ( A ))   R n

 Ex 6: (Fundamental subspaces)
Find the four fundamental subspaces of the matrix.
1 2 0
0 0 1 
A (reduced row-echelon form)
0 0 0
 
0 0 0
Sol:
CS ( A)  span 1,0,0,0 0,1,0,0 is a subspace of R 4
CS ( A )  RS  A  span 1,2,0 0,0,1 is a subspace of R3
NS ( A)  span  2,1,0 is a subspace of R3

1 0 0 0 1 0 0 0
A  2 0 0 0 ~ R  0 1 0 0
0 1 0 0 0 0 0 0
s t
NS ( A )  span 0,0,1,0 0,0,0,1 is a subspace of R 4
 Check:
(CS ( A))   NS ( A )
(CS ( A ))   NS ( A)
CS ( A)  NS ( AT )  R 4
CS ( AT )  NS ( A)  R3

 Ex 3 & Ex 4:
W  span ({w1 , w 2 })
Let W is a subspace of R4 and w1  (1, 2, 1, 0), w 2  (0, 0, 0, 1) .
(a) Find a basis for W
(b) Find a basis for the orthogonal complement of W.
Sol:
1 0 1 0
2 0 0 1
A ~R (reduced row-echelon form)
1 0 0 0
   
0 1 0 0
w1 w 2

(a) W  CS  A
 1,2,1,0, 0,0,0,1 is a basis for W
(b) W   CS  A  NS A 


 x1   2 s  t   2  1
x   s   1  0
1 2 1 0 
A  
  2     s   t  
 0 0 0 1   x3   t   0  1
x   0   0  0
 4      
  2,1,0,0   1,0,1,0  is a basis for W 
 Notes:
(1) dim(W )  dim(W  )  dim( R 4 )
(2) W  W   R 4
 Least squares problem:
Ax  b (A system of linear equations)
m  n n 1 m 1
(1) When the system is consistent, we can use the Gaussian

elimination with back-substitution to solve for x
(2) When the system is inconsistent, how to find the “best possible”
solution of the system. That is, the value of x for which the
difference between Ax and b is small.

 Least squares solution:
Given a system Ax = b of m linear equations in n unknowns,
the least squares problem is to find a vector x in Rn that
minimizes Ax  b with respect to the Euclidean inner
product on Rn. Such a vector is called a least squares
solution of Ax = b.
 Notes:
The least square problem is to find a vector xˆ in R n such that
Axˆ  projCS ( A ) b in the column space of A (i.e., Axˆ  CS ( A))
is as close as possible to b. That is,
b  projCS ( A ) b  b  Axˆ  minn b  Ax
xR

A  M mn
x  Rn
Ax  CS ( A) (CS  A is a subspace of R m )
 b  CS ( A) ( Ax  b is an inconsiste nt system )
Let Axˆ  projCS ( A ) b
 (b  Axˆ )  CS ( A)
 b  Axˆ  (CS ( A))   NS ( A )
 A (b  Axˆ )  0
i.e. A Axˆ  A b (the normal system associated with Ax = b)

 Note: (Ax = b is an inconsistent system)
The problem of finding the least squares solution of Ax  b
is equal to he problem of finding an exact solution of the
associated normal system A Axˆ  A b .

 Ex 7: (Solving the normal equations)
Find the least squares solution of the following system
Ax  b
1 1  0 
1 2 c 0   1
  c    (this system is inconsistent)
1 3  1  3
and find the orthogonal projection of b on the column space of A.

Sol:
1 1 
1 1 1   3 6 
A A
T
 1 2   
1 2 3 1 3  6 14 
 
0 
1 1 1    4 
A b
T
  1   
1 2 3 3 11
 
the associated normal system

AT Axˆ  AT b
3 6  c0   4 
6 14  c   11
  1   

the least squares solution of Ax = b
 53 
x̂   3 
 2 
the orthogonal projection of b on the column space of A
1 1 5  61 
  3 8
projCS ( A ) b  Ax  1 2  3    6 
ˆ
1 3    176 
2

▪ Define the least squares problem.
▪ Find the orthogonal complement of a subspace and the
projection of a vector onto a subspace.
▪ Find the four fundamental subspaces of a matrix.
▪ Solve a least squares problem.
▪ Use least squares for mathematical modeling.
84/101
 orthogonal to W: 正交於W
 orthogonal complement: 正交補集
 direct sum: 直和
 projection onto a subspace: 在子空間的投影
 fundamental subspaces: 基本子空間
 least squares problem: 最小平方問題
 normal equations: 一般方程式
85/101
5.5 Applications of Inner Product Spaces
 The cross product of two vectors in R3
A vector product that yields a vector in R3 is orthogonal to two

vectors. This vector product is called the cross product, and it
is most conveniently defined and calculated with vectors
written in standard unit vector form
v  (v1, v2 , v3 )  v1i  v2 j  v3k i  (1,0,0), j  (0,1,0), k  (0,0,1)

 Cross product of two vectors in R3:
Let u  u1i  u2 j  u3k and v  v1i  v2 j  v3k be vectors in R3.
The cross product of u and v is the vector
u  v  (u2v3  u3v2 )i  (u1v3  u3v1 ) j  (u1v2  u2v1 )k
i j k
u  v  u1 u2 u3 Components of u
v1 v2 v3 Components of v
u2 u3 u1 u3 u1 u2
u v  i j k
v2 v3 v1 v3 v1 v2
 (u2v3  u3v2 )i  (u1v3  u3v1 ) j  (u1v2  u2v1 )k
 Notes:
(1) The cross product is defined only for vectors in R3.
(2) The cross product of two vectors in R3 is orthogonal to two vectors.
(3) The cross product of two vectors in Rn, n ≠ 3 is not defined here.
 Ex 1: (Finding the Cross Product of Two Vectors)
u  i  2j  k v  3i  j  2k
Sol:
i j k
2 1 1 1 1 2
u v  1  2 1  i j k  3i  5 j  7k
1 2 3 2 3 1
3 1 2
i j k
1 2 3 2 3 1
vu  3 1  2  i j k  3i  5 j  7k
2 1 1 1 1 2
1 2 1
i j k
1 2 3 2 3 1
v v  3 1  2  i j k  0i  0 j  0k  0
1 2 3 2 3 1
2 1 2
 Thm 5.17: (Algebraic Properties of the Cross Product)
If u, v, and w are vectors in R3 and c is a scalar, then the following
properties are true.
1. u  v  ( v  u )
2. u  ( v  w )  (u  v)  (u  w )
3. c(u  v )  cu  v  u  cv
4. u  0  0  u  0
5. u  u  0
6. u  ( v  w )  (u  v )  w

Pf:
u  u1i  u2 j  u3k v  v1i  v2 j  v3k
i j k
u  v  u1 u2 u3  (u2v3  u3v2 )i  (u1v3  u3v1 ) j  (u1v2  u2v1 )k
v1 v2 v3
i j k
v  u  v1 v2 v3  (v2u3  v3u2 )i  (v1u3  v3u1 ) j  (v1u2  v2u1 )k
u1 u2 u3
 (u2v3  u3v2 )i  (u1v3  u3v1 ) j  (u1v2  u2v1 )k
 ( v  u )
 Note:
The vectors u × v and v × u have equal lengths but
opposite directions.
Elementary Linear Algebra: Section 5.5, p.278-279 90/101
 Thm 5.18: (Geometric Properties of the Cross Product)
If u and v are nonzero vectors in R3, then the following properties

are true.
1. u × v is orthogonal to both u and v.

2. The angle θ between u and v is given by u  v  u v sin  .
3. u and v are parallel if and only if u  v  0 .
4. The parallelogram having u and v as adjacent sides has an
area of u v .

Pf:
Base Height
Area  u v sin   u  v
 Notes:
(1) The three vectors u, v, and u × v form a right-handed
system.
(2) The three vectors u, v, and v × u form a left-handed system.
Ex 2: (Finding a Vector Orthogonal to Two Given Vectors)
u  i  4j  k v  2i  3j
Sol: i j k
u  v  1  4 1  3i  2 j  11k
length 2 3 0
u  v  (3)2  22  112  134
unit vector
u v 3 2 11
 i j k
u v 134 134 134
3 2 11
( , , )  (1,  4, 1)  0
134 134 134
3 2 11
( , , )  (2, 3, 0)  0
134 134 134
Ex 3: (Finding the Area of a Parallelogram)
u  3i  4 j  k v  2 j  6k
Sol:
i j k
u  v   3 4 1  26i  18 j  6k
0 2 6
area
u  v  262  182  6 2
 1036  32.19

 Find the cross product of two vectors in R3.
 Find the linear or quadratic least squares approximation of a
function.
 Find the nth-order Fourier approximation of a function.
95/101
 cross product: 外積
 parallelogram: 平行四邊形
96/101
5.1 Linear Algebra Applied
 Electric/Magnetic Flux
Electrical engineers can use the dot product to calculate

electric or magnetic flux, which is a measure of the
strength of the electric or magnetic field penetrating a
surface. Consider an arbitrarily shaped surface with an
element of area dA, normal (perpendicular) vector dA,
electric field vector E and magnetic field vector B. The
electric flux Φe can be found using the surface integral Φe
= ∫ E‧dA and the magnetic flux can be found using the
surface integral Φe = ∫ B‧dA. It is interesting to note that
for a given closed surface that surrounds an electrical
charge, the net electric flux is proportional to the charge,
but the net magnetic flux is zero. This is because electric
fields initiate at positive charges and terminate at
negative charges, but magnetic fields form closed loops,
so they do not initiate or terminate at any point. This
means that the magnetic field entering a closed surface
must equal the magnetic field leaving the closed surface.

 Work
The concept of work is important to scientists and

engineers for determining the energy needed to
perform various jobs. If a constant force F acts at an
angle  with the line of motion of an object to move
the object from point A to point B (see figure below),
then the work done by the force is given by
W  (cos  ) F AB
=F  AB
where AB represents the directed line segment from A
to B. The quantity (cos  ) F is the length of the
orthogonal projection of F onto AB Orthogonal
projections are discussed on the next page.

 Heart Rhythm Analysis
Time-frequency analysis of irregular physiological
signals, such as beat-to-beat cardiac rhythm variations
(also known as heart rate variability or HRV), can be
difficult. This is because the structure of a signal can
include multiple periodic, nonperiodic, and pseudo-
periodic components. Researchers have proposed and
validated a simplified HRV analysis method called
orthonormal-basis partitioning and time-frequency
representation (OPTR). This method can detect both
abrupt and slow changes in the HRV signal’s
structure, divide a nonstationary HRV signal into
segments that are “less nonstationary,” and determine
patterns in the HRV. The researchers found that
although it had poor time resolution with signals that
changed gradually, the OPTR method accurately
represented multicomponent and abrupt changes in
both real-life and simulated HRV signals.

 Revenues
The least squares problem has a wide variety of real-
life applications. To illustrate, in Examples 9 and 10
and Exercises 39, 40, and 41, are all least squares
analysis problems, and they involve such diverse
subject matter as world population, astronomy,
master’s degrees awarded, company revenues, and
galloping speeds of animals. In each of these
applications, you will be given a set of data and you
are asked to come up with mathematical model(s) for
the data. For example, in Exercise 40, you are given
the annual revenues from 2008 through 2013 for
General Dynamics Corporation. You are asked to find
the least squares regression quadratic and cubic
polynomials for the data, to predict the revenue for the
year 2018, and to decide which of the models appears
to be more accurate for predicting future revenues.

 Torque
In physics, the cross product can be used to measure

torque—the moment M of a force F about a point A as
shown in the figure below. When the point of
application of the force is B, the moment of F about A
is given by
M  AB  F
where AB represents the vector whose initial point is
A and whose terminal point is B. The magnitude of the
moment M measures the tendency of AB to rotate
counterclockwise about an axis directed along the
vector M.

Ch5 Inner Product Spaces

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ch5 Inner Product Spaces

Uploaded by

Copyright:

Available Formats

CHAPTER 5

Elementary Linear Algebra 投影片設計製作者

Electric/Magnetic Flux (p.240) Heart Rhythm Analysis (p.255)

Revenue (p.266) Torque (p.277)

 Notes: The length of a vector is also called its norm.

 Notes: Properties of length

(b) In R3 the length of v  ( 2

Elementary Linear Algebra: Section 5.1, p.232 4/101

the standard unit vector in R3: i, j, k  1,0,0, 0,1,0, 0,0,1

 Notes: (Two nonzero vectors are parallel)

Elementary Linear Algebra: Section 5.1, p.233 8/101

 Notes: (Properties of distance)

Elementary Linear Algebra: Section 5.1, p.234 10/101

Elementary Linear Algebra: Section 5.1, p.234 11/101

 Ex 4: (Finding the dot product of two vectors)

u  v  (1)(3)  (2)(2)  (0)(4)  (3)(2)  7

Elementary Linear Algebra: Section 5.1, p.235 12/101

Elementary Linear Algebra: Section 5.1, p.235 13/101

Elementary Linear Algebra: Section 5.1, p.235 14/101

Elementary Linear Algebra: Section 5.1, p.236 15/101

 3(39)  7(3)  2(79)  254

Elementary Linear Algebra: Section 5.1, p.236 16/101

 Ex 7: (An example of the Cauchy - Schwarz inequality)

Elementary Linear Algebra: Section 5.1, p.237 17/101

Elementary Linear Algebra: Section 5.1, p.238 18/101

    u and v have opposite directions. (u  2 v )

Elementary Linear Algebra: Section 5.1, p.238 19/101

Elementary Linear Algebra: Section 5.1, p.238 20/101

Elementary Linear Algebra: Section 5.1, p.238 21/101

Elementary Linear Algebra: Section 5.1, p.239 22/101

Elementary Linear Algebra: Section 5.1, p.239 23/101

Elementary Linear Algebra: Section 5.1, p.240 24/101

Elementary Linear Algebra: Section 5.2, p.243 27/101

Elementary Linear Algebra: Section 5.2, Addition 28/101

Elementary Linear Algebra: Section 5.2, p.243 29/101

Elementary Linear Algebra: Section 5.2, p.244 30/101

 Note: (An inner product on Rn)

〈u , v〉 c1u1v1  c2u2v2    cnun vn , ci  0

Elementary Linear Algebra: Section 5.2, p.244 31/101

Axiom 4 is not satisfied.

Elementary Linear Algebra: Section 5.2, p.244 32/101

Elementary Linear Algebra: Section 5.2, p.245 33/101

 Angle between two nonzero vectors u and v:

u and v are orthogonal if〈u , v〉 0 .

Elementary Linear Algebra: Section 5.2, p.246 34/101

(2) v 1 v (the unit vector in the

Elementary Linear Algebra: Section 5.2, p.246 35/101

Elementary Linear Algebra: Section 5.2, p.246 36/101

Elementary Linear Algebra: Section 5.2, p.247 37/101

Elementary Linear Algebra: Section 5.2, p.248 38/101

Elementary Linear Algebra: Section 5.2, p.249 39/101

Elementary Linear Algebra: Section 5.2, p.249 40/101

Elementary Linear Algebra: Section 5.2, p.260 41/101

Elementary Linear Algebra: Section 5.3, p.255 45/101

Thus S is an orthonormal set.

Elementary Linear Algebra: Section 5.3, p.255 46/101

The standard basis B  {1, x, x 2 } is orthonormal.

Elementary Linear Algebra: Section 5.3, p.255 47/101

Elementary Linear Algebra: Section 5.3, p.255 48/101

Elementary Linear Algebra: Section 5.3, p.257 50/101

Elementary Linear Algebra: Section 5.3, p.257 51/101

Elementary Linear Algebra: Section 5.3, p.258 53/101

Elementary Linear Algebra: Section 5.3, p.258 54/101