Chap.

5
Inner Product Spaces
n
5.1 Length and Dot Product in R
5.2 Inner Product Spaces
5.3 Orthonormal Bases: Gram-Schmidt Process
5.4 Mathematical Models & Least Squares Analysis
5.5 Applications of Inner Product Spaces
 5.1 Length and Dot Product
 Vectors in the plane can be characterized as directed line
segments having a certain length and direction.
 If v = (v1, v2), then the length, or magnitude, of v, denoted
by v , is defined to be v  v12  v22
v  v1  v2  v12  v22
2 2 2

(v1, v2 )
 The length or norm of a vector
v  (v1 , v2 , ..., vn ) in Rn is given
v
by v  v1  v2    vn
2 2 2
v2
 Unit vector: v  1
 Zero vector: v  0 v
1

Ming-Feng Yeh Chapter 5 5-2

Section 5-1

Standard Unit Vector

 Each vector in the standard basis for Rn has length 1 and is
called a standard unit vector.
 R2: {i, j} = {(1, 0), (0, 1)}
R3: {i, j, k} = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}
 (1, 1) = (1, 0) + (0, 1) = i + j
 (2, 2) = 2(1, 1) = 2i + 2j
(2, 2) = 2(1, 1)
 (1, 1)  2 , (2, 2)  2 2
 (2, 2)  2(1, 1)  2 (1, 1)
(1, 1)

1
2
(1, 1)  ( 2 , 2 )
2 2
j

 ( 2
, 2
) 1 i
2 2
Ming-Feng Yeh Chapter 5 5-3
Section 5-1

Length of a Scalar Multiple

 Two nonzero vectors u and v in Rn are parallel if
one is a scalar multiple of the other, i.e., u = cv.
If c > 0, then u and v have the same direction, and
if c < 0, then u and v have the opposite direction.
Theorem 5.1: Length of a Scalar Multiple
Let v be a vector in Rn and c be a scalar. Then
cv  c v
where c is the absolute value of c.

Ming-Feng Yeh Chapter 5 5-4

Section 5-1

Theorem 5.2
Unit Vector in the Direction of v
If v is a nonzero vector in Rn, then the vector u  v v
has length 1 and has the same direction as v. This
vector u is called the unit vector in the direction of v.
pf: 1. v is nonzero 1 v is positive and u  1v v  
Therefore u has the same direction as v.
v 1
2. u  v  v v  1
 The above process is called normalizing the vector v.

Ming-Feng Yeh Chapter 5 5-5

Section 5-1

Example 2
Find the unit vector in the direction of v = (3, 1, 2), and
verify that this vector has length 1.
Sol: The unit vector in the direction of v is
v

(3,  1, 2)

 3 14 ,  1 14 , 2 14 
v 3  (1)  2
2 2 2

which is a unit vector because

     
3 2
14
1 2
14
2 2
14

14
14
1

Ming-Feng Yeh Chapter 5 5-6

Section 5-1

Distance Between 2 Vectors: R 2

 The distance between two points in the plane,

(u1, u2) and (v1, v2), is given by
d  (u1  v1 )2  (u2  v2 )2
(v1, v2 )
d (u, v )
 In vector terminology, (u1 , u2 )
this distance can be viewed v
as the length of u – v, i.e., u

u  v  (u1  v1 ) 2  (u2  v2 ) 2

Ming-Feng Yeh Chapter 5 5-7

Section 5-1

Distance Between 2 Vectors: R n

 The distance between two vectors u and v in Rn is

d (u, v)  u  v
 Properties of distance:
1. d (u, v)  0
2. d (u, v)  0 if and only if u = v.
3. d (u, v )  d ( v, u)
 Let u = (0, 2, 2) and v = (2, 0, 1). Then
d (u, v)  u  v  (0  2, 2  0, 2  1)
 (2) 2  22  12  3
Ming-Feng Yeh Chapter 5 5-8
Section 5-1

Angle Between Two Vectors: R 2

 The angle  (0     ) between two nonzero vector

u = (u1, u2) and v = (v1, v2) is
u1v1  u2v2 v u
cos 
u v
The Law of Cosines
 v 
v  u  u  v  2 u v cos
2 2 2
u
u  v  vu
2 2 2

cos 
2u v
(u12  u22 )  (v12  v22 )  [( v1  u1 ) 2  (v2  u2 ) 2 ] u1v1  u2v2
 
2u v u v
Ming-Feng Yeh Chapter 5 5-9
Section 5-1

Dot Product
 The dot product of u = (u1, u2, …, un) and
v = (v1, v2, …, vn) is the scalar quantity
u  v  u1v1  u2v2    unvn
 Theorem 5.3: Properties of Dot Product
1. u  v  v  u
2. u  ( v  w )  u  v  u  w
3. c(u  v )  (cu)  v  u  (cv )
4. v  v  v  0
2

5. v  v  0 if and only if v = 0.

Ming-Feng Yeh Chapter 5 5-10

Section 5-1

Example 5
Given u = (2, –2), v = (5, 8) and w = (– 4, 3).
1. u  v  2(5)  (2)(8)  6
2. (u  v)w  6w  (24,  18)
3. u  (2 v )  2(u  v )  12
4. w  w  w  (4)(4)  3(3)  25
2

5. v  2w  5  (8), 8  6  (13, 2)
u  ( v  2w )  (2,  2)  (13, 2)  2(13)  (2)(2)  22

Ming-Feng Yeh Chapter 5 5-11

Section 5-1

Example 6
Given two vectors u and v in Rn such that u  u  39,
u  v  3, and v  v  79, evaluate (u  2 v)  (3u  v)
Sol: (u  2 v)  (3u  v)  u  (3u  v)  (2 v)  (3u  v)
 u  (3u)  u  v  (2 v)  (3u)  (2 v)  v
 3(u  u)  7(u  v)  2( v  v)
 3(39)  7(3)  2(79)
 254

Ming-Feng Yeh Chapter 5 5-12

Section 5-1

Angle Between Two Vectors: R n

 The angle  between two nonzero vectors u and v in Rn is

uv
cos  , 0  
u v

 0 0    2   2 
2
   
cos  1 cos  0 cos  0 cos  0 cos  1

 Two vectors u and v in Rn is orthogonal if u  v  0

 The zero vector 0 is orthogonal to every vector.

Ming-Feng Yeh Chapter 5 5-13

Section 5-1

Theorem 5.4
Cauchy-Schwarz Inequality
If u and v are vectors in Rn, then u  v  u v
where u  v denotes the absolute value of u  v
pf: If u = 0, then u  v  0  v  0, u v  0 v  0.
Hence the theorem is true if u = 0.
If u  0, from u  v  u v cos and cos  1,
we have u  v  u v cos  u v cos  u v .

Ming-Feng Yeh Chapter 5 5-14

Section 5-1

Example 7
Verify the Cauchy-Schwarz Inequality for u = (1, –1, 3) and
v = (2, 0, –1).
Sol: Because u  v  1, u  u  11 and v  v  5 , we have
u  v  1  1

u v  u  u v  v  11 5  55

Therefore u  v  u v

Ming-Feng Yeh Chapter 5 5-15

Section 5-1

Examples 8 & 9
Ex 8: The angle between u = (–4, 0, 2, –2) and
v = (2, 0, –1, 1) is given by
uv  12
cos    1    
u v 24 6
u and v should have opposite directions, because u = –2v.

Ex 9: The vectors u = (3, 2, –1, 4) and v = (1, –1, 1, 0) are

orthogonal because
u  v  (3)(1)  (2)(1)  (1)(1)  (4)(0)  0

Ming-Feng Yeh Chapter 5 5-16

Section 5-1

Example 10
Determine all vectors in R2 that are orthogonal to u = (4, 2).
Sol: Let v = (v1, v2) be orthogonal to u. Then
u  v  (4, 2)  (v1, v2 )  4v1  2v2  0
This implies that v2  2v1.
Therefore every vector that is
orthogonal to (4, 2) is of the form
v  (t ,  2t )  t (1,  2), t  R

Ming-Feng Yeh Chapter 5 5-17

Section 5-1

Theorem 5.5: Triangle Inequality

If u and v are vectors in Rn, then u  v  u  v .
pf: u  v  (u  v )  (u  v )  u  (u  v)  v  (u  v )
2

uv  u  u  2(u  v )  v  v
 u  2(u  v )  v
2 2

 u  2u  v  v u  v   u  v
2 2

 u 2u v  v uv  u v
2 2
u
v
 ( u  v ) 2 Equality occurs if and only if
 u  v  u  v the vectors u and v have the
same directions.
Ming-Feng Yeh Chapter 5 5-18
Section 5-1

Theorem 5.6: Pythagorean Thm

If u and v are vectors in Rn, then u and v are
orthogonal iff u  v  u  v .
2 2 2

pf: If u and v are orthogonal, then u  v  0

u  v  u  2(u  v)  v
2 2 2

 u  v
2 2

Ming-Feng Yeh Chapter 5 5-19

Section 5-1

Dot Product and

Matrix Multiplication
 Represent a vector in Rn as an n  1 column matrix.
 u1   v1 
u  v 
Let u    and v    . Then
2 2

 
   
 n
u vn 
 v1 
v 
u  v  uT v  u1 u2  un  2   [u1v1  u2v2    un vn ]

 
vn 
Ming-Feng Yeh Chapter 5 5-20
 5.2 Inner Product Spaces
 u  v = dot product (Euclidean inner product for Rn)
u, v = general inner product for vector space V.
 Definition: Let u, v, and w be vectors in a vector space V,
and let c be any scalar. An inner product on V is a
function that associates a real number <u, v> with pair of
vectors u and v and satisfies the following axioms.
1. u, v  v, u
2. u, v  w  v, u  u, w
3. c u, v  cu, v
4. v, v  0 and v, v  0 iff v = 0.

Ming-Feng Yeh Chapter 5 5-21

Section 5-2

Inner Product Space

 A vector space V with an inner product is called an inner
product space.
 Show that the following function defines an inner product
on R2: u, v  u1v1  2u2v2  u  v  u1v1  u2v2
where u  (u1 , u2 ) and v  (v1, v2 ).
pf: 1. u, v  u1v1  2u2v2  v1u1  2v2u2  v, u
2. Let w  (w1, w2 ) . Then
u, v  w  u1 (v1  w1 )  2u2 (v2  w2 )
 (u1v1  2u2v2 )  (u1w1  2u2  w2 )
 v , u  u, w
Ming-Feng Yeh Chapter 5 5-22
Section 5-2

Proof of Theorem 2
3. If c is any scalar, then
c u, v  c(u1v1  2u2v2 )
 (cu1 )v1  2(cu 2 )v2
 cu, v
4. Because the square of a real number is nonnegative,
v, v  v12  2v22  0
Moreover, this expression is equal to zero iff v = 0.

Ming-Feng Yeh Chapter 5 5-23

Section 5-2

Example 3
Show that the following function is not an inner product on R3:
u, v  u1v1  2u2v2  u3v3
where u  (u1 , u2 , u3 ) and v  (v1 , v2 , v3 ).
pf: Let v = (1, 2, 1). Then
v, v  (1)(1)  2(2)(2)  (1)(1)  6  0
Axiom 4 is not satisfied.

Ming-Feng Yeh Chapter 5 5-24

Section 5-2

Inner Product on M2,2 & Pn

 a11 a12  b11 b12 
 Let A    and B   
 21
a a 22   21 22 
b b
be matrices in the vector space M2,2. The function given by
A, B  a11b11  a21b21  a12b12  a22b22
is an inner product on M2,2.
 Let p  a0  a1 x    an x n
and q  b0  b1 x    bn x n

be polynomials in the vector space Pn. The function given

by p, q  a0b0  a1b1    anbn
is an inner product on Pn.
 The verification of the four inner product axioms is left to you.

Ming-Feng Yeh Chapter 5 5-25

Section 5-2

Theorem 5.7
Properties of Inner Products
Let u, v, and w be vectors in an inner product space
V, and let c be any real number.
1. 0, v  v, 0  0
2. u  v, w  u, w  v, w
3. u, cv  c u, v

Ming-Feng Yeh Chapter 5 5-26

Section 5-2

Norm, Distance & Angle

Let u and v be vectors in an inner product space V.
1. The norm (or length) of u is u  u, u
2. The distance between u and v is d (u, v)  u  v
3. The angle between two nonzero vectors u and v is given by
u, v
cos  , 0  
u v
4. u and v are orthogonal if u, v  0
 If v  1 , then v is called a unit vector.

 If v is any nonzero vector, then the vector u  v v

is called the unit vector in the direction of v.

Ming-Feng Yeh Chapter 5 5-27
Section 5-2

Example 6
Let p( x)  1  2 x 2 , q( x)  4  2 x  x 2 , and r ( x)  x  2 x 2
be polynomial in P2, and determine the following.
1. p, q  a0b0  a1b1  a2b2  (1)(4)  (0)(2)  (2)(1)  2
2. q, r  (4)(0)  (2)(1)  (1)(2)  0
3. q  q, q  42  (2)2  12  21
4. d ( p, q)  p  q  (1  2 x 2 )  (4  2 x  x 2 )
  3  2 x  3x 2

 (3) 2  22  (3) 2  22

Ming-Feng Yeh Chapter 5 5-28

Section 5-2

Theorem 5.8
Let u and v be vectors in an inner product space V.
1. Cauchy-Schwarz Inequality: u, v  u v
2. Triangle Inequality: u  v  u  v
3. Pythagorean Theorem:
u and v are orthogonal iff u  v  u  v
2 2 2

Ming-Feng Yeh Chapter 5 5-29

Section 5-2

Orthogonal Projections: R 2

 Let u and v be vectors in the plane. If v is nonzero, then u

can be orthogonally projected onto v. This projection is
denoted by projvu.
 projvu is a scalar multiple of v, i.e., projvu = av.
 If a > 0, then cos  0 and
u v cos u  v
av  a v  u cos  
v v
 a  u  v v  (u  v) ( v  v)
2
u
uv 
Therefore projvu  v v
vv projvu
Ming-Feng Yeh Chapter 5 5-30
Section 5-2

Projvu in R2 & Example 9

 If a < 0, then cos  0. The orthogonal projection of u
onto v is given by the same formula.
 Example 9: The orthogonal projection of u = (4, 2) onto
v = (3, 4) is given by
uv
projvu  v  20 (3, 4 )  ( 12 16
, 5)
vv 25 5
v

u u
projvu

v
projvu
Ming-Feng Yeh Chapter 5 5-31
Section 5-2

Orthogonal Projection & Ex 10

 Let u and v be vectors in an inner product space V, such
that v  0. Then the orthogonal projection of u onto v is
u, v
given by proj vu  v
v, v
 Example 10: The orthogonal projection of u = (6, 2, 4)
onto v = (1, 2, 0) is given by
uv
projvu  v  105 (1, 2, 0)  (2, 4, 0)
vv

Ming-Feng Yeh Chapter 5 5-32

Section 5-2

Remark of Example 10
 In Example 10, u = (6, 2, 4), v = (1, 2, 0), and
projvu = (2, 4, 0).
u – projvu = (6, 2, 4) – (2, 4, 0) = (4, –2, 4) is orthogonal
to v = (1, 2, 0).
 If u and v are nonzero vectors in an inner product space,
then u – projvu is orthogonal to v.

u d(u, projvu)
 v
projvu

Ming-Feng Yeh Chapter 5 5-33

Section 5-2

Theorem 5.9
Orthogonal Projection and Distance
Let u and v be vectors in an inner product space V,
such that v  0. Then
u, v
d (u, projvu)  d (u, cv), c 
v, v
u

d(u, cv)
d(u, projvu)

v
Ming-Feng Yeh
projvu Chapter 5 5-34
 5.3 Orthogonal Bases
 The standard basis for R3: B = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}
1. The three vectors are mutually orthogonal.
2. Each vector in the basis is a unit vector.

 Definition: Orthogonal & Orthonormal Sets

A set S of vectors in an inner product space V is called
orthogonal if every pair of vectors in S is orthogonal. If,
in addition, each vector in the set is a unit vector, then S is
called orthonomal.

Ming-Feng Yeh Chapter 5 5-35

Section 5-3

Orthonormal Basis
 For S = {v1, v2, …, vn},
Orthogonal Orthonormal
1. <vi, vj> = 0, i  j 1. <vi, vj> = 0, i  j
2. v i  1, i  1,2,..., n
If S is a basis, then it is called an orthogonal basis or an
orthonormal basis.
 The standard basis for Rn is orthomormal, but it is not the
only orthonormal basis for Rn. For example,
B  {(cos , sin , 0), ( sin , cos , 0), (0, 0, 1)}
is a nonstandard orthonormal basis for R3.

Ming-Feng Yeh Chapter 5 5-36

Section 5-3

Examples 1 & 2
 Example 1: Show that the following set
   
S  {v1 , v 2 , v 3}  12 , 12 , 0 ,  62 , 62 , 2 3 2 ,  23 ,  23 , 13 
is an orthonormal basis for R3.
Sol: 1. v1  v 2  0, v1  v 3  0, v 2  v 3  0
2. v1  v 2  v3  1
Therefore S is an orthonormal set.

 Example 2: In P3, with inner product

<p, q> = a0b0 + a1b1 + a2b2 + a3b3,
the standard basis B = {1, x, x2, x3} is orthonormal

Ming-Feng Yeh Chapter 5 5-37

Section 5-3

Theorem 5.10
Orthogonal Sets Are Linearly Independent
If S = {v1, v2, …, vn} is an orthogonal set of nonzero vectors
in an inner product space V, then S is linearly independent.
pf: Because S is orthogonal, <vi, vj> = 0, i  j.
<( c1v1+ c2v2 + … + cnvn), vi>
= c1<v1, vi>+ c2<v2, vi>+ …+ ci<vi, vi>+…+ cn<vn, vi>
= ci <vi, vi> = 0
  vi , vi  vi  0,  ci  0
2

Hence every ci must be zero and the set must be linearly

independent.

Ming-Feng Yeh Chapter 5 5-38

Section 5-3

Corollary 5.10 & Example 4

 Corollary 5.10
If V is an inner product space of dimension n, then any
orthogonal set of n nonzero vectors is a basis for V.
 Example 4: Show that the following set is a basis for R4.

S  {( 2, 3, 2,  2), (1, 0, 0, 1), (1, 0, 2, 1), (1, 2,  1, 1)}

Sol: Because v1  v 2  0, v1  v 3  0, v1  v 4  0
v 2  v 3  0, v 2  v 4  0
v3  v 4  0
Thus S is orthogonal.

Ming-Feng Yeh Chapter 5 5-39

Section 5-3

Theorem 5.11
Coordinates Relative to an Orthonormal Basis
If B = {v1, v2, …, vn} is an orthonormal basis for an inner
product space V, then the coordinate representation of a
vector w with respect to B is
w = <w, v1>v1 + <w, v2>v2 + … + <w, vn>vn
pf: Because B is a basis for V, then there exists unique scalars
c1, c2, …, cn, such that w = c1v1 + c2v2 + … + cnvn.
Taking the inner product of the both sides of this equation,
<w, vi> = <(c1v1 + c2v2 + … + cnvn), vi>
= c1<v1, vi> + c2<v2, vi> + … + cn<vn, vi>
= ci<vi, vi>
Because <vi, vi> = 1, <w, vi> = ci.
Ming-Feng Yeh Chapter 5 5-40
Section 5-3

Coordinate Matrix & Example 5

 The coordinates representation of w relative to the
orthonormal basis B = {v1, v2, …, vn} is
w = <w, v1>v1 + <w, v2>v2 + … + <w, vn>vn
The corresponding coordinate matrix of w relative to B is
wB  c1 c2  cn    w, v1   w, v 2    w, v n 
T T

 Example 5: Find the coordinates of w = (5, –5, 2) relative

to B  ( 53 , 54 , 0), ( 54 , 53 , 0), (0, 0, 1)
Sol: Because B is orthonormal,
w  v1  1, w  v 2  7, w  v 3  2
Thus [w]B   1  7 2
T

Ming-Feng Yeh Chapter 5 5-41

Section 5-3

Gram-Schmidt
Orthonormal Process #1
 Let B = {v1, v2, …, vn} be a basis for an inner product
space V.
 Let B  {w1 , w 2 , ..., w n } is given by
w1  v1
 v 2 , w1 
w2  v2  w1
 w 1 , w1 
 v 3 , w1   v3 , w 2 
w 3  v3  w1  w2
 w1 , w1   w2, w2 

 v n , w1   vn , w2   v n 1 , w n 1 
wn  vn  w1  w2    w n 1
 w 1 , w1   w2, w2   w n 1 , w n 1 
Then B  is an orthogonal basis for V.
Ming-Feng Yeh Chapter 5 5-42
Section 5-3

Gram-Schmidt
Orthonormal Process #2
 Let ui  w i w i . Then the set B  {u1 , u 2 , ..., u n }
is an orthonormal basis for V. Moreover,
span{v1, v2, …, vk} = span{u1, u2, …, uk} for k = 1,2,…,n.
 Let {v1, v2} be a basis for R2.
w1  v1
w 2  v 2  projv1 v 2 v2
 v 2 , w1  w2 v1  w1
 v2  w1
 w1 , w1 
{w1, w2} is an orthogonal basis. projv1 v 2
 w1 w 2 
u1, u 2    ,  is an orthonormal basis
 w1 w 2 
Ming-Feng Yeh Chapter 5 5-43
Section 5-3

Example 6
Apply the Gram-Schmidt orthogonormal process to the
following basis for R2: B = {(1, 1), (0, 1)}.
Sol: w1  v1  (1, 1)
(1,1)
 u1 
w1
w1
 
 22 , 22 (0,1)

v 2  w1
w2  v2  w1
w1  w 1

 (0, 1)  12 (1, 1)   21 , 12   2
2
, 
2
2
2
2
, 2

2

 u2 
w2
w2
 
  2 2 , 22

Ming-Feng Yeh Chapter 5 5-44

Section 5-3

Example 7
Apply the Gram-Schmidt orthogonormal process to the
following basis for R3: B = {(1, 1, 0), (1, 2, 0), (0, 1, 2)}.
Sol: w1  v1  (1, 1, 0)

 u1  w1 w1  22 , 22 , 0 
v w
w 2  v 2  2 1 w1  (1, 1, 0)  32 (1, 1, 0)   21 , 12 , 0
w1  w1

 u 2  w 2 w 2   2 2 , 22 , 0 
v 3  w1 v3  w 2 12
w 3  v3  w1  w 2  v 3  2 w1 
1
w2
w1  w1 w2  w2 12
 (0, 0, 2)
 u 3  w 3 w 3  0, 0, 1
Ming-Feng Yeh Chapter 5 5-45
Section 5-3

Example 8
The vectors v1 = (0, 1, 0) and v2 = (1, 1, 1) span a plane in R3.
Find an orthonormal basis for this subspace.
Sol: w1  v1  (0, 1, 0)
 u1  w1 w1  0, 1, 0  z
1
2
(1, 0,1) (1, 1, 1)
v 2  w1
w2  v2  w1 (0, 1, 0)
w1  w1 y
 (1, 1, 1)  11 (0, 1, 0)
 1, 0, 1

 u 2  w 2 w 2  22 , 0, 22 x
Ming-Feng Yeh Chapter 5 5-46
Section 5-3

Example 10
Find an orthonormal basis for the solution space of the
following homogeneous system of linear equations
x1  x2  7 x4  0
2 x1  x2  2 x3  6 x4  0
Sol: 1 1 0 7 0
  1 0 2  1 0
2 1 2 6 0  0 1  2 8 0
   
Let x3 = s and x4 = t,  x1    2  1 
x   2   8
 2   s   t  
 x3  1 0
     
 x4  0 1
Ming-Feng Yeh Chapter 5 5-47
Section 5-3

Example 10 (cont.)
One basis for solution space is
B  {v1, v 2}  {(2, 2, 1, 0), (1,  8, 0, 1)}
Apply the Gram-Schmidt orthonormalization process process
to the basis B:
w1  v1  (2, 2, 1, 0)
 u1  w1 w1  (2, 2, 1, 0)
v 2  w1
w2  v2  w1  (1,  8, 0, 1)  918  32 , 23 , 13 , 0
w1  w1
 (3,  4, 2, 1)
 u2  w 2 w 2  ( 3
30
, 4
30
, 2
30
, 1
30
)
Ming-Feng Yeh Chapter 5 5-48