Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

MATH 4A - Linear Algebra with Applications

Lecture 25: Angles and orthogonality

31 May 2019

Reading: §6.1-6.3
Recommended problems from §6.2: 1-29 odd
Announcement: please fill out online evaluations!
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Lecture plan

1 Angles between vectors

2 Orthogonal vectors and orthogonal complements

3 Orthogonal sets
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Recall: law of cosines

Consider a triangle, with side lengths a, b and c, such that c is

opposite an angle θ. The law of cosines says

c 2 = a2 + b 2 − 2ab cos θ.

(You don’t need to remember this.)

Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Applying the law of cosines to vectors

Let u and v be two nonzero vectors in Rn . Consider the triangle

with vertices determined by u, v and 0. The lengths of the sides of
this triangle are kuk, kvk and ku − vk, and, by the law of cosines,
the angle θ between u and v satisfies

ku − vk2 = kuk2 + kvk2 − 2kukkvk cos θ.

Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Rearranging and simplifying

We can rearrange

ku − vk2 = kuk2 + kvk2 − 2kukkvk cos θ

and then simplify:

1
kuk2 + kvk2 − ku − vk2

kukkvk cos θ =
2
1 2
u1 + · · · + un2 + v12 + · · · + vn2 − (u1 − v1 )2 − · · · − (un − vn )2

=
2
= u1 v1 + · · · + un vn = u · v.
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

The angle between two nonzero vectors vectors

We conclude that the angle θ between two vectors u and v in Rn

satisfies
kukkvk cos θ = u · v.
Since kuk =
6 0 6= kvk,
 
−1 u·v
θ = cos .
kukkvk

Note: there’s still some ambiguity because cos θ is not a one-to-one

function. We usually resolve this by requiring 0 ≤ θ ≤ π.
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

iClicker

Are the two vectors    

1 −3
−2 2
   
3 1
2 −2
orthogonal?
(a) Yes
(b) No
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Perpendicular lines and orthogonal vectors

Two lines in Rn are perpendicular when the angle between them is

π/2. We can reframe this geometric condition in terms of inner
products if we describe each line as the span (i.e. set of scalar
multiples) of a vector. See board.

The calculation we just did motivates the following definition: Two

vectors u and v in Rn are orthogonal to each other if u · v = 0.

In fact, using the law of cosines and some geometry, we can prove
a different characterization of orthogonality:
Theorem (Pythagorean theorem)
Two vectors u and v are orthogonal if and only if
ku + vk2 = kuk2 + kvk2 .
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Orthogonality with respect to subspaces

Whenever two lines meet, it make sense to ask about the angle
between them. In fact, this can be generalized to any two
intersecting hyperplanes (of possibly different dimensions), e.g.
two planes in R3 , or a line and a 3-dimensional hyperplane in R4 .
We will be especially interested in understanding when a line and a
hyperplane are orthogonal.

We recast this in terms of linear algebra as follows: if a vector z in

Rn is orthogonal to every vector in a subspace W of Rn , then z is
said to be orthogonal to W .

The orthogonal complement of a subspace W of Rn is the set of

all vectors in Rn that are orthogonal to W and is denoted by W ⊥
(usually read as “W perp”).
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Two important properties of orthogonal complements

1 A vector x is in W ⊥ if and only if x is orthogonal to every
vector in a spanning set of W .

(Why? One direction is easy. For the other direction, linearity

of dot product implies if x is orthogonal to some vectors, it is
also orthogonal to any linear combination of those vectors.)

2 If W is a subspace of Rn , then W ⊥ is also subspace of Rn ,

and (W ⊥ )⊥ = W .
(Why? Let B = {w1 , w2 , . . . , wp } be a basis of W . Then W ⊥
is the set of solutions to the homogeneous system of equations
 T   
w1 x1 0
w2T  x2  0
 ..   ..  =  ..  .)
    
 .   .  .
wpT xn 0
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Definition

A set of vectors {u1 , . . . , up } in Rn is said to be an orthogonal set

if each pair of distinct vectors from the set is orthogonal, that is, if
ui · uj = 0 whenever i 6= j.
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Testing orthogonality using matrices

Given the set of vectors {u1 , . . . , up } in Rn , form the n × p matrix

U = u1 u2 · · · up .

Then {u1 , . . . , up } is an orthogonal set if and only if the p×p

matrix
 T  
u1 u1 · u1 u1 · u2 ··· u1 · up
uT 
 2 · u1 u2 · u2
 u ··· u2 · up 
 2
U T U =  .  u1 u2 · · · up =  .

 ..   .. .. 
. 
uTp up · u1 up · u2 ··· up · up

is diagonal.
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Example

Show the set of vectors consisting of

     
3 −1 −1/2
u1 = 1 u2 =  2  u3 =  −2 
1 1 7/2

is orthogonal.
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Why we care: orthogonal is “even better” than linearly

independent

Theorem
If S = {u1 , . . . , up } is an orthogonal set of nonzero vectors in Rn ,
then S is linearly independent and hence is a basis for the subspace
spanned by S.

Motivates a definition: an orthogonal basis for a subspace W of

Rn is a basis for W that is also an orthogonal set.

Intuitively: an orthogonal basis is a basis where the angles between

any two of the vectors are all simultaneously as large as possible
(that is, they are all π/2). This makes such bases especially useful
in numerical linear algebra.
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Recall

Given a (not necessarily orthogonal) basis B = {b1 , b2 , . . . , bn } of

Rn , computing the B-coordinate vector of a vector v requires
solving the linear system


b1 b2 · · · bn x = v.

We know how to do this (use the row reduction algorithm) and it’s
not so bad...
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

But life is better with orthogonal bases

If B = {b1 , b2 , . . . , bn } is an orthogonal basis of Rn , then there are

explicit formulas for writing v as a linear combination of B. More
generally, we have the following:
Theorem
Let {u1 , u2 , . . . , up } be an orthogonal basis for a subspace W of
Rn . For each y in W , the weights in the linear combination

y = c1 u1 + · · · + cp up

are given by
y · uj
cj =
uj · uj
for each j = 1, . . . , p.
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

iClicker
Recall from earlier that the set of vectors B consisting of
     
3 −1 −1/2
u1 = 1 u2 =  2  u3 =  −2 
1 1 7/2
is an orthogonal basis of R3 . What is the u2 coordinate of
 
1
v = 2

1
in the B-basis?
(a) uu22·u·v2
(b) v·u
v·v
2

(c) uv·u
2 ·u2
1

v·u2
(d) u2 ·u2
(e) uv·u
3 ·u3
2
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Orthonormal bases

There is one more thing we can impose on an orthogonal set to

make it even nicer: require each of the vectors to have unit length.

A set {u1 , . . . , up } is an orthonormal set if it is an orthogonal set

of unit vectors. If W is spanned by such a set, we call the set
{u1 , . . . , up } an orthonormal basis of W .
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Important properties of orthonormal sets

1 An m × n matrix U has orthonormal columns if and only if

UT U = I .
2 (Ux) · (Uy) = x · y for any two vectors x and y in Rn . In
particular:
(i) kUxk = kxk for any vector x in Rn .
(ii) (Ux) · (Uy) = 0 if and only if x · y = 0
Intuition: when m = n, we call a matrix U with orthonormal
columns an orthogonal matrix (although a better name would be
orthonormal matrix...). The transformation x 7→ Ux is a rotation
of Rn .
Angles between vectors Orthogonal vectors and orthogonal complements Orthogonal sets

Examples

The standard basis of Rn is an orthonormal basis:

InT In = In In = In .

The set consisting of the vectors

 √   √   √ 
3/√11 −1/√ 6 −1/√66
v1 = 1/√11 v2 =  2/√6  v3 = −4/√ 66
1/ 11 1/ 6 7/ 66

is orthonormal. It forms an orthonormal basis of R3 .