MTH 309 Y

Recall:

1) If

LECTURE 17.1

1
.
..

u=

are vectors in R then

u v = 11 +

||u|| = u u

THU 2014.04.03

1
.
..

v=

dist(u v) = ||u v||

2) u, v are orthogonal vectors if u v = 0.

3) Pythagorean theorem: if u, v are orthogonal then

||u + v||2 = ||u||2 + ||v||2

MTH 309 Y

LECTURE 17.2

THU 2014.04.03

4) If V R is a subspace then an orthogonal basis of V is

a basis of V that consists of vectors that are orthogonal to
one another.

5) If V R is a subspace and = {v1 v } is an orthogonal

basis of V then for w V we have

1

.

w = ..

w v
where =
.
v v

6) Gram-Schmidt process:
arbitrary basis
= {v1 v }

Gram-Schmidt
process

of V R

w1 := v1

w2 := v2
w3 := v3

w1 v2
w1 w1
w1 v3
w1 w1

w1

w1

w2 v3
w2 w2

w2

orthogonal basis
= {w1 w }
of V R

w1 v
w1 v
w2 v
w := v w1w1 w1 w2w2 w2 w1w1 w1

MTH 309 Y

LECTURE 17.3

THU 2014.04.03

Next: Orthogonal projections of vectors.

Motivation.

90 cars/h
x2

72

A
C

r
a
c

s/h

x1

x3

x5

B
D

x4

0
2
1

h
/
rs

ca

45 cars/h

Problem. Find the flow rate of cars on each segment of streets.

MTH 309 Y

LECTURE 17.4

Upshot.

1) Recall: a matrix equation

THU 2014.04.03

Ax = b

has a solution i b Col(A).

2) In practical applications we may obtain a matrix equation Ax =

b that have no solutions, i.e. where b Col(A).

3) In such cases we may look for approximate solutions as follows:

replace b by a vector b such that b Col(A) and

dist(b b) is as small as possible;
then solve the equation Ax = b

Problem. How to find the vector b?

MTH 309 Y

LECTURE 17.5

THU 2014.04.03

Definition. Let V be a subspace of R. A vector w R is

orhogonal to V if
for all v V .

wv =0

Proposition. If V = Span(v1 v ) then a vector w is orthogonal to V i w is orthogonal to v1 v .

MTH 309 Y

LECTURE 17.6

THU 2014.04.03

Definition. Let V be a subspace of R and let w R. The

orthogonal projection of w onto V is the vector projV w such
that:
1) projV w V

2) the vector z = w projV w is orthogonal to V .

projV w

MTH 309 Y

LECTURE 17.7

The Best Approximation Theorem.

THU 2014.04.03

If V R is a subspace and w R, then projV w is the vector

of V which is the closest to w:
for all v V .

dist( w projV w ) 6 dist( w v )

MTH 309 Y

LECTURE 17.8

THU 2014.04.03

Upshot.

If A is an matrix and b R then a least square solution

of the equation
is a solution of the equation
where b = projCol(A) b.

Ax = b

Ax = b

b
projCol(A) b

Next. If V is a subspace of R and w R how can we compute

projV w?

MTH 309 Y

LECTURE 17.9

THU 2014.04.03

Theorem. If {w1 w } is an orthogonal basis of a subspace V

of R then for u R we have

u w2
u w
u w1
w1 +
w2 + +
w
projV u =
w1 w1
w2 w2
w w

Corollary. If {w1 w } is an orthonormal basis of a subspace

V of R then for u R we have
projV u = (u w1) w1 + (u w2) w2 + + (u w ) w

MTH 309 Y

Example.

B=

1
2
0
3

LECTURE 17.10

2
4

4 1

5 0

2
2

THU 2014.04.03

u=

1
2
2
1

B is an orthogonal basis of some subspace V R4. Find projV u.

MTH 309 Y

LECTURE 17.11

THU 2014.04.03

Note. In general, if V is a subspace of R and u R then to find

projV u we need to do the following:
1) find a basis of V ;
2) use Gram-Schmidt process to get an orthogonal basis of V ;
3) use the orthogonal basis to compute projV u.

Example. Consider the following matrix A and vector u:

0 0 1 1
0
A = 1 3 4 2 u = 3
2 6 3 1
0
Compute projV u where V = Col(A).

MTH 309 Y

LECTURE 17.12

THU 2014.04.03

Example. Find least square solutions of the matrix equation Ax = b

where

A=

1 1 0 0 0
1 1 1 1 0
0 0 0 0 1
1 0 1 1 0

1
0

90

72

b=

45
120