Homework 9

due Friday Nov 25, 10pm

1. This first problem

 is a preparation for determinants.
a b
Let A = .
c d
Let P be the parallelogram with a vertex at the origin and sides given by the columns
of A, as drawn on the left side of the figure:

c+d
d
` `
` ' `
c

b a a+b

       
0 a b a+b
Thus the four vertices of P lie at , , , and .
0 c d c+d
Use basic geometry to find the area of P.
Hint: One way to proceed is to first find the length of the horizontal line segments de-
noted ` in the figure. (They both have the same length.) Then notice that we can cut the
shaded triangle off the top of the parallelogram and re-attach it to the bottom, to get a more
standard parallelogram of the same area, as on the right side of the figure. The area of this
more standard parallelogram is (base)×(height) = ` × d. Thus area(P) = ` d.

Solution. We follow the hint. Consider the figure below.

 The (bottom) line segment 
of length

a b
` corresponds to the vector v1 − p. We know v1 = . We also know that v2 = .
c d

v2
`
p ` v1

The vector p is proportional to v2 and its second entry must be c (since it’s at the same
“height” as v1 ):    
∗ b
p= =α for some α.
c d

1
  
bc/d
From the second entry we find that α = c/d, and so p = . Now we find the length `
c
as
bc
!  
a− ` bc
v1 − p = d = 0 ⇒ `=a− .
0 d

Finally, following the hint, we get

area(P) = ` d = ad − bc .
 
a b
The crucial point is that ad − bc is the determinant of the matrix .
c d

A  b
1 x

1 1
x1
2. Consider the equation  −2 −1 = 0.

x2
3 2 0
a) Show that Ax = b has no solutions.

Sol. We go to row-echelon form.

   1 0 0   1 0 0  
1 1 1 2 1 0 1 1 1 0 1 0 1 1 1
−3 0 1 0 1 1 
(A | b) =  −2 −1 0  −→  0 1 2  −→ 0 1 2 
3 2 0 0 −1 −3 0 0 −1

We could keep reducing, but it’s already clear that there’s an unsolvable equation 0 = −1
coming from the bottom row.

b) The least-squares solution is the next best thing. What equation do we solve to find it?

Sol. We want to solve AT A x

b = AT b. This looks like
   
14 9 1
x
b= .
9 6 1

c) Find the least-squares solution x

b.
      
T −1 1 2 −3 1 −1
Sol. x
b = (A A) = = .
1 −3 14/3 1 5/3
d) Find the error b − A x
b.
     
1 1   2/3 1/3
−1
Sol. Here b
b = Ab x = −2 −1 b = −1/3.
= 1/3. The error is b − b
5/3
3 2 1/3 −1/3

2
3. A physicist is measuring the electric resistance r (in Ohms) of a wire as a function of
temperature t (in degrees Celsius). She finds that

t r
◦
−10 5
0◦ 5.2
20◦ 5.5

In other words, at temperatures t1 = −10, t2 = 0, t3 = 20 she finds resistances r1 = 5,

r2 = 5.2, r3 = 5.5. She guesses that the dependence of resistance on temperature should be
roughly linear
r(t) = c + dt
a) Set up the matrix problem that solves for the coefficients c, d given the data. It should
look like  
  5
c
A = 5.2
d
5.5
for an appropriate A.
   
1 −10   5
c
Sol. 1 0
  = 5.2

d
1 20 5.5
b) The matrix problem does not have an exact solution, only an approximate (least-
squares) solution. Find the least-squares solution for c and d. Write down the best-fit
function r(t) = c + dt.
 
  5  
T 3 10 T  15.7
Sol. Compute: A A = , and A 5.2 =
 . Thus
10 500 60
5.5
     
c T −1 15.7 5.179
= (A A) ≈ , r(t) = 5.179 + (0.016)t
d 60 0.016

c) What is the error between the best-fit function and the actual data, at each measured
value of t?

Sol. We can compute individual values r(−10) = 5.014, r(0) = 5.179, r(20) = 5.507, or all
at once  
  5.014
bb = A 5.179 = 5.179 .
0.016
5.507
     
5 5.014 −0.014
The vector of errors is 5.2 − 5.179 =  0.021 .
5.5 5.507 −0.006

3
4. You throw a ball into the air in order to measure the gravitational acceleration g. Re-
member that motion of a ball thrown upward takes the form
1
h(t) = d + vt − gt2 ,
2
where h(t) denotes the height as a function of time, d is the height the ball was thrown from,
v is the initial velocity, and g is the gravitational acceleration.
You’re not totally sure about the initial height d or the initial velocity v (when flinging
the ball into the air, it is hard to judge exactly how hard you throw it, or where it left your
hand). By looking at photos, you find that

t (seconds) h (meters)
.2 2.5
.4 3.1
.6 3.3
1 2.6
1.2 1.65

Find the best guesses for d, v, and (your real goal) g by using the following steps
a) Set up a matrix equation that captures this problem, in the form
 
  2.5
d  3.1 
 
A v =
 
 3.3 .

g  2.6 
1.65

Hint: write the fit function as h(t) = d · 1 + v · t + g · (− 12 t2 ). The first column of A

should consist of values of ‘1’ at the five measurement times (so the first column just contains
1’s); the second column consists of ‘t’ evaluated at the five measurement times; and the last
column consists of ‘− 21 t2 ’ evaluated at the five measurement times.

1 t1 − 21 t21
   
1 .2 −.02
1 t2 − 1 t22  1 .4 −.08
 2   
1 2
Sol. We want A =   1 t 3 − t
2 3
= 
 1 .6 −.18.

1 2
1 t4 − t4  1 1 −.5 
2
1 t5 − 12 t25 1 1.2 −.72
b) Find the least-squares solution for d, v, g.
 
  2.5  
d  3.1 
  1.5
T −1 T 
Sol. v = (A A) A  3.3  ≈ 5.6 . So the ball was released at a height of about
   
g  2.6  9.6
1.65
1.5 meters, with a velocity of 5.6m/sec, and the acceleration of gravity is about 9.6m/sec2 .

4
5. Suppose v1 , v2 , ..., vn are orthogonal and none of these vectors are zero. In order to
prove that they’re independent, we need to show that c1 v1 + c2 v2 + ... + cn vn = 0 implies
c1 = c2 = ... = cn = 0.
a) Compute (simplify as much as possible) (c1 v1 + c2 v2 + ... + cn vn ) · v1 .

Sol. (c1 v1 + c2 v2 + ... + cn vn ) · v1 = c1 ||v1 ||2 .

b) Compute (simplify as much as possible) (c1 v1 + c2 v2 + ... + cn vn ) · vi , for any i.

Sol. (c1 v1 + c2 v2 + ... + cn vn ) · vi = ci ||vi ||2 .

c) Try to use the result of (b) to argue that if c1 v1 + c2 v2 + ... + cn vn = 0 than every
coefficient ci must equal zero.

Sol. If c1 v1 + c2 v2 + ... + cn vn = 0 then (c1 v1 + c2 v2 + ... + cn vn ) · vi = 0 for each i. Using

(b), this implies that ci ||vi ||2 = 0 for each i. Since the norms ||vi ||2 are nonzero, this must
imply that ci = 0 for each i. Thus orthogonal vectors are independent.
   
1 3
6. Let v = and w = . These vectors span R2 , so any vector in R2 can be written
2 4  
5
as a linear combination of v and w. Let u = . Find c and d such that u = c v + d w.
 6
   
1 3 c 5
Hint: this is the same as solving = .
2 4 d 6
   −1    
c 1 3 5 −1
Sol. Let’s use the hint. Solving the matrix equation we get = = .
  d 2 4 6 2
5
And we can indeed check that (−1)v + (2)w = = u.
6

7. Suppose that v, w are orthogonal vectors and that u = c v + d w.

a) Compute u · v and use the result to find c.

Sol. u · v = c||v||2 , so c = (u · v)/||v||2 .

b) Compute u · w and use the result to find d. (So far, everything is a variable.)

Sol. u · w = d||w||2 , so d = (u · w)/||w||2 .

     
1 2 5
c) Let v = and w = . Note that v, w are orthogonal. Let u = . Find
2 −1 6
the coefficients c and d for which u = c v + d w.

Sol. c = (5 · 1 +6 ·2)/5 =17/5  · 2

 and d = (5 + 6· (−1))/5 = 4/5.
17 1 4 2 25/5 5
Let’s check: + = = = u.
5 2 5 −1 30/5 6

5
8. We can do the same thing in matrix form. Remember that if a matrix Q has orthogonal
columns then QT Q is diagonal. Part (c) of the previous problem amounts to solving

 Q    
1 2 c 5
= .
2 −1 d 6

Multiply both sides by QT to get a matrix equation involving a diagonal matrix QT Q. Then
solve to find c, d.
   
T c T 5
Sol. The equation Q Q =Q looks like
d 6
    
5 0 c 17
= .
0 5 d 4

Since QT Q is diagonal, this is easy to solve: c = 17/5, d = 4/5.

     
1 1 1
9. Let v1 =  1 , v2 = −1 , v3 = 1.
   
−1 0 2

vectors are orthogonal. Sol. v1 · v2 = v1 · v3 = v2 · v3 = 0.
a) Check that these three
b) Let Q = v1 v2 v3 . Compute the diagonal matrix QT Q.
Notice that the diagonal entries are precisely the norms ||v1 ||2 , ||v2 ||2 , ||v3 ||2 .
   
1 1 1 3 0 0
Sol. Q =  1 −1 1 and QT Q = 0 2 0.
−1 0 2 0 0 6
   
x1 b1
c) Solve Q x2  = b2  for any b1 , b2 , b3 by multiplying both sides by QT .
x4 b3
   
x1 b1
Sol. The equation QT Q x2  = QT b2  looks like
x4 b3
       1 
3 0 0 x1 b1 + b2 − b3 x1 3
(b 1 + b 2 − b 3 )
0 2 0 x2  =  b1 − b2  ⇒ x2  =  1 (b1 − b2 )  .
2
1
0 0 6 x4 b1 + b2 + 2b3 x4 (b
6 1
+ b2 + 2b3 )

10. Consider the equation Qx = b in the form

 Q   
1 1   b1
 1 −1 x1 = b2  .
x2
−1 0 b3

This is the sort of matrix equation that may not have solutions, and requires a least-
squares approximation.

6
 a) What condition must the entries of b satisfy in order for a solution to exist?

Sol. We go to row-echelon form

!
   1 0 0   1 0 0  
1 1 b1 −1 1 0 1 1 b1 0 1 0
0 1/2 1
1 1 b1
1 0 1 
(Q | b) =  1 −1 b2  −→ 0 −2 −b1 + b2  −→  0 −2 −b1 + b2 
1 1
−1 0 b3 0 1 b1 + b3 0 0 b + 2 b2 + b 3
2 1

The condition is b3 = − 12 (b1 + b2 ).

b) Compute the diagonal matrix QT Q.

 
T 3 0
Sol. Q Q = .
0 2
c) Multiply both sides of Qx = b by QT to get an equation QT Qx = QT b involving a
diagonal matrix.
   
3 0 b1 + b2 − b3
Sol. The equation looks like x= .
0 2 b1 − b2
 
2
T T
d) Solve Q Qx = Q b when b = 3. This produces the least-squares solution.

5
(Note: since QT is not invertible, QT Qx = QT b does not imply that Qx = b. As usual,
having a least-squares solution does not lead to an actual solution.)
       
3 0 0 x1 0
Sol. For this b, our least-squares equation looks like x= . So = .
0 2 −1 x2 − 12
  
2 1
1  1  
11. Let q1 = −1 and q2 = √ 2 . Find a third vector q3 such that
3 5 0
2


Q = q 1 q 2 q3
is an orthogonal matrix.
Hint: First find a vector v3 orthogonal to q1 and q2 . Then divide by its norm to get
q3 = v3 /||v3 ||, which has ||q3 || = 1.  
2 −1 2
Sol. To find v3 in a systematic way, we could solve for the nullspace of the matrix
1 2 0
(whose rows are proportional to q1 and q2 ). But we can also   do this by inspired guessing,
2
as follows. In order to be orthogonal to q2 , v3 can look like −1 for any a. Then in order
a
to be orthogonal to q1 , we must have 2 · 2 + (−1) · (−1) + 2 · a = 0, which implies a = −5/2.
     √4 
2 2
v3 2  3 5 
v3 =  −1  , q3 = = √  −1  = − 3√√2 5  . (Note −q3 works too.)
||v3 || 3 5 −5/2
−5/2 − 35

7
12. a) What is the maximum number of orthonormal vectors we can find in Rn ?
Since orthonormal vectors must be independent, there can be at most n of them.
b) True or false: if dim(V) = n, and v1 , ..., vn is a collection of orthogonal vectors in V
(and none of the vectors are zero), then V = hv1 , v2 , ..., vn i.
TRUE: the vectors must be independent; and any collection of n independent vectors in
V will span V.
c) True of false: If Q is a p × n matrix with orthonormal columns, then p ≥ n.
TRUE: p is the dimension of the vector space that the columns live in (in other words: the
columns are vectors in Rp ). Since the columns are orthonormal, they must be independent.
So p ≥ n.
d) True or false: if Q is a square matrix and it has orthonormal columns, then it has
orthonormal rows.
TRUE: In this case Q is an invertible matrix. Orthonormal columns means QT Q = I, so
Q−1 = QT , so QQT = I as well. That last equation means Q has orthonormal rows.
e) True or false: if Q is a rectangular matrix (meaning not a square matrix) with or-
thonormal columns, then it has orthonormal rows.
FALSE: if Q is rectangular, it is not invertible. So while QT Q = I, it is not true that
−1
Q = QT , and QQT need not equal I. (In fact, QQT cannot equal I because it can’t have
maximal rank.)
f) True or false: if Q is an orthogonal matrix, then Q−1 is an orthogonal matrix.
TRUE: If Q’s orthogonal then Q−1 = QT . We then check (Q−1 )T Q−1 = QQT = I and
Q−1 (Q−1 )T = QT Q = I.
g) True or false: if Q is an orthogonal matrix, then for any vector v, ||Qv|| = ||v||.
TRUE: ||Qv||2 = (Qv)T (Qv) = vT QT Qv = vT Iv = vT v = ||v||2 .

The Gram-Schmidt algorithm provides a systematic way to take a basis for some vector space
W and modify it to get an orthonormal basis for the same vector space. (In applications, W
will often be a subspace of some bigger space V.) Please see the book for a precise description
of the algorithm. The next problem illustrates the idea.

13. Strang Section 4.4 Problem 18 (both editions), reworded 

here.

1
1
Suppose V = R4 and W is the orthogonal complement to  1. A simple basis for W is


1
given by the three vectors
     
1 0 0
−1 1 0
a1 = 
 0  , a2 = −1 , a3 =  1  .
    

0 0 −1

However, this basis is not orthonormal. We get  an orthonormal

 basis q1 , q2 , q3 as follows.
1
−1
a) Define c1 = a1 . [nothing to do: c1 = 
 0 .]


8
 cT1 a2
b) Define (and compute) c2 = a2 − c1 . Check that c2 · c1 = 0.
cT1 c1
This is less scary than it seems! We are taking a2 and removing from it just the right
amount of c1 (i.e. removing some multiple of c1 ) to make it orthogonal to c1 .
     
0 1 1/2
1 −1 1/2
Sol. c2 = −1 − (−1/2)  0  =  −1  . Now c1 · c2 = 0.
    

0 0 0
T
c a3 cT a3
c) Define (and compute) c3 = a3 − 1T c1 − 2T c2 . Check that c3 ·c1 = 0 and c3 ·c2 = 0.
c c1 c2 c2
   1  
0 1/2 1/3
0 −1 1/2 1/3
Sol. c3 =  1  − (0)c1 − 3/2  −1  = 1/3 . Now c3 · c1 = 0 and c3 · c2 = 0.
    

−1 0 −1
d) We now have W = hc1 , c2 , c3 i, where the c’s are orthogonal vectors. As a final step,
c1 c2 c3
compute q1 = , q2 = , q3 = to get three orthonormal vectors that span W.
||c1 || ||c2 || ||c3 ||
     √     √ 
1 √ 1/2 1/√6 √ 1/3 1/(2√3)
1 −1  , q2 = √2 1/2 =  1/ 3
1/3 = 1/(2√3)
p 6  , q3 =
      
Sol. q1 = √ 
2 0  3  −1  − 2/3 2 1/3 1/(2 √ 3)

0 0 0 −1 − 3/2
 
1 0 2
2 1 −1
14. Let A =  0 1 −1. This matrix has independent columns.


1 3 1
a) Use Gram-Schmidt to find an orthonormal basis for C(A).
   
1 −5/6
T
. Then c2 = a2 − c1 a2 c1 = a2 − ( 5 )c1 = −2/3.
2  
Sol. First, c1 = 0 cT1 c1 6  1 
1 13/6
 
76
cT a3 cT a3 1 1 1 −54 .
And c3 = a3 − 1T c1 − 2T c2 = a3 − ( )c1 − ( )c2 =

c1 c1 c2 c2 6 41 41 −42
32
     
1 r −5/6 76
−2/3, q3 = √ 1
1 2 6   −54
Finally, q1 = √  , q 2 =  .
6 0
  41  1  2 41 · 70 −42
1 13/6 32

9
  
1
1
b) Find the point b
b in C(A) that is closest to b =  .
1
1
Sol. We have C(A) = hq1 , q2 , q3 i.

Let’s place the q’s as columns of a matrix Q = q1 q2 q3 .
The projection of b to C(A) is
    
27 2 −14 4 1 19
b = Q(QT Q)−1 QT b = QQT b = 1  2 69/2 7/2 −1 1 = 1 39 .
     
b
35 −14 7/2 21/2 7  1 35  7 
4 −1 7 33 1 43

10