Linear Algebra and Geometry 3

Inner product spaces, quadratic forms, and more advanced problem solving

T
More new facts about A A:
six equivalent statements

Hania Uscka-Wehlou, Ph.D. (2009, Uppsala University: Mathematics)

University teacher in mathematics, Sweden
 y 4y = te + sin 2t ) y= e t + sin 2t + C 1 e + C 2 e
3 9 8
2 3
y 00 t t 1 2
4y = te + sin 2t ) y = e 3 t + 9 1
sin 2t +aC 2t
111 e a+
21C2 e
2t
a11 a12 a13 8
A= A = 4a12 a22 5
T
a21 a22 a23
a13 a23
2 3
 a11 a21 
Ta11 a12 a13 4a12 5 r 1 · r 1 r 1 · r 2
AA = · 2 a22 3 =
2 a 21 a 22 a
3 23 aa11 a r 2 · r 1 r 2 · r 2
a aa12 a 13 a23
21
r · r r · r
T a
AA = 11 11 21 13
· 4a12 a22 5 = 1 1 1 2
T 4aa21 aa22 5a23 r2 · r1 r2 · r2
A = 12 22 a13 a23
a13 a23    
x 2 3 x 2x + 3y
2 3 TA 2= = 3
a11 a21  y c1 · c41 1c1 · cy2 c1 · c3 4x + y
4 5 a a a
T
A A = a12 a22 · 11 12 13
= 4c2 · c1 c2 · c2 c2 · c3 5
2 a a3 a21 a22 a23 2c · c c · c c · c 3
a1113 a2123  c31 · c1 1 3c1 ·2c2 3c1 ·3 c3
a11 a12 a13
A A = 4a12 a22 5 ·
T
 =
 4 c ·
2 1c c 2 · c c
2 2 · c 3
5 
a21 a22 a23
  a 13 a
 23  TA 2t 2 c33 · c1 2t
c3 · c2 c34t· c+3 3t
x 2 3 x 2x + 3yt23 = = =t
4 1 t 8t + t
 Video 188
Least Squares ⊥
b − Ax̂ ∈ C

⊥ T
C = Null(A )

T
A (b − Ax)̂ = 0

T T
A Ax̂ = A b Video 194

T −1 T
The normal equation ? x̂ = (A A) A b
 7·6 22 7=6 22 22 7 Example: Compute the inverse of the matrix
Extra material: notes with the motivation
A:
for the for
algorithm (for 2-by-2anand 3-by-3
540 0 b33 05 4 0 0 a33 b33 0 5 88 Algorithm 2 inverse matrices,
3 example.
0 0 0 b44 0 0 0 a44 b44 88 Algorithm for inverse matrices, an example. 2 1 the inverse
Example: Compute 1 of the matrix
4
The invertible matrix theorem
Example: Compute the inverse of the matrix A: A = 4 0 6 4 5 .
generalised for product of any number of diagonal 2 0 23 2 A
Let A be a square n-by-n matrix over ℝ. The following statements are equivalent 2 1 1
(i.e., they are either all true or all false for any given matrix): A = 40 6 4 5.
ent n 2 N+ of a diagonal matrix A is the diagonal 89 Matrix inverse, Problem 1. 0 2 2
wers anii along the diagonal: 1. A is invertible, that is, A has an inverse, is nonsingular. 89 Matrix inverse, Problem 1.
2. There is an n-by-n matrix B such that Problem
AB = In = BA. 1: Determine if the matrix A is invertible and, if it is invertible, co
Problem 1: 2Determine if the matrix A is inve
3n 2 n 3 89 Matrix inverse, Problem 1. 3
0 0 0 a11 0 3. 0A is row-equivalent
0 to the n-by-n identity matrix In.
2 0 1
7 6 n 4. A has n pivot
7 positions (or: n leading 1’s Problem 1: Determine if the matrix A is invertible and, if it is invertible, compute i
after row reduction to RREF).
a22 0 07 6 0 a22 5. 0A has0full7 rank; that is, rank(A) = n. A 2= 4 0 1 3 1 5. A
=
0 a33 0 5 40 n
0 6.aThe 0 5 A can be expressed as a finite product of elementary matrices.
33 matrix
2 0
3 0 1
1
0 0 a44 0 an44
0 7. 0The equation Ax = 0 has only the trivial solution x = 0. A = 4 0 1 1 5.
n
Solve the system of
8. The equation Ax = b has exactly one solution for some b in ℝ . linear equations
Video 89 Solve
Ax = the
b system
for 0 of 1RHS
3 every linear b
equations
= (b , Ax
b
1 2 3 , b=)bT
n
eating Rule 2. 9. The equation Ax = b has exactly one solution for each b in ℝ .
Solve the system of linear equations AxExtra
Extra material: notes with solved Problem 1. material:
= b for notesbwith
every RHS = (bsolved
, b , b Problem
1 )T .
2 3
1.
10. detA ≠ 0. Extra material: notes with solved Problem 90 Matrix1. inverse, Problem 2.
sponding to eigenvalue are non-trivial 90 Matrix inverse, Problem 2.
Problem 2: Determine all the values of c for w
90 Matrix inverse, Problem 2.
⇥ n system of equations Problemdeterminant
11. The columns of A are linearly independent. 2: Determine all the
testallfor values of c for which the following
ℝ n matrix is in
12. The columns of A span ℝn. Problem 2: Determine thenvalues
vectorsof cgenerating
for which 2 thethe entire
following
3 matrix is invertible
13. Col(A) = ℝn. 2 c c3 c
n c4 c c
14. The columns of A form a basis of ℝ . 1 c 1 5 .
4 5
(A I)v = 0. 15. The transpose AT is an invertible matrix (hence rows of A are linearly independent, spanExtra 1 c 1 .
ℝn, material:
00 0notes cwith solved Problem 2
and form a basis of ℝ ). n 0 c
must have infinitely 16. A is column-equivalent to the n-by-n Extra 91 Matrix equations, Problem 3.
many solutions
identity material:
matrix I n .
Extra material: notes notes n
with
withn
solved Problem
solved Problem 2. 2.
17. The linear transformation mapping x to Ax is a bijection from ℝ to ℝ .
if λ is an eigenvalue 18. The kernel of A is trivial, that is,91 91 Matrix
Matrix
it contains only equations,
equations,
the null vector Problem
Problem 3.
3. ker(A) =Problem
as an element, null(A) = {0}
{0}. 3: Find the matrix A that solves the
  T 
19. The number 0 is not an eigenvalue of A. 2 with
1 solved T 2 7 3.

Problem Extra material: notes 2 1 Problem
det( I A) = det(A I) 20. The matrix A has a left inverse (that Problem
is, there 3: a3:Find
exists BFind the matrix
suchthe
that matrix
BA = I) orAathat
A right solves
that inverse
solvesthethe
equation
equation 5 3
= A +=AA + A .
4 8
(that is, there exists a C such that AC = I), in which case both left and right inverses exist 92 and B = Cequations,
Matrix =A .−1 5 4.
Problem 3
e, the system of equations (A I)v = 0 must have Extra material: notes with solved Problem 3.
Extra material: notes with solved Problem 3.
the trivial one (v = 0). 92 Matrix equations, Problem 4. Problem 4: Find the matrix A that solves the
Theorem The following are equivalent for A 2 Rm⇥n :
1. rankA = n.
2. The rows of A span Rn .
3. The columns of A are linearly independent in Rm .
4. The n ⇥ n matrix AT A is invertible.
5. CA = In for some n ⇥ m matrix C.
6. If Ax = 0 for some x 2 Rn then x = 0.

Theorem The following are equivalent for A 2 Rm⇥n :

1. rankA = n.
2. The rows of A span Rn .
3. The columns of A are linearly independent in Rm .
4. The n ⇥ n matrix AT A is invertible.
5. CA = In for some n ⇥ m matrix C.
6. If Ax = 0 for some x 2 Rn then x = 0.

Problem 4: Find the largest and the smallest value of the quadratic
 Video 188
Least Squares ⊥
b − Ax̂ ∈ C

⊥ T
C = Null(A )

T
A (b − Ax)̂ = 0

T T
A Ax̂ = A b Video 194

T −1 T
The normal equation ? x̂ = (A A) A b
If the columns of A
are linearly independent
 Video 188
Least Squares ⊥
b − Ax̂ ∈ C

⊥ T
C = Null(A )

T
A (b − Ax)̂ = 0

T T
A Ax̂ = A b Video 194

T −1 T
The normal equation ? x̂ = (A A) A b
If matrix A If the columns of A
has full column rank are linearly independent
 m×n
Video 90 Parts 1 and 2 of the Theorem:
dim r dim r
Part 2 About 4 fundamental matrix spaces
and their dimensions
Col(A)
T
Col(A ) all Ax
Row(A)
T
all A y
n m
ℝ ℝ

such y that
Null(A) such x that ATy = 0
Ax = 0 T
Null(A )

dim n − r dim m − r
Theorem The following are equivalent for A 2 Rm⇥n :
1. rankA = n. n⩽m
2. The rows of A span Rn .
3. The columns of A are linearly independent in Rm .
4. The n ⇥ n matrix AT A is invertible.
5. CA = In for some n ⇥ m matrix C.
6. If Ax = 0 for some x 2 Rn then x = 0.

Theorem The following are equivalent for A 2 Rm⇥n :

1. rankA = n.
2. The rows of A span Rn .
3. The columns of A are linearly independent in Rm .
4. The n ⇥ n matrix AT A is invertible.
5. CA = In for some n ⇥ m matrix C.
6. If Ax = 0 for some x 2 Rn then x = 0.

Problem 4: Find the largest and the smallest value of the quadratic
Theorem The following are equivalent for A 2 Rm⇥n :

T
Similar theorem for AA
1. rankA = m.
2. The columns of A span Rm .
3. The rows of A are linearly independent in Rn .
4. The m ⇥ m matrix AAT is invertible.
5. AC = Im for some n ⇥ m matrix C.
6. The system Ax = b is consistent for every b 2 Rm .

Problem 4: Find the largest and the smallest value of the quadratic
form q(x1 , x2 , x3 ) for arguments on the sphere x21 + x22 + x23 = 1.
Here q(x1 , x2 , x3 ) = 5x21 + 14x1 x2 + 4x1 x3 + 5x22 4x2 x3 4x23 .

Problem 3: Find the largest and the smallest value of the quadratic
form q(x, y, z) = x2 + y 2 4xy + z 2 for arguments on the unit
sphere x2 + y 2 + z 2 = 1.