*mnemonics

PDQ^t
= PAE DAE QI

: A = m x n = a real matrix

mxm
D = sqrt(eigenvalue) =norm of “Aq” mxn
nxn

P = orthogonal matrix
D = diagonal matrix
“Q” = orthogonal matrix

P= Aq1 /|Aq1| = extend from “r” to “m” thru GSP

: Its orthogonality is proved through inner product calculation.

: “r” = the number of positive eigenvalues = the number of non-zero eigenvalues
D = Norm of A*q = sqrt (eigenvalue of gram)
“Q” = q = eigenvector of gram = orthonormal (Principal Axes Theorem; gram=symmetric matrix)

gram = (A^t)(A)
Gram*q = (lambda)*q
===================================================
*Key

PD = A”Q”
PDQ^t = AQQ^t
PDQ^t = A

|Aq|^2 = (Aq)*(Aq) = (Aq)^t (Aq) =(q^t)(A^t)(A)(q) = (q^t)(lambda)(q) = lambda*(1)

Thus, |Aq| = sqrt(lambda)

The following is an element of “PD”

[Aq1 *(1/|Aq1|) ] * sqrt(lambda1)
= (A)(q1)

===================================================
With regard to “P”:

: the extension through GSP thereafter

: qi = an eigenvector of (A^t)(A)
-----------------------------------------------------------------------------
With regard to “Sigma”:

-----------------------------------------------------------------------------
With regard to Q^t:

: qi = an eigenvector of (A^t)(A)
: These orthonormalities are possible bcs A^tA is a symmetric matrix (per Principal Axes).
-----------------------------------------------------------------------------

MAIN PROOF

*singular values are either positive or zero. We pick only positive singular values to construct D.

: Aqn = (lambda n)(qn) = (0)(qn) = 0

: lambda’s greater than “r” are all zeros.
: This is established, because by definition, Q comprises the following.
: Q = <q1, q2, … , qn>

===================================================

With regard to “Ax”

: “x” comprises the necessary coefficients for the linear combination of the columns of “A.”

===================================================
We show the following in order.
1. consists of positive eigenvalues only.
2. The number of the positive eigenvalues = rank of “A”

===================================================

“lambda” = eigenvalue of (A^t)(A)

“q” = eigenvector of (A^t)(A)

(A^t)(A) q = (lambda) q

(1) Key: Use the Norm.

: According to the equation, the eigenvalue cannot help but be non-negative

: |q|^2 = 1, because it is orthonormal.

(2) (This does not seem to be important.)

: Originally, lambda is an eigenvalue of (A^t)(A) with respect to “q” as an eigenvector.

: Also, lambda is an eigenvalue of (A)(A^t) with respect to “Aq” as an eigenvector.
: (A)(A^t) has a lambda with respect to Aq and NOT “q” itself.

===================================================
Technique: Align the lambdas according to a particular order.
: There are “n” number of eigenvectors because A^tA (n x n) is orthogonally diagonalizable.

There can be eigenvalues that are zero. (Key notion)

Align the eigenvalues.

: “r” may equal “n” or be less.

===================================================
Fact: Rank of “A” = “r” (the number of non-zero eigenvalues of (A^t)(A) )

---------------------------------------------------------------
*Key

(1) Prove “im A = U” through and “im A > U” and “im A < U”
(2) Show that “U” is an orthogonal set.
---------------------------------------------------------------
“im A > U”
: This is obvious.
---------------------------------------------------------------
“im A < U”

Aq = (lambda)(q) = (0)q = 0
---------------------------------------------------------------
(2) Show that “U” is an orthogonal set.

(qi)(qj) = 0

---------------------------------------------------------------

Conclusion:
Rank A = dim (col A) = dim (im A) = r

===================================================