1.

“Best Approximation Theorem”

2. “Least Squares Method”
3. “Least Squares Approximating Polynomials”

------------------------------
1. “Best Approximation Theorem”

We are discussing a particular closest solution to a homogeneous system.

Ax = b A=mxn x=nx1 b=mx1

(n x m) (m x n) (n x 1) = (n x m) (m x 1) = (n x 1)
: Az = b (“z” as the closest to “x” even if there may be no such “x”)
: AtA is invertible if the columns of A are independent (See theorem 5.4.3 below).

*Mnemonics
: “Ah tee Ah zee” = Ah tee Bee
------------------------------
: (A^t)*(Az) = (A^t)*b
: (A^t)(A) is a square matrix
: (m x n) x (n x m) = m x m

= distance between the tips of “Az” and “b”

: “z” is such that |b-Az| becomes minimal.
: “b-Az” is orthogonal to every vector “Ax” as well as “Az” in U”.
: “x” is a non-zero vector.

Accordingly:

----------------------

: This has no solution.

A=3x2 z=2x1
================================================
2. “Least Squares Method”

We are discussing a particular line through the coordinates.

The term “least squares” comes from the following:

: This should be minimized.

: This is a least squares approximating line.

: matrix “M” is to be multiplied with the vector “z”.

Mz = y

Ax=b

------------
================================================
3. “Least Squares Approximating Polynomials”

We are discussing a particular approximate polynomial (rather than a line) for the coordinates.
M = (m+1) x n, n = m+1 or greater
(M^t)M = n x n

With regard to 2:
: (Mt)(M) is invertible if the columns of “M” are independent (see Theorem 5.4.3 below)

Define:

: Assume that the polynomial applies to a particular entry such as “x1.”

: Upon the plugging in of “x1”…and ”xn”, the following holds based on the above assumption.

“n” = “m+1’ or greater

: However, q(x) is a polynomial of degree “m.”
: “the number of roots > the degree” => “it is a zero polynomial”
: Hence, the columns are independent.