Level-up!

Linear Algebra 1
Morris Alper

Morris Alper – ITC Level-up December 2020 cohort

Agenda

● Linear Equations
● Systems of Linear Equations
● Vectors
● Matrices
● Row Reduction
● Vectors spans and bases
● Gram-Schmidt
● Further Reading
Linear Equations
Linear Equations

Recall that we can represent a line in the 𝑥, 𝑦 plane with an

equation in the form 𝑦 = 𝑚𝑥 + 𝑏, where 𝑚 is the line’s slope and
𝑏 its y-intercept.
Linear Equations

Recall that we can represent a line in the 𝑥, 𝑦 plane with an

equation in the form 𝑦 = 𝑚𝑥 + 𝑏, where 𝑚 is the line’s slope and
𝑏 its y-intercept.

𝑦 = 2𝑥 − 1
Linear Equations

Recall that we can represent a line in the 𝑥, 𝑦 plane with an

equation in the form 𝑦 = 𝑚𝑥 + 𝑏, where 𝑚 is the line’s slope and
𝑏 its y-intercept.

𝑦 = 4𝑥 − 1
Linear Equations

Recall that we can represent a line in the 𝑥, 𝑦 plane with an

equation in the form 𝑦 = 𝑚𝑥 + 𝑏, where 𝑚 is the line’s slope and
𝑏 its y-intercept.

𝑦 = 4𝑥 − 4
Linear Equations

Equivalently, we can move all variables (𝑥, 𝑦) to the left-hand-

side:

𝑦 = 2𝑥 − 1 ⇒ −𝟐𝒙 + 𝒚 = 𝟏

𝑦 = 4𝑥 − 1 ⇒ −𝟒𝒙 + 𝒚 = −𝟏

𝑦 = 4𝑥 − 4 ⇒ −𝟒𝒙 + 𝒚 = −𝟒
Linear Equations

In general, a line in (𝑥, 𝑦) two-dimensional space can be written

as:

𝑎1 𝑥 + 𝑎2 𝑦 = 𝑏

for constants 𝑎1 , 𝑎2 , 𝑏,
Linear Equations

In general, a plane in (𝑥, 𝑦, 𝑧) three-dimensional space can be

written as:

𝑎1 𝑥 + 𝑎2 𝑦 + 𝑎3 𝑧 = 𝑏

for constants 𝑎1 , 𝑎2 , 𝑎3 , 𝑏,
Linear Equations

𝑥 + 𝑦 − 3𝑧 = −1
Linear Equations

2𝑥 + 𝑦 + 𝑧 = 3
Linear Equations

A linear equation in n variables is an equation

𝑎1 𝑥1 + 𝑎2 𝑥2 + ⋯ + 𝑎𝑛 𝑥𝑛 = 𝑏

for constants 𝑎1 , 𝑎2 , … , 𝑎𝑛 , 𝑏.

Geometrically this represents a line (n=2), plane (n=3), or

hyperplane (n>3).
Linear algebra studies these equations and their properties.
Systems of Linear Equations
Systems of Linear Equations

Consider the two linear equations:

3𝑥 − 2𝑦 = −2
𝑥+𝑦 =6
Systems of Linear Equations

Consider the two linear equations:

3𝑥 − 2𝑦 = −2
𝑥+𝑦 =6
Systems of Linear Equations

Consider the two linear equations:

3𝑥 − 2𝑦 = −2
𝑥+𝑦 =6

Geometrically this represents two

lines which intersect at 𝑥, 𝑦 = (2, 4).
Systems of Linear Equations

In general, a system of linear equations is a set of 𝑚 linear

equations in 𝑛 variables:
𝑎1,1 𝑥1 + 𝑎1,2 𝑥2 + ⋯ + 𝑎1,𝑛 𝑥𝑛 = 𝑏1
𝑎2,1 𝑥1 + 𝑎2,2 𝑥2 + ⋯ + 𝑎2,𝑛 𝑥𝑛 = 𝑏2
⋯
𝑎𝑚,1 𝑥1 + 𝑎𝑚,2 𝑥2 + ⋯ + 𝑎𝑚,𝑛 𝑥𝑛 = 𝑏𝑚

A solution is a set of values 𝑥1 , 𝑥2 , … , 𝑥𝑛 = (𝑐1 , 𝑐2 , … , 𝑐𝑛 ) that

satisfy the equations.
Systems of Linear Equations

A system of linear equations can have:

One solution
Systems of Linear Equations

A system of linear equations can have:

No solutions
Systems of Linear Equations

A system of linear equations can have:

Infinite solutions
Vectors
Vectors

A vector is a tuple (pair/triplet/…) of numbers such as:

(1, 2)
(5, 6, 7)
0.4, −0.1, 2.3,4.0

We can think of these either as points or as arrows in n-

dimensional space.
Vectors

Example: The vector 𝒗 = (1, 2) can be graphed in the (x,y)-plane

as

or
Vectors

Notation:

The set of vectors with real-valued coordinates in n-dimensional space is

written as ℝ𝑛 .

We normally write vectors as bold lowercase letters*, so for example 𝒗 ∈ ℝ2

means that 𝒗 is a vector containing two real numbers, i.e. 𝒗 = 𝑣1 , 𝑣2 .

*You may also see vectors written with a superscript arrow, e.g. 𝑣.
Ԧ
Vectors

Vector addition, subtraction, and scalar multiplication are defined

elementwise.

For example, for 𝒗 = (𝑣1 , 𝑣2 ), 𝒘 = 𝑤1 , 𝑤2 , and scalar 𝑐:

● 𝒗 + 𝒘 = (𝑣1 + 𝑤1 , 𝑣2 + 𝑤2 )
● 𝒗 − 𝒘 = (𝑣1 − 𝑤1 , 𝑣2 − 𝑤2 )
● 𝑐𝒗 = (𝑐𝑣1 , 𝑐𝑣2 )
Vectors

Example:

● 𝒗 = 1, 2
● 𝒘 = 1, 0.5
● 𝒗 + 𝒘 = 1 + 1, 2 + 0.5 = (2, 2.5)
Vectors

Example:

● 𝒗 = 1, 2
● 𝒘 = 1, 0.5
● 𝒗 − 𝒘 = 1 − 1, 2 − 0.5 = (0, 1.5)
Vectors

Example:

● 𝒘 = 1, 0.5
● 𝟐𝒘 = 2 ⋅ 1, 2 ⋅ 0.5 = (2, 1)
Vectors

One property of vectors is their Euclidean norm*, which represents their

length in Euclidean space. In general, the norm of a vector 𝒗 ∈ ℝ𝑛 is defined
as

𝒗 = 𝑣12 + 𝑣22 + ⋯ + 𝑣𝑛2

Example: For 𝒗 = (3, 4), 𝒗 = 32 + 42 = 5

*Also known as the L2 norm. For more information see Wikipedia.

Vectors

A vector of norm one is called a unit vector.

We can turn any vector into a unit vector by dividing it by its norm. We
usually use a superscript hat to indicate a unit vector, as shown below:
1
ෝ=
𝒗 𝒗
𝒗
1 1
Then 𝒗
ෝ = 𝒗 = 𝒗 = 1.
𝒗 𝒗
Vectors

Example:

𝒗 = 3, 4

1 3 4
ෝ=
𝒗 3, 4 = ( , ) is a unit vector
5 5 5
pointing in the same direction as 𝒗.
Vectors

The dot product is an operation which takes two vectors as inputs and
returns a scalar value as output.

Definition: For 𝒗, 𝒘 ∈ ℝ𝑛 ,
𝑛

𝒗 ⋅ 𝒘 = 𝑣1 𝑤1 + 𝑣2 𝑤2 + ⋯ + 𝑣𝑛 𝑤𝑛 = ෍ 𝑣𝑖 𝑤𝑖
𝑖=1

Example:
1, 3, 4 ⋅ −1, 2, 0 = 1 ⋅ −1 + 3 ⋅ 2 + 4 ⋅ 0 = 5
Vectors

Properties of the dot product:

● 𝒗⋅𝒘=𝒘⋅𝒗 (commutativity)

● 𝑐𝒗 ⋅ 𝒘 = 𝒗 ⋅ 𝑐𝒘 = 𝑐(𝒗 ⋅ 𝒘) (linearity)

● 𝒖+𝒗 ⋅𝒘= 𝒖⋅𝒘+𝒗⋅𝒘 (linearity)

● 𝒗⋅𝒗= 𝒗 2

● 𝒗⋅𝒘 ≤ 𝒗 𝒘 (Cauchy-Schwartz inequality)

Vectors

It can be shown that 𝒗 ⋅ 𝒘 = 𝒗 𝒘 cos 𝜃, where 𝜃 is the angle between the

two vectors.

Example:
𝒗 = 1, 1 , 𝒘 = (0, 2)
𝒗⋅𝒘=1⋅0+1⋅2=2 𝜋
𝜃= (45° )
𝒗 = 2 𝒘 =2 4

𝜋
𝜃= (45° )
4
2
𝒗 𝒘 cos 𝜃 = 2 ⋅ 2 ⋅ =2
2
Vectors

For 𝒗, 𝒘 ∈ ℝ𝑛 , we say that 𝒗 and 𝒘 are orthogonal if 𝒗 ⋅ 𝒘 = 0.

𝜋
This is equivalent to saying that there is a right angle ( radians, 90° degrees)
4
between 𝒗 and 𝒘.

Example:
𝒗 = 1, 1 , 𝒘 = (1, −1)
𝒗 ⋅ 𝒘 = 1 ⋅ 1 + 1 ⋅ −1 = 0

So 𝒗 and 𝒘 are orthogonal.

Vectors

The projection of a vector 𝒗 ∈ ℝ𝑛 onto a vector 𝒘 ∈ ℝ𝑛 is defined by:

𝒗⋅𝒘
𝑝𝑟𝑜𝑗𝒘 𝒗 = 2
𝒘
𝒘
Note that if 𝒘
ෝ is a unit vector then this simplifies to 𝑝𝑟𝑜𝑗𝒘ෝ 𝒗 = (𝒗 ⋅ 𝒘)
ෝ 𝒘.
ෝ
Matrices
Matrices

Recall: a system of linear equations is a set of 𝑚 linear

equations in 𝑛 variables:
𝑎1,1 𝑥1 + 𝑎1,2 𝑥2 + ⋯ + 𝑎1,𝑛 𝑥𝑛 = 𝑏1
𝑎2,1 𝑥1 + 𝑎2,2 𝑥2 + ⋯ + 𝑎2,𝑛 𝑥𝑛 = 𝑏2
⋯
𝑎𝑚,1 𝑥1 + 𝑎𝑚,2 𝑥2 + ⋯ + 𝑎𝑚,𝑛 𝑥𝑛 = 𝑏𝑚

A solution is a set of values 𝑥1 , 𝑥2 , … , 𝑥𝑛 = (𝑐1 , 𝑐2 , … , 𝑐𝑛 ) that

satisfy the equations.
Matrices

We can write the coefficients of the linear equations as a matrix:

𝑎1,1 ⋯ 𝑎1,𝑛
𝐴= ⋮ ⋱ ⋮
𝑎𝑚,1 ⋯ 𝑎𝑚,𝑛

The matrix 𝐴 is a rectangular array of numbers. 𝐴 has m rows

and n columns, so we say that 𝐴 is a m × n (“m by n”) matrix.
Matrices

Example 1: For the system of equations

𝑥 + 2𝑦 + 5𝑧 = 1
1
2𝑥 − 𝑦 = 3
2
1 2 5
The coefficient matrix is 𝐴 = 1 , which is a 2 × 3 matrix.
2 − 0
2
Matrices

Example 2: For the system of equations

𝑥1 + 2𝑥2 − 2𝑥3 + 3𝑥4 = 4
−𝑥1 + 2𝑥2 − 𝑥3 + 7𝑥4 = 3
3𝑥1 + 3𝑥2 + 𝑥4 = 2
−𝑥1 − 𝑥2 + 5𝑥3 + 5𝑥4 = 1

1 2 −2 3
The coefficient matrix is the 4 × 4 matrix 𝐴 = −1 2 −1 7 .
3 3 0 1
−1 −1 5 5
Matrices

We can also write vectors as matrices, in two formats:

1 × n matrices (row vectors)

𝒗 = 4 3 −1

n × 1 matrices (column vectors)

4
𝒗= 3
−1
Matrices

We often write the variables and values of a system of linear

equations as column vectors.

Example 1:

1 2 5 𝑥
1
𝑥 + 2𝑦 + 5𝑧 = 1 𝐴 = 2 −1 0 𝒗= 𝑦 𝒃=
2 𝑧 3
1
2𝑥 − 𝑦 = 3
2
Matrices

Example 2: 1 2 −2 3
𝑥1 + 2𝑥2 − 2𝑥3 + 3𝑥4 = 4 𝐴 = −1 2 −1 7
3 3 0 1
−𝑥1 + 2𝑥2 − 𝑥3 + 7𝑥4 = 3
−1 −1 5 5
3𝑥1 + 3𝑥2 + 𝑥4 = 2
−𝑥1 − 𝑥2 + 5𝑥3 + 5𝑥4 = 1 𝑥1 4
𝑥2
𝒗= 𝑥 𝒃= 3
3 2
𝑥4 1
Matrices

The transpose 𝑀𝑇 of a matrix 𝑀 is defined by flipping its rows

and columns as shown below:

4
𝑇 = 3
4 3 −1
−1
𝑇
1 2 −2 3 1 −1 3 −1
−1 2 −1 7 = 2 2 3 −1
3 3 0 1 −2 −1 0 5
−1 −1 5 5 3 7 1 5
Row Reduction
Row Reduction

Consider the system of linear equations

3𝑥 + 11𝑦 + 5𝑧 = 35

2𝑥 + 2𝑦 − 2𝑧 = 2

𝑥 + 3𝑦 + 𝑧 = 9

Q: How can we efficiently find a solution for 𝑥, 𝑦, 𝑧 ?

Row Reduction

Consider how we can “change” the equations while keeping the solutions the
same.

Operation 1: Change the order of the equations

3𝑥 + 11𝑦 + 5𝑧 = 35 2𝑥 + 2𝑦 − 2𝑧 = 2

2𝑥 + 2𝑦 − 2𝑧 = 2 ⇒ 3𝑥 + 11𝑦 + 5𝑧 = 35
𝑥 + 3𝑦 + 𝑧 = 9 𝑥 + 3𝑦 + 𝑧 = 9
Row Reduction

Consider how we can “change” the equations while keeping the solutions the
same.

Operation 2: Multiply/divide both sides of one equation by a constant

3𝑥 + 11𝑦 + 5𝑧 = 35 3𝑥 + 11𝑦 + 5𝑧 = 35

𝟐𝒙 + 𝟐𝒚 − 𝟐𝒛 = 𝟐 ⇒ 𝒙+𝒚−𝒛=𝟏
𝑥 + 3𝑦 + 𝑧 = 9 𝑥 + 3𝑦 + 𝑧 = 9
Row Reduction

Consider how we can “change” the equations while keeping the solutions the
same.

Operation 3: Add/subtract a multiple of one equation to another

𝟑𝒙 + 𝟏𝟏𝒚 + 𝟓𝒛 = 𝟑𝟓 𝟐𝒚 + 𝟐𝒛 = 𝟖 (−3𝑒𝑞3 )

2𝑥 + 2𝑦 − 2𝑧 = 2 ⇒ 2𝑥 + 2𝑦 − 2𝑧 = 2
𝒙 + 𝟑𝒚 + 𝒛 = 𝟗 𝑥 + 3𝑦 + 𝑧 = 9
Row Reduction

We can represent this succinctly using augmented matrix notation:

3𝑥 + 11𝑦 + 5𝑧 = 35
3 11 5 | 35
2𝑥 + 2𝑦 − 2𝑧 = 2 ⇒ 2 2 −2 | 2
1 3 1 | 9
𝑥 + 3𝑦 + 𝑧 = 9

The three operations we listed are called elementary row operations on the
augmented matrix.
Row Reduction

Elementary Row Operations

1. Exchange rows
2. Multiply/divide row by a (nonzero) constant
3. Add/subtract multiple of one row from another
Row Reduction

Elementary Row Operations

1. Exchange rows

3 11 5 | 35 2 2 −2 | 2
2 2 −2 | 2 ⇒ 3 11 5 | 35
1 3 1 | 9 1 3 1 | 9
Row Reduction

Elementary Row Operations

2. Multiply/divide row by a (nonzero) constant

3 11 5 | 35 3 11 5 | 35
𝟐 𝟐 −𝟐 | 𝟐 ⇒ 𝟏 𝟏 −𝟏 | 𝟏
1 3 1 | 9 1 3 1 | 9
Row Reduction

Elementary Row Operations

3. Add/subtract multiple of one row from another

𝟑 𝟏𝟏 𝟓 | 𝟑𝟓 𝟎 𝟐 𝟐 |𝟖
2 2 −2 | 2 ⇒ 2 2 −2 |2
𝟏 𝟑 𝟏 | 𝟗 1 3 1 |9

(𝑟𝑜𝑤1 ⇒ 𝑟𝑜𝑤1 − 3𝑟𝑜𝑤3 )

Row Reduction

We can use elementary row operations to simplify the augmented matrix, converting it
into row-echelon form: the first nonzero coefficient in each row is 1, with 0’s
underneath it, and any rows of all zeros at the bottom of the matrix.

3 11 5 | 35 1 3 1 |9 1 3 1 | 9 1 3 1 | 9
2 2 −2 | 2 ⇒ 2 2 −2 |2 ⇒ 0 −4 −4 | − 16 ⇒ 0 1 1 | 4
1 3 1 | 9 3 11 5 | 35 3 11 5 | 35 3 11 5 | 35

1 3 1 | 9 1 3 1 | 9 1 3 1 | 9
⇒ 0 1 1 | 4 ⇒ 0 1 1 | 4 ⇒ 0 1 1 | 4
0 2 2 |8 0 1 1 |4 0 0 0 | 0

Q: Which elementary row operations were used at each step?

 Row Reduction

We can simplify further with elementary row operations to put the augmented matrix in
reduced row-echelon form, where each leading coefficient is 1 and all other elements
in that column are 0
1 3 1 | 9 1 0 −2 | − 3
0 1 1 | 4 ⇒ 0 1 1 | 4
0 0 0 | 0 0 0 0 | 0

So in this case the system of linear equations is equivalent to the system

𝑥 − 2𝑧 = −3
𝑦+𝑧 =4

So the solutions are all vectors of the form 𝑥, 𝑦, 𝑧 = (2𝑧 − 3, 4 − 𝑧, 𝑧).

Row Reduction

Row Reduction (a.k.a. Gauss-Jordan elimination):

Solving a system of linear equations by using elementary row

operations to convert the corresponding augmented matrix to
reduced row-echelon form.
Row Reduction

Example:
2𝑥 + 𝑦 − 𝑧 = 8
−3𝑥 − 𝑦 + 2𝑧 = −11
−2𝑥 + 𝑦 + 2𝑧 = −3

This system of equations has augmented matrix

2 1 −1 | 8
−3 −1 2 | − 11
−2 1 2 | −3
Row Reduction

Example:

Using row reduction:

2 1 −1 | 8 2 1 −1 | 8 2 1 −1 | 8
−3 −1 2 | − 11 ⇒ 0 1/2 1/2 | 1 ⇒ 0 1/2 1/2 | 1
−2 1 2 | −3 −2 1 2 |−3 0 2 1 |5
2 1 −1 | 8 2 1 −1 |8 2 1 −1 | 8
⇒ 0 1 1 |2 ⇒ 0 1 1 |2 ⇒ 0 1 1 | 2
0 2 1 |5 0 0 −1 |1 0 0 1 |−1
Row Reduction

Example:

Using row reduction:

2 1 −1 | 8 2 1 −1 | 8 2 1 −1 | 8
−3 −1 2 | − 11 ⇒ ⋯ ⇒ 0 1 1 | 2 ⇒ 0 1 0 | 3
−2 1 2 | −3 0 0 1 |−1 0 0 1 |−1
2 1 0 | 7 2 0 0 | 4 1 0 0 | 2
⇒ 0 1 0 | 3 ⇒ 0 1 0 | 3 ⇒ 0 1 0 | 3
0 0 1 |−1 0 0 1 |−1 0 0 1 |−1
Row Reduction

Example:

Using row reduction:

2 1 −1 | 8 1 0 0 | 2
−3 −1 2 | − 11 ⇒ ⋯ ⇒ 0 1 0 | 3
−2 1 2 | −3 0 0 1 |−1

Therefore the unique solution is 𝑥, 𝑦, 𝑧 = (2, 3, −1).

Vector Spans and Bases
Vector Spans and Bases

The span of a set of vectors is defined as the set of all linear combinations
of the vectors.

𝑠𝑝𝑎𝑛 𝒗 𝟏 , 𝒗(𝟐) , … , 𝒗(𝒌) = 𝑐1 𝒗 𝟏 + 𝑐2 𝒗 𝟐 + ⋯ , 𝑐𝑘 𝒗 𝒌 : 𝑐1 , … , 𝑐𝑘 ∈ ℝ

Examples:
3 𝑥
𝑠𝑝𝑎𝑛 ={ ∶ 𝑥 ∈ ℝ}
0 0

5 −1 1 𝑥
𝑠𝑝𝑎𝑛 0 , 0 , 2 = { 𝑦 ∶ 𝑥, 𝑦 ∈ ℝ}
0 0 0 0
Vector Spans and Bases

The span is invariant under:

1. Changing order of vectors

2. Multiplying/dividing vector by a scalar
3. Adding/subtracting a multiple of one vector from another
Example:
5 −1 1 −1 5 1
𝑠𝑝𝑎𝑛 0 , 0 , 2 = 𝑠𝑝𝑎𝑛 0 , 0 , 2
0 0 0 0 0 0
Vector Spans and Bases

The span is invariant under:

1. Changing order of vectors

2. Multiplying/dividing vector by a scalar
3. Adding/subtracting a multiple of one vector from another
Example:
5 −1 1 1 −1 1
𝑠𝑝𝑎𝑛 0 , 0 , 2 = 𝑠𝑝𝑎𝑛 0 , 0 , 2
0 0 0 0 0 0
Vector Spans and Bases

The span is invariant under:

1. Changing order of vectors

2. Multiplying/dividing vector by a scalar
3. Adding/subtracting a multiple of one vector from another
Example:
5 −1 1 5 −1 0
𝑠𝑝𝑎𝑛 0 , 0 , 2 = 𝑠𝑝𝑎𝑛 0 , 0 , 2
0 0 0 0 0 0
Vector Spans and Bases

Therefore we can use row reduction with the vectors as rows to simplify the
span:

Example:

3 2 1 3 11 5 1 0 −2
𝑠𝑝𝑎𝑛 11 , 2 , 3 2 2 −2 ⇒ 0 1 1
5 −2 1 1 3 1 0 0 0

1 0
= 𝑠𝑝𝑎𝑛 0 , 1
−2 1
Vector Spans and Bases

A basis is a set of vectors of minimal size that spans a vector space.

The dimension of a vector space is the number of vectors in its bases.

Examples:

1 0 2 −3
{ , } and { , } are both bases for ℝ2 (2-dimensional).
0 1 1 0

1 0 3 2 1
{ 0 , 1 } is a basis for 𝑠𝑝𝑎𝑛 11 , 2 , 3 (2-dimensional).
−2 1 5 −2 1
Vector Spans and Bases

Therefore we can use row reduction with the vectors as rows to find a basis
for the span:

Example 1:

5 −1 1 5 0 0 1 0 0
𝑠𝑝𝑎𝑛 0 , 0 , 2 −1 0 0 ⇒ 0 1 0
0 0 0 1 2 0 0 0 0

1 0
= 𝑠𝑝𝑎𝑛 0 , 1
0 0
Vector Spans and Bases

Any vector in a vector space can be expressed uniquely as linear combination of

basis vectors. The coefficients are its coordinates in the basis.

Example:

1 0 2 −3
{ , } and { , } are both bases for ℝ2
0 1 1 0

11 1 0
The vector 𝒗 = is equal to 11 + 1 , so it has coordinates (11, 1) in the
1 0 1
1 0
basis { , }.
0 1

2 −3 2 −3
Also, 𝒗 = 1 −3 , so it has coordinates (1, −3) in the basis { , }.
1 0 1 0
 Vector Spans and Bases

An orthonormal basis is a basis of unit vectors which are all orthogonal to each other.

Example 1:
1 0
𝒗= 𝒘=
0 1
{𝒗, 𝒘} is an orthonormal basis for ℝ2 because they are unit vectors and orthogonal to
each other:
𝒗 = 𝒘 =1
𝒗⋅𝒘=0
 Vector Spans and Bases

An orthonormal basis is a basis of unit vectors which are all orthogonal to each other.

Example 2:
2/2 2/2
𝒗= 𝒘=
2/2 − 2/2

{𝒗, 𝒘} is an orthonormal basis for ℝ2 because they are unit vectors and orthogonal to
each other:
𝒗 = 𝒘 =1
𝒗⋅𝒘=0
 Vector Spans and Bases

An orthonormal basis is a basis of unit vectors which are all orthogonal to each other.

Example 3:

1 2/2
𝒗= 𝒘=
0 − 2/2

2
{𝒗, 𝒘} is not orthonormal basis for ℝ2 because 𝒗 ⋅ 𝒘 = >0
2
 Vector Spans and Bases

𝒗 𝟏 ,𝒗
If {ෝ ෝ 𝟐 ,…,𝒗
ෝ 𝒌 } is an orthonormal basis for a vector space, then the coordinates of
any vector 𝒘 from the vector space in this basis are given by:

ෝ
𝒘= 𝒘⋅𝒗 𝟏 ෝ
𝒗 𝟏 ෝ
+ 𝒘⋅𝒗 𝟐 ෝ
𝒗 𝟐 ෝ
+ ⋯+ 𝒘 ⋅ 𝒗 𝒌 ෝ
𝒗 𝒌

𝟏 𝟐 𝒌
i.e. 𝒘 has coordinates (𝒘 ⋅ 𝒗
ෝ ෝ
,𝒘⋅ 𝒗 ෝ
,…,𝒘 ⋅ 𝒗 ) in the basis.
 Vector Spans and Bases

Example:
2/2 2/2
𝒗= 𝒘=
2/2 − 2/2

4
In the orthonormal basis {𝒗, 𝒘} of ℝ2 , the vector 𝒖 = has coordinates:
−2

𝒖 ⋅ 𝒗, 𝒖 ⋅ 𝒘 = ( 2, 3 2)
Gram-Schmidt
Gram-Schmidt

The Gram–Schmidt process is a method for obtaining

an orthonormal basis from any basis for a vector
space.

Given a basis {𝒗𝟏 , 𝒗𝟐 , … , 𝒗𝒌 }, we will construct an

orthonormal basis {𝒆𝟏 , 𝒆𝟐 , … , 𝒆𝒌 } that spans the same
space. Reproduced from
https://en.wikipedia.org/wiki/Gram%E2%80%93
Algorithm: Schmidt_process#/media/File:Gram%E2%80%
93Schmidt_process.svg

Let 𝒖𝟏 = 𝒗𝟏 and for 𝑖 > 1, let 𝒖𝒊 = 𝒗𝒊 − σ𝑖−1

𝑗=1 𝑝𝑟𝑜𝑗𝒖𝒋 (𝒗𝒊 )

1
For all 𝑖, let 𝒆𝒊 = 𝒖𝒊 .
𝒖𝒊
Gram-Schmidt

Example:
3 −1
𝒗𝟏 = 𝒗𝟐 =
4 1
3
𝒖𝟏 = 𝒗𝟏 =
4
−1 3
𝒖𝟏 ⋅ 𝒗𝟐 ⋅
𝑝𝑟𝑜𝑗𝒖𝟏 𝒗𝟐 = 𝒖 = 1 4 3 = 1 3 = 3/25
𝒖𝟏 𝟐 𝟏 52 4 25 4 4/25

−1 3/25 −28/25
𝒖𝟐 = 𝒗𝟐 − 𝑝𝑟𝑜𝑗𝒖𝟏 (𝒗𝟐 ) = − =
1 4/25 21/25
Gram-Schmidt

Example:
3 −1
𝒗𝟏 = 𝒗𝟐 =
4 1

3 𝟏 1 3 3/5
𝒖𝟏 = 𝒆𝟏 = 𝒖𝟏 = =
4 𝒖𝟏 5 4 4/5

−28/25 𝟏 1 −28/25 −4/5

𝒖𝟐 = 𝒆𝟐 = 𝒖𝟐 = =
21/25 𝒖𝟐 7/5 21/25 3/5

3/5 −4/5
So Gram-Schmidt yields the orthonormal basis {𝒆𝟏 , 𝒆𝟐 } = { , }.
4/5 3/5
Further Reading
Further Reading

● https://tutorial.math.lamar.edu/classes/calcii/dotproduct.
aspx

● https://textbooks.math.gatech.edu/ila/row-reduction.html

● http://ltcconline.net/greenl/courses/203/vectors/linindspa
n.htm

● http://ltcconline.net/greenl/courses/203/vectors/orthonor
malbases.htm