Vector and Matrix Norms

ME 2004
Outline Vector Norms Matrix Norms Purpose

Outline
• 1.1: Vector Norms
• 1.2: Matrix Norms
• 1.3: Purpose of Norms

Vector/Matrix Norms 2
1.1: Vector Norms
Outline Vector Norms Matrix Norms Purpose

Vector Norms
• Norm: measures the size or “length” of vectors/matrices

𝑃 = 1 0 −1
𝑄= 1 4 0
Are 𝑃 and 𝑄 similar?

• Generalized form of a vector 𝑝-norm:

1
𝑛 𝑝
𝑝
𝑋 𝑝 = ෍ 𝑥𝑖
𝑖=1
Vector/Matrix Norms 4
Outline Vector Norms Matrix Norms Purpose

Vector Norms
• 1-norm/𝐿1 norm/Manhattan norm:
𝑛

𝑋 1 = ෍ 𝑥𝑖
𝑖=1

• “Total length” of the vector

Vector/Matrix Norms 5
Outline Vector Norms Matrix Norms Purpose

Vector Norms
• 2-norm/𝐿2 norm/Euclidean norm:

𝑋 = ෍ 𝑥𝑖 2
2
𝑖=1

• Shortest distance from the origin

Vector/Matrix Norms 6
Outline Vector Norms Matrix Norms Purpose

Vector Norms
• ∞-norm/𝐿∞ norm/max norm:

𝑋 ∞ = max 𝑥𝑖
1≤𝑖≤𝑛

• Absolute value of the largest element in the vector

Vector/Matrix Norms 7
Outline Vector Norms Matrix Norms Purpose

Vector Norms
• Given 𝑋 = [−1,4,2], calculate the 1-
norm, 2-norm, and ∞-norm.
• 1-norm: 𝑛

𝑋 1 = ෍ 𝑥𝑖 = 1 + 4 + 2 = 7
𝑖=1

• 2-norm:
𝑛

𝑋 = ෍ 𝑥𝑖 2 = −1 2 + 42 + 22 = 4.58
2
𝑖=1

• ∞-norm:
𝑋 ∞ = max 𝑥𝑖 = max(−1,2,4) = 4
1≤𝑖≤𝑛
Vector/Matrix Norms 8
1.2: Matrix Norms
Outline Vector Norms Matrix Norms Purpose

Matrix Norms
• 1-norm/column-sum norm:
𝑛

𝐴 1 = max ෍ 𝑎𝑖𝑗
1≤𝑗≤𝑛
𝑖=1

• Sum the absolute values of each column, then take the largest
summation

Vector/Matrix Norms 10
Outline Vector Norms Matrix Norms Purpose

Matrix Norms
• 2-norm/Frobenius norm
• ∞-norm/row-sum norm:
𝑛

𝐴 ∞ = max ෍ 𝑎𝑖𝑗
1≤𝑖≤𝑛
𝑗=1

• Sum the absolute values of each row, then take the largest
summation

Vector/Matrix Norms 11
Outline Vector Norms Matrix Norms Purpose

Matrix Norms
7 1 8
• Given A = 4 5 8 , calculate the 1-norm and ∞-norm.
10 4 2
• 1-norm:
𝑛

𝐴 1 = max ෍ 𝑎𝑖𝑗
1≤𝑗≤𝑛
𝑖=1

= max 7 + 4 + 10 1+5+4 8+8+2

→ 𝐴 1 = 21
Vector/Matrix Norms 12
Outline Vector Norms Matrix Norms Purpose

Matrix Norms
7 1 8
• Given A = 4 5 8 , calculate the 1-norm and ∞-norm.
10 4 2
• ∞-norm: 𝑛

𝐴 ∞ = max ෍ 𝑎𝑖𝑗
1≤𝑖≤𝑛
𝑗=1

7+1+8 16
= max 4+5+8 = max 17
10 + 4 + 2 16

→ 𝐴 ∞ = 17
Vector/Matrix Norms 13
Outline Vector Norms Matrix Norms Purpose

Matrix Norms
• In MATLAB:

norm(Z,p)

• Z = vector or matrix
• p = 1,2 (default),inf = which norm to compute
• Can also supply –inf and other positive scalars (vectors only)
• Can also supply 'fro' to compute matrix Frobenius norm (not used in this
class)

Vector/Matrix Norms 14
Outline Vector Norms Matrix Norms Purpose

Matrix Norms

Vector/Matrix Norms 15
Outline Vector Norms Matrix Norms Purpose

Matrix Norms

Vector/Matrix Norms 16
1.3: Purpose of Norms
Outline Vector Norms Matrix Norms Purpose

Applications
• Many linear algebra applications (machine learning, image
classification, etc.) rely on computing the (dis)similarity between
items
• Norms are useful because they’re a single number related to the size
of the vector/matrix, so they can be used as a comparison metric
Norm Application
𝐿1 LASSO (Least Absolute Shrinkage and Selection Operator)
(machine learning)
𝐴 ∞ Optimizing performance objectives
(control systems)

Vector/Matrix Norms 18
Outline Vector Norms Matrix Norms Purpose

Applications
• In MATLAB Grader (Robot Navigation):

• Absolute value of the largest element in the error vector

Vector/Matrix Norms 19
Outline Vector Norms Matrix Norms Purpose

Applications
• Condition number: measures how much the 𝑥 changes if 𝑏 changes
slightly

𝐶𝑜𝑛𝑑 𝐴 = 𝐴 ∗ 𝐴−1 ≥ 1

• Large condition number: small error in 𝑏 → large change in 𝑥

Vector/Matrix Norms 20
Outline Vector Norms Matrix Norms Purpose

Summary
• Norms measure the “size” of a vector/matrix
• Useful to directly compare vectors/matrices (such as the ones in your
Workshops ☺)
• Different types of norms: 1-norm, 2-norm, ∞-norm
• Can be computed by hand or in MATLAB

Vector/Matrix Norms 21