Introduction to Numerical Analysis

(Linear Systems: Matrix Norms)

MA 214, Spring 2023-24.

MA 214 - NA Spring 2022-23 1 / 27

Vector Norms

Definition (Vector Norm)

MA 214 - NA Spring 2022-23 2 / 27

Vector Norms

Definition (Vector Norm)

A vector norm on a vector space V is a function ∥ · ∥ : V → [0, ∞)
having the following properties:

MA 214 - NA Spring 2022-23 2 / 27

Vector Norms

Definition (Vector Norm)

A vector norm on a vector space V is a function ∥ · ∥ : V → [0, ∞)
having the following properties:
1 ∥x∥ ≥ 0 for all x ∈ V.

MA 214 - NA Spring 2022-23 2 / 27

Vector Norms

Definition (Vector Norm)

A vector norm on a vector space V is a function ∥ · ∥ : V → [0, ∞)
having the following properties:
1 ∥x∥ ≥ 0 for all x ∈ V.
2 ∥x∥ = 0 if and only if x = 0.

MA 214 - NA Spring 2022-23 2 / 27

Vector Norms

Definition (Vector Norm)

A vector norm on a vector space V is a function ∥ · ∥ : V → [0, ∞)
having the following properties:
1 ∥x∥ ≥ 0 for all x ∈ V.
2 ∥x∥ = 0 if and only if x = 0.
3 ∥αx∥ = |α|∥x∥ for all x ∈ V and for all α ∈ R.

MA 214 - NA Spring 2022-23 2 / 27

Vector Norms

Definition (Vector Norm)

A vector norm on a vector space V is a function ∥ · ∥ : V → [0, ∞)
having the following properties:
1 ∥x∥ ≥ 0 for all x ∈ V.
2 ∥x∥ = 0 if and only if x = 0.
3 ∥αx∥ = |α|∥x∥ for all x ∈ V and for all α ∈ R.
4 ∥x + y∥ ≤ ∥x∥ + ∥y∥ for all x, y ∈ V.

MA 214 - NA Spring 2022-23 2 / 27

Vector Norms (contd.)
Notation : x = (x1 , x2 , · · · , xn )T ∈ Rn .

MA 214 - NA Spring 2022-23 3 / 27

Vector Norms (contd.)
Notation : x = (x1 , x2 , · · · , xn )T ∈ Rn .
Example:
1 Euclidean norm on Rn is denoted
v by ∥ · ∥2 , and is defined by
u n
uX
∥x∥2 = t |xi |2 .
i=1

MA 214 - NA Spring 2022-23 3 / 27

Vector Norms (contd.)
Notation : x = (x1 , x2 , · · · , xn )T ∈ Rn .
Example:
1 Euclidean norm on Rn is denoted
v by ∥ · ∥2 , and is defined by
u n
uX
∥x∥2 = t |xi |2 .
i=1
2 l∞ norm, which is also called maximum norm, on Rn is denoted
by ∥ · ∥∞ , and is defined by
∥x∥∞ = max { |x1 |, |x2 |, · · · , |xn | } .

MA 214 - NA Spring 2022-23 3 / 27

Vector Norms (contd.)
Notation : x = (x1 , x2 , · · · , xn )T ∈ Rn .
Example:
1 Euclidean norm on Rn is denoted
v by ∥ · ∥2 , and is defined by
u n
uX
∥x∥2 = t |xi |2 .
i=1
2 l∞ norm, which is also called maximum norm, on Rn is denoted
by ∥ · ∥∞ , and is defined by
∥x∥∞ = max { |x1 |, |x2 |, · · · , |xn | } .
3 l1 norm on Rn is denoted by ∥ · ∥1 , and is defined by
∥x∥1 = |x1 | + |x2 | + · · · + |xn |.
MA 214 - NA Spring 2022-23 3 / 27
Equivalent Norms
Two norms ∥ · ∥ and ∥ · ∥′ on a vector space V are said to be
equivalent if there exist positive constants m and M such that

m∥x∥ ≤ ∥x∥′ ≤ M∥x∥ ∀x ∈ V

♣ All norms are equivalent on Rn

♣ ∥ · ∥ℓ∞ and ∥ · ∥ℓp norms are equivalent:
! p1
X
n r √
max |xi | ≤ |xi |p ≤ ∥x∥ℓ∞ 1
p |
+ · ·
{z }· + 1 = p
n ∥x∥ℓ∞
1≤i≤n
i=1 n terms

MA 214 - NA Spring 2022-23 4 / 27

 Thus we have √
∥x∥ℓ∞ ≤ ∥x∥ℓp ≤ p
n ∥x∥ℓ∞
proving that ∥ · ∥ℓ∞ and ∥ · ∥ℓp norms are equivalent

MA 214 - NA Spring 2022-23 5 / 27

 equivalent norms (contd.)
♣ ∥ · ∥ℓ1 and ∥ · ∥ℓ2 norms are equivalent:
Recall the identity:
!2
X
n X
n X
n X
i−1
|xi | = 2
|xi | + 2 |xj | |xi |
i=1 i=1 i=1 j=1
| {z }
≥0

i.e. ∥x∥2ℓ1 ≥ ∥x∥2ℓ2

Cauchy-Schwarz inequality
! 12 ! 21
X
n X
n X
n
|⟨f, g⟩| = fi gi ≤ |fi |2 |gi |2
i=1 i=1 i=1
! 12 ! 21
X
n X
n X
n
√
=⇒ |xi | ≤ 2
|xi | 1 = n ∥x∥ℓ2
i=1 i=1
MA 214 - NA i=1 Spring 2022-23 6 / 27
So, together we have
√
∥x∥ℓ2 ≤ ∥x∥ℓ1 ≤ n∥x∥ℓ2

MA 214 - NA Spring 2022-23 7 / 27

Matrix Norms

Definition (Matrix Norm)

A matrix norm on the vector space of all n × n real matrices Mn (R)
is a function ∥ · ∥ : Mn (R) → [0, ∞) having the following properties:

MA 214 - NA Spring 2022-23 8 / 27

Matrix Norms

Definition (Matrix Norm)

A matrix norm on the vector space of all n × n real matrices Mn (R)
is a function ∥ · ∥ : Mn (R) → [0, ∞) having the following properties:
1 ∥A∥ ≥ 0 for all A ∈ Mn (R).

MA 214 - NA Spring 2022-23 8 / 27

Matrix Norms

Definition (Matrix Norm)

A matrix norm on the vector space of all n × n real matrices Mn (R)
is a function ∥ · ∥ : Mn (R) → [0, ∞) having the following properties:
1 ∥A∥ ≥ 0 for all A ∈ Mn (R).
2 ∥A∥ = 0 if and only if A = 0.

MA 214 - NA Spring 2022-23 8 / 27

Matrix Norms

Definition (Matrix Norm)

A matrix norm on the vector space of all n × n real matrices Mn (R)
is a function ∥ · ∥ : Mn (R) → [0, ∞) having the following properties:
1 ∥A∥ ≥ 0 for all A ∈ Mn (R).
2 ∥A∥ = 0 if and only if A = 0.
3 ∥αA∥ = |α|∥A∥ for all A ∈ Mn (R) and for all α ∈ R.

MA 214 - NA Spring 2022-23 8 / 27

Matrix Norms

Definition (Matrix Norm)

A matrix norm on the vector space of all n × n real matrices Mn (R)
is a function ∥ · ∥ : Mn (R) → [0, ∞) having the following properties:
1 ∥A∥ ≥ 0 for all A ∈ Mn (R).
2 ∥A∥ = 0 if and only if A = 0.
3 ∥αA∥ = |α|∥A∥ for all A ∈ Mn (R) and for all α ∈ R.
4 ∥A + B∥ ≤ ∥A∥ + ∥B∥ for all A, B ∈ Mn (R).

MA 214 - NA Spring 2022-23 8 / 27

Matrix Norms

Definition (Matrix Norm)

A matrix norm on the vector space of all n × n real matrices Mn (R)
is a function ∥ · ∥ : Mn (R) → [0, ∞) having the following properties:
1 ∥A∥ ≥ 0 for all A ∈ Mn (R).
2 ∥A∥ = 0 if and only if A = 0.
3 ∥αA∥ = |α|∥A∥ for all A ∈ Mn (R) and for all α ∈ R.
4 ∥A + B∥ ≤ ∥A∥ + ∥B∥ for all A, B ∈ Mn (R).
We use the notation
A = (aij )1≤i≤n
1≤j≤n

MA 214 - NA Spring 2022-23 8 / 27

Matrix Norms (contd.)

Example:
The following define norms on Mn (R).

MA 214 - NA Spring 2022-23 9 / 27

Matrix Norms (contd.)

Example:
The following define norms on Mn (R).
v
uX
u n X n
1 ∥A∥ = t |aij |2 .
i=1 j=1

MA 214 - NA Spring 2022-23 9 / 27

Matrix Norms (contd.)

Example:
The following define norms on Mn (R).
v
uX
u n X n
1 ∥A∥ = t |aij |2 .
i=1 j=1

2 ∥A∥ = max { |aij | : 1 ≤ i ≤ n, 1 ≤ j ≤ n } .

MA 214 - NA Spring 2022-23 9 / 27

Matrix Norms (contd.)

Example:
The following define norms on Mn (R).
v
uX
u n X n
1 ∥A∥ = t |aij |2 .
i=1 j=1

2 ∥A∥ = max { |aij | : 1 ≤ i ≤ n, 1 ≤ j ≤ n } .

Pn Pn
3 ∥A∥ = i=1 j=1 |aij |.

MA 214 - NA Spring 2022-23 9 / 27

——————————————————————————

MA 214 - NA Spring 2022-23 10 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

An important norm:

MA 214 - NA Spring 2022-23 10 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

An important norm:

Given an n × n matrix A with real entries, let

λ1 , λ2 , · · · , λn
be eigenvalues of the matrix AT A.

MA 214 - NA Spring 2022-23 10 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

An important norm:

Given an n × n matrix A with real entries, let

λ1 , λ2 , · · · , λn
be eigenvalues of the matrix AT A.

Define q
∥A∥2 = max |λi |.
1≤i≤n

MA 214 - NA Spring 2022-23 10 / 27

Matrix Norms Subordinate to a Vector Norm

An important matrix norm

MA 214 - NA Spring 2022-23 11 / 27

Matrix Norms Subordinate to a Vector Norm

An important matrix norm

Definition (Matrix Norm subordinate to a vector norm)

Let ∥ · ∥ be a vector norm on Rn . The matrix norm subordinate to
the vector norm ∥ · ∥ is defined by

∥A∥ := sup {∥Ax∥ : x ∈ Rn , ∥x∥ = 1} for A ∈ Mn (R).

MA 214 - NA Spring 2022-23 11 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Lemma
For any A ∈ Mn (R) and a given vector norm ∥ · ∥, we have

∥Az∥
∥A∥ = max .
z̸=0 ∥z∥

Proof:

MA 214 - NA Spring 2022-23 12 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Lemma
For any A ∈ Mn (R) and a given vector norm ∥ · ∥, we have

∥Az∥
∥A∥ = max .
z̸=0 ∥z∥

Proof: For any z ̸= 0, we have x = z/||z|| as a unit vector.

MA 214 - NA Spring 2022-23 12 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Lemma
For any A ∈ Mn (R) and a given vector norm ∥ · ∥, we have

∥Az∥
∥A∥ = max .
z̸=0 ∥z∥

Proof: For any z ̸= 0, we have x = z/||z|| as a unit vector. Hence

max ||Ax|| =
||x||=1

MA 214 - NA Spring 2022-23 12 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Lemma
For any A ∈ Mn (R) and a given vector norm ∥ · ∥, we have

∥Az∥
∥A∥ = max .
z̸=0 ∥z∥

Proof: For any z ̸= 0, we have x = z/||z|| as a unit vector. Hence

 
z
max ||Ax|| = max A
||x||=1 ||z||̸=0 ||z||
=
MA 214 - NA Spring 2022-23 12 / 27
Matrix Norms Subordinate to a Vector Norm (contd.)

Lemma
For any A ∈ Mn (R) and a given vector norm ∥ · ∥, we have

∥Az∥
∥A∥ = max .
z̸=0 ∥z∥

Proof: For any z ̸= 0, we have x = z/||z|| as a unit vector. Hence

 
z
max ||Ax|| = max A
||x||=1 ||z||̸=0 ||z||
||Az||
= max .
||z||̸=0 ||z||
MA 214 - NA Spring 2022-23 12 / 27
Matrix Norms Subordinate to a Vector Norm (contd.)

The matrix norm subordinate to a vector norm has additional properties.

MA 214 - NA Spring 2022-23 13 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

The matrix norm subordinate to a vector norm has additional properties.

Theorem
Let ∥ · ∥ be a matrix norm subordinate to a vector norm. Then
1 ∥Ax∥ ≤ ∥A∥ ∥x∥ for all x ∈ Rn .

MA 214 - NA Spring 2022-23 13 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

The matrix norm subordinate to a vector norm has additional properties.

Theorem
Let ∥ · ∥ be a matrix norm subordinate to a vector norm. Then
1 ∥Ax∥ ≤ ∥A∥ ∥x∥ for all x ∈ Rn .
2 ∥I∥ = 1 where I is the Identity matrix.

MA 214 - NA Spring 2022-23 13 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

The matrix norm subordinate to a vector norm has additional properties.

Theorem
Let ∥ · ∥ be a matrix norm subordinate to a vector norm. Then
1 ∥Ax∥ ≤ ∥A∥ ∥x∥ for all x ∈ Rn .
2 ∥I∥ = 1 where I is the Identity matrix.
3 ∥AB∥ ≤ ∥A∥ ∥B∥ for all A, B ∈ Mn (R).

MA 214 - NA Spring 2022-23 13 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the maximum norm)

The matrix norm subordinate to the l∞ norm (also called maximum
norm) on Rn

MA 214 - NA Spring 2022-23 14 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the maximum norm)

The matrix norm subordinate to the l∞ norm (also called maximum
norm) on Rn
∥x∥∞ = max { |x1 |, |x2 |, · · · , |xn | }

MA 214 - NA Spring 2022-23 14 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the maximum norm)

The matrix norm subordinate to the l∞ norm (also called maximum
norm) on Rn
∥x∥∞ = max { |x1 |, |x2 |, · · · , |xn | }
is denoted by ∥A∥∞ and is given by

X
n
∥A∥∞ = max |aij |.
1≤i≤n
j=1

MA 214 - NA Spring 2022-23 14 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the maximum norm)

The matrix norm subordinate to the l∞ norm (also called maximum
norm) on Rn
∥x∥∞ = max { |x1 |, |x2 |, · · · , |xn | }
is denoted by ∥A∥∞ and is given by

X
n
∥A∥∞ = max |aij |.
1≤i≤n
j=1

The norm ∥A∥∞ given by the above formula is called

maximum-of-row-sums norm.
MA 214 - NA Spring 2022-23 14 / 27
Matrix Norms Subordinate to a Vector Norm (contd.)

 ∥A∥∞ for
Example: Let us now compute  the matrix
1 1 −1
 1 2 −2 .
−2 1 1

MA 214 - NA Spring 2022-23 15 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

 ∥A∥∞ for
Example: Let us now compute  the matrix
1 1 −1
 1 2 −2 .
−2 1 1
X 3
|a1j | = |1| + |1| + | − 1| = 3,
j=1

MA 214 - NA Spring 2022-23 15 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

 ∥A∥∞ for
Example: Let us now compute  the matrix
1 1 −1
 1 2 −2 .
−2 1 1
X 3
|a1j | = |1| + |1| + | − 1| = 3,
j=1

X
3
|a2j | = |1| + |2| + | − 2| = 5,
j=1

MA 214 - NA Spring 2022-23 15 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

 ∥A∥∞ for
Example: Let us now compute  the matrix
1 1 −1
 1 2 −2 .
−2 1 1
X 3
|a1j | = |1| + |1| + | − 1| = 3,
j=1

X
3
|a2j | = |1| + |2| + | − 2| = 5,
j=1

X
3
|a3j | = | − 2| + |1| + |1| = 4.
j=1
MA 214 - NA Spring 2022-23 15 / 27
Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix Norm Subordinate to the l1 -norm)

The matrix norm subordinate to the l1 -norm on Rn

∥x∥1 = |x1 | + |x2 | + · · · + |xn |. (1)

is denoted by ∥A∥1 and is given by

X
n
∥A∥1 = max |aij |. (2)
1≤j≤n
i=1

The norm ∥A∥1 is called the maximum-of-column-sums norm of A.

MA 214 - NA Spring 2022-23 16 / 27
Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the Euclidean norm)

The matrix norm subordinate to the l2 norm (also called Euclidean
norm) on Rn

MA 214 - NA Spring 2022-23 17 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the Euclidean norm)

The matrix norm subordinate to the l2 norm (also called Euclidean
norm) on Rn v
u n
uX
∥x∥2 = t |xi |2
i=1

MA 214 - NA Spring 2022-23 17 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the Euclidean norm)

The matrix norm subordinate to the l2 norm (also called Euclidean
norm) on Rn v
u n
uX
∥x∥2 = t |xi |2
i=1

is denoted by ∥A∥2 q
∥A∥2 = max |λi |,
1≤i≤n

MA 214 - NA Spring 2022-23 17 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Theorem (Matrix norm subordinate to the Euclidean norm)

The matrix norm subordinate to the l2 norm (also called Euclidean
norm) on Rn v
u n
uX
∥x∥2 = t |xi |2
i=1

is denoted by ∥A∥2 q
∥A∥2 = max |λi |,
1≤i≤n

where λ1 , λ2 , · · · , λn are eigenvalues of the matrix AT A.

MA 214 - NA Spring 2022-23 17 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Example:
Let us now compute ∥A∥2 for the matrix
 
1 1 −1
1 2 −2 .
−2 1 1

MA 214 - NA Spring 2022-23 18 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Example:
Let us now compute ∥A∥2 for the matrix
 
1 1 −1
1 2 −2 .
−2 1 1

The eigenvalues of AT A are

MA 214 - NA Spring 2022-23 18 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Example:
Let us now compute ∥A∥2 for the matrix
 
1 1 −1
1 2 −2 .
−2 1 1

The eigenvalues of AT A are

λ1 ≈ 0.0616, λ2 ≈ 5.0256 and λ3 ≈ 12.9128.

MA 214 - NA Spring 2022-23 18 / 27

Matrix Norms Subordinate to a Vector Norm (contd.)

Example:
Let us now compute ∥A∥2 for the matrix
 
1 1 −1
1 2 −2 .
−2 1 1

The eigenvalues of AT A are

λ1 ≈ 0.0616, λ2 ≈ 5.0256 and λ3 ≈ 12.9128.

√
Hence ∥A∥2 ≈ 12.9128 ≈ 3.5934.
MA 214 - NA Spring 2022-23 18 / 27
Condition Number of a Matrix

Theorem
Let A be an invertible n × n matrix.

MA 214 - NA Spring 2022-23 19 / 27

Condition Number of a Matrix

Theorem
Let A be an invertible n × n matrix. Let x and x̃ be the solutions of the
systems
Ax = b and Ax̃ = b̃,

MA 214 - NA Spring 2022-23 19 / 27

Condition Number of a Matrix

Theorem
Let A be an invertible n × n matrix. Let x and x̃ be the solutions of the
systems
Ax = b and Ax̃ = b̃,
respectively, where b and b̃ are given vectors.

MA 214 - NA Spring 2022-23 19 / 27

Condition Number of a Matrix

Theorem
Let A be an invertible n × n matrix. Let x and x̃ be the solutions of the
systems
Ax = b and Ax̃ = b̃,
respectively, where b and b̃ are given vectors. Then

∥x − x̃∥ ∥b − b̃∥
≤ ∥A∥ ∥A−1 ∥
∥x∥ ∥b∥

MA 214 - NA Spring 2022-23 19 / 27

Condition Number of a Matrix (contd.)
Proof:

MA 214 - NA Spring 2022-23 20 / 27

Condition Number of a Matrix (contd.)
Proof:
Since A is invertible, we have  
−1
x − x̃ = A b − b̃ .

MA 214 - NA Spring 2022-23 20 / 27

Condition Number of a Matrix (contd.)
Proof:
Since A is invertible, we have  
−1
x − x̃ = A b − b̃ .
Take norm on both sides

MA 214 - NA Spring 2022-23 20 / 27

Condition Number of a Matrix (contd.)
Proof:
Since A is invertible, we have  
−1
x − x̃ = A b − b̃ .
Take norm on both sides
∥x − x̃∥ ≤ ∥A−1 ∥∥b − b̃∥

MA 214 - NA Spring 2022-23 20 / 27

Condition Number of a Matrix (contd.)
Proof:
Since A is invertible, we have  
−1
x − x̃ = A b − b̃ .
Take norm on both sides
∥x − x̃∥ ≤ ∥A−1 ∥∥b − b̃∥
Since Ax = b, we get
∥b∥ = ∥Ax∥ ≤ ∥A∥∥x∥.

MA 214 - NA Spring 2022-23 20 / 27

Condition Number of a Matrix (contd.)
Proof:
Since A is invertible, we have  
−1
x − x̃ = A b − b̃ .
Take norm on both sides
∥x − x̃∥ ≤ ∥A−1 ∥∥b − b̃∥
Since Ax = b, we get
∥b∥ = ∥Ax∥ ≤ ∥A∥∥x∥.

1 ∥A∥
⇒ ≤
∥x∥ ∥b∥
MA 214 - NA Spring 2022-23 20 / 27
Condition Number of a Matrix (contd.)
Proof:
Since A is invertible, we have  
−1
x − x̃ = A b − b̃ .
Take norm on both sides
∥x − x̃∥ ≤ ∥A−1 ∥∥b − b̃∥
Since Ax = b, we get
∥b∥ = ∥Ax∥ ≤ ∥A∥∥x∥.

1 ∥A∥ ∥x − x̃∥ ∥b − b̃∥

⇒ ≤ ⇒ ≤ ∥A∥ ∥A−1 ∥
∥x∥ ∥b∥ ∥x∥ ∥b∥
MA 214 - NA Spring 2022-23 20 / 27
Condition Number of a Matrix (contd.)
Theorem
Let A and à be non-singular square matrices, and b ̸= 0.
If x and x̃ are the solutions of the systems
Ax = b and Ãx̃ = b,
respectively, then
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥

MA 214 - NA Spring 2022-23 21 / 27

Condition Number of a Matrix (contd.)
Theorem
Let A and à be non-singular square matrices, and b ̸= 0.
If x and x̃ are the solutions of the systems
Ax = b and Ãx̃ = b,
respectively, then
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥
Proof:
Ax − Ãx̃ = 0

MA 214 - NA Spring 2022-23 21 / 27

Condition Number of a Matrix (contd.)
Theorem
Let A and à be non-singular square matrices, and b ̸= 0.
If x and x̃ are the solutions of the systems
Ax = b and Ãx̃ = b,
respectively, then
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥
Proof:
Ax − Ãx̃ = 0 ⇒ A(x − x̃) = −(A − Ã)x̃

MA 214 - NA Spring 2022-23 21 / 27

Condition Number of a Matrix (contd.)
Theorem
Let A and à be non-singular square matrices, and b ̸= 0.
If x and x̃ are the solutions of the systems
Ax = b and Ãx̃ = b,
respectively, then
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥
Proof:
Ax − Ãx̃ = 0 ⇒ (x − x̃) = −A−1 (A − Ã)x̃

MA 214 - NA Spring 2022-23 22 / 27

Condition Number of a Matrix (contd.)
Theorem
Let A and à be non-singular square matrices, and b ̸= 0.
If x and x̃ are the solutions of the systems
Ax = b and Ãx̃ = b,
respectively, then
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥
Proof:
Ax − Ãx̃ = 0 ⇒ ∥x − x̃∥ ≤ ∥A−1 ∥ ∥A − Ã∥ ∥x̃∥

MA 214 - NA Spring 2022-23 23 / 27

Condition Number of a Matrix (contd.)
Theorem
Let A and à be non-singular square matrices, and b ̸= 0.
If x and x̃ are the solutions of the systems
Ax = b and Ãx̃ = b,
respectively, then
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥
Proof:
Ax − Ãx̃ = 0 ⇒ ∥x − x̃∥ ≤ ∥A−1 ∥ ∥A − Ã∥ ∥x̃∥
∥x − x̃∥
⇒ ≤ ∥A−1 ∥ ∥A − Ã∥
∥x̃∥
MA 214 - NA Spring 2022-23 23 / 27
Condition Number of a Matrix (contd.)

Definition
Let A be an n × n invertible matrix. Let a matrix norm be given that is
subordinate to a vector norm. Then the condition number of the matrix
A is defined as
κ(A) := ∥A∥ ∥A−1 ∥.

MA 214 - NA Spring 2022-23 24 / 27

Condition Number of a Matrix (contd.)
∥x − x̃∥ ∥b − b̃∥
≤ ∥A∥ ∥A−1 ∥
∥x∥ ∥b∥
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥

MA 214 - NA Spring 2022-23 25 / 27

Condition Number of a Matrix (contd.)
∥x − x̃∥ ∥b − b̃∥
≤ ∥A∥ ∥A−1 ∥
∥x∥ ∥b∥
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥

It is clear that if the condition number is small, then the relative error in
the solution will also be small whenever the relative error in the right
hand side vector is small.

MA 214 - NA Spring 2022-23 25 / 27

Condition Number of a Matrix (contd.)
∥x − x̃∥ ∥b − b̃∥
≤ ∥A∥ ∥A−1 ∥
∥x∥ ∥b∥
∥x̃ − x∥ ∥A − Ã∥
≤ ∥A∥ ∥A−1 ∥ .
∥x̃∥ ∥A∥

It is clear that if the condition number is small, then the relative error in
the solution will also be small whenever the relative error in the right
hand side vector is small.
On the other hand, if the condition number is large, then the relative
error could be very large even though the relative error in the right hand
side vector is small.
MA 214 - NA Spring 2022-23 25 / 27
Condition Number of a Matrix (contd.)

Example: Consider 5x1 + 7x2 = 0.7, 7x1 + 10x2 = 1

T
Solution: x = (0, 0.1) .

MA 214 - NA Spring 2022-23 26 / 27

Condition Number of a Matrix (contd.)

Example: Consider 5x1 + 7x2 = 0.7, 7x1 + 10x2 = 1

T
Solution: x = (0, 0.1) .
The perturbed system with RHS vector b̃ = (0.69, 1.01)T given by
5x1 + 7x2 = 0.69, 7x1 + 10x2 = 1.01

has the solution x̃ = (−0.17, 0.22)T .

MA 214 - NA Spring 2022-23 26 / 27

Condition Number of a Matrix (contd.)

Example: Consider 5x1 + 7x2 = 0.7, 7x1 + 10x2 = 1

T
Solution: x = (0, 0.1) .
The perturbed system with RHS vector b̃ = (0.69, 1.01)T given by
5x1 + 7x2 = 0.69, 7x1 + 10x2 = 1.01

has the solution x̃ = (−0.17, 0.22)T .

∥x − x̃∥∞
The relative error in solution with the l∞ vector norm is = 1.7,
∥x∥∞

MA 214 - NA Spring 2022-23 26 / 27

Condition Number of a Matrix (contd.)

Example: Consider 5x1 + 7x2 = 0.7, 7x1 + 10x2 = 1

T
Solution: x = (0, 0.1) .
The perturbed system with RHS vector b̃ = (0.69, 1.01)T given by
5x1 + 7x2 = 0.69, 7x1 + 10x2 = 1.01

has the solution x̃ = (−0.17, 0.22)T .

∥x − x̃∥∞
The relative error in solution with the l∞ vector norm is = 1.7,
∥x∥∞
∥b − b̃∥∞
The relative error in RHS vector is = 0.01.
∥b∥∞

MA 214 - NA Spring 2022-23 26 / 27

Condition Number of a Matrix (contd.)

Example: Consider 5x1 + 7x2 = 0.7, 7x1 + 10x2 = 1

T
Solution: x = (0, 0.1) .
The perturbed system with RHS vector b̃ = (0.69, 1.01)T given by
5x1 + 7x2 = 0.69, 7x1 + 10x2 = 1.01

has the solution x̃ = (−0.17, 0.22)T .

∥x − x̃∥∞
The relative error in solution with the l∞ vector norm is = 1.7,
∥x∥∞
∥b − b̃∥∞
The relative error in RHS vector is = 0.01.
∥b∥∞
The condition number of the coefficient matrix of the system is 289.
MA 214 - NA Spring 2022-23 26 / 27
Condition Number of a Matrix (contd.)
Definition
A matrix with a large condition number is said to be ill conditioned. A
matrix with a small condition number is said to be well conditioned.

MA 214 - NA Spring 2022-23 27 / 27

Condition Number of a Matrix (contd.)
Definition
A matrix with a large condition number is said to be ill conditioned. A
matrix with a small condition number is said to be well conditioned.
Example: The Hilbert matrix is given by
 
1 12 1
3 · · · 1
n
1 1 1
· · · 1 
2 3 4 n+1 
· ··· 
Hn = ·
;

 ··· 
· ··· 
1 1 1 1
n n+1 n+1 · · · 2n−1

MA 214 - NA Spring 2022-23 27 / 27

Condition Number of a Matrix (contd.)
Definition
A matrix with a large condition number is said to be ill conditioned. A
matrix with a small condition number is said to be well conditioned.
Example: The Hilbert matrix is given by
 
1 12 1
3 · · · 1
n
1 1 1
· · · 1 
2 3 4 n+1 
· ··· 
Hn = ·
;

 ··· 
· ··· 
1 1 1 1
n n+1 n+1 · · · 2n−1
25
κ(H4 ) = ∥H4 ∥∥H−14 ∥ = 13620 ≈ 28000
12
MA 214 - NA Spring 2022-23 27 / 27
Introduction to MA-414

January 8, 2021

Introduction to MA-414
Welcome to the first lecture of MA-414. This is mainly a course on
Field and Galois theory and its connections to classical geometry
and the theory of equations. Galois theory is indeed one of the
jewels of mathematics and hopefully by the end of the course I will
convince you of this.
Let me outline the topics in this course, beginning with a historical
overview.

Introduction to MA-414
Solving equations is something that occupied us from school days.
The simplest equations were linear equations in one variable and
they are very easy to solve. They also enlarge the number system
from integers to the rational numbers. The formula
√
−b ± b 2 − 4ac
x=
2a
for the roots of the quadratic equation ax 2 + bx + c = 0 was
known to Babylonians. During the reign of King Hammurabi (1750
B. C.), Babylonian mathematicians found methods of solving linear
and quadratic equations in one and two variables. They described
algorithms to solve specific examples. From these examples it is
clear that they knew the formula for the roots of quadratic
equations.

Introduction to MA-414
In 1494, an Italian mathematician Franciscan Luca Pacioli
published the book Summa de Arithmetica, Geometria, Proportioni
et Proportionalita containing all that was known in that period in
arithmetic, algebra, geometry and trigonometry. Paciolo ended his
book with a remark that solutions of cubic equations seemed
impossible.

Generations of mathematicians at the University of Bologna in

Italy tried to find solutions of cubic equations. This was the largest
and one of the most famous universities at the turn of sixteenth
century in Europe. Scipio Del Ferro at this univerisity solved the
cubic but never published his findings.

Introduction to MA-414
In 1535, Niccolo Tartaglia, a mathematician from Venice proved in
a public demonstration that he could solve cubic equations. But he
kept his formula a secret. In 1539 Cardano cajoled Tartaglia into
revealing his solution to the cubic equations by promising not to
publish them. Tartaglia divulged the secrets of the solutions of
three different forms of the cubic equation in verse. Several years
later, Cardano happened to see unpublished work by Scipione del
Ferro who independently came up with the same solution as
Tartaglia. As the unpublished work was dated before Tartaglia’s,
Cardano decided his promise could be broken and included
Tartaglia’s solution in his next publication. Even though Cardano
credited his discovery , Tartaglia was extremely upset. Widespread
stories that Tartaglia devoted the rest of his life to ruining
Cardano, however, appear to be completely fabricated.
Mathematical historians now credit both Cardano and Tartaglia
with the formula to solve cubic equations, referring to it as the
”Cardano-Tartaglia formula”.

Introduction to MA-414
Here’s the recipe to solving the cubic equation.
Let ax 3 + bx 2 + cx + d be a cubic whose roots we wish to find. By
substituting x = y − b/3a we get a so called ’depressed cubic’ ,
one with missing x 2 coefficient. Thus we are left to solve a cubic
of the form y 3 + Ay = B. We now find s and t so that

3st = A and s 3 − t 3 = B

Then s − t is a solution of this polynomial, for

(s − t)3 + 3st(s − t) = s 3 − t 3
We are thus reduced to finding s and t. The above equation
A 3
yields: became ( 3t ) − t 3 = B. Simplifying gives
3
t 6 + Bt 3 − A27 = 0
This is a quadratic in t 3 and hence can be solved for t 3 and hence
for t. Hence we can find s and t with desired property.
The quartic equation is little more difficult and we will come back
to this later.

Introduction to MA-414
In 1536 Lodovico Ferrari entered Cardano’s house as a servant.
Due to his extraordinary mathematical abilities he became a
mathematician under Cardano’s guidance. Ferrari showed that a
quartic equation can be reduced to a cubic equation and therefore
it can be solved by means of four arithmetic operations and
extraction of square and cube roots. We will derive the formulas of
Cardano and Ferrari later.
Some of the greatest mathematicians, e.g., Euler and Lagrange
attempted to find similar formula for the roots of quintic
equations. Lagrange gave a general method to solve equations of
degree atmost four. But this method did not work for quintic
equations. Using clever substitutions, mathematicians were able to
reduced the solution of the general quintic to the very degenerate
quintic z 5 + pz + q = 0. However solving this seems intractable for
over a century.

Introduction to MA-414
Mathematicians became skeptical about existence of such formulas
for equations of degree five and higher. Paolo Ruffini, born 1765
was a student of Lagrange. He published several papers(1802,
1813) about insolvability of general quintic equation. His proof
was not complete. The first complete proof was given by Neils
Henrik Abel (1802-1829) in 1824. Abel also proved that if the
’Galois group’ of the polynomial is commutative then the
polynomial is solvable by radicals. Roughly, to every polynomial
defined over a field k, one can associate an extension field F which
is the smallest field which contains the roots of this polynomial.
The field automorphisms of F which fix every element of k form a
group which is called the Galois group of the polynomial. It is a
finite group. This sets up a connection between field theory and
group theory which we will explore in detail. Commutative groups
are called Abelian groups to honour Abel for his deep work in many
branches of mathematics. Abel died at a young age of 27, but
already he had many outstanding works. A very prestigious prize
called the Abel prize is instituted in his honor.
Introduction to MA-414
Gauss made two fundamental contributions to the theory of
equations. He provided complete solution by means of radicals of
the cyclotomic equation

x n − 1 = 0.

The roots of this equation are the complex numbers represented by

the vertices of a regular polygon of n sides centered at the origin.
Gauss analysis of the roots of cyclotomic equation led him to find
a criterion of the constructibility of regular polygons of n sides. We
will discuss this later.

Introduction to MA-414
The second contribution was first rigorous proof of the
Fundamental Theorem of Algebra: Every non-constant polynomial
with complex coefficients is a product of linear factors with
complex coefficients. He gave the first proof of this in 1799 which
was not completely rigorous by modern day standards. He later
went on give many more (complete) proofs. As most of you know,
complex analysis provides many proofs of this theorem. In this
course we will discuss an almost algebraic proof of the fundamental
theorem of algebra using Galois theory. A purely algebraic proof of
the theorem does not exist !

Introduction to MA-414
The most decisive results in the theory of equations were found by
Evariste Galois (1811-1832). Modern algebra began with the work
of Galois. He introduced the Galois group of a polynomial which
connected field theory with group theory. In 1829, Galois presented
two papers to the Paris Academy of Sciences. These were sent to
Cauchy who lost them. In 1830 he sent another paper to the
Academy whose secretary was Fourier who died before he could
examine this paper. The manuscript has never been found. In
1830, Galois published a summary of his results. The first theorem
in this account is: An equation of prime degree is solvable by
radicals if and only if if two of its roots are known then the others
are rational functions of them. This implied that a general
equation of degree five cannot be solved by means of radicals. The
most decisive result is the solvability criterion: A polynomial is
solvable by radicals if and only if its Galois group is solvable. Here
solvability by radicals roughly means roots can be computed using
only the operations +, -, ., /, and nth roots, in a finite number of
steps, using the coefficients of the equation.
Introduction to MA-414
We will also study the solutions of several problems in Greek
Geometry using rudiments of field theory. In Euclidean Geometry,
we carry out several geometric constructions with a ruler
(unmarked) and compass such as bisection of line segments and
angles, constructions of certain angles, triangles, quadrilaterals and
circles. Ancient Greeks posed the following four problems:
1 The Delian Problem : Construct the side of a cube of
volume 2.
2 The angle trisection problem : Divide a given angle in
three equal parts.
3 Squaring a circle : Construct a square having same area as
that of a given circle.
4 Constructible regular polygons : Find n for which regular
polygon of n sides can be constructed by ruler and compass
and describe their constructions.

Introduction to MA-414
The above problems remained open for almost 2200 years. The
Delian problem owes its name to a story concerning the citizens of
Delos, who consulted the oracle at Delphi in order to learn how to
defeat a plague sent by Apollo. Others say the citizens of Delos
consulted the oracle at Delphi, seeking a solution for their internal
political problems at the time, which had intensified relationships
among the citizens. The oracle responded that they must double
the size of the altar to Apollo, which was a regular cube. The
answer seemed strange to the Delians and they consulted Plato,
who was able to interpret the oracle as the mathematical problem
of doubling the volume of a given cube, thus explaining the oracle
as the advice of Apollo for the citizens of Delos to occupy
themselves with the study of geometry and mathematics in order
to calm down their passions !! But Pierre Wantzel proved in 1837
that the cube root of 2 is not constructible; that is, it cannot be
constructed with straightedge and compass.

Introduction to MA-414
The second and the third problems also circulated among Greek
geometers around the same time. The angle trisection problem was
settled by Pierre Laurent Wantzel in 1837. It was probably also
solved by Gauss as a special case of his remarkable solution of the
fourth problem. Gauss, barely 19, provided a construction of the
17-sided regular polygon. He also characterized n for which regular
n−gons are constructible by ruler and compass. Recall that a prime
m
of the form 22 + 1 is called a Fermat prime. Gauss proved that a
regular n−gon is constructible if and only if n = 2r p1 p2 . . . pg
where n ≥ 0 and p1 , p2 , . . . , pg are distinct Fermat primes. Gauss’s
Theorem solves the angle trisection problem. The values of n for
which regular n−gons were known to be constructible upto the
time of Gauss were n = 2m , 2m .3, 2m .5 and 2m .15. No one was
able to construct a heptagon or a regular 17−gon. In 1796 Gauss
made his first mathematical discovery : construction of a 17-sided
regular polygon by ruler and compass. He began noting down his
mathematical discoveries in a diary which he maintained for the
next 19 years. Gauss published “Disquisitiones Arithmaticae” in
1801 which has become a classic in mathematical literature.
Introduction to MA-414
The last result of this is his solution to the fourth problem. Gauss
was very proud of this discovery. He desired that a regular polygon
of 17 sides be engraved on his tombstone. This wish was not
fulfilled. It was fulfilled when a monument to Gauss was built in
his birth place Braunschwig. Explicit construction of 17−sided
regular polygon was given by Erchinger in 1800. In 1892 Richelot
and Schwendenwein constructed a regular 257−gon. Around 1900
Hermes constructed a regular 65537-gon. The manuscript fills a
box and it is found in Góttingen. The construction has now been
computerized.
Lindemann proved in 1882 that π is not a root of any polynomial
with rational coefficients. Such numbers are called transcendental
numbers. This proved the impossibility of squaring a circle.

Introduction to MA-414
This was a historical account of some of the topics we will study in
the course. I should add that Galois theory has very deep
connections to areas like Number theory, Algebraic Geometry and
Representation Theory. Some of the deepest mathematics in the
last 25 years, especially in the so called Langlands program, has
been on the connections between these three areas of
mathematics. In the first half of this course, we will study field
extensions (mostly finite extensions) and their connections to
polynomials. We will apply this knowledge to understand the
solutions to some of the problems of classical geometry discussed
before. In the second part of the course we will discuss the basics
of Galois theory and make connections with group theory. In
particular we will study the fundamental theorem of Galois theory
will will make precise connections between subfields of a Galois
extension and subgroups of the Galois group. This connection
between field theory and group theory will allow us to understand
the non-solvability of the quintic equation using radicals (loosely
speaking this means there is no formula for the roots in terms of
coefficients and square roots, cube roots etc).
Introduction to MA-414
References:
1. Micheal Artin - Algebra
2. Dummit and Foote - Abstract Algebra
3. Serge Lang - Algebra
4. Emil Artin - Galois theory
5. J.A Gallian - Contemporary Abstract Algebra

Introduction to MA-414