Lecture 5

Null space of 𝐴𝐴 and finding all

solutions of 𝐴𝐴𝐴𝐴 = 𝑏𝑏

Kang ZHOU

1
 Overview of linear algebraic equations
Find all the
END
solutions
No
Is there only
one solution?
Yes
Yes
END
Is there any solution?
Using null space
No
Using column space How to find the
closest
'solution'?
END

2
 What is Null space?
The null space of a matrix 𝐴𝐴 is the vector space made up of all
solutions to 𝐴𝐴𝐴𝐴 = 0. It can be written as 𝑁𝑁(𝐴𝐴).
Vector space is a collection of vectors that meet the following criteria:
1. it must contain origin (zero vector)
2. linear combination of its vectors must be a vector
that is still in the vector space

Linear combination of vectors of 𝑁𝑁(𝐴𝐴) is still in 𝑁𝑁(𝐴𝐴)

𝑝𝑝 is a solution to 𝐴𝐴𝐴𝐴 = 0, i.e. 𝐴𝐴𝐴𝐴 = 0, 𝑝𝑝 ∈ 𝑁𝑁(𝐴𝐴)
𝑞𝑞 is a solution to 𝐴𝐴𝐴𝐴 = 0, i.e. 𝐴𝐴𝐴𝐴 = 0, 𝑞𝑞 ∈ 𝑁𝑁(𝐴𝐴)
Then, α𝑝𝑝 + 𝛽𝛽𝑞𝑞 is also a solution to 𝐴𝐴𝐴𝐴 = 0, i.e. 𝐴𝐴(𝛼𝛼𝑝𝑝 + 𝛽𝛽𝑞𝑞) = 0
Because 𝐴𝐴 𝛼𝛼𝑝𝑝 + 𝛽𝛽𝑞𝑞 = 𝛼𝛼𝐴𝐴𝐴𝐴 + 𝛽𝛽𝐴𝐴𝐴𝐴 = 𝛼𝛼 × 0 + 𝛽𝛽 × 0 = 0
3
 Zero vector is always a solution to 𝐴𝐴𝐴𝐴 = 0
𝑥𝑥1
𝐴𝐴𝐴𝐴 = 0 𝐴𝐴 = 𝑣𝑣1 𝑣𝑣2 𝑣𝑣3 𝑣𝑣1 𝑣𝑣2 𝑣𝑣3 𝑥𝑥2 = 0
𝑥𝑥3
0 0 0 0
0 0 0 0
𝑣𝑣1 × 0 + 𝑣𝑣2 × 0 + 𝑣𝑣3 × 0 = + + =
⋮ ⋮ ⋮ ⋮
0 0 0 0

Therefore, zero vector is always a solution to 𝐴𝐴𝐴𝐴 = 0. However, this solution is not very
useful, so it is called trivial solution.

4
 Why are we interested in Null space?

We are interested in it because it helps us find all the solutions

to 𝐴𝐴𝐴𝐴 = 𝑏𝑏 when there are infinitely many solutions. Why?

𝑝𝑝 is a solution to 𝐴𝐴𝐴𝐴 = 𝑏𝑏, i.e. 𝐴𝐴𝐴𝐴 = 𝑏𝑏

𝑞𝑞 is a solution to 𝐴𝐴𝐴𝐴 = 0, i.e. 𝐴𝐴𝐴𝐴 = 0, 𝑞𝑞 ∈ 𝑁𝑁(𝐴𝐴)
Then, 𝑝𝑝 + 𝑞𝑞 is also a solution to 𝐴𝐴𝐴𝐴 = 𝑏𝑏, i.e. 𝐴𝐴(𝑝𝑝 + 𝑞𝑞) = 𝑏𝑏
Because 𝐴𝐴 𝑝𝑝 + 𝑞𝑞 = 𝐴𝐴𝐴𝐴 + 𝐴𝐴𝐴𝐴 = 𝑏𝑏 + 0 = 𝑏𝑏
All the solutions to 𝐴𝐴𝐴𝐴 = 𝑏𝑏 can be expressed as 𝑝𝑝 + 𝑁𝑁(𝐴𝐴)

5
 How to find all the vectors in 𝑁𝑁(𝐴𝐴) ?
Formula Examples Conditions
1 0 0 2 5
𝐼𝐼 𝐹𝐹 𝑟𝑟 = 3 0 1 0 3 6
𝑟𝑟 < 𝑚𝑚, 𝑟𝑟 < 𝑛𝑛
0 0 1 4 7
0 0 𝑚𝑚 = 4, 𝑛𝑛 = 5 0 0 0 0 0

1 0 0 2 5
row 𝐼𝐼 𝐹𝐹 𝑟𝑟 = 3 0 1 0 3 6 𝑟𝑟 = 𝑚𝑚 < 𝑛𝑛
𝑚𝑚 = 3, 𝑛𝑛 = 5 0 0 1 4 7
elimination*
𝐴𝐴 𝑅𝑅
𝑚𝑚 × 𝑛𝑛 1 0 0
𝐼𝐼 𝑟𝑟 = 3 0 1 0
𝑟𝑟 = 𝑛𝑛 < 𝑚𝑚
0 0 1
[m, n] = size(A);
0 𝑚𝑚 = 4, 𝑛𝑛 = 3 0 0 0
r = rank(A);
r is the same to # of 1 0 0
independent columns of A
𝐼𝐼 𝑟𝑟 = 3 0 1 0 𝑟𝑟 = 𝑛𝑛 = 𝑚𝑚
𝑚𝑚 = 3, 𝑛𝑛 = 3 0 0 1 6
* exchange of columns may be needed
 How to find all the vectors in 𝑁𝑁(𝐴𝐴) ?
Formula Examples Solution formula Examples
1 0 0 2 5
𝐼𝐼 𝐹𝐹 0 1 0 3 6
0 0 1 4 7
0 0 2 5
0 0 0 0 0 𝐹𝐹 3 6 𝛼𝛼
𝑤𝑤 4 7 𝛽𝛽
1 0 0 2 5
−𝐼𝐼 −1 0
row 0 −1
𝐼𝐼 𝐹𝐹 0 1 0 3 6
elimination* 0 0 1 4 7 𝑤𝑤 𝑖𝑖𝑖𝑖 𝑎𝑎 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣 𝛼𝛼, 𝛽𝛽 𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏 𝑎𝑎𝑎𝑎𝑎𝑎 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛
𝐴𝐴 𝑅𝑅
1 0 0
𝐼𝐼 0 1 0
0 0 1
0 0 0 0 The only 0
solution is zero 0
1 0 0 vector, a trivial 0
𝐼𝐼 0 1 0 solution
0 0 1 7
* exchange of columns may be needed
 How to derive the solution formula? (𝑟𝑟 = 𝑛𝑛)
𝑟𝑟 = 𝑛𝑛 means all the columns of 𝑅𝑅 are independent, which also
means all the columns of 𝐴𝐴 are independent because row
elimination do not affect relationship of the columns.

1 0 0
row 𝐼𝐼 0 1 0
The only
elimination 0 0 0 1 0
0 0 0 solution is zero
𝐴𝐴 𝑅𝑅 vector, a trivial
0
1 0 0 solution 0
𝐼𝐼 0 1 0
0 0 1

8
 How to derive the solution formula? (𝑟𝑟 = 𝑛𝑛)
𝐴𝐴 = 𝑣𝑣1 𝑣𝑣2 𝑣𝑣3 𝑟𝑟 = 𝑛𝑛 𝑣𝑣1, 𝑣𝑣2, 𝑣𝑣3 are independent
𝛾𝛾 ≠ 0
𝑣𝑣1 ≠ 𝛼𝛼𝑣𝑣2 + 𝛽𝛽𝑣𝑣3 𝛾𝛾𝑣𝑣1 ≠ 𝛾𝛾𝛼𝛼𝑣𝑣2 + 𝛾𝛾𝛽𝛽𝑣𝑣3
𝛼𝛼, 𝛽𝛽 can be any real number 𝛾𝛾 can be any real number except zero

𝛾𝛾𝑣𝑣1 − 𝛾𝛾𝛼𝛼𝑣𝑣2 − 𝛾𝛾𝛽𝛽𝑣𝑣3 ≠ 0

𝛾𝛾
𝑣𝑣1 𝑣𝑣2 𝑣𝑣3 −𝛾𝛾𝛼𝛼 ≠ 0 𝐴𝐴𝐴𝐴 ≠ 0
−𝛾𝛾𝛽𝛽 𝑥𝑥 can be any 3x1 vector except 3x1 zero vector

There is no solution to 𝐴𝐴𝐴𝐴 = 0 except zero vector

9
 How to derive the solution formula? (𝑟𝑟 < 𝑛𝑛)
Formula Examples Solution formula Examples
1 0 0 2 5
𝐼𝐼 𝐹𝐹 0 1 0 3 6
row 0 0 1 4 7
elimination
0 0 2 5
0 0 0 0 0 𝐹𝐹 3 6 𝛼𝛼
𝐴𝐴 𝑅𝑅 𝑤𝑤 4 7 𝛽𝛽
1 0 0 2 5
−𝐼𝐼 −1 0
0 −1
𝐼𝐼 𝐹𝐹 0 1 0 3 6
0 0 1 4 7 𝑤𝑤 𝑖𝑖𝑖𝑖 𝑎𝑎 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣 𝛼𝛼, 𝛽𝛽 𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏 𝑎𝑎𝑎𝑎𝑎𝑎 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛

10
 How to derive the solution formula? (𝑟𝑟 < 𝑚𝑚, 𝑟𝑟 < 𝑛𝑛)
𝐼𝐼 𝐹𝐹 𝐹𝐹
𝐴𝐴𝐴𝐴 = 0 𝑁𝑁 𝐴𝐴 = 𝑁𝑁(𝑅𝑅) 𝑅𝑅𝑅𝑅 = 0 𝑅𝑅 = 𝑥𝑥 = 𝑤𝑤
0 0 −𝐼𝐼
𝑟𝑟 𝑛𝑛 − 𝑟𝑟 𝑛𝑛 − 𝑟𝑟
1
𝑟𝑟 𝐼𝐼 𝐹𝐹 𝐹𝐹 𝑟𝑟
𝑤𝑤 𝑛𝑛 − 𝑟𝑟
𝑚𝑚 − 𝑟𝑟 0 0 −𝐼𝐼 𝑛𝑛 − 𝑟𝑟

𝑟𝑟 𝑛𝑛 − 𝑟𝑟 𝑛𝑛 − 𝑟𝑟 𝑛𝑛 − 𝑟𝑟 𝑛𝑛 − 𝑟𝑟
𝐼𝐼 𝐹𝐹 𝐹𝐹 𝐼𝐼 𝑟𝑟 𝐹𝐹 𝐹𝐹 −𝐹𝐹 0
= 𝐹𝐹 + (−𝐼𝐼) 𝑛𝑛 − 𝑟𝑟 = + = 𝑚𝑚
0 0 −𝐼𝐼 0 0 0 0 0
𝑛𝑛 − 𝑟𝑟 1
𝐼𝐼 𝐹𝐹 𝐹𝐹 0 𝑛𝑛 − 𝑟𝑟 0
𝑤𝑤 = 𝑤𝑤 = 𝑚𝑚
0 0 −𝐼𝐼 0 0

11
 Demonstration 1
2𝑥𝑥1 + 3𝑥𝑥2 = 5 2 3 𝑥𝑥 5 2 3 5 1 0 1
�5𝑥𝑥1 + 4𝑥𝑥2 = 9 5 4 𝑥𝑥1 = 9 5 4 9 0 1 1
7𝑥𝑥1 + 2𝑥𝑥2 = 9 7 2 2 9 7 2 9 0 0 0
𝐴𝐴𝐴𝐴 = 𝑏𝑏 One solution to 𝐴𝐴𝐴𝐴 = 𝑏𝑏 is found
To check if there is any other solution, we need to find 𝑁𝑁(𝐴𝐴)
2 3 1 0 𝑟𝑟 = 𝑛𝑛 = 2, the only solution to 𝐴𝐴𝐴𝐴 = 0 is zero vector,
5 4 → 0 1 so the solution to 𝐴𝐴𝐴𝐴 = 𝑏𝑏 is unique.
7 2 0 0

12
 Demonstration 2
𝑥𝑥1
2𝑥𝑥1 + 3𝑥𝑥2 + 8𝑥𝑥3 + 11𝑥𝑥4 = 5 2 3 8 11 𝑥𝑥 5 2 3 8 11 5
�5𝑥𝑥1 + 4𝑥𝑥2 + 13𝑥𝑥3 + 17𝑥𝑥4 = 9 5 4 13 17 𝑥𝑥2 = 9 5 4 13 17 9
7𝑥𝑥1 + 2𝑥𝑥2 + 11𝑥𝑥3 + 13𝑥𝑥4 = 9 7 2 11 13 𝑥𝑥3 9 7 2 11 13 9
4
1
1 0 1 1 1 1 0 1 1 1 1
0 1 2 3 1 0 𝑥𝑥1 + 1 𝑥𝑥2 + 2 𝑥𝑥3 + 3 𝑥𝑥4 = 1 𝑥𝑥 =
0
0 0 0 0 0 0 0 0 0 0 0
One solution to 𝐴𝐴𝐴𝐴 = 𝑏𝑏 is found
To check if there is any other solution, we need to find 𝑁𝑁(𝐴𝐴)

2 3 8 11 1 0 1 1 𝑟𝑟 = 2, 𝑛𝑛 = 4, there are infinitely many solutions to

5 4 13 17 0 1 2 3 𝐴𝐴𝐴𝐴 = 0 and 𝐴𝐴𝐴𝐴 = 𝑏𝑏
7 2 11 13 0 0 0 0

13
 Demonstration 2
2 3 8 11 1 0 1 1 𝑟𝑟 = 2, 𝑛𝑛 = 4, there are infinitely many solutions to
5 4 13 17 0 1 2 3 𝐴𝐴𝐴𝐴 = 0 and 𝐴𝐴𝐴𝐴 = 𝑏𝑏
7 2 11 13 0 0 0 0

We can use the formula All the solutions to 𝐴𝐴𝐴𝐴 = 𝑏𝑏

1 1 1 1 1 1 1
𝐹𝐹 2 3 2 3 𝛼𝛼 1 2 3 𝛼𝛼
𝑁𝑁 𝐴𝐴 = 𝑤𝑤 = 𝑤𝑤 = 𝑝𝑝 + 𝑁𝑁 𝐴𝐴 = +
−𝐼𝐼 −1 0 −1 0 𝛽𝛽 0 −1 0 𝛽𝛽
0 −1 0 −1 0 0 −1
𝛼𝛼, 𝛽𝛽 𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏 𝑎𝑎𝑎𝑎𝑎𝑎 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝛼𝛼, 𝛽𝛽 𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏 𝑎𝑎𝑎𝑎𝑎𝑎 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛

14
 Demonstration 2
All the solutions to 𝐴𝐴𝐴𝐴 = 𝑏𝑏
𝑥𝑥1
2 3 8 11 𝑥𝑥 5 1 1 1
5 4 13 17 𝑥𝑥2 = 9 1 2 3 𝛼𝛼
𝑝𝑝 + 𝑁𝑁 𝐴𝐴 = +
7 2 11 13 𝑥𝑥3 9 0 −1 0 𝛽𝛽
4 0 0 −1
𝐴𝐴𝐴𝐴 = 𝑏𝑏 𝛼𝛼, 𝛽𝛽 𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏 𝑎𝑎𝑎𝑎𝑎𝑎 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛

Let's find four different solutions to 𝐴𝐴𝐴𝐴 = 𝑏𝑏

1 2
5 5
1 3
𝛼𝛼 = 0, 𝛽𝛽 = 0, 𝑥𝑥 = , 𝐴𝐴𝐴𝐴 = 9 𝛼𝛼 = 1, 𝛽𝛽 = 0, 𝑥𝑥 = , 𝐴𝐴𝐴𝐴 = 9
0 −1
9 9
0 0
2 3
5 5
4 6
𝛼𝛼 = 0, 𝛽𝛽 = 1, 𝑥𝑥 = , 𝐴𝐴𝐴𝐴 = 9 𝛼𝛼 = 1, 𝛽𝛽 = 1, 𝑥𝑥 = , 𝐴𝐴𝐴𝐴 = 9
0 −1
9 9 15
−1 −1
 Demonstration 2
2 3 8 11 1 0 1 1 𝑟𝑟 = 2, 𝑛𝑛 = 4, there are infinitely many solutions to
5 4 13 17 0 1 2 3 𝐴𝐴𝐴𝐴 = 0 and 𝐴𝐴𝐴𝐴 = 𝑏𝑏
7 2 11 13 0 0 0 0

We can also understand the formula for 𝑁𝑁(𝐴𝐴) this way:

𝑥𝑥1
1 0 1 1 𝑥𝑥 0 1 0 1 1 0
0 1 2 3 𝑥𝑥2 = 0 0 𝑥𝑥1 + 1 𝑥𝑥2 + 2 𝑥𝑥3 + 3 𝑥𝑥4 = 0
0 0 0 0 𝑥𝑥3 0 0 0 0 0 0
4
1 0 1 1 0
0 1 + 1 2 + 2 (−1) + 3 0 = 0 1 1 1 1
0 0 0 0 0 2 3 2 3 𝛼𝛼
𝑁𝑁 𝐴𝐴 = 𝛼𝛼 + 𝛽𝛽 =
1 0 1 1 0 −1 0 −1 0 𝛽𝛽
0 1 + 1 3 + 2 0 + 3 (−1) = 0 0 −1 0 −1
0 0 0 0 0 𝛼𝛼, 𝛽𝛽 𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏 𝑎𝑎𝑎𝑎𝑎𝑎 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛
16
 Workflow to solve 𝐴𝐴𝐴𝐴 = 𝑏𝑏
𝐴𝐴𝐴𝐴 = 𝑏𝑏

𝐴𝐴 𝑏𝑏 𝑝𝑝 is the only
solution to
𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅 𝐴𝐴𝐴𝐴 = 𝑏𝑏
At least one No
Is there any No Is 𝑟𝑟 < 𝑛𝑛 true?
All the solutions can be
inconsistency? solution to 𝐹𝐹
Yes described as 𝑝𝑝 + 𝑤𝑤
Yes 𝐴𝐴𝐴𝐴 = 𝑏𝑏 (𝑝𝑝) −𝐼𝐼

No solution

𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅: 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑟𝑟𝑟𝑟𝑟𝑟 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 17