LINEAR ALGEBRA, NUMERICAL AND COMPLEX

ANALYSIS
MA11004
SECTION 1 and 2
Dr. Jitendra Kumar

Professor
Department of Mathematics
Indian Institute of Technology Kharagpur
West Bengal 721302, India

Webpage: http://www.facweb.iitkgp.ac.in/~jkumar/
Linear Algebra

 System of Linear Equation - Introduction

 Solution – Geometrical Interpretation

 Solution of the System - Gauss Elimination

 Consistency of Solution

Dr. Jitendra Kumar – IIT KHARAGPUR 2

 System of Linear Equations 𝑎11 𝑥1 + 𝑎12 𝑥2 + ⋯ + 𝑎1𝑛 𝑥𝑛 = 𝑏1
𝑎21 𝑥1 + 𝑎22 𝑥2 + ⋯ + 𝑎2𝑛 𝑥𝑛 = 𝑏2
Matrix form: 𝑎11 𝑎12 … 𝑎1𝑛
⋮
𝑎21 𝑎22 … 𝑎2𝑛 𝑎𝑚1 𝑥1 + 𝑎𝑚2 𝑥2 + ⋯ + 𝑎𝑚𝑛 𝑥𝑛 = 𝑏𝑚
𝐴𝑥 =𝑏 𝐴= ⋮ ⋮ ⋱ ⋮
𝑎𝑚1 𝑎𝑚2 ⋯ 𝑎𝑚𝑛
𝑚 equations and 𝑛 unknowns
𝑥1 𝑏1
𝑥2 𝑏2
𝑥= ⋮ 𝑏=
⋮
𝑥𝑛 𝑏𝑚

A system of equation is consistent if it has at least one solution, and

inconsistent if it has no solution.

Dr. Jitendra Kumar – IIT KHARAGPUR 3

System of Linear Equations 𝑦

Consider 𝑥 + 2𝑦 = 4 (𝐿1 )
𝑥−𝑦 =1 (𝐿2 ) (2,1)

OR
1 2 𝑥 4 𝑥
=
1 −1 𝑦 1

Case of Unique Solution

Dr. Jitendra Kumar – IIT KHARAGPUR 4

System of Linear Equations (vectors interpretation)
𝑦
𝑥 + 2𝑦 = 4 𝑥−𝑦 =1

OR

1 2 𝑥 4 1
4
= 1
1 −1 𝑦 1 1

OR 𝑥
1
2 2
1
1 2 4 −1
𝑥 +𝑦 =
1 −1 1

4
The vector can only be produced by ONE linear combination of
1
col-1 & col-2. Hence this is the case of unique solution.
Dr. Jitendra Kumar – IIT KHARAGPUR 5
System of Linear Equations 𝑦

Consider 𝑥−𝑦 =1 (𝐿1 )

−𝑥 + 𝑦 = 2 (𝐿2 )

Lines do not intersect.

Case of No Solution

Dr. Jitendra Kumar – IIT KHARAGPUR 6

 𝑦
System of Linear Equations (vectors interpretation)

Consider 𝑥−𝑦 =1 −𝑥 + 𝑦 = 2
1
2

−1
1 −1 1 1
𝑥 +𝑦 = 𝑥
−1 1 2
1
−1
1
It is not possible to produce by any linear combination
2
of col-1 & col-2. Hence this is the case of no solution.

Dr. Jitendra Kumar – IIT KHARAGPUR 7

System of Linear Equations 𝑦

Consider 𝑥−𝑦 =2 (𝐿1 )

−𝑥 + 𝑦 = −2 (𝐿2 )

𝑥
Both are the same equation. Any point on
the line is a solution of the given system

Case of Infinitely Many Solutions

Dr. Jitendra Kumar – IIT KHARAGPUR 8

System of Linear Equations (vectors interpretation) 𝑦

𝑥−𝑦 =2 −𝑥 + 𝑦 = −2
−1
1

1 −1 2 𝑥
𝑥 +𝑦 = 1
−1 1 −2
−1

2
−2

2
The vector can be produced by many linear combinations of
−2
col-1 & col-2. Hence this is the case of infinitely many solution.

Dr. Jitendra Kumar – IIT KHARAGPUR 9

Summary:

System of Linear Equations

• Unique Solution

• Infinitely Many Solutions

• No Solution

Dr. Jitendra Kumar – IIT KHARAGPUR 10

System of Linear Equations: Solution Methods

• Method of Determinants: Cramer’s rule

direct method
• Matrix Inversion Method: 𝐴𝑥 = 𝑏 ⇒ 𝑥 = 𝐴−1 𝑏
(exact solution)

• Gauss Elimination Method

approximate
• Iterative Method – Jacobi & Gauss-Seidel method
solution

Dr. Jitendra Kumar – IIT KHARAGPUR 11

System of Linear Equations: Gauss Elimination Method

Elementary Row Operations

• Interchange of 𝑖-th and 𝑗-th rows 𝑅𝑖 ↔ 𝑅𝑗

• Multiplication of the 𝑖-th row by a nonzero number 𝜆 𝑅𝑖 ← 𝜆𝑅𝑖

• Addition of 𝜆 times the 𝑗-th row to the 𝑖-th row 𝑅𝑖 ← 𝑅𝑖 + 𝜆𝑅𝑗

Dr. Jitendra Kumar – IIT KHARAGPUR 12

Gauss Elimination Method

Gauss Elimination: Example - 1

Augmented Matrix
𝑥1 + 𝑥2 + 𝑥3 = 4 1 1 1 4
2𝑥1 + 3𝑥2 + 𝑥3 = 7 𝐴𝑏 = 2 3 1 7
𝑥1 + 2𝑥2 + 3𝑥3 = 9 1 2 3 9
𝑅2 ← 𝑅2 − 2𝑅1
𝑅3 ← 𝑅3 − 𝑅1
𝑥1 + 𝑥2 + 𝑥3 = 4
1 1 1 4
𝑥2 − 𝑥3 = −1 ~ 0 1 −1 −1
𝑥2 + 2𝑥3 = 5 0 1 2 5
𝑅3 ← 𝑅3 − 𝑅2
𝑥1 + 𝑥2 + 𝑥3 = 4 1 1 1 4
𝑥2 − 𝑥3 = −1 ~ 0 1 −1 −1
3𝑥3 = 6 0 0 3 6

Dr. Jitendra Kumar – IIT KHARAGPUR 13

Gauss Elimination Method
Gauss Elimination: Example - 1

𝑥1 + 𝑥2 + 𝑥3 = 4 1 1 1 4
𝑥2 − 𝑥3 = −1 𝐴𝑏 ~ 0 1 −1 −1 Echelon form
3𝑥3 = 6 0 0 3 6

Solution: Back substitution:

𝑥3 = 2 𝑥3 = 2
𝑥2 = −1 + 2 = 1 𝑥2 = −1 + 𝑥3 = 1
𝑥1 = 4 − 1 − 2 = 1 𝑥1 = 4 − 𝑥2 − 𝑥3 = 1

Number of Pivots = Number of Unknowns ⟹ Unique Solution

OR every column has a pivot
Dr. Jitendra Kumar – IIT KHARAGPUR 14
Gauss Elimination Method

Gauss Elimination: Example - 2

1 1 1 4
𝐴𝑏 = 2 3 1 7
1 2 0 9 𝑅2 ← 𝑅2 − 2𝑅1
𝑅3 ← 𝑅3 − 𝑅1
Equations are inconsistent and
1 1 1 4
hence the solution does not exist.
~ 0 1 −1 −1
0 1 −1 5
𝑅3 ← 𝑅3 − 𝑅2

1 1 1 4
~ 0 1 −1 −1 right-most column has a pivot
0 0 0 6 ⟹ No Solution

Dr. Jitendra Kumar – IIT KHARAGPUR 15

Gauss Elimination Method

Gauss Elimination: Example - 3

number of pivots (𝑟)
1 1 1 4
𝐴𝑏 = 2 3 1 7 𝑅2 ← 𝑅2 − 2𝑅1 < numer of unknowns (𝑛)
1 2 0 3 𝑅3 ← 𝑅3 − 𝑅1

number of free variable = (𝑛 − 𝑟)

1 1 1 4
~ 0 1 −1 −1
0 1 −1 −1 𝑅3 ← 𝑅3 − 𝑅2

1 1 1 4
~ 0 1 −1 −1 Equations are Consistent
0 0 0 0
𝑥3 Free variable
Dr. Jitendra Kumar – IIT KHARAGPUR 16
Gauss Elimination Method

1 1 1 4 𝑥1 + 𝑥2 + 𝑥3 = 4
~ 0 1 −1 −1 𝑥3 Free variable 2𝑥1 + 3𝑥2 + 𝑥3 = 7
0 0 0 0 𝑥1 + 2𝑥2 =3

Choose 𝑥3 = 𝛼 𝑥1 5 −2
𝑥2 = −1 + 𝛼 𝑥2 = −1 + 𝛼 1
𝑥1 = 5 − 2𝛼 𝑥3 0 1
particular solution of
solution 𝐴𝑥 = 0
(Null Space)

𝑥 = 𝑥𝑝 + 𝑥ℎ

Dr. Jitendra Kumar – IIT KHARAGPUR 17

Solution of System of Linear Equations: 𝐴𝑚×𝑛 𝑥𝑛×1 = 𝑏𝑚×1

Echelon Form:
⊠ ∗ ∗ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 ⊠ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 0 ⊠ ∗ ⋯ ⋯ ⋯ ∗ | ∗ pivot rows
0 0 0 0 ⋮ ⋮ ⋮ ⋮ ⋮ | ∗ (𝑟)
[𝐴ሚ 𝑏෨ = ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ | ∗
0 ⋯ 0 0 0 ⊠ ∗ ⋯ ∗ | ∗
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗ zero rows
⋮ ⋮ | ⋮ (𝑚 − 𝑟)
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗

⊠ − pivot element ≠ 0 ⊗ & ∗ − other elements (may be zero)

Dr. Jitendra Kumar – IIT KHARAGPUR 18
Echelon Form

⊠ ∗ ∗ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗ Def. Rank (A) = r (number of pivots)

0 0 ⊠ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 0 ⊠ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 0 0 ⋮ ⋮ ⋮ ⋮ ⋮ | ∗
(𝑟)
𝐴𝑏~ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ | ∗
0 ⋯ 0 0 0 ⊠ ∗ ⋯ ∗ | ∗
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗
⋮ ⋮ | ⋮ (𝑚 − 𝑟)
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗

 If ⊗≠ 0 the equations become inconsistent and hence the system has no solution

OR in terms of rank: Rank (𝐴) ≠ Rank 𝐴 𝑏

Dr. Jitendra Kumar – IIT KHARAGPUR 19

Echelon Form

⊠ ∗ ∗ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗ Def. Rank (A) = r (number of pivots)

0 0 ⊠ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 0 ⊠ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 0 0 ⋮ ⋮ ⋮ ⋮ ⋮ | ∗
(𝑟) If ⊗= 0
𝐴𝑏~ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ | ∗ The system will be consistent
0 ⋯ 0 0 0 ⊠ ∗ ⋯ ∗ | ∗
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗ (𝐴𝑥 = 0 is always consistent)
⋮ ⋮ | ⋮ (𝑚 − 𝑟)
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗

 If ⊗= 0 and number of pivot elements (𝑟) = number of unknowns (𝑛)

OR each column has a pivot Then the system has a unique solution

OR in terms of rank: Rank 𝐴 = Rank 𝐴 𝑏 =𝑛

Dr. Jitendra Kumar – IIT KHARAGPUR 20

Echelon Form
⊠ ∗ ∗ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗ Def. Rank (A) = r (number of pivots)
0 0 ⊠ ∗ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 0 ⊠ ∗ ⋯ ⋯ ⋯ ∗ | ∗
0 0 0 0 ⋮ ⋮ ⋮ ⋮ ⋮ | ∗
(𝑟)
𝐴𝑏~ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ | ∗
0 ⋯ 0 0 0 ⊠ ∗ ⋯ ∗ | ∗
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗
⋮ ⋮ | ⋮ (𝑚 − 𝑟)
0 ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 0 | ⊗

 If ⊗= 0 and number of pivot elements (𝑟) < number of unknowns (𝑛)

Then the system has infinitely many solutions

OR in terms of rank: Rank 𝐴 = Rank 𝐴 𝑏 <𝑛

Dr. Jitendra Kumar – IIT KHARAGPUR 21
Gauss Elimination Method

Problem -1 Solve the system of equations 𝐴𝑥 = 𝑏 with

1 2 −2 −1 1 1
2 4 −4 0 3 2
𝐴𝑏 = ; 𝛽∈ℝ
−1 −2 3 3 4 3
3 6 −7 1 1 𝛽

𝑅2 → 𝑅2 − 2𝑅1 𝑅3 → 𝑅3 + 𝑅1 𝑅4 → 𝑅4 − 3𝑅1

1 2 −2 −1 1 1
0 0 0 2 1 0
𝐴𝑏 =
0 0 1 2 5 4
0 0 −1 4 −2 𝛽−3

Dr. Jitendra Kumar – IIT KHARAGPUR 22

Gauss Elimination Method

1 2 −2 −1 1 1 𝑅4 → 𝑅4 − 3𝑅3
0 0 0 2 1 0
𝐴𝑏 =
0 0 1 2 5 4
𝛽−3 1 2 −2 −1 1 1
0 0 −1 4 −2
0 0 1 2 5 4
𝑅2 ↔ 𝑅3 𝐴𝑏 =
0 0 0 2 1 0
1 2 −2 −1 1 1 0 0 0 0 0 𝛽+1
0 0 1 2 5 4
𝐴𝑏 =
0 0 0 2 1 0
0 0 −1 4 −2 𝛽−3
𝑅4 → 𝑅4 + 𝑅2
1 2 −2 −1 1 1
0 0 1 2 5 4
𝐴𝑏 =
0 0 0 2 1 0
0 0 0 6 3 𝛽+1

Dr. Jitendra Kumar – IIT KHARAGPUR 23

Gauss Elimination Method

1 2 −2 −1 1 1
0 0 1 2 5 4
𝐴𝑏 =
0 0 0 2 1 0
0 0 0 0 0 𝛽+1

Case – I: 𝛽 ≠ −1 ⟹ No Solution

Case – II: 𝛽 = −1 𝑥1 𝑥3 𝑥4 dependent variables

1 2 −2 −1 1 1
0 0 1 2 5 4
𝐴𝑏 =
0 0 0 2 1 0
0 0 0 0 0 0
𝑥2 𝑥5 free variables

Dr. Jitendra Kumar – IIT KHARAGPUR 24

Gauss Elimination Method

Case – II: 𝛽 = −1
Take 𝑥2 = 𝛼1 , 𝑥5 = 𝛼2 , then
dep. variables 𝑥1 𝑥3 𝑥4
𝑥4 = −12𝛼2 𝑥3 = 4 − 4𝛼2
1 2 −2 −1 1 1
0 0 1 2 5 4 𝑥1 = 9 − 2𝛼1 − 9.5𝛼2
𝐴𝑏 =
0 0 0 2 1 0
0 0 0 0 0 0
free variables 𝑥2 𝑥5

𝑥1 9 −2 −9.5
𝑥2 0 1 0
𝑥3 = 4 + 𝛼1 0 + 𝛼2 −4
𝑥4 0 0 −0.5
𝑥5 0 0 1

Dr. Jitendra Kumar – IIT KHARAGPUR 25

Gauss Elimination Method

𝑥1 9 −2 −9.5 Solution of nonhomogeneous linear system

𝑥2 0 1 0
𝑥3 = 4 + 𝛼1 are of the form 𝑥 = 𝑥𝑝 + 𝑥ℎ , where 𝑥𝑝 is
0 + 𝛼2 −4
𝑥4 0 0 −0.5 any fixed solution of 𝐴𝑥 = 𝑏 and 𝑥ℎ runs
𝑥5 0 0 1
through all the solutions corresponding to
𝑥= 𝑥𝑝 + 𝑥ℎ homogeneous system 𝐴𝑥 = 0.

𝐴𝑥𝑝 = 𝑏 𝐴𝑥ℎ = 0

Dr. Jitendra Kumar – IIT KHARAGPUR 26

Gauss Elimination Method

Remarks:

 Free variable(s) is (are) responsible for infinitely many solutions

 An invertible matrix has no free variable (𝐴𝑥 = 𝑏 ⟹ 𝑥 = 𝐴−1 𝑏) unique solution

 Vectors that generate solutions of 𝐴𝑥 = 0 are −2, 1, 0, 0, 0 𝑇 & −9.5, 0, −4, −0.5, 1 𝑇

 These generators are called BASIS of solution space of 𝐴𝑥 = 0 (NULL Space)

 NULL Space is a vector space (next lecture)

Dr. Jitendra Kumar – IIT KHARAGPUR 27

SUMMARY:

System of Linear Equations

• Gauss Elimination

• Echelon form

• Solution (Consistency & Inconsistency)

• Free variables – Infinitely many solutions

• 𝑥 = 𝑥𝑝 + 𝑥ℎ

Dr. Jitendra Kumar – IIT KHARAGPUR 28

Thank You
29