Mathematical Foundations

for Data Science

BITS Pilani
Pilani|Dubai|Goa|Hyderabad
 Linear Algebraic Equations
• Solved single equations previously
f x  0
• Now consider more than one variable and more
than one equation

f 1 x1 , x 2 ,..., x n   0
f 2 x1 , x 2 ,..., x n   0

f n x1 , x 2 ,..., x n   0
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Linear Systems
• Linear equations and constant coefficients

a11 x1  a12 x2  ... a1n xn  b1

a21 x1  a22 x2  ... a2 n xn  b2

an1 x1 a n2 x2  ... ann xn  bn

• aij and bi are constants

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Row-Echelon Form(REF)
• A matrix is in row-echelon form if
it satisfies the following conditions.
1. The first nonzero number in each row
(reading from left to right) is 1.
This is called the leading entry.
2. The leading entry in each row is to the right of
the leading entry in the row immediately above
it.
3. All rows consisting entirely of zeros are at
the bottom of the matrix.
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
Reduced Row-Echelon Form(RREF)
• A matrix is in reduced row-echelon
form if it is in row-echelon form and
also satisfies the following condition.
Previous conditions 1, 2, 3 and
4. Every number above and below
each leading entry is a 0.

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Row Echelon Form
a 11 x 1  a 12 x 2  a 13 x 3  a 14 x 4  b1
a 21 x 1  a 22 x 2  a 23 x 3  a 24 x 4  b2
a 31 x 1  a 32 x 2  a 33 x 3  a 34 x 4  b3
a 41 x 1  a 42 x 2  a 43 x 3  a 44 x 4  b4
 a 11 a 12 a 13 a 14 b1 
a b2 
A b   21
a 22 a 23 a 24 
a 31 a 32 a 33 a 34 b3 
 
 a 41 a 42 a 43 a 44 b4 
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Elimination of first column

 a 11 a12 a13 a14 b1 

a b2 
 21 a 22 a23 a 24  f 21  a21 / a 11
a 31 a 32 a 33 a 34 b3  f31  a31 / a11
a 
 41 a42 a43 a44 b4  f41  a41 / a 11

a11 a12 a13 a14 b1 


 0 a22 a23 a24 b2  (2)  f 21  (1)
 0 a32 a33 a34 b3  (3)  f31  (1)
    b4 
 0 a42 a43 a44 (4)  f 41  (1)
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Elimination of second column

 a11 a12 a13 a14 b1 

 0 a22 a23 a24 b2 
 
 0 a32 a33 a34 b3  f 32  a32 / a22

 0 a42 a43
 a44 b4  f 42  a42 / a22
 a11 a12 a13 a14 b1 
 b2 
 0 a22 a23 a24 
 0 0 a33 a34 b3  (3)  f 32 (2)
 
 0  a44
0 a43  
b4  (4)  f 42  (2)
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Elimination of third column

a11 a12 a13 a14 b1

 0 a 22 a 23 a 24 b2 
 
 0 0 a33 a34 a3
  
 0 0 a 43 a 44 a4  f 43  a43 / a33
a11 a12 a13 a14 b1

0 a 22 a 23 a 24 b2  Upper triangular
  matrix
 0 0 a33 a34 b3
 0
 0 
0 a 44 b4 (4)  f 43  (3)
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Back-Substitution

a 11 a 12 a 13 a 14 b1 
 0 a a 
a 24 b   Upper triangular
 22 23 2 matrix
 0 0 a33 a34 b3
 
 0 0 0 a 
44 b4
x 4  b4 / a44
a 11 ,a22 ,a33 ,a44  0
x 3  ( b3  a 34 x 4 ) / a33
x 2  ( b2  a23 x 3  a24 x 4 ) / a22
x1  ( b1  a 12 x 2  a 13 x 3  a 14 x 4 ) / a 11
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Example for REF
1 a1 2 a1 3 a1 4 | b1 
0 1 a23 a 24 | b2 
 
0 0 1 a 3 4 | b3
0 
 0 0 1 | b4 

• The matrix in the above form is called

row echelon form (REF).
• No. of non-zero rows in echelon form is called
rank of the matrix.
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Example for RREF
 1 0 0 0 | b1 
0 1 0 0 | b 
 2
 0 0 1 0 | b3
0 
 0 0 1 | b 
4
• The matrix in the above form is called
reduced row echelon form (RREF).
• No. of non-zero rows in RREF is called rank of
the matrix.
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
• We use Row Echelon Form in Gauss
Elimination method.
• In this method we need use Back ward
substitution to solve the system.
• We use Reduced Row Echelon Form in Gauss
Jordan elimination method.
• In this method we don’t need to back-
substitute to solve the system.

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 EXERCISE
Solve the linear system by Gauss Elimination /
Gauss Jordan elimination method.
(i) 2x  y  3z  8 (ii) y  z  2
5x 2z  3 4 y  6z  12
8x  y  7z  0 x yz 2

(iii) 4 y  4z  24 (iv) x  y  z 6
3x 11y  2z  6 3 x  3 y  4z  20
6x 17 y  z  18 2 x  y  3z  13

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Solutions of Linear Systems:
Existence, Uniqueness

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Linear Systems
• Linear equations and constant coefficients

a11 x1  a12 x2  ... a1n xn  b1

a21 x1  a22 x2  ... a2 n xn  b2

an1 x1 a n2 x2  ... ann xn  bn

• aij and bi are constants

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Mathematical background
• It is convenient to write system of equations
in matrix-vector form
[A][x]  [b] or [A]{x}  {b}
 a11 a12 a1n  x1  b1 
a 
a2 n x2   b 
 21 a 22    2 
     
    
    
an1 a n2 ann xn  bn
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Graphical Method
2 x1  x 2  3 x  2x  3
 x  x  3 rearrange   x 2  3 1 x
 1 2  2 1

x1 + x 2 = 3 2x1 – x2 = 3

One solution

BITS Pilani, Deemed to be University under Section 3 of UGC A ct, 1956

 Graphical Method
No solution

2x1 – x2 = – 1

2x1 – x2 = 3

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Graphical Method
Infinite many solution

2x1 – x2 = 3 6x1 – 3x2 = 9

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

• The matrix form of system of equations is
[ A][x]  [b] -- (1)
• If b’s are all zero in equation (1) then,
(1) is called homogeneous linear system otherwise
(1) is called non-homogeneous system.
• If (1) has a solution, then the linear system is said to
be consistent otherwise the linear system is said to
be inconsistent.

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Non Homogeneous System
[ A][x]  [b]

• If rank of A=rank of [A/b], then non-

homogeneous system is consistent.
• If rank of A=rank of [A/b]=n, then the system
has unique solution. (n is the no. of unknowns)
• If rank of A=rank of [A/b]<n ,then the system
has infinite no. of solutions.
• If rank of A ≠ rank of *A/b+ , then the system
has no solution.

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 EXERCISE
• Solve the Non homogeneous linear system

(i) 2 x  y  3z  8 (ii) x  4 y  7z  14
5x  2z  3 3 x  8 y  2z  13
8 x  y  7z  0 7 x  8 y  26z  5

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Homogeneous System
[ A][x]  0

• Homogeneous system is always consistent

X=0, is called trivial solution
• If rank of A=n , then the system has unique
solution. (Trivial solution)
• If rank of A < n , then the system has infinite
no. of solutions.
• A homogeneous linear system with fewer
equations than unknowns has always non-
trivial solutions.
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 EXERCISE
• Solve the Homogeneous linear system

(i) x  y  2z  3w  0 (ii) x  y  w 0
x  2y  z  w  0 y  z 0
4 x  y  5z  8w  0 x yzw 0
5 x  7 y  2z  w  0 x  y  2z  0

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Inverse of Matrix
• The inverse of an n x n matrix A is denoted by
A1 and is an n x n matrix such that
AA1  A1 A  I
• If A has an inverse, then A is called a
non singular matrix
• If A has no inverse A is called singular matrix
• If A has inverse , then it is inverse is unique

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Gauss Jordan elimination to find
Inverse of a Matrix
 3 1 1 
1. 15 6  5
 
 5  2 2 
1 0 0 1 2 5 

2. 2 1 0 
3.0 1 2 

5 4 1 2 4 10
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
Thank you

28

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956