Mathematical Foundations for

Machine Learning
BITS Pilani MFDS Team
Pilani Campus
 BITS Pilani
Pilani Campus

Mathematical Foundations for Machine Learning

Webinar#1
 Agenda

Discussion on
• Solving system of Linear equations (Non Homogeneous)

• Traffic Flow Problem.

• Vector Space, basis , dimension, Linearly independent

and dependent Vectors.

• Eigen values and Eigen vectors,Diagonalization and

Power of Matrix by diagonalization.

• Cholesky’s Method.
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
Solving system of Linear equations (Non Homogeneous)
x + 2 y + 3z = 14
1. Test for consistency and solve 4 x + 5 y + 7 z = 35
Solution: 3x + 3 y + 4 z = 21

1 2 3 : 14 Here (n - r) = 1 and hence one of the variables can

[ A : B ] = 4 5 7 : 35 is the augmented matrix.
take arbitrary values.
3 3 4 : 21
We now have, x + 2 y + 3z = 14.......( 1 )
R 2 → R2 − 4 R1 and R 3 → R3 − 3R1 − 3 y − 5 z = −21.......( 2 )
1 2 3 : 14 

[ A : B ] ~ 0 − 3 − 5 : − 21
0 − 3 − 5 : − 21

R 3 → R3 − R2

1 2 3 : 14 

[ A : B ] ~ 0 − 3 − 5 : − 21
0 0 0 : 0 
2. Test for consistency and solve x − 4 y + 7 z = 14
3x + 8 y − 2 z = 13
7 x − 8 y + 26z = 5
Solution:
1 − 4 7 : 14
[ A : B] = 3 8 − 2 : 13 is the augmented matrix.
7 − 8 26 : 5 

R 2 → R2 − 3R1 and R 3 → R3 − 7 R1

1 − 4 7 : 14 

[ A : B] ~ 0 20 − 23 : − 29
0 20 − 23 : − 93

R 3 → R3 − R2

1 − 4 7 : 14 

[ A : B] ~ 0 20 − 23 : − 29
0 0 0 : − 64
 3. Test for consistency and solve 5 x1 + x 2 + 3 x 3 = 20
2 x1 + 5 x 2 + 2 x 3 = 18
3x1 + 2 x 2 + x 3 = 14

Solution:
5 1 3 : 20
[ A : B] = 2 5 2 : 18 is the augmented matrix. Let us now convert the prevailing form of [A : B]
3 2 1 : 14  into a set of equations as follows,
R 2 → 5R2 − 2 R1 and R 3 → 5R3 − 3R1 5 x1 + x 2 + 3 x 3 = 20.......( 1 )
23x 2 + 4 x 3 = 50.......( 2 )
5 1 3 : 20
− 120x 3 = −120.......... ..( 3 )
[ A : B ] ~ 0 23 4 : 50
0 7 4 : 10 

R 3 → 23R3 + 7 R2

5 1 3 : 20 

[ A : B ] ~ 0 23 4 : 50 
0 0 − 120 : − 120
 Traffic Flow Problem.(Non-Homogeneous system)
4. Model the following traffic flow problem would lead to a linear system
Ax=B and hence solve.

Solution:

Consider the following traffic flow diagram and construct the linear system AX=B.
Hence solve.

BITS Pilani, Pilani Campus

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
The above system will have infinitely many solution , Since rank of [A:B] and A =5 and
which is less than no of Unknowns i.e equal to 7. You can check the answers by changing
values of k and l.
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
Problem on Vector Space

BITS Pilani, Pilani Campus

BITS Pilani, Pilani Campus
Problem on Linear Independent and Linear dependent Vectors

Q7. Show that the vectors (1, 1, 3), (1,-2, 6) and (-1, 1,-5) are linearly dependent.
Also find the relation between them.

BITS Pilani, Pilani Campus

BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
Q8. Show that the vectors (2,2,1), (1, -1,1) and (1,0,1) in R3 are linearly independent.

BITS Pilani, Pilani Campus

BITS Pilani, Pilani Campus
 Problems on Basis and Dimension

 1 −2 −1 3 2
 2 −2 −3 6 1 
Q6. Compute four fundamental vector spaces associated with matrix A, where 
 −1 −4 4 −3 7 

BITS Pilani, Pilani Campus

BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
 Problems
Problems on Eigen Values, Eigen Vectors, Diagonalization and Power of Matrix
 −1 3 
91.. Reduce the matrix A = 
4
 to the diagonal form and hence find A .
Solution:  −2 4 
3 1
P = X 1 X2=  
The characteristic equation of A is A − I =0 Modal matrix: 2 1

(−1 −  )
1
3 We have P = 1 and P −1 = ( AdjP)
=0 P
−2 4−
 1 − 1
(−1 − )(4 − ) + 6 = 0 P −1 =  
− 2 3 
2 − 3 + 2 = 0
 1 − 1  − 1 3 3 1 1 − 1 3 1 1 0 
𝑁𝑜𝑤 P −1 AP =  = − 4 2 1 =
3  0 2 
  = 1, 2 are the eigen values of A .     
− 2 3  − 2 4 2 1     
Now consider A − I  [ X ] = [0]
(−1 − ) 3   x  0  Thus P −1 AP = D
 −2 =
 (4 − )  y  0 1 0 
=  is the diagonal matrix.
 0 2 
(−1 − ) x + 3 y = 0
− 2 x + (4 − ) y = 0 P −1 AP = Diag(1 2)

Case-(i): Let  = 1, Now to find out A4, consider the following

x y we have An = PD n P −1
We get − 2 x + 3 y = 0 or 2 x = 3 y or = .
3 2
−1
14 0  1 0 
 X 1 = (3 2)' is the eigen vector corresponding to  = 1 . A = PD P
4 4
where D =  4
= 
0 2 4  0 16
Case-(ii): Let  = 2 , 3 1 1 0   1 − 1 − 29 45
A4 =   0 16 − 2 3  =  − 30 46
x y  2 1    
We get − 3x + 3 y = 0 or x = y =
or
1 1 − 29 45
 X 2 = (1 1)' is the eigen vector corresponding to  = 2 . Thus A 4 =  
 − 30 46 .
  − 2 2 − 3
10. Reduce the given matrix in diagonal form A =  2 1 − 6
 − 1 − 2 0 

Solution: The Characteristic equation of A is A − I = 0 Case (i): Let  = −3 and the corresponding equations are
x + 2 y − 3z = 0
(−2 − ) 2 −3
2x + 4 y − 6z = 0
2 (1 − ) −6 = 0
− x − 2 y + 3z = 0 .
−1 −2 ( − )
It should be observed that the equations are all same
⇒ (−2 − )[−(1 − ) − 12] − 2[−2 − 6] − 3[−4 + 1 − ] = 0
and we have only one independent equation x + 2 y − 3 z = 0
⇒ (−2 − )[− + 2 − 12) + (4 + 12) + (9 + 3) = 0 (In case the rule of cross multiplication is applied,
⇒ (−2 −  )( + 3)( − 4) + 4( + 3) + 3( + 3) = 0 we get x = y = z = 0 which is a trivial solution.)
⇒ ( + 3)(−2 −  )( − 4) + 4 + 3 = 0
⇒ ( + 3)(− 2 + 2 + 15) = 0
Two variables can be arbitrary.
⇒ ( + 3)( + 3) (  − 5) = 0 Let z = k1 , y = k 2  x = 3k1 − 2k 2
 = −3 −3 5.
The eigen vector corresponding to the coincident
eigen value  = −3 be denoted by
We now form the system of equations. X 1, 2 and we have X 1, 2 = (3k1 − 2K 2 k2 k1 )'
(−2 − ) + 2 y − 3z = 0
where k1 , k2 are arbitray. We choose convenient
2 x + (1 −  ) y − 6 z = 0
values for k1 and k2 to obtain two distinct eigen vectors.
− 1x − 2 y − z = 0 .
(i) Let k1 = 1, k 2 = 1  X 1 = (1 1 1)'

(ii) Let k1 = 1, k 2 = 0  X 2 = (3 0 1)'

 Case (ii): Let  = 5 and the corresponding equations are
− 7 x + 2 y − 3z = 0 …… (1)
2x − 4 y − 6z = 0 ……. (2)
− x − 2 y − 5z = 0 Diagonalization of A is given by P −1 AP ,
x −y z
Solving (1) and (2) , = = − 2 4 6 − 2 2 − 3 1 3 1 
− 12 − 12 42 + 6 28 − 4
=  3 − 2 − 1
1 2
 1 − 6 1 0 2 
x −y z x y z 8
= = or = =  1 2 − 3  − 1 − 2 0  1 1 −1
− 24 48 24 1 2 −1
X 3 = (1 2 − 1) ' is the eigen vector corresponding to  = 5 .  6 − 12 − 18 1 3 1 
1
= − 9 6 3  1 0 2 
8
We have modal matrix  5 10 − 15 1 1 −1

1 3 1 
− 24 − 0 0 
P = [X1 X 3 ] = 1 0 2  1
− 24 0 
X2
=  0
1 1 − 1 8
 0 0 40

P = 1(−2) − 3(−3) + 1(1) = 8

.  − 3 0 0
=  0 − 3 0 = D
 + (0 − 2) − (−3 − 1) + (6 − 0) − 2 4 6
 0 0 5
AdjP = − (−1 − 2) + (−1 − 1) − (2 − 1)  =  3 − 2 − 1
  
 + (1 − 0) − (1 − 3) + (0 − 3)   1 2 − 3
Thus P −1 AP = D = Diag(− 3 − 3 5) .
Problems on Cholesky’s Method

BITS Pilani, Pilani Campus

BITS Pilani, Pilani Campus
11.

BITS Pilani, Pilani Campus

BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
Thank You

BITS Pilani, Pilani Campus