MA 106: Linear Algebra

Instructors: Prof. Akhil Ranjan and Prof. S. Krishnan

January 5, 2017 1 / 42
Text and references

Main Text:
E. Kreyszig, Advanced Engineering Mathematics, 8th ed. (Chapters 6 and
7)
Additional references:
1) Notes by Prof. I.K. Rana
2) S. Kumaresan, Linear Algebra- A geometric approach.
3) Wylie and Barrett, Advanced Engineering Mathematics, 6th ed.
(Chapter 13)

January 5, 2017 2 / 42
More course details

Syllabus: see Math Dept webpage.

Grading policy:
Study hours:
1 Lectures - 21 hours?
2 Tutorials - 7 hours
3 Independent study - 42 hours

January 5, 2017 3 / 42
Outline of Week-1

1 Matrices
2 Addition, multiplication, transposition
3 Linear transformations and matrices
4 Linear equations and Gauss’ elimination
5 Row echelon forms and elementary row matrices
6 Reduced REF
7 Gauss-Jordan method for finding inverse

January 5, 2017 4 / 42
Matrices

Definition 1
A rectangular array of numbers,real or complex, is called a matrix.

An array could be of any type of non mathematical objects too. Or more

sophisticated mathematical objects like functions instead of numbers. E.g.
 
sin x − cos x
.
cos x sin x

Most of the topics today (sec 6.1 and 6.2) will be briefly reviewed and the
details will be left for self study

January 5, 2017 5 / 42
Basic Notation

A = [ajk ], 1 ≤ j ≤ m, 1 ≤ k ≤ n

denotes an m × n matrix whose entry in j th row and the k th column is the

number ajk . (Equivalently, k th entry in the j th row or equivalently j th · · · .)

a11 · · · a1k · · · a1n

 .. .. .. 
 . . . 
 
A =  aj1 · · · ajk · · · ajn 


 . .. .. 
 .. . . 
am1 · · · amk · · · amn

January 5, 2017 6 / 42
Examples

Some special cases are sometimes named differently

Definition 2
An m × 1 matrix is referred to as a column vector while a 1 × n matrix is
referred to as a row vector .

Examples :
 
1
1  0  is a column.

−1
2 [0 1 − 1 3 0] is a row.

Two matrices are said to be equal if and only if their corresponding entries
are same.

January 5, 2017 7 / 42
Transposition

Definition 3
A matrix B is a transpose of A if the rows of B are the columns of A and
vice versa.
Thus, is A = [ajk ] is an m × n then B is n × m matrix [brs ] where
brs = asr ; 1 ≤ r ≤ n, 1 ≤ s ≤ m.

The transpose of A is unique and is denoted by AT .

 
  5 4
5 −8 1
Example: A = =⇒ AT = −8 0 .
4 0 0
1 0
T T
Exercise: Show that (A ) = A.

January 5, 2017 8 / 42
Symmetry, Addition, Scalar multiplication

Definition 4
A matrix A is called symmetric (resp. skew-symmetric) if A = AT (resp.
A = −AT ).

These are necessarily square matrices i.e.

no. of rows=no. of columns.
Let A, B be real (or complex) matrices and λ ∈ R (or C) be a scalar.

Definition 5
(Addition) If A = [ajk ] and B = [bjk ] have the same order m × n, we
define their addition to be A + B = [cjk ] = [ajk + bjk ].

Definition 6
(Scalar multiplication) The scalar multiplication of λ with A is defined as
λA = [λajk ].

January 5, 2017 9 / 42
Matrix multiplication

If A = [ajk ] is m × n and B = [bk` ] is n × p, then the product C := AB is

a well defined m × p matrix cooked by the following recipe (called row by
column multiplication):
n
X
C = [cj` ] where cj` = ajk bk` ; 1 ≤ j ≤ m, 1 ≤ ` ≤ p.
k=1

 
  b1`
 .. 





 . 

aj1 · · · ajk · · · ajn   bk` 
..
   
   
 . 
bn`

January 5, 2017 10 / 42
Associativity, Distributivity

Theorem 7 (Associativity)
If A, B, C are real (or complex) matrices such that A is m × n, B is n × p
and C is p × q, then the products AB and BC are defined and in turn the
products A(BC ) and (AB)C are also defined and the latter two are equal.

In other words: A(BC ) = (AB)C

Proof: Exercise.
Theorem 8 (Distributivity)
If B, C are real (or complex) m × n matrices, then
A(B + C ) = AB + AC if A is p × m.
(B + C )A = BA + CA if A is n × q.

Proof: Exercise.

January 5, 2017 11 / 42
Transpose of a product

Theorem 9
Let A be m × n and B be n × p, then AB and B T AT are well defined and
in fact
(AB)T = B T AT .

Proof:
Omitted.  
4 9  
3 7
Exercise: Let A = 0
 2 ,B =
 . Compute AT , B T , AB, (AB)T ,
2 8
1 6
T T
B A and A B toT T verify the claim.

January 5, 2017 12 / 42
Example: Dot product as a matrix product

Definition 10
   
v1 w1
 ..   .. 
Let v =  .  and w =  .  be column vectors of the same size n. Their
vn wn
dot product (or inner product or scalar product) is defined as
n
v qw =
X
vj wj .
j=1

It is interesting to observe that v qw = vT w as a 1 × 1 matrix, which being

symmetric also equals wT v = w qv.
Question: What about vwT ? Is it defined? Is it also the dot product?

[1.0]

January 5, 2017 13 / 42
Linear transformations and associated matrices

From now on the elements of Rn will be written as the column vectors of

length n.
Definition 11
A map f : Rn −→ R is called linear if it is of the form
f (x) = a1 x1 + a2 x2 · · · + an xn for suitable constants a1 , a2 , ..., an .

Here x1 , x2 , ..., xn are the entries of x usually called variables. If we view

A = [a1 a2 · · · an ] as a row vector, then f (x) = Ax in terms of matrix
multiplication.

More generally, an m × n matrix A can be viewed as a linear map

Rn −→ Rm via x 7→ Ax.

This viewpoint allows us to study matrices geometrically.

January 5, 2017 14 / 42
Linear transformations and associated matrices
Formal definition of a linear map (also called transformation)

Definition 12
A map f : Rn −→ Rm is called linear if it satisfies
(i) f(x + y) = f(x) + f(y) and
(ii) f(λx) = λf(x)

Remark: For an m × n matrix A, f(x) = Ax is an example of a linear map.

These are no other examples of linear maps. A is called the matrix
associated to the linear transformation f.
 
1
Example 1: Show that the range of as a linear map R −→ R2 is a
−1
line through 0.
     
1 t x(t)
A(t) = [t] = = , say. Thus x(t) = t, y (t) = −t which
−1 −t y (t)
are parametric equations of the line x + y = 0 through 0.
January 5, 2017 15 / 42
Linear transformations and associated matrices

 
1 −1
Example 2:Determine domain and range and also show that −1 2 
0 1
has a plane through 0 as its range.
2 3
 is 3 × 2,so that’s a linear transformation
(i) The matrix    R −→ R .
  1 −1   u−v x(u, v )
u u
(ii) B = −1 2  = −u + 2v  = y (u, v ), say.
v v
0 1 v z(u, v )
• Thus x = u − v , y = 2v − u, z = v are the parametric equations of a
plane through 0.
• On eliminating u, v the equation of the plane is x + y − z = 0 .

January 5, 2017 16 / 42
Linear transformations and associated matrices
 
1 1
Example 3:Consider the matrix: . Determine the images of
0 1
(i) Unit square {0 ≤ x ≤ 1, 0 ≤ y ≤ 1},
(ii) Unit circle {x 2 + y 2 = 1} and (iii) Unit disc {x 2 + y 2 ≤ 1}.
(i)    {0≤ x 
Unitsquare ≤ 1, 0 ≤ y ≤ 1} has vertices
0 1 1 0
, , , . Therefore the image has vertices
0 0 1 1
       
0 1 2 1
, , , respectively. The full image is a parallelogram
0 0 1 1
with
 these vertices.
      
x x +y u x u−v
(ii) A = = =⇒ = and
y y v y v
x 2 + y 2 = 1 =⇒ u 2 − 2uv + 2v 2 = 1 which is an ellipse.
(iii) Elliptic disc u 2 − 2uv + 2v 2 ≤ 1 enclosed by the ellipse
u 2 − 2uv + 2v 2 = 1.

January 5, 2017 17 / 42
Linear transformations and associated matrices
 
1 1
Example 4: Consider the matrix: Determine the images of
1 1
(i) Unit square {0 ≤ x ≤ 1, 0 ≤ y ≤ 1},
(ii) Unit circle {x 2 + y 2 = 1} and (iii) Unit disc {x 2 + y 2 ≤ 1}.
(i)    {0≤ x 
Unitsquare ≤ 1, 0 ≤ y ≤ 1} has vertices
0 1 1 0
, , , . Therefore the image has ’vertices’
0 0 1 1
       
0 1 2 1
, , , which are collinear. The full image is a line
0 1 2 1
   
0 2
segment to .
0 2
     
x x +y u
(ii) A = = and
y x +y v
√ √
x 2 + y 2 = 1 =⇒ √ y ) = − 2, max (x + y ) = 2 =⇒ a line
min (x + √
2 2
segment from − √ to √ .
2 2
(iii) the line segment above in (ii). [1.5]
January 5, 2017 18 / 42
Linear systems and matrices

Consider m linear equations in n variables x1 , x2 , ..., xn :

a11 x1 + a12 x2 + · · · + a1n xn = b1 (1)

a21 x1 + a22 x2 + · · · + a2n xn = b2 (2)
.. .. ..
. . .
am1 x1 + am2 x2 + · · · + amn xn = bm (m)

In matrix language, we can write

Ax = b
1 where A = [ajk ] is the m × n coefficient matrix,
2 x ∈ Rn is the unknown vector and b ∈ Rm is a given i.e. a known vector.
3 Also A+ = [A|b] is an m × (n + 1) matrix, called the augmented matrix
and it completely describes the above system of equations.

January 5, 2017 19 / 42
Gauss’ elimination

In order to solve the above system of linear equations, Gauss proposed 3

kinds of operations each of which gives a new linear system, (hopefully)
simpler and equivalent to the original one.
1 Pij : Interchanging two different equations, say equations (i) and (j).
2 Ejk (c): Multiplying an equation say (k), by a scalar c and adding it
to some other equation say (j) of the system.
3 Mj (c): Multiplying an equation by a non-zero number c. (E.g. to
clear the denominators.)

Each of the above operations is equivalent to an obvious corresponding

operation on the rows of the augmented matrix A+ .

January 5, 2017 20 / 42
Elementary row operations

Let A be any matrix. The elementary row operations on A are the

following:
1 Applying Pjk to A. This means Interchanging the j th and the k th
rows.
2 Applying Ejk (c) to A for j 6= k. This means multiplying the k th row
by c and adding it to the j th row.
3 And lastly, applying Mj (c), c 6= 0 to A. This implies that we multiply
the j th row by a scalar c 6= 0.
Exercise: Show that for j 6= k, Pjk Ejk (c)Pjk = Ekj (c). Thus we need Ejk (c) only
for j > k. (This can be useful sometimes.)

January 5, 2017 21 / 42
Row echelon form
Any matrix after a sequence of ERO’s (done cleverly) gets reduced to
what is known as a Row Echelon form. • In this form each row, except
perhaps the first, starts with a string of zeroes.
• Each row
starts with strictly more number of zeroes than the previous row.
• The first non-zero entry in the j th row is known as the j th pivot.
• The j th pivot is below and strictly to the right of (j − 1)th pivot. All the
entries below a pivot are zeroes.
Conclusion: The no. of pivots in a REF of A
≤ the no. of rows as well as the no.
of columns in A.  
3 2 1 3
 0 1 −1 6 
 
Example: The matrix   is in REF. The three pivots
 0 0 0 12 
0 0 0 0
are indicated.
January 5, 2017 22 / 42
Row echelon form ctd.

 
3 2 1 3
 0 1 −1 6
 
The matrix   is NOT in REF.

 0 0 0 0 
0 0 0 12
Properties of the REF:
Given a matrix A, its REF is NOT unique. However, the position of
each of its pivots is unalterable.
Reduced REF: A matrix in REF can be further row-operated upon to
ensure that (i) each pivot becomes 1 and (ii) all the entries above
each pivot become 0. This is the reduced REF and it is unique.
Reduced REF is mainly of theoretical interest only.

January 5, 2017 23 / 42
Example 1
Reduce the augmented matrix to the REF:

3x1 + 2x2 + x3 = 3
2x1 + x2 + x3 = 0
6x1 + 2x2 + 4x3 = 6

   
3 2 1 3 3 2 1 3
+ Row Ops 
A = 2 1 1 0 7 →  0
− 1 −1 6 
  
6 2 4 6 0 0 0 12
The matrix on the right is a REF of A+ . The pivots are highlighted. The
third equation of the equivalent system of equations reads:

0x1 + 0x2 + 0x3 = 12

which is absurd. The given system is inconsistent and hence has no

solutions.
January 5, 2017 24 / 42
Example 2

Solve the system of linear equations:

x1 − x2 + x3 = 0
−x1 + x2 − x3 = 0
10x2 + 25x3 = 90
20x1 + 10x2 = 80.

 
−1
 
1 −1 1 0 -1 1 0
−1 1 −1 0  Row Ops 
0 10 25 90

A+ = 
 0 10 25 90 7−→
  
 
 0 0 -95 −190 
20 10 0 80 0 0 0 0

January 5, 2017 25 / 42
Example 2, details

     
1 −1 1  0
 1 −1 1  0

−1 1 −1  0 E21 (1),E41 (−20) 0 0 0  0

0
  7−→   
10 25  90

0 10 25  90

20 10 0  80 0 30 −20  80
     
1 −1 1  0
 1 −1 1 
 0
P24  0 30 −20  80 M2 (−1),E23 (3) 0 0 95  190
7−→  0
 7−→   
10 25  90 0 10 25 
 90 
0 0 0  0 0 0 0  0
    
1 −1 1  0 1 −1 1  0

P23  0 10 25  90  M2 (1/5),M3 (1/95) 0 2 5  18
7−→   7−→  
0 0 95  190 0 
0 1  2
0 0 0  0

0 0 0  0

January 5, 2017 26 / 42
Example 2 contd.

Equations become

x1 − x2 + x3 = 0
2x2 + 5x3 = 18
x3 = 2
0 = 0.

The last equation shows consistency and the solution(s) may be obtained
by the back substitution method as

x3 = 2 =⇒ x2 = 4 =⇒ x1 = 2.

The solution is unique.

January 5, 2017 27 / 42
Example 3

Solve the system of linear equations:

3x1 + 2x2 + 2x3 − 5x4 = 8

0.6x1 + 1.5x2 + 1.5x3 − 5.4x4 = 2.7
1.2x1 − 0.3x2 − 0.3x3 + 2.4x4 = 2.1.

 
3 2 2 −5 8
+
A = 0.6 1.5
 1.5 −5.4 2.7 
1.2 −0.3 −0.3 2.4 2.1
 
3 2 2 −5 8
Row Ops
7−→  0 1.1 1.1 −4.4 1.1  .
0 0 0 0 0

January 5, 2017 28 / 42
Example 3 contd.

The equivalent system is:

3.0x1 + 2.0x2 + 2.0x3 − 5.0x4 = 8.0

1.1x2 + 1.1x3 − 4.4x4 = 1.1
0 = 0 (consistency).

Solution set, by the back substitution method, is

x2 = 1 − x3 + 4x4
x1 = 2 − x4 ; (x3 , x4 ) ∈ R2 .

The solution set is a 2-parameter family! [2.0]

January 5, 2017 29 / 42
Elementary row matrices

Recall the elementary row operations on matrices:

1 Pjk : Interchanging the j th and the k th rows.
2 Ejk (c): Adding c times the k th row to the j th row.
3 And Mj (λ): Multiplying the j th row by a scalar λ 6= 0.
Denote by Ejk the standard basic matrix whose (jk)th entry is 1 and the
rest 0’s. We need only m × m or square matrices in the following
m
X
discussion. Let I = Im (= Ejj ) be the m × m identity matrix.
j=1

January 5, 2017 30 / 42
Elementary row matrices, contd.

For a scalar c and 1 ≤ j 6= k ≤ m let us define some m × m matrices as

follows:
1 Pjk : The matrix Pjk is the matrix obtained by interchanging the j th
and the k th rows of I.
2 Ejk (c): The matrix Ejk (c) = I + cEjk , j 6= k i.e. the matrix obtained
from I by adding c times the k th row to the j th row.
3 Mj (λ): The matrix Mj (λ) is the matrix obtained from I by
multiplying its j th row by λ(6= 0).
The above matrices are known as the elementary row matrices (of order
m if you like.)

January 5, 2017 31 / 42
Examples of ERM 0 s

Among 3 × 3 matrices we notice that

 
0 1 0
P12 = 1 0 0
0 0 1.
 
1 0 0
E32 (c) = 0 1 0
0 c 1.
 
1 0 0
M2 (λ) = 0 λ 0
0 0 1.

January 5, 2017 32 / 42
ERM 0 s contd.

Theorem 13
Let A be any m × n matrix and Pjk , Ejk (c) and Mj (λ) be the m × m
ERM’s (j 6= k, λ 6= 0).
1 The product Pjk A is the m × n matrix obtained by interchanging the
j th and the k th rows of A.
2 The product Ejk (c)A is the m × n matrix obtained by adding c times
the k th row of A to the j th row of A.
3 The product Mj (λ)A is the m × n matrix obtained by multiplying the
j th row of A by λ.

Proof: Omitted.

January 5, 2017 33 / 42
Reduced Row Echelon form

Theorem 14
Let A be an m × n matrix. There exist ERM’s E1 , E2 , ..., EN of order m
such that the product EN · · · E2 E1 A is a row echelon form of A.

Proof: Obvious.
Reduced Row Echelon form: Once an echelon form of A is obtained, we
can by further row operations ensure that (i) each pivot becomes 1 and
(ii) all the entries above each pivot become zeroes. This is called the
Reduced Row Echelon form  of A and it isunique.
1 2 3 4
Example: Suppose A =  0 5 6 0  is in REF. Then by EROps we
 
0 0 0 7
 
1 2 3 4
can further reduce it to  0 1 1.2 0  and then again to
 
0 0 0 1
January 5, 2017 34 / 42
Reduced Row Echelon form contd.
 
1 0 0.6 0
0 1 1.2 0  which is THE reduced REF.
 
0 0 0 1

Theorem 15 (REF of square matrices)

Let A be a square matrix, say n × n. There exist ERM’s E1 , E2 , ..., EN of
order n such that the product EN · · · E2 E1 A is either (i) the n × n identity
matrix I or (ii) its last row is 0.

Proof:
Consider the reduced row echelon form of A. Let p(≤ n) be the number of
pivots. (i) If p = n then perforce each row and column will have a pivot
and the reduced REF must be I. (ii) If there are p < n pivots, then the
last n − p ≥ 1 rows must vanish.

January 5, 2017 35 / 42
Inverse of a square matrix

Definition 16
(Inverse of a square matrix) Let A be a square matrix. Its inverse is
another square matrix B, if it exist, satisfying AB = BA = I.

We say that A is invertible with an inverse B.

Properties:
1 If A has an inverse, then it is unique. It is denoted by A−1 .
2 If A and B are invertible, then so is AB and (AB)−1 = B −1 A−1 .
−1
3 Each ERM is invertible. In fact (i) Pjk = Pjk ,
−1 −1
(ii) Ejk (c) = Ejk (−c) and (iii) Mj (λ) = Mj (1/λ). Observe that
the inverses of ERM’s are also ERM’s.

January 5, 2017 36 / 42
Gauss Jordan Method

Theorem 17
A square matrix A is invertible if and only if it is a product of ERM’s.

Proof: If A is a product of ERM’s, then it is clearly invertible.

For the converse let A be invertible and consider its reduced REF, Â say.
(
I or
 = EN · · · E2 E1 A =
the last row is 0.

In the first case A = E1−1 E2−1 · · · EN−1 is a product of ERM’s as required

while the second case can not occur for invertible A. WHY?

 being a product of invertible matrices is invertible. ∃B such that

ÂB = I. then the last row of LHS is [0 0 ... 0] while that of RHS is
[0 0 ... 1] a contradiction.
Exercise: If k th row of any matrix M is 0, then so is the case with MP for
any matrix P which can multiply M from the right.
January 5, 2017 37 / 42
Gauss Jordan Method
 
a b
Example: Write as a product of 4 elementary row matrices if
c d
D = ad − bc 6= 0.
E21 ( −c 1 b/a E12 ( −b
     
a b a
) a b M1 ( 1a ),M2 ( Da ) )
Case 1 (a 6= 0): 7−→ 7−→ 7−→a I2 .
c d 0 D/a 0 1
Hence
 
A = E21 (c/a) M1 (a)M2 (D/a) E12 (b/a)
    
1 0 1/a 0 1 0 1 b/a
= .
c/a 1 0 1 0 D/a 0 1

Case 2: (a = 0) =⇒ D = −bc 6= 0 =⇒ c 6= 0: Then

 
A = P12 M2 (b)M1 (c) E12 (d/c). (Exercise)

January 5, 2017 38 / 42
Finding inverse by Gauss Jordan Method

Gauss-Jordan Method:

Let A be a square n × n matrix.

(1) Augment it on the right by I to obtain n × 2n matrix [A|I].

(2) Apply row operations to [A|I] till the left half becomes a REF of A.

(3a) If the last row of the left half is 0, STOP as A is not invertible.

(3b) Else continue till the left half becomes I.

(4) The right half is A−1 .

January 5, 2017 39 / 42
An example of Gauss Jordan Method
 
−1 1 2
Example: Find the inverse of A =  3 −1 1 .
−1 3 4
Solution:
     
−1 1 2  1 0 0 −1 1 2  1 0 0
 
   E (3),E (−1)   
 21 31
 3 −1 1  0 1 0 −→ 0 2 7  3 1 0
    
     
−1 3 4  0 0 1 0 2 2  −1 0 1
 
  
-1 1 2  1 0 0

E32 (−1)
  
−→  0 2 7  3 0 1
  
  
0 0 -5  −4 −1 1


January 5, 2017 40 / 42
Example contd.
The REF of A obtained has the last row non zero, hence CONTINUE.
  
1 −1 −2  −1 0 0


M1 (−1),M2 (1/2),M3 (−1/5) 
 
−→ 0 1 7/2  3/2 1/2 0 
 
  
0 0 1  4/5 1/5 −1/5

  
1 −1 0  3/5 2/5 −2/5

E13 (2),E23 (−7/2)
  
−→ 0 1 0  −13/10 −1/5 7/10 
  
  
0 0 1  4/5 1/5 −1/5

  
1 0 0  −7/10 1/5 3/10

E12 (1)
  
−→ 0 1 0  −13/10 −1/5 7/10 
  
  
0 0 1  4/5 1/5 −1/5


January 5, 2017 41 / 42
Example contd.

It follows that  
−7 2 3
1 
A−1 = −13 −2 7 .
10
8 2 −2
Also putting inverses of all the row ops together in reverse order,

A−1 = E12 (1) E13 (2)E23 (−7/2)

 
   
× M1 (−1)M2 (1/2)M3 (−1/5) E32 (−1) E21 (3)E31 (−1)

 
A = E31 (1)E21 (−3) E32 (1)
  
× M3 (−5)M2 (2)M1 (−1) E23 (7/2)E13 (−2) E12 (−1)

as products of ERM’s. [3.0]

January 5, 2017 42 / 42