ADDIS ABABA UNIVERSITY

COMPUTATIONAL DATA SCIENCE

PROGRAM
Matrix Algebra
Instructor: Hailemichael K. (PhD)

(hailekebede1995@gmail.com)

CDSC 603: Mathematical Methods (7 ECTS)

2021
 Outline
1. Scalars, vectors and matrices
2. Operation on Matrices (Addition, (Scalar) multiplication,
power, Transpose)
3. Special Types of Matrices (square, diagonal, symmetric,
triangular, identity )
4. Determinants and Inverse of Matrices
5. Some Application Areas of Matrix Algebra
6. Eigenvectors & Eigenvalues
Scalars, vectors and matrices
• Scalar: Variable described by a single number– e.g. density, temperature,
time, mass, etc . They possess only magnitude.

Vector: Variable described by magnitude and direction (Force, displacement,

velocity, acceleration etc)

• Matrix: Rectangular array of scalars (numbers)

2

Square (3 x 3) Rectangular (3 x 2) d r c : rth row, cth column

Operation on Matrices

Matrices
• A matrix is defined by the number of Rows and the
number of Columns.
• An mxn matrix has m rows and n columns.
A = 4x3 matrix

21 2 53

5 34 12

6 33 55

74 27 3

• A square matrix of order n, is an nxn matrix.

Matrix addition

Addition (matrix of same size)

– Commutative: A+B=B+A
– Associative: (A+B)+C=A+(B+C)

Subtraction consider as the addition of a negative matrix

Matrix multiplication

Constant (or Scalar)

multiplication of a matrix:

Matrix multiplication rule:

When A is a mxn matrix & B is a kxl matrix, the multiplication of
AB is only viable if n=k. The result will be an mxl matrix.
Transposition

column → row row → column

Example

Two vectors:

Inner product = scalar Note: (1xn)(nx1)  (1X1)

Outer product = matrix Note: (nx1)(1xn)  (nXn)

 Special Matrices:
Identity matrices
• Is there a matrix which plays a similar role as
the number 1 in number multiplication?
Consider the nxn matrix:

A square nxn matrix A has one

A In = I n A = A
An nxm matrix A has two!!
In A = A & A I m = A

1 2 3 1 0 0 1+0+0 0+2+0 0+0+3

Worked example
A In = A 4 5 6 X 0 1 0 = 4+0+0 0+5+0 0+0+6

for a 3x3 matrix: 7 8 9 0 0 1 7+0+0 0+8+0 0+0+9

 Matrix inverse
• For a XxX square matrix:

• The inverse matrix is:

• E.g.: 2x2 matrix

Inverse matrices
• Definition. A matrix A is nonsingular or invertible if there exists a
matrix B such that: worked example:
-1
2 2+1 -1 + 1
1 1 X = = 1 0
3 3 3 3 3
3
-2+ 2
1 1 1+2
-1 2 3 0 1
3 3 3 3
3

• Notation. A common notation for the inverse of a matrix A is A-1.

• The inverse matrix A-1 is unique when it exists.

• If A is invertible, A-1 is also invertible  A is the inverse matrix of A-1.

• If A is an invertible matrix, then (AT)-1 = (A-1)T

Matrix inverse

• Definition. A matrix A is called nonsingular or invertible if there

exists a matrix B such that:
-1
2 2+1 -1 + 1
1 1 X = = 1 0
3 3 3 3 3
3
-2+ 2
1 1 1+2
-1 2 3 0 1
3 3 3 3
3
• Notation. A common notation for the
inverse of a matrix A is A-1. So:

• The inverse matrix is unique when it exists. So if A is invertible, then A-1 is

also invertible and then (AT)-1 = (A-1)T

• In Matlab: A-1 = inv(A) •Matrix division: A/B= A*B-1

Matrix inverse
• For a XxX square matrix:

• The inverse matrix is:

• E.g.: 2x2 matrix

 Determinants
• Determinants are mathematical objects that are very useful in the analysis and
solution of systems of linear equations (i.e. GLMs).

• The determinant is a function that associates a scalar det(A) to every square

matrix A.

– Input is nxn matrix

– Output is a single number
(real or complex) called
the determinant
 Determinants
•Determinants can only be found for square matrices.
•For a 2x2 matrix A

det(A) =
[ ]
a
c
b
d
= ad - bc

How to get Determinant of A?

n n
A   a1 j C1 j 
j 1
j 1
a1 j  ( 1)1 j  M 1 j

• A matrix A has an inverse matrix A-1 if and only if det(A)≠0.

Solving for X
x = A-1 d ,
where A-1 is the inverse (matrix) of A

 x1   a11 a12  a1n 

1

x   d1 
a a 22  a 2 n  d 
 2   21  2
        
     
 xn  a n1 a n 2  a nn  d n 

A-1 = ad joint A / |A|,

where

|A| = ai1ci1 + …. + aincin (Determinant)

And, ad joint A

= transposed cofactor matrix of A

Determinant, Cofactor, and Minor

a11 a12 a13

a 22 a 23
A  a 21 a 22 a 23 M 11 
a32 a33
a31 a32 a33
a 21 a 23
n M 12 
A   a1 j C1 j
a31 a33
j 1 a 21 a 22
M 13 
a31 a32
C ij    1
i j
M ij
Pattern of the signs for cofactor minors

focus _ on _   1
i j
:

        
      
    
   
   
Ad joint of A: the transposed cofactor matrix

a11 a12  a1n  C11 C12  C1x 

 
a21 a22  a2 n  C 21 C 22  C 2n 
A C
    
nxn    
am1 am 2 am  C mi Cm2  C mn 

 C11 C 21  C m1 
 
 C12 C 22 Cm2 
C     adj A
nxn
   
C C2n C nn 
 1n 
Calculating Ad joint is hard!
Is there any easier way to solve for x or specifically
one of x, that is, xi ?
Cramer's Rule for each of x, say, x1 :
“The easy way”

Ai
xi 
A

The numerator represents a determinant of A in which the ith column is replaced by the vector
of constants, i.e., no need to invert A!
Solving for x1 using Cramer’s rule

a11 a12  a1n d1 a12  a1n

a 21 a 22  a2n d2 a 22  a2n
A A1 
nxn   nxn  
a m1 am2 a m dn an2 a nn

Find the determinant |A|

Find the determinant |A1|

where di is the constant vector substituted for the 1st col.

X1 = |A1|/|A|

Repeat for X2 by substituting the constant vector for the 2nd col. And
 Some Application Areas

Matrix algebra can be used:

 To express the system of equations in a

compact manner.
 To find out whether solution to a system of
equations exist.
 To obtain the solution if it exists.
 Some Application Areas

 Simultaneous Equations

 Simple Neural Network (Machine Learning)

 General Linear Model (GLM)

 System of linear equations
Resolving simultaneous equations can be applied using Matrices:

• Multiply a row by a non-zero constant Also known as

• Interchange two rows Gaussian Elimination
• Add a multiple of one row to another row

…
 Example: linear network:
inner product point of view

• What is the response of cell yi of the second layer?

• The response is the dot product of

the ith row of W with the vector x
 THANK YOU!

ANY

QUESTIONS COMMENTS