You are on page 1of 87

NUMERICAL METHODS:

GEC320
Matrices, Jacobi &
Gauss-Siedel
Matrix algebra has at least two advantages:
•Reduces complicated systems of equations to
simple expressions
•Adaptable to systematic method of mathematical
treatment and well suited to computers

Definition:
A matrix is a set or group of numbers arranged in a
square or rectangular array enclosed by two
brackets
 4 2 a b 
1 1  3 0 c d 
   
Properties:
•A specified number of rows and a specified
number of columns
•Two numbers (rows x columns) describe the
dimensions or size of the matrix.

Examples:
3x3 matrix 1 2 4

2x4 matrix
4  1 5 1 1
  
3  3
 1 1
3 3 3 0 0 3 2 
1x2 matrix
A matrix is denoted by a bold capital letter and the
elements within the matrix are denoted by lower case
letters
e.g. matrix [A] with elements aij

Amxn=  a11 a12 ... aij ain 


mA
n a a ... a 
a2 n 
 21 22 ij

     
 
am1 am 2 aij amn 
i goes from 1 to m
j goes from 1 to n
TYPES OF MATRICES

1. Column matrix or vector:


The number of rows may be any integer but the number of
columns is always 1

1   a11 
4 1  a21 
   3  
 2     
 am1 
TYPES OF MATRICES

2. Row matrix or vector


Any number of columns but only one row

1 1 6 0 3 5 2

a11 a12 a13  a1n 


TYPES OF MATRICES

3. Rectangular matrix
Contains more than one element and number of rows is not
equal to the number of columns

1 1 
3 7  1 1 1 0 0 
   2 0 3 3 0
7  7   
 
7 6 
mn
TYPES OF MATRICES
4. Square matrix
The number of rows is equal to the number of columns
(a square matrix A has an order of m)
mxm

1 1 1 1 1
9 9 0
3 0  
  6 6 1
The principal or main diagonal of a square matrix is composed of all
elements aij for which i=j
TYPES OF MATRICES

5. Diagonal matrix
A square matrix where all the elements are zero except those
on the main diagonal
3 0 0 0
1 0 0 0 3 0 
0
0 2 0  
  0 0 5 0
0 0 1  
0 0 0 9
i.e. aij =0 for all i = j
aij = 0 for some or all i = j
TYPES OF MATRICES

6. Unit or Identity matrix - I


A diagonal matrix with ones on the main diagonal

1 0 0 0
0 
 1 0 0 1 0 aij 0
0 0 0 1  0 
0 1    aij 
 
0 0 0 1
i.e. aij =0 for all i = j
aij = 1 for some or all i = j
TYPES OF MATRICES

7. Null (zero) matrix - 0


All elements in the matrix are zero

0  0 0 0 
0  0 0 0 
   
0 
0 0 0

aij  0 For all i,j


TYPES OF MATRICES

8. Triangular matrix
A square matrix whose elements above or below the main
diagonal are all zero

1 0 0  1 0 0  1 8 9
 2 1 0  2 1 0 0 1 6 
     
5 2 3 5 2 3 0 0 3
TYPES OF MATRICES

8a. Upper triangular matrix


A square matrix whose elements below the main
diagonal are all zero
1 7 4 4
aij aij aij  1 8 7 0 
  0 1 8   1 7 4
0 aij aij 
  0
0 0 7 8
 0 aij  0 0 3  
0 0 0 3
i.e. aij = 0 for all i > j
TYPES OF MATRICES

8b. Lower triangular matrix

A square matrix whose elements above the main diagonal are


all zero

aij 0 0 1 0 0 
   2 1 0
aij aij 0
 
aij aij aij  5 2 3

i.e. aij = 0 for all i < j
TYPES OF MATRICES
9. Scalar matrix
A diagonal matrix whose main diagonal elements are
equal to the same scalar
A scalar is defined as a single number or constant

aij 0 0 1 0 0 6 0 0 0
  0 1 0  0 
0 aij 0
   6 0 0
0 aij  0 0 1
 0 0 0 6 0
 
i.e. aij = 0 for all i = j 0 0 0 6
aij = a for all i = j
Matrices

Matrix Operations
Matrices - Operations

EQUALITY OF MATRICES
Two matrices are said to be equal only when all
corresponding elements are equal
Therefore their size or dimensions are equal as well

1 0 0  1 0 0 
A=  2 1 0 B=  2 1 0 A=B
   
5 2 3 5 2 3
Matrices - Operations
Some properties of equality:
•IIf A = B, then B = A for all A and B
•IIf A = B, and B = C, then A = C for all A, B and C

1 0 0  b11 b12 b13 


A=  2 1 0 B= b 
   21 b22 b23 
5 2 3 b31 b32 b33 

If A = B then aij  bij


Matrices - Operations
ADDITION AND SUBTRACTION OF MATRICES

The sum or difference of two matrices, A and B of the same


size yields a matrix C of the same size

cij  aij  bij


Matrices of different sizes cannot be added or subtracted
Matrices - Operations
Commutative Law:
A+B=B+A

Associative Law:
A + (B + C) = (A + B) + C = A + B + C

7 3  1  1 5 6  8 8 5
2  5 6    4  2 3   2  7 9
     
A B C
2x3 2x3 2x3
Matrices - Operations
A+0=0+A=A
A + (-A) = 0 (where –A is the matrix composed of –aij as elements)

6 4 2 1 2 0 5 2 2 
3 2 7  1 0 8  2 2  1
     
Matrices - Operations
SCALAR MULTIPLICATION OF MATRICES

Matrices can be multiplied by a scalar (constant or single


element)
Let k be a scalar quantity; then
kA = Ak

Ex. If k=4 and


 3  1
2 1 
A 
2  3
 
4 1 
Matrices - Operations
3  1 3  1 12  4 
2 1  2 1  8 4 
4   4   
2  3 2  3  8  12
     
4 1  4 1  16 4 

Properties:
• k (A + B) = kA + kB
• (k + g)A = kA + gA
• k(AB) = (kA)B = A(k)B
• k(gA) = (kg)A
Matrices - Operations
MULTIPLICATION OF MATRICES

The product of two matrices is another matrix


Two matrices A and B must be conformable for multiplication to
be possible
i.e. the number of columns of A must equal the number of rows
of B
Example.
A x B = C
(1x3) (3x1) (1x1)
Matrices - Operations
B x A = Not possible!
(2x1) (4x2)

A x B = Not possible!
(6x2) (6x3)

Example
A x B = C
(2x3) (3x2) (2x2)
Matrices - Operations
b11 b12 
 a11 a12 a13    c11 c12 
a   b21 b22    
 21 a22 a23 
b31 b32  c21 c22 

(a11  b11 )  (a12  b21 )  (a13  b31 )  c11


(a11  b12 )  (a12  b22 )  (a13  b32 )  c12
(a21  b11 )  (a22  b21 )  (a23  b31 )  c21
(a21  b12 )  (a22  b22 )  (a23  b32 )  c22

Successive multiplication of row i of A with column j of B – row by


column multiplication
Matrices - Operations
4 8 
1 2 3    (1 4)  (2  6)  (3  5) (1 8)  (2  2)  (3  3) 
4 2 7 6 2  (4  4)  (2  6)  (7  5) (4  8)  (2  2)  (7  3)
  5 3  
 

31 21
 
63 57

Remember also:
IA = A

1 0 31 21 31 21


0 1 63 57  
    63 57
Matrices - Operations
Assuming that matrices A, B and C are conformable for
the operations indicated, the following are true:
1. AI = IA = A
2. A(BC) = (AB)C = ABC - (associative law)
3. A(B+C) = AB + AC - (first distributive law)
4. (A+B)C = AC + BC - (second distributive law)
Caution!
1. AB not generally equal to BA, BA may not be conformable
2. If AB = 0, neither A nor B necessarily = 0
3. If AB = AC, B not necessarily = C
Matrices - Operations
AB not generally equal to BA, BA may not be conformable

1 2
T  
 5 0 
3 4
S 
 0 2 
1 2 3 4  3 8
TS       
5 0 0 2 15 20
3 4 1 2 23 6
ST       
0 2 5 0 10 0
Matrices - Operations
If AB = 0, neither A nor B necessarily = 0

1 1  2 3  0 0 
0 0  2  3  0 0
    
Matrices - Operations
TRANSPOSE OF A MATRIX

If :
2 4 7 
A 2 A   3

2x3 5 3 1 
Then transpose of A, denoted AT is:

2 5
A 2 A
T 3T
 4 3
7 1

aij  a T
ji For all i and j
Matrices - Operations
To transpose:
Interchange rows and columns
The dimensions of AT are the reverse of the dimensions of A

2 4 7 
A 2 A  
3
 2x3
 5 3 1 
2 5
A 3 A
T T2 
 4 3
3x2
7 1
Matrices - Operations
Properties of transposed matrices:
1. (A+B)T = AT + BT
2. (AB)T = BT AT
3. (kA)T = kAT
4. (AT)T = A
Matrices - Operations
1. (A+B)T = AT + BT

7 3  1  1 5 6  8 8 5 8  2
2  5 6    4  2 3   2  7 9 8  7
       
5 9 

7 2  1  4 8  2
 3  5  5  2  8  7
     
 1 6  6 3  5 9 
Matrices - Operations
(AB)T = BT AT

1 
1 1 0   2
0 2 3 1   8   2 8
   2  
 
1 0
1 1 21 2  2 8
0 3
Matrices - Operations
SYMMETRIC MATRICES
A Square matrix is symmetric if it is equal to its
transpose:
A = AT

a b 
A 
b d 
a b 
A 
T

 b d 
Matrices - Operations

When the original matrix is square, transposition does not


affect the elements of the main diagonal

a b 
A 
 c d 
a c 
A 
T

b d 
The identity matrix, I, a diagonal matrix D, and a scalar matrix, K,
are equal to their transpose since the diagonal is unaffected.
Matrices - Operations
INVERSE OF A MATRIX
Consider a scalar k. The inverse is the reciprocal or division of 1 by
the scalar.
Example:
k=7 the inverse of k or k-1 = 1/k = 1/7
Division of matrices is not defined since there may be AB = AC
while B = C
Instead matrix inversion is used.
The inverse of a square matrix, A, if it exists, is the unique matrix
A-1 where:
AA-1 = A-1 A = I
Matrices - Operations
Example:
3 1
A 2 A  
2

2 1
1  1  1
A  
 2 3 
Because:
 1  1 3 1 1 0
 2 3  2 1  0 1
   
3 1  1  1 1 0
2 1  2 3   0 1
   
Matrices - Operations
Properties of the inverse:

( AB) 1  B 1 A1
1 1
(A )  A
T 1 1 T
(A )  (A )
1 1 1
( kA)  A
k
A square matrix that has an inverse is called a nonsingular matrix
A matrix that does not have an inverse is called a singular matrix
Square matrices have inverses except when the determinant is zero
When the determinant of a matrix is zero the matrix is singular
Matrices - Operations
DETERMINANT OF A MATRIX

To compute the inverse of a matrix, the determinant is


required
Each square matrix A has a unit scalar value called the
determinant of A, denoted by det A or |A|
1 2
If A 
 6 5 
1 2
then A
6 5
Matrices - Operations
If A = [A] is a single element (1x1), then the
determinant is defined as the value of the
element
Then |A| =det A = a11
If A is (n x n), its determinant may be defined in
terms of order (n-1) or less.
Matrices - Operations
MINORS
If A is an n x n matrix and one row and one
column are deleted, the resulting matrix is an
(n-1) x (n-1) submatrix of A.
The determinant of such a submatrix is called a
minor of A and is designated by mij , where i
and j correspond to the deleted
row and column, respectively.
mij is the minor of the element aij in A.
Matrices - Operations
eg.
 a11 a12 a13 

A  a21 a22 
a23 
a31 a32 a33 
Each element in A has a minor
Delete first row and column from A .
The determinant of the remaining 2 x 2 submatrix is the minor of
a11
a22 a23
m11 
a32 a33
Matrices - Operations
Therefore the minor of a12 is:

a21 a23
m12 
a31 a33
And the minor for a13 is:

a21 a22
m13 
a31 a32
Matrices - Operations
COFACTORS

The cofactor Cij of an element aij is defined as:

Cij  (1)i  j mij

When the sum of a row number i and column j is even, cij = mij and
when i+j is odd, cij =-mij
c11 (i  1, j  1)  (1)11 m11   m11
1 2
c12 (i  1, j  2)  (1) m12  m12
1 3
c13 (i  1, j  3)  (1) m13   m13
Matrices - Operations
DETERMINANTS CONTINUED

The determinant of an n x n matrix A can now be defined as

A  det A  a11c11  a12c12    a1nc1n


The determinant of A is therefore the sum of the products of the
elements of the first row of A and their corresponding cofactors.
(It is possible to define |A| in terms of any other row or column
but for simplicity, the first row only is used)
Matrices - Operations
Therefore the 2 x 2 matrix :

 a11 a12 
A 
a21 a22 
Has cofactors :
c11  m11  a22  a22
And:
c12  m12   a21  a21
And the determinant of A is:

A  a11c11  a12c12  a11a22  a12a21


Matrices - Operations
Example 1:
3 1
A 
1 2
A  (3)(2)  (1)(1)  5
Matrices - Operations
For a 3 x 3 matrix:
 a11 a12 a13 

A  a21 a22 
a23 
a31 a32 a33 
The cofactors of the first row are:
a22 a23
c11   a22a33  a23a32
a32 a33
a21 a23
c12    (a21a33  a23a31 )
a31 a33
a21 a22
c13   a21a32  a22a31
a31 a32
Matrices - Operations
The determinant of a matrix A is:

A  a11c11  a12c12  a11a22  a12a21


Which by substituting for the cofactors in this case is:

A  a11(a22a33  a23a32 )  a12 (a21a33  a23a31)  a13 (a21a32  a22a31)


Matrices - Operations

Example 2:
 1 0 1
A   0 2 3
 1 0 1

A  a11(a22a33  a23a32 )  a12 (a21a33  a23a31)  a13 (a21a32  a22a31)

A  (1)(2  0)  (0)(0  3)  (1)(0  2)  4


Matrices - Operations
ADJOINT MATRICES

A cofactor matrix C of a matrix A is the square matrix of the same


order as A in which each element aij is replaced by its cofactor cij .

Example:
 1 2
If A 
  3 4 

 4 3
The cofactor C of A is C 
  2 1
Matrices - Operations
The adjoint matrix of A, denoted by adj A, is the transpose of its
cofactor matrix
adjA  C T

It can be shown that:


A(adj A) = (adjA) A = |A| I

Example:
 1 2
A 
  3 4 
A  (1)( 4)  (2)( 3)  10
 4  2
adjA  C  T

 3 1 
Matrices - Operations

 1 2 4  2 10 0 
A(adjA)         10 I
 3 4 3 1   0 10

4  2  1 2 10 0 
(adjA) A         10 I
3 1   3 4  0 10
Matrices - Operations
USING THE ADJOINT MATRIX IN MATRIX INVERSION
Since
AA-1 = A-1 A = I

and
A(adj A) = (adjA) A = |A| I

then
1adjA
A 
A
Matrices - Operations
Example
 1 2
A=   3 4
 

1 1 4  2 0.4  0.2
A   

10 3 1  0.3 0.1 
To check AA-1 = A-1 A = I
 1 2 0.4  0.2 1
1 0
AA        I
 3 4 0.3 0.1  0 1
1 0.4  0.2  1 2 1 0
A A      I
0.3 0.1   3 4 0 1
Matrices - Operations
Example 2
3  1 1 
A  2 1 0 
1 2  1
The determinant of A is
|A| = (3)(-1-0)-(-1)(-2-0)+(1)(4-1) = -2

The elements of the cofactor matrix are


c11  (1), c12  (2), c13   (3),
c21  (1), c22  (4), c23  (7),
c31  (1), c32  (2), c33   (5),
Matrices - Operations
The cofactor matrix is therefore
 1 2 3
C   1  4  7
 1 2 5 

so
 1 1  1
adjA  C T   2  4 2 
 3  7 5 

and
 1 1  1  0.5  0.5 0.5 
1 adjA 1     1.0 2.0  1.0 
A   2  4 2
A 2    
 3  7 5   1.5 3.5  2.5
Matrices - Operations
The result can be checked using

AA-1 = A-1 A = I

The determinant of a matrix must not be zero for the inverse to


exist as there will not be a solution
Nonsingular matrices have non-zero determinants
Singular matrices have zero determinants
Matrix Inversion
Simple 2 x 2 case
Let
and
a b  w x
1
A  A  
c d   y z

Since it is known that


A A-1 = I

then
a b   w x  1 0
 c d   y z   0 1 
    
Simple 2 x 2 case
Multiplying gives

aw  by  1
ax  bz  0
cw  dy  0
cx  dz  1

It can simply be shown that


A  ad  bc
Simple 2 x 2 case
thus
1  aw
y
b
 cw
y
d
1  aw  cw

b d
d d
w 
da  bc A
Simple 2 x 2 case

 ax
z
b
1  cx
z
d
 ax 1  cx

b d
b b
x 
 da  bc A
Simple 2 x 2 case

1  by
w
a
 dy
w
c
1  by  dy

a c
c c
y 
 ad  cb A
Simple 2 x 2 case

 bz
x
a
1  dz
x
c
 bz 1  dz

a c
a a
z 
ad  bc A
Simple 2 x 2 case
So that for a 2 x 2 matrix the inverse can
be constructed in a simple fashion as
d b
 A A  1  d  b
1w x  
A     c a  A  c a 
 y z  A
 A 

•Exchange elements of main diagonal


•Change sign in elements off main diagonal
•Divide resulting matrix by the determinant
Simple 2 x 2 case
Example
2 3
A 
 4 1
1 1  1  3  0.1 0.3 
A     
10  4 2   0.4  0.2
Check inverse
A-1 A=I

1  1  3 2 3 1 0
       I
10  4 2  4 1 0 1
Matrices and Linear Equations
Linear Equations
Linear equations are common and important for survey
problems
Matrices can be used to express these linear equations and
aid in the computation of unknown values
Example
n equations in n unknowns, the aij are numerical
coefficients, the bi are constants and the xj are unknowns

a11x1  a12 x2    a1n xn  b1


a21x1  a22 x2    a2 n xn  b2

an1 x1  an 2 x2    ann xn  bn
Linear Equations
The equations may be expressed in the form
AX = B
where

 a11 a12  a1n   x1   b1 


a21 a22  a2 n   x2  b2 
A , X   , and B 
       
an1 an1  ann   xn  bn 

nxn nx1 nx1

Number of unknowns = number of equations = n


Linear Equations
If the determinant is nonzero, the equation can be solved to
produce n numerical values for x that satisfy all the simultaneous
equations
To solve, pre-multiply both sides of the equation by A-1 which exists
because |A| = 0
A-1 AX = A-1 B
Now since
A-1 A = I

We get
X = A-1 B

So if the inverse of the coefficient matrix is found, the unknowns,


X would be determined
Summarily, to obtain the inverse of a matrix:

1) Form the determinant of the matrix and evaluate it


2) Form a new matrix consisting of the cofactors of the elements in the
matrix
3) Form the transpose of the cofactors
Then the inverse of the original matrix is
Linear Equations
Example
3x1  x2  x3  2
2 x1  x2  1
x1  2 x2  x3  3

The equations can be expressed as

3  1 1   x1  2
2 1 0   x   1
  2   
1 2  1  x3  3
Linear Equations
When A-1 is computed the equation becomes

 0.5  0.5 0.5  2  2 


X  A1 B   1.0 2.0  1.0  1   3
 1.5 3.5  2.5 3  7 

Therefore
x1  2,
x2  3,
x3  7
Linear Equations
The values for the unknowns should be checked by substitution
back into the initial equations

x1  2, 3x1  x2  x3  2
x2  3, 2 x1  x2  1
x3  7 x1  2 x2  x3  3

3  (2)  (3)  (7)  2


2  (2)  (3)  1
(2)  2  (3)  (7)  3
Write out the augmented matrix for the system of equations below and
obtain the values of the unknows by
1) Row operation method
2) Method of determinant
3) Matrix inversion method
Jacobi iteration
&
Gauss-Siedel
Jacobi iteration
a11x1  a12 x2    a1n xn  b1  x10 
a21x1  a22 x2    a2 n xn  b2  0
 x2 
x 
0
 
 0
an1 x1  an 2 x2    ann xn  bn  xn 

x11 
1
(b1  a12 x20    a1n xn0 ) 1  i 1 n 
a11
k 1
xi  bi 
aii 
 aij x   aij x 
k
j
k
j

1
j 1 j  i 1 
x12  (b2  a21 x10  a23 x30    a2 n xn0 )
a22
1
x1n  (bn  an1 x10  an 2 x20    ann1 xn01 )
ann
80
Example 1
 4  1 1  x1   7  0
 4  8 1  x    21
  2    x 0  0
 2 1 5  x3   15  0

Diagonally dominant matrix


7  x20  x30 7
x 1
1   1.75
4 4
21  4 x 0
 x 0
21
x12  1 3
  2.625
8 8
15  2 x 0
 x 0
15
x31  1 2
  3 .0
5 5
7  x1
 x1
7  2.625  3
x1 
2 2 3
  1.65625
4 4
21  4 x1  x3 21  4 1.75  3
1 1
x2 
2
  3.875
8 8
15  2 x 1
 x1
15  2 1.75  2.625
x3 
2 1 2
  4.225
5 5
7  3.875  4.225
x 
3
1  1.6625
4
21  4 1.65625  4.225
x2 
3
 3.98125
8
15  2 1.65625  3.875
x3 
3
 2.8875
5
Matrix is diagonally dominant, Jacobi iterations are converging
Example 2

 2 1 5  x1   15  0
 4  8 1  x    21 x 0  0
  2     
 4  1 1  x3   7  0
The matrix is not diagonally dominant

 15  x  5 x
0 0
 15
x 
1
1
2 3
  7.5
2 2
21  4 x 0
 x 0
21
x2 
1 1 3
  2.625
8 8
x31  7  4 x10  x20  7.0 83
 15  2.625  5  7
x 
1
1  11.3125
2
21  4  7.5  7
x2 
1
 0.25
8
x3  7  4  7.5  2.625  39.625
1

The residual term is increasing at each iteration, so the iterations are


diverging.
Note that the matrix is not diagonally dominant
84
Gauss-Seidel (GS) iteration
a11x1  a12 x2    a1n xn  b1  x10 
Use the latest a21x1  a22 x2    a2 n xn  b2  0
 x2 
update x 
0
 
 0
an1 x1  an 2 x2    ann xn  bn  xn 

x11 
1
(b1  a12 x20    a1n xn0 ) 1  i 1 n 
a11
k 1
xi  bi 
aii 
a ij x k 1
j   aij x  k
j
1 j 1 j  i 1 
x12  (b2  a21 x11  a23 x30    a2 n xn0 )
a22
1
x1n  (bn  an1 x11  an 2 x12    ann1 x1n 1 )
ann
85
Example 1:

 4  1 1  x1   7  0
 4  8 1  x    21 0  
  2    x  0
 2 1 5  x3   15  0
Diagonally dominant matrix
7  x 0
 x 0
7
x1 
1 2 3
  1.75
4 4
21  4 x 1
 x 0
21  4 1.75
x12  1 3
  3.5
8 8
15  2 x1  x2
1 1
15  2 1.75  3.5
x3 
1
  3.0
5
7  x 1
 x 1
7  3.5  3
x1 
2 2 3
  1.875
4 4
21  4 x1  x3
2 1
21  4 1.875  3
x2 
2
  3.9375
8 8
15  2 x 2
 x 2
15  2 1.875  3.9375
x3 
2 1 2
  2.9625
5 5