SILLIMAN UNIVERSITY

COLLEGE OF ENGINEERING AND DESIGN

Course Title: NUMERICAL METHODS

Prepared by: Engr. Mylane A. Farolan, MEE, REE

CHAPTER 2: LINEAR AND NONLINEAR SYSTEMS AND MATRICES

A. DIRECT SOLUTION TO LINEAR ALGEBRAIC EQUATIONS: GAUSS

ELIMINATION
 For an equation:

 Supposing are given, we want to solve for . And assuming that the
is nonzero, then a unique solution to exists.
 In matrix form of

[ ][ ] [ ]

 Converting the matrix into an UPPER TRIANGULAR matrix with non-zero

diagonal elements and using BACK SUBSTITUTION to solve for the
solution, the matrix becomes

[ ][ ] [ ]

or in general form,

∑
GAUSS ELIMINATION METHOD

 The process of converting a matrix into an upper triangular matrix is called

Gauss Elimination
 The general process is to multiply eqn. k by and subtract the product
from eqn. n where and k is the
kth step of the elimination process.
 There are elimination steps.
 The division operation of rows by the pivot element, is called
normalization.

EXAMPLE 1: Use Gauss Elimination to triangularize the matrix given below.

[ ][ ] [ ]

1st elimination step :

[ ][ ] [ ]

[ ][ ] [ ]

2nd elimination step :

[ ][ ] [ ]

[ ][ ] [ ]

By Back Substitution:

; ;
 B. PITFALLS OF GAUSS ELIMINATION
1. Division by zero (That is, division by where or very near to zero)
 The technique to solve this problem is by pivoting. The process is to
determine the largest available coefficient in the column where the pivot
element belongs. The rows can also be interchanged so that the largest
element is the pivot element.
2. Round-off Errors
 There is a slight discrepancy in the result of the solution depending on the
number of significant figures used, which means that more significant figures
reduce error.
C. GAUSS JORDAN
 This method is a slight variation of Gauss Elimination method. The
elimination steps result in an identity matrix rather than an upper triangular
matrix as in Gauss Elimination.
 In the elimination process, all rows are normalized by dividing them by their
pivot elements.
 The general process is to multiply eqn. k by and subtract the product
from eqn. n where and k
is the kth step of the elimination process.
 The result is an identity matrix:

[ ][ ] [ ]

EXAMPLE 2: Use Gauss Jordan to solve the matrix given below.

[ ][ ] [ ]

 Normalize the 1st pivot element: Multiply the first row by

[ ][ ] [ ]

 1st elimination step :

 [ ][ ] [ ]

[ ][ ] [ ]

 Normalize the 2nd pivot element: Multiply the 2nd row by

[ ][ ] [ ⁄ ]

 2nd elimination step :

[ ][ ] [ ]

[ ][ ] [ ]

 Normalize the 3rd pivot element: Multiply the 3rd row by

[ ][ ] [ ]

 3rd elimination step :

[ ][ ]

[ ]
 [ ][ ] [ ]

 By Direct Substitution:

; ;

D. LU DECOMPOSITION
 A method of solving algebraic equations by manipulating the elimination
process on the right side of the equation.
;
 Express matrix A as an upper triangular matrix:

[ ][ ] [ ]

 Assuming that a unit lower triangular matrix is multiplied to

[ ]

{ }

 Collecting like terms:

 Solution Steps:
1. LU Decomposition step: Matrix is factored or decomposed into lower
and upper triangular matrices.
2. Substitution Step: Matrices and are used to determine a solution
for the right hand side of the equation where the equation
is used to generate intermediate vector by forward
substitution.
o The result is substituted into , which can be solved
by back substitution.

EXAMPLE 3: Derive an LU Decomposition based on Gauss Elimination.

 [ ]

 Use forward elimination to derive an upper triangular matrix:

[ ]

 Use forward elimination to derive a unit lower triangular matrix:

[ ]

 By the formula

[ ]

[ ][ ]

EXAMPLE 4: Use LU substitution to solve for .

[ ][ ] [ ]

 Making use of the formula :

[ ][ ] [ ]

[ ] [ ]

 Making use of the formula :

[ ][ ] [ ]
  By back substitution and making use of the formula:
∑

[ ] [ ]

E. TRIDIAGONAL MATRIX
 A banded matrix is a square matrix that has all elements equal to zero, with
the exemption of a band centered on the main diagonal.
 A tridiagonal system – that is, one with a bandwidth of 3 – can be expressed
as

 This was done to avoid storing large numbers of useless zeros in the square
matrix . This space saving modification is advantageous because the
resulting algorithm requires less computer memory.

F. CHOLESKY DECOMPOSITION
 This is a very popular approach of solving system of equations by using
symmetric matrices.
 A symmetric matrix is one where . In other words,
.
 Cholesky Decomposition method has lower triangular factors of matrix
which are Transpose of each other.

 The elements of the Lower triangular matrix can be derived using the
formulas:
∑

√ ∑
G. ITERATIVE SOLUTION TO LINEAR ALGEBRAIC EQUATIONS: JACOBI
AND GAUSS-SEIDEL
 The stopping criterion:

| |

Where is called the tolerance level

1. JACOBI METHOD
 The solution is obtained by considering the kth equation of the Linear
Algebraic Function
or

∑ ∑

 The Jacobi Method uses the “previous” value of at iteration on the

right hand side of the equation to generate “new” value, on the left
side.
∑ ∑ ]
 Jacobi Iteration in Matrix Form:

Where D contains only the diagonal elements of matrix A and all other
elements are zero (0) and matrix A is the coefficient matrix of the given sets
of equations.

[ ]

[ ]

EXAMPLE 5: Using Jacobi Iteration, Find the values of . Start with the
values of and continue until . Given are:
Rewriting the equations:

@ :

@ :

Check for tolerance:

| |

| | = | | (False, therefore continue

iteration.)

| | = | | (False, therefore continue

iteration.)

Note: Continue iteration until tolerance value is acceptable (that is, less than
).

In Matrix Form:
[ ][ ] [ ]

[ ]

[ ] [ ] [ ] [ ]

[ ]

[ ]

[ ] [ ]
From the formula:
 [ ] [ ][ ][ ] [ ][ ]

@ Iteration 1:
[ ] [ ][ ][ ] [ ][ ] [ ]

@ Iteration 2:
[ ] [ ][ ][ ] [ ][ ]

[ ]

@ Iteration 3:
[ ] [ ][ ][ ] [ ][ ]

[ ]

@ Iteration 4:
[ ] [ ][ ][ ] [ ][ ]

[ ]

Iteration 0 1 2 3 4 …
0 0.6 0.434 … 0.4875
0 0.3333 0.2003 … 0.225
---

2. GAUSS SEIDEL METHODS

 This method is similar to Jacobi Iteration method, however, iteration is
improved by substituting the new value of the solution immediately in the
calculation of the second variable instead of completing the calculations using
the values from the previous set.
 The Gauss Seidel Method is has the general equation:

∑ ∑

 In Matrix Form:
 Where D is a Lower Triangular Matrix of matrix A and matrix A is the
coefficient matrix of the given sets of equations.

[ ]

[ ]

EXAMPLE 6: Using Gauss Seidel Iteration method, Find the values of .

Start with the values of and continue until . Given are:

Rewriting the equations:

@ :

@ :

Check for tolerance:

| |

| | = | | (False, therefore continue

iteration.)

| | = | | (False, therefore continue

iteration.)
Note: Continue iteration until tolerance value is acceptable (that is, less than
).

In Matrix Form:
[ ][ ] [ ]

[ ]

[ ] [ ] [ ] [ ]

[ ]

[ ]

[ ] [ ]
From the formula:

[ ]

[ ][ ][ ]

[ ][ ]

@ Iteration 1:
[ ] [ ][ ][ ] [ ][ ]

[ ]

@ Iteration 2:
[ ]

[ ][ ][ ]

[ ][ ] [ ]
@ Iteration 3:
[ ]

[ ][ ][ ]

[ ][ ] [ ]
 @ Iteration 4:
[ ] [ ][ ][ ] [ ][ ]

[ ]

Iteration 0 1 2 3 4 …
0 0.6 … 0.4875
0 … 0.225
---

DIVERGENCE OF JACOBI AND GAUSS SEIDEL

 These two methods may diverge if any of these can occur:

a. If any diagonal element then Jacobi and Gauss Seidel are
undefined.
b. If the diagonal coefficients are larger than the other coefficients

H. ITERATIVE SOLUTION TO NONLINEAR ALGEBRAIC EQUATIONS:

NEWTON RAPHSON
 A set of nonlinear equations in matrix format is given by:

where are both N vectors and is an N

[ ]
vector of functions

 Adding to both sides:

 Multiplying both sides by (or Inverse of D)

[ ]

 The previous value is used in the right side of the equation to generate
the new value of on the left side.

 For Linear Equations:

Thus,
 This equation is similar to the Jacobi and Gauss Seidel methods (linear
methods) used for nonlinear equations, where matrix D has to be specified.

 THE NEWTON RAPHSON

o Newton Raphson is a method for specifying D to solve nonlinear
system of algebraic equations.
o This method is based on Taylor Series Expansion of about an
operating point .

o Solving for (Neglecting higher order terms):

[ | ]

[ | ]

o Newton Raphson replaces by the previous value of and replaces

by the new value of .

[ | ]

[ ]
o The matrix , whose elements are partial derivatives is
called Jacobian Matrix.
o Newton Raphson is similar to extended Gauss Seidel method where D
is replaced by J(i).

EXAMPLE 7: Using Newton Raphson method, find the solution to the given nonlinear
algebraic equations. Assume initial values of .
 [ | ] [ ]

[ ]

[ ]

[ ]

[ ]

From the Newton Raphson Formula:

[ ( )]

[ ] [ ] [ ] {[ ] [ ]}

@ Iteration 1: [ ] [ ] [ ] {[ ] [ ]}

[ ] [ ] [ ][ ] [ ]

@ Iteration 2: [ ] [ ] [ ] {[ ]

[ ]}

[ ] [ ] [ ][ ] [ ]

Note: Continue iteration until tolerance value is acceptable (that is, less than
).

Iteration 0 1 2 3 4 …
4 5.2 … 0.4875
9 .8 … 0.225
--- --- --- --- --- --- ---
ASSIGNMENT 2:
Problem 1: Develop a program using Jacobi Method and 2) Gauss-Seidel Method to solve
for the unknown variables of the given system of equations below. Make initial guess that allows
your solution to converge. Set the stopping criterion equal to .

[ ][ ] [ ]

Problem 2: Develop a Newton-Raphson Program that finds the solution of the given non-
linear system of equations (in matrix form) where are in radians. Start with
and continue until .

[ ] [ ]

GOD BLESS