Professional Documents
Culture Documents
Module 2
Systems of Linear
Algebraic Equations
Eduardo E. Descalsota, Jr.
Subtopics
2.1 Introduction
2.2 Methods of Solution
2.3 Matrix Inversion Method
2.4 Gauss Elimination Method
2.5 LU Decomposition Methods
2.6 Gauss-Jordan Method
2.7 Jacobis Iteration Method
2.8 Gauss-Seidel Iteration Method
Notations
Algebraic Notation:
Matrix Notation:
or simply Ax = b
where:
coefficients Aij and
constants bj are known
xi represent the unknowns
Augmented Matrix
obtained by adjoining the constant vector b
to the coefficient matrix A
Uniqueness of Solution
A system of n linear equations in n unknowns
has a unique solution, provided that:
determinant of the coefficient matrix is
nonsingular, i.e., if |A| 0
rows and columns of a nonsingular matrix
are linearly independent
no row (or column) is a linear combination of
other rows (or columns)
x = 1 x = 2 x = 3 x = 0 x = -5
y = 1 y = -1 y = -3 y = 3 y = 13
infinite number of combinations of x and y
Example 2:
2x + y = 3
4x + 2y = 0
No Solution:
4x + 2y = 0 is equivalent to 2x + y = 0
hence, any solution that satisfies one
equation cannot satisfy the other one
2.2
Methods of Solution
Direct Methods
1. Matrix Inverse Method
2. Gauss Elimination Method
3. LU Decomposition Methods
4. Gauss-Jordan Method
(1/2)
(2/2)
2.3
Matrix Inversion
Method
1
0
0
0
1
0
0
0
1
Adet = ad bc
A-1 = Aadj/Adet
A-1 =
1
0
0
1
1
0
Example:
Example:
Find the inverse of the matrix
1)
2
5
3
1
3)
4
2
1
3
1
1
4)
15
2
1
2)
3
10
2
2
1
1
2
5
1
4
2
1
8
10
Solve:
x y + 3z = 5
4x + 2y z = 0
x + 3y + z = 5
11
2.4
Gauss Elimination
Method
Gauss Elimination
a popular technique for solving simultaneous
linear algebraic equations (Ax = b)
reduces the coefficient matrix into an upper
triangular matrix (Ux = c)
consists of two parts:
elimination phase
solution phase
Initial Form: Ax = b
Final Form: Ux = c
12
Gauss Elimination
Process
1. Eliminate x1 from the second
and third equations
assuming a11 0.
2. Eliminate x2 from the third
row assuming a'22 0.
3. Apply back substitution:
x3 from a''33x3 = b''3
x2 from a'22x2 + a'23x3 = b'2
x1 from a11x1 + a12x2 + a13x3 = b1
13
Pivoting
Gauss elimination method fails if any one of
the pivots becomes zero
What if pivot is zero?
Solution: interchange the equation with its lower
equations such that the pivots are not zero
Example:
Solve the following equations
by Gauss elimination method:
2x + 4y 6z = -4
x + 5y + 3z = 10
x + 3y + 2z = 5
1
2
3
2x + 4y 6z = -4
-6y 12z = -24
-2y 10z = -14
-6y 12z = -24
18z = 18
z=1
y = (24 12z)/6
2x + 4y 6z = -4
x = 3z 2y 2
y=2
x = 3(1)2(2)2
x = -3
y = (4 2(1))
14
Problem 1:
Use the method of Gaussian elimination to
solve the following system of linear equations:
x1 + x2 + x3 x4 = 2
4x1 + 4x2 + x3 + x4 = 11
x1 x2 x3 + 2x4 = 0
2x1 + x2 + 2x3 2x4 = 2
Problem 2:
Using the Gaussian elimination method, solve
the system of equations [A] {x} = {b} where
15
2.5
LU Decomposition
Methods
Doolittles Method
Choleskis Method
Crouts Method
LU Decomposition
aka LU Factorization
process of computing L and U for a given A
expressed as a product of a lower triangular matrix
L and an upper triangular matrix U
A = LU
General LU Process:
Ax = b
LUx = b
Ux = y
Ly = b
16
Constraints
LU decomposition is not unique unless certain
constraints are placed on L or U
Doolittles Decomposition
Method
(1/2)
transforms Ax = b to LUx = b to Ux = y
Consider a 3x3 matrix A:
and assume that there exist triangular matrices:
17
Example:
Use Doolittles decomposition method
to solve the equations Ax = b, where
DECOMPOSITION PHASE
1st pass:
1 4 1
pivot: row1
0 2 -2
row2 row2 1 row1
0 -9 0
(eliminates A21)
row3 row3 2 row1
(eliminates A31)
A = A replacing the eliminated
1 4 1
terms with multipliers
0 2 -2
2nd pass:
0 0 -9
pivot: row2
row3 row3 (4.5) row2
(eliminates A32)
1
A = 1
2
4 1
2 -2
-9 0
1 4 1
A = 1 2 -2
2 -4.5 -9
18
SOLUTION PHASE
Forward substitution: Ly = b
y1
=7
y1 + y2
= 13
2y1 4.5y2 + y3 = 5
Solving for y2:
y2 = 13 y1 = 13 7
y2 = 6
Solving for y3:
y3 = 5 2y1 + 4.5y2
y3 = 5 2(7) + 4.5(6)
y3 = 18
Problem:
Solve AX = B with Doolittles decomposition
and compute |A|, where
19
or
u13 = a13/l11
u23 = (a23 l21u13) / l22
l33 = a33 l31u13 l32u23
A=LU
l11 = 2
l21 = 8
l31 = 4
l11u12 = 1
u12 = 1/2
l22 + l21u12 = -3
l22 = -3 8(1/2)
l22 = -7
l32 + l31u12 = 11
l32 = 11 4(1/2)
l32 = 9
l11u13 = 4
u13 = 4/2 = 2
l21u13 + l22u23 = 2
u23 = (2 8(2))/-7
u23 = 2
l31u13+ l32u23+ l33= -1
l33 = -1 4(2) 9(2)
l33 = -27
20
2
8
4
1/2 2
0 0
-7 -14
9 -9
2
8
4
2
0 00
-7 -14
0
9 -27
-10
Ly = b: forward subst.:
2y1 = 12
y1 = 6
8y1 7y2 = 20
-7y2 = 20 8(6)
y2 = -28/-7 = 4
4y1 + 9y2 27y3 = 33
-27y3 = 33 4(6) 9(4)
y3 = -27/-27 = 1
Ux = y: backward subst:
z = y3 = 1
y + 2z = y2
y = 4 2(1)
y=2
x + y + 2z = y1
x = 6 (2) 2(1)
x=3
21
Problem:
Solve the following set of equations by using the Crouts
method:
2x1 + x2 + x3 = 7
x1 + 2x2 + x3 = 8
x1 + x2 + 2x3 = 9
Choleskys Decomposition
A = LLT where U=LT
not a particularly popular means of solving
simultaneous equations
but invaluable in certain other applications
(e.g., in the transformation of eigenvalue problems)
Limitations:
requires A to be symmetric since the matrix product LLT
is symmetric
involves taking square roots of certain combinations of
the elements of A
square roots of negative numbers can be avoided only if A is
positive definite
22
Example:
Compute Choleskys
decomposition of the matrix
Note that A is symmetric, so Choleskis applicable:
Equating the given matrix A to LLT:
23
Problem:
Solve the equations Ax = b by Choleskys
decomposition method, where:
1)
2)
Problem:
Given the LU decomposition A = LU, determine A and |A|.
24
2.6
Gauss-Jordan
Method
Gauss-Jordan Method
extension of the Gauss elimination method
Ax = b is reduced to a diagonal set Ix = b'
where:
I = a unit matrix
Ix = b' equivalent to x = b' where b'
solution vector
25
Example:
Solve the following equations
by Gauss-Jordan method.
1 3 2 17
A|b = 1 2 3 16
2 -1 4 13
1 3 2 17
1 2 3 16
2 -1 4 13
Eliminating x in 2 & 3:
1 3 2 17
0 -1 1 -1
0 -7 0 -21
x + 3y + 2z =17
x + 2y + 3z =16
2x y + 4z =13
Normalize 2:
1 3 2 17
0 1 -1 1
0 -7 0 -21
Eliminating y in 1 & 3:
1 0 5 14
0 1 -1 1
0 0 -7 -14
1
2
3
Normalize 3:
1 0 5 14
0 1 -1 1
0 0 1 2
Eliminating z in 1 & 2:
1 0 0 4
0 1 0 3
0 0 1 2
x
y
=4
=3
z =2
Gauss-Jordan Process
26
Problems:
Solve the following system of equations using
the Gauss-Jordan method.
1) x 2y = -4
5y + z = -9
4x 3z = -10
2) 2x1 + x2 3x3 = 11
4x1 2x2 + 3x3 = 8
-2x1 + 2x2 x3 = -6
Problem:
Solve the following system of equations
2x + 6y + z = 7
x + 2y z = -1
5x + 7y 4z = 9
using:
(a) Gaussian elimination and
(b) Gauss-Jordan elimination
27
2.7
Jacobis Iteration
Method
28
Jacobi Iteration
Consider the equation:
3x + 1 = 0
which can be cast into
an iterative scheme as:
2x = -x 1
x=
x +1
2
1
1
x k +1 = x k
2
2
Iterations:
x1 = -2-1 2-1x0
where x0 is the initial guess
x2 = -2-1 2-1x1
x3 = -2-1 2-1x2
Will it always converge?
another iterative scheme
x = -2x 1
xk+1 = -2xk 1
Will this converge?
29
30
Example:
Solve the following equations
by Jacobis method.
15x + 3y 2z = 85
2x + 10y + z = 51
x 2y + 8z = 5
------------------------------------------------------------------
= 5.045
y3 = [51 2(4.73) 1(1.192)]/10
= 4.035
z3 = [5 1(4.73) + 2(3.904)]/8
= 1.010
------------------------------------------------------------------
------------------------------------------------------------------
= 4.73
= 4.994
y4 = [51 2(5.045) 1(1.010)]/10
= 3.99
= 3.904
z4 = [5 1(5.045) + 2(4.035)]/8
z2 = [5 17/3 (-2)(51/10)]/8
= 1.003
= 1.192
y
5.100
z
0.625
4.730
5.045
4.994
3.904
4.035
3.990
1.192
1.010
1.003
5.002
5.000
5.000
5.000
4.001
4.000
4.000
4.000
0.998
1.000
1.000
1.000
5.000
4.000
1.000
To check:
15x + 3y 2z = 85
2x + 10y + z = 51
x 2y + 8z = 5
15(5)+3(4)2(1) = 85
75 + 12 2 = 85
2(5)+10(4)+1(1) = 85
10 + 40 + 1 = 51
5 2(4) + 8(1) = 85
58+8=5
31
Problem:
Use the Jacobi iterative scheme to obtain
the solutions of the system of equations
correct to three decimal places.
x + 2y + z = 0
3x + y z = 0
x y + 4z = 3
Problem:
Use Jacobi iterative scheme to obtain the
solution of the system of equations correct to
two decimal places.
32
2.8
Gauss-Seidel
Iteration Method
33
Gauss-Seidel method
aka the method of successive approximations
applicable to predominantly diagonal systems
PDS has large diagonal elements
34
8x + 2y 2z = 8
x 8y + 3z = -4
2x + y + 9z = 12
Iteration 2:
x2 = (b1 a12y1 a13z1)/a11
= [8 2(5/8) + 2(1.042)]/8 = 1.104
y2 = (b2 a21x2 a23z1)/a22
= [-4 1(1.104) 3(1.042)]/-8 =1.029
z2 = (b3 a31x2 a32y2)/a33
= [12 2(1.104) 1(1.029)]/9 = 0.974
Let: x0 = y0 = z0 = 0
3:
i Iteration
1
2
3
4
5
6
7
Iteration 1:
(b1 1.104
a12y20.986
a131.004
z2)/a11
x x3 =
1.000
0.999 1.000 1.000
x1 = b1/a11 = 8/8 = 1
=
[8
2(1.029)
+
2(0.974)]/8
= 0.986
y 0.625 1.029 0.988 1.002 0.999 1.000
1.000
y1 = (b2 a21x1)/a22
y
=
(b
a
x
a
z
)/a
21 31.004 230.999
2
22
z 3 1.042 2 0.974
1.000 1.000 1.000
= [-4 1(1)]/-8 = 5/8
= [-4 1(0.986) 3(0.974)]/-8 =0.988
z1 = (b3 a31x1 a32y1)/a33
z3 = (b3 a31x3 a32y3)/a33
= [12 2(1) 1(5/8)]/9 = 1.042 = [12 2(0.986) 1(0.988)]/9 = 1.004
35
Gauss-Seidel Method
The equations Ax = b are in scalar notation
36
Iterative Scheme
start by choosing the starting vector x
if a good guess is not available, x can be
chosen randomly
recompute each element of x, always using
the latest available values of xj
procedure is repeated until the changes in x
between successive iteration cycles become
sufficiently small
Relaxation
a technique used to improve the convergence of
the Gauss-Seidel method
take the new value of xi as a weighted average of
its previous value and the initial guess
37
Convergence Considerations
Convergence of the iterative schemes is ensured:
38
MatLab Functions
1/2
x = A\b
returns the solution x of Ax =b
obtained by Gauss elimination
[L,U] = lu(A)
Doolittle s decomposition A = LU
L = chol(A)
Choleski s decomposition A = LLT
B = inv(A)
returns B as the inverse of A
c = cond(A)
returns the condition number of the matrix A
MatLab Functions
2/2
A = spdiags(B,d,n,n)
creates a n n sparse matrix from the columns of matrix
B by placing the columns along the diagonals specified
by d
A = full(S)
converts the sparse matrix S into a full matrix A
S = sparse(A)
converts the full matrix A into a sparse matrix S
x = lsqr(A,b)
conjugate gradient method for solving Ax = b
spy(S)
draws a map of the nonzero elements of S
39
Exercises:
1) Determine the inverse
of the following matrices:
a)
Set 1
2) Solve the following
set of simultaneous
linear equations by the
matrix inverse method.
(a) 2x + 3y z = -10
-x + 4y + 2z = -4
2x 2y + 5z = 35
b)
(b) 10x + 3y + 10z = 5
8x 2y + 9z = 2
8x + y 10z =35
Exercises:
1) Solve using
Gaussian elimination:
a) 2x + y 3z = 11
4x 2y + 3z = 8
-2x + 2y z = -6
b) 6x + 3y + 6z = 30
2x + 3y + 3z = 17
x + 2y + 2z = 11
Set 2
2) Solve using GaussJordan method:
a) 4x 3y + 5z = 34
2x y z = 6
x + y + 4z = 15
b) 2x y + z = -1
3x + 3y + 9z = 0
3x + 3y + 5z = 4
40
Exercises:
Set 3
1) Solve using
Choleskis method:
a) 2x y = 3
-x + 2y z = -3
-y + z = 2
b) x + y + z = 7
3x + 3y + 4z = 23
2x + y + z = 10
b) x + y + z = 9
2x 3y + 4z = 13
3x + y + 5z = 40
Exercises:
Set 4
b)
b)
20x + y 2z = 17
3x + 20y z = -18
2x 3y + 20z = 25
2x y + 5z = 15
2x + y + z = 7
x + 3y + z = 10
41
Exercises:
Set 5
42