Professional Documents
Culture Documents
Na-Chapter 1
Na-Chapter 1
1. Introduction
Many of the problems of the numerical analysis can be reduced to the problem
of solving linear systems of equations. Among the problems, which could be treated
are the solution of ordinary and/or partial differential equations by using finite
difference methods, the solution of systems of equations, the eigen-value problems of
mathematical physics, least squares fitting of data and polynomial approximation.
In the above problem there are many several different right hand sides b - for
example, k of them – and therefore k unknown vectors x to be found. If we
let B be the n-rowed, k-column matrix of right-hand sided, and let X be the
corresponding n–rowed, k –column matrix of solution vectors, then we are to
solve the linear system AX = B.
By a direct method, we mean a method, which after a certain finite number of steps
gives the exact solution, disregarding rounding errors. For systems
Axb
where the matrix A is full (that is, most of the elements of A are non-zero), direct
elimination methods are almost always the most efficient, but when A is sparse (that
is, a large proportion of the elements of A are zeros), iterative methods offer certain
advantages. Iterative methods give a sequence of approximate solutions, converging
when the number of steps tends to infinity. Thus the choice between direct and
iterative methods depends on the proportion and distribution as well as sign and size
of non-zero elements of A.
Linear system of equations where the matrix is triangular are particularly simple to
solve. A system
Uxb,
u n -1,n -1 x n -1 u n -1,n x n b n -1
u n,n x n b n
If we assume that ui,i 0, for i=1 (1) n, then the unknowns could be computed in the
order xn, xn-1, …, x1 (backward order) as follows
bn
xn
u n, n
b n -1 u n -1, n x n
x n -1
u n -1,n -1
....
n
bi u
j i 1
i, j xj
xi
u i,i
...
n
b1 u i, j x j
j 2
x1
u 1,1
Now we are ready to write a simple algorithm to find xi, i=n (-1) 1
x ( n) b(n)/u(n, n);
for i (n - 1) : (-1) : 1
{s 0;
for j i 1 : n
{s s u(i, j) * x(j); }
x(i) b(i) - s)/u(i, i) ;
}
It is very simply to convert the above algorithm to MATLAB code (m file to solve the
a linear system of equations with an upper triangle matrix form - save the file as
uppersys.m)
» n=3;
» b=[1;2;3]
b =
1
2
3
u =
» x=uppersys(n,u,b)
x =
-0.0100
0.0500
0.3333
» check = u*x-b
check =
1.0e-015 *
-0.1110
0
0
It is easy to show that the number of division and multiplication is 0.5*(n2 + n), we
left the proof as an exercise.
The most important among the direct methods for solving a general linear system of
equations is Gaussian elimination. The idea behind this method is to eliminate the
unknowns in a systematic way, in such a way that we end up with a triangular system,
which we know how to solve. Consider the system,
A n,n x n,1 b n,1 ,
We assume in the following that the matrix A is non-singular (det(A) 0), so the
system A x b has a unique solution (only one solution).
Now we assume a1,1 0.Then we could eliminate x1 from the last (n-1) equations by
subtracting from the ith equation the multiple
a i,1
m i,1 , i 2 (1) n
a 1,1
of the first equation. So the pattern for the last (n-1) equations become
. .
. .
. .
a (2)
n,2 xn a (2)
n,3 x 3 ... a (2)
n, n x n b n
(2)
( 2)
This is a system of (n-1) equations in the (n-1) unknowns xi , i =2 (1) n. If the a 2 , 2 0,
in a similar way, as in the elimination of x1from equations 2 (1) n, one could eliminate
x2 from the last (n-2) of these equations (n-1 equations). We get a system of (n-2)
equations in the unknowns xi , i= 3 (1) n, if we let
(2)
a i,2
m i,2 , i 3 (1) n
a (2)
2,2
n, n x n b n
a (n) (n)
Now, we gather the first equation from each step to get the following new system
(1)
a 1,1 x 1 a 1,2
(1)
x 2 ... a 1,(1)n x n b1(1)
2,2 x 2 ... a 2, n x n b 2
a (2) (2) (2)
...
n, n x n b n
a (n) (n)
Thus we have reduced the original system to triangular system, which as we have
seen could be easily solved.
for j k 1 : n
( k 1 )
a i, j a ( k)
i, j m i, k a
}
for k 1 : (n - 1)
for i k 1 : n
a i,k
{a i, k ;
a k, k
b i b i - a i, k b k ;
f or j k 1 : n
a i, j a i, j a i,k a k, j ;
}
the MATLAB code for the modified algorithm is as follows (save with fmatsys.m)
» n=3;
» a=magic(n)
a=
8 1 6
3 5 7
4 9 2
» b=[1;2;3]
b=
1
2
3
» [a,b]=fmatsys(n,a,b)
a=
b=
1.0000
1.6250
-0.4865
to find the unknown xi , i = 1 (1) n; we combine the code of triangularization and the
code of solving the upper triangular system, and the MATLAB code (save it as
gausseli.m) is as follows
function [x]=gausseli(n,a,b)
for k=1:n-1
for i=k+1:n
a(i,k)=a(i,k)/a(k,k);
b(i)=b(i)-a(i,k)*b(k);
for j=k+1:n
a(i,j)=a(i,j)-a(i,k)*a(k,j);
end;
end;
end;
x(n)=b(n)/a(n,n);
for i=n-1:-1:1
s=0;
for j=i+1:n
s=s+a(i,j)*x(j);
end;
x(i)=(b(i)-s)/a(i,i);
end;
x=x';
» n=3;
» a=magic(n)
a=
8 1 6
3 5 7
4 9 2
» b=[1;2;3]
b=
1
2
3
» x=gausseli(n,a,b)
x=
0.0500
0.3000
0.0500
» check=a*x-b
check =
1.0e-015 *
0
-0.2220
-0.4441
x 1 x 2 x 3 1
x 1 x 2 2 x 3 2 (* )
x 1 2 x 2 2 x 3 1
x 3 1
x 2 x 3 0
(2)
Which means that a 2,2 = 0, and we could not proceed as usual. The way to proceed
obviously is to interchange equations 2 and 3 before the next step, which in this case
actually gives us the triangular system directly.
(k)
Now we consider the general case that in step k we have a k,k = 0, but for some other
(k)
element a i, k , i = k+1 (1) n in column k must be non-zero, otherwise the matrix is
(k)
singular (det(A) = 0), assume that a r, k , so we could interchange rows k and r and
proceed with the elimination. Using the above technique, we can conclude that any
non-singular matrix could be reduced to triangular form using Gaussian elimination
combined with row interchange.
x 1 x 2 x 3 1
0 . 00 0 1 x 2 2 x 3 1 ( ** )
9 99 9 x 3 1 0 00 0
x1 0 , x2 0 , x 3 1.000
where the true solution rounded to four decimals is
It is left to the reader to verify that if we interchange rows 2 and 3, then using same
precision the computed solution becomes
r, k max a i, k , k i n .
a (k) (k)
p(i)
k k
or
r r
Now, we are ready to write the algorithm for Gaussian elimination (with partial
pivoting – permutation vector) and back-substitution
for k 1 : n
{ p(k) k; }
for k 1 : n - 1
{m ax0 a k
l ocmax p
for i k 1
{ i f a i , k
{ max0
l ocm ax
}
if p(k)
{ r p (
for i k 1
{ a p (i ) ,k
for j k
a p ( i ) , j
}
}
a
p (n ) ,n 1
x n ;
a p ( n ) ,n
for k n - 1 : -1
{s 0 ;
for j k 1
s s a
a p( k ) , j
x k
a p ( k
Now, it is easy to write the MATLAB code for the general direct algorithm to solve
system of linear equations, a possible code for the above algorithm is as follows (save
the file with gepartpb.m, we augment the Matrix A to include b as the (n+1)th column
» n=3;
» a=[1 1 1 1; 1 1 2 2; 1 2 2 1]
a=
1 1 1 1
1 1 2 2
1 2 2 1
» x=gepartpb(n,a)
locmax =
x=
1
-1
1
» check=a(:,1:3)*x-a(:,4)
check =
0
0
0
Now we test the code for, when we change the value of a(2,2) to 1.0001)
» a(2,2)=1.0001
a=
» x=gepartpb(n,a)
locmax =
x=
1.0000
-1.0001
1.0001
» check=a(:,1:3)*x-a(:,4)
check =
0
0
0
We consider very important special case of system of linear equation, in which the
elements distributed on the diagonal and above and below that diagonal, and the
system has the following form ( for a system of n = 4)
b1 c1 0 0 x1 d1
a b2 c2 0 x d
2 2 2
0 a3 b3 c3 x 3 d 3
0 0 a4 b4 x 4 d 4
Instead of storing the above system in matrix form with a very large number of zero
elements (especially in case of large n), we store the system in vector format: the
diagonal elements in vector b of order n, and the elements below the diagonal a vector
a also of order n and the elements above the diagonal in a vector c of order n with a(1)
= c(n) = 0, and we apply the elimination process of one row below the diagonal. The
benefits for solving such systems using this technique (Thomas algorithm) instead of
using the general elimination (Gauss elimination on a full matrix) is to reduce the
number of arithmetic operation form O(n3) to O(n), and we sketch the algorithm in the
following steps
Now we are ready to develop the algorithm to solve the tri-diagonal system (Thomas
algorithm), as follows
c1 c 1 /b 1 ; d 1 d 1 /b 1 ;
for i 2 : n
{ b i b i c i 1 * a i ;
ci c i /b i ;
d i d i d i 1 * a i /b i ;
}
xn d n ;
for k n - 1 : -1 : 1
{x k d k ck * xk 1 ;}
function x=thomas(n,a,b,c,d)
c(1)=c(1)/b(1);
d(1)=d(1)/b(1);
for i=2:n
b(i)=b(i)-a(i)*c(i-1);
c(i)=c(i)/b(i);
d(i)=(d(i)-d(i-1)*a(i))/b(i);
end;
x(n)=d(n);
for k=n-1:-1:1
x(k)=d(k)-c(k)*x(k+1);
end;
x=x';
» n=5;
» a=[0;1;1;1;1]
a =
0
1
1
1
1
» b=[-4;-4;-4;-4;-4]
b =
-4
-4
-4
-4
-4
» c=[1;1;1;1;0]
c =
1
1
1
1
0
» d=[1;1;1;1;1]
d =
1
1
1
1
1
» x=thomas(n,a,b,c,d)
x =
-0.3654
-0.4615
-0.4808
-0.4615
-0.3654
» check=a(2)*x(1)+b(2)*x(2)+c(2)*x(3)-d(2)
check =
-1.1102e-016