MAT 637: NUMERICAL LINEAR ALGEBRA

MAT613 ASSIGNMENT III:CHRISTMAS

ASSIGNMENT.
MABAKOU TCHIO LYNDA LAURIANE
Matricule: SC23P190
January 3, 2024

1. We are given A be a non singular n x n matrix and a column vector ~b ∈ Rn

(a) Define
i. the p-norm of the vector ~x ∈ Rn
Pn 1
It is define as ||~x||p = ( i=1 |xi |p ) p

ii. The surbordinate matrix norm ||A||p for the square ma-
trix A
For each p-morm in Rn there is an induced or surbordinate ma-
trix p-norm ||A||p where A ∈ Rn×n defined as

||A~x||p
||A||p = max ||A~x||p or ||A||p = max
||~
x||p =1 x||p 6=0 ||~
||~ x||p
iii. The spectral raduis ρ(A)
Let A be an n x n real matrix . Then if there exist a vector
x ∈ Rn and a scalar λ ∈ C such that Ax = λx then x and
λ are called the eigen vector and eigen value of A respectively.
Then the spectral raduis of A is defined as the maximum of the
absolute values of all the eigen values of A that is

ρ(A) = max{|λ| : Ax = λx}

iv. The Condition number κ(A)

This is a measure of how the pertubation of ~b would affect the

solution ~a
Required to show that

1
 a). ρ ≤ ||Ap ||

Let λi , i = 1, 2, · · · , n be the eigen values of the matrix A.

Then for each xi , Ax~i = λi x~i

=⇒ ||Ax~i ||p = ||λi x~i ||p

=⇒ ||Ax~i ||p = |λi |||x~i ||p since ||λi x~i ||p = |λi |||x~i ||p

=⇒ ||A||p ||x~i ||p ≥ |λi |||x~i ||p

=⇒ ||A||p ≥ |λi |
Thus for each i , ||A||p ≥ |λi |
b). κ(A) ≥ 1
By definition, κ(A) = ||A||||A−1 ||

Observe that for an n x n matrix A and a invertible this

means that the inverse exists and

AA−1 = In

||AA−1 || = ||In ||

=⇒ ||A||||A−1 || ≥ 1 since ||A||||A−1 || ≥ ||AA−1 || and ||In || = 1

=⇒ κ(A) ≥ 1
(b) Find ~x so that A~x = ~b
We are oing to briefly describe two direct methods and one iterative
method.

i). Gauss Elimination

We have two parts that is Elimination and the backward substitution

Elimination we proceed as follows ;

i) Write the given system A~x = ~b down.

2
2i) Reduce the augmented matrix using elementary row opera-
tions .

3i) Repeat step 2i) continuously until you obatain an upper tri-
angular matrix .

Remark

* Elementary row operations means here that at each step we

eliminate particular aij , i > j below the Leading diagonal using
aspecific multiplier and subtract or add from the next row.

** An upper triangular matrix is one in which the entries below

the leading diagonal are all zeros.
Backward Substitution we proceed as follows ;

After obtaining the upper triangular matrix above , the last en-
try in the leading diagonal can be use easily to solve the last
entry in the vector in xT and then substituted in the next equa-
tion with one extra unknown.

This can be shown explictly using the system A~x = ~b for an n×n
matrix below.

Consider the n × n system of equations A~x = ~b that is

    
a11 a12 . . . a1n x1 b1
 a21 a22 . . . a2n   x2   b2 
..   ..  =  ..  (1)
    
 .. ..
 . . ... .  .   . 
an1 an2 ... ann xn bn
Assume a11 6= 0 and define

mi1 = ai1 /a11 , i = 2, 3, . . . , n (multipliers)

Then we calculate a2ij = aij − mi1 ai1 , i = 2, 3, . . . , n

b2i = bi − mi1 b1 , i = 2, 3, . . . , n

this is done to obtain an (n−1)×(n−1) subsystem in x2 , x3 , . . . , xn .

We can adjoin the first equation in this subsystem to obtain the

n × n system A2 x = b2 .

Continuing recursively , and the next step assumes a222 6= 0 and

define.

3
 mi2 = a2i2 /a222 , i = 3, 4, . . . , n

Then we calculate
a3ij = a2ij − mi1 a22j , i = 3, 4, . . . , n

b3i = b2i − mi2 b22 , i = 3, 4, . . . , n

To obtain an (n − 2) × (n − 2) subsystem in x3 , x4 , . . . , xn .

We can adjoin the first two equations in this subsystem to obtain

the n × n system A(3) x = b3 .
We will repeat the step above until we find annn xn = bnn assuming
that annn 6= 0, as first step of the back substitution.

The whole back-substitution process is applied to system of equa-

tions of the form

   b 
a11 a12 a13 ... a1n x1 1
(2) 
 (2)
a22
(2)
a23 ...
(2)
a2n   x2  
   b2 
(3) 

  x3  
A(n) x =  (3)
a33 ...
(3)
a3n    =  b3 
 
  ..   . 
. .   .   .. 

 
(n)
ann xn bn
(n)

With anij = 0 ∀i > j, so that we have reduced the system to an

upper triangular system of equations . The last equation contain
only one unknown xn which is solved and the value used to solve
the rest of the unknown that is .

~b(n)
n
xn = (n)
ann
(i) Pn (i)
bi − j=i+1 aij xj
xi = (n)
aii
ii). Crammers Rule
We explain briefly how to use the Crammers rule to find the vec-
tor xT ∈ Rn .

This method consist in calculating determinants and dividing by

determinant of A. We proceed as follows.

4
 i) Write the given system A~x = ~b down.

2i) Calculate the determinant of the matrix A. and let det(A) =

∆

3i) To have the value of the first entry in the column vector (x1 )
of unknowns , we replace the firdt column of the matrix with the
column vector of constants then find its determinant∆1 .

4i) Divide the new determinant obtained in 3i) above by the

det(A) in 2i) so as to have the value of (x1 ).

5i) To calculate the second unknown , Repeat step 3i) but this
time put b in the place of the second column of A then do step
4i) to have the valuee of (x2 ).
∆n
6ı̂) For n vectors do the same that is xn = .
∆
This can be shown explictly using the system A~x = ~b for an n×n
(1).

System (1) will have a unique solution only when the determi-
nant of coefficients is not zero that is det(A) = ∆ 6= 0

Suppose det(A) = ∆ 6= 0, we compute ∆1 , ∆2 , . . . ∆n and that

our matrix A has i-rows and j-columns , we replace the 1st col-
umn of A by b to obtain A1 , then computing the det(A1 ) =
∆1 .Repeating this process just like in 5i) we obtain det(A1 ) = ∆1
. Hence det(An ) = ∆n

Thus the solution of our system is given as

∆1 ∆2 ∆n
x1 = , x2 = , . . . , xn =
∆ ∆ ∆
iii). The Jacobi Iterative Scheme
Recall that Iterative methods consist in splitting the matrix
A=N-P where N is a simple and invertible .For special itera-
tive scheme like The Jacobi Iterative Scheme we are going to use
the splitting A = D - L -U that is
   
a11 a12 . . . a1n a11 0 . . . 0
 a21 a22 . . . a2n   0 a22 . . . 0 
A= . , D =
   
.. .. .. .. .. 
 ..
 
. ... .   . . ... . 
an1 an2 ... ann 0 0 ... ann

5
    
0 0 ... 0 0 a12 ... a1n
 a21 0 ... 0 0 0 ... a2n 
−L =  . ..  , −U =  ..
   
.. .. .. 
 .. . ... . . . ... . 
an1 an2 ... 0 0 0 ... 0
For the Jocobi Iteration scheme, N = D , P = L+U

We have the following iteration for this scheme

A=N −P (1)

A~x = ~b (2)

Sub A in (2)

(2) =⇒ (N − P )x = b

~ = P x~(k) + b for the k - iteration , k =0,1,2,....

=⇒ N x(k+1)

~ = N −1 P x~(k) + N −1 b
=⇒ x(k+1)

~ = M x~(k) + C where M = N −1 P and N −1 b

=⇒ x(k+1)

M is further called the iteration matrix

To generate the Jacobi iteration sequence we consider the case

where A is a 3 x 3 matrix as shown for the system a~x = ~b as
shown below
    
a11 a12 a13 x1 b1
a21 a22 a23  x2  = b2 
a31 a32 a33 x3 b3

a11 x1 + a12 x2 + a13 x3 = b1 (1)

a21 x1 + a22 x2 + a23 x3 = b2 (2)

a31 x1 + a32 x2 + a33 x3 = b3 (3)

Assume a11 6= 0, a22 6= 0, a33 6= 0

Then from (1) , (2) ,(3) we gwt

6
 1
x1 = {b1 − a12 x2 − a13 x3 }
a11
1
x2 = {b2 − a21 x1 − a23 x3 }
a22
1
x3 = {b3 − a31 x1 − a32 x2 }
a33
(0) (0) (0)
Let (x1 , x2 , x3 ) be the initial guess to the solution ~xu and
this will help define the sequence of k -iterate below

(k+1) 1 (k) (k)

x1 = {b1 − a12 x2 − a13 x3 }
a11

(k+1) 1 (k) (k)

x2 = {b2 − a21 x1 − a23 x3 }
a22

(k+1) 1 (k) (k)

x3 = {b3 − a31 x1 − a32 x2 }
a33
Which is the Jacobi iteration sequence and generally for the i-th
componenet we have

(k+1) 1 (k) (k) (k)

xi = {bi −ai1 x1 −ai,i−1 xi −ai,i+1 xi+1 +· · ·+ai,n x(k)
n } (∗)
aii
(k)
(k+1) (k) ri
(∗) can also be written as xi = xi + (2∗)
aii
(k) Pn (k)
Where ri = bi − j=1 aij xj , i = 1, 2, · · · , n is the i-th com-
ponent of the residual.

To monitor errors and ensure convergence, we are going to state

a theorem and prove it.

Theorem: Suppose A =N-P and suppose ||N −1 P ||∞ =

λ < 1 Then the folllowing assertions holds

. A is non-singular.

. ||x(j) − x||∞ ≤ λ(j) ||x(0) − x||∞ or ||e(j) ||∞ ≤ λj ||e(0) ||∞

. If x is a solution of the Linear system Ax = b and (x(j) )j≥1 is a

sequence of points generated by the relation N x(k+1) = P x(k) +b
then lim x(j) = x
j7−→∞

7
Proof

. A is non-singular.

We assume contradiction that A is singular .If A is singular ,

then there exists y 6= 0 such that Ay = 0 .Therefore (N − P )y =
0 or y = N −1 P and so ||y||∞ = ||N −1 P y||∞ ≤ ||N −1 P ||∞ ||y||∞
since y 6= 0 , we must have λ ≥ 1 and this brings a contradiction
according to our hypothesis hence A is non-singular.

——e(j) ||∞ ≤ λj ||e(0) ||∞

Let e(k) be the error in the approximation at step k then e(k) =
~x − x~(k)

For N - P we have A~x = ~b to be

N~x = P ~x + b (1)

N~x(k+1) = P ~x(k) + b (2)

(1) - (2) =⇒ N (~x − ~x(k+1) ) = P (~x − ~x(k) )

N e(k+1) = P e(k)

e(k+1) = N −1 P e(k) = M e(k)

where M = N −1 P thus at step j we have j = k+1 =⇒ k = j−1,

e(j) = M e(j−1)
Now

e(j) = M e(j−1) = M 2 e(j−2) = M 3 e(j−3) = · · · = M (j) e(0)

e(j) = M (j) e(0)

Thus
||e(j) ||∞ = ||M (j) ||∞ ||e(0) ||∞ = λ(j) ||e(0) ||∞
If λ < 1, then lim λ(j) = 0 and we have lim x(j) = x
j7−→∞ j7−→∞

Which ensures errors and convergence in the Jacobi Iteration

matrix.

8
2(a) (i) The answer is NO this is because the convergence of the Jacobi and
Gauss-Seidel methods depends on the spectral raduis of the iteration
matrix which is different for each method
(ii) Let a be a 3 by 3 as shown below
   
1 2 3 1
A = 2 1 2 , b = 1
3 2 1 1
Observe that the coefficient matrix A is not diagonally dorminant ,
nor symmetric positive definite so the convergence of the Jacobi and
Gauss-Seidel methods not guaranteed.

For convergence of both methods the spectral raduis must be less

than 1 . For the coefficient matrix A , the spectral raduis for the Ja-
cobi and Gauss-Seidel methods are 0.82 and 1.24 respectively which
indicates that the Jacobi will converge but Gauss-Seidel method will
diverge for any initial guess.
 
0 1
(iii) We want to show that the matrix A = , has no LU decom-
1 0
position.

Recall that LU decomposition is the splitting of a matrix into a unit

lower triangular and upper triangular matrix.

Using contradiction we suppose A has an LU decomposition then A

= LU with L = a unit lower triangular matrix and U = upper trian-
gular matrix.

Then for a 2 by 2 matrix we have

   
l11 0 u11 u12
L= ,U=
l21 l22 0 u22
    
l 0 u11 u12 l u l11 u12
=⇒ A = 11 = 11 11
l21 l22 0 u22 l21 u11 l21 u12 + l22 u22

Comparing the above resukts with the matrix A given we observe

that a11 = l11 u11 = 0 =⇒ either l11 = 0 or u11 = 0.

This means that l11 u12 = 0 =⇒ a12 = 0 but a12 = 1from A which
contradics the assumption that a has an LU decomposition.

We want to show that A + I2 has an LU decomposition .

     
0 1 1 0 1 1
A + I2 = + =
1 0 0 1 1 1

9
 Let A + I2 = A1 and assume have an LU decomposition that is
    
l 0 u11 u12 l11 u11 l11 u12
A = LU = 11
1
=
l21 l22 0 u22 l21 u11 l21 u12 + l22 u22

Note that a11 = l11 u11 = 1 =⇒ l11 = 1, u11 = 1

a12 = l11 u12 = 1 =⇒ l11 = 1u12 = 1

a21 = l21 u11 = 1 =⇒ l21 = 1, u11 = 1

a22 = l21 u12 +l22 u22 since l21 = 1 and u12 = 1 then l22 = 0 and u22 =
0

With this information we have

   
1 0 1 1
L= and U =
1 0 0 0

    
1 0 1 1 1 1
=⇒ LU = = = A1
1 0 0 0 1 1

Therefore the singular matrix A + I2 = A1 admitts and LU decom-

position.
3. (a) Tthe first degree interpolatting polynomial Pm which interpolates u
is

P1 (x) = u[x0 ] + u[x0 , x1 ](x − x0 )

where m =1
4. f : R −→ R;
b
x −→ ax +
x

(0.5, 100), (1, 7), (3, 9), (6, 20), (10, 70)

We are required to use the method of least square approximations to

determine the constants a and b.
Wrinting the above information in matrix form we have

10
   
1 0.5 100
1 1    7 
  a  
1 3 b = 9  (1)
 

1 6   20 
1 10 70
Suppose ~x∗ is the least square solution then it solves the system A~x∗ = ~b

=⇒ AT A~x∗ = AT ~b

=⇒ ~x∗ = (AT A)−1 AT ~b (2)

From (1) we have
 
1 0.5
  1 1  
T 1 1 1 1 1  1
 5 20.5
A A= 3 =

0.5 1 3 6 10 1 20.5 147.25
6
1 10

|AT A| = (147.25 × 5) − (20.5 × 20.5) = 316

 
T 147.25 −20.5
Adj(A A) =
−20.5 5
 
T −1 1 147.25 −20.5
=⇒ (A A) =
316 −20.5 5
Substituting in (2) we get

 
100
   7 
1 147.25 −20.5 1 1 1 1 1 
~x∗ =

 9 
316 −20.5 5 0.5 1 3 6 10  
 20 
70
  
1 147.25 −20.5 206
=
316 −20.5 5 904
   
a 37.35
=⇒ =
b 0.94
0.94
Hence f (x) = 37.35x +
x

11