iterative solutions of linear systems

© All Rights Reserved

4 views

iterative solutions of linear systems

© All Rights Reserved

- Teknik Jwb Maths Spm-PANG
- Algebra Lineal
- Mathlab Sor
- Differential-Equations-with-Bou-John-Polking.pdf
- Elzaki Transform and a Bulge Function on Volterra Integral Equations of the Second Kind
- Cambridge Maths 7 Chapter 9
- NM Unit-I-1
- Unit_2.3
- MATLAB Applications in Chemical Engineering
- Chapter 8 - Two Dimensional Arrays
- a new approximation to the solution of the lineear matrix equation AXB%3dC.pdf
- diffyqs
- 472017.pdf
- Matlab Coding -- First Slopes
- diffyqs
- ch 6 cfa template
- T_Algebra II January 2012 State Wide Individual Test
- Cgl Answer Key
- solving system of linear equations unit plan template
- Matrix Structure

You are on page 1of 6

143

Consistent linear systems in real life are solved in one of two ways: by direct calculation

(using a matrix factorization, for example) or by an iterative procedure that generates

a sequence of vectors that approach the exact solution. When the coefficient matrix

is large and sparse (with a high proportion of zero entries), iterative algorithms can be

more rapid than direct methods and can require less computer memory. Also, an iterative

process may be stopped as soon as an approximate solution is sufficiently accurate for

practical work. However, iterative methods can also fail or be extremely slow for some

problems.

The simple iterative methods below were discovered long ago, but they provided the

foundation for what today is an active area of research at the interface of mathematics

and computer science.

Throughout the section, A is an invertible matrix. The goal of an iterative algorithm is

to produce a sequence of vectors,

x(0) , x(1) , . . . , x(k) , . . .

that converges to the unique solution of Ax b, say x , in the sense that the entries in

x(k) are as close as desired to the corresponding entries in x for all k sufficiently large.

To describe a recursion algorithm that produces x(k1) from x(k) , we write A

M N for suitable matrices M and N, and then we rewrite the equation Ax b as

Mx Nx b and

Mx Nx b

If a sequence x(k) satisfies

Mx(k1) Nx(k) b

(k 0, 1, . . .)

(1)

and if the sequence converges to some vector x , then it can be shown that Ax b. [The

vector on the left in (1) approaches Mx , while the vector on the right in (1) approaches

Nx b. This implies that Mx Nx b and Ax b.]

For x(k1) to be uniquely specified in (1), M must be invertible. Also, M should

be chosen so that x(k1) is easy to calculate. The iterative methods below illustrate two

simple choices for M.

Jacobis Method

This method assumes that the diagonal entries of A are all nonzero. Let D be the diagonal

matrix formed from the diagonal entries of A. Jacobis method uses D for M and D A

for N in (1), so that

Dx(k1) (D A)x(k) b

(k 0, 1, . . .)

In a real-life problem, available information may suggest a value for x(0) . For simplicity,

we take the zero vector as x(0) .

144

2

Chapter 2

Matrix Algebra

EXAMPLE 1

10x1

x2

x1 15x2

x1

x3

18

x3 12

x2 20x3

(2)

17

Take x(0) (0, 0, 0) as an initial approximation to the solution, and use six iterations

(that is, compute x(1) , . . . , x(6) ).

For some k, denote the entries in x(k) by (x1 , x2 , x3 ) and the entries in x(k1)

by (y1 , y2 , y3 ). The recursion

0 1

1 x1

18

10

0

0 y1

0 15

0 1 x2 12

0 y2 1

1 1

0 x3

17

0

0 20 y3

Solution

x(k1)

(D A)

x (k)

can be written as

10y1

x2 x3 18

15y2 x1

x3 12

20y3 x1 x2

17

and

y1 (x2 x3 18) 10

y2 (x1 x3 12) 15

y3 (x1 x2 17) 20

(3)

A faster way to get (3) is to solve the first equation in (2) for x1 , the second equation

for x2 , the third for x3 , and then rename the variables on the left as y1 , y2 , and y3 ,

respectively.

For k 0, take x(0) (x1 , x2 , x3 ) (0, 0, 0), and compute

x(1) (y1 , y2 , y3 ) (18 10, 12 15, 17 20) (1.8, .8, .85)

For k 1, use the entries in x(1) as x1 , x2 , x3 in (3) and compute the new y1 , y2 , y3 :

y1 [(.8) (.85) 18] 10 1.965

y2 [(1.8) (.85) 12] 15 .9767

y3 [(1.8) (.8) 17] 20 .98

Thus x(2) (1.965, .9767, .98). The entries in x(2) are used on the right in (3) to

compute the entries in x(3) , and so on. Here are the results, with calculations using

MATLAB and results reported to four decimal places:

x(0)

x(1)

x(2)

x(3)

x(4)

x(5)

x(6)

0

1.8

1.965

1.9957

1.9993

1.9999

2.0000

0 .8 .9767 .9963 .9995 .9999 1.0000

0

.85

.98

.9971

.9996

.9999

1.0000

2.6

145

3

If we decide to stop when the entries in x(k) and x(k1) differ by less than .001, then we

need five iterations (k 5).

This method uses the recursion (1) with M the lower triangular part of A. That is, M has

the same entries as A on the diagonal and below, and M has zeros above the diagonal.

See Fig. 1. As in Jacobis method, the diagonal entries of A must be nonzero in order

for M to be invertible.

*

*

A= *

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

F I GUR E 1

*

*

* ,

*

*

*

*

M= *

*

*

0

*

* *

* * *

* * * *

of A.

EXAMPLE 2 Apply the GaussSeidel method to the system in Example 1 with x(0)

0 and six iterations.

10x1 x2

x1 15x2

x1

Solution

x3 18

x3 12

x2 20x3

(4)

17

10

1

1

0

15

1

0 y1

0 1

1 x1

18

0 y2 0 0 1 x2 12

20 y3

0 0

0 x3

17

x(k1)

(M A)

x (k)

or

10y1 x2 x3 18

y1 15y2 x3 12

y1 y2 20y3 17

We will work with one equation at a time. When we reach the second equation, y1 is

already known, so we can move it to the right side. Likewise, in the third equation y1 and

y2 are known, so we move them to the right. Dividing by the coefficients of the terms

146

4

Chapter 2

Matrix Algebra

y1 (x2 x3 18) 10

y2 (y1 x3 12) 15

(5)

y3 (y1 y2 17) 20

Another way to view (5) is to solve each equation in (4) for x1 , x2 , x3 , respectively, and

regard the highlighted xs as the new values:

x1 (x2 x3 18) 10

x2 ( x1 x3 12) 15

x3 ( x1 x2 17) 20

(6)

Use the first equation to calculate the new x1 [called y1 in (5)] from x2 and x3 . Then use

this new x1 along with x3 in the second equation to compute the new x2 . Finally, in the

third equation, use the new values for x1 and x2 to compute x3 . In this way, the latest

information about the variables is used to compute new values. [A computer program

would use statements corresponding to the equations in (6).]

From x(0) (0, 0, 0), we obtain

x1 [(0) (0) 18] 10 1.8

x2 [(1.8)

Thus x(1) (1.8, .92, .986). The entries in x(1) are used in (6) to produce x(2) , and so

on. Here are the MATLAB calculations reported to four decimal places:

x(1)

x(2)

x(3)

x(4)

x(5)

x(6)

x(0)

2.0000

2.0000

2.0000

0

1.8

1.9906

1.9998

0 .92 .9984 .9999 1.0000 1.0000 1.0000

.986

.9995

1.0000

1.0000

1.0000

1.0000

0

Observe that when k is 4, the entries in x(k) and x(k1) differ by less than .001. The values

in x(6) in this case happen to be accurate to eight decimal places.

There exist examples where Jacobis method is faster than the GaussSeidel method,

but usually a GaussSeidel sequence converges faster, as in Example 2. (If parallel

processing is available, Jacobi might be faster because the entries in x(k) can be computed

simultaneously.) There are also examples where one or both methods fail to produce a

convergent sequence, and other examples where a sequence is convergent, but converges

too slowly for practical use.

Fortunately, there is a simple condition that guarantees (but is not essential for)

the convergence of both Jacobi and GaussSeidel sequences. This condition is often

satisfied, for instance, in large-scale systems that can occur during numerical solutions

of partial differential equations (such as Laplaces equation for steady-state heat flow).

2.6

5

147

of each diagonal entry exceeds the sum of the absolute values of the other entries in the

same row. In this case it can be shown that A is invertible and that both the Jacobi and

GaussSeidel sequences converge to the unique solution of Ax b, for any initial x(0) .

(The speed of the convergence depends on how much the diagonal entries dominate the

corresponding row sums.)

The coefficient matrix in Examples 1 and 2 is strictly diagonally dominant, but the

following matrix is not. Examine each row:

6 2 3

|6| |2| |3|

1 4 2

|4| |1| |2|

3 5

8

|8| |3| |5|

The problem lies in the third row, because |8| is not larger than the sum of the magnitudes of the other entries. The practice problem below suggests a trick that sometimes

works when a system is not strictly diagonally dominant.

PRACTICE PROBLEM

Show that the GaussSeidel method will produce a sequence converging to the solution

of the following system, provided the equations are arranged properly:

x1 3x2

x3 2

5x1 2x2 2x3

2.6

1

9

EXERCISES

x(0) 0 and three iterations. [M] Repeat the iterations until two

successive approximations agree within a tolerance of .001 in each

entry.

1. 4x1 x2 7

x1 5x2 7

2. 10x1 x2 25

x1 8x2 43

3. 3x1 x2

11

x1 5x2 2x3 15

3x2 7x3 17

x2

149

4. 50x1

x1 100x2 2x3 101

2x2 50x3 98

In Exercises 58, use the GaussSeidel method, with x(0) 0 and

two iterations. [M] Compare the number of iterations needed by

GaussSeidel and Jacobi to make two successive approximations

agree within a tolerance of .001.

5. The system in Exercise 1.

6. The system in Exercise 2.

8. The system in Exercise 4.

Determine which of the matrices in Exercises 9 and 10 are strictly

diagonally dominant.

9. a.

10. a.

5

4

4

3

3 2

2

3

b.

b.

9 5

2

5 8 1

2

1

4

5

3

1

3

6 4

1 4

7

written in Exercises 11 and 12. Compute the first three iterates

for GaussSeidel, with x(0) 0. Then rearrange the equations to

make GaussSeidel work, and compute two iterations. [M] Find

an approximation accurate to three decimal places.

11. 2x1 6x2 4

5x1 x2 6

12. x1 4x2 x3 3

4x1 x2

10

x2 4x3 6

148

6

Chapter 2

Matrix Algebra

dominant, and no rearrangement of the equations will make them

so. Nevertheless, both Jacobi and GaussSeidel produce convergent sequences. [M] Find how many GaussSeidel iterations are

needed to produce an approximation accurate to three decimal

places. Set x(0) 0.

13.

4x1 2x2 10

5x1 4x2 2

x1 5x2 3x3 10

x1 4x2 4x3 18

[M] In Exercises 15 and 16, try both Jacobis method and the

GaussSeidel method and describe what happens. Set x(0) 0. If

a sequence seems to converge, try to find two successive approximations that agree within a tolerance of .001.

x3 6

15. x1

x1 x2

3

x1 2x2 3x3 9

16.

2x1 3x2 2x3 7

2x1 2x2 3x3 4

Cx d. Under suitable hypotheses on C, this equation can

x(k1) Cx(k) d

(7)

A?

b. Take x(0) 0 and use (7) to find a formula for x(2) not

involving x(1) .

c. Verify by induction that if x(0) 0, then

x(k) d Cd C 2 d C k1 d

(k 1, 2, . . .)

recursion relation (1), where A is invertible, A M N, and

M is invertible. The vector e(k) x(k) x is the error in the

approximation of x by x(k) . The steps below imply that if

(M 1 N)k 0 as k , then e(k) 0, which means that the

sequence x(k) converges to the solution x .

(k)

b. Prove by induction that e(k) (M 1 N)k e(0) .

The system is not strictly diagonally dominant, so neither Jacobi nor GaussSeidel

is guaranteed to work. In fact, both iterative methods produce sequences that fail to

converge, even though the system has the unique solution x1 3, x2 2, x3 1.

However, the equations can be rearranged as

5x1 2x2 2x3

x1 3x2 x3 2

6x1 4x2 11x3 1

Now the coefficient matrix is strictly diagonally dominant, so we know GaussSeidel

works with any initial vector. In fact, if x(0) 0, then x(8) (2.9987, 1.9992, .9996).

- Teknik Jwb Maths Spm-PANGUploaded byhidayah
- Algebra LinealUploaded byIsaac Reyna de Garcia
- Mathlab SorUploaded byYohannes
- Differential-Equations-with-Bou-John-Polking.pdfUploaded byالعلم نور
- Elzaki Transform and a Bulge Function on Volterra Integral Equations of the Second KindUploaded byIOSRjournal
- Cambridge Maths 7 Chapter 9Uploaded byRughfidd Jfdhbigfhg
- NM Unit-I-1Uploaded byJanani sathya
- Unit_2.3Uploaded byLaxman Naidu N
- MATLAB Applications in Chemical EngineeringUploaded byJuanMuñoz
- Chapter 8 - Two Dimensional ArraysUploaded byAhmadali
- a new approximation to the solution of the lineear matrix equation AXB%3dC.pdfUploaded byAnonymous Vd26Pzpx80
- diffyqsUploaded byHamza Zahid
- Matlab Coding -- First SlopesUploaded byTheodor Munteanu
- 472017.pdfUploaded byKOLLA Venkatesh
- diffyqsUploaded bydjalilkadi
- ch 6 cfa templateUploaded byapi-254097238
- T_Algebra II January 2012 State Wide Individual TestUploaded byAnonScr573
- Cgl Answer KeyUploaded byDadi Rambabu
- solving system of linear equations unit plan templateUploaded byapi-351781754
- Matrix StructureUploaded byAbhay Parashar
- AHP newUploaded byAhmad Shahlan Peniaga Kereta
- Calculation of the Solution of Two-Dimensional Helmholtz Equation (Hegedus G., Kuczmann M., 2010)Uploaded bysamouille6666
- 2001-Investigation of the Stability and Steady State Response of Asymmetric Rotors, Using Finite Element FormulationUploaded byJoel Morales Perez
- 7th and 8th Compacted Scope and Sequence (1).pdfUploaded byAnonymous aheD8bm
- resumenEjecutivoIO2.docxUploaded byErikaUruchi
- Pure2x2 Working SystemUploaded byDOLARIZADOS Comunidad Networking Para Ganar Dinero En Equipo
- Introductory API for MatlabUploaded byRaymond Wong
- R_ Apply a Function to a Data Frame Split by FactorsUploaded byloshude
- oleson instructions project matlabUploaded byapi-273162418
- 12438_DS7Uploaded bynitish-bhardwaj-9810

- Job CreationUploaded byNaj Retxed Dargup
- Typesetting Documents in Scientific Work PlaceUploaded bymscafe
- Beamer Appearance Cheat SheetUploaded bygladioacuto
- poisson regression models.pdfUploaded byNaj Retxed Dargup
- Summated Rating Scales.pdfUploaded byNaj Retxed Dargup
- 4 11 PermutationsUploaded byNaj Retxed Dargup
- Preparation for Yellow Ginger DecoctionUploaded byNaj Retxed Dargup
- img149Uploaded byNaj Retxed Dargup
- pidsdps1313Uploaded byNaj Retxed Dargup
- survival.pdfUploaded bykhitish19
- CongratulationsUploaded byNaj Retxed Dargup
- Voice Qualities.pdfUploaded byNaj Retxed Dargup
- Camille-Problem-6.pdfUploaded byNaj Retxed Dargup
- Quantitative Critique ExampleUploaded byNaj Retxed Dargup
- ExhibitUploaded byNaj Retxed Dargup
- Sample Letter of IntentUploaded byNaj Retxed Dargup
- SamplingUploaded byNaj Retxed Dargup
- Sample Size FormulaUploaded byMuhammad Azeem
- Cybertec Universal Remote Controller_0005Uploaded byNaj Retxed Dargup
- Cybertec Universal Remote Controller_0004Uploaded byNaj Retxed Dargup
- Cybertec Universal Remote Controller_0003Uploaded byNaj Retxed Dargup
- Cybertec Universal Remote Controller_0002Uploaded byNaj Retxed Dargup
- Cybertec Universal Remote Controller_0001Uploaded byNaj Retxed Dargup
- HP Ink Cartridge Installation Guide_0001Uploaded byNaj Retxed Dargup
- HP Ink Cartridge Installation Guide_0002Uploaded byNaj Retxed Dargup
- Sample Gift CardUploaded byNaj Retxed Dargup

- WHP196_3Uploaded byTiệp Phạm Xuân
- Satellite communication pptUploaded bybhanu455
- Automatic BalancingUploaded bySamehibrahem
- Multi Stage Centrifugal PumpUploaded byTg Tarro
- Closed-Book Practice-Ch 07 (2016-12-24)(2).pdfUploaded byJuan
- kemagnetanUploaded byAnastasya Amalia
- nUploaded byVyne Nguyen
- TrussUploaded bycjcute91
- 260661Uploaded byOscar Salazar
- engr2227_apr03Uploaded byMohamed Alqaisi
- 9709_w15_ms_43Uploaded byyuke kristina
- Chandra Resume VIT(M.tech-VLSI Design) Analog DesignUploaded byAjay Kumar Mattupalli
- Guidance Note on Garden & Retaining WallsUploaded bythowchin
- Mobility Between UMTS and LTE RAN13!0!02Uploaded byklajdi
- Newton's CradleUploaded bybhattnp_620044101
- Test AnalysisUploaded byAldrin Paguirigan
- IMG_20171022_0001Uploaded byreza786
- Kinetic DataUploaded byCharlie CB Portner
- annurev-fluid-122414-034558Uploaded byTeja Dasari
- Source Test ManualUploaded bySudipto Ghosh
- 2accc1244ba9add521ca5605f397374de518Uploaded byArdhi Wiratama Yudha
- PhyUploaded byriya
- 155_BPD_02_EN_RUUploaded byakashverma
- Octane RatingUploaded bytaufany99
- 164747 Cambridge Learner Guide for o Level ChemistryUploaded byALI MASUM
- Analisi Sismica Wanxian BridgeUploaded byShadi
- MATH23X-EXIT-EXAM-REVIEW-MATERIAL-MULTIVARIABLE-CALCULUS.docxUploaded byJoachim De Leon
- Native PAGEUploaded byCeciliaLuna
- 01_02elements of VectorsUploaded byshahbazalam4a5
- science testUploaded byJasvinder Singh