Contents 113. TWO STORAGE SCHEMES FOR GIVENS PLANE ROTATIONS ceveeeee 233 13.1, The George-Heath implementation of the Givens method 233 13.2. Use of @ dynamic storage schewe in connection with the Gentleman version of the Givens plane rotations ............. 238 13.3, Numerical results 240 13/4) Concluding remarks and references concening the implementation of orthogonslization methods in the solution of Linear least squares problems ......e..cccsses++ 242 14, PIVOTAL STRATEGIES FOR GIVENS PLANE ROTATIONS .....0.00000 243 14,1, Definition of two pivotal strategies 16.2. Computations with the same column interchanges 16.3) choice of test-matrices a 14.4. Comparison of the storage required by the two strategies 252 14:5. Comparison of the computing times 253 14.6. Comparison of the accuracy achieved by the two seeategiea "254 14.7. Robustness of the computations .... ws 255 14.8. Concluding remarks and references concerning the pivotal strategies in sparse codes based on orthogonal ization ....... 257 15. ITERATIVE REFINEMENT AFTER THE PLANE ROTATIONS ....0.00.cc0cese002 259 15.1, Statement of the problem Li) 259 15.2. Introduction of the iterative refinement process 260 15.3. Stopping criteria perenne . 2 261 15.4. Dropping "small" elements” 2) 262 15.5. Nunerical experiments .... TT 262 15.6. Comparison of the storage required by the two pivotal strategies . 263 15.7. Comparison of ‘the computing time required by the two pivotal strategies... cesses 265 15.8. Comparison of the accuracy achieved by the two pivotal strategies 265 15.9, Comparison of the robustneas of the two pivotal strategies. 266 15.10, Concluding remarks and references concerning the iterative refineaent process for orthogonal ization methods = 26 116. PRECONDITIONED CONJUGATE GRADIENTS FOR GIVENS PLANE ROTATIONS .... 269 16.1. The classical conjugate gradient algorithm ....... 269 16.2, Transforaation of the original problem on 16.3. A conjugate gradient algorithm for the transformed problem ... 273 16-4. Stopping criteria .... wee 216 16.5. Numerical results DOI oe 16.6. Concluding remarks and references concerning the conjugate gradient algorithm for orthogonslization methods 293 ‘REFERENCES: . ceeteeeseeeesees 295) AUTHOR INDEX a9 SUBJECT INDEX 323,PREFACE Matrices that contain many zero elements are called sparse. If the matrix is sparse then it is often possible to exploit this property so as to reduce (A) the storage and/or (44) the computing time needed vhen basic Linear algebra operations are carried out. Various sparse matrix techniques can be applied in the effort to achieve these two aims. Sparse matrix techniques that are designed for general matrices are the main topte of this book ‘The attempt to reduce the storage and/or the computing time in the computations involving sparse matrices 1s often carried out by avoiding both the storage of zero elements and the computations with zero elements. In many situations this approach, vhich will be called the classical manner of exploiting sparsity, is very efficient. However, in sone cases many fill-ins fare created during the performance of certain linear algebra operations vhen the classical manner of exploiting sparsity is used. 4 fill-in is a new non zero element, created at a position where the original matrix contains a zero element. Experience indicates that many fill-ins are often small in some Sense. Therefore it may sometines be worthwhile to apply @ second approach: check for small elements and reuove these from the arrays where the non-zero elements are kept (or, in other words, to consider these elements as zero elements during the subsequent computation). Sone of the snall elements produced during the calculations may be caused by rounding errors. Assume that fan element will be a zero element if the computations are performed by exact arithmetic. The computer vill as a rule produce a non-zero element (instead of a zero clement) in this situation because the computations are performed with rounding errors. Thus, some non-zeros appear only because the computer is in general not able to treat correctly the cases where zero elements will be produced by exact arithmetic, Hovever, such non-zero elements are small elements and they will normally be removed when the second approach is applied. This is the reason for calling the second approach the computer- oriented manner of exploiting sparsity. Both approaches are studied in detail in this book and many numerical results achieved in comparing of the two approaches are presented. Assume that the problem x-A'b is solved by a direct method (AcR™™, bene, xe", men, rank(A)=n). Then the coaputer-oriented manner can be used as an attempt to detect cancellations (1.e. to detect the situations vhere the modified elements would be zero elements if exact arithmetic were used). One should renove only very small elements in such an application. However, the computer-oriented manner of exploiting sparsity is normally more efficient with regard to storage and computing time when it is allowed to remove also elements that are not very small. In the latter situation both the storage and ‘the computing time may be reduced considerably, but, on the other hand, the solution computed in this way may be inaccurate. Therefore it is necess: try to improve the solution iteratively. The simplest way to do this is to apply straightforvard iterative refinement, which is studied in detail in Chapters 3, 5, 8 and 15. Hovever, other iterative methods can also be applied. Conjugate gradient-type methods’ are discussed in Chapters 11 and 16xi Preface ‘The computer-oriented manner of exploiting sparsity (together with sone procedure for iterative improvement) is not alvays better than the first approach (where no fill-ins are discarded and the solution obtained by the direct method is accepted without any attempt to improve it iteratively). How- fever, many experiments indicate that when the first approach performs better than the second one then the difference is not great. On the other hand, some very large scientific and engineering problems can be handled numerically only ‘when the second approach is applied. An example, a large-scale problem arising in nuclear magnetic resonance spectroscopy, is given as an illustration of this statement in Chapter 8. Even on large modern computers it is extrenely expensive, in terms of both storage and computing tine, to treat this problem numerically when the first approach is used, while the second approach performs quite satisfactorily. The conclusion ia that in a good package for sparse matrices it is desirable to have options in vhich the first approach is applied, but there must also be options in which the second approach is applied and the solution is improved iteratively. The second approach, the computer-oriented manner of exploiting sparsity, may be preferred even in the case where the classical manner of exploiting sparsity performs better with regard to storage and computing time. This is 0 because the accuracy that can be achieved by the second approach is normally higher than that achieved by the first approach and, what is even more important, a reliable error estimate can be obtained when the second approach is used (no error estimate can be found in an easy way when the first approach is chosen). Consider a system of Linear algebraic equations Ax-b (which can be obtained from the more general problem mentioned above by setting mn). Assume ‘that Gaussian elimination (GE) is used. Then it is popular to use an ILD (incomplete LU) factorization as a preconditioner and apply an iterative sethod. The computer-oriented manner of exploiting sparsity can be considered 45 an ILU factorization. The name ILU factorization is not used in this book, hovever, in order to avoid misunderstanding. Indeed, all versions of the TU factorization proposed in the literature are in fact applicable to only very special matrices. This is so because by definition any ILU factorization is performed under the assumption that at least two conditions is satisfied: (A) the GE process is carried out without pivoting, (At) a sparsity pattern is prescribed before the beginning fof the computations and all nevly created non-z0ro elements that do not fit into this pattern are removed (ie. non-zero elements are removed not because they are small, but because there 1s no place for them in the pattern prescribed in advance) Consider the first condition. Tt i# well known that the completion of the Ge process without pivoting can be guaranteed when the matrix is either symietric and positive definite or diagonally dominant. Assume that the matrix to be factorized belongs to the first of these two classes. Assune also that the ‘Cholesky factorization ia used. Then some extra requirements mist be imposed in order to guarantee the completion of the factorization when the sparsity pattern is prescribed in advance and all elements that do not fit into it are reaoved (as, for example, the requirement that the matrix is an Mematrix, Le. ays0 for ij and the entries of the inverse of A are