SERIES EDITOR'S PREFACE. “BL my 8 Fat ou comment en revit, ‘One service mathematics bat render the je ny srs pint ale" aman race It as pt common some bac Jeles Vere whore belong, othe topmost shelf tex {o the dy canter Ibelled "vero non: ‘The seri i divergent; therfore we may be oe “histo do something with eT. Ba 0. Meaiite ‘Mathematics is a tool for thought. A highly necessary tool in a world where both feedback and non= linearties abound. Similarly all kinds of parts of mathematics serve as tools for other parts and for other sciences. Applying a simple rewriting rule to the quote on the right above one finds such statements as: “One service topology has rendered mathematical physis .."; ‘One service logic has rendered com- puter science . (One service category theory has rendered mathematics .”. All arguably true, And all statements obtainable this way form part of the raison d'&re ofthis series ‘This series, Mathematics and 11s Appicavions, started in 1977. Now that over one hundred volumes have appeared it seems opportune to reexamine its scope. At the time T wrote “Growing specialization and diversification have brought host of monographs and textbooks on increasingly specialized topics. However, the ‘irec’ of knowledge of mathematics and related fields does not grow only by putting forth new branches. It also happens, quite often in fact, that branches which were thought 10 be completely ‘disparate are'suddenly seen to be related. Further, the kind and level of sophistication ‘of mathematics applied in various sciences has changed drastically in recent years: measure theory is used (non-trivally) in regional and theoretical economics; algebraic geometry interacts with physics; the Minkowsky lemma, coding theory and the structure fof water meet one another in packing and covering theory; quantum fields, crystal defects and mathematical programming profit from homotopy theory: Lie algebras are relevant to filtering; and prediction and electrical engineering can use Stein spaces. And in addition to this there are such new emerging subdisciplines as. ‘experimental mathematics’, ‘CFD’, ‘completely integrable systems’, ‘chaos, synergetics and large-scale ‘order’, which are almost impossible to fit into the existing classification schemes, ‘They ‘draw upon widely different sections of mathematics By and large, allthis still applies today. It is still true that at first sight mathematics seems rather fragmented and that to find, see, and exploit the deeper underlying interelations more effort is needed and so are books that can help mathematicians and scientists do so, Accordingly MIA will continue to try to make such books available. If anything, the description T gave in 1977 is now an understatement. To the examples of interaction areas one should add string theory where Riemann surfaces, algebraic geometry, modu- Jar functions, knots, quantum field theory, Kac-Moody algebras, monstrous moonshine (and more) all. come together. And to the examples of things which can be usefully applied let me add the topic “finite geometry’; a combination of words which sounds like it might not even exist, let alone be applicable. And yet itis being applied: to statistics via designs, to radar/ sonar detection arrays (via finite projective planes), and to bus connections of VLSI chips (via difference ses). There seems to ‘be no part of (so-called pure) mathematics that is not in immediate danger of being applied. And, ‘accordingly, the applied mathematician needs to be aware of much more. Besides analysis and traditional workhorses, he may need all kinds of combinatorics, algebra, probability, In addition, the applied scientist needs to cope increasingly with the nonlinear world and thevi SERIES EDITOR'S PREFACE ‘extra mathematical sophistication that this requires. For that is where the rewards are. Linear ‘models are honest and a bit sad and depressing: proportional efforts and results. It is in the non- linear workd that infinitesimal inputs may result in macroscopic outputs (or vice versa). To appreci- ate what I am hinting at: if electronics were linear we would have no fun with transistors and com- puters; we would have no TV; in fact you would not be reading these lines. ‘There is also no safety in ignoring such outlandish things as nonstandard analysis, superspace and anticommuting integration, p-adic and ultrametric space. All three have applications in both electrical engineering and physics. Once, complex numbers were equally outlandish, but they fe~ quently proved the shortest path between ‘real’ results. Similar amed have already provided a number of ‘wormhole’ paths. There is no telling where all this is leading - ortunately. ‘Thus the original scope of the series, which for various (Sound) reasons now comprises five sub- series: white Gapan), yellow (China), red (USSR), blue (Eastern Europe), and green (everything clse), still applies. It has been enlarged a bit to include books treating of the tools from one subdis- cipline which are used in others. Thus the series still aims at books dealing with: = a central concept which plays an important role in several different mathemat scieniic specialization areas; = new applications of the results and ideas from one area of scientific endeavour into another; + influences which the results, problems and concepts of one field of enquiry have, and have had, ‘on the development of another al and/or ‘Mathematics is like Janus. It is about understanding structures, which helps to calculate them; it is, also about calculating structures in order to understand them. By and large we understand ‘mathematics quite well. But the structures one might meet in practice can get quite large (size 10°x 10° for instance) and calculating with these becomes both a computational and theoretical challenge. Fortunately, many of these large matrices arising from applications tend to have special properties. For instance they can be sparse, which, roughly, means los of zeros’. And the theoreti cal and computational challenge becomes how to make effective use of that property. ‘There are several ways of exploiting sparsity. This book, by a world expert, surveys them and compares them and doing soit discusses quite a few topics which are not included (Wo date) in other books on sparse matrices; for instance complexity, parallelism, orthogonalization, and condition ‘umber aspects. ‘As almost always there is no best way of exploiting sparseness and no single algorithm is satis- factory in all cases. Thus one needs guidance in using computer packages and devising one's own algorithms. This book will provide that “The soret path Retcen wo taht in the [Never lend bask, for no one ever return real domain patios through the compen than: the only books T have ia my brary oma sre book ha ter fk ve let me- 5. Hadamard ‘Anatole France La physique ne nove donne pat seement ‘The fnaion ofan expert not 10 be more Tocasion de retoudee det problimes elle ‘ight than eter people, but tobe wrong for ‘ou fit preset a saat. smote sophiicala resent. Poincaré David Baer ‘Amsterdam, August 1991 Michiel HazewinkelCONTENTS 1, EXPLOITING SPARSITY 2, STORAGE SCHEMES ‘an input storage schene fT Performance of static linear algebra operations Dynamic Linear algebra operations Dynamic storage scheme .....-..-+ Storage of fi11-ins Performance of garbage collections Need for elbow room Application of the storage scheme on a small matrix Initialization of the dynamic storage scheme Use of Linked lists in the dynamic storage schemes application of static storage schemes vs.......+ The storage scheme of Gilbert-Peierls The vectorizable storage scheme in ITPACK Other storage schemes Using principles of graph theory Reduction of the integer locations used in the Lists hy are the methods studied here based on the dynamic ‘storage schene? : Concluding remarks and references for the storay schemes 3. GENERAL SCHEME FOR LINEAR ALGEBRAIC PROBLEMS 3. 3 3 3 1 Application to some particular methods Introduction of the general scheme General approach for solving sparse problems by using particular algorithms from the general k-stage scheme Application of iterative refinement in connection with the general k-stage scheme . Relationship between the convergence of the iterative | refinenent process and the drop-tolerance used . Inplenentation of the comon aproach in the solution of sparse algebrate problems by some particular direct methods ‘Testing the subroutines in which the common approach for solving sparse problems x= ATb is implemented |. Amumerical illustration of the relationship between the drop-tolerance and the condition number . Concluding remarks and references concerning the general scheme for solving linear algebraic problems 4, PIVOTAL STRATEGIES FOR GAUSSIAN ELIMINATION 4 a 1. 2. Pivoting for dense matrices Common principles in the construction of pivotal | strategies for sparse matrices Pivotal strategies with a priori interchanges : Pivotal strategies based on the use of the Markowitz cost-fFunction 50 32 33 36 37 62 6 6 69 n 1B. Pivotal. sexs Choice of a 1 Goneluding gies OF ITERATIVE Convergence The drop-tol Storage comparisons Computing Different. When is the Large drop- Concluding contents ategies based on a local minimization of fill-ins pivotal strategy in a package for sparse matrices eaarks and references concerning the pivotal for Gaussian elimination fetes REFINEMENT IN THE GE PROCESS of the IR process sLerance whee ime BIN lays of using the IR’ process IR process efficient? : : ‘tolerance versus large stability factor remarks and references concerning the IR process IMPLEMENTATION OF THE ALGORITHMS TA 72 13 1a. 1.3. Inplenentat: |; Implenentat On the stor: Solving syst |; Inplenentat Inplenentat! Concluding ‘the implemes - SOLVING LEAST SQUARES PROBLEMS BY AUGMENTATION Forming the Using. packa, squares prot The choice problems by Using small Least square Concluding of Linear 1 Transition Comparing tt Comparing th 2 Numerical {Dropping small elements during the GE proc on of Gaussian elimination ion of a pivotal strategy .... age of the lover triangular matrix. L stems with triangular matrices on of iterative improvement. ‘ion of parallel algorithas ‘remarks and references concerning mntation of the algorithas auguented matrix ge Y12M in the solution of linear least blems by augmentation ‘of a@ in the solution of linear least squares ‘the method of augmentation values of a in the solution of Linear e3 problems by augmentation a remarks and references concerning the solution feast squares problems by augmentation . PARSE MATRIX TECHNIQUE YOR ORDINARY DIFFERENTIAL BQUATIONS . Statement of . The integra of the problem... stion algorithm used in the software from @ dense code to a sparse code |. he storage requirenents of the two codes the computing tine for the two codé sxperiments with DENSL and SPARL Dropping small elements before the start of the GE process” 2 automatic dé Application to difficult problems Sone extensi * Concluding of ODE's jevermination of the drop-tolerance ‘ions of the results fo ‘emarks and references concerning the solution by using sparse matrix technique . 2 102 104 + 106 87 87 90 92 96 107 109 109 ere 3 aa 216 ur us - 1a Daa 123 - 1s 17 130 ai fast 133 D139 2 140 1a arr Das 7 148. D150 152 153