Chebyshev and Fourier Spectral Methods: Second Revised Edition

Chebyshev and Fourier

Spectral Methods

Second Edition (Revised)

JOHN P. BOYD

DOVER PUBLICATIONS, INC.

Mineola, New York

Copyright

Copyright © 2001 by John P. Boyd

All rights reserved under Pan American and International Copyright Conventions.

Bibliographical Note

This Dover edition, first published in 2001, is a revised and enlarged second edition of the work first published in 1969 by Springer-Verlag, Berlin, in the series Lecture Notes in Engineering.

Library of Congress Cataloging-in-Publication Data

Boyd, J. P. (John Philip), 1941-

Chebyshev and Fourier spectral methods / John P. Boyd.—2nd ed., rev.

            p. cm.

Includes bibliographical references and index.

ISBN 0-486-41183-4 (pbk.)

1. Chebyshev polynomials. 2. Fourier analysis. 3. Spectral theory (Mathematics) I. Title.

QA404.5 .B69 2001

515'.55—dc21

00-027475

Manufactured in the United States of America

Dover Publications, Inc., 31 East 2nd Street, Mineola, N.Y. 11501

Dedication

To Marilyn, Ian, and Emma

A computation is a temptation that should be resisted as long as possible.

— J. P. Boyd, paraphrasing T. S. Eliot

Contents

PREFACE

Acknowledgments

Errata and Extended-Bibliography

1    Introduction

1.1   Series expansions

1.2   First Example

1.3   Comparison with finite element methods

1.4   Comparisons with Finite Differences

1.5   Parallel Computers

1.6   Choice of basis functions

1.7   Boundary conditions

1.8   Non-Interpolating and Pseudospectral

1.9   Nonlinearity

1.10   Time-dependent problems

1.11   FAQ: Frequently Asked Questions

1.12   The Chrysalis

2     Chebyshev & Fourier Series

2.1   Introduction

2.2   Fourier series

2.3   Orders of Convergence

2.4   Convergence Order

2.5   Assumption of Equal Errors

2.6   Darboux’s Principle

2.7   Why Taylor Series Fail

2.8   Location of Singularities

2.8.1   Corner Singularities & Compatibility Conditions

2.9   FACE: Integration-by-Parts Bound

2.10   Asymptotic Calculation of Fourier Coefficients

2.11   Convergence Theory: Chebyshev Polynomials

2.12   Last Coefficient Rule-of-Thumb

2.13   Convergence Theory for Legendre Polynomials

2.14   Quasi-Sinusoidal Rule of Thumb

2.15   Witch of Agnesi Rule–of–Thumb

2.16   Boundary Layer Rule-of-Thumb

3     Galerkin & Weighted Residual Methods

3.1   Mean Weighted Residual Methods

3.2   Completeness and Boundary Conditions

3.3   Inner Product & Orthogonality

3.4   Galerkin Method

3.5   Integration-by-Parts

3.6   Galerkin Method: Case Studies

3.7   Separation-of-Variables & the Galerkin Method

3.8   Heisenberg Matrix Mechanics

3.9   The Galerkin Method Today

4     Interpolation, Collocation & All That

4.1   Introduction

4.2   Polynomial interpolation

4.3   Gaussian Integration & Pseudospectral Grids

4.4   Pseudospectral Is Galerkin Method via Quadrature

4.5   Pseudospectral Errors

5     Cardinal Functions

5.1   Introduction

5.2   Whittaker Cardinal or Sinc Functions

5.3   Trigonometric Interpolation

5.4   Cardinal Functions for Orthogonal Polynomials

5.5   Transformations and Interpolation

6     Pseudospectral Methods for BVPs

6.1   Introduction

6.2   Choice of Basis Set

6.3   Boundary Conditions: Behavioral & Numerical

6.4   Boundary-Bordering I

6.5   Basis Recombination

6.6   Transfinite Interpolation

6.7   The Cardinal Function Basis

6.8   The Interpolation Grid

6.9   Computing Basis Functions & Derivatives

6.10   Higher Dimensions: Indexing

6.11   Higher Dimensions

6.12   Corner Singularities

6.13   Matrix methods

6.14   Checking

6.15   Summary

7     Linear Eigenvalue Problems

7.1   The No-Brain Method

7.2   QR/QZ Algorithm

7.3   Eigenvalue Rule-of-Thumb

7.4   Four Kinds of Sturm-Liouville Problems

7.5   Criteria for Rejecting Eigenvalues

7.6   Spurious Eigenvalues

7.7   Reducing the Condition Number

7.8   The Power Method

7.9   Inverse Power Method

7.10   Combining Global & Local Methods

7.11   Detouring into the Complex Plane

7.12   Common Errors

8     Symmetry & Parity

8.1   Introduction

8.2   Parity

8.3   Modifying the Grid to Exploit Parity

8.4   Other Discrete Symmetries

8.5   Axisymmetric & Apple-Slicing Models

9     Explicit Time-Integration Methods

9.1   Introduction

9.2   Spatially-Varying Coefficients

9.3   The Shamrock Principle

9.4   Linear and Nonlinear

9.5   Example: KdV Equation

9.6   Implicitly-Implicit: RLW & QG

10   Partial Summation, the FFT and MMT

10.1   Introduction

10.2   Partial Summation

10.3   The Fast Fourier Transform: Theory

10.4   Matrix Multiplication Transform

10.5   Costs of the Fast Fourier Transform

10.6   Generalized FFTs and Multipole Methods

10.7   Off-Grid Interpolation

10.8   Fast Fourier Transform: Practical Matters

10.9   Summary

11   Aliasing, Spectral Blocking, & Blow-Up

11.1   Introduction

11.2   Aliasing and Equality-on-the-Grid

11.3   2 h-Waves and Spectral Blocking

11.4   Aliasing Instability: History and Remedies

11.5   Dealiasing and the Orszag Two-Thirds Rule

11.6   Energy-Conserving: Constrained Interpolation

11.7   Energy-Conserving Schemes: Discussion

11.8   Aliasing Instability: Theory

11.9   Summary

12   Implicit Schemes & the Slow Manifold

12.1   Introduction

12.2   Dispersion and Amplitude Errors

12.3   Errors & CFL Limit for Explicit Schemes

12.4   Implicit Time-Marching Algorithms

12.5   Semi-Implicit Methods

12.6   Speed-Reduction Rule-of-Thumb

12.7   Slow Manifold: Meteorology

12.8   Slow Manifold: Definition & Examples

12.9   Numerically-Induced Slow Manifolds

12.10   Initialization

12.11   The Method of Multiple Scales (Baer-Tribbia)

12.12   Nonlinear Galerkin Methods

12.13   Weaknesses of the Nonlinear Galerkin Method

12.14   Tracking the Slow Manifold

12.15   Three Parts to Multiple Scale Algorithms

13   Splitting & Its Cousins

13.1   Introduction

13.2   Fractional Steps for Diffusion

13.3   Pitfalls in Splitting, I: Boundary Conditions

13.4   Pitfalls in Splitting, II: Consistency

13.5   Operator Theory of Time-Stepping

13.6   High Order Splitting

13.7   Splitting and Fluid Mechanics

14   Semi-Lagrangian Advection

14.1   Concept of an Integrating Factor

14.2   Misuse of Integrating Factor Methods

14.3   Semi-Lagrangian Advection: Introduction

14.4   Advection & Method of Characteristics

14.5   Three-Level, 2D Order Semi-Implicit

14.6   Multiply-Upstream SL

14.7   Numerical Illustrations & Superconvergence

14.8   Two-Level SL/SI Algorithms

14.9   Noninterpolating SL & Numerical Diffusion

14.10   Off-Grid Interpolation

14.10.1   Off-Grid Interpolation: Generalities

14.10.2   Spectral Off-grid

14.10.3   Low-order Polynomial Interpolation

14.10.4   McGregor’s Taylor Series Scheme

14.11   Higher Order SL Methods

14.12   History and Relationships to Other Methods

14.13   Summary

15   Matrix-Solving Methods

15.1   Introduction

15.2   Stationary One-Step Iterations

15.3   Preconditioning: Finite Difference

15.4   Computing Iterates: FFT/Matrix Multiplication

15.5   Alternative Preconditioners

15.6   Raising the Order Through Preconditioning

15.7   Multigrid: An Overview

15.8   MRR Method

15.9   Delves-Freeman Block-and-Diagonal Iteration

15.10   Recursions & Formal Integration: Constant Coefficient ODEs

15.11   Direct Methods for Separable PDE’s

15.12   Fast Iterations for Almost Separable PDEs

15.13   Positive Definite and Indefinite Matrices

15.14   Preconditioned Newton Flow

15.15   Summary & Proverbs

16   Coordinate Transformations

16.1   Introduction

16.2   Programming Chebyshev Methods

16.3   Theory of 1-D Transformations

16.4   Infinite and Semi-Infinite Intervals

16.5   Maps for Endpoint & Corner Singularities

16.6   Two-Dimensional Maps & Corner Branch Points

16.7   Periodic Problems & the Arctan/Tan Map

16.8   Adaptive Methods

16.9   Almost-Equispaced Kosloff/Tal-Ezer Grid

17   Methods for Unbounded Intervals

17.1   Introduction

17.2   Domain Truncation

17.2.1   Domain Truncation for Rapidly-decaying Functions

17.2.2   Domain Truncation for Slowly-Decaying Functions

17.2.3   Domain Truncation for Time-Dependent Wave Propagation: Sponge Layers

17.3   Whittaker Cardinal or Sinc Functions

17.4   Hermite functions

17.5   Semi-Infinite Interval: Laguerre Functions

17.6   New Basis Sets via Change of Coordinate

17.7   Rational Chebyshev Functions: TBn

17.8   Behavioral versus Numerical Boundary Conditions

17.9   Strategy for Slowly Decaying Functions

17.10   Numerical Examples: Rational Chebyshev Functions

17.11   Semi-Infinite Interval: Rational Chebyshev TLn

17.12   Numerical Examples: Chebyshev for Semi-Infinite Interval

17.13   Strategy: Oscillatory, Non-Decaying Functions

17.14   Weideman-Cloot Sinh Mapping

17.15   Summary

18   Spherical & Cylindrical Geometry

18.1   Introduction

18.2   Polar, Cylindrical, Toroidal, Spherical

18.3   Apparent Singularity at the Pole

18.4   Polar Coordinates: Parity Theorem

18.5   Radial Basis Sets and Radial Grids

18.5.1   One-Sided Jacobi Basis for the Radial Coordinate

18.5.2   Boundary Value & Eigenvalue Problems on a Disk

18.5.3   Unbounded Domains Including the Origin in Cylindrical Coordinates

18.6   Annular Domains

18.7   Spherical Coordinates: An Overview

18.8   The Parity Factor for Scalars: Sphere versus Torus

18.9   Parity II: Horizontal Velocities & Other Vector Components

18.10   The Pole Problem: Spherical Coordinates

18.11   Spherical Harmonics: Introduction

18.12   Legendre Transforms and Other Sorrows

18.12.1   FFT in Longitude/MMT in Latitude

18.12.2   Substitutes and Accelerators for the MMT

18.12.3   Parity and Legendre Transforms

18.12.4   Hurrah for Matrix/Vector Multiplication

18.12.5   Reduced Grid and Other Tricks

18.12.6   Schuster-Dilts Triangular Matrix Acceleration

18.12.7   Generalized FFT: Multipoles and All That

18.12.8   Summary

18.13   Equiareal Resolution

18.14   Spherical Harmonics: Limited-Area Models

18.15   Spherical Harmonics and Physics

18.16   Asymptotic Approximations, I

18.17   Asymptotic Approximations, II

18.18   Software: Spherical Harmonics

18.19   Semi-Implicit: Shallow Water

18.20   Fronts and Topography: Smoothing/Filters

18.20.1   Fronts and Topography

18.20.2   Mechanics of Filtering

18.20.3   Spherical splines

18.20.4   Filter Order

18.20.5   Filtering with Spatially-Variable Order

18.20.6   Topographic Filtering in Meteorology

18.21   Resolution of Spectral Models

18.22   Vector Harmonics & Hough Functions

18.23   Radial/Vertical Coordinate: Spectral or Non-Spectral?

18.23.1   Basis for Axial Coordinate in Cylindrical Coordinates

18.23.2   Axial Basis in Toroidal Coordinates

18.23.3   Vertical/Radial Basis in Spherical Coordinates

18.24   Stellar Convection in a Spherical Annulus: Glatzmaier (1984)

18.25   Non-Tensor Grids: Icosahedral, etc

18.26   Robert Basis for the Sphere

18.27   Parity-Modified Latitudinal Fourier Series

18.28   Projective Filtering for Latitudinal Fourier Series

18.29   Spectral Elements on the Sphere

18.30   Spherical Harmonics Besieged

18.31   Elliptic and Elliptic Cylinder Coordinates

18.32   Summary

19   Special Tricks

19.1   Introduction

19.2   Sideband Truncation

19.3   Special Basis Functions, I: Corner Singularities

19.4   Special Basis Functions, II: Wave Scattering

19.5   Weakly Nonlocal Solitary Waves

19.6   Root-Finding by Chebyshev Polynomials

19.7   Hilbert Transform

19.8   Spectrally-Accurate Quadrature Methods

19.8.1   Introduction: Gaussian and Clenshaw-Curtis Quadrature

19.8.2   Clenshaw-Curtis Adaptivity

19.8.3   Mechanics

19.8.4   Integration of Periodic Functions and the Trapezoidal Rule

19.8.5   Infinite Intervals and the Trapezoidal Rule

19.8.6   Singular Integrands

19.8.7   Sets and Solitaries

20   Symbolic Calculations

20.1   Introduction

20.2   Strategy

20.3   Examples

20.4   Summary and Open Problems

21   The Tau-Method

21.1   Introduction

21.2   τ-Approximation for a Rational Function

21.3   Differential Equations

21.4   Canonical Polynomials

21.5   Nomenclature

22   Domain Decomposition Methods

22.1   Introduction

22.2   Notation

22.3   Connecting the Subdomains: Patching

22.4   Weak Coupling of Elemental Solutions

22.5   Variational Principles

22.6   Choice of Basis & Grid

22.7   Patching versus Variational Formalism

22.8   Matrix Inversion

22.9   The Influence Matrix Method

22.10   Two-Dimensional Mappings & Sectorial Elements

22.11   Prospectus

23   Books and Reviews

A   A Bestiary of Basis Functions

A.1   Trigonometric Basis Functions: Fourier Series

A.2   Chebyshev Polynomials: Tn(x)

A.3   Chebyshev Polynomials of the Second Kind: Un(x)

A.4   Legendre Polynomials: Pn(x)

A.5   Gegenbauer Polynomials

A.6   Hermite Polynomials: Hn (x)

A.7   Rational Chebyshev Functions: TBn (y)

A.8   Laguerre Polynomials: Ln(x)

A.9   Rational Chebyshev Functions: TLn(y)

A.10   Graphs of Convergence Domains in the Complex Plane

B   Direct Matrix-Solvers

B.1   Matrix Factorizations

B.2   Banded Matrix

B.3   Matrix-of-Matrices Theorem

B.4   Block-Banded Elimination: the Lindzen-Kuo Algorithm

B.5   Block and Bordered Matrices

B.6   Cyclic Banded Matrices (Periodic Boundary Conditions)

B.7   Parting shots

C   Newton Iteration

C.1   Introduction

C.2   Examples

C.3   Eigenvalue Problems

C.4   Summary

D   The Continuation Method

D.1   Introduction

D.2   Examples

D.3   Initialization Strategies

D.4   Limit Points

D.5   Bifurcation points

D.6   Pseudoarclength Continuation

E   Change-of-Coordinate Derivative Transformations

F   Cardinal Functions

F.1   Introduction

F.2   General Fourier Series: Endpoint Grid

F.3   Fourier Cosine Series: Endpoint Grid

F.4   Fourier Sine Series: Endpoint Grid

F.5   Cosine Cardinal Functions: Interior Grid

F.6   Sine Cardinal Functions: Interior Grid

F.7   Sinc(x): Whittaker cardinal function

F.8   Chebyshev Gauss-Lobatto (Endpoints)

F.9   Chebyshev Polynomials: Interior or Roots Grid

F.10   Legendre Polynomials: Gauss-Lobatto Grid

G   Transformation of Derivative Boundary Conditions

Glossary

Index

References

Preface

[Preface to the First Edition (1988)]

The goal of this book is to teach spectral methods for solving boundary value, eigenvalue and time-dependent problems. Although the title speaks only of Chebyshev polynomials and trigonometric functions, the book also discusses Hermite, Laguerre, rational Chebyshev, sine, and spherical harmonic functions.

These notes evolved from a course I have taught the past five years to an audience drawn from half a dozen different disciplines at the University of Michigan: aerospace engineering, meteorology, physical oceanography, mechanical engineering, naval architecture, and nuclear engineering. With such a diverse audience, this book is not focused on a particular discipline, but rather upon solving differential equations in general. The style is not lemma-theorem-Sobolev space, but algorithm-guidelines-rules-of-thumb.

Although the course is aimed at graduate students, the required background is limited. It helps if the reader has taken an elementary course in computer methods and also has been exposed to Fourier series and complex variables at the undergraduate level. However, even this background is not absolutely necessary. Chapters 2 to 5 are a self-contained treatment of basic convergence and interpolation theory.

Undergraduates who have been overawed by my course have suffered not from a lack of knowledge, but a lack of sophistication. This volume is not an almanac of unrelated facts, even though many sections and especially the appendices can be used to look up things, but rather is a travel guide to the Chebyshev City where the individual algorithms and identities interact to form a community. In this mathematical village, the special functions are special friends. A differential equation is a pseudospectral matrix in drag. The program structure of grids point/basisset/collocation matrix is as basic to life as cloud/rain/river/sea.

It is not that spectral concepts are difficult, but rather that they link together as the components of an intellectual and computational ecology. Those who come to the course with no previous adventures in numerical analysis will be like urban children abandoned in the wilderness. Such innocents will learn far more than hardened veterans of the arithmurgical wars, but emerge from the forests with a lot more bruises.

In contrast, those who have had a couple of courses in numerical analysis should find this book comfortable: an elaboration of familiar ideas about basis sets and grid point representations. Spectral algorithms are a new worldview of the same computational landscape.

These notes are structured so that each chapter is largely self-contained. Because of this and also the length of this volume, the reader is strongly encouraged to skip-and-choose.

The course on which this book is based is only one semester. However, I have found it necessary to omit seven chapters or appendices each term, so the book should serve equally well as the text for a two-semester course.

Although these notes were written for a graduate course, this book should also be useful to researchers. Indeed, half a dozen faculty colleagues have audited the course.

The writing style is an uneasy mixture of two influences. In private life, the author has written fourteen published science fiction and mystery short stories. When one has described zeppelins jousting in the heavy atmosphere of another world or a stranded explorer alone on an artificial toroidal planet, it is difficult to write with the expected scientific dullness.

Nonetheless, I have not been too proud to forget most of the wise precepts I learned in college English: the book makes heavy use of both the passive voice and the editorial we. When I was still a postdoc, a kindly journal editor took me in hand, and circled every single I in red. The scientific abhorrence of the personal pronoun, the active voice, and lively writing is as hypocritical as the Victorian horror of breast and pregnant. Nevertheless, most readers are so used to the anti-literature of science that what would pass for good writing elsewhere would be too distracting. So I have done my best to write a book that is not about its style but about its message.

Like any work, this volume reflects the particular interests and biases of the author. While a Harvard undergraduate, I imagined that I would grow up in the image of my professors: a pillar of the A. M. S., an editorial board member for a dozen learned journals, and captain and chief executive officer of a large company of graduate students and postdocs. My actual worldline has been amusingly different.

I was once elected to a national committee, but only after my interest had shifted. I said nothing and was not a nuisance. I have never had any connection with a journal except as a reviewer. In twelve years at Michigan, I have supervised a single Ph. D. thesis. And more than three-quarters of my 65 papers to date have had but a single author.

This freedom from the usual entanglements has allowed me to follow my interests: chemical physics as an undergraduate, dynamic meteorology as a graduate student, hydro-dynamic stability and equatorial fluid mechanics as an assistant professor, nonlinear waves and a stronger interest in numerical algorithms after I was tenured. This book reflects these interests: broad, but with a bias towards fluid mechanics, geophysics and waves.

I have also tried, not as successfully as I would have wished, to stress the importance of analyzing the physics of the problem before, during, and after computation. This is partly a reflection of my own scientific style: like a sort of mathematical guerrilla, I have ambushed problems with Padé approximants and perturbative derivations of the Korteweg-deVries equation as well as with Chebyshev polynomials; numerical papers are only half my published articles.

However, there is a deeper reason: the numerical agenda is always set by the physics. The geometry, the boundary layers and fronts, and the symmetries are the topography of the computation. He or she who would scale Mt. Everest is well-advised to scout the passes before beginning the climb.

When I was an undergraduate — ah, follies of youth — I had a quasi-mystical belief in the power of brute force computation.

Fortunately, I learned better before I could do too much damage. Joel Primack (to him be thanks) taught me John Wheeler’s First Moral Principle: Never do a calculation until you already know the answer.

The point of the paradox is that one can usually deduce much about the solution — orders-of-magnitude, symmetries, and so on — before writing a single line of code. A thousand errors have been published because the authors had no idea what the solution ought to look like. For the scientist, as for Sherlock Holmes, it is the small anomalies that are the clues to the great pattern. One cannot appreciate the profound significance of the unexpected without first knowing the expected.

The during-and-after theory is important, too. My thesis advisor, Richard Lindzen, never had much interest in computation per se, and yet he taught me better than anyone else the art of good scientific number-crunching. When he was faced with a stiff boundary value problem, he was not too proud to run up and down the halls, knocking on doors, until he finally learned of a good algorithm: centered differences combined with the tridiagonal elimination described in Appendix B. This combination had been known for twenty years, but was only rarely mentioned in texts because it was hard to prove convergence theorems.¹ He then badgered the programming staff at the National Center for Atmospheric Research to help him code the algorithm for the most powerful computer then available, the CDC 7600, with explicit data swaps to and from the core.

A scientist who is merely good would have stopped there, but Lindzen saw from the numerical output that equatorial waves in vertical shear satisfied the separation-of-scales requirement of singular perturbation theory. He then wrote two purely analytical papers to derive the perturbative approximation, and showed it agreed with his numerical calculations. The analysis was very complicated — a member of the National Academy of Sciences once described it to me, laughing, as the most complicated damn thing I’ve ever seen — but the final answers fits on one line.

In sad contrast, I see far too many students who sit at their workstation, month after month, trying to batter a problem into submission. They never ask for help, though Michigan has one of the finest and broadest collections of arithmurgists on the planet. Nor will they retreat to perturbation theory, asymptotic estimates, or even a little time alone in the comer.

It is all too easy to equate multiple windows with hard work, and multiple contour plots with progress. Nevertheless, a scientist by definition is one who listens for the voice of God. It is part of the fallen state of man that He whispers.

In order that this book may help to amplify those whispers, I have been uninhibited in expressing my opinions. Some will be wrong; some will be soon outdated.² Nevertheless, I hope I may be forgiven for choosing to stick my neck out rather than drown the reader in a sea of uninformative blandness. The worst sin of a thesis advisor or a textbook writer is to have no opinions.

[Preface to the Second Edition, January, 1999]

In revising this book ten years after, I deleted the old Chapter 11 (case studies of fluid computations) and Appendix G (least squares) and added four new chapters on eigenvalue problems, aliasing and spectral blocking, the slow manifold and Nonlinear Galerkin theory, and semi-Lagrangian spectral methods. All of the chapters have been updated and most have been rewritten. Chapter 18 has several new sections on polar coordinates. Appendix E contains a new table giving the transformations of first and second derivatives for a two-dimensional map. Appendix F has new analytical formulas for the Legendre-Lobatto grid points up to nine-point grids, which is sufficient for most spectral element applications.

My second book, Weakly Nonlocal Solitary Waves and Beyond-All-Orders-Asymptotics (Kluwer, 1998) has two chapters that amplify on themes in this volume. Chapter 8 is an expanded version of Appendices C and D here, describing a much wider range of strategies for nonlinear algebraic equations and for initializing interations. Chapter 9 explains how a standard infinite interval basis can be extended to approximate functions that oscillate rather than decay-to-zero at infinity.

Other good books on spectral methods have appeared in recent years. These and a selection of review articles are catalogued in Chapter 23.

My original plan was to build a bibliographical database on spectral methods and applications of spectral algorithms that could be printed in full here. Alas, this dream was overtaken by events: as the database grew past 2000 items, I was forced to limit the bibliography to 1025 references. Even so, this partial bibliography and the Science Citation Index should provide the reader with ample entry points into any desired topic. The complete database is available online at the author’s homepage, currently at http://www-personal.engin.umich.edu/~jpboyd. To paraphrase Newton, it is better to stand on the shoulders of giants than to try to recreate what others have already done better.

Spectral elements have become an increasingly important part of the spectral world in the last decade. However, the first edition, with but a single chapter on spectral elements, was almost 800 pages long. (Students irrevently dubbed it the Encyclopedia Boydica.) So, I have reluctantly included only the original chapter on domain decomposition in this edition. A good treatment of spectral elements in the lowbrow spirit of this book will have to await another volume.

Perhaps it is just as well. The bibliographic explosion is merely a symptom of a field that is still rapidly evolving. The reader is invited to use this book as a base camp for his or her own expeditions.

The Heart of Africa has lost its mystery; the planets of Tau Ceti are currently unknown and unreachable. Nevertheless, the rise of digital computers has given this generation its galleons and astrolabes. The undiscovered lands exist, in one sense, only as intermittent electric rivers in dendritic networks of copper and silicon, invisible as the soul. And yet the mystery of scientific computing is that its new worlds over the water, wrought only of numbers and video images, are as real as the furrowed brow of the first Cro-Magnon who was mystified by the stars, and looked for a story.

¹Alas, numerical analysis is still more proof-driven than accomplishment-driven even today.

²Surely, too, the book has typographical errors, and the reader is warned to check formulas and tables before using them.

Acknowledgments

The author’s work has been supported by the National Science Foundation through the Physical Oceanography, Meteorology, Computational Engineering and Computational Mathematics programs via grants OCE7909191, OCE8108530, OCE8305648, OCE8509923, OCE812300, DMS8716766 and by the Department of Energy. My leave of absence at Harvard in 1980 was supported through grant NASA NGL-22-007-228 and the hospitality of Richard Lindzen. My sabbatical at Rutgers was supported by the Institute for Marine and Coastal Sciences and the hospitality of Dale Haidvogel.

I am grateful for the comments and suggestions of William Schultz, George Delic, and the students of the course on which this book is based, especially Ahmet Selamet, Mark Storz, Sue Haupt, Mark Schumack, Hong Ma, Beth Wingate, Laila Guessous, Natasha Flyer and Jeff Hittinger. I thank Andreas Chaniotis for correcting a formula

I am also appreciative of the following publishers and authors for permission to reproduce figures or tables. Fig. 3.3: C. A. Coulson, Valence (1973), Oxford University Press. Fig. 7.3: H. Weyl, Symmetry (1952) [copyright renewed, 1980], Princeton University Press. Tables 9.1 and Figs. 9.1 and 9.2: D. Gottlieb and S. A. Orszag, Numerical Analysis of Spectral Methods (1977), Society for Industrial and Applied Mathematics. Fig. 12-4: C. Canuto and A. Quarteroni, Journal of Computational Physics (1985), Academic Press. Tables 12.2 and 12.3: T. Z. Zang, Y. S. Wong and M. Y. Hussaini, Journal of Computational Physics (1984), Academic Press. Fig. 13.1 and Table 13.2: J. P. Boyd, Journal of Computational Physics (1985), Academic Press. Fig. 14.3: E. Merzbacher, Quantum Mechanics (1970), John Wiley and Sons. Figs. 14.4, 14.5, 14.7, 14.8, 14.9, 14.10, and 14.11: J. P. Boyd Journal of Computational Physics (1987), Academic Press. Fig. 15.1: W. D’Arcy Thompson, Growth and Form (1917), Cambridge University Press. Fig. D.1 (wth changes): J. P. Boyd, Physica D (1986), Elsevier. Fig. D.2: E. Wasserstrom, SIAM Review (1973), Society for Industrial and Applied Mathematics.

I thank Gene, Dale, Dave and Terry of the Technical Illustration Dept., DRDA [now disbanded], for turning my rough graphs and schematics into camera-ready drawings.

I also would like to acknowledge a debt to Paul Bamberg of the Harvard Physics department. His lecturing style strongly influenced mine, especially his heavy reliance on class notes both as text and transparencies.

I thank Joel Primack, who directed my undergraduate research, for his many lessons. One is the importance of preceding calculation with estimation. Another is the need to write quick-and-rough reports, summary sheets and annotations for even the most preliminary results. It is only too true that otherwise, in six months all your computer output and all your algebra will seem the work of a stranger.

I am also thankful for Richard Goody’s willingness to humour an undergraduate by teaching him in a reading course. Our joint venture on tides in the Martian atmosphere was scooped, but I found my calling.

I am grateful for Richard Lindzen’s patient tolerance of my first experiments with Chebyshev polynomials. His running commentary on science, scientists, and the interplay of numerics and analysis was a treasured part of my education.

I thank Steven Orszag for accepting this manuscript for the Lecture Notes in Engineering series (Springer-Verlag) where the first edition appeared. The treatment of time-stepping methods in Chapter 10 is heavily influenced by his MIT lectures of many years ago, and the whole book is strongly shaped by his many contributions to the field.

I am appreciative of John Grafton and the staff of Dover Press for bringing this book back into print in an expanded and corrected form.

Lastly, I am grateful for the support of the colleagues and staff of the University of Michigan, particularly Stan Jacobs for sharing his knowledge of nonlinear waves and perturbation theory, Bill Schultz for many fruitful collaborations in applying spectral methods to mechanical engineering, and Bill Kuhn for allowing me to introduce the course on which this book is based.

Errata and Extended-Bibliography

These may be found on author’s homepage, currently at

http://www-personal.engin.umich.edu/jpboyd

Errata and comments may be sent to the author at the following:

jpboyd@umich.edu

Thank you!

Chapter 1

Introduction

I have no satisfaction in formulas unless I feel their numerical magnitude.

–Sir William Thomson, 1st Lord Kelvin (1824–1907)

It is the increasingly pronounced tendency of modem analysis to substitute ideas for calculation; nevertheless, there are certain branches of mathematics where calculation conserves its rights.

–P. G. L. Dirichlet (1805–1859)

1.1   Series expansions

Our topic is a family of methods for solving differential and integral equations. The basic idea is to assume that the unknown u(x) can be approximated by a sum of N + 1 basis functions ϕn(x):

When this series is substituted into the equation

where L is the operator of the differential or integral equation, the result is the so–called residual function defined by

Since the residual function R(x; an) is identically equal to zero for the exact solution, the challenge is to choose the series coefficients {an} so that the residual function is minimized. The different spectral and pseudospectral methods differ mainly in their minimization strategies.

1.2   First Example

These abstract ideas can be made concrete by a simple problem. Although large problems are usually programmed in FORTRAN and C, it is very educational to use an algebraic manipulation language like Maple, Mathematica, Macsyma or Reduce. In what follows, Maple statements are shown in bold face. The machine’s answers have been converted into standard mathematical notation.

The example is the linear, one-dimensional boundary value problem:

The exact solution is (Scraton, 1965)

Polynomial approximations are recommended for most problems, so we shall choose a spectral solution of this form. In order to satisfy the boundary conditions independently of the unknown spectral coefficients, it is convenient to write the approximation as u2:=1 + (1-x*x)*(a0 + a1*x + a2*x*x);

where the decision to keep only three degrees of freedom is arbitrary.

The residual for this approximation is Resid:= diff(u,x,x) – (x**6 + 3*x**2)*u;

As error minimization conditions, we choose to make the residual zero at a set of points equal in number to the undetermined coefficients in u2(x). This is called the collocation or pseudospectral method. If we arbitrarily choose the points xi = (–1/2,0,1/2), this gives the three equations: eq1:=subs(x=-1/2,Resid);

eq2:=subs(x=0,Resid); eq3:=subs(x=1/2,Resid);

The coefficients are then determined by solving eq1 = eq2 = eq3 = 0;

solutionarray:= solve({eq1, eq2, eq3}, {a0, a1, a2}); yields

Figure 1.1 shows that this low order approximation is quite accurate.

However, the example raises a whole host of questions including:

1. What is an optimum choice of basis functions?

2. Why choose collocation as the residual-minimizing condition?

3. What are the optimum collocation points?

4. Why is a1 zero? Could we have anticipated this, and used a trial solution with just two degrees of freedom for the same answer?

5. How do we solve the algebraic problem for the coefficients when the Maple solve function isn’t available?

The answer to the first question is that choosing powers of x as a basis is actually rather dangerous unless N, the number of degrees-of–freedom, is small or the calculations are being done in exact arithmetic, as true for the Maple solution here. In the next section, we describe the good choices. In an algebraic manipulation language, different rules apply as explained in Chapter 20.

The second answer is: Collocation is the simplest choice which is guaranteed to work, and if done right, nothing else is superior. To understand why, however, we shall have to understand both the standard theory of Fourier and Chebyshev series and Galerkin methods (Chapters 2 and 3) and the theory of interpolation and cardinal functions (Chapters 4 and 5).

The third answer is: once the basis set has been chosen, there are only two optimal sets of interpolation points for each basis (the Gauss-Chebyshev points and the Gauss-Lobatto points); both are given by elementary formulas in Appendices A and F, and which one is used is strictly a matter of convenience.