George Bachman Lawrence Narici Edward Beckenstein FOURIER AND WAVELET ANALYSISGeorge Bachman Lawrence Narici Edward Beckenstein Fourier and Wavelet Analysis 6 SpringerGeorge Bachman Lawrence Narici Edward Beckenstein Professor Emeritus Department of Mathematics Science Division of Mathematics and Computer Science St. John’s University Polytechnic University St. John’s University Staten Island, NY 10301 5 Metrotech Center Jamaica, NY 11439 USA Brooklyn, NY 11201 USA USA Editorial Board (North America): S. Axler FW. Gehring Mathematics Department Mathematics Department San Francisco State University East Hall San Francisco, CA 94132 University of Michigan USA Ann Arbor, MI 48109-1109 USA K.A. Ribet Department of Mathematics University of California at Berkeley Berkeley, CA 94720-3840 USA Mathematics Subject Classification (1991): 42Axx, 42Cxx, 41-xx Library of Congress Cataloging-in-Publication Data Bachman, George, 1929- Fourier and wavelet analysis / George Bachman, Lawrence Narici, Edward Beckenstein. P._ em. — Universitext) Includes bibliographical references and index. ISBN 0-387-98899-8 (alk. paper) I. Fourier, analysis. 2. Wavelets (Mathematics)_I. Beckenstein, Edward, 1940- IL. Narici, Lawrence, IIL. Title, IV. Series. QA403.5.828 2000 515/.2433—de21 99-36217 Printed on acid-free paper. © 2000 Springer-Verlag New York, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. ‘The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Production managed by Michael Koy; manufacturing supervised by Jeffrey Taub, ‘Camera-ready copy prepared by the author. Printed and bound by R.R. Donnelley and Sons, Harrisonburg, VA. Printed in the United States of America. 987654321 ISBN 0-387-98899-8 Springer-Verlag New York Berlin Heidelberg SPIN 10491067Not long ago many thought that the mathematical world was created out of analytic functions. It was the Fourier series which disclosed a terra incognita in a second hemisphere. —E. B. van Vleck, 1914 The Fast Fourier transform—the most valuable numerical algo- rithm of our lifetime. —G. Strang, 1993 . wavelets are without any doubt an exciting and intuitive concept. This concept brings with it a new way of thinking... —Y. Meyer, 1993 Foreword Fourier, the nineteenth (and not the last!) child in his family, wanted to join an artillery regiment. His commoner status prevented it and he went on to other things. Goethe’s dictum that boldness has a magic all its own found life in Fourier. He was so rash at times that his work was rejected by his peers (see the introduction to Chapter 4). He never worried about convergence and said that any periodic function could be expressed in a Fourier series. Nevertheless he was so original that others—Cauchy and Lagrange, among them—were inspired to attempt to place his creations on a firm foundation. They both failed. The first proof that Fourier series converged pointwise was Dirichlet’s in 1829 for piecewise smooth functions (Sec. 4.6). As a result of Dirichlet’s work, the idea of function was trans- mogrified. No more would it apply only to the aristocratic society of poly- nomials, exponentials and sines and cosines; disorderly conduct now had to be tolerated. By the mid-nineteenth century, it inspired (as a trigonomet- ric series) Weierstrass’s continuous-but-not-differentiable map (Sec. 4.3). It was such a shock at the time that Weierstrass was apparently in no hurry to disseminate it widely. Inorder to generalize Dirichlet’s pointwise convergence theorem for piece- wise smooth functions to a wilder sort, Jordan invented the concept of ‘function of bounded variation;’ he proved his pointwise convergence the- orem of Fourier series for such functions in 1881 (Sec. 4.6). As it became necessary to deal with this wider class of functions, the conception of in- tegral was also transmuted. At Dirichlet’s urging, it went from integral-as- antiderivative to being defined as area under a curve. Cauchy developed the integral from this perspective for continuous functions. Riemann extended it to discontinuous functions, although not too discontinuous. Fejér (1904) went beyond functions of bounded variation. He discovered that for many functions f, f can be recovered by summing the arithmetic means of its Fourier series; even if the Fourier series diverges at a point, the series of arithmetic means converges to (f (t~) + f (t+)) /2 (Sec. 4.15). What happens at t’s where the one-sided limits do not exist? By removing the requirement concerning the existence of f(t-) and f (t+), Lebesguevi Foreword globalized Fejér’s theorem; he showed that the Fourier series for any f € L,[—7, 7] converges (C, 1) to f (t) a.e. The desire to do this was part of the reason that Lebesgue invented his integral; the theorem mentioned above was one of the first uses he made of it (Sec. 4.18). Denjoy, with the same motivation, extended the integral even further. Concurrently, the emerging point of view that things could be decom- posed into waves and then reconstituted infused not just mathematics but all of science. It is impossible to quantify the role that this perspective played in the development of the physics of the nineteenth and twentieth centuries, but it was certainly great. Imagine physics without it. We develop the standard features of Fourier analysis—Fourier series, Fourier transform, Fourier sine and cosine transforms. We do NOT do it in the most elegant way. Instead, we develop it for the reader who has never seen them before. We cover more recent developments such as the discrete and fast Fourier transforms and wavelets in Chapters 6 and 7. Our treatment of these topics is strictly introductory, for the novice. (Wavelets for idiots?) To do them properly, especially the applications, would take at least a whole book. What do you need to read the book? Not a lot of facts per se, but a little sophistication. We have helped ourselves to what we needed about the Lebesgue integral and given references. It’s not much and if you don’t know them exactly—if you know the analogous results (when they exist) for the Riemann integral—you can still read the book. We use some things about Hilbert space, too, and we have included a short development of what is needed in the first three short chapters. You can use them as a short course in functional analysis or start in Chapter 4 on Fourier series, referring to them on an as-needed basis. The chapters are sufficiently independent so that you could start in Chapter 5 (Fourier transform) or 6 (discrete, fast) or 7 (wavelets) and refer back as needed. One caveat: To appreciate Chapter 7, you should read the theory of the Ly Fourier transform in Chapter 5. The Ly theory is really quite pretty, anyway. Notation: Our notation is all standard. On some rare occasions we use C for complement. If we say “by Exercise 3,” we mean Exercise 3 at the end of the current section; otherwise we say “Exercise 2.4-3,” meaning Exercise 3 at the end of Sec. 2.4. Hints are provided for lots (not all) of the exercises. Rather than a separate index of symbols, the symbols are blended into the index. We prepared the book using Scientific Word and Scientific Workplace. The experience has been... .interesting. We hope that the result is fun.Contents Foreword 1 Metric and Normed Spaces Ll Metric Spaces... ee 1.2 Normed Spaces... 0.0... bees 1.3. Inner Product Spaces... 2.2... 0020000 14 Orthogonality. 2... eee ee 1.5 Linear Isometry. 2.0. ee 16 Holder and Minkowski Inequalities; L, and ¢, Spaces. 2° Analysis 21 Balls... 2.20.00... . 22 Convergence and Continuity... ..........-~ 2.3 Bounded Sets... 0.0.0. 2.4 Closure and Closed Sets... 2.2.2.2 ee 25 OpenSets.. 2... ee 26 Completeness... 0... ..0.0-- bees 2.7 Uniform Continuity. ©... 22 ee 28 Compactness ........ Lees 2.9 Equivalent Noms... . 2... ...00 00000005 2.10 Direct Sums... 2. ee eee 3 Bases 3.1 Best Approximation... ......... . 3.2 Orthogonal Complements and the Projection Theorem 3.3 Orthonormal Sequences... ..........- Lee 3.4 Orthonormal Bases... 2.2.22... eee 3.5 The Haar Basis... 22... ..0..0..0.000. 3.6 Unconditional Convergence... .....-..-.0-- 3.7 Orthogonal Direct Sums... 2... ...0..0-000- 3.8 Continuous Linear Maps ...... 2... 3.9 Dual Spaces... 0... .. bene Lee 3.10 Adjoints. 22... 0. ee eee 103 107 14 119 123 126 131 1354 Contents Fourier Series 4.1 Warmup . . 4.2. Fourier Sine Series and Cosine Series 4.3 Smoothness... 2... 2.0.00 an 44 The Riemann-Lebesgue Lemma... ..... 4.5 The Dirichlet and Fourier Kernels 50 4.6 Pointwise Convergence of Fourier Series. . . . 4.7 Uniform Convergence... . 2... --. 48 The Gibbs Phenomenon... .... . Bo 4.9 A Divergent Fourier Series . 2... 2... 4.10 Termwise Integration... 2.2.0.2... 4.11 Trigonometric vs. Fourier Series... 0... 4.12 Termwise Differentiation 4.13 Dido’s Dilemma. . 4.14 Other Kinds of Summability..... 2... . 4.15 Fejér Theory . . 4.16 The Smoothing Effect of (C, 1) Summation . 4.17 Weierstrass’s Approximation Theorem. . . . . 4.18 Lebesgue’s Pointwise Convergence Theorem. . . . 4.19 Higher Dimensions. Lee 4.20 Convergence of Multiple Series... - The Fourier Transform 5.1 The Finite Fourier Transform. . . 5.2 ConvolutiononT....... 5.3. The Exponential Form of Lebesgue’ 's Theorem. oe 5.4 Motivation and Definition. .......... 5.5 Basics/Examples .... 0.0.0.0... 5.6 The Fourier Transform and Residues... . . 5.7 TheFourierMap...........-.-.% 5.8 Convolution on R sees 5.9 Inversion, Exponential Form... 2... 5.10 Inversion, Trigonometric Form . 5.11 (C, 1) Summability for Integrals... . . 5.12 The Fejér—Lebesgue Inversion Theorem . . . - 5.13 Convergence Assistance . : 5.14 Approximate Identity... 0.0.02... 5.15 Transforms of Derivatives and Integrals . 5.16 Fourier Sine and Cosine Transforms . 5.17 Parseval’s Identities... . .. . . 139 143 154 159 169 174 188 202 207 210 213 218 221 224 226 235 242 244 245, 249 257 263 264 267 273 275 278 284 289 291 294 295 303 306 317 330 334 340 351Contents 5.18 The I, Theory . . . Sees 5.19 The Plancherel Theorem... . a 5.20 Pointwise Inversion and Summability..... 00... 5.21 A Sampling Theorem . 5.22 The Mellin Transform . bee ee 5.23 Variations. . . ae . a a as The Discrete and Fast Fourier Transforms 6.1 The Discrete Fourier Transform. . . : Lae 6.2 The Inversion Theorem for the DFT... 0.0... 5. 6.3. Cyclic Convolution... ee 64 Fast Fourier Transform for N=2!. 0... 11. 6.5 The Fast Fourier Transform for N=RC... 0.0... Wavelets 7.1 Orthonormal Basis from One Function ...... - 7.2 Multiresolution Analysis... .........- a 7.3 Mother Wavelets Yield Wavelet Bases... ........- 7.4 From MRA to Mother Wavelet... 2... 2... -- 7.5 Construction of a Scaling Function with Compact Support... 2.2... . 7.6 Shannon Wavelets... 2.2.22. ee ee 7.7 Riesz Bases and MRAs eee eee eee 7.8 Franklin Wavelets... 2... ......0-- 7.9 Frames...... . we . 7.10 Splines... 0... ee 7.11 The Continuous Wavelet Transform Index ix 356 361 366 372 375 378 383 383, 390 396 399 406 4i1 413 414 419 422 435 448 449 459 464 476 480 4971 Metric and Normed Spaces It is natural to think about distance between physical objects—people, say, or buildings or stars. In what follows, we explore a notion of “closeness” for such things as functions and sequences. (How far is f(z) = 2° from g(z) = sinz? How far is the sequence (1/n) from (2/n?)?) The way we answer such a question is through the idea of a metric space. In principle, it enables us to talk about the distance between colors or ideas or songs. When we can measure “distance,” we can take limits or “perform analysis.” Special distance-measuring devices called norms are introduced for vector spaces. The analysis we care most about in this book involves norms. This type of analysis is known as functional analysis because the vector spaces of greatest interest are spaces of functions. 1.1 Metric Spaces We define a metric space here, then give some examples. The idea is to abstract the properties of the usual notion of distance. Definition 1.1.1 Metric Space ‘Two things are required to define a METRIC SPACE: a set X of elements called POINTS or ELEMENTS and a METRIC (or DISTANCE FUNCTION) 4, defined on pairs (2, y) of points that satisfy the following conditions. For all z and y in X, « (Positive) d(z,y) > 0, and d(x,y) = 0 if and only if z = y. Thus, not only is the distance from any point to itself 0, but z is the only point 0 units. away from z. « (Symmetric) d(x, y) = d(y, 2). (Triangle Inequality) For any z € X, d(e,y) < d(z,2)+d(z,y). 0 As a tule, the only property that is difficult to verify about a proposed metric is the triangle inequality. We use the following notation throughout the book. ¢ R denotes the real numbers. © © denotes the complex numbers.2 1. Metric and Normed Spaces * K = Ror C without specifying which. Example 1.1.2 THe EUCLIDEAN SPACES (K", dz) = fz (n), nEN With distance defined by dy(z, y) = |x — yl, (K, di) is a metric space. Let K? denote the Cartesian product K x K ={(z,y):2,y@K}. A metric dz is defined on K? by 1p do(z,y) = [ln — 21? +Iye—2el"] where 2 = (21,22) and y = (y2,y2)- Let K? = Kx K x K. A distance is defined between the points s = (#1, 22,23) and y = (yi, ¥2,¥s) of K* by y(z,y) = [lyn — 21!” + lye — al? + lye — eal }/?. The triangle inequality is verified by using the Minkowski inequality 1.6.3(a) with p = 2. In either case dy is called the EUCLIDEAN or USUAL metric on K? and K®. The spaces (K?,d2) and (K°,dz) are called EUCLIDEAN 2-SPACE (the EUCLIDEAN PLANE if K = R) and EUCLIDEAN 3-SPACE, re- spectively. If we say only R? or R°, we mean (R?, dz) and (R¥, dz), respec- tively. We extend this idea of distance to n-dimensional space K” by taking, for (2:), (us) EK", n 12 do(z,y) = (= ly - ai) ; The triangle inequality follows from the Minkowski inequality with p = 2. The metric dp is also called the EUCLIDEAN METRIC, the pair (K", dz) EUCLIDEAN n-SPACE. Many authors reserve Buclidean for (R",d2) and refer to (C”,dz) as UNITARY n-SPACE, We say real or compler Euclidean n-space, respectively; we also denote (K", d2) by ¢2(n). 0 In many important respects, the spaces we care most about in this book can be thought of as though they were Euclidean 2- or 3-space, a point that will become clearer as our story unfolds. For now, you might glance at Exercise 1.3-8. It is common to view Ras a “subset” of the plane R?, even though it is actually R. x {0}, not R, that is the subset of R®. As metric spaces, there is no difference between (R,d1) and Rx {0} C ¢2 (2): For two real numbers a and b compare the distance d; (a,b) = |a —6| between them with the dy-distance between (a, 0) and (b,0): ds ((a, 0), (6, 0)] = (a —)? + (0 - 0)? = Ja — 8].1. Metric and Normed Spaces 3 Likewise, as metric spaces, there is no difference between (R?,d2) and the complex numbers C—distance is measured in exactly the same way in each case. The point is that there can be concretely different things that are the same as metric spaces. Definition 1.1.3 IsomETRY Let (X,d) and (¥,d’) be metric spaces. A map f : (X,d) — (¥,d’) is an ISOMETRY if for all z,y € X, d' (F(z), F(y)) = d(z,y). If the isometry f is onto (surjective) in the sense that for all y in Y there is some x in X such that f(x) = y, then the spaces (X,d) and (Y,d’) are called ISOMETRIC. 0 Notice that isometries f must be 1-1 (injective), since rxy oe d(z,y) £0 > d' (F(z), f(y)) #0 = F(z) F Sy)- Some obvious isometries of R? onto R? are translation, rotation through a fixed angle, and reflection about a line. Indeed, these are the only ones, something we say a little more about in Exercise 15-2. We commonly identify—i.e., treat as equal—isometric spaces, as in “R is a subspace of R” rather than “R is isometric to a subspace of R?.” Let Py denote the set of polynomials p(t) of degree less than or equal to n with real coefficients and distance defined by d (Soe. dm) = The space Py is isometric to R"+! = & (n + 1) under the mapping f: Ret — Pp (a9,41,---,4n) —> Dito a;ti * At times it is preferable to view P, as a set of polynomials, at other times as a set of (n + 1)-tuples. As metric spaces, they are the same. Example 1.1.4 Propuct METRICS The square-root-of-the-sum-of-the-squares method used to create a met- ric for the Cartesian products R x R and R x R x R in Example 1.1.2 can be generalized to Cartesian products of arbitrary metric spaces as follows. Suppose that (X,d) and (Y,d’) are metric spaces. A metric dz on X x Y = {(2,y):2€X, y€Y} is defined by da((e1sm) (22,00) = (d(e20)? +€'(099)?) (1)